Guy Mardiks-Many Objective Neural Architecture Search for Deep Classifiers

Electrical Engineering Systems Seminar

Speaker: Guy Mardiks

M.Sc. student under the supervision of Dr. Amiram Moshaiov

Wednesday, 6^th March 2024, at 15:00

Room 011, Kitot Building, Faculty of Engineering

Many Objective Neural Architecture Search for Deep Classifiers

Abstract

Deep Neural Networks (DNNs) have been shown to achieve excellent results in solving real-life problems such as classification, segmentation, and natural language processing. The complex architecture design of a DNN plays a crucial role in the network performance. Commonly, such architectures are manually designed by experts based on a trial-and-error process. In order to reduce human efforts and improve performance, neural architecture search (NAS) methods have been developed to automate the design of DNN architectures without or with little domain expertise. Among the different methods to implement NAS, evolutionary computation (EC) methods have been gaining increasing attention and success.

This study focusses on the development and investigation of an evolutionary NAS technique that deals with the design of DNNs for Multi-Class Classification (MCC). The traditional approach to define MCC problems is based on a scalar loss function. In contrast, this study addresses the MCC problem as a many-objective optimization problem. In particular, the problem is defined as a vector optimization problem in which the classification performance in each class is considered as a separate objective. This problem definition aims at a set of Pareto-optimal classifiers. To solve the considered problem, this study proposes and investigates a unique evolutionary NAS technique that is combined with a transfer learning approach. The knowledge transfer was done using a deep feature extractor, based on the removal of the final layers from an existing (source) deep network. This feature extractor was kept as-is for each of the evolving networks. The evolutionary NAS was done by pruning of a classification network that was amended to the feature extractor. This allows a major reduction of the required computational resources. Yet, it raised a research question regarding the accuracy cost especially when facing a classification problem that the source classifier was not trained for. This study provides an answer to this question. It shows that the proposed many-objective search approach finds optimal architectures in terms of network complexity, without a performance compromise relative to the unpruned network. Furthermore, this study shows that the resulting set of deep classifiers can be used to devise ensemble models to achieve improved performance.

השתתפות בסמינר תיתן קרדיט שמיעה = עפ"י רישום שם מלא + מספר ת.ז. בדף הנוכחות שיועבר באולם במהלך הסמינר