A comprehensive survey of neural architecture search: Challenges and solutions

Ren, P; Xiao, Y; Chang, X; Huang, PY; Li, Z; Chen, X; Wang, X

A comprehensive survey of neural architecture search: Challenges and solutions

Ren, P Xiao, Y Chang, X

Huang, PY Li, Z Chen, X Wang, X

Permalink

Publisher:: ASSOC COMPUTING MACHINERY
Publication Type:: Journal Article
Citation:: ACM Computing Surveys, 2021, 54, (4)
Issue Date:: 2021-07-01

Closed Access

	Filename	Description	Size
	3447582 (1).pdf	Published version	4.36 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Ren, P
dc.contributor.author	Xiao, Y
dc.contributor.author	Chang, X https://orcid.org/0000-0002-7778-8807
dc.contributor.author	Huang, PY
dc.contributor.author	Li, Z
dc.contributor.author	Chen, X
dc.contributor.author	Wang, X
dc.date.accessioned	2022-02-06T02:04:44Z
dc.date.available	2022-02-06T02:04:44Z
dc.date.issued	2021-07-01
dc.identifier.citation	ACM Computing Surveys, 2021, 54, (4)
dc.identifier.issn	0360-0300
dc.identifier.issn	1557-7341
dc.identifier.uri	http://hdl.handle.net/10453/154209
dc.description.abstract	Deep learning has made substantial breakthroughs in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers' prior knowledge and experience. And due to the limitations of humans' inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture. Neural Architecture Search (NAS) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. In addition, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.
dc.language	English
dc.publisher	ASSOC COMPUTING MACHINERY
dc.relation	http://purl.org/au-research/grants/arc/DE190100626
dc.relation.ispartof	ACM Computing Surveys
dc.relation.isbasedon	10.1145/3447582
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences
dc.subject.classification	Information Systems
dc.title	A comprehensive survey of neural architecture search: Challenges and solutions
dc.type	Journal Article
utslib.citation.volume	54
utslib.for	08 Information and Computing Sciences
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2022-02-06T02:04:41Z
pubs.issue	4
pubs.publication-status	Published
pubs.volume	54
utslib.citation.issue	4

Abstract:

Deep learning has made substantial breakthroughs in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers' prior knowledge and experience. And due to the limitations of humans' inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture. Neural Architecture Search (NAS) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. In addition, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/154209