MTLP-JR: Multi-task learning-based prediction for joint ranking in neural architecture search

Lyu, B; Lu, L; Hamdi, M; Wen, S; Yang, Y; Li, K

MTLP-JR: Multi-task learning-based prediction for joint ranking in neural architecture search

Lyu, B Lu, L Hamdi, M Wen, S

Yang, Y Li, K

Permalink

Publisher:: Elsevier
Publication Type:: Journal Article
Citation:: Computers and Electrical Engineering, 2023, 105, pp. 108474
Issue Date:: 2023-01-01

Embargoed

	Filename	Description	Size
	MTLP-JR Multi-task learning-based prediction for joint ranking in neural architecture search.pdf	Accepted version	3.68 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Embargoed
Open Access

This item is currently unavailable due to the publisher's embargo.

The embargo period expires on 1 Jan 2025

Full metadata record

Field	Value	Language
dc.contributor.author	Lyu, B
dc.contributor.author	Lu, L
dc.contributor.author	Hamdi, M
dc.contributor.author	Wen, S https://orcid.org/0000-0001-8077-7001
dc.contributor.author	Yang, Y
dc.contributor.author	Li, K
dc.date.accessioned	2023-03-13T12:10:40Z
dc.date.available	2023-03-13T12:10:40Z
dc.date.issued	2023-01-01
dc.identifier.citation	Computers and Electrical Engineering, 2023, 105, pp. 108474
dc.identifier.issn	0045-7906
dc.identifier.uri	http://hdl.handle.net/10453/167160
dc.description.abstract	At present, great attentions have been paid to multi-objective neural architecture search (NAS) and resource-aware NAS for their comprehensive consideration of the overall evaluation of architectures, including inference latency, precision, and model scale. However NAS also exacerbates the ever-increasing cost (engineering, time complexity, computation resource). Aiming to alleviate this, the reproducible NAS research releases the benchmark, which includes the metrics (e.g. Accuracy, Latency, and Parameters) of representative models from the typical search space on specific tasks. Motivated by the multi-objective NAS, resource-aware NAS, and reproducible NAS, this paper dedicates to binary-relation prediction (Latency, Accuracy), which is a more reasonable and effective way to satisfy the general NAS scenarios with less cost. We conduct a reproducible NAS study on the MobileNet-based search space and release the dataset. Further, we first propose the modeling of common features among prediction tasks (Latency, Accuracy, Parameters, and FLOPs), which will facilitate the prediction of individual tasks, and creatively formulate the architecture ranking prediction with a multi-task learning framework. Eventually, the proposed multi-task learning based binary-relation prediction model reaches the performance of 94.3% on Latency and 85.02% on Top1 Accuracy even with only 100 training points, which outperforms the single-task learning based model.
dc.language	en
dc.publisher	Elsevier
dc.relation.ispartof	Computers and Electrical Engineering
dc.relation.isbasedon	10.1016/j.compeleceng.2022.108474
dc.rights	info:eu-repo/semantics/embargoedAccess
dc.subject	0803 Computer Software, 0805 Distributed Computing, 0906 Electrical and Electronic Engineering
dc.subject.classification	Electrical & Electronic Engineering
dc.title	MTLP-JR: Multi-task learning-based prediction for joint ranking in neural architecture search
dc.type	Journal Article
utslib.citation.volume	105
utslib.for	0803 Computer Software
utslib.for	0805 Distributed Computing
utslib.for	0906 Electrical and Electronic Engineering
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	embargoed	*
utslib.copyright.embargo	2025-01-01T00:00:00+1000Z
dc.date.updated	2023-03-13T12:10:37Z
pubs.publication-status	Published
pubs.volume	105

Abstract:

At present, great attentions have been paid to multi-objective neural architecture search (NAS) and resource-aware NAS for their comprehensive consideration of the overall evaluation of architectures, including inference latency, precision, and model scale. However NAS also exacerbates the ever-increasing cost (engineering, time complexity, computation resource). Aiming to alleviate this, the reproducible NAS research releases the benchmark, which includes the metrics (e.g. Accuracy, Latency, and Parameters) of representative models from the typical search space on specific tasks. Motivated by the multi-objective NAS, resource-aware NAS, and reproducible NAS, this paper dedicates to binary-relation prediction (Latency, Accuracy), which is a more reasonable and effective way to satisfy the general NAS scenarios with less cost. We conduct a reproducible NAS study on the MobileNet-based search space and release the dataset. Further, we first propose the modeling of common features among prediction tasks (Latency, Accuracy, Parameters, and FLOPs), which will facilitate the prediction of individual tasks, and creatively formulate the architecture ranking prediction with a multi-task learning framework. Eventually, the proposed multi-task learning based binary-relation prediction model reaches the performance of 94.3% on Latency and 85.02% on Top1 Accuracy even with only 100 training points, which outperforms the single-task learning based model.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/167160