A lightweight Max-Pooling method and architecture for Deep Spiking Convolutional Neural Networks

Nguyen, D-A; Tran, X-T; Dang, KN; Iacopi, F

A lightweight Max-Pooling method and architecture for Deep Spiking Convolutional Neural Networks

Nguyen, D-A Tran, X-T Dang, KN Iacopi, F

Permalink

Publisher:: IEEE
Publication Type:: Conference Proceeding
Citation:: 2020 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), 2020, 00, pp. 209-212
Issue Date:: 2020-12-29

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

The embargo period expires on 31 Dec 2022

Adobe PDF

Download Accepted Manuscript VersionAdobe PDF (8.26 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Nguyen, D-A
dc.contributor.author	Tran, X-T
dc.contributor.author	Dang, KN
dc.contributor.author	Iacopi, F https://orcid.org/0000-0002-3196-0990
dc.date	2020-12-08
dc.date.accessioned	2021-03-29T02:08:55Z
dc.date.available	2021-03-29T02:08:55Z
dc.date.issued	2020-12-29
dc.identifier.citation	2020 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), 2020, 00, pp. 209-212
dc.identifier.isbn	9781728193960
dc.identifier.uri	http://hdl.handle.net/10453/147605
dc.description.abstract	The training of Deep Spiking Neural Networks (DSNNs) is facing many challenges due to the non-differentiable nature of spikes. The conversion of a traditional Deep Neural Networks (DNNs) to its DSNNs counterpart is currently one of the prominent solutions, as it leverages many state-of-the-art pre-trained models and training techniques. However, the conversion of max-pooling layer is a non-trivia task. The state-of-the-art conversion methods either replace the max-pooling layer with other pooling mechanisms or use a max-pooling method based on the cumulative number of output spikes. This incurs both memory storage overhead and increases computational complexity, as one inference in DSNNs requires many timesteps, and the number of output spikes after each layer needs to be accumulated. In this paper1, we propose a novel max-pooling mechanism that is not based on the number of output spikes but is based on the membrane potential of the spiking neurons. Simulation results show that our approach still preserves classification accuracies on MNIST and CIFARIO dataset. Hardware implementation results show that our proposed hardware block is lightweight with an area cost of 15.3kGEs, at a maximum frequency of 300 MHz.
dc.language	en
dc.publisher	IEEE
dc.relation.ispartof	2020 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS)
dc.relation.ispartof	2020 IEEE Asia Pacific Conference on Circuits and Systems
dc.relation.isbasedon	10.1109/apccas50809.2020.9301703
dc.rights	© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.	en_US
dc.rights	info:eu-repo/semantics/embargoedAccess
dc.title	A lightweight Max-Pooling method and architecture for Deep Spiking Convolutional Neural Networks
dc.type	Conference Proceeding
utslib.citation.volume	00
utslib.location.activity	Ha Long, Vietnam
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	open_access	*
utslib.copyright.embargo	2022-12-31T00:00:00+1000Z
dc.date.updated	2021-03-29T02:08:49Z
pubs.finish-date	2020-12-10
pubs.publication-status	Published
pubs.start-date	2020-12-08
pubs.volume	00

Abstract:

The training of Deep Spiking Neural Networks (DSNNs) is facing many challenges due to the non-differentiable nature of spikes. The conversion of a traditional Deep Neural Networks (DNNs) to its DSNNs counterpart is currently one of the prominent solutions, as it leverages many state-of-the-art pre-trained models and training techniques. However, the conversion of max-pooling layer is a non-trivia task. The state-of-the-art conversion methods either replace the max-pooling layer with other pooling mechanisms or use a max-pooling method based on the cumulative number of output spikes. This incurs both memory storage overhead and increases computational complexity, as one inference in DSNNs requires many timesteps, and the number of output spikes after each layer needs to be accumulated. In this paper1, we propose a novel max-pooling mechanism that is not based on the number of output spikes but is based on the membrane potential of the spiking neurons. Simulation results show that our approach still preserves classification accuracies on MNIST and CIFARIO dataset. Hardware implementation results show that our proposed hardware block is lightweight with an area cost of 15.3kGEs, at a maximum frequency of 300 MHz.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/147605