Deep Multidilation Temporal and Spatial Dependence Modeling in Stereoscopic 3-D EEG for Visual Discomfort Assessment

Zheng, Y; Zhao, X; Yao, L; Cao, L

Deep Multidilation Temporal and Spatial Dependence Modeling in Stereoscopic 3-D EEG for Visual Discomfort Assessment

Zheng, Y Zhao, X Yao, L Cao, L

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2024, 54, (4), pp. 2125-2136
Issue Date:: 2024-04-01

Closed Access

	Filename	Description	Size
	1691998.pdf	Published version	3.16 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zheng, Y
dc.contributor.author	Zhao, X
dc.contributor.author	Yao, L
dc.contributor.author	Cao, L https://orcid.org/0000-0003-1562-9429
dc.date.accessioned	2024-08-07T04:34:39Z
dc.date.available	2024-08-07T04:34:39Z
dc.date.issued	2024-04-01
dc.identifier.citation	IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2024, 54, (4), pp. 2125-2136
dc.identifier.issn	2168-2216
dc.identifier.issn	2168-2232
dc.identifier.uri	http://hdl.handle.net/10453/180205
dc.description.abstract	Visual discomfort assessment in stereoscopic three-dimensional (3-D) electroencephalography (EEG) data is challenging. The SOTA research applies deep neural networks to learn temporal information in continuous EEG signals and global spatial information about electrode locations. This work makes the first attempt to jointly deeply model spatio-Temporal coupling relationships and select strong dependencies in 3-D EEG data. We explore whether modeling such temporal and spatial dependencies would improve visual discomfort assessment. To address these issues, this work introduces multidilation temporal and spatial dependence-based convolutional neural networks (MTSD) to explore spatio-Temporal couplings by 1) learning hierarchical temporal relations within both continuous and interval data and 2) learning and selecting strong spatial dependencies between electrodes (their locations). MTSD captures 1) multidilation temporal relations in continuous and interval receptive fields by a parallel convolution module and 2) spatial dependencies between electrodes in stereoscopic 3-D EEG data by a constrained self-Attention module with a copula-based variable dependence strength filter for visual discomfort assessment. Experiments compare five EEG-based deep networks and three MTSD variants. MTSD makes improvement in discriminating visual discomfort by capturing strong spatio-Temporal couplings but uses significantly less computational resources.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation	http://purl.org/au-research/grants/arc/DP190101079
dc.relation.ispartof	IEEE Transactions on Systems, Man, and Cybernetics: Systems
dc.relation.isbasedon	10.1109/TSMC.2023.3340710
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Deep Multidilation Temporal and Spatial Dependence Modeling in Stereoscopic 3-D EEG for Visual Discomfort Assessment
dc.type	Journal Article
utslib.citation.volume	54
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Strength - AAI - Advanced Analytics Institute Research Centre
pubs.organisational-group	University of Technology Sydney/All Manual Groups
pubs.organisational-group	University of Technology Sydney/All Manual Groups/Data Science Institute (DSI)
pubs.organisational-group	University of Technology Sydney/Strength - DSI - Data Science Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2024-08-07T04:34:35Z
pubs.issue	4
pubs.publication-status	Published
pubs.volume	54
utslib.citation.issue	4

Abstract:

Visual discomfort assessment in stereoscopic three-dimensional (3-D) electroencephalography (EEG) data is challenging. The SOTA research applies deep neural networks to learn temporal information in continuous EEG signals and global spatial information about electrode locations. This work makes the first attempt to jointly deeply model spatio-Temporal coupling relationships and select strong dependencies in 3-D EEG data. We explore whether modeling such temporal and spatial dependencies would improve visual discomfort assessment. To address these issues, this work introduces multidilation temporal and spatial dependence-based convolutional neural networks (MTSD) to explore spatio-Temporal couplings by 1) learning hierarchical temporal relations within both continuous and interval data and 2) learning and selecting strong spatial dependencies between electrodes (their locations). MTSD captures 1) multidilation temporal relations in continuous and interval receptive fields by a parallel convolution module and 2) spatial dependencies between electrodes in stereoscopic 3-D EEG data by a constrained self-Attention module with a copula-based variable dependence strength filter for visual discomfort assessment. Experiments compare five EEG-based deep networks and three MTSD variants. MTSD makes improvement in discriminating visual discomfort by capturing strong spatio-Temporal couplings but uses significantly less computational resources.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/180205