A 3D Vector Field and Gaze Data Fusion Framework for Hand Motion Intention Prediction in Human-Robot Collaboration

Jayasuriya, M; Hu, G; Le, DDK; Ang, K; Sankaran, S; Liu, D

A 3D Vector Field and Gaze Data Fusion Framework for Hand Motion Intention Prediction in Human-Robot Collaboration

Jayasuriya, M Hu, G Le, DDK Ang, K Sankaran, S Liu, D

Permalink

Publisher:: IEEE
Publication Type:: Conference Proceeding
Citation:: 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, 00, pp. 5637-5643
Issue Date:: 2024-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Accepted versionAdobe PDF (896.09 kB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Jayasuriya, M
dc.contributor.author	Hu, G
dc.contributor.author	Le, DDK
dc.contributor.author	Ang, K
dc.contributor.author	Sankaran, S
dc.contributor.author	Liu, D https://orcid.org/0000-0002-1581-5582
dc.date	2024-05-13
dc.date.accessioned	2024-10-09T03:37:14Z
dc.date.available	2024-10-09T03:37:14Z
dc.date.issued	2024-01-01
dc.identifier.citation	2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, 00, pp. 5637-5643
dc.identifier.isbn	979-8-3503-8458-1
dc.identifier.issn	1050-4729
dc.identifier.uri	http://hdl.handle.net/10453/181315
dc.description.abstract	In human robot collaboration HRC settings hand motion intention prediction HMIP plays a pivotal role in ensuring prompt decision making safety and an intuitive collaboration experience Precise and robust HMIP with low computational resources remains a challenge due to the stochastic nature of hand motion and the diversity of HRC tasks This paper proposes a framework that combines hand trajectories and gaze data to foster robust real time HMIP with minimal to no training A novel 3D vector field method is introduced for hand trajectory representation leveraging minimum jerk trajectory predictions to discern potential hand motion endpoints This is statistically combined with gaze fixation data using a weighted Naive Bayes Classifier NBC Acknowledging the potential variances in saccadic eye motion due to factors like fatigue or inattentiveness we incorporate stationary gaze entropy to gauge visual concentration thereby adjusting the contribution of gaze fixation to the HMIP Empirical experiments substantiate that the proposed framework robustly predicts intended endpoints of hand motion before at least 50 of the trajectory is completed It also successfully exploits gaze fixations when the human operator is attentive and mitigates its influence when the operator loses focus A real time implementation in a construction HRC scenario collaborative tiling showcases the intuitive nature and potential efficiency gains to be leveraged by introducing the proposed HMIP into HRC contexts The opens ource implementation of the framework is made available at https github com maleenj hmip ros git
dc.language	en
dc.publisher	IEEE
dc.relation.ispartof	2024 IEEE International Conference on Robotics and Automation (ICRA)
dc.relation.ispartof	2024 IEEE International Conference on Robotics and Automation
dc.relation.isbasedon	10.1109/icra57147.2024.10609996
dc.rights	info:eu-repo/semantics/openAccess
dc.title	A 3D Vector Field and Gaze Data Fusion Framework for Hand Motion Intention Prediction in Human-Robot Collaboration
dc.type	Conference Proceeding
utslib.citation.volume	00
utslib.location.activity	Yokohama, Japan
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Mechanical and Mechatronic Engineering
pubs.organisational-group	University of Technology Sydney/UTS Groups
pubs.organisational-group	University of Technology Sydney/UTS Groups/Robotics Institute (RI)
utslib.copyright.status	open_access	*
dc.date.updated	2024-10-09T03:37:10Z
pubs.finish-date	2024-05-17
pubs.place-of-publication	Piscataway, USA
pubs.publication-status	Published
pubs.start-date	2024-05-13
pubs.volume	00
dc.location	Piscataway, USA

Abstract:

In human robot collaboration HRC settings hand motion intention prediction HMIP plays a pivotal role in ensuring prompt decision making safety and an intuitive collaboration experience Precise and robust HMIP with low computational resources remains a challenge due to the stochastic nature of hand motion and the diversity of HRC tasks This paper proposes a framework that combines hand trajectories and gaze data to foster robust real time HMIP with minimal to no training A novel 3D vector field method is introduced for hand trajectory representation leveraging minimum jerk trajectory predictions to discern potential hand motion endpoints This is statistically combined with gaze fixation data using a weighted Naive Bayes Classifier NBC Acknowledging the potential variances in saccadic eye motion due to factors like fatigue or inattentiveness we incorporate stationary gaze entropy to gauge visual concentration thereby adjusting the contribution of gaze fixation to the HMIP Empirical experiments substantiate that the proposed framework robustly predicts intended endpoints of hand motion before at least 50 of the trajectory is completed It also successfully exploits gaze fixations when the human operator is attentive and mitigates its influence when the operator loses focus A real time implementation in a construction HRC scenario collaborative tiling showcases the intuitive nature and potential efficiency gains to be leveraged by introducing the proposed HMIP into HRC contexts The opens ource implementation of the framework is made available at https github com maleenj hmip ros git

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/181315