A Template-Based 3D Reconstruction of Colon Structures and Textures from Stereo Colonoscopic Images

Zhang, S; Zhao, L; Huang, S; Ye, M; Hao, Q

A Template-Based 3D Reconstruction of Colon Structures and Textures from Stereo Colonoscopic Images

Zhang, S Zhao, L

Huang, S

Ye, M Hao, Q

Permalink

Publisher:: Institute of Electrical and Electronics Engineers (IEEE)
Publication Type:: Journal Article
Citation:: IEEE Transactions on Medical Robotics and Bionics, 2021, 3, (1), pp. 85-95
Issue Date:: 2021-02-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Accepted versionAdobe PDF (10.92 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Zhang, S
dc.contributor.author	Zhao, L https://orcid.org/0000-0003-4063-8183
dc.contributor.author	Huang, S https://orcid.org/0000-0002-6124-4178
dc.contributor.author	Ye, M
dc.contributor.author	Hao, Q
dc.date.accessioned	2022-03-08T05:23:09Z
dc.date.available	2022-03-08T05:23:09Z
dc.date.issued	2021-02-01
dc.identifier.citation	IEEE Transactions on Medical Robotics and Bionics, 2021, 3, (1), pp. 85-95
dc.identifier.issn	2576-3202
dc.identifier.issn	2576-3202
dc.identifier.uri	http://hdl.handle.net/10453/155057
dc.description.abstract	This article presents a framework for 3D reconstruction of colonic surface using stereo colonoscopic images. Due to the limited overlaps between consecutive frames and the nonexistence of large loop closures during a normal screening colonoscopy, the state-of-art simultaneous localization and mapping (SLAM) algorithms cannot be directly applied to this scenario, thus a colon model segmented from CT scans is used together with the colonosocopic images to achieve the colon 3D reconstruction with high accuracy. The proposed framework includes 3D scan (point cloud with RGB information) reconstruction from stereo images, a visual odometry (VO) based camera pose initialization module, a 3D registration scheme for matching texture scans to the segmented colon model, and a barycentric-based texture rendering module for mapping textures from colonoscopic images to the reconstructed colonic surface. A realistic simulator is developed using Unity to simulate the procedures of colonoscopy and used to provide experimental datasets in different scenarios. Experimental results demonstrate the good performance of the proposed 3D colonic surface reconstruction method in terms of accuracy and robustness. Currently, the framework requires a pre-operative colon model as the template for colon reconstruction and can reconstruct 3D colon maps when the colon has no large deformation and the colon structure is clearly visible. The datasets used in this article and the developed simulator are made publicly available for other researchers to use (https://github.com/zsustc/colon_reconstruction_dataset).
dc.language	en
dc.publisher	Institute of Electrical and Electronics Engineers (IEEE)
dc.relation	http://purl.org/au-research/grants/arc/DP200100982
dc.relation.ispartof	IEEE Transactions on Medical Robotics and Bionics
dc.relation.isbasedon	10.1109/TMRB.2020.3044108
dc.rights	info:eu-repo/semantics/openAccess
dc.title	A Template-Based 3D Reconstruction of Colon Structures and Textures from Stereo Colonoscopic Images
dc.type	Journal Article
utslib.citation.volume	3
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - CAS - Centre for Autonomous Systems
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Mechanical and Mechatronic Engineering
utslib.copyright.status	open_access	*
dc.date.updated	2022-03-08T05:23:04Z
pubs.issue	1
pubs.publication-status	Published
pubs.volume	3
utslib.citation.issue	1

Abstract:

This article presents a framework for 3D reconstruction of colonic surface using stereo colonoscopic images. Due to the limited overlaps between consecutive frames and the nonexistence of large loop closures during a normal screening colonoscopy, the state-of-art simultaneous localization and mapping (SLAM) algorithms cannot be directly applied to this scenario, thus a colon model segmented from CT scans is used together with the colonosocopic images to achieve the colon 3D reconstruction with high accuracy. The proposed framework includes 3D scan (point cloud with RGB information) reconstruction from stereo images, a visual odometry (VO) based camera pose initialization module, a 3D registration scheme for matching texture scans to the segmented colon model, and a barycentric-based texture rendering module for mapping textures from colonoscopic images to the reconstructed colonic surface. A realistic simulator is developed using Unity to simulate the procedures of colonoscopy and used to provide experimental datasets in different scenarios. Experimental results demonstrate the good performance of the proposed 3D colonic surface reconstruction method in terms of accuracy and robustness. Currently, the framework requires a pre-operative colon model as the template for colon reconstruction and can reconstruct 3D colon maps when the colon has no large deformation and the colon structure is clearly visible. The datasets used in this article and the developed simulator are made publicly available for other researchers to use (https://github.com/zsustc/colon_reconstruction_dataset).

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/155057