TY - GEN
T1 - Wide-area shape reconstruction by 3D endoscopic system based on CNN decoding, shape registration and fusion
AU - Furukawa, Ryo
AU - Mizomori, Masaki
AU - Hiura, Shinsaku
AU - Oka, Shiro
AU - Tanaka, Shinji
AU - Kawasaki, Hiroshi
N1 - Funding Information:
This work was supported byJSPS/KAKENHI 16H02849, 16KK0151, 18H04119, 18K19824, and MSRA CORE14.
Funding Information:
Acknowledgment. This work was supported by JSPS/KAKENHI 16H02849,
Publisher Copyright:
© Springer Nature Switzerland AG 2018.
PY - 2018
Y1 - 2018
N2 - For effective in situ endoscopic diagnosis and treatment, dense and large areal shape reconstruction is important. For this purpose, we develop 3D endoscopic systems based on active stereo, which projects a grid pattern where grid points are coded by line gaps. One problem of the previous works was that success or failure of 3D reconstruction depends on the stability of feature extraction from the images captured by the endoscope camera. Subsurface scattering or specularities on bio-tissues make this problem difficult. Another problem was that shape reconstruction area was relatively small because of limited field of view of the pattern projector compared to that of the camera. In this paper, to solve the first problem, learning-based approach, i.e., U-Nets, for efficient detection of grid lines and codes at the detected grid points under severe conditions, is proposed. To solve the second problem, an online shape-registration and merging algorithm for sequential frames is proposed. In the experiments, we have shown that we can train U-Nets to extract those features effectively for three specimens of cancers, and also conducted 3D scanning of shapes of a stomach phantom model and a surface inside a human mouth, in which wide-area surfaces are successfully recovered by shape registration and merging.
AB - For effective in situ endoscopic diagnosis and treatment, dense and large areal shape reconstruction is important. For this purpose, we develop 3D endoscopic systems based on active stereo, which projects a grid pattern where grid points are coded by line gaps. One problem of the previous works was that success or failure of 3D reconstruction depends on the stability of feature extraction from the images captured by the endoscope camera. Subsurface scattering or specularities on bio-tissues make this problem difficult. Another problem was that shape reconstruction area was relatively small because of limited field of view of the pattern projector compared to that of the camera. In this paper, to solve the first problem, learning-based approach, i.e., U-Nets, for efficient detection of grid lines and codes at the detected grid points under severe conditions, is proposed. To solve the second problem, an online shape-registration and merging algorithm for sequential frames is proposed. In the experiments, we have shown that we can train U-Nets to extract those features effectively for three specimens of cancers, and also conducted 3D scanning of shapes of a stomach phantom model and a surface inside a human mouth, in which wide-area surfaces are successfully recovered by shape registration and merging.
UR - http://www.scopus.com/inward/record.url?scp=85054882518&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85054882518&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-01201-4_16
DO - 10.1007/978-3-030-01201-4_16
M3 - Conference contribution
AN - SCOPUS:85054882518
SN - 9783030012007
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 139
EP - 150
BT - OR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis - 1st International Workshop, OR 2.0 2018 5th International Workshop, CARE 2018, 7th International Workshop, CLIP 2018, 3rd International Workshop, ISIC 2018 Held in Conjunction with MICCAI 2018
A2 - Malpani, Anand
A2 - Zenati, Marco A.
A2 - Oyarzun Laura, Cristina
A2 - Celebi, M. Emre
A2 - Sarikaya, Duygu
A2 - Codella, Noel C.
A2 - Halpern, Allan
A2 - Erdt, Marius
A2 - Maier-Hein, Lena
A2 - Xiongbiao, Luo
A2 - Wesarg, Stefan
A2 - Stoyanov, Danail
A2 - Taylor, Zeike
A2 - Drechsler, Klaus
A2 - Dana, Kristin
A2 - Martel, Anne
A2 - Shekhar, Raj
A2 - De Ribaupierre, Sandrine
A2 - Reichl, Tobias
A2 - McLeod, Jonathan
A2 - González Ballester, Miguel Angel
A2 - Collins, Toby
A2 - Linguraru, Marius George
PB - Springer Verlag
T2 - 1st International Workshop on OR 2.0 Context-Aware Operating Theaters, OR 2.0 2018, 5th International Workshop on Computer Assisted Robotic Endoscopy, CARE 2018, 7th International Workshop on Clinical Image-Based Procedures, CLIP 2018, and 1st International Workshop on Skin Image Analysis, ISIC 2018, held in conjunction with the 21st International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2018
Y2 - 16 September 2018 through 20 September 2018
ER -
