TY - JOUR
T1 - Virtual IR Sensing for Planetary Rovers
T2 - Improved Terrain Classification and Thermal Inertia Estimation
AU - Iwashita, Yumi
AU - Nakashima, Kazuto
AU - Gatto, Joseph
AU - Higa, Shoya
AU - Stoica, Adrian
AU - Khoo, Norris
AU - Kurazume, Ryo
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2020/10
Y1 - 2020/10
N2 - Terrain classification is critically important for Mars rovers, which rely on it for planning and autonomous navigation. On-board terrain classification using visual information has limitations, and is sensitive to illumination conditions. Classification can be improved if one fuses visual imagery with additional infrared (IR) imagery of the scene, yet unfortunately there are no IR image sensors on the current Mars rovers. A virtual IR sensor, estimating IR from RGB imagery using deep learning, was proposed in the context of a MU-Net architecture. However, virtual IR estimation was limited by the fact that slope angle variations induce temperature differences within the same terrain. This paper removes this limitation, giving good IR estimates and as a consequence improving terrain classification by including the additional angle from the surface normal to the Sun and the measurement of solar radiation. The estimates are also useful when estimating thermal inertia, which can enhance slip prediction and small rock density estimation. Our approach is demonstrated in two applications. We collected a new data set to verify the effectiveness of the proposed approach and show its benefit by applying to the two applications.
AB - Terrain classification is critically important for Mars rovers, which rely on it for planning and autonomous navigation. On-board terrain classification using visual information has limitations, and is sensitive to illumination conditions. Classification can be improved if one fuses visual imagery with additional infrared (IR) imagery of the scene, yet unfortunately there are no IR image sensors on the current Mars rovers. A virtual IR sensor, estimating IR from RGB imagery using deep learning, was proposed in the context of a MU-Net architecture. However, virtual IR estimation was limited by the fact that slope angle variations induce temperature differences within the same terrain. This paper removes this limitation, giving good IR estimates and as a consequence improving terrain classification by including the additional angle from the surface normal to the Sun and the measurement of solar radiation. The estimates are also useful when estimating thermal inertia, which can enhance slip prediction and small rock density estimation. Our approach is demonstrated in two applications. We collected a new data set to verify the effectiveness of the proposed approach and show its benefit by applying to the two applications.
UR - http://www.scopus.com/inward/record.url?scp=85089946506&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85089946506&partnerID=8YFLogxK
U2 - 10.1109/LRA.2020.3013909
DO - 10.1109/LRA.2020.3013909
M3 - Article
AN - SCOPUS:85089946506
SN - 2377-3766
VL - 5
SP - 6302
EP - 6309
JO - IEEE Robotics and Automation Letters
JF - IEEE Robotics and Automation Letters
IS - 4
M1 - 9158384
ER -