Virtual IR Sensing for Planetary Rovers: Improved Terrain Classification and Thermal Inertia Estimation

Yumi Iwashita, Kazuto Nakashima, Joseph Gatto, Shoya Higa, Adrian Stoica, Norris Khoo, Ryo Kurazume

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


Terrain classification is critically important for Mars rovers, which rely on it for planning and autonomous navigation. On-board terrain classification using visual information has limitations, and is sensitive to illumination conditions. Classification can be improved if one fuses visual imagery with additional infrared (IR) imagery of the scene, yet unfortunately there are no IR image sensors on the current Mars rovers. A virtual IR sensor, estimating IR from RGB imagery using deep learning, was proposed in the context of a MU-Net architecture. However, virtual IR estimation was limited by the fact that slope angle variations induce temperature differences within the same terrain. This paper removes this limitation, giving good IR estimates and as a consequence improving terrain classification by including the additional angle from the surface normal to the Sun and the measurement of solar radiation. The estimates are also useful when estimating thermal inertia, which can enhance slip prediction and small rock density estimation. Our approach is demonstrated in two applications. We collected a new data set to verify the effectiveness of the proposed approach and show its benefit by applying to the two applications.

Original languageEnglish
Article number9158384
Pages (from-to)6302-6309
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number4
Publication statusPublished - Oct 2020

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'Virtual IR Sensing for Planetary Rovers: Improved Terrain Classification and Thermal Inertia Estimation'. Together they form a unique fingerprint.

Cite this