TY - GEN
T1 - Model-free reinforcement learning with ensemble for a soft continuum robot arm
AU - Morimoto, Ryota
AU - Nishikawa, Satoshi
AU - Niiyama, Ryuma
AU - Kuniyoshi, Yasuo
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021/4/12
Y1 - 2021/4/12
N2 - Soft robots have more passive degrees of freedom (DoFs) than rigid-body robots, which makes controller design difficult. Model-free reinforcement learning (RL) is a promising tool to resolve control problems in soft robotics alongside detailed and elaborate modeling. However, the adaptation of RL to soft robots requires consideration of the unique nature of soft bodies. In this work, a continuum robot arm is used as an example of a soft robot, and we propose an Ensembled Light-weight model-Free reinforcement learning Network (ELFNet), which is an RL framework with a computationally light ensemble. We demonstrated that the proposed system could learn control policies for a continuum robot arm to reach target positions using its tip not only in simulations but also in the real world. We used a pneumatically controlled continuum robot arm that operates with nine flexible rubber artificial muscles. Each artificial muscle can be controlled independently by pressure control valves, demonstrating that the policy can be learned using a real robot alone. We found that our method is more suitable for compliant robots than other RL methods because the sample efficiency is better than that of the other methods, and there is a significant difference in the performance when the number of passive DoFs is large. This study is expected to lead to the development of model-free RL in future soft robot control.
AB - Soft robots have more passive degrees of freedom (DoFs) than rigid-body robots, which makes controller design difficult. Model-free reinforcement learning (RL) is a promising tool to resolve control problems in soft robotics alongside detailed and elaborate modeling. However, the adaptation of RL to soft robots requires consideration of the unique nature of soft bodies. In this work, a continuum robot arm is used as an example of a soft robot, and we propose an Ensembled Light-weight model-Free reinforcement learning Network (ELFNet), which is an RL framework with a computationally light ensemble. We demonstrated that the proposed system could learn control policies for a continuum robot arm to reach target positions using its tip not only in simulations but also in the real world. We used a pneumatically controlled continuum robot arm that operates with nine flexible rubber artificial muscles. Each artificial muscle can be controlled independently by pressure control valves, demonstrating that the policy can be learned using a real robot alone. We found that our method is more suitable for compliant robots than other RL methods because the sample efficiency is better than that of the other methods, and there is a significant difference in the performance when the number of passive DoFs is large. This study is expected to lead to the development of model-free RL in future soft robot control.
UR - http://www.scopus.com/inward/record.url?scp=85114205322&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85114205322&partnerID=8YFLogxK
U2 - 10.1109/RoboSoft51838.2021.9479340
DO - 10.1109/RoboSoft51838.2021.9479340
M3 - Conference contribution
AN - SCOPUS:85114205322
T3 - 2021 IEEE 4th International Conference on Soft Robotics, RoboSoft 2021
SP - 141
EP - 148
BT - 2021 IEEE 4th International Conference on Soft Robotics, RoboSoft 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 4th IEEE International Conference on Soft Robotics, RoboSoft 2021
Y2 - 12 April 2021 through 16 April 2021
ER -