Model-free reinforcement learning with ensemble for a soft continuum robot arm

Ryota Morimoto, Satoshi Nishikawa, Ryuma Niiyama, Yasuo Kuniyoshi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

32 Citations (Scopus)

Abstract

Soft robots have more passive degrees of freedom (DoFs) than rigid-body robots, which makes controller design difficult. Model-free reinforcement learning (RL) is a promising tool to resolve control problems in soft robotics alongside detailed and elaborate modeling. However, the adaptation of RL to soft robots requires consideration of the unique nature of soft bodies. In this work, a continuum robot arm is used as an example of a soft robot, and we propose an Ensembled Light-weight model-Free reinforcement learning Network (ELFNet), which is an RL framework with a computationally light ensemble. We demonstrated that the proposed system could learn control policies for a continuum robot arm to reach target positions using its tip not only in simulations but also in the real world. We used a pneumatically controlled continuum robot arm that operates with nine flexible rubber artificial muscles. Each artificial muscle can be controlled independently by pressure control valves, demonstrating that the policy can be learned using a real robot alone. We found that our method is more suitable for compliant robots than other RL methods because the sample efficiency is better than that of the other methods, and there is a significant difference in the performance when the number of passive DoFs is large. This study is expected to lead to the development of model-free RL in future soft robot control.

Original languageEnglish
Title of host publication2021 IEEE 4th International Conference on Soft Robotics, RoboSoft 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages141-148
Number of pages8
ISBN (Electronic)9781728177137
DOIs
Publication statusPublished - Apr 12 2021
Externally publishedYes
Event4th IEEE International Conference on Soft Robotics, RoboSoft 2021 - New Haven, United States
Duration: Apr 12 2021Apr 16 2021

Publication series

Name2021 IEEE 4th International Conference on Soft Robotics, RoboSoft 2021

Conference

Conference4th IEEE International Conference on Soft Robotics, RoboSoft 2021
Country/TerritoryUnited States
CityNew Haven
Period4/12/214/16/21

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Mechanical Engineering
  • Control and Optimization
  • Modelling and Simulation

Fingerprint

Dive into the research topics of 'Model-free reinforcement learning with ensemble for a soft continuum robot arm'. Together they form a unique fingerprint.

Cite this