Robust visual servoing for object manipulation against temporary loss of sensory information using a multi-fingered hand-arm

This paper proposes a robust visual servoing method for object manipulation against temporary loss of sensory information. It is known that visual information is useful for reliable object grasping and precise manipulation. Visual information becomes unavailable, however when occlusion occurs or a grasped object disappears during manipulation. In that case, the behavior of the visual servoing system becomes unstable. Our proposed method enables an object to be grasped and manipulated stably even if visual information is temporarily unavailable during manipulation. This method is based on dynamic stable object grasping and manipulation proposed in our previous work and the concept of virtual object information. A dynamic model of the overall system is first formulated. A new controller using both actual and virtual object information is proposed next. The usefulness of this method is finally verified through both numerical simulation and experiments using a triple-fingered mechanical hand.