Dexterous object manipulation by a multi-fingered robotic hand with visual-tactile fingertip sensors

Seung hyun Choi, Kenji Tahara

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)


In this paper, a novel visual-tactile sensor is proposed; additionally, an object manipulation method for a multi-fingered robotic hand grasping an object is proposed by detecting a contact position using the visual-tactile sensor. The visual-tactile sensor is composed of a hemispheric fingertip made of soft silicone with a hollow interior and a general USB camera located inside the fingertip to detect the displacement of the many point markers embedded in the silicone. The deformation of each point marker due to a contact force is measured, and a contact position is estimated reliably through a novel method of creating virtual points to determine the point clouds. The aim is to demonstrate both the estimation performance of the new visual-tactile sensor and its usefulness in a grasping and manipulation task. By using the contact position obtained from the proposed sensor and the position of each fingertip obtained from kinematics, the position and orientation of a grasped object are estimated and controlled. The effectiveness of the method is illustrated through numerical simulation and its practical use is demonstrated through grasping and manipulating experiments.

Original languageEnglish
Article number14
JournalROBOMECH Journal
Issue number1
Publication statusPublished - Dec 1 2020

All Science Journal Classification (ASJC) codes

  • Modelling and Simulation
  • Instrumentation
  • Mechanical Engineering
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'Dexterous object manipulation by a multi-fingered robotic hand with visual-tactile fingertip sensors'. Together they form a unique fingerprint.

Cite this