A grasping and manipulation scheme that is robust against time delays of sensing information: An application of a controller based on finger-thumb opposability

Visual sensing information is useful for acquiring an object's position and orientation when a robot grasps and manipulates an object. However, it is still difficult to obtain such information accurately in real time. This is because visual sensing information generally includes a considerable time delay due to the low sampling rate of the sensing system, the computational cost of image processing, and the latency of data transmission from the sensor to the processor. It also includes noise, which generally originates from ambient light conditions as well as the object's color, texture, pattern, and light reflection characteristics, among other sources. Furthermore, the most important concern is that such time delay and noise sometimes induce unstable behavior of the system as a whole when the information is used directly in a real-time servo controller. One practical way to avoid such unstable behavior is to choose each feedback gain to be sufficiently low, and many visual servo controllers have been designed in this way. This approach, however, introduces another problem; namely, it sacrifices control performance. In this chapter, we present a robust grasping and manipulation controller based on finger-thumb opposability. This approach can effectively employ visual sensing information even when it includes considerable time delay or noise without any performance sacrifice. First, a control scheme for object manipulation using a virtual-object frame is designed. Next, numerical simulations conducted to verify the effectiveness of the control scheme are reported. Finally, the practical usefulness of the proposed scheme is illustrated through experimental results, which emphasize that the proposed scheme is an effective way to exploit visual sensing information that includes considerable time delay and noise in a manipulation controller.