Real-time 3D hand shape estimation based on image feature analysis and inverse kinematics

Vision-based hand shape estimation is a challenging task, since the hand presents a motion of high degrees of freedom and since self-occlusions of different fingers bring a lot of uncertainty for the occluded parts. Considering that the influence of self-occlusions may be reduced by observing multiple images, we propose a multiple view system to obtain hand features, with using previous information that facilitates very robust feature extraction. The extracted features are then used to compute approximate global state of hand and perform preliminary estimation of the local state of each finger by Inverse Kinematics (IK). By minimizing the estimation error between groups of model features and groups of image features, some model parameters are refined and IK is recomputed, which contributes to enhance estimation accuracy. To reduce the estimation complexity due to the high degrees of freedom of the hand, we combine IK with the motion constraints of articulated hand. The effectiveness of our approach is demonstrated with experiments on a number of different hand motions with finger articulation and global hand rotation under complex background.