Action Input Interface of IntelligentBox Using 360-Degree VR Camera and OpenPose for Multi-persons’ Collaborative VR Applications

This paper proposes action input interface of IntelligentBox for multi-persons’ collaborative VR applications. IntelligentBox ia a component-based constructive 3D graphics application development system. One of the application fields of IntelligentBox is VR. VR applications should support various types of VR peripherals, e.g., HMD (Head Mount Display), Data Gloves and so on. IntelligentBox supports most of them as dedicated software components. As for action input interfaces of VR applications, they support a motion capture system and MS Kinect. However, basically, such input interface supports one person’s actions. For multi-persons’ collaborative VR applications, the other action input interface is needed. Then, this paper proposes such an action input interface using 360° VR camera and OpenPose. In the interface, first a 360° VR camera captures all the persons’ images in the real world at the same time. Then, OpenPose recognizes all the persons’ poses in real time and output those poses as their skeletons in the json format. This time, the authors developed one dedicated software that generates multi-persons’ motion data from the output json format data and send them into a certain component of IntelligentBox through socket communication. The developed software works as a server of the server-client framework so that it can be used as a general action input interface for other client applications.