Artificial intelligence facilitates the potential of simulator training: An innovative laparoscopic surgical skill validation system using artificial intelligence technology

Purpose: The development of innovative solutions, such as simulator training and artificial intelligence (AI)-powered tutoring systems, has significantly changed surgical trainees’ environments to receive the intraoperative instruction necessary for skill acquisition. In this study, we developed a new objective assessment system using AI for forceps manipulation in a surgical training simulator. Methods: Laparoscopic exercises were recorded using an iPad®, which provided top and side views. Top-view movies were used for AI learning of forceps trajectory. Side-view movies were used as supplementary information to assess the situation. We used an AI-based posture estimation method, DeepLabCut (DLC), to recognize and positionally measure the forceps in the operating field. Tracking accuracy was quantitatively evaluated by calculating the pixel differences between the annotation points and the points predicted by the AI model. Tracking stability at specified key points was verified to assess the AI model. Results: We selected a random sample to evaluate tracking accuracy quantitatively. This sample comprised 5% of the frames not used for AI training from the complete set of video frames. We compared the AI detection positions and correct positions and found an average pixel discrepancy of 9.2. The qualitative evaluation of the tracking stability was good at the forceps hinge; however, forceps tip tracking was unstable during rotation. Conclusion: The AI-based forceps tracking system can visualize and evaluate laparoscopic surgical skills. Improvements in the proposed system and AI self-learning are expected to enable it to distinguish the techniques of expert and novice surgeons accurately. This system is a useful tool for surgeon training and assessment.