Virtual Shadow Drawing System Using Augmented Reality for Laparoscopic Surgery

Satoshi Miura, Masaki Seki, Yuta Koreeda, Yang Cao, Kazuya Kawamura, Yo Kobayashi, Masakatsu G. Fujie, Tomoyuki Miyashita
Vol. 11 (2022) p.87-97

Laparoscopic surgery holds great promise in medicine but remains challenging for surgeons because it is difficult to perceive depth while suturing. In addition to binocular parallax, such as three-dimensional vision, shadow is essential for depth perception. This paper presents an augmented reality system that draws virtual shadows to aid depth perception. On the visual display, the system generates shadows that mimic actual shadows by estimating shadow positions using image processing. The distance and angle between the forceps tip and the surface were estimated to evaluate the accuracy of the system. To validate the usefulness of this system in surgical applications, novices performed suturing tasks with and without the augmented reality system. The system error and delay were sufficiently small, and the generated shadows were similar to actual shadows. Furthermore, the suturing error decreased significantly when the augmented reality system was used. The shadow-drawing system developed in this study may help surgeons perceive depth during laparoscopic surgery.