Although flexible ureteroscopy is the most common surgical procedure for treating urolithiasis, no method has ever been validated for assessing ureteroscopic skills.

Researchers at Brigham and Women’s Hospital previously addressed this problem in a pilot study published in 2022. In that study, Marie-Therese Valovska, MD, chief resident in the Department of Urology at Brigham and Women’s Hospital, Daniel A. Wollin, MD, MS, a urologic surgeon in the Department and an assistant professor of Surgery at Harvard Medical School, and colleagues established the feasibility of using motion capture data to understand surgeons’ efficiency during simulated ureteroscopy better.

In a new effort reported in the Journal of Endourology, the same lead researchers devised an even more robust surgeon efficiency score, applying computer vision and artificial intelligence (AI). This latest work takes them closer to their goal of creating a fully automated simulation-based system that can reliably differentiate between novice and expert ureteroscope handling.

Methods

The researchers enlisted the help of 12 urologic surgeons in different stages of their careers (average years of experience, 10; range, 1– 35). The surgeons were asked to guide a flexible ureteroscope to various target stones inside a simulation box, contact each stone in order and remove the ureteroscope from the box, then repeat the exercise twice.

A camera in a fixed position in the simulation box during all experiments captured ureteroscope movements during each task. Another was placed on the surgeon’s side to capture surgeon movements during the tasks.

Task videos were analyzed with computer vision techniques to extract three metrics automatically:

• Ureteroscope tip travel distance (DIST)
• Task time (TIME)
• Spectral arc length (SPARC, a measure of the overall smoothness of the scope’s movement)

A computer algorithm was devised to automatically calculate a fourth metric: the percentage of purposeful or efficient collisions with the edge of the simulator box compared with accidental collisions (COLL).

The normalized values of these metrics were aggregated (SPARC + COLL − DIST − TIME) to create a composite ureteroscopy efficiency score (CUES).

Results

Compared with the first attempt, both the second and third attempts showed statistically significant improvements in mean DIST, TIME, SPARC, and COLL. This progressive improvement suggests CUES holds promise for assessing ureteroscopic skills.

The mean CUES of residents denoted significantly less efficiency compared with attending physicians (0.02 vs. 0.44; P=0.04), further suggesting an association between skill and CUES.

Future Directions

These findings serve as proof of concept that a higher CUES correlates with improved efficiency.

The research team aims for this novel method to be integrated into simulation training or operating rooms to provide trainees and established urologists with automated, objective assessment that encourages lifelong learning and skills improvement.