76
· DOS Abstracts
Testing basic competency in knee arthroscopy using
a virtual reality simulator: Exploring reliability and
validity
Mads Emil Jacobsen, Morten Jon Andersen, Claus Ol Hansen, Lars Konge
Center of Clinical Education, Copenhagen University Hospital Rigshospitalet,
Denmark; Department of Orthopedic Surgery, Copenhagen University Hospital
Hvidovre, Denmark; Arthroscopic Center Amager, Copenhagen University
Hospital Amager, Denmark
Background:
Orthopedic residents are confronted with diagnostic arthrosco-
pies early in their training. Arthroscopy differs from traditional open surgery and
many residents feel less prepared for arthroscopic procedures. Virtual reality
(VR) training and testing could ensure basic competency before proceeding to
supervised operations in patients.
Purpose / Aim of Study:
The aim of this study was to create a reliable and
valid simulation-based test for diagnostic knee arthroscopy.
Materials and Methods:
Twenty-six physicians (13 untrained novices and 13
experienced surgeons) performed a test consisting of five procedures on a VR
knee arthroscopy simulator. Performance was evaluated by obtaining predefined
metrics from the simulator for each procedure from which z-scores, describing
inappropriate behavior, were calculated. Reliability was explored by calculating
an Intra-class Correlation Coefficient. Validity was explored by comparing the
performances of novices and experienced surgeons using independent samples
t-tests. Using the contrasting groups method a pass/fail standard was set and
its consequences explored.
Findings / Results:
One procedure was excluded from the final test due to lack
of validity. Z-scores showed an inter-case reliability of 0.87 between the four
procedures in the final test. A total z-score was 38.6 ± 27.3 for the novices and
0.0 ± 9.1 for the experienced surgeons (p < 0.0005). The pass/fail standard
was set at a total z-score of 15.5 resulting in two of the novices passing the
test and a single experienced surgeon failing.
Conclusions:
By combining four procedures on a VR arthroscopy simulator we
were able to create a valid and reliable test of basic arthroscopic competency,
and establish a credible pass/fail standard. The test could help assess and ensure
basic competency of junior surgeons before supervised operations in patients.
23.
