Quality Assurance of Operating Room-Based Learning: Development & Validation of ‘Surgical Operating Room Educational Experience Measure’ for Medical Students (SOREEMST)

Abstract

Introduction: Operating Room (OR) based learning experience has traditionally remained challenging owing to complex medical, psychosocial, educational and administrative factors and, so is its measurement of success. Recently, mini-STEEM, an abbreviated form of Surgical Theater Educational Experience Measure (STEEM) has been employed to evaluate the OR-based learning experience of medical students. However, its content and construct validity has been challenged, in light of updated extensive literature review, justifying the need for a more robust, reliable and content valid instrument.

Methods: Extensive literature review was done to identify and evaluate various factors affecting students’ OR based learning within the OR setting and a conceptual framework was developed. Artino et al’s (AMEE Guide 87) seven-step approach was used to develop and validate this new instrument, which has been named Surgical Operating Room Educational Experience Measure (SOREEM^ST). Content Validity Index (I-CVI) was measured, with a range between 0.68-1 for all items. Following principles of Structural Equation Modeling (SEM) and piloting on 535 students, Exploratory Factor Analysis (EFA) was carried out to finally retain 50 items for subsequent model fitness in Confirmatory Factor Analysis (CFA) through SPSS-AMOS 24.

Results: Against previously identified 26 constructs, 100 items were initially designed and expert-validated for clarity, relevance and comprehension. Cognitive interviews were done to optimize item clarity and comprehension, and finally the items were piloted on a pool of 536 students. Reliability and internal consistency were analyzed through Cronbach’s alpha and exploratory factor analysis (EFA) to refine and choose final 50 items for SOREEM^ST. Which model best fit to conceptual framework was confirmed through Confirmatory Factor Analysis (CFA) and Structural Equation Modeling (SEM).

Conclusion: SOREEM^ST is a reliable, novel, content/construct valid instrument to assess the quality of the OR based learning experience of medical students and may be used to quality assure the OR-based learning process.