Utility of Workplace-Based Assessment Among Surgical Residents: A Systematic Literature Review
DOI:
https://doi.org/10.48111/2022.02.03Keywords:
Resident Training, Surgical Training, Assessment, Work-based assessment, Direct Observation of Procedural Skills, Clinical Evaluation Exercise, Case based discussion, Procedure based assessmentAbstract
IMPORTANCE This review article aims to assess the usefulness of Workplace-Based Assessment (WBA) among surgical residents. Multiple studies have been done in the past to determine the utility of WBAs across different fields of medicine. But it hasn’t been done among surgical specialties on large scale, and the present paper fills in this research gap.
METHODS Articles were searched using PubMed and Google Scholar databases up to 30th June 2022. Following headings were used for the literature search i.e work-based assessment or workplace-based assessment or WBAs or direct observation of procedural skills or DOPS or procedure-based assessments or PBA or Case-based discussion or CBD or clinical evaluation exercise or CEX or Mini CEX, and surgical training. The utility formula of Vleuten was used to assess the usefulness of WBA. It is a product of validity, reliability, educational impact, acceptability, cost-effectiveness, and feasibility.
RESULTS For reviewing purposes, 29 studies were selected. There were 8 studies on PBA, 2 on Mini-CEX, 2 on CBD, 6 on DOPS, and 11 studies on multiple methods of WBA. PBA had positive satisfaction levels and was valid, reliable, feasible, and acceptable. CBD. Mini-CEX and CBD had very few studies in which they were proved to be reliable and valid but had mixed responses in terms of satisfaction levels. DOPS had a level 1 educational impact and also proved to be valid. When all components of WBAs were used together, a negative level 1 educational impact was observed. The concerns that were identified include lack of time, training and evidence of validity, design of the tool, and perception of WBA as a summative tool.
DISCUSSION Work-based Assessments should be implemented and used properly after the training of faculty. The focus should be on the quality of the assessments, not on the quantity. Large-scale studies should be done to assess level 3 and 4 of educational impact from WBAs.
References
Ahmed I. Post Graduate Surgical Training: Past, Present and Future in Pakistan. www.jpmsonline.com
childhood MBA of disease in, 2006 undefined. What’s happening in postgraduate medical education? adc.bmj.com. Accessed July 5, 2022. https://adc.bmj.com/content/91/1/68?max-results=200
Swanwick T, Practice NCBJ of G, 2005 undefined. Workplace assessment for licensing in general practice. bjgp.org. Accessed July 5, 2022. https://bjgp.org/content/55/515/461.short
Swanwick T, Chana N. Workplace-based assessment. British Journal of Hospital Medicine. 2009;70(5):290-293. doi:10.12968/HMED.2009.70.5.42235
ISCP. Intercollegiate surgical curriculum overview, The Intercollegiate Curriculum, Educating the Surgeons of the Future, Intercollegiate Surgical Curriculum Project, 2017. Accessed in may 2021 from, https://www.iscp.ac.uk/curriculum/surgica l/assessment_wbas.aspx. Published 2021. Accessed July 5, 2022. https://www.iscp.ac.uk/curriculum/surgica l/assessment_wbas.aspx.
Rowley D, Wass V, Myerson Keith. Supplementary Guidance Workplace Based Assessment: A Guide for Implementation.; 2010.
Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63-7. doi:10.1097/00001888-199009000-00045
Saedon H, Salleh S, Balakrishnan A, Imray CH, Saedon M. The role of feedback in improving the effectiveness of workplace based assessments: A systematic review. BMC Medical Education. 2012;12(1). doi:10.1186/1472-6920-12-25
Satpathy A, Kneebone R. Workplace-Based Assessment Insurgical Training: A Qualitative Study. The Bulletin of the Royal College of Surgeons of England. 2011;93(10):1-5. doi:10.1308/147363511x595082
Raj Aryal K. The Utility of Work-Based assessments in higher general surgical training-a systematic review. Published online 2019. doi:10.21203/rs.2.16654/v1
Norcini JJ. The Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation. Annals of Internal Medicine. 1995;123(10):795. doi:10.7326/0003-4819-123-10-199511150-00008
Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Medical Teacher. 2007;29(9-10):855-871. doi:10.1080/01421590701775453
Mathew A, Beard J, Bussey M. An evaluation of the use of direct observation of procedural skills in the intercollegiate surgical curriculum programme. Bulletin of The Royal College of Surgeons of England. 2014;96(9):10-13. doi:10.1308/147363514x14042954769519
Barrett A, Galvin R, Steinert Y, et al. A BEME (Best Evidence in Medical Education) review of the use of workplace-based assessment in identifying and remediating underperformance among postgraduate medical trainees: BEME Guide No. 43. Medical Teacher. 2016;38(12):1188-1198. doi:10.1080/0142159X.2016.1215413
Miller A, Archer J. Impact of workplace based assessment on doctors’ education and performance: a systematic review. BMJ. 2010;341(sep24 1):c5064-c5064. doi:10.1136/bmj.c5064
Aryal K, Hamed M, Currow C. The usefulness of work-based assessments in higher surgical training: A systematic review. International Journal of Surgery. 2021;94. doi:10.1016/j.ijsu.2021.106127
Satpathy A, Kneebone R. Workplace-Based Assessment Insurgical Training: A Qualitative Study. The Bulletin of the Royal College of Surgeons of England. 2011;93(10):1-5. doi:10.1308/147363511X595082
Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic Reviews. 2021;10(1):89. doi:10.1186/s13643-021-01626-4
van der Vleuten CPM. The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education. 1996;1(1):41-67. doi:10.1007/BF00596229
Fowell SL, Southgate LJ, Bligh JG. Evaluating assessment: the missing link? Med Educ. 1999;33(4):276-281. doi:10.1046/j.1365-2923.1999.00405.x
Everitt S, Howell C. Encyclopedia of Statistics in Behavioral Science. 2nd Edition.; 2005.
Kirkpatrick. Evaluation of training. In: R.L. Craig, L.R. Bittel, eds. Training and Development Handbook. ; 1967:87-112.
Crossley J, Humphris G, Jolly B. Assessing health professionals. Medical Education. 2002;36(9):800-804. doi:10.1046/j.1365-2923.2002.01294.x
Medeiros DCA de, Medeiros PJ de, Diniz RVZ. Procedure-based assessment (PBA) in urology residency: initial experience. Revista Brasileira de Educação Médica. 2021;45(4). doi:10.1590/1981-5271v45.4-20210346.ing
Ramhamadany E. Orthopaedic Trainee Opinion on the Current Procedure-based Assessment Compared to its Predecessor: A UK Trauma Centre Perspective. Cureus. Published online April 23, 2020. doi:10.7759/cureus.7799
Shalhoub J, Marshall DC, Ippolito K. Perspectives on procedure-based assessments: a thematic analysis of semistructured interviews with 10 UK surgical trainees. BMJ Open. 2017;7:13417. doi:10.1136/bmjopen-2016
Roushdi I, Tennent D. Current usage patterns of procedure-based assessments in the orthopaedic community. Bulletin of The Royal College of Surgeons of England. 2015;97(2):1-3. doi:10.1308/147363515x14134529300148
Hunter AR, Baird EJ, Reed MR. Procedure-based assessments in trauma and orthopaedic training-The trainees’ perspective. Medical Teacher. 2015;37(5):444-449. doi:10.3109/0142159X.2014.956055
Marriott J, Purdie H, Crossley J, Beard JD. Evaluation of procedure-based assessment for assessing trainees skills in the operating theatre. British Journal of Surgery. 2011;98(3):450-457. doi:10.1002/bjs.7342
Beard J, Marriott J, Purdie H, Crossley J. Assessing the surgical skills of trainees in the operating theatre: a prospective observational study of the methodology. Health Technology Assessment. 2011;15(01). doi:10.3310/hta15010
James K, Cross K, Lucarotti ME, Fowler AL, Cook TA. Undertaking procedure-based assessment is feasible in clinical practice. Ann R Coll Surg Engl. 2009;91(2):110-112. doi:10.1308/003588409X359286
Deshpande S, Gudhe M. Utility of mini-CEX in assessing postgraduates competency in orthopedics. Journal of Datta Meghe Institute of Medical Sciences University. 2021;16(1):52-56. doi:10.4103/jdmimsu.jdmimsu_240_20
Joshi M, Singh T, Badyal D. Acceptability and feasibility of mini-clinical evaluation exercise as a formative assessment tool for workplace-based assessment for surgical postgraduate students. Journal of Postgraduate Medicine. 2017;63(2):100. doi:10.4103/0022-3859.201411
Phillips A, Lim J, Madhavan A, Macafee D. Case-based discussions: UK surgical trainee perceptions. The Clinical Teacher. 2016;13(3):207-212. doi:10.1111/tct.12411
Awad Z, Hayden L, Muthuswamy K, Ziprin P, Darzi A, Tolley NS. Utilisation and outcomes of Case-based discussion in otolaryngology training. Clinical Otolaryngology. 2015;40(2):86-92. doi:10.1111/coa.12321
Inamdar P, Hota PK, Undi M. Feasibility and Effectiveness of Direct Observation of Procedure Skills (DOPS) in General Surgery discipline: a Pilot Study. Indian Journal of Surgery 2021 84:1. 2021;84(1):109-114. doi:10.1007/S12262-021-02953-5
Sethi S, Badyal DK. Clinical procedural skills assessment during internship in ophthalmology. J Adv Med Educ Prof. 2019;7(2):56-61.
Hussain Waqar S, Ali SZ. Direct Observation of Procedural Skills As Assessment Tool Pak Armed Forces Med. Vol 66.; 2016.
Ali L, Ali S, Orakzai N, Ali N. Effectiveness of Direct Observation of Procedural Skills (DOPS) in Postgraduate Training in Urology at Institute of Kidney Diseases, Peshawar. Journal of the College of Physicians and Surgeons Pakistan. 2019;29(6):516-519. doi:10.29271/jcpsp.2019.06.516
Awad Z, Hayden L, Muthuswamy K, Ziprin P, Darzi A, Tolley NS. Does direct observation of procedural skills reflect trainee’s progress in otolaryngology? Clinical Otolaryngology. 2014;39(3):169-173. doi:10.1111/coa.12251
Aryal KR, Currow C, Downey S, Praseedom R, Seager A. Work-Based Assessments in Higher General Surgical Training Program: A Mixed Methods Study Exploring Trainers’ and Trainees’ Views and Experiences. The Surgery Journal. 2020;06(01):e49-e61. doi:10.1055/s-0040-1708062
Nathoo NA, Sidhu R, Gingerich A. Educational Impact Drives Feasibility of Implementing Daily Assessment in the Workplace. Teaching and Learning in Medicine. 2020;32(4):389-398. doi:10.1080/10401334.2020.1729162
Gaunt A, Patel A, Rusius V, Royle TJ, Markham DH, Pawlikowska T. ‘Playing the game’: How do surgical trainees seek feedback using workplace-based assessment? Medical Education. 2017;51(9):953-962. doi:10.1111/medu.13380
Gaunt A, Patel A, Royle J, et al. What do surgeons and trainees think of WBAs and how do they use them? The Bulletin of the Royal College of Surgeons of England. 2016;98(9):408-414. doi:10.1308/rcsbull.2016.408
Pentlow A, Field J. The educational value of work-based assessments: a survey of orthopaedic trainees and their consultant trainers. Journal of Surgical Simulation. 2015;2. doi:10.1102/2051-7726.2015.0013
Phillips A, Jones A. The validity and reliability of workplacebased assessments in surgical training. The Bulletin of the Royal College of Surgeons of England. 2015;97(3):e19-e23. doi:10.1308/147363515X14134529301147
Eardley I, Bussey M, Woodthorpe A, Munsch C, Beard J. Workplace-based assessment in surgical training: experiences from the Intercollegiate Surgical Curriculum Programme. ANZ Journal of Surgery. 2013;83(6):448-453. doi:10.1111/ans.12187
Ali AS, Bussey M, O’Flynn KJ, Eardley I. Quality of Feedback Using Workplace Based Assessments in Urological Training. British Journal of Medical and Surgical Urology. 2012;5(1):39-43. doi:10.1016/j.bjmsu.2011.10.001
Pereira EAC, Dean BJF. British Surgeons’ Experiences of a Mandatory Online Workplace Based Assessment Portfolio Resurveyed Three Years On. Journal of Surgical Education. 2013;70(1):59-67. doi:10.1016/j.jsurg.2012.06.019
Pereira EA, Dean BJ. British surgeons’ experiences of mandatory online workplace-based assessment. J R Soc Med. 2009;102(7):287-293. doi:10.1258/jrsm.2009.080398
Sultan AS, Ali R, Shakil S, Khan RN. Workplace Based Assessment: Tools to assess competencies in a clinical setting. J Pak Med Assoc. 2021;71(Suppl 1)(1):S89-S93.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Nabila Talat, Muhammad Usama Aziz
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
