Resident evaluations in the age of competency-based medical education: faculty perspectives on minimizing burdens

View More View Less
  • 1 Injury Prevention Research Office, St. Michael's Hospital, Toronto;
  • 2 Department of Surgery, Division of Neurosurgery, University of Ottawa; and
  • 3 Department of Surgery, Division of Neurosurgery, University of Toronto, Ontario, Canada
Restricted access

Purchase Now

USD  $45.00

JNS + Pediatrics - 1 year subscription bundle (Individuals Only)

USD  $505.00

JNS + Pediatrics + Spine - 1 year subscription bundle (Individuals Only)

USD  $600.00
Print or Print + Online

OBJECTIVE

Competency-based medical education (CBME), an outcomes-based approach to medical education, continues to be implemented across many postgraduate medical education programs worldwide, including a recent introduction into Canadian neurosurgical training programs (July 2019). The success of this educational paradigm shift requires frequent faculty observation and evaluation of residents performing defined tasks of the specialty. A main challenge involves providing residents with frequent performance evaluations and feedback that are feasible for faculty to complete. This study aims to define what is currently happening and what changes are needed to make CBME successful for the certification of neurosurgeons' competence.

METHODS

A 55-item questionnaire was emailed nationwide to survey Canadian neurosurgical faculty.

RESULTS

Fifty-two complete responses were received and achieved a distribution highly correlated with the number of faculty neurosurgeons practicing in each Canadian province (Pearson's r = 0.94). Two-thirds (35/52) of faculty reported currently taking a median of 10 minutes to complete evaluation forms at the end of a resident's rotation block. Regardless of the faculty's province of practice (p = 0.50) or years of experience (p = 0.06), they reported 3 minutes (minimum 1 minute, maximum 10 minutes, interquartile range [IQR] 3 minutes) as a feasible amount of time to spend completing an evaluation form following an observation of a resident's performance of an entrustable professional activity (EPA). If evaluation forms took 3 minutes to complete, 85% of respondents (44/52) would complete EPA evaluations weekly or daily. The faculty recommended 5 minutes as a feasible amount of time to provide oral feedback (minimum 1 minute, maximum 20 minutes, IQR 3.25 minutes), which was significantly higher (p = 0.00099) than their recommended amount of time for completing evaluation forms. The majority of faculty (71%) stated they would prefer to access resident evaluation forms through a mobile application compared to a paper form (12%), an evaluation website (8%), or through a URL link sent via email (10%; p = 0.0032).

CONCLUSIONS

To facilitate the successful implementation of CBME into a neurosurgical training curriculum, resident EPA assessment forms should take 3 minutes or less to complete and be accessible through a mobile application.

ABBREVIATIONS CBME = competency-based medical education; EPA = entrustable professional activity; IQR = interquartile range; TBI = traumatic brain injury.

Supplementary Materials

    • Supplemental material (PDF 510 KB)

JNS + Pediatrics - 1 year subscription bundle (Individuals Only)

USD  $505.00

JNS + Pediatrics + Spine - 1 year subscription bundle (Individuals Only)

USD  $600.00

Contributor Notes

Correspondence Jessica Rabski: The Ottawa Hospital, Ottawa, ON, Canada. jrabs066@uottawa.ca.

INCLUDE WHEN CITING Published online December 11, 2020; DOI: 10.3171/2020.7.JNS201688.

Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this aper

  • 1

    Bloom BS. Learning for mastery. UCLA Evaluation Comment-CSEIP. 1968;1(2).

  • 2

    McGaghie WC, Miller GE, Sajid AW, Telder TW. Competency-Based Curriculum Development in Medical Education. World Health Organization; 1978. Accessed August 24, 2020. http://whqlibdoc.who.int/php/WHO_PHP_68.pdf

    • Search Google Scholar
    • Export Citation
  • 3

    Carraccio C, Englander R, Van Melle E, . Advancing competency-based medical education: a charter for clinician-educators. Acad Med. 2016;91(5):645649.

    • Search Google Scholar
    • Export Citation
  • 4

    Parent F, Jouquan J, De Ketele JM. CanMEDS and other “competency and outcome-based approaches” in medical education: clarifying the ongoing ambiguity. Adv Health Sci Educ Theory Pract. 2013;18(1):115122.

    • Search Google Scholar
    • Export Citation
  • 5

    Biemans H, Wesselink R, Gulikers J, . Towards competence-based VET: dealing with the pitfalls. J Vocat Educ Train. 2009;61(3):267286.

    • Search Google Scholar
    • Export Citation
  • 6

    Frank JR, Snell L, Englander R, Holmboe ES. Implementing competency-based medical education: moving forward. Med Teach. 2017;39(6):568573.

    • Search Google Scholar
    • Export Citation
  • 7

    Hodge, S.. The origins of competency-based training. Aust J Adult Learn. 2007;47(2):179209.

  • 8

    Maudsley RF, Dagg P, Wilson J, . Addressing societal health needs. In: Competence by Design: Reshaping Canadian Medical Education. Royal College of Physicians and Surgeons of Canada, 2014:817.

    • Search Google Scholar
    • Export Citation
  • 9

    ten Cate O.. Entrustability of professional activities and competency-based training. Med Educ. 2005;39(12):11761177.

  • 10

    Cate OT. A primer on entrustable professional activities. Korean J Med Educ. 2018;30(1):110.

  • 11

    Pichette J, Watkins EK. Competency-Based Education: Driving the Skills-Measurement Agenda. Higher Education Quality Council of Ontario; October 2018. Accessed August 24, 2020. http://www.heqco.ca/en-ca/Research/ResPub/Pages/Competency-based-Education-Driving-the-Skills-measurement-Agenda.aspx

    • Search Google Scholar
    • Export Citation
  • 12

    Selden N, Abosch A, Bambakidis N, . Neurological Surgery Milestones. The Accreditation Council for Graduate Medical Education. Second Revision: March 2018. Accessed August 24, 2020. https://www.acgme.org/Portals/0/PDFs/Milestones/NeurologicalSurgeryMilestones.pdf

    • Search Google Scholar
    • Export Citation
  • 13

    Pangaro L, ten Cate, O.. Frameworks for learner assessment in medicine: AMEE Guide No. 78. Med Teach. 2013;35(6): e1197e1210.

  • 14

    Bindal T, Wall D, Goodyear HM. Trainee doctors' views on workplace-based assessments: Are they just a tick box exercise? Med Teach. 2011;33(11):919927.

    • Search Google Scholar
    • Export Citation
  • 15

    Barrett A, Galvin R, Scherpbier AJJA, . Is the learning value of workplace-based assessment being realised? A qualitative study of trainer and trainee perceptions and expe riences. Postgrad Med, J. 2017;93(1097):138142.

    • Search Google Scholar
    • Export Citation
  • 16

    Menon S, Winston M, Sullivan, G.. Workplace-based assessment: attitudes and perceptions among consultant trainers and comparison with those of trainees. Psychiatr Bull. 2012; 36(1):1624.

    • Search Google Scholar
    • Export Citation
  • 17

    Abdelaal, A.. Procedure-based assessments: the past, the present and the future. Postgrad Med, J. 2020;96(1131):78.

  • 18

    Naeem, N.. Validity, reliability, feasibility, acceptability and educational impact of direct observation of procedural skills (DOPS). J Coll Physicians Surg Pak. 2013;23(1):7782.

    • Search Google Scholar
    • Export Citation
  • 19

    Sharma VK, Krishna M, Lepping P, . Validation and feasibility of the Global Mental Health Assessment Tool— Primary Care Version (GMHAT/PC) in older adults. Age Ageing. 2010;39(4):496499.

    • Search Google Scholar
    • Export Citation
  • 20

    Hadley C, Lam SK, Briceño V, . Use of a formal assessment instrument for evaluation of resident operative skills in pediatric neurosurgery. J Neurosurg Pediatr. 2015;16(5): 497504.

    • Search Google Scholar
    • Export Citation
  • 21

    Canadian Medical Association . Neurosurgery Profile. August 2018. Accessed August 24, 2020. https://www.cma.ca/sites/default/files/2019-01/neurosurgery-e.pdf

    • Search Google Scholar
    • Export Citation
  • 22

    Association of American Medical Colleges. Physician Specialty Data Report—Number of People per Active Physician by Specialty, 2017. Accessed August 24, 2020. https://www.aamc.org/data-reports/workforce/interactive-data/number-people-active-physician-specialty-2017

    • Search Google Scholar
    • Export Citation
  • 23

    Saleh A, Bista, K.. Examining factors impacting online survey response rates in educational research: perceptions of graduate students. J Multidiscip Eval. 2017;13(29):6374.

    • Search Google Scholar
    • Export Citation
  • 24

    Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med. 1995;123(10):795799.

    • Search Google Scholar
    • Export Citation
  • 25

    Downs SH, Black, N.. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52(6):377384.

    • Search Google Scholar
    • Export Citation
  • 26

    Maruthappu M, Trehan A, Barnett-Vanes A, . The impact of feedback of surgical outcome data on surgical performance: a systematic review. World J Surg. 2015;39(4): 879889.

    • Search Google Scholar
    • Export Citation
  • 27

    Yoo J-H. The meaning of information technology (IT) mobile devices to me, the infectious disease physician. Infect Chemother. 2013;45(2):244251.

    • Search Google Scholar
    • Export Citation
  • 28

    Ventola CL. Mobile devices and apps for health care professionals: uses and benefits. P T. 2014;39(5):356364.

  • 29

    Ten Cate O, Graafmans L, Posthumus I, . The EPA-based Utrecht undergraduate clinical curriculum: development and implementation. Med Teach. 2018;40(5):506513.

    • Search Google Scholar
    • Export Citation
  • 30

    Blades ML, Glaze S, McQuillan SK. Resident perspectives on competency-by-design curriculum. J Obstet Gynaecol Can. 2020;42(3):242247.

    • Search Google Scholar
    • Export Citation
  • 31

    Hanley M, Shearer C, Livingston, P.. Faculty perspectives on the transition to competency-based medical education in anesthesia. Can J Anaesth. 2019;66(11):13201327.

    • Search Google Scholar
    • Export Citation
  • 32

    Holmboe ES, Ward DS, Reznick RK, . Faculty development in assessment: the missing link in competency-based medical education. Acad Med. 2011;86(4):460467.

    • Search Google Scholar
    • Export Citation
  • 33

    Tomiak A, Braund H, Egan R, . Exploring how the new Entrustable Professional Activity assessment tools affect the quality of feedback given to medical oncology residents. J Cancer Educ. 2020;35(1):165177.

    • Search Google Scholar
    • Export Citation
  • 34

    Diaz Voss Varela DA, Malik MU, Thompson CB, . Comprehensive assessment of thyroidectomy skills development: a pilot project. Laryngoscope. 2012;122(1):103109.

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 260 260 79
Full Text Views 35 35 9
PDF Downloads 23 23 10
EPUB Downloads 0 0 0