Resident evaluations in the age of competency-based medical education: faculty perspectives on minimizing burdens

Jessica E. Rabski Injury Prevention Research Office, St. Michael's Hospital, Toronto;
Injury Prevention Research Office, St. Michael's Hospital, Toronto;

Search for other papers by Jessica E. Rabski in
Current site
Google Scholar
PubMed
Close
 MD, MEd
,
Ashirbani Saha Injury Prevention Research Office, St. Michael's Hospital, Toronto;

Search for other papers by Ashirbani Saha in
Current site
Google Scholar
PubMed
Close
 PhD
,
Michael D. Cusimano
Search for other papers by Michael D. Cusimano in
Current site
Google Scholar
PubMed
Close
 MD
, and
FRCSC DABNS Injury Prevention Research Office, St. Michael's Hospital, Toronto;
Department of Surgery, Division of Neurosurgery, University of Toronto, Ontario, Canada

Search for other papers by FRCSC DABNS in
Current site
Google Scholar
PubMed
Close
 PhD, MHPE
Restricted access

Purchase Now

USD  $45.00

JNS + Pediatrics - 1 year subscription bundle (Individuals Only)

USD  $536.00

JNS + Pediatrics + Spine - 1 year subscription bundle (Individuals Only)

USD  $636.00
USD  $45.00
USD  $536.00
USD  $636.00
Print or Print + Online Sign in

OBJECTIVE

Competency-based medical education (CBME), an outcomes-based approach to medical education, continues to be implemented across many postgraduate medical education programs worldwide, including a recent introduction into Canadian neurosurgical training programs (July 2019). The success of this educational paradigm shift requires frequent faculty observation and evaluation of residents performing defined tasks of the specialty. A main challenge involves providing residents with frequent performance evaluations and feedback that are feasible for faculty to complete. This study aims to define what is currently happening and what changes are needed to make CBME successful for the certification of neurosurgeons' competence.

METHODS

A 55-item questionnaire was emailed nationwide to survey Canadian neurosurgical faculty.

RESULTS

Fifty-two complete responses were received and achieved a distribution highly correlated with the number of faculty neurosurgeons practicing in each Canadian province (Pearson's r = 0.94). Two-thirds (35/52) of faculty reported currently taking a median of 10 minutes to complete evaluation forms at the end of a resident's rotation block. Regardless of the faculty's province of practice (p = 0.50) or years of experience (p = 0.06), they reported 3 minutes (minimum 1 minute, maximum 10 minutes, interquartile range [IQR] 3 minutes) as a feasible amount of time to spend completing an evaluation form following an observation of a resident's performance of an entrustable professional activity (EPA). If evaluation forms took 3 minutes to complete, 85% of respondents (44/52) would complete EPA evaluations weekly or daily. The faculty recommended 5 minutes as a feasible amount of time to provide oral feedback (minimum 1 minute, maximum 20 minutes, IQR 3.25 minutes), which was significantly higher (p = 0.00099) than their recommended amount of time for completing evaluation forms. The majority of faculty (71%) stated they would prefer to access resident evaluation forms through a mobile application compared to a paper form (12%), an evaluation website (8%), or through a URL link sent via email (10%; p = 0.0032).

CONCLUSIONS

To facilitate the successful implementation of CBME into a neurosurgical training curriculum, resident EPA assessment forms should take 3 minutes or less to complete and be accessible through a mobile application.

ABBREVIATIONS

CBME = competency-based medical education ; EPA = entrustable professional activity ; IQR = interquartile range ; TBI = traumatic brain injury .

Supplementary Materials

    • Supplemental material (PDF 510 KB)
  • Collapse
  • Expand

Artist’s illustration of the classic mulberry appearance of a cavernoma. This illustration represents the Seven Cavernomas series by Dr. Michael Lawton, a collection of articles defining the tenets and techniques for the treatment of cavernous malformations, a taxonomy for classifying these lesions, and the nuances of their surgical approaches. Artist: Peter M. Lawrence. Used with permission from Barrow Neurological Institute, Phoenix, Arizona. See the article by Garcia et al. (pp 671–682).

  • 1

    Bloom BS . Learning for mastery . UCLA Evaluation Comment-CSEIP . 1968 ;1 (2 ).

  • 2

    McGaghie WC , Miller GE , Sajid AW , Telder TW . Competency-Based Curriculum Development in Medical Education . World Health Organization ; 1978 . Accessed August 24, 2020 . http://whqlibdoc.who.int/php/WHO_PHP_68.pdf

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Carraccio C , Englander R , Van Melle E , et al. Advancing competency-based medical education: a charter for clinician-educators . Acad Med . 2016 ;91 (5 ):645 649 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Parent F , Jouquan J , De Ketele JM . CanMEDS and other “competency and outcome-based approaches” in medical education: clarifying the ongoing ambiguity . Adv Health Sci Educ Theory Pract . 2013 ;18 (1 ):115 122 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5

    Biemans H , Wesselink R , Gulikers J , et al. Towards competence-based VET: dealing with the pitfalls . J Vocat Educ Train . 2009 ;61 (3 ):267 286 .

  • 6

    Frank JR , Snell L , Englander R , Holmboe ES . Implementing competency-based medical education: moving forward . Med Teach . 2017 ;39 (6 ):568 573 .

  • 7

    Hodge , S. . The origins of competency-based training . Aust J Adult Learn . 2007 ;47 (2 ):179 209 .

  • 8

    Maudsley RF , Dagg P , Wilson J , et al. Addressing societal health needs. In: Competence by Design: Reshaping Canadian Medical Education . Royal College of Physicians and Surgeons of Canada , 2014 :8 17 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    ten Cate O. . Entrustability of professional activities and competency-based training . Med Educ . 2005 ;39 (12 ):1176 1177 .

  • 10

    Cate OT . A primer on entrustable professional activities . Korean J Med Educ . 2018 ;30 (1 ):1 10 .

  • 11

    Pichette J , Watkins EK . Competency-Based Education: Driving the Skills-Measurement Agenda . Higher Education Quality Council of Ontario; October 2018 . Accessed August 24, 2020 . http://www.heqco.ca/en-ca/Research/ResPub/Pages/Competency-based-Education-Driving-the-Skills-measurement-Agenda.aspx

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12

    Selden N , Abosch A , Bambakidis N , et al. Neurological Surgery Milestones. The Accreditation Council for Graduate Medical Education . Second Revision : March 2018 . Accessed August 24, 2020 . https://www.acgme.org/Portals/0/PDFs/Milestones/NeurologicalSurgeryMilestones.pdf

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 13

    Pangaro L , ten Cate , O. . Frameworks for learner assessment in medicine: AMEE Guide No. 78 . Med Teach . 2013 ;35 (6 ): e1197e1210 .

  • 14

    Bindal T , Wall D , Goodyear HM . Trainee doctors' views on workplace-based assessments: Are they just a tick box exercise? Med Teach . 2011 ;33 (11 ):919 927 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15

    Barrett A , Galvin R , Scherpbier AJJA , et al. Is the learning value of workplace-based assessment being realised? A qualitative study of trainer and trainee perceptions and expe riences . Postgrad Med, J . 2017 ;93 (1097 ):138 142 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16

    Menon S , Winston M , Sullivan , G. . Workplace-based assessment: attitudes and perceptions among consultant trainers and comparison with those of trainees . Psychiatr Bull . 2012 ; 36 (1 ):1624 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17

    Abdelaal , A. . Procedure-based assessments: the past, the present and the future . Postgrad Med, J . 2020 ;96 (1131 ):7 8 .

  • 18

    Naeem , N. . Validity, reliability, feasibility, acceptability and educational impact of direct observation of procedural skills (DOPS) . J Coll Physicians Surg Pak . 2013 ;23 (1 ):77 82 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 19

    Sharma VK , Krishna M , Lepping P , et al. Validation and feasibility of the Global Mental Health Assessment Tool— Primary Care Version (GMHAT/PC) in older adults . Age Ageing . 2010 ;39 (4 ):496 499 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20

    Hadley C , Lam SK , Briceño V , et al. Use of a formal assessment instrument for evaluation of resident operative skills in pediatric neurosurgery . J Neurosurg Pediatr . 2015 ;16 (5 ): 497 504 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21

    Canadian Medical Association . Neurosurgery Profile . August 2018 . Accessed August 24, 2020 . https://www.cma.ca/sites/default/files/2019-01/neurosurgery-e.pdf

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22

    Association of American Medical Colleges. Physician Specialty Data Report—Number of People per Active Physician by Specialty , 2017 . Accessed August 24, 2020 . https://www.aamc.org/data-reports/workforce/interactive-data/number-people-active-physician-specialty-2017

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23

    Saleh A , Bista , K. . Examining factors impacting online survey response rates in educational research: perceptions of graduate students . J Multidiscip Eval . 2017 ;13 (29 ):63 74 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24

    Norcini JJ , Blank LL , Arnold GK , Kimball HR . The mini-CEX (clinical evaluation exercise): a preliminary investigation . Ann Intern Med . 1995 ;123 (10 ):795 799 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 25

    Downs SH , Black , N. . The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions . J Epidemiol Community Health . 1998 ;52 (6 ):377 384 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26

    Maruthappu M , Trehan A , Barnett-Vanes A , et al. The impact of feedback of surgical outcome data on surgical performance: a systematic review . World J Surg . 2015 ;39 (4 ): 879 889 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27

    Yoo J-H . The meaning of information technology (IT) mobile devices to me, the infectious disease physician . Infect Chemother . 2013 ;45 (2 ):244 251 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 28

    Ventola CL . Mobile devices and apps for health care professionals: uses and benefits . P T . 2014 ;39 (5 ):356 364 .

  • 29

    Ten Cate O , Graafmans L , Posthumus I , et al. The EPA-based Utrecht undergraduate clinical curriculum: development and implementation . Med Teach . 2018 ;40 (5 ):506 513 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30

    Blades ML , Glaze S , McQuillan SK . Resident perspectives on competency-by-design curriculum . J Obstet Gynaecol Can . 2020 ;42 (3 ):242 247 .

  • 31

    Hanley M , Shearer C , Livingston , P. . Faculty perspectives on the transition to competency-based medical education in anesthesia . Can J Anaesth . 2019 ;66 (11 ):1320 1327 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32

    Holmboe ES , Ward DS , Reznick RK , et al. Faculty development in assessment: the missing link in competency-based medical education . Acad Med . 2011 ;86 (4 ):460 467 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33

    Tomiak A , Braund H , Egan R , et al. Exploring how the new Entrustable Professional Activity assessment tools affect the quality of feedback given to medical oncology residents . J Cancer Educ . 2020 ;35 (1 ):165 177 .

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 34

    Diaz Voss Varela DA , Malik MU , Thompson CB , et al. Comprehensive assessment of thyroidectomy skills development: a pilot project . Laryngoscope . 2012 ;122 (1 ):103 109 .

    • PubMed
    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 1196 220 18
Full Text Views 137 27 0
PDF Downloads 214 40 0
EPUB Downloads 0 0 0