Correlation of personality assessments with standard selection criteria for neurosurgical residency applicants

Full access

OBJECTIVE

Neurosurgery is among the most competitive residencies, as evidenced by the high number of applicants for relatively few positions. Although it is important to recruit candidates who have the intellectual capacity and drive to succeed, traditional objective selection criteria, such as US Medical Licensing Examination (USMLE) (also known as Step 1) score, number of publications, and class ranking, have not been shown to consistently predict clinical and academic success. Furthermore, these traditional objective parameters have not been associated with specific personality traits.

METHODS

The authors sought to determine the efficacy of a personality assessment in the selection of neurosurgery residents. Specifically, the aim was to determine the correlation between traditional measures used to evaluate an applicant (e.g., USMLE score, number of publications, MD/PhD status) and corresponding validated personality traits.

RESULTS

Fifty-four neurosurgery residency applicants were interviewed at the Cleveland Clinic during the 2014–2015 application cycle. No differences in validated personality scores were identified between the 46 MD applicants and 8 MD/PhD applicants. The mean USMLE score (± SD) was 252.3 ± 11.9, and those in the high-USMLE-score category (USMLE score ≥ 260) had a significantly lower “imaginative” score (a stress measure of eccentric thinking and impatience with those who think more slowly). The average number of publications per applicant was 8.6 ± 7.9, and there was a significant positive correlation (r = 0.339, p = 0.016) between greater number of publications and a higher “adjustment” score (a measure of being even-tempered, having composure under pressure). Significant negative correlations existed between the total number of publications and the “excitable” score (a measure of being emotionally volatile) (r = −0.299, p = 0.035) as well as the “skeptical” score (measure of being sensitive to criticism) (r = −0.325, p = 0.021). The average medical school rank was 25.8, and medical school rankings were positively correlated with the “imaginative” score (r = 0.287, p = 0.044).

CONCLUSIONS

This is the first study to investigate the use of personality scores in the selection of neurosurgical residents. The use of personality assessments has the potential to provide insight into an applicant's future behavior as a resident and beyond. This information may be useful in the selection of neurosurgical residents and can be further used to customize the teaching of residents and for enabling them to recognize their own strengths and weaknesses for self-improvement.

ABBREVIATIONSERAS = Electronic Residency Application Service; HDS = Hogan Development Survey; HPI = Hogan Personality Inventory; LOR = letters of recommendation; MVPI = Motives, Values, Preferences Inventory; SME = subject matter expert; USMLE = US Medical Licensing Examination.

OBJECTIVE

Neurosurgery is among the most competitive residencies, as evidenced by the high number of applicants for relatively few positions. Although it is important to recruit candidates who have the intellectual capacity and drive to succeed, traditional objective selection criteria, such as US Medical Licensing Examination (USMLE) (also known as Step 1) score, number of publications, and class ranking, have not been shown to consistently predict clinical and academic success. Furthermore, these traditional objective parameters have not been associated with specific personality traits.

METHODS

The authors sought to determine the efficacy of a personality assessment in the selection of neurosurgery residents. Specifically, the aim was to determine the correlation between traditional measures used to evaluate an applicant (e.g., USMLE score, number of publications, MD/PhD status) and corresponding validated personality traits.

RESULTS

Fifty-four neurosurgery residency applicants were interviewed at the Cleveland Clinic during the 2014–2015 application cycle. No differences in validated personality scores were identified between the 46 MD applicants and 8 MD/PhD applicants. The mean USMLE score (± SD) was 252.3 ± 11.9, and those in the high-USMLE-score category (USMLE score ≥ 260) had a significantly lower “imaginative” score (a stress measure of eccentric thinking and impatience with those who think more slowly). The average number of publications per applicant was 8.6 ± 7.9, and there was a significant positive correlation (r = 0.339, p = 0.016) between greater number of publications and a higher “adjustment” score (a measure of being even-tempered, having composure under pressure). Significant negative correlations existed between the total number of publications and the “excitable” score (a measure of being emotionally volatile) (r = −0.299, p = 0.035) as well as the “skeptical” score (measure of being sensitive to criticism) (r = −0.325, p = 0.021). The average medical school rank was 25.8, and medical school rankings were positively correlated with the “imaginative” score (r = 0.287, p = 0.044).

CONCLUSIONS

This is the first study to investigate the use of personality scores in the selection of neurosurgical residents. The use of personality assessments has the potential to provide insight into an applicant's future behavior as a resident and beyond. This information may be useful in the selection of neurosurgical residents and can be further used to customize the teaching of residents and for enabling them to recognize their own strengths and weaknesses for self-improvement.

Neurosurgical training is challenging because of the critically ill patient population, the technically demanding operations, and the high-volume work demands placed on residents. Predicting the ultimate success of neurosurgery residency applicants is difficult, because most medical student applicants have never experienced the full extent of the pressure, demands, and unpredictable environment that neurosurgery residents face. Nonetheless, most neurosurgery program directors and faculty seek candidates with particular traits that they believe are necessary for achieving success. A successful resident should be able to perform well under stress, be organized and efficient, empathically communicate at all levels, be technically adept, and be able and willing to make difficult life-or-death decisions.32

Neurosurgery is among the most competitive residencies, as evidenced by the high number of applicants for relatively few positions and the applicants' high average US Medical Licensing Examination (USMLE) (also known as Step 1) scores, number of publications, and grades.12 Both program directors and medical graduates are aware of these criteria, and as such, the applicants work to maximize these factors to increase their chances of matching the neurosurgical program of their choice. Although it is important to recruit candidates who have the intellectual capacity and drive to succeed, the question of whether these traditional selection criteria identify those applicants who are best suited for neurosurgical residency remains. Although the aim of interviewing is to explore candidates' personality traits and fit with a program, it is a largely subjective process.4 Furthermore, only a limited amount can be learned during a brief interview period.

In this pilot study, we sought to determine the effectiveness of the Residency Select tool, which includes the Hogan Personality Assessment, in the selection of neurosurgery residents. Specifically, our aim was to determine the correlation between traditional measures used to evaluate an applicant (e.g., USMLE score, number of publications, MD/PhD status) and corresponding validated personality traits. Through this approach, we sought to determine if relationships exist between traditional Electronic Residency Application Service (ERAS) criteria and personality traits, which we felt would enable us to begin to recognize which personality traits surface within current selection criteria for applicants who are selected for an interview.

Methods

Working in collaboration with J3Personica and using its Residency Select tool, we distributed Hogan Personality assessments to all interviewing applicants. The applicants were asked to complete the assessment no later than 1 week in advance of the interview date so that the assessment could be used in addition to the standard ERAS application. The goal was to use this assessment as an additional tool to improve the selection process by determining (1) the applicant's strengths and possible areas for development and (2) the applicant's “fit” with our program.

Each applicant completed 3 assessments through the J3Personica Residency Select tool, which includes the Hogan Personality Inventory (HPI),21 the Hogan Development Survey (HDS),20 and the Motives, Values, Preferences Inventory (MVPI).17 These 3 inventories measure normal personality traits or usual tendencies, qualities that emerge under stress and that can be disruptive, and the values/drivers that motivate one to behave in certain ways, respectively (Table 1). The following is a brief description of each of these inventories.

TABLE 1.

Hogan assessment descriptions

AssessmentDescription
HPI
  AdjustmentConfidence, self-esteem, and composure under pressure
  AmbitionInitiative, competitiveness, desire for leadership roles
  SociabilityExtraversion, gregariousness, and need for social interaction
  SensitivityTact, perceptiveness, and ability to maintain relationships
  PrudenceSelf-discipline, responsibility, and thoroughness
  InquisitiveCreative and interested in problems
  Learning approachConcerned with building job-related knowledge
HDS
  ExcitementMoody, hard to please, and emotionally volatile
  SkepticalSuspicious, sensitive to criticism, and expecting betrayal
  CautiousRisk averse, resistant to change, and slow to make decisions
  ReservedAloof, uncommunicative, and indifferent to the feelings of others
  LeisurelyOvertly cooperative but privately irritable, stubborn, and uncooperative
  BoldOverly self-confident, arrogant, and entitled
  MischievousCharming, risk-taking, and excitement-seeking
  ColorfulDramatic, attention-seeking, and interruptive
  ImaginativeCreative but thinking and acting in unusual or eccentric ways
  DiligentMeticulous, precise, hard to please, and micromanaging
  DutifulEager to please and reluctant to act independently or against popular opinion
MVPI
  RecognitionResponsive to attention, approval, and praise
  PowerDesiring success, accomplishment, status, and control
  HedonismOriented for fun, pleasure, and enjoyment
  AltruisticWanting to help others and contribute to society
  AffiliationEnjoying and seeking out social interaction
  TraditionDedicated to strong personal beliefs
  SecurityNeeding predictability, structure, and order
  CommerceInterested in money, profits, investment, and business opportunities
  AestheticsNeeding self-expression, concerned with look and design of work products
  ScienceWanting knowledge, research, technology, and data

The HPI, a validated measure of personality, is based on the Five-Factor Model22 and was designed specifically to identify which personality constructs are strongly correlated with job performance. The HPI has 206 true-or-false items with no psychiatric components. It has been normalized with more than 500,000 people and validated in greater than 200 occupations to predict future job performance across these numerous occupations.18 This inventory measures normal personality along the following 7 scales: adjustment, ambition, sociability, interpersonal sensitivity, prudence, inquisitive, and learning approach (Table 1 lists descriptions for each of the scales based on the previously published Hogan manuals17,20,21). Recent studies found that job applicants do not intentionally distort results, and they cannot successfully “fake” their scores.16,28 The HDS has been normalized with 10,000 managers and measures how people behave when under stress/pressure, and it includes the following 11 scales: excitable, skeptical, cautious, reserved, leisurely, bold, mischievous, colorful, imaginative, diligent, and dutiful. These personality traits are difficult to detect from brief interactions/interviews, but they are important to recognize because they can inhibit an individual's job performance and disrupt the workplace. The MVPI measures the core values and interests that motivate the individual; it is used mainly to determine fit with a specific work culture and also enables prediction of success and job satisfaction. The 10 scales used in the MVPI are recognition, power, hedonism, altruistic, affiliation, tradition, security, commerce, aesthetics, and science.

Before initiation of this pilot study with the Cleveland Clinic Neurosurgical Residency Program, the Residency Select tool (which administers the HPI, HDS, and MVPI) was validated in 12 orthopedic residency programs for screening orthopedic residents (JD Zuckerman et al., unpublished data).15 Although these assessments were shown in numerous studies to be highly valid, they had not been used as a tool for surgical residency selection. Therefore, for this validation, a criterion-related validity study was completed. As a part of the criterion-validation study, a job-evaluation process was completed by 174 subject matter experts (SMEs) (including orthopedic surgery chairmen, program directors, and senior faculty). The specific survey completed by the SMEs is called the Job Evaluation Tool, which is a revised version of a long-established job-evaluation process, tailored specifically for this context.14 This survey included 4 sections. One was linked to the HPI and asked questions related to the characteristics that are needed to improve the performance of an orthopedic resident; the second section was linked to the HDS and asked questions related to what would impede or degrade the performance of the orthopedic resident; the third section was linked to the MVPI and asked questions related to job characteristics or values; and the fourth section asked questions specifically designed to identify the competencies necessary for success as an orthopedic resident. Once this initial analysis was completed, a profile that indicated the extent to which an orthopedic resident would need each of the measured personality scales (e.g., 69% adjustment, 36% sociability, 70% interpersonal sensitivity, and 81% prudence) was created. In addition, 266 orthopedic residents completed the Residency Select/Hogan assessments. These residents also had their performance evaluated by several faculty members (range 1–17) on the basis of 38 behavioral competencies (selected by the orthopedic surgery faculty) using a 5-point rating scale. Statistical correlation was calculated between the Residency Select/Hogan personality scales and the 38 competencies in the Job Performance Indicator, as well as between the personality scales and the job-analysis profile. Thus, criterion-related validity was established because the data supported the idea that residents who were a closer fit to the SME-identified profile (predictor variable) were rated to be higher performers (criterion variable). The significant predictors of job performance were adjustment, interpersonal sensitivity, prudence, and learning approach (for the HPI scales); excitable, skeptical, and imaginative (for the HDS); and altruistic and tradition (for the MVPI).

After validation in the orthopedic surgery residency programs, a validation analysis of neurosurgery residency programs, including the Cleveland Clinic program, was performed.13 The analysis compared the characteristics deemed to be important for the neurosurgery and orthopedic surgery programs. Results of the validity-generalization study supported the idea that the profiles for orthopedics and neurosurgery aligned very closely, and there were no significant differences for any of the characteristics. This result implies that the customized assessment tool can be used in the neurosurgery residency population just as it was in the orthopedic residency population. In our study, we used the validated assessment for the Cleveland Clinic neurosurgery program interview applicants (n = 54). An outline of the analysis follows.

Statistical Analysis

For this study, our neurosurgery residency program collected the Residency Select/Hogan personality scores of each applicant who was interviewed for a position and paired these data with the interviewee's respective ERAS application. Applicants were grouped based on USMLE score (high, medium, or low), number of published/accepted peer-reviewed scientific article publications (high, medium, or low), medical school rank (high, medium, or low), and whether the applicant had an MD or MD/PhD degree. Grouping for continuous variables was done before statistical analysis, and an effort was made to have similar numbers of applicants in each category by using lower/upper limits that were clinically/academically relevant. For USMLE scores, the high category included USMLE scores of ≥ 260 (n = 16), the low category included USMLE scores of ≤ 249 (n = 19), and the medium category included all scores between 250 and 260 (n = 19). For publication categories, the high category included those with more than 10 publications (n = 16), the low category included those with fewer than 5 publications (n = 20), and the medium category included those with 5–10 publications (n = 18). Medical school ranks were based on the 2014 US News and World Report medical school research rankings. For this category, a higher number was equivalent to a lower-ranked medical school (i.e., 1 is the highest ranked medical school). The high-rank category included ranks between 1 and 20 (n = 21), the low-rank category was associated with ranks lower than 40 (n = 21), and the medium-rank category included programs ranked between 21 and 40 (n = 12). For all categories, high, medium, and low groupings were made based on the average number, which allowed for an approximately even number of applicants in each group.

Average personality scores were compared between groups by using independent-sample t-tests and by 1-way ANOVA. Furthermore, all validated and nonvalidated personality scores were correlated to the aforementioned ERAS application variables by using Pearson correlation. Any p value of ≤ 0.05 was considered statistically significant.

Results

A total of 54 applicants were interviewed during the 2014–2015 application cycle, including 46 MD applicants and 8 MD/PhD applicants. The mean (± SD) USMLE score was 252.3 ± 11.9. The average number of publications was 8.6 ± 7.9. The average medical school rank (among ranked programs) was 25.8 ± 1.3. Comparisons between the groups on the 9 validated personality scores are presented in Fig. 1A–D. Graphical depictions of the linear correlations for the personality measures are shown in Figs. 2 and 3. No significant differences in personality between the MD and MD/PhD applicants were identified for any of the 9 validated measures (Fig. 1A). Among the criteria that are not validated, there was a significant negative association between security (a measure of needing predictability, structure, and order) and those with an MD/PhD degree (r = −0.341, p = 0.015; Table 2).

FIG. 1.
FIG. 1.

Graphs showing correlations between the 9 validated personality measures and various traditional selection criteria. Statistical significance (p < 0.05) is indicated with asterisks. Figure is available in color online only.

FIG. 2.
FIG. 2.

Significant linear correlations are shown between the total number of publications and personality measures. Correlation coefficients (r) and p values are shown to indicate levels of correlation and statistical significance. Figure is available in color online only.

FIG. 3.
FIG. 3.

Significant linear correlations between USMLE score and skeptical and tradition scores and between medical school rank and imaginative and bold scores. Figure is available in color online only.

TABLE 2.

Significant correlations between personality scores and traditional measures*

AssessmentPearson Correlation (r)p Value
MD vs MD/PhD
  Security−0.3410.015
USMLE score
  Skeptical0.3360.017
  Tradition0.2820.047
No. of publications
  Adjustment0.3390.016
  Excitable−0.2990.035
  Skeptical−0.3250.021
  Leisurely−0.3380.016
  Power−0.9790.008
  Science−0.3270.021
Medical school rank
  Imaginative0.2870.044
  Bold0.3810.006

Italicized words refer to nonvalidated personality measures.

A negative Pearson r value represents a negative association. For the MD vs MD/PhD comparison, the negative correlation indicates that those with an MD/PhD degree have a negative association with the security measure, which indicates that they are less likely to need predictability and structure.

Comparing groups based on USMLE score, we found that those in the low category (≤ 249) had significantly (p = 0.04) lower skeptical scores (a measure of being suspicious, sensitive to criticism) than the rest of the applicants. Comparing the actual USMLE scores revealed a positive correlation between skeptical and increasing scores (r = 0.336, p = 0.017). Those in the high category (USMLE score ≥ 260) had significantly (p = 0.05) lower imaginative scores (a measure of creativity and eccentric thinking) than the other applicants (Fig. 1B). For the nonvalidated criteria, there was a positive correlation of higher USMLE scores with the tradition score, a measure of dedication to strong personal beliefs (r = 0.282, p = 0.047).

The applicants' number of publications was categorized into the low-, medium-, or high-publishing group (< 5, 5–10, or > 10, respectively). Those applicants in the high-publishing group had significantly (p = 0.03) greater adjustment scores (a measure of being consistent and predictable, even-tempered, and displaying composure under pressure) than the applicants in the other 2 groups. When evaluating the implications of the actual number of publications, a significant positive correlation with adjustment (r = 0.339, p = 0.016) was observed. The high-publishing group had significantly (p = 0.05) lower excitable scores (a measure of being moody and emotionally volatile) and significantly (p = 0.03) lower skeptical scores (a measure of being suspicious and sensitive to criticism) than the other 2 groups (Fig. 1C). Significant negative correlations were observed between total publications and excitable (r = −0.299, p = 0.035) and skeptical (r = −0.325, p = 0.021). For the nonvalidated measures, there were negative correlations between total publications and leisurely (a measure of being overtly cooperative but privately irritable, stubborn, and uncooperative; r = −0.338, p = 0.016), power (a measure of desiring success, accomplishment, and status; r = −0.373, p = 0.008), and science (a measure of wanting knowledge, research, and data; r = −0.327, p = 0.021).

For the medical school rank, those in the low-rank group (below 40th) had significantly (p = 0.01) higher imaginative scores (a measure of creative but unusual thinking) than the applicants from the other groups (Fig. 1D). Specifically, as the rank decreased (i.e., from 20 to 30 to 40), there was a positive correlation with imaginative scores (r = 0.287, p = 0.044) and with the nonvalidated bold scores (a measure of arrogance/overestimating competence; r = 0.381, p = 0.006).

There were no significant correlations identified between personality scores and medical school or home geographical location.

Discussion

According to the 2014 National Resident Matching Program survey of neurosurgical program directors,29 the most commonly used selection factors are USMLE score (93%), letters of recommendation (LOR) in the specialty (98%), deans' letters (81%), Alpha Omega Alpha status (81%), and involvement and interest in research (80%). Yet, although these factors have been used historically as selection criteria, it is unclear whether they predict success as a neurosurgical resident.

Al Khalili et al.1 surveyed neurosurgery residency directors and found that the most important factors used in the selection process are interviews, USMLE scores, and LOR. However, based on multivariate analysis, the authors found that the predictors of long-term satisfaction with resident selection were placing less emphasis on LOR and placing greater emphasis on applicant extramural activity. Although this result may be true, a review of the National Resident Matching Program survey of neurosurgery residency directors reveals that using LOR, although among the least valid selection techniques, is the most common factor cited as important for selection, whereas extracurricular activities are among the least cited (only 56% of residency directors cited these activities as important for applicant selection).29

There have been several papers from various medical and surgical specialties that evaluated the use of USMLE scores as predictors of resident success (defined most commonly as good work performance as a resident when assessed by their respective faculty). Although these studies have shown conflicting results,2,5,10,23,27,35,36 the majority of them have found that the USMLE score is not predictive.27 Furthermore, a study in orthopedic residencies found that overall academic success in medical school was not correlated with faculty-rated performance of the individuals during their residency.8 Recent studies have supported the idea that the applicants' personal characteristics and reference letters, which are assessed during the interview process, were the best predictors of clinical performance during residency.5 Overall, this sentiment is repeated in numerous studies, from various surgical specialties, wherein the program directors value interviewing as the most critical process for selecting applicants.1,33,34 With an interview duration of only 4–8 hours in most neurosurgery programs (frequently comprising 15- to 20-minute individual interviews),1 there is likely insufficient time to assess the candidate's fitness with the program or his or her ability to persevere with the stresses of residency or to properly assess any of the other characteristics that are important for a resident to be successful. To underscore the shortcomings of this selection process and the limitations in identifying those who are best suited for the field, a recent study found that there is a 14% attrition rate among neurosurgical residents.25

There is little evidence that an unstandardized and subjective review based on a resume or an interview can predict job performance. For this reason, the corporate world frequently uses personality assessments for selection and development that have been validated to predict occupational satisfaction, job performance, and ability to work in teams.9,30 Organizations also use personality measures to recognize which employees may be more likely to engage in counterproductive behaviors.3 Furthermore, cognitive and work sample measures tend to discriminate in terms of race and ethnicity, whereas personality measures decrease the chance of such bias.7,11

Recent studies have investigated the role of personality tests in supplementing interviews in the residency-selection process. Quintero et al.31 had applicants to their orthopedic surgery program take Myers-Briggs personality instruments before their interview. The authors found that clinicians ranked candidates higher when they shared personality traits. This result further highlights the subjectivity and bias associated with interviews, but it also suggests the potential utility of personality tests in the selection of residents. Spiotta et al.32 administered Myers-Briggs personality instruments to neurosurgical residents throughout the United States. Of the 108 residents who completed the survey, the most common personality types were scientist, executive, and duty fulfiller. These results have value in the selection of residents and in helping newly starting residents identify their strengths and weaknesses to help them improve their performance.

This study helped detect which personality traits are being identified when using the standard evaluation criteria. MD/PhD candidates had no significant personality differences (for the validated criteria) compared with the MD candidates. The only significant difference identified was the negative association with the security motivator (Table 2), which suggests that MD/PhD candidates more commonly prefer faster-paced environments in which they quickly adapt to change. They are more likely to be stifled by environments that are overly structured. Although the presence of the advanced degree has been shown not to correlate with an increased likelihood of staying in academia,24 it is possible that their training or motivators can make them stronger residents because of their adaptability or lesser need for security.

Those in the low-rank medical school group (below 40th) had high imaginative scores, whereas those in the high-USMLE groups had low imaginative scores. The imaginative trait is part of the HDS, which is a measure of behavioral tendencies under stress. Those who score high in imaginative tend to share ideas before spending sufficient time thinking through their practicality and can have little patience for individuals who have difficulty thinking quickly. It is interesting to note the trends. As the rank of the school increased (lower number, e.g., 1) and as the USMLE scores increased, the validated imaginative stress-trait scores decreased, which suggests that those applicants may have more patience with those who think more slowly or have less eccentric thinking under stress. In contrast, the trend was reversed for the skeptical trait, another significant validated personality stress trait. Under stress, those with high skeptical scores tend to be more cynical or mistrustful and give others the impression that they do not fully trust them. For this trait, there was a positive association between increasing USMLE scores and high skeptical scores. Although some studies have shown that greater USMLE scores predict only greater scores on future standardized tests,10 the association seen in our study suggests possible personality traits among those with very high scores.

Finally, those in the high-publishing group had significant associations with 3 validated personality traits, which shows a positive correlation with adjustment and negative correlations with excitable and skeptical. Adjustment is one of the scales associated with the HPI, which evaluates a person's normal behavior or usual tendencies. Those individuals who are high in adjustment tend to be calm, consistent, and predictable and are able to handle pressure and stressful conditions. Excitable is an HDS scale and, under stress, individuals who are high in excitable vacillate quickly between optimism and pessimism and can seem moody and unpredictable. The combination of these 3 personality traits among those with a high number of publications is logical and shows that they are able to handle stress and work well with others, even under stress. Other studies developed varying conclusions regarding the impact of publications on postresidency choice of academic career. Lawton et al.24 and McClelland26 found no association between preresidency publications and choice of academic career. Crowley et al.6 found that in-training publications were associated with choice of academic career and becoming a professor or chair/chief. The results of our study are consistent with these previous findings. Although not a validated measure, a higher number of publications was negatively correlated with science scores. Those with high science scores derive motivation from conducting research, analyzing information, and uncovering underlying causes, whereas those with lower scores tend to prefer to rely on intuition, leaving analysis to others.

Accordingly, although number of publications has been used historically by programs to help identify candidates who are more likely to be academically oriented, with more programs using personality assessments in their selection process, number of publications may be discovered to be a better indicator of a candidate's ability to work with people and handle stress rather than his or her particular scientific curiosity.

This study is the first such pilot investigation to use personality tests in the evaluation of neurosurgery residency applicants. The results provide insight into what personality traits we are selecting for when using traditional selection measures found in the ERAS. They also help elucidate what traits may be overlooked. Ultimately, however, this study raises questions that will need to be answered in the coming years by organized neurosurgery. For example, what personality traits are important for success in neurosurgical residency? The Job Evaluation Tool, which surveyed chairman and program directors and correlated the personality scores to assessments of current resident performance, may begin to answer that question. Different programs, however, will prioritize different values; for some, work ethic may be the most important value, whereas for others, it will be scientific curiosity. Will including the personality assessments in the selection criteria lead to more resident satisfaction with the current program, enable residents to be more reflective in improving their own weaknesses, and/or lead to fewer disciplinary issues? In addition, which measures will correlate with long-term academic productivity? Only longer-term studies with larger cohorts will be able to answer those questions. This was the first pilot study in the field, and we examined a single institution's experience with its interviewees. Future studies should be able to compare the traits of all applicants with those who are selected for interview and should be more suited for determining whether a program can use personality tests to select interview candidates.

There were a number of limitations in this study that should be considered. The study included only the data from a single institution and from a single application cycle. Accordingly, there was likely an element of selection bias. The sample size is also relatively small, because only those applicants selected for an interview were asked to take the personality assessments. Future larger studies with all neurosurgery applicants are needed to validate these findings. Last, the Hogan assessment is a self-report measure, which introduces validity concerns such as response bias. For example, social desirability bias occurs when the participant responds in a way that he or she perceives as favorable to the experimenter. However, as discussed earlier, recent research has supported the idea that job applicants do not intentionally distort results and that they cannot successfully fake their scores.16,28 In addition, Hogan personality assessments specifically measure a person's reputation, which is public and based on observable behaviors and therefore can be measured reliably.19

Conclusions

Personality assessments reflect how a person presents himself or herself to others in a job setting. The Hogan personality assessments have been validated to successfully predict a person's future behavior. However, it is important to recognize that there is no “best” personality profile for a career in neurosurgery. There are variations among faculty and institutions. Furthermore, the relative strengths and weaknesses and personality characteristics of each resident, if known, can be used to the benefit of the resident as a coaching tool by enhancing self-awareness and understanding stress tendencies. This information can be used to customize the methodology by which a given resident is taught and may help the resident improve in various aspects of patient care and interpersonal interaction. Ultimately, such data could provide additional meaningful and objective information that may help in the selection and development of neurosurgical residents. This is the first study to investigate the use of personality scores in the selection of neurosurgical residents. Future studies with larger sample sizes can be used longitudinally to investigate the impact of using personality tests in the selection of neurosurgical applicants, their respective success as residents, and their overall career trajectory.

References

  • 1

    Al Khalili KChalouhi NTjoumakaris SGonzalez LFStarke RMRosenwasser R: Programs selection criteria for neurological surgery applicants in the United States: a national survey for neurological surgery program directors. World Neurosurg 81:473477477.e1477.e22014

    • Search Google Scholar
    • Export Citation
  • 2

    Alterman DMJones TMHeidel REDaley BJGoldman MH: The predictive value of general surgery application data for future resident performance. J Surg Educ 68:5135182011

    • Search Google Scholar
    • Export Citation
  • 3

    Berry CMOnes DSSackett PR: Interpersonal deviance, organizational deviance, and their common correlates: a review and meta-analysis. J Appl Psychol 92:4104242007

    • Search Google Scholar
    • Export Citation
  • 4

    Blouin D: Reliability of a structured interview for admission to an emergency medicine residency program. Teach Learn Med 22:2462502010

    • Search Google Scholar
    • Export Citation
  • 5

    Brothers TEWetherholt S: Importance of the faculty interview during the resident application process. J Surg Educ 64:3783852007

  • 6

    Crowley RWAsthagiri ARStarke RMZusman EEChiocca EALonser RR: In-training factors predictive of choosing and sustaining a productive academic career path in neurological surgery. Neurosurgery 70:102410322012

    • Search Google Scholar
    • Export Citation
  • 7

    Dean MARoth PLBobko P: Ethnic and gender subgroup differences in assessment center ratings: a meta-analysis. J Appl Psychol 93:6856912008

    • Search Google Scholar
    • Export Citation
  • 8

    Dirschl DRCampion ERGilliam K: Resident selection and predictors of performance: can we be evidence based?. Clin Orthop Relat Res 449:44492006

    • Search Google Scholar
    • Export Citation
  • 9

    Dudley NMOrvis KALebiecki JECortina JM: A meta-analytic investigation of conscientiousness in the prediction of job performance: examining the intercorrelations and the incremental validity of narrow traits. J Appl Psychol 91:40572006

    • Search Google Scholar
    • Export Citation
  • 10

    Egol KACollins JZuckerman JD: Success in orthopaedic training: resident selection and predictors of quality performance. J Am Acad Orthop Surg 19:72802011

    • Search Google Scholar
    • Export Citation
  • 11

    Foldes HJDuehr EEOnes DS: Group differences in personality: meta-analyses comparing five U.S. racial groups. Person Psychol 61:5796162008

    • Search Google Scholar
    • Export Citation
  • 12

    Green MJones PThomas JX Jr: Selection criteria for residency: results of a national program directors survey. Acad Med 84:3623672009

    • Search Google Scholar
    • Export Citation
  • 13

    Hogan Assessment Systems: Job Analysis of the Hogan Personality Inventory Hogan Development Survey and Motives Values Preferences Inventory for Selecting Neurological Surgery Residents Eatontown, NJJ3Personica/Hogan Assessment Systems2014

    • Search Google Scholar
    • Export Citation
  • 14

    Hogan Assessment Systems: Job Evaluation Tool Manual TulsaHogan Assessment Systems2000

  • 15

    Hogan Assessment Systems: Technical Report: Job Analysis Validity Generalization and Criterion-Related Validation of the Hogan Personality Inventory Hogan Development Survey and the Motives Values Preferences Inventory for Selecting Candidates for Orthopaedic Resident Programs Eatontown, NJJ3Personica/Hogan Assessment Systems2014

    • Search Google Scholar
    • Export Citation
  • 16

    Hogan JBarrett PHogan R: Personality measurement, faking, and employment selection. J Appl Psychol 92:127012852007

  • 17

    Hogan JHogan R: Motives Values Preferences Inventory Manual ed 2TulsaHogan Assessment Systems2010

  • 18

    Hogan JHolland B: Using theory to evaluate personality and job-performance relations: a socioanalytic perspective. J Appl Psychol 88:1001122003

    • Search Google Scholar
    • Export Citation
  • 19

    Hogan R: In defense of personality measurement: new wine for old whiners. Hum Perform 18:3313412005

  • 20

    Hogan RHogan J: Hogan Development Survey Manual TulsaHogan Assessment Systems2009

  • 21

    Hogan RHogan J: Hogan Personality Inventory Manual ed 3TulsaHogan Assessment Systems2007

  • 22

    Hurtz GMDonovan JJ: Personality and job performance: the Big Five revisited. J Appl Psychol 85:8698792000

  • 23

    Janis JEHatef DA: Resident selection protocols in plastic surgery: a national survey of plastic surgery program directors. Plast Reconstr Surg 122:192919412008

    • Search Google Scholar
    • Export Citation
  • 24

    Lawton MTNarvid JQuiñones-Hinojosa A: Predictors of neurosurgical career choice among residents and residency applicants. Neurosurgery 60:9349392007

    • Search Google Scholar
    • Export Citation
  • 25

    Lynch GNieto KPuthenveettil SReyes MJureller MHuang JH: Attrition rates in neurosurgery residency: analysis of 1361 consecutive residents matched from 1990 to 1999. J Neurosurg 122:2402492015

    • Search Google Scholar
    • Export Citation
  • 26

    McClelland S III: Pre-residency peer-reviewed publications are associated with neurosurgery resident choice of academic compared to private practice careers. J Clin Neurosci 17:2872892010

    • Search Google Scholar
    • Export Citation
  • 27

    McGaghie WCCohen ERWayne DB: Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions?. Acad Med 86:48522011

    • Search Google Scholar
    • Export Citation
  • 28

    McGrath REMitchell MKim BHHough L: Evidence for response bias as a source of error variance in applied assessment. Psychol Bull 136:4504702010

    • Search Google Scholar
    • Export Citation
  • 29

    National Resident Matching Program Data Release and Research Committee: Results of the 2014 NRMP Program Director Survey Washington, DCNational Resident Matching Program2014

    • Search Google Scholar
    • Export Citation
  • 30

    Peeters MAGvan Tuijl HFJMRutte CGReymen IMMJ: Personality and team performance: a meta-analysis. Eur J Pers 20:3773962006

  • 31

    Quintero AJSegal LSKing TSBlack KP: The personal interview: assessing the potential for personality similarity to bias the selection of orthopaedic residents. Acad Med 84:136413722009

    • Search Google Scholar
    • Export Citation
  • 32

    Spiotta AMullin JWeil RJSchlenk RGrady MSBenzel EC: Insights into the contemporary neurosurgical trainee: personality typing of residents across the United States. JSM Neurosurg Spine 2:10222014

    • Search Google Scholar
    • Export Citation
  • 33

    Wagoner NEGray GT: Report on a survey of program directors regarding selection factors in graduate medical education. J Med Educ 54:4454521979

    • Search Google Scholar
    • Export Citation
  • 34

    Wagoner NESuriano JRStoner JA: Factors used by program directors to select residents. J Med Educ 61:10211986

  • 35

    Wood PSSmith WLAltmaier EMTarico VSFranken EA Jr: A prospective study of cognitive and noncognitive selection criteria as predictors of resident performance. Invest Radiol 25:8558591990

    • Search Google Scholar
    • Export Citation
  • 36

    Yindra KJRosenfeld PSDonnelly MB: Medical school achievements as predictors of residency performance. J Med Educ 63:3563631988

Disclosures

Mr. Friedman has ownership in J3Personica, and Ms. Ferraris has served as a consultant for J3Personica. No grants or technical or corporate financial support was received for conducting this study or writing this manuscript. J3Personica collected and provided the data from its Residency Select tool (Hogan Personality Assessments). The analysis and writing of this manuscript were performed independently.

Author Contributions

Conception and design: Schlenk, Lubelski, Friedman, Benzel. Acquisition of data: Schlenk, Lubelski, Friedman, Ferraris. Analysis and interpretation of data: all authors. Drafting the article: Lubelski. Critically revising the article: all authors. Reviewed submitted version of manuscript: all authors. Statistical analysis: Lubelski.

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

INCLUDE WHEN CITING Published online February 5, 2016; DOI: 10.3171/2015.7.JNS15880.

Correspondence Richard Schlenk, Center for Spine Health, Department of Neurological Surgery, Cleveland Clinic, 9500 Euclid Ave., S-40, Cleveland, OH 44195. email: schlenr@ccf.org.

© AANS, except where prohibited by US copyright law.

Headings

Figures

  • View in gallery

    Graphs showing correlations between the 9 validated personality measures and various traditional selection criteria. Statistical significance (p < 0.05) is indicated with asterisks. Figure is available in color online only.

  • View in gallery

    Significant linear correlations are shown between the total number of publications and personality measures. Correlation coefficients (r) and p values are shown to indicate levels of correlation and statistical significance. Figure is available in color online only.

  • View in gallery

    Significant linear correlations between USMLE score and skeptical and tradition scores and between medical school rank and imaginative and bold scores. Figure is available in color online only.

References

  • 1

    Al Khalili KChalouhi NTjoumakaris SGonzalez LFStarke RMRosenwasser R: Programs selection criteria for neurological surgery applicants in the United States: a national survey for neurological surgery program directors. World Neurosurg 81:473477477.e1477.e22014

    • Search Google Scholar
    • Export Citation
  • 2

    Alterman DMJones TMHeidel REDaley BJGoldman MH: The predictive value of general surgery application data for future resident performance. J Surg Educ 68:5135182011

    • Search Google Scholar
    • Export Citation
  • 3

    Berry CMOnes DSSackett PR: Interpersonal deviance, organizational deviance, and their common correlates: a review and meta-analysis. J Appl Psychol 92:4104242007

    • Search Google Scholar
    • Export Citation
  • 4

    Blouin D: Reliability of a structured interview for admission to an emergency medicine residency program. Teach Learn Med 22:2462502010

    • Search Google Scholar
    • Export Citation
  • 5

    Brothers TEWetherholt S: Importance of the faculty interview during the resident application process. J Surg Educ 64:3783852007

  • 6

    Crowley RWAsthagiri ARStarke RMZusman EEChiocca EALonser RR: In-training factors predictive of choosing and sustaining a productive academic career path in neurological surgery. Neurosurgery 70:102410322012

    • Search Google Scholar
    • Export Citation
  • 7

    Dean MARoth PLBobko P: Ethnic and gender subgroup differences in assessment center ratings: a meta-analysis. J Appl Psychol 93:6856912008

    • Search Google Scholar
    • Export Citation
  • 8

    Dirschl DRCampion ERGilliam K: Resident selection and predictors of performance: can we be evidence based?. Clin Orthop Relat Res 449:44492006

    • Search Google Scholar
    • Export Citation
  • 9

    Dudley NMOrvis KALebiecki JECortina JM: A meta-analytic investigation of conscientiousness in the prediction of job performance: examining the intercorrelations and the incremental validity of narrow traits. J Appl Psychol 91:40572006

    • Search Google Scholar
    • Export Citation
  • 10

    Egol KACollins JZuckerman JD: Success in orthopaedic training: resident selection and predictors of quality performance. J Am Acad Orthop Surg 19:72802011

    • Search Google Scholar
    • Export Citation
  • 11

    Foldes HJDuehr EEOnes DS: Group differences in personality: meta-analyses comparing five U.S. racial groups. Person Psychol 61:5796162008

    • Search Google Scholar
    • Export Citation
  • 12

    Green MJones PThomas JX Jr: Selection criteria for residency: results of a national program directors survey. Acad Med 84:3623672009

    • Search Google Scholar
    • Export Citation
  • 13

    Hogan Assessment Systems: Job Analysis of the Hogan Personality Inventory Hogan Development Survey and Motives Values Preferences Inventory for Selecting Neurological Surgery Residents Eatontown, NJJ3Personica/Hogan Assessment Systems2014

    • Search Google Scholar
    • Export Citation
  • 14

    Hogan Assessment Systems: Job Evaluation Tool Manual TulsaHogan Assessment Systems2000

  • 15

    Hogan Assessment Systems: Technical Report: Job Analysis Validity Generalization and Criterion-Related Validation of the Hogan Personality Inventory Hogan Development Survey and the Motives Values Preferences Inventory for Selecting Candidates for Orthopaedic Resident Programs Eatontown, NJJ3Personica/Hogan Assessment Systems2014

    • Search Google Scholar
    • Export Citation
  • 16

    Hogan JBarrett PHogan R: Personality measurement, faking, and employment selection. J Appl Psychol 92:127012852007

  • 17

    Hogan JHogan R: Motives Values Preferences Inventory Manual ed 2TulsaHogan Assessment Systems2010

  • 18

    Hogan JHolland B: Using theory to evaluate personality and job-performance relations: a socioanalytic perspective. J Appl Psychol 88:1001122003

    • Search Google Scholar
    • Export Citation
  • 19

    Hogan R: In defense of personality measurement: new wine for old whiners. Hum Perform 18:3313412005

  • 20

    Hogan RHogan J: Hogan Development Survey Manual TulsaHogan Assessment Systems2009

  • 21

    Hogan RHogan J: Hogan Personality Inventory Manual ed 3TulsaHogan Assessment Systems2007

  • 22

    Hurtz GMDonovan JJ: Personality and job performance: the Big Five revisited. J Appl Psychol 85:8698792000

  • 23

    Janis JEHatef DA: Resident selection protocols in plastic surgery: a national survey of plastic surgery program directors. Plast Reconstr Surg 122:192919412008

    • Search Google Scholar
    • Export Citation
  • 24

    Lawton MTNarvid JQuiñones-Hinojosa A: Predictors of neurosurgical career choice among residents and residency applicants. Neurosurgery 60:9349392007

    • Search Google Scholar
    • Export Citation
  • 25

    Lynch GNieto KPuthenveettil SReyes MJureller MHuang JH: Attrition rates in neurosurgery residency: analysis of 1361 consecutive residents matched from 1990 to 1999. J Neurosurg 122:2402492015

    • Search Google Scholar
    • Export Citation
  • 26

    McClelland S III: Pre-residency peer-reviewed publications are associated with neurosurgery resident choice of academic compared to private practice careers. J Clin Neurosci 17:2872892010

    • Search Google Scholar
    • Export Citation
  • 27

    McGaghie WCCohen ERWayne DB: Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions?. Acad Med 86:48522011

    • Search Google Scholar
    • Export Citation
  • 28

    McGrath REMitchell MKim BHHough L: Evidence for response bias as a source of error variance in applied assessment. Psychol Bull 136:4504702010

    • Search Google Scholar
    • Export Citation
  • 29

    National Resident Matching Program Data Release and Research Committee: Results of the 2014 NRMP Program Director Survey Washington, DCNational Resident Matching Program2014

    • Search Google Scholar
    • Export Citation
  • 30

    Peeters MAGvan Tuijl HFJMRutte CGReymen IMMJ: Personality and team performance: a meta-analysis. Eur J Pers 20:3773962006

  • 31

    Quintero AJSegal LSKing TSBlack KP: The personal interview: assessing the potential for personality similarity to bias the selection of orthopaedic residents. Acad Med 84:136413722009

    • Search Google Scholar
    • Export Citation
  • 32

    Spiotta AMullin JWeil RJSchlenk RGrady MSBenzel EC: Insights into the contemporary neurosurgical trainee: personality typing of residents across the United States. JSM Neurosurg Spine 2:10222014

    • Search Google Scholar
    • Export Citation
  • 33

    Wagoner NEGray GT: Report on a survey of program directors regarding selection factors in graduate medical education. J Med Educ 54:4454521979

    • Search Google Scholar
    • Export Citation
  • 34

    Wagoner NESuriano JRStoner JA: Factors used by program directors to select residents. J Med Educ 61:10211986

  • 35

    Wood PSSmith WLAltmaier EMTarico VSFranken EA Jr: A prospective study of cognitive and noncognitive selection criteria as predictors of resident performance. Invest Radiol 25:8558591990

    • Search Google Scholar
    • Export Citation
  • 36

    Yindra KJRosenfeld PSDonnelly MB: Medical school achievements as predictors of residency performance. J Med Educ 63:3563631988

TrendMD

Cited By

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 659 659 29
PDF Downloads 283 283 21
EPUB Downloads 0 0 0

PubMed

Google Scholar