Estimating prognosis for traumatic brain injury patients in a low-resource setting: how do providers compare to the CRASH risk calculator?

View More View Less
  • 1 Duke University Division of Global Neurosurgery and Neurology;
  • 2 Duke Global Health Institute;
  • 3 Department of Neurosurgery, Duke University Hospital;
  • 4 Duke University School of Medicine, Durham, North Carolina;
  • 5 Mbarara University of Science and Technology, Mbarara Regional Referral Hospital, Mbarara, Uganda;
  • 6 Paul L. Foster School of Medicine, El Paso, Texas; and
  • 7 Duke Surgery, Division of Emergency Medicine, Durham, North Carolina
Free access

OBJECTIVE

Traumatic brain injury (TBI), a burgeoning global health concern, is one condition that could benefit from prognostic modeling. Risk stratification of TBI patients on presentation to a health facility can support the prudent use of limited resources. The CRASH (Corticosteroid Randomisation After Significant Head Injury) model is a well-established prognostic model developed to augment complex decision-making. The authors’ current study objective was to better understand in-hospital decision-making for TBI patients and determine whether data from the CRASH risk calculator influenced provider assessment of prognosis.

METHODS

The authors performed a choice experiment using a simulated TBI case. All participant doctors received the same case, which included a patient history, vitals, and physical examination findings. Half the participants also received the CRASH risk score. Participants were asked to estimate the patient prognosis and decide the best next treatment step. The authors recruited a convenience sample of 28 doctors involved in TBI care at both a regional and a national referral hospital in Uganda.

RESULTS

For the simulated case, the CRASH risk scores for 14-day mortality and an unfavorable outcome at 6 months were 51.4% (95% CI 42.8%, 59.8%) and 89.8% (95% CI 86.0%, 92.6%), respectively. Overall, participants were overoptimistic when estimating the patient prognosis. Risk estimates by doctors provided with the CRASH risk score were closer to that score than estimates made by doctors in the control group; this effect was more pronounced for inexperienced doctors. Surgery was selected as the best next step by 86% of respondents.

CONCLUSIONS

This study was a novel assessment of a TBI prognostic model’s influence on provider estimation of risk in a low-resource setting. Exposure to CRASH risk score data reduced overoptimistic prognostication by doctors, particularly among inexperienced providers.

ABBREVIATIONS HIC = high-income country; LMICs = low- and middle-income countries; LST = life-sustaining treatment; MNRH = Mulago National Referral Hospital; MRRH = Mbarara Regional Referral Hospital; TBI = traumatic brain injury.

OBJECTIVE

Traumatic brain injury (TBI), a burgeoning global health concern, is one condition that could benefit from prognostic modeling. Risk stratification of TBI patients on presentation to a health facility can support the prudent use of limited resources. The CRASH (Corticosteroid Randomisation After Significant Head Injury) model is a well-established prognostic model developed to augment complex decision-making. The authors’ current study objective was to better understand in-hospital decision-making for TBI patients and determine whether data from the CRASH risk calculator influenced provider assessment of prognosis.

METHODS

The authors performed a choice experiment using a simulated TBI case. All participant doctors received the same case, which included a patient history, vitals, and physical examination findings. Half the participants also received the CRASH risk score. Participants were asked to estimate the patient prognosis and decide the best next treatment step. The authors recruited a convenience sample of 28 doctors involved in TBI care at both a regional and a national referral hospital in Uganda.

RESULTS

For the simulated case, the CRASH risk scores for 14-day mortality and an unfavorable outcome at 6 months were 51.4% (95% CI 42.8%, 59.8%) and 89.8% (95% CI 86.0%, 92.6%), respectively. Overall, participants were overoptimistic when estimating the patient prognosis. Risk estimates by doctors provided with the CRASH risk score were closer to that score than estimates made by doctors in the control group; this effect was more pronounced for inexperienced doctors. Surgery was selected as the best next step by 86% of respondents.

CONCLUSIONS

This study was a novel assessment of a TBI prognostic model’s influence on provider estimation of risk in a low-resource setting. Exposure to CRASH risk score data reduced overoptimistic prognostication by doctors, particularly among inexperienced providers.

ABBREVIATIONS HIC = high-income country; LMICs = low- and middle-income countries; LST = life-sustaining treatment; MNRH = Mulago National Referral Hospital; MRRH = Mbarara Regional Referral Hospital; TBI = traumatic brain injury.

In Brief

The authors performed a choice experiment using a simulated traumatic brain injury (TBI) case and tested how healthcare providers respond to patient management questions when given output from a risk calculator. This study is important because it offers insight into the utility of risk calculators in the management of TBI in low- and middle-income countries.

Traumatic brain injury (TBI) is an intricate pathology with regard to etiology, mechanism, and severity.1,2 This heterogeneity poses a particular challenge to healthcare providers, who must simultaneously predict which adverse events their patients may suffer and the risk associated with each complication. To estimate risk, providers in high-income countries (HICs) rely on CT scanners, intracranial pressure monitoring, electroencephalography, and other key prognostic technologies.3 Such technologies are often limited or unavailable in low- and middle-income countries (LMICs), which suffer a disproportionate burden of TBI, exacerbating the difficulty inherent to making key treatment decisions.4,5

Healthcare providers in low-resource settings currently estimate risk using patient data, the literature, experience, and intuition.5,6 While these methods can be accurate, they are inherently subjective, prone to human error and biases, and subject to external pressures. To mitigate the limitations of human decision-making and provide objective data to support decisions, researchers have developed TBI risk calculators. These calculators, developed from TBI prognostic models, combine readily available data using an algorithm trained on previous patient encounters to estimate a patient’s risk.7 Therefore, accurate assessment of risk using a TBI risk calculator has great potential to augment decision-making, particularly in resource-poor settings.

However accurate a prognostic model may be, its utility is ultimately determined by its generalizability and ability to produce a change in behavior.3 An example of a robust TBI risk calculator constructed using data from high- and low-resource settings is the CRASH (Corticosteroid Randomisation After Significant Head Injury) risk calculator.8 Although hundreds of TBI prognostic models and risk calculators such as CRASH,9 as well as other support tools for surgical specialties, have been available for decades, few studies have evaluated their impact on decision-making.10,11 In 1993 a study on severe TBI patients in Great Britain showed that providers were more likely to increase resources administered to patients expected to have favorable outcomes and reduce resources for patients expected to have poor outcomes as estimated by a decision support tool.10 A 2016 randomized controlled trial asked providers to estimate the risk for general surgery pathologies in hypothetical patients and found that providers given a risk calculator score approximated risk more closely to the risk calculator and had less interrater variability than providers who did not receive the risk calculator output.11 One study of 138 neurosurgical attendings, fellows, and residents attending the 2017 Annual Meeting of the Congress of Neurological Surgeons asked providers to predict 30-day survival and functional recovery at 6 months and recommend craniotomy, nonsurgical management, or comfort care for two hypothetical patients with severe TBI and subdural hematoma; participants provided with the CRASH risk calculator estimates were significantly more likely to recommend nonsurgical treatment (T. Williamson et al., unpublished data). To our knowledge, no study in a low-resource setting has investigated whether a TBI risk calculator influences a provider’s assessment of patient risk.

To investigate the impact of prognostic modeling—presented here as a risk calculator—on provider assessment of risk, we conducted an experimental study at one national and one regional referral hospital in Uganda. We hypothesized that providers presented with the risk calculator output would approximate patient risk closer to the risk calculator estimate than the providers without such information. We also administered a survey to understand other factors influencing decision-making.

Methods

Our study received ethical clearance from the institutional review boards of Makerere University, Kampala, Uganda, and Duke University, Durham, North Carolina. We additionally received hospital clearance from Mbarara Regional Referral Hospital (MRRH) and Mulago National Referral Hospital (MNRH).

Clinical Setting

The study took place at the two largest hospitals in Uganda, that is, MRRH and MNRH. The MRRH is a 360-bed referral hospital located 250 km southwest of the capital city, Kampala. The annual volume of TBI patients is approximately 600 and includes patients from neighboring Rwanda, Democratic Republic of the Congo, and Tanzania. There is one doctor trained in neurosurgery and no neurosurgery residents; instead, third-year general surgery residents and interns complete rotations in the neurosurgery department. The casualty department (emergency department), the location of first triage for TBI patients, is staffed by general surgery residents, emergency medicine residents, and interns.

The MNRH is a 1500-bed national referral hospital located in the capital city of Kampala. The hospital treats about 1500 TBI cases annually through the casualty and neurosurgery departments, with patients from throughout Uganda and neighboring East African countries. The neurosurgery department, staffed by 6 residents, 5 consultants, and 2 neurosurgery-trained providers in the casualty department, performs all the neurosurgical procedures at MNRH.

Study Population

There are only 12 residency-trained neurosurgeons in Uganda. However, providers across the healthcare system, including those in multiple levels of training and other specialties, are closely involved in the care of TBI patients. To capture the gamut of providers and increase sample size, we included doctors working in emergency medicine and general surgery. A provider must have completed a rotation in either the neurosurgery department or the casualty department to be included. The levels of training comprised interns, residents, and consultants. An intern doctor is in his or her first year post–medical school graduation. Residents are 3–5 years removed from medical school. Consultants are the equivalent to attending physicians. We divided the study population into experienced and inexperienced doctors, defining “experienced” doctors as those in residency or who had completed residency training and “inexperienced” doctors as those who had not finished their intern year.

Participant Recruitment

We used purposive and snowball sampling to recruit doctors for the study. Participants provided written informed consent and were compensated for their time.

Participant Demographics and Background Characteristics

Collected background information included provider sex, age, medical specialty, level of experience, years of experience (calculated by subtracting year of medical school graduation from 2018), and location of the experiment. As previous studies have demonstrated the impact of providers’ religious values on end-of-life decision-making,4,12,13 and given that over 99% of the Ugandan population endorses religious affiliation,2 the importance of each provider’s religious faith in life was also included.

Study Design

We created one detailed clinical vignette of a TBI patient (Supplemental Fig. 1). A panel of emergency medicine doctors and neurosurgeons from the United States with experience providing care in LMICs created the vignette to represent a realistic patient case. The hypothetical case was a patient with severe TBI with an estimated risk of mortality of 51.4% per the CRASH calculator. We chose this level of severity to provide near-equal ranges to over- or underestimate patient risk. In addition to the vignette, we administered a separate survey to capture participant background information and other considerations for decision-making. We did not collect identifying participant information to preserve anonymity. The primary author (C.E.) administered all surveys in person between October and December 2018.

Randomization

We randomly assigned participants to one of two groups, the clinical vignette alone (control group) or the clinical vignette plus the output from the CRASH risk calculator (experimental group). We chose the CRASH8 risk calculator because it incorporates data from LMICs and estimates acute outcomes, whereas IMPACT (International Mission for Prognosis and Analysis of Clinical Trials),14 another well-known TBI prognostic model, does not. For the experimental group, we entered the patient information from the vignette into the CRASH risk calculator prior to the experiment. The risk calculator estimated the patient risk for mortality at 14 days (i.e., acute outcome) and for an unfavorable outcome (death, vegetative state, or severe disability) at 6 months (i.e., chronic outcomes) after surgery for hematoma evacuation. We included the output from the calculator in the vignette and provided a brief written introduction to risk calculators. Other than the risk output and the description of risk calculators, the vignettes were identical between the two groups. A co-investigator (S.W.) randomized the order of clinical vignettes with or without the risk score using a computer program and placed them in envelopes. We kept the survey administrator (C.E.) blinded from the randomization process.

Prognostication Exercise and Follow-Up Survey

For the clinical vignette, we asked providers what was the best next step for the patient: surgery, admittance to the intensive care unit, initiation of comfort measures, or observation. We then asked providers to estimate how many out of 100 patients similar to the vignette case would survive the hospitalization, survive to 6 months, survive to 30 days, be able to perform activities of daily living within the next 6 months, and remain on a ventilator were they to undergo surgery. We subtracted these numbers from 100 to calculate the providers’ estimated risk for each outcome.

To understand the influence of resources on the participants’ decision-making, we administered a follow-up survey in which we asked providers what percentage of patients with an identical presentation would undergo CT within 2 days of admission and what percentage would undergo surgery within 2 days of admission if needed, considering cost and availability of resources. The time point of 2 days was chosen because the time from a patient’s hospital arrival to CT is often up to 24 hours and the time from hospital arrival to surgery is often 4 days or more;15 therefore, 2 days represents an intermediate time. Finally, we asked if patient prognosis, resources available, or both influenced their decisions.

Clinical Decision-Making Survey

We administered a survey to assess providers’ decision-making process in the care of patients with severe TBI. The survey asked providers to select whether they strongly agreed, agreed, were neutral, disagreed, or strongly disagreed with the following statements: “I have recommended surgery in patients that I should not have,” “I have not recommended surgery in patients that I should have,” “I am responsible if a patient dies within 30 days of any surgery I recommend,” “I am willing to not offer surgery even if it means a patient will die,” “I am obligated to offer surgery if there is a chance the patient will survive,” “I worry about incorrectly estimating prognosis,” “I use available literature and data to make a prognosis,” “I use prior experience with patients to make a prognosis,” and “I feel that there are adequate quality data upon which to base prognostication.”

Statistical Analysis

We used chi-square tests to compare demographic and medical training background between the experimental and control groups to determine if our randomization was effective. We used unpaired t-tests to compare the risk estimates between the two provider groups. We performed an additional exploratory analysis in which we repeated the t-tests after dividing the study population into experienced and inexperienced providers. Data were summarized with descriptive statistics or graphically illustrated as appropriate using the ggplot2 and Likert packages in R 3.5.3 (R Foundation for Statistical Computing). A p value less than 0.05 was considered statistically significant.

Results

Between the MNRH and MRRH locations, we enrolled 28 doctors to participate in this study (Table 1). Of these, 14 received the patient vignette with the CRASH risk calculator output; 14 received the patient vignette alone. Two interns in the control group were excluded from the prognostication exercise: one did not understand the exercise, and one was uncomfortable with the exercise (Fig. 1). One intern in the control group did not complete the clinical decision-making survey. The majority of doctors were male (21 [75%]), residents (13 [46%]), and working at MRRH (19 [68%]). The most common medical specialty represented in the study was neurosurgery (6 [21%]), and intern doctors represented 39% of the study population. We deemed the randomization successful as there was no significant difference between the groups with respect to sex, medical specialty, level of experience, importance of religion in life, or location of experiment.

TABLE 1.

Demographic and medical training background of participants in the study

CategoryTotalw/ CRASH (experimental group)w/o CRASH (control group)
Total no. of participants281414
Demographics, no. (%)
 Female7 (25)3 (21)4 (29)
 Mean age in yrs (SD)30.2 (6.7)31.6 (7.4)28.9 (5.9)
Medical specialty, no. (%)
 Emergency medicine5 (18)2 (14)3 (21)
 General surgery6 (21)5 (36)1 (7)
 Intern doctor11 (39)4 (29)7 (50)*
 Neurosurgery6 (21)3 (21)3 (21)
Level of experience, no. (%)
 Consultant3 (11)2 (14)1 (7)
 Fellow1 (4)0 (0)1 (7)
 Resident13 (46)8 (57)5 (36)
 Intern11 (39)4 (29)7 (50)
 Mean yrs of experience (SD)4.6 (5.7)5.2 (6.1)4.1 (5.6)
Importance of religious faith in life, no. (%)
 Very important19 (68)11 (79)8 (57)
 Important6 (21)2 (14)4 (29)
 Neutral2 (7)1 (7)1 (7)
 Not important0 (0)0 (0)0 (0)
 Not available1 (4)0 (0)1 (7)
Location of experiment, no. (%)
 MRRH19 (68)9 (64)10 (71)
 MNRH9 (32)5 (36)4 (29)

MNRH = Mulago National Referral Hospital; MRRH = Mbarara Regional Referral Hospital.

Two interns in this group did not estimate prognosis: one was uncomfortable with the exercise, and one did not understand the exercise. One intern in this group did not complete the clinical decision-making study.

FIG. 1.
FIG. 1.

Flowchart depicting the number of participants in the experimental and control groups completing each part of the study: prognostication exercise, follow-up survey, and clinical decision-making survey.

Table 2 reports the mean estimated risk for different clinical outcomes provided by doctors in the experimental (with CRASH risk score; n = 14) and control (without CRASH risk score; n = 12) groups. Overall, participants were more optimistic than the CRASH risk score in estimating patient risk for both acute and chronic outcomes. The experimental and control groups estimated the risk of inpatient mortality as 40% and 32%, respectively, whereas the CRASH estimate was 51.4% (95% CI 42.8%, 59.8%). The experimental and control groups estimated the risk of unfavorable outcome at 6 months as 69% and 63%, respectively, compared to the CRASH estimate of 89.8% (95% CI 86.0%, 92.6%). When comparing the experimental and control groups, the former reported higher risk estimates for each clinical outcome except the risk for permanently needing a ventilator; however, these findings were not statistically significant for estimation of risk of death at 30 days (p = 0.050) and 6 months (p = 0.056).

TABLE 2.

Providers’ mean estimated risk for each clinical outcome

Clinical Outcomew/o CRASH (control group)w/ CRASH (experimental group)Difference (95% CI)p Value
Risk of inpatient mortality32%40%−8 (−25.4, 8.9)0.327
Risk of death at 30 days39%60%−21 (−42.1, −0.03)0.050
Risk of unfavorable outcome at 6 mos63%69%−6 (−27.0, 13.6)0.494
Risk of death at 6 mos43%63%−20 (−39.5, 0.57)0.056
Risk of permanently needing ventilator91%80%11 (−3.2, 26.8)0.120

CRASH estimated 14-day mortality was 51.4% (95% CI 42.8%, 59.8%). CRASH estimated risk of unfavorable outcome at 6 months was 89.8% (95% CI 86.0%, 92.6%).

When comparing risk estimation within levels of training, intern doctors in the experimental group (n = 4) estimated the risk of inpatient mortality at 44% compared to 14% for those in the control group (n = 5, p = 0.037). Similarly, interns in the experimental group estimated the risk of death at 6 months at 56% compared to 27% for those in the control group (p = 0.071). There was a similar difference between experimental and control intern doctor estimates for the risk of unfavorable outcome at 6 months; however, this finding was not statistically significant (p = 0.217). Risk estimates by intern doctors given the CRASH risk output more closely mirrored risk estimates by more experienced residents and consultants (Fig. 2).

FIG. 2.
FIG. 2.

Comparison of inexperienced and experienced providers’ estimated risk for each clinical outcome. Providers’ estimated risk for each clinical outcome without (control, upper) and with (experimental, lower) the CRASH risk output. The difference between the resident/consultant estimated risk and the intern estimated risk is listed on the right side of the plot.

Among 28 respondents (100%) completing the follow-up survey (Table 3), the most commonly selected best next step for the patient was surgery (24 [86%]). The participants estimated that this hypothetical patient had a 60% chance of undergoing CT and a 48% chance of undergoing TBI surgical intervention within 2 days of admission. Twenty-seven participants (96%) reported considering patient prognosis and resources available when deciding the best course of action for the patient. One participant (4%) reported considering resources only, and none of the participants reported considering prognosis alone. There were no significant differences between the groups with regard to these responses.

TABLE 3.

Providers’ responses to follow-up questions about the patient vignette

CategoryTotalw/ CRASH (experimental group)w/o CRASH (control group)
Total no. of participants281414
Best next step, no. (%)
 Surgery24 (86)12 (86)12 (86)
 ICU3 (11)2 (14)1 (7)
 Observe1 (4)0 (0)1 (7)
 Comfort measures0 (0)0 (0)0 (0)
Estimate resources
 % w/ CT in 2 days605664
 % w/ surgery in 2 days484353
What factors into decision, no. (%)
 Resources only1 (4)0 (0)1 (7)
 Prognosis only0 (0)0 (0)0 (0)
 Both27 (96)14 (100)13 (93)

ICU = intensive care unit.

A total of 27 participants (96%) completed the survey on clinical decision-making (Fig. 3). The majority of participants agreed or strongly agreed that they worry about accurately assessing prognosis in TBI patients (22 [81%]). Most participants also agreed or strongly agreed that they use patient data and the literature (25 [93%]) and prior experience (21 [81%]); one participant did not respond to estimate patient risk. A majority (24 [89%]) of participants reported a strong sense of obligation to offer surgery if there was a chance of survival. Only 8 participants (30%) agreed or strongly agreed that there are adequate quality data to estimate patient prognosis. Also, only 3 participants (11%) agreed or strongly agreed that they are willing to not offer surgery even if it means the patient will die.

FIG. 3.
FIG. 3.

Provider responses to a Likert scale survey about their decision-making in the care of patients with severe TBI. Percentages on the left represent participants who responded with “strongly disagree” or “disagree,” those in the middle represent participants who responded with “neutral,” and those on the right represent participants who responded with “agree” or “strongly agree.”

Discussion

We have performed one of the first experiments testing the impact of a TBI risk calculator on provider estimation of patient risk and prognosis in a low-resource setting. Additionally, we explored key factors influencing point-of-care decision-making for low-resource setting providers, including the perceived need for increased data to support this process. Finally, we captured contextual data that could inform the implementation of a TBI risk calculator in this setting.

Assessment of TBI severity and risk for complications is a complex process requiring the integration of clinical data and knowledge of the relevant literature with personal experience.5,6 A study performed in the United States has shown that emergency department provider characteristics such as age and years of experience impact their disposition decisions (a similarly complex process), with greater variation in risk perception among younger providers.12 Similarly, while participants in our study were overly optimistic in their risk estimation, this effect was particularly pronounced in inexperienced doctors. However, we found that interns presented with the CRASH risk output estimated risk at a level closer to the estimates of experienced doctors. Shortages of experienced doctors in LMICs have resulted in the increased use of task shifting, or the delegation of decision-making, to less specialized health workers.16–18 Interns, although less than 1 year removed from medical school, are heavily involved in the patient care in hospitals in LMICs. Given that initial decisions are informed by the perceived severity of injury, and since most providers endorsed using prior experience in prognostication, our findings highlight the potential of risk calculators in augmenting risk assessment and thus empowering inexperienced providers. Risk calculators can also serve as an anchor point for communication with more senior providers during decision-making in individual cases, as well as educational opportunities between providers at different levels.

For risk calculators to provide meaningful use in this setting, their implementation must be contextually and culturally relevant. In high-resource settings, providers have recognized the potential for clinical decision support systems to improve communication with families and justify treatment decisions.19 Concurrently, providers have expressed concern with these technologies as a final decision-maker, particularly in terms of end-of-life care.19 One study comparing LMICs in Asia to high-income ones revealed that physicians in LMICs were overall less likely to withhold aggressive life-sustaining treatment (LST), but were more likely than providers in HICs to consider the financial burden to patients and/or families and the institution (i.e., lack of resources) in their decision to limit LST.13 Similarly, while our study captured opposition to withholding care and a sense of duty among providers to perform surgery if there was some chance of survival, all participants considered the availability of resources when making treatment decisions. Although withholding surgery and withholding LST may not be perceived as morally equivalent, the results of these studies highlight two important concepts: that providers in low-resource settings are more reluctant than those in high-resource settings to withhold necessary care despite limited resources and that, again due to limited resources, these same providers may be more willing to withhold care they believe will offer little benefit relative to cost. Therefore, in this setting, long-term implementation of a risk calculator will depend on a greater understanding of how providers determine the necessity and potential benefit of surgery and how they reconcile severe resource constraints with an obligation to offer surgery to any patient with some chance of survival.

In both high- and low-resource settings, providers assess risk by using data from diagnostic tests, the literature, experience, and clinical findings. The majority of our study population reported inadequate data for decision-making and concern about accurately assessing prognosis. Risk calculators, by providing data on the potential for survival and poor outcomes, may inform providers’ perceptions of the benefits of a particular intervention. However, this knowledge may not automatically translate into a change in attitudes or practice. Indeed, our study showed that while providers given the CRASH risk score reported higher estimated risks for some outcomes than those reported by the control group, these results were not significant for the outcomes for which the CRASH risk estimate was provided. Further, providers were still relatively optimistic, and as discussed earlier, most reported a strong sense of obligation to offer surgery if there was a chance of survival. The findings of this study can be compared with those of a similar survey of neurosurgeons from multiple countries, which showed considerable variability in prognostication and only a modest effect from the CRASH risk calculator output (T. Williamson et al., unpublished data). While most providers in that study also recommended surgery, both the provider’s perception of a poor prognosis and provision of the risk calculator output reduced the likelihood of this recommendation. The limitations of the potential of risk assessment to influence practice are magnified in settings with limited resources. The physicians in the present study estimated that many patients would not undergo CT or surgery within 2 days, even if needed, because of resource constraints. These are important contextual considerations when designing the implementation of decision support technology in this setting. Future research should explore which decisions on the care continuum a risk calculator can support, how providers’ disagreement with the risk calculator score impacts decision-making, and how providers would incorporate the risk score in discussions with patients and families.

Study Limitations

The limitations of this study include the sample size and heterogeneity of the participants. The final sample size was 28, which decreased further for some analyses after excluding participants and subsetting the data by level of experience. The difference in risk estimation between the experimental and control groups did not reach statistical significance. Although one finding was significant when comparing the experimental and control groups of inexperienced doctors, this analysis was exploratory in nature. The small sample size warrants further research to confirm our findings in a larger population and determine why the effects of the CRASH risk output and provider experience were greater for some outcomes, but not others. In an effort to increase sample size, we included providers at two different academic hospitals and from different medical specialties. However, providers from different medical specialties likely weigh clinical findings differently when estimating patient risk and may use different decision algorithms when determining the best next steps for patients. Further, the observed differences in prognostication between groups may be attributable to an inherent difference in the aggressiveness of providers in each group. We believed these limitations were partly mitigated by the provider group randomization procedure and the level of task sharing between specialties and experience levels in this setting, but having each provider estimate patient risk both with and without the CRASH output would have offered more insight into its effect on prognostication despite individual and group variation. Finally, we used a single clinical vignette from a patient with an intermediate prognosis, whereas multiple vignettes also representing patients with very poor and excellent prognoses would have more accurately represented trends in prognostication. Future investigations should use providers as their own controls to estimate prognosis for patients with a range of outcomes, allowing more accurate conclusions to be drawn regarding the true potential of TBI risk calculators to enhance decision-making.

Conclusions

As data quality and infrastructure in LMICs improve, there is increased potential in developing and implementing risk calculators and decision support tools. In this study, we explored how a risk calculator impacts provider estimation of TBI patient risk at the two largest government hospitals in Uganda and found that such a tool may be most beneficial to providers with limited clinical experience from which to estimate prognosis. Moving forward, we aim to perform an expanded study in which individual providers estimate prognosis in patients across the spectrum of prognoses, both with and without information from a risk calculator. Such a study could be performed in neighboring countries and outside of East Africa, as well as in various LMICs and HICs, for a compelling comparison. Future investigations could also compare the impact of different risk calculators (e.g., CRASH, IMPACT) or locally derived prognostic models20 on provider prognostication. Finally, an understanding of the ethical framework in which these providers estimate prognosis and make treatment decisions is requisite to the successful implementation of decision support tools; this will be the focus of a future study. Although there is potential for a risk calculator to support decision-making and subsequent allocation of resources, thorough research on the feasibility and acceptability of this technology, as well as its advantages over prognostication by providers alone, is essential to transitioning these technologies from development to practice.

Acknowledgments

Mr. Elahi acknowledges scholarship support from the Duke University Center for International and Global Studies Research Award. Dr. Staton would like to acknowledge salary support provided by the Fogarty International Center of the National Institutes of Health under Award Number K01TW010000 (PI, Staton). Dr. Haglund acknowledges salary support provided by the Duke Department of Neurosurgery and the Duke Global Health Institute.

We would like to thank the Duke Global Health Institute, the Duke University Division of Global Neurosurgery and Neurology, Mbarara Regional Referral Hospital, and Mulago National Referral Hospital for facilitating the conduction of this investigation and the creation of this manuscript.

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Haglund, Elahi, Williamson, Nambi Najjuma, Nickenig Vissoci, Fuller, Kitya. Acquisition of data: Elahi, Williams, Nambi Najjuma, Fuller, Kitya. Analysis and interpretation of data: all authors. Drafting the article: Elahi, Spears, Nickenig Vissoci. Critically revising the article: all authors. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Haglund. Statistical analysis: Elahi, Spears, Nickenig Vissoci, Fuller. Administrative/technical/material support: Haglund, Elahi, Williamson, Spears, Williams, Staton, Nickenig Vissoci, Fuller, Kitya. Study supervision: Haglund, Elahi, Williamson, Nambi Najjuma, Nickenig Vissoci, Fuller, Kitya.

Supplemental Information

Online-Only Content

Supplemental material is available with the online version of the article.

Previous Presentations

An abstract based on this work was presented as a poster at the Congress of Neurological Surgeons Annual Meeting held in San Francisco, California, on October 19–23, 2019.

References

  • 1

    Maas AIR , Stocchetti N , Bullock R . Moderate and severe traumatic brain injury in adults . Lancet Neurol . 2008 ;7 (8 ):728 741 .

    • Search Google Scholar
    • Export Citation
  • 2

    Wheble JLC , Menon DK . TBI—the most complex disease in the most complex organ: the CENTER-TBI trial—a commentary . J R Army Med Corps . 2016 ;162 (2 ):87 89 .

    • Search Google Scholar
    • Export Citation
  • 3

    Lingsma HF , Roozenbeek B , Steyerberg EW , Early prognosis in traumatic brain injury: from prophecies to predictions . Lancet Neurol . 2010 ;9 (5 ):543 554 .

    • Search Google Scholar
    • Export Citation
  • 4

    Fuller A , Tran T , Muhumuza M , Haglund MM . Building neurosurgical capacity in low and middle income countries . eNeurologicalSci . 2015 ;3 :1 6 .

    • Search Google Scholar
    • Export Citation
  • 5

    Ramesh A , Fezeu F , Fidele B , Challenges and solutions for traumatic brain injury management in a resource-limited environment: example of a public referral hospital in Rwanda . Cureus . 2014 ;6 (5 ):e179 .

    • Search Google Scholar
    • Export Citation
  • 6

    Qureshi JS , Ohm R , Rajala H , Head injury triage in a sub Saharan African urban population . Int J Surg . 2013 ;11 (3 ):265 269 .

    • Search Google Scholar
    • Export Citation
  • 7

    Moons KGM , Royston P , Vergouwe Y , Prognosis and prognostic research: what, why, and how? BMJ . 2009 ;338 :b375 .

  • 8

    Perel P , Arango M , Clayton T , Predicting outcome after traumatic brain injury: practical prognostic models based on large cohort of international patients . BMJ . 2008 ;336 (7641 ):425 429 .

    • Search Google Scholar
    • Export Citation
  • 9

    Perel P , Edwards P , Wentz R , Roberts I . Systematic review of prognostic models in traumatic brain injury . BMC Med Inform Decis Mak . 2006 ;6 :38 .

    • Search Google Scholar
    • Export Citation
  • 10

    Murray LS , Teasdale GM , Murray GD , Does prediction of outcome alter patient management? Lancet . 1993 ;341 (8859 ):1487 1491 .

    • Search Google Scholar
    • Export Citation
  • 11

    Sacks GD , Dawes AJ , Ettner SL , Impact of a risk calculator on risk perception and surgical decision making: a randomized trial . Ann Surg . 2016 ;264 (6 ):889 895 .

    • Search Google Scholar
    • Export Citation
  • 12

    Capan M , Pigeon J , Marco D , We all make choices: a decision analysis framework for disposition decision in the ED . Am J Emerg Med . 2018 ;36 (3 ):450 454 .

    • Search Google Scholar
    • Export Citation
  • 13

    Phua J , Joynt GM , Nishimura M , Withholding and withdrawal of life-sustaining treatments in low-middle-income versus high-income Asian countries and regions . Intensive Care Med . 2016 ;42 (7 ):1118 1127 .

    • Search Google Scholar
    • Export Citation
  • 14

    Yeoman P , Pattani H , Silcocks P , Validation of the IMPACT outcome prediction score using the Nottingham Head Injury Register dataset . J Trauma . 2011 ;71 (2 ):387 392 .

    • Search Google Scholar
    • Export Citation
  • 15

    Vaca SD , Kuo BJ , Nickenig Vissoci JR , Temporal delays along the neurosurgical care continuum for traumatic brain injury patients at a tertiary care hospital in Kampala, Uganda . Neurosurgery . 2019 ;84 (1 ):95 103 .

    • Search Google Scholar
    • Export Citation
  • 16

    Ashengo T , Skeels A , Hurwitz EJH , Bridging the human resource gap in surgical and anesthesia care in low-resource countries: a review of the task sharing literature . Hum Resour Health . 2017 ;15 (1 ):77 .

    • Search Google Scholar
    • Export Citation
  • 17

    Mundeva H , Snyder J , Ngilangwa DP , Kaida A . Ethics of task shifting in the health workforce: exploring the role of community health workers in HIV service delivery in low- and middle-income countries . BMC Med Ethics . 2018 ;19 (1 ):71 .

    • Search Google Scholar
    • Export Citation
  • 18

    Sayed S , Field A , Rajab J , Task sharing and shifting to provide pathology diagnostic services: the Kenya fine-needle aspiration biopsy cytology and bone marrow aspiration and trephine biopsy training program . J Glob Oncol . 2018 ;4 :1 11 .

    • Search Google Scholar
    • Export Citation
  • 19

    Weber S : Impacts of clinical decision support technology on nursing and medical practice in U.S. critical care . Can J Nurs Inform . 2011 ;5 (4 ):3 . http://cjni.net/journal/?p=1094. Accessed February 21, 2020 .

    • Search Google Scholar
    • Export Citation
  • 20

    Hernandes Rocha TA , Elahi C , Cristina da Silva N , A traumatic brain injury prognostic model to support in-hospital triage in a low-income country: a machine learning-based approach [published online May 10, 2019] . J Neurosurg . doi:10.3171/2019.2.JNS182098

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Supplementary Materials

Contributor Notes

Correspondence Michael M. Haglund: Duke University, Durham, NC. michael.haglund@dm.duke.edu.

INCLUDE WHEN CITING Published online April 3, 2020; DOI: 10.3171/2020.2.JNS192512.

Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

  • View in gallery

    Flowchart depicting the number of participants in the experimental and control groups completing each part of the study: prognostication exercise, follow-up survey, and clinical decision-making survey.

  • View in gallery

    Comparison of inexperienced and experienced providers’ estimated risk for each clinical outcome. Providers’ estimated risk for each clinical outcome without (control, upper) and with (experimental, lower) the CRASH risk output. The difference between the resident/consultant estimated risk and the intern estimated risk is listed on the right side of the plot.

  • View in gallery

    Provider responses to a Likert scale survey about their decision-making in the care of patients with severe TBI. Percentages on the left represent participants who responded with “strongly disagree” or “disagree,” those in the middle represent participants who responded with “neutral,” and those on the right represent participants who responded with “agree” or “strongly agree.”

  • 1

    Maas AIR , Stocchetti N , Bullock R . Moderate and severe traumatic brain injury in adults . Lancet Neurol . 2008 ;7 (8 ):728 741 .

    • Search Google Scholar
    • Export Citation
  • 2

    Wheble JLC , Menon DK . TBI—the most complex disease in the most complex organ: the CENTER-TBI trial—a commentary . J R Army Med Corps . 2016 ;162 (2 ):87 89 .

    • Search Google Scholar
    • Export Citation
  • 3

    Lingsma HF , Roozenbeek B , Steyerberg EW , Early prognosis in traumatic brain injury: from prophecies to predictions . Lancet Neurol . 2010 ;9 (5 ):543 554 .

    • Search Google Scholar
    • Export Citation
  • 4

    Fuller A , Tran T , Muhumuza M , Haglund MM . Building neurosurgical capacity in low and middle income countries . eNeurologicalSci . 2015 ;3 :1 6 .

    • Search Google Scholar
    • Export Citation
  • 5

    Ramesh A , Fezeu F , Fidele B , Challenges and solutions for traumatic brain injury management in a resource-limited environment: example of a public referral hospital in Rwanda . Cureus . 2014 ;6 (5 ):e179 .

    • Search Google Scholar
    • Export Citation
  • 6

    Qureshi JS , Ohm R , Rajala H , Head injury triage in a sub Saharan African urban population . Int J Surg . 2013 ;11 (3 ):265 269 .

    • Search Google Scholar
    • Export Citation
  • 7

    Moons KGM , Royston P , Vergouwe Y , Prognosis and prognostic research: what, why, and how? BMJ . 2009 ;338 :b375 .

  • 8

    Perel P , Arango M , Clayton T , Predicting outcome after traumatic brain injury: practical prognostic models based on large cohort of international patients . BMJ . 2008 ;336 (7641 ):425 429 .

    • Search Google Scholar
    • Export Citation
  • 9

    Perel P , Edwards P , Wentz R , Roberts I . Systematic review of prognostic models in traumatic brain injury . BMC Med Inform Decis Mak . 2006 ;6 :38 .

    • Search Google Scholar
    • Export Citation
  • 10

    Murray LS , Teasdale GM , Murray GD , Does prediction of outcome alter patient management? Lancet . 1993 ;341 (8859 ):1487 1491 .

    • Search Google Scholar
    • Export Citation
  • 11

    Sacks GD , Dawes AJ , Ettner SL , Impact of a risk calculator on risk perception and surgical decision making: a randomized trial . Ann Surg . 2016 ;264 (6 ):889 895 .

    • Search Google Scholar
    • Export Citation
  • 12

    Capan M , Pigeon J , Marco D , We all make choices: a decision analysis framework for disposition decision in the ED . Am J Emerg Med . 2018 ;36 (3 ):450 454 .

    • Search Google Scholar
    • Export Citation
  • 13

    Phua J , Joynt GM , Nishimura M , Withholding and withdrawal of life-sustaining treatments in low-middle-income versus high-income Asian countries and regions . Intensive Care Med . 2016 ;42 (7 ):1118 1127 .

    • Search Google Scholar
    • Export Citation
  • 14

    Yeoman P , Pattani H , Silcocks P , Validation of the IMPACT outcome prediction score using the Nottingham Head Injury Register dataset . J Trauma . 2011 ;71 (2 ):387 392 .

    • Search Google Scholar
    • Export Citation
  • 15

    Vaca SD , Kuo BJ , Nickenig Vissoci JR , Temporal delays along the neurosurgical care continuum for traumatic brain injury patients at a tertiary care hospital in Kampala, Uganda . Neurosurgery . 2019 ;84 (1 ):95 103 .

    • Search Google Scholar
    • Export Citation
  • 16

    Ashengo T , Skeels A , Hurwitz EJH , Bridging the human resource gap in surgical and anesthesia care in low-resource countries: a review of the task sharing literature . Hum Resour Health . 2017 ;15 (1 ):77 .

    • Search Google Scholar
    • Export Citation
  • 17

    Mundeva H , Snyder J , Ngilangwa DP , Kaida A . Ethics of task shifting in the health workforce: exploring the role of community health workers in HIV service delivery in low- and middle-income countries . BMC Med Ethics . 2018 ;19 (1 ):71 .

    • Search Google Scholar
    • Export Citation
  • 18

    Sayed S , Field A , Rajab J , Task sharing and shifting to provide pathology diagnostic services: the Kenya fine-needle aspiration biopsy cytology and bone marrow aspiration and trephine biopsy training program . J Glob Oncol . 2018 ;4 :1 11 .

    • Search Google Scholar
    • Export Citation
  • 19

    Weber S : Impacts of clinical decision support technology on nursing and medical practice in U.S. critical care . Can J Nurs Inform . 2011 ;5 (4 ):3 . http://cjni.net/journal/?p=1094. Accessed February 21, 2020 .

    • Search Google Scholar
    • Export Citation
  • 20

    Hernandes Rocha TA , Elahi C , Cristina da Silva N , A traumatic brain injury prognostic model to support in-hospital triage in a low-income country: a machine learning-based approach [published online May 10, 2019] . J Neurosurg . doi:10.3171/2019.2.JNS182098

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 10 10 0
Full Text Views 90 90 66
PDF Downloads 47 47 34
EPUB Downloads 0 0 0