The vast amount of information available via the internet has made it an increasingly popular resource for patients seeking health information. It is estimated that around 52 million people in the United States alone have utilized online resources to seek medical information, with 70% of these individuals stating that online information has influenced their medical decision-making.1,2 Online patient education resources can include both written and video material. One of the largest online video platforms, YouTube, hosts over 2 billion users every month and offers a unique avenue to reach patients searching for online health information.3 However, it is vital that the information from online patient education videos be of high quality. Poor-quality health information can mislead patients and/or create misconceptions regarding their treatment, which may negatively impact patient perceptions of treatment and clinical improvement after treatment.4,5 Several studies have investigated the reliability and quality of videos appearing on YouTube that provide patient education information for a variety of medical topics and specialty fields.6–11 However, there is scarce literature addressing YouTube content on minimally invasive spine surgery.
With spinal pain remaining the leading cause of global disability since 1990,12 many advances in minimally invasive spine surgery have been developed to treat a variety of lumbar spine pathologies. Among these new minimally invasive treatment options, lateral lumbar interbody fusion (LLIF) offers a lateral approach to the lumbar disc space and has grown in popularity because it offers several advantages over posterior and anterior approaches, such as preserving the annular/ligamentous structures, not requiring an access surgeon, and lowering risks of dural tear, muscle denervation, and infections.13–16 With the increasing popularity of the LLIF option for spine surgery, patients are turning to online patient education resources for supplemental information regarding this procedure, and it is vital that the quality of available information be properly assessed.
Our aim was to gauge the reliability and quality of online LLIF educational videos. We approached this task with the DISCERN tool, a validated measure of reliability and quality for online patient education resources, which are rated on the basis of 15 specific criteria (http://www.discern.org.uk/). According to the DISCERN instrument, a resource is reliable when it provides appropriate and influential information that is based on good evidence. Quality measurements performed with the DISCERN tool reflect how comprehensively the resource describes all aspects of a treatment, including risks, benefits, alternatives, and effects on quality of life. The DISCERN tool consists of 2 sections, with the first measuring reliability and the second measuring quality; the average score of the 15 questions places a resource in 1 of 3 overall quality categories: poor, moderate, or high.
Given the increasing numbers of people using online patient education resources, the reliability of online medical information should be regularly measured by qualified medical personnel, particularly information that may be the patient’s first source of knowledge about a complex surgical procedure. Previous studies analyzing the reliability of online patient education videos in other areas of neurosurgery have demonstrated the need for quality improvement based on the DISCERN criteria,17,18 and we hypothesized that a similar trend would be observed among LLIF videos.
Methods
Search Strategy and Data Selection
In December 2018, we conducted an online video query on the YouTube platform using 3 separate search terms: “lateral lumbar interbody fusion,” “LLIF surgery,” and “LLIF.” After excluding all videos unrelated to neurosurgery or spinal surgery, a total of 234, 15, and 47 unique videos were further analyzed for each of the above search terms, respectively. Duplicate videos among the searches were included only once. Videos were sorted using the relevance-based ranking filter, which is the default search option on YouTube and provides results that are most likely to match the viewer’s original query. Because previous internet search engine analysis has shown that fewer than 17% of users browse beyond the first 3 pages of the results, only the first 3 pages for each search term were analyzed.19 After all of the videos were watched, they were sorted into 3 categories according to source of origin: 1) videos made by universities/academic institutions, 2) videos made by hospitals, and 3) videos made by educational associations. Videos were also sorted into “surgical videos” and “nonsurgical videos” based on whether the videos included intraoperative surgical clips of the procedure.
Inclusion and Exclusion Criteria
Video inclusion criteria were adopted and adjusted from previous studies.8,9 According to these criteria, only educational videos released in English by universities/academic institutions, hospitals, and educational associations were considered for final inclusion. Upon review of the 296 videos to assess whether they met these criteria, a final list of 10 videos met inclusion criteria and were investigated for further analysis using the DISCERN tool and YouTube data. Placement into these categories was determined by the author and the name of each video; for example, LLIF education videos posted by the University of Pittsburgh Neurosurgery Department could reliably be classified as produced by an academic center. Videos made by private practice groups and those with a focus on a different operative technique were excluded from analysis. Duplicate videos were analyzed only once, and videos in multiple parts were analyzed together and counted as a single video.
Scoring System
Videos were independently assessed by 3 medical providers using the validated DISCERN tool. For each video, the 3 final scores were then averaged (Table 1). DISCERN is an instrument intended to evaluate the quality of health information. The questionnaire presents 15 quality indicators, which the evaluators score on a 5-point scale ranging from information of poor quality (< 3.0). to moderate (3.0–3.99), and to good quality (≥ 4.0). Fulfillment of most of the DISCERN criteria indicates that an information source has high overall quality and thus is an evidence-supported resource about treatment choices. A moderate overall quality source of information indicates the midranges of quality and that the video suffers some limitations, and thus further evidence would be needed to clearly understand treatment choices. A source with low overall quality of information, indicated by a DISCERN score of less than 3.0, offers poor and confusing advice supported by low-level scientific evidence. The first section of the DISCERN tool, consisting of 8 criteria, focuses on reliability: for a video to be reliable it must state its aims clearly and be objective, unbiased, and supported by strong evidence. In the second section, 7 questions focus on information quality: for a video to be high quality, it must present a clear description of the treatment, its benefits and risks, and consequences of delayed treatment. At the end of the assessment of each video, the evaluators provided an overall averaged score to demonstrate the overall quality of information using the above-mentioned 5-point scale. Further detail regarding this tool is available in the original study by Charnock et al.20
Total DISCERN scores from the 3 evaluators and average score for each video
| Total DISCERN Score | ||||
|---|---|---|---|---|
| Institution | Rater 1 | Rater 2 | Rater 3 | Average DISCERN Score |
| Tampa General Hospital | 3.73 | 3.23 | 4.20 | 3.72 |
| HCA Virginia Health System | 3.40 | 3.42 | 3.13 | 3.31 |
| University of Southern California Neurosurgery | 1.00 | 1.00 | 1.13 | 1.04 |
| AANS Neurosurgery | 4.10 | 4.50 | 4.26 | 4.28 |
| AANS Neurosurgery | 2.90 | 3.57 | 3.20 | 3.22 |
| University of Pittsburgh Medical Center | 4.20 | 4.10 | 2.73 | 3.67 |
| University of Pittsburgh Medical Center | 4.50 | 4.20 | 3.06 | 3.92 |
| Barrow Neurological Institute | 3.00 | 2.50 | 3.33 | 2.94 |
| University of Pittsburgh Neurosurgery | 4.20 | 3.90 | 3.86 | 3.98 |
| University of Pittsburgh Neurosurgery | 4.60 | 4.20 | 3.53 | 4.11 |
Intraclass correlation coefficient of interrater reliability 0.981 (CI 0.945–0.995).
Data Collection and Outcomes
Information retrieved for each of the 10 videos in the final evaluation included the title, author, year of publication, number of days online, video duration, number of views, number of likes, and number of dislikes. Videos were sorted into 3 subcategories according to their source: universities/academic institutions, hospitals, and educational associations. Audience interaction with the video was assessed by examining the numbers of comments, likes per day, and views per day. The numbers of views and likes per day was used as an index of “popularity,” and the association between video quality (based on the DISCERN score) and popularity was measured to determine whether videos presenting a higher educational content engaged more with the audience. We also documented inclusion in the video of physician spokespersons and patients reporting their own experience of the surgical intervention, to test the association between their presence and video popularity. We hypothesized that higher video popularity (more likes and views) would be correlated with higher quality (DISCERN score) and that videos with patient testimonials would be correlated with higher popularity (more likes and views).
Statistical Analysis
Means, medians, and percentages were generated for continuous and discrete variables, where appropriate. Continuous variables were compared via the Student t-test and discrete variables were analyzed with the chi-square test. Where appropriate, linear regression analysis was performed to test the relationships between quantitative variables. Finally, interrater reliability was analyzed by calculating intraclass correlation coefficients. A p value < 0.05 was considered statistically significant.
Results
Included Studies
After exclusion of inappropriate results (i.e., all videos unrelated to neurosurgery or spinal surgery), the initial search yielded 296 videos, 234 for the search term “lateral lumbar interbody fusion,” 15 for “LLIF surgery,” and 47 for “LLIF.” Following further assessment of each video, a total of 10 of 296 videos (3.37%) met inclusion criteria and were included in the final evaluation. Most exclusions of videos were due to content involving discussion of different approaches or having been produced by commercial companies or private practice physicians. Of the 10 videos included, 5 resulted from the search term “lateral lumbar interbody fusion” (2.1% of videos for that search term and 50% of videos included). The other 5 videos were acquired from the search term “LLIF surgery” (33.3% of videos for that search term and 50% of videos included). No videos resulting from the search term “LLIF” were included due to overlap or being considered not educationally valuable by the authors (Table 2).
Main characteristics of videos sorted by search term
| Search Term | |||
|---|---|---|---|
| Variable | Lateral Lumbar Interbody Fusion | LLIF Surgery | LLIF |
| Videos, no. (%) | |||
| Found | 234 | 15 | 47 |
| Met final inclusion criteria | 5 (2.1%) | 5 (33.3%) | 0 |
| Surgical | 4 (80%) | 1 (20%) | NA |
| Average DISCERN score | 3.1 | 3.7 | NA |
NA = not applicable.
Characteristics of the 10 videos selected for the final analysis, including video category, year of publication, number of days online, video length, number of views, and number of likes and dislikes, are shown in Table 3. The total number of views for the final 10 videos was 175,941 (133,718 from the search term “lateral lumbar interbody fusion” and 42,223 for the search term “LLIF surgery”). The average quality scores for the videos found with the search terms “lateral lumbar interbody fusion” and “LLIF surgery” were 3.1 and 3.7, respectively. A linear regression analysis found no correlation between the number of views and the quality score of the video (r2 = 0.01, p = 0.80). Videos included for analysis were released in a total span of 8 years, from 2011 to 2018.
Characteristics of videos included in the analysis
| Institution | Category | Publication Yr | No. of Days Online | Duration (mins:secs) | No. of Views | No. of Likes | No. of Dislikes |
|---|---|---|---|---|---|---|---|
| Tampa General Hospital | Hospital | 2011 | 3067 | 59:47 | 87,910 | 203 | 14 |
| HCA Virginia Health System | Hospital | 2013 | 2132 | 2:35 | 6202 | 4 | 3 |
| University of Southern California Neurosurgery | Academic institution | 2013 | 2143 | 3:02 | 12,662 | 0 | 0 |
| AANS Neurosurgery | Educational association | 2012 | 2747 | 31:58 | 11,911 | 37 | 3 |
| AANS Neurosurgery | Educational association | 2016 | 1130 | 7:52 | 15,033 | 41 | 9 |
| University of Pittsburgh Medical Center | Academic institution | 2018 | 429 | 1:34 | 367 | 1 | 0 |
| University of Pittsburgh Medical Center | Academic institution | 2014 | 1705 | 2:47 | 1675 | 7 | 0 |
| Barrow Neurological Institute | Academic institution | 2018 | 545 | 34:13 | 3774 | 35 | 4 |
| University of Pittsburgh Neurosurgery | Academic institution | 2015 | 1676 | 18:45 | 34,917 | 27 | 2 |
| University of Pittsburgh Neurosurgery | Academic institution | 2017 | 830 | 18:35 | 1490 | 8 | 0 |
Analysis of Videos
The DISCERN scores for each of the 10 videos selected for the final analysis are shown in Table 1. Analysis of interrater reliability was performed, and the intraclass correlation coefficient and 95% confidence interval between the 3 scorers was 0.981 (CI 0.945–0.995), indicating a high level of interrater reliability. Of the 10 videos, 7 were released by academic institutions: 1 by the University of Southern California, 1 by the Barrow Neurological Institute, 1 by Tampa General Hospital (University of South Florida), and 4 by the University of Pittsburgh; 2 were released by an educational association, the American Association of Neurological Surgeons (AANS); and 1 was released by a hospital: HCA Virginia Health System. Only 2 videos (20%) were rated highly, with an average DISCERN score of 4 or above. The video with the highest average score was rated 4.3 and was released by the AANS, which published the video in 2012 (2747 days online). The video with the second highest average score was rated 4.1 and was released by the University of Pittsburgh, which published the video in 2017 (830 days online). No significant correlation between video quality (DISCERN score ≥ 4) and video popularity (as defined by the numbers of views and likes per video age) could be drawn (p value = 0.104; Table 4).
Video characteristics sorted by DISCERN score
| DISCERN Score | |||
|---|---|---|---|
| Variable | 4–5 | 3 | 1–2 |
| Videos | 2 (20%) | 6 (60%) | 2 (20%) |
| Surgical videos | 0% | 4 (67%) | 2 (100%) |
| Median length, mins:secs | 25:16 | 13:09 | 18:37 |
| Median views | 6701 (1490–11,911) | 24,351 (367–87,910) | 8218 (3774–12,662) |
| Median likes | 22.5 (37–8) | 52.16 (1–203) | 17.5 (0–35) |
| Median dislikes | 1.5 (0–3) | 4.83 (0–14) | 2 (0–4) |
| Median comments | 0 | 11.8 (0–57) | 0.5 (0–1) |
Values are presented as number of videos (%) or median (range) unless otherwise indicated.
Five videos contained actual operating room footage of the LLIF procedure, and the association between intraoperative recordings and video popularity was tested. For this purpose, multiple-part videos were considered as a single item. Of the 10 videos, 5 (50%) contained clips from the operating room and were thus classified as “surgical videos.” Accordingly, the remaining 5 videos (50%) were classified as nonsurgical. We performed an unpaired single-tail Student t-test assuming unequal variance. Videos focusing on the surgical procedure were shown to be significantly correlated with greater popularity (p value = 0.01), even though the 2 videos with scores indicating them to be poorly informative (DISCERN < 3.0) both contained intraoperative clips. No significant difference in popularity, determined by the number of likes and total number of views, was found between poor quality surgical videos and moderately good quality surgical videos. Most videos (90%) included a physician spokesperson. Patients’ direct experiences of the procedure were reported in 2 videos (20%). However, the presence of patients sharing personal testimonials was not associated with increased video popularity.
Discussion
Several studies in the literature have assessed the quality of YouTube content focusing on surgical procedures, including lumbar discectomy, pediatric adenotonsillectomy, tympanostomy tube placement, glioblastoma treatment, cataract surgery, and anterior cruciate ligament injury and reconstruction.6,8,10,21,22 Overall, it has been shown that the internet allows patients to readily access a consistent amount of videos dispensing medical information. However, many of these videos provide limited medical advice, and the information that is provided is sometimes confusing and requires further evidence. To our knowledge, no previously reported studies to date have addressed the quality of the content of online educational videos for LLIF.
In current methods for estimating quality of care and patient satisfaction, patients’ individual preoperative expectations and subjective perceptions of their postoperative outcomes are becoming increasingly important in addition to the customarily used objective and technical measures of surgical outcomes.4,5,23,24 There is evidence in the literature showing that failure to reach patients’ expectations leads to diminished patient compliance with medical recommendations and worse clinical outcomes.23 Concurrently, literature has shown that patients who harbor an optimistic perspective on a specific treatment strategy often show improved outcomes.25,26 Thus, because the extent of patients’ medical knowledge and expectations has proven to be crucial with regard to the perceived outcome of a surgical procedure, it is imperative that the easily accessible health information on the internet be critically reviewed. Ultimately, although medical decision-making should always be centered on a thorough conversation between patient and physician, online educative materials have allowed patients to access supplemental information while at home, thus allowing them to gain a better understanding of their treatment options and to prepare thoughtful questions regarding their treatment to bring up during discussions with their surgeons.
Our analysis reveals that online videos pertaining to LLIF present an overall moderate quality of information, which should be supplemented with additional information. Moderate quality information harbors a range of limitations, including lack of clear presentations of information pertaining to treatment risks, consequences of treatment delay, and postoperative modifications.
The 2 videos rated to be good quality (DISCERN score > 4) were not found to be significantly more popular than those with scores indicating lower quality. Notably, both of the good quality videos are recordings of lectures given during neurosurgical symposia. Hypothetically, the perceived specificity of the presentations may lead patients to skip these higher-quality videos and seek information elsewhere.
The results of this study have shown that intraoperative surgical videos are significantly more popular than nonsurgical videos. Notably, the intraoperative surgical videos included both of the videos found to be of poor quality (DISCERN score < 2, considered to be poorly informative for patients). No significant difference in popularity, as defined by numbers of views and likes per video age, was found between surgical videos with DISCERN scores indicating poor versus moderately good quality (p > 0.05). This finding may suggest that the quality of information provided is not a key factor in determining video popularity. Instead, the insertion of video clips from the operating room may play a larger role in popularity.
When academic institutions release online educational content to support knowledge on pathologies and their corresponding therapeutic strategies, all quality criteria should be fully met to maximize the usefulness of the educational material in enabling patients to set reasonable expectations and make well-informed decisions. A strategy to improve the quality of uploaded online videos would be to implement the release of peer-reviewed educational contents produced by academically recognized institutions. The educational contents should be developed in accordance with and fulfill the DISCERN assessment criteria, thus providing quality assurance for patients and enabling them to promptly access exhaustive and evidence-based advice.
Study Limitations
Our study addresses only a portion of the information available on YouTube. Of the total videos (296) resulting from our selected search terms, 10 videos (3.37%) were selected for the final list included in our analysis. This restricted sample size limits the sensitivity of our study. Videos released in languages other than English were excluded from evaluation. In addition, it is noteworthy that the algorithm of the YouTube search engine changes according to the geographic location of the viewer, time of posting, and level of interaction with the video. Because our data were captured on a single day, our study provides information that was exclusively present at the time of the study. Finally, only the YouTube platform was analyzed, thus excluding other online resources to which patients may refer.
Conclusions
Our analysis shows that the vast majority (60%) of LLIF educational content available online presents a moderate overall quality. Although YouTube content on LLIF is informative and a relevant resource for patients, the goal of presenting more content of high quality requires the inclusion of further evidence and more information on resources and treatments. To achieve an online presence that provides patients with optimal and sustained guidance regarding minimally invasive spine surgeries, including LLIF, we encourage academic institutions in the United States to implement the release of peer-reviewed online educational content. We believe that the creation of peer-reviewed educational content utilizing validated assessment tools such as the DISCERN criteria will provide patients and their families with the most reliable and high-quality online medical information.
Disclosures
The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.
Author Contributions
Conception and design: Agarwal, White. Acquisition of data: White, Latour. Analysis and interpretation of data: White, Latour. Drafting the article: White, Latour, Giordano. Critically revising the article: Agarwal. Reviewed submitted version of manuscript: Agarwal, White, Latour, Giordano. Approved the final version of the manuscript on behalf of all authors: Agarwal. Statistical analysis: White, Latour, Taylor. Study supervision: Agarwal.
References
- 1↑
Rainie L, Fox S. The online health care revolution. Pew Research Center. Accessed April 21, 2020. http://www.pewinternet.org/2000/11/26/the-online-health-care-revolution/
- 2↑
Rice RE. Influences, usage, and outcomes of Internet health information searching: multivariate results from the Pew surveys. Int J Med Inform. 2006;75(1):8–28.
- 3↑
Spangler T. YouTube now has 2 billion monthly users, who watch 250 million hours on TV screens daily. Variety. May 3, 2019. Accessed April 21, 2020. https://variety.com/2019/digital/news/youtube-2-billion-users-tv-screen-watch-time-hours-1203204267/
- 4↑
Franz EW, Bentley JN, Yee PP, et al. Patient misconceptions concerning lumbar spondylosis diagnosis and treatment. J Neurosurg Spine. 2015;22(5):496–502.
- 5↑
Myles PS, Williams DL, Hendrata M, et al. Patient satisfaction after anaesthesia and surgery: results of a prospective survey of 10,811 patients. Br J Anaesth. 2000;84(1):6–10.
- 6↑
Bae SS, Baxter S. YouTube videos in the English language as a patient education resource for cataract surgery. Int Ophthalmol. 2018;38(5):1941–1945.
- 7
MacLeod MG, Hoppe DJ, Simunovic N, et al. YouTube as an information source for femoroacetabular impingement: a systematic review of video content. Arthroscopy. 2015;31(1):136–142.
- 8↑
ReFaey K, Tripathi S, Yoon JW, et al. The reliability of YouTube videos in patients education for glioblastoma treatment. J Clin Neurosci. 2018;55:1–4.
- 9↑
Rittberg R, Dissanayake T, Katz SJ. A qualitative analysis of methotrexate self-injection education videos on YouTube. Clin Rheumatol. 2016;35(5):1329–1333.
- 10↑
Sorensen JA, Pusz MD, Brietzke SE. YouTube as an information source for pediatric adenotonsillectomy and ear tube surgery. Int J Pediatr Otorhinolaryngol. 2014;78(1):65–70.
- 11
Stamelou M, Edwards MJ, Espay AJ, et al. Movement disorders on YouTube—caveat spectator. N Engl J Med. 2011;365(12):1160–1161.
- 12↑
Briggs AM, Woolf AD, Dreinhöfer K, et al. Reducing the global burden of musculoskeletal conditions. Bull World Health Organ. 2018;96(5):366–368.
- 13
Humphreys SC, Hodges SD, Patwardhan AG, et al. Comparison of posterior and transforaminal approaches to lumbar interbody fusion. Spine (Phila Pa 1976). 2001;26(5):567–571.
- 14
Lehmen JA, Gerber EJ. MIS lateral spine surgery: a systematic literature review of complications, outcomes, and economics. Eur Spine J. 2015;24(suppl 3):287–313.
- 15
Rajaraman V, Vingan R, Roth P, et al. Visceral and vascular complications resulting from anterior lumbar interbody fusion. J Neurosurg. 1999;91(1)(suppl):60–64.
- 16
Sasso R, Kenneth Burkus J, LeHuec J. Retrograde ejaculation after anterior lumbar interbody fusion: transperitoneal versus retroperitoneal exposure. Spine (Phila Pa 1976). 2003;28(10):1023–1026.
- 17↑
Szmuda T, Rosvall P, Hetzger TV, et al. YouTube as a source of patient information for hydrocephalus: a content-quality and optimization analysis. World Neurosurg. Published online March 5, 2020. doi:10.1016/j.wneu.2020.02.149
- 18↑
Tripathi S, ReFaey K, Stein R, et al. The reliability of deep brain stimulation YouTube videos. J Clin Neurosci. 2020;74:202–204.
- 19↑
Jansen BJ, Spink A. An analysis of web documents retrieved and viewed. In Proceedings of the International Conference on Internet Computing; June 23–26, 2003; Las Vegas, NV: 65–69.
- 20↑
Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105–111.
- 21↑
Brooks FM, Lawrence H, Jones A, McCarthy MJ. YouTube™ as a source of patient information for lumbar discectomy. Ann R Coll Surg Engl. 2014;96(2):144–146.
- 22↑
Cassidy JT, Fitzgerald E, Cassidy ES, et al. YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surg Sports Traumatol Arthrosc. 2018;26(3):840–845.
- 23↑
Bell RA, Kravitz RL, Thom D, et al. Unmet expectations for care and the patient-physician relationship. J Gen Intern Med. 2002;17(11):817–824.
- 24↑
McKinley RK, Stevenson K, Adams S, Manku-Scott TK. Meeting patient expectations of care: the major determinant of satisfaction with out-of-hours primary medical care? Fam Pract. 2002;19(4):333–338.
- 25↑
Myers SS, Phillips RS, Davis RB, et al. Patient expectations as predictors of outcome in patients with acute low back pain. J Gen Intern Med. 2008;23(2):148–153.
- 26↑
Verbeek J, Sengers MJ, Riemens L, Haafkens J. Patient expectations of treatment for back pain: a systematic review of qualitative and quantitative studies. Spine (Phila Pa 1976). 2004;29(20):2309–2318.
