Accuracy analysis of fMRI and MEG activations determined by intraoperative mapping

Free access

OBJECTIVE

By looking at how the accuracy of preoperative brain mapping methods vary according to differences in the distance from the activation clusters used for the analysis, the present study aimed to elucidate how preoperative functional neuroimaging may be used in such a way that maximizes the mapping accuracy.

METHODS

The eloquent function of 19 patients with a brain tumor or cavernoma was mapped prior to resection with both functional MRI (fMRI) and magnetoencephalography (MEG). The mapping results were then validated using direct cortical stimulation mapping performed immediately after craniotomy and prior to resection. The subset of patients with equivalent MEG and fMRI tasks performed for motor (n = 14) and language (n = 12) were evaluated as both individual and combined predictions. Furthermore, the distance resulting in the maximum accuracy, as evaluated by the J statistic, was determined by plotting the sensitivities and specificities against a linearly increasing distance threshold.

RESULTS

fMRI showed a maximum mapping accuracy at 5 mm for both motor and language mapping. MEG showed a maximum mapping accuracy at 40 mm for motor and 15 mm for language mapping. At the standard 10-mm distance used in the literature, MEG showed a greater specificity than fMRI for both motor and language mapping but a lower sensitivity for motor mapping. Combining MEG and fMRI showed a maximum accuracy at 15 mm and 5 mm—MEG and fMRI distances, respectively—for motor mapping and at a 10-mm distance for both MEG and fMRI for language mapping. For motor mapping, combining MEG and fMRI at the optimal distances resulted in a greater accuracy than the maximum accuracy of the individual predictions.

CONCLUSIONS

This study demonstrates that the accuracy of language and motor mapping for both fMRI and MEG is heavily dependent on the distance threshold used in the analysis. Furthermore, combining MEG and fMRI showed the potential for increased motor mapping accuracy compared to when using the modalities separately.

Clinical trial registration no.: NCT01535430 (clinicaltrials.gov)

ABBREVIATIONS BVS = blood vessel segmentation; DCS = direct cortical stimulation; fMRI = functional MRI; MEG = magnetoencephalography; T1W = T1-weighted.

OBJECTIVE

By looking at how the accuracy of preoperative brain mapping methods vary according to differences in the distance from the activation clusters used for the analysis, the present study aimed to elucidate how preoperative functional neuroimaging may be used in such a way that maximizes the mapping accuracy.

METHODS

The eloquent function of 19 patients with a brain tumor or cavernoma was mapped prior to resection with both functional MRI (fMRI) and magnetoencephalography (MEG). The mapping results were then validated using direct cortical stimulation mapping performed immediately after craniotomy and prior to resection. The subset of patients with equivalent MEG and fMRI tasks performed for motor (n = 14) and language (n = 12) were evaluated as both individual and combined predictions. Furthermore, the distance resulting in the maximum accuracy, as evaluated by the J statistic, was determined by plotting the sensitivities and specificities against a linearly increasing distance threshold.

RESULTS

fMRI showed a maximum mapping accuracy at 5 mm for both motor and language mapping. MEG showed a maximum mapping accuracy at 40 mm for motor and 15 mm for language mapping. At the standard 10-mm distance used in the literature, MEG showed a greater specificity than fMRI for both motor and language mapping but a lower sensitivity for motor mapping. Combining MEG and fMRI showed a maximum accuracy at 15 mm and 5 mm—MEG and fMRI distances, respectively—for motor mapping and at a 10-mm distance for both MEG and fMRI for language mapping. For motor mapping, combining MEG and fMRI at the optimal distances resulted in a greater accuracy than the maximum accuracy of the individual predictions.

CONCLUSIONS

This study demonstrates that the accuracy of language and motor mapping for both fMRI and MEG is heavily dependent on the distance threshold used in the analysis. Furthermore, combining MEG and fMRI showed the potential for increased motor mapping accuracy compared to when using the modalities separately.

Clinical trial registration no.: NCT01535430 (clinicaltrials.gov)

ABBREVIATIONS BVS = blood vessel segmentation; DCS = direct cortical stimulation; fMRI = functional MRI; MEG = magnetoencephalography; T1W = T1-weighted.

Noninvasive eloquent function mapping is often used prior to brain tumor resections in order to avoid disrupting brain functions critical for patients’ well-being. The most prevalent form of noninvasive mapping is functional MRI (fMRI), but despite its utilization among surgeons, the reliability of fMRI is unestablished and varies widely in published reports. By using intraoperative direct cortical stimulation (DCS) as the gold standard, fMRI language mapping has been reported to have sensitivities ranging from 5%8 to 100%2 and specificities ranging from 0%16 to 98%,10 depending on the published article.1,4,12,14,17,20,23,24

One of the limitations of previous studies that have quantified fMRI accuracy with DCS is that the studies have used arbitrary distance thresholds to determine accuracy.1,8,10,14,18 Namely, if a DCS location is within a set threshold (usually 10 mm) of an fMRI activation cluster, then that DCS location is said to be a positively detected location. While choosing such a distance threshold is necessary to establish accuracy metrics, having a set distance threshold leaves out important information as to how the accuracy changes when varying distances are considered. By looking at how accuracy metrics vary according to changes in the distance threshold used, the present study aimed to elucidate how preoperative imaging may be used in such a way that maximizes the mapping accuracy. In addition to optimizing the distance threshold, the present study will look at the general accuracy of fMRI and magnetoencephalography (MEG) compared to each other, as well as the accuracy of fMRI and MEG when used in combination.

Methods

This study is part of an ongoing clinical trial (registration no. NCT01535430; clinicaltrials.gov).

Patient Selection

Patients presenting with a brain tumor in or near an eloquent area (sensory, motor, language) planning to undergo resection were referred preoperatively for fMRI and MEG as part of their routine preoperative evaluation. Clinical evaluation and testing, in conjunction with neuroimaging and patients’ wishes, determined the operative plan. Appropriate candidates were offered resection with intraoperative mapping while either asleep (motor) or awake (language and/or motor). If they had undergone preoperative fMRI and MEG and intraoperative functional mapping, they provided consent for this IRB-approved study.

MEG Data Acquisition

All MEG data were acquired prior to surgery using an Elekta Neuromag system, which consists of 102 magnetometers and 204 planar radiometers (306 total magnetic sensors). The data were acquired within a Lindgren magnetically shielded room, composed of aluminum and μ-metal layers, that is equipped with an active-shielding component for advanced environmental noise compensation. During data acquisition, participants were monitored via real-time audio and video feeds from inside the magnetically shielded room. Neuromagnetic responses were sampled continuously at 1 kHz with an acquisition bandwidth of 0.1–330 Hz.

MEG motor tasks consisted of finger tapping, foot movement, and face movement (face “squenching”) and were all performed contralateral to the lesion. Motor tasks were paced utilizing a beep that occurs every 3.5–4.0 seconds. For each body part, the participant completes about 120 movements, and an accelerometer is used to quantify each movement onset. The MEG language task consisted of word reading at a rate of one word every 3.0 seconds.

MEG Source Analyses

In order to map the MEG source data, a dipole method of analysis was performed. As the first step in the analysis, artifact rejection was performed based on a fixed threshold method, supplemented with visual inspection. After the artifact-free epochs from each limb per modality were time-domain averaged, baseline corrected, and filtered, source localization was performed using the time period around the signal maximum. Essentially, continuous flux contour maps for each extremity were examined for temporal periods containing strictly dipolar field distributions above or near the rolandic regions of the head. These temporal periods were then subjected to single-moving, equivalent-current dipole analyses that employ the Nelder-Mead nonlinear search algorithm to obtain the optimal dipole position and orientation parameters per data-point (i.e., millisecond) in the least-squares sense. All dipole analyses utilized a spherically symmetric single-conductor head model that had been co-registered to the MEG data and the individual subject’s MRI data. Only equivalent-current dipoles that account for at least 90% of the variance in the filtered MEG data and had 95% confidence volumes of less than 3 cm were accepted into the final model.

MRI Data Acquisition

All MR images were acquired prior to surgery using a Philips 3-T scanner. The fMRI sequences were a single-shot, fast field echo technique (TR 3000 msec; TE 35 msec; echo planar imaging (EPI) bandwidth 3693 Hz; acquired voxel 3 × 3 mm; reconstructed voxel 1.67 × 1.67 mm; field of view 240 × 240 mm; section thickness 3 mm; section gap 0 mm; signal averaging–number of acquisitions; sensitivity encoding factor 1.8; 44 slices). Computer-driven fMRI directions and stimuli are presented utilizing visual and auditory systems. The fMRI examinations consisted of acquiring two dummy scans before obtaining 100 scans utilizing the block-design approach of motor and language paradigms.1,3,13,17,22 These consisted of using blocks of 5 brain volumes (15 seconds) in the rest mode and 5 brain volumes (15 seconds) in the functional mode repeated 10 times for a total of 100 brain volumes. A camera system was used to confirm subject performance or attention to task. In addition, T1-weighted (T1W) volumes were collected with and without contrast medium.

fMRI motor mapping consisted of a combination of finger tapping, foot movement, and face movement (lip pursing) paced utilizing a beep that occurs once per second. The finger tapping and foot movement tasks are performed on the side of the body contralateral to the lesion. The fMRI language task consisted of word mapping where words were displayed and silently read by the patient.

MRI Data Processing

fMRI data were processed separately for each task using Invivo DynaSuite Neuro (Philips). The statistical map for each individual subject and task was thresholded by eye to identify responses that were as follows: 1) statistically robust, 2) localized to the expected region, and 3) accounted for idiosyncratic changes to brain tissue distorted by nearby tumor. This mapping was performed by a trained analyst, and the final activation maps were reviewed, edited, and approved by a neuroradiologist (M.L.W.). In order to create a visualization for reviewing intraoperative data, the cortical surfaces were reconstructed from the T1W volumes with the FreeSurfer reconstruction workflow. Inaccuracies of the cortical surface reconstruction were identified with manual review and then fixed by manually segmenting areas missing in the reconstruction and rerunning FreeSurfer. All of the other structural data, along with the functional activation masks, were then registered to the T1W volume used to reconstruct the cortical surface. A reconstruction of the blood vessel segmentation was acquired by subtracting the T1W scan from the T1W scan that was acquired after a contrast agent had been administered. A threshold was then applied to the resulting image to isolate the voxels where the contrast agent increased the signal response the most. A visualization with the cortical blood vessels, the cortical surface, and the DCS functional activations was then created using 3DSlicer5 software (see Fig. 1C).

FIG. 1.
FIG. 1.

A: Axial scan showing a nonenhancing tumor in the left frontal operculum, raising concern for motor and language function. B: Intraoperative DCS mapping findings for speech hesitation (Sp), speech arrest (SpA), and motor tongue (Mf). C: A 3D rendering of the surgical area showing the BVS and intraoperative DCS locations. The blue spheres represent positive motor face DCS mapping, and the green spheres represent positive language DCS mapping.

Intraoperative DCS Mapping and Data Collection

The procedures were performed by one surgeon (M.R.A.). The asleep-awake mapping technique was utilized for awake mapping. Briefly, the patient is sedated and has a laryngeal mask airway placed for the pinning, positioning, and registration to the navigation system. The patient is then awakened prior to the start of the procedure, and the laryngeal mask airway is removed. We have found that this protocol reduces the amount of potential brain shift after opening, as the changes that occur during the wake-up process when the dura is opened are avoided. For asleep motor mapping, general anesthesia without paralytics was employed with somatosensory evoked potentials for phase reversal and DCS with electromyography.

Asleep mapping was performed using electromyography with bipolar stimulation from the Ojemann Cortical Stimulator (Integra LifeSciences) from 2 to 14 mA in 1- to 2-mA increments and biphasic square wave pulses of 1 msec at 60 Hz of a 4-second maximum duration. For awake mapping, stimulation was from 1 to 10 mA in 0.5- to 1-mA increments. For motor mapping, the surgeon applied stimulation at specific sites in the brain and then observed any involuntary movements. Language mapping consisted of number counting, naming months or days of week, picture naming, word cards, word mapping, spontaneous speech with storytelling, and directed questions. The positive sites were labeled, and the coordinates of those locations were acquired and stored using the Brainlab Curve navigation system. Sites that tested negative for eloquent function were also recorded. After all the exposed area of the brain had been mapped and labeled, an intraoperative photograph of the mapped labeled locations on the cortical surface was obtained, which illustrated the cortical venous pattern and labeled positive and negative sites for reference during co-registration (Fig. 1B).

Verifying DCS Locations

In order to account for any brain shift, we acquired intraoperative photographs that visualized the mapped eloquent points in relation to the cortical surface vessels, which serve as static landmarks. The vessels are segmented out and visualized along with the reconstructed cortical surface. The blood vessel segmentation (BVS) is unaffected by brain shift and was used to verify the accuracy of the transformation. The intraoperative photograph provided a means of verifying that the registration of brain, intraoperative points, and BVS were accurate after reconstruction.

First, the functional DCS locations recorded on the surface of the brain were registered onto the reconstructed cortical surface using a rigid iterative closest-point registration. The registered DCS locations were then visualized in a 3D rendering along with the BVS and cortical surface. Finally, the picture containing the labeled DCS locations and intraoperative cortical vessel landmarks (Fig. 1B) were compared to those shown in the visualization (Fig. 1C), and the DCS locations were manually corrected as necessary to account for any shift. In some patients, the negative mapping locations were not recorded intraoperatively. In these cases, the negative locations were recorded after the surgery by the surgeon using the visualization software.

Accuracy Analysis

In order to measure the accuracy of the MEG and fMRI data to preoperatively detect eloquent function, activations were grouped into motor- or speech-related functions. For each neuroimaging modality, each intraoperatively mapped location, with or without eloquent function, was evaluated. If an intraoperative location had a neuroimaging activation within x millimeters, then that location was labeled as a positive location for that mapping modality. If no activation was found within x millimeters, then the location was labeled as a negative location for that particular eloquent function. Figure 2 shows an example of a language mapping case, with the distances to the MEG and fMRI clusters shown in color on the cortical surface.

FIG. 2.
FIG. 2.

Examples for MEG (A and C) and fMRI (B and D) clusters for the word mapping language task. Intraoperative DCS locations are shown on the surface in red for speech-positive DCS locations and gray for speech-negative locations. MEG (A) and fMRI (B) activation clusters as 3D renderings along with the cortical surface. MEG (C) and fMRI (D) distances from the activation clusters to the cortical surface. For this patient, MEG mapping has much longer distances to the majority of DCS speech locations than fMRI mapping.

In order to explore the influence of a designated distance used for accuracy analysis, we varied the distance from the DCS points. By linearly sampling for x from 0 to 70 mm at 5-mm intervals, a range of sensitivity, specificity, and Youden’s J statistic values were calculated from the resulting detections. Youden’s J statistic (J = sensitivity + specificity − 1) combines sensitivity and specificity to measure the overall performance of a given test at each sampled distance.25 A mapping that provides no improvement over random chance has a J statistic value of 0, while a perfect mapping with no false-positive or false-negative errors has a J statistic value of 1. Accuracy analysis with the J statistic provides an informative summary of the performance that can then be used to specify at what distance the maximum sensitivity and specificity can be obtained.

Results

Mapping Outcomes

The 19 patients included in this study had a variety of primary or metastatic brain tumors. There was also one patient with a cavernoma. All 19 patients underwent preoperative mapping with both fMRI and MEG. For motor mapping, 14 patients were mapped doing comparable fMRI and MEG motor mapping tasks (Table 1). Eleven of those patients had motor hand mapping, 6 had motor face mapping, and 3 had motor foot mapping. In these 14 patients with preoperative motor mapping, 175 DCS locations were mapped intraoperatively, with 29 of those locations being identified as motor locations according to DCS. For language mapping, 12 patients were mapped doing comparable word mapping fMRI and MEG tasks. In these 12 patients, there were 117 locations mapped intraoperatively with DCS, and 16 of those locations were determined to be eloquent areas for speech (Table 1). Seven patients had both motor and language mapping.

TABLE 1.

Summary of collected data

CategoryPatientsPositive LocationsNegative LocationsTotal Locations
Motor1429146175
Language1216101117
Total1945168213
The “Category” column indicates the category of paradigm employed for the preoperative imaging data. The “Patients” column refers to the number of individuals who had that specific paradigm mapped prior to surgery. “Positive Locations” refers to the total number of locations that were positively mapped using DCS for the same category of function (language or motor) tested for by the imaging paradigm. “Negative Locations” refers to the total number of locations that were tested for by DCS but did not detect function corresponding to the category of function being tested for by the imaging paradigm. “Total Locations” refers to the total number of DCS mapped locations that were acquired in the patients who had been mapped with the corresponding paradigm. “Total Locations” is the sum of both positive and negative locations.

Motor Mapping

When analyzed using a linearly increasing distance threshold with 5-mm intervals, fMRI achieved a maximum J statistic of 0.21 at a distance of 5 mm with a sensitivity of 0.34 and a specificity of 0.87 (Fig. 3). MEG achieved a maximum J statistic of 0.28 at 40 mm with a sensitivity of 0.86 and a specificity of 0.42 (Fig. 3). When combining the fMRI and MEG, a maximum J statistic of 0.35 (sensitivity of 0.55, specificity of 0.79) was achieved using a 5-mm distance threshold for fMRI and a 15-mm distance threshold for MEG (Fig. 4). This J statistic of the combined detection with optimized distance thresholds was significantly greater than the maximum J statistic using the arbitrary 10-mm thresholds (p < 0.05) and the maximum J statistic of fMRI and MEG at 5-mm and 15-mm thresholds (p < 0.05). However, it was not greater than the maximum J statistics of fMRI and MEG using the individually optimized thresholds of 5 mm and 40 mm (p = 0.27).

FIG. 3.
FIG. 3.

A–D: Sensitivity, specificity, and J statistic (sensitivity + specificity − 1) with respect to the distance for MEG and fMRI language and motor. The shaded areas indicate 95% confidence intervals. The distance resulting in the maximum J statistic is annotated and marked with the vertical gray dashed line in each subplot.

FIG. 4.
FIG. 4.

J statistic results of the combined MEG and fMRI detection with respect to the analysis of distance thresholds for both motor (A) and language (B). The location of the maximum J statistic is shown by the red dot to be 0.35 with the MEG distance set at 15 mm and the fMRI distance set at 5 mm for motor (A). Also shown by the red dot, the maximum J statistic was 0.40 at a 10-mm distance for both fMRI and MEG language mapping (B).

Using the arbitrary distance threshold of 10 mm from the previous literature,1,8,10,14,18 MEG motor mapping resulted in a specificity of 0.97 and a sensitivity of 0.14. fMRI motor mapping resulted in a lower specificity of 0.68 (p < 0.0001) but a higher sensitivity of 0.48 (p < 0.05). Overall, the J statistic was 0.11 for MEG and 0.16 for fMRI. Combining the two modalities resulted in a specificity of 0.67 and a sensitivity of 0.62 (Table 2). As shown in Fig. 5, the combined sensitivity was greater than that of MEG (p < 0.001) but not that of fMRI (p = 0.12). The combined specificity was less than that of MEG (p < 0.0001) but not different from that of fMRI (p = 1.00). This resulted in a J statistic of 0.29, which was not significantly different from that of the maximum of MEG and fMRI (p = 0.07). Overall, of the 29 DCS mapped motor locations, 14 were identified by fMRI and 4 were identified by MEG. None of the DCS motor locations identified by MEG were identified by fMRI, and thus the MEG and fMRI combination identified 18 of 29 motor locations.

TABLE 2.

Summary of motor mapping results

ModalityDistanceSensitivitySpecificityJ Statistic
MEG10 mm0.140.970.11
fMRI10 mm0.480.680.16
fMRI & MEG10 mm & 10 mm0.620.670.29
MEG40 mm0.860.420.28
fMRI5 mm0.340.870.21
fMRI & MEG5 mm & 15 mm0.550.790.35
Summary of motor mapping results at the arbitrary 10-mm distance used in previous studies for MEG individually and combined, the optimal individual distances (40 mm for MEG and 5 mm for fMRI), and the optimal combined distance (15 mm for MEG and 5 mm for fMRI). Combining MEG and fMRI at the optimal combined distances resulted in the highest J statistic (sensitivity + specificity − 1). Maximum scores are shown in boldface.
FIG. 5.
FIG. 5.

Summary bar chart of the fMRI and MEG motor (A) and language (B) accuracy analysis. The specificity, sensitivity, and J statistic were taken using a 10-mm distance and the distances that maximized the individual and combined J statistics. The MEG and fMRI detections were combined such that when either MEG or fMRI resulted in a positive detection, the combined detection was considered to be positive (an “either positive” approach). The significance is annotated for comparing the results at the 10-mm distance. *p < 0.05; **p < 0.001; ***p < 0.0001.

Language Mapping

When analyzed using a linearly increasing distance threshold with 5-mm intervals, fMRI achieved a maximum J statistic of 0.28 at a 5-mm distance with a sensitivity of 0.31 and specificity of 0.97 (Fig. 3). MEG achieved a maximum J statistic of 0.28 at 15 mm with a sensitivity of 0.56 and a specificity of 0.71 (Fig. 3). When combining the fMRI and MEG, a maximum J statistic of 0.40 (sensitivity of 0.55, specificity of 0.79) was achieved using a 10-mm distance threshold for both fMRI and MEG (Fig. 4). The J statistic of the combined detection with optimized distance thresholds was not significantly greater than the maximum J statistic using the arbitrary 10-mm thresholds (p = 0.10) and was also not significantly greater than the maximum J statistic of the individually optimized thresholds for fMRI and MEG of 5 mm and 15 mm (p = 0.14).

Using the arbitrary distance threshold of 10 mm from the previous literature,1,8,10,14,18 MEG language mapping resulted in a specificity of 0.90 and a sensitivity of 0.25. fMRI language mapping resulted in a lower specificity of 0.77 (p < 0.05) and a sensitivity of 0.50, which was not significantly different from that of MEG (p = 0.34). Overall, the J statistic was 0.15 for MEG and 0.27 for fMRI. Combining the two modalities resulted in a specificity of 0.71 and a sensitivity of 0.69 (Table 3). As shown in Fig. 5, the combined sensitivity was greater than that of MEG (p < 0.05) but not that of fMRI (p = 0.25). The combined specificity was less than that of MEG (p < 0.0001) and fMRI (p < 0.05). This resulted in a J statistic of 0.40, which was not significantly different than the maximum of MEG and fMRI (p = 0.10). Overall, of the 16 DCS mapped language locations, 8 were identified by fMRI and 4 were identified by MEG. Only one of the DCS language locations identified by MEG was identified by fMRI, and thus the MEG and fMRI combination identified 11 of 16 language locations.

TABLE 3.

Summary of language mapping results

ModalityDistanceSensitivitySpecificityJ Statistic
MEG10 mm0.250.900.15
fMRI10 mm0.500.770.27
fMRI & MEG10 mm & 10 mm0.690.710.40
MEG15 mm0.560.710.28
fMRI5 mm0.310.970.28
Summary of language mapping results at the arbitrary 10-mm distance used in previous studies for MEG individually and combined and the optimal individual distances (15 mm for MEG and 5 mm for fMRI). The optimal combined distance for language mapping was 10 mm for both fMRI and MEG. Combining MEG and fMRI at the 10-mm distance resulted in the highest J statistic. Maximum scores are shown in boldface.

Discussion

Our study shows that the accuracy of language and motor mapping for both fMRI and MEG is heavily dependent on the distance threshold used in the analysis. Using shorter thresholds results in fewer noneloquent locations being incorrectly detected as locations containing eloquent function and thus greater specificity. However, shorter distance thresholds also result in the correct detection of fewer eloquent locations and thus lower sensitivity. Not surprisingly, increasing the distance threshold from the eloquent clusters increases the sensitivity but decreases the specificity. In this study, the distance that resulted in the highest combinations of sensitivity and specificity (J statistic) was achieved for both language and motor fMRI at a 5-mm distance. In contrast, maximum accuracy for MEG was achieved at 40 mm for motor and 15 mm for language mapping. The larger optimal distances for MEG mapping are likely the result of a highly specific initial mapping that benefits from the increased sensitivity provided by the larger distance thresholds.

Combining the fMRI and MEG modalities showed promise for improving the overall mapping accuracy. By combining the modalities and optimizing the distance threshold, the combined fMRI and MEG motor mapping had a higher overall accuracy than the individual mapping modalities when using the 10-mm distance threshold reported in the literature. Furthermore, of the 8 motor and language locations accurately identified by MEG at 10 mm, only one of those locations was also identified by fMRI, despite fMRI having higher or equal sensitivity. This lack of overlap in eloquent locations may indicate that due to their respective imaging and mapping methods, MEG and fMRI are providing complementary rather than redundant mapping information, as has been postulated in previous research.21 However, more data are needed to confirm this hypothesis.

It is important to note that the distance a surgeon chooses to evaluate for eloquent function location when reviewing preoperative mapping may not be the distance that gives the highest accuracy. Instead, it is suggested a surgeon consider the distance that will give the closest to ideal combination of sensitivity and specificity according to their expertise and the individual case. Furthermore, the optimal distances obtained in this study may not apply to mapping data acquired with different protocols and scanners. Previous research has shown that language fMRI mapping, in particular, can vary significantly due to varied statistical thresholds, imaging times, and specific language task being performed.23 Nevertheless, the present study does provide valuable information as how the distance used has a profound impact on the accuracy measures that surgeons may take into account even without assuming the uniformity of their data to that of the present study.

Also of value is that this is the first study to report the accuracy of MEG mapping in terms of sensitivity and specificity as determined by DCS. While several studies had empirically validated MEG localization using various methods,6,7,11,15,19 our literature search revealed no publications reporting detailed sensitivity and specificity analysis for either MEG motor or language localization using DCS as the gold standard. For the sake of comparison with the fMRI accuracy literature, we reported accuracy metrics for both MEG and fMRI using the arbitrary 10-mm threshold that has been used previously.1,8,10,14,18 At this distance, MEG mapping showed decreased sensitivity when compared to fMRI for motor mapping but increased specificity for both motor and language mapping.

According to a meta-analysis by Weng et al., the sensitivity for fMRI language mapping is estimated to be 51%–80% with a specificity of 25%–82%.23 Compared to the results of the fMRI meta-analysis, the fMRI language mapping in the present study had a similar sensitivity (50%) and specificity (77%), while the MEG language mapping had a lower sensitivity (25%) but a higher specificity (90%). However, the fMRI motor mapping accuracy, with a sensitivity of 48% and specificity of 67%, was lower than that reported in previous studies with sensitivities ranging from 88%1 to 100%10 and specificities ranging from 66%10 to 95%.8,9 The poor accuracy of fMRI motor mapping may be the result of bias that was inherent in the data collection, as presurgical motor mapping was only performed in patients presenting with tumors near the known anatomical sensory and motor locations. It is possible that the tumors distorted anatomy to the extent that the mapping was more complex than would be expected in patients with less distorted anatomy.

Due to the dependency of accuracy metrics on various analysis parameters, including the analysis distance, as was demonstrated, future studies should be careful to note that any resulting accuracy scores are not set in stone but, rather, are highly dependent on the analysis parameters utilized to achieve those scores. Future studies should also be careful to report both the sensitivity and the specificity because there is an inherent trade-off between these measures that is often the result of the given study’s parameters. Reporting sensitivity without specificity gives very limited information about the accuracy of a method as compared to other methods.

Conclusions

This study demonstrates that the accuracy of language and motor mapping with both fMRI and MEG is heavily dependent on the distance threshold used in the analysis. Furthermore, combining MEG and fMRI shows the potential for increased motor mapping accuracy over using the modalities separately.

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Aizenberg. Acquisition of data: Aizenberg, White, Wilson. Analysis and interpretation of data: Aizenberg, Ellis, White, Warren, Wilson. Drafting the article: Aizenberg, Ellis. Critically revising the article: Aizenberg, Ellis. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Aizenberg. Statistical analysis: Ellis, Hayasaka. Administrative/technical/material support: Ellis. Study supervision: Aizenberg.

Supplemental Information

Previous Presentations

Portions of this work were presented in abstract form at the Radiological Society of North America 105th Scientific Assembly and Annual Meeting, December 1–6, 2019, Chicago, Illinois. Portions of this work were presented as an oral abstract presentation at the Midwest Neurosurgical Society Annual Meeting, August 17, 2019, Kansas City, Missouri.

References

  • 1

    Bizzi ABlasi VFalini AFerroli PCadioli MDanesi U: Presurgical functional MR imaging of language and motor functions: validation with intraoperative electrocortical mapping. Radiology 248:5795892008

    • Search Google Scholar
    • Export Citation
  • 2

    FitzGerald DBCosgrove GRRonner SJiang HBuchbinder BRBelliveau JW: Location of language in the cortex: a comparison between functional MR imaging and electrocortical stimulation. AJNR Am J Neuroradiol 18:152915391997

    • Search Google Scholar
    • Export Citation
  • 3

    Giussani CRoux FEOjemann JSganzerla EPPirillo DPapagno C: Is preoperative functional magnetic resonance imaging reliable for language areas mapping in brain tumor surgery? Review of language functional magnetic resonance imaging and direct cortical stimulation correlation studies. Neurosurgery 66:1131202010

    • Search Google Scholar
    • Export Citation
  • 4

    Ille SSollmann NHauck TMaurer STanigawa NObermueller T: Impairment of preoperative language mapping by lesion location: a functional magnetic resonance imaging, navigated transcranial magnetic stimulation, and direct cortical stimulation study. J Neurosurg 123:3143242015

    • Search Google Scholar
    • Export Citation
  • 5

    Kikinis RPieper SDVosburgh KG: 3D Slicer: a platform for subject-specific image analysis, visualization, and clinical support in Jolesz FA (ed): Intraoperative Imaging and Image-Guided Therapy. New York: Springer New York2014 pp 277289

    • Search Google Scholar
    • Export Citation
  • 6

    Kirsch HEZhu ZHonma SFindlay ABerger MSNagarajan SS: Predicting the location of mouth motor cortex in patients with brain tumors by using somatosensory evoked field measurements. J Neurosurg 107:4814872007

    • Search Google Scholar
    • Export Citation
  • 7

    Korvenoja AKirveskari EAronen HJAvikainen SBrander AHuttunen J: Sensorimotor cortex localization: comparison of magnetoencephalography, functional MR imaging, and intraoperative cortical mapping. Radiology 241:2132222006

    • Search Google Scholar
    • Export Citation
  • 8

    Kuchcinski GMellerio CPallud JDezamis ETurc GRigaux-Viodé O: Three-tesla functional MR language mapping: comparison with direct cortical stimulation in gliomas. Neurology 84:5605682015

    • Search Google Scholar
    • Export Citation
  • 9

    Lehéricy SDuffau HCornu PCapelle LPidoux BCarpentier A: Correspondence between functional magnetic resonance imaging somatotopy and individual brain anatomy of the central region: comparison with intraoperative stimulation in patients with brain tumors. J Neurosurg 92:5895982000

    • Search Google Scholar
    • Export Citation
  • 10

    Meier MPIlmberger JFesl GRuge MI: Validation of functional motor and language MRI with direct cortical stimulation. Acta Neurochir (Wien) 155:6756832013

    • Search Google Scholar
    • Export Citation
  • 11

    Nagarajan SKirsch HLin PFindlay AHonma SBerger MS: Preoperative localization of hand motor cortex by adaptive spatial filtering of magnetoencephalography data. J Neurosurg 109:2282372008

    • Search Google Scholar
    • Export Citation
  • 12

    Pernet CRGorgolewski KJJob DRodriguez DStorkey AWhittle I: Evaluation of a pre-surgical functional MRI workflow: from data acquisition to reporting. Int J Med Inform 86:37–422016

    • Search Google Scholar
    • Export Citation
  • 13

    Pillai JJZaca D: Relative utility for hemispheric lateralization of different clinical fMRI activation tasks within a comprehensive language paradigm battery in brain tumor patients as assessed by both threshold-dependent and threshold-independent analysis methods. Neuroimage 54 (Suppl 1):S136S1452011

    • Search Google Scholar
    • Export Citation
  • 14

    Roux FEBoulanouar KLotterie JAMejdoubi MLeSage JPBerry I: Language functional magnetic resonance imaging in preoperative assessment of language areas: correlation with direct cortical stimulation. Neurosurgery 52:133513472003

    • Search Google Scholar
    • Export Citation
  • 15

    Schiffbauer HBerger MSFerrari PFreudenstein DRowley HARoberts TPL: Preoperative magnetic source imaging for brain tumor surgery: a quantitative comparison with intraoperative sensory and motor mapping. J Neurosurg 97:133313422002

    • Search Google Scholar
    • Export Citation
  • 16

    Schlosser MJLuby MSpencer DDAwad IAMcCarthy G: Comparative localization of auditory comprehension by using functional magnetic resonance imaging and cortical stimulation. J Neurosurg 91:6266351999

    • Search Google Scholar
    • Export Citation
  • 17

    Spena GNava ACassini FPepoli ABruno MD’Agata F: Preoperative and intraoperative brain mapping for the resection of eloquent-area tumors. A prospective analysis of methodology, correlation, and usefulness based on clinical outcomes. Acta Neurochir (Wien) 152:183518462010

    • Search Google Scholar
    • Export Citation
  • 18

    Stevens MTClarke DBStroink GBeyea SDD’Arcy RC: Improving fMRI reliability in presurgical mapping for brain tumours. J Neurol Neurosurg Psychiatry 87:2672742016

    • Search Google Scholar
    • Export Citation
  • 19

    Tarapore PETate MCFindlay AMHonma SMMizuiri DBerger MS: Preoperative multimodal motor mapping: a comparison of magnetoencephalography imaging, navigated transcranial magnetic stimulation, and direct cortical stimulation. J Neurosurg 117:3543622012

    • Search Google Scholar
    • Export Citation
  • 20

    Trinh VTFahim DKMaldaun MVCShah KMcCutcheon IERao G: Impact of preoperative functional magnetic resonance imaging during awake craniotomy procedures for intraoperative guidance and complication avoidance. Stereotact Funct Neurosurg 92:3153222014

    • Search Google Scholar
    • Export Citation
  • 21

    Tuunanen PIKavec MJousmäki VUsenius JPHari RSalmelin R: Comparison of BOLD fMRI and MEG characteristics to vibrotactile stimulation. Neuroimage 19:177817862003

    • Search Google Scholar
    • Export Citation
  • 22

    Tyndall AJReinhardt JTronnier VMariani LStippich C: Presurgical motor, somatosensory and language fMRI: technical feasibility and limitations in 491 patients over 13 years. Eur Radiol 27:2672782017

    • Search Google Scholar
    • Export Citation
  • 23

    Weng HHNoll KRJohnson JMPrabhu SSTsai YHChang SW: Accuracy of presurgical functional MR imaging for language mapping of brain tumors: a systematic review and meta-analysis. Radiology 286:5125232018

    • Search Google Scholar
    • Export Citation
  • 24

    Xia HHuang WWu LMa HWang XChen X: Preoperative functional MRI localization of language areas in Chinese patients with brain tumors: validation with intraoperative electrocortical mapping. Neural Regen Res 7:156315692012

    • Search Google Scholar
    • Export Citation
  • 25

    Youden WJ: Index for rating diagnostic tests. Cancer 3:32351950

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

Contributor Notes

Correspondence Michele R. Aizenberg: University of Nebraska Medical Center, Omaha, NE. maizenberg@unmc.edu.INCLUDE WHEN CITING DOI: 10.3171/2019.11.FOCUS19784.Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.
Headings
Figures
  • View in gallery

    A: Axial scan showing a nonenhancing tumor in the left frontal operculum, raising concern for motor and language function. B: Intraoperative DCS mapping findings for speech hesitation (Sp), speech arrest (SpA), and motor tongue (Mf). C: A 3D rendering of the surgical area showing the BVS and intraoperative DCS locations. The blue spheres represent positive motor face DCS mapping, and the green spheres represent positive language DCS mapping.

  • View in gallery

    Examples for MEG (A and C) and fMRI (B and D) clusters for the word mapping language task. Intraoperative DCS locations are shown on the surface in red for speech-positive DCS locations and gray for speech-negative locations. MEG (A) and fMRI (B) activation clusters as 3D renderings along with the cortical surface. MEG (C) and fMRI (D) distances from the activation clusters to the cortical surface. For this patient, MEG mapping has much longer distances to the majority of DCS speech locations than fMRI mapping.

  • View in gallery

    A–D: Sensitivity, specificity, and J statistic (sensitivity + specificity − 1) with respect to the distance for MEG and fMRI language and motor. The shaded areas indicate 95% confidence intervals. The distance resulting in the maximum J statistic is annotated and marked with the vertical gray dashed line in each subplot.

  • View in gallery

    J statistic results of the combined MEG and fMRI detection with respect to the analysis of distance thresholds for both motor (A) and language (B). The location of the maximum J statistic is shown by the red dot to be 0.35 with the MEG distance set at 15 mm and the fMRI distance set at 5 mm for motor (A). Also shown by the red dot, the maximum J statistic was 0.40 at a 10-mm distance for both fMRI and MEG language mapping (B).

  • View in gallery

    Summary bar chart of the fMRI and MEG motor (A) and language (B) accuracy analysis. The specificity, sensitivity, and J statistic were taken using a 10-mm distance and the distances that maximized the individual and combined J statistics. The MEG and fMRI detections were combined such that when either MEG or fMRI resulted in a positive detection, the combined detection was considered to be positive (an “either positive” approach). The significance is annotated for comparing the results at the 10-mm distance. *p < 0.05; **p < 0.001; ***p < 0.0001.

References
  • 1

    Bizzi ABlasi VFalini AFerroli PCadioli MDanesi U: Presurgical functional MR imaging of language and motor functions: validation with intraoperative electrocortical mapping. Radiology 248:5795892008

    • Search Google Scholar
    • Export Citation
  • 2

    FitzGerald DBCosgrove GRRonner SJiang HBuchbinder BRBelliveau JW: Location of language in the cortex: a comparison between functional MR imaging and electrocortical stimulation. AJNR Am J Neuroradiol 18:152915391997

    • Search Google Scholar
    • Export Citation
  • 3

    Giussani CRoux FEOjemann JSganzerla EPPirillo DPapagno C: Is preoperative functional magnetic resonance imaging reliable for language areas mapping in brain tumor surgery? Review of language functional magnetic resonance imaging and direct cortical stimulation correlation studies. Neurosurgery 66:1131202010

    • Search Google Scholar
    • Export Citation
  • 4

    Ille SSollmann NHauck TMaurer STanigawa NObermueller T: Impairment of preoperative language mapping by lesion location: a functional magnetic resonance imaging, navigated transcranial magnetic stimulation, and direct cortical stimulation study. J Neurosurg 123:3143242015

    • Search Google Scholar
    • Export Citation
  • 5

    Kikinis RPieper SDVosburgh KG: 3D Slicer: a platform for subject-specific image analysis, visualization, and clinical support in Jolesz FA (ed): Intraoperative Imaging and Image-Guided Therapy. New York: Springer New York2014 pp 277289

    • Search Google Scholar
    • Export Citation
  • 6

    Kirsch HEZhu ZHonma SFindlay ABerger MSNagarajan SS: Predicting the location of mouth motor cortex in patients with brain tumors by using somatosensory evoked field measurements. J Neurosurg 107:4814872007

    • Search Google Scholar
    • Export Citation
  • 7

    Korvenoja AKirveskari EAronen HJAvikainen SBrander AHuttunen J: Sensorimotor cortex localization: comparison of magnetoencephalography, functional MR imaging, and intraoperative cortical mapping. Radiology 241:2132222006

    • Search Google Scholar
    • Export Citation
  • 8

    Kuchcinski GMellerio CPallud JDezamis ETurc GRigaux-Viodé O: Three-tesla functional MR language mapping: comparison with direct cortical stimulation in gliomas. Neurology 84:5605682015

    • Search Google Scholar
    • Export Citation
  • 9

    Lehéricy SDuffau HCornu PCapelle LPidoux BCarpentier A: Correspondence between functional magnetic resonance imaging somatotopy and individual brain anatomy of the central region: comparison with intraoperative stimulation in patients with brain tumors. J Neurosurg 92:5895982000

    • Search Google Scholar
    • Export Citation
  • 10

    Meier MPIlmberger JFesl GRuge MI: Validation of functional motor and language MRI with direct cortical stimulation. Acta Neurochir (Wien) 155:6756832013

    • Search Google Scholar
    • Export Citation
  • 11

    Nagarajan SKirsch HLin PFindlay AHonma SBerger MS: Preoperative localization of hand motor cortex by adaptive spatial filtering of magnetoencephalography data. J Neurosurg 109:2282372008

    • Search Google Scholar
    • Export Citation
  • 12

    Pernet CRGorgolewski KJJob DRodriguez DStorkey AWhittle I: Evaluation of a pre-surgical functional MRI workflow: from data acquisition to reporting. Int J Med Inform 86:37–422016

    • Search Google Scholar
    • Export Citation
  • 13

    Pillai JJZaca D: Relative utility for hemispheric lateralization of different clinical fMRI activation tasks within a comprehensive language paradigm battery in brain tumor patients as assessed by both threshold-dependent and threshold-independent analysis methods. Neuroimage 54 (Suppl 1):S136S1452011

    • Search Google Scholar
    • Export Citation
  • 14

    Roux FEBoulanouar KLotterie JAMejdoubi MLeSage JPBerry I: Language functional magnetic resonance imaging in preoperative assessment of language areas: correlation with direct cortical stimulation. Neurosurgery 52:133513472003

    • Search Google Scholar
    • Export Citation
  • 15

    Schiffbauer HBerger MSFerrari PFreudenstein DRowley HARoberts TPL: Preoperative magnetic source imaging for brain tumor surgery: a quantitative comparison with intraoperative sensory and motor mapping. J Neurosurg 97:133313422002

    • Search Google Scholar
    • Export Citation
  • 16

    Schlosser MJLuby MSpencer DDAwad IAMcCarthy G: Comparative localization of auditory comprehension by using functional magnetic resonance imaging and cortical stimulation. J Neurosurg 91:6266351999

    • Search Google Scholar
    • Export Citation
  • 17

    Spena GNava ACassini FPepoli ABruno MD’Agata F: Preoperative and intraoperative brain mapping for the resection of eloquent-area tumors. A prospective analysis of methodology, correlation, and usefulness based on clinical outcomes. Acta Neurochir (Wien) 152:183518462010

    • Search Google Scholar
    • Export Citation
  • 18

    Stevens MTClarke DBStroink GBeyea SDD’Arcy RC: Improving fMRI reliability in presurgical mapping for brain tumours. J Neurol Neurosurg Psychiatry 87:2672742016

    • Search Google Scholar
    • Export Citation
  • 19

    Tarapore PETate MCFindlay AMHonma SMMizuiri DBerger MS: Preoperative multimodal motor mapping: a comparison of magnetoencephalography imaging, navigated transcranial magnetic stimulation, and direct cortical stimulation. J Neurosurg 117:3543622012

    • Search Google Scholar
    • Export Citation
  • 20

    Trinh VTFahim DKMaldaun MVCShah KMcCutcheon IERao G: Impact of preoperative functional magnetic resonance imaging during awake craniotomy procedures for intraoperative guidance and complication avoidance. Stereotact Funct Neurosurg 92:3153222014

    • Search Google Scholar
    • Export Citation
  • 21

    Tuunanen PIKavec MJousmäki VUsenius JPHari RSalmelin R: Comparison of BOLD fMRI and MEG characteristics to vibrotactile stimulation. Neuroimage 19:177817862003

    • Search Google Scholar
    • Export Citation
  • 22

    Tyndall AJReinhardt JTronnier VMariani LStippich C: Presurgical motor, somatosensory and language fMRI: technical feasibility and limitations in 491 patients over 13 years. Eur Radiol 27:2672782017

    • Search Google Scholar
    • Export Citation
  • 23

    Weng HHNoll KRJohnson JMPrabhu SSTsai YHChang SW: Accuracy of presurgical functional MR imaging for language mapping of brain tumors: a systematic review and meta-analysis. Radiology 286:5125232018

    • Search Google Scholar
    • Export Citation
  • 24

    Xia HHuang WWu LMa HWang XChen X: Preoperative functional MRI localization of language areas in Chinese patients with brain tumors: validation with intraoperative electrocortical mapping. Neural Regen Res 7:156315692012

    • Search Google Scholar
    • Export Citation
  • 25

    Youden WJ: Index for rating diagnostic tests. Cancer 3:32351950

TrendMD
Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 136 136 136
PDF Downloads 161 161 161
EPUB Downloads 0 0 0
PubMed
Google Scholar