Regret is a fundamental experience of people with addiction disorders.1–3 Regret emerges at multiple stages of addiction, with regret at relapse after a failed attempt to quit the addiction highlighted as a major hurdle clinicians must address when treating a patient with an addictive disorder.4 Commercial gambling interfaces such as slot machines provide counterfactual information as feedback that suggests near misses, which elicits thoughts and feelings akin to “what if” or “almost hits” that are associated with continued gambling.5 Counterfactual prediction error (CPE) is a term that has been used to quantify an individual’s response to an outcome that is compared with potential alternative outcomes.6–11 CPEs may be associated with one’s experience of regret when the alternative options may have yielded better results than the experienced outcome; conversely, CPE may be associated with an experience of relief when alternative options are worse than the selected outcome.10,11 Recent investigations into a concept related to CPE, termed “fictive prediction error” in alcohol use disorder (AUD), revealed differences in the processing of potential regrets associated with gambling choices.3,12 Namely, individuals with AUD do not associate regret with their actions.12
The advent of human voltammetry, as a tool to measure dopamine release in the human brain, has revealed nuances in the role of phasic subsecond changes in dopamine levels in human decision-making10,13–15 and suggests a unique approach to investigate the role of dopaminergic signaling in AUD. Human voltammetry provides subsecond measurements of neurotransmitters, including dopamine, during stereotactic neurosurgery such as deep brain stimulation (DBS) electrode implantation procedures.10,13–15 Because many DBS surgeries take place while the patient is awake, human electrochemistry allows for the investigation of rapid (10 Hz) changes in dopamine levels with real-time changes in human behavior.10,13–15 In a study pairing a stock market investment task with subsecond dopamine measurements, Kishida and colleagues discovered that subsecond dopamine fluctuations following variable stock market investments encode the integration of CPEs with reward prediction errors (RPEs).10 Prior to the use of human voltammetry, subsecond changes in dopamine signals were believed to encode RPE alone based on foundational work showing spiking dopaminergic neuron activity following unexpected rewards.9 Just as human voltammetry has changed our understanding of dopamine as it relates to RPE, human voltammetry has the potential to uncover new findings about the role of dopamine as it relates to CPE.16,17
In the current study, we utilized a previously published approach to human voltammetry10,13–15 and a “sure bet or gamble” (SBORG) decision-making task to investigate dopamine’s response to CPEs following different types of decision outcomes. Voltammetric measurements were made in the caudate in patients undergoing DBS electrode implantation surgery for the treatment of essential tremor (ET) or Parkinson’s disease (PD); 2 of the 4 patients enrolled had a history of AUD (AUD1 and AUD2; Table 1). During the decision-making task, participants chose between a guaranteed gain option (sure bet) or a 50%-chance gamble option on each trial. In this study, we investigated subsecond dopamine time series data associated with SBORG outcomes with a particular focus on counterfactual information and differences observed in patients with versus without a history of AUD.
Human electrochemistry characteristics by patient
| Variable | Research ID | |||
|---|---|---|---|---|
| AUD1 | AUD2 | Non-AUD1 | Non-AUD2 | |
| Sex | M | M | M | M |
| Age (yrs) | 50s | 60s | 80s | 60s |
| Diagnosis for DBS surgery | ET | PD primary, potentially mixed PD/ET | PD | PD |
| Dopamine measurement target | Rt caudate | Lt caudate | Lt caudate | Rt caudate |
| QUIP-RS | — | 16 | Neuropsychology note states negative history of ICD, no QUIP-RS score given | 8 |
| ICD | — | Yes | No | No |
| Dopamine agonist | — | Previously taking 0.25 mg of pramipexole 3×/day, discontinued “too sleepy to function” | None | 0.5 mg of ropinirole 2×/day |
| Substance use disorder | AUD (active even after DBS) | 40-yr history of AUD (quit 7 yrs prior to DBS) | None | None |
ICD = impulse control disorder; QUIP-RS = Questionnaire for Impulsive-Compulsive Disorders in Parkinson’s Disease–Rating Scale.
Methods
Participants
The IRB at Wake Forest School of Medicine approved all participant recruitment, consent, and experimental procedures. The research team performed recruitment and informed consent procedures after patients consented to receive DBS to treat their movement disorder. In the present work, we report on 4 patients who completed this task. Two patients had a history of AUD and 2 did not (non-AUD; Table 1). We identified a history of AUD through chart review of patients who had given consent and performed our study. We reviewed available medical notes from providers in the departments of neurosurgery, neurology, psychiatry, and neuropsychology to identify a documented clinical history of AUD in patients’ electronic medical records. After patients signed informed consent forms, clinical information was collected through an electronic medical record search and IRB-approved survey tools (Table 1).
SBORG Decision-Making Task
Participants completed a decision-making task delivered through a computerized interface while electrochemical measurements were made in the operating room (Fig. 1). The behavioral task was adapted from the work of Rutledge and colleagues.18 At each trial, patients were presented with two options: a guaranteed gain and a 50%-chance gamble. The guaranteed gain, or sure bet, represents a value equal to a set dollar amount (ranging from $1 to $6 in $1 increments), and the 50%-chance gamble represents two values each with an equal probability of being the final outcome. The values in the gamble option range from $0 to $6 in $1 increments. All values (the sure bet and the two values in the gamble option) were determined by a computerized random draw from a uniform distribution over the range of possible values at each trial. After participants made their choice, the trial ended with feedback about the outcome of their choice. For the sure bet, there was no surprise. For the gamble option, one of the two possible outcomes was chosen and shown as a statement, “You won $X,” where $X is the amount won in that particular gamble. Patients were instructed that one of the many trials they completed would be chosen at random (by the computer) at the completion of the task and used as an actual bonus payment; thus, each decision should have been made as though that choice may be realized.
Timeline of events for human electrochemistry during the SBORG decision-making task. A and B: The SBORG task was composed of independent trials that participants interacted with (A) using a game controller (B). Participants chose between a sure bet (a single number with 100% probability if selected) and a gamble (two numbers each with a 50% probability if selected). Consistent feedback about choice selection and outcome was given for each trial, followed by a screen showing the “+” symbol to indicate transition between trials. Task timing was noted in the text above each screen (in seconds). C: Human electrochemistry measurements of dopamine were made in the DBS operating room. Custom carbon fiber microelectrodes for research were placed by the neurosurgeons along the same trajectory as the clinical electrodes used for microelectrode recording. For patient “non-AUD1,” the research electrode trajectory and recording location are shown by the yellow trace and red crosshairs, respectively, terminating in the caudate. The possible clinical electrode trajectory is shown in green, with the clinical DBS target (in the subthalamic nucleus) represented by a green crosshair. Views are in plane with the electrode trajectory.
Visual stimuli were presented on a monitor that hung from the ceiling in the operating room. The side of the screen where the two options were presented were randomized for each trial. Participants made their choices using a button box controller by pressing a left or right button to indicate the choice shown on the left or right side of the screen, respectively. Participants were instructed to make their choices on each trial as quickly as possible. Options were presented for a randomized amount of time (determined by a Poisson distribution, with λ = 6 seconds). If participants failed to respond within this time constraint, a screen displayed the message “too late” and advanced to the next trial. An intertrial interval screen appeared between each trial that contained the “+” symbol in the center of a blank screen. The duration of the intertrial interval was randomized following a Poisson distribution with λ = 3 seconds.
Human Voltammetry
Electrochemical measurements of dopamine were made following previously published methods.10,13–15 Briefly, dopamine measurements are made during the microelectrode recording phase of DBS surgery. Carbon fiber microelectrodes were placed in the caudate during this phase of the procedure within a fixed 30-minute window of dedicated research time. The recording location in the caudate was on the same trajectory as the clinical microelectrode, but at an anatomically superior location (Fig. 1, Supplemental Fig. 1). The clinical trajectories were not modified for research purposes. While the task was performed, electrochemical measurements using the carbon fiber microelectrode and a fast-scan cyclic voltammetry (FSCV) protocol were performed at one dopamine measurement every 100 msec.10,13–15 The carbon fiber research electrodes were constructed in the laboratory of the corresponding author (K.T.K.). These customized carbon fiber microelectrodes had passed a successful ethylene oxide sterilization exposure and sterility audit conducted by BioLabs to ensure that preoperative ethylene oxide treatment fully sterilized the electrodes. These electrodes were also validated and approved for autoclave and hydrogen peroxide sterilization. Notably, recent work has demonstrated that this research protocol does not increase the infection rate compared with DBS surgeries without this research protocol.19
The FSCV protocol and analysis used in this study followed prior work.10,13–15 Carbon fiber microsensors were conditioned using a 60-Hz application of a triangular voltage protocol for approximately 10 minutes. The triangular voltage protocol was as follows: the electrode potential was held at −0.6 V for 16.6 or 90 msec (for the 60-Hz conditioning or 10-Hz measurement protocol, respectively); from the −0.6-V holding potential, the voltage was linearly increased to +1.4 V at 400 V/sec, and ramped back down to −0.6 V at −400 V/sec. Once the 60-Hz conditioning phase was completed, the measurement protocol and task were initiated. The measurement protocol used the same triangular voltage protocol as the conditioning protocol, but with the triangular waveform once every 100 msec (i.e., 10 Hz). Within the 10-msec triangular voltage protocol, the current was measured at a sampling rate of either 100 kHz or 250 kHz per electrode (maximum range of electrochemical currents ± 2000 nA). These electrochemical current spectra were analyzed after a machine learning–based approach based on prior work.10,13–15 Briefly, dopamine concentration estimates were made using multivariate regression models that were fit and cross-validated using an elastic net algorithm from the “glmnet” package in MATLAB (The MathWorks) and calibration data sets generated in our laboratory.15,20–22
CPEs and Corresponding Dopamine Responses
We calculated CPEs based on the choice of the participant and what they received compared with what they could have received had they selected the other option. Specifically, for trials in which participants chose the sure bet, the equation for sure bet CPE is: sure bet value − expected value of the gamble option. When participants chose the gamble, the equation for gamble CPE is: gamble outcome − sure bet value. Within the sure bet CPE and gamble CPE events, we further sorted trials into three categories based on the valence of the CPE: positive CPE (CPE > 0), zero CPE (CPE = 0), and negative CPE (CPE < 0).
We compiled dopamine measurements corresponding to these CPE categories for each participant and pooled responses across all participants. We removed any rounds in which a participant did not respond in time (Supplemental Table 1). Dopamine measurement time series spanning a single trial are z-score normalized using the “zscore()” MATLAB function.23 Finally, we compared dopamine time series (1-second duration) time-locked to the SBORG outcome presentation.
Statistical Analyses
We investigated dopamine responses in the caudate to counterfactual information in patients with and without a history of AUD. We compared dopamine responses within non-AUD or AUD participants according to CPE valence categories (i.e., positive CPE, neutral CPE, and negative CPE). We also compared dopamine response with CPE valence categories across AUD versus non-AUD participants.
All statistical analyses and graphing were conducted using RStudio and MATLAB.23,24 We pooled trial data across patients separately for the non-AUD and AUD groups. We conducted behavioral analyses in RStudio, including calculations of mean, standard error of the mean (SEM), and gambling percentage.24 We compared the values selected for sure bet and gamble options within the AUD pair or separately within the non-AUD pair using Welch’s two-sample t-tests. We compared the difference in gambling rates across these two groups using the Wilcoxon rank-sum test.
We conducted dopamine time series analyses in MATLAB.23 For within–AUD group dopamine time series comparisons, we implemented two-way ANOVAs using time series and the CPE group using the MATLAB function “anovan()” to determine significant differences across dopamine time series data. To compare AUD versus non-AUD dopamine time series, we implemented two-way ANOVAs using time series and the AUD group separately for each CPE category using the MATLAB function “anovan().” We conducted post hoc identification of significant comparisons with correction for multiple comparisons with Tukey’s honestly significant difference (HSD) using the MATLAB function “multcompare().”
Results
Participants With Versus Without AUD Facing a Gamble Versus a Sure Bet Opportunity
We compared choice behavior and the counterfactual outcome value for choices made across the AUD and non-AUD participants. We compared CPE distributions for trials in which individual participants chose the sure bet (Fig. 2) or gamble (Fig. 3) options (group distributions presented in Supplemental Figs. 2 and 3). We also compared the rate of gambling across these participants.
Distribution of sure bet CPEs by patient. The distribution of the CPE values for all trials in which participants selected the sure bet option, separated by individual patients with (AUD) and without (non-AUD) a history of AUD. Overall, patients experienced a similar distribution of CPE when they chose the sure bet option.
Non-AUD participants chose the sure bet option in 307 trials (Fig. 2). AUD participants chose the sure bet option in 337 trials (Fig. 2). The values of selected sure bet CPEs were higher in the AUD group (mean 0.4169, SEM 0.1101) than in the non-AUD group (mean −0.1962, SEM 0.1223; Welch’s two-sample t-test, t[724.2668] = 3.7265, p = 0.0002).
Non-AUD participants chose the gamble option in 84 trials (Fig. 2). AUD participants chose the gamble option in 103 trials (Fig. 2). Welch’s two-sample t-test revealed there was no significant difference between the mean value of the gamble CPEs chosen by the non-AUD participants (mean 1.1785, SEM 0.2575) compared with the AUD participants (mean 1.2816, SEM 0.2144; t[171.0570] = 0.3073, p = 0.7590).
Distribution of gamble CPEs by patient. The distribution of the CPE values for all trials in which participants selected the gamble option, separated by individual patients with (AUD) and without (non-AUD) a history of AUD. Overall, patients experienced a similar distribution of CPE when they chose the gamble option, although patient AUD2 chose the gamble very few times and the non-AUD2 patient is not shown because that patient selected the gamble option only once.
We also calculated the gambling percentage for the non-AUD and AUD groups as the number of trials a participant chose a gamble as a percentage of the total number of trials in which a participant made a choice ("too late" trials were excluded). The gambling percentages for the non-AUD and AUD groups were 21.4834% and 23.4091%, respectively. There was no significant difference between gambling percentages across the non-AUD and AUD groups on a Wilcoxon rank-sum test (W statistic = 2, p > 0.9999).
Subsecond Dopamine Responses to Counterfactual Information
Discrimination of CPE Valence
We compared dopamine time series measurements by CPE valence in non-AUD and AUD participants to determine if CPE valence is associated with changes in the dopamine response in this task. Dopamine fluctuations distinguished CPE valence in the non-AUD group for both the sure bet (Fig. 4) and gamble (Fig. 5) trials, but less so for the AUD group.
Subsecond dopamine fluctuations for patients with AUD (n = 2) and non-AUD (n = 2) for all trials in which participants selected the sure bet option. Trials are pooled within the AUD and non-AUD groups. Dopamine fluctuations are separated within each group by CPE valence (positive and negative). Dopamine fluctuations are well delineated by CPE valence in the non-AUD group. This separation is lost in the AUD group, with noisier, nondiscriminating dopamine fluctuations across CPE valence.
Subsecond dopamine fluctuations for 1 patient with AUD (AUD1) and 1 without AUD (non-AUD1) in the trials in which participants selected the gamble option. Dopamine fluctuations are separated within each group by CPE valence (positive and negative). Patients AUD2 and non-AUD2 were excluded from this analysis because neither selected the gamble option a sufficient number of times for these comparisons to be valid. Patient AUD1 showed a significantly lower dopamine response to positive CPEs following the choice to gamble compared with patient non-AUD1.
During sure bet trials, comparisons of dopamine time series responses across CPE valence within the non-AUD participants revealed a significant difference between CPE valence categories (two-way ANOVA, f[2] = 4.1066, p = 0.0166; Supplemental Table 2). A post hoc Tukey HSD revealed significant differences between dopamine time series responses comparing neutral versus positive and neutral versus negative valence CPEs (Supplemental Table 3). However, within the AUD participants, comparing dopamine time series responses across CPE valence categories revealed that only the interaction of time series and CPE valence was significant (two-way ANOVA, f[18] = 2.4564, p < 0.001).
Among the 4 participants, 2 (1 each with AUD and non-AUD) expressed very few choices to gamble; thus, we restricted our statistical analyses comparing trials across the 2 participants who did express a significant number of choices to gamble. Between these 2 participants, the dopamine responses appear similar across AUD status, but do separate according to CPE valence. During the gamble trials, comparisons of dopamine time series responses across CPE valence within the non-AUD and AUD participants revealed a significant difference between CPE valence categories (non-AUD: two-way ANOVA, f[2] = 4.2525, p = 0.0146; AUD: two-way ANOVA, f[2] = 6.1467, p = 0.0022; Supplemental Table 4). A post hoc Tukey HSD revealed significant differences between dopamine time series responses to positive versus negative valence CPEs in both the non-AUD and AUD participants (Supplemental Table 5).
Subsecond Dopamine Responses to Counterfactual Information Across Patients With Versus Without AUD
We compared subsecond dopamine time series measurements with CPE valence across non-AUD versus AUD participants to determine if the AUD status was associated with changes in processing signals related to regret and relief. For sure bet trials (Fig. 4), we compared dopamine time series across non-AUD versus AUD participants for each of the CPE valence categories (i.e., positive, neutral, or negative valence CPEs; neutral CPE data are available in Supplemental Figs. 4 and 5). A two-way ANOVA for dopamine time series responses to positive valence CPEs in non-AUD versus AUD participants revealed no significant difference (f[1] = 3.3836, p = 0.0659). However, the interaction of the dopamine time series and AUD category (non-AUD vs AUD) was significant for the comparison of neutral valence CPE (two-way ANOVA, f[9] = 3.2217, p = 0.0009). A two-way ANOVA for negative valence CPEs found the time series (f[9] = 2.1661, p = 0.0218) and interaction between time series and AUD category (f[9] = 2.6268, p = 0.0051) to be significant, but AUD status alone was not significant (f[1] =3.4895, p = 0.0619).
Again, because 2 participants did not show a sufficient number of gamble choices (1 with AUD and 1 with non-AUD), we compared gamble trial dopamine responses from 1 patient with AUD versus 1 patient without AUD (Fig. 5). We compared dopamine time series across non-AUD versus AUD participants for each of the CPE valence categories (i.e., positive, neutral, or negative valence CPEs; neutral CPE data are available in Supplemental Figs. 4 and 5). A two-way ANOVA for dopamine time series responses to positive valence CPEs revealed a significant difference between AUD status (f[1] = 6.5953, p = 0.0104). There were no significant differences on the two-way ANOVA for dopamine time series responses to neutral or negative valence CPEs comparing non-AUD and AUD participants.
Discussion
The present study investigated subsecond dopamine fluctuations in an SBORG decision-making task in 2 patients with AUD and 2 patients without a history of AUD. We examined dopamine time series data collected at a rate of 10 Hz in reaction to CPEs concerning the outcomes of the patients’ decisions. We report modulation of dopamine responses according to AUD status and the valence of CPEs. The relationship of CPEs to psychological constructs of regret and relief, and prior work suggesting that counterfactual information modulates RPE learning signals in humans,10,14 suggests that a history of prolonged alcohol use may be associated with fundamental changes in human neuromodulatory physiology that is critical for emotional reactivity and adaptive learning. Of course, a major limitation of the current work is the limited sample size and the heterogeneity of the participants’ movement disorder symptoms (Table 1). However, to our knowledge, data pertaining to fast dopamine fluctuations in humans with AUD in any context have not been previously reported, making these findings critical first steps and important for the unique insight we may gain.
CPEs are consistent with these psychological constructs of regret and relief. Future work will be needed to better quantify the connection between these quantifiable error signals regarding objective monetary outcomes and the subjective feelings associated with the notions of regret and relief. We report that subsecond dopamine responses encode counterfactual information about monetary outcomes that are better or worse than “what could have been.” This finding corroborates prior work, but the present work extends our knowledge of human dopamine by providing the first insights into dopamine signaling about regret and relief in patients with a history of AUD. Looking within non-AUD participants, better than ”could have been” outcomes (positive CPEs) versus worse than “could have been” outcomes (negative CPEs) are distinguishable in the dopamine time series response. However, this distinction is diminished within the AUD participants (Fig. 4).
In prior addiction work pertaining to regret and relief, regret is typically characterized as a defining feature of addiction disorders, including AUD.1–3 These prior studies primarily use survey instruments or other conceptual frameworks that assess feelings about regret on timescales over days to years.1–3 In addition, no prior work has been able to investigate dopaminergic signals with the temporal resolution presented here. Our results reveal that dopamine responses (measured in hundreds of milliseconds) in AUD following relief-related events (i.e., positive CPE) are lower than in patients without AUD. It appears that in these patients, dopaminergic encoding of regret on fast moment-to-moment timescales is unaltered in AUD, but on the same timescales it is the encoding of signals related to relief that is altered. The connection between moment-to-moment encoding of counterfactual outcomes and the long-term consequences of these events on one’s assessment of their psychological state (as measured through survey instruments) was not investigated in the present study and remains an area that requires further investigation.
Prior work suggests that learning from regret is impaired in patients with AUD.3,12,25 The current study probes subsecond dopamine fluctuations in the context of individual decisions and the impact that quantifiable monetary outcomes and explicit counterfactual monetary outcomes related to a decision made have on dopaminergic signals.10,13,15 The task we used does not lend itself to investigating learning effects; i.e., each trial has complete information such that prior trials and subsequent trials are independent of any one trial decision. Furthermore, while the valence of a CPE is likely a component of subjective feelings of regret or relief, we did not directly measure participants’ feelings of regret or relief on any given trial. A limitation of the present work is that we cannot determine whether the CPEs are large enough to elicit significant feelings of regret or relief, nor have we determined the effect of these feelings on participants’ behavior.
CPE as a behavioral screening tool has potential translational applications in DBS to treat neuropsychiatric disorders. For DBS to treat both PD and ET, tremor symptoms are assessed intraoperatively to determine if the DBS electrode will effectively manage symptoms.26,27 Although DBS has been extended to treat neuropsychiatric disorders such as Tourette syndrome and obsessive-compulsive disorder,28–30 there is not currently a comparable intraoperative test to predict efficacy. Given our findings, neuronal activity or dopamine measurements following decision-making outcomes may be investigated as a potential intraoperative measure in the application of DBS-based treatments of neuropsychiatric disorders. This will be particularly beneficial as the neurosurgical field moves toward using DBS to treat addictive disorders, including AUD.31,32
Conclusions
The present study proposes relief as a differentiating factor for patients with and without AUD. We found that dopamine fluctuations following outcomes that were better than they could have been otherwise were lower in patients with AUD. Future work with larger sample sizes is necessary to determine the broader generalizability of our findings. However, these first-of-their-kind data from patients with AUD suggest potential applications for the use of decision-making tasks that incorporate counterfactual information and subsecond neurochemical recordings during DBS procedures to optimize treatments of addiction and other debilitating psychiatric conditions.
Acknowledgments
Dr. Kishida reported receiving funding from the National Institutes of Health (NIH): 1) National Institute on Drug Abuse (R01DA048096, P50DA006634); 2) National Institute of Mental Health (R01MH121099, R01MH124115); 3) National Center for Advancing Translational Sciences (KL2TR001420); 4) National Institute on Alcohol Abuse and Addiction (P50AA026117). Dr. Liebenow reported receiving grants from the National Institute on Drug Abuse of the NIH (no. F30DA053176) during the conduct of the study. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosures
Dr. Haq reported personal fees and nonfinancial support from Boston Scientific and Medtronic, and nonfinancial support from Abbott outside the submitted work. Dr. Tatter reported grants from Monteris Medical and Arbor Pharmaceuticals outside the submitted work.
Author Contributions
Conception and design: Kishida, Liebenow, Jiang, Laxton, Tatter. Acquisition of data: Kishida, Liebenow, Jiang, DiMarco, Wilson, Laxton, Tatter. Analysis and interpretation of data: Kishida, Liebenow, Jiang, ul Haq, Laxton, Tatter. Drafting the article: Kishida, Liebenow, Siddiqui, ul Haq, Laxton, Tatter. Critically revising the article: Kishida, Liebenow, Jiang, DiMarco, ul Haq, Laxton, Tatter. Reviewed submitted version of manuscript: Kishida, Liebenow, Jiang, DiMarco, Siddiqui, ul Haq, Laxton. Approved the final version of the manuscript on behalf of all authors: Kishida. Statistical analysis: Kishida, Liebenow, Jiang. Administrative/technical/material support: Kishida, Liebenow, Jiang. Study supervision: Kishida, Liebenow.
Supplemental Information
Online-Only Content
Supplemental material is available online.
Supplemental Tables and Figures. https://thejns.org/doi/suppl/10.3171/2022.11.FOCUS22614.
References
- 1.↑
Orphanides A, Zervos D. Rational addiction with learning and regret. J Polit Econ. 1995;103(4):739–758.
- 2.
Fong GT, Hammond D, Laux FL, et al. The near-universal experience of regret among smokers in four countries: findings from the International Tobacco Control Policy Evaluation Survey. Nicotine Tob Res. 2004;6(suppl 3):S341-S351.
- 3.↑
Galandra C, Basso G, Cappa S, Canessa N. The alcoholic brain: neural bases of impaired reward-based decision-making in alcohol use disorders. Neurol Sci. 2018;39(3):423–435.
- 4.↑
Volkow ND. Personalizing the treatment of substance use disorders. Am J Psychiatry. 2020;177(2):113–116.
- 7.
Sutton RS, Barto AG. Reinforcement Learning: An Introduction. 2nd ed. MIT Press; 2018.
- 8.
Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci. 1996;16(5):1936–1947.
- 9.↑
Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275(5306):1593–1599.
- 10.↑
Kishida KT, Saez I, Lohrenz T, et al. Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward. Proc Natl Acad Sci U S A. 2016;113(1):200–205.
- 11.↑
Lohrenz T, McCabe K, Camerer CF, Montague PR. Neural signature of fictive learning signals in a sequential investment task. Proc Natl Acad Sci U S A. 2007;104(22):9493–9498.
- 12.↑
Galandra C, Crespi C, Basso G, Canessa N. Impaired learning from regret and disappointment in alcohol use disorder. Sci Rep. 2020;10(1):12104.
- 13.↑
Kishida KT, Sandberg SG, Lohrenz T, et al. Sub-second dopamine detection in human striatum. PLoS One. 2011;6(8):e23291.
- 14.↑
Moran RJ, Kishida KT, Lohrenz T, et al. The protective action encoding of serotonin transients in the human brain. Neuropsychopharmacology. 2018;43(6):1425–1435.
- 15.↑
Bang D, Kishida KT, Lohrenz T, et al. Sub-second dopamine and serotonin signaling in human striatum during perceptual decision-making. Neuron. 2020;108(5):999–1010.e6.
- 16.↑
Kishida KT, Sands LP. A dynamic affective core to bind the contents, context, and value of conscious experience. In: Waugh CE, Kuppens P, eds. Affect Dynamics. Springer International Publishing; 2021:293-328.
- 17.↑
Collins AG, Frank MJ. Opponent actor learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychol Rev. 2014;121(3):337–366.
- 18.↑
Rutledge RB, Skandali N, Dayan P, Dolan RJ. A computational and neural model of momentary subjective well-being. Proc Natl Acad Sci U S A. 2014;111(33):12252–12257.
- 19.↑
Liebenow B, Williams M, Wilson T, et al. Intracranial approach for sub-second monitoring of neurotransmitters during DBS electrode implantation does not increase infection rate. PLoS One. 2022;17(8):e0271348.
- 20.↑
Friedman J, Hastie T, Tibshirani R. glmnet: lasso and elastic-net regularized generalized linear models. Accessed December 2, 2022. https://cran.r-project.org/web/packages/glmnet/glmnet.pdf
- 21.
Qian J, Hastie T, Friedman J, Tibshirani R, Simon N. Glmnet in Matlab. Accessed December 2, 2022. http://hastie.su.domains/glmnet_matlab/
- 22.↑
Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Series B Stat Methodol. 2005;67(2):301–320.
- 25.↑
Tochkov K. The effects of anticipated regret on risk preferences of social and problem gamblers. Judgm Decis Mak. 2009;4:227–234.
- 26.↑
Wan KR, Maszczyk T, See AAQ, Dauwels J, King NKK. A review on microelectrode recording selection of features for machine learning in deep brain stimulation surgery for Parkinson’s disease. Clin Neurophysiol. 2019;130(1):145–154.
- 27.↑
Deuschl G, Raethjen J, Hellriegel H, Elble R. Treatment of patients with essential tremor. Lancet Neurol. 2011;10(2):148–161.
- 28.↑
Bari AA, Mikell CB, Abosch A, et al. Charting the road forward in psychiatric neurosurgery: proceedings of the 2016 American Society for Stereotactic and Functional Neurosurgery workshop on neuromodulation for psychiatric disorders. J Neurol Neurosurg Psychiatry. 2018;89(8):886–896.
- 29.
Baldermann JC, Schüller T, Huys D, et al. Deep brain stimulation for Tourette-Syndrome: a systematic review and meta-analysis. Brain Stimul. 2016;9(2):296–304.
- 30.↑
Bina RW, Langevin JP. Closed loop deep brain stimulation for PTSD, addiction, and disorders of affective facial interpretation: review and discussion of potential biomarkers and stimulation paradigms. Front Neurosci. 2018;12:300.
- 31.↑
Navarro PA, Paranhos T, Lovo E, et al. Safety and feasibility of nucleus accumbens surgery for drug addiction: a systematic review. Neuromodulation. 2022;25(2):171–184.
- 32.↑
Voges J, Müller U, Bogerts B, Münte T, Heinze HJ. Deep brain stimulation surgery for alcohol addiction. World Neurosurg. 2013;80(3-4):S28.e21–e31.
