Dual Imaging of Dopamine Release and PKA Activity Provides New Insights into Reinforcement Learning

 In Chapter 14 of Principles of Neurobiology, we learned about many key techniques used in the studies of different aspects of the nervous system. Among them, we learned ways to introduce transgene expression in the brain via transgenic recombinase line (14.9) and engineered viruses (14.10); optical imaging of neuronal activity (14.22); neuronal inactivation and activation via transgene expression, chemogenetics, and optogenetics methods (14.23-25); behavior paradigms in controlled conditions for complex behavior studies (14.28), etc.

In the real-world scenario, we typically need to combine the use of multiple techniques to solve complex neurobiology questions. In this recent study by Lee et al. (2021), the authors masterfully combined several techniques to study how dopamine (DA) modulates learning via protein kinase A (PKA) in the nucleus accumbens (NAc).

Dopamine-releasing neurons (DAN) in the ventral tegmental area (VTA) are known to regulate action reinforcement and reward prediction behaviors via controlling DA release in the NAc (see PoN Section 11.24). NAc contains two major classes of spiny projection neurons (SPNs), D1R-SPN (in which DA enhances cAMP production and PKA activity) and D2R-SPN (in which DA inhibits cAMP production and PKA activity). Yet the real-time relationship between DA level and PKA activity in SPNs remains untested owing to the technical challenges of monitoring intracellular signaling in behaving animals.

The authors investigated this question by developing a sensor to measure PKA activity in vivo. They first introduced transgene expression in either SPN class via Cre-recombinase mouse line and stereotaxic injection of AAV virus. The transgene, a fluorescent reporter, enables multichannel fiber photometry and fluorescence lifetime photometry (FLiP) to measure net PKA activity in SPNs. In FLiP, an optic fiber is implanted at the virus injection site to measure the fluorescent lifetime of the reporter by time-correlated single-photon counting. A shortened lifetime indicates PKA phosphorylation in the target neurons. With this technique, the authors observed bidirectional DA-receptor-dependent regulation of PKA activity in real-time by administering DA agonist and antagonist, confirming the performance of the PKA sensor system.

Next, the authors studied how DAN activity and DA levels in the NAc change across learning. They implemented a dual-fiber photometry setup and simultaneously recorded DAN activity in VTA and DA release in NAc during the learning process. By recording both signals during a behavior task where the mice gradually learned to respond to LED cues and get food rewards, the authors revealed a gradual shift in DA release from reward to reward anticipation (cue) across training; and DAN activity and DA levels had similar patterns during and after learning. These experiments confirmed the sensitivity of the DA sensor system.

The authors then coupled both PKA and DA sensor systems in the same brain by expressing two respective fluorescent sensors in different hemispheres with fiber implantations (see figure). With the help of D1R-SPN or D2R-SPN specific Cre-line, antagonists for D1R or D2R, and the previous learning task, the authors revealed that PKA activities in D1R- and D2R-SPNs respond to different DA dynamics and are asynchronously modulated: PKA in D1R-SPNs is activated at early learning stages by rewards as well as by each reward-predictive cue and reward after learning (which increase DA levels), whereas PKA in D2R-SPNs is activated only at late learning stages by failures to achieve expected rewards (which decrease DA below baseline).

A setup used in the study of the DA- and PKA-dependent learning circuit in the NAc: a fluorescent DA reporter (dLight1.1) was virally expressed in one side of NAc, with fiber photometry implantation to serve as a sensor for DA level; another fluorescent reporter (FLIM-AKAR) was virally expressed in the other side of NAc, with FliP setup to detect PKA activities in D1R- or D2R-SPNs, depending on the specific cre line used. This parallel recording setup enables simultaneous measurement of DA levels and PKA activities during learning in behaving animals. Figures adapted from Lee et. al., 2021.

Finally, using optogenetics techniques to activate or inactivate DAN in VTA and simultaneously measured DA level or PKA activity in SPNs, the authors showed that DA levels can be bidirectionally modulated by different opsins: DAN activation leads to DA release and increased net PKA activity in D1R-SPNs; while DAN inactivation reduces DA level and significantly increased net PKA activity in D2R-SPNs. Expressing a viral inhibitory peptide to block PKA activation also confirmed that PKA inhibition in SPNs slows learning.

In summary, this paper elegantly exemplified a combinatory approach for studying complex neurobiology questions by combining multiple techniques that we learned in PoN Chapter 14. The multifaceted data collectively revealed a mechanistic model of the DA/NAc-dependent learning process: learning-evoked positive and negative DA transients in NAc asynchronously modulate PKA activities in D1R- and D2R-SPNs, thus control the learning process.

Reference

1.   Lee, S. J., Lodder, B., Chen, Y., Patriarchi, T., Tian, L., & Sabatini, B. L., Cell-type-specific asynchronous modulation of PKA by dopamine in learning. Nature, 590, 451–456 (2021). Link