We combined tetrode recordings in dorsal CA1 (dCA1) and chemogenetic silencing of VTA dopamine neurons to determine whether reward-related changes in hippocampal ripples and replay required VTA dopamine signaling. Transgenic rats expressing cre-recombinase under the tyrosine hydroxylase (TH) promoter were stereotactically injected with cre-dependent virus containing the inhibitory DREADD hM4Di (Experimental, n=4) or mCherry-only control (Control, n=3) into bilateral VTA (Figure 1A). We observed widespread expression across the extent of VTA and co-localization with TH (Figure 1B), enabling specific and reversible inactivation of VTA dopamine signaling. Recording microdrives containing 6-32 independently adjustable tetrodes (bilateral 32 tetrode, n=4; unilateral 20 tetrode, n=2; unilateral 6 tetrode, n=1) were implanted above dCA1 (Fig. 1A). Each tetrode was lowered into the pyramidal cell layer of dCA1 to collect single unit and LFP data.

Before each experimental session, rats were given intraperitoneal injection of CNO, to activate hM4Di receptors and suppress VTA dopamine neuron activity, or saline, then performed a simple task on linear tracks (1.5 to 2.5 m in length), collecting liquid chocolate rewards from each end (Fig. 1C). Each session began with equal 0.1 ml reward volume at each end (Epoch 1) for 10-20 laps (1 lap was comprised of reward collection at both ends; mean, 16 laps). This was followed by unsignaled quadrupling of reward at one end to 0.4 ml (Incr. end), while reward at the other end remained unchanged (Unch. end), for 10-20 laps (Epoch2; mean, 16.7 laps). Finally, reward was equalized again to 0.1 ml at both ends (Epoch 3) for up to 20 laps (mean, 11.6 laps).

Each animal performed this task on familiar linear tracks (>2 sessions on track; total session count for each condition: Experimental rats: 36 saline, 34 CNO; Control rats: 23 saline, 25 CNO) and novel linear tracks (1^st or 2^nd session on track; Experimental rats: 9 saline, 10 CNO; Control rats: 12 saline, 12 CNO). During stopping periods at either end of the track (velocity ≤ 8 cm/s, position ≤ 10 cm from end), SWR were identified as peaks in the ripple band (150-250 hz) in LFP (Fig. 1D-E; see Methods).

Gross behavior was largely unaffected by VTA suppression (e.g., all reward consumed on each lap), but CNO in experimental animals systematically affected stopping period duration. Visits to the Unch. end in Epoch 2 were significantly shorter than in Epoch 1, despite unchanged reward volume there, and while this reduction was present in CNO sessions, overall visit durations were increased (Fig. 1F-G). CNO did not affect control animals (Fig. 1F-G). Visits to the Incr. end were significantly longer in Epoch 2 than in Epoch 1 in all conditions, owing to the increased reward consumption time (Fig. 1H-I; Epoch 1 vs. Epoch 2, two sample t-test, all p<10^-10). Changes in stopping period duration in Epoch 3 were similar across all conditions: Unch. visit duration increased from Epoch 2 to Epoch 3 (Fig. S1E), while Incr. visit duration decreased (Fig. S1F). Separate analysis of novel and familiar sessions revealed the pattern of shorter duration Unch. visits in Epoch 2 compared to Epoch 1 did not depend on novelty (Fig. S1A-D). However, the main effect of CNO in experimental rats of prolonging stopping periods occurred in novel sessions (Fig. S1A-B), not in familiar sessions (Fig. S1C-D). Rats consistently ran slightly faster towards the Incr. end than the Unch. end in Epoch 2, across all conditions (Fig. S2).

We interpret the reduction in visit duration as a behavioral signature of the Unch. end becoming relatively less valuable during Epoch 2, when the reward volume is larger at the Incr. end. Though visit durations were slightly longer in CNO sessions in experimental rats, particularly in novel track sessions, this behavioral effect of a relative value decrease remained, indicating VTA inactivation did not prevent rats from recognizing a devalued location.

We analyzed the rate of SWR occurrence during the first 10 s of each stopping period, when rats were consuming reward. In individual sessions, SWR rate increased robustly in all conditions shortly after stopping at the reward wells and beginning reward consumption (Fig. 2A). Relative to Epoch 1, SWR rate during Epoch 2 tended to increase dramatically at Incr. end visits and decrease at Unch. end visits. During Epoch 3, SWR rate at the Incr. end dropped precipitously relative to Epoch 2, while often increasing at the Unch. end (Fig. S4A).

Surprisingly, during novel sessions, VTA inactivation often led to increased SWR rate at both reward ends (Fig. 2A, right). SWR rate still increased in Epoch 2 at the Incr. end even without normal VTA signaling, indicating reward sensitivity per se was not abolished, but suggesting the localization of this increased SWR rate to where reward increased was disrupted.

Pooling across sessions revealed this dramatic increase in SWR rate at the Unch. end was typical with CNO in novel sessions, and further suggested there was a reduction in the difference in SWR rate between the Incr. and Unch. ends in Epoch 2 in both familiar and novel experiences (Fig. 2B). We therefore used a Poisson generalized linear model (GLM) to quantify the changes in SWR rate across reward end, epoch, drug condition, and novelty (see Methods). In experimental rats, CNO and reward both influenced SWR rate, with significant effects for the CNO main effect (z=3.19, p=0.0014), the interaction between Incr. end and Epoch 2 (z=9.02, p<10^-10), the three-way interaction between Incr. end, Epoch 2, and CNO (z=-2.06, p=0.0396), and marginally by the interaction between Incr. end and CNO (z=-1.92, p=0.055). Control animals showed no apparent effect of CNO (Fig. 2C). The same Poisson GLM fit to control rat data confirmed this, with significant coefficients only for Incr. end (z=-2.42, p=0.0156) and the interaction between Incr. end and Epoch 2 (z=7.64, p<10^-10).

To assess the interaction between VTA inactivation and novelty, we fit the Poisson GLM separately to novel and familiar sessions, then used bootstrapping to generate distributions of SWR rates for each reward end and condition under the null hypothesis that CNO had no effect (see Methods). We found the actual difference in SWR rates between saline and CNO sessions in experimental animals during epoch 2 was significantly greater than chance (Fig. S3A) in novel sessions for both the Unch. end (one-tailed test, CNO>saline, p=0.0006) and the Incr. end (one-tailed test, saline>CNO, p=0.004), as well as at the Unch. end in familiar sessions (one-tailed test, CNO>saline, p=0.002). There was no significant difference between saline and CNO in control animals at either reward end in either familiar or novel sessions (Fig. S3B; one-tailed tests, all p>0.17).

A potential functional role for the reward-related changes in SWR rate is to strengthen downstream representations of particularly rewarding locations at the expense of less rewarding locations. We tested whether VTA suppression blunted the SWR rate difference between reward ends in Epoch 2. Across both familiar and novel environments, CNO reduced the difference in SWR rate between the Incr. end and Unch. end in experimental rats but not in control rats (Fig. 2D-E, left panels; three-way ANOVA with animal group, drug, and novelty: drug, F[1,153]=5.19, p=0.024, group by drug, F[1,153]=5.16, p=0.025, all other terms n.s.). To control for the possibility that VTA inactivation caused changes in locomotor or other non-consummatory behavior at reward wells that might affect SWR emission, we omitted the first and last 1 s of each stopping period to isolate the reward consumption period. The effect of CNO remained in experimental rats, reducing SWR rate discrimination between Incr. and Unch. ends (Fig. 2D-E, right panels; three-way ANOVA with animal group, drug, and novelty: drug, F[1,153]=3.41, p=0.067; group by drug, F[1,153]=5.58, p=0.019).

We next looked for within-epoch changes in SWR rate to determine whether VTA inactivation altered the dynamics of the response to reward changes. We calculated the difference in SWR rate at the Incr. and Unch. reward ends (each with its Epoch 1 mean subtracted) in a 5-lap sliding window. In all time windows and conditions except novel CNO sessions in experimental rats, the SWR rate at the Incr. end was significantly greater than at the Unch. end (Fig. 2F-G; Incr. – Unch. significantly greater than 0, one-sample t-test, p<0.05, uncorrected for multiple comparisons).

In novel sessions, VTA inactivation did not prevent an initially larger increase in SWR rate at the Incr. end than the Unch. end, but caused that difference to diminish over laps (Fig. 2F). By the middle of the epoch, there was no statistically-significant difference in reward modulation between the reward ends (one-sample t-test, p>0.05, for 5-lap windows centered on laps 8-13 and 15), consistent with an initial appropriately-localized reaction to reward change that eventually spread across the track. We found a similar deficit in Epoch 3, with SWR rate decreasing significantly more compared to Epoch 2 at the Incr. end than the Unch. end (Incr. – Unch. significantly below 0, one-sample t-test, p<0.05) for almost every task condition and timepoint except in novel CNO sessions in experimental rats (Fig. S4B-C). This suggests VTA inactivation may disrupt the normal magnitude or time course of the SWR response to negative value changes as well.

Overall, VTA inactivation spared the capacity for increased reward to modulate SWR rate but led to decreased differentiation of low and high value locations, particularly in novel environments where SWR rate increased spatially-indiscriminately.

Taken together, the above results demonstrate VTA inactivation caused changes in the normal dynamics of the response of SWR rate to positive and negative changes in reward value (Fig. 2). However, because each session had only two timepoints when reward value changed by fixed amounts, our experiment was not optimized to probe the precise relationship between SWR rate and reward changes. Additionally, the effect of VTA inactivation was particularly prominent with novelty, when both SWR rate and its modulation by reward changes were greater, raising the possibility that large SWR rates and fluctuations, rather than novelty per se, depend on VTA dopamine signaling.

To address these questions, we designed a volatile reward schedule (Experiment 2) with frequent, large reward changes at one end of the linear track, and tested whether VTA inactivation impacted the capacity for SWR rate to track value (Fig. 3A, top). The “stable end” delivered 0.2 ml every lap, while the “volatile end” reward volume was drawn pseudorandomly from 0, 0.1, 0.2, 0.4, and 0.8 ml (mean 0.37 ml; blocks of 20 laps were comprised of 3 laps x 0 ml, 4 laps x 0.1 ml, 3 laps x 0.2 ml, 4 laps x 0.4 ml, and 6 laps x 0.8 ml). The position of the stable and volatile ends randomly varied across sessions.

This reward schedule also allowed us to test whether SWR rate was correlated with value, RPE, or neither. We expected SWR rate at the volatile end would be predominantly determined by the current reward volume there, but potentially also modulated by previous reward volumes (Fig. 3A, bottom). If SWR rate is correlated with value, then for a given current volume, larger reward volumes at the last visit will lead to higher SWR rates compared to when the last visit was a smaller reward volume. Conversely, if SWR rate is correlated with RPE, the opposite modulation by last reward volume will be observed: the larger the previous reward volume, the lower the current SWR rate.

A subset of rats performed sessions of the modified reward schedule (mean 53.4 laps per session; Total sessions per condition: Experimental rats 2 and 4: 6 saline, 7 CNO; Control rats 1-3: 16 saline, 18 CNO). Each rat completed 1-2 saline sessions before any CNO sessions and all sessions were on the same linear track, meaning almost all CNO sessions took place on a familiar track. As expected, SWR rate at the volatile end was predominantly determined by the current volume (Fig. 3B-C). There was little obvious difference between saline and CNO in either experimental (Fig. 3B) or control rats (Fig. 3C). SWR rate at the stable end was largely stable across laps, although there was a trend towards higher SWR rate if the most recent volatile end visit was lower volume, consistent with lap-by-lap changes in the relative value of the stable end (Fig. S5).

We next investigated how SWR rate at the volatile end varied as a function of both current and immediately previous volatile end volume in experimental rats (Fig. 3D-E, top panels). For each current volume, we subtracted the mean SWR rate across all previous volumes, and examined how previous volume affected the mean-subtracted SWR rates across all current volumes (Fig. 3D-E, middle panels).

The mean-subtracted SWR rate was modestly negatively correlated with the previous volume (Pearson correlation: saline, r=-0.076, p=0.177; CNO, r=-0.109, p=0.049). A GLM found the mean-subtracted SWR rate was significantly affected by previous volume (z=-2.3, p=0.02), but not drug (z=-0.24, p=0.81) or current volume (z=0.196, p=0.844). There was no similar behavioral effect: for each current volume, we subtracted the mean reward end visit duration, and found no correlation between the previous reward volume and mean-subtracted visit duration (Pearson correlation; saline, r=0.011, p=0.844; CNO, r=-0.075, p=0.175).

We next separated visits to the volatile end using a median split based on the recent volumes (mean of previous 3 visits). SWR rates were higher for a given current reward volume when the recent reward history was low (Fig. 3D-E, bottom panels). A Poisson GLM predicting SWR rate as a function of current volume, drug condition, and whether reward history was low or high (relative to the median for the session) revealed significant effects of current volume (z=13.86, p<10^-10), as expected, and reward history split (z=-2.23, p=0.026), but not drug condition (z=-1.05, p=0.29).

Finally, we separated combinations of current and previous volume into those with negative RPE (current < previous) and positive RPE (current > previous) and found mean-subtracted SWR rate was significantly affected by RPE sign (Fig. 3F; two-way ANOVA with drug and RPE sign, RPE sign: F[1,518]=6.42, p=0.012), but not drug (F[1,518]=0.3, p=0.582; drug X RPE sign, F[1,518]=0.07, p=0.785).

Given the lack of an effect of drug in experimental rats, we pooled all sessions (both animal groups, both drug conditions) in Experiment 2 to maximize experimental power and found similar results as in just experimental rats (Fig. S6). On top of a large increase in SWR rate with current volume at the volatile end (Fig. S6A-B), SWR rate was also significantly negatively correlated with the previous volatile volume, both at the volatile end (Fig. S6C) and at the stable end (Fig. S6E). Accordingly, SWR rates were significantly lower for negative RPE than for positive/non-negative RPE (Fig. S6D-F). Finally, recent reward history at the volatile end significantly affected SWR rate (Fig. S6G), with higher SWR rate when recent rewards were lower.

Taken together, SWR rate was modulated by reward volume changes consistent with RPE-like coding. This modulation did not require normal VTA dopamine signaling, at least in familiar environments. The lack of effect of VTA inactivation, even with frequent, large swings in value and SWR rate, corroborates the results from Experiment 1 that novel experiences are particularly susceptible to disruption, indicating VTA dopamine release is critical when learning new reward locations.

Previous work discovered the incidence rate of reverse replay, but not forward replay, was increased at locations with increased reward (Ambrose et al., 2016). We therefore analyzed single unit data collected in Experiment 1 (excluding experimental rat 2, who had a 6-tetrode recording drive) to determine whether this modulation of replay required VTA dopamine signaling. As previously observed (e.g., Ambrose et al., 2016), place cells in dCA1 had directional fields, such that the location a neuron was active while the rat moved in one direction on the track (e.g., “upward”) was often distinct from its activity when the rat moved in the other direction (e.g., “downward”). This directionality was apparent in both familiar and novel sessions, including in experimental rats with either saline or CNO (Fig. 4A). We found no effect of CNO on within-session field reliability, but significantly less reliability in novel compared to familiar sessions (Fig. S7A). Field similarity across running directions was slightly but significantly increased by both CNO and novelty (Fig. S7B).

Two place fields were defined for each neuron, one for each running direction, permitting Bayesian decoding methods to estimate both position and direction from neural activity (Fig. 4B-C). Sessions with accurate position and direction decoding during run, primarily due to sufficiently high cell yield, were included for replay analysis (Fig. S8; total sessions included: Experimental rats: novel saline, n=8; novel CNO, n=8; familiar saline, n=18; familiar CNO, n=23; Control rats: novel saline, n=12; novel CNO, n=11; familiar saline, n=16; familiar CNO, n=17).

Candidate replay events were periods of high population spiking activity while the rat was not running (z-scored spike count>3, minimum duration of 50 ms, rat velocity≤8 cm/s). We used a memory-less Bayesian decoder, with 40 ms decoding windows advancing by 5 ms steps, to estimate position and direction from neural activity.

Replays were defined as candidate events with spatial trajectories meeting a threshold for motion and minimum total posterior in one running direction map 13 (see Methods), with the running direction with greater posterior probability used to classify replay directionality, described below.

Forward replays were spatial trajectories moving across the track in the same direction as the rat when fields were calculated, e.g., moving “downward” with posterior probability in the “downward” place field map (Fig. 4B, left panel). Reverse replays were trajectories that moved in the opposite direction as the rat, e.g., moving “upward” with posterior probability in the “downward” place field map or vice versa (Fig. 4B, middle and right panels). We found forward and reverse replays occurred in all conditions, including in novel sessions with saline (Fig. 4B) or CNO (Fig. 4C). We therefore asked how novelty and drug condition influenced the effect of reward change on rates of reverse and forward replay.

Consistent with previous work (Ambrose et al., 2016), the rate of reverse replay was strongly modulated by the reward volumes on the track. Excluding novel CNO sessions for the moment, in all other conditions in experimental rats, when reward was larger at the Incr. end than the Unch. end (unequal reward), reverse replay was significantly increased at the Incr. end relative to when rewards were equal (novel saline: equal reward, 0.0019±0.0009 replay/s; unequal reward, 0.0121±0.004 replay/s; two-sample t-test, t(420)=3.235, p=0.0013; familiar saline: equal reward, 0.0033±0.0012 replay/s; unequal reward, 0.0128±0.0025 replay/s; two-sample t-test, t(907)=3.822, p=0.0001; familiar CNO: equal reward, 0.0033±0.001 replay/s; unequal reward, 0.0139±0.0022 replay/s; two-sample t-test, t(1153)=5.234, p<10^-6). The rate of reverse replay was not significantly increased at the Unch. end with unequal rewards in any of these conditions (all p>0.05). This led to a bias for reverse replay to preferentially occur at the Incr. end when rewards were unequal (Fig. 4D-E). In control rats, reward changes caused similar changes to the balance of reverse replay (Fig. 4H-I), with a significantly larger swing in reverse replay bias in novel sessions (three-way ANOVA with drug, novelty, and replay directionality: novelty X directionality, F[1,101]=9.04, p=0.0034; all other terms, p>0.05). Reward changes caused no consistent effects in the rates of forward replay in either animal group (Fig. 4F-G, Fig. 4J-K).

Conversely, in novel CNO sessions in experimental rats, reverse replay rate failed to be biased towards the larger reward location (Fig. 4D). With unequal reward, the rate of reverse replay did increase at the Incr. end (equal reward, 0.0053±0.0016 replay/s; unequal reward, 0.0116±0.0029 replay/s; two-sample t-test, t(332)=2.043, p=0.0419), but also increased somewhat at the Unch. end (equal reward, 0.0173±0.004 replay/s; unequal reward, 0.0249±0.0068; two-sample t-test, t(319)=1.019, p=0.309), leading to no consistent change in the difference at the two reward ends. This effect of VTA inactivation on the bias of replay between the reward ends when reward contingencies changed was specific to novel sessions and reverse replay (three-way ANOVA with drug, novelty, and replay directionality: drug X novelty X directionality, F[1,106]=4.64, p=0.0335; all other terms, p>0.05). VTA dopamine signaling was therefore required to direct reward-related changes in reverse replay, specifically in novel environments.