To reinforce SWR activity, our neurofeedback paradigm linked the online detection of large SWRs with positive reinforcement: delivery of a tone and food reward (Figure 1A). We embedded the neurofeedback requirement into a flexible spatial memory task (Figure 1B) that we have studied extensively (Gillespie et al., 2021), allowing us to compare neural and behavioral data from subjects that perform the task with neurofeedback manipulation to those without the manipulation component. All subjects performed the same overall task, which comprised structured trials consisting of three steps (Figure 1C): first, initiating a trial by nosepoking at the home port; second, maintaining a nosepoke at whichever center port illuminates (randomly assigned on each trial) for a duration determined by the port’s criteria (see below); and finally, choosing one of eight arms to visit before returning to the home port to initiate the next trial.

Here we present data from two cohorts of animals. In the manipulation cohort, one center port served as the neurofeedback port and the other served as the delay port. At the neurofeedback port, the subject had to maintain its nose in the port until a suprathreshold SWR was detected, which triggered delivery of a tone and food reward (Figure 1A, Figure 1—figure supplement 1A and B; and see Methods). The pre-reward period at the center port on neurofeedback trials is thus considered the ‘targeted interval’ for neurofeedback. At the delay port, the subject was required to maintain its nose in the port for an unpredictable amount of time. The delay period length was randomly chosen from the pre-reward duration of recent neurofeedback trials, such that the amount of time spent waiting at each of the center ports was approximately matched (Figure 1—figure supplement 1C; and see Methods). Delay trials thus serve as an internal control for each subject, allowing us to assess the specificity of any neurofeedback effect. In the control cohort, whose data have been presented previously (Gillespie et al., 2021), both center ports required the subject to maintain its nose in the port for an unpredictable delay period (see Methods), after which a tone and food reward was delivered.

The memory aspect of this task is required at the third step of each trial: at any given time, one of the eight arms provides a large food reward; the others provide nothing. Over multiple trials, the subject must sample arms to discover which one provides reward and remember to return to that location on subsequent trials. Occasionally, the rewarded arm changes, prompting the subject to return to sampling different arms to find the new goal location (Gillespie et al., 2021). Before implant, all subjects were pretrained on a delay-only version of this task, as summarized in the experimental timelines in Figure 1D. Once performance stabilized, a microdrive containing 30 independently movable tetrodes was implanted targeted to dorsal CA1 (Figure 1—figure supplement 1D). After implant, subjects were given several days to recover; during this time tetrodes were gradually lowered into the CA1 cell layer and the animal was briefly re-trained to perform the delay-only version of the task. The control cohort of subjects continued such performance for many days (Gillespie et al., 2021).

For subjects in the manipulation cohort, the neurofeedback component of the task was then introduced. Each day, 4–6 CA1 tetrodes were used for online SWR detection (Figure 1—figure supplement 1A; and see Methods). Because we use just a small subset of tetrodes, our online detection algorithm does not detect every SWR event identified by our offline detection strategy, which incorporates all CA1 tetrode data. This strategy provides two important benefits. First, it means that rewards are provided to SWR events on an unpredictable, variable schedule – an operant conditioning regime which drives robust, persistent task engagement (Ferster and Skinner, 1957). Second, we choose a different subset of tetrodes each day (see Methods) to ensure that subjects cannot learn to trigger the algorithm and earn reward by repeatedly producing a single firing pattern corresponding to specific units on those tetrodes. Initially, the threshold for online SWR detection is set to a low value, requiring only small SWRs to trigger reward. SWR size, as defined by the peak amplitude of the event, can be generally described by a long-tailed distribution, with larger events occurring less frequently than smaller events (Yu et al., 2017). Starting with a low threshold ensures that the animals received feedback relatively rapidly and remained engaged early in training.

Over subsequent days of training, the size threshold for SWR detection at the neurofeedback port was gradually increased, requiring the subject to generate larger and larger SWR events on neurofeedback trials in order to proceed with the task (Figure 1—figure supplement 1E). We then maintained the threshold at a high level for several additional days of behavior (‘stable period’).

In response to the increasing SWR detection threshold, subjects in the manipulation cohort successfully produced increasingly large trigger events at the neurofeedback port over the course of training (Figure 2—figure supplement 1A). This result confirms that our feedback criteria were enforced as designed and that subjects were capable of meeting the criteria. As the detection threshold was raised, subjects could have met the neurofeedback criteria simply by waiting much longer for increasingly rare suprathreshold events. This was not the case: instead, subjects produced large SWR events much faster than would have been predicted based on the initial prevalence of similarly sized events (Figure 2—figure supplement 1B). This suggests that subjects learned a strategy for generating large events that allowed them to meet the neurofeedback criteria more efficiently.

We reasoned that two candidate strategies could have enabled subjects to limit their wait times: learning to increase the average size of SWR events or learning to increase the rate of SWR event generation across all sizes. To evaluate these possibilities, we quantified differences both between the two cohorts and between trial types of the neurofeedback cohort. When evaluating the significance of differences between the neurofeedback and delay trial types within each manipulation cohort subject, we used ranksum tests, since this approach allows us to capitalize on the large numbers of measurements collected from each rat without assuming that they were normally distributed. To quantify systematic group effects of cohort, we used linear mixed-effects models (see Methods), since this approach allows us to appropriately account for correlations between measurements taken from each subject (Yu et al., 2022).

First, to assess whether subjects learned to directly increase the size of SWR events, we quantified the mean size of SWRs that occured pre-reward at the center ports on neurofeedback and delay trials during behavioral epochs from the stable period. Importantly, to account for the structure of the task, we excluded suprathreshold events detected by our online algorithm during both neurofeedback and delay trials (see Methods). These analyses revealed that subjects did not consistently increase SWR size: only two out of four subjects showed very slight but significant increases (Figure 2—figure supplement 1C). We also found no difference in SWR size between either trial type in the manipulation cohort and the control cohort (Figure 2—figure supplement 1C, inset). Instead, when we quantified the number of events of each size per behavioral epoch, we found that within the manipulation cohort, neurofeedback trials contained roughly twice as many events relative to delay trials for all SWR sizes (Figure 2—figure supplement 1D). These results rule out our first candidate strategy and indicate that subjects met the neurofeedback criteria efficiently by modulating SWR rate, not event size.

The increase in SWR rate pre-reward was associated with striking differences in when SWRs occurred both within and across cohorts that were evident even in individual trials (Figure 2A). To assess how SWR rate was changing during the time spent at the center ports, we calculated SWR rate in 0.5 s bins throughout the time subjects spent nosepoked at the center ports, aligning each trial to reward delivery (Figure 2B). We excluded the trigger SWR of neurofeedback trials from analysis, since it is guaranteed to occur just prior to reward delivery in each neurofeedback trial and causes a large spike in SWR rate. Subjects in the manipulation cohort all showed high, stable SWR rates before reward delivery, with neurofeedback trials showing higher SWR rate than delay trials. By contrast, subjects in the control cohort rarely generated SWRs during the pre-reward period but showed much higher SWR rates than the manipulation cohort after reward delivery.

To quantify these effects, we calculated the mean SWR rate over each portion of the time at the center ports (Figure 2C): the pre-reward period (left), the post-reward period (middle), and the total port period (pre +post; right). Indeed, we found that during the pre-reward period, SWR rates were approximately twice as high on neurofeedback compared to delay trials, and both trial types from the manipulation cohort had much higher rates than trials from the control cohort (Figure 2C, left). This effect was not dependent on the trigger SWR, as excluding suprathreshold events from both neurofeedback and delay trials did not change the result (Figure 2—figure supplement 1E). This finding indicates that the neurofeedback training caused far more SWRs to occur prior to reward delivery at both center ports in the manipulation cohort, with the most pronounced effect occurring on neurofeedback trials.

These patterns differed after the reward at the center ports, where SWR rate was slightly but significantly higher during delay trials compared to neurofeedback trials in the manipulation cohort subjects but was overall indistinguishable between the cohorts (Figure 2C, middle). Relative to the pre-reward period, SWR rates were maintained at a high level during neurofeedback trials, while they increased substantially during delay and control trials (Figure 2—figure supplement 1F). Note that the slightly lower SWR rate post-reward on neurofeedback trials compared to delay trials could not be explained by a refractory period following the trigger SWR (Figure 2—figure supplement 1G). When considering both pre- and post-reward periods together, the neurofeedback trials still showed higher SWR rates than delay trials within the manipulation cohort, but the differences between each trial type and the control cohort were not significant (Figure 2C, right). This surprising result suggested that the pre-reward effects on SWR rate in the manipulation cohort were followed by some compensatory effects post-reward that resulted in a net normalization of total SWR rate across cohorts.

Quantifying SWR count, rather than rate, provided further explanation for these surprising findings. We quantified the number of SWRs during pre-reward, post-reward, and total center port times (Figure 2D). Our SWR count results were completely consistent with the SWR rate findings during the pre-reward period (Figure 2D, left) and largely consistent when considering the total center port time (Figure 2D, right). However, we found striking differences during the post-reward period, during which we saw a clear reversal of the pre-reward effect on SWR counts: neurofeedback trials contained fewer SWRs than delay trials, and both trial types contained far fewer SWRs post-reward than trials from the control cohort (Figure 2D, center). Since post-reward SWR rates were not different between the cohorts, the count results indicated that the compensatory effect must involve the amount of time spent at the port post-reward.

Indeed, we found clear differences in the amount of time that subjects chose to spend at the port after reward delivery. Here it is important to remember that in contrast to the pre-reward dwell time, which is imposed by task constraints, the post-reward dwell time is not externally constrained. It was notable, therefore, that neurofeedback trials had slightly but significantly shorter post-reward periods than delay trials (Figure 2E). Even more strikingly, subjects in the manipulation cohort remained at the port for approximately half as long as subjects in the control cohort (Figure 2E, inset), despite all trial types receiving the same amount of reward. The difference in dwell time could not be explained by the control cohort having fewer days of task experience than the manipulation cohort, since post-reward dwell time tended to increase, rather than decrease, over subsequent days of performance within the control cohort (Figure 2—figure supplement 1H). Together, these findings suggest that both covert and overt processes regulate SWR generation. Animals learned to modulate their SWR rate when required during the pre-reward period of neurofeedback trials and did so covertly, without displaying obvious behavioral correlates. A subsequent overt change in behavior – leaving the well early – was then associated with lower numbers of SWRs post-reward in the neurofeedback cohort.

Of course, the subjects’ behavior during the time spent at the ports could have substantial impact on SWR generation processes. In particular, since SWRs are associated with periods of immobility, we wanted to ensure that the increase in SWRs prior to reward delivery during neurofeedback trials was not driven by differences in movement during this time. In general, the requirement for subjects to remain nosepoked forces subjects to remain in the same position; however, small head movements are possible. The mean speeds of the head-mounted LEDs while the subjects were nosepoked were below our cutoff for immobility (4 cm/s) for all trial types, however, we did observe a slightly but significantly lower mean speed during the pre-reward period on neurofeedback trials compared to delay trials and trials from the control cohort (Figure 3A). To assess whether this difference could affect SWR rates, we selected the quartile of neurofeedback trials with the highest mean speeds and compared them to the quartiles of delay and control cohort trials with the lowest mean speeds. Even with much higher mean speeds (Figure 3B), the neurofeedback trials showed higher SWR rates than control and delay trials (Figure 3C). This analysis suggests that movement differences during the time spent at the center ports cannot explain the increased SWR rates observed prior to reward during neurofeedback trials.

The surprising stability of total SWR rate and count at both center ports across the two cohorts, despite changes in when those SWRs occurred relative to reward delivery, suggests that some regulatory mechanisms keep SWR generation within a set range for the center port phase of this task. We therefore asked whether we could detect evidence for this regulation on a trial-by-trial basis. Strict regulation within a single trial would predict that, for instance, trials with many SWRs prior to reward would likely have fewer SWRs post-reward. However, we did not find evidence for such a relationship consistently: the correlation coefficients between pre-reward and post-reward SWR counts were positive in some subjects and negative in others, and always very small in magnitude, indicating a weak relationship (r=–0.084, 0.130. 0.118, and –0.061 and p=1.47 × 10^–7, 3.84x10^–6, 8.60x10^–9, and 6.01x10^–4, for each subject in the manipulation cohort, respectively). Similarly, we did not find consistent correlations between pre-and post-reward SWR rate nor dwell time. These results suggest that while compensatory regulation of SWR generation may be evident across hundreds of trials, high variability in SWR rate, count, and dwell time across trials may prevent consistent detection of this effect at the level of single trials.

SWRs are thought to support memory storage as a result of the specific patterns of spiking activity (replay) seen during these events (Gridchyn et al., 2020). Thus, we wanted to assess whether our SWR manipulation also increased replay during the targeted pre-reward period. As hippocampal CA1 neurons often fire reliably at particular locations in space (‘place cells’) while an animal moves through an environment, replay content has most often been studied with regard to its spatial content. Specifically, spatial content can be assessed by using a model that relates neural firing to specific locations in space. Inverting this ‘encoding model’ yields a ‘decoding model’ that can predict the spatial locations represented by the neural spiking within SWRs (Zhang et al., 1998).

We used a clusterless state-space model (Chen et al., 2012; Deng et al., 2016; Denovellis et al., 2021; Kloosterman et al., 2014) identical to the one we used previously to characterize the spatial content of replay for the animals in the control cohort (Gillespie et al., 2021). Briefly, we first project the 2D maze environment to 1D (Figure 4A). We next use times when the animal is moving through the maze to build a model that relates amplitude features of detected spikes to the location on the 1D maze where they were observed. We then can predict the spatial representation of spikes observed during movement (Figure 4B) and during SWRs (see Methods).

As expected, we observed a wide range of spatial representations during SWRs (Figure 4C). We define replay events as SWRs which predominantly contain spatial representations that are consistent with how a subject can experience the environment: those which represent a single location or a continuous trajectory through space (see Methods). These events frequently represent the subject’s current location (‘local’ replay; Figure 4C, left two events) as well as locations or trajectories that encompass distant regions of the maze environment (‘remote’ replay; Figure 4C, middle four events). In small subset of cases, SWRs contain a mixture of disjoint spatial representations with no single location clearly dominant (fragmented events; Figure 4C, right), which do not meet our replay criteria and are excluded from analysis.

Across a variety of measures, we found that neurofeedback training preserved replay content. In SWRs during the pre-reward period, there was no difference in the fraction of SWRs that qualified as replay between the two cohorts, and within each manipulation cohort subject, there was no consistent difference between neurofeedback and delay trials (Figure 4D).

Replay of remote locations has been proposed to play a particularly important role in updating or maintaining memory in the absence of direct experience (Gillespie et al., 2021; Gupta et al., 2010; Ólafsdóttir et al., 2015). Consistent with our SWR findings, we find much higher rates of remote replay during the pre-reward period in neurofeedback trials compared to delay trials within the manipulation cohort, and low rates of remote replay during this time in trials from the control cohort (Figure 4E). However, if we consider both the pre-and post-reward periods combined, we find no difference in remote replay rate between the two cohorts for either trial type (Figure 4F). Within the manipulation cohort, neurofeedback trials have slightly but significantly higher rates of remote replay than delay trials over the combined pre- and post-reward period.

Prior work demonstrated that replay events in this task were enriched for specific, relevant past experiences that change as a function of goal location (Gillespie et al., 2021), and we saw the same biases in replay content in the manipulation cohort. We classified remote replay events based on the behavioral significance of the maze arm they represented and used a generalized linear model to assess how behavioral categories influence the likelihood of replay of a given arm (see Methods). As previously seen in the control cohort, we observed consistently enhanced representation of previous goal locations in replay events during the pre-reward period in both neurofeedback and delay trials within the manipulation cohort (Figure 4G). Similarly, we replicated the reduced prevalence of replay representing the immediately previously visited arm and a slightly increased prevalence of replay of the arm associated with the upcoming choice in both trial types of the manipulation cohort. These results also remain unchanged when we analyzed replay events during both pre- and post-reward time at the center ports (Figure 4—figure supplement 1).

Together, these results indicate that while the timing of replay is altered by the neurofeedback training, the content and behavioral relevance is preserved.

Finally, we wanted to assess whether neurofeedback training would have an impact on the performance of the spatial memory task. Since SWRs and remote replay rate were generally indistinguishable between the manipulation and control cohorts when the total time at the center ports was considered, we did not expect to find a large difference in behavior between the two cohorts. Within the manipulation cohort, however, neurofeedback trials generally showed higher SWR and remote replay rates, providing an opportunity to measure any behavioral consequences of the neurofeedback training. Our previous findings (Gillespie et al., 2021), suggested that SWRs and replay do not drive trial-by-trial choice behavior in this task, so we did not predict that the increase in SWRs on individual neurofeedback trials would necessarily improve performance on those trials. On the other hand, we speculated that the requirement to generate SWRs could serve as a distracting or cognitively demanding additional aspect of neurofeedback trials that could impair subsequent arm choice on those trials, and if so, this would be an important factor to consider when designing future implementations of the neurofeedback paradigm.

We used several metrics to quantify task performance. First, during trials when the subject had not yet discovered a new goal arm (‘search’ trials), optimal behavior dictates that the animal should sample each arm without repeating visits to unrewarded arms until the goal arm is found. To measure performance on these trials, we quantified search efficiency: the fraction of search trials when the subject visits an arm not yet sampled. The manipulation cohort subjects show a slight improvement in search efficiency compared to control subjects, which only reached significance on delay trials (Figure 5A). However, rather than relating to the neurofeedback training, this difference could have been driven by the additional days of experience that the manipulation cohort has on the task, since the control cohort showed a subtle but consistent increase in this performance measure over time (Figure 5—figure supplement 1). Consistent with this possibility, we find no significant difference in this metric between neurofeedback and delay trials. We next asked whether ‘redundant’ search trials – when the subject chooses to sample an arm it has already visited – tended to be neurofeedback or delay trials. We found no bias toward either of the two trial types: neurofeedback trials accounted for approximately half of redundant search trials for each subject (Figure 5B), indicating that neither the neurofeedback nor delay trial type was consistently overrepresented among the trials with suboptimal behavior.

To quantify performance during trials when the goal arm had already been discovered by the subject and should be chosen again (‘repeat’ trials), we measured the fraction of subsequent trials during which the subject correctly visited the goal arm. We also saw no difference between the two cohorts or between neurofeedback and delay trials on this measure (Figure 5C). Within the manipulation cohort, we also found that error trials, when subjects do not choose to visit a known goal arm, were equally distributed between neurofeedback and delay trials (Figure 5D). Overall, these results suggest that on a single-trial timescale, the neurofeedback manipulation had neither a beneficial nor detrimental effect on behavior, in line with our previous findings (Gillespie et al., 2021).

Data from four male Long-Evans rats (Charles River RRID:RGD_2308852; aged 4–9 months; 450–550 g) comprise the neurofeedback cohort of animals included in this study. We also include data from a cohort of four additional male Long Evans rats used for a previous study who did not receive neurofeedback training (Gillespie et al., 2021). All animals were kept on a 12 hr light-dark cycle (lights on 6am-6pm) with ad libitum access to food (standard rat chow) in a temperature- and humidity- controlled facility. Beginning 1 week prior to and extending throughout behavioral training and data acquisition, rats were singly housed and food restricted to a maximum of 85% of free-feeding weight. All procedures were approved by the Institutional Animal Care and Use Committee at the University of California, San Francisco (Protocol # AN191130).

All analyses were performed using custom code written in Matlab 2020a (Mathworks) and Python 3.6.