We consider neural networks performing sequences of tasks. The neural networks are standard multi-layer perceptrons, with layers defined by the equation: Each layer output y is sum of inputs x weighted according to matrix w, subject to activation function ϕ. We define task performance in terms of loss functions denoted L. Here we use a supervised learning paradigm, making these losses explicit, but our results apply equally to reinforcement learning and unsupervised settings, where one can often consider learning in terms of implicit loss functions (indeed we demonstrate analogous effects in RL contexts in the appendices, together with a fuller explanation of the application to RL).

The gradient of network parameters with respect to a loss can be computed using the chain rule: This equation is summed over all indices. Performing some partial sums, as shown explicitly in the appendix, gives an outer-product in terms of a column vector g and the layer’s input x: In this equation the column vector g summarizes how y should change, given input x, for the loss to change maximally. The matrix D_ϕ is a diagonal matrix of the local slopes of ϕ at each component of z. The gradient of the loss with respect to y (the term dL/dy) captures how downstream processing makes use of the local post-synaptic firing rates, ultimately manifesting in a loss value. In the reinforcement learning and unsupervised cases, this g is typically accumulated through experience, whereas supervision and backpropagation implicitly determine it. This representation of the gradient is striking in relation to neuroscience, because the x vector denotes how a given neuron’s inputs should be re-weighted (i.e., plasticity of dendritic processes and receptive fields), whereas the g vector denotes how a layer of neurons’ collective output is changed (i.e., plasticity of population response patterns). In particular, this parallels the fact that individual neurons can independently exhibit both gain (or excitability) changes and re-weighting of their inputs (see, for example, Cichon & Gan 2015; Yang et al. 2014; Sehgal, Filho, et al. 2021; Scott & Frank 2022). As such, processing can change when population responses change, inputs are re-weighted with respect to one-another, or both. For example, a weight change Δw = uv^T, can involve input reweighting (changes in the row space of w arising from v), population response changes (differences in the column space of w arising from u) or both. We hypothesized that this would allow interference and generalization between tasks to be managed at these two distinct loci, and we show that this is the case below.

For a weight change to impact performance, by definition, it must have a non-zero inner product with the gradient of a loss. Indeed, the loss gradient is defined to be the direction in weight space along which that loss increases maximally. Hence an orthogonal change in weights (i.e., in the loss’ null-space) will neither increase nor decrease the loss. If we consider two rank-1 weight changes that are outer products (such as gradients for two tasks), denoted and , then these have an inner product: This equation shows that both the input and population response components of plasticity must overlap for interference or generalization to occur. Intuitively, gradient updates interfere with one another when they change network responses to the same input in opposite ways, for example when u₁ = g₁ = −g₂ = u₂ and v₁ = x₁ = x₂ = v₂. Updates generalize across tasks when they change the network in similar ways, such as when u₁ = g₁ ∝ g₂ = u₂ and v₁ = x₁ = x₂ = v₂. Both situations are illustrated in figure 2A, using the minimal example of trying to learn two conflicting (non-proportional) input-output mappings using one input and one output neuron (see the caption for further details). In multi-dimensional networks, interference occurs when weights are changed in opposite directions by receptive field plasticity, population response plasticity, or both. These situations are illustrated in figure 2B and 2C, and described further in the caption. Note, however, that if a network only changes responses to two completely distinct stimuli, as when v₁ · v₂ = x₁ · x₂ = 0, then no interference occurs. The same is true when the stimuli themselves are identical, but their processing is completely segregated.

Observing the facts above about gradients, interference, and generalization, along with the many biological plasticity controllers, we formalize a coordinated eligibility theory of plasticity. Note that “eligibility” refers to the capacity for plasticity, whereas plasticity refers to synaptic weight (and hence network) change itself. In this theory, plasticity takes the form: Here, u and v are generic vectors, α is a scalar, and i is an index with an arbitrary range.

Many specific forms of plasticity are instances of equation (4). For example, when there are as many terms i as synapses, and u_i and v_i are arbitrary, we can capture every possible weight update. When i = 1, u = −g and v = x we recover gradient descent. Other choices interpolate between these two extremes of flexibility, and other canonical models can also be described using equation (4). Setting i = 1, (with r denoting “reward”), , and v = x gives a REINFORCE algorithm (Williams 1992), which can approximate gradient descent by estimating g over time. Specifically, in this latter case, population response variability drives sample-based integration of the gradient term (α and combine in expectation to form g). By generalizing this classic REINFORCE model to use non-isotropic sampling (Cov(Δy) ≠ I), one arrives at a coordinated eligibility model that effectively projects g, which can have much lower sample complexity, along with some of the geometric properties we discuss here.

More broadly, a reasonable set of models to consider might take α, u, and v, to be functions of g (without necessarily requiring g to be known explicitly) and x, the weight update to have many fewer degrees of freedom than n× m for w ∈R^n×m, and the update itself to be distinct from the loss gradient of some task at hand. We investigate several such models below. Specifically, in our simulations we take α to be 1, and the factors u and v to be functions of the gradient components g and x. These gradient components are projected onto different subspaces according to task demands by matrices P and Q, and normalized (projected onto the unit sphere via transform S). Mathematically: This is one of the simplest coordinated eligibility models, but is flexible enough to avoid interference and promote generalization. Specifically, neglecting some technicality related to normalization, the inner product between two weight updates (with subscripts 1 and 2) will then be: The column spaces (or, roughly, outputs) of the transformation matrices can therefore be used to increase or decrease either quantity in parentheses.

There are many ways these transformations could be manipulated biologically, but we observe that simple, surprise-based, unsupervised metaplasticity can manage interference and generalization. (Both unsupervised and surprised based metaplasticity are well established empirically. See, for example, Yagishita et al. 2014; Jaskir & Frank 2021; Scott & Frank 2022) In our simulations below, networks compute surprise over inputs and feedback signals, in order to determine when and how to change the transforms P and Q. This allows them to perform patternseparation (in the high-surprise case) or integration-based coarse-coding (in the low-surprise case) of plasticity components u and v, by progressively removing dimensions from the column spaces of P and Q.

To quantify the impacts of coordinating eligibility, we first simulated one of the simplest continual learning problems, online linear regression. Simulating a linear regression problem had the advantages of allowing us to use an easy-to-understand linear neural network, and therefore to compare the network’s solution at each time to the true (minimum norm) solution in terms of both the network’s function and its weights. We demonstrate below that our coordinated eligibility model converges to the true optimal solution for the whole curriculum, whereas gradient descent does not (despite learning locally optimal solutions for each task, sequentially). By comparing plasticity models, this analysis also reveals important properties of both learning algorithms.

Our linear regression problem was composed of 80 input-output associations (“tasks”) with 100-dimensional unit-normal input vectors and 20-dimensional unit-normal target vectors, drawn at uniform angles (figure 3A-3B). These pairs were learned completely and sequentially in one sweep through the data (figure 3C), using either gradient descent or the coordinated eligibility model. A sweep was defined as reduction of the RMSE for a task from an initial value of 1 to a criterion value of 0.05. Mathematically, this resulted in neural networks approximately solving the ordinary least squares regression equations:

The minimal norm solution (and hence the optimal network weight matrix) is given via a multiplication of T (a matrix of target vectors) by the pseudo-inverse of X (a matrix of input vectors): This equation is appropriate regardless of the number of tasks (columns of T and X) the network has seen, and therefore defines an optimal solution after every individual task is learned.

To solve this problem, we simulated a single-headed, single-layer linear network with a squared-error loss. (We also obtain similar results in reinforcement learning simulations using categorical action choice, cross-entropy loss, and nonlinear activation functions, however, which we discuss in the appendices.) In this setting the transformation from weights to a loss value has a large null space, meaning many weights are compatible with solving the task (which is why the weight matrix is a pseudo-inverse rather than an inverse). As a result, upon learning the first input-output pair, the weights “listening to” every other input dimension are free to vary, because the first input has no projection along these. Each of these additional dimensions is constrained only once it is associated with an input, and a learning algorithm can therefore proceed by “listening” only to unique dimensions in new inputs (learning with orthogonal input plasticity vectors u) to avoid interference. Given these observations, we set the network’s surprise-based metaplasticity to detect input change-points (using the surprise function in figure 3E), and to progressively remove dimensions of input plasticity when these change-points occurred (figure 3D). How and when networks would know to use this exact form of metaplasticity is an interesting open question, but we introduce it here to demonstrate our points about the geometry of our plasticity factors. We expected this geometry to produce a path toward the optimal solution in weight space that would stay within the solution manifolds identified for previously learned tasks (figure 3F).

We compared the properties of our coordinated eligibility model with gradient descent by examining training error, forward and backward transfer, and network weights, averaged over 200 simulations (figure 3G-K). Training error at the end of training for each task was consistently just below criterion, per expectations, with the margin decreasing over task number, consistent with networks taking steps which were less direct relative to gradients as fewer dimensions of the input space became available for learning (figure 3G). Backward transfer was assessed by testing each task after training the entire curriculum. Networks using gradient descent completely unlearned task 1 by this time, and showed graded forgetting as a function of time-since training for each task (figure 3F). By contrast, the coordinated eligibility model showed minimal interference (figure 3H). We also observed that both gradient descent and the CEM introduced significant negative forward transfer (figure 3I). That is, solving tasks made the initial, untrained error on new tasks larger. This is expected, because each task defines a solution space of weight matrices, and the intersection of a set of these task solution spaces gets further from the origin as their number increases. Because the tasks were normalized, with unit-normal inputs and targets, predicting the 0 vector for new inputs will result in an error of 1, and the further from the origin new untrained task outputs are, the larger the initial error is likely to be. This led us to suspect that gradient-descent was under-fitting the data, and remained in the vicinity of the origin by virtue of moving directly towards each new task’s individual solution subspace.

To test this hypothesis about the nature of the forward transfer problem, we computed the norm of the network’s weights after completing every task, and similarly checked the distance from (curriculum) optimality at each point. We observed that the weight norms under gradient descent were consistently low, relative to the norm of the full solution, whereas the CEM weight norms grew to match this norm (figure 3J). Intuitively, this resulted in the gradient-based solution becoming more distant from the optimal solution as more tasks were completed (figure 3K).

Because of the symmetry of relationship (3) between population-response and receptive-field changes, interference and generalization can also be manipulated by changing population responses. Indeed, there are some data suggesting this may occur in the brain, with memories being increasingly segregated according to their distance in time as a result of (for example, CREB-based) excitability drift (Sehgal, Zhou, et al. 2018; Lisman et al. 2018). Here, we model a complementary (and analogous) situation to the previous simulation, involving population-response based pattern separation, using task-dependent readouts. We describe a pure form of this scenario below, in which interference can only be managed via population response plasticity, because network inputs don’t change. As such, our simulation is a model of purely contextual behavior, with context inferred based on network history.

Specifically, we simulate a continual learning scenario in which the input is constant, but target outputs change across tasks. To do so, we used a single linear layer with a constant scalar input and 100 response neurons (figure 4A). The response neurons were subject to a sequence of unit-random, 100 dimensional readout weights, one defined for each subsequent task, of which there were 80. Without loss of generality, each readout had a target response of 1, defining a 100 dimensional target at the layer output (figure 4B). Networks received each input, were trained to produce the paired target (up to an SSE of 0.05), and then were trained on subsequent input-target pairs, until the whole curriculum had been seen (4C). Mathematically, jointly solving all tasks in the curriculum is equivalent to solving the following equation for w:

As with the simulation above, this is a standard linear regression problem. Applying a continual learning algorithm will therefore approximate the solution: To remove interference in this setting, networks computed surprise over feedback change-points (figure 4D), using a logistic curve over output-plasticity cosine similarity (i.e., in a manner analogous to the previous simulation) (figure 4E). As with the previous simulation, this surprise signal was used to orthogonalize ongoing plasticity from previous plasticity, following these change-points.

After training networks to solve this sequence of tasks, we observe that backward transfer again completely abolishes earlier learning under gradient descent (figure 4H). Here, too, we have averaged over simulations (400), and again, the coordinated eligibility model does not suffer from negative backward transfer (figure 4H). Forward transfer results are also analogous to the previous simulation (figure 4I), and appear to be explained by the same difference in the algorithms’ abilities to effectively leave the origin of the networks’ weight spaces (figure 4J-K). As with our previous simulations, we note that we obtained similar results in reinforcement learning simulations (also discussed in the appendices).

A network that perfectly pattern separates all learning will completely avoid interference, at the potential cost of also removing positive transfer between tasks (i.e., generalization). By contrast, a network that coarse-codes or generalizes receptive field plasticity will produce generalized learning, while potentially risking negative transfer. In any given setting, there are therefore sets of desirable basis functions for receptive-field plasticity, and sets of undesirable ones. In the previous section, we addressed the removal of undesirable generalization (i.e., interference) by restricting plasticity, and in this section, we explore the promotion of desirable generalization via coarse-coding. In the pattern-separation simulations above, our network had a built-in assumption that inputs with sufficiently different angles represented different tasks, defining boundaries across which learning should be separated. However, there are many complementary situations in which superficially dissimilar states should instead be considered equivalent in a more abstract sense. This is typically studied in the context of state abstraction and task-set clustering across contexts in reinforcement learning Lehnert et al. 2020; Collins & Frank 2013, whereby such abstraction is needed for generalization.

To motivate this situation, consider the case of object affordances. Many classes of objects are loosely defined by their uses, such that (for example) the category of mugs is a set of things one drinks from. Such uses create natural categories for generalizing learning. If we interpret a target network output as an action, use, or label, then the inputs which should prompt a given output can often be inferred by learning the input transformations which the target output is invariant to. For example, if I see several distinct mugs, I can note that those physical transformations taking one mug to another should generally leave their shared property of being open-top vessels intact; this will allow me to drink out of the transformed objects. Knowing this set of transformations will increase my ability to say which other sorts of objects might be mugs, and therefore to generalize inferences or associations over this class. In this case, learning (plasticity) could be coarse-coded by yoking receptive-field changes v (which represent changes to how mug features are detected) for a second labeling task over category members from the first (i.e., according to similarity over vectors u, representing the mug-category response). For example, by observing that I can’t drink coffee out of a particular floppy mug, I may be justified in learning that “can be floppy” is generally false for open-top-drinking-vessels broadly, and therefore other mugs specifically. A similar plasticity-component logic would apply in other generalization scenarios, and structurally equivalent forms of generalization have been studied empirically in state abstraction and task-set clustering situations Collins & Frank 2016).

In analogy with this example, we simulated a continual learning problem in which object classes had (by definition) the same target outputs, and were seen in blocks. During the first block, networks learned one distinct target output for each group of inputs. We refer to this first block as an association block. Target similarity, computed as cosine angle, was then used to induce a similarity measure on inputs, which itself defined coarse-coded input plasticity in a subsequent block of trials (a generalization block). In this second block of trials, we model a new group of tasks using a separate regression head, and demonstrate that the coordinated eligibility model is able to effectively transfer learning within classes defined by the first block.

Concretely, we simulated two linear regression problems, using a network with 100 dimensional inputs, 20 dimensional outputs, and two readouts (figure 5A). All of these were random unit vectors (figure 5B). Inputs during the association task (“object exemplars”) were randomly paired (and hence had no intrinsic similarity), and pairs had the same associated target (“object property or class”). During the association curriculum, networks were trained to produce the correct target for each object, sequentially (figure 5C). Surprise was computed over targets (technically, population response feedback) to establish class-presentation change-points. Future receptive field plasticity was then coarse-coded over the inputs that had been seen during the previous low-surprise period (figure 5D). Subsequently, during a generalization task, new targets were generated for each existing pair of inputs, on a new readout (i.e., old inputs/classes were conserved) (figure 5E). These new input-target associations were trained (sequentially, in a curriculum) on one of each pair of inputs and tested on the other (figure 5F). Performance was assessed at each stage of the task (figure 5G), verifying that both gradient descent and the coordinated eligibility model performed identically at initialization, after association training, and after training during the generalization phase. The algorithms differed in terms of their generalization however (by construction), with coarse-coded plasticity ensuring that held-out inputs were properly associated with trained targets during the generalization phase.

As with pattern separation, pattern completion (or coarse-coding) can also be applied over population responses. We illustrate this by simulating a network performing tasks analogous to the RF pattern completion case just discussed, with PR and RF roles reversed. Specifically, we first simulated a curriculum in which pairs of targets (on independent readouts) were associated (via training) with individual inputs (figures 6A-C). At every input transition, input-based surprise was used to coarse-code plasticity for the previously seen targets (which shared the previous input) (figure 6D). As previously, networks made use of this coarse-coding during training on a generalization curriculum (figure 6E-f). During these tasks, networks were given a sequence of new inputs, and were required to produce one out of of each of 40 pairs of targets. Plasticity during training was constrained to the coarse-coded dimensions discovered previously. Testing the network on the held-out targets verified that it properly produced each second target, given each held-out readout, and that performance at other phases of learning was otherwise identical (figure 6G).

Moving beyond simple subspace restrictions in our coordinated eligibility theory, we examined tasks with compositional inputs, requiring compositional responses. Many objects can be described as bundles of features of varying statistical interdependence, and when learning a task, subsets of these features might dictate separate elements of an appropriate response (figure 7B). Brain regions with convergent input from diverse pre-synaptic partners would be well served by plasticity mechanisms that could manage responsiveness to the compositional building blocks in such mixed inputs, and the resulting compositionality would also facilitate combinatorial generalization (as in e.g. O’reilly 2001). While there are circuit level architectural mechanisms that can gate attention to specific features in order to govern responding (Frank & Badre 2012; Niv et al. 2015), just as there are for pattern separation vs completion O’Reilly & Norman 2002, we again consider here how learning rules themselves can implement compositional inductive biases. As we show, associations between population-response plasticity and receptive field plasticity can produce compositional learning.

Our basic insight is that (unlike the earlier simulations) population response plasticity need not be independent of receptive field plasticity. Indeed, there is little reason to think these two aspects of response change are, in general, independent. Instead, as we consider here, subspaces of candidate population response changes (i.e., span({u_i}), for some set u_i) may be associated with subspaces of receptive field changes (span({ v_j }), for some set v_j), leading to weight update matrices in span , for u_i ∈ {u_i} and v_j∈{ v_j}. When subspace associations such as these exist, the building blocks of representation changes are inherently yoked to those of receptive field changes. Learning multiple such updates simultaneously produces compositional learning, along with dimensionality reduction; instead of learning on the complete bipartite graph of possible population-response and receptive field updates, some smaller eligibility graph (with fewer edges) is thereby assumed.

As a biological example, striatal neurons multiplexing sensory information from different areas may have different population-level representations related to different sensory modalities, and it may be useful to explore holistic changes in these representations in a way that keeps learning somewhat separated by (for example, internal to) different modalities. This would imply yoking a representation change associated with one modality to a set of receptive field changes “listening” to that same modality. As noted above, we are not suggesting that other mechanisms cannot accomplish similar goals. Instead, we show that synaptic plasticity could be either pre-configured or tuned online to perform such functions in some circuits.

To explore this idea, we simulated simple, single-layer linear networks (figure 7A) solving compositional task sets. Tasks were constructed by generating random mathematical bases for network inputs and hidden layer responses, and associating elements of each basis with elements of the other. This defined how features (latent inputs, the input basis components), translated into latent task demands (the output basis components) in a 1-to-1 fashion (figure 7B). Since basis components for each feature were unit vectors, the inputs for each feature were generated as compositions of rotations and reflections (elements of the orthogonal group O(n)). The desired input-to-hidden transformation was thus an orthogonal matrix , with input feature vectors encoded in A and task demands in B (figure 7B). Stimuli were generated as compositions of the basic features, with 10 random loadings per task, and target outputs were generated as compositions of graded responses to these input features (using the same loadings) (figure 7B). The strength of an input feature therefore determined the extent to which its associated output was required. Because there are 40-choose-10 features per task (which precludes using averages over draws to avoid introducing non-uniform sampling effects in our results), we selected stimuli to include all 40 features with equal frequency (figure 7C). Specifically, we did so by circularly shifting feature loadings to define a curriculum. These tasks were then learned sequentially (figure 7D), with plasticity in our coordinated eligibility model restricted to feature-demand subspaces which included the correct associations (figure 7E). We ran a series of such simulations, varying a “linking number” parameter, l, which determined the degree of plasticity restriction. This parameter varied between all-to-all (gradient) connectivity and 1-to-1 (completely accurate prior associations), and allowed us to examine the impact of restriction on learning.

Our results are illustrated in figure 7F-I, showing performance of the coordinated eligibility model relative to GD, averaged over 60 random task and network initializations. The coordinated eligibility model converges to its asymptotic error in fewer passes over the data relative to gradient descent, with less interference (figure 7F). This can be observed in the weights of the network, as relevant weights (defined as those connecting input features with their associated output demands) rapidly increase and plateau for the coordinated eligibility model, whereas gradient descent increases them much more slowly (figure 7G). Similarly, irrelevant weights (or spurious associations) grow much more rapidly for gradient descent, then decay, whereas the coordinated eligibility model suffers less from this problem (figure 7H) (the scales differ because there are many more irrelevant weights). Linking number impacted average cumulative error as expected, with even limited dimensionality reduction providing significant decreases in cumulative training error (figure 7I). Thus, feature matching via coordinated receptive field and response eligibility can drastically improve learning in compositional tasks relative to gradient descent.

All simulations were carried out using PyTorch, running custom network layers. Code for these simulations can be found at www.github.com/DanielNScott/beyond-grads. Network layers implementing coordinated eligibility used forward and backward hooks to compute surprise as a function of inputs during network forward passes or as a function of supervision mismatch during backward passes. To do so, networks maintained histories of inputs and supervision, generally for one time-step of training, and compared new inputs or supervisory signals with those seen just prior. To compute gradient manipulations, layers also maintained state in the form of projection matrices. For pattern-separation simulations, these had previously seen dimensions removed, whereas for pattern-completion simulations, dimensions were merged by removing the independent (to-be-merged) dimensions and replacing them with one (joint) dimension. In our compositional plasticity simulation (simulation 5), the independent projections were not learned, being directly implemented instead.

During blocked learning, the appearance of a new task can be detected as a change-point in inputs or feedbacks, and likewise, these can be grouped over time according to contiguity. Empirical evidence suggests the locus coeruleus is heavily involved in signaling such (unvalenced) surprise (reviewed in Sara & Bouret 2012), whereas midbrain dopamine projections signal both valenced (reward-prediction error) and unvalenced surprise, associated with reinforcement learning and stimulus-stimulus association learning, respectively (Diederen & Fletcher 2021; Schultz et al. 1997). The neuromodulators these areas release, norepinephrine, and dopamine, have major impacts on plasticity (reviewed in Scott & Frank 2022). Our networks compute such surprise signals, and use them to perform pattern-separation (in the high-surprise case) or integration-based coarse-coding (in the low-surprise case) on plasticity components u and v, by changing the transformations P and Q over time.

Specifically, each layer computes a running average µ of its inputs over time. When new inputs arrive, surprise s is computed as the logistic function applied to cosine-angle dissimilarity between new and old inputs. That is: This surprise function is shown in figure 3E. The parameters θ₀ and m define the shape (cutoff location and cutoff sharpness) of the sigmoid. The running mean computation is weighted by surprise, such that surprising events more fully replace the running average, and the transform T_u has the most-recent running mean removed according to surprise as well. That is, for a normalized mean . An analogous computation is performed on the v vectors, as we describe in the relevant section below. As we show below, these operations allow our networks to flexibly manage interference and generalization across tasks.

For each repetition of simulation 1, we initialized our network’s input eligibility transform as the identity matrix, T_u = I. After training on input x_t during the t-th task, T_u was updated as , where u_t represents the vector scaled to have unit norm, ρ is a mixture parameter, s is surprise (which was very close to 1 at task transitions, and 0 otherwise), and was the average input over the immediately preceding period of low-surprise (i.e., , the input during the t-th task).

Plasticity in simulation 2 was treated analogously to plasticity in simulation 1, with the transform T_v being updated according to surprising output update supervision signals g_t, rather than T_u being updated according to surprising inputs.

For simulation 5, we generated orthogonal bases A and B for the inputs and targets by randomly sampling multivariate normal distributions and applying the Gram-Schmidt procedure to orthogonalize them. The columns of A served as vectors mapping latent features to observed inputs, and the columns of B as vectors mapping latent demands to observed targets. We generated compositional input stimuli by circularly shifting (“rolling”) a fixed vector with C non-zero random weights w_i, and using each of these as weights applied to the latent features. The same procedure was used to generate target outputs. For example, if stimuli are denoted s_i, then in the n = 3, C = 2 case, we would have a set of inputs: Juxtaposition here represents matrix multiplication, and parenthesis denote vectors (not indexing). Targets were generated using the same set of weights and the demand-target map B instead of the feature-input map A. Matching the inputs and targets shows that the learned optimal weight matrix will be , because this matrix first inverts the feature-to-input map (A) then applies to feature-to-demand map (I), and finally the demand-to-target map (B). This weight matrix was learned directly in the gradient descent simulations, and indirectly, in the latent feature-demand space, using the CEM. Since the basis transormations are arbitrary, both cases are equivalent to learning the identity map.

Linking numbers were free parameters that we used to encode prior information about the target weight transformation. In a simulation with linking number l, for each index t, plasticity was restricted to retain only terms between features [t − l, t + l] and their associated demands. For example, given a linking number of 1, plasticity would include terms in the feature-demand space. A linking number of l = 0 therefore provided complete information about which features should be associated with which demands , again, in the feature-demand space), such that only the strengths of these relationships required learning. A linking number such that 2l + 1 ≥ n was equivalent to gradient descent (, since p_i and q_j span all feature-demand basis pairs). These restrictions can be accomplished at by taking 2l + 1 projections of g and 2l + 1 projections of x, then summing all of their pairwise products, or they can be implemented parsimoniously using a mask M. The t-th row of this mask is then a vector of length n with 2l + 1 ones centered on the t-th component, with zeros elsewhere. We took the latter approach, transforming gradients into the feature-demand space, masking them, then transforming them back. Denoting the full gradient G, this meant that at each backward pass of the CEM network, we set G↦ B(M ⊙ B^T GA^T)A.