In a computer-based task, subjects are asked to predict which of two rods the lightning will strike. On each trial, the subject first selects by a key press the left- or right-hand-side rod presented on screen. A lightning symbol (which is here the stimulus) then randomly strikes either of the two rods. The trial is a success if the lightning strikes the rod selected by the subject (Figure 1a). The location of the lightning strike (left or right) is a Bernoulli random variable whose parameter p (the stimulus generative probability) we manipulate across blocks of 200 trials: in each block, p is a multiple of 0.05 chosen between 0.05 and 0.95. Changes of block are explicitly signaled to the subjects: each block is presented as a different town exposed to lightning strikes. The subjects are not told that the locations of the strikes are Bernoulli-distributed (in fact no information is given to them regarding how the locations are determined). Moreover, in order to capture the ‘stationary’ behavior of subjects, which presumably prevails after ample exposure to the stimulus, each block is preceded by 200 passive trials in which the stimuli (sampled with the probability chosen for the block) are successively shown with no action from the subject (Figure 1b); this is presented as a ‘useful track record’ of lightning strikes in the current town. (To verify the stationarity of subjects’ behavior, we compare their responses in the first and second halves of the 200 trials in which they are asked to make predictions. In most cases we find no significant differences. See Appendix.) We provide further details on the task in Methods.

The behavior of subjects varies with the stimulus generative probability, p. In our analyses, we are interested in how the subjects’ predictions of an event (left or right strike) vary with the probability of this event, regardless of its nature (left or right). Thus, for instance, we would like to pool together the trials in which a subject makes a rightward prediction when the probability of a rightward strike is 0.7, and the trials in which a subject makes a leftward prediction when the probability of a leftward strike is also 0.7. Therefore, throughout the paper, we do not discuss whether subjects predict ‘right’ or ‘left’, and instead we discuss whether they predict the event ‘A’ or the complementary event ‘B’: in different blocks of trials, A (and similarly B) may refer to different locations; but importantly, B always corresponds to the location opposite to A, and p denotes the probability of A (thus B has probability 1−p). This allows us, given a probability p, to pool together the responses obtained in blocks of trials in which one of the two locations has probability p. One advantage of this pooling is that it reduces the noise in data. Looking at the unpooled data, however, does not change our conclusions; see Appendix.

Turning to the behavior of subjects, we denote by p¯(A) the proportion of trials in which a subject predicts the event A. In the equiprobable condition (p=0.5), the subjects predict either side on about half the trials (p¯(A)=.496, subjects pooled; standard error of the mean (sem): 0.008; p-value of t-test of equality with 0.5: 0.59). In the non-equiprobable conditions, the optimal behavior is to predict A on none of the trials (p¯(A)=0) if p<0.5, or on all trials (p¯(A)=1) if p>0.5. The proportion of predictions A adopted by the subjects also increases as a function of the stimulus generative probability (Pearson correlation coefficient between p and p¯(A), subjects pooled: .97; p-value: 3.3e-6; correlation between the ‘logits’, ln⁡p1−p: 0.994, p-value: 5.7e-9.), but not as steeply: it lies between the stimulus generative probability p, and the optimal response 0 (if p<0.5) or 1 (if p>0.5; Figure 2a).

The sequences presented to subjects correspond to independent, Bernoulli-distributed random events. Having shown that the subjects’ predictions follow (in a non-optimal fashion) the stimulus generative probability, we now test whether they also exhibit the non-independence of consecutive trials featured by the Bernoulli process. Under this hypothesis and in the stationary regime, the proportion of predictions A conditional on the preceding stimulus being A, p¯(A|A), should be no different than the proportion of predictions A conditional on the preceding stimulus being B, p¯(A|B). (Here and below, p¯(X|Y) denotes the proportion of predictions X conditional on the preceding observation being Y, and not on the preceding response being Y. For the possibility that subjects’ responses depend on the preceding response, see Methods.)

In other words, conditioning on the preceding stimulus should have no effect. In subjects’ responses, however, these two conditional proportions are markedly different for all stimulus generative probabilities (Fisher exact test, subjects pooled: all p-values < 1e-10; Figure 2a). Both quantities increase as a function of the stimulus generative probability, but the proportions of predictions A conditional on an A are consistently greater than the proportions of predictions A conditional on a B, i.e., p¯(A|A)−p¯(A|B)>0 (Figure 2b). (We note that because the stimulus is either A or B, it follows that, symmetrically, the proportions of predictions B conditional on a B are consistently greater than the proportions of predictions B conditional on an A.) In other words, the preceding stimulus has an ‘attractive’ sequential effect. In addition, this attractive sequential effect seems stronger for values of the stimulus generative probability closer to the equiprobable case (p = 0.5), and to decrease for more extreme values (p closer to 0 or to 1; Figure 2b). The results in Figure 2 are obtained by pooling together the responses of the subjects. Results derived from an across-subjects analysis are very similar; see Appendix.

The attractive effect of the preceding stimulus on subjects’ responses suggests that the subjects have not correctly inferred the Bernoulli statistics of the process generating the stimuli. We investigate the hypothesis that their ability to infer the underlying statistics of the stimuli is hampered by cognitive constraints. We assume that these constraints can be understood as a cost, bearing on the representation, by the brain, of the subject’s beliefs about the statistics. Specifically, we derive an array of models from a framework of inference under costly posteriors (Prat-Carrabin et al., 2021a), which we now present. We consider a model subject who is presented on each trial t with a stimulus xt∈{0,1} (where 0 and 1 encode for B and A, respectively) and who uses the sequence of stimuli x1:t=(x1,…,xt) to infer the stimulus statistics, over which she holds the belief distribution P^t. A Bayesian observer equipped with this belief P^t and observing a new observation xt+1 would obtain its updated belief Pt+1 through Bayes’ rule. However, a cognitive cost C(P) hinders our model subject’s ability to represent probability distributions P. Thus, she approximates the posterior Pt+1 through another distribution P^t+1 that minimizes a loss function L defined as(1)L(P^t+1)=D(P^t+1;Pt+1)+λC(P^t+1),

where D is a measure of distance between two probability distributions, and λ≥0 is a coefficient specifying the relative weight of the cost. (We are not proposing that subjects actively minimize this quantity, but rather that the brain’s inference process is an effective solution to this optimization problem.) Below, we use the Kullback-Leibler divergence for the distance (i.e. D(P^t+1;Pt+1)=DKL(P^t+1||Pt+1)). If λ=0, the solution to this minimization problem is the Bayesian posterior; if λ≠0, the cost distorts the Bayesian solution in ways that depend on the form of the cost borne by the subject (we detail further below the two kinds of costs we investigate).

In our framework, the subject assumes that the m preceding stimuli (xt−m+1:t with m≥0) and a vector of parameters q jointly determine the distribution of the stimulus at trial t+1, p(xt+1|xt−m+1:t,q). Although in our task the stimuli are Bernoulli-distributed (thus they do not depend on preceding stimuli) and a single parameter determines the probability of the outcomes (the stimulus generative probability), the subject may admit the possibility that more complex mechanisms govern the statistics of the stimuli, for example transition probabilities between consecutive stimuli. Therefore, the vector q may contain more than one parameter and the number m of preceding stimuli assumed to influence the probability of the following stimulus, which we call the ‘Markov order’, may be greater than 0.

Below, we call ‘Bernoulli observer’ any model subject who assumes that the stimuli are Bernoulli-distributed (m=0); in this case the vector q consists of a single parameter that determines the probability of observing A, which we also denote by q for the sake of concision. The bias and variability in the inference of the Bernoulli observer is studied in Prat-Carrabin et al., 2021a. We call ‘Markov observer’ any model subject who posits that the probability of the stimulus depends on the preceding stimuli (m>0). In this case, the vector q contains the 2m conditional probabilities of observing A after observing each possible sequence of m stimuli. For instance, with m=1 the vector q is the pair of parameters (qA,qB) denoting the probabilities of observing a stimulus A after observing, respectively, a stimulus A and a stimulus B. In the absence of a cost, the belief over the parameter(s) eventually converges towards the parameter vector that is consistent with the generative Bernoulli statistics governing the stimulus (except if the prior precludes this parameter vector). Below, we assume a uniform prior.

To understand how the costs contort the inference process, it is useful to have in mind the solution to the ‘unconstrained’ inference problem (with λ=0), i.e., the Bayesian posterior, which we denote by Pt∗(q). In the case of a Bernoulli observer (m=0), after t trials, the Bayesian posterior is a Beta distribution,(2)Pt∗(q)∝qntA(1−q)ntB,

where ntX is the number of stimuli X observed up to trial t, that is, ntA=∑i=1txi, and ntB=∑i=1t(1−xi). As more evidence is accumulated, the Bayesian posterior gradually narrows and converges towards the value of the stimulus generative probability (Figure 3c and d, grey lines).

The ways in which the Bayesian posterior is distorted, in our models, depend on the nature of the cost that weighs on the inference process. Although many assumptions could be made on the kind of constraint that hinders human inference, and on the cost it would entail in our framework, here we examine two costs that stem from two possible principles: that the cost is a function of the beliefs held by the subject, or that it is a function of the environment that the subject is inferring. We detail, below, these two costs.

A first hypothesis about the inference process of subjects is that the brain mobilizes resources to represent probability distributions, and that more ‘precise’ distributions require more resources. We write the cost associated with a distribution, P^(q), as the negative of its entropy,(3)Cp(P^)=−H[P^(q)]=∫P^(q)lnP^(q)dq,

which is a measure of the amount of certainty in the distribution. Wider (less concentrated) distributions provide less information about the probability parameter and are thus less costly than narrower (more concentrated) distributions (Figure 3b). As an extreme case, the uniform distribution is the least costly.

With this cost, the loss function (Equation 1) is minimized by the distribution equal to the product of the prior and the likelihood, raised to the exponent 1/(λ+1), and normalized, i.e.,(4)P^t+1(q)∝[P^t(q)p(xt+1|xt−m+1:t,q)]1/(λ+1).

Since λ is strictly positive, the exponent is positive and lower than 1. As a result, the solution ‘flattens’ the Bayesian posterior, and in the extreme case of an unbounded cost (λ→∞) the posterior is the uniform distribution.

Furthermore, in the expression of our model subject’s posterior, the likelihood p(xt+1|xt−m+1:t,q) is raised after k trials to the exponent 1/(λ+1)k+1, it thus decays to zero as the number k of new stimuli increases. One can interpret this effect as gradually forgetting past observations. Specifically, we recover the predictions of leaky-integration models, in which remote patterns in the sequence of stimuli are discounted through an exponential filter (Yu and Cohen, 2008; Meyniel et al., 2016); here, we do not posit the gradual forgetting of remote observations, but instead we derive it as an optimal solution to a problem of constrained inference. We illustrate leaky integration in the case of a Bernoulli observer (m=0): in this case, the posterior after t trials, P^t(q), is a Beta distribution,(5)P^t(q)∝qn~tA(1−q)n~tB,

where n~tA and n~tB are exponentially-filtered counts of the number of stimuli A and B observed up to trial t, i.e.,(6)n~tA=∑k=0t−1(1λ+1)kxt−k,andn~tB=∑k=0t−1(1λ+1)k(1−xt−k).

In other words, the solution to the constrained inference problem, with the precision cost, is similar to the Bayesian posterior (Equation 2), but with counts of the two stimuli that gradually ‘forget’ remote observations (in the absence of a cost, that is, λ=0, we have n~tA=ntA and n~tB=ntB, and thus we recover the Bayesian posterior). As a result, these counts fluctuate with the recent history of the stimuli. Consequently, the posterior P^t(q) is dominated by the recent stimuli: it does not converge, but instead fluctuates with the recent stimulus history (Figure 3c and d, purple lines; compare with the green and gray lines). Hence, this model implies predictions about subsequent stimuli that depend on the stimulus history, i.e., it predicts sequential effects.

A different hypothesis is that the subjects favor, in their inference, parameter vectors q that correspond to more predictable outcomes. We quantify the outcome unpredictability by the Shannon entropy (Shannon, 1948) of the outcome implied by the vector of parameters q, which we denote by H(X;q). (In the Bernoulli-observer case, H(X;q)=−qln⁡q−(1−q)ln⁡(1−q); for the Markov-observer cases, see Methods.) The cost associated with the distribution P^(q) is the expectation of this quantity averaged over beliefs, i.e.,(7)Cu(P^)=EP^[H(X;q)]=∫H(X;q)P^(q)dq,

which we call the ‘unpredictability cost’. For a Bernoulli observer, a posterior concentrated on extreme values of the Bernoulli parameter (toward 0 or 1), thus representing more predictable environments, comes with a lower cost than a posterior concentrated on values of the Bernoulli parameter close to 0.5, which correspond to the most unpredictable environments (Figure 3a).

After t trials, the loss function (Equation 1) under this cost is minimized by the posterior(8)P^t(q)∝Pt∗(q)e−tλH(X;q),

i.e., the product of the Bayesian posterior, which narrows with t around the stimulus generative probability, and of a function that is larger for values of q that imply less entropic (i.e. more predictable) environments (see Methods). In short, with the unpredictability cost the model subject’s posterior is ‘pushed’ towards less entropic values of q.

In the Bernoulli case (m=0), the posterior after t stimuli has a global maximum, q∗(nt/t), that depends on the proportion nt/t of stimuli A observed up to trial t. As the number of presented stimuli t grows, the posterior P^t becomes concentrated around this maximum. The proportion nt/t naturally converges to the stimulus generative probability, p, thus our subject’s inference converges towards the value q∗(p) which is different from the true value p, in the non-equiprobable case (p≠.5). The equiprobable case (p=.5) is singular, in that with a weak cost (λ<1) the inferred probability is unbiased (q∗(p)=.5), while with a strong cost (λ>1) the inferred probability does not converge but instead alternates between two values above and below 0.5; see Prat-Carrabin et al., 2021a. In other words, except in the equiprobable case, the inference converges but it is biased, i.e., the posterior peaks at an incorrect value of the stimulus generative probability (Figure 3c and d, green lines). This value is closer to the extremes (0 and 1) than the stimulus generative probability, that is, it implies an environment more predictable than the actual one (Figure 3d).

In the case of a Markov observer (m>0), the posterior also converges to a vector of parameters q which implies not only a bias but also that the conditional probabilities of a stimulus A (conditioned on different stimulus histories) are not equal. The prediction of the next stimulus being A on a given trial depends on whether the preceding stimulus was A or B: this model therefore predicts sequential effects. We further examine below the behavior of this model in the cases of a Bernoulli observer and of different Markov observers. We refer the reader interested in more details on the Markov models, including their mathematical derivations, to the Methods section.

In short, with the unpredictability-cost models, when p≠0.5, the inference process converges to an asymptotic posterior q∗(p) which does not itself depend on the history of the stimulus, but that is biased (Figure 3c, d, green lines). In particular, for Markov observers (m>0), the asymptotic posterior corresponds to an erroneous belief about the dependency of the stimulus on the recent stimulus history, which results in sequential effects in behavior.

Although the two families of models derived from the two costs both potentially generate sequential effects, they do so by giving rise to qualitatively different inference processes. Under the unpredictability cost, the inference converges to a posterior that, in the Bernoulli case (m=0), implies a biased estimate of the stimulus generative probability (Figure 3d, green lines), while in the Markov case (m>0) it implies the belief that there are serial dependencies in the stimuli: predictions therefore depend on the recent stimulus history. By contrast, the precision cost prevents beliefs from converging (Figure 3c, purple lines). As a result, the subject’s predictions vary with the recent stimulus history (Figure 3d). This inference process amounts to an exponential discount of remote observations, or equivalently, to the overweighting of recent observations (Equation 6).

To investigate in more detail the sequential effects that these two costs produce, we implement two families of inference models derived from the two costs. Each model is characterized by the type of cost (unpredictability cost or precision cost), and by the assumed Markov order (m): we examine the case of a Bernoulli observer (m=0) and three cases of Markov observers (with m= 1, 2, and 3). We thus obtain 2×4=8 models of inference. Each of these models has one parameter λ controlling the weight of the cost. (We also examine a ‘hybrid’ model that combines the two costs; see below.)

We assume that the subject’s response on a given trial depends on the inferred posterior according to a generalization of ‘probability matching’ implemented in other studies (Battaglia et al., 2011; Yu and Huang, 2014; Prat-Carrabin et al., 2021b). In this response-selection strategy, the subject predicts A with the probability p¯tκ/(p¯tκ+(1−p¯t)κ), where p¯t is the expected probability of a stimulus A derived from the posterior, i.e., p¯t≡∫p(xt+1=1|xt−m+1:t,q)P^t(q)dq. The single parameter κ controls the randomness of the response: with κ=0 the subject predicts A and B with equal probability; with κ=1 the response-selection strategy corresponds to probability matching, that is, the subject predicts A with probability p¯t; and as κ increases toward infinity the choices become optimal, that is, the subjects predicts A if the expected probability of observing a stimulus A, p¯t, is greater than 0.5, and predicts B if it is lower than 0.5 (if p¯t=0.5 the subject chooses A or B with equal probability). In our investigations, we also implement several other response-selection strategies, including one in which subjects have a propensity to repeat their preceding response, or conversely, to alternate; these analyses do not change our conclusions (see Methods).

Each of our eight models has two parameters: the factor weighting the cost, λ, and the exponent of the generalized probability-matching, κ. We fit the parameters of each model to the responses of each subject, by maximizing their likelihoods. We find that 60% of subjects are best fitted by one of the unpredictability-cost models, while 40% are best fitted by one of the precision-cost models. When pooling the two types of cost, 65% of subjects are best fitted by a Markov-observer model. We implement a ‘Bayesian model selection’ procedure (Stephan et al., 2009), which takes into account, for each subject, the likelihoods of all the models (and not only the maximum among them) in order to obtain a Bayesian posterior over the distribution of models in the general population (see Methods). The derived expected probability of unpredictability-cost models is 57% (and 43% for precision-cost models) with an exceedance probability (i.e. probability that unpredictability-cost models are more frequent in the general population) of 78%. The expected probability of Markov-observer models, regardless of the cost used in the model, is 70% (and 30% for Bernoulli-observer models) with an exceedance probability (i.e. probability that Markov-observer models are more frequent in the general population) of 98%. These results indicate that the responses of subjects are generally consistent with a Markov-observer model, although the stimuli used in the experiment are Bernoulli-distributed. As for the unpredictability-cost and the precision-cost families of models, Bayesian model selection does not provide decisive evidence in favor of either model, indicating that they both capture some aspects of the responses of the subjects. Below, we examine more closely the behaviors of the models, and point to qualitative differences between the predictions resulting from each model family.

Before turning to these results, we validate the robustness of our model-fitting procedure with several additional analyses. First, we estimate a confusion matrix to examine the possibility that the model-fitting procedure could misidentify the models which generated test sets of responses. We find that the best-fitting model corresponds to the true model in at least 70% of simulations (the chance level is 12.5%=1/8 models), and actually more than 90% for the majority of models (see Appendix).

Second, we seek to verify whether the best-fitting cost factor, λ, that we obtain for each subject is consistent across the range of probabilities tested. Specifically, we fit separately the models to the responses obtained in the blocks of trials whose stimulus generative probability was ‘medium’ (between 0.3 and 0.7, included) on the one hand, and to the responses obtained when the probability was ‘extreme’ (below 0.3, and above 0.7) on the other hand; and we compare the values of the best-fitting cost factors λ in these two cases. More precisely, for the precision-cost family, we look at the inverse of the decay time, ln⁡(1+λ), which is the inverse of the characteristic time over which the model subject ‘forgets’ past observations. With both families of models, we find that on a logarithmic scale the parameters in the medium- and extreme-probabilities cases are significantly correlated across subjects (Pearson’s r, precision-cost models: 0.75, p-value: 1e-4; unpredictability-cost models: r=0.47, p-value: .036). In other words, if a subject is best fitted by a large cost factor in medium-probabilities trials, he or she is likely to be also best fitted by a large cost factor in extreme-probabilities trials. This indicates that our models capture idiosyncratic features of subjects that generalize across conditions instead of varying with the stimulus probability (see Appendix).

Third, as mentioned above we examine a variant of the response-selection strategy in which the subject sometimes repeats the preceding response, or conversely alternates and chooses the other response, instead of responding based on the inferred probability of the next stimulus. This propensity to repeat or alternate does not change the best-fitting inference model of most subjects, and the best-fitting values of the parameters λ and κ are very stable when allowing or not for this propensity. This analysis supports the results we present here, and speaks to the robustness of the model-fitting procedure (see Methods).

Finally, as the unpredictability-cost family and the precision-cost family of models both seem to capture the responses of a sizable share of the subjects, one might assume that the behavior of most subjects actually fall ‘somewhere in between’, and would be best accounted for by a hybrid model combining the two costs. In our investigations, we have implemented such a model, whereby the subject’s approximate posterior P^t results from the minimization of a loss function that includes both a precision cost, with weight λp, and an unpredictability cost, with weight λu (and the response-selection strategy is the generalized probability matching, with parameter κ). We do not find that most subjects’ responses are better fitted (as measured by the Bayesian Information Criterion Schwarz, 1978) by a combination of the two costs: instead, for more than two thirds of subjects, the best-fitting model features just one cost (see Methods). In other words, the two cost seems to capture different aspects of the behavior that are predominant in different subpopulations. Below, we examine the behavioral patterns resulting from each cost type, in comparison with the behavior of the subjects.

We now examine the behavioral patterns resulting from the models. All the models we consider predict that the proportion of predictions A, p¯(A), is a smooth, increasing function of the stimulus generative probability (when λ<∞ and 0<κ<∞; Figure 4a–d, grey lines), thus we focus, here, on the ability of the models to reproduce the subjects’ sequential effects. With the unpredictability-cost model of a Bernoulli observer (m=0), the belief of the model subject, as mentioned above, asymptotically converges in non-equiprobable cases to an erroneous value of the stimulus generative probability (Figure 3d, green lines). After a large number of observations (such as the 200 ‘passive’ trials, in our task), the sensitivity of the belief to new observations becomes almost imperceptible; as a result, this model predicts practically no sequential effects (Figure 4b), that is, p¯(A|A)≃p¯(A|B). With the unpredictability-cost model of a Markov observer (e.g. m=1), the belief of the model subject also converges, but to a vector of parameters q that implies a sequential dependency in the stimulus, that is, qA≠qB, resulting in sequential effects in predictions, that is, p¯(A|A)≠p¯(A|B). The parameter vector q yields a more predictable (less entropic) environment if the probability conditional on the more frequent outcome (say, A) is less entropic than the probability conditional on the less frequent outcome (B). This is the case if the former is greater than the latter, resulting in the inequality p¯(A|A)>p¯(A|B), that is, in sequential effects of the attractive kind (Figure 4d). (The case in which B is the more frequent outcome results in the inequality p¯(B|B)>p¯(B|A), i.e., 1−p¯(A|B)>1−p¯(A|A), i.e., the same, attractive sequential effects.)

Turning to the precision-cost models, we have noted that in these models the posterior fluctuates with the recent history of the stimuli (Figure 3c): as a result, sequential effects are obtained, even with a Bernoulli observer (m=0; Figure 4a). The most recent stimulus has the largest weight in the exponentially filtered counts that determine the posterior (Equation 6), thus the model subject’s prediction is biased towards the last stimulus, that is, the sequential effect is attractive (p¯(A|A)>p¯(A|B)). With the traditional probability-matching response-selection strategy (i.e. κ=1), the strength of the attractive effect is the same across all stimulus generative probabilities (i.e. the difference p¯(A|A)−p¯(A|B) is constant; Figure 4a, dotted lines and light-red dots). With the generalized probability-matching response-selection strategy, if κ>1, proportions below and above 0.5 are brought closer to the extremes (0 and 1, respectively), resulting in larger sequential effects for values of the stimulus generative probability closer to 0.5 (Figure 4a, solid lines and red dots; the model is simulated with κ=2.8, a value representative of the subjects’ best-fitting values for this parameter). We also find stronger sequential effects closer to the equiprobable case in subjects’ data (Figure 2b).

The precision-cost model of a Markov observer (m=1) also predicts attractive sequential effects (Figure 4c). While the behavior of the Bernoulli observer (with a precision cost) is determined by two exponentially-filtered counts of the two possible stimuli (Equation 6), that of the Markov observer with m=1 depends on four exponentially filtered counts of the four possible pairs of stimuli. After observing a stimulus B, the belief that the following stimulus should be A or B is determined by the exponentially filtered counts of the pairs BA and BB. If p is large, i.e., if the stimulus B is infrequent, then the BA and BB pairs are also infrequent and the corresponding counts are close to zero: the model subject thus behaves as if only very little evidence had been observed about the transitions B to A and B to B in this case, resulting in a proportion of predictions A conditional on a preceding B, p¯(A|B), close to 0.5 (Figure 4c, orange line). Consequently, the sequential effects are stronger for values of the stimulus generative probabilities closer to the extreme (Figure 4c, red dots).

Both families of costs are thus able to produce attractive sequential effects, albeit with some qualitative differences. (In Figure 4a–d we show the behaviors resulting from the two costs for a Bernoulli observer and a Markov observer of order m=1; the Markov observers of higher order exhibit qualitatively similar behaviors; see Methods.) As the model fitting indicates that different groups of subjects are best fitted by models belonging to the two families, we examine separately the behaviors of the subjects whose responses are best fitted by each of the two costs (Figure 4e and f), in comparison with the behaviors of the corresponding best-fitting models (Figure 4g and h). This provides a finer understanding of the behavior of subjects than the group average shown in Figure 2. For the subjects best fitted by precision-cost models, the proportion of predictions A, p¯(A), when the stimulus generative probability is close to 0.5, is a less steep function of this probability than for the subjects best-fitted by unpredictability-cost models (Figure 4e and f, grey lines); furthermore, their sequential effects are larger (as measured by the difference p¯(A|A)−p¯(A|B)), and do not depend much on the stimulus generative probability (Figure 4e and f, red dots). The corresponding models reproduce the behavioral patterns of the subjects that they best fit (Figure 4g and h). Each family of models seems to capture specific behaviors exhibited by the subjects: when fitting the unpredictability-cost models to the responses of the subjects that are best fitted by precision-cost models, and conversely when fitting the precision-cost models to the responses of the subjects that are best fitted by unpredictability-cost models, the models do not reproduce well the subjects’ behavioral patterns (Figure 4i and j). The precision-cost models, however, seem slightly better than the unpredictability-cost models at capturing the behavior of the subjects that they do not best fit (Figure 4, compare panel j to panel f, and panel i to panel e). Substantiating this observation, the examination of the distributions of the models’ BICs across subjects shows that when fitting the models onto the subjects that they do not best fit, the precision-cost models fare better than the unpredictability-cost models (see Appendix).

Notwithstanding the quantitative differences just presented, both families of models yield qualitatively similar attractive sequential effects: the model subjects’ predictions are biased towards the preceding stimulus. Does this pattern also apply to the longer history of the stimulus, i.e., do more distant trials also influence the model subjects’ predictions? To investigate this hypothesis, we examine the difference between the proportion of predictions A after observing a sequence of length n that starts with A, minus the proportion of predictions A after the same sequence, but starting with B, i.e., p¯(A|Ax)−p¯(A|Bx), where x is a sequence of length n−1, and Ax and Bx denote the same sequence preceded by A and by B. This quantity enables us to isolate the influence of the n-to-last stimulus on the current prediction. If the difference is positive, the effect is ‘attractive’; if it is negative, the effect is ‘repulsive’ (in this latter case, the presentation of an A decreases the probability that the subjects predicts A in a later trial, as compared to the presentation of a B); and if the difference is zero there is no sequential effect stemming from the n-to-last stimulus. The case n=1 corresponds to the immediately preceding stimulus, whose effect we have shown to be attractive, i.e., p¯(A|A)−p¯(A|B)>0, in the responses both of the best-fitting models and of the subjects (Figures 2b, 4g and h).

We investigate the effect of the n-to-last stimulus on the behavior of the two families of models, with n=1, 2, and 3. We present here the main results of this investigation; we refer the reader to Methods for a more detailed analysis. With unpredictability-cost models of Markov order m, there are non-vanishing sequential effects stemming from the n-to-last stimulus only if the Markov order is greater than or equal to the distance from this stimulus to the current trial, i.e., if m≥n. In this case, the sequential effects are attractive (Figure 5).

With precision-cost models, the n-to-last stimuli yield non-vanishing sequential effects regardless of the Markov order, m. With n=1, the effect is attractive, i.e., p¯(A|A)−p¯(A|B)>0. With n=2 (second-to-last stimulus), the effect is also attractive, i.e., in the case of the pair of sequences AA and BA, p¯(A|AA)−p¯(A|BA)>0 (Figure 5a). By symmetry, the difference is also positive for the other pair of relevant sequences, AB and BB (e.g. we note that p¯(A|AB)=1−p¯(B|AB), and that p¯(B|AB) when the probability of A is p is equal to p¯(A|BA) when the probability of A is 1−p. We detail in Methods such relations between the proportions of predictions A or B in different situations. These relations result in the symmetries of Figure 2, for the sequential effect of the last stimulus, while for higher-order sequential effects they imply that we do not need to show, in Figure 5, the effects following all possible past sequences of two or three stimuli, as the ones we do not show are readily derived from the ones we do.)

As for the third-to-last stimulus (n=3), it can be followed by four different sequences of length two, but we only need to examine two of these four, for the reasons just presented. We find that for the precision-cost models, with all the Markov orders we examine (from 0 to 3), the probability of predicting A after observing the sequence AAA is greater than that after observing the sequence BAA, i.e., p¯(A|AAA)−p¯(A|BAA)>0, that is, there is an attractive sequential effect of the third-to-last stimulus if the sequence following it is AA (and, by symmetry, if it is BB; Figure 5b). So far, thus, we have found only attractive effects. However, the results are less straightforward when the third-to-last stimulus is followed by the sequence BA. In this case, for a Bernoulli observer (m=0), the effect is also attractive: p¯(A|ABA)−p¯(A|BBA)>0 (Figure 5c, white circles). With Markov observers (m≥1), over a range of stimulus generative probability p, the effect is repulsive: p¯(A|ABA)−p¯(A|BBA)<0, that is, the presentation of an A decreases the probability that the model subject predicts A three trials later, as compared to the presentation of a B (Figure 5c, filled circles). The occurrence of the repulsive effect in this particular case is a distinctive trait of the precision-cost models of Markov observers (m≥1); we do not obtain any repulsive effect with any of the unpredictability-cost models, nor with the precision-cost model of a Bernoulli observer (m=0).

We now examine the sequential effects in subjects’ responses, beyond the attractive effect of the preceding stimulus (n=1; discussed above). With n=2 (second-to-last stimulus), for the majority of the 19 stimulus generative probabilities p, we find attractive sequential effects: the difference p¯(A|AA)−p¯(A|BA) is significantly positive (Figure 6a; p-values <0.01 for 11 stimulus generative probabilities, <0.05 for 13 probabilities; subjects pooled). With n=3 (third-to-last stimulus), we also find significant attractive sequential effects in subjects’ responses for some of the stimulus generative probabilities, when the third-to-last stimulus is followed by the sequence AA (Figure 6b; p-values <0.01 for four probabilities, <0.05 for seven probabilities). When it is instead followed by the sequence BA, we find that for eight stimulus generative probabilities, all between 0.25 and 0.75, there is a significant repulsive sequential effect: p¯(A|ABA)−p¯(A|BBA)<0 (p-values <0.01 for six probabilities, <0.05 for eight probabilities; subjects pooled). Thus, in these cases, the occurrence of A as the third-to-last stimulus increases (in comparison with the occurrence of a B) the proportion of the opposite prediction, B. For the remaining stimulus generative probabilities, this difference is in most cases also negative although not significantly different from zero (Figure 6c). (An across-subjects analysis yields similar results; see Supplementary Materials.) Figure 6d summarizes subjects’ sequential effects, and exhibits the attractive and repulsive sequential effects in their responses (compare solid and dotted lines). (In this tree-like representation, we show averages across the stimulus generative probabilities; a figure with the individual ‘trees’ for each probability is provided in the Appendix.)

The repulsive sequential effect of the third-to-last stimulus in subjects’ predictions only occurs when the third-to-last stimulus is A followed by the sequence BA. It is also only in this case that the repulsive effect appears with the precision-cost models of a Markov observer (while it never appears with the unpredictability-cost models). This qualitative difference suggests that the precision-cost models offer a better account of sequential effects in subjects. However, model-fitting onto the overall behavior presented above showed that a fraction of the subjects is better fitted by the unpredictability-cost models. We investigate, thus, the presence of a repulsive effect in the predictions of the subjects best fitted by the precision-cost models, and of those best fitted by the unpredictability-cost models. For the subjects best fitted by the precision-cost models, we find (expectedly) that there is a significant repulsive sequential effect of the third-to-last stimulus (p¯(A|ABA)−p¯(A|BBA)<0; p-values <0.01 for two probabilities, <0.05 for four probabilities; subjects pooled; Figure 6e, left panel). For the subjects best fitted by the unpredictability-cost models (a family of model that does not predict any repulsive sequential effects), we also find, perhaps surprisingly, a significant repulsive effect of the third-to-last stimulus (p-values <0.01 for three probabilities, <0.05 for five probabilities; subjects pooled), which demonstrates the robustness of this effect (Figure 6e, right panel). Thus, in spite of the results of the model-selection procedure, some sequential effects in subjects’ predictions support only one of the two families of model. Regardless of the model that best fits their overall predictions, the behavior of the subjects is consistent only with the precision-cost family of models with Markov order equal to or greater than 1, that is, with a model of inference of conditional probabilities hampered by a cognitive cost weighing on the precision of belief distributions.

The computer-based task was programmed using the Python library PsychoPy (Peirce, 2008). The experiment comprised ten blocks of trials, which differed by the stimulus generative probability, p, used in all the trials of each block. The probability p was chosen randomly among the ten values ranging from 0.50 to 0.95 by increments of 0.05, excluding the values already chosen; and with probability 1/2 the stimulus generative probability 1−p was used instead. Each block started with 200 passive trials, in which the subject was only asked to look at the 200 stimuli sampled with the block’s probability and successively presented. No action from the subject was required for these passive trials. The subject was then asked to predict, in each of 200 trials, the next location of the stimulus. Subjects provided their responses by a keypress. The task was presented as a game to the subjects: the stimulus was a lightning symbol, and predicting correctly whether the lightning would strike the left or the right rod resulted in the electrical energy of the lightning being collected in a battery (Figure 1). A gauge below the battery indicated the amount of energy accumulated in the current block of trials (Figure 1a). Twenty subjects (7 women, 13 men; age: 18–41, mean 25.5, standard deviation 6.2) participated in the experiment. All subjects completed the ten blocks of trials, except one subject who did not finish the experiment and was excluded from the analysis. The study was approved by the ethics committee Île de France VII (CPP 08–021). Participants gave their written consent prior to participating. The number of blocks of trials and the number of trials per block were chosen as a trade-off between maximizing the statistical power of the study, scanning the values of the generative probability parameter from 0.05 to 0.95 with a satisfying resolution, and maintaining the duration of the experiment under a reasonable length of time. The number of subjects was chosen consistently with similar studies and so as to capture individual variability. Throughout the study, we conduct Student’s t-tests when comparing the subjects’ proportion of predictions A to a given value (e.g. 0.5). When comparing two proportions of predictions A obtained under different conditions (e.g. depending on whether the preceding stimulus is A or B), we accordingly conduct Fisher exact tests. The trials in which subjects failed to respond within the limit of 1 s were not included in the analysis. They represented 1.27% of the trials, on average (across subjects); and for 95% of the subjects these trials represented less than 2.5% of the trials.

We run simulations of the eight models and look at the predictions they yield. To reproduce the conditions faced by the subjects, which included 200 passive trials, we start each simulation by showing to the model subject 200 randomly sampled stimuli (without collecting predictions at this stage). We then show an additional 200 samples, and obtain a prediction from the model subject after each sample. The sequential effects of the most recent stimulus, with the different models, are shown in Figure 7. With the precision-cost models, the posterior distribution of the model subject does not converge, but fluctuates instead with the recent history of the stimulus. This results in attractive sequential effects (Figure 7a), including for the Bernoulli observer, who assumes that the probability of A does not depend on the most recent stimulus. With the unpredictability-cost models, the posterior of the model subject does converge. With Markov observers, it converges toward a parameter vector q that implies that the probability of observing A depends on the most recent stimulus, resulting in the presence of sequential effects of the most recent stimulus (Figure 7b, second to fourth row). With a Bernoulli observer, the posterior of the model subject converges toward a value of the stimulus generative probability that does not depend on the stimulus history. As more evidence is accumulated, the posterior narrows around this value but not without some fluctuations that depend on the sequence of stimuli presented. In consequence the model subject’s estimate of the stimulus generative probability is also subject to fluctuations, and depends on the history of stimuli (including the most recent stimulus), although the width of the fluctuations tend to zero as more stimuli are observed. After the 200 stimuli of the passive trials, the sequential effects of the most recent stimulus resulting from this transient regime appear small in comparison to the sequential effects obtained with the other models (Figure 7b, first row). The Figure 7 also shows the behaviors of the models when augmented with a propensity to repeat the preceding response: we comment on these in the section dedicated to these models, below.

Turning to higher-order sequential effects, we look at the influence on predictions of the second- and third-to-last stimulus (Figure 8). As mentioned, only precision-cost models of Markov observers yield repulsive sequential effects, and these occur only when the third-to-last-stimulus is followed by BA. They do not occur with the second-to-last stimulus, nor with the third-to-last-stimulus when it is followed by AA (Figure 8a); and they do not occur in any case with the unpredictability-cost models (Figure 8b).

We derive the solution to the constrained optimization problem, in the general case of a ‘hybrid’ model subject who bears both a precision cost, with weight λp, and an unpredictability cost, with weight λu. Thus the subject minimizes the loss function(9)L(P^t+1)=∫P^t+1(q)ln⁡P^t+1(q)Pt+1(q)dq+λu∫H(X;q)P^t+1(q)dq+λp∫P^t+1(q)ln⁡P^t+1(q)dq+μ(∫P^t+1(q)dq−1),

in which we have included a Lagrange multiplier, μ, corresponding to the normalization constraint, ∫P^t+1(q)dq=1. Taking the functional derivative of L and setting to zero, we obtain(10)ln⁡P^t+1(q)+1−ln⁡Pt+1(q)+λuH(X;q)+λpln⁡P^t+1(q)+λp+μ=0,

and thus we write the approximate posterior as(11)P^t+1(q)∝Pt+1(q)11+λpexp⁡(−λu1+λpH(X;q)),

where Pt+1(q) is the Bayesian update of the preceding belief, P^t(q), i.e.,(12)Pt+1(q)∝P^t(q)p(xt+1|q,xt−m+1:t).

Setting the weight of the unpredictability cost to zero (i.e., λu=0), we obtain the posterior in presence of the precision cost only, as(13)P^t+1prec(q)∝Pt+1(q)11+λp.

The main text provides more details about the posterior in this case (Equation 4), in particular with a Bernoulli observer (m=0; Equation 5, Equation 6).

For the hybrid model (in which both λu and λp are potentially different from zero), we obtain(14)P^t(q)∝P^tprec(q)exp⁡(−λuH(X;q)∑i=1t(11+λp)i).

With λp=0, the sum in the exponential is equal to t, and the precision-cost posterior, P^tprec(q), is the Bayesian posterior, Pt∗(q), and thus we obtain the posterior in presence of the unpredictability cost only (see Equation 8).

The hybrid model, described above, features both a precision cost and an unpredictability cost, with respective weights λp and λu. As with the models that include only one type of cost, we consider a Bernoulli observer (m=0), and three Markov observers (m=1,2, and 3). As for the response-selection strategy, we use, here also, the generalized probability-matching strategy parameterized by κ. We thus obtain four new models; each one has three parameters (λp, λu, and κ), while the non-hybrid models (featuring only one type of cost) have only two parameters.

We fit these models to the responses of subjects. For 68% of subjects, the BIC of the best-fitting hybrid model is larger than the BIC of the best-fitting non-hybrid model, indicating a worse fit, by this measure. This suggests that for these subjects, allowing for a second type of cost result in a modest improvement of the fit that does not justify the additional parameter. For the remaining 32% of subjects, the hybrid models yield a better fit (a lower BIC) than the non-hybrid models, although for half of these, the difference in BICs is lower than 6, which is only weak evidence in favor of the hybrid models.

Moreover, we compute the exceedance probability, defined below in the section ‘Bayesian Model Selection’, of the hybrid models (together with the complementary probability of the non-hybrid models). We find that the exceedance probability of the hybrid models is 8.1% while that of the non-hybrid models is 91.9%, suggesting that subjects best-fitted by non-hybrid models are more prevalent.

In summary, we find that for more than two thirds of subjects, allowing for a second cost type does not improve much the fit to the behavioral data (the BIC is higher with the best-fitting hybrid model). These subjects are best-fitted by non-hybrid models, that is, by models featuring only one type of cost, instead of ‘falling in between’ the two cost types. This suggests that for most subjects, only one of the two costs, either the prediction cost or the unpredictability cost, dominates the inference process.

In addition to the generalized probability-matching response-selection strategy presented in the main text, in our investigations we also implement several other response-selection strategies. First, a strategy based on a ‘softmax’ function that smoothes the optimal decision rule; it does not yield, however, a behavior substantially different from that of the generalized probability-matching response-selection strategy. Second, we examine a strategy in which the model subject chooses the optimal response with a probability that is fixed across conditions, which we fit onto subjects’ choices. No subject is best-fitted by this strategy. Third, another possible strategy proposed in the game-theory literature (Nowak and Sigmund, 1993) is ‘win-stay, lose-shift’: it prescribes to repeat the same response as long as it proves correct and to change otherwise. In the context of our binary-choice prediction task, it is indistinguishable from a strategy in which the model subject chooses a prediction equal to the outcome that last occurred. This strategy is a special case of our Bernoulli observer hampered by a precision cost whose weight λ is large combined with the optimal response-selection strategy (κ→∞). Since the generalized probability-matching strategy parameterized by the exponent κ appears either more general, better than or indistinguishable from those other response-selection strategies, we selected it to obtain the results presented in the main text.

Furthermore, we consider the possibility that subjects may have a tendency to repeat their preceding response, or, conversely, to alternate and choose the other response, independently from their inference of the stimulus statistics. Specifically, we examine a generalization of the response-selection strategy, in which a parameter η, with −1<η<1, modulates the probability of a repetition or of an alternation. With probability 1−|η|, the model subject chooses a response with the generalized probability-matching response-selection strategy, with parameter κ. With probability |η|, the model subject repeats the preceding response, if η is positive; or chooses the opposite of the preceding response, if η is negative. With η=0, there is no propensity for repetition nor alternation, and the response-selection strategy is the same as the one we have considered in the main text. We have allowed for alternations (η<0) in this model for the sake of generality, but for all the subjects the best-fitting value of η is non-negative, thus henceforth we only consider the possibility of repetitions, i.e., non-negative values of the parameter (η≥0).

We note that with a repetition probability η, such that 0≤η<1, the unconditional probability of a prediction A, which we denote by p¯η(A), is not different from the unconditional probability of a prediction A in the absence of a repetition probability η, p¯(A), as in the event of a repetition, the response that is repeated is itself A with probability p¯(A); formally, p¯η(A)=(1−η)p¯(A)+ηp¯η(A), which implies the equality p¯η(A)=p¯(A).

Now turning to sequential effects, we note that with a repetition probability η, the probability of a prediction A conditional on an observation A is(15)p¯η(A|A)=(1−η)p¯(A|A)+ηp¯(A).

In other words, when introducing the repetition probability η, the resulting probability of a prediction A conditional on observing A is a weighted mean of the unconditional probability of a prediction A and of the conditional probability of a prediction A in the absence of a repetition probability. Figure 7 (dotted lines) illustrates this for the eight models, with η=0.2. Consequently the sequential effects with this response-selection strategy are more modest (Figure 7, light-red dots).

We fit (by maximizing their likelihoods) our eight models now equipped with a propensity for repetition (or alternation) parameterized by η. The average best-fitting value of η, across subjects, is 0.21 (standard deviation: 0.19; median: 0.18); as mentioned, no subjects have a negative best-fitting value of η. In order to assess the degree to which the models with repetition propensity are able to capture subjects’ data, in comparison with the models without such propensity, we use the Bayesian Information Criterion (BIC) (Schwarz, 1978), which penalizes the number of parameters, as a comparative metric (a lower BIC is better). For 26% of subjects, the BIC with this response-selection strategy (allowing for η≠0) is higher than with the original response-selection strategy (which sets η=0,) suggesting that the responses of these subjects do not warrant the introduction of a repetition (or alternation) propensity. In addition, for these subjects the best-fitting inference model, characterized by a cost type and a Markov order, is the same when the response-selection strategy allows for repetition or alternation (η≠0) and when it does not (η=0). For 47% of subjects, the BIC is lower when including the parameter η (suggesting that allowing for η≠0 results in a better fit to the data), and importantly, here also the best-fitting inference model (cost type and Markov order) is the same with η≠0 and with η=0. For 11% of subjects, a better fit (lower BIC) is obtained with η≠0; and the best-fitting inference models, with η≠0 and with η=0, belong to the same family of models, that is, they have the same cost type (precision cost or unpredictability cost), and only their Markov orders differ. Finally, only for the remaining 16% does the cost type change when allowing for η≠0. In other words, for 84% of subjects the best-fitting cost type is the same whether or not η is allowed to differ from 0.

Furthermore, the best-fitting parameters λ and κ are also stable across these two cases. Among the 73% of subjects whose best-fitting inference model (including both cost type and Markov order) remains the same regardless of the presence of a repetition propensity, we find that the best-fitting values of κ, with η≠0 and with η=0, differ by less than 10% for 93% of subjects, and the best-fitting values of λ differ by less than 10% for 71% of subjects. For these two parameters, the correlation coefficient (between the best-fitting value with η=0 and the best-fitting value with η≠0) is above 0.99 (with p-values lower than 1e-19).

The responses of a majority of subjects are thus better reproduced by a response-selection strategy that includes a probability of repeating the preceding response. The impact of this repetition propensity on sequential effects is relatively small in comparison to the magnitude of these effects (Figure 7). For most subjects, moreover, the best-fitting inference model, characterized by its cost type and its Markov order, is the same — with or without repetition propensity —, and the best-fitting parameters λ and κ are very close in the two cases. Therefore, this analysis supports the results of the model-fitting and model-selection procedure, and validates its robustness. We conclude that the models of costly inference are essential in reproducing the behavioral data, notwithstanding a positive repetition propensity in a fraction of subjects.

Model fitting is conducted by maximizing for each model the likelihood of the subject’s choices. With the precision-cost models, the likelihood can be derived analytically and thus easily computed: the model’s posterior is a Dirichlet distribution of order 2m+1, whose parameters are exponentially filtered counts of the observed sequences of length m+1. With a Bernoulli observer, i.e., m=0, this is the Beta distribution presented in Equation 5. The expected probability of a stimulus A, conditional on the sequence of m stimuli most recently observed, is a simple ratio involving the exponentially filtered counts, for example (n~tA+1)/(n~tA+n~tB+2) in the case of a Bernoulli observer. This probability is then raised to the power κ and normalized (as prescribed by the generalized probability-matching response-selection strategy) in order to obtain the probability of a prediction A.

As for the unpredictability-cost models, the posterior is given in Equation 8 up to a normalization constant. Unfortunately, the expected probability of a stimulus A implied by this posterior does not come in a closed-form expression. Thus we compute the (unnormalized) posterior on a discretized grid of values of the vector q. The dimension of the vector q is 2m, and each element of q is in the segment [0,1]. If we discretize each dimension into n bins, we obtain n2m different possible values of the vector q; for each of these, at each trial, we compute the unnormalized value of the posterior (as given by Equation 8). As m increases, this becomes computationally prohibitive: for instance, with n=100 bins and m=3, the multidimensional grid of values of q contains 10¹⁶ numbers (with a typical computer, this would represent 80,000 terabytes). In order to keep the needed computational resources within reasonable limits, we choose a lower resolution of the grid for larger values of m. Specifically, for m=0 we choose a grid (over [0,1]) with increments of 0.01; for m=1, increments of 0.02 (in each dimension); for m=2, increments of 0.05; and for m=3, increments of 0.1. We then compute the mean of the discretized posterior and pass it through the generalized probability-matching response-selection model to obtain the choice probability.

To find the best-fitting parameters λ and κ, the likelihood was maximized with the L-BFGS-B algorithm (Byrd et al., 1995; Zhu et al., 1997). These computations were run using Python and the libraries Numpy and Scipy (Harris et al., 2020; Virtanen et al., 2020).

Throughout the paper, we leverage the symmetry inherent to the Bernoulli prediction task to present results in a condensed manner. Specifically, in our analysis, the proportion of predictions A when the probability of A (the stimulus generative probability) is p, which we denote here by p¯(A|p), is equal to the proportion of predictions B when the probability of A is 1−p, which we denote by p¯(B|1−p); i.e., p¯(A|p)=p¯(B|1−p). More generally, the predictions conditional on a given sequence when the probability of A is p are equal to the predictions conditional on the ‘mirror’ sequence (in which A and B have been swapped), when the probability of A is 1−p, for example extending our notation, p¯(A|AAB,p)=p¯(B|BBA,1−p). Here, we show how this results in the symmetries in Figure 2, and in the fact that in Figures 5 and 6, it suffices to plot the sequential effects obtained with only a fraction of all the possible sequences of two or three stimuli.

First, we note that(16)p¯(A|p)=1−p¯(B|p)=1−p¯(A|1−p),

which implies the symmetry of p¯(A) in Figure 2a (grey line). Turning to conditional probabilities (and thus sequential effects), we have(17)p¯(A|A,p)=1−p¯(B|A,p)=1−p¯(A|B,1−p)and p¯(A|B,p)=1−p¯(A|A,1−p).

As a result, the lines representing p¯(A|A) (blue) and p¯(A|B) (orange) in Figure 2a are reflections of each other. In addition, these equations result in the equality(18)p¯(A|A,p)−p¯(A|B,p)=p¯(A|A,1−p)−p¯(A|B,1−p),

which implies the symmetry in Figure 2b.

As for the sequential effect of the second-to-last stimulus, we show in Figures 5a and 6a the difference in the proportions of predictions A conditional on two past sequences of two stimuli, AA and BA; i.e., p¯(A|AA)−p¯(A|BA). There are two other possible sequences of two stimuli: AB and BB. The difference in the proportions conditional on these two sequences is implied by the former difference, as:(19)p¯(A|AB,p)−p¯(A|BB,p)=1−p¯(B|AB,p)−(1−p¯(B|BB,p))=1−p¯(A|BA,1−p)−(1−p¯(A|AA,1−p))=p¯(A|AA,1−p)−p¯(A|BA,1−p).

As for the sequential effect of the third-to-last stimulus, we show in Figures 5b and 6b the difference in the proportions conditional on the sequences AAA and BAA, and in Figures 5c and 6c the difference in the proportions conditional on the sequences ABA and BBA. The differences in the proportions conditional on the sequences AAB and BAB, and conditional on the sequences ABB and BBB, are recovered as a function of the former two, as(20)p¯(A|AAB,p)−p¯(A|BAB,p)=1−p¯(B|AAB,p)−(1−p¯(B|BAB,p))=1−p¯(A|BBA,1−p)−(1−p¯(A|ABA,1−p))=p¯(A|ABA,1−p)−p¯(A|BBA,1−p),and p¯(A|ABB,p)−p¯(A|BBB,p)=1−p¯(B|ABB,p)−(1−p¯(B|BBB,p))=1−p¯(A|BAA,1−p)−(1−p¯(A|AAA,1−p))=p¯(A|AAA,1−p)−p¯(A|BAA,1−p).

We implement the Bayesian model selection (BMS) procedure described in Stephan et al., 2009. Given M models, this procedure aims at deriving a probabilistic belief on the distribution of these models among the general population. This unknown distribution is a categorical distribution, parameterized by the probabilities of the M models, denoted by r=(r1,…,rM), with ∑rm=1. With a finite sample of data, one cannot determine with infinite precision the values of the probabilities rm. The BMS, thus, computes an approximation of the Bayesian posterior over the vector r, as a Dirichlet distribution parameterized by the vector α=(α1,…,αM), i.e., the posterior distribution(21)p(r|α)=1Z(α)∏m=1Mrmαm−1.

Computing the parameters αk of this posterior makes use of the log-evidence of each model for each subject, i.e., the logarithm of the joint probability, p(y|m), of a given subject’s responses, y, under the assumption that a given model, m, generated the responses. We use the model’s maximum likelihood to obtain an approximation of the model’s log-evidence, as (Balasubramanian, 1997)(22)ln⁡p(y|m)≃maxθ[ln⁡p(y|m,θ)]−d2ln⁡N,

where θ denotes the parameters of the model, p(y|m,θ) is the likelihood of the model when parameterized with θ, d is the dimension of θ, and N is the size of the data, that is, the number of responses. (The well-known Bayesian Information Criterion Schwarz, 1978 is equal to this approximation of the model’s log-evidence, multiplied by −1/2.)

In our case, there are M=8 models, each with d=2 parameters: θ=(λ,κ). The posterior distribution over the parameters of the categorical distribution of models in the general population, p(r|α), allows for the derivation of several quantities of interest; following Stephan et al., 2009, we derive two types of quantities. First, given a family of models, that is, a set M={mi} of different models (for instance, the prediction-cost models, or the Bernoulli-observer models), the expected probability of this class of model, that is, the expected probability that the behavior of a subject randomly chosen in the general population follows a model belonging to this class, is the ratio(23)∑m∈Mαm∑m=1Kαm.

We compute the expected probability of the precision-cost models (and the complementary probability of the unpredictability-cost models), and the expected probability of the Bernoulli-observer models (and the complementary probability of the Markov-observer models; see Results).

Second, we estimate, for each family of models M, the probability that it is the most likely, i.e., the probability of the inequality(24)∑m∈Mrm>1/2,

which is called the ‘exceedance probability’. We compute an estimate of this probability by sampling one million times the Dirichlet belief distribution (Equation 21), and counting the number of samples in which the inequality is verified. We estimate in this way the exceedance probability of the precision-cost models (and the complementary probability of the unpredictability-cost models), and the exceedance probability of the Bernoulli-observer models (and the complementary probability of the Markov-observer models; see Results).

Here we derive the expression of the unpredictability cost for Markov observers as a function of the elements of the parameter vector q. For an observer of Markov order 1 (m=1), the vector q has two elements, which are the probability of observing A at a given trial conditional on the preceding outcome being A, and the probability of observing A at a given trial conditional on the preceding outcome being B, which we denote by qA and qB, respectively. The Shannon entropy, H(X;q), implied by the vector q, is the average of the conditional entropies implied by each conditional probability, i.e.,(25)H(X;q)=pBH(X;qB)+pAH(X;qA),

where pA and pB are the unconditional probabilities of observing A and B, respectively (see below), and(26)H(X;qX)=−qXln⁡qX−(1−qX)ln⁡(1−qX),

where X is A or B.

The unconditional probabilities pA and pB are functions of the conditional probabilities qA and qB. Indeed, at trial t+1, the marginal probability of the event xt+1=A, P(xt+1=A), is a weighted average of the probabilities of this event conditional on the preceding stimulus, xt, as given by the law of total probability:(27)P(xt+1=A)=P(xt+1=A|xt=A)P(xt=A)+P(xt+1=A|xt=B)P(xt=B),

i.e.(28)pA=qApA+qB(1−pA).

Solving for pA, we find:(29)pA=qB1+qB−qA, and pB=1−pA=1−qA1+qB−qA.

The entropy H(X;q) implied by the vector q is obtained by substituting these quantities in Equation 25.

Similarly, for m=2 and 3, the 2m elements of the vector q are the parameters qij and qijk, respectively, where i,j,k∈{A,B}, and where qij is the probability of observing A at a given trial conditional on the two preceding outcomes being the sequence ‘ij’, and qijk is the probability of observing A at a given trial conditional on the three preceding outcomes being the sequence ‘ijk’. The Shannon entropy, H(X;q), implied by the vector q, is here also the average of the conditional entropies implied by each conditional probability, as(30)H(X;q)=∑ijpijH(X;qij), for m=2,(31) and H(X;q)=∑ijkpijkH(X;qijk), for m=3,

where pij and pijk are the unconditional probabilities of observing the sequence ‘ij’, and of observing the sequence ‘ijk’, respectively. These unconditional probabilities verify a system of linear equations whose coefficients are given by the conditional probabilities. For instance, for m=2, we have the relation(32)P(xt=A,xt+1=A)=P(xt−1=A,xt=A,xt+1=A)+P(xt−1=B,xt=A,xt+1=A)=P(xt−1=A,xt=A)P(xt+1=A|xt−1=A,xt=A)+P(xt−1=B,xt=A)P(xt+1=A|xt−1=B,xt=A),

i.e.,(33)pAA=pAAqAA+pBAqBA.

The system of linear equations can be written as(34)(pAApABpBApBB)=(qAA0qBA01−qAA01−qBA00qAB0qBB01−qAB01−qBB)(pAApABpBApBB).

The solution is the eigenvector corresponding to the eigenvalue equal to 1 of the matrix in the equation above, with the additional constraint that the unconditional probabilities must sum to 1, i.e., ∑ijpij=1. We find: pBB=(1−2qBB1−qAB+qBA1−qAAqBB1−qAB)−1pBA=qBB1−qABpBB,pAB=pBA,andpAA=qBA1−qAAqBB1−qABpBB.

For m=3, we find the relations:pBBA=pBBBqBBB1−qABB,pBAB=pBBBqBBB1−qABB1−qBBA(1−qAAB)−qABAqAAB1−qBAB(1−qABA)−qABAqAAB,pBAA=pBABqABAqBAB1−qABAqAAB+pBBAqBBA1−qABAqAAB,pAAB=pBAA,pABA=pBAB+pBAA−pBBA,pAAB=pBAA,andpAAA=pBAAqBAA1−qAAA.