elifeeLifeeLife2050-084XeLife Sciences Publications, Ltd8803910.7554/eLife.8803910.7554/eLife.88039.3version of recordResearch ArticleGenetics and GenomicsThe impact of stability considerations on genetic fine-mappingAwAlan Jhttps://orcid.org/0000-0001-9455-787812†JinLionel Chentian3IoannidisNilahnilah@berkeley.edu24SongYun Shttps://orcid.org/0000-0002-0734-9868yss@berkeley.edu124https://ror.org/01an7q238Department of Statistics, University of CaliforniaBerkeleyUnited Stateshttps://ror.org/01an7q238Center for Computational Biology, University of CaliforniaBerkeleyUnited Stateshttps://ror.org/01gmv5d77McKinsey & CompanyNew YorkUnited Stateshttps://ror.org/01an7q238Computer Science Division, University of CaliforniaBerkeleyUnited StatesYoungAlexanderReviewing Editorhttps://ror.org/046rm7j60University of California, Los AngelesUnited StatesWeigelDetlefSenior Editorhttps://ror.org/0243gzr89Max Planck Institute for Biology TübingenGermany
Department of Genetics, University of Pennsylvania, Philadelphia, United States
Fine-mapping methods, which aim to identify genetic variants responsible for complex traits following genetic association studies, typically assume that sufficient adjustments for confounding within the association study cohort have been made, for example, through regressing out the top principal components (i.e., residualization). Despite its widespread use, however, residualization may not completely remove all sources of confounding. Here, we propose a complementary stability-guided approach that does not rely on residualization, which identifies consistently fine-mapped variants across different genetic backgrounds or environments. Simulations show that stability guidance neither outperforms nor underperforms residualization, but each approach picks up different variants considerably often. Critically, prioritizing variants that match between the residualization and stability-guided approaches enhances recovery of causal variants. We further demonstrate the utility of the stability approach by applying it to fine-map eQTLs in the GEUVADIS data. Using 378 different functional annotations of the human genome, including recent deep learning-based annotations (e.g., Enformer), we compare enrichments of these annotations among variants for which the stability and traditional residualization-based fine-mapping approaches agree against those for which they disagree and find that the stability approach enhances the power of traditional fine-mapping methods in identifying variants with functional impact. Finally, in cases where the two approaches report distinct variants, our approach identifies variants comparably enriched for functional annotations. Our findings suggest that the stability principle, as a conceptually simple device, complements existing approaches to fine-mapping, reinforcing recent advocacy of evaluating cross-population and cross-environment portability of biological findings. To support visualization and interpretation of our results, we provide a Shiny app, accessible at https://github.com/songlab-cal/StableFM.
fine-mappingfunctional genomicscomputational predictorsvariant interpretationResearch organismHumanhttps://ror.org/01cwqze88National Institutes of HealthR35-GM134922SongYun Shttps://ror.org/02qenvm24Chan Zuckerberg InitiativeCZF2019-002449SongYun SThe funders had no role in study design, data collection, and interpretation, or the decision to submit the work for publication.Author impact statementIn statistical fine-mapping, signals stable across stratified subgroups can capture functionally important loci missed by covariate adjustment approaches, and prioritizing agreement between both approaches enhances functional variant discovery.publishing-routeprcIntroduction
An important challenge faced by computational precision health research is the lack of generalizability of biological findings, which are often obtained by studying cohorts that do not include particular communities of individuals. Known as the cross-population generalizability or portability problem, the challenge persists in multiple settings, including gene expression prediction (Keys et al., 2020) and polygenic risk prediction (Mostafavi et al., 2020). Generalizable biological signals, such as the functional impact of a variant, are important, as they ensure that general conclusions drawn from cohort-specific analyses are not based on spurious discoveries. Portability problems are potentially attributable to cohort-biased discoveries being treated as generalizable true signals. Efforts to address such problems have included the use of diverse cohorts typically representing multiple population ancestries (Márquez-Luna et al., 2017), meta-analyses of earlier studies across diverse cohorts (Turley et al., 2021; Han and Eskin, 2012; Morris, 2011; Willer et al., 2010), or focusing on biological markers that are more likely a priori to play a causal role (including proxy variables), for example, rare variants (Zaidi and Mathieson, 2020) or the transcriptome (Liang et al., 2022).
In this paper, we consider an approach based on the notion of stability to improve generalizability. Being a pillar of veridical data science (Yu and Kumbier, 2020), stability refers to the robustness of statistical conclusions to perturbations of the data. Perturbations are not arbitrary, but instead, they encode the practitioner’s beliefs about the quality of the data and the nature of the relationship between variables. Well-chosen perturbation schemes will help the practitioner obtain statistical conclusions that are robust, in the sense that they are generalizable rather than spurious findings, and therefore more likely to capture the true signal. For example, in sparse linear models, prioritizing the stability of effect sizes to cross-validation ‘perturbations’ leads to a much smaller set of selected features without reducing predictive performance (Lim and Yu, 2016). In another example involving the application of random forests to detect higher-order interactions between gene regulation features (Basu et al., 2018), interactions stable to bootstrap perturbations are also largely consistent with known physical interactions between the associated transcription factor binding or enhancer sites.
To further investigate the utility of the stability approach, we focus on a procedure known as (genetic) fine-mapping (Schaid et al., 2018). Fine-mapping is the task of identifying genetic variants that causally affect some trait of interest. From the viewpoint of stability, previous works focusing on cross-population stability of fine-mapped variants typically use Bayesian linear models. The linear model has allowed modeling of heterogeneous effect sizes across different user-defined populations (e.g., ethnic groups, ancestrally distinct populations, or study cohorts), and it is common to assume that causal variants share correlated effects across populations (see, e.g., eq. (24) of LaPierre et al., 2021 or eq. (9) of Wen et al., 2015).
Unlike the parametric approaches described above, we here apply a non-parametric fine-mapping method to GEUVADIS (Lappalainen et al., 2013), a database of gene expression traits measured across individuals of diverse genetic ancestries and from different geographical environments, whose accompanying genotypes are available through the 1000 Genomes Project (1 kGP) (Auton et al., 2015). We apply fine-mapping through two approaches, one that is commonly used in practice and another that is guided by stability. First, we perform simulations using 1kGP genotypes to evaluate the strength of each approach. We next evaluate the agreement of fine-mapped variants between the two approaches on GEUVADIS data, and then measure the functional significance of variants picked by both approaches using a much wider range of functional annotations than considered in previous studies. By performing various statistical tests on the functional annotations, we evaluate the advantages brought about by the incorporation of stability. Finally, we investigate the broader applicability of the stability approach by implementing and analyzing a stability-guided version of the fine-mapping algorithm SuSiE on simulated data. To support visualization of results obtained from the GEUVADIS analysis at both the single gene and genome-wide levels, we provide an interactive Shiny app which is accessible at: https://github.com/songlab-cal/StableFM. Our app is open-source and designed to support geneticists in interpreting our results, thereby also addressing a growing demand for accessible, integrative software to interpret genetic findings in the age of big data and variant annotation databases.
ResultsExperimental design
We apply the fine-mapping method PICS (Taylor et al., 2021; Farh et al., 2015; see Algorithm 1 in Probabilistic Identification of Causal SNPs) to the GEUVADIS data (Lappalainen et al., 2013), which consists of T=22,720\begin{document}$T=22,720$\end{document} gene expression traits measured across N=445\begin{document}$N=445$\end{document} individuals, with their accompanying genotypes obtained from the 1000 Genomes Project (Auton et al., 2015). These individuals are of either European or African ancestry, with about four-fifths of the cohort made up of individuals of (self-identified) European descent. In particular, these ancestrally different subpopulations have distinct linkage disequilibrium patterns and environmental exposures, which constitute potential confounders that we wish to stabilize the fine-mapping procedure against.
PICS, like many eQTL analysis methods, requires the lead variant at a locus to compute posterior probabilities, so we perform marginal regressions of each gene expression trait against variants within the fine-mapping locus. Our implementation of PICS generates three sets of variants, which represent putatively causal variants that are (marginally) highly associated, moderately associated, and weakly associated with the gene expression phenotype. (See Materials and methods for details.) As illustrated in Figure 1A, we compare two versions of the fine-mapping procedure—one that is typically performed in practice, and another that is motivated by the stability approach.
An overview of our study of the impact of stability considerations on genetic fine-mapping.
(A) The two ways in which we perform fine-mapping, the first of which (colored in green) prioritizes the stability of variant discoveries to subpopulation perturbations. The data illustrates the case where there are two distinct environments, or subpopulations (denoted E1\begin{document}$E_{1}$\end{document} and E2\begin{document}$E_{2}$\end{document}), that split the observations. (B) Key steps in our comparison of the stability-guided approach with the popular residualization approach.
Stable variant
Specifically, to incorporate stability into fine-mapping and obtain what we call the stable variant or stable SNP, we encode into the algorithm our belief that for a gene expression trait, a causal variant would presumably act through the same mechanism regardless of the population from which they originate. Indeed, if a set of genetic variants were causal, the same algorithm should report it, if run on subsets of the data corresponding to heterogeneous populations. This belief is consistent with recent simulation studies showing that the inclusion of GWAS variants discovered in diverse populations both mitigates false discoveries driven by linkage disequilibrium differences (Li et al., 2022) and improves generalizability of polygenic score construction (Cavazos and Witte, 2021). Hence, the stable variant is the variant with the highest probability of being causal, conditioned on being reported by PICS in the most number of subpopulations. Incorporating stability provides a formal definition of the stable variant.
Top variant
The stability consideration we have described is closely related to a popular procedure known as correction for population structure, which residualizes the trait using measured confounders (e.g., top principal components computed from the genotype matrix). We call the variant returned by the residualization approach the top variant or top SNP. The residualization approach removes the effects of genetic ancestry and environmental exposures on a trait and is used to avoid the risk of low statistical power borne by stratified analyses. The top variant is formally defined toward the end of PICS. We remark that the top variant is a function of the residualized trait, as opposed to the stable variant, which is a function of the unresidualized trait (see Figure 1A).
Simulation study
We simulated 100 genes from GEUVADIS gene expression data, sampled proportionally across the 22 autosomes. Mirroring the approach described in Section 4 of Wang et al., 2020, synthetic gene expression phenotypes were simulated based on the sampled genes and involved two parameters: C\begin{document}$C$\end{document}, the number of causal variants, and ϕ\begin{document}$\phi$\end{document}, the proportion of variance in gene expression explained by variants in cis. Whenever available, gene expression canonical TSS was used to include only variants lying within 1 Mb upstream and downstream when simulating gene expression phenotypes. We considered all combinations of C∈{1,2,3}\begin{document}$C\in\{1,2,3\}$\end{document} and ϕ∈{0.05,0.1,0.2,0.4}\begin{document}$\phi\in\{0.05,0.1,0.2,0.4\}$\end{document}, and simulated two replicates for each gene and for each combination of C\begin{document}$C$\end{document} and ϕ\begin{document}$\phi$\end{document}. Altogether, we generated 100 × 3 × 4 × 2 = 2400 datasets, on which we ran PICS and SuSiE. For PICS, we ran four versions: the stability-guided version and residualization version, which return the stable variant and top variant, respectively; a combined version, which runs stability-guided PICS on covariate-residualized phenotypes; as well as a plain version, where no correction for population stratification or stability guidance was applied. For SuSiE, we ran two versions: the stability-guided version and the residualization version. We also ran a separate set of simulations comprising six scenarios of environmental heterogeneity by ancestry on 10 genes from Chromosomes 1, 20, 21, and 22 (6 × 10 × 3 × 4 × 2 = 1440 generated datasets), on which we ran the plain and stability-guided versions of PICS.
Evaluation on simulated genes
Following Mazumder, 2020, we analyze the recovery probability of all fine-mapping algorithms as a function of the signal-to-noise ratio (SNR), defined here as the ratio of true signal to background noise: SNR=var(Xb)/σ2\begin{document}$\text{SNR}=\text{var}(\mathbf{X}\mathbf{b})/\sigma^{2}$\end{document} (see Simulation study and evaluation details in Materials and methods for details).
Analysis of GEUVADIS data
We compare the results obtained by using each approach, across a range of categories of functional annotations including conservation, pathogenicity, chromatin accessibility, transcription factor binding, and histone modification, and across biological assays and computational predictions. See Table 1 for a full list of annotations covered. (Appendix 5 contains a full description of each quantity and its interpretation.)
A list of 378 functional annotations across which the biological significances of stable and top fine-mapped single nucleotide polymorphisms are compared.
Annotations that report multiple scores have the total number of scores reported shown in parentheses. Scores mined from the FAVOR database (Zhou et al., 2023) are indicated by an asterisk (TSS = transcription start site, bp = base pair).
Functional annotation type
Functional annotation
Ensembl
Distance to Canonical TSS (Cunningham et al., 2022)
Regulatory Features (6; Cunningham et al., 2022)
Computational predictions
CADD∗ (2; Rentzsch et al., 2019)
SIFTVal∗ (Ng and Henikoff, 2003)
FATHMM-XF∗ (Rogers et al., 2018)
LINSIGHT∗ (Huang et al., 2017)
Polyphen∗ (Adzhubei et al., 2010)
PhyloP∗ (3; Pollard et al., 2010)
Gerp∗ (2; Davydov et al., 2010)
B Statistic∗ (McVicker et al., 2009)
FunSeq2∗ (Fu et al., 2014)
ALoFT∗ (Balasubramanian et al., 2017)
Percent CpG in 75 bp window∗ (Rentzsch et al., 2019)
Percent GC in 75 bp window∗ (Rentzsch et al., 2019)
FIRE (Ioannidis et al., 2017)
Enformer (177 tracks × 2 scores per track; Avsec et al., 2021)
Figure 1B summarizes the key steps of our investigation. To be clear, a first comparison is between the set of genes for which the top and stable variants match and the set of genes for which the top and stable variants disagree; comparisons are made between matching variants and one of the non-matching sets of variants (top or stable). A second comparison is restricted to the set of genes with non-matching top and stable variants, that is, between paired sets of variants fine-mapped to a gene, where one set of a pair corresponds to the output of the residualization approach whereas the other corresponds to the output of the stability-guided approach. We additionally compare across three fine-mapping output sets, corresponding to variants that have a high, moderate, or weak marginal association with the expression phenotype. For brevity, we term these three sets Potential Set 1, Potential Set 2, and Potential Set 3, respectively. (See Algorithm 1 for details.) For each Potential Set, we find the top and stable variants as described in Incorporating stability.
Simulations justify exploration of stability guidance
To better understand how stability guidance may support the discovery of causal variants across ancestrally diverse cohorts that possess confounding exogenous factors, we selected 10 genes from Chromosomes 1, 20, 21, and 22 to simulate six scenarios where environmental heterogeneity by ancestry influences gene expression. We consider four scenarios where environmental variance differed between ancestries and two scenarios where environmental mean differed between ancestries. We also vary the number of causal variants and proportion of variance in gene expression expressed in cis. We run Stable PICS, which returns stable variants; alongside a ‘plain’ version of PICS (Plain PICS), which neither performs PC-residualization of phenotypes nor incorporates stability.
We find that, across these simulations, performance was not significantly different between Stable PICS and Plain PICS: both approaches recover the true causal variant in Potential Set 1 with similar frequencies in simulations involving one causal variant, and in simulations involving two or more causal variants the leading potential sets recover at least one causal variant with similar frequency (Figure 2A). These also hold for lower potential sets (Figure 2—figure supplement 1). While the two approaches agree more often than not, there is greater disagreement when considering simulations with low SNR (which may potentially reflect real gene expression data—see Stable variants frequently do not match top variants in GEUVADIS). For example, we identified 83% matching variants in Potential Set 1 across all simulations, but this dropped to 74% when restricting to simulations with ϕ=0.05\begin{document}$\phi=0.05$\end{document}. We next split results by whether the variant returned by Plain PICS had a large posterior probability (PP)—defined as >0.9 to reflect thresholds reported in practice—and found strong agreement between the two approaches when PP was large: for Potential Set 1 the agreement was 98%, Potential Set 2 was 90%, and Potential Set 3 was 93% (Appendix 12—table 1). These observations together imply that the two approaches disagree particularly when SNR is low and the PP of variants returned by Plain PICS is not large, and Stable PICS recovers causal variants accurately in certain cases where Plain PICS fails to do so (otherwise their performances would be different).
Simulation study results.
(A) The frequency with which at least one causal variant is recovered in Potential Set 1 by Plain PICS and Stable PICS, across 1440 simulated gene expression data that incorporate ancestry-mediated environmental heterogeneity. Recovery frequencies are stratified by simulations differing in the number of causal variants, and the Venn diagram reports the number of matching and non-matching variants in Potential Set 1 across all simulations. (B) The frequency with which at least one causal variant is recovered in Potential Set 1 by Combined PICS, Stable PICS, and Top PICS, across 2400 simulated gene expression data. Recovery frequencies are stratified by the SNR parameter ϕ\begin{document}$\phi$\end{document} used in simulations, and the Venn diagram reports the number of matching and non-matching variants in Potential Set 1 across all simulations. (C) The frequency with which at least one causal variant is recovered in Credible Set 1 by Stable SuSiE and Top SuSiE. Venn diagram reports the number of matching and non-matching variants in Potential Set 1 across all simulations. (D) The frequency with which matching and non-matching variants in the first credible or potential set recover a causal variant, obtained from comparing top and stable approaches to an algorithm. We report approximate 95% confidence intervals for each point estimate, by multiplying the associated standard error of the estimate by 1.96.
Plain PICS vs Stable PICS (Potential Sets 2 and 3).
Frequency with which at least one causal variant is recovered in Potential Sets 2 and 3 by Plain PICS and Stable PICS, across 1440 simulated gene expression phenotypes. Recovery frequencies are stratified by simulations differing in the number of causal variants, but Venn diagrams report the number of matching and non-matching variants across all simulations.
Performance of PICS algorithms (Potential Sets 2 and 3).
Frequency with which at least one causal variant is recovered in Potential Sets 2 and 3 by Combined PICS, Stable PICS, and Top PICS, across 2400 simulated gene expression phenotypes. Recovery frequencies are stratified by simulations differing in the signal-to-noise ratio (SNR) parameter ϕ\begin{document}$\phi$\end{document}, but Venn diagrams report the number of matching and non-matching variants across all simulations.
Performance of SuSiE algorithms (Credible Sets 2 and 3).
Frequency with which at least one causal variant is recovered in Potential Sets 2 and 3 by Stable SuSiE and Top SuSiE, across 2400 simulated gene expression phenotypes. Recovery frequencies are stratified by simulations differing in the signal-to-noise ratio (SNR) parameter ϕ\begin{document}$\phi$\end{document}, but Venn diagrams report the number of matching and non-matching variants across all simulations.
Matching vs non-matching variants (Potential and Credible Sets 1 and 2).
Frequencies with which matching and non-matching variants in the credible or potential set recover a causal variant, obtained from comparing top and stable approaches to an algorithm. Analysis is performed over 2400 simulated gene expression phenotypes, and recovery frequencies are stratified by simulations differing in the signal-to-noise ratio (SNR) parameter ϕ\begin{document}$\phi$\end{document}. (A) Credible or Potential Set 2. (B) Credible or Potential Set 3.
Stable PICS vs Stable SuSiE (one causal variant).
Empirical discrete probability distributions over the number of causal variants (0 or 1) recovered by stability-guided algorithms in simulations with one causal variant, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible or potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Stable PICS vs Stable SuSiE (two causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, or 2) recovered by stability-guided algorithms in simulations with two causal variants, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible or potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Stable PICS vs Stable SuSiE (three causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by stability-guided algorithms in simulations with three causal variants, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible or potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Matching Top vs Stable SNP posterior probabilities.
Posterior probabilities of matching top and stable variants across 2400 simulated gene expression phenotypes. Points are colored by the number of causal variants,,S∈{1,2,3}\begin{document}$S\in\{1,2,3\}$\end{document} set in simulations.
Non-matching Top vs Stable SNP posterior probabilities.
Posterior probabilities of non-matching top and stable variants across 2400 simulated gene expression phenotypes. Points are colored by the number of causal variants,,S∈{1,2,3}\begin{document}$S\in\{1,2,3\}$\end{document} set in simulations.
Stable PICS vs SuSiE (one causal variant).
Empirical discrete probability distributions over the number of causal variants (0 or 1) recovered by Stable PICS or Top SuSiE in simulations with one causal variant, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible or potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Stable PICS vs SuSiE (two causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, or 2) recovered by Stable PICS or Top SuSiE in simulations with two causal variants, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible or potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Stable PICS vs SuSiE (three causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by Stable PICS or Top SuSiE in simulations with three causal variants, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible or potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Distribution of the number of variants recovered by PICS (one causal variant).
Empirical discrete probability distributions over the number of causal variants (0 or 1) recovered by PICS algorithms in simulations with one causal variant, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a larger number of potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Distribution of the number of variants recovered by PICS (two causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, or 2) recovered by PICS algorithms in simulations with two causal variants, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a larger number of potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Distribution of the number of variants recovered by PICS (three causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by PICS algorithms in simulations with three causal variants, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Distribution of the number of variants recovered by SuSiE (one causal variant).
Empirical discrete probability distributions over the number of causal variants (0 or 1) recovered by SuSiE algorithms in simulations with one causal variant, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible sets on the distribution is shown (increasing number of included sets from top to bottom).
Distribution of number of variants recovered by SuSiE (two causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, or 2) recovered by SuSiE algorithms in simulations with two causal variants, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible sets on the distribution is shown (increasing number of included sets from top to bottom).
Distribution of number of variants recovered by SuSiE (three causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by SuSiE algorithms in simulations with three causal variants, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of credible sets on the distribution is shown (increasing number of included sets from top to bottom).
Variant recovery frequency of PICS and SuSiE matching and non-matching variants (one causal variant).
Frequency with which the causal variant is recovered by a matching variant, non-matching top variant, or non-matching stable variant in simulations involving one causal variant. Recovery frequencies are stratified by simulations differing in the signal-to-noise ratio (SNR) parameter ϕ\begin{document}$\phi$\end{document}.
Distribution of number of variants recovered by PICS matching and non-matching variants (two causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, or 2) recovered by matching Top and Stable PICS variants, non-matching Top PICS variants, and non-matching Stable PICS variants, in simulations involving two causal variants. Empirical distributions are stratified by simulations differing in the signal-to-noise ratio (SNR) parameter ϕ\begin{document}$\phi$\end{document} (increasing SNR from left to right).
Distribution of the number of variants recovered by SuSiE matching and non-matching variants (two causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, or 2) recovered by matching Top and Stable SuSiE variants, non-matching Top SuSiE variants, and non-matching Stable SuSiE variants, in simulations involving two causal variants. Empirical distributions are stratified by simulations differing in the signal-to-noise ratio (SNR) parameter ϕ\begin{document}$\phi$\end{document} (increasing SNR from left to right).
Distribution of number of variants recovered by PICS matching and non-matching variants (three causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by matching Top and Stable PICS variants, non-matching Top PICS variants, and non-matching Stable PICS variants, in simulations involving three causal variants. Empirical distributions are stratified by simulations differing in the signal-to-noise ratio (SNR) parameter ϕ\begin{document}$\phi$\end{document} (increasing SNR from left to right).
Distribution of number of variants recovered by SuSiE matching and non-matching variants (three causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by matching Top and Stable SuSiE variants, non-matching Top SuSiE variants, and non-matching Stable SuSiE variants, in simulations involving three causal variants. Empirical distributions are stratified by simulations differing in the signal-to-noise ratio (SNR) parameter ϕ\begin{document}$\phi$\end{document} (increasing SNR from left to right).
Plain PICS vs Stable PICS in environmental heterogeneity simulations (one causal variant).
Empirical discrete probability distributions over the number of causal variants (0 or 1) recovered by Plain or Stable PICS in simulations with one causal variant and environmental heterogeneity, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Plain PICS vs Stable PICS in environmental heterogeneity simulations (two causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, or 2) recovered by Plain or Stable PICS in simulations with two causal variants and environmental heterogeneity, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a greater number of potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Plain PICS vs Stable PICS in environmental heterogeneity simulations (three causal variants).
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by Plain or Stable PICS in simulations with three causal variants and environmental heterogeneity, stratified by the SNR parameter used in simulations (increasing SNR from left to right). The impact of including a larger number of potential sets on the distribution is shown (increasing number of included sets from top to bottom).
Plain PICS vs Stable PICS in ‘variance shift (<italic>t</italic> = 8)’ environmental heterogeneity simulations.
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by Plain or Stable PICS in ‘t=8\begin{document}$t=8$\end{document}’ simulations involving environmental heterogeneity (variance shift scenario), stratified by the SNR parameter used in simulations (increasing SNR from left to right). Each row reports the distribution for simulations with a specific number of causal variants (1, 2, or 3), and we use all three potential sets to compute the number of causal variants recovered in each case.
Plain PICS vs Stable PICS in ‘variance shift (<italic>t</italic> = 16)’ environmental heterogeneity simulations.
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by Plain or Stable PICS in ‘t=16\begin{document}$t=16$\end{document}’ simulations involving environmental heterogeneity (variance shift scenario), stratified by the SNR parameter used in simulations (increasing SNR from left to right). Each row reports the distribution for simulations with a specific number of causal variants (1, 2, or 3), and we use all three potential sets to compute the number of causal variants recovered in each case.
Plain PICS vs Stable PICS in ‘variance shift (<italic>t</italic> = 128)’ environmental heterogeneity simulations.
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by Plain or Stable PICS in ‘t=128\begin{document}$t=128$\end{document}’ simulations involving environmental heterogeneity (variance shift scenario), stratified by the SNR parameter used in simulations (increasing SNR from left to right). Each row reports the distribution for simulations with a specific number of causal variants (1, 2, or 3), and we use all three potential sets to compute the number of causal variants recovered in each case.
Plain PICS vs Stable PICS in ‘variance shift (<italic>t</italic> = 256)’ environmental heterogeneity simulations.
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by Plain or Stable PICS in ‘t=256\begin{document}$t=256$\end{document}’ simulations involving environmental heterogeneity (variance shift scenario), stratified by the SNR parameter used in simulations (increasing SNR from left to right). Each row reports the distribution for simulations with a specific number of causal variants (1, 2, or 3), and we use all three potential sets to compute the number of causal variants recovered in each case.
Plain PICS vs Stable PICS in ‘mean shift (|<italic>i</italic> − 3|)’ environmental heterogeneity simulations.
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by Plain or Stable PICS in ‘|i−3|\begin{document}$|i-3|$\end{document}’ simulations involving environmental heterogeneity (mean shift scenario), stratified by the SNR parameter used in simulations (increasing SNR from left to right). Each row reports the distribution for simulations with a specific number of causal variants (1, 2, or 3), and we use all three potential sets to compute the number of causal variants recovered in each case.
Plain PICS vs Stable PICS in ‘mean shift (<italic>i</italic> = 3)’ environmental heterogeneity simulations.
Empirical discrete probability distributions over the number of causal variants (0, 1, 2, or 3) recovered by Plain or Stable PICS in ‘i=3\begin{document}$i=3$\end{document}’ simulations involving environmental heterogeneity (mean shift scenario), stratified by the SNR parameter used in simulations (increasing SNR from left to right). Each row reports the distribution for simulations with a specific number of causal variants (1, 2, or 3), and we use all three potential sets to compute the number of causal variants recovered in each case.
Looking closely at each simulation scenario, we see that Stable PICS recovers a causal variant more frequently than Plain PICS in scenarios where there are multiple causal variants and environmental heterogeneity is driven by a ‘spiked mean shift’ (Simulation study and evaluation details), while Plain PICS generally outperforms when environmental heterogeneity is driven by differences in environmental variation. For example, in a set of spiked mean shift simulations with three causal variants, with low SNR (ϕ=0.1\begin{document}$\phi=0.1$\end{document}) and with exogenous variable drawn from N(2σ,σ2)\begin{document}$N(2\sigma,\sigma^{2})$\end{document} for only GBR individuals (rest are drawn from N(0,σ2)\begin{document}$N(0,\sigma^{2})$\end{document}), the frequencies at which at least one causal variant is recovered were 80% and 75% for Stable PICS and Plain PICS, respectively (see Figure 2—figure supplement 32). However, none of these differences in performance is statistically significant. In summary, these findings demonstrate that non-genetic confounding in cohorts can reduce power in methods not adjusting or accounting for ancestral confounding but can be remedied by approaches that do so. (Performances stratified by SNR and number of potential sets included are summarized in Figure 2—figure supplements 24–26; performances stratified by each scenario are summarized in Figure 2—figure supplements 27–32).
Simulations reveal advantages (and disadvantages) of stability guidance
We next conduct a larger set of simulations using 100 genes selected across all 22 autosomes. Similar to the earlier set of simulations, we vary both the number of causal variants and proportion of variance in gene expression explained by variants in cis. However, here we compare stability-guided fine-mapping against residualization fine-mapping, a commonly used approach.
No difference in power between stability guidance and residualization, although considerably many variants do not match
Comparing stable and top variants returned by Stable PICS and PICS run via the residualization approach (Top PICS), we observed similar causal variant recovery rates. Across all 2400 simulated gene expression phenotypes, the stable variant in Potential Set 1 was causal with frequency 0.63, which is close to and not significantly different from the frequency at which the Potential Set 1 top variant was causal (0.64; McNemar test p-value = 0.38). Similarly close frequencies were observed for lower potential sets (Potential Set 2: stable=0.12,top=0.13\begin{document}$\text{stable}=0.12,\text{top}=0.13$\end{document}; Potential Set 3: stable=0.041,top=0.047\begin{document}$\text{stable}=0.041,\text{top}=0.047$\end{document}), as well as when results were stratified by SNR (Figure 2B). However, top and stable variants often disagreed, with Potential Set 1 having 72% matching top and stable variants, Potential Set 2 matching at 40% and Potential Set 3 matching at 25% (Figure 2—figure supplement 2). Stratifying results by simulation parameters (number of causal variants, S\begin{document}$S$\end{document}, and SNR, ϕ\begin{document}$\phi$\end{document}), we generally observed matching fractions increasing with SNR, at least in Potential Set 1: for example, in simulations involving one causal variant, the matching fraction is 69.5% under SNR=0.05/(1−0.05)≈0.053\begin{document}$\text{SNR}=0.05/(1-0.05)\approx 0.053$\end{document}, and it increases monotonically to 89.5% under SNR=0.4/(1−0.4)≈0.67\begin{document}$\text{SNR}=0.4/(1-0.4)\approx 0.67$\end{document} (Appendix 12—table 2). There was no clear relationship between matching fractions and the number of causal variants simulated.
Searching for matching variants between Top PICS and Stable PICS improves causal variant recovery
Given that stable and top variants frequently do not match despite achieving similar performance, we suspect that, similar to the smaller set of simulations on which Plain PICS and Stable PICS were compared, each approach can recover causal variants in scenarios where the other does not. We thus explore ways to combine the residualization and stability-driven approaches, by considering (1) combining them into a single fine-mapping algorithm (we call the resulting procedure Combined PICS); and (2) prioritizing matching variants between the two algorithms. Comparing the performance of Combined PICS against both Top and Stable PICS, however, we find no significant difference in its ability to recover causal variants (Figure 2B). This conclusion held even when we analyzed performance by stratifying simulations by SNR and number of causal variants simulated (ϕ\begin{document}$\phi$\end{document} and S\begin{document}$S$\end{document} parameters; see Figure 2—figure supplements 13–15). On the other hand, matching variants between Top and Stable PICS are significantly more likely to be causal. Across all simulations, a matching variant in Potential Set 1 was 2.5× as likely to be causal than either a non-matching top or stable variant (Figure 2D)—a result that was qualitatively consistent even when we stratified simulations by SNR and number of causal variants simulated (Figure 2—figure supplements 19, 20 and Figure 2—figure supplement 22). A similar trend was observed for Potential Set 2, although we do not see much higher causal variant recovery when comparing matching and non-matching variants in Potential Set 3 (Figure 2—figure supplement 4).
Stability guidance improves causal variant recovery in SuSiE
To explore the applicability of stability guidance beyond PICS, we developed a similar procedure that runs the fine-mapping algorithm SuSiE (Wang et al., 2020) on multiple slices and returns a stable variant. We call this approach Stable SuSiE. We then compared Stable SuSiE against SuSiE applied to residualized phenotypes (Top SuSiE), analogous to our comparisons for PICS earlier. First, similar to PICS, we generally observe no significant differences in performance between Stable and Top SuSiE, although empirically more causal variants are recovered by Top SuSiE in larger SNR simulation settings (Figure 2C), while more causal variants are recovered by Stable SuSiE in lower credible sets (Figure 2—figure supplement 3; see also Figure 2—figure supplements 16–18). Similar to PICS, we also observed matching fractions increasing with SNR in Potential Set 1 (Appendix 12—table 3). Next, comparing matching vs non-matching variants between the approaches, we observe a boost in causal variant recovery similar to that observed in PICS (Figure 2D), which persisted even when we stratified results by simulation parameters (Figure 2—figure supplements 19 and 21, Figure 2—figure supplement 23). However, one difference is that, unlike in PICS, this boost was unique to Credible Set 1—for the other two credible sets, improvements in causal variant recovery were observed only when SNR is high, with non-matching Stable SuSiE variants recovering at least one causal variant with the highest frequency in low SNR scenarios (Figure 2—figure supplement 4).
Other analyses
Our key findings from the simulation study are as follows. Stability guidance, on its own, does not significantly outperform nor underperform other standard approaches at causal variant recovery. However, prioritizing variants that agree between stability-guided fine-mapping and standard fine-mapping can significantly boost causal variant recovery. In the Supplement, we also describe findings from investigations into the impact of including more potential sets on matching frequency and causal variant recovery (Appendix 2, with Figure 2—figure supplements 5–7 discussed), the differences in PP between non-matching variants (Appendix 3, with Figure 2—figure supplements 8 and 9 discussed), the relative performance of SuSiE and Stable PICS (Appendix 4, with Figure 2—figure supplements 10–12 discussed), and interpreting matching variants with very low (stable) PP (Appendix 11).
Stable variants frequently do not match top variants in GEUVADIS
Having identified scenarios that favor stability guidance, as well as strategies for using stability guidance to recover causal variants, we now turn to analysis of real gene expression data from GEUVADIS. As a preliminary analysis, when interrogating if the residualization and stability-guided approaches produce the same candidate causal variants, we find that for Potential Set 1, which corresponds to the set typically reported in fine-mapping studies, 56.2% of genes had matching top and stable variants (see Figure 3). Moving down potential sets, we find less agreement (Potential Set 2: 36.2%, Potential Set 3: 25.6%), providing evidence that as the marginal association of a variant with the expression phenotype decreases, the two approaches prioritize different signals when searching for putatively causal variants. This result is most consistent with our simulations using S=3\begin{document}$S=3$\end{document} causal variants and low SNR (ϕ=0.05\begin{document}$\phi=0.05$\end{document}), where we observed 56.5%, 37%, and 24.5% matching top and stable variants in Potential Sets 1, 2, and 3, respectively.
Venn diagram showing the number of matching and non-matching variants for Potential Set 1 in GEUVADIS fine-mapped variants.
Matching GEUVADIS Top vs Stable SNP posterior probabilities.
Pair density plot of posterior probabilities of the top variant and the stable variant, in case they match.
Non-matching GEUVADIS Top vs Stable SNP posterior probabilities.
Pair density plot of posterior probabilities of the top variant and the stable variant, in case they do not match.
Because our simulations demonstrate that prioritizing matching variants boosts causal variant recovery, we proceeded with comparing functional impact of matching stable and top variants against non-matching variants. For the rest of this section, we report only results for Potential Set 1, given that it corresponds to the set typically reported in fine-mapping. Results for the other potential sets are provided in Appendices 7 and 8.
Matching vs non-matching variants
For each gene, our algorithm finds the top eQTL variant and the stable eQTL variant, which may not coincide. We thus run (one-sided) unpaired Wilcoxon tests on matching and non-matching sets of variants to detect significant functional enrichment of one set of variants over the other. We find that the top variants that are also stable for the corresponding gene (Nmatch=12743\begin{document}$N_{\text{match}}=12743$\end{document} maximum annotatable) score significantly higher in functional annotations than the top variants that are not stable (Nnon-match=9921\begin{document}$N_{\text{non-match}}=9921$\end{document} maximum annotatable). Notably, 361 out of 378 functional annotations report one-sided greater p-values <0.05 for the matching (i.e., both top and stable) variants after correcting for multiple testing using the Benjamini–Hochberg (BH) procedure, with many of these annotations measuring magnitudes of functional impact or functional enrichment (e.g., Enformer perturbation scores, FATHMM.XF score). Among the 17 remaining functional annotations, none has significantly lower scores for the matching variants. (Appendix 7 lists these functional annotations in detail.) We also find that empirically, the matching variants tend to have greater agreement in posterior probabilities than non-matching variants (Figure 2—figure supplements 1 and 2).
As an example of a significant functional annotation, consider raw CADD scores (Rentzsch et al., 2019)—a higher value of which indicates a greater likelihood of deleterious effects. Out of the 22,559 genes for which both the top and the stable variant are annotatable, looking at the distribution of top variant scores, the one corresponding to the 12,685 genes with matching top and stable variants stochastically dominates the one corresponding to the remaining 9874 genes (Figure 4A). This relationship is more pronounced when we inspect PHRED-scaled CADD scores, where we apply a sliding cutoff threshold for calling variant deleteriousness (i.e., potential pathogenicity—see Rentzsch et al., 2019). We find that a greater proportion of matching variants than of either non-matching variant is classified as deleterious under the typical range of deleteriousness cutoffs (Figure 4B).
Distribution of computational VEP scores across matching and non-matching variants.
Top row. CADD scores. (A) Empirical cumulative distribution functions of raw CADD scores of matching and non-matching variants across all genes, for Potential Set 1. Non-matching variants are further divided into stable and top variants, with a score lower threshold of 1.0 and upper threshold of 5.0 used to improve visualization. (B) For a deleteriousness cutoff, the percent of (1) all matching variants, (2) all non-matching top variants, and (3) all non-matching stable variants, which are classified as deleterious. We use a sliding cutoff threshold ranging from 10 to 20 as recommended by CADD authors. For each value along the x-axis, 95% confidence intervals for point estimates on the y-axis were obtained using the Sison-Glaz method for constructing multinomial distribution standard errors (R command DescTools::MultinomCI(...)). Bottom row. Empirical cumulative distribution functions of perturbation scores of Enformer-predicted H3K27me3 ChIP-seq track. Score upper threshold of 0.015 and empirical CDF lower threshold of 0.5 used to improve visualization. (C) Perturbation scores computed from predictions based on centering input sequences on the gene TSS as well as its two flanking positions. (D) Perturbation scores computed from predictions based on centering input sequences on the gene TSS only.
Another example demonstrating significant functional enrichment of matching variants over non-matching variants is the perturbation scores on H3K9me3 ChIP-seq peaks, as predicted by the Enformer (Avsec et al., 2021). Out of the 6364 genes for which the distance of both the top and the stable variant to the TSS are within the Enformer input sequence length constraint, looking at the distribution of top variant scores, the one corresponding to the 4491 genes with matching top and stable variants stochastically dominates the one corresponding to the remaining 1873 genes (Figure 4C,D). This relationship is true regardless of whether perturbation scores are calculated from an average of input sequences centered on the gene TSS and its two flanking positions (Figure 4C), or from input sequences centered on the gene TSS only (Figure 4D). (See Appendix 6 for details on Enformer annotation calculation.)
Similarly, amongst all stable variants, those variants that match the top variant are significantly more enriched in functional annotations: 363 functional annotations report one-sided greater BH-adjusted p-values <0.05 for the matching variants, and none of the remaining 15 annotations present significant depletion for the matching variants set.
Top vs stable variants when they do not match
Focusing on the genes for which the top and stable variants are different, we run (one-sided) paired Wilcoxon tests to detect significant functional enrichment of one set of variants over the other. We find in general that for some genes, stable variants can carry more functional impact than top variants, and for other genes, top variants carry more functional impact—although neither of these patterns is statistically significant genome-wide after multiple testing correction using the BH procedure. For example, for raw CADD scores, out of the 9874 genes for which the top and the stable variants do not match, 4906 genes have higher scoring stable variants than top variants, whereas 4968 genes have higher scoring top variants than stable variants (Figure 5A). Looking at the accompanying PHRED-scaled CADD scores (CADD PHRED) for Potential Set 1, when applying a sliding cutoff threshold for calling variant deleteriousness, we find that even though a higher fraction of genes have top but not stable variant classified as deleterious, the difference is usually not significant (Figure 5B). Figure 5B also demonstrates that no matter how the cutoff is chosen, there are genes for which the top variant is not classified as deleterious while the stable variant is. Taken together, these observations suggest that the stability-guided approach can sometimes be more useful at identifying variants of functional significance, and in a broad sense, both fine-mapped variants should be equally prioritized for potential of carrying functional impact.
Comparison of CADD scores across non-matching top and stable variants.
(A) Paired scatterplot of raw CADD scores of both top and stable variant for each gene, for Potential Set 1. (B) Percent of genes that are classified as (1) having deleterious top variant only, (2) having deleterious stable variant only, and (3) having both top and stable variant deleterious, using a sliding cutoff threshold ranging from 10 to 20 as recommended by CADD authors.
Because our comparison between the top and stable variants yielded no significant functional enrichment of one over the other, we investigate whether external factors—for example, the posterior probability of the variants—might moderate the relative enrichment of one variant over the other (i.e., we perform trend analysis—see Section for a list of all moderators). Here, we find that all but one of the moderators considered—namely, the Posterior Probability of Top Variant (see Table 2)—did not produce any significant trends for Potential Set 1. There is a small but significant positive correlation (Pearson’s r=0.05\begin{document}$r=0.05$\end{document}) between the posterior probability of the top variant and the difference between the stable variant and top variant FIRE scores (Ioannidis et al., 2017), and a small but significant negative correlation (Pearson’s r=−0.07\begin{document}$r=-0.07$\end{document}) between the posterior probability of the top variant and the difference between the stable variant and top variant Absolute Distance to Canonical TSS. (Details are reported in Appendix 8.)
List of six moderating factors considered.
Moderator
Quantity/statistic computed
(1) Degree of Stability
No. subpopulations for which stable variant has positive probability
(2) Population Diversity
Maximum of pairwise allele frequency difference between subpopulations for which stable variant has positive posterior probability
(3) Population Differentiation
Maximum FST\begin{document}$F_{ST} $\end{document} between subpopulations forwhich stable variant has positive posterior probability
(4) Inclusion of Distal Subpopulations (Top)
Whether or not the top variant also had positive probability in Yoruban subpopulation when the stability-guided approach was used
(5) Inclusion of Distal Subpopulations (Stable)
Whether or not the stable variant had positive probability in Yoruban subpopulation when the stability-guided approach was used
(6) Degree of Certainty of Causality Using Residualization Approach
Posterior probability of top variant
Additional comparisons
We perform various conditional analyses to evaluate whether additional restrictions to characteristics of fine-mapping outputs may boost the power of either approach over the other. Such characteristics include (1) the positive posterior probability support (i.e., how many variants reported a positive posterior probability from fine-mapping), and (2) the posterior probability of the top or stable variant. Results from our conditional analysis of (1) are reported in Appendix 9. As an example, for (2) we further perform a comparison between top and stable variants, by restricting to genes where the posterior probability of the top variant or the stable variant exceeds 0.9. Such restriction of valid fine-mapped variant-gene pairs is useful in training variant effect prediction models requiring reliable positive and negative examples, as seen in Wang et al., 2021. We find that for genes where the top variant posterior probability exceeds 0.9, there is no significant enrichment of the top variant over the stable variant across the 378 annotations considered (all BH-adjusted p-values exceed 0.05). Interestingly, when focusing on genes where the stable variant posterior probability exceeds 0.9, FIRE scores of the top variant are significantly larger than the stable variant (BH-adjusted p-value =1×10−6\begin{document}$=1\times 10^{-6}$\end{document}). Detailed results are reported in Appendix 10.
Discussion
We have shown that a stability-guided approach complements existing approaches to detect biologically meaningful variants in genetic fine-mapping. Through various statistical comparisons, we have found that prioritizing the agreement between existing approaches and a stability-guided approach enhances the functional impact of the fine-mapped variant. Incorporating stability into fine-mapping also provides an adjuvant approach that helps discover variants of potential functional impact in case standard approaches fail to pick up variants of functional significance. Our findings are consistent with earlier reports of stable discoveries having the tendency to capture actual physical or mechanistic relationships, potentially making such discoveries generalizable or portable (Basu et al., 2018).
The link between stability and generalizability is not new. In the machine learning and statistics literature, it has been shown that stable algorithms provably lead to generalization errors that are well-controlled (Bousquet and Elisseeff, 2002, Theorem 17, p. 510). Furthermore, in certain classes of algorithms (e.g., sparse regression), it has been demonstrated empirically that this mathematical relationship is explained precisely by the stable algorithm removing spurious discoveries (Lim and Yu, 2016).
The stability-guided approach is also distinct from other subsampling approaches, such as the bootstrap (Efron and Tibshirani, 1994). First, the bootstrap is more often deployed as a method for calibrating uncertainty surrounding a prediction, which is not the objective of our method. Next, in settings where the bootstrap is deployed as a type of perturbation against which a prediction or estimand is expected to be stable (Basu et al., 2018), a stability threshold is implicitly needed and would require tuning to be chosen (e.g., in Basu et al., 2018 this threshold was chosen to be 50%). Here, we leverage interpretable existing external annotations to define the perturbation against which we expect the fine-mapped variant to be stable—in other words, the user relates what is meant by the fine-mapped variant being stable to a biologically meaningful concept like ‘portable across environmentally and LD pattern-wise heterogeneous populations’.
The last sentence in the preceding paragraph suggests there are teleological similarities between the stability-guided approach and meta-analysis approaches (Turley et al., 2021). We emphasize that meta-analyses rely on already analyzed cohorts, thereby implicitly assuming that within-cohort heterogeneities have been sufficiently accounted for prior to the reporting of findings for that cohort. The stability-guided approach, however, is relevant to the cohort-specific analysis itself, where existing approaches may present methodological insufficiencies resulting in inflated false discoveries. In other words, whereas the goal of meta-analysis may be stated as identifying consistent hits across cohorts while also assuming that findings specific to each cohort are reliable, the goal of a stability-guided approach is to search for consistent signals despite the presence of potential confounders within a single cohort. Our analysis has focused on comparing the stability-guided approach against residualization, a convenient and popular approach to account for population structure, which reflects this difference in purpose.
Related to the previous point, while revising our work, we came across a few recent papers that developed multi-ancestry fine-mapping algorithms to investigate causal variant heterogeneity across ancestries. Gao and Zhou, 2024 and Yuan et al., 2024 found that complex traits have both shared and non-negligible ancestry-specific causal signals, although the proportion of trait-specific causal variants that are ancestry-specific is probably low (Hu et al., 2025; Shi et al., 2021). On the other hand, for molecular quantitative traits, which include RNA sequencing reads that we have studied in this work, Lu et al., 2025 reported highly correlated effect sizes in cis QTLs across ancestries, with effect heterogeneity concentrated at predicted loss-of-function-intolerant genes. These findings are all consistent with our goal of leveraging stability guidance to identify generalizable causal signals and also imply that stability guidance would be particularly useful for molecular quantitative trait fine-mapping.
There are several limitations to our present work. First, we have chosen to focus on a non-parametric approach to fine-mapping, but multiple parametric fine-mapping approaches have been extended to incorporate cross-population heterogeneity (e.g., LaPierre et al., 2021; Lu et al., 2022; Wen et al., 2015). While our present work is a proof-of-concept of the applicability of the stability to genetic fine-mapping, we believe that future work focusing on comparing functional impact of variants prioritized by our stability-driven approach or these parametric methods will shed more light on the efficacy of the stability principle at detecting generalizable biological signals. Our results for SuSiE on simulated gene expression data are promising in this regard. Second, our analyses assume that there is at most one cis eQTL prioritized by each potential set of PICS. This is owing to the implicit assumption that all other variants in high LD with the causal variant are simply tagging it, though we acknowledge the possibility of relaxing this assumption. Understanding the impact of this relaxation requires extensive investigation; hence, we defer it to future work. Third, while we have relied on numerous computational and experimental annotations to evaluate the functional impact of our fine-mapped variants, some computational variant effect predictors may themselves be biased owing to the lack of ancestral diversity of training data. Recent work has found that deep learning-based gene expression predictors—despite outperforming traditional statistical models at inferring regulatory tracks—explain surprisingly little of gene expression variation across ancestries (Huang et al., 2023) and yield limited power at predicting individual gene expression levels more broadly (Sasse et al., 2023). While this complicates the use of such predictors in their present form as strong evidence of functional impact, it is more constructive to understand their shortcomings and develop strategies to fine-tune their predictions for future use as trustworthy functional impact metrics—an emerging desideratum in the era of computational biomedicine (Barbadilla-Martínez et al., 2025; Benegas et al., 2025; Katsonis et al., 2022; Livesey et al., 2025).
In closing, while our work explores stability to subpopulation perturbations where subpopulations are defined by ancestry, we emphasize that our stability-guided slicing methodology is applicable to all settings where meaningful external labels are available to the data analyst. For instance, environmental or geographical variables, which are well-recognized determinants of some health outcomes (Abdellaoui et al., 2022; Favé et al., 2018) and arguably better measure potential confounders than ancestry labels, can be the basis on which slices are defined in the stability-guided approach. As the barriers to access larger biobank-scale datasets, which contain such aforementioned variables, continue to be lowered, we expect stability-driven analyses conducted on such data and relying on carefully defined slices will help users better understand genetic drivers of complex traits. Given its utility in our present work, we believe that the stability approach in precision medicine may find uses beyond genetic fine-mapping and few other biological tasks previously studied, ultimately empowering the discovery of veridical effects not previously known.
Materials and methods
We use publicly available GEUVADIS B-lymphocyte RNA-seq measurements from 445 individuals (Lappalainen et al., 2013), whose genotypes are also available from the 1000 Genomes Project (Auton et al., 2015). The 445 individuals come from five populations: Tuscany Italian (TSI, N=91\begin{document}$N=91$\end{document}), Great British (GBR, N=86\begin{document}$N=86$\end{document}), Finnish (FIN, N=92\begin{document}$N=92$\end{document}), Utah White American (CEU, N=89\begin{document}$N=89$\end{document}), and Yoruban (YRI, N=87\begin{document}$N=87$\end{document}). The mRNA measurements are normalized for library depth, expression frequency across individuals as well as PEER factors, as reported in the GEUVADIS Project (here). Of the available normalized gene expression phenotypes, only those with non-empty sets of variants lying within 1 Mb upstream or downstream of the canonical transcription start site were kept for (cis) fine-mapping. This process yielded 22,664 genes used in all subsequent analyses involving the PICS fine-mapping algorithm.
Probabilistic identification of causal SNPs
We implement PICS (Farh et al., 2015), which is based on an earlier genome-wide analysis identifying genetic and epigenetic maps of causal autoimmune disease variants (Farh et al., 2015).
We assume that X\begin{document}$\mathbf{X}$\end{document} and y\begin{document}$\mathbf{y}$\end{document} are the N×P\begin{document}$N\times P$\end{document} locus haplotype (or genotype) matrix and the N×1\begin{document}$N\times 1$\end{document} vector of phenotype values, respectively, with y\begin{document}$\mathbf{y}$\end{document} possibly already adjusted by relevant covariates. For consistency of exposition, let the SNPs be named A1,…,AP\begin{document}$A_{1},\ldots,A_{P}$\end{document}. Recall the goal of fine-mapping is to return information about which SNP(s) in the set {A1,…,AP}\begin{document}$\{A_{1},\ldots,A_{P}\}$\end{document} is (are) causal, given the input pair {X,y}\begin{document}$\{\mathbf{X},\mathbf{y}\}$\end{document} and possibly other external information such as functional annotations or, more generally, prior biological knowledge.
Overview
PICS is a Bayesian, non-parametric approach to fine-mapping. Given the observed patterns of association at a locus, and furthermore not assuming a parametric model relating the causal variants to the trait itself, one can estimate the probability that any SNP is causal by performing permutations that preserve its marginal association with the trait as well as the LD patterns at the locus.
To see how this is accomplished, suppose that A1\begin{document}$A_{1}$\end{document} is the lead SNP. We are interested in P(Aicausal|A1=Alead)\begin{document}$\mathbb{P}(A_{i}^{\text{causal}}|A_{1}=A^{\text{lead}})$\end{document}, the probability that Ai\begin{document}$A_{i}$\end{document} is causal, for each i∈[P]\begin{document}$i\in[P]$\end{document}. By Bayes’ theorem,P(Aicausal|A1=Alead)∝P(A1=Alead|Aicausal)×P(Aicausal).\begin{document}$$\displaystyle \mathbb{P}(A_{i}^{\text{causal}}|A_{1}=A^{\text{lead}}) \propto \mathbb{P}(A_{1}=A^{\text{lead}}|A_{i}^{\text{causal}}) \times \mathbb{P}(A_{i}^{\text{causal}}). $$\end{document}
Focusing on the focal SNP i\begin{document}$i$\end{document}, permute the rows of X\begin{document}$\mathbf{X}$\end{document} such that the association between the focal SNP and the trait y\begin{document}$\mathbf{y}$\end{document} is invariant; see Appendix 1 for a concrete mathematical description of this set of constrained permutations. Then the first term on the RHS of Equation 1, P(A1=Alead|Aicausal)\begin{document}$\mathbb{P}(A_{1}=A^{\text{lead}}|A_{i}^{\text{causal}})$\end{document}, is estimated by the proportion of all permutations where A1\begin{document}$A_{1}$\end{document} emerges as the lead SNP. The second term, P(Aicausal)\begin{document}$\mathbb{P}(A_{i}^{\text{causal}})$\end{document}, is the prior probability of the focal SNP being causal, which the user can choose based on prior knowledge. The default setting in PICS is P(A1causal)=…=P(APcausal)\begin{document}$\mathbb{P}(A_{1}^{\text{causal}})=\ldots=\mathbb{P}(A_{P}^{\text{causal}})$\end{document}. Figure 6 provides a visual summary of the method.
Visual summary of the PICS algorithm described in Probabilistic Identification of Causal SNPs.
(A) Breakdown of the calculation of the probability of a focal SNP Ai\begin{document}$A_{i}$\end{document} being causal. (B) Illustration of the permutation procedure used to generate the null distribution. An example N×P\begin{document}$N\times P$\end{document} genotype array with N=P=6\begin{document}$N=P=6$\end{document} is used, with two valid row shuffles, or permutations, of the original array shown. Entries affected by the shuffle are highlighted, as is the focal SNP (A3\begin{document}$A_{3}$\end{document}).
By running the permutation procedure across all SNPs in the locus, one obtains P(Aicausal|A1=Alead)\begin{document}$\mathbb{P}(A_{i}^{\text{causal}}|A_{1}=A^{\text{lead}})$\end{document} for each i\begin{document}$i$\end{document}. These posterior probabilities are then normalized so that ∑i=1NP(Aicausal|A1=Alead)=1\begin{document}$\sum_{i=1}^{N}\mathbb{P}(A_{i}^{\text{causal}}|A_{1}=A^{\text{lead}})=1$\end{document}.
Finally, the above procedure is performed after restricting the set of all SNPs to only those with correlation magnitude |r|>0.5\begin{document}$|r| \gt 0.5$\end{document} to the lead SNP.
Algorithm
Based on the computation of posterior probabilities described above (Equation 1), the full PICS algorithm returns putatively causal variants as follows.
The last line of Algorithm 1 returns a list of posterior probability vectors corresponding to each potential set. For our work, we set |r|=0.5\begin{document}$|r|=0.5$\end{document}, C=3\begin{document}$C=3$\end{document}, R=500\begin{document}$R=500$\end{document} and p0=(1/P,...,1/P)\begin{document}$\mathbf{p}_0=(1/P,...,1/P)$\end{document} throughout implementations of Algorithm 1.
Algorithm 1. PICS.
1: Input: Individual-by-genotype array XN×P\begin{document}$ \underset{N\times P}{\mathbf{X}} $\end{document}, phenotype array yN×1\begin{document}$\underset{N\times 1}{\mathbf{y}}$\end{document}, LD threshold |r|\begin{document}$|r|$\end{document}, resampling number R\begin{document}$R$\end{document}, number of potential sets C\begin{document}$C$\end{document}, prior causal probabilities p0P×1\begin{document}$\underset{P\times 1}{\mathbf{p}_{0}}$\end{document}.2: c←1\begin{document}$c\leftarrow 1$\end{document}3: whilec⩽C\begin{document}$c\leqslant C$\end{document}do4: M←No. columns of X\begin{document}$M\leftarrow\text{No. columns of }\mathbf{X}$\end{document}5: Identify lead SNP, ℓ∈{1,…,M}\begin{document}$\ell\in\{1,\ldots,M\}$\end{document}6: Identify neighboring SNPs, Nℓc={k∈[M]:cor(Ak,Aℓ)2>r2}\begin{document}$\mathcal{N}^{c}_{\ell}=\{k\in[M]:\text{cor}(A_{k},A_{\ell})^{2} \gt r^{2}\}$\end{document}.7: forj∈Nℓc\begin{document}$j\in\mathcal{N}^{c}_{\ell}$\end{document}do8: Compute pj^=P(Ajcausal|Aℓ=Alead)\begin{document}$\widehat{p_{j}}=\mathbb{P}(A_{j}^{\text{causal}}|A_{\ell}=A^{\text{lead}})$\end{document} as described below Equation 1. Use R\begin{document}$R$\end{document} permutations.9: end for10: Record vector of posterior probabilities, pc→=[pj^:j∈Nℓc]\begin{document}$\overrightarrow{p^{c}}=[\widehat{p_{j}}:j\in\mathcal{N}^{c}_{\ell}]$\end{document}11: X←X[:,{1,…,M}∖Nℓc]\begin{document}$\mathbf{X}\leftarrow\mathbf{X}[:,\{1,\ldots,M\}\setminus\mathcal{N}_{\ell}^{c}]$\end{document}12: c←c+1\begin{document}$c\leftarrow c+1$\end{document}13: end while14: Output:[pc→:c⩽C]\begin{document}$[\overrightarrow{p^{c}}:c\leqslant C]$\end{document} (a list, where component c\begin{document}$c$\end{document} corresponds to Potential Set c\begin{document}$c$\end{document})
As mentioned in the Introduction, we apply two different approaches to implementing Algorithm 1. One is guided by stability and will be introduced shortly in Incorporating stability. The other, which we now describe, is based on regressing out confounders, i.e., residualization. In implementing the residualization approach, we regress the top five principal components, obtained from the genotype matrix X\begin{document}$\mathbf{X}$\end{document}, from the gene expression phenotype, y\begin{document}$\mathbf{y}$\end{document}. We choose five principal components based on the elbow method, an approach described in Brown et al., 2018. This yields residuals yr\begin{document}$\mathbf{y}^{r}$\end{document} (Figure 1A), which we use as the input phenotype array in Algorithm 1. We subsequently report the variant with the highest posterior probability in each potential set as the putatively causal variant for that potential set. This variant is referred to as the top variant.
We remark that there are other ways of using potential confounders in variable selection, such as inclusion as covariates in a linear model. Because our algorithm explicitly avoids assuming a linear model, we have chosen the residualization approach just described.
Incorporating stability
Our stability-guided approach to implementing PICS follows the steps outlined below. Let (X,y)\begin{document}$(\mathbf{X},\mathbf{y})$\end{document} be the genotype array and gene expression array. Assume that there are K\begin{document}$K$\end{document} subpopulations making up the dataset, and let Ek\begin{document}$E_{k}$\end{document} denote the set of row indices of X\begin{document}$\mathbf{X}$\end{document} corresponding to individuals from subpopulation k\begin{document}$k$\end{document}. (In our present work, K=5\begin{document}$K=5$\end{document}. The five subpopulations are Utahns (NCEU=89\begin{document}$N_{\text{CEU}}=89$\end{document}), Finns (NFIN=92\begin{document}$N_{\text{FIN}}=92$\end{document}), British (NGBR=86\begin{document}$N_{\text{GBR}}=86$\end{document}), Toscani (NTSI=91\begin{document}$N_{\text{TSI}}=91$\end{document}), and Yoruban (NYRI=87\begin{document}$N_{\text{YRI}}=87$\end{document}).)
Run Algorithm 1 on the pair (X,y)\begin{document}$(\mathbf{X},\mathbf{y})$\end{document}. Obtain a list of posterior probability vectors, L=[pc→:c⩽C]\begin{document}$\mathcal{L}=[\overrightarrow{p^{c}}:c\leqslant C]$\end{document}. Recall that C\begin{document}$C$\end{document} is the number of potential sets.
For each subpopulation k=1,…,K\begin{document}$k=1,\ldots,K$\end{document}, run Algorithm 1 on the pair (X|Ek,y|Ek)\begin{document}$(\mathbf{X}|_{E_{k}},\mathbf{y}|_{E_{k}})$\end{document}. Obtain L(k)=[pkc→:c⩽C]\begin{document}$\mathcal{L}^{(k)}=[\overrightarrow{p^{c}_{k}}:c\leqslant C]$\end{document} for each k\begin{document}$k$\end{document}.
(Stability-guided choice of putatively causal variants) Collect the probability vectors in lists L\begin{document}$\mathcal{L}$\end{document} and L(k)\begin{document}$\mathcal{L}^{(k)}$\end{document} (k∈[K]\begin{document}$k\in[K]$\end{document}). Operationalizing the principle that a stable variant has positive probability across multiple slices, we pick causal variants as follows. For potential set c\begin{document}$c$\end{document}, pick the variants that have (1) positive probability in pc→\begin{document}$\overrightarrow{p^{c}}$\end{document} in Step 1, and moreover (2) have positive probability in the most number of probability vectors; call this set of variants Sc\begin{document}$\mathcal{S}_{c}$\end{document} (note Sc\begin{document}$\mathcal{S}_{c}$\end{document} is a subset of the support of pc→\begin{document}$\overrightarrow{p^{c}}$\end{document} for Potential Set c\begin{document}$c$\end{document}). Among members of Sc\begin{document}$\mathcal{S}_{c}$\end{document}, select the variant that had the highest posterior probability in pc→\begin{document}$\overrightarrow{p^{c}}$\end{document}.
In other words, the stability-guided approach reports the variant that not only appears with positive probability in the most number of subsets including the pooled sample, but also has the largest probability in the pooled sample. This variant is referred to as the stable variant.
To illustrate the last step, suppose there are K=2\begin{document}$K=2$\end{document} subpopulations, C=1\begin{document}$C=1$\end{document} potential set, and P=5\begin{document}$P=5$\end{document} SNPs. Let the outputs from Steps 1 and 2 beL:p1→=(0.45,0.43,0.02,0.10,0.00)L(1):p11→=(0.00,0.40,0.10,0.25,0.25)L(2):p21→=(0.20,0.60,0.00,0.10,0.10)\begin{document}$$\displaystyle \begin{array}{ll}\mathcal{L}: & \overrightarrow{p^1}=(0.45,0.43,0.02,0.10,0.00)\\ \mathcal{L}^{(1)}: & \overrightarrow{p^1_1}=(0.00,0.40,0.10,0.25,0.25)\\ \mathcal{L}^{(2)}: & \overrightarrow{p^1_2}=(0.20,0.60,0.00,0.10,0.10)\end{array}$$\end{document}
In this example, the second and fourth variants have positive probability in the most number of probability vectors (S1={A2,A4}\begin{document}$\mathcal{S}_{1}=\{A_{2},A_{4}\}$\end{document}). Among A2\begin{document}$A_{2}$\end{document} and A4\begin{document}$A_{4}$\end{document}, A2\begin{document}$A_{2}$\end{document} has a higher posterior probability (i.e., 0.43) in p1→\begin{document}$\overrightarrow{p^{1}}$\end{document}, so the stable variant reported is A2\begin{document}$A_{2}$\end{document}. As a side remark, the posterior probability vector L\begin{document}$\mathcal{L}$\end{document} does not generally agree with the posterior probability vector computed using the residualization approach, because the latter is computed by running Algorithm 1 on the pair (X,yr)\begin{document}$(\mathbf{X},\mathbf{y}^{r})$\end{document} rather than (X,y)\begin{document}$(\mathbf{X},\mathbf{y})$\end{document}, as described earlier.
Stability-guided SuSiE
We apply the stability principle to SuSiE (Wang et al., 2020) in a similar manner as described above. Concretely, we run SuSiE on the pair (X,y)\begin{document}$(\mathbf{X},\mathbf{y})$\end{document}, before running it again on each subpopulation k=1,…,K\begin{document}$k=1,\ldots,K$\end{document}. Collecting the posterior inclusion probability (PIP) vectors across each subpopulation and the entire sample, we then keep only variants that have posterior probability at least 1/(no. variants included)\begin{document}$1/(\text{no. variants included})$\end{document}, owing to SuSiE tending to return nonzero probabilities to all variants included in the fine-mapping. These ‘good’ variants (i.e., variants with PIP at least as large as the prior inclusion probability) are analogous to the ‘variants with positive probability’ in PICS. The remaining steps are identical to stability-guided PICS.
Simulation study and evaluation detailsMain simulation study
We simulate 2400 synthetic gene expression data, which vary by two parameters: C\begin{document}$C$\end{document}, the number of causal variants, and ϕ\begin{document}$\phi$\end{document}, the proportion of variance in y\begin{document}$\mathbf{y}$\end{document} explained by the genotypes X\begin{document}$\mathbf{X}$\end{document}. Phenotypes are simulated as follows.
For a gene, whenever its canonical transcription start site (TSS) is available, restrict the genotype matrix X\begin{document}$\mathbf{X}$\end{document} to only variants lying within 1 Mb upstream and downstream. Else, use variants spanning the entire autosome associated with that gene. This ensures that cis eQTLs are used in simulation, for the most part.
Sample the indices of the C\begin{document}$C$\end{document} causal variants, C\begin{document}$\mathscr{C}$\end{document}, uniformly at random from {1,…,p}\begin{document}$\{1,\ldots,p\}$\end{document}, where p\begin{document}$p$\end{document} is the number of variants included in X\begin{document}$\mathbf{X}$\end{document} from Step 1.
For each j∈C\begin{document}$j\in\mathscr{C}$\end{document}, independently draw bj∼N(0,0.62)\begin{document}$b_{j}\sim N(0,0.6^{2})$\end{document} and, for all j∉C\begin{document}$j\not\in\mathscr{C}$\end{document}, set bj=0\begin{document}$b_{j}=0$\end{document}.
Set σ2\begin{document}$\sigma^{2}$\end{document} to achieve the desired proportion of variance explained ϕ\begin{document}$\phi$\end{document}.
Generate the phenotype by drawing y∼N(Xb,σ2In×n)\begin{document}$\mathbf{y}\sim N(\mathbf{X}\mathbf{b},\sigma^{2}\mathbf{I}_{n\times n})$\end{document}.
After gene expression data were simulated, we ran three versions of PICS: the stability-guided version, the residualization version, and an uncorrected version, where no correction for population stratification or stability guidance was applied. To investigate the broader applicability of the stability approach, we also ran two versions of SuSiE (Wang et al., 2020), a popular fine-mapping method relying on variational approximation techniques. The first version runs SuSiE on residualized phenotypes, while the second version incorporates stability in selecting the most likely causal variant(s). In both approaches, we rely on default parameters of SuSiE while setting the number of credible sets to return to 3 (i.e., L=3 in susieR::susie).
Smaller simulation study
To investigate whether non-genetic confounding between ancestries in a multi-ancestry cohort can hinder the performance of fine-mapping algorithms that do not correct for potential confounding by ancestry, we simulate a smaller set of synthetic gene expression data. We select 10 genes from Chromosomes 1, 20, 21, and 22 and repeat the simulation steps as described above, up to Step 4. At Step 4, instead of choosing a single σ2\begin{document}$\sigma^{2}$\end{document}, we allow σ2\begin{document}$\sigma^{2}$\end{document} to depend on the ancestry of the individual in order to capture environmental heterogeneity. We specifically consider six scenarios that we classify as (environmental) variance heterogeneity and mean heterogeneity.
Variance heterogeneity. Let α(t)=(αTSI(t),αGBR(t),αFIN(t),αCEU(t),αYRI(t))\begin{document}$\boldsymbol{\alpha}(t)=(\alpha_{\text{TSI}}(t),\alpha_{\text{GBR}}(t),\alpha_{ \text{FIN}}(t),\alpha_{\text{CEU}}(t),\alpha_{\text{YRI}}(t))$\end{document} be a vector of exponentiated proportions, where we define the exponentiated proportion of a subpopulation POP (POP∈{TSI,FIN,GBR,CEU,YRI}\begin{document}$\text{POP}\in\{\text{TSI},\text{FIN},\text{GBR},\text{CEU},\text{YRI}\}$\end{document}) as αPOP(t)=NPOPt/(NTSIt+NGBRt+NFINt+NCEUt+NYRIt)\begin{document}$\alpha_{\text{POP}}(t)=N_{\text{POP}}^{t}/(N_{\text{TSI}}^{t}+N_{\text{GBR}}^{ t}+N_{\text{FIN}}^{t}+N_{\text{CEU}}^{t}+N_{\text{YRI}}^{t})$\end{document}. In Step 4, we define a vector of ancestry-specific exogenous variances, σ2(t)=5σ2α(t)\begin{document}$\boldsymbol{\sigma^{2}}(t)=5\sigma^{2}\boldsymbol{\alpha}(t)$\end{document}, where σ2\begin{document}$\sigma^{2}$\end{document} is calculated as in Step 4 of the original simulation setup. We then generate phenotypes independently in Step 5 by conditioning on the individual’s population membership (POP): yi∼N(xib,σ2(t)(k))\begin{document}$y_{i}\sim N(\boldsymbol{x}_{i}\mathbf{b},\boldsymbol{\sigma^{2}}(t)(k))$\end{document}, where k\begin{document}$k$\end{document} is the index that corresponds to the population membership of individual i\begin{document}$i$\end{document} and xi\begin{document}$\boldsymbol{x}_{i}$\end{document} denotes the genotypes of individual i\begin{document}$i$\end{document}. For example, if t=0\begin{document}$t=0$\end{document}, α(t)=α(0)=(1/5,1/5,1/5,1/5,1/5)\begin{document}$\boldsymbol{\alpha}(t)=\boldsymbol{\alpha}(0)=(1/5,1/5,1/5,1/5,1/5)$\end{document} and σ2(t)=(σ2,σ2,σ2,σ2,σ2)\begin{document}$\boldsymbol{\sigma^{2}}(t)=(\sigma^{2},\sigma^{2},\sigma^{2},\sigma^{2},\sigma ^{2})$\end{document}, which reduces to our original simulation setup. The choice of t>0\begin{document}$t \gt 0$\end{document} determines the heterogeneity of the exogenous variance among the subpopulations, with a larger t\begin{document}$t$\end{document} producing higher heterogeneity. Here, we consider four choices of t\begin{document}$t$\end{document}: 8, 16, 128, and 256.
Mean heterogeneity. Step 5 of the original simulation setup is modified to include an ancestry-dependent mean vector. We arbitrarily let GBR be the ‘focal’ population and generate phenotypes in two ways.
‘Smooth mean’: After σ2\begin{document}$\sigma^{2}$\end{document} is calculated in Step 4 of the original simulation setup, define the mean exogenous noise vector as μ=(μTSI,μGBR,μFIN,μCEU,μYRI)=(2σ,0,σ,σ,2σ)\begin{document}$\boldsymbol{\mu}=(\mu_{\text{TSI}},\mu_{\text{GBR}},\mu_{\text{FIN}},\mu_{ \text{CEU}},\mu_{\text{YRI}})=(2\sigma,0,\sigma,\sigma,2\sigma)$\end{document}. An individual i\begin{document}$i$\end{document} has their phenotype simulated as yi∼N(xib+μ(k),σ2)\begin{document}$y_{i}\sim N(\boldsymbol{x}_{i}\mathbf{b}+\boldsymbol{\mu}(k),\sigma^{2})$\end{document}, where again k\begin{document}$k$\end{document} is the index that corresponds to the population membership of individual i\begin{document}$i$\end{document}. This effectively increases the gene expression of all but GBR individuals in a positive direction with varying amounts of shift.
‘Spiked mean’: After σ2\begin{document}$\sigma^{2}$\end{document} is calculated in Step 4 of the original simulation setup, define μ=(0,2σ,0,0,0)\begin{document}$\boldsymbol{\mu}=(0,2\sigma,0,0,0)$\end{document} and simulate individual i\begin{document}$i$\end{document}’s phenotype as yi∼N(xib+μ(k),σ2)\begin{document}$y_{i}\sim N(\boldsymbol{x}_{i}\mathbf{b}+\boldsymbol{\mu}(k),\sigma^{2})$\end{document}. This effectively increases the gene expression of GBR individuals only in a positive direction, while the other subpopulation individuals share the same zero mean (as in the original simulation setup).
For brevity, we refer to these six scenarios by t=8, t=16, t=128, t=256, |i-3| and i=3.
Evaluation
Several metrics have been used to evaluate Bayesian fine-mapping algorithms (see e.g., Benner et al., 2016; Cui et al., 2024; Hormozdiari et al., 2015; Wang et al., 2020; Wen et al., 2016; Yang et al., 2023), many of which compute credible sets and analyze their coverage. A challenge in using credible sets for our study is that the posterior probabilities returned by the PICS algorithm are not posterior inclusion probabilities in a strict sense: they do not measure the frequency with which a variant should be included in a linear model (Griffin and Steel, 2021; note PICS does not assume a model relating features to outcome). This complicates defining credible sets and coverage, so we follow Mazumder, 2020 instead and evaluate performance of all algorithms by computing the signal recovery probability as a function of the signal to background noise (SNR). By the definition of ϕ\begin{document}$\phi$\end{document} (see Wang et al., 2020 for details), we may compute SNR directly from ϕ\begin{document}$\phi$\end{document} as SNR=ϕ/(1−ϕ)\begin{document}$\text{SNR}=\phi/(1-\phi)$\end{document}.
Statistical comparison methodology
We rely on 378 external functional annotations to interpret biological significance of our variants, summarized in Table 1. Appendix 5 provides interpretation for the functional annotations, while Appendix 6 describes in greater detail how we generate annotations from Enformer predictions.
Our comparison of the top and stable variants is twofold. First, we evaluate the relative significance of the stable variant against the top variant by running paired Wilcoxon two-sample tests across all pairs of top and stable variants across all GEUVADIS gene expression phenotypes. We compute one-directional p-values in either direction to check for significant depletion or enrichment of the stable variant with respect to a particular annotation. Raw p-values are adjusted for false discovery rate control by applying the standard BH procedure (R command p.adjust(…,method=’BH’)) to p-values across all 378 annotations and all potential sets.
To investigate whether various external factors might moderate the differences in functional annotations, we next perform trend tests. Basically, for some external factor F\begin{document}$F$\end{document}, we run a trend test to see if values of F\begin{document}$F$\end{document} are associated with attenuation or augmentation of differences in functional annotation of the top and stable variants. The list of all external factors F\begin{document}$F$\end{document} is provided in Table 2.
For (1) Degree of Stability, we perform a Jonckheere–Terpstra test (R command clinfun::jonckheere.test(…,nperm=5000)). For (2) Population Diversity, (3) Population Differentiation, and (6) Degree of Certainty of Causality Using Residualization Approach, we perform a correlation test (R command cor.test(…)). For (4) Inclusion of Distal Subpopulations (Top) and (5) Inclusion of Distal Subpopulations (Stable), we perform an unpaired Wilcoxon test (R command wilcox.test(…)). Finally, for each moderator, we again compute one-directional p-values in either direction, before applying the BH procedure to all p-values across annotations and potential sets for that moderator only.
Additional informationCompeting interests
No competing interests declared
Lionel Chentian Jin is affiliated with McKinsey & Company. The author has no other competing interests to declare
We provide the following data and scripts on the GitHub repository https://github.com/songlab-cal/StableFM (copy archived at Aw, 2026): fine-mapped variants with their functional annotations and moderator variable quantities, code for reproducing Main Text figures in this manuscript and building our visualization app.
The following previously published dataset was used:
DermitzakisEKurbatovaNLappalainenT2013RNA-sequencing of 465 lymphoblastoid cell lines from the 1000 Genomes Project (GEUVADIS)ArrayExpressE-GEUV-1
Acknowledgements
This research is supported in part by an NIH grant R35-GM134922 and grant number CZF2019-
002449 from the Chan Zuckerberg Initiative Foundation. We thank Carlos Albors, Gonzalo Benegas,
Ryan Chung, Ziyue Gao, Iain Mathieson, and Yutong Wang for helpful discussions; members of the
Yu Group at Berkeley for feedback on the work; and Aniketh Reddy for help with data processing.
ReferencesAbdellaouiADolanCVVerweijKJHNivardMG2022Gene-environment correlations across geographic regions affect genome-wide association studiesNature Genetics541345135410.1038/s41588-022-01158-035995948AdzhubeiIASchmidtSPeshkinLRamenskyVEGerasimovaABorkPKondrashovASSunyaevSR2010A method and server for predicting damaging missense mutationsNature Methods724824910.1038/nmeth0410-24820354512AutonABrooksLDDurbinRMGarrisonEPKangHMKorbelJOMarchiniJLMcCarthySMcVeanGAAbecasisGR1000 Genomes Project Consortium2015A global reference for human genetic variationNature526687410.1038/nature1539326432245AvsecŽAgarwalVVisentinDLedsamJRGrabska-BarwinskaATaylorKRAssaelYJumperJKohliPKelleyDR2021Effective gene expression prediction from sequence by integrating long-range interactionsNature Methods181196120310.1038/s41592-021-01252-x34608324AwAJ2026StableFMswh:1:rev:49b7f17640d96e3148ad54981c847272a7898beaSoftware Heritagehttps://archive.softwareheritage.org/swh:1:dir:8fd96eb5631490d35e00714f9e33185fc2b5c5c5;origin=https://github.com/songlab-cal/StableFM;visit=swh:1:snp:5df7c652f2f78013716ef00b2e017736b79cc89d;anchor=swh:1:rev:49b7f17640d96e3148ad54981c847272a7898beaBalasubramanianSFuYPawasheMMcGillivrayPJinMLiuJKarczewskiKJMacArthurDGGersteinM2017Using ALoFT to determine the impact of putative loss-of-function variants in protein-coding genesNature Communications838210.1038/s41467-017-00443-528851873Barbadilla-MartínezLKlaassenNvan SteenselBde RidderJ2025Predicting gene expression from DNA sequence using deep learning modelsNature Reviews. Genetics2666668010.1038/s41576-025-00841-240360798BasuSKumbierKBrownJBYuB2018Iterative random forests to discover predictive and stable high-order interactionsPNAS1151943194810.1073/pnas.171123611529351989BenegasGYeCAlborsCLiJCSongYS2025Genomic language models: opportunities and challengesTrends in Genetics4128630210.1016/j.tig.2024.11.01339753409BennerCSpencerCCAHavulinnaASSalomaaVRipattiSPirinenM2016FINEMAP: efficient variable selection using summary data from genome-wide association studiesBioinformatics321493150110.1093/bioinformatics/btw01826773131BousquetOElisseeffA2002Stability and generalizationThe Journal of Machine Learning Research2499526BrownBCBrayNLPachterL2018Expression reflects population structurePLOS Genetics14e100784110.1371/journal.pgen.100784130566439CavazosTBWitteJS2021Inclusion of variants discovered from diverse populations improves polygenic risk score transferabilityHGG Advances210001710.1016/j.xhgg.2020.10001733564748CuiRElzurRAKanaiMUlirschJCWeissbrodODalyMJNealeBMFanZFinucaneHK2024Improving fine-mapping by modeling infinitesimal effectsNature Genetics5616216910.1038/s41588-023-01597-338036779CunninghamFAllenJEAllenJAlvarez-JarretaJAmodeMRArmeanIMAustine-OrimoloyeOAzovAGBarnesIBennettRBerryABhaiJBignellABillisKBodduSBrooksLCharkhchiMCumminsCDa Rin FiorettoLDavidsonCDodiyaKDonaldsonSEl HoudaiguiBEl NaboulsiTFatimaRGironCGGenezTMartinezJGGuijarro-ClarkeCGymerAHardyMHollisZHourlierTHuntTJuettemannTKaikalaVKayMLavidasILeTLemosDMarugánJCMohananSMushtaqANavenMOgehDNParkerAPartonAPerryMPiližotaIProsovetskaiaISakthivelMPSalamAIASchmittBMSchuilenburgHSheppardDPérez-SilvaJGStarkWSteedESutinenKSukumaranRSumathipalaDSunerM-MSzpakMThormannATricomiFFUrbina-GómezDVeidenbergAWalshTAWaltsBWillhoftNWinterbottomAWassEChakiachviliMFlintBFrankishAGiorgettiSHaggertyLHuntSEIIsleyGRLovelandJEMartinFJMooreBMudgeJMMuffatoMPerryERuffierMTateJThybertDTrevanionSJDyerSHarrisonPWHoweKLYatesADZerbinoDRFlicekP2022Ensembl 2022Nucleic Acids Research50D988D99510.1093/nar/gkab104934791404DavenportEEAmariutaTGutierrez-ArcelusMSlowikowskiKWestraH-JLuoYShenCRaoDAZhangYPearsonSvon SchackDBeebeJSBingNJohnSVincentMSZhangBRaychaudhuriS2018Discovering in vivo cytokine-eQTL interactions from a lupus clinical trialGenome Biology1916810.1186/s13059-018-1560-830340504DavydovEVGoodeDLSirotaMCooperGMSidowABatzoglouS2010Identifying a high fraction of the human genome to be under selective constraint using GERP++PLOS Computational Biology6e100102510.1371/journal.pcbi.100102521152010EfronBTibshiraniRJ1994An Introduction to the BootstrapCRC pressFarhKK-HMarsonAZhuJKleinewietfeldMHousleyWJBeikSShoreshNWhittonHRyanRJHShishkinAAHatanMCarrasco-AlfonsoMJMayerDLuckeyCJPatsopoulosNADe JagerPLKuchrooVKEpsteinCBDalyMJHaflerDABernsteinBE2015Genetic and epigenetic fine mapping of causal autoimmune disease variantsNature51833734310.1038/nature1383525363779FavéM-JLamazeFCSoaveDHodgkinsonAGauvinHBruatVGrenierJ-CGbehaESkeadKSmargiassiAJohnsonMIdaghdourYAwadallaP2018Gene-by-environment interactions in urban populations modulate risk phenotypesNature Communications982710.1038/s41467-018-03202-229511166FuYLiuZLouSBedfordJMuXJYipKYKhuranaEGersteinM2014FunSeq2: a framework for prioritizing noncoding regulatory variants in cancerGenome Biology1548010.1186/s13059-014-0480-525273974GaoBZhouX2024MESuSiE enables scalable and powerful multi-ancestry fine-mapping of causal variants in genome-wide association studiesNature Genetics5617017910.1038/s41588-023-01604-738168930GongJMeiSLiuCXiangYYeYZhangZFengJLiuRDiaoLGuoA-YMiaoXHanL2018PancanQTL: systematic identification of cis-eQTLs and trans-eQTLs in 33 cancer typesNucleic Acids Research46D971D97610.1093/nar/gkx86129036324GriffinJESteelMF2021Adaptive Computational Methods for Bayesian Variable Selection, Handbook of Bayesian Variable SelectionChapman and Hall/CRC10.1201/9781003089018-5HanBEskinE2012Interpreting meta-analyses of genome-wide association studiesPLOS Genetics8e100255510.1371/journal.pgen.100255522396665HormozdiariFKichaevGYangWYPasaniucBEskinE2015Identification of causal genes for complex traitsBioinformatics31i206i21310.1093/bioinformatics/btv24026072484HuSFerreiraLAFShiSHellenthalGMarchiniJLawsonDJMyersSR2025Fine-scale population structure and widespread conservation of genetic effect sizes between human groups across traitsNature Genetics5737938910.1038/s41588-024-02035-839901012HuangYFGulkoBSiepelA2017Fast, scalable prediction of deleterious noncoding variants from functional and population genomic dataNature Genetics4961862410.1038/ng.381028288115HuangCShuaiRWBaokarPChungRRastogiRKathailPIoannidisNM2023Personal transcriptome variation is poorly explained by current genomic deep learning modelsNature Genetics552056205910.1038/s41588-023-01574-w38036790IoannidisNMDavisJRDeGorterMKLarsonNBMcDonnellSKFrenchAJBattleAJHastieTJThibodeauSNMontgomerySBBustamanteCDSiehWWhittemoreAS2017FIRE: functional inference of genetic variants that regulate gene expressionBioinformatics333895390110.1093/bioinformatics/btx53428961785KarollusAMauermeierTGagneurJ2023Current sequence-based models capture gene expression determinants in promoters but mostly ignore distal enhancersGenome Biology245610.1186/s13059-023-02899-936973806KatsonisPWilhelmKWilliamsALichtargeO2022Genome interpretation using in silico predictors of variant impactHuman Genetics1411549157710.1007/s00439-022-02457-635488922KeysKLMakACYWhiteMJEckalbarWLDahlAWMeffordJMikhaylovaAVContrerasMGElhawaryJREngCHuDHuntsmanSOhSSSalazarSLenoirMAYeJCThorntonTAZaitlenNBurchardEGGignouxCR2020On the cross-population generalizability of gene expression prediction modelsPLOS Genetics16e100892710.1371/journal.pgen.100892732797036KheradpourPKellisM2014Systematic discovery and characterization of regulatory motifs in ENCODE TF binding experimentsNucleic Acids Research422976298710.1093/nar/gkt124924335146LaPierreNTaraszkaKHuangHHeRHormozdiariFEskinE2021Identifying causal variants by fine mapping across multiple studiesPLOS Genetics17e100973310.1371/journal.pgen.100973334543273LappalainenTSammethMFriedländerMR’t HoenPACMonlongJRivasMAGonzàlez-PortaMKurbatovaNGriebelTFerreiraPGBarannMWielandTGregerLvan ItersonMAlmlöfJRibecaPPulyakhinaIEsserDGigerTTikhonovASultanMBertierGMacArthurDGLekMLizanoEBuermansHPJPadioleauISchwarzmayrTKarlbergOOngenHKilpinenHBeltranSGutMKahlemKAmstislavskiyVStegleOPirinenMMontgomerySBDonnellyPMcCarthyMIFlicekPStromTMGeuvadis ConsortiumLehrachHSchreiberSSudbrakRCarracedoAAntonarakisSEHäslerRSyvänenA-Cvan OmmenG-JBrazmaAMeitingerTRosenstielPGuigóRGutIGEstivillXDermitzakisET2013Transcriptome and genome sequencing uncovers functional variation in humansNature50150651110.1038/nature1253124037378LiSSesiaMRomanoYCandèsESabattiC2022Searching for robust associations with a multi-environment knockoff filterBiometrika10961162910.1093/biomet/asab05538633763LiangYPividoriMManichaikulAPalmerAACoxNJWheelerHEImHK2022Polygenic transcriptome risk scores (PTRS) can improve portability of polygenic risk scores across ancestriesGenome Biology232310.1186/s13059-021-02591-w35027082LimCYuB2016Estimation stability with cross-validation (ESCV)Journal of Computational and Graphical Statistics2546449210.1080/10618600.2015.1020159LiveseyBJBadonyiMDiasMFrazerJKumarSLindorff-LarsenKMcCandlishDMOrenbuchRShearerCAMuffleyLForemanJGlazerAMLehnerBMarksDSRothFPRubinAFStaritaLMMarshJA2025Guidelines for releasing a variant effect predictorGenome Biology269710.1186/s13059-025-03572-z40234898LuZGopalanSYuanDContiDVPasaniucBGusevAMancusoN2022Multi-ancestry fine-mapping improves precision to identify causal genes in transcriptome-wide association studiesAmerican Journal of Human Genetics1091388140410.1016/j.ajhg.2022.07.00235931050LuZWangXCarrMKimAGazalSMohammadiPWuLPirruccelloJKachuriLGusevAMancusoN2025Improved multiancestry fine-mapping identifies cis-regulatory variants underlying molecular traits and disease riskNature Genetics571881188910.1038/s41588-025-02262-740691406Márquez-LunaCLohPRPriceALConsortium SATDSConsortium STD2017Multiethnic polygenic risk scores improve risk prediction in diverse populationsGenetic Epidemiology4181182310.1002/gepi.2208329110330MazumderR2020Discussion of “best subset, forward stepwise or lasso? analysis and recommendations based on extensive comparisons”Statistical Science3560260810.1214/20-STS807McVickerGGordonDDavisCGreenP2009Widespread genomic signatures of natural selection in hominid evolutionPLOS Genetics5e100047110.1371/journal.pgen.100047119424416MorrisAP2011Transethnic meta-analysis of genomewide association studiesGenetic Epidemiology3580982210.1002/gepi.2063022125221MostafaviHHarpakAAgarwalIConleyDPritchardJKPrzeworskiM2020Variable prediction accuracy of polygenic scores within an ancestry groupeLife9e4837610.7554/eLife.4837631999256NgPCHenikoffS2003SIFT: Predicting amino acid changes that affect protein functionNucleic Acids Research313812381410.1093/nar/gkg50912824425PollardKSHubiszMJRosenbloomKRSiepelA2010Detection of nonneutral substitution rates on mammalian phylogeniesGenome Research2011012110.1101/gr.097857.10919858363RentzschPWittenDCooperGMShendureJKircherM2019CADD: predicting the deleteriousness of variants throughout the human genomeNucleic Acids Research47D886D89410.1093/nar/gky101630371827RogersMFShihabHAMortMCooperDNGauntTRCampbellC2018FATHMM-XF: accurate prediction of pathogenic point mutations via extended featuresBioinformatics3451151310.1093/bioinformatics/btx53628968714SasseANgBSpiroAETasakiSBennettDAGaiteriCDe JagerPLChikinaMMostafaviS2023Benchmarking of deep neural networks for predicting personal gene expression from DNA sequence highlights shortcomingsNature Genetics552060206410.1038/s41588-023-01524-638036778SchaidDJChenWLarsonNB2018From genome-wide associations to candidate causal variants by statistical fine-mappingNature Reviews. Genetics1949150410.1038/s41576-018-0016-z29844615ShiHGazalSKanaiMKochEMSchoechAPSiewertKMKimSSLuoYAmariutaTHuangHOkadaYRaychaudhuriSSunyaevSRPriceAL2021Population-specific causal disease effect sizes in functionally important regions impacted by selectionNature Communications12109810.1038/s41467-021-21286-1TaylorKEAnselKMMarsonACriswellLAFarhKKH2021PICS2: next-generation fine mapping via probabilistic identification of causal SNPsBioinformatics373004300710.1093/bioinformatics/btab12233624747TurleyPMartinARGoldmanGLiHKanaiMWaltersRKJalaJBLinKMillwoodIYCareyCEPalmerDSZacherMAtkinsonEGChenZLiLAkiyamaMOkadaYKamataniYWaltersRGCallierSLaibsonDMeyerMNCesariniDDalyMBenjaminDJNealeBM2021Multi-ancestry meta-analysis yields novel genetic discoveries and ancestry-specific associationsbioRxiv10.1101/2021.04.23.441003WangGSarkarACarbonettoPStephensM2020A simple new approach to variable selection in regression, with application to genetic fine mappingJournal of the Royal Statistical Society. Series B, Statistical Methodology821273130010.1111/rssb.1238837220626WangQSKelleyDRUlirschJKanaiMSadhukaSCuiRAlborsCChengNOkadaYAguetFArdlieKGMacArthurDGFinucaneHKBiobank Japan Project2021Leveraging supervised learning for functionally informed fine-mapping of cis-eQTLs identifies an additional 20,913 putative causal eQTLsNature Communications12339410.1038/s41467-021-23134-834099641WenXLucaFPique-RegiR2015Cross-population joint analysis of eQTLs: fine mapping and functional annotationPLOS Genetics11e100517610.1371/journal.pgen.100517625906321WenXLeeYLucaFPique-RegiR2016Efficient integrative multi-snp association analysis via deterministic approximation of posteriorsAmerican Journal of Human Genetics981114112910.1016/j.ajhg.2016.03.02927236919WillerCJLiYAbecasisGR2010METAL: fast and efficient meta-analysis of genomewide association scansBioinformatics262190219110.1093/bioinformatics/btq34020616382YangZWangCLiuLKhanALeeAVardarajanBMayeuxRKirylukKIonita-LazaI2023CARMA is a new Bayesian model for fine-mapping in genome-wide association meta-analysesNature Genetics551057106510.1038/s41588-023-01392-037169873YuBKumbierK2020Veridical data sciencePNAS1173920392910.1073/pnas.1901326117YuanKLongchampsRJPardiñasAFYuMChenT-TLinS-CChenYLamMLiuRXiaYGuoZShiWShenCSchizophrenia Workgroup of Psychiatric Genomics ConsortiumDalyMJNealeBMFengY-CALinY-FChenC-YO’DonovanMCGeTHuangH2024Fine-mapping across diverse ancestries drives the discovery of putative causal variants underlying human complex traits and diseasesNature Genetics561841185010.1038/s41588-024-01870-z39187616ZaidiAAMathiesonI2020Demographic history mediates the effect of stratification on polygenic scoreseLife9e6154810.7554/eLife.6154833200985ZhouHArapoglouTLiXLiZZhengXMooreJAsokAKumarSBlueEEBuyskeSCoxNFelsenfeldAGersteinMKennyELiBMatiseTPhilippakisARehmHLSofiaHJSnyderGNHGRI Genome Sequencing Program Variant Functional Annotation Working GroupWengZNealeBSunyaevSRLinX2023FAVOR: functional annotation of variants online resource and annotator for variation across the human genomeNucleic Acids Research51D1300D131110.1093/nar/gkac96636350676Appendix 1Permuting while preserving marginal correlation and LD
For a focal SNP i\begin{document}$i$\end{document}, its N\begin{document}$N$\end{document} entries (x1i,…,xNi)\begin{document}$(x_{1i},\ldots,x_{Ni})$\end{document} take on only D\begin{document}$D$\end{document} values, where D\begin{document}$D$\end{document} is the ploidy of the data (for human genotypes D=2\begin{document}$D=2$\end{document} and for human haplotypes D=1\begin{document}$D=1$\end{document}). Suppose D=2\begin{document}$D=2$\end{document} for exposition. For d=0,1,2\begin{document}$d=0,1,2$\end{document}, let Id⊂[N]\begin{document}$I_{d}\subset[N]$\end{document} denote the indices of those elements of the allelic dosage vector (x1i,…,xNi)\begin{document}$(x_{1i},\ldots,x_{Ni})$\end{document} taking on value d\begin{document}$d$\end{document}. Then, any permutation σ∈SN\begin{document}$\sigma\in\mathfrak{S}_{N}$\end{document} of the rows of X\begin{document}$\mathbf{X}$\end{document} that preserves marginal correlation with the phenotype and LD maps indices from Id\begin{document}$I_{d}$\end{document} to Id\begin{document}$I_{d}$\end{document}, because
Each distinct entry in the allelic dosage vector of the focal SNP is still assigned the same phenotype entry;
Row shuffles do not change column-column covariances, the latter of which determines LD.
To be precise, σ∈S|I0|×S|I1|×S|I2|\begin{document}$\sigma\in\mathfrak{S}_{|I_{0}|}\times\mathfrak{S}_{|I_{1}|}\times\mathfrak{S}_ {|I_{2}|}$\end{document}, a direct product of the three permutation groups corresponding to the different allelic dosages.
In practice, we approximate the permutation distribution by sampling permutations uniformly at random from the group defined above. We set the sampling number to 500 for all our fine-mapping experiments.
Appendix 2Impact of including more potential sets on matching frequency and causal variant recovery
The number of credible or potential sets is a parameter in many fine-mapping algorithms. Focusing on stability-guided approaches, we consider how including more potential sets for stable fine-mapping algorithms affects both causal variant recovery and matching frequency in simulations. For the latter, we will specifically investigate whether including more potential sets in PICS and searching for matching variants across different potential sets for Top and Stable PICS will improve causal variant recovery.
Causal variant recovery
We investigate both Stable PICS and Stable SuSiE. Focusing first on simulations with one causal variant, we observe a modest gain in causal variant recovery for both Stable PICS and Stable SuSiE, most noticeably when the number of sets was increased from 1 to 2 under the lowest signal-to-noise ratio setting (SNR = 0.053, see Figure 2—figure supplement 5). For example, for Stable PICS, we observed 4 genes—ENSG00000203760.4 in Chr6, ENSG00000132912.8 in Chr5, and ENSG00000168614.12 and ENSG00000171502.10 in Chr1—where the most stable variant in Potential Set 2 was the causal variant, which we would not have recovered had we only told Stable PICS to return one potential set.
When we consider simulations with two and three causal variants, we observe modest gains when increasing from 1 to 2 sets (Figure 2—figure supplement 6) and from 2 to 3 sets (Figure 2—figure supplement 7). Bigger gains were observed for Stable SuSiE than for Stable PICS, especially from 1 to 2 sets, where Stable SuSiE saw larger increases in probability of recovering all causal variants across all SNR settings.
Matching frequency
In PICS, we assume that each potential set returns just one causal variant, because PICS explicitly groups variants with very high LD within the same potential set. Here, we ask whether non-matching variants between the same potential sets returned by Top and Stable PICS are owing to the lack of multiple causal variant modeling; in other words, whether non-matching variants would match across different Top PICs and Stable PICS potential sets.
Computing fractions of non-matching variants for which the stable variant matched the top variant of another potential set, we observed generally small matching fractions (see Appendix 12—table 4). The mean matching fraction between Potential Set 1 stable variant and a top variant in Potential Set 2 or 3 across all simulations was 11%, with larger matching fractions occurring between Potential Set 1 stable and Potential Set 2 top variants. Mean matching fractions for Potential Sets 2 and 3 stable variant and a different potential set top variant were 23% and 14%, with larger matching fractions occurring between Potential Set 2 stable and Potential Set 3 top variants and Potential Set 3 stable and Potential Set 2 top variants, respectively.
We next checked if these ‘off-diagonal’ matching variants are enriched in causal variants. Across all simulations, an off-diagonal matching Potential Set 1 stable variant recovered a causal variant 54% of the time, while for Potential Set 2 stable variant and Potential Set 3 stable variant the recovery percentages were 16% and 8%. These fractions are larger than if the stable variant did not match any top variant (24%, 5%, and 3% for Potential Sets 1, 2, and 3, respectively). However, these fractions are generally smaller when compared against causal variant recovery percentages for matching variants: we observed smaller fractions for Potential Set 1 (54% < 77%) and Potential Set 2 (16% < 20%), and only for Potential Set 3 did we observe a slightly larger fraction (8% > 6%).
Taken together, these findings demonstrate that it is helpful to condition on matching variants between ‘off-diagonal’ potential sets to recover causal variants in case the top and stable variants in the same potential set do not match, but such events occur infrequently and do not enrich for causal variants as much as do matching variants from the same potential sets. Therefore, in our real data analysis, we proceed with comparing matching variants from the same potential sets.
Appendix 3Difference in posterior probability of non-matching variants
We investigate whether there are differences in posterior probabilities between non-matching variants in simulations. Focusing first on Potential Set 1 non-matching variants, we observe that posterior probabilities tend to be clustered around two regions: (1) similar posterior probabilities that are less than 0.5; and (2) low stable variant posterior probabilities (Figure 2—figure supplement 9). This clustering pattern differs from matching variants, which tend to have more similar and larger posterior probabilities (Figure 2—figure supplement 8). Moving down to Potential Sets 2 and 3, we observe a similar clustering around region (2) for non-matching variants, with the remaining points distributed more randomly within the rectangle [0,1] × [0,1] (Figure 2—figure supplement 9). Matching variants for these two potential sets remain more similar (lying close to y=x\begin{document}$y=x$\end{document} diagonal of the rectangle, see Figure 2—figure supplement 8), suggesting that a salient feature of non-matching variants is a small stable variant posterior probability (despite possibly large top variant posterior probability).
Appendix 4Comparing SuSiE to stable PICS
We investigate how PICS performs relative to SuSiE, a widely used fine-mapping algorithm, in simulations. Our comparison is between Stable PICS and Top SuSiE, the latter of which performs residualization on gene expression phenotypes to remove potential confounding by ancestry. Starting with simulations involving one causal variant, we observe slightly better causal variant recovery frequency for SuSiE at larger SNR, although the opposite is true at smaller SNR (Figure 2—figure supplement 1). Notably, PICS always recovers causal variants with probability at least 0.5. Moving onto simulations with two causal variants (Figure 2—figure supplement 2) and three causal variants (Figure 2—figure supplement 3), we observe similar patterns, where PICS recovers more causal variants at small SNR and SuSiE recovers more causal variants at large SNR. (Comparisons between Stable PICS and Stable SuSiE are summarized in Figure 2—figure supplements 5–7.)
Appendix 5List of annotations
We measure the biological significance of a variant using a wide range of available functional annotations (see Appendix 12—table 6). The annotations cover potential biological activity in the local vicinity of the variant position along the genome, estimated model-based biological quantities (e.g., selection and conservation scores), and predicted effects of mutagenesis from the reference to the alternate allele at the variant.
Appendix 6Generating annotations from enformer predictions
The Enformer (Avsec et al., 2021) is a sequence-based prediction model, which leverages the attention mechanism in a transformer neural network to capture long-range effects on gene regulation.
For Enformer predictions, we subset the 5313 ChIP-seq, DNase-seq, ATAC-seq, and CAGE tracks to only those relevant to the lymphoblastoid cell line (GM12878), the cell line with respect to which GEUVADIS gene expression phenotypes are measured. We also restrict to genes whose top and stable variants are within the 196,608 bp input length limit of the Enformer from the corresponding GEUVADIS gene’s TSS. This restriction allows us to obtain Enformer predictions on three sequences: a null sequence, a sequence where the REF allele is replaced with the ALT allele at the top variant (top sequence), and a sequence where the REF allele is replaced with the ALT allele at the stable variant (stable sequence).
To compare the top and the stable variants with respect to a particular track, we take the predictions obtained from the three input sequences (null prediction, top prediction, stable prediction—a triplet), and compute the magnitude of change in predictions between both the top and null and the stable and null. We use the magnitude of change here, rather than the change itself, to capture the impact the mutation associated with the variant has on the track. Our Enformer functional annotations can thus be interpreted as measures of mutagenic impact on a track’s prediction, without considering the directionality of the impact. We refer to these annotations as perturbation scores.
We compute perturbation scores in two ways. The first way is to center all input sequences on the transcription start site of the gene (occasionally referred to as ‘TSS’ in our work), thus allowing the model to measure the impact of mutagenesis at the (top or stable) variant on predicted gene expression profile for the sequence. The second way is to average the perturbation scores over three triplets, one centered on the gene TSS, and two centered on the flanking positions of the gene TSS (i.e., the two neighboring bins). The second way (occasionally referred to as ‘AVE’ in our work) accounts for errors arising from imprecise TSS positioning and the instability of the model to small changes in input—a technique used in Karollus et al., 2023 and by the Enformer team (Avsec et al., 2021).
Appendix 7Matching vs non-matching variant results
As described in the main text, we compare between one set of genes where the top and stable variants match and another set of genes where the top and stable variants disagree; comparisons are made between matching variants and one of the non-matching sets of variants (top or stable).
Concretely, for Potential Set 1, we compare between Nmatch=12743\begin{document}$N_{\text{match}}=12743$\end{document} genes with matching variants and Nnon-match=9921\begin{document}$N_{\text{non-match}}=9921$\end{document} genes with non-matching variants; for Potential Set 2, we compare between Nmatch=8197\begin{document}$N_{\text{match}}=8197$\end{document} genes with matching variants and Nnon-match=14452\begin{document}$N_{\text{non-match}}=14452$\end{document} genes with non-matching variants; and for Potential Set 3, we compare between Nmatch=5807\begin{document}$N_{\text{match}}=5807$\end{document} genes with matching variants and Nnon-match=16812\begin{document}$N_{\text{non-match}}=16812$\end{document} genes with non-matching variants. Below, we summarize our findings.
Matching variants vs non-matching top variants
We observe the following significant trends: as mentioned in the main text, for Potential Set 1361 functional annotations reported significantly higher scores for the matching variants, with all BH-adjusted p-values less than 0.05. The remaining 17 = 378 − 361 annotations not exhibiting significantly higher scores are: Distance to Canonical TSS, CTCF Binding Enrichment, Enhancer Enrichment, Open Chromatin Enrichment, TF Binding Enrichment, Promoter Flanking Enrichment, CADD raw score, PHRED-normalized CADD score, SIFTVal, priPhyloP, mamPhyloP, verPhyloP, GerpS, LINSIGHT, Funseq2, ALoft, and FIRE.
For Potential Set 2, 132 functional annotations reported significantly higher scores for the matching variants. For Potential Set 3, 9 functional annotations reported significantly higher scores for the matching variants: B Statistic, percent GC content, and Enformer tracks ENCFF776DPQ, ENCFF831ZHL, ENCFF601YET, ENCFF945XXY, ENCFF676UXN, ENCFF151LGF, ENCFF876DXW. For all three potential sets, there were no functional annotations reporting significantly lower scores for the matching variants, even for annotations where a low score implies greater biological functionality (e.g., B Statistic).
Matching variants vs non-matching stable variants
We observe the following significant trends: as mentioned in the main text, for Potential Set 1, 363 functional annotations reported significantly higher scores for the matching variants, with all BH-adjusted p-values less than 0.05. The remaining 15 annotations not exhibiting significantly higher scores are: Distance to Canonical TSS, CTCF Binding Enrichment, Enhancer Enrichment, Open Chromatin Enrichment, TF Binding Enrichment, Promoter Flanking Enrichment, CADD raw score, PHRED-normalized CADD score, SIFTVal, mamPhyloP, verPhyloP, GerpS, LINSIGHT, ALoft, and FIRE.
For Potential Set 2, 359 functional annotations reported significantly higher scores for the matching variants (all BH-adjusted p-values <0.05), while only one functional annotation—Distance to Canonical TSS—reported significantly lower scores for the matching variants (BH-adjusted p-value =8×10−4\begin{document}$=8\times 10^{-4}$\end{document}). For Potential Set 3, 259 functional annotations reported significantly higher scores for the matching variants (all BH-adjusted p-values <0.05), while only one functional annotation—Distance to Canonical TSS—reported significantly lower scores for the matching variants (BH-adjusted p-value =8×10−4\begin{document}$=8\times 10^{-4}$\end{document}).
Appendix 8Top variant vs stable variant results
As described in the main text, we compare, across all genes, the annotations of the top and the stable variant. We reported no significant differences in enrichment or trends in enrichment differences driven by moderators for Potential Set 1. For completeness, we report here significant results for all three potential sets.
Paired analysis
After BH adjustment, there are no significant differences across all three potential sets using a threshold of α=0.05\begin{document}$\alpha=0.05$\end{document}.
Trend analysis
For the following moderators, we observe some significant trends.
Inclusion of Distal Subpopulations (Top)
For Potential Set 3, FIRE scores exhibited a greater difference between stable and top variant scores (i.e., stable score−top score\begin{document}$\text{stable score}-\text{top score}$\end{document}) for genes with top variants not discovered in YRI than those with top variants discovered in YRI (one-sided BH-adjusted p=0.03\begin{document}$p=0.03$\end{document}). So did TSS-centered perturbation scores for 121 Enformer tracks and AVE perturbation scores for 121 Enformer tracks. (These tracks and their corresponding BH-adjusted p-values are available here.)
Posterior Probability of Top Variant
For all potential sets, there are small but significant positive correlations between the posterior probability of the top variant and the difference between the stable variant FIRE score and the top variant FIRE score (Potential Set 1: Pearson’s r=0.05\begin{document}$r=0.05$\end{document}, BH-adjusted p=3×10−5\begin{document}$p=3\times 10^{-5}$\end{document}; Potential Set 2: Pearson’s r=0.07\begin{document}$r=0.07$\end{document}, BH-adjusted p=3×10−14\begin{document}$p=3\times 10^{-14}$\end{document}; Potential Set 3: Pearson’s r=0.05\begin{document}$r=0.05$\end{document}, BH-adjusted p=1×10−7\begin{document}$p=1\times 10^{-7}$\end{document}). For all potential sets, there are small but significant negative correlations between the posterior probability of the top variant and the difference between the stable variant Absolute Distance to TSS and the top variant Absolute Distance to TSS (Potential Set 1: Pearson’s r=−0.07\begin{document}$r=-0.07$\end{document}, BH-adjusted p=1×10−10\begin{document}$p=1\times 10^{-10}$\end{document}; Potential Set 2: Pearson’s r=−0.05\begin{document}$r=-0.05$\end{document}, BH-adjusted p=9×10−7\begin{document}$p=9\times 10^{-7}$\end{document}; Potential Set 3: Pearson’s r=−0.06\begin{document}$r=-0.06$\end{document}, BH-adjusted p=2×10−12\begin{document}$p=2\times 10^{-12}$\end{document}). For Potential Set 2, there are significant negative correlations between the posterior probability of the top variant and the difference between the stable variant and top variant Percent GC (Pearson’s r=−0.04\begin{document}$r=-0.04$\end{document}, BH-adjusted p=5×10−5\begin{document}$p=5\times 10^{-5}$\end{document}) and FATHMM-XF scores (Pearson’s r=−0.04\begin{document}$r=-0.04$\end{document}, BH-adjusted p=7×10−4\begin{document}$p=7\times 10^{-4}$\end{document}). For Potential Set 3, there is a significant negative correlation between the posterior probability of the top variant and the difference between the stable variant and top variant FATHMM-XF scores (Pearson’s r=−0.04\begin{document}$r=-0.04$\end{document}, BH-adjusted p=2×10−5\begin{document}$p=2\times 10^{-5}$\end{document}).
Appendix 9Impact of positive posterior probability support on results
One key consideration in statistical fine-mapping is the number of variants possessing positive posterior probability, which we refer to as the positive posterior probability support (or support, for short). Indeed, a larger support may indicate greater uncertainty in the assignment of causal variant to the allele with largest posterior probability. To evaluate the potential utility of the stability-guided approach in settings where the residualization approach leads to large support, we repeat our analysis on two restricted sets of genes, namely (1) those genes where the top variant has support greater than 10; or (2) those genes where the top variant has support greater than 50.
Paired analysis
Gene Set (1). For all Potential Sets, Distance to TSS is significantly smaller for top variant (all three BH-adjusted p-values <10−6\begin{document}$ \lt 10^{-6}$\end{document}). For Potential Set 3, FATHMM-XF score is significantly larger for stable variant (BH-adjusted p<10−6\begin{document}$p \lt 10^{-6}$\end{document}). For all Potential Sets, FIRE score is significantly larger for top variant (BH-adjusted p-values: Potential Set 1 = 0.02, Potential Set 2 =2×10−10\begin{document}$=2\times 10^{-10}$\end{document}, Potential Set 3 =1.7×10−4\begin{document}$=1.7\times 10^{-4}$\end{document}). Gene Set (2). For all Potential Sets 2 and 3, FIRE score is significantly larger for top variant (both BH-adjusted p-values are 0.04).
Trend analysis
Gene Set (1). For the following moderators, we observe some significant trends.
Posterior Probability of Top Variant
For Potential Sets 2 and 3, there is a significant negative correlation between the posterior probability of the top variant and the difference between the stable variant FIRE score and the top variant FIRE score (Potential Set 2: Pearson’s r=0.08\begin{document}$r=0.08$\end{document}, BH-adjusted p=1.9×10−5\begin{document}$p=1.9\times 10^{-5}$\end{document}; Potential Set 3: Pearson’s r=0.06\begin{document}$r=0.06$\end{document}, BH-adjusted p=0.003\begin{document}$p=0.003$\end{document}). For Potential Set 2, there is a significant positive correlation between the posterior probability of the top variant and the difference between the stable variant’s FATHMM-XF score and the top variant’s FATHMM-XF score (Pearson’s r=−0.06\begin{document}$r=-0.06$\end{document}, BH-adjusted p=0.01\begin{document}$p=0.01$\end{document}).
Gene Set (2). For the following moderators, we observe some significant trends.
Posterior Probability of Top Variant
For Potential Set 2, TSS-centered perturbation scores for 8 Enformer tracks exhibited a positive correlation between the posterior probability of the top variant and the difference between stable and top variant perturbation scores. Track names (with empirical Pearson’s r\begin{document}$r$\end{document}) are ENCFF915DFR (-0.40), ENCFF107LDM (-0.41), ENCFF821PRO (-0.42), ENCFF171MDW (-0.43), ENCFF170NTY (-0.43), ENCFF935KTD (-0.41), ENCFF676GTP (-0.41), ENCFF013ZOI (-0.42). Averaged perturbation scores for two Enformer tracks also exhibited negative correlation; these two tracks are ENCFF107LDM (-0.40) and ENCFF170NTY (-0.40). All BH-adjusted p-values are 0.0495.
Appendix 10Impact of posterior probability on results
We perform a comparison between top and stable variants, by restricting to genes where the posterior probability of the top variant or the stable variant exceeds 0.9. Specifically, we repeat our analysis on two restricted sets of genes, namely (1) those genes where the top variant reported a posterior probability exceeding 0.9; or (2) those genes where the stable variant reported a posterior probability exceeding 0.9. For reference, we plot in Figure 3—figure supplement 2 the joint distribution of posterior probabilities of top and stable variants, across all genes for which the two fine-mapping approaches returned distinct variants.
Paired analysis
Gene Set (1). For Potential Set 3, Distance to TSS is significantly smaller for stable variant (BH-adjusted p=0.01\begin{document}$p=0.01$\end{document}). Gene Set (2). For Potential Sets 1 and 2, FIRE scores of the top variant are significantly larger than the stable variant (BH-adjusted p-values: Potential Set 1 =1×10−6\begin{document}$=1\times 10^{-6}$\end{document}, Potential Set 2 = 0.02). For Potential Set 2, the distance to TSS of the top variant is significantly smaller (BH-adjusted p=3×10−4\begin{document}$p=3\times 10^{-4}$\end{document}). For Potential Set 3, FATHMM-XF scores of the stable variant are significantly larger (BH-adjusted p=0.006\begin{document}$p=0.006$\end{document}).
Trend analysis
Gene Set (1). For the following moderators, we observe some significant trends.
Inclusion of Distal Subpopulations (Top)
For Potential Set 3, TSS-centered perturbation scores for nine Enformer tracks exhibited a higher difference between stable and top variant perturbation score for genes with top variants not discovered in YRI vs those discovered in YRI. Track names (with BH-adjusted unpaired Wilcoxon test p-values) are ENCFF279CYY (0.03), ENCFF417WYL (0.04), ENCFF782WWH (0.03), ENCFF629RRF (0.03), ENCFF676GTP (0.03), ENCFF984HLU (0.03), ENCFF700YOH (0.03), ENCFF848LJL (0.0477), ENCFF038IYA (0.03). Averaged perturbation scores for 10 Enformer tracks also a higher difference; these are ENCFF776DPQ (0.02), ENCFF279CYY (0.02), ENCFF629RRF (0.03), ENCFF319YAI (0.04), ENCFF676GTP (0.046), ENCFF367WTF (0.04), ENCFF984HLU (0.04), ENCFF917YSR (0.03), ENCFF613CYH (0.04), and ENCFF700YOH (0.04).
Gene Set (2). For the following moderators, we observe some significant trends.
Posterior Probability of Top Variant
For Potential Set 3, there is a significant negative correlation between the posterior probability of the top variant and the difference between the stable variant Distance to TSS and the top variant Distance to TSS (Pearson’s r=−0.09\begin{document}$r=-0.09$\end{document}, BH-adjusted p=0.004\begin{document}$p=0.004$\end{document}).
Appendix 11On matching variants with very low (stable) posterior probability
In both simulations and analysis of GEUVADIS data, some matching variants have low stable posterior probability. To better interpret such variants, we look at allele frequency heterogeneity by ancestry, posterior probability support size, and (in Stable PICS) number of slices containing the variant. For simulated data, we additionally consider if such low posterior probabilities can be predictive of causal variant recovery. For GEUVADIS data, we compare functional enrichments of such variants relative to other matching variants.
Simulated gene expression
By defining a very low stable posterior probability as having a probability <0.01, we found two expression phenotypes that returned matching top and stable variants in Potential Set 1: rs8008094 (using TSS metadata for gene ENSG00000151413.12) and rs4758290 (using TSS metadata for gene ENSG00000258659.1). Both simulations involved two causal variants and >1000 background variants. Allele frequencies for both variants did not differ substantially across ancestry slices: (fYRI,fTSI,fGBR,fFIN,fCEU)=(0.408,0.302,0.198,0.25,0.208)\begin{document}$(f_{\text{YRI}},f_{\text{TSI}},f_{\text{GBR}},f_{\text{FIN}},f_{\text{CEU}})= \linebreak(0.408,0.302,0.198,0.25,0.208)$\end{document} for rs8008094 with pooled allele frequency f=0.273\begin{document}$f=0.273$\end{document}, while (fYRI,fTSI,fGBR,fFIN,fCEU)=(0.540,0.527,0.523,0.571,0.528)\begin{document}$(f_{\text{YRI}},f_{\text{TSI}},f_{\text{GBR}},f_{\text{FIN}},f_{\text{CEU}})=(0.540,0.527,0.523,0.571,0.528)$\end{document} for rs4758290 with pooled allele frequency f=0.538\begin{document}$f=0.538$\end{document}. However, for rs8008094, the positive posterior probability support size was 47, whereas for rs4758290, the positive posterior probability support size was 7. Additionally, Top PICS returned a posterior probability of 0.359 for rs8008094 and 0.846 for rs4758290. The latter variant also appeared in five slices for Stable PICS, while the former appeared in three slices. It turns out that rs4758290 was one of the causal variants in the simulations whereas rs8008094 was not.
Moving onto Potential Sets 2 and 3 matching variants, using the same cutoff, we identified five (rs4687770, rs2834344, rs1125036, rs7963386, rs74080151) and one (rs1404862) variant(s), respectively, that had low stable posterior probability. All but two simulations involved three causal variants—rs4687770 and rs1404862 involved two causal variants. Inspecting Top PICS posterior probabilities for these variants, we observed a modest spread of values (all contained in [0.33, 0.85]). Each variant was also contained in at least three slices in Stable PICS (three were contained in five slices), while posterior probability support sizes ranged from 6 to 57. Apart from rs4687770, which had a considerable difference in allele frequency between YRI and GBR (fYRI−fGBR=0.36\begin{document}$f_{\text{YRI}}-f_{\text{GBR}}=0.36$\end{document}), all other variants did not report substantial difference in allele frequencies by ancestry. None of these variants were causal variants used in simulations.
While we cannot derive any generalizable conclusions from such a small number of observations, this analysis suggests that matching variants with very low stable posterior probability are largely depleted in causal variants. However, at least for Potential Set 1, other factors, such as the number of slices including the stable variant as well as the top variant posterior probability, may still be useful for causal variant enrichment.
GEUVADIS
Again using a very low stable posterior probability cutoff of 0.01, we found 17 matching variants with very low stable posterior probability across all three potential sets. These are listed in Appendix 12—table 5. Briefly, the Top PICS posterior probabilities ranged from 0.23 to 1, with Stable PICS support sizes ranging from as small as 4 to as large as 45. The number of slices containing the stable variant (including the ALL slice) ranged from 3 to 6, while the maximum allele frequency difference between any pair of ancestry slices ranged from 0.084 to 0.43. Comparing these quantities potential set by potential set, allele frequencies tend to be less homogeneous and support sizes tend to be in the upper half of the distribution across the complementary sets of matching variants with stable posterior probability ≥0.01. Notably, all eight Potential Set 1 variants had support sizes lying above the third quartile of the distribution of support sizes for Potential Set 1 matching variants with stable posterior probability ≥0.01. While we cannot derive any generalizable conclusions from such a small number of observations, this suggests that a larger support size could result in stable variants with low stable posterior probability.
Because we do not know the causal variants, we performed functional enrichment analyses of the 17 matching variants. Similar to the above analysis, we cannot make generalizable claims owing to small sample sizes, but we can nonetheless report some patterns. By inspecting each functional annotation, we did not notice systematic patterns in either extreme direction relative to matching variants with stable posterior probability ≥0.01. However, we occasionally found variants that stand out for having impactful functional annotation scores. We list one below for each potential set.
Potential Set 1 reported the variant rs12224894 from fine-mapping ENSG00000255284.1 (accession code AP006621.3) in Chromosome 11. This variant stood out for lying in the promoter flanking region of multiple cell types and being relatively enriched for GC content within a 75 bp flanking region. This variant has been reported as a cis eQTL for AP006621 (using whole blood gene expression, rather than lymphoblastoid cell line gene expression in this study) in a clinical trial study of patients with systemic lupus erythematosus (Davenport et al., 2018). Its nearest gene is GATD1, a ubiquitously expressed gene that codes for a protein and is predicted to regulate enzymatic and catabolic activity. This variant appeared in all 6 slices, with a moderate support size of 23.
Potential Set 2 reported the variant rs9912201 from fine-mapping ENSG00000108592.9 (mapped to FTSJ3) in Chromosome 17. Its FIRE score is 0.976, which is close to the maximum FIRE score reported across all Potential Set 2 matching variants. This variant has been reported as an SNP in high LD to a GWAS hit SNP rs7223966 in a pan-cancer study (Gong et al., 2018). This variant appeared in all six slices, with a moderate support size of 32.
Potential Set 3 reported the variant rs625750 from fine-mapping ENSG00000254614.1 (mapped to CAPN1-AS1, an RNA gene) in Chromosome 11. Its FIRE score is 0.971 and its B statistic is 0.405 (region under selection), which lie at the extreme quantiles of the distributions of these scores for Potential Set 3 matching variants with stable posterior probability ≥0.01. Its associated mutation has been predicted to affect transcription factor binding, as computed using several position weight matrices (Kheradpour and Kellis, 2014). This variant appeared in just three slices, possibly owing to the considerable allele frequency difference between ancestries (maximum AF difference = 0.22). However, it has a small support size of 4, with moderately high Top PICS posterior probability of 0.64.
To summarize, our analysis of GEUVADIS fine-mapped variants demonstrates that matching variants with very low stable posterior probability could still be functionally important, even for lower potential sets, conditional on supportive scores in interpretable features such as the number of slices containing the stable variant and the posterior probability support size. It would be interesting to scale up such analyses in future work to investigate the generalizability of these conclusions beyond just 17 data points.
Appendix 12Supplementary tables
Plain and Stable PICS matching frequencies.
Below reports the frequencies with which Plain and Stable PICS have matching variants for the same potential set. The numbers of matching variants for each SNR scenario are reported in the parentheses. The bottom two rows show matching frequencies when results are stratified by posterior probability (PP) of the Plain PICS variant. The numbers of matching variants for each PP stratum are reported in the parentheses.
Stratified by signal-to-noise ratio (SNR) of simulations
Potential Set 1
Potential Set 2
Potential Set 3
SNR = 0.053
0.736 (265)
0.803 (289)
0.797 (287)
SNR = 0.111
0.775 (279)
0.753 (271)
0.758 (273)
SNR = 0.25
0.903 (325)
0.714 (257)
0.728 (262)
SNR = 0.667
0.906 (326)
0.753 (271)
0.744 (268)
Stratified by posterior probability (PP) of plain PICS variant
p > 0.9
0.978 (441)
0.899 (286)
0.927 (307)
p ≤ 0.9
0.762 (754)
0.715 (802)
0.706 (783)
Stable and Top PICS matching frequencies.
Below reports the frequencies with which Stable and Top PICS have matching variants for the same potential set. The numbers of matching variants for each SNR/‘No. Causal Variants’ scenario are reported in the parentheses.
Stratified by signal-to-noise ratio (SNR) and No. Causal Variants (S) in simulations
Potential Set 1
Potential Set 2
Potential Set 3
One causal variant (S = 1)
SNR = 0.053
0.695 (139)
0.41 (82)
0.22 (44)
SNR = 0.111
0.75 (150)
0.405 (81)
0.24 (48)
SNR = 0.25
0.825 (165)
0.45 (90)
0.26 (52)
SNR = 0.667
0.895 (179)
0.405 (81)
0.275 (55)
Two causal variants (S = 2)
SNR = 0.053
0.545 (109)
0.36 (72)
0.225 (45)
SNR = 0.111
0.68 (136)
0.38 (76)
0.215 (43)
SNR = 0.25
0.79 (158)
0.435 (87)
0.27 (54)
SNR = 0.667
0.78 (156)
0.41 (82)
0.26 (52)
Three causal variants (S = 3)
SNR = 0.053
0.565 (113)
0.37 (74)
0.245 (49)
SNR = 0.111
0.655 (131)
0.36 (72)
0.265 (53)
SNR = 0.25
0.72 (144)
0.39 (78)
0.22 (44)
SNR = 0.667
0.785 (157)
0.48 (96)
0.255 (51)
Stable and Top SuSiE matching frequencies.
Below reports the frequencies with which Stable and Top SuSiE have matching variants for the same potential set. The numbers of matching variants for each SNR/‘No. Causal Variants’ scenario are reported in the parentheses.
Stratified by signal-to-noise ratio (SNR) and No. Causal Variants (S) in simulations
Potential Set 1
Potential Set 2
Potential Set 3
One causal variant (S = 1)
SNR = 0.053
0.55 (110)
0.115 (23)
0.095 (19)
SNR = 0.111
0.725 (145)
0.14 (28)
0.095 (19)
SNR = 0.25
0.84 (168)
0.145 (29)
0.17 (34)
SNR = 0.667
0.875 (175)
0.14 (28)
0.19 (38)
Two causal variants (S = 2)
SNR = 0.053
0.505 (101)
0.095 (19)
0.065 (13)
SNR = 0.111
0.73 (146)
0.14 (28)
0.095 (19)
SNR = 0.25
0.875 (175)
0.33 (66)
0.105 (21)
SNR = 0.667
0.875 (175)
0.425 (85)
0.115 (23)
Three causal variants (S = 3)
SNR = 0.053
0.345 (69)
0.075 (15)
0.085 (17)
SNR = 0.111
0.68 (136)
0.23 (46)
0.145 (29)
SNR = 0.25
0.795 (159)
0.37 (74)
0.185 (37)
SNR = 0.667
0.85 (170)
0.585 (117)
0.25 (50)
Off-diagonal matching frequencies and causal variant recovery.
Below reports the number of Stable and Top PICS non-matching variants that match across different, or ‘off-diagonal’, potential sets. Frequencies are computed across simulations with the same number of causal variants (S = 1, 2, or 3), with numbers along the yellow-shaded diagonal reporting the number of non-matching variants between the same potential sets. Each off-diagonal element reports both the number of matching variants for the pair of potential sets listed as well as the percentage of these matches that also correspond to the causal variant.
Simulations with one causal variant
Top PICS potential set compared against
Potential Set 1
Potential Set 2
Potential Set 3
Stable PICS potential set
Potential Set 1
167
5 (60%)
3 (67%)
Potential Set 2
4 (25%)
466
104 (0.96%)
Potential Set 3
0
94 (0%)
601
Simulations with two causal variants
Top PICS potential set compared against
Potential Set 1
Potential Set 2
Potential Set 3
Stable PICS potential set
Potential Set 1
241
24 (46%)
5 (60%)
Potential Set 2
29 (52%)
483
88 (10%)
Potential Set 3
9 (11%)
84 (13%)
606
Simulations with three causal variants
Top PICS potential set compared against
Potential Set 1
Potential Set 2
Potential Set 3
Stable PICS potential set
Potential Set 1
255
29 (66%)
7 (14%)
Potential Set 2
30 (40%)
480
79 (18%)
Potential Set 3
4 (0%)
65 (12%)
603
List of matching variants with low stable posterior probability.
Below summarizes the genes and potential sets for which Stable and Top PICS returned matching variants, along with SNP-level and fine-mapping features for interpretation. Five statistics are reported: posterior probability of the stable variant (Stable PP); posterior probability of the top variant (Top PP); posterior probability support size, defined as the number variants with positive probability (Support Size); the number of ancestry slices, including the ALL slice, for which the stable variant had positive posterior probability from running Stable PICS (Number of Slices); the maximum difference in allele frequency between any pair of subpopulations among YRI, TSI, GBR, FIN, and CEU (Max AF Difference).
Potential Set
Gene
Matching variant
Stable PP
Top PP
Support size
Number of slices
Max AF Difference
1
ENSG00000134762.11
rs61731921
0.0028
0.76
23
4
0.22
1
ENSG00000197847.8
rs7130955
0.0075
0.23
45
3
0.18
1
ENSG00000255284.1
rs12224894
0.0067
0.65
23
6
0.14
1
ENSG00000104442.5
rs6995242
0.0092
0.31
42
4
0.34
1
ENSG00000146733.9
rs10239528
0.0031
0.53
27
5
0.24
1
ENSG00000248468.1
rs9853505
0.0099
0.29
39
3
0.43
1
ENSG00000122224.10
rs57449
0.0089
0.50
25
4
0.31
1
ENSG00000134262.8
rs17464525
0.0030
0.45
23
4
0.15
2
ENSG00000216522.3
rs5751902
0.0052
1
7
3
0.16
2
ENSG00000108592.9
rs9912201
0.0022
0.32
32
6
0.27
2
ENSG00000134551.7
rs7315843
0.0019
0.58
10
5
0.22
2
ENSG00000221947.3
rs3103860
0.0018
0.99
4
4
0.084
2
ENSG00000081791.4
rs2270113
0.0059
0.77
11
4
0.33
3
ENSG00000140368.8
rs62027296
0.0069
0.29
21
4
0.15
3
ENSG00000254614.1
rs625750
0.0017
0.64
4
3
0.22
3
ENSG00000133835.9
rs2451818
0.0036
0.40
32
3
0.41
3
ENSG00000158234.8
rs693293
0.0069
0.55
24
4
0.13
List of variant annotations with interpretations.
Functional annotation
Interpretation
Distance to Canonical Transcription Start Site (TSS)
-
Percent CpG in 75 bp window centered on variant position
-
Percent GC in 75 bp window centered on variant position
-
CTCF Binding Enrichment
Whether the variant lies within a CTCF binding site region as predicted by Ensembl
Enhancer Enrichment
Whether the variant lies within an enhancer region as predicted by Ensembl
Open Chromatin Enrichment
Whether the variant lies within an open chromatin region as predicted by Ensembl
Promoter Enrichment
Whether the variant lies within a promoter region as predicted by Ensembl
TF Binding Enrichment
Whether the variant lies within a TF-binding site region as predicted by Ensembl
Promoter Flanking Enrichment
Whether the variant lies within a promoter flanking region as predicted by Ensembl
CADD (2 scores)
Whether the variant is likely to be simulated or not, and hence likely deleterious or not. One score is raw while the other is rank-normalized
SIFTVal
Whether the variant affects protein function, and hence deleterious
Polyphen2
Posterior probability that the variant is damaging
LINSIGHT
Probability that the variant site is under selection, thus having functional consequence
PhyloP (3 scores)
Substitution rates measuring cross-species evolutionary conservation at the site of the variant. Each score is computed with respect to a clade (vertebrate, mammal, primate)
GerpN
Estimated neutral substitution rate at variant position, with higher value implying greater conservation
GerpS
Estimated rejected substitution rate at variant position, with positive value implying a deficit in substitutions
B Statistic
Background selection at variant position, with smaller value indicating larger impact of selection
FATHMM-XF
Integrative score measuring deleteriousness of the variant
Funseq2
Integrative score measuring deleteriousness of the variant
ALoft
Integrative score measuring loss of function associated with the variant
FIRE
Integrative score measuring deleteriousness of the variant
Magnitude of Effect on Enformer Track Prediction (177 tracks)
Change in prediction of a gene regulatory track when performing in silico mutagenesis on the variant in a 196,608 bp sequence
10.7554/eLife.88039.3.sa0eLife AssessmentYoungAlexanderReviewing EditorUniversity of California, Los AngelesUnited StatesConvincingImportant
This important study presents a methodologically rigorous framework for stability-guided fine-mapping, extending PICS and generalizing to methods such as SuSiE, supported by comprehensive simulations and functional enrichment analyses. The evidence is now convincing, demonstrating improved causal variant recovery and offering a robust alternative for cross-population fine-mapping. The approach will be of particular interest to statistical geneticists, computational biologists, and biomedical researchers who rely on fine-mapping to interpret genetic association signals.
Aw et al. have proposed that utilizing stability analysis can be useful for fine-mapping of cross populations. In addition, the authors have performed extensive analyses to understand the cases where the top eQTL and stable eQTL are the same or different via functional data.
Aw et al presents a new stability-guided fine-mapping method by extending the previously proposed PICS method. They applied their stability-based method to fine-map cis-eQTLs in the GEUVADIS dataset and compared it against residualization-based approaches. They evaluated the performance of the proposed method using publicly available functional annotations and demonstrated that the variants identified by their stability-based method show enrichment for these functional annotations.
The authors have substantially strengthened the manuscript by addressing the major concerns raised in the initial review. I acknowledge that they have conducted comprehensive simulation studies to show the performance of their proposed approach and that they have extended their approach to SuSiE ("Stable SuSiE") to demonstrate the broader applicability of the stability-guided principle beyond PICS.
One remaining question is the interpretation of matching variants with very low stable posterior probabilities (~0), which the authors have analyzed in detail but without fully conclusive findings. I agree with the authors that this event is relatively rare and the current sample size is limited but this might be something to keep in mind for future studies.
10.7554/eLife.88039.3.sa3Author responseAwAlan JAuthorUniversity of California, BerkeleyBerkeleyUnited StatesJinLionel ChentianAuthorhttps://ror.org/01gmv5d77McKinsey & Company (United States)New YorkUnited StatesIoannidisNilahAuthorUniversity of California, BerkeleyBerkeleyUnited StatesSongYun SAuthorUniversity of California, BerkeleyBerkeleyUnited States
The following is the authors’ response to the latest reviews:
"One remaining question is the interpretation of matching variants with very low stable posterior probabilities (~0), which the authors have analyzed in detail but without fully conclusive findings. I agree with the authors that this event is relatively rare and the current sample size is limited but this might be something to keep in mind for future studies."
Fine-mapping stability – on matching variants with very low stable posterior probability
We thank Reviewer 2 for encouraging us to think more about how low stable posterior probability matching variants can be interpreted. We describe a few plausible interpretations, even though – as Reviewer 2 and we have both acknowledged – our present experiments do not point to a clear and conclusive account.
One explanation is that the locus captured by the variant might not be well-resolved, in the sense that many correlated variants exist around the locus. Thus, the variant itself is unlikely causal, but the set of variants in high LD with it may contain the true causal variant, or it's possible that the causal variant itself was not sequenced but lies in that locus. A comparison of LD patterns across ancestries at the locus would be helpful here.
Another explanation rests on the following observation. For a variant to be matching between top and stable PICS and to also have very small stable PP, it has to have the largest PP after residualization on the ALL slice but also have positive PP with gene expression on many other slices. In other words, failing to control for potential confounders shrinks the PP. If one assumes that the matching variant is truly causal, then our observation points to an example of negative confounding (aka suppressor effect). This can occur when the confounders (PCs) are correlated with allele dosage at the causal variant in a different direction than their correlation with gene expression, so that the crude association between unresidualized gene expression and causal variant allele dosage is biased toward 0.
Although our present study does not allow us to systematically confirm either interpretation – since we found that matching variants were depleted in causal variants in our simulations, violating the second argument, but we also found functional enrichment in analyses of GEUVADIS data though only 17 matching variants with low stable PP were reported – we believe a larger-scale study using larger cohort sizes (at least 1000 individuals per ancestry) and many more simulations (to increase yield of such cases) would be insightful.
———
The following is the authors’ response to the original reviews:
Reviewer #1:
Major comments:
(1) It would be interesting to see how much fine-mapping stability can improve the fine-mapping results in cross-population. One can simulate data using true genotype data and quantify the amount the fine-mapping methods improve utilizing the stability idea.
We agree, and have performed simulation studies where we assume that causal variants are shared across populations. Specifically, by mirroring the simulation approach described in Wang et al. (2020), we generated 2,400 synthetic gene expression phenotypes across 22 autosomes, using GEUVADIS gene expression metadata (i.e., gene transcription start site) to ensure largely cis expression phenotypes were simulated. We additionally generated 1,440 synthetic gene expression phenotypes that incorporate environmental heterogeneity, to motivate our pursuit of fine-mapping stability in the first place (see Response to Reviewer 2, Comment 6). These are described in Results section “Simulation study”:
We evaluated the performance of the PICS algorithm, specifically comparing the approach incorporating stability guidance against the residualization approach that is more commonly used — similar to our application to the real GEUVADIS data. We additionally investigated two ways of “combining” the residualization and stability guidance approaches: (1) running stability-guided PICS on residualized phenotypes; (2) prioritizing matching variants returned by both approaches. See Response to Reviewer 2, Comment 5.
(2) I would be very interested to see how other fine-mapping methods (FINEMAP, SuSiE, and CAVIAR) perform via the stability idea.
Thank you for this valuable comment. We ran SuSiE on the same set of simulated datasets. Specifically, we ran a version that uses residualized phenotypes (supposedly removing the effects of population structure), and also a version that incorporates stability. The second version is similar to how we incorporate stability in PICS. We investigated the performance of Stable SuSiE in a similar manner to our investigation of PICS. First we compared the performance relative to SuSiE that was run on residualized phenotypes. Motivated by our finding in PICS that prioritizing matching variants improves causal variant recovery, we did the same analysis for SuSiE. This analysis is described in Results section “Stability guidance improves causal variant recovery in SuSiE.”
We reported overall matching frequencies and causal variant recovery rates of top and stable variants for SuSiE in Figures 2C&D.
Frequencies with which Stable and Top SuSiE variants match, stratified by the simulation parameters, are summarized in Supplementary File 2C (reproduced for convenience in Response to Reviewer 2, Comment 3). Causal variant recovery rates split by the number of causal variants simulated, and stratified by both signal-to-noise ratio and the number of credible sets included, are reported in Figure 2—figure supplements 16-18. We reproduce Figure 2—figure supplement 18 (three causal variants scenario) below for convenience. Analogous recovery rates for matching versus non-matching top or stable variants are reported in Figure 2—figure supplements 19, 21 and 23.
(3) I am a little bit concerned about the PICS's assumption about one causal variant. The authors mentioned this assumption as one of their method limitations. However, given the utility of existing fine-mapping methods (FINEMAP and SuSiE), it is worth exploring this domain.
Thank you for raising this fair concern. We explored this domain, by considering simulations that include two and three causal variants (see Response to Reviewer 2, Comment 3). We looked at how well PICS recovers causal variants, and found that each potential set largely does not contain more than one causal variant (Figure 2—figure supplements 20 and 22). This can be explained by the fact that PICS potential sets are constructed from variants with a minimum linkage disequilibrium to a focal variant. On the other hand, in SuSiE, we observed multiple causal variants appearing in lower credible sets when applying stability guidance (Figure 2—figure supplements 21 and 23). A more extensive study involving more fine-mapping methods and metrics specific to violation of the one causal variant assumption could be pursued in future work.
Reviewer #2:
Aw et al. presents a new stability-guided fine-mapping method by extending the previously proposed PICS method. They applied their stability-based method to fine-map cis-eQTLs in the GEUVADIS dataset and compared it against what they call residualization-based method. They evaluated the performance of the proposed method using publicly available functional annotations and claimed the variants identified by their proposed stability-based method are more enriched for these functional annotations.
While the reviewer acknowledges the contribution of the present work, there are a couple of major concerns as described below.
Major:
(1) It is critical to evaluate the proposed method in simulation settings, where we know which variants are truly causal. While I acknowledge their empirical approach using the functional annotations, a more unbiased, comprehensive evaluation in simulations would be necessary to assess its performance against the existing methods.
Thank you for this point. We agree. We have performed a simulation study where we assume that causal variants are shared across populations (see response to Reviewer 1, Comment 1). Specifically, by mirroring the simulation approach described in Wang et al. (2020), we generated 2,400 synthetic gene expression phenotypes across 22 autosomes, using GEUVADIS gene expression metadata (i.e., gene transcription start site) to ensure cis expression phenotypes were simulated.
(2) Also, simulations would be required to assess how the method is sensitive to different parameters, e.g., LD threshold, resampling number, or number of potential sets.
Thank you for raising this point. The underlying PICS algorithm was not proposed by us, so we followed the default parameters set (LD threshold, r2 = 0.5; see Taylor et al., 2021 Bioinformatics) to focus on how stability considerations will impact the existing fine-mapping algorithm. We attempted to derive the asymptotic joint distribution of the p-values, but it was too difficult. Hence, we used 500 permutations because such a large number would allow large-sample asymptotics to kick in. However, following your critical suggestion we varied the number of potential sets in our analyses of simulated data. We briefly mention this in the Results.
“In the Supplement, we also describe findings from investigations into the impact of including more potential sets on matching frequency and causal variant recovery…”
A detailed write-up is provided in Supplementary File 1 Section S2 (p.2):
“The number of credible or potential sets is a parameter in many fine-mapping algorithms. Focusing on stability-guided approaches, we consider how including more potential sets for stable fine-mapping algorithms affects both causal variant recovery and matching frequency in simulations…
Causal variant recovery. We investigate both Stable PICS and Stable SuSiE. Focusing first on simulations with one causal variant, we observe a modest gain in causal variant recovery for both Stable PICS and Stable SuSiE, most noticeably when the number of sets was increased from 1 to 2 under the lowest signal-to-noise ratio setting…”
We observed that increasing the number of potential sets helps with recovering causal variants for Stable PICS (Figure 2—figure supplements 13-15). This observation also accounts for the comparable power that Stable PICS has with SuSiE in simulations with low signal-to-noise ratio (SNR), when we increase the number of credible sets or potential sets (Figure 2—figure supplements 10-12).
(3) Given the previous studies have identified multiple putative causal variants in both GWAS and eQTL, I think it's better to model multiple causal variants in any modern fine-mapping methods. At least, a simulation to assess its impact would be appreciated.
We agree. In our simulations we considered up to three causal variants in cis, and evaluated how well the top three Potential Sets recovered all causal variants (Figure 2—figure supplements 13-15; Figure 2—figure supplement 15). We also reported the frequency of variant matches between Top and Stable PICS stratified by the number of causal variants simulated in Supplementary File 2B and 2C. Note Supplementary File 2C is for results from SuSiE fine-mapping; see Response to Reviewer 1, Comment 2.
Supplementary File 2B. Frequencies with which Stable and Top PICS have matching variants for the same potential set. For each SNR/ “No. Causal Variants” scenario, the number of matching variants is reported in parentheses.
Supplementary File 2C. Frequencies with which Stable and Top SuSiE have matching variants for the same credible set. For each SNR/ “No. Causal Variants” scenario, the number of matching variants is reported in parentheses.
(4) Relatedly, I wonder what fraction of non-matching variants are due to the lack of multiple causal variant modeling.
PICS handles multiple causal variants by including more potential sets to return, owing to the important caveat that causal variants in high LD cannot be statistically distinguished. For example, if one believes there are three causal variants that are not too tightly linked, one could make PICS return three potential sets rather than just one. To answer the question using our simulation study, we subsetted our results to just scenarios where the top and stable variants do not match. This mimics the exact scenario of having modeled multiple causal variants but still not yielding matching variants, so we can investigate whether these non-matching variants are in fact enriched in the true causal variants.
Because we expect causal variants to appear in some potential set, we specifically considered whether these non-matching causal variants might match along different potential sets across the different methods. In other words, we compared the stable variant with the top variant from another potential set for the other approach (e.g., Stable PICS Potential Set 1 variant vs Top PICS Potential Set 2 variant). First, we computed the frequency with which such pairs of variants match. A high frequency would demonstrate that, even if the corresponding potential sets do not have a variant match, there could still be a match between non-corresponding potential sets across the two approaches, which shows that multiple causal variant modeling boosts identification of matching variants between both approaches — regardless of whether the matching variant is in fact causal.
Low frequencies were observed. For example, when restricting to simulations where Top and Stable PICS Potential Set 1 variants did not match, about 2-3% of variants matched between the Potential Set 1 variant in Stable PICS and Potential Sets 2 and 3 variants in Top PICS; or between the Potential Set 1 variant in Top PICS and Potential Sets 2 and 3 variants in Stable PICS (Supplementary File 2D). When looking at non-matching Potential Set 2 or Potential Set 3 variants, we do see an increase in matching frequencies (between 10-20%) between Potential Set 2 variants and other potential set variants between the different approaches. However, these percentages are still small compared to the matching frequencies we observed between corresponding potential sets (e.g., for simulations with one causal variant this was 70-90% between Top and Stable PICS Potential Set 1, and for simulations with two and three causal variants this was 55-78% and 57-79% respectively).
We next checked whether these “off-diagonal” matching variants corresponded to the true causal variants simulated. Here we find that the causal variant recovery rate is mostly less than the corresponding rate for diagonally matching variants, which together with the low matching frequency suggests that the enrichment of causal variants of “off-diagonal” matching variants is much weaker than in the diagonally matching approach. In other words, the fraction of non-matching (causal) variants due to the lack of multiple causal variant modeling is low.
We discuss these findings in Supplementary File 1 Section S2 (bottom of p.2).
(5) I wonder if you can combine the stability-based and the residualization-based approach, i.e., using the residualized phenotypes for the stability-based approach. Would that further improve the accuracy or not?
This is a good idea, thank you for suggesting it. We pursued this combined approach on simulated gene expression phenotypes, but did not observe significant gains in causal variant recovery (Figure 2B; Figure 2—figure supplements 2, 13 and 15). We reported this Results “Searching for matching variants between Top PICS and Stable PICS improves causal variant Recovery.”
“We thus explore ways to combine the residualization and stability-driven approaches, by considering (i) combining them into a single fine-mapping algorithm (we call the resulting procedure Combined PICS); and (ii) prioritizing matching variants between the two algorithms. Comparing the performance of Combined PICS against both Top and Stable PICS, however, we find no significant difference in its ability to recover causal variants (Figure 2B)...”
However, we also confirmed in our simulations that prioritizing matching variants between the two approaches led to gains in causal variant recovery (Figure 2D; Figure 2—figure supplements 4, 19, 20 and 22). We reported this Results “Searching for matching variants between Top PICS and Stable PICS improves causal variant Recovery.”
“On the other hand, matching variants between Top and Stable PICS are significantly more likely to be causal. Across all simulations, a matching variant in Potential Set 1 is 2.5X as likely to be causal than either a non-matching top or stable variant (Figure 2D) — a result that was qualitatively consistent even when we stratified simulations by SNR and number of causal variants simulated (Figure 2—figure supplements 19, 20 and 22)...”
This finding is consistent with our analysis of real GEUVADIS gene expression data, where we reported larger functional significance of matching variants relative to non-matching variants returned by either Top of Stable PICS.
(6) The authors state that confounding in cohorts with diverse ancestries poses potential difficulties in identifying the correct causal variants. However, I don't see that they directly address whether the stability approach is mitigating this. It is hard to say whether the stability approach is helping beyond what simpler post-hoc QC (e.g., thresholding) can do.
Thank you for raising this fair point. Here is a model we have in mind. Gene expression phenotypes (Y) can be explained by both genotypic effects (G, as in genotypic allelic dosage) and the environment (E): Y = G + E. However, both G and E depend on ancestry (A), so that Y = G|A+E|A. Suppose that the causal variants are shared across ancestries, so that (G|A=a)=G for all ancestries a. Suppose however that environments are heterogeneous by ancestry: (E|A=a) = e(a) for some function e that depends non-trivially on a. This would violate the exchangeability of exogenous E in the full sample, but by performing fine-mapping on each ancestry stratum, the exchangeability of exogenous E is preserved. This provides theoretical justification for the stability approach.
We next turned to simulations, where we investigated 1,440 simulated gene expression phenotypes capturing various ways in which ancestry induces heterogeneity in the exogenous E variable (simulation details in Lines 576-610 of Materials and Methods). We ran Stable PICS, as well as a version of PICS that did not residualize phenotypes or apply the stability principle. We observed that (i) causal variant recovery performance was not significantly different between the two approaches (Figure 2—figure supplements 24-32); but (ii) disagreement between the approaches can be considerable, especially when the signal-to-noise ratio is low (Supplementary File 2A). For example, in a set of simulations with three causal variants, with SNR = 0.11 and E heterogeneous by ancestry by letting E be drawn from N(2σ,σ2) for only GBR individuals (rest are N(0,σ2)), there was disagreement between Potential Set 1 and 2 variants in 25% of simulations — though recovery rates were similar (Probability of recovering at least one causal variant: 75% for Plain PICS and 80% for Stable PICS). These points suggest that confounding in cohorts can reduce power in methods not adjusting or accounting for ancestral heterogeneity, but can be remedied by approaches that do so. We report this analysis in Results “Simulations justify exploration of stability guidance”
In the current version of our work, we have evaluated, using both simulations and empirical evidence, different ways to combine approaches to boost causal variant recovery. Our simulation study shows that prioritizing matching variants across multiple methods improves causal variant recovery. On GEUVADIS data, where we might not know which variants are causal, we already demonstrated that matching variants are enriched for functional annotations. Therefore, our analyses justify that the adverse consequence of confounding on reducing fine-mapping accuracy can be mitigated by prioritizing matching variants between algorithms including those that account for stability.
(7) For non-matching variants, I wonder what the difference of posterior probabilities is between the stable and top variants in each method. If the difference is small, maybe it is due to noise rather than signal.
We have reported differences in posterior probabilities returned by Stable and Top PICS for GEUVADIS data; see Figure 3—figure supplement 1. For completeness, we compute the differences in posterior probabilities and summarize these differences both as histograms and as numerical summary statistics.
Potential Set 1
- Number of non-matching variants = 9,921
- Table of Summary Statistics of (Stable Posterior Probability – Top Posterior Probability)
Min
1st Qu.
Median
Mean
3rd Qu.
Max
-0.999
-0.342
-0.107
-0.117
0.067
0.949
- Histogram of (Stable Posterior Probability – Top Posterior Probability)
Potential Set 2
- Number of non-matching variants = 14,454
- Table of Summary Statistics of (Stable Posterior Probability – Top Posterior Probability)
Min
1st Qu.
Median
Mean
3rd Qu.
Max
-1.000
-0.334
-0.0741
-0.0778
0.162
0.976
- Histogram of (Stable Posterior Probability – Top Posterior Probability)
Potential Set 3
- Number of non-matching variants = 16,814
- Table of Summary Statistics of (Stable Posterior Probability – Top Posterior Probability)
Min
1st Qu.
Median
Mean
3rd Qu.
Max
-0.998
-0.327
-0.0564
-0.0629
0.191
0.968
- Histogram of (Stable Posterior Probability – Top Posterior Probability)
We also compared the difference in posterior probabilities between non-matching variants returned by Stable PICS and Top PICS for our 2,400 simulated gene expression phenotypes. Focusing on just Potential Set 1 variants, we find two equally likely scenarios, as demonstrated by two distinct clusters of points in a “posterior probability-posterior probability” plot. The first is, as pointed out, a small difference in posterior probability (points lying close to y=x). The second, however, reveals stable variants with very small posterior probability (of order 4 x 10–5 to 0.05) but with a non-matching top variant taking on posterior probability well distributed along [0,1]. Moving down to Potential Sets 2 and 3, the distribution of pairs of posterior probabilities appears less clustered, indicating less tendency for posterior probability differences to be small (Figure 2—figure supplement 8).
Here are the histograms and numerical summary statistics.
Potential Set 1
- Number of non-matching variants = 663 (out of 2,400)
- Table of Summary Statistics of (Stable Posterior Probability – Top Posterior Probability)
Min
1st Qu.
Median
Mean
3rd Qu.
Max
-0.994
-0.266
-0.00645
-0.109
0.049
0.924
- Histogram of (Stable Posterior Probability – Top Posterior Probability)
Potential Set 2
Number of non-matching variants = 1,429 (out of 2,400)
- Table of Summary Statistics of (Stable Posterior Probability – Top Posterior Probability)
Min
1st Qu.
Median
Mean
3rd Qu.
Max
-0.998
-0.376
-0.0944
-0.121
0.115
0.903
- Histogram of (Stable Posterior Probability – Top Posterior Probability)
Potential Set 3
- Number of non-matching variants = 1,810 (out of 2,400)
- Table of Summary Statistics of (Stable Posterior Probability – Top Posterior Probability)
Min
1st Qu.
Median
Mean
3rd Qu.
Max
-0.998
-0.389
-0.100
-0.114
0.146
0.915
- Histogram of (Stable Posterior Probability – Top Posterior Probability)
(8) It's a bit surprising that you observed matching variants with (stable) posterior probability ~ 0 (SFig. 1). What are the interpretations for these variants? Do you observe functional enrichment even for low posterior probability matching variants?
Thank you for this question. We have performed a thorough analysis of matching variants with very low stable posterior probability, which we define as having a posterior probability < 0.01 (Supplementary File 1 Section S11). Here, we briefly summarize the analysis and key findings.
Analysis
First, such variants occur very rarely — only 8 across all three potential sets in simulations, and 17 across all three potential sets for GEUVADIS (the latter variants are listed in Supplementary 2E). We begin interpreting these variants by looking at allele frequency heterogeneity by ancestry, support size — defined as the number of variants with positive posterior probability in the ALL slice* — and the number of slices including the stable variant (i.e., the stable variant reported positive posterior probability for the slice).
*Note that the stable variant posterior probability need not be at least 1/(Support Size). This is because the algorithm may have picked a SNP that has a lower posterior probability in the ALL slice (i.e., not the top variant) but happens to appear in the most number of other slices (i.e., a stable variant).
For variants arising from simulations, because we know the true causal variants, we check if these variants are causal. For GEUVADIS fine-mapped variants, we rely on functional annotations to compare their relative enrichment against other matching variants that did not have very low stable posterior probability.
Findings
While we caution against generalizing from observations reported here, which are based on very small sample sizes, we noticed the following. In simulations, matching variants with very low stable posterior probability are largely depleted in causal variants, although factors such as the number of slices including the stable variant may still be useful. In GEUVADIS, however, these variants can still be functionally enriched. We reported three examples in Supplementary File 1 Section S11 (pp. 8-9 of Supplement), where the variants were enriched in either VEP or biologically interpretable functional annotations, and were also reported in earlier studies. We partially reproduce our report below for convenience.
“However, we occasionally found variants that stand out for having large functional annotation scores. We list one below for each potential set.
- Potential Set 1 reported the variant rs12224894 from fine-mapping ENSG00000255284.1 (accession code AP006621.3) in Chromosome 11. This variant stood out for lying in the promoter flanking region of multiple cell types and being relatively enriched for GC content with a 75bp flanking region. This variant has been reported as a cis eQTL for AP006632 (using whole blood gene expression, rather than lymphoblastoid cell line gene expression in this study) in a clinical trial study of patients with systemic lupus erythematosus (Davenport et al., 2018). Its nearest gene is GATD1, a ubiquitously expressed gene that codes for a protein and is predicted to regulate enzymatic and catabolic activity. This variant appeared in all 6 slices, with a moderate support size of 23.
- Potential Set 2 reported the variant rs9912201 from fine-mapping ENSG00000108592.9 (mapped to FTSJ3) in Chromosome 17. Its FIRE score is 0.976, which is close to the maximum FIRE score reported across all Potential Set 2 matching variants. This variant has been reported as a SNP in high LD to a GWAS hit SNP rs7223966 in a pan-cancer study (Gong et al., 2018). This variant appeared in all 6 slices, with a moderate support size of 32.
- Potential Set 3 reported the variant rs625750 from fine-mapping ENSG00000254614.1 (mapped to CAPN1-AS1, an RNA gene) in Chromosome 11. Its FIRE score is 0.971 and its B statistic is 0.405 (region under selection), which lie at the extreme quantiles of the distributions of these scores for Potential Set 3 matching variants with stable posterior probability at least 0.01. Its associated mutation has been predicted to affect transcription factor binding, as computed using several position weight matrices (Kheradpour and Kellis, 2014). This variant appeared in just 3 slices, possibly owing to the considerable allele frequency difference between ancestries (maximum AF difference = 0.22). However, it has a small support size of 4 and a moderately high Top PICS posterior probability of 0.64.
To summarize, our analysis of GEUVADIS fine-mapped variants demonstrates that matching variants with very low stable posterior probability could still be functionally important, even for lower potential sets, conditional on supportive scores in interpretable features such as the number of slices containing the stable variant and the posterior probability support size…”