Gaining insights into microbial metabolism requires accurate estimates of the presence/absence and completion of metabolic modules. While a myriad of tools address this task for single genomes (Machado et al., 2018; Aziz et al., 2008; Arkin et al., 2018; Palù et al., 2022; Shaffer et al., 2020; Geller-McGrath et al., 2023; Zorrilla et al., 2021; Zhou et al., 2022; Zimmermann et al., 2021), working with complex environmental metagenomes poses additional challenges due to the large number of organisms that are present in metagenomic assemblies. A few tools can estimate community-level metabolic potential from metagenomes without relying on the reconstruction of individual population genomes or reference-based approaches (Ye and Doak, 2009; Karp et al., 2021; Supplementary file 5). These high-level summaries of module presence and redundancy in a given environment are suitable for most surveys of metabolic capacity, particularly for microbial communities of similar richness. However, since the frequency of observed metabolic modules will increase as the number of distinct microbial populations in a habitat increases, investigations of metabolic determinants of survival across environmental conditions with substantial differences in microbial richness may suffer from ambiguous observations from quantitative data. For instance, the estimated copy number of a given metabolic module may be identical between two metagenomes but its enrichment may be relatively higher in the metagenome with a lower alpha diversity, revealing its potential role in overcoming environment-specific selective pressures that influence an entire community. Working solely with raw copy numbers of metabolic modules without a normalization step that considers the microbial richness will thus shroud potentially critical insights. To quantify the differential enrichment of metabolic modules between metagenomes generated from healthy individuals and those from individuals diagnosed with IBD, we implemented a new software framework (https://anvio.org/m/anvi-estimate-metabolism) that reconstructs metabolic modules from genomes and metagenomes, and a means to calculate the per-population copy number (PPCN) of modules in metagenomes to account for potential differences in microbial richness (Methods, Appendices 1 and 2). Briefly, the PPCN estimates the proportion of microbes in a community with a particular metabolic capacity (Figure 1, Appendix 1—figure 2) by normalizing observed metabolic module copy numbers with the ‘number of microbial populations in a given metagenome’, which we estimate using the SCGs without relying on the reconstruction of individual genomes. Our validation of this method using simulated metagenomic data demonstrated that it is accurate in capturing metagenome-level metabolic capacity relative to genome-level metabolic capacity estimated from the same data (Appendix 2, Supplementary file 6).

To gain insight into potential metabolic determinants of microbial survival in the IBD gut environment, we assessed the distribution of metabolic modules within samples from each group (IBD and healthy) with and without using PPCN normalization. Without normalizing, module copy numbers were overall higher in healthy samples (Figure 2A) and modules exhibited weak differential occurrence between cohorts (Figure 2B and C, Appendix 1—figure 3). The application of PPCN reversed this trend, and most metabolic modules were elevated in IBD (Appendix 1—figure 5). This observation is influenced by two independent aspects of the healthy and IBD microbiota. The first one is the increased representation of microbial organisms with smaller genomes in healthy individuals (Watson et al., 2023), which increases the likelihood that the overall copy number of a given metabolic module is below the actual number of populations. In contrast, one of the hallmarks of the IBD microbiota is the generally increased representation of organisms with larger genomes (Watson et al., 2023). The second aspect is that the generally higher diversity of microbes in healthy individuals increases the denominator of the PPCN. This results in a greater reduction in the PPCN of metabolic modules that are not shared across all members of the diverse gut microbial populations in health.

To go beyond this general trend and identify modules that were highly conserved in the IBD group, we first selected those that passed a relatively high statistical significance threshold in our enrichment test (Wilcoxon rank-sum test, FDR-adjusted p-value < 2e-10). We then accounted for effect size by ranking these modules according to the difference between their median PPCN in IBD samples and their median PPCN in healthy samples, and keeping only those in the top 50% (which translated to an effect size threshold of > 0.12). This stringent filtering revealed a set of 33 metabolic modules that were significantly enriched in metagenomes obtained from individuals diagnosed with IBD (Figure 2D and E), 17 of which matched the modules that were associated with HMI previously (Watson et al., 2023; Figure 2F). This result suggests that the PPCN normalization is an important step in comparative analyses of metabolisms between samples with different levels of microbial diversity.

The majority of the metabolic modules that were significantly enriched in the microbiomes of IBD patients encoded biosynthetic capabilities (23 out of 33) that resolved to amino acid metabolism (33%), carbohydrate metabolism (21%), cofactor and vitamin biosynthesis (15%), nucleotide biosynthesis (12%), lipid biosynthesis (6%), and energy metabolism (6%) (Supplementary file 2a). In contrast to previous reports based on reference genomes (Gevers et al., 2014; Morgan et al., 2012), amino acid synthesis and carbohydrate metabolism were not reduced in the IBD gut microbiome in our dataset. Rather, our results were in accordance with a more recent finding that predicted amino acid secretion potential is increased in the microbiomes of individuals with IBD (Heinken et al., 2021).

Within our set of 33 modules that were enriched in IBD, it is notable that all the biosynthesis and central carbohydrate modules are directly or indirectly linked via shared enzymes and metabolites. Each enriched module shared on average 25.6% of its enzymes and 40.2% of metabolites with the other enriched modules, and overall 18.2% of enzymes and 20.4% of compounds across these modules were shared (Supplementary file 2a). Thus, modules may be enriched not just due to the importance of their immediate end products, but also because of their role in the larger metabolic network. The few standalone modules that were enriched included the efflux pump MepA and the beta-lactam resistance system, which are associated with drug resistance. These capacities may provide an advantage since antibiotics are a common treatment for IBDs (Nitzan et al., 2016), but are not necessarily related to the systematic enrichment of biosynthesis modules that likely provide resilience to general environmental stress rather than to a specific stressor such as antibiotics.

Microbiome data generated by different groups can result in systematic biases that may outweigh biological differences between otherwise similar samples (Lozupone et al., 2013; Sinha et al., 2017; Clausen and Willis, 2022). The potential impact of such biases constitutes an important consideration for meta-analyses such as ours that analyze publicly available metagenomes from multiple sources. To account for cohort biases, we conducted an analysis of our data on a per-cohort basis. All cohorts within a given group exhibited similar distributions of PPCN values, which indicates that the trends we observed above result from an overall between-group difference in signal rather than a cohort-specific signal (Appendix 1—figure 6B and C). Another source of potential bias stems from the annotation efficiency of gene function. For instance, we noticed that, independent of the annotation strategy, a smaller proportion of genes resolved to known functions in metagenomic assemblies of samples from healthy individuals compared to the samples from individuals who were diagnosed with IBD (Appendix 1—figure 4). This highlights the possibility that samples from healthy individuals merely appear to harbor less metabolic capabilities due to missing annotations. Indeed, we found that the normalized copy numbers of most metabolic modules were reduced in the healthy group, where 84% of KEGG modules (98 out of 118) have significantly lower median copy numbers (Appendix 1—figure 5C, Appendix 1). While the presence of a bias between the two cohorts is clear, the source of this bias and its implications are not. One hypothesis that could explain this phenomenon is that the increased proportion of unknown functions in environments where populations with low metabolic independence (LMI) thrive is due to our inability to identify distant homologs of even well-studied functions in poorly studied novel genomes through public databases. If true, this would indeed impair our ability to annotate genes using state-of-the-art functional databases and bias metabolic module completion estimates. Such a limitation would warrant a careful reconsideration of common workflows and studies that rely on public resources to characterize gene function in complex environments. Another hypothesis that could explain our observation is that the general absence of microbes with smaller genomes in culture had a historical impact on the characterization of novel functions that represent a relatively larger fraction of their gene repertoire. If true, this would suggest that the unknown functions are unlikely essential for well-studied metabolic capabilities. Furthermore, HMI and LMI genomes may be indistinguishable with respect to the distribution of such novel genes, but the increased number of genes in HMI genomes that resolve to well-studied metabolisms would reduce the proportion of known functions in LMI genomes, and thus in metagenomes where they thrive. While testing these hypotheses falls outside the scope of our work, we find the latter hypothesis more likely due to examples in literature that have successfully identified genes that belong to known metabolisms in some of the most obscure organisms via annotation strategies similar to those we have used in our work (Jaffe et al., 2020; Farag et al., 2020).

Taken together, these results (1) demonstrate that the PPCN normalization is an important consideration for investigations of metabolic enrichment in complex microbial communities as a function of microbial diversity, and (2) reveal that the enrichment of HMI populations in an environment offers a high-resolution marker to resolve different levels of environmental stress.

So far, our findings demonstrate an overall, metagenome-level trend of increasing HMI within gut microbial communities as a function of IBD status without considering the individual genomes that contribute to this signal. Since we can measure the extent of metabolic independence as defined in our study based on the completion of a few key metabolic modules for any given genome, we next considered a genome-based approach to further benchmark our findings by investigating whether publicly available microbial genomes that appear to have properties of HMI are more commonly found in individuals diagnosed with IBD.

To identify a set of microbial genomes that are generally associated with the human gut environment, we cast a broad net by surveying the ecology of 19,226 genomes in the Genome Taxonomy Database (GTDB) (Parks et al., 2022) that belonged to three major phyla: Bacteroidetes, Firmicutes, and Proteobacteria, which represent the vast majority of microbial diversity in the human gut environment (Woting and Blaut, 2016; Turnbaugh et al., 2009). As these phyla also include a large number of taxa that primarily occur outside of the human gut, we only kept for downstream analyses those that were detected in at least 2% of the participants of the Human Microbiome Project (HMP) (Human Microbiome Project Consortium, 2012; Appendix 1—figure 8, Methods). Of the final set of 338 reference genomes that passed our filters, 258 (76.3%) resolved to Firmicutes, 60 (17.8%) to Bacteroidetes, and 20 (5.9%) to Proteobacteria. Most of these genomes resolved to families common to the colonic microbiota, such as Lachnospiraceae (30.0%), Ruminococcaceae/Oscillospiraceae (23.1%), and Bacteroidaceae (10.1%) (Arumugam et al., 2011), while 5.9% belonged to poorly studied families with temporary code names (Supplementary file 3a). Finally, we performed a more comprehensive read recruitment analysis on this smaller set of genomes using all deeply sequenced metagenomes from cohorts that included healthy, non-IBD, and IBD samples (Figure 3). This provided us with a quantitative summary of the detection patterns of GTDB genome representatives common to the human gut across our dataset.

We assumed that a given genome had HMI if its average completeness of the 33 HMI-associated metabolic modules was at least 80%, equivalent to a summed metabolic independence score of 26.4 (Methods). Given the number of ways a genome can pass or fail this threshold, this arbitrary cutoff has significant shortcomings, which was demonstrated by the fact that several species in the Bacteroides group were not classified as HMI despite their frequent dominance of the gut microbiome of individuals with IBD (Saitoh et al., 2002; Wexler, 2007; Vineis et al., 2016) (Appendix 1). That said, the genomes that were classified as HMI by this approach were consistently higher in their detection and abundance in IBD samples (Figure 3A). It is likely that there are multiple ways to have HMI which are not fully captured by the 33 IBD-enriched metabolic modules identified in this study. Across all genomes, the mean metabolic independence score was 24.0 (Q1: 19.9, Q3: 25.7). We identified 17.5% (59) of the reference genomes as HMI. HMI genomes were on average substantially larger (3.8 Mbp) than non-HMI genomes (2.9 Mbp) and encoded more genes (3634 vs 2683 genes, respectively), which is in accordance with the reduced metabolic potential of non-HMI populations (Supplementary file 3a). Our read recruitment analysis showed that HMI reference genomes were present in a significantly higher proportion of IBD samples compared to non-HMI genomes (Figure 3B, p < 1e-5, Wilcoxon rank-sum test). Similarly, the fraction of HMI populations was significantly higher within a given IBD sample compared to samples classified as ‘non-IBD’ and those from healthy individuals (Figure 3C, p < 1e-24, Kruskal-Wallis rank-sum test). In contrast, the detection of HMI populations and non-HMI populations was similar in healthy individuals (Figure 3B, p = 0.267, Wilcoxon rank-sum test). The intestinal environment of healthy individuals likely supports both HMI and non-HMI populations, wherein ‘metabolic diversity’ is maintained by metabolic interactions such as cross-feeding. Indeed, loss of cross-feeding interactions in the gut microbiome appears to be associated with a number of human diseases, including IBD (Marcelino et al., 2023). This interpretation is further supported by the fact that the top two HMI-associated modules are required for the synthesis of cobalamin from glutamate. Auxotrophy for cobalamin biosynthesis is common among gut bacteria that rely on cross-feeding for this essential cofactor (Degnan et al., 2014a; Magnúsdóttir et al., 2015; Kelly et al., 2019) (Appendix 1).

Overall, the classification of reference gut genomes as HMI and their enrichment in individuals diagnosed with IBD strongly supports the contribution of HMI to stress resilience of individual microbial populations. We note that survival in a disturbed gut environment will likely require a wide variety of additional functions that are not covered in the list of metabolic modules we consider to determine HMI status – e.g., see Degnan et al., 2014b; Martens et al., 2014; Zong et al., 2020; Feng et al., 2020; Goodman et al., 2009; Powell et al., 2016. Indeed, there may be many ways for a microbe to be metabolically independent, and our strategy likely failed to identify some HMI populations. Nonetheless, these data suggest that HMI serves as a reliable proxy for the identification of microbial populations that are particularly resilient.

Our analysis identified HMI as an emergent property of gut microbial communities associated with individuals diagnosed with IBD. This community-level signal translates to individual microbial populations and provides insights into the microbial ecology of stressed gut environments. HMI-associated metabolic modules were enriched at the community level, and microbial populations encoding these modules were more prevalent in individuals with IBD than in healthy individuals. Furthermore, the copy number of these modules and the proportion of HMI populations reflect the severity of environmental stress and translate to host health states (Appendix 1—figure 5B, Figure 3C). The ecological implications of these observations suggest that HMI may serve as a predictor of general stress in the human gut environment.

So far, efforts to identify IBD using microbial markers have presented classifiers based on (1) taxonomy in pediatric IBD patients (Papa et al., 2012; Gevers et al., 2014), (2) community composition in combination with clinical data (Halfvarson et al., 2017), (3) untargeted metabolomics and/or species-level relative abundance from metagenomes (Franzosa et al., 2019), and (4) k-mer-based sequence variants in metagenomes that can be linked to microbial genomes associated with IBD (Reiter et al., 2022). Performance varied both between and within studies according to the target classes and data types used for training and validation of each classifier (Supplementary file 4a). For those studies reporting accuracy, a maximum accuracy of 77% was achieved based on either metabolite profiles (for prediction of IBD subtype) (Franzosa et al., 2019) or k-mer-based sequence variants (for differentiating between IBD and non-IBD samples) (Reiter et al., 2022). Some studies reported performance as area under the receiver operating characteristic curve (AUROCC), a typical measure of classifier utility describing both sensitivity (ability to correctly identify the disease) and specificity (ability to correctly identify absence of disease). For this metric the highest value was 0.92, achieved by Franzosa et al., 2019, when using metabolite profiles, with or without species abundance data, for classifying IBD vs non-IBD. However, the majority of these classifiers were trained and tested on a relatively small group of individuals that all come from the same region, i.e., clinical studies confined to a specific hospital. Though some had high performance, they either relied on data that are inaccessible to most laboratories and clinics considering that untargeted metabolomics analyses are difficult to reproduce (Koek et al., 2011; Lin et al., 2020), or they required complex k-mer-based models without the resolution to differentiate gradients in host health (Reiter et al., 2022). These classifiers thus have limited translational potential across global clinical settings and do not provide an ecological framework to explain the observed shifts in community composition and activity. For practical use as a diagnostic tool, a microbiome-based classifier for IBD should rely on an ecologically meaningful, easy to measure, and high-level signal that is robust to host variables like lifestyle, geographical location, and ethnicity. HMI could potentially fill this gap as a metric related to the ecological filtering that defines microbial community changes in the IBD gut microbiome.

We trained a logistic regression classifier to explore the applicability of HMI as a noninvasive diagnostic tool for IBD. The classifier’s predictors were the PPCNs of IBD-enriched metabolic modules in a given metagenome. Across the 330 deeply sequenced IBD and healthy samples included in this analysis, the classifier had high sensitivity and specificity (Figure 4). It correctly identified (on average) 76.8% of samples from individuals diagnosed with IBD and 89.5% of samples representing healthy individuals, for an overall accuracy of 85.6% and an average AUROCC of 0.832 (Supplementary file 4c). Our model outperforms (Gevers et al., 2014; Halfvarson et al., 2017; Reiter et al., 2022) or has comparable performance to Franzosa et al., 2019; Papa et al., 2012 the previous attempts to classify IBD from fecal samples in more restrictively defined cohorts. It also has the advantage of being a simple model, utilizing a relatively low number of features compared to the other classifiers. Determining whether such a model has broader utility as a diagnostic tool requires further research and validation; however, these results demonstrate the potential of HMI as an accessible diagnostic marker of IBD. Due to the lack of time-series studies that include individuals in the pre-diagnosis phase of IBD development, we cannot test the applicability of HMI to predict IBD onset (Lloyd-Price et al., 2019).

Yet, the gradient of metabolic independence reflected by per-population module copy number and the relative increase in the number of HMI populations detected in non-IBD samples (Appendix 1—figure 5B, Figure 3C) suggests that the degree of HMI in the gut microbiome may be indicative of general gut stress, such as the stress induced by antibiotic use. Antibiotics can cause long-lasting perturbations of the gut microbiome – including reduced diversity, emergence of opportunistic pathogens, increased microbial load, and development of highly resistant strains – with potential implications for host health (Ramirez et al., 2020). We applied our metabolism classifier to a metagenomic dataset that reflects the changes in the microbiome of healthy people before, during, and up to 6 months following a 4-day antibiotic treatment (Palleja et al., 2018). The resulting pattern of sample classification corresponds to the posttreatment decline and subsequent recovery of species richness documented in the study by Palleja et al., 2018.

All pretreatment samples were classified as ‘healthy’ followed by a decline in the proportion of ‘healthy’ samples to a minimum 8 days posttreatment, and a gradual increase until 180 days posttreatment, when over 90% of samples were classified as ‘healthy’ (Figure 5, Supplementary file 4b). In other words, the increase in the HMI metric serves as an indicator of stress in the gut microbiome, regardless of whether that stress arises from the IBD condition or the application of antibiotics. These observations support the role of HMI as an ecological driver of microbial resilience during gut stress caused by a variety of environmental perturbations and demonstrate its diagnostic power in reflecting gut microbiome state.

Overall, our observations that stem from the analysis of hundreds of reference genomes, deeply sequenced gut metagenomes, and multiple categories of human disease states suggest that environmental stress in the human gut – whether it is associated with inflammation, cancer, or antibiotic use – promotes the survival and relative expansion of microbial populations with HMI. These results establish HMI as a high-level metric to classify gradients of human health states through the gut microbiota that is robust to ethnic, geographical, or lifestyle factors. Taken together with recent evidence that models altered ecological relationships within gut microbiomes under stress due to disrupted metabolic cross-feeding (Heinken et al., 2021; Marcelino et al., 2023), our data support the hypothesis that the reduction in microbial diversity, or more generally ‘dysbiosis’, is an emergent property of microbial communities responding to disease pathogenesis or other external factors such as antibiotic use that disrupt the gut microbial ecosystem. This paradigm depicts microbes as bystanders by default, rather than perpetrators or drivers of noncommunicable human diseases, and provides an ecological framework to explain the frequently observed reduction in microbial diversity associated with IBD and other noncommunicable human diseases and disorders.

We developed a new program ‘anvi-estimate-metabolism’ (https://anvio.org/m/anvi-estimate-metabolism), which uses gene annotations to estimate ‘completeness’ and ‘copy number’ of metabolic modules that are defined in terms of enzyme accession numbers. By default, this tool works on metabolic modules from the KEGG MODULE database (Kanehisa et al., 2012; Kanehisa et al., 2023) which are defined by KEGG KOfams (Aramaki et al., 2020), but user-defined modules based on a variety of functional annotation sources are also accepted as input. Completeness estimates describe the percentage of steps (typically, enzymatic reactions) in a given metabolic module that are encoded in a genome or a metagenome. Likewise, copy number summarizes the number of distinct sets of enzyme annotations that collectively encode the complete module. This program offers two strategies for estimating metabolic potential: a ‘stepwise’ strategy with equivalent treatment for alternative enzymes – i.e., enzymes that can catalyze the same reaction in a given metabolic module – and a ‘pathwise’ strategy that accounts for all possible variations of the module. Appendix 1 file includes more information on these two strategies and the completeness/copy number calculations. For the analysis of metagenomes, we used stepwise copy number of KEGG modules. Briefly, the calculation of stepwise copy number is done as follows: the copy number of each step in a module (typically, one chemical reaction or conversion) is individually evaluated by translating the step definition into an arithmetic expression that summarizes the number of annotations for each required enzyme. In cases where multiple enzymes or an enzyme complex are needed to catalyze the reaction, we take the minimum number of annotations across these components. In cases where there are alternative enzymes that can each catalyze the reaction individually, we sum the number of annotations for each alternative. Once the copy number of each step is computed, we then calculate the copy number of the entire module by taking the minimum copy number across all the individual steps. The use of minimums results in a conservative estimate of module copy number such that only copies of the module with all enzymes present are counted. For the analysis of genomes, we calculated the stepwise completeness of KEGG modules. This calculation is similar to the one described above for copy number, except that the step definition is translated into a Boolean expression that, once evaluated, indicates the presence or absence of each step in the module. Then, the completeness of the modules is computed as the proportion of present steps in the module.

We acquired publicly available gut metagenomes from 13 different studies (Le Chatelier et al., 2013; Feng et al., 2015; Franzosa et al., 2019; Lloyd-Price et al., 2019; Qin et al., 2012; Quince et al., 2015; Rampelli et al., 2015; Raymond et al., 2016; Schirmer et al., 2018; Vineis et al., 2016; University of Sydney, 2022; Wen et al., 2017; Xie et al., 2016). The studies were chosen based on the following criteria: (1) they included shotgun metagenomes of fecal matter (primarily stool, but some ileal pouch luminal aspirate samples (Vineis et al., 2016) are also included); (2) they sampled from people living in industrialized countries in the case where a study (Rampelli et al., 2015) included samples from hunter-gatherer populations, only the samples from industrialized areas were included in our analysis; (3) they included samples from people with IBD and/or they included samples from people without gastrointestinal (GI) disease or inflammation; and (4) clear metadata differentiating between case and control samples was available. A full description of the studies and samples can be found in Supplementary file 1a–c. We grouped samples according to the health status of the sample donor. Briefly, the ‘IBD’ group of samples includes those from people diagnosed with Crohn’s disease, ulcerative colitis (UC), or pouchitis. The ‘non-IBD’ group contains non-IBD controls, which includes both healthy people presenting for routine cancer screenings and people with benign or nonspecific symptoms that are not clinically diagnosed with IBD. Colorectal cancer patients from Feng et al., 2015, were also put into the ‘non-IBD’ group on the basis that tumors in the GI tract may arise from local inflammation (Kraus and Arber, 2009) and represent a source of gut stress without an accompanying diagnosis of IBD. Finally, the ‘HEALTHY’ group contains samples from people without GI-related diseases or inflammation. Note that only control or pretreatment samples were taken from the studies covering type 2 diabetes (Qin et al., 2012), ankylosing spondylitis (Wen et al., 2017), antibiotic treatment (Raymond et al., 2016), and dietary intervention University of Sydney, 2022; these controls were all assigned to the ‘HEALTHY’ group. At least one study (Le Chatelier et al., 2013) included samples from obese people, and these were also included in the ‘HEALTHY’ group.

We made single assemblies of most gut metagenomes using the anvi’o metagenomics workflow implemented in the program ‘anvi-run-workflow’ (Shaiber et al., 2020). This workflow uses Snakemake (Köster and Rahmann, 2012), and a tutorial is available at the URL https://merenlab.org/anvio-workflows/. Briefly, the workflow includes quality filtering using ‘iu-filter-quality-minoche’ (Eren et al., 2013); assembly with IDBA-UD (Peng et al., 2012) (using a minimum contig length of 1000); gene calling with Prodigal v2.6.3 (Hyatt et al., 2010); tRNA identification with tRNAscan-SE v2.0.7 (Chan and Lowe, 2019); as well as annotations of ribosomal RNAs (Seemann, 2018), single-copy core genes (SCGs) for Bacteria, Archaea, and Protista, KEGG KOfams (Aramaki et al., 2020), NCBI Clusters of Orthologous Group (COGs) (Galperin et al., 2021), and Pfam protein families (release 33.1, Mistry et al., 2021). The aforementioned annotations relied on HMMER v3.3.2 (Eddy, 2011) as well as Diamond v0.9.14.115 (Buchfink et al., 2015). As part of this workflow, all single assemblies were converted into anvi’o contigs databases. Samples from Vineis et al., 2016, were processed differently because they contained merged reads rather than individual paired-end reads: no further quality filtering was run on these samples, we assembled them individually using MEGAHIT (Li et al., 2015), and we used the anvi’o contigs workflow to perform all subsequent steps described for the metagenomics workflow above. Note that we used a version of KEGG downloaded in December 2020 (for reproducibility, the hash of the KEGG snapshot available via ‘anvi-setup-kegg-kofams’ is 45b7cc2e4fdc). Additionally, the annotation program ‘anvi-run-kegg-kofams’ includes a heuristic for annotating hits with bitscores that are just below the KEGG-defined threshold (Kananen et al., 2025), which is further described at https://anvio.org/m/anvi-run-kegg-kofams/.

We downloaded all reference genomes for ‘species’ cluster representatives from the GTDB, release 95.0 (Parks et al., 2018; Parks et al., 2020), and processed them with the same anvi’o gene annotation workflow described above.

We used SCG sets belonging to each domain of microbial life (Bacteria, Archaea, Protista) to estimate the number of populations from each domain present in a given metagenomic sample. For each domain, we calculated the number of populations by taking the mode of the number of copies of each SCG in the set. We then summed the number of populations from each domain to get a total number of microbial populations within each sample. We accomplished this using SCG annotations provided by ‘anvi-run-hmms’ (which was run during metagenome processing) and a custom script relying on the anvi’o class ‘NumGenomesEstimator’ (see reproducible workflow).

We observed that, at lower sequencing depths, our estimates for the number of populations in a metagenomic sample were moderately correlated with sequencing depth (Appendix 1—figure 1, R > 0.5). These estimates rely on having accurate counts of SCGs, so we hypothesized that lower-depth samples were systematically missing SCGs, especially from populations with lower abundance. Since accurate population number estimates are critical for proper normalization of module copy numbers, keeping these lower-depth samples would have introduced a bias into our metabolism analyses. To address this, we removed samples with low sequencing depth from downstream analyses using a sequencing depth threshold of 25 million reads, such that the remaining samples exhibited a weaker correlation (R < 0.5) between sequencing depth and number of estimated populations. We kept samples for which both the R1 file and the R2 file contained at least 25 million reads (and for the Vineis et al., 2016, dataset, we kept samples containing at least 25 million merged reads). This produced our final sample set of 408 metagenomes.

We ran ‘anvi-estimate-metabolism’, in genome mode and with the ‘--add-copy-number’ flag, on each individual metagenome assembly to compute stepwise copy numbers for KEGG modules from the combined gene annotations of all populations present in the sample. We then divided these copy numbers by the number of estimated populations within each sample to obtain a PPCN for each module.

We used a one-sided Mann-Whitney-Wilcoxon test with an FDR-adjusted p-value threshold of p ≤ 2e-10 on the per-sample PPCN values for each module individually to identify the modules that were most significantly enriched in the IBD sample group compared to the healthy group. We calculated the median PPCN of each metabolic module in the IBD samples, and again in the healthy samples. After filtering for p-values ≤ 2e-10, we also applied a minimum effect size threshold based on the median PPCN in each group (M_IBD - M_Healthy ≥ 0.12) – this threshold was calculated by taking the mean effect size over all modules that passed the p-value threshold. The set originally contained 34 modules that passed both thresholds, but we removed one redundant module (M00006) which represents the first half of another module in the set (M00004).

We used a one-sided Fisher’s exact test (also known as hypergeometric test, see e.g., Boyle et al., 2004) for testing the independence between the metabolic modules identified to be IBD-enriched (i.e. using the methods described in ‘Selection of IBD-enriched modules’) and functionality (i.e. modules annotated to be involved in biosynthesis).

Because the 33 IBD-enriched modules were selected using PPCNs of healthy and IBD samples, statistical tests comparing PPCN distributions for these modules need to be interpreted with care, because the hypotheses were selected and tested on the same dataset (Fithian et al., 2014). Therefore, to assess the statistical validity of the identified IBD-enriched modules, we performed the following repeated sample-split analysis: we first randomly split the IBD and healthy samples into the equal-sized training and validation sets. We select IBD-enriched modules in the training set using the Mann-Whitney-Wilcoxon test, and then compute the p-values on the validation set. We repeat this sample split analysis 1000 times with an FDR-adjusted p-value threshold of 1e-10 on the first split; most identified modules (89.4%; 95% CI: [87.5%, 91.3%]) on the training sets remain significant at a slightly less stringent threshold (1e-8) on the validation sets. This indicates that the approach we used to identify IBD-enriched modules yields stable and statistically significant results on this dataset.

We trained logistic regression models to classify samples as ‘IBD’ or ‘healthy’ using PPCNs of IBD-enriched modules as features. We ran a 25-fold cross-validation pipeline on the set of 330 healthy and IBD metagenomes in our analysis, using an 80% train – 20% test random split of the data in each fold. The pipeline included selection of IBD-enriched modules within the training samples using the same strategy as described above, followed by training and testing of a logistic regression model as implemented in the ‘sklearn’ Python package. We set the ‘penalty’ parameter of the model to ‘None’ and the ‘max_iter’ parameter to 20,000 iterations, and we used the same random state in each fold to ensure changes in performance only come from differences in the training data rather than differences in model initialization. To summarize the overall performance of the classifier, we took the mean (over all folds) of each performance metric.

We trained a final classifier using the 33 IBD-enriched modules selected earlier from the entire set of 330 healthy and IBD metagenomes. We then applied this classifier to the metagenomic samples from Palleja et al., 2018, which we processed in the same way as the other samples in our analysis (including removal of samples with low sequencing depth and calculation of PPCNs of KEGG modules for use as input features to the classifier model).

We took 19,226 representative genomes from the GTDB species clusters belonging to the phyla Firmicutes, Bacteroidetes, and Proteobacteria, which are most common in the human gut microbiome (Woting and Blaut, 2016). To evaluate which of these genomes might represent gut microbes in a computationally tractable manner, we ran the anvi’o ‘EcoPhylo’ workflow (https://anvio.org/m/ecophylo) to contextualize these populations within 150 healthy gut metagenomes from the HMP (Human Microbiome Project Consortium, 2012). Briefly, the EcoPhylo workflow (1) recovers sequences of a gene family of interest from each genome and metagenomic sample in the analysis, (2) clusters resulting sequences and picks representative sequences using mmseqs2 (Steinegger and Söding, 2017), and (3) uses the representative sequences to rapidly summarize the distribution of each population cluster across the metagenomic samples through metagenomic read recruitment analyses. Here, we used the Ribosomal Protein S6 as our gene of interest, since it was the most frequently assembled SCG in our set of GTDB genomes. We clustered the Ribosomal Protein S6 sequences from GTDB genomes at 94% nucleotide identity.

To identify genomes that were likely to represent gut microbes, we selected genomes whose ribosomal protein S6 belonged to a gene cluster where at least 50% of the representative sequence was covered (i.e. detection ≥ 0.5×) in more than 10% of samples (i.e. n > 15). There are 100 distinct individuals represented in the 150 HMP gut metagenomes – 56 of which were sampled just once and 46 of which were sampled at 2 or 3 time points – so this threshold is equivalent to detecting the genome in 5–15% of individuals. From this selection we obtained a set of 836 genomes; however, these were not exclusively gut microbes, as some non-gut populations have similar ribosomal protein S6 sequences to gut microbes and can therefore pass this selection step. To eliminate these, we mapped our set of 330 healthy and IBD metagenomes to the 836 genomes using the anvi’o metagenomics workflow and extracted genomes whose entire sequence was at least 50% covered (i.e. detection ≥ 0.5×) in over 2% (n > 6) of these samples. Our final set of 338 genomes was used in downstream analysis.

To create the phylogeny, we identified the following ribosomal proteins that were annotated in at least 90% (n = 304) of the genomes: Ribosomal_S6, Ribosomal_S16, Ribosomal_L19, Ribosomal_L27, Ribosomal_S15, Ribosomal_S20p, Ribosomal_L13, Ribosomal_L21p, Ribosomal_L20, and Ribosomal_L9_C. We used ‘anvi-get-sequences-for-hmm-hits’ to extract the amino acid sequences for these genes, align the sequences using MUSCLE v3.8.1551 (Edgar, 2004), and concatenate the alignments. We used trimAl v1.4.rev15 (Capella-Gutiérrez et al., 2009) to remove any positions containing more than 50% of gap characters from the final alignment. Finally, we built the tree with IQtree v2.2.0.3 (Minh et al., 2020), using the WAG model and running 1000 bootstraps.

We estimated metabolic potential for each genome with ‘anvi-estimate-metabolism’ (in genome mode) to get stepwise completeness scores for each KEGG module, and then we used the script ‘anvi-script-estimate-metabolic-independence’ to give each genome a metabolic independence score based on completeness of the 33 IBD-enriched modules. Briefly, the latter script calculates the score by summing the completeness scores of each module of interest. Genomes were classified as having HMI if their score was greater than or equal to 26.4. We calculated this threshold by requiring these 33 modules to be, on average, at least 80% complete in a given genome.

We mapped the gut metagenomes from the healthy, non-IBD, and IBD groups to each genome using the anvi’o metagenomics workflow in reference mode. We used ‘anvi-summarize’ to obtain a matrix of genome detection across all samples. We summarized this data as follows: for each genome, we computed the proportion of samples in each group in which at least 50% of the genome sequence was covered by at least 1 read (≥ 50% detection). For each sample, we calculated the proportion of detected genomes that were classified as HMI. We also computed the percent abundance of each genome in each sample by dividing the number of reads mapping to that genome by the total number of reads in the sample.

We used ggplot2 (Wickham, 2016) to generate most of the initial data visualizations. The phylogeny and heatmap in Figure 3 were generated by the anvi’o interactive interface and the ROC curves in Figure 4 were generated using the pyplot package of matplotlib (Hunter, 2007). These visualizations were refined for publication using Inkscape, an open-source graphical editing software that is available at https://inkscape.org/.

Supplementary Table files and our Appendix files can be accessed at https://doi.org/10.6084/m9.figshare.22679080.