The landscape of biology foundation models: do they actually beat the old specialized baselines?

Evidence-first notes on bioscience and deep tech, at the edge of the lab and the market. Information only — not investment advice.

The 30-second version

• What. Protein, genome and cell “foundation models” (bio-FMs) are usually sold with headlines like “simulating millions of years of evolution.” The firm’s one question is narrower: on a specific task, do they actually beat the old domain-specialized baseline (a docking program, an alignment-based conservation score, a linear regression)? The key finding is that the answer differs by axis: protein structure “wins, but leans on memorization”; genome variants “partly win, but with a short-range blind spot”; cell perturbation “does not yet beat even a simple linear baseline.”
• So what. “Bio-FMs beat the baselines” collapses into a misleading one-liner. Each quantitative superiority claim needs three angles kept apart — vendor paper vs. independent benchmark vs. critique. In leakage-controlled independent benchmarking (FoldBench, Nat Commun 2025), AlphaFold3 does lead protein-ligand (64.9% success), but accuracy rises with ligand similarity (a memorization signal) and collapses on nucleic acids (DNA LDDT 0.53 / RNA 0.61 vs. 0.88 for protein monomers). For cells, Nature Methods 2025 is titled to the effect that deep-learning perturbation prediction still does not beat a simple linear baseline.
• Now what. Treat “benchmark win” as separate from “generalization to a novel target.” Vendor figures such as STATE’s “+50% / 2x DEG” and ESM3’s “1 trillion teraflops” phrasing remain vendor/unverified. Note one correction of the correction: Evo2 was peer-reviewed and published in Nature (652:1349–1361, 2026-03; PMID 41781614) — initially recorded conservatively as a bioRxiv preprint, then reversed on primary evidence. Verification status is PARTIAL; per-axis 3-vote adversarial checks are finalized in the sub-parts.

[demo-gap note] A leaderboard number is not real-world use. AF3’s lead is measured on curated structure sets and rises with training-set ligand similarity; ESM3’s de novo esmGFP is a generation claim whose wet-lab validation is a Part 1 question; STATE’s virtual-cell figures are vendor/unverified research-demo results, not an independent readout. “Beats the baseline” is claimed on benchmarks, not yet demonstrated as generalization to new targets.

The five-minute read

The one question: does it actually beat the old baseline?

Bio-FMs across protein, genome and cell get compressed into headlines about “predicting every molecule of life.” One layer in, the useful question is whether, on a defined task, the model beats a long-standing specialized baseline. The firm has not previously tested this head-on. The core landscape finding is that the three axes give different answers, so any single-sentence verdict misleads.

Three axes, three different answers

Protein — “wins, but leans on memorization.” AlphaFold3 (Google DeepMind + Isomorphic Labs, Nature 2024) extended to all-atom complexes with ligands, nucleic acids and ions; vendors report ~84% PoseBusters protein-ligand top-1. In the leakage-controlled independent FoldBench, AF3 leads protein-ligand at 64.9% (about 10 points over Boltz-1) — but accuracy climbs with ligand similarity, antibody-antigen sits at 47.9%, and nucleic-acid LDDT drops sharply. A separate critique (Inductive Bio) argues the AF3 paper’s out-of-box Vina baseline was not the strongest classical method, and a “100-line strong baseline” beats blind AF3 by +4.2% [blog/med, single source, needs replication]. Data leakage is real: on a decontaminated set the optimal IDR pLDDT threshold shifts 0.89 to 0.69 (Brief Bioinform 2025).

Genome — “partly wins, with a short-range blind spot.” Evo2 (Arc Institute + NVIDIA; Nature 2026, 7B/40B params, 9.3 trillion nucleotides, 1M bp context) claims zero-shot noncoding variant-effect SOTA, but coding SNV trails conservation-specialized baselines (GPN-MSA, PhyloP) and ESM-1b. An independent preprint reports a systematic short-range blind spot, and a deployment benchmark finds 40B barely beats 7B (variant-classification AUC 0.86–0.87), weakening the “bigger is better” story.

Cell — “does not yet beat a linear baseline.” Nature Methods 2025 compares scGPT/scFoundation against GEARS/CPA and a deliberately simple baseline and finds deep learning does not yet win on gene-perturbation effect prediction. Arc’s STATE virtual-cell model reports vendor figures (+50% perturbation discrimination, 2x true-DEG identification) that remain unverified.

[diagram: does the bio-FM beat the specialized baseline? — by axis]

  Axis            Flagship FM      Beats baseline?        Real bottleneck
  ─────────────────────────────────────────────────────────────────────
  Protein struct  AF3 / ESM3       PARTLY YES  ██████░░   ligand-similarity
                  (FoldBench lead)                        memorization; NA drop
  Genome variant  Evo2             PARTLY YES  ████░░░░   short-range blind
                  (noncoding only)                        spot; 40B ~= 7B
  Cell perturb.   STATE / scGPT    NO (not yet) ██░░░░░░   below linear baseline;
                  (100M+ cells)                           attention = co-expr
  ─────────────────────────────────────────────────────────────────────
  through-line:  benchmark win  !=  generalization to a new target

Deep dive

1. Background — porting the “real bottleneck” lens to AI x Bio

This Part 0 transplants the computing series’ “the real bottleneck next to the headline” lens onto biology foundation models. The headline is always evolutionary scale or all-atom coverage; the firm’s question is whether a bio-FM beats an old, cheap, specialized baseline on a concrete task. Verification status is PARTIAL — independent benchmarks and primary papers were cross-checked, but the full 3-vote adversarial pass is finalized in the sub-parts.

2. What this landscape newly establishes

Core answer: there is no single verdict. The three axes diverge, and every “superiority” claim must separate vendor paper, independent benchmark and critique.

• Protein — AF3 genuinely leads [independent/high]: in leakage-controlled FoldBench (Nat Commun 2025; curated to 1,522 structures after AF3’s 2023-01-13 cutoff), AF3 reaches 64.9% protein-ligand success, about 10 points over Boltz-1.
• …but with a generalization gap [independent/high]: accuracy rises with ligand similarity (memorization signal), antibody-antigen is 47.9%, and nucleic-acid LDDT falls to DNA 0.53 / RNA 0.61 vs. protein-monomer 0.88.
• …and the baseline may have been unfair [blog/med, single source]: Inductive Bio argues out-of-box Vina was not the strongest classical method; a 100-line strong baseline (ensemble conformation + Gnina rescoring) beats blind AF3 by +4.2% (and is −7.1% vs. pocket-specified AF3). Replication needed.
• …and leakage is real [independent/high]: on the decontaminated BETA set the optimal IDR pLDDT threshold shifts 0.89 to 0.69 (KS-test P<0.0001); the literature “commonly ignores training-set contamination” (Brief Bioinform 2025, bbaf104).
• Genome — noncoding claim vs. blind spot: Evo2’s noncoding-SOTA claim is vendor/med (ClinVar, SpliceVarDB, BRCA1/2). An independent preprint (“Biological Blind Spots in Evo2 Variant-Effect Prediction,” bioRxiv 2026-03) reports a systematic short-range blind spot and sensitivity to contextually-neutral features [independent/med]. A deployment benchmark (bioRxiv 2025-09) finds 40B barely beats 7B (AUC 0.86–0.87) [independent/med].
• Cell — the strongest skeptic signal [primary/high]: Nature Methods 2025 (s41592-025-02772-6) reports that deep-learning gene-perturbation prediction does not yet beat a simple linear baseline; single-cell FM attention captures co-expression rather than intrinsic regulatory signal (arXiv).

3. Strengths and limits of the methodology

Strengths: each superiority claim is split into vendor paper vs. independent benchmark vs. critique, with evidence tiers attached, and leakage-controlled benchmarks (FoldBench, decontaminated BETA) are foregrounded. Limits: much of what is confirmed is “the accuracy of what is claimed,” and several key figures remain vendor/unverified (STATE’s +50% and 2x DEG, ESM3’s “1 trillion teraflops” phrasing, Isomorphic’s clinical-entry timing as forward-looking). The Inductive Bio strong-baseline result is a single blog source needing independent replication, and two of the sharpest critiques (Evo2 blind spot, 40B scaling) are preprints. Data leakage and evaluation-set contamination cut across all three axes, so leaderboard positions must be read against contamination controls.

4. Neighbouring domains

The cross-domain hook is methodological rather than forced: the same “headline metric is not usefulness” structure recurs from the computing (post-CMOS) series, and the leakage-control problem here mirrors benchmark-contamination debates in general ML evaluation. Whether these bio-FM methods transfer to CKM (cardio-renal-metabolic) and drug discovery is deferred to Part 5; the through-line recorded now is only the structural similarity, not a claimed clinical payoff.

5. Commercialization and market context (actors, platforms)

Clinical readouts are out of scope here; this is the model/platform angle only.

Deployment-readiness spectrum: research demo (STATE and some cell FMs) to open weights (ESM3-open, Evo2, Geneformer) to API/product (AF3 server, Isomorphic’s internal engine, BioNeMo). Open weights and closed commercial licensing (the AF3 server carries commercial-use restrictions) split by axis. No single-winner conclusion is asserted; vendor commercial figures are attributed.

6. The skeptic’s bottom line

• No axis is “verified-clean.” None has yet shown a clean win on a leakage-controlled prospective benchmark. Fan-out tally: 14 confirmed / 2 corrected (refuted) / 3 unverified-vendor.
• Proceed-with-caveats — protein and genome: there is evidence they beat the baseline, but the memorization (protein) and short-range blind-spot (genome) caveats are mandatory. The Tier-1 headline must carry the demo-gap: a benchmark win is not generalization to a new target.
• Hold — cell: claims that “the virtual cell beats the baseline” are on hold and escalated while the Nature Methods 2025 refutation stands.
• Corrections logged: (1) the Inductive Bio strong baseline refutes “out-of-box Vina was a fair baseline”; (2) Evo2’s venue was reversed from “bioRxiv preprint” to Nature 652:1349–1361 (2026-03, PMID 41781614) on primary evidence — a correction of the correction, per evidence-first.

7. What to watch (falsifiable predictions)

1. If leakage-controlled prospective benchmarks become standard, AF3-style protein-ligand leads should shrink (the 64.9% edge falls materially once ligand similarity is controlled). Testable via FoldBench follow-ups and strong-baseline replication.
2. Over the next 12–18 months, cell-perturbation FMs will not significantly beat a linear baseline across tasks in a published benchmark. Falsifiable by Virtual Cell Challenge public results.
3. Evo2 40B will show no significant cost-adjusted benefit over 7B on real-use tasks (AUC gap stays <0.02). Testable via deployment-benchmark follow-ups.

References

• Abramson, Josh, et al. 2024. “Accurate structure prediction of biomolecular interactions with AlphaFold3.” Nature. https://www.nature.com/articles/s41586-024-07487-w
• Hayes, Thomas, et al. 2025. “Simulating 500 million years of evolution with a language model (ESM3).” Science. https://www.science.org/doi/10.1126/science.ads0018
• Brixi, Garyk, et al. 2026. “Genome modeling and design across all domains of life with Evo2.” Nature 652:1349–1361 (2026-03; PMID 41781614; original preprint bioRxiv 2025.02.18.638918). https://www.nature.com/articles/s41586-026-10176-5
• 2025. “Deep learning does not (yet) outperform a simple linear baseline for perturbation effect prediction.” Nature Methods. https://www.nature.com/articles/s41592-025-02772-6
• 2025. “FoldBench: a leakage-controlled benchmark for biomolecular structure prediction.” Nature Communications. https://www.nature.com/articles/s41467-025-67127-3
• Inductive Bio. “A strong baseline for AlphaFold3 docking.” https://www.inductive.bio/blog/strong-baseline-for-alphafold-3-docking
• 2025. “Data leakage in AlphaFold benchmarking.” Briefings in Bioinformatics 26(2):bbaf104. https://academic.oup.com/bib/article/26/2/bbaf104/8069415
• Arc Institute. “STATE: a virtual cell model.” https://arcinstitute.org/news/virtual-cell-model-state
• Isomorphic Labs $600M raise (2025-04). PR Newswire / TechCrunch, 2025-03-31.

Source knowledge asset: knowledge-base/deep-dives/bio-foundation-models/part0-landscape.md (generated 2026-07-12, verification PARTIAL — 14 confirmed / 2 corrected / 3 unverified-vendor). This draft inherits the figures and source attributions of the original part and creates no new figures, numbers or sources.

Disclosure

This post is for information only and is not investment advice. The author holds no position in, and no financial interest in, the companies mentioned (NVIDIA, Recursion RXRX, Alphabet, and others).

COI note: This document describes listed companies (Alphabet/DeepMind, NVIDIA, Recursion RXRX) and private or non-profit actors (Isomorphic Labs, EvolutionaryScale, Arc Institute, CZI) in a factual, neutral technology/platform context. There is no buy/sell implication. Vendor quantitative claims (for example STATE’s “+50% / 2x DEG,” ESM3’s “1 trillion teraflops” phrasing, Isomorphic’s clinical-entry timing) are attributed as “vendor/author/preprint claims” and are not independently verified.