Uthando Nethemba

Genome foundation models and the “noncoding SOTA” claim: what Evo2 really beats, and where it does not

—

by

itembalabantu
in

Evidence-first notes on bioscience and deep tech, at the edge of the lab and the market. Information only — not investment advice.

The 30-second version

  • What. Evo2 (Arc Institute · NVIDIA · Stanford · UC Berkeley · UCSF; 7B / 40B parameters, ~9.3 trillion nucleotides of training data) claims a new state of the art (SOTA) in zero-shot noncoding variant effect prediction. The claim is real — but the paper itself scopes it as SOTA among unsupervised (zero-shot) methods, and Evo2 is now peer-reviewed in Nature (652:1349–1361, 2026-03; PMID 41781614).
  • So what. Drop the bracket and it misleads. On coding single-nucleotide variants (SNVs) Evo2 trails conservation-based specialist baselines (GPN-MSA, PhyloP) and the protein language model ESM-1b; on noncoding SNVs it trails supervised specialist models. An independent preprint also reports a systematic short-range blind spot and a severity inversion (accuracy falls as variants get clinically more severe) — the wrong direction for a pathogenicity predictor.
  • Now what. Evo2 clearly leads only on zero-shot scoring of noncoding non-SNVs and splice variants. A separate deployment benchmark finds 40B ≈ 7B (little gain from scale, and 40B collapses to AUC 0.48 on non-FP8 hardware). Treat Evo2 as a research-grade open-weight model (TRL 3–4), not a clinical pathogenicity caller. The metric to watch is leakage-controlled prospective benchmarks that include supervised baselines and severity-stratified subsets.

[demo-gap note] Evo2’s headline is a benchmark result, not a clinical demonstration. The “noncoding SOTA” is largely “SOTA among zero-shot methods”; older alignment/conservation baselines still win on coding variants, and independent work reports the model performing worse on the most severe (most clinically relevant) variants. Standalone zero-shot clinical pathogenicity use is not supported at this stage.

The five-minute read

A real SOTA — with a bracket

Genome foundation models (FMs) are language models trained on raw DNA instead of text. The central question for the field is the same one that dogs protein FMs: do they actually beat purpose-built specialist baselines? Evo2’s answer is a conditional yes. The paper reports genuine, non-fabricated gains — but it scopes the “SOTA” to zero-shot / unsupervised methods. The moment that bracket is removed, the claim tilts from true to misleading: supervised and conservation-based specialists still lead on the variant classes that dominate clinical interpretation.

What Evo2 wins, and what it loses

By the paper’s own stratified results, Evo2 (40B) is competitive on coding SNVs but trails ESM-1b, GPN-MSA and some PhyloP settings there. On noncoding SNVs it is first among unsupervised methods but trails supervised specialists. Where it leads outright is noncoding non-SNVs (structural / insertion-deletion classes) and splice variants scored zero-shot — precisely the regions where multiple-sequence alignment is hard. In other words, the old alignment- and conservation-based baselines (GPN-MSA is alignment-based; PhyloP is phylogenetic conservation) still beat the genome LM on coding variants. That is the honest through-line: headline SOTA ≠ specialist-beating across the board.

[diagram: where the genome LM leads vs. trails, by variant class]

  Variant class            Evo2 (40B) position         Still loses to
  ─────────────────────────────────────────────────────────────────────
  coding SNV               competitive                 ESM-1b · GPN-MSA · PhyloP
  noncoding SNV            #1 among UNSUPERVISED  ◄──── supervised specialists
  noncoding non-SNV        #1 overall  ███████████     —
  splice (SpliceVarDB)     best zero-shot  █████████   (separate supervised)
  ─────────────────────────────────────────────────────────────────────
  "noncoding SOTA"  ≈  "SOTA among zero-shot methods"   (bracket is load-bearing)
The bracket is load-bearing. Evo2’s clear wins are confined to zero-shot scoring of noncoding non-SNVs and splice variants. On coding variants, conservation/alignment baselines (GPN-MSA, PhyloP) and the protein LM ESM-1b still lead. Exact AUROC values are not reproduced here — the Nature full text was gated at verification time, so the ordering (not absolute numbers) is what is attributed.

Deep dive

1. Background — the specialist-baseline test

This is Part 2 of the bio-foundation-models series, drilling into the genome axis. Part 0 left it as an open contest: a vendor “noncoding variant SOTA” claim versus an independent “short-range blind spot” critique. The question that runs through the whole series is whether a general FM actually beats purpose-built baselines. On the genome axis the answer turns out to be a scoped, conditional yes — true only once you name the regime (unsupervised) and the variant type (non-SNV, splice).

2. What this deep dive newly establishes

Core answer: the noncoding SOTA is real but self-limited to zero-shot methods; coding and supervised regimes still belong to specialist baselines; and independent work raises a short-range blind spot plus a severity inversion.

  • Peer-review status corrected forward — CONFIRMED (PubMed): a provisional Part 0 correction (“Evo2 not in Nature“) is itself reversed by a primary source. Evo2 is published in Nature 652:1349–1361 (online 2026-03-04), DOI 10.1038/s41586-026-10176-5, PMID 41781614. The bioRxiv preprint (2025-02-18) and the journal paper are the normal preprint → peer-review path, not a contradiction. This raises the evidence tier of the headline claim to peer-reviewed/high; the independent critiques below remain preprints (not peer-reviewed).
  • Model and scale — CONFIRMED (Nature abstract · NVIDIA · Hugging Face): 7B / 40B parameters, StripedHyena 2 architecture, ~9.3 trillion nucleotides across ~128,000 genomes (eukaryotic, prokaryotic, archaeal, metagenomic), 1M-token context at single-nucleotide resolution.
  • The “SOTA” is scoped — CONFIRMED (paper as summarized): noncoding SNV is first among unsupervised methods but trails supervised specialists; coding SNV trails ESM-1b / GPN-MSA / PhyloP; clear wins are noncoding non-SNV and splice, zero-shot. The exact per-tier AUROC values are unverified (full text gated at verification time) — only the ordering is attributed.
  • Short-range blind spot and severity inversion — INDEPENDENT preprint: a benchmarking preprint stratifies signals into short (1–5 nt), medium (~30 nt) and long (>60 nt). It reports a codon-usage-bias deficit (mean Jensen-Shannon divergence 0.254; preferred codon predicted at only 24.4% of positions) and a severity inversion — mild pathogenic variants scored 100% correct, but performance falls progressively for moderate and severe variants. For a pathogenicity predictor that direction is backwards.
  • Scale skepticism — INDEPENDENT preprint: a deployment benchmark finds 7B AUC ~0.87 versus 40B 0.86–0.87 (little gain), and a hardware fragility — on FP8 / H800 the 40B stays at AUC 0.88–0.95 with ~4× faster inference, but on non-FP8 hardware the 40B collapses to AUC 0.48 while the 7B stays robust.

3. Strengths and limits of the methodology

Strengths: the paper attributes its own stratified ordering and separates zero-shot from supervised regimes, so the “SOTA” is honestly bracketed at the source. The single-nucleotide-resolution, long-context design genuinely helps where alignment is hard (non-SNV, splice). Limits: what is confirmed here is mostly the ordering of claims, not the absolute AUROC figures (the Nature full text and the blind-spots PDF were gated / 403 at verification, so secondary indices — PubMed, press summaries — were cross-checked instead; deep_read_required). The two independent critiques are themselves preprints from small author groups (Horace Mann School / NY Medical College for the blind-spots work), so their weight is also held conservatively. And the comparison fairness question — whether supervised baselines and zero-shot models are benchmarked without data leakage — is exactly what a standardized evaluation still has to settle.

4. Neighbouring domains

Two patterns recur across the series. First, the “bigger is always better” narrative fails: genome 40B ≈ 7B mirrors the open question of whether scaling justifies large cell FMs (STATE, scGPT) in Part 3. Second, old baselines still win: conservation (PhyloP / GPN-MSA) leads on coding variants just as a linear baseline reportedly leads specialist FMs on cell perturbation prediction (Nature Methods 2025). The clinical hook — cardio-renal-metabolic (CKM) genotype interpretation — is where this matters most: dropping a zero-shot genome FM into standalone clinical variant calling is premature given the severity-inversion risk, and is deferred to the Part 5 drug/clinical angle.

5. Commercialization and market context (TRL, companies)

ItemStatusReal bottleneckHonest readiness
OpennessWeights, training data and inference code released (arcinstitute/evo2_7b · evo2_40b on Hugging Face)License is the NVIDIA Open Model License (not pure-permissive; commercial-use conditions, not Apache)Open-weight, but with license conditions
Deployment pathHosted as an NVIDIA BioNeMo NIM microservice; fine-tunable via BioNeMo Framework (build.nvidia.com/arc/evo2-40b demo)FP8-precision hardware dependence (§ scale skepticism)API + open-weight coexist
Deployment readinessResearch-grade40B accuracy can break down on A100 / non-FP8 hardwareTRL 3–4 (not clinical grade)

Company context (factual, neutral, no buy/sell implication): Evo2 is a collaboration led by Arc Institute (a non-profit) with NVIDIA (listed), Stanford, UC Berkeley and UCSF. NVIDIA appears here only in an infrastructure / hosting / marketplace role (BioNeMo, NIM); any stock implication belongs to the investment layer, not this technical description. The practical caveat that matters for buyers: an open-weight download does not guarantee the headline accuracy — on non-FP8 hardware the 40B can collapse, so the 7B is arguably the more robust practical default.

6. The skeptic’s bottom line

  • Adversarial verification complete (11 confirmed / 1 corrected-forward / 2 unverified-vendor), but the “noncoding SOTA” headline is only true with the bracket. Misreading warning: “noncoding SOTA” without “among zero-shot methods” hides the fact that supervised and conservation specialists still lead on coding and noncoding SNVs. The exact per-tier AUROC values are attributed to the paper but were not independently read (gated full text).
  • demo-gap: this is a benchmark, not a clinical demonstration. The independent blind-spots preprint reports a severity inversion (worse on more severe variants) and a codon-usage short-range deficit — so a standalone zero-shot clinical pathogenicity claim is on hold and should be escalated, not published as ready.
  • Scale skepticism: a deployment benchmark finds 40B ≈ 7B (0.86–0.87) with the 40B fragile on non-FP8 hardware (AUC 0.48) — weakening the “bigger is better” story and pointing to 7B as the practical default.
  • Source discipline: the two critiques are preprints from small author groups, so their weight is held conservatively; the peer-reviewed status is the Nature paper only.

7. What to watch (falsifiable predictions)

  1. If a leakage-controlled, regime-controlled prospective benchmark becomes standard, Evo2’s noncoding-SNV “SOTA” falls out of the top tier once supervised specialists are included (it stays #1 only within the unsupervised bracket).
  2. In a subsequent clinical-style benchmark, Evo2’s zero-shot pathogenicity trails a conservation-plus-supervised ensemble on severe / loss-of-function subsets (the severity inversion reproduces).
  3. Evo2 40B keeps an AUC gap of <0.02 over 7B in real variant classification, and its accuracy advantage disappears on non-FP8 hardware (no scale premium).

References

Source knowledge asset: knowledge-base/deep-dives/bio-foundation-models/part2-genome-fm.md (generated 2026-07-12, VERIFIED: 11 confirmed / 1 corrected-forward / 2 unverified-vendor; deep_read_required — Nature full text and blind-spots PDF gated at verification, secondary indices cross-checked). This draft inherits the figures and source attributions of the original part and creates no new figures or sources.

Disclosure

This post is for information only and is not investment advice. The author holds no position in, and no financial interest in, the entities mentioned.

COI note: Evo2 is a collaboration of Arc Institute (non-profit), NVIDIA (listed), Stanford, UC Berkeley and UCSF. NVIDIA and Arc are described in a factual, neutral technology / platform context with no buy/sell implication. Vendor quantitative claims (the “noncoding SOTA”) are attributed to the paper, with independent-verification status flagged; exact per-tier AUROC values remain unverified (gated full text). The independent critiques are preprints from small author groups, weighted conservatively.

Comments

Leave a comment