Uthando Nethemba

A concept-bottleneck AI predicts immunotherapy response across cancers — but every result is retrospective, and the survival win rests on one sponsor’s own trial

—

by

itembalabantu
in

Evidence-first notes on bioscience and deep tech, at the edge of the lab and the market. Information only — not investment advice.

The 30-second version

  • What. A Harvard (Zitnik lab) and Roche team published COMPASS in Nature Medicine (2026-07-03, doi:10.1038/s41591-026-04502-7): a lightweight (~1.02M-parameter) “concept-bottleneck” transformer that reads a tumour’s bulk RNA-seq and predicts response to immune-checkpoint inhibitors (ICIs). It is self-supervised-pretrained on TCGA (10,184 tumours, 33 cancer types), then parameter-efficiently fine-tuned on 16 clinical cohorts (1,133 patients, 7 cancers, six ICI regimens). Per the authors, in leave-one-cohort-out testing it beats 22 existing methods by +8.5% accuracy and +15.7% AUPRC, and in a held-out phase-2 cohort it stratifies survival at a hazard ratio (HR) of 4.7 versus 1.67 for tumour mutational burden (TMB).
  • So what. Read the fine print before the headline. First, every result is retrospective — there is no prospective companion-diagnostic RCT, so a benchmark win is not clinical actionability. Second, the validation cohorts are fresh / flash-frozen tissue, which is incompatible with the FFPE (formalin-fixed paraffin-embedded) blocks that routine oncology actually uses. Third, the flagship survival result (HR 4.7) rests on a single Roche-sponsored trial (IMvigor210, atezolizumab), and the corresponding author is a Roche employee — a conflict-of-interest plus a selective-flagship risk, with no multi-cohort prospective survival validation. Fourth, roughly one-third of pairwise cohort transfers failed (163/240 succeeded), tempering the “generalizable” headline. All quantitative figures below are attributed to the authors and their cohorts.
  • Now what. This is a retrospective, research-cohort benchmark — not a prospective, clinical-grade companion diagnostic, and there is no FDA pathway (no established route for a pan-cancer transcriptomic companion Dx). Code and an interactive web tool are open (CC BY-NC 4.0), so reproducibility is a relative strength. Because of listed-equity COI adjacency (Roche), framing is kept strictly neutral and factual, and this draft is held for Principal review.

The five-minute read

The design: a foundation-model recipe with an interpretable bottleneck

COMPASS ports the foundation-model grammar — pretrain on abundant unlabelled -omics, then transfer to scarce labelled clinical cohorts — into tumour immunology. Its distinctive feature is a concept bottleneck: rather than a black box, the model projects 15,672 protein-coding genes through 132 curated gene signatures onto 44 interpretable immune “concepts” (43 tumour-immune-microenvironment concepts plus one cancer-type concept). That bottleneck does two jobs at once — it yields human-readable immune-state and pathway axes, and it acts as a strong inductive bias that curbs overfitting on the very small ICI cohorts (some with fewer than 30 patients). The model is not a large language model: at roughly one million parameters, with fine-tuning shrinking to as few as 182 parameters in a linear-probe setting, it is a deliberately lightweight concept-bottleneck model.

The performance case has two prongs. On classification, leave-one-cohort-out testing (train on 15 cohorts, evaluate on the held-out 16th) shows, per the authors, +8.5% accuracy, +15.7% AUPRC and +12.3% Matthews correlation over 22 baselines such as TIDE, IMPRES, GEP and NetBio — and it reports transfer to an unseen cancer (gastric, 83.7%) and an unseen drug class (anti-CTLA4, 76.1%). On survival, holding out IMvigor210 (metastatic urothelial cancer, atezolizumab) entirely, predicted responders versus non-responders split 1-year overall survival 86% versus 40%, an HR of 4.7 (p = 1.7×10⁻⁷), ahead of TMB (1.67), PD-L1 IC2+ (1.75) and IHC immune phenotype (1.85).

Self-authored clinical-evidence ladder, illustrative not a scoreboard. Figures attributed to the study’s open-access text (Nature Medicine 2026-07-03, doi:10.1038/s41591-026-04502-7; PMC12083594): leave-one-cohort-out +8.5% accuracy / +15.7% AUPRC vs 22 baselines; IMvigor210 held-out HR 4.7 vs TMB 1.67, PD-L1 1.75, IHC 1.85; pairwise transfer 163/240 (LFT) succeeded. Rung = evidence-strength status only, not a ranking. Not a buy/sell signal (Roche).

Three fault lines that govern the story

First, everything is retrospective. IMvigor210 is itself a post-hoc analysis of a closed phase-2 trial; there is no prospective, real-world deployment evidence. Second, specimen compatibility: bulk RNA-seq needs fresh or flash-frozen tissue, whereas community oncology runs on FFPE paraffin blocks — so there is a physical gap between the controlled research setting where the model was validated and the practice setting where it would be used (a point an independent critique also raised). Third, conflict of interest: the survival flagship, IMvigor210 (and IMmotion150), are Roche/Genentech-sponsored trials of the company’s own atezolizumab (Tecentriq), and the corresponding author is a Roche employee. The classification accuracy — a retrospective, code-open, leave-one-cohort-out result — is comparatively mature; but the “prediction leads to a treatment change leads to better outcomes” prospective interventional evidence is simply absent. That absence is the oncology instance of what our ledger calls the L-001 predictive-diagnostic actionability gap.

Deep dive

1. Background

ICIs produce durable remissions in some patients but the overall response rate is low (30.5% responders in this pooled cohort). The established biomarkers — TMB, PD-L1 IHC, gene-expression profiling, TIDE — are cancer- and drug-dependent, use different cut-offs, and no single predictor generalises across cancers and agents. COMPASS aims to fill that gap by compressing the transcriptome onto biologically grounded concept axes rather than a black-box embedding.

2. What this study newly established

Two axes, graded separately. On generalisation, the contribution is real: a single model that transfers, in leave-one-cohort-out testing, to an unseen cancer type (gastric) and an unseen drug class (anti-CTLA4), which prior single-marker methods do not. On interpretability, the concept bottleneck exposes which immune concepts drive a prediction, an axis that regulators and clinicians tend to demand. What the study does not establish is any prospective or interventional claim — the design is retrospective throughout, and the authors position COMPASS as a research and translational tool (e.g. trial-endpoint selection and patient stratification), not a clinical product.

3. Methods — strengths and limits

Strengths. The paper is peer-reviewed and open access (PMC), the code is public (GitHub, mims-harvard/COMPASS-web), the preprocessing pipeline is standardised, and an interactive response-map web tool is available — so the core quantitative claims were cross-checkable against the full text. Leave-one-cohort-out is a leakage-controlled design that approximates a genuine held-out test.

Limits, in order of severity. (i) Everything is retrospective — no prospective companion-diagnostic RCT, so bench accuracy does not equal clinical actionability. (ii) FFPE incompatibility — validation used fresh / flash-frozen tissue, not the paraffin blocks community oncology relies on. (iii) COI and selective-flagship risk — the HR 4.7 survival flagship rests on a single Roche-sponsored cohort with a Roche corresponding author, and there is no multi-cohort prospective survival validation. (iv) Transfer is not universal — roughly one-third of the 240 pairwise cohort transfers failed (163/240 succeeded under linear fine-tuning), which sits uneasily beside the “generalizable” framing. (v) Baseline fairness and batch effects — whether all 22 baselines were tuned equally, and whether cross-cohort performance reflects biology rather than sequencing-batch artefacts, remain to be confirmed against the supplement. Three items are logged as unverified in the source analysis: the pretraining compute/GPU-hours (no value in the paper — not estimated), whether pretraining weights are deposited, and the exact “six ICIs” accounting (combinations appear to be counted separately).

4. Connection to neighbouring domains

Methodologically, the recipe — large unlabelled -omics self-supervised pretraining, plus an interpretable concept layer, plus parameter-efficient transfer to label-scarce clinical cohorts — is directly portable to other label-poor biology predictions (neuro biomarker stratification, drug response, minimal residual disease). The concept bottleneck is attractive precisely because it supplies the interpretability that clinical and regulatory acceptance require. It also sits against our cell-transcriptome foundation-model thread (whether -omics FMs actually beat linear baselines remains the open L-007 question): COMPASS is notable for winning small cohorts through inductive bias rather than raw scale, making it both a candidate counter-example to “scale equals performance” and a test case for “retrospective bench win does not equal real-world generalisation.”

5. Commercialization and investment lens (TRL, related companies)

TRLs, kept separate: COMPASS as a retrospective, multi-cohort-validated predictive-biomarker research tool is TRL 3–4; there is no prospective validation, no regulatory submission, no clinical integration. An independent critique (Clinical Trial Vanguard) flags the companion-diagnostic bottlenecks specifically: there is no established FDA pathway for a pan-cancer transcriptomic biomarker (pursuing CDx approval is described as a multi-year detour), the FFPE incompatibility creates an adoption friction, and a sponsor whose trial succeeds on COMPASS could inherit a post-approval obligation to formally validate the assay. Licensing is CC BY-NC 4.0 — non-commercial reuse only — so commercial companion-Dx development would require separate licensing. On companies, stated as neutral facts: Roche/Genentech contributed two co-authors and the survival-validation trials of atezolizumab (Tecentriq). Roche (SIX: ROG; OTC: RHHBY) is a listed company; that a company’s own phase-2 trial was used to validate a promising biomarker carries a mild-positive read, but at a retrospective, single-cohort survival stage any product or trading implication is premature and is deferred to the investment layer. This is neutral factual context, not a negative characterization.

6. The counter-view (skeptic block)

The source analysis passed its adversarial gate as proceed-with-caveats (conditional), not verified-clean, and required these caveats to travel with any published version:

  1. Every result is retrospective and drawn from research cohorts — there is no prospective companion-diagnostic RCT, so a benchmark win is not clinical actionability (the L-001 predictive-Dx gap).
  2. The validation cohorts are fresh / flash-frozen tissue, which is incompatible with the FFPE paraffin blocks routine oncology actually uses.
  3. The flagship survival result (HR 4.7) rests on a single Roche-sponsored cohort (IMvigor210) with a Roche corresponding author — a conflict of interest plus a selective-flagship risk, with no multi-cohort prospective survival validation.
  4. Roughly one-third of pairwise cohort transfers failed (163/240 succeeded), which limits the “generalizable” headline.
  5. All quantitative figures are attributed to the authors and their cohorts, not independently reproduced.

The gate noted that the science (open code, leave-one-cohort-out design) is partly separable from the Roche-linked survival narrative — which is why the verdict is proceed-with-caveats rather than hold — but the survival quantities must be reported as vendor-cohort claims. The residual replication risks the gate flagged are cross-cohort transcriptome batch effects, baseline-tuning fairness across the 22 comparators, and possible selection bias in which of the 132 signatures were curated.

7. What to watch next

  1. Whether a prospective, multi-cohort survival validation replicates the HR outside a single sponsor’s trial.
  2. Whether the model is shown to work on FFPE specimens, not only fresh/frozen tissue.
  3. A batch-effect ablation and a fair, equally tuned baseline comparison (supplement-pending).
  4. Any move toward an FDA biomarker-qualification or companion-diagnostic pathway.

References

Disclosure

This post is for information only and is not investment advice, and it is not a buy/sell view on any company named. Conflict-of-interest facts, stated neutrally: the study’s corresponding author (D. Marbach) and a co-author (N. Nair) are employees of Roche; the survival-validation cohorts (IMvigor210, IMmotion150) are Roche/Genentech-sponsored trials of the company’s own atezolizumab (Tecentriq). The HR 4.7 and related survival figures are therefore reported as vendor-cohort claims and were not independently reproduced. Roche (SIX: ROG; OTC: RHHBY) is a listed company; because the survival evidence is retrospective and single-cohort, any adjacency or competition implication is premature and is deferred to the investment layer. All quantitative figures are attributed to the open-access text of the study; the paywalled Nature Medicine final was not separately collated against the open version. Pretraining compute and weight-deposit status were not stated in the paper and are not estimated here. The author holds no position in any company mentioned (default: no interest).