Uthando Nethemba

The generative engine for protein design — backbone diffusion + inverse folding + a self-consistency filter that only checks folding, not function

—

by

itembalabantu
in

Evidence-first notes on bioscience and deep tech, at the edge of the lab and the market. Information only — not investment advice, not medical advice. Every in-silico metric (scRMSD, pLDDT, sequence recovery, designability) is separated from wet-lab validation (X-ray/cryo-EM, binding, activity), and any vendor/lab claim is attributed. Benchmark or preprint SOTA is not read as experimental success.

The 30-second version

  • What. The modern generative engine for de novo protein design is an assembly line of three parts: (1) backbone generation by diffusion (RFdiffusion, Chroma, Genie) creates a 3D main-chain from noise; (2) inverse folding (ProteinMPNN, LigandMPNN) fills in an amino-acid sequence that should fold to that shape; and (3) a self-consistency filter re-folds the sequence with AlphaFold2/RoseTTAFold and keeps it only if the prediction matches the original backbone (scRMSD < 2 Å and pLDDT > 70) — a design→predict→keep loop.
  • So what. The filter validates only designability — “does this sequence fold back to this backbone.” It structurally cannot see binding, catalysis, expression or dynamics. So the metric–reality gap is built into the engine itself: passing the filter means “well-designed shape,” not “functional protein.” The engine’s strong zone (rigid scaffolds, symmetric assemblies, motif scaffolding) maps exactly onto what the filter can check; its weak zone (flexible/disordered regions, precise functional-site placement, conformational dynamics) maps onto what the filter is blind to.
  • Now what. The engine is genuinely impressive, reproducible (all open weights on GitHub), and some outputs reach crystallographic confirmation — so this is not a demo. But “self-consistent = well-designed” quietly slides into “= functional,” and that slide is the engine-layer root of the series’ bottleneck. In-silico designability (scRMSD/pLDDT/sequence recovery) is not wet-lab validation; benchmark designability SOTA (notably Genie 2/3, which are preprints) is not experimental hit rate; and quantitative wet-lab success rates remain deferred to Part 4.

The five-minute read

The engine is assembled from three parts — and only one of them is a gate

Part 0 of this series graded the design axis as “wet-lab demonstrated, but experimental success rate is the bottleneck.” Part 2 asks which component of the engine that bottleneck lives in. The answer: the generative engine is an assembly line. Backbone generation uses a diffusion model to denoise a 3D main-chain from pure noise (RFdiffusion, RFdiffusionAA, Chroma, Genie). Inverse folding then answers “which sequence folds into this shape” (ProteinMPNN, LigandMPNN). Finally, the self-consistency filter re-folds each candidate sequence with AlphaFold2/RoseTTAFold and keeps only those whose prediction matches the original backbone. Parts (1) and (2) are cheap and produce thousands to tens of thousands of candidates in seconds-to-minutes; the filter culls the majority in silico, shrinking thousands down to tens or hundreds.

That filter is where this analysis concentrates, because it is the only real gate — and it is a narrow one. The community’s central claim is that the AF2/RF filter raised binder success rates markedly (Cao et al. 2022 reported roughly a 10x improvement in binder hit rate). But even the minority that pass the filter still fail downstream in the wet lab — poor expression, non-specific binding, undetectable affinity. The filter raises hit rate but does not remove the wet-lab bottleneck. This chapter dissects parts (1)–(3); Parts 3 and 4 cover the wet-lab reality.

What the self-consistency filter can and cannot see

The design→predict→keep loop checks whether a designed sequence can exist in the intended shape. It does not check what that protein does. And because the filter re-uses a prediction foundation model (AF2/RF) that is itself weak on disorder, protein-protein interactions, ligands and dynamics, the axes the filter cannot see coincide almost exactly with the axes the prediction model cannot solve. That coincidence is why the metric–reality gap is structurally embedded in the engine, not a fixable tuning problem. This is the design-side homolog of the bio-foundation-models series’ esmGFP wet-lab gap: good predicted metrics, separate real-world function.

What the filter catches (in-silico) What the filter misses (decided in the wet lab)
Whether the sequence folds to the target backbone (designability) Recombinant expression and solubility (a fold-able design can still fail to express)
Prediction confidence (pLDDT) and structural match (scRMSD) Target-specific binding and affinity (Kd) — folding is not binding
A rough fold-stability signal Catalytic activity (kcat/Km) — structure is not function
— Conformational dynamics, off-target, aggregation
Self-consistency predicts whether a protein “can exist in the designed shape,” not “what it does.” scRMSD, pLDDT, sequence recovery and designability are all in-silico metrics and do not entail expression, binding, catalysis or clinical value. Benchmark designability SOTA (particularly Genie 2/3, which are arXiv/GitHub preprints) is not wet-lab success. Cross-tool superiority claims (RFdiffusion vs Chroma vs Genie) are not head-to-head unless run on the same benchmark and filter. Quantitative wet-lab success rates are deferred to Part 4.

Deep dive

1. Background — the end-to-end pipeline from nothing to a real structure

A typical Baker-Lab-style de novo binder/scaffold flow runs in four stages. (1) Backbone generation (diffusion): RFdiffusion / Chroma / Genie produce thousands to tens of thousands of novel main-chain coordinates. (2) Sequence design (inverse folding): ProteinMPNN / LigandMPNN assign a sequence to each backbone. (3) Self-consistency filter (design→predict→keep): AF2/RF re-fold each design, and only candidates with scRMSD < 2 Å and pLDDT > 70 survive, shrinking the pool from thousands to tens or hundreds. (4) Wet lab: expression → binding/activity assay → X-ray/cryo-EM structural confirmation — this is where the success rate is actually decided (Parts 3 and 4).

Separation rule (analysis-standards §2): everything in stages (1)–(3) — scRMSD, pLDDT, sequence recovery, designability — is an in-silico metric. Wet-lab validation (X-ray/cryo-EM, binding, activity) is reported separately, and where absent is labeled “in-silico only.” Benchmark designability SOTA (especially Genie 2/3 preprints) must not be read as experimental success.

2. Backbone generation (1) — RFdiffusion (Nature 2023)

RFdiffusion fine-tunes the RoseTTAFold structure-prediction network into a denoising diffusion generative model: starting from pure noise, it reverse-denoises into physically plausible main-chain coordinates (Watson et al., Nature 2023, s41586-023-06415-8). A network that used to predict structure was retrained to make it. Training data is the Protein Data Bank (PDB). It supports unconditional generation, motif scaffolding (grafting a functional motif onto a new scaffold), symmetric oligomers, and binder design.

The in-silico definition of “designable” (= self-consistency): re-fold the designed sequence (AF2/RF) and require scRMSD < 2 Å AND pLDDT > 70. This is the engine-wide standard in-silico success metric.

Wet-lab validation (not a demo): de novo binders against influenza HA, IL-7Rα, PD-L1 and TrkA were confirmed by X-ray crystallography and cryo-EM — notably an influenza-HA binder whose cryo-EM structure closely matched the design model (Nature 2023). This axis reaches crystallography, clearing the analysis-standards §2 demonstration threshold. Caveat: RFdiffusion binder in-silico designability is high, but reported wet-lab hit rates (fraction of high-affinity binders) are on the order of low single-digit percent and are strongly target/protocol dependent; precise quantitation and independent reproduction are deferred to Part 4 (unverified). Run-to-run success variation under identical parameters has also been reported anecdotally (GitHub Issue #391, not peer-reviewed).

2b. All-atom extension — RFdiffusionAA / RoseTTAFold All-Atom (attribution correction)

Correction to the Part 0 / series frame (refuted item): the Part 0 table and task framing labeled the all-atom version “RFdiffusion-AA (Nature 2023, all-atom).” That is an error. The all-atom work is Krishna et al., Science 2024 (science.adl2528, April 2024, vol. 384 iss. 6693), “Generalized biomolecular modeling and design with RoseTTAFold All-Atom.” Baker Lab first released all-atom RoseTTAFold/RFdiffusion as a bioRxiv preprint in October 2023 (2023.10.09.561603); peer review is Science 2024 — not Nature 2023 (no-fabrication correction).

RoseTTAFold All-Atom (RFAA) represents amino acids and DNA bases as residues but everything else (small molecules, metals, covalent modifications) at atomic resolution, jointly modeling protein + nucleic acid + small-molecule + metal complexes. RFdiffusionAA fine-tunes RFAA with diffusion to build a binding pocket tailored around a small-molecule ligand (generating a pocket per ligand rather than recycling a native/pre-designed scaffold). Wet-lab validation (not a demo): proteins binding digoxigenin (a cardiac-glycoside derivative), heme (an enzyme cofactor) and bilin (a light-harvesting pigment) were designed and validated by crystallography and binding measurement (Science 2024). Engine implication: all-atom lets functional-site (ligand/metal/cofactor) placement be handled during backbone generation, partly relaxing the §6 “precise functional-site placement” weakness — but the self-consistency-type filter still validates structural match, not catalysis or affinity itself.

3. Backbone generation (2) — Chroma and Genie (alternative architectures)

Chroma (Generate Biomedicines, Nature 2023): a programmable generative model that samples backbone and sequence directly and steers design via Bayesian inference under external constraints (Ingraham et al., Nature 2023, 623:1070–1078, s41586-023-06728-8). Architecturally it uses a diffusion process that respects polymer-ensemble statistics, plus a molecular neural network with sub-quadratic scaling (long-range interactions at near-linear cost) and a 3D synthesis layer from inter-residue geometry. It can be conditioned by symmetry, substructure, shape, semantics and natural-language prompts, with all-atom design integrated. Where RFdiffusion belongs to the “retrained predictor” lineage, Chroma is a frame designed for generation from the start; its developer is Generate Biomedicines (private vendor), and Chroma is a platform asset. Caveat: Chroma’s paper presents varied conditional generation and some experimental validation, but independent-group wet-lab success-rate reproduction is a separate matter (unverified, Parts 4–5). It is not a same-benchmark head-to-head with RFdiffusion, so no superiority verdict.

Genie / Genie 2·3 (Lin & AlQuraishi — preprint lineage): diffusion over an oriented residue cloud that is SE(3)-equivariant — simple Gaussian forward noising, with an asymmetric SE(3)-equivariant attention backward process (Lin & AlQuraishi, arXiv 2301.12485, ICML 2023). Genie claims more designable, novel and diverse backbones than prior models; Genie 2 (arXiv 2405.15489) claims SOTA on designability, diversity and novelty for unconditional and motif-scaffolding tasks; Genie 3 extends to all-atom and binders (GitHub aqlaboratory/genie3). Media and metric rule: the Genie line is arXiv/GitHub (preprint/open-source), not Nature/Science, and its reported metrics (designability, diversity, novelty) are all in-silico and self-consistency-based. “SOTA designability” is not wet-lab success (analysis-standards §2). Genie’s strength is architectural efficiency and reproducibility; these preprint numbers do not guarantee experimental hit rate, and cross-tool superiority is not assertable without the same benchmark and same filter.

4. Sequence design (inverse folding) — ProteinMPNN and LigandMPNN

A backbone gives only “shape.” Inverse folding fills in the sequence that folds into it.

ProteinMPNN (Baker Lab, Science 2022): a message-passing deep-learning inverse-folding model (structure→sequence, autoregressive) (Dauparas et al., Science 2022, 378:49–56, science.add2187). Its key in-silico metric is native-backbone sequence recovery of 52.4%, versus physics-based Rosetta at 32.9%, at far lower compute; improved solubility/expression tendencies are also reported. It is the de facto standard partner that assigns sequences to RFdiffusion-generated backbones, supplying the “re-folding candidate sequence” for the self-consistency filter. Note: sequence recovery is an in-silico metric — high recovery is not expression, binding or activity.

LigandMPNN (Baker Lab/IPD, Nature Methods 2025): an atomic-context extension of ProteinMPNN that explicitly models non-protein atomic context (small molecules, nucleic acids, metals) and also generates sidechain conformations (Dauparas et al., Nature Methods 2025, s41592-025-02626-1; code github.com/dauparas/LigandMPNN). Native-backbone sequence recovery (LigandMPNN vs Rosetta / ProteinMPNN): small-molecule-contact residues 63.3% vs 50.4% / 50.5%; nucleic-acid contact 50.5% vs 35.2% / 34.0%; metal contact 77.5% vs 36.0% / 40.6% (all in-silico). Wet lab: 100+ small-molecule/DNA-binding proteins designed, several confirmed by X-ray. Vendor/IPD claim (attributed): redesigning Rosetta-based binders yielded “up to 100x improvement in binding affinity” — this is IPD/paper framing, and independent reproduction is a separate matter (unverified); the “100x” figure is for specific redesign cases and must not be generalized to overall hit-rate improvement. Engine view: LigandMPNN directly targets the functional-site (ligand/metal) sequence-placement weakness of §6 — real progress on functional sites, but the filter still does not validate catalysis or affinity itself.

5. The self-consistency filter loop — the core

The design→predict→keep loop runs: (1) inverse folding (ProteinMPNN) produces several candidate sequences per backbone; (2) each sequence is re-folded (predict) by AF2/RoseTTAFold into a predicted structure; (3) if the prediction matches the original backbone within scRMSD < 2 Å and pLDDT > 70, it is “self-consistent” and kept. The central proposition: self-consistency predicts only whether a protein “can exist in the designed shape,” not “what it does.” And the prediction FM used as the filter (AF2/RF) is itself weak on disorder, PPI, ligands and dynamics (inherited from bio-FM Part 1). Therefore the axes the filter cannot see overlap precisely with the axes the prediction FM cannot solve — which is exactly why the metric–reality gap is structurally embedded in the engine. This is the design-engine analog of Part 1’s esmGFP wet-lab gap (good predicted metric, separate real-world fluorescence), and the engine-layer recurrence of the neuro series’ “surrogate marker → hard outcome” translation bottleneck.

6. Strength vs weakness — where the engine wins and where it loses

Resolving Part 0’s layer verdict down to engine components, one pattern is consistent: the engine is strong on the axes the filter can validate (shape) and weak on the axes it cannot see (function, dynamics).

Strong (shape-dominated — self-consistency works well): rigid scaffolds / stable idealized folds (well-folding scaffolds without demanding targets have high designability and relatively easy realization); symmetric assemblies / oligomers (symmetry constraints narrow the search space, where diffusion is especially strong — RFdiffusion/Chroma symmetric conditioning); motif scaffolding (grafting a fixed functional motif onto a new scaffold, where motif coordinates act as anchors and structural constraints are clear).

Weak (function/dynamics-dominated — the filter is blind): flexible / intrinsically disordered regions (a single-backbone, scRMSD frame is itself ill-suited — it cannot capture an ensemble); precise functional-site placement (enzyme active-site geometry, catalytic-residue angles, where sub-angstrom precision drives catalysis but self-consistency does not reward it — the axis where de novo enzyme activity trails nature, Part 3); conformational dynamics / allostery / induced fit (outside the static single-structure paradigm — the filter cannot see it in principle); PPI and antibody CDR loops (flexible loops + expression + humanization compound into the largest bottleneck — RFdiffusion-antibody “low success rate,” Part 3). Summary: the engine’s strong/weak boundary coincides with the boundary of “what self-consistency can validate.” Shape-dominated problems are solved well; function/dynamics-dominated problems get no signal from the filter and are decided in the wet lab.

7. Verdict table (Part 2 confirmed values)

Item Status Basis
RFdiffusion = RoseTTAFold→diffusion, trained on PDB Confirmed Watson et al. Nature 2023 s41586-023-06415-8
Self-consistency “designable” = scRMSD < 2 Å, pLDDT > 70 Confirmed (in-silico metric) RFdiffusion paper / field standard
RFdiffusion binder wet-lab structural confirmation (HA cryo-EM etc.) Demonstrated (not a demo) Nature 2023 — X-ray/cryo-EM
All-atom = RFdiffusionAA / RFAA Confirmed · Science 2024 (not Nature 2023 — corrected) Krishna et al. Science 2024 adl2528
ProteinMPNN seq recovery 52.4% vs Rosetta 32.9% Confirmed (in-silico metric) Dauparas et al. Science 2022 378:49–56
LigandMPNN seq recovery (small-mol 63.3 / nucleic 50.5 / metal 77.5%) Confirmed (in-silico metric) Dauparas et al. Nature Methods 2025
Chroma programmable diffusion (symmetry, natural language, all-atom) Confirmed Ingraham et al. Nature 2023 623:1070–1078
Genie SE(3)-equivariant diffusion, designability strength Confirmed (preprint · in-silico only) Lin & AlQuraishi arXiv 2301.12485 · 2405.15489
Entire engine is open weight (GitHub) Confirmed · reproducibility strength RFdiffusion · ProteinMPNN · LigandMPNN · Chroma · Genie
Self-consistency validates binding/catalysis/expression Does not (principled limit) Filter = re-folding only; function is wet-lab (§5·§6)
Quantitative wet-lab success rate after filter pass Undetermined · bottleneck bioRxiv 2025 6-target (Part 0) · deferred to Part 4
Overall verdict: proceed-with-caveats (conditional). The engine components (architecture, in-silico metrics, open weights) are confirmed against primary sources, reproducibility is a genuine strength, and some outputs reach crystallography (not a demo). But the metric–reality gap — the self-consistency filter validates only “folding,” not “function” — is embedded in the engine, so the slide from “passes filter = well-designed = functional” must be guarded against. Quantitative wet-lab success rates depend on the Part 4 decisive treatment.

8. The skeptic’s bottom line (five caveats at Tier-1 exposure)

  • All in-silico. scRMSD, pLDDT, sequence recovery and designability are entirely in-silico metrics and are unrelated to expression, binding, catalysis or commercialization.
  • Folding only. Self-consistency validates only “folding” in principle, inheriting the prediction FM’s limits (disorder, PPI, dynamics) directly (§5).
  • Preprint SOTA ≠ experiment. Genie 2/3 and similar preprint SOTA are benchmark/in-silico only — not wet-lab success (§3).
  • No cross-tool superiority. RFdiffusion, Chroma and Genie are not same-benchmark, same-filter head-to-heads; no superiority verdict.
  • Vendor/IPD figures are attributed. Claims such as “up to 100x affinity improvement” are attributed and await independent verification (§4).

9. Falsifiable predictions

  • P1: if the correlation between self-consistency (scRMSD/pLDDT) and wet-lab results (expression/binding) holds predictively even for novel targets far from the training set, the filter’s reliability is strengthened (toward a discovery engine); if it holds only for near targets, the metric–reality gap (Part 0 hypothesis “c”) is strengthened. (Test: Part 4, inheriting the bio-FM Part 4 leakage methodology.)
  • P2: if the all-atom / functional-site components (RFdiffusionAA, LigandMPNN) produce reproducible double-digit-percent success on enzyme activity or ligand affinity across independent groups, the §6 functional-site weakness is relaxed; if low activity/affinity reproduces, the weakness is confirmed. (Test: Parts 3–4.)
  • P3: if Genie 2/3 preprint designability SOTA translates into an experimental hit-rate advantage over RFdiffusion under the same wet-lab protocol, the predictive power of designability metrics is supported; if the difference vanishes experimentally, “benchmark designability ≠ experimental success” (§3 caveat) is confirmed. (Test: Part 4.)

10. Framing — connection to Parts 3 and 4

Part 3 (function): as the engine’s candidates ((1)–(3)) are realized into binders, enzymes and antibodies, the §6 strong/weak boundary reappears — de novo binder (strong), serine-hydrolase low activity (functional-site weak), RFdiffusion-antibody low success rate (flexible loop + expression weak) — with §6 signaling that the causes live in the engine components. Part 4 (validation reality / bottleneck layer): the decisive treatment is quantifying the self-consistency↔wet-lab gap flagged in §5 — the concrete hit rates of the bioRxiv 2025 six-target study, the novel-target generalization of in-silico–wet-lab correlation, and data-leakage control. The distinction between “success rate per filter-passing candidate” and “per generated candidate” (Part 0 §4) is also quantified in Part 4.

References

Disclosure

This post is for information only and is not investment advice, and not medical advice.

COI note: this post describes academic groups (Baker Lab / Institute for Protein Design — RFdiffusion, ProteinMPNN, LigandMPNN, RFdiffusionAA) and listed and private companies (Alphabet/Google DeepMind, Meta, NVIDIA; Generate Biomedicines, the developer of Chroma; EvolutionaryScale) in a descriptive, neutral context. In-silico metrics (scRMSD, pLDDT, sequence recovery, designability) are separated from wet-lab validation (X-ray/cryo-EM, binding, activity). Media and review status is labeled: peer-reviewed (Nature/Science/Nature Methods) versus preprint (arXiv/bioRxiv) versus lab/company blog. Vendor/IPD quantitative claims (e.g., “up to 100x affinity improvement”) are attributed and their independent-verification status is stated; benchmark or preprint SOTA is not read as experimental success. Attributions are factual and neutral and are not buy/sell implications for any security. Quantitative claims are attributed to the vendor, author or preprint. The author holds no position in, and has no financial interest in, the companies named.

Comments

Leave a comment