Foundation Models for Biology

Mamba/SSM-based, bidirectional, reverse-complement equivariant, outperforms 10× larger transformers on long-range tasks.

Evo 2 has 40B parameters, 1 megabase context length, trained on 9 trillion nucleotides spanning eukaryotes and prokaryotes. Single-nucleotide resolution via StripedHyena architecture (hybrid attention + signal-processing operators). Capable of zero-shot prediction *and* whole-gen

variant effect across DNA, RNA, and protein in a single model.

various long-context generative DNA models.

various long-context generative DNA models.

various long-context generative DNA models.

transformer family trained on multispecies genomes; strong on regulatory element prediction.

various long-context generative DNA models.

alternative PLMs with different scaling/data tradeoffs.

ESM-1b, ESM-2 (15B params), **ESM-3** (multimodal: jointly reasons over sequence, structure, and function via discrete tokenization; published in *Science* 2025), ESM-C (efficient successor). The dominant family.

alternative PLMs with different scaling/data tradeoffs.

alternative PLMs with different scaling/data tradeoffs.

T5/GPT-style architectures for protein.

T5/GPT-style architectures for protein.

T5/GPT-style architectures for protein.

the active toolkit. Accuracy degrades sharply on novel folds without homologs.

language model + structure prediction for RNA; RhoFold+ is currently SOTA for single-RNA tertiary prediction.

the active toolkit. Accuracy degrades sharply on novel folds without homologs.

transformer-based, trained on 30-100M cells.

transformer-based, trained on 30-100M cells.

transformer-based, trained on 30-100M cells.

transformer-based, trained on 30-100M cells.