Alex Hao
Brian Hie
Evo 2 uses in-context learning to design de novo regulatory sequences with only tens of examples
Imagine composing a cell-type-specific enhancer from scratch and taking it directly from the computer to the mouse brain–no intermediates, no large-scale screen, just you and a handful of promising designs to test in vivo with a simple delivery protocol. Today, in a preprint out on bioRxiv, we reveal that Evo 2 is capable of designing–from scratch–enhancers, or cis-regulatory elements, that successfully target specific neuronal populations in the living mouse brain. Then, in a companion preprint on bioRxiv, we extend these findings to novel proteins, protein-RNA systems like CRISPR-Cas9 loci, epitope-specific antibodies, and whole bacteriophage genomes in silico.
Designing the spatiotemporal pattern of gene expression is highly desired in both basic biology and therapeutic engineering. It can be used to create targeted cancer therapies, drive cell-type-specific expression of actuators like channelrhodopsin, or generate neuronal subtype-level voltage sensors to acquire whole-brain recordings of highly specific circuits. In nature, enhancers are an endogenous mechanism for cell-type-specific gene expression. But designing cell-type-specific enhancers that behave as intended in animals is extremely challenging.
When we started this project, we knew Evo 2 had already designed chromatin accessibility patterns at the multi-kilobase scale. However, this result came from inference-time guidance, which is relatively slow and reliant on the model’s ability to generate elements from nature. We were looking for a higher-efficiency alternative, but it wasn’t clear how to interact with Evo 2 effectively to accomplish this goal, until we drew inspiration from natural language models.
A powerful way to prompt a natural language model is by giving it examples within the prompt itself, so that it can learn the pattern from just a few examples and apply that to a similar task without further training. In other words, the language model can learn in context. Like these models, Evo 2 is an autoregressive model, but instead of being trained on written words, it is trained on a large amount of genomic DNA. A natural next step is therefore to ask whether Evo 2 can also learn sequence design from just a few examples in its prompt context. There were hints that it could (we know, for example, that Evo 2 performs in-context learning for synthetic string-manipulation tasks), but nobody had shown that this capability extended from purely synthetic contexts to generating sequences with defined biological function. So, we set out to test that first.
In-context learning is an emergent property of Evo 2
Until now, getting Evo 2 to generate a particular class of sequence meant fine-tuning it on a large, labeled dataset. Conditioning Evo 2 to generate a phage, for instance, required training it on 15,000 examples from nature. But for many biologically-useful sequences, both natural and synthetic, there are only tens of cases. In the past, this has made them nearly impossible to train on. If Evo 2 could generate working sequences from a few examples in its prompt, it would be a huge time- and cost-efficient solution to this design problem.
To test Evo 2’s capability, we focused on non-coding DNA as a powerful genomic regulator, and we started by simply concatenating sequences from the same class (5’ UTRs, IRESs, promoters, and enhancers) together in the model’s prompt. Strikingly, as we fed Evo 2 with more and more in-context examples, the novel sequences gained higher and higher predicted performance in each class. Since natural genome sequences do not have a series of explicitly delineated regulatory elements, the model was picking up on a pattern that was not seen in its training data–providing further confirmation that Evo 2 is capable of in-context learning on biologically meaningful sequence-generation tasks. From less than 100 in-context examples, Evo 2 had generated functional sequences spanning five regulatory modalities with 30-50% yield and <60% sequence similarity.
Nobody built this into Evo 2. It is an emergent property that appears to arise from the scale of the model and the natural genomic data it was trained on. And Evo 2 seems to accomplish it in the same way as a language model would. We ran two separate tests to probe how.
First, we searched for induction heads, a known circuit for in-context learning in language models, where attention modules use repeated patterns to predict the next token. Studying the attention heads of Evo 2, we were able to isolate a few candidates, and ablating one actually impacted the performance of sequence design in Evo 2. In contrast, ablating another random head in the same layer caused essentially no change, something that is also seen in smaller-scaled natural language models.
Our second test traced how individual examples in context influence the generated sequence at nucleotide resolution (known as a categorial Jacobian). With just a few examples in the prompt, high-influence nucleotides were only delivered in a single example sequence. But, as more examples entered the context, high-influence nucleotides also became more distributed across the prompt. In other words, Evo 2 continued to integrate the examples we gave it, synthesizing information from multiple prompts rather than leaning on any one of them.
Incentivizing Evo 2’s in-context learning with virtual rewards
To see if we could improve both the efficiency and quality of the sequences generated using in-context learning, we turned to the idea of using rewards as incentives for the model. We repurposed several sequence-to-function models as reward models for Evo 2, combining it with other rule-based rewards to create a reinforcement learning loop that incentivized it to generate better biological sequences (try out Evo 2 in-context design with this design tool).
For tasks where Evo 2 already does pretty well, reinforcement can take less than 20 minutes. For more difficult tasks, it can still take hours. But, either way, it doesn’t require intensive training (e.g. training a new Evo 2 or large-scale fine-tuning), and the rewards can actually transfer in some cases. When we post-trained Evo 2 on a human promoter dataset, it also boosted the performance of enhancer design, even though we didn’t give it any information on enhancers during the reinforcement learning. And when we post-trained on two cell types simultaneously, holding out a third, the predicted specificity rose in all three. Reinforcement learning had increased specificity even in a cell type that the model was never trained on.
From the computer straight to the mouse brain
The brain has the highest-cell type diversity in the body, and neurons are extremely heterogenous, meaning that labeling a single neuronal subtype in the cortex is quite a feat. We wanted to see whether Evo 2 was up to the task: could it design neuron-specific enhancers that could be easily delivered to the mouse brain?
Because there was no existing large-scale predictive model for scoring whether a DNA sequence might work in a mouse, we independently fine-tuned AlphaGenome to reward Evo 2 and used fine-tuned Enformer and Borzoi models to evaluate the designs. Together, they predicted the chromatin accessibility pattern across 107 neuron types in the human brain. We then packaged nine designs into AAVs for delivery into the mouse brain via a simple retroorbital injection. Out of the nine designs tested, seven drove detectable expression and six produced spatially localized labeling. With only tens of designs, two were able to specifically label layer 5 extratelencephalic neurons with 73% and 95% specificity, at the level of or higher than the current most specific synthetic enhancer on the market.
Given that we designed this for a cell type in the human brain, it was surprising to see this work in the mouse brain as well, and it suggests that there is some conservation across evolution in the brain’s regulatory codes. While we targeted layer 5 in this publication, we plan to extend this work and generate specific and strong enhancers for more cell types in the brain.
More broadly, in-context learning could transform pre-trained genomic language models into universal generators of biological sequences, including DNA, RNA, and protein. In fact, in a companion preprint, we show that this same in-context learning extends well past regulatory DNA: prompting Evo 2 with as few as four examples generates novel proteins, protein-RNA systems like CRISPR-Cas9 loci, epitope-specific antibodies, and whole bacteriophage genomes. Since these example sequences can range from hundreds to tens of thousands of bases, we also developed Evo2-flash, an efficient inference engine for Evo 2 that alleviates the burden of using long-context prompts. It generates sequences up to 35x faster than before and runs on GPUs available in Google Colab.
At this stage, designs beyond enhancers are computational and have not yet undergone experimental testing. Still, together, our work suggests that in-context learning is a general property of Evo 2 that can be applied across the central dogma. And the implications extend beyond generation: it may be possible to leverage in-context learning for predictive tasks in data-scarce regimes, opening up a whole new range of applications for Evo 2.
Hao, Y. A., Liu, M., Nori, D., Brixi, G., Lau, T., Ramakrishnan, C., Chang, D., Lee, R., Allen, W. E., & Hie, B. L. (2026). Few-shot regulatory sequence design with a genomic language model. bioRxiv. https://doi.org/10.64898/2026.10.28.762639
Brixi, G., Hao, Y. A., Nori, D., Merchant, A., Widatalla, T., King, S. H., Shalvarjian, K. E., Park, D. M., & Hie, B. L. (2026). Efficient biological design via genomic in-context learning. bioRxiv. https://doi.org/10.64898/2026.10.28.762643
Evo 2 In-context Design Tool: https://proto.evodesign.org/tools/evo2-icl
Evo2-flash on Github: https://github.com/evo-design/evo2_flash
Alex Hao is a Postdoctoral Researcher in Chemical Engineering in the Hie Lab at Stanford University and Arc Institute.
Brian Hie (X: @BrianHie), an Assistant Professor of Chemical Engineering at Stanford University, the Dieter Schwarz Foundation Stanford Data Science Faculty Fellow, and Arc Institute Innovation Investigator in Residence.