InstaDeep Launches Nucleotide Transformer v3: A Multi-Species AI Model for Long-Range Genomic Analysis

Can artificial intelligence models trained on trillions of DNA base pairs unlock new insights into genomic regulation across diverse species, from humans to plants?

Revolutionizing Genomics Through AI-Driven Foundation Models

The introduction of Nucleotide Transformer v3 (NTv3) by InstaDeep represents a significant step in applying transformer-based architectures to genomics. This foundation model processes up to 1 million base pair (Mb) contexts at single-nucleotide resolution, enabling unified tasks such as representation learning, functional track prediction, genome annotation, and controllable sequence generation. By integrating self-supervised pretraining with supervised fine-tuning, NTv3 addresses the challenge of connecting local DNA motifs with broader regulatory contexts, potentially accelerating research in molecular biology and personalized medicine.

Model Architecture and Technical Specifications

NTv3 employs a U-Net-inspired architecture tailored for extended genomic sequences. It features a convolutional downsampling tower to compress input data, a central transformer stack to capture long-range dependencies, and a deconvolutional upsampling tower to restore base-level resolution for outputs. This design allows the model to handle sequences that are multiples of 128 tokens, using character-level tokenization over the nucleotides A, T, C, G, N, along with special tokens like , , , , , and . The model family spans a range of sizes to balance computational efficiency and performance:

• The smallest variant, NTv3 8M, contains approximately 7.69 million parameters, with a hidden dimension of 256, feed-forward network (FFN) dimension of 1,024, 2 transformer layers, 8 attention heads, and 7 downsampling stages.
• Larger models, such as NTv3 650M, scale up to 650 million parameters, featuring a hidden dimension of 1,536, FFN dimension of 6,144, 12 transformer layers, 24 attention heads, and species-specific conditioning layers for targeted predictions.

Training Data, Performance Benchmarks, and Generative Applications

NTv3’s pretraining phase utilized 9 trillion base pairs from the OpenGenome2 dataset, covering over 128,000 species, through masked language modeling at base resolution. This was followed by post-training on approximately 16,000 functional tracks and annotation labels from 24 animal and plant species, incorporating a joint objective that blends continued self-supervision with supervised signals across about 10 assay types and 2,700 tissues. Performance evaluations demonstrate NTv3’s superiority on the newly introduced Ntv3 Benchmark, comprising 106 long-range, single-nucleotide resolution tasks across species and assays, using standardized 32 kilobase (kb) input windows.

The model achieves state-of-the-art accuracy in functional track prediction and genome annotation, outperforming baselines like sequence-to-function models and earlier genomic foundation models on both public benchmarks and this suite. For instance, post-training enables coherent inference of regulatory grammar transferable between organisms, highlighting the value of multi-species exposure during training. Beyond prediction, NTv3 supports fine-tuning as a controllable generative model via masked diffusion language modeling. Conditioning on signals for desired enhancer activity and promoter selectivity allows it to infill masked DNA spans. In validation experiments, the model generated 1,000 enhancer sequences, which were tested in vitro using STARR-seq assays in collaboration with external labs. Results indicated recovery of intended activity level orderings and over twofold improvement in promoter specificity compared to baselines, suggesting practical utility in designing regulatory elements.

Broader Implications for AI in Genomics

The development of NTv3 underscores a trend toward scalable, multi-modal foundation models in bioinformatics, where pretraining on vast, diverse datasets enhances zero-shot transferability. With genomics data volumes projected to exceed exabytes by 2025, models like NTv3 could streamline annotation pipelines and reduce experimental costs, potentially impacting fields from crop engineering to rare disease diagnostics. The emphasis on controllable generation opens avenues for hypothesis-driven sequence design, though challenges remain in generalizing beyond the 24 supervised species and ensuring ethical handling of multi-species data.

• Statistical Edge: NTv3’s post-training yields SOTA results on 106 benchmark tasks, with cross-species transfer reducing the need for organism-specific retraining.
• Resource Efficiency: Smaller variants (e.g., 8M parameters) enable deployment on standard hardware, democratizing access for academic researchers.
• Validation Metrics: In enhancer design, generated sequences achieved >2x promoter specificity, validated through in vitro assays, indicating reliable predictive power.