RNA Foundation Models for Oncology: ML Explainer

Picture a language model, but instead of predicting the next word in a sentence, it is predicting the next nucleotide in a strand of RNA pulled from a cancer patient's tumor. The vocabulary is only four letters long (A, U, G, C), but the sequences run tens of thousands of tokens, the grammar rules are deeply nonlinear, and a wrong prediction does not produce a mildly awkward sentence. It produces a missed splice site that could mean the difference between a protein that suppresses tumors and one that promotes them. This is the problem that Blank Bio, a computational biology startup, is now tackling in collaboration with PacBio, and their recently announced seed financing signals that the broader ML community is starting to take RNA seriously as a modeling domain.

For practitioners who have been living in text-and-image land, the RNA world can feel like a foreign country where all your favorite intuitions are technically correct but subtly wrong in ways that will embarrass you at conferences. Let us fix that.

What RNA Foundation Models Actually Are (And Are Not)

A foundation model, in the now-standard sense, is a large model pretrained on massive unlabeled data that can be fine-tuned for downstream tasks. You know this story. BERT learns language from text, then gets fine-tuned for sentiment analysis. The same logic applies to genomics: pretrain on enormous amounts of biological sequence data, then fine-tune for specific tasks like predicting splice sites, classifying tumor subtypes, or flagging isoforms associated with poor prognosis.

RNA foundation models specifically treat RNA sequences (or transcriptomic expression profiles) as the pretraining corpus. The insight is that RNA carries far more information than its reputation suggests. It is not just a passive messenger shuttling DNA instructions to ribosomes. RNA folds into three-dimensional structures, interacts with proteins, regulates gene expression, gets spliced in dozens of context-dependent ways, and behaves differently in a lung cancer cell than in a healthy lung cell sitting two centimeters away. A*STAR researchers recently demonstrated a novel technique to decode how RNA secondary structure influences disease states, reinforcing that structure-function relationships in RNA are rich modeling targets, not background noise to be filtered out.

What Blank Bio is building sits at the intersection of sequence modeling and transcriptomics: models that learn the statistical regularities of RNA biology at scale, then apply those representations to patient-level cancer prediction tasks. Think of it less like GPT-for-genomes and more like a very well-read bioinformatician who has internalized ten million transcriptomes and can tell you, with calibrated uncertainty, what a given RNA expression profile is probably doing.

Why Long-Read Sequencing Data Changes the Training Equation Here is where

PacBio's role becomes technically non-trivial and worth understanding properly. Sequencing technologies broadly split into two camps: short-read (Illumina dominates here, reading about 150 base pairs at a time) and long-read (PacBio and Oxford Nanopore, reading thousands to tens of thousands of base pairs in a single continuous pass). For most genomics work over the past decade, short-read won on cost and throughput. But for training RNA foundation models aimed at oncology, long-read has structural advantages that are hard to paper over.

The key issue is isoforms. A single gene can produce multiple distinct RNA transcripts depending on which exons get included or excluded during splicing. This is called alternative splicing, and it is not a rare edge case. It is pervasive. A recent Nature paper on modeling context-dependent RNA splicing using deep learning found that splicing decisions are highly sensitive to sequence context in ways that simple linear models systematically miss. Short-read sequencing struggles to assign reads to specific isoforms because the reads are too short to span multiple exon junctions unambiguously. You end up with a blurry aggregate signal where you actually needed a precise isoform-resolved picture. Long-read sequencing reads entire transcripts in one pass, which means the training data preserves full isoform structure. For a foundation model trying to learn the biological grammar of tumor transcriptomes, that is not a minor data-quality footnote. It is load-bearing.

Blank Bio's collaboration with PacBio is specifically about accessing and generating long-read bulk RNA sequencing data at the scale needed to train these models meaningfully. As Blank Bio stated in their announcement, the collaboration is intended to advance RNA foundation models for precision oncology by combining PacBio's sequencing capabilities with Blank Bio's modeling infrastructure. This is the kind of data-compute-model flywheel that any serious foundation model effort requires, and it is notable that the genomics world is now building the same infrastructure scaffolding that the NLP world built between 2018 and 2022.

The Tumor Transcriptome

as a Modeling Target Cancer is not one disease. It is thousands of diseases that happen to share a general mechanism (cells growing when they should not). This heterogeneity is exactly why transcriptomic modeling is attractive: gene expression profiles capture the specific biological state of a specific tumor in a specific patient, not just the mutation it inherited from a parent. Two patients with the same lung cancer diagnosis can have radically different transcriptome signatures, and those signatures predict treatment response in ways that genomic mutation data alone does not.

Training a foundation model on tumor transcriptomes means learning what normal RNA expression looks like, what cancer-specific dysregulation looks like, what treatment-associated changes look like, and crucially, how all of these patterns interact at the patient level. The downstream task here is not just tumor classification (though that is a starting point). The richer goal is patient-level prediction: which patients will respond to immunotherapy, which isoforms are driving resistance, which expression subtypes map to clinical outcomes.

This is adjacent to work happening in the broader multi-omics data space. Rancho BioSciences recently launched OmicsHQ, a curated multi-omics data platform designed to accelerate drug discovery, signaling that the industry recognizes clean, well-annotated biological data as a genuine infrastructure problem, not just a preprocessing inconvenience. The model is only as good as the transcriptomes you fed it, and the transcriptomes are only as informative as the sequencing technology used to generate them. The whole stack matters.

What ML Practitioners Should Actually Take Away

If you are an ML engineer or data scientist looking to understand this space or potentially work in it, a few things are worth internalizing that will save you from reinventing painful lessons the genomics field learned the hard way.

First, tokenization is not obvious. When you tokenize RNA sequences for a transformer-style model, the choice of k-mer size (how many nucleotides constitute one token) meaningfully affects what biological features the model can capture. Tokenizing at single nucleotide resolution preserves the most information but produces very long sequences. Larger k-mers reduce sequence length but potentially obscure functional motifs. This is a real hyperparameter with biological consequences, not just an efficiency knob.

Second, the label problem is severe. Unlike image classification where labels are relatively cheap to generate, labeled transcriptomic data tied to clinical outcomes requires actual patients, actual tissue samples, actual sequencing runs, and actual follow-up data collected over years. The data scarcity problem is not going away, which is exactly why self-supervised pretraining on large unlabeled corpora (the foundation model approach) is so appealing here. You pretrain on everything, fine-tune on the rare labeled subset.

Third, interpretability is not optional. In cancer diagnostics, a model that produces a prediction without a biologically coherent explanation is clinically useless and potentially dangerous. Attention maps and feature importance scores are not sufficient. The field increasingly demands mechanistic interpretability: can you point to specific RNA features the model weighted, and do those features correspond to known biology? This is a harder bar than most ML interpretability work is currently built for.

The collaboration between Blank Bio and PacBio is still early-stage, and the road from seed funding to clinically validated models is long and full of regulatory checkpoints that would make any startup founder's hair go gray. But the technical direction is sound, the data strategy is smart, and the problem is real. For learners interested in computational biology, genomics, or applied deep learning, this is one of the most technically rich domains you can start building skills in right now. Pick up a bioinformatics fundamentals course, get comfortable with sequencing data formats (FASTQ, BAM, GTF files are your new best friends), and read the RNA splicing deep learning paper out of Nature. The biology and the ML are finally speaking the same language.

In the meantime, the most honest summary of where RNA foundation models stand today: enormously promising, technically serious, and approximately four years behind text models in infrastructure maturity. Which, if you think about it, means the interesting engineering work is happening right now.