NYU Tandon's Orion Framework Lets AI Train on Encrypted Data Without Ever Decrypting It

Imagine a hospital feeding patient records into an AI diagnostic model, and the model never once sees the actual data. Not a redacted version, not a synthetic substitute: the genuine, sensitive records, processed in their fully encrypted state, producing accurate results without a single byte of plaintext ever touching the compute layer. That is not a hypothetical from a research wishlist. It is what the NYU Tandon School of Engineering demonstrated with a framework called Orion, and the implications for every industry that runs AI on data it legally cannot expose are enormous.

What Orion Actually Does (and

Why It Is Hard) The technique at the center of this work is fully homomorphic encryption, or FHE. The concept has been around for decades in cryptographic theory: it allows computations to be performed directly on encrypted data, so the result, when decrypted, matches what you would have gotten if you had computed on the plaintext version. The problem has always been speed. As IEEE Spectrum has reported on the trajectory of FHE research, the technology has historically been thousands of times too slow for practical deployment, which is precisely why it has stayed inside research labs rather than shipping in production systems. Orion is the NYU Tandon team's answer to that performance wall. PhD students Austin Ebel and Karthik Garimella, working with Assistant Professor Brandon Reagen, built a framework that enables secure neural network computation without sacrificing accuracy, according to the NYU Tandon School of Engineering. The headline demonstration is striking: the researchers achieved the first-ever high-resolution FHE object detection run using YOLO-v1, a deep learning model with 139 million parameters. Running a model of that scale under full homomorphic encryption, without degrading the output, is the kind of result that moves FHE from theoretical curiosity to plausible engineering tool. > "Encryption breakthrough lays groundwork for privacy-preserving AI models. New AI framework enables secure neural network computation without sacrificing accuracy." (NYU Tandon School of Engineering) The team's work earned a Best Paper award at ASPLOS 2025 in Rotterdam, one of the most competitive systems architecture venues in computer science, a recognition noted publicly by NYU Senior Vice Provost for Research Juan J. de Pablo on April 3, 2025. A Best Paper at ASPLOS is not a participation ribbon. It means the systems research community, the people who build the hardware and software stacks that everything else runs on, looked at this work and decided it moved the needle.

Why This Matters

for Regulated Industries Right Now The privacy problem in production AI is not abstract. Healthcare organizations subject to HIPAA, financial institutions navigating GLBA and a thickening tangle of state-level regulations, and any enterprise handling personal data under GDPR face a structural tension: the most useful AI models require large volumes of sensitive training data, but exposing that data to a training pipeline creates legal and security liability. The standard workarounds, anonymization, differential privacy, federated learning, each involve tradeoffs in either data utility or model accuracy. FHE, if it can be made fast enough, removes the tension entirely. The data never leaves its encrypted state, so the compliance question changes from "how do we protect data during processing" to "we never processed unencrypted data in the first place." That is a genuinely different compliance posture, and regulators are increasingly interested in technical controls that provide cryptographic guarantees rather than policy promises. Privacy-preserving AI techniques including FHE are already being explored across sectors where data sensitivity is non-negotiable, according to XenonStack's overview of privacy-preserving AI approaches. The gap between "being explored" and "deployed in production" has historically been the performance problem. Orion's 139-million-parameter demonstration does not close that gap overnight, but it redraws the boundary of what is feasible.

The Infrastructure Problem That Comes Next

A framework demonstration, however impressive, is not an infrastructure ecosystem. This is where the NYU Tandon team's parallel work becomes relevant. The same institution is leading a separate initiative funded by a 3.8 million dollar grant from the National Science Foundation, in collaboration with Stanford University and the City University of New York, to build open-source hardware design tools for privacy-preserving computation under a platform called Cryptolets, according to NYU Tandon's announcement of the project. The premise of Cryptolets is direct: today's most advanced cryptographic computing technologies are trapped in research labs by one critical barrier, they are thousands of times too slow for everyday use. The NSF-funded project aims to build the missing infrastructure, specifically hardware-level design tools, that could make those technologies practical at scale. Taken together, Orion and Cryptolets represent two layers of the same ambition: prove the computation works at meaningful scale, then build the chip-level tooling to make it fast enough to run outside a research environment. > "Today's most advanced cryptographic computing technologies, which enable privacy-preserving computation, are trapped in research labs by one critical barrier: they're thousands of times too slow for everyday use." (NYU Tandon School of Engineering) For learners building toward careers in AI security, privacy engineering, or compliance technology, this is the frontier worth watching. The interesting engineering problems here sit at the intersection of cryptography, computer architecture, and machine learning systems: exactly the kind of cross-disciplinary territory that produces genuinely difficult and genuinely rewarding work.

What It Actually Means for You If

you are studying AI, security, or both, Orion is a useful case study in what real privacy engineering looks like. It is not a privacy policy. It is not a checkbox on a compliance form. It is a cryptographic guarantee enforced at the mathematical level, meaning an attacker who intercepts the computation gets nothing useful, because the data was never decrypted to begin with. That distinction matters enormously as AI adoption accelerates in healthcare, finance, and any domain where the underlying data carries legal protection. For learners in these fields, the practical takeaway is this: the tools for privacy-preserving AI are maturing faster than the curricula that teach them. FHE, secure multi-party computation, and differential privacy are no longer purely academic topics. They are becoming engineering requirements, and the institutions that produce practitioners who understand them have a significant advantage. NYU Tandon's Orion work, presented at ASPLOS 2025 and backed by NSF infrastructure funding, is a signal that the field is moving from theory to systems. The next step is watching whether the hardware tooling from Cryptolets can bring FHE performance close enough to conventional inference to make it a realistic default for sensitive deployments, not just a proof of concept. That is the development worth tracking as this research matures.