Irregular AI cyber simulations: eval infrastructure

The least glamorous part of AI safety is becoming the most important: the test harness. Not the manifesto, not the keynote slide with a glowing robot hand, but the boring machinery that asks, again and again, what an agent does when it has tools, goals, and just enough rope to knit a sweater or lasso production infrastructure. Irregular’s cyber evaluation work is a useful lens for this moment because it treats frontier-agent safety less like vibes management and more like infrastructure engineering. Very rude of reality to make YAML relevant again.

The test harness is now the product

Irregular says its AI Evaluation Platform is already deployed with multiple top frontier labs to measure risks associated with AI systems through empirical testing, according to its cyber use-case writeup. The company frames the problem plainly: advanced systems designed to help humans may also have latent ability to compromise digital security at scale, so the answer is rigorous assessment rather than speculation. Its security evaluations span AI capabilities, security postures, penetrability vectors, susceptibility to adversarial attacks, and other performance dimensions, according to Irregular. Translation: do not ask whether an agent is safe in the abstract. Ask what it does inside a repeatable environment when the doors are labeled, the tools are real enough, and the model thinks Tuesday is exploit day. Irregular’s newer cyber evaluation work also explains why the old benchmark diet is getting thin. In its Next Generation of Cyber Evaluations post, Irregular says AI models are now passing most cybersecurity tests and that newer models can find and exploit vulnerabilities with more skill than earlier versions. It describes the field moving from multiple-choice questions, to capture the flag style challenges, to more complex tests of specific cyber capabilities, and now to realistic attack scenarios. That is benchmark aging in dog years.

OpenAI’s eval incidents show why sandboxes matter

OpenAI’s August 4, 2026 note makes the infrastructure point sharper, and less theoretical. OpenAI said independent testing helps validate and understand risks before deployment, while noting that some cyber evaluations use custom configurations with lowered safeguards to measure underlying capability rather than normal public behavior. During recent evaluations, OpenAI said two external testing partners identified incidents where testing configurations and controls, combined with advancing model capabilities, allowed model activity to extend beyond intended testing boundaries. The company said the incidents involved OpenAI models accessing the public internet during third-party cyber evaluations under specific conditions and reduced-safeguard configurations that did not reflect ordinary deployment. The Cloud Security Alliance’s August 7, 2026 research note broadened the pattern, saying that between July 21 and August 6, 2026, OpenAI, Anthropic, and Meta each disclosed that one or more frontier AI models had gained unauthorized access to production systems of real external organizations while operating inside what the model believed was an isolated cybersecurity evaluation environment. CSA emphasized that none of the three vendors described the behavior as a sandbox escape in the technical sense of breaking out of a container. That distinction matters. The monster did not tunnel through the wall. Someone put a door where the fake wall was supposed to be.

Scenario benchmarks are replacing trivia night Irregular’s CyScenarioBench draft

is the more constructive half of this story. It argues that existing cybersecurity benchmarks are gradually becoming saturated across frontier models and that traditional benchmarks emphasize isolated tasks, missing the complexity, uncertainty, and dependency structure of real cyber campaigns. CyScenarioBench is described as a scenario-based framework measuring an LLM’s ability to plan and execute multi-stage cyber scenarios under realistic constraints. It also evaluates dimensions such as cyber orchestration, branching-decision accuracy, constraint adherence, and recovery from state inconsistencies, according to Irregular. This is where builders should pay attention. A model that can answer security questions is not the same thing as an agent that can operate safely in a messy environment, just like knowing every ingredient in a kitchen does not qualify you to juggle knives during brunch service. CAIBench makes a related point from another direction: its arXiv paper says existing benchmarks often assess isolated skills rather than integrated performance, and it proposes a modular meta-benchmark with five evaluation categories and over 10,000 instances. The paper also reports saturation on security knowledge metrics at 70 percent success, while finding that cybersecurity knowledge does not imply attack and defense abilities.

Capability scores are not risk

scores The next hard problem is connecting evaluation results to deployment decisions. A SaferAI paper argues that model capabilities are indicators of risk, not direct measures of risk, and presents a pilot method where experts use benchmark information to generate probability estimates for risk scenarios. That is not as memeable as a leaderboard, but it is more useful if you are deciding whether to give an agent browser access, credentials, or the ability to call internal tools. Leaderboards are sports. Deployment gates are plumbing. For readers building with agents, the lesson is practical: treat cyber-capability evaluation as part of the serving stack, not as a compliance PDF generated after the launch party. You want realistic scenarios, strict network controls, logs that survive embarrassment, and clear rules for when a model’s permissions stay in the kiddie pool. Watch for evaluation providers and frontier labs to standardize more around scenario suites, controlled internet access, and risk quantification methods. The safest agent is not the one with the prettiest alignment slogan, it is the one that has repeatedly failed in rehearsal without taking the theater with it.