Continuous Evals: Coding Agents' Operating System

The old coding assistant bargain was simple: it suggested a line, you squinted, and maybe nobody got hurt. Now coding agents can call tools, modify state, and stumble through a repository like a very confident intern with root access. Vibes are no longer a QA strategy. They are a scented candle in a wind tunnel.

The useful framing from LetsLearnGenAI is that evals are becoming the operating system of AI-assisted coding. Not the shiny app layer, not the leaderboard wallpaper, but the control surface that tells teams what changed, what broke, and whether the agent should keep working or be handed a juice box and escorted from the build pipeline.

Anthropic Gives the Boring Definition We Need

Anthropic defines an eval as a test for an AI system: provide an input, then apply grading logic to the output to measure success. In its Jan 09, 2026 engineering post, Anthropic says good evaluations help teams ship agents more confidently by making failures and behavioral changes visible before they hit users. The company also points out why agents are harder to measure than chatbots: they operate over many turns, call tools, modify state, and adapt based on intermediate results.

@title Evaluation structure
@source Demystifying evals for AI agents

  Input
    │
    ▼
  AI system
    │
    ▼
  Output
    │
    ▼
  Grading logic
    │
    ▼
  Success measure

@caption Anthropic describes evals as input, output, grading logic, and success measurement.

That definition sounds almost offensively plain, which is why it matters. Repository agents are not merely predicting the next token, they are taking actions inside environments full of brittle tests, spooky dependencies, and one file named final_final_really.py. If you cannot replay a task and grade the outcome consistently, you are not adopting an agent. You are summoning one.

Task Type Is the Sneaky Variable

A task-stratified arXiv study compared OpenAI Codex, GitHub Copilot, Devin, Cursor, and Claude Code across 7,156 pull requests from the AIDev dataset. The paper found task type had a large effect: documentation tasks reached 82.1% acceptance, while new features reached 66.1%, a 16 percentage point gap that exceeded typical inter-agent variance for most tasks. It also reported that Devin showed the only consistent positive trend in acceptance rate, at 0.77% per week over 32 weeks, while the other agents were largely stable.

That is the part teams should tattoo on the inside of their CI dashboard. Choosing a coding agent by average benchmark score is like choosing a restaurant by the number of forks in the drawer. Your local eval suite should split tasks by the work you actually do: docs, bug fixes, refactors, migrations, tests, and new features. Otherwise, the agent that looks brilliant on easy maintenance chores may quietly eat your architecture like a raccoon in a server room.

Production Evals Need Production Shaped Tasks

The REAP paper argues that production deployment of AI coding agents needs fast, reproducible evaluation signals. It says online A/B testing can take weeks and risk user experience, shadow deployment does not produce reproducible signals across runs, and public benchmarks can diverge from real workloads in language distribution, prompt style, and codebase structure. REAP proposes automatically curating production-derived benchmarks from real developer-agent sessions without manual labeling.

This is the bridge from research evals to operating discipline. A useful team eval is not a museum of puzzle problems, it is a living regression suite built from the mess your codebase actually produces. The REAP paper also flags practical landmines: untestable prompts, misaligned tests, and flaky tests can compromise reliability. Translation: if your eval cannot tell a bad agent from a bad test, congratulations, you built a fog machine with YAML.

Benchmarks Are Necessary, Not Sufficient

ProjDevBench pushes on a different weakness: end-to-end project development. According to its arXiv abstract, the benchmark gives project requirements to coding agents and evaluates resulting repositories using Online Judge testing plus LLM-assisted code review. It covers 20 programming problems across 8 categories, evaluates six coding agents, and reports an overall acceptance rate of 27.38%.

That low acceptance rate is not a reason to panic, it is a reason to scope responsibly. The paper says agents handle basic functionality and data structures but struggle with complex system design, time complexity optimization, and resource management. For builders, the practical move is obvious: start with narrow, high-signal tasks where correctness can be checked, then expand only when your evals show the agent is improving. Autonomy without measurement is just autocomplete wearing a trench coat.

Keep the Human in the Loop, Ideally Awake

The Agents That Teach paper adds a softer but important failure mode: developer learning. It argues that as developers delegate substantial coding tasks to autonomous agents, incidental learning can be short-circuited, creating what the authors call Knowledge Debt. The paper proposes six design principles and presents SHIELD, a multi-agent system meant to surface contextual, out-of-band learning from the coding agent's own reasoning.

That matters because evals should measure more than whether the tests are green. Teams also need to ask whether developers can explain the change, maintain it, and notice when the agent confidently invents a tiny cathedral of nonsense. The next practical step is not a massive eval empire. Build a small local suite, run it on real tasks, stratify by task type, track regressions, and keep adding cases from production misses.

If the agent is going to drive, evals are the steering wheel, not the fuzzy dice.