What can we learn from autonomous software factories?

TL;DR: AI agents are making code cheap, but testing and human review are becoming more expensive in cost, compute and attention. Inspired by Yossi Eliaz's research into autonomous software factories, I explore three principles: compute once, count once, publish once.

What can we learn from his work? We can use foundational engineering and CI/CD practices, including caching and collecting immutable evidence from diverse sources, to build better pipelines. These pipelines should provide clear evidence of what agents (and humans) have actually completed and make their work easier for humans to verify. More automation shouldn't mean more invisible work for engineers.

One of the best discoveries from Cambridge, bar the Chelsea buns and a surprising number of bike lanes, has been talks.cam.ac.uk, a central repository of public and semi-public talks happening across the university. From linguistics and Roman history to Middle Eastern art and computer science, there's something interesting happening almost every day (and you, dear reader, can access some of these too).

Amidst all the Hugging Face palaver, I attended (virtually, having failed to find the room) a seminar by Yossi Eliaz on Forkable Sandboxes: The Runtime Layer for AI Software Factories, hosted by the Cambridge Systems Research Group on 15 October 2026.

Eliaz's research explores self-driving compute: infrastructure and environments capable of exploring alternative solutions without unnecessarily repeating work. One application is building more efficient and verifiable autonomous software factories, where agents can propose changes, execute experiments and evaluate results safely and securely.

So let's wonder: what can those of us who aren't building the latest AI factory at the mythical research frontier apply to human-agent hybrid development in more conventional software?

Too many agents spoil the broth

Once upon a time, Amazon popularised the idea of a team small enough to feed with two American pizzas. In this engineering fairytale, a developer picks up a ticket (hopefully a discrete piece of work), creates a branch, writes some code and tests, opens a pull request (PR), and waits for CI to validate it. Git manages changes, while tools such as GitHub Actions coordinate builds, security scans, tests and deployments.

Fast-forward to today, and a mixture of humans and agents can produce an extraordinary amount of code. I remember mourning the death of the PR as I reviewed yet another AI-generated change, only for my comments to be absorbed by another AI with seemingly minimal context of the situation at hand. Fighting AI code enthusiasm with AI PR review might be one solution, but surely there must be a better, more robust way. The crux is that code is becoming cheaper to produce, but verification remains expensive and human-centred. Amongst other things, Eliaz's framework offers three principles for this dilemma: compute once, count once, publish once.

Compute once: stop repeating expensive work

Caching is not new. We use it to reuse dependencies, compiled artefacts and build outputs. Eliaz's proposal extends this principle to execution states through entire forkable sandboxes.

Rather than repeatedly preparing an environment for every proposed change, Eliaz creates a checkpointed, prepared environment and allows multiple agents to explore alternative solutions independently. He illustrates that restoring execution states can outperform cold starts while remaining slightly slower than an existing warm cache. The opportunity here is to identify where we can eliminate computation and create safe, ephemeral sandboxes for exploration on top of our existing infrastructure.

Thinking about the Hugging Face incident earlier this year, sandbox isolation is a key security consideration, particularly when agents have a little too much creative freedom in pursuing their goal-seeking tendencies. Isolation alone does not guarantee safety, but combined with credential boundaries, access controls and monitoring (it should always be supervised), it can help contain experiments.

Count once: what sort of evidence?

Suppose we ask twenty agents to investigate a bug (why we're asking twenty agents to look at one bug is another question, but alas). Ten identify the same failure, five discover another and the remaining five return inconclusive results. Have we discovered fifteen problems, or repeatedly observed just two?

Eliaz's second principle concerns aggregating results without double-counting observations. He proposes evidence contracts: structured and standardised records describing what was checked, the resulting artefact, who or what reported it, and which observations contributed to the result.

In conventional software development, we already capture similar information through test reports, logs and commit hashes. In regulated environments, these artefacts are frequently stored in evidence stores and associated with commits and deployments.

For agentic software development, I think there's an opportunity to standardise evidence contracts for discrete items of work, such as bug fixes, rather than accepting a natural-language assertion from an agent that the work has been completed.

By bringing together the original failure, code, test results, execution logs and resulting artefact—ideally produced or verified by systems independent of the agent itself—we could provide a holistic view of the agentic change. This would make agent-generated work more auditable while allowing CI pipelines to make informed decisions about which checks need repeating and which results can safely be reused.

Publish once: who or what decides what gets merged?

If ten agents propose ten correct solutions, we don't want ten changes merged into our codebase. Instead, we need to evaluate the candidates, determine which satisfy our acceptance criteria and ensure that only approved results are published.

Eliaz's publish once principle concerns the integrity of this final step. Only verified artefacts should survive beyond temporary sandboxes, and outdated workers shouldn't be able to overwrite newer accepted results. Think The Hunger Games for candidate changes, except the winner should be determined by evidence. This is where acceptance criteria and evidence contracts need to work together. The evidence contract establishes what happened and what was tested. The acceptance criteria determine whether that evidence is sufficient to approve the result. Only then should the accepted artefact be promoted into the codebase or delivery pipeline.

Meeting people where they are: making human verification easier

In AI agent–human hybrid development, discussions about agentic development often assume there will be a human in the loop, conveniently available to review the work and take responsibility when a feature behaves in weird and wonderful ways. But human review cannot scale at the same rate as code generation, and I don't think we should expect it to.

I'm currently reading Kate Crawford's book Atlas of AI, where she examines the often invisible human labour underpinning supposedly autonomous AI systems. While Crawford's critique concerns forms of labour and exploitation quite different from software engineering, there is a parallel.

We celebrate agents autonomously producing entire features, while engineers are left reviewing enormous PRs, correcting mistakes, reconstructing missing implicit context and, at times, verifying whether the software (and the tests) actually work. The human labour hasn't disappeared. We've simply moved it further down the development lifecycle.

It makes me sceptical of how readily we invoke the human in the loop as a governance safeguard. Is the engineer exercising meaningful judgement, or performing the repetitive work necessary to make an unreliable system appear autonomous? I don't think the solution is to remove humans from the process. Instead, we should design our technical and organisational infrastructure to make their participation easier and more valuable.

For example, Playwright's headed mode can visibly demonstrate how an application behaves. With Vercel's ephemeral preview environments, we can give business colleagues a temporary, working version of a feature to explore without requiring them to review code, interpret test reports or understand the deployment infrastructure. Engineers, meanwhile, could inspect the automated test results, traces and artefact provenance.

Some final thoughts

There is already evidence that traditional developer infrastructure is under pressure to augment and change for agentic times. In October 2026, Anthropic described its migration away from Git to an in-house version-control system after the volume of merged changes grew exponentially. Of course, most organisations are unlikely to be in the market to build their own source-control systems.

For those of us mere mortals using existing platforms, I think there's plenty we could improve without reinventing Git. Better caching, independent evidence collection, clearer acceptance criteria and accessible preview environments are all places to start.

The point from Eliaz's talk that stayed with me most was the need to make evidence and acceptance first-class citizens in computational infrastructure for deployment. Rather than simply running tests and returning a green tick, we should define what evidence is required, which systems produce it and how that evidence establishes successful completion.

Crawford's critique also reminds us to be more measured in how we evaluate productivity. If an agent produces a feature in minutes but an engineer spends hours making it safe, understandable and deployable, how much work have we really automated?

And perhaps, after all these years of maintaining CI/CD pipelines, the humble test suite is about to become more important than ever. It might just need a better user interface.

Further reading: Yossi Eliaz · Cambridge seminar · Anthropic's source-control redesign · Hugging Face incident analysis · Kate Crawford, Atlas of AI (2021)

Disclaimer: This post reflects my own interpretation and learnings from Dr Yossi Eliaz's talk and my further reading. It is not intended to be a comprehensive summary of his research or necessarily the central argument of his seminar.