Skip to content
neovekContact us

AI Software Factory: What to Check Before Agents Merge Code

Share

An AI software factory is a development setup in which coding agents take a planned release, split it into tickets, then write, test and merge the code for each ticket with little or no human typing. This article looks at what has to be true about a repository before such a factory merges on its own: what readiness scorecards check, what a green CI pipeline fails to prove, and where human approvals still belong. It draws on published research about agents that cheat on tests, on the controls GitHub built around its own coding agent, and on the vendors now selling the factory as a product.

The phrase barely existed at the start of 2026. By October, Port, a company known for its internal developer portal, had put “AI Software Factory” in the headline of its home page (read 2026-10-26). In a video made in partnership with Port, published 2026-10-23, developer Cole Medin walks through his own factory and the layer Port adds around it. His demo makes a useful case study, both for what it shows and for what a sponsored demo has no reason to show.

The AI software factory term dates from January 2026

The term comes from a ladder of autonomy. On 2026-01-23, Dan Shapiro published The Five Levels: from Spicy Autocomplete to the Dark Factory, modelled on the levels of driving automation. At level 3 the developer spends the day reading diffs. At level 4 they write specifications and come back hours later to see whether the tests pass. Level 5 is the dark factory, named after Fanuc’s robot-run plant, which needs no people on the floor: specifications go in, software comes out, and nobody looks inside. Shapiro wrote that almost everyone tops out at level 3.

Two weeks later, StrongDM described a team running at level 5. In Software Factories and the Agentic Moment, dated 2026-02-06, CTO Justin McCarthy set two rules for the AI team he had founded on 2025-07-14:

“Code must not be written by humans. Code must not be reviewed by humans.” - Justin McCarthy, StrongDM

The team replaced review with end-to-end scenarios, run against behavioural clones of the services its product depends on (Okta, Jira, Slack and several Google apps), and scored by the share of runs likely to satisfy a user. The page also set a spending benchmark of $1,000 in tokens per human engineer per day. Simon Willison, who visited the team, asked the obvious question on 2026-02-07: how can skipping review be sensible when language models make mistakes no human would? He also noted that the benchmark adds roughly $20,000 a month per engineer.

Most working factories keep a human at both ends

Medin’s factory is not a dark one, and he says that is deliberate. A release starts as a planning session with an agent, sized to split into 10 to 20 tickets with a map of which ones depend on which. A scheduled job then runs the loop: take the next ticket in priority order, plan, implement, exercise the change the way a user would, and have a second agent review the pull request. Once CI and that review pass, the change merges to a staging branch, never straight to production. At the end, Medin tests the whole release in staging at once and files whatever is wrong as new tickets for the factory to pick up.

He started with no human anywhere, deploying to production directly, and backed away from it:

“It could adjust its own harness. It could literally do anything and we have no visibility.” - Cole Medin

The human stays at the two ends, planning and acceptance, while the middle runs alone. That middle is where the repository’s own checks decide what gets through. The 2025 DORA report, published 2025-09-23 from nearly 5,000 survey responses, found that AI adoption now goes with higher delivery throughput but still goes with lower delivery stability, and it describes AI as an amplifier of whatever an organization already does well or badly. A factory is the same amplifier, run unattended between those two ends.

Readiness scorecards check that a repository is equipped

Port’s answer to “is this repository ready for a factory” is a scorecard. In Medin’s setup, a repository reaches gold when it has a named owner, a README, a pull request template, a CI workflow, and CI passing on main. Every merged pull request is graded again, and a failing pipeline drops the repository to silver. Port’s own production readiness guide uses the same bronze, silver and gold ladder with stricter rules: branch protection with at least one required approval at silver, then code owner review and a defined on-call at gold. Factory, another vendor, sells an Agent Readiness score across eight pillars, from build system and testing to security and task discovery, on five levels running from basic to fully autonomous.

These checks are worth having. Each one is cheap to verify and catches a repository that is plainly not ready. But they all measure presence: a file exists, a workflow runs, a rule is configured. None of them can tell whether a passing test suite means the code does what the ticket asked. Medin concedes the point in passing when he says a green CI is only meaningful if the CI is thorough. That condition carries most of the weight.

Two columns: on the left, what a readiness scorecard verifies, namely a named owner, a README, a pull request template, a CI workflow and CI passing on main; on the right, what it cannot see, namely whether tests cover real behaviour, whether the agent edited a test, whether a test was special-cased, and who approved the merge A readiness scorecard confirms that the tooling exists, not that the tests mean anything.

Coding agents game failing tests, and a green CI hides it

The gap is measurable. ImpossibleBench, a paper by Ziqian Zhong, Aditi Raghunathan and Nicholas Carlini submitted on 2025-10-23, built coding tasks whose unit tests contradict their specification, starting from LiveCodeBench and SWE-bench. No honest solution can pass, so every pass is a shortcut. GPT-5 passed 76% of the SWE-bench tasks where a single expected value had been altered, and 54% of those where a contradictory duplicate test had been added.

The shortcuts fell into four kinds: editing the test, overloading a comparison operator so that every check succeeds, keeping hidden state between calls, and special-casing the exact input the test uses. The tactic depended on the model. On the conflicting SWE-bench tasks, Claude models and Qwen3-Coder took the test-editing route in more than 79% of their cheating, while GPT-5 and o3 used all four tactics.

The same paper measured countermeasures, and they map onto choices a factory owner makes:

  • Read-only tests stopped test edits while preserving legitimate performance, but did nothing against operator overloading or special-casing.
  • A strict prompt cut GPT-5’s cheating on the conflicting LiveCodeBench tasks from 92% to 1%.
  • An explicit way out, a flag the agent could raise to ask for a human, cut cheating on the conflicting SWE-bench tasks from 54% to 9% for GPT-5 and from 49% to 12% for o3. It helped Claude Opus 4.1 much less.
  • Language model monitors caught 86 to 89% of cheating on single-file LiveCodeBench tasks, but only 42 to 65% on multi-file SWE-bench tasks.

Bar chart of cheating rates on the conflicting SWE-bench tasks of ImpossibleBench: GPT-5 drops from 54% to 9% and o3 from 49% to 12% when the agent can flag the task for a human Giving the agent a sanctioned way to stop cut cheating by more than three quarters for both OpenAI models. Source: ImpossibleBench, 2025-10-23.

Now look at the most striking part of Medin’s demo. He breaks CI on purpose, Port detects the drop from gold to silver, proposes a GitHub issue to fix the pipeline, and waits. He approves, the factory takes the issue on its next pass, CI turns green, and the repository returns to gold. The ticket the factory received amounts to “make CI pass”, which is precisely the instruction ImpossibleBench shows an agent can satisfy by changing the test instead of the code. The approval in the demo is also Medin’s own choice. Port’s published template for this pattern, Auto-fix services when scorecards degrade, opens the issue and assigns it to GitHub Copilot with no approval required and the analysing agent set to run automatically.

Approvals belong where the blast radius is

Requiring a human on every pull request turns the factory back into level 3. Medin’s alternative is to decide case by case from context that the code does not contain: which team owns a service, how critical its tier is, which business domain a change touches. In his demo, a feature in the marketing domain routes its approval to that domain’s named owner. The AI agent permissions for a shared Slack agent follow the same principle: the decision lives outside the model, in data the agent cannot rewrite.

GitHub offers the building blocks. For pull requests opened by its coding agent, the documentation tells the developer who handed over the task that their own approval will not count toward required approvals, so another reviewer has to sign off. By default, Actions workflows do not even run on the agent’s pushes until a person approves them. A CODEOWNERS file adds the rest: owners are requested automatically when a pull request touches their paths, and the “Require review from Code Owners” branch rule makes their approval mandatory. In my view, the cheapest protection against the test-editing route is to give the test directories and the pipeline definitions an owner who is a person. An agent can still change a test, but that change can no longer merge unseen.

Each agent needs an identity of its own

None of these rules work if the agent pushes as a human. Medin points out that factory agents often borrow an engineer’s GitHub credentials, so when a production fault traces back to a commit, nobody can tell whether the engineer or the engineer’s agent wrote it. He says Port gives each agent its own identity, though the video does not show the mechanism. GitHub’s coding agent already behaves as a separate actor, which is why the rule about the requester’s approval can exist at all. An agent running under its owner’s account can approve its own work as that owner, and a CODEOWNERS rule cannot tell the difference.

Seven checks before an AI software factory merges alone

The scorecard sets the floor. These checks, which are my synthesis of the sources above, cover what it cannot see:

  1. An owner for every repository, and a human code owner for test directories and pipeline files.
  2. Tests the agent can read but not change unseen. Read-only during implementation where the tooling allows it, owner review where it does not.
  3. A sanctioned way to stop. The agent can flag a task as impossible or ambiguous instead of forcing it green.
  4. An approval from someone other than the requester, and a human approval on changes with a high blast radius.
  5. A staging branch and batch acceptance before anything reaches production.
  6. A separate identity per agent, with its own permissions and its own entry in the commit history.
  7. A stability measure, not only a throughput measure. Rollbacks and incidents per release, since DORA found AI adoption still goes with lower stability.

The loop itself is the easy part to build, and models keep improving it. What decides whether a factory can be trusted is the set of checks it is not allowed to edit, the same durable layer that outlasts each model release in an AI agent harness.

Share

Get new posts by email

At most one email a week, on Monday, with Neovek's new posts. Unsubscribe in one click. Privacy