AI Agent Harness: Which Parts Survive the Next Model Release

An AI agent harness is the software that turns a language model into an agent: the loop that calls the model, the tools it may use, the way context is trimmed and carried between sessions, the permissions that gate each action and the checks that decide when a task is done. This article looks at where performance and money actually sit in that layer, using benchmark data, Harvey’s path in legal AI and the moves the model labs made in 2026. It ends with a practical test for telling a durable harness business from one that the next model release will erase.

The question is live because founders and investors now treat the harness as the place to build. In a video listing thirteen kinds of businesses worth owning as agents spread, published on 2026-10-06, Greg Isenberg puts domain-specific harnesses on his list and presents them as the next chapter of a familiar story:

“I believe that harnesses are the new GPT wrapper.” - Greg Isenberg

His argument is that the best of the early wrappers did not die. They kept the model underneath and piled industry tools, workflow rules and memory on top until they became something sturdier. That claim is half right, and the half that is wrong matters more to anyone deciding what to build.

An AI agent harness is everything around the weights

Isenberg’s own image is a car: the model is the engine, and the harness is the rest of the vehicle. The analogy holds better than he lets on, because the vendors now describe their products in the same terms. Anthropic’s documentation presents its Agent SDK as Claude Code’s agent loop, tools and context management packaged as a library, and lists what comes with it: built-in tools, hooks that run custom code at points in the agent’s lifecycle, subagents, connections to outside systems through the Model Context Protocol, permissions that decide which tools run without approval, resumable sessions, and memory loaded from project files (Agent SDK overview). Anthropic’s engineering blog calls that same SDK a general-purpose agent harness (Justin Young, Anthropic, 2025-11-26), and OpenAI titled a February 2026 post about building software with Codex “Harness engineering”.

The term is settled at the labs. What it covers falls into three groups, and the rest of this article relies on the sorting:

• Mechanics: the loop, tool calling, context compaction, retries, sessions.
• Workarounds: prompt chains, task decomposition and checklists that exist because the current model fails without them.
• Commitments to a customer: who the agent may act for, what it may touch, which systems it writes to, and how a result is judged correct.

The grouping is mine, not a vendor’s. It matters because the three groups age at very different speeds.

The same model scores differently in another harness

The harness is not decoration: it moves results. A 2026 study, shown as a poster at an ICML workshop on coding agents, held two models fixed (Qwen 3.6 Plus and MiniMax M2.5) and ran each through three open-source harnesses, Goose, OpenCode and OpenHands-SDK, on 50 tasks drawn from Terminal-Bench Pro (Vats and Golev, arXiv 2607.22585). Changing only the harness produced up to a 40-fold difference in tokens spent per solved task. Pass rates moved far less, between 0 and 8 percentage points for a given model, and the authors report that only the largest of those gaps clears their confidence interval. Each harness also left its own failure signature, such as hitting the turn limit or looping idle, whichever model sat inside it.

That result cuts both ways. On cost, the harness dominates: a 40x spread in tokens separates a product that pays for itself from one that cannot. On accuracy, the effect is real but modest, which should temper anyone who expects clever scaffolding to make a weak model strong.

Anthropic’s experiments with long-running coding agents show the same pattern from the inside. Its agents failed in two predictable ways. Either they tried to build an entire application in one pass and ran out of context halfway, or a later session saw partial progress and declared the job finished. Every fix was harness work: an initializer agent that writes a feature list, a startup script, a progress log and a first commit; one feature per session after that; and a rule that a feature counts as passing only after an end-to-end test driven through a browser (Anthropic, 2025-11-26). None of it required a different model.

Harvey sells a harness built on three labs’ models

Harvey is the clearest case of a domain harness becoming a large business. The legal AI company’s product ran on OpenAI models until May 2025, when it added Anthropic models through AWS Bedrock and Google models through Vertex. Its announcement explained the move with a result from BigLaw Bench, its proprietary legal benchmark: in less than a year, seven foundation models, three of them from outside OpenAI, had overtaken the Harvey system it first benchmarked (Harvey, 2025-05-13). The conclusion it drew:

“there is no longer a single “best” model, but an array of premier models” - Harvey, 2025

By March 2026, Harvey described itself as multi-model by design: it routes each task to the model that suits it, fails over to another provider during an outage, and decides those routes through continuous BigLaw Bench testing and preference studies run by its own legal research team (Niko Grupen, Harvey, 2026-03-10). On 2026-09-09 it announced a $550 million round at a $15.5 billion valuation, co-led by Diffusion and Lightspeed, and said that 80% of Am Law 100 firms use it (Harvey, 2026-09-09).

Read closely, that story supports Isenberg’s thesis and complicates it. Harvey’s value plainly does not sit in one model, since it rents three. But the BigLaw Bench result also means that off-the-shelf models caught up with Harvey’s original system within a year. What kept the company ahead was not its first scaffold. It was the benchmark that told it the scaffold had been overtaken, and the customer relationships that let it swap engines without losing the car.

Thin GPT wrappers lost to the product they wrapped

The cautionary case is the first wave of AI writing tools. Jasper raised $125 million at a $1.5 billion valuation in October 2022, about seven weeks before OpenAI released ChatGPT. By July 2023 it was laying off staff and repositioning itself as a copilot for marketing teams rather than individual writers, after ChatGPT had reshaped the market it sold into (Eric Hal Schwartz, Voicebot.ai, 2023-07-17).

At that stage, Jasper’s product was mostly the second group of components: templates and prompt chains that coaxed good copy from a model that needed coaxing. Once the lab’s own interface produced comparable output for less, the coaxing was worth little. Isenberg’s account of wrappers maturing into harnesses is drawn from the companies that made it. It says nothing about how many startups added tools and memory and folded anyway. That is not a detail: it is the selection problem underneath the whole thesis.

Better models delete harness code

Workarounds expire. Rich Sutton’s 2019 essay “The Bitter Lesson” argued that, over decades of AI research, methods that scale with computation have repeatedly beaten methods that encode human knowledge of the problem. A harness full of hand-built task decomposition is human knowledge of the problem, and the essay predicts what happens to it.

Vercel documented one instance. Its internal text-to-SQL agent, d0, had 17 specialized tools and still answered only four of its five test queries correctly. The team rebuilt it around Claude Opus 4.5 with two tools, one that runs shell commands and one that runs SQL. On the same five queries it went to five out of five; average run time fell from 274.8 to 77.4 seconds, tokens from about 102,000 to about 61,000, and steps from about 12 to about 7 (Andrew Qu, Vercel, 2025-12-22). Five queries is a small sample, and the post is a field report, not a study. The direction is still the one Sutton predicted: a stronger model made most of the harness unnecessary.

With fewer tools and a stronger model, d0 got faster, cheaper and more accurate on its five test queries. Source: Vercel, 2025-12-22.

Labs now ship their own harnesses

The mechanics group is being absorbed from the other side. Anthropic sells Claude Code’s harness as the Agent SDK and, through Managed Agents, offers a hosted harness that runs the agent loop in a sandbox it operates (Agent SDK overview). A startup whose main asset is a well-built agent loop now competes with the model vendor’s own library.

The labs are moving into domains as well. On 2026-02-03, Anthropic released a legal plugin for Claude Cowork that handles contract review, NDA triage and compliance checks, with the warning that a licensed attorney must review what it produces. eWEEK reported that Thomson Reuters lost nearly 18% in a day, RELX 14% and Wolters Kluwer about 13% (eWEEK, 2026-02-04). Those are incumbents rather than harness startups, and a one-day drop measures investor fear, not lost revenue. The signal is plain all the same: a lab can publish a domain workflow as a plugin, and the market reads that as a threat to everyone selling the same workflow.

Evaluations, permissions and integrations are the durable layer

What survives is the third group, the commitments to a particular customer. My view, drawn from the cases above rather than from any single source, is that three parts of a harness resist both forces.

• Domain evaluations. BigLaw Bench is what let Harvey notice that models had overtaken its system and route around the change. A lab can ship a legal plugin; it cannot ship one firm’s judgement of what a correct merger memo looks like.
• Permissions tied to real people. Which client files an agent may read, on whose behalf it acts and which actions wait for a human are decisions about the customer’s organization, not about the model. That is why AI agent permissions have to live outside the model, and a smarter model does not change it.
• Integrations into systems of record. Writing to a practice-management system, an ERP or a ticket queue, with the audit trail a regulated customer expects, is slow work that accumulates instead of expiring.

Each of these gains value as models improve. A stronger model can be trusted with more actions, and every extra action needs a scope, a check and a destination.

Of the three groups in a harness, only the commitments to a customer gain value as models improve.

A test for any AI agent harness business

Isenberg is right that the harness is where a vertical AI company does its work, and Harvey’s valuation is evidence enough. He is wrong to treat it as one thing. The three groups suggest a simple test: imagine the next model release. If it would let you delete code, that code was a workaround, and its value was always temporary. If it would make your evaluation suite, your permission model and your integrations more useful, because the agent can now be trusted with more, you are building the part of the AI agent harness that lasts.