Coding agent harness, and which parts of it actually change the output

A coding agent harness is everything in a coding agent except the model. Martin Fowler's site puts it as "Agent = Model + Harness". The harness is the loop that calls the model, the tools it can use, the files it reads at startup, how it trims its own history, whether it plans first, and what it is allowed to touch. Claude Code, Codex, OpenCode and Pi are harnesses. The model inside them is a separate choice.

The word is everywhere this month and rarely defined. Two studies from the same week of September 2026 measured which parts of a harness change the result. In short: on a strong model with a big context window, the harness changes your bill far more than your pass rate. On a weak model or a tight window, it can decide whether the task gets done at all.

What is a harness in a coding agent stack?

Microsoft's definition is the most concrete: the harness "drives model and tool calls, manages conversation state and context, applies approval policies, and can keep the agent progressing through a multi-step task." In practice that breaks into a handful of parts:

• The loop: gather context, act, check the result, repeat.

• The tool set: a few general tools like read, edit and bash, or dozens of specific ones.

• Startup context: the system prompt, plus instruction files like CLAUDE.md or AGENTS.md.

• Context management: what gets trimmed or summarised when the window fills up.

• Planning: whether the agent writes a plan before it edits.

• Subagents: separate contexts for noisy side work.

• Permissions and sandbox: what the agent may run without asking.

The main harnesses have converged on most of these. Claude Code, Codex and OpenCode all read AGENTS.md, compact automatically, offer a plan mode and support subagents. Claude Code only started reading AGENTS.md on 18 September, as a built-in example of "Claude Code mods", its upcoming way to customise the harness itself.

Does the harness or the model change the result more?

For pass rates on strong models, the model. For cost, the harness, by a wide margin.

HarnessTax, from a UC Berkeley team including Ion Stoica and Matei Zaharia, ran seven models through three harnesses: Claude Code, Codex and Pi. On SWE-bench Lite the harness moved success rates by no more than about 2 percent either way, and on Terminal-Bench by about 5. Cost moved much more. Claude Code cost about twice as much as Pi on SWE-bench Lite for near-identical results. Claude Fable 5 solved 97.8 percent of tasks in Claude Code and 96.7 percent in Pi, at $1.33 against $0.67. The authors' summary: "The same model can achieve similar success rates at up to 5x costs."

The likely reason is startup context. Claude Code's mean initial context is more than ten times Pi's, and Pi ships four tools: read, write, edit and bash. The study did not isolate the prompt on its own, so treat that as a strong correlation, not proof. The sample is also small, 30 tasks per benchmark, and the traces had not been released by 23 September.

Unreal Agent makes the same point as a product. Its launch post claims "up to 40% cost savings compared to Codex" from harness design alone: asynchronous tool calls, trimmed tool output and no subagents. The pass rates in its table are close. The footnotes matter. The 40 percent is 39, on one benchmark, against the Codex leaderboard entry. Against Unreal's own Codex runs the savings were 16 to 28 percent, with one model and a vendor running the test.

When does context compaction actually help?

When the window is tight. Otherwise, barely.

The arXiv study by Run-Ze Fan and colleagues built a minimal harness and switched its parts on and off across 176 settings, on SWE-Bench Verified and Terminal-Bench, with open-weight Nemotron and Mistral models. With a 32k context window, any form of context management raised SWE-Bench success by 35.7 points on average over none. At 128k the gain was 2.7 points. Without management, 78.7 percent of runs at 32k simply overflowed and died. In the authors' words: "Context management matters most when the context-window budget is tight."

Two more findings. Trimming stale tool output and then summarising was the most efficient strategy. Making trimmed content recoverable added nothing, -0.36 points on average, because the models rarely asked for it back. A separate August paper by Sydney Lewis found the same shape. At a 20,480-token window, trimming older tool results took one Qwen model from solving 43 tasks to 72. At 262k tokens the difference disappeared.

For most people on a frontier model with a large window, compaction is a way to keep long sessions alive, not a way to get better code. Our compaction explainer covers what it loses when it does run.

When is a planning step worth its tokens?

It depends on the model. For the weakest model in the Fan study, Nemotron 30B, planning added 11.6 points on SWE-Bench. Without it, the model's median run collapsed from 40 turns to 5, and 68.6 percent of runs ended without a single edit. For the two strongest models, planning barely changed accuracy, at -2.0 and -0.4 points, and cut cost by about 30 percent. The saving came from dropping redundant checks after each edit, not from better code.

The authors' summary: "Planning shifts from an accuracy scaffold for weaker models to a cost saver for stronger models." On a frontier model, plan mode is mostly a budget tool that also lets you review the approach before any file changes.

The same study found a similar flip for tools. The weak model did 15 percent better with predefined tools than with bash alone. The strongest did 3.6 percent better with bash alone, at 53 percent lower cost. A harness tuned for a small model can be the wrong harness for a big one.

Which harnesses do developers run side by side?

On Stackness, more than one at a time is common. As of 23 September 2026, 10 real public profiles list at least one coding agent harness, after removing accounts that say they are bots or test users. Four of the ten list more than one.

• Claude Code and Cursor: six profiles each, and three list both.

• Codex and OpenCode: two profiles each. Both Codex users also list Claude Code, and one of them is the founder of Stackness.

• GitHub Copilot and Windsurf: one profile each, and it is the same profile, which lists five harnesses.

These numbers are tiny, and we are not reading market share into them. They do match what people are building: tools for running several harnesses at once. CC Switch, a desktop app that manages providers, MCP servers, skills and prompts across Claude Code, Codex, Gemini CLI, OpenCode and more, had over 135,000 GitHub stars on 23 September. wshobson/agents, a library of agents and skills, now ships for seven harnesses from one source. ZCode, Z.ai's harness, published its source on 20 September, after a blogger reported it uploading whole workspaces, which Z.ai said was for codebase indexing. Running several harnesses is easier when their configuration is portable, and when you can read what each one does.

What can you change in a harness without changing models?

Quite a lot. The studies suggest where to start.

• Trim what loads at startup. Initial context is the clearest cost driver in HarnessTax. Keep CLAUDE.md or AGENTS.md short and move long workflows into skills that load on demand, which is the move Package domain workflows as reusable agent skills.

• Filter tool output before it lands. A hook that cuts test output to the failures does a version of what the best context strategy in the Fan study did, before the tokens are spent.

• Use plan mode as a budget tool on strong models, and as a lifeline on small ones.

• Change the compaction strategy if you work in tight windows. Plugins like fast-jev-compaction prune instead of summarising.

• Measure your own tasks. Every study here warns that its results belong to its own setup. Run the same task through two harnesses and compare the bill.

That last point matters most. As the Fan paper puts it, its results are "conditional effects of the specific implementations studied rather than identifying a universally optimal harness". The harness is now a part of your stack worth choosing on purpose, next to the model rather than after it.

Key numbers

• 35.7 points of extra SWE-Bench success from context management at a 32k window, against 2.7 at 128k (Fan et al., September 2026).

• About 30 percent lower cost from planning on the strongest models, for a 0.4 to 2.0 point drop in success.

• +11.6 points from planning on the weakest model tested, Nemotron 30B.

• Within about 2 percent is how much the harness moved success rates on SWE-bench Lite, while Claude Code cost about twice as much as Pi (HarnessTax).

• More than 10 times Pi's initial context is what Claude Code starts with.

• 4 of 10 real Stackness profiles with a coding agent harness list more than one, as of 23 September 2026 (see data sources).

Quick answers

What is a coding agent harness? Everything in a coding agent except the model: the loop, the tools, the startup instructions, context management, planning, subagents and permissions. Claude Code, Codex, OpenCode and Pi are harnesses.

Does the harness matter more than the model? For cost, usually yes. HarnessTax found the same model scoring within about 2 percent across harnesses at up to five times the cost. For success rates on strong models, the model still matters more.

When does context compaction help? Mostly when the context window is small. One study found a 35.7 point gain at 32k tokens and 2.7 points at 128k.

Is plan mode worth it? On weaker models it can decide whether the task gets done. On strong models it barely changes accuracy but cut cost by about 30 percent in one study.

Why do developers run more than one harness? Each is tuned differently, and tools like CC Switch make it cheap to move between them. On Stackness, 4 of 10 profiles with a harness list more than one.