Harness Engineering: What Actually Drives Agent Performance

2026년 초, OpenAI and Anthropic published posts on the same topic within a month of each other. The core message is identical: "It's not the model — it's the harness that determines agent performance."

The fact that two companies independently reached the same conclusion is worth paying attention to. This post compares the two approaches and distills the principles you can apply in practice right now.

What Is a Harness?

A harness is not the same as crafting a good prompt. It refers to the entire design of the environment that controls how a model's capabilities are directed toward a specific task.

Concretely, it includes:

• Prompt structure
• Repository directory layout
• Documentation conventions (AGENTS.md, design docs, etc.)
• Linters and CI pipelines
• Inter-agent feedback loops
• Context management strategies

In experiments by the LangChain team, holding the model constant and changing only the harness raised benchmark scores from 52.8 to 66.5. That gap came purely from environment design — no model swap required.

OpenAI: "Humans steer. Agents execute"

OpenAI shared what they learned from putting the Codex agent into production.

• 1 million lines of product code generated with zero lines of manual code over 5 months
• ~1,500 PRs merged, with the team growing from only 3 to 7 engineers
• 3.5 PRs per engineer per day

That level of productivity wasn't just a function of model capability — it came from a structured environment designed for agents to work effectively in.

AGENTS.md: A Table of Contents, Not an Encyclopedia

The centerpiece of OpenAI's approach is the AGENTS.md file. It's not a document that explains the entire codebase — it's a table of contents that tells the agent where to look.

Information the agent can't reach might as well not exist. The design principle is to keep the entry point small and let the agent navigate to what it needs through linked documents.

Enforcing Architecture: Using Linters to Preserve Invariants

If architectural decisions live only in documentation, agents will eventually violate them by accident. OpenAI moved architectural invariants — dependency direction, layer separation, and the like — into linters and CI checks for automatic enforcement.

Anthropic: A 3-Agent Architecture

Anthropic's concern was different: how do you maintain quality over long-horizon autonomous coding sessions that run for hours?

Their experiments surfaced two problems:

• Context anxiety: As the context window fills up, the agent's coherence degrades.
• Self-evaluation bias: Agents tend to overrate their own output.

A Structure Inspired by GANs

Anthropic drew inspiration from GANs (Generative Adversarial Networks) and designed a three-agent structure with explicit role separation.

Separating generation from evaluation eliminates self-evaluation bias.

Cost and Outcome

The difference became clear in an experiment building a 2D retro game from scratch.

The cost is 22× higher, but the single-agent approach never produced a working artifact to begin with.

Comparing the Two Approaches

The two approaches aren't mutually exclusive. OpenAI's approach tends to fit large-codebase maintenance; Anthropic's tends to fit complex greenfield feature development.

7 Principles for Practical Application

1. Give agents a map, not an encyclopedia

Keep AGENTS.md to under 100 lines as a table of contents. Structuring things so the agent reads everything at once hurts performance. Keep the entry point small and design the information architecture so the agent can navigate to what it needs through linked documents.

2. Enforce invariants in code

Rules like layer dependency constraints, left only in documentation, will eventually be broken. Add architecture boundary checks to your linter and CI so that agent mistakes are caught automatically.

3. Separate generation from evaluation

When an agent generates and reviews its own output, bias creeps in. Adding an independent Evaluator is often enough on its own to meaningfully improve output quality.

4. Let agents see the running app

Connecting Playwright MCP or Chrome DevTools Protocol to your agents lets them interact with the live application and catch issues that way. This is effective for surfacing runtime bugs that code review alone will miss.

5. Revalidate your harness when the model changes

Harness components are often designed to compensate for a specific model's weaknesses. When you upgrade model versions, remove components one by one and verify whether each is still necessary.

Anthropic's experience: upgrading from Opus 4.5 to 4.6, they were able to drop sprint decomposition and context resets — but the Evaluator remained essential.

6. Garbage-collect entropy

Code generated by agents accumulates technical debt over time. A background agent that periodically scans and refactors the codebase is an effective way to maintain long-term consistency.

7. Choose boring technology

Agents perform better with technologies that have abundant training data and stable APIs. Choosing a mature stack — PostgreSQL, FastAPI, React — measurably reduces agent mistakes.

Limitations

• Long-term consistency over multiple years has not yet been validated.
• The cost is significant. $200 for a single full-stack app is high by current standards.
• There are reproducibility concerns around lock-in to specific structures and tooling.
• Research from ETH Zurich noted that automatically generated configurations can actually hurt performance.
• Some argue this is simply a rebranding of established software engineering concepts.

Summary

Contrary to the expectation that better models would make harnesses unnecessary, what actually happens is that the harness's role shifts to a different layer. The era where context resets were the key concern gives way to an era where the design of evaluation criteria matters most.

Actions you can start today: