Harness2: Google open-sourced the agent that rewrites its own harness

Until now, the harness was the part of the agent you wrote by hand. The model just followed orders.

Google just open-sourced a harness that rewrites itself between one task and the next. Without touching a single model weight.

It's called Harness2 (Harness² in the paper). It comes out of Google Cloud AI Research, with one author from Northwestern University, code on GitHub under Apache 2.0 and a paper on arXiv published on October 9, 2026. The idea fits in one sentence: the agent proposes changes to its own system prompt, tools, memory, skills and subagents, tests those changes on the same task, keeps what held up and carries the lessons into the next one.

In this post you'll see exactly what Harness2 edits, how it decides a change is better without ground truth, what the numbers prove (and what they don't), how to run the repo today, and the problem I think matters most in this whole story: governance of a harness that edits itself.

TL;DR

• What it is: a Google research framework that refines the agent's harness (not the model) through a Propose-Probe-Compose cycle, with no ground truth.
• Stack/Models: Python 3.12 + uv, OpenCode or Codex CLI as executor; tested with GLM-5 and Gemini 3.5 Flash via Vertex AI.
• Cost/Access: open source (Apache 2.0). Running a real benchmark burns API credits. There's a local demo that needs no credentials.
• Repository: github.com/google-research/harness2

What Harness2 changes: the harness stops being fixed

I'm not going to re-explain what a harness is here. If you need the basics, start with what an AI harness is and then the anatomy of an agent harness.

The point is this: the thesis we keep repeating on this blog is that the harness is what separates a demo from production. Same model, different harness, completely different result. The paper itself opens with that: agent performance depends on the model and on the execution harness.

But until now that work was human. You looked at the trace, saw the agent botching a tool call, tweaked the prompt, added a guardrail, wrote a skill. Repeat.

Harness2 automates that cycle. According to the README, it touches eight components:

• Context construction: system prompt, guardrails, memory, skills
• Tool interface: tool guidance, scripts
• Orchestration: subagents
• Middleware control: plugins

The model stays frozen. Everything around it is what evolves.

And it works at two levels of recursion:

1. Task level: refines the execution harness for the current task.
2. Harnessing level: updates a persistent "improvement harness" with the adopted harnesses and the lessons that had evidence behind them, to guide refinement on the next task.

The second level is the real trick. It's not just "the agent adapts to this task". It's "the agent learns to adjust its own harness better next time".

Some context: Google isn't alone here, and it isn't first. Google Research itself has RRSI, another approach to recursive harness self-improvement, and the Harness2 paper compares against Meta-Harness, ReasoningBank and Harness Scaling. Harness2's stated difference is targeting the "open world": tasks the harness has never seen, with no verifier to say whether it got them right.

If you're building harnesses by hand today (and you should be, because that's where your edge lives), this is exactly the kind of thing we take apart and rebuild live every week in the Clã Beer and Code: agents, harnesses and pipelines running, with people reviewing together instead of everyone getting stuck alone in their own trace.

Propose-Probe-Compose with no ground truth: how Harness2 decides a change is better

This is the part I find most interesting, and it's what sets Harness2 apart from the classic self-improvement loop.

In the agent improvement loop we already showed here, the agent improves the code and has an eval with ground truth to tell it whether it got better. Passed the test, it stays. Broke something, it gets reverted.

Harness2 improves the harness and has no ground truth. In production, a new task doesn't come with the right answer attached. So how does it decide?

Per the paper, each step works like this:

Propose. The current harness runs once on the task and produces a reference execution. On top of it, an editor proposes two candidates:

• an invariant one, targeting recurring agent weaknesses (planning, tool use, context management, delegation);
• a domain-specific one, targeting the knowledge and constraints of that field.

Probe. Each candidate runs on the same task, same model, same environment. The trajectory and the deliverable of each are compared against the reference. That produces what the paper calls behavioral contrast: did the change help, make no difference, or cause a regression?

Compose. A composer reads the reference, the candidates, the diffs and the task evidence. It can merge the changes that held up, revise the ones that conflict, adopt an entire candidate, or keep the current harness. Every decision has to be tied to a concrete observation, not to "I think the score will go up".

After each task, a reflector reviews everything and writes lessons into a library. Each entry has an executable harness and an EXPERIENCE.md with the situation where it applied, the lesson and the evidence. If the evidence is weak, nothing goes in.

Notice how honest the paper is here: the reference "reveals how the harness behaves, not whether the deliverable is correct". In other words, without ground truth, the signal is observed behavior + model judgment. It's evidence engineering, not proof.

A few safety choices baked into the design that are worth copying into your own project: candidate harnesses and artifacts are treated as data, not instructions, scripts coming from candidates are never executed by the improver, and no grader, rubric or score is available to it.

The Harness2 paper numbers (and what they don't prove)

The numbers that went viral are real, but they need context. With a single Propose-Probe-Compose step (k=1), the authors report:

With a bigger recursion budget (k=5 in parallel), the numbers go up further. On WorkBuddy-Bench, for example, GLM-5 + OpenCode gets close to 79.6. In an appendix, JobBench shows the tool error rate dropping from 14.4% to 9.1%.

Now the part the hype skips:

• The biggest gains come from different configurations. There's no single setup that got +11.5, +4.5 and +11.8 at the same time.
• The weaker model gains the most. GLM-5 starts well below Gemini 3.5 Flash. A bad harness has more room to improve. Makes sense, but it changes how you read the results.
• Two of the judges are models. The LAB judge and the JobBench rubric judge are LLM-based. An LLM judge grading an LLM agent has shared bias, and the paper itself admits this in its limitations.
• No absolute cost numbers. The paper shows Harness2 on the cost vs. performance Pareto frontier, but gives no token or dollar figures. And it says it plainly: more recursion = more inference.
• It's three benchmarks and four configurations. Generalizing from there to your domain is a bet, not a conclusion.
• Self-reported numbers, v1 on arXiv, no independent replication yet.

So: promising results, and a methodology that's honest about its own limits. It's not "the agent now fixes itself".

How to run the Harness2 repo on your agent today

Requirements per the README: Linux, Python 3.12 and uv. For benchmarks with a real model: Node.js 20+, Google Cloud CLI and a Vertex AI project. LAB needs Pandoc and Poppler; WorkBuddy-Bench needs Docker.

Step 1: local demo, no credentials

Start here. It costs nothing and shows you the output structure.

git clone https://github.com/google-research/harness2
cd harness2
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

Step 2: configure the model

cp .env.example .env
# edit .env with your Vertex AI project and model IDs
gcloud auth application-default login

Step 3: run a small slice

The README itself recommends starting with just a few tasks, because each refinement step runs the agent several times. Use --dry-run first to see what will be selected without calling the model.

harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
  --only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

Flags that matter:

• --k: recursion budget per task. --k 1 is a single Propose-Probe-Compose step.
• --mode parallel: tests K general candidates and K domain candidates, then generates K compositions. --mode sequential refines over K consecutive steps.
• --substrate cx: swaps the default executor (OpenCode) for Codex.

Common mistake: running the judge or the report with different parameters from the refinement run. Benchmark, executor, subset, --k, --mode and seed all have to match. And if you change the model or the task selection, point HARNESS2_RUNS_DIR to a new directory.

One honest caveat: today the repo runs on the three benchmarks from the paper, plugged into OpenCode or Codex. "Running it on your agent" in practice means studying what it produces in runs/ (adopted harnesses, reflection notes, deliverables) and bringing the pattern into your own harness. It's not a library you import into your Laravel app tomorrow.

The problem nobody is talking about: auditing, versioning and rolling back a harness that edits itself

This is where I want you to stop for a minute.

If the harness is what separates a demo from production, a self-tuning harness changes who does that work. And it raises a question the paper doesn't answer, because it's out of its scope:

When the agent changed its own system prompt on Tuesday, who reviewed it?

Think about what Harness2 can edit: guardrails, memory, tool guidance, middleware plugins. That's exactly the layer where your agent's safety and behavior in production live. A guardrail loosened because "the behavioral contrast showed the task went faster without it" is an incident waiting to happen.

What I'd do before letting anything like this near production:

1. The harness is code. It goes in git. Every adopted harness becomes a commit, with the reflection note as the message. A readable diff is a prerequisite.
2. Separate the zones. Style prompts and domain skills can evolve on their own in staging. Guardrails, tool permissions and middleware plugins only change with human approval.
3. Gate with your eval, not its own. Harness2 decides without ground truth. You, before promoting to production, decide with ground truth. This is where the agent improvement loop and Harness2 complement each other.
4. Rollback is a tag. If harness version N made a production metric worse, going back to N-1 has to be one command, not an archaeology dig.

A minimal skeleton of that discipline, adjusting the path to wherever your harness lives:

# after each refinement round
HARNESS_DIR=./agent-harness   # system prompt, skills, AGENTS.md, guardrails...
cd "$HARNESS_DIR"

git add -A
git commit -m "harness: automated refinement $(date +%F)" \
           -m "$(cat ../ultima-reflexao.md)"

# mandatory human review when a sensitive zone is touched
if git diff HEAD~1 --name-only | grep -Eq 'guardrails|plugins|permissions'; then
  echo "Change in a sensitive zone. Opening a PR instead of promoting." && exit 1
fi

# promote only if the ground-truth eval passes
./run-evals.sh && git tag "harness-$(date +%Y%m%d-%H%M)"

# rollback: git checkout <previous-tag> -- .

The README itself helps you gauge how careful you need to be: the executor's network access is unrestricted and Codex runs with the sandbox turned off. The authors also warn that public benchmark rubrics may be reachable online and ask you to restrict network access before evaluating. And the project is explicit: "this is not an officially supported Google product". It's research. Treat it like research.

Limitations and things to watch

• Cost. Each step runs the agent several times (reference + candidates + composition). More k, more tokens. The paper gives no absolute numbers.
• Noisy signal. Without ground truth, the decision depends on behavioral contrast and model judgment, both sensitive to run-to-run variance and shared biases. The authors themselves say this guarantees neither correctness nor transfer.
• Execution safety. Open network and sandbox off in Codex. Don't run this on a machine with production credentials.
• Data. If you test on internal tasks, the editor will read trajectories and deliverables. Don't feed it customer data without masking it.

Quick FAQ

Does Harness2 train or fine-tune the model? No. The weights stay frozen. What changes is the harness: system prompt, guardrails, memory, skills, tool guidance, scripts, subagents and plugins. That's why it works with closed models over an API, like Gemini 3.5 Flash.

Can I use it commercially? The code is licensed under Apache 2.0. But third-party code in third_party/, benchmarks and datasets keep their original licenses. And the project is not a Google-supported product. Check the README first.

How is this different from the eval-driven self-improvement loop? In the agent improvement loop the agent improves the code and an eval with ground truth decides. In Harness2 the agent improves the harness and decides without ground truth, by contrasting executions.

Does it run with a local model? The documented flow uses Vertex AI and the OpenCode or Codex executors. The paper tested GLM-5, which has open weights, but through the authors' setup. Local models aren't on the documented path today.

Conclusion

Harness2 takes the most hand-crafted part of agent engineering, reading traces and tuning the harness, and shows that a good chunk of it can be automated without ground truth. The numbers are good, especially for weaker models, and the paper is honest about its own limits.

But the lesson for people building products isn't "let the agent figure it out". It's that the harness has become a living artifact. And a living artifact in production needs versioning, review and rollback, like any other code.

The next step for this technology won't be an agent that improves itself more. It'll be whoever can prove what the agent changed.

If your harness fundamentals aren't solid yet, go back to the anatomy of an agent harness before letting anything edit yours.