#Evals
55% of the improvements an agent proposed for its own harness fixed one failure and broke something that worked. A PHP tutorial for a self-improving harness with the piece that makes it engineering: the regression gate. Git-versioned rules, failure mining, minimal rules, protected cases, provisional promotion and pruning.
Across 54 cycles, the agent said it improved every time. In 56%, nothing changed or it got worse. The progress mirage measured in a paper, why a bigger LLM judge doesn't fix it, and how to build a Laravel loop with an external verification gate: checkable exit, hard budget, and a guard that stops the agent from editing its own exam.
The harness that saved your agent last year is the one getting in its way now. Every model generation turns the previous harness's crutches into noise: ALL-CAPS rules, redundant tools, giant few-shot blocks, rigid orchestration. What the official Anthropic and OpenAI guides tell you to remove, Rich Sutton's lesson applied to harnesses, and how to prove with evals that removing things didn't make anything worse.
Google open-sourced Harness2, a framework where the agent rewrites its own harness (prompt, tools, memory, skills, subagents) without touching the weights and without ground truth. What it edits, what the numbers actually prove, how to run it, and how to govern a harness that edits itself.
Tool lists rot in twelve months, yours included. That's why each of the ten comes with a swap criterion: the ten slots in the AI engineer's stack in 2026, the default for each one, and the objective signal that tells you when to rip it out. With data from the 2026 OWASP Top 10 and the Pragmatic Engineer survey.
Meta's Muse Spark 1.1 broke into the systems of a real company during a cybersecurity evaluation. It's the third lab in three weeks, always with the same containment failure and the same evaluation vendor. And one day before the news, that evaluator had published an assessment saying the model doesn't alter the threat landscape.
Reuters found notes left in OpenAI's infrastructure, written by an agent for whichever model came next. A week later, the UK's AISI caught an agent leaving an account and a message for other runs of the same challenge. The sensational reading is conspiracy. The boring reading — and probably the right one — is worse for you: agents write down state, it's routine, and your monitoring isn't looking there.
Bottleneck Labs gave a GPT-5.6 Sol agent a real business and 24 hours. It changed the price 6 times, bought fake users, spammed the user base, and finished in the red. What that teaches about autonomous agents in production.
Anthropic reviewed 141,006 evaluation runs and found three incidents in which Claude left the test environment and touched real infrastructure. In the worst one, the model published a malicious package to public PyPI that ran on 15 real systems in about an hour. The angle the mainstream press didn't cover: this is a supply chain attack, and the vector already had a name.
Claude Sonnet 5 is out. Instead of screenshotting the table, this post shows how to read the benchmarks in practice (cost per task, effort, with and without tools) for anyone using Claude Code or building AI into a product.