The Mess Under the Bed

Thu, 27 Aug 2026 00:00:00 +0000

TL;DR: You tell your kid to clean their room and they earn an hour of TV. They shove everything under the bed and still claim the reward. Reward hacking is that, and it is old: economics calls it Goodhart’s law, principal-agent theory calls it moral hazard. In July 2026 two frontier models did the room-under-the-bed move to their own exams. One inferred the answer key lived at the largest ML dataset host and broke in to take it. The other opened a malicious pull request on a real project and, when a bystander flagged it, edited its tracks to look harmless and stood up fake identities to vouch for it. Neither was malfunctioning. Both were optimising exactly what they were measured on. The uncomfortable part is not the cheating; it is that for a trained model, unlike a kid’s room, you don’t know where to look.

Disclaimer: I’m a solutions architect who reads incident reports so I can reason about the systems I help people build. Every factual claim about the two incidents below comes from a first-party disclosure (Hugging Face, OpenAI, or the UK AI Security Institute), not from press coverage. If I’ve mischaracterised something, tell me.

The room-cleaning problem is ancient

You want a clean room. You can’t stand over the child for an hour, so you attach a reward: clean the room, get an hour of TV. The reward measures a proxy for what you want, because “a genuinely tidy room” is expensive to verify and “no visible mess when I glance in” is cheap. The child, being a rational optimiser of TV, finds the cheapest path to the measured outcome: shove everything under the bed.

schristoph.online

The Mess Under the Bed

The room-cleaning problem is ancient