Your Agent's Notes Are the Closest Thing It Has to Training Data

Some of the rules in my agent setup exist because the agent made a mistake. And then the agent made the same mistake again, with the rule sitting in its own files. One of them has been flagged more than ten times by the analyzer that reviews my sessions, and it isn’t a hard rule to follow.

I described that analyzer loop in How My Setup Learns While You Sleep [2]: sessions get captured, a background agent compares them against my skill and steering files, and it proposes edits that I approve or reject. That post was about the loop working. This one is about how it fails, prompted by a paper that measures exactly this kind of learning in a controlled setting [1].

Learning without touching the weights

The paper is Agent Plasticity: Measuring Self-Improvement Through Experience, by Harman Singh and colleagues at UC Berkeley, Meta Superintelligence Labs, the University of Washington and Princeton, posted on 6 October 2026 [1]. The appendices contain the actual tools and notes the agents wrote, and those examples are the most useful part of the paper.

An agent is a model with fixed weights plus an inventory of artifacts: executable Python tools, skills or strategies written in plain language, and text memory. Every game starts in a fresh context, and only the inventory carries over from one game to the next. Between checkpoints the same model reviews its training games, the engine feedback and its current inventory, and proposes an update, which has to pass structural and executable checks before it is kept [1]. This is the scaffolding side of self-improvement that I contrasted with weight updates in an earlier post on MiniMax M2.7 [4]: the weights stay fixed while the code and notes around the model change.

The agents played chess, Go, Hex and NetHack. In NetHack they ran in their vendors’ coding-agent harnesses and could edit their own CLAUDE.md or AGENTS.md, SKILL.md files, Python tools and memory between checkpoints [1]. If you run an agent with skills and steering files, that is your setup, pointed at a dungeon.

Every checkpoint is scored on held-out games that the reflection step never sees, both at the training difficulty and against stronger opponents, so a rule only earns credit if it helps on games it wasn’t written from. The paper also counts what learning costs: the model calls spent playing training games and reflecting on them, including reflection attempts that got rejected [1]. The feedback differed by game. In chess a separate engine rated every training move, Go and Hex got lighter diagnostics, and NetHack had no oracle for the best action [1].

Final score versus learning efficiency

The paper defines agent plasticity as the gain on held-out games per unit of learning cost, reported in percentage points per $1,000 [1]. That separates how good the agent is at the end from how efficiently it got there, two things that usually get blurred together. I find the distinction useful beyond the paper. “Which model is best” and “which setup gets better fastest” are different questions, and only the second one tells you what a learning loop is worth.

The results spread widely. In hard chess the strongest lineage went from 37.5% at the start to 73.3%, averaged over the last five checkpoints. The second went from 25.0% to 66.4%. Another lineage started near 0% and climbed to 36.9%, so it gained about as much as the leaders and simply started lower. Several others stayed close to where they began, and a few got worse, despite twenty rounds of feedback [1].

On the combined chess, Go and Hex curve, the agent with the highest final score had a plasticity of 57 points per $1,000 [1]. The most efficient learner reached 298 [1]. In NetHack, one lineage roughly tripled its mean score over 40 checkpoints and the other five showed no reliable rise [1]. The authors are careful about what this means. Plasticity depends on pricing, the curve fit and the task, so they describe it as a property of an agent under a given protocol and warn against reading small differences as a model ranking [1].

Three ways the loop breaks

The failure analysis is the part I keep coming back to. For every identified mistake in a game, the authors ask whether a relevant artifact existed, whether it was used, and whether the mistake happened anyway [1]. That gives three buckets:

1. No covering artifact existed.
2. An artifact covered the mistake but the agent did not use it.
3. The agent used a covering artifact and still made the mistake.

The weak improvers stand out at step two. Pooled over both chess settings, four lineages made between about 28% and 98% of their relevant decisions without touching an artifact that applied [1]. The other four, which include the strongest improvers, did so in 2-6% of decisions [1]. Across 19 model and game combinations, reuse on held-out games correlated with improvement [1]. The authors call low reuse a candidate bottleneck, an association rather than a proven cause [1].

Reuse isn’t enough, though. Those four lineages used artifacts at 94-98% of relevant chess decisions and still ended up with very different scores [1]. They made fewer mistakes overall, about 0.16 per relevant decision against 0.38 for the rest. But 83-99% of the mistakes they did make happened while a relevant artifact was in use [1].

The appendix shows what that looks like, and it reads like a review of any agent’s steering files.

• One lineage kept adding rules against a specific losing chess move, in its notes and as code checks. From checkpoint 5 onwards the number of rules grew, and the mistake continued, because the agent’s opening book returned a move before the checks ran, so the checks never saw it [1]. The rules were fine. They sat behind a code path that skipped them.
• Another lineage, worried about running out of model calls, wrote notes telling itself to play the first legal move in the list without comparing options. By checkpoint 20, 65% of its held-out moves were the first one listed, up from under 20% before, and one of them allowed checkmate [1]. The note traded play quality for finished games, and some games still ran out of calls [1].
• A third lineage added an “avoid” entry for an exact board position after a costly mistake. When that position came up again in a held-out game, the engine played a better move, and removing the entry brought the mistake back. As the authors point out, this fix memorizes one position and only helps if it recurs [1].
• In NetHack, one agent died praying through raw keystrokes that bypassed its own safety check. It added a rule to pray only through its controller, and at a later checkpoint the same situation ended with a successful prayer [1]. The rule helped because it pointed at a code path that enforced it (though the paper notes that later games still sometimes bypassed the check).

One agent’s reflection step kept telling it to load its saved move-picking program immediately, and the agent still spent its first steps listing and reading files. Another lineage used its artifacts at 96-100% of relevant decisions for a while, then dropped to 32-50% from checkpoint 9 on, although the artifacts hadn’t changed [1]. An agent can stop using notes it once used reliably.

The same three buckets in my own setup

My setup has no held-out set and no cost accounting, so this is an anecdote sitting next to a study. Still, the paper’s buckets fit what I have seen better than any vocabulary I had before.

The second bucket, a rule exists and gets ignored, is the common one. My rule files record two rules that kept recurring after they were written down. One asks a background run to write a first file to disk early, before it spends turns on reading. The analyzer kept flagging violations, and the rule file records more than ten of those flags. Some of those flags are duplicates, so it’s a rough count. Restating the rule more forcefully didn’t help. So I moved it into the launcher that starts every background run, which now appends the rule to every prompt automatically. That guarantees every run receives the rule, but it is still text the agent has to follow, and the rule file records violations after the change. The NetHack prayer rule worked because it pointed at a code path that checked the action; my launcher doesn’t check anything yet. I wrote about this pattern from the code side in Architecting Skills [3], where each step moved from prose into a script removed a class of failures.

A related version: a failure diagnosis was already recorded, including which hypothesis had been tested and ruled out, and a later session retried that same hypothesis anyway, which the paper would count in the same bucket.

The third bucket, a rule that is used and is wrong, shows up less often, and it costs more because the agent looks like it’s doing the right thing. One of my rules told the agent where to save browser screenshots, and the tool refuses to write there. The agent followed the rule faithfully and failed every time, until the rule was corrected. Rules can also be too narrow, like the paper’s single-position guard, which only covers the exact case that went wrong.

There is also a volume problem the paper hints at. In one round the analyzer proposed eight separate suggestions that all restated the same two rules that already existed. Each suggestion was reasonable, but eight of them for two existing rules meant that writing more text wasn’t working. That has the same shape as the chess lineage whose rules piled up while the mistake continued. The paper has a related data point from NetHack: one lineage carried about three times as much tool code as the leader and scored far lower, and the authors write that more code or tool use does not explain the lead [1].

What I would measure

The paper argues that validating persistent artifacts should also check whether they change behaviour, beyond whether they are well formed or executable [1]. My own loop has a human approval step, and that step mostly checks whether a proposed rule is sensible. It does not check whether the rule works. These are the checks I would add, and they apply to any agent that learns through files.

• Count the mistakes that recur although a rule covers them and the agent didn’t consult or apply it. That is the “available but unused” bucket, and if the count rises, the problem is reuse, so look at enforcement or retrieval before you write another rule.
• Track failures that happen while a rule is in use. When the agent followed a rule and still failed, the rule, or the way it was retrieved and applied, is the suspect. Review it, narrow it, or delete it.
• Keep a small held-out check, a handful of tasks the improvement loop never sees, and run it before and after a batch of rule changes. Rules that only help the sessions they were written from will show up here.

I would also watch the cost per kept rule. The paper counts rejected reflection attempts as part of the cost [1], and a loop that proposes eight versions of one rule is expensive even when only one survives. Removal tests belong in the mix as well. The paper’s most direct evidence comes from removing one entry and watching the old move come back on that position [1]. Doing that with a rule you think matters, repeated on the same tasks, tells you much more than one run.

Games make this easier than most agent work, because a chess engine scores every move and an office task rarely comes with a scorer. So outside games the held-out check takes more work to build, and I think it’s worth more for that reason. None of the rest needs research infrastructure. Most of it is logging which rule applied to a decision and whether the decision went wrong. The paper is also clear that its own diagnostics are observational: an artifact being used doesn’t prove it helped, and a failure after use doesn’t show whether the artifact or its application was at fault [1]. Repeated removal tests are the cheap way to get closer.

If you’re running this on AWS

If you want the learning loop as a managed service rather than as files on your own machine, two parts of Amazon Bedrock AgentCore map onto it.

AgentCore Memory has an episodic memory strategy that does the capture and consolidate steps. It detects when an episode in a conversation is complete, turns it into a structured episode record, and generates reflections across episodes: insights about what happened and how it should influence future behaviour [5]. Those reflections are a managed version of one part of the paper’s inventory, the text notes, and the same questions apply. Are they retrieved when relevant, and are they right?

AgentCore Evaluations is where the measurement goes. It supports on-demand, online, batch and dataset evaluation of agent interactions, with built-in and custom evaluators [6] and code-based evaluators that run as AWS Lambda functions [7]. Batch evaluation over a fixed set of sessions is the natural place for a held-out check, used for baseline measurement and for before-and-after regression testing [7]. Run the same set before and after a change to what the agent remembers and you have a small version of the paper’s held-out score. It stays held out only while nobody tunes rules against those sessions, so keep a second set you leave alone.

Notes as training data

The paper frames self-improvement without weight updates as a property of the whole loop: the model, the artifacts it writes, and whether it uses them [1]. For a setup like mine, with weights that never change, that makes the agent’s notes the closest thing it has to training data. If they are not read, they can’t change the next decision. If they are wrong, they steer it the wrong way, very consistently.

The improvements I trust most in my own setup came from moving a step into code that runs whether or not the agent reads it. For the rules that can’t move into code, my next step is to log which rule applied to a decision and whether it was used and worked.

How do you find out whether a rule in your agent’s files is actually doing anything?

Sources

• [1] Agent Plasticity: Measuring Self-Improvement Through Experience — Singh et al., arXiv 2610.08902 (2026) — protocol, plasticity definition, results and the artifact examples in appendices D and E (PDF)
• [2] How My Setup Learns While You Sleep — the capture, consolidate and approve loop in my own agent setup
• [3] Architecting Skills: How Code Makes AI Agents More Reliable Over Time — moving mechanical steps from prose into scripts
• [4] Self-Improving Models: What MiniMax M2.7 Actually Does — scaffolding improvement versus weight improvement
• [5] Episodic memory strategy — Amazon Bedrock AgentCore documentation — extraction, consolidation and reflection across episodes
• [6] Evaluate agent performance with Amazon Bedrock AgentCore Evaluations — AWS documentation — on-demand, online, batch and dataset evaluation
• [7] Automated agent evaluation with Amazon Bedrock AgentCore and GitHub Actions — AWS Machine Learning Blog — batch evaluation for baselines and before-and-after regression testing