schristoph.online

AgenticAI ยท 10 min read

How My Setup Learns While You Sleep

On this page
TL;DR: The thing that makes an agent get better over time is a loop, and the loop is the same whether you buy it or build it: capture raw interaction, consolidate it on a schedule, and keep the lesson while the transcript fades. Amazon Bedrock AgentCore Memory is the managed substrate for it โ€” storage plus automatic insight extraction โ€” with the skill-consolidation step layered on top as your own architecture. My own kiro-cli and Obsidian setup is the do-it-yourself version, and it ran this exact loop on me at the start of the session where I wrote this post. Here is the loop, both ways, and how to choose between them.
Disclaimer: I’m a solutions architect who builds things to understand them. This is a builder’s field report on how I think about agent memory and a setup I run for my own work, not authoritative guidance. If I’ve got something wrong, tell me.

The brilliant amnesiac

Davide Gallitelli has a good phrase for most production agents: brilliant amnesiacs [1]. An agent solves a fiddly problem at two in the afternoon and rediscovers it from scratch at four, because the lesson never left the session where it was learned. A bigger context window helps it remember more inside one conversation. Retrieval helps it look things up โ€” and unless something updates what gets retrieved between sessions, neither makes it better across conversations, which is the thing we actually mean when we say an agent learns.

This is the last rung of a ladder I have been climbing across a few posts: the spectrum that runs from an agent with no memory at all, through one that reads a knowledge base, up to one that improves between runs. The top rung is the interesting one, and it is also the one most teams skip. This post is about that rung, and about the loop that gets you there.

Three steps, one loop

The loop borrows its shape from how humans consolidate memory in sleep. You go through a day, you capture a pile of raw episodes, and overnight something sorts them: the useful pattern gets filed away as a skill you keep, and the minute-by-minute detail is allowed to fade. Agents can run the same three steps.

  1. Capture the raw interaction as it happens. The session, the trajectory, what was tried and what worked.
  2. Consolidate on a schedule. Step back from the transcripts, find the patterns that recur, and distill each one into something reusable.
  3. Keep the distilled lesson as durable knowledge the next session loads, and let the raw episodes age out.

The vocabulary for this comes from a taxonomy IBM’s Martin Keen lays out cleanly โ€” mapped here onto the mechanisms that typically implement each type in an agent system: working memory lives in the context window, semantic memory in the knowledge base, episodic memory is the record of past interactions and what was learned, and procedural memory is how the agent knows how to do things, expressed as skills [2]. The loop above is the bridge between the last two. It turns episodic memory (what happened) into procedural memory (what to do next time). As Keen puts it, “memory is what separates a chatbot from an agent, because a chatbot gives a response, but an agent can give a response shaped by persistent knowledge and accumulated experience” [2].

A loop: sessions feed short-term episodic capture, periodic consolidation produces long-term reusable skills, which make the next run better, which feeds back into sessions

The consolidation loop: episodes in, reusable skills out, a better next run.

The shape is the same everywhere. What changes is whether you buy the machinery or build it.

The managed version: AgentCore Memory

Amazon Bedrock AgentCore Memory is the managed answer. It is a fully managed service that gives an agent two kinds of memory that work together [3]. Short-term memory captures turn-by-turn interaction within a single session, so the agent holds immediate context without making the user repeat themselves. Long-term memory automatically extracts and stores key insights across multiple sessions: user preferences, important facts, and session summaries, retained for the next conversation [3]. The pitch is that you offload the state-keeping and get to spend your time on the agent’s actual job.

One precision point, because it is easy to blur. Native AgentCore Memory gives you short-term context plus long-term insight extraction. The specific move of distilling recurring episodes into a named, reusable skill and publishing it to a registry that future agents load, that is Gallitelli’s reference architecture built on top of AgentCore, not a button in the service [1]. He uses the Memory service for storage and a nightly schedule for the consolidation job, and adds the episodic-to-procedural step himself. Worth keeping straight: the managed primitive is memory; the skill-consolidation pattern is an architecture you assemble from it.

The do-it-yourself version: my kiro-cli and Obsidian setup

I run my Solutions Architect workday through a set of skills on a terminal agent, kiro-cli, over an Obsidian vault [4], part of the larger content and research pipeline I have written about before [5]. That setup runs the same loop, built from public Kiro features โ€” hooks that fire shell commands or agent prompts on session events are documented Kiro functionality [6] โ€” plus my own scripts around them, and I can watch every step of it because it is all plain files I own.

It works like this. A stop hook fires when a session ends and captures what happened. A headless analyzer, a Kiro hook that runs a separate agent, reads that transcript later and compares it against my current skills and steering files, looking for the same recurring patterns: a mistake I corrected, a workflow that drifted, a capability I reached for and did not have. It writes its findings as proposed edits to those skill and steering files. The next time I open a session, the pending suggestions surface as a nudge.

I am not describing this from the documentation. That nudge ran at the start of the session where I wrote this post, and six suggestions were waiting for me, things earlier sessions had learned and queued up. Episodes from past runs, consolidated into proposed changes to how the agent behaves. That is episodic memory on its way to becoming procedural memory, on my own machine, while I was not looking โ€” pending, as the next paragraph explains, my approval.

One honest caveat, because the headline invites the wrong picture. The loop does not edit itself. It proposes; I approve. A human stays in the seat for the step where a learned lesson becomes a standing instruction, and that is on purpose. “Learns while you sleep” describes the capture-and-consolidate work that happens between sessions, not an agent quietly rewriting its own rules unsupervised. It is a strong assistant that does its homework, not an autonomous system, and I would not run it any other way yet.

Managed or DIY: how to choose

Both sit on the same line, and the capture-consolidate-keep pattern really is shared. What they produce is not. AgentCore Memory extracts durable preferences, facts and summaries; Gallitelli’s consolidator on top of it distils trajectories into skills; my own version emits proposals that change nothing until I approve them. That last difference is the one that matters most in practice, so read the table below as a comparison of three pipelines that share a shape, rather than three implementations of one mechanism.

AgentCore Memory (managed)kiro-cli + Obsidian (DIY)
OperationsService handles storage and automatic insight extraction; you schedule any skill-consolidation job yourselfYou own the hook, the schedule, the quality bar
GovernanceManaged access controls, namespacing, encryptionYou inspect and version plain files in git
PortabilityTied to the AgentCore stackPlain markdown and shell; the data moves anywhere, the loop itself needs kiro-cli, a shell, a scheduler, and model access
Best whenYou are building a real agent product and want the heavy lifting goneYou want full visibility and the data lives on your own machine

The managed path is the right call when you are shipping an agent to users and the last thing you want is to build and run a memory tier yourself โ€” assuming its regions, latency, cost, and compliance posture fit your requirements. The service gives you consolidation and reflection without the infrastructure, and it sits next to the rest of your agent stack. The DIY path suits a different situation: a personal or small-team system where you value being able to read every file, diff every proposed change, and keep the whole thing in version control. I run the DIY version because my workday is the workload and I like watching the loop work. If I were putting this in front of customers, I would reach for the managed service. Neither one is simply the safe choice and the other the risky one โ€” but they are not equivalent either, and “ownership, not trust” would be too neat a way to end it. The threat models genuinely differ. The managed path puts your consolidated lessons behind a provider’s access controls, tenant isolation and encryption, and inside its audit surface. The DIY path puts them on a disk you control, in a git history that will cheerfully retain a lesson forever, backed up wherever your backups happen to go. Pick the set of risks you would rather reason about, and then actually check the controls on that side, rather than assuming the two cancel out.

If you’re running this on AWS

If you want the managed loop, the entry point is AgentCore Memory. The two memory types map directly onto the loop above: short-term memory is your capture step within a session, and long-term memory is the consolidation that carries insights forward across sessions [3]. You create a memory store, attach it to your agent, and let the service extract long-term insights rather than writing that extraction yourself. To add the episodic-to-procedural skill step on top, Gallitelli’s repository is a working reference: worker agents emit trajectories, a scheduled job harvests and distills them into skills, and a registry serves those skills back at session start [1]. Start from the official AgentCore Memory guide for the primitives [3], then layer the consolidation pattern if you need agents that compound what they learn.

What it actually buys you

An agent that does not learn is a tool you re-teach every morning. An agent that runs this loop can show up a little sharper than it left, because the work of yesterday’s sessions has been folded into how it behaves today โ€” when the folded-in lessons are good ones. Consolidation can just as well entrench a bad lesson, which is exactly why the approval step exists. That is the top rung of the memory spectrum, and it turns out to be reachable two ways: buy the managed tier, or build it from files you can read. The pattern is the same either way.

What would you put on the consolidation schedule first, the mistakes worth never repeating, or the wins worth turning into a skill?

Sources

๐ŸŽฌ Also available as a blog walkthrough video on YouTube

โค Created with the support of AI (Kiro)

๐Ÿ“ Last updated: November 18, 2026 โ€” Redrew the diagrams in a compact, consistent style

AgenticAIAgentMemoryAgentCoreKiroSkills