
WeeklyReview · 8 min read
Weekly Review — Nov 16–22, 2026
On this page
On Tuesday evening I spoke at CAIML #45 in Köln, to a full room: 75 registrations, a waiting list and people standing at the back. The rest of the posts were about numbers, and three of them came down to measuring the thing you actually run.
This Week on the Blog
Serving a Decision Model on SageMaker AI, and Measuring What You Serve
Part 1 ended with a calibrated decision model and nothing an agent could call. Part 2 converts it to a 2.7 GB GGUF file, serves it from a small child image of the AWS llama.cpp container with one changed nginx route, and measures it on three CPU instance types: on ml.c5.9xlarge a case with five questions took 4.7 seconds on average, about $2.70 per 1,000 cases with the instance kept busy. The served model kept part 1’s top answer on 88 percent of 2,000 decisions, and its accuracy dropped from 0.787 to 0.764. The largest shift came before llama.cpp, between the training-time model and the merged checkpoint, so the post ends with one rule: evaluate the file you deploy, and set the escalation threshold from its numbers.
How AI Agents Pay with HTTP 402 and x402: My CAIML #45 Talk as a Take-Home Handout
HTTP 402 “Payment Required” has been reserved for future use since 1997, and x402 finally gives it a job: an agent reads the price in a response header, checks its budget, pays and retries. The talk splits every paid request into two decisions. The agent decides whether a purchase is worth it, and the site decides the price and the license terms. On the buy side Amazon Bedrock AgentCore payments enforces a session budget and expiry before anything is signed, and on the sell side AWS WAF prices requests at the edge and pays out in USDC or, since November, in local currency through Stripe. The handout has one section per part of the talk and every link, grouped by where you want to start.
The Pipeline Behind My Demo Videos Is Now Open Source
The narrated walkthroughs on my agent posts come out of a small pipeline that now sits, MIT licensed, in my blog-samples repository. It works audio-first: each narration segment is synthesized and measured, and Playwright holds each scene on screen for exactly that long, so the voice stays in step with the picture as long as each scene’s steps finish inside its audio. The voice is a parameter with three backends, Amazon Polly by default, a Qwen3-TTS speaker or a clone of your own voice on a SageMaker endpoint that scales to zero, and a Kiro CLI agent can run the whole thing from a plain-language request.
The 6x Math Jump Looks Like a Loop Fix
An MIT CSAIL team reported that an AI model edited 0.016 percent of Qwen3.5-0.8B’s weights and lifted its MATH-500 score from 7.9 to 48.1 percent. The last column of the same table shows the share of answers that ran into the token limit falling from 93 to 33 percent, and the model card already warns that this model gets stuck in thinking loops. A small model that rarely loops showed no established gain under the same procedure, which supports the reading that the edit mostly let the model finish. The post ends with five checks for the next big jump, and with the point that your agent evals need the same column: record why each run ended, with max-token truncation, timeout, step limit and budget counted separately next to correctness.
What We Build With AI Today Is Tomorrow’s Legacy and Attack Surface
Calif closed its Java4Shell disclosure with a question: everyone is asking what we can build with AI, but for security the better question may be what we can delete. The bug sat in a Base64 fast path that few applications need and that is less than four years old, and an expert with an agent found it in about an hour. Since February I have added 75 skills to my own agent toolkit and deleted none, and Git can tell me which ones nobody edited but not which ones nobody runs. The post proposes a small loop: count each use, review the least used every month, and let a person decide what to keep, archive or delete.
The Thread This Week
Looking back, three of these posts started by measuring the wrong thing. Part 1’s evaluation was sound, but it measured a different file from the one I deployed. The headline math score was real, but the column that points to the likely cause sat at the end of the table. And my own toolkit records every edit and none of the uses, so I can’t say what is safe to delete. In each case the fix was to measure what actually runs.
Further Reading
Things worth your time that did not get their own post, all public:
- We Saved Millions: Marc Brooker and Carr Onstott on Strands Decider (AWS Developers): A production counterpart to this week’s decision-model posts. A ticketing system replaced a large-model classifier for 20 problem labels with a small decider that emits one output token, shadowed it on 5 percent and then all of the traffic, and switched over about two and a half weeks after the first commit. Agreement with the old model was about 76 percent, and the episode is careful about what that does and doesn’t show.
- AI Model Debugging: Find the First Divergence Across AI Accelerators (IBM Technology): A model with the same weights and the same input runs on a new accelerator and gives a different answer, and nothing crashes. Kaoutar El Maghraoui’s method is to localize first and diagnose second: compare intermediate tensors against a validated reference at chosen boundaries, with tolerances that fit each op and precision, and keep the minimal failing test in CI.
- How to Build Agents That Write Their Own Tools (AWS Developers): A Strands agent gets three tools, a shell, an editor and a loader, and writes a new tool at runtime. What I liked most is the promotion rule. In the presenter’s words, “A changed agent is not the same as an improved agent,” so a generated tool stays in staging until tool-level and session-level evaluations show it helped.
- Quit Tokenmaxxing: 4 Context Engineering Techniques for AI Agents (AWS Developers): Four families, each with a Strands example: compress older history, externalize large tool results to storage and keep a preview, select what the task needs through skills that load on demand, and isolate noisy work in sub-agents. Summaries lose detail, so the advice is to offload the data you can’t afford to lose.
- Welcome to the B2A Economy (a16z): a16z argues that agents are becoming customers and calls the next distribution shift business-to-agent. Among its signals: Shopify reported AI-driven traffic to its stores up eightfold year over year and orders from AI-powered searches up nearly thirteenfold, and payments, fraud and identity are named as the infrastructure already under strain.
- Advertising in the Agentic Web: Who Wins, Who Loses, and What Comes Next? (Matthias Jung): If an assistant compares products without a site visit, the publisher whose reviews it read loses the impression. Jung walks through six emerging models, from ads inside assistants to charging AI systems for content and building services agents can use, says whom each one serves, and recommends starting with a measurement baseline from your server or CDN logs.
- This Algorithm Changed How I Think About Compression, Information Theory Part 3 (3Blue1Brown): The third part of Grant Sanderson’s information theory series builds arithmetic coding from scratch, with a shared next-token model on both ends and each text mapped to an interval whose width is its probability. Seen that way, pretraining loss, the cross-entropy converted to bits (the nats usually reported divided by ln 2), is the number of bits per token the model would need to compress its training data.
- What a Squirrel Knows That Language Models Skip (Machine Learning Street Talk): Developmental psychologist Michael Tomasello defines agency as an organization of perception, goal and decision in a feedback loop, and traces four kinds of it from reptiles to humans. His critique is that AI work focuses on learning, which serves decision-making, and that language models start at language without the tiers of control underneath.
Until Next Sunday
Pick one tool you built with an agent this year and add a single log line that records each time it runs. A month from now that counter will tell you more about what to delete than the file dates do.
This is the Weekly Review, and it also goes out as Sunday’s newsletter.
❤ Created with the support of AI (Kiro)