schristoph.online

Tag

Evaluation

4 posts · all tags · all posts

2026

  1. Your Agent's Notes Are the Closest Thing It Has to Training Data

    TL;DR: A new paper from Meta Superintelligence Labs, UC Berkeley and others measures how well frontier-model agents improve when their weights stay frozen and everything they learn goes into files: …

    AgenticAISelfImprovingAgentsAgentMemorySkillsEvaluationReliability

  2. When Your Benchmark Runs for 88 Hours

    TL;DR: On the Dwarkesh podcast, Noam Brown of OpenAI described a system of roughly 10,000 agents that spent 130 billion tokens over 88 hours on a single problem. This is his account, not independently …

    AgenticAIMultiAgentAIAgentsEvaluationAmazonBedrock

  3. The Gate That Couldn't See the Defect

    TL;DR: I am building narrated demo videos for an upcoming conference, and one of them went back for rework six times. Three of my own quality checks were involved, and each failed differently. One ran …

    EvaluationVoiceCloningTTSQualityEngineeringSageMaker

  4. Your AI Models Have an Expiry Date — A Practical Guide to Model Lifecycle Management

    Introduction, The Promise I Made In my previous article [1], I explored the maintenance trap in IT, how software systems are more like plants than stones, requiring constant care. I ended with a …

    AIModelLifecycleEvaluationAmazonBedrockSageMakerArchitecture