2026
- The 6x Math Jump Looks Like a Loop Fix
TL;DR: An MIT CSAIL team reported that GPT-6 Astra edited 0.016% of Qwen3.5-0.8B’s weights and lifted its MATH-500 score from 7.9% to 48.1%, with GSM8K and IFEval rising too. The last column of the …
- Wrong, Not Broken: An Agent Evaluator Is Only as Good as Its Reference
TL;DR: Matt Wood’s “Wrong, not broken” describes a refund agent that answers in 600 ms with zero errors and quotes $40 where the policy says $140, because it retrieved last quarter’s document [1]. …
- Who Referees the AI?
TL;DR: In July 2026 a purported disproof of the Collatz conjecture got through Lean’s official proof checker, and through an independent second checker as well. It was wrong. The submission exploited …
- Your AI Judge Needs a Judge
TL;DR: Most teams I see ship LLM judges without testing them against human labels. The result: judges that are confidently, consistently wrong on 30%+ of cases. Hamel Husain’s “critique shadowing” …