schristoph.online

Evaluation · 15 min read

The 6x Math Jump Looks Like a Loop Fix

On this page
TL;DR: An MIT CSAIL team reported that GPT-6 Astra edited 0.016% of Qwen3.5-0.8B’s weights and lifted its MATH-500 score from 7.9% to 48.1%, with GSM8K and IFEval rising too. The last column of the same table points to the likely cause: the share of answers that ran into the 32,768-token limit fell from 93% to 33% after the edit removed one direction tied to the model’s looping, and a model that rarely loops (Gemma 3 270M) showed no established gain from the same procedure. An independent evaluation of the released checkpoint reproduced the score. As far as the evidence goes, the edit mostly changed how often the model finishes an answer. The aggregate results can’t say how much of the gain that explains, and the evidence for better math ability is thin. Before you believe the next capability jump, in a paper or your own agent evals, check the truncation share, the sampling settings, the runs per condition, a negative control, and what the method was given. On AWS, Amazon Bedrock Evaluations can score “reached a final answer” as a custom judge metric on your own prompts, as a completion proxy you read next to correctness.

This autumn I wrote two posts about reading eval results before trusting them, one on benchmarks that run for days [7] and one on evaluators that need a reference answer [8]. Both came down to the same habit: before you trust a number, find out what produced it. A result published on 2 November is a good test of that habit, because the explanation sits in the same table as the headline.

What the MIT Team Reported

Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL published “Sparks of Weight-Space Programming” as a blog post with eight experiments, in which OpenAI’s GPT-6 Astra reads or edits the weights of other networks [1] [5]. The headline is Experiment 8. Astra got a Python session over the weights of Qwen3.5-0.8B, practice problems from the GSM8K and MATH training sets, and a budget of 10, 25 or 50 forward passes. Gradients and training were not allowed [1]. The evaluation ran in thinking mode with up to 32,768 tokens per answer, on test sets Astra never saw [1]. The final edit changed 131,072 of the model’s 0.8 billion weights, which is 0.016% [1].

Qwen3.5-0.8BMATH-500GSM8KIFEvalHit token limit
Base7.9%13.8%21.0%93.0%
Edited, 10 passes12.3%25.7%49.0%87.2%
Edited, 25 passes14.1%25.1%39.9%86.4%
Edited, 50 passes48.1%67.1%62.3%33.1%

Experiment 8 results from the MIT post [1]. Each edited row is a separate run with a different edit [5].

Read only the first three columns and you see a small model that got six times better at math and three times better at following instructions. Those are the numbers that made the headlines.

The Token-Limit Column Tracks the Jump

The last column is the share of MATH-500 answers that ran to the 32,768-token limit without finishing. For the base model it is 93.0%. After the 50-pass edit it is 33.1% [1]. Turned around, the share of answers that finished went from about 7% to about 67%. The two columns are separate measurements and overlap a little (in the base row they add up to 100.9%), but the direction is clear.

So what happened? The write-up says it plainly. The base model loops: it repeats the same text and rarely reaches an answer, and 85 of its first 96 practice answers ran to the limit [1]. Qwen’s model card warns about exactly this. In thinking mode, the card says, the 0.8B model “is more prone to entering thinking loops compared to other Qwen3.5 models, which may prevent it from terminating generation properly” [3]. The model runs in non-thinking mode by default, and the experiment switched thinking on [5], so the baseline is the model in the mode its card warns about. For math that’s a reasonable choice, but it isn’t the default configuration. Astra traced the looping to one direction written by the MLPs in layers 15 and 19 and subtracted that direction from 64 output columns in each of the two layers [1]. Those are the 131,072 weights.

The simplest reading of the table, then: the model now finishes answers it used to abandon. My best guess is that the loop used to cut off some solutions the model could already reach, and the edit let them through. The table itself doesn’t split correctness by finished and unfinished answers, so it can’t say how many, and it can’t rule out that part of the gain comes from better reasoning as well. The aggregate numbers point strongly at the loop reading, but they can’t measure how much of the gain it accounts for. The authors lean the same way: “Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked.” [1]

Two bar charts. Left, Qwen3.5-0.8B: for the base model and edits at 10, 25 and 50 forward passes, MATH-500 score is 7.9, 12.3, 14.1 and 48.1 percent, and the share of answers hitting the token limit is 93.0, 87.2, 86.4 and 33.1 percent. Right, Gemma 3 270M: base score 6.8 percent with 8.5 percent of answers hitting the limit, best edit 7.7 percent.

The largest score rise is in the 50-pass run, which is also the only run with a large drop in token-limit hits. Gemma rarely hits its limit, and the same procedure barely moved its score.

Why Instruction Following Jumped Too

IFEval measures instruction following, and it rose from 21.0% to 62.3% [1]. If the edit had taught the model math, an IFEval gain that size would be a surprise. If the edit stopped the loops, you’d expect it, because a model that loops fails any task that needs it to stop and answer. The authors make the same point: the loops aren’t specific to math, so performance beyond math improved too [1]. When unrelated benchmarks jump together after one small change, I’d look for something general first, and the finishing rate is the obvious candidate here.

Gemma 3 270M as a Comparison

The authors also ran a comparison that bears directly on this reading. Gemma 3 270M is an instruction-tuned model about a third the size of Qwen3.5-0.8B, and it rarely loops: 8.5% of its answers reach its token limit, against 93% for Qwen [1]. Astra worked on it with budgets from 5 to 50 forward passes. MATH-500 went from 6.8% to at most 7.7%, a gain Astra itself reported as not established [1]. Reading Gemma’s failures, Astra found ordinary reasoning errors, such as wrong arithmetic and invalid algebra steps, with no single component behind them to switch off [1].

Gemma works as an imperfect negative control: a model without the failure mode, run through the same procedure. Its score didn’t move. That supports the loop explanation without settling it: Gemma is a different and smaller model, sampled with its own model card’s settings and up to 8,192 tokens [1], and a method can work on one model and not another for reasons that have nothing to do with looping, such as architecture, baseline ability or how hard the search is.

Open Questions and What Checks Out

Some parts of the result rest on very little.

One Edit Search per Budget

There was one Astra run per budget [4], and the budget curves are uneven. “One run” here means one edit search. The resulting models were each scored over several samples per problem, four on MATH-500 [1], which reduces sampling noise for each checkpoint. The post reports no confidence intervals, though, and the larger uncertainty sits in the search. MATH-500 reached 12.3% with 10 passes, 14.1% with 25 and 48.1% with 50, so it rises at every budget but jumps only at the last one. IFEval is non-monotonic: it went from 49.0% at 10 passes down to 39.9% at 25 [1]. The token-limit share barely moved until the 50-pass run (87.2% and 86.4%) [1]. To me that looks less like a smooth budget curve and more like one run that happened to find the loop direction. How often a second run would find it is unknown. The logs also leave out trial runs made with earlier prompt versions [4], so we can’t see whether other attempts were made or how they went.

No Cheaper Baseline

The post doesn’t compare the edit against other decoding settings or against a standard direction ablation found with existing interpretability tools [4]. Either might match the result or fall well short. The post doesn’t run either comparison.

Sampling Settings

The evaluation used temperature 0.6, top-p 0.95, top-k 20 and presence penalty 1.5, for the base and edited model alike [1]. The model card recommends temperature 1.0 with the same top-p, top-k and presence penalty for thinking mode on text tasks, and lists temperature 0.6 (with presence penalty 0) for vision and precise coding [3]. The card also says the loops show up “when using the recommended sampling parameters” [3]. So the eval’s settings differ from the card’s text recommendation, but the card doesn’t suggest its own settings would stop the looping. I’d treat that as a question for a re-run. It isn’t evidence that the base score was held down.

Other parts hold up well.

An Independent Evaluation of the Checkpoint

Hugging Face’s akhaliq evaluated the released edited checkpoint on all 500 MATH-500 problems, 2,000 generations in total, and got 51.0%, with 31.6% of answers hitting the token limit [5]. Both the score and the token-limit share reproduced. Keep in mind what that covers: the released checkpoint. Astra’s edit search itself wasn’t repeated.

Open Logs

For every reported run, the repository includes the prompts, the replies with their reasoning summaries, the code the model ran with its output, and the evaluation behind each number. All 1,125 model turns were audited as tool-free [2]. The Experiment 8 edit is included as an exact diff against the public base weights, with a script that applies it [2]. The trial runs with earlier prompt versions are outside that record [4]. With these files anyone can inspect the mechanism and evaluate the released checkpoint independently [4].

None of this makes the edit less real. Finding the one direction tied to a small model’s looping, removing it, and getting a measured improvement that an independent evaluation reproduced is a useful debugging technique. Sean Goedecke argues it follows a playbook a capable ML engineer would recognize, steering and direction ablation [6]. For a debugging tool, that counts in its favour. Where I’d push back is the framing. “Better at math” describes the benchmark row. “Stopped looping” is the better-supported description of what changed in the model.

Five Checks for the Next Big Jump

When a jump like this shows up in a paper, a vendor announcement or your own eval dashboard, five questions cover most of the ground, and none of them costs much to ask.

  1. Look at the truncation share before and after. Find the share of answers that hit the token or time limit. If it moved about as much as the score, you’re looking at a finishing-rate change first. If the report doesn’t include that column, ask for it.
  2. Compare the sampling settings with the model card. Put the eval’s temperature, top-p and penalties next to what the card recommends for that mode, and read the card’s warnings. A baseline run under poor settings is easy to beat.
  3. Count the runs per condition and look at the shape of the curve. One run per condition tells you what happened once. A curve that jumps at one budget and dips at another needs repeats before you read a trend into it. Ask about runs that weren’t reported, too.
  4. Look for a negative control. Run the same method on a model that doesn’t have the failure mode. If that model doesn’t move, the likeliest story is that the method fixes the failure and adds no ability. If it does move, you may have found something bigger.
  5. Write down what tools and feedback the method had. Here that was a Python session over the weights, practice problems from the training sets and up to 50 forward passes [1]. That tells you which simpler methods to compare it with.
A scorecard of five cards applying the checks to Experiment 8. Truncation share: 93 to 33 percent, the gain tracks finishing. Sampling: temperature 0.6 versus the card's 1.0 for text, loops documented either way, open. Runs per condition: one per budget, 12.3, 14.1, 48.1 percent, an abrupt increase at 50 passes. Negative control: Gemma 3 270M 6.8 to at most 7.7 percent, consistent with the loop reading. Tools and feedback: Python over weights, up to 50 forward passes, no training.

The five checks applied to Experiment 8. Orange cards change how you read the headline, the blue card supports the loop reading, and the grey cards are context or still open.

Your Agent Evals Have a Token-Limit Column Too

The same checks apply to evals you run yourself, and agent evals have their own version of the token limit. An agent that times out, hits a step limit or runs out of budget is in the same position as an answer that ran to 32,768 tokens. In “When Your Benchmark Runs for 88 Hours” I suggested treating runs that time out without converging as censored instead of counting them as converged, and summing cost per solved task over the failed runs too [7]. Both need the eval to record why each run ended.

So when a new model or prompt raises your pass rate, check the timeout share alongside correctness and the other reasons runs ended. A change that stops an agent from stalling is worth shipping. It’s still a different change from one that makes the agent reason better, and it needs different follow-up.

“Wrong, Not Broken” covered the other half: an evaluator is only as good as the reference it compares against [8]. An answer that finishes still has to be checked against ground truth. A useful report shows finishing and correctness as two columns, side by side.

Running These Checks in Amazon Bedrock Evaluations

Amazon Bedrock Evaluations can run these checks on your own prompts instead of only on public benchmarks [9].

Your Own Prompt Dataset

A model evaluation job that uses a judge model reads a custom prompt dataset from Amazon S3 in JSON Lines format, with up to 1,000 prompts per job. Each line has a prompt, and optionally a referenceResponse (your ground truth) and a category [10]. Scores are reported per category, so math, instruction following and your own task types each get their own number [10].

Comparing Models and Configurations

An LLM-as-a-judge job evaluates one model, and you run several jobs on the same dataset to compare models [14]. Generators can be Amazon Bedrock foundation models, Marketplace models, customized or imported models, prompt routers, or models with Provisioned Throughput [12]. You can also bring inference responses you generated yourself. In that case Bedrock skips the model call and evaluates the responses you supply [10] [12].

A “Reached a Final Answer” Metric

Custom metrics are judge prompts you write, up to 10 per job [11]. They can use the input variables {{prompt}}, {{prediction}} (the generator’s response) and {{ground_truth}} (from referenceResponse) [11] [13]. Add a judge metric for whether the response reached a final answer, and you have a completion proxy to read next to a correctness metric:

A custom metric definition in the JSON format the console accepts, with a two-value rating scale [11]:

{
  "customMetricDefinition": {
    "metricName": "reached_final_answer",
    "instructions": "You are checking whether a model response is complete.\nYou are given a prompt and the response a model produced for it. Decide whether the response ends with a clear final answer to the prompt. A response that stops in the middle of its reasoning, or repeats the same text without concluding, has not reached a final answer. Do not judge whether the answer is correct.\n\nRate the response as follows:\n- Final answer given: the response ends with a clear final answer.\n- No final answer: the response stops mid-reasoning or repeats itself without concluding.\n\nHere is the actual task:\nPrompt: {{prompt}}\nResponse: {{prediction}}",
    "ratingScale": [
      { "definition": "Final answer given", "value": { "floatValue": 1 } },
      { "definition": "No final answer", "value": { "floatValue": 0 } }
    ]
  }
}

The judge only sees the response text, so this metric measures apparent completion. It can’t tell whether generation stopped at the max-token limit: a response can contain a final answer and still hit the limit, or stop unfinished for another reason. The actual truncation column has to come from the generation metadata your inference harness records, such as the stop reason for each response. If you generate the responses in your own harness, record that before you upload them and keep it as a separate measurement next to the judge’s score. Writing it into the category field is one way to get scores split by finished and truncated responses in the report.

Before you plan a job, check the user guide for which judge models support custom metrics and which models and Regions support evaluation. The judge-model list for custom metrics is separate from the one for built-in metrics [9] [12].

What’s the token-limit column in your own eval reports, and who looks at it when the score jumps?

Sources

❤ Created with the support of AI (Kiro)

EvaluationLLMEvalsLLMLimitationsReasoningVerificationAmazonBedrock