It Started with a Saturday Morning Experiment

I recently ran a simple test. I asked a small language model the same questions three times, with zero, one, and three rounds of self-reflection, and published the results. The pattern was clear: self-reflection helped when the model already knew the topic. It did nothing when it didn’t. And on bleeding-edge questions, more thinking just produced more confidently wrong answers.

That experiment raised a question I couldn’t shake: if “thinking harder” only works sometimes, what exactly is happening when a model reasons, and when is it just pretending?