Introduction — The Promise I Made

In my previous article [1], I explored the maintenance trap in IT — how software systems are more like plants than stones, requiring constant care. I ended with a cliffhanger: “What is open from the article is how to specifically test and evaluate models — something to be picked up in the next article.”

This is that article.

Since publishing the first piece, something happened that made this topic very real for many of my customers. Anthropic announced the deprecation of Claude 3.5 Sonnet — a model that had become the backbone of countless production applications. Teams that had built their systems around a specific model version suddenly faced a hard deadline to migrate. Some were prepared. Most were not.