“UPTIME? We don’t care - we’re a subscription business.”

This sounds wrong in so many aspects. Still it has some truth to it. Let me start with a disclaimer: It’s neither originated from a current or former customer, nor does it reflect how current or prior employers of mine are operating or thinking. It’s just me exaggerating a brainstorming with some folks.

But let’s start with what feels wrong with it? Where to start and where to end? “Shows disregard for customer experience and service reliability” “Undermines customer trust and loyalty” “Could lead to increased customer churn” “May violate terms of service” [..]

LLMs for the rescue?! Or are we actually building Compound AI Systems?

LLMs rule the world, right?! - Only thing what matters is using the most powerful LLM available and everything falls in place. Looking for numbers - just consult the latest LLM benchmark. Hmm - or do we need to build systems?!

I think it’s not just a matter of choosing an LLM, or any foundation model for that matter, and if you are following me, you already know that. E.g. in my medium post on “How do you choose the foundation model for your Generative AI App — like your car?"[2], I already argued how 1/ LLMs are just one part of your Generative AI application, but the overall application requires so much more components and engineering excellence and 2/ capabilities of frontier models become commodity with a ever increasing pace.

What happens in Las Vegas … Nah - let’s have a look. All things (Gen) AI.

At the time of starting this article, the first day of AWS re:Invent is over. AWS re:invent is the cloud computing conference, hosted annually by Amazon Web Services (AWS) in Las Vegas, Nevada. The 13th annual event takes place December 2-6, 2024. This year I’m not on-site, but still curious what it is happening there. Plan for the week is to update this article on a daily basis with the new things announced there. Let’s see how this goes.

RAG - just a poor engineering workaround?

My week kicked off nicely with some inspiring talks on an internal conference. In one of the talks Johannes Langer dived deep on how to build production-ready RAG systems. I answered his opening questions to the audience - “What is RAG?” - with “𝗞𝗼𝗳𝗳𝗲𝗿𝗸𝗹𝗮𝘂𝘀𝘂𝗿”, which translates to 𝗼𝗽𝗲𝗻 𝗯𝗼𝗼𝗸 𝘁𝗲𝘀𝘁 in my head.

Thinking more about this analogy, I find it is helpful to approach the question if RAG is just a workaround to overcome limitations of our current foundation models or is here to stay, one a more conceptual level. The German wikipedia article on “𝗞𝗼𝗳𝗳𝗲𝗿𝗸𝗹𝗮𝘂𝘀𝘂𝗿” talks about some of the motivations for this kind of test: huge efforts for students on memorising independent facts are eliminated, the test scope can be wider and the test is focussing more on the ability to creatively think and find new solutions approaches. In other words this approach is frugal with students resources and incentives creation of new solutions.

❓ Successful Building a GenAI use cases just requires the latest and greatest frontier model, right?!

🔍 In my conversations with customers I often realize that the choice of the best & shiniest model is highly occupying their resources, while thinking and focusing on architecting and building the use cases which ultimately should fulfil users needs gets very little attention. This naturally fuelled by the fast-pace and loud announcements of new frontier models, which come with superior benchmark results and new capabilities. What we often forget is that the majority of capabilities become commodity among different models very fast. Hence it often makes much more sense to focus on the use cases and building the generative application for the use case in a way it really addresses the user’s need and is able to evolve to utilize new release models where this makes sense. I wrote about this in my blog post (https://lnkd.in/e_YhCinM.)