The ones who joined Philipp and me in our session at the #AWSSUMMIT in Berlin last week already got
written by Stefan Christoph
The ones who joined Philipp and me in our session at the #AWSSUMMIT in Berlin last week already got a preview of the blog post as we run it as a demo. Nice that it is now published and you all can get hands on it. Kudos!
AWS Inferentia2 is a great way to optimize the inference part of your (gen)AI workloads on AWS and the blog post helps you to dive straight into deploying a LLM (in this case Meta’s Llama 3) model. But it is not “just” Llama 3. From Hugging Face recent blog post: “Enabling over 100,000 models on AWS Inferentia2 with Amazon SageMaker” - https://lnkd.in/ePYb6TFs. So there is a good chance that you can benefit from AWS Inferentia 2 today :)
As we also discussed in out talk, inference is just one side of the coin. If you after training foundation models on AWS, have a look into AWS Trainium.