AWS Expands Amazon Bedrock with Automated Reasoning, Multi-Agent Collaboration, and Model Distillation

Amazon Bedrock, the fully managed service for building and scaling generative AI applications, is receiving new features to help developers move more quickly from proof of concept to go-to-market. The company announced at its 2024 re:Invent conference that the platform is gaining enhanced safeguards, the ability to orchestrate multiple agents, and the ability to create smaller, task-specific models that perform as well as large models but are cheaper.

Disclosure: I attended Amazon's 2024 re:Invent as a guest, with a portion of my travel expenses covered by the company. However, Amazon had no influence over the content of this post—these thoughts are entirely my own.

With Amazon Web Services viewing inference as a core part of any application’s functionality, these capabilities are meant to provide developers with the tools needed to deliver inference at scale while also leveraging their organization’s data in a way that provides a differentiated experience.

“Amazon Bedrock has become essential for customers who want to make generative AI a core part of their applications and businesses,” Dr. Swami Sivasubramanian, AWS’ Vice President of AI and Data, says. “Over time, as generative AI transforms more companies and customer experiences, inference will become a core part of every application. With the launch of these new capabilities, we are innovating on behalf of customers to solve some of the top challenges, like hallucinations and cost, that the entire industry is facing when moving generative AI applications to production.”

Automated Reasoning

AWS calls this an industry-first AI safeguard. It’s a protective measure designed to prevent factual errors due to model hallucinations.

Any model can provide inaccurate responses, misinformation, or misleading answers. Sometimes, what a model spits out can decide between life and death. With automated reasoning, the system checks against hallucinations using logically accurate and verifiable reasoning. It’s another protocol to minimize the chances of faulty information being shared.

This technology isn’t new for AWS. The company has deployed automated reasoning across multiple other products, including Amazon VPC, Codeguru, Verified Permissions, S3, and AWS Identity and Access Management. And Amazon isn’t the only one utilizing it: Microsoft, Google, IBM, OpenAI, Nvidia, and Intel are a handful of companies leveraging it.

Automated reasoning is a branch of AI that uses math to prove something is correct. AWS claims that it really shines with problems in which users require precise answers to a large and complex topic with a well-defined set of rules or a collection of knowledge about a subject. As AWS Chief Executive Matt Garman highlighted during his re:Invent keynote, following a user prompt, the response provided by the AI would be validated against a company’s automated reasoning policy. If it’s found in violation, a new response is generated. Otherwise, it’s verified. Regardless of the outcome, feedback is provided back to the system.

Agent Orchestration

Are there agents managing other agents? You better believe it! It’s called agent orchestration, and with Amazon Bedrock, developers can use multi-agent collaboration to get bots to solve problems collectively. While you can program agents to tackle functions such as handling sales orders, compiling financial reports, or analyzing customer retention, sometimes tasks are too complex to require more complicated and multiple steps.

Developers can establish a supervisory agent within Amazon Bedrock, which functions like a brain. The agent breaks up the complex project into tasks and routes them to other agents.

The proliferation of agents across the internet will make managing them unwieldy, so orchestration creates the equivalent of a project manager overseeing many specialized agents and managing their coordination.

Model Distillation

The final update to Amazon Bedrock involves model distillation, enabling developers to use large models to train smaller models to create an AI dataset that meets their needs. Previously, this work was difficult, requiring machine learning, in-house expertise, time iteration, and qualified and expensive professionals. With Bedrock, the process is made simpler.

AWS claims that with Amazon Bedrock Model Distillation, anyone can create a model that is 500 percent faster and 75 percent cheaper to run than the original model. “Now, customers can optimize to achieve the best combination of capabilities, accuracy, latency, and cost for their use case—no ML expertise required,” the company writes.

Today, only models from Amazon, Anthropic, and Meta work with AWS’ Model Distillation.

All of these three new features are available as a preview. Amazon has not disclosed how long it will be before they hit general availability.

Featured Image: Amazon Web Services Chief Executive Matt Garman gestures on stage at the company's re:Invent conference on Dec. 3, 2024. Photo credit: Ken Yeung