If you thought Amazon Web Services (AWS) was done revealing updates to Amazon Bedrock, you’re in for a surprise. After all, it’s the centerpiece of the company’s push to bring inference to all applications. That said, the generative AI development platform is receiving another slate of features and capabilities now generally available or in preview.
“Amazon Bedrock is helping to tackle the biggest roadblocks developers face today, so customers can realize the full potential of generative AI,” AWS Vice President of AI and Data, Dr. Swami Sivasubramanian, says. “With this new set of capabilities, we are empowering customers to develop more intelligent AI applications that will deliver greater value to their end users.”
Among the updates are support for prompt caching and intelligent prompt routing, structured data retrieval, GraphRAG, and data automation. Several new models, specifically from Poolside, Stability AI, and Luma AI, are also coming to Amazon Bedrock. It’s here where we start our story.
Disclosure: I attended Amazon's 2024 re:Invent as a guest, with a portion of my travel expenses covered by the company. However, Amazon had no influence over the content of this post—these thoughts are entirely my own.New Amazon Bedrock Models
The generative AI app development platform already supports many models from leading companies, including AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, and Stability AI, not to mention Amazon’s new foundation model family, Nova. However, AWS believes developers should be free to choose from whichever model they want. Therefore, the company continues to add more options.
Coming soon to Amazon Bedrock are Poolside’s Malibu and Point, an engineering AI for code generation, testing, documentation, and real-time code completion; Stability AI’s Stable Diffusion 3.5 Large, an advanced text-to-image model; and Luma AI’s Ray 2, the second-generation model that turns text and images into video clips. AWS declined to specify precisely when these models would be available, only saying that it’ll be “soon,” except Poolside—its model is believed to be released in early 2025.
New Bedrock Tools
AWS is adding new ways for developers to tackle inference at scale. After all, the company feels inference is core to applications. Amazon Bedrock now includes prompt caching and intelligent prompt routing—the former caches frequently used prompts in a manner that doesn’t reduce repeated processing without also sacrificing accuracy. The latter involves automatically routing simple questions to smaller foundation models within the same family, optimizing for response quality and cost.
Any organization can deploy prompt caching and routing, though it requires specialized expertise. In place of this expensive price tag, AWS believes its offering is more palatable and autonomous—it’s too good an offer for developers to pass up, right?
Improved Workability with Enterprise Data
Three new features for Amazon Bedrock are designed to make working with enterprise data smoother and ensure that all data can be used to train intelligent applications.
Two involve Amazon Bedrock Knowledge Bases, the system in which proprietary information is integrated into generative AI applications. The first is support for structured data retrieval. Now generally available, it eliminates a problem developers have with large language models: They aren’t always great with structured data infrastructure such as databases, data warehouses, and data lakes. With structured data retrieval, developers only need a text prompt to run queries into databases.
Another feature involves a Retrieval Augmented Generation (RAG) variation called GraphRAG. With Amazon Bedrock now supporting it, developers can leverage the knowledge graph—a model representing the relationship between data points—to generate graphs through Amazon Neptune. This automates a task that normally would require someone with graphing expertise. GraphRAG is available in preview mode.
Lastly, we move from tackling structured data to unstructured. Amazon Bedrock is receiving a feature called Data Automation. Available as a preview today, it rapidly extracts information from documents, images, videos, and audio and converts it into structured data. From there, the data can be used for intelligent workflows, including document processing and RAG. This solves a problem most enterprise companies have: How can they convert their unstructured data into something usable with AI?
The latest updates to Amazon Bedrock highlight AWS’s focus on expanding the capabilities of generative AI for developers and enterprises. With new models, tools for inference optimization, and features to simplify working with structured and unstructured data, the platform is clearly evolving to meet a broad range of needs. However, questions about timelines for model availability and the practical ease of integrating these features into existing workflows will be key factors in determining their impact. As AWS continues to refine its offering, these additions set the stage for further advancements in AI-powered application development.
Featured Image: An AI-generated image representing Amazon Bedrock overlaid with the AWS logo. Image credit: Adobe Firefly
Leave a Reply
You must be logged in to post a comment.