Microsoft’s platform for building and deploying AI apps and agents is getting an update. Announced at this year’s Build conference, Azure AI Foundry (née Azure AI Studio) has new tools to simplify agent development, automate model selection, enhance monitoring, and support on-device AI. It’s all aimed at helping companies scale their generative AI solutions with greater speed, flexibility, and control.
First-Party Models Come to the Foundry
It starts with the foundation models, of which more than 1,900 are available in the foundry’s library. And Microsoft isn’t stopping there—it’s adding “direct, first-party offerings,” models hosted and operated by Microsoft and backed by its service level agreements (SLAs). The first of these is xAI’s Grok 3 and Grok 3 Mini.
Grok 3 is the latest LLM from Elon Musk’s xAI. It was trained using 100,000 Nvidia H100 GPUs, ten times more compute power than its predecessor. The model family includes multiple variants, including Grok 3, Grok 3 mini, Grok 3 Reasoning, and Grok 3 mini Reasoning. However, Microsoft will only host the first two.
Under this arrangement, Microsoft will be responsible for the model’s performance, uptime, and integration within the Azure ecosystem. This gives enterprise customers peace of mind when using these models, knowing that the AI is being maintained by a company with an extensive history and understanding of regulatory and compliance standards. Hosting the models also allows Microsoft to ensure they’re effectively and adequately integrated across its AI stack.
First-party offerings also make it easier for the model makers. Developers can focus more on training their LLMs and let Microsoft handle the billing and distribution.
Subscribe to The AI Economy
Foundry Agent Service Now Generally Available
In November 2024, Microsoft introduced Azure AI Agent Service, a platform for building, deploying, and scaling enterprise-grade AI agents. For the past six months, it was only available as a public preview, but today, it’s generally available.
The offering’s multi-agent workflow support means developers can instruct multiple specialized agents to handle different and complex tasks. As it’s known now, Azure AI Foundry Agent Service also integrates with Google’s Agent2Agent (A2A) protocol and Anthropic’s Model Context Protocol (MCP), enabling interoperability across agent frameworks and applications.
New Monitoring and Evaluation Tools
Microsoft is releasing new tools to help developers manage their agents better. As part of the company’s Azure AI Foundry Observability, these tools give developers a view of their bots’ performance, quality, cost, and safety. Microsoft explains that these capabilities “are designed to provide deeper insights into the quality, performance, and safety of agents.”
Developers will also receive end-to-end support tools, which will enable them to run evaluations during model tuning, improve system prompts, and manage transitions between models. When agents are out in the wild, Azure AI Foundry Observability provides them with a single dashboard for continuous monitoring. It only requires one configuration step to have the system’s quality and safety evaluators running continuously.
The new monitoring and evaluation tools are in preview today.
Automate Model Selection and AI App Design
For those companies that prefer to use multiple models in their applications but aren’t sure which one is the best, Microsoft has a tool to help. Called Model Router, it will automatically select the optimal OpenAI model for prompts with the goal of higher quality and lower cost outputs. It’s available in preview today.
In addition, Azure AI Foundry now has new AI templates designed for “common, high-value use cases and technical patterns.” Created by customers, Microsoft says these templates enable developers to design, customize, and deploy AI solutions quickly.
Azure AI Foundry Services, Models, and Tools Come to Copilot Studio
Microsoft is integrating Azure AI Foundry Services to Copilot Studio, allowing developers to build low-code copilots using any of the 1,900 Foundry Models, vectorized indices for Retrieval-Augmented Generation (RAG), and multi-agent orchestration through the Azure AI Foundry Agent Service.
Running Models Locally on Client Devices
Developers looking to implement AI at the edge will soon be able to use Azure AI Foundry Local. Available in preview, this capability makes running open-source models, tools, and agents easy on any Windows 11 or macOS device. Powered through ONNX Runtime, Foundry Local is intended to support cases where internet data usage is a concern, prioritizing privacy and reducing costs. This could benefit organizations with field personnel in rural areas where connectivity is limited and AI use is vital—perhaps similar to Meta’s former program Free Basics, but specific to AI.
Featured Image: AI-generated image of a futuristic foundry. Credit: Adobe Firefly
Subscribe to “The AI Economy”
Exploring AI’s impact on business, work, society, and technology.
Leave a Reply
You must be logged in to post a comment.