Sierra Debuts Voice Sims to Train AI Agents for Chaotic Customer Calls

Before an AI voice agent ever picks up the phone, companies need to know it is well-suited to handling the chaos of a real customer call. To help ensure this happens, Sierra is launching Voice Sims, a new tool that uses AI to train and refine these bots. It acts like a quality assurance system for conversations, allowing businesses to test agents under real-world conditions so they’re fully prepared for live service.

Founded by former Salesforce co-CEO Bret Taylor and ex-Google Labs lead Clay Bavor, Sierra is a platform used to build enterprise-grade customer service agents. In its nearly two-year existence, the company has made significant strides in developing AI-powered solutions, including those focused on voice conversations. Last October, it debuted voice capabilities for its customer service bots.

Getting an AI agent ready for prime time, however, requires more than providing it instructions. In a blog post, Sierra notes that, unlike text, it’s critical to pay attention to the cadence of the conversations, such as “when to stop and start talking, or pause.” In addition, it’s vital to recognize that words could be misunderstood due to someone’s accents, background noise, industry acronyms, and/or bad connections—anyone who uses voice transcription services will know this to be true. Moreover, a flat tone or a poor word choice could also impact the response given.

With Voice Sims, Sierra provides multiple virtual “users,” who will put the voice agent through the ringer. These ad hoc bots can simulate customers with diverse languages, needs, and environments—whether they’re at home with the TV blasting, on the street, or riding a train—as well as varying emotional states and unique situations. Voice Sims promises to save businesses time by eliminating the need for manual QA testing, a time-intensive process.

“Sierra’s Simulations offering uses AI to create synthetic conversations with your agent,” a company spokesperson tells me. “This can be used in various ways, including testing and evaluating your agent end-to-end, coming up with training data, and avoiding regressions pre-launch. After simulating the conversation, we use an LLM as a judge to evaluate whether the agent passed the simulation and met its expected criteria.”

There are two ways to set up Voice Sims: auto-generated or manual. With the former, Sierra will automatically generate test cases for AI agents based on existing Standard Operating Procedures (SOPs), knowledge bases, coaching transcripts, conversational flows, and other policies. The latter provides more granular control, allowing admins to input any scenario they wish to test repeatedly. They only need to give the scenario description, success criteria, and the desired location or device information to get started.

Sierra says Voice Sims integrates with its Agent Studio and a company’s existing CI/CD infrastructure, so whatever other channels are being tested (e.g., chat, email, or messaging), voice simulations can be evaluated within the same system.

Improving call quality is vital. Companies are sacrificing customer support staff in favor of AI, and if they want to realize the value of those choices, the bots need to sound human-like, even though they’re not. We’ve seen examples where companies have backtracked on their decisions, such as with Klarna. Taylor’s former boss, Marc Benioff, recently commented that Salesforce let go of 4,000 customer support roles in favor of AI because “I need less heads.”

Ensuring that these AI agents can perform many of the same tasks as well as, if not better than, humans is necessary for businesses to justify layoffs or to ensure that their AI strategies are yielding positive results. A tool like Voice Sims could help do that.

“The impact of Voice Sims is simple but profound: agents become more reliable and more empathetic, and conversations more natural,” Sierra explains in a blog post. “By continuously creating simulated conversations, you catch brittle “only in calls” bugs early. By testing noisy, emotional, interrupted conversations on repeat, you refine prompts, and delivery until they sound right. And because it’s automated, teams can run fast—re-doing the same tough scenarios on every change, just like with unit tests, to ensure the agent is launch ready and improving overtime.”

Today’s announcement comes days after Sierra announced it raised $350 million in new funding at a $10 billion valuation. To date, investors have put in $635 million in the startup. The company boasts that 20 percent of its customers have over $10 billion in revenue, and half have over $1 billion in revenue. It plans to use the new investment to expand its platform and accelerate growth in both the U.S. and internationally.

Sierra’s Voice Sims is available free of charge to customers.

Featured Image: An AI-generated image of audio soundwaves on a smartphone. Credit: Adobe Firefly