Voice agents have finally reached the awkward job interview stage: they sound confident, they interrupt less, and then one of them forgets why it called the dentist. That is why Artificial Analysis launching a Speech Agent Arena matters. It moves the conversation from demo theater to measurable behavior, which is nice because vibes are not an evaluation metric, despite what several pitch decks and one haunted sales webinar keep insisting. ## Artificial Analysis turns voice demos into blind trials Artificial Analysis says its Speech Agent Arena compares speech to speech models by preference across blind, live voice conversations, including tasks like booking a dental appointment and ordering takeout, according to the Speech Agent Arena Leaderboard. AlphaSignal reports that the arena covers 35 real tasks, which makes it more useful than the usual booth demo where the model orders a pizza with the emotional range of a cruise director. The key design choice is that humans judge conversations without seeing which model produced them, while task success is tracked separately. That separation is the whole story. A voice model can be pleasant, fast, and convincingly alive while still fumbling the actual errand, which is basically the AI version of a charming intern with admin permissions. For product teams, this is the difference between “users liked the call” and “the appointment was actually booked,” a distinction your support queue will discover very quickly if you do not. ## Google wins preference, but completion tells a different joke The Speech Agent Arena Leaderboard lists Google’s Gemini 3.1 Flash Live Preview - Minimal in first place with 1046 Elo, 768 samples, and a 74.6% task success rate. The same leaderboard puts Google’s Gemini 3.1 Flash Live Preview - High second with 1014 Elo, 763 samples, and a 71.8% task success rate, while OpenAI’s GPT-Realtime-1.5 sits third with 1000 Elo, 762 samples, and an 85.1% task success rate. In other words, the model people prefer is not automatically the one that gets more work done. AlphaSignal sharpens that contrast by reporting that Grok Voice Think Fast 2.0 High leads task completion at 94.7% while ranking ninth on preference. That is a useful little grenade lobbed into the voice agent hype machine. If you are building a receptionist, travel assistant, or ordering agent, you may care less about sparkling banter than whether the thing can survive contact with a menu, a calendar, and a human who says “actually, never mind” three times. ## Responsiveness is eating the preference score AlphaSignal reports that preference tracks responsiveness closely, with lower Time to First Audio correlating with higher Elo. This should surprise nobody who has ever waited on a phone tree while a synthetic voice pauses like it is consulting an oracle made of wet cardboard. Latency is not a polish feature in voice agents; it is part of the product’s perceived intelligence. Artificial Analysis already frames speech to speech evaluation across reasoning quality, conversational dynamics, generation time, and price in its Speech to Speech Models and Providers Analysis. That broader context matters because no single leaderboard row can answer every production question. A model that feels great in a blind preference test may still be wrong for a workflow if it costs too much, responds too slowly under load, or completes tasks unreliably. ## What builders should do with the leaderboard AlphaSignal reports that prices across leaderboard models span $1.50 to $10.75 per hour of input audio, which is the part where the demo suddenly develops accounting consequences. For teams evaluating voice agents, the sane approach is to treat Speech Agent Arena as a shortlist generator, not a procurement spell. Start with preference and task success, then run your own calls with your scripts, your accents, your failure modes, and your customers’ magnificent talent for saying things no benchmark author predicted. The bigger signal is methodological. Artificial Analysis is giving builders a way to compare live, blind speech interactions against task outcomes, and that is exactly where voice AI needs to be tested if it is going anywhere near customer facing work. Watch how fast the rankings move, but watch the gaps more closely: preference, latency, completion, and price are now tugging in different directions. Voice agents are no longer just learning to talk; they are learning that talking is the easy part. ## Sources - Artificial Analysis' Speech Arena Reveals Voice AI's Uncomfortable Split Brain Problem | AlphaSignal

Sources