AssemblyAI says voice teams can cut LLM costs with workload routing

AssemblyAI said voice-product teams can lower LLM spending by matching each post-transcription task to the smallest model that still meets quality requirements, rather than sending every workload to a frontier model by default. The company released Voice Standard Bench on 2026-10-13, a benchmark that scores more than 50 models on the tasks voice products commonly run, with quality, speed and cost shown side by side, according to AssemblyAI's release material.

What AssemblyAI released

AssemblyAI described Voice Standard Bench as a ranking system for models on LLM Gateway across common voice workflows such as transcript cleanup, summarization, information extraction and classification. The company said the benchmark evaluates models at four transcript lengths and is intended to help teams identify where higher-cost models are unnecessary. The related blog post, published on 2026-10-13, argues that the largest voice-LLM bills often come from downstream processing after speech-to-text, not from transcription itself.

AssemblyAI also highlighted that its LLM Gateway exposes more than 50 models from OpenAI, Anthropic, Google and other providers through an OpenAI-compatible API. Among the lower-cost options it listed were Qwen3.5 4B Fast at $0.10 input and $0.50 output per 1 million tokens, Gemma 4 31B at $0.14 and $0.40, GPT OSS 20B at $0.07 and $0.30, and Nemotron Lightning 3.5 30B A3B at $0.05 and $0.20, with prices stated as of 2026-10-08 in the source material.

Why transcript length matters

The company said transcript length changes which model is most suitable. In its benchmark table, AssemblyAI reported that for 30-second transcripts the recommended pick scored 96, while a small open model scored 94 at a much lower cost, and Gemma 4 31B scored 93. For 2-minute transcripts, the recommended pick scored 94, while smaller models remained close behind. At 15 minutes, the gap widened, and at 60 minutes larger models pulled ahead, with AssemblyAI noting that GPT-6.1 Sol matched the top score of 88 at a lower listed cost than the top recommendation in that row.

AssemblyAI said the same pattern applies to structured extraction as well as summarization. It warned that a model can appear inexpensive on paper but become more costly if it produces malformed JSON, refusals or timeouts that trigger retries and downstream failures. The company said its quality score counts those failures rather than only successful completions.

Cost example and operating caveats

In one worked example, AssemblyAI compared monthly summarization costs for 10,000 support calls of about 15 minutes each, assuming 3,200 input tokens and 300 output tokens per call. Under those assumptions, the company calculated costs of $141 for Claude Sonnet 5, $35 for Gemini 3.8 Flash, $5.68 for Gemma 4 31B and $4.70 for GPT-6 Luna. AssemblyAI said that at 1 million calls a month, the difference would scale to roughly $14,100 versus $470 to $568 before any regional surcharge.

The blog post said teams should test cheaper models on their own transcripts before switching, because vocabulary, accents and output requirements vary by customer base and use case. It recommended a four-step process: audit token usage, shortlist two or three candidates per workload, test them on real transcripts, and then migrate one workload at a time with a fallback model in place. AssemblyAI also said some third-party-served models can face capacity constraints under sudden load, even when account rate limits are higher.

As a timing note, AssemblyAI said Claude Sonnet 4.5 on LLM Gateway is scheduled to retire on 2026-11-30, making that one opportunity to re-evaluate which model should replace it. Beyond that date, no further catalyst was provided in the source material.