Small LLMs stay competitive on short transcripts, then fade at meeting length

AssemblyAI said its benchmark data shows small language models can match larger systems closely on short voice transcripts, but lose that edge as transcript length increases. In its Voice Standard Bench run published Oct. 13, 2026, the company said the performance gap widens on longer meetings, while the cost advantage of smaller models narrows once the task becomes more complex.

Evidence and context

The post, published Oct. 20, 2026, frames the question as one of transcript length rather than model class alone. AssemblyAI said the current Voice Standard Bench run compares models on real speech-to-text output at about 30 seconds, 2 minutes, 15 minutes and 60 minutes. In the chart it provided, GPT-6 Luna scored 96, 94, 89 and 88 across those lengths, while gemma-4-31b scored 95, 92, 87 and 85, Qwen3.5 4B Fast scored 93, 89, 64 and 63, and another model listed in the post scored 74, 64, 47 and 31. AssemblyAI said scores within 2 to 3 points are effectively tied, and that gemma-4-31b stays close to the leader on short inputs before dropping more sharply by 15 minutes.

The company also said the economics change with length. In the current run, the top-scoring model costs 15 to 77 times more per hour of audio than a model within two quality points of it, according to AssemblyAI. On hour-long transcripts, that premium narrows to about 4 times, which the company said is where larger models begin to earn their higher cost. AssemblyAI also said GPT-6 Luna scores 85 at 60 minutes for about $0.004 per hour of audio, compared with GPT-6.1 Sol at 87 for about $0.09 per hour, and that the top 60-minute score in the run was GPT-6 Astra at 88.

For small models, AssemblyAI highlighted Qwen3.5 4B Fast as the fastest option in the benchmark up to 15 minutes, with median response times of 0.28 seconds at 30 seconds, 0.60 seconds at 2 minutes and 1.49 seconds at 15 minutes. The company said it costs $0.10 per million input tokens and $0.50 per million output tokens, and that it is hosted by AssemblyAI on its own GPUs with a 32,768-token context window. AssemblyAI also said the model is intended for rewrite-style work such as dictation cleanup, transcript rewrites into emails or notes, and live formatting, but that it lacks tool calling and structured output, making it unsuitable for JSON extraction or voice agents.

AssemblyAI said the main reason small models underperform on long transcripts is not simply context-window overflow. The post notes that an hour-long meeting still fits within Qwen3.5 4B Fast's 32,768-token window, and argues that the harder problem is keeping track of decisions, action items and named entities over long conversations. The company said its methodology measures tasks such as summarization coverage and extraction completeness, which become more demanding as transcripts grow.

The post also distinguishes size from serving setup. AssemblyAI said the same open-model families hosted elsewhere can take 10 to 20 seconds to respond, while Qwen3.5 4B Fast is tuned for sub-second latency on AssemblyAI infrastructure. It added that hosting determines which provider processes a transcript, and that its own benchmark includes models served through AssemblyAI, AWS Bedrock, DigitalOcean and Fireworks.

AssemblyAI's routing guidance is to use the fastest model that fits the task for transcripts under about 2 minutes, compare budget and recommended models for 2 to 15 minutes, and start with a larger model for meeting-length audio. The company said benchmarks are a starting point rather than a final verdict, and recommended testing on an application's own transcripts before deciding where small models are sufficient.