AssemblyAI outlines browser voice assistant build in JavaScript

AssemblyAI published a JavaScript tutorial on Oct. 13, 2026, showing how to build a browser voice assistant that listens through the microphone, sends audio over a single WebSocket and plays replies back in the tab. The guide argues that the approach is aimed at conversational assistants rather than simple command-and-response voice controls.

Evidence and context

In the tutorial, AssemblyAI contrasts the browser’s built-in Web Speech API with its Voice Agent API. The Web Speech API can handle speech recognition and browser text-to-speech, but the company says it leaves understanding logic, interruption handling and LLM integration to the developer. The Voice Agent API instead runs speech recognition, LLM orchestration and text-to-speech server-side behind one WebSocket, with JavaScript only capturing audio bytes and rendering transcripts.

The tutorial says the browser client connects to wss://agents.assemblyai.com/v1/ws using a temporary token minted by a small Node server. The API key stays on the server, and the browser sends 24 kHz, 16-bit PCM audio while receiving 24 kHz PCM back. AssemblyAI says the flow is designed to reduce latency and simplify the pipeline by avoiding separate STT, LLM and TTS vendors.

AssemblyAI also describes browser support and implementation details. Its example uses an AudioWorklet for mic capture, resamples audio inside the worklet so the code behaves consistently across Chrome, Firefox and Safari, and schedules playback so reply.audio chunks play gaplessly. The tutorial says the assistant supports barge-in, built-in turn detection and tool calling, and that a session can be ended cleanly by sending session.end and waiting for session.ended before releasing the microphone and audio context.

The guide places the browser build in the context of its broader Voice Agent API documentation, which on Oct. 13, 2026 documents a WebSocket conversation endpoint, API-key or temporary-token authentication, binary audio streaming, runtime prompt updates and function calling. Background material supplied with the tutorial says Deepgram made related Voice Agent SDK changes on Oct. 12, 2026, including custom Think-provider messages, after an earlier documentation correction on Sept. 18 clarified that its Browser Agent SDK never shipped client-side VAD. It also notes earlier Voice Agent SDK releases on Sept. 14 added ForceEndTurn support and Flux TTS expressivity controls.

The tutorial does not present performance comparisons against independent benchmarks. It does state a flat Voice Agent API price of $4.50 per hour for speech recognition, LLM and text-to-speech, while noting that new accounts receive free credits. The article’s own example is framed as a production-shaped browser assistant built in roughly 250 lines of code, though the exact maintenance burden and reliability in real deployments will depend on the developer’s prompt design, tools and UI choices.

Technical details highlighted in the tutorial

AssemblyAI’s example server mints single-use temporary tokens from a Node process, with an expires_in_seconds window of 1 to 600 seconds. The browser fetches a fresh token for each connection, including reconnects. The sample client requests microphone access with echoCancellation enabled, sets a system prompt and greeting on session.update, then streams mic audio only after session.ready arrives.

The guide also calls out practical limits and troubleshooting points: getUserMedia requires a secure origin, local development is allowed on http://localhost, and production deployment needs HTTPS. It warns that stale or reused tokens can cause unauthorized errors or a 1006 WebSocket close, and that sample-rate mismatches can distort audio if raw 44.1 kHz or 48 kHz samples are mislabeled as 24 kHz PCM.

Because the article is a product tutorial rather than a market event, it does not include customer numbers, adoption data or analyst commentary. Its significance is technical: it shows how AssemblyAI is positioning the Voice Agent API as a browser-native route for conversational assistants without exposing API keys in the client or chaining together multiple vendors for recognition, reasoning and speech output.