AssemblyAI guide details how text shapes TTS delivery

AssemblyAI has published guidance arguing that text, not just voice selection, is what makes text-to-speech output sound human. In the company’s view, the most natural results come from writing replies the way people speak them, rather than the way they write emails or scripts, because the model takes pacing, tone and hesitation cues from the input text.

Evidence and context

The guide says its approach applies to AssemblyAI-TTS-1, which the company says was trained on millions of hours of audio, including real conversational data. It also says the API does not use a markup language or style tags, and that bracketed stage directions such as [laughs] or [pause] are read aloud as words after square brackets are converted to round ones. AssemblyAI’s guidance therefore places most of the burden on sentence structure, with punctuation and spelling used as secondary controls for emphasis, pauses and how specific entities are spoken.

Among the punctuation rules described, an exclamation mark adds emphasis, a question mark signals question tone, and the combination ?! can add a small note of surprise. Commas and ellipses create short pauses, while a trailing double dash indicates a sudden cutoff. The company also warns that stacking punctuation does not increase the effect: !!! collapses to one exclamation mark, while multiple commas or ellipses do not lengthen the pause. The guide says CAPS can add stress to a single word, but should be used sparingly.

AssemblyAI’s examples also focus on disfluencies. It says fillers should be written as plain words between commas, such as "It's, uh, right here," with "um" written as "um" rather than "ummm." Stutters use dashes inside the word, false starts repeat the first word before a dash and a space, and trailing off uses an ellipsis. The guide says these elements can be combined, and notes that short conversational words such as "Okay," "And," "So" and "Yeah" are often reliable in spoken-style replies.

The article also includes before-and-after examples intended to make replies sound more conversational. In one appointment-confirmation example, the after version replaces a dense first turn with a shorter opening, spells out the date as "November 3rd," and separates "Okay?" into its own check-in. In a payment example, the visible text is cut off in the source material after "So your current balance is $1,248.50, and it's due on the 15th," so the remainder of that example was not disclosed.

AssemblyAI says one practical rule is to open softly, keep a light stumble, use fragments when they fit the meaning, say the words rather than shorthand, and keep the first reply short. The guide also says voices differ in how they render the same text, and recommends listening tests before settling on a style. It adds that some observations, including a note that certain voices can produce more upward inflection than desired, came from internal testing and are not documented behavior.

Broader technical context supplied alongside the guide aligns with AssemblyAI’s position that natural-sounding TTS depends on both wording and model-specific delivery controls. Separate documentation from ElevenLabs says some of its models do not support SSML break tags and instead rely on audio tags, punctuation, ellipses and text structure to shape pauses and pacing, while other models can use for pauses up to three seconds. OpenAI said on 2025-03-20 that gpt-4o-mini-tts supports instructions about delivery. The guidance also notes that punctuation and tags are not universal timing guarantees and should be tested on the exact engine and voice selected.

The announcement did not disclose any pricing, usage limits or rollout timing beyond stating that the guidance applies to AssemblyAI-TTS-1. It also did not specify when, if ever, the company plans to update the prompting guide beyond the reference already available in its docs.