A guest calls the hotline at 11 PM. In Polish. Switching to German mid-sentence because they don't know the word for the key safe. The AI bot handles the call, the conversation ends, and LiveKit sends us the transcript.
The transcript looks like this:
caller: Dzień dobry yyy jak się yyy ten der schlüsselkasten
caller: yes yes yyy like the box the thing for the key
ai: I understand you have a question about access. Can you confirm
your apartment address?
caller: yes the number uh the number i forgot uh Główna five or
was it six yes six i thinkYou can roughly understand it. An LLM can too — with effort. But “the thing for the key,” “yyy,” and an address that contradicts itself two turns later are not useful signal for an AI that's trying to decide whether the bot handled that call well.
We run a bot-improvement loop that reads recent transcripts and proposes prompt rewrites. When I looked at what it was reading, I realised it was spending half its attention on noise. Filler words. Garbled place names. Partial sentences that STT cut mid-breath. The suggestions it produced were reasonable — but they were reasonable responses to a mangled version of the call, not the actual call.
That's when we decided to clean the transcripts before any AI reads them for analysis.
The problem: garbage in, garbage out
Speech-to-text engines are trained on clear, well-paced speech. Real support calls are not that. Guests call in the middle of the night, with background noise, from non-native speakers switching between three languages, about problems they're stressed about. The STT engine does its best, and its best is often wrong in specific, predictable ways:
- Filler words. Every “yyy,” “uh,” and “um” makes it into the transcript verbatim. They carry zero information but fragment the sentence for any downstream NLP.
- Phonetic errors on proper nouns. Apartment names, street names, Polish surnames — the model hallucinates plausible-sounding replacements. “Główna” becomes “Glovna” or “Gwowna” depending on the day.
- Code-switching artifacts. A sentence that starts in Polish and ends in German breaks the speaker model. Turns get split mid-thought or merged across speakers.
- Short utterances. Single-word confirmations (“Yes,” “OK,” “Right”) frequently get transcribed as something phonetically adjacent but semantically different.
- Plausibility failures. Occasionally a turn has no coherent meaning at all — it's just noise the STT engine tried to interpret as speech.
None of these individually kill a transcript. But when an LLM reads twenty turns of filler-word-contaminated, phonetically- garbled, mid-sentence-split dialogue and tries to extract what the guest needed, the signal-to-noise ratio is bad enough that the conclusions are unreliable.
The improvement loop was proposing prompt changes based on “problems” that were never problems — they were transcription errors. That's the failure mode we wanted to fix.
The naive approach — and why it fails
The first thing most people try is cleaning at the turn level. Parse the transcript into structured speaker turns, then run a cleaning pass on each turn's text field independently.
We considered it and rejected it for one reason: context. A single turn in isolation doesn't give you enough to decide whether “Glovna five” is a mishearing of an apartment name or a legitimate answer to a different question. You need the surrounding turns. The AI agent's last question. What the guest said three turns earlier. That context exists in the full transcript string — it disappears when you chop it into individual turns.
We also considered post-hoc correction via a second read after the turns were already stored. That would mean maintaining a separate async pipeline, reconciling IDs, and handling the failure mode where the correction runs but the original transcript is already being read downstream. More moving parts than we wanted.
The approach we chose is simpler: pass the full raw transcript string to an LLM, ask it to return the same transcript with STT artifacts corrected. Then parse both versions into turns and store them side by side. The normalisation happens synchronously in the same request that saves the transcript — no second pipeline.
The normalisation pipeline
When LiveKit sends us a completed call, the pipeline runs in this order:
The entry point is the LiveKit webhook handler. It receives the raw transcript as a plain string — each line prefixed with either “AI:” or “caller:”. Before we parse that into structured turns, we pass the whole string to the TranscriptNormalisationService:
const transcriptNormalized =
await this.transcriptNormalisationService.normalizeTranscript(
data.transcript,
);
const turns = transcriptToTurns(data.transcript);
const turnsNormalized = transcriptNormalized
? transcriptToTurns(transcriptNormalized)
: [];The service sends the full transcript to the OpenAI API with a system prompt that instructs the model to return the same transcript with STT artifacts cleaned — no summarising, no paraphrasing, no removing content that's awkward. Just fix what the microphone got wrong:
// TranscriptNormalisationService
async normalizeTranscript(rawTranscript: string): Promise<string | undefined> {
return this.chatGptService.generateResponse(
[
'You are a transcript editor. Clean the following voice call transcript:',
'- Remove filler words (yyy, uh, um, hmm)',
'- Fix obvious phonetic STT errors on proper nouns and addresses',
'- Preserve the exact speaker prefixes (AI: / caller:) and line structure',
'- Do not paraphrase, summarise, or remove any substantive content',
'- If a turn is unintelligible noise with no recoverable meaning, keep it as-is',
'- Return only the corrected transcript, nothing else',
].join('
'),
[{ role: 'user', content: rawTranscript }],
);
}Both sets of turns get stored in the call_transcripts table: raw turns from the original, and turns_normalized from the cleaned version. They're both JSONB columns — same schema, separate fields.
Downstream readers that need the most accurate signal — specifically the bot-improvement data service that feeds the improvement loop — read turns_normalized first and fall back to raw turns for older transcripts that predate the normalisation step:
const fullTurns = Array.isArray(row.turns_normalized)
? row.turns_normalized
: Array.isArray(row.turns)
? row.turns
: [];The decisions that mattered
A few choices here weren't obvious. Here's what we decided, what we rejected, and why.
1. Clean the string, not the turns
We normalise before transcriptToTurns(), not after. Rejected: cleaning turn objects one at a time. Reason: inter-turn context is essential for resolving ambiguous corrections. A phonetic error on a street name can only be confidently fixed if you know what the AI agent asked in the previous turn. Single-turn cleaners can't see that.
2. Store both raw and normalised — never overwrite
The raw turns are the audit record. We never overwrite them with the normalised version. Rejected: a single turns column that gets updated after normalisation. Reason: if a guest or operator ever disputes what was said on a call, the raw STT output is the closest thing we have to a verbatim record. Normalisation is a lossy process — it's right most of the time but not always. Keep both.
3. Empty array, not null, as the graceful fallback
If normalisation fails — the LLM API is down, the response times out, the model returns nothing useful — we store an empty array in turns_normalized. Rejected: storing null or propagating the error up. Reason: the entity schema requires a non-null array, and downstream readers already know how to fall back to raw turns when the normalised array is empty. An empty array is an honest signal that says “normalisation didn't run.” Null would be ambiguous — is it not run, or not applicable?
4. A dedicated service, not a method on EscalationService
Normalisation lives in TranscriptNormalisationService, not bolted onto the existing escalation service. Rejected: adding a normalisation method to the escalation or hotline service. Reason: transcript normalisation is a standalone concern — it touches no escalation logic, no hotline entities, no support cases. It takes a string and returns a string. Keeping it separate makes it independently testable and reusable if another service ever needs to clean a transcript.
What we haven't built yet
A few things we're aware of and haven't done:
- Normalisation quality scoring. We don't currently measure whether the LLM improved the transcript or made it worse. In rare cases it over-corrects and removes something meaningful. A turn-by-turn diff against the raw version would let us flag suspicious normalisations for manual review.
- Backfill for existing transcripts. Transcripts stored before we shipped this pipeline have
turns_normalized = []. The improvement loop falls back to raw turns for those, which is correct but suboptimal. A one-time migration job could normalise the backlog. - Language-aware prompting. We know the call's guest language from the LiveKit metadata. We don't currently pass it to the normalisation prompt. A Polish-specific hint would help with street name corrections; a German hint would help with compound nouns that STT splits incorrectly.
The takeaway
AI analysis is downstream of data quality. You can have the most sophisticated improvement loop, the most carefully tuned prompts, the best flagging workflow — and if the transcripts it reads are full of filler words and phonetic guesses, the suggestions it produces will reflect that noise.
The normalisation pipeline is three things: a service that calls an LLM with a focused system prompt, a parser that runs twice on the same content, and a fallback that stores an empty array when something goes wrong. The whole thing is under a hundred lines.
The unglamorous truth: making AI analysis better often means making the data better. Not a new model, not a more elaborate prompt. Just cleaning what's there before anyone reads it.