What Is WER: How Speech Recognition Accuracy Is Measured

The short answer

WER (Word Error Rate) is the main accuracy metric for speech recognition (ASR). It shows what share of words were recognised wrongly compared with a reference transcript made by a human.

The formula:

WER = (S + D + I) / N

• S (substitutions) — words replaced by other words;
• D (deletions) — words that were dropped;
• I (insertions) — extra words that were added;
• N — the number of words in the reference.

Lower is better. Because of insertions, WER can exceed 100%.

A worked example

Reference: "please transfer the payment today" — 5 words. Recognised: "please transfer payments today now".

• "payment" → "payments" — substitution (S = 1);
• "the" was dropped (D = 1);
• "now" was added (I = 1).

WER = 3 / 5 = 60%. Small details count: a plural ending is a full word error. Likewise, differences in capitalisation, punctuation or how numbers are written will inflate WER unless both texts are normalised the same way. Without a shared normalisation, comparing numbers is meaningless.

Related metrics

For a business, the overall WER often matters less than whether the system gets the contract number, the amount and the service name right.

How to measure WER on your own calls

1. Collect a sample. Take real recordings that reflect your conditions: different agents and customers, noise, poor connections, Russian, Uzbek and mixed speech. Do not pick only the clean ones.
2. Create reference transcripts. A person transcribes recordings verbatim using fixed rules: how to write numbers, abbreviations, filler words and language switches.
3. Normalise the text. Bring the reference and system output to the same form: lowercase, no punctuation, consistent spelling of apostrophes, letters like "ё" and numbers.
4. Compute metrics separately for each group of conditions: noisy vs clean calls, Russian, Uzbek, mixed.
5. Review errors by hand. See which words get confused most — that tells you where a custom vocabulary or fine-tuning is needed.

The Python library jiwer is one convenient option:

import re
import jiwer

def normalize(text: str) -> str:
    text = text.lower()
    text = re.sub(r"[^\w\s]", "", text)
    return " ".join(text.split())

reference = normalize("Please transfer the payment today.")
hypothesis = normalize("please transfer payments today now")

print(jiwer.wer(reference, hypothesis))  # 0.6
print(jiwer.cer(reference, hypothesis))

Why vendor figures don't match yours

Advertised accuracy is usually measured on public benchmarks — mostly clean speech read aloud into a good microphone. Your calls differ in almost every way:

• Phone channel. Narrow bandwidth and compression degrade audio quality.
• Noise. Streets, shops, cars, background conversations.
• Mixed languages. A customer starts in Uzbek, drops in Russian words and terms — models trained on one language make more mistakes on such phrases.
• Uzbek specifics. It is agglutinative: one wrong suffix makes the whole word an error, so WER looks worse than actual understanding. Add Latin and Cyrillic scripts and several apostrophe variants in o‘ and g‘.
• Domain vocabulary. Your product names, tariffs, street names and personal names.
• Different normalisation. A vendor may compute WER with its own rules, which cannot be compared with yours.

So choose a system only after testing it on your own sample, applying the same rules to every candidate.

Common mistakes

• Comparing WER from different vendors calculated on different data.
• Testing on a handful of lucky recordings.
• Looking only at average WER without splitting by conditions.
• Forgetting that the task depends on specific entities, not every word.