This section is about two jobs that every voice model needs done before it ships. The first is to measure how good its speech is. People rate generated speech in listening tests, and automatic scores stand in for those people between tests. The second is to stop the speech from being misused. A model that speaks well enough to fool a listener can also fool a bank, a voter, or a parent. Both jobs come down to statistics done carefully, and both come up in interviews for speech and safety roles.

A text-to-speech (TTS) system turns written text into audio. The standard way to judge one is a listening test. People hear its output and rate each clip from 1 (bad) to 5 (excellent). The average rating is the system's mean opinion score, or MOS. Every TTS paper reports one, and most of them read it too confidently. Here is a small case that shows why.

Those effect sizes are realistic. Across 46 listening tests, the highest-scoring and the lowest-scoring rater in a test differed on average by more than 2 points on the 5-point scale (Rosenberg and Ramabhadran 2017). The test settings matter as much as the arithmetic. One study reran MOS tests on three well-known TTS systems under different but reasonable settings and got at least three different rankings (Chiang, Huang and Lee 2023).

The second job exists because the first one succeeds. The zero-shot models of §37.4 copy a voice they never heard in training from a 3-second recording. The better they score in listening tests, the better they impersonate. In January 2024, voters in New Hampshire received robocalls in a cloned voice of the US president telling them to skip the primary. Within weeks the Federal Communications Commission ruled that AI-generated voices count as "artificial" voices under the Telephone Consumer Protection Act. Such calls now need the prior consent of the person called (FCC 2024).

Here is the plan. We first set up the listening tests and derive the variance of a MOS when both listeners and sentences vary. We build intervals that contain the truth as often as they claim, show why rating two systems on the same listeners and sentences is so powerful, and turn "human parity" into a test that can fail. Then we take apart the automatic metrics that stand in for listeners and see how each one is gamed. The second half lays out the risks of voice cloning and the layered defenses against it. It ends with the exact false-alarm and detection rates of a watermark detector, the numbers that decide whether a watermark is usable on a platform that sees a billion clips a day.

What a listening test measures

A listening test is a survey. We play audio to people, ask them one question, and average the answers. Whatever makes surveys hard makes listening tests hard too. It matters who answers, what exactly they are asked, and what they compare against. Each clip played to a listener is called a stimulus. There are six common designs, and the question you want answered decides which one to run.

The WaveNet vocoder (van den Oord et al. 2016) was rated with exactly these five labels. Tacotron 2 (Shen et al. 2018) used the same scale with half-point steps, on 100 sentences with at least 8 raters each. An ACR test is cheap and gives an absolute level. But each stimulus is judged alone. Two good systems both land near the top, and the small difference between them drowns in the spread of opinions. That is why side-by-side tests exist.

CMOS answers a narrower question, whether B is better than A. It answers that question far more precisely, because each listener compares the two systems on the same sentence, and the section on pairing below proves why that helps so much. Tacotron 2 ran a side-by-side test against the original recordings and reported $-0.270 \pm 0.155$. That is a small but statistically significant preference for the recordings, which the raters' comments put down mostly to occasional mispronunciations. Seed-TTS (Anastassiou et al. 2024) used a scale from $-2$ to $+2$ instead, and asked which clip was closer to a reference recording of the speaker in similarity and expressiveness. So check the scale and the question before you compare CMOS values from two papers.

The hidden reference checks that a listener can hear the differences at all. The anchors pin the bottom of the scale, so a score of 60 means roughly the same thing in two different tests. MUSHRA is built for intermediate quality, where the damage is clearly audible. That is the regime of low-bitrate neural codecs, the networks of §37.3 that compress audio into a few integers per frame. EnCodec (Défossez et al. 2022) and Moshi's Mimi codec (Défossez et al. 2024) were both evaluated with MUSHRA. Near transparency, where the copy is hard to tell from the original and most systems score 80 to 100, BS.1534 itself warns that the results may be invalid.

The last design is the arena. Visitors type a prompt of their choosing, hear two anonymous systems speak it, and vote for one. A ranking model then gives each system a strength $s$ and says that A beats B with probability $e^{s_A}/(e^{s_A} + e^{s_B})$. This is the Bradley-Terry model that ranks chat models in arenas (§22.7). An arena ranks many systems with no fixed panel, at the price of raters and prompts that nobody controls.

Who listens, and how they listen

A listening test is only as good as its listeners. WaveNet's authors threw out the roughly 40 percent of their crowdsourced ratings that were made without headphones. MUSHRA's exclusion rule above is the formal version of the same idea. The usual guards are native speakers of the test language, a quiet room, and a short training round. Another guard is the catch trial, a stimulus with a known answer that reveals careless raters. A recording played twice should get the same score, and a clip with an obvious defect should score low.

These details then have to be reported. Chiang, Huang and Lee read the 80 TTS papers presented at INTERSPEECH 2022 that ran MOS tests. None of them gave enough detail to repeat the test. Most left out how the listeners were recruited and filtered, what instructions and payment they got, or where they came from and what languages they spoke. The reruns above show why that matters, since the ranking of three systems changed with these settings. NaturalSpeech (Tan et al. 2022) proposed a minimum of 50 test utterances, each compared side by side by more than 20 native speakers. You cannot judge a MOS without these facts, any more than an accuracy without its test set. An interviewer who asks "how would you run a MOS test?" wants this list before any statistics.

A MOS with the usual interval

Most papers report a MOS with an interval computed as if every rating were an independent draw. Recall from §0.11 that a confidence interval at level $1-\alpha$ is a recipe for an interval that contains the true value in a fraction $1-\alpha$ of repeated experiments. The usual recipe takes $n$ independent ratings with sample mean $\overline{u}$ and sample standard deviation $s$, computed with the $n-1$ denominator. The $t$ interval is

where $t_{n-1,\,q}$ is the $q$ quantile of Student's $t$ distribution with $n - 1$ degrees of freedom. In plain words, the interval is the mean plus or minus a multiple of the standard error $s/\sqrt n$, the estimated standard deviation of the mean itself. The multiple is a little larger than the normal 1.96, because $s$ is itself estimated. With 8 ratings it is $t_{7,\,0.975} = 2.36$, and once $n$ is in the hundreds the two agree.

Tacotron 2's MOS of $4.526 \pm 0.066$ is this interval. It rests on at least 800 ratings, at least 8 for each of 100 sentences. With 800, a half-width of 0.066 means a rating standard deviation of about $0.066\sqrt{800}/1.96 \approx 0.95$, a typical spread. Exercise 37.6a implements the interval, and the rest of the section shows what it leaves out.

import numpy as np
from scipy import stats

def mos_ci(ratings, level=0.95):
    raise NotImplementedError

Listeners and sentences both move a MOS

The naive interval assumes that every rating is a fresh, independent draw. Real ratings share listeners and sentences, and that sharing is what this part measures. Start with a test small enough to work by hand.

That movement is exactly what a confidence interval must capture. We want a statement about listeners like these and sentences like these in general, not about these three people and these four sentences. So the listeners and the sentences are both random samples, and the model of the ratings must say so.

In the hand table, listener 3 has a large positive $a_3$, sentence 3 has a negative $b_3$, and the residuals are what is left over in each cell. The model is additive. A harsh listener is equally harsh on every sentence, up to the residual. That is an assumption, and the section on pairing shows where it matters.

The theorem below says how much each part moves the MOS. It also gives a way to estimate each part from the table, using mean squares, the spreads of the listener means, of the sentence means, and of what neither explains. The method is the two-way analysis of variance (ANOVA).

In plain words, each source of variation is averaged over only as many independent draws as it has. Listener effects are averaged over $I$ listeners, however many sentences each one rates. Sentence effects are averaged over $J$ sentences. Only the residual gets the full $IJ$. So collecting more ratings from the same 20 people cannot shrink the first term. The second half of the theorem turns three spreads you can compute from the table into an estimate of that variance.

Unbiased is not the same as sensible. In a small test, $\mathrm{MS}_{\text{lis}}$ can come out below $\mathrm{MS}_{\text{res}}$ by chance, and then the estimated listener variance is negative. The standard fix, and the one we use, is to estimate each component on its own and clamp it at zero,

then plug the clamped values into the variance formula. The clamped estimator is slightly biased upward, so it errs on the side of wider intervals.

The cell checks the arithmetic.

How wrong the naive interval is

The naive standard error divides the sample variance of all $IJ$ ratings by $IJ$. That sample variance estimates the total variance $\sigma^2_{\text{lis}} + \sigma^2_{\text{sen}} + \sigma^2_{\text{res}}$, up to a small correction given in the box. Dividing the true variance of the MOS by the naive one gives the factor by which the naive variance is too small.

This ratio is a design effect, the factor by which correlated data inflate a variance. §0.11 derived it for one kind of cluster, such as several sampled answers per question. Here there are two kinds of cluster at once, listeners and sentences. It is the speech version of the clustered standard errors of §14.7. §36.5 found the same thing for word error rate. There, errors cluster by speaker, and an interval that treated utterances as independent was about four times too narrow.

The practical rule follows from the last paragraph. To narrow a MOS interval, recruit more listeners, not more ratings per listener. An interviewer who asks how to make a listening test more precise wants to hear exactly that.

Intervals that cover

An interval covers when it contains the true value. A 95 percent interval should cover in 95 of 100 repeated tests, and its coverage is the fraction of repeats in which it actually does. The next cell checks three intervals by simulating the same test many times.

The first interval is the naive one and the second uses the ANOVA estimate above. The third is a bootstrap. The bootstrap of §0.11 estimates the uncertainty of a statistic by resampling the test items with replacement and recomputing the statistic many times. It must resample the unit that was drawn independently. Here no single rating is independent, so the fix is to resample both random factors, listeners and sentences, at once.

No bootstrap is exact for crossed random effects (McCullagh 2000). This one errs on the safe side. For a mean it counts the residual term $\sigma^2_{\text{res}}/(IJ)$ about three times instead of once, so its variance is a little too large. When listeners and sentences have real effects, that term is the smallest of the three and the excess hardly shows (Owen 2007). Only when they have no effects at all does it matter, and then the bootstrap interval is about $\sqrt3 \approx 1.7$ times too wide. That holds even when the design is unbalanced and the variances differ across listeners, and no random-effects model has to be fit. Resampling single ratings instead, the ordinary bootstrap applied blindly, reproduces the naive interval.

The naive 95 percent interval covers the truth about a third of the time. That is what an interval 4.2 times too narrow should do. Its half-width is $1.96/4.2 = 0.47$ true standard errors, and a normal variable lands that close to its mean with probability $2\Phi(0.47) - 1 \approx 0.36$, where $\Phi$ is the standard normal distribution function. The ANOVA interval and the two-way bootstrap both cover about 92 percent of the time. They fall a little short of 95 because, with 20 listeners, the listener variance is itself estimated from only 19 degrees of freedom.

The widget runs the same experiment live, forty times over, with the residual standard deviation fixed at 0.7. It opens on the example's test, 20 listeners and 50 sentences with both effects present. Each line is one test's interval, pink for the naive one, indigo for the ANOVA one, and green for the two-way bootstrap. A line drawn dark misses the true MOS.

import numpy as np

def crossed_mos_se(U):
    U = np.asarray(U, float)
    return U.std(ddof=1) / np.sqrt(U.size)       # the naive standard error, which is wrong

def two_way_bootstrap_ci(U, B, rng, level=0.95):
    raise NotImplementedError

Comparing two systems, and why pairing wins

The real-scale example above compared two systems rated by different panels on different sentences. Suppose instead that the same 20 listeners rate the same 50 sentences from both systems. That happens in a CMOS test, or in a MOS test that mixes the two systems' clips in one session. Then the harshness of listener 7 and the difficulty of sentence 31 hit both systems alike, and they cancel in the difference. This design is called pairing.

Proof. Cell by cell, $u^{B}_{ij} - u^{A}_{ij} = (\mu_B - \mu_A) + (\varepsilon^{B}_{ij} - \varepsilon^{A}_{ij})$, because $a_i$ and $b_j$ cancel. Averaging over the $IJ$ cells gives a constant plus the mean of $IJ$ independent differences, each of variance $2\sigma^2_{\text{res}}$. ∎

In plain words, a harsh person is harsh on both systems, and the difference does not care. With the example's numbers, the gap now has a standard error of $\sqrt{0.98/1000} = 0.031$ instead of 0.172, more than five times smaller. A gap of 0.09 is then almost three standard errors.

The assumption that does the work is additivity. Real listeners may differ in which system they prefer, not only in how harsh they are. That is an interaction. Listener $i$'s preference for B over A then varies across listeners, with standard deviation $\sigma_{\text{int}}$, and the variance of the difference keeps a term $\sigma^2_{\text{int}}/I$. The difference matrix $u^{B}_{ij} - u^{A}_{ij}$ is itself a crossed design. So the ANOVA estimator applied to it measures whatever listener and sentence structure remains, and the interval stays honest.

The cell compares three designs at a true gap of 0.09. It reports the power of each, the probability that the test declares a significant difference (here $z > 1.96$) when the difference is real.

Separate panels detect a real gap of 0.09 less than one time in ten. One panel with additive effects detects it about three times in four. A modest interaction costs much of that gain. When listeners' preference for B varies with a standard deviation of 0.25 points, the standard error nearly doubles and the power falls to about a third. The paired design is still far ahead.

So the design rule is to pair whenever you compare. Use the same listeners and the same sentences for every system, and analyze the per-cell differences. When several systems are compared, each pairwise test is one more chance of a false positive, and the corrections of §0.11 apply. Holm's method keeps the chance of even one false claim below $\alpha$. The Benjamini-Hochberg procedure instead keeps the expected share of false claims below a target, which finds more real differences when there are many.

Human parity is a claim of equivalence

"Human-level quality" is the headline every TTS paper wants. NaturalSpeech wrote down a definition. A TTS system reaches human-level quality on a test set if there is no statistically significant difference between the quality scores of its speech and of the matching recordings. On LJSpeech it reported a MOS of $4.56 \pm 0.13$ against $4.58 \pm 0.13$ for the recordings ($p = 0.71$), and a CMOS of $-0.01$ against them ($p = 0.69$). On that basis it claimed human-level quality on that test set. Seed-TTS reported a CMOS against human speech of $-0.07$ in English and $-0.08$ in Mandarin, on its $-2$ to $+2$ scale of similarity and expressiveness. It added that, empirically, an absolute CMOS below 0.1 is considered an insignificant difference. VALL-E 2 (Chen et al. 2024) claims human parity on two read-speech datasets, LibriSpeech and VCTK.

The trouble is the logic of "no significant difference". A test that fails to reject equality could mean the systems are equal. It could also mean the test had too little data to see the difference. A test with five listeners will "show parity" between almost anything.

The test that does this is the equivalence test. It turns the burden around. Instead of asking whether the data rule out a zero difference, it asks whether they rule out every difference as large as the margin.

In plain words, to claim two systems are equal, show that the difference is confidently above $-\delta$ and confidently below $+\delta$. Failing to show that they differ is not enough.

So look for three things in a parity claim. There should be a margin chosen in advance, an interval that accounts for listeners and sentences, and a statement of the population and protocol the claim applies to. NaturalSpeech's claim, read carefully, is about LJSpeech's single speaker, its test sentences, and 20 judges per utterance, and the authors say so.

The cell simulates a CMOS test of NaturalSpeech's size, 20 listeners on 50 sentences, with a true gap of $-0.05$. It passes the "no significant difference" bar and fails a reasonable equivalence margin.

An interviewer who hears "our CMOS was not significantly different from zero" will ask for the margin, the interval, and how many listeners and sentences were used. Exercise 37.6c builds the test.

import numpy as np
from scipy import stats

def paired_diff_ci(a, b, level=0.95):
    raise NotImplementedError

def tost(diffs, delta, alpha=0.05):
    # placeholder: "not significantly different" taken as parity

Automatic metrics, and how each is gamed

Listening tests are slow and expensive. So every TTS paper also reports automatic metrics, scores computed by programs. And every speech model trained with rewards, the post-training of §37.4, optimizes them. Each metric models one aspect of what listeners hear, and each fails in a specific, predictable way.

Intelligibility through a recognizer

To check that a TTS system said the words it was given, transcribe its output with a speech recognizer and compare the transcript with the input text. The word error rate (WER) counts the fewest word substitutions, deletions, and insertions that turn the transcript into the reference, and divides by the number of reference words (§36.1). Say "the cat sat on the mat" is transcribed as "the cat sat on a mat". One substitution in six words gives a WER of 1/6, or 16.7 percent.

Two systems' WERs are comparable only if both were scored with the same text normalizer, the rules that lowercase text, strip punctuation, and spell out numbers before scoring (§36.5). For Mandarin, which is written without spaces between words, the character error rate replaces WER. Three more details decide what the number means.

Which recognizer. VALL-E (Wang et al. 2023) used HuBERT-Large, a self-supervised speech encoder (§35.5). It was fine-tuned as a recognizer on 960 hours of LibriSpeech, a corpus of read audiobooks, with the CTC loss (§36.2) and no language model. The Seed-TTS evaluation set, which many 2025 and 2026 papers report on, uses Whisper-large-v3 (§36.4) for English and the Paraformer-zh recognizer for Mandarin. A recognizer with a strong language model guesses the intended word from context and so fixes some of the TTS system's errors for it. WERs from different recognizers are therefore different metrics.

The floor. Real recordings do not score zero. Ground-truth speech had a WER of 2.2 percent in VALL-E's LibriSpeech setup and 2.143 percent on Seed-TTS's English set. A system at the floor is as intelligible to that recognizer as the recordings, and differences smaller than the floor's own noise mean nothing.

The direction of the error. WER rewards speech that is easy for a recognizer, which is not the same as speech that sounds natural. Seed-TTS reports that a lower WER typically meant more standardized speech. For prompts with a strong accent or a very expressive delivery, a lower WER usually meant a less accented output with less variation, which sounded less natural and less like the speaker. When the authors used WER as a reinforcement learning reward, the model learned to speak more slowly and more clearly at the expense of naturalness. That is a textbook case of reward hacking (§16.5), where a model raises a proxy score in a way that departs from what the score was meant to capture. Voicebox (Le et al. 2023) makes the mirror-image point. More varied or more expressive samples can be harder to recognize, and they raise WER without being worse.

Speaker similarity

A cloning system should sound like the speaker of its prompt. The automatic measure uses a speaker verification model, a network trained to map each utterance to an embedding, a vector that is close for the same speaker and far for different speakers. Embed two utterances and take the cosine similarity of the embeddings, the cosine scoring of §35.6. A cosine of 1 means the two embeddings point the same way, and 0 means they are unrelated. VALL-E used a verifier built on WavLM, a self-supervised speech encoder of the same family as HuBERT (§35.5). The Seed-TTS evaluation uses WavLM-large fine-tuned for speaker verification. The subtle part is which recording to compare against.

VALL-E computed speaker similarity against the decompressed prompt, which is SIM-r in this notation. Voicebox introduced the two names and argued for SIM-o, because SIM-r is not comparable across models that use different vocoders. Every codec leaves its own small, consistent traces in the audio, a fingerprint. A generated utterance and a resynthesized prompt share that fingerprint, and a verifier partly hears it as "the same speaker". So SIM-r rises with the codec, not only with the cloning. In Voicebox's own table, its English model had SIM-o 0.662 and SIM-r 0.681 in the cross-sentence setup, and VALL-E's reported 0.580 is a SIM-r.

The ceiling is a second trap. Two real utterances of the same LibriSpeech speaker scored 0.754 in VALL-E's setup, and Seed-TTS's human references scored 0.730 on its English set. Seed-TTS's generated speech scored 0.762, above the human ceiling. The authors explain why. The real target utterance and the prompt can differ in speaking style and background, while the generator copies the prompt's characteristics faithfully. A generator that reproduces the prompt's room and microphone can beat a real recording of the same person made on another day. Speaker similarity measures similarity to the prompt, recording channel included, and that is a different thing from identity.

The cell builds toy embeddings as a sum of parts, namely who is speaking, the room, the codec's fingerprint, and noise. Then it computes the three numbers.

So report SIM-o with the verifier named, report the ceiling next to it, and never compare SIM-r across models with different decoders. An interviewer who sees a similarity above the human ceiling should ask what the model copied besides the voice. Exercise 37.6d builds the report you would put in a paper.

import numpy as np

def similarity_report(gen, prompt_orig, prompt_resyn, real_pairs, rng, B=1000):
    raise NotImplementedError

Predicted MOS

If listeners are expensive, train a model to imitate them. A MOS predictor maps an utterance to the MOS that listeners would give it. It is trained on the ratings of past listening tests. The VoiceMOS Challenge 2022 (Huang et al. 2022) made this a benchmark, with listening-test data from 187 TTS and voice conversion systems spanning a decade. It scored predictors at two levels. The utterance level is the correlation between the predicted and the true MOS of each utterance. The system level is the correlation between the predicted and the true mean of each system.

UTMOS (Saeki et al. 2022), an ensemble of predictors built on fine-tuned self-supervised speech models, ranked first on several of the challenge's metrics. It was scored by Spearman correlation, the correlation of the two rankings. On the main track its utterance-level correlation was 0.897 and its system-level one 0.936. On the out-of-domain track, made of newer systems, the two were 0.893 and 0.988.

The gap between the two levels is the lesson. Errors on single utterances average out over a system's many utterances, so predictors rank systems far better than they score clips. The organizers also found unseen speakers, listeners, and systems hard. A predictor trained on past systems rates new kinds of artifacts badly, and a new model is exactly where you would use it. It is worse still when you train against it, since the generator will find the artifacts the predictor cannot hear.

Two other predictors are common. DNSMOS (Reddy et al. 2021) needs no reference recording and was built to judge noise suppression. NISQA (Mittag et al. 2021) predicts overall quality and four dimensions, noisiness, coloration, discontinuity, and loudness, for the damage typical of telephone calls. Each is trustworthy only near the data it was trained on.

Distribution metrics and codec metrics

For music and general audio there is often no reference text or speaker. The question becomes whether a set of generated clips looks like a set of real ones. §34.6 derived the Fréchet distance between two Gaussians, which measures how far apart their means and spreads are. Applied to the features of an image classifier it is the FID score for images. The audio version uses the same formula on the embeddings of an audio classifier.

FAD was proposed for music enhancement. Its correlation with human judgments was 0.52, above signal-based metrics such as the signal-to-distortion ratio at 0.39 (Kilgour et al. 2019). It inherits every weakness of FID. It depends on the embedding model, on the reference set, and on the number of clips, since the estimate is biased upward at small sample sizes. Sample-size bias, a poor choice of embedding, and biased or low-quality reference sets all hurt FAD's agreement with perceived quality (Gui et al. 2024). Extrapolating the score to an infinite sample size reduces the first problem.

MusicGen (Copet et al. 2023) reports three numbers that answer three different questions. FAD with VGGish asks whether the audio is realistic. A KL divergence between the label distributions that an audio classifier assigns to generated and to reference clips asks whether the content matches. A CLAP score, the similarity of audio and text in a shared embedding space (§35.6), asks whether the audio fits the prompt.

Codecs and vocoders are judged against the original signal, with full-reference metrics that compare the output with the input. ViSQOL and PESQ model perceived quality, STOI models intelligibility, and the mel-spectrogram distance measures spectral error directly. They can disagree badly with listeners, as §37.3 showed. Moshi's authors trained a Mimi codec with adversarial losses only, where a second network learns to tell real audio from decoded audio and the codec learns to fool it. Its ViSQOL was a very low 1.84, which did not reflect its high perceived quality. In their MUSHRA test it scored 81.0, against 58.8 for the same codec trained with EnCodec's mix of losses. A decoder trained adversarially does not reproduce the waveform, only something that sounds like it, and full-reference metrics punish exactly that.

The risk this chapter creates

Everything in this chapter so far has been about making speech that people cannot tell from a real person's. That is also the definition of a convincing fake. VALL-E clones a voice from 3 seconds of audio, and the hybrid systems of §37.4 do it better. The GPT-4o system card (OpenAI 2024) names the harms that voice generation from a short clip could enable, more fraud through impersonation and the spread of false information. The chairwoman's statement attached to the FCC's ruling cites the New Hampshire calls. It also cites scams in which parents heard a cloned child's voice asking for money. Cloning also enables harassment with a person's own voice, and the use of a voice without its owner's consent, for example an actor's.

There is also an accidental version. OpenAI reports that during testing, GPT-4o in rare instances produced output that imitated the user's voice. A model that continues an audio stream (§37.5) has the user's voice in its context, and continuing in that voice is a plausible completion. A voice agent that acts on spoken instructions also inherits prompt injection, instructions hidden in the input that hijack the model (§19.3), now delivered through audio. Limiting what such an agent may do is the same sandboxing problem as for code agents.

Defenses, layer by layer

No single defense works. §19.7 built the case for defense in depth, stacking input and output classifiers, monitoring, and policy so that each layer catches some of what the others miss. Voice adds layers of its own. They differ in who must cooperate, which may be the model provider, the platform that distributes the audio, or nobody at all. That decides what each layer can stop.

Preset voices, consent, and output voice classifiers

The strongest control is not to offer arbitrary cloning at all. GPT-4o's voice mode allows only preset voices recorded with voice actors. The model was post-trained with those voices as the ideal completions. A separate output classifier runs while the audio streams out and blocks it if the speaker does not match the chosen preset voice. OpenAI reports that the system caught 100 percent of meaningful deviations from the system voice in internal evaluations. Measured over conversations, the classifier's precision (the share of its blocks that were real deviations) was 0.96 in English, and its recall (the share of real deviations it blocked) was 1.0. In other languages the two were 0.95 and 1.0. The same card describes post-training the model to refuse to identify a person from their voice.

Products that do offer custom voices rely on consent. The usual practice has five parts. The speaker records a consent statement in their own voice. Speaker verification checks that the consenting voice is the voice being cloned. The cloned voice stays private to that account. Use is rate-limited and monitored. The voice is deleted on request. Several of these steps are speaker verification problems of the kind §35.6 scored, now with the attacker on the other side.

Passive detection, or anti-spoofing

Passive detection tries to tell synthetic from real speech with no help from the generator. It trains a classifier, called a countermeasure, on examples of both. The ASVspoof challenges define the field, where ASV stands for automatic speaker verification. ASVspoof 2019 (Wang et al. 2020) covered two threats. Logical access feeds synthetic or converted speech straight into a speaker verifier, and physical access plays recordings back to its microphone. The challenge found spoofed speech that even human listeners could not tell from real speech.

Countermeasures are scored by the equal error rate (EER, §35.6). It is the error rate at the threshold where the rate of accepted spoofs equals the rate of rejected real speech. They are also scored by detection costs that combine the countermeasure with the verifier it protects. AASIST (Jung et al. 2021), a graph attention network over spectral and temporal features, is a widely used strong countermeasure.

The weakness is generalization. A countermeasure learns the artifacts of the generators in its training data, and new generators have new artifacts or none. Published detectors were re-evaluated on 37.9 hours of real-world audio of celebrities and politicians, 17.2 hours of it deepfakes (Müller et al. 2022). Their equal error rates rose by up to one thousand percent, which suggests the field had tailored its detectors to the benchmark. ASVspoof 5 (Wang et al. 2024) moved to crowdsourced speech from many more speakers in diverse acoustic conditions. Its attacks were tuned against stand-in detectors, and for the first time they included adversarial attacks, small changes to the audio designed to fool a detector. Those attacks significantly compromised the baseline systems. Passive detection is useful as one layer, and it is a contest the defender never gets to end.

Active watermarking

The alternative is to make the generator cooperate. A watermark is a signal added at generation time that people cannot hear and that a detector holding the right key can find later.

A watermark works only if whoever runs the generator applies it. So it covers the models of providers that choose to mark, and the main open question is how well the mark survives edits. Three systems mark the state of the art.

• AudioSeal (San Roman et al. 2024) trains an embedder and a detector together. A localization loss teaches the detector to say which samples are marked, at a resolution of one sample (1/16,000 s at 16 kHz). So a short cloned phrase spliced into a real recording can be found. A perceptual loss, inspired by the way loud sounds mask quiet ones in human hearing, hides the mark where the signal is loud. The detector runs in a single pass, up to two orders of magnitude faster than WavMark's. The released models carry an optional 16-bit message that can identify, say, a model version.
• WavMark (Chen et al. 2023) encodes up to 32 bits in each second of audio. On hosts, the audio that carries the mark, of 10 to 20 seconds, it reported an average bit error rate of 0.48 percent across ten common attacks. That is the share of payload bits the detector decodes wrongly.
• SynthID (Google DeepMind) watermarks audio from Google's Lyria music model, among other media. Google describes the audio mark as inaudible and robust to common changes such as added noise, MP3 compression, and speed changes.

The detector is what makes a watermark useful, and its error rates can be computed exactly. We compute them after the last two layers of defense.

Provenance metadata

A watermark says that a cooperating generator made a clip. Provenance says what happened to a file. C2PA content credentials, the standard of the Coalition for Content Provenance and Authenticity covered in §34.7, attach a manifest to a file. The manifest is a list of claims, such as which tool created the content and which edits were made. It is bound to the content by a cryptographic hash and signed with a certificate. A valid manifest proves the claimed history of a file that carries it. It says nothing about a file that does not, and re-encoding, screen recording, or any platform that drops metadata strips it. Provenance is strongest for the opposite claim, that a recording is authentic, since a recording app can sign the audio at the moment of capture.

Law

Rules now reach both the callers and the providers. In the United States, the FCC adopted its declaratory ruling FCC-24-17 on February 2, 2024, and released it on February 8. It confirmed that the Telephone Consumer Protection Act's limits on calls using an "artificial or prerecorded voice" cover current AI technologies that generate human voices. Such calls therefore need the called party's prior express consent, unless they serve an emergency or an exemption applies.

In the European Union, Article 50(2) of the AI Act (Regulation 2024/1689) covers providers of AI systems that generate synthetic audio, image, video, or text. They must ensure that the outputs are marked in a machine-readable format and detectable as artificially generated or manipulated. The technical solutions must be effective, interoperable, robust, and reliable as far as technically feasible. Article 50(4) separately requires deployers of systems that generate deep fakes to disclose that the content is artificial. Article 113 applies Article 50 from 2 August 2026.

Detection statistics of a watermark

Here is the question a platform must answer. It sees $10^9$ audio clips a day and wants to flag the ones made by its own generator. How many bits should the watermark carry, and how many must match? AudioSeal's paper states the stakes. Take a platform that processes a billion samples a day, and a detector with an FPR of $10^{-3}$ and a TPR of 0.5. It sends a million clips a day to manual review, and it still catches only half of the marked ones. The false-positive rate is the number that matters, and it has to be tiny.

Start with the intuition. On unmarked audio, the bits a decoder reads out have nothing to do with the expected payload, so each one matches by luck, like a fair coin. On marked audio, each bit matches unless the edit flipped it. The detector fires when enough bits match.

Proof. The number of agreeing bits is a sum of $b_{\text{wm}}$ independent Bernoulli variables, each with success probability $\tfrac12$ on unmarked audio and $1 - e$ on marked audio, so it is binomial with those parameters. The detector fires exactly when the count is at least $h_{\text{thr}}$, so each rate is the binomial upper tail, the sum of the probabilities of $k = h_{\text{thr}}, \dots, b_{\text{wm}}$ matches. With $p = \tfrac12$ every one of the $2^{b_{\text{wm}}}$ bit patterns is equally likely, which gives the first formula. ∎

In plain words, on unmarked audio the detector flips $b_{\text{wm}}$ fair coins and fires on at least $h_{\text{thr}}$ heads. On marked audio it flips coins biased toward heads. Raising the threshold makes false alarms rarer and real detections rarer too. The only ways to improve both are more bits or a lower bit error rate.

The cell computes the exact rates for several payloads and thresholds.

The table makes the trade-off concrete. At a fixed false-positive budget, each doubling of the payload raises the bit error rate the detector can tolerate. A 128-bit payload meets $10^{-9}$ with 30 errors allowed (98 matches), and it still detects essentially every clip at a 10 percent bit error rate. An interviewer who asks how to make a watermark robust wants this answer. Spend extra bits on tolerating errors, not on a longer message.

In the widget, the pink bars count the matching bits on unmarked audio, assumed to be fair coins. The green bars count them on marked audio after an edit that flips a fixed share of the bits, an illustrative rate rather than a measured one. Every bar at or to the right of the threshold line is flagged, and the platform checks $10^9$ clips a day. Each time the payload changes, the threshold jumps to the fewest matches that allow at most one false flag a day. You can still drag it by hand.

The theorem's assumption is its weak point, and it fails in a measurable way. On unmarked audio, a real detector's decoded bits need not be fair coins. AudioSeal's authors read WavMark's soft bits, the decoder's raw outputs between 0 and 1 before rounding, from 10,000 genuine samples. They found them centered around 0.38 instead of 0.5. The bits leaned toward 0, so the theoretical FPR would greatly underestimate the actual one. The practical rule follows. Compute the threshold from the binomial formula as a starting point. Then calibrate it on a large set of real, unmarked audio from the platform's own traffic. Draw the payload at random. Whatever the decoder's lean, each decoded bit then matches a random payload bit with probability exactly one half, so the binomial rate holds on average over payloads. A fixed payload that is mostly zeros would instead match a decoder that leans toward 0 far too often.

import numpy as np
from math import comb

def fpr(b, h):
    # placeholder: a normal approximation with a strict inequality
    from scipy import stats
    return float(stats.norm.sf((h - b / 2) / np.sqrt(b / 4)))

A spread-spectrum watermark from scratch

Where do the decoded bits and scores come from? One of the oldest robust methods, and still the clearest, is spread spectrum (Cox et al. 1997). The idea is to add a faint, known noise pattern to the whole signal and later look for it by correlation. The name comes from the mark's energy being spread thinly over every frequency. Derive a pseudorandom sequence $\mathbf{w} \in \{-1, +1\}^N$ from the secret key, so that only the key holder can regenerate it, and add a scaled copy to the $N$ samples of the audio $\mathbf{x}$,

Here $g_{\text{wm}}$ is the gain of the mark and $P_x = \frac1N\sum_n(x_n - \overline{x})^2$ is the power of the signal's fluctuations. Since $w_n^2 = 1$, the mark has power exactly $g_{\text{wm}}^2$. So the signal-to-noise ratio of the mark, $10\log_{10}(P_x/g_{\text{wm}}^2)$ in decibels (§35.1), equals SNR. A mark at 20 dB has one hundredth of the signal's power.

To detect, regenerate $\mathbf{w}$ from the key. Take the received audio $\mathbf{y}'$, which may have been edited, and remove its mean, $\mathbf{c} = \mathbf{y}' - \overline{y'}\mathbf{1}$. Then compute the normalized correlation

Proof. Given $\mathbf{y}'$, the vector $\mathbf{c}$ is fixed and $z = \sum_n (c_n/\lVert\mathbf{c}\rVert)\,w_n$ is a weighted sum of independent signs with weights whose squares sum to 1. Each term has mean 0 and variance $c_n^2/\lVert\mathbf{c}\rVert^2$, so the sum has mean 0 and variance 1. The central limit theorem, in Lindeberg's form for weights none of which is large, gives the normal shape. ∎

So the threshold for a target false-positive rate is a normal quantile, $z_{\text{thr}} = \Phi^{-1}(1 - \mathrm{FPR})$. It is one-sided, because only large positive correlations indicate the mark, and it is 4.75 for an FPR of $10^{-6}$. On marked audio, $\mathbf{c}^\top\mathbf{w} \approx g_{\text{wm}} N$ plus the host's own random correlation with the key, since $\mathbf{w}^\top\mathbf{w} = N$. The norm is $\lVert\mathbf{c}\rVert \approx \sqrt{N(P_x + g_{\text{wm}}^2)}$. So

In plain words, the detector adds up a faint, known pattern over many samples. The evidence grows like $\sqrt N$, and the host audio acts as noise. The host's own correlation with the key, which has standard deviation 1 by the theorem, is the main source of error. That is why practical schemes subtract an estimate of the host before correlating, or shape the mark to fit the host.

import numpy as np
from scipy import stats

def embed(x, key, snr_db):
    w = key_sequence(key, len(x))
    return x + np.std(x) * 10 ** (-snr_db / 10) * w        # scales the amplitude by a power ratio

def detect(y, key):

What does a simple mark survive? The cell embeds a key at 20 dB in one second of a synthetic voiced sound, applies a few edits, and prints the score after each.

Additive noise costs little. Even noise as loud as the speech leaves the mark detectable ($z = 9.2$), because the correlation adds up evidence over 16,000 samples. Filtering is another matter. The mark is white, meaning its energy is spread evenly over all frequencies, so a low-pass filter removes much of it. A moving average over 2 samples already halves the score, and one over 8 samples destroys it. So does a shift of a single sample, because a white sequence is uncorrelated with a shifted copy of itself.

Real schemes answer each of these edits. They shape the mark to survive the edits that matter. They put its energy in the bands that codecs keep, and they make it louder where the signal masks it. They add synchronization patterns or repeated blocks to undo shifts and time stretching. And they train the detector against those edits, which is what learned watermarks do.

Learned watermarks

AudioSeal and WavMark replace the fixed sequence with two networks trained together. The embedder proposes a mark. An augmentation layer applies the edits the mark must survive. The detector is trained to find the mark, while a perceptual loss keeps it inaudible.

The block below is a toy version, on random smoothed noise standing in for audio. The embedder multiplies a fixed high-frequency carrier, a sequence that alternates $+1$ and $-1$, by an envelope it learns from the signal. The envelope is bounded so that the mark stays at least 20 dB below the signal. The detector measures energy in frequency bands it learns, and light noise added during training plays the attacker.

In 150 steps the pair reaches about 97 percent accuracy with the mark 20 dB down. Accuracy falls, to about 93 percent, once the noise is five times stronger than the noise used in training. That is the lesson of every learned watermark. It survives what it was trained against. A production system runs the same loop with real speech, a perceptual loss, and a long list of augmentations, such as codecs, resampling, filtering, and time stretching. Its detector outputs a score per sample. AudioSeal's released package exposes the result in a few lines.

Attacks on watermarks

Watermarks face three kinds of attack, described here at the level of research findings.

Removal edits the audio until the detector fails while the audio still sounds acceptable. The standard threats are re-encoding with a codec, resampling, filtering, and time stretching. Above all there is regeneration, passing the audio through a neural codec or a voice conversion model, which reproduces what it hears and not the inaudible mark. Robustness papers report the bit error rate or the TPR after each edit.

Forgery adds a mark to real audio to make it look generated. That matters if a mark is used to dismiss real evidence. It is why the key must stay secret, and why detection results need provenance of their own.

Open models are the gap that no technique closes. Nobody can force them to watermark, since anyone can run them with the embedder removed. A watermark can show that a cooperating generator made a clip. Its absence shows nothing.

Read more

• Rosenberg & Ramabhadran (2017). Bias and Statistical Significance in Evaluating Speech Synthesis with Mean Opinion Scores. Measures rater and text bias across 46 listening tests and argues for tests that account for both.
• Chiang, Huang & Lee (2023). Why We Should Report the Details in Subjective Evaluation of TTS More Rigorously. The INTERSPEECH 2022 reporting survey and the reruns that change the ranking of three systems.
• Owen (2007). The pigeonhole bootstrap. The two-way bootstrap for crossed, unbalanced random effects, and why resampling single cells misleads.
• Tan et al. (2022). NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality. Section 2 defines human-level quality as no significant difference, and Table 1 shows MOS missing what CMOS catches. Read it alongside the equivalence argument above.
• Le et al. (2023). Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale. Section 5 introduces SIM-o and SIM-r and discusses what WER and similarity do and do not show.
• Anastassiou et al. (2024). Seed-TTS: A Family of High-Quality Versatile Speech Generation Models. The evaluation protocol many papers now report on (with the seed-tts-eval configuration), human floors and ceilings, and the observation that WER rewards get hacked.
• San Roman et al. (2024). Proactive Detection of Voice Cloning with Localized Watermarking. AudioSeal, with the platform-scale argument for low false-positive rates and an appendix on why theoretical FPRs fail for real decoders.
• Wang et al. (2024). ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale. The current anti-spoofing benchmark, with adversarial attacks for the first time.
• OpenAI (2024). GPT-4o System Card. Section 3.3 describes unauthorized voice generation, preset voices, the streaming output classifier, and speaker identification refusals.

Watch next

▶

▶

▶