You have a fixed amount of compute to answer one hard question. You could run a 70-billion-parameter model once. You could run a 7-billion-parameter model and let it think ten times longer. You could run the small model 32 times and vote. Or you could let it write an answer, read its own answer, and revise it, eight times in a row. Which is best?

For most of deep learning's history this question did not come up, because inference compute was a rounding error next to training. Reasoning models changed that. A model that thinks for 20,000 tokens per question and serves heavy traffic can spend, over its deployment, a large multiple of the compute it took to train it, and how that compute is spent is now a design decision with its own scaling laws. §9.6 introduced test-time compute as a scaling axis. This section works out how to spend it, with the parallel and sequential strategies and their trade-offs, the compute-optimal allocation that depends on how hard the question is, budget forcing as the simplest sequential control, and the accounting that decides when a bigger model beats a longer think.

Two ways to spend compute at test time

Every strategy is a combination of two primitives.

• Parallel. Generate $n$ independent responses and aggregate them, by majority vote, by best-of-$n$ with a verifier, or by weighted voting (§18.2). The responses do not see each other.
• Sequential. Generate one response whose later parts depend on earlier ones, a longer chain of thought (§18.1), or a series of revisions in which each attempt is conditioned on the previous attempts.

Search methods mix the two. Beam search and tree search (§18.6) run many branches in parallel and extend the promising ones in sequence, and "parallel thinking" systems such as Gemini Deep Think explore several solutions at once and combine them before answering. The two primitives fail in different ways, which is what makes the choice interesting.

A model of the trade-off

Here is the smallest model that shows why the answer depends on the problem. A chain starts with an attempt that is right with probability $p_1$. Each revision reads the current answer and either fixes it, if it was wrong, with probability $p_{\text{fix}}$, or breaks it, if it was right, with probability $p_{\text{break}}$. Write $p_m$ for the probability that the chain's answer is right after $m$ attempts. Then

This is a linear recurrence with fixed point $p^\star = p_{\text{fix}}/(p_{\text{fix}} + p_{\text{break}})$, found by setting $p_{m+1} = p_m$. Subtracting the fixed point from both sides gives $p_{m+1} - p^\star = (1 - p_{\text{fix}} - p_{\text{break}})(p_m - p^\star)$, so the gap shrinks geometrically,

In plain words, revisions move the success probability toward $p^\star$ exponentially fast, and then stop helping. A model that fixes 30% of its mistakes and breaks 3% of its right answers saturates near $0.3/0.33 = 0.91$, however many revisions it gets. Now run $n$ such chains in parallel, each with $m$ attempts, for a budget of $C = nm$ generations, and pick one final answer. With a perfect verifier the result is right if any chain is, with probability $1 - (1 - p_m)^n$.

Simulating the chains directly confirms the saturation.

Two limits are easy to read off. If revisions do nothing ($p_{\text{fix}} = p_{\text{break}} = 0$), then $p_m = p_1$ and all of the budget should go to parallel samples. If first attempts are hopeless but revisions are powerful (say $p_1 = 0.01$, $p_{\text{fix}} = 0.5$, $p_{\text{break}} = 0$), then two sequential attempts give $p_2 = 0.505$, while two parallel samples give $1 - 0.99^2 \approx 0.02$, so all of the budget should go to sequence. Real problems sit in between, and where they sit depends on how hard they are for the model.

The closed form also says exactly when revising helps at all. Subtracting $p_1$ from the formula for $p_m$ gives $p_m - p_1 = (p^\star - p_1)\big(1 - (1 - p_{\text{fix}} - p_{\text{break}})^{m-1}\big)$, and the second factor is positive for $m \ge 2$ whenever $0 < p_{\text{fix}} + p_{\text{break}} < 2$. So revisions raise accuracy if and only if the first attempt is worse than the fixed point, $p_1 < p_{\text{fix}}/(p_{\text{fix}} + p_{\text{break}})$. A model that is right 80% of the time on its first try, fixes 20% of its wrong answers, and breaks 10% of its right ones has $p^\star = 0.2/0.3 = 0.67$, so asking it to check its work pulls its accuracy down from 0.80 toward 0.67.

That is roughly what happens with prompted self-correction. Huang et al. (2024) asked models to review and revise their own answers to reasoning questions with no external feedback, and accuracy usually did not improve and sometimes fell, because on several benchmarks the models changed right answers to wrong ones more often than the reverse. Revision has to be learned. Snell et al. fine-tuned their revision model on sequences of attempts, SCoRe (Kumar et al. 2024) trained self-correction with multi-turn RL and reported gains in self-correction of 15.6% on MATH and 9.1% on HumanEval over the base model, and reasoning models do much of their revising inside one chain, the "wait" and re-check of §18.5. The wider evidence on when self-correction works is the subject of self-refinement methods.

def p_after(p1, fix, brk, m):
    return p1

def best_allocation(p1, fix, brk, budget):
    return budget, 1, 1 - (1 - p1) ** budget

A perfect verifier flatters parallel sampling, because coverage then counts directly as accuracy. With a realistic verifier, selection becomes the bottleneck (§18.2), and sequential compute gains value, since it concentrates probability on one answer instead of relying on someone to pick it out. The widget uses a noisy verifier by default for that reason.

Compute-optimal test-time scaling

Snell et al. (2024) asked the question empirically, on the MATH benchmark with PaLM 2-S*. They studied two mechanisms. The first is search against a process verifier, best-of-$N$, step-level beam search, and lookahead search with a PRM (§18.6). The second is refining the model's own proposal distribution, by fine-tuning it to revise its previous answers in sequence and then choosing how to split a budget between sequential revisions and parallel chains. Their central idea was to make the strategy a function of the question. Write $\theta$ for the strategy's settings (which search method, the beam width, the split between sequential and parallel), $N$ for the generation budget, and $q$ for the question. The compute-optimal strategy is

the setting that maximizes the chance of a correct final answer given the budget, where $\text{Target}(\theta, N, q)$ is the distribution of final answers that strategy produces. Since $y^\star(q)$ is unknown at test time, they approximated the question-dependence through difficulty. They binned questions into five quantiles of the model's own pass@1 (estimated from 2,048 samples, an oracle notion of difficulty) and also tried a difficulty predicted from verifier scores, which needs no ground truth, and they chose the best strategy per bin.

The findings line up with the toy model.

• Search. Beam search against the PRM beat best-of-$N$ on harder questions and at lower budgets, and best-of-$N$ won on easier questions and at higher budgets, where beam search began to exploit the verifier. On the hardest bin, no method made much progress.
• Revisions. Easier questions benefited from purely sequential compute. Harder questions did best with some ratio of sequential to parallel, and the ideal ratio depended on difficulty and budget.
• Efficiency. Choosing the strategy per difficulty bin improved test-time compute efficiency by more than four times compared with a best-of-$N$ baseline.

When a bigger model beats a longer think

Snell et al. also compared test-time compute with model size at matched FLOPs, and the accounting is worth deriving because it comes up in every design discussion. Use the standard counts from §9.2, about $6N$ FLOPs per token to train a model of $N$ parameters and $2N$ per token to run it (from here on $N$ is a parameter count again, not a generation budget). Let $D_{\text{pre}}$ be the pretraining tokens and $D_{\text{inf}}$ the total tokens the model will generate over its life, and write $R = D_{\text{inf}}/D_{\text{pre}}$ for the ratio. A model $M$ times larger, trained on the same data and serving the same tokens, costs

The small model, generating $k$ times more tokens per query at test time, costs $6ND_{\text{pre}} + 2kND_{\text{inf}}$. Setting them equal and dividing by $2ND_{\text{pre}}$,

In plain words, the extra test-time multiplier the small model can afford is the size ratio plus a term that grows as inference becomes a smaller share of total compute. When the model will serve few tokens relative to its training set ($R \ll 1$), the big model's extra pretraining is the dominant cost, and the small model can think hundreds of times longer for the same total. When the model serves enormous traffic ($R \gg 1$), $k$ falls to about $M$, and the small model can only afford to think $M$ times longer.

Snell et al.'s comparison was against a model 14 times larger. On easy and medium questions, where the small model already had a nontrivial success rate, and at low inference loads ($R \ll 1$), test-time compute outperformed the larger model at matched FLOPs. On the hardest questions, or at high inference loads ($R \gg 1$), more pretraining was the better buy. Test-time compute substitutes for parameters on problems within the model's reach, and cannot conjure capability the model does not have, which is also what the "sharpening" debate of §18.5 would predict.

The same trade-off had been measured outside language. Jones (2021), studying AlphaZero agents on the board game Hex, found that train-time and test-time compute trade off linearly in log-compute, with each additional 10 times of train-time compute removing about 15 times of test-time compute, down to a floor of a minimal tree search. Wu et al. (2024) found that for math, smaller models with advanced inference algorithms gave Pareto-optimal cost and accuracy. Llemma-7B with their REBASE tree search outperformed Llemma-34B across inference strategies on MATH, and matched it with about half the FLOPs.

import numpy as np

def test_time_multiplier(M, R):
    return M

def small_model_wins(M, R, k_needed):
    return k_needed <= test_time_multiplier(M, R)

Budget forcing

The simplest sequential control is to decide how long the model thinks. Muennighoff et al. (2025) showed in s1 how far that alone can go. They built s1K, 1,000 questions selected from a pool of about 59,000 for difficulty, diversity, and quality, with reasoning traces from Gemini 2.0 Flash Thinking (a later version, s1.1, regenerated them with DeepSeek-R1), and fine-tuned Qwen2.5-32B-Instruct on them, which took 26 minutes on 16 H100 GPUs. At test time they controlled the thinking length with budget forcing.

• To cap thinking, once the model has used its budget, append the end-of-thinking delimiter (optionally followed by an answer prompt), so the model must answer with what it has.
• To extend thinking, when the model tries to end its thinking early, suppress the end-of-thinking delimiter and append "Wait" instead, which usually makes the model re-examine its work.

With budget forcing, s1-32B's accuracy on AIME 2024 went from 50% to 57%, and the model exceeded o1-preview on competition math (MATH and AIME 2024) by up to 27%. Accuracy rose with the thinking budget, a clean sequential scaling curve from a model trained on a thousand examples. Majority voting on the base Qwen2.5-32B-Instruct model, by contrast, could not catch up with s1-32B however many samples it used, though the authors also found that parallel methods complement sequential scaling rather than replace it. Forcing extension cannot go on forever. The authors report that the gains flatten after about six forced extensions, and that suppressing the end of thinking too often can push the model into repetitive loops instead of more reasoning. The context window is a hard limit as well.

Budget forcing is a decoding-time intervention, which makes it a good small coding problem. It needs no access to the model's weights, only control over the token stream.

def budget_force(next_token, prompt, end_think, wait, min_think, max_think):
    seq, think = list(prompt), []
    for _ in range(max_think):            # placeholder: ignores min_think and never forces the end
        tok = next_token(seq + think)
        think.append(tok)
        if tok == end_think:
            break
    return think

Parallel and sequential in reasoning models

Reasoning models do both kinds of scaling, and their papers report both. DeepSeek-R1-Zero's AIME 2024 accuracy went from 71.0% for a single sample to 86.7% with majority voting over 64 samples (§18.5), parallel scaling on top of a model that had already learned long sequential chains. OpenAI's o1 announcement reported 74% with one sample, 83% with consensus among 64, and 93% when re-ranking 1,000 samples with a learned scoring function, alongside a plot of accuracy rising smoothly with thinking time. Product features expose both dials, thinking budgets and effort levels for the sequential one, and "deep" or "pro" modes that spend more compute per query, some of which explore several lines of reasoning in parallel.

Three practical considerations decide the mix in deployment.

Latency. Parallel samples finish in about the time of one, if the hardware has room to batch them, while a longer chain takes proportionally longer, since decoding is sequential (§6.5). Interactive products cap sequential thinking for that reason.

Memory. A long chain's KV cache grows with every token, and at tens of thousands of tokens it dominates serving memory, which is why the techniques of §12.5 matter so much for reasoning models. Parallel samples that share a prompt can share its cache.

It is worth one calculation. The KV cache of §6.5 holds $2Ln_{kv}d_ks$ bytes per token, for $L$ layers, $n_{kv}$ key-value heads of width $d_k$, and $s$ bytes per number.

A single long chain costs as much cache as twenty parallel short answers, and sampling many long chains in parallel multiplies the two.

Selection. Parallel compute is only as good as the aggregation. Majority voting needs answers that can be compared, and verifiers can be exploited. Sequential compute relies on the model's own checking, which RL has trained, and needs no external selector.

In PyTorch

Budget forcing in a real serving stack is a logits processor, a function applied to the next-token logits at every decoding step. To extend thinking, it masks the end-of-thinking logit to $-\infty$ until the minimum length, and to cap thinking, it forces that token once the maximum is reached. Here it is on a tiny random language model.

(Here masking the logit plays the role of s1's "Wait". A production processor usually also inserts the actual "Wait" tokens, because a model that is merely prevented from stopping tends to keep writing whatever it was writing, while the word "Wait" invites it to re-check.)

Read more

• Snell et al. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. Compute-optimal strategies by difficulty, search against PRMs, revisions, and the comparison with a larger model.
• Muennighoff et al. (2025). s1: Simple test-time scaling. s1K, budget forcing, and sequential scaling from a thousand examples.
• Brown et al. (2024). Large Language Monkeys: Scaling Inference Compute with Repeated Sampling. Parallel scaling and its selection bottleneck.
• Wu et al. (2024). Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models. Small models with tree search on the cost-accuracy frontier.
• Huang et al. (2024). Large Language Models Cannot Self-Correct Reasoning Yet and Kumar et al. (2024), Training Language Models to Self-Correct via Reinforcement Learning. Why prompted revision fails and how RL fixes it.
• Jones (2021). Scaling Scaling Laws with Board Games. The train-time and test-time compute trade-off measured in Hex.
• OpenAI (2024). Learning to reason with LLMs. Accuracy rising with both RL compute and thinking time.

Watch next

▶

▶

▶