A long-running agent works on one task for hours or days. It might move a codebase to a new API, build a web application one feature at a time, or run a small simulated business. Two things stop it. Every step has some chance of going wrong, and over hundreds of steps those chances multiply. And the machinery under the run fails too, because processes crash, context windows fill up, and tools time out halfway through a payment. This section first measures how long a task an agent can finish, and then builds the engineering that keeps the work alive through its own failures.

Start with the arithmetic. Say an agent gets each step right 99% of the time, and you give it a task of 500 steps. If every step must go right and nothing is ever caught or fixed, the task succeeds with probability $0.99^{500} \approx 0.0066$, less than once in a hundred tries. Make the agent ten times more reliable, 99.9% per step, and the same task succeeds with probability $0.999^{500} \approx 0.61$. On a benchmark of short tasks, a gain from 99.0% to 99.9% looks like noise. On a long task it turns a hopeless job into a likely one.

Here are the words we need, restated from Chapter 24. An agent is a language model that works in a loop. It writes a request to run a tool, a program runs the tool and returns the result as text, and the model reads the result and writes again (§24.1). One pass through that loop is a step. The context window is the text the model reads in one call, and in a plain loop it holds the whole history of the task. §24.1 treated a task as a chain of steps that fail independently, and in its model only the errors nobody catches end the task.

That section also read the measurements of METR, a nonprofit that times skilled people on a suite of software tasks and reports how long a task each frontier agent can finish. Those measurements hold a puzzle. The task length an agent finishes four times in five is a tenth to a quarter of the length it finishes half the time. If the agent failed at a constant rate per minute of work, that fraction would be about a third.

This section answers five questions in turn.

1. What is a small gain in step accuracy worth?
2. Why do measured horizons fall off faster than a constant failure rate allows, and what does that predict for the 99% reliability a product needs?
3. What happens when an agent's past errors make its next error more likely?
4. How should an agent save its progress, and how often, so that a crash costs little and a card is charged exactly once?
5. How do you notice an agent that is stuck, and how often should a person look in?

The two previous sections kept the agent's memory in order. §25.2 kept the window small by sliding it, clearing old tool results, and summarizing the history. §25.3 built stores outside the window and read them back. This section keeps the task itself alive. Interviews for agent teams probe every part of it, from the reliability arithmetic to the bug that charges a customer twice.

What a gain in step accuracy is worth

Begin with the simplest model, the one of §24.1. The step accuracy $p$ is the probability that a step goes right, and $q = 1 - p$ is the step error rate (that section writes $\varepsilon$ for it). Assume that steps succeed independently and that any error ends the task. Then a task of $n$ steps succeeds with probability $p^n$, the chance that all $n$ steps go right.

A product rarely cares about succeeding half the time. It needs a target, such as 80% or 99% of tasks done. So the number to track is the longest task the agent finishes at that target.

Take $p = 0.99$ and $s = 0.5$. Then $\ln 0.5/\ln 0.99 = 68.97$, so $H_{0.5} = 68$. The agent finishes a 68-step task at least half the time, and a 69-step task less often than that.

When errors are rare, the horizon is close to $\ln(1/s)/q$. §24.1 proved this for the half-life, from the series $\ln(1/p) = q + q^2/2 + \dots$, which lies between $q$ and $q/(1-q)$. The same two bounds hold at any target,

At $q = 0.01$ and $s = 0.5$ the bounds are 68.6 and 69.3, and the true value 68.97 sits between them. In plain words, the horizon is nearly inversely proportional to the error rate. Halve the errors and the agent finishes tasks twice as long at the same target. What §24.1 did not ask is how fast the horizon moves when the accuracy does.

In plain words, the horizon grows like $1/q$, and $1/q$ shoots up as $q$ goes to zero. A gain of a tenth of a percentage point adds 0.07 steps to the 50% horizon at 90% accuracy, and 347 steps from 99.8% to 99.9%. So a benchmark of short tasks can show step accuracy creeping from 99.0% to 99.9%, which looks like saturation, while the length of task the model can carry out grows tenfold. Sinha et al. (2025) named this the illusion of diminishing returns.

The cell tabulates the 50% and 99% horizons across step accuracies. Its last column is the number of steps that one hundredth of a percentage point of accuracy buys, a change of $10^{-4}$ in $p$.

The last column runs from less than a hundredth of a step at 90% to about 6,900 steps at 99.99%. The middle column holds a second lesson. Even at 99.99% per step, the agent finishes only 100 steps with 99% reliability, against 6,931 steps half the time. For any accuracy near 1, the 99% horizon is about 69 times shorter than the 50% one, because $\ln(1/0.99)/\ln 2 = 0.0145$. If your product needs a task to succeed 99 times in 100, the middle column sets the length it can promise, not the headline one. An interviewer who hears "the benchmark is saturated at 99%" expects you to ask which horizon the product needs.

From steps to minutes, and the ratio of two horizons

Nobody counts the steps of a real task in advance. What you can measure is how long the task takes a person, and that is how METR measures agents (Kwa et al. 2025), as §19.4 described. Skilled professionals are timed on a suite of software and research tasks. The length $t$ of a task is the geometric mean of their successful times, the exponential of the average log time, so times of 30 and 120 minutes give 60. Each model attempts every task several times. Then METR fits a logistic curve of the model's success against $\log_2 t$,

Here $\sigma(z) = 1/(1 + e^{-z})$ is the logistic function, $h$ is the length at which the curve passes 50%, and $\beta > 0$ sets how steeply success falls with length. In plain words, success is one half at length $h$, and each doubling of the task length lowers the log-odds of success by $\beta$. The $u$ time horizon $h_u$ is the length at which the fitted success rate equals $u$. So $h_{0.5} = h$ is the headline "50% time horizon", and $h_{0.8}$ is the length the model finishes four times in five. This section follows the notation of §19.4. §24.1 writes $t_{50}$ and $t_{80}$ for $h_{0.5}$ and $h_{0.8}$, because there $h$ already names the agent's history, and it writes $\lambda$ for the hazard rate $\lambda_h$ defined next.

To connect minutes with steps, picture an attempt working through a task one human-minute at a time. At each moment the attempt is either still on track or has already failed. The hazard rate is the chance per minute that an attempt still on track fails in the next minute.

A constant hazard is the step chain in continuous time. Suppose each step of agent work stands for $t_{\text{step}}$ minutes of human work. Then a task of $t$ minutes has $t/t_{\text{step}}$ steps, and $p^{t/t_{\text{step}}} = e^{-\lambda_h t}$ with $\lambda_h = \ln(1/p)/t_{\text{step}}$. Ord (2025) proposed reading METR's curves this way.

§24.1 compared this model with METR's measured ratios and found every one below its prediction. This section extends it to the stricter targets a product needs. Here is a tiny case. An agent that fails once per 1,000 human-minutes on average has $\lambda_h = 0.001$. It succeeds half the time at $h_{0.5} = \ln 2/0.001 = 693$ minutes, and four times in five at $h_{0.8} = \ln 1.25/0.001 = 223$ minutes. The ratio is $223/693 = 0.322$, and it would be the same for any hazard.

Two facts proved in §24.1 carry over. First, suppose tasks differ, each with its own constant hazard. Then the ratio $h_{0.8}/h_{0.5}$ of the whole suite can only fall below 0.322, and it equals 0.322 only when every task has the same hazard. Second, METR's logistic curve has ratio $h_{0.8}/h_{0.5} = 2^{-\ln 4/\beta}$, so the ratio is a reading of the slope $\beta$. Either way, the ratio, not the headline horizon, tells you how evenly an agent fails.

A fixed ratio also says how far behind the headline a stricter horizon runs. Suppose the 50% horizon doubles every $D$ days. A horizon that is always half as long reaches any given length exactly one doubling time later.

In plain words, each halving of the ratio costs one doubling time. METR's 50% horizon has doubled every 128.7 days since 2023. Under a constant hazard the 80% horizon would trail it by $128.7 \times \log_2(1/0.322) \approx 210$ days. The 99% horizon would trail by $128.7 \times \log_2 69 \approx 786$ days, more than two years. The measured ratios are smaller than 0.322, so the real lags are longer.

Two warnings apply to every number here. A horizon measures the difficulty of tasks in human time, not how long the agent runs. And the intervals widen at the top, to a factor of five or more, because the suite has few tasks that long. METR calls estimates above 16 hours unreliable with its current suite. The cell lists the two horizons of nine models, copied from METR's data file, and sets each beside the 80% horizon a constant hazard would predict.

Every ratio is below 0.322, by a lot. So the constant-hazard model overstates every 80% horizon, by more than three times for Claude Opus 4.6. No single per-minute failure rate can produce these numbers. A spread of hazards across tasks pushes the ratio the right way. To fit the data, and to predict the 99% horizon a product cares about, we need a spread concrete enough to compute with.

Uneven tasks bend the curve

A constant hazard says every minute of every task is equally dangerous. Real tasks differ. Some are long and routine, and some are short and treacherous. METR's FAQ gives a direct picture. On its tasks that take humans 90 minutes to 3 hours, a GPT-5 agent succeeds every time on about a third of them, fails every time on another third, and is inconsistent on the rest. That is what it looks like when each task has its own hazard.

Now let every task have its own constant hazard $\lambda$, drawn from a distribution across the suite. The suite's success curve is the average of the tasks' curves, $S(t) = \mathbb{E}_\lambda[e^{-\lambda t}]$. This average of exponentials is called the Laplace transform of the hazard distribution. For one family of distributions it has a closed form.

The Gamma distribution $\operatorname{Gamma}(\kappa, b)$ is a flexible family on the positive numbers. It has a shape $\kappa > 0$ and a rate $b > 0$, here measured in minutes. Its density is $f(\lambda) = b^\kappa\lambda^{\kappa-1}e^{-b\lambda}/\Gamma(\kappa)$ and its mean is $\kappa/b$. Its coefficient of variation, the standard deviation divided by the mean, is $1/\sqrt{\kappa}$. So a small shape means very uneven tasks, and $\kappa \to \infty$ means every task has the same hazard.

A Gamma distribution with a finite shape always spreads its hazards, so the mixture bound keeps its ratio strictly below 0.322 at every finite $\kappa$. The bound itself has a second proof worth knowing. It needs no derivatives and shows where the inequality comes from.

In plain words, mixing easy and hard tasks makes the suite's success curve fall fast at first, as the hard tasks fail, and then flatten, as the easy ones keep going. So the 80% point arrives much earlier, relative to the 50% point, than a single hazard allows. Solving the theorem's ratio for $\kappa$ turns each measured ratio into a shape. The next cell does that for each model. Then it uses the shape to predict the 99% horizon, $h_{0.99} = h_{0.5}\,(0.99^{-1/\kappa} - 1)/(2^{1/\kappa} - 1)$.

The measured ratios give shapes between about 0.25 and 1, and around 0.5 for a typical frontier model. A shape of 0.5 is a coefficient of variation of hazards near 1.4. So two tasks of the same length often differ several times over in how dangerous they are. This reading is ours, computed from METR's published numbers. It is not a claim of METR or of Ord, and the deeper box below says how far to trust it.

The practical consequence concerns high reliability. At $\kappa = 0.5$, $h_{0.99}/h_{0.5} = 0.0068$, so the 99% horizon is about $1/148$ of the 50% horizon instead of $1/69$. For the newest model in the table, a constant hazard predicts a 99% horizon of 15 minutes and the matched Gamma about 7. Both are extrapolations far below the 80% point. METR does not measure there, because that would need many very short, very reliable tasks. Read them as a reason for caution before you promise high reliability on long tasks. In an interview, the point to make is that the spread of difficulty across tasks, not the headline horizon, sets what you can promise at 99%.

The widget puts both models side by side, with task length counted in steps. Its left panel plots success against task length for a constant hazard (solid) and for Gamma-distributed hazards (dashed), matched at the 50% point. Its right panel plots the ratio $h_{0.8}/h_{0.5}$ that each shape implies, with METR's models placed on it and a ring at the shape you choose.

import numpy as np

def horizon_ratio(u, v, shape):
    return u / v                              # placeholder

def fit_shape(ratio, u=0.8, v=0.5):
    raise NotImplementedError

Errors that feed on themselves

So far each step failed independently of the others. Inside a real task that assumption breaks in a specific way. An agent's mistakes do not vanish once made. They stay in its context window, and the model reads them again on every later step.

Sinha et al. (2025) isolated this effect with a task that needs no knowledge and no planning. The model gets a dictionary that maps common five-letter words to integers between $-99$ and $99$. On each turn it receives a few words and must add their values to a running sum. The plan and the knowledge are both in the prompt. So the only thing that can fail is execution, carrying out the steps it was told. Even so, accuracy per turn fell as the turns went on.

To find out why, they ran a counterfactual. They replaced the model's own history with a fabricated one at a controlled error rate. With a fully correct history, accuracy at turn 100 still dropped somewhat. That is the long-context degradation of §25.1, where a model's accuracy falls as its context grows. With errors planted in the history, accuracy dropped further, and more planted errors meant a larger drop. The model was more likely to err after seeing its own past errors.

The effect is not confined to toy tasks. In Vending-Bench (Backlund and Petersson 2025) an agent runs a small vending business. It orders stock, sets prices, and pays a daily fee, through tools that advance simulated time. It keeps the last 30,000 tokens of its history in its context, plus a scratchpad, a key-value store, and a vector database for memory. Each run is capped at 2,000 messages, and most use about 25 million tokens and 5 to 10 hours of real time.

The best models made a profit in most runs, yet every model had runs that derailed. Agents misread delivery schedules, forgot orders, or fell into what the authors call "meltdown" loops, from which they rarely recovered. The day a run stopped selling barely tracked the day its memory filled up. Across runs the Pearson correlation of the two was 0.167, where 1 would mean they move in lockstep and 0 means no linear relation. So the breakdowns were not a matter of running out of window. The authors trace most of them to the agent misreading its own situation, for instance believing an order had arrived when it had not, and then veering off from that mistake.

A small simulation puts numbers on the mechanism. It needs two more terms. A verifier is a check, such as a test, that catches an error so the agent can repair it. A context reset starts a fresh session from a short, clean handoff, a note of what is done and what comes next, instead of the full transcript.

Here is the model. Each step errs with probability $q_0(1 + \alpha n_e)$, capped at 1. Here $q_0$ is the base error rate, $n_e$ is the number of error traces in the context, and $\alpha$ is the strength of self-conditioning. A verifier catches and repairs an error with probability $r$, but the trace of the repaired error stays in the context. An error that is not caught ends the run. With $\alpha = 0$ this is the step chain with a fatal error rate of $q_0(1 - r)$, so $H_{0.5} \approx \ln 2/(q_0(1-r))$. A reset every so often sets $n_e$ back to zero.

The reset is free in this model, and in reality it is not, because the handoff can drop something the agent needed. That trade is the compaction problem of §25.2. Compaction replaces a long history with a summary written by the model, and now it has a second purpose. A summary that states what is true now, without the history of failed attempts, also removes the errors the model would otherwise condition on. An interviewer asking how to make an agent last longer wants to hear this, along with the cost of a lossy handoff.

Practitioners report the same lever. Anthropic's account of its long-running coding harness (Anthropic 2026) separates two operations. A context reset clears the window and starts a fresh agent from a structured handoff. Compaction summarizes the history in place, so the same agent keeps going. Claude Sonnet 4.5 began wrapping up work early as it sensed the end of its window, which the authors call "context anxiety". It did so strongly enough that resets were essential. Claude Opus 4.5 largely lost that behavior, so automatic compaction was enough.

The simulation estimates these horizons by sampling 4,000 runs. They can be computed exactly, because a run that is still alive is fully described by one number, the count $n_e$ of error traces in its context.

import numpy as np

def survival_exact(q0, alpha, r, n_steps, reset_every=None):
    return (1 - q0) ** np.arange(1, n_steps + 1)     # placeholder: no verifier, no self-conditioning

def horizon_exact(q0, alpha, r, s=0.5, reset_every=None, max_steps=2000):
    raise NotImplementedError

The exact horizons are 346 steps with $\alpha = 0$ and 178 with $\alpha = 1$. With resets every 50 and every 20 steps they are 286 and 320. All four are within the sampling noise of the simulation.

Checkpoints and restarts

So far an error was either fatal or quietly repaired. Most engineered agents sit in between. An error is detected, because a test fails or a tool returns an error, but the damage cannot be fixed in place. So the agent rolls back to the last saved state and redoes the work since then. Saving often wastes time on saves, and saving rarely wastes time on redone work. How often should it save?

Model it in steps. A task has $n$ steps, each succeeding independently with probability $p$. Every $k$ steps the agent writes a checkpoint, a saved state it can return to, which costs $c$ steps' worth of time. A failed step is noticed at once, and the agent restarts from the last checkpoint. We call the $k$ steps between two checkpoints a segment.

Start with one segment, by hand. Each attempt at the segment runs until its first failure or until it has $k$ successes in a row. At $p = 0.99$ and $k = 50$, a single attempt gets through with probability $0.99^{50} = 0.605$, so about two attempts in five fail somewhere and start over. The theorem below counts the cost exactly. The segment takes $(0.99^{-50} - 1)/0.01 = (1.6529 - 1)/0.01 = 65.29$ steps on average, 31% more than its 50 useful steps.

Item 3 is the same square-root rule that sets how often a large training job saves its weights. In §10.7, a job that loses everything since its last save when a GPU fails should save every $\sqrt{2\delta M}$ seconds. There $\delta$ is the time one save takes and $M$ is the mean time between failures. The rule was first derived by Young (1974), and the cell below labels it with his name. For an agent, the checkpoint cost $c$ plays the part of $\delta$, a failed step plays the part of a failed GPU, and the mean time between failures is $M = 1/q$ steps.

In plain words, checkpointing turns an exponential cost into a nearly linear one. Without checkpoints a task of $n$ steps costs $(p^{-n} - 1)/(1-p)$ steps, which grows exponentially in $n$. With them it pays a small overhead per step, set by balancing saving too often against redoing too much. The best interval is the square root of twice the checkpoint cost times the mean number of steps between failures. Detecting failures only at the checkpoint costs as much as failing twice as often.

The cell checks the closed form against simulation, and the approximate optimum against an exact search.

The simulation agrees with 65.29 to within its noise. The square-root rule is exact at $c = 1$, slightly high at $c = 5$ (45 against 42), and well off at $c = 20$ (89 against 77), where $k^\star q \approx 0.4$ is no longer small. When checkpoints are expensive relative to the failure rate, search the exact formula instead of trusting the square root. Interviewers like this question because it has a clean derivation and a practical caveat.

The widget plots the extra steps a 1,000-step task costs against the checkpoint interval, once for failures noticed at once and once for failures noticed only at the checkpoint.

import numpy as np

def expected_steps(n, k, p, c=0.0):
    return n / k * p ** -k / (1 - p)          # placeholder with two classic bugs

def best_interval(n, p, c):
    return int(round(np.sqrt(2 * c / (1 - p))))

Durable state, and side effects that happen exactly once

A training job's checkpoint is a set of tensors (§10.7). What is an agent's checkpoint? It is not the context window. A window can be rebuilt, compacted, or thrown away, and after a crash it is gone anyway. What must survive is whatever a fresh process needs, starting with an empty window, to carry on as if nothing had happened. We call a record durable when it is written somewhere that outlives the process, such as a disk or a database.

The hard part is item 5. A side effect is an action that changes the world outside the agent. Inside a sandbox, an isolated machine where the agent's code runs (§24.5), most side effects can be undone by restoring a snapshot. Outside it, a charged card, a sent email, a shipped order, or a migrated table cannot be undone.

Distributed systems have precise words for what that example does (Kleppmann 2021).

Most real actions are not idempotent, so services deduplicate them. Payment APIs work this way, and §24.6 builds timeouts, backoff, and retries around them. What remains for the agent is to make sure the key survives a crash. The restarted process must send the same key, so it must have written the key somewhere durable before calling. This is the write-ahead rule of database recovery, and ARIES, the recovery method most databases follow, is built on it (Mohan et al. 1992). Log your intent before you change the world.

The protocol has three rules.

1. On start, read the log. Every step with a result record is done, and its result is replayed from the log, never recomputed.
2. For the next step, reuse the key of its intent record if there is one. Otherwise create a key and append the intent record durably.
3. Call the tool with the key, then append the result record.

That four-case argument is what an interviewer wants to hear, and each assumption matters. Without atomic appends, a half-written intent record could hide a key. Without deduplication by the service, case 3 performs the effect twice. If keys were not unique, two steps could collide. And if a step's arguments came from a fresh model call, a retry could send different contents under the same key, which is why results are replayed rather than recomputed. The widget lets you place the crashes yourself and watch each restart.

The cell runs a five-step plan against a flaky service. The service crashes the agent's process a quarter of the time, either before or after acting. The plan runs once with no log, so every attempt gets a new key, and once with the write-ahead log.

Without the log, the customer is charged $40 four times. With it, a run with as many crashes produces each effect exactly once. The price is a few repeated calls, which the service answers from its records.

Production systems package this pattern as durable execution. Workflow engines such as Temporal, which run long business processes as ordinary code, record the result of every step in an event history. After a crash they rerun the workflow code and feed it the recorded results instead of calling again. Agent frameworks such as LangGraph, a library that runs an agent as a graph of steps, save the agent's state after each step for the same purpose.

One rule matters more for agents than for ordinary programs. Replay results, never recompute them. A model call is not a deterministic function. Sampling at a temperature above zero varies by design. Even at temperature zero, a served model can return different tokens from one call to the next. Its kernels give slightly different results depending on how many requests share a batch, and your request lands in batches of different sizes (He 2025). If a restarted agent regenerates step 7 instead of reading it from the log, it may write a different email under the same idempotency key. Everything after it can then diverge.

When you do want to recompute, for instance to debug a run, everything random must be in the checkpoint. The block below runs a small sampled policy for 60 steps. Then it repeats the run with a crash after step 25, a checkpoint that includes the random generator's state, and a resume in a fresh process.

With the generator state restored, the resumed run is identical to the uninterrupted one. Reseeding instead changes 25 of the 35 resumed actions, a different trajectory from the same checkpoint. With a hosted model you cannot checkpoint its sampler at all, which is why the log of results is the source of truth.

Some actions cannot be made safe by any key, because their effect cannot be undone. There are two tools for those. A compensating action undoes the business effect after the fact, such as a refund for a charge or a correction for a wrong email. Long business transactions that cannot hold locks are built this way, as a chain of steps each paired with its compensation. That design is called a saga (Garcia-Molina and Salem 1987).

An approval gate stops the agent before an irreversible or expensive action until a person approves it. Typical gated actions are a deploy, a payment above a limit, a deletion, or a message to a customer. Gates sit before the action, never after. They are also the one place where the check-in arithmetic later in this section does not apply. The cost of an unreviewed irreversible mistake is not bounded by the work done since the last check.

def run_durable(plan, log, service, new_key):
    results = []
    for step in plan:                             # placeholder: no log, a new key every attempt
        op, arg = step(results)
        results.append(service.call(op, arg, new_key()))
    return results

Detecting a stuck agent

Everything so far protects safety, in the sense used for concurrent programs, that nothing bad happens. No state is lost and no effect is doubled. A long-running agent also needs liveness, the property that something good eventually happens. Both terms come from the study of concurrent programs (Lamport 1977).

A failure of liveness is quieter than a crash. The agent retries the same failing command, rereads the same file, or keeps "fixing" a test by reverting its last fix. In Vending-Bench some runs fell into such loops and rarely recovered. Nothing crashes, so no restart helps, and the budget drains.

Four cheap mechanisms catch the common cases. Each belongs outside the model, where a confused agent cannot argue with it.

1. Budgets. Cap each task's steps, tokens, dollars, and wall time, the episode budgets of §24.1. Keep the counts in the durable log so a restart cannot reset them. Frameworks ship one. LangGraph, for instance, stops a graph after a configurable number of steps and raises an error.
2. Repeat detection. Hash each canonicalized (tool, arguments) pair, keep the hashes of the last $W$ actions, and flag a loop as soon as one hash occurs more than $R$ times. Canonicalization matters. A model that varies the order of the keys in an argument dictionary would otherwise slip past an exact comparison. A cycle of two actions, A, B, A, B, is caught the same way, since each of its actions repeats.
3. A heartbeat and a watchdog. The agent appends a heartbeat record, with its progress marker, to the durable log at every step. A separate supervisor process, the watchdog, checks how long ago the last heartbeat arrived and whether the marker has moved. Temporal does the same for long activities. A worker sends heartbeats that can carry a progress payload. When the heartbeat timeout passes without one, the attempt fails and is retried, and the retry can resume from the last payload.
4. Escalation. When any of these trips, the agent does not just stop. It writes a short account of what it tried. The supervisor then either resets the context, since repeats are often self-conditioning that a clean handoff removes, or asks a person. That is a check-in triggered by an event rather than by a clock.

def detect_loop(actions, window, max_repeats):
    seen = {}
    for i, (tool, args) in enumerate(actions):           # placeholder: whole history, raw text
        k = (tool, str(args))
        seen[k] = seen.get(k, 0) + 1
        if seen[k] >= max_repeats:
            return i
    return None

A production detector also puts the observation into the key. The same command run after a fix is progress, and run against an unchanged workspace it is not. It also treats a stall in the progress marker as the stronger signal, because an agent can loop through many different actions without getting anywhere.

Progress files and human check-ins

Durable state answers "where was I". A long task also needs answers to "how far along am I, really" and "should a person look at this now".

The most detailed public account of progress tracking for long-running agents is Anthropic's harness for building a web application across many context windows (Anthropic 2025). A harness is the program around the model that runs the loop, supplies the tools, and decides what goes into each context. Out of the box, a frontier model running in a loop with compaction failed in two ways. It tried to build everything at once, ran out of context in the middle of a feature, and left the next session to guess what had happened. And later sessions looked around, saw progress, and declared the job done.

The fix was a first session with a different prompt, an initializer. It wrote an init.sh script that starts the app, a progress log, and an initial git commit. It also wrote a feature list of more than 200 end-to-end features in JSON, every one marked as failing. Every later session starts with the same routine.

1. Read the progress log and the git history.
2. Start the app with init.sh and run a basic end-to-end test, to catch anything left broken.
3. Pick the highest-priority failing feature and work on that one feature only.
4. Test it as a user would, through browser automation, and mark it passing only after the test passes.
5. Commit, and update the log.

Sessions may change only the pass or fail status of a feature, and the prompt forbade editing or deleting the tests. JSON was chosen because the model was less likely than with Markdown to rewrite the file. The last failure the authors saw was a feature marked complete without an end-to-end test. Prompting the agent to test through browser automation, as a user would, fixed most of it.

The follow-up (Anthropic 2026) separated the agent that does the work from an evaluator agent that tests it. Agents grading their own work praised it even when it was mediocre, and a skeptical outside grader was easier to tune than a self-critical worker. On one prompt, a retro game maker, a single agent ran for 20 minutes at a cost of about $9 and produced an app whose core feature, playing the game, was broken. The full harness of planner, generator, and evaluator ran for 6 hours at about $200 and produced a far richer app whose play mode worked. Even the evaluator needed several rounds of prompt tuning. At first it found real bugs and then talked itself into approving the work anyway. Long runs are expensive, which is one more reason to put a number on how often a person should look.

Here is that number. A check-in costs the reviewer's time and the agent's idle wait, $c_h$ dollars in all. Not checking has a cost too. Some errors slip past the agent's own tests, such as a misread requirement, a wrong approach, or a quiet regression. Suppose they arrive at rate $\lambda_e$ per hour of agent work. Work done after such an error is wasted until a person catches it, and an hour of work is worth $w$ dollars. A person checks every $\tau$ hours.

This is the square-root rule of checkpoints again, $\tau^\star = \sqrt{2\delta M}$. The check-in cost in hours of work, $c_h/w$, plays the part of the checkpoint time $\delta$. The mean time between undetected errors, $1/\lambda_e$, plays the part of the mean time between failures $M$. In plain words, take two lengths of time. One is what a check costs in hours of work, and the other is how long the agent typically goes between mistakes it cannot catch itself. Check in at $\sqrt 2$ times their geometric mean.

The widget draws both cost curves for this example. Shorten the time between uncaught errors, and the first-order interval drifts from the exact optimum while its cost stays close.

The table collects every mechanism in this section, with what each protects against and what it costs.

Measuring the horizon of your own agent

Everything above can be measured for your agent, not only for frontier models on METR's suite. A sound recipe has six parts.

1. A task suite with lengths. Use real tasks from your product, each with a human time estimate (time a few experts) or, for synthetic tasks, a step count. Cover lengths over several factors of two, and include short tasks so the fit is anchored at both ends.
2. Repeated runs. Run each task several times, as METR's original study did with about eight runs per task. A single run cannot tell a task the agent solves half the time from one it always solves.
3. Fits. Fit the logistic curve of §19.4 and a constant hazard by maximum likelihood, and compare them by held-out likelihood. Report $h_{0.5}$, $h_{0.8}$, and their ratio, which tells you how uneven your tasks are.
4. Intervals. Use a bootstrap, which resamples the data with replacement and refits many times to see how much the answer moves. Resample task families, then tasks within them, then runs, because tasks in a family succeed and fail together (§14.7).
5. Reliability. Report pass$^k$, the probability that all of $k$ runs of a task succeed (§22.5), and the cost per success. A product needs the agent to work every time, at a price.
6. Checks of the runs. Read failed and successful transcripts for broken tasks and for reward hacking, an agent passing the scorer without doing the task. METR reviews flagged runs by hand before publishing any number.

For a step-level view that separates execution from planning, the running-sum task of Sinha et al. (2025) is cheap to generate and free of contamination, and it reports $H_s$ directly.

The last exercise does the cross-model version of this analysis on METR's own data. It ends where a product team would, with a forecast and an honest interval.

import numpy as np
from scipy.optimize import brentq

def implied_shapes(models):
    raise NotImplementedError

def lag_days(ratio, doubling_days):
    raise NotImplementedError

The interval understates the uncertainty. Resampling the models captures how the trend and the median ratio move with the choice of models. It does not capture the uncertainty of each model's own estimate, and the 95% interval for Claude Opus 4.6 alone spans a factor of eleven. Nor does it capture the extrapolation of a Gamma fitted at 50% and 80% down to 99%, or whether the trend continues past the 16 hours the suite can measure.

Where the models break

Every model in this section is a useful simplification, and each fails in a specific, predictable way.

• Steps are not independent. Errors cluster on hard steps and feed on themselves through the context. The step chain describes independent errors only. Dependence can make things worse, as self-conditioning does. It can also make things better, when errors concentrate on a few hard steps and leave the rest of the task safe.
• Human time is not model difficulty. A task that takes a person an hour of reading may take an agent one tool call, and a two-minute task that needs a visual judgment may be beyond it. Horizons are measured on clean, self-contained, automatically scored tasks, and Kwa et al. (2025) found agents did much worse on messier ones.
• Checkpoints do not help against silent errors. Rollback needs detection. An error that passes every test, such as a feature marked done without being tested, is preserved by every checkpoint after it. That is why verification and outside evaluators matter as much as saving state.
• Some effects cannot be rolled back, only compensated. That is why irreversible actions need approval gates rather than check-in intervals.
• The horizon is not a duration. A model with a 17-hour horizon is not one that works for 17 hours. An agent that runs for days is chaining many tasks, and each must be within its horizon at the reliability the product needs.

Read more

• Kwa et al. (2025). Measuring AI Ability to Complete Long Software Tasks, with METR's page Task-Completion Time Horizons of Frontier AI Models and its Time Horizon 1.1 data file. The metric, the logistic fit, and the doubling trend, the source of every METR number in this section, and an FAQ that answers most misreadings of the metric.
• Ord (2025). Is there a half-life for the success rates of AI agents?. The constant-hazard model of time horizons and what it predicts for higher reliability.
• Sinha et al. (2025). The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs. Step horizons, the counterfactual experiment that isolates self-conditioning, and single-turn execution lengths of frontier models.
• Backlund and Petersson (2025). Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents. Runs of more than 20 million tokens, high variance, and breakdowns that do not track the context filling up.
• Anthropic (2025). Effective harnesses for long-running agents, and its follow-up (2026), Harness design for long-running application development. Initializer and coding agents, feature lists, progress files, context resets, and separate evaluators, from practice.
• Young (1974). A first order approximation to the optimum checkpoint interval. The square-root rule for how often to save, in two pages.
• Kleppmann (2021). Distributed Systems, Cambridge lecture notes, section 5.1. Idempotence and at-most-once, at-least-once, and exactly-once semantics, in a few clear pages.
• He (2025). Defeating Nondeterminism in LLM Inference. Why a served model at temperature zero still varies from call to call, and why an agent's log, not a rerun, is the record of what happened.
• Garcia-Molina and Salem (1987). Sagas, and Mohan et al. (1992), ARIES. Compensating actions for long transactions, and the write-ahead logging that makes recovery exact.

Watch next

▶

▶

▶

▶