A sparse autoencoder (SAE) is a tool for finding the variables a trained network computes with. It takes the network's internal activation vectors, one per token, and rewrites each one as a short sum of directions taken from a long learned list. It needs no labels. Each direction on the list is a candidate feature, a concept the network may represent, such as "this text is base64" or "this is the Golden Gate Bridge". SAEs are the main unsupervised tool of interpretability today. They have been trained with 34 million directions on Claude 3 Sonnet (Templeton et al. 2024) and 16 million on GPT-4 (Gao et al. 2024), and they are the building block of the attribution graphs of §23.8.

The need for such a tool comes from §23.5. A layer with $d$ coordinates can store far more than $d$ features if each feature is rarely active. It gives every feature its own direction, lets the directions overlap a little, and tolerates the small errors the overlap causes. This packing is called superposition. Its price is that single neurons mix unrelated concepts, which is called being polysemantic. One neuron in a one-layer Transformer studied by Bricken et al. (2023) responds to academic citations, English dialogue, HTTP requests, and Korean text. Reading neurons one at a time cannot untangle that. The features are directions, there are more of them than coordinates, and nobody told us which directions they are.

So the question for this section is concrete. Suppose a 4,096-dimensional activation is a sparse sum of hundreds of thousands of feature directions. Can we learn those directions from the activations alone? We answer in steps. We state the underlying problem, called dictionary learning, turn it into a network, and show by hand that its usual penalty biases every answer downward. Then we train SAEs from scratch on data whose true features we planted, remove the bias with newer rules, keep the SAE's units from going permanently silent, and measure the result honestly. The last parts cover how dictionaries fail, two relatives called transcoders and crosscoders, the results at frontier scale, and the 2025 evidence on what SAEs are and are not good for.

Keep in mind what an SAE can prove. Section 23.1 set out a ladder of evidence with four rungs, association (some quantity tracks a concept), direct effect, causal effect, and finally mechanism. An SAE on its own sits on the first rung. It proposes candidate variables, directions whose activations track concepts in the input. Only interventions on those variables, like the patching of §23.4, can show that the model uses them. As in the rest of this chapter, vectors are rows and a matrix acts on the right, $\mathbf{x}\mathbf{W}$.

Dictionary learning is the problem underneath

Start with a case small enough to check by hand. A layer has two coordinates but stores three features. Their unit directions are $\mathbf{g}_1 = (1, 0)$, $\mathbf{g}_2 = (-\tfrac12, \tfrac{\sqrt3}{2})$, and $\mathbf{g}_3 = (-\tfrac12, -\tfrac{\sqrt3}{2})$, spaced 120 degrees apart. On one token, feature 1 is on with strength 1.0 and feature 2 with strength 0.5. The activation is $\mathbf{x} = \mathbf{g}_1 + 0.5\,\mathbf{g}_2 = (0.75, 0.433)$.

That is the whole section in miniature. With more features than coordinates, many codes rebuild an activation equally well, and a preference for sparse codes chooses among them.

Now the general case. This chapter's synthetic data uses planted features. An activation is $\mathbf{x} = \sum_{i=1}^n a_i\mathbf{g}_i + \boldsymbol\epsilon \in \R^d$, where the $\mathbf{g}_i$ are $n > d$ unit feature directions. Each coefficient $a_i \ge 0$ is zero with probability $S$, called the sparsity, and $\boldsymbol\epsilon$ is small noise. Section 23.5, which wrote $m$ for the width, showed two things about such data. A model trained on it stores the features in superposition. And a sparse code can be recovered from fewer measurements than its length, which is the subject of compressed sensing. That recovery assumed the directions were known. Here we must learn them too.

In plain words, write every activation as a short nonnegative recipe over a shared set of ingredient directions. Choose the ingredients so that every recipe is both short and accurate. The L1 term is the sum of the entries, the quantity the hand example minimized.

Three facts about this problem shape everything that follows.

• For a fixed dictionary, finding one token's code is a convex problem. It is a least-squares fit with an L1 penalty and nonnegative entries, called a nonnegative lasso. The iterative shrinkage-thresholding algorithm (ISTA) solves it by alternating a gradient step on the squared error with a step that pulls every entry toward zero. Exercise 23.5d implements ISTA for codes of either sign, and the nonnegative version only adds a clip at zero.
• Jointly in the dictionary and the codes, the problem is not convex. Training finds a local optimum that depends on where it starts.
• The answer is defined only up to the order of the dictionary entries and their scale, since $f_i\mathbf{d}_i = (f_i/c)(c\,\mathbf{d}_i)$ for any $c > 0$. The unit-norm constraint removes the scale ambiguity, and we treat the dictionary as an unordered set.

The idea is old. Olshausen and Field (1996) learned a sparse code for small patches of natural images. The dictionary entries came out looking like the oriented, localized edge detectors that simple cells in the visual cortex implement. Nobody had told the algorithm about edges. Sparsity alone, applied to the right data, recovered the variables the data was built from. Interpretability bets that the same will happen with a network's activations. Classic sparse coding had already been run on the hidden states of BERT (Yun et al. 2021). In 2023 two groups showed, at about the same time, that sparse autoencoders find interpretable features in language models, Bricken et al. and Cunningham et al. (2023).

An interviewer may ask why sparsity is the right prior. The hand example is the answer. With more features than dimensions, reconstruction alone cannot choose a code. The assumption that each feature is rarely active is what makes the problem well posed.

An autoencoder predicts the code in one step

Solving a lasso for every token is expensive. ISTA needs tens to hundreds of iterations, each costing two products with the $F\times d$ dictionary, and an SAE at frontier scale is trained on billions of tokens. The fix is amortization. Instead of optimizing each token's code separately, learn a function that predicts the code in one matrix product.

That function is the encoder of an autoencoder. An autoencoder is a network with two halves. The encoder maps an input to a code. The decoder maps the code back to a reconstruction of the input. Training makes the reconstruction match the input, and §32.1 develops the idea in full. An ordinary autoencoder squeezes the input through a code smaller than the input. Ours is wider than the input, $F \gg d$, and what keeps it from copying the input is sparsity, not width. A vector-quantized autoencoder, which replaces each code by the nearest entry of a learned codebook, is the extreme case in which every code has exactly one nonzero entry, and that entry is 1.

The sparse autoencoder

In plain words, center the activation, then read off how strongly each of $F$ detectors responds. That response, before the ReLU, is the pre-activation. Keep only the positive responses, and rebuild the activation as a weighted sum of the latents' directions. Pay for errors with the squared distance, and for every unit of activation with $\lambda$.

Each piece has a reason.

• Subtracting $\mathbf{b}_{\text{dec}}$ before encoding makes the latents describe how an activation differs from a typical one. Bricken et al. found that this pre-encoder bias improved results in toy models. They start it at the geometric median of the dataset, the point with the smallest total distance to all the activations.
• The ReLU makes activations nonnegative, so a latent means "this feature is present, this much".
• The encoder bias $\mathbf{b}_{\text{enc}}$, usually negative, acts as a threshold. It keeps a latent silent until its detector's response is large enough.
• The fixed rescaling makes $\lambda$ mean the same thing across models and layers, whose raw activation sizes differ by orders of magnitude.

Two words need care. A latent is a unit of the SAE, one entry of $\mathbf{f}$. A feature is a variable the model is hypothesized to compute. The whole enterprise is the hope that latents are features, and much of this section is about when they are not. Papers often say "feature" for both.

The encoder and the decoder do different jobs, so they are kept as separate matrices. The decoder row $\mathbf{d}_i$ is where the feature points. The encoder column is a detector. A good detector for a feature in superposition must respond to $\mathbf{d}_i$ while ignoring the neighboring directions that overlap with it. Only if the dictionary were orthonormal would the best detector be $\mathbf{d}_i$ itself. Bricken et al. found that the learned encoder columns are often far from their decoder rows, and that leaving the two untied matters for capacity. In the runs below, an encoder column ends up on average at a cosine of about 0.9 with its decoder row, not 1.

In practice, the first thing anyone reports about an SAE is three numbers. They are how many latents fire per token, how much of the activation the SAE fails to explain, and how many latents never fire at all. The first exercise computes them.

import numpy as np

def sae_forward(X, W_enc, b_enc, W_dec, b_dec):
    raise NotImplementedError

def sae_metrics(X, f, X_tilde):
    raise NotImplementedError

An L1 penalty shrinks every activation

The L1 penalty charges for every unit of activation. So the cheapest answer under-reports every feature a little. This bias is called shrinkage. Start with the smallest possible case, by hand.

With an orthonormal dictionary the same calculation holds for every latent at once.

In plain words, an L1 SAE with an orthonormal dictionary is exactly the lasso, solved in one step. It zeroes every projection below $\lambda/2$ and subtracts $\lambda/2$ from every projection above it. The first part is the sparsity we wanted. The second part is a systematic bias we did not want. The cell checks the formula against a numerical minimizer.

In superposition the dictionary is not orthonormal, and shrinkage gets worse for a second reason. When feature $j$ is active, its overlap with $\mathbf{d}_i$ leaks into latent $i$'s pre-activation. This leak is called interference, and its size is about $a_j\,\mathbf{d}_i\cdot\mathbf{d}_j$. To stay silent on interference, latent $i$ needs a threshold, a negative encoder bias $b_{\text{enc}, i}$. But in $\operatorname{ReLU}(z_i)$ the same bias also sets the zero from which the magnitude is measured. So a threshold high enough to reject interference is subtracted from every genuine activation too. One parameter is doing two jobs. It decides whether a feature is present, and it sets how large the feature is reported to be. Rajamanoharan et al. (2024a) put the problem in exactly these terms, and every architecture after the plain ReLU SAE is a way to separate the two jobs.

In practice, an L1 SAE reports every feature as weaker than it is. If you use a latent's activation to measure how strongly a concept is present, that measurement is biased low. "Write the SAE loss" is a common interview question, and "what is shrinkage?" is almost always the follow-up.

The penalty must carry the decoder norm

The penalty in the definition box is $\lambda\sum_i f_i\lVert\mathbf{d}_i\rVert$, not the plain $\lambda\sum_i f_i$. The reason is a loophole, and numbers show it best. Take a latent whose activation is 0.8 and whose decoder row has norm 1, so its plain penalty is $0.8\lambda$. Divide its encoder column and bias by 10, and multiply its decoder row by 10. A ReLU of a number divided by 10 is the ReLU divided by 10, so the activation becomes 0.08. The reconstruction $0.08 \times 10\,\mathbf{d}_i$ is the same vector as before. But the plain penalty has fallen to $0.08\lambda$.

Repeat with any factor $c > 1$, and the plain penalty falls to $\lambda f_i/c$ at no cost in reconstruction. Left alone, gradient descent will grow the decoder norms to dodge the penalty. The L1 term then measures nothing but the arbitrary scale of the decoder. The cell runs the loophole on one random latent.

There are two standard fixes, and they agree whenever the norms are 1. Bricken et al. keep the plain penalty and constrain $\lVert\mathbf{d}_i\rVert = 1$, renormalizing each decoder row after every optimizer step. Templeton et al. drop the constraint and put the norm into the penalty. That makes $f_i\lVert\mathbf{d}_i\rVert$ the natural "feature activation" and $\mathbf{d}_i/\lVert\mathbf{d}_i\rVert$ the feature direction.

The constraint needs one more detail. The part of a row's gradient that points along the row only changes the row's length, and the renormalization immediately undoes that. The Adam optimizer (§0.6), which scales each step by running averages of past gradients, would still fold that wasted part into its averages. So Bricken et al. remove it before the step,

where $\mathbf{g}_i$ is the gradient of the loss with respect to $\mathbf{d}_i$. In plain words, keep only the part of the gradient that turns the row, and drop the part that stretches it. Geometrically, this projects the gradient onto the plane tangent to the unit sphere at $\mathbf{d}_i$. Bricken et al. report a small but real reduction in total loss from it, and Gao et al. use it too.

Training an SAE on planted features

On synthetic data we know the right answer, so we can train SAEs and grade them. The data has $n = 128$ planted features in $d = 32$ dimensions, four features per dimension. Each feature is on with probability $1 - S = 0.015$, and when on, its coefficient is uniform on $[0.5, 1.5]$. A token has $128 \times 0.015 = 1.92$ active features in expectation, and 1.95 in this sample.

The trainer below implements the derivation line by line, with unit-norm decoder rows, the tangent projection, and Adam. It also supports a second rule, set by rule="topk". That rule keeps the $k$ largest pre-activations of each token and uses no penalty. It is called TopK and is defined properly in the next section, but having it here lets us compare the two on the same data. As in the equation of Gao et al., the TopK encoder has no bias.

We grade a dictionary in six ways.

• L0 and FVU are the usual sparsity and fidelity numbers. FVU, the fraction of variance unexplained, is the squared reconstruction error divided by the total variance of the data.
• Recovery is the mean, over the 128 true features, of the largest cosine between the feature's direction and any decoder row. A score of 1 means every true feature has a latent pointing exactly along it.
• Shrinkage is measured on the tokens that contain exactly one true feature. It is the mean ratio of the best-matching latent's activation to the true coefficient, so a ratio of 1 means unbiased.
• Dead is the fraction of latents that never fire on the data.
• Detector alignment is the mean cosine between each latent's encoder column and its decoder row.

Even TopK reports only 0.90 of the true value on single-feature tokens, and TopK has no penalty. The likely reason is that the encoder learns to discount every reading a little. A reading of feature $i$ usually also contains interference from other active features. For a reading $a + e$ with independent zero-mean noise $e$, the gain $c$ that minimizes $\mathbb{E}(c(a + e) - a)^2$ is $\mathbb{E}a^2/(\mathbb{E}a^2 + \mathbb{E}e^2)$, which is below 1. A noisy measurement is best shrunk a little before you use it.

The widget lets you watch training with every direction visible. Five features live in a plane, arranged as the pentagon that §23.5 showed a toy model building, so the superposition is extreme. You choose the rule, the penalty or $k$, and the number of latents $F$. Every change restarts training, and Train again starts over from a new random initialization.

import numpy as np

def sae_l1_grads(X, params, lam):
    raise NotImplementedError

def train_sae(X, F, lam, steps, lr, rng, B=256):
    raise NotImplementedError

TopK, JumpReLU, and gates separate detection from size

The fixes for shrinkage all attack the same coupling, one parameter deciding both whether a latent fires and how large it is. They do it in three ways. TopK fixes how many latents may fire. JumpReLU learns a separate firing threshold for each latent. Gated SAEs give detection its own path through the network.

TopK sidesteps shrinkage because nothing penalizes magnitude, so the kept values pass through unchanged. The gradient flows only through the $k$ selected latents. The selection itself is treated as fixed, the same way backpropagation through a maximum sends gradient only to the entry that won. Gao et al. report a better trade-off between reconstruction and sparsity than ReLU SAEs, and clean scaling laws in $F$ and $k$. They also note a practical advantage. L0 becomes a number you set, not one you discover after a sweep over $\lambda$.

The drawback is rigidity. Every token gets $k$ latents whether it needs them or not, the beginning-of-sequence token included. And an SAE trained at one $k$ reconstructs substantially worse when run at a much larger one. BatchTopK relaxes the first problem. Bussmann et al. found that their BatchTopK SAEs used one latent on some tokens and more than 80 on others. They reconstructed better than TopK at the same average L0 on GPT-2 Small and Gemma 2 2B.

Selecting the top $k$ correctly, without loops and in both forms, is a common coding question.

import numpy as np

def topk_encode(Z, k):
    raise NotImplementedError

def batch_topk_encode(Z, k):
    raise NotImplementedError

JumpReLU learns a threshold for each latent

The second fix gives each latent its own threshold and stops subtracting that threshold from the magnitude.

In plain words, the threshold decides whether the latent fires, and once it fires, its value is left alone. The L0 penalty counts active latents instead of charging for their size, which removes the incentive to shrink. For example, take $\theta_i = 0.3$. A pre-activation of 0.25 gives 0, and a pre-activation of 1.0 gives 1.0. A ReLU latent that used a bias of $-0.3$ as its threshold would turn the same response of 1.0 into 0.7.

The difficulty is that the loss is a step function of $\theta_i$. Nudging a threshold changes nothing until some pre-activation crosses it. So the derivative with respect to $\theta_i$ is zero almost everywhere, and plain backpropagation never moves the thresholds. The way out is to remember what we are minimizing. It is an expected loss over the data, which is smooth in $\theta_i$ even though each sample's loss is not.

In the original paper, on Gemma 2 9B, JumpReLU SAEs reconstructed more faithfully than Gated SAEs and at least as well as TopK SAEs at the same sparsity, measured by the increase in the model's loss. They are the architecture of Gemma Scope, Google DeepMind's open suite of SAEs for its Gemma 2 models.

Two more designs complete the family. A Gated SAE (Rajamanoharan et al. 2024a) computes which latents are on in one path and how large they are in another, and puts the L1 penalty only on the first. A Matryoshka SAE (Bussmann et al. 2025) trains nested dictionaries so that the first few latents must reconstruct the input on their own, which pushes them toward general concepts. Matryoshka SAEs fight a failure called absorption that we meet below. The box has the details.

import numpy as np

def jumprelu_forward(Z, theta):
    return np.maximum(Z, 0)            # placeholder: a plain ReLU

def jumprelu_theta_grad(Z, theta, eps, lam, G_f):
    return np.zeros(Z.shape[1])

Compare SAEs at equal sparsity

Every SAE sits somewhere on a trade-off between sparsity (low L0) and fidelity (low FVU or low added loss), so no single number ranks two of them. Compare them at equal L0, or compare whole curves. The widget shows such curves for runs of each rule on the planted data of the cells above. The runs were trained when the book was written, with 3,000 Adam steps each, and the JumpReLU runs used the straight-through estimator just derived.

Before you look, predict two things. The data has 1.95 active features per token, so at an L0 of about 2, which rule should reconstruct best? And why should a rule that can vary its L0 from token to token have an edge over one that cannot? Then switch the dictionary from 128 to 512 latents and watch how many TopK latents are dead.

In an interview, the expected summary is one sentence. L1 couples "is it on" with "how big is it", and every later rule separates the two, by fixing the count (TopK), by learning a threshold (JumpReLU), or by a separate gate (Gated).

Dead latents

A latent is dead if it never fires. The gradient derivation showed why that state is permanent. A ReLU latent whose pre-activation is negative on every token gets zero gradient in its encoder column. A TopK latent that is never among the top $k$ is in the same position. Nothing ever moves it again. In practice a latent is declared dead if it has not fired on any of a long window of tokens, $10^7$ in Templeton et al. and Gao et al.

At large scale the problem is severe. Templeton et al.'s dictionaries of about 1, 4, and 34 million features ended training with roughly 2, 35, and 65 percent dead. Gao et al. saw up to 90 percent dead in experiments without countermeasures. The opposite case also exists. Dense latents fire on a large fraction of tokens, 10 percent or more by the usual cutoff. They were long treated as training artifacts, but Sun et al. (2025) found that many track real properties, such as the position of a token in the context or the part of speech.

Four cures are in use.

• Resampling (Bricken et al.). Every so often, find the latents that have not fired recently. Reset each one's decoder row to an input the SAE currently reconstructs badly, chosen with probability proportional to its squared error. Give it a small encoder norm, so the revived latent fires only weakly at first.
• Tied initialization (Conerly et al. 2024, Gao et al.). Start the encoder at the transpose of the decoder, $\mathbf{W}_{\text{enc}} = \mathbf{W}_{\text{dec}}^\top$. Each latent then initially detects its own direction and fires on the inputs that point along it. The two matrices are trained separately afterward.
• AuxK (Gao et al.). Add a loss that asks the dead latents to explain what the live ones missed, defined next.
• Gentler optimization. A learning rate that is too high kills latents (Bricken et al.). Conerly et al. raise the L1 coefficient linearly from zero over the first 5 percent of training. Together with tied initialization and the decoder-norm penalty, that left under 1 percent of their latents dead, so they stopped resampling. An earlier idea, the "ghost grads" of Jermyn and Templeton (2024), gave dead latents a separate gradient signal, and was dropped two months later when it no longer lowered the loss (March 2024 update).

In plain words, the dead latents are given the job of modeling what the live ones missed. A dead latent whose direction would help gets gradient, and its pre-activation rises. Eventually it wins a place in the main top $k$ and comes back to life. With tied initialization and AuxK, Gao et al.'s 16-million-latent SAE on GPT-4 ended with only 7 percent dead.

How much does initialization alone matter? The cell trains the TopK SAE from before again, with a random encoder instead of a tied one, and prints it next to the tied run.

With a random encoder, 28 percent of the latents never fire on the data. Recovery falls from 0.989 to 0.833, and FVU more than doubles, from 0.174 to 0.375. Only the initialization changed. A random encoder column is unrelated to its decoder row, so many latents never make it into the top $k$ during the first steps, and once out they stay out. In practice, the dead fraction is the first number to check when a run looks worse than expected. "How do you prevent dead latents?" is a standard interview question.

import numpy as np

def dead_mask(fire_counts):
    raise NotImplementedError

def auxk_loss(err, Z, W_dec, dead, k_aux):
    raise NotImplementedError

Measuring whether the dictionary is any good

A real model has no ground truth, so evaluation is a set of partial views. These are the ones every SAE paper reports.

Each metric can be gamed, the lesson of §22.1 in a new setting. L0 and FVU say nothing about whether the model's computation survives the reconstruction, which is why the spliced-in loss matters. The residual stream is the running sum that every layer of a Transformer reads from and writes to (§5.1). Gemma Scope's evaluation found that SAEs on the residual stream have a consistently higher delta loss than SAEs on the outputs of the attention and MLP sublayers. (An MLP sublayer is the two-layer feed-forward network inside each Transformer block, §5.3.) Their FVUs are roughly comparable. The authors conjecture that even small errors in reconstructing the residual stream can have a large effect on the loss.

Loss recovered against zero ablation flatters an SAE. Zeroing a residual stream destroys the model, so the denominator is huge, and almost any reconstruction looks like it recovers most of the loss. Mean ablation, which replaces the activation by its average, is a much harder baseline. The torch block below shows the gap on a small model, and Exercise 23.6f asks you to compute it.

Gao et al. proposed a more honest reading. Express the spliced-in loss as the pretraining compute a model would need to reach the same loss. Their 16-million-latent SAE, spliced into GPT-4, gave a loss equal to that of a model trained with about 10 percent of GPT-4's pretraining compute. That is a sobering way to say how much of the model the dictionary misses.

Spliced-in loss on a small model

The torch block trains a small classifier that reads a 16-dimensional stream holding 64 sparse features in superposition. It then trains a TopK SAE with AuxK on the stream. On a real model, the PyTorch hooks of §0.9 would do the splicing. Here the model is two functions, so we call them directly. The block reports the SAE's own metrics, then the classifier's cross-entropy on the original stream, on the reconstruction, and with the stream zero-ablated and mean-ablated.

import numpy as np

def loss_recovered(lp_orig, lp_sae, lp_abl):
    raise NotImplementedError

def compute_equivalent(ce, a, b, c):
    raise NotImplementedError

How dictionaries fail

The failures teach as much as the successes, and interviewers ask about them.

Feature splitting. As the dictionary grows, one latent becomes several more specific ones. Bricken et al. saw the number of latents specific to base64 text go from 1 in their 512-latent run to 3 in the 4,096-latent run, and many more at 16,384. Templeton et al. found that whether a concept gets its own latent at all depends on how often it appears in the training data. Claude 3 Sonnet could list every London borough, yet the 34-million-feature SAE had features for only about 60 percent of them. Splitting means there is no single right dictionary size, only a resolution.

Absorption is splitting with a hole in it. A researcher who trusts the "starts with S" latent as a detector is misled exactly on the tokens it absorbed. The cell builds such a hierarchy. There are four parent features, each with three children, and a child occurs only together with its parent. All 16 directions are orthogonal, so no interference is involved. The cell trains L1 SAEs with 16 latents, one per true feature.

Chanin et al. found absorption across hundreds of real SAEs on a first-letter task, and found that changing the size or the sparsity of the SAE did not remove it. Matryoshka training is one of the most direct fixes proposed so far.

Dark matter (Engels et al. 2024). The reconstruction error $\mathbf{x} - \tilde{\mathbf{x}}$ is not noise. About half of the error vector, and more than 90 percent of its norm, can be predicted linearly from the input activation. Larger SAEs mostly fail on the same tokens as smaller ones. So part of what the SAE misses is linear structure it could in principle learn. The authors suggest the rest, the part that is not linearly predictable, may be a different kind of thing that more latents will not fix.

Seed dependence (Paulo and Belrose 2025). Two SAEs trained on the same activations with different random seeds learn different dictionaries. In an SAE with about 131,000 latents on an MLP of Llama 3 8B, only 30 percent of the latents had a close counterpart in the other seed's dictionary. TopK SAEs were more seed-dependent than L1 SAEs. The authors draw the right conclusion. A dictionary is a useful decomposition of the activation space, not a list of the variables the model truly uses.

Errors that compound. An SAE spliced into one layer perturbs everything downstream, and SAEs at many layers at once compound their errors. That is why circuit work keeps an explicit error term at each site, and §23.8 makes those error terms nodes of the graph.

Transcoders and crosscoders

An SAE describes one activation. Circuits are about how activations at one place cause activations at another, and there an SAE on the output of an MLP sublayer has a problem. To see how a feature before the MLP produces a feature after it, you must pass through the MLP's thousands of neurons and their nonlinearities. The answer you get by patching or gradients then holds only for the one input you tried. Dunefsky, Chlenski, and Nanda (2024) proposed replacing the MLP itself.

The payoff is linearity. Suppose latent $i$ of a transcoder at layer $\ell$ is active with value $f_i$ and writes $f_i\mathbf{d}_i$ into the residual stream. Let $\mathbf{w}_j$ be the encoder column of latent $j$ in a later transcoder that reads the same stream (we write $\mathbf{w}$ here because $\mathbf{e}$ is the reconstruction error). Freeze the normalization in front of the later MLP at its value on this input, the convention of §23.3, and fold its scale into $\mathbf{w}_j$. Then, through the residual connection, the contribution of latent $i$ to latent $j$'s pre-activation is

In plain words, the effect of one feature on another is an activation that depends on the input, times a weight that does not. The weight $\mathbf{d}_i\cdot\mathbf{w}_j$ comes from the two transcoders alone, so it can be read once and holds on every input. Dunefsky et al. call this the factorization of a circuit into input-dependent and input-invariant terms. Summed over all sources, the contributions add up exactly to the later pre-activation, as the cell checks. Paths that pass through attention add a factor for the attention pattern, which §23.8 handles.

Their transcoders on models from 120 million to 1.4 billion parameters were at least on par with SAEs in sparsity, faithfulness, and interpretability. They used them to find new structure in the greater-than circuit of GPT-2 small (§23.4). A transcoder replaces one MLP with one wide sparse layer. The cross-layer transcoder of §23.8 lets each latent read at one layer and write to all later ones, and attribution graphs are built from it.

Two uses follow. Across layers, a feature that persists through the residual stream gets one latent instead of one per layer. Across models, a crosscoder on a base model and its chat-tuned version supports model diffing, finding what fine-tuning changed. Chat tuning is the supervised and reinforcement-learning fine-tuning that turns a base model into an assistant.

The recipe compares, for each latent, the norms of its two decoder rows. A latent with nearly all its norm in the chat model is a candidate for something chat tuning created. One with all its norm in the base model is a candidate for something it removed, and one with equal norms is shared. On the middle layer of Claude 3 Sonnet and its base model, a crosscoder with 1 million latents showed exactly these three clusters, with four to five thousand model-specific latents for each model.

"Model-specific" is a strong claim, though, and the sparsity penalty can manufacture it. Before trusting a chat-only latent, check how much it actually explains in each model, and prefer a crosscoder trained with BatchTopK, as the box explains.

A plain SAE can diff models too, and the most striking case so far concerns emergent misalignment. Emergent misalignment (§19.6) is the finding that fine-tuning a model on a narrow bad task, such as writing insecure code or giving subtly wrong advice, can make it broadly misaligned on unrelated prompts.

SAEs at frontier scale

Everything above was checked on small models whose features we planted. The same recipes have now run on production models, at sizes that change what an SAE costs and what it can find. Four reference points follow, in the order they were published. They run from a one-layer model to Claude 3 Sonnet and GPT-4, and then to an open suite for every layer of two Gemma 2 models.

Those dictionary sizes are easy to read past. A quick count of parameters shows why covering a whole frontier model this way is expensive.

The features themselves were the headline. Claude 3 Sonnet's dictionaries held features that were multilingual and multimodal, responding to the same concept in many languages and in images. They also held features relevant to safety, such as deception, sycophancy, bias, and dangerous content. The most famous is the Golden Gate Bridge feature 34M/31164353, which fires on mentions and images of the bridge. Templeton et al. clamped it, fixing its activation at 10 times its maximum on every token, and the model began to describe itself as the Golden Gate Bridge. Clamping is a form of steering. Section 23.7 takes it up properly, along with the evidence on whether SAE features steer better than simpler directions.

The open suites mean you can run a production SAE at home. The block below needs a machine with a GPU and access to Gemma 2 on Hugging Face. It loads one Gemma Scope SAE and encodes a prompt. The released weights have the training-time rescaling and pre-encoder subtraction folded in, so the code encodes the raw activation, and it uses the JumpReLU rule.

The latent indices it prints can be looked up on Neuronpedia, which hosts explanations and top activations for the Gemma Scope dictionaries.

What SAEs have and have not delivered

By 2025 there was enough evidence to judge SAEs against simpler tools, and much of it was negative. Most of it compares SAEs with linear probes, the classifiers of §23.2 trained to read a labeled concept off the activations.

• Kantamneni et al. (2025) tested probes built on SAE latents in four hard regimes, scarce data, class imbalance, label noise, and covariate shift. SAEs occasionally helped on individual datasets. But no ensemble that added SAEs consistently beat ensembles of the baselines alone. The apparent wins, finding spurious correlations, detecting bad labels, and multi-token probes, were matched by simple methods without SAEs.
• The Google DeepMind interpretability team (2025) asked whether SAE probes detect harmful user intent better out of distribution, on new jailbreaks. Dense linear probes on the raw residual stream performed nearly perfectly, even out of distribution. A single SAE latent used as a probe could not fit the training set. Probes on about 20 SAE latents fit it but generalized worse, and SAEs fine-tuned on chat data closed only about half the gap. Probes trained on the SAE reconstruction were worse than probes on the stream, so the SAE was discarding information about the concept. The team deprioritized fundamental SAE research while keeping SAEs as a tool. They noted one positive result, that SAE probes quickly exposed spurious correlations in their dataset.
• AxBench (Wu et al. 2025), a benchmark for concept detection and steering, found SAEs not competitive for either on Gemma 2 2B and 9B. Prompting was the best steering method, and the difference of class means the best detector. Section 23.7 has the details.

The positive evidence is about discovery, not detection.

• Marks et al. (2024) built sparse feature circuits from SAE latents and introduced SHIFT. It removes a classifier's reliance on a spurious signal (gender, in a profession classifier) by ablating the latents a human judges irrelevant to the task. It needs no examples that break the correlation.
• In the blind auditing game of Marks et al. (2025), a model was trained with a hidden objective, exploiting reward-model biases it had read about. Four teams investigated it. On one team allowed to use SAEs, a single researcher found the key training documents in 90 minutes. They did it by inspecting the SAE features active on a prompt about the "potentially concerning" behaviors. A second SAE team also succeeded, and so did a team barred from SAEs that used only black-box queries plus the training data. The team with API access and no data did not. The authors note that the fastest team's search could also have been done with a simpler search over text embeddings. SAEs helped, and they were not the only route.
• Wang et al.'s persona latents, above, explained a surprising generalization and gave an early-warning signal.
• Anthropic has reported using SAE features in pre-deployment audits of its models, which §23.8 describes.

In practice the two tools combine. Use an SAE to generate hypotheses about what a model represents. Then train a probe on labeled data, validate it on held-out distributions, and deploy the probe as the monitor.

Read more

• Bricken et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. The detailed case for SAEs on a one-layer model, with the training recipe (pre-encoder bias, unit-norm decoder with the gradient projection, resampling) and feature splitting.
• Templeton et al. (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. SAEs with up to 34 million features on a production model, the decoder-norm penalty, scaling laws, feature completeness, and the Golden Gate Bridge feature.
• Gao et al. (2024). Scaling and evaluating sparse autoencoders. TopK SAEs, tied initialization and AuxK against dead latents, scaling laws in latents and sparsity, and the compute-equivalent reading of spliced-in loss.
• Rajamanoharan et al. (2024). Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders. The JumpReLU rule and the straight-through estimator as kernel density estimation of the expected loss's gradient.
• Lieberum et al. (2024). Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2. The open JumpReLU suite, its training details, and evaluation by delta loss and FVU across sites.
• Dunefsky, Chlenski, Nanda (2024). Transcoders Find Interpretable LLM Feature Circuits. Transcoders and the factorization of circuits into input-dependent and input-invariant terms.
• Chanin et al. (2024). A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders. Why sparsity makes general latents fail on specific cases, measured on hundreds of real SAEs.
• Google DeepMind mechanistic interpretability team (2025). Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2). Probes versus SAEs out of distribution, frequency-weighted autointerp, and an honest strategic update.

Watch next

▶

▶

▶