∇
Made for larger screens
Open this page on a tablet or computer.
Home
Book
∇
Home
Book
Notes
Exercises
Search
⌘K
Sign in
The book
Chapters
0
The Toolkit
›
1
Embeddings
›
2
Tokenization
›
3
From Static to Contextual Meaning
›
4
Attention from First Principles
›
5
The Transformer Block
›
6
Build a GPT from Scratch
›
7
Encoders
›
8
Pretraining Recipes
›
9
Scaling Laws
›
10
Distributed Training
›
11
GPU Programming for ML
›
12
FlashAttention and Efficient Attention
›
13
Mixture of Experts
›
14
Post-training and Supervised Fine-tuning
›
15
Reinforcement Learning Foundations for LLMs
›
16
Reinforcement Learning from Human Feedback
›
17
Direct Preference Optimization
›
18
Reasoning Models and RL with Verifiable Rewards
›
19
Alignment and Safety
›
20
Fast
›
21
State-Space and Linear-Time Models
›
22
Evaluation
›
23
Interpretability
›
24
Tool Use and Agents
›
25
Memory and Long-Running Agents
›
26
Self-Improvement Loops
›
27
Images as Tokens
›
28
Self-supervised Vision
›
29
Video Representations
›
30
Joint Text-Image Embeddings
›
31
Vision-Language Models
›
32
Autoencoders
›
33
Diffusion and Flow Matching
›
34
Image and Video Generation at Scale
›
35
How Sound Becomes Vectors
›
36
Speech to Text
›
37
Text to Speech
›
‹
All chapters
Chapter 15
Reinforcement Learning Foundations for LLMs
15.1
The reinforcement learning problem
15.2
Bellman equations and dynamic programming
15.3
Policy gradients and REINFORCE
15.4
Baselines, advantages, and actor-critic
15.5
Generalized advantage estimation
15.6
Trust regions and PPO
15.7
Language models as policies