Imagine watching a photograph slowly dissolve into television static — the colors bleeding, the shapes blurring, the detail vanishing until nothing remains but noise. Now imagine teaching a computer to play that film in reverse, so it can start with static and conjure a photograph from nothing.
That, in essence, is a diffusion model. It is the technology behind image generators like Stable Diffusion, DALL-E, and Midjourney. And while the mathematics underlying it is genuinely sophisticated, the core idea is something you can understand without writing a single equation.
The Forward Process: Destroying Information
Every diffusion model begins its life by practicing destruction. During training, the model takes a clean image — a photograph of a cat, say — and gradually adds noise to it over many small steps. At step one, the image looks mostly like a cat with a faint haze. By step fifty, the cat is barely visible. By step one thousand, the image is pure, unstructured static — indistinguishable from the random noise you'd see on an old analog TV tuned to a dead channel.
This gradual destruction is called the forward diffusion process. It is mathematically well-defined: at each step, a small amount of Gaussian noise (random static following a bell-curve distribution) is added to every pixel. The process is also reversible in principle — if you knew exactly what noise was added at each step, you could subtract it all back and recover the original image.
Here is the crucial insight: the forward process is a Markov chain. Each step depends only on the previous one. The model doesn't need to remember the entire history — it only needs to understand the transition from one state to the next. This simplicity is what makes the whole approach tractable.
The Reverse Process: Learning to Reconstruct
Now comes the clever part. A diffusion model is trained to reverse the forward process. Given a noisy image, the model learns to predict what the image looked like one step earlier — slightly less noisy. Do this over and over, starting from pure noise, and the model can generate an entirely new image.
The model never learns to generate an image in one shot. It learns a single, simple skill: "given a noisy image, remove a little bit of noise." Stitch together a thousand applications of that skill, and you've gone from static to photograph.
Think of it like a restorer cleaning an old painting. They don't remove centuries of grime in a single wipe. They work patiently, removing a thin layer at a time, and at each step they make an educated guess about what the paint underneath should look like. A diffusion model does the same thing, but with noise instead of grime, and with a neural network making each guess.
Where Does the "Diffusion" Come From?
The name "diffusion model" is not metaphorical — it is literal. The same mathematics that describes how ink spreads through water also describes how the forward process adds noise to an image. In physics, Fick's laws of diffusion describe how particles move from regions of high concentration to low. In a diffusion model, the "concentration" is the structure in the image, and it gradually diffuses into uniform noise.
This connection is more than a naming convention. The Fokker-Planck equation, a cornerstone of statistical physics, appears directly in the mathematical formulation of diffusion models. The researchers who developed these models — Jascha Sohl-Dickstein at Stanford, and later Jonathan Ho and colleagues at Google — drew explicitly on the physics literature. The connection runs deep, and we explore it fully in our article on the connection between physical and AI diffusion.
How Does the Model Know What to Generate?
If the model just removes noise, why does it produce a cat instead of a dog or a landscape? The answer is conditioning. The model is trained on millions of labeled images, and when you ask it to generate an image, you provide a text prompt that guides each denoising step.
Think of it as giving the restorer a description of the painting they're uncovering. At each step, as they remove noise, they steer their guess toward "something that looks like a cat." The text prompt acts as a compass, gently pulling the denoising process toward images that match the description. Without this conditioning, the model would still generate images — but they'd be random samples from whatever data it was trained on, not specific subjects you requested.
Why So Many Steps?
Early diffusion models required a thousand or more denoising steps to produce an image. Each step involves running the neural network once, so generating a single image could take minutes. Modern techniques have reduced this dramatically — Stable Diffusion can produce good results in 20 to 50 steps, and newer methods like DPM-Solver can do it in as few as ten.
The reason many steps are needed is that each individual denoising guess is imperfect. If the model tries to remove too much noise in one step, it has to make a bigger guess, and bigger guesses are harder to get right. By taking many small steps, each guess is small and manageable, and errors don't compound catastrophically. It's the difference between carving a sculpture with a chisel and trying to do it with a sledgehammer.
The Latent Space Shortcut
Stable Diffusion made a critical improvement over earlier diffusion models: instead of diffusing and denoising in pixel space (where each image is thousands of pixels wide and tall), it works in a compressed latent space. A separate encoder first compresses the image into a much smaller representation, the diffusion happens in that compressed space, and then a decoder expands the result back into pixels.
This is why Stable Diffusion can run on consumer graphics cards while earlier models required massive data center GPUs. We cover this architecture in detail in our explainer on how Stable Diffusion works.
Why Diffusion Models Matter
Diffusion models didn't invent AI image generation — GANs (generative adversarial networks) came before them. But diffusion models offered two enormous advantages: they produce more diverse and higher-quality images, and they are dramatically more stable to train. GANs were notorious for "mode collapse," where the generator would find a few images that fooled the discriminator and produce them endlessly. Diffusion models don't have this problem.
The trade-off is speed: GANs generate images in a single forward pass, while diffusion models require many steps. But as sampling techniques improve, that gap is closing. And the quality advantage of diffusion models has made them the dominant approach for image generation since 2022.
A Process Older Than AI
What makes diffusion models so elegant is that they repurpose one of the oldest ideas in physics. Diffusion — the spreading of particles from high concentration to low — was first described mathematically by Adolf Fick in 1855. The same equations that govern a drop of ink spreading through water now power the AI that generates art, edits photographs, and designs molecules. The technology feels futuristic, but its mathematical roots are older than the light bulb.
If you'd like to go deeper, start with our explainers on how Stable Diffusion works in practice and the denoising process in detail. Or, if you're curious about the physics side of the story, read about Brownian motion — the random dance that first revealed the atomic nature of matter.