weatherlinguist

From weather analogs to diffusion models

At the end of 2022 you could count the machine learning weather models on the fingers of one hand: FourCastNet, Keisler's graph network, Pangu-Weather and GraphCast. They had a lot in common. All of them were global, all were trained on ERA5 (ECMWF's reconstruction of the weather since 1940), all were deterministic (one input, one forecast), and all borrowed their architecture from computer vision or graph learning: vision transformers, Fourier neural operators, graph neural networks.

Four years later there are too many to count. There are regional models, ensemble models, nowcasting models, models that go straight from observations to forecasts, and people now talk about foundation models of the atmosphere: large models trained once on many kinds of data and then adapted to different tasks. Microsoft's Aurora is one example. Another is WeatherGenerator, an EU-funded project coordinated by ECMWF that is building a machine learning Earth system model trained on reanalyses, forecast data and observations (https://github.com/ecmwf/WeatherGenerator). For a sense of scale, see this curated list: https://github.com/shengchaochen82/Awesome-Foundation-Models-for-Weather-and-Climate.

timeline A selection of machine learning weather models by year of first preprint, coloured by type

With so many names around it is easy to lose track of what actually differs between them, so I want to write a short refresher on the fundamentals. I start with diffusion models. Not because they came first (they did not), but because the idea behind them is the easiest to picture.

One clarification first. "Diffusion", "transformer" and "graph network" are not three competing kinds of model. Transformers and graph networks are architectures: the shape of the network. Diffusion is a way of training and using a network: instead of predicting one answer, it learns to draw samples from a distribution. Many diffusion weather models have a transformer or a graph network inside; GenCast, for example, is a diffusion model built on a graph transformer. The more useful split is between models that predict the single most likely next state and models that learn what the weather can look like and draw from it. This post is about the second kind, and it starts from an idea forecasters have used for decades: look at what the weather has done before.

In this post I go from that old idea, through the image generators, to a recent paper by an NVIDIA team that uses a diffusion model to draw a full km-scale weather map from a handful of weather stations [1].

The oldest trick: find a day like today

Long before machine learning, forecasters asked a simple question: when did the weather last look like this, and what happened next? This is the analog method. Lorenz used naturally occurring analogs in 1969 to estimate how predictable the atmosphere is [2], and analog methods are still used today to correct model output, for example by searching past forecasts for ones that resemble today's and looking at what was actually observed [3].

Analogs have a hard limit. The atmosphere almost never repeats itself, so the closest past day is usually not that close. And you can only ever get back something that already happened.

Learning the weather from past data instead of solving the equations is not a new ambition either. In the 1950s statistical forecasting was a major research topic, and some forecasters believed that linear regression on today's weather would do at least as well as numerical models ever could [4]. Lorenz, put in charge of a statistical forecasting project at MIT, doubted it. He tested the idea on a small model of the atmosphere, 12 equations run on an early desk computer: the solutions never repeated themselves, and regression could only reproduce them poorly [4, 5]. While rerunning one of those simulations in 1961 from slightly rounded numbers, he got a completely different result, and that is how chaos was found. Physics won the first round, partly because the statistical tools of the time were linear and the data and computers were small.

Today's machine learning models are, in a sense, the second round. A neural network is a statistical model too, but a very nonlinear one, trained on decades of reanalysis on hardware nobody had in the 1950s. And the generative models in this post go one step further than predicting one number from others.

Now imagine instead that you could look at every past weather map at once, learn what weather in general looks like (how fronts curve, how rain cells cluster, how wind changes around a thunderstorm), and then draw a new map that never happened but could have. That is what a generative model does. It does not retrieve a past situation. It learns the distribution of past situations and samples from it.

How diffusion came to be

The method behind Stable Diffusion is called diffusion, and its history is short:

The idea in plain words:

  1. Take a real picture.
  2. Add some random noise to it, a little or a lot.
  3. Train a network to recover the clean picture from the noisy one.
  4. Repeat for millions of pictures and every amount of noise.

noising A synthetic weather-like field with increasing amounts of noise added. Training goes left to right, generating goes right to left.

The network only ever learns to denoise. But to remove noise well it has to know what real pictures look like, which shapes belong and which are noise. To make a new picture you start from pure noise and denoise in small steps, a few dozen times. Each step uses the network to nudge the picture towards "more like real data". A different starting noise gives a different picture.

To control what gets drawn, you give the network extra input while it denoises. In Stable Diffusion that input is your text prompt. In weather it can be a coarse global forecast, the current state of the atmosphere, or, as we will see, a list of weather station readings.

From pictures to weather maps

To a neural network a weather field looks a lot like an image: a grid of numbers. Temperature, wind components and precipitation are just channels, like red, green and blue, except there may be dozens of them. So it was natural to try the image machinery on weather, and from 2023 it took off:

Why bother, when deterministic models like GraphCast are already so good? Because of blur. A deterministic model trained to minimise squared error learns the average of all the futures that could happen. Averages are smooth: storms turn into blobs, peak winds and rain get underestimated, and it gets worse with lead time. A diffusion model does not average. Each sample is sharp, and if you want the uncertainty, you draw many samples and get an ensemble (a set of possible forecasts whose spread shows the uncertainty).

blur Toy example: four possible futures with a sharp rain cell in different places, and their average, where the cell is smeared out and its peak halve

Diffusion is not the only way out of the blur, though. Another route is to keep a model that produces a forecast in one pass, feed it some random noise so every run is different, and train it with a score that rewards a good ensemble instead of a good average, such as the continuous ranked probability score (CRPS). ECMWF's AIFS-CRPS [14] and Google DeepMind's FGN [15] work this way and also give sharp, skilful ensembles, at a fraction of the sampling cost. More on that in a later post.

The showcase: drawing a weather map from 40 stations

This is the paper I really want to talk about. Manshausen et al. (2025) [1] use diffusion not to forecast but for the step before the forecast: data assimilation.

To be clear from the start: nothing in this paper forecasts. The output is a 2D snapshot of the surface weather at one moment in time, the best guess of what the weather is right now given the stations. There is no time evolution in it. Moving that snapshot forward is a separate job for a forecast model (more on that in the Earth2Studio section below).

Every forecast needs an estimate of the current state of the atmosphere, called the analysis. Producing it means combining a short forecast with whatever observations are available, weighting each by how much you trust it. In operational regional models such as HRRR (NOAA's hourly 3 km model for the US) this is expensive and carefully tuned machinery.

The question in the paper: over a 384 x 384 km box in Oklahoma there are about 50 weather stations reporting wind and precipitation. The grid has 3 km spacing, so 128 x 128 = 16384 points per variable. Can you go from ~40 station readings to a realistic map of surface wind and rain?

You have far fewer numbers than unknowns, so the only way to fill the gaps is to know what weather usually looks like. Classical data assimilation writes this knowledge down by hand, as a background error covariance, and assumes it is Gaussian. The paper replaces that with a diffusion model that has learned it.

stations Toy example with a simple Gaussian prior, not the paper's model: a hidden field, the 40 stations that sample it (crosses are held-out stations), and two different maps that both agree with the stations

The figure shows the problem with a toy version. Many different maps agree with the same stations. Which ones you get depends entirely on what the model believes weather looks like between the stations. Here that belief is a simple smooth Gaussian; in the paper it is a diffusion model trained on millions of real maps, so the gaps get filled with fronts, rain cells and gust fronts instead of smooth blobs.

step 1: learn what the weather looks like

They train a diffusion model, with the EDM recipe, on about 2.5 million HRRR snapshots of three fields: the two surface wind components and precipitation. No prompt, no conditioning: the model just learns to generate plausible km-scale Oklahoma weather.

step 2: prompt it with the stations

If you have used the "fill in the missing part" tool of a photo editor, you already know the idea. Data assimilation of this kind is very close to inpainting: you know a few pixels (the stations) and ask the model to complete the rest of the picture so that it looks real and matches the pixels you know [16]. Diffusion models were already doing this with photos of bedrooms and faces in 2021 [8].

This is also where the Stable Diffusion analogy is closest. Instead of a text prompt, the prompt is the station readings. At every denoising step the map is pulled in two directions at once: towards "look like real weather", which is what the trained model knows, and towards "agree with the stations", which is added during sampling. The model guesses the clean map, checks it against the stations, and nudges the sample to reduce the mismatch. After a few dozen steps you have a map that is both realistic and consistent with the observations.

For the technically curious: the maths

This is Bayes' rule written in terms of scores:

grad log p(x | y) = grad log p(x) + grad log p(y | x)

where x is the full map and y the station readings. The first term is the learned score of the unconditional model. The second is the likelihood of the observations. If the observation operator A simply picks the grid points where the stations are, and the observation errors are Gaussian with standard deviation sigma, it is

grad log p(y | x) = A^T (y - A x) / sigma^2

that is, the mismatch at the stations spread back onto the grid. During sampling x is noisy, so the mismatch is computed on the denoiser's estimate of the clean map instead. This is score-based data assimilation [16], with the likelihood approximated as in diffusion posterior sampling [17].

The nice part: the model is trained once, without any observations. You can change which stations you use, or add new ones, without retraining. Just as Stable Diffusion was never trained on your particular prompt.

What they found

They tested on all of 2017, assimilating 40 stations and keeping 10 aside to check the result [1]:

And what does not work yet [1]:

Dense or sparse: the spread tells you

Because you get an ensemble and not a single map, you also get a map of the uncertainty: where the members disagree, the stations did not pin the weather down. Close to a station the members agree; far from all stations they are free to differ. With more stations the whole map tightens.

spread Toy example with the same Gaussian prior as above: ensemble spread with 40 stations is large away from the stations; with 400 stations it is small almost everywhere

What Earth2Studio actually ships

NVIDIA's Earth2Studio [18] is not a single model but a framework that runs many, deterministic and generative. If you look in it for this paper, what you find is StormCastSDA [19], and it is not quite the same thing. It is the combination the paper's outlook asks for: a diffusion forecast model with diffusion data assimilation attached. Instead of starting from nothing every time, it takes the current weather, forecasts one hour ahead, and lets whatever stations are reporting nudge that forecast while it is being drawn.

For the technically curious: how StormCastSDA works

From the code (commit 95f74e6, August 2026):

So the paper is the proof of concept, and Earth2Studio turns it into a forecast where every hourly step is nudged by whatever stations are reporting. The documentation flags the data assimilation models as new, with APIs that may change [20].

Where the picture analogy breaks

Weather maps are not cat pictures, and the differences matter:

From remembering to imagining

The analog forecaster asked: when did the weather last look like this? A diffusion model asks something slightly different: given what I can see right now, what could the weather look like? It answers with many maps, not one, each of them sharp and each of them plausible. The Manshausen paper shows how far that idea goes with very little: three surface variables, one small box in Oklahoma, forty stations, and a model that was never shown a single observation during training. What I find most interesting is not the 10% over HRRR but the last part of that sentence. A model that learns what the weather looks like once, and can then be steered by whatever measurements you happen to have, is something anyone with their own sensor network (a wind farm, a city, an insurer with hail sensors) could use. It is worth repeating what it is not: it tells you what the weather is now, not what it will be. And it is not finished yet. The ensembles are too confident, the rain is too light, and nobody has shown it holds up in the extremes that matter most. But the old idea of learning from past weather has come a long way since Lorenz went looking for analogs, and it is worth watching where it goes next.

References

[1] Manshausen, P., Cohen, Y., Harrington, P., Pathak, J., Pritchard, M., Garg, P., Mardani, M., Kashinath, K., Byrne, S. and Brenowitz, N. (2025). Generative data assimilation of sparse weather station observations at kilometer scales. Journal of Advances in Modeling Earth Systems. https://doi.org/10.1029/2024MS004505

[2] Lorenz, E. N. (1969). Atmospheric predictability as revealed by naturally occurring analogues. Journal of the Atmospheric Sciences, 26, 636-646.

[3] Delle Monache, L., Nipen, T., Liu, Y., Roux, G. and Stull, R. (2011). Kalman filter and analog schemes to postprocess numerical weather predictions. Monthly Weather Review, 139, 3554-3570.

[4] Emanuel, K. (2011). Edward Norton Lorenz 1917-2008. Biographical Memoirs, National Academy of Sciences. https://texmex.mit.edu/pub/emanuel/PAPERS/Lorenz_Edward.pdf

[5] Lorenz, E. N. (1962). The statistical prediction of solutions of dynamic equations. Proceedings of the International Symposium on Numerical Weather Prediction, Tokyo (November 1960), 629-635.

[6] Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N. and Ganguli, S. (2015). Deep unsupervised learning using nonequilibrium thermodynamics. ICML 2015. arXiv:1503.03585.

[7] Ho, J., Jain, A. and Abbeel, P. (2020). Denoising diffusion probabilistic models. NeurIPS 2020. arXiv:2006.11239.

[8] Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S. and Poole, B. (2021). Score-based generative modeling through stochastic differential equations. ICLR 2021. arXiv:2011.13456.

[9] Rombach, R., Blattmann, A., Lorenz, D., Esser, P. and Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. CVPR 2022. arXiv:2112.10752.

[10] Karras, T., Aittala, M., Aila, T. and Laine, S. (2022). Elucidating the design space of diffusion-based generative models. NeurIPS 2022. arXiv:2206.00364.

[11] Mardani, M. et al. (2025). Residual corrective diffusion modeling for km-scale atmospheric downscaling. Communications Earth & Environment. arXiv:2309.15214.

[12] Price, I. et al. (2025). Probabilistic weather forecasting with machine learning. Nature, 637, 84-90.

[13] Pathak, J. et al. (2024). Kilometer-scale convection allowing model emulation using generative diffusion modeling. arXiv:2408.10958.

[14] Lang, S. et al. (2024). AIFS-CRPS: Ensemble forecasting using a model trained with a loss function based on the Continuous Ranked Probability Score. arXiv:2412.15832.

[15] Alet, F. et al. (2025). Skillful joint probabilistic weather forecasting from marginals. arXiv:2506.10772.

[16] Rozet, F. and Louppe, G. (2023). Score-based data assimilation. NeurIPS 2023. arXiv:2306.10574.

[17] Chung, H., Kim, J., McCann, M. T., Klasky, M. L. and Ye, J. C. (2023). Diffusion posterior sampling for general noisy inverse problems. ICLR 2023. arXiv:2209.14687.

[18] NVIDIA Earth2Studio. https://github.com/NVIDIA/earth2studio

[19] Earth2Studio, StormCastSDA, source earth2studio/models/da/sda_stormcast.py, commit 95f74e6 (15 August 2026).

[20] Earth2Studio data assimilation models documentation. https://nvidia.github.io/earth2studio/modules/models_da.html