weatherlinguist

When AI models miss the butterfly effect

I keep hearing "the model is learning the physics" from machine learning people talking about AI weather models. What they usually mean is a specific kind of model: the generative, transformer-based emulators trained end-to-end on reanalysis data, no dynamical core, no equations, just a network fit to the data. GraphCast, Pangu, GenCast, and the generative-data-assimilation models coming out of places like NVIDIA (Manshausen et al.'s model is one example, not the point) all fall in that bucket. That's the question I actually want to answer for myself, and it's narrower than "do AI weather models learn physics" as a blanket claim, because some AI weather models explicitly aren't in that bucket. NeuralGCM (Kochkov et al. 2024) wires an actual differentiable dynamical core into the model and only learns the subgrid closure. Caltech, MIT, and JPL's CliMA earth-system model, still in development, is built the same way: physics as far as it can go, learning only where it can't. Those get to claim "physics" by construction, that's not in dispute. My question is about the ones that don't have a dynamical core anywhere in them: does a purely data-driven network implicitly learn physics just from being trained on physically generated data?

This is where Edward Lorenz's butterfly effect actually belongs in the story, and it's the reason for the title. In 1963 Lorenz ran a toy weather model twice from almost-identical starting conditions, differing only in rounding error, and got wildly different forecasts within days: the errors didn't just persist, they grew, faster the bigger they got, because a chaotic system keeps feeding its small-scale disturbances back onto the large-scale flow. That accelerating growth isn't a bug in the atmosphere, it's the mechanism that makes weather unpredictable past a couple of weeks no matter how good the model is. So the question gets sharper: not just whether these physics-free models learn physics in some general sense, but specifically whether they reproduce that mechanism, or whether they've learned something that looks skillful while quietly missing the butterfly.

I do not believe it does. The denoising and noising procedure in a generative model is not akin to learning physics, it is akin to imitating 2D snapshots of what is going on (in the case of the NVIDIA model above).

I found some backing for this in Hassanzadeh et al. 2026 (arXiv:2608.25835). They actually measure it, rather than leaving it as a hunch: they track ensemble spread (they call it DKE, difference kinetic energy) across ERA5-trained transformers, a GCM emulator, Lorenz 96, and Pangu-Weather, none of which have a dynamical core, and compare it to the real numerical models (ICON, the GCM itself). In a real atmosphere, or in a real NWP model, two initial states that are almost identical diverge fast, and the divergence accelerates as it grows, that's the whole point of chaos: small errors don't just grow, they grow faster the bigger they get. Every numerical model in their comparison shows exactly that: fast, accelerating growth that saturates. Every physics-free model in their comparison, generative or not, shows something else instead: slow, flat exponential growth, almost like a fixed decay rate. So the butterfly effect really is missing, across this whole category, not just in the generative NVIDIA model I picked on above. That part of my hunch holds up.

butterfly_schematic The shape that matters, stripped of all the real data below: a real chaotic system's error curve bends upward, growing faster as it grows (the butterfly effect). A physics-free AI model's error curve keeps the same growth rate throughout, a straight line on this log scale, never accelerating. Everything that follows is evidence for which shape each model actually produces.

Where the paper made me rethink things is why. My story was: the denoising, the generative sampling is just glorified image synthesis, of course it doesn't have real dynamics. But the paper's cleanest experiment has nothing to do with denoising at all. They train twelve plain regression networks (same architecture, same MSE loss, no diffusion in sight) on the two-scale Lorenz 96 system, and vary only one thing: how much of the fast, small-scale variability survives in the training data. That one knob is enough to turn the butterfly effect on and off. More complete data, closer to the real multi-scale system, and the butterfly-like growth shows up; coarsen the data and it vanishes, along with a chunk of the model's apparent skill. So it isn't the generative machinery that's the culprit, it's what got averaged out of the training data before the model ever saw it. A boring MSE-trained network has exactly the same problem. That's the direct link back to the butterfly effect: accelerating, chaotic error growth is generated by exactly the fast, small-scale coupling that this knob controls, so a model trained on data with that coupling averaged out never had the mechanism to reproduce it in the first place, no matter how well it fits everything else. That takes some of the wind out of picking on the NVIDIA model specifically, and out of picking on "generative" models as a subcategory, it's a property of physics-free, end-to-end training in general.

It also made me soften "not learning physics" a bit. What the paper argues these models pick up instead is something like an implicit subgrid parameterization: the averaged effect that the fast scales have on the slow ones, without the fast scales themselves and their error growth along for the ride. That's not nothing. It's basically the same move a GCM's convection scheme makes below its own grid box, except learned implicitly instead of designed. So the honest version of my complaint isn't "there's no physics in there," it's "there's a coarse, homogenized physics in there, and it's specifically missing the multi-scale error growth that produces chaos." Different claim than what I originally wrote.

Two more things worth mentioning:

One more thing worth flagging, since it's adjacent rather than direct evidence: OpenAI announced in September 2026 that one of their models produced a proof, formally verified in Lean, that the idealized three-dimensional Navier-Stokes equations can develop a finite-time singularity, a vortex that spins up and collapses in finite time from smooth initial data. That's not evidence about chaos or about turbulence closure specifically, and it says nothing about real atmospheric flows, which have viscosity, rotation, and stratification that the idealized inviscid limit doesn't. But it's a useful reminder that "the physics" isn't a complete, closed system that a network is failing to learn: even the equations we'd treat as ground truth have open gaps at the smallest scales. Closure at some scale is unavoidable in principle, not just an engineering shortcut, which is part of why every model in this post, learned or hand-built, has to draw that line somewhere.

The Chen et al. paper is worth a paragraph of its own, because it ran the same DKE test across four current probabilistic ensemble models (NeuralGCM-ENS, FourCastNet 3, AIFS-ENS, GenCast) against ECMWF's real ensemble, IFS-ENS, and split the results by scale. All four models do show upscale error growth at large, synoptic scales, DKE peak shifting toward larger wavelengths over time, roughly tracking IFS-ENS. It's specifically at small scales where the physics-free ones come apart from the real atmosphere, and they come apart in different ways depending on how each model generates its ensemble spread. AIFS-ENS and GenCast, both purely data-driven and both sampling spatially uncorrelated Gaussian noise per member, show the opposite of a cascade: kinetic energy piles up at high wavenumbers instead of transferring to larger scales. FourCastNet 3, also purely data-driven but generating its stochasticity from a spatially- and temporally-correlated diffusion field instead, doesn't show that pileup. The interesting bit is that AIFS-ENS isn't diffusion-based at all, it's a transformer trained with a CRPS loss, and it still fails the same way GenCast does. So the shared failure isn't about being generative or doing denoising, it's the specific choice of spatially uncorrelated noise as the source of ensemble spread. Change that one design choice and the small-scale pathology changes shape, independent of the training-data averaging argument above.

So, to answer the question I started with: no, I don't think a physics-free, end-to-end generative or transformer weather emulator is learning physics in any meaningful sense. But it's not because of the denoising specifically, that part of my hunch didn't survive contact with the evidence. It's that every current model in this category is trained on data that has had the fast scales averaged out of it, and that averaging is exactly what's buying the forecast skill and exactly what's costing the butterfly effect. On top of that, for the ensemble versions specifically, there's a second, independent knob: how the ensemble spread itself gets generated. Spatially uncorrelated noise is its own way to get the small-scale physics wrong, piling energy up instead of cascading it, on top of whatever the training data already averaged away. None of this touches NeuralGCM or CliMA, they were never trying to answer this question the hard way, they just wired the physics in directly.

References