Magic’s latest pretraining run hit DeepSeek V4 Pro Base’s eval numbers using about 50x fewer FLOPs to get there. On some held-out eval sets the efficiency gap runs as high as 127x.
Where the 50x comes from
Magic measures pretraining quality with bits-per-byte loss on heldout data, lower is better. Their V5 e24 model matched DeepSeek V4 Pro Base on that metric using roughly 1.63x10^24 FLOPs against DeepSeek’s 9.67x10^24, a run that cost about 4M, and it beat every public open-weight base model on perplexity evals.
Break the efficiency down by eval category and it’s not flat: 29-48x on private code, 24-45x on research papers, 15-127x on math reasoning. That spread is the actual finding. A single average multiplier would suggest one clean trick. The spread says the gains compound differently depending on what you’re testing, which is what you’d expect if the source isn’t one algorithm.
And it isn’t. Magic is explicit that there’s no single technique behind the number. It’s “tens of changes across model architecture, optimizer, training objective, and data curation,” stacked.
Small wins, stacked
The instinct to look for the one big algorithmic win is usually wrong once a system is past its first working version. Early gains come from picking the right algorithm. Later gains come from refusing to accept any single source of overhead, including the ones too small to be worth a slide. Magic’s post reads like the pretraining-scale version of that: nobody wants to write “we made 30 small things each 5-10% better,” but that’s apparently what a 50x compute multiplier looks like when you unpack it.
Efficiency is the moat now
Compute efficiency is turning into the moat, more than parameter count. If a $0.5M training run can match a model that reportedly cost an order of magnitude more, the barrier to competing at the frontier isn’t capital anymore, it’s whether your team does the unglamorous work of stacking dozens of small wins instead of waiting for one big one. Look for the boring 5% improvements. They’re the ones that compound.