---
title: Magic Matched DeepSeek V4 Pro Base With 50x Fewer FLOPs
tags: [AI Systems, Efficiency, Pretraining, Benchmarks]
date: 2026-09-10
---

Magic's latest pretraining run hit DeepSeek V4 Pro Base's eval numbers using about 50x fewer FLOPs to get there. On some held-out eval sets the efficiency gap runs as high as 127x.

## What actually happened

Magic measures pretraining quality with bits-per-byte loss on heldout data, lower is better. Their V5 e24 model matched DeepSeek V4 Pro Base on that metric using roughly 1.63x10^24 FLOPs against DeepSeek's 9.67x10^24, a run that cost about $0.5M on GB200 hardware, in the same ballpark as GPT-3's original training bill. Scale the same recipe to 10x that compute, about $4M, and it beat every public open-weight base model on perplexity evals.

Break the efficiency down by eval category and it's not flat: 29-48x on private code, 24-45x on research papers, 15-127x on math reasoning. That spread is the actual finding. A single average multiplier would suggest one clean trick. The spread says the gains compound differently depending on what you're testing, which is what you'd expect if the source isn't one algorithm.

And it isn't. Magic is explicit that there's no single technique behind the number. It's "tens of changes across model architecture, optimizer, training objective, and data curation," stacked.

## Why that matches what I've seen in my own systems

I've heard this before, just at a different layer. My matching engine's 3.31x throughput jump didn't come from one rewrite. It came from cache-line alignment, lock-free SPSC queues replacing mutex contention, branch prediction hints, and memory layout changes, applied one at a time and measured after each. None of those individually would headline a blog post. Stacked, they moved p99 latency from microseconds to 900 nanoseconds.

The instinct to look for the one big algorithmic win is usually wrong once a system is past its first working version. Early gains come from picking the right algorithm. Later gains come from refusing to accept any single source of overhead, including the ones too small to be worth a slide. Magic's post reads like the pretraining-scale version of that same discipline: nobody wants to write "we made 30 small things each 5-10% better," but that's apparently what a 50x compute multiplier looks like when you unpack it.

## The takeaway

Compute efficiency is turning into the actual moat, not parameter count. If a $0.5M training run can match a model that reportedly cost an order of magnitude more, the barrier to competing at the frontier isn't capital anymore, it's whether your team does the unglamorous work of stacking dozens of small wins instead of waiting for one big one. That applies whether you're pretraining a language model or optimizing an order book. Look for the boring 5% improvements. They're the ones that compound.

More on how I chase the boring 5% improvements in my own systems: [github.com/saksham10arora-dotcom](https://github.com/saksham10arora-dotcom).