Charts like this, from your own numbers.
Connect a spreadsheet or the tools you run on, and Parity builds the dashboard and writes the report, with every number checked.
Try Parity freeThe first row of Epoch AI's table of notable AI models, sorted newest first, is Gemini 4 Carbon, announced by Google DeepMind on October 28, 2026. Its training-compute cell is empty. The last row is Theseus, a maze-solving mechanical mouse that Claude Shannon built at Bell Laboratories in 1950, and its cell is filled: Epoch puts its "training" at 4×10¹ operations, counting each relay switch the mouse flipped while it bumped along the walls. Between those two rows sit 1,092 models. For 543 of them, Epoch has published an estimate of how much arithmetic went into training them. For the other 549, nobody outside the company that built them has enough information to estimate it.
The chart above plots every model with an estimate published since 2010 (481 of them) as a grey dot. Time runs left to right. The vertical axis is training compute on a logarithmic scale: each horizontal line is ten times the one below, so the chart covers fourteen powers of ten. The thick orange staircase is the record: it steps up whenever a model's estimate beats every earlier one. The dashed violet line is the trend for the typical model. Five models are called out; the tables below name every record-setter.
This piece climbs the same axis, one power of ten at a time.
10¹ to 10¹⁸: sixty years in the margin
First, the unit. A floating-point operation, or FLOP, is one step of arithmetic on numbers with decimal points: one multiplication, or one addition. Training a neural network means showing it examples and nudging millions or billions of internal numbers after each one, and every nudge takes several of these steps. "Training compute" is the total count of them across the whole run. It says nothing about how long training took or how many chips were used, only how much arithmetic was done.
Epoch describes its approach plainly: "Where possible, we collect details such as training compute directly from publications. Otherwise, we estimate details from information such as model architecture and training data." Hardware counts and training time are another route. The note on Google's 2007 language model SB-LM, for example, multiplies a processor's speed by 1,500 machines running for a day.
The chart starts in 2010, but the table goes back much further. The record started at Theseus's 4×10¹ and passed through the Perceptron Mark I at 6.9×10⁵ in January 1957, NetTalk at 2.8×10¹⁰ in June 1987 and the LSTM paper at 3.2×10¹³ in November 1997, among others, before reaching SB-LM at 1.4×10¹⁸ in June 2007. Fitted across the 62 pre-2010 models with estimates, compute grew about 1.5 times a year, doubling roughly every 20 months. That is the slow lane. The rest of this story happens in the fast one.
10¹⁸: where the chart begins
The orange line enters 2010 at SB-LM's 1.4×10¹⁸, a record from 2007 that stood for more than five years. AlexNet, the 2012 image classifier from the University of Toronto, does not touch it: Epoch estimates 4.7×10¹⁷, below the line. The first model published from 2010 on to set a new record was Google's DistBelief language model in January 2013, at 2.6×10¹⁸.
Then came a crowded 2014. Microsoft's SPPNet, Oxford's two VGG networks (VGG16 at 1.2×10¹⁹ in September 2014), Google's sequence-to-sequence translation model (5.6×10¹⁹, also September 2014) and Google's SNM-skip language model pushed the record up five times in under six months.
This rung, from 10¹⁸ up to 10¹⁹, holds 30 models with estimates. The original Transformer, published by Google in June 2017, sits here at 7.4×10¹⁸, below the record that had been set three years earlier.
10²¹: games and translation
The next three steps of the staircase belong to DeepMind's Go programs and Google's translation system. AlphaGo Fan is estimated at 3.8×10²⁰ (October 2015). AlphaGo Lee reached 1.9×10²¹ in January 2016, though Epoch grades that estimate speculative, meaning it could be off by up to 30 times either way. Google's Neural Machine Translation system took the record at 6.6×10²¹ in September 2016 and held it for 19 months, the longest wait between records since 2013.
This rung has 57 models, including OpenAI's GPT-2 (1.9×10²¹, February 2019) and DeepMind's AlphaFold 2 (3×10²¹, November 2020). Facebook's image model ResNeXt-101, trained on Instagram photos, set the record in May 2018 at 8.7×10²¹.
| Published | Model | Developer | Training compute (FLOP) | × previous record | Epoch confidence |
|---|---|---|---|---|---|
| 1950-07-02 | Theseus | Bell Laboratories | 4×10¹ | Confident | |
| 1957-01-01 | Perceptron Mark I | Cornell Aeronautical Laboratory,Cornell University | 6.9×10⁵ | Likely | |
| 1959-02-01 | Pandemonium (morse) | Massachusetts Institute of Technology (MIT) | 6×10⁸ | Speculative | |
| 1960-03-30 | Perceptron (1960) | Cornell Aeronautical Laboratory | 7.2×10⁸ | Speculative | |
| 1987-06-06 | NetTalk (dictionary) | Princeton University | 2.8×10¹⁰ | Confident | |
| 1987-06-06 | NetTalk (transcription) | Princeton University | 2.8×10¹⁰ | Confident | |
| 1989-11-27 | Handwritten digit recognition network | AT&T | 1.8×10¹¹ | Confident | |
| 1989-12-01 | Zip CNN | AT&T,Bell Laboratories | 1.5×10¹² | Confident | |
| 1992-05-01 | TD-Gammon | IBM | 1.8×10¹³ | Speculative | |
| 1994-12-02 | Predictive Coding NN | Technical University of Munich | 1.9×10¹³ | Confident | |
| 1997-11-15 | LSTM | Technical University of Munich | 3.2×10¹³ | Confident | |
| 2000-11-28 | PoE MNIST | University College London (UCL) | 5.2×10¹³ | Confident | |
| 2000-11-28 | Neural LM | University of Montreal / Université de Montréal | 6.3×10¹⁵ | Confident | |
| 2007-06-22 | KN-LM | 7.7×10¹⁷ | Likely | ||
| 2007-06-22 | SB-LM | 1.4×10¹⁸ | Likely | ||
| 2013-01-16 | DistBelief NNLM | 2.6×10¹⁸ | Likely | ||
| 2014-06-18 | SPPNet | Microsoft,Xi’an Jiaotong University,University of Science and Technology of China (USTC) | 3.4×10¹⁸ | Confident | |
| 2014-09-04 | VGG19 | University of Oxford | 1.1×10¹⁹ | Likely | |
| 2014-09-04 | VGG16 | University of Oxford | 1.2×10¹⁹ | Confident | |
| 2014-09-10 | Seq2Seq LSTM | 5.6×10¹⁹ | Confident | ||
| 2014-12-03 | SNM-skip | 3×10²⁰ | Speculative | ||
| 2015-10-01 | AlphaGo Fan | DeepMind | 3.8×10²⁰ | Likely | |
| 2016-01-27 | AlphaGo Lee | DeepMind | 1.9×10²¹ | Speculative | |
| 2016-09-26 | GNMT | 6.6×10²¹ | Likely | ||
| 2018-05-02 | ResNeXt-101 32x48d | 8.7×10²¹ | Confident | ||
| 2019-09-17 | Megatron-LM (8.3B) | Nvidia | 9.1×10²¹ | Likely | |
| 2019-09-17 | Megatron-BERT | Nvidia | 2.2×10²² | Confident | |
| 2019-10-23 | T5-11B | 3.3×10²² | Confident | ||
| 2019-10-30 | AlphaStar | DeepMind | 1.1×10²³ | Confident | |
| 2020-01-28 | Meena | Google Brain | 1.1×10²³ | Confident | |
| 2020-05-28 | GPT-3 175B (davinci) | OpenAI | 3.1×10²³ | Confident | |
| 2021-08-11 | Jurassic-1-Jumbo | AI21 Labs | 3.7×10²³ | Confident | |
| 2021-09-03 | FLAN 137B | Google Research | 2×10²⁴ | Confident | |
| 2022-03-15 | GPT-3.5 (davinci-002) | OpenAI | 2.6×10²⁴ | Speculative | |
| 2022-06-29 | Minerva (540B) | 2.7×10²⁴ | Confident | ||
| 2023-03-15 | GPT-4 (Mar 2023) | OpenAI | 2.1×10²⁵ | Likely | |
| 2023-12-06 | Gemini 1.0 Ultra | Google DeepMind | 5×10²⁵ | Speculative | |
| 2025-02-17 | Grok 3 | xAI | 3.5×10²⁶ | Likely | |
| 2025-02-27 | GPT-4.5 | OpenAI | 3.8×10²⁶ | Likely | |
| 2025-07-09 | Grok 4 | xAI | 5×10²⁶ | Speculative | |
| 2026-09-03 | GPT-6 Astra | OpenAI | 9.4×10²⁶ | Likely |
Each row beat every earlier estimate in Epoch AI's notable-models table. Bars are scaled to the log of the jump. Chip colour = builder type (violet companies, pink company + university, teal universities, grey other).
10²³: the language models arrive
Between September 2019 and January 2020, the record changed hands five times. Nvidia's Megatron models (2.2×10²², September 2019), Google's T5 (3.3×10²², October 2019), DeepMind's StarCraft player AlphaStar (1.1×10²³, October 2019) and Google's chatbot Meena (1.1×10²³, January 2020) each took a turn. AlphaStar was the first model in the table to reach 10²³.
Then came GPT-3, published in May 2020 at 3.1×10²³. It held the record until AI21 Labs' Jurassic-1-Jumbo edged past it in August 2021 at 3.7×10²³. Since GPT-3, every record-setter on the chart has been a language or multimodal model.
157 models in the table have estimates at or above 10²³. That count is less than a third of all models with estimates, yet every one was published from October 2019 on. In their 2024 analysis, Epoch's Jaime Sevilla and Edu Roldán found that notable language models grew "9.5 x/year" between June 2017 and May 2024, much faster than models overall, which they read as language models catching up with the rest of the field.
10²⁴: where universities stop
FLAN 137B, a Google model, was the first to pass 10²⁴, in September 2021. Google's Minerva set the record at 2.7×10²⁴ in June 2022, and PaLM sits on the same rung at 2.5×10²⁴. DeepSeek-V3, released in December 2024, is here too at 3.3×10²⁴.
This is also where universities drop out. The largest model built by universities alone is GLM-130B from Tsinghua University, at 3.5×10²³ in 2022. The number of university-only models at or above 10²⁴ is 0.
The yearly counts show the same shift by headcount. In 2014, universities alone built 21 of the 33 notable models published that year, and companies alone built 5. Companies working without academic partners first outnumbered everyone else combined in 2018. The total peaked at 119 in 2023, when universities alone still accounted for 18. In 2025 the figure was 2 of 109. In 2026, through October 28, 2026, it is 0: companies alone built 93 of the 95 notable models, and one more was a collaboration with universities.
| Year | Notable models | Companies only | Universities only | With compute estimate | Largest estimate that year |
|---|---|---|---|---|---|
| 2010 | 1 | 8 | 5 (50%) | 5.4×10¹⁶ | |
| 2011 | 0 | 8 | 5 (45%) | 5.2×10¹⁶ | |
| 2012 | 3 | 8 | 8 (62%) | 6×10¹⁷ | |
| 2013 | 7 | 10 | 10 (53%) | 2.6×10¹⁸ | |
| 2014 | 5 | 21 | 14 (42%) | 3×10²⁰ | |
| 2015 | 13 | 9 | 11 (38%) | 3.8×10²⁰ | |
| 2016 | 21 | 18 | 16 (33%) | 6.6×10²¹ | |
| 2017 | 26 | 18 | 20 (36%) | 8.4×10²⁰ | |
| 2018 | 23 | 4 | 24 (65%) | 8.7×10²¹ | |
| 2019 | 30 | 12 | 38 (56%) | 1.1×10²³ | |
| 2020 | 25 | 9 | 33 (66%) | 3.1×10²³ | |
| 2021 | 46 | 9 | 62 (78%) | 2×10²⁴ | |
| 2022 | 50 | 17 | 60 (66%) | 2.7×10²⁴ | |
| 2023 | 69 | 18 | 65 (55%) | 5×10²⁵ | |
| 2024 | 82 | 3 | 46 (46%) | 3.8×10²⁵ | |
| 2025 | 100 | 2 | 41 (38%) | 5×10²⁶ | |
| 2026 | 93 | 0 | 23 (24%) | 9.4×10²⁶ |
2026 runs to October 28, 2026. "Companies only" and "Universities only" exclude collaborations, which make up most of the remainder.
10²⁵: the rung with a rulebook
GPT-4 was the first model in the table to reach 10²⁵, in March 2023, at an estimated 2.1×10²⁵. That matches Epoch's separate tracker of large models, which says "The first model trained at this scale was GPT-4, released in March 2023."
This is the first power of ten written into law. Under Article 51 of the EU AI Act, a general-purpose model is presumed to have high-impact capabilities, and so to carry systemic risk, "when the cumulative amount of computation used for its training measured in floating point operations is greater than 10^25."
26 models in the notable table sit at or above it, from 17 different developers. 19 were built in the United States alone and 5 in China, with the largest Chinese estimate being ByteDance's Doubao-pro at 2.5×10²⁵ (October 2024). Epoch's large-model tracker had already counted 33 such models by June 2025, more than this table holds today. The two lists answer different questions: the tracker tries to catch every model over the line, while the notable table only includes models that meet Epoch's notability criteria, such as a state-of-the-art result, over 1,000 citations, historical relevance or significant use.
Of those 26, Epoch grades 8 confident and 6 speculative. Gemini 1.0 Ultra, which took the record in December 2023 at 5×10²⁵, is one of the speculative ones.
Epoch's country field gives the wider geography. From 2010 to 2017, organisations in the United States alone produced 43% of notable models and China 3%. Between 2023 and 2026, the United States alone produced 56% and China 26%: 236 and 111 models out of 422. The United Kingdom, with 26 notable models in the first period, had 2 in the last.
10²⁶: four models
Only 4 models in the table pass 10²⁶: Grok 3 (xAI, 3.5×10²⁶), GPT-4.5 (OpenAI, 3.8×10²⁶), Grok 4 (xAI, 5×10²⁶), GPT-6 Astra (OpenAI, 9.4×10²⁶). Grok 3 got there first, in February 2025. GPT-4.5 followed ten days later. xAI's Grok 4 is graded speculative, so its estimate is the loosest of the four.
The current record, GPT-6 Astra, published on September 3, 2026, is estimated at 9.4×10²⁶. Epoch's note on the entry cites OpenAI and its partners as saying the model was trained on at least 100,000 Nvidia GB200 chips in Abilene, Texas, and works forward from there with assumptions about training time and how fully the chips were used. The estimate rests on what was said about the hardware, not on anything published about the model itself.
This rung once had a rule too. In 2023, Executive Order 14110 asked for reports on "any model that was trained using a quantity of computing power greater than 10^26 integer or floating-point operations." It was rescinded on January 20, 2025, four weeks before Grok 3 became the first notable model to cross that line.
10⁰·⁶³: the slope between the rungs
Fit a straight line through all 481 estimates since 2010 and its slope is 0.63 powers of ten a year. In plainer terms, the compute behind a typical notable model grew about 4.3 times a year, doubling every 5.7 months. That is the dashed line. Over a decade, that rate compounds to roughly 2.2 million-fold.
Epoch's own published estimates use different samples and periods, and they land close. Sevilla and Roldán found notable models grew "about 4.1x/year" from 2010 to May 2024 and suggested summarising recent growth as 4-5x a year. Epoch's trends page now says "Training compute for frontier language models has been growing at 5× per year since 2020" and "doubling every 5.2 months since 2020."
The table gives similar answers when cut differently. Since 2018 the fit steepens to 5.8 times a year (doubling every 4.7 months). Since 2020 it is 5.2 times. For models Epoch flags as frontier, meaning in the top five by compute at release, the fit since 2020 is 4.6 times a year across 34 estimates. These are our fits, made with ordinary least squares on the logarithm of compute. They are not Epoch's, and the narrower cuts rest on fewer points.
The record itself moved more unevenly than the trend. It rose 26 times after 2010, sometimes several times in a season and sometimes not for more than a year. From SB-LM to GPT-6 Astra, it grew 649 million-fold.
| Training compute | Models | First to arrive | Built by companies only | Arrivals by year, 2010–2026 |
|---|---|---|---|---|
| 10²⁶ to 10²⁷ | Grok 3 (2025) | 100% | ||
| 10²⁵ to 10²⁶ | GPT-4 (Mar 2023) (2023) | 95% | ||
| 10²⁴ to 10²⁵ | FLAN 137B (2021) | 89% | ||
| 10²³ to 10²⁴ | AlphaStar (2019) | 82% | ||
| 10²² to 10²³ | Megatron-BERT (2019) | 63% | ||
| 10²¹ to 10²² | AlphaGo Lee (2016) | 49% | ||
| 10²⁰ to 10²¹ | SNM-skip (2014) | 47% | ||
| 10¹⁹ to 10²⁰ | VGG19 (2014) | 42% | ||
| 10¹⁸ to 10¹⁹ | SB-LM (2007) | 33% | ||
| 10¹⁷ to 10¹⁸ | KN-LM (2007) | 29% | ||
| 10¹⁶ to 10¹⁷ | RNN LM (2010) | 24% | ||
| 10¹⁵ to 10¹⁶ | Neural LM (2000) | 14% | ||
| 10¹⁴ to 10¹⁵ | NPLM (Brown) (2003) | 0% | ||
| 10¹³ to 10¹⁴ | TD-Gammon (1992) | 38% |
Models with a training-compute estimate, by order of magnitude. Sparkline counts arrivals per year from 2010 to 2026.
The rung nobody can measure
The chart only shows models with an estimate, and the part it leaves out is getting bigger. In 2018, 65% of notable models published that year had a compute estimate. In 2021 it was 78%. In 2025 it was 38%, and in 2026 so far it is 24%: 23 of 95.
Who is missing matters. Of OpenAI's 19 notable models in 2026, 18 have no estimate. None of Anthropic's 9 do. Google accounts for 7, of which 6 have none. Meanwhile, of the 23 models from 2026 that do have estimates, 9 come from China and 5 from South Korea, against 7 from the United States alone. The 2026 dots therefore lean toward the developers that disclose enough to be estimated, which is not the same as the developers running the largest training runs.
Several other limits apply to everything above:
- Estimates, not measurements. Epoch grades each figure. "Confident" means within a factor of 3, "Likely" within 10, and "Speculative" within 30, larger or smaller. Of the 543 estimates, 323 are confident, 153 likely and 67 speculative. On a log chart, a 30-fold error is about one and a half gridlines.
- A selected sample. The notable table is a subset of Epoch's database of over 3,700 models, chosen by the notability criteria above. A model that was large but never cited or used widely may not appear.
- Publication date, not training date. Each dot sits at the date the model was published, not when its training run happened.
- Our trend line. The fitted rates are ours, from this snapshot. Epoch revises entries as new information appears, and a refreshed download may move both the dots and the slope.
The data is published by Epoch AI under a Creative Commons Attribution licence, with the request that "the source and authors are credited". The full table behind the graphic, including models without estimates, can be downloaded below.
Back at the top of the table, Gemini 4 Carbon's blank cell is not unusual. It is what most of 2026 looks like. The staircase can still be drawn because some companies describe their hardware and some researchers publish their methods. Whether the next step up shows on this chart depends as much on disclosure as on how much compute anyone uses.
Sources and method
- Epoch AI, 'Data on AI models': notable AI models (CSV). Downloaded November 4, 2026 from https://epoch.ai/data/notable_ai_models.csv. Creative Commons Attribution licence; credit Epoch AI
- Epoch AI, Trends in AI. Published growth estimates for frontier language models
- Sevilla and Roldán, Training compute of frontier AI models grows by 4-5x per year (Epoch AI, May 2024)
We use every row of Epoch AI's notable-models table that has a training-compute estimate, placed at its publication date. Epoch estimates compute from reported hardware and training time or from model size and training data, and grades each estimate Confident (within 3×), Likely (within 10×) or Speculative (within 30×). The record staircase steps up whenever a model's estimate exceeds every earlier one. The trend is an ordinary least-squares fit of log10(compute) against publication date for all models with an estimate published from 2010 on; it is our fit, not Epoch's, and differs from Epoch's published rates because the sample and period differ. Builder type comes from Epoch's organisation categorisation: models with both a company and a university count as collaborations. Models without a compute estimate cannot be placed on the log scale; they are counted in the yearly table and the data download.
Charts like this, from your own numbers.
Connect a spreadsheet or the tools you run on, and Parity builds the dashboard and writes the report, with every number checked.
Try Parity free