Neon breaks the rule13 minCheaper, faster models are supposed to be worse. Google's new Gemini 4 Neon, released November 18, is cheaper and faster than Carbon and still tops leaderboards from tax law to running a vending business. Here's what it is, what the numbers really say, and the bot-fighting arena where it's being put to the test.Read now →

Stanford's 'Next Concept' AI Solved 86 of 100 Hard Sudokus Where a Normal Model Solved One. Did It Really Break LLMs?
A Stanford and Together AI paper says language models don't have to write one word at a time. Its Next-Concept Modeling sketches a whole answer, then sharpens it in a dozen parallel steps, running up to 31 times faster than diffusion rivals. The internet declared ChatGPT dead. The tables tell a more careful, and more interesting, story.
Here's a Sudoku puzzle so hard that a normal language model of the same size solves it about one time in a hundred.
A new system from Stanford solves it 86 times out of a hundred.
It doesn't fill the board one digit at a time, the way every chatbot writes. It sketches the whole grid at once, as a blur, and then sharpens it in just 12 steps.
When the paper behind it appeared in October, the internet did what the internet does. Posts declared that every chatbot had been "writing on a typewriter," that the old method was dead, and that Stanford may have just killed ChatGPT.
It didn't. But the real story, the one in the tables most people skipped, may matter more than the hype. Here's what Next-Concept Modeling actually is, what it proved, what it didn't, and why AI insiders are paying close attention.
The typewriter problem
Start with how every major AI chatbot talks to you.
ChatGPT, Claude and Gemini all work the same way underneath. They predict one token, roughly a piece of a word, then the next, then the next, always left to right. Each new piece waits for the one before it.
Ask for a 500-word answer and the model runs its giant network hundreds of times in a row, one small guess after another. It's like writing a novel on a typewriter: one letter at a time, never looking ahead, never going back.
This method, called autoregressive generation, works astonishingly well. But it's slow by design. And the bottleneck isn't raw computing power. It's memory: for ordinary chatbots, the speed limit is how fast the chip can move the model's weights out of memory for every single token.
Every one of those tokens also costs electricity. The International Energy Agency estimates that data centers used about 415 terawatt-hours in 2024 and projects demand roughly doubling to around 945 TWh by 2030, with AI the biggest driver. A big share of AI's load is inference: answering questions, billions of times a day.
The industry squeezes the typewriter hard. Prices keep falling. Tricks like speculative decoding, where a small model guesses ahead and a big model checks the guesses, can make translation and summarization two to three times faster.
But every one of those tricks still obeys the same rule: one token, then the next. Nobody threw the typewriter away. They just learned to type faster.
The first escape: diffusion
The first big attempt to break the rule is called diffusion, borrowed from image generators.
A diffusion language model starts with a scrambled block of text and cleans the whole thing up over many passes, fixing many words at once instead of one at a time.
And it works commercially:
- Inception, a Palo Alto startup co-founded by Stanford professor Stefano Ermon, sells diffusion models called Mercury, claiming speeds well over 1,000 tokens per second on standard Nvidia chips. Inception says its latest Mercury release beats small, fast models like GPT-6 Luna and Claude Haiku 5.5 on speed-focused benchmarks.
- Google DeepMind showed Gemini Diffusion in May 2025, performing on par with its small Flash-Lite model at the time.
- ByteDance said its Seed Diffusion preview reached 2,146 tokens per second.
- Together AI published its own technique to make diffusion text generation cheaper.
But diffusion has a hidden bill. Each cleanup pass usually churns through the whole sequence. Quality tends to drop when you cut the number of passes. Steps are expensive, the usual memory-saving shortcut doesn't apply, and training is costly. Speed claims also need context: one independent tester measured an older Mercury release far below its advertised rate.
So a Stanford team asked a different question.
The idea: paint, don't type
What if a model didn't refine the whole page over and over? What if it worked like a painter: rough shapes first, then detail, one layer at a time?
That's the core of Next-Concept Modeling (NCM): Fast, Parallel Coarse-to-Fine Text Generation, posted as a preprint in October 2026 by Stanford's Hazy Research lab and the cloud company Together AI. The first author is doctoral student Hermann Kumbong, and the senior authors include Simran Arora, Azalia Mirhoseini and Christopher Ré.
That lineup is why the industry didn't laugh it off:
- Christopher Ré runs Hazy Research, is a MacArthur Fellow and co-founded Together AI. His group is behind FlashAttention, a technique that cuts how often a model reads and writes graphics memory. If you've used a modern chatbot, you've almost certainly benefited from it.
- Azalia Mirhoseini runs Stanford's Scaling Intelligence Lab, previously co-led Google's AlphaChip project, which uses AI to lay out computer chips, and has contributed to mixture-of-experts research now common in frontier models.
- Kumbong, a Stanford PhD student originally from Cameroon, has published at top AI venues with a focus on making AI cheaper and faster to build and run, so more people can use it.
The headline claim was blunt: NCM generates text 10 to 31 times faster than strong rival models of similar size and needs 4 to 5 times fewer training tokens. The authors called it a new paradigm for fast, parallel, coarse-to-fine generation.
How it works, step by step
The inspiration comes from images. In 2024, an image generator called Visual AutoRegressive modeling (VAR) won a best paper award at NeurIPS. Its trick: draw a tiny blurry version of the picture first, then add detail scale by scale.
NCM does the same for text, in two stages.
Stage 1: build a coarse-to-fine compressor.
- The team trains a compressor that turns a passage of text into a stack of codes, from coarse to fine.
- The top level holds only a handful of codes summarizing the broad meaning.
- Each level below stores only what the levels above got wrong.
- At the bottom, there's one code per token. Stack all the levels and you rebuild the original text almost exactly.
Stage 2: generate the stack top-down.
- They freeze the compressor and train a standard transformer, the same engine inside every chatbot, to predict the stack from top to bottom.
- The breakthrough: within each level, it predicts every code at the same time. Sequential across levels, parallel within a level.
- A 128-token passage takes 12 steps instead of 128.
That flips the usual math. The number of steps is set by the depth of the stack, not the length of the answer. For a word-by-word model, a longer answer always means a longer wait. Here, a nine-level model takes nine passes regardless of length.
Two clever training tricks make it work:
- Random truncation. The stack is chopped at random depths during training, so the model learns to produce something sensible at every level, not just the last.
- Built-in humility. Guessing a whole level at once lets several mistakes slip in together. So the team deliberately swaps in random wrong codes and teaches the model to repair them. The model learns to catch its own errors before they reach the page.
What "concept" really means
The word "concept" is what made the paper go viral, and it's the most misunderstood part.
In this paper, a concept isn't a human idea or a readable outline hidden in the machine. It's the rough information in the top levels of the learned stack. The codes come from a vocabulary of roughly 4,000 entries. They're compression units, not thoughts.
It's also completely different from Meta's earlier "Large Concept Models," which treat each whole sentence as a concept.
Still, watching it work is striking. Given a German sentence to translate, NCM's first level is a near-meaningless scramble. Level by level, the sentence takes shape. By the 11th level, it closely matches the human translation, scoring 77.8 on a standard translation-similarity measure.
It looks like thinking: a blur becomes a draft, a draft becomes a sentence. And unlike models that reason silently inside hidden states, every level can be decoded back into text, so you can watch it work.
The numbers
The team measured speed as the real time to generate 1,024 sequences on a single Nvidia H100 GPU. The main models have only about 100 million parameters.
- News sentences (One Billion Words benchmark): 12 steps versus 128 for the strongest diffusion model. 21 times faster, with slightly better quality.
- News summarization: 8 steps instead of 128, 31 times faster, matching the best rivals' quality. With 16 drafts and a pick of the best, its summary score climbed to 29.6, above the best rival.
- Open web text: 32 passes instead of 1,024, about 10 times faster.
- Training efficiency: 13.36 billion training tokens on the news benchmark versus 66 billion for the diffusion models, almost five times fewer. On web text, around 127 billion versus more than 500 billion.
The Sudoku result, with a tiny model of about 25 million parameters, all in 12 steps:
| Difficulty | NCM | Word-by-word model (128 steps) |
|---|---|---|
| Easy | 99.5% | 14.6% |
| Medium | 96.5% | 5.1% |
| Hard | 86.1% | 1.0% |
The best diffusion rival reached 81.3% on hard boards.
Watching one hard board solve is revealing. After step one, 16 of 81 cells are right. After step two, it actually slips back to 15. Then it climbs, correcting itself, until all 81 are right at step 12.
That's why the result sends a chill through people who build these systems. A word-by-word model commits to its first digit before it knows anything about the last. If that first guess is wrong, everything after inherits the mistake. NCM sees the whole board from the start. As the authors put it, coarse-to-fine generation is well suited to tasks that need global planning.
The cold water
Now read the fine print, because one fact changes everything.
NCM never went head-to-head with ChatGPT, Claude or Gemini. Its opponents are diffusion and flow models of similar size. The headline speedups are measured against them, not against optimized word-by-word models. And the comparison doesn't include standard tricks like speculative decoding, which alone delivers two to three times faster translation and summarization.
Scale is untested. The main models have about 100 million parameters. One run reached 725 million, and quality did improve from 97 million to 317 million to 725 million. But no billion-parameter model has been tested. Popular open models people run daily have tens of billions; frontier chatbots are bigger still. Nvidia's Nemotron 3 Ultra, announced in May 2026, has 550 billion parameters, more than 700 times larger. Scaling is where promising architectures live or die.
The speedups don't stack. They come from three separate experiments.
On translation, the old method still wins. The strongest diffusion rival scored 24.8, NCM 22.6, and a plain word-by-word model of the same size 25.2. NCM only tied by drafting 12 times, which took 132 steps against the plain model's 64. The paper's own tables also show the same-size word-by-word model writing better open-ended text.
What outsiders found
Then outside researchers tested it with the released code and models, and found something more interesting than the headlines.
- Handling one request at a time, NCM ran about four times faster than a matched word-by-word model on translation and close to twice as fast on summarization. That's real.
- But with 64 requests at once, the way a busy server works, the old word-by-word model came out ahead.
The reason is almost poetic. When many requests share a chip, it stops waiting on memory and starts doing pure arithmetic, and NCM does more arithmetic, processing roughly four times more positions than the number of words it finally writes.
So NCM shines for a single user on a local machine. It isn't yet a clear win for data centers serving millions. That fits the authors' own setup, which timed batches of 16 on one GPU.
Even the Sudoku magic got a second look. A matching-size diffusion model filling cells in random order solves about a quarter of the hard boards. Tell it to fill the cell it's most confident about first, and it approaches NCM's score, with more refinement passes pushing it higher. A good part of the dramatic gap may come from the order cells get filled, not the architecture alone.
The small, human-sized failures
- Names and numbers. The compressor rebuilds 99.99% of web-text tokens exactly, but the errors land where you'd least want them. In one example, Melbourne became Brisbane and 1957 became 1956. Fine for a sports summary. A deal-breaker for medical records or contracts.
- Fixed depth. Even a one-word answer passes through the whole stack and pays full cost.
- No streaming. Nothing is readable until the final pass, so no word-by-word output like today's chatbots.
- One model per task. No general assistant and no public demo.
- Visible seams where coarse and fine levels meet.
- Evaluation-only code. The release supports testing, not training.
So, did this paper break large language models? No. Not even close. Not yet.
Why it still matters
That's the wrong question, though.
What the team showed isn't that the typewriter is dead. It's that the typewriter isn't the only option, and alternatives are improving faster than expected. For years, word-by-word generation was treated like a law of nature. Diffusion cracked that belief. NCM shows a third road, and on its home ground it beats diffusion by an order of magnitude.
It's also open. Code and checkpoints are on GitHub and Hugging Face under the permissive MIT license. Within weeks the repository drew hundreds of stars and over a hundred forks. Models this small run on a single home graphics card. A student anywhere can download them and watch a sentence crystallize out of noise.
Speed also buys second chances. Because each answer is so fast, the model can generate several candidates and pick the best one, improving translation and summarization while keeping total time competitive. Many fast drafts plus a judge, instead of one slow, careful attempt.
Who feels this first
Probably not the giant chatbot in your browser. Think of places where waiting hurts most:
- Voice assistants that need to answer before you lose patience.
- Coding tools, where a two-second pause breaks concentration.
- On-device AI on laptops and, eventually, phones.
Those are exactly the markets diffusion companies are chasing, and a method that beats diffusion at small sizes on a single GPU points straight at them.
What comes next
The authors' roadmap attacks the critics' main points:
- Scale up and test whether the speedups survive at billions of parameters.
- Train the compressor and generator together instead of in separate stages.
- Adaptive effort, spending more steps on hard problems and fewer on easy ones, so depth isn't fixed.
The broader pattern is worth watching. The next big leap may not come from a bigger model. It may come from changing how a model decides what to say, and in what order. And if even part of the world's text could be generated in a dozen parallel steps instead of hundreds of sequential ones, the math behind power-hungry data center build-outs could start to shift. Not overnight, but the direction is real.
Practical takeaways
- Ignore "X killed ChatGPT" headlines. Check what a new method was actually compared against.
- Watch batch size. Single-user speedups and data-center speedups are different things.
- Developers can try it now. The checkpoints are small, open and run on one consumer GPU.
- Don't trust names and numbers from early compression-based generators without checking.
FAQ
What is Next-Concept Modeling?
A text-generation method from Stanford's Hazy Research and Together AI that predicts a coarse-to-fine stack of codes, filling each level in parallel, instead of writing one token at a time.
Is NCM faster than ChatGPT?
It wasn't compared with ChatGPT. Its big speedups are against diffusion models of similar size. Against matched word-by-word models, independent tests found real gains for single requests but not for heavily batched server workloads.
Is it a product I can use?
No. It's a research method with small, task-specific models. Code and checkpoints are open for researchers and developers.
Why did it do so well on Sudoku?
Sudoku needs global planning, and NCM sees the whole board at every step. Part of the gap may also come from the order in which cells are filled.
Has it been peer reviewed?
It was posted as an openly available preprint. Most numbers come from the authors, and outside testing is still early.
The bottom line
Next-Concept Modeling isn't the most powerful AI ever built. It's a small research model that can't hold a conversation, can't stream a sentence and still confuses Melbourne with Brisbane.
But it's one of the clearest demonstrations yet that language doesn't have to be built one word at a time. On a puzzle that demands seeing the whole picture, it solved 86 hard boards in a hundred, where a word-by-word model of the same size solved one. That's not the end of the typewriter. It might be the first sketch of what replaces it.





