This week: Researchers from Tel Aviv University and Stanford University showed that two AI models can start from random weights, train only on puzzles they generate themselves, and get better at predicting real-world data they never saw.

Elsewhere: How the teacher-student game works, its honest limits, a few ways to explore the paper yourself, and three fresh research drops worth a skim.

The New Prime Signal

Modern AI is a famously hungry reader. It consumes huge piles of text, photos, and code, and the industry keeps worrying about running out of good stuff to feed it. So here is a delightfully weird question: could a model learn useful skills without training on any human-made text, images, or other datasets? A team from Tel Aviv University and Stanford University says a small "yes, sort of" in a preprint called Self-Play Pretraining with Zero Data, which Tel Aviv University announced this week. Think of it as two study buddies locked in a room with no books, who invent their own homework, study only that, and somehow come out better at the real exam.

What shipped: The method starts with two models that know nothing, randomly initialized. One, the generator, writes short programs in a tiny, Brainf*ck-like programming language; running those programs spits out sequences of bytes. The other, the learner, practices predicting the next byte in each sequence, the same basic skill chatbots use to predict the next word. The generator is trained with reinforcement learning and rewarded for producing exercises at the edge of what the learner can handle: not so easy it's bored, not so random it's hopeless. Then came the real test. After training only on these made-up sequences, the learner was checked, zero-shot, on natural data it had never seen: language, images, speech, melodies, DNA, and math sequences. Per the paper, its loss on that data improved predictably as the team spent more compute, with scaling exponents comparable to those obtained by training directly on natural data. The learner also showed in-context learning, picking up rules from examples in a prompt without any extra training.

Why it matters: Today's AI progress mostly comes from feeding models more human-made data, and that supply is finite. This work hints at a second fuel source: compute alone, used to cook up practice problems. The authors frame it as an initial proof of concept toward training data that is "limited by compute rather than human knowledge." The models still learn from data, just data they make themselves. The researchers' hunch is that general tricks like copying, recursion, and nesting patterns can be learned from pure structure and then reused anywhere. There is also a cute bonus finding: while hunting for useful exercises, the generator rediscovered familiar math. The paper's table shows arithmetic sequences appearing from the start, geometric sequences by round 256, and Fibonacci-style sequences by round 512. Nobody told it about Fibonacci. It just found him useful, which is more than most of us can say about high school algebra.

The fine print: Keep the confetti in the box for a minute. The experiments are small: the paper says they are confined to models below 25 million parameters with a 4K context, tiny next to today's chatbots, and the authors list scaling up as the most immediate open question. Tel Aviv University and the researchers are also clear that this method cannot teach a system facts about the world. It learns how to spot patterns, not what the capital of France is. For that, a model still needs real data. And "zero data" means no natural data in training updates: the paper says natural data are never used for gradient updates, though held-out natural datasets are used to evaluate the models and pick settings. The likely near-term use is as a warm-up: the paper explores starting normal training from a self-play checkpoint, and the university says the approach may eventually help AI learn from real data more efficiently. It is also a preprint, first posted in September, so it has not been through peer review.

We love this one because it flips the usual AI story. Instead of “bigger pile of internet, bigger brain,” it asks what a mind could learn from pure puzzles. The answer so far is: real but modest skills, in small models, with no knowledge of the world attached. That’s not a replacement for data. It’s more like stretching before a run. If it scales, future models might show up to their real training already limber, and that could mean less data, less cost, or both.

One model writes puzzles, the other solves them, and the puzzle-writer gets paid only when its puzzles actually teach something.

How it works: Each round, the generator samples a batch of programs. Each program runs on a fixed universal machine with a random input tape, so one program can produce a whole family of outputs, and the programs are built so they can't crash. The learner takes one training step on those outputs. The generator then gets a learning-progress reward, which roughly asks whether a puzzle built on what the learner has actually been absorbing, instead of just being noisy. To keep exploring without forgetting, the pool mixes fresh programs, small mutations of high-scoring ones, and replays of older winners. The team also found that averaging several independently trained models into an ensemble helped.

Try it: Start with the Tel Aviv University explainer for the plain-language version, then open the full paper on arXiv and jump to Figure 1, which shows the generator-learner loop and the scaling curves on one page. The discovered-sequences table is the fun part: it lists example programs and the first round each math family showed up. For broader context, The Jerusalem Post and Ynetnews both covered the study.