Randall Balestriero and Yann LeCun report that a text model trained on different descriptions of the same image reached roughly the same alignment with a vision model using 42 times fewer training samples than next-token prediction. Their October 29 arXiv preprint gives researchers a reason to examine how they pair training examples before increasing model size. The saving concerns representation alignment, not general language-model performance.
The paper combines a conditional mathematical result with experiments on Joint-Embedding Predictive Architectures, or JEPAs. These learn by bringing representations of related inputs together while preventing every input from collapsing into the same representation. MrKeyoor read the paper and its accompanying repository; we haven't reproduced the training runs.
What the 42-fold comparison measures
The authors' results table compares the same 109-million-parameter encoder and corpus under two training approaches. The JEPA uses different descriptions of one image as related examples, which the authors call good views. Next-token prediction learns from a text prefix and its continuation.
In section 4.2, the good-view model reaches a WIT alignment score of 0.109 after 21.6 million training samples. The next-token model reaches 0.111 after 900 million. The benchmark measures agreement between text and image representations; those scores aren't percentages of correct answers to general questions.
The accounting matters. Each good-view sample reads three texts, so the authors report a 14-fold reduction in text tokens alongside the 42-fold sample reduction. The runs also use different learning rates and batch sizes, and the researchers didn't score an earlier next-token checkpoint. The comparison therefore establishes the reported checkpoint results, without establishing the minimum training needed by either approach.
Why the theorem has conditions
The Platonic Representation Hypothesis asks whether models trained on different kinds of input converge toward a shared representation of the world. The authors prove a version for an idealized JEPA setting: separately trained encoders can learn matching representations, up to a rotation.
Their formal setup assumes a finite world, observations that identify its states, shared positive-pair relationships, and optimal solutions unique up to rotation. It doesn't cover the learned, sometimes action-conditioned predictors used in JEPA world models. Applying the result to those systems would require further analysis.
There is also a practical qualification to the text-only training. The web-scale model's views are grouped by image, and two of the three descriptions are generated by a vision–language model. The encoder never sees pixels, but its training data still carry information supplied by an image-aware system, as the authors disclose.
A result researchers can investigate
The public repository includes training and evaluation scripts, executable theory checks, and reproduction instructions. Its documented requirements separate CPU tests from training, which needs CUDA GPUs. Passing those component tests would still leave the training results to reproduce.
The next useful check is whether the sample advantage survives tighter comparisons of training recipes and independently constructed views. The current paper supplies a specific target for that work: measure text–image alignment while tracking samples and tokens separately, and record how the paired descriptions were produced.