Skipping Language School May Speed Up Image Tools
Original title: Text-to-Image Generative Modeling Without Language Pretraining
How AI worksImages, video & soundScience
Read by Roger, a synthetic voice
The paper asks whether image generation tools need huge text training first. The authors built Halcyon, trained from scratch on paired pictures and descriptions. They report it beat strong existing systems with far less training effort, and that text-only training did not help.
Our take: Evidence is mixed. The authors report controlled tests and direct comparisons, but they used one picture set and one resolution. It matters because lower training cost could make image tools more common.
- Halcyon learned only from paired pictures and words, with no big text library.
- The authors report it beat strong systems with about 18 and 275 times less training effort.
- Text-only training did not improve results, but the authors tested only one picture set and size.
9 claims checked against the source Read the paper on arXiv PDF
Checked against the source
Every statement in this brief had to match a word-for-word passage from the source; code checks each match, and a second AI model reviews the whole brief for errors. A brief that fails is not published. These are the 9 passages it rests on.
Show the 9 passages
The authors ask whether language pretraining is necessary for strong text-to-image generation.
We ask whether language pretraining is necessary for strong T2I generation.
Halcyon is a family of models pretrained entirely from scratch on GPIC, a dataset of 100M image-text pairs.
We introduceHalcyon, a family of early-fusion multimodal autoregressive models pretrained entirely from scratch on GPIC, a dataset of 100M image-text pairs.
Halcyon-0.52B outperforms LlamaGen-XL/8 with about 18 times less pretraining compute.
Halcyon-0.52B achieves 48.0 gFD, outperforming LlamaGen-XL/8 (FLAN-T5-XL 1.22B + 0.90B image generator; 63.3 gFD) using18.2×lesstotal pretraining compute.
Halcyon-1.6B outperforms JiT-T2I-XXL/8++ with about 275 times less pretraining compute.
Halcyon-1.6B achieves 35.3 gFD, outperforming JiT-T2I-XXL/8++ (Qwen3-1.7B + 1.10B image generator; 41.0 gFD) using275×lesstotal pretraining compute.
Controlled experiments show that text-only pretraining does not improve text-to-image generation.
Controlled experiments show that text-only pretraining does not improve T2I generation.
Extending the findings to other datasets and higher resolutions remains important.
extending these findings to other datasets and higher resolutions remains important.
Halcyon uses no text-only language pretraining; its only language supervision comes from caption tokens paired with images.
Halcyonuses no text-only language pretraining; its only language supervision comes from caption tokens paired with images
Language pretraining accounts for over 80% of total pretraining FLOPs in representative T2I systems, supporting the cost claim.
language pretraining dominates the compute budget, accounting for over 80% of total pretraining FLOPs in representative T2I systems
The lower-cost recipe lowers the barrier to developing capable generative systems.
By providing a significantly lower-cost recipe for training strong T2I models, our work lowers the barrier to developing capable generative systems, including for malicious use.