Representation Conditioning for Generation
Synthetic voice
The authors propose RECON (representation conditioning), which reuses the MLP projection head that REPA normally discards after training, feeding its pooled semantic tokens into the later layers of the diffusion transformer so they can access vision-encoder semantics. During training, semantic tokens from the vision encoder are injected only below a timestep threshold t_max; at inference, with no clean image, the MLP head serves as a proxy and tokens are injected at every step. The method generally adds no parameters (Small models add a linear layer) and is reported to converge 63× faster than SiT and 25× faster than REPA, achieving gFID 3.25 at 80 epochs on ImageNet 256×256 without guidance.
- RECON reuses the discarded MLP projection head at inference to condition later layers on semantic tokens; during training it uses pooled vision encoder features injected only below a timestep threshold t_max, while at inference it injects at every step using the MLP head as a proxy.
- The authors report RECON converges 63× faster than SiT and 25× faster than REPA, with gFID 3.25 at 80 epochs (400K steps) without CFG on ImageNet 256×256.
- The method generally adds no parameters for B, L, and XL models, though the authors note certain S models require a linear layer.