Skill Self-Play combines task generation, solution search, and dynamic skill control in a reinforcement learning loop to achieve both task diversity and training reliability.
CausalMix uses causal modeling instead of static assumptions to find optimal data mix ratios that generalize across different data pool sizes and model scales.
iLLaDA demonstrates that fully bidirectional diffusion training from scratch can be a competitive path to strong language models, even without autoregressive training.
Blackwell’s 180–268 GB memory per GPU enables larger batch sizes and longer sequences during model training, reducing communication overhead and allowing single-node training for models…
Structured curriculum learning strategies that leverage task relationships in latent space achieve better downstream performance than pure difficulty prioritization.
Uniform 4-bit formats eliminate the systematic shrinkage bias of E2M1 in FP4 LLM training and enable consistently better convergence across all model sizes.
STARE uses surprisal metrics and selective advantage reweighting to maintain policy entropy stability across long training sequences while improving accuracy by 4–8%.
A self-learning framework for code-repair agents leverages their solution traces directly to generate targeted training tasks, achieving higher accuracy than previous approaches.
Hidden-state alignment reduces sampling variance, closes the student-teacher gap more effectively, and trains with less memory and computational time than output-only distillation.
