Posts tagged "grpo"
2 posts found
Kolibri Post-Training from Scratch: Effort Labels, Format Rewards and a Gradient-Norm Trust Region
Building Aleph Alpha's Kolibri post-training recipe from the ground up: soft reasoning-effort labels drawn from log-length percentiles, attention-cost ordering for packed SFT batches, segment-aware group advantages, the p-norm format multiplier, the B-TV trust region, a closed-form per-token bound on the logit gradient, targeted on-policy self-distillation, the three-bucket curriculum, and the Merlin–Arthur game against hallucination. Every piece is derived on one German baker problem and a group of four rollouts.
MiMo-V2.6: Scaling Reinforcement Learning from Scratch
Building Xiaomi's MiMo-V2.6 RL recipe from the ground up — the importance-sampled GRPO objective, group-relative advantages and dynamic sampling, hack zeroing, Groupwise Reward Synthesis, Groupwise Advantage Redistribution, the group-relative length penalty, mass-conserving segment penalties, top-p candidate replay, router freezing, and the Little's-law Sample Mixer — all derived step by step on one four-rollout coding group.