Posts tagged "reinforcement-learning"
5 posts found
Kolibri Post-Training from Scratch: Effort Labels, Format Rewards and a Gradient-Norm Trust Region
Building Aleph Alpha's Kolibri post-training recipe from the ground up: soft reasoning-effort labels drawn from log-length percentiles, attention-cost ordering for packed SFT batches, segment-aware group advantages, the p-norm format multiplier, the B-TV trust region, a closed-form per-token bound on the logit gradient, targeted on-policy self-distillation, the three-bucket curriculum, and the Merlin–Arthur game against hallucination. Every piece is derived on one German baker problem and a group of four rollouts.
MiMo-V2.6: Scaling Reinforcement Learning from Scratch
Building Xiaomi's MiMo-V2.6 RL recipe from the ground up — the importance-sampled GRPO objective, group-relative advantages and dynamic sampling, hack zeroing, Groupwise Reward Synthesis, Groupwise Advantage Redistribution, the group-relative length penalty, mass-conserving segment penalties, top-p candidate replay, router freezing, and the Little's-law Sample Mixer — all derived step by step on one four-rollout coding group.
MaxRL: From REINFORCE to Maximum Likelihood
Why dividing by the number of successes instead of the batch size changes what your gradient estimator optimizes — and how this connects REINFORCE, maximum likelihood, and pass@k through one clean mathematical identity.
Reinforcement Learning from Scratch
Building RL from the ground up — actions, rewards, policies, expected reward, the policy gradient theorem, and REINFORCE — all derived step by step with concrete examples.
Mathematical Prerequisites for Reinforcement Learning
Building the math foundations you need for RL — probability, expected value, derivatives, the log trick, and Monte Carlo estimation — all through one consistent example.