Posts tagged "probability"
6 posts found
Mathematical Prerequisites for Mixture of Experts — Part 3
Building the math foundations you need for understanding why MoEs work — orthogonality, vector norms, asymptotic notation, Lipschitz continuity, and dispatch entropy — all derived step by step with one consistent example.
Mathematical Prerequisites for Mixture of Experts — Part 2
Building the math foundations you need for sparse MoEs and the Switch Transformer — softplus, top-k masking, mean and variance, coefficient of variation, indicator functions, argmax, differentiability, and the dot-product loss — all derived step by step with one consistent example.
Mathematical Prerequisites for Mixture of Experts
Building the math foundations you need for Mixture of Experts — expected value, Gaussian densities, likelihood, Bayes' theorem, softmax, mixture models, conditional probability, multinomial distributions, and the Sherman-Morrison-Woodbury formula — all derived step by step with one consistent example.
Foundation Prior: How LLM Outputs Reshape Bayesian Beliefs
Deriving the Foundation Prior framework from scratch — why synthetic data is not real evidence, the constrained optimization that produces exponential tilting, the trust parameter λ, prompt heterogeneity through mixtures, calibration via real data, and the final posterior — all step by step with one coin example.
Mathematical Prerequisites for Foundation Prior
Building the math foundations for understanding how LLMs reshape Bayesian priors — parameters, likelihood, Beta distributions, KL divergence, entropy, exponential tilting, and marginal likelihood — all derived step by step with one coin example.
Mathematical Prerequisites for Reinforcement Learning
Building the math foundations you need for RL — probability, expected value, derivatives, the log trick, and Monte Carlo estimation — all through one consistent example.