Posts tagged "llm"
2 posts found
X-Token: Cross-Tokenizer Knowledge Distillation from Scratch
Building cross-tokenizer distillation from the ground up — why per-position KL breaks across tokenizers, DP span alignment, the chain-rule chunk merge, the projection matrix W, and two complementary losses (P-KL and H-KL) — with a full proof of GOLD's suppressive gradient and one running 2+3 example derived step by step.
Foundation Prior: How LLM Outputs Reshape Bayesian Beliefs
Deriving the Foundation Prior framework from scratch — why synthetic data is not real evidence, the constrained optimization that produces exponential tilting, the trust parameter λ, prompt heterogeneity through mixtures, calibration via real data, and the final posterior — all step by step with one coin example.