Posts tagged "neural-networks"
3 posts found
Mixture of Experts from Scratch — Part 1: The Foundations (1991–1993)
Building Mixture of Experts from the ground up — adaptive expert networks, gating functions, the mixture-of-Gaussians interpretation, hierarchical mixtures, and the EM algorithm — all derived step by step with a 2-expert regression example.
Manifold-Constrained Hyper-Connections: Stabilizing Deep Networks Beyond ResNets (with the actual math)
From residual identity paths to Hyper-Connections and mHC — now with the paper's exact equations, fully unrolled products, and concrete numeric examples
Manifold-Constrained Hyper-Connections: Stabilizing Deep Networks Beyond ResNets
A deep dive into why residual connections work, how Hyper-Connections generalize them, and why constraining learned skip paths to doubly stochastic matrices solves the instability problem