Mathematical Prerequisites for the Delta Rule
Building the foundations for the delta rule and gated attention — the Frobenius norm and squared reconstruction loss, the gradient of a quadratic loss with respect to a matrix state, online stochastic gradient descent as a one-step update, and the chain rule for gated identity paths — all derived step by step with one consistent two-dimensional key-value example.