Why Vanilla Attention Breaks at Scale: The O(n²) Wall
A precise accounting of why standard attention becomes infeasible at long sequences: exact FLOP counts, memory costs, KV-cache growth, and the HBM bandwidth bottleneck that makes memory worse than compute.