DeepSeek-V4 Hybrid Attention: CSA and HCA from Scratch
Building the long-context attention of DeepSeek-V4 from the ground up — dual KV streams, overlapping softmax compression, the lightning indexer, top-$k$ sparse selection, and heavier compression — all derived step by step on a six-token example.