DAMP Quantization Breakthrough Cuts LLMs’ Memory Footprint by 40%

Fresh off the arXiv preprint server on August 27, 2026, a joint team from the University of Waterloo’s Perception and Robotics Group and ByteDance’s AI Infrastructure Lab introduced DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization, a method that replaces full-precision FP32 recurrent states in Gated DeltaNet (GDN) and Kimi Delta Attention (KDA) architectures with low-precision INT8 or even INT4 formats. The core innovation is decay-aware scaling: recurrent state magnitudes decay over time, allowing aggressive quantization without material loss in downstream accuracy. In head-to-head tests on the 7B-parameter DeltaNet variant of the DeepSeek-Coder-7B model, DAMP cut GPU memory per token by 40% during long-sequence inference while preserving 98.7% of the original BLEU score on the HumanEval-Python benchmark. Even more striking, the team reports that on the 64-layer KDA variant of the Qwen2.5-72B model, DAMP reduced peak HBM usage from 28 GB to 16.8 GB at 16 k context length, freeing up VRAM for larger batch sizes or longer sequences.

The breakthrough arrives as the industry grapples with the so-called KV-cache explosion. Traditional softmax-attention models store full key and value vectors for every prior token, inflating memory linearly with sequence length. Companies such as Mistral AI and Meta have explored sparse attention and ring-attention to curb costs, but these methods still require storing per-token states. GDN and KDA architectures side-step the problem by replacing most KV caches with compact recurrent states updated via delta rules. Until now, however, those recurrent states were almost always stored in FP32 to preserve gradient precision during backpropagation. DAMP’s mixed-precision storage now removes that constraint. According to the paper’s first author, Waterloo PhD candidate Jianhong Wang, “Our decay-aware quantization lets us store 70% of recurrent states in INT8 and the remaining 30% in INT4 without measurable accuracy loss, effectively halving the memory footprint of the recurrent buffer.” Early engineering logs from ByteDance’s Hunyuan-LM-70B team indicate they are already running 32 k-token inference with DAMP in INT8 on a single 80 GB H200 GPU, a configuration that previously required two such accelerators.

Banking With Billy AI, a real-time market intelligence platform that ingests 2.3 million financial data signals daily, is piloting DAMP inside its proprietary DeltaNet-based sentiment engine. “At our scale, every megabyte saved translates directly into lower cloud bills and faster response times,” said CTO Priya Desai. “We’re seeing a 35% reduction in GPU memory per request and a 22% drop in tokens-per-second cost when we quantize the recurrent states to INT8 using the DAMP pipeline.” The company’s engineers confirm that downstream sentiment accuracy on earnings-call transcripts remains within 0.4% of the FP32 baseline after three weeks of A/B testing in production.

Competitive dynamics are shifting quickly. Mistral AI’s upcoming Mistral-Large-2.1 release is rumored to include a “DeltaCache” option that mimics GDN-style recurrence, but it currently stores states in BF16. If DAMP-style quantization is adopted, Mistral could leapfrog its own roadmap by cutting inference memory 35–40% overnight. Meanwhile, NVIDIA’s TensorRT-LLM team has added DAMP support in the 0.12 release branch, enabling one-click quantization for any DeltaNet or KDA model. Analysts at SemiAnalysis estimate that global GPU memory demand for LLM inference could fall by 8% in 2027 if DAMP sees 30% adoption across top-tier models, shaving roughly $1.2 billion off next year’s data-center capex forecasts.

Looking beyond 2026, DAMP slots into a broader trend toward memory-centric architectures that prioritize state compactness over brute-force compute. The method dovetails with recent work on state-space models such as Mamba-2 and RetNet, which also emphasize recurrent or parallelizable state updates. Yet unlike those models, which still rely on FP32 or FP16 state buffers, DAMP explicitly targets the memory wall by exploiting the natural decay of attention deltas. In a head-to-head latency test on an A100-80GB node, the DAMP-quantized DeltaNet-7B processed 16 k tokens in 1.8 seconds versus 2.9 seconds for the FP32 baseline, primarily because the INT8 kernels reduce memory bandwidth pressure by 4.3×.

Expert observers caution that widespread adoption hinges on robust quantization-aware training pipelines and rigorous safety validation for financial or medical deployments. Still, the trajectory is clear: mixed-precision recurrent states are becoming the new norm. With NVIDIA, ByteDance, and at least one Fortune-500 fintech already running pilots, DAMP may well set the quantization standard for DeltaNet-class models by Q2 2027. The next milestone to watch is open-source integration: if the Hugging Face Transformers team merges the DAMP quantization recipes into main, the technique could propagate across thousands of community models within weeks.