EAGLE 3.1 fixes attention drift to deliver stronger long-context performance, greater stability, and higher throughput for speculative decoding in production LLM serving.

It achieves this through two architectural fixes: post-norm hidden-state feedback feeding normalized states into the next decoding step, and FC normalization after each target hidden state and before the FC layer, making the drafter behave more like recursive invocations.

Training and benchmarks are contextualized within the SPEED-Bench coding dataset, including the targeted hardware and configuration used.

Deployment examples show a practical pipeline using vLLM to run the Kimi-K2.6-Eagle3.1-mla model, illustrating serving at scale.

TorchSpec now supports efficient training for EAGLE 3.1 and future speculative decoding, with an open-sourced draft model on HuggingFace to demonstrate deployment with TorchSpec and vLLM.

Benchmark results on Kimi K2.6-NVFP4 show EAGLE 3.1 delivering notable per-user throughput gains over non-speculative baselines at various concurrency levels.

VLLM integration remains backward-compatible: EAGLE 3.1 extends EAGLE 3 via a config-driven approach, preserving checkpoints and allowing draft models to plug into the same speculative-decoding path with the new normalization features.

Compared with EAGLE 3, EAGLE 3.1 offers better robustness to long-context inputs, improved stability across serving environments, and resilience to prompt variations, including up to twice the accepted long-context length.

Attention drift was identified as the main instability limiting EAGLE 3’s real-world performance, particularly with long contexts, diverse chat templates, and out-of-distribution prompts.