Speculative Decoding

https://developer.nvidia.com/blog/an-introduction-to-speculative-decoding-for-reducing-latency-in-ai-inference/

An Introduction to Speculative Decoding for Reducing Latency in AI Inference | NVIDIA Technical Blog