Google Research has unveiled TurboQuant—a set of theoretically grounded quantization algorithms that compress large language models and vector search engines to extreme degrees without accuracy loss. The method combines PolarQuant for high-quality compression with Quantized Johnson-Lindenstrauss (QJL) for bias-free error correction, enabling key-value cache compression down to just 3 bits while delivering up to 8x speedup on H100 GPUs. TurboQuant achieves near-optimal results across benchmarks like LongBench and Needle In A Haystack, outperforming state-of-the-art baselines without requiring fine-tuning or training. The work will be presented at ICLR 2026.

TurboQuant: Redefining AI efficiency with extreme compression