Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillatio

arXiv:2606.05682v1 Announce Type: new Abstract: Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments. Quantization-aware distillation (QAD) helps recover accuracy lost under low bit quantization by training a...

Læs hele artiklen hos kilden.