A new paper introduces Hybrid Policy Distillation (HPD), a method for compressing large language models (LLMs) into smaller, more efficient student models without significant performance loss. The approach reinterprets existing knowledge distillation (KD) techniques as token-level reweighted log-likelihood objectives, combining the strengths of forward and reverse Kullback-Leibler (KL) divergence.
HPD balances mode coverage and mode-seeking behavior by using both off-policy data and lightweight approximate on-policy sampling. This hybrid strategy improves optimization stability, computational efficiency, and final performance across tasks such as reasoning, dialogue, and code generation. The method addresses key challenges in model compression, offering a practical pathway to deploy powerful LLMs in resource-constrained environments.