Model distillation: the direction of per-token KL
A systematic study published on October 3, 2026, reports that the direction of per-token KL divergence matters more than rollout policy in on-policy versus off-policy distillation.
Source: Estudo de distilação destaca a direção do KL por token (x.com). Text prepared with AI from this source.
What happened and what to do
The systematic study, published on October 3, 2026, reports that the direction of per-token KL divergence matters more than rollout policy in on-policy versus off-policy distillation. The finding challenges the assumption that on-policy rollouts are inherently better and may guide the choice of KL objectives and rollout policies.
A company training or adapting models can turn this finding into controlled experiments: compare KL directions and rollout policies using the same data and evaluation criteria, record metrics, and track quality, cost, and stability. The practical need is an evaluation and monitoring routine for the distillation recipe, not the automatic adoption of a particular policy.
How the consultancy can help
Wendelmaques can assess the opportunity in your distillation recipe, scope experiments, and implement metric collection, testing, and operational monitoring.
Next step
Send a short description of your distillation case to receive a scoped proposal for diagnosis and implementation.
Consulting for your project
Infrastructure review, deployment and ongoing operations, with scope and pricing defined in the proposal.
Quoted per project
Request a proposal