Rejection sampling for on-policy LLM alignment
A paper published on July 7, 2026 proposes using rejection sampling to treat accepted off-policy tokens as on-policy and address the variance of accumulated per-token importance sampling ratios.
Source: Uma abordagem com rejection sampling para alinhamento on-policy de LLMs (arxiv.org). Text prepared with AI from this source.
What happened and what to do
The paper, published on July 7, 2026, proposes converting off-policy tokens into on-policy tokens for LLM alignment using rejection sampling. The approach aims to address variance associated with accumulated per-token importance sampling ratios and help engineers understand differences between data and policy in RL post-training.
A company developing or evaluating this kind of training could run a controlled experiment against its current process, recording acceptance rates, variance, and divergence between data and policy. Evaluation pipelines and dashboards can make these signals observable without assuming the method suits every model or dataset.
How the consultancy can help
Wendelmaques can diagnose a training and evaluation pipeline, define a scope for testing the approach, and implement the required collection, indicators, and monitoring components. Ongoing operation can be included in the scope as needed.
Next step
Send a short description of your LLM alignment or evaluation case to receive a scoped proposal.
Let's assess your project
I reply with scope and a proposal.