ORPO combines SFT and preference alignment in one stage
Published on September 19, 2026, the paper introduces ORPO, a method that combines supervised fine-tuning and preference optimization without a reference model.
Source: ORPO combina otimização de preferências com supervised fine-tuning (arxiv.org). Text prepared with AI from this source.
What happened and what to do
Published on September 19, 2026, the paper introduces ORPO, an approach that combines preference alignment with supervised fine-tuning (SFT) without using a reference model. During preference-aligned SFT, the method applies a penalty to disfavored generations. The proposal offers an alternative to performing SFT and preference alignment in separate stages.
A company can assess whether this approach fits its model training by comparing it with its current process on representative data and tasks. Implementation could include data preparation, controlled experiments, and results monitoring to support a technical decision.
How the consultancy can help
Wendelmaques can diagnose the current training process and the opportunity to evaluate ORPO, define an implementation scope, and support experiment operations and monitoring.
Next step
Send a short description of your model training case to receive a scoped proposal.
Let's assess your project
I reply with scope and a proposal.