llama.cpp adds Multi-Token Prediction support
On May 17, 2026, a pull request to the llama.cpp repository added support for MTP heads. The report cites 65 tokens per second for Qwen3.6-27B on an RTX 3090, versus 38 without MTP.
Source: llama.cpp adiciona suporte a Multi-Token Prediction (github.com). Text prepared with AI from this source.
What happened and what to do
On May 17, 2026, a pull request to the llama.cpp repository added support for Multi-Token Prediction (MTP) heads, which can predict multiple tokens per execution, according to the report. With MTP enabled, Qwen3.6-27B reached 65 tokens per second on an RTX 3090, compared with 38 tokens per second in the cited comparison.
Companies serving compatible models can test MTP on their own infrastructure, comparing throughput, latency, memory use, and response quality against their current setup. A controlled deployment with metrics and rollback criteria can help determine whether the change suits the service’s requirements.
How the consultancy can help
Wendelmaques can assess the opportunity across a company’s infrastructure and models, scope an evaluation, and implement serving with monitoring. Operations and ongoing improvements can be included in the agreed scope.
Next step
Send a short description of your model, infrastructure, and performance goal to receive a scoped proposal.
Let's assess your project
I reply with scope and a proposal.