HOLA pairs linear attention with an exact, bounded KV cache
Published on July 3, 2026, the paper presents HOLA, an architecture combining recurrent state and a bounded key-value cache, with reported gains in perplexity and long-range recall.
Source: HOLA adiciona um cache KV exato e limitado à atenção linear (arxiv.org). Text prepared with AI from this source.
What happened and what to do
Published on July 3, 2026, the paper describes HOLA, which combines a recurrent state based on the delta rule with an exact, bounded key-value cache. It reports improved Wikitext perplexity and robust recall of specific information on RULER up to 32k tokens. The design aims to improve long-range recall while preserving a compressed state for linear attention.
For a company, the practical next step is to evaluate this design on real long-context tasks: define test sets, measure recall, quality, and memory use, and compare results with the current solution. An evaluation and monitoring pipeline can inform whether to integrate this kind of architecture into an inference service, without assuming it suits every use case.
How the consultancy can help
Wendelmaques can diagnose the opportunity and requirements, define an evaluation with relevant data and metrics, and scope an implementation, including integration and operation of the selected pipeline or service.
Next step
Send a short description of your long-context use case to receive a scoped proposal for diagnosis and implementation.
Let's assess your project
I reply with scope and a proposal.