HERMES: component-level harness with a resident LLM per code module

What happened and what to do

The arXiv paper presents HERMES, a code-agent harness in which each repository component is paired with a resident LLM that knows that code. According to the authors, the approach yields gains on migration tasks and on four software engineering benchmarks, compared with equivalent baseline harnesses. The central finding is that harness design, not only the choice of model, can produce large improvements in long-horizon code agents.

For a company with legacy systems or large monorepos, the applicable idea is to organize the agent by module, with local context and clear responsibilities, instead of a single agent with access to the whole repository. This can be prototyped on private repositories, using models hosted on the client's own infrastructure, with evaluation pipelines that measure migration and test tasks before any adoption. Before scaling, it is necessary to measure cost, accuracy and regressions per component, with an auditable record of every proposed change.

How the consultancy can help

Diagnosis of how code agents are used today in repositories and migrations, followed by a prototype per-component harness on client code, with an internal benchmark and cost and quality metrics. Operation with monitoring, context versioning and human review of changes, on the client's own infrastructure.

Next step

Send a short description of your repository, the migration or maintenance work you want to automate, and the infrastructure available. We will then prepare a scoped proposal with an initial diagnosis, timelines and a project or monthly retainer format.