MTEB v2 expands embedding evaluation
On October 20, 2025, an MTEB team post introduced version 2 of its evaluation suite, with multimodal support, rerankers, and migration guidance.
Source: MTEB v2 atualiza a suíte de avaliação de embeddings (x.com). Text prepared with AI from this source.
What happened and what to do
On October 20, 2025, the MTEB team published a post about version 2 of its suite for evaluating embedding models. The update includes simplified evaluation, multimodal support, rerankers, new interfaces and documentation, as well as dataset statistics and a migration guide.
A company can use these features to structure model comparisons around its tasks and data. A project could diagnose evaluation criteria, implement a reproducible testing and monitoring pipeline, and organize results in a dashboard to support adoption or migration decisions.
How the consultancy can help
Wendelmaques can diagnose how your team evaluates embeddings, define the testing scope, and implement the pipeline, monitoring, and results dashboard on your infrastructure.
Next step
Send a short description of your evaluation or migration case to receive a scoped proposal.
Let's assess your project
I reply with scope and a proposal.