HijackRAG highlights risks in RAG systems
Published on December 1, 2024, the post describes an attack that uses injected texts, attention diversion, and instructions to manipulate RAG system outputs.
Source: HijackRAG usa textos injetados para atacar sistemas RAG (x.com). Text prepared with AI from this source.
What happened and what to do
Published on December 1, 2024, the post describes HijackRAG, an attack that combines injected retrieval texts, attention diversion, and instructions to manipulate RAG system outputs. It presents black-box and white-box modes using gradient-based optimization, and highlights the risks of malicious content in knowledge bases and the limits of existing defenses.
A company using RAG can assess how documents enter its knowledge base, test whether malicious content changes responses, and monitor sources and outputs. Practical measures include ingestion controls, adversarial tests, query and response logging, and alerts for unexpected behavior; the approach should fit the system and the risks identified.
How the consultancy can help
Wendelmaques can diagnose a RAG system’s exposure, scope implementation of ingestion controls, testing, and monitoring, and support operation on the client’s infrastructure.
Next step
Send a short description of your RAG system and concerns to receive a scoped proposal.
Let's assess your project
I reply with scope and a proposal.