Haystack: deepset's framework for RAG and AI agents

A search pioneer that grew into agents

Haystack has been around since 2019, well before the chatbot boom; it started out as a way to find answers in large document collections. The framework was rebuilt from the ground up with version 2 in March 2024, and version 3 followed in July 2026. deepset GmbH, the company behind it, now describes it as open source orchestration for production-grade agents and applications.

Components and pipelines

The core idea is simple: every task (splitting a document, computing embeddings, querying a vector database, calling a model, formatting an answer) is a component. You connect components into a pipeline, a deterministic graph where every step is known in advance. That’s the classic way to build RAG: retrieve the right passages, then ask the model to answer from them.

For less predictable requests, the Agent component hands the choice of steps to the model. An agent can be wrapped as a tool for another agent, which gives you a coordinator-and-specialists multi-agent system. Tools can also come from MCP servers through the official integration.

Getting started

Install the haystack-ai package (Python 3.11 or later), then work through the official tutorials: a first question-answering pipeline over a few documents, then an agent with tools. Pipelines are serializable, which makes them easier to ship in containers or on Kubernetes.

Strengths and limits

Strengths: years of maturity, an explicit and testable architecture, lots of integrations, and independence from any single model provider or cloud.

Limits: the component mindset takes time to learn; major version changes require migrations; and the visual interface is only available in deepset’s paid platform.

Alternatives

LlamaIndex is the closest competitor for RAG, LangChain offers a wider ecosystem, LangGraph gives fine control over agents, and DSPy takes an approach built on automatic prompt optimization.