Haystack

Haystack is an open-source framework for building agentic, production-grade AI systems that combine retrieval, generation, and orchestration in a modular way. It enables developers to design “compound” AI applications by connecting LLMs, vector stores, document stores, tools, and pipelines into robust, end-to-end workflows. Its primary purpose is to help teams move from prototype to scalable, maintainable AI services suitable for real-world environments.

At its core, Haystack provides configurable pipelines for tasks such as retrieval-augmented generation (RAG), question answering, semantic search, document ranking, and summarization. It offers integrations with major LLM providers, vector databases, and storage backends, along with components for preprocessing, routing, and post-processing. The framework supports agents that can call tools, perform multi-step reasoning, and interact with external systems via APIs. Strong observability, evaluation utilities, and support for deployment in Docker, Kubernetes, and cloud environments make it suitable for continuous improvement and monitoring in production.