What is Haystack? An introduction to the NLP framework

What is NLP?

If you've ever used a voice application like Siri, Alexa, or Google Assistant, or if you've used a large language model like ChatGPT (and who hasn't these days?), then even if you didn't realize it, you're somewhat familiar already with NLP (natural language processing).

NLP is an area of machine learning (even if many put it under the broader umbrella of AI) that enables machines to read, understand, and interpret human language.

Among the most common NLP-based solutions are chatbots, question-answer (QA) systems, translation, speech recognition, and automatic text classification, to name but a few.

In a previous blog post, I introduced you to one of the most renowned platforms for natural language processing: Hugging Face. Now it's Haystack's turn!

What is the Haystack NLP framework?

Haystack is an open-source end-to-end Python framework for natural language processing made by deepset, a startup launched in 2018. It was designed to leverage Transformers, which are the deep-learning architectures behind large language models like ChatGPT.

Haystack's range of pre-built components for natural language processing tasks and support for pre-trained LLMs enables developers, researchers, and NLP enthusiasts to build and deploy sophisticated applications powered by large language models, Transformer models, and vector databases.

Haystack's modular design lets you customize pipelines and try a range of different models, such as those hosted on platforms like Hugging Face. This flexibility makes it easier to build, fine-tune, and scale QA systems.

How Haystack works

Components

Components are at the core of Haystack. These are the fundamental building blocks that can perform tasks like document retrieval, text generation, or summarization. Even a single component is powerful enough to manage local language models or communicate with a hosted model through an API.

Haystack provides components you can use out of the box, but it also allows you to create your own custom components with its range of integrations.

In a similar fashion to LangChain, Haystack can chain components together to build pipelines. These pipelines are the foundation of Haystack's NLP app architecture.

Pipelines

Haystack works by leveraging Retriever-Reader pipelines. Pipelines are powerful structures made up of components that connect to infrastructure building blocks, such as Elasticsearch or Weaviate. They are the standard structure used for connecting your data and performing your NLP tasks.

You can build very reliable and sometimes quite elaborate NLP pipelines with Haystack (extractive or generative QA, summarization, document similarity, semantic search, FAQ-style search, to name just a few examples). You can do this with Transformer models or large language models.

Readers and Retrievers

Readers are powerful models based on Transformer models. They analyze documents and perform QA tasks based on them.

Retrievers are filters that scan the documents in Haystack Document Stores (more about that below) to identify relevant data and pass them on to the Reader. That reduces the number of documents the Reader needs to process.

Haystack achieves this by using a framework called FARM (Framework for Adapting Representation Models). FARM facilitates transfer learning on representation models.

Agents

Haystack Agents use LLMs to resolve complex tasks. You can put Haystack Agents on top of your pipelines and use a prompt-defined control to find the best tool or pipeline for your task. The Agents can then determine which tools are useful to answer the query and call them in a loop to produce an answer. That means they can achieve much more than extractive or generative QA pipelines.

You can use Haystack Agents with different LLM providers. You just need to implement one standardized wrapper class for your chosen model provider.

Document Stores

Document Stores are databases where you can store your text data for Haystack to access it. Among the wide range of stores available are Pinecone, FAISS, and the aforementioned Elasticsearch and Weaviate.

Illustration of human hand and NLP - Natural Language Processing for What is Haystack, an introduction.

Why use Haystack?

Large neural networks, especially those with transformer-based architectures, are great for both Extractive and Generative QA. However, they're computationally expensive and time-consuming, making them pretty much unusable in latency-sensitive applications. Haystack solves this problem by prefiltering the documents using faster (albeit less powerful) solutions. That means neural models can complete inference in a short amount of time.

Let's make this a little plainer, shall we? Here's a list of things that Haystack can help you with (this list is by no means exhaustive):

Enables easy retrieval of information from unstructured text data
Handles large volumes of text data on a large scale
Offers pre-trained models for various NLP tasks (entity recognition or text classification, for example)
Lets you deploy models from Hugging Face or other providers into your NLP pipeline
Allows you to create dynamic templates for LLM prompting
Provides cleaning and preprocessing functions for multiple formats
Offers seamless integrations with vector databases and indexes
Provides tooling for a faster and more structured annotation process
Lets you deploy your system with Haystack's REST API, so you can query it with a user-facing interface

Is Haystack worth the hype?

Yep! Natural language processing requires more than just language models. As an end-to-end framework, Haystack lets you build your system from start to finish and offers tooling for every stage of your NLP project life cycle.

What is Haystack? An introduction to the NLP framework

Content

What is NLP?

What is the Haystack NLP framework?