Does Haystack Remember Between Runs?

No, not on its own. A Haystack Agent keeps conversation history, including tool calls, while a run is going. When the run ends, Haystack hands that history back as messages output but does not store it or link it to the next run. A Haystack agent starts cold across runs unless you add a memory layer.

That distinction is the whole question for anyone shipping a Haystack agent. An agent that remembers what a user said five turns ago is doing its job. An agent that asks the same user the same questions next week, because the new run has no link to the old one, is not.

What does a Haystack Agent keep during a run?

Quite a lot, and this is where the confusion starts. Haystack is deepset's open-source framework for building production LLM applications: pipelines, agents, and retrieval. Its Agent component is a loop: it takes messages, calls tools, updates state, and generates replies until an exit condition is met.

During that loop, the Agent maintains conversation history. It stores tool outputs it is configured to keep, tracks step counts and token usage, and returns the full chat history along with its final message. A follow-up question in the same run can use pronouns and build on earlier work. That is real memory, but it is memory with a lifespan of one run.

Older Haystack documentation describes the same idea in a different shape: memory injected into the prompt with every new query, sometimes summarized to fit the context window. The mechanism changed across versions. The boundary did not: the memory belongs to the run, not to the user or the project.

What happens when the run ends?

The run returns its messages and its state, and then the Agent forgets. The next run starts with whatever messages you pass into it, and nothing else. There is no built-in store the next run consults and no link between a run on Tuesday and a run on Friday.

This is easy to miss in development, where you keep one process alive and keep talking to it. The gap shows up when the agent ships: a service restarts, a scheduled job fires, a new session opens, and the agent meets a returning user as a stranger. A June 2026 walkthrough puts it plainly: the Agent handles tool use well, but it is stateless across runs, and the next session starts cold.

How do you test whether your Haystack agent remembers?

Run this test before your users do it for you. It takes two runs and one fact.

1. In the first run, tell the agent a specific, checkable detail, such as a customer's preferred plan. Let the run finish completely.
2. Start a second run with fresh messages only, as a real new session would. Ask about the detail from the first run.
3. If the agent answers correctly, something in your setup carried the detail across the boundary. If it asks you to repeat it, the default is confirmed: nothing persists between runs.

Repeat the test after you add any memory option below: it measures the boundary, not the feature list.

What are your options for memory that survives the run?

There are four honest paths, and they differ in who operates the memory.

First, pass the messages back in yourself. The Agent returns its full chat history, so you can store those messages and feed the relevant ones into the next run. This costs nothing extra and works for short histories. It breaks down as histories grow: every run re-reads the whole transcript, token costs climb, and old instructions from months ago compete with today's context.

Second, build a store yourself. Save the facts worth keeping to a database or document store at the end of each run, and load the relevant ones at the start of the next. The agent receives compact facts instead of sprawling transcripts. The price is ownership: the schema, the retrieval logic, and the routine that prunes stale facts are all yours.

Third, use memory tools built for the platform. On deepset's cloud platform, Mem0 memory tools give an Agent long-term memory: a store step that persists facts, preferences, and context, and a retrieve step that searches them in future conversations. Memories are scoped to individual users, so one user's facts stay separate from another's. Third-party drop-in tool sets offer the same shape for self-hosted agents: retain, recall, and reflect tools.

Fourth, plug in a hosted memory layer that sits outside Haystack entirely. This is where a hosted memory layer fits. A built agent connects to Vilix AI with an API key as a Bearer token to https://api.vilix.ai/mcp and gets full memory with zero memory infrastructure to run. The memory lives outside any single run, so the agent can pull relevant context from earlier runs and other tools before it answers and save what it learned. Retrieval is semantic as well as keyword-based, so it finds what was meant, not just what was typed.

How should you choose?

Ask what your agent needs to remember, not just what it chats about.

• If every run is standalone and nothing from one run should carry into the next, Haystack's built-in run history is enough.
• If the same people return and expect the agent to know them, you need memory keyed to the person, not the run.
• If your agents also run in other tools, you need memory that follows the work across tools, not one pipeline.
• If you do not want to run and maintain another database, a hosted layer removes that work.

Frequently asked questions

Does Haystack remember a conversation after the process restarts?

No, not by default. A new run starts with only the messages you pass in. Anything from earlier runs must have been saved by you, or by a memory tool you added, and loaded back explicitly.

Is passing the previous messages back in enough for a production agent?

It is enough for short, single-user conversations. It stops being enough when histories grow long, when many users share one agent, or when the agent needs an old fact without re-reading everything since.

What is the difference between run history and user memory?

Run history is the transcript of one execution. User memory is what the agent knows about a person across runs: preferences, decisions, and open items. Production agents usually need the second one.

Do I need to run my own database for Haystack memory?

Not necessarily. Passing messages back in needs no database, memory tools can run on a subscribed service, and a hosted layer means no database to run at all. Running your own store only makes sense if you specifically want to operate that infrastructure yourself.

The short answer

Haystack remembers within a run and hands you the full history when the run ends. It does not, by default, remember a person between runs. If your agent needs to recognize returning users, add a memory layer scoped to the person and the work, not just to the run.

Learn more about shared agent memory at https://vilix.ai/?utm_source=vilix-blog&utm_medium=article&utm_campaign=icp1-visibility