Does LiveKit Agents Remember Between Sessions?

No, not on its own. A LiveKit Agents session remembers everything inside one call and nothing between calls. ChatContext and userdata both end with the session, and a new AgentSession starts empty. To recognize a returning caller, your entrypoint must load that caller's history before the session starts and save the call after it ends.

That distinction is the whole design. LiveKit Agents is a framework for realtime voice agents, available for Python and Node.js. AgentSession runs the speech-to-text, language model, and text-to-speech stages, handles turn detection and interruptions, and lets the model call typed function tools while the caller is still on the line. It is built to run a call well, not to store what happened on the last one.

What does LiveKit Agents keep during a call?

Two things, and both belong to one session.

ChatContext is the conversation the language model sees. You build it with add_message, copy it, trim it with truncate (which always keeps the system instructions), and change it mid-session with update_chat_ctx. The full history across all agents in a session is available on session.history. For a long conversation, the documented advice is to summarize the context before passing it on, which is code you write plus one more model call.

Userdata is typed per-session state. You define a dataclass, pass it as the session's userdata, and read it as session.userdata or through RunContext inside a function tool. It is the right place for what the agent collects during this call: the account number the caller confirmed, the appointment time it booked, the option the caller picked. It ends with the session.

Inside one call, this is plenty. The agent can follow a conversation, remember what was said five minutes ago, and use what it collected earlier in the call. Nothing here is broken. It is simply scoped to the session.

What happens when the call ends?

The session ends, and its state goes with it. A new AgentSession begins with an empty chat context, and userdata belongs to the session that created it. When the same caller rings back next week, the agent knows only what your code loads for it.

This is where voice agents earn their reputation for amnesia. A caller explains a billing dispute on Monday, is promised a discount on Wednesday, and calls back on Friday to find an agent that has never heard of either. The audio pipeline worked. The memory did not, because there was no memory to work. Each call is a fresh job with its own context, its own state, and no link to the calls before it.

LiveKit is exact about where its responsibility ends. It documents the pattern for loading data before a session starts and points builders to third-party memory providers for the storage itself. The framework gives you the hooks. The store is your job.

What does a new session start with?

Only what your entrypoint puts into it. Before session.start(), you can build an initial ChatContext from stored history, read user-specific values from job metadata, room metadata, or participant attributes, and pass that context to the agent. During the call, the on_user_turn_completed hook gives you a place to run a lookup on each user turn, for speech-to-text, language model, and text-to-speech pipelines where the turn exists as text.

That is the shape of every working answer: load before the first reply, record during and after the call, and keep the caller identity stable across calls. The identity step is easy to skip and hard to recover from. A phone number, a SIP identity, or an authenticated app user must map to one stable caller id before the session starts, or two calls from the same person look like two strangers.

How do you give a returning caller memory?

1. Map the call to a stable caller id first. Resolve the phone number, SIP identity, or authenticated user to your own id in the entrypoint, before anything else. Every later read and write keys off that id.
2. Build the initial ChatContext from stored history. Load the caller's relevant past calls, decisions, and promises, and put a short version into the context before session.start(). Do this fetch once, before the caller hears anything, so no retrieval cost lands between a question and an answer.
3. Record the call as it happens. Save committed turns, tool calls and their results, and the durable facts the caller states: preferences, promises made, issues resolved, and issues left open. A transcript alone is raw material; the facts are what the next call needs.
4. Put that memory in a layer outside any single session when the agent also runs elsewhere. A built agent connects to Vilix AI with an API key as a Bearer header to https://api.vilix.ai/mcp and gets full memory with zero memory infrastructure to run. Retrieval is semantic as well as keyword-based, so it finds what was meant, not just what was typed, and it stores the full conversation history, not just extracted facts, so the reasoning behind a decision survives too.
5. Test the second call, not just the first. Call once, state a fact, hang up, and call again from a fresh session. If the agent greets the caller as a stranger, the load step is missing or keyed to the wrong id. If it repeats a resolved issue, the record step is missing.

Voice adds one constraint that text agents do not face: latency. A caller tolerates a pause of well under a second, so mid-call lookups must be fast and rare. That is why the preload matters so much. The expensive retrieval happens once, before the conversation begins, and the call itself only pays for short, targeted lookups when the model truly needs one.

The short answer

LiveKit Agents remembers one call well. It does not remember a caller across calls on its own. ChatContext and userdata are session-scoped, and a new session starts empty. Load the caller's history into the initial context before session.start(), record the call under a stable caller id, and put that memory in a store that outlives any single session.

Learn more about shared agent memory at https://vilix.ai/?utm_source=vilix-blog&utm_medium=article&utm_campaign=icp1-visibility