Does Apache Flink Remember Between Runs? State Survives in Checkpoints and Savepoints. The Next Run Still Starts With Only Its State.

Apache Flink remembers state on purpose, and in more detail than most streaming tools. A job can keep a running count per key, a list of recent events, or a half-built session, and that state survives failures, restarts, and even planned upgrades when a savepoint carries it forward. That state is real memory, engineered carefully. What it never keeps is anything an AI agent concluded while working outside the job, so a scheduled agent fed by a Flink pipeline still starts every run with only the state the job was coded to hold.

A stream processor built around state

Flink is a framework for stateful computation over unbounded and bounded streams. Any job that does more than transform one event at a time needs to remember earlier events or intermediate results, and Flink treats that need as a first-class feature. State comes in two main shapes. Keyed state is partitioned by key, so every customer, device, or account gets its own values, lists, and maps, readable only by operations on that same key. Operator state belongs to a parallel operator instance rather than to a key.

The values themselves live in a state backend. The HashMap backend keeps state as objects on the TaskManager JVM heap, fast and bounded by memory. The EmbeddedRocksDB backend keeps serialized state in an embedded RocksDB store on local disk, so state can grow far beyond the heap and checkpoints can be incremental, uploading only what changed. A newer disaggregated backend, ForSt, moves state toward remote storage. In every case, the backend answers the same narrow question: where do the values a job was programmed to keep sit while the job runs?

Notice the boundary in that sentence. Flink state holds exactly the fields the job code declares: a counter, a buffer, a deduplication set, a window accumulator. It is structured, typed, and scoped to one job. There is no place in it for a note, a reason, or a judgment that was never written into a state descriptor.

Checkpoints recover a job, and savepoints move it

Flink protects that state with snapshots. Checkpoints are automatic, taken at a configured interval, and capture a consistent picture of all operator state together with the positions reached in the input streams. If a TaskManager fails, the job restarts from the most recent completed checkpoint and continues with exactly-once state consistency. Checkpoints are recovery machinery: older ones are pruned as newer ones complete, and their format serves the runtime first.

Savepoints are the manual counterpart. An operator triggers one deliberately, in a canonical and portable format, to stop and later resume a job, upgrade its code, change its parallelism, or move it to another cluster, keeping state intact across the move. A savepoint stays on durable storage, in a file system or object store such as S3, until someone deletes it.

State can also expire by design. State TTL, available since Flink 1.6, lets a descriptor declare that a value untouched for a set time, counted in processing time, becomes inaccessible and is cleaned up later, in the background or on a snapshot. Read the whole system together and the pattern holds. Backends, checkpoints, savepoints, and TTL all answer runtime questions: where does this value live, how is it recovered after a failure, how long should it stay? None of them records what a reader outside the job decided, discovered, or produced while using the job's output.

What this means for a scheduled AI agent

Consider a common automation shape on Flink. A job reads a stream of support events all day, keeps per-customer state, and flags accounts whose event pattern crosses a threshold. Every hour, a scheduled agent takes the flagged accounts, checks each case against account history, and decides whether to offer a credit, open an investigation, or escalate to a human.

The Flink side of that work is covered: counts and patterns survive any restart, and a savepoint can carry them through an upgrade. The agent side is not. The next run begins an hour later with a fresh list of flags. Restoring a checkpoint could hand back the job's counters, but the agent would still have to redo the entire review to recover what it concluded last time. What the run actually needs is small and specific: this account already received a credit in March, the exception was approved for this reason, a direct call worked better than a template reply. None of that was ever declared as Flink state, so none of it is in any checkpoint or savepoint, no matter how reliably they are taken.

Operators often close the gap by writing summary records back into a stream for the job to keep. That helps one pipeline, in one schema, at whatever moment the summary was written. It breaks down when a second tool needs the same history, when the schema changes, or when the reasoning behind a decision matters more than the decision itself. A state backend answers whether a value exists for a key. Agent work depends on what has been learned so far. Those are two different jobs, and they need two different systems.

Give the agent a memory that outlives the state

Vilix AI is the shared memory layer that sits beside the job and keeps what the job never contained. Where Flink keeps the typed values a job was coded to hold, Vilix AI keeps the full conversation history, not just extracted facts, so the reasoning behind a decision stays readable next to the decision itself. The next run does not need to restore a snapshot and re-derive its conclusions. It retrieves the relevant past at the start of the run.

Vilix AI is cloud-hosted with zero infrastructure to run: no state backend to tune for memory, no checkpoint storage to size, and no savepoint schedule to manage. Agents connect over MCP, so the same memory follows the work everywhere, from a scheduled step reading Flink output to an n8n flow, a Make scenario, or a chat session the next day, across tools and devices. Every step reads and writes one shared state instead of a summary buried in one job's keyed state.

The working pattern stays simple. Events still stream through Flink, which remains the right tool for stateful, fault-tolerant stream processing. At the start of a run, the agent step pulls its context from Vilix AI. At the end of the run, it saves what it did, what it decided, and what it noticed. Checkpoints keep recovering the job, exactly as designed, and the agent forgets nothing, exactly as required.

Starting takes minutes. The free plan is free forever, and the 7-day Pro trial needs no credit card. Data stays under full control, with export of everything or deletion of everything available at any time. Let Flink remember the state until a checkpoint, a savepoint, or a TTL says otherwise. Let Vilix AI remember everything the state taught the agent.