Does DolphinScheduler Remember Between Runs? Parameters Pass Inside One Instance. The Next Instance Starts Blank.

Apache DolphinScheduler remembers a great deal about your workflows: definitions, schedules, and a record of every instance it has run. What it does not do is hand the next scheduled instance the working context of the last one. The gap is easy to miss, because inside a single instance, passing values between tasks works well. Between instances, the slate is clean again.

What DolphinScheduler actually keeps

The platform keeps its records in a metadata database, MySQL or PostgreSQL in a real deployment. Workflow definitions live there, along with schedules, tenants, data sources, and the history of workflow and task instances. When an instance finishes, its status, timing, and task logs stay on record. An operator can open yesterday's instance, see which tasks ran, read the logs, and check the values its parameters ended with.

Global parameters sit on the workflow definition itself: named values, such as a business date or an environment flag, that tasks across the workflow can reference. System parameters add values the scheduler fills in at runtime, such as ${system.biz.date}, which resolves from the scheduled time of that instance. These are configuration, set in advance or derived from the clock. They describe how an instance should run, not what a previous instance learned.

So the stored picture works as an audit trail: what was defined, when it ran, and how each task ended. An audit trail answers questions after the fact, when a person goes looking. It is not loaded into the next run as a briefing.

Passing values inside one instance works

Within a single workflow instance, DolphinScheduler has a real parameter system. A task can define local parameters for its own inputs. An upstream task can pass a value downstream by declaring an OUT parameter and emitting it in the format ${setValue(key=value)}; a dependent downstream task then reads that value. When names collide, resolution follows a clear priority: local parameter first, then the parameter context passed from upstream, then the global parameter.

That machinery is genuinely useful. A SQL task can hand a row count to a shell task, and a shell task can hand a file path to the step that processes it. For the length of one instance, the workflow has something that feels like memory.

The boundary is the instance itself. When the instance ends, the OUT value remains visible in its record, but nothing promotes it into the starting state of the next one.

What the next instance starts with

When the schedule fires again, DolphinScheduler creates a new workflow instance. Global parameters resolve again from the definition. System parameters resolve again from the new scheduled time. Local parameters come from the task definitions. The downstream parameter context starts empty, because there is no upstream task yet in this instance.

Nothing in that starting state says what the last instance decided. Backfill and complement runs follow the same rule: each date gets its own instance, and one date's instance does not brief the next. Serial execution modes only control whether instances overlap; they do not pass working context.

Consider a nightly workflow with an AI step that reviews open incidents and drafts follow-ups. On Monday the step finds that one alert was already explained by a maintenance window, drafts a question for a second incident, and notes that a third was promised an update on Tuesday. DolphinScheduler records that Monday's instance succeeded. On Tuesday, the new instance starts from its parameters and fresh query results. The maintenance explanation, the question already sent, and the promise are not in its starting state, because none of them are parameters. The AI step can ask the already-answered question again, or miss the follow-up it promised, while the platform logs another successful instance.

The layer operators end up building by hand

The standard fix is to persist the context yourself. The last task writes a summary row, a status file, or a key in an external store, and the first task of the next instance reads it back into the AI step's prompt. That works, until it becomes a second system: a schema to maintain, a prompt builder to test, and a cleanup policy for stale rows.

It also stores a compressed ending. A summary row keeps the conclusion of a run. The next decision often depends on the exchange in the middle: the wording a person used when correcting the agent, the option that was rejected and why, the follow-up that was promised. If only final values were written, the next instance receives the result without the reasoning.

And the table belongs to this one workflow. The same operator usually runs AI steps in other schedulers, in n8n or Make scenarios, and in AI tools used directly. A correction given in a chat tool never reaches tonight's DolphinScheduler instance unless someone copies it across.

Give the instance a memory that lives outside it

The cleaner shape is to keep scheduling and execution in DolphinScheduler, and give the AI steps a shared memory that lives outside any single instance, workflow, or tool, which every run reads at the start and updates at the end.

Vilix AI is built for that layer. It is cloud-hosted with zero infrastructure to run, so there is no extra database to provision or keep alive between schedules. It is available over MCP, so the same memory follows the work across a DolphinScheduler task, another automation platform, and the AI tools used directly. One memory, reachable everywhere the work happens.

Vilix AI stores full conversation history, not just facts or final parameter values. A scheduled agent can revisit the actual exchange: what was asked, what was answered, what a person corrected, and what was promised. That history stops a Tuesday instance from repeating Monday's settled question, and makes a promised follow-up visible to the run that owes it.

In a DolphinScheduler setup the pattern stays small. The AI task retrieves the relevant history for that service, project, or workflow before it acts, does the work with the past in view, and saves the new exchange and outcome back before the instance ends. DolphinScheduler keeps defining, scheduling, and logging. The continuing story lives in the memory layer, where the next instance in any tool can find it.

Data in Vilix AI can be exported or deleted at any time. The free plan is free forever, and Pro offers a 7-day trial with no credit card required, so the pattern can be tested on one nightly workflow first.

The short answer

Does DolphinScheduler remember between runs? As records, yes: definitions, schedules, instance history, and logs persist in the metadata database, and parameters pass cleanly between tasks inside one instance. As working memory for an AI step, no: the next instance resolves its parameters from the definition and the clock, and starts without anything the last instance learned, decided, or promised.

Let DolphinScheduler do scheduling and orchestration. Give the agents inside its workflows a shared memory outside them, and each instance can start where the last one actually left off.

Start at Vilix AI and connect one scheduled workflow first.