Does Arq Remember Between Runs? A Job Keeps Its Arguments in Redis, but the Next Job Never Gets the Last Job's Findings

Arq is a job queue and RPC library for Python, built on asyncio and Redis. Code enqueues a job by name, a worker picks it up, and the function runs outside the request that asked for it. Jobs can be deferred, retried, and scheduled on a repeating cron pattern. For a team whose enqueued function does AI work, that raises a practical question: when the same job runs again tomorrow, does it know anything about today's run? No. Arq stores the job in Redis until it runs, and it can keep a finished job's result for a short window. The next job still starts with only the arguments it was enqueued with. What the last run found, decided, or already handled is not passed along.

What an arq job actually carries into Redis

Enqueueing a job serializes a small, exact package. That package holds the function name, the positional and keyword arguments, a try counter, and the enqueue time. Arq writes that package to a Redis key for the job and adds the job ID to the queue, which is a sorted set scored by the time the job should run. A worker polls that set, claims due jobs, and runs them.

Read that package closely and its scope becomes clear. Every field answers an execution question. Which function should run. With which inputs. At what time. Which attempt is this. None of the fields answers a work question: which records did yesterday's run already process, which exception did a person approve, or which conclusion should today's run build on instead of rediscovering? The job knows how to start. It does not carry a briefing from the job before it.

Custom job IDs and deferral follow the same boundary. A custom ID can stop the same job from being enqueued twice until the first one finishes and its result clears. Deferring by a duration or until a date changes when a job runs. Both are useful controls over timing and duplication, and neither one adds anything the previous run learned to the next run's inputs.

A result is kept for the caller, not handed to the next job

Arq does keep one thing after a job finishes: its return value. The worker stores the result in Redis under a result key, and code that holds the job handle can poll for it, check the job's status, or read its info. That is how an enqueueing process learns that a job returned 42, or raised an exception, without running the work itself.

The limits of that result matter for recurring work. On a worker, the default setting keeps results for 3600 seconds, one hour. For cron jobs the default is sharper still: the result is not kept at all. Even inside the window when a result does survive, nothing delivers it to the next job. The queue never attaches last time's return value to this time's arguments on its own. A return value is also a narrow thing to build a past from: one serialized answer, not the reasoning inside the run, the sources checked, or the correction made halfway through.

Cron repeats the call, and ctx is a worker resource, not a past

Arq cron jobs make the gap easiest to see. A function registered with cron runs at set times, for example at 9, 12, and 18 past the hour, and each firing enqueues a fresh job for that function. The schedule is real and reliable. What repeats is the call itself, with the same setup each time. Tuesday's firing does not open Monday's run, summarize it, and brief itself before starting.

The one place that can look like shared memory is ctx, the dictionary passed as the first argument to every job function. Ctx is set up once per worker process. An on_startup coroutine typically puts shared resources into it, such as an HTTP client session, and on_shutdown closes those resources when the worker stops. Jobs in that process can all reach the same client, pool, or connection through ctx.

That sharing is about resources inside one running worker, not history across runs. Ctx is rebuilt when the worker restarts, and it holds tools for doing work, not a record of work already done. A session object in ctx can fetch a page. It cannot tell the job which pages yesterday's run already fetched. Pessimistic execution draws the same line: arq does not remove a job until it has succeeded or failed, so an interrupted job stays queued and runs again. That protects a single job from being lost. It does not connect finished jobs to future ones.

Where this leaves a scheduled AI job

Picture a nightly arq job that checks new signups, enriches each account, and drafts follow-up notes. On Monday it processes thirty accounts, finds that five match an exception a person approved, and learns that one domain should never be contacted again. On Tuesday the cron firing arrives with the same function name and arguments it always carries. Monday's accounts, the approved exception, and the blocked domain are nowhere in that package. Unless the job reads them from a store the team built, it repeats work already done or acts without the correction it was already given. Teams often patch this with a database table or a Redis key they manage themselves. That patch can hold a few fields for one job. It rarely holds the full context behind those fields, and it does not travel when a second tool needs the same past.

Give the job a memory that exists outside the queue

Keep arq for what it does well, dependable async execution on Redis, and give the work itself a separate memory layer that every run reads before acting and writes to after finishing. That layer is Vilix AI.

Vilix AI is cloud-hosted, so there is zero infrastructure to run or maintain. The same memory is available everywhere over MCP, so an arq job, a coding agent, and a chat assistant can all work from one shared past instead of separate local notes. It stores full conversation history, not just extracted facts, so a later run can revisit what was actually said and decided, in context.

The nightly job changes in one plain way. Before enriching anything, it retrieves the relevant history from Vilix AI: the accounts already processed, the approved exception, the blocked domain. After finishing, it saves the new run back. The next firing still arrives as a fresh arq job, and that no longer matters, because the briefing lives in memory the job reads first.

Vilix AI offers a free plan forever, plus a 7-day Pro trial with no credit card required. Data stays under the operator's control, with the ability to export everything or delete it anytime. Let arq keep the schedule. Let shared memory carry the past. Start with Vilix AI and give the next run the last run's findings from the start.