Does Apache RocketMQ Remember Between Runs? Messages Stay in the CommitLog for Days. The Next Run Still Starts With Only Its Message.

Apache RocketMQ remembers the message, in full, on disk. A broker writes every message it receives into a shared physical log, keeps it for the whole storage duration whether anyone consumed it or not, and lets a consumer go back and read it again inside that window. That is real retention, and it is stronger than a queue that deletes on delivery. What RocketMQ never keeps is everything around the message: why it was sent, what the last run decided while handling the previous one, and what should happen differently this time. The next run still starts with only its message.

Every message goes into one physical log on the broker

RocketMQ separates the work into producers, brokers, and consumers, with a name server holding the routing information that connects them. Producers send messages to a topic. The broker that owns that topic appends each message, in arrival order, to the CommitLog, a set of physical files on the broker's local disk under the configured store path. A lighter logical index, the ConsumeQueue, points into the CommitLog by topic and queue so consumers can pull efficiently without scanning the physical log.

That design explains both the strength and the limit of RocketMQ memory. The strength is durability with order: messages for many topics share one sequential log, which is fast to write and straightforward to replicate, and a consumed message is still physically there. The limit is scope. The CommitLog records message bodies and properties, in arrival order. It does not record processing. Nothing in it notes what a consumer concluded, which branch it took, or what it promised to do next.

Consumed does not mean deleted

By default, RocketMQ retains all messages for the storage duration. When a consumer finishes a message and commits its result, the broker logically marks that position as consumed instead of deleting the message at once. The consumer offset, the number that says how far a consumer group has read in a queue, simply moves forward. The bytes stay in the CommitLog until they age out.

Because the bytes stay, backtracking works. A consumer group can reset its offset and consume the same messages again inside the retention window, the standard recovery after a bad deploy or a downstream outage. Message query and tracing can also find a specific message by key or ID while it is still stored. For a scheduled automation, a failed run's input can be replayed exactly, without asking the producer to send it again.

Replay is still not memory. Re-reading the same message gives the next run the same starting point as the last run, not the accumulated knowledge of the last run. If the previous attempt already checked three sources, ruled out two causes, and narrowed the problem to one account, the replayed message contains none of that. The work starts over from the same square, only later.

The clock deletes by age, not by understanding

Retention in RocketMQ is a storage duration managed per broker node. It starts when the message is produced and runs independently of consumption status: a message consumed in the first minute and a message nobody read expire on the same schedule. When the duration passes, the broker deletes the oldest files in a rolling cleanup. When disk space runs short, it deletes earlier to protect the service, so actual lifetime can be shorter than configured.

Exact defaults depend on the deployment. The open-source broker's file retention is commonly set to 72 hours, and managed RocketMQ services often default to 48 hours with a configurable ceiling. The number matters less than the principle behind it: the broker keeps messages for a window sized for delivery problems, redelivery, and short-term replay. It was never sized, or shaped, to be the long-term memory of the systems consuming from it.

RocketMQ adds delivery features around that core. Ordered messages preserve sequence inside a queue. Transactional messages coordinate a send with a local transaction. Delayed messages wait until their delivery time. Failed consumption retries, and messages that exhaust retries move to a dead letter queue. Each feature makes delivery more reliable. None records what a run learned.

What a scheduled agent actually needs between runs

Consider a scheduled automation that wakes every few hours, pulls a RocketMQ message describing new work, and processes it with an AI agent: checking accounts, comparing sources, drafting output, and noting exceptions. RocketMQ handles the trigger well. The message arrives in order, survives a broker restart, and can be redelivered if the run fails.

The gap appears in the second week, not the first run. The agent needs to know that this customer was already contacted twice, that a particular data source was wrong last time and had to be corrected by hand, and that a decision made three runs ago changed how this category of work should be handled. None of that lives in a topic. The topic holds the next message. The history of decisions, evidence, and corrections has no queue to sit in, so teams end up rebuilding it from logs, spreadsheets, and re-briefing prompts at the start of every run.

That re-briefing is the hidden cost. Each run pays it in time, tokens, and repeated mistakes, because the agent cannot tell work that is genuinely new from work that only looks new.

Give the runs a memory built for runs

That is the gap Vilix AI fills, alongside the broker rather than instead of it. RocketMQ keeps delivering the trigger. Vilix AI keeps what the runs learned from every trigger before it, as one shared memory every agent and run reads from.

Vilix AI is cloud-hosted with zero infrastructure to run, so there is no second storage system to deploy and maintain next to the broker cluster. Agents connect over MCP, and the same memory follows them everywhere: the same context is available to a scheduled automation at night, a coding agent in the morning, and a chat session in the afternoon, because the memory lives in one hosted layer rather than inside any single tool.

It also stores the right material. Vilix AI keeps full conversation history, not just extracted facts, so a later run can revisit what was actually said, checked, and decided, in context. Decisions, corrections, and dead ends stay retrievable by meaning, ready at the start of the next run instead of reconstructed under pressure.

Starting takes minutes. There is a free plan that stays free forever, and a 7-day Pro trial with no credit card required. Data stays under the operator's control, with export or delete at any time. The broker window can stay what it was designed to be, a few days of reliable delivery and replay, while the memory of the work stops expiring with it.

RocketMQ remembers the message, completely, until the clock removes it. Let it do that job. Give the agents consuming from it a memory that keeps the rest: what the last run found, what it decided, and what the next run should do differently because of it.