In-memory server for agentsPostino Port 2790 v0.1.0

Guides

Queues and work distribution

Postino offers four ways to hand work from one agent to another. They differ in how much they guarantee.

MechanismOrderIf the worker crashesBest for
List with BLPOPfirst in, first outthe item is lostbest-effort jobs, fast and simple
List with BLMOVEfirst in, first outthe item stays in a processing listreliable jobs with a little bookkeeping
Sorted set with BZPOPMINby prioritythe item is lostthe most urgent work first
Stream with a consumer groupby idthe entry stays pending and can be claimedseveral workers, acknowledgements, retries

For work aimed at one named agent, its inbox already provides leases, redelivery, and dead letters.

A list as a queue

Producers push on the right; workers pop from the left and block while the queue is empty:

postino db RPUSH jobs '{"task": 42, "action": "test"}'
postino db BLPOP jobs 30
1) "jobs"
2) "{\"task\": 42, \"action\": \"test\"}"

BLPOP takes several keys and serves the first non-empty one, so a worker can watch jobs:urgent jobs:normal and drain the urgent list first. The last argument is the timeout in seconds; 0 waits as long as the transport allows (60 seconds over HTTP and MCP).

Reliable processing with BLMOVE

Move the job to a per-worker processing list instead of removing it. If the worker dies, the job is still there:

postino db BLMOVE jobs jobs:processing:w1 LEFT RIGHT 30
# ... work on it ...
postino db LREM jobs:processing:w1 1 '{"task": 42, "action": "test"}'

A supervisor that notices a dead worker pushes its processing list back with LMOVE jobs:processing:w1 jobs RIGHT LEFT.

Priority queues

Score each job by urgency and pop the lowest score first:

postino db ZADD work 1 deploy-hotfix 5 refactor-tests 3 update-docs
postino db BZPOPMIN work 30
1) "work"
2) "deploy-hotfix"
3) "1"

A timestamp as the score turns the same structure into a schedule. ZRANGE work -inf <now> BYSCORE lists what is due.

Streams with consumer groups

A stream keeps every job, and a consumer group gives each job to exactly one member and tracks it until it is acknowledged. This is the most robust choice when several agents share the work.

Create the group once. $ starts it at the end of the stream, 0 at the beginning:

postino db XGROUP CREATE tasks workers '$' MKSTREAM

Producers append:

postino db XADD tasks '*' task 42 action test

Each worker reads under its own name. > means entries nobody in the group has received yet:

postino db XREADGROUP GROUP workers builder-1 COUNT 1 BLOCK 30000 STREAMS tasks '>'

The entry is now pending for builder-1. Acknowledge it when the work is done:

postino db XACK tasks workers 1793431332310-0

Recovering stalled work

XPENDING tasks workers shows how many entries are pending and for whom. A worker that finds entries idle for longer than they should take claims them:

postino db XAUTOCLAIM tasks workers builder-2 60000 0 COUNT 10

This hands every entry that has been pending for more than 60 seconds to builder-2, which then processes and acknowledges it. Each entry counts its deliveries. An entry delivered many times probably cannot be processed: move it aside with XADD tasks:dead ... and XACK it.

A worker that restarts under the same name should first read its own pending entries with id 0 instead of >, so it finishes what it had started.

See Streams for reading without groups, trimming, and following a stream live.