Queue

queue/v1 is a partitioned message queue built on an immutable edge table. Messages are ordered within a partition and parallel across partitions, with an explicit lifecycle: poll → process → commit.

See the Queue API Reference for endpoints.

Architecture

A queue is an immutable edge table with a fixed schema and a fixed access pattern. Define a queue, and each part is exactly one thing on the underlying immutable edge table:

queue/v1	Immutable edge table
namespace	database
queue	table, type IMMUTABLE_EDGE
partition = xxhash32(key) mod partitions	source (LONG)
message id	target (STRING) — a server-assigned ULID
seq (order / due time)	property (LONG) — the single seq ASC index
value	property (STRING) — opaque JSON payload
enqueue	INSERT one index row (append, lock-free)
poll (seq > offset, optional until)	bounded scan of the seq ASC index (OUT)
commit (seq <= offset)	scan-delete over that index range

The backing table is append-only and index-only: an enqueue is a single index-row write, and a poll is a single bounded index scan. There is no per-message state, no ack flags, and no consumer registry.

Partitioning

A message is routed by its key:

partition = xxhash32(key) mod partitions

• Messages with the same key land in the same partition and are read back in seq order.
• partitions is fixed at queue creation (default 30 = 2·3·5, whose divisors allow many balanced consumer-shard splits).
• Consumers fan out by polling partitions 0 .. partitions-1 independently.

Consumer Lifecycle

poll(partition, offset) → process the batch → commit(partition, offset = batch offset)

1. Poll scans one partition forward by seq. offset is exclusive (seq > offset); the response returns the batch and its next offset (the highest seq seen). An optional until bound (inclusive) restricts a poll to due messages when seq encodes a due time.
2. Process the batch.
3. Commit up to the batch’s offset: every message with seq <= offset is deleted. There is no separate ack state; the delete is the committed position, and the same operation serves retention.

Caution

Commit deletes a prefix of the partition, so it assumes one logical consumer per partition (Kafka-style). Independent consumers sharing a partition would delete each other’s unprocessed messages.

Delivery Semantics

• At-least-once: commit only after processing. If a consumer crashes between poll and commit, the next poll re-reads the uncommitted messages.
• Ordered within a partition: polls return messages in seq order; there is no ordering across partitions.
• Ordering key: seq is a client-supplied, increasing key (a timestamp, LSN, or sequence number). Uniqueness is not required; the server-assigned ULID id keeps same-seq messages distinct.
• Commit is a real delete: after a commit, re-polling from the start of the partition does not return the committed prefix.
• No CDC: immutable edge tables do not emit CDC; the queue itself is the log.

Use Cases

A durable, partitioned, append-only log with a due-time filter — a log primitive, not a full message broker.

Use case	Fit	Notes
Refresh / delay scheduler	Good	until withholds not-yet-due messages; commit clears fired ones
Durable per-partition log	Partial	Replay by offset; retention is manual via commit (a delete), one consumer per partition
Ack-based work queue	No	No per-message ack, visibility timeout, redelivery, or dead-letter

Not provided: message ack / redelivery, deduplication, independent consumer groups on one partition, and server-side long-poll or streaming.

Next Steps

• Queue API Reference: Endpoints
• Schema: Immutable edge tables
• Mutation API Reference: The underlying scan-delete primitive