DriftQ logo
DriftQ

Topics, Groups, Owners

The core consumer model behind DriftQ-Core.

DriftQ-Core’s consumer model is built around three concepts: topics, consumer groups, and owners. If these click, the rest of the API feels obvious.

Quick mental model: topic = stream, group = subscription, owner = worker identity.

Topics

A topic is a named stream of messages.

  • Producers write messages to a topic.
  • Consumers read messages from a topic.
  • Topics can have partitions to scale throughput and parallelism.

Partitions (quick mental model)

If a topic has multiple partitions, it’s effectively multiple parallel logs.

  • Each partition has its own ordered offsets: 0, 1, 2, 3, …
  • Consumers typically read per-partition in order.

Rule of thumb: ordering is per-partition, not across the entire topic.

Consumer groups

A group is a logical subscription to a topic. Each group maintains its own committed offset per partition.

That means:
  • g1 and g2 can independently consume the same topic.
  • Each group sees the stream from its own position, at its own pace.

Example

If topic t has partition p0 and messages exist at offsets 0..100:

group=g1  -> offset 10
    group=g2  -> offset 72
Both are valid and normal.

What this enables: multiple independent workflows reading the same event stream.

Owners

An owner identifies a specific consumer instance inside a group (think: a worker process).

DriftQ uses owner to enforce safe ack/nack behavior with leases:

  • /v1/consume requires owner
  • /v1/ack and /v1/nack must use the same owner
  • If the owner doesn’t match the current lease holder → 409 conflict

Why this exists

Without an owner + lease model, you risk:
  • two consumers processing the same message at once
  • a consumer acking work it doesn’t actually own
  • silent double-processing after crashes/retries

Minimal mental model

  1. Consumer calls consume with (topic, group, owner) → receives a leased message
  2. Same consumer later calls ack/nack with the same (topic, group, owner)
  3. If a different consumer tries to ack it → 409 conflict

Bottom line: owner is what makes retries safe and prevents “wrong consumer acked it.”

Common combinations

  • Scale processing: same group, different owners (multiple workers)
  • Independent workflows: different groups (separate offsets)
  • Strict safety: keep owner stable per consumer process instance