Skip to content
All articles

Kafka · Topics and partitions

Ordering is a property of a partition

A Kafka topic is a name. A partition is the log, and ordering lives there. Twelve partitions are twelve independent logs that happen to share a name, so across them no order is defined.

· 3 min read

Topic orders.v1 drawn as three independent logs. A producer picks a partition with hash(key) % 3. Partition P0 holds records 0 to 4, P1 holds 0 to 2 and P2 holds 0 to 5, each with its next append at the right. Two readers keep one integer per partition: fulfilment is at P0:3, P1:1, P2:5 and analytics is at P0:0, P1:0, P2:0, and reading deletes nothing. Order lives inside one partition; across partitions there is no defined order.

Four nouns carry most of Kafka, and one of them becomes expensive to change.

A topic is a named stream such as orders.v1. It is a logical name and has no single ordering of its own. A partition is the actual log, an append-only sequence of messages on disk, and it is where ordering lives. A topic with twelve partitions is twelve independent logs that happen to share a name. An offset is a record's position in one partition, a monotonically increasing integer the broker assigns on append. A broker is one server, and spreading partitions across brokers is how a topic scales past one machine's disk and network.

One sentence explains the difficulty

Everything difficult about Kafka follows from this: ordering is a property of a partition, not a topic. A consumer reads the records present in one partition in offset order. That is broker append order, not automatically business-time order across producers, and retries can reorder batches when producer idempotence is disabled and more than one request is in flight.

Records in different partitions have no defined relative order. If you see a merged order, it is a property of timing, not a Kafka guarantee.

What the diagram shows

The producer chooses a partition with hash(key) % 3. Each of the three logs grows on its own: P0 holds offsets 0 to 4, P1 holds 0 to 2 and P2 holds 0 to 5.

Readers keep one integer per partition. The fulfilment group is at P0:3 P1:1 P2:5, and analytics is at P0:0 P1:0 P2:0. Reading does not delete anything, so a new group set to start from the earliest offset replays everything retention still holds.

Mental Model

Offsets name positions inside one partition, not across the topic. Ask "in which partition?" before you ask "in what order?", because the second question has no answer across partitions.

Offsets are useful only while that log history exists. Retention and exceptional log truncation can remove the record an offset once named, and compaction does not renumber the records that survive.

Get one diagram a week

A short article built around one engineering diagram, from the same library as these courses.

One diagram-led article a week on AI and systems engineering. We email you once to confirm, and every newsletter has an unsubscribe link. Privacy policy