Skip to content
All articles

Transactions · Dual writes

Paid in the database, never announced

Commit the order, then publish its event. When the publish fails, the order is paid and nobody downstream is told. No ordering of two separate writes closes that gap.

· 3 min read

A dual write. The order service first commits the order row to the orders database, then publishes an event to the event broker, which feeds email and the warehouse. For ord_1001 the commit, the publish and the email all succeed. For ord_1002 the commit succeeds but the publish fails, so nobody is told. Totals: 2 orders committed, 1 event delivered, 1 email sent, 1 order with no event. No exception reached the customer and no alert fired. Publishing first instead means a crash can announce an order that doesn't exist.

When a customer pays, your order service does two things: it commits the order row, then it publishes an event so the email service can confirm the order and the warehouse can pick it. Two systems, two writes, and no transaction spanning both.

The run

The course's lab, outbox_dual_write.py, does exactly that for two orders:

commit order row for ord_1001
published event for ord_1001
consumer: sends confirmation email for ord_1001
commit order row for ord_1002
publish FAILED for ord_1002 — the row is committed and nobody was told

orders committed    : 2
events delivered    : 1
emails sent         : 1
orders with no event: 1   <- silent drift

ord_1002 is paid in your database and does not exist as far as every downstream system is concerned. No exception reached the customer, and no alert fired. The drift is silent, which is what makes it dangerous: you find it weeks later, when someone asks why a paid order never shipped.

Swapping the order does not help

Publish first and commit second, and a crash in between announces an order that does not exist. There is no ordering of two independent writes that removes the window between them. The window is the problem.

Mental Model

Whenever one action writes to two systems and no transaction spans both, ask what each system believes if the process dies between the two writes. If the answers differ, you have a dual write, and a lucky test run proves nothing about it.

Where the course goes next

The repair is to stop making two independent writes. The course's next section makes the event a row written in the same database transaction as the order, then publishes it from there with a separate relay, so a crash delays the event instead of losing it. That trade has a price the lab makes visible: an event can now be delivered twice, so the consumers that receive it have to deduplicate.

Get one diagram a week

A short article built around one engineering diagram, from the same library as these courses.

One diagram-led article a week on AI and systems engineering. We email you once to confirm, and every newsletter has an unsubscribe link. Privacy policy