The problem: dual writes
await db.orders.insertOne(order); // ✅ committed
await broker.publish("OrderPlaced", ...); // 💥 process crashes → event lost foreverFlip the order and you can publish an event for data that never got saved. Two systems can't be updated atomically without 2PC.
How it works
Two relay options:
- Polling publisher: query
outbox WHERE sent = false. Simple, but adds DB load and latency. - Change Data Capture (CDC): tail the DB log (Debezium, MongoDB Change Streams, DynamoDB Streams). Lower latency, more infrastructure.
MongoDB note: multi-document transactions (replica set required) let you write the entity and the outbox doc atomically. Alternatively, embed pending events in the same document and use Change Streams.
Guarantees and consequences
- Delivery is at-least-once. The relay may publish, crash, and publish again, so consumers must be idempotent (dedupe on event ID).
- Ordering per aggregate is preserved if the relay publishes in order and the broker partitions by aggregate ID.
- A DB change must reliably produce an event
- Implementing sagas or event-driven integration
- You can't tolerate lost events
- Fire-and-forget telemetry where loss is acceptable
- You already use event sourcing (the event store is the outbox)
Sources & further learning
Videos, courses, docs and books I recommend for this topic.
Related topics
Saga Pattern
Keep data consistent across services without distributed transactions — a sequence of local transactions, each with a compensating action if a later step fails.
Event-Driven Architecture
Services announce facts ("EntryPublished") and others react asynchronously — decoupling producers from consumers in time, space and knowledge.
Idempotency
Doing an operation twice has the same effect as doing it once. The foundation of safe retries, at-least-once messaging and reliable APIs.