Distributed Systems

An evolving map of hard-won lessons about building systems that span more than one machine - where the network, not your code, decides the failure modes.

Failure handling & load

• Timeouts are a design decision, not a default
• Retries without backoff turn a blip into an outage
• A circuit breaker protects the caller, not the callee
• A queue is a loan against future capacity

Delivery & correctness

• Idempotency is a property, not a mechanism
• At-least-once delivery makes deduplication your job
• The outbox pattern gets you at-least-once without a shared database

Data & coupling

• A shared database is distributed coupling in disguise
• Cache invalidation fails at boundaries you don't own