🌿Timeouts are a design decision, not a default
An unset timeout is not "no timeout." It's the longest timeout in your stack - inherited from whatever library gives up last. So one slow dependency can pin a thread, exhaust the connection pool, and cascade a local hiccup into a system-wide stall. The absence of a decision is still a decision, and usually the worst one.
A good timeout is derived, not guessed: shorter than your caller's timeout (so you fail before they give up on you), and longer than the p99 of a healthy call (so you don't kill requests that would have succeeded).
Timeouts must also compose. A chain of five services each patiently waiting 30 seconds means the user waits over two minutes for an answer that stopped being useful after the first ten. Budget the deadline across the whole call chain and pass it down.
Set them deliberately, and pair them with backoff so a timeout doesn't immediately become a retry storm.
Related
Linked from
- 🌿 Distributed Systems
An evolving map of hardwon lessons about building systems that span more than one machine…
- 🌿 Retries without backoff turn a blip into an outage
A single failed request retried immediately is harmless. Ten thousand clients retrying a…
- 🌿 A circuit breaker protects the caller, not the callee
It's tempting to think a circuit breaker shields a struggling downstream service from load.…
- 🌿 Cache invalidation fails at boundaries you don't own
You can invalidate what you control. Across a boundary you don't own (a CDN, a client, a…
- 🌿 A queue is a loan against future capacity
Putting a queue in front of an overloaded service feels like relief spikes disappear, callers…