20 reusable building blocks, each framed the way an interview answer should be: the problem it solves, the mechanism, and - the part that actually distinguishes candidates - when not to reach for it.
Read Replicas
Scaling
Problem: Read traffic saturates the primary database.
Solution: Replicate the primary asynchronously to N read-only copies and route reads to them, keeping writes on the primary.
Use when
Read:write ratio above roughly 10:1
A few hundred milliseconds of staleness is acceptable
You have not yet exhausted vertical scaling and caching
Avoid when
×Writes are the bottleneck - replicas do not help at all
×Every read must reflect the latest write with no session routing
×Replication lag would exceed your consistency window under load
Postgres streaming replicationMySQL read replicas behind a proxyAurora reader endpoints
Horizontal Partitioning (Sharding)
Scaling
Problem: A single primary can no longer absorb the write volume, or the data no longer fits on one machine.
Solution: Split rows across independent databases by a shard key, mapping keys to logical shards and logical shards to physical servers.
Use when
Write throughput or data size genuinely exceeds one node
A shard key exists that appears on most queries
You can tolerate losing cross-shard joins and transactions
Avoid when
×Replicas, caching and vertical scaling still have headroom
×Queries have no common key, forcing scatter-gather on everything
×Strong cross-entity transactions are core to the domain
Vitess for MySQLCitus for PostgresCassandra partition keys
Consistent Hashing
Scaling
Problem:hash(key) % N remaps almost every key whenever the node count changes.
Solution: Map keys and nodes onto a ring with virtual nodes; a key belongs to the first node clockwise, so resizing moves only ~1/N of keys.
Use when
Node membership changes - autoscaling, failures, rolling deploys
You need cache locality or sticky routing by key
Distributing data across a dynamic cluster
Avoid when
×The node set is genuinely fixed
×You need range scans across keys
×A small cluster where rendezvous hashing is simpler and more even
CassandraDynamoDBmemcached clientsEnvoy ring hash
Cell-Based Architecture
Scaling
Problem: A single bug, poison record or bad deploy can affect every user at once.
Solution: Partition users into independent cells, each a complete vertical stack, so failures are contained to one cell's population.
Use when
Blast radius is a first-class requirement
Very large multi-tenant systems
You want to canary at the level of infrastructure, not just code
Avoid when
×The system is small enough that a rollback is fast
×Cross-cell queries would be common
×The operational overhead exceeds the reliability benefit
AWS service cellsSlack shardsSalesforce pods
Cache-Aside
Data
Problem: The database answers the same expensive question repeatedly.
Solution: The application checks the cache, reads the database on a miss and populates the cache; on write it updates the database then deletes the key.
Use when
Reads dominate writes and the same keys are hot
The system must keep working when the cache is unavailable
You want to cache only what is actually read
Avoid when
×Data changes more often than it is read
×Even brief staleness is unacceptable
×The underlying read is already sub-millisecond
Redis in front of PostgresMemcached at Facebook
Materialised View / Precomputation
Data
Problem: An expensive aggregation is recomputed on every read.
Solution: Compute the result once on write or on a schedule, store it, and serve reads from the stored result.
Use when
The same expensive query runs far more often than the data changes
The result space is small enough to precompute
Slight staleness is acceptable
Avoid when
×Results are highly personalised, so the space explodes combinatorially
×Writes are frequent, making refresh cost dominate
×The query is cheap already
Postgres materialised viewsPrecomputed news feedsRollup tables for analytics
Change Data Capture
Data
Problem: Derived stores - search indexes, caches, warehouses - must stay in sync without dual writes.
Solution: Tail the database's write-ahead log and publish every change as an event stream that derived stores consume.
Use when
Several derived stores need the same changes
You must avoid dual writes from application code
You need an ordered, replayable history of changes
Avoid when
×One derived store and a transactional outbox is simpler
×The team cannot operate the CDC pipeline
×Schema changes are frequent and would break downstream consumers
Problem: One slow dependency consumes the whole thread or connection pool and takes down unrelated endpoints.
Solution: Give each dependency a bounded, separate resource pool sized by its importance.
Use when
Several dependencies with different reliability profiles
Some features are optional and some are critical
You have seen one slow call take down a whole service
Avoid when
×There is only one dependency
×Idle capacity per pool is genuinely unaffordable
Per-dependency thread poolsEnvoy connection pools per cluster
Graceful Degradation
Resilience
Problem: A non-essential dependency failing takes down the whole user experience.
Solution: Define, per feature, what happens when its dependency is unavailable - hide it, use a default, serve stale, or fail closed if correctness demands.
Use when
Some parts of the page are cosmetic or supplementary
Partial success is more useful to the user than an error
You can define the degraded state precisely, in advance
Avoid when
×A wrong answer is worse than no answer - prices, balances, permissions
×Silent degradation would hide a real outage from operators
Netflix rendering without personalised rowsAmazon hiding recommendations under load
Rate Limiting
Resilience
Problem: A single abusive or buggy client can consume capacity intended for everyone.
Solution: Enforce a token bucket per client, endpoint and global dimension, returning 429 with Retry-After when exceeded.
Use when
Any public API
Any endpoint with expensive operations
Protecting a downstream with a hard capacity ceiling
Avoid when
×Internal traffic already bounded by a known caller set
×The limit would block security-critical flows such as password reset
Stripe API limitsGitHub APICloudflare rate limiting
Transactional Outbox
Messaging
Problem: You must commit a state change and publish an event atomically, but they live in different systems.
Solution: Insert the event as a row in the same transaction as the state change; a relay reads the outbox and publishes at-least-once.
Use when
Any time a database write must produce an event
You cannot tolerate a published event for an uncommitted change
Consumers are idempotent
Avoid when
×The event is purely advisory and loss is acceptable