When do you scale vertically and when do you scale horizontally?

Scale vertically first because it requires no code changes and modern machines are enormous - hundreds of cores and terabytes of RAM. Switch to horizontal when you hit the ceiling of a single machine, when the cost curve turns superlinear (the biggest instances cost disproportionately more per unit of capacity), or when you need redundancy, because one machine is a single point of failure regardless of how big it is. In practice, redundancy forces horizontal scaling long before capacity does. | Vertical (scale up) | Horizontal (scale out). Change required | None - resize and restart | Statelessness, load balancing, coordination. Ceiling | Hard: largest available machine | Practically none. Cost curve | Superlinear at the top end | Roughly linear. Failure domain | One machine = total outage | Lose one of N, degrade gracefully. Downtime to scale | Usually a restart | Add an instance, no disruption. Best for | Databases, early-stage apps, stateful services | Stateless app tiers, workers, caches Redundancy, not capacity, is usually the trigger: A single modern server can handle traffic that would have needed a datacentre in 2005. Most teams go horizontal not because one box cannot cope, but because one box means one power supply, one kernel, one deploy that can go wrong. The moment you need to survive losing a machine, you need at least two - and that is a horizontal architecture whether or not you needed the capacity. The one precondition: statelessness A stateless server keeps no request-specific data in local memory or on local disk between requests. Any instance can serve any request; instances are interchangeable and disposable. This is what makes load balancing, autoscaling, rolling deploys and instance failure boring instead of catastrophic. State | Wrong place

Horizontal vs Vertical Scaling and Stateless Architecture

BeginnerscalingarchitecturecostAsked at Amazon, Netflix

30-second answer

Scale vertically first because it requires no code changes and modern machines are enormous - hundreds of cores and terabytes of RAM. Switch to horizontal when you hit the ceiling of a single machine, when the cost curve turns superlinear (the biggest instances cost disproportionately more per unit of capacity), or when you need redundancy, because one machine is a single point of failure regardless of how big it is. In practice, redundancy forces horizontal scaling long before capacity does.

	Vertical (scale up)	Horizontal (scale out)
Change required	None - resize and restart	Statelessness, load balancing, coordination
Ceiling	Hard: largest available machine	Practically none
Cost curve	Superlinear at the top end	Roughly linear
Failure domain	One machine = total outage	Lose one of N, degrade gracefully
Downtime to scale	Usually a restart	Add an instance, no disruption
Best for	Databases, early-stage apps, stateful services	Stateless app tiers, workers, caches

The one precondition: statelessness

A stateless server keeps no request-specific data in local memory or on local disk between requests. Any instance can serve any request; instances are interchangeable and disposable. This is what makes load balancing, autoscaling, rolling deploys and instance failure boring instead of catastrophic.

State	Wrong place	Right place
Session / login	In-process memory	Signed cookie, or Redis keyed by session ID
Uploaded files	Local disk	Object storage
Background job state	A thread on one box	A queue plus a durable job record
Rate-limit counters	Per-instance counters	Redis, or a coordinated token bucket
WebSocket connections	Unavoidably local	Accept it; add a registry and pub/sub (see the real-time lesson)
Caches	Local is fine	Local L1 + shared L2, with short local TTLs

Autoscaling done properly

Scale on the right signal. CPU is a poor proxy for an I/O-bound service. Prefer requests per instance, queue depth, or concurrency from Little's Law.
Scale out fast, scale in slowly. Adding capacity you did not need costs a little money; removing capacity you did need costs an outage. Asymmetric thresholds and a long scale-in cooldown.
Account for warm-up. A new instance needs to boot, connect pools, JIT-compile and warm caches. If that takes 90 seconds, your autoscaler must trigger 90 seconds before saturation - which means scaling on a leading indicator, not a lagging one.
Set a floor and a ceiling. A minimum of three across zones for redundancy; a maximum so a traffic anomaly or a retry storm cannot scale you into a very large bill.

Say these points

Scale vertically first; it needs no code change and modern machines are huge.
Redundancy usually forces horizontal scaling before capacity does.
Statelessness is the precondition - move session, files and counters out of the instance.
Autoscale on a leading signal, out fast and in slow, with a floor and a ceiling.
The database is the hard part; the app tier is the easy part.

Avoid these mistakes

Sticky sessions as a permanent design.
Autoscaling on CPU for an I/O-bound service.
No maximum instance count, allowing a retry storm to become a bill.
Assuming a new instance is useful the moment it is running.

Expect these follow-ups

Your service takes two minutes to warm up. How do you autoscale it?
How do you migrate a session-in-memory app to a stateless one with no downtime?
When is vertical scaling of a database still the right answer at large scale?

Horizontal vs Vertical Scaling and Stateless Design