Real-Time Communication: Polling, SSE and WebSockets
Four ways to push data to a client, the scaling cost of each, and how to handle the hard part - a million long-lived connections that all reconnect at once.
Covers: Short and long polling, Server-Sent Events, WebSockets, WebRTC, connection scaling, presence, reconnection and backfill
HTTP is request/response: the client asks, the server answers. Chat, notifications, live scores and collaborative editing all need the opposite - the server has news and must tell the client. There are exactly four mainstream ways to do that, and choosing between them is a standard interview beat.
30-second answer
Short polling asks on a timer - simple, stateless, wasteful, and latency equal to half the interval. Long polling holds the request open until there is news, giving near-real-time delivery over ordinary HTTP but tying up a connection per client. Server-Sent Events is a one-way server-to-client stream over plain HTTP with automatic reconnection and event IDs built in, which covers most notification use cases. WebSockets give a full-duplex persistent connection and are the right answer only when the client also needs to send frequently - chat, collaborative editing, multiplayer. Choose the simplest one that meets the latency requirement, because each step up costs you statefulness.
| Short polling | Long polling | SSE | WebSocket | |
|---|---|---|---|---|
| Direction | Client pulls | Client pulls | Server → client | Bidirectional |
| Protocol | HTTP | HTTP | HTTP (text/event-stream) | Upgraded TCP (ws://) |
| Latency | ½ × interval | Near real time | Near real time | Near real time |
| Server cost | Wasted empty requests | Held connection per client | Held connection per client | Held connection per client |
| Auto-reconnect | N/A | Manual | Built in, with Last-Event-ID | Manual |
| Works through proxies | Always | Usually | Usually | Sometimes needs config |
| Binary payloads | Yes | Yes | No (text only) | Yes |
| Best for | Low-frequency status | Legacy real-time | Notifications, live feeds, progress | Chat, editing, games |
// Client - the browser reconnects automatically and replays Last-Event-ID.
const es = new EventSource("/api/notifications");
es.addEventListener("notification", (e) => render(JSON.parse(e.data)));
es.onerror = () => {
// No manual reconnect logic needed; the browser handles backoff itself.
console.log("disconnected, browser will retry");
};
// Server - the id: field is what enables gap-free resumption.
res.writeHead(200, {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache, no-transform", // no-transform stops proxies buffering
"Connection": "keep-alive",
"X-Accel-Buffering": "no", // nginx: disable response buffering
});
const since = req.headers["last-event-id"]; // resume exactly where we left off
for (const n of await getNotificationsSince(userId, since)) {
res.write(`id: ${n.id}\nevent: notification\ndata: ${JSON.stringify(n)}\n\n`);
}
// Heartbeat so idle proxies do not close the connection.
setInterval(() => res.write(": ping\n\n"), 20_000);The decision, in order
Can the user tolerate 30–60 seconds of staleness?
Then short poll. It is stateless, trivially scalable, survives deploys with zero effort, and costs nothing operationally. Do not be embarrassed by the simple answer.Does data only flow server → client?
Then SSE. You get sub-second delivery, automatic reconnection and resumption without writing any of it yourself.Does the client also send frequently?
Then WebSockets. Chat, cursors in a shared document, game input - anything where the client is a first-class publisher.Is it peer-to-peer media?
Then WebRTC, with your servers only doing signalling and TURN relay. Do not proxy audio and video through your application tier.
Say these points
- Pick the simplest option meeting the latency requirement; each step up adds statefulness.
- SSE covers most one-way cases and gives reconnection plus resumption for free.
- WebSockets are justified by frequent client→server traffic, not by 'real time'.
- Long polling is a legacy fallback, not a default.
Avoid these mistakes
- Reaching for WebSockets when the client never sends anything.
- Forgetting SSE is text-only and needs heartbeats to survive idle proxy timeouts.
- Short polling at one-second intervals - that is just an expensive, high-latency stream.
- Ignoring that any held-connection approach makes your tier stateful.
Expect these follow-ups
- How would you stream LLM tokens to a browser, and why?
- What happens to an SSE connection through a proxy that buffers responses?
- How does HTTP/2 change the cost calculation for SSE?
30-second answer
Separate the connection tier from the business logic. Stateless gateway nodes hold the sockets and do nothing but authenticate, decode and forward; a registry maps user ID to the gateway currently holding that user's connection; and a pub/sub layer fans messages out to the right gateway. Each node can hold roughly 50,000–200,000 connections given enough file descriptors and memory, so a million connections is 10–20 nodes. The genuinely hard parts are not throughput but reconnect storms, deploys that drop every connection at once, and presence - knowing who is actually online.
A connection tier that scales
The key move: gateways are dumb and horizontally scalable; all routing goes through pub/sub so no gateway needs to know about any other.
Flow
- ClientsL4 load balancer
- L4 load balancerGateway nodes
- Gateway nodesConnection registry(register)
- Gateway nodesBusiness services(inbound)
- Business servicesMessage store
- Business servicesPub/sub bus(publish)
- Pub/sub busGateway nodes(fan-out)
- Message storeClients(backfill on reconnect)
The three problems that actually bite
Reconnect storms
If a gateway holding 100,000 connections dies, all 100,000 clients reconnect immediately - and can take down the replacement, then the next one. Fix with exponential backoff plus full jitter on the client, and a connection rate limit at the gateway. Jitter is the essential part: synchronised backoff is still synchronised.Deploys
Restarting a gateway drops every socket on it. Roll one node at a time, send acloseframe with a reconnect hint before shutting down, and drain slowly. Keeping business logic out of the gateway means gateways restart rarely - which is the real fix.Missed messages
A socket is not a durable channel. Every message gets a monotonic sequence number per conversation; on reconnect the client sends its last-seen number and the server backfills from the store. Without this, a five-second network blip silently loses messages.
let attempt = 0;
function connect() {
const ws = new WebSocket(url + "?since=" + lastSeq);
ws.onopen = () => { attempt = 0; };
ws.onclose = () => {
// Full jitter: pick uniformly in [0, cap]. Without the random factor every
// client retries at the same instant and the storm simply repeats.
const cap = Math.min(30_000, 1_000 * 2 ** attempt++);
const delay = Math.random() * cap;
setTimeout(connect, delay);
};
ws.onmessage = (e) => {
const msg = JSON.parse(e.data);
lastSeq = msg.seq; // persisted so a reload also resumes correctly
render(msg);
};
}| Resource | Per connection | At 100k connections/node |
|---|---|---|
| File descriptor | 1 | Raise ulimit -n well above 100k |
| Kernel socket buffers | ~4–16 KB | 0.4–1.6 GB |
| Application state | ~1–10 KB | 0.1–1 GB |
| Heartbeat traffic | ~1 frame / 30 s | ~3,300 frames/s per node |
Connections are cheap in CPU and expensive in memory and file descriptors - the opposite of a request-serving tier.
Say these points
- Split dumb connection gateways from stateless business logic.
- Registry (user → gateway) plus pub/sub fan-out removes gateway-to-gateway coupling.
- Exponential backoff with full jitter is what prevents reconnect storms.
- Sequence numbers plus a durable store make reconnects gap-free.
- Presence is approximate: heartbeat plus TTL, never a correctness dependency.
Avoid these mistakes
- Putting business logic in the socket tier, forcing disconnects on every deploy.
- Backoff without jitter - the storm just repeats on a timer.
- Treating the socket as durable and losing messages on a blip.
- Forgetting file-descriptor limits and load-balancer idle timeouts.
- Building features that assume presence is accurate.
Expect these follow-ups
- How do you deliver a message to a user connected on three devices?
- A single chat room has 500,000 subscribers. What breaks, and how do you fix it?
- How would you support graceful gateway shutdown without any client noticing?
Showing 2 of 2 questions for realtime-communication-websockets-sse-polling.
Check your understanding
4 questions · no sign-up, nothing stored
Hands-on challenge
Build it - this is what you talk about in a deep-dive round.
Design the real-time layer for a collaborative document editor
Multiple users edit one document simultaneously and see each other's cursors. Design only the transport and connection layer - not the conflict-resolution algorithm.
Requirements
- Choose a transport for edits and for cursor positions, and justify treating them differently.
- Design the gateway, registry and pub/sub topology, and state how a message reaches exactly the right sockets.
- Specify the reconnect protocol so no edit is lost after a 10-second disconnection.
- Define presence semantics and be explicit about what is approximate.
- Describe a zero-disruption deploy for the connection tier.
Stretch goals
- Handle a document with 10,000 concurrent viewers and 5 editors.
- Add cursor-position throttling and explain the bandwidth saving.