Long Polling

Long polling is a client-driven update mechanism in which the server keeps an HTTP request open for a bounded period while it waits for a change. When new information appears, the server answers; the client handles the response and immediately sends another request. If nothing changes before the timeout, the server sends an empty or unchanged response and the client reconnects. Unlike ordinary polling, it does not need a new request every few seconds while the system is idle.

Picture a support dashboard waiting for a new ticket. With five-second polling it keeps asking “anything new?” and mostly hears “no.” With long polling it asks once, and the server waits up to, say, 25 seconds before replying. If a ticket arrives after three seconds, the response goes out at once. That is near-immediate notification, but it is not a permanently open streaming response: each HTTP response completes one request, and the client has to ask again. Everything hard about the technique lives in the seams between those requests: the gap, the timeouts and the number of requests parked at any moment.

The comparison below uses one minute of traffic. Two events happen, at 3 s and 41 s. Polling every five seconds sends twelve requests and sees them at 5 s and 45 s, which is the average-case cost of an interval: a delay of anywhere from zero up to one full interval. The long poll needs four requests, and each event ends the request that was waiting for it.

Two things in the figure matter later. The grey request that ends empty at 28 s is not an error: it is the server saying “still nothing” so that the connection can be renewed before a proxy or a client timer decides to end it. And the request count only stays low while events are rare. In a busy stream each event still costs one request, so a long poll under heavy traffic behaves like back-to-back polling.

A practical long-poll request carries a cursor that identifies the last event the client has seen. The request is an ordinary GET; the response is an ordinary 200. Nothing in HTTP knows that this one is slow on purpose.

GET /ticket-events?after=ev_104 HTTP/1.1
Host: api.example.com
Authorization: Bearer <access token>
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: no-store

{"events":[{"id":"ev_105","type":"ticket_created","ticket_id":"T9"}],"next_cursor":"ev_105"}

The server first checks for events after ev_104. If any exist it returns them immediately, with no waiting at all. Otherwise it registers a bounded wait, wakes when a new event is committed, rechecks durable state and responds. On timeout it returns an empty result carrying the same cursor (or an appropriate no-content response; a 200 with an empty list is easier for clients to handle uniformly). The client reconnects using the latest successfully processed cursor.

GET /ticket-events?after=ev_105 HTTP/1.1
Host: api.example.com
Authorization: Bearer <access token>

   … 25 seconds pass, no new event, no bytes sent …

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: no-store

{"events":[],"next_cursor":"ev_105"}

Why a long poll can miss events: the reconnect gap

There is a gap between receiving one response and establishing the next request. During that gap the server holds no request for this client. If the design relies only on transient in-memory notifications (“push to whoever is waiting right now”), an event arriving in the gap has nobody to be pushed to and is lost. A durable event log or a queryable change table closes the gap: every request checks for events after the cursor before it waits. The wakeup signal is an optimisation that makes delivery fast; the durable query is the source of truth that makes it correct.

Follow event 106 in the sequence. It commits while the browser is between requests, so no wakeup reaches anyone. When the browser comes back with after=ev_105, the very first thing the gateway does is read the store, and 106 is sitting there. The gap lasted one network round trip plus whatever the client spent processing, and the cursor made its length irrelevant to correctness. It still matters to latency: a client that spends two seconds applying a heavy batch before reconnecting delays the next event by about two seconds even though nothing is lost.

What the cursor must guarantee

The store's cursor needs a documented ordering scope. An opaque, sortable event ID works if the backend guarantees the order. A random UUID is not inherently an ordered cursor, so “everything after this UUID” has no meaning. If several partitions are involved, a single scalar cursor may not be enough: use a merged stream with a defined order, or a vector of per-partition offsets.

Even a simple auto-increment column has a trap. In PostgreSQL, sequence values are handed out when a row is inserted, not when its transaction commits, so a transaction holding seq=105 can commit after another that holds seq=106. A reader that sees 106, saves it as its cursor and asks for seq > 106 will skip 105 forever. Serialising writers (as below), assigning the sequence inside a single writer process, or reading only up to a watermark that every older transaction has passed are all ways to make “after this cursor” safe.

-- PostgreSQL. Runnable as written.
CREATE TABLE ticket_event (
  seq        bigserial   PRIMARY KEY,          -- the cursor: ev_<seq> on the wire
  tenant_id  text        NOT NULL,
  type       text        NOT NULL,
  payload    jsonb       NOT NULL,
  created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX ticket_event_tenant_seq ON ticket_event (tenant_id, seq);

-- Writer: one transaction. The advisory lock makes seq order equal commit order,
-- so a reader can never see seq 106 while seq 105 is still uncommitted.
BEGIN;
SELECT pg_advisory_xact_lock(hashtext('ticket_event_writer'));
INSERT INTO ticket_event (tenant_id, type, payload)
VALUES ('acme', 'ticket_created', '{"ticket_id":"T9"}');
SELECT pg_notify('ticket_events', 'acme');   -- delivered to listeners only if/when this commits
COMMIT;

-- Reader: the durable catch-up query, scoped to the caller's tenant.
SELECT seq, type, payload
  FROM ticket_event
 WHERE tenant_id = 'acme' AND seq > 104
 ORDER BY seq
 LIMIT 100;

A browser limit that shapes the design

A parked request occupies a connection for its whole life. Over HTTP/1.1, browsers typically cap concurrent connections per origin (commonly six), and that cap is shared by every tab on the same origin. Two long-poll endpoints in each of three tabs can starve ordinary API calls. HTTP/2 and HTTP/3 multiplex many streams over one connection, which removes most of that pressure, but the server still pays for each parked stream, as the capacity section below shows. In practice, fold every update the page needs into one long-poll endpoint with one cursor, rather than one per widget.

A subtle race occurs when a handler checks the database, finds no new events and only then subscribes to a notification. An event can arrive between those two steps. If no later event follows, the request waits until timeout even though data was available for most of that time. It is invisible in light testing, because the window is microseconds wide, and shows up in production as sporadic delays of a whole timeout period.

One safe strategy is to register the wakeup mechanism first and recheck the durable store before sleeping. Another is to use a transactionally coordinated change feed or a sequence-based wait primitive. On any wakeup, read from the durable source again. Notification delivery may be duplicated, reordered or dropped; correctness comes from the cursor-based catch-up query, not from the notification. Pick an ordering below to see where each failing or passing event lands.

Subscribing first turns every interleaving into a harmless one: an event committed before the read is found by the read, and an event committed after the subscription produces a wakeup that is either consumed or ignored. The cost is occasional spurious wakeups, where the handler reads, finds nothing for its own cursor and goes back to sleep. That is why the loop in the pseudocode re-checks instead of assuming a wakeup means data.

function longPoll(after, deadline):
    verifyAuthorization()
    watcher = subscribeToWakeups()       # register before final check
    try:
        while now() < deadline:
            events = durableStore.readAfter(after, limit=100)
            if events not empty:
                return 200 {events, next_cursor: events.last.id}
            await watcher.waitUntil(deadline)  # may wake spuriously
        return 200 {events: [], next_cursor: after}
    finally:
        watcher.close()

Note limit=100. A client returning after a long absence can have thousands of events pending. Returning a bounded batch with a next_cursor lets it come back immediately for the next batch without building a huge response. The waiting period only begins once a read returns nothing.

The server below is a complete Node program with no dependencies. Its in-memory array is a stand-in for the durable store; everything around it (cursor parsing, expiry, register-then-read, timeout, cleanup on disconnect, graceful drain) is the real shape of the handler. The event-driven structure is the same algorithm as the pseudocode: wakeups.on registers the watcher, then check() reads the store, and each later wakeup calls check() again.

// Node 18+, ES modules, no dependencies. Run: node server.mjs
import http from 'node:http';
import { EventEmitter } from 'node:events';

const WAIT_MS = 25_000;

// Stand-in for the durable store: an append-only log, seq starts at 1.
// In production this is a table, a stream or a topic; the handler only needs readAfter().
const log = [];
let nextSeq = 1;
const wakeups = new EventEmitter();   // best-effort signal, not a source of truth
wakeups.setMaxListeners(0);

export function append(type, data) {
  const ev = { seq: nextSeq, id: `ev_${nextSeq}`, type, data };
  nextSeq += 1;
  log.push(ev);                       // commit first ...
  wakeups.emit('append');             // ... then wake waiters
  return ev;
}

function parseCursor(raw) {           // 'ev_104' -> 104; missing -> 0; garbage -> NaN
  if (raw === null || raw === '') return 0;
  const m = /^ev_(\d+)$/.exec(raw);
  return m ? Number(m[1]) : NaN;
}

function readAfter(seq, limit = 100) {
  if (log.length === 0) return [];
  const i = Math.max(0, seq + 1 - log[0].seq);
  return log.slice(i, i + limit);
}

// Older than retention, or ahead of anything this store ever issued.
const cursorExpired = (seq) =>
  seq > nextSeq - 1 || (log.length > 0 && seq + 1 < log[0].seq);

function send(res, status, body) {
  res.writeHead(status, { 'Content-Type': 'application/json', 'Cache-Control': 'no-store' });
  res.end(JSON.stringify(body));
}

function longPoll(res, after) {
  let finished = false;
  const finish = (status, body) => {
    if (finished) return;
    finished = true;
    clearTimeout(timer);
    wakeups.off('append', check);
    wakeups.off('drain', onTimeout);
    if (body && !res.writableEnded && !res.destroyed) send(res, status, body);
  };
  const check = () => {
    const events = readAfter(after);
    if (events.length > 0) finish(200, { events, next_cursor: events.at(-1).id });
  };
  const onTimeout = () => finish(200, { events: [], next_cursor: `ev_${after}` });

  wakeups.on('append', check);        // 1. register for wakeups
  wakeups.on('drain', onTimeout);     //    (also used for graceful shutdown)
  const timer = setTimeout(onTimeout, WAIT_MS + Math.random() * 2000);  // jittered deadline
  res.on('close', () => finish());    // client went away: release the waiter
  check();                            // 2. then read the durable store
}

export const server = http.createServer((req, res) => {
  const url = new URL(req.url, 'http://localhost');
  if (url.pathname !== '/ticket-events') return send(res, 404, { error: 'not_found' });
  if (req.headers.authorization !== 'Bearer demo-token') return send(res, 401, { error: 'unauthorized' });

  const after = parseCursor(url.searchParams.get('after'));
  if (Number.isNaN(after)) return send(res, 400, { error: 'bad_cursor' });
  if (cursorExpired(after)) return send(res, 410, { error: 'cursor_expired', snapshot_url: '/tickets/snapshot' });
  longPoll(res, after);
});

process.on('SIGTERM', () => {         // answer every waiter now so clients reconnect elsewhere
  wakeups.emit('drain');
  server.close();
});

server.listen(3000);
setInterval(() => append('ticket_created', { ticket_id: `T${nextSeq}` }), 8000);  // demo traffic

Three details deserve attention. The handler removes its listeners in one place, finish, which is called on response, timeout, drain and client disconnect; forgetting one path leaks a listener per request and shows up as a slow memory climb. The res.on('close') hook matters because a client that navigates away does not tell the server, and without it the waiter lives until its timeout. And wakeups.setMaxListeners(0) is required because the default limit of ten would print warnings with ten waiters.

Authorization is checked before any waiting, on every request, and the read is scoped to what the caller may see (the SQL above filters by tenant). A cursor is a position, never a permission: handing someone ev_104 must not let them read another tenant's events after it.

Wakeups that survive more than one process

With a database as the store, a natural wakeup channel is the database's own notification feature. In PostgreSQL, NOTIFY issued inside the writing transaction is delivered to listeners only after commit, which is exactly the ordering this design wants. Use one LISTEN connection per instance and fan out in memory instead of one per waiter, or the connection count becomes the bottleneck.

// One LISTEN connection per instance, fanned out locally to every waiter on that instance.
import pg from 'pg';
import { EventEmitter } from 'node:events';

export const wakeups = new EventEmitter();
wakeups.setMaxListeners(0);

const listener = new pg.Client({ connectionString: process.env.DATABASE_URL });
await listener.connect();
await listener.query('LISTEN ticket_events');
listener.on('notification', () => wakeups.emit('append'));
listener.on('error', () => process.exit(1));   // supervisor restarts; waiters then fall back to timeouts + reads

The client loop

The browser has no built-in reconnect for long polling; the loop is yours to write, and it is where most bugs live. The client timer is deliberately longer than the server hold so the normal outcome is a server response, not a client abort. The cursor moves only after the events were applied and the new position was saved. A guard against instant empty answers prevents a misconfigured cache or a proxy returning a fixed 200 from turning the loop into a tight spin. Failures use exponential backoff with full jitter, the policy described in Retry & Backoff.

// Browser or Node 18+. The cursor is saved only after the events are applied.
async function longPollForever({ url, token, signal, applyEvent, loadCursor, saveCursor, resync }) {
  const CLIENT_TIMEOUT_MS = 30_000;   // a little longer than the ~25-27 s server hold
  const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
  let cursor = (await loadCursor()) ?? 'ev_0';
  let failures = 0;

  while (!signal.aborted) {
    const startedAt = Date.now();
    const ctl = new AbortController();
    const timer = setTimeout(() => ctl.abort(), CLIENT_TIMEOUT_MS);
    const stop = () => ctl.abort();
    signal.addEventListener('abort', stop, { once: true });

    try {
      const res = await fetch(`${url}?after=${encodeURIComponent(cursor)}`, {
        headers: { Authorization: `Bearer ${token()}` },
        cache: 'no-store',
        signal: ctl.signal,
      });
      if (res.status === 410) { cursor = await resync(); await saveCursor(cursor); continue; }
      if (res.status === 401) throw new Error('unauthorized');   // refresh credentials in a real app
      if (!res.ok) throw new Error(`HTTP ${res.status}`);

      const body = await res.json();
      for (const ev of body.events) await applyEvent(ev);   // must be idempotent per ev.id
      cursor = body.next_cursor;
      await saveCursor(cursor);                             // advance only after applying
      failures = 0;

      // An empty answer that came back instantly is not a normal timeout: do not spin.
      if (body.events.length === 0 && Date.now() - startedAt < 1000) await sleep(1000);
    } catch (err) {
      if (signal.aborted) return;
      failures += 1;
      const cap = Math.min(30_000, 500 * 2 ** failures);
      await sleep(Math.random() * cap);                     // full jitter
    } finally {
      clearTimeout(timer);
      signal.removeEventListener('abort', stop);
    }
  }
}

The state machine has one invariant worth stating: the only edge that changes the cursor runs from Processing to Cursor saved. Everything else either retries with the old cursor or replaces the whole state through Resync. If a code review ever finds a place where the cursor is updated while parsing the response, that is the bug that eventually loses an event.

Long polling involves several timeout layers: the application wait duration, the server's request timeout, the load balancer idle timeout, any reverse proxy timeout, the client's abort timer and the behaviour of mobile networks, where a radio sleeping through an idle period can silently drop a connection. If the application waits 60 seconds but the gateway ends idle requests at 30, clients see avoidable disconnects. A common approach is to choose an application wait shorter than the relevant infrastructure limits, with a client timeout slightly longer than the application wait. Move the sliders to see which layer wins.

Do not prescribe one universal timeout. The right value depends on the provider, the intermediaries, the client SDK and connection reuse behaviour. Some infrastructure supports long-held requests; other infrastructure imposes strict limits. Verify the real deployment settings and monitor which component closes the request: a rising count of gateway-generated 504s or resets, as opposed to application 200s with empty bodies, is the signature of a mismatched layer. Idle timeouts are also usually measured from the last byte, so if your infrastructure only counts activity, an occasional keep-alive byte can extend a hold, but a bounded wait that returns a clean empty response is simpler to reason about.

A timeout is not a data error. A successful empty response means “no event observed within this wait,” not “there will never be an event.” Clients should reconnect promptly after a success, but use jitter after repeated network failures so a gateway restart does not create a synchronised surge. The same logic applies to the server side: the random extra wait in the handler spreads timeouts out, and a rolling deploy should answer parked requests during shutdown (the drain event above) rather than leave clients hanging until their own timers fire.

Long polling reduces the frequency of empty requests but holds many requests concurrently. If 100,000 clients are waiting, the service may need to maintain roughly 100,000 active request contexts, sockets or multiplexed streams. Asynchronous event-loop servers handle many waiting requests more efficiently than a thread-per-request model, which would need one blocked thread per waiter, but there is still memory, file-descriptor, gateway and load-balancer state to pay for. Operating-system limits such as the per-process file-descriptor cap and ephemeral port ranges on proxies are often the first walls you hit.

Little's Law gives a useful sizing intuition: concurrent_requests ≈ arrival_rate × average_request_duration under stable conditions. With an average hold of 20 seconds and 5,000 new requests per second, approximately 100,000 requests are in flight. Real sizing also depends on event arrival rates (an event ends a hold early, so average duration falls and the arrival rate for the same client count rises), reconnect overhead and infrastructure limits. An HTTP/2 connection may multiplex streams, but it does not eliminate per-request server state.

The same arithmetic shows why the technique is attractive: 100,000 idle clients on a 20-second hold arrive at 5,000 requests per second, against 20,000 per second for the same clients polling every five seconds. It also shows the price: the state for 100,000 live requests. Whether that is cheap or expensive is a matter of measurement on your stack, including TLS session state, buffers and any per-request framework objects, and of whether a load-shedding or backpressure policy exists for the moment all clients reconnect together.

Waiters across instances

Distribute waiting clients across instances. If wakeups are local to one process, events handled on another process must still reach the correct waiters through a broker or a shared notification mechanism. Sticky sessions are not a substitute for a durable event source: they make failover harder, they pile the waiters of a restarted instance onto its neighbours at once, and they do nothing to prevent missed events.

Notice what is deliberately weak in this picture. The broker links are dashed because the design assumes they can drop or repeat messages. Each instance, on a wakeup, re-reads the store for every waiter it holds, so for hot streams consider batching: one store read per wakeup tick per distinct cursor, rather than per request, and shedding waiters that exceed per-user or per-tenant limits before they consume capacity. Queue-based fan-out is covered in Queues & Workers and Pub/Sub.

Long polling alone does not guarantee exactly-once event delivery. A server can send an event and crash, or the connection can break, before it knows whether the client processed it. The client may reconnect with an older cursor and receive the same event again. Event IDs, deduplication and idempotent handling therefore matter. Advance the cursor after successful processing or durable local acceptance, not merely after bytes arrive. This is the distinction between a protocol guarantee (HTTP delivered a response body) and an application guarantee (the effect was applied once).

When a cursor is too old because retention has expired, return an explicit resynchronisation signal and have the client fetch a fresh snapshot. Silently starting from “now” may conceal missing changes; silently starting from the oldest retained event conceals them just as well. The status code is a convention (410 Gone here; a 409 or a typed error body works too), but it must be distinct from a plain empty result, so that clients cannot mistake data loss for quiet.

GET /ticket-events?after=ev_0040 HTTP/1.1
Host: api.example.com
Authorization: Bearer <access token>

HTTP/1.1 410 Gone
Content-Type: application/json
Cache-Control: no-store

{"error":"cursor_expired","snapshot_url":"/tickets/snapshot"}

The two scenarios map to the two ways an event stream can fail the client. A replay after a crash is benign when handlers are idempotent, and costs a duplicate response. An expired cursor is not benign: no amount of retrying will produce the missing events, so the protocol must say so and the client must replace its state from a snapshot. Retention length is therefore a product decision: it defines the longest offline period a client can recover from incrementally. Choose it from real reconnect data, not guesswork.

Authorization must be checked on every request and again when events are read; a cursor must not grant access to another user's events. Tokens expire while requests are parked, so decide whether a hold may outlive its token (usually not: the hold length should be shorter than the minimum token lifetime you accept, or the handler should re-verify on wakeup) and have the client answer a 401 by refreshing and retrying with the same cursor.

Long polling works when low-latency notification is useful but WebSockets or SSE are unavailable, when the existing HTTP request/response infrastructure is preferred, or when clients need a broadly compatible fallback: corporate proxies that mishandle upgrades, constrained clients with only a plain HTTP stack, or an API consumed by simple scripts that can loop on a GET. Typical payloads are notifications, job and order status, and low-volume dashboards, where events are sparse enough that one request per event is cheap.

• If updates are frequent and flow in one direction, prefer Server-Sent Events: a single long response with browser-managed reconnection and Last-Event-ID, no per-event request cost.
• If the client also sends frequent messages, WebSockets give a bidirectional channel, at the price of a separate protocol to operate and secure.
• If freshness of a few seconds or minutes is acceptable, plain polling with conditional requests is simpler and holds no connections.

Long polling remains valuable when its operational characteristics match the environment: it adds almost no new infrastructure beyond a wakeup channel and an ordered store, it degrades into ordinary polling when things go wrong, and every part of it can be tested with curl. The costs are the ones this lesson walked through: reconnect handling, aligned timeouts, parked-request capacity and application-level deduplication. Whichever transport you choose, the cursor-and-catch-up pattern here carries over unchanged.

Signals that it is time to move to a persistent channel: direction (server-only news points to SSE, frequent client-to-server messages on the same session point to WebSockets), update frequency (if events arrive faster than the reconnect round trip, every wakeup costs a full request, response and re-parking, and bursts leave a reconnect gap to cover with the cursor), client count (a parked request holds a connection for the whole wait, so per-client cost is already similar to a stream, and switching to a stream mainly saves the per-event request overhead) and proxy constraints (if intermediaries allow long responses but block or buffer streaming or upgrades, long polling may be the only thing that works). The first two usually decide it.