Long Polling and Fallbacks — Real-Time APIs

Long polling: the client sends a request that the server holds open until data is available (or a timeout, e.g., 25 s), then the client immediately reconnects. It works everywhere HTTP works and is a solid fallback for restrictive networks. Use cursors so each poll resumes from the last event, and keep server timeouts below proxy/LB idle timeouts. Libraries like Socket.IO and SignalR negotiate WebSockets and fall back automatically.

Polling styles

Criterion	Short polling	Long polling
Latency	Up to interval	Near real-time
Empty responses	Many	Few
Server resources	Per request	Held connections
Compatibility	Universal	Universal

@app.get("/updates")
async def updates(cursor: int = 0, user: CurrentUser = None):
    try:
        async with asyncio.timeout(25):                   # below LB idle timeout
            items = await feed.wait_for_items(user.id, after=cursor)
    except TimeoutError:
        items = []
    next_cursor = items[-1]["seq"] if items else cursor
    return {"items": items, "cursor": next_cursor}         # client immediately polls again

async function poll(cursor = 0): Promise<never> {
  for (;;) {
    try {
      const r = await fetch(`/updates?cursor=${cursor}`, { signal: AbortSignal.timeout(30_000) });
      const { items, cursor: next } = await r.json();
      items.forEach(render);
      cursor = next;
    } catch {
      await new Promise((r) => setTimeout(r, 1000 + Math.random() * 2000));   // jittered backoff
    }
  }
}

Common pitfalls

Hold times longer than load balancer idle timeouts.
No cursor (missed or duplicate updates).
Thundering reconnects after deploys (add jitter).

Key points

Hold, respond, reconnect
Cursors for continuity
Jittered reconnects

Further reading: RFC 6202: Known issues and best practices for long polling

Practice questions

Why should the long-poll hold time be shorter than the load balancer's idle timeout?

Answer

Otherwise the LB closes the connection first, causing errors and reconnect churn instead of a clean empty response.

How do cursors prevent missed updates in long polling?

Answer

Each response returns the last delivered sequence; the next request asks for items after it, so updates arriving between polls are still delivered.