Forward Deployed Engineering Tutorial · Integration Engineering · Lesson 9 of 24
Pagination, Rate Limits and Retries
Read large data sets from customer APIs politely: cursor pagination, rate limits, Retry-After, exponential backoff with jitter.
Pagination
APIs return data in pages. Cursor-based pagination (a next token) is reliable while data changes;
offset-based pagination can skip or repeat records when items are inserted during the sync.
DATA = [f"invoice-{i}" for i in range(1, 12)]
def api_list(cursor=None, limit=4): # GET /invoices?cursor=...
start = int(cursor or 0)
page = DATA[start:start + limit]
nxt = str(start + limit) if start + limit < len(DATA) else None
return {"items": page, "next": nxt}
cursor, pages, items = None, 0, []
while True:
resp = api_list(cursor)
pages += 1
items += resp["items"]
cursor = resp["next"]
if cursor is None:
break
print(f"{len(items)} items in {pages} pages; last: {items[-1]}")
Output
11 items in 3 pages; last: invoice-11
Rate limits
Respect the customer's API limits: they protect systems their business runs on. When you get 429 Too Many
Requests (or 503), wait for Retry-After if present, otherwise back off exponentially with jitter so many
workers don't retry in lockstep. Cap retries and send failures to a dead-letter list for review.
import random
random.seed(2)
def backoff_delays(attempts, base=0.5, cap=30):
# "full jitter": a random delay between 0 and the exponential ceiling
return [round(random.uniform(0, min(cap, base * 2 ** a)), 2) for a in range(attempts)]
responses = [429, 429, 503, 200]
delays = backoff_delays(len(responses))
for status, delay in zip(responses, delays):
if status == 200:
print("200 OK")
break
print(f"{status} -> wait {delay}s then retry")
Output
429 -> wait 0.48s then retry 429 -> wait 0.95s then retry 503 -> wait 0.11s then retry 200 OK
Large historical backfills should be throttled, run off-peak, and agreed with the customer's API owner first.
Test yourself
A customer API returns 429 with Retry-After: 30. What should your integration do?
Retry immediatelyWait at least 30 seconds before retryingOpen more connectionsGive up permanently
Show answer
Answer: Wait at least 30 seconds before retrying
Retry-After is the server telling you exactly when to come back.