MongoDB from Basics to Production · MongoDB in Production
Pagination and Bounded Arrays
Paginate large collections with range queries instead of skip, and replace ever-growing arrays with a bucket pattern.
What you'll build
A paginated activity feed: skip-based paging compared with keyset paging (continue after the last seen value), and sensor readings stored in hourly buckets instead of one ever-growing array per device.
The idea
skip(n)still walks over the firstnentries: page 1,000 is slow, and results shift when new documents arrive.- Keyset pagination: sort by an indexed, unique-ish key (
createdAt,_id) and ask for documents after the last one you returned. - Bucket pattern: group many small items (readings, events) into documents per time window with a bounded count, keeping documents small and indexes efficient. (MongoDB also offers time series collections for this.)
The code
const events = [];
for (let i = 1; i <= 25; i++) events.push({ _id: i, createdAt: new Date(Date.UTC(2026, 0, 15, 0, i)), text: "event " + i });
db.events.insertMany(events);
db.events.createIndex({ createdAt: -1, _id: -1 });
function page(after, size) {
const filter = after ? { $or: [{ createdAt: { $lt: after.createdAt } }, { createdAt: after.createdAt, _id: { $lt: after._id } }] } : {};
return db.events.find(filter).sort({ createdAt: -1, _id: -1 }).limit(size).toArray();
}
let last = null;
for (let p = 1; p <= 3; p++) {
const docs = page(last, 10);
print(`page ${p}:`, docs.map(d => d._id).join(" "));
last = docs[docs.length - 1];
}
const plan = db.events.find({ createdAt: { $lt: new Date("2026-01-15T00:16:00Z") } }).sort({ createdAt: -1, _id: -1 }).limit(10).explain("executionStats").executionStats;
print("keyset page examined", plan.totalKeysExamined, "keys");
for (let m = 0; m < 130; m++) {
const at = new Date(Date.UTC(2026, 0, 15, 8, 0) + m * 60000);
const hour = new Date(Date.UTC(2026, 0, 15, at.getUTCHours()));
db.readings.updateOne({ device: "d1", hour, count: { $lt: 60 } },
{ $push: { values: { at, temp: 20 + (m % 5) } }, $inc: { count: 1 } }, { upsert: true });
}
db.readings.find({}, { _id: 0, hour: 1, count: 1 }).sort({ hour: 1 }).forEach(b => print(b.hour.toISOString(), b.count, "readings"));
Output
page 1: 25 24 23 22 21 20 19 18 17 16 page 2: 15 14 13 12 11 10 9 8 7 6 page 3: 5 4 3 2 1 keyset page examined 10 keys 2026-01-15T08:00:00.000Z 60 readings 2026-01-15T09:00:00.000Z 60 readings 2026-01-15T10:00:00.000Z 10 readings
Run this on your computer — see “Get Started” for setup.
Common mistakes
skip-based deep pagination on large collections.- Sorting only by a non-unique field (like
createdAt) without a tiebreaker, skipping or repeating documents. - One document per device with an array of every reading ever; it grows without bound.
Key points
- Use keyset pagination with a unique sort key and an index.
- Bound array growth with the bucket pattern (or time series collections).
- Check pagination queries with explain.
Further reading: Bucket pattern · Time series collections
Practice questions
Why is skip-based pagination slow for deep pages?
Answer
The server still walks over all skipped entries before returning results, so cost grows with the page number.
What does keyset pagination need?
Answer
A stable sort on indexed fields with a unique tiebreaker (such as _id) and a filter for values after the last returned document.
What problem does the bucket pattern solve?
Answer
It groups many small items into bounded documents per time window, avoiding unbounded arrays and huge numbers of tiny documents.
Open in the app to mark lessons complete, take notes and track your progress.