Keyset pagination is a technique for reading large datasets in pages by remembering the last key seen and asking for rows after it, instead of counting an offset from the start. Its defining property is that page ten thousand costs the same as page one.
Offset versus keyset#
Offset pagination asks for rows 100,000 to 100,050, which forces the database to walk and discard everything before the offset; latency grows with depth. Keyset pagination asks for the 50 rows after a remembered key, which an index answers directly regardless of how deep into the dataset the cursor sits.
For interactive tools over large collections, the difference is architectural: offset pagination gets slower exactly when tenants get bigger, keyset pagination does not degrade.
VisualizerEngine
Why VisualizerEngine uses it
The read path uses keyset pagination throughout, which is how the engine stays interactive at scale: performance-verified at 50,000 groups and 250,000 memberships with sub-25 ms page reads, where deep pages measure as fast as shallow ones.