API Pagination: How to Navigate a Collection

What it means to receive a paginated collection

A paginated collection is returned across multiple responses rather than in one response. An integration must process the results it receives and follow the endpoint’s documented instructions to determine whether another request is needed. The exact continuation method and stopping condition depend on the service and operation; the word “pagination” alone does not identify them.

Sensedia recommends pagination for services that return large amounts of data. That is a recommendation from Sensedia, not a universal rule that describes every API: https://www.sensedia.com.es/post/api-buenas-practicas-de-paginacion-y-filtros.

Before implementing a traversal, locate the contract for the specific operation. You need to understand how to make the initial request, what response information signals continuation, what information to use in a subsequent request, and what marks completion. A successful response does not, on its own, show that the full collection has been retrieved.

• Treat the operation’s documentation as the source for its pagination behavior.
• Write down the documented continuation signal and completion condition before coding.

Turn the endpoint documentation into a traversal plan

Start with the operation you intend to call. Note its method, endpoint, and documented initial parameters. Then inspect the response description for any pagination-related information and identify whether the documentation explains how that information affects the next request. Do not assume that a response field, header, page number, cursor, or URL has a particular meaning unless the service documents it.

A useful implementation plan separates the traversal into four steps: make the initial request; process the results returned; follow the documented continuation procedure when the response indicates that more results are available; and stop when the documented completion condition is met. Keep the continuation value and the rule for using it tied to the operation’s contract. Do not invent a parameter name or construct a request value based on a familiar pattern from another API.

For example, a planning note can use neutral labels rather than guessed syntax: “initial request: as documented,” “continue when: [documented signal],” “next request uses: [documented instruction],” and “stop when: [documented condition].” Replace each bracketed item with details from the actual technical documentation. If a required detail is absent or ambiguous, record the question and seek clarification before treating the traversal as complete.

This plan is also useful during review. Another person can compare the implementation with the stated contract and see whether the code follows the documented sequence, rather than having to infer the intended behavior from a loop.

• Keep initial-request details separate from continuation instructions.
• Use placeholders in planning notes until the service documentation supplies the actual values and rules.
• Do not use a small result set as a stopping condition unless the contract says to.

A documented example: New Relic REST API v2

New Relic’s REST API v2 documentation says that a paginated response includes a Link header reporting the total number of pages and which page is being queried. This describes the behavior documented for that API; it is not evidence that every service uses a Link header or follows the same pagination procedure: https://docs.newrelic.com/es/docs/apis/rest-api-v2/basic-functions/pagination-api-output/.

When reviewing this example, the two details to record from the cited description are the total page count and the page being queried. The description alone does not explain how to construct the next request for every operation or establish a general completion rule for other APIs. For the operation you plan to call, check its own documentation for those instructions.

The example illustrates why it is useful to distinguish a documented response detail from an implementation rule. A header may provide information, but an integration still needs to know what the endpoint expects in the next request and when the traversal is finished. Only use instructions that the relevant service documentation supports.

• The Link-header behavior described here is specific to New Relic REST API v2.
• Do not transfer its details to a different service without checking that service’s documentation.

Plan for interruptions and repeated records

If a process stops before reaching its documented completion condition, mark the run as interrupted rather than complete. Record where the interruption occurred and review the endpoint documentation before attempting to continue. The available evidence does not establish that a position or cursor can be retained, recovered, or reused after an interruption, so do not build a resumption method on that assumption.

If records appear more than once, compare the identifiers available in the returned data and review the run history. This can help your team understand what the integration processed, but it does not establish that the service guarantees repeats or guarantees that repeats will never occur. Decide how the application should handle repeated records based on its requirements and the documented behavior of the endpoint.

Keep these decisions distinct: whether the traversal reached the documented end, whether it was interrupted, and how the application handles repeated records. If the service documentation does not explain recovery or repeat behavior, note that limitation and seek clarification rather than treating a guessed approach as a provider guarantee.

• Label a run interrupted if it did not reach the documented completion condition.
• Review available identifiers and run history before deciding how to handle repeated records.
• Treat resumption and deduplication as implementation decisions, not assumed service guarantees.

Review the traversal and investigate discrepancies

For troubleshooting, keep a concise record of which operation ran, when the traversal started and ended, how many requests it made, and which documented condition was used to stop. If a request failed or the process ended early, note that too. These are engineering observations that make a run easier to review; they do not prove that every expected record was received.

If the service provides a reference count, compare it with the number of items your integration processed. Treat a difference as a prompt to investigate, not as proof of a particular cause. A count by itself does not establish that a collection is complete, and matching counts alone do not demonstrate that the same items were received.

When reviewing results, keep provider-reported values separate from totals calculated by your own integration. This avoids confusing what the service reported with what the client observed. If the results raise questions, return to the documented continuation procedure and the run record before drawing conclusions.

• Record the stopping condition and any interruptions or request failures.
• Use available counts as comparison points, not proof of completeness.
• Label provider-reported values and locally calculated totals separately.

What to check when integrating with Apification Cloud

Apification supports integration with Cloud and its services through REST API, OpenAPI, webhooks, iframe, and JavaScript. These are integration options; they do not specify how a particular operation returns a collection or handles pagination.

For automated collection retrieval, consult the technical documentation for the operation you plan to use. Identify its initial request, continuation procedure, and completion condition, then use those details in the traversal plan. Keep endpoint behavior distinct from platform integration capabilities: the available information does not establish a pagination, resumption, or integrity guarantee for a particular Apification endpoint.

If you cannot find a required pagination detail in the operation’s documentation, treat it as an open question. That is more reliable than borrowing a method from New Relic or another service and assuming it applies. Once the service-specific contract is available, the same planning and review steps in this guide can be applied to it.

• Use operation-specific technical documentation to establish pagination behavior.
• Do not attribute a pagination method or guarantee to Apification Cloud without supporting documentation.

Frequently asked questions