Process File Batches with Controlled Concurrency

Why You Shouldn’t Send Everything at Once

When automating work on many files, sending all requests at once can make it difficult to track what has completed, what is still in progress, and what has failed. A burst of requests also makes it hard to know for certain what happened if the process is interrupted. Designing a queue with a limit on active tasks and a status for each file can help organize the work, but these are decisions for your integration, not behavior guaranteed by every API.

APIs offer different models. The [OpenAI Batch API](https://help.openai.com/es-es/articles/9197833-batch-api-faq) lets you submit requests for asynchronous processing. Microsoft documents batch requests for queries and operations in the [SharePoint Online REST/OData API](https://learn.microsoft.com/en-us/sharepoint/dev/sp-add-ins/make-batch-requests-with-the-rest-apis). These examples have different scopes and do not establish a common concurrency rule for other services.

• First, check the documentation for the endpoint you plan to use.
• Track each task by file.
• Separate API-documented functionality from decisions made for your queue.

Before You Start: Read the API Contract

Check which inputs the endpoint accepts, what responses it returns, and whether the result is delivered directly or processed asynchronously. Also look for any documented limits, identifiers for checking progress, or conditions for repeating an operation. If the documentation does not specify a behavior, do not assume it: ask the provider or design a controlled test.

Record these conditions in an integration specification, and distinguish between a request being accepted and the work being completed. Apification lets you integrate Cloud and its services through REST APIs, OpenAPI, and webhooks; that does not mean all services share the same limits, statuses, or retry policies. Consult the reference for the specific service.

• Record the endpoint, method, input format, and expected response.
• Note only limits and retry rules that are explicitly specified.
• Identify which response confirms acceptance and which confirms completion.

Design a Queue with a Status for Each File

As a design decision for your application, you can represent each file as a separate task. Store an internal identifier, a reference to the file, and its parameters; also record the local status, attempts, and relevant result or error. If the API returns a remote identifier, keeping it may help with tracking when the service documents a way to check status.

Use local statuses with clear meanings, such as pending, in progress, completed, and needs review. Do not present them as provider statuses unless the provider’s documentation defines them that way. Persisting these statuses lets your application identify incomplete tasks after an interruption, but it does not by itself determine whether a remote operation was carried out. Protect secrets and credentials when storing records.

• Record the result of each task, not just a batch total.
• Distinguish local statuses from statuses returned by the API.
• Keep the context needed for investigation without recording secrets.

Choose and Adjust Concurrency

There is no universal starting number that can be recommended without knowing the API and its conditions. As a design guideline, make the number of simultaneous tasks configurable and start by testing with a small workload. Observe the responses and any restrictions communicated by the provider; increase or reduce the limit only in accordance with documented rules and your test results.

The total batch size and the number of simultaneous tasks are different concepts: a large batch can be processed in smaller groups. If your architecture uses multiple consumers, decide how the aggregate limit will be coordinated; do not assume that each consumer separately has access to all the capacity the API allows.

• Start with a small number of items and a configurable limit.
• Adjust the limit based on the documentation and observed responses.
• Take the combined activity of all consumers into account.

Classify Errors and Limit Retries

Classify errors according to the responses and rules of the specific API. Some may require correcting the file or parameters; others may justify another attempt if the documentation allows it. Do not infer a universal policy from the error text. Save the code and relevant context for diagnosis without exposing sensitive data.

As a design decision, you can set a maximum number of attempts and a delay between them for cases that allow retries. If an operation is left in an uncertain state, a timeout does not prove that the service did not process it: avoid resending it blindly. If the API documents an idempotency mechanism, follow its conditions; otherwise, do not assume that repeating the request is safe.

• For each error, decide whether to retry, correct, or review.
• Record every attempt and avoid indefinite retry loops.
• Handle requests with uncertain outcomes cautiously.

Resume an Interrupted Batch

When restarting your application, load the persisted status and keep completed tasks. For tasks left in progress, check whether the API offers a documented way to verify their status using an identifier. If there is no way to resolve the uncertainty, define an explicit policy for these cases or send them for review instead of resubmitting them automatically.

Keep a stable identity for each task, and record file or parameter changes as distinct jobs. Apification lets you review the history of Cloud items, download previous versions, and restore content. These features help you review changes in Cloud, but they do not replace request tracking or guarantee that an API operation is idempotent.

• Skip completed tasks and review those left in progress.
• Do not interpret a lost response as proof that the operation failed.
• Record changes to inputs or parameters to distinguish jobs.

Operational Workflow and Testing

One possible workflow is to validate the files, create a persisted task for each item, process tasks within the limit you have defined, and record every response. At the end, show the results and separate completed tasks from those that need correction and those that remain uncertain. These are design decisions for your integration, not guarantees about how an external API works.

Before using the workflow in production, test interruptions and uncertain responses, such as a timeout or an outage between submitting a request and recording its result. Verify that the application retains enough context for investigation and that its policy does not automatically resubmit completed tasks or tasks with unknown outcomes. Tailor each case to the documented capabilities of the service. Apification offers a guided assistant for processing documents, images, video, audio, and data, as well as API integration options; consult the service documentation to confirm which operations are available through the API.

• Validate inputs and define what it means for each task to be complete.
• Test interruptions and uncertain statuses without assuming how the provider responds.
• At the end of the batch, identify pending items, errors that can be corrected, and tasks for review.

Frequently asked questions