How to receive webhooks without duplicating actions in file workflows

The problem: a delivery is not always the same as an event

In a real file workflow, a webhook may arrive more than once, arrive late or appear in a different order than expected. This is not necessarily a design error: retries exist to overcome network outages, temporary receiver failures or ambiguous HTTP responses. The problem appears when the endpoint treats every delivery as a new action and downloads a file again, converts it, creates a ticket, sends a notification or records an operation in the ERP.

The first architectural decision is to separate event, delivery and side effect. The event represents something that happened at the source; the delivery is an attempt to communicate it; the side effect is what your system does as a consequence. A robust receiver does not ask “have I received this request before?”, but rather “has this event already been accepted and what authorized effects can it still produce?”. That distinction is the foundation of idempotent webhooks.

• Typical risk: creating two internal records for the same transformed file.
• Typical risk: sending several customer notifications for a single Cloud action.
• Typical risk: overwriting a new version with a late response from an earlier event.

What your receiver must guarantee before processing

A webhook receiver must guarantee four things: authenticity, traceability, idempotency and a predictable HTTP response. Authenticity means checking that the message comes from the expected source and was not modified in transit. Traceability means preserving identifiers, delivery date, internal status and result. Idempotency means that a repetition of the same event does not duplicate effects. The HTTP response tells the source whether the delivery was accepted or should be treated as failed.

The operational rule is strict: first verify, then accept durably and only then respond correctly. If the transformation, download or synchronization with the CRM may take time, do not run it inside the webhook’s critical window. Record the event in a transactional table or queue, mark its initial status and delegate the heavy work to a background process. For asynchronous cases, 202 Accepted is an appropriate response when the request was received but another process will handle it later.

• Check the signature before reading or saving the payload as trusted data.
• Persist acceptance of the event before responding successfully.
• Do not run long jobs in the endpoint’s main thread.
• Use internal statuses that are visible to support and operations.

HMAC signature: do not trust the payload without validating it

The signature is the first filter. In Apification, HMAC signatures are documented as a mechanism for verifying the origin and integrity of the payload. The operational recommendation is to validate the HMAC signature before reading or saving the content as if it were trusted. If validation fails, the receiver must not process the event, trigger downloads or start internal jobs. A signature failure is not a business problem; it is a security rejection.

Robust verification should not depend only on the body. In technical webhook specifications, it is recommended that the signature cover the identifier, the timestamp and the body, because the timestamp helps reduce replay attacks and may differ from the original event date when retries occur. In practice, your implementation must reconstruct the signed message exactly according to the source documentation, compare the signature securely and record only the metadata needed for diagnostics, never secrets.

• Reject events with a missing, malformed or non-matching signature.
• Validate the delivery timestamp according to a tolerance defined by your team.
• Do not include secrets, tokens or complete signatures in shared logs.
• Keep webhook secrets separated by environment and destination.

Practical deduplication with persistent statuses

To deduplicate, you need a stable key. A webhook specification includes a unique identifier associated with the event that remains the same even when a failed delivery is retried. That identifier can be used as an idempotency key so the consumer processes an event only once, even if it is received because of network problems, by mistake or maliciously. If your source also provides a delivery identifier, keep it for auditing, but do not use it as the only event key.

The minimum pattern is an events table with a unique key, status and result. When receiving a valid webhook, try to insert the event_id. If it already exists and is processed, respond correctly without repeating effects. If it exists in processing, respond consistently and avoid launching another worker. If it is in error, decide whether it can be requeued manually or after an internal policy. This approach turns duplicate deliveries into a status query, not a repetition of work.

• Recommended fields: event_id, delivery_id if it exists, type, resource, date, status, internal attempts and last error.
• Useful statuses: received, processing, processed, error, discarded.
• Key constraint: unique index on the stable event identifier.
• Support rule: every manual action must leave a record of who retried it and when.

Idempotency applied to file actions

Files add specific risks. The same notification can end up downloading the same resource twice, generating two conversions or notifying two different URLs for an equivalent result. Design each step with a “create if it does not exist” or “advance only if the status allows it” operation. For example, create the internal file record only once, associate the version or stable resource identifier and store the transformation result as a referenced artifact, not as a blind write over the latest available value.

It is also advisable to separate download, transformation and notification. The download obtains the input and confirms that it corresponds to the accepted event. The transformation produces a controlled output, ideally with a work record. The notification to the CRM, ERP or document management system is performed at the end and only if the previous steps reached the expected status. If a late event arrives, compare it against dates, status and stable identifiers before modifying a version or reporting a result.

• Do not overwrite versions without checking the current status of the internal resource.
• Do not send external notifications until the result is persisted.
• Store the link between file, Cloud project, event and internal result.
• Treat transformations as traceable jobs, not as immediate endpoint responses.

Retries: when to accept, when to fail and when to pause

Retries are a tool, but they also amplify defects if the receiver is not idempotent. Apification documents automatic and manual retries for temporary webhook failures. That is why your endpoint must distinguish between “I cannot accept the event” and “I already accepted it, but I will process it later”. If the signature is valid and you can save the event durably, respond successfully or with 202 Accepted and let your internal queue handle the work. This prevents a slow conversion from causing unnecessary repeated deliveries.

If your database, queue or event storage is not available, do not pretend to have accepted the event. Under temporary server conditions, 503 Service Unavailable is the appropriate code and can be accompanied by Retry-After when you have an estimate. 4xx errors should be reserved for problems attributable to the request, such as invalid format or rejected signature. The consistency of these responses makes it easier to interpret the delivery history and avoids mixing security incidents with operational saturation.

• Accept only when the event has been durably persisted or queued.
• Return an error if you cannot record the event and need the source to retry.
• Do not use external retries to compensate for poorly designed internal processes.
• Review events in error before retrying manually so you do not duplicate effects.

How Apification fits into a secure architecture

Apification lets you integrate Cloud and its services through REST API, OpenAPI, webhooks, iframe and JavaScript. For server-to-server integrations, the REST API lets you manage Cloud resources, users, configuration and transformation jobs from the backend. In event-oriented workflows, signed webhooks help you react to changes without continuously polling resources or background jobs. Apification documents events related to files, services, forms, signatures and processes.

The operational part matters too. Apification documents webhook delivery history with destination URL, date, status and response body, as well as statistics and automatic and manual retries. It also documents idempotent commands using a stable key in writes so network retries do not repeat the action. In projects with large files, asynchronous jobs make it possible to import and transform resources in the background while preserving progress and detailed errors.

• Use downloadable OpenAPI 3.1 as a contract for request and response schemas.
• Combine REST API for actions initiated by your backend and webhooks for relevant changes.
• Check history and statistics to debug delivery failures without relying only on internal logs.
• Apply idempotent keys in writes when an operation may be retried.

Example operational implementation

A reasonable architecture is: Apification webhook, verification endpoint, events table or queue, worker, internal API or CRM and final record by file or Cloud project. The endpoint validates the HMAC signature, checks timestamp and structure, extracts the stable event identifier, tries to insert it with a unique constraint and responds when acceptance is persisted. The worker takes events in received status, marks them as processing, runs the necessary download or query through APIs, launches transformations if applicable and records the result.

Expected failures must be defined before production. If the signature fails, it is rejected and not processed. If the event already exists, a correct response is returned without repeating the work. If the CRM is down, the worker keeps the event in error or pending according to your internal policy. If an old event arrives, it is compared against status, dates and identifiers before anything is modified. No secret should appear in logs, public URL parameters or visible error messages.

• Pre-checklist: signature validated, unique key created, statuses defined and logs without secrets.
• Test checklist: duplicate delivery, late delivery, invalid signature, database outage and CRM outage.
• Operations checklist: review delivery history, events in error, manual retries and queue times.
• Exit criterion: every file has a single highlighted result or an explainable and traceable error.

Frequently asked questions