How to Prevent Duplicate Webhook Executions in n8n

Build around the side effect

The four-part idempotency architecture

A webhook can be delivered more than once even when the sender and n8n are both behaving normally. Providers retry after timeouts, operators replay deliveries, and two requests can arrive close enough together to overlap. The right objective is therefore not “make every URL receive exactly one HTTP request.” It is “make one logical event produce the protected side effect at most once.” If you still need to identify why requests are repeating, start with the separate guide to why an n8n webhook fires twice. This page begins after repeat delivery is already possible.

The claim must sit immediately before the irreversible or expensive action. A duplicate check near the beginning followed by several slow nodes leaves a race window: two executions can both observe “not seen,” then both continue. Put authentication and lightweight normalization first, claim next, and let only the winning branch reach the protected node.

Identity before storage

Choose a stable idempotency key

Prefer an identifier issued by the webhook provider for the logical event or delivery. Do not generate a random UUID inside n8n: every retry would receive a new value and look unique. Avoid hashing an entire payload unless the provider gives no durable identifier, because harmless changes in field order, timestamps, or metadata can change the hash even when the business event is the same.

Scope the key narrowly enough to avoid collisions and broadly enough to catch retries. A practical normalized value often has this shape:

{{ $json.provider }}:{{ $json.account_id }}:{{ $json.event_type }}:{{ $json.event_id }}

For example, an event ID that is unique only within one connected account needs that account ID in the key. If the provider documents a globally unique delivery or event identifier, the extra scope may be unnecessary. Preserve the original value separately for audit; normalization should not destroy evidence.

Match the guard to the risk

Three implementation options

Option 1: Remove Duplicates for bounded, low-risk history

Set the Remove Duplicates node to Remove Items Processed in Previous Executions, choose Value Is New, and point Value to Dedupe On at the normalized key. The official documentation says the node can store history at node scope or workflow scope and stores 10,000 items by default for this mode. That makes it convenient for feed items, notifications, and other cases where bounded history is acceptable.

There are two important boundaries. First, this operation compares the current input with stored values from previous executions; it does not remove repeated items within the same current input. n8n recommends chaining a current-input Remove Duplicates node before the previous-execution node when both forms matter. Second, clearing deduplication history intentionally makes older keys eligible again. Treat history size and clearing as business rules, not housekeeping details.

Option 2: n8n Data Table for visible workflow state

A Data Table can store a row per idempotency key together with a state such as processing, completed, or failed_retryable. The Data Table node supports checking whether a row exists, inserting, updating, and upserting. This is easier to inspect than an opaque deduplication history and supports retention and recovery fields.

However, a separate “check row, then insert row” sequence is not automatically safe under simultaneous executions. Two executions can complete the check before either inserts. Use a datastore operation with a real uniqueness constraint or atomic conditional write when duplicate side effects would be costly. Do not describe a two-node lookup-and-insert sequence as an atomic lock unless the backing system guarantees it.

Option 3: External database or destination idempotency for critical actions

For payments, inventory, provisioning, or any action that must survive multiple workers and restarts, use a durable store designed for atomic claims. A database table can enforce a unique constraint on the idempotency key. The winning insert owns the action; a unique-key conflict sends the other execution to the duplicate branch. When the destination API accepts an idempotency key, pass the same normalized key to it as a second line of defense.

A boolean is not enough

Failure and recovery states

A key recorded simply as “seen” can cause data loss. Imagine that execution A claims the event and then the protected API call times out. If every later delivery is discarded forever, the action may never complete. Store a state machine instead:

1. 01

   Claim as processing

   Create the key with processing, the current execution ID, and a lease or review timestamp. Only the successful claimant continues.

2. 02

   Run the protected action

   Pass the idempotency key downstream where supported. Capture a safe external reference, not credentials or a full sensitive payload.

3. 03

   Mark completed

   After a confirmed success, change the state to completed. Later attempts can acknowledge the event without repeating the action.

4. 04

   Classify failure

   Use failed_retryable only when retrying is safe. Use a terminal state for validation failures or permanently rejected requests. A timeout with an unknown destination outcome needs reconciliation before retry.

5. 05

   Recover abandoned claims

   Define what happens when processing outlives its expected window. Reconcile the destination first; do not simply expire the key and repeat an irreversible action.

If the provider expects a quick HTTP response, apply the acknowledgement pattern in How to Respond to Webhooks Immediately in n8n, but authenticate before acknowledging and persist enough durable state before detaching slow work. A fast response reduces provider retries; it does not replace idempotency.

Prove the side effect, not the request count

Duplicate-delivery verification procedure

1. Use a non-production destination or a reversible test action.
2. Prepare one valid webhook payload with a stable test event ID and valid authentication for the test environment.
3. Send the identical logical event twice sequentially. Confirm two requests can be observed but only one protected action occurs.
4. Send the same two requests as close together as your test tooling allows. This is the test that exposes a non-atomic check-then-write race.
5. Confirm the first attempt owns the key, the duplicate branch retains an audit record, and the final state becomes completed.
6. Force a safe downstream failure. Verify the state and recovery path match the documented retry policy rather than silently discarding the event.
7. Repeat with a genuinely new event ID and confirm it is not falsely blocked.

Execution count is not the success metric. Two webhook deliveries can legitimately create two n8n execution records. The pass condition is one protected business action for one logical event, with a traceable outcome for every attempt.

False fixes

Common mistakes that still allow repeats

• Generating the key inside the workflow. A fresh random value makes every retry unique.
• Checking only the current input. That removes duplicates within one execution but does not remember a delivery from an earlier execution.
• Checking history too late. If an email, charge, or row insert happens before the guard, deduplication cannot undo it.
• Using workflow static data as a lock. Treat process-local or workflow-local convenience state as unsuitable for critical cross-worker atomic claims unless you have verified its guarantees for your deployment.
• Calling concurrency limits idempotency. Queuing excess executions changes when they run; it does not make two events with the same ID become one.
• Discarding without evidence. Keep the key, attempt time, reason, and outcome needed to investigate false positives.
• Retaining sensitive payloads unnecessarily. Store the minimum identifiers and audit metadata needed for the guard and comply with the provider’s retention rules.

Short answers

FAQs

Does Remove Duplicates guarantee exactly-once processing?

No. It provides documented deduplication modes, including comparison with previous executions, but exactly-once business behavior depends on key quality, history lifetime, concurrency, storage guarantees, failure recovery, and the destination system.

Should the duplicate branch return an error?

Usually a known duplicate is an accepted, already-handled event rather than a server failure, but the correct response depends on the provider’s documented retry contract. Return the provider-appropriate success response only after authentication and your durable handling rule are satisfied.

How long should keys be retained?

At least for the provider’s documented replay or retry window, plus any operator replay window you intentionally support. Critical business identifiers may need longer retention subject to privacy and storage policies.