How to test webhook reliability

1. Define the business action and identity scope

Start with a concrete action: create an order, grant an entitlement or update a CRM record. Define the expected durable effect count for each eligible event, including zero for intentionally ignored events. Keep provider, tenant, environment and endpoint separate. Distinct event IDs can represent the same business intent, so event-level deduplication alone may not cover your domain.

Choose an idempotency key that follows the business operation. Couple the durable effect and deduplication record atomically where your datastore supports it. Where an external action cannot share a transaction, design explicit outbox, reconciliation and recovery behavior around that boundary.

2. Capture delivery and completion independently

Retain an expected event manifest and an immutable attempt log. Record attempt identity, event identity, start time, response status and the number of new durable effects attributable to that attempt. Preserve raw evidence privately, then normalize only the metadata needed for analysis.

A queued receiver may acknowledge before a worker finishes. Measure worker completion and reconciliation separately. Do not substitute a successful HTTP response, a processing-start message or a cumulative database count for per-attempt committed effects. Unknown remains unknown.

3. Exercise failures deliberately in a test environment

1. Replay the same event. Repeat a recorded delivery and confirm that it does not create an extra durable effect.
2. Lose the response after commit. Simulate the business action succeeding while the sender sees a timeout; retry should not repeat the action.
3. Fail before commit. Retry a transient failure and verify that the intended effect eventually appears.
4. Restart the worker. Confirm that durable queue and idempotency state survive process interruption.
5. Change ordering. Test later events arriving first and define whether to defer, reconcile or retrieve authoritative state.
6. Test rejection. Verify signatures, scope, malformed inputs and expired replay protection at the actual receiver. This analyzer cannot perform those checks.

Use test accounts and explicit fixtures. This page does not resend events or touch a live endpoint.

4. Inspect the evidence, not a green HTTP status

The analyzer retains missing events and flags excess known effects even when other telemetry is absent. Repeated delivery alone is not an idempotency failure. A timeout can occur after a successful commit. A 202 response can mean work was only queued.

The illustrative fixture contains five expected events: one recorded-effect match after a retry, one excess-effect event, one telemetry gap, one missing delivery and one recorded effect shortfall. One unexpected attempt is excluded from the expected summaries. These are instructional fixtures, not client results.

Choose an observation window long enough for your provider and workflow. This tool bounds offsets to 30 days; that is a format limit, not a recommended retention or retry policy. Preserve events outside the window separately rather than claiming they never occurred.

5. Reconcile, recover and retain a reproducible packet

Compare authoritative source state, your durable queue, business records and the attempt log. Recover only the specific missing actions through an approved operation; do not blindly replay an entire history. Preserve the source-hashed report, input manifests, environment, time boundary, receiver version and known telemetry omissions alongside the release.

For AI agents and automation tools, apply the same separation: an accepted tool call is not necessarily a completed external action. Record action identity, authorization, durable receipt and reconciliation state. Keep unrelated API/MCP contract checks and business-effect checks distinct.

Provider-specific references: Stripe webhook delivery documentation discusses duplicate events, ordering and acknowledgements. GitHub redelivery documentation describes its own recovery process. Verify the rules for your provider; neither provider's behavior is a universal webhook guarantee.

Use the same engine offline

Download the self-contained Node.js 22+ module. No package install, credentials or provider connection is required. The CLI reads two bounded JSON files and writes its report to standard output; it never edits source files. Its analyzeDelivery(expectedText, attemptsText) export is also available to your own JavaScript scripts. It is not an MCP server.

node webhook-delivery-audit.mjs --expected webhook-expected.json --attempts webhook-attempts.json > report.json

For repeatable use, retain the module version, both input files and your observation-window notes. A valid report can contain problems; the CLI exits nonzero for invalid inputs or I/O errors, not for operational findings.

Version 1.0.0. Module SHA-256: 27e6ee8ec70483e069ada6098f45a044899be8ae4e77a547576e6be9d7bd5f40. The hash identifies these file bytes, not a security certification.

Keep it self-service, or commission the integration

For reusable API/schema, MCP catalog and AI regression checks, the Developer Assurance Kit remains a separate $79 one-time purchase with five local MCP tools. This webhook analyzer is free and is not an additional MCP tool in Kit 1.1.0.