How to Verify a Webhook Chain: Declared, Registered, Delivered, Recorded

Our idempotent-handlers guide starts at the moment a webhook lands on your handler. This article is about everything before and after that moment, because that's where webhooks actually die: not in the handler, but in the gap between configuration and reality. A webhook that "isn't working" is four separate claims wearing one sentence, and each is proven at a different surface: the declaration in your config file, the subscription the platform actually created, the request that actually arrived at your edge, and the row that actually got written. We recently worked this checklist against two live systems we run — a Shopify app whose subscriptions are declared in TOML, and a FastAPI service taking Stripe events — and the punchline is worth stating up front: three of the four links can be verified from inside your own infrastructure. The second one cannot.

Link 1: Declared — a Config File Is a Request, Not a Fact

The declaration is the easy link: it's in your repo. A Shopify app declares subscriptions as [[webhooks.subscriptions]] blocks in its shopify.app.toml; a FastAPI service declares a route. The trap is believing the declaration does anything. On Shopify, it does not — and the platform says so, twice, in sentences worth pinning to the wall. The CLI reference for shopify app deploy: the command "creates an app version, which is a snapshot of your app configuration and all extensions. This version is then released to users" — and, separately, "This command doesn't deploy your web app. You need to deploy your web app to your own hosting solution." The webhooks getting-started page completes the picture: "When working in development mode, webhook subscriptions are automatically updated when you save your TOML file," but for production, "when you're ready to release your changes to users, you can create and release an app version."

Read those together and the failure mode writes itself: shipping your server and shipping your configuration are two different deploys. Edit the TOML, deploy your web app, and the platform's registry hasn't changed — dev mode's auto-sync trains you to expect an update that production only performs when a new app version is released. We know because we did it: one of our apps declared a new subscription topic in its TOML, and the commit message itself ends with the reminder that the topics "require a separate shopify app deploy to register." That note exists because the gap had already bitten once before, in a nastier costume: a checkout Validation Function that was deployed but never activated — the platform only runs one after a validation record is created, which the app had never done — so the server-side block was dormant on every install, and an App Store review caught what no local test could. Deployed is not registered; registered is not active. Link 1 verifies only that you asked.

Link 2: Registered — the Link You Cannot Grep

Here is the uncomfortable one. From the receiving end, a registered subscription that has had no qualifying events and an unregistered subscription look exactly the same: silence. Nothing in your logs, your database, or your code distinguishes them, because both produce zero requests. The only authority on link 2 is the platform's own registry, which for Shopify means asking the Admin API:

{ webhookSubscriptions(first: 20) { nodes { topic endpoint { __typename } } } }

Run that with the app's own authenticated session (a Partner-dashboard check of the released app version's configuration answers the same question). One tempting shortcut deserves a warning label: don't try to shortcut the library by reading an access token out of your own session table and curling the API with it. When we audited ours, every stored access token was expired — normal, since they're short-lived and refresh on use — and refreshing out-of-band rotates the stored refresh token underneath a running production app. Ask through the framework's session machinery or through the dashboard, not around them.

Link 3: Delivered — Your Edge Log, and What Its Silence Doesn't Mean

Delivery is checkable with a one-line grep against your access log — count the POSTs to your webhook paths — but interpreting the count takes more care than producing it. The worked example from our own fleet: that TOML-declared subscription topic from link 1 has, as of this morning, delivered zero requests in the eight days since it was declared. Is it broken? Here's the discipline the whole article exists to teach: that zero proves nothing on its own. The topic fires on subscription changes, and no install of that app has changed a subscription in the window. Meanwhile the same topic demonstrably registers and delivers on sibling apps of the same fleet, sharing the same server and the same deploy process. And the second data source we might be tempted to lean on — a backfill that refreshes the same table — runs on embedded page loads, of which the window has also had zero. Two absences, two different mechanisms, neither one evidence about registration. Silence at link 3 sends you back to link 2's probe; it never answers it.

When delivery fails rather than merely not happening, the log has a signature worth learning to read. Shopify documents its policy precisely: "If Shopify receives no response or an error, it retries 8 times over the next 4 hours. After 8 consecutive failures, the subscription is automatically deleted if it was configured using the Admin API" — note the qualifier; config-file subscriptions aren't in that deletion clause — with "a one-second connection timeout and a five-second timeout for the entire request," and "any response outside the 200 range, including 3XX codes, is treated as an error." Our own log carries a textbook specimen: nine 500s to one webhook path inside a four-hour window, the gaps between them doubling from a minute toward an hour. The access log records no delivery ID, so "one delivery, nine attempts" is an inference from that backoff shape rather than a field we can point to — but it matches the documented policy too well to be much else.

Link 4: Recorded — Arrival Is Not a Ledger Entry

The last link is the one the idempotency article lives inside, so we'll add only the verification method. The framework-neutral example is our FastAPI service taking Stripe events, and it starts with a mundane trap that has eaten real debugging hours: print the mounted path, not the decorator string. The handler's decorator reads /webhook; the router is created with prefix="/stripe"; the only path that exists in production is POST /stripe/webhook, which is the one the access log confirms. Grep for the decorator's path and you'll conclude nothing is arriving while the log fills with deliveries one string away.

Past the path, the receipt structure matters more than the code. The handler verifies the signature (against the production secret, then the test secret) and answers 401 when neither matches; it short-circuits on an already-seen event ID; and — the part most setups skip — it writes a success ledger row only when processing actually succeeded, distinct from the arrival record written when the event landed. That gives you two tables answering two different questions: what arrived, and what was fully processed. The verification method is then a windowed LEFT JOIN from arrivals to the ledger over a common time range: every arrival without a ledger row is a delivery your system accepted and then failed to finish, which is precisely the population "we got the webhook" hides. (Two practical notes if you build this: window both sides on comparable timestamps — an arrival-time column joined against a processed-time column will manufacture phantom gaps at the window edges — and treat the join as an operational report, not a data-integrity alarm.)

The Checklist

The general lesson costs one sentence: declared configuration is a request, not a fact. Every system that accepts a declaration — a TOML block, a route, a cron entry, a manifest — has a registry somewhere that decides whether the request became real, and the registry is the only surface that can say so. Config drift is a whole genre of outage precisely because the declaration keeps looking correct the entire time nothing happens. We've written before about how fast claims rot; configuration is the special case where the claim rots the moment it's written, because it was never more than a claim to begin with. Deploy the config, probe the registry, grep the edge, join the ledger — in that order, every time it matters.