Implementation recipes
These are not feature bullets. They are the guides that ship inside the product, on the Implement tab of each module. Each one carries a goal, prerequisites, the full file inventory, ordered steps with real code, verification at every stage, and a definition of done. Below is what each one covers.
Showing 39 of 173 recipes.
Provider calls settle their own state. A real email send runs under the retrier's supervision: its record starts as queued and moves to sent or failed exactly once, from the completion hook rather than the attempt itself, so nothing is marked delivered by an attempt that later failed and no record is stranded.
state settled outside the action body
A timed-out attempt may already have reached the provider, and a naive retry sends twice. Stable idempotency keys, derived from the work itself rather than the clock, let a repeated attempt recognise the send already happened and return the original result, one delivery no matter how many attempts run.
timeouts that already reached the provider
Supervised work can start after a delay or at an exact future timestamp instead of immediately. A provider's Retry-After cooldown gets one scheduled run rather than four attempts burned against a wall, deadlines that come from data fire on time, and a run still waiting reads as scheduled, not stuck.
honouring a Retry-After cooldown
Users get an abort control for supervised work they started, and cancellation settles through the same completion path as success and failure. A cancel that loses the race to an already-finished run is reported calmly rather than thrown, and settled run records are reclaimed on demand instead of lingering for a week.
reclaiming completed run storage
Customers generate a key in settings, see the secret exactly once, and call your public endpoints with a bearer token. The database stores only a hash and a display prefix, so a stolen backup yields no working credentials, and keys can be listed, rate limited and revoked per organization.
authenticating from a digest lookup
A key that inherits everything its creator can do is a full-account credential sitting in a config file. Every key instead carries an explicit scope list checked on each route, scope names stay stable for your docs, and creators can only grant scopes their own role already covers.
no privilege escalation through a key
Keys carry an expiry date the customer picks, enforced on every request rather than by an overnight sweep. Rotation mints a linked successor while the old key stays valid through a grace window, owners are warned by email before the cutoff, and a report surfaces keys unused for ninety days.
overlap window so rotation never breaks
Per-key ceilings derived from the customer's plan stop one retry loop becoming everyone's outage, and keying by key means a runaway script cannot throttle its owner's other traffic. Every call is metered for billing, and responses carry the standard rate limit headers, so well-behaved SDKs back off on their own.
headers SDKs back off on by themselves
Customers debug themselves from their own request history: method, path, status and latency per key, with call volume, error rate and p95 latency for the last day. No column can hold a token or request body, every route is logged without exception, and old rows are swept on schedule.
logging requests without logging secrets
Keys end up in public repositories, CI logs and screenshots. Secret-scanning reports in the format GitHub already sends revoke the matching key on arrival, owners are notified with an audit trail naming where it leaked, and admins hold an organization-wide revoke-everything button behind a fresh re-authentication.
matching a leaked secret to a stored hash
Any mutation can become automatable, shown here by marking an invoice paid. That event fires customers' own when-this-then-that rules, email, signed webhook, agent or task, only after the write commits, the trigger appears as an option in the builder, and every run leaves a history summary a human can read.
trigger summaries a human can read
Long-running exports and reports survive restarts, and customers watch progress move from queued to complete live. Permission and plan checks run before anything is enqueued, so a denied organization leaves no half-created records, and you get durable background work without standing up a job runner of your own.
progress the UI can trust
Take the durable job pattern to any multi-step process: onboarding checklists, invoice PDFs, CRM syncs. Each new job keeps tenant isolation, reports its status to a live list in the UI, and sits behind a plan feature flag, so an expensive job can be switched off per customer without a deploy.
multi-step work that resumes mid-flight
Failed and canceled jobs settle into a final state carrying the real error text, so a step that throws never leaves a customer watching a run sit at queued. The status badge renders failure and cancellation distinctly from work in progress, and every run terminates exactly once, whatever its outcome.
terminating a run exactly once
A Cancel control that stops the engine itself rather than only the status row, so the record never lies about what is still running. The action is gated by the same permission that started the run, rejected server-side without it, and the button only appears while a run can still be stopped.
stopping the engine, not just the row
Nightly runs fan out across every organization still entitled to the feature, with no human clicking Start. The same durable job now serves system-triggered work, runs exist cleanly without a requesting user, and an organization whose plan or kill switch denies the feature gets zero runs, not a failed one.
skipping tenants who lost entitlement
Job finished, requester told. The person who clicked Start gets an in-app notification on success and a distinctly worded one on failure or cancellation, delivered through the same channel every other module uses, so mute preferences and duplicate suppression still apply, and nobody keeps a tab open watching a list.
different wording for the failure path
Exceptions reach Sentry in production tagged with release and source, and stay on the console locally when no DSN is configured. Events carry an organization tag for triage without any personal data, payloads never include secrets, and a built-in test capture proves the path before a real incident does.
keeping secrets out of error payloads
Denials become actionable screens — upgrade, ask an admin, sign in again — rather than raw stack traces. Feature, permission, plan-limit and session errors each get their own recovery path, unexpected failures still reach Sentry with full detail, and internal messages never leak to the customer.
a recovery path per denial reason
Your customers receive signed events within seconds of the product action that caused them. Endpoint registration, HMAC-signed payloads and a delivery queue come as one piece: failures retry with backoff up to eight attempts, dead deliveries stay visible with their last status code, and a slow receiver never slows your app.
dead letters visible to operators
Signing secrets cannot be hashed away like API keys, so rotation needs its own design. Customers add endpoints, choose event types, pause and rotate secrets from a self-service panel, every delivery carries signatures for both old and new secrets during the rotation window, and expired secrets are pruned automatically.
signing with two secrets at once
The recent-deliveries panel customers debug from without filing a ticket. Every attempt keeps its own record — outgoing headers, response status, a bounded response snippet, timing, and connection failures marked distinctly — one click redelivers the original payload with a fresh signature, and a retention sweep stops the log growing forever.
keeping an attempt log from growing forever
One abandoned customer URL can absorb queue capacity forever. Failing endpoints move through progressively longer cool-off periods, owners are warned in-product and by email before a sustained outage disables the endpoint with its last error named, and reactivating replays the deliveries missed during the outage within a bounded window.
replaying deliveries missed while down
The first payload change after customers integrate is the one that breaks them. Events live in a typed catalog with one shape per version, endpoints stay pinned to the version they integrated against until the customer opts to upgrade, and a test ping exercises a new receiver before real events exist.
test pings before a real event exists
A webhook sender posts wherever the customer says, handing every signup a request-forgery primitive. Private, loopback, link-local and cloud metadata addresses are refused at registration and re-checked at delivery time so DNS rebinding gains nothing, redirects are never followed, and every request is bounded by a timeout and a response size cap.
DNS rebinding between check and send
Protects an abuse-prone action end to end. You choose between token bucket and fixed window from the shape of the abuse, enforcement runs after authorization and before the write so anonymous traffic never touches limit state, and a denial reaches the user as a try-again countdown rather than a generic error.
where the check sits versus authorization
Users see remaining capacity before they click: the control disables itself, shows how many attempts are left, and counts down to the moment it re-enables — without polling. The hint corrects for client clock skew, each user can only read their own bucket, and the server keeps doing the enforcement that actually matters.
client hint, server stays the authority
Background work reserves a future slot when the bucket is empty and schedules itself for the exact moment capacity frees up, so a burst spreads out instead of being dropped or retried blindly. Reservations are capped, and once they run out the caller gets honest backpressure with a wait time attached.
spreading a burst instead of retrying blind
Support can clear one customer's throttle after a false positive, and the customer's very next request succeeds against a fresh bucket. Only an explicit allowlist of limits can be reset — abuse-containment limits stay off the table — behind fresh password step-up and a required reason written to the immutable audit trail.
step-up and reason on a support override
"Where is my export?" gets an answer on screen. Every queued job owns a record that moves from pending through running to succeeded, failed or canceled, carrying attempt counts and the real error text, and a live dashboard table shows each organization its own jobs without polling.
closing the row when the job dies
Backfills across fifty thousand rows run as a self-resuming paginated driver that enqueues fixed-size batches into a capped pool, so a re-index or bulk send never blows a single transaction's budget. Progress persists between passes, survives interruption, and reads out as processed and remaining counts on the admin screen.
transaction budget on huge fan-outs
Pools cap execution, not enqueueing, so a runaway producer can turn a five-second job into a four-hour wait. Each enqueue path gets an arrival limit sized to the drain rate plus a per-tenant depth ceiling, denials render as a wait estimate, and an admin panel shows depth and oldest-pending age before users feel it.
queue depth nobody is watching
Retries end; the work still matters. Jobs that exhaust their retry budget land in a dead-letter table carrying everything needed to run them again days later, an operator replays a single job or a whole outage window from the admin screen, and replaying work that actually succeeded produces no duplicate side effect.
replay that cannot double-fire
A five-second thumbnail should not wait behind a ten-thousand-row export. Interactive and bulk work run in separate lanes with independent caps so a burst in one class never delays the other, users cancel their own queued jobs from the UI, and deleting a tenant cancels everything it still had queued.
orphaned jobs after a tenant is deleted
A 500-member broadcast should not become 500 simultaneous calls and a wall of 429s. A dedicated pool sized to Resend's documented rate turns that burst into an orderly drain, transient provider errors retry with backoff while bad addresses fail fast, and every send records its final outcome.
sizing a pool to a provider's limit
Queues drain in enqueue order, so one enterprise account's forty-thousand-row reindex parks ahead of everyone else's three-second jobs. Round-robin dispatch with a per-tenant in-flight cap keeps every account moving — a tenant queueing forty thousand rows delays a neighbour by at most one dispatch cycle — and a stalled queue recovers on its own.
fairness needs dispatch, not a bigger cap
A saved file is not a finished file. Thumbnails, text extraction and search indexing run as chained stages after an instant upload, each stage retries on its own and reports its own status in the file list, and a failed stage leaves the file usable with its partial results intact.
a failed stage must not lose the file
Pool retries are at-least-once, so a job that times out after the provider already accepted it will run again. Every external effect claims a stable key before calling out, providers that accept idempotency keys receive one, and the ambiguous timeout case is reconciled instead of retried into a second charge.
the ambiguous timeout after a charge
When a downstream provider is on fire, stop consuming without dropping work. An operator switch halts admission while enqueues keep succeeding, in-flight jobs finish and the drain reads out live on the admin screen, resuming runs the backlog in its original order, and the pause records who set it and why.
halting admission while enqueues keep working