Node · jobs
API platform
The request is not the job
The product API did HTTP, long jobs, and inbound webhooks in one request cycle. Reports timed out, a failed webhook was gone, and one shared key could touch every resource. The request path is now Fastify. Slow work is a BullMQ job the client polls.
- Node.js
- Fastify
- BullMQ
- PostgreSQL
- Redis
- OpenAPI
REQUEST
Fastify, OpenAPI from the route schemas
JOBS
BullMQ with a status endpoint
WEBHOOKS
Accept, then retry with backoff
ACCESS
Scoped keys, per-resource, per-key limits
01The problem
The backend was one synchronous app. A request, a report, a bulk import, a notification, and an inbound webhook all ran inside the same cycle.
Anything longer than the client would wait timed out. A webhook that hit a slow downstream call was dropped, and nothing recorded that it had arrived. The same aggregate queries ran on every read. Auth was one shared API key. Any integration could touch any resource.
02What we designed
Fastify handles the request. Route schemas are the OpenAPI document the client teams consume.
A report, an import, or a notification is a BullMQ job. The response returns a job id. The client polls a status endpoint. The HTTP handler does not wait for the work.
Webhooks are two steps. The handler stores the delivery and returns. A worker processes it, and a failure retries with backoff. Receiving the call and finishing the work are separate events.
Aggregates that were expensive enough to cache sit in Redis. A write that can change one deletes that key. Numbers close to billing should change when the write commits, not when a timer fires.
Keys are scoped. A key names the resources it can touch, and it has its own rate limit.
03What I owned
The split between the request and the job, the webhook accept-then-process path, which writes invalidate which cache keys, and the scoped-key model.