Summarize webhook events with an LLM before they hit Slack.

The alert worth having says what happened in language: the invoice sync failed on every call since 2:10 — the Stripe key it uses was rotated, so last night's invoices never imported. One glance and you know what broke, who it affects, and whether it can wait until morning. The alert most channels carry instead is the payload itself — a wall of JSON with the one line that matters buried somewhere between retry_count and a stack trace.

The distance between those two messages is exactly one reader. Someone has to read the payload before it becomes a sentence — and until now that someone was you, at 2 a.m., squinting at your phone. The move this post walks through is to summarize webhook events with an LLM while they're still in flight, so the sentence is already written by the time anything pings you.

the sentence you actually want

Start from the message you'd be glad to receive. It leads with the job's name and a severity you can trust, because something judged it rather than defaulting to red. Then two sentences of plain prose: what failed, the likely why, what it took down with it. Not the stack — the point of the stack. Anyone on the team can read it, including the person who didn't write the job, and nobody has to expand a collapsed JSON blob to decide whether dinner is over.

why templates can't write it

The usual tool for making payloads readable is a message template, and templates are fine at what they do: pick fields, rearrange them, drop the rest. But a template can only repeat what a field already says. {{ $event.payload.error }} prints 401 Unauthorized; it can't notice that the same 401 has fired on every call since 2:10 and that the key was rotated yesterday, because that story is smeared across the stack, the job args, and the timestamps. No single field contains it.

Truncation, the other reflex, is worse — the first 200 characters of a stack trace are the least informative 200 characters in the payload. The payload was written for machines. Turning it into something written for people is a reading job, and templates don't read.

summarize events with an llm, mid-pipe

So put a reader in the pipe. A summarize step hands the payload to a model with tight bounds — you set the sentence count and the register, it returns prose that fits them, and the result lands on $summarize.text as just another field. Downstream steps and destination templates use it exactly like they'd use $event.payload.job. The model call happens after the event is acknowledged, on the pipe's clock, so the sender's timeout never meets the model's thinking time — the same reason classifying in a handler doesn't work and classifying in a pipe is boring.

Order matters more than usual once a model is involved, because every summary costs a call. dedupe goes first: a job that fails identically on five retries is one story, and should be one summary, not five. classify reads the error and writes down a severity you can gate on — so the channel only hears about events worth hearing about — and throttle caps the storm a bad deploy sets off. The model turns each event into a sentence; the control steps decide which sentences deserve a ping.

the pipeline

What lands in #ops is *sync-invoices* — page_me over two casual sentences a model wrote after actually reading the stack. What lands in Postgres is everything — the full payload, trace and all — for every failure, including the ones classified as noise that never made a sound. The fan-out is the quiet trick: the humans get the summary, the warehouse gets the truth, and neither destination compromises for the other.

how ingestlayer does this

summarize is a first-class action: you give it a field, a sentence budget, and a style, and it puts the result on $summarize.text for anything downstream. It stacks with classify, dedupe, and throttle in whatever order you choose, and any webhook you point at a pipeline gets the same treatment — your job runner today, a third-party service with payloads you didn't design tomorrow. It's the same shape as Slack pings without the noise, with one addition: there the pipe decided whether to speak, and here it also decides what to say.

Point a failure hook at a pipeline — the quickstart takes about five minutes.

il/eng