ingestlayer/docs

docs · 03 actions & templates

All fourteen actions.

The full set, grouped by category. Three control actions that can drop the event, three enrichment actions that add fields, four more for LLM-backed enrichment, one that calls any HTTP API, three mutation actions that rewrite the body. Run in declared order; first drop wins.

Control

Three ways to drop or yield.

Each can short-circuit the pipeline. The delivery is marked delivered on a drop — the pipeline ran to completion, the event correctly went nowhere.

filter — keep events that match

control · may drop the event

Evaluate one or more conditions against the event. If they fail (per the AND/OR combinator), the pipeline short-circuits — no destinations are hit, the delivery is marked delivered.

field is a $-rooted ref. value is a string; numeric ops coerce both sides via Number().

dedupe — drop if seen recently

control · may drop the event

Reads the resolved key off the event and drops the event if it's been seen inside the configured window. Same idea as the idempotency key at ingest, but evaluated per pipeline against any event field.

throttle — rate-limit per key

control · may drop the event

Counts events per key per window. Once the configured max is hit, further events either drop or have their delivery delayed to the next window.

Delay mode pushes the next attempt forward without counting it as a retry — see /docs/deliveries.

Enrichment

Eight ways to add fields.

Three read from your data: enrich.entity against your identity graph, enrich.person / enrich.company from public signal (email parsing, homepage scrape). Four call out to an LLM: classify, summarize, translate, and ai.transform for a free-form prompt. And http.request calls any external API, merging the response into $http.

enrich.entity — lookup against your identity graph

enrichment · adds new fields

Resolves a key field to an external_id, looks up the matching entities row, merges its traits onto $entity.<kind>. Missing rows are silent no-ops; downstream templates read undefined.

Read /docs/entities for how to populate the table in the first place.

enrich.person — profile waterfall

enrichment · adds new fields

Walks the providers list to assemble a person profile. `entity` looks up your identity graph by (kind, email) and merges traits. `internal` derives from public signal: email parse, name from the local-part, personal/disposable domain classification, optional Gravatar. First hit wins per field. Lands on $enrich.person.

Entity-tier traits are read live, so upsert.entity earlier in the same pipeline is visible here without a cache flush. Only the slower internal-tier derivations cache.

enrich.company — profile waterfall

enrichment · adds new fields

Same shape as enrich.person but keyed by domain. Accepts an email (takes the part after @) or a bare domain. `entity` looks up your identity graph; `internal` derives public signal from the domain — homepage title and description, branding, the email provider (Google Workspace, Microsoft 365, etc.). Lands on $enrich.company.

The entity tier looks up by the resolved domain, so an entity row with external_id='acme.com' matches every email from that domain.

classify — LLM labels with enum schema

enrichment · adds new fields

Sends the input to an LLM under a schema that constrains each dimension to its configured values. The output is validated against the schema before it lands on $classify.<dimension>, so downstream steps see a label from your enum or nothing.

summarize — LLM short summary

enrichment · adds new fields

Sends the configured input to an LLM and returns a sentence-bounded summary in your chosen style. Lands on $summarize.text.

translate — LLM translation

enrichment · adds new fields

Sends the source text and target language to an LLM and returns the translation only, no preamble. Lands on $translate.text.

ai.transform — free-form LLM step

enrichment · adds new fields

Run an arbitrary prompt over the event (or part of it). The model's reply is stored under a key you name, addressable downstream as $ai.<output>. Turn on JSON mode to parse the reply as structured data.

JSON mode is best-effort: clean JSON, fenced JSON, and JSON embedded in prose are all recovered; if nothing parses, the step warns and leaves $ai.<output> unset. Empty prompt or input skips the step.

http.request — call any API

enrichment · adds new fields

Calls an external HTTP API and merges the response into $http.<name> for later steps. A JSON response is parsed; anything else lands as raw text. The body and headers are templated, so you can send event fields and reference the result downstream.

Requests to private/internal hosts are blocked by default. Secret values injected via {{secrets.NAME}} never appear in run notes or logs.

Mutation

Three ways to rewrite.

redact.pii applies per destination (so Slack and the warehouse can receive different bodies from one event). upsert.entity writes the event into the identity graph. transform renders the outbound body with the $-dialect.

redact.pii — per-destination PII matrix

mutation · rewrites in place

Applied per destination, not per pipeline. Two detection mechanisms: content rules map each detectable PII type (email, phone, ip, credit_card) to a per-destination action, matched by regex on field values; key rules redact a field by its name (at any depth) regardless of content. Each action is preserve, mask, hash, or drop.

Content matching is regex on string values, so name and company can't be detected from a value alone — redact those by field name via keys.

upsert.entity — write to your identity graph

mutation · rewrites in place

Maps event fields onto trait names; upserts the entities row keyed by (kind, externalIdField). Traits deep-merge one level: partial updates don't blank out fields a prior write set.

The just-written traits are visible on $entity.<kind> for any later step in the same pipeline — no need to follow with enrich.entity.

transform — the JSON-ish escape hatch

mutation · rewrites in place

Renders a custom template into the outbound body. $-rooted refs are substituted with JSON-encoded values; now() yields an ISO timestamp; bare object keys are auto-quoted so the result parses as JSON.

Order

Declared order, first drop wins.

Actions run top-to-bottom as ordered in the dashboard. Drag-reorder the rows to change. Common idioms:


deliveriesback to docsnext: template syntax