ingestlayer/docs/v0.4

docs · 03 actions & templates

All twelve actions.

The full set, grouped by category. Three control actions that can drop the event, four enrichment actions that add fields, two more for LLM-backed enrichment, three mutation actions that rewrite the body. Run in declared order; first drop wins.

Control

Three ways to drop or yield.

Each can short-circuit the pipeline. The delivery is marked delivered on a drop — the pipeline ran to completion, the event correctly went nowhere.

filter — keep events that match

control · may drop the event

Evaluate one or more conditions against the event. If they fail (per the AND/OR combinator), the pipeline short-circuits — no destinations are hit, the delivery is marked delivered.

field is a $-rooted ref. value is a string; numeric ops coerce both sides via Number().

dedupe — drop if seen recently

control · may drop the event

Reads the resolved key off the event and drops the event if it's been seen inside the configured window. Same idea as the idempotency key at ingest, but evaluated per pipeline against any event field.

throttle — rate-limit per key

control · may drop the event

Counts events per key per window. Once the configured max is hit, further events either drop or have their delivery delayed to the next window.

Delay mode pushes the next attempt forward without counting it as a retry — see /docs/deliveries.

Enrichment

Five ways to add fields.

Two read from your data: enrich.entity against your identity graph, enrich.person / enrich.company from public signal (email parsing, homepage scrape). Three call out to an LLM: classify, summarize, translate.

enrich.entity — lookup against your identity graph

enrichment · adds new fields

Resolves a key field to an external_id, looks up the matching entities row, merges its traits onto $entity.<kind>. Missing rows are silent no-ops; downstream templates read undefined.

Read /docs/entities for how to populate the table in the first place.

enrich.person — profile waterfall

enrichment · adds new fields

Walks the providers list to assemble a person profile. `entity` looks up your identity graph by (kind, email) and merges traits. `internal` derives from public signal: email parse, name from the local-part, personal/disposable domain classification, optional Gravatar. First hit wins per field. Lands on $enrich.person.

Entity-tier traits are read live, so upsert.entity earlier in the same pipeline is visible here without a cache flush. Only the slower internal-tier derivations cache.

enrich.company — profile waterfall

enrichment · adds new fields

Same shape as enrich.person but keyed by domain. Accepts an email (takes the part after @) or a bare domain. `entity` looks up your identity graph; `internal` derives public signal from the domain — homepage title and description, branding, the email provider (Google Workspace, Microsoft 365, etc.). Lands on $enrich.company.

The entity tier looks up by the resolved domain, so an entity row with external_id='acme.com' matches every email from that domain.

classify — LLM labels with enum schema

enrichment · adds new fields

Sends the input to an LLM under a schema that constrains each dimension to its configured values. The output is validated against the schema before it lands on $classify.<dimension>, so downstream steps see a label from your enum or nothing.

summarize — LLM short summary

enrichment · adds new fields

Sends the configured input to an LLM and returns a sentence-bounded summary in your chosen style. Lands on $summarize.text.

translate — LLM translation

enrichment · adds new fields

Sends the source text and target language to an LLM and returns the translation only, no preamble. Lands on $translate.text.

Mutation

Three ways to rewrite.

redact.pii applies per destination (so Slack and the warehouse can receive different bodies from one event). upsert.entity writes the event into the identity graph. transform renders the outbound body with the $-dialect.

redact.pii — per-destination PII matrix

mutation · rewrites in place

Applied per destination, not per pipeline. The configured matrix maps each PII type (email, phone, ip, credit_card) to a per-destination action: preserve, mask, hash, or drop. Detection is regex-based on string values in the payload.

name and address PII types sit in the matrix but require NER to detect from values alone — currently no-ops.

upsert.entity — write to your identity graph

mutation · rewrites in place

Maps event fields onto trait names; upserts the entities row keyed by (kind, externalIdField). Traits deep-merge one level: partial updates don't blank out fields a prior write set.

The just-written traits are visible on $entity.<kind> for any later step in the same pipeline — no need to follow with enrich.entity.

transform — the JSON-ish escape hatch

mutation · rewrites in place

Renders a custom template into the outbound body. $-rooted refs are substituted with JSON-encoded values; now() yields an ISO timestamp; bare object keys are auto-quoted so the result parses as JSON.

Order

Declared order, first drop wins.

Actions run top-to-bottom as ordered in the dashboard. Drag-reorder the rows to change. Common idioms:


deliveriesback to docsnext: template syntax