Build09 Testing · pipelines · 3 of 3

Failure semantics

Whether an update ends is not a scheduling preference. It decides whether a failing rule produces a red build, a stale table, or a retry loop nobody is watching.

Stand 29.07.2026

#Triggered versus continuous

What

A triggered pipeline refreshes its datasets once and stops. A continuous pipeline keeps running and processes data as it arrives. The scheduling difference is obvious; the failure difference is not, and it is what decides whether any of this route applies.

What happens when an expectation with a fail action is violated

	Triggered	Continuous
The update	fails and terminates	the flow fails and is retried automatically
The job task	fails — the build goes red	there is no completion for a downstream task to wait for
Downstream tables	keep yesterday's data: stale, but internally consistent	partially updated, and the boundary moves while you look at it
What you notice	an alert, within minutes	rising cost and a pipeline that has been RUNNING for a suspiciously long time
Testable this way	yes — seed, run, assert	no — nothing ends, so nothing can be asserted after it

A deterministic data defect in continuous mode is the bad case: the record that violates the rule is still there on the retry, so it fails again, and the loop is paid for by the hour. The failure is not hidden — it is in the event log every time — but nothing pages anyone, because from the outside the pipeline is running.

Why

Teams choose continuous for freshness and inherit an operational model they did not choose: every retry costs compute, no retry produces an alert, and the SLA breach is found by whoever opens the dashboard rather than by monitoring.

For this warehouse the freshness requirement is a daily SAP extract and a webshop feed. Triggered, on a schedule, with a 26-hour freshness SLA asserted separately — see the observability route — is both cheaper and more testable. Continuous is a choice to be argued for on a named requirement, not a default to drift into.

Doing it

Declare continuous: false in the bundle for every pipeline in this warehouseRequire a written requirement to change it.
Use fail actions on rules where processing wrong data is worse than processing nonefx_rate_resolved is the candidate here, because an unresolved rate silently understates revenue. Use drop where a bad row is a nuisance and stale totals are worse.
If a pipeline must be continuous, alert on flow retry counts from the event logThe pipeline's own state will read RUNNING throughout.
Alert on the job, not on the taskA triggered pipeline that fails at task two of five should page once, with the chain's name on it.

Embedding it

Ask the team what happens to yesterday's dashboard when tonight's update failsIf they do not know, that is the runbook's first paragraph and they should write it.
Run the failure once in test: seed a row that violates a fail rule and watch the update stopThen check what the gold tables show. Everyone has a first-hand answer to the question the business will ask.
When someone proposes continuous, ask which decision gets made faster on minutes-old dataThat is, faster than it would on hours-old data. If nobody can name one, the requirement does not exist yet.
Give the retry-cost query to whoever owns the budgetCost is the fastest teacher of failure semantics.

Defaults

Triggered by default; continuous only against a written latency requirement with a named owner
continuous: false stated explicitly in the bundle, not inherited from a default
Fail actions on correctness rules, drop actions on hygiene rules, warn actions on anything you are still calibrating
Validation datasets live in a separate pipeline from the ones that feed goldA failing check then does not stop the warehouse — orchestrate the two with a job.

Gotchas

Flipping a pipeline to continuous for a latency experiment and leaving itThe CI job's assert task now has nothing to wait against, and the harness degrades to 'the pipeline started' without anybody editing a test.
A fail expectation on a deterministic defect in a continuous pipelineIt retries forever, the pipeline reports RUNNING, the bill grows, and the only symptom anyone sees is a table that stopped moving.
Assuming a triggered failure leaves gold half-writtenIt does not — the update stops and the previous version stands — which is good, and which means the real risk is silent staleness. That is a freshness check's job, not an expectation's.
Deploying the pipeline on CORE or PROExpectations require ADVANCED, and the FIRST UPDATE fails with an explanatory error — not the deploy. bundle deploy writes the pipeline definition and returns green; the graph is only compiled when an update runs, so the mismatch surfaces minutes later in a run, which is exactly why the build needs an update in it and not just a deploy step. The error itself is the right behaviour, and it is the one someone under deadline pressure resolves by deleting the expectations rather than by fixing one line of YAML.

Depends on