Runnable against the canonical model, as they would be committed. Each returns zero rows when the warehouse is healthy, so each is simultaneously a monitor and a test. That convention is worth exactly as much as the wrapper that carries it into the alert.
Stand 29.07.2026
sqlmonitors/_alert_wrapper.sql — the shape every one of the four is scheduled in
-- The check file is pasted verbatim into the subquery. The wrapper is what
-- the alert points at, so 'zero rows = healthy' survives the trip into the UI.
SELECT count(*) AS violations
FROM (
-- verbatim contents of monitors/freshness_fct_order_line.sql
) AS check_result;
-- Alert condition : aggregation MAX, column violations, operator >, threshold 0.
-- Empty result : cannot arise — an aggregate with no GROUP BY always returns
-- one row. Set the option anyway, so the next person sees a
-- decision rather than a default.
--
-- Do NOT choose aggregation COUNT here: it counts the wrapper's rows, which is
-- always 1, so the alert is permanently triggered and gets muted in a week.
Freshness is two numbers. When the pipeline last wrote and how recent the newest business event is fail differently, and most teams measure only the first.
Why
This catches an extract that stopped being scheduled. Nobody disables a job on purpose — a workspace is migrated, a service principal's token rotated, a bundle target redeployed without the schedule block. The pipeline does not fail, it does not run, and a job-failure alert has nothing to fire on.
The event clock catches the other half. bronze.sap_kna1 is a daily full extract with no change timestamp, so the pipeline cannot distinguish 'nothing changed' from 'this is Tuesday's file again'. Only MAX(order_ts) on the fact side notices.
How
sqlmonitors/freshness_fct_order_line.sql — zero rows = fresh
-- Two clocks, deliberately. A full refresh that rewrites yesterday's rows
-- keeps _tech_loaded_at current while the newest order is 40 hours old.
--
-- 26 h = 24 (run to run) + 1 (early finish one night, late the next)
-- + 1 (October: two 03:00 Europe/Berlin starts are 25 h apart).
SELECT
'gold.fct_order_line' AS object_name,
MAX(_tech_loaded_at) AS last_write,
MAX(order_ts) AS newest_event,
timestampdiff(HOUR, MAX(_tech_loaded_at), current_timestamp()) AS write_age_h,
timestampdiff(HOUR, MAX(order_ts), current_timestamp()) AS event_age_h
FROM dwh_prod.gold.fct_order_line
HAVING timestampdiff(HOUR, MAX(_tech_loaded_at), current_timestamp()) > 26
OR timestampdiff(HOUR, MAX(order_ts), current_timestamp()) > 26;
The SLA is 26 hours and the two extra hours are arithmetic, not slack. The chain starts at 03:00 Europe/Berlin and runs up to ninety minutes, so _tech_loaded_at lands somewhere between about 03:05 and 04:30. Two consecutive nights are therefore 24 hours apart give or take that hour of spread — an early finish followed by a late one is 25 — and a 24-hour SLA fires every time the run starts ten minutes late.
The 26th hour is October's. The check measures a wall-clock interval, current_timestamp() - MAX(_tech_loaded_at), so with the schedule pinned to Europe/Berlin the gap between two 03:00 starts is 25 hours on the night the clocks go back and 23 hours on the night they go forward. March's short night can only make the age smaller, which is why spring-forward can never trip a freshness SLA and is not what the extra hour is for. October's long night can, and is.
Doing it
Commit the check per gold table a report reads, with the SLA as a literalSo the diff shows when someone loosens it.
Derive the SLA from the schedule, not from a round numberRun-to-run gap + the spread between an early finish one night and a late one the next + the October hour, and only that hour if the schedule is pinned to a DST-observing timezone.
Pin the schedule's timezone deliberately and record the choice beside the SLAEurope/Berlin buys business-day alignment and one 25-hour night in October; UTC buys a constant 24-hour gap and an hour of seasonal drift against the business day.
Add the same check on silver.fx_rate — no downstream reader, largest blast radius
Embedding it
Ask the team to derive the 26 from their own run history rather than accepting itWhoever runs that query owns the number afterwards.
Have them pause a job in dwh_test for a day and confirm the alert reaches a phoneAn untested alert is a belief.
When the first weekend false positive lands, do not fix itHand it to whoever is on the rota and review their fix.
Defaults
Two clocks on every fact: load recency and event recency
SLA = run-to-run gap + run-duration spread + the October hour, written down with that reasoning
Schedules and freshness checks reason in one explicitly stated timezone, and the SLA names which one
Freshness on the bronze tables too — the earliest signal is the cheapest to act on
Gotchas
Measuring only _tech_loaded_atA full-refresh pipeline stamps every row on every run, so the table reports itself as minutes old forever — including the morning the source stopped delivering.
A 24-hour SLA on a daily jobIt fires on ordinary jitter, the team learns the alert means nothing, and it is muted before the month it would have mattered.
Pinning the schedule to Europe/Berlin and then deriving the SLA as if the run-to-run gap were a constant 24 hoursIt is 25 on the October night the clocks go back, so a 25-hour SLA false-fires on exactly one morning a year — the morning nothing is wrong — and the fix applied under pressure is usually to widen the threshold for the other 364 nights. March is the harmless direction: spring-forward shortens the gap to 23 hours, which can only make the measured age smaller.
A freshness check that itself runs once a dayWorst case you learn about a stale table 24 hours late. Run it hourly — it is one aggregate over a clustered table.
One event-age SLA on a B2B order bookSaturday and Sunday produce almost no orders, so MAX(order_ts) is past sixty hours old at 08:00 every Monday: the check fires weekly, on healthy data, until someone mutes it. Reason the event clock in business days off gold.dim_date, or evaluate it only on business days — and leave the write clock at 26 hours, because the pipeline does run at the weekend.
Row count is the highest-yield check per line of SQL in the warehouse. Median, not mean: one Black Friday drags a mean baseline for a week, and a median does not care.
Why
Volume catches failures that leave a table syntactically perfect. A CDC connector on bronze.sap_vbap that resumes from the wrong offset delivers a real, well-typed day missing four hours of items. Freshness green, schema unchanged, every expectation passing — because every row that arrived was valid.
The upper band matters as much. A ratio above 2.0 is usually a re-run that duplicated a day, and the fact table accepts it silently: Unity Catalog primary keys are informational, so nothing rejects the duplicate and revenue doubles.
How
sqlmonitors/volume_fct_order_line.sql — yesterday vs the 7 days before it
-- The calendar is the spine, not the fact table. A day on which nothing
-- landed produces no group, so a GROUP BY over the fact alone cannot emit a
-- row for the total outage this check exists to catch.
WITH calendar AS (
SELECT dd.date AS d
FROM dwh_prod.gold.dim_date dd
WHERE dd.date >= current_date() - INTERVAL 40 DAYS
AND dd.date < current_date()
),
daily AS (
SELECT c.d, count(f.order_line_sk) AS line_count
FROM calendar c
LEFT JOIN (
SELECT date_sk, order_line_sk
FROM dwh_prod.gold.fct_order_line
WHERE date_sk >= current_date() - INTERVAL 40 DAYS
AND date_sk < current_date()
) f
ON f.date_sk = c.d
GROUP BY c.d
),
baseline AS (
SELECT t.d, t.line_count, median(h.line_count) AS median_7d
FROM daily t
JOIN daily h
ON h.d >= t.d - INTERVAL 7 DAYS
AND h.d < t.d
GROUP BY t.d, t.line_count
)
SELECT d, line_count, median_7d,
round(line_count / nullif(median_7d, 0), 2) AS ratio
FROM baseline
WHERE d = current_date() - INTERVAL 1 DAY
AND (line_count < 0.5 * median_7d OR line_count > 2.0 * median_7d);
Holidays have no arithmetic at all. This warehouse sells into DE, AT, CH and NL: 1 May is a public holiday in DE and AT, an ordinary working day in NL, and cantonal in CH — so 'is today a holiday in Switzerland' has no single answer. A dayofweek() rule cannot express that. A per-country holiday column on gold.dim_date can.
Doing it
Set the band from history, not from instinctCompute the observed ratio over 90 days, then pick bounds that would have fired on the incidents you remember and nothing else.
Run the check on each bronze table as well as the factBronze is where a per-source outage is visible first.
Alert on the ratio, not the raw countA count threshold needs rewriting every time the business grows.
Group by date_sk, never by _tech_loaded_at, so a late batch does not read as a spike
Bound the fact scan on order_ts as well as date_skfct_order_line is clustered by (customer_sk, order_ts), so a date_sk-only window reads files an order_ts range would have skipped — free today, a per-run cost once the fact is years deep.
Embedding it
Give the team the 90-day ratio distribution and let them argue the bounds out loudThe argument is the transfer; the numbers are secondary.
Have them replay a real incident against the check before trusting itIf it would not have fired, the check is decoration.
Rotate who reviews the weekly false-positive listOtherwise one person ends up tuning thresholds nobody else understands.
Defaults
Median over mean, ratio over absolute count, band derived from observed history
Like-for-like baselines: same weekday, business days only, one shared calendar on dim_date
Volume on bronze as well as gold
Both directions bandedA doubling is a duplicate load and it is the expensive one.
Gotchas
A mean baselineOne promotional spike raises it for a week, so the real drop that follows sits inside the band and never fires.
Weekend and holiday false positives left unfixedThe check becomes noise, the noise gets muted, and the muting is invisible in code review because it happened in the alert UI.
Counting rows written instead of rows for the business dayA late batch lands two days of orders in one run; the load count spikes while every per-day count is fine.
Only banding the low sideDuplicate loads survive longest, because everyone's instinct is that more data is not a problem.
Grouping the fact by day and waiting for the missing day to appearA day with no rows produces no group, so the total outage — the failure everyone assumes this check is for — returns zero rows and reads as healthy. Drive the check off gold.dim_date and LEFT JOIN the fact, so an absent day arrives as a count of 0 and trips the lower band.
An aggregate is an average of the working and the broken. DE is roughly two thirds of this order book, so NL going to zero moves the total by a few percent and sits inside any sane band. Per slice, or it does not catch segment failure at all.
Why
Distribution catches the fx_rate incident from the first topic: pct_amount_null goes from 0 to the CHF share of the day, on the morning it happens rather than at month end. pct_unknown_customer catches the mirror failure — a customer load that stopped, so new orders resolve to the unknown member and revenue is attributed to nobody.
Slices also change what an alert costs to act on. 'Volume down 8%' sends someone into the whole pipeline. 'NL is at zero, DE and AT normal' names the country and the source in one sentence.
How
sqlmonitors/volume_by_country.sql — the same test, one row per country
-- Same spine problem, one dimension worse: a country that goes to zero stops
-- producing rows at all. Build the country x day grid first, join the fact in.
WITH calendar AS (
SELECT dd.date AS d
FROM dwh_prod.gold.dim_date dd
WHERE dd.date >= current_date() - INTERVAL 40 DAYS
AND dd.date < current_date()
),
countries AS (
SELECT DISTINCT country
FROM dwh_prod.gold.dim_customer
WHERE _tech_is_current
),
grid AS (
SELECT k.country, c.d
FROM countries k CROSS JOIN calendar c
),
fact_lines AS (
SELECT c.country, f.date_sk AS d, f.order_line_sk
FROM dwh_prod.gold.fct_order_line f
JOIN dwh_prod.gold.dim_customer c
ON c.customer_sk = f.customer_sk
AND c._tech_is_current -- customer_sk is stable across SCD2 versions;
-- without this predicate every version fans out
WHERE f.date_sk >= current_date() - INTERVAL 40 DAYS
AND f.date_sk < current_date()
),
daily AS (
SELECT g.country, g.d, count(l.order_line_sk) AS line_count
FROM grid g
LEFT JOIN fact_lines l
ON l.country = g.country
AND l.d = g.d
GROUP BY g.country, g.d
),
baseline AS (
SELECT t.country, t.d, t.line_count, median(h.line_count) AS median_7d
FROM daily t
JOIN daily h
ON h.country = t.country
AND h.d >= t.d - INTERVAL 7 DAYS
AND h.d < t.d
GROUP BY t.country, t.d, t.line_count
)
SELECT country, d, line_count, median_7d
FROM baseline
WHERE d = current_date() - INTERVAL 1 DAY
AND (line_count < 0.5 * median_7d OR line_count > 2.0 * median_7d);
The other slice everyone wants is source system, and this model does not have it: gold.fct_order_line carries no source column. Source identity is the bronze table — SAP is bronze.sap_vbak / sap_vbap, the webshop is bronze.shop_order — so per-source volume is monitored there. Carrying source onto the fact is a data-model change, not a monitoring setting.
The fourth signal is shape rather than count, and it comes in two files that are easy to conflate. The panel below is eight days side by side — a human artefact for an incident review, and it returns eight rows every time by construction, so it is a report and is never wired to an alert. The monitor after it applies a threshold to the same percentages and returns zero rows when the shape held.
sqlreports/distribution_panel_fct_order_line.sql — a REPORT: eight rows always
-- Eight days side by side, for a human to read during an incident.
-- pct_amount_null is the column that shows the missing fx_rate.
-- This file is deliberately NOT an alert: it has no threshold and always
-- returns eight rows, so scheduling it would fire every night forever.
SELECT
date_sk,
count(*) AS line_count,
round(100.0 * count_if(amount_eur IS NULL) / count(*), 2) AS pct_amount_null,
round(100.0 * count_if(customer_sk = 'UNKNOWN') / count(*), 2) AS pct_unknown_customer,
round(100.0 * count_if(status = 'CANCELLED') / count(*), 2) AS pct_cancelled,
round(100.0 * count_if(currency = 'CHF') / count(*), 2) AS pct_chf
FROM dwh_prod.gold.fct_order_line
WHERE date_sk >= current_date() - INTERVAL 8 DAYS
AND date_sk < current_date() -- today is a partial day and always reads
-- as a step change; it is not one
GROUP BY date_sk
ORDER BY date_sk;
sqlmonitors/distribution_fct_order_line.sql — the same percentages, thresholded: zero rows = shape held
-- Yesterday's shape against the median of the seven days before it, one row
-- per percentage that moved. Median for the same reason volume uses one: a
-- single odd day must not redefine normal.
WITH daily AS (
SELECT
date_sk,
round(100.0 * count_if(amount_eur IS NULL) / count(*), 2) AS pct_amount_null,
round(100.0 * count_if(customer_sk = 'UNKNOWN') / count(*), 2) AS pct_unknown_customer,
round(100.0 * count_if(status = 'CANCELLED') / count(*), 2) AS pct_cancelled,
round(100.0 * count_if(currency = 'CHF') / count(*), 2) AS pct_chf
FROM dwh_prod.gold.fct_order_line
WHERE date_sk >= current_date() - INTERVAL 8 DAYS
AND date_sk < current_date()
GROUP BY date_sk
),
long AS (
SELECT date_sk, metric, pct
FROM daily
UNPIVOT (pct FOR metric IN (
pct_amount_null, pct_unknown_customer, pct_cancelled, pct_chf))
),
baseline AS (
SELECT metric, median(pct) AS median_7d
FROM long
WHERE date_sk < current_date() - INTERVAL 1 DAY
GROUP BY metric
)
SELECT y.metric,
y.pct AS pct_yesterday,
b.median_7d,
round(y.pct - b.median_7d, 2) AS points_moved
FROM long y
JOIN baseline b
ON b.metric = y.metric
WHERE y.date_sk = current_date() - INTERVAL 1 DAY
-- Two percentage points, or a quarter of the metric's own baseline,
-- whichever is larger. The absolute floor is what makes 0 -> 8 fire on
-- pct_amount_null; the relative term is what stops pct_cancelled at 30%
-- alarming on ordinary daily wobble.
AND abs(y.pct - b.median_7d) > greatest(2.0, 0.25 * b.median_7d);
Doing it
Slice by country via dim_customer, and by status and currency on the fact itself
Always add _tech_is_current when a monitor joins a fact to an SCD2 dimensionOr an interval predicate on _tech_valid_from / _tech_valid_to.
Monitor volume per bronze table so a webshop outage is visible before it reaches gold
Alert on a step change in a percentage, not on its level3% unknown customers may be normal; 3% after a month at 0% is not.
Keep the panel and the monitor as two files in two foldersA file with no threshold in monitors/ becomes an alert that fires every night, and the channel is muted before anyone notices which file did it.
Embedding it
Ask which slice would have made last quarter's worst incident obvious in one lineBuild that one first.
Have someone break a single country in dwh_test and see whether the unsliced check noticesIt will not, and that ends the argument about whether slices are worth the cost.
Make the SCD2 join predicate a review checklist itemEveryone gets it wrong once; the point is that review catches it, not a false alarm.
Defaults
One slice per dimension the business steers by — here country, and status
Percentages compared against their own recent history, never against a fixed target
Per-source monitoring on bronze, because the model deliberately does not carry source into gold
Cap the slice countFour countries is a check; four hundred customers is a noise generator.
Gotchas
Joining fct_order_line to dim_customer on customer_sk aloneThe surrogate key is a hash of customer_id and is stable across SCD2 versions, so the join fans out one row per version — and the monitor reports a volume spike that is a defect in the monitor.
Slicing by a high-cardinality columnPer-customer checks fire somewhere every day, which teaches the team that this channel is always red.
Assuming an aggregate inside its band is healthyAny slice smaller than the band width is invisible to the total: NL can go to zero without moving the number you are watching.
Alerting on an absolute percentageIt is right for one quarter, drifts with the business, and is then rewritten under pressure during an incident.
Scheduling the eight-day panel as an alertIt returns eight rows on the healthiest night of the year, so it fires nightly, and the threshold someone invents to silence it is a number nobody derived from anything.
Expecting a fact-driven GROUP BY to report a slice that vanishedNL at zero emits no NL rows, so the country disappears from the result instead of appearing at zero, and the alert built for exactly this case never fires. The countries x days grid has to exist before the fact is joined onto it.
Schema drift is usually diffed against a snapshot someone remembered to take. A better reference already exists in the estate: the schema CI actually tested. Diff dwh_prod against dwh_test, and zero rows means production is the shape the suite ran against.
Why
The concrete case is SAP. A field is added to KNA1 or VBAP by an ERP team who have never heard of this warehouse. CDC adds the column to bronze.sap_vbap and nothing complains. On bronze.shop_order, Auto Loader's default addNewColumns evolution stops the stream with an UnknownFieldException and picks the column up on the next start — which under a job retry reads as a transient failure that fixed itself, and is the only notification anyone gets. Either way silver ignores the new column, because the transform names its columns. Six months later someone wants that field and finds it has been arriving unmapped since spring.
The expensive version is a type change. A source field that widens upstream while the target stays narrow does not fail in one recognisable way: losing scale rounds silently, while a value that no longer fits the precision raises under ANSI mode — the serverless default this warehouse tests against — and returns NULL wherever ANSI is off. Two endings, one consequence: amount_eur is wrong for exactly the largest orders, the ones a spot check is least likely to include.
How
sqlmonitors/schema_drift.sql — symmetric, because one direction sees half
WITH prod AS (
SELECT table_schema, table_name, column_name, ordinal_position,
full_data_type, is_nullable
FROM dwh_prod.information_schema.columns
WHERE table_schema IN ('bronze', 'silver', 'gold')
),
test AS (
SELECT table_schema, table_name, column_name, ordinal_position,
full_data_type, is_nullable
FROM dwh_test.information_schema.columns
WHERE table_schema IN ('bronze', 'silver', 'gold')
)
SELECT 'only_in_prod' AS side, * FROM (SELECT * FROM prod EXCEPT SELECT * FROM test)
UNION ALL
SELECT 'only_in_test' AS side, * FROM (SELECT * FROM test EXCEPT SELECT * FROM prod);
full_data_type, not data_type: the latter reports DECIMAL for both DECIMAL(18,2) and DECIMAL(12,2), so a narrowing of amount_eur passes the diff untouched. ordinal_position is in the list to catch a positional insert about to write currency into status.
Doing it
Run the diff after every deployment and nightlyTreat a non-empty result as a deployment defect until proven otherwise.
Include ordinal_position and full_data_type; exclude nothing you cannot name a reason for
Use INSERT INTO … BY NAME wherever a target is written from a SELECTSo a reordered column fails loudly instead of shifting values.
Add the comment column to the diffGrain and ownership live in comments, so a dropped comment is a dropped contract.
Embedding it
Have the team add a column in dwh_test and watch the diff catch itThen drop it and watch the other direction catch that. The symmetry stops being theoretical in two minutes.
Ask in review where the source-owner conversation is recordedDrift is an organisational signal: someone changed something and did not tell you.
Let them find the permission gotcha themselvesRun the monitor as the service principal rather than as themselves.
Defaults
Diff prod against test, not against a snapshot file somebody has to maintain
Symmetric EXCEPT, alwaysOne direction reports additions and is blind to removals.
full_data_type over data_type; ordinal_position included
Writes by name, never by position
Gotchas
information_schema shows only objects the caller has privileges onA monitor running as a service principal without SELECT on a schema returns zero rows — indistinguishable from healthy. Assert the row count of both CTEs, not just the diff.
Diffing data_type instead of full_data_typeDECIMAL(18,2) narrowed to DECIMAL(12,2) is invisible, and the symptom is wrong totals confined to the largest orders.
Ignoring ordinal_position because 'nothing depends on column order'Positional INSERT … SELECT * does, and the failure is currency landing in status: same type, plausible values, no error.
Auto Loader schema evolution treated as a solution rather than an eventIt keeps the pipeline running, which is right, and it means new columns arrive with nobody deciding whether they belong in silver.