Crosshire
Testing · SQL
Build08 Testing · SQL · 3 of 3

Comparing two result sets

Half the assertions above are one question — do these two queries return the same thing — and the SQL for that question has three traps in it, each of which produces a green test.

Stand 29.07.2026

#EXCEPT, symmetrically and with counts

What
A EXCEPT B returns rows in A that are not in B and says nothing about rows in B that are not in A. A one-directional comparison of a deployed table against a rebuild therefore catches missing rows and is blind to extra ones — the more common failure, because extra rows are what a re-run produces.
Second trap: EXCEPT is EXCEPT DISTINCT. It deduplicates both inputs before subtracting, so one copy of a row and three copies compare as identical. Whenever counts matter — in a warehouse, always — the operator is EXCEPT ALL.
Why
Every refactor of a gold model ends with the same question: is the new table the same as the old one? Getting that comparison wrong is worse than not running it, because it produces a signed-off migration — the rebuild is verified, the extra rows the old job left behind are not, and they ship.
The three traps share a property that makes them expensive: each one fails toward green. A one-directional comparison, a deduplicating operator and a NULL-unsafe join all return fewer rows than the correct query, and an assertion that returns fewer rows is an assertion that passes. Nothing about a green parity check tells you which of the three you wrote.
How
sql
tests/sql/dim_customer_rebuild_parity.sql — symmetric, count-preserving
WITH rebuilt AS (
  SELECT
    -- make_sk() verbatim: trim, upper, coalesce to the sentinel, concat_ws,
    -- sha2. Any deviation and EVERY row reports as both missing and extra.
    sha2(concat_ws('||', COALESCE(UPPER(TRIM(customer_id)), '~')), 256)
                                        AS customer_sk,
    customer_id,
    customer_name,
    country,
    customer_segment,
    CAST(_tech_valid_from AS TIMESTAMP) AS _tech_valid_from,
    CAST(_tech_valid_to   AS TIMESTAMP) AS _tech_valid_to
  FROM silver.customer
),
deployed AS (
  SELECT
    customer_sk, customer_id, customer_name, country, customer_segment,
    _tech_valid_from, _tech_valid_to
  FROM gold.dim_customer
)
SELECT 'missing_from_gold' AS side, *
FROM (SELECT * FROM rebuilt  EXCEPT ALL SELECT * FROM deployed)
UNION ALL
SELECT 'extra_in_gold' AS side, *
FROM (SELECT * FROM deployed EXCEPT ALL SELECT * FROM rebuilt)
The side column matters more than it looks: missing and extra have different causes and different fixes.
Third trap, and the subtlest. Set operators treat NULL as equal to NULL, so EXCEPT is already NULL-safe. An equality join is not. Rewriting this as LEFT JOIN ... WHERE b.key IS NULL — which reads as the same check — changes the answer for every nullable column, because NULL = NULL is NULL and the row never matches:
sql
The same comparison as a join — NULL-safe AND symmetric
-- same WITH rebuilt AS (...), deployed AS (...) header as above
SELECT 'extra_in_gold' AS side, d.customer_sk, d._tech_valid_from
FROM deployed d
LEFT JOIN rebuilt r
       ON  r.customer_sk       IS NOT DISTINCT FROM d.customer_sk
       AND r._tech_valid_from  IS NOT DISTINCT FROM d._tech_valid_from
       AND r.customer_segment  IS NOT DISTINCT FROM d.customer_segment
-- customer_sk is a hash and is never NULL, which is the ONLY reason it can
-- serve as the did-it-match sentinel. Pick a nullable column here and the
-- anti-join reports matched rows as missing.
WHERE r.customer_sk IS NULL

UNION ALL

-- The second half is not optional. The half above finds rows gold has and the
-- rebuild does not; alone it is blind to rows the rebuild has and gold lost.
-- Stopping there fixes trap #3 and hands you trap #1 with NULL-safe predicates
-- bolted on — which is worse, because it now looks deliberate.
SELECT 'missing_from_gold' AS side, r.customer_sk, r._tech_valid_from
FROM rebuilt r
LEFT JOIN deployed d
       ON  d.customer_sk       IS NOT DISTINCT FROM r.customer_sk
       AND d._tech_valid_from  IS NOT DISTINCT FROM r._tech_valid_from
       AND d.customer_segment  IS NOT DISTINCT FROM r.customer_segment
WHERE d.customer_sk IS NULL
Even symmetric, this form is still weaker than the set operator, and the reason is trap #2 wearing a different hat: an anti-join asks did any row match, so three copies in deployed against one in rebuilt all match and nothing is returned. EXCEPT ALL subtracts multiplicity and returns the two surplus copies. Reach for the join form only when you need the matched-column detail an anti-join can carry — and when you do, write both directions in the same file, as above.
Doing it
  • Compare in both directions, always, with a side column naming the direction
  • EXCEPT ALL by defaultReach for EXCEPT DISTINCT only when you can say out loud why duplicates are acceptable.
  • Cast both sides to the same declared types inside the CTEs, before the set operator sees them
  • Keep the column list explicit on both sidesSELECT * over two tables that have drifted by one column fails with a positional type error naming the wrong column.
Embedding it
  • Show it rather than say it: duplicate one row in a copy of dim_customer and run EXCEPT and EXCEPT ALL side by sideTen seconds, and the rule never has to be repeated.
  • Add 'is the comparison symmetric?' to the review checklist for any migration pull request
  • Have the team write the parity assertion for the next migration themselvesReview only the direction and the operator.
Defaults
  • Symmetric EXCEPT ALL with a side column, as one file
  • Explicit column lists and explicit casts in both CTEs
  • IS NOT DISTINCT FROM in join predicates; IS DISTINCT FROM in filters
  • Parity assertions are temporary by design — delete them when the migration is done
Gotchas
  • One-directional EXCEPTIt catches missing rows and is blind to extra ones, so a re-run that duplicated a day's load passes the parity check written specifically to catch it.
  • Plain EXCEPT when counts matterIt deduplicates both sides first; one row versus three compares as identical, and the report double-counts revenue with a green test behind it.
  • Implicit coercion across the set operatorDECIMAL(18,2) on one side and DOUBLE on the other coerce silently to DOUBLE, cent-level differences appear that exist nowhere in the data, and you spend a day looking for a rounding bug in a correct transform.
  • Column order: set operators match by position, not by nameTwo SELECTs with the same columns in a different order compare successfully and compare the wrong things whenever the types line up.
  • Fixing one trap and inheriting anotherThe NULL-safe anti-join rewrite reads as the finished article, so it gets lifted into the next migration one-directional — extra rows invisible again — and duplicate-blind on top, because an anti-join asks whether ANY row matched and three copies against one all match. Two directions and EXCEPT ALL, or you have swapped which trap you are in.
  • Retyping the key recipe in the rebuild CTETRIM(UPPER(x)) where make_sk does COALESCE(UPPER(TRIM(x)), '~') hashes every row to a different value, so the parity check reports 100 % missing and 100 % extra. That reads as a catastrophic migration and is a paraphrased expression — paste the recipe with a comment naming its source, or call the function.

#Testing a metric view against an oracle

What
Unity Catalog metric views are GA, and every evaluation of a measure goes through the MEASURE() function. gold.mv_sales is where revenue is defined once for AI/BI dashboards, Genie, notebooks and SQL — so a mistake in it is a mistake in all of them at once. Third-party BI is the exception to that sentence rather than part of it: BI compatibility mode is Beta, it needs SET metric_view_bi_compatibility_mode = true re-set on every connection, and Microsoft has removed the option from the Power BI connector, so the surviving Power BI paths are a Native query using MEASURE() in DirectQuery, or an AI/BI dashboard. Do not promise a client one definition across five tools without checking which of their tools can still read it.
The test is an oracle: an independent expression of the same number, in plain SQL against gold.fct_order_line, compared symmetrically with the view.
Why
The argument for defining revenue once is that every consumer gets the same number. That only holds if the definition is right, and a definition used by five tools is a defect multiplied by five. Unlike a table, a metric view has no rows to inspect — the only way to see what it computes is to compute it another way.
The independence requirement is not process theatre. Measures fail through misunderstood semantics far more often than through broken SQL, and that is exactly what a second author with a different mental model catches.
How
sql
tests/sql/mv_sales_oracle.sql
-- The CTE is mandatory, not stylistic: a metric view cannot be joined by the
-- consuming query, so it is read once here and only the CTE meets the set
-- operator. The casts below are NOT doc-confirmed — verify on the workspace.
WITH from_view AS (
  SELECT
    country,
    CAST(month AS DATE)                      AS month,
    CAST(MEASURE(revenue) AS DECIMAL(18,2))  AS revenue,
    CAST(MEASURE(order_count) AS BIGINT)     AS order_count
  FROM gold.mv_sales
  GROUP BY country, month
),
oracle AS (
  SELECT
    c.country,
    CAST(DATE_TRUNC('MONTH', f.order_ts) AS DATE)  AS month,
    CAST(SUM(f.amount_eur) AS DECIMAL(18,2))       AS revenue,
    CAST(COUNT(DISTINCT f.order_id) AS BIGINT)     AS order_count
  FROM gold.fct_order_line f
  JOIN gold.dim_customer c
    ON  c.customer_sk = f.customer_sk
    -- the view joins the CURRENT version (see the mv_sales YAML), so the
    -- oracle does too. Join on the interval here and the oracle is red for
    -- every customer who ever moved country — a real question, but a
    -- different assertion, not this one.
    AND c._tech_is_current
  WHERE f.status IS DISTINCT FROM 'CANCELLED'
  GROUP BY 1, 2
)
SELECT 'metric_view_only' AS side, *
FROM (SELECT * FROM from_view EXCEPT ALL SELECT * FROM oracle)
UNION ALL
SELECT 'oracle_only' AS side, *
FROM (SELECT * FROM oracle EXCEPT ALL SELECT * FROM from_view)
Two deliberate asymmetries, and they are where this oracle earns its cost. The view filters with status <> 'CANCELLED'; the oracle filters with IS DISTINCT FROM. A row whose status is NULL is therefore counted by the oracle and dropped by the view, so the gap between them is the exact revenue that three-valued logic is deleting — invisible in the view itself, because a metric view has no rows to count.
The second is the dimension join. gold.mv_sales resolves country through _tech_is_current, which means a customer who moves from AT to DE takes three years of history with them, retroactively. That is a business decision, not a bug, and the oracle must join the way the view joins or it reports the decision as a defect every night. Write the interval-join version as a separate file whose returned rows quantify the shift — then take that number to the business rather than to the engineer.
Doing it
  • Write one oracle per measure — revenue, order_count, average_order_value, active_customersNot one oracle for the view as a whole.
  • Test average_order_value separatelyAVG of line amounts and revenue over order_count are different numbers, and the view has to say which it means.
  • Test active_customers on a month containing a cancelled-only customerWhether they count is a business decision, and this is where you find out it was never made.
  • Write the oracle's filter NULL-safely even though the view's is notThe difference between the two is the measurement of the problem; making them identical hides it.
  • Run a new oracle interactively on the target workspace before it joins the suiteThe CTE wrapper is required, the casts around MEASURE() are not documented either way, and a shape that fails at parse time is a red suite nobody can act on at 05:30.
Embedding it
  • Pair the view's author with the analyst who requested the measure and have the analyst write the oracle unaidedThe conversation after the first red result is the workshop.
  • When oracle and view disagree, do not decide who is right — take both to the business ownerHalf the time the view is correct and the requirement was wrong.
  • Make the oracle a required artefact for any new measureA measure without one is an opinion published to five tools.
Defaults
  • One oracle per measure, written by someone other than the view's author
  • Symmetric EXCEPT ALL, the metric view wrapped in a CTE because it cannot be joined, and explicit casts on both sidesThe cast placement is confirmed on the workspace, not assumed from the docs.
  • The oracle joins dimensions exactly as the metric view doesA different join is a different question and belongs in its own file.
  • Oracles run on metric-view change as well as on schedule — a definition moves without any data moving
Gotchas
  • The author writing their own oracleThe test proves internal consistency, passes forever, and gives false confidence to five downstream tools at once.
  • Type coercion across the set operatorMEASURE() returning DOUBLE against an oracle's DECIMAL(18,2) produces differences of 0.000000001 on every row — red on correct data, and within a fortnight someone adds a tolerance that hides a real break.
  • Testing only the totalRevenue can be right in aggregate and wrong per country when a dimension resolves through the wrong customer version, so the oracle must group by the dimensions people slice by.
  • Putting metric-view materialization or third-party metric import into a sizing estimateMaterialization is Public Preview, and metric import from third-party tools is Beta — neither belongs in one. If the oracle suite starts failing intermittently after someone enables materialization, suspect a stale materialization before you suspect the data.
  • Reading the CTE wrapper as a materialisation boundaryIt is there because a metric view cannot be joined by the consuming query, not because it freezes a result set — Spark inlines CTEs — so a plan you assumed was 'compute the view, then subtract' can be one plan, and the shape of this oracle is worth confirming on the workspace rather than assuming from the SQL.
  • Promising the definition to a BI tool that can no longer read itBI compatibility mode is Beta, its session flag has to be re-set on every connection, and the Power BI connector option has been removed — a Tableau Top-N filter or LOD expression compiles to a join, and a metric view cannot be joined, so it fails rather than degrades. Confirm the client's actual BI path before the oracle is used as evidence that everyone gets the same number.
Platform facts on this page verified 29.07.2026 against the official documentation. Volatile claims are anchored to the currency register. This is section 3 of 3 in 08 Testing · SQL.