19Requirements
This route decides how a sentence spoken in a meeting becomes a table in gold — and how it gets refused when it cannot. Three artefacts do the work: an intake short enough that people actually fill it in, five feasibility questions asked in a fixed order, and a link between the requirement and the object that satisfies it that survives both of them leaving the company.
Intake
The request arrives as one sentence in a meeting. Everything expensive that follows is decided by how much of the rest gets written down in the next twenty minutes.
#The seven-field intake
| Field | What it must contain | REQ-118 |
|---|---|---|
| 1 · Requester + owner | two names; the owner is whoever signs the number off | requester: Head of Sales DE. Owner: Finance Controlling. |
| 2 · The question | one sentence, worded as the report title would be | Monthly revenue by customer segment and country, DE / AT / CH / NL. |
| 3 · Decision it changes | what somebody does differently depending on the answer | the quarterly segment-discount review, run today off a spreadsheet. |
| 4 · Grain, slices, as-of | one row per what, sliced by what, and as of when | one row per month × customer_segment × country; segment as of the order date, not today. |
| 5 · Measure definition | the arithmetic and the edge cases, citing the business rule | SUM(amount_eur) excluding status = 'CANCELLED' (rule 1); amount_eur already converted at the order-date rate (rule 2). |
| 6 · Source + freshness | which objects, and how late is too late | gold.fct_order_line + gold.dim_customer; loaded daily by 07:00 CET; month closes on working day 3. |
| 7 · Acceptance test | a query, a number, and who runs it | the query below; the June 2026 total agreed with Finance to the cent; the requester runs it. |
-- Rule 1: revenue excludes CANCELLED.
-- Field 4: customer_segment AS OF the order date.
SELECT
DATE_TRUNC('MONTH', f.order_ts) AS month,
c.customer_segment,
c.country,
ROUND(SUM(f.amount_eur), 2) AS revenue_eur,
COUNT(DISTINCT f.order_id) AS order_count
FROM gold.fct_order_line f
JOIN gold.dim_customer c
ON c.customer_sk = f.customer_sk
AND f.order_ts >= c._tech_valid_from
AND (c._tech_valid_to IS NULL OR f.order_ts < c._tech_valid_to)
WHERE f.status <> 'CANCELLED'
AND f.order_ts >= TIMESTAMP '2026-06-01'
AND f.order_ts < TIMESTAMP '2026-07-01'
GROUP BY ALL
ORDER BY month, customer_segment, country;
-- The other reading of the same sentence -- segment as of TODAY -- is
-- ON c.customer_sk = f.customer_sk AND c._tech_is_current
-- and returns a different number. Field 4 exists so that choice is made once,
-- by the owner, and not by whoever writes the join.Doing it
- Fill the form in the meeting, out loud, with the requester watching your screen. Emailing the template gets it back in three weeks with fields four and five empty.
- Write and run the acceptance query during intake, even against an empty gold — running it proves the objects and the join are agreed.
- Force the as-of decision explicitly whenever the request touches an SCD2 dimension, and record which reading was chosen.
- Restate the measure in your own words and wait. There is always a correction, and it is always cheaper now.
- Issue the requirement ID before anyone leaves the room; every branch, PR and table tag downstream references that string.
Defaults
- Seven fields, one screen, completed in the meeting — not sent as homework.
- The acceptance test is a query and a number, agreed before a line of build.
- Measure definitions cite the business rule number instead of restating prose per ticket.
- Every requirement has one named human owner. 'The business' is not a name.
Gotchas
- 'Segment as of today' versus 'segment as of the order date'. Both joins are correct SQL and they return different revenue. The divergence grows silently as customers move segment, and it surfaces months later as a trend chart that restates itself — which reads exactly like the business changing.
- An acceptance test written after the build. It is then written against what was built, passes by construction, and proves only that the code does what the code does.
- Field five answered with 'revenue, the usual'. Rule 1 (exclude CANCELLED) and rule 2 (the rounding in amount_eur) are precisely the 'usual' on which finance and sales differ by a few percent — enough for the report to be disbelieved once and then never trusted again.
- Skipping field six because latency 'is a technical detail'. A daily 07:00 load against a request that assumed intraday is discovered in UAT, by the requester, in front of their own stakeholders.
Feasibility, in order
Five questions, cheapest first. The order is the technique: three of the five can kill a request outright, and asking them out of order is how a sprint gets spent building something the data-protection officer was always going to veto.
#The five questions
| # | Question | Answered by | Cost | What 'no' means |
|---|---|---|---|---|
| 1 | Does the data exist in any source system at all? | the source owner, in a message | minutes | this is a source project, not a warehouse project — it leaves your backlog today |
| 2 | Does it reach us at the grain and latency the request needs? | the bronze table and the job schedule | an hour | a new feed or a cadence change: costed, scheduled, and dependent on someone else |
| 3 | Does the source carry the history the question needs? | the extract's specification — not its contents | a morning | frequently fatal, always a surprise; see bronze.sap_kna1 below |
| 4 | Do the keys survive conformance across sources? | profiling the join | a day | the answer stays per-source until someone makes an identity decision |
| 5 | Is anyone allowed to see it? | the classification and the access policy | a week — it involves people outside the team | a masked or aggregated version of the request, or nothing |
dim_customer.email, .phone, .street — ask question five first. It is the only one of the five whose answer is not yours to give, and the only one that can be reversed by someone who was not in the room.Doing it
- Run the five in order, out loud, with the requester present. The sequence is the argument.
- Timebox each: question one is a message, question five is a meeting. Two days into question three IS the answer — the history is not there.
- Record which question failed on the requirement itself. In six months nobody remembers whether it was a source problem or a governance one, and the request comes back.
- Jump straight to question five whenever email, phone or street appears in the request.
Defaults
- Fixed order, cheapest first; anything naming personal data jumps to question five.
- Every failed question is recorded with its evidence, not summarised as 'not feasible'.
- Feasibility ends in three options, never in a verdict.
- Question three is asked about semantics, question four about data — do not swap them.
Gotchas
- Answering question one by looking in the warehouse. The warehouse shows only what was already ingested, so you get a 'no' that actually means 'not yet' and a viable request dies of a lookup.
- Profiling data to answer question three. A snapshot source looks history-rich — many rows per customer — while the thing being asked for, when and why it changed, is not in there at any row count.
- Question five answered by an engineer. 'We can mask it' is true and is not the same statement as 'you may have it'. The engineer's yes gets quoted back six weeks later by someone who never spoke to the data-protection owner.
- Treating question four as a modelling task instead of a feasibility one. If SAP customers and webshop customers share no resolvable key, no amount of dimension design produces conformed revenue — and that is a finding for the intake, not a ticket for the sprint.
- changedUnity Catalog ABAC — row filter and column mask policies — plus governed tags and automated data classification are GENERALLY AVAILABLE.Stand 29.07.2026
- verifiedLineage system tables keep a rolling 1-year window. Catalog Explorer and the lineage API retain lineage captured after 01.09.2024 indefinitely.Stand 29.07.2026
#Question three failing: bronze.sap_kna1
bronze.sap_kna1, the SAP customer master as a daily full extract with no change timestamp. Nothing in the file says a change happened; the change is the difference between two files.- Answerable: that a customer's segment differs between two extracts we hold — to a resolution of one day, forward from the first snapshot retained.
- Not answerable: anything before that first retained snapshot. No engineering recovers it, at any budget. The information was never sent.
- Not answerable: a change that happened and reversed between two extracts. A → B → A over a weekend is, to this source, no change at all.
- Silently misdated: a change on a day whose extract never arrived is dated to the next extract we hold, and nothing marks it as approximate.
_tech_extract_date — the fourth technical column this source carries beyond the standard three, the extract's business date parsed from the filename by the Ingestion job. It takes a minute and it converts the estimate from a hope into a date:-- How far back can this source answer 'when did it change'?
-- _tech_extract_date is the extract's BUSINESS date, taken from the filename.
-- _tech_loaded_at would answer a different question -- when we loaded it --
-- and would move every answer on the day the extracts are re-run.
SELECT
MIN(_tech_extract_date) AS earliest_snapshot,
MAX(_tech_extract_date) AS latest_snapshot,
COUNT(DISTINCT _tech_extract_date) AS snapshots_held,
DATEDIFF(MAX(_tech_extract_date), MIN(_tech_extract_date)) + 1
- COUNT(DISTINCT _tech_extract_date) AS missing_days
FROM bronze.sap_kna1;earliest_snapshot is the honest start of the answer. missing_days is the honest caveat: each missing day is a day on which a real change gets stamped with the wrong date, invisibly. If the probe returns 2026-03-14 and four missing days, the sentence you owe the requester is 'from 14.03.2026, to the day, except across four days where the date may be late' — not 'partially'.Doing it
- Ask the source team for the extract's specification, not a sample. A sample cannot tell you whether a change timestamp exists.
- Run the probe before the estimate. Quoting a history start date you have not queried is how the March boundary becomes a surprise in the demo.
- State the limitation as a date and a count, in the requirement, in the requester's language.
- Record intra-extract reversal as a known limitation — it is invisible by construction, so it will never be found later by testing.
Defaults
- Interrogate the source's semantics before its contents.
- Every promise about history cites a queried date, never an adjective.
- Limitations that follow from the source live on the requirement and later in the table COMMENT.
- Snapshot sources make bronze retention a correctness requirement, not a storage preference — say so at intake, because it is a cost.
Gotchas
- 'We have a year of data' offered as the feasibility answer. It is a year of states. The request wanted changes, and the two coincide only where the source records a change event — which this one does not.
- Deriving the change date from _tech_loaded_at instead of the extract's business date. Re-running last month's extracts on a Friday stamps every change that Friday, the report shows one enormous migration day, and every job was green.
- Promising a backfill from the source archive before anyone has opened it. Archives are routinely incomplete, differently formatted, or kept for thirty days — all of which you discover after the commitment, not before.
- Treating missing_days as an operations problem to fix later. For this request it is a correctness caveat owed to the business now, because the misdating is silent and no downstream test can detect it.
#Never a flat no
| What you get | Cost | Depends on | Still not answered | |
|---|---|---|---|---|
| A · compare snapshots forward | segment changes to the day, from the first retained extract | ~2 days, this sprint | nothing outside the team | anything before that date; reversals inside one extract interval |
| B · request change documents from SAP | true change dates including history, back to SAP's own retention | 1 day our side, 4–8 weeks lead time | the SAP team's backlog | nothing — if it arrives, and it arrives later than anyone plans for |
| C · reconstruct from the extract archive | possibly several extra months of history | 1 sprint, quality unknown until it is opened | the archive existing and being complete | whatever the archive lacks — and you find that out at the end |
Doing it
- Never present fewer than three, and never pad to three with an option you would not take. A straw option is spotted instantly and it costs you the credibility of the other two.
- Cost in days and dependencies, not in t-shirt sizes. 'Large' is not a number anyone can plan against.
- State what each option still does not answer. That is the sentence people quote back to you, correctly, in six months.
- Recommend one. Three options with no recommendation is abdication wearing the costume of neutrality.
- Write the decision, the date, the decider and the rejected options into the requirement before the meeting ends.
Defaults
- Three real options, honest costs, one stated recommendation, one written decision.
- Every option names what it still leaves unanswered.
- Rejected options stay on the requirement — they are the answer when the same question returns next year.
- Anything depending on a third party is quoted as a lead time, never as an estimate.
Gotchas
- The straw option. Padding to three with something obviously unserious teaches the room that the exercise is theatre, and the next set of options gets read as sales material.
- Costing option B at one day because your part is one day. The SAP team's queue is the schedule; a plan that quietly assumes otherwise slips in public, and it slips at the demo.
- Options with no recommendation. The business picks the cheapest, hits its limitation, and feels misled — correctly, because you already knew which limitation they were choosing.
- Letting the chosen option live only in a slide. The chart's history starts in March forever, and whoever inherits it treats March as a bug and re-investigates the whole thing from scratch.
From requirement to table
Two objects with different lifetimes, and one link that has to survive a backlog migration, a table rename, and both original humans leaving.
#Ticket workflow and the link that survives
- Intake — seven fields complete. The requirement does not exist until they are.
- Feasible — five questions run, the failing one recorded with its evidence, three options presented, one chosen and signed.
- Specified — the acceptance test runs. Returning the wrong number is fine at this stage; failing to execute is not.
- In build — one or more issues, each carrying the requirement ID in the branch name and the PR title.
- Accepted — the acceptance test runs green with the requester watching. Not when the PR merges.
- Live — object tagged, owner group set, runbook entry exists.
gold.mv_sales, while the monthly pre-aggregate under the same dashboard is its own requirement with its own owner. That is normal, and it is why the link is carried by the object rather than by the report.ALTER TABLE gold.agg_revenue_month
SET TAGS ('requirement' = 'REQ-121', 'owner' = 'grp_dwh_gold');
COMMENT ON TABLE gold.agg_revenue_month IS
'One row per month x region_sk x product group. Revenue excludes
status = CANCELLED (rule 1). Requirement REQ-121. Owner grp_dwh_gold.
Loaded daily 07:30 CET from gold.fct_order_line, after the 07:00 gold load.';SELECT t.table_name
FROM dwh_prod.information_schema.tables t
LEFT JOIN dwh_prod.information_schema.table_tags g
ON g.schema_name = t.table_schema
AND g.table_name = t.table_name
AND g.tag_name = 'requirement'
WHERE t.table_schema = 'gold'
AND t.table_type = 'MANAGED'
AND g.tag_name IS NULL
ORDER BY t.table_name;gold.agg_revenue_month excludes cancelled orders. Without the link the answer is archaeology: git blame, then a chat search, then a person who has left. With it, the tag names REQ-121, and REQ-121 names the rule, the owner, the acceptance number and the date it was agreed.Doing it
- Put the requirement ID in the branch name, the PR title and the table tag. One string, three places, all greppable.
- Set tags and comments in deployed DDL. A tag clicked into Catalog Explorer survives only until the pipeline next recreates the table.
- Close issues on merge and requirements on acceptance — never the other way round, and never both at once.
- Run the orphan sweep monthly and put the output into the team's review, not into your status report.
Defaults
- Requirement and issue are separate objects with separate owners and separate lifetimes.
- Accepted means the acceptance test ran green with the requester in the room.
- The link lives in the warehouse as a governed tag, not only in the tracker.
- Rejected and cancelled requirements are kept — they are the record of what was already asked and answered.
- The orphan sweep is a standing monthly item owned by the team.
Gotchas
- Closing the requirement when the PR merges. 'Done' then means 'deployed', and the first person to notice the number is wrong is the person who asked for it — in a meeting, with an audience.
- The requirement living in a chat thread. In eighteen months the reason revenue excludes cancelled orders sits in a channel of a tool the client has since replaced, and the rule gets re-decided differently.
- One epic per report with forty issues under it. When a rule turns out to be wrong you cannot tell which change implemented it, so the investigation is a re-read of everything and the estimate for a one-line fix is a week.
- Tags applied by hand in Catalog Explorer, or a tagged table renamed without touching the requirement. Both fail the same way: nothing errors, the link simply stops existing, and the orphan sweep is the only thing that notices.
- Deploying DDL that sets a governed tag key nobody has created yet. Governed tags are defined once at account level and assigned by whoever holds that permission — not by the engineer writing the ALTER TABLE — so the release fails on the tag rather than on the table, and unblocking it needs a person who is not in the sprint. Create the keys with the catalogs, before the first table needs one.
Straight to the documentation
The generic platform material lives in the official docs and is not restated here. These are the pages worth having open while you work through this route.
- Attribute-based access control in Unity Catalog — feasibility question five — what a tag-driven policy can actually do, before you promise it at intake
- ALTER TABLE — exact SET TAGS syntax, so the requirement link is deployable DDL rather than a click
- Information schema — the column names behind the orphan sweep — tables and table_tags differ in how they name catalog and schema
- Lineage system tables reference — the two retention regimes that decide whether a historical audit question is answerable at all
- System tables — check the platform-side question — who read it, what it cost — is answerable before you accept a request that assumes it