Crosshire
Unity Catalog topology
Govern15 Unity Catalog topology · 3 of 3

Metadata that survives a redeploy

The catalog is the only documentation an analyst reads at 02:00. It is worth exactly as much as the deployment pipeline that maintains it.

Stand 29.07.2026

#Grain, owner and cadence, in the DDL

What
Three facts belong on every table, and only one of them is technical: what one row is, which group owns it, and how often it refreshes. They go in the table COMMENT, in a fixed format, deployed as DDL. Anything a consumer could misread goes in a column comment.
Why
Comments typed into the UI are deleted by the next CREATE OR REPLACE TABLE. Nobody notices, because the person who typed them is not the person who runs the deploy and the table still works. Months of accumulated knowledge disappear in one release, with no diff, no error and nothing to review.
The second reason pays back weekly. Catalog Explorer, Genie and every LLM-assisted query tool read this metadata, as does every new joiner. One column comment saying amount_eur includes cancelled lines is the difference between a wrong revenue dashboard and none.
How
sql
gold.fct_order_line — the convention (column list cut for length)
CREATE OR REPLACE TABLE gold.fct_order_line (
  order_line_sk STRING COMMENT 'Surrogate key: sha2 of order_id + line_number',
  customer_sk   STRING COMMENT 'FK -> gold.dim_customer. INFORMATIONAL ONLY, not enforced. UNKNOWN where the customer did not resolve (rule 7).',
  order_ts      TIMESTAMP COMMENT 'Order timestamp from silver.sales_order. Second clustering key — do not drop it from a rebuild.',
  amount_eur    DECIMAL(18,2) COMMENT 'ROUND(quantity * unit_price * fx_rate, 2) (rule 2). Includes CANCELLED lines — revenue must filter them (rule 1).',
  status        STRING COMMENT 'OPEN | SHIPPED | CANCELLED'
  -- remaining columns exactly as declared in the model
)
CLUSTER BY (customer_sk, order_ts)
COMMENT 'Grain: one row per order line. Owner: dwh_gold_owners. Cadence: hourly, freshness SLA 26h.';
sql
The CI check — zero rows means pass
SELECT table_name, comment
FROM   dwh_prod.information_schema.tables
WHERE  table_schema = 'gold'
  AND  (comment IS NULL
        OR comment NOT LIKE 'Grain:%'
        OR comment NOT LIKE '%Owner:%'
        OR comment NOT LIKE '%Cadence:%');
Comments and tags are not interchangeable. A comment is prose for the human reading Catalog Explorer; a governed tag is a queryable key-value a policy can act on. Owner and cadence live in the comment because a person reads them; the pii tag lives in the tag because a policy reads it.
Doing it
  • Fix the table-comment format — Grain / Owner / Cadence — and check it by pattern rather than by review
  • Comment every key, every measure that carries a business rule, and every _tech_ column
  • Deploy comments in the same DDL as the tableIf a comment is worth typing, it is worth a pull request.
  • Treat AI-suggested comments as a draft to editThey describe the values, not the rule, and a plausible wrong comment is worse than an empty one.
Embedding it
  • Make the grain sentence the first review comment on every new tableRefuse to review anything else until it is right.
  • Ask a new joiner to explain amount_eur using only Catalog ExplorerTheir questions are the missing comments — have them write the ALTER statements themselves.
  • Let the team add the CI checkTwenty minutes of work, and a standard becomes something that fails a build.
  • When a schedule changes, ask in review where else that fact is written downThe second time, someone updates the comment in the same pull request without being asked.
Defaults
  • Table COMMENT format fixed and pattern-checked in CI
  • Column comments carry the rule, not a restatement of the column name
  • Comments in DDL, in the repo, reviewedThe UI is for reading metadata, not writing it.
  • Tags for machines, comments for humans — and never the same fact in both
Gotchas
  • CREATE OR REPLACE TABLE wipes UI-typed commentsThe release is green, the table is fine, and the documentation is gone with no diff and no error to notice.
  • A cadence in the comment that nobody updates when the schedule changesWrong documentation outlives missing documentation, because the on-call engineer at 02:00 believes it — tie the comment to the schedule change in the same pull request.
  • Comments that restate the column name'customer_sk: the customer surrogate key' costs a reader's attention and buys nothing; the useful version says it is informational-only and what UNKNOWN means.
  • Documenting the table on a wiki insteadThe page is correct on the day it is written; the COMMENT is correct on the day the DDL deploys — and only one of them is next to the data when somebody is actually looking at it.