Four tag keys and two queries. This sits in the foundation group rather than in run because it is the one thing here that genuinely cannot be fixed later.
Four keys, lowercase, fixed value sets. Not five, and no free text — a tag with uncontrolled values is a tag you cannot GROUP BY.
The taxonomy
Key
Values
Set where
Answers
domain
sales | finance | platform
compute policy, fixed per policy
whose budget is this
environment
dev | test | prod
bundle target
production spend or experiment
cost_centre
CC-4711 and friends
compute policy
which line in the finance system
owner
group address, never a person
compute policy
who to ask before switching it off
Why
The failure mode is precise and it has a date on it. You reach the end of a quarter, finance asks which team spent the Databricks budget, and a third of the usage carries a NULL tag. The only honest answer is that it cannot be determined; fixing the tags today fixes next quarter and leaves this one unattributable permanently. At a DACH client with an internal cost-allocation process that is the meeting where the platform stops being able to justify its own budget, with the consultant who set it up in the room.
How
yamldatabricks.yml — tags per target, so environment cannot be wrong
# Declarative Automation Bundles (formerly Asset Bundles; the CLI
# command and the file format are unchanged).
targets:
prod:
variables:
catalog: dwh_prod
resources:
jobs:
silver_customer:
job_clusters:
- job_cluster_key: main
new_cluster:
custom_tags:
domain: sales
environment: prod
cost_centre: CC-4711
owner: dwh-sales@example.com
Doing it
Agree the four keys in week one, before any compute existsIt is a thirty-minute conversation that cannot be had retroactively.
Fix the values in the compute policy, rather than documenting the conventionA cluster then physically cannot be created untagged.
Set environment from the bundle target, so a dev job cannot claim to be prodIt holds even if someone copies the YAML.
Attach a serverless usage policy to everyone running serverless workloadsThen check custom_tags a day later to confirm it landed.
Run the untagged-spend query weekly for the first monthIt reaches zero, or there is a compute surface nobody has covered.
Embedding it
Have the team produce the untagged-spend percentage themselves in week twoThey present it to their own manager. A number they found is a number they fix.
Ask 'which cost_centre?' at every new-job reviewWhen nobody knows, the finding is not the missing tag — it is that the job has no owner.
Take the four keys to the client's finance contact and let them reject your value listsIt is their taxonomy; yours only has to be queryable.
Defaults
Four keys, fixed value sets, lowercase snake_case
owner is always a group address — a person leaves and the tag becomes a riddle
Tags enforced in policy and bundle, never in a wiki page
Every new compute surface gets a tagging answer before it gets a workload
Gotchas
Tags cannot be backfilledUsage rows are immutable, so spend that was untagged when you found the gap is unattributable for the rest of time. Discovering it in month four means months one to three are simply gone, and the number you say out loud is a percentage of the client's quarterly platform spend.
Serverless compute ignores cluster custom tagsA team with a perfect policy-based taxonomy migrates jobs to serverless and watches attribution collapse without a single tag being changed — and the natural conclusion, that someone broke the tagging, sends them looking in the wrong place.
Two different things called tagsCompute custom tags land in system.billing.usage.custom_tags and drive cost; Unity Catalog object tags, including governed tags, sit on catalogs, schemas, tables and columns and drive governance. Neither appears in the other's queries, so 'we tagged everything' can be true and useless at once.
Free-text values — Sales, sales and sales-team are three rows in a GROUP BYThey are one team in reality, and the report understates every one of them.
Attribution is one query. It joins usage in DBUs to the price in force at the time — the validity window in the join is the part people leave out.
Why
A budget alert is a smoke detector with a day-long fuse: the Friday-afternoon cluster is a finished incident by the time it goes off. That does not make budgets useless — they are the right instrument for a trend across a month — but the fast alarm is a scheduled query with an alert on its result, and you have to build it.
The second reason to own the query rather than a dashboard: it produces list cost, ignoring whatever commitment discount was negotiated. As a relative split between domains it is exactly right; as an absolute figure handed to a controller it is wrong, and being wrong about someone's bill is a credibility event you get to have once.
How
sqlCost by domain and environment, last 30 days
SELECT
u.custom_tags['domain'] AS domain,
u.custom_tags['environment'] AS environment,
u.billing_origin_product AS product,
ROUND(SUM(u.usage_quantity * p.pricing.effective_list.default), 2) AS list_cost
FROM system.billing.usage u
JOIN system.billing.list_prices p
ON u.cloud = p.cloud
AND u.sku_name = p.sku_name
AND u.usage_start_time >= p.price_start_time
AND (p.price_end_time IS NULL OR u.usage_start_time < p.price_end_time)
WHERE u.usage_date >= current_date() - INTERVAL 30 DAYS
GROUP BY 1, 2, 3
ORDER BY list_cost DESC;
sqlThe number that must be zero: unattributable spend this quarter
SELECT
COALESCE(custom_tags['domain'], '<untagged>') AS domain,
ROUND(SUM(usage_quantity), 2) AS dbus,
ROUND(100 * SUM(usage_quantity)
/ SUM(SUM(usage_quantity)) OVER (), 1) AS pct
FROM system.billing.usage
WHERE usage_date >= date_trunc('QUARTER', current_date())
GROUP BY 1
ORDER BY dbus DESC;
Account budgets sit on top: filter by workspace and by custom tag, set a threshold, get an email. They do not stop spend, and the notification carries up to a 24-hour delay between the usage happening and the mail arriving.
Doing it
Schedule both queries as SQL alertsThe cost breakdown weekly, the untagged percentage daily until it is zero.
Create budgets per domain tag rather than one for the accountAn account-level budget says spend rose and nothing about where.
Set the threshold at 80% of planAt 100% the alert and the overspend arrive together.
Report list cost internally and let procurement apply the discountNever send the list figure to a client's finance team as their bill.
Embedding it
Hand over the query, not the dashboardA team that can edit the SQL owns the number; a team with a dashboard owns a picture.
Have someone from finance in the room for the first monthly reviewThe taxonomy either survives contact with them or it was never real.
Ask the team to predict next month's figure before running the queryThe gap measures how well they understand their own platform, and it closes fast once it is public.
Defaults
Budgets scoped by domain tag, threshold at 80%, alerts routed to a rota
A daily scheduled query as the fast alarm; the budget as the slow trend
list_prices joined on the price validity window, always
Internal reporting in list cost, clearly labelled as list cost
Gotchas
Budget alerts do not stop spend and can lag usage by up to 24 hoursA cluster left running on Friday afternoon costs the whole weekend before the mail is sent, so treating the budget as your incident alarm means every cost incident runs to completion.
Joining usage to list_prices on sku_name alonePrices change, so without the price_start_time / price_end_time window you multiply last year's usage by today's price and silently restate history. Nothing errors; the number is simply wrong.
Quoting list price as the billsystem.billing.list_prices knows nothing about the client's commit discount — the relative split is right, the absolute number is not, and the two look identical on a slide.
Chasing the discrepancy between the budget page, the usage system table and yesterday's emailSystem tables refresh every few hours and the surfaces genuinely disagree about today; compare windows that are at least a day closed instead of debugging the difference.
One budget for the whole accountIt fires, everyone looks, nobody can say which team moved, and by the third time people have stopped looking.