Crosshire
← Databricks
— Crosshire / Learn · Tech · Databricks

Databricks Audit

96 copy-paste SQL queries that audit a Databricks account's cost, performance, compute, jobs, serving, storage and governance — built entirely on system.* tables. No app to install, no dashboard to wire up: open a query, read what it does and how to read the result, copy the SQL, and run it.

96 queries7 domains★ 10 first-audit picks
Start hereThe first audit — a guided runbookRun the ten highest-signal queries in order — trust check, then the money, then the risk — and walk away with a filled findings sheet in about 90 minutes. No system-tables background needed.

Everything here is open source — grab all 96 queries from the GitHub repository ↗. Clone the repo and every .sql file is ready to paste; each query below also links straight to its own source.

Who this is for

FinOps & cost owners

Find where the bill actually goes, prove the negotiated discount is landing, charge back by team, and hunt recoverable waste.

CostJobsComputeStorage

Platform & workspace admins

Right-size compute, move jobs off expensive all-purpose clusters, and shut down idle warehouses and dormant serving endpoints.

ComputeJobsServing

Data engineers

Tune the heaviest queries with real pruning / spill / shuffle signals, harden pipeline reliability, and keep table maintenance efficient.

PerformanceJobsStorage

Security, governance & compliance

Audit grants, masks and row filters, trace how far PII propagates, and catch run-as escalation and network exfiltration attempts.

Governance

The library, by domain

Seven domains. Pick one to browse its queries — almost every one is a copy-paste SELECT that takes a :period_days bind parameter.

Cost & Billing23Dollarized DBU spend by SKU / product / job / notebook / endpoint / tag / identity; list vs account price; egress; GenAI tokensQuery Performance12Costly statements, bytes scanned / pruning / spill / shuffle, cache cold-start, queuing, failed queries, workload mixCompute10Cluster & SQL-warehouse config, node utilization, idle ratio, autoscale churn, instance pools/eventsJobs & Pipelines21Job/pipeline runs, failures, timeouts, retries, queue/cold-start, orphans, wasted DBUs, all-purpose placementModel Serving & AI4Serving endpoint traffic, dormant endpoints, AI-gateway token usageStorage & Optimization9Predictive Optimization (clustering / compaction / VACUUM), table inventory, time-travel, Iceberg, ANALYZEGovernance, Access & Security17Grants, column masks / row filters, tags, data classification, lineage blast-radius, network denials, run-as escalation, admin changes

★ The first audit

New to the account? These ten are the highest-signal queries to run first — the biggest money and the biggest risk.

Discount Realization by SKU (actual vs list)cost_actual_vs_list_by_skuPer-Job DBU Cost Rankingcost_by_jobChargeback Coverage by Tagcost_chargeback_by_tagDollarizing DBUs at List Pricecost_dollarized_by_sku_dayServerless, Photon & Tier Premiumscost_premium_serverless_photonYour Heaviest Statements, Rankedquery_costly_statementsWarehouse Idle-Tail Secondscompute_warehouse_idle_gapsFailing Jobs Ranked by Wasted DBUslakeflow_failed_jobs_wasted_dbusJob Tasks On All-Purpose Computelakeflow_jobs_on_all_purposeRun-As Privilege Escalation Watchaccess_runas_escalation

How to run

  1. Open a Databricks SQL editor (or a notebook on a SQL warehouse / UC-enabled cluster).
  2. Make sure you have SELECT on the system.* schema the query reads — some schemas must be enabled by an account admin first.
  3. Open a query here, copy its SQL, set the look-back window on the :period_days parameter, and run.
Best-effort by design. Databricks system tables aren't uniformly available: a query may return nothing or raise TABLE_OR_VIEW_NOT_FOUND / insufficient_privileges on your account — that's expected, not a bug. Every dollar figure is est · at list (pre-discount, DBU-only) unless a query joins account_prices, and usage_quantity is DBUs / bytes / hours / tokens — never dollars. Sample-output tables are illustrative synthetic rows, never real account data.

Conventions

  • Read-only, with one exception. Every query is a plain SELECT except storage_breakdown_analyze, which runs ANALYZE TABLE … COMPUTE STORAGE METRICS — it writes table statistics (flagged in red on its page). Run it only against tables you are allowed to analyze.
  • Every dollar figure is est · at list (pre-discount, DBU-only from system.billing.list_prices) unless a query explicitly joins account_prices.
  • usage_quantity is DBUs / bytes / hours / tokens — never dollars.
  • Query output is sensitive: identities / emails are not masked by these raw queries.
  • Sample-output tables are illustrative synthetic rows, never real account data.
First-audit runbookTen queries, in order, in ~90 minutes.War storiesThree findings told as detective cases.Reference & glossaryParameters, terms, and the est · at list label.