Crosshire
← Databricks Audit
— Crosshire / Learn · Tech · Databricks · Audit

Query lineage

A dbt-style map of how the hundred queries connect. Every arrow is a query’s own where-to-go-next link — the same graph the investigation trails walk as curated paths, drawn here in full. Start with the domain-flow overview, then drop into a domain to see its queries wired together. Click any node to open the query.

The graph is derived from the manifest, not hand-drawn: nodes are queries, arrows are their next edges. Solid arrows move you forward to the next step; dashed arrows point back to an earlier query you have already seen (a “see also” cross-reference). Hover an arrow to read the condition under which you’d follow it. Node colour is the domain; ★ marks a first-audit pick.

  • Next step
  • See also (back-reference)
  • First-audit pick
  • Leaves the domain (see the overview)

How the domains connect

The twenty-two cross-domain links, aggregated. An investigation fans in toward cost — where the money question is settled — then a few return branches (dashed) send you back out to check the compute, jobs or serving behind a bill. Thicker arrows carry more links; the number is how many.

52222Governance, Access & Security21Storage & Optimization9Query Performance12Jobs & Pipelines21Compute10Model Serving & AI4Cost & Billing23

Each domain, wired together

One graph per domain, laid out left→right by depth: the queries you start from are on the left, the ones you reach by following links are further right. Wide graphs scroll sideways.

Cost & Billing23 queries

if for the full dollarized cost series across all SKUs → Dollarizing DBUs at List Priceif for the underlying raw price rows behind this comparison → Your List-Price Basis by SKUif for the total DBU denominator to compute an untagged % → The Base Billing Rollupif for a DBU-only actual-vs-list comparison → Discount Realization by SKU (actual vs list)if to see which specific clusters/warehouses carry the premium usage → Cost Per Cluster and Warehouseif for the model-serving-specific premium/cost-mode cut → Model-Serving Cost Mode Per Endpointif to compare this rate against list/effective pricing → Discount Realization by SKU (actual vs list)if for the full raw pricing struct including effective_list and promotional → The List-Price Rate Bookif for the same window broken out by SKU and workspace → The Base Billing Rollupif to drill a JOBS spike down to individual jobs → Per-Job DBU Cost Rankingif to see which cluster is absorbing the notebook's cost → Cost Per Cluster and Warehouseif to compare against scheduled-job DBU cost in the same window → Per-Job DBU Cost Rankingif for the dollarized VECTOR_SEARCH cut → Vector Search Spend by Endpointif for a tag-based cut of the same window → Chargeback Coverage by Tagif to resolve workspace_id to a human name → Workspace ID to Name Lookupif for the same window broken out by SKU/workspace instead of cloud/currency → The Base Billing Rollupif for the DBU-billed egress slice specifically → Billed Data Egress Signalif uncovered serverless DBSQL usage is one likely driver of the gap → Serverless Policy Coverage Gapsif for an identity-level cut of the same SQL usage → Chargeback DBUs by Identityif for the total spend context → The Base Billing Rollupif for the dollarized VECTOR_SEARCH cut → Vector Search Spend by Endpointif for the broader DBU-derived cloud cost estimate → Estimated Cloud Spend, DBU-Derivedif for total usage by product line in the same window → Where the Money Goes by Productif for the same net-corrected totals broken out by SKU/workspace → The Base Billing Rollupif for the net totals by product line → Where the Money Goes by Productif for the same window rolled up to product line only → Where the Money Goes by Productif to resolve workspace_id to a human name → Workspace ID to Name Lookupif tag coverage is tracked separately from policy coverage here → Chargeback Coverage by Tagif for the total DBU denominator → The Base Billing Rollupif to join workspace_id to workspace_name onto the cost totals → The Base Billing Rollupif to join the same onto the identity chargeback cut → Chargeback DBUs by Identityif for a ready-made actual-vs-list comparison built from this same table → Discount Realization by SKU (actual vs list)Default-Storage DSU by API Tiercost_default_storage_dsuDiscount Realization by SKU (actual vs list)cost_actual_vs_list_by_skuCan You Trust the Bill?cost_restatement_trust_metricEstimated Cloud Spend, DBU-Derivedcost_cloud_infraAd-hoc notebook DBU costcost_by_notebookDBSQL Allocation Gap Checkcost_dbsql_allocation_gapServerless, Photon & Tier Premiumscost_premium_serverless_photonServing & Vector Spend by Endpointcost_by_serving_endpointGenAI Token & GPU Spendcost_genai_token_gpuDollarizing DBUs at List Pricecost_dollarized_by_sku_dayYour List-Price Basis by SKUcost_account_prices_rawServerless Policy Coverage Gapscost_usage_policy_coverageBilled Data Egress Signalcost_networking_egressCost Per Cluster and Warehousecost_by_compute_resourceModel-Serving Cost Mode Per Endpointcost_serving_mode_by_endpointVector Search Spend by Endpointcost_vector_search_spendThe List-Price Rate Bookpricing_list_prices_rawChargeback Coverage by Tagcost_chargeback_by_tagThe Base Billing Rollupcost_totals_by_sku_dayWhere the Money Goes by Productcost_by_billing_origin_productWorkspace ID to Name Lookupcost_workspace_namesPer-Job DBU Cost Rankingcost_by_jobChargeback DBUs by Identitycost_chargeback_by_identity
Leaves Cost & Billing →

Query Performance12 queries

if to roll repeat shapes up by fingerprint → Heaviest Query Shapes, Grouped by Fingerprintif read_files far exceeds pruned_files → Are Your Scans Skipping Files?if spilled_local_bytes is high → Disk Spill Memory-Pressure Hotspotsif you want to see the audit's own queries ranked individually → Your Heaviest Statements, Rankedif to see how the audit's load compares to the rest of your workload → When And What Runsif waiting_for_compute_ms_sum is also high → Capacity Queuing vs Cold-Start Waitsif the same warehouse also spills → Disk Spill Memory-Pressure Hotspotsif you want the individual failed statements → Your Heaviest Statements, Rankedif failures cluster with queuing or cold-start → Capacity Queuing vs Cold-Start Waitsif shuffle_read_bytes_sum is also high → Shuffle And Small-Files Hotspotsif you want the exact statements behind a spilling warehouse → Your Heaviest Statements, Rankedif for the same per-query granularity with pruning/shuffle/spill signals attached → Your Heaviest Statements, Rankedif for the same volume broken out by hour-of-day instead of source → When And What Runsif to turn this volume into a cost estimate → Per-Query Cost Allocation Inputsif the same reads are also shuffle-heavy → Shuffle And Small-Files Hotspotsif to see if waits line up with a load spike by hour → When And What Runsif the same warehouse also spills once queries do get compute → Disk Spill Memory-Pressure Hotspotsif the same warehouse also spills - shuffle and spill often co-occur → Disk Spill Memory-Pressure Hotspotsif a peak hour also shows queuing → Capacity Queuing vs Cold-Start Waitsif to see the individual runs behind a fingerprint → Your Heaviest Statements, Rankedif a hot shape scans far more files than it prunes → Are Your Scans Skipping Files?What This Audit Costs Youaudit_self_costCache Hits vs. Cold-Start Latencyquery_cache_coldstartDaily Failed & Canceled Queriesquery_failed_queries_dailyWhere Query Spend Comes Fromquery_provenance_by_sourcePer-Query Cost Allocation Inputsquery_per_query_estimate_laneWhen And What Runsquery_workload_mix_hoursYour Heaviest Statements, Rankedquery_costly_statementsCapacity Queuing vs Cold-Start Waitsquery_queuing_waitsHeaviest Query Shapes, Grouped by Fingerprintquery_costly_statements_groupedAre Your Scans Skipping Files?query_pruning_effectivenessShuffle And Small-Files Hotspotsquery_shuffle_write_amplificationDisk Spill Memory-Pressure Hotspotsquery_local_spillage
Leaves Query Performance →

Compute10 queries

if to check this warehouse's auto_stop_minutes → Live SQL Warehouse Config Baselineif for the raw event-type breakdown → SQL Warehouse Event Activityif the same warehouse is also thrashing clusters up/down → SQL Warehouses Thrashing Clustersif to size vCPU/memory/GPU for driver_node_type/worker_node_type → Node Type Hardware Spec Catalogif to see idle time on these classic clusters → Idle Classic Clusters, Rankedif to see this warehouse's min_clusters/max_clusters → Live SQL Warehouse Config Baselineif the same warehouse also shows a long RUNNING idle tail → Warehouse Idle-Tail Secondsif to see auto_termination_minutes for clusters using this node_type → Live Classic Cluster Config Baselineif these instances are pool-managed → Idle Instance-Pool Waste Snapshotif to see which clusters draw from this pool via driver_instance_pool_id/worker_instance_pool_id → Live Classic Cluster Config Baselineif to size the node_type this pool uses → Node Type Hardware Spec Catalogif to rank the same clusters by idle-slice ratio and wasted list-price $ → Idle Classic Clusters, Rankedif to see which clusters use a given node_type → Live Classic Cluster Config Baselineif for per-node utilization by node_type → Right-Sizing Evidence From Node Telemetryif to see this warehouse's actual RUNNING idle tail against its auto_stop_minutes → Warehouse Idle-Tail Secondsif to see if min_clusters/max_clusters is causing thrash → SQL Warehouses Thrashing Clustersif for the raw event history → SQL Warehouse Event Activityif to check auto_stop_minutes/warehouse_size for a dormant warehouse → Live SQL Warehouse Config Baselineif for the RUNNING idle-tail duration instead of just event counts → Warehouse Idle-Tail Secondsif for the scaling-specific churn signal → SQL Warehouses Thrashing ClustersWarehouse Idle-Tail Secondscompute_warehouse_idle_gapsSpot Mix and Instance Idle Signalinstance_events_idle_activeLive SQL Warehouse Config Baselinesql_warehouse_config_currentIdle Instance-Pool Waste Snapshotinstance_pools_idle_capacitySQL Warehouse Event Activitysql_warehouse_events_activityLive Classic Cluster Config Baselineclassic_clusters_config_currentSQL Warehouses Thrashing Clusterscompute_warehouse_autoscale_churnNode Type Hardware Spec Catalognode_types_referenceRight-Sizing Evidence From Node Telemetrynode_timeline_utilizationIdle Classic Clusters, Rankedcompute_idle_node_ratio
Leaves Compute →

Jobs & Pipelines21 queries

if for the org-wide termination_code breakdown → Job Failure Root-Cause Breakdownif to see how many attempts each failing run actually took → Flapping Jobs That Retryif these jobs also fail → Failing Jobs Ranked by Wasted DBUsif for the account-wide termination_code breakdown → Job Failure Root-Cause Breakdownif for the DBUs each failing job is burning → Failing Jobs Ranked by Wasted DBUsif for the related governance-coverage picture → Orphaned And Handed-Off Jobsif another active-job config gap worth checking alongside → Jobs Missing a Timeoutif for the related governance-coverage picture → Job Health-Rule Coverageif orphaned jobs are often also stale → Hunting Stale Zombie Jobsif for the full setup/queue/execution/cleanup breakdown → Where Job Time Actually Goesif the job-level version of this same gap → Jobs Missing a Timeoutif for tasks that DO have a timeout but are running close to it → Tasks Running Near Timeoutif the task-level version of this same gap → Task-Level Timeout Coverage Gapsif a no-timeout job that also never runs is lower priority → Hunting Stale Zombie Jobsif for the account-wide termination_code picture → Job Failure Root-Cause Breakdownif queueing and never-started runs often share a capacity root cause → Jobs Stuck Waiting On Capacityif to see whether the affected job runs often enough to be worth the fix → Daily Job Run Mix And Hoursif for cluster lingering after the pipeline's active window → Pipeline Active-Time and Idle-Tail Signalif for how much of that DBU spend is retries → Pipeline Refresh Failures And Retriesif for the full per-pipeline DBU picture → Per-Pipeline Cost and Over-Refreshif for the pipeline's serverless/continuous/development settings → DLT Pipeline Fleet Inventoryif to see the DBU cost of a pipeline that keeps retrying → Per-Pipeline Cost and Over-Refreshif for the same pipeline's idle/active time split → Pipeline Active-Time and Idle-Tail Signalif drill into DBU cost per pipeline → Per-Pipeline Cost and Over-Refreshif drill into idle-tail exposure per pipeline → Pipeline Active-Time and Idle-Tail Signalif for the DBU-waste view of the same failing jobs → Failing Jobs Ranked by Wasted DBUsif for the account-wide termination_code picture behind the retries → Job Failure Root-Cause Breakdownif stale jobs are often also ownership orphans → Orphaned And Handed-Off Jobsif a stale job is a low priority for health-rule coverage effort → Job Health-Rule Coverageif a task timing out is one common cause of a masked task failure → Tasks Running Near Timeoutif for the run-level setup/queue/execution breakdown → Where Job Time Actually Goesif drill into which jobs/triggers a given termination_code hits → Job Failure Breakdown by Root Causeif drill into the DBU cost of jobs terminating this way → Failing Jobs Ranked by Wasted DBUsif drill into runs behind a given termination_type once confirmed → Job Failure Breakdown by Root Causeif drill into failures within a given run_type/trigger_type → Job Failure Breakdown by Root Causeif drill into DBU cost for the pipeline-driven share of this mix → Per-Pipeline Cost and Over-RefreshJob Tasks On All-Purpose Computelakeflow_jobs_on_all_purposeGreen Runs Hiding Failed Taskslakeflow_succeeded_with_failed_tasksDiscover Job Termination-Type Valueslakeflow_termination_type_probeJob Health-Rule Coveragelakeflow_health_rule_coverageRuns Killed Before They Launchedlakeflow_never_started_runsFailing Jobs Ranked by Wasted DBUslakeflow_failed_jobs_wasted_dbusJobs Stuck Waiting On Capacitylakeflow_job_queue_timeOrphaned And Handed-Off Jobslakeflow_job_ownership_orphansJobs Missing a Timeoutlakeflow_jobs_no_timeoutFlapping Jobs That Retrylakeflow_retries_repairsHunting Stale Zombie Jobslakeflow_stale_zombie_jobsTask-Level Timeout Coverage Gapslakeflow_job_tasks_no_timeoutTasks Running Near Timeoutlakeflow_tasks_near_timeoutJob Failure Root-Cause Breakdownlakeflow_termination_taxonomyWhere Job Time Actually Goeslakeflow_phase_cold_startDaily Job Run Mix And Hourslakeflow_workload_mix_hoursPer-Pipeline Cost and Over-Refreshlakeflow_pipeline_costJob Failure Breakdown by Root Causelakeflow_failed_runsPipeline Refresh Failures And Retrieslakeflow_pipeline_update_failures_retriesPipeline Active-Time and Idle-Tail Signallakeflow_pipeline_idle_tail_durationDLT Pipeline Fleet Inventorylakeflow_pipelines_inventory_tier
Leaves Jobs & Pipelines →

Model Serving & AI4 queries

if you want the underlying serving-endpoint request/error/token detail → Daily Serving Endpoint Trafficif per-endpoint request and token detail → Daily Serving Endpoint Trafficif you want to know which of these endpoints have zero traffic → What Your Serving Endpoints Cost — and Which Are Idleif you want the per-served-entity dormancy read → What Your Serving Endpoints Cost — and Which Are IdleAI Gateway Abuse Signalscompute_ai_gateway_usageServing Endpoint Traffic Rollupserving_endpoint_traffic_by_endpointDaily Serving Endpoint Trafficcompute_serving_endpoint_usageWhat Your Serving Endpoints Cost — and Which Are Idlecompute_serving_endpoint_cost_status
Leaves Model Serving & AI →

Storage & Optimization9 queries

if to find EXTERNAL / non-DELTA tables worth checking → Table Inventory by Type and Formatif for a fuller per-table metadata pass → Delta Time-Travel & Optimize Settingsif churn is suspected → Clustering Key Churn Detectorif for the full per-table maintenance-cost picture across all PO operation types → Predictive Optimization Cost & Coverageif see the DBU cost this churn is generating → Liquid Clustering Work and Costif check other per-table config while you are looking at this table → Delta Time-Travel & Optimize Settingsif compaction/clustering activity on the same table → Liquid Clustering Work and Costif drill into CLUSTERING rows for a specific table → Liquid Clustering Work and Costif drill into VACUUM rows for a specific table → VACUUM Storage Reclaimed Per Tableif pair reclaimed bytes against vacuumable_bytes still on disk for the same table → Real Table Size Breakdownif check the retention config driving VACUUM eligibility → Delta Time-Travel & Optimize Settingsif see whether VACUUM already reclaimed this table recently → VACUUM Storage Reclaimed Per Tableif check the retention config driving time_travel_bytes → Delta Time-Travel & Optimize Settingsif drill into EXTERNAL / non-DELTA tables to confirm UniForm → Detecting UniForm & Managed-Iceberg Tablesif cross-check PO coverage against the EXTERNAL count here → Predictive Optimization Cost & Coverageif pairs this retention config with the actual time-travel / vacuumable bytes it produces → Real Table Size Breakdownif see whether VACUUM is actually running against this retention window → VACUUM Storage Reclaimed Per TableDetecting UniForm & Managed-Iceberg Tablesiceberg_uniform_metadataData-Skipping Stats Backfill Trackerpo_data_skipping_backfillTable Inventory by Type and Formattable_inventory_typePredictive Optimization Cost & Coveragepo_maintenance_cost_by_tableLiquid Clustering Work and Costpo_clustering_activityClustering Key Churn Detectorpo_clustering_column_churnDelta Time-Travel & Optimize Settingstable_props_time_travel_configReal Table Size Breakdownstorage_breakdown_analyzeVACUUM Storage Reclaimed Per Tablepo_vacuum_reclaimed_bytes
Leaves Storage & Optimization →

Governance, Access & Security21 queries

if check if the same run_by identity also touched admin roles → Who Changed Permissions, From Whereif see what the run_as identity can do → Who Holds Which Grantsif the same actor also shows run-as differences → Run-As Privilege Escalation Watchif see what that actor's role change actually granted → Who Holds Which Grantsif see every mask already in place → Which Columns Are Actually Maskedif the specific untagged-target finding this feeds → PII Leaking Into Untagged Columnsif the table-level rollup → Table Lineage Blast Radiusif cross-check which classified columns are missing from this list → Sensitive Columns Missing Their Maskif the row-level equivalent → Row Filter Coverage Inventoryif see which of these are masked → Sensitive Columns Missing Their Maskif see where this sensitive data propagates from here → PII Leaking Into Untagged Columnsif see the table's full lineage graph if it does show up elsewhere → Table Lineage Blast Radiusif schema/connection/credential/external-location grants → Grants on Schemas and Integrationsif cross-check a broadly-granted identity against run-as activity → Run-As Privilege Escalation Watchif table/catalog grants → Who Holds Which Grantsif cross-check a broadly-granted identity against run-as activity → Run-As Privilege Escalation Watchif check if the same principal shows run-as activity → Run-As Privilege Escalation Watchif check if they also touched admin roles → Who Changed Permissions, From Whereif check the egress side → Blocked Egress Exfiltration Attemptsif see if the same principal also touched admin roles → Who Changed Permissions, From Whereif check the ingress side → Blocked Inbound Network Attemptsif the blocked destination is a Vector Search endpoint → Vector Search Query Traffic by Endpointif see the full tag inventory → Manual Governance Tag Inventoryif check whether the untagged target is also unmasked → Sensitive Columns Missing Their Maskif the column-level equivalent → Which Columns Are Actually Maskedif see who has access to these filtered tables → Who Holds Which Grantsif the column-level equivalent → Column-Level Lineage Blast Radiusif check if this edge is also an untagged PII gap → PII Leaking Into Untagged Columnsif see where tagged sensitive data flows into untagged targets → PII Leaking Into Untagged Columnsif the auto-detected counterpart to these manual tags → Where Sensitive Data Actually Livesif the internal grant side of the same objects → Grants on Schemas and Integrationsif whether the shared tables carry sensitivity tags → Manual Governance Tag Inventoryif the table lineage these views mediate → Table Lineage Blast Radiusif unused views/tables to retire → Dead-Table Cleanup Candidatesif the account's tag vocabulary these should use → Manual Governance Tag Inventoryif table-level classification these volume files are invisible to → Where Sensitive Data Actually Livesif the full volume inventory this is derived from → Volumes — the Files Outside Your Tablesif the table-column PII this does NOT cover → Sensitive Columns Missing Their Maskif the account's tag vocabulary → Manual Governance Tag InventoryLogin Concentration & Anomaly Rollupaccess_login_concentrationWhat You're Sharing Outside — and With Whomaccess_delta_sharing_exposureBlocked Inbound Network Attemptsaccess_network_inbound_denialsPII the Masks Can't Reachaccess_pii_outside_tablesColumn-Level Lineage Blast Radiusaccess_column_lineage_sensitive_reachEvery View, and Which Are Materializedaccess_views_inventoryRun-As Privilege Escalation Watchaccess_runas_escalationVolumes — the Files Outside Your Tablesaccess_volumes_inventoryBlocked Egress Exfiltration Attemptsaccess_network_outbound_denialsDead-Table Cleanup Candidatesaccess_dead_table_candidatesWho Changed Permissions, From Whereaccess_admin_role_change_eventsTable Lineage Blast Radiusaccess_table_lineage_blast_radiusVector Search Query Traffic by Endpointaccess_vector_search_trafficPII Leaking Into Untagged Columnsaccess_pii_propagation_untaggedManual Governance Tag Inventoryaccess_tags_inventoryWhere Sensitive Data Actually Livesaccess_data_classification_inventorySensitive Columns Missing Their Maskaccess_classified_unmaskedWhich Columns Are Actually Maskedaccess_column_masks_inventoryRow Filter Coverage Inventoryaccess_row_filters_inventoryWho Holds Which Grantsaccess_grants_inventoryGrants on Schemas and Integrationsaccess_grants_inventory_extended
Leaves Governance, Access & Security →

Want the words, not the map? The investigation trails walk four of these paths step by step, and the first-audit runbook runs the ★10 in order. The coverage page shows which queries can come back empty, and why. Prefer the data model? The system-table map draws the tables and their joins, ER-style.