21Enablement & coaching
This route decides how the team ends up owning the warehouse rather than watching it get built. It sets the engagement as a measurable curve, the four formats that actually move skill, and the prepared answers to the six objections that arrive in every project. The deliverable is not the warehouse — anyone can sell them that. It is the week their output overtakes yours.
The shape of the engagement
Enablement fails quietly: everything ships, everyone is happy, and six months later every change still arrives as a ticket addressed to you. The fix is to make transfer a number that is visible from week one.
#The declining-contribution model
| Week | Your share of merged PRs | What the team owns | Signal |
|---|---|---|---|
| 1–2 | ~90% | nothing yet — reading, then workshop 1 | the layer contract is committed, by them |
| 3–5 | ~70% | one silver table end to end | the first PR you did not have to touch |
| 6–8 | ~45% | two gold tables and the CI job | they review each other before you see it |
| 9 | ~30% — crossover | the alert rota | you come off the rota and nothing changes |
| 10–12 | ~15% | all pipelines, all runbooks | an incident resolved without you |
| 13–16 | ≤10% | everything; you review by exception | game day survived; your name on no runbook |
-- Action names differ per service. List yours first:
-- SELECT DISTINCT service_name, action_name FROM system.access.audit
-- WHERE event_date >= current_date() - INTERVAL 7 DAYS;
SELECT date_trunc('WEEK', event_date) AS week,
user_identity.email AS actor,
count(*) AS write_actions
FROM system.access.audit
WHERE event_date >= current_date() - INTERVAL 90 DAYS
AND service_name IN ('unityCatalog', 'jobs')
AND action_name RLIKE '(?i)create|update|delete|run'
GROUP BY ALL
ORDER BY week, write_actions DESC;silver.customer needs a new attribute and nobody but you has ever opened src/dwh/common/scd2.py.Doing it
- Put the curve and its measurement in the statement of work, with the week-9 crossover named.
- Run the audit query every Friday and publish it, including the weeks it moves the wrong way.
- From week 5, stop taking tickets that touch gold. Review them instead; the queue slowing down is the point.
- Leave the alert rota in week 9, not in the final week — standby for a fortnight, then off.
Defaults
- Numbers in the statement of work, measured weekly, visible to the sponsor.
- Off the alert rota by week 9 at the latest.
- One named human owner per gold table by week 6, written into the table COMMENT.
- The final quarter is theirs; you review by exception and write nothing.
Gotchas
- The line that never falls. You are faster, so you do it, so they never do. It does not show in the standup mood — it shows in the merged-PR count, which is why you count.
- Reaching the crossover by removing yourself rather than raising them. It measures as success and arrives as a 3am incident nobody can resolve. Tell them apart by asking who wrote the last three runbooks.
- Holiday as the real test. If the daily load cannot survive a week without you, week 9 did not happen whatever the chart says. Take that week off deliberately, in week 11.
- A plateau presented as ownership: they say they own it while every non-trivial PR waits on your approval. If you are a required reviewer everywhere, you are still the bottleneck under a new title.
#Eight workshops with run sheets
| # | Week · length | Subject | Artefact — merged, or it did not happen |
|---|---|---|---|
| 1 | 2 · 90 min | The layer contract | the contract table, filled in by them, committed to the repo |
| 2 | 3 · 120 min | Grain and the star | gold.fct_order_line DDL with a grain COMMENT, merged |
| 3 | 4 · 120 min | Keys and SCD2 | the boundary table, plus two-changes-in-one-day written as a test |
| 4 | 5 · 90 min | Ingestion decision table | all four sources placed, each with a written reason |
| 5 | 6 · 90 min | Testing I — the pure function | one transform extracted to src/dwh/pipelines/transforms/silver_sales_order.py, with a test |
| 6 | 7 · 90 min | Testing II — SQL assertions, then CI | three .sql assertions from RULES 1–7, running in the bundle test target |
| 7 | 8 · 90 min | Governed tags, ABAC, PII | gold.dim_customer.email tagged; a column mask proven from two accounts |
| 8 | 9 · 90 min | Runbooks and the re-run question | one runbook per pipeline, written by whoever gets woken |
gold.dim_customer. Every session runs the same six-segment sheet, and the segment that gets cut when you run late is the one that must not be:WORKSHOP 2 — Grain and the star week 3 · 120 min · max 6 people
PREREQ everyone has run 'databricks bundle validate' and can query dwh_dev.silver
ARTEFACT gold.fct_order_line DDL merged to main, with a grain COMMENT
00:00 frame One sentence on the whiteboard: "one row per order line."
Nothing else may be written until everyone agrees it.
00:10 hands Team draws the star. Consultant holds the pen for zero minutes.
00:40 break it "Sales wants revenue per delivery, not per order line."
New fact, or does that change this grain?
01:10 build Write the DDL together. CLUSTER BY (customer_sk, order_ts).
01:40 commit PR opened by a team member, reviewed by a second team member.
Consultant comments; consultant does not approve.
02:00 close Each person names the one thing they would still get wrong.- Layer contract — 'a dashboard needs a column that only exists in bronze. Where does it go, and what does the contract say?'
- Grain and the star — 'Sales wants revenue per delivery, not per order line.'
- Keys and SCD2 — 'the same customer changes customer_segment twice on one day. How many versions, and what are the interval bounds?'
- Ingestion — 'bronze.sap_kna1 arrives with no change timestamp. How do you know which customers changed?'
- Testing I — 'the transform needs today's date. Where does the clock come from, and how do you test yesterday?'
- Testing II — 'your assertion returns zero rows against an empty table. Did it pass, or did it not run?'
- Governed tags and ABAC — 'the analyst is excluded by the column mask. Can they still CLONE the table or TIME TRAVEL past it?'
- Runbooks — 'the load failed at 02:00 and nobody knows why. Is it safe to re-run blind?'
gold.fct_order_line is — discovered in month four, when the first aggregate double-counts and nobody can explain it.Doing it
- Publish all eight dates and artefacts before week 2, so the calendar is defensible when delivery pressure arrives.
- Check the prerequisite the day before. A room where half the people cannot connect loses forty minutes and its credibility.
- Cap attendance at six. The seventh person turns a workshop into a presentation.
- Keep your hands off the keyboard during the hands and build segments, without exception.
Defaults
- Weekly cadence, never a two-day bootcamp. Hands need a week between them.
- Ninety minutes default, 120 for modelling, six people, one merged artefact each.
- The same six-segment run sheet every time, so nobody spends attention on the format.
- Four sessions on architecture and modelling — protect that proportion when the schedule is squeezed.
Gotchas
- The two-day kickoff bootcamp. Retention is near zero because nothing is applied between sessions; you re-teach SCD2 in week 7 and the client concludes their team is slow.
- Workshops that drift into demos. The tell is the consultant being the only person typing for more than ten minutes. Say it out loud and hand over the keyboard.
- Cutting break-it to finish on time. They then learn only the happy path and meet the edge case in production, at the worst hour, without you.
- Inviting the department for visibility. Above six people nobody types, it becomes a status update, and the artefact is not built.
- Teaching workshop 7 as a preview briefing. Governed tags, ABAC policies and classification went GA in May 2026; the old fallback wastes the session and installs a workaround the team keeps for years.
- changedUnity Catalog ABAC — row filter and column mask policies — plus governed tags and automated data classification are GENERALLY AVAILABLE.Stand 29.07.2026
- verifiedDatabricks Asset Bundles are now Declarative Automation Bundles. The rename is non-breaking: the bundle CLI command and all existing configuration are unchanged.Stand 29.07.2026
The weekly rhythm
Workshops move understanding once a week. Skill moves in the loop that runs every day: who holds the keyboard, what a review comment says, and forty-five protected minutes where nothing has to ship.
#The pairing protocol
- Ninety minutes, in the calendar, one named outcome. Opportunistic pairing evaporates the week things get busy — which is the week it matters.
- The team member drives. Always. To show something you take the keyboard for under two minutes and hand it back with the change unmerged.
- Narrate intent, not keystrokes: 'we have to close the open interval before inserting the new version', never 'line 40, change the mode'.
- When the driver is stuck, count to twenty before speaking. Usually you never reach twenty.
- The driver writes the commit message unaided. If they cannot describe what changed, the session is not finished.
- Next session, verify: same driver, same class of change, you out of the room, and you read the PR afterwards.
src/dwh/common/scd2.py and gold.fct_order_line — the code they must change first and understand least — not on gold.dim_date, which pairs pleasantly and teaches nothing.git shortlog -sn with your name on every commit. A passenger learns what a passenger learns: they recognise the route and cannot navigate it.customer_segment change creates a version and an address change does not — one sentence separating a dimension that answers historical questions from one that quietly lies about them.Doing it
- Book the sessions for the whole engagement on day one; never week to week.
- Check whose name is on the commits. git shortlog -sn --since='4 weeks ago' takes five seconds and is not flattering.
- Pair on the on-call path — the MERGE, the retry logic, the currency conversion.
- After each session write the one thing the driver could not do alone. That list is your coaching backlog, and later your handover risk register.
Defaults
- Ninety minutes, booked, one outcome, the team member always driving.
- Under two minutes of consultant keyboard time, and the demonstration is discarded rather than merged.
- Rotate through everyone early, then concentrate on the future owners.
- Pair on the code that wakes people up, not the code that demos well.
Gotchas
- Consultant-drives pairing. Faster in the session, and it appears in the git log as your authorship during a week invoiced as enablement — exactly what a client checks when they later ask what they got.
- Pairing all day. Both people degrade after ninety minutes, and the afternoon session teaches the driver that pairing is exhausting, so they stop asking for it.
- Pairing only with the strongest engineer because it goes fastest. The person who needed it does not get it, and they are frequently the one still there in three years.
- Never pairing on the on-call path. You pair on the code you enjoy; the code that fails at 3am is the MERGE and the retry, and it stays untransferred.
#What a review comment says
| Bad | Why it fails | Good |
|---|---|---|
| Use liquid clustering here. | gives the answer, teaches nothing, and they ask again for the next table | This is PARTITIONED BY (year, month, day) — what does DESCRIBE DETAIL say about numFiles? Compare with gold.fct_order_line and tell me which you would rather query. |
| This is wrong. | no failure mode, no reproduction; invites defence instead of thought | This MERGE fails with 'multiple source rows matched' the first time bronze.sap_vbap sends two changes for one line in a batch. Run it against the two-changes-same-day fixture. |
| nit: naming | vague, and costs a full round trip to resolve | customer_key should be customer_sk — the prefix table in the architecture route. One line, then this is mergeable. |
| LGTM (on a 400-line PR) | approves what nobody read; teaches the team that approval is free | I read the SCD2 close and the tests. I did not read the FX join — @maria, can you take that half? |
| I've pushed a fix to your branch. | removes the learning and the authorship; the author finds out at merge | amount_eur is rounded before the FX multiply, so RULE 2 does not hold for CHF orders. Pair for twenty minutes, or shall I point you at the assertion that would have caught it? |
nit:. Give someone three hard questions and they answer the easy two and negotiate the third.amount_eur is rounded before the fx_rate multiply rather than after it — RULE 2 read backwards. Every CHF line is then wrong by a rounding step: too small to catch by eye in a spot check, systematic enough that the quarter's revenue disagrees with SAP, and nobody can say by how much or since when.Doing it
- Cite the rule number and link the test that would have caught it, instead of describing the fix.
- Never push to someone else's branch. There is no version of this that is worth it.
- State what you read and what you did not, and name who should read the rest.
- Ask for the missing test rather than the fix. The bug comes back; the test does not.
Defaults
- One thinking-question per review; everything mechanical prefixed nit: so the author can triage.
- Comments cite RULES 1–7, the layer contract or a route — never seniority.
- Reviews end in a test, not in a fix.
- Two team reviewers by week 4; the consultant is a required approver on nothing.
Gotchas
- The consultant as required reviewer everywhere. It looks like quality control and behaves like a queue: merge latency climbs, the team routes around you by splitting PRs into 'small' ones, and the risky change waits longest.
- Style-only reviews. They block a variable name and pass a wrong join, and the team correctly concludes review is theatre and starts approving each other unread.
- Pushing fixes to the author's branch. They never see the failure, git blame now points at someone who leaves in eight weeks, and the next person has nobody to ask.
- Teaching in a direct message. The lesson reaches one person and disappears. Put it in the PR thread, where the person who hits it in March can find it.
#Weekly katas on the canonical model
- Write the grain sentence for gold.agg_revenue_month, then find the two ways it can double-count. (RULE 1)
- Write the SCD2 close-and-insert MERGE on silver.customer for a changed customer_segment — and prove an address change on the same row creates no version. (RULE 4)
- Break RULE 2 deliberately: compute amount_eur before joining silver.fx_rate. Find the assertion that catches it; if none does, write it.
- Declare the primary key on gold.dim_customer, then insert a duplicate customer_sk. Watch it succeed. Now write the assertion that would have failed.
- Land an order line whose customer_id is not yet in gold.dim_customer on customer_sk = 'UNKNOWN' instead of letting the join drop it. (RULE 7)
- Two changes to one order line in a single bronze.sap_vbap batch. Make the MERGE fail with the real error, then make it pass without losing a change.
- Query system.billing.usage for last week by tag and explain the largest line in one sentence.
- Cold modelling on samples.nyctaxi.trips — one of the few places this handbook deliberately leaves the canonical model, and it says so, because this exercise needs a table nobody in the room has modelled before. Grain sentence and star, twenty minutes.
Doing it
- Book the recurring 45-minute slot for the whole engagement on day one, owned by someone who is not you.
- Throw the code away at the end. Every time.
- Keep the list of katas nobody finished. That list is next month's workshop.
- Retire a kata once two people solve it cold; keep the one that still hurts.
Defaults
- Forty-five minutes, weekly, thrown away, on the canonical model.
- One rule from RULES 1–7 per kata, so practice and production enforce the same seven sentences.
- Difficulty from the edge case, never from data volume.
- The last ten minutes of discussion is the exercise; the code was the pretext.
Gotchas
- Cancelled 'just this sprint'. It never returns. The recurring booking with a non-consultant owner exists to survive the sprint where it is inconvenient.
- Katas that turn into work. Once one result is merged, the next gets scheduled around delivery and the format is dead within a month.
- Toy data. A two-row fixture teaches syntax and hides every timing problem; you discover in production that nobody has seen an overlapping SCD2 interval.
- Only running katas the team already passes. It feels good and transfers nothing. If everyone finishes in twenty minutes, retire it that day.
Resistance, and what to stop doing
Six objections arrive in every warehouse engagement, usually in this order; the answers you improvise are the ones that sound like sales. Then subtract — enablement is mostly removing work, and every row below got there for a reason that made sense at the time.
| Stop | Why it started | Do instead |
|---|---|---|
| Weekly status decks | someone asked for visibility once | the declining-contribution query and the PR count, five lines in a channel |
| Consultant on the alert rota past week 9 | you are faster at 3am | team on the rota, you on standby a fortnight, then off |
| %run notebook chains | it worked in the prototype | workspace files and imports — %run hides the dependency graph and cannot be unit-tested |
| Scheduled OPTIMIZE on UC managed tables | necessary before predictive optimization existed | check whether predictive optimization is already on for this account first — if it is, the scheduled job pays a second time to compact files that were already compacted, and the two writers start losing to each other on concurrent-modification conflicts |
| A shared team mailbox for alerts | it felt like coverage | a named rota — a mailbox is where alerts go to be unread |
| Consultant as required PR approver | quality | two team reviewers; you comment without approving |
| Copying production data into dev | debugging | seeded fixtures in dwh_test.fixtures — a production copy in dev is a GDPR event with a ticket number |
| Documentation in the consultant's deck | it is what got sold | repository pages the team edits; anything only you can change dies at handover |
#Six objections, and what to say
| Objection | Response |
|---|---|
| You want to rewrite our working code. | No — I want to pin what it does today, then change it without changing that. Characterization test first, refactor second. |
| We don't have time to write tests. | You already write them: you eyeball the output in a notebook after every run. Same cost, except mine runs again next month. One assertion per bug from here; nothing retroactive. |
| Our data is too complex for a star schema. | Possibly. Take the three hardest questions the business asked last quarter and model only those — two hours. Usually the complexity is in the source, not the model; if it is not, we have learned something real. |
| Databricks does that automatically now. | Often true, and worth checking rather than arguing: predictive optimization runs OPTIMIZE, VACUUM and ANALYZE on Unity Catalog managed tables, on by default for accounts created on or after 11.11.2024. It does not decide your grain or notice that revenue is wrong. |
| The previous consultant said the opposite. | What problem was that solving, and does it still exist? Partitioning advice was right before liquid clustering; Databricks now recommends clustering over Z-ordering for new tables, and the two cannot be combined. |
| We'll do it properly later. | Name the date and price it. Retrofitting catalog-per-environment after go-live moves every table, reissues every grant, re-points every report, and has no partial rollout. 'Later' with a number attached stops being free. |
"""Pins TODAY's behaviour of the existing customer load.
Asserts nothing about correctness -- only that the refactor changes nothing
the business can see. Delete it once the refactor is merged and reviewed.
"""
from pyspark.testing import assertDataFrameEqual
from dwh.silver.customer import clean_customer
def test_clean_customer_matches_pinned_output(spark):
source = spark.read.table("dwh_test.fixtures.sap_kna1_2026_07_01")
pinned = spark.read.table("dwh_test.fixtures.silver_customer_pinned")
actual = clean_customer(source).drop("_tech_loaded_at")
assertDataFrameEqual(
actual,
pinned.drop("_tech_loaded_at"),
checkRowOrder=False,
)Doing it
- Write the six responses down before week one. You will be asked all six, usually in this order.
- Answer with a demonstration on a named date, not with an argument in the room.
- Concede the true part out loud, first, every time.
- Never a flat no: three options with their costs, per the requirements route.
- Check the currency register before contradicting any claim about what the platform now does by itself.
Defaults
- Six prepared answers, each conceding something true.
- Schedule the demonstration rather than winning the meeting.
- Platform claims get checked against the currency register before they get contradicted.
- A characterization test before any refactor of code that currently runs in production.
Gotchas
- Winning the argument. They concede in the meeting and revert in the code, and you find out at handover. The tell is agreement with no follow-up question.
- Treating 'no time for tests' as a values problem. It is a schedule problem and it is usually accurate. Answer with scope — one assertion per bug — not with principle.
- Dismissing the previous consultant. The team hears a verdict on their last two years, and it guarantees the next consultant dismisses you.
- Flatly denying that Databricks now handles something. Half of these claims are true and more become true each quarter; being wrong about one costs you the other five arguments.
- Answering the rewrite objection with reassurance instead of a test. Reassurance expires the first time a refactor breaks a report, and nothing you propose afterwards gets adopted.
- verifiedPredictive optimization runs OPTIMIZE, VACUUM and ANALYZE automatically on Unity Catalog managed tables. It is on by default for accounts created on or after 11.11.2024.Stand 29.07.2026
- verifiedLiquid clustering replaces partitioning and ZORDER. Databricks recommends it over Z-ordering for all new tables, and CLUSTER BY AUTO lets predictive optimization choose the keys.Stand 29.07.2026
Straight to the documentation
The generic platform material lives in the official docs and is not restated here. These are the pages worth having open while you work through this route.
- Introduction to the Databricks well-architected framework — hand this to the team as reading before workshop 1, so the workshop itself can be hands rather than exposition
- Databricks Git folders — pairing and review only transfer anything if the daily loop is branch → PR → review; this is how you get there from notebooks
- Predictive optimization for Unity Catalog managed tables — read before contradicting 'Databricks does that automatically' — check whether it is on for this account first
- Use liquid clustering for tables — where the partitioning kata and the previous-consultant objection both land
- What are Declarative Automation Bundles? — workshop 6's second half, and the rename that confuses anyone who learned this as Asset Bundles last year