EngagementSynthesis · phases · handover
23The engagement, end to end
Everything else in this handbook is a subject. This page is the sequence. It answers the two questions a client actually asks — what happens in what order, and how will we know it worked — and it links rather than repeats, because the detail is one click away and does not belong twice.
Stand 29.07.2026
The arc
Five phases. The dates are illustrative for a roughly one-quarter engagement; the ordering is not, because each phase produces the input the next one needs.
| Phase | Weeks | Produces | Routes |
|---|---|---|---|
| Assess | 1–2 | Evidence about the platform as it is, not opinion about it. A written intake for what the warehouse must answer. | Audit & health check · Requirements |
| Decide | 2–4 | The expensive-to-reverse choices, written down: layer contract, model, naming, environment topology, ingestion pattern per source, semantic layer ownership. | Architecture · Data model · Ingestion · BI & semantics |
| Build | 3–10 | Working pipelines, plus the standards and CI that keep the tenth one as good as the first. | Standards & CI/CD · Storage & cost · Unity Catalog · Access & ABAC |
| Prove | 5–12 | Tests that fail for the right reasons, monitoring that pages a person, and observability that catches wrong numbers behind green jobs. | The four testing routes · Observability · Monitoring |
| Hand over | 10–13 | Named owners, runbooks written by the people who get woken, a survived game day, and the consultant off the rota. | Operations · Enablement · Written concepts |
#The audit is the opening move
What
Week one is a read-only sweep of the account using
system.* tables: where the money goes, what compute exists, which jobs fail, what is granted to whom, what is governed and what is not.Nothing is installed and nothing is changed. The output is a short document of findings with the query beside each one, so any of it can be re-run by the client the day after you leave.
Why here
It converts opinion into evidence before anyone proposes changes, which changes the politics of the whole engagement. "Your clusters are oversized" is an argument with the person who sized them. "Here is the query, here is the number, here is the same query you can run next month" is not.
It also protects you. The findings are dated and reproducible, so improvements later in the engagement have a baseline that nobody has to take on trust.
Doing it
- Enable the system schemas first — they are not retroactive, so every day you wait is a day of history you will not have.
- Run the sweep read-only and say so explicitly; it lowers the barrier to being given access at all.
- Report findings with the query attached, never as bare assertions.
- Separate 'no evidence' from 'evidence of none' in every finding — they are different sentences and only one is defensible.
Defaults
- Read-only, evidence-first, in week one.
- Every finding carries its query and its date.
- Baseline at the start, re-run at the end, show the delta.
- Findings are ranked by cost or risk, not by how interesting they were to find.
Gotchas
- Leading with the audit findings in a kickoff before you have any relationship. It reads as an indictment of the incumbent team, who then spend the engagement defending rather than learning. Share it with the team first, then jointly with management.
- Quoting numbers from a schema enabled last week as though they covered last quarter. System tables are not retroactive, and the gap always hides spend rather than revealing it.
- Turning the audit into the engagement. It is a two-week input, not a deliverable to polish — the value is in what changes afterwards.
The decisions that are expensive to reverse
Most of what a warehouse team does is reversible in an afternoon. This list is not. Each of these gets written down, with the reasoning, in the first month.
#The eight that need writing down
WhatDecide once, in phase two, in writing
| Decision | Cost of getting it wrong | Route |
|---|---|---|
| Environment topology — catalog per environment | Retrofit means moving every table, reissuing every grant, re-pointing every report. No partial rollout. | Architecture · Unity Catalog |
| Layer contract — what may enter and leave each layer | Without it, gold becomes silver with a prefix and the first cross-source question cannot be answered. | Architecture |
| Naming and identifier standard | Renaming after BI connects is a fortnight of coordination, not one statement. | Architecture |
| Modelling approach — Kimball, Vault, or wide | Changes the shape of every table and every load. Effectively a rebuild. | Data model |
| Key strategy — hash surrogate keys and their normalisation | A collision or an un-normalised input corrupts joins silently, and the fix is a full reload. | Data model |
| Historisation — which dimensions are SCD2 | History not captured cannot be recovered later. The data is simply gone. | Data model |
| Semantic layer ownership — metric views, BI model, or both | 'Both' means two definitions of revenue and a standing argument about which is right. | BI & semantics |
| Tag taxonomy for cost attribution | Tags cannot be backfilled — untagged spend is unattributable forever. | Storage & cost |
Why here
Every one of these is cheap in week three and expensive in month six, and none of them is technically hard. They get missed because they are not urgent — no ticket says "decide the historisation policy", so the first pipeline quietly decides it by accident.
Two of them are worse than merely expensive, because they destroy information rather than costing effort: history you did not capture cannot be reconstructed, and tags cannot be backfilled. Everything else on the list is money and time. Those two are permanent.
Doing it
- Put all eight on one page with the decision, the date, and one paragraph of reasoning. Not a slide deck — a file in the repository.
- Decide the two irreversible ones first: historisation, and the tag taxonomy.
- Build one vertical slice against the decisions before writing them down as final. A decision no code has touched is a guess.
- Record what you rejected and why. The question comes back in month four, reliably.
Defaults
- Eight decisions, one page, in the repository, with dates and reasoning.
- Irreversible-information decisions first: historisation and tagging.
- Prove each decision with one vertical slice before declaring it.
- Rejected options are recorded, not discarded.
Gotchas
- Deferring historisation until 'we know the requirements'. Meanwhile the loads overwrite, and when the requirement arrives the history it needs was never written. There is no fix — only a restart of the clock.
- Deciding the semantic layer late. By then Power BI has its own model, the warehouse has metric views, the two disagree by a rounding rule, and every number in every meeting is now contestable.
- Writing the decisions in a consultant's deck rather than the client's repository. They are not findable at the moment someone needs them, so they get re-decided differently.
- Treating the eight as sequential. They interact — the key strategy depends on the modelling approach, which depends on the layer contract. Do them in one workshop block, not one per sprint.
What good looks like when you leave
The engagement is not finished when the pipelines run. It is finished when the team keeps them running without you, and that is checkable rather than felt.
#The handover test
What
Nine checks, run in the final fortnight. Every one is observable — no item on this list is a judgement call, which is deliberate, because "the team feels confident" is not a handover criterion.
- Every production object is owned by a group, not a person. Verify it with a query, not a promise.
- The consultant is not on the alert rota, and has not been for two weeks.
- A game day has been run and survived — a real pipeline broken deliberately, diagnosed and fixed by the team.
- Every pipeline has a runbook, written by whoever gets woken, and each one answers the re-run question: safe to re-run blind, or needs intervention first.
- The last three incidents produced runbook updates in the same week they happened.
- CI fails for the right reasons — someone can point at a build that correctly blocked a merge in the last month.
- A team member has added a new source end to end — bronze through gold, tested, granted, monitored — without pairing.
- The eight irreversible decisions are in the client's repository, and someone who was not in the room can explain them.
- The currency register has been re-verified by the team at least once, so the habit transferred and not just the document.
Why here
The failure this prevents is the standard consulting failure, and it is rarely technical: the platform works, the team can operate it while things go well, and the first genuinely novel incident after the consultant leaves takes four days instead of four hours.
Checks 3, 5 and 7 are the load-bearing ones. They are the only three that cannot be satisfied by documentation — each requires the team to have already done the thing unaided. Everything else on the list can be faked by a diligent writer.
Doing it
- Run the nine checks in week 10, not week 13. Failures need time to become passes.
- Take yourself off the rota deliberately and early enough that a real incident lands while you are still reachable but not responsible.
- Schedule the game day; it never happens if it is left to a quiet week.
- Write nothing in the final fortnight that the team could write instead.
Defaults
- Nine observable checks, shared in week one, verified in week ten.
- Group ownership everywhere in production, proven by query.
- Consultant off the rota with two weeks of runway remaining.
- The final fortnight is the team writing and you reviewing — never the reverse.
Gotchas
- A handover that is a document rather than a demonstration. The document is read once and never again; the pipeline someone built unaided is remembered. If you only have time for one, build the second.
- Staying on the alert rota 'just in case' through the final week. It is kind, and it guarantees the team never experiences an incident as theirs — so the first one they own happens with no one to ask.
- Named owners who were never asked. Someone discovers they own a pipeline when it pages them at 03:00, and the runbook they never wrote is the one they now need.
- Skipping the game day because the quarter is busy. The quarter is always busy. Its absence is only discovered during the first real incident, which is the worst possible time to learn that the runbook was never tested.
Straight to the documentation
The generic platform material lives in the official docs and is not restated here. These are the pages worth having open while you work through this route.
- Databricks Well-Architected Lakehouse — the platform's own architecture pillars — useful as a checklist to compare your decisions against
- Unity Catalog best practices — the topology and ownership guidance behind the handover checks
- Databricks release notes — hand this to the team as the input to their quarterly re-verification