Every number, traceable back to the row it came from.

Getting data out of operational systems without hurting them and without silently losing rows. Change data capture where the source supports it, batch extraction where it does not, idempotent loads so a rerun does not double count, and a landing zone that keeps the raw payload exactly as it arrived.

The single most common defect at this layer is an extract that quietly truncates when the source changes a column type, and a pipeline that reports success anyway. We catch it by asserting the row count and the schema on arrival, not by reading the log afterwards.

Deciding what the warehouse or lakehouse is actually shaped like. Raw, staging and serving separation, partitioning and clustering that match how the tables are read rather than how they are written, surrogate keys, slowly changing dimensions where history genuinely matters, and a naming convention a new analyst can predict.

Modelling is where cost is decided. A badly partitioned fact table is a recurring monthly invoice, not a one off mistake.

A data contract is a written, enforced agreement about the schema, the semantics, the freshness and the nullability of a dataset, owned by the team that produces it. Without one, every downstream break is discovered by whoever is unlucky enough to open the dashboard on a Monday.

We build the tests that enforce the contract: uniqueness and referential checks, accepted value sets, row count and distribution shifts, freshness thresholds, and a failure that blocks the release rather than posting a warning nobody reads.

Scheduling is the easy half. The hard half is dependency ordering, backfills that do not stampede the warehouse, retries with sensible backoff, alerting that distinguishes a transient network failure from a genuine data defect, and column level lineage so an impact assessment takes minutes instead of a week.

We treat a pipeline that cannot be safely rerun from any point in its history as unfinished work.

Role based access at the table, column and row level, so that the finance team and the support team see different things from the same model. Classification of personal and special category data at the point it enters the platform, not retrospectively. Masking and pseudonymisation policies expressed in code rather than in a spreadsheet of promises.

This layer is where UK GDPR obligations become real engineering: retention that actually deletes, deletion that actually propagates, and an audit trail that survives an ICO question.

Warehouse spend attributed to the team, model and query that caused it. Slow query identification, materialisation decisions made on measured cost rather than habit, and warehouse or cluster sizing that reflects the actual concurrency profile.

Platform metrics treated the same way as application metrics: freshness, job duration, failure rate and cost per model, all on a dashboard the platform owner sees before the finance director does.