How to Make Sure Your dbt Semantic Layer Metrics Are Actually Right
Use AI to check a dbt MetricFlow metric against independent SQL, then build a canary that fails when a reconciled number moves.
Who it's for: Analytics and data engineers who define metrics in dbt, or let AI write them.
The problem: A metric can pass dbt build and mf validate-configs with zero errors and still be wrong. A plain average of daily view duration came out about 38% low on real YouTube data in BigQuery.
Takeaways:
- Reconcile each metric against hand-written SQL first.
- Pin only months whose data has stopped changing.
- A canary catches a silent filter that build and validation both pass.