How to Make Sure Your dbt Semantic Layer Metrics Are Actually Right

Use AI to check a dbt MetricFlow metric against independent SQL, then build a canary that fails when a reconciled number moves.

Who it's for: Analytics and data engineers who define metrics in dbt, or let AI write them.

The problem: A metric can pass dbt build and mf validate-configs with zero errors and still be wrong. A plain average of daily view duration came out about 38% low on real YouTube data in BigQuery.

Takeaways:

  • Reconcile each metric against hand-written SQL first.
  • Pin only months whose data has stopped changing.
  • A canary catches a silent filter that build and validation both pass.

Resources