Chess.com's analytics ran on hand-managed SQL pipelines. Backfills were manual, testing was thin, and every change went through an engineer. We rebuilt the stack around dbt: a medallion architecture on BigQuery, custom macros for incremental processing, isolated CI/CD, and a governed QA environment. Today 30 to 40 people across experimentation, analytics reporting, marketing and CRM build on it every week.
The mandate: one place where models are tested, documented, incremental by default and safe to change. That platform now feeds experiment readouts, executive reporting, marketing audiences and CRM. When someone at Chess.com asks whether a number is right, this is the system that answers.
We wrote the architecture proposal that made dbt the foundation of analytical engineering at Chess.com, then implemented it. Model lineage, data tests and standardized materializations replaced duplicated SQL and manually tracked dependencies. Custom Jinja macros handle incremental logic once, so every model gets it right instead of reinventing it.
Bronze, silver and gold layers with a unified event contract underneath. Daily and realtime processing resolve the same definitions, so intraday numbers and historical reports agree with each other. Terraform governs the warehouse itself: datasets, permissions and cost controls live in code.
Core models are fully incremental and idempotent. A normal daily run finishes in roughly two minutes, and history is never reprocessed without a reason. Analysts ship model changes on their own, without waiting on an engineer to wire up a DAG.
A dedicated CI/CD project runs the test suite against production-shaped fixtures before anything merges. A governed QA environment separates personal development, staging backfills, release QA and production, with clear pass and fail outcomes and named reviewer sign-off on releases.
Centralized backfill DAGs in Airflow, permissioned and observable, running production backfills on dedicated QA reservations so they never fight the daily workload. Reprocessing a year of history went from a risky weekend job to a normal ticket.