Broken data used to be found by whoever hit it first, usually an analyst in the middle of a report. We turned quality into infrastructure at Chess.com: healed and cleaned historical data, an automated watchdog that investigates anomalies on its own, validation wired into every release, and one governed registry of event definitions. The watchdog caught real production issues in its first week of operation.
Most teams treat data quality as cleanup work that happens after something breaks. At Chess.com we made it part of the platform itself. Detection runs automatically, investigations produce evidence instead of guesses, releases carry sign-off, and definitions live in one governed place instead of in tribal knowledge.
Historical data was healed rather than discarded. Contaminated sessions were untangled, sessionless events were absorbed into valid nearby sessions, and passive server noise was suppressed from active user counts. Raw data stays untouched, and audit columns explain every correction that was applied.
An event-driven watchdog listens for completed pipeline runs, applies deterministic anomaly detection, and then runs bounded AI-assisted investigation on what it finds. It produces a daily digest with lineage, impact and suggested ownership, suppresses known noise, and operates under strict query cost limits. It caught multiple real production issues in its first week.
A governed Data QA environment separates personal development, staging backfills, release QA and production. dbt test runs are linked to releases, outcomes are recorded as pass, fail, unsupported or insufficient traffic, and a named reviewer signs off. Quality stopped being a matter of trust and became a matter of record.
A governed event registry separates raw instrumentation from analytical meaning. Canonical concepts survive platform changes and event redesigns, rules are effective-dated, and both daily and realtime processing resolve the same definitions. The registry was validated against a 1.6-billion-row production partition with zero metric disagreements, at about 0.1% additional build cost.
A cleanup audit identified roughly 20.8 TiB of obsolete storage while explicitly protecting every active dbt, Airflow, dashboard, ML and analyst dependency. Cost went down. Nothing broke.