Incident Lab · SEV2 · Airflow
Midnight renewals missing from every daily load
Billing: every subscription renewal is missing from the warehouse. Card payments during the day look fine.
[2026-10-13 01:00:12] INFO - load_payments: rendered WHERE created_at > '2026-10-12 00:00:00' AND created_at < '2026-10-13 00:00:00'
The load payments task runs once a day and loads the rows created between Airflow's data interval start and data interval end . dag runs holds one row per run with its window. Each window ends exactly where the next one starts . The query uses on the start and < on the end. A row created at exactly 00:00:00 falls outside both neighboring windows. Most payments don't land on the exact second, but the billing system charges every renewal at midnight sharp , so all of them disappear. Investigate: Fix the window condition in loaded payments so every payment is loaded by exactly one run. A row on a boundary belongs to the run that starts there. Keep the columns run date , payment id , created at , amount , ordered by payment id .