Incident Lab · SEV2 · Airflow
Revenue doubled overnight
Finance: yesterday's revenue on the dashboard is twice what the payment provider settled.
[2026-03-02 02:14:07] INFO - Task load_orders failed (timeout), retrying (1/3) ... SUCCESS
The nightly load orders task timed out after writing half its batch, and Airflow retried it. The retry loaded the same orders again, so raw orders now holds some orders twice (same order id , a later loaded at ). daily revenue sums everything it finds in raw orders , so every duplicated order is counted twice. Investigate (run these on their own first): Fix the model: rewrite daily revenue so each order id counts once (keep its latest load). Keep the columns order date and revenue (rounded to 2 decimals).