We only learned our AI agent broke when a customer emailed us
A team shares how they stopped shipping agent regressions to real customers: adversarial testing before release, reading the full trace instead of just the final answer, and the extra step of running new versions in shadow mode so the diff between old and new behaviour becomes the eval set.