Testing a new AI agent skill: hand-written tests, shadow mode, then canary rollout
A group member asked how to test a new agent skill before trusting it on real tasks. A three-layer approach won the thread: a small hand-written test set for the happy path, a shadow mode where the skill runs on real inputs but only logs what it would do, and a canary percentage of live traffic before full rollout. Shadow runs catch the edge cases a test set never imagines.