New models ship faster than my eval process — how do you decide when to switch?
Citing Vercel’s AI Gateway report that most LLM requests now go to models under 60 days old, a builder asks how others decide when a new model is worth switching to. The practical answer emerging from the discussion is to judge candidates on a small set of real weekly jobs, not benchmarks, and to treat speed or price gains as billing decisions rather than reasons to migrate. A timely question as model churn keeps accelerating.