Cut LLM routing waste with Vercel AI Gateway provider sorting

Vercel added provider sorting to AI Gateway. Set providerOptions.gateway.sort to cost, ttft, or tps and the gateway ranks available providers behind a model by the selected metric before it falls back to the next provider.

Use it when a model has multiple providers and your workflow has a clear priority: lowest listed input cost for batch jobs, lower time to first token for interactive chat, or higher tokens per second for long outputs. The routing metadata can show which providers were considered, attempted, deprioritized, or selected.

This is useful, but it is not a magic optimizer. Provider sorting works best after you define the workload, check data-retention constraints such as ZDR, and verify model/provider behavior with your own prompts and token mix.