Cut LLM routing waste with Vercel AI Gateway provider sorting
Vercel AI Gateway can now rank providers by cost, time to first token, or throughput before fallback, so agent apps can optimize routing without hard-coding provider order.
- LinkLoot access
- Free
- Provider costs
- Unknown
What you get from it
Practical testing not documented.
LinkLoot assessment · not a user rating.
Vercel added provider sorting to AI Gateway. Set providerOptions.gateway.sort to cost, ttft, or tps and the gateway ranks available providers behind a model by the selected metric before it falls back to the next provider.
Use it when a model has multiple providers and your workflow has a clear priority: lowest listed input cost for batch jobs, lower time to first token for interactive chat, or higher tokens per second for long outputs. The routing metadata can show which providers were considered, attempted, deprioritized, or selected.
This is useful, but it is not a magic optimizer. Provider sorting works best after you define the workload, check data-retention constraints such as ZDR, and verify model/provider behavior with your own prompts and token mix.
Discussion
Share practical experience, questions, or warnings with the community.
Sign in to join the discussion and vote on comments.
Sign in