How Switchboard works

You tell us which LLM you use today, roughly what you spend, and what you use it for. We tell you whether a cheaper open-weight model would do the job just as well — and what switching would save you. This page explains how we get to that answer, in plain terms, so you can defend the recommendation to your team. Exact formulas and thresholds follow in the fine print below.

The one rule: performance first, then price

A model that is 10× cheaper but fails at your actual work isn’t a saving — it’s a cost waiting to happen. So we never lead with the dollar figure. We first establish how a candidate performs on your kind of work, and only then compare prices. Every number is computed from public, auditable sources, and every figure on the results page can be traced back to its source.

What you tell us

Your current model — e.g. Claude Opus 4.8 or GPT-5.5.
Your usage — whichever is easiest to estimate: requests per day, tokens per month, or just your monthly bill. We convert them all to the same monthly volume.
What you use it for — a category like coding, general chat, or customer support. You can also describe your work in a sentence and we’ll build the mix for you.
How long your requests are — short, medium, or long.
What matters more to you — closest quality, balanced, or maximum savings. This sets how conservative the final recommendation is: choose closest quality and we demand a bigger performance margin before we’ll suggest switching.

Where the data comes from

Pricing (OpenRouter) — what each model actually charges per million input and output tokens.
Real-world quality (LMArena) — ratings built from millions of blind, head-to-head human votes, broken down by task type. This is people judging real outputs, not lab tests.
Benchmark scores (Artificial Analysis) — structured tests of coding, multi-step agent work, and reasoning, which fill in where head-to-head data is thin.

The results page shows the date of the data behind your report, so you always know how fresh the comparison is.

Step 1 — Who makes the shortlist

We only consider open-weight models as candidates, because that’s the decision this tool exists to inform. We also quietly remove models that couldn’t handle your work in practice: models that can’t fit your typical request size, models without the tool-calling support your tasks depend on, and models without reliable public pricing. If you like, a closed model can be added to the chart as a sanity check.

Step 2 — Scoring performance on your work

We express performance as one intuitive number: the expected win-rate — “this candidate matches or beats your current model X% of the time on your kind of work.”

Behind that number: we compare each candidate to your current model on every task category in your mix, using the head-to-head human ratings, then weight the categories by how much they matter to you. Ratings backed by only a handful of votes are discounted rather than taken at face value. Where benchmark data adds real signal — coding, agent-style workflows, math — it’s blended in at a fixed, disclosed share for that category (up to about half for agent-heavy work, less elsewhere). When the data is thin, we say so with a confidence flag and a plain-language caveat instead of silently guessing.

Step 3 — Projecting your costs

Your input volume stays the same — you’d send the same prompts to any model.
Your output volume is adjusted per candidate, because some models are wordier than others. We measure each model’s typical verbosity relative to your current one and scale accordingly, capped so no outlier distorts the estimate.
Monthly cost is then just tokens × price, and savings are shown in $ / month, $ / year, and %.

If you gave us a monthly bill rather than token counts, we work backwards from your current model’s pricing — so your current cost always reconstructs to exactly the number you told us. No phantom math.

Step 4 — The three recommendations

We keep only candidates that are actually cheaper than your current model, plot them on performance versus cost, and drop any that another candidate beats on both. From what’s left, we surface three picks for three kinds of decisions:

Closest match — the safest bet: the strongest performer among the options we have solid data on. Pick this if quality is non-negotiable.
Value pick — the recommended sweet spot: the best overall trade-off between savings and performance, tilted by the priority you chose in setup.
Cost-efficient — the aggressive play: weighted heavily toward savings, but still required to clear a minimum performance floor. Nothing below that floor is ever recommended, however cheap.

Step 5 — The verdict: stay, pilot, or switch

We don’t leave you to interpret a chart. The report ends in one word, decided by clear rules:

Stay — the savings are too small to justify a migration (under $5,000/year and under 50% of your bill), or no alternative clears the minimum performance bar.
Pilot — the savings are real, but something isn’t proven yet — the performance margin is modest, or the absolute dollars are small. Try a candidate on a real workload before committing.
Switch — a candidate clears both tests at once: at least $5,000/year in savings and performance solidly on par with what you have. The math says go.

The performance bar moves with your stated priority: quality-focused users need to see a stronger candidate before we’ll say switch.

What we deliberately leave out

Self-hosting never affects the verdict. For models you could run on your own hardware, we show a one-line break-even estimate as a side note only.
Switching costs aren’t modeled. The savings shown are gross: they don’t include engineering time to migrate, re-testing your prompts, or vendor review. For most teams these are one-off costs against a recurring saving, but budget for them.
No hidden thumbs on the scale. There are no tuned knobs pushing you toward switching, and edge cases aren’t hidden — missing data and low-confidence calls are flagged right on the results page and in the PDF.

The fine print — exact formulas and thresholds

Every headline number traces to a source row. No bounds-juggling, no voiding thresholds, no modifiers — point-only arena Elo blended with Artificial Analysis indices, applied consistently. The full blend weights, per-category Elo, AA share, and data dates are visible inside the calculator under “How we scored this” and reproduced in the decision PDF.

Performance axis — expected win-rate vs incumbent

Text arena ratings come from LMArena's style-controlled leaderboard (length/markdown effects regressed out — the raw ladder flatters style-tuned models); the webdev arena has no style-controlled variant and is used raw. Per arena category in the use-case blend: ΔElo = candidate.rating − incumbent.rating, shrunk toward 0 by vote support (ΔElo′ = ΔElo × votes/(votes+3000), using the thinner side's count — an unconverged rating on few votes is not trusted at face value), then win_rate = 1/(1+10^(−ΔElo′/400)). Weighted average across categories, renormalised over the categories where both models have data. Every use case blends in an Artificial Analysis index at a stated share (specialised indices for agentic / coding / math / web, a general intelligence hedge elsewhere; the engine falls back to the intelligence index when a specialised one is missing) — arena preference and capability evals measure different things, so neither is trusted alone.

Cost axis

monthly_cost = in_tok/1M · input_price + out_tok/1M · output_price. Savings = incumbent − candidate (absolute and percent). Per-request token volumes are derived from your use-case blend × your request-size preset.

Cost per task — verbosity-adjusted

Models differ several-fold in tokens consumed per task (reasoning tokens especially), so per-token prices alone mislead. Each candidate's OUTPUT volume is scaled by a verbosity ratio: implied tokens/task (Artificial Analysis cost-per-task ÷ a blended price at the index's ~2.3:1 input:output mix) for the candidate ÷ the same for your current model, clamped to 0.5–2.5×. Input volume is never scaled — your prompts don't change with the model. When either side lacks AA cost data the ratio is 1 (volumes assumed equal) and the card says so. This is a free-tier estimate; AA's measured token counts would replace it.

Candidate set

Open-weight only — the decision this tool answers is closed → open. An optional cheaper-closed sanity row is available on the results chart.

Speed — an annotation, never a filter

Estimated response time = time-to-first-answer-token + output tokens ÷ tokens/second, both Artificial Analysis medians across serving providers; the candidate's generation term uses its verbosity-adjusted output volume. TTFAT includes thinking time for reasoning models regardless of whether the provider streams it. Shown per card with a faster/on-par/slower band; for latency-sensitive use-case mixes (≥50% weighted) the verdict may cite the champion's speed. Speed never filters candidates, never re-ranks tiers, never flips the verdict — ~40% of models lack the data, and gating on it would select by data availability, not merit. Open-weight hosting speed varies by provider; treat figures as directional.

Tiers — three named picks from the Pareto frontier

Closest match: highest win-rate (flagged with a caveat when below the ~50% on-par bar — not a gate). Value pick: savings_pct^α × max(0, win_rate − 0.30), where α follows the Cost-vs-quality preference (Maximize savings α=1, Balanced α=0.6, Closest quality α=0.35 — smaller α compresses the savings axis so win-rate decides more of the pick). Cost-efficient: a cost-vs-quality blend `cost_weight · savings_pct + perf_weight · win_rate`, gated by a ≥35% win-rate floor, with weights from the same preference: Maximize savings (0.70/0.30, the default for legacy callers), Balanced (0.50/0.50), or Closest quality (0.30/0.70). Each card is a non-dominated point.

Verdict — stay / pilot / switch

Each tier pick is evaluated standalone on its own figures — absolute ($5k/yr) or proportional (50%) savings bar plus win-rate thresholds — and the best single outcome wins; the verdict names that champion model. The stay/pilot/switch win-rate thresholds shift with the Cost-vs-quality preference: Maximize savings (stay < 35%, pilot 35–45%, switch ≥ 45%), Balanced (stay < 40%, pilot 40–50%, switch ≥ 50%), Closest quality (stay < 45%, pilot 45–55%, switch ≥ 55%). Switch additionally requires the absolute savings bar cleared.

Self-host callout — secondary, not an axis

For open-weight picks with selfhostable flag and ram_q4 ≤ 128 GB: break-even months = hardware cost / monthly savings vs the incumbent. A note, not a recommendation.

Confidence flags

Driven by arena coverage fraction and per-category vote counts. High coverage falls to medium/low when any counted category has thin (<300) head-to-head data or the blend is largely eval-based.