Cut Gemini API Costs by Routing Tasks to the Right Model

If you build a coding agent, every loop iteration, tool call and retry sends tokens to Gemini. The bill is the sum of all those steps, not the one prompt you see. In Google's own test, a tiny classifier called Jev sat in front of Gemini and decided which model to call. Jev is a small classifier API from TypeSafe that decides which Gemini model to call.

It solved all 50 coding tasks with a lower bill than using the strongest model on everything. You can build this pattern yourself — here is how.

Google ran these tests on Gemini 3.8 Flash and Flash-Lite in November 2026. Your results may vary with a different version.

The bill problem is cost per task, not token price

According to Google's Nov 3 research post, per-token prices keep falling but coding agent bills keep climbing because each task spends tokens across many loop iterations, tool calls, retries and thinking passes.

Every retry after an error sends more tokens. The bill is the sum of all those steps, not the one prompt you see. That sum divided by tasks that work is your cost per successful task. A single cheap prompt is not the cost; the loop around it is.

Our earlier post on Particle-1 Token Efficiency: Cheaper Coding or Just Fewer Tokens? makes the same point: fewer tokens per call is not the same as a lower bill. Routing only helps if the classifier sends each task to the right model.

Set thinkingLevel LOW on Flash before you build any router

Google's strongest no-router baseline was Gemini 3.8 Flash on low thinking, which solved all 50 test tasks and costs far less than the default high thinking, per the same post.

This is rule one from Google's post, and it is the cheapest change you can make today. No router, no classifier, no new code. One parameter on your existing Gemini calls:

const response = await model.generateContent({
  contents: [{ role: "user", parts: [{ text: prompt }] }],
  thinkingLevel: "LOW",
});

Flash-Lite alone missed 6 of 25 hard tasks in Google's test. If you are a student with a small side project, start here. Only add routing when the low-thinking baseline stops being enough. Once you lower thinking, the next instinct is to cap the output length to save more. Google tested that too — and it backfired.

Never cap maxOutputTokens hoping to save money

Lowering maxOutputTokens from 4096 to 300 did not reduce thinking tokens on easy tasks and broke 7 of 25 hard ones, per Google's measurements.

Google found the 300-token cap spent almost identical thinking tokens on easy tasks. The cap is a ceiling, not a thinking-budget dial. On hard tasks, 6 ran out of room mid-reasoning and 1 squeezed its reasoning and got the answer wrong.

The temptation is intuitive: cap the output, save tokens. Google's data shows the model still thinks as much as it needs, and the cap only cuts off the answer. Set the token limit to what a correct answer needs, and control thinking cost with the thinkingLevel parameter instead.

Route short tasks to Flash-Lite, keep long tasks on one model

Send short, self-contained requests to Flash-Lite. Keep long, context-heavy turns on one model so context caching keeps giving you a 75–90% discount, per Google's post. (Context caching stores prior conversation turns so you are not re-sent them; it is a 75–90% discount on repeated tokens.)

This is how you apply the routing pattern to your own agent.

The answer tells the router which model to call. A broad question like "is this simple or complex?" failed because it sent nearly every task to the cheap model, even hard ones. A specific question about whether the task needs more than one sequential step worked.

Google's post uses this exact question: "Does answering this require performing more than one sequential calculation, simulation step or logical deduction, rather than a single direct lookup, format check or one-step classification?" The rule: decompose broad questions into atomic checks.

Context caching has a catch. A cache belongs to one model, so switching Flash-Lite and Flash per turn means two caches or losing the discount. That is why short, self-contained tasks are the right routing targets.

1. Log your task mix

   Write down the kinds of tasks your agent handles: a code-generation call, a code review, a refactor.

2. Ask one yes/no question per factor

   One question for difficulty, one for context length. Do not combine them into a single broad question.

3. Route short tasks to Flash-Lite

   Send self-contained requests there. Escalate only the flagged hard ones to Flash on low thinking.

Test security prompts before you route

Google notes one more risk in its test run, also from the post: the Pro configuration declined two short code-security review prompts that Flash answered. Test security prompts on the configuration you plan to use before you route them away. For example, if you route a prompt that asks "is this code vulnerable to SQL injection?" to Flash-Lite, test that Flash-Lite actually answers it. In Google's test, the Pro model refused two such prompts while Flash answered them.

A simple heuristic classifier is not enough

A heuristic is a rule you write by hand, like "if the prompt contains the word sequence, use the strong model." A regex is the pattern that rule matches. The problem is that it can only match patterns you already thought of, so it cannot see a new kind of hard task.

A rule you write by hand can only catch the patterns you already thought of, so it misses new kinds of hard tasks. In Google's test, a hand-written rule failed to catch 10 of 34 unseen tasks (29.4%). A classifier like Jev caught all of them.

The daily.dev writeup independently confirms these numbers.

Measure what your agent actually spends

First, print token usage from your API response and add it up. For example, console.log(response.usageMetadata); shows the token count. For a fuller picture, create a Cloud Logging sink that captures latency and token counts, then query the logs in BigQuery. That tells you your real spend per task, so you know whether routing is working, per Google's guidance.

You cannot optimize what you cannot see. That data tells you your actual cost per successful task, before and after you add routing.

Your move:

1. Find one existing Gemini call in your code.
2. Add thinkingLevel: "LOW".
3. Run it on your hardest task.
4. Note the token count in the API response.
5. Compare with the default setting.

FAQ

If I have only one script, does routing help? Routing only pays off when you have a mix of easy and hard tasks that repeats many times. For a single script, you would pay the classifier call on top of the Gemini call, so it is not worth it. Set thinkingLevel LOW instead — that works on a single script.

What is a classifier? It is not a language model generating text; it just makes a decision. It reads the prompt and answers a yes/no question, like "does this need more than one step?" Your code then uses that answer to choose which model to call.

Does routing work for small projects? Yes. If you have a handful of tasks, you can test manually first. The cost savings come from not paying for high thinking on easy tasks.

What if I don't use Google Cloud? The examples here use Gemini, so the parameter names are Gemini-specific. If you use OpenAI or Anthropic, the same idea applies: route simple tasks to their cheapest model.

Why not just cap maxOutputTokens? Google's test showed the cap doesn't reduce thinking on easy tasks and breaks hard ones. The cap is a ceiling — the model still thinks as much as it needs, so the cap only cuts off the answer before it finishes. Use thinkingLevel instead.