Why big tool lists hurt

If you've connected an agent to a CRM, a billing system, a ticket queue and a few MCP servers, you'll know the problem. Every request sends every tool's full JSON schema. Google's own guide warns that a few dozen declarations can use up tens of thousands of tokens. It also says the model gets worse at choosing the right tool as the list grows. You pay for that extra weight on every call, and you wait for it too.

On 19 November 2026 Google released a preview feature for the Gemini API called tool retrieval. Lucia Loher, a Gemini API product manager, and Nora Kassner, a Google DeepMind research scientist, wrote the announcement. The idea is that an agent keeps its whole tool library on hand, but the model only sees short summaries until it decides it needs a tool. We've read through the documentation to see how it works and where builders are likely to run into trouble.

Before you start

Three things need to be true first. The feature is in preview, it needs the v1beta API version, and it only works through the Interactions API. If your agent calls generateContent, you'll need to move it over before you can use retrieval. It runs on Gemini 3.6 and later: gemini-4-carbon, gemini-4-neon, gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash. Older models return an error saying deferred loading isn't supported.

Two ways to run the search

Server-side is the default. You add {"type": "tool_search"} to your tools array and set defer_loading: true on each function you want to hold back. The model sees each deferred tool's name and a short description. When it wants one, Gemini searches the library, loads the full schema and produces the function call, all within one API request. Your app only runs the final function_call step, and you don't have to build or host a search index.

Client-side puts the search in your hands. You declare a tool_search tool with execution: "client", along with a name, a description and optional parameters. Google's example takes a query and a namespace. When the model calls it, your code runs its own lookup and returns the matching function declarations in function_result.result. The model can then call the tool it found. Each search adds a round trip. In return you can filter tools per user, group them by namespace, or add new ones to your registry partway through a conversation.

You can't use both modes in the same request, and defer_loading only works with server-side search. Google suggests server-side when latency matters and client-side when your registry changes often. If you have fewer than about 20 tools that rarely change, the guide says you're better off loading them all up front.

Fields worth knowing

  • short_description is a brief summary the model reads while searching. It's only allowed on deferred functions, and it helps when your main description runs long. If you leave it out, the model sees the first sentence of description instead.
  • Setting defer_loading on an mcp_server entry holds back every tool that remote server offers. It still needs a server-side tool_search in the same request. When the model calls one of those tools, the API runs it remotely and returns mcp_server_tool_call and mcp_server_tool_result steps.
  • You can only have one tool_search entry per request.

The response's interaction.steps list shows what happened in order: tool_search_call, then tool_search_result, then function_call. Google's sample response shows status: "requires_action", but the guide says to look in steps for a function_call instead of relying on the status field. The steps are also useful for debugging, because they show what the model searched for and what came back.

Multi-turn traps

Most of the errors builders will hit come from how conversation history is handled.

The API doesn't keep tool declarations from one turn to the next. Every follow-up request has to resend the full tools array, including the one that carries your function_result. That means tool_search, the deferred tools and any tools that are already loaded. If you leave tool_search out of a later turn, you'll get "tool_search tool must be specified when using defer_loading."

If you use previous_interaction_id, Google stores the earlier steps for you, search steps included. Stateless runs with store=false take more care:

  • Add every step the model generates to your history exactly as it came back.
  • Keep tool_search_call and tool_search_result in their original order, before the function_call.
  • Keep the signature field on search, thought and function call steps.

If you drop a tool_search_call or strip a signature, you'll get a 400 error when you send your function_result. If you drop a tool_search_result, you get no error at all, and the loaded schemas silently disappear from context. That second case is harder to spot.

Two more points to keep in mind. Once a tool is loaded, it stays loaded for the rest of the conversation, and there's no way to remove it. Also, don't defer any tool that your system instructions or tool outputs mention by name. The model may try to call it without searching first, which causes the "Server side tool call context must be circulated" error.

What it costs

Retrieval itself is free. Everything bills at the model's usual input, cached input and output rates. What changes is which tokens you pay for:

  • Deferred tools only add their name and short description to total_input_tokens. You don't pay for their parameter schemas until they're loaded.
  • Turning on tool_search adds a small, fixed number of input tokens.
  • Each server-side search runs an extra model call. Those tokens show up under total_tool_use_tokens and bill at the input rate.
  • Once a schema is loaded, you pay for it as input on every remaining turn.

Loaded schemas are added inline, so the start of your prompt doesn't change and implicit caching keeps working across turns. There's one catch. Cutting a medium-sized prompt down can drop it below the model's minimum size for caching, and that can eat into the savings.

Google hasn't published any savings figures. It says the gain depends on how many schemas go unused, and it recommends testing against your own traffic. For scale, gemini-4-hydrogen costs US$1.50 per million input tokens on the standard tier. As a hypothetical, cutting 20,000 schema tokens from each request would save about three US cents per call. Prices are also changing soon. Standard input for Carbon and Argon rises from US$2 to US$4 per million on 1 December 2026, and the 3.6 and 3.8 Flash rates go up on 1 January 2027.

Retrieval pays off when you have a large library and each request only uses a few tools. With a small library, or a long conversation that searches on every turn, loading everything up front can work out cheaper.

How we'd approach it

  • Keep the few tools used on almost every request loaded, and defer the rest.
  • Start related tool names with the service name, such as billing_get_customer, and start each short_description with the service too. Google says this helps the model choose correctly.
  • Always include full parameters for deferred tools. Without them, the model can't call a tool after it finds it.
  • Expect the model to run several searches in one turn. Plan for searches that find nothing, too: those come back as a plain text reply, so your code needs to check for it.
  • Log interaction.steps and compare token counts with retrieval on and off before you roll it out.

Because this is a preview, field names and behaviour could still change. We'd keep it behind a feature flag in production for now.

Sources

Build with people working on the same things

Join the Auckland Society for AI: NZ$599 a year with a ForgeBox and the members' projects area. Our Auckland meetups are open to everyone.

Become a member