Google Introduces Tool Retrieval in Gemini API

Google has launched Tool Retrieval in preview within the Gemini Interactions API, allowing developers to load tool parameter schemas on demand rather than sending the entire tool library with each request. This feature aims to reduce input tokens, improve latency, and enhance tool-selection accuracy as the number of tools increases.

Orin CodewellTools & Coding

1 min read1 source

Key takeaways

30-second read
  1. 1Developers can optimize API usage by reducing unnecessary token counts, leading to potential cost savings when using the API.
  2. 2The ability to load tool schemas on demand allows for more efficient handling of large tool libraries, improving application performance.
  3. 3This feature simplifies the integration of new tools mid-conversation, enhancing user experience and flexibility in tool usage.
  • Confirmed

    Official source: Google AI Blog

  • Reported

    No secondary reporting linked

  • Verify yourself

    Eligibility, local prices and dates for your car, state or utility

Google Introduces Tool Retrieval in Gemini API

Google has introduced Tool Retrieval in the Gemini Interactions API, currently available in preview. This new feature allows developers to load tool parameter schemas only when needed, rather than sending the entire tool library with each prompt. This approach is designed to reduce input token counts, decrease latency, and improve tool-selection accuracy, especially when an agent is connected to numerous tools.

To implement Tool Retrieval, developers need to add a tool_search tool to their requests and set defer_loading: true for the tools they wish to hold back. This means that while the model can see the names and descriptions of all deferred tools, the detailed parameter schemas are not included in the initial context. When a tool is required, the model requests it by name, prompting the API to load the full schema within the same request. This streamlines the process, allowing for efficient function calling and debugging.

Additionally, Tool Retrieval integrates well with prefix caching, ensuring that deferred schemas do not clutter the start of the prompt, thus maintaining a consistent cached prefix throughout the conversation. It also supports remote MCP servers, enabling developers to connect large servers without needing to list every tool individually.

For those who maintain their own tool registries or embedding indexes, client-side retrieval is also an option. This allows for dynamic tool exposure mid-conversation, enhancing flexibility in tool management. Importantly, Tool Retrieval does not incur additional charges; it operates under the standard billing rates for input and output tokens, making it a cost-effective solution for developers managing extensive tool libraries.

0 people like this

Share this story

Gemini APITool RetrievalGoogle AIAPI DevelopmentToken Optimization

Sources

1 official
  1. [1]
    Introducing Tool Retrieval in the Gemini API

    Google AI Blog · blog.google · published Nov 19

    Official
This brief was drafted with AI and published automatically after checks on its sources and content; an editor has not reviewed it. Articles are expected to link back to source material. Read our editorial standards or contact us with corrections at [email protected].

Newsletter

Weekly AI signal. No spam.

Comments

Sign in to join the discussion

Loading comments…