Augmenting massive language fashions (LLMs) with exterior instruments, moderately than relying solely on their inner information, may unlock their potential to resolve more difficult issues. Widespread approaches for such “instrument studying” fall into two classes: (1) supervised strategies to generate instrument calling capabilities, or (2) in-context studying, which makes use of instrument paperwork that describe the meant instrument utilization together with few-shot demonstrations. Software paperwork present directions on instrument’s functionalities and the way to invoke it, permitting LLMs to grasp the person instruments.
Nevertheless, these strategies face sensible challenges when scaling to a lot of instruments. First, they undergo from enter token limits. It’s unattainable to feed your entire record of instruments inside a single immediate, and, even when it had been potential, LLMs nonetheless usually wrestle to successfully course of related info from prolonged enter contexts. Second, the pool of instruments is evolving. LLMs are sometimes paired with a retriever educated on labeled question–instrument pairs to advocate a shortlist of instruments. Nevertheless, the perfect LLM toolkit needs to be huge and dynamic, with instruments present process frequent updates. Offering and sustaining labels to coach a retriever for such an intensive and evolving toolset could be impractical. Lastly, one should cope with ambiguous consumer intents. Consumer context within the queries may obfuscate the underlying intents, and failure to determine them may result in calling the wrong instruments.
In “Re-Invoke: Software Invocation Rewriting for Zero-Shot Software Retrieval”, offered at EMNLP 2024, we introduce a novel unsupervised retrieval methodology particularly designed for instrument studying to deal with these distinctive challenges. Re-Invoke leverages LLMs for each instrument doc enrichment and consumer intent extraction to boost instrument retrieval efficiency throughout varied use instances. We exhibit that the proposed Re-Invoke methodology persistently and considerably improves upon the baselines overlaying each single- and multi-tool retrieval duties on instrument use benchmark datasets.
