Small Models, Big Payoff: Teamwork Fixs AI Tool-Calling Errors

Large language models have dazzled the world with their ability to write, reason and converse, but when it comes to actually doing things—booking a flight, querying a database, triggering a smart-home routine—they often stumble on something almost embarrassingly mundane: formatting. A new study published in the open-access journal Vicinagearth argues that the single biggest bottleneck in letting AI agents call external tools is not intelligence at all, but the rigid syntactic discipline required to produce a machine-readable API request. The research team, led by Yudian Zhang and Xuelong Li at the Institute of Artificial Intelligence (TeleAI) of China Telecom, together with Haijiang Zhu of Beijing University of Chemical Technology, proposes an elegantly simple remedy: let a large model think and a small model tidy up.

The work arrives at a moment when the AI industry is pouring enormous resources into so-called agentic systems—models that autonomously plan tasks and invoke software tools on the user’s behalf. Tool invocation sits at the heart of this vision. In the standard tool-learning pipeline, which researchers typically divide into task planning, tool selection, tool invocation and response generation, the invocation stage is the make-or-break moment. The model must extract parameters from a natural-language query, match them to a tool’s specification, and emit a request so precisely structured that a downstream server can parse it without error. Any stray character, a missing parenthesis, or a misplaced comma can cause the entire call to fail silently.

What the researchers discovered through systematic perturbation experiments is striking: the success of a tool call is far more sensitive to format standardization than to semantic accuracy. When they fine-tuned the Llama3.1-8B-Instruct model on the ToolACE dataset using LoRA, randomly altering numbers in the training labels left accuracy nearly untouched, and shuffling parameter strings produced only a modest decline. But when they changed the format itself—swapping bracket types, converting integers to floating-point numbers, or reordering parameters—performance collapsed. Simply changing bracket styles dragged live-task accuracy down to 42.51 percent from a much higher baseline. Converting numbers to floats proved most devastating of all, with one metric plunging to 25.39 percent, because the abstract syntax tree evaluation used by the benchmark flags data-type mismatches instantly.

The most dramatic result came from compounding these perturbations. In a double mixed-modification experiment that first randomized bracket usage and then converted numbers to floats, live accuracy cratered to just 3.02 percent—essentially total failure. The lesson, the authors argue, is that conventional fine-tuning creates what they call format fragility: models rigidly cling to whatever format patterns they saw in training data, and even when prompts explicitly specify an output format, fine-tuned models frequently ignore those instructions and emit unparsable output. Earlier studies have described this phenomenon as format specialization or task locking, where intense fine-tuning erodes a model’s general in-context learning ability on non-target tasks.

Recognizing that reasoning and formatting are fundamentally different skills, the team designed a division-of-labor architecture that separates them. In their collaborative framework, the large language model receives the user’s question and a list of available tools, then produces an intermediate output containing its thought process, the selected tool name and the parameter information. Crucially, this intermediate output need not follow any format at all—the large model is freed from worrying about syntax. That freedom is precisely what preserves its generalization. The intermediate result is then handed to a small, specialized format model whose sole job is to normalize it into a strict, predefined structure that can be parsed directly into a callable API request.

The experimental payoff was substantial. When the same perturbed models were paired with the formatting model, accuracy rebounded dramatically. The Random Mix Twice configuration, which had fallen to 3.02 percent, soared to 73.42 percent once the small model normalized the output. The formatting step effectively absorbs all the chaotic variations—missing parentheses, wrong number formats, unexpected parameter orders—that would otherwise doom the invocation. The authors also contrast their approach with in-context learning, noting that few-shot examples struggle to exhaustively cover complex, nested parameter schemas, whereas a dedicated format model explicitly models the output structure and separates tool selection from argument generation.

The study used the Berkeley Function Call Leaderboard, a benchmark of more than 1,700 instances spanning simple, multiple, parallel and parallel-multiple function calls in Python, as well as REST API, JavaScript and Java tasks. Evaluation relied on the benchmark’s abstract syntax tree methodology, which checks whether function names, required parameters and data types all conform to the function documentation. The experiments ran on a single RTX 4090 GPU, underscoring that the collaborative method is computationally modest: instead of retraining a giant model, it attaches a lightweight normalizer to the end of the pipeline.

The implications reach across the AI industry. Giants including IBM’s Granite-20B-FunctionCalling, ToolLLM, APIGen and ToolACE have all pursued better function-calling models through increasingly sophisticated fine-tuning and dataset synthesis. But the new study suggests a quiet vulnerability running through that entire paradigm: as long as a single model is asked to be both reasoner and formatter, it will remain brittle. A wrong parameter value may still pass parsing and merely yield an irrelevant result, but a wrong bracket is fatal. The finding that format correctness outranks content accuracy inverts a common assumption that semantic quality is the primary axis of model quality, and it offers a practical, modular fix that developers could retrofit onto existing systems without touching the underlying model weights.

The authors are candid about the limits of their approach. Their evaluation remains confined to static, single-turn settings on specific test sets, while real-world agents must handle multi-turn dialogues that demand consistent tracking of context and parameters across turns. Real APIs also evolve their schemas over time, and format learning grounded in fixed training data struggles to adapt. Yet the multi-model framework points toward a natural solution: the large model can continue to handle contextual reasoning and intent understanding while the small model operates as a lightweight, updatable formatter that maps intent to whatever the current API schema requires. The team plans to integrate schema-based validation checkers and explore online adaptation techniques, including few-shot in-context learning and parameter-efficient fine-tuning, to keep inference costs low while boosting robustness in dynamic deployments.

For a field fixated on scale, the takeaway is refreshingly counterintuitive. Sometimes the fastest way to make a giant AI smarter is to pair it with a tiny, single-minded helper obsessed with punctuation. By decoupling what a model knows from how it says it, the study reframes tool learning as a coordination problem rather than a capability problem—and suggests that the next leap in autonomous AI agents may come not from bigger brains, but from better teamwork.

Subject of Research: Enhanced tool invocation method through multi-model collaboration

Article Title: Enhanced tool invocation method through multi-model collaboration

Article References: Enhanced tool invocation method through multi-model collaboration. (n.d.). https://doi.org/10.1007/s44336-025-00028-7

Image Credits: AI Generated

DOI: 10.1007/s44336-025-00028-7

Keywords: Enhanced, tool, invocation, method, multi-model, collaboration, scientific research