Nate Swenson

Put your agent's tool list on a byte budget

Local Fitness · No. 123 · 6 min read ·v0.58.0

Shipped

This release put my fitness agent’s MCP server on a diet: two pairs of near-twin tools merged (48 tools became 46), the biggest list payloads compacted by a quarter to a third, and the eval suite’s hallucination gate went from advisory to enforced. The part worth teaching is how the cuts were chosen. Not by taste; by recording 264 real tool calls and measuring where the bytes actually went. One tool turned out to be 4.6% of the calls and 24% of all returned characters.

Why the tool list itself costs money

An MCP client discovers tools with a tools/list request, and every tool ships a name, a description, and a JSON Schema for its inputs, per the MCP specification. All of that lands in the model’s context before it reads your first message. Anthropic’s code execution with MCP post describes agents burning through hundreds of thousands of tokens on tool definitions alone, and their guidance on writing tools for agents warns that too many tools, or overlapping tools, actively distract agents from efficient strategies.

So a fat tool surface costs you twice. Once per session for the definitions, and once per call for the payloads. Both are measurable, and you should measure before cutting anything.

Record the traffic

The cheapest instrument is a wrapper that logs one JSON line per call. If your server already routes every handler through a decorator, this is a few lines:

import functools
import json
import pathlib
import time

LOG = pathlib.Path("tool_calls.jsonl")

def recorded(fn):
    @functools.wraps(fn)
    def wrapper(**kwargs):
        result = fn(**kwargs)
        with LOG.open("a") as f:
            f.write(json.dumps({
                "tool": fn.__name__,
                "ts": time.time(),
                "request_chars": len(json.dumps(kwargs)),
                "response_chars": len(json.dumps(result, default=str)),
            }) + "\n")
        return result
    return wrapper

Apply @recorded where your tools register and let it run for a couple of weeks of normal use. Characters are a fine proxy for tokens here; you are hunting for ratios, not exact counts.

Find where the bytes go

Then aggregate by tool, comparing each tool’s share of calls against its share of returned bytes:

import collections
import json
import pathlib
import statistics

calls = [json.loads(line) for line in pathlib.Path("tool_calls.jsonl").open()]
by_tool = collections.defaultdict(list)
for c in calls:
    by_tool[c["tool"]].append(c["response_chars"])

total_calls = len(calls)
total_chars = sum(c["response_chars"] for c in calls)
print(f"{'tool':<28}{'calls':>7}{'chars':>7}{'median':>9}")
for tool, sizes in sorted(by_tool.items(), key=lambda kv: -sum(kv[1])):
    print(f"{tool:<28}{len(sizes)/total_calls:>6.1%}{sum(sizes)/total_chars:>6.1%}"
          f"{int(statistics.median(sizes)):>9}")

The interesting rows are the ones where the two percentages disagree. On my server, get_training_plan_progress was 4.6% of calls but 24% of all returned characters, with a median payload around 11 KB. That is the tool to open up first.

Three cuts the numbers picked

Drop the raw-plus-formatted duplication. Every workout row was shipping raw columns next to their display twins: distance_meters beside distance_mi, duration_seconds beside duration_formatted. Roughly a quarter of every list payload was the same fact twice. The fix is one pure module that owns two row shapes, a detail shape with both forms and a list shape with display only, nulls omitted. Measured on the live database: the plan-progress payload went from 11,156 to 7,547 characters (−32%), workout queries dropped 26%, the daily snapshot 12%.

Merge twin tools. Two pairs of tools were the same job with two names: a chart tool and a generate_chart sibling sharing the same fetch helper and style enums, and a raw get_metric whose schema was identical to get_metric_trend with a flag. Anthropic’s tool-writing guidance says consolidate functionality rather than exposing one tool per operation, and the failure mode is worse than bloat: given two tools for one job, the agent sometimes picks the weaker one, and you can watch it happen in the logs. A merge usually looks like a parameter, not a loss (fetch_metric_series, render_png, and render_ascii stand for the shared helpers your twins already call):

@recorded
def chart(metric: str, days: int = 30, format: str = "ascii"):
    series = fetch_metric_series(metric, days)
    if format == "png":
        return render_png(series)          # the old twin's body, verbatim
    if format != "ascii":
        raise ValueError("format must be 'ascii' or 'png'")
    return render_ascii(series)

Cap the unbounded payloads. The raw-series tool would happily return ten years of rows; I measured 63 KB for one call. A cap with an honesty flag keeps the tool useful and the payload bounded:

MAX_ROWS = 120

def raw_series(rows: list[dict]) -> dict:
    truncated = len(rows) > MAX_ROWS
    return {"values": rows[-MAX_ROWS:], "values_truncated": truncated}

The same pass also shrank the fixed per-session cost: one tool’s description inlined a 1.6 KB dump of every table’s columns, re-shipped in every session’s preamble. The columns moved to an MCP resource the agent can read when it needs them, and the description now carries table names only. Net for the release: two fewer tools and about 2.4 KB less preamble in every session, before the first call is made.

Verify it on your own logs

Run the aggregator against a few days of recorded calls. Here it is against a synthetic log of 172 calls across six tools (your table will have your tools, but the shape to hunt for is the same):

tool                          calls  chars   median
list_items                   32.0% 33.0%     2525
get_report                    7.0% 31.1%    10837
get_status                   23.8% 23.5%     2358
search                       17.4%  6.5%      929
chart                         5.8%  3.6%     1633
sync                         14.0%  2.3%      401

The row to open first is get_report: 7% of the calls carrying 31% of the bytes, the same disagreement my real server showed on its plan-progress tool. After a cut, re-run the same aggregation on fresh traffic and compare medians per tool. If a merge worked, the retired tool’s calls reappear under the survivor, and total characters drop while call counts stay flat.

Gotchas

  • Some tool names are load-bearing outside your code. I evaluated merging two more tools and deliberately did not: one of them is named in an agent’s permission grant, where a rename fails silently. Mine did exactly that once, for three weeks, before anyone noticed. Grep your grants, prompts, and configs for the tool name before you retire it.
  • A success payload that says "error": null reads as a failure. My sync tool shipped that field on every success, and naive pattern-matching flagged it. Omit the key entirely on success; only ship error when there is one.
  • Repeat calls are a payload multiplier you can remove server-side. 8 of 24 recorded sync calls were pure repeats minutes apart, the user re-asks and the agent re-syncs. A short-circuit keyed on the last successful run’s timestamp (about 10 minutes, with a force bypass) removed a third of that tool’s traffic without touching the agent.
  • Response caps need a flag, not silence. Anthropic caps tool responses in Claude Code at 25,000 tokens by default and recommends pagination and truncation with sensible defaults. Whatever cap you pick, ship a values_truncated style marker; a silently-shortened series looks complete and quietly changes downstream math.

Sources

Changelog

  • release: 0.58.0 — MCP efficiency, tool consolidation, docs sweep, eval gates (dev → main) (#215) (9bc0011)