Let the model phrase the judgment, not derive it

Shipped

This release bundles three versions of cleanup (0.22.0 through 0.22.2), but they share one spine: stop asking the language model to compute things a plain function can compute. My fitness agent has analysis tools like training_load_status, correlate, and compare_periods. Each returned raw numbers and, in a couple of cases, a static legend string like “TSB > 5 fresh, -10…5 neutral, < -10 fatigued.” Then it left the connecting LLM to look at a float and apply the legend by hand.

The brief-generation path never worked that way. It computed the zones and trend arrows in tested Python and left the model to phrase them. v0.22.0 makes the ad-hoc tools agree with the brief by extracting all of it into one pure module, agent/interpret.py, that both sides call. The two patch releases behind it (a run_sql error pointer and a table-rendering repair) are small, but the interpretation module is the real story.

Why not just let the model classify

LLMs are optimized for fluent language, not rule-based precision. Ask one whether -12.4 falls below your -10 fatigue threshold and it will usually get it right, and occasionally not, because it is pattern-matching over tokens rather than executing a comparison. Research on deterministic computation in LLMs is blunt about this: prompting alone is insufficient for reliable deterministic computation, and accuracy degrades as the arithmetic gets less trivial. The reliable pattern is a division of labor: the model plans and interprets, a deterministic component does the exact computation.

There is a second, sneakier reason. When the brief classified a TSB of -12 as “fatigued” but an ad-hoc tool call left the same number to the model and it said “a bit worn down,” those are two different reads of one fact. The system contradicted itself depending on which door you came in. Centralizing the judgment kills that class of bug by construction.

Build it: a pure classifier module

The rule I settled on: the LLM phrases a judgment, it never derives one that tested Python can compute. So every classifier moves into a module with no I/O, no SDK, and no schemas, just stdlib. Named constants for every band, so a boundary can be pinned by a test on both sides.

# agent/interpret.py
TSB_VERY_FATIGUED = -20.0
TSB_FATIGUED = -10.0
TSB_FRESH = 5.0


def tsb_zone(tsb: float | None) -> str:
    """Plain-English read of training stress balance (CTL - ATL)."""
    if tsb is None:
        return "no training-load data yet"
    if tsb < TSB_VERY_FATIGUED:
        return "very fatigued"
    if tsb < TSB_FATIGUED:
        return "fatigued"
    if tsb > TSB_FRESH:
        return "fresh"
    return "neutral"


def pct_change(now: float | None, then: float | None) -> float | None:
    """Percent change from `then` to `now`; None on a zero or missing base."""
    if now is None or then is None or then == 0:
        return None
    return (now - then) / then * 100

Two details matter more than they look. First, every classifier handles None and degenerate input without raising, the same guarantee the read paths make against an empty database. A judgment layer that throws on missing data is worse than no layer. Second, the inclusivity conventions are chosen on purpose and documented: the SD bands are strict (> +1, < -1), while the correlation and effect-size bands are inclusive on the lower bound. Writing those boundaries down once, with a constant, means the brief and the tools can never drift apart on where “moderate” ends.

Wire it into a tool

The point of a shared module is that the caller gets thinner. Here is training_load_status after the change: it fetches numbers, then hands each one to interpret instead of shipping a legend for the model to apply.

async def training_load_status(_args: dict) -> dict:
    from . import brief_planner

    cutoff = (date.today() - timedelta(days=30)).isoformat()
    with db.connect() as conn:
        recent = [dict(r) for r in conn.execute(
            "SELECT date, ctl, atl, tsb FROM baselines "
            "WHERE date >= ? AND ctl IS NOT NULL ORDER BY date DESC",
            (cutoff,),
        ).fetchall()]
        if not recent:
            return _err("no training-load data yet — call sync_garmin_data")
        # Single-source the 14-day "then" CTL from the same query the brief
        # uses, so the tool and the brief agree even on gappy baselines.
        anchor = (date.today() - timedelta(days=brief_planner._LOOKBACK_DAYS)).isoformat()
        ctl_then = brief_planner.ctl_at_or_before(conn, anchor)

    current = recent[0]
    ctl_pct = interpret.pct_change(current.get("ctl"), ctl_then)
    if ctl_pct is not None:
        ctl_pct = round(ctl_pct, 1)

    return _text({
        "current": current,
        "tsb_zone": interpret.tsb_zone(current.get("tsb")),
        "ctl_pct_change_14d": ctl_pct,
        "ctl_direction": interpret.delta_direction(ctl_pct),
        "history_30d": recent,
    })

Notice ctl_direction uses delta_direction, not trend_direction. A 14-day percent change is a single scalar, not a fitted slope over a series, so it gets the classifier built for scalars. Same idea, different shape, and keeping them as two functions stops a caller from quietly feeding a scalar into a slope classifier. Across the release the same treatment landed on correlate (gains strength and direction, drops its legend string), find_anomalies (rows gain sd_distance and direction), compare_periods (gains delta_pct and Cohen’s d magnitude), and get_metric_trend. Every float is rounded at the payload boundary, skipping None rather than raising.

Verify it: pin both sides of every band

Because the bands are the whole point, the tests assert the boundary, not a happy-path value in the middle of a zone:

def test_tsb_zone_boundaries():
    # strictly-less-than: exactly -20 is NOT yet "very fatigued"
    assert interpret.tsb_zone(-20.0) == "fatigued"
    assert interpret.tsb_zone(-20.01) == "very fatigued"
    assert interpret.tsb_zone(5.0) == "neutral"      # exactly +5 is not "fresh"
    assert interpret.tsb_zone(5.01) == "fresh"
    assert interpret.tsb_zone(None) == "no training-load data yet"


def test_pct_change_zero_base_is_none():
    assert interpret.pct_change(10.0, 0.0) is None   # no defined % change
    assert interpret.pct_change(None, 5.0) is None
    assert interpret.pct_change(11.0, 10.0) == 10.0

Each of those would fail if someone flipped a < to a <= or dropped the zero-base guard. That is the bar: a test that only passes because the code is correct, not because it echoes a mocked return.

The failure mode this whole release is insurance against is silent divergence. Before, nothing broke loudly when the brief and an ad-hoc tool disagreed about the same TSB, they just told me two slightly different stories and I had no way to notice. Now there is exactly one function that decides what -12 means, both paths call it, and a boundary test guards it. The model still writes the sentence, which is the part it is actually good at. It just no longer gets a vote on the math.

Sources

Evaluating Prompting and Execution-Based Methods for Deterministic Computation in LLMs — prompting alone is insufficient for reliable deterministic computation; execution-based approaches do far better.
Why LLMs Are Bad at Math, and How They Can Be Better — the model-plans / calculator-computes division of labor.
Why LLMs Struggle: Math, Structured Data & AI Reasoning Limits — LLMs are probabilistically optimized for fluency, not rule-based precision.

Changelog

Deterministic interpretation module agent/interpret.py, MCP payload-quality upgrades, and agent UX fixes across 0.22.0–0.22.2 (d4c631e)