nate.swenson_
Local Fitness

Never let one LLM call sink a whole report

· 9 min read ·v0.20.0

Shipped

This release reworked my fitness agent’s daily brief PDF. The takeaway cards now reflow into a two-column grid instead of one long stack, and a new Training Plan section shows adherence percentage, days to race, this week’s planned versus actual mileage, and the last seven days graded against prescription. One piece of that section is generated at render time: a short Claude-written paragraph prepping me for today’s prescribed run, called fresh on every render with a deterministic fallback if the call fails. That fallback is the part worth teaching. A single LLM call inside a synchronous render is a single point of failure, and the pattern that defuses it fits in one small module you can add to your own report generator this week.

Why a render path can’t wait on a model

Generating the report is a synchronous, user-facing action: I ask for the PDF, I want the file. By the time the coaching line is requested, the charts, the adherence math, and the graded table are already computed and correct. Losing the whole PDF because one paragraph didn’t come back is a bad trade, and a blank spot where the paragraph should be isn’t much better.

The general shape of the fix is old. Martin Fowler’s circuit breaker writeup describes wrapping a risky remote call so that once failures trip the breaker, callers get an immediate error “without the protected call being made at all”, instead of hanging on a resource that isn’t coming back. The Google SRE book’s overload chapter pushes the same idea one step further: serve a degraded response, one that is less accurate or carries less data but is cheap to compute, rather than serving nothing. A once-a-day report doesn’t need the breaker’s stateful trip-and-reset machinery. It needs the degraded response: a try/except around the model call, a hard deadline, and a template that can’t fail.

You need Python 3.10+ to follow along. The primary path uses pip install claude-agent-sdk plus an authenticated Claude Code install, but here’s the part I like about this build: the failure path, which is the part worth testing, runs with no SDK and no credentials at all. You can verify the whole degraded path before you set up the happy one.

Build the primary call with a hard deadline

Start a file called coaching.py. The primary path is a single-turn call: no tools, one turn, and a deadline enforced from outside the SDK.

import asyncio
import logging

log = logging.getLogger(__name__)


async def generate_line(system_prompt: str, user_prompt: str,
                        *, model: str, timeout: float = 30.0) -> str:
    """Single-turn LLM call with a hard deadline. Raises on timeout, on a
    transport error, and on an empty response; it never returns a blank
    string as if it succeeded."""
    from claude_agent_sdk import (AssistantMessage, ClaudeAgentOptions,
                                  TextBlock, query)

    options = ClaudeAgentOptions(
        system_prompt=system_prompt,
        model=model,
        max_turns=1,
    )

    async def _run() -> str:
        chunks: list[str] = []
        async for message in query(prompt=user_prompt, options=options):
            if isinstance(message, AssistantMessage):
                for block in message.content:
                    if isinstance(block, TextBlock):
                        chunks.append(block.text)
        return "".join(chunks).strip()

    text = await asyncio.wait_for(_run(), timeout=timeout)
    if not text:
        raise RuntimeError("LLM returned an empty response")
    return text

Three decisions in that block carry the reliability story. First, the SDK import lives inside the function, not at module scope, so any process that imports this module without ever rendering a report never pays for the dependency; the gotchas below cover a second, sneakier reason to defer imports here. Second, the deadline comes from asyncio.wait_for, which cancels the task and raises TimeoutError when the clock runs out, so a hung connection becomes an ordinary exception instead of a hung report. Anthropic’s error reference is the motivation: a call like this can hit rate limits (429), an overloaded API (529), a server-side timeout (504), and, on streaming responses, errors that arrive mid-stream after the API already returned a 200. There is no single exception that means “the network failed”, so the caller is going to catch broadly rather than enumerate.

Third, an empty response raises. An empty string is a successful call at the transport layer and useless as content. If you treat it as success, your fallback fires on network errors and never on the quieter failure.

Build the fallback that can’t fail

The fallback is a plain function in the same file. No I/O, no randomness, no code path that raises:

_VERDICT_PHRASE = {
    "done": "You hit yesterday's session clean.",
    "partial": "Yesterday came up short of the prescription.",
    "missed": "Yesterday was a skip.",
    "rest": "Yesterday was a scheduled rest day.",
}


def fallback_line(today: dict, history: list[dict],
                  days_to_race: int | None) -> str:
    """Deterministic template line, used only when generate_line fails.
    Pure and total: same inputs always give the same output, and no input
    makes it raise."""
    parts: list[str] = []
    prior = next((d for d in history if d.get("verdict") != "pending"), None)
    if prior is not None:
        phrase = _VERDICT_PHRASE.get(prior.get("verdict"))
        if phrase:
            parts.append(phrase)

    prescription = today["type"]
    if today.get("distance_mi") is not None:
        prescription += f" {today['distance_mi']} mi"
    parts.append(f"Today: {prescription}.")

    if days_to_race is not None:
        parts.append(f"{days_to_race} days out.")
    return " ".join(parts)

Every lookup that could throw is written not to: next() gets a None default, the verdict phrase uses .get(), and the optional fields are guarded. This function only ever runs on a day something else already failed, so it doesn’t get to have failure modes of its own.

Wire the seam, then run it

The two paths meet in one place, and it’s deliberately boring: try the model, catch everything, fall back, and log so the failure stays visible even though the reader of the PDF never sees it.

async def build_coaching_line(today: dict, history: list[dict],
                              days_to_race: int | None,
                              *, model: str = "claude-sonnet-5") -> str:
    """The seam where the two paths meet: try the model, catch everything,
    fall back, and log so the failure stays visible."""
    system_prompt = (
        "You are a running coach. Write ONE short paragraph (2-4 sentences) "
        "prepping the runner for today's prescribed workout. Output only the "
        "paragraph itself: no headline, no markdown, no preamble."
    )
    lines = [f"Today's prescribed workout: {today['type']} "
             f"{today.get('distance_mi') or ''} mi.".replace("  ", " ")]
    if days_to_race is not None:
        lines.append(f"Days to race: {days_to_race}.")
    for d in history:
        lines.append(f"  {d['date']}: planned {d.get('planned_mi')} mi, "
                     f"actual {d.get('actual_mi')} mi, verdict {d['verdict']}")

    try:
        return await generate_line(system_prompt, "\n".join(lines), model=model)
    except Exception:
        log.warning("coaching line generation failed, using fallback",
                    exc_info=True)
        return fallback_line(today, history, days_to_race)

Here’s the early win: put this in demo.py next to coaching.py and run it before you’ve installed the SDK.

import asyncio
import logging

from coaching import build_coaching_line

logging.basicConfig(level=logging.WARNING, format="%(levelname)s %(message)s")

today = {"type": "easy", "distance_mi": 4.0}
history = [
    {"date": "2026-07-08", "planned_mi": 4.0, "actual_mi": 3.0, "verdict": "partial"},
    {"date": "2026-07-07", "planned_mi": None, "actual_mi": None, "verdict": "rest"},
]

print(asyncio.run(build_coaching_line(today, history, days_to_race=6)))

python3 demo.py on a machine with no SDK installed prints (traceback elided):

WARNING coaching line generation failed, using fallback
Traceback (most recent call last):
  ...
ModuleNotFoundError: No module named 'claude_agent_sdk'
Yesterday came up short of the prescription. Today: easy 4.0 mi. 6 days out.

That’s the whole pattern working end to end: the primary path failed for a real reason, the log captured it, and the caller still got a sane line. Once you install and authenticate the SDK, the same script prints a model-written paragraph instead, and nothing else about the call site changes.

Verify the path you hope never runs

A fallback that only gets exercised in production, on the one day the primary call actually breaks, is a fallback you’ve never tested. Because fallback_line is pure, testing it costs almost nothing, and the seam test just forces the primary to blow up. Put this in test_coaching.py:

import asyncio

import coaching

TODAY = {"type": "easy", "distance_mi": 4.0}
HISTORY = [
    {"date": "2026-07-08", "planned_mi": 4.0, "actual_mi": 3.0, "verdict": "partial"},
]


def test_fallback_used_when_primary_raises(monkeypatch):
    async def boom(*args, **kwargs):
        raise RuntimeError("simulated transport failure")

    monkeypatch.setattr(coaching, "generate_line", boom)

    line = asyncio.run(coaching.build_coaching_line(TODAY, HISTORY, days_to_race=6))
    assert line == coaching.fallback_line(TODAY, HISTORY, 6)


def test_fallback_is_pure():
    a = coaching.fallback_line(TODAY, HISTORY, 6)
    b = coaching.fallback_line(TODAY, HISTORY, 6)
    assert a == b == (
        "Yesterday came up short of the prescription. Today: easy 4.0 mi. 6 days out."
    )


def test_fallback_never_raises_on_unknown_verdict():
    history = [{"date": "2026-07-08", "verdict": "totally-unexpected"}]
    line = coaching.fallback_line(TODAY, history, None)
    assert line == "Today: easy 4.0 mi."

Run it with python3 -m pytest test_coaching.py -q (or uv run --with pytest -- pytest test_coaching.py -q if you don’t keep pytest installed globally). Real output from my run:

...                                                                      [100%]
3 passed in 0.01s

The first test is the one that earns its keep: it pins the contract that any exception from the primary, not just the ones you thought to enumerate, lands on the fallback. The third pins the “never raises” claim against a verdict value the phrase table has never seen.

Gotchas

  • An empty response is a transport-level success. Symptom: the PDF renders “successfully” with a blank coaching box, and your fallback never fired because nothing raised. Escape: check the joined text and raise on empty, the way generate_line does. In the shipped version this is an explicit RuntimeError with its own unit test, because it’s the failure nobody’s retry logic notices.
  • No deadline means a hung render, not a failed one. Networks drop idle connections, and streaming errors can arrive after the API already said 200, per Anthropic’s error reference. Symptom: the report request just never returns. Escape: wrap the whole call in asyncio.wait_for so a hang becomes a TimeoutError your except clause converts into the fallback.
  • Module-scope imports can close a circular-import loop. The shipped version reads the model name from the same constants module the daily brief generator uses, and that module imports the tools module, which imports the coaching module. Importing it at module scope would have completed the cycle and broken the always-running server at startup, in a process that never renders PDFs at all. Symptom: an import error at process start, nowhere near the code you changed. Escape: defer both the SDK import and the shared-constants import to call time, by which point every module involved has finished initializing.
  • The fallback itself is the code that must not throw. The tempting version indexes the verdict table directly with prior["verdict"]; one unexpected verdict value and the “can’t fail” path dies with a KeyError on exactly the day it’s the only path left. Symptom: your report fails anyway, from inside the except clause that was supposed to save it. Escape: total functions only. .get() with defaults, guards on optional fields, and unit tests that feed it degenerate inputs; the shipped version pins empty history and all-pending history the same way.

Sources

Changelog

  • release: 0.20.0 — brief PDF redesign + ephemeral reports (dev → main) (#93) (d418ac4)