Nate Swenson

Two answers for one morning's resting heart rate

Local Fitness · No. 124 · 7 min read ·v0.60.1

Shipped

This release taught my fitness agent the difference between data that is late and data that is still changing. Garmin revises some of today’s values through the morning rather than accumulating them, so the agent now carries a settled-data contract: trend and anomaly tools exclude today’s not-yet-settled readings from their verdicts, the morning brief moved from 06:30 to 08:30 so it runs after the numbers stop moving, and the eval baseline was recaptured to an honest floor. The contract is the part worth building yourself, because most pipelines have at least one metric like this and treat it like all the others.

The morning that produced two todays

Here is the failure, measured live. My 06:30 scheduled pull ran while I was still asleep and stored a resting heart rate of 54. At 10:04, the trend tool served that snapshot as today’s value: elevated, nearly two standard deviations above baseline. At 10:09, the next pull revised it to 50, and the morning brief reported 50. Two contradictory readings of the same morning, both presented with full confidence, and no signal on either that anything was provisional.

Nothing failed. The syncs succeeded, the queries were correct, the storage was intact. The bug is in a hidden assumption: that a stored value for today is today’s value. For this class of metric it is not; the device keeps reprocessing the night until you are properly awake, and the number moves.

Late data and revised data are different problems

Stream-processing systems have a vocabulary for part of this. A watermark is an estimate that all data for a window has arrived; Dataflow’s docs define it as the threshold after which anything else for that window is late data. My agent already handled lateness: step counts are running tallies, partial all day, so the tools anchor on yesterday and label today pending.

But a settling metric is not late; it has arrived and is wrong. That is closer to what Martin Fowler describes in bitemporal history: the record says one thing at 06:30 and another at 10:09, and both were honestly what we knew at the time. The practical question for a serving layer is simpler than full bitemporality, though. You only need to answer: has this value settled yet, and if not, what may read it?

Three behaviors, three contracts:

  • Accumulating (steps): partial all day. Anchor on yesterday, always.
  • Settling (resting heart rate, sleep, battery min/max): moving until mid-morning, then exactly what you want. Readable today, but only once fresh and labeled until the day closes.
  • Immutable once written (a finished workout): no special handling.

Build the settling contract

Declare which metrics settle. This is domain knowledge, and it belongs in code rather than in a prompt:

SETTLING_METRICS = {"rhr", "sleep_seconds", "body_battery_min", "body_battery_max"}
SYNC_WINDOW_MIN = 10

The rule for serving today’s value: it counts only if a successful pull that covered today finished within the last few minutes. Both halves matter. Recency alone is not enough, because a pull can succeed while fetching only old days:

import datetime as dt

def fresh_covering_today(last_pull: dict | None, now: dt.datetime) -> bool:
    """A pull counts only if it succeeded, recently, AND its fetch range
    reached today. `last_pull` is your sync log's most recent success:
    {"completed_at": datetime, "last_date_fetched": date}."""
    if last_pull is None:
        return False
    age_min = (now - last_pull["completed_at"]).total_seconds() / 60
    return age_min <= SYNC_WINDOW_MIN and last_pull["last_date_fetched"] >= now.date()

Then the serving function computes its statistics on the settled series, and carries the unsettled value only as an explicitly-labeled extra:

import statistics

def metric_trend(metric: str, rows: list[dict], last_pull: dict | None,
                 now: dt.datetime) -> dict:
    """rows: [{"date": date, "value": float}] sorted ascending, ending today."""
    today = now.date()
    settled = [r for r in rows if r["date"] < today]
    result = {
        "metric": metric,
        "mean": round(statistics.mean(r["value"] for r in settled), 1),
        "current": settled[-1]["value"],
    }
    todays = next((r for r in rows if r["date"] == today), None)
    if todays is None or metric not in SETTLING_METRICS:
        if todays is not None:
            result["current"] = todays["value"]
        return result
    if fresh_covering_today(last_pull, now):
        result["current"] = todays["value"]
        result["current_provisional"] = True
        result["data_as_of"] = last_pull["completed_at"].isoformat()
    else:
        result["provisional_today_value"] = todays["value"]
        result["note"] = "today's value is stale for a settling metric; sync first"
    return result

The key design choice: when the snapshot is stale, the verdict fields (current, mean, and in the real version slope and versus-baseline) are computed on the settled series. Exclusion is deterministic. The alternative, serving the raw number with a warning flag, hands the judgment to whatever reads the payload, and a model reading it will sometimes skip the flag. Downstream of an LLM, a value you should not use must not be sitting in the field it would be used from.

Anomaly detection gets the strictest form. An anomaly is a claim that something settled out of range, so for settling metrics the scan simply never reads today:

def anomaly_rows(metric: str, rows: list[dict], today: dt.date) -> list[dict]:
    if metric in SETTLING_METRICS:
        return [r for r in rows if r["date"] < today]
    return rows

Then fix the schedule, not just the reads

The serving contract protects ad-hoc reads, but my flagship surface was a scheduled morning brief firing at 06:30, before I was awake. Every guard above would have correctly labeled its data provisional, and it would have shipped a hedged brief every single day. The better fix was embarrassingly simple: move the job to 08:30, past typical wake, so the numbers it reads are settled by construction, and keep a 09:30 backstop. When a schedule and a settling window conflict, reschedule; do not teach every consumer to apologize.

Verify it

Feed the trend function a stale morning and a fresh one; the verdict must not move between them, only the labels:

rows = [{"date": dt.date(2026, 8, d), "value": v}
        for d, v in [(7, 49.0), (8, 50.0), (9, 51.0), (10, 54.0)]]
now = dt.datetime(2026, 8, 10, 10, 4)
stale = {"completed_at": dt.datetime(2026, 8, 10, 6, 30),
         "last_date_fetched": dt.date(2026, 8, 10)}
fresh = {"completed_at": dt.datetime(2026, 8, 10, 10, 1),
         "last_date_fetched": dt.date(2026, 8, 10)}
print(metric_trend("rhr", rows, stale, now))
print(metric_trend("rhr", rows, fresh, now))

Running exactly that prints:

{'metric': 'rhr', 'mean': 50.0, 'current': 51.0, 'provisional_today_value': 54.0, 'note': "today's value is stale for a settling metric; sync first"}
{'metric': 'rhr', 'mean': 50.0, 'current': 54.0, 'current_provisional': True, 'data_as_of': '2026-08-10T10:01:00'}

The stale read serves the most recent settled value (51.0) with the 54 quarantined in provisional_today_value; the fresh read serves 54.0 but stamped current_provisional with a data_as_of. If your stale line serves 54 unlabeled, the guard is not wired into the verdict path.

Gotchas

  • A successful recent sync does not mean today was fetched. My first freshness check keyed on the pull’s timestamp alone; a backfill of last month stamps that fresh while covering nothing recent. That is why fresh_covering_today also checks last_date_fetched. This one is invisible in testing unless a fixture has a backfill in it.
  • Opt-in beats blanket enforcement when some callers pull first. My brief pipeline syncs immediately before reading, so its data is always fresh and its planner fixtures are byte-identical with the guard off. The guard defaults off on the shared assembly path and is switched on only by the surfaces that serve ad-hoc reads. Turning it on globally would have churned every fixture to protect a path that could not be stale.
  • This is a different defect than a running tally. I had already shipped partial-today handling for accumulating metrics and assumed it covered the morning readings. It cannot: a tally is partial all day and anchors on yesterday forever, while a settling metric becomes exactly what you want mid-morning and should be served then. Two contracts, and collapsing them loses either the morning’s data or its correctness.

Sources

Changelog

  • release: 0.60.1 — settled-data contract, 08:30 brief, grounding unit binding, honest eval floor (dev → main) (#220) (d9d7c40)