Green CI, two releases behind

Press · No. 087

Shipped

press 0.4.0 added propagate, which re-emits every generated region in a consumer’s checkout, bumps the version it finds pinned in that repo’s CI, and reports what actually moved. A workflow turns that into a pull request in each consumer whose bytes changed. The gap it closed was concrete: one consumer sat two releases behind the shared brand with entirely green CI, because its check was pinned to the version it had adopted and passed forever against exactly that.

The pin is not the mistake. Consumers pin an exact version deliberately, since a mutable reference in a repo that auto-deploys to production is a supply-chain hole, and a reproducible check is the only kind worth gating a merge on. The mistake is expecting one mechanism to answer two questions. This guide builds the second one.

Two questions a pinned check cannot both answer

Write them down separately, because the whole design falls out of the distinction:

Integrity. Does this region match the version this repo adopted? Answered inside the consumer, against a pinned version, on every PR. This is what a drift check is for, and pinning is what makes it reproducible.

Freshness. Has this repo adopted the current release? A pinned check structurally cannot answer this. It compares the consumer against the version named in its own config, so a repo pinned at 0.2.0 reports a clean run for as long as it stays at 0.2.0, no matter how far the source of truth has moved.

This trade is not a surprise to anyone who has thought about pinning. The OpenSSF Scorecard’s Pinned-Dependencies check argues for pinning because it “can help mitigate compromised dependencies from undermining the security of the project”, and in the same breath concedes that “pinning dependencies can inhibit software updates”, recommending you employ “automated notification tools when dependencies become outdated” to compensate. The propagator below is that compensating half, written by the producer instead of bolted onto each consumer.

Since the consumer can’t answer the second question without giving up the property that makes the first one useful, the source of truth answers it instead and pushes the result out. That’s the same shape Dependabot uses from the other direction: “When Dependabot identifies an outdated dependency, it raises a pull request to update the manifest to the latest version of the dependency.” The difference is who holds the knowledge. Dependabot watches a registry on the consumer’s behalf; here the producer already knows it changed and fans that out, which also lets it tell a rendered change apart from a version bump nobody would see.

Two rules keep that from turning into noise, and they are the whole reason this is worth writing rather than just running the emitter everywhere:

  • Content decides, not the version receipt. A region written by 0.1.0 that is still byte-correct today is current. A pull request for it changes nothing anyone can see.
  • A stale pin alone is not “behind”. It alters no shipped artifact, so bump it quietly and never open a pull request just for it.

Build the propagator

Everything here is Node’s standard library. Save it as propagate.mjs:

#!/usr/bin/env node
/**
 * Two different questions, and conflating them is what leaves a consumer stale:
 *
 *   INTEGRITY  "does this region match the version this repo adopted?"
 *              Answered inside the consumer, against a PINNED version.
 *   FRESHNESS  "has this repo adopted the CURRENT release?"
 *              Answered here, by running the newest release against its
 *              checkout. A pinned check can never answer it: it passes forever
 *              against the version it was pinned to.
 */
import { readFileSync, writeFileSync, readdirSync, existsSync } from 'node:fs';
import { join } from 'node:path';

const PIN_RE = /(@acme\/brand@)(\d+\.\d+\.\d+)/g;

export const emitBody = (tokens) =>
  Object.entries(tokens).map(([k, v]) => `  --${k}: ${v};`).join('\n');

export function findRegion(text) {
  const lines = text.split('\n');
  const start = lines.findIndex((l) => /^\/\* >>> brand /.test(l));
  const end = lines.findIndex((l) => /^\/\* <<< brand \*\/\s*$/.test(l));
  if (start === -1 || end === -1 || end < start) return null;
  return {
    start, end,
    body: lines.slice(start + 1, end).join('\n'),
    version: (/v(\d+\.\d+\.\d+)/.exec(lines[start]) ?? [])[1] ?? null,
  };
}

export function spliceRegion(text, body, version) {
  const found = findRegion(text);
  const lines = text.split('\n');
  lines.splice(found.start, found.end - found.start + 1,
    `/* >>> brand v${version} GENERATED, do not edit */`, body, '/* <<< brand */');
  return lines.join('\n');
}

/** Bump every pinned reference found under .github/workflows. Returns how many. */
function bumpPins(root, version, dryRun) {
  const dir = join(root, '.github', 'workflows');
  if (!existsSync(dir)) return 0;
  let bumped = 0;
  for (const file of readdirSync(dir)) {
    const path = join(dir, file);
    const before = readFileSync(path, 'utf8');
    const after = before.replace(PIN_RE, (m, prefix, found) =>
      found === version ? m : `${prefix}${version}`);
    if (after !== before) {
      bumped += 1;
      if (!dryRun) writeFileSync(path, after);
    }
  }
  return bumped;
}

export function propagate({ tokens, consumer, root, version, dryRun = false }) {
  const path = join(root, consumer.path);
  const before = readFileSync(path, 'utf8');
  const found = findRegion(before);
  if (!found) return { id: consumer.id, verdict: 'missing' };

  const body = emitBody(tokens);
  // RULE 1: content decides, not the version receipt. A region written by an
  // old release that is still byte-correct today IS current; opening a pull
  // request for it would be pure noise.
  //
  // Carry this as a boolean. Deriving it later by matching a status string
  // (/updated$/) silently misses "would update", so a dry run reports
  // "nothing to do" while printing a changed region.
  const changed = found.body.replace(/\s+$/, '') !== body.replace(/\s+$/, '');
  if (changed && !dryRun) writeFileSync(path, spliceRegion(before, body, version));

  const pinsBumped = bumpPins(root, version, dryRun);

  // RULE 2: a stale pin alone is not "behind". It changes no shipped artifact,
  // so bump it quietly and never open a pull request just for it.
  const verdict = changed ? 'brand' : pinsBumped > 0 ? 'stale' : 'current';
  return { id: consumer.id, verdict, changed, pinsBumped, wroteBy: found.version };
}

const EXPLAIN = {
  brand: 'BRAND VALUES CHANGED -> open a pull request',
  stale: 'pin was behind, no rendered change -> bumped quietly',
  current: 'already current',
  missing: 'no brand region in this checkout',
};

if (import.meta.url === `file://${process.argv[1]}`) {
  const dryRun = process.argv.includes('--dry-run');
  const version = JSON.parse(readFileSync('release.json', 'utf8')).version;
  const tokens = JSON.parse(readFileSync('tokens.json', 'utf8'));
  const consumers = JSON.parse(readFileSync('consumers.json', 'utf8'));

  const results = consumers.map((c) =>
    propagate({ tokens, consumer: c, root: c.root, version, dryRun }));

  for (const r of results) {
    const tag = r.verdict === 'brand' ? 'PR  ' : '    ';
    console.log(`${tag}${r.id.padEnd(10)} wrote-by ${String(r.wroteBy).padEnd(7)} ${EXPLAIN[r.verdict]}`);
  }
  const prs = results.filter((r) => r.verdict === 'brand').length;
  console.log(`\n${results.length} consumers, ${prs} pull request${prs === 1 ? '' : 's'}${dryRun ? ' (dry run)' : ''}`);
}

Run it against two consumers that disagree

Set up two checkouts that are wrong in opposite directions. One has an old pin but correct content; the other is pinned at the current release and carries an accent color that has since moved.

cat > release.json <<'EOF'
{ "version": "0.4.0" }
EOF
cat > tokens.json <<'EOF'
{ "paper": "#F5F0E6", "ink": "#181510", "accent": "#E8501F" }
EOF
cat > consumers.json <<'EOF'
[
  { "id": "site",   "root": "consumers/site",   "path": "brand.css" },
  { "id": "report", "root": "consumers/report", "path": "brand.css" }
]
EOF
mkdir -p consumers/site/.github/workflows consumers/report/.github/workflows

# site: region already byte-correct, but its CI pin is two releases old
cat > consumers/site/brand.css <<'EOF'
/* >>> brand v0.2.0 GENERATED, do not edit */
  --paper: #F5F0E6;
  --ink: #181510;
  --accent: #E8501F;
/* <<< brand */

.masthead { color: var(--ink); }
EOF
echo "      - run: npx -y @acme/[email protected] check" > consumers/site/.github/workflows/ci.yml

# report: pinned at the current release, but carrying an accent that moved
cat > consumers/report/brand.css <<'EOF'
/* >>> brand v0.4.0 GENERATED, do not edit */
  --paper: #F5F0E6;
  --ink: #181510;
  --accent: #FF6B35;
/* <<< brand */

.figure { border-color: var(--accent); }
EOF
echo "      - run: npx -y @acme/[email protected] check" > consumers/report/.github/workflows/ci.yml

node propagate.mjs --dry-run
    site       wrote-by 0.2.0   pin was behind, no rendered change -> bumped quietly
PR  report     wrote-by 0.4.0   BRAND VALUES CHANGED -> open a pull request

2 consumers, 1 pull request (dry run)

That is the payoff, and it inverts the obvious reading. site looks neglected: a pin two releases old and a region stamped by 0.2.0. It needs no review, because its bytes are already right. report looks impeccable: pinned at the current release, region stamped with the current version. It’s the one shipping the wrong color. Version numbers describe what a repo ran, not what it has, and only re-deriving the content tells you which.

Apply it, then run again:

node propagate.mjs
echo
cat consumers/report/brand.css
cat consumers/site/.github/workflows/ci.yml
echo
node propagate.mjs
    site       wrote-by 0.2.0   pin was behind, no rendered change -> bumped quietly
PR  report     wrote-by 0.4.0   BRAND VALUES CHANGED -> open a pull request

2 consumers, 1 pull request

/* >>> brand v0.4.0 GENERATED, do not edit */
  --paper: #F5F0E6;
  --ink: #181510;
  --accent: #E8501F;
/* <<< brand */

.figure { border-color: var(--accent); }
      - run: npx -y @acme/[email protected] check

    site       wrote-by 0.2.0   already current
    report     wrote-by 0.4.0   already current

2 consumers, 0 pull requests

Both are current on the second pass, and site still reports wrote-by 0.2.0. That’s rule one behaving correctly rather than a bug: its content never changed, so its marker was never rewritten, and a marker that lags reality is not itself a reason to touch a file.

Wire it to the release

The fan-out belongs on the release, plus a schedule as a safety net for consumers whose pull requests were closed unmerged.

name: propagate
on:
  workflow_call:        # called by the release job, in the same run
  schedule:
    - cron: "0 13 * * 1"
  workflow_dispatch:

jobs:
  fan-out:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: node propagate.mjs --dry-run

Note what is not there: a push: tags: trigger. That is the subject of the first gotcha, and it is the one that cost a whole release.

Gotchas

A tag pushed by your own release workflow starts nothing. The natural trigger for a fan-out is the release tag, so on: push: tags: ["v*"] is what you write. GitHub’s docs are explicit that this cannot work when the tag comes from CI: “When you use the repository’s GITHUB_TOKEN to perform tasks, events triggered by the GITHUB_TOKEN will not create a new workflow run”, with only workflow_dispatch and repository_dispatch exempted. Symptom: the trigger reads correctly, the release publishes, and the fan-out runs exactly zero times with no error anywhere, because a workflow that never started has no failed run to look at. press shipped this in 0.4.0 and confirmed it on that release, which tagged and published with no propagate run at all. The escape is either to call the workflow from the release job in the same run, as the workflow_call above does, or to use a GitHub App installation token or a personal access token, which the same docs recommend for exactly this.

Deriving a boolean from a human-readable status string. It is tempting to compute the status once and infer everything else from it. press wrote /updated$/ against a status that could read "updated" or "would update", and the two differ by more than tense:

true  "updated"
false "would update"

Symptom: --dry-run printed a changed region on one line and “nothing to do” on the next, which reads as a display bug rather than the state error it is. The escape is to carry changed as a boolean from the one place that actually compared the bytes, and let the display string derive from it rather than the reverse.

Open the pull request against the branch the consumer integrates on. The default branch is the obvious target and is frequently wrong. Every consumer here runs feature → dev → main, so a PR opened straight into main violates the flow it is supposed to respect. Worse, the default branch can legitimately lag: press’s first real fan-out reported two consumers as missing a region entirely, only because their migration was still sitting on dev. A shallow clone makes this invisible, since --depth=1 cannot see other branches at all. The escape is to check out the integration branch when the remote has one, fall back to the default branch otherwise, and clone deep enough to look.

A bot that opens pull requests has to close its own. Each release opening a fresh PR per consumer sounds harmless until one repo goes two releases without merging. Symptom: stacked pull requests, where the older ones fail by construction, since an older PR’s pin predates the targets the newer release declares, so its check is being asked about regions that did not exist when it was written. The reviewer sees red on a bot PR and stops trusting bot PRs. The escape is to make opening a new one close any older one it supersedes, with a comment saying what replaced it.

Sources

Changelog

  • feat(press): push brand updates to every consumer (0.4.0) (#121) (03af5ef)