Nate Swenson

Lane two waited on an edge that was never there

Issue Flow · No. 116 · 9 min read ·v0.2.0

Shipped

issueflow splits an issue too large for one change into stacked lanes, each with its own branch, its own implement and test stages, and its own pull request. 0.2.0 was written against a measured run of the previous release: 57 minutes end to end, of which 23 were subagent time and 31 were me reading artifacts. The release added checkpointing to the issue, reconciliation with GitHub before every advance, a git worktree per lane, and stage durations.

It also fixed the reason that run only ever had one lane moving. The gate asked whether every step listed above this one was approved, and lane two’s implement sits below lane one’s test in that list. So lane two waited for lane one’s tests, a dependency that does not exist in any form. This guide is about the difference between those two questions, and how to make a scheduler answer the right one.

Ordering by position is not ordering by need

Any pipeline with stages eventually grows a list, and the list is almost always in a sensible order. The temptation is to let the list be the schedule: a step may run when everything before it is done. That is one line of code and it is correct for exactly as long as the work is a single file.

Build systems solved this a long time ago by refusing to conflate the two. Build Systems à la Carte describes Make’s approach plainly: it “constructs a task dependency graph from the information contained in the makefile and executes tasks in a topological order”. The makefile’s line order has nothing to do with it. A prerequisite in Make is a file that is used as input to create the target, and the graph of those inputs, not the file’s layout, is what decides both the order and what may run at the same time.

Agent pipelines have the same shape and a higher price for getting it wrong, because each step is a model call that costs money and minutes. Anthropic’s agent guide calls the pattern sectioning, “breaking a task into independent subtasks run in parallel”, and notes parallelization is effective “when the divided subtasks can be parallelized for speed”. A false edge in your graph quietly deletes that entire capability, and it does it without an error message. The pipeline still works. It is just serial.

Model the run, then declare the edges

Two shared stages that are about the issue, then one lane per work item, each lane stacked on the one below it.

// graph.mjs
export const createRun = (lanes) => ({
  shared: [
    { key: 'investigate', state: 'pending' },
    { key: 'design', state: 'pending' },
  ],
  lanes: lanes.map((l) => ({
    slug: l.slug,
    base: l.base, // the branch this lane stacks on, or null for the bottom lane
    steps: [
      { key: `${l.slug}/implement`, state: 'pending' },
      { key: `${l.slug}/test`, state: 'pending' },
    ],
  })),
});

// Every step in board order: shared first, then each lane in landing order.
// This is a display order. It is NOT the dependency order, which is the whole
// point of this file.
export const gateSteps = (run) => [...run.shared, ...run.lanes.flatMap((l) => l.steps)];

const at = (run, key) => gateSteps(run).find((s) => s.key === key) ?? null;
const laneOf = (run, step) => run.lanes.find((l) => l.steps.includes(step)) ?? null;

Keeping gateSteps and calling it a display order in a comment is worth the two seconds. The board renders in that order and humans read it in that order; the mistake is only in letting it also schedule.

Now the edges, written out one at a time:

// graph.mjs, continued
//   investigate     <- nothing
//   design          <- investigate
//   lane.implement  <- design, plus the implement of the lane it stacks on
//   lane.test       <- that same lane's implement
//
// The stacked-parent edge is real: a lane branches off the branch below it, so
// its commits cannot exist until that branch does. What is NOT real is waiting
// for the parent lane to have been tested.
export function dependencies(run, step) {
  const out = [];
  const lane = laneOf(run, step);

  if (step.key === 'design') out.push(at(run, 'investigate'));

  if (step.key.endsWith('/implement')) {
    out.push(at(run, 'design'));
    const parent = run.lanes.find((l) => l.slug === lane.base);
    if (parent) out.push(at(run, `${parent.slug}/implement`));
  }

  if (step.key.endsWith('/test')) out.push(at(run, `${lane.slug}/implement`));

  return out.filter(Boolean);
}

Write these as prose first and translate them one line at a time. Every edge you can defend in a sentence is real; every edge you cannot is the one costing you an hour. The one that survives scrutiny here is the stacked-parent edge, because lane two’s branch is cut from lane one’s branch and cannot hold commits before it exists. Lane one being correct is a different claim, and nothing about lane two’s code depends on it.

Walk it transitively

// graph.mjs, continued
// Walks the graph transitively, so a hole two levels down is still named rather
// than hidden behind an intermediate step that looks approved. Reported in board
// order, because that is the order a reader fixes them in.
export function blockers(run, step) {
  const seen = new Set([step.key]);
  const found = [];
  const walk = (from) => {
    for (const dep of dependencies(run, from)) {
      if (seen.has(dep.key)) continue;
      seen.add(dep.key);
      if (dep.state === 'approved') continue;
      found.push(dep);
      walk(dep);
    }
  };
  walk(step);
  const order = gateSteps(run).map((s) => s.key);
  return found.sort((a, b) => order.indexOf(a.key) - order.indexOf(b.key));
}

// The version this replaced: everything listed above it in board order.
export function flatBlockers(run, step) {
  const steps = gateSteps(run);
  const index = steps.findIndex((s) => s.key === step.key);
  return steps.slice(0, index).filter((s) => s.state !== 'approved');
}

// Every step that could be dispatched right now: the fan-out set. More than one
// entry means those stages are genuinely independent and should go out together.
export const readySteps = (run) =>
  gateSteps(run).filter(
    (s) => s.state !== 'approved' && s.state !== 'skipped' && blockers(run, s).length === 0,
  );

Three details in blockers earn their keep. The seen set stops a diamond reporting the same missing step twice. Not recursing past an approved dependency is safe because a step cannot be approved unless its own dependencies were, so approval is a proof about the whole subtree beneath it. And sorting back into board order means the error message lists holes in the sequence a human would fix them, rather than in whatever order the walk happened to find them.

readySteps is the part that turns a correct graph into saved time. A graph nothing queries still runs one step at a time, because the orchestrator has no way to learn that two things are dispatchable. Give it a command that says so.

Run it

// demo.mjs
import { createRun, gateSteps, blockers, flatBlockers, readySteps } from './graph.mjs';

const run = createRun([
  { slug: 'lane-a', base: null },
  { slug: 'lane-b', base: 'lane-a' },
]);

const approve = (key) => {
  gateSteps(run).find((s) => s.key === key).state = 'approved';
};

// Where the measured run actually stood: the design is in, lane-a is built, and
// lane-a's tests are still sitting at the gate awaiting a human.
approve('investigate');
approve('design');
approve('lane-a/implement');

const step = gateSteps(run).find((s) => s.key === 'lane-b/implement');
const names = (list) => (list.length ? list.map((s) => s.key).join(', ') : '(none)');

console.log('state:');
for (const s of gateSteps(run)) console.log(`  ${s.key.padEnd(18)} ${s.state}`);

console.log('\nlane-b/implement');
console.log(`  flat prefix scan blocks on : ${names(flatBlockers(run, step))}`);
console.log(`  dependency graph blocks on : ${names(blockers(run, step))}`);

console.log(`\nready to dispatch now: ${names(readySteps(run))}`);

// The stacked edge is still enforced. Rewind lane-a's implement and lane-b must
// wait, because its branch cannot exist yet.
gateSteps(run).find((s) => s.key === 'lane-a/implement').state = 'pending';
console.log(`\nwith lane-a/implement rewound, lane-b/implement blocks on: ${names(blockers(run, step))}`);

node demo.mjs, and this is the real output:

state:
  investigate        approved
  design             approved
  lane-a/implement   approved
  lane-a/test        pending
  lane-b/implement   pending
  lane-b/test        pending

lane-b/implement
  flat prefix scan blocks on : lane-a/test
  dependency graph blocks on : (none)

ready to dispatch now: lane-a/test, lane-b/implement

with lane-a/implement rewound, lane-b/implement blocks on: lane-a/implement

The same run, scored two ways. The old gate names lane-a/test, and there is no argument to be had with it, because a human really did have to approve that before anything else moved. The graph names nothing, and readySteps hands back two steps to dispatch together.

The last line is the half people skip. A scheduler that unblocks everything is not a fixed scheduler, it is a deleted one, so prove the real edge still bites before you believe the loosened one.

Gotchas

Fixing the graph is what creates the concurrency bug. While lanes ran one at a time, sharing a checkout was free. The moment two lanes are genuinely dispatchable, two subagents are editing one working tree, and the test stage’s revert-and-rerun proof is reverting files another lane is mid-change on. That proof had been running in the live tree all along, in a repo whose own contributor guide warns that parallel sessions hold uncommitted work there; serial execution was hiding it. The escape is a git worktree per lane, added in the same release as the graph, and the ordering matters: ship the isolation with the parallelism, never after it.

The gate’s latency is a window, and the world moves inside it. On the measured run the change was merged and the issue closed while the run sat at the implement gate. Four minutes later the run approved that stage and dispatched a subagent against a branch that no longer mattered. Nothing was wrong with the state machine; its state was simply stale, because approval had been treated as a fact about the run rather than a fact about the world at the moment it was given. The escape is to re-ask the remote what is true immediately before every advance and refuse on a closed issue or an already-merged pull request. Human review time is exactly the interval in which someone else finishes your work.

A slug truncated at a fixed length cuts mid-word. A real run produced the branch feature/issue-173-shipflow-refuses-the-ambiguous-f. It is harmless and it looks broken, which is its own cost when the name appears in every pull request. Truncate on a word boundary instead, and do it in the function that builds the name rather than at each call site.

Check what your automation writes into public places. The pull request body carried an absolute local path, /Users/<someone>/.claude/issueflow/…, into every pull request the skill opened. Run directories are a natural thing to reference while debugging and a strange thing to publish. Grep your emitted templates for homedir() and absolute paths before the first real run, not after.

Sources

Changelog

  • feat(issueflow): checkpoint every gate to GitHub, and let independent lanes run at once (0.2.0) (#177) (8408c93)