The rule generator almost binned the mail a rule was protecting

Shipped

Three versions of my Gmail triage skill landed on main together: sub-labels with a retroactive pass over already-filed mail, an audit command that grades whether a label system is still coherent, and the fix this post is about. The propose command, which clusters bulk senders and suggests new filing rules, now reads the user’s existing rule file first and drops every sender those rules already claim. On my real mailbox, the version without that filter proposed trashing a sender whose mail an existing rule was deliberately keeping.

The proposal that contradicted a decision

propose looks at a sample of threads, clusters them by sender, and suggests rules for the senders that show up in bulk: file this newsletter, trash that notifier. It took the thread sample, a minimum cluster size, and the label list. It did not take the rule set. It had no idea what the user had already decided.

On a live run, that blind spot produced this: an existing sort rule was filing [email protected], an e-signature service, into the folder for my home sale. Those threads matched the rule, left the inbox, and looked exactly like handled bulk mail. propose clustered the leftover volume and suggested a trash rule for the sender. Accepting it would have put a trash rule into the same rule set as a keep rule, and every future signing document would have been binned. The summary table would have looked completely correct while it happened.

The near-miss reframed the problem for me. I had thought of duplicate proposals as the failure mode, and duplicates are merely annoying: a proposed rule that repeats an existing one changes nothing. Firewall policy analysis makes this distinction precisely. In the classic taxonomy from Al-Shaer and Hamed’s work on policy anomalies, overlapping rules with the same action are redundancy; overlapping rules with different actions are conflicts, the class that includes shadowing, where one rule makes another unreachable. A proposal generator that cannot see the existing rules will eventually generate both kinds, and only one of them destroys mail.

Subtract the claimed set before clustering

The fix is a filter in front of the clustering, built from pieces the codebase already had. First, propose loads the same rule file every other command uses:

import { readFileSync } from "node:fs";

export function loadRules(path) {
  const { rules } = JSON.parse(readFileSync(path, "utf8"));
  return rules; // [{ id, match: { from }, action, label }]
}

Then it computes which senders are claimed (matches(thread, rule, opts) is your rule engine’s existing predicate, and cluster is whatever sender-grouping step your propose already has). The granularity decision is the important one: claimed is judged per sender, not per thread. A rule that matches only some of a sender’s mail still proves the user has made a decision about that sender, and the unmatched leftovers are exactly the threads that would otherwise cluster into a contradicting rule:

export function claimedSenders(threads, rules) {
  const claimed = new Map(); // sender -> rule id that claims it
  for (const t of threads) {
    for (const rule of rules) {
      if (matches(t, rule, { ignoreFiled: true })) {
        claimed.set(t.sender, rule.id);
        break;
      }
    }
  }
  return claimed;
}

export function propose(threads, rules, { minCount }) {
  const claimed = claimedSenders(threads, rules);
  const candidates = threads.filter((t) => !claimed.has(t.sender));
  const excluded = threads.filter((t) => claimed.has(t.sender));
  return { clusters: cluster(candidates, minCount), claimed, excluded };
}

That ignoreFiled: true option carries its own lesson. The default matches answers the question “is there work to do”, and says no for a thread already sitting in the folder its rule files into. That is the right answer for planning and exactly the wrong one here: an already-filed thread is the most claimed thread in the mailbox. My audit command had hit the same trap one version earlier, reporting 47 of 48 correctly-filed threads as unclaimed because it reused the planner’s question. Same predicate, two questions; the option makes the second one askable.

Report what you excluded, and rank the reasons

A filter that silently shrinks the candidate table has a UX bug: mail disappearing from a report with no stated cause reads as mail gone missing. So the exclusions print as their own table, naming each dropped sender, its thread count, and the rule that claims it. From my frozen baseline run (the corpus is invented, every domain reserved):

| Already claimed                         | Threads | By rule              |
|-----------------------------------------|---------|----------------------|
| [email protected]                | 9       | trash-packages       |
| [email protected]      | 4       | sort-initech         |
| [email protected]               | 4       | sort-paperwork       |
| [email protected]                | 4       | trash-leadgen        |

The empty case needs its own words too. Before this release, a sample where every sender was already claimed fell through to the generic “no bulk mail in the sample at all. Widen the fetch”, sending the user off to re-fetch a mailbox that was working perfectly. Outcome reasons are ranked, and full coverage outranks emptiness:

export function proposeReason({ clusters, excluded }, sampleSize) {
  if (clusters.length > 0) return { kind: "candidates" };
  if (excluded.length === sampleSize && sampleSize > 0)
    return { kind: "all-claimed" };   // your rules cover this sample; nothing to add
  return { kind: "no-bulk" };          // genuinely thin sample; widen the fetch
}

Verify it with a pair of runs

One fixture, two rule sets. The pair is what proves the filter fires on coverage rather than on the fixture, and I froze exactly this pair as the skill’s baseline. The run with an explicitly empty rule set proposes 12 rules over the corpus (5 trash, 7 sort). The run with the corpus’s full rule set proposes nothing, and says why, from the frozen output:

66 of 66 thread(s) are already claimed by 31 sender(s) your rules cover
— excluded from everything below, so a rule you already wrote is never
re-proposed or contradicted.

TRASH — bulk mail a rule would move to the trash

No trash candidates — every sender in the sample is already claimed by
one of your 24 rules. Nothing new has arrived — that is what a covered
mailbox looks like.

Same input, opposite rule knowledge, opposite output, and the covered run names its reason instead of telling you to widen the fetch. Your version needs the same pair: if both runs propose the same candidates, the filter is reading the fixture, not the rules.

Gotchas

• The default rules path poisons hermetic tests. propose defaults to the user’s real rule file in their home directory, which is right interactively and wrong in a frozen baseline: the fixture would read the personal rules of whoever refreshes it. A first run genuinely has no rules, so the fixture passes --rules rules-none.json explicitly and says so out loud, instead of relying on a file happening to be absent.
• Per-thread claiming reopens the hole. If claimed were judged thread by thread, a sender with a partial-match rule splits: matched threads excluded, unmatched threads clustered, and the contradiction gets proposed anyway from the leftovers. Sender granularity closes it, at the acceptable cost of never proposing a second rule for an already-managed sender.
• Reusing a predicate reuses its question. Both audit and propose broke by borrowing matches from the planner, inheriting “does this thread need work” when they needed “has the user decided about this”. If a helper’s answer encodes a purpose, borrowing it across purposes is where the wrong answers come from; the ignoreFiled option exists to make the purpose explicit at the call site.
• Gmail’s model makes ordering mistakes quiet. A Gmail filter applies all criteria to individual messages, one user label per filter, and nothing in the API objects when two filters disagree about a sender; both simply run. Whatever safety you want has to live in the layer that writes the rules, because the platform will execute the contradiction without comment.

Sources

• Discovery of policy anomalies in distributed firewalls — Al-Shaer and Hamed’s classification of rule-set anomalies (INFOCOM 2004)
• Ant colony optimization-based firewall anomaly mitigation engine — working definitions of shadowing, redundancy, correlation and generalization, and why different-action overlaps are the conflicts
• Gmail API: Managing filters — filter criteria and actions, and the constraints the rule-writing layer has to compensate for

Changelog

• fix(gmailtriage): propose must not contradict a rule the user already wrote (0.5.0) (#199) (190d286)
• security(gmailtriage): genericise the last employer initialism in comments and tests (2aa0bee)
• security(gmailtriage): scrub the mailbox owner’s real addresses from source and docs too (6ccab3e)
• test(gmailtriage): restore the baseline corpus, invented rather than redacted (e67d1fb)
• feat(gmailtriage): a label system that stays clean, and a run that notices what is new (0.4.0) (#193) (6656758)
• feat(gmailtriage): sub-labels, and a retroactive pass over mail already filed (0.3.0) (#192) (9d58252)