Everything I built to catch leaks, leaked
2026-09-23. A day spent building a publishing pipeline, and a longer one spent finding out what it did not catch.
Background: the hallucination log for the running record of this class of mistake, and the companion entry on the dashboard built over the same gates.
The job was supposed to be presentation
The project has 96 markdown documents outside the archives, about 283,000 words, and 206 links between them. It also had no front door. Everything lived in a private repository, readable as raw markdown on a page with no search, no navigation and no way to hover a term and see what it means.
So: a static site generator over docs/, rendering in place, nothing migrated.
Quartz v5, publication opt-in per file via frontmatter. An afternoon’s work.
The interesting part was never the site. It was that these documents are about a real person’s cancer, and roughly a fifth of them say so in detail.
The first gate, and why it was not enough
Publication is opt-in: a page ships only if its frontmatter says publish: true.
An allowlist, because a denylist fails open and the failure mode here is
publishing a medical record.
Then a second gate, deliberately built on different logic: a scanner that ignores what the frontmatter claims and reads the content. That asymmetry is the whole point — the first gate shares its assumptions with whoever wrote the frontmatter, and this repository has an expensive precedent for self-consistent checks that cannot fail.
It caught a researcher’s email address on its first run, transcribed into an internal note months ago. Good start.
Then I built the site and looked at what it had actually emitted.
What a markdown gate cannot see
Quartz copies every non-markdown file under the content root to the output verbatim. Gate or no gate. The first build shipped an unpublished draft, an EHR OAuth test matrix, a personal fitness-data census, extracted text from copyrighted papers, and a scatter plot of the patient’s measured drug sensitivities.
None of them are markdown, so no frontmatter gate would ever have seen them.
That is the moment the design changed. The authoritative check moved from the sources to the built artifact — walk what is about to ship and fail on anything unrecognised. Denylists fail open; the staging tree became an allowlist by construction, where a file is present only because something deliberately copied it there.
Three more of the same shape
The pattern repeated all day, and I did not see it as a pattern until late.
The audit passed an empty build. It checked only that nothing unexpected shipped. A site with zero pages satisfies that perfectly: nothing unexpected, nothing ungated, nothing dead, because nothing at all. It now asserts that every cleared page was actually emitted.
The scanner reported clean on a file it never opened. scan_doc returned
early when a document was unpublished, before reading the body. So --explain on
the flagship document printed “no identifier hits” for a file it had not read.
Richard ran it, saw the clean result, and asked why it disagreed with what I had
told him.
My own verification was a broken command. I had been confirming leak-freedom
with grep -rniF. In this environment that combination of flags silently matches
nothing — for every pattern. I ran it against a string I knew was present, and it
returned zero. Every “verified 0 occurrences across all 9 pages” I had reported
was a command that never ran. The claims happened to be true, which is worse:
they would have been believed the next time too.
The thread connecting all four: absence of findings and absence of looking render identically unless you make them different. A gate has to assert positively — what did ship, what was read — and carry a control string that is known to be present, so a scan matching nothing reports unknown rather than clean.
The line that actually mattered
Underneath the tooling was a decision I got wrong in both directions before getting it right.
I started too broad, treating everything clinical as untouchable. Richard pushed back, and was right on three counts: the regulatory framing I had reached for does not apply to him; somatic mutations are tumour-only, not heritable, not usable in underwriting, and are the scientifically useful part; and a genotype does not tell anyone where you live, which is the threat he actually cares about.
What survives is narrower and sharper. An inherited variant is not one person’s data. Each child has a fifty per cent chance of carrying it, as do siblings. The exposure is not health insurance — the relevant US statute covers health insurance and employment and explicitly does not cover life, disability or long-term-care underwriting. For children who did not consent, and in some cases cannot, it would be the only adverse genetic fact on record. And publication is irrevocable.
So the rule became: publish nearly all of it, generalise exactly one class of fact, and do it at build time so the source documents keep full fidelity. The technical arguments survive generalisation completely — the passage explaining that a position-keyed builder recovers a variant an rsID-keyed path dropped makes exactly as much sense without the gene, the coordinate and the zygosity.
Then I found that rule had been failing quietly all afternoon.
One specificity level at a time
The germline rules were correct four separate times, and each correct version left the next gap open.
First they matched the coordinate and the rsID. Then a document stated the genotype in plain English, which discloses precisely what the VCF notation does, and matched nothing. Then a log named the gene with neither coordinate nor zygosity — the gene and nothing else — and matched nothing again. Then my own redaction, written to fix the first case, still asserted “the patient’s germline pathogenic variant”, and matched nothing.
Every one of those was live on the public site before it was caught. One of them went out with the first deploy, and the reason nothing stopped it is instructive: the germline rules were warnings, warnings do not block, and the artifact scan only failed on blocking rules. Every layer had the same blind spot as the layer above it.
The structural fix separates two bars that I had been treating as one. Severity governs the source gate, where full fidelity is correct and a warning means a human should look. The artifact has a harder bar: some findings may not reach built HTML whatever the source says. Applying that immediately caught two more, including my own redaction.
And a person who does not exist
Late in the day Richard asked, roughly: who is this person?
An earlier session had read his GitHub handle as an initial plus a surname, invented a first name to fit, and signed a document with it. He spotted that and asked whether there were more. There were: the same invention had produced a repository URL built from the same invention, cited twice as a source in the same document. It 404s. It was live.
It escaped for two reasons that are worth keeping. Word-boundary anchors do not exist inside a concatenated handle, so the rule looking for the name could not see it. And the content scan stripped HTML tags before searching, so it never inspected a link target at all. A URL is content. I had not been reading it.
Where it ended
The site is private now, and verified private — Pages disabled first, then the repository flipped, then checked anonymously rather than assumed, because whether a private repository keeps serving its pages depends on the account plan and “private repo, still-live site” is the worst outcome available: you stop worrying.
Sixteen pages built and waiting. Richard wants to go through them one at a time before any of it is public again, which after today seems exactly right.
The thing I want to remember is not any individual bug. It is that I kept fixing the disclosure I had just been shown and calling the rule complete. The correct posture is to assume there is another level below the one I just closed, and to trust a scan of the artifact over my own confidence about the rules.