The question was whether a column called hm_source can tell us which DNA strand a scoring file’s alleles were written on. If it could, about 45 % of the variants we currently refuse to score would become scoreable. If it can’t, they stay refused and the next gate has to plan around a permanent hole.

The issue laid out an experiment to settle it, and the experiment was sound. Take the 167 scoring files that declare their own allele frequencies, orient their ambiguous variants two ways — once by comparing the declared frequency against a reference panel, once by assuming the column already fixed it — and measure how often the two agree. Ground truth from data we already hold.

Except we don’t hold it. The panel comparison is the next gate’s entire job; the 15 GB archive is extracted on disk and nothing has read a genotype out of it yet. Building that path to run the experiment would have been a day, and it would have been the next gate’s day spent early and badly.

So before doing that I went to read what actually writes the column.

Ten minutes

The PGS Catalog’s harmonizer is public. It runs in two passes. The first, HmPOS, works out where each variant sits on the target genome build. The second, HmVCF, looks the variant up in a reference VCF and checks its alleles — matches, reverse-complement, ambiguous.

hm_source is written by the first one. Its function returns five values and none of them is an allele; the effect allele goes in and comes out untouched. The Catalog’s own column documentation says the same thing in six words — data source of the variant position.

That is the whole answer. hm_source = ENSEMBL means the coordinate came from an rsID lookup. It has never made a claim about alleles, so there is nothing to trust or distrust, and the hypothesis was not wrong so much as malformed.

Two more things fell out of the same reading, and both foreclose a follow-up someone would otherwise propose. The second pass does have a code for ambiguous variants — and beside it in the source sits an unimplemented note to resolve them by allele frequency one day. The Catalog has the same open problem we do. And the files that pass carries are not published: the FTP serves two flavours per score and both are first-pass output. So hm_code is not available to anybody, and running the second pass ourselves would return “ambiguous” for exactly the rows we wanted resolved.

The part I nearly shipped as the answer

A code reading is an argument about what should be in the data. I wanted the data.

The cheap version took six seconds: scan all 5,377 cached scoring files for any column or header key the second pass leaves behind. Zero. Not one file we have ever scored has been through the stage that looks at an allele. Then nine files downloaded in their original author-submitted form and diffed row by row against our harmonized copies — 401 rows, three different hm_source values, zero allele differences. Coordinates moved by megabases; every allele letter identical.

At that point I had the answer three ways and could have stopped. What I did instead was ask a question the issue hadn’t: never mind whether the column certifies forward strand, are these files forward strand anyway? Because if they are, the refusal is over-caution rather than necessity.

That question had a free answer sitting in the last fleet run. When a variant is not ambiguous, we can check its strand against the reference genome, and we already count the rows that would only have matched reverse-complemented. Fleet-wide that count is 0.024 % — reassuringly tiny. Per score it is not tiny at all. Thirty-seven scores sit above 5 %. Six sit near 50 %.

Fifty per cent is not noise. Fifty per cent is a coin.

One file, 53,705 rows, not a single C/T

I pulled the worst one and looked at its allele pairs directly against the pinned FASTA. Every row is A/G or A/C. Out of 53,705 rows there is not one C/T and not one G/T.

On the forward strand those are the same variants seen from the two sides and they occur about equally often — three normal files I checked came in at 0.4991, 0.5017, 0.5018. A file that is all A/x was written in a convention that always reports the A-bearing strand, which means its strand assignment is per variant and is recorded nowhere. Half its rows are reverse-complemented and nothing in the file says which half.

Then the line that made the whole day worth it. Split those rows by hm_source:

• 26,901 rows labelled ENSEMBL, forward strand.
• 26,801 rows labelled ENSEMBL, reverse strand.

Same label. Same file. Opposite strands, split down the middle.

That is the refutation I would have wanted from the start, and it needed no reference panel and no ground truth — just the reference genome and a column I’d been treating as uninteresting. Three other files in the same shape carry ambiguous variants that are all ENSEMBL-sourced. Under the hypothesis those were the safe ones to orient. They are the most dangerous rows in the fleet.

The detector I nearly shipped was wrong

I liked the all-A/x test enough to write it into a recommendation: flag any file whose unambiguous variants are all A/G or A/C, and refuse to orient its ambiguous ones. Then, before writing it down as settled, I ran it over the full fleet and looked at what it caught.

Three of the six it flagged have a zero per cent reverse-strand rate. I pulled one apart: 5,326 of 5,326 checkable rows have their non-effect allele equal to the reference base. Its alleles genuinely are the forward-strand pair. The all-A/x shape describes what variants that score contains, not a relabelling of them, and its ambiguous rows are no worse off than anyone’s.

So the shape names a convention; it does not prove a file is mixed. What proves it is rows that only match complemented — which we already measure against the pinned genome and store on every score. The right rule needs both signals, and the one I nearly shipped would have refused three innocent files to guard against three guilty ones.

Two of the guilty ones carry six ambiguous rows each.

The honest size of it

Which brings me to the number I had been quietly inflating. Under the corrected rule, three of the 1,687 affected scores are strand-mixed, and they carry 21 ambiguous rows between them. Loosen it to any file above a 5 % reverse rate and it is 18 scores and 62 rows — 0.0001 % of the 64.8 million we defer.

The finding changes the arithmetic by nothing. It changes the warrant. “They’re probably forward strand” cannot be defended as empirically safe once you can name the files where it is false, and being right 99.996 % of the time with no way to tell which case you are in is a different object from being right.

There is a reason the exposure is small, and it is not luck. The big file — 53,705 rows, half of them reversed — has no ambiguous variants at all. A submitter who knows their data is in A/B coding also knows the ambiguous ones are unrecoverable and drops them. The files that kept theirs are the ones that did not know.

What it cost to get there

Three things I’d do differently.

I launched the 645-million-row fleet scan, watched it for ten minutes, and then realised I needed one more field per file — the allele-pair composition that turned into the detector above. Killing and relaunching cost twelve minutes, which is cheap only because I noticed early. The real lesson is that I decided the output shape before I knew what question the output was for. A 10 GB pass deserves five minutes of asking what else it should carry while it is reading anyway.

I spent the first part of the session planning how to build a panel-frequency path because the issue described an experiment that needed one. The issue was right about what would settle the hypothesis if the hypothesis were the kind of thing data settles. It wasn’t. Ten minutes with the pipeline that writes the field would have saved that, and “read the thing that produces this” should come before “design the experiment that tests it” every time the field has a producer I can read.

And the detector. I had three confirming examples and a mechanism that explained them, and that is exactly the state in which a rule feels finished. It wasn’t finished; it was untested against the population it would run on. Running it over all 1,687 files cost one command and turned a recommendation I would have had to retract into one I can defend.

Where it leaves the gate

The ambiguous variants stay deferred. The frequency route survives and is worth taking for the 167 files that have frequencies — priced at 8.48 % of the deferred rows once you exclude frequencies too close to 0.5 to decide anything. Everything else becomes a coverage number to gate on rather than a decision still pending, which is a better place to be than a maybe.

One useful thing came out of it that nobody asked for. The strand-mixed files are also quietly losing half their ordinary rows to a mismatch reason that reads like the score and the genome disagree about the variant, when the real cause is a known convention. Across the fleet a per-file, evidence-gated flip would recover 56,411 rows in 37 scores (the final rule, which needs both signals, recovers 26,885 rows in 5 scores; see the PRS primer’s §5.4). That is coverage, not correctness, and it is not this gate’s problem — but it should not stay invisible either.

And a smaller one: the fleet distribution the issue asked for came back at 21.7 % for the value its 150-file sample had put at 44.2 %. Both are correct. The sample capped each file at 300 rows, which quietly made it file-weighted — and on the file measure it was nearly exact, 56 % against the fleet’s 58.3 %. Row-weighted and file-weighted answer different questions about the same table, which is a trap this project has now walked into from three directions.

The calculator the Catalog ships drops ambiguous variants by default and documents its only alternative as an assumption. We arrived at the same refusal independently. It is a strange kind of reassurance to find that the field’s answer to your open problem is the same shrug, but it beats being the only one shrugging.

---

Later the same day: building the guard I had just argued for

The spike merged, and the two follow-ups I had filed came straight back as work. They are one finding split in two — make the A/B allele convention visible per score, then make the rows it loses recoverable — and the second depends on the first.

The design question was where the evidence lives

My first instinct was a boolean on the manifest: strand_mixed, computed at classify time. That is wrong for a reason the morning had already taught me: the classifier reads the scoring file and nothing else, so it can see the allele convention but cannot see whether rows actually fail against the genome. Half the evidence is in a different gate.

So the manifest stores counts — n_snv_pair_top, n_snv_pair_bot — and never the verdict. The ratio is derived on read, which is the rule the coverage payload already follows, and the verdict is assembled in the service by joining those counts to the complement-only rate the scorer measured. Two gates, one conclusion, neither able to reach it alone.

That also gave the honest answer for a score whose manifest row predates the change: strand_mixed is null, not false. The panel says “not measured” in the slot where a number would go. A file we have not looked at is not a file we have cleared.

The bit I keep having to re-learn

I wrote the recovery as a set of score ids on the policy rather than a second boolean, and I had to talk myself into the extra awkwardness. The argument that settled it: the existing global flag is an assumption about the entire catalog, and this is a finding about four named files. If both were booleans, a run’s provenance would render them identically, and six months from now nobody could tell which one a number rested on. So the ids go in the provenance, and the set is small by construction.

It is off by default, which makes the feature look like a no-op. It is not — turning it on means re-scoring, and re-scoring under a changed policy is exactly the mixed-fleet hazard that nearly bit the last fleet run. The switch belongs to whoever runs that, not to me.

Two tests broke, and both were supposed to

The classifier version is pinned by a test. I bumped it to v5 and the test failed, which is its entire job: the manifest build skips rows already at the target version, so a silent version change would leave the fleet on the old meaning forever.

The second one I did not see locally. The invariant count is pinned too — I added a tenth and a test asserting exactly nine went red in CI. It went red in CI rather than on my machine because the local test database has started rejecting its own credentials, so every DB-backed test errors there. That is worth saying plainly instead of reporting a clean run: I ran 124 unit tests, not the 153 I would have liked to.

Then I noticed the new invariant would never actually fire in any test, which is the failure mode I had written a memory about that morning. So it now has a test that flips a fixture row to v5 with null counters, asserts that specific check fails, fills the counters in, and asserts health goes green — testing “v5 without counters”, not “v5”.

Verifying the panel cost more than writing it

No node_modules anywhere in the checkout, and the backend already running on :8000 serves the main checkout, so it has none of this. I stood up a second stack on :8011/:5173 from temporary launch entries.

It died immediately. python main.py on Windows exits before serving because run_migrations() prints a ✓ into a cp1252 console — and the error handler then prints a ❌ and raises again, so the traceback buries the real cause. PYTHONIOENCODING=utf-8 fixed it. Then the frontend got “Failed to fetch”: port 3002 is not in the backend’s CORS allow-list. Moved it to 5173, which is.

Screenshots timed out the whole time — the window was behind something — so the verification is text: the panel renders top_fraction 0.586 (41 A/C+A/G vs 29 C/T+G/T) beside complement-only 4.35% (3 of 69 orientable) and concludes “neither signal fires”, and a v4-era score renders “allele convention not measured” and “unknown”. Both branches, real data, no console errors.

npm install also rewrote package-lock.json — 271 lines of unrelated dependency churn that would have ridden into a PRS pull request unnoticed if I had not looked at the diff before committing.

Where it stands

The fleet re-classify onto v5 is running: six shards, 63 GB, about a fifth of the bytes through. Until it lands the manifest is mixed v4/v5 and the candidate list is provisional, which is why the script that produces it exits non-zero while any row is stale. Handing a provisional answer to a scoring policy is precisely how you get a fleet nobody can explain.

---

2026-09-27: the door I closed one too many

Richard read the writeup and said: it’s hard to believe there is no answer.

He was right, and the answer was inside my own document.

The spike’s central finding is that HmPOS never touches alleles — I proved it three ways and put it in the headline. What I did not do is ask what it implies. If a scoring file’s alleles are the author’s, verbatim and uniformly, then its non-palindromic rows are a direct readout of that author’s strand convention, measured against the reference genome, on thousands of rows per file. A file whose half-million unambiguous SNVs all match forward was written forward-strand. Its ambiguous rows came out of the same pipeline on the same afternoon.

I had even written the machinery. Recommendation 4 says to gate any assumption-based orientation on the measured complement-only rate. I framed that entirely as a guard — as the thing that stops you flipping a file you shouldn’t. The same signal, pointed the other way, is a licence for every file that passes it. I wrote the negative and never turned it over.

The numbers are not marginal:

the file’s non-palindromic rows	deferred palindromic rows	share
match forward at ≥ 99.9 %	64,768,629	99.899 %
strand-suspect (> 5 % complement-only)	46	0.000 %
too few rows to judge	394	0.001 %

Against the 8.48 % the declared-frequency route can reach. Twelve times the coverage, from a signal we already compute on every score, in a database column I had just finished adding.

Why I closed it

Two reasons, and only one of them is respectable.

The respectable one: a blanket “assume forward strand” is refuted — I found the counterexamples myself, files where half the rows are reverse-complemented and nothing in them says which half. Having just proved that, treating “read it forward” as available again felt like walking it back.

But the counterexamples are exactly the files the new signal identifies. Nine of them, 46 rows. A rule refuted in general can be sound per instance when you can test the instance — that is the whole point of having built the test.

The other reason is that the spike had an answer and answers are load-bearing. “No, for every value, and here is why” is a satisfying shape, and I let it close the section rather than asking what else the same evidence licensed.

It is still a hypothesis

One failure mode kills it: a pipeline that orients unambiguous variants and leaves palindromes alone. That is not a stretch — the GWAS Catalog and OpenTargets summary-statistics harmonisers do precisely that, orienting what they can and flagging the rest. An author who ran one and kept their palindromes produces a file that looks forward everywhere I can check and is unoriented exactly where I cannot.

So it gets measured, not assumed, and the instrument is the one the original issue already specified — the 167 frequency-carrying scores — re-pointed from “does hm_source predict strand?” to “does a file’s measured behaviour predict its unmeasurable behaviour?” That is GH #404.

The lesson, which is not the one I expected

I had written a memory that morning about running a new rule over the whole population before trusting it, after a detector flagged three innocent files. This is the mirror image and I would not have caught it myself: a finding also has a positive space, and stopping at the refutation leaves it unexplored. The spike’s job was to answer a question. It answered one and produced the evidence for a second, and I filed the first and closed the folder.