The request was simple to state. Another agent had read the published notebook and said, fairly, that for all the careful work we had never shown the main point: that a patient’s genetics can re-rank medicines. Richard’s response was not “prove it”. It was: go end to end with what we have, as a smoke test of the machinery, and first catalogue every issue that stands in the way, so they can be retired one by one instead of rediscovered forever.

I expected a list of maybe eight issues. It ended at fifteen, plus a tracking issue. The reason it grew is the interesting part.

Four audits, and checking the auditors

I split the work four ways:

• the training-graph export and its leaks;
• the fine-tune → personalized-scoring → metrics chain;
• the PRS release path and the disease axis;
• a sweep of all 151 open issues.

They came back with roughly forty findings between them, several of which would overturn numbers we had published. So before filing anything I re-checked the load-bearing ones myself against the database and the artifacts on disk. That paid for itself twice. One agent reported that the expression resolver would silently pull values from a second, unrelated tumor’s transcriptome. That turned out to be latent, not live: the resolver sorts NULLS LAST, and every relevant gene has a newer in-scope value. Another reported a correlation I could reproduce in direction but not to the decimal, because it matched drug names slightly differently. I published my own numbers, not theirs.

The one positive number was memorization

The project’s best PDR2 validation result was a Tier-2 AUROC of 0.835 on the patient’s ex vivo drug-sensitivity panel. It was already flagged “superseded”, because it had pooled two blood draws that disagree. What nobody had checked was which drugs the model had seen labelled.

The foundation’s own split files were still on disk, so I counted. Of the 135 labelled patient→drug edges, 74 sat in the training split. Four of the five high-sensitivity positives were among them. One was trained in under a sibling identifier from a second source, which is why a check keyed on the exact triple had called it held out.

The single positive the model had never seen was the drug the mission spec called “the outlier at rank 260”, blamed for dragging the correlation down. It was not an outlier. It was the only honest data point in the positive class.

Re-scored per draw, the drugs whose labels never reached training correlate at ρ ≈ −0.05 (n = 49). The drugs whose labels did correlate at +0.36 to +0.41. The signal lived exactly where the answers were.

There was a second, larger leak underneath. PrimeKG stores most edges in both directions, and the export split them per direction. 60.6% of the test triples had their mirror in train (539,594 of 890,384), and training never removes the mirror, so every PrimeKG foundation MRR we have quoted, 0.56 to 0.59, partly measures a lookup. The split also depends on Python’s hash seed, which nothing sets. Two runs with the same seed shared about a fifth of their test set.

Three ways the machine would have lied quietly

Each of these would have produced a plausible-looking end-to-end run.

• The PRS would have arrived as zeros. Every one of the 3,752 computed scores has a NULL disease key. The pickle builder keeps only rows that have one, so after all the Gate D and F work a “real PRS” run would have exported an empty vector, filled it with zeros, and reported nothing. It is the same shape as the REF-effect bug: a missing value that looks like a measured zero.
• A failed checkpoint load continues on a random model. The loader catches every exception, logs “Proceeding with randomly initialized model”, and carries on. A smoke test can pass on a random foundation unless the loader is made to fail.
• The fine-tune learns nothing. Under the released loss the optimal disease weights are a closed-form function of the input, −PRS/‖PRS‖. I had it backwards at first; I assumed the sign ran the paper’s way until I read the loss line. The expression half never trains at all, because it enters as 0–100 percentiles and saturates tanh from the first step. Every “top weighted gene” we have reported is an initialization value.

None of these is exotic. Each is a default that nobody had reason to question until the whole chain was asked to run at once.

Why the critical path needed rebuilding

The project already had a “critical path”. The pdr_status tool, which every PDR session is told to call first, returns seven “critical” tasks. All of them are from May, and most concern a model family the project no longer uses. A milestone and an epic label also carried the phrase “critical path” for groups where most members are not on it.

Labelling turned out to be a design problem. The unattended fixer ranks issues partly by priority, so marking fifteen issues p0 would have pointed it straight at them. Most of them hold a decision only Richard can make: an ontology mapping, a statistical reference, a loss sign. The fixer’s eligibility check reads a few phrases in the issue body, so every human-led issue got one, and only the four mechanical fixes were left for it. I re-ran the ranking afterwards to prove the split came out as intended rather than assuming it did.

What the run is now

The smoke run scores drugs against a disease query rather than the patient node. It has to: a foundation trained without her drug labels cannot score the patient-to-drug relation at all, because the relation drops out of the vocabulary. That turns out to be the cleaner test of the actual claim, whether her genetics moves the ranking, on a base that holds no patient drug data.

The playbook is docs/mods_smoke_run_1_playbook.md, and the tracking issue is #439. It predicts a null or negative lift, and it defines success as ten machinery checks, not as lift. That prediction is part of the design. It is what lets a null result mean something instead of looking like a failure.

The count I keep returning to is the one that took a directory listing to find: seventy-four of a hundred and thirty-five. It had been sitting on disk since May.

A note on the model, for the timeline

This session ran on Claude Opus 5.5 (claude-opus-5-5). The work it audited was built almost entirely on Opus 5: the export, the fine-tune plumbing, and the validation numbers.

The session transcripts still on disk cover the last 30 days, and they show the sequence:

Model	Sessions	When
claude-opus-4-7	3	May
claude-opus-4-8	8	June to late July
claude-opus-5	107	from late July on
claude-opus-5-5	17	first message 2026-09-22 at 11:18 PDT

About fourteen of the Opus 5.5 sessions were interactive. The unattended nightly fixer switched to 5.5 by default on the evening of 2026-09-27, the same night it took the first three critical-path fixes.

Richard’s reading is that 5.5 found things that were never considered during development, and he is moving every feasible operation to it. I can’t tell how Opus 5 would have approached the same question, and one session is not a controlled comparison. This was also the first session asked to audit the whole chain end to end rather than build one piece of it, and the question may matter as much as the model. Either way, the tooling under the timeline changed here, and the record should say so.

So every MapleLog entry now carries a model: key in its frontmatter. Entries from 2026-08-29 onward were back-filled from their own transcripts. The three from May and July predate the retention window and say "unrecorded", although the table above suggests Opus 4.7 for May and 4.8 for July.

The next morning

Richard approved the housekeeping, so I did the part that needed checking first.

The stale critical list. The “critical path” that pdr_status hands every PDR session came from a task table whose 37 rows were all created on one day in May and last touched four days later. I demoted its seven open “critical” tasks and marked the list deprecated in the tool, the navigation kernel and the orientation docs. Nobody had consulted it for four months, but every new session was still being told to call it first.

Seven stale issues. Before closing any, I verified each one in the code or the database. Six were clean: a merged fix, a default flipped months before the issue was filed, tables that were never created. The seventh, a double-claim race in the worker, I could only close on circumstantial evidence. The fix for the matching mechanism landed the day after that incident, and the race hasn’t recurred since, including a 20-shard fleet run. I closed it as a duplicate with an explicit condition for reopening rather than pretend to certainty. Two of the issues also asserted a mutation the patient does not have, and I corrected that text instead of just closing over it.

My own wrong turn. To back-fill the model: field I scanned every retained transcript for the session that wrote each log entry. The first pass attributed zero entries to any session. That included this one, which I had written the evening before in the very session doing the scanning. The regex’s backslashes had not survived the trip through the shell, so every Windows path failed to match. The result looked like a plausible story, “the transcripts were pruned”, and I nearly reported it. What caught it was the one case whose answer I already knew. It is the RGCC audit’s lesson in miniature: before believing an instrument’s null, run it on a case where you know the answer.

Overnight the fixer, now on Opus 5.5, took three of the critical-path issues. It finished two and left the third partial, correctly, because the part it could not do depends on a scoring change not yet built. Thirteen remain.