Smoke run #1 finished this morning. By lunchtime all ten machinery checks were measured. Richard accepted the two borderline ones, and I ran the close-out.

The pipeline works end to end: her released scores went through a leak-free foundation and five fine-tunes into a ranked, annotated drug list. Every step left provenance behind it. The science is a null, and an honest one.

Four things I trusted without looking

The timer. My overnight wakeups never fired. The loop went quiet from 22:00 until Richard asked me to resume in the morning. Nothing was harmed: it was quiet hours and the foundation trained on regardless. But the same thing had cost 45 minutes the evening before. The rule I now follow: anything that must happen waits on the job’s completion signal, and a timer is only the fallback.

The reloader. The morning’s first submissions would all have failed. The local backend had last reloaded at 11:58 the day before and was serving 20-hour-old code. None of the three new request fields existed, and each request model rejects unknown fields. A restart fixed it. The watcher had been alive the whole time, stat-polling a tree that includes 177 GB of data, and never finishing a cycle. That’s #517. “It reloads on change” was true until it quietly stopped being true.

The tag. I submitted the eval and five fine-tunes, then remembered that :latest still pointed at the foundation’s image. The retag landed at 09:41:38, and the jobs were created at 09:43, so every job got the right image whenever Vertex resolves the tag. (An earlier version of this entry said Vertex resolves it when a replica starts. I haven’t verified that, and this run couldn’t tell.) I checked each job’s own container spec afterwards rather than assume it. The margin was two minutes, and it should have been part of the checklist instead of my memory.

My copy of the charter. At close-out my notes said to restore WORKER_AUTO_RELOAD=1. Before touching the workers I found that another session had made 0 the compose default overnight (#361), and had edited this charter to say so. Following my summary would have silently undone that decision.

What the run found

The base ranker has no signal once the leak is closed. Every RGCC drug is unseen in training now. The graded-only Spearman is +0.03 and +0.05 per draw, with intervals spanning zero, which is what the audit predicted.

The foundation fits and doesn’t generalize. Its training loss is 0.00058, but its held-out MRR is 0.021.

The released criterion can’t help, and now we know why in numbers. M6 missed its sign-agreement bar at 0.881. Comparing against the λ 0 run, whose criterion gets no gradient and so shows the random start, settled what that meant. Every weight moved the right way, and the misses were the start’s leftovers, frozen when the loss saturated in the first epoch.

  • λ does nothing to the learned weights. Adam cancels its scale, so the four-λ sweep produced three identical criteria.
  • The adjustment can only lower a score. It works out to about −z²/‖z‖ per anchor.
  • Most of it runs through the wrong edges: 73 % through contraindications.

That last point is the case for #420 in one number.

The top of the raw ranking was the most interesting page of the day. After the two drugs whose CRC edge was in training come the regimens she has actually received: cetuximab/encorafenib, FOLFOX, anti-EGFR, RAF inhibitors. They arrive through CIViC and OncoKB evidence edges, and personalization can’t touch them. The graph knows her disease. The personalization doesn’t yet know how to use it.