Population-Supervised Outcome Prediction: A Planning Doc

Context

This doc started as paan_planning §5a — a short section sketching a reframe: instead of building a KG that fits Maureen and using her labels for validation, build a KG trained on thousands of (genome, drug, outcome) triples from diverse populations, and let Maureen’s predictions emerge from where her genome lands in that learned geometry.

It got promoted to its own doc because you flagged the underlying prompt as “one of the most important… may be our greatest project”, and because the prompt actually contains three intertwined threads that all need real space to breathe:

The SHEPHERD lift. SHEPHERD is the closest prior art we found (paan_planning §8a), but it’s diagnostic, phenotype-only, and trained on 40,000 synthetic patients. What does it look like to lift the general shape of the idea into therapeutic repurposing? And specifically: why did SHEPHERD need synthetic patients in the first place?
Patient cohorts as training data. The user’s framing: “real patients resembling Maureen, with data we can also get about Maureen, trained on that to predict what an arbitrary example would generate.” This is standard supervised ML — train/val/test, batched, loss-on-outcome. It feels surprising that nobody’s done this cleanly for individual repurposing.
Architecture and unification. Are NBFNet/ULTRA still right? Now that the supervision isn’t “is this KG triple true” but “did drug Y produce outcome Z in subject X” — should the fundamental neural net architecture change? And across the many data sources we’ve identified, what’s the common pattern, and how do we handle the calibration/scale mess?

Three threads, one underlying question: how do we scale training signal beyond Maureen’s own labels to predict her outcomes?

This doc is exploratory — it walks the paths honestly before recommending a starting point in §7.

1. Lifting SHEPHERD — what synthetic patients gave them, and the analog that fits us

1.1 Why SHEPHERD needed synthetic patients

SHEPHERD (Zitnik lab, npj Digital Medicine 2025) trained on ~40,000 simulated rare-disease patients and evaluated on 465 real UDN patients. Why simulate?

Two reasons, in order of importance:

Reason A — Scale for metric learning. SHEPHERD’s loss is contrastive: pull (patient embedding, true causal gene embedding) together; push non-matches apart. Contrastive losses need many examples to populate the metric space meaningfully. 465 patients is not enough; 40,000 is. Real UDN patients are bottlenecked by the rate of clinical enrollment plus the rarity of definitive causal-gene diagnosis — they couldn’t get to 40K with real patients even in principle on a publication timeline.

Reason B — Supervision target is exactly known from the generator. This is the load-bearing observation. In SHEPHERD’s domain:

Sample a disease D (one of 2,134 rare-disease nodes).
Sample HPO phenotype terms for D using the known disease→phenotype edges, with some noise/dropout.
Sample candidate Ensembl genes including the actual causal gene for D plus distractors.

The “causal gene” label for each synthetic patient is known by construction, not estimated. There’s no measurement noise; the supervision signal is exact. Synthetic patients work because the simulator produces labeled training examples without ambiguity.

The key generalization: synthetic supervision is well-defined for problems where the simulator dictates the label. Diagnosis from phenotypes is one such problem. Many others — especially most therapeutic ones — are not.

1.2 Why the synthetic-patient trick doesn’t lift to drug repurposing

Try to imagine what “synthetic patient with synthetic drug response” would look like.

Sample a patient genome G.
Sample a drug D.
Generate an outcome label O.

What’s O? If O is “the response a real patient with genome G would have to drug D,” that’s the function we’re trying to learn. We can’t synthesize it; we’d need a calibrated mechanistic simulator from (G, D) to O, and that doesn’t exist at the scale or breadth we need. If such a simulator existed, we’d already have solved the problem and wouldn’t need ML.

So the SHEPHERD playbook doesn’t extend cleanly. Synthetic patients buy you scale only when the label comes free.

1.3 The right analog: proxy patients, not synthetic ones

What SHEPHERD solved with simulation, we solve with proxies. Cell lines, patient-derived xenografts (PDX), organoids, TCGA patients on standard-of-care, GENIE patients on named regimens — these are real biological entities with real measurements that produce real labels. They’re not the target patient (Maureen), but they’re not synthetic either.

Scale comparison:

What	SHEPHERD analog	Us
Target	one real UDN patient at inference time	one real patient (Maureen) at inference time
Training scale	40,000 synthetic patients	~1,000 cell lines (GDSC, CCLE) + ~1,000 cell lines × CRISPR (DepMap) + ~11,000 TCGA patients + ~5,500 CRC patients in MSK-CHORD and ~1,500-3,000 in GENIE BPC (not the ~100K main GENIE registry — see §4.5) + hundreds of PDX/organoids
Label quality	exact (by construction)	real measurements, but with assay noise + biological gaps
Label semantics	causal gene rank	IC50 / AUC / % cell death / RECIST / OS / TGI — heterogeneous

We get more training data than SHEPHERD did, and it’s real biology, not simulated. The cost: heterogeneous label scales, distribution shift from cell line to patient, and identifier-reconciliation overhead. Those costs are addressable. The benefit is the supervision is grounded in measurement.

1.4 What does actually transfer from SHEPHERD

Three pieces survive the lift:

A patient can be meaningfully embedded into a biomedical KG, and the embedding can drive useful ranking. SHEPHERD shows this empirically for diagnostic ranking; the inductive bias is preserved across diagnostic and therapeutic tasks.
Inductive scoring on novel patients without retraining. Critical for clinical workflows — when Maureen’s data updates (new RGCC test, new lab value), we don’t want to retrain.
Patient-as-subgraph is a real architectural option. PDR2 (and our current direction) makes patient an explicit node. SHEPHERD makes patient = a set of phenotype terms that induce a subgraph. The patient gets no learned embedding of their own — their representation is constructed at inference by aggregating over the induced subgraph. This option is worth seriously considering — see §5.

1.5 The architectural choice this forces

Patient-as-subgraph vs patient-as-explicit-node:

	Patient-as-subgraph	Patient-as-explicit-node
Inference flexibility	Excellent — new patient = new subgraph, no retraining	Limited — need the patient in the training graph
Learned representation	None of their own; built from subgraph at inference	Each patient gets a learned embedding
PDR2 trick	Doesn’t apply	Foundation frozen + small learnable loss weights (1,802 of them) → exactly works
Maureen-specific overfitting	Low	High if not careful
Generalization to a brand-new patient	Free	Requires graph augmentation + (per PDR2) fine-tune the loss weights

PDR2 picks the second column because its trick — freezing the foundation and only training the loss — lets the patient’s edges influence the model without learning a patient-specific embedding that wouldn’t generalize.

We could pick either. We could pick both: subgraph-style at inference, explicit-node-style at training for the cohort of proxies. The recommendation lands in §7.

2. Patient cohorts as training data — the standard ML framing

2.1 The framing in one paragraph

Input: (subject_features, drug_features). Output: response_label. Train on many (subject, drug) pairs from many subjects; validate on held-out (subject, drug) pairs; test on a held-out cohort or a strictly held-out drug list. Loss is regression on response, or pairwise/listwise ranking, or survival, depending on the response type. At inference, encode Maureen → predict her response → rank drugs by predicted response.

This is the standard cancer drug response prediction (CDRP) setup. The user’s framing is correct.

2.2 What the user thinks vs what the field has actually done

The user wrote: “It’s hard to believe nobody else has thought about doing this for drug repurposing to the individual.”

The honest answer: many people have thought about it, just not exactly this. The cancer drug response prediction literature is large. Representative papers:

PaccMann (Manica et al., Mol. Pharm. 2019; arXiv:1811.05741) — attention over gene expression + drug SMILES → IC50.
DeepDR (Chiu et al., BMC Med. Genomics 2019) — autoencoder for mutation + expression → drug response.
MOLI (Sharifi-Noghabi et al., Bioinformatics 2019) — multi-omics late-integration network for drug response.
DeepCDR (Liu et al., Bioinformatics 2020) — drug molecular graph + omics + ConvNet → drug response.
CDRscan, tCNNs, CaDRReS-Sc, scDEAL, scPDS — variations on the same theme.

What’s actually under-served — and where our setup is unusually well-positioned — is the intersection of:

Drug repurposing. Existing approved or already-in-trial drugs, not de novo discovery.
Individual patient resolution. Not “what works for KRAS-mutant CRC” but “what works for this CRC patient with this expression profile and this germline.”
Outcome-supervised ranking, not IC50 regression in cell lines. The deliverable is “rank these 500 drugs for this patient by expected effect,” not “predict log(IC50) accurately across the GDSC test set.”
A patient with our depth of supervision. Maureen has RGCC (224 drug x % cell death labels over 112 drugs), Caris WTS (whole-transcriptome), Caris WES (somatic + germline variants), Invitae germline, treatment history with response, longitudinal labs. Most patients in any cohort lack one or more of these. The user’s “real patients resembling Maureen, with data we can also get about Maureen” criterion is genuinely restrictive — and is the empirical reason nobody’s done this cleanly. There aren’t many Maureens.

So: the field has done the encoder + drug-feature + response-head architecture many times. What we’d add is (a) the repurposing framing, (b) the individual focus driven by an unusually-instrumented anchor patient, and (c) outcome-ranking rather than IC50-regression objectives.

2.3 What does the loss function look like?

Several options, and the right one depends on what we want to deliver:

MSE / Huber regression on response. Standard. Works when the response is on a single scale (e.g., log(IC50) within GDSC). Breaks across sources with different scales unless we calibrate first.
Pairwise ranking (BPR, RankNet). Each training example is a triplet (subject, drug_a, drug_b) where drug_a was better. The loss pushes the model to score drug_a higher than drug_b for that subject. Robust to scale differences — only the order matters. This is probably the right primary loss for individual repurposing: we don’t actually care about predicted IC50; we care about ranking the candidate drugs.
Listwise (NDCG-based). Generalizes pairwise to “for this subject, the top-K drugs should be these.” Good if we have many drugs tested in each subject, which we do in RGCC and GDSC.
Survival losses (Cox, DeepSurv, DeepHit). Needed if we use TCGA or GENIE OS/PFS as labels. Handles censoring.
PDR2-style composite. L = L_base − λ₁·L_prs − λ₂·L_expr where L_base is any of the above and the patient-prior terms modulate scores based on PRS and biomarker deviations. Compatible with the above as a wrapper.

If we had to pick one for v0: pairwise BPR within source. Rank-based, scale-robust, well-understood, and aligned with the actual deliverable (drug ranking for a patient).

2.4 Validation, in layers

The whole point of the population-tapestry approach is that we have thousands of proxy subjects (cell lines, PDX, organoids, TCGA patients), so the standard held-out-test-set move is available and used. The reason validation in this setup still deserves its own section is that there are at least four distinct layers of validation, each progressively harder, and it’s easy to conflate them. The patient-of-1 framing only bites at the last layer.

Layer 1 — Standard held-out within the training source. Train on 80% of GDSC, validate on 10%, test on 10%. Standard ML practice. This is exactly the same proxy-patient data used for training, partitioned. There is no problem here — it’s the cleanest validation we have, and it’s what we’d report as the primary internal metric for the trained model.
Layer 2 — Cross-source generalization. Train on GDSC, validate on CCLE over the shared cell-line subset. Tests whether the model generalized or just learned GDSC’s idiosyncrasies. Harder because of the Haibe-Kains 2013 receipt — measurements disagree across panels even on the same cell lines. Per-source decoder heads (Path B in §3.3) keep this layer from becoming an architectural problem.
Layer 3 — Cell-line → patient transfer. Train on cell lines, evaluate on TCGA real-patient outcomes (OS, PFS) under the corresponding regimens where known. This is the field’s central hard problem (gotcha #2 in §6). Many published cell-line-trained models look great on Layers 1–2 and fail Layer 3 badly. It’s why the PDX and organoid consortia exist.

Data source for this layer (added 2026-09-24). Layer 3 was specified here without a dataset attached to it. The candidates are MSK-CHORD (5,543 CRC patients) and GENIE BPC CRC (1,485→3,000+, curated line-of-therapy). Note this is not in tension with §9’s exclusion of EHR mining: that exclusion is about using real-world cohorts as training labels, where line-of-therapy confounding is fatal. Using them to check whether a cell-line-trained ranker holds up against real outcomes is a weaker claim that tolerates confounding far better — and it is the only way to measure Layer 3 at all. PDX (Novartis PDXE: ~1,000 models, 62 treatments, mRECIST incl. CRC) and the CRC organoid studies (OPTIC: PPV 0.78 / NPV 0.80 / AUC 0.78–0.88 vs patient response) sit between Layers 2 and 3 and are the cleanest bridge, because they carry molecular depth and per-drug response.
Layer 4 — Population → this specific patient. Predict Maureen’s RGCC drug rankings; predict her future RECIST outcomes from pre-treatment features. This is the layer where “patient-of-1” actually bites: RGCC gives us 224 labels for Maureen — meaningful internal signal — but there is only one Maureen. We can’t validate “did the model learn precision medicine for her” the way we’d validate a normal model, because there’s no cohort of comparably-instrumented patients to test against.

Layers 1–3 are standard machine learning territory and don’t require any special framing. Layer 4 is where the discomfort lives. Realistic moves for Layer 4:

Drug-held-out within Maureen. Train on 80% of her RGCC drugs (and on all the proxy cohort data); test on the held-out 20%. Tests whether the model generalizes across drugs for the same patient.
Time-held-out. Use Maureen’s pre-2026 data to predict her 2026 RECIST outcomes. Strict but low-resolution (few clinical episodes).
Held-out variant signature. Hold out all proxy subjects sharing a Maureen-defining variant or signature from training; test on Maureen. Tests variant-specific generalization vs cohort averages.
RGCC-as-proxy-for-clinical-outcome. Even Maureen’s RGCC isn’t the actual deliverable (clinical response is). The ex-vivo CTC assay → in-vivo response gap (gotcha #8) is part of the Layer 4 validation budget, not separate from it.

The point of separating these layers: don’t muddle them. Standard supervised ML on proxy patients gives us Layer 1 cleanly. The further you climb, the more uncomfortable validation becomes, but each layer is a separately reportable signal — and Layer 4 being hard is not a reason to skip Layers 1–3.

2.5 The intended-output problem (the deeper one)

Layer 4 above actually understates what’s happening. The model’s drug ranking for Maureen is the intended output of the system. Validating it the standard ML way would require a ground-truth label for “the best drug for Maureen.” That label doesn’t exist — if it did, we wouldn’t need the model. This isn’t a defect in our setup; it’s the same problem all of personalized medicine prediction faces. We never observe the counterfactual outcomes of what would have happened to her on the drugs she didn’t take. The standard ML loop (train → predict → check) breaks because the “check” step has no ground-truth target.

Terminology note. This is, technically, zero-shot per-subject inference: Maureen wasn’t in training; we encode her features and produce a ranking without ever having seen labels for her. ULTRA and PDR2 already use “zero-shot” for the KG-level analog (apply a pre-trained model to a never-seen KG); the subject-level version is the same idea, one rung down. What separates our setup from standard zero-shot ML benchmarks is that those benchmarks do have ground truth on held-out classes — zero-shot CUB-200 birds, you know what species the test image actually is. We don’t. So we’re doing zero-shot inference in a regime where standard zero-shot evaluation doesn’t apply. The one validation path the zero-shot literature does offer that survives this gap is reasoning-chain / attribute-attribution validation — validate the reasoning even when you can’t validate the answer (“the model ranks drug X high because Maureen has biomarker Y, and drug X is known to act on the Y pathway; is that chain biologically plausible?”). That’s what the four shifted answers below are circling.

So what does validation mean here? Four shifted answers:

Validate the learning, not the answer. Layers 1–3 show the model captures (subject, drug) → response patterns where we have labels. That’s necessary for trusting its Maureen prediction, not sufficient. The right analog is a clinical trial: we don’t validate “this drug works for patient X”; we validate “this drug works in a population that includes patient X,” and then trust the inference one step further.
Validate against the imperfect proxy we do have. RGCC gives 224 labels for Maureen. Drug-held-out within her tests whether the model can recover the rest. This validates against an ex-vivo CTC assay whose own concordance with in-vivo clinical outcome is an open empirical question (gotcha #8). Useful, partial, not definitive.
Reframe to decision-support validation. The deliverable isn’t “the right drug.” It’s “candidates worth investigating, with reasoning a clinician/patient can evaluate.” Validation under this framing is downstream: does the recommendation change action and improve outcomes vs. the alternative process (SOC, intuition, a coin flip)? Different statistics, different cadence — the framing used for clinical decision support systems, not pure prediction models.
Eventually, prospective outcomes — slowly. Predictions made at time T checked against what was tried and how Maureen responded. One patient is statistically thin, but a pattern accumulates over years of decisions. The counterfactual outcomes (what the other drugs would have done) remain unobserved.

The right mental model: this is a decision support tool whose final output is itself the deliverable. The model surfaces a best guess from population evidence. Validation of the learning process is necessary and reportable (Layers 1–3). Full validation of the specific prediction for Maureen is partial at best — and that’s not a defect, it’s the epistemology of the problem. Anyone who claims they can fully validate a personalized prediction without observing counterfactuals is selling something.

2.6 The structural consequence — PDR2’s paths are not optional

The framing above has a sharp architectural implication that’s worth naming directly. If the system’s job is to surface recommendations that we’d only act on if they look like good ideas, and the way they “look like good ideas” is by being defensible through inspectable reasoning a domain expert can evaluate, then the reasoning surface is structurally necessary — not a nice-to-have, not interpretability theatre, but the actual mechanism of validation.

Put plainly: by definition, the validation of any personalized recommendation we’d ever act on is validation of the logical path that produced it. If the path looks like a good idea, we try it. If it doesn’t, we don’t. There is no other operating mode.

This makes PDR2/ULTRA’s KG-completion track (Track 1 in §5.4) load-bearing in a way that wasn’t obvious before. NBFNet-style path-based models don’t just produce a score for (Maureen, treats, drug_X) — they produce one or more paths through the KG that justify the score:

Maureen → has_variant → BRAF_V600E → activates → MAPK_pathway → inhibited_by → BRAF_inhibitors → drug_X

A clinician (or the user, or Maureen herself) reads that path and either says “yes, this is biologically defensible” or “no, that path goes through a known-spurious edge, skip this drug.” The path is the unit of validation. The score, on its own, is unactionable.

Track 2 (cell-line bi-encoder, §5.4) does not have this property. It produces an opaque score from a learned (subject, drug) → response function. Without Track 1, we’d have a number with no reasoning behind it — and a number without reasoning is not something we’d act on.

This refines the two-track architecture recommendation:

Track 1 is not replaceable by Track 2. Even if Track 2 had higher Layer 1–3 accuracy, removing Track 1 would remove the reasoning surface — and the reasoning surface is the validation mechanism.
The combiner at inference must preserve at least one Track 1 path for every top candidate, not just rank by combined score.
Track 2’s scores are supplementary confidence; Track 1’s paths are the validation surface. A drug that Track 2 ranks high but for which Track 1 produces no defensible path is a flagged candidate worth investigating further, not an actionable recommendation.
Layers 1–3 (§2.4) remain necessary in this picture, but for a narrower reason than before: they validate that the model’s learning process generalized, so that the paths it produces aren’t memorized noise. Without that prior, even a beautiful-looking path is suspect.

This also gives a structural answer to a question we hadn’t quite asked directly: why a KG at all? The KG isn’t here because graphs are trendy or because PrimeKG was available. It’s here because the KG is the substrate on which reasoning paths exist, and reasoning paths are the only thing that survives as a validation mechanism when the intended output has no ground truth.

3. The data sources, examined — what’s the common pattern?

Don’t re-list every source — KG_CATALOG.md and the memory entries already do that. Here we focus on the unification question.

3.1 A common schema across all sources

Every patient-outcome data source can be cast as:

Field	What it is	Examples
`subject_id`	A biological entity that was assayed	`CCLE_HEPG2`, `TCGA-AA-1234`, `RGCC_maureen`, `PDX_017`, `ORG_AC042`
`subject_type`	What kind	`cell_line`, `patient`, `pdx`, `organoid`
`subject_features`	What we know about it	gene expression vector, variant calls, methylation, mutational burden
`drug_id`	A chemical entity	DrugBank or ChEMBL ID (normalized)
`drug_features`	What we know about the drug	SMILES + fingerprint, ATC code, target list, MoA
`response_label`	What was measured	log(IC50), AUC, % cell death, RECIST category, OS_months, TGI%
`response_unit`	Scale type	µM, dimensionless [0,1], %, ordinal, months, %
`response_source`	Where it came from	`gdsc_v2`, `ccle`, `depmap`, `rgcc`, `tcga`, `genie`, `pdxnet`, `optic`
`response_context`	Other things that matter	concentration, exposure duration, line-of-therapy, prior treatments

The response_source field is load-bearing for the calibration strategy in §4. Almost everything else is normalizable in standard ways; the response_source field is the one that determines whether we can fuse losses across sources.

3.2 Two architectural paths through unification

Path A — Unify into a single response target. Force everything to a common abstraction. Example: convert all responses to “is this drug in the top-20% of effective drugs for this subject?” — a binary label that’s defined regardless of whether the underlying scale was IC50, AUC, RECIST, or OS. Trainable end-to-end with a single loss. Lossy: we throw away absolute response magnitude. Hard: the threshold (top-20%) is arbitrary and may not transfer biological meaning across sources.

Path B — Per-source decoder heads with shared encoder. Each source keeps its own scale. The (subject, drug) encoder is shared across sources — that’s where the generalization comes from. The response head is per-source: one head emits log(IC50), another emits RECIST probability, another emits hazard ratio. At training time, each batch goes through the appropriate head. At inference, we either pick one source’s head (the closest match to our deliverable) or combine multiple via meta-model.

Recommendation: Path B for MVP, Path A as a stretch goal. Path B is what the multi-task literature recommends when scales differ. It also makes the calibration question (§4) optional rather than required.

4. Calibration and scale — the genuinely hard problem

The user asked specifically: “Isn’t this what z-scaling is for? If that doesn’t get us all the way there, what do other people do?”

The honest answer is that z-scaling is necessary but nowhere near sufficient, and there’s a famous receipt for why.

4.1 What z-scaling does and doesn’t

Z-scaling centers and rescales a vector to mean 0, std 1. It’s the right answer when you want to compare two things on different scales but with the same meaning. It does nothing to fix two sources that disagree on the same measurement.

Concretely: if GDSC says paclitaxel has IC50 = 5 nM in cell line X and CCLE says paclitaxel has IC50 = 50 nM in the same cell line X, z-scaling each panel within itself doesn’t reconcile them — z(GDSC X) and z(CCLE X) come out at different points relative to their own panels’ distributions.

4.2 The Haibe-Kains receipt

Haibe-Kains et al. (Nature 2013, “Inconsistency in large pharmacogenomic studies”) showed exactly this empirically: GDSC and CCLE drug responses disagree on the same cell lines for many drugs, with Spearman correlations on shared subset ranging from near-zero to modest. The paper kicked off a multi-year reconciliation effort and motivated PharmacoDB and similar harmonization projects.

This isn’t a “we need better z-scoring” problem. It’s “the underlying assays disagree because of differences in concentration ranges tested, exposure duration, viability readout (CellTiter-Glo vs SRB vs syto60), and analysis pipeline.” Calibration in this space is genuinely hard, and people who say “just z-score” have not actually tried to do it across panels.

4.3 Quantile normalization

Force the marginal distributions to match. Preserve ranks within each panel, destroy the absolute scale. This is OK if downstream is rank-based (BPR/NDCG). It’s not OK if downstream needs to predict an absolute IC50 because the absolute information is gone.

This pairs well with Path B + pairwise ranking loss. We don’t need absolute scale to rank drugs for a patient.

4.4 Linker drugs (empirical calibration)

Pick drugs that were tested in multiple panels. For GDSC ↔ CCLE there are ~250 shared drugs. Fit a regression (linear, monotonic, or isotonic) for each shared drug mapping source A’s response to source B’s response on shared subjects. Apply the regression to unshared drugs as a calibration prior.

This is analogous to ComBat batch correction in genomics or ratio normalization in cross-platform proteomics. It’s the empirical answer to the user’s question “can we calibrate things empirically somehow?” — yes, and there’s a literature on it.

4.5 Start with the biggest, cleanest single source

The user explicitly asked: “What if we started with the biggest / best / richest data set, and used only that?”

This is the right MVP move. Single-source training side-steps the calibration nightmare entirely. The candidate sources, ranked by suitability for our purpose:

GDSC v2 — ~1,000 cell lines × ~500 drugs, well-curated, open license, log(IC50) is a single coherent scale, and a substantial subset of the RGCC drug list (estimate ≥60 drugs; verify against actual RGCC panel before committing). Recommended starting point.
CCLE — ~1,000 cell lines, narrower drug panel than GDSC v2 but richer multi-omics on the cell lines. Strong second option, especially if we want to leverage transcriptome features the same way Caris WTS gives us them for Maureen.
DepMap — gene-knockout viability, not drug response. Different question. Useful for target-relevance reasoning, not direct drug-response prediction.
TCGA — real patients, outcome labels. Confounded by line-of-therapy + supportive care; mostly first-line outcomes; censoring; selection effects.
AACR GENIE (main registry) — 100K+ patient genomes, but essentially no treatment or outcome data at all: it is genomics plus minimal clinical. Not a supervised training source. (Corrected 2026-09-24 — an earlier revision of this doc counted these 100K as labelled proxies and separately dismissed the labels as “inconsistent”. Both were wrong.)
GENIE BPC — the curated subset that does carry outcomes, via PRISSMM: treatment histories with timing, imaging-based disease status, PFS, plus MSI/MMR/PD-L1. The CRC cohort is released (BPC CRC v2.0-public, 1,485 patients; phase II expanded past 3,000). Free via Synapse. Labels are deliberately curated, not inconsistent. Good v2 source.
MSK-CHORD — 5,543 CRC patients, MSK-IMPACT + NLP-derived outcomes, treatments and timelines; public via cBioPortal, CC BY-NC-ND 4.0 (Nature 2024). Largest free CRC clinicogenomic cohort. Panel sequencing only, so no transcriptome — which matters, because Maureen’s strongest feature is her full WTS. Best used for Layer 3 (§2.4), not as a training source.
Project Data Sphere — patient-level control-arm data from phase III oncology trials including CRC, free and with no research proposal required. RECIST response + regimen, usually no genomics.

Recommendation: GDSC v2 alone, train bi-encoder regression on log(IC50), validate on cell-line held-out, test on Maureen RGCC. If signal looks good, add CCLE next via linker-drug calibration or per-source decoder heads.

4.6 Could ensembles avoid the calibration problem?

The user asked: “Maybe an ensemble of models, each trained on its own data set, could avoid calibration issues?”

Yes — this is a real strategy. Train one model per source (GDSC model, CCLE model, TCGA model, …) and combine at inference time via mean rank, learned meta-model, or voting. No cross-source calibration is required at training time; each model only sees its own coherent scale.

The cost is that we lose cross-source generalization at training time. A drug tested only in CCLE doesn’t get the benefit of GDSC patterns. But for MVP, this loss is acceptable — and the ensemble is a defensible final architecture regardless. Many production recommender systems work this way.

4.7 Recommended calibration approach

Start single-source (GDSC v2). No calibration.
Add a second source (CCLE) via per-source decoder heads. Shared encoder, separate heads. No calibration between heads.
Add TCGA via survival head. Still no cross-source calibration; just another decoder head.
At inference, combine via rank fusion or learned meta-model. Don’t try to force calibration earlier than necessary.
Only attempt unified scaling (Path A) if rank-fusion is empirically worse than a unified target. Let empirics decide.

5. Architecture — is NBFNet/ULTRA still the right primitive?

The user asked: “Now that we aren’t exactly talking about predicting paths through graphs (or are we?), should we be rethinking the fundamental neural net architecture?”

The honest answer: NBFNet/ULTRA is right for one of the two things we want to do, and wrong for the other. The doc should be honest that we need both.

5.1 What NBFNet/ULTRA is good at

Path reasoning over a discrete KG: “drug treats disease that has gene that interacts with gene that has variant in patient.”
Compositional inductive bias — the same path patterns generalize to new entities.
Inductive in the new-entity sense — ULTRA generalizes to KGs it’s never seen.
Mature, runnable, already integrated (PDR2 foundation training is operational per memory).

For the KG-completion side of the problem — “given this KG, what drugs should rank highly for this disease?” — NBFNet/ULTRA is the right tool. PDR2 has shown it can be personalized with the patient-specific loss.

5.2 What NBFNet/ULTRA is bad at

Continuous-valued tabular regression. The output is a triple-existence probability, not a calibrated drug-response prediction.
Scaling to millions of labeled (subject, drug, response) triples where the labels are continuous and the subjects don’t have a natural KG-node representation. Cell lines aren’t really “in the KG” in a useful sense — they’re tabular objects with omics vectors.
Multi-modal fusion at the node level. A patient with gene expression, variants, methylation, and treatment history isn’t a single KG node — it’s a structured object with several modalities.

For the cell-line-to-patient supervised-learning side — “I have GDSC, learn (cell_line_features, drug_features) → log(IC50)” — NBFNet/ULTRA is not the natural primitive.

5.3 Alternatives, briefly

Bi-encoder regression. Industry-standard recommender pattern. Subject encoder (MLP or transformer over omics + variants) + drug encoder (MLP or GNN over molecular graph) + scalar head (dot product or MLP). Cheap to train. Scales to millions of pairs. The right primitive for the cell-line track.
Multi-modal fusion transformer. Token streams from each modality (genome tokens, expression tokens, drug structure tokens) + self-attention + head. PaccMann and DeepCDR are in this family. More expressive, more data-hungry, harder to debug.
Pretrained omics foundation models. scGPT (single-cell transcriptome, Cui et al. Nature Methods 2024), Geneformer (bulk transcriptome from large public corpora, Theodoris et al. Nature 2023). Use them as the subject encoder. Particularly relevant because Maureen’s WTS is one sample — having a pretrained encoder that’s seen millions of transcriptomes is a meaningful inductive bias.
Hybrid KG + tabular. Use KG embeddings (ULTRA over PrimeKG) as features for a tabular learner (transformer or gradient-boosting) on the (subject, drug, response) data. Two-stage but lets us reuse what we’ve already built and lets each piece do what it’s good at.
Equivariant / geometric (EquiBind, DiffDock). Operates on 3D molecular structure for binding-pose prediction. Tangential to our question — we’re predicting outcomes, not binding poses. Mention and move on.

5.4 Recommendation — a two-track architecture

The right move is not to pick one. It’s to run two tracks and combine at inference time. Note that §2.6 sharpens the role of Track 1: it’s not “the KG track we happen to have running” — it’s the load-bearing source of reasoning paths, which are the system’s actual validation surface. Track 2 cannot stand alone.

Track 1 — KG (PDR2 / ULTRA, existing). Patient-as-node + patient-specific loss. Optimized for KG-completion-style ranking. Foundation training is already operational. Outputs a relevance score and at least one explanatory path per (patient, drug, treats-relation). The path is the surface a clinician inspects to judge whether the recommendation is defensible.
Track 2 — Cell-line bi-encoder (new). Subject encoder over Caris-aligned expression features + variant features + (optional) pretrained scGPT/Geneformer embedding. Drug encoder over SMILES → Morgan fingerprint (v0) or learned drug graph embedding (v1). Output: predicted log(IC50) or pairwise rank. Trained on GDSC v2 first. Scores are supplementary confidence; this track does not produce paths and so cannot stand alone as a recommendation source.
Combiner at inference. Take Track 1 score, Track 1 path(s), and Track 2 score per drug. Fuse the scores via simple averaging or a learned meta-model, but require at least one Track 1 path per candidate that’s surfaced as actionable. Drugs that Track 2 ranks high but for which Track 1 has no defensible path are surfaced as a separate “flagged candidates worth investigating further” list, not as actionable recommendations.

This is the cell-line-to-patient transfer pattern used in scDEAL/scPDS/CaDRReS-Sc — pretrain on cell lines, condition on patient. The PaaN architecture is the unification of that with KG-based reasoning, with the KG-based reasoning playing the dominant role for validation reasons (§2.6).

The two-track approach has another property: each track can be debugged, validated, and ablated independently. If Track 1’s paths are biologically implausible, we can see it. If Track 2 has poor cell-line-to-patient transfer (the field’s central problem), we can see that too.

6. Critical gotchas — surface honestly, don’t bury

Haibe-Kains 2013 is real. Drug response measurements disagree across panels. Don’t assume calibration is solvable with z-scoring; it isn’t.
Cell-line → patient distribution shift is the field’s central problem. Many CDRP models look great on cell-line held-out and degrade sharply when moved to patient outcomes. Anything we build must take this seriously. The PDX consortia exist because of this.
The CDRP literature is large. Push back on “nobody’s done this.” The novelty is in the intersection (repurposing + individual + RGCC-anchored), not in the bi-encoder + drug-response framing per se.
Patient-of-1 only bites at Layer 4 of validation. Standard held-out test sets on proxy patients (cell lines, TCGA, etc.) work normally — that’s Layers 1–3 in §2.4. Layer 4 — does the proxy-trained model actually help this patient — is the uncomfortable one, because we have one Maureen.
Outcome semantics are heterogeneous. IC50, AUC, % cell death, RECIST, OS, TGI — different scales, different biological meaning. Path B (per-source heads) is the safe move.
NBFNet vs regression mismatch. Link prediction is not outcome regression. Architecture must match the task, not the comfort zone.
Maureen’s variants are rare in cell lines. Her inherited variant, her exact tumor profile, her CMS subtype — no cell line is a clean match. The model has to learn at a functional level (pathway, expression, signature), not at variant-identity level.
RGCC ≠ in vivo response. It’s an ex-vivo CTC assay. Concordance with clinical outcome is itself an open empirical question. Phase 7.5 has begun characterizing this (pdr_phase7_5_rgcc_writeup.md).
Identifier reconciliation is real work. Cell lines (CCLE/COSMIC/DepMap), drugs (DrugBank/ChEMBL/PubChem/NCIt), genes (Ensembl/HGNC/Entrez). The Composition primitive helps but isn’t free.
Ensemble combination is hard. Combining an IC50 prediction + a survival hazard + a KG-completion score into a single rank requires care. Rank-fusion is the cheap first move; learned combiners come later.
SHEPHERD’s synthetic-patient trick does not lift. Therapeutic supervision isn’t simulatable. Real proxies (cell lines, PDX, organoids, real cohorts) are the answer.
TCGA outcomes are mostly first-line. Most drug × patient combinations are unobserved. Causal-inference territory if we lean on TCGA seriously.
AACR GENIE’s main registry has no outcome labels at all — it is genomics plus minimal clinical, so it is not a training set on any day. The outcomes live in GENIE BPC, a separately curated subset whose CRC cohort is released and is consistent. Corrected 2026-09-24; the earlier “inconsistent labels” framing was wrong in both directions.
Patient-as-node requires the patient in the training graph. PDR2’s trick (foundation frozen, 1,802 learnable loss weights) is what makes this actually work without overfitting. If we switch backbones (bi-encoder, transformer), we lose this architecture and need to redesign personalization. Worth knowing before we commit.
Removing the KG-path track removes the system’s validation surface. Track 1 (PDR2/ULTRA) is not replaceable by Track 2 (bi-encoder regression) alone — see §2.6. A black-box regressor produces opaque scores that no domain expert can sanity-check, and we will not act on un-reasoned scores. Don’t optimize Track 1 away in pursuit of Track 2 accuracy, and don’t replace path-based KG models with non-explanatory KG models (plain DistMult/ComplEx without path tracing) for the same reason.

7. Recommended minimum viable experiment

Cheap, falsifiable, runs in days.

Source: GDSC v2 only. Single source first. No calibration. ~1,000 cell lines × ~500 drugs.
Subject features: gene expression vector (CCLE-aligned where overlap exists) + variant burden. For Maureen: Caris WTS-aligned expression + Caris/Invitae variant calls. Compute the feature vector once; cache.
Drug features: SMILES → 1024-bit Morgan fingerprint. Simple, fast, no model needed. Upgrade to learned drug graph embedding in v1.
Architecture: bi-encoder regression. Two small MLPs (subject encoder: 512 → 256 → 128; drug encoder: 1024 → 256 → 128). Head: concatenate + 2-layer MLP → predict log(IC50). Total params well under 1M.
Loss: MSE on log(IC50) for v0. Pairwise BPR on (subject, drug_better, drug_worse) for v1.
Train/val/test: blind on both subjects and drugs — strictly held-out cell lines AND strictly held-out drugs. Hardest split. Reports true generalization.
Maureen evaluation: encode her via Caris WTS-aligned features. Predict log(IC50) for the drug overlap with her RGCC panel. Compute Spearman ρ between predicted log(IC50) and her RGCC % cell death. Positive ρ with reasonable confidence = the approach is alive.
Compare to: PDR2 foundation Track-A baseline (Phase 7.5 ρ = +0.351 on PrimeKG) — ⚠ that comparator is superseded pending restatement (GH #324): it was computed on RGCC labels averaged across her two draws, which flattened the 12 drugs that inverted by class. Restate it per timepoint before using it as the bar. If the bi-encoder is competitive or complementary, the two-track combination is worth pursuing.
Cost: a few days for v0. Roughly one foundation-training-equivalent in compute. Cheap relative to the foundation work already running.

Decision gate: if ρ on Maureen is meaningfully positive (say > +0.15) with the cell-line-only model, the cell-line → patient transfer works for our setup and the two-track architecture is worth building out. If ρ is near zero or negative, the transfer doesn’t work for Maureen and we need to think harder (organoid data, PDX, cohort matching, or restricting the model to a Caris-similarity-weighted subset).

8. Open questions, consolidated

Not pre-decided. These survive into the next round of thinking.

Patient-as-node (PDR2-style) vs patient-as-subgraph (SHEPHERD-style) at inference? Recommendation defaulted to PDR2-style for Track 1 because it’s running, but the subgraph approach may matter when we add new patients.
Path A (unified target) vs Path B (per-source heads)? Recommendation defaulted to Path B; revisit if signal is strong enough that calibration would unlock more.
Cell lines first, or jump to PDX/organoids? Cell-line first for scale; PDX/organoid as v2 when we’ve learned what the cell-line-only model can’t do.
Maureen’s RGCC: train on it (drug-held-out) or strictly validate-only? v0 keeps RGCC strictly held-out so we know the cell-line model alone can transfer; v1 may use drug-held-out RGCC for fine-tuning.
TCGA outcomes useful or hopelessly confounded? Worth a separate explore — survival head trained on TCGA could complement IC50 head; the confounding is real but maybe addressable with line-of-therapy stratification.
How aggressively to lean on omics foundation models (scGPT, Geneformer) for the subject encoder? Worth a single-sweep comparison early; if pretrained encoders dominate, the architecture pivots toward foundation-model-fine-tuning.
Does the two-track architecture stay two tracks, or do we eventually fuse into a single unified model? Probably stays two tracks for a long time — each is debuggable on its own.

9. What we are explicitly not proposing

To prevent scope creep and to be honest about what we ruled out:

Synthetic patients à la SHEPHERD. The supervision target isn’t simulatable for therapy. Proxy patients (cell lines, PDX, organoids, real cohorts) replace this role.
A pivot away from PDR2. PDR2 stays as Track 1. We run a parallel Track 2 (bi-encoder regression on cell lines) and combine at inference. Both contribute; neither is sacrificed.
Mining EHR for “patients like Maureen” who got drug X and predicting OS — as a training signal. Too confounded by line-of-therapy, supportive care, selection effects, and lack of matched-control patients. Refined 2026-09-24: this exclusion is about training. The same cohorts (MSK-CHORD, GENIE BPC) are legitimate for Layer 3 validation (§2.4), where the claim is weaker and confounding is tolerable. Do not conflate the two uses.
A unified single-loss model spanning all sources. Calibration is too hard for MVP. Per-source decoder heads with shared encoder is the safer move.
Equivariant / geometric models (DiffDock, EquiBind). Wrong task — they’re for binding pose prediction, not outcome prediction.

TL;DR

The SHEPHERD lift exists, but the synthetic-patient piece doesn’t transfer. The analog for us is proxy patients (cell lines / PDX / organoids / TCGA / GENIE), which are real biology with real labels at population scale.
Patient cohorts as training data is the standard cancer drug response prediction (CDRP) setup, and the literature is large (PaccMann, DeepDR, MOLI, DeepCDR, …). What’s novel about our setup is the intersection: repurposing + individual + RGCC-anchored + outcome-ranking objective.
The right loss is probably pairwise ranking (rank drug A > drug B for this patient), not absolute response regression. Rank-based losses sidestep the calibration nightmare.
Calibration is genuinely hard — z-scoring is necessary but nowhere near sufficient (Haibe-Kains 2013). Best practical answer: single-source first (GDSC v2); add sources via per-source decoder heads + rank-fusion at inference; only attempt unified calibration via linker drugs if rank-fusion empirically underperforms.
NBFNet/ULTRA is right for Track 1 (KG path reasoning) and wrong for Track 2 (cell-line tabular regression). Use a bi-encoder for Track 2 and combine at inference. Don’t make one architecture do both jobs.
Validation is layered (§2.4) and bounded by an epistemic ceiling (§2.5). Layers 1–3 work the standard way on proxy patients (held-out from training source, cross-source, cell-line→patient). The model’s prediction for Maureen IS the intended output — there’s no ground-truth label for “her best drug” to check against. The right framing is decision support, not closed-loop prediction.
The validation surface is the reasoning, not the answer (§2.6). Because we’d only act on a recommendation that looks defensible by inspectable reasoning, validation is — by definition — validation of the logical path. This makes PDR2/ULTRA’s KG paths structurally non-negotiable. Track 2’s scores are supplementary confidence; Track 1’s paths are the validation surface. A black-box regressor without paths is unactionable regardless of accuracy. This also answers “why a KG at all?”: the KG is the substrate on which reasoning paths exist.
MVP: GDSC v2 + bi-encoder regression + Maureen RGCC as held-out validation. A few days of compute. If Spearman ρ > +0.15, the approach is alive.

author	maple
last-verified	2026-09-25
status	planning
description	Using cell lines, PDX models and public cohorts as proxy patients, so the model learns from thousands of outcomes rather than one.

MODS Notebook

Explorer