PDR2 v2 Foundation — RGCC Tier-1 / Tier-2 Validation

Authors: whafa, Claude (Maple project) Date: 2026-05-20 Status: CLOSED. End-to-end pipeline built, container rebuilt (gcr.io/PROJECT/maple-pdr2-training:3fe1dc8-score-edges, Cloud Build 5ac4f1e2), Vertex job 1913996564225851392 ran on A100 in 4 min (after ~5 min PENDING). Tier-1 ρ = +0.181 (n=107, p=0.062 — marginal); Tier-2 AUROC = 0.835 (PASS). ⚠#324 — both computed on pooled RGCC draws; unrestated. ⚠ 2026-09-27: and on memorized labels. See the second correction below. Tier-2 beats every PDR1 Run-B/A1/v1 baseline; Tier-1 is about half of PDR1 Run B’s +0.351 — see §4.

Companion documents:

• docs/pdr_phase7_5_rgcc_writeup.md — PDR1 Phase 7.5 (the validation framework + baseline numbers we’re comparing against).
• docs/pdr2_track_a_v2_repro_writeup.md — the foundation run that produced the checkpoint being validated. §6 (open questions) explicitly tracked this Tier-1 validation as the missing piece.
• Memory: the patient-specific validation writeup.

---

⚠ Correction, 2026-09-24 (GH #317). Every Tier-1 / Tier-2 number below was computed on RGCC labels averaged across Maureen’s two draws (2025-02-15, 2025-09-19). The draws are not replicates: 12 drugs move ≥25 points and they move as two coherent classes — four taxanes down, eight anthracycline / topoisomerase-II agents up — a shift RGCC states in its own page-1 summaries. Averaging sends both classes to mid-range. Test-retest is ρ = +0.9616 over all 112 drugs, ρ = +0.7635 over the 59 graded ones. compute_metrics_crc.py now evaluates per timepoint and refuses to pool; these results need restating against it. The 112-drug / 109-matched panel figures below are correct and unchanged.

⚠ Correction, 2026-09-27 (GH #214, #439). These numbers measure memorization, not generalization. The split flags in §4 were computed per exact triple. So a drug counted as held out whenever its sensitive_to_drug edge was absent, even when the same drug had a partially_sensitive_to_drug or resistant_to_drug edge in train, or was trained under a sibling identifier (oxaliplatin exists both as CHEBI:31941 and as a PharmGKB duplicate). Recounted per drug, across every predicate and identifier, against the foundation’s actual split files:

• 74 of the patient’s 135 RGCC edges were in the training split. By predicate: sensitive 5, partially 35, resistant 34. The §2 table’s figures are approximations.
• 58 of the 107 scored drugs had a label edge in training.
• Four of the five Tier-2 positives were memorized. Cisplatin, oxaliplatin and mitomycin were trained under sensitive_to_drug directly; pemetrexed through its PharmGKB duplicate. Carboplatin, the one held-out positive, ranked 260.
• Per draw, Tier-1 on the 49 drugs whose label never reached training is ρ = −0.051 (2025-02-15) and −0.045 (2025-09-19), neither significant. On the 56 trained drugs with labels it is +0.363 and +0.407.

Claim (a) in §4.1 is therefore wrong, and the “top picks” in (b) are the memorized drugs. The project has no clean RGCC number yet. The first will come from smoke run #1 (docs/mods_smoke_run_1_playbook.md), on a foundation exported without these edges.

Summary

We built the missing per-edge scoring pipeline so the v2 PDR2 foundation (maureen_focus_v1, 2026-05-18, A100-80GB, 21 hr) can be scored against Maureen’s RGCC drug-sensitivity panel under the same Tier-1 / Tier-2 framework that PDR1 was scored against in Phase 7.5. The pipeline is:

1. backend/scripts/score_pdr2_edges.py build-edges — pulls Maureen’s RGCC sensitivities (112 distinct drugs from 224 agent_result rows, identical count to PDR1 Phase 7.5), resolves them to KG drug CURIEs via the v2 composition (mods_kg_entity drug-domain, with the same 30-entry alias map from compute_metrics_crc.py), and emits one edges CSV row per (drug CURIE × predicate). 110/112 drugs resolved (98%), 534 unique edges after collapsing duplicates (e.g., Doxorubicin

   • Liposomal Doxorubicin → same CHEBI).

2. MODE=score_edges in the PDR2 container (new — see lib/pdr2/docker/entrypoint.sh

   • lib/pdr2/code/script/score_edges.py) — loads the foundation checkpoint via the same initialize_model() / entity_relation_dicts.pt path used by eval_patient.py (preserving the Maple entity_dict_forward vocab-key fallback), groups the input edges by (h, r), and does ONE forward pass per group — 3 forward passes total for our 3 predicates. Emits a scores.csv with raw model score, raw rank, filtered rank, and the train/valid/test split each edge falls in.

3. backend/scripts/compute_metrics_rgcc_pdr2.py — joins scores ↔ RGCC ground truth, collapses multiple CURIEs per drug down to one row per drug (max raw_score across CURIEs), computes Tier 1 (Spearman ρ vs score_pct) and Tier 2 (AUROC, high-sens vs low+resistant) per-predicate and per-split-class. Headlines use MAPLE:sensitive_to_drug because that’s the predicate whose semantics align with “rank drugs by predicted sensitivity to Maureen’s tumor.”

A synthetic smoke test (where raw_score = score_pct/100 + N(0, 0.05)) correctly recovers Spearman ρ ≈ +0.91 and AUROC = 1.0, confirming the metrics + join layer wiring is correct end-to-end.

The real run landed on 2026-05-20. Cloud Build 5ac4f1e2-806a-4bd5-9ee7-7536f0f18d2a produced gcr.io/PROJECT/maple-pdr2-training:3fe1dc8-score-edges; Vertex customJob/1913996564225851392 waited ~5 min for A100 capacity and ran for ~4 min (69 s pure scoring across 3 (h, r) groups; the rest was dataset load). 447/534 input edges scored, 87 skipped (drugs whose CURIE wasn’t in the v2 foundation’s training vocab — primarily HMDB and a subset of pharmgkb_variants identifiers, see scores.csv status column).

---

1. Why a Custom Per-Edge Scorer Was Needed

eval_patient.py already exists and reports filtered MRR per predicate on Maureen’s outbound edges. So why couldn’t we use its output for Tier-1 / Tier-2?

Because eval_patient.json records per-predicate aggregates only (MRR, Hits@1/3/10, mean rank). It does not record the raw model score or the raw rank for any individual (h, r, t) triple, which is what Tier 1 / Tier 2 need:

• Tier 1 (Spearman ρ between model score and RGCC % cell death): needs the model’s raw score for each drug, paired with RGCC’s continuous % cell death measurement for the same drug.
• Tier 2 (AUROC, high-sens vs low+resistant): needs the same per-drug raw scores so we can rank them and compute the binary separation.

Per-predicate MRR collapses the per-edge information that the metrics require. So we need a scorer that emits scores at edge granularity.

This is exactly the missing piece called out in [[results/pdr2_track_a_v2_repro_writeup#6-open-questions-and-risks|PDR2 Track A v2 §6 open question 4]]:

“RGCC Tier-1 validation against v2 (Spearman ρ between predicted score and RGCC sensitivity class on Maureen’s 224 drug sensitivities) needs a custom (h, r, t)-scoring script — eval_patient.json records per-predicate aggregates but not per-edge raw scores. Tracked separately.”

This commit set ships that custom scorer.

---

2. Pipeline Architecture

                  ┌─────────────────────────────────────────┐
                  │  mods_functional_test_agent_result      │
                  │  (112 distinct drugs for Maureen,       │
                  │   224 agent_result rows, mean score_pct)│
                  └────────────────┬────────────────────────┘
                                   ↓
   ┌──────────────────────────────────────────────────────────────┐
   │ backend/scripts/score_pdr2_edges.py build-edges              │
   │   • mods_kg_entity drug-domain index (595K names)            │
   │   • RGCC_TO_KG_ALIASES (30 entries, lifted from PDR1)        │
   │   • Collapses duplicate (curie, predicate)                   │
   └────────────────────────────┬─────────────────────────────────┘
                                ↓
       runs/2026-05-20_rgcc_tier1_v2/edges.csv  (534 rows)
                                │
              ┌─────────────────┼──────────────────┐
              ↓ (upload to GCS) ↓                  │
   ┌──────────────────────────────────────┐        │
   │ Vertex CustomJob MODE=score_edges    │        │
   │   • lib/pdr2/code/script/score_edges │        │
   │     .py loads checkpoint + vocab     │        │
   │   • Groups by (h, r) → 3 fwd passes  │        │
   │   • Writes scores.csv to GCS         │        │
   └──────────────────────┬───────────────┘        │
                          ↓                        │
   gs://BUCKET/runs/                  │
       2026-05-20_rgcc_tier1_v2/                   │
       score_edges/scores.csv                      │
                          │                        │
                          ↓ (gsutil cp)            │
   runs/2026-05-20_rgcc_tier1_v2/scores.csv ←──────┘
                          │
                          ↓
   ┌────────────────────────────────────────────────┐
   │ backend/scripts/compute_metrics_rgcc_pdr2.py   │
   │   • Inner-join scores ↔ edges                  │
   │   • Collapse CURIEs (max raw_score per drug)   │
   │   • Tier 1 / 2 per predicate × split-class     │
   └────────────────────────┬───────────────────────┘
                            ↓
              runs/2026-05-20_rgcc_tier1_v2/
              {metrics.json, metrics.matched.csv}

Why score under all 3 predicates

We score each drug under all of MAPLE:sensitive_to_drug, MAPLE:partially_sensitive_to_drug, MAPLE:resistant_to_drug — not just sensitive_to_drug. The cost is free (3 forward passes vs 1 — the (h, r) batching makes per-tail scoring constant), and it lets us diagnose:

• Does the model agree with the overlay’s predicate assignment for each drug? (Compare raw_score under each predicate to the natural_predicate we recorded at edge-build time.)
• Does Spearman ρ hold under the inverse predicate too? (For MAPLE:resistant_to_drug, we expect ρ < 0 against RGCC % cell death — drugs the model says are “more resistant” should have lower % cell death.)
• Are the per-predicate score distributions on the same scale, or does the model give systematically higher scores to one predicate?

Cross-predicate consistency is one of the cleanest tests of whether the model has learned semantics or just memorized labels.

Why the held-out split matters

Maureen’s overlay was built from RGCC sensitivities. The maureen_focus_v1 training data contains:

Predicate	edges in train	edges in valid	edges in test
MAPLE:sensitive_to_drug	5	1	2
MAPLE:partially_sensitive_to_drug	≈ 45	≈ 11	≈ 11
MAPLE:resistant_to_drug	≈ 40	≈ 10	≈ 10

(Exact counts from grep -c "MAPLE:.*_to_drug" runs/2026-05-18_maureen_focus_v1/data/{train,valid,test}.txt.)

So the model has trained on most of these labels. Tier 1 ρ computed over all-drugs is biased by training-set leakage — the model will appear to do better than it has learned to generalize. The honest measurement is Tier 1 ρ computed only on the held-out (valid+test) split. The metrics script reports both, so the reader can see the train-vs-held-out gap directly.

For drugs that are in RGCC but were NOT in the maureen_focus_v1 overlay (CURIEs the overlay materializer couldn’t resolve), the scorer flags split = "unknown" — these are pure generalization-from-graph-context tests.

---

3. Implementation

Files added this commit set

Path	Purpose
lib/pdr2/code/script/score_edges.py	In-container per-edge scorer (mirrors eval_patient.py setup; (h, r) batching)
lib/pdr2/docker/entrypoint.sh	MODE=score_edges dispatch (additive — leaves other modes untouched)
backend/scripts/score_pdr2_edges.py	Local orchestrator: build-edges + submit-job subcommands
backend/scripts/compute_metrics_rgcc_pdr2.py	Tier 1 + Tier 2 metrics (PDR2 port of compute_metrics_crc.py)
backend/scripts/vertex_maureen_focus_v1_score_edges_job.json	Reference Vertex job spec; regenerate with score_pdr2_edges.py submit-job

Files NOT added (deliberately reused or shared)

• No new migrations — the schema is unchanged. mods_functional_test_agent_result is the same table PDR1 reads from.
• No new MCP tool / API route / frontend component — this is a CLI-only analysis pipeline.
• No new dataset class — uses PrimeKGCIViCPharmGKBMaureen identically to the foundation run.

Identity of the model-scoring math

score_edges.py:225-241 and eval_patient.py:236-244 are intentionally identical at the model-call layer:

# both files
triple = torch.tensor([[h_idx, t_idx, r_idx]], device=device)
t_batch, _ = tasks.all_negative(test_data_gpu, triple)
t_pred = model(test_data_gpu, t_batch)  # [1, num_entities]
scores = t_pred[0]                       # [num_entities]
raw_score = float(scores[t_idx].item())
raw_rank = int((scores > scores[t_idx]).sum().item()) + 1

A sample edge scored through score_edges.py will produce the same raw_score and raw_rank that eval_patient.py would compute for the same (h, r, t) triple. The only difference is score_edges.py emits the raw score whereas eval_patient.py keeps only the rank.

Why a container rebuild is required

The c5c17a5-eval-fix image baked eval_patient.py, evaluate_personalized_maple.py, and the other current scripts in at /opt/pdr2/code/script/. score_edges.py is new and won’t appear in that image. The entrypoint dispatch also needs the new MODE=score_edges case.

Build:

gcloud builds submit \
    --config cloudbuild-pdr2-training.yaml \
    --substitutions=_TAG=$(git rev-parse --short HEAD)-score-edges

Then bump the imageUri in vertex_maureen_focus_v1_score_edges_job.json to the new tag.

---

4. Results

4.1 Headline (predicate: MAPLE:sensitive_to_drug, all 107 matched drugs)

Metric	Value	Threshold	Decision
Tier 1 — Spearman ρ vs RGCC % cell death	+0.181 (p = 0.062, n = 107)	ρ > 0 and p < 0.05	FAIL (marginal — just over the 0.05 threshold)
Tier 2 — AUROC (high-sens vs low + resistant)	0.835 (AUPR = 0.606, n_pos = 5 / n_neg = 85)	AUROC > 0.6	PASS

Two facts dominate the interpretation:

(a) Under MAPLE:sensitive_to_drug, only 5 of 107 matched drugs appear in the model’s training overlay (3 train + 1 valid + 1 test). The other 102 are in the unknown split — drugs whose KG CURIE the maureen_focus_v1 overlay never reached. So both metrics are measuring graph-context generalization, not memorization. (Counts under partially_sensitive_to_drug and resistant_to_drug are denser by training overlay — see §4.3 splits.) (⚠ 2026-09-27: wrong; see the correction at the top. Counted per drug across all predicates and identifiers, 58 of the 107 drugs had a label edge in training, and four of the five positives were memorized.)

(b) The model’s top-4 picks under MAPLE:sensitive_to_drug are Cisplatin (rank 1), Pemetrexed (rank 1), Gemcitabine (rank 3), Oxaliplatin (rank 2). Three of those four (Cisplatin, Pemetrexed, Oxaliplatin) are RGCC high-sensitivity drugs (81+% cell death). The exception is Gemcitabine (RGCC low, 10%) — a known CRC-relevant nucleoside analog with strong KG support that doesn’t kill her CTCs in vitro. Mitomycin (RGCC high, 81.5%) lands at rank 42; Carboplatin (RGCC high, 80.5%) is the dramatic outlier at rank 260, single-handedly suppressing the Spearman ρ.

4.2 Score distribution by RGCC class (under MAPLE:sensitive_to_drug)

RGCC class	n	mean raw_score	median raw_score	median rank
high_sensitivity	5	+2.55	+4.00	2
low_sensitivity	35	−0.25	+0.11	91
moderate_sensitivity	17	−0.36	+0.22	82
resistant	50	−0.40	−0.26	119.5

The mean-score stratification is clean and monotone in the right direction — high-sensitivity drugs sit at +2.55, with low / moderate / resistant clustering near zero and resistant landing the lowest. This is the signature of a model with strong binary discrimination (driving the Tier 2 AUROC = 0.835) but lossy continuous ranking (driving the Tier 1 ρ = +0.18 instead of something larger).

4.3 Per-predicate breakdown (cross-predicate consistency check)

We scored each drug under all three predicates (~free under (h, r) batching). The question: does the model respect predicate semantics, or is it just scoring “this drug is relevant to Maureen”?

Predicate	n	ρ vs RGCC %	AUROC (high-sens vs low+resistant)	overlay splits (train/valid/test/unknown)
MAPLE:sensitive_to_drug (headline)	107	+0.181	0.835	3 / 1 / 1 / 102
MAPLE:partially_sensitive_to_drug	107	+0.158	0.706	26 / 6 / 8 / 67
MAPLE:resistant_to_drug	107	−0.006	0.673	27 / 12 / 4 / 64

Read this table as the diagnostic for predicate semantics:

• The sensitive_to_drug predicate yields the strongest positive ρ and highest AUROC, as semantics would predict. ✓
• The resistant_to_drug predicate yields a near-zero ρ — exactly the “noise floor” we’d expect if the model had completely decorrelated the score from RGCC % under this predicate. We did not see ρ go negative (which is what would happen if the model also correctly learned the inverse semantics of “resistant”). Conclusion: the foundation has learned that sensitive_to_drug-relevant drugs are different from resistant_to_drug-relevant drugs, but the resistant predicate acts more as “drugs not related to me” than “drugs that affect me but negatively.”
• AUROC > 0.5 under all three predicates (including resistant_to_drug at 0.67) confirms the model does score Maureen’s high-RGCC drugs higher than her low-RGCC drugs regardless of predicate — i.e., there’s a strong “is this drug Maureen-relevant?” component shared across all three predicates, on top of the predicate-specific signal.

The high-overlay-count predicates (partially and resistant) get worse Tier-1 ρ than the low-overlay-count predicate (sensitive_to_drug) despite having more training signal, which is another datapoint suggesting cross-predicate noise rather than per-predicate learning — the same architectural finding the v1 writeup made about λ=0.01 personalization saturating.

4.4 Comparison to PDR1 Phase 7.5

PDR1 Phase 7.5 (2026-05-09, writeup) reported the following on the SAME RGCC drug panel (Maureen’s 112 distinct drugs, 109 of which resolved to PrimeKG):

Checkpoint	KG	Tier 1 ρ	Tier 2 AUROC
primekg v1 (corr 1L 300d)	PrimeKG	+0.204	0.649
primekg A1 (sub 1L 300d)	PrimeKG	+0.184	0.751
primekg Run B (corr 1L 400d)	PrimeKG	+0.351 ⚠#324	0.689 ⚠#324
gpkg v4 (no skip_head)	GP-KG	+0.391	n/a (4 of 5 high-sens drugs absent)
PDR2 v2 (this run, ULTRA on maureen_focus_v1)	PrimeKG+CIViC+PharmGKB+OncoKB+ClinVar	+0.181	0.835

Split decision: PDR2 v2 wins Tier 2 by a solid margin (0.835 vs A1’s 0.751 — best PDR1; +0.084 absolute / +11% relative) and loses Tier 1 to PDR1 Run B by about half (+0.181 vs +0.351).

This split is consistent with the §4.3 finding that PDR2 v2’s strength is binary discrimination of extremes (high-sens vs resistant) and its weakness is fine-grained continuous ranking. Why this asymmetry might exist:

• Architecture. PDR1 is CompGCN+ConvE; PDR2 is ULTRA+NBFNet. ULTRA was designed for inductive transfer across KGs — it learns relation types as a graph of meta-relations and embeds nodes via the labeling-trick. NBFNet’s iterative message passing tends to produce a few high-scoring tails per query and many low-scoring ones — a “spiky” score distribution. That’s exactly the pattern that helps AUROC (you can clearly separate the few high-scoring drugs from the bulk) and hurts Spearman ρ (the bulk has uninformative continuous order).
• Carboplatin specifically. Removing Carboplatin from the matched set (n=106, RGCC high) recovers ρ to ≈ +0.26. The model places it at rank 260, which is wildly wrong even for a noisy ranker. Cisplatin and Oxaliplatin both rank in the top 4, so the model understands platinum-class CRC drugs in general; it’s missing something specific about Carboplatin’s KG embedding. Worth a dedicated dive — possibly an entity-alias issue in the overlay (Carboplatin in the v2 KG has different CURIE source coverage).
• More raw signal, but on a sparser learning target. v2’s training data has 4.55M edges vs PrimeKG-only’s 8.1M. PDR2 v2 has more Maureen-specific information (variant-aware overlay, OncoKB LEVEL_1 edges) but less general drug-disease-gene density to ground its continuous score on. Less density typically hurts ρ-style metrics more than AUROC-style metrics.

The headline takeaway: PDR2 v2 succeeds where it matters most clinically (binary high-sens vs resistant discrimination) and lags PDR1 Run B on the more general continuous-ranking task. For a decision-support use case (which CRC drugs should we consider for Maureen?), AUROC = 0.835 is operationally far more useful than ρ = +0.351 with AUROC = 0.689; the model with the higher Tier 2 is better at surfacing the actual responders even when its Tier 1 is weaker.

4.5 Caveats inherited from PDR1 Phase 7.5

All of Phase 7.5 §5.4 applies verbatim here. Most pointed:

• n_pos = 5 for Tier 2. The AUROC of 0.835 is computed against 5 high-sensitivity drugs. With this sample size the metric has high sampling variance; a single misclassified high-sens drug (Carboplatin at rank 260, in this run) significantly moves the number. The qualitative story (high-sens drugs cluster at the top of the model’s ranking) is robust; the exact AUROC is not.
• In vitro CTC ≠ in vivo response. The model’s high-confidence picks include Cetuximab (RGCC low, 12.5%), Bevacizumab (RGCC low, 22.5%), and Ramucirumab — three first/second-line CRC biologics that don’t kill cells in a 48-h Annexin-V assay but are clinically central. The model is correctly surfacing them from KG context; the RGCC assay can’t credit them. Same observation as PDR1 Run B (Phase 7.5 §4.8).
• No personalization in this score. This evaluates the baseline foundation. The fine-tune bundle (evaluate_personalized at λ=0.01) showed Δ_MRR ≈ -0.01% in v2 §4.5 so we wouldn’t expect personalization to materially change these numbers either — but a higher-λ sweep on the scoring side would be the natural follow-up.

---

5. Limitations and Caveats

1. Training-set leakage on overlay drugs. The maureen_focus_v1 overlay materialized 135 sensitivity edges into the training set, built from the same RGCC data. Tier-1 ρ on those drugs is partly a memorization test. The metrics script reports valid+test as the honest held-out measurement. The 2 RGCC drugs that didn’t resolve (Catumaxumab, Everolimus/Temsirolimus combo cell) and any RGCC drugs that resolved to KG entities NOT in the overlay are pure generalization tests (split = "unknown").
2. In vitro CTC ≠ in vivo response. Inherited from Phase 7.5 §5.4. RGCC measures Annexin-V+ at fixed concentrations in a 72-h (chemo) / 48-h (targeted) assay on Maureen’s CTCs. It does not measure pharmacokinetic exposure, drug-drug interactions, or metastatic-niche effects. Prodrugs (Capecitabine → 5-FU) typically underperform in vitro vs their active metabolite; the model can’t distinguish them mechanistically.
3. Drug-name normalization is conservative. The 30-entry alias map from compute_metrics_crc.py was hand-curated for PrimeKG drug names. v2’s composition includes pharmgkb_variants and HMDB sources whose naming conventions are slightly different; we don’t yet have an exhaustive map for those edges. Two drugs (Catumaxumab, Everolimus/Temsirolimus combo cell) remain unmatched.
4. Multi-CURIE-per-drug collapsing. v2’s KG carries many drugs under multiple CURIEs (Cisplatin: CHEBI:27899, HMDB:HMDB0014656, pharmgkb_variants:pharmgkb_drug:cisplatin). We collapse to the max raw_score across CURIEs at metrics time. The rationale: alias mismatch between sources shouldn’t penalize the drug. Downside: a poorly-connected CURIE never gets credit for being the “less informative” one, even if its low score is more honest. Inspect metrics.matched.csv if a high-sens drug looks anomalous; the n_curies column shows how many CURIEs were collapsed.
5. Single patient. Same as Phase 7.5 §5.4. All findings pertain to Maureen specifically; cross-patient validation needs a cohort that does not currently exist in the Maple data.

---

6. Reproducibility

A) Build the edges CSV (local, requires Cloud SQL proxy)

.\start_local_prod_db.ps1    # if not already running
cd backend
python scripts/score_pdr2_edges.py build-edges `
    --out ../runs/2026-05-20_rgcc_tier1_v2/edges.csv `
    --preview

Expected output: Built 534 edges (110 drugs × ~1.7 CURIEs × 3 predicates), Wrote 2 unmatched diagnostics to ...edges.unmatched.json.

B) Rebuild the container

gcloud builds submit \
    --config cloudbuild-pdr2-training.yaml \
    --substitutions=_TAG=$(git rev-parse --short HEAD)-score-edges

This commit set’s build: 5ac4f1e2-806a-4bd5-9ee7-7536f0f18d2a → gcr.io/PROJECT/maple-pdr2-training:3fe1dc8-score-edges.

Heads-up (one-time .dockerignore patch required): the working tree at the time of this commit had lib/ in .dockerignore (to exclude the 69 MB PDR1 paper mirror at lib/pdr/), which also excludes lib/pdr2/ that the PDR2 image needs. A !lib/pdr2/ re-include line was added at .dockerignore:91. If you regenerate the dockerignore from scratch, keep that re-include or the build will succeed but produce a broken image (Dockerfile COPY of lib/pdr2/code/ finds nothing).

C) Render + submit the Vertex job

# Upload edges CSV
gsutil cp runs/2026-05-20_rgcc_tier1_v2/edges.csv `
    gs://BUCKET/runs/2026-05-20_rgcc_tier1_v2/edges.csv
 
# Render job spec
cd backend
python scripts/score_pdr2_edges.py submit-job `
    --edges ../runs/2026-05-20_rgcc_tier1_v2/edges.csv `
    --out ../runs/2026-05-20_rgcc_tier1_v2/vertex_job.json `
    --ckpt-uri gs://BUCKET/runs/2026-05-18_maureen_focus_v1_foundation_scratch/train/latest_state_dict.pt `
    --data-uri gs://BUCKET/runs/2026-05-18_maureen_focus_v1_foundation_scratch/data `
    --output-uri gs://BUCKET/runs/2026-05-20_rgcc_tier1_v2 `
    --edges-uri gs://BUCKET/runs/2026-05-20_rgcc_tier1_v2/edges.csv `
    --run-id 2026-05-20_rgcc_tier1_v2_score_edges `
    --image-uri gcr.io/PROJECT/maple-pdr2-training:<sha>-score-edges
 
# Submit (paste from script's stdout, or directly)
gcloud ai custom-jobs create `
    --region=us-central1 `
    --display-name="2026-05-20_rgcc_tier1_v2_score_edges" `
    --config=runs/2026-05-20_rgcc_tier1_v2/vertex_job.json

Expected wallclock: ~15 min (dataset loading dominates; the scoring itself is 3 forward passes ≈ seconds).

D) Download scores + compute metrics (local)

gsutil cp gs://BUCKET/runs/2026-05-20_rgcc_tier1_v2/score_edges/scores.csv `
    runs/2026-05-20_rgcc_tier1_v2/scores.csv
gsutil cp gs://BUCKET/runs/2026-05-20_rgcc_tier1_v2/score_edges/scores.summary.json `
    runs/2026-05-20_rgcc_tier1_v2/scores.summary.json
 
python backend/scripts/compute_metrics_rgcc_pdr2.py `
    --scores runs/2026-05-20_rgcc_tier1_v2/scores.csv `
    --edges  runs/2026-05-20_rgcc_tier1_v2/edges.csv `
    --out    runs/2026-05-20_rgcc_tier1_v2/metrics.json `
    --run-id 2026-05-20_rgcc_tier1_v2

Reference artifacts

Path	Contents
runs/2026-05-20_rgcc_tier1_v2/edges.csv	534 edges built from RGCC + KG drug index
runs/2026-05-20_rgcc_tier1_v2/edges.unmatched.json	2 unmatched RGCC drugs (Catumaxumab, Everolimus/Temsirolimus combo cell)
runs/2026-05-20_rgcc_tier1_v2/scores.csv	447 scored + 87 skipped (CURIE not in foundation vocab)
runs/2026-05-20_rgcc_tier1_v2/scores.summary.json	scorer telemetry (3 (h,r) groups, 69 s eval, 102,914 entities)
runs/2026-05-20_rgcc_tier1_v2/stdout.log	container stdout (downloaded from GCS)
runs/2026-05-20_rgcc_tier1_v2/metrics.json	Tier 1 + Tier 2 per predicate × split
runs/2026-05-20_rgcc_tier1_v2/metrics.matched.csv	per-drug join (107 rows × 3 predicates)
mods_functional_test_agent_result	Source of truth for RGCC sensitivities
mods_kg_entity (drug domain)	Source of truth for KG drug CURIEs in the v2 composition
gs://BUCKET/runs/2026-05-20_rgcc_tier1_v2_score_edges/	GCS mirror of all of the above (edges + scores + summary + stdout)

---

7. Parallelization Analysis (CLAUDE.md §4)

Surfaces touched

• Backend API: none
• Frontend component: none
• Worker handler: none
• Migration (backend/migrations.py): none — schema unchanged
• MCP tool: none
• DB-stored config: none
• Docs: new file docs/pdr2_rgcc_tier1_v2_writeup.md (this file)
• Tests: none added (the in-container scorer is hard to unit-test outside the PDR2 container; the smoke test against synthetic scores is documented in §4.4 of the pipeline implementation rather than as pytest)
• CLI scripts: new backend/scripts/score_pdr2_edges.py, new backend/scripts/compute_metrics_rgcc_pdr2.py, new lib/pdr2/code/script/score_edges.py, edit lib/pdr2/docker/entrypoint.sh, new backend/scripts/vertex_maureen_focus_v1_score_edges_job.json

Parallel split

• The 4 new files (score_pdr2_edges.py, compute_metrics_rgcc_pdr2.py, score_edges.py, the Vertex job spec) are independent and could in principle be developed in parallel sessions.
• The entrypoint.sh edit is the only shared-file change — additive (new MODE case + new dispatch block) so low collision risk, but serialize with any other entrypoint.sh edit.

Collision hotspots (CLAUDE.md §Planning §3)

• lib/pdr2/docker/entrypoint.sh — recently touched in commit c5c17a5 (2026-05-19). Verified clean in main before editing; the new score_edges MODE block sits between the eval_patient and evaluate_personalized blocks and does not modify any existing dispatch.
• No other hotspots from CLAUDE.md’s list are touched (no migrations.py, component_registry.py, main.py, ComponentLoader.jsx, Terminal.jsx, MEMORY.md, CLAUDE.md, package.json, requirements.txt).

Verify-how per surface

• backend/scripts/score_pdr2_edges.py build-edges — local invocation produces a valid edges CSV (verified — 534 rows, 110/112 drugs resolved, 2 unmatched drugs in diagnostic JSON).
• backend/scripts/compute_metrics_rgcc_pdr2.py — synthetic-score smoke test recovers ρ ≈ +0.91 / AUROC = 1.0 against a deliberately noisy raw_score = score_pct/100 + N(0, 0.05) (verified). The validate=“one_to_one” merge correctly catches duplicate keys.
• lib/pdr2/docker/entrypoint.sh — verified end-to-end by the real Vertex run (build 5ac4f1e2 → image 3fe1dc8-score-edges → job 1913996564225851392 completed SUCCEEDED in 4 min).
• lib/pdr2/code/script/score_edges.py — produced scores.csv with 447/534 input edges scored (87 skipped CURIEs flagged in status column), exactly the (h, r) batching contract (3 groups, 1 forward pass per group, 69 s total).
• .dockerignore — required a one-line !lib/pdr2/ re-include fix (line 91) on top of the working-tree-already-present lib/ exclusion. Without it, the Dockerfile’s COPY lib/pdr2/code/ would silently produce an image missing the training code. Build 5ac4f1e2 was the first run with the fix.

---

8. Open Questions After This Run

1. Carboplatin rank 260 outlier. RGCC high-sensitivity, 80.5% cell death, KG should know it as a platinum-class drug closely related to Cisplatin (rank 1) and Oxaliplatin (rank 2). Why does the v2 foundation place it at rank 260? Inspect the CURIE resolution: is the Carboplatin entity in mods_kg_entity poorly connected in the v2 composition relative to Cisplatin / Oxaliplatin? Removing Carboplatin from the n=107 set lifts Tier-1 ρ to ≈ +0.26 — meaningfully closer to PDR1 Run B’s +0.351. If this is a CURIE-coverage gap, it likely affects other drugs too and is a composition-quality improvement target.
2. Why does partially_sensitive_to_drug underperform sensitive_to_drug despite having 5× the training overlay (33 overlay drugs vs 5)? Either the moderate-sensitivity edges in training are too noisy (RGCC moderate = 35-79% spans a wide range collapsed into one predicate), or cross-predicate noise is leaking signal from the better-trained predicate. The §4.3 cross-predicate consistency pattern is suggestive but doesn’t prove which.
3. Predicate semantics under resistant_to_drug. The model correctly decorrelates score from RGCC % (ρ ≈ -0.006) under this predicate but does not invert the sign. A model that fully learned semantics should produce ρ < 0 — drugs the model says are most-resistant should have LOWER RGCC %, not random. Worth designing a targeted test for this (e.g., compare per-drug sensitive_to_drug score MINUS resistant_to_drug score against RGCC %; if semantics is learned, this difference should correlate much better than either score alone).
4. The 87 skipped CURIEs. Drugs whose KG CURIE wasn’t in the foundation’s training vocab. Inspecting scores.csv status column shows the breakdown (mostly HMDB and some pharmgkb_variants). Either (a) the foundation should ingest these CURIE families in the next composition, or (b) the orchestrator should filter to the CURIE families known to be in-vocab before emitting the edge list. Trade-off: option (b) sacrifices coverage for clean reporting; option (a) needs another full training run.
5. Personalization re-test. This evaluated baseline foundation only. Re-run with the fine-tune bundle (evaluate_personalized-style per-entity score adjustment) at higher λ values to see whether the criterion adjustment can lift Carboplatin specifically (the ATM criterion weight at +0.0249 in v2’s fine-tune signature should boost platinum-class drugs given Maureen’s ATM-low expression). The structural finding from v2 §4.5 says λ=0.01 is too small; the λ sweep results pending separately.

---

9. References

• Phase 7.5 writeup (pdr_phase7_5_rgcc_writeup.md)
• PDR2 Track A v2 writeup (pdr2_track_a_v2_repro_writeup.md)
• Galkin M et al. Towards foundation models for knowledge graph reasoning (ULTRA). ICLR 2024.
• Oğuztüzün C et al. PDR2 — the paper whose mechanism we are validating against patient-specific ground truth.
• RGCC International. Onconomics Plus chemosensitivity assay. Methodology in lib/pdr/Supplementary_Material/.