MODS Notebook

Properties4
authormaple
last-verified2026-09-25
statuswriteup
descriptionHow PDR1 and PDR2 differ in where the patient enters the model, and what each mechanism actually does at training time.

28 min read

PDR Personalization Process — Technical Writeup

Authors: whafa, Claude (Maple project) Date: 2026-05-16 Status: Reference document. Describes the personalization mechanisms used by PDR1 (Oğuztüzün et al., J Biomed Inform 2025) and PDR2 (Oğuztüzün et al., J Biomed Inform 179, 2026) — verbatim from the upstream code — and audits Maureen’s data inventory against the inputs each model demands.

Companion documents:

Summary

PDR1 and PDR2 personalize a knowledge-graph link-prediction model through two structurally different mechanisms. PDR1 puts the patient signal into the graph — every personalized fact (a PRS, a biomarker value, a diagnosis, a medication) becomes a per-patient self-loop edge, and the scalar magnitudes are loaded as a sidecar tensor that multiplicatively modulates the relation embedding inside forward_embedd. PDR2 keeps the patient signal outside the graph — the patient appears in the KG only as a node with a small set of WAS_DIAGNOSED edges, and the PRS / expression vectors enter only at fine-tune time through a personalized loss term whose learnable per-disease and per-protein weights ride on top of a frozen foundation model. Both models accept the same four discrete patient data types (PRS, biomarker expression, diagnosis history, medication), but PDR1 needs them shaped as a CSV row and a .npy scalar tensor, while PDR2 needs them shaped as a pickle dict and a per-patient overlay subgraph.

For Maureen we have abundant somatic data (84 nonzero gene-expression values from Caris MI Profile, 8 actionable somatic + germline variants across Caris / Invitae / Guardant, 224 RGCC drug-sensitivity readings) but the PRS column is presently all zeros because no genotype VCF has been uploaded — making Track A (synthetic_zeros PRS + real biomarkers) the only personalization track that runs today. The PRS pipeline (PGS Catalog connector + mods_polygenic_score table + 13-disease default scoring-file set) is built and waiting for genotypes; Track B unblocks the moment a 23andMe / WGS / VCF lands. Additional comorbidity / variant-level coverage is obtainable with targeted work — re-resolving Maureen’s HGVS-p strings through ClinGen, finishing the OncoKB / PharmGKB ingest, and ordering missing PGS Catalog scoring files — but the patient→variant edge fix and the OncoKB ingest are the two near-term unblocks for moving from “gene-level personalization” to the variant-level personalization that the somatic data actually merits.

1. Purpose and Scope

This writeup is a reference for what data each PDR generation actually consumes for personalization, in what shape, and where in the codebase that consumption happens. It is the document to consult before any future debate of the form “could we just feed Maureen’s X into PDR-Y?” because it pins down the exact tensor / table / column / line of upstream code that X has to land in for Y to use it.

Scope:

  • §2 catalogs the four discrete patient data types both papers use, with the source-of-truth column / field for each.
  • §3 documents PDR1’s scalar-modulated graph convolution personalization — the patient_merge.py self-loop construction, the scalars_<pid>.npy tensor, and the multiplicative use inside forward_embedd.
  • §4 documents PDR2’s frozen-foundation, learned-weights personalization — the pickle dict format, the PersonalizedRepurposingLoss, the patient-specific fine-tune (1,802 trainable weights), and the test-time apply_personalization adjustment.
  • §5 is a side-by-side comparison.
  • §6 audits Maureen’s data against both schemas — what is materialized today, what is held at zero as a controlled placeholder, what is obtainable with bounded follow-up work, and what is blocked on infrastructure / quota.
  • §7 lists open questions and risks.

Out of scope: PDR0 / KG-Predict, the in-graph link-prediction mechanics themselves (CompGCN+ConvE in PDR1; ULTRA+NBFNet in PDR2; both covered in pdr_phase0_gpkg_writeup.md and nbfnet_understanding.md), and our own Phase 12 / Phase 13 patient-as-node extensions (covered in paan_planning.md and temporal_patient_modeling_planning.md).

2. The Four Discrete Patient Data Types

PDR1 and PDR2 accept the same four data types for personalization. They differ only in how the data is shaped and where it lands inside the model.

TypeGranularityModalitySource-of-truth (this paper)Source-of-truth (Maple DB)
Polygenic Risk Score (PRS)One scalar per diseaseContinuous, often z-scoredUK Biobank Showcase fields 26200–26285 (Standard PRS Release; 28 binary + 25 quantitative traits)mods_polygenic_score(patient_id, pgs_id, disease_cui, raw_score, z_score, percentile, snps_used, ...)
Biomarker expressionOne scalar per geneContinuous (TPM, qPCR ΔΔCt, or assay-dependent)UK Biobank Olink protein panel (Phase III)¹ / blood biomarker fieldsmods_gene_expression(patient_id, gene, expression_value, test_type, report_date)
Diagnosis historySet membership (binary)CategoricalUMLS CUI list parsed from the CUIs cell of the patient CSVmods_condition, mods_kgia_profile_service._build_* outputs
Treatment / medication historySet membership (binary)CategoricalDrugBank IDs in the Treatment/medication code | Instance 0 cellmods_medication, mods_treatment_drug, mods_treatment_episode

¹ The PDR papers operationalize “biomarker” as protein / gene expression, not the broader clinical use of “biomarker” (CEA, CA 19-9, LDH, etc., which Maple tracks separately under mods_biomarker). When a writeup uses the unqualified word “biomarker” in PDR context it means a gene-keyed expression value.

These four are what the papers implement. They are point-in-time — neither paper consumes trajectories, change rates, or repeated measures (covered in temporal_patient_modeling_planning.md as our Phase 13 research direction).

3. PDR1 — Scalar-Modulated Graph Convolution

PDR1 personalizes a single global model by rewriting the knowledge graph per patient. Every personalized fact becomes a self-loop edge (h, scalar, h) (where h is a disease CUI or gene symbol and scalar is the patient’s value for that feature), then a sidecar tensor of scalars is loaded into the model and used to multiplicatively modulate the relation embedding during message passing.

3.1 Input shape — the patient CSV row

patient_merge.py reads a CSV with one row per UK Biobank participant. Relevant columns:

ColumnTypeMeaning
EIDintUK Biobank patient ID
Standard PRS for alzheimer's disease (AD)floatThe headline PRS for the target disease (AD case study)
PRS_<CUI> (×~13)floatComorbidity PRS columns, keyed by UMLS CUI (e.g. PRS_C0006142 = breast cancer)
Treatment/medication code | Instance 0string-encoded listPython-literal list of DrugBank IDs (['DB00001', 'DB00945', …])
CUIsstring-encoded listPython-literal list of diagnosis CUIs from the patient’s encounter history
<gene_symbol> (×~hundreds)floatOne column per gene with a biomarker / expression value

The Maple builder for this row is backend/scripts/build_pdr1_input_csv.py; the staged Track A row for Maureen is at runs/2026-05-09_phase9/maureen_track_a_pdr1.csv.

3.2 In-graph representation — patient_merge.py

Patient_Merge.merge_nodes() (docs/References/PDR1/patient_merge.py:63) materializes the per-patient KG by appending five families of self-loop edges to the global kg_df:

MethodSource columnEdge producedRelation column type
filter_and_match_disease_nodes_PRSStandard PRS for alzheimer's disease (AD)(AD_CUI, prs_value, AD_CUI)scalar (float)
filter_and_match_disease_nodes_PRS_comorbitieseach PRS_<CUI>(CUI, prs_value, CUI)scalar (float)
filter_and_match_disease_nodes_DisHistCUIs list(CUI, 'HAS_DIAGNOSED', CUI)symbolic
filter_and_match_drug_nodesTreatment/medication code(intersection only — no edges written in shipped code; commented out at L91–96)n/a
filter_and_match_gene_nodes_Biomarkerseach gene-keyed expression column(gene, expr_value, gene)scalar (float)

The patient’s drug history is the one type the shipped PDR1 code does not actually write into the graph — the self-loop creation block is commented out. The drug list is used only for entity-set intersection with the KG’s DB* prefix (DrugBank-style nodes); GP-KG instead uses CID* prefix (PubChem + 100M offset, the reproducibility and ent2id notes), which is one of the two known patches required before patient_merge.py will run on GP-KG (the other is the dead UMLS REST endpoint at L111).

3.3 The scalars_<pid>.npy sidecar tensor

Patient_Merge.save() (patient_merge.py:206) emits a 1-D tensor scalars of length len(kg_df) — one entry per triple in the patient-merged KG. For each row:

if isinstance(value, (int, float)):
    scalars[i] = value   # PRS or biomarker value
else:
    scalars[i] = 1.0     # symbolic relation (TARGET, ASSOCIATE, HAS_DIAGNOSED, ...)

Numeric “relation” cells are then replaced with the literal string 'SELF' before the merged KG is written to disk, because the downstream data_loader.py expects a categorical relation column. The scalar magnitude survives only in the .npy sidecar; the merged KG itself looks like a vanilla CompGCN input.

Output files:

  • outputs/patient_merged_kg_<pid>.txt — tab-separated triples, scalars replaced with 'SELF'
  • outputs/scalars_<pid>.npy — float tensor aligned to the triple order

3.4 Training-time use — forward_embedd

The scalar tensor is loaded into the model via the -scalars CLI flag (PDR1-specific marker; absent from kg-predict, the reproducibility and ent2id notes). Inside GPKG_EMBEDD:

# In __init__:
loaded_tensor_numpy = np.load(self.p.scalars)
self.scalars = torch.from_numpy(loaded_tensor_numpy).to('cuda:0')
 
# Relation embedding is sized to scalars.size()[0] (not 2*num_rel) when scalars are present:
self.init_rel = get_param((self.scalars.size()[0], self.p.init_dim))

Then in forward_embedd (predict.py:51):

r = self.init_rel
if self.scalars is not None:
    r = r * self.scalars.unsqueeze(1)   # ← THE PERSONALIZATION STEP
x, r = self.conv1(self.init_embed, self.edge_index, self.edge_type, rel_embed=r)

The unsqueeze broadcasts a [len(triples), 1] scalar column against the [len(triples), init_dim] relation embedding matrix, multiplying each relation’s embedding by the patient’s scalar value for that edge. Symbolic edges (scalar = 1.0) are unchanged; scalar edges scale by the patient’s PRS / expression magnitude. Higher PRS for disease d → larger gradient flow through edges incident to d → the model’s downstream drug ranking is pulled toward drugs that connect strongly to high-magnitude diseases for this patient.

The Cohen’s d ≈ 1.05 lift the paper reports for PRS is the strongest of any single personalizer in their ablation. The mechanism is unambiguously the relation-embedding scale — there is no other personalization pathway in PDR1’s architecture.

3.5 Implementation status (Maple)

  • ✓ build_pdr1_input_csv.py builds Maureen’s CSV (Track A row staged; PRS columns zero pending genotypes).
  • ✓ Migration 052 + Caris / RGCC parsers populate mods_gene_expression (84 nonzero values for Maureen).
  • ✗ patient_merge.py patches deferred: startswith('DB') drug filter doesn’t match GP-KG’s CID*; UMLS REST endpoint with embedded API key at L111 is dead (UMLS_apikey = "68297838-..." — replace with _resolve_cuis_via_translator(...) or pre-compute the AD CUI list).
  • ✗ Phase 8 AD repro deferred — establishes the wiring gate before Phase 9.
  • ✓ compute_metrics_crc.py exists for Tier 1 / Tier 2 RGCC scoring once a -scalars-trained checkpoint is produced.

4. PDR2 — Patient-as-Node + Frozen-Foundation Fine-Tune

PDR2 personalizes through a structurally different mechanism. The patient appears in the KG as a node (not just as a row in a CSV), but with minimal edges (WAS_DIAGNOSED only); the PRS and biomarker vectors are not put into the graph at all. They enter through the loss function during a per-patient fine-tune that freezes the entire foundation model and trains only 1,802 personalization weights.

4.1 Input shape — the patient pickle

utils.load_patient_data expects a dict with four required keys:

KeyShapeTypeSemantics
prs_values[len(disease_names)]np.float32Patient’s PRS for each disease, in the same order as disease_names
expr_values[len(protein_names)]np.float32Patient’s expression value for each gene, in the same order as protein_names
disease_names[len(disease_names)]list[str]UMLS CUIs — e.g. ['C0006142', 'C0011860', ...]
protein_names[len(protein_names)]list[str]HGNC gene symbols — e.g. ['TP53', 'BRCA1', ...]

PDR2 z-scores both vectors at load time (old_runs/run_drugrec.py:331–332), so the builder emits raw values.

The Maple builder is backend/scripts/build_pdr2_input_pickle.py — three source modes (synthetic_zeros, maureen, ukb_sample). The default 13-disease CUI set (CRC + 12 comorbidities) is sourced from backend/connectors/pgs_catalog_connector.py:261 (UKB_TRAIT_TO_PGS_DEFAULTS). Genes are sourced from mods_gene_expression with a test-type priority (Caris MI Profile > Caris WTS > RGCC qPCR).

4.2 In-graph representation — patient as a node, minimally

Unlike PDR1’s self-loops, PDR2 has the patient appear as a first-class graph entity with outbound edges. From the personalization mechanism reference: foundation training augments the GP-KG with patient nodes that each carry ~37 WAS_DIAGNOSED edges (in the original UK Biobank corpus). The foundation is trained on this augmented graph end-to-end — i.e. the model does see patient nodes during foundation training; it just sees nothing else about them beyond their diagnosis set.

PRS and biomarker values are not in the graph. They appear nowhere as nodes, edges, or scalar modulators. This is a deliberate architectural choice (and a major contrast with PDR1).

For Maureen, the canonical KG carries MAPLE_PATIENT:maureen-1173a8db with 250 outbound canonical relations materialized by backend/mods_patient_kg_service.py::materialize_patient_kg (the variant-aware KG extension) — already substantially richer than the 37-edge UK Biobank patient overlay, even before any of the variant resolver fix or OncoKB ingest lands.

4.3 The personalized loss — PersonalizedRepurposingLoss

losses.py:37 defines the personalization mechanism. The class holds two parameter tensors:

self.disease_weights = nn.Parameter(torch.randn(len(disease_names)) * 0.01)
self.protein_weights = nn.Parameter(torch.randn(len(protein_names)) * 0.01)

These are the only learnable parameters during patient-specific fine-tune. In the paper’s setup len(disease_names) = 341 and len(protein_names) = 1,461, giving 1,802 total trainable weights. The forward pass:

def forward(self, pos_scores, neg_scores, prs_values, expr_values):
    base_loss = StandardKGLoss()(pos_scores, neg_scores)
    
    prs_term = torch.tensor(0.0, device=pos_scores.device)
    expr_term = torch.tensor(0.0, device=pos_scores.device)
    
    if self.disease_weights is not None and len(prs_values) > 0:
        disease_weights_norm = F.normalize(self.disease_weights, p=2, dim=0)
        prs_impact = torch.sum(disease_weights_norm * prs_values)
        prs_term = torch.tanh(prs_impact)
        
    if self.protein_weights is not None and len(expr_values) > 0:
        protein_weights_norm = F.normalize(self.protein_weights, p=2, dim=0)
        expr_impact = torch.sum(protein_weights_norm * expr_values)
        expr_term = torch.tanh(expr_impact)
    
    return base_loss + self.lambda1 * prs_term + self.lambda2 * expr_term

Total loss:

L_total = L_base(pos_scores, neg_scores)
       + λ₁ · tanh( ⟨w_disease_norm, PRS⟩ )
       + λ₂ · tanh( ⟨w_protein_norm, EXPR⟩ )

Both lambdas default to 0.01 in the constructor; the paper sweeps lambda1 (PRS weight) and lambda2 (expression weight) per-patient as documented at PDR_PIPELINE_PLAN.md L973.

What this loss is doing semantically. Each learnable weight tells the model how much that particular disease’s PRS (or that particular gene’s expression) should pull the next training-step gradient on the current triple. The tanh bounds the contribution to [-λ, +λ] regardless of how extreme the PRS or expression value is, preventing a single high-z-score feature from dominating. The L2 normalization on the weight vector keeps the magnitudes from drifting; only the direction of the weighting matters. After fine-tune convergence the learned weight pattern is effectively a per-patient summary: “for this individual, prioritize the bipolar-disorder PRS axis and the BDNF expression axis when ranking drugs.”

4.4 Patient-specific fine-tune — fine_tune_patient_specific

training.py:1193 — invoked from run.py when --use_ploss is set. Pre-conditions: a foundation checkpoint must already exist (trained on the augmented KG via train_foundation_model). Then:

# 7. CRITICAL OPTIMIZATION: Freeze the base model parameters
model.eval()
for param in model.parameters():
    param.requires_grad = False

All 168,705 foundation parameters become non-trainable. Only criterion.disease_weights and criterion.protein_weights (1,802 total) collect gradients. The fine-tune CLI from the personalization mechanism reference:

python script/run.py --patient_id X --dataset KG_ZORO \
  --use_ploss -l1 0.1 -l2 0.1 --ckpt <foundation>

Reported AUPRC lifts (paper §3): Robin 0.076 → 0.251 (3.3×), Zoro 0.277 → 0.368 (1.3×), Usopp 0.065 → 0.353 (5.4×). MRR / Hits stay similar because the relation-aware filter limits how much head-rank can move — the lift surfaces in AUPRC, which is unfiltered.

4.5 Test-time score adjustment — apply_personalization

A second personalization touch point exists at evaluation time: training.py:198 apply_personalization directly adjusts batch scores by a precomputed per-entity adjustment:

for drug_id, disease_rels in drug_disease_rels.items():
    prs_adjustment = 0.0
    for disease_idx, strength in disease_rels:
        prs_adjustment += (
            lambda1 *
            disease_weights[disease_idx] *
            prs_values[disease_idx] *
            strength            # currently always 1.0
        )
    precomputed_adjustments[drug_id] += prs_adjustment

The mechanism iterates the graph’s edges, finds every drug node that is connected to one of the disease entities in criterion.disease_names (or to one of the gene entities in criterion.protein_names), and accumulates a score offset proportional to the learned weight × patient value. The offset is added to the model’s raw prediction at test time.

This is structurally similar to PDR1’s scalar modulation, except (a) it acts at the score level not the embedding level, (b) it uses learned weights from the fine-tune rather than raw scalars, and (c) it only counts drug→disease and drug→protein direct edges (no message-passing propagation). It is more like a post-hoc reweighting of the model output than a personalization of the model itself.

CRITICAL — λ is one-per-experiment, not separable across train and eval (corrected 2026-05-20)

The paper’s apply_personalization reads lambda1 = criterion.lambda1 directly from the criterion state — i.e., the same λ used during fine-tuning training rides on the criterion through evaluation (training.py:206-207). A “λ sweep” in the paper means N separate fine-tune → eval pipelines, each with one λ value baked in throughout. It does NOT mean “fine-tune once at λ=0.01, then evaluate the same checkpoint at λ ∈ {0.01, 0.1, 1.0, 10.0}”.

Maple’s lib/pdr2/code/script/evaluate_personalized_maple.py accepts λ from the CLI (--lambda1_prs, --lambda2_biomarker), so it can be invoked with a λ that mismatches the fine-tune’s training λ. Doing this produces an off-spec eval-time-only sweep: the criterion weights are still those learned under the fine-tune’s λ, but the inference-time scaling is rescaled. This is not what the paper measures.

The May-2026 maureen_focus_v1 run did exactly this off-spec sweep — one fine-tune at λ=0.01, four evals at λ ∈ {0.01, 0.1, 1.0, 10.0}. The reported Δ MRR ≈ −0.4% across all four evals is therefore suggestive, not conclusive. It tells us that no post-hoc scaling rescues the weights learned at λ=0.01; it does NOT tell us how a fine-tune at λ=10.0 (with weights learned under that stronger regularization) would perform. The eval-time-only sweep artifacts have been deleted from the DB (see scripts/strip_evaltime_lambda_sweep.py).

The paper-faithful path runs through Maple’s worker queue as N pairs of pdr_finetune_run + pdr_eval_personalized_run tasks, where the eval task defaults its λ to the source fine-tune’s training λ. Submit endpoints: POST /api/mods/research/runs/{foundation_id}/submit_finetune then POST /api/mods/research/runs/{ft_id}/submit_eval_personalized. Each fine-tune becomes its own mods_research_run row with parent_run_id set; the personalized eval attaches to the fine-tune (not the foundation), so the Overview matrix correctly fans out per-fine-tune in a sweep.

4.6 Implementation status (Maple)

  • ✓ build_pdr2_input_pickle.py ships with synthetic_zeros / maureen / ukb_sample modes.
  • ✓ Maureen’s patient node is in canonical KG (MAPLE_PATIENT:maureen-1173a8db, 250 relations).
  • ✓ Phase 1 training automation code-complete; awaiting Vertex L4 quota grant (an internal note on the training-automation status).
  • ✓ eval_patient.py shipped (Maple-original) for per-patient outbound-edge scoring.
  • ✗ Patient → variant edges still gene-level — _resolve_variant_canonical HGVS-p prefix mismatch (the variant-aware KG extension).
  • ✗ OncoKB / PharmGKB connectors coded but not ingested.

5. Side-by-Side Comparison

AspectPDR1PDR2
Where the patient signal livesInside the KG, as self-loop edges per fact + a sidecar scalar tensor aligned to the triple listOutside the KG, as feature vectors prs_values / expr_values consumed by the loss function
Patient as graph node?No — patient is a row in a CSV; nodes are diseases / genes / drugs that the patient connects to via self-loopsYes — patient is a first-class node with WAS_DIAGNOSED edges (PDR2 paper); Maple’s overlay adds variant, expression, medication, sensitivity, and phenotype edges
How values modulate the modelMultiplicatively scale relation embeddings in forward_embedd: r = r * scalarsLoss-additive personalization terms: + λ₁ tanh(⟨w_d, PRS⟩) + λ₂ tanh(⟨w_p, EXPR⟩)
What is learnable on a per-patient basisNothing — the scalars are inputs, the model is shared1,802 weights: disease_weights (one per disease in disease_names) + protein_weights (one per gene in protein_names)
What is frozen on a per-patient basisn/a — no per-patient training stepThe entire foundation model (168,705 parameters)
Per-patient training timeEach patient is a new full retrain (or a fine-tune from the foundation that retrains all params; the paper trains end-to-end with -scalars set)A 3-epoch fine-tune of 1,802 weights only — orders of magnitude cheaper
Foundation training inputGlobal KG (no patient overlay)Augmented KG with patient nodes for the entire patient cohort during foundation training
Patient CSV / pickle filenamepatient_data.csv (one row per patient) + outputs/scalars_<pid>.npypatient_<id>_data.pkl
Empty-PRS behaviorA zero scalar makes r * 0 = 0 → the edge contributes no message. Heavy regularization toward “no signal”A zero PRS vector makes tanh(⟨w, 0⟩) = tanh(0) = 0 → the personalization term vanishes, the loss reduces to the base KG loss
Validation reported in paperCohen’s d on case-study ranks; AD-NCT AUROC = 0.868; PRS is strongest personalizer (Cohen’s d ≈ 1.05)AUPRC lifts on Robin / Zoro / Usopp (1.3×–5.4×); lithium case study for Usopp
What it does not personalizeTreatment / medication list (the drug-self-loop block in patient_merge.py is shipped commented out); no temporal signalTreatment / medication list (no shipped equivalent of disease_weights for drugs); no temporal signal

The structural takeaway. PDR1’s mechanism is richer per fact (every personalized scalar enters the convolution) but crude per patient (no learning, no per-patient parameters). PDR2’s mechanism is poorer per fact (only PRS and expression terms, only via a tanh-bounded loss adjustment) but richer per patient (it learns 1,802 patient-specific weights and freezes the foundation, which makes per-patient fine-tune cheap enough to do at clinic scale). Phases 12 (patient-as-node V2) and 13 (temporal modeling) are designed to combine the per-fact richness of PDR1 with the per-patient learning of PDR2 — see paan_planning.md.

6. Maureen’s Data Inventory

For each of the four discrete data types, this section enumerates what we have, what is placeholder, what is obtainable, and what is pending.

6.1 PRS — Polygenic Risk Scores

Have: Nothing computed today. mods_polygenic_score row count for Maureen is 0 as of 2026-05-09 (pdr_prs_stage0_findings.md, PDR_PIPELINE_PLAN.md L918).

Placeholder (the Track A baseline): All PRS columns set to 0.0. After z-scoring (PDR2) or self-loop modulation (PDR1) this collapses to no contribution:

  • PDR2: tanh(⟨w, 0⟩) = 0 → personalization term vanishes; loss reduces to base KG loss.
  • PDR1: r * 0 = 0 → message passing through a PRS-keyed edge contributes nothing. Symbolic edges (HAS_DIAGNOSED, biomarker self-loops) carry through unchanged.

This is a controlled baseline, not a corrupted run. Track A is the “biomarker-only personalization” probe; Track B with real PRS is the unblock once genotypes land.

Obtainable with more digging:

  • Maureen uploads a 23andMe / WGS / VCF file to the mods_genotype table. mods_prs_service.compute_prs() then computes raw + z-score + percentile against the PGS Catalog scoring file (PGS000074 for CRC, PGS002280 for AD, etc. — full default set in connectors/pgs_catalog_connector.py, now SMOKE_TEST_PGS_SET). PRS pipeline shipped (commit 1504d93, UI in 02a7824). (2026-09-29: that function no longer exists. The scorer was rebuilt as Gate C in mods_prs_scorer.py, and normalization is Gate D; mods_prs_service.py now stores scores, records their Gate D normalization and serves the released-only reads. See `prs_primer.md` §5.)
  • Default disease set covers 13 traits (CRC + 12 comorbidities). For broader comorbidity coverage, additional PGS Catalog scoring files can be added to UKB_TRAIT_TO_PGS_DEFAULTS and re-resolved per Maureen’s clinical history. CAD, AF, T2D, HT, and AST are already in the default set; psychiatric / autoimmune / cancer-specific scores can be added by extending the dict.

Pending / blocked:

  • Genotype upload from the patient. This is the single largest blocker on Track B and is purely a data-flow question, not a code question.
  • patient_merge.py UMLS REST endpoint fix (uses a dead API key) — when this is fixed PDR1 can resolve the CUI list for AD comorbidities at build time. See runs/2026-05-09_phase9/README.md.

6.2 Biomarker / gene expression

Have: 84 nonzero biomarker values for Maureen across two source-of-truth tests:

  • Caris MI Profile (WES + WTS) — 73 selected genes with TPM percentiles; 23K+ genes available in the underlying WTS data. From the second Caris report (Sept 2025): CMS1 classification. (an internal note on the project research assumptions)
  • RGCC Onconomics + OncoTrace qPCR — Livak 2^-ΔΔCt (CTC vs PBMC ratio) for a smaller gene set; methodology in an internal note on genomic-report parsing.

The builder (build_pdr2_input_pickle.py:resolve_expression_by_gene) picks the highest-priority test type per gene (Caris MI Profile > Caris WTS > RGCC qPCR by default), then within a test type picks the most recent report.

Placeholder: None needed — biomarker data is fully populated.

Obtainable with more digging:

  • Full Caris WTS export — Maureen’s WTS reports include 23K+ gene measurements but the parser only ingests 73 selected genes (those highlighted on the Caris PDF). Extending the parser to grab the full WTS table would multiply the protein_names vector ~300× and dramatically expand the biomarker personalization surface. The data is in hand; the parser is the bottleneck. (an internal note on the project research assumptions)
  • Tempus xT whole transcriptome (if ordered) — would provide a second independent WTS for redundancy and trajectory.
  • LINCS L1000 reverse-signature lookup — covered as a downstream lead, not a PDR personalization input; depends on having a clean expression signature, which the Caris WTS supplies.

Pending / blocked:

  • Phase 13 trajectory modeling — Maureen’s two CRC Caris reports (Jan 2024 + Sept 2025) provide repeat measurements for CMS classification drift and CEA dynamics, but neither PDR1 nor PDR2 consumes temporal signal. Modeling this requires Tier 1+ work from temporal_patient_modeling_planning.md.
  • Snapshot bug — mods_kgia_profile_service._build_gene_expression (line ~265) currently picks the earliest report per gene when multiple reports exist. Fix is Phase 13.1.

6.3 Diagnosis history

Have: Maureen’s diagnoses are in mods_condition and surface through mods_patient_kg_service::_build_diagnosis_edges. The KG overlay materializes 6 phenotype-style edges on her node (the variant-aware KG extension, 6 phenotype out of 250 total). The full clinical condition list (CRC, comorbidities, history) is available via the Cerbo / the integrative clinic EHR import path (an internal note on the integrative clinic’s EHR integration) and through Maple’s mods_condition table.

Placeholder: None needed — diagnosis history is the type with the cleanest source data (clinical encounters are well-recorded).

(2026-10-05, GH #624 I-3: until I-3, Maple’s copy of the Kaiser conditions was one row per name, each filed as a problem-list item. It is now one row per record, with Kaiser’s own category, and the patient KG emits one phenotype edge per resolved disease from every record that is not void, refuted or unconfirmed. The count above predates that.)

Obtainable with more digging:

  • Kaiser EHR integration (Epic FHIR + Kaiser HTML import) — for any conditions tracked at Kaiser that aren’t in the the integrative clinic records. OAuth client ID story is documented in an internal note on the health plan’s EHR integration.
  • Phenotype-level mapping — HPO term resolution per condition would let us connect to Translator-curated drug→phenotype evidence beyond CUI matching.

Pending / blocked: Nothing structurally blocking. For PDR1’s CSV the CUIs cell needs the Translator-resolved CUI list once the dead UMLS endpoint in patient_merge.py is replaced.

6.4 Treatment / medication history

Have: Treatment data in mods_medication, mods_treatment_drug, mods_treatment_episode. The patient KG overlay carries 9 + 59 + 8 = 76 medication-related edges for Maureen (the variant-aware KG extension).

(2026-10-05, GH #624 I-3: mods_medication is one row per Kaiser order, and current_medication / past_medication are decided per drug by written rules: current if any order is current, otherwise past if any is evidence she took it, never both. The counts above predate that. The smoke-run training exports exclude the medication predicates (playbook D2).)

Placeholder: None.

Obtainable with more digging: Drug-bank ID normalization for medications recorded only as free-text names — mods_treatment_drug.drugbank_id is the target column but isn’t always populated by the parser. build_pdr1_input_csv.py currently falls back to an empty list rather than failing when this column is missing (one of the two builder-bug patches landed 2026-05-09).

Pending / blocked:

  • Neither PDR1 nor PDR2 actually consumes drug history as a personalization input. PDR1’s filter_and_match_drug_nodes (patient_merge.py:70) is implemented but the self-loop creation is commented out. PDR2 has no equivalent drug_weights parameter. So drug history is currently a contextual feature (visible in the KG, useful for ranking-context dashboards, used in our Phase 10 publishing UI) but not a personalization feature. This is a gap that Phase 12 (patient-as-node V2) could close by adding a TAKES / HAS_USED edge type with a learned weight per-patient.

6.5 Variant data (PDR1+2 input via gene-level approximation only)

This is a fifth data type that neither paper consumes natively but that we care about deeply for Maureen — somatic and germline variants from Caris, Invitae, and Guardant360.

Have: 8 actionable variants materialized as gene-level MAPLE:has_genetic_alteration_in edges on Maureen’s node (the variant-aware KG extension). Includes BRAF V600E, APC c.4666dupA, KRAS Q22K, and others.

Placeholder: Variants currently resolve to gene entities in the KG, not variant entities — even though CIViC’s 4,124 variant nodes are in the canonical KG and CAID:CA123643 (BRAF V600E) is queryable. The resolver in _resolve_variant_canonical looks for {gene} {variant} name matches; Maureen’s HGVS-p strings carry p. prefix which CIViC stores without.

Obtainable with more digging:

  • Patient → variant edge resolver fix (Phase 8.5.2) — Translator/ClinGen spike validated 71.4% CAID resolution on Maureen’s 14 eligible variants. Fix is documented in the variant-aware KG extension: use HGNC REST → transcript + version-fallback HGVS-c → ClinGen Allele Registry → upsert if missing. Resolved CAIDs include BRAF V600E (CAID:CA123643, present in KG), and APC c.4666dupA (CAID:CA16021317, not in KG).
  • OncoKB ingest (Phase 8.5.4) — connector coded (backend/connectors/oncokb_connector.py, 403 lines); requires ONCOKB_API_TOKEN from https://www.oncokb.org/account/register. Levels 1–4 → biolink:predicts_sensitivity_to; R1/R2 → biolink:predicts_resistance_to. License posture: flagged commercial / free academic tier.
  • PharmGKB ingest (Phase 8.5.6) — connector coded; not ingested.
  • Mutalyzer integration — for the 5 HGVS-p-only variants that can’t currently be resolved through ClinGen.

Pending / blocked:

  • Phase 8.5 retrain — IN PROGRESS 2026-05-14 with Maureen-noded + CIViC + PrimeKG (no OncoKB yet). OncoKB-augmented retrain is a follow-up. Note: current training did not pre-hold-out RGCC; lift measurement is suggestive, not clean.
  • This is the personalization route that actually matters for Maureen’s specific variant biology, but PDR1 and PDR2 can’t consume variant data directly. The right architectural answer is Phase 12 (patient-as-node) with variant entities in the KG; gene-level approximation is the interim.

6.6 RGCC drug sensitivities — not a personalization input, but the validation label set

Have: 224 drug sensitivities parsed from Maureen’s RGCC reports (the patient-specific validation writeup). 112 distinct drug names after dedup; 109 matched to PrimeKG; 36 to GP-KG.

These are not an input to PDR1 or PDR2’s personalization machinery. They are the patient-specific ground-truth labels that lift the validation from population-level (FDA approvals, ClinicalTrials.gov) to Maureen’s tumor specifically (pdr_phase7_5_rgcc_writeup.md). Three validation tiers exist; Tier 3 (leave-one-out personalization) is the only one that can validate whether personalization machinery itself helps, and is feasible only on PDR2’s cheap fine-tune.

7. Open Questions and Risks

Q1. Does PDR1’s mechanism transfer to oncology? UK Biobank PRS was developed against late-onset chronic disease (AD, CAD, T2D) where polygenic accumulation explains much of risk. For mCRC, somatic-driver biology (KRAS / BRAF / MSI) probably dominates the actionable signal far more than germline polygenic risk does. If Maureen’s Track B run shows no Tier 1 ρ lift over Track A, that’s not necessarily a wiring failure — it may be that PDR1’s scalar mechanism is a poor fit for an oncology patient. (PDR_PIPELINE_PLAN.md L924–925 documents this as the explicit Phase 9 fallback decision.)

Q2. PDR2’s apply_personalization only counts direct drug↔disease and drug↔gene edges. Multi-hop reasoning (drug → target → pathway → disease) is the strength of NBFNet message passing in PDR2’s base model, but the test-time score adjustment doesn’t propagate through it — it only sees adjacency. This may understate the lift from learned weights. We have not audited what fraction of Maureen’s clinically relevant drugs are directly connected to her diagnoses / expressed genes in PrimeKG vs reachable through intermediate entities.

Q3. The 1,802-weight ceiling. PDR2’s personalization has at most len(disease_names) + len(protein_names) learnable weights per patient. For Maureen with the 13-disease default and ~84 genes, that’s 97 weights, not 1,802. The cohort the paper trained against had richer disease / gene vocabularies via the UK Biobank Olink panel. Whether 97 weights is enough to capture Maureen-specific signal in a 3-epoch fine-tune is empirically unknown.

Q4. Patient-network leakage. PDR2’s foundation training augments the KG with all patients during foundation training, then fine-tunes per-patient. If we eventually move toward a population tapestry (Scenario Z, paan_planning.md) the foundation could see patient↔patient similarity via shared neighbors, opening test-set leakage if validation isn’t held out by patient ID. PDR2’s paper handled this implicitly by training on a fixed UK Biobank cohort; we’d need an explicit hold-out policy.

Q5. Variant vs gene granularity for biomarker expression. PDR1 and PDR2 both key expression by gene, but Maureen’s actionable variants (e.g. BRAF V600E vs wild-type BRAF) have radically different drug-response biology. A gene-level expression of BRAF tells you nothing about V600E status. The full architectural fix is variants-as-nodes (Phase 8.5) — and that fix is needed for the variant-level edges first, not for re-keying expression. Re-keying expression by variant is an open research question for Phase 12+.

Q6. Phase 1 quota grant. PDR2 training automation is code-complete through aiplatform.CustomJob.submit() but blocked on Vertex AI L4 / A100 quota grants (an internal note on the training-automation status). Without the grant, the per-patient fine-tune machinery cannot be tested end-to-end against Maureen’s pickle. Smoke test command is staged.

8. Cross-References

Code

  • docs/References/PDR1/patient_merge.py — PDR1 KG personalization pre-processor (upstream pristine)
  • docs/References/PDR1/code/model/predict.py — PDR1 GPKG_EMBEDD.forward_embedd with r = r * scalars
  • docs/References/PDR2/reasoning/script/losses.py — PDR2 PersonalizedRepurposingLoss
  • docs/References/PDR2/reasoning/script/training.py — PDR2 fine_tune_patient_specific and apply_personalization
  • docs/References/PDR2/reasoning/script/utils.py — PDR2 load_patient_data pickle contract
  • backend/scripts/build_pdr1_input_csv.py — Maple builder for the PDR1 CSV row
  • backend/scripts/build_pdr2_input_pickle.py — Maple builder for the PDR2 pickle
  • backend/connectors/pgs_catalog_connector.py — default 13-disease set
  • backend/mods_prs_service.py — PRS compute pipeline
  • backend/mods_patient_kg_service.py — patient overlay materialization
  • lib/pdr2/code/script/eval_patient.py — Maple-original per-patient outbound-edge scorer
  • lib/pdr/code/scripts/compute_metrics_crc.py — RGCC Tier 1 / Tier 2 scoring against any trained checkpoint

Documents

Memory pointers

Graph View

Backlinks