Building a Defensible KG Composition Pipeline — Formal Writeup

Authors: whafa, Claude (Maple project) Date: 2026-05-09 Status: Phases A–E CLOSED (deployed to prod). Phase C4 (consumer cutover for the Explorer) CLOSED 2026-05-13 — canonical schema is the Explorer default, composition picker live, Maureen browsable alongside any composition; legacy tables preserved behind USE_CANONICAL_KG=false / ?canonical=false. Phase F (patient-anchor scoping) still planned. Related planning docs:

an internal memory note — the parent plan
pdr_phase8.5_variant_kg_planning.md — variant entry point (CIViC/OncoKB/PharmGKB connectors land here)
kg_anchor_scoping_planning.md — Phase F (patient-anchor scoping; written 2026-05-09)
KG_CATALOG.md — the source registry, now backed by a database table

Template note. This document follows the Purpose / Materials & Methods / Procedure / Results / Discussion structure used by pdr_phase0_gpkg_writeup.md so results across phases stay directly comparable.

Summary

Before this session, the Maple MODS pipeline had two biomedical knowledge graphs loaded — GP-KG and PrimeKG — via ad-hoc per-source code with no registry, no canonical CURIE layer, no composition concept, and no programmatic way to swap or combine sources for a training run. Adding a third source (CIViC, OncoKB, PharmGKB, or RTX-KG2) would have meant rewriting the export script. Verifying license posture, deduplicating across sources, or understanding “which entities does this composition cover?” required ad-hoc SQL.

We built a six-phase refactor (A–E with one planning-only Phase F) that introduces:

A source registry (mods_kg_source) — the canonical record of every biomedical KG we know about, with status, license, citation, and live row counts.
A domain taxonomy (mods_kg_domain) — Tier 1/2/3 classification of entity domains so compositions can filter by gene, drug, disease, etc. without per-source string-matching.
A canonical entity layer (mods_kg_entity / mods_kg_entity_source / mods_kg_relation) — one row per real-world thing keyed by Biolink CURIE, with provenance moved to a join table. Cross-source deduplication becomes a single GROUP BY instead of fuzzy name-matching.
A composition primitive (mods_kg_composition / mods_kg_composition_member) — named, versioned graph mixes that training runs reference by ID. Six baseline compositions seeded.
An Entity Resolver with pluggable upstream (NCATS Node Normalizer, ClinGen Allele Registry, predicate mapper) for canonical CURIE resolution at ingest time.
Four new source connectors (RTX-KG2 KGX, CIViC, OncoKB, PharmGKB) implementing a BaseKGConnector ABC.
A REST API + dashboard tab + MCP tools for inspecting the canonical schema and composition validator output without dropping into psql.
A legacy → canonical migration that backfilled all 129,375 PrimeKG nodes and 8,099,878 / 8,100,128 (99.997%) of PrimeKG edges into the new schema with zero synthetic CURIEs and zero unmapped predicates. The 0.003% gap is fully accounted for by intentional canonicalization dedup.

The session also surfaced (and fixed) two operational bugs worth recording: a per-row insert pattern that turned a should-be-10-minute migration into a 6.8-hour one (fixed: ~750× bulk-insert speedup), and a cursor advancement bug that caused all 8M+ PrimeKG edges to be permanently orphaned on the first migration attempt (fixed: auto-reset edges cursor when nodes are added, plus a --rewalk-edges escape hatch).

origin/main HEAD at writeup time: e8ff568.

1. Purpose

A ranked list of drugs is a claim about biology, and anyone asked to act on it is entitled to ask what it was computed from. Which sources went in. What each one actually contributed. How two records naming the same drug were decided to be the same drug. What is still missing, and whether the gap matters. If the answer to any of those is “there is a markdown table someone maintained by hand, and a set of ingest scripts that each made their own decisions,” then the honest response to why should anyone believe this is that they should not — and that is true no matter how good the model on top turns out to be.

The stake is specific rather than abstract. The output of this pipeline is meant to reach one patient’s oncologist, and the first question a clinician will ask about a surprising candidate is where it came from. A pipeline that cannot answer that has not earned the conversation. So this work is not infrastructure for its own sake: it is the difference between a number and a number somebody can check.

The objectives below all reduce to making the data layer auditable end to end — which, put less grandly, means being able to answer the four questions in the first paragraph from the database rather than from memory.

The concrete objectives going into the session:

Registry and provenance — every KG we’ve ingested or are considering should be a database row with license, citation, and status, not a markdown table maintained by hand.
Cross-source identity — when two KGs both contain Olaparib, the system should know it’s the same drug. Not via string-match heuristics; via canonical Biolink CURIE.
Composition — training runs should be able to declare “I want PrimeKG plus CIViC variants, filtered to Tier 1 domains” and have that be a configuration value, not a script edit.
Defensible domain coverage — for any composition, we should be able to say which Tier 1 domains have how many entities, what the license posture is, and where the gaps are.
Pluggable connectors — adding a new source should be a one-day task: write a connector that subclasses a base ABC, register it in the source registry, run an ingest job. No rewrites elsewhere.
Variant-graph readiness — Phase 8.5 wants variants (BRAF V600E, DPYD*2A, UGT1A1*28) as graph-native nodes. The schema should support this without a follow-on refactor; the connector framework should make CIViC/OncoKB/PharmGKB ingest trivial.

The scope did not include rewriting the existing KG explorer or MCP tools that read from the legacy mods_kg_node / mods_kg_edge tables — that consumer cutover (Phase C4 in the plan) is real work whose value depends on the new schema actually getting populated and exercised, so it was deferred until after the headline goal lands.

2. Materials and Methods

2.1 Pre-session state (what existed)

Two KGs loaded into mods_kg_node / mods_kg_edge: GP-KG (61,146 nodes, 1,246,726 edges) and PrimeKG (129,375 nodes, 8,100,128 edges). Both keyed by (kg_source, external_id) — the same canonical entity (e.g., Olaparib) appeared as two rows linked only by name match.
One markdown registry: docs/KG_CATALOG.md, hand-maintained.
One export script: backend/scripts/export_primekg_for_pdr.py — hard-coded WHERE kg_source = 'prime_kg'.
One MCP-aware translator query connector: backend/connectors/translator_connector.py — query-time enrichment only, never bulk ingest.
Three planning docs that referenced this work: pdr_phase8.5_variant_kg_planning.md, temporal_patient_modeling_planning.md, and the parent we-need-more-rigor-fuzzy-nygaard.md (drafted at session start).

The legacy schema, in shape:

erDiagram
    mods_kg_node {
        bigserial id PK
        text kg_source "source baked into the row"
        text external_id "native ID, not canonicalized"
        text entity_type
        text name
        jsonb synonyms
    }
    mods_kg_edge {
        bigserial id PK
        text kg_source
        bigint head_id FK
        bigint tail_id FK
        text relation_type "raw per-source predicate"
        float weight
    }

    mods_kg_node ||--o{ mods_kg_edge : "head_id"
    mods_kg_node ||--o{ mods_kg_edge : "tail_id"

Two tables, no registry, no domain taxonomy, no notion of canonical identity. The same real-world drug appears as N separate rows (one per kg_source), so corroboration (“how many sources assert this edge?”) requires guessing at name/synonym equality after the fact. Adding a third KG meant adding rows with a new kg_source string — nothing in the schema reasons about it.

2.2 New schema (migrations 071–076)

Six migrations, all idempotent, all additive (no destructive operations on existing tables):

Migration	Tables created	Seed
071 `kg_source_registry`	`mods_kg_source`	9 rows: gp_kg, prime_kg, rtx_kg2, civic, oncokb, pharmgkb_variants, drkg, translator_ncats, txgnn
072 `kg_domain_taxonomy`	`mods_kg_domain`	22 rows: 8 Tier 1 (gene, variant, protein, pathway, drug, disease, toxicity, biomarker), 5 Tier 2 (PK, metabolism enzyme, transporter, resistance mechanism, drug-drug interaction), 9 Tier 3 supporting domains
073 `kg_canonical_schema`	`mods_kg_entity`, `mods_kg_entity_source`, `mods_kg_relation`, `mods_kg_curie_cache`	(empty; populated by backfill or new-source ingest)
074 `kg_composition`	`mods_kg_composition`, `mods_kg_composition_member`	6 compositions: gp_kg_baseline_v1, primekg_baseline_v1, rtx_kg2_full, rtx_kg2_tier1, tier1_mvp_v1, tier1_mvp_with_primekg_v1 + 13 member rows
075 `research_run_composition`	(column add) `mods_research_run.composition_id`	backfilled from existing `kg_source` strings
076 `kg_legacy_backfill_state`	`mods_kg_migration_state`	(empty; populated by backfill cursors)

The resulting shape:

erDiagram
    mods_kg_domain {
        text name PK
        smallint tier "1 = MVP, 2 = deferred, 3 = supporting"
        text biolink_category
    }
    mods_kg_entity {
        bigserial id PK
        text curie UK "canonical CURIE"
        text name
        text domain FK
        text biolink_category
        bool is_synthetic_curie
    }
    mods_kg_entity_source {
        bigint entity_id PK, FK
        text source_id PK, FK
        text external_id PK "native ID in that source"
        text source_name
        text source_entity_type
    }
    mods_kg_source {
        text id PK
        text display_name
        text version
        text status "loaded / planned / api_only / deprecated"
        text license
        text citation
        bool biolink_compliant
        jsonb domains
        text connector_path
        bigint node_count
        bigint edge_count
    }
    mods_kg_relation {
        bigserial id PK
        bigint subject_id FK
        bigint object_id FK
        text predicate "Biolink"
        text source_predicate "raw, preserved"
        text source_id FK
        float weight
    }
    mods_kg_composition_member {
        text composition_id PK, FK
        text source_id PK, FK
        jsonb include_domains
        jsonb exclude_domains
        float weight
    }
    mods_kg_composition {
        text id PK
        text display_name
        text version
        bool is_locked
    }
    mods_research_run {
        bigserial id PK
        text composition_id FK
        text kg_source "denormalized legacy field"
    }
    mods_kg_curie_cache {
        text query_curie PK
        text preferred_curie
        text biolink_category
        jsonb equivalent_curies
        text resolver
    }
    mods_kg_migration_state {
        text key PK
        text value
    }

    mods_kg_entity     ||--o{ mods_kg_entity_source       : "is also known as"
    mods_kg_source     ||--o{ mods_kg_entity_source       : "provenance"
    mods_kg_domain     ||--o{ mods_kg_entity              : "categorizes"
    mods_kg_entity     ||--o{ mods_kg_relation            : "subject_id"
    mods_kg_entity     ||--o{ mods_kg_relation            : "object_id"
    mods_kg_source     ||--o{ mods_kg_relation            : "asserted by"
    mods_kg_source     ||--o{ mods_kg_composition_member  : "ingredient"
    mods_kg_composition||--o{ mods_kg_composition_member  : "recipe"
    mods_kg_composition||--o{ mods_research_run           : "trained on"

Three things to point at:

Identity moved into its own table. mods_kg_entity is one row per real-world thing keyed by canonical CURIE; mods_kg_entity_source is the many-to-one provenance join. The same drug now collapses to one entity, with N source rows under it — corroboration is a GROUP BY entity_id.
Sources and domains are first-class. Adding a new KG is now an INSERT into mods_kg_source + a connector, not a new magic string scattered through queries. Domain taxonomy with tiers makes composition filters declarative.
Compositions decouple “what’s in the graph” from “what we train on.” mods_research_run.composition_id points at a versioned recipe; the same canonical store backs every experiment.

Schema rationale. The new design separates two concerns the legacy schema conflated:

Identity — what real-world thing is this? Canonical CURIE in mods_kg_entity.curie. UNIQUE index. One row per drug/gene/disease/variant in the universe.
Provenance — which sources have it, under what native ID, with what source-specific metadata? mods_kg_entity_source rows, one per (entity, source, external_id) triple.

This means cross-source corroboration becomes a single SQL aggregate (the headline win):

SELECT subject_id, object_id, predicate,
       COUNT(DISTINCT source_id) AS support
FROM mods_kg_relation
GROUP BY 1, 2, 3
HAVING COUNT(DISTINCT source_id) >= 2;

Without the refactor, “find drugs that PrimeKG and CIViC both link to BRAF V600E” was a JOIN on lowercased names with hand-written aliases. With it, it’s a join on shared entity_id.

2.3 Entity Resolver (`backend/kg_normalize/entity_resolver.py`)

The resolver mediates between connector output (EntityRecord dataclasses with optional curie_hint and biolink_category_hint) and canonical storage. Resolution flow per record:

Cache lookup — has this CURIE hint been resolved before? mods_kg_curie_cache is a memoizing layer keyed on hint.
Upstream resolver — pluggable. In production this is ChainedResolver([ClinGenAlleleRegistryClient(), NodeNormalizerClient(), PredicateMapper()]). The default for tests / cold-start is SyntheticOnlyResolver() (no-op).
Hint as canonical — if no upstream confirmed but the connector supplied a CURIE hint (e.g., PrimeKG’s metadata gives us MONDO:0005148), use it directly. This is the key to dedup working without a live Node Normalizer.
Synthetic fallback — <source_id>:<external_id>, tagged is_synthetic_curie=true so the validator can surface the rate.

The resolver supports both per-row (resolve_entity) and bulk (bulk_resolve_entities) entry points. The bulk path issues exactly four SQL statements per batch regardless of size; we’ll come back to why that matters in §3.4.

2.4 Connector framework (`backend/connectors/base.py`)

BaseKGConnector is an ABC with three required methods (manifest, is_downloaded, download, stream_relations) plus optional hints. The manifest mirrors the columns of mods_kg_source so registry rows can be auto-upserted on first ingest. EntityRecord and RelationRecord are dataclasses connectors emit; the resolver consumes them.

Connector inventory at end of session:

Connector	Source	License	Status	Domains contributed
`gp_kg_connector.py`	GP-KG (Gao 2022)	Academic-only	loaded (legacy schema; not yet backfilled)	gene, drug, disease, phenotype
`prime_kg_connector.py`	PrimeKG (Harvard Zitnik)	CC-BY 4.0	loaded (legacy + canonical)	gene, drug, disease, pathway, phenotype, anatomy, biological_process, molecular_function, cellular_component, exposure
`rtx_kg2_connector.py`	RTX-KG2 (NCATS Translator canonicalized)	CC-BY 4.0	code complete, not yet ingested	all Tier 1 + variant + supporting
`civic_connector.py`	CIViC (clinical variant-evidence)	CC-BY 4.0	code complete, not yet ingested	variant, gene, drug, disease
`oncokb_connector.py`	OncoKB (MSK actionable variants)	Commercial (free academic)	code complete, not yet ingested	variant, gene, drug, biomarker, disease
`pharmgkb_connector.py`	PharmGKB CPIC variant guidelines	CC-BY-SA 4.0	code complete, not yet ingested	variant, gene, drug
`translator_connector.py`	NCATS Translator TRAPI 1.6 federated	mixed per source	api_only (query-time enrichment)	drug, disease, gene, protein, pathway
`txgnn_connector.py`	TxGNN (predicted indications)	academic-only	api_only	drug, disease

2.5 Composition layer (`backend/mods_kg_composition_service.py`)

A composition is a named, versioned recipe: “use these sources, apply these per-source domain and predicate filters.” Materializing it runs a CTE that walks mods_kg_entity_source and mods_kg_relation filtered by composition members. The service exposes:

CRUD (create_composition, get_composition, lock, unlock)
materialize(composition_id) → entity_ids + relation_ids ready for export
validate(composition_id) → non-blocking report covering domain coverage, license posture, dedup stats, missing-CURIE percentage, orphan count, coverage gaps
coverage_report(composition_id) → counts by (source, domain) for UX surfaces

Per the parent plan’s user direction, validate() is informational only — it never blocks ingest or composition. License posture (e.g., “OncoKB is in this composition; OncoKB is non-permissive”) is surfaced for humans to decide on, not enforced.

2.6 Export pipeline

backend/scripts/export_composition_for_training.py replaces the legacy export_primekg_for_pdr.py. Takes --composition-id and --format pdr|kgx. Produces train/valid/test splits using the same greedy + 60/20/20 algorithm the legacy script used, so existing PDR training notebooks accept the output unchanged. The legacy export script is preserved as a thin shim for backward compatibility.

2.7 UX surface (Phase E)

Three additions:

REST API (backend/mods_kg_api.py) at /api/mods/kg/*: list/get sources + compositions, validation report, coverage report, lock/unlock, global stats.
Lab dashboard tab (frontend/src/components/ModsKGCompositionTab.jsx): three-panel view — overview tiles (entity/relation/source/synthetic-CURIE counts), sources table with status badges, expandable composition cards with validate/lock actions and rendered validator reports.
MCP tools in backend/mcp_servers/mods_server.py: mods_kg_list_sources, mods_kg_list_compositions, mods_kg_get_composition, mods_kg_validate_composition, mods_kg_canonical_stats. All callable from the terminal as /<tool> slash commands.

2.8 Legacy backfill (`backend/scripts/migrate_legacy_kg_to_canonical.py`)

A two-phase walk per source:

Phase 1 (nodes). Read mods_kg_node ordered by id, build a CURIE hint from metadata.original_source + metadata.original_id (PrimeKG carries these natively) or by parsing the external_id (DB\d+ → DRUGBANK:, C\d{7} → UMLS:, etc.). Pass through the resolver, populate mods_kg_entity and mods_kg_entity_source.
Phase 2 (edges). Build a legacy_node_id → canonical entity_id map by joining the populated mods_kg_entity_source to mods_kg_node. Read mods_kg_edge ordered by id, resolve each edge’s subject/object via the map, normalize the relation_type via the predicate mapper, bulk-insert into mods_kg_relation.

Both phases use a cursor in mods_kg_migration_state for resumability. Both are idempotent (ON CONFLICT DO NOTHING on the canonical tables).

A PowerShell wrapper backfill_kg.ps1 at the repo root handles Cloud SQL Proxy startup, loads DATABASE_URL from .env, and forwards arguments to the Python script — the user invokes the entire pipeline as one command.

3. Procedure

3.1 Sequencing decisions

The plan defined eight phases (A through F) with ~5 sub-phases each. We executed in dependency order, committing at each phase boundary so any rollback would have minimal blast radius:

Commit	Phase	Highlight
617419b	A + B + C1-C2	Foundation: schema + service + resolver + clients
e0db107	C3	Legacy backfill script + predicate mapper
82e43e9	D0	RTX-KG2 KGX bulk connector
bdf68a9	D1	CIViC variant-evidence connector
672f369	D2	OncoKB variant-actionability connector
800ec09	D3	PharmGKB CPIC variant connector
7e21d6c	E	REST + MCP + dashboard tab
75d49fe + a0af749	(ops)	`backfill_kg.ps1` wrapper + ASCII fix
bfa52c8	F (planning)	Patient-anchor scoping planning doc
e8ff568	(fix)	Bulk node insert + cursor reset

Phase C4 (consumer cutover for the legacy KG explorer / MCP tools / saved subgraphs to read from the canonical schema) was deferred — its value depends on the new schema being populated and exercised, so it makes sense to land after we have real data and patient use cases driving requirements.

3.2 Test coverage

128 unit tests against the test database (PostgreSQL with pgvector):

14 EntityResolver tests (per-row + bulk paths, dedup, idempotency, cache write-through)
16 composition service tests (CRUD, materialize with various filter combinations, validator dimensions)
15 normalizer client tests (Node Normalizer + ClinGen, with HTTP stubs)
12 legacy backfill tests (synthetic legacy KG, cross-source dedup, predicate mapping, resume cursor)
9 + 19 + 18 + 20 connector tests for RTX-KG2, CIViC, OncoKB, PharmGKB respectively (synthetic data fixtures, no network)
5 bulk-insert tests added during the perf fix

Plus 9 existing KG-pathways tests verified still passing — the additive schema design didn’t break the legacy explorer.

3.3 Deploy

origin/main was force-fast-forwarded after each phase commit, triggering the standard Cloud Run container reload, which runs backend/migrations.py on startup. Migrations 071–076 applied cleanly to the production Cloud SQL instance with no errors and no rollbacks.

The user verified the new “KG Composition” tab in the mods-lab dashboard immediately after deploy and saw the seeded sources / compositions appear with zero entity counts (as expected — the canonical schema was empty until backfill ran).

3.4 The migration: what went wrong and how it got fixed

This is the part worth dwelling on — both bugs are common patterns and both have lessons that generalize past this work.

Bug 1: per-row INSERT vs cloud DB latency.

The first migration attempt was structured as a per-record loop:

for row in legacy_node_rows:
    record = build_entity_record(row)
    resolver.resolve_entity(record)   # 3-4 SQL round-trips per call

Each resolve_entity call did roughly: cache lookup, INSERT entity ON CONFLICT DO NOTHING, SELECT entity id, INSERT entity_source ON CONFLICT DO UPDATE. With ~30 ms of round-trip latency per statement to the Cloud SQL proxy, that’s ~120 ms per record. For PrimeKG’s 129,375 nodes, total wall-clock was 6 hours, 49 minutes, 42 seconds (24,582.8 s as logged).

The fix was to batch: bulk_resolve_entities(records) issues exactly four SQL statements regardless of batch size — bulk cache lookup, multi-value INSERT entities, SELECT ids by curie, multi-value INSERT entity_sources. For a 1000-record batch that’s 3,000 round-trips → 4 round-trips ≈ 750× speedup at cloud latency. The estimate held: re-running PrimeKG’s nodes phase after the fix takes minutes rather than hours.

Worth stating plainly, because the intuition runs the other way: inserting a thousand rows is not a thousand times the work of inserting one. On the wire it is very nearly the same work. What costs is the round trip — the ~30 ms of waiting between “here is a statement” and “here is your answer” — and that price is paid per statement, not per row. A loop issuing four statements per record pays it four times per record. A batch issuing four statements per thousand records pays it four times, full stop.

The practical consequence is that profiling misleads in this class of workload. Nothing in the per-row path is slow; no single call is doing anything wasteful, and a CPU profile of the migration would have shown a program spending almost all of its time idle and told you nothing about what to fix. The unit that matters is not operations, or rows, or CPU seconds. It is statements crossing the network, and it is worth counting them explicitly before concluding a job is inherently long-running.

Bug 2: cursor advancement on skipped edges.

The edges phase walked all 8.1M PrimeKG edges in id order, advancing a cursor in mods_kg_migration_state every batch. Conceptually:

for batch in stream_edges(start=cursor):
    for edge in batch:
        if both_endpoints_exist_in_canonical_map(edge):
            insert_relation(edge)
        else:
            skip()
        cursor = edge.id    # advance regardless

The problem: the first migration attempt had been run with --limit-nodes 1000 (a smoke test). That populated only 1000 canonical entities; the edges phase walked all 8.1M edges, found that ~99.7% of them referenced the other 128,375 yet-to-be-backfilled nodes, skipped them, and advanced the cursor to the maximum edge id. The follow-up “now do the rest” run correctly added the remaining 128,375 entities — but the edges phase saw cursor = max_id, fetched zero edges, and exited.

The legacy mods_kg_edge table still contained all 8.1M rows; the canonical mods_kg_relation table only had the 20,874 edges from the limited first run. 8M+ relations were silently orphaned.

This was caught by the user noticing the dashboard’s Canonical relations: 20,874 was implausibly low compared to PrimeKG’s stated 8.1M edges. We diagnosed by inspecting the cursor row — it was at the very last edge id, not at “the last successful insert” — and the architecture flaw became immediately visible.

The fix has two parts: (a) when the nodes phase processes > 0 rows in a run, automatically reset the edges cursor for that source so previously-skipped edges get re-walked; (b) a --rewalk-edges manual flag for cases like the user’s, where nodes are already complete but edges are stuck. Both rely on ON CONFLICT DO NOTHING for idempotency — re-walking already-inserted edges is a no-op.

The narrow bug is one line. The wider one is a conflation, and it is the part worth carrying: a cursor recording where the walk reached is not the same object as a cursor recording what the walk accomplished, and this loop used one variable for both. A seek position is a statement about the input; a bookmark is a statement about the output. They agree exactly as long as nothing is ever skipped — which is to say they agree right up until the first run that skips something, and that run was the smoke test.

What makes it dangerous rather than merely wrong is that both failures are quiet. The first run skips 99.7% of its input and reports success, because skipping was a legitimate outcome for every edge it saw. The second run fetches zero rows and also reports success, because reaching the end of a stream is not an error. Nothing throws, no count is obviously wrong in isolation, and the only surviving evidence is a total that happens to look implausible next to a number published elsewhere — which is exactly how it was found, and not a detection method anyone should rely on twice.

The general form: when a loop can decline to act on an item, “I have seen everything” and “I have done everything” stop being the same claim, and a resumable job needs to persist the second one.

4. Results

4.1 PrimeKG canonical backfill — full audit (BASELINE; first migration with coarse mapper)

Reader note: this section is the baseline audit from the first PrimeKG migration (2026-05-08). It reflects the coarse predicate mapping that was in place at the time, which produced a 250-edge canonicalization gap and 16 distinct Biolink predicates. The refined mapper + re-migration was triggered by analysis of these numbers; see §4.3 for the final audit on the refined data. Both audits are preserved so the reader can verify the gap, the cause, and the fix end-to-end.

After the bulk-insert fix and --rewalk-edges rerun, the audit (run via psql against the production database):

Dimension	Legacy schema	Canonical schema	Gap	Status
PrimeKG nodes / entity_source rows	129,375	129,375	0	✅ 100%
Distinct canonical entities (PrimeKG-sourced)	—	129,312	-63	ℹ️ explained
PrimeKG edges / canonical relations	8,100,128	8,099,878	-250	ℹ️ explained
Synthetic CURIE % (PrimeKG entities)	n/a	0.0000%	—	✅
Unmapped predicate % (PrimeKG relations)	n/a	0.00%	—	✅

The 63-entity gap. PrimeKG’s x_index is its internal guaranteed-unique identifier (the connector uses it as the canonical external_id). But several x_index values resolve to the same (original_source, original_id) pair when we extract the CURIE hint from the row’s metadata. In other words, PrimeKG’s own data contains two distinct x_index rows for the same real-world entity in 63 cases.

PrimeKG’s authors documented this as approximately 64 collisions; we measured 63. They’re not data loss — they’re correct deduplication of source-side duplicates. The user’s KG_CATALOG.md flagged this as a known issue when PrimeKG was first imported.

The 250-edge gap. This was the more interesting number. We confirmed via SQL audit that the gap is entirely caused by predicate canonicalization dedup, not by any data loss:

WITH joined AS (
  SELECT es_h.entity_id AS s, es_t.entity_id AS o,
         e.relation_type AS raw_p,
         /* mirror predicate_mapper.py for prime_kg */
         CASE LOWER(e.relation_type)
           WHEN 'indication' THEN 'biolink:treats'
           WHEN 'off-label use' THEN 'biolink:treats'
           ...
         END AS canonical_p
  FROM mods_kg_edge e
  JOIN mods_kg_node h ON h.id = e.head_id
  JOIN mods_kg_node t ON t.id = e.tail_id
  JOIN mods_kg_entity_source es_h
    ON es_h.source_id = h.kg_source AND es_h.external_id = h.external_id
  JOIN mods_kg_entity_source es_t
    ON es_t.source_id = t.kg_source AND es_t.external_id = t.external_id
  WHERE e.kg_source = 'prime_kg'
)
SELECT
  COUNT(*)                              AS legacy_total,
  COUNT(DISTINCT (s, o, raw_p))         AS distinct_raw_quads,
  COUNT(DISTINCT (s, o, canonical_p))   AS distinct_biolink_quads;

Result: 8,100,128 / 8,100,128 / 8,099,878.

The 250 missing edges are pairs where PrimeKG records the same (drug, disease) relation as both indication AND off-label use. Both raw predicates map to biolink:treats per the predicate mapper, and the unique constraint on (subject, object, predicate, source) collapses them to one canonical row. This is the correct semantic behavior — those are genuinely the same fact recorded twice in the source.

Other potential collapse vectors in the predicate mapper (e.g., bioprocess_protein + pathway_protein both → biolink:participates_in, or protein_protein + drug_drug both → biolink:interacts_with) would only collide if the same (subject, object) pair appeared with multiple raw predicates. PrimeKG mostly avoids this by domain-typing its edges (a protein-protein edge can’t share endpoints with a drug-drug edge), so those didn’t contribute to the dedup count.

4.2 Tier 1 domain coverage from PrimeKG alone (BASELINE)

Domain	Tier	Entities from PrimeKG	Filled	Notes
gene	1	27,671	✅	PrimeKG normalizes gene + protein into one node type
disease	1	17,080	✅
drug	1	7,957	✅
pathway	1	2,516	✅
variant	1	0	❌	Needs CIViC + OncoKB + PharmGKB (D1–D3)
protein	1	0	❌	PrimeKG normalizes to gene; RTX-KG2 will split them out (D0)
toxicity	1	0	❌	PrimeKG has `phenotype` (Tier 3); CTCAE / SIDER would fill this
biomarker	1	0	❌	Needs OncoKB (D2)

PrimeKG fills 4 of 8 Tier 1 domains. The other 4 are exactly what the new connectors are designed to populate — RTX-KG2 for proteins, CIViC + OncoKB + PharmGKB for variants and biomarkers, with Tier 1 toxicity coverage from CTCAE / SIDER as a future addition. Total per-domain breakdown including the supporting Tier 3 categories sums to PrimeKG’s exact 129,375 node count.

4.3 Re-migration with refined predicate mapping + GP-KG (2026-05-09 through 2026-05-11)

The §4.1 audit triggered a closer look at the predicate mapper: the 250-edge gap was explainable but stemmed from over-coarse Biolink mappings (indication + off-label use both → biolink:treats, plus several *_* ontology hierarchies all mapped to the generic biolink:related_to). Three real bugs were fixed and several over-coarse defaults refined; details in kg_predicate_mapping_report.md.

The refined mapper was deployed; both canonical schemas were wiped (via wipe_canonical_kg.ps1 — supports per-source via -Source <id> flag) and re-migrated. The §4.1 audit was repeated post-refinement and GP-KG was migrated for the first time.

PrimeKG canonical schema — final audit (verbatim from production DB on 2026-05-11):

Dimension	Legacy schema	Canonical schema (refined)	Gap	Status
PrimeKG nodes / entity_source rows	129,375	129,375	0	✅ 100%
Distinct canonical entities (PrimeKG-sourced)	—	129,312	-63	ℹ️ PrimeKG’s documented internal collisions (same 63 as §4.1)
PrimeKG edges / canonical relations	8,100,128	8,100,128	0	✅ 100% (was -250 in §4.1; gap closed by separating `off-label use` from `treats`)
Distinct Biolink predicates in data	n/a	20 (up from 16 in §4.1)	—	✅ refined mapper preserves all source-native distinctions
Synthetic CURIE % (PrimeKG entities)	n/a	0.0000%	—	✅
Unmapped predicate % (PrimeKG relations)	n/a	0.000%	—	✅

GP-KG canonical schema — first migration audit (verbatim from production DB on 2026-05-11):

The first GP-KG migration attempt surfaced two further bugs in the hint code and mapper that the PrimeKG path hadn’t exercised: 87.26% synthetic CURIE rate (the hint code didn’t recognize GP-KG’s external_id formats — CID100002907 with 100M PubChem offset, bare HGNC symbols like BRAF, underscore-not-colon CURIEs like HP_0001128, and lowercase UMLS CUIs like c0009402) and 98.76% unmapped predicate rate (the mapper had been written against the published GP-KG schema names like treat/target/associate, but the actual data ships verbose <entity>_<vocabulary>_phenotype relations like gene_uao_phenotype, gene_mp_phenotype, etc.). Both fixed (commit e234bbd); GP-KG was wiped and re-migrated.

Dimension	Legacy schema	Canonical schema (post-fix)	Gap	Status
GP-KG nodes / entity_source rows	61,146	61,146	0	✅ 100%
GP-KG edges / canonical relations	1,246,726	1,246,726	0	✅ 100% (0 edges skipped)
Distinct Biolink predicates in data	n/a	7	—	✅ (4 EXACT + 1 DELIBERATE-LOSS for GO + 2 AMBIGUITY-VIA-DOMAIN for has_phenotype)
Synthetic CURIE rate (GP-KG entities)	n/a	0.013% (8 of 61,146)	—	✅
Unmapped predicate %	n/a	0.000%	—	✅

The 8 synthetic-CURIE GP-KG entities are all defensible non-fixes (six complement proteins miscategorized as entity_type='disease' in GP-KG’s source data; two viral genes — NS3/4A and HIV-1 PROTEASE — that aren’t in HGNC). Full audit table including the verbatim CURIEs is in kg_predicate_mapping_report.md section “The 8 GP-KG entities that DID land as synthetic”.

Cross-source state across the combined canonical schema:

Metric	Value	Reproducibility query
Total canonical entities (both sources)	188,270	`SELECT COUNT(*) FROM mods_kg_entity`
Total canonical relations (both sources)	9,346,854	`SELECT COUNT(*) FROM mods_kg_relation`
Cross-source-shared entities (PrimeKG ∩ GP-KG)	2,188	`SELECT COUNT(*) FROM (SELECT entity_id FROM mods_kg_entity_source WHERE source_id='gp_kg' INTERSECT SELECT entity_id FROM mods_kg_entity_source WHERE source_id='prime_kg') s`
Math check: 129,375 + 61,146 − 188,270 =	2,251 (= 2,188 cross-source + 63 PrimeKG internal)	✓

The 2,188 cross-source-shared entities are the first concrete demonstration that the canonical-schema refactor is doing its job: when both KGs reference the same drug, disease, or phenotype via a CURIE either side knows, the row collapses to one canonical entity with two provenance rows. The relation table’s unique (subject, object, predicate, source) constraint then supports COUNT(DISTINCT source_id) corroboration queries — “how many sources independently assert this fact?” becomes a one-liner.

4.4 Composition-as-config validation

The tier1_mvp_v1 composition (the headline target — RTX-KG2 + CIViC + OncoKB + PharmGKB filtered to Tier 1 domains) was created and persisted at deploy time but is currently empty because none of D0–D3 have been ingested yet. The validator surfaces this correctly: hitting “Validate” in the dashboard reports coverage_gaps: ["rtx_kg2 has 0 entities loaded (status=planned)", "civic has 0 entities loaded (status=planned)", ...].

The primekg_baseline_v1 composition (the existing PDR2 training-set definition) was likewise persisted and now materializes against the canonical schema. Running export_composition_for_training.py --composition-id primekg_baseline_v1 produces a triples file ready for PDR1/PDR2 training; the export is byte-equivalent (same row count, same connectivity structure) to the legacy export_primekg_for_pdr.py output, just with canonical Biolink CURIEs as token strings instead of PrimeKG x_index integers.

4.5 First connector-driven ingest — CIViC (Phase D1, 2026-05-12)

CIViC was the first source to land via the new connector framework rather than the legacy backfill path. It’s a clean test of the abstraction: a fresh ingest with no precedent in the legacy schema, exercising the full flow (manifest → download → stream → resolve → upsert → registry update).

Final state in canonical tables:

	Count
Entities	6,014
Relations	11,474
Variants	4,421 (~36% with real canonical CURIE: CAID/CLINVAR/HGVS; rest synthetic `civic:civic_variant:N`)
Genes	910
Drugs	640
Diseases	330

Biolink predicate distribution (no unmapped relations):

Predicate	Count
`biolink:has_sequence_variant`	4,045
`biolink:predicts_sensitivity_to`	2,788
`biolink:predicts_resistance_to`	1,507
`biolink:gene_associated_with_condition`	963
`biolink:biomarker_for`	812
`biolink:contributes_to`	570
`biolink:predicts_toxicity_for`	33
`biolink:related_to`	21

The headline contribution: 4,421 variant entities — the entity class neither PrimeKG nor GP-KG carries. The connector framework’s EntityRecord schema with curie_hint and biolink_category_hint handled the variant-as-graph-node case without any schema changes.

Two problems surfaced during the ingest that are worth recording — both quiet failure modes that the framework’s defensive design caught only after the data landed:

1. CIViC’s TSV download URL had changed. The connector hardcoded https://civicdb.org/downloads/nightly/EvidenceSummaries.tsv from when it was first written; CIViC has since reorganized to a nightly-AcceptedAndSubmittedClinicalEvidenceSummaries.tsv naming. The old URL now 200-OKs with the 1.7 KB SPA HTML, so the connector cheerfully reported a successful download with evidence_size_mb: 0.0. First fix: update the filenames to the current convention (verified via the CIViC GraphQL dataReleases query). Lesson: connectors should sanity-check downloaded payloads (content-type, minimum-size) before declaring success.

2. CIViC’s variant TSV uses placeholder strings (N/A, NONE FOUND, unregistered) in identifier columns. The connector was building CLINVAR:N/A as a CURIE hint and the EntityResolver dutifully treated it as a canonical identifier — collapsing 283 distinct biological variants onto one shared mods_kg_entity row. The validator surfaced this as an unusually high variant_sources count per CLINVAR:* curie. Second fix: a _clean_id() helper that filters known placeholder values to None before forming the CURIE; variants with no real ID then fall through to the synthetic fallback (as they should). Wipe + re-ingest yielded the correct 4,421 distinct variants.

The two bugs together pushed the first-pass node count from 5,724 to the correct 6,014 — a 5% identity-graph correction that would have silently degraded downstream cross-source matching once OncoKB and PharmGKB land.

Schema evolution in the upstream source. CIViC’s 2025 schema introduced a molecular_profile layer that abstracts over variant (1 MP = 1+ variants). The connector originally read variant_id from evidence rows; the new schema ships molecular_profile_id and pushes variant_id to the join via single_variant_molecular_profile_id in VariantSummaries. The patched connector now resolves MP → variant for single-variant profiles (the common case) and falls through to MP-as-opaque-entity for multi-variant profiles. Two-line change to _row_to_relations plus a new mp_to_variant index. Source-schema evolution is going to be a recurring theme as we add more connectors; the connector framework’s separation of “where does the data live” from “what shape is it” keeps this kind of upstream churn local to the connector file.

Performance. The end-to-end ingest took 14 minutes for 10,943 evidence rows at ~14 relations/sec — driven by the per-row EntityResolver.resolve_entity round-trip cost (3-4 SQL statements per record × 30ms cloud RTT). Fine for CIViC’s scale. RTX-KG2’s 6.5M entities × 140M relations would extrapolate to ~150 hours at this rate — the bulk path (bulk_resolve_entities, ~750x faster) becomes mandatory for Phase D0. Tracked as an explicit pre-D0 prerequisite.

mods_kg_source.status is now loaded for CIViC with node_count=6014, edge_count=11474, ingested_at=2026-05-12 23:13:35Z. The tier1_mvp_v1 composition now materializes with non-zero CIViC contribution.

4.6 The composition pipeline as a research lever

The composition primitive immediately enables a class of experiments that were friction before:

Cross-source corroboration scoring. Train on primekg_baseline_v1, then for each predicted candidate, count how many sources independently assert the same (subject, object, predicate) tuple. Higher = more confidence. This is now a GROUP BY query, not a custom script.
License-aware compositions. “What does my graph look like if I exclude OncoKB and any PharmGKB-derived edges?” One composition definition; immediate answer.
Domain ablation. Train on tier1_mvp_v1, then on tier1_mvp_v1 excluding variant domain, and measure the lift.
Patient-anchor scoping. Phase F (planning doc landed) extends the composition primitive with an anchor_set_id so a composition can mean “all of X intersected with N hops of Maureen’s anchor entities.” Critically, this is NOT a way to define the production training set (that would destroy the discovery loop) — it’s a post-hoc filter / fine-tune target / explainability surface. The planning doc covers six distinct use cases; F.5 (inference candidate filtering) is the single most product-relevant.

5. Discussion

5.1 What we’d do differently

Bulk-insert from day one. The per-row resolver was correct but operationally unusable at production-DB scale. Adding the bulk_resolve_entities method as a separate code path (vs retrofitting resolve_entity to bulk under the hood) felt right because it keeps the per-row API for streaming connectors that genuinely yield records one at a time. But the bulk path should have existed before the first migration attempt rather than being added as a fix after a 6-hour wait.

Cursor semantics need to be explicit. “Where I left off” and “what I successfully did” are different things and cursor design has to pick one and commit to it. Our fix (reset on nodes-added + manual --rewalk-edges flag) treats the cursor as a seek hint, not a finality marker, which works for our use case because every operation is idempotent. If we ever add a non-idempotent operation downstream, this cursor design will need a re-think.

Test coverage was good but missed the operational shape. We had 128 passing unit tests against synthetic data. They didn’t catch either bug because both manifest only at scale (the per-row latency penalty) or in a specific interaction sequence (cursor advancement after a partial run). Adding even one integration test that runs the backfill against a 10,000-row synthetic dataset and times it would have flagged the perf issue immediately.

5.2 Limitations

Phase C4 (consumer cutover) is deferred. The legacy KG explorer, MCP tools (mods_kg_search, mods_kg_find_path, etc.), and saved-subgraphs (mods_kg_subgraph / mods_kg_subgraph_item from migration 069) all still read the legacy mods_kg_node / mods_kg_edge tables. The new dashboard tab and REST API read the canonical schema. Both data layers coexist correctly; they just don’t share a UI yet. The cutover is real work, and it should be driven by which canonical-schema feature gets used first.
No new sources actually ingested yet. D0–D3 connectors are code-complete and unit-tested but haven’t been pointed at real data. RTX-KG2 in particular needs a multi-GB KGX bundle download; CIViC needs a few-MB nightly TSV pull; OncoKB needs an academic API token; PharmGKB needs a manual download from their downloads page. The dashboard will surface “loaded” vs “planned” status correctly once each lands.
The legacy backfill script is one-shot. It exists to migrate the existing GP-KG and PrimeKG data into the canonical schema. After GP-KG runs (the only remaining backfill), the script can be retired. New sources go through the connector framework directly — they never touch mods_kg_node / mods_kg_edge.
The predicate mapper is hand-curated. Six sources × ~20 raw predicates each = ~120 mapping entries currently. As we add more sources or PrimeKG/CIViC/OncoKB ship updates with new relation types, the mapper needs maintenance. The validator surfaces the unmapped-predicate rate per composition, so coverage gaps are at least visible — but a more principled approach (e.g., LLM-suggested Biolink mappings with human review) is a candidate future improvement.
The Entity Resolver’s hint-as-canonical fallback can produce CURIE drift. If the connector’s hint uses a different prefix casing than the canonical Biolink form (e.g., NCBIGene:673 vs NCBIGENE:673), and Node Normalizer isn’t wired in to canonicalize, two effectively-identical hints land as two distinct entities. We saw this once during testing and fixed it by removing the over-aggressive .upper() call in the hint construction. With Node Normalizer wired in (currently optional via --use-node-normalizer), this drift is invisible because every hint round-trips through a canonical-form authority. Worth wiring as default for a future ingest pass.

5.3 Retiring the legacy backfill

After GP-KG is migrated (the last remaining source in mods_kg_node / mods_kg_edge), the path forward is:

Verify GP-KG migration with the same audit pattern — total node count, edge count, synthetic CURIE rate, unmapped predicate rate, expected-collapse delta.
Lock the baseline compositions (gp_kg_baseline_v1 and primekg_baseline_v1) — mods_kg_composition_service.lock(id) flips is_locked = true so accidental edits can’t shift training sets out from under reproducibility.
Stop using mods_kg_node / mods_kg_edge for new work. New sources go through the BaseKGConnector framework into the canonical schema directly.
backend/scripts/migrate_legacy_kg_to_canonical.py can be retired — kept in git history but removed from the active codebase. The backfill_kg.ps1 wrapper retires with it.
Phase C4 (consumer cutover) becomes the next blocker for legacy table removal. Until KG explorer / MCP tools / saved subgraphs are migrated to read canonical, migrations 079 (rename to *_legacy) and 080 (drop *_legacy) stay deferred.

5.4 What’s next

In order of expected value:

D1 CIViC ingest. Public, permissive, the easiest of the variant connectors to bring online. Closes the variant-domain gap immediately for cancer variants. Estimated: half a day including audit.
D2 OncoKB ingest. Requires the academic API token. Adds FDA-aligned actionability (Level 1/2/3A/3B/4/R1/R2) which directly maps to Maureen’s clinical decision context. Estimated: half a day.
D3 PharmGKB ingest. Requires manual download. Closes the chemo-safety gap (DPYD, UGT1A1, TPMT, CYP2D6) — directly relevant to her CRC treatment plan. Estimated: half a day.
Re-run training on tier1_mvp_v1. Once D1–D3 land, the headline composition is no longer empty. Compare downstream metrics (RGCC validation, AD case study Spearman ρ) against primekg_baseline_v1 to measure variant-extension lift.
Phase F.1–F.4 (anchor scoping inspection layer). Even before full Phase F, getting “Maureen’s 2-hop neighborhood” visible in the dashboard delivers immediate value for inspection and explainability.
Phase C4 (consumer cutover). Once the canonical schema is the primary data substrate by usage, the legacy explorer / MCP tools / saved subgraphs cutover becomes the gating step for retiring mods_kg_node / mods_kg_edge.
D0 RTX-KG2 ingest. Lower priority than D1–D3 because RTX-KG2 overlaps PrimeKG significantly and the marginal value is mostly in the protein domain (which mods_kg_entity will keep separate from gene, unlike PrimeKG’s normalization). Worth doing for completeness and as a third corroboration source.

6. Acknowledgments and methodological notes

Code and infrastructure throughout this work were authored collaboratively with Claude (Anthropic Sonnet 4.5 via Claude Code). All schema decisions, architecture, and final commits were reviewed by the human author. The 6.8-hour first-attempt migration was a real production cost; the audit was real verification; the bugs were real bugs.
Test database for the unit suite is a separate maple_test instance. Production database is the Cloud SQL maple_db instance backing the deployed Maple stack.
The full session including diagnosis of both bugs and the audit queries that confirmed the 250-edge gap were authored within a single ~16-hour pairing window starting 2026-05-08.
Contributions of intermediate commits, test counts, and migration timings reported above are taken from the git log and the deployed mods_kg_migration_state table at writeup time. Independently reproducible via the queries shown in §4.1 against the production database.

7. Cross-references

Parent plan: an internal memory note
Variant entry-point planning: pdr_phase8.5_variant_kg_planning.md
Patient-anchor scoping planning: kg_anchor_scoping_planning.md
KG source registry (auto-generatable from mods_kg_source going forward): KG_CATALOG.md
Prior writeup template: pdr_phase0_gpkg_writeup.md
Prior substack-format draft: essays/training_first_neural_network.md

End of writeup. The two bugs in §3.4 are the parts most likely to generalise: the round-trip-vs-compute point applies to any cloud-database workload, and the seek-versus-bookmark distinction applies to any resumable job whose loop can skip an item. §4.1 and §4.3 are preserved as a pair on purpose — the first audit is what made the mapping gap visible, and removing it would leave the fix without the evidence that motivated it.

MODS Notebook

Explorer

Building a Defensible KG Composition Pipeline — Formal Writeup

Summary

1. Purpose

2. Materials and Methods

2.1 Pre-session state (what existed)

2.2 New schema (migrations 071–076)

2.3 Entity Resolver (`backend/kg_normalize/entity_resolver.py`)

2.4 Connector framework (`backend/connectors/base.py`)

2.5 Composition layer (`backend/mods_kg_composition_service.py`)

2.6 Export pipeline

2.7 UX surface (Phase E)

2.8 Legacy backfill (`backend/scripts/migrate_legacy_kg_to_canonical.py`)

3. Procedure

3.1 Sequencing decisions

3.2 Test coverage

3.3 Deploy

3.4 The migration: what went wrong and how it got fixed

4. Results

4.1 PrimeKG canonical backfill — full audit (BASELINE; first migration with coarse mapper)

4.2 Tier 1 domain coverage from PrimeKG alone (BASELINE)

4.3 Re-migration with refined predicate mapping + GP-KG (2026-05-09 through 2026-05-11)

4.4 Composition-as-config validation

4.5 First connector-driven ingest — CIViC (Phase D1, 2026-05-12)

4.6 The composition pipeline as a research lever

5. Discussion

5.1 What we’d do differently

5.2 Limitations

5.3 Retiring the legacy backfill

5.4 What’s next

6. Acknowledgments and methodological notes

7. Cross-references

Graph View

Table of Contents

Backlinks

MODS Notebook

Explorer

Building a Defensible KG Composition Pipeline — Formal Writeup

Summary

1. Purpose

2. Materials and Methods

2.1 Pre-session state (what existed)

2.2 New schema (migrations 071–076)

2.3 Entity Resolver (backend/kg_normalize/entity_resolver.py)

2.4 Connector framework (backend/connectors/base.py)

2.5 Composition layer (backend/mods_kg_composition_service.py)

2.6 Export pipeline

2.7 UX surface (Phase E)

2.8 Legacy backfill (backend/scripts/migrate_legacy_kg_to_canonical.py)

3. Procedure

3.1 Sequencing decisions

3.2 Test coverage

3.3 Deploy

3.4 The migration: what went wrong and how it got fixed

4. Results

4.1 PrimeKG canonical backfill — full audit (BASELINE; first migration with coarse mapper)

4.2 Tier 1 domain coverage from PrimeKG alone (BASELINE)

4.3 Re-migration with refined predicate mapping + GP-KG (2026-05-09 through 2026-05-11)

4.4 Composition-as-config validation

4.5 First connector-driven ingest — CIViC (Phase D1, 2026-05-12)

4.6 The composition pipeline as a research lever

5. Discussion

5.1 What we’d do differently

5.2 Limitations

5.3 Retiring the legacy backfill

5.4 What’s next

6. Acknowledgments and methodological notes

7. Cross-references

Graph View

Table of Contents

Backlinks

2.3 Entity Resolver (`backend/kg_normalize/entity_resolver.py`)

2.4 Connector framework (`backend/connectors/base.py`)

2.5 Composition layer (`backend/mods_kg_composition_service.py`)

2.8 Legacy backfill (`backend/scripts/migrate_legacy_kg_to_canonical.py`)