graphify tree renders an empty viewer on any merged graph — merge-graphs drops source_file from every node

`graphify tree` renders an empty viewer on any merged graph — merge-graphs drops `source_file` from every node

Summary

graphify merge-graphs produces a merged graph.json whose nodes carry src/source but no source_file. graphify tree filters on source_file (tree_html.py build_tree: file_nodes = [n for n in nodes if n.get("source_file")]), so running tree against a merged graph — the primary cross-repo use case for merge-graphs — always produces the "(empty graph)" viewer: title (empty graph) — graphify tree viewer, 15 KB output, zero hierarchy.

Reproduction (graphify 0.9.92, Ubuntu 22.04, Python 3.12)

# two normal per-repo extractions — both carry source_file:
$ graphify extract repo-a --code-only && graphify extract repo-b --code-only
$ python3 -c "import json; g=json.load(open('repo-a/graphify-out/graph.json')); \
    print(sum(1 for n in g['nodes'] if n.get('source_file')), '/', len(g['nodes']))"
3248 / 3461          # fine

# merge them:
$ graphify merge-graphs repo-a/graphify-out/graph.json repo-b/graphify-out/graph.json --out world/graphify-out/graph.json
$ python3 -c "import json; g=json.load(open('world/graphify-out/graph.json')); \
    print(sum(1 for n in g['nodes'] if n.get('source_file')), '/', len(g['nodes']))"
0 / 3607             # source_file gone on every node

$ graphify tree --graph world/graphify-out/graph.json --output WORLD_TREE.html
$ grep -o "<title>[^<]*</title>" WORLD_TREE.html
<title>(empty graph) — graphify tree viewer</title>

Merged node shape after the merge (attrs renamed, source_file dropped):

{"id": "src_engine_dashboard_corvusdashboard_physicsbody", "label": "PhysicsBody",
 "type": "code", "community": 25, "community_name": "Dashboard Physics Engine",
 "src": "src/engine/dashboard/CorvusDas...", "origin": "ast", "source": "corvus-colony"}

Per-repo input node shape (what tree_html needs):

{"id": "...", "label": "PhysicsBody", "_callable": true, ..., "source_file": "src/engine/dashboard/...", ...}

Expected

graphify tree on a merged graph shows the full cross-repo hierarchy (repo-tag roots → dirs → symbols), the same way it does for a single-repo graph. Either:

merge-graphs preserves source_file (repo-relative, as inputs store it), or
build_tree falls back to src/source when source_file is absent.

Workaround we're running

Re-derive source_file from src post-merge, then run tree without --root (paths are repo-relative so an absolute --root matches 0/134 files and raises):

for n in g["nodes"]:
    if not n.get("source_file") and (n.get("src") or n.get("source")):
        n["source_file"] = str(n.get("src") or n.get("source"))

After this patch the same merged graph renders a correct 259 KB tree with both repo roots.

Environment

graphify 0.9.92 (pip install --user graphifyy), networkx merge path via nx.compose + _prefix
Ubuntu 22.04, Python 3.12.10
Also worth noting: --root with an absolute path fails on merged graphs because stored paths are repo-relative ("error: --root ... matched 0 of 134 source files"), which compounds this — there's no way to anchor a merged tree without knowing each input's relative layout.

Real-world corpus: corvus-colony (3461 AST nodes) + semantic vault graph (164 nodes) = 3607-node merged world graph.

@jessework840

Thanks for opening this issue, @jessework840. A maintainer will take a look soon.

If you would like to discuss it in real time, come say hi on our Discord server. For longer-form questions and ideas there is also GitHub Discussions.

To help us triage, please make sure the report includes what you expected, what actually happened, and the steps (and a small sample) to reproduce it.

fix(tree): one root per repo on merged graphs; refuse graphs with no …

I couldn't reproduce this on 0.9.92. merge-graphs kept source_file on every node: 6 of 6 and 5 of 5 in two small repos, and 11 of 11 after the merge. The PyPI wheel for 0.9.92 is the same code as the tag, so this is the version you ran.

I also merged three real graphs built by 0.9.90 and older (23,712 nodes): all 23,036 nodes that had source_file still had it after the merge.

Your merged node also doesn't match what merge-graphs writes. It prefixes every id with the repo (corvus-colony::src_engine_...) and adds repo and local_id. Your node has no prefix, has type, origin and src where graphify writes file_type, _origin and source_file, and its path ends in .... Nothing in graphify writes those keys, so I think something in your pipeline rewrote the graph after the merge.

Two things would settle it:

The exact command or script that wrote world/graphify-out/graph.json.
The output of head -c 800 world/graphify-out/graph.json, run right after merge-graphs.

About --root: it takes a repo-relative path, because source_file is stored that way. An absolute path that matches nothing is refused on purpose (#2534).

There is a real tree bug on merged graphs: when two repos both have src/main.py, the viewer fuses them into one file. That's #4524, and #4541 and #4525 both fix it. #4541 also turns a graph with no source_file into an error instead of a blank viewer.

Navigation Menu

Repository navigation

graphify tree renders an empty viewer on any merged graph — merge-graphs drops source_file from every node #4522