POC exploration
I Built LabGo to Learn GraphRAG. Writing It Down Is What Made It Stick.
236 cases · hybrid 43.3% recall · agent 30.3%
From the file map
The charts that changed my mind
The fair comparison
Holding the prediction budget roughly constant changed the story. The hybrid found different true positives from the graph and improved both recall and precision.
The agent had no budget
The agent returned more files and still had worse precision. That pointed to a missing size constraint, not a reason to pretend the experiment had won.
I started LabGo because I wanted to understand GraphRAG beyond diagrams and framework names. I knew the rough vocabulary: embeddings find similar things, graphs connect related things, and agents can choose tools. What I did not have was a feel for where one method stopped being useful and another one had to take over.
The question I chose was concrete: if I change this file, what else breaks, which tests should run, and who should review it? That forced the graph to do real work. Vector similarity can find code that looks related. It cannot follow a chain of callers four steps across a repository. Once I had a question with that shape, the architecture stopped feeling like a collection of fashionable parts.
I built the exam before the impressive part. Git history became the answer key. Given one file from an old commit, LabGo predicts the other files that changed with it. That gave the call graph a baseline, and it meant vectors and the agent had to face the same cases instead of winning through a better demo prompt.
That measurement caught me several times. A 94.9% result included evidence from the commit it was being asked to predict. Another high score came from returning so many files that recall had nowhere to go but up. Both numbers looked good at first glance. Fixing them taught me more about evaluation than getting a clean result would have.
The agent did not win either. The tuned deterministic hybrid reached 43.3% recall and 15.4% precision. The agent landed at 30.3% and 5.8% on a 40-case sample because its final answer had no size cap. I do not read that as a verdict on language models. I read it as a missing budget control, the same class of mistake the earlier retrieval experiments had already exposed.
The field guide came after the build. I wrote it because I could run the commands and explain the broad architecture, but I still had to look things up when I moved between stages. Walking through every file and every decision forced me to reconstruct the reasoning instead of remembering the outcome. Reading that explanation back a few times did something the build alone had not done: the ideas became easier to retrieve without the repository open beside me.
That is the part of this project I care about most. LabGo is useful code, but it was also a way to turn a loose understanding into something I could test, explain, forget, revisit, and finally retain. The repository, decision log, file map, and short field guide are public for anyone who wants to follow the same path without pretending the mistakes were not part of it.
The decision
Build the measuring stick first. Every new retrieval layer had to face the same pinned benchmark before it earned a place in the system.
Alternatives considered
- ·Start with a polished agent demo. Faster to show, but there would be no honest way to know whether the agent helped.
- ·Use vector search for everything. Simpler, but similarity cannot answer which code depends on the file being changed.
- ·Treat the README as enough documentation. It explained how to run the project, but writing the longer guide exposed gaps in what I thought I understood.
Tech stack
The challenge
Learn GraphRAG well enough to explain where graph traversal, vector similarity, git history, and an agent each belonged without hiding behind framework vocabulary.
Architecture approach
- ·Parsed source into a code graph and mined git history into a repeatable impact-prediction exam.
- ·Measured call traversal, co-change evidence, vectors, and their union at comparable prediction budgets.
- ·Added a bounded LangGraph tool loop only after the deterministic methods had established the standard.
- ·Wrote a file-by-file field guide and reread the project through the decisions that changed its direction.
Results
- Call-graph floor: 22.4% recall and 12.2% precision
- Hybrid plus co-change: 43.3% recall and 15.4% precision
- Agent: 30.3% recall and 5.8% precision on a 40-case sample
- Nineteen append-only decisions record the fixes, reversals, and claims the evidence did not support
What I'd do differently at production scale
- ·Cap the agent's final answer and rerun it at the same prediction budget as the deterministic methods.
- ·Measure a non-Python corpus before turning multi-language ingestion into a performance claim.
- ·Keep the decision log append-only. The corrected numbers are useful because the believable wrong numbers remain visible beside them.