Hebra: Repository-aware context engine for AI coding agents

AI coding agents work inside large repositories but can only see a limited context window. When the wrong files are packed into that window, tokens are wasted, agents miss the code that actually needs changing, and edits become unreliable. Many context tools also lack verifiable token budgets, traceability back to source, or a way to detect when retrieved context has gone stale after files change. Claims that more sophisticated retrieval (such as code-graph expansion) beats simple keyword search are rarely tested on real tasks. The goal was to build a context engine that reliably surfaces the code an agent needs within a fixed budget, and to measure honestly which retrieval strategies actually work.

Built Hebra, a repository-aware context engine in Python: - Indexing: SQLite FTS5 full-text index over the repository, with BM25 ranking for lexical retrieval. - Code structure: Python AST analysis to build a code graph of symbols, definitions and references, used for graph-based context expansion. - Source-linked memory: every retrieved snippet carries provenance (file, symbol, location) so the agent's context can be traced back to source. - Budget packing: tokenizer-verified context budgets, so packed context is guaranteed to fit the target window rather than estimated. - Freshness: stale-file detection flags context derived from files that changed after indexing. - Evaluation harness: a benchmark built from 120 real bug-fix commits across 3 open-source repositories, using the files and symbols each fix touched as ground truth. 90 tasks were held out for evaluation. Six retrieval/packing configurations were compared, with paired bootstrap analysis to test whether differences were statistically meaningful.

- Delivered a working context engine with verifiable token budgets, source provenance and stale-context detection. - Built a reproducible benchmark from real bug-fix history instead of synthetic queries. - The strongest BM25 baseline achieved 73.9% target-symbol recall at an 8K-token budget on 90 held-out tasks. - Paired bootstrap analysis showed where graph-based optimisation underperformed the BM25 baseline. This negative result was documented rather than hidden, giving a clear, evidence-based picture of which retrieval strategies are worth their complexity. - Lesson learned: for code-context retrieval under tight budgets, a well-tuned lexical baseline is hard to beat, and any added complexity should be justified against it on held-out data.