mirror of
https://github.com/Gitlawb/openclaude.git
synced 2026-08-24 02:34:15 -05:00
fb40d49e68
* feat: add Codebase Intelligence — repo map with PageRank-ranked structural summaries
Adds a new module that builds a structural map of the repository by parsing
source files with tree-sitter, building a cross-file reference graph weighted
by IDF, ranking files with PageRank, and rendering a token-budgeted summary
of the most important files and their signatures.
Surface:
- RepoMap tool the model can call on-demand, with focus_files / focus_symbols
- /repomap slash command with --tokens, --focus, --stats, --invalidate
- Auto-injection into session system context, gated by REPO_MAP=1 env var
(compile-time feature('REPO_MAP') flag stays off in scripts/build.ts)
How it works:
git ls-files → tree-sitter WASM parse → extract defs/refs →
IDF-weighted directed graph → PageRank → render top files until token budget
Files imported by many others rank highest. Common symbol names (get, set,
map, value) are down-weighted via IDF. Results cached to disk keyed by
(path, mtime, size) — only changed files are re-parsed.
Supported languages: TypeScript, JavaScript, Python.
Tree-sitter tag queries are inlined as string constants in queries.ts so
they ship inside dist/cli.mjs and work after npm install — the .scm source
files are kept for readability/Aider attribution but are not required at
runtime. A drift-guard test (queries.test.ts) asserts byte-equality between
the inlined strings and the .scm source files.
Dependencies added: web-tree-sitter, tree-sitter-wasms, graphology,
graphology-pagerank, graphology-operators, js-tiktoken.
* fix(repomap): invalidate rendered cache on file edits + Windows test fix
- computeMapHash now folds per-file mtime+size into the cache key so a
source edit (without changing the file list) no longer returns the
prior rendered map. Adds a regression test that edits a file and
confirms the second build reflects the new symbol without manual
invalidateCache().
- queries.test.ts byte-for-byte drift guard normalizes CRLF -> LF when
reading the .scm source so Windows checkouts pass. .gitattributes
also pins *.scm to LF on future checkouts.
- Externals: declare web-tree-sitter, tree-sitter-wasms, graphology*,
and js-tiktoken in scripts/externals.ts so build validation passes.
* fix(repomap): expand directory focus paths
* fix(repomap): satisfy deadcode check
* Fix repo map review findings
* Resolve remaining repo map review findings
* fix(repomap): address review findings
* fix(repomap): address review findings
* fix(repomap): resolve smoke and review follow-ups
* fix(repomap): preserve cached tag order
* fix(repomap): resolve review follow-ups
* fix(repomap): satisfy query promise lint
* Fix repo map context timeout cleanup
* fix: address repo map review findings
* fix: cancel timed-out repo map context builds
* fix(repomap): preserve git file path whitespace
* fix(repomap): handle graph and parsing edge cases
* fix(repomap): preserve shell token positions
* fix(repomap): respect configured cache home
* fix(repomap): address review findings
- Add explicit 10000ms timeout to the feature-flag-off context test to avoid cold-import flakes.
- Add --focus-symbols flag to /repomap and forward it to buildRepoMap, matching the RepoMap tool.
- Add parsing/command tests and docs coverage for --focus-symbols.
---------
Co-authored-by: gnanam1990 <gnanasekaran.sekareee@gmail.com>
69 lines
4.0 KiB
Markdown
69 lines
4.0 KiB
Markdown
# Codebase Intelligence — Repo Map
|
|
|
|
The repo map feature gives the AI model structural awareness of your codebase at the start of each session. Instead of the model needing to explore the repository with `Grep`, `Glob`, and `Read` calls, it starts with a ranked summary of the most important files and their key signatures.
|
|
|
|
## How it works
|
|
|
|
1. **File enumeration** — Lists tracked files plus untracked, unignored files via `git ls-files --cached --others --exclude-standard` (falls back to a manual directory walk when not in a git repo)
|
|
2. **Symbol extraction** — Parses each supported source file with tree-sitter to extract function, class, type, and interface definitions, plus cross-file references
|
|
3. **Reference graph** — Builds a directed graph where an edge from file A to file B means A references a symbol defined in B. Edges are weighted by reference count multiplied by the IDF (inverse document frequency) of the symbol name — common names like `get`, `set`, `value` contribute less
|
|
4. **PageRank** — Ranks files by structural importance using PageRank. Files imported by many others rank highest
|
|
5. **Rendering** — Walks ranked files top-down, emitting file paths and definition signatures, stopping when the token budget is reached
|
|
|
|
Results are cached to disk (`~/.openclaude/repomap-cache/`) keyed by file path, mtime, and size. Only changed files are re-parsed on subsequent runs.
|
|
|
|
## Supported languages
|
|
|
|
- TypeScript (`.ts`, `.tsx`)
|
|
- JavaScript (`.js`, `.jsx`, `.mjs`, `.cjs`)
|
|
- Python (`.py`)
|
|
|
|
Additional language grammars will be added in future releases.
|
|
|
|
## Enabling auto-injection
|
|
|
|
The repo map is gated behind the `REPO_MAP` feature flag, **off by default**. To enable auto-injection into the session context:
|
|
|
|
Set the environment variable before launching:
|
|
|
|
```bash
|
|
REPO_MAP=1 openclaude
|
|
```
|
|
|
|
Or add it to your shell profile for persistent use.
|
|
|
|
When enabled, the map is built once per session and prepended to the system context alongside git status and CLAUDE.md content. The auto-injected map uses a 1024-token budget.
|
|
|
|
Auto-injection is skipped in:
|
|
- Bare mode (`--bare`)
|
|
- Remote sessions (`CLAUDE_CODE_REMOTE`)
|
|
|
|
## The /repomap slash command
|
|
|
|
The `/repomap` command is always available regardless of the feature flag. It lets you inspect and tune the map interactively.
|
|
|
|
```text
|
|
/repomap # Show the map with default settings (2048 tokens)
|
|
/repomap --tokens 4096 # Increase the token budget for a larger map
|
|
/repomap --focus src/tools/ # Boost specific paths in the ranking
|
|
/repomap --focus src/context.ts # Can use multiple --focus flags
|
|
/repomap --focus-symbols buildTool # Boost files that define specific symbols
|
|
/repomap --stats # Show cache statistics
|
|
/repomap --invalidate # Clear cache and rebuild from scratch
|
|
```
|
|
|
|
## The RepoMap tool
|
|
|
|
The model can also call the `RepoMap` tool on demand during a session. This is useful when:
|
|
- The model needs structural context mid-conversation
|
|
- The user asks about specific areas (the model can pass `focus_files` or `focus_symbols`)
|
|
- A larger token budget is needed than the auto-injected default
|
|
|
|
## Known limitations
|
|
|
|
- **Signatures only** — The map shows function/class/type declarations, not implementations. The model still needs `Read` to see function bodies.
|
|
- **Cold build time** — First build on large repos (2000+ files) can take 20-30 seconds due to WASM-based parsing. Subsequent builds use the disk cache and complete in under 100ms.
|
|
- **Language coverage** — Only TypeScript, JavaScript, and Python are supported. Files in other languages are skipped.
|
|
- **TypeScript references** — The TypeScript tree-sitter query captures type annotations and `new` expressions as references, but not plain function calls. This means the ranking slightly favors type-heavy hub files.
|
|
- **Git dependency** — File enumeration uses `git ls-files --cached --others --exclude-standard` by default, so untracked files that are not ignored can appear in the map. Non-git repos fall back to a directory walk with hardcoded exclusions.
|