perf(context): share project-wide aggregates across processes - #2349
Draft
gtarpenning wants to merge 1 commit into
Draft
gtarpenning wants to merge 1 commit into
gtarpenning wants to merge 1 commit into
Conversation
getDominantFile and getStats scan the whole graph and were recomputed by every fresh process, such as the prompt hook. They are now kept in a sidecar file tagged with a graph_epoch bumped by every graph writer, and warmed after an index or sync. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
getDominantFile()andgetStats()aggregate the whole graph, and neither depends on the query. The prompt hook is a fresh process per prompt, so the in-process memo from #1864 never helps it: every prompt recomputes both (about 1.3 s warm, more cold). This keeps the results where a new process can read them, and refreshes them when the graph changes.Independent of the trigram PR; either can merge first. #2115 (an in-process
getStatscache) and #2260 touch nearby code inqueries.ts; this stays minimal around them.What changed
graph_epochis aproject_metadatarow replaced whenever the graph changes:indexAll,sync(unless it changed nothing),indexFiles,resolveReferences,resolveReferencesBatchedandclear. No schema change.<db>.memo.json) holds both results, tagged with the epoch. A tag mismatch, a missing file or unreadable JSON recomputes and rewrites it. It is a file, not a metadata row, because the hook's connection would otherwise need the database write lock the daemon's writer holds.indexAllrecomputes inline; other writers do it after 15 s of quiet on an unref'd timer, so a burst of edits pays once.indexFilesnow awaitsorchestrator.indexFilesinside itstry, so the file lock is held until indexing finishes instead of being released as the promise is created.Measured
Prompt hook on a large TS/Go/Python monorepo index (about 1.5 GB, 350k nodes, 1.2M edges), same JSON on stdin:
The first run after a real change pays the recompute once. Hook stdout is byte-identical to the unmodified build on every run.
Compatibility
codegraph syncexits before the deferred warm; the next hook recomputes once.Testing
npx tsc --noEmitnpx vitest run __tests__/persisted-memo.test.ts __tests__/dominant-file-cache.test.ts __tests__/sync.test.ts: a stale tag and a corrupt sidecar both recompute and repair; a fresh process sees the new graph afterindexFilesandclear; the deferred warm populates the sidecar.npx vitest run: 5747 passed, 23 failed. 20 are indaemon-older-version,daemon-registryandindex-commandand fail the same way on unmodifiedmain; 3 aregit-index-currencytimeouts under load that pass in isolation on both.🤖 Generated with Claude Code