1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

Add persistent source generations

This commit is contained in:
Andraxion 2026-07-29 04:00:23 -04:00
parent 3bee200234
commit ad4f52b239
7 changed files with 770 additions and 231 deletions

View file

@ -70,6 +70,73 @@ Repository Pyright intentionally covers `src` and `tools`, so that command repor
untyped test-result indexing rather than a source defect. Rerunning the repository-configured type
gate produced zero diagnostics.
#### Persistent generic source generations
Generic projects now persist a version-1 source-generation receipt only after complete source
loading and fully verified index publication. The receipt binds the explicit generic source
contract, project and root identity, adapter, revision, source hash, every canonical/authority/
descriptor regular-file identity, and every source-membership directory identity.
A normal warm check reads no canonical source bytes. It validates the known directories and files
directly using device, inode, mode, size, nanosecond modification time, and nanosecond change time.
Directory identities detect add, delete, and rename operations without an `rglob`. Any missing,
malformed, incompatible, foreign, or dirty receipt becomes a cache miss and falls back to the full
canonical load and row-verification oracle. Successful fallback verification repairs the disposable
receipt.
The receipt is deliberately generic-project behavior. Incremental adapter manifests retain
authority over generated or specialist source identities. A one-method legacy adapter continues to
work even when it cannot provide a cheap generation.
#### Request-scoped immutable reads
Index reads now use one read-only SQLite transaction pinned to one verified file signature and one
source identity. Existence checks and queries share that connection. Before returning, the request
rechecks the index signature and current cheap source generation. A concurrent source or index
change fails closed.
Context compilation hydrates nodes and edges from the pinned derived snapshot while retaining
profiles from the immutable descriptor. It no longer loads or parses canonical sources. The public
full `Project.load()` and deep `ProjectIndex.check()` behavior remains the recovery and equivalence
oracle.
Focused tests prove that fresh-process-style generic reads can run exact, search, filter,
backlinks, dependency, impact, context, and no-change synchronization operations while
`Project.load()` is forbidden. They also prove a final source-generation change is rejected before
return and missing/corrupt receipts fall back and repair.
On the maintained 1,000-file fixture, the current work-in-progress measurements are:
| Operation | Milestone 0 median | Milestone 1 WIP median |
|---|---:|---:|
| Warm no-change synchronize | 142.479 ms | 20.007 ms |
| Exact node | 286.306 ms | 40.277 ms |
| Search, limit 20 | 288.793 ms | 41.455 ms |
| Dependencies, depth 8 | 287.791 ms | 41.551 ms |
| Context, 32k | 436.897 ms | 46.245 ms |
| MCP exact node | 287.094 ms | 40.051 ms |
| MCP context, 32k | 434.853 ms | 46.454 ms |
The three-sample WIP run is directional, not the final Milestone 1 baseline. The final evidence run
will use the maintained sample counts and committed clean-tree revision.
#### Read-only audit reconciliation
The three Milestone 1 audits agreed on the main architecture:
- Keep complete loading and deep checking as independent truth oracles.
- Trust only versioned, identity-bound disposable generation receipts.
- Use one pinned read transaction and retain a final dirty check.
- Hydrate context from the current index.
- Replace full-edge traversal scans with bounded indexed frontier reads.
- Add compact success receipts before allowing large mutations to report post-write size errors.
- Replace hidden render-status rendering with a receipt comparison.
- Add algorithmic counters and parse-count gates alongside wall-clock thresholds.
One audit identified a correctness risk beyond latency: a large mutating MCP operation can commit
successfully and then be replaced by `result_too_large`. This must be fixed in Milestone 1 so
exactly-once operations never report a false failure after mutation.
### Initial design constraints
- Full rebuild remains the recovery and equivalence oracle.