Close Milestone 1 fast core

ACTIVE_SLICE.md

@@ -1,12 +1,12 @@
 # Active milestone
 
 ```text
-Milestone: 1 — fast, observable core
+Milestone: 2 — agent retrieval and MCP experience
-Goal: Make warm retrieval immediate by removing repeated whole-project work without changing graph meaning.
+Goal: Let one project-bound server return compact, task-shaped, explainable context under an explicit effective policy.
-In scope: Structured profiling; immutable request snapshots; duplicate-check elimination; linear validation; indexed traversal; compact receipts and bounded pagination where measurements require them; receipt-based status; persistent source generations; cheap no-change detection.
+In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks.
-Out of scope: Speculative storage replacement; task-shaped agent retrieval; independent render-plan packages; adapter SDK expansion; self-hosting; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
+Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
-Done when: Routine warm reads parse zero canonical sources; exact, search, traversal, context, synchronization, and status paths are bounded and measured; stale and corrupt state still fail closed or recover safely; legacy and no-AST adapters remain compatible; the complete repository gate passes.
+Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass.
-Status: Active. Repository audits and design reconciliation are in progress.
+Status: Active. Read-only contract audits begin from the verified Milestone 1 boundary.
 ```
 
-Milestones 2–5 remain directional context and are not active.
+Milestones 3–5 remain directional context and are not active.

DEVELOPMENT_NOTES.md

@@ -26,7 +26,7 @@ The central measurement was decisive: a 1,000-node warm exact lookup took about
 generation-pinned SQLite query path took about 0.4–1.4 ms. Repeated whole-source loading and
 validation, not SQLite, is the first optimization target.
 
-## Milestone 1 — active: fast, observable core
+## Milestone 1 — complete: fast, observable core
 
 ### Outcome
 
@@ -45,13 +45,12 @@ of project size.
   and exact retrieval sub-millisecond on a tiny fixture.
 - Pinned viewer queries prove the current SQLite schema can serve bounded reads quickly.
 
-### Current work
+### Final outcome
 
-1. Audit request-scoped immutable snapshot and persistent-generation options.
+Routine generic reads now parse zero canonical source files, use one generation-pinned SQLite
-2. Audit result receipts, pagination, bounded response contracts, and side-effect-free status.
+snapshot, and return bounded results. Status checks perform no hidden rendering or rebuilding.
-3. Audit graph validation complexity, indexed traversal, profiling, and zero-source-parse proofs.
+Structured counters prove those invariants independently of machine timing. Complete loading and
-4. Reconcile the audits into the smallest additive design that preserves v1 behavior.
+deep validation remain recovery and equivalence oracles.
-5. Implement and measure coherent slices, committing only after their gates pass.
 
 ### Work log
 
@@ -116,20 +115,20 @@ backlinks, dependency, impact, context, and no-change synchronization operations
 `Project.load()` is forbidden. They also prove a final source-generation change is rejected before
 return and missing/corrupt receipts fall back and repair.
 
-On the maintained 1,000-file fixture, the current work-in-progress measurements are:
+The final clean 1,000-file run recorded:
 
-| Operation | Milestone 0 median | Milestone 1 WIP median |
+| Operation | Milestone 0 median | Milestone 1 median | Milestone 1 p95 |
-|---|---:|---:|
+|---|---:|---:|---:|
-| Warm no-change synchronize | 142.479 ms | 20.007 ms |
+| Warm no-change synchronize | 142.479 ms | 9.192 ms | 9.324 ms |
-| Exact node | 286.306 ms | 40.277 ms |
+| Exact node | 286.306 ms | 17.887 ms | 18.577 ms |
-| Search, limit 20 | 288.793 ms | 41.455 ms |
+| Search, limit 20 | 288.793 ms | 19.518 ms | 19.884 ms |
-| Dependencies, depth 8 | 287.791 ms | 41.551 ms |
+| Dependencies, depth 8 | 287.791 ms | 18.117 ms | 19.061 ms |
-| Context, 32k | 436.897 ms | 46.245 ms |
+| Context, 32k, page 20 | 436.897 ms | 25.395 ms | 25.867 ms |
-| MCP exact node | 287.094 ms | 40.051 ms |
+| Render status | 150.591 ms | 18.969 ms | 19.613 ms |
-| MCP context, 32k | 434.853 ms | 46.454 ms |
+| Visualization status | — | 9.875 ms | 10.836 ms |
 
-The three-sample WIP run is directional, not the final Milestone 1 baseline. The final evidence run
+The recorded run came from clean commit `6253c45a5eca01efa8c73ea3dfe4d85c55878ada`.
-will use the maintained sample counts and committed clean-tree revision.
+Every measured operation passed its p95 threshold and work-counter contract.
 
 #### Read-only audit reconciliation
 
@@ -245,8 +244,8 @@ The result JSON schema contains the same closed operation, stage, and counter se
 implementation.
 
 The maintained `tools/milestone1_benchmark.py` harness adds hard counter and p95 latency gates to a
-disposable generic project. The smoke target is part of `make gate`; the 1,000-node evidence run
+disposable generic project. The smoke target is part of `make gate`; the final 1,000-node evidence
-will be recorded only from a clean committed revision. The historical Milestone 0 harness remains
+is recorded in `benchmarks/milestone1-2026-07-29.json`. The historical Milestone 0 harness remains
 behaviorally unchanged as comparison evidence; it only exposes shared fixture and measurement
 helpers to the Milestone 1 harness.
 
@@ -308,6 +307,17 @@ incoming traversal, and measures current/stale/missing/corrupt render receipts p
 visualization lifecycle states. A maintained query-plan test prevents the incoming traversal
 temporary sort from returning.
 
+#### Milestone closeout
+
+The complete repository gate passed with 138 tests and 77 subtests, zero Pyright diagnostics,
+warning-strict execution, package builds, contract checks, and both benchmark smoke gates. The
+clean 1,000-node benchmark passed all maintained thresholds and is interpreted in
+`docs/MILESTONE_1_BASELINE.md`. Compatibility, measured decisions, limitations, and scope evidence
+are frozen in `docs/MILESTONE_1_CLOSEOUT.md`.
+
+Milestone 1 made no storage rewrite, self-hosting change, production integration change, tag, or
+release.
+
 ### Initial design constraints
 
 - Full rebuild remains the recovery and equivalence oracle.

benchmarks/README.md

@@ -36,6 +36,12 @@ mutate repository content.
 `time.perf_counter_ns()` for durations. The file is data, not a performance threshold. Later work
 must explain fixture or environment changes before comparing results.
 
+`milestone1-2026-07-29.json` is the clean-tree fast-core baseline captured from commit
+`6253c45a5eca01efa8c73ea3dfe4d85c55878ada`. Unlike the historical baseline, the Milestone 1
+harness enforces operation-specific p95 ceilings and fixed zero-work counter invariants. Its
+human-readable interpretation is in
+[`docs/MILESTONE_1_BASELINE.md`](../docs/MILESTONE_1_BASELINE.md).
+
 The generic fixture exposes whole-source scaling. It does not replace the incremental adapter
 equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has
 no portable graph-planning or graph-rendering contract.

docs/MILESTONE_1_BASELINE.md

@@ -0,0 +1,102 @@
+# DocForge2 Milestone 1 baseline
+
+Milestone 1 removes repeated whole-project work from routine warm reads while retaining complete
+loading and deep validation as recovery and equivalence oracles. The maintained machine-readable
+evidence is
+[`benchmarks/milestone1-2026-07-29.json`](../benchmarks/milestone1-2026-07-29.json), captured from
+clean commit `6253c45a5eca01efa8c73ea3dfe4d85c55878ada`.
+
+## Environment and method
+
+- Platform: x86-64 Linux 7.1.3 with glibc 2.43.
+- Python: CPython 3.14.6.
+- Fixture: 1,000 Markdown files, 1,000 nodes, and 999 dependency edges.
+- Samples: ten measured invocations after validated warmups.
+- Duration clock: `time.perf_counter_ns()`.
+- Percentile: nearest rank, so p95 is the maximum with ten samples.
+- Response size: UTF-8 bytes of compact, sorted JSON.
+- Process memory: cumulative main-process `RUSAGE_SELF` high-water mark.
+
+All canonical sources, caches, indexes, changesets, renders, and viewer state were created in a
+disposable temporary directory. The run did not read WorldForge, ScrapeStation, legacy DocForge
+indexes, or production MCP state.
+
+The benchmark validates every warmup and measured result. It also fails when a routine warm
+operation performs a project load, parses canonical source, rebuilds an adapter projection,
+extracts adapter sources, builds an index, prepares a render, constructs rendered output, or hashes
+complete rendered output.
+
+## Maintained 1,000-node results
+
+| Operation | Median | p95 | Gate | Response |
+|---|---:|---:|---:|---:|
+| Warm no-change synchronization | 9.192 ms | 9.324 ms | 100 ms | 1,570 B |
+| Exact node | 17.887 ms | 18.577 ms | 50 ms | 1,487 B |
+| Missing-node error | 18.170 ms | 18.320 ms | 50 ms | 1,130 B |
+| Search, limit 20 | 19.518 ms | 19.884 ms | 100 ms | 11,164 B |
+| Filter, limit 20 | 18.274 ms | 19.534 ms | 100 ms | 8,572 B |
+| Backlinks, limit 20 | 18.039 ms | 18.455 ms | 100 ms | 1,210 B |
+| Dependencies, depth 8 | 18.117 ms | 19.061 ms | 100 ms | 2,559 B |
+| Impact, depth 8 | 18.059 ms | 18.716 ms | 100 ms | 2,553 B |
+| Context, 32,000-token budget, page 20 | 25.395 ms | 25.867 ms | 250 ms | 13,152 B |
+| Current render receipt status | 18.969 ms | 19.613 ms | 50 ms | 1,603 B |
+| Stale render receipt status | 18.910 ms | 19.087 ms | 50 ms | 1,551 B |
+| Missing render receipt status | 17.986 ms | 18.746 ms | 50 ms | 1,352 B |
+| Corrupt render receipt status | 18.007 ms | 18.511 ms | 50 ms | 1,352 B |
+| Current visualization status | 9.875 ms | 10.836 ms | 50 ms | 1,446 B |
+| Stale visualization status | 10.050 ms | 10.472 ms | 50 ms | 1,438 B |
+| Not-running visualization status | 0.258 ms | 0.288 ms | 50 ms | 1,008 B |
+| Unavailable visualization status | 0.053 ms | 0.081 ms | 50 ms | 1,138 B |
+
+Every measured p95 passed its maintained ceiling. Exact retrieval is 15.4 times faster than the
+Milestone 0 median. Warm synchronization is 15.5 times faster. The paged context response is 17.2
+times faster and 19.6 times smaller than the inherited full response.
+
+The cumulative process peak was 940,116 KiB. This is not an operation-local steady-state value. It
+includes fixture construction, all benchmark phases, and Python allocator high-water behavior. It
+excludes the detached visualization worker. Milestone 0's isolated subprocess measurements remain
+the better evidence for per-operation steady-state memory until a maintained operation-local memory
+harness is added.
+
+## Work-proof counters
+
+Routine retrieval and status operations recorded:
+
+- Zero complete project loads.
+- Zero canonical files or bytes parsed.
+- Zero adapter projection loads and source extractions.
+- Zero index builds.
+- Zero render preparations, output bytes constructed, or complete output bytes hashed.
+- One index check and two cheap source-generation checks for each pinned retrieval.
+- One viewer-manager request for each running visualization-status query.
+
+No-change synchronization recorded one synchronization and no build. Receipt and visualization
+status recorded no hidden synchronization. The counter contract is fixed, schema-validated, and
+executed by the repository gate.
+
+## Meaning of the result
+
+The Milestone 0 evidence showed that SQLite queries were already fast after a generation was
+pinned. Milestone 1 confirms that repeated source discovery, parsing, and validation were the
+dominant cost. A versioned source-generation receipt, immutable SQLite read snapshot, and bounded
+indexed operations remove that cost without changing graph authority or storage.
+
+The evidence still does not justify replacing SQLite. Complete project loading, complete index
+checking, full adapter projection, and deep render validation remain independent truth and recovery
+oracles.
+
+## Known limits
+
+- The context compiler still materializes its bounded selected graph before transport pagination.
+  A streaming planner requires separate scale evidence.
+- Generic stat identities are cheap publication proofs, not cryptographic integrity scans.
+- Legacy non-incremental adapters may not provide a cheap generation identity.
+- Incremental adapter manifest, invalidation, extraction, and assembly are not yet measured at
+  1,000-source scale.
+- Pagination cursors detect corruption and stale generations. They are not authenticated
+  authorization tokens.
+- An individually oversized context entry is returned as explicit hash-identified omission
+  evidence. Targeted retrieval is required for its content.
+- The maintained process peak is cumulative and excludes detached worker memory.
+- Manual and graph render plans do not exist until Milestone 3.
+

docs/MILESTONE_1_CLOSEOUT.md

@@ -0,0 +1,99 @@
+# DocForge2 Milestone 1 closeout
+
+Milestone 1 establishes a fast, observable core without changing graph meaning, canonical
+authority, the supported `docforge` identity, or the legacy adapter boundary.
+
+## Completed contracts
+
+- Generic projects publish a versioned source-generation receipt only after complete stable
+  verification.
+- Routine reads validate file and membership-directory identities without parsing canonical
+  sources.
+- Every indexed read uses one immutable read-only SQLite transaction pinned between source and
+  index identity checks.
+- Dependency validation is linear in nodes and edges and uses an iterative deterministic cycle
+  check.
+- Search, filtering, backlinks, dependency, impact, context, and changeset review results are
+  bounded independently of project size.
+- Version-1 cursors bind the project, adapter, canonical generation, query, collection, and
+  position. Corrupt and stale cursors fail closed.
+- Mutation tools preflight their minimum receipt and never report `result_too_large` after a
+  committed operation.
+- Render status uses a publication receipt and performs no hidden render, source parse, index
+  rebuild, or repair.
+- Visualization status separates lifecycle from source and index freshness and performs no hidden
+  SQLite validation or source parse.
+- Request-local diagnostics expose fixed bounded stage timings and work counters without recording
+  source text, paths, node IDs, queries, or SQL.
+
+Complete loading, deep index checking, deep render status, and full adapter projection remain the
+recovery and equivalence oracles.
+
+## Compatibility
+
+The distribution, import package, three executable names, existing CLI commands, existing MCP tool
+names, and required arguments remain supported. New limits, cursors, deep-status switches, and
+diagnostics are additive.
+
+A one-method `load_projection()` adapter remains supported. Incremental behavior remains optional.
+The no-AST binding continues to reject Logic publication and every Logic retrieval surface,
+including application refresh and live visualization.
+
+The disposable SQLite index schema is version 3. Version 2 indexes rebuild automatically. No
+canonical source or stored proposal is migrated to satisfy the new index.
+
+## Verification
+
+The complete repository gate passed at clean commit
+`6253c45a5eca01efa8c73ea3dfe4d85c55878ada`:
+
+- Ruff formatting and lint.
+- HTML, rendered-manual HTML, CSS, and JavaScript lint.
+- Pyright with zero diagnostics.
+- Python compilation.
+- Public-contract and no-AST checks.
+- 138 tests and 77 subtests under warnings-as-errors.
+- Lock and JavaScript dependency-tree checks.
+- Wheel and source-distribution builds.
+- Milestone 0 benchmark smoke.
+- Milestone 1 counter and latency smoke.
+
+The maintained clean 1,000-node benchmark passed every latency and work-counter threshold. Exact
+retrieval measured 18.577 ms p95. Paged 32,000-token context measured 25.867 ms p95. Warm no-change
+synchronization measured 9.324 ms p95. Render receipt status measured 19.613 ms p95.
+Visualization status measured 10.836 ms p95.
+
+Detailed evidence is in
+[`MILESTONE_1_BASELINE.md`](MILESTONE_1_BASELINE.md) and
+[`benchmarks/milestone1-2026-07-29.json`](../benchmarks/milestone1-2026-07-29.json).
+
+## Measured decisions
+
+SQLite remains the derived retrieval store. The benchmark demonstrates that whole-source
+validation around SQLite, not SQLite retrieval itself, caused the inherited latency. No speculative
+storage rewrite was made.
+
+Receipt caches remain disposable and fail closed. A missing, corrupt, incompatible, foreign, or
+changed receipt falls back to the complete oracle or reports an explicit unverified state according
+to the operation's safety contract.
+
+Pagination is transport state, not project authority. It adds no database and grants no
+authorization.
+
+## Remaining weaknesses
+
+- Context selection is bounded but not yet streaming internally.
+- Scaled incremental-adapter performance remains unmeasured.
+- Generic cheap generation proof uses filesystem identity rather than content hashing on every
+  read.
+- Legacy adapters without incremental state cannot always prove current identity cheaply.
+- One oversized context entry requires targeted retrieval after an explicit omission.
+- Operation-local and detached-worker memory need a maintained isolated harness.
+- Tree-sitter and its JavaScript and C++ grammars remain mandatory package dependencies.
+- `ManualRenderPlan`, `GraphViewPlan`, and independent renderer packages remain Milestone 3 work.
+
+## Scope confirmation
+
+Milestone 1 did not self-host DocForge2, change WorldForge or ScrapeStation, repoint a production
+MCP integration, modify the legacy Forgejo repository, create a tag, or create a release.
+