Close Milestone 2 with measured evidence

docs/MILESTONE_2_BASELINE.md

@@ -0,0 +1,85 @@
+# Milestone 2 baseline
+
+## Scope and method
+
+This baseline records the agent-retrieval and client-integration behavior added in Milestone 2.
+It was captured on 2026-07-29 from clean candidate commit
+`fb0df5e4a1c591c2a84788fd4814d98550f11863`.
+
+The maintained command was:
+
+```bash
+.venv/bin/python tools/milestone2_benchmark.py \
+  --nodes 1000 \
+  --samples 10 \
+  --output /tmp/docforge-milestone2-final.json
+```
+
+The fixture contains 1,000 Markdown nodes and 999 edges in a direct fan-in around one focus node.
+The configured MCP response limit is 200,000 characters. Durations use
+`time.perf_counter_ns()` and nearest-rank p95. Peak memory uses an isolated child process and
+`RUSAGE_SELF`. Every warmup and measured invocation is validated.
+
+Environment:
+
+- Linux 7.1.3-200.nobara.fc44.x86_64.
+- CPython 3.14.6.
+- x86_64.
+- Ten warm samples after one warmup.
+- Isolated memory ceiling: 262,144 KiB.
+
+The complete machine-readable result is
+[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json).
+
+## Results
+
+| Operation | Median | p95 | Limit | Maximum response |
+|---|---:|---:|---:|---:|
+| Read bootstrap | 9.406 ms | 9.884 ms | 100 ms | 7,495 B |
+| No-AST bootstrap | 9.153 ms | 9.379 ms | 100 ms | 8,796 B |
+| Task diagnostic page | 73.326 ms | 93.115 ms | 500 ms | 7,190 B |
+| Task complete traversal | 703.561 ms | 721.847 ms | 2,500 ms | 151,172 B/page |
+| Generation diagnostic page | 40.961 ms | 41.650 ms | 100 ms | 66,516 B |
+| Generation maximum page | 55.565 ms | 59.184 ms | 100 ms | 199,566 B |
+| Generation complete traversal | 418.607 ms | 425.315 ms | 500 ms | 66,516 B/page |
+| Codex configuration preview | 314.365 ms | 364.383 ms | 500 ms | 2,627 B |
+| Claude configuration preview | 314.326 ms | 364.365 ms | 500 ms | 2,748 B |
+| OpenClaw configuration preview | 314.401 ms | 364.532 ms | 500 ms | 2,869 B |
+| Codex doctor | 0.421 ms | 0.556 ms | 100 ms | 3,595 B |
+| Claude doctor | 0.364 ms | 0.446 ms | 100 ms | 3,669 B |
+| OpenClaw doctor | 0.384 ms | 0.484 ms | 100 ms | 3,602 B |
+
+Isolated peak RSS was 86,448 KiB.
+
+Task traversal returned 108 evidence records and 892 explicit omissions across 11 pages. One
+individually oversized focus record became a response-limit surrogate bound to the original record
+hash. The remaining omissions were token-budget evidence. The benchmark verified every unique
+subject, reconstructed the original collection hash, and matched the exact 1,000-node fixture.
+
+Generation traversal returned all 1,000 changed-node details across 10 pages. It reconstructed the
+stored retained-collection hash. The maximum generation page approached the response limit and
+proved that optional diagnostics were dropped before the primary result.
+
+## Structured-work gates
+
+Configuration preview and doctor performed zero project loads, source parses, adapter projection
+loads, adapter extraction, index checks, synchronization, index builds, render preparation,
+rendered-byte construction or hashing, and viewer-manager requests.
+
+Task-context pages performed exactly one index check and two cheap source-generation checks. They
+performed none of the hidden work above. Generation-diff pages performed exactly two cheap
+source-generation checks and no index check or hidden work.
+
+## Measured limits and future notes
+
+- Continuation is stateless and regenerates the task capsule for each page. The complete
+  11-page traversal remains within its gate, but later work can avoid repeated planning without
+  weakening generation binding.
+- Configuration preview deliberately spends about 314 ms proving that the exact isolated
+  interpreter can import the MCP module. Discovery-only checks were rejected as unsafe.
+- Claude configuration syntax is supported, but its timeout representation remains unverified.
+  Doctor therefore reports degraded rather than healthy.
+- Doctor is a configuration inspector, not an MCP connection or SQLite integrity test.
+- Legacy adapters without cheap source-generation identity report unknown for generation-diff
+  freshness.
+- The results do not justify a storage rewrite. SQLite remains fast after one generation is pinned.