DocForge2/docs/MILESTONE_2_BASELINE.md

Milestone 2 baseline

Scope and method

This baseline records the agent-retrieval and client-integration behavior added in Milestone 2. It was captured on 2026-07-29 from clean candidate commit fb0df5e4a1c591c2a84788fd4814d98550f11863.

The maintained command was:

.venv/bin/python tools/milestone2_benchmark.py \
  --nodes 1000 \
  --samples 10 \
  --output /tmp/docforge-milestone2-final.json

The fixture contains 1,000 Markdown nodes and 999 edges in a direct fan-in around one focus node. The configured MCP response limit is 200,000 characters. Durations use time.perf_counter_ns() and nearest-rank p95. Peak memory uses an isolated child process and RUSAGE_SELF. Every warmup and measured invocation is validated.

Environment:

• Linux 7.1.3-200.nobara.fc44.x86_64.
• CPython 3.14.6.
• x86_64.
• Ten warm samples after one warmup.
• Isolated memory ceiling: 262,144 KiB.

The complete machine-readable result is benchmarks/milestone2-2026-07-29.json.

Results

Operation	Median	p95	Limit	Maximum response
Read bootstrap	9.406 ms	9.884 ms	100 ms	7,495 B
No-AST bootstrap	9.153 ms	9.379 ms	100 ms	8,796 B
Task diagnostic page	73.326 ms	93.115 ms	500 ms	7,190 B
Task complete traversal	703.561 ms	721.847 ms	2,500 ms	151,172 B/page
Generation diagnostic page	40.961 ms	41.650 ms	100 ms	66,516 B
Generation maximum page	55.565 ms	59.184 ms	100 ms	199,566 B
Generation complete traversal	418.607 ms	425.315 ms	500 ms	66,516 B/page
Codex configuration preview	314.365 ms	364.383 ms	500 ms	2,627 B
Claude configuration preview	314.326 ms	364.365 ms	500 ms	2,748 B
OpenClaw configuration preview	314.401 ms	364.532 ms	500 ms	2,869 B
Codex doctor	0.421 ms	0.556 ms	100 ms	3,595 B
Claude doctor	0.364 ms	0.446 ms	100 ms	3,669 B
OpenClaw doctor	0.384 ms	0.484 ms	100 ms	3,602 B

Isolated peak RSS was 86,448 KiB.

Task traversal returned 108 evidence records and 892 explicit omissions across 11 pages. One individually oversized focus record became a response-limit surrogate bound to the original record hash. The remaining omissions were token-budget evidence. The benchmark verified every unique subject, reconstructed the original collection hash, and matched the exact 1,000-node fixture.

Generation traversal returned all 1,000 changed-node details across 10 pages. It reconstructed the stored retained-collection hash. The maximum generation page approached the response limit and proved that optional diagnostics were dropped before the primary result.

Structured-work gates

Configuration preview and doctor performed zero project loads, source parses, adapter projection loads, adapter extraction, index checks, synchronization, index builds, render preparation, rendered-byte construction or hashing, and viewer-manager requests.

Task-context pages performed exactly one index check and two cheap source-generation checks. They performed none of the hidden work above. Generation-diff pages performed exactly two cheap source-generation checks and no index check or hidden work.

Measured limits and future notes

• Continuation is stateless and regenerates the task capsule for each page. The complete 11-page traversal remains within its gate, but later work can avoid repeated planning without weakening generation binding.
• Configuration preview deliberately spends about 314 ms proving that the exact isolated interpreter can import the MCP module. Discovery-only checks were rejected as unsafe.
• Claude configuration syntax is supported, but its timeout representation remains unverified. Doctor therefore reports degraded rather than healthy.
• Doctor is a configuration inspector, not an MCP connection or SQLite integrity test.
• Legacy adapters without cheap source-generation identity report unknown for generation-diff freshness.
• The results do not justify a storage rewrite. SQLite remains fast after one generation is pinned.