DocForge2/docs/MILESTONE_1_BASELINE.md

DocForge2 Milestone 1 baseline

Milestone 1 removes repeated whole-project work from routine warm reads while retaining complete loading and deep validation as recovery and equivalence oracles. The maintained machine-readable evidence is benchmarks/milestone1-2026-07-29.json, captured from clean commit 6253c45a5eca01efa8c73ea3dfe4d85c55878ada.

Environment and method

• Platform: x86-64 Linux 7.1.3 with glibc 2.43.
• Python: CPython 3.14.6.
• Fixture: 1,000 Markdown files, 1,000 nodes, and 999 dependency edges.
• Samples: ten measured invocations after validated warmups.
• Duration clock: time.perf_counter_ns().
• Percentile: nearest rank, so p95 is the maximum with ten samples.
• Response size: UTF-8 bytes of compact, sorted JSON.
• Process memory: cumulative main-process RUSAGE_SELF high-water mark.

All canonical sources, caches, indexes, changesets, renders, and viewer state were created in a disposable temporary directory. The run did not read WorldForge, ScrapeStation, legacy DocForge indexes, or production MCP state.

The benchmark validates every warmup and measured result. It also fails when a routine warm operation performs a project load, parses canonical source, rebuilds an adapter projection, extracts adapter sources, builds an index, prepares a render, constructs rendered output, or hashes complete rendered output.

Maintained 1,000-node results

Operation	Median	p95	Gate	Response
Warm no-change synchronization	9.192 ms	9.324 ms	100 ms	1,570 B
Exact node	17.887 ms	18.577 ms	50 ms	1,487 B
Missing-node error	18.170 ms	18.320 ms	50 ms	1,130 B
Search, limit 20	19.518 ms	19.884 ms	100 ms	11,164 B
Filter, limit 20	18.274 ms	19.534 ms	100 ms	8,572 B
Backlinks, limit 20	18.039 ms	18.455 ms	100 ms	1,210 B
Dependencies, depth 8	18.117 ms	19.061 ms	100 ms	2,559 B
Impact, depth 8	18.059 ms	18.716 ms	100 ms	2,553 B
Context, 32,000-token budget, page 20	25.395 ms	25.867 ms	250 ms	13,152 B
Current render receipt status	18.969 ms	19.613 ms	50 ms	1,603 B
Stale render receipt status	18.910 ms	19.087 ms	50 ms	1,551 B
Missing render receipt status	17.986 ms	18.746 ms	50 ms	1,352 B
Corrupt render receipt status	18.007 ms	18.511 ms	50 ms	1,352 B
Current visualization status	9.875 ms	10.836 ms	50 ms	1,446 B
Stale visualization status	10.050 ms	10.472 ms	50 ms	1,438 B
Not-running visualization status	0.258 ms	0.288 ms	50 ms	1,008 B
Unavailable visualization status	0.053 ms	0.081 ms	50 ms	1,138 B

Every measured p95 passed its maintained ceiling. Exact retrieval is 15.4 times faster than the Milestone 0 median. Warm synchronization is 15.5 times faster. The paged context response is 17.2 times faster and 19.6 times smaller than the inherited full response.

The cumulative process peak was 940,116 KiB. This is not an operation-local steady-state value. It includes fixture construction, all benchmark phases, and Python allocator high-water behavior. It excludes the detached visualization worker. Milestone 0's isolated subprocess measurements remain the better evidence for per-operation steady-state memory until a maintained operation-local memory harness is added.

Work-proof counters

Routine retrieval and status operations recorded:

• Zero complete project loads.
• Zero canonical files or bytes parsed.
• Zero adapter projection loads and source extractions.
• Zero index builds.
• Zero render preparations, output bytes constructed, or complete output bytes hashed.
• One index check and two cheap source-generation checks for each pinned retrieval.
• One viewer-manager request for each running visualization-status query.

No-change synchronization recorded one synchronization and no build. Receipt and visualization status recorded no hidden synchronization. The counter contract is fixed, schema-validated, and executed by the repository gate.

Meaning of the result

The Milestone 0 evidence showed that SQLite queries were already fast after a generation was pinned. Milestone 1 confirms that repeated source discovery, parsing, and validation were the dominant cost. A versioned source-generation receipt, immutable SQLite read snapshot, and bounded indexed operations remove that cost without changing graph authority or storage.

The evidence still does not justify replacing SQLite. Complete project loading, complete index checking, full adapter projection, and deep render validation remain independent truth and recovery oracles.

Known limits

• The context compiler still materializes its bounded selected graph before transport pagination. A streaming planner requires separate scale evidence.
• Generic stat identities are cheap publication proofs, not cryptographic integrity scans.
• Legacy non-incremental adapters may not provide a cheap generation identity.
• Incremental adapter manifest, invalidation, extraction, and assembly are not yet measured at 1,000-source scale.
• Pagination cursors detect corruption and stale generations. They are not authenticated authorization tokens.
• An individually oversized context entry is returned as explicit hash-identified omission evidence. Targeted retrieval is required for its content.
• The maintained process peak is cumulative and excludes detached worker memory.
• Manual and graph render plans do not exist until Milestone 3.