DocForge2/benchmarks/README.md

# Benchmarks

Milestone 0 records measurements before changing compiler, storage, rendering, or response
contracts.

Run the maintained smoke benchmark:

```bash
make benchmark-smoke
```

Run the 1,000-node generic baseline:

```bash
make benchmark
```

Run the Milestone 1 warm-operation counter and latency smoke gate:

```bash
make benchmark-m1-smoke
```

Run the maintained 1,000-node Milestone 1 benchmark:

```bash
make benchmark-m1
```

Run the Milestone 2 agent-workflow smoke and full gates:

```bash
make benchmark-m2-smoke
make benchmark-m2
```

The benchmark creates canonical sources, derived state, changesets, rendered output, and caches
only in a disposable temporary directory. It does not read another project, self-host DocForge, or
mutate repository content.

`milestone0-2026-07-29.json` is the clean-tree baseline captured from commit
`fd4759096e90edb13a745621aae4872f23079357`. It uses compact sorted JSON for response sizes and
`time.perf_counter_ns()` for durations. The file is data, not a performance threshold. Later work
must explain fixture or environment changes before comparing results.

`milestone1-2026-07-29.json` is the clean-tree fast-core baseline captured from commit
`6253c45a5eca01efa8c73ea3dfe4d85c55878ada`. Unlike the historical baseline, the Milestone 1
harness enforces operation-specific p95 ceilings and fixed zero-work counter invariants. Its
human-readable interpretation is in
[`docs/MILESTONE_1_BASELINE.md`](../docs/MILESTONE_1_BASELINE.md).

`milestone2-2026-07-29.json` is the clean-tree agent-retrieval and client-integration baseline
captured from commit `fb0df5e4a1c591c2a84788fd4814d98550f11863`. It gates every warmup and
sample, reconstructs complete task-context and generation-diff collections across bounded pages,
records whether diagnostics were dropped for response budget, checks all hidden-work counters,
and measures isolated-process peak RSS. Its interpretation is in
[`docs/MILESTONE_2_BASELINE.md`](../docs/MILESTONE_2_BASELINE.md).

The generic fixture exposes whole-source scaling. It does not replace the incremental adapter
equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has
no portable graph-planning or graph-rendering contract.

The Milestone 1 harness treats wall time and structured work counters as separate gates. Warm
operations fail if they load a complete project, parse source files, reconstruct an adapter
projection, extract adapter sources, build an index, prepare a render, construct rendered output,
or hash complete rendered output. Its latency ceilings are the Milestone 1 targets, not claims
about all hardware.

Every warmup and measured invocation is validated. The recorded counter ranges also require one
index synchronization for the synchronization operation, no hidden synchronization for reads and
status, one index check for each retrieval snapshot, and exactly one manager request for viewer
status. The 1,000-node run records bounded semantic summaries for exact errors, search, filtering,
backlinks, both traversal directions, paged context, render receipt states, and visualization
freshness. The reported p95 uses the nearest-rank method; with ten samples it is the maximum.
`process_peak_rss_kib` is the cumulative main-process `RUSAGE_SELF` high-water mark and excludes the
detached viewer worker.