Close Milestone 2 with measured evidence
This commit is contained in:
parent
fb0df5e4a1
commit
4c5773c865
10 changed files with 771 additions and 9 deletions
|
|
@ -6,14 +6,14 @@ Goal: Let one project-bound server return compact, task-shaped, explainable cont
|
||||||
In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks.
|
In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks.
|
||||||
Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
|
Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
|
||||||
Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass.
|
Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass.
|
||||||
Status: Candidate frozen. Effective policy, versioned task retrieval, latest-generation diff
|
Status: Complete. Effective policy, versioned task retrieval, latest-generation diff receipts,
|
||||||
receipts, and logarithmic task-context page packing are committed and pushed on `dev`.
|
logarithmic bounded page packing, deterministic client configuration, and the read-only integration
|
||||||
Deterministic client configuration and the read-only integration doctor now pass their bounded
|
doctor are implemented and contract-tested. The complete repository gate passes with 205 tests and
|
||||||
publication, path-race, malformed-input, redaction, and no-hidden-work audits. The complete
|
120 subtests. Three independent adversarial audits found no remaining implementation blocker. The
|
||||||
repository gate passes with 205 tests and 120 subtests. A disposable 1,000-node audit sample passes
|
clean 1,000-node baseline is recorded against candidate commit
|
||||||
the maintained task-context, generation-diff, response-size, counter, and memory gates. Final
|
`fb0df5e4a1c591c2a84788fd4814d98550f11863`, including task/generation reconstruction,
|
||||||
clean-revision benchmark evidence and documentation closeout remain before the milestone is marked
|
response-size behavior, zero-hidden-work counters, and isolated memory. No tag or release was
|
||||||
complete.
|
created, no production integration was repointed, and self-hosting remains out of scope.
|
||||||
```
|
```
|
||||||
|
|
||||||
Milestones 3–5 remain directional context and are not active.
|
Milestones 3–5 remain directional context and are not active.
|
||||||
|
|
|
||||||
|
|
@ -583,3 +583,30 @@ compilation, lock and dependency checks, package builds, and all three milestone
|
||||||
Three independent final audits approve client publication and policy binding, doctor fail-closed
|
Three independent final audits approve client publication and policy binding, doctor fail-closed
|
||||||
behavior, and benchmark/contract coverage. Clean-revision benchmark evidence is still required
|
behavior, and benchmark/contract coverage. Clean-revision benchmark evidence is still required
|
||||||
before closeout.
|
before closeout.
|
||||||
|
|
||||||
|
### Milestone 2 closeout
|
||||||
|
|
||||||
|
Candidate commit `fb0df5e4a1c591c2a84788fd4814d98550f11863` passed the clean ten-sample
|
||||||
|
Milestone 2 benchmark. Task-context complete traversal measured 703.561 ms median and 721.847 ms
|
||||||
|
p95 across 11 bounded pages. It reconstructed the exact 1,000-candidate collection from 108 cited
|
||||||
|
evidence records, 891 original token-budget omissions, and one hash-attested response-limit
|
||||||
|
surrogate. Generation-diff complete traversal measured 418.607 ms median and 425.315 ms p95 across
|
||||||
|
10 pages.
|
||||||
|
|
||||||
|
Read and no-AST bootstrap remained below 10 ms p95. The maximum generation page used 199,566 bytes
|
||||||
|
of the 200,000-byte budget and correctly discarded diagnostics before primary evidence.
|
||||||
|
Configuration preview measured about 314 ms median and 365 ms p95 because it proves the real
|
||||||
|
isolated interpreter import on every invocation. Codex and OpenClaw doctor checks remained below
|
||||||
|
0.6 ms p95; Claude remained explicitly degraded because its timeout format is unverified.
|
||||||
|
Isolated-process peak RSS was 86,448 KiB against the 262,144 KiB gate.
|
||||||
|
|
||||||
|
All measured configuration and doctor counters were zero. Task-context pages performed one index
|
||||||
|
check and two cheap generation checks with no loads, parses, synchronization, builds, extraction,
|
||||||
|
rendering, or viewer work. Generation-diff pages performed two cheap generation checks and no
|
||||||
|
index check. The canonical machine-readable result is
|
||||||
|
`benchmarks/milestone2-2026-07-29.json`.
|
||||||
|
|
||||||
|
Milestone 2 is complete. Follow-up ideas stay explicitly later-scope: avoid recomputing the
|
||||||
|
task-shaped capsule for every continuation page, add authenticated continuation when the threat
|
||||||
|
model requires it, verify a native Claude timeout representation, and introduce adapter-owned
|
||||||
|
launcher metadata before generating configurations for custom adapters.
|
||||||
|
|
|
||||||
20
README.md
20
README.md
|
|
@ -13,6 +13,10 @@ declared manuals, visualizes project structure, and manages reviewable documenta
|
||||||
continuation.
|
continuation.
|
||||||
- Records one bounded, versioned latest-generation graph transition without creating a history
|
- Records one bounded, versioned latest-generation graph transition without creating a history
|
||||||
database.
|
database.
|
||||||
|
- Generates deterministic project-bound Codex, Claude, and OpenClaw client fragments without
|
||||||
|
copying ambient secrets.
|
||||||
|
- Diagnoses one client binding through bounded read-only checks without starting MCP or rebuilding
|
||||||
|
project state.
|
||||||
- Automatically synchronizes disposable indexes before MCP work.
|
- Automatically synchronizes disposable indexes before MCP work.
|
||||||
- Creates, validates, diffs, and previews isolated changesets.
|
- Creates, validates, diffs, and previews isolated changesets.
|
||||||
- Registers complete proposals atomically without caller-managed hash chaining.
|
- Registers complete proposals atomically without caller-managed hash chaining.
|
||||||
|
|
@ -135,6 +139,16 @@ Start an MCP server for one project:
|
||||||
Add `--canonical-applier project-editor` only when that MCP integration should expose the
|
Add `--canonical-applier project-editor` only when that MCP integration should expose the
|
||||||
hash-bound `docforge_apply_changeset` tool.
|
hash-bound `docforge_apply_changeset` tool.
|
||||||
|
|
||||||
|
Preview a read-only Codex fragment and diagnose an installed binding:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/docforge configure codex --project "$PROJECT"
|
||||||
|
.venv/bin/docforge doctor --client codex --project "$PROJECT"
|
||||||
|
```
|
||||||
|
|
||||||
|
Pass `--output /absolute/path/docforge.toml` only when creating a standalone fragment. DocForge
|
||||||
|
never replaces or merges an existing different client file.
|
||||||
|
|
||||||
For an unconfigured codebase, begin with a read-only language and documentation assessment:
|
For an unconfigured codebase, begin with a read-only language and documentation assessment:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -156,6 +170,10 @@ DocForge describes them as a source graph.
|
||||||
performance, memory, rendering and response sizes, bottlenecks, and missing coverage.
|
performance, memory, rendering and response sizes, bottlenecks, and missing coverage.
|
||||||
- [Milestone 0 closeout](docs/MILESTONE_0_CLOSEOUT.md) — lineage, migration, security scan,
|
- [Milestone 0 closeout](docs/MILESTONE_0_CLOSEOUT.md) — lineage, migration, security scan,
|
||||||
repository state, and fresh-clone proof.
|
repository state, and fresh-clone proof.
|
||||||
|
- [Milestone 2 baseline](docs/MILESTONE_2_BASELINE.md) — task context, generation diff, client
|
||||||
|
configuration, doctor, response-size, counter, and memory measurements.
|
||||||
|
- [Milestone 2 closeout](docs/MILESTONE_2_CLOSEOUT.md) — implemented contracts, adversarial
|
||||||
|
validation, exclusions, and exact candidate evidence.
|
||||||
- [MCP contract](docs/MCP_CONTRACT.md) — exact tool and process boundary.
|
- [MCP contract](docs/MCP_CONTRACT.md) — exact tool and process boundary.
|
||||||
- [Viewer manager](docs/VIEWER_MANAGER.md) — native service setup and lifecycle.
|
- [Viewer manager](docs/VIEWER_MANAGER.md) — native service setup and lifecycle.
|
||||||
- [Adapter decision](docs/APPLICATION_DECISION.md) — why custom adapters own canonical
|
- [Adapter decision](docs/APPLICATION_DECISION.md) — why custom adapters own canonical
|
||||||
|
|
@ -178,7 +196,7 @@ make gate
|
||||||
|
|
||||||
Focused entry points are available as `make contract`, `make test`, `make type`,
|
Focused entry points are available as `make contract`, `make test`, `make type`,
|
||||||
`make benchmark-smoke`, `make benchmark`, `make benchmark-m1-smoke`, and
|
`make benchmark-smoke`, `make benchmark`, `make benchmark-m1-smoke`, and
|
||||||
`make benchmark-m1`.
|
`make benchmark-m1`. Milestone 2 adds `make benchmark-m2-smoke` and `make benchmark-m2`.
|
||||||
|
|
||||||
The committed 1,000-node baseline and its measurement method are under `benchmarks/`.
|
The committed 1,000-node baseline and its measurement method are under `benchmarks/`.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -27,6 +27,13 @@ Run the maintained 1,000-node Milestone 1 benchmark:
|
||||||
make benchmark-m1
|
make benchmark-m1
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Run the Milestone 2 agent-workflow smoke and full gates:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make benchmark-m2-smoke
|
||||||
|
make benchmark-m2
|
||||||
|
```
|
||||||
|
|
||||||
The benchmark creates canonical sources, derived state, changesets, rendered output, and caches
|
The benchmark creates canonical sources, derived state, changesets, rendered output, and caches
|
||||||
only in a disposable temporary directory. It does not read another project, self-host DocForge, or
|
only in a disposable temporary directory. It does not read another project, self-host DocForge, or
|
||||||
mutate repository content.
|
mutate repository content.
|
||||||
|
|
@ -42,6 +49,13 @@ harness enforces operation-specific p95 ceilings and fixed zero-work counter inv
|
||||||
human-readable interpretation is in
|
human-readable interpretation is in
|
||||||
[`docs/MILESTONE_1_BASELINE.md`](../docs/MILESTONE_1_BASELINE.md).
|
[`docs/MILESTONE_1_BASELINE.md`](../docs/MILESTONE_1_BASELINE.md).
|
||||||
|
|
||||||
|
`milestone2-2026-07-29.json` is the clean-tree agent-retrieval and client-integration baseline
|
||||||
|
captured from commit `fb0df5e4a1c591c2a84788fd4814d98550f11863`. It gates every warmup and
|
||||||
|
sample, reconstructs complete task-context and generation-diff collections across bounded pages,
|
||||||
|
records whether diagnostics were dropped for response budget, checks all hidden-work counters,
|
||||||
|
and measures isolated-process peak RSS. Its interpretation is in
|
||||||
|
[`docs/MILESTONE_2_BASELINE.md`](../docs/MILESTONE_2_BASELINE.md).
|
||||||
|
|
||||||
The generic fixture exposes whole-source scaling. It does not replace the incremental adapter
|
The generic fixture exposes whole-source scaling. It does not replace the incremental adapter
|
||||||
equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has
|
equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has
|
||||||
no portable graph-planning or graph-rendering contract.
|
no portable graph-planning or graph-rendering contract.
|
||||||
|
|
|
||||||
457
benchmarks/milestone2-2026-07-29.json
Normal file
457
benchmarks/milestone2-2026-07-29.json
Normal file
|
|
@ -0,0 +1,457 @@
|
||||||
|
{
|
||||||
|
"benchmark": "docforge2_milestone2",
|
||||||
|
"environment": {
|
||||||
|
"implementation": "CPython",
|
||||||
|
"machine": "x86_64",
|
||||||
|
"platform": "Linux-7.1.3-200.nobara.fc44.x86_64-x86_64-with-glibc2.43",
|
||||||
|
"python": "3.14.6"
|
||||||
|
},
|
||||||
|
"fixture": {
|
||||||
|
"edge_count": 999,
|
||||||
|
"kind": "synthetic_generic_focus_fan_in",
|
||||||
|
"max_tool_output_chars": 200000,
|
||||||
|
"node_count": 1000,
|
||||||
|
"source_file_count": 1000
|
||||||
|
},
|
||||||
|
"isolated_process_peak_rss_kib": 86448,
|
||||||
|
"method": {
|
||||||
|
"clock": "time.perf_counter_ns",
|
||||||
|
"memory": "isolated child-process resource.getrusage(RUSAGE_SELF).ru_maxrss",
|
||||||
|
"memory_limit_kib": 262144,
|
||||||
|
"memory_probe_samples": 10,
|
||||||
|
"percentile": "nearest-rank",
|
||||||
|
"response_size": "UTF-8 bytes of compact sorted JSON",
|
||||||
|
"samples": 10,
|
||||||
|
"warmups": 1,
|
||||||
|
"zero_work_counters": [
|
||||||
|
"project_loads",
|
||||||
|
"source_files_parsed",
|
||||||
|
"source_bytes_parsed",
|
||||||
|
"adapter_projection_loads",
|
||||||
|
"adapter_source_extractions",
|
||||||
|
"index_synchronizations",
|
||||||
|
"index_builds",
|
||||||
|
"render_prepare_calls",
|
||||||
|
"render_output_bytes_built",
|
||||||
|
"render_output_bytes_hashed",
|
||||||
|
"viewer_manager_requests"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"operations": {
|
||||||
|
"bootstrap_no_ast": {
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 1, "minimum": 1},
|
||||||
|
"index_synchronizations": {"maximum": 1, "minimum": 1},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 1, "minimum": 1},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"max_ms": 9.379,
|
||||||
|
"maximum_response_bytes": 8796,
|
||||||
|
"median_ms": 9.153,
|
||||||
|
"min_ms": 9.026,
|
||||||
|
"p95_limit_ms": 100,
|
||||||
|
"p95_ms": 9.379,
|
||||||
|
"response_bytes": 8795,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"bootstrap_read": {
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 1, "minimum": 1},
|
||||||
|
"index_synchronizations": {"maximum": 1, "minimum": 1},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 1, "minimum": 1},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"max_ms": 9.884,
|
||||||
|
"maximum_response_bytes": 7495,
|
||||||
|
"median_ms": 9.406,
|
||||||
|
"min_ms": 9.053,
|
||||||
|
"p95_limit_ms": 100,
|
||||||
|
"p95_ms": 9.884,
|
||||||
|
"response_bytes": 7491,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"configuration_preview": {
|
||||||
|
"claude": {
|
||||||
|
"artifact_format": "claude-json-fragment-v1",
|
||||||
|
"configuration_hash": "4dbb4ed0f38264fdba350de8904cc898493d620194b5105b7889c54bd5913c9c",
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"max_ms": 364.365,
|
||||||
|
"maximum_response_bytes": 2748,
|
||||||
|
"median_ms": 314.326,
|
||||||
|
"min_ms": 314.278,
|
||||||
|
"p95_limit_ms": 500,
|
||||||
|
"p95_ms": 364.365,
|
||||||
|
"response_bytes": 2748,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"codex": {
|
||||||
|
"artifact_format": "codex-toml-fragment-v1",
|
||||||
|
"configuration_hash": "3e1dd5191021da1778cd1c4f4658768537775e5e16252e42d4c80e328841145b",
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"max_ms": 364.383,
|
||||||
|
"maximum_response_bytes": 2627,
|
||||||
|
"median_ms": 314.365,
|
||||||
|
"min_ms": 314.248,
|
||||||
|
"p95_limit_ms": 500,
|
||||||
|
"p95_ms": 364.383,
|
||||||
|
"response_bytes": 2627,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"openclaw": {
|
||||||
|
"artifact_format": "openclaw-json-fragment-v1",
|
||||||
|
"configuration_hash": "6f269e90a55088c5d517f91761c53a3b90036d62fd80b5b9d094668267257b99",
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"max_ms": 364.532,
|
||||||
|
"maximum_response_bytes": 2869,
|
||||||
|
"median_ms": 314.401,
|
||||||
|
"min_ms": 314.251,
|
||||||
|
"p95_limit_ms": 500,
|
||||||
|
"p95_ms": 364.532,
|
||||||
|
"response_bytes": 2869,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"doctor": {
|
||||||
|
"claude": {
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"doctor_state": "degraded",
|
||||||
|
"max_ms": 0.446,
|
||||||
|
"maximum_response_bytes": 3669,
|
||||||
|
"median_ms": 0.364,
|
||||||
|
"min_ms": 0.352,
|
||||||
|
"p95_limit_ms": 100,
|
||||||
|
"p95_ms": 0.446,
|
||||||
|
"response_bytes": 3669,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"summary": {"failed": 0, "passed": 11, "skipped": 1, "warning": 2},
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"codex": {
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"doctor_state": "healthy",
|
||||||
|
"max_ms": 0.556,
|
||||||
|
"maximum_response_bytes": 3595,
|
||||||
|
"median_ms": 0.421,
|
||||||
|
"min_ms": 0.404,
|
||||||
|
"p95_limit_ms": 100,
|
||||||
|
"p95_ms": 0.556,
|
||||||
|
"response_bytes": 3595,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"summary": {"failed": 0, "passed": 13, "skipped": 1, "warning": 0},
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"openclaw": {
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"doctor_state": "healthy",
|
||||||
|
"max_ms": 0.484,
|
||||||
|
"maximum_response_bytes": 3602,
|
||||||
|
"median_ms": 0.384,
|
||||||
|
"min_ms": 0.353,
|
||||||
|
"p95_limit_ms": 100,
|
||||||
|
"p95_ms": 0.484,
|
||||||
|
"response_bytes": 3602,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"summary": {"failed": 0, "passed": 13, "skipped": 1, "warning": 0},
|
||||||
|
"validated_invocations": 11
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"generation_diff_complete": {
|
||||||
|
"max_ms": 425.315,
|
||||||
|
"maximum_response_bytes": 984,
|
||||||
|
"median_ms": 418.607,
|
||||||
|
"min_ms": 410.974,
|
||||||
|
"p95_limit_ms": 500,
|
||||||
|
"p95_ms": 425.315,
|
||||||
|
"response_bytes": 983,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"result_summary": {
|
||||||
|
"aggregate_page_bytes": 664715,
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 2, "minimum": 2},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"elapsed_ms": 419.04,
|
||||||
|
"item_count": 1000,
|
||||||
|
"maximum_cursor_bytes": 448,
|
||||||
|
"maximum_page_bytes": 66516,
|
||||||
|
"ordered_item_hash": "1ac48cc72532809ef5d3e949756e536eec819f348eaf06338c9b39b14e63b2c7",
|
||||||
|
"page_count": 10,
|
||||||
|
"receipt_hash": "6913c962972d8255f56966e5cfab5ac8293e41bdfb5f39ef91a4f33a3b092f88",
|
||||||
|
"status": "ok"
|
||||||
|
},
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"generation_diff_diagnostic_page": {
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 2, "minimum": 2},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"max_ms": 41.65,
|
||||||
|
"maximum_response_bytes": 66516,
|
||||||
|
"median_ms": 40.961,
|
||||||
|
"min_ms": 40.239,
|
||||||
|
"p95_limit_ms": 100,
|
||||||
|
"p95_ms": 41.65,
|
||||||
|
"response_bytes": 66516,
|
||||||
|
"response_limit_bytes": 200000,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"generation_diff_maximum_page": {
|
||||||
|
"diagnostics_dropped_for_budget": true,
|
||||||
|
"max_ms": 59.184,
|
||||||
|
"maximum_response_bytes": 199566,
|
||||||
|
"median_ms": 55.565,
|
||||||
|
"min_ms": 54.68,
|
||||||
|
"p95_limit_ms": 100,
|
||||||
|
"p95_ms": 59.184,
|
||||||
|
"response_bytes": 199566,
|
||||||
|
"response_limit_bytes": 200000,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"task_context_complete": {
|
||||||
|
"max_ms": 721.847,
|
||||||
|
"maximum_response_bytes": 1325,
|
||||||
|
"median_ms": 703.561,
|
||||||
|
"min_ms": 688.975,
|
||||||
|
"p95_limit_ms": 2500,
|
||||||
|
"p95_ms": 721.847,
|
||||||
|
"response_bytes": 1324,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"result_summary": {
|
||||||
|
"aggregate_page_bytes": 348845,
|
||||||
|
"capsule_hash": "20663ed685a255f7cb8a0e8d78262bf7bf26863d728f0159e0b29000f6a52b0a",
|
||||||
|
"collection_hash": "9dbc46eb5b1ac8c8340bb149205d3599fffaf67e3b363de2d035475da274857c",
|
||||||
|
"collection_hash_reconstructed": true,
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 1, "minimum": 1},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 2, "minimum": 2},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"elapsed_ms": 719.55,
|
||||||
|
"evidence_count": 108,
|
||||||
|
"item_count": 1000,
|
||||||
|
"maximum_cursor_bytes": 1066,
|
||||||
|
"maximum_page_bytes": 151172,
|
||||||
|
"omission_count": 892,
|
||||||
|
"ordered_candidate_hash": "bdeb3a8f4018000f72a5ff1891aa800b6edf98070b9814f443c6bac4e52c38f3",
|
||||||
|
"ordered_evidence_hash": "ec07bac7f528f5a3afbc083ea5ad60541335c8789d42980fe7bae0476e0a5331",
|
||||||
|
"page_count": 11,
|
||||||
|
"plan_hash": "84dbe60267e5f8359adcf30d4d395f1ac5d5bea957beaa3888d2938120860a9c",
|
||||||
|
"status": "ok"
|
||||||
|
},
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"task_context_diagnostic_page": {
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 1, "minimum": 1},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 2, "minimum": 2},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"max_ms": 93.115,
|
||||||
|
"maximum_response_bytes": 7190,
|
||||||
|
"median_ms": 73.326,
|
||||||
|
"min_ms": 72.264,
|
||||||
|
"p95_limit_ms": 500,
|
||||||
|
"p95_ms": 93.115,
|
||||||
|
"response_bytes": 7190,
|
||||||
|
"response_limit_bytes": 200000,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
},
|
||||||
|
"task_context_maximum_page": {
|
||||||
|
"counter_ranges": {
|
||||||
|
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_builds": {"maximum": 0, "minimum": 0},
|
||||||
|
"index_checks": {"maximum": 1, "minimum": 1},
|
||||||
|
"index_synchronizations": {"maximum": 0, "minimum": 0},
|
||||||
|
"project_loads": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
|
||||||
|
"render_prepare_calls": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_files_parsed": {"maximum": 0, "minimum": 0},
|
||||||
|
"source_generation_checks": {"maximum": 2, "minimum": 2},
|
||||||
|
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
|
||||||
|
},
|
||||||
|
"diagnostics_dropped_for_budget": false,
|
||||||
|
"max_ms": 87.372,
|
||||||
|
"maximum_response_bytes": 7192,
|
||||||
|
"median_ms": 84.933,
|
||||||
|
"min_ms": 83.652,
|
||||||
|
"p95_limit_ms": 500,
|
||||||
|
"p95_ms": 87.372,
|
||||||
|
"response_bytes": 7192,
|
||||||
|
"response_limit_bytes": 200000,
|
||||||
|
"samples": 10,
|
||||||
|
"validated_invocations": 11
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"process_peak_rss_kib": 86072,
|
||||||
|
"schema_version": 1,
|
||||||
|
"source": {
|
||||||
|
"dirty": false,
|
||||||
|
"revision": "fb0df5e4a1c591c2a84788fd4814d98550f11863"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -60,6 +60,17 @@ MCP results retain:
|
||||||
The result schema describes the common envelope. Operation-specific fields are additive and remain
|
The result schema describes the common envelope. Operation-specific fields are additive and remain
|
||||||
bounded by the configured tool-output limit.
|
bounded by the configured tool-output limit.
|
||||||
|
|
||||||
|
The following Milestone 2 CLI additions do not change existing command signatures:
|
||||||
|
|
||||||
|
- `docforge configure codex|claude|openclaw --project ROOT`
|
||||||
|
- `docforge doctor --client codex|claude|openclaw`
|
||||||
|
|
||||||
|
Configuration output is a new version-1 machine-local contract. It preserves the `docforge`
|
||||||
|
package and executable names and emits the existing `docforge.mcp_server` module entrypoint.
|
||||||
|
Existing hand-written client configurations remain valid and are never rewritten automatically.
|
||||||
|
Doctor is inspection-only and does not become a hidden bootstrap, synchronization, or migration
|
||||||
|
path.
|
||||||
|
|
||||||
## Versioned data contracts
|
## Versioned data contracts
|
||||||
|
|
||||||
Milestone 0 preserves:
|
Milestone 0 preserves:
|
||||||
|
|
|
||||||
|
|
@ -22,6 +22,8 @@ commit when Git is available; it cannot change repository state.
|
||||||
- Task context capsule: `schemas/context-capsule.schema.json`, version 1.
|
- Task context capsule: `schemas/context-capsule.schema.json`, version 1.
|
||||||
- Latest generation diff: `schemas/generation-diff.schema.json`, version 1.
|
- Latest generation diff: `schemas/generation-diff.schema.json`, version 1.
|
||||||
- Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1.
|
- Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1.
|
||||||
|
- Generated client configuration: `schemas/client-configuration.schema.json`, version 1.
|
||||||
|
- Client doctor result: `schemas/doctor-result.schema.json`, version 1.
|
||||||
- Index schema: version 3, disposable and reproducible.
|
- Index schema: version 3, disposable and reproducible.
|
||||||
- Index attestation: schema version 1, disposable and reproducible.
|
- Index attestation: schema version 1, disposable and reproducible.
|
||||||
- Core, CLI, and MCP server: version 1.3.0.dev0.
|
- Core, CLI, and MCP server: version 1.3.0.dev0.
|
||||||
|
|
@ -75,6 +77,25 @@ not history and contains no Logic details or source text. Public pages carry one
|
||||||
`receipt_header`; its `stored_receipt_hash` identifies the complete persisted receipt rather than
|
`receipt_header`; its `stored_receipt_hash` identifies the complete persisted receipt rather than
|
||||||
the header alone. One top-level pagination object carries the only continuation cursor.
|
the header alone. One top-level pagination object carries the only continuation cursor.
|
||||||
|
|
||||||
|
## Machine-local client integration
|
||||||
|
|
||||||
|
Generated Codex, Claude, and OpenClaw fragments are machine-local projections. They are not
|
||||||
|
canonical project content. Version 1 binds the selected project, exact isolated Python
|
||||||
|
interpreter, canonical argument layout, effective policy, no-AST projection, render policy,
|
||||||
|
timeouts, artifact bytes, and configuration hash.
|
||||||
|
|
||||||
|
Preview is side-effect free. Explicit publication creates only one new private standalone
|
||||||
|
fragment in an existing real directory. It never merges or replaces different content. Descriptor,
|
||||||
|
parent, target, content, ownership, permission, and link identities are checked before and after
|
||||||
|
the directory durability boundary. A failure rolls back when that can be proven and otherwise
|
||||||
|
returns bounded unconfirmed publication evidence.
|
||||||
|
|
||||||
|
Doctor is a bounded read-only inspector with one fixed check inventory. It uses stable no-follow
|
||||||
|
descriptor and configuration reads plus stat-only derived-index evidence. It never loads a
|
||||||
|
complete projection, opens SQLite, starts MCP, executes the configured command, synchronizes,
|
||||||
|
builds, renders, starts a viewer, or writes configuration. Unprovable client behavior is a warning,
|
||||||
|
not an invented success.
|
||||||
|
|
||||||
## Isolated proposal model
|
## Isolated proposal model
|
||||||
|
|
||||||
Create, update, move, and delete are ordered node operations inside an isolated changeset. Every
|
Create, update, move, and delete are ordered node operations inside an isolated changeset. Every
|
||||||
|
|
|
||||||
85
docs/MILESTONE_2_BASELINE.md
Normal file
85
docs/MILESTONE_2_BASELINE.md
Normal file
|
|
@ -0,0 +1,85 @@
|
||||||
|
# Milestone 2 baseline
|
||||||
|
|
||||||
|
## Scope and method
|
||||||
|
|
||||||
|
This baseline records the agent-retrieval and client-integration behavior added in Milestone 2.
|
||||||
|
It was captured on 2026-07-29 from clean candidate commit
|
||||||
|
`fb0df5e4a1c591c2a84788fd4814d98550f11863`.
|
||||||
|
|
||||||
|
The maintained command was:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/python tools/milestone2_benchmark.py \
|
||||||
|
--nodes 1000 \
|
||||||
|
--samples 10 \
|
||||||
|
--output /tmp/docforge-milestone2-final.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The fixture contains 1,000 Markdown nodes and 999 edges in a direct fan-in around one focus node.
|
||||||
|
The configured MCP response limit is 200,000 characters. Durations use
|
||||||
|
`time.perf_counter_ns()` and nearest-rank p95. Peak memory uses an isolated child process and
|
||||||
|
`RUSAGE_SELF`. Every warmup and measured invocation is validated.
|
||||||
|
|
||||||
|
Environment:
|
||||||
|
|
||||||
|
- Linux 7.1.3-200.nobara.fc44.x86_64.
|
||||||
|
- CPython 3.14.6.
|
||||||
|
- x86_64.
|
||||||
|
- Ten warm samples after one warmup.
|
||||||
|
- Isolated memory ceiling: 262,144 KiB.
|
||||||
|
|
||||||
|
The complete machine-readable result is
|
||||||
|
[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json).
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
| Operation | Median | p95 | Limit | Maximum response |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| Read bootstrap | 9.406 ms | 9.884 ms | 100 ms | 7,495 B |
|
||||||
|
| No-AST bootstrap | 9.153 ms | 9.379 ms | 100 ms | 8,796 B |
|
||||||
|
| Task diagnostic page | 73.326 ms | 93.115 ms | 500 ms | 7,190 B |
|
||||||
|
| Task complete traversal | 703.561 ms | 721.847 ms | 2,500 ms | 151,172 B/page |
|
||||||
|
| Generation diagnostic page | 40.961 ms | 41.650 ms | 100 ms | 66,516 B |
|
||||||
|
| Generation maximum page | 55.565 ms | 59.184 ms | 100 ms | 199,566 B |
|
||||||
|
| Generation complete traversal | 418.607 ms | 425.315 ms | 500 ms | 66,516 B/page |
|
||||||
|
| Codex configuration preview | 314.365 ms | 364.383 ms | 500 ms | 2,627 B |
|
||||||
|
| Claude configuration preview | 314.326 ms | 364.365 ms | 500 ms | 2,748 B |
|
||||||
|
| OpenClaw configuration preview | 314.401 ms | 364.532 ms | 500 ms | 2,869 B |
|
||||||
|
| Codex doctor | 0.421 ms | 0.556 ms | 100 ms | 3,595 B |
|
||||||
|
| Claude doctor | 0.364 ms | 0.446 ms | 100 ms | 3,669 B |
|
||||||
|
| OpenClaw doctor | 0.384 ms | 0.484 ms | 100 ms | 3,602 B |
|
||||||
|
|
||||||
|
Isolated peak RSS was 86,448 KiB.
|
||||||
|
|
||||||
|
Task traversal returned 108 evidence records and 892 explicit omissions across 11 pages. One
|
||||||
|
individually oversized focus record became a response-limit surrogate bound to the original record
|
||||||
|
hash. The remaining omissions were token-budget evidence. The benchmark verified every unique
|
||||||
|
subject, reconstructed the original collection hash, and matched the exact 1,000-node fixture.
|
||||||
|
|
||||||
|
Generation traversal returned all 1,000 changed-node details across 10 pages. It reconstructed the
|
||||||
|
stored retained-collection hash. The maximum generation page approached the response limit and
|
||||||
|
proved that optional diagnostics were dropped before the primary result.
|
||||||
|
|
||||||
|
## Structured-work gates
|
||||||
|
|
||||||
|
Configuration preview and doctor performed zero project loads, source parses, adapter projection
|
||||||
|
loads, adapter extraction, index checks, synchronization, index builds, render preparation,
|
||||||
|
rendered-byte construction or hashing, and viewer-manager requests.
|
||||||
|
|
||||||
|
Task-context pages performed exactly one index check and two cheap source-generation checks. They
|
||||||
|
performed none of the hidden work above. Generation-diff pages performed exactly two cheap
|
||||||
|
source-generation checks and no index check or hidden work.
|
||||||
|
|
||||||
|
## Measured limits and future notes
|
||||||
|
|
||||||
|
- Continuation is stateless and regenerates the task capsule for each page. The complete
|
||||||
|
11-page traversal remains within its gate, but later work can avoid repeated planning without
|
||||||
|
weakening generation binding.
|
||||||
|
- Configuration preview deliberately spends about 314 ms proving that the exact isolated
|
||||||
|
interpreter can import the MCP module. Discovery-only checks were rejected as unsafe.
|
||||||
|
- Claude configuration syntax is supported, but its timeout representation remains unverified.
|
||||||
|
Doctor therefore reports degraded rather than healthy.
|
||||||
|
- Doctor is a configuration inspector, not an MCP connection or SQLite integrity test.
|
||||||
|
- Legacy adapters without cheap source-generation identity report unknown for generation-diff
|
||||||
|
freshness.
|
||||||
|
- The results do not justify a storage rewrite. SQLite remains fast after one generation is pinned.
|
||||||
68
docs/MILESTONE_2_CLOSEOUT.md
Normal file
68
docs/MILESTONE_2_CLOSEOUT.md
Normal file
|
|
@ -0,0 +1,68 @@
|
||||||
|
# Milestone 2 closeout
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Milestone 2 is complete. One project-bound server can expose an explicit effective policy and
|
||||||
|
return compact, task-shaped, explainable context. Users can generate deterministic client
|
||||||
|
fragments and inspect their bindings without hidden runtime work.
|
||||||
|
|
||||||
|
Implemented contracts:
|
||||||
|
|
||||||
|
- Version-1 effective policy and capability-aware bootstrap.
|
||||||
|
- Version-1 retrieval plans and context capsules.
|
||||||
|
- Bounded task-context pagination with evidence gaps and explicit omissions.
|
||||||
|
- One disposable latest-generation transition receipt and paged read surface.
|
||||||
|
- Deterministic Codex, Claude, and OpenClaw standalone configuration fragments.
|
||||||
|
- Fixed-inventory read-only doctor results.
|
||||||
|
- Dedicated configuration and doctor JSON schemas.
|
||||||
|
- Repository-native Milestone 2 contract, smoke, scale, response-size, counter, and memory gates.
|
||||||
|
|
||||||
|
## Candidate evidence
|
||||||
|
|
||||||
|
The frozen implementation candidate is
|
||||||
|
`fb0df5e4a1c591c2a84788fd4814d98550f11863`.
|
||||||
|
|
||||||
|
The complete repository gate passed:
|
||||||
|
|
||||||
|
- Ruff formatting and lint.
|
||||||
|
- HTML, rendered-manual HTML, CSS, and JavaScript checks.
|
||||||
|
- Pyright with zero diagnostics.
|
||||||
|
- Warning-strict compilation and tests.
|
||||||
|
- 205 tests and 120 subtests.
|
||||||
|
- Lock and npm dependency-tree checks.
|
||||||
|
- Wheel and source-distribution builds.
|
||||||
|
- Milestone 0, 1, and 2 smoke benchmarks.
|
||||||
|
|
||||||
|
Three independent read-only adversarial audits covered client publication and policy binding,
|
||||||
|
doctor race and malformed-input behavior, and benchmark/contract evidence. Reproduced descriptor,
|
||||||
|
parent, target, filesystem, policy, secret-redaction, ambiguity, parser, response-size, and hidden
|
||||||
|
work defects were fixed and regression-tested before the candidate was frozen.
|
||||||
|
|
||||||
|
The clean ten-sample 1,000-node benchmark passed every threshold. Exact measurements and counter
|
||||||
|
ranges are recorded in
|
||||||
|
[`MILESTONE_2_BASELINE.md`](MILESTONE_2_BASELINE.md) and
|
||||||
|
[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json).
|
||||||
|
|
||||||
|
## Preserved boundaries
|
||||||
|
|
||||||
|
- The `docforge` package, imports, CLI executable, MCP executable, and existing tool names remain.
|
||||||
|
- Legacy one-method `load_projection()` adapters remain supported.
|
||||||
|
- The no-AST shorthand and legacy adapter-policy payload remain compatible.
|
||||||
|
- Project descriptor schema version 1 remains unchanged.
|
||||||
|
- No storage replacement was introduced.
|
||||||
|
- No legacy DocForge MCP or DocForge2 self-hosting was used.
|
||||||
|
- WorldForge and ScrapeStation were not touched.
|
||||||
|
- No production MCP integration was repointed.
|
||||||
|
- The legacy Forgejo repository and `legacy` remote were not changed.
|
||||||
|
- No tag, release, release announcement, or visibility change was created.
|
||||||
|
|
||||||
|
## Known follow-up work
|
||||||
|
|
||||||
|
The next active milestone may improve projection independence. It must not silently absorb these
|
||||||
|
separate future ideas:
|
||||||
|
|
||||||
|
- Avoid recomputing a complete task capsule for every continuation page.
|
||||||
|
- Add authenticated cursors only if a stronger threat model requires them.
|
||||||
|
- Verify Claude's native timeout representation.
|
||||||
|
- Add versioned adapter-owned launcher metadata before generating custom-adapter configurations.
|
||||||
|
- Keep doctor read-only; a live connection test must be an explicit separate operation.
|
||||||
|
|
@ -508,6 +508,67 @@ visualization-status
|
||||||
visualization-stop
|
visualization-stop
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Client configuration and doctor
|
||||||
|
|
||||||
|
Preview one deterministic standalone client fragment:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docforge configure codex --project /absolute/path/MyProject
|
||||||
|
docforge configure claude --project /absolute/path/MyProject
|
||||||
|
docforge configure openclaw --project /absolute/path/MyProject
|
||||||
|
```
|
||||||
|
|
||||||
|
Preview is the default. Add `--output /absolute/path/fragment` to create a new private fragment in
|
||||||
|
an existing real directory. Publication is create-only. DocForge accepts an identical existing
|
||||||
|
private single-link file as unchanged, but it never merges, replaces, broadens permissions, or
|
||||||
|
follows a symlink. Descriptor, parent, and target identities are revalidated across the
|
||||||
|
publication commit.
|
||||||
|
|
||||||
|
The generated command uses the exact current Python interpreter with isolated module startup.
|
||||||
|
Generation first proves that this interpreter can import `docforge.mcp_server`. The result binds
|
||||||
|
the project root, effective policy, arguments, artifact bytes, and all hashes. It copies no ambient
|
||||||
|
environment values.
|
||||||
|
|
||||||
|
Select authority explicitly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docforge configure codex \
|
||||||
|
--project /absolute/path/MyProject \
|
||||||
|
--capability-mode proposal \
|
||||||
|
--proposal-writer project-editor
|
||||||
|
|
||||||
|
docforge configure codex \
|
||||||
|
--project /absolute/path/MyProject \
|
||||||
|
--capability-mode application \
|
||||||
|
--proposal-writer project-editor \
|
||||||
|
--canonical-applier project-editor
|
||||||
|
```
|
||||||
|
|
||||||
|
Read mode is the default. Proposal and application modes fail closed unless the descriptor
|
||||||
|
declares the named writer, and application requires the same writer/applier identity. Add
|
||||||
|
`--no-ast` to preserve the no-AST binding. Generic CLI generation refuses project-owned adapters
|
||||||
|
because it cannot safely reconstruct their composition.
|
||||||
|
|
||||||
|
Inspect one configured client binding:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docforge doctor --client codex --project /absolute/path/MyProject
|
||||||
|
docforge doctor --client codex \
|
||||||
|
--project /absolute/path/MyProject \
|
||||||
|
--config /absolute/path/config.toml \
|
||||||
|
--server-name my-project-docforge
|
||||||
|
```
|
||||||
|
|
||||||
|
Doctor returns `healthy`, `degraded`, or `unhealthy` with exit codes 0, 1, or 2. Its fixed
|
||||||
|
version-1 inventory checks project and descriptor binding, the client driver and entry, executable
|
||||||
|
and arguments, project root, effective policy, no-AST state, timeouts, environment-key names,
|
||||||
|
tool-filter representation, and stat-only index presence.
|
||||||
|
|
||||||
|
Doctor is intentionally not a connection test. It never loads canonical sources, opens SQLite,
|
||||||
|
starts MCP, executes the configured command, synchronizes, builds, renders, starts a viewer, or
|
||||||
|
writes configuration. Claude timeout representation and client filtering that cannot be proved
|
||||||
|
locally remain explicit warnings.
|
||||||
|
|
||||||
## MCP usage
|
## MCP usage
|
||||||
|
|
||||||
Run one MCP server per project with absolute paths:
|
Run one MCP server per project with absolute paths:
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue