From 4c5773c8652aedfeab82cf87ee7da1b7bc01eddc Mon Sep 17 00:00:00 2001 From: Andraxion Date: Wed, 29 Jul 2026 10:22:56 -0400 Subject: [PATCH] Close Milestone 2 with measured evidence --- ACTIVE_SLICE.md | 16 +- DEVELOPMENT_NOTES.md | 27 ++ README.md | 20 +- benchmarks/README.md | 14 + benchmarks/milestone2-2026-07-29.json | 457 ++++++++++++++++++++++++++ docs/COMPATIBILITY.md | 11 + docs/CONTRACT.md | 21 ++ docs/MILESTONE_2_BASELINE.md | 85 +++++ docs/MILESTONE_2_CLOSEOUT.md | 68 ++++ docs/USER_MANUAL.md | 61 ++++ 10 files changed, 771 insertions(+), 9 deletions(-) create mode 100644 benchmarks/milestone2-2026-07-29.json create mode 100644 docs/MILESTONE_2_BASELINE.md create mode 100644 docs/MILESTONE_2_CLOSEOUT.md diff --git a/ACTIVE_SLICE.md b/ACTIVE_SLICE.md index 97be32f..71c0e5e 100644 --- a/ACTIVE_SLICE.md +++ b/ACTIVE_SLICE.md @@ -6,14 +6,14 @@ Goal: Let one project-bound server return compact, task-shaped, explainable cont In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks. Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases. Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass. -Status: Candidate frozen. Effective policy, versioned task retrieval, latest-generation diff -receipts, and logarithmic task-context page packing are committed and pushed on `dev`. -Deterministic client configuration and the read-only integration doctor now pass their bounded -publication, path-race, malformed-input, redaction, and no-hidden-work audits. The complete -repository gate passes with 205 tests and 120 subtests. A disposable 1,000-node audit sample passes -the maintained task-context, generation-diff, response-size, counter, and memory gates. Final -clean-revision benchmark evidence and documentation closeout remain before the milestone is marked -complete. +Status: Complete. Effective policy, versioned task retrieval, latest-generation diff receipts, +logarithmic bounded page packing, deterministic client configuration, and the read-only integration +doctor are implemented and contract-tested. The complete repository gate passes with 205 tests and +120 subtests. Three independent adversarial audits found no remaining implementation blocker. The +clean 1,000-node baseline is recorded against candidate commit +`fb0df5e4a1c591c2a84788fd4814d98550f11863`, including task/generation reconstruction, +response-size behavior, zero-hidden-work counters, and isolated memory. No tag or release was +created, no production integration was repointed, and self-hosting remains out of scope. ``` Milestones 3–5 remain directional context and are not active. diff --git a/DEVELOPMENT_NOTES.md b/DEVELOPMENT_NOTES.md index 536100e..8e15aea 100644 --- a/DEVELOPMENT_NOTES.md +++ b/DEVELOPMENT_NOTES.md @@ -583,3 +583,30 @@ compilation, lock and dependency checks, package builds, and all three milestone Three independent final audits approve client publication and policy binding, doctor fail-closed behavior, and benchmark/contract coverage. Clean-revision benchmark evidence is still required before closeout. + +### Milestone 2 closeout + +Candidate commit `fb0df5e4a1c591c2a84788fd4814d98550f11863` passed the clean ten-sample +Milestone 2 benchmark. Task-context complete traversal measured 703.561 ms median and 721.847 ms +p95 across 11 bounded pages. It reconstructed the exact 1,000-candidate collection from 108 cited +evidence records, 891 original token-budget omissions, and one hash-attested response-limit +surrogate. Generation-diff complete traversal measured 418.607 ms median and 425.315 ms p95 across +10 pages. + +Read and no-AST bootstrap remained below 10 ms p95. The maximum generation page used 199,566 bytes +of the 200,000-byte budget and correctly discarded diagnostics before primary evidence. +Configuration preview measured about 314 ms median and 365 ms p95 because it proves the real +isolated interpreter import on every invocation. Codex and OpenClaw doctor checks remained below +0.6 ms p95; Claude remained explicitly degraded because its timeout format is unverified. +Isolated-process peak RSS was 86,448 KiB against the 262,144 KiB gate. + +All measured configuration and doctor counters were zero. Task-context pages performed one index +check and two cheap generation checks with no loads, parses, synchronization, builds, extraction, +rendering, or viewer work. Generation-diff pages performed two cheap generation checks and no +index check. The canonical machine-readable result is +`benchmarks/milestone2-2026-07-29.json`. + +Milestone 2 is complete. Follow-up ideas stay explicitly later-scope: avoid recomputing the +task-shaped capsule for every continuation page, add authenticated continuation when the threat +model requires it, verify a native Claude timeout representation, and introduce adapter-owned +launcher metadata before generating configurations for custom adapters. diff --git a/README.md b/README.md index d08b2ab..16a03a6 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,10 @@ declared manuals, visualizes project structure, and manages reviewable documenta continuation. - Records one bounded, versioned latest-generation graph transition without creating a history database. +- Generates deterministic project-bound Codex, Claude, and OpenClaw client fragments without + copying ambient secrets. +- Diagnoses one client binding through bounded read-only checks without starting MCP or rebuilding + project state. - Automatically synchronizes disposable indexes before MCP work. - Creates, validates, diffs, and previews isolated changesets. - Registers complete proposals atomically without caller-managed hash chaining. @@ -135,6 +139,16 @@ Start an MCP server for one project: Add `--canonical-applier project-editor` only when that MCP integration should expose the hash-bound `docforge_apply_changeset` tool. +Preview a read-only Codex fragment and diagnose an installed binding: + +```bash +.venv/bin/docforge configure codex --project "$PROJECT" +.venv/bin/docforge doctor --client codex --project "$PROJECT" +``` + +Pass `--output /absolute/path/docforge.toml` only when creating a standalone fragment. DocForge +never replaces or merges an existing different client file. + For an unconfigured codebase, begin with a read-only language and documentation assessment: ```bash @@ -156,6 +170,10 @@ DocForge describes them as a source graph. performance, memory, rendering and response sizes, bottlenecks, and missing coverage. - [Milestone 0 closeout](docs/MILESTONE_0_CLOSEOUT.md) — lineage, migration, security scan, repository state, and fresh-clone proof. +- [Milestone 2 baseline](docs/MILESTONE_2_BASELINE.md) — task context, generation diff, client + configuration, doctor, response-size, counter, and memory measurements. +- [Milestone 2 closeout](docs/MILESTONE_2_CLOSEOUT.md) — implemented contracts, adversarial + validation, exclusions, and exact candidate evidence. - [MCP contract](docs/MCP_CONTRACT.md) — exact tool and process boundary. - [Viewer manager](docs/VIEWER_MANAGER.md) — native service setup and lifecycle. - [Adapter decision](docs/APPLICATION_DECISION.md) — why custom adapters own canonical @@ -178,7 +196,7 @@ make gate Focused entry points are available as `make contract`, `make test`, `make type`, `make benchmark-smoke`, `make benchmark`, `make benchmark-m1-smoke`, and -`make benchmark-m1`. +`make benchmark-m1`. Milestone 2 adds `make benchmark-m2-smoke` and `make benchmark-m2`. The committed 1,000-node baseline and its measurement method are under `benchmarks/`. diff --git a/benchmarks/README.md b/benchmarks/README.md index 05f5c55..9d78eda 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -27,6 +27,13 @@ Run the maintained 1,000-node Milestone 1 benchmark: make benchmark-m1 ``` +Run the Milestone 2 agent-workflow smoke and full gates: + +```bash +make benchmark-m2-smoke +make benchmark-m2 +``` + The benchmark creates canonical sources, derived state, changesets, rendered output, and caches only in a disposable temporary directory. It does not read another project, self-host DocForge, or mutate repository content. @@ -42,6 +49,13 @@ harness enforces operation-specific p95 ceilings and fixed zero-work counter inv human-readable interpretation is in [`docs/MILESTONE_1_BASELINE.md`](../docs/MILESTONE_1_BASELINE.md). +`milestone2-2026-07-29.json` is the clean-tree agent-retrieval and client-integration baseline +captured from commit `fb0df5e4a1c591c2a84788fd4814d98550f11863`. It gates every warmup and +sample, reconstructs complete task-context and generation-diff collections across bounded pages, +records whether diagnostics were dropped for response budget, checks all hidden-work counters, +and measures isolated-process peak RSS. Its interpretation is in +[`docs/MILESTONE_2_BASELINE.md`](../docs/MILESTONE_2_BASELINE.md). + The generic fixture exposes whole-source scaling. It does not replace the incremental adapter equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has no portable graph-planning or graph-rendering contract. diff --git a/benchmarks/milestone2-2026-07-29.json b/benchmarks/milestone2-2026-07-29.json new file mode 100644 index 0000000..f4196ac --- /dev/null +++ b/benchmarks/milestone2-2026-07-29.json @@ -0,0 +1,457 @@ +{ + "benchmark": "docforge2_milestone2", + "environment": { + "implementation": "CPython", + "machine": "x86_64", + "platform": "Linux-7.1.3-200.nobara.fc44.x86_64-x86_64-with-glibc2.43", + "python": "3.14.6" + }, + "fixture": { + "edge_count": 999, + "kind": "synthetic_generic_focus_fan_in", + "max_tool_output_chars": 200000, + "node_count": 1000, + "source_file_count": 1000 + }, + "isolated_process_peak_rss_kib": 86448, + "method": { + "clock": "time.perf_counter_ns", + "memory": "isolated child-process resource.getrusage(RUSAGE_SELF).ru_maxrss", + "memory_limit_kib": 262144, + "memory_probe_samples": 10, + "percentile": "nearest-rank", + "response_size": "UTF-8 bytes of compact sorted JSON", + "samples": 10, + "warmups": 1, + "zero_work_counters": [ + "project_loads", + "source_files_parsed", + "source_bytes_parsed", + "adapter_projection_loads", + "adapter_source_extractions", + "index_synchronizations", + "index_builds", + "render_prepare_calls", + "render_output_bytes_built", + "render_output_bytes_hashed", + "viewer_manager_requests" + ] + }, + "operations": { + "bootstrap_no_ast": { + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 1, "minimum": 1}, + "index_synchronizations": {"maximum": 1, "minimum": 1}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 1, "minimum": 1}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "max_ms": 9.379, + "maximum_response_bytes": 8796, + "median_ms": 9.153, + "min_ms": 9.026, + "p95_limit_ms": 100, + "p95_ms": 9.379, + "response_bytes": 8795, + "response_limit_bytes": 32768, + "samples": 10, + "validated_invocations": 11 + }, + "bootstrap_read": { + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 1, "minimum": 1}, + "index_synchronizations": {"maximum": 1, "minimum": 1}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 1, "minimum": 1}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "max_ms": 9.884, + "maximum_response_bytes": 7495, + "median_ms": 9.406, + "min_ms": 9.053, + "p95_limit_ms": 100, + "p95_ms": 9.884, + "response_bytes": 7491, + "response_limit_bytes": 32768, + "samples": 10, + "validated_invocations": 11 + }, + "configuration_preview": { + "claude": { + "artifact_format": "claude-json-fragment-v1", + "configuration_hash": "4dbb4ed0f38264fdba350de8904cc898493d620194b5105b7889c54bd5913c9c", + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 0, "minimum": 0}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 0, "minimum": 0}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "max_ms": 364.365, + "maximum_response_bytes": 2748, + "median_ms": 314.326, + "min_ms": 314.278, + "p95_limit_ms": 500, + "p95_ms": 364.365, + "response_bytes": 2748, + "response_limit_bytes": 32768, + "samples": 10, + "validated_invocations": 11 + }, + "codex": { + "artifact_format": "codex-toml-fragment-v1", + "configuration_hash": "3e1dd5191021da1778cd1c4f4658768537775e5e16252e42d4c80e328841145b", + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 0, "minimum": 0}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 0, "minimum": 0}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "max_ms": 364.383, + "maximum_response_bytes": 2627, + "median_ms": 314.365, + "min_ms": 314.248, + "p95_limit_ms": 500, + "p95_ms": 364.383, + "response_bytes": 2627, + "response_limit_bytes": 32768, + "samples": 10, + "validated_invocations": 11 + }, + "openclaw": { + "artifact_format": "openclaw-json-fragment-v1", + "configuration_hash": "6f269e90a55088c5d517f91761c53a3b90036d62fd80b5b9d094668267257b99", + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 0, "minimum": 0}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 0, "minimum": 0}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "max_ms": 364.532, + "maximum_response_bytes": 2869, + "median_ms": 314.401, + "min_ms": 314.251, + "p95_limit_ms": 500, + "p95_ms": 364.532, + "response_bytes": 2869, + "response_limit_bytes": 32768, + "samples": 10, + "validated_invocations": 11 + } + }, + "doctor": { + "claude": { + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 0, "minimum": 0}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 0, "minimum": 0}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "doctor_state": "degraded", + "max_ms": 0.446, + "maximum_response_bytes": 3669, + "median_ms": 0.364, + "min_ms": 0.352, + "p95_limit_ms": 100, + "p95_ms": 0.446, + "response_bytes": 3669, + "response_limit_bytes": 32768, + "samples": 10, + "summary": {"failed": 0, "passed": 11, "skipped": 1, "warning": 2}, + "validated_invocations": 11 + }, + "codex": { + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 0, "minimum": 0}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 0, "minimum": 0}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "doctor_state": "healthy", + "max_ms": 0.556, + "maximum_response_bytes": 3595, + "median_ms": 0.421, + "min_ms": 0.404, + "p95_limit_ms": 100, + "p95_ms": 0.556, + "response_bytes": 3595, + "response_limit_bytes": 32768, + "samples": 10, + "summary": {"failed": 0, "passed": 13, "skipped": 1, "warning": 0}, + "validated_invocations": 11 + }, + "openclaw": { + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 0, "minimum": 0}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 0, "minimum": 0}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "doctor_state": "healthy", + "max_ms": 0.484, + "maximum_response_bytes": 3602, + "median_ms": 0.384, + "min_ms": 0.353, + "p95_limit_ms": 100, + "p95_ms": 0.484, + "response_bytes": 3602, + "response_limit_bytes": 32768, + "samples": 10, + "summary": {"failed": 0, "passed": 13, "skipped": 1, "warning": 0}, + "validated_invocations": 11 + } + }, + "generation_diff_complete": { + "max_ms": 425.315, + "maximum_response_bytes": 984, + "median_ms": 418.607, + "min_ms": 410.974, + "p95_limit_ms": 500, + "p95_ms": 425.315, + "response_bytes": 983, + "response_limit_bytes": 32768, + "result_summary": { + "aggregate_page_bytes": 664715, + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 0, "minimum": 0}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 2, "minimum": 2}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "elapsed_ms": 419.04, + "item_count": 1000, + "maximum_cursor_bytes": 448, + "maximum_page_bytes": 66516, + "ordered_item_hash": "1ac48cc72532809ef5d3e949756e536eec819f348eaf06338c9b39b14e63b2c7", + "page_count": 10, + "receipt_hash": "6913c962972d8255f56966e5cfab5ac8293e41bdfb5f39ef91a4f33a3b092f88", + "status": "ok" + }, + "samples": 10, + "validated_invocations": 11 + }, + "generation_diff_diagnostic_page": { + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 0, "minimum": 0}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 2, "minimum": 2}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "max_ms": 41.65, + "maximum_response_bytes": 66516, + "median_ms": 40.961, + "min_ms": 40.239, + "p95_limit_ms": 100, + "p95_ms": 41.65, + "response_bytes": 66516, + "response_limit_bytes": 200000, + "samples": 10, + "validated_invocations": 11 + }, + "generation_diff_maximum_page": { + "diagnostics_dropped_for_budget": true, + "max_ms": 59.184, + "maximum_response_bytes": 199566, + "median_ms": 55.565, + "min_ms": 54.68, + "p95_limit_ms": 100, + "p95_ms": 59.184, + "response_bytes": 199566, + "response_limit_bytes": 200000, + "samples": 10, + "validated_invocations": 11 + }, + "task_context_complete": { + "max_ms": 721.847, + "maximum_response_bytes": 1325, + "median_ms": 703.561, + "min_ms": 688.975, + "p95_limit_ms": 2500, + "p95_ms": 721.847, + "response_bytes": 1324, + "response_limit_bytes": 32768, + "result_summary": { + "aggregate_page_bytes": 348845, + "capsule_hash": "20663ed685a255f7cb8a0e8d78262bf7bf26863d728f0159e0b29000f6a52b0a", + "collection_hash": "9dbc46eb5b1ac8c8340bb149205d3599fffaf67e3b363de2d035475da274857c", + "collection_hash_reconstructed": true, + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 1, "minimum": 1}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 2, "minimum": 2}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "elapsed_ms": 719.55, + "evidence_count": 108, + "item_count": 1000, + "maximum_cursor_bytes": 1066, + "maximum_page_bytes": 151172, + "omission_count": 892, + "ordered_candidate_hash": "bdeb3a8f4018000f72a5ff1891aa800b6edf98070b9814f443c6bac4e52c38f3", + "ordered_evidence_hash": "ec07bac7f528f5a3afbc083ea5ad60541335c8789d42980fe7bae0476e0a5331", + "page_count": 11, + "plan_hash": "84dbe60267e5f8359adcf30d4d395f1ac5d5bea957beaa3888d2938120860a9c", + "status": "ok" + }, + "samples": 10, + "validated_invocations": 11 + }, + "task_context_diagnostic_page": { + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 1, "minimum": 1}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 2, "minimum": 2}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "max_ms": 93.115, + "maximum_response_bytes": 7190, + "median_ms": 73.326, + "min_ms": 72.264, + "p95_limit_ms": 500, + "p95_ms": 93.115, + "response_bytes": 7190, + "response_limit_bytes": 200000, + "samples": 10, + "validated_invocations": 11 + }, + "task_context_maximum_page": { + "counter_ranges": { + "adapter_projection_loads": {"maximum": 0, "minimum": 0}, + "adapter_source_extractions": {"maximum": 0, "minimum": 0}, + "index_builds": {"maximum": 0, "minimum": 0}, + "index_checks": {"maximum": 1, "minimum": 1}, + "index_synchronizations": {"maximum": 0, "minimum": 0}, + "project_loads": {"maximum": 0, "minimum": 0}, + "render_output_bytes_built": {"maximum": 0, "minimum": 0}, + "render_output_bytes_hashed": {"maximum": 0, "minimum": 0}, + "render_prepare_calls": {"maximum": 0, "minimum": 0}, + "source_bytes_parsed": {"maximum": 0, "minimum": 0}, + "source_files_parsed": {"maximum": 0, "minimum": 0}, + "source_generation_checks": {"maximum": 2, "minimum": 2}, + "viewer_manager_requests": {"maximum": 0, "minimum": 0} + }, + "diagnostics_dropped_for_budget": false, + "max_ms": 87.372, + "maximum_response_bytes": 7192, + "median_ms": 84.933, + "min_ms": 83.652, + "p95_limit_ms": 500, + "p95_ms": 87.372, + "response_bytes": 7192, + "response_limit_bytes": 200000, + "samples": 10, + "validated_invocations": 11 + } + }, + "process_peak_rss_kib": 86072, + "schema_version": 1, + "source": { + "dirty": false, + "revision": "fb0df5e4a1c591c2a84788fd4814d98550f11863" + } +} diff --git a/docs/COMPATIBILITY.md b/docs/COMPATIBILITY.md index 99053b1..12c6247 100644 --- a/docs/COMPATIBILITY.md +++ b/docs/COMPATIBILITY.md @@ -60,6 +60,17 @@ MCP results retain: The result schema describes the common envelope. Operation-specific fields are additive and remain bounded by the configured tool-output limit. +The following Milestone 2 CLI additions do not change existing command signatures: + +- `docforge configure codex|claude|openclaw --project ROOT` +- `docforge doctor --client codex|claude|openclaw` + +Configuration output is a new version-1 machine-local contract. It preserves the `docforge` +package and executable names and emits the existing `docforge.mcp_server` module entrypoint. +Existing hand-written client configurations remain valid and are never rewritten automatically. +Doctor is inspection-only and does not become a hidden bootstrap, synchronization, or migration +path. + ## Versioned data contracts Milestone 0 preserves: diff --git a/docs/CONTRACT.md b/docs/CONTRACT.md index 832a5af..ec2b062 100644 --- a/docs/CONTRACT.md +++ b/docs/CONTRACT.md @@ -22,6 +22,8 @@ commit when Git is available; it cannot change repository state. - Task context capsule: `schemas/context-capsule.schema.json`, version 1. - Latest generation diff: `schemas/generation-diff.schema.json`, version 1. - Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1. +- Generated client configuration: `schemas/client-configuration.schema.json`, version 1. +- Client doctor result: `schemas/doctor-result.schema.json`, version 1. - Index schema: version 3, disposable and reproducible. - Index attestation: schema version 1, disposable and reproducible. - Core, CLI, and MCP server: version 1.3.0.dev0. @@ -75,6 +77,25 @@ not history and contains no Logic details or source text. Public pages carry one `receipt_header`; its `stored_receipt_hash` identifies the complete persisted receipt rather than the header alone. One top-level pagination object carries the only continuation cursor. +## Machine-local client integration + +Generated Codex, Claude, and OpenClaw fragments are machine-local projections. They are not +canonical project content. Version 1 binds the selected project, exact isolated Python +interpreter, canonical argument layout, effective policy, no-AST projection, render policy, +timeouts, artifact bytes, and configuration hash. + +Preview is side-effect free. Explicit publication creates only one new private standalone +fragment in an existing real directory. It never merges or replaces different content. Descriptor, +parent, target, content, ownership, permission, and link identities are checked before and after +the directory durability boundary. A failure rolls back when that can be proven and otherwise +returns bounded unconfirmed publication evidence. + +Doctor is a bounded read-only inspector with one fixed check inventory. It uses stable no-follow +descriptor and configuration reads plus stat-only derived-index evidence. It never loads a +complete projection, opens SQLite, starts MCP, executes the configured command, synchronizes, +builds, renders, starts a viewer, or writes configuration. Unprovable client behavior is a warning, +not an invented success. + ## Isolated proposal model Create, update, move, and delete are ordered node operations inside an isolated changeset. Every diff --git a/docs/MILESTONE_2_BASELINE.md b/docs/MILESTONE_2_BASELINE.md new file mode 100644 index 0000000..44089f6 --- /dev/null +++ b/docs/MILESTONE_2_BASELINE.md @@ -0,0 +1,85 @@ +# Milestone 2 baseline + +## Scope and method + +This baseline records the agent-retrieval and client-integration behavior added in Milestone 2. +It was captured on 2026-07-29 from clean candidate commit +`fb0df5e4a1c591c2a84788fd4814d98550f11863`. + +The maintained command was: + +```bash +.venv/bin/python tools/milestone2_benchmark.py \ + --nodes 1000 \ + --samples 10 \ + --output /tmp/docforge-milestone2-final.json +``` + +The fixture contains 1,000 Markdown nodes and 999 edges in a direct fan-in around one focus node. +The configured MCP response limit is 200,000 characters. Durations use +`time.perf_counter_ns()` and nearest-rank p95. Peak memory uses an isolated child process and +`RUSAGE_SELF`. Every warmup and measured invocation is validated. + +Environment: + +- Linux 7.1.3-200.nobara.fc44.x86_64. +- CPython 3.14.6. +- x86_64. +- Ten warm samples after one warmup. +- Isolated memory ceiling: 262,144 KiB. + +The complete machine-readable result is +[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json). + +## Results + +| Operation | Median | p95 | Limit | Maximum response | +|---|---:|---:|---:|---:| +| Read bootstrap | 9.406 ms | 9.884 ms | 100 ms | 7,495 B | +| No-AST bootstrap | 9.153 ms | 9.379 ms | 100 ms | 8,796 B | +| Task diagnostic page | 73.326 ms | 93.115 ms | 500 ms | 7,190 B | +| Task complete traversal | 703.561 ms | 721.847 ms | 2,500 ms | 151,172 B/page | +| Generation diagnostic page | 40.961 ms | 41.650 ms | 100 ms | 66,516 B | +| Generation maximum page | 55.565 ms | 59.184 ms | 100 ms | 199,566 B | +| Generation complete traversal | 418.607 ms | 425.315 ms | 500 ms | 66,516 B/page | +| Codex configuration preview | 314.365 ms | 364.383 ms | 500 ms | 2,627 B | +| Claude configuration preview | 314.326 ms | 364.365 ms | 500 ms | 2,748 B | +| OpenClaw configuration preview | 314.401 ms | 364.532 ms | 500 ms | 2,869 B | +| Codex doctor | 0.421 ms | 0.556 ms | 100 ms | 3,595 B | +| Claude doctor | 0.364 ms | 0.446 ms | 100 ms | 3,669 B | +| OpenClaw doctor | 0.384 ms | 0.484 ms | 100 ms | 3,602 B | + +Isolated peak RSS was 86,448 KiB. + +Task traversal returned 108 evidence records and 892 explicit omissions across 11 pages. One +individually oversized focus record became a response-limit surrogate bound to the original record +hash. The remaining omissions were token-budget evidence. The benchmark verified every unique +subject, reconstructed the original collection hash, and matched the exact 1,000-node fixture. + +Generation traversal returned all 1,000 changed-node details across 10 pages. It reconstructed the +stored retained-collection hash. The maximum generation page approached the response limit and +proved that optional diagnostics were dropped before the primary result. + +## Structured-work gates + +Configuration preview and doctor performed zero project loads, source parses, adapter projection +loads, adapter extraction, index checks, synchronization, index builds, render preparation, +rendered-byte construction or hashing, and viewer-manager requests. + +Task-context pages performed exactly one index check and two cheap source-generation checks. They +performed none of the hidden work above. Generation-diff pages performed exactly two cheap +source-generation checks and no index check or hidden work. + +## Measured limits and future notes + +- Continuation is stateless and regenerates the task capsule for each page. The complete + 11-page traversal remains within its gate, but later work can avoid repeated planning without + weakening generation binding. +- Configuration preview deliberately spends about 314 ms proving that the exact isolated + interpreter can import the MCP module. Discovery-only checks were rejected as unsafe. +- Claude configuration syntax is supported, but its timeout representation remains unverified. + Doctor therefore reports degraded rather than healthy. +- Doctor is a configuration inspector, not an MCP connection or SQLite integrity test. +- Legacy adapters without cheap source-generation identity report unknown for generation-diff + freshness. +- The results do not justify a storage rewrite. SQLite remains fast after one generation is pinned. diff --git a/docs/MILESTONE_2_CLOSEOUT.md b/docs/MILESTONE_2_CLOSEOUT.md new file mode 100644 index 0000000..8ff2f8f --- /dev/null +++ b/docs/MILESTONE_2_CLOSEOUT.md @@ -0,0 +1,68 @@ +# Milestone 2 closeout + +## Outcome + +Milestone 2 is complete. One project-bound server can expose an explicit effective policy and +return compact, task-shaped, explainable context. Users can generate deterministic client +fragments and inspect their bindings without hidden runtime work. + +Implemented contracts: + +- Version-1 effective policy and capability-aware bootstrap. +- Version-1 retrieval plans and context capsules. +- Bounded task-context pagination with evidence gaps and explicit omissions. +- One disposable latest-generation transition receipt and paged read surface. +- Deterministic Codex, Claude, and OpenClaw standalone configuration fragments. +- Fixed-inventory read-only doctor results. +- Dedicated configuration and doctor JSON schemas. +- Repository-native Milestone 2 contract, smoke, scale, response-size, counter, and memory gates. + +## Candidate evidence + +The frozen implementation candidate is +`fb0df5e4a1c591c2a84788fd4814d98550f11863`. + +The complete repository gate passed: + +- Ruff formatting and lint. +- HTML, rendered-manual HTML, CSS, and JavaScript checks. +- Pyright with zero diagnostics. +- Warning-strict compilation and tests. +- 205 tests and 120 subtests. +- Lock and npm dependency-tree checks. +- Wheel and source-distribution builds. +- Milestone 0, 1, and 2 smoke benchmarks. + +Three independent read-only adversarial audits covered client publication and policy binding, +doctor race and malformed-input behavior, and benchmark/contract evidence. Reproduced descriptor, +parent, target, filesystem, policy, secret-redaction, ambiguity, parser, response-size, and hidden +work defects were fixed and regression-tested before the candidate was frozen. + +The clean ten-sample 1,000-node benchmark passed every threshold. Exact measurements and counter +ranges are recorded in +[`MILESTONE_2_BASELINE.md`](MILESTONE_2_BASELINE.md) and +[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json). + +## Preserved boundaries + +- The `docforge` package, imports, CLI executable, MCP executable, and existing tool names remain. +- Legacy one-method `load_projection()` adapters remain supported. +- The no-AST shorthand and legacy adapter-policy payload remain compatible. +- Project descriptor schema version 1 remains unchanged. +- No storage replacement was introduced. +- No legacy DocForge MCP or DocForge2 self-hosting was used. +- WorldForge and ScrapeStation were not touched. +- No production MCP integration was repointed. +- The legacy Forgejo repository and `legacy` remote were not changed. +- No tag, release, release announcement, or visibility change was created. + +## Known follow-up work + +The next active milestone may improve projection independence. It must not silently absorb these +separate future ideas: + +- Avoid recomputing a complete task capsule for every continuation page. +- Add authenticated cursors only if a stronger threat model requires them. +- Verify Claude's native timeout representation. +- Add versioned adapter-owned launcher metadata before generating custom-adapter configurations. +- Keep doctor read-only; a live connection test must be an explicit separate operation. diff --git a/docs/USER_MANUAL.md b/docs/USER_MANUAL.md index 48e26b5..a92ed1b 100644 --- a/docs/USER_MANUAL.md +++ b/docs/USER_MANUAL.md @@ -508,6 +508,67 @@ visualization-status visualization-stop ``` +### Client configuration and doctor + +Preview one deterministic standalone client fragment: + +```bash +docforge configure codex --project /absolute/path/MyProject +docforge configure claude --project /absolute/path/MyProject +docforge configure openclaw --project /absolute/path/MyProject +``` + +Preview is the default. Add `--output /absolute/path/fragment` to create a new private fragment in +an existing real directory. Publication is create-only. DocForge accepts an identical existing +private single-link file as unchanged, but it never merges, replaces, broadens permissions, or +follows a symlink. Descriptor, parent, and target identities are revalidated across the +publication commit. + +The generated command uses the exact current Python interpreter with isolated module startup. +Generation first proves that this interpreter can import `docforge.mcp_server`. The result binds +the project root, effective policy, arguments, artifact bytes, and all hashes. It copies no ambient +environment values. + +Select authority explicitly: + +```bash +docforge configure codex \ + --project /absolute/path/MyProject \ + --capability-mode proposal \ + --proposal-writer project-editor + +docforge configure codex \ + --project /absolute/path/MyProject \ + --capability-mode application \ + --proposal-writer project-editor \ + --canonical-applier project-editor +``` + +Read mode is the default. Proposal and application modes fail closed unless the descriptor +declares the named writer, and application requires the same writer/applier identity. Add +`--no-ast` to preserve the no-AST binding. Generic CLI generation refuses project-owned adapters +because it cannot safely reconstruct their composition. + +Inspect one configured client binding: + +```bash +docforge doctor --client codex --project /absolute/path/MyProject +docforge doctor --client codex \ + --project /absolute/path/MyProject \ + --config /absolute/path/config.toml \ + --server-name my-project-docforge +``` + +Doctor returns `healthy`, `degraded`, or `unhealthy` with exit codes 0, 1, or 2. Its fixed +version-1 inventory checks project and descriptor binding, the client driver and entry, executable +and arguments, project root, effective policy, no-AST state, timeouts, environment-key names, +tool-filter representation, and stat-only index presence. + +Doctor is intentionally not a connection test. It never loads canonical sources, opens SQLite, +starts MCP, executes the configured command, synchronizes, builds, renders, starts a viewer, or +writes configuration. Claude timeout representation and client filtering that cannot be proved +locally remain explicit warnings. + ## MCP usage Run one MCP server per project with absolute paths: