Close Milestone 2 with measured evidence
This commit is contained in:
parent
fb0df5e4a1
commit
4c5773c865
10 changed files with 771 additions and 9 deletions
|
|
@ -60,6 +60,17 @@ MCP results retain:
|
|||
The result schema describes the common envelope. Operation-specific fields are additive and remain
|
||||
bounded by the configured tool-output limit.
|
||||
|
||||
The following Milestone 2 CLI additions do not change existing command signatures:
|
||||
|
||||
- `docforge configure codex|claude|openclaw --project ROOT`
|
||||
- `docforge doctor --client codex|claude|openclaw`
|
||||
|
||||
Configuration output is a new version-1 machine-local contract. It preserves the `docforge`
|
||||
package and executable names and emits the existing `docforge.mcp_server` module entrypoint.
|
||||
Existing hand-written client configurations remain valid and are never rewritten automatically.
|
||||
Doctor is inspection-only and does not become a hidden bootstrap, synchronization, or migration
|
||||
path.
|
||||
|
||||
## Versioned data contracts
|
||||
|
||||
Milestone 0 preserves:
|
||||
|
|
|
|||
|
|
@ -22,6 +22,8 @@ commit when Git is available; it cannot change repository state.
|
|||
- Task context capsule: `schemas/context-capsule.schema.json`, version 1.
|
||||
- Latest generation diff: `schemas/generation-diff.schema.json`, version 1.
|
||||
- Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1.
|
||||
- Generated client configuration: `schemas/client-configuration.schema.json`, version 1.
|
||||
- Client doctor result: `schemas/doctor-result.schema.json`, version 1.
|
||||
- Index schema: version 3, disposable and reproducible.
|
||||
- Index attestation: schema version 1, disposable and reproducible.
|
||||
- Core, CLI, and MCP server: version 1.3.0.dev0.
|
||||
|
|
@ -75,6 +77,25 @@ not history and contains no Logic details or source text. Public pages carry one
|
|||
`receipt_header`; its `stored_receipt_hash` identifies the complete persisted receipt rather than
|
||||
the header alone. One top-level pagination object carries the only continuation cursor.
|
||||
|
||||
## Machine-local client integration
|
||||
|
||||
Generated Codex, Claude, and OpenClaw fragments are machine-local projections. They are not
|
||||
canonical project content. Version 1 binds the selected project, exact isolated Python
|
||||
interpreter, canonical argument layout, effective policy, no-AST projection, render policy,
|
||||
timeouts, artifact bytes, and configuration hash.
|
||||
|
||||
Preview is side-effect free. Explicit publication creates only one new private standalone
|
||||
fragment in an existing real directory. It never merges or replaces different content. Descriptor,
|
||||
parent, target, content, ownership, permission, and link identities are checked before and after
|
||||
the directory durability boundary. A failure rolls back when that can be proven and otherwise
|
||||
returns bounded unconfirmed publication evidence.
|
||||
|
||||
Doctor is a bounded read-only inspector with one fixed check inventory. It uses stable no-follow
|
||||
descriptor and configuration reads plus stat-only derived-index evidence. It never loads a
|
||||
complete projection, opens SQLite, starts MCP, executes the configured command, synchronizes,
|
||||
builds, renders, starts a viewer, or writes configuration. Unprovable client behavior is a warning,
|
||||
not an invented success.
|
||||
|
||||
## Isolated proposal model
|
||||
|
||||
Create, update, move, and delete are ordered node operations inside an isolated changeset. Every
|
||||
|
|
|
|||
85
docs/MILESTONE_2_BASELINE.md
Normal file
85
docs/MILESTONE_2_BASELINE.md
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
# Milestone 2 baseline
|
||||
|
||||
## Scope and method
|
||||
|
||||
This baseline records the agent-retrieval and client-integration behavior added in Milestone 2.
|
||||
It was captured on 2026-07-29 from clean candidate commit
|
||||
`fb0df5e4a1c591c2a84788fd4814d98550f11863`.
|
||||
|
||||
The maintained command was:
|
||||
|
||||
```bash
|
||||
.venv/bin/python tools/milestone2_benchmark.py \
|
||||
--nodes 1000 \
|
||||
--samples 10 \
|
||||
--output /tmp/docforge-milestone2-final.json
|
||||
```
|
||||
|
||||
The fixture contains 1,000 Markdown nodes and 999 edges in a direct fan-in around one focus node.
|
||||
The configured MCP response limit is 200,000 characters. Durations use
|
||||
`time.perf_counter_ns()` and nearest-rank p95. Peak memory uses an isolated child process and
|
||||
`RUSAGE_SELF`. Every warmup and measured invocation is validated.
|
||||
|
||||
Environment:
|
||||
|
||||
- Linux 7.1.3-200.nobara.fc44.x86_64.
|
||||
- CPython 3.14.6.
|
||||
- x86_64.
|
||||
- Ten warm samples after one warmup.
|
||||
- Isolated memory ceiling: 262,144 KiB.
|
||||
|
||||
The complete machine-readable result is
|
||||
[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json).
|
||||
|
||||
## Results
|
||||
|
||||
| Operation | Median | p95 | Limit | Maximum response |
|
||||
|---|---:|---:|---:|---:|
|
||||
| Read bootstrap | 9.406 ms | 9.884 ms | 100 ms | 7,495 B |
|
||||
| No-AST bootstrap | 9.153 ms | 9.379 ms | 100 ms | 8,796 B |
|
||||
| Task diagnostic page | 73.326 ms | 93.115 ms | 500 ms | 7,190 B |
|
||||
| Task complete traversal | 703.561 ms | 721.847 ms | 2,500 ms | 151,172 B/page |
|
||||
| Generation diagnostic page | 40.961 ms | 41.650 ms | 100 ms | 66,516 B |
|
||||
| Generation maximum page | 55.565 ms | 59.184 ms | 100 ms | 199,566 B |
|
||||
| Generation complete traversal | 418.607 ms | 425.315 ms | 500 ms | 66,516 B/page |
|
||||
| Codex configuration preview | 314.365 ms | 364.383 ms | 500 ms | 2,627 B |
|
||||
| Claude configuration preview | 314.326 ms | 364.365 ms | 500 ms | 2,748 B |
|
||||
| OpenClaw configuration preview | 314.401 ms | 364.532 ms | 500 ms | 2,869 B |
|
||||
| Codex doctor | 0.421 ms | 0.556 ms | 100 ms | 3,595 B |
|
||||
| Claude doctor | 0.364 ms | 0.446 ms | 100 ms | 3,669 B |
|
||||
| OpenClaw doctor | 0.384 ms | 0.484 ms | 100 ms | 3,602 B |
|
||||
|
||||
Isolated peak RSS was 86,448 KiB.
|
||||
|
||||
Task traversal returned 108 evidence records and 892 explicit omissions across 11 pages. One
|
||||
individually oversized focus record became a response-limit surrogate bound to the original record
|
||||
hash. The remaining omissions were token-budget evidence. The benchmark verified every unique
|
||||
subject, reconstructed the original collection hash, and matched the exact 1,000-node fixture.
|
||||
|
||||
Generation traversal returned all 1,000 changed-node details across 10 pages. It reconstructed the
|
||||
stored retained-collection hash. The maximum generation page approached the response limit and
|
||||
proved that optional diagnostics were dropped before the primary result.
|
||||
|
||||
## Structured-work gates
|
||||
|
||||
Configuration preview and doctor performed zero project loads, source parses, adapter projection
|
||||
loads, adapter extraction, index checks, synchronization, index builds, render preparation,
|
||||
rendered-byte construction or hashing, and viewer-manager requests.
|
||||
|
||||
Task-context pages performed exactly one index check and two cheap source-generation checks. They
|
||||
performed none of the hidden work above. Generation-diff pages performed exactly two cheap
|
||||
source-generation checks and no index check or hidden work.
|
||||
|
||||
## Measured limits and future notes
|
||||
|
||||
- Continuation is stateless and regenerates the task capsule for each page. The complete
|
||||
11-page traversal remains within its gate, but later work can avoid repeated planning without
|
||||
weakening generation binding.
|
||||
- Configuration preview deliberately spends about 314 ms proving that the exact isolated
|
||||
interpreter can import the MCP module. Discovery-only checks were rejected as unsafe.
|
||||
- Claude configuration syntax is supported, but its timeout representation remains unverified.
|
||||
Doctor therefore reports degraded rather than healthy.
|
||||
- Doctor is a configuration inspector, not an MCP connection or SQLite integrity test.
|
||||
- Legacy adapters without cheap source-generation identity report unknown for generation-diff
|
||||
freshness.
|
||||
- The results do not justify a storage rewrite. SQLite remains fast after one generation is pinned.
|
||||
68
docs/MILESTONE_2_CLOSEOUT.md
Normal file
68
docs/MILESTONE_2_CLOSEOUT.md
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
# Milestone 2 closeout
|
||||
|
||||
## Outcome
|
||||
|
||||
Milestone 2 is complete. One project-bound server can expose an explicit effective policy and
|
||||
return compact, task-shaped, explainable context. Users can generate deterministic client
|
||||
fragments and inspect their bindings without hidden runtime work.
|
||||
|
||||
Implemented contracts:
|
||||
|
||||
- Version-1 effective policy and capability-aware bootstrap.
|
||||
- Version-1 retrieval plans and context capsules.
|
||||
- Bounded task-context pagination with evidence gaps and explicit omissions.
|
||||
- One disposable latest-generation transition receipt and paged read surface.
|
||||
- Deterministic Codex, Claude, and OpenClaw standalone configuration fragments.
|
||||
- Fixed-inventory read-only doctor results.
|
||||
- Dedicated configuration and doctor JSON schemas.
|
||||
- Repository-native Milestone 2 contract, smoke, scale, response-size, counter, and memory gates.
|
||||
|
||||
## Candidate evidence
|
||||
|
||||
The frozen implementation candidate is
|
||||
`fb0df5e4a1c591c2a84788fd4814d98550f11863`.
|
||||
|
||||
The complete repository gate passed:
|
||||
|
||||
- Ruff formatting and lint.
|
||||
- HTML, rendered-manual HTML, CSS, and JavaScript checks.
|
||||
- Pyright with zero diagnostics.
|
||||
- Warning-strict compilation and tests.
|
||||
- 205 tests and 120 subtests.
|
||||
- Lock and npm dependency-tree checks.
|
||||
- Wheel and source-distribution builds.
|
||||
- Milestone 0, 1, and 2 smoke benchmarks.
|
||||
|
||||
Three independent read-only adversarial audits covered client publication and policy binding,
|
||||
doctor race and malformed-input behavior, and benchmark/contract evidence. Reproduced descriptor,
|
||||
parent, target, filesystem, policy, secret-redaction, ambiguity, parser, response-size, and hidden
|
||||
work defects were fixed and regression-tested before the candidate was frozen.
|
||||
|
||||
The clean ten-sample 1,000-node benchmark passed every threshold. Exact measurements and counter
|
||||
ranges are recorded in
|
||||
[`MILESTONE_2_BASELINE.md`](MILESTONE_2_BASELINE.md) and
|
||||
[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json).
|
||||
|
||||
## Preserved boundaries
|
||||
|
||||
- The `docforge` package, imports, CLI executable, MCP executable, and existing tool names remain.
|
||||
- Legacy one-method `load_projection()` adapters remain supported.
|
||||
- The no-AST shorthand and legacy adapter-policy payload remain compatible.
|
||||
- Project descriptor schema version 1 remains unchanged.
|
||||
- No storage replacement was introduced.
|
||||
- No legacy DocForge MCP or DocForge2 self-hosting was used.
|
||||
- WorldForge and ScrapeStation were not touched.
|
||||
- No production MCP integration was repointed.
|
||||
- The legacy Forgejo repository and `legacy` remote were not changed.
|
||||
- No tag, release, release announcement, or visibility change was created.
|
||||
|
||||
## Known follow-up work
|
||||
|
||||
The next active milestone may improve projection independence. It must not silently absorb these
|
||||
separate future ideas:
|
||||
|
||||
- Avoid recomputing a complete task capsule for every continuation page.
|
||||
- Add authenticated cursors only if a stronger threat model requires them.
|
||||
- Verify Claude's native timeout representation.
|
||||
- Add versioned adapter-owned launcher metadata before generating custom-adapter configurations.
|
||||
- Keep doctor read-only; a live connection test must be an explicit separate operation.
|
||||
|
|
@ -508,6 +508,67 @@ visualization-status
|
|||
visualization-stop
|
||||
```
|
||||
|
||||
### Client configuration and doctor
|
||||
|
||||
Preview one deterministic standalone client fragment:
|
||||
|
||||
```bash
|
||||
docforge configure codex --project /absolute/path/MyProject
|
||||
docforge configure claude --project /absolute/path/MyProject
|
||||
docforge configure openclaw --project /absolute/path/MyProject
|
||||
```
|
||||
|
||||
Preview is the default. Add `--output /absolute/path/fragment` to create a new private fragment in
|
||||
an existing real directory. Publication is create-only. DocForge accepts an identical existing
|
||||
private single-link file as unchanged, but it never merges, replaces, broadens permissions, or
|
||||
follows a symlink. Descriptor, parent, and target identities are revalidated across the
|
||||
publication commit.
|
||||
|
||||
The generated command uses the exact current Python interpreter with isolated module startup.
|
||||
Generation first proves that this interpreter can import `docforge.mcp_server`. The result binds
|
||||
the project root, effective policy, arguments, artifact bytes, and all hashes. It copies no ambient
|
||||
environment values.
|
||||
|
||||
Select authority explicitly:
|
||||
|
||||
```bash
|
||||
docforge configure codex \
|
||||
--project /absolute/path/MyProject \
|
||||
--capability-mode proposal \
|
||||
--proposal-writer project-editor
|
||||
|
||||
docforge configure codex \
|
||||
--project /absolute/path/MyProject \
|
||||
--capability-mode application \
|
||||
--proposal-writer project-editor \
|
||||
--canonical-applier project-editor
|
||||
```
|
||||
|
||||
Read mode is the default. Proposal and application modes fail closed unless the descriptor
|
||||
declares the named writer, and application requires the same writer/applier identity. Add
|
||||
`--no-ast` to preserve the no-AST binding. Generic CLI generation refuses project-owned adapters
|
||||
because it cannot safely reconstruct their composition.
|
||||
|
||||
Inspect one configured client binding:
|
||||
|
||||
```bash
|
||||
docforge doctor --client codex --project /absolute/path/MyProject
|
||||
docforge doctor --client codex \
|
||||
--project /absolute/path/MyProject \
|
||||
--config /absolute/path/config.toml \
|
||||
--server-name my-project-docforge
|
||||
```
|
||||
|
||||
Doctor returns `healthy`, `degraded`, or `unhealthy` with exit codes 0, 1, or 2. Its fixed
|
||||
version-1 inventory checks project and descriptor binding, the client driver and entry, executable
|
||||
and arguments, project root, effective policy, no-AST state, timeouts, environment-key names,
|
||||
tool-filter representation, and stat-only index presence.
|
||||
|
||||
Doctor is intentionally not a connection test. It never loads canonical sources, opens SQLite,
|
||||
starts MCP, executes the configured command, synchronizes, builds, renders, starts a viewer, or
|
||||
writes configuration. Claude timeout representation and client filtering that cannot be proved
|
||||
locally remain explicit warnings.
|
||||
|
||||
## MCP usage
|
||||
|
||||
Run one MCP server per project with absolute paths:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue