Close Milestone 3 with measured projection evidence
This commit is contained in:
parent
f5dccb5e1c
commit
d6d9f47672
11 changed files with 1115 additions and 47 deletions
|
|
@ -1,16 +1,13 @@
|
||||||
# Active milestone
|
# Milestone state
|
||||||
|
|
||||||
```text
|
```text
|
||||||
Milestone: 3 — independent projections
|
Last completed milestone: 3 — independent projections
|
||||||
Goal: Make manual output, portable graph artifacts, and the live viewer independent generation-pinned consumers of the validated graph.
|
Outcome: Manual output, portable graph artifacts, and the live viewer are independent generation-pinned consumers of the validated graph.
|
||||||
In scope: Versioned ManualRenderPlan and GraphViewPlan; immutable projection packages and receipts; independent manual and graph renderers; projection policies; incremental fragments; equivalence, recovery, accessibility, response-size, performance, and memory gates.
|
Evidence: Clean candidate f5dccb5e1c312121f1af63780162f593d9363b98; 281 tests and 272 subtests; 3 accessibility flows; clean 1,000-node ten-sample benchmark; isolated wheel proof; no secret-scan findings.
|
||||||
Out of scope: Adapter SDK expansion; remote render services; shared render farms; third-party renderer ecosystems; self-hosting; storage replacement; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
|
Active milestone: None.
|
||||||
Done when: Manual and graph plans are versioned and bounded; renderers cannot crawl project state or mutate canonical facts; portable and live graph modes remain separate; policies are enforced independently; status is receipt-only; full/incremental output is equivalent; accessibility and maintained scale gates pass.
|
Next directional milestone: 4 — adapter SDK and product documentation.
|
||||||
Status: Active implementation. Three independent audits were reconciled before source changes. The
|
Status: Milestone 3 is closed. Milestone 4 has not started.
|
||||||
versioned plan/package/receipt contracts, pure manual and graph planners, isolated manual renderer,
|
|
||||||
legacy byte-compatibility shim, packaged schemas, and pinned live-source correction are implemented
|
|
||||||
and focused-green. Durable publication, portable graph artifacts, detached workers, fragment
|
|
||||||
equivalence, independent policy enforcement, accessibility, and maintained scale gates remain.
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Milestones 4–5 remain directional context and are not active.
|
Milestones 4–5 remain directional context. Do not begin Milestone 4 without a new active-slice
|
||||||
|
contract.
|
||||||
|
|
|
||||||
|
|
@ -345,7 +345,7 @@ These are notes, not commitments:
|
||||||
than bounded positions, they will need a different versioned security contract and persisted key
|
than bounded positions, they will need a different versioned security contract and persisted key
|
||||||
lifecycle.
|
lifecycle.
|
||||||
|
|
||||||
## Milestone 2 — active: agent retrieval and MCP experience
|
## Milestone 2 — complete: agent retrieval and MCP experience
|
||||||
|
|
||||||
### Audit reconciliation
|
### Audit reconciliation
|
||||||
|
|
||||||
|
|
@ -611,10 +611,10 @@ task-shaped capsule for every continuation page, add authenticated continuation
|
||||||
model requires it, verify a native Claude timeout representation, and introduce adapter-owned
|
model requires it, verify a native Claude timeout representation, and introduce adapter-owned
|
||||||
launcher metadata before generating configurations for custom adapters.
|
launcher metadata before generating configurations for custom adapters.
|
||||||
|
|
||||||
## Milestone 3 — active: independent projections
|
## Milestone 3 — complete: independent projections
|
||||||
|
|
||||||
Milestone 3 began only after `main` and `dev` were aligned at the verified Milestone 2 closeout.
|
Milestone 3 began only after `main` and `dev` were aligned at the verified Milestone 2 closeout.
|
||||||
Three read-only audits are running before source changes:
|
Three read-only audits ran before source changes:
|
||||||
|
|
||||||
- Manual planning, immutable packages, renderer isolation, receipts, preview/application
|
- Manual planning, immutable packages, renderer isolation, receipts, preview/application
|
||||||
integration, and full/incremental equivalence.
|
integration, and full/incremental equivalence.
|
||||||
|
|
@ -692,8 +692,95 @@ The first slice now implements:
|
||||||
- A live-viewer correction: source evidence now comes from the pinned index generation. The
|
- A live-viewer correction: source evidence now comes from the pinned index generation. The
|
||||||
viewer no longer reopens mutable canonical files behind an older graph snapshot.
|
viewer no longer reopens mutable canonical files behind an older graph snapshot.
|
||||||
|
|
||||||
The new repository-native contract target passes 91 tests and 120 subtests. The combined
|
The new repository-native contract target passed 91 tests and 120 subtests at the slice boundary.
|
||||||
projection, rendering, and live-viewer focus passes with byte-exact compatibility and no hidden
|
The combined projection, rendering, and live-viewer focus passed with byte-exact compatibility and
|
||||||
source/path authority. This is not Milestone 3 closeout: durable multi-artifact publication,
|
no hidden source/path authority.
|
||||||
portable graph rendering, detached workers, fragment reuse/equivalence, policy version 2,
|
|
||||||
accessibility, and maintained scale evidence remain active work.
|
### Durable portable graph publication
|
||||||
|
|
||||||
|
The portable graph path now has its own declared `graph_render` views, pure plans, fixed
|
||||||
|
`portable_graph_html` renderer, content-addressed artifact store, renderer receipts, and one bounded
|
||||||
|
generation/view manifest as the publication commit. It supports Nodes, Flow, and Web without
|
||||||
|
including Logic. Static HTML contains the complete pre-rendered graph and treats JavaScript as
|
||||||
|
progressive enhancement.
|
||||||
|
|
||||||
|
Publication revalidates source, view, artifact, receipt, and output identities across replacement.
|
||||||
|
Status reads only bounded manifest and receipt evidence. It never plans or renders. Repair may
|
||||||
|
restore a declared output from its content-addressed artifact. A post-artifact failure that cannot
|
||||||
|
be rolled back returns explicit degraded committed evidence rather than reporting an ordinary
|
||||||
|
failed mutation.
|
||||||
|
|
||||||
|
### Detached workers and incremental fragments
|
||||||
|
|
||||||
|
Manual and portable graph packages execute through one fixed one-request child protocol. The
|
||||||
|
parent launches isolated Python from a trusted working directory with a sanitized environment,
|
||||||
|
spools stdout to disk, reads one bounded canonical response, and validates the complete artifact
|
||||||
|
and receipt identity. The worker accepts only the two built-in renderer identities. Requests are
|
||||||
|
bounded by the 24,000,000-byte package contract, actual artifact transfer by 20,000,000 bytes, and
|
||||||
|
execution by a 30-second timeout.
|
||||||
|
|
||||||
|
Manual fragment records are semantic, versioned, canonical, hash-bound, and stored below a
|
||||||
|
dedicated confined cache. The worker independently recomputes the expected page fragment before
|
||||||
|
using a record. Corrupt, forged, oversized, stale, or aggregate-oversized records fall back to the
|
||||||
|
full detached render. Cold fragment creation is compared byte-for-byte with that full oracle before
|
||||||
|
cache publication. The cache retains only the current inventory and is capped at 10,000 entries
|
||||||
|
and 64,000,000 bytes.
|
||||||
|
|
||||||
|
### Independent policies and accessibility
|
||||||
|
|
||||||
|
Projection policy version 2 independently composes manual `auto|explicit|disabled`, portable graph
|
||||||
|
`explicit|disabled`, and live viewer `on-demand|disabled`. CLI, MCP, generated client
|
||||||
|
configuration, doctor, render services, canonical application, onboarding, and viewer-manager
|
||||||
|
entry points enforce their relevant policy. Status remains available when an active operation is
|
||||||
|
disabled.
|
||||||
|
|
||||||
|
Generated client evidence binds the projection policy, its hash, projection availability, and the
|
||||||
|
current descriptor hash into the configuration hash. Validation cross-checks omitted default
|
||||||
|
selectors against the bound descriptor so coordinated policy and availability drift fails closed.
|
||||||
|
The version-1 effective-policy payload remains unchanged for existing clients.
|
||||||
|
|
||||||
|
Pinned Playwright 1.62.0 and axe-core 4.12.1 gates exercise the frozen manual, portable graph, and
|
||||||
|
live viewer with selected WCAG A/AA axe tags and keyboard interaction flows. Portable and live
|
||||||
|
graph presentation received only the minimal contrast and nested-role corrections needed by those
|
||||||
|
gates.
|
||||||
|
|
||||||
|
### Scale and runtime hardening
|
||||||
|
|
||||||
|
The first 1,000-node full benchmark exposed recursive strongly connected-component traversal in
|
||||||
|
manual planning. Cycle detection now uses an iterative two-pass traversal. A regression covers the
|
||||||
|
descriptor maximum of 10,000 nodes as both a deep acyclic chain and one strongly connected
|
||||||
|
component.
|
||||||
|
|
||||||
|
The isolated wheel proof also exposed a Python `runpy` warning when the worker module was imported
|
||||||
|
during package initialization before `-m` execution. A private fixed module entrypoint now owns
|
||||||
|
child startup. Malformed child input returns code 2 with empty stdout and stderr.
|
||||||
|
|
||||||
|
Configured render ceilings above 20,000,000 bytes remain accepted for compatibility, and small
|
||||||
|
actual artifacts render normally. The detached protocol still rejects an actual transfer beyond
|
||||||
|
its fixed 20,000,000-byte boundary.
|
||||||
|
|
||||||
|
### Milestone 3 closeout
|
||||||
|
|
||||||
|
Candidate `f5dccb5e1c312121f1af63780162f593d9363b98` passed the complete repository gate: formatting,
|
||||||
|
Python and web lint, strict types, compilation, 281 tests and 272 subtests, three accessibility
|
||||||
|
flows, lock and dependency checks, package builds, and all milestone smoke benchmarks. The
|
||||||
|
maintained projection contract subset passed 142 tests and 236 subtests.
|
||||||
|
|
||||||
|
The clean ten-sample 1,000-node benchmark passed every latency, memory, response-size, no-work, and
|
||||||
|
equivalence gate. Manual full rendering measured 810.490 ms p95, portable graph full rendering
|
||||||
|
323.690 ms p95, and receipt-only status 111.381 ms and 59.331 ms p95 respectively. Direct detached
|
||||||
|
worker peaks were 88,580,096 and 89,583,616 bytes. The separately gated production manual worker
|
||||||
|
peak was 104,771,584 bytes. Production cold, warm, forced-full, add, change, delete, and reorder
|
||||||
|
outputs were byte-identical.
|
||||||
|
|
||||||
|
Production warm fragment rendering measured 2,206.540 ms p95 versus 978.870 ms for forced full.
|
||||||
|
Milestone 3 therefore closes the fragment isolation, invalidation, equivalence, and recovery
|
||||||
|
contract without claiming a throughput win. Later optimization must begin from that evidence.
|
||||||
|
|
||||||
|
The exact method and measurements are recorded in `docs/MILESTONE_3_BASELINE.md` and
|
||||||
|
`benchmarks/milestone3-2026-07-29.json`. The candidate passed an isolated wheel CLI/MCP/worker
|
||||||
|
proof. Gitleaks 8.30.1 found no findings across the six Milestone 3 commits or candidate tree.
|
||||||
|
|
||||||
|
Milestone 3 is complete. No tag, release, production integration repointing, WorldForge change,
|
||||||
|
ScrapeStation change, storage rewrite, or self-hosting dependency was introduced. Milestone 4
|
||||||
|
remains directional and has not started.
|
||||||
|
|
|
||||||
73
README.md
73
README.md
|
|
@ -26,6 +26,12 @@ declared manuals, visualizes project structure, and manages reviewable documenta
|
||||||
- Detects project-local adapter implementation and configuration changes and requires a fresh
|
- Detects project-local adapter implementation and configuration changes and requires a fresh
|
||||||
project-bound process before any further MCP work.
|
project-bound process before any further MCP work.
|
||||||
- Keeps function-scoped control-flow projections separate from the primary architecture graph.
|
- Keeps function-scoped control-flow projections separate from the primary architecture graph.
|
||||||
|
- Compiles manuals and portable graph artifacts from separate versioned, generation-pinned plans
|
||||||
|
and immutable packages.
|
||||||
|
- Runs built-in manual and graph renderers in fixed detached workers with validated receipts,
|
||||||
|
bounded transfer, and no project-path authority.
|
||||||
|
- Enforces independent manual, portable-graph, and live-viewer policy while keeping status
|
||||||
|
receipt-only.
|
||||||
- Runs a managed loopback graph browser with neighborhood, semantic Flow, convergence Web,
|
- Runs a managed loopback graph browser with neighborhood, semantic Flow, convergence Web,
|
||||||
function-scoped Logic, source inspection, and branch-aware node hiding.
|
function-scoped Logic, source inspection, and branch-aware node hiding.
|
||||||
- Supports generic documentation projects and project-owned source adapters.
|
- Supports generic documentation projects and project-owned source adapters.
|
||||||
|
|
@ -73,6 +79,63 @@ details, and explicit truncation. Paged results use one top-level cursor and a v
|
||||||
`receipt_header`; `stored_receipt_hash` identifies the complete persisted receipt. The read never
|
`receipt_header`; `stored_receipt_hash` identifies the complete persisted receipt. The read never
|
||||||
exposes Logic details, loads canonical source, repairs derived state, or invents history.
|
exposes Logic details, loads canonical source, repairs derived state, or invents history.
|
||||||
|
|
||||||
|
## Independent projections
|
||||||
|
|
||||||
|
Manual compilation, portable graph rendering, and the live viewer consume the same validated graph
|
||||||
|
generation through separate boundaries:
|
||||||
|
|
||||||
|
```text
|
||||||
|
validated generation
|
||||||
|
├── ManualRenderPlanV1 → immutable package → detached manual renderer
|
||||||
|
├── GraphViewPlanV1 → immutable package → detached portable graph renderer
|
||||||
|
└── pinned index → managed read-only live viewer
|
||||||
|
```
|
||||||
|
|
||||||
|
Plans, packages, and receipts are canonical, versioned, hash-identified, bounded, and contain no
|
||||||
|
project object, SQLite handle, absolute project path, command, or caller-selected renderer module.
|
||||||
|
Renderers cannot select graph facts, crawl canonical sources, choose publication paths, or mutate
|
||||||
|
project state.
|
||||||
|
|
||||||
|
Declare portable graph output separately from manual views:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[graph_render]
|
||||||
|
output_root = ".docforge/portable-graph"
|
||||||
|
|
||||||
|
[[graph_render.views]]
|
||||||
|
id = "architecture"
|
||||||
|
renderer = "portable_graph_html"
|
||||||
|
output = "architecture.html"
|
||||||
|
title = "Architecture"
|
||||||
|
root = "architecture.overview"
|
||||||
|
initial_mode = "web"
|
||||||
|
depth = 3
|
||||||
|
max_nodes = 250
|
||||||
|
max_edges = 1000
|
||||||
|
max_work = 100000
|
||||||
|
include_logic = false
|
||||||
|
```
|
||||||
|
|
||||||
|
Plan, publish, and inspect it explicitly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/docforge --project-root "$PROJECT" graph-plan architecture
|
||||||
|
.venv/bin/docforge --project-root "$PROJECT" graph-render architecture
|
||||||
|
.venv/bin/docforge --project-root "$PROJECT" graph-render-status architecture
|
||||||
|
```
|
||||||
|
|
||||||
|
The version-2 projection policy independently selects manual
|
||||||
|
`auto|explicit|disabled`, portable graph `explicit|disabled`, and live viewer
|
||||||
|
`on-demand|disabled`. Use `--manual-render-policy`, `--portable-graph-policy`, and
|
||||||
|
`--live-viewer-policy` on CLI/MCP startup or generated client configuration. Status remains
|
||||||
|
available when the corresponding active operation is disabled.
|
||||||
|
|
||||||
|
Manual fragment reuse is disposable. Cold record publication is guarded by byte-exact comparison
|
||||||
|
with a full detached render; warm records are independently recomputed and validated inside the
|
||||||
|
worker. Full rendering remains the recovery and equivalence oracle. Portable artifacts commit
|
||||||
|
content-addressed output and renderer evidence before one bounded generation/view manifest; status
|
||||||
|
does not plan or render.
|
||||||
|
|
||||||
## Graph views
|
## Graph views
|
||||||
|
|
||||||
The browser presents the primary architecture graph through three complementary views and loads a
|
The browser presents the primary architecture graph through three complementary views and loads a
|
||||||
|
|
@ -170,10 +233,18 @@ DocForge describes them as a source graph.
|
||||||
performance, memory, rendering and response sizes, bottlenecks, and missing coverage.
|
performance, memory, rendering and response sizes, bottlenecks, and missing coverage.
|
||||||
- [Milestone 0 closeout](docs/MILESTONE_0_CLOSEOUT.md) — lineage, migration, security scan,
|
- [Milestone 0 closeout](docs/MILESTONE_0_CLOSEOUT.md) — lineage, migration, security scan,
|
||||||
repository state, and fresh-clone proof.
|
repository state, and fresh-clone proof.
|
||||||
|
- [Milestone 1 baseline](docs/MILESTONE_1_BASELINE.md) — warm operation latency, structured work,
|
||||||
|
status, retrieval, and memory measurements.
|
||||||
|
- [Milestone 1 closeout](docs/MILESTONE_1_CLOSEOUT.md) — fast-core contracts, adversarial
|
||||||
|
validation, compatibility boundaries, and exact candidate evidence.
|
||||||
- [Milestone 2 baseline](docs/MILESTONE_2_BASELINE.md) — task context, generation diff, client
|
- [Milestone 2 baseline](docs/MILESTONE_2_BASELINE.md) — task context, generation diff, client
|
||||||
configuration, doctor, response-size, counter, and memory measurements.
|
configuration, doctor, response-size, counter, and memory measurements.
|
||||||
- [Milestone 2 closeout](docs/MILESTONE_2_CLOSEOUT.md) — implemented contracts, adversarial
|
- [Milestone 2 closeout](docs/MILESTONE_2_CLOSEOUT.md) — implemented contracts, adversarial
|
||||||
validation, exclusions, and exact candidate evidence.
|
validation, exclusions, and exact candidate evidence.
|
||||||
|
- [Milestone 3 baseline](docs/MILESTONE_3_BASELINE.md) — manual, fragment, worker, portable graph,
|
||||||
|
status, equivalence, response-size, and memory measurements.
|
||||||
|
- [Milestone 3 closeout](docs/MILESTONE_3_CLOSEOUT.md) — independent projection contracts,
|
||||||
|
adversarial validation, compatibility boundaries, and exact candidate evidence.
|
||||||
- [MCP contract](docs/MCP_CONTRACT.md) — exact tool and process boundary.
|
- [MCP contract](docs/MCP_CONTRACT.md) — exact tool and process boundary.
|
||||||
- [Viewer manager](docs/VIEWER_MANAGER.md) — native service setup and lifecycle.
|
- [Viewer manager](docs/VIEWER_MANAGER.md) — native service setup and lifecycle.
|
||||||
- [Adapter decision](docs/APPLICATION_DECISION.md) — why custom adapters own canonical
|
- [Adapter decision](docs/APPLICATION_DECISION.md) — why custom adapters own canonical
|
||||||
|
|
@ -197,6 +268,8 @@ make gate
|
||||||
Focused entry points are available as `make contract`, `make test`, `make type`,
|
Focused entry points are available as `make contract`, `make test`, `make type`,
|
||||||
`make benchmark-smoke`, `make benchmark`, `make benchmark-m1-smoke`, and
|
`make benchmark-smoke`, `make benchmark`, `make benchmark-m1-smoke`, and
|
||||||
`make benchmark-m1`. Milestone 2 adds `make benchmark-m2-smoke` and `make benchmark-m2`.
|
`make benchmark-m1`. Milestone 2 adds `make benchmark-m2-smoke` and `make benchmark-m2`.
|
||||||
|
Milestone 3 adds `make accessibility`, `make benchmark-m3-smoke`, `make benchmark-m3`, and
|
||||||
|
`make benchmark-m3-full`.
|
||||||
|
|
||||||
The committed 1,000-node baseline and its measurement method are under `benchmarks/`.
|
The committed 1,000-node baseline and its measurement method are under `benchmarks/`.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,51 @@
|
||||||
# Completed slices
|
# Completed slices
|
||||||
|
|
||||||
|
## DocForge2 Milestone 3 independent projections
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Added strict version-1 manual plan, graph plan, projection package, and projection receipt
|
||||||
|
contracts with canonical identities and packaged schemas.
|
||||||
|
- Kept the legacy manual API and exact alpha bytes as a compatibility wrapper over a pure planner,
|
||||||
|
immutable package, and independent renderer.
|
||||||
|
- Added generation-pinned portable graph planning, detached rendering, content-addressed durable
|
||||||
|
publication, repair, and receipt-only status.
|
||||||
|
- Added a fixed isolated worker protocol with bounded request, response, artifact, timeout,
|
||||||
|
environment, and renderer inventory.
|
||||||
|
- Added semantic manual fragments with independent worker validation, corruption recovery,
|
||||||
|
full-render equivalence, and bounded cache retention.
|
||||||
|
- Added version-2 independent manual, portable-graph, and live-viewer policy with descriptor-bound
|
||||||
|
generated-client evidence while retaining effective policy version 1.
|
||||||
|
- Corrected live source reads to use the pinned index generation.
|
||||||
|
- Added static and interactive accessibility gates for the manual, portable graph, and live viewer.
|
||||||
|
- Replaced recursive cycle planning with an iterative traversal proven at 10,000 nodes.
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
- The complete gate passed 281 tests, 272 subtests, three accessibility flows, dependency checks,
|
||||||
|
builds, and all milestone smoke benchmarks.
|
||||||
|
- The maintained projection contract subset passed 142 tests and 236 subtests.
|
||||||
|
- The clean ten-sample 1,000-node benchmark passed every latency, memory, response-size, no-work,
|
||||||
|
deterministic-output, and full/incremental equivalence threshold.
|
||||||
|
- An isolated wheel passed CLI, MCP, detached manual rendering, and malformed worker startup.
|
||||||
|
- Three adversarial review tracks were reconciled, and Gitleaks found no findings in the six
|
||||||
|
milestone commits or candidate tree.
|
||||||
|
|
||||||
|
### Limits
|
||||||
|
|
||||||
|
- Production fragment reuse is correct and recoverable but is slower than forced full rendering at
|
||||||
|
the maintained 1,000-page fixture. No speedup is claimed.
|
||||||
|
- Actual detached artifact transfer is capped at 20,000,000 bytes.
|
||||||
|
- Portable graph publication is CLI-owned. MCP exposes read-only plan and status tools.
|
||||||
|
- Remote render services, render farms, third-party renderer ecosystems, storage replacement,
|
||||||
|
self-hosting, adapter SDK expansion, and production integration changes remain out of scope.
|
||||||
|
- No tag or release was created.
|
||||||
|
|
||||||
|
### Next gate
|
||||||
|
|
||||||
|
Milestone 4 remains directional and is not active. Create a new active-slice contract before
|
||||||
|
starting adapter SDK or product-documentation implementation.
|
||||||
|
|
||||||
## DocForge2 Milestone 0 successor foundation
|
## DocForge2 Milestone 0 successor foundation
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
|
||||||
|
|
@ -34,6 +34,14 @@ make benchmark-m2-smoke
|
||||||
make benchmark-m2
|
make benchmark-m2
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Run the Milestone 3 independent-projection smoke and maintained full gates:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make benchmark-m3-smoke
|
||||||
|
make benchmark-m3
|
||||||
|
make benchmark-m3-full
|
||||||
|
```
|
||||||
|
|
||||||
The benchmark creates canonical sources, derived state, changesets, rendered output, and caches
|
The benchmark creates canonical sources, derived state, changesets, rendered output, and caches
|
||||||
only in a disposable temporary directory. It does not read another project, self-host DocForge, or
|
only in a disposable temporary directory. It does not read another project, self-host DocForge, or
|
||||||
mutate repository content.
|
mutate repository content.
|
||||||
|
|
@ -56,9 +64,16 @@ records whether diagnostics were dropped for response budget, checks all hidden-
|
||||||
and measures isolated-process peak RSS. Its interpretation is in
|
and measures isolated-process peak RSS. Its interpretation is in
|
||||||
[`docs/MILESTONE_2_BASELINE.md`](../docs/MILESTONE_2_BASELINE.md).
|
[`docs/MILESTONE_2_BASELINE.md`](../docs/MILESTONE_2_BASELINE.md).
|
||||||
|
|
||||||
The generic fixture exposes whole-source scaling. It does not replace the incremental adapter
|
`milestone3-2026-07-29.json` is the clean-tree independent-projection baseline captured from commit
|
||||||
equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has
|
`f5dccb5e1c312121f1af63780162f593d9363b98`. It measures versioned manual and graph planning,
|
||||||
no portable graph-planning or graph-rendering contract.
|
in-process and detached rendering, production cold/warm/forced-full behavior, fragment cache
|
||||||
|
sweeps, add/change/delete/reorder equivalence, portable publication, receipt-only status, traced
|
||||||
|
memory, detached worker peak memory, and response size. Its interpretation is in
|
||||||
|
[`docs/MILESTONE_3_BASELINE.md`](../docs/MILESTONE_3_BASELINE.md).
|
||||||
|
|
||||||
|
The generic fixtures expose whole-source and projection scaling. They do not replace incremental
|
||||||
|
adapter equivalence tests. Milestone 3's portable fixture contains 1,000 Nodes/Flow/Web nodes and
|
||||||
|
999 edges; portable version 1 deliberately excludes Logic.
|
||||||
|
|
||||||
The Milestone 1 harness treats wall time and structured work counters as separate gates. Warm
|
The Milestone 1 harness treats wall time and structured work counters as separate gates. Warm
|
||||||
operations fail if they load a complete project, parse source files, reconstruct an adapter
|
operations fail if they load a complete project, parse source files, reconstruct an adapter
|
||||||
|
|
@ -73,4 +88,5 @@ status. The 1,000-node run records bounded semantic summaries for exact errors,
|
||||||
backlinks, both traversal directions, paged context, render receipt states, and visualization
|
backlinks, both traversal directions, paged context, render receipt states, and visualization
|
||||||
freshness. The reported p95 uses the nearest-rank method; with ten samples it is the maximum.
|
freshness. The reported p95 uses the nearest-rank method; with ten samples it is the maximum.
|
||||||
`process_peak_rss_kib` is the cumulative main-process `RUSAGE_SELF` high-water mark and excludes the
|
`process_peak_rss_kib` is the cumulative main-process `RUSAGE_SELF` high-water mark and excludes the
|
||||||
detached viewer worker.
|
detached viewer worker. It is diagnostic and not operation-local. Milestone 3 memory gates use
|
||||||
|
per-operation `tracemalloc` peaks and detached worker receipt peaks instead.
|
||||||
|
|
|
||||||
442
benchmarks/milestone3-2026-07-29.json
Normal file
442
benchmarks/milestone3-2026-07-29.json
Normal file
|
|
@ -0,0 +1,442 @@
|
||||||
|
{
|
||||||
|
"benchmark": "docforge2_milestone3",
|
||||||
|
"environment": {
|
||||||
|
"implementation": "CPython",
|
||||||
|
"machine": "x86_64",
|
||||||
|
"platform": "Linux-7.1.3-200.nobara.fc44.x86_64-x86_64-with-glibc2.43",
|
||||||
|
"python": "3.14.6"
|
||||||
|
},
|
||||||
|
"equivalence": {
|
||||||
|
"manual_full_vs_fragment_assisted": true,
|
||||||
|
"manual_in_process_vs_detached": true,
|
||||||
|
"manual_production_cold_warm_full": true,
|
||||||
|
"manual_production_variants": {
|
||||||
|
"add": true,
|
||||||
|
"change": true,
|
||||||
|
"delete": true,
|
||||||
|
"reorder": true
|
||||||
|
},
|
||||||
|
"portable_graph_in_process_vs_detached": true
|
||||||
|
},
|
||||||
|
"fixture": {
|
||||||
|
"edge_count": 999,
|
||||||
|
"full_coverage": true,
|
||||||
|
"kind": "synthetic_generic_projection",
|
||||||
|
"manual_page_count": 1000,
|
||||||
|
"node_count": 1000,
|
||||||
|
"portable_graph_edge_count": 999,
|
||||||
|
"portable_graph_node_count": 1000
|
||||||
|
},
|
||||||
|
"memory": {
|
||||||
|
"manual_production_worker_peak_bytes": 104771584,
|
||||||
|
"manual_worker_peak_bytes": 88580096,
|
||||||
|
"portable_graph_worker_peak_bytes": 89583616,
|
||||||
|
"process_peak_rss_kib": 102692
|
||||||
|
},
|
||||||
|
"method": {
|
||||||
|
"clock": "time.perf_counter_ns",
|
||||||
|
"detached_peak_memory": "worker receipt resource peak RSS",
|
||||||
|
"determinism": "stable semantic summaries must match across samples; report JSON uses sorted keys",
|
||||||
|
"full_mode_node_requirement": 1000,
|
||||||
|
"in_process_peak_memory": "tracemalloc per measured invocation",
|
||||||
|
"maximum_worker_artifact_bytes": 20000000,
|
||||||
|
"process_peak_memory": "resource.getrusage(RUSAGE_SELF).ru_maxrss",
|
||||||
|
"response_size": "UTF-8 bytes of canonical compact sorted JSON",
|
||||||
|
"samples": 10
|
||||||
|
},
|
||||||
|
"mode": "full",
|
||||||
|
"operations": {
|
||||||
|
"fragment_assisted_equivalence": {
|
||||||
|
"max_ms": 666.119,
|
||||||
|
"maximum_response_bytes": 664,
|
||||||
|
"maximum_traced_peak_bytes": 5332792,
|
||||||
|
"median_ms": 638.999,
|
||||||
|
"min_ms": 633.157,
|
||||||
|
"p95_limit_ms": 15000,
|
||||||
|
"p95_ms": 666.119,
|
||||||
|
"response_limit_bytes": 128000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"artifact_bytes": 583149,
|
||||||
|
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"package_bytes": 2364302,
|
||||||
|
"package_id": "ecc21e58106c420a9f78ffbba997778ca98b61586ecfdb23a15b08f431a52c42",
|
||||||
|
"plan_bytes": 1006393,
|
||||||
|
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"fragment_cache_hit_sweep": {
|
||||||
|
"max_ms": 514.324,
|
||||||
|
"maximum_response_bytes": 145,
|
||||||
|
"maximum_traced_peak_bytes": 6941174,
|
||||||
|
"median_ms": 498.911,
|
||||||
|
"min_ms": 495.447,
|
||||||
|
"p95_limit_ms": 5000,
|
||||||
|
"p95_ms": 514.324,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"aggregate_content_bytes": 516921,
|
||||||
|
"fragment_count": 1000,
|
||||||
|
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"fragment_cache_miss_sweep": {
|
||||||
|
"max_ms": 106.888,
|
||||||
|
"maximum_response_bytes": 145,
|
||||||
|
"maximum_traced_peak_bytes": 146245,
|
||||||
|
"median_ms": 106.203,
|
||||||
|
"min_ms": 105.383,
|
||||||
|
"p95_limit_ms": 5000,
|
||||||
|
"p95_ms": 106.888,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"aggregate_content_bytes": 516921,
|
||||||
|
"fragment_count": 1000,
|
||||||
|
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"fragment_cache_put_sweep": {
|
||||||
|
"max_ms": 539.212,
|
||||||
|
"maximum_response_bytes": 145,
|
||||||
|
"maximum_traced_peak_bytes": 509548,
|
||||||
|
"median_ms": 539.212,
|
||||||
|
"min_ms": 539.212,
|
||||||
|
"p95_limit_ms": 10000,
|
||||||
|
"p95_ms": 539.212,
|
||||||
|
"response_limit_bytes": 32768,
|
||||||
|
"samples": 1,
|
||||||
|
"stable_result": {
|
||||||
|
"aggregate_content_bytes": 516921,
|
||||||
|
"fragment_count": 1000,
|
||||||
|
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"manual_detached_worker": {
|
||||||
|
"child_peak_limit_bytes": 268435456,
|
||||||
|
"max_ms": 599.394,
|
||||||
|
"maximum_child_peak_bytes": 88580096,
|
||||||
|
"maximum_response_bytes": 667,
|
||||||
|
"maximum_traced_peak_bytes": 28203708,
|
||||||
|
"median_ms": 591.317,
|
||||||
|
"min_ms": 585.123,
|
||||||
|
"p95_limit_ms": 20000,
|
||||||
|
"p95_ms": 599.394,
|
||||||
|
"response_limit_bytes": 128000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"artifact_bytes": 583149,
|
||||||
|
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"package_bytes": 1007297,
|
||||||
|
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
|
||||||
|
"plan_bytes": 1006393,
|
||||||
|
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"manual_forced_full": {
|
||||||
|
"child_peak_limit_bytes": 268435456,
|
||||||
|
"max_ms": 978.87,
|
||||||
|
"maximum_child_peak_bytes": 104771584,
|
||||||
|
"maximum_response_bytes": 668,
|
||||||
|
"maximum_traced_peak_bytes": 30425714,
|
||||||
|
"median_ms": 944.135,
|
||||||
|
"min_ms": 934.773,
|
||||||
|
"p95_limit_ms": 20000,
|
||||||
|
"p95_ms": 978.87,
|
||||||
|
"response_limit_bytes": 128000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"output_bytes": 583149,
|
||||||
|
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"manual_full_render": {
|
||||||
|
"max_ms": 810.49,
|
||||||
|
"maximum_response_bytes": 664,
|
||||||
|
"maximum_traced_peak_bytes": 4412754,
|
||||||
|
"median_ms": 801.948,
|
||||||
|
"min_ms": 773.7,
|
||||||
|
"p95_limit_ms": 15000,
|
||||||
|
"p95_ms": 810.49,
|
||||||
|
"response_limit_bytes": 128000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"artifact_bytes": 583149,
|
||||||
|
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"package_bytes": 1007297,
|
||||||
|
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
|
||||||
|
"plan_bytes": 1006393,
|
||||||
|
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"manual_incremental_cold": {
|
||||||
|
"child_peak_limit_bytes": 268435456,
|
||||||
|
"max_ms": 2827.152,
|
||||||
|
"maximum_child_peak_bytes": 99454976,
|
||||||
|
"maximum_response_bytes": 668,
|
||||||
|
"maximum_traced_peak_bytes": 35160716,
|
||||||
|
"median_ms": 2827.152,
|
||||||
|
"min_ms": 2827.152,
|
||||||
|
"p95_limit_ms": 20000,
|
||||||
|
"p95_ms": 2827.152,
|
||||||
|
"response_limit_bytes": 128000,
|
||||||
|
"samples": 1,
|
||||||
|
"stable_result": {
|
||||||
|
"output_bytes": 583149,
|
||||||
|
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"manual_incremental_warm": {
|
||||||
|
"child_peak_limit_bytes": 268435456,
|
||||||
|
"max_ms": 2206.54,
|
||||||
|
"maximum_child_peak_bytes": 104767488,
|
||||||
|
"maximum_response_bytes": 669,
|
||||||
|
"maximum_traced_peak_bytes": 34676043,
|
||||||
|
"median_ms": 2143.388,
|
||||||
|
"min_ms": 2125.466,
|
||||||
|
"p95_limit_ms": 20000,
|
||||||
|
"p95_ms": 2206.54,
|
||||||
|
"response_limit_bytes": 128000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"output_bytes": 583149,
|
||||||
|
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"manual_status_no_work": {
|
||||||
|
"max_ms": 111.381,
|
||||||
|
"maximum_response_bytes": 1542,
|
||||||
|
"maximum_traced_peak_bytes": 1792025,
|
||||||
|
"median_ms": 62.881,
|
||||||
|
"min_ms": 60.111,
|
||||||
|
"p95_limit_ms": 500,
|
||||||
|
"p95_ms": 111.381,
|
||||||
|
"response_limit_bytes": 256000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"counters": {
|
||||||
|
"adapter_projection_loads": 0,
|
||||||
|
"adapter_source_extractions": 0,
|
||||||
|
"index_builds": 0,
|
||||||
|
"index_checks": 0,
|
||||||
|
"index_synchronizations": 0,
|
||||||
|
"project_loads": 0,
|
||||||
|
"render_output_bytes_built": 0,
|
||||||
|
"render_output_bytes_hashed": 0,
|
||||||
|
"render_prepare_calls": 0,
|
||||||
|
"source_bytes_parsed": 0,
|
||||||
|
"source_files_parsed": 0,
|
||||||
|
"source_generation_checks": 2,
|
||||||
|
"viewer_manager_requests": 0
|
||||||
|
},
|
||||||
|
"response": {
|
||||||
|
"adapter": "generic",
|
||||||
|
"configured": true,
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"actual_output_hash": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"expected_output_hash": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"path": ".docforge/rendered/manual.html",
|
||||||
|
"projection_receipt": {
|
||||||
|
"artifacts": [
|
||||||
|
{
|
||||||
|
"artifact_id": "manual.html",
|
||||||
|
"bytes": 583149,
|
||||||
|
"media_type": "text/html; charset=utf-8",
|
||||||
|
"sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"contract": "docforge.projection-receipt",
|
||||||
|
"diagnostics": {
|
||||||
|
"warnings": []
|
||||||
|
},
|
||||||
|
"kind": "manual",
|
||||||
|
"package_id": "b30c65390570537a16ce03afe6a992599c6ca0c4ef5ea2f06d14b33e248bc9aa",
|
||||||
|
"peak_memory_bytes": 104771584,
|
||||||
|
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
|
||||||
|
"receipt_id": "ead4c25b34ada8df674903a592e25dd6bbf9817df2b04e5ebc66cde9e41c1ed1",
|
||||||
|
"renderer": {
|
||||||
|
"renderer_id": "generic_html",
|
||||||
|
"renderer_version": "1+markdown-it-py-4.2.0"
|
||||||
|
},
|
||||||
|
"schema_version": 1,
|
||||||
|
"timing": {
|
||||||
|
"elapsed_ns": 114544865
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"reason": null,
|
||||||
|
"receipt_schema_version": 1,
|
||||||
|
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0",
|
||||||
|
"renderer": "generic_html",
|
||||||
|
"renderer_version": "1+markdown-it-py-4.2.0",
|
||||||
|
"state": "current",
|
||||||
|
"template_hash": "ae0ebb3eadeb530e9d033c8fbd21321d406e29d04c0ae7d07a9a0f929f0ffa9f",
|
||||||
|
"verification": "receipt",
|
||||||
|
"view_id": "manual"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"project_id": "synthetic-1000",
|
||||||
|
"project_root_fingerprint": "0747acfd975703c0",
|
||||||
|
"revision": "unversioned",
|
||||||
|
"source_hash": "a314da4ffa8fcf291ef7a7b0fc87737caeaa89fa3c558ea442afeb0e5ae49c2d",
|
||||||
|
"state": "current",
|
||||||
|
"status": "ok",
|
||||||
|
"verification": "receipt"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"portable_graph_detached_worker": {
|
||||||
|
"child_peak_limit_bytes": 268435456,
|
||||||
|
"max_ms": 268.428,
|
||||||
|
"maximum_child_peak_bytes": 89583616,
|
||||||
|
"maximum_response_bytes": 660,
|
||||||
|
"maximum_traced_peak_bytes": 27587227,
|
||||||
|
"median_ms": 265.448,
|
||||||
|
"min_ms": 263.761,
|
||||||
|
"p95_limit_ms": 20000,
|
||||||
|
"p95_ms": 268.428,
|
||||||
|
"response_limit_bytes": 128000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"artifact_bytes": 718383,
|
||||||
|
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
|
||||||
|
"package_bytes": 398715,
|
||||||
|
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
|
||||||
|
"plan_bytes": 398158,
|
||||||
|
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"portable_graph_full_render": {
|
||||||
|
"max_ms": 323.69,
|
||||||
|
"maximum_response_bytes": 657,
|
||||||
|
"maximum_traced_peak_bytes": 3490976,
|
||||||
|
"median_ms": 303.736,
|
||||||
|
"min_ms": 301.215,
|
||||||
|
"p95_limit_ms": 10000,
|
||||||
|
"p95_ms": 323.69,
|
||||||
|
"response_limit_bytes": 128000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"artifact_bytes": 718383,
|
||||||
|
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
|
||||||
|
"package_bytes": 398715,
|
||||||
|
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
|
||||||
|
"plan_bytes": 398158,
|
||||||
|
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b"
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
},
|
||||||
|
"portable_graph_status_no_work": {
|
||||||
|
"max_ms": 59.331,
|
||||||
|
"maximum_response_bytes": 879,
|
||||||
|
"maximum_traced_peak_bytes": 685012,
|
||||||
|
"median_ms": 58.467,
|
||||||
|
"min_ms": 58.052,
|
||||||
|
"p95_limit_ms": 500,
|
||||||
|
"p95_ms": 59.331,
|
||||||
|
"response_limit_bytes": 256000,
|
||||||
|
"samples": 10,
|
||||||
|
"stable_result": {
|
||||||
|
"counters": {
|
||||||
|
"adapter_projection_loads": 0,
|
||||||
|
"adapter_source_extractions": 0,
|
||||||
|
"index_builds": 0,
|
||||||
|
"index_checks": 0,
|
||||||
|
"index_synchronizations": 0,
|
||||||
|
"project_loads": 0,
|
||||||
|
"render_output_bytes_built": 0,
|
||||||
|
"render_output_bytes_hashed": 0,
|
||||||
|
"render_prepare_calls": 0,
|
||||||
|
"source_bytes_parsed": 0,
|
||||||
|
"source_files_parsed": 0,
|
||||||
|
"source_generation_checks": 2,
|
||||||
|
"viewer_manager_requests": 0
|
||||||
|
},
|
||||||
|
"response": {
|
||||||
|
"adapter": "generic",
|
||||||
|
"configured": true,
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"artifact": {
|
||||||
|
"artifact_id": "portable-graph.html",
|
||||||
|
"bytes": 718383,
|
||||||
|
"media_type": "text/html; charset=utf-8",
|
||||||
|
"sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30"
|
||||||
|
},
|
||||||
|
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
|
||||||
|
"path": ".docforge/portable-graph/architecture.html",
|
||||||
|
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b",
|
||||||
|
"publication_id": "2620ce106593d5499931b145a42fc1eccd31f5e356b19bf1d76687f8d45bcad2",
|
||||||
|
"reason": null,
|
||||||
|
"renderer": "portable_graph_html",
|
||||||
|
"renderer_version": "1",
|
||||||
|
"state": "current",
|
||||||
|
"verification": "manifest",
|
||||||
|
"view_id": "architecture"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"project_id": "synthetic-1000",
|
||||||
|
"project_root_fingerprint": "0747acfd975703c0",
|
||||||
|
"revision": "unversioned",
|
||||||
|
"source_hash": "a314da4ffa8fcf291ef7a7b0fc87737caeaa89fa3c558ea442afeb0e5ae49c2d",
|
||||||
|
"state": "current",
|
||||||
|
"status": "ok"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"traced_peak_limit_bytes": 268435456
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"schema_version": 1,
|
||||||
|
"sizes": {
|
||||||
|
"fragment_assisted_manual": {
|
||||||
|
"aggregate_fragment_content_bytes": 516921,
|
||||||
|
"artifact_bytes": 583149,
|
||||||
|
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"fragment_count": 1000,
|
||||||
|
"package_bytes": 2364302,
|
||||||
|
"package_id": "ecc21e58106c420a9f78ffbba997778ca98b61586ecfdb23a15b08f431a52c42",
|
||||||
|
"plan_bytes": 1006393,
|
||||||
|
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
|
||||||
|
"receipt_bytes": 664
|
||||||
|
},
|
||||||
|
"manual": {
|
||||||
|
"artifact_bytes": 583149,
|
||||||
|
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||||
|
"package_bytes": 1007297,
|
||||||
|
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
|
||||||
|
"plan_bytes": 1006393,
|
||||||
|
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
|
||||||
|
"receipt_bytes": 664
|
||||||
|
},
|
||||||
|
"manual_status_response_bytes": 1542,
|
||||||
|
"portable_graph": {
|
||||||
|
"artifact_bytes": 718383,
|
||||||
|
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
|
||||||
|
"package_bytes": 398715,
|
||||||
|
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
|
||||||
|
"plan_bytes": 398158,
|
||||||
|
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b",
|
||||||
|
"receipt_bytes": 657
|
||||||
|
},
|
||||||
|
"portable_graph_status_response_bytes": 879
|
||||||
|
},
|
||||||
|
"source": {
|
||||||
|
"dirty": false,
|
||||||
|
"revision": "f5dccb5e1c312121f1af63780162f593d9363b98"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -71,6 +71,19 @@ Existing hand-written client configurations remain valid and are never rewritten
|
||||||
Doctor is inspection-only and does not become a hidden bootstrap, synchronization, or migration
|
Doctor is inspection-only and does not become a hidden bootstrap, synchronization, or migration
|
||||||
path.
|
path.
|
||||||
|
|
||||||
|
The following Milestone 3 CLI additions are also additive:
|
||||||
|
|
||||||
|
- `docforge graph-plan VIEW_ID`
|
||||||
|
- `docforge graph-render VIEW_ID`
|
||||||
|
- `docforge graph-render-status [VIEW_ID]`
|
||||||
|
- `--manual-render-policy auto|explicit|disabled`
|
||||||
|
- `--portable-graph-policy explicit|disabled`
|
||||||
|
- `--live-viewer-policy on-demand|disabled`
|
||||||
|
|
||||||
|
MCP adds the read-only `docforge_graph_plan` and `docforge_graph_render_status` tools. Portable
|
||||||
|
graph publication remains an explicit local CLI integration action. Existing manual render,
|
||||||
|
preview, visualization, and status names remain supported.
|
||||||
|
|
||||||
## Versioned data contracts
|
## Versioned data contracts
|
||||||
|
|
||||||
Milestone 0 preserves:
|
Milestone 0 preserves:
|
||||||
|
|
@ -96,6 +109,11 @@ Milestone 0 preserves:
|
||||||
- Latest-generation-diff page schema version 1. Pages use one top-level pagination object and a
|
- Latest-generation-diff page schema version 1. Pages use one top-level pagination object and a
|
||||||
nested `receipt_header`. `stored_receipt_hash` names the complete stored receipt. Opaque cursors
|
nested `receipt_header`. `stored_receipt_hash` names the complete stored receipt. Opaque cursors
|
||||||
may be restarted after a server or receipt change and are not durable public identifiers.
|
may be restarted after a server or receipt change and are not durable public identifiers.
|
||||||
|
- Manual render-plan schema version 1.
|
||||||
|
- Graph view-plan schema version 1.
|
||||||
|
- Projection-package schema version 1.
|
||||||
|
- Projection-receipt schema version 1.
|
||||||
|
- Independent projection-policy schema version 2. Effective policy version 1 remains frozen.
|
||||||
|
|
||||||
Indexes, attestations, extraction caches, previews, and rendered artifacts are disposable. A schema
|
Indexes, attestations, extraction caches, previews, and rendered artifacts are disposable. A schema
|
||||||
change may rebuild them. Canonical project content and stored proposals may not be silently
|
change may rebuild them. Canonical project content and stored proposals may not be silently
|
||||||
|
|
@ -161,12 +179,24 @@ The following guarantees remain stable:
|
||||||
|
|
||||||
## Rendering and visualization
|
## Rendering and visualization
|
||||||
|
|
||||||
The `generic_html` renderer remains the supported version-1 manual projection. It retains confined
|
The `generic_html` renderer remains the supported version-1 manual projection. Its public
|
||||||
paths, raw-HTML suppression, fixed template tokens, deterministic identities, atomic replacement,
|
`GenericHtmlRenderer.prepare()` signature, renderer identity, frozen alpha bytes, confined paths,
|
||||||
and side-effect-free status.
|
raw-HTML suppression, fixed template tokens, deterministic identities, atomic replacement, and
|
||||||
|
side-effect-free status remain compatible. It now delegates through a versioned manual plan,
|
||||||
|
immutable package, and independent renderer.
|
||||||
|
|
||||||
|
The `portable_graph_html` renderer and `graph_render` descriptor table are additive. Manual and
|
||||||
|
portable graph declarations, plans, policies, publication receipts, and status remain separate.
|
||||||
|
The portable renderer does not replace the existing live viewer or `docforge_visualize`.
|
||||||
|
|
||||||
|
Existing project descriptors may retain any positive `max_render_bytes` accepted by schema version
|
||||||
|
1. A value above 20,000,000 bytes does not make the descriptor invalid, and a smaller actual
|
||||||
|
artifact still renders. Actual detached worker transfer is a separate fixed 20,000,000-byte
|
||||||
|
runtime boundary.
|
||||||
|
|
||||||
The live graph viewer remains a read-only consumer of a generation-pinned validated index. It does
|
The live graph viewer remains a read-only consumer of a generation-pinned validated index. It does
|
||||||
not become project authority or MCP retrieval authority.
|
not become project authority or MCP retrieval authority. Source reads use the pinned index
|
||||||
|
generation instead of reopening mutable canonical files behind that generation.
|
||||||
|
|
||||||
## Task-context compatibility
|
## Task-context compatibility
|
||||||
|
|
||||||
|
|
@ -199,8 +229,9 @@ The exact version-1 relation aliases are frozen by the MCP contract and reposito
|
||||||
Changing an alias category requires a new planner version; it is not a silent implementation
|
Changing an alias category requires a new planner version; it is not a silent implementation
|
||||||
detail.
|
detail.
|
||||||
|
|
||||||
`ManualRenderPlan`, `GraphViewPlan`, a portable graph renderer, and independently packaged
|
Milestone 3 adds `ManualRenderPlanV1`, `GraphViewPlanV1`, projection package and receipt version 1,
|
||||||
renderers are later-milestone direction. Milestone 0 does not claim that those contracts exist.
|
and projection policy version 2. These are additive submodule and schema contracts. They do not
|
||||||
|
change the legacy task-context, adapter, changeset, or effective-policy contracts described above.
|
||||||
|
|
||||||
## Safety boundary
|
## Safety boundary
|
||||||
|
|
||||||
|
|
@ -221,6 +252,8 @@ Milestone 0 records rather than redesigns these areas:
|
||||||
callers use targeted retrieval for that node.
|
callers use targeted retrieval for that node.
|
||||||
- One individually oversized changeset diff is transported as reconstructable canonical-JSON
|
- One individually oversized changeset diff is transported as reconstructable canonical-JSON
|
||||||
chunks. Cursors are corruption-detecting read tokens, not authenticated authorization tokens.
|
chunks. Cursors are corruption-detecting read tokens, not authenticated authorization tokens.
|
||||||
- Manual planning is not separated from rendering.
|
- Production fragment validation is currently slower than forced-full rendering at the maintained
|
||||||
- There is no portable graph-planning or graph-rendering contract.
|
1,000-page fixture. Full rendering remains the equivalence and recovery oracle.
|
||||||
|
- Remote render services, render farms, third-party renderer ecosystems, and a separate render MCP
|
||||||
|
remain deferred.
|
||||||
- DocForge2 does not self-host its bootstrap documentation.
|
- DocForge2 does not self-host its bootstrap documentation.
|
||||||
|
|
|
||||||
|
|
@ -24,6 +24,11 @@ commit when Git is available; it cannot change repository state.
|
||||||
- Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1.
|
- Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1.
|
||||||
- Generated client configuration: `schemas/client-configuration.schema.json`, version 1.
|
- Generated client configuration: `schemas/client-configuration.schema.json`, version 1.
|
||||||
- Client doctor result: `schemas/doctor-result.schema.json`, version 1.
|
- Client doctor result: `schemas/doctor-result.schema.json`, version 1.
|
||||||
|
- Manual render plan: `schemas/manual-render-plan.schema.json`, version 1.
|
||||||
|
- Graph view plan: `schemas/graph-view-plan.schema.json`, version 1.
|
||||||
|
- Projection package: `schemas/projection-package.schema.json`, version 1.
|
||||||
|
- Projection receipt: `schemas/projection-receipt.schema.json`, version 1.
|
||||||
|
- Independent projection policy: `schemas/projection-policy.schema.json`, version 2.
|
||||||
- Index schema: version 3, disposable and reproducible.
|
- Index schema: version 3, disposable and reproducible.
|
||||||
- Index attestation: schema version 1, disposable and reproducible.
|
- Index attestation: schema version 1, disposable and reproducible.
|
||||||
- Core, CLI, and MCP server: version 1.3.0.dev0.
|
- Core, CLI, and MCP server: version 1.3.0.dev0.
|
||||||
|
|
@ -82,7 +87,9 @@ the header alone. One top-level pagination object carries the only continuation
|
||||||
Generated Codex, Claude, and OpenClaw fragments are machine-local projections. They are not
|
Generated Codex, Claude, and OpenClaw fragments are machine-local projections. They are not
|
||||||
canonical project content. Version 1 binds the selected project, exact isolated Python
|
canonical project content. Version 1 binds the selected project, exact isolated Python
|
||||||
interpreter, canonical argument layout, effective policy, no-AST projection, render policy,
|
interpreter, canonical argument layout, effective policy, no-AST projection, render policy,
|
||||||
timeouts, artifact bytes, and configuration hash.
|
timeouts, artifact bytes, and configuration hash. Milestone 3 adds the version-2 projection policy,
|
||||||
|
its hash, projection availability, and the exact descriptor hash to that attested configuration
|
||||||
|
evidence. Omitted default selectors are recomposed against the bound descriptor.
|
||||||
|
|
||||||
Preview is side-effect free. Explicit publication creates only one new private standalone
|
Preview is side-effect free. Explicit publication creates only one new private standalone
|
||||||
fragment in an existing real directory. It never merges or replaces different content. Descriptor,
|
fragment in an existing real directory. It never merges or replaces different content. Descriptor,
|
||||||
|
|
@ -135,10 +142,12 @@ preview root, and one or more stable view IDs. Each view names a built-in render
|
||||||
derived output file, title, and optional family filter. Paths are resolved under the project root
|
derived output file, title, and optional family filter. Paths are resolved under the project root
|
||||||
and may not overlap canonical content, authority files, changesets, templates, or previews.
|
and may not overlap canonical content, authority files, changesets, templates, or previews.
|
||||||
|
|
||||||
The initial `generic_html` renderer uses pinned CommonMark parsing with raw HTML disabled. Templates
|
The `generic_html` renderer uses pinned CommonMark parsing with raw HTML disabled. Templates are
|
||||||
are UTF-8 files with a fixed token vocabulary; they cannot name commands, modules, or executable
|
UTF-8 files with a fixed token vocabulary; they cannot name commands, modules, or executable
|
||||||
renderers. Render identity covers the canonical source hash, optional changeset hash, selected node
|
renderers. Render identity covers the canonical source hash, optional changeset hash, selected node
|
||||||
and edge identities, view configuration, template hash, renderer contract, and exact parser version.
|
and edge identities, view configuration, template hash, renderer contract, and exact parser
|
||||||
|
version. The frozen version-1 API and alpha bytes are preserved by a compatibility wrapper over the
|
||||||
|
manual plan/package/renderer path.
|
||||||
|
|
||||||
An explicit CLI render atomically replaces one declared derived output. MCP can render a validated
|
An explicit CLI render atomically replaces one declared derived output. MCP can render a validated
|
||||||
changeset only to its isolated preview path. Normal status verifies bounded source, configuration,
|
changeset only to its isolated preview path. Normal status verifies bounded source, configuration,
|
||||||
|
|
@ -146,6 +155,56 @@ template, output, renderer, and publication-receipt identities without reconstru
|
||||||
Explicit deep status remains the side-effect-free full-render oracle. Input changes detected before
|
Explicit deep status remains the side-effect-free full-render oracle. Input changes detected before
|
||||||
atomic replacement fail without publishing a current receipt for stale output.
|
atomic replacement fail without publishing a current receipt for stale output.
|
||||||
|
|
||||||
|
## Independent projection boundary
|
||||||
|
|
||||||
|
Manual and portable graph plans are separate version-1 contracts over one immutable validated
|
||||||
|
generation. They use canonical JSON, deterministic ordering, fixed structural and serialized-size
|
||||||
|
bounds, and content-derived identities. Plans contain selected graph facts and bounded content.
|
||||||
|
They contain no live project object, database handle, absolute project or index path, arbitrary
|
||||||
|
query, command, executable path, or caller-selected module.
|
||||||
|
|
||||||
|
Projection packages bind one plan to inert assets, a closed built-in renderer identity, declared
|
||||||
|
component versions, and an artifact inventory with a byte allowance. Receipts bind the exact
|
||||||
|
package, plan, renderer, artifact hashes and sizes, diagnostics, timing, and detached peak memory.
|
||||||
|
Manual and graph renderer modules accept only validated packages. They cannot select nodes, invent
|
||||||
|
relationships, read project state, choose publication paths, or write canonical files.
|
||||||
|
|
||||||
|
Detached execution uses one fixed private Python module, isolated mode, a trusted working
|
||||||
|
directory, a sanitized environment, exactly one canonical newline-terminated JSON request and
|
||||||
|
response, a closed renderer allowlist, a 30-second timeout, disk-spooled stdout, and bounded reads. The package
|
||||||
|
contract is capped at 24,000,000 bytes and actual detached artifact transfer at 20,000,000 bytes.
|
||||||
|
Project descriptors may retain a larger `max_render_bytes` compatibility allowance, but an actual
|
||||||
|
detached transfer above the fixed worker boundary fails closed.
|
||||||
|
|
||||||
|
Portable graph configuration is independent of manual render configuration. One view selects
|
||||||
|
either an exact root or a bounded metadata-only lexical query plus closed filters and node, edge,
|
||||||
|
depth, and work limits. Logic is excluded. The renderer emits a complete static Nodes, Flow, or Web
|
||||||
|
artifact and uses JavaScript only as progressive enhancement.
|
||||||
|
|
||||||
|
Portable publication commits a content-addressed artifact, renderer receipt, and one bounded
|
||||||
|
generation/view manifest in that order. The manifest is the publication commit. Status reads only
|
||||||
|
bounded manifest and receipt evidence and never plans or renders. Repair restores declared output
|
||||||
|
only from validated content-addressed evidence. A failure after a replacement that cannot be
|
||||||
|
proven rolled back returns explicit degraded committed evidence.
|
||||||
|
|
||||||
|
Manual fragment records are disposable semantic cache entries. Their keys bind the renderer,
|
||||||
|
component version, and complete page semantics. The detached renderer recomputes the expected page
|
||||||
|
fragment before accepting cached bytes. Cold creation is compared with a full detached render
|
||||||
|
before cache publication. Invalid, corrupt, forged, stale, individually oversized, or
|
||||||
|
aggregate-oversized records fall back to the full oracle. The dedicated cache retains only current
|
||||||
|
keys and is capped at 10,000 entries and 64,000,000 bytes.
|
||||||
|
|
||||||
|
Projection policy version 2 composes manual `auto|explicit|disabled`, portable graph
|
||||||
|
`explicit|disabled`, and live viewer `on-demand|disabled` independently. Active plan, render,
|
||||||
|
application, onboarding, and viewer-start operations enforce the relevant policy before hidden
|
||||||
|
work. Receipt-only status and explicit viewer stop remain available. Effective policy version 1
|
||||||
|
and its legacy projections remain unchanged.
|
||||||
|
|
||||||
|
The live viewer remains separate from portable graph publication. It consumes one
|
||||||
|
generation-pinned validated index through the viewer manager. Source reads come from that pinned
|
||||||
|
generation and do not reopen mutable canonical files behind an older snapshot. Neither live nor
|
||||||
|
portable visualization is retrieval or canonical authority.
|
||||||
|
|
||||||
Normal MCP access does not expose canonical application. An explicitly configured canonical
|
Normal MCP access does not expose canonical application. An explicitly configured canonical
|
||||||
applier registers one hash-bound application tool. No MCP mode exposes arbitrary renderer
|
applier registers one hash-bound application tool. No MCP mode exposes arbitrary renderer
|
||||||
execution, arbitrary file writes, shell commands, Git mutation, build commands, deployment, or
|
execution, arbitrary file writes, shell commands, Git mutation, build commands, deployment, or
|
||||||
|
|
|
||||||
90
docs/MILESTONE_3_BASELINE.md
Normal file
90
docs/MILESTONE_3_BASELINE.md
Normal file
|
|
@ -0,0 +1,90 @@
|
||||||
|
# Milestone 3 baseline
|
||||||
|
|
||||||
|
## Scope and method
|
||||||
|
|
||||||
|
This baseline records the independent-projection behavior completed in Milestone 3. It was
|
||||||
|
captured on 2026-07-29 from clean candidate commit
|
||||||
|
`f5dccb5e1c312121f1af63780162f593d9363b98`.
|
||||||
|
|
||||||
|
The maintained command was:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
.venv/bin/python tools/milestone3_benchmark.py \
|
||||||
|
--mode full \
|
||||||
|
--nodes 1000 \
|
||||||
|
--samples 10 \
|
||||||
|
--output benchmarks/milestone3-2026-07-29.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The synthetic generic fixture contains 1,000 manual pages, 1,000 portable-graph nodes, and 999
|
||||||
|
edges. Durations use `time.perf_counter_ns()` and nearest-rank p95. In-process peak memory uses
|
||||||
|
`tracemalloc`; detached worker peak memory comes from the worker receipt and `RUSAGE_SELF`.
|
||||||
|
Every measured result is checked for deterministic semantic identity and bounded response size.
|
||||||
|
|
||||||
|
Environment:
|
||||||
|
|
||||||
|
- Linux 7.1.3-200.nobara.fc44.x86_64.
|
||||||
|
- CPython 3.14.6.
|
||||||
|
- x86_64.
|
||||||
|
- Ten samples except the one-time production cold render and fragment-cache population.
|
||||||
|
- In-process and detached-worker memory ceiling: 268,435,456 bytes.
|
||||||
|
- Detached artifact-transfer ceiling: 20,000,000 bytes.
|
||||||
|
|
||||||
|
The complete machine-readable result is
|
||||||
|
[`benchmarks/milestone3-2026-07-29.json`](../benchmarks/milestone3-2026-07-29.json).
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
| Operation | Median | p95 | Limit | Maximum response |
|
||||||
|
|---|---:|---:|---:|---:|
|
||||||
|
| Manual full plan/package/render | 801.948 ms | 810.490 ms | 15,000 ms | 664 B |
|
||||||
|
| Manual detached worker | 591.317 ms | 599.394 ms | 20,000 ms | 667 B |
|
||||||
|
| Fragment-assisted equivalence | 638.999 ms | 666.119 ms | 15,000 ms | 664 B |
|
||||||
|
| Production incremental cold | 2,827.152 ms | 2,827.152 ms | 20,000 ms | 668 B |
|
||||||
|
| Production incremental warm | 2,143.388 ms | 2,206.540 ms | 20,000 ms | 669 B |
|
||||||
|
| Production forced full | 944.135 ms | 978.870 ms | 20,000 ms | 668 B |
|
||||||
|
| Fragment cache miss sweep | 106.203 ms | 106.888 ms | 5,000 ms | 145 B |
|
||||||
|
| Fragment cache hit sweep | 498.911 ms | 514.324 ms | 5,000 ms | 145 B |
|
||||||
|
| Portable graph full plan/package/render | 303.736 ms | 323.690 ms | 10,000 ms | 657 B |
|
||||||
|
| Portable graph detached worker | 265.448 ms | 268.428 ms | 20,000 ms | 660 B |
|
||||||
|
| Manual receipt-only status | 62.881 ms | 111.381 ms | 500 ms | 1,542 B |
|
||||||
|
| Portable graph receipt-only status | 58.467 ms | 59.331 ms | 500 ms | 879 B |
|
||||||
|
|
||||||
|
The largest traced in-process peak was 35,160,716 bytes. The direct manual worker track peaked at
|
||||||
|
88,580,096 bytes and the portable graph worker at 89,583,616 bytes. The production manual paths,
|
||||||
|
including cold, warm, forced-full, and mutation variants, peaked at 104,771,584 bytes. Every child
|
||||||
|
peak was validated from its projection receipt against the 268,435,456-byte gate.
|
||||||
|
|
||||||
|
The manual artifact was 583,149 bytes. The portable graph artifact was 718,383 bytes. The manual
|
||||||
|
plan was 1,006,393 bytes and its ordinary package was 1,007,297 bytes. The graph plan was 398,158
|
||||||
|
bytes and its package was 398,715 bytes.
|
||||||
|
|
||||||
|
## Equivalence and no-work gates
|
||||||
|
|
||||||
|
The benchmark proved exact output equivalence for:
|
||||||
|
|
||||||
|
- Manual in-process and detached rendering.
|
||||||
|
- Manual full and fragment-assisted rendering.
|
||||||
|
- Production cold, warm, and forced-full rendering.
|
||||||
|
- Production add, change, delete, and reorder variants.
|
||||||
|
- Portable graph in-process and detached rendering.
|
||||||
|
|
||||||
|
Manual and portable-graph status each performed zero project loads, source parses, adapter
|
||||||
|
projection loads, adapter extraction, index checks, synchronization, index builds, render
|
||||||
|
preparation, output construction, output hashing, and viewer-manager requests. Each status path
|
||||||
|
performed only two cheap source-generation checks and verified committed receipt or manifest
|
||||||
|
evidence.
|
||||||
|
|
||||||
|
## Measured limits and future notes
|
||||||
|
|
||||||
|
- Fragment reuse is a correctness, isolation, and recovery boundary in this milestone. At 1,000
|
||||||
|
pages, production warm fragment validation is slower than the forced-full path. Later
|
||||||
|
optimization must start from this measurement and preserve byte equivalence.
|
||||||
|
- Full rendering remains the oracle and recovery path. Invalid, corrupt, oversized, stale, or
|
||||||
|
mismatched fragment records fall back without changing canonical facts.
|
||||||
|
- The benchmark main-process `ru_maxrss` value was 102,692 KiB. It is cumulative across all
|
||||||
|
main-process operations and is recorded only as diagnostic context. Detached child peaks are
|
||||||
|
measured separately. Per-operation traced peaks and every detached receipt peak own the memory
|
||||||
|
gates.
|
||||||
|
- The results do not justify a storage rewrite, render farm, remote renderer, or separate render
|
||||||
|
MCP.
|
||||||
89
docs/MILESTONE_3_CLOSEOUT.md
Normal file
89
docs/MILESTONE_3_CLOSEOUT.md
Normal file
|
|
@ -0,0 +1,89 @@
|
||||||
|
# Milestone 3 closeout
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Milestone 3 is complete. Manual compilation, portable graph rendering, and the live viewer are
|
||||||
|
separate generation-pinned consumers of the validated graph. They cannot become canonical or
|
||||||
|
retrieval authority.
|
||||||
|
|
||||||
|
Implemented contracts:
|
||||||
|
|
||||||
|
- Version-1 `ManualRenderPlan`, `GraphViewPlan`, projection package, and projection receipt.
|
||||||
|
- Strict canonical JSON identities and packaged Draft 2020-12 schemas.
|
||||||
|
- Independent manual and portable-graph renderer import boundaries.
|
||||||
|
- One isolated, fixed, one-request detached worker protocol with bounded request, response,
|
||||||
|
artifact, timeout, environment, and renderer inventory.
|
||||||
|
- Content-addressed portable graph artifacts, renderer receipts, generation/view manifests,
|
||||||
|
receipt-only status, repair, and degraded committed-publication evidence.
|
||||||
|
- Disposable semantic fragment records with bounded cache inventory, corruption recovery, and
|
||||||
|
full-render equivalence.
|
||||||
|
- Version-2 independent projection policy while preserving version-1 effective-policy behavior.
|
||||||
|
- Generation-pinned live source reads and a separate read-only viewer-manager lifecycle.
|
||||||
|
- Automated axe-tag and keyboard gates for the manual, portable graph, and live viewer.
|
||||||
|
- Repository-native contract, smoke, scale, response-size, memory, and equivalence gates.
|
||||||
|
|
||||||
|
## Candidate evidence
|
||||||
|
|
||||||
|
The frozen implementation candidate is
|
||||||
|
`f5dccb5e1c312121f1af63780162f593d9363b98`.
|
||||||
|
|
||||||
|
The complete repository gate passed:
|
||||||
|
|
||||||
|
- Ruff formatting and lint.
|
||||||
|
- HTML, rendered-manual HTML, portable-graph HTML, CSS, and JavaScript checks.
|
||||||
|
- Pyright with zero diagnostics.
|
||||||
|
- Warning-strict compilation and tests.
|
||||||
|
- 281 tests and 272 subtests.
|
||||||
|
- Three Playwright and axe accessibility flows. The alpha manual is checked with WCAG 2.0/2.1
|
||||||
|
A/AA axe tags; portable and live graph flows add WCAG 2.2 A/AA tags and keyboard interaction.
|
||||||
|
- Lock and npm dependency-tree checks.
|
||||||
|
- Wheel and source-distribution builds.
|
||||||
|
- Milestone 0, 1, 2, and 3 smoke benchmarks.
|
||||||
|
|
||||||
|
The maintained projection contract subset passed 142 tests and 236 subtests. A 10,000-node deep
|
||||||
|
chain and one 10,000-node strongly connected component prove that manual cycle planning has no
|
||||||
|
recursion-depth failure.
|
||||||
|
|
||||||
|
An isolated wheel installation passed CLI and MCP startup, a real detached manual render, and the
|
||||||
|
closed malformed-worker-request contract. Six Milestone 3 commits and the complete candidate tree
|
||||||
|
passed Gitleaks 8.30.1 with no findings.
|
||||||
|
|
||||||
|
Three independent adversarial review tracks covered manual isolation and fragment integrity,
|
||||||
|
portable publication and policy binding, and worker/accessibility/benchmark gates. Reproduced
|
||||||
|
project import, hostile environment, unbounded stdout, fragment forgery, cache growth, aggregate
|
||||||
|
overflow, coordinated policy drift, render-limit compatibility, deep-graph, and module-startup
|
||||||
|
defects were fixed and regression-tested before closeout.
|
||||||
|
|
||||||
|
The clean ten-sample 1,000-node benchmark passed every threshold. Exact measurements, equivalence
|
||||||
|
results, memory peaks, and response sizes are recorded in
|
||||||
|
[`MILESTONE_3_BASELINE.md`](MILESTONE_3_BASELINE.md) and
|
||||||
|
[`benchmarks/milestone3-2026-07-29.json`](../benchmarks/milestone3-2026-07-29.json).
|
||||||
|
|
||||||
|
## Preserved boundaries
|
||||||
|
|
||||||
|
- The `docforge` distribution, package, CLI, MCP executable, and existing tool names remain.
|
||||||
|
- The frozen alpha manual remains exactly 2,043 bytes with its legacy output hash and render
|
||||||
|
identity.
|
||||||
|
- Legacy one-method `load_projection()` adapters remain supported.
|
||||||
|
- Effective policy version 1, no-AST behavior, and existing client bindings remain compatible.
|
||||||
|
- Project descriptor schema version 1 and SQLite index schema version 3 remain unchanged.
|
||||||
|
- Configured `max_render_bytes` values above the detached transfer ceiling still load; a small
|
||||||
|
actual artifact renders normally. Actual detached transfer remains capped at 20,000,000 bytes.
|
||||||
|
- No storage replacement or self-hosting dependency was introduced.
|
||||||
|
- WorldForge and ScrapeStation were not touched.
|
||||||
|
- No production MCP integration was repointed.
|
||||||
|
- The legacy Forgejo repository and `legacy` remote were not changed.
|
||||||
|
- No tag, release, release announcement, or visibility change was created.
|
||||||
|
|
||||||
|
## Known follow-up work
|
||||||
|
|
||||||
|
Milestone 4 remains directional and is not active. Its adapter SDK and product-documentation work
|
||||||
|
must not silently absorb these separate future ideas:
|
||||||
|
|
||||||
|
- Optimize production fragment reuse only from measured profiles while preserving the forced-full
|
||||||
|
oracle.
|
||||||
|
- Add authenticated cursors only if a stronger threat model requires them.
|
||||||
|
- Verify Claude's native timeout representation.
|
||||||
|
- Add versioned adapter-owned launcher metadata before generating custom-adapter configurations.
|
||||||
|
- Keep remote render services, shared render farms, third-party renderers, storage replacement,
|
||||||
|
and self-hosting deferred until their own evidence justifies them.
|
||||||
|
|
@ -27,6 +27,10 @@ incremental methods while retaining the full loader as a fallback.
|
||||||
- Opt-in incremental adapter extraction with reverse-dependency invalidation.
|
- Opt-in incremental adapter extraction with reverse-dependency invalidation.
|
||||||
- Lazy function-scoped logic projections that do not densify the primary graph.
|
- Lazy function-scoped logic projections that do not densify the primary graph.
|
||||||
- Declared HTML render views. Arbitrary templates, render commands, and output paths are rejected.
|
- Declared HTML render views. Arbitrary templates, render commands, and output paths are rejected.
|
||||||
|
- Separate versioned manual and portable graph plans, immutable packages, detached built-in
|
||||||
|
renderers, and validated receipts.
|
||||||
|
- Content-addressed portable Nodes/Flow/Web artifacts with receipt-only status and repair.
|
||||||
|
- Independent manual, portable-graph, and live-viewer policy.
|
||||||
- A loopback-only graph browser with Nodes, semantic Flow, and convergence Web views,
|
- A loopback-only graph browser with Nodes, semantic Flow, and convergence Web views,
|
||||||
relationship keys, source inspection, branch-aware node hiding, panel resizing, zooming, and
|
relationship keys, source inspection, branch-aware node hiding, panel resizing, zooming, and
|
||||||
managed idle shutdown.
|
managed idle shutdown.
|
||||||
|
|
@ -47,7 +51,11 @@ Canonical files own facts:
|
||||||
canonical Markdown/TOML or adapter sources
|
canonical Markdown/TOML or adapter sources
|
||||||
↓ validate
|
↓ validate
|
||||||
disposable SQLite graph
|
disposable SQLite graph
|
||||||
↓ query / visualize / compile context
|
├── query / compile context
|
||||||
|
├── ManualRenderPlanV1 → detached manual renderer → declared manual
|
||||||
|
├── GraphViewPlanV1 → detached graph renderer → portable Nodes/Flow/Web artifact
|
||||||
|
└── pinned index → managed live Nodes/Flow/Web/Logic viewer
|
||||||
|
↓
|
||||||
people and agents
|
people and agents
|
||||||
↓ propose
|
↓ propose
|
||||||
isolated changeset + preview
|
isolated changeset + preview
|
||||||
|
|
@ -77,8 +85,8 @@ source format.
|
||||||
Clone and verify DocForge:
|
Clone and verify DocForge:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone forgejo@repo.andraxion.net:administrator/DocForge.git /absolute/path/DocForge
|
git clone forgejo@repo.andraxion.net:administrator/DocForge2.git /absolute/path/DocForge2
|
||||||
cd /absolute/path/DocForge
|
cd /absolute/path/DocForge2
|
||||||
uv sync --group dev
|
uv sync --group dev
|
||||||
npm ci
|
npm ci
|
||||||
|
|
||||||
|
|
@ -159,6 +167,27 @@ output = "Docs/Rendered/Manual.html"
|
||||||
title = "My Project Manual"
|
title = "My Project Manual"
|
||||||
families = ["architecture", "system", "operations", "roadmap"]
|
families = ["architecture", "system", "operations", "roadmap"]
|
||||||
|
|
||||||
|
[graph_render]
|
||||||
|
output_root = ".docforge/portable-graph"
|
||||||
|
|
||||||
|
[[graph_render.views]]
|
||||||
|
id = "architecture"
|
||||||
|
renderer = "portable_graph_html"
|
||||||
|
output = "architecture.html"
|
||||||
|
title = "Architecture"
|
||||||
|
root = "architecture.overview"
|
||||||
|
initial_mode = "web"
|
||||||
|
depth = 3
|
||||||
|
max_nodes = 250
|
||||||
|
max_edges = 1000
|
||||||
|
max_work = 100000
|
||||||
|
families = ["architecture", "system"]
|
||||||
|
relations = ["depends_on", "owns", "calls", "reads", "writes", "tested_by", "relates_to"]
|
||||||
|
authorities = []
|
||||||
|
statuses = ["current", "active", "verified"]
|
||||||
|
tags = []
|
||||||
|
include_logic = false
|
||||||
|
|
||||||
[graph]
|
[graph]
|
||||||
allowed_relations = ["depends_on", "owns", "calls", "reads", "writes", "tested_by", "relates_to"]
|
allowed_relations = ["depends_on", "owns", "calls", "reads", "writes", "tested_by", "relates_to"]
|
||||||
|
|
||||||
|
|
@ -492,13 +521,25 @@ builds or repairs the index.
|
||||||
```text
|
```text
|
||||||
render-status [VIEW_ID] [--deep]
|
render-status [VIEW_ID] [--deep]
|
||||||
render VIEW_ID
|
render VIEW_ID
|
||||||
|
graph-plan VIEW_ID
|
||||||
|
graph-render VIEW_ID
|
||||||
|
graph-render-status [VIEW_ID]
|
||||||
preview CHANGESET_ID VIEW_ID
|
preview CHANGESET_ID VIEW_ID
|
||||||
apply CHANGESET_ID --changeset-hash SHA256 --applier WRITER_ID
|
apply CHANGESET_ID --changeset-hash SHA256 --applier WRITER_ID
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`graph-plan` validates and returns one declared `GraphViewPlanV1` without publishing. A portable
|
||||||
|
view must select exactly one stable `root` or metadata-only lexical `query`. It may use only Nodes,
|
||||||
|
Flow, or Web as `initial_mode`; portable version 1 excludes function-scoped Logic.
|
||||||
|
|
||||||
|
`graph-render` explicitly publishes the declared static artifact, content-addressed renderer
|
||||||
|
evidence, and generation/view manifest. `graph-render-status` verifies only bounded committed
|
||||||
|
evidence and never plans or renders. Portable publication is a local CLI action.
|
||||||
|
|
||||||
The CLI apply command supports the generic adapter. It verifies that the configured writer owns the
|
The CLI apply command supports the generic adapter. It verifies that the configured writer owns the
|
||||||
changeset, applies the exact reviewed hash, rebuilds the index, checks it, and regenerates every
|
changeset, applies the exact reviewed hash, rebuilds the index, checks it, and regenerates every
|
||||||
declared render. It does not commit or push the result.
|
declared manual render when manual policy is `auto`. It does not publish portable graphs, commit,
|
||||||
|
or push the result.
|
||||||
|
|
||||||
### Viewer commands
|
### Viewer commands
|
||||||
|
|
||||||
|
|
@ -549,6 +590,20 @@ declares the named writer, and application requires the same writer/applier iden
|
||||||
`--no-ast` to preserve the no-AST binding. Generic CLI generation refuses project-owned adapters
|
`--no-ast` to preserve the no-AST binding. Generic CLI generation refuses project-owned adapters
|
||||||
because it cannot safely reconstruct their composition.
|
because it cannot safely reconstruct their composition.
|
||||||
|
|
||||||
|
Select projection behavior independently:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docforge configure codex \
|
||||||
|
--project /absolute/path/MyProject \
|
||||||
|
--manual-render-policy explicit \
|
||||||
|
--portable-graph-policy disabled \
|
||||||
|
--live-viewer-policy on-demand
|
||||||
|
```
|
||||||
|
|
||||||
|
The generated version-1 configuration result carries an additive version-2 `projection_policy`,
|
||||||
|
its hash, projection availability, and the exact descriptor hash. Omitted default selectors are
|
||||||
|
validated against that descriptor rather than trusted as self-reported output.
|
||||||
|
|
||||||
Inspect one configured client binding:
|
Inspect one configured client binding:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -569,6 +624,54 @@ starts MCP, executes the configured command, synchronizes, builds, renders, star
|
||||||
writes configuration. Claude timeout representation and client filtering that cannot be proved
|
writes configuration. Claude timeout representation and client filtering that cannot be proved
|
||||||
locally remain explicit warnings.
|
locally remain explicit warnings.
|
||||||
|
|
||||||
|
## Independent projection behavior
|
||||||
|
|
||||||
|
Manual and portable graph renderers consume immutable, path-free packages. A package binds one
|
||||||
|
generation-pinned plan, inert assets, fixed component versions, a built-in renderer identity, and
|
||||||
|
an exact artifact inventory. The detached child cannot select nodes, open the project or index,
|
||||||
|
choose a publication path, execute project code, or mutate canonical facts.
|
||||||
|
|
||||||
|
Child startup is fixed to isolated Python, a private module entrypoint, a trusted working
|
||||||
|
directory, and a sanitized environment. One request and response use canonical newline-terminated
|
||||||
|
JSON. The request, response, receipt, execution time, and disk-spooled stdout are bounded. Actual
|
||||||
|
artifact transfer is capped at 20,000,000 bytes even when the descriptor retains a larger
|
||||||
|
`max_render_bytes` compatibility value.
|
||||||
|
|
||||||
|
Manual fragment records are disposable semantic cache entries. On a cold miss, DocForge performs a
|
||||||
|
trusted full detached render, extracts candidate page fragments, and compares fragment-assisted
|
||||||
|
output byte-for-byte before publishing records. On a warm hit, the worker recomputes each expected
|
||||||
|
page fragment before accepting cached bytes. Corrupt, forged, stale, individually oversized, or
|
||||||
|
aggregate-oversized records fall back to the full oracle. Fragment reuse is currently a correctness
|
||||||
|
and recovery boundary, not a promised speedup.
|
||||||
|
|
||||||
|
Projection policy version 2 is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
manual: auto | explicit | disabled
|
||||||
|
portable_graph: explicit | disabled
|
||||||
|
live_viewer: on-demand | disabled
|
||||||
|
```
|
||||||
|
|
||||||
|
For ordinary CLI commands, place the corresponding global flag before the subcommand:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docforge --project-root "$PROJECT" --manual-render-policy disabled render manual
|
||||||
|
docforge --project-root "$PROJECT" --portable-graph-policy disabled graph-plan architecture
|
||||||
|
docforge --project-root "$PROJECT" --live-viewer-policy disabled visualize
|
||||||
|
```
|
||||||
|
|
||||||
|
An active operation blocked by policy returns `projection_policy_forbids_operation` before hidden
|
||||||
|
work. Manual and portable receipt-only status remain available. Viewer status and explicit stop
|
||||||
|
remain available when viewer start is disabled.
|
||||||
|
|
||||||
|
A non-disabled projection also requires its declared configuration or runtime. Manual `explicit`
|
||||||
|
requires manual render configuration. Manual `auto` additionally requires canonical application in
|
||||||
|
the current operation or server capability. Portable graph `explicit` requires portable graph
|
||||||
|
render configuration, and live viewer `on-demand` requires its runtime. An unavailable selection
|
||||||
|
returns `projection_policy_unavailable` before work begins. In particular, ordinary CLI `render`
|
||||||
|
operations cannot select manual `auto`; use `explicit`, or let a configured canonical `apply`
|
||||||
|
operation own automatic regeneration.
|
||||||
|
|
||||||
## MCP usage
|
## MCP usage
|
||||||
|
|
||||||
Run one MCP server per project with absolute paths:
|
Run one MCP server per project with absolute paths:
|
||||||
|
|
@ -586,7 +689,10 @@ Select the session's declared surface explicitly when useful:
|
||||||
```bash
|
```bash
|
||||||
docforge-mcp \
|
docforge-mcp \
|
||||||
--project-root /absolute/path/MyProject \
|
--project-root /absolute/path/MyProject \
|
||||||
--capability-mode read
|
--capability-mode read \
|
||||||
|
--manual-render-policy explicit \
|
||||||
|
--portable-graph-policy explicit \
|
||||||
|
--live-viewer-policy on-demand
|
||||||
```
|
```
|
||||||
|
|
||||||
Supported modes are `read`, `proposal`, `application`, and `operator`. Existing startup defaults
|
Supported modes are `read`, `proposal`, `application`, and `operator`. Existing startup defaults
|
||||||
|
|
@ -613,8 +719,9 @@ identity, not a command. The changeset creator, configured writer, and canonical
|
||||||
|
|
||||||
Call `docforge_bootstrap` first. Its version-1 `session_contract` contains the fixed binding,
|
Call `docforge_bootstrap` first. Its version-1 `session_contract` contains the fixed binding,
|
||||||
current graph generation, effective policy, actual capabilities, render policies, prohibitions,
|
current graph generation, effective policy, actual capabilities, render policies, prohibitions,
|
||||||
and a recommended first operation. Workflow guidance does not recommend registration or
|
and a recommended first operation. The result also carries the independently composed version-2
|
||||||
application when those startup capabilities are unavailable.
|
`projection_policy` and hash. Workflow guidance does not recommend registration or application
|
||||||
|
when those startup capabilities are unavailable.
|
||||||
|
|
||||||
Example MCP client configuration:
|
Example MCP client configuration:
|
||||||
|
|
||||||
|
|
@ -653,11 +760,16 @@ Example MCP client configuration:
|
||||||
- `docforge_get_task_context`
|
- `docforge_get_task_context`
|
||||||
- `docforge_validate_project`
|
- `docforge_validate_project`
|
||||||
- `docforge_render_status`
|
- `docforge_render_status`
|
||||||
|
- `docforge_graph_plan`
|
||||||
|
- `docforge_graph_render_status`
|
||||||
- `docforge_visualize`
|
- `docforge_visualize`
|
||||||
- `docforge_visualization_status`
|
- `docforge_visualization_status`
|
||||||
- `docforge_stop_visualization`
|
- `docforge_stop_visualization`
|
||||||
- `docforge_get_generation_diff`
|
- `docforge_get_generation_diff`
|
||||||
|
|
||||||
|
MCP graph plan and status are read-only. MCP does not expose portable graph publication; use the
|
||||||
|
explicit local `graph-render` CLI command.
|
||||||
|
|
||||||
### Proposal tools
|
### Proposal tools
|
||||||
|
|
||||||
- `docforge_create_changeset`
|
- `docforge_create_changeset`
|
||||||
|
|
@ -976,8 +1088,29 @@ docforge --project-root "$PROJECT" render-status
|
||||||
docforge --project-root "$PROJECT" render VIEW_ID
|
docforge --project-root "$PROJECT" render VIEW_ID
|
||||||
```
|
```
|
||||||
|
|
||||||
Successful canonical apply regenerates all declared views automatically. A manual canonical edit
|
Successful canonical apply regenerates declared manual views only when manual policy is `auto`.
|
||||||
requires reindexing and rendering.
|
A manual canonical edit requires reindexing and explicit rendering. Portable graph publication
|
||||||
|
always remains a separate explicit CLI action.
|
||||||
|
|
||||||
|
Portable graph publication has separate status and policy:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docforge --project-root "$PROJECT" graph-render-status
|
||||||
|
docforge --project-root "$PROJECT" graph-render architecture
|
||||||
|
```
|
||||||
|
|
||||||
|
### `projection_policy_forbids_operation`
|
||||||
|
|
||||||
|
The process was deliberately started with the relevant manual, portable-graph, or live-viewer
|
||||||
|
operation disabled. Restart with an allowed selector after confirming that the integration should
|
||||||
|
receive that capability. Status and explicit stop operations remain available as described above.
|
||||||
|
|
||||||
|
### `projection_policy_unavailable`
|
||||||
|
|
||||||
|
The selected non-disabled projection has no matching project configuration or runtime. Add the
|
||||||
|
declared manual or portable graph render configuration, or make the live viewer runtime available,
|
||||||
|
before selecting that mode. Manual `auto` also requires an operation or MCP server with canonical
|
||||||
|
application enabled. Use manual `explicit` for a standalone CLI render.
|
||||||
|
|
||||||
### Descriptor changed after startup
|
### Descriptor changed after startup
|
||||||
|
|
||||||
|
|
@ -992,8 +1125,11 @@ Run the complete release gate from the DocForge repository:
|
||||||
make gate
|
make gate
|
||||||
```
|
```
|
||||||
|
|
||||||
Use `make benchmark` for the historical Milestone 0 baseline and `make benchmark-m1` for the
|
Use `make benchmark` for the historical Milestone 0 baseline, `make benchmark-m1` for the
|
||||||
counter-gated 1,000-node warm-operation benchmark.
|
counter-gated warm-operation benchmark, `make benchmark-m2` for agent workflow gates, and
|
||||||
|
`make benchmark-m3-full` for the ten-sample 1,000-node projection, worker, fragment, status,
|
||||||
|
equivalence, response-size, and memory gates. `make accessibility` runs the generated manual,
|
||||||
|
portable graph, and live viewer axe and keyboard flows.
|
||||||
|
|
||||||
Project-specific vocabulary, extraction rules, and serialization belong in the project adapter.
|
Project-specific vocabulary, extraction rules, and serialization belong in the project adapter.
|
||||||
Generic core behavior must remain deterministic, project-bound, and recoverable.
|
Generic core behavior must remain deterministic, project-bound, and recoverable.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue