Compare commits
6 commits
96e3965855
...
d6d9f47672
| Author | SHA1 | Date | |
|---|---|---|---|
| d6d9f47672 | |||
| f5dccb5e1c | |||
| d65b83a140 | |||
| 5b43c44dd7 | |||
| f1fabaf0ca | |||
| 1134c2d375 |
61 changed files with 8668 additions and 171 deletions
|
|
@ -1,16 +1,13 @@
|
|||
# Active milestone
|
||||
# Milestone state
|
||||
|
||||
```text
|
||||
Milestone: 3 — independent projections
|
||||
Goal: Make manual output, portable graph artifacts, and the live viewer independent generation-pinned consumers of the validated graph.
|
||||
In scope: Versioned ManualRenderPlan and GraphViewPlan; immutable projection packages and receipts; independent manual and graph renderers; projection policies; incremental fragments; equivalence, recovery, accessibility, response-size, performance, and memory gates.
|
||||
Out of scope: Adapter SDK expansion; remote render services; shared render farms; third-party renderer ecosystems; self-hosting; storage replacement; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
|
||||
Done when: Manual and graph plans are versioned and bounded; renderers cannot crawl project state or mutate canonical facts; portable and live graph modes remain separate; policies are enforced independently; status is receipt-only; full/incremental output is equivalent; accessibility and maintained scale gates pass.
|
||||
Status: Active implementation. Three independent audits were reconciled before source changes. The
|
||||
versioned plan/package/receipt contracts, pure manual and graph planners, isolated manual renderer,
|
||||
legacy byte-compatibility shim, packaged schemas, and pinned live-source correction are implemented
|
||||
and focused-green. Durable publication, portable graph artifacts, detached workers, fragment
|
||||
equivalence, independent policy enforcement, accessibility, and maintained scale gates remain.
|
||||
Last completed milestone: 3 — independent projections
|
||||
Outcome: Manual output, portable graph artifacts, and the live viewer are independent generation-pinned consumers of the validated graph.
|
||||
Evidence: Clean candidate f5dccb5e1c312121f1af63780162f593d9363b98; 281 tests and 272 subtests; 3 accessibility flows; clean 1,000-node ten-sample benchmark; isolated wheel proof; no secret-scan findings.
|
||||
Active milestone: None.
|
||||
Next directional milestone: 4 — adapter SDK and product documentation.
|
||||
Status: Milestone 3 is closed. Milestone 4 has not started.
|
||||
```
|
||||
|
||||
Milestones 4–5 remain directional context and are not active.
|
||||
Milestones 4–5 remain directional context. Do not begin Milestone 4 without a new active-slice
|
||||
contract.
|
||||
|
|
|
|||
|
|
@ -345,7 +345,7 @@ These are notes, not commitments:
|
|||
than bounded positions, they will need a different versioned security contract and persisted key
|
||||
lifecycle.
|
||||
|
||||
## Milestone 2 — active: agent retrieval and MCP experience
|
||||
## Milestone 2 — complete: agent retrieval and MCP experience
|
||||
|
||||
### Audit reconciliation
|
||||
|
||||
|
|
@ -611,10 +611,10 @@ task-shaped capsule for every continuation page, add authenticated continuation
|
|||
model requires it, verify a native Claude timeout representation, and introduce adapter-owned
|
||||
launcher metadata before generating configurations for custom adapters.
|
||||
|
||||
## Milestone 3 — active: independent projections
|
||||
## Milestone 3 — complete: independent projections
|
||||
|
||||
Milestone 3 began only after `main` and `dev` were aligned at the verified Milestone 2 closeout.
|
||||
Three read-only audits are running before source changes:
|
||||
Three read-only audits ran before source changes:
|
||||
|
||||
- Manual planning, immutable packages, renderer isolation, receipts, preview/application
|
||||
integration, and full/incremental equivalence.
|
||||
|
|
@ -692,8 +692,95 @@ The first slice now implements:
|
|||
- A live-viewer correction: source evidence now comes from the pinned index generation. The
|
||||
viewer no longer reopens mutable canonical files behind an older graph snapshot.
|
||||
|
||||
The new repository-native contract target passes 91 tests and 120 subtests. The combined
|
||||
projection, rendering, and live-viewer focus passes with byte-exact compatibility and no hidden
|
||||
source/path authority. This is not Milestone 3 closeout: durable multi-artifact publication,
|
||||
portable graph rendering, detached workers, fragment reuse/equivalence, policy version 2,
|
||||
accessibility, and maintained scale evidence remain active work.
|
||||
The new repository-native contract target passed 91 tests and 120 subtests at the slice boundary.
|
||||
The combined projection, rendering, and live-viewer focus passed with byte-exact compatibility and
|
||||
no hidden source/path authority.
|
||||
|
||||
### Durable portable graph publication
|
||||
|
||||
The portable graph path now has its own declared `graph_render` views, pure plans, fixed
|
||||
`portable_graph_html` renderer, content-addressed artifact store, renderer receipts, and one bounded
|
||||
generation/view manifest as the publication commit. It supports Nodes, Flow, and Web without
|
||||
including Logic. Static HTML contains the complete pre-rendered graph and treats JavaScript as
|
||||
progressive enhancement.
|
||||
|
||||
Publication revalidates source, view, artifact, receipt, and output identities across replacement.
|
||||
Status reads only bounded manifest and receipt evidence. It never plans or renders. Repair may
|
||||
restore a declared output from its content-addressed artifact. A post-artifact failure that cannot
|
||||
be rolled back returns explicit degraded committed evidence rather than reporting an ordinary
|
||||
failed mutation.
|
||||
|
||||
### Detached workers and incremental fragments
|
||||
|
||||
Manual and portable graph packages execute through one fixed one-request child protocol. The
|
||||
parent launches isolated Python from a trusted working directory with a sanitized environment,
|
||||
spools stdout to disk, reads one bounded canonical response, and validates the complete artifact
|
||||
and receipt identity. The worker accepts only the two built-in renderer identities. Requests are
|
||||
bounded by the 24,000,000-byte package contract, actual artifact transfer by 20,000,000 bytes, and
|
||||
execution by a 30-second timeout.
|
||||
|
||||
Manual fragment records are semantic, versioned, canonical, hash-bound, and stored below a
|
||||
dedicated confined cache. The worker independently recomputes the expected page fragment before
|
||||
using a record. Corrupt, forged, oversized, stale, or aggregate-oversized records fall back to the
|
||||
full detached render. Cold fragment creation is compared byte-for-byte with that full oracle before
|
||||
cache publication. The cache retains only the current inventory and is capped at 10,000 entries
|
||||
and 64,000,000 bytes.
|
||||
|
||||
### Independent policies and accessibility
|
||||
|
||||
Projection policy version 2 independently composes manual `auto|explicit|disabled`, portable graph
|
||||
`explicit|disabled`, and live viewer `on-demand|disabled`. CLI, MCP, generated client
|
||||
configuration, doctor, render services, canonical application, onboarding, and viewer-manager
|
||||
entry points enforce their relevant policy. Status remains available when an active operation is
|
||||
disabled.
|
||||
|
||||
Generated client evidence binds the projection policy, its hash, projection availability, and the
|
||||
current descriptor hash into the configuration hash. Validation cross-checks omitted default
|
||||
selectors against the bound descriptor so coordinated policy and availability drift fails closed.
|
||||
The version-1 effective-policy payload remains unchanged for existing clients.
|
||||
|
||||
Pinned Playwright 1.62.0 and axe-core 4.12.1 gates exercise the frozen manual, portable graph, and
|
||||
live viewer with selected WCAG A/AA axe tags and keyboard interaction flows. Portable and live
|
||||
graph presentation received only the minimal contrast and nested-role corrections needed by those
|
||||
gates.
|
||||
|
||||
### Scale and runtime hardening
|
||||
|
||||
The first 1,000-node full benchmark exposed recursive strongly connected-component traversal in
|
||||
manual planning. Cycle detection now uses an iterative two-pass traversal. A regression covers the
|
||||
descriptor maximum of 10,000 nodes as both a deep acyclic chain and one strongly connected
|
||||
component.
|
||||
|
||||
The isolated wheel proof also exposed a Python `runpy` warning when the worker module was imported
|
||||
during package initialization before `-m` execution. A private fixed module entrypoint now owns
|
||||
child startup. Malformed child input returns code 2 with empty stdout and stderr.
|
||||
|
||||
Configured render ceilings above 20,000,000 bytes remain accepted for compatibility, and small
|
||||
actual artifacts render normally. The detached protocol still rejects an actual transfer beyond
|
||||
its fixed 20,000,000-byte boundary.
|
||||
|
||||
### Milestone 3 closeout
|
||||
|
||||
Candidate `f5dccb5e1c312121f1af63780162f593d9363b98` passed the complete repository gate: formatting,
|
||||
Python and web lint, strict types, compilation, 281 tests and 272 subtests, three accessibility
|
||||
flows, lock and dependency checks, package builds, and all milestone smoke benchmarks. The
|
||||
maintained projection contract subset passed 142 tests and 236 subtests.
|
||||
|
||||
The clean ten-sample 1,000-node benchmark passed every latency, memory, response-size, no-work, and
|
||||
equivalence gate. Manual full rendering measured 810.490 ms p95, portable graph full rendering
|
||||
323.690 ms p95, and receipt-only status 111.381 ms and 59.331 ms p95 respectively. Direct detached
|
||||
worker peaks were 88,580,096 and 89,583,616 bytes. The separately gated production manual worker
|
||||
peak was 104,771,584 bytes. Production cold, warm, forced-full, add, change, delete, and reorder
|
||||
outputs were byte-identical.
|
||||
|
||||
Production warm fragment rendering measured 2,206.540 ms p95 versus 978.870 ms for forced full.
|
||||
Milestone 3 therefore closes the fragment isolation, invalidation, equivalence, and recovery
|
||||
contract without claiming a throughput win. Later optimization must begin from that evidence.
|
||||
|
||||
The exact method and measurements are recorded in `docs/MILESTONE_3_BASELINE.md` and
|
||||
`benchmarks/milestone3-2026-07-29.json`. The candidate passed an isolated wheel CLI/MCP/worker
|
||||
proof. Gitleaks 8.30.1 found no findings across the six Milestone 3 commits or candidate tree.
|
||||
|
||||
Milestone 3 is complete. No tag, release, production integration repointing, WorldForge change,
|
||||
ScrapeStation change, storage rewrite, or self-hosting dependency was introduced. Milestone 4
|
||||
remains directional and has not started.
|
||||
|
|
|
|||
22
Makefile
22
Makefile
|
|
@ -5,7 +5,10 @@ NPM := npm
|
|||
PYTHONPYCACHEPREFIX := /tmp/docforge-quality-pycache
|
||||
PYTEST_BASETEMP := /tmp/docforge-quality-pytest
|
||||
|
||||
.PHONY: benchmark benchmark-m1 benchmark-m1-smoke benchmark-m2 benchmark-m2-smoke benchmark-smoke build compile contract dependencies format-check gate lint lock test type
|
||||
.PHONY: accessibility benchmark benchmark-m1 benchmark-m1-smoke benchmark-m2 benchmark-m2-smoke benchmark-m3 benchmark-m3-full benchmark-m3-smoke benchmark-smoke build compile contract dependencies format-check gate lint lock test type
|
||||
|
||||
accessibility:
|
||||
$(NPM) run test:accessibility
|
||||
|
||||
format-check:
|
||||
$(PYTHON) -m ruff format --check src tests tools
|
||||
|
|
@ -25,12 +28,18 @@ contract:
|
|||
-p no:cacheprovider --basetemp=$(PYTEST_BASETEMP) \
|
||||
tests/test_public_contract.py \
|
||||
tests/test_policy.py \
|
||||
tests/test_projection_policy.py \
|
||||
tests/test_projection_policy_integration.py \
|
||||
tests/test_projection_worker.py \
|
||||
tests/test_projection_fragments.py \
|
||||
tests/test_retrieval.py \
|
||||
tests/test_generation_diff.py \
|
||||
tests/test_client_integration.py \
|
||||
tests/test_projection_contract.py \
|
||||
tests/test_projection_schemas.py \
|
||||
tests/test_graph_projection.py \
|
||||
tests/test_graph_rendering.py \
|
||||
tests/test_graph_publication.py \
|
||||
tests/test_observability.py::TelemetryContractTests::test_schema_fixed_names_match_the_implementation \
|
||||
tests/test_adapter_contract.py::AdapterContractTests::test_no_ast_index_policy_rejects_logic_publication \
|
||||
tests/test_adapter_contract.py::AdapterContractTests::test_no_ast_index_accepts_legacy_and_non_logic_incremental_adapters \
|
||||
|
|
@ -71,4 +80,13 @@ benchmark-m2-smoke:
|
|||
benchmark-m2:
|
||||
$(PYTHON) tools/milestone2_benchmark.py --nodes 1000 --samples 10
|
||||
|
||||
gate: format-check lint type compile contract test lock dependencies build benchmark-smoke benchmark-m1-smoke benchmark-m2-smoke
|
||||
benchmark-m3-smoke:
|
||||
$(PYTHON) tools/milestone3_benchmark.py --mode smoke \
|
||||
--output /tmp/docforge-milestone3-smoke.json > /dev/null
|
||||
|
||||
benchmark-m3:
|
||||
$(PYTHON) tools/milestone3_benchmark.py --mode full
|
||||
|
||||
benchmark-m3-full: benchmark-m3
|
||||
|
||||
gate: format-check lint type compile contract test accessibility lock dependencies build benchmark-smoke benchmark-m1-smoke benchmark-m2-smoke benchmark-m3-smoke
|
||||
|
|
|
|||
73
README.md
73
README.md
|
|
@ -26,6 +26,12 @@ declared manuals, visualizes project structure, and manages reviewable documenta
|
|||
- Detects project-local adapter implementation and configuration changes and requires a fresh
|
||||
project-bound process before any further MCP work.
|
||||
- Keeps function-scoped control-flow projections separate from the primary architecture graph.
|
||||
- Compiles manuals and portable graph artifacts from separate versioned, generation-pinned plans
|
||||
and immutable packages.
|
||||
- Runs built-in manual and graph renderers in fixed detached workers with validated receipts,
|
||||
bounded transfer, and no project-path authority.
|
||||
- Enforces independent manual, portable-graph, and live-viewer policy while keeping status
|
||||
receipt-only.
|
||||
- Runs a managed loopback graph browser with neighborhood, semantic Flow, convergence Web,
|
||||
function-scoped Logic, source inspection, and branch-aware node hiding.
|
||||
- Supports generic documentation projects and project-owned source adapters.
|
||||
|
|
@ -73,6 +79,63 @@ details, and explicit truncation. Paged results use one top-level cursor and a v
|
|||
`receipt_header`; `stored_receipt_hash` identifies the complete persisted receipt. The read never
|
||||
exposes Logic details, loads canonical source, repairs derived state, or invents history.
|
||||
|
||||
## Independent projections
|
||||
|
||||
Manual compilation, portable graph rendering, and the live viewer consume the same validated graph
|
||||
generation through separate boundaries:
|
||||
|
||||
```text
|
||||
validated generation
|
||||
├── ManualRenderPlanV1 → immutable package → detached manual renderer
|
||||
├── GraphViewPlanV1 → immutable package → detached portable graph renderer
|
||||
└── pinned index → managed read-only live viewer
|
||||
```
|
||||
|
||||
Plans, packages, and receipts are canonical, versioned, hash-identified, bounded, and contain no
|
||||
project object, SQLite handle, absolute project path, command, or caller-selected renderer module.
|
||||
Renderers cannot select graph facts, crawl canonical sources, choose publication paths, or mutate
|
||||
project state.
|
||||
|
||||
Declare portable graph output separately from manual views:
|
||||
|
||||
```toml
|
||||
[graph_render]
|
||||
output_root = ".docforge/portable-graph"
|
||||
|
||||
[[graph_render.views]]
|
||||
id = "architecture"
|
||||
renderer = "portable_graph_html"
|
||||
output = "architecture.html"
|
||||
title = "Architecture"
|
||||
root = "architecture.overview"
|
||||
initial_mode = "web"
|
||||
depth = 3
|
||||
max_nodes = 250
|
||||
max_edges = 1000
|
||||
max_work = 100000
|
||||
include_logic = false
|
||||
```
|
||||
|
||||
Plan, publish, and inspect it explicitly:
|
||||
|
||||
```bash
|
||||
.venv/bin/docforge --project-root "$PROJECT" graph-plan architecture
|
||||
.venv/bin/docforge --project-root "$PROJECT" graph-render architecture
|
||||
.venv/bin/docforge --project-root "$PROJECT" graph-render-status architecture
|
||||
```
|
||||
|
||||
The version-2 projection policy independently selects manual
|
||||
`auto|explicit|disabled`, portable graph `explicit|disabled`, and live viewer
|
||||
`on-demand|disabled`. Use `--manual-render-policy`, `--portable-graph-policy`, and
|
||||
`--live-viewer-policy` on CLI/MCP startup or generated client configuration. Status remains
|
||||
available when the corresponding active operation is disabled.
|
||||
|
||||
Manual fragment reuse is disposable. Cold record publication is guarded by byte-exact comparison
|
||||
with a full detached render; warm records are independently recomputed and validated inside the
|
||||
worker. Full rendering remains the recovery and equivalence oracle. Portable artifacts commit
|
||||
content-addressed output and renderer evidence before one bounded generation/view manifest; status
|
||||
does not plan or render.
|
||||
|
||||
## Graph views
|
||||
|
||||
The browser presents the primary architecture graph through three complementary views and loads a
|
||||
|
|
@ -170,10 +233,18 @@ DocForge describes them as a source graph.
|
|||
performance, memory, rendering and response sizes, bottlenecks, and missing coverage.
|
||||
- [Milestone 0 closeout](docs/MILESTONE_0_CLOSEOUT.md) — lineage, migration, security scan,
|
||||
repository state, and fresh-clone proof.
|
||||
- [Milestone 1 baseline](docs/MILESTONE_1_BASELINE.md) — warm operation latency, structured work,
|
||||
status, retrieval, and memory measurements.
|
||||
- [Milestone 1 closeout](docs/MILESTONE_1_CLOSEOUT.md) — fast-core contracts, adversarial
|
||||
validation, compatibility boundaries, and exact candidate evidence.
|
||||
- [Milestone 2 baseline](docs/MILESTONE_2_BASELINE.md) — task context, generation diff, client
|
||||
configuration, doctor, response-size, counter, and memory measurements.
|
||||
- [Milestone 2 closeout](docs/MILESTONE_2_CLOSEOUT.md) — implemented contracts, adversarial
|
||||
validation, exclusions, and exact candidate evidence.
|
||||
- [Milestone 3 baseline](docs/MILESTONE_3_BASELINE.md) — manual, fragment, worker, portable graph,
|
||||
status, equivalence, response-size, and memory measurements.
|
||||
- [Milestone 3 closeout](docs/MILESTONE_3_CLOSEOUT.md) — independent projection contracts,
|
||||
adversarial validation, compatibility boundaries, and exact candidate evidence.
|
||||
- [MCP contract](docs/MCP_CONTRACT.md) — exact tool and process boundary.
|
||||
- [Viewer manager](docs/VIEWER_MANAGER.md) — native service setup and lifecycle.
|
||||
- [Adapter decision](docs/APPLICATION_DECISION.md) — why custom adapters own canonical
|
||||
|
|
@ -197,6 +268,8 @@ make gate
|
|||
Focused entry points are available as `make contract`, `make test`, `make type`,
|
||||
`make benchmark-smoke`, `make benchmark`, `make benchmark-m1-smoke`, and
|
||||
`make benchmark-m1`. Milestone 2 adds `make benchmark-m2-smoke` and `make benchmark-m2`.
|
||||
Milestone 3 adds `make accessibility`, `make benchmark-m3-smoke`, `make benchmark-m3`, and
|
||||
`make benchmark-m3-full`.
|
||||
|
||||
The committed 1,000-node baseline and its measurement method are under `benchmarks/`.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,51 @@
|
|||
# Completed slices
|
||||
|
||||
## DocForge2 Milestone 3 independent projections
|
||||
|
||||
### Changed
|
||||
|
||||
- Added strict version-1 manual plan, graph plan, projection package, and projection receipt
|
||||
contracts with canonical identities and packaged schemas.
|
||||
- Kept the legacy manual API and exact alpha bytes as a compatibility wrapper over a pure planner,
|
||||
immutable package, and independent renderer.
|
||||
- Added generation-pinned portable graph planning, detached rendering, content-addressed durable
|
||||
publication, repair, and receipt-only status.
|
||||
- Added a fixed isolated worker protocol with bounded request, response, artifact, timeout,
|
||||
environment, and renderer inventory.
|
||||
- Added semantic manual fragments with independent worker validation, corruption recovery,
|
||||
full-render equivalence, and bounded cache retention.
|
||||
- Added version-2 independent manual, portable-graph, and live-viewer policy with descriptor-bound
|
||||
generated-client evidence while retaining effective policy version 1.
|
||||
- Corrected live source reads to use the pinned index generation.
|
||||
- Added static and interactive accessibility gates for the manual, portable graph, and live viewer.
|
||||
- Replaced recursive cycle planning with an iterative traversal proven at 10,000 nodes.
|
||||
|
||||
### Verification
|
||||
|
||||
- The complete gate passed 281 tests, 272 subtests, three accessibility flows, dependency checks,
|
||||
builds, and all milestone smoke benchmarks.
|
||||
- The maintained projection contract subset passed 142 tests and 236 subtests.
|
||||
- The clean ten-sample 1,000-node benchmark passed every latency, memory, response-size, no-work,
|
||||
deterministic-output, and full/incremental equivalence threshold.
|
||||
- An isolated wheel passed CLI, MCP, detached manual rendering, and malformed worker startup.
|
||||
- Three adversarial review tracks were reconciled, and Gitleaks found no findings in the six
|
||||
milestone commits or candidate tree.
|
||||
|
||||
### Limits
|
||||
|
||||
- Production fragment reuse is correct and recoverable but is slower than forced full rendering at
|
||||
the maintained 1,000-page fixture. No speedup is claimed.
|
||||
- Actual detached artifact transfer is capped at 20,000,000 bytes.
|
||||
- Portable graph publication is CLI-owned. MCP exposes read-only plan and status tools.
|
||||
- Remote render services, render farms, third-party renderer ecosystems, storage replacement,
|
||||
self-hosting, adapter SDK expansion, and production integration changes remain out of scope.
|
||||
- No tag or release was created.
|
||||
|
||||
### Next gate
|
||||
|
||||
Milestone 4 remains directional and is not active. Create a new active-slice contract before
|
||||
starting adapter SDK or product-documentation implementation.
|
||||
|
||||
## DocForge2 Milestone 0 successor foundation
|
||||
|
||||
### Changed
|
||||
|
|
|
|||
|
|
@ -34,6 +34,14 @@ make benchmark-m2-smoke
|
|||
make benchmark-m2
|
||||
```
|
||||
|
||||
Run the Milestone 3 independent-projection smoke and maintained full gates:
|
||||
|
||||
```bash
|
||||
make benchmark-m3-smoke
|
||||
make benchmark-m3
|
||||
make benchmark-m3-full
|
||||
```
|
||||
|
||||
The benchmark creates canonical sources, derived state, changesets, rendered output, and caches
|
||||
only in a disposable temporary directory. It does not read another project, self-host DocForge, or
|
||||
mutate repository content.
|
||||
|
|
@ -56,9 +64,16 @@ records whether diagnostics were dropped for response budget, checks all hidden-
|
|||
and measures isolated-process peak RSS. Its interpretation is in
|
||||
[`docs/MILESTONE_2_BASELINE.md`](../docs/MILESTONE_2_BASELINE.md).
|
||||
|
||||
The generic fixture exposes whole-source scaling. It does not replace the incremental adapter
|
||||
equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has
|
||||
no portable graph-planning or graph-rendering contract.
|
||||
`milestone3-2026-07-29.json` is the clean-tree independent-projection baseline captured from commit
|
||||
`f5dccb5e1c312121f1af63780162f593d9363b98`. It measures versioned manual and graph planning,
|
||||
in-process and detached rendering, production cold/warm/forced-full behavior, fragment cache
|
||||
sweeps, add/change/delete/reorder equivalence, portable publication, receipt-only status, traced
|
||||
memory, detached worker peak memory, and response size. Its interpretation is in
|
||||
[`docs/MILESTONE_3_BASELINE.md`](../docs/MILESTONE_3_BASELINE.md).
|
||||
|
||||
The generic fixtures expose whole-source and projection scaling. They do not replace incremental
|
||||
adapter equivalence tests. Milestone 3's portable fixture contains 1,000 Nodes/Flow/Web nodes and
|
||||
999 edges; portable version 1 deliberately excludes Logic.
|
||||
|
||||
The Milestone 1 harness treats wall time and structured work counters as separate gates. Warm
|
||||
operations fail if they load a complete project, parse source files, reconstruct an adapter
|
||||
|
|
@ -73,4 +88,5 @@ status. The 1,000-node run records bounded semantic summaries for exact errors,
|
|||
backlinks, both traversal directions, paged context, render receipt states, and visualization
|
||||
freshness. The reported p95 uses the nearest-rank method; with ten samples it is the maximum.
|
||||
`process_peak_rss_kib` is the cumulative main-process `RUSAGE_SELF` high-water mark and excludes the
|
||||
detached viewer worker.
|
||||
detached viewer worker. It is diagnostic and not operation-local. Milestone 3 memory gates use
|
||||
per-operation `tracemalloc` peaks and detached worker receipt peaks instead.
|
||||
|
|
|
|||
442
benchmarks/milestone3-2026-07-29.json
Normal file
442
benchmarks/milestone3-2026-07-29.json
Normal file
|
|
@ -0,0 +1,442 @@
|
|||
{
|
||||
"benchmark": "docforge2_milestone3",
|
||||
"environment": {
|
||||
"implementation": "CPython",
|
||||
"machine": "x86_64",
|
||||
"platform": "Linux-7.1.3-200.nobara.fc44.x86_64-x86_64-with-glibc2.43",
|
||||
"python": "3.14.6"
|
||||
},
|
||||
"equivalence": {
|
||||
"manual_full_vs_fragment_assisted": true,
|
||||
"manual_in_process_vs_detached": true,
|
||||
"manual_production_cold_warm_full": true,
|
||||
"manual_production_variants": {
|
||||
"add": true,
|
||||
"change": true,
|
||||
"delete": true,
|
||||
"reorder": true
|
||||
},
|
||||
"portable_graph_in_process_vs_detached": true
|
||||
},
|
||||
"fixture": {
|
||||
"edge_count": 999,
|
||||
"full_coverage": true,
|
||||
"kind": "synthetic_generic_projection",
|
||||
"manual_page_count": 1000,
|
||||
"node_count": 1000,
|
||||
"portable_graph_edge_count": 999,
|
||||
"portable_graph_node_count": 1000
|
||||
},
|
||||
"memory": {
|
||||
"manual_production_worker_peak_bytes": 104771584,
|
||||
"manual_worker_peak_bytes": 88580096,
|
||||
"portable_graph_worker_peak_bytes": 89583616,
|
||||
"process_peak_rss_kib": 102692
|
||||
},
|
||||
"method": {
|
||||
"clock": "time.perf_counter_ns",
|
||||
"detached_peak_memory": "worker receipt resource peak RSS",
|
||||
"determinism": "stable semantic summaries must match across samples; report JSON uses sorted keys",
|
||||
"full_mode_node_requirement": 1000,
|
||||
"in_process_peak_memory": "tracemalloc per measured invocation",
|
||||
"maximum_worker_artifact_bytes": 20000000,
|
||||
"process_peak_memory": "resource.getrusage(RUSAGE_SELF).ru_maxrss",
|
||||
"response_size": "UTF-8 bytes of canonical compact sorted JSON",
|
||||
"samples": 10
|
||||
},
|
||||
"mode": "full",
|
||||
"operations": {
|
||||
"fragment_assisted_equivalence": {
|
||||
"max_ms": 666.119,
|
||||
"maximum_response_bytes": 664,
|
||||
"maximum_traced_peak_bytes": 5332792,
|
||||
"median_ms": 638.999,
|
||||
"min_ms": 633.157,
|
||||
"p95_limit_ms": 15000,
|
||||
"p95_ms": 666.119,
|
||||
"response_limit_bytes": 128000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"artifact_bytes": 583149,
|
||||
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"package_bytes": 2364302,
|
||||
"package_id": "ecc21e58106c420a9f78ffbba997778ca98b61586ecfdb23a15b08f431a52c42",
|
||||
"plan_bytes": 1006393,
|
||||
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"fragment_cache_hit_sweep": {
|
||||
"max_ms": 514.324,
|
||||
"maximum_response_bytes": 145,
|
||||
"maximum_traced_peak_bytes": 6941174,
|
||||
"median_ms": 498.911,
|
||||
"min_ms": 495.447,
|
||||
"p95_limit_ms": 5000,
|
||||
"p95_ms": 514.324,
|
||||
"response_limit_bytes": 32768,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"aggregate_content_bytes": 516921,
|
||||
"fragment_count": 1000,
|
||||
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"fragment_cache_miss_sweep": {
|
||||
"max_ms": 106.888,
|
||||
"maximum_response_bytes": 145,
|
||||
"maximum_traced_peak_bytes": 146245,
|
||||
"median_ms": 106.203,
|
||||
"min_ms": 105.383,
|
||||
"p95_limit_ms": 5000,
|
||||
"p95_ms": 106.888,
|
||||
"response_limit_bytes": 32768,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"aggregate_content_bytes": 516921,
|
||||
"fragment_count": 1000,
|
||||
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"fragment_cache_put_sweep": {
|
||||
"max_ms": 539.212,
|
||||
"maximum_response_bytes": 145,
|
||||
"maximum_traced_peak_bytes": 509548,
|
||||
"median_ms": 539.212,
|
||||
"min_ms": 539.212,
|
||||
"p95_limit_ms": 10000,
|
||||
"p95_ms": 539.212,
|
||||
"response_limit_bytes": 32768,
|
||||
"samples": 1,
|
||||
"stable_result": {
|
||||
"aggregate_content_bytes": 516921,
|
||||
"fragment_count": 1000,
|
||||
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"manual_detached_worker": {
|
||||
"child_peak_limit_bytes": 268435456,
|
||||
"max_ms": 599.394,
|
||||
"maximum_child_peak_bytes": 88580096,
|
||||
"maximum_response_bytes": 667,
|
||||
"maximum_traced_peak_bytes": 28203708,
|
||||
"median_ms": 591.317,
|
||||
"min_ms": 585.123,
|
||||
"p95_limit_ms": 20000,
|
||||
"p95_ms": 599.394,
|
||||
"response_limit_bytes": 128000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"artifact_bytes": 583149,
|
||||
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"package_bytes": 1007297,
|
||||
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
|
||||
"plan_bytes": 1006393,
|
||||
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"manual_forced_full": {
|
||||
"child_peak_limit_bytes": 268435456,
|
||||
"max_ms": 978.87,
|
||||
"maximum_child_peak_bytes": 104771584,
|
||||
"maximum_response_bytes": 668,
|
||||
"maximum_traced_peak_bytes": 30425714,
|
||||
"median_ms": 944.135,
|
||||
"min_ms": 934.773,
|
||||
"p95_limit_ms": 20000,
|
||||
"p95_ms": 978.87,
|
||||
"response_limit_bytes": 128000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"output_bytes": 583149,
|
||||
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"manual_full_render": {
|
||||
"max_ms": 810.49,
|
||||
"maximum_response_bytes": 664,
|
||||
"maximum_traced_peak_bytes": 4412754,
|
||||
"median_ms": 801.948,
|
||||
"min_ms": 773.7,
|
||||
"p95_limit_ms": 15000,
|
||||
"p95_ms": 810.49,
|
||||
"response_limit_bytes": 128000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"artifact_bytes": 583149,
|
||||
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"package_bytes": 1007297,
|
||||
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
|
||||
"plan_bytes": 1006393,
|
||||
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"manual_incremental_cold": {
|
||||
"child_peak_limit_bytes": 268435456,
|
||||
"max_ms": 2827.152,
|
||||
"maximum_child_peak_bytes": 99454976,
|
||||
"maximum_response_bytes": 668,
|
||||
"maximum_traced_peak_bytes": 35160716,
|
||||
"median_ms": 2827.152,
|
||||
"min_ms": 2827.152,
|
||||
"p95_limit_ms": 20000,
|
||||
"p95_ms": 2827.152,
|
||||
"response_limit_bytes": 128000,
|
||||
"samples": 1,
|
||||
"stable_result": {
|
||||
"output_bytes": 583149,
|
||||
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"manual_incremental_warm": {
|
||||
"child_peak_limit_bytes": 268435456,
|
||||
"max_ms": 2206.54,
|
||||
"maximum_child_peak_bytes": 104767488,
|
||||
"maximum_response_bytes": 669,
|
||||
"maximum_traced_peak_bytes": 34676043,
|
||||
"median_ms": 2143.388,
|
||||
"min_ms": 2125.466,
|
||||
"p95_limit_ms": 20000,
|
||||
"p95_ms": 2206.54,
|
||||
"response_limit_bytes": 128000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"output_bytes": 583149,
|
||||
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"manual_status_no_work": {
|
||||
"max_ms": 111.381,
|
||||
"maximum_response_bytes": 1542,
|
||||
"maximum_traced_peak_bytes": 1792025,
|
||||
"median_ms": 62.881,
|
||||
"min_ms": 60.111,
|
||||
"p95_limit_ms": 500,
|
||||
"p95_ms": 111.381,
|
||||
"response_limit_bytes": 256000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"counters": {
|
||||
"adapter_projection_loads": 0,
|
||||
"adapter_source_extractions": 0,
|
||||
"index_builds": 0,
|
||||
"index_checks": 0,
|
||||
"index_synchronizations": 0,
|
||||
"project_loads": 0,
|
||||
"render_output_bytes_built": 0,
|
||||
"render_output_bytes_hashed": 0,
|
||||
"render_prepare_calls": 0,
|
||||
"source_bytes_parsed": 0,
|
||||
"source_files_parsed": 0,
|
||||
"source_generation_checks": 2,
|
||||
"viewer_manager_requests": 0
|
||||
},
|
||||
"response": {
|
||||
"adapter": "generic",
|
||||
"configured": true,
|
||||
"outputs": [
|
||||
{
|
||||
"actual_output_hash": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"expected_output_hash": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"path": ".docforge/rendered/manual.html",
|
||||
"projection_receipt": {
|
||||
"artifacts": [
|
||||
{
|
||||
"artifact_id": "manual.html",
|
||||
"bytes": 583149,
|
||||
"media_type": "text/html; charset=utf-8",
|
||||
"sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c"
|
||||
}
|
||||
],
|
||||
"contract": "docforge.projection-receipt",
|
||||
"diagnostics": {
|
||||
"warnings": []
|
||||
},
|
||||
"kind": "manual",
|
||||
"package_id": "b30c65390570537a16ce03afe6a992599c6ca0c4ef5ea2f06d14b33e248bc9aa",
|
||||
"peak_memory_bytes": 104771584,
|
||||
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
|
||||
"receipt_id": "ead4c25b34ada8df674903a592e25dd6bbf9817df2b04e5ebc66cde9e41c1ed1",
|
||||
"renderer": {
|
||||
"renderer_id": "generic_html",
|
||||
"renderer_version": "1+markdown-it-py-4.2.0"
|
||||
},
|
||||
"schema_version": 1,
|
||||
"timing": {
|
||||
"elapsed_ns": 114544865
|
||||
}
|
||||
},
|
||||
"reason": null,
|
||||
"receipt_schema_version": 1,
|
||||
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0",
|
||||
"renderer": "generic_html",
|
||||
"renderer_version": "1+markdown-it-py-4.2.0",
|
||||
"state": "current",
|
||||
"template_hash": "ae0ebb3eadeb530e9d033c8fbd21321d406e29d04c0ae7d07a9a0f929f0ffa9f",
|
||||
"verification": "receipt",
|
||||
"view_id": "manual"
|
||||
}
|
||||
],
|
||||
"project_id": "synthetic-1000",
|
||||
"project_root_fingerprint": "0747acfd975703c0",
|
||||
"revision": "unversioned",
|
||||
"source_hash": "a314da4ffa8fcf291ef7a7b0fc87737caeaa89fa3c558ea442afeb0e5ae49c2d",
|
||||
"state": "current",
|
||||
"status": "ok",
|
||||
"verification": "receipt"
|
||||
}
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"portable_graph_detached_worker": {
|
||||
"child_peak_limit_bytes": 268435456,
|
||||
"max_ms": 268.428,
|
||||
"maximum_child_peak_bytes": 89583616,
|
||||
"maximum_response_bytes": 660,
|
||||
"maximum_traced_peak_bytes": 27587227,
|
||||
"median_ms": 265.448,
|
||||
"min_ms": 263.761,
|
||||
"p95_limit_ms": 20000,
|
||||
"p95_ms": 268.428,
|
||||
"response_limit_bytes": 128000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"artifact_bytes": 718383,
|
||||
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
|
||||
"package_bytes": 398715,
|
||||
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
|
||||
"plan_bytes": 398158,
|
||||
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"portable_graph_full_render": {
|
||||
"max_ms": 323.69,
|
||||
"maximum_response_bytes": 657,
|
||||
"maximum_traced_peak_bytes": 3490976,
|
||||
"median_ms": 303.736,
|
||||
"min_ms": 301.215,
|
||||
"p95_limit_ms": 10000,
|
||||
"p95_ms": 323.69,
|
||||
"response_limit_bytes": 128000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"artifact_bytes": 718383,
|
||||
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
|
||||
"package_bytes": 398715,
|
||||
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
|
||||
"plan_bytes": 398158,
|
||||
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b"
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
},
|
||||
"portable_graph_status_no_work": {
|
||||
"max_ms": 59.331,
|
||||
"maximum_response_bytes": 879,
|
||||
"maximum_traced_peak_bytes": 685012,
|
||||
"median_ms": 58.467,
|
||||
"min_ms": 58.052,
|
||||
"p95_limit_ms": 500,
|
||||
"p95_ms": 59.331,
|
||||
"response_limit_bytes": 256000,
|
||||
"samples": 10,
|
||||
"stable_result": {
|
||||
"counters": {
|
||||
"adapter_projection_loads": 0,
|
||||
"adapter_source_extractions": 0,
|
||||
"index_builds": 0,
|
||||
"index_checks": 0,
|
||||
"index_synchronizations": 0,
|
||||
"project_loads": 0,
|
||||
"render_output_bytes_built": 0,
|
||||
"render_output_bytes_hashed": 0,
|
||||
"render_prepare_calls": 0,
|
||||
"source_bytes_parsed": 0,
|
||||
"source_files_parsed": 0,
|
||||
"source_generation_checks": 2,
|
||||
"viewer_manager_requests": 0
|
||||
},
|
||||
"response": {
|
||||
"adapter": "generic",
|
||||
"configured": true,
|
||||
"outputs": [
|
||||
{
|
||||
"artifact": {
|
||||
"artifact_id": "portable-graph.html",
|
||||
"bytes": 718383,
|
||||
"media_type": "text/html; charset=utf-8",
|
||||
"sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30"
|
||||
},
|
||||
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
|
||||
"path": ".docforge/portable-graph/architecture.html",
|
||||
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b",
|
||||
"publication_id": "2620ce106593d5499931b145a42fc1eccd31f5e356b19bf1d76687f8d45bcad2",
|
||||
"reason": null,
|
||||
"renderer": "portable_graph_html",
|
||||
"renderer_version": "1",
|
||||
"state": "current",
|
||||
"verification": "manifest",
|
||||
"view_id": "architecture"
|
||||
}
|
||||
],
|
||||
"project_id": "synthetic-1000",
|
||||
"project_root_fingerprint": "0747acfd975703c0",
|
||||
"revision": "unversioned",
|
||||
"source_hash": "a314da4ffa8fcf291ef7a7b0fc87737caeaa89fa3c558ea442afeb0e5ae49c2d",
|
||||
"state": "current",
|
||||
"status": "ok"
|
||||
}
|
||||
},
|
||||
"traced_peak_limit_bytes": 268435456
|
||||
}
|
||||
},
|
||||
"schema_version": 1,
|
||||
"sizes": {
|
||||
"fragment_assisted_manual": {
|
||||
"aggregate_fragment_content_bytes": 516921,
|
||||
"artifact_bytes": 583149,
|
||||
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"fragment_count": 1000,
|
||||
"package_bytes": 2364302,
|
||||
"package_id": "ecc21e58106c420a9f78ffbba997778ca98b61586ecfdb23a15b08f431a52c42",
|
||||
"plan_bytes": 1006393,
|
||||
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
|
||||
"receipt_bytes": 664
|
||||
},
|
||||
"manual": {
|
||||
"artifact_bytes": 583149,
|
||||
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
|
||||
"package_bytes": 1007297,
|
||||
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
|
||||
"plan_bytes": 1006393,
|
||||
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
|
||||
"receipt_bytes": 664
|
||||
},
|
||||
"manual_status_response_bytes": 1542,
|
||||
"portable_graph": {
|
||||
"artifact_bytes": 718383,
|
||||
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
|
||||
"package_bytes": 398715,
|
||||
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
|
||||
"plan_bytes": 398158,
|
||||
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b",
|
||||
"receipt_bytes": 657
|
||||
},
|
||||
"portable_graph_status_response_bytes": 879
|
||||
},
|
||||
"source": {
|
||||
"dirty": false,
|
||||
"revision": "f5dccb5e1c312121f1af63780162f593d9363b98"
|
||||
}
|
||||
}
|
||||
|
|
@ -71,6 +71,19 @@ Existing hand-written client configurations remain valid and are never rewritten
|
|||
Doctor is inspection-only and does not become a hidden bootstrap, synchronization, or migration
|
||||
path.
|
||||
|
||||
The following Milestone 3 CLI additions are also additive:
|
||||
|
||||
- `docforge graph-plan VIEW_ID`
|
||||
- `docforge graph-render VIEW_ID`
|
||||
- `docforge graph-render-status [VIEW_ID]`
|
||||
- `--manual-render-policy auto|explicit|disabled`
|
||||
- `--portable-graph-policy explicit|disabled`
|
||||
- `--live-viewer-policy on-demand|disabled`
|
||||
|
||||
MCP adds the read-only `docforge_graph_plan` and `docforge_graph_render_status` tools. Portable
|
||||
graph publication remains an explicit local CLI integration action. Existing manual render,
|
||||
preview, visualization, and status names remain supported.
|
||||
|
||||
## Versioned data contracts
|
||||
|
||||
Milestone 0 preserves:
|
||||
|
|
@ -96,6 +109,11 @@ Milestone 0 preserves:
|
|||
- Latest-generation-diff page schema version 1. Pages use one top-level pagination object and a
|
||||
nested `receipt_header`. `stored_receipt_hash` names the complete stored receipt. Opaque cursors
|
||||
may be restarted after a server or receipt change and are not durable public identifiers.
|
||||
- Manual render-plan schema version 1.
|
||||
- Graph view-plan schema version 1.
|
||||
- Projection-package schema version 1.
|
||||
- Projection-receipt schema version 1.
|
||||
- Independent projection-policy schema version 2. Effective policy version 1 remains frozen.
|
||||
|
||||
Indexes, attestations, extraction caches, previews, and rendered artifacts are disposable. A schema
|
||||
change may rebuild them. Canonical project content and stored proposals may not be silently
|
||||
|
|
@ -161,12 +179,24 @@ The following guarantees remain stable:
|
|||
|
||||
## Rendering and visualization
|
||||
|
||||
The `generic_html` renderer remains the supported version-1 manual projection. It retains confined
|
||||
paths, raw-HTML suppression, fixed template tokens, deterministic identities, atomic replacement,
|
||||
and side-effect-free status.
|
||||
The `generic_html` renderer remains the supported version-1 manual projection. Its public
|
||||
`GenericHtmlRenderer.prepare()` signature, renderer identity, frozen alpha bytes, confined paths,
|
||||
raw-HTML suppression, fixed template tokens, deterministic identities, atomic replacement, and
|
||||
side-effect-free status remain compatible. It now delegates through a versioned manual plan,
|
||||
immutable package, and independent renderer.
|
||||
|
||||
The `portable_graph_html` renderer and `graph_render` descriptor table are additive. Manual and
|
||||
portable graph declarations, plans, policies, publication receipts, and status remain separate.
|
||||
The portable renderer does not replace the existing live viewer or `docforge_visualize`.
|
||||
|
||||
Existing project descriptors may retain any positive `max_render_bytes` accepted by schema version
|
||||
1. A value above 20,000,000 bytes does not make the descriptor invalid, and a smaller actual
|
||||
artifact still renders. Actual detached worker transfer is a separate fixed 20,000,000-byte
|
||||
runtime boundary.
|
||||
|
||||
The live graph viewer remains a read-only consumer of a generation-pinned validated index. It does
|
||||
not become project authority or MCP retrieval authority.
|
||||
not become project authority or MCP retrieval authority. Source reads use the pinned index
|
||||
generation instead of reopening mutable canonical files behind that generation.
|
||||
|
||||
## Task-context compatibility
|
||||
|
||||
|
|
@ -199,8 +229,9 @@ The exact version-1 relation aliases are frozen by the MCP contract and reposito
|
|||
Changing an alias category requires a new planner version; it is not a silent implementation
|
||||
detail.
|
||||
|
||||
`ManualRenderPlan`, `GraphViewPlan`, a portable graph renderer, and independently packaged
|
||||
renderers are later-milestone direction. Milestone 0 does not claim that those contracts exist.
|
||||
Milestone 3 adds `ManualRenderPlanV1`, `GraphViewPlanV1`, projection package and receipt version 1,
|
||||
and projection policy version 2. These are additive submodule and schema contracts. They do not
|
||||
change the legacy task-context, adapter, changeset, or effective-policy contracts described above.
|
||||
|
||||
## Safety boundary
|
||||
|
||||
|
|
@ -221,6 +252,8 @@ Milestone 0 records rather than redesigns these areas:
|
|||
callers use targeted retrieval for that node.
|
||||
- One individually oversized changeset diff is transported as reconstructable canonical-JSON
|
||||
chunks. Cursors are corruption-detecting read tokens, not authenticated authorization tokens.
|
||||
- Manual planning is not separated from rendering.
|
||||
- There is no portable graph-planning or graph-rendering contract.
|
||||
- Production fragment validation is currently slower than forced-full rendering at the maintained
|
||||
1,000-page fixture. Full rendering remains the equivalence and recovery oracle.
|
||||
- Remote render services, render farms, third-party renderer ecosystems, and a separate render MCP
|
||||
remain deferred.
|
||||
- DocForge2 does not self-host its bootstrap documentation.
|
||||
|
|
|
|||
|
|
@ -24,6 +24,11 @@ commit when Git is available; it cannot change repository state.
|
|||
- Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1.
|
||||
- Generated client configuration: `schemas/client-configuration.schema.json`, version 1.
|
||||
- Client doctor result: `schemas/doctor-result.schema.json`, version 1.
|
||||
- Manual render plan: `schemas/manual-render-plan.schema.json`, version 1.
|
||||
- Graph view plan: `schemas/graph-view-plan.schema.json`, version 1.
|
||||
- Projection package: `schemas/projection-package.schema.json`, version 1.
|
||||
- Projection receipt: `schemas/projection-receipt.schema.json`, version 1.
|
||||
- Independent projection policy: `schemas/projection-policy.schema.json`, version 2.
|
||||
- Index schema: version 3, disposable and reproducible.
|
||||
- Index attestation: schema version 1, disposable and reproducible.
|
||||
- Core, CLI, and MCP server: version 1.3.0.dev0.
|
||||
|
|
@ -82,7 +87,9 @@ the header alone. One top-level pagination object carries the only continuation
|
|||
Generated Codex, Claude, and OpenClaw fragments are machine-local projections. They are not
|
||||
canonical project content. Version 1 binds the selected project, exact isolated Python
|
||||
interpreter, canonical argument layout, effective policy, no-AST projection, render policy,
|
||||
timeouts, artifact bytes, and configuration hash.
|
||||
timeouts, artifact bytes, and configuration hash. Milestone 3 adds the version-2 projection policy,
|
||||
its hash, projection availability, and the exact descriptor hash to that attested configuration
|
||||
evidence. Omitted default selectors are recomposed against the bound descriptor.
|
||||
|
||||
Preview is side-effect free. Explicit publication creates only one new private standalone
|
||||
fragment in an existing real directory. It never merges or replaces different content. Descriptor,
|
||||
|
|
@ -135,10 +142,12 @@ preview root, and one or more stable view IDs. Each view names a built-in render
|
|||
derived output file, title, and optional family filter. Paths are resolved under the project root
|
||||
and may not overlap canonical content, authority files, changesets, templates, or previews.
|
||||
|
||||
The initial `generic_html` renderer uses pinned CommonMark parsing with raw HTML disabled. Templates
|
||||
are UTF-8 files with a fixed token vocabulary; they cannot name commands, modules, or executable
|
||||
The `generic_html` renderer uses pinned CommonMark parsing with raw HTML disabled. Templates are
|
||||
UTF-8 files with a fixed token vocabulary; they cannot name commands, modules, or executable
|
||||
renderers. Render identity covers the canonical source hash, optional changeset hash, selected node
|
||||
and edge identities, view configuration, template hash, renderer contract, and exact parser version.
|
||||
and edge identities, view configuration, template hash, renderer contract, and exact parser
|
||||
version. The frozen version-1 API and alpha bytes are preserved by a compatibility wrapper over the
|
||||
manual plan/package/renderer path.
|
||||
|
||||
An explicit CLI render atomically replaces one declared derived output. MCP can render a validated
|
||||
changeset only to its isolated preview path. Normal status verifies bounded source, configuration,
|
||||
|
|
@ -146,6 +155,56 @@ template, output, renderer, and publication-receipt identities without reconstru
|
|||
Explicit deep status remains the side-effect-free full-render oracle. Input changes detected before
|
||||
atomic replacement fail without publishing a current receipt for stale output.
|
||||
|
||||
## Independent projection boundary
|
||||
|
||||
Manual and portable graph plans are separate version-1 contracts over one immutable validated
|
||||
generation. They use canonical JSON, deterministic ordering, fixed structural and serialized-size
|
||||
bounds, and content-derived identities. Plans contain selected graph facts and bounded content.
|
||||
They contain no live project object, database handle, absolute project or index path, arbitrary
|
||||
query, command, executable path, or caller-selected module.
|
||||
|
||||
Projection packages bind one plan to inert assets, a closed built-in renderer identity, declared
|
||||
component versions, and an artifact inventory with a byte allowance. Receipts bind the exact
|
||||
package, plan, renderer, artifact hashes and sizes, diagnostics, timing, and detached peak memory.
|
||||
Manual and graph renderer modules accept only validated packages. They cannot select nodes, invent
|
||||
relationships, read project state, choose publication paths, or write canonical files.
|
||||
|
||||
Detached execution uses one fixed private Python module, isolated mode, a trusted working
|
||||
directory, a sanitized environment, exactly one canonical newline-terminated JSON request and
|
||||
response, a closed renderer allowlist, a 30-second timeout, disk-spooled stdout, and bounded reads. The package
|
||||
contract is capped at 24,000,000 bytes and actual detached artifact transfer at 20,000,000 bytes.
|
||||
Project descriptors may retain a larger `max_render_bytes` compatibility allowance, but an actual
|
||||
detached transfer above the fixed worker boundary fails closed.
|
||||
|
||||
Portable graph configuration is independent of manual render configuration. One view selects
|
||||
either an exact root or a bounded metadata-only lexical query plus closed filters and node, edge,
|
||||
depth, and work limits. Logic is excluded. The renderer emits a complete static Nodes, Flow, or Web
|
||||
artifact and uses JavaScript only as progressive enhancement.
|
||||
|
||||
Portable publication commits a content-addressed artifact, renderer receipt, and one bounded
|
||||
generation/view manifest in that order. The manifest is the publication commit. Status reads only
|
||||
bounded manifest and receipt evidence and never plans or renders. Repair restores declared output
|
||||
only from validated content-addressed evidence. A failure after a replacement that cannot be
|
||||
proven rolled back returns explicit degraded committed evidence.
|
||||
|
||||
Manual fragment records are disposable semantic cache entries. Their keys bind the renderer,
|
||||
component version, and complete page semantics. The detached renderer recomputes the expected page
|
||||
fragment before accepting cached bytes. Cold creation is compared with a full detached render
|
||||
before cache publication. Invalid, corrupt, forged, stale, individually oversized, or
|
||||
aggregate-oversized records fall back to the full oracle. The dedicated cache retains only current
|
||||
keys and is capped at 10,000 entries and 64,000,000 bytes.
|
||||
|
||||
Projection policy version 2 composes manual `auto|explicit|disabled`, portable graph
|
||||
`explicit|disabled`, and live viewer `on-demand|disabled` independently. Active plan, render,
|
||||
application, onboarding, and viewer-start operations enforce the relevant policy before hidden
|
||||
work. Receipt-only status and explicit viewer stop remain available. Effective policy version 1
|
||||
and its legacy projections remain unchanged.
|
||||
|
||||
The live viewer remains separate from portable graph publication. It consumes one
|
||||
generation-pinned validated index through the viewer manager. Source reads come from that pinned
|
||||
generation and do not reopen mutable canonical files behind an older snapshot. Neither live nor
|
||||
portable visualization is retrieval or canonical authority.
|
||||
|
||||
Normal MCP access does not expose canonical application. An explicitly configured canonical
|
||||
applier registers one hash-bound application tool. No MCP mode exposes arbitrary renderer
|
||||
execution, arbitrary file writes, shell commands, Git mutation, build commands, deployment, or
|
||||
|
|
|
|||
90
docs/MILESTONE_3_BASELINE.md
Normal file
90
docs/MILESTONE_3_BASELINE.md
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
# Milestone 3 baseline
|
||||
|
||||
## Scope and method
|
||||
|
||||
This baseline records the independent-projection behavior completed in Milestone 3. It was
|
||||
captured on 2026-07-29 from clean candidate commit
|
||||
`f5dccb5e1c312121f1af63780162f593d9363b98`.
|
||||
|
||||
The maintained command was:
|
||||
|
||||
```bash
|
||||
.venv/bin/python tools/milestone3_benchmark.py \
|
||||
--mode full \
|
||||
--nodes 1000 \
|
||||
--samples 10 \
|
||||
--output benchmarks/milestone3-2026-07-29.json
|
||||
```
|
||||
|
||||
The synthetic generic fixture contains 1,000 manual pages, 1,000 portable-graph nodes, and 999
|
||||
edges. Durations use `time.perf_counter_ns()` and nearest-rank p95. In-process peak memory uses
|
||||
`tracemalloc`; detached worker peak memory comes from the worker receipt and `RUSAGE_SELF`.
|
||||
Every measured result is checked for deterministic semantic identity and bounded response size.
|
||||
|
||||
Environment:
|
||||
|
||||
- Linux 7.1.3-200.nobara.fc44.x86_64.
|
||||
- CPython 3.14.6.
|
||||
- x86_64.
|
||||
- Ten samples except the one-time production cold render and fragment-cache population.
|
||||
- In-process and detached-worker memory ceiling: 268,435,456 bytes.
|
||||
- Detached artifact-transfer ceiling: 20,000,000 bytes.
|
||||
|
||||
The complete machine-readable result is
|
||||
[`benchmarks/milestone3-2026-07-29.json`](../benchmarks/milestone3-2026-07-29.json).
|
||||
|
||||
## Results
|
||||
|
||||
| Operation | Median | p95 | Limit | Maximum response |
|
||||
|---|---:|---:|---:|---:|
|
||||
| Manual full plan/package/render | 801.948 ms | 810.490 ms | 15,000 ms | 664 B |
|
||||
| Manual detached worker | 591.317 ms | 599.394 ms | 20,000 ms | 667 B |
|
||||
| Fragment-assisted equivalence | 638.999 ms | 666.119 ms | 15,000 ms | 664 B |
|
||||
| Production incremental cold | 2,827.152 ms | 2,827.152 ms | 20,000 ms | 668 B |
|
||||
| Production incremental warm | 2,143.388 ms | 2,206.540 ms | 20,000 ms | 669 B |
|
||||
| Production forced full | 944.135 ms | 978.870 ms | 20,000 ms | 668 B |
|
||||
| Fragment cache miss sweep | 106.203 ms | 106.888 ms | 5,000 ms | 145 B |
|
||||
| Fragment cache hit sweep | 498.911 ms | 514.324 ms | 5,000 ms | 145 B |
|
||||
| Portable graph full plan/package/render | 303.736 ms | 323.690 ms | 10,000 ms | 657 B |
|
||||
| Portable graph detached worker | 265.448 ms | 268.428 ms | 20,000 ms | 660 B |
|
||||
| Manual receipt-only status | 62.881 ms | 111.381 ms | 500 ms | 1,542 B |
|
||||
| Portable graph receipt-only status | 58.467 ms | 59.331 ms | 500 ms | 879 B |
|
||||
|
||||
The largest traced in-process peak was 35,160,716 bytes. The direct manual worker track peaked at
|
||||
88,580,096 bytes and the portable graph worker at 89,583,616 bytes. The production manual paths,
|
||||
including cold, warm, forced-full, and mutation variants, peaked at 104,771,584 bytes. Every child
|
||||
peak was validated from its projection receipt against the 268,435,456-byte gate.
|
||||
|
||||
The manual artifact was 583,149 bytes. The portable graph artifact was 718,383 bytes. The manual
|
||||
plan was 1,006,393 bytes and its ordinary package was 1,007,297 bytes. The graph plan was 398,158
|
||||
bytes and its package was 398,715 bytes.
|
||||
|
||||
## Equivalence and no-work gates
|
||||
|
||||
The benchmark proved exact output equivalence for:
|
||||
|
||||
- Manual in-process and detached rendering.
|
||||
- Manual full and fragment-assisted rendering.
|
||||
- Production cold, warm, and forced-full rendering.
|
||||
- Production add, change, delete, and reorder variants.
|
||||
- Portable graph in-process and detached rendering.
|
||||
|
||||
Manual and portable-graph status each performed zero project loads, source parses, adapter
|
||||
projection loads, adapter extraction, index checks, synchronization, index builds, render
|
||||
preparation, output construction, output hashing, and viewer-manager requests. Each status path
|
||||
performed only two cheap source-generation checks and verified committed receipt or manifest
|
||||
evidence.
|
||||
|
||||
## Measured limits and future notes
|
||||
|
||||
- Fragment reuse is a correctness, isolation, and recovery boundary in this milestone. At 1,000
|
||||
pages, production warm fragment validation is slower than the forced-full path. Later
|
||||
optimization must start from this measurement and preserve byte equivalence.
|
||||
- Full rendering remains the oracle and recovery path. Invalid, corrupt, oversized, stale, or
|
||||
mismatched fragment records fall back without changing canonical facts.
|
||||
- The benchmark main-process `ru_maxrss` value was 102,692 KiB. It is cumulative across all
|
||||
main-process operations and is recorded only as diagnostic context. Detached child peaks are
|
||||
measured separately. Per-operation traced peaks and every detached receipt peak own the memory
|
||||
gates.
|
||||
- The results do not justify a storage rewrite, render farm, remote renderer, or separate render
|
||||
MCP.
|
||||
89
docs/MILESTONE_3_CLOSEOUT.md
Normal file
89
docs/MILESTONE_3_CLOSEOUT.md
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
# Milestone 3 closeout
|
||||
|
||||
## Outcome
|
||||
|
||||
Milestone 3 is complete. Manual compilation, portable graph rendering, and the live viewer are
|
||||
separate generation-pinned consumers of the validated graph. They cannot become canonical or
|
||||
retrieval authority.
|
||||
|
||||
Implemented contracts:
|
||||
|
||||
- Version-1 `ManualRenderPlan`, `GraphViewPlan`, projection package, and projection receipt.
|
||||
- Strict canonical JSON identities and packaged Draft 2020-12 schemas.
|
||||
- Independent manual and portable-graph renderer import boundaries.
|
||||
- One isolated, fixed, one-request detached worker protocol with bounded request, response,
|
||||
artifact, timeout, environment, and renderer inventory.
|
||||
- Content-addressed portable graph artifacts, renderer receipts, generation/view manifests,
|
||||
receipt-only status, repair, and degraded committed-publication evidence.
|
||||
- Disposable semantic fragment records with bounded cache inventory, corruption recovery, and
|
||||
full-render equivalence.
|
||||
- Version-2 independent projection policy while preserving version-1 effective-policy behavior.
|
||||
- Generation-pinned live source reads and a separate read-only viewer-manager lifecycle.
|
||||
- Automated axe-tag and keyboard gates for the manual, portable graph, and live viewer.
|
||||
- Repository-native contract, smoke, scale, response-size, memory, and equivalence gates.
|
||||
|
||||
## Candidate evidence
|
||||
|
||||
The frozen implementation candidate is
|
||||
`f5dccb5e1c312121f1af63780162f593d9363b98`.
|
||||
|
||||
The complete repository gate passed:
|
||||
|
||||
- Ruff formatting and lint.
|
||||
- HTML, rendered-manual HTML, portable-graph HTML, CSS, and JavaScript checks.
|
||||
- Pyright with zero diagnostics.
|
||||
- Warning-strict compilation and tests.
|
||||
- 281 tests and 272 subtests.
|
||||
- Three Playwright and axe accessibility flows. The alpha manual is checked with WCAG 2.0/2.1
|
||||
A/AA axe tags; portable and live graph flows add WCAG 2.2 A/AA tags and keyboard interaction.
|
||||
- Lock and npm dependency-tree checks.
|
||||
- Wheel and source-distribution builds.
|
||||
- Milestone 0, 1, 2, and 3 smoke benchmarks.
|
||||
|
||||
The maintained projection contract subset passed 142 tests and 236 subtests. A 10,000-node deep
|
||||
chain and one 10,000-node strongly connected component prove that manual cycle planning has no
|
||||
recursion-depth failure.
|
||||
|
||||
An isolated wheel installation passed CLI and MCP startup, a real detached manual render, and the
|
||||
closed malformed-worker-request contract. Six Milestone 3 commits and the complete candidate tree
|
||||
passed Gitleaks 8.30.1 with no findings.
|
||||
|
||||
Three independent adversarial review tracks covered manual isolation and fragment integrity,
|
||||
portable publication and policy binding, and worker/accessibility/benchmark gates. Reproduced
|
||||
project import, hostile environment, unbounded stdout, fragment forgery, cache growth, aggregate
|
||||
overflow, coordinated policy drift, render-limit compatibility, deep-graph, and module-startup
|
||||
defects were fixed and regression-tested before closeout.
|
||||
|
||||
The clean ten-sample 1,000-node benchmark passed every threshold. Exact measurements, equivalence
|
||||
results, memory peaks, and response sizes are recorded in
|
||||
[`MILESTONE_3_BASELINE.md`](MILESTONE_3_BASELINE.md) and
|
||||
[`benchmarks/milestone3-2026-07-29.json`](../benchmarks/milestone3-2026-07-29.json).
|
||||
|
||||
## Preserved boundaries
|
||||
|
||||
- The `docforge` distribution, package, CLI, MCP executable, and existing tool names remain.
|
||||
- The frozen alpha manual remains exactly 2,043 bytes with its legacy output hash and render
|
||||
identity.
|
||||
- Legacy one-method `load_projection()` adapters remain supported.
|
||||
- Effective policy version 1, no-AST behavior, and existing client bindings remain compatible.
|
||||
- Project descriptor schema version 1 and SQLite index schema version 3 remain unchanged.
|
||||
- Configured `max_render_bytes` values above the detached transfer ceiling still load; a small
|
||||
actual artifact renders normally. Actual detached transfer remains capped at 20,000,000 bytes.
|
||||
- No storage replacement or self-hosting dependency was introduced.
|
||||
- WorldForge and ScrapeStation were not touched.
|
||||
- No production MCP integration was repointed.
|
||||
- The legacy Forgejo repository and `legacy` remote were not changed.
|
||||
- No tag, release, release announcement, or visibility change was created.
|
||||
|
||||
## Known follow-up work
|
||||
|
||||
Milestone 4 remains directional and is not active. Its adapter SDK and product-documentation work
|
||||
must not silently absorb these separate future ideas:
|
||||
|
||||
- Optimize production fragment reuse only from measured profiles while preserving the forced-full
|
||||
oracle.
|
||||
- Add authenticated cursors only if a stronger threat model requires them.
|
||||
- Verify Claude's native timeout representation.
|
||||
- Add versioned adapter-owned launcher metadata before generating custom-adapter configurations.
|
||||
- Keep remote render services, shared render farms, third-party renderers, storage replacement,
|
||||
and self-hosting deferred until their own evidence justifies them.
|
||||
|
|
@ -27,6 +27,10 @@ incremental methods while retaining the full loader as a fallback.
|
|||
- Opt-in incremental adapter extraction with reverse-dependency invalidation.
|
||||
- Lazy function-scoped logic projections that do not densify the primary graph.
|
||||
- Declared HTML render views. Arbitrary templates, render commands, and output paths are rejected.
|
||||
- Separate versioned manual and portable graph plans, immutable packages, detached built-in
|
||||
renderers, and validated receipts.
|
||||
- Content-addressed portable Nodes/Flow/Web artifacts with receipt-only status and repair.
|
||||
- Independent manual, portable-graph, and live-viewer policy.
|
||||
- A loopback-only graph browser with Nodes, semantic Flow, and convergence Web views,
|
||||
relationship keys, source inspection, branch-aware node hiding, panel resizing, zooming, and
|
||||
managed idle shutdown.
|
||||
|
|
@ -47,7 +51,11 @@ Canonical files own facts:
|
|||
canonical Markdown/TOML or adapter sources
|
||||
↓ validate
|
||||
disposable SQLite graph
|
||||
↓ query / visualize / compile context
|
||||
├── query / compile context
|
||||
├── ManualRenderPlanV1 → detached manual renderer → declared manual
|
||||
├── GraphViewPlanV1 → detached graph renderer → portable Nodes/Flow/Web artifact
|
||||
└── pinned index → managed live Nodes/Flow/Web/Logic viewer
|
||||
↓
|
||||
people and agents
|
||||
↓ propose
|
||||
isolated changeset + preview
|
||||
|
|
@ -77,8 +85,8 @@ source format.
|
|||
Clone and verify DocForge:
|
||||
|
||||
```bash
|
||||
git clone forgejo@repo.andraxion.net:administrator/DocForge.git /absolute/path/DocForge
|
||||
cd /absolute/path/DocForge
|
||||
git clone forgejo@repo.andraxion.net:administrator/DocForge2.git /absolute/path/DocForge2
|
||||
cd /absolute/path/DocForge2
|
||||
uv sync --group dev
|
||||
npm ci
|
||||
|
||||
|
|
@ -159,6 +167,27 @@ output = "Docs/Rendered/Manual.html"
|
|||
title = "My Project Manual"
|
||||
families = ["architecture", "system", "operations", "roadmap"]
|
||||
|
||||
[graph_render]
|
||||
output_root = ".docforge/portable-graph"
|
||||
|
||||
[[graph_render.views]]
|
||||
id = "architecture"
|
||||
renderer = "portable_graph_html"
|
||||
output = "architecture.html"
|
||||
title = "Architecture"
|
||||
root = "architecture.overview"
|
||||
initial_mode = "web"
|
||||
depth = 3
|
||||
max_nodes = 250
|
||||
max_edges = 1000
|
||||
max_work = 100000
|
||||
families = ["architecture", "system"]
|
||||
relations = ["depends_on", "owns", "calls", "reads", "writes", "tested_by", "relates_to"]
|
||||
authorities = []
|
||||
statuses = ["current", "active", "verified"]
|
||||
tags = []
|
||||
include_logic = false
|
||||
|
||||
[graph]
|
||||
allowed_relations = ["depends_on", "owns", "calls", "reads", "writes", "tested_by", "relates_to"]
|
||||
|
||||
|
|
@ -492,13 +521,25 @@ builds or repairs the index.
|
|||
```text
|
||||
render-status [VIEW_ID] [--deep]
|
||||
render VIEW_ID
|
||||
graph-plan VIEW_ID
|
||||
graph-render VIEW_ID
|
||||
graph-render-status [VIEW_ID]
|
||||
preview CHANGESET_ID VIEW_ID
|
||||
apply CHANGESET_ID --changeset-hash SHA256 --applier WRITER_ID
|
||||
```
|
||||
|
||||
`graph-plan` validates and returns one declared `GraphViewPlanV1` without publishing. A portable
|
||||
view must select exactly one stable `root` or metadata-only lexical `query`. It may use only Nodes,
|
||||
Flow, or Web as `initial_mode`; portable version 1 excludes function-scoped Logic.
|
||||
|
||||
`graph-render` explicitly publishes the declared static artifact, content-addressed renderer
|
||||
evidence, and generation/view manifest. `graph-render-status` verifies only bounded committed
|
||||
evidence and never plans or renders. Portable publication is a local CLI action.
|
||||
|
||||
The CLI apply command supports the generic adapter. It verifies that the configured writer owns the
|
||||
changeset, applies the exact reviewed hash, rebuilds the index, checks it, and regenerates every
|
||||
declared render. It does not commit or push the result.
|
||||
declared manual render when manual policy is `auto`. It does not publish portable graphs, commit,
|
||||
or push the result.
|
||||
|
||||
### Viewer commands
|
||||
|
||||
|
|
@ -549,6 +590,20 @@ declares the named writer, and application requires the same writer/applier iden
|
|||
`--no-ast` to preserve the no-AST binding. Generic CLI generation refuses project-owned adapters
|
||||
because it cannot safely reconstruct their composition.
|
||||
|
||||
Select projection behavior independently:
|
||||
|
||||
```bash
|
||||
docforge configure codex \
|
||||
--project /absolute/path/MyProject \
|
||||
--manual-render-policy explicit \
|
||||
--portable-graph-policy disabled \
|
||||
--live-viewer-policy on-demand
|
||||
```
|
||||
|
||||
The generated version-1 configuration result carries an additive version-2 `projection_policy`,
|
||||
its hash, projection availability, and the exact descriptor hash. Omitted default selectors are
|
||||
validated against that descriptor rather than trusted as self-reported output.
|
||||
|
||||
Inspect one configured client binding:
|
||||
|
||||
```bash
|
||||
|
|
@ -569,6 +624,54 @@ starts MCP, executes the configured command, synchronizes, builds, renders, star
|
|||
writes configuration. Claude timeout representation and client filtering that cannot be proved
|
||||
locally remain explicit warnings.
|
||||
|
||||
## Independent projection behavior
|
||||
|
||||
Manual and portable graph renderers consume immutable, path-free packages. A package binds one
|
||||
generation-pinned plan, inert assets, fixed component versions, a built-in renderer identity, and
|
||||
an exact artifact inventory. The detached child cannot select nodes, open the project or index,
|
||||
choose a publication path, execute project code, or mutate canonical facts.
|
||||
|
||||
Child startup is fixed to isolated Python, a private module entrypoint, a trusted working
|
||||
directory, and a sanitized environment. One request and response use canonical newline-terminated
|
||||
JSON. The request, response, receipt, execution time, and disk-spooled stdout are bounded. Actual
|
||||
artifact transfer is capped at 20,000,000 bytes even when the descriptor retains a larger
|
||||
`max_render_bytes` compatibility value.
|
||||
|
||||
Manual fragment records are disposable semantic cache entries. On a cold miss, DocForge performs a
|
||||
trusted full detached render, extracts candidate page fragments, and compares fragment-assisted
|
||||
output byte-for-byte before publishing records. On a warm hit, the worker recomputes each expected
|
||||
page fragment before accepting cached bytes. Corrupt, forged, stale, individually oversized, or
|
||||
aggregate-oversized records fall back to the full oracle. Fragment reuse is currently a correctness
|
||||
and recovery boundary, not a promised speedup.
|
||||
|
||||
Projection policy version 2 is:
|
||||
|
||||
```text
|
||||
manual: auto | explicit | disabled
|
||||
portable_graph: explicit | disabled
|
||||
live_viewer: on-demand | disabled
|
||||
```
|
||||
|
||||
For ordinary CLI commands, place the corresponding global flag before the subcommand:
|
||||
|
||||
```bash
|
||||
docforge --project-root "$PROJECT" --manual-render-policy disabled render manual
|
||||
docforge --project-root "$PROJECT" --portable-graph-policy disabled graph-plan architecture
|
||||
docforge --project-root "$PROJECT" --live-viewer-policy disabled visualize
|
||||
```
|
||||
|
||||
An active operation blocked by policy returns `projection_policy_forbids_operation` before hidden
|
||||
work. Manual and portable receipt-only status remain available. Viewer status and explicit stop
|
||||
remain available when viewer start is disabled.
|
||||
|
||||
A non-disabled projection also requires its declared configuration or runtime. Manual `explicit`
|
||||
requires manual render configuration. Manual `auto` additionally requires canonical application in
|
||||
the current operation or server capability. Portable graph `explicit` requires portable graph
|
||||
render configuration, and live viewer `on-demand` requires its runtime. An unavailable selection
|
||||
returns `projection_policy_unavailable` before work begins. In particular, ordinary CLI `render`
|
||||
operations cannot select manual `auto`; use `explicit`, or let a configured canonical `apply`
|
||||
operation own automatic regeneration.
|
||||
|
||||
## MCP usage
|
||||
|
||||
Run one MCP server per project with absolute paths:
|
||||
|
|
@ -586,7 +689,10 @@ Select the session's declared surface explicitly when useful:
|
|||
```bash
|
||||
docforge-mcp \
|
||||
--project-root /absolute/path/MyProject \
|
||||
--capability-mode read
|
||||
--capability-mode read \
|
||||
--manual-render-policy explicit \
|
||||
--portable-graph-policy explicit \
|
||||
--live-viewer-policy on-demand
|
||||
```
|
||||
|
||||
Supported modes are `read`, `proposal`, `application`, and `operator`. Existing startup defaults
|
||||
|
|
@ -613,8 +719,9 @@ identity, not a command. The changeset creator, configured writer, and canonical
|
|||
|
||||
Call `docforge_bootstrap` first. Its version-1 `session_contract` contains the fixed binding,
|
||||
current graph generation, effective policy, actual capabilities, render policies, prohibitions,
|
||||
and a recommended first operation. Workflow guidance does not recommend registration or
|
||||
application when those startup capabilities are unavailable.
|
||||
and a recommended first operation. The result also carries the independently composed version-2
|
||||
`projection_policy` and hash. Workflow guidance does not recommend registration or application
|
||||
when those startup capabilities are unavailable.
|
||||
|
||||
Example MCP client configuration:
|
||||
|
||||
|
|
@ -653,11 +760,16 @@ Example MCP client configuration:
|
|||
- `docforge_get_task_context`
|
||||
- `docforge_validate_project`
|
||||
- `docforge_render_status`
|
||||
- `docforge_graph_plan`
|
||||
- `docforge_graph_render_status`
|
||||
- `docforge_visualize`
|
||||
- `docforge_visualization_status`
|
||||
- `docforge_stop_visualization`
|
||||
- `docforge_get_generation_diff`
|
||||
|
||||
MCP graph plan and status are read-only. MCP does not expose portable graph publication; use the
|
||||
explicit local `graph-render` CLI command.
|
||||
|
||||
### Proposal tools
|
||||
|
||||
- `docforge_create_changeset`
|
||||
|
|
@ -976,8 +1088,29 @@ docforge --project-root "$PROJECT" render-status
|
|||
docforge --project-root "$PROJECT" render VIEW_ID
|
||||
```
|
||||
|
||||
Successful canonical apply regenerates all declared views automatically. A manual canonical edit
|
||||
requires reindexing and rendering.
|
||||
Successful canonical apply regenerates declared manual views only when manual policy is `auto`.
|
||||
A manual canonical edit requires reindexing and explicit rendering. Portable graph publication
|
||||
always remains a separate explicit CLI action.
|
||||
|
||||
Portable graph publication has separate status and policy:
|
||||
|
||||
```bash
|
||||
docforge --project-root "$PROJECT" graph-render-status
|
||||
docforge --project-root "$PROJECT" graph-render architecture
|
||||
```
|
||||
|
||||
### `projection_policy_forbids_operation`
|
||||
|
||||
The process was deliberately started with the relevant manual, portable-graph, or live-viewer
|
||||
operation disabled. Restart with an allowed selector after confirming that the integration should
|
||||
receive that capability. Status and explicit stop operations remain available as described above.
|
||||
|
||||
### `projection_policy_unavailable`
|
||||
|
||||
The selected non-disabled projection has no matching project configuration or runtime. Add the
|
||||
declared manual or portable graph render configuration, or make the live viewer runtime available,
|
||||
before selecting that mode. Manual `auto` also requires an operation or MCP server with canonical
|
||||
application enabled. Use manual `explicit` for a standalone CLI render.
|
||||
|
||||
### Descriptor changed after startup
|
||||
|
||||
|
|
@ -992,8 +1125,11 @@ Run the complete release gate from the DocForge repository:
|
|||
make gate
|
||||
```
|
||||
|
||||
Use `make benchmark` for the historical Milestone 0 baseline and `make benchmark-m1` for the
|
||||
counter-gated 1,000-node warm-operation benchmark.
|
||||
Use `make benchmark` for the historical Milestone 0 baseline, `make benchmark-m1` for the
|
||||
counter-gated warm-operation benchmark, `make benchmark-m2` for agent workflow gates, and
|
||||
`make benchmark-m3-full` for the ten-sample 1,000-node projection, worker, fragment, status,
|
||||
equivalence, response-size, and memory gates. `make accessibility` runs the generated manual,
|
||||
portable graph, and live viewer axe and keyboard flows.
|
||||
|
||||
Project-specific vocabulary, extraction rules, and serialization belong in the project adapter.
|
||||
Generic core behavior must remain deterministic, project-bound, and recoverable.
|
||||
|
|
|
|||
|
|
@ -21,4 +21,26 @@ export default [
|
|||
"prefer-const": "error",
|
||||
},
|
||||
},
|
||||
{
|
||||
files: ["playwright.accessibility.config.mjs", "tests/accessibility.spec.mjs"],
|
||||
...js.configs.recommended,
|
||||
languageOptions: {
|
||||
ecmaVersion: 2024,
|
||||
sourceType: "module",
|
||||
globals: {
|
||||
...globals.browser,
|
||||
...globals.node,
|
||||
},
|
||||
},
|
||||
linterOptions: {
|
||||
reportUnusedDisableDirectives: "error",
|
||||
},
|
||||
rules: {
|
||||
...js.configs.recommended.rules,
|
||||
eqeqeq: "error",
|
||||
"no-implicit-coercion": "error",
|
||||
"no-var": "error",
|
||||
"prefer-const": "error",
|
||||
},
|
||||
},
|
||||
];
|
||||
|
|
|
|||
88
package-lock.json
generated
88
package-lock.json
generated
|
|
@ -8,7 +8,9 @@
|
|||
"name": "docforge-web-quality",
|
||||
"version": "0.0.0",
|
||||
"devDependencies": {
|
||||
"@axe-core/playwright": "4.12.1",
|
||||
"@eslint/js": "10.0.1",
|
||||
"@playwright/test": "1.62.0",
|
||||
"eslint": "10.8.0",
|
||||
"globals": "17.7.0",
|
||||
"html-validate": "11.5.6",
|
||||
|
|
@ -18,6 +20,19 @@
|
|||
"stylelint-csstree-validator": "4.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@axe-core/playwright": {
|
||||
"version": "4.12.1",
|
||||
"resolved": "https://registry.npmjs.org/@axe-core/playwright/-/playwright-4.12.1.tgz",
|
||||
"integrity": "sha512-rMd7xriptqKpP+w5265i4Hdkv2X5kbu6uiBi/B2I7uf3hieRBM3qDCfaKPtxfiYb2mKXfF+yLODJwIx+Jv1GDw==",
|
||||
"dev": true,
|
||||
"license": "MPL-2.0",
|
||||
"dependencies": {
|
||||
"axe-core": "~4.12.1"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"playwright-core": ">= 1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@babel/code-frame": {
|
||||
"version": "7.29.7",
|
||||
"resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.29.7.tgz",
|
||||
|
|
@ -515,6 +530,22 @@
|
|||
"node": ">= 8"
|
||||
}
|
||||
},
|
||||
"node_modules/@playwright/test": {
|
||||
"version": "1.62.0",
|
||||
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.62.0.tgz",
|
||||
"integrity": "sha512-9zOJ6ZQRAena31MpOH9VSzIz8Ou3YJ/wtY/eQm5T2uhfhG7/U3COrMS8xOtUrZrp9OgdmzEnIYODye3nY1VqzA==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"playwright": "1.62.0"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
},
|
||||
"node_modules/@sindresorhus/merge-streams": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@sindresorhus/merge-streams/-/merge-streams-4.0.0.tgz",
|
||||
|
|
@ -635,6 +666,16 @@
|
|||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/axe-core": {
|
||||
"version": "4.12.1",
|
||||
"resolved": "https://registry.npmjs.org/axe-core/-/axe-core-4.12.1.tgz",
|
||||
"integrity": "sha512-s7iGf5GaVMxEG0ENN9x+xTr7GFZCb1ZP/1uATUpCEK2X78nDB3RwbtFCo9pGAf9ru+VwoQ464DkaLEeRM08wJA==",
|
||||
"dev": true,
|
||||
"license": "MPL-2.0",
|
||||
"engines": {
|
||||
"node": ">=4"
|
||||
}
|
||||
},
|
||||
"node_modules/balanced-match": {
|
||||
"version": "4.0.4",
|
||||
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz",
|
||||
|
|
@ -1943,6 +1984,53 @@
|
|||
"url": "https://github.com/sponsors/jonschlinkert"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright": {
|
||||
"version": "1.62.0",
|
||||
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.0.tgz",
|
||||
"integrity": "sha512-Z14dG305dgaLu6foB1TXQagFiW8JfSUIUaUuPaKQ6NtBPKF1P/qXcqfh6c6K/icPqdy37JmjbiBXf6JNg6Sylw==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"playwright-core": "1.62.0"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"fsevents": "2.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright-core": {
|
||||
"version": "1.62.0",
|
||||
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.0.tgz",
|
||||
"integrity": "sha512-nsNRyq0r2zsG8AcRHWknc9QRA5XCueC7gWMrs+Gx2tlZn9hcl8zudfh00lhJPY1DE7NmZ6bDsT9g2yey8mXljA==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"playwright-core": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright/node_modules/fsevents": {
|
||||
"version": "2.3.2",
|
||||
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
|
||||
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
|
||||
"dev": true,
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/postcss": {
|
||||
"version": "8.5.23",
|
||||
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.23.tgz",
|
||||
|
|
|
|||
|
|
@ -4,10 +4,14 @@
|
|||
"private": true,
|
||||
"packageManager": "npm@10.9.7",
|
||||
"scripts": {
|
||||
"lint:web": "uv run python tools/check_web_assets.py"
|
||||
"install:accessibility-browser": "playwright install chromium",
|
||||
"lint:web": "uv run python tools/check_web_assets.py && eslint --max-warnings=0 playwright.accessibility.config.mjs tests/accessibility.spec.mjs",
|
||||
"test:accessibility": "npm run install:accessibility-browser && playwright test --config=playwright.accessibility.config.mjs"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@axe-core/playwright": "4.12.1",
|
||||
"@eslint/js": "10.0.1",
|
||||
"@playwright/test": "1.62.0",
|
||||
"eslint": "10.8.0",
|
||||
"globals": "17.7.0",
|
||||
"html-validate": "11.5.6",
|
||||
|
|
|
|||
24
playwright.accessibility.config.mjs
Normal file
24
playwright.accessibility.config.mjs
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
import { defineConfig } from "@playwright/test";
|
||||
|
||||
export default defineConfig({
|
||||
testDir: "./tests",
|
||||
testMatch: "accessibility.spec.mjs",
|
||||
fullyParallel: false,
|
||||
workers: 1,
|
||||
retries: 0,
|
||||
reporter: "line",
|
||||
outputDir: "/tmp/docforge-playwright-accessibility",
|
||||
timeout: 30_000,
|
||||
expect: {
|
||||
timeout: 5_000,
|
||||
},
|
||||
use: {
|
||||
browserName: "chromium",
|
||||
bypassCSP: true,
|
||||
headless: true,
|
||||
viewport: {
|
||||
width: 1440,
|
||||
height: 1000,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
|
@ -127,6 +127,17 @@
|
|||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"projection_policy": {
|
||||
"type": "object",
|
||||
"required": ["schema_version", "manual", "portable_graph", "live_viewer"],
|
||||
"properties": {
|
||||
"schema_version": { "const": 2 },
|
||||
"manual": { "enum": ["auto", "explicit", "disabled"] },
|
||||
"portable_graph": { "enum": ["explicit", "disabled"] },
|
||||
"live_viewer": { "enum": ["on-demand", "disabled"] }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"diagnostics": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
|
|
@ -211,6 +222,9 @@
|
|||
"project",
|
||||
"binding",
|
||||
"effective_policy",
|
||||
"projection_policy",
|
||||
"projection_policy_hash",
|
||||
"projection_availability",
|
||||
"artifact",
|
||||
"configuration_hash",
|
||||
"warnings"
|
||||
|
|
@ -231,7 +245,8 @@
|
|||
"project_id",
|
||||
"project_root",
|
||||
"project_root_fingerprint",
|
||||
"adapter"
|
||||
"adapter",
|
||||
"descriptor_hash"
|
||||
],
|
||||
"properties": {
|
||||
"project_id": { "type": "string", "minLength": 1 },
|
||||
|
|
@ -240,7 +255,8 @@
|
|||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{16}$"
|
||||
},
|
||||
"adapter": { "type": "string", "minLength": 1 }
|
||||
"adapter": { "type": "string", "minLength": 1 },
|
||||
"descriptor_hash": { "$ref": "#/$defs/sha256" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
|
|
@ -317,6 +333,24 @@
|
|||
"additionalProperties": false
|
||||
},
|
||||
"effective_policy": { "$ref": "#/$defs/effective_policy" },
|
||||
"projection_policy": { "$ref": "#/$defs/projection_policy" },
|
||||
"projection_policy_hash": { "$ref": "#/$defs/sha256" },
|
||||
"projection_availability": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"manual_configured",
|
||||
"portable_graph_configured",
|
||||
"application_enabled",
|
||||
"live_viewer_available"
|
||||
],
|
||||
"properties": {
|
||||
"manual_configured": { "type": "boolean" },
|
||||
"portable_graph_configured": { "type": "boolean" },
|
||||
"application_enabled": { "type": "boolean" },
|
||||
"live_viewer_available": { "const": true }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"artifact": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
|
|
|
|||
|
|
@ -74,6 +74,82 @@
|
|||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"graph_render": {
|
||||
"type": "object",
|
||||
"required": ["output_root", "views"],
|
||||
"properties": {
|
||||
"output_root": { "$ref": "#/$defs/relativePath" },
|
||||
"views": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["id", "renderer", "output", "title"],
|
||||
"properties": {
|
||||
"id": { "type": "string", "pattern": "^[a-z0-9][a-z0-9._-]{1,127}$" },
|
||||
"renderer": { "const": "portable_graph_html" },
|
||||
"output": {
|
||||
"allOf": [
|
||||
{ "$ref": "#/$defs/relativePath" },
|
||||
{ "pattern": "\\.html$" }
|
||||
]
|
||||
},
|
||||
"title": { "type": "string", "minLength": 1, "maxLength": 1024 },
|
||||
"root": { "type": "string", "minLength": 1, "maxLength": 1024 },
|
||||
"query": { "type": "string", "minLength": 1, "maxLength": 10000 },
|
||||
"initial_mode": { "enum": ["nodes", "flow", "web"] },
|
||||
"depth": { "type": "integer", "minimum": 1, "maximum": 32 },
|
||||
"max_nodes": { "type": "integer", "minimum": 1, "maximum": 1000 },
|
||||
"max_edges": { "type": "integer", "minimum": 0, "maximum": 4000 },
|
||||
"max_work": { "type": "integer", "minimum": 1, "maximum": 1000000 },
|
||||
"families": {
|
||||
"type": "array",
|
||||
"maxItems": 64,
|
||||
"uniqueItems": true,
|
||||
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
|
||||
},
|
||||
"relations": {
|
||||
"type": "array",
|
||||
"maxItems": 64,
|
||||
"uniqueItems": true,
|
||||
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
|
||||
},
|
||||
"authorities": {
|
||||
"type": "array",
|
||||
"maxItems": 64,
|
||||
"uniqueItems": true,
|
||||
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
|
||||
},
|
||||
"statuses": {
|
||||
"type": "array",
|
||||
"maxItems": 64,
|
||||
"uniqueItems": true,
|
||||
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
|
||||
},
|
||||
"tags": {
|
||||
"type": "array",
|
||||
"maxItems": 64,
|
||||
"uniqueItems": true,
|
||||
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
|
||||
},
|
||||
"include_logic": { "const": false }
|
||||
},
|
||||
"oneOf": [
|
||||
{
|
||||
"required": ["root"],
|
||||
"not": { "required": ["query"] }
|
||||
},
|
||||
{
|
||||
"required": ["query"],
|
||||
"not": { "required": ["root"] }
|
||||
}
|
||||
],
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"graph": {
|
||||
"type": "object",
|
||||
"required": ["allowed_relations"],
|
||||
|
|
|
|||
19
schemas/projection-policy.schema.json
Normal file
19
schemas/projection-policy.schema.json
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://docforge.local/schema/projection-policy-v2.json",
|
||||
"title": "DocForge independent projection policy",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schema_version",
|
||||
"manual",
|
||||
"portable_graph",
|
||||
"live_viewer"
|
||||
],
|
||||
"properties": {
|
||||
"schema_version": { "const": 2 },
|
||||
"manual": { "enum": ["auto", "explicit", "disabled"] },
|
||||
"portable_graph": { "enum": ["explicit", "disabled"] },
|
||||
"live_viewer": { "enum": ["on-demand", "disabled"] }
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
|
|
@ -20,6 +20,7 @@
|
|||
"test",
|
||||
"benchmark.m1",
|
||||
"benchmark.m2",
|
||||
"benchmark.m3",
|
||||
"mcp.invoke",
|
||||
"mcp.bootstrap",
|
||||
"mcp.sync",
|
||||
|
|
@ -37,6 +38,8 @@
|
|||
"mcp.generation_diff",
|
||||
"mcp.validate_project",
|
||||
"mcp.render_status",
|
||||
"mcp.graph_plan",
|
||||
"mcp.graph_render_status",
|
||||
"mcp.visualize",
|
||||
"mcp.visualization_status",
|
||||
"mcp.stop_visualization",
|
||||
|
|
@ -58,6 +61,9 @@
|
|||
"cli.impact",
|
||||
"cli.context",
|
||||
"cli.generation-diff",
|
||||
"cli.graph-plan",
|
||||
"cli.graph-render",
|
||||
"cli.graph-render-status",
|
||||
"cli.configure",
|
||||
"cli.doctor",
|
||||
"cli.render",
|
||||
|
|
|
|||
|
|
@ -3,7 +3,10 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import secrets
|
||||
import stat
|
||||
from collections.abc import Callable
|
||||
from contextlib import suppress
|
||||
from pathlib import Path
|
||||
|
||||
from .errors import DocForgeError
|
||||
|
|
@ -69,3 +72,177 @@ def require_bound_directory(path: Path, directory_fd: int) -> None:
|
|||
"path_escape",
|
||||
"Derived cache root disappeared during publication",
|
||||
) from error
|
||||
|
||||
|
||||
def open_confined_directory(root: Path, path: Path, *, create: bool) -> int:
|
||||
"""Open a descendant directory through stable no-follow directory descriptors."""
|
||||
|
||||
try:
|
||||
unsafe = (
|
||||
root.is_symlink()
|
||||
or root.resolve(strict=True) != root
|
||||
or not path.is_relative_to(root)
|
||||
or path == root
|
||||
)
|
||||
except OSError as error:
|
||||
raise DocForgeError("path_escape", "Project root cannot be resolved safely") from error
|
||||
if unsafe:
|
||||
raise DocForgeError("path_escape", "Derived output directory is not confined")
|
||||
relative = path.relative_to(root)
|
||||
try:
|
||||
descriptor = os.open(root, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
|
||||
except OSError as error:
|
||||
raise DocForgeError("path_escape", "Project root cannot be opened safely") from error
|
||||
try:
|
||||
for part in relative.parts:
|
||||
if part in {"", ".", ".."}:
|
||||
raise DocForgeError("path_escape", "Derived output directory is not confined")
|
||||
if create:
|
||||
try:
|
||||
os.mkdir(part, mode=0o700, dir_fd=descriptor)
|
||||
except FileExistsError:
|
||||
pass
|
||||
except OSError as error:
|
||||
raise DocForgeError(
|
||||
"publication_failure",
|
||||
"Derived output directory could not be created",
|
||||
) from error
|
||||
try:
|
||||
next_descriptor = os.open(
|
||||
part,
|
||||
os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW,
|
||||
dir_fd=descriptor,
|
||||
)
|
||||
except OSError as error:
|
||||
raise DocForgeError(
|
||||
"path_escape",
|
||||
"Derived output directory is missing or unsafe",
|
||||
) from error
|
||||
os.close(descriptor)
|
||||
descriptor = next_descriptor
|
||||
require_bound_directory(path, descriptor)
|
||||
return descriptor
|
||||
except Exception:
|
||||
os.close(descriptor)
|
||||
raise
|
||||
|
||||
|
||||
def safe_file_identity_at(
|
||||
directory: Path,
|
||||
directory_fd: int,
|
||||
name: str,
|
||||
) -> dict[str, object] | None:
|
||||
"""Return one no-follow regular-file identity relative to a bound directory."""
|
||||
|
||||
del directory
|
||||
if not name or "/" in name or name in {".", ".."}:
|
||||
raise DocForgeError("path_escape", "Derived artifact name is unsafe")
|
||||
try:
|
||||
current = os.stat(name, dir_fd=directory_fd, follow_symlinks=False)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except OSError as error:
|
||||
raise DocForgeError("path_escape", "Derived artifact cannot be inspected") from error
|
||||
if not stat.S_ISREG(current.st_mode):
|
||||
raise DocForgeError("path_escape", "Derived artifact is not a safe regular file")
|
||||
return {
|
||||
"path": name,
|
||||
"device": current.st_dev,
|
||||
"inode": current.st_ino,
|
||||
"mode": current.st_mode,
|
||||
"size": current.st_size,
|
||||
"mtime_ns": current.st_mtime_ns,
|
||||
"ctime_ns": current.st_ctime_ns,
|
||||
}
|
||||
|
||||
|
||||
def read_bounded_file_at(
|
||||
directory_fd: int,
|
||||
name: str,
|
||||
maximum_bytes: int,
|
||||
) -> bytes | None:
|
||||
"""Read one regular file through a bound directory without following links."""
|
||||
|
||||
try:
|
||||
descriptor = os.open(name, os.O_RDONLY | os.O_NOFOLLOW, dir_fd=directory_fd)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except OSError as error:
|
||||
raise DocForgeError("path_escape", "Derived artifact cannot be opened safely") from error
|
||||
with os.fdopen(descriptor, "rb") as handle:
|
||||
current = os.fstat(handle.fileno())
|
||||
if not stat.S_ISREG(current.st_mode) or current.st_size > maximum_bytes:
|
||||
raise DocForgeError("invalid_projection", "Derived artifact is invalid or oversized")
|
||||
content = handle.read(maximum_bytes + 1)
|
||||
if len(content) > maximum_bytes:
|
||||
raise DocForgeError("invalid_projection", "Derived artifact is oversized")
|
||||
return content
|
||||
|
||||
|
||||
def atomic_replace_bytes_at(
|
||||
path: Path,
|
||||
directory_fd: int,
|
||||
name: str,
|
||||
content: bytes,
|
||||
*,
|
||||
verify: Callable[[], None],
|
||||
) -> dict[str, object]:
|
||||
"""Durably replace one file inside an already bound directory."""
|
||||
|
||||
if not name or "/" in name or name in {".", ".."}:
|
||||
raise DocForgeError("path_escape", "Derived artifact name is unsafe")
|
||||
existing = safe_file_identity_at(path, directory_fd, name)
|
||||
del existing
|
||||
temporary = f".docforge-projection-{secrets.token_hex(12)}"
|
||||
descriptor: int | None = None
|
||||
committed = False
|
||||
try:
|
||||
descriptor = os.open(
|
||||
temporary,
|
||||
os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW,
|
||||
0o600,
|
||||
dir_fd=directory_fd,
|
||||
)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
descriptor = None
|
||||
handle.write(content)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
verify()
|
||||
require_bound_directory(path, directory_fd)
|
||||
os.replace(
|
||||
temporary,
|
||||
name,
|
||||
src_dir_fd=directory_fd,
|
||||
dst_dir_fd=directory_fd,
|
||||
)
|
||||
committed = True
|
||||
os.fsync(directory_fd)
|
||||
identity = safe_file_identity_at(path, directory_fd, name)
|
||||
if identity is None:
|
||||
raise DocForgeError(
|
||||
"publication_failure",
|
||||
"Derived artifact disappeared after publication",
|
||||
mutation_committed=True,
|
||||
)
|
||||
return identity
|
||||
except DocForgeError as error:
|
||||
if committed:
|
||||
raise DocForgeError(
|
||||
"publication_failure",
|
||||
"Derived artifact was replaced but final publication verification failed",
|
||||
mutation_committed=True,
|
||||
cause=error.code,
|
||||
) from error
|
||||
raise
|
||||
except OSError as error:
|
||||
raise DocForgeError(
|
||||
"publication_failure",
|
||||
"Derived artifact publication failed",
|
||||
mutation_committed=committed,
|
||||
) from error
|
||||
finally:
|
||||
if descriptor is not None:
|
||||
os.close(descriptor)
|
||||
with suppress(OSError):
|
||||
os.unlink(temporary, dir_fd=directory_fd)
|
||||
|
|
|
|||
8
src/docforge/_projection_worker_main.py
Normal file
8
src/docforge/_projection_worker_main.py
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
"""Private module entry point for the detached projection worker."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .projection_worker import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -15,6 +15,7 @@ from .changesets import ChangesetStore
|
|||
from .errors import DocForgeError
|
||||
from .index import ProjectIndex
|
||||
from .models import Node, ProjectService, ProjectSnapshot
|
||||
from .projection_policy import ManualProjectionMode, validate_manual_projection_mode
|
||||
from .rendering import RenderService
|
||||
|
||||
|
||||
|
|
@ -353,13 +354,19 @@ class CanonicalApplicationService:
|
|||
applier_id: str | None,
|
||||
applier: CanonicalApplier | None,
|
||||
index: ProjectIndex | None = None,
|
||||
manual_policy: ManualProjectionMode = "auto",
|
||||
) -> None:
|
||||
self.project = project
|
||||
self.applier_id = applier_id
|
||||
self.applier = applier
|
||||
self.changesets = ChangesetStore(project, applier_id)
|
||||
self.index = index or ProjectIndex(project)
|
||||
self.rendering = RenderService(project, self.changesets)
|
||||
self.manual_policy = validate_manual_projection_mode(manual_policy)
|
||||
self.rendering = RenderService(
|
||||
project,
|
||||
self.changesets,
|
||||
manual_policy=self.manual_policy,
|
||||
)
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
|
|
@ -402,7 +409,9 @@ class CanonicalApplicationService:
|
|||
)
|
||||
renders: list[dict[str, object]] = []
|
||||
config = self.project.descriptor.render
|
||||
if config is not None:
|
||||
render_action = "not_configured"
|
||||
if config is not None and self.manual_policy == "auto":
|
||||
render_action = "rendered"
|
||||
for view in config.views:
|
||||
try:
|
||||
rendered = self.rendering.render(view.view_id)
|
||||
|
|
@ -435,6 +444,10 @@ class CanonicalApplicationService:
|
|||
"error": error.as_dict(),
|
||||
}
|
||||
)
|
||||
elif config is not None:
|
||||
render_action = (
|
||||
"skipped_explicit" if self.manual_policy == "explicit" else "skipped_disabled"
|
||||
)
|
||||
return {
|
||||
**applied,
|
||||
"derived_refresh": {
|
||||
|
|
@ -442,6 +455,10 @@ class CanonicalApplicationService:
|
|||
"index": index_result,
|
||||
"check": index_check,
|
||||
"renders": renders,
|
||||
"render_policy": {
|
||||
"mode": self.manual_policy,
|
||||
"action": render_action,
|
||||
},
|
||||
"errors": refresh_errors,
|
||||
},
|
||||
}
|
||||
|
|
|
|||
|
|
@ -95,7 +95,7 @@
|
|||
aria-label="Visible relationship color and symbol key"></ul>
|
||||
</details>
|
||||
<svg id="graph" viewBox="-600 -410 1200 820"
|
||||
role="img" aria-label="Node neighborhood"></svg>
|
||||
role="group" aria-label="Interactive node neighborhood"></svg>
|
||||
<div class="empty" id="empty">Search for a node to inspect its neighborhood.</div>
|
||||
<div class="connection-state" id="connection-state" role="alert" hidden>
|
||||
<strong>Visualization disconnected</strong>
|
||||
|
|
|
|||
|
|
@ -13,9 +13,11 @@ from .client_config import CLIENT_NAMES, generate_client_configuration
|
|||
from .context import compile_context
|
||||
from .doctor import run_doctor
|
||||
from .errors import DocForgeError
|
||||
from .graph_rendering import GraphRenderService
|
||||
from .index import ProjectIndex
|
||||
from .onboarding import assess_project, scaffold_project
|
||||
from .project import Project, project_root_fingerprint
|
||||
from .projection_policy import compose_projection_policy
|
||||
from .rendering import RenderService
|
||||
from .telemetry import request
|
||||
from .viewer_manager import ViewerManagerClient
|
||||
|
|
@ -29,6 +31,18 @@ def _parser() -> argparse.ArgumentParser:
|
|||
action="store_true",
|
||||
help="Attach bounded request-local stage timings and counters",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--manual-render-policy",
|
||||
choices=("auto", "explicit", "disabled"),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--portable-graph-policy",
|
||||
choices=("explicit", "disabled"),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--live-viewer-policy",
|
||||
choices=("on-demand", "disabled"),
|
||||
)
|
||||
commands = parser.add_subparsers(dest="command", required=True)
|
||||
configure = commands.add_parser("configure")
|
||||
configure.add_argument("client", choices=CLIENT_NAMES)
|
||||
|
|
@ -42,6 +56,21 @@ def _parser() -> argparse.ArgumentParser:
|
|||
configure.add_argument("--proposal-writer")
|
||||
configure.add_argument("--canonical-applier")
|
||||
configure.add_argument("--no-ast", action="store_true")
|
||||
configure.add_argument(
|
||||
"--manual-render-policy",
|
||||
choices=("auto", "explicit", "disabled"),
|
||||
default=argparse.SUPPRESS,
|
||||
)
|
||||
configure.add_argument(
|
||||
"--portable-graph-policy",
|
||||
choices=("explicit", "disabled"),
|
||||
default=argparse.SUPPRESS,
|
||||
)
|
||||
configure.add_argument(
|
||||
"--live-viewer-policy",
|
||||
choices=("on-demand", "disabled"),
|
||||
default=argparse.SUPPRESS,
|
||||
)
|
||||
configure.add_argument("--startup-timeout", type=int, default=30)
|
||||
configure.add_argument("--tool-timeout", type=int, default=300)
|
||||
configure.add_argument("--output", type=Path)
|
||||
|
|
@ -95,6 +124,12 @@ def _parser() -> argparse.ArgumentParser:
|
|||
render_status = commands.add_parser("render-status")
|
||||
render_status.add_argument("view_id", nargs="?")
|
||||
render_status.add_argument("--deep", action="store_true")
|
||||
graph_plan = commands.add_parser("graph-plan")
|
||||
graph_plan.add_argument("view_id")
|
||||
graph_render = commands.add_parser("graph-render")
|
||||
graph_render.add_argument("view_id")
|
||||
graph_render_status = commands.add_parser("graph-render-status")
|
||||
graph_render_status.add_argument("view_id", nargs="?")
|
||||
preview = commands.add_parser("preview")
|
||||
preview.add_argument("changeset_id")
|
||||
preview.add_argument("view_id")
|
||||
|
|
@ -124,6 +159,9 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
|
|||
proposal_writer=arguments.proposal_writer,
|
||||
canonical_applier=arguments.canonical_applier,
|
||||
no_ast=arguments.no_ast,
|
||||
manual_render_policy=arguments.manual_render_policy,
|
||||
portable_graph_policy=arguments.portable_graph_policy,
|
||||
live_viewer_policy=arguments.live_viewer_policy,
|
||||
startup_timeout=arguments.startup_timeout,
|
||||
tool_timeout=arguments.tool_timeout,
|
||||
output=arguments.output,
|
||||
|
|
@ -152,12 +190,39 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
|
|||
content_root=arguments.content_root,
|
||||
)
|
||||
project = Project.open(arguments.project_root)
|
||||
projection_policy = compose_projection_policy(
|
||||
manual=arguments.manual_render_policy,
|
||||
portable_graph=arguments.portable_graph_policy,
|
||||
live_viewer=arguments.live_viewer_policy,
|
||||
manual_configured=project.descriptor.render is not None,
|
||||
portable_graph_configured=project.descriptor.graph_render is not None,
|
||||
application_enabled=False,
|
||||
)
|
||||
build = ProjectIndex(project).build()
|
||||
render = RenderService(project).render("manual")
|
||||
render = (
|
||||
{
|
||||
"status": "ok",
|
||||
"state": "skipped",
|
||||
"reason": "projection_policy_disabled",
|
||||
}
|
||||
if projection_policy.manual == "disabled"
|
||||
else RenderService(
|
||||
project,
|
||||
manual_policy=projection_policy.manual,
|
||||
).render("manual")
|
||||
)
|
||||
return {**scaffold, "build": build, "render": render}
|
||||
return assess_project(arguments.project_root, requested_languages=languages)
|
||||
project = Project.open(arguments.project_root)
|
||||
index = ProjectIndex(project)
|
||||
projection_policy = compose_projection_policy(
|
||||
manual=arguments.manual_render_policy,
|
||||
portable_graph=arguments.portable_graph_policy,
|
||||
live_viewer=arguments.live_viewer_policy,
|
||||
manual_configured=project.descriptor.render is not None,
|
||||
portable_graph_configured=project.descriptor.graph_render is not None,
|
||||
application_enabled=arguments.command == "apply",
|
||||
)
|
||||
if arguments.command == "info":
|
||||
snapshot = project.load()
|
||||
return {
|
||||
|
|
@ -250,24 +315,52 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
|
|||
cursor=arguments.cursor,
|
||||
)
|
||||
if arguments.command == "render":
|
||||
return RenderService(project).render(arguments.view_id)
|
||||
return RenderService(
|
||||
project,
|
||||
manual_policy=projection_policy.manual,
|
||||
).render(arguments.view_id)
|
||||
if arguments.command == "render-status":
|
||||
rendering = RenderService(project)
|
||||
rendering = RenderService(
|
||||
project,
|
||||
manual_policy=projection_policy.manual,
|
||||
)
|
||||
return (
|
||||
rendering.deep_status(arguments.view_id)
|
||||
if arguments.deep
|
||||
else rendering.status(arguments.view_id)
|
||||
)
|
||||
if arguments.command == "graph-plan":
|
||||
return GraphRenderService(
|
||||
project,
|
||||
portable_graph_policy=projection_policy.portable_graph,
|
||||
).plan(arguments.view_id)
|
||||
if arguments.command == "graph-render":
|
||||
return GraphRenderService(
|
||||
project,
|
||||
portable_graph_policy=projection_policy.portable_graph,
|
||||
).render(arguments.view_id)
|
||||
if arguments.command == "graph-render-status":
|
||||
return GraphRenderService(
|
||||
project,
|
||||
portable_graph_policy=projection_policy.portable_graph,
|
||||
).status(arguments.view_id)
|
||||
if arguments.command == "preview":
|
||||
return RenderService(project).preview(arguments.changeset_id, arguments.view_id)
|
||||
return RenderService(
|
||||
project,
|
||||
manual_policy=projection_policy.manual,
|
||||
).preview(arguments.changeset_id, arguments.view_id)
|
||||
if arguments.command == "apply":
|
||||
return CanonicalApplicationService(
|
||||
project,
|
||||
applier_id=arguments.applier,
|
||||
applier=GenericCanonicalApplier(project),
|
||||
manual_policy=projection_policy.manual,
|
||||
).apply(arguments.changeset_id, arguments.changeset_hash)
|
||||
if arguments.command == "visualize":
|
||||
visualization = ViewerManagerClient(index).start(
|
||||
visualization = ViewerManagerClient(
|
||||
index,
|
||||
live_viewer_policy=projection_policy.live_viewer,
|
||||
).start(
|
||||
node_id=arguments.node,
|
||||
query=arguments.query,
|
||||
depth=arguments.depth,
|
||||
|
|
@ -284,9 +377,15 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
|
|||
"visualization": visualization,
|
||||
}
|
||||
if arguments.command == "visualization-status":
|
||||
return ViewerManagerClient(index).status()
|
||||
return ViewerManagerClient(
|
||||
index,
|
||||
live_viewer_policy=projection_policy.live_viewer,
|
||||
).status()
|
||||
if arguments.command == "visualization-stop":
|
||||
return ViewerManagerClient(index).stop()
|
||||
return ViewerManagerClient(
|
||||
index,
|
||||
live_viewer_policy=projection_policy.live_viewer,
|
||||
).stop()
|
||||
raise DocForgeError("invalid_command", "Unknown command")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -18,9 +18,10 @@ from typing import Literal, cast
|
|||
|
||||
from .changeset_contract import document_hash
|
||||
from .errors import DocForgeError
|
||||
from .models import ProjectService
|
||||
from .models import ProjectDescriptor, ProjectService
|
||||
from .policy import CapabilityMode, compose_effective_policy
|
||||
from .project import project_root_fingerprint, validate_descriptor_binding
|
||||
from .project import Project, project_root_fingerprint, validate_descriptor_binding
|
||||
from .projection_policy import compose_projection_policy
|
||||
|
||||
ClientName = Literal["codex", "claude", "openclaw"]
|
||||
CLIENT_NAMES: tuple[ClientName, ...] = ("codex", "claude", "openclaw")
|
||||
|
|
@ -605,11 +606,37 @@ def _atomic_write(
|
|||
os.close(directory_fd)
|
||||
|
||||
|
||||
def _validate_configuration_result(result: dict[str, object]) -> None:
|
||||
def _validate_configuration_result(
|
||||
result: dict[str, object],
|
||||
*,
|
||||
trusted_descriptor: ProjectDescriptor | None = None,
|
||||
) -> None:
|
||||
artifact = cast(dict[str, object], result["artifact"])
|
||||
binding = cast(dict[str, object], result["binding"])
|
||||
policy = cast(dict[str, object], result["effective_policy"])
|
||||
projection_policy = cast(dict[str, object], result["projection_policy"])
|
||||
projection_availability = cast(
|
||||
dict[str, object],
|
||||
result["projection_availability"],
|
||||
)
|
||||
project = cast(dict[str, object], result["project"])
|
||||
if trusted_descriptor is None:
|
||||
try:
|
||||
bound_descriptor = Project.open(cast(str, project["project_root"])).descriptor
|
||||
except (DocForgeError, KeyError, TypeError) as error:
|
||||
raise AssertionError(
|
||||
"Generated client project binding cannot be independently validated"
|
||||
) from error
|
||||
else:
|
||||
bound_descriptor = trusted_descriptor
|
||||
if (
|
||||
project["project_id"] != bound_descriptor.project_id
|
||||
or project["project_root"] != str(bound_descriptor.root)
|
||||
or project["project_root_fingerprint"] != project_root_fingerprint(bound_descriptor.root)
|
||||
or project["adapter"] != bound_descriptor.adapter
|
||||
or project["descriptor_hash"] != bound_descriptor.descriptor_hash
|
||||
):
|
||||
raise AssertionError("Generated client project binding drifted")
|
||||
content = cast(str, artifact["content"])
|
||||
if artifact["content_sha256"] != hashlib.sha256(content.encode("utf-8")).hexdigest():
|
||||
raise AssertionError("Generated client content hash drifted")
|
||||
|
|
@ -644,6 +671,29 @@ def _validate_configuration_result(result: dict[str, object]) -> None:
|
|||
if remaining[-1:] != ["--no-ast"] or arguments.count("--no-ast") != 1:
|
||||
raise AssertionError("Generated no-AST argument layout drifted")
|
||||
remaining = remaining[:-1]
|
||||
projection_arguments: dict[str, str] = {}
|
||||
authority_arguments: list[str] = []
|
||||
position = 0
|
||||
projection_options = {
|
||||
"--manual-render-policy": "manual",
|
||||
"--portable-graph-policy": "portable_graph",
|
||||
"--live-viewer-policy": "live_viewer",
|
||||
}
|
||||
while position < len(remaining):
|
||||
option = remaining[position]
|
||||
field = projection_options.get(option)
|
||||
if field is None:
|
||||
authority_arguments.append(option)
|
||||
position += 1
|
||||
continue
|
||||
if position + 1 >= len(remaining) or option in projection_arguments:
|
||||
raise AssertionError("Generated projection policy argument layout drifted")
|
||||
value = remaining[position + 1]
|
||||
projection_arguments[option] = value
|
||||
if projection_policy[field] != value:
|
||||
raise AssertionError("Generated projection policy argument drifted")
|
||||
position += 2
|
||||
remaining = authority_arguments
|
||||
mode = binding["capability_mode"]
|
||||
if (
|
||||
(mode == "read" and remaining)
|
||||
|
|
@ -663,6 +713,34 @@ def _validate_configuration_result(result: dict[str, object]) -> None:
|
|||
)
|
||||
):
|
||||
raise AssertionError("Generated authority argument layout drifted")
|
||||
expected_projection_policy = compose_projection_policy(
|
||||
manual=projection_arguments.get("--manual-render-policy"),
|
||||
portable_graph=projection_arguments.get("--portable-graph-policy"),
|
||||
live_viewer=projection_arguments.get("--live-viewer-policy"),
|
||||
manual_configured=cast(bool, projection_availability["manual_configured"]),
|
||||
portable_graph_configured=cast(
|
||||
bool,
|
||||
projection_availability["portable_graph_configured"],
|
||||
),
|
||||
application_enabled=cast(
|
||||
bool,
|
||||
projection_availability["application_enabled"],
|
||||
),
|
||||
live_viewer_available=cast(
|
||||
bool,
|
||||
projection_availability["live_viewer_available"],
|
||||
),
|
||||
)
|
||||
if (
|
||||
projection_policy != expected_projection_policy.as_dict()
|
||||
or projection_availability["manual_configured"] != (render_policy["manual"] != "disabled")
|
||||
or projection_availability["manual_configured"] != (bound_descriptor.render is not None)
|
||||
or projection_availability["portable_graph_configured"]
|
||||
!= (bound_descriptor.graph_render is not None)
|
||||
or projection_availability["application_enabled"] != (mode == "application")
|
||||
or projection_availability["live_viewer_available"] is not True
|
||||
):
|
||||
raise AssertionError("Generated projection policy drifted from its availability")
|
||||
composed_policy = compose_effective_policy(
|
||||
selected_mode=cast(CapabilityMode, mode),
|
||||
capability_source="explicit",
|
||||
|
|
@ -698,6 +776,18 @@ def _validate_configuration_result(result: dict[str, object]) -> None:
|
|||
)
|
||||
):
|
||||
raise AssertionError("Generated client policy drifted from its binding")
|
||||
if (
|
||||
result["projection_policy_hash"]
|
||||
!= hashlib.sha256(
|
||||
json.dumps(
|
||||
projection_policy,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
).encode("utf-8")
|
||||
).hexdigest()
|
||||
):
|
||||
raise AssertionError("Generated projection policy hash drifted")
|
||||
expected_hash = document_hash(
|
||||
{
|
||||
"schema_version": 1,
|
||||
|
|
@ -706,6 +796,9 @@ def _validate_configuration_result(result: dict[str, object]) -> None:
|
|||
"project": project,
|
||||
"binding": binding,
|
||||
"effective_policy": policy,
|
||||
"projection_policy": projection_policy,
|
||||
"projection_policy_hash": result["projection_policy_hash"],
|
||||
"projection_availability": projection_availability,
|
||||
"artifact_format": artifact["format"],
|
||||
"artifact_content_sha256": artifact["content_sha256"],
|
||||
}
|
||||
|
|
@ -723,6 +816,9 @@ def generate_client_configuration(
|
|||
proposal_writer: str | None = None,
|
||||
canonical_applier: str | None = None,
|
||||
no_ast: bool = False,
|
||||
manual_render_policy: str | None = None,
|
||||
portable_graph_policy: str | None = None,
|
||||
live_viewer_policy: str | None = None,
|
||||
startup_timeout: int = 30,
|
||||
tool_timeout: int = 300,
|
||||
output: Path | None = None,
|
||||
|
|
@ -839,9 +935,6 @@ def generate_client_configuration(
|
|||
arguments.extend(("--proposal-writer", proposal_writer))
|
||||
if canonical_applier is not None:
|
||||
arguments.extend(("--canonical-applier", canonical_applier))
|
||||
if no_ast:
|
||||
arguments.append("--no-ast")
|
||||
|
||||
policy = compose_effective_policy(
|
||||
selected_mode=selected_mode,
|
||||
capability_source="explicit",
|
||||
|
|
@ -850,6 +943,40 @@ def generate_client_configuration(
|
|||
render_configured=descriptor.render is not None,
|
||||
application_enabled=canonical_applier is not None,
|
||||
)
|
||||
projection_policy = compose_projection_policy(
|
||||
manual=manual_render_policy,
|
||||
portable_graph=portable_graph_policy,
|
||||
live_viewer=live_viewer_policy,
|
||||
manual_configured=descriptor.render is not None,
|
||||
portable_graph_configured=descriptor.graph_render is not None,
|
||||
application_enabled=canonical_applier is not None,
|
||||
)
|
||||
default_projection_policy = compose_projection_policy(
|
||||
manual_configured=descriptor.render is not None,
|
||||
portable_graph_configured=descriptor.graph_render is not None,
|
||||
application_enabled=canonical_applier is not None,
|
||||
)
|
||||
for option, selected, default in (
|
||||
(
|
||||
"--manual-render-policy",
|
||||
projection_policy.manual,
|
||||
default_projection_policy.manual,
|
||||
),
|
||||
(
|
||||
"--portable-graph-policy",
|
||||
projection_policy.portable_graph,
|
||||
default_projection_policy.portable_graph,
|
||||
),
|
||||
(
|
||||
"--live-viewer-policy",
|
||||
projection_policy.live_viewer,
|
||||
default_projection_policy.live_viewer,
|
||||
),
|
||||
):
|
||||
if selected != default:
|
||||
arguments.extend((option, selected))
|
||||
if no_ast:
|
||||
arguments.append("--no-ast")
|
||||
artifact_format, content, warning = _artifact(
|
||||
selected_client,
|
||||
server_name=selected_name,
|
||||
|
|
@ -902,6 +1029,7 @@ def generate_client_configuration(
|
|||
"project_root": str(descriptor.root),
|
||||
"project_root_fingerprint": fingerprint,
|
||||
"adapter": descriptor.adapter,
|
||||
"descriptor_hash": descriptor.descriptor_hash,
|
||||
}
|
||||
policy_payload = policy.as_dict()
|
||||
plan_hash = document_hash(
|
||||
|
|
@ -912,6 +1040,14 @@ def generate_client_configuration(
|
|||
"project": project_binding,
|
||||
"binding": binding,
|
||||
"effective_policy": policy_payload,
|
||||
"projection_policy": projection_policy.as_dict(),
|
||||
"projection_policy_hash": projection_policy.policy_hash,
|
||||
"projection_availability": {
|
||||
"manual_configured": descriptor.render is not None,
|
||||
"portable_graph_configured": descriptor.graph_render is not None,
|
||||
"application_enabled": canonical_applier is not None,
|
||||
"live_viewer_available": True,
|
||||
},
|
||||
"artifact_format": artifact_format,
|
||||
"artifact_content_sha256": artifact["content_sha256"],
|
||||
}
|
||||
|
|
@ -926,6 +1062,14 @@ def generate_client_configuration(
|
|||
"project": project_binding,
|
||||
"binding": binding,
|
||||
"effective_policy": policy_payload,
|
||||
"projection_policy": projection_policy.as_dict(),
|
||||
"projection_policy_hash": projection_policy.policy_hash,
|
||||
"projection_availability": {
|
||||
"manual_configured": descriptor.render is not None,
|
||||
"portable_graph_configured": descriptor.graph_render is not None,
|
||||
"application_enabled": canonical_applier is not None,
|
||||
"live_viewer_available": True,
|
||||
},
|
||||
"artifact": artifact,
|
||||
"configuration_hash": plan_hash,
|
||||
"warnings": [
|
||||
|
|
@ -933,5 +1077,5 @@ def generate_client_configuration(
|
|||
*([] if publication_warning is None else [{"code": publication_warning}]),
|
||||
],
|
||||
}
|
||||
_validate_configuration_result(result)
|
||||
_validate_configuration_result(result, trusted_descriptor=descriptor)
|
||||
return result
|
||||
|
|
|
|||
|
|
@ -20,6 +20,7 @@ from .project import (
|
|||
project_root_fingerprint,
|
||||
validate_descriptor_binding,
|
||||
)
|
||||
from .projection_policy import compose_projection_policy
|
||||
|
||||
MAX_CLIENT_CONFIG_BYTES = 1_000_000
|
||||
MAX_CLIENT_SERVERS = 256
|
||||
|
|
@ -614,6 +615,9 @@ def _parse_binding(arguments: list[str]) -> dict[str, object]:
|
|||
"--proposal-writer",
|
||||
"--canonical-applier",
|
||||
"--capability-mode",
|
||||
"--manual-render-policy",
|
||||
"--portable-graph-policy",
|
||||
"--live-viewer-policy",
|
||||
}
|
||||
flag_options = {"--no-ast", "--diagnostics"}
|
||||
position = 0
|
||||
|
|
@ -662,6 +666,9 @@ def _parse_binding(arguments: list[str]) -> dict[str, object]:
|
|||
"capability_mode_implicit": implicit,
|
||||
"no_ast": "--no-ast" in flags,
|
||||
"diagnostics": "--diagnostics" in flags,
|
||||
"manual_render_policy": values.get("--manual-render-policy"),
|
||||
"portable_graph_policy": values.get("--portable-graph-policy"),
|
||||
"live_viewer_policy": values.get("--live-viewer-policy"),
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -749,6 +756,16 @@ def _runtime_policy_check(
|
|||
canonical_applier is not None and selected_mode in {"application", "operator"}
|
||||
),
|
||||
)
|
||||
compose_projection_policy(
|
||||
manual=cast(str | None, binding["manual_render_policy"]),
|
||||
portable_graph=cast(str | None, binding["portable_graph_policy"]),
|
||||
live_viewer=cast(str | None, binding["live_viewer_policy"]),
|
||||
manual_configured=project.descriptor.render is not None,
|
||||
portable_graph_configured=project.descriptor.graph_render is not None,
|
||||
application_enabled=(
|
||||
canonical_applier is not None and selected_mode in {"application", "operator"}
|
||||
),
|
||||
)
|
||||
if cast(bool, binding["capability_mode_implicit"]):
|
||||
return (
|
||||
"warning",
|
||||
|
|
|
|||
|
|
@ -9,18 +9,20 @@ from typing import Literal, cast
|
|||
from .errors import DocForgeError
|
||||
from .models import Edge, Node, ProjectSnapshot
|
||||
from .project import project_root_fingerprint
|
||||
from .projection_contract import GraphViewPlanV1
|
||||
from .projection_contract import (
|
||||
MAX_GRAPH_VIEW_DEPTH,
|
||||
MAX_GRAPH_VIEW_EDGES,
|
||||
MAX_GRAPH_VIEW_FILTERS,
|
||||
MAX_GRAPH_VIEW_NODES,
|
||||
MAX_GRAPH_VIEW_QUERY_CHARS,
|
||||
MAX_GRAPH_VIEW_STRING_CHARS,
|
||||
MAX_GRAPH_VIEW_WORK,
|
||||
GraphViewPlanV1,
|
||||
ProjectionPackageV1,
|
||||
)
|
||||
|
||||
GraphViewMode = Literal["nodes", "flow", "web", "logic"]
|
||||
|
||||
MAX_GRAPH_VIEW_DEPTH = 32
|
||||
MAX_GRAPH_VIEW_NODES = 1_000
|
||||
MAX_GRAPH_VIEW_EDGES = 4_000
|
||||
MAX_GRAPH_VIEW_WORK = 1_000_000
|
||||
MAX_GRAPH_VIEW_FILTERS = 64
|
||||
MAX_GRAPH_VIEW_STRING_CHARS = 1_024
|
||||
MAX_GRAPH_VIEW_QUERY_CHARS = 10_000
|
||||
|
||||
_QUERY_TOKEN = re.compile(r"\w+", re.UNICODE)
|
||||
_DETAIL_FIELDS = (
|
||||
"node_id",
|
||||
|
|
@ -495,3 +497,29 @@ def build_graph_view_plan(
|
|||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def build_graph_projection_package(
|
||||
plan: GraphViewPlanV1,
|
||||
*,
|
||||
renderer_id: str,
|
||||
renderer_version: str,
|
||||
max_output_bytes: int,
|
||||
) -> ProjectionPackageV1:
|
||||
"""Bind a graph plan to the fixed portable renderer without adding runtime authority."""
|
||||
|
||||
return ProjectionPackageV1.create(
|
||||
kind="graph",
|
||||
plan=plan,
|
||||
renderer={"renderer_id": renderer_id, "renderer_version": renderer_version},
|
||||
components=[
|
||||
{"component_id": "graph.portable-document@1"},
|
||||
{"component_id": "graph.accessible-list@1"},
|
||||
{"component_id": "graph.relationship-table@1"},
|
||||
],
|
||||
assets=[],
|
||||
output_policy={
|
||||
"artifact_ids": ["portable-graph.html"],
|
||||
"max_total_bytes": max_output_bytes,
|
||||
},
|
||||
)
|
||||
|
|
|
|||
285
src/docforge/graph_render_config.py
Normal file
285
src/docforge/graph_render_config.py
Normal file
|
|
@ -0,0 +1,285 @@
|
|||
"""Strict parsing and confinement for optional portable graph artifacts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
from .config_validation import (
|
||||
ID_PATTERN,
|
||||
confined_path,
|
||||
positive_int,
|
||||
require_string,
|
||||
string_list,
|
||||
)
|
||||
from .errors import DocForgeError
|
||||
from .models import GraphRenderConfig, GraphRenderView, Limits, RenderConfig
|
||||
from .projection_contract import (
|
||||
MAX_GRAPH_VIEW_DEPTH,
|
||||
MAX_GRAPH_VIEW_EDGES,
|
||||
MAX_GRAPH_VIEW_FILTERS,
|
||||
MAX_GRAPH_VIEW_NODES,
|
||||
MAX_GRAPH_VIEW_QUERY_CHARS,
|
||||
MAX_GRAPH_VIEW_STRING_CHARS,
|
||||
MAX_GRAPH_VIEW_WORK,
|
||||
)
|
||||
|
||||
_CONFIG_KEYS = frozenset({"output_root", "views"})
|
||||
_VIEW_KEYS = frozenset(
|
||||
{
|
||||
"id",
|
||||
"renderer",
|
||||
"output",
|
||||
"title",
|
||||
"root",
|
||||
"query",
|
||||
"initial_mode",
|
||||
"depth",
|
||||
"max_nodes",
|
||||
"max_edges",
|
||||
"max_work",
|
||||
"families",
|
||||
"relations",
|
||||
"authorities",
|
||||
"statuses",
|
||||
"tags",
|
||||
"include_logic",
|
||||
}
|
||||
)
|
||||
_MODES = frozenset({"nodes", "flow", "web"})
|
||||
|
||||
|
||||
def _overlaps(first: Path, second: Path) -> bool:
|
||||
return first == second or first.is_relative_to(second) or second.is_relative_to(first)
|
||||
|
||||
|
||||
def _optional_string(document: dict[str, object], key: str, source: Path) -> str | None:
|
||||
if key not in document:
|
||||
return None
|
||||
return require_string(document, key, source)
|
||||
|
||||
|
||||
def _bounded_string(
|
||||
document: dict[str, object],
|
||||
key: str,
|
||||
source: Path,
|
||||
*,
|
||||
maximum: int,
|
||||
) -> str:
|
||||
value = require_string(document, key, source)
|
||||
if len(value) > maximum:
|
||||
raise DocForgeError("invalid_config", f"{key} exceeds its fixed character limit")
|
||||
return value
|
||||
|
||||
|
||||
def _bounded_strings(
|
||||
value: object,
|
||||
*,
|
||||
key: str,
|
||||
source: Path,
|
||||
) -> tuple[str, ...]:
|
||||
values = string_list(value, key=key, source=source)
|
||||
if len(values) > MAX_GRAPH_VIEW_FILTERS or any(
|
||||
len(item) > MAX_GRAPH_VIEW_STRING_CHARS for item in values
|
||||
):
|
||||
raise DocForgeError("invalid_config", f"{key} exceeds its fixed bounds")
|
||||
return values
|
||||
|
||||
|
||||
def load_graph_render_config(
|
||||
root: Path,
|
||||
document: object,
|
||||
*,
|
||||
descriptor_path: Path,
|
||||
content_roots: tuple[Path, ...],
|
||||
authority_files: tuple[Path, ...],
|
||||
cache_root: Path,
|
||||
index_path: Path,
|
||||
changeset_root: Path,
|
||||
manual_render: RenderConfig | None,
|
||||
limits: Limits,
|
||||
) -> GraphRenderConfig | None:
|
||||
if document is None:
|
||||
return None
|
||||
if not isinstance(document, dict):
|
||||
raise DocForgeError("invalid_config", "graph_render must be a table")
|
||||
document = cast(dict[str, object], document)
|
||||
unknown = sorted(set(document) - _CONFIG_KEYS)
|
||||
if unknown:
|
||||
raise DocForgeError("invalid_config", "graph_render has unknown fields", fields=unknown)
|
||||
output_root = confined_path(
|
||||
root,
|
||||
document.get("output_root"),
|
||||
field="graph_render.output_root",
|
||||
must_exist=False,
|
||||
)
|
||||
protected = [*content_roots, cache_root, changeset_root]
|
||||
if manual_render is not None:
|
||||
protected.extend((manual_render.template_root, manual_render.preview_root))
|
||||
protected.extend(view.output_path for view in manual_render.views)
|
||||
if any(_overlaps(output_root, path) for path in protected):
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph output must not overlap canonical or other derived roots",
|
||||
)
|
||||
protected_files = (descriptor_path, index_path, *authority_files)
|
||||
if any(path == output_root or path.is_relative_to(output_root) for path in protected_files):
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph output overlaps a protected project path",
|
||||
)
|
||||
view_values_value = document.get("views")
|
||||
if not isinstance(view_values_value, list) or not view_values_value:
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"graph_render.views must contain at least one view",
|
||||
)
|
||||
view_values = cast(list[object], view_values_value)
|
||||
if len(view_values) > limits.max_render_views:
|
||||
raise DocForgeError("invalid_config", "graph_render.views exceeds the configured limit")
|
||||
views: list[GraphRenderView] = []
|
||||
view_ids: set[str] = set()
|
||||
outputs: set[Path] = set()
|
||||
for value in view_values:
|
||||
if not isinstance(value, dict):
|
||||
raise DocForgeError("invalid_config", "Each portable graph view must be a table")
|
||||
view = cast(dict[str, object], value)
|
||||
unknown_view = sorted(set(view) - _VIEW_KEYS)
|
||||
if unknown_view:
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph view has unknown fields",
|
||||
fields=unknown_view,
|
||||
)
|
||||
view_id = require_string(view, "id", descriptor_path)
|
||||
if ID_PATTERN.fullmatch(view_id) is None or view_id in view_ids:
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph view ID is invalid or duplicated",
|
||||
id=view_id,
|
||||
)
|
||||
view_ids.add(view_id)
|
||||
renderer = require_string(view, "renderer", descriptor_path)
|
||||
if renderer != "portable_graph_html":
|
||||
raise DocForgeError(
|
||||
"unsupported_renderer",
|
||||
"Portable graph view names an unsupported built-in renderer",
|
||||
renderer=renderer,
|
||||
)
|
||||
output = confined_path(
|
||||
output_root,
|
||||
view.get("output"),
|
||||
field="graph_render.view.output",
|
||||
must_exist=False,
|
||||
)
|
||||
if output.suffix != ".html" or output in outputs:
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph outputs must be unique HTML files",
|
||||
)
|
||||
outputs.add(output)
|
||||
root_node_id = _optional_string(view, "root", descriptor_path)
|
||||
query = _optional_string(view, "query", descriptor_path)
|
||||
if (root_node_id is None) == (query is None):
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph view requires exactly one root or query",
|
||||
)
|
||||
if root_node_id is not None and len(root_node_id) > MAX_GRAPH_VIEW_STRING_CHARS:
|
||||
raise DocForgeError("invalid_config", "Portable graph root exceeds its fixed limit")
|
||||
if query is not None and len(query) > MAX_GRAPH_VIEW_QUERY_CHARS:
|
||||
raise DocForgeError("invalid_config", "Portable graph query exceeds its fixed limit")
|
||||
initial_mode_value = view.get("initial_mode", "nodes")
|
||||
if not isinstance(initial_mode_value, str):
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph initial mode is unsupported",
|
||||
)
|
||||
initial_mode = cast(
|
||||
Literal["nodes", "flow", "web", "logic"],
|
||||
initial_mode_value,
|
||||
)
|
||||
if initial_mode not in _MODES:
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph initial mode is unsupported",
|
||||
)
|
||||
depth = positive_int(view.get("depth", 1), "graph_render.view.depth")
|
||||
max_nodes = positive_int(view.get("max_nodes", 100), "graph_render.view.max_nodes")
|
||||
max_edges = positive_int(
|
||||
view.get("max_edges", 400),
|
||||
"graph_render.view.max_edges",
|
||||
allow_zero=True,
|
||||
)
|
||||
max_work = positive_int(view.get("max_work", 100_000), "graph_render.view.max_work")
|
||||
if (
|
||||
depth > min(limits.max_traversal_depth, MAX_GRAPH_VIEW_DEPTH)
|
||||
or max_nodes > min(limits.max_nodes, MAX_GRAPH_VIEW_NODES)
|
||||
or max_edges > MAX_GRAPH_VIEW_EDGES
|
||||
or max_work > MAX_GRAPH_VIEW_WORK
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph view exceeds project or fixed safety limits",
|
||||
)
|
||||
include_logic = view.get("include_logic", False)
|
||||
if type(include_logic) is not bool:
|
||||
raise DocForgeError(
|
||||
"invalid_config",
|
||||
"Portable graph include_logic must be Boolean",
|
||||
)
|
||||
if include_logic:
|
||||
raise DocForgeError(
|
||||
"unsupported_renderer",
|
||||
"Portable graph renderer version 1 does not support Logic projections",
|
||||
)
|
||||
views.append(
|
||||
GraphRenderView(
|
||||
view_id=view_id,
|
||||
renderer=renderer,
|
||||
output_path=output,
|
||||
title=_bounded_string(
|
||||
view,
|
||||
"title",
|
||||
descriptor_path,
|
||||
maximum=MAX_GRAPH_VIEW_STRING_CHARS,
|
||||
),
|
||||
root_node_id=root_node_id,
|
||||
query=query,
|
||||
initial_mode=initial_mode,
|
||||
depth=depth,
|
||||
max_nodes=max_nodes,
|
||||
max_edges=max_edges,
|
||||
max_work=max_work,
|
||||
families=_bounded_strings(
|
||||
view.get("families", []),
|
||||
key="graph_render.view.families",
|
||||
source=descriptor_path,
|
||||
),
|
||||
relations=_bounded_strings(
|
||||
view.get("relations", []),
|
||||
key="graph_render.view.relations",
|
||||
source=descriptor_path,
|
||||
),
|
||||
authorities=_bounded_strings(
|
||||
view.get("authorities", []),
|
||||
key="graph_render.view.authorities",
|
||||
source=descriptor_path,
|
||||
),
|
||||
statuses=_bounded_strings(
|
||||
view.get("statuses", []),
|
||||
key="graph_render.view.statuses",
|
||||
source=descriptor_path,
|
||||
),
|
||||
tags=_bounded_strings(
|
||||
view.get("tags", []),
|
||||
key="graph_render.view.tags",
|
||||
source=descriptor_path,
|
||||
),
|
||||
include_logic=include_logic,
|
||||
)
|
||||
)
|
||||
return GraphRenderConfig(
|
||||
output_root=output_root,
|
||||
views=tuple(sorted(views, key=lambda item: item.view_id)),
|
||||
)
|
||||
815
src/docforge/graph_rendering.py
Normal file
815
src/docforge/graph_rendering.py
Normal file
|
|
@ -0,0 +1,815 @@
|
|||
"""Declared portable graph planning, publication, and receipt-only status."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
from collections.abc import Callable, Generator
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from ._fs_safety import (
|
||||
atomic_replace_bytes_at,
|
||||
open_confined_directory,
|
||||
read_bounded_file_at,
|
||||
require_bound_directory,
|
||||
safe_file_identity_at,
|
||||
)
|
||||
from .errors import DocForgeError
|
||||
from .graph_projection import (
|
||||
GraphViewRequestV1,
|
||||
build_graph_projection_package,
|
||||
build_graph_view_plan,
|
||||
)
|
||||
from .models import (
|
||||
GenerationRecordingProject,
|
||||
GraphRenderConfig,
|
||||
GraphRenderView,
|
||||
IncrementalStateProject,
|
||||
ProjectService,
|
||||
ProjectSnapshot,
|
||||
ProjectState,
|
||||
)
|
||||
from .project import project_root_fingerprint
|
||||
from .projection_contract import GraphViewPlanV1, ProjectionReceiptV1, projection_hash
|
||||
from .projection_policy import (
|
||||
PortableGraphProjectionMode,
|
||||
validate_portable_graph_projection_mode,
|
||||
)
|
||||
from .projection_worker import render_projection_in_worker
|
||||
|
||||
GRAPH_RENDERER_ID = "portable_graph_html"
|
||||
GRAPH_RENDERER_VERSION = "1"
|
||||
GRAPH_PUBLICATION_MANIFEST_VERSION = 1
|
||||
GRAPH_PUBLICATION_CONTRACT = "docforge.graph-publication"
|
||||
MAX_GRAPH_PUBLICATION_BYTES = 256_000
|
||||
|
||||
|
||||
class GraphRenderService:
|
||||
"""Publish one declared artifact while keeping planning and rendering independent."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
project: ProjectService,
|
||||
*,
|
||||
allow_logic: bool = False,
|
||||
portable_graph_policy: PortableGraphProjectionMode = "explicit",
|
||||
) -> None:
|
||||
self.project = project
|
||||
self.allow_logic = allow_logic
|
||||
self.portable_graph_policy = validate_portable_graph_projection_mode(portable_graph_policy)
|
||||
|
||||
def _require_rendering(self, operation: str) -> None:
|
||||
if self.portable_graph_policy == "disabled":
|
||||
raise DocForgeError(
|
||||
"projection_policy_forbids_operation",
|
||||
"Portable graph projection policy disables rendering work",
|
||||
projection="portable_graph",
|
||||
mode=self.portable_graph_policy,
|
||||
operation=operation,
|
||||
)
|
||||
|
||||
def plan(self, view_id: str) -> dict[str, object]:
|
||||
self._require_rendering("plan")
|
||||
snapshot = self.project.load()
|
||||
view = self._view(self._config(snapshot), view_id)
|
||||
plan = self._plan(snapshot, view)
|
||||
return {
|
||||
"status": "ok",
|
||||
**self._identity(snapshot),
|
||||
"view_id": view.view_id,
|
||||
"plan": plan.as_dict(),
|
||||
}
|
||||
|
||||
def status(self, view_id: str | None = None) -> dict[str, object]:
|
||||
config = self.project.descriptor.graph_render
|
||||
current = self._current_state()
|
||||
if config is None:
|
||||
return self._status_result(
|
||||
current,
|
||||
configured=False,
|
||||
state="not_configured",
|
||||
outputs=[],
|
||||
)
|
||||
views = config.views if view_id is None else (self._view(config, view_id),)
|
||||
first_outputs = [self._manifest_status(view, current) for view in views]
|
||||
outputs = [self._manifest_status(view, current) for view in views]
|
||||
if outputs != first_outputs:
|
||||
for output in outputs:
|
||||
if output["state"] == "current":
|
||||
output["state"] = "stale"
|
||||
output["reason"] = "publication_changed_during_status"
|
||||
final = self._current_state()
|
||||
if final != current:
|
||||
for output in outputs:
|
||||
if output["state"] == "current":
|
||||
output["state"] = "stale"
|
||||
output["reason"] = "source_changed_during_status"
|
||||
identity = final if final is not None else current
|
||||
return self._status_result(
|
||||
identity,
|
||||
configured=True,
|
||||
state="current" if all(item["state"] == "current" for item in outputs) else "stale",
|
||||
outputs=outputs,
|
||||
)
|
||||
|
||||
def render(self, view_id: str) -> dict[str, object]:
|
||||
self._require_rendering("render")
|
||||
with self._lock():
|
||||
current_status = self.status(view_id)
|
||||
current_outputs = cast(list[dict[str, object]], current_status["outputs"])
|
||||
if current_status["state"] == "current" and current_outputs:
|
||||
return {
|
||||
**current_status,
|
||||
"publication": "unchanged",
|
||||
"output": current_outputs[0],
|
||||
}
|
||||
snapshot = self.project.load()
|
||||
view = self._view(self._config(snapshot), view_id)
|
||||
plan = self._plan(snapshot, view)
|
||||
package = build_graph_projection_package(
|
||||
plan,
|
||||
renderer_id=GRAPH_RENDERER_ID,
|
||||
renderer_version=GRAPH_RENDERER_VERSION,
|
||||
max_output_bytes=snapshot.descriptor.limits.max_render_bytes,
|
||||
)
|
||||
result = render_projection_in_worker(package)
|
||||
if len(result.artifacts) != 1:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Portable graph renderer returned an unsupported artifact set",
|
||||
)
|
||||
artifact = result.artifacts[0]
|
||||
|
||||
def verify() -> None:
|
||||
current = self.project.load()
|
||||
if (
|
||||
current.revision != snapshot.revision
|
||||
or current.source_hash != snapshot.source_hash
|
||||
):
|
||||
raise DocForgeError(
|
||||
"render_input_changed",
|
||||
"Canonical input changed during portable graph rendering",
|
||||
)
|
||||
|
||||
verify()
|
||||
if isinstance(self.project, GenerationRecordingProject):
|
||||
self.project.record_generation(snapshot)
|
||||
artifact_evidence = artifact.evidence()
|
||||
try:
|
||||
store_identity = self._publish_artifact(
|
||||
snapshot,
|
||||
artifact_evidence["sha256"],
|
||||
artifact.content,
|
||||
verify=verify,
|
||||
)
|
||||
except DocForgeError as error:
|
||||
if self._mutation_committed(error):
|
||||
return self._degraded_publication(
|
||||
snapshot,
|
||||
view,
|
||||
plan,
|
||||
package.package_id,
|
||||
result.receipt.as_dict(),
|
||||
artifact_evidence,
|
||||
stage="artifact_store",
|
||||
error=error,
|
||||
output_published=False,
|
||||
)
|
||||
raise
|
||||
try:
|
||||
output_identity = self._publish_output(
|
||||
snapshot,
|
||||
view,
|
||||
artifact.content,
|
||||
verify=verify,
|
||||
)
|
||||
except DocForgeError as error:
|
||||
if self._mutation_committed(error):
|
||||
return self._degraded_publication(
|
||||
snapshot,
|
||||
view,
|
||||
plan,
|
||||
package.package_id,
|
||||
result.receipt.as_dict(),
|
||||
artifact_evidence,
|
||||
stage="output",
|
||||
error=error,
|
||||
output_published=True,
|
||||
)
|
||||
raise
|
||||
manifest = self._manifest(
|
||||
snapshot,
|
||||
view,
|
||||
plan,
|
||||
package.package_id,
|
||||
result.receipt.as_dict(),
|
||||
artifact_evidence,
|
||||
store_identity,
|
||||
output_identity,
|
||||
)
|
||||
try:
|
||||
self._publish_manifest(snapshot, view, manifest, verify=verify)
|
||||
except DocForgeError as error:
|
||||
return self._degraded_publication(
|
||||
snapshot,
|
||||
view,
|
||||
plan,
|
||||
package.package_id,
|
||||
result.receipt.as_dict(),
|
||||
artifact_evidence,
|
||||
stage="manifest",
|
||||
error=error,
|
||||
output_published=True,
|
||||
)
|
||||
return {
|
||||
"status": "ok",
|
||||
**self._identity(snapshot),
|
||||
"view_id": view.view_id,
|
||||
"state": "current",
|
||||
"publication": "published",
|
||||
"plan_id": plan.plan_id,
|
||||
"package_id": package.package_id,
|
||||
"output": {
|
||||
**artifact.evidence(),
|
||||
"path": view.output_path.relative_to(snapshot.descriptor.root).as_posix(),
|
||||
},
|
||||
"receipt": result.receipt.as_dict(),
|
||||
"manifest": {
|
||||
"state": "current",
|
||||
"publication_id": manifest["publication_id"],
|
||||
},
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _mutation_committed(error: DocForgeError) -> bool:
|
||||
return error.details.get("mutation_committed") is True
|
||||
|
||||
def _degraded_publication(
|
||||
self,
|
||||
snapshot: ProjectSnapshot,
|
||||
view: GraphRenderView,
|
||||
plan: GraphViewPlanV1,
|
||||
package_id: str,
|
||||
receipt: dict[str, object],
|
||||
artifact: dict[str, object],
|
||||
*,
|
||||
stage: str,
|
||||
error: DocForgeError,
|
||||
output_published: bool,
|
||||
) -> dict[str, object]:
|
||||
return {
|
||||
"status": "ok",
|
||||
**self._identity(snapshot),
|
||||
"view_id": view.view_id,
|
||||
"state": "degraded",
|
||||
"publication": "published" if output_published else "partial",
|
||||
"committed_stage": stage,
|
||||
"plan_id": plan.plan_id,
|
||||
"package_id": package_id,
|
||||
"artifact": artifact,
|
||||
"output": {
|
||||
**artifact,
|
||||
"path": view.output_path.relative_to(snapshot.descriptor.root).as_posix(),
|
||||
"state": "unverified" if output_published else "not_published",
|
||||
},
|
||||
"receipt": receipt,
|
||||
"manifest": {
|
||||
"state": "failed",
|
||||
"error": error.as_dict(),
|
||||
},
|
||||
}
|
||||
|
||||
def _manifest_status(
|
||||
self,
|
||||
view: GraphRenderView,
|
||||
current: ProjectState | None,
|
||||
) -> dict[str, object]:
|
||||
manifest = self._read_manifest(view)
|
||||
base = {
|
||||
"view_id": view.view_id,
|
||||
"renderer": GRAPH_RENDERER_ID,
|
||||
"renderer_version": GRAPH_RENDERER_VERSION,
|
||||
"path": view.output_path.relative_to(self.project.descriptor.root).as_posix(),
|
||||
"verification": "manifest",
|
||||
}
|
||||
if manifest is None:
|
||||
return {**base, "state": "missing", "reason": "manifest_missing"}
|
||||
if not self._valid_manifest(view, manifest):
|
||||
return {**base, "state": "unverified", "reason": "manifest_invalid"}
|
||||
if current is None:
|
||||
reason = (
|
||||
"source_generation_changed"
|
||||
if isinstance(self.project, GenerationRecordingProject)
|
||||
else "source_generation_unavailable"
|
||||
)
|
||||
return {
|
||||
**base,
|
||||
"state": (
|
||||
"stale"
|
||||
if isinstance(self.project, GenerationRecordingProject)
|
||||
else "unverified"
|
||||
),
|
||||
"reason": reason,
|
||||
"plan_id": manifest.get("plan_id"),
|
||||
"package_id": manifest.get("package_id"),
|
||||
}
|
||||
project = cast(dict[str, object], manifest["project"])
|
||||
if project["revision"] != current.revision or project["source_hash"] != current.source_hash:
|
||||
return {
|
||||
**base,
|
||||
"state": "stale",
|
||||
"reason": "source_generation_changed",
|
||||
"plan_id": manifest["plan_id"],
|
||||
"package_id": manifest["package_id"],
|
||||
}
|
||||
artifact = cast(dict[str, object], manifest["artifact"])
|
||||
store = cast(dict[str, object], manifest["store"])
|
||||
artifact_root = self.project.descriptor.cache_root / "projection-artifacts"
|
||||
if not artifact_root.exists():
|
||||
return {
|
||||
**base,
|
||||
"state": "stale",
|
||||
"reason": "artifact_store_missing",
|
||||
"plan_id": manifest["plan_id"],
|
||||
"package_id": manifest["package_id"],
|
||||
}
|
||||
if artifact_root.is_symlink() or not artifact_root.is_dir():
|
||||
return {
|
||||
**base,
|
||||
"state": "unsafe",
|
||||
"reason": "artifact_store_unsafe",
|
||||
"plan_id": manifest["plan_id"],
|
||||
"package_id": manifest["package_id"],
|
||||
}
|
||||
artifact_directory: int | None = None
|
||||
try:
|
||||
artifact_directory = open_confined_directory(
|
||||
self.project.descriptor.root,
|
||||
artifact_root,
|
||||
create=False,
|
||||
)
|
||||
artifact_identity = safe_file_identity_at(
|
||||
artifact_root,
|
||||
artifact_directory,
|
||||
f"{artifact['sha256']}.html",
|
||||
)
|
||||
except DocForgeError:
|
||||
return {
|
||||
**base,
|
||||
"state": "unsafe",
|
||||
"reason": "artifact_store_unsafe",
|
||||
"plan_id": manifest["plan_id"],
|
||||
"package_id": manifest["package_id"],
|
||||
}
|
||||
finally:
|
||||
if artifact_directory is not None:
|
||||
os.close(artifact_directory)
|
||||
if artifact_identity is None:
|
||||
return {
|
||||
**base,
|
||||
"state": "stale",
|
||||
"reason": "artifact_store_missing",
|
||||
"plan_id": manifest["plan_id"],
|
||||
"package_id": manifest["package_id"],
|
||||
}
|
||||
if artifact_identity != store:
|
||||
return {
|
||||
**base,
|
||||
"state": "stale",
|
||||
"reason": "artifact_store_changed",
|
||||
"plan_id": manifest["plan_id"],
|
||||
"package_id": manifest["package_id"],
|
||||
}
|
||||
try:
|
||||
directory = open_confined_directory(
|
||||
self.project.descriptor.root,
|
||||
view.output_path.parent,
|
||||
create=False,
|
||||
)
|
||||
except DocForgeError:
|
||||
return {**base, "state": "unsafe", "reason": "output_root_unsafe"}
|
||||
try:
|
||||
identity = safe_file_identity_at(
|
||||
view.output_path.parent, directory, view.output_path.name
|
||||
)
|
||||
except DocForgeError:
|
||||
return {**base, "state": "unsafe", "reason": "output_unsafe"}
|
||||
finally:
|
||||
os.close(directory)
|
||||
expected = cast(dict[str, object], manifest["output"])
|
||||
if identity != expected:
|
||||
return {
|
||||
**base,
|
||||
"state": "stale",
|
||||
"reason": "output_changed",
|
||||
"plan_id": manifest["plan_id"],
|
||||
"package_id": manifest["package_id"],
|
||||
}
|
||||
return {
|
||||
**base,
|
||||
"state": "current",
|
||||
"reason": None,
|
||||
"plan_id": manifest["plan_id"],
|
||||
"package_id": manifest["package_id"],
|
||||
"publication_id": manifest["publication_id"],
|
||||
"artifact": manifest["artifact"],
|
||||
}
|
||||
|
||||
def _read_manifest(self, view: GraphRenderView) -> dict[str, object] | None:
|
||||
root = self._manifest_root()
|
||||
if not root.is_dir() or root.is_symlink():
|
||||
return None
|
||||
try:
|
||||
descriptor = open_confined_directory(
|
||||
self.project.descriptor.root,
|
||||
root,
|
||||
create=False,
|
||||
)
|
||||
except DocForgeError:
|
||||
return None
|
||||
try:
|
||||
raw = read_bounded_file_at(
|
||||
descriptor,
|
||||
f"{view.view_id}.json",
|
||||
MAX_GRAPH_PUBLICATION_BYTES,
|
||||
)
|
||||
except DocForgeError:
|
||||
return None
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
if raw is None:
|
||||
return None
|
||||
try:
|
||||
value: object = json.loads(raw)
|
||||
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||
return None
|
||||
return cast(dict[str, object], value) if isinstance(value, dict) else None
|
||||
|
||||
def _valid_manifest(self, view: GraphRenderView, manifest: dict[str, object]) -> bool:
|
||||
required = {
|
||||
"schema_version",
|
||||
"contract",
|
||||
"publication_id",
|
||||
"project",
|
||||
"view_id",
|
||||
"view_config_hash",
|
||||
"plan_id",
|
||||
"package_id",
|
||||
"renderer",
|
||||
"receipt",
|
||||
"artifact",
|
||||
"store",
|
||||
"output",
|
||||
}
|
||||
try:
|
||||
if (
|
||||
set(manifest) != required
|
||||
or manifest.get("schema_version") != GRAPH_PUBLICATION_MANIFEST_VERSION
|
||||
or manifest.get("contract") != GRAPH_PUBLICATION_CONTRACT
|
||||
or manifest.get("view_id") != view.view_id
|
||||
or manifest.get("view_config_hash") != self._view_hash(view)
|
||||
or not self._hash(manifest.get("plan_id"))
|
||||
or not self._hash(manifest.get("package_id"))
|
||||
):
|
||||
return False
|
||||
project = manifest.get("project")
|
||||
descriptor = self.project.descriptor
|
||||
if not isinstance(project, dict):
|
||||
return False
|
||||
project_document = cast(dict[str, object], project)
|
||||
if (
|
||||
set(project_document)
|
||||
!= {
|
||||
"project_id",
|
||||
"project_root_fingerprint",
|
||||
"adapter",
|
||||
"revision",
|
||||
"source_hash",
|
||||
}
|
||||
or project_document.get("project_id") != descriptor.project_id
|
||||
or project_document.get("project_root_fingerprint")
|
||||
!= project_root_fingerprint(descriptor.root)
|
||||
or project_document.get("adapter") != descriptor.adapter
|
||||
or not isinstance(project_document.get("revision"), str)
|
||||
or not project_document["revision"]
|
||||
or not self._hash(project_document.get("source_hash"))
|
||||
):
|
||||
return False
|
||||
renderer = manifest.get("renderer")
|
||||
if renderer != {
|
||||
"renderer_id": GRAPH_RENDERER_ID,
|
||||
"renderer_version": GRAPH_RENDERER_VERSION,
|
||||
}:
|
||||
return False
|
||||
receipt_value = manifest.get("receipt")
|
||||
if not isinstance(receipt_value, dict):
|
||||
return False
|
||||
receipt = ProjectionReceiptV1.from_dict(
|
||||
dict(cast(dict[str, object], receipt_value))
|
||||
).as_dict()
|
||||
artifacts = receipt.get("artifacts")
|
||||
if not isinstance(artifacts, list):
|
||||
return False
|
||||
artifact_values = cast(list[object], artifacts)
|
||||
if (
|
||||
receipt.get("kind") != "graph"
|
||||
or receipt.get("plan_id") != manifest["plan_id"]
|
||||
or receipt.get("package_id") != manifest["package_id"]
|
||||
or receipt.get("renderer") != renderer
|
||||
or len(artifact_values) != 1
|
||||
or manifest.get("artifact") != artifact_values[0]
|
||||
):
|
||||
return False
|
||||
artifact = artifact_values[0]
|
||||
if not isinstance(artifact, dict):
|
||||
return False
|
||||
artifact_document = cast(dict[str, object], artifact)
|
||||
if (
|
||||
artifact_document.get("artifact_id") != "portable-graph.html"
|
||||
or artifact_document.get("media_type") != "text/html; charset=utf-8"
|
||||
):
|
||||
return False
|
||||
artifact_hash = artifact_document.get("sha256")
|
||||
artifact_bytes = artifact_document.get("bytes")
|
||||
store = manifest.get("store")
|
||||
output = manifest.get("output")
|
||||
if (
|
||||
not self._file_identity(store, expected_name=f"{artifact_hash}.html")
|
||||
or not self._file_identity(output, expected_name=view.output_path.name)
|
||||
or type(artifact_bytes) is not int
|
||||
or cast(dict[str, object], store)["size"] != artifact_bytes
|
||||
or cast(dict[str, object], output)["size"] != artifact_bytes
|
||||
):
|
||||
return False
|
||||
body = dict(manifest)
|
||||
publication_id = body.pop("publication_id", None)
|
||||
return self._hash(publication_id) and publication_id == projection_hash(body)
|
||||
except (DocForgeError, KeyError, TypeError, ValueError):
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _hash(value: object) -> bool:
|
||||
return (
|
||||
isinstance(value, str)
|
||||
and len(value) == 64
|
||||
and all(character in "0123456789abcdef" for character in value)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _file_identity(value: object, *, expected_name: str) -> bool:
|
||||
if not isinstance(value, dict):
|
||||
return False
|
||||
document = cast(dict[str, object], value)
|
||||
required = {"path", "device", "inode", "mode", "size", "mtime_ns", "ctime_ns"}
|
||||
return (
|
||||
set(document) == required
|
||||
and document.get("path") == expected_name
|
||||
and all(
|
||||
type(document.get(field)) is int and cast(int, document[field]) >= 0
|
||||
for field in required - {"path"}
|
||||
)
|
||||
)
|
||||
|
||||
def _manifest(
|
||||
self,
|
||||
snapshot: ProjectSnapshot,
|
||||
view: GraphRenderView,
|
||||
plan: GraphViewPlanV1,
|
||||
package_id: str,
|
||||
receipt: dict[str, object],
|
||||
artifact: dict[str, object],
|
||||
store: dict[str, object],
|
||||
output: dict[str, object],
|
||||
) -> dict[str, object]:
|
||||
body: dict[str, object] = {
|
||||
"schema_version": GRAPH_PUBLICATION_MANIFEST_VERSION,
|
||||
"contract": GRAPH_PUBLICATION_CONTRACT,
|
||||
"project": self._identity(snapshot),
|
||||
"view_id": view.view_id,
|
||||
"view_config_hash": self._view_hash(view),
|
||||
"plan_id": plan.plan_id,
|
||||
"package_id": package_id,
|
||||
"renderer": {
|
||||
"renderer_id": GRAPH_RENDERER_ID,
|
||||
"renderer_version": GRAPH_RENDERER_VERSION,
|
||||
},
|
||||
"receipt": receipt,
|
||||
"artifact": artifact,
|
||||
"store": store,
|
||||
"output": output,
|
||||
}
|
||||
return {**body, "publication_id": projection_hash(body)}
|
||||
|
||||
def _publish_artifact(
|
||||
self,
|
||||
snapshot: ProjectSnapshot,
|
||||
artifact_hash: object,
|
||||
content: bytes,
|
||||
*,
|
||||
verify: Callable[[], None],
|
||||
) -> dict[str, object]:
|
||||
if not isinstance(artifact_hash, str):
|
||||
raise DocForgeError("invalid_projection", "Artifact hash is invalid")
|
||||
root = snapshot.descriptor.cache_root / "projection-artifacts"
|
||||
descriptor = open_confined_directory(snapshot.descriptor.root, root, create=True)
|
||||
name = f"{artifact_hash}.html"
|
||||
try:
|
||||
try:
|
||||
existing = read_bounded_file_at(descriptor, name, len(content))
|
||||
except DocForgeError as error:
|
||||
if error.code != "invalid_projection":
|
||||
raise
|
||||
existing = None
|
||||
if existing == content:
|
||||
identity = safe_file_identity_at(root, descriptor, name)
|
||||
assert identity is not None
|
||||
return identity
|
||||
return atomic_replace_bytes_at(root, descriptor, name, content, verify=verify)
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
def _publish_output(
|
||||
self,
|
||||
snapshot: ProjectSnapshot,
|
||||
view: GraphRenderView,
|
||||
content: bytes,
|
||||
*,
|
||||
verify: Callable[[], None],
|
||||
) -> dict[str, object]:
|
||||
root = view.output_path.parent
|
||||
descriptor = open_confined_directory(snapshot.descriptor.root, root, create=True)
|
||||
try:
|
||||
try:
|
||||
existing = read_bounded_file_at(
|
||||
descriptor,
|
||||
view.output_path.name,
|
||||
len(content),
|
||||
)
|
||||
except DocForgeError as error:
|
||||
if error.code != "invalid_projection":
|
||||
raise
|
||||
existing = None
|
||||
if existing == content:
|
||||
identity = safe_file_identity_at(root, descriptor, view.output_path.name)
|
||||
assert identity is not None
|
||||
return identity
|
||||
return atomic_replace_bytes_at(
|
||||
root,
|
||||
descriptor,
|
||||
view.output_path.name,
|
||||
content,
|
||||
verify=verify,
|
||||
)
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
def _publish_manifest(
|
||||
self,
|
||||
snapshot: ProjectSnapshot,
|
||||
view: GraphRenderView,
|
||||
manifest: dict[str, object],
|
||||
*,
|
||||
verify: Callable[[], None],
|
||||
) -> None:
|
||||
raw = json.dumps(manifest, sort_keys=True, indent=2).encode() + b"\n"
|
||||
if len(raw) > MAX_GRAPH_PUBLICATION_BYTES:
|
||||
raise DocForgeError(
|
||||
"projection_too_large",
|
||||
"Portable graph publication manifest exceeds its fixed limit",
|
||||
)
|
||||
root = self._manifest_root()
|
||||
descriptor = open_confined_directory(snapshot.descriptor.root, root, create=True)
|
||||
try:
|
||||
atomic_replace_bytes_at(
|
||||
root,
|
||||
descriptor,
|
||||
f"{view.view_id}.json",
|
||||
raw,
|
||||
verify=verify,
|
||||
)
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
@contextmanager
|
||||
def _lock(self) -> Generator[None]:
|
||||
root = self.project.descriptor.cache_root
|
||||
descriptor = open_confined_directory(self.project.descriptor.root, root, create=True)
|
||||
lock_descriptor: int | None = None
|
||||
try:
|
||||
lock_descriptor = os.open(
|
||||
"graph-render.lock",
|
||||
os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW,
|
||||
0o600,
|
||||
dir_fd=descriptor,
|
||||
)
|
||||
fcntl.flock(lock_descriptor, fcntl.LOCK_EX)
|
||||
require_bound_directory(root, descriptor)
|
||||
yield
|
||||
except OSError as error:
|
||||
raise DocForgeError(
|
||||
"publication_failure",
|
||||
"Portable graph render lock is unavailable",
|
||||
) from error
|
||||
finally:
|
||||
if lock_descriptor is not None:
|
||||
os.close(lock_descriptor)
|
||||
os.close(descriptor)
|
||||
|
||||
def _plan(self, snapshot: ProjectSnapshot, view: GraphRenderView) -> GraphViewPlanV1:
|
||||
return build_graph_view_plan(
|
||||
snapshot,
|
||||
GraphViewRequestV1(
|
||||
view_id=view.view_id,
|
||||
title=view.title,
|
||||
root_node_id=view.root_node_id,
|
||||
query=view.query,
|
||||
initial_mode=view.initial_mode,
|
||||
depth=view.depth,
|
||||
max_nodes=view.max_nodes,
|
||||
max_edges=view.max_edges,
|
||||
max_work=view.max_work,
|
||||
families=view.families,
|
||||
relations=view.relations,
|
||||
authorities=view.authorities,
|
||||
statuses=view.statuses,
|
||||
tags=view.tags,
|
||||
include_logic=view.include_logic,
|
||||
),
|
||||
self.allow_logic,
|
||||
)
|
||||
|
||||
def _current_state(self) -> ProjectState | None:
|
||||
if isinstance(self.project, IncrementalStateProject):
|
||||
return self.project.incremental_state()
|
||||
return None
|
||||
|
||||
def _config(self, snapshot: ProjectSnapshot) -> GraphRenderConfig:
|
||||
config = snapshot.descriptor.graph_render
|
||||
if config is None:
|
||||
raise DocForgeError(
|
||||
"graph_render_not_configured",
|
||||
"Project has no portable graph render configuration",
|
||||
)
|
||||
return config
|
||||
|
||||
@staticmethod
|
||||
def _view(config: GraphRenderConfig, view_id: str) -> GraphRenderView:
|
||||
for view in config.views:
|
||||
if view.view_id == view_id:
|
||||
return view
|
||||
raise DocForgeError(
|
||||
"unknown_graph_render_view",
|
||||
"Portable graph view is not declared",
|
||||
view_id=view_id,
|
||||
)
|
||||
|
||||
def _manifest_root(self) -> Path:
|
||||
return self.project.descriptor.cache_root / "projection-publications" / "graph"
|
||||
|
||||
@staticmethod
|
||||
def _view_hash(view: GraphRenderView) -> str:
|
||||
return projection_hash(
|
||||
{
|
||||
"view_id": view.view_id,
|
||||
"renderer": view.renderer,
|
||||
"title": view.title,
|
||||
"root_node_id": view.root_node_id,
|
||||
"query": view.query,
|
||||
"initial_mode": view.initial_mode,
|
||||
"depth": view.depth,
|
||||
"max_nodes": view.max_nodes,
|
||||
"max_edges": view.max_edges,
|
||||
"max_work": view.max_work,
|
||||
"families": list(view.families),
|
||||
"relations": list(view.relations),
|
||||
"authorities": list(view.authorities),
|
||||
"statuses": list(view.statuses),
|
||||
"tags": list(view.tags),
|
||||
"include_logic": view.include_logic,
|
||||
}
|
||||
)
|
||||
|
||||
def _status_result(self, current: ProjectState | None, **payload: object) -> dict[str, object]:
|
||||
descriptor = self.project.descriptor
|
||||
return {
|
||||
"status": "ok",
|
||||
"project_id": descriptor.project_id,
|
||||
"project_root_fingerprint": project_root_fingerprint(descriptor.root),
|
||||
"adapter": descriptor.adapter,
|
||||
"revision": current.revision if current is not None else "unknown",
|
||||
"source_hash": current.source_hash if current is not None else None,
|
||||
**payload,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _identity(snapshot: ProjectSnapshot) -> dict[str, object]:
|
||||
return {
|
||||
"project_id": snapshot.descriptor.project_id,
|
||||
"project_root_fingerprint": project_root_fingerprint(snapshot.descriptor.root),
|
||||
"adapter": snapshot.descriptor.adapter,
|
||||
"revision": snapshot.revision,
|
||||
"source_hash": snapshot.source_hash,
|
||||
}
|
||||
|
|
@ -4,6 +4,7 @@ from __future__ import annotations
|
|||
|
||||
import hashlib
|
||||
from collections import defaultdict
|
||||
from collections.abc import Sequence
|
||||
|
||||
from .errors import DocForgeError
|
||||
from .models import Edge, ProjectSnapshot, RenderView
|
||||
|
|
@ -23,47 +24,51 @@ def _cycles(node_ids: tuple[str, ...], edges: tuple[Edge, ...]) -> list[list[str
|
|||
"""Return deterministic strongly connected components that represent cycles."""
|
||||
|
||||
adjacency: dict[str, list[str]] = {node_id: [] for node_id in node_ids}
|
||||
reverse_adjacency: dict[str, list[str]] = {node_id: [] for node_id in node_ids}
|
||||
for edge in edges:
|
||||
adjacency[edge.source_id].append(edge.target_id)
|
||||
for targets in adjacency.values():
|
||||
reverse_adjacency[edge.target_id].append(edge.source_id)
|
||||
for targets in (*adjacency.values(), *reverse_adjacency.values()):
|
||||
targets.sort()
|
||||
|
||||
index = 0
|
||||
indexes: dict[str, int] = {}
|
||||
lowlinks: dict[str, int] = {}
|
||||
stack: list[str] = []
|
||||
on_stack: set[str] = set()
|
||||
components: list[list[str]] = []
|
||||
visited: set[str] = set()
|
||||
finished: list[str] = []
|
||||
for node_id in node_ids:
|
||||
if node_id in visited:
|
||||
continue
|
||||
visited.add(node_id)
|
||||
traversal: list[tuple[str, int]] = [(node_id, 0)]
|
||||
while traversal:
|
||||
current, position = traversal[-1]
|
||||
targets = adjacency[current]
|
||||
if position < len(targets):
|
||||
target = targets[position]
|
||||
traversal[-1] = (current, position + 1)
|
||||
if target not in visited:
|
||||
visited.add(target)
|
||||
traversal.append((target, 0))
|
||||
continue
|
||||
finished.append(current)
|
||||
traversal.pop()
|
||||
|
||||
def visit(node_id: str) -> None:
|
||||
nonlocal index
|
||||
indexes[node_id] = index
|
||||
lowlinks[node_id] = index
|
||||
index += 1
|
||||
stack.append(node_id)
|
||||
on_stack.add(node_id)
|
||||
for target_id in adjacency[node_id]:
|
||||
if target_id not in indexes:
|
||||
visit(target_id)
|
||||
lowlinks[node_id] = min(lowlinks[node_id], lowlinks[target_id])
|
||||
elif target_id in on_stack:
|
||||
lowlinks[node_id] = min(lowlinks[node_id], indexes[target_id])
|
||||
if lowlinks[node_id] != indexes[node_id]:
|
||||
return
|
||||
assigned: set[str] = set()
|
||||
components: list[list[str]] = []
|
||||
for node_id in reversed(finished):
|
||||
if node_id in assigned:
|
||||
continue
|
||||
assigned.add(node_id)
|
||||
component: list[str] = []
|
||||
while stack:
|
||||
member = stack.pop()
|
||||
on_stack.remove(member)
|
||||
component.append(member)
|
||||
if member == node_id:
|
||||
break
|
||||
component_stack = [node_id]
|
||||
while component_stack:
|
||||
current = component_stack.pop()
|
||||
component.append(current)
|
||||
for target in reversed(reverse_adjacency[current]):
|
||||
if target not in assigned:
|
||||
assigned.add(target)
|
||||
component_stack.append(target)
|
||||
component.sort()
|
||||
if len(component) > 1 or component[0] in adjacency[component[0]]:
|
||||
components.append(component)
|
||||
|
||||
for node_id in node_ids:
|
||||
if node_id not in indexes:
|
||||
visit(node_id)
|
||||
return sorted(components)
|
||||
|
||||
|
||||
|
|
@ -161,6 +166,8 @@ def build_manual_projection_package(
|
|||
renderer_id: str,
|
||||
renderer_version: str,
|
||||
max_output_bytes: int,
|
||||
render_identity: str | None = None,
|
||||
fragment_records: Sequence[dict[str, object]] = (),
|
||||
) -> ProjectionPackageV1:
|
||||
"""Bind one plan and inert template asset for a path-free manual renderer."""
|
||||
|
||||
|
|
@ -168,6 +175,23 @@ def build_manual_projection_package(
|
|||
template = template_bytes.decode("utf-8")
|
||||
except UnicodeDecodeError as error:
|
||||
raise DocForgeError("invalid_template", "Render template is not valid UTF-8") from error
|
||||
template_asset: dict[str, object] = {
|
||||
"asset_id": "manual.template",
|
||||
"media_type": "text/html; charset=utf-8",
|
||||
"sha256": hashlib.sha256(template_bytes).hexdigest(),
|
||||
"text": template,
|
||||
}
|
||||
if render_identity is not None:
|
||||
template_asset["render_identity"] = render_identity
|
||||
assets = [template_asset]
|
||||
if fragment_records:
|
||||
assets.append(
|
||||
{
|
||||
"asset_id": "manual.fragments",
|
||||
"media_type": "application/vnd.docforge.projection-fragments.v1+json",
|
||||
"records": list(fragment_records),
|
||||
}
|
||||
)
|
||||
return ProjectionPackageV1.create(
|
||||
kind="manual",
|
||||
plan=plan,
|
||||
|
|
@ -176,14 +200,7 @@ def build_manual_projection_package(
|
|||
{"component_id": "manual.document@1"},
|
||||
{"component_id": "manual.commonmark@1"},
|
||||
],
|
||||
assets=[
|
||||
{
|
||||
"asset_id": "manual.template",
|
||||
"media_type": "text/html; charset=utf-8",
|
||||
"sha256": hashlib.sha256(template_bytes).hexdigest(),
|
||||
"text": template,
|
||||
}
|
||||
],
|
||||
assets=assets,
|
||||
output_policy={
|
||||
"artifact_ids": ["manual.html"],
|
||||
"max_total_bytes": max_output_bytes,
|
||||
|
|
|
|||
|
|
@ -15,11 +15,13 @@ from .application import CanonicalApplicationService, CanonicalApplier, GenericC
|
|||
from .changesets import ChangesetStore
|
||||
from .context import compile_context
|
||||
from .errors import DocForgeError
|
||||
from .graph_rendering import GraphRenderService
|
||||
from .index import ProjectIndex
|
||||
from .models import IncrementalStateProject, ProjectService, RuntimeValidatedProject
|
||||
from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt
|
||||
from .policy import CapabilityMode, capability_mode, compose_effective_policy
|
||||
from .project import Project, project_root_fingerprint
|
||||
from .projection_policy import compose_projection_policy
|
||||
from .rendering import RenderService
|
||||
from .retrieval import MAX_TASK_EVIDENCE, TaskKind, build_retrieval_plan
|
||||
from .telemetry import request, stage
|
||||
|
|
@ -47,6 +49,8 @@ READ_TOOLS = (
|
|||
"docforge_get_task_context",
|
||||
"docforge_validate_project",
|
||||
"docforge_render_status",
|
||||
"docforge_graph_plan",
|
||||
"docforge_graph_render_status",
|
||||
"docforge_visualize",
|
||||
"docforge_stop_visualization",
|
||||
"docforge_visualization_status",
|
||||
|
|
@ -134,6 +138,9 @@ class DocForgeService:
|
|||
no_ast: bool = False,
|
||||
diagnostics: bool = False,
|
||||
capability_mode_name: str | None = None,
|
||||
manual_projection_policy: str | None = None,
|
||||
portable_graph_policy: str | None = None,
|
||||
live_viewer_policy: str | None = None,
|
||||
) -> None:
|
||||
self.project = project
|
||||
default_mode: CapabilityMode = (
|
||||
|
|
@ -153,19 +160,40 @@ class DocForgeService:
|
|||
render_configured=project.descriptor.render is not None,
|
||||
application_enabled=application_enabled,
|
||||
)
|
||||
self.projection_policy = compose_projection_policy(
|
||||
manual=manual_projection_policy,
|
||||
portable_graph=portable_graph_policy,
|
||||
live_viewer=live_viewer_policy,
|
||||
manual_configured=project.descriptor.render is not None,
|
||||
portable_graph_configured=project.descriptor.graph_render is not None,
|
||||
application_enabled=application_enabled,
|
||||
)
|
||||
self.index = ProjectIndex(self.project, allow_logic=not self.policy.no_ast)
|
||||
self.changesets = ChangesetStore(
|
||||
self.project,
|
||||
proposal_writer if selected_mode != "read" else None,
|
||||
)
|
||||
self.rendering = RenderService(self.project, self.changesets)
|
||||
self.rendering = RenderService(
|
||||
self.project,
|
||||
self.changesets,
|
||||
manual_policy=self.projection_policy.manual,
|
||||
)
|
||||
self.graph_rendering = GraphRenderService(
|
||||
self.project,
|
||||
allow_logic=not self.policy.no_ast,
|
||||
portable_graph_policy=self.projection_policy.portable_graph,
|
||||
)
|
||||
self.application = CanonicalApplicationService(
|
||||
self.project,
|
||||
applier_id=canonical_applier_id if application_enabled else None,
|
||||
applier=canonical_applier if application_enabled else None,
|
||||
index=self.index,
|
||||
manual_policy=self.projection_policy.manual,
|
||||
)
|
||||
self.visualization = ViewerManagerClient(
|
||||
self.index,
|
||||
live_viewer_policy=self.projection_policy.live_viewer,
|
||||
)
|
||||
self.visualization = ViewerManagerClient(self.index)
|
||||
self.context_provider = context_provider
|
||||
self.task_context_available = context_provider is compile_context
|
||||
self.binding_metadata = dict(binding_metadata or {})
|
||||
|
|
@ -621,6 +649,7 @@ class DocForgeService:
|
|||
}
|
||||
capabilities = self.capabilities()
|
||||
effective_policy = self.policy.as_dict()
|
||||
projection_policy = self.projection_policy.as_dict()
|
||||
session_contract: dict[str, object] = {
|
||||
"schema_version": 1,
|
||||
"binding": binding,
|
||||
|
|
@ -630,6 +659,8 @@ class DocForgeService:
|
|||
"freshness": "current",
|
||||
},
|
||||
"effective_policy": effective_policy,
|
||||
"projection_policy": projection_policy,
|
||||
"projection_policy_hash": self.projection_policy.policy_hash,
|
||||
"capabilities": capabilities,
|
||||
"render_policies": {
|
||||
"manual": effective_policy["manual_render"],
|
||||
|
|
@ -652,6 +683,8 @@ class DocForgeService:
|
|||
"canonical_paths": [str(path) for path in descriptor.content_roots],
|
||||
"adapter_policy": self.adapter_policy(),
|
||||
"effective_policy": effective_policy,
|
||||
"projection_policy": projection_policy,
|
||||
"projection_policy_hash": self.projection_policy.policy_hash,
|
||||
"capabilities": capabilities,
|
||||
"session_contract": session_contract,
|
||||
"proposal_access": proposal_access,
|
||||
|
|
@ -714,6 +747,8 @@ class DocForgeService:
|
|||
),
|
||||
"adapter_policy": self.adapter_policy(),
|
||||
"effective_policy": self.policy.as_dict(),
|
||||
"projection_policy": self.projection_policy.as_dict(),
|
||||
"projection_policy_hash": self.projection_policy.policy_hash,
|
||||
"capabilities": self.capabilities(),
|
||||
"canonical_paths": [
|
||||
*(relative(path) for path in snapshot.descriptor.content_roots),
|
||||
|
|
@ -829,6 +864,29 @@ class DocForgeService:
|
|||
operation_name="mcp.render_status",
|
||||
)
|
||||
|
||||
def graph_plan(self, view_id: str) -> dict[str, object]:
|
||||
"""Plan one declared portable graph without publishing derived output."""
|
||||
|
||||
return self.invoke(
|
||||
lambda: self.graph_rendering.plan(view_id),
|
||||
synchronize=False,
|
||||
load_error_identity=False,
|
||||
operation_name="mcp.graph_plan",
|
||||
)
|
||||
|
||||
def graph_render_status(
|
||||
self,
|
||||
view_id: str | None = None,
|
||||
) -> dict[str, object]:
|
||||
"""Report portable-graph publication state without planning or rendering."""
|
||||
|
||||
return self.invoke(
|
||||
lambda: self.graph_rendering.status(view_id),
|
||||
synchronize=False,
|
||||
load_error_identity=False,
|
||||
operation_name="mcp.graph_render_status",
|
||||
)
|
||||
|
||||
def context(
|
||||
self,
|
||||
profile: str,
|
||||
|
|
@ -1575,6 +1633,18 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
|
|||
|
||||
return service.render_status(view_id, deep=deep)
|
||||
|
||||
@server.tool(name="docforge_graph_plan")
|
||||
def graph_plan(view_id: str) -> dict[str, Any]:
|
||||
"""Plan one declared portable graph without publishing derived output."""
|
||||
|
||||
return service.graph_plan(view_id)
|
||||
|
||||
@server.tool(name="docforge_graph_render_status")
|
||||
def graph_render_status(view_id: str | None = None) -> dict[str, Any]:
|
||||
"""Report portable-graph publication state without rendering."""
|
||||
|
||||
return service.graph_render_status(view_id)
|
||||
|
||||
@server.tool(name="docforge_visualize")
|
||||
def visualize(
|
||||
node_id: str | None = None,
|
||||
|
|
@ -1622,6 +1692,8 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
|
|||
get_task_context,
|
||||
validate_project,
|
||||
render_status,
|
||||
graph_plan,
|
||||
graph_render_status,
|
||||
visualize,
|
||||
stop_visualization,
|
||||
visualization_status,
|
||||
|
|
@ -2004,6 +2076,9 @@ def create_server(
|
|||
no_ast: bool = False,
|
||||
diagnostics: bool = False,
|
||||
capability_mode: str | None = None,
|
||||
manual_projection_policy: str | None = None,
|
||||
portable_graph_policy: str | None = None,
|
||||
live_viewer_policy: str | None = None,
|
||||
) -> FastMCP:
|
||||
project = Project.open(project_root)
|
||||
return create_project_server(
|
||||
|
|
@ -2020,6 +2095,9 @@ def create_server(
|
|||
no_ast=no_ast,
|
||||
diagnostics=diagnostics,
|
||||
capability_mode=capability_mode,
|
||||
manual_projection_policy=manual_projection_policy,
|
||||
portable_graph_policy=portable_graph_policy,
|
||||
live_viewer_policy=live_viewer_policy,
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -2034,6 +2112,9 @@ def create_project_server(
|
|||
no_ast: bool = False,
|
||||
diagnostics: bool = False,
|
||||
capability_mode: str | None = None,
|
||||
manual_projection_policy: str | None = None,
|
||||
portable_graph_policy: str | None = None,
|
||||
live_viewer_policy: str | None = None,
|
||||
) -> FastMCP:
|
||||
"""Create the full fixed MCP surface for one explicitly configured project service."""
|
||||
|
||||
|
|
@ -2047,6 +2128,9 @@ def create_project_server(
|
|||
no_ast=no_ast,
|
||||
diagnostics=diagnostics,
|
||||
capability_mode_name=capability_mode,
|
||||
manual_projection_policy=manual_projection_policy,
|
||||
portable_graph_policy=portable_graph_policy,
|
||||
live_viewer_policy=live_viewer_policy,
|
||||
)
|
||||
return _create_bound_server(
|
||||
service,
|
||||
|
|
@ -2062,6 +2146,9 @@ def create_read_only_server(
|
|||
no_ast: bool = False,
|
||||
diagnostics: bool = False,
|
||||
capability_mode: str | None = None,
|
||||
manual_projection_policy: str | None = None,
|
||||
portable_graph_policy: str | None = None,
|
||||
live_viewer_policy: str | None = None,
|
||||
) -> FastMCP:
|
||||
"""Create an adapter-capable MCP server exposing only the fixed read tool surface."""
|
||||
|
||||
|
|
@ -2073,6 +2160,9 @@ def create_read_only_server(
|
|||
no_ast=no_ast,
|
||||
diagnostics=diagnostics,
|
||||
capability_mode_name="read" if capability_mode is None else capability_mode,
|
||||
manual_projection_policy=manual_projection_policy,
|
||||
portable_graph_policy=portable_graph_policy,
|
||||
live_viewer_policy=live_viewer_policy,
|
||||
)
|
||||
if service.policy.capability_mode != "read":
|
||||
raise DocForgeError(
|
||||
|
|
@ -2106,6 +2196,18 @@ def main() -> None:
|
|||
choices=("read", "proposal", "application", "operator"),
|
||||
help="Expose the versioned project-bound capability surface",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--manual-render-policy",
|
||||
choices=("auto", "explicit", "disabled"),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--portable-graph-policy",
|
||||
choices=("explicit", "disabled"),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--live-viewer-policy",
|
||||
choices=("on-demand", "disabled"),
|
||||
)
|
||||
arguments = parser.parse_args()
|
||||
create_server(
|
||||
arguments.project_root,
|
||||
|
|
@ -2114,6 +2216,9 @@ def main() -> None:
|
|||
no_ast=arguments.no_ast,
|
||||
diagnostics=arguments.diagnostics,
|
||||
capability_mode=arguments.capability_mode,
|
||||
manual_projection_policy=arguments.manual_render_policy,
|
||||
portable_graph_policy=arguments.portable_graph_policy,
|
||||
live_viewer_policy=arguments.live_viewer_policy,
|
||||
).run(transport="stdio")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ from __future__ import annotations
|
|||
from collections.abc import Mapping
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
from typing import Protocol, runtime_checkable
|
||||
from typing import Literal, Protocol, runtime_checkable
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
|
@ -49,6 +49,33 @@ class RenderConfig:
|
|||
views: tuple[RenderView, ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GraphRenderView:
|
||||
view_id: str
|
||||
renderer: str
|
||||
output_path: Path
|
||||
title: str
|
||||
root_node_id: str | None
|
||||
query: str | None
|
||||
initial_mode: Literal["nodes", "flow", "web", "logic"]
|
||||
depth: int
|
||||
max_nodes: int
|
||||
max_edges: int
|
||||
max_work: int
|
||||
families: tuple[str, ...]
|
||||
relations: tuple[str, ...]
|
||||
authorities: tuple[str, ...]
|
||||
statuses: tuple[str, ...]
|
||||
tags: tuple[str, ...]
|
||||
include_logic: bool
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GraphRenderConfig:
|
||||
output_root: Path
|
||||
views: tuple[GraphRenderView, ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContextProfile:
|
||||
profile_id: str
|
||||
|
|
@ -78,6 +105,7 @@ class ProjectDescriptor:
|
|||
allowed_relations: tuple[str, ...]
|
||||
profiles: tuple[ContextProfile, ...]
|
||||
limits: Limits
|
||||
graph_render: GraphRenderConfig | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ from .config_validation import (
|
|||
string_list,
|
||||
)
|
||||
from .errors import DocForgeError
|
||||
from .graph_render_config import load_graph_render_config
|
||||
from .models import (
|
||||
ContextProfile,
|
||||
Edge,
|
||||
|
|
@ -66,6 +67,7 @@ _DESCRIPTOR_KEYS = frozenset(
|
|||
"derived",
|
||||
"changesets",
|
||||
"render",
|
||||
"graph_render",
|
||||
"graph",
|
||||
"limits",
|
||||
"profiles",
|
||||
|
|
@ -548,7 +550,6 @@ def _load_descriptor(root: Path) -> ProjectDescriptor:
|
|||
for field in defaults.__dataclass_fields__
|
||||
}
|
||||
)
|
||||
|
||||
render = load_render_config(
|
||||
root,
|
||||
document.get("render"),
|
||||
|
|
@ -560,6 +561,18 @@ def _load_descriptor(root: Path) -> ProjectDescriptor:
|
|||
changeset_root=changeset_root,
|
||||
limits=limits,
|
||||
)
|
||||
graph_render = load_graph_render_config(
|
||||
root,
|
||||
document.get("graph_render"),
|
||||
descriptor_path=descriptor_path,
|
||||
content_roots=content_roots,
|
||||
authority_files=authority_files,
|
||||
cache_root=cache_root,
|
||||
index_path=index_path,
|
||||
changeset_root=changeset_root,
|
||||
manual_render=render,
|
||||
limits=limits,
|
||||
)
|
||||
|
||||
profile_documents = document.get("profiles", [])
|
||||
if not isinstance(profile_documents, list):
|
||||
|
|
@ -623,6 +636,7 @@ def _load_descriptor(root: Path) -> ProjectDescriptor:
|
|||
changeset_root=changeset_root,
|
||||
proposal_writers=tuple(sorted(proposal_writers, key=lambda writer: writer.writer_id)),
|
||||
render=render,
|
||||
graph_render=graph_render,
|
||||
allowed_relations=allowed_relations,
|
||||
profiles=tuple(profiles),
|
||||
limits=limits,
|
||||
|
|
|
|||
|
|
@ -19,6 +19,13 @@ MAX_PLAN_BYTES = 16_000_000
|
|||
MAX_PACKAGE_BYTES = 24_000_000
|
||||
MAX_RECEIPT_BYTES = 128_000
|
||||
MAX_PROJECTION_ARTIFACTS = 32
|
||||
MAX_GRAPH_VIEW_DEPTH = 32
|
||||
MAX_GRAPH_VIEW_NODES = 1_000
|
||||
MAX_GRAPH_VIEW_EDGES = 4_000
|
||||
MAX_GRAPH_VIEW_WORK = 1_000_000
|
||||
MAX_GRAPH_VIEW_FILTERS = 64
|
||||
MAX_GRAPH_VIEW_STRING_CHARS = 1_024
|
||||
MAX_GRAPH_VIEW_QUERY_CHARS = 10_000
|
||||
|
||||
ProjectionKind = Literal["manual", "graph"]
|
||||
|
||||
|
|
@ -468,16 +475,39 @@ def validate_projection_receipt(document: dict[str, object]) -> dict[str, object
|
|||
if not isinstance(artifacts_value, list):
|
||||
raise DocForgeError("invalid_projection", "Projection receipt structure is invalid")
|
||||
artifacts = cast(list[object], artifacts_value)
|
||||
renderer = document.get("renderer")
|
||||
diagnostics = document.get("diagnostics")
|
||||
timing = document.get("timing")
|
||||
if (
|
||||
not isinstance(renderer, dict)
|
||||
or not isinstance(diagnostics, dict)
|
||||
or not isinstance(timing, dict)
|
||||
):
|
||||
raise DocForgeError("invalid_projection", "Projection receipt structure is invalid")
|
||||
renderer_document = cast(dict[str, object], renderer)
|
||||
diagnostics_document = cast(dict[str, object], diagnostics)
|
||||
timing_document = cast(dict[str, object], timing)
|
||||
warnings = diagnostics_document.get("warnings")
|
||||
if (
|
||||
len(artifacts) > MAX_PROJECTION_ARTIFACTS
|
||||
or not isinstance(document.get("renderer"), dict)
|
||||
or not isinstance(document.get("diagnostics"), dict)
|
||||
or not isinstance(document.get("timing"), dict)
|
||||
or set(renderer_document) != {"renderer_id", "renderer_version"}
|
||||
or not all(
|
||||
isinstance(renderer_document.get(field), str) and renderer_document[field]
|
||||
for field in ("renderer_id", "renderer_version")
|
||||
)
|
||||
or set(diagnostics_document) != {"warnings"}
|
||||
or not isinstance(warnings, list)
|
||||
or len(cast(list[object], warnings)) > 10_000
|
||||
or not all(isinstance(item, str) for item in cast(list[object], warnings))
|
||||
or set(timing_document) != {"elapsed_ns"}
|
||||
or type(timing_document.get("elapsed_ns")) is not int
|
||||
or cast(int, timing_document["elapsed_ns"]) < 0
|
||||
):
|
||||
raise DocForgeError("invalid_projection", "Projection receipt structure is invalid")
|
||||
peak = document.get("peak_memory_bytes")
|
||||
if peak is not None and (type(peak) is not int or peak < 0):
|
||||
raise DocForgeError("invalid_projection", "Projection receipt memory value is invalid")
|
||||
artifact_ids: set[str] = set()
|
||||
for artifact in artifacts:
|
||||
if not isinstance(artifact, dict):
|
||||
raise DocForgeError("invalid_projection", "Projection receipt artifact is invalid")
|
||||
|
|
@ -494,6 +524,10 @@ def validate_projection_receipt(document: dict[str, object]) -> dict[str, object
|
|||
or cast(int, item["bytes"]) < 0
|
||||
):
|
||||
raise DocForgeError("invalid_projection", "Projection receipt artifact is invalid")
|
||||
artifact_id = cast(str, item["artifact_id"])
|
||||
if artifact_id in artifact_ids:
|
||||
raise DocForgeError("invalid_projection", "Projection receipt artifacts are duplicated")
|
||||
artifact_ids.add(artifact_id)
|
||||
return _validated_identity(
|
||||
document,
|
||||
identity_field="receipt_id",
|
||||
|
|
|
|||
489
src/docforge/projection_fragments.py
Normal file
489
src/docforge/projection_fragments.py
Normal file
|
|
@ -0,0 +1,489 @@
|
|||
"""Bounded, path-free, disposable projection fragment caching."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
from ._fs_safety import (
|
||||
atomic_replace_bytes_at,
|
||||
open_confined_directory,
|
||||
read_bounded_file_at,
|
||||
require_bound_directory,
|
||||
)
|
||||
from .errors import DocForgeError
|
||||
from .projection_contract import canonical_projection_bytes, projection_hash
|
||||
|
||||
FRAGMENT_SCHEMA_VERSION = 1
|
||||
FRAGMENT_KEY_CONTRACT = "docforge.projection-fragment-key"
|
||||
FRAGMENT_RECORD_CONTRACT = "docforge.projection-fragment-record"
|
||||
FRAGMENT_CACHE_DIRECTORY = "projection-fragments-v1"
|
||||
MAX_FRAGMENT_CONTENT_BYTES = 4_000_000
|
||||
MAX_FRAGMENT_ID_CHARS = 256
|
||||
MAX_FRAGMENT_CACHE_ENTRIES = 10_000
|
||||
MAX_FRAGMENT_CACHE_BYTES = 64_000_000
|
||||
_MAX_RECORD_OVERHEAD_BYTES = 8_192
|
||||
|
||||
ProjectionKind = Literal["manual", "graph"]
|
||||
|
||||
|
||||
def _is_hash(value: object) -> bool:
|
||||
return (
|
||||
isinstance(value, str)
|
||||
and len(value) == 64
|
||||
and all(character in "0123456789abcdef" for character in value)
|
||||
)
|
||||
|
||||
|
||||
def _version_string(value: object, *, field: str) -> str:
|
||||
if (
|
||||
not isinstance(value, str)
|
||||
or not value
|
||||
or value != value.strip()
|
||||
or len(value) > MAX_FRAGMENT_ID_CHARS
|
||||
or any(ord(character) < 32 for character in value)
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment identity is invalid",
|
||||
field=field,
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def fragment_semantic_hash(value: object) -> str:
|
||||
"""Hash one complete semantic input using the projection canonical JSON form."""
|
||||
|
||||
try:
|
||||
return hashlib.sha256(canonical_projection_bytes(value)).hexdigest()
|
||||
except (TypeError, ValueError) as error:
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment semantic input is not canonical JSON",
|
||||
) from error
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FragmentKey:
|
||||
"""Versioned identity for one renderer component's complete semantics."""
|
||||
|
||||
projection_kind: ProjectionKind
|
||||
renderer_id: str
|
||||
renderer_version: str
|
||||
component_version: str
|
||||
semantic_input_hash: str
|
||||
key_id: str
|
||||
|
||||
@classmethod
|
||||
def create(
|
||||
cls,
|
||||
*,
|
||||
projection_kind: ProjectionKind,
|
||||
renderer_id: str,
|
||||
renderer_version: str,
|
||||
component_version: str,
|
||||
semantic_input_hash: str,
|
||||
) -> FragmentKey:
|
||||
body = cls._body(
|
||||
projection_kind=projection_kind,
|
||||
renderer_id=renderer_id,
|
||||
renderer_version=renderer_version,
|
||||
component_version=component_version,
|
||||
semantic_input_hash=semantic_input_hash,
|
||||
)
|
||||
return cls._from_validated({**body, "key_id": projection_hash(body)})
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, value: object) -> FragmentKey:
|
||||
if not isinstance(value, dict):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment key is invalid",
|
||||
)
|
||||
return cls._from_validated(dict(cast(dict[str, object], value)))
|
||||
|
||||
@staticmethod
|
||||
def _body(
|
||||
*,
|
||||
projection_kind: object,
|
||||
renderer_id: object,
|
||||
renderer_version: object,
|
||||
component_version: object,
|
||||
semantic_input_hash: object,
|
||||
) -> dict[str, object]:
|
||||
if projection_kind not in {"manual", "graph"}:
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment kind is invalid",
|
||||
)
|
||||
if not _is_hash(semantic_input_hash):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment semantic input hash is invalid",
|
||||
)
|
||||
return {
|
||||
"schema_version": FRAGMENT_SCHEMA_VERSION,
|
||||
"contract": FRAGMENT_KEY_CONTRACT,
|
||||
"projection_kind": projection_kind,
|
||||
"renderer_id": _version_string(renderer_id, field="renderer_id"),
|
||||
"renderer_version": _version_string(
|
||||
renderer_version,
|
||||
field="renderer_version",
|
||||
),
|
||||
"component_version": _version_string(
|
||||
component_version,
|
||||
field="component_version",
|
||||
),
|
||||
"semantic_input_hash": semantic_input_hash,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def _from_validated(cls, value: dict[str, object]) -> FragmentKey:
|
||||
required = {
|
||||
"schema_version",
|
||||
"contract",
|
||||
"projection_kind",
|
||||
"renderer_id",
|
||||
"renderer_version",
|
||||
"component_version",
|
||||
"semantic_input_hash",
|
||||
"key_id",
|
||||
}
|
||||
if (
|
||||
set(value) != required
|
||||
or value.get("schema_version") != FRAGMENT_SCHEMA_VERSION
|
||||
or value.get("contract") != FRAGMENT_KEY_CONTRACT
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment key contract is incompatible",
|
||||
)
|
||||
body = cls._body(
|
||||
projection_kind=value.get("projection_kind"),
|
||||
renderer_id=value.get("renderer_id"),
|
||||
renderer_version=value.get("renderer_version"),
|
||||
component_version=value.get("component_version"),
|
||||
semantic_input_hash=value.get("semantic_input_hash"),
|
||||
)
|
||||
key_id = value.get("key_id")
|
||||
if not _is_hash(key_id) or key_id != projection_hash(body):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment key does not match its semantics",
|
||||
)
|
||||
return cls(
|
||||
projection_kind=cast(ProjectionKind, body["projection_kind"]),
|
||||
renderer_id=cast(str, body["renderer_id"]),
|
||||
renderer_version=cast(str, body["renderer_version"]),
|
||||
component_version=cast(str, body["component_version"]),
|
||||
semantic_input_hash=cast(str, body["semantic_input_hash"]),
|
||||
key_id=cast(str, key_id),
|
||||
)
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"schema_version": FRAGMENT_SCHEMA_VERSION,
|
||||
"contract": FRAGMENT_KEY_CONTRACT,
|
||||
"projection_kind": self.projection_kind,
|
||||
"renderer_id": self.renderer_id,
|
||||
"renderer_version": self.renderer_version,
|
||||
"component_version": self.component_version,
|
||||
"semantic_input_hash": self.semantic_input_hash,
|
||||
"key_id": self.key_id,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FragmentRecord:
|
||||
"""One path-free fragment payload with complete byte evidence."""
|
||||
|
||||
key: FragmentKey
|
||||
content: bytes
|
||||
byte_count: int
|
||||
content_sha256: str
|
||||
record_id: str
|
||||
|
||||
@classmethod
|
||||
def create(cls, key: FragmentKey, content: bytes) -> FragmentRecord:
|
||||
if len(content) > MAX_FRAGMENT_CONTENT_BYTES:
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment content is invalid or oversized",
|
||||
maximum_bytes=MAX_FRAGMENT_CONTENT_BYTES,
|
||||
)
|
||||
body = cls._body(key, content)
|
||||
return cls(
|
||||
key=key,
|
||||
content=content,
|
||||
byte_count=len(content),
|
||||
content_sha256=hashlib.sha256(content).hexdigest(),
|
||||
record_id=projection_hash(body),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, value: object) -> FragmentRecord:
|
||||
if not isinstance(value, dict):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment record is invalid",
|
||||
)
|
||||
document = dict(cast(dict[str, object], value))
|
||||
required = {
|
||||
"schema_version",
|
||||
"contract",
|
||||
"record_id",
|
||||
"key",
|
||||
"content_encoding",
|
||||
"content",
|
||||
"byte_count",
|
||||
"content_sha256",
|
||||
}
|
||||
if (
|
||||
set(document) != required
|
||||
or document.get("schema_version") != FRAGMENT_SCHEMA_VERSION
|
||||
or document.get("contract") != FRAGMENT_RECORD_CONTRACT
|
||||
or document.get("content_encoding") != "base64"
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment record contract is incompatible",
|
||||
)
|
||||
encoded = document.get("content")
|
||||
if not isinstance(encoded, str):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment content encoding is invalid",
|
||||
)
|
||||
try:
|
||||
content = base64.b64decode(encoded.encode("ascii"), validate=True)
|
||||
except (UnicodeEncodeError, ValueError) as error:
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment content encoding is invalid",
|
||||
) from error
|
||||
if len(content) > MAX_FRAGMENT_CONTENT_BYTES:
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment content is oversized",
|
||||
maximum_bytes=MAX_FRAGMENT_CONTENT_BYTES,
|
||||
)
|
||||
key = FragmentKey.from_dict(document.get("key"))
|
||||
body = cls._body(key, content)
|
||||
record_id = document.get("record_id")
|
||||
if (
|
||||
type(document.get("byte_count")) is not int
|
||||
or document.get("byte_count") != len(content)
|
||||
or document.get("content_sha256") != hashlib.sha256(content).hexdigest()
|
||||
or not _is_hash(record_id)
|
||||
or record_id != projection_hash(body)
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment byte evidence is invalid",
|
||||
)
|
||||
return cls(
|
||||
key=key,
|
||||
content=content,
|
||||
byte_count=len(content),
|
||||
content_sha256=hashlib.sha256(content).hexdigest(),
|
||||
record_id=cast(str, record_id),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_bytes(cls, raw: bytes) -> FragmentRecord:
|
||||
try:
|
||||
value: object = json.loads(raw)
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as error:
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment record is not valid JSON",
|
||||
) from error
|
||||
record = cls.from_dict(value)
|
||||
if record.to_bytes() != raw:
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment record is not canonically serialized",
|
||||
)
|
||||
return record
|
||||
|
||||
@staticmethod
|
||||
def _body(key: FragmentKey, content: bytes) -> dict[str, object]:
|
||||
return {
|
||||
"schema_version": FRAGMENT_SCHEMA_VERSION,
|
||||
"contract": FRAGMENT_RECORD_CONTRACT,
|
||||
"key": key.as_dict(),
|
||||
"content_encoding": "base64",
|
||||
"content": base64.b64encode(content).decode("ascii"),
|
||||
"byte_count": len(content),
|
||||
"content_sha256": hashlib.sha256(content).hexdigest(),
|
||||
}
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
**self._body(self.key, self.content),
|
||||
"record_id": self.record_id,
|
||||
}
|
||||
|
||||
def to_bytes(self) -> bytes:
|
||||
return canonical_projection_bytes(self.as_dict())
|
||||
|
||||
|
||||
class ProjectionFragmentCache:
|
||||
"""Confined best-effort storage for immutable projection fragments."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
project_root: Path,
|
||||
cache_root: Path,
|
||||
*,
|
||||
maximum_content_bytes: int = MAX_FRAGMENT_CONTENT_BYTES,
|
||||
) -> None:
|
||||
if (
|
||||
type(maximum_content_bytes) is not int
|
||||
or not 1 <= maximum_content_bytes <= MAX_FRAGMENT_CONTENT_BYTES
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_projection_fragment",
|
||||
"Projection fragment cache byte limit is invalid",
|
||||
maximum_bytes=MAX_FRAGMENT_CONTENT_BYTES,
|
||||
)
|
||||
self.project_root = project_root
|
||||
self.cache_root = cache_root
|
||||
self.fragment_root = cache_root / FRAGMENT_CACHE_DIRECTORY
|
||||
self.maximum_content_bytes = maximum_content_bytes
|
||||
|
||||
@property
|
||||
def maximum_record_bytes(self) -> int:
|
||||
encoded = ((self.maximum_content_bytes + 2) // 3) * 4
|
||||
return encoded + _MAX_RECORD_OVERHEAD_BYTES
|
||||
|
||||
def get(self, key: FragmentKey) -> FragmentRecord | None:
|
||||
"""Return one exact compatible fragment, treating every cache defect as a miss."""
|
||||
|
||||
descriptor: int | None = None
|
||||
try:
|
||||
descriptor = self._open(create=False)
|
||||
raw = read_bounded_file_at(
|
||||
descriptor,
|
||||
f"{key.key_id}.json",
|
||||
self.maximum_record_bytes,
|
||||
)
|
||||
if raw is None:
|
||||
return None
|
||||
record = FragmentRecord.from_bytes(raw)
|
||||
if record.key != key or record.byte_count > self.maximum_content_bytes:
|
||||
return None
|
||||
require_bound_directory(self.fragment_root, descriptor)
|
||||
return record
|
||||
except (DocForgeError, OSError):
|
||||
return None
|
||||
finally:
|
||||
if descriptor is not None:
|
||||
os.close(descriptor)
|
||||
|
||||
def put(self, key: FragmentKey, content: bytes) -> FragmentRecord | None:
|
||||
"""Durably publish one fragment, returning ``None`` on disposable cache failure."""
|
||||
|
||||
if len(content) > self.maximum_content_bytes:
|
||||
return None
|
||||
try:
|
||||
record = FragmentRecord.create(key, content)
|
||||
except DocForgeError:
|
||||
return None
|
||||
raw = record.to_bytes()
|
||||
if len(raw) > self.maximum_record_bytes:
|
||||
return None
|
||||
descriptor: int | None = None
|
||||
try:
|
||||
descriptor = self._open(create=True)
|
||||
name = f"{key.key_id}.json"
|
||||
try:
|
||||
existing = read_bounded_file_at(
|
||||
descriptor,
|
||||
name,
|
||||
self.maximum_record_bytes,
|
||||
)
|
||||
except DocForgeError as error:
|
||||
if error.code == "path_escape":
|
||||
return None
|
||||
existing = None
|
||||
if existing == raw:
|
||||
require_bound_directory(self.fragment_root, descriptor)
|
||||
return record
|
||||
atomic_replace_bytes_at(
|
||||
self.fragment_root,
|
||||
descriptor,
|
||||
name,
|
||||
raw,
|
||||
verify=lambda: require_bound_directory(self.fragment_root, descriptor),
|
||||
)
|
||||
return record
|
||||
except (DocForgeError, OSError):
|
||||
return None
|
||||
finally:
|
||||
if descriptor is not None:
|
||||
os.close(descriptor)
|
||||
|
||||
def prune(self, keep: tuple[FragmentKey, ...]) -> bool:
|
||||
"""Remove every stale entry and prove the retained inventory is bounded."""
|
||||
|
||||
keep_ids = {key.key_id for key in keep}
|
||||
if len(keep_ids) > MAX_FRAGMENT_CACHE_ENTRIES:
|
||||
return False
|
||||
descriptor: int | None = None
|
||||
try:
|
||||
descriptor = self._open(create=False)
|
||||
retained_entries = 0
|
||||
retained_bytes = 0
|
||||
with os.scandir(descriptor) as entries:
|
||||
for entry in entries:
|
||||
name = entry.name
|
||||
retained = (
|
||||
len(name) == 69
|
||||
and name.endswith(".json")
|
||||
and name[:-5] in keep_ids
|
||||
and all(character in "0123456789abcdef" for character in name[:-5])
|
||||
)
|
||||
if retained:
|
||||
identity = entry.stat(follow_symlinks=False)
|
||||
if not entry.is_file(follow_symlinks=False):
|
||||
return False
|
||||
retained_entries += 1
|
||||
retained_bytes += identity.st_size
|
||||
continue
|
||||
try:
|
||||
os.unlink(name, dir_fd=descriptor)
|
||||
except OSError:
|
||||
return False
|
||||
if (
|
||||
retained_entries > MAX_FRAGMENT_CACHE_ENTRIES
|
||||
or retained_bytes > MAX_FRAGMENT_CACHE_BYTES
|
||||
):
|
||||
return False
|
||||
require_bound_directory(self.fragment_root, descriptor)
|
||||
os.fsync(descriptor)
|
||||
return True
|
||||
except (DocForgeError, OSError):
|
||||
return False
|
||||
finally:
|
||||
if descriptor is not None:
|
||||
os.close(descriptor)
|
||||
|
||||
def _open(self, *, create: bool) -> int:
|
||||
if self.cache_root == self.project_root or not self.cache_root.is_relative_to(
|
||||
self.project_root
|
||||
):
|
||||
raise DocForgeError(
|
||||
"path_escape",
|
||||
"Projection fragment cache is not confined to a derived project root",
|
||||
)
|
||||
return open_confined_directory(
|
||||
self.project_root,
|
||||
self.fragment_root,
|
||||
create=create,
|
||||
)
|
||||
234
src/docforge/projection_policy.py
Normal file
234
src/docforge/projection_policy.py
Normal file
|
|
@ -0,0 +1,234 @@
|
|||
"""Independent version-2 policy for manual, portable graph, and live projections."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal, cast
|
||||
|
||||
from .errors import DocForgeError
|
||||
|
||||
ManualProjectionMode = Literal["auto", "explicit", "disabled"]
|
||||
PortableGraphProjectionMode = Literal["explicit", "disabled"]
|
||||
LiveViewerProjectionMode = Literal["on-demand", "disabled"]
|
||||
|
||||
MANUAL_PROJECTION_MODES: tuple[ManualProjectionMode, ...] = (
|
||||
"auto",
|
||||
"explicit",
|
||||
"disabled",
|
||||
)
|
||||
PORTABLE_GRAPH_PROJECTION_MODES: tuple[PortableGraphProjectionMode, ...] = (
|
||||
"explicit",
|
||||
"disabled",
|
||||
)
|
||||
LIVE_VIEWER_PROJECTION_MODES: tuple[LiveViewerProjectionMode, ...] = (
|
||||
"on-demand",
|
||||
"disabled",
|
||||
)
|
||||
|
||||
|
||||
def _invalid_mode(field: str, value: object, allowed: tuple[str, ...]) -> DocForgeError:
|
||||
return DocForgeError(
|
||||
"invalid_projection_policy",
|
||||
"Projection policy mode is unsupported",
|
||||
projection=field,
|
||||
mode=value,
|
||||
allowed=list(allowed),
|
||||
)
|
||||
|
||||
|
||||
def _select_mode(
|
||||
value: object | None,
|
||||
*,
|
||||
field: str,
|
||||
default: str,
|
||||
allowed: tuple[str, ...],
|
||||
) -> str:
|
||||
selected: object = default if value is None else value
|
||||
if not isinstance(selected, str) or selected not in allowed:
|
||||
raise _invalid_mode(field, selected, allowed)
|
||||
return selected
|
||||
|
||||
|
||||
def _require_boolean(field: str, value: object) -> bool:
|
||||
if type(value) is not bool:
|
||||
raise DocForgeError(
|
||||
"invalid_projection_policy",
|
||||
"Projection policy availability must be Boolean",
|
||||
field=field,
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def _unavailable(field: str, mode: str, required: str) -> DocForgeError:
|
||||
return DocForgeError(
|
||||
"projection_policy_unavailable",
|
||||
"Projection policy mode is unavailable",
|
||||
projection=field,
|
||||
mode=mode,
|
||||
required=required,
|
||||
)
|
||||
|
||||
|
||||
def validate_manual_projection_mode(value: object) -> ManualProjectionMode:
|
||||
"""Validate one direct manual-service policy selection."""
|
||||
|
||||
return cast(
|
||||
ManualProjectionMode,
|
||||
_select_mode(
|
||||
value,
|
||||
field="manual",
|
||||
default="explicit",
|
||||
allowed=MANUAL_PROJECTION_MODES,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def validate_portable_graph_projection_mode(
|
||||
value: object,
|
||||
) -> PortableGraphProjectionMode:
|
||||
"""Validate one direct portable-graph service policy selection."""
|
||||
|
||||
return cast(
|
||||
PortableGraphProjectionMode,
|
||||
_select_mode(
|
||||
value,
|
||||
field="portable_graph",
|
||||
default="explicit",
|
||||
allowed=PORTABLE_GRAPH_PROJECTION_MODES,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def validate_live_viewer_projection_mode(value: object) -> LiveViewerProjectionMode:
|
||||
"""Validate one direct live-viewer service policy selection."""
|
||||
|
||||
return cast(
|
||||
LiveViewerProjectionMode,
|
||||
_select_mode(
|
||||
value,
|
||||
field="live_viewer",
|
||||
default="on-demand",
|
||||
allowed=LIVE_VIEWER_PROJECTION_MODES,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ProjectionPolicyV2:
|
||||
"""One immutable policy for three independent projection consumers."""
|
||||
|
||||
manual: ManualProjectionMode
|
||||
portable_graph: PortableGraphProjectionMode
|
||||
live_viewer: LiveViewerProjectionMode
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.manual not in MANUAL_PROJECTION_MODES:
|
||||
raise _invalid_mode("manual", self.manual, MANUAL_PROJECTION_MODES)
|
||||
if self.portable_graph not in PORTABLE_GRAPH_PROJECTION_MODES:
|
||||
raise _invalid_mode(
|
||||
"portable_graph",
|
||||
self.portable_graph,
|
||||
PORTABLE_GRAPH_PROJECTION_MODES,
|
||||
)
|
||||
if self.live_viewer not in LIVE_VIEWER_PROJECTION_MODES:
|
||||
raise _invalid_mode(
|
||||
"live_viewer",
|
||||
self.live_viewer,
|
||||
LIVE_VIEWER_PROJECTION_MODES,
|
||||
)
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"schema_version": 2,
|
||||
"manual": self.manual,
|
||||
"portable_graph": self.portable_graph,
|
||||
"live_viewer": self.live_viewer,
|
||||
}
|
||||
|
||||
@property
|
||||
def policy_hash(self) -> str:
|
||||
raw = json.dumps(
|
||||
self.as_dict(),
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
).encode("utf-8")
|
||||
return hashlib.sha256(raw).hexdigest()
|
||||
|
||||
|
||||
def compose_projection_policy(
|
||||
*,
|
||||
manual: str | None = None,
|
||||
portable_graph: str | None = None,
|
||||
live_viewer: str | None = None,
|
||||
manual_configured: bool,
|
||||
portable_graph_configured: bool,
|
||||
application_enabled: bool,
|
||||
live_viewer_available: bool = True,
|
||||
) -> ProjectionPolicyV2:
|
||||
"""Compose compatible defaults with explicit availability-checked selections."""
|
||||
|
||||
manual_available = _require_boolean("manual_configured", manual_configured)
|
||||
graph_available = _require_boolean(
|
||||
"portable_graph_configured",
|
||||
portable_graph_configured,
|
||||
)
|
||||
application_available = _require_boolean("application_enabled", application_enabled)
|
||||
viewer_available = _require_boolean("live_viewer_available", live_viewer_available)
|
||||
|
||||
default_manual = (
|
||||
"auto"
|
||||
if manual_available and application_available
|
||||
else ("explicit" if manual_available else "disabled")
|
||||
)
|
||||
default_graph = "explicit" if graph_available else "disabled"
|
||||
default_viewer = "on-demand" if viewer_available else "disabled"
|
||||
|
||||
manual_mode = cast(
|
||||
ManualProjectionMode,
|
||||
_select_mode(
|
||||
manual,
|
||||
field="manual",
|
||||
default=default_manual,
|
||||
allowed=MANUAL_PROJECTION_MODES,
|
||||
),
|
||||
)
|
||||
graph_mode = cast(
|
||||
PortableGraphProjectionMode,
|
||||
_select_mode(
|
||||
portable_graph,
|
||||
field="portable_graph",
|
||||
default=default_graph,
|
||||
allowed=PORTABLE_GRAPH_PROJECTION_MODES,
|
||||
),
|
||||
)
|
||||
viewer_mode = cast(
|
||||
LiveViewerProjectionMode,
|
||||
_select_mode(
|
||||
live_viewer,
|
||||
field="live_viewer",
|
||||
default=default_viewer,
|
||||
allowed=LIVE_VIEWER_PROJECTION_MODES,
|
||||
),
|
||||
)
|
||||
|
||||
if manual_mode != "disabled" and not manual_available:
|
||||
raise _unavailable("manual", manual_mode, "manual_render_config")
|
||||
if manual_mode == "auto" and not application_available:
|
||||
raise _unavailable("manual", manual_mode, "canonical_application")
|
||||
if graph_mode == "explicit" and not graph_available:
|
||||
raise _unavailable(
|
||||
"portable_graph",
|
||||
graph_mode,
|
||||
"portable_graph_render_config",
|
||||
)
|
||||
if viewer_mode == "on-demand" and not viewer_available:
|
||||
raise _unavailable("live_viewer", viewer_mode, "live_viewer_runtime")
|
||||
|
||||
return ProjectionPolicyV2(
|
||||
manual=manual_mode,
|
||||
portable_graph=graph_mode,
|
||||
live_viewer=viewer_mode,
|
||||
)
|
||||
436
src/docforge/projection_worker.py
Normal file
436
src/docforge/projection_worker.py
Normal file
|
|
@ -0,0 +1,436 @@
|
|||
"""One-shot detached execution for the fixed built-in projection renderers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import json
|
||||
import os
|
||||
import resource
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from importlib.metadata import version
|
||||
from typing import cast
|
||||
|
||||
from .errors import DocForgeError
|
||||
from .projection_contract import (
|
||||
MAX_PACKAGE_BYTES,
|
||||
MAX_PROJECTION_ARTIFACTS,
|
||||
MAX_RECEIPT_BYTES,
|
||||
ProjectionArtifact,
|
||||
ProjectionPackageV1,
|
||||
ProjectionReceiptV1,
|
||||
ProjectionRenderResult,
|
||||
canonical_projection_bytes,
|
||||
)
|
||||
|
||||
WORKER_PROTOCOL_VERSION = 1
|
||||
MAX_WORKER_ARTIFACT_BYTES = 20_000_000
|
||||
MAX_WORKER_REQUEST_BYTES = MAX_PACKAGE_BYTES + 1
|
||||
MAX_WORKER_RESPONSE_BYTES = 4 * ((MAX_WORKER_ARTIFACT_BYTES + 2) // 3) + MAX_RECEIPT_BYTES + 256_000
|
||||
WORKER_TIMEOUT_SECONDS = 30
|
||||
|
||||
_GENERIC_HTML_RENDERER_ID = "generic_html"
|
||||
_PORTABLE_GRAPH_RENDERER_ID = "portable_graph_html"
|
||||
_PORTABLE_GRAPH_RENDERER_VERSION = "1"
|
||||
|
||||
|
||||
def _generic_html_renderer_version() -> str:
|
||||
return f"1+markdown-it-py-{version('markdown-it-py')}"
|
||||
|
||||
|
||||
def _worker_failure(message: str, **details: object) -> DocForgeError:
|
||||
return DocForgeError("projection_worker_failure", message, **details)
|
||||
|
||||
|
||||
def _renderer_identity(package: ProjectionPackageV1) -> dict[str, object]:
|
||||
renderer_value = package.document.get("renderer")
|
||||
if not isinstance(renderer_value, dict):
|
||||
raise DocForgeError("unsupported_renderer", "Projection renderer identity is invalid")
|
||||
renderer = cast(dict[str, object], renderer_value)
|
||||
if set(renderer) != {"renderer_id", "renderer_version"}:
|
||||
raise DocForgeError("unsupported_renderer", "Projection renderer identity is invalid")
|
||||
renderer_id = renderer.get("renderer_id")
|
||||
renderer_version = renderer.get("renderer_version")
|
||||
if not isinstance(renderer_id, str) or not isinstance(renderer_version, str):
|
||||
raise DocForgeError("unsupported_renderer", "Projection renderer identity is invalid")
|
||||
supported = (
|
||||
package.kind == "manual"
|
||||
and renderer_id == _GENERIC_HTML_RENDERER_ID
|
||||
and renderer_version == _generic_html_renderer_version()
|
||||
) or (
|
||||
package.kind == "graph"
|
||||
and renderer_id == _PORTABLE_GRAPH_RENDERER_ID
|
||||
and renderer_version == _PORTABLE_GRAPH_RENDERER_VERSION
|
||||
)
|
||||
if not supported:
|
||||
raise DocForgeError(
|
||||
"unsupported_renderer",
|
||||
"Projection worker supports only the fixed built-in renderer versions",
|
||||
)
|
||||
return dict(renderer)
|
||||
|
||||
|
||||
def _output_policy(package: ProjectionPackageV1) -> tuple[tuple[str, ...], int]:
|
||||
policy_value = package.document.get("output_policy")
|
||||
if not isinstance(policy_value, dict):
|
||||
raise DocForgeError("invalid_projection", "Projection output policy is invalid")
|
||||
policy = cast(dict[str, object], policy_value)
|
||||
if set(policy) != {"artifact_ids", "max_total_bytes"}:
|
||||
raise DocForgeError("invalid_projection", "Projection output policy is invalid")
|
||||
artifact_ids_value = policy.get("artifact_ids")
|
||||
maximum = policy.get("max_total_bytes")
|
||||
if not isinstance(artifact_ids_value, list):
|
||||
raise DocForgeError("invalid_projection", "Projection artifact inventory is invalid")
|
||||
artifact_ids_objects = cast(list[object], artifact_ids_value)
|
||||
if (
|
||||
not artifact_ids_objects
|
||||
or len(artifact_ids_objects) > MAX_PROJECTION_ARTIFACTS
|
||||
or not all(
|
||||
isinstance(artifact_id, str)
|
||||
and bool(artifact_id)
|
||||
and "/" not in artifact_id
|
||||
and artifact_id not in {".", ".."}
|
||||
for artifact_id in artifact_ids_objects
|
||||
)
|
||||
):
|
||||
raise DocForgeError("invalid_projection", "Projection artifact inventory is invalid")
|
||||
artifact_ids = tuple(cast(list[str], artifact_ids_objects))
|
||||
if len(set(artifact_ids)) != len(artifact_ids):
|
||||
raise DocForgeError("invalid_projection", "Projection artifact inventory is duplicated")
|
||||
if type(maximum) is not int or maximum < 1:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Projection output byte allowance is invalid",
|
||||
)
|
||||
return artifact_ids, maximum
|
||||
|
||||
|
||||
def _validated_package(package: object) -> ProjectionPackageV1:
|
||||
if not isinstance(package, ProjectionPackageV1):
|
||||
raise TypeError("Projection worker requires ProjectionPackageV1")
|
||||
validated = ProjectionPackageV1.from_dict(package.as_dict())
|
||||
_renderer_identity(validated)
|
||||
_output_policy(validated)
|
||||
return validated
|
||||
|
||||
|
||||
def _validate_result(
|
||||
package: ProjectionPackageV1,
|
||||
result: ProjectionRenderResult,
|
||||
*,
|
||||
require_peak_memory: bool,
|
||||
) -> ProjectionRenderResult:
|
||||
artifact_ids, maximum = _output_policy(package)
|
||||
if (
|
||||
len(result.artifacts) != len(artifact_ids)
|
||||
or tuple(artifact.artifact_id for artifact in result.artifacts) != artifact_ids
|
||||
):
|
||||
raise _worker_failure("Projection worker returned an invalid artifact inventory")
|
||||
total_bytes = 0
|
||||
evidence: list[dict[str, object]] = []
|
||||
for artifact in result.artifacts:
|
||||
if not artifact.media_type or type(artifact.content) is not bytes:
|
||||
raise _worker_failure("Projection worker returned an invalid artifact")
|
||||
total_bytes += len(artifact.content)
|
||||
if total_bytes > maximum or total_bytes > MAX_WORKER_ARTIFACT_BYTES:
|
||||
raise _worker_failure("Projection worker artifact transfer exceeded its fixed boundary")
|
||||
evidence.append(artifact.evidence())
|
||||
|
||||
try:
|
||||
receipt = ProjectionReceiptV1.from_dict(result.receipt.as_dict())
|
||||
except DocForgeError as error:
|
||||
raise _worker_failure("Projection worker receipt is invalid") from error
|
||||
receipt_document = receipt.document
|
||||
renderer = _renderer_identity(package)
|
||||
timing_value = receipt_document.get("timing")
|
||||
if not isinstance(timing_value, dict):
|
||||
raise _worker_failure("Projection worker receipt timing is invalid")
|
||||
timing = cast(dict[str, object], timing_value)
|
||||
elapsed = timing.get("elapsed_ns")
|
||||
peak_memory = receipt_document.get("peak_memory_bytes")
|
||||
if (
|
||||
receipt_document.get("kind") != package.kind
|
||||
or receipt_document.get("package_id") != package.package_id
|
||||
or receipt_document.get("plan_id") != package.document.get("plan_id")
|
||||
or receipt_document.get("renderer") != renderer
|
||||
or receipt_document.get("artifacts") != evidence
|
||||
or type(elapsed) is not int
|
||||
or elapsed < 0
|
||||
or (require_peak_memory and (type(peak_memory) is not int or peak_memory <= 0))
|
||||
):
|
||||
raise _worker_failure("Projection worker receipt does not attest the requested package")
|
||||
return ProjectionRenderResult(tuple(result.artifacts), receipt)
|
||||
|
||||
|
||||
def _decode_canonical_line(raw: bytes, *, maximum: int, label: str) -> dict[str, object]:
|
||||
if type(raw) is not bytes or len(raw) > maximum:
|
||||
raise _worker_failure(f"{label} exceeded its fixed boundary", maximum_bytes=maximum)
|
||||
if not raw or not raw.endswith(b"\n") or raw.count(b"\n") != 1:
|
||||
raise _worker_failure(f"{label} framing is invalid")
|
||||
payload = raw[:-1]
|
||||
try:
|
||||
value: object = json.loads(payload)
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as error:
|
||||
raise _worker_failure(f"{label} is not valid JSON") from error
|
||||
if not isinstance(value, dict):
|
||||
raise _worker_failure(f"{label} must be one JSON object")
|
||||
document = cast(dict[str, object], value)
|
||||
if canonical_projection_bytes(document) != payload:
|
||||
raise _worker_failure(f"{label} is not canonical JSON")
|
||||
return document
|
||||
|
||||
|
||||
def _encode_request(package: ProjectionPackageV1) -> bytes:
|
||||
encoded = canonical_projection_bytes(package.as_dict()) + b"\n"
|
||||
if len(encoded) > MAX_WORKER_REQUEST_BYTES:
|
||||
raise DocForgeError(
|
||||
"projection_too_large",
|
||||
"Projection worker request exceeds its fixed boundary",
|
||||
maximum_bytes=MAX_WORKER_REQUEST_BYTES,
|
||||
)
|
||||
return encoded
|
||||
|
||||
|
||||
def _invoke_worker(request: bytes) -> subprocess.CompletedProcess[bytes]:
|
||||
environment = {key: os.environ[key] for key in ("SYSTEMROOT", "WINDIR") if key in os.environ}
|
||||
environment.update(
|
||||
{
|
||||
"PYTHONIOENCODING": "utf-8",
|
||||
"PYTHONUTF8": "1",
|
||||
}
|
||||
)
|
||||
command = [sys.executable, "-I", "-m", "docforge._projection_worker_main"]
|
||||
with tempfile.TemporaryFile() as output:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
input=request,
|
||||
stdout=output,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
timeout=WORKER_TIMEOUT_SECONDS,
|
||||
shell=False,
|
||||
cwd=sys.prefix,
|
||||
env=environment,
|
||||
)
|
||||
output.seek(0)
|
||||
stdout = output.read(MAX_WORKER_RESPONSE_BYTES + 1)
|
||||
return subprocess.CompletedProcess(
|
||||
command,
|
||||
completed.returncode,
|
||||
stdout=stdout,
|
||||
)
|
||||
|
||||
|
||||
def _decode_response(package: ProjectionPackageV1, raw: bytes) -> ProjectionRenderResult:
|
||||
document = _decode_canonical_line(
|
||||
raw,
|
||||
maximum=MAX_WORKER_RESPONSE_BYTES,
|
||||
label="Projection worker response",
|
||||
)
|
||||
if (
|
||||
set(document) != {"schema_version", "artifacts", "receipt"}
|
||||
or document.get("schema_version") != WORKER_PROTOCOL_VERSION
|
||||
):
|
||||
raise _worker_failure("Projection worker response contract is invalid")
|
||||
artifact_values = document.get("artifacts")
|
||||
receipt_value = document.get("receipt")
|
||||
if not isinstance(artifact_values, list) or not isinstance(receipt_value, dict):
|
||||
raise _worker_failure("Projection worker response structure is invalid")
|
||||
artifacts: list[ProjectionArtifact] = []
|
||||
total_bytes = 0
|
||||
for value in cast(list[object], artifact_values):
|
||||
if not isinstance(value, dict):
|
||||
raise _worker_failure("Projection worker artifact envelope is invalid")
|
||||
artifact = cast(dict[str, object], value)
|
||||
if set(artifact) != {"artifact_id", "media_type", "content_base64"}:
|
||||
raise _worker_failure("Projection worker artifact envelope is invalid")
|
||||
artifact_id = artifact.get("artifact_id")
|
||||
media_type = artifact.get("media_type")
|
||||
encoded = artifact.get("content_base64")
|
||||
if (
|
||||
not isinstance(artifact_id, str)
|
||||
or not isinstance(media_type, str)
|
||||
or not isinstance(encoded, str)
|
||||
):
|
||||
raise _worker_failure("Projection worker artifact envelope is invalid")
|
||||
try:
|
||||
content = base64.b64decode(encoded.encode("ascii"), validate=True)
|
||||
except (UnicodeEncodeError, binascii.Error, ValueError) as error:
|
||||
raise _worker_failure("Projection worker artifact encoding is invalid") from error
|
||||
total_bytes += len(content)
|
||||
if total_bytes > MAX_WORKER_ARTIFACT_BYTES:
|
||||
raise _worker_failure("Projection worker artifact transfer exceeded its fixed boundary")
|
||||
artifacts.append(ProjectionArtifact(artifact_id, media_type, content))
|
||||
try:
|
||||
receipt = ProjectionReceiptV1.from_dict(cast(dict[str, object], receipt_value))
|
||||
except DocForgeError as error:
|
||||
raise _worker_failure("Projection worker receipt is invalid") from error
|
||||
return _validate_result(
|
||||
package,
|
||||
ProjectionRenderResult(tuple(artifacts), receipt),
|
||||
require_peak_memory=True,
|
||||
)
|
||||
|
||||
|
||||
def render_projection_in_worker(package: ProjectionPackageV1) -> ProjectionRenderResult:
|
||||
"""Render one validated path-free package in a fixed one-shot child process."""
|
||||
|
||||
validated = _validated_package(package)
|
||||
request = _encode_request(validated)
|
||||
try:
|
||||
completed = _invoke_worker(request)
|
||||
except subprocess.TimeoutExpired as error:
|
||||
raise DocForgeError(
|
||||
"projection_worker_timeout",
|
||||
"Detached projection worker exceeded its fixed timeout",
|
||||
timeout_seconds=WORKER_TIMEOUT_SECONDS,
|
||||
) from error
|
||||
except OSError as error:
|
||||
raise _worker_failure("Detached projection worker could not be launched") from error
|
||||
if completed.returncode != 0:
|
||||
if completed.returncode == 3 and completed.stdout:
|
||||
try:
|
||||
failure = _decode_canonical_line(
|
||||
completed.stdout,
|
||||
maximum=MAX_RECEIPT_BYTES,
|
||||
label="Projection worker error response",
|
||||
)
|
||||
error = failure.get("error")
|
||||
error_document = cast(dict[str, object], error) if isinstance(error, dict) else None
|
||||
if (
|
||||
set(failure) == {"schema_version", "error"}
|
||||
and failure.get("schema_version") == WORKER_PROTOCOL_VERSION
|
||||
and error_document is not None
|
||||
and set(error_document) == {"code", "message", "details"}
|
||||
and isinstance(error_document.get("code"), str)
|
||||
and bool(error_document["code"])
|
||||
and isinstance(error_document.get("message"), str)
|
||||
and bool(error_document["message"])
|
||||
and isinstance(error_document.get("details"), dict)
|
||||
):
|
||||
raise DocForgeError(
|
||||
cast(str, error_document["code"]),
|
||||
cast(str, error_document["message"]),
|
||||
**cast(dict[str, object], error_document["details"]),
|
||||
)
|
||||
except DocForgeError as error:
|
||||
if error.code != "projection_worker_failure":
|
||||
raise
|
||||
if completed.returncode < 0:
|
||||
raise _worker_failure(
|
||||
"Detached projection worker terminated by signal",
|
||||
signal=-completed.returncode,
|
||||
)
|
||||
raise _worker_failure(
|
||||
"Detached projection worker exited unsuccessfully",
|
||||
exit_code=completed.returncode,
|
||||
)
|
||||
if type(completed.stdout) is not bytes:
|
||||
raise _worker_failure("Detached projection worker returned invalid output")
|
||||
return _decode_response(validated, completed.stdout)
|
||||
|
||||
|
||||
def _render_package(package: ProjectionPackageV1) -> ProjectionRenderResult:
|
||||
renderer = _renderer_identity(package)
|
||||
if renderer["renderer_id"] == _GENERIC_HTML_RENDERER_ID:
|
||||
from docforge_renderers.manual import ManualHtmlRenderer
|
||||
|
||||
result = ManualHtmlRenderer(cast(str, renderer["renderer_version"])).render(package)
|
||||
else:
|
||||
from docforge_renderers.graph import PortableGraphHtmlRenderer
|
||||
|
||||
result = PortableGraphHtmlRenderer().render(package)
|
||||
return _validate_result(package, result, require_peak_memory=False)
|
||||
|
||||
|
||||
def _peak_memory_bytes() -> int:
|
||||
peak = int(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
|
||||
return max(1, peak if sys.platform == "darwin" else peak * 1024)
|
||||
|
||||
|
||||
def _child_response(package: ProjectionPackageV1) -> bytes:
|
||||
result = _render_package(package)
|
||||
original = result.receipt.document
|
||||
receipt = ProjectionReceiptV1.create(
|
||||
kind=package.kind,
|
||||
package_id=package.package_id,
|
||||
plan_id=cast(str, package.document["plan_id"]),
|
||||
renderer=cast(dict[str, object], original["renderer"]),
|
||||
artifacts=[artifact.evidence() for artifact in result.artifacts],
|
||||
diagnostics=cast(dict[str, object], original["diagnostics"]),
|
||||
timing=cast(dict[str, object], original["timing"]),
|
||||
peak_memory_bytes=_peak_memory_bytes(),
|
||||
)
|
||||
validated = _validate_result(
|
||||
package,
|
||||
ProjectionRenderResult(result.artifacts, receipt),
|
||||
require_peak_memory=True,
|
||||
)
|
||||
document: dict[str, object] = {
|
||||
"schema_version": WORKER_PROTOCOL_VERSION,
|
||||
"artifacts": [
|
||||
{
|
||||
"artifact_id": artifact.artifact_id,
|
||||
"media_type": artifact.media_type,
|
||||
"content_base64": base64.b64encode(artifact.content).decode("ascii"),
|
||||
}
|
||||
for artifact in validated.artifacts
|
||||
],
|
||||
"receipt": validated.receipt.as_dict(),
|
||||
}
|
||||
encoded = canonical_projection_bytes(document) + b"\n"
|
||||
if len(encoded) > MAX_WORKER_RESPONSE_BYTES:
|
||||
raise _worker_failure(
|
||||
"Projection worker response exceeded its fixed boundary",
|
||||
maximum_bytes=MAX_WORKER_RESPONSE_BYTES,
|
||||
)
|
||||
return encoded
|
||||
|
||||
|
||||
def _read_child_request() -> ProjectionPackageV1:
|
||||
raw = sys.stdin.buffer.read(MAX_WORKER_REQUEST_BYTES + 1)
|
||||
document = _decode_canonical_line(
|
||||
raw,
|
||||
maximum=MAX_WORKER_REQUEST_BYTES,
|
||||
label="Projection worker request",
|
||||
)
|
||||
return _validated_package(ProjectionPackageV1.from_dict(document))
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
"""Run the closed one-request child protocol."""
|
||||
|
||||
arguments = sys.argv[1:] if argv is None else argv
|
||||
if arguments:
|
||||
return 2
|
||||
try:
|
||||
package = _read_child_request()
|
||||
except Exception:
|
||||
return 2
|
||||
try:
|
||||
response = _child_response(package)
|
||||
sys.stdout.buffer.write(response)
|
||||
sys.stdout.buffer.flush()
|
||||
except DocForgeError as error:
|
||||
response = (
|
||||
canonical_projection_bytes(
|
||||
{
|
||||
"schema_version": WORKER_PROTOCOL_VERSION,
|
||||
"error": error.as_dict(),
|
||||
}
|
||||
)
|
||||
+ b"\n"
|
||||
)
|
||||
if len(response) <= MAX_RECEIPT_BYTES:
|
||||
sys.stdout.buffer.write(response)
|
||||
sys.stdout.buffer.flush()
|
||||
return 3
|
||||
except Exception:
|
||||
return 2
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -3,15 +3,28 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import html
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from importlib.metadata import version
|
||||
from pathlib import Path
|
||||
from typing import Protocol
|
||||
from typing import Protocol, cast
|
||||
|
||||
from .errors import DocForgeError
|
||||
from .manual_projection import build_manual_projection_package, build_manual_render_plan
|
||||
from .models import ProjectSnapshot, RenderView
|
||||
from .projection_contract import (
|
||||
ManualRenderPlanV1,
|
||||
ProjectionPackageV1,
|
||||
ProjectionRenderResult,
|
||||
)
|
||||
from .projection_fragments import (
|
||||
FragmentKey,
|
||||
FragmentRecord,
|
||||
ProjectionFragmentCache,
|
||||
fragment_semantic_hash,
|
||||
)
|
||||
from .projection_worker import render_projection_in_worker
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
|
@ -22,6 +35,7 @@ class PreparedRender:
|
|||
renderer: str
|
||||
renderer_version: str
|
||||
template_hash: str
|
||||
projection_receipt: dict[str, object] | None = None
|
||||
|
||||
|
||||
class Renderer(Protocol):
|
||||
|
|
@ -46,10 +60,13 @@ class GenericHtmlRenderer:
|
|||
renderer_id = "generic_html"
|
||||
contract_version = "1"
|
||||
|
||||
def __init__(self) -> None:
|
||||
page_component_version = "manual.page@1"
|
||||
|
||||
def __init__(self, *, incremental: bool = True) -> None:
|
||||
self.renderer_version = (
|
||||
f"{self.contract_version}+markdown-it-py-{version('markdown-it-py')}"
|
||||
)
|
||||
self.incremental = incremental
|
||||
|
||||
def prepare(
|
||||
self,
|
||||
|
|
@ -98,19 +115,24 @@ class GenericHtmlRenderer:
|
|||
json.dumps(identity_payload, sort_keys=True, separators=(",", ":")).encode("utf-8")
|
||||
).hexdigest()
|
||||
plan = build_manual_render_plan(snapshot, view, changeset_hash=changeset_hash)
|
||||
package = build_manual_projection_package(
|
||||
full_package = build_manual_projection_package(
|
||||
plan,
|
||||
template_bytes,
|
||||
renderer_id=self.renderer_id,
|
||||
renderer_version=self.renderer_version,
|
||||
max_output_bytes=snapshot.descriptor.limits.max_render_bytes,
|
||||
)
|
||||
from docforge_renderers.manual import ManualHtmlRenderer
|
||||
|
||||
result = ManualHtmlRenderer(self.renderer_version).render(
|
||||
package,
|
||||
render_identity=render_identity,
|
||||
)
|
||||
if self.incremental:
|
||||
result = self._incremental_result(
|
||||
snapshot,
|
||||
plan,
|
||||
template_bytes,
|
||||
full_package=full_package,
|
||||
render_identity=render_identity,
|
||||
)
|
||||
else:
|
||||
result = render_projection_in_worker(full_package)
|
||||
if len(result.artifacts) != 1:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
|
|
@ -124,21 +146,128 @@ class GenericHtmlRenderer:
|
|||
renderer=self.renderer_id,
|
||||
renderer_version=self.renderer_version,
|
||||
template_hash=template_hash,
|
||||
projection_receipt=result.receipt.as_dict(),
|
||||
)
|
||||
|
||||
def _incremental_result(
|
||||
self,
|
||||
snapshot: ProjectSnapshot,
|
||||
plan: ManualRenderPlanV1,
|
||||
template_bytes: bytes,
|
||||
*,
|
||||
full_package: ProjectionPackageV1,
|
||||
render_identity: str,
|
||||
) -> ProjectionRenderResult:
|
||||
cache = ProjectionFragmentCache(
|
||||
snapshot.descriptor.root,
|
||||
snapshot.descriptor.cache_root,
|
||||
)
|
||||
pages = cast(list[object], plan.document["pages"])
|
||||
records: list[FragmentRecord | None] = []
|
||||
missing: list[tuple[int, FragmentKey]] = []
|
||||
for value in pages:
|
||||
page = cast(dict[str, object], value)
|
||||
key = FragmentKey.create(
|
||||
projection_kind="manual",
|
||||
renderer_id=self.renderer_id,
|
||||
renderer_version=self.renderer_version,
|
||||
component_version=self.page_component_version,
|
||||
semantic_input_hash=fragment_semantic_hash(page),
|
||||
)
|
||||
record = cache.get(key)
|
||||
if record is None:
|
||||
missing.append((len(records), key))
|
||||
records.append(record)
|
||||
if not records:
|
||||
return render_projection_in_worker(full_package)
|
||||
full_result: ProjectionRenderResult | None = None
|
||||
new_records: list[FragmentRecord] = []
|
||||
if missing:
|
||||
full_result = render_projection_in_worker(full_package)
|
||||
fragments = self._extract_page_fragments(
|
||||
pages,
|
||||
full_result.artifacts[0].content,
|
||||
)
|
||||
try:
|
||||
for position, key in missing:
|
||||
record = FragmentRecord.create(key, fragments[position])
|
||||
records[position] = record
|
||||
new_records.append(record)
|
||||
except DocForgeError:
|
||||
return full_result
|
||||
complete_records = [record for record in records if record is not None]
|
||||
if len(complete_records) != len(records):
|
||||
return full_result or render_projection_in_worker(full_package)
|
||||
try:
|
||||
incremental_package = build_manual_projection_package(
|
||||
plan,
|
||||
template_bytes,
|
||||
fragment_records=[record.as_dict() for record in complete_records],
|
||||
renderer_id=self.renderer_id,
|
||||
renderer_version=self.renderer_version,
|
||||
max_output_bytes=snapshot.descriptor.limits.max_render_bytes,
|
||||
render_identity=render_identity,
|
||||
)
|
||||
incremental_result = render_projection_in_worker(incremental_package)
|
||||
except DocForgeError:
|
||||
cache.prune(())
|
||||
return full_result or render_projection_in_worker(full_package)
|
||||
if full_result is None:
|
||||
cache.prune(tuple(record.key for record in complete_records))
|
||||
return incremental_result
|
||||
if tuple(
|
||||
(artifact.artifact_id, artifact.media_type, artifact.content)
|
||||
for artifact in incremental_result.artifacts
|
||||
) != tuple(
|
||||
(artifact.artifact_id, artifact.media_type, artifact.content)
|
||||
for artifact in full_result.artifacts
|
||||
):
|
||||
cache.prune(())
|
||||
return full_result
|
||||
for record in new_records:
|
||||
cache.put(record.key, record.content)
|
||||
cache.prune(tuple(record.key for record in complete_records))
|
||||
return incremental_result
|
||||
|
||||
@staticmethod
|
||||
def _extract_page_fragments(
|
||||
pages: list[object],
|
||||
output: bytes,
|
||||
) -> list[bytes]:
|
||||
"""Extract exact deterministic page sections from one trusted full artifact."""
|
||||
|
||||
fragments: list[bytes] = []
|
||||
cursor = 0
|
||||
closing = b"</section>"
|
||||
for value in pages:
|
||||
page = cast(dict[str, object], value)
|
||||
node_id = cast(str, page["node_id"])
|
||||
marker = f'<section id="node-{html.escape(node_id, quote=True)}">'.encode()
|
||||
start = output.find(marker, cursor)
|
||||
end = output.find(closing, start + len(marker)) if start >= 0 else -1
|
||||
if start < 0 or end < 0:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Full manual artifact does not contain its planned page fragments",
|
||||
)
|
||||
end += len(closing)
|
||||
fragments.append(output[start:end])
|
||||
cursor = end
|
||||
return fragments
|
||||
|
||||
|
||||
_RENDERERS: dict[str, type[GenericHtmlRenderer]] = {
|
||||
GenericHtmlRenderer.renderer_id: GenericHtmlRenderer
|
||||
}
|
||||
|
||||
|
||||
def renderer_for(view: RenderView) -> Renderer:
|
||||
def renderer_for(view: RenderView, *, incremental: bool = True) -> Renderer:
|
||||
factory = _RENDERERS.get(view.renderer)
|
||||
if factory is None:
|
||||
raise DocForgeError(
|
||||
"unsupported_renderer", "View does not name a supported built-in renderer"
|
||||
)
|
||||
return factory()
|
||||
return factory(incremental=incremental)
|
||||
|
||||
|
||||
def relative_output(snapshot: ProjectSnapshot, path: Path) -> str:
|
||||
|
|
|
|||
|
|
@ -26,6 +26,8 @@ from .models import (
|
|||
RenderView,
|
||||
)
|
||||
from .project import project_root_fingerprint
|
||||
from .projection_contract import ProjectionReceiptV1
|
||||
from .projection_policy import ManualProjectionMode, validate_manual_projection_mode
|
||||
from .render_contract import PreparedRender, relative_output, renderer_for
|
||||
from .telemetry import increment, stage
|
||||
|
||||
|
|
@ -36,9 +38,26 @@ MAX_RENDER_RECEIPT_BYTES = 64_000
|
|||
class RenderService:
|
||||
"""Render only declared views through fixed built-in renderer implementations."""
|
||||
|
||||
def __init__(self, project: ProjectService, changesets: ChangesetStore | None = None) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
project: ProjectService,
|
||||
changesets: ChangesetStore | None = None,
|
||||
*,
|
||||
manual_policy: ManualProjectionMode = "explicit",
|
||||
) -> None:
|
||||
self.project = project
|
||||
self.changesets = changesets or ChangesetStore(project)
|
||||
self.manual_policy = validate_manual_projection_mode(manual_policy)
|
||||
|
||||
def _require_rendering(self, operation: str) -> None:
|
||||
if self.manual_policy == "disabled":
|
||||
raise DocForgeError(
|
||||
"projection_policy_forbids_operation",
|
||||
"Manual projection policy disables rendering work",
|
||||
projection="manual",
|
||||
mode=self.manual_policy,
|
||||
operation=operation,
|
||||
)
|
||||
|
||||
def status(self, view_id: str | None = None) -> dict[str, object]:
|
||||
"""Report publication state from bounded receipts without rendering canonical content."""
|
||||
|
|
@ -86,6 +105,7 @@ class RenderService:
|
|||
def deep_status(self, view_id: str | None = None) -> dict[str, object]:
|
||||
"""Recompute render output as the explicit side-effect-free equivalence oracle."""
|
||||
|
||||
self._require_rendering("deep_status")
|
||||
snapshot = self.project.load()
|
||||
config = snapshot.descriptor.render
|
||||
if config is None:
|
||||
|
|
@ -107,7 +127,12 @@ class RenderService:
|
|||
snapshot.descriptor.root,
|
||||
view.output_path,
|
||||
)
|
||||
prepared, _ = self._prepare(snapshot, view, changeset_hash=None)
|
||||
prepared, _ = self._prepare(
|
||||
snapshot,
|
||||
view,
|
||||
changeset_hash=None,
|
||||
incremental=False,
|
||||
)
|
||||
state = "missing"
|
||||
actual_hash: str | None = None
|
||||
output = view.output_path
|
||||
|
|
@ -151,6 +176,7 @@ class RenderService:
|
|||
)
|
||||
|
||||
def render(self, view_id: str) -> dict[str, object]:
|
||||
self._require_rendering("render")
|
||||
with self._lock():
|
||||
snapshot = self.project.load()
|
||||
config = self._config(snapshot)
|
||||
|
|
@ -373,6 +399,8 @@ class RenderService:
|
|||
"template_file": final_template_file,
|
||||
"output_file": final_output_file,
|
||||
}
|
||||
if prepared.projection_receipt is not None:
|
||||
payload["projection_receipt"] = prepared.projection_receipt
|
||||
raw = json.dumps(payload, sort_keys=True, indent=2).encode("utf-8") + b"\n"
|
||||
if len(raw) > MAX_RENDER_RECEIPT_BYTES:
|
||||
raise DocForgeError(
|
||||
|
|
@ -572,8 +600,10 @@ class RenderService:
|
|||
renderer = renderer_for(view)
|
||||
template_file = receipt.get("template_file")
|
||||
output_file = receipt.get("output_file")
|
||||
fields = set(receipt)
|
||||
projection_receipt = receipt.get("projection_receipt")
|
||||
return (
|
||||
set(receipt) == required
|
||||
fields in (required, required | {"projection_receipt"})
|
||||
and receipt.get("schema_version") == RENDER_RECEIPT_SCHEMA_VERSION
|
||||
and receipt.get("project_id") == descriptor.project_id
|
||||
and receipt.get("project_root_fingerprint") == project_root_fingerprint(descriptor.root)
|
||||
|
|
@ -602,6 +632,50 @@ class RenderService:
|
|||
descriptor.limits.max_render_bytes,
|
||||
)
|
||||
and cast(dict[str, object], output_file)["size"] == receipt.get("output_bytes")
|
||||
and (
|
||||
projection_receipt is None
|
||||
or self._valid_projection_receipt(
|
||||
projection_receipt,
|
||||
renderer_id=renderer.renderer_id,
|
||||
renderer_version=renderer.renderer_version,
|
||||
output_hash=cast(str, receipt["output_hash"]),
|
||||
output_bytes=cast(int, receipt["output_bytes"]),
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _valid_projection_receipt(
|
||||
value: object,
|
||||
*,
|
||||
renderer_id: str,
|
||||
renderer_version: str,
|
||||
output_hash: str,
|
||||
output_bytes: int,
|
||||
) -> bool:
|
||||
if not isinstance(value, dict):
|
||||
return False
|
||||
try:
|
||||
receipt = ProjectionReceiptV1.from_dict(cast(dict[str, object], value))
|
||||
except DocForgeError:
|
||||
return False
|
||||
document = receipt.document
|
||||
return (
|
||||
document.get("kind") == "manual"
|
||||
and document.get("renderer")
|
||||
== {
|
||||
"renderer_id": renderer_id,
|
||||
"renderer_version": renderer_version,
|
||||
}
|
||||
and document.get("artifacts")
|
||||
== [
|
||||
{
|
||||
"artifact_id": "manual.html",
|
||||
"media_type": "text/html; charset=utf-8",
|
||||
"sha256": output_hash,
|
||||
"bytes": output_bytes,
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
|
|
@ -663,6 +737,7 @@ class RenderService:
|
|||
"reason": reason,
|
||||
"verification": "receipt",
|
||||
"receipt_schema_version": payload.get("schema_version"),
|
||||
"projection_receipt": payload.get("projection_receipt"),
|
||||
}
|
||||
|
||||
def _current_state(self) -> ProjectState | None:
|
||||
|
|
@ -687,6 +762,7 @@ class RenderService:
|
|||
}
|
||||
|
||||
def preview(self, changeset_id: str, view_id: str) -> dict[str, object]:
|
||||
self._require_rendering("preview")
|
||||
with self._lock():
|
||||
snapshot, changeset_hash = self.changesets.projected_snapshot(changeset_id)
|
||||
config = self._config(snapshot)
|
||||
|
|
@ -731,11 +807,12 @@ class RenderService:
|
|||
view: RenderView,
|
||||
*,
|
||||
changeset_hash: str | None,
|
||||
incremental: bool = True,
|
||||
) -> tuple[PreparedRender, bytes]:
|
||||
increment("render_prepare_calls")
|
||||
template = self._template_bytes(snapshot, view)
|
||||
with stage("render.prepare"):
|
||||
prepared = renderer_for(view).prepare(
|
||||
prepared = renderer_for(view, incremental=incremental).prepare(
|
||||
snapshot,
|
||||
view,
|
||||
template,
|
||||
|
|
@ -850,6 +927,7 @@ class RenderService:
|
|||
"expected_output_hash": prepared.output_hash,
|
||||
"actual_output_hash": actual_hash,
|
||||
"template_hash": prepared.template_hash,
|
||||
"projection_receipt": prepared.projection_receipt,
|
||||
"path": relative_output(snapshot, view.output_path),
|
||||
"state": state,
|
||||
}
|
||||
|
|
|
|||
|
|
@ -79,6 +79,7 @@ OPERATION_NAMES = frozenset(
|
|||
"test",
|
||||
"benchmark.m1",
|
||||
"benchmark.m2",
|
||||
"benchmark.m3",
|
||||
"mcp.invoke",
|
||||
"mcp.bootstrap",
|
||||
"mcp.sync",
|
||||
|
|
@ -96,6 +97,8 @@ OPERATION_NAMES = frozenset(
|
|||
"mcp.generation_diff",
|
||||
"mcp.validate_project",
|
||||
"mcp.render_status",
|
||||
"mcp.graph_plan",
|
||||
"mcp.graph_render_status",
|
||||
"mcp.visualize",
|
||||
"mcp.visualization_status",
|
||||
"mcp.stop_visualization",
|
||||
|
|
@ -117,6 +120,9 @@ OPERATION_NAMES = frozenset(
|
|||
"cli.impact",
|
||||
"cli.context",
|
||||
"cli.generation-diff",
|
||||
"cli.graph-plan",
|
||||
"cli.graph-render",
|
||||
"cli.graph-render-status",
|
||||
"cli.configure",
|
||||
"cli.doctor",
|
||||
"cli.render",
|
||||
|
|
|
|||
|
|
@ -28,6 +28,10 @@ from .errors import DocForgeError
|
|||
from .index import ProjectIndex
|
||||
from .models import IncrementalStateProject
|
||||
from .project import project_root_fingerprint
|
||||
from .projection_policy import (
|
||||
LiveViewerProjectionMode,
|
||||
validate_live_viewer_projection_mode,
|
||||
)
|
||||
from .telemetry import increment, stage
|
||||
from .visualization import VISUALIZATION_TEMPLATE, VisualizationIndexSnapshot
|
||||
|
||||
|
|
@ -623,9 +627,16 @@ class ViewerManager:
|
|||
class ViewerManagerClient:
|
||||
"""Project-bound MCP-side client for the separately supervised manager service."""
|
||||
|
||||
def __init__(self, index: ProjectIndex, *, state_path: Path | None = None) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
index: ProjectIndex,
|
||||
*,
|
||||
state_path: Path | None = None,
|
||||
live_viewer_policy: LiveViewerProjectionMode = "on-demand",
|
||||
) -> None:
|
||||
self.index = index
|
||||
self.state_path = state_path or default_state_path()
|
||||
self.live_viewer_policy = validate_live_viewer_projection_mode(live_viewer_policy)
|
||||
|
||||
def start(
|
||||
self,
|
||||
|
|
@ -634,6 +645,14 @@ class ViewerManagerClient:
|
|||
query: str | None = None,
|
||||
depth: int = 1,
|
||||
) -> dict[str, object]:
|
||||
if self.live_viewer_policy == "disabled":
|
||||
raise DocForgeError(
|
||||
"projection_policy_forbids_operation",
|
||||
"Live viewer projection policy disables viewer startup",
|
||||
projection="live_viewer",
|
||||
mode=self.live_viewer_policy,
|
||||
operation="start",
|
||||
)
|
||||
if node_id is not None and query is not None:
|
||||
raise DocForgeError(
|
||||
"invalid_visualization_target",
|
||||
|
|
|
|||
321
src/docforge_renderers/graph.py
Normal file
321
src/docforge_renderers/graph.py
Normal file
|
|
@ -0,0 +1,321 @@
|
|||
"""Deterministic self-contained renderer for one portable graph package."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import html
|
||||
import json
|
||||
from time import perf_counter_ns
|
||||
from typing import cast
|
||||
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.projection_contract import (
|
||||
ProjectionArtifact,
|
||||
ProjectionPackageV1,
|
||||
ProjectionReceiptV1,
|
||||
ProjectionRenderResult,
|
||||
)
|
||||
|
||||
PORTABLE_GRAPH_CSS = """
|
||||
:root { color-scheme: light dark; font-family: system-ui, sans-serif; }
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; background: Canvas; color: CanvasText; }
|
||||
.skip { position: absolute; left: -10000px; top: auto; }
|
||||
.skip:focus { left: 1rem; top: 1rem; z-index: 2; padding: .5rem; background: Canvas; }
|
||||
header, main { width: min(96%, 1100px); margin: 0 auto; }
|
||||
header { padding: 1rem 0; }
|
||||
.controls { display: flex; flex-wrap: wrap; gap: .75rem; align-items: end; }
|
||||
label { display: grid; gap: .25rem; font-weight: 600; }
|
||||
input, select, button { font: inherit; min-height: 2.75rem; padding: .45rem .65rem; }
|
||||
button { cursor: pointer; }
|
||||
button:focus-visible, input:focus-visible, select:focus-visible { outline: .2rem solid Highlight; }
|
||||
.summary { margin: 1rem 0; }
|
||||
.layout { display: grid; grid-template-columns: minmax(16rem, 1fr) minmax(20rem, 2fr); gap: 1rem; }
|
||||
.panel { border: 1px solid GrayText; border-radius: .5rem; padding: 1rem; overflow: auto; }
|
||||
html[data-enhanced="true"] main[data-mode="nodes"] .layout,
|
||||
html[data-enhanced="true"] main[data-mode="flow"] .layout { grid-template-columns: 1fr; }
|
||||
html[data-enhanced="true"] main[data-mode="nodes"] [data-panel="relationships"] { display: none; }
|
||||
html[data-enhanced="true"] main[data-mode="flow"] [data-panel="nodes"] { display: none; }
|
||||
.node-list { list-style: none; padding: 0; margin: 0; display: grid; gap: .5rem; }
|
||||
.node-list button {
|
||||
width: 100%; text-align: left; border: 1px solid GrayText; border-radius: .35rem;
|
||||
}
|
||||
.node-list button[aria-current="true"] { border-width: .2rem; }
|
||||
table { border-collapse: collapse; width: 100%; }
|
||||
th, td { text-align: left; border-bottom: 1px solid GrayText; padding: .5rem; vertical-align: top; }
|
||||
caption { text-align: left; font-weight: 700; margin-bottom: .5rem; }
|
||||
.muted { color: CanvasText; }
|
||||
dialog {
|
||||
max-width: min(42rem, calc(100% - 2rem));
|
||||
border: 1px solid GrayText; border-radius: .5rem;
|
||||
}
|
||||
dialog::backdrop { background: rgb(0 0 0 / 55%); }
|
||||
@media (max-width: 48rem) { .layout { grid-template-columns: 1fr; } }
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
*, *::before, *::after { scroll-behavior: auto !important; }
|
||||
}
|
||||
@media (forced-colors: active) {
|
||||
.panel, .node-list button, dialog { border: 2px solid CanvasText; }
|
||||
}
|
||||
""".strip()
|
||||
|
||||
PORTABLE_GRAPH_JAVASCRIPT = r"""
|
||||
(() => {
|
||||
"use strict";
|
||||
const plan = JSON.parse(document.getElementById("docforge-graph-plan").textContent);
|
||||
const nodes = plan.graph.nodes;
|
||||
const edges = plan.graph.edges;
|
||||
const list = document.getElementById("node-list");
|
||||
const rows = document.getElementById("edge-rows");
|
||||
const filter = document.getElementById("filter");
|
||||
const mode = document.getElementById("mode");
|
||||
const main = document.getElementById("main");
|
||||
const status = document.getElementById("status");
|
||||
const dialog = document.getElementById("node-dialog");
|
||||
const detail = document.getElementById("node-detail");
|
||||
const close = document.getElementById("close-dialog");
|
||||
let opener = null;
|
||||
document.documentElement.dataset.enhanced = "true";
|
||||
|
||||
const matches = (node) => {
|
||||
const query = filter.value.trim().toLocaleLowerCase();
|
||||
const fields = [
|
||||
node.node_id, node.title, node.summary, node.family, node.status, ...node.tags
|
||||
];
|
||||
return !query || fields
|
||||
.join(" ").toLocaleLowerCase().includes(query);
|
||||
};
|
||||
const selectedIds = () => new Set(nodes.filter(matches).map((node) => node.node_id));
|
||||
const render = () => {
|
||||
main.dataset.mode = mode.value;
|
||||
const visible = nodes.filter(matches);
|
||||
const ids = selectedIds();
|
||||
list.replaceChildren(...visible.map((node) => {
|
||||
const item = document.createElement("li");
|
||||
const button = document.createElement("button");
|
||||
button.type = "button";
|
||||
button.textContent = `${node.title} (${node.node_id})`;
|
||||
button.dataset.nodeId = node.node_id;
|
||||
button.addEventListener("click", () => inspect(node, button));
|
||||
item.append(button);
|
||||
return item;
|
||||
}));
|
||||
const visibleEdges = edges.filter((edge) => ids.has(edge.source_id) && ids.has(edge.target_id));
|
||||
rows.replaceChildren(...visibleEdges.map((edge) => {
|
||||
const row = document.createElement("tr");
|
||||
[edge.source_id, edge.relation, edge.target_id].forEach((value) => {
|
||||
const cell = document.createElement("td");
|
||||
cell.textContent = value;
|
||||
row.append(cell);
|
||||
});
|
||||
return row;
|
||||
}));
|
||||
status.textContent = `${visible.length} nodes and ${visibleEdges.length} relationships `
|
||||
+ `shown in ${mode.value} mode.`;
|
||||
};
|
||||
const inspect = (node, button) => {
|
||||
opener = button;
|
||||
detail.replaceChildren();
|
||||
const heading = document.createElement("h2");
|
||||
heading.id = "node-dialog-title";
|
||||
heading.textContent = node.title;
|
||||
const identity = document.createElement("p");
|
||||
identity.textContent = `${node.node_id} · ${node.family} · ${node.status}`;
|
||||
const summary = document.createElement("p");
|
||||
summary.textContent = node.summary;
|
||||
detail.append(heading, identity, summary);
|
||||
dialog.showModal();
|
||||
close.focus();
|
||||
};
|
||||
close.addEventListener("click", () => dialog.close());
|
||||
dialog.addEventListener("close", () => opener?.focus());
|
||||
filter.addEventListener("input", render);
|
||||
mode.addEventListener("change", render);
|
||||
render();
|
||||
})();
|
||||
""".strip()
|
||||
|
||||
|
||||
def _csp_hash(content: str) -> str:
|
||||
digest = hashlib.sha256(content.encode("utf-8")).digest()
|
||||
return base64.b64encode(digest).decode("ascii")
|
||||
|
||||
|
||||
def _embedded_json(value: object) -> str:
|
||||
return (
|
||||
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
|
||||
.replace("&", "\\u0026")
|
||||
.replace("<", "\\u003c")
|
||||
.replace(">", "\\u003e")
|
||||
)
|
||||
|
||||
|
||||
def _static_node_markup(plan: dict[str, object]) -> str:
|
||||
graph = cast(dict[str, object], plan["graph"])
|
||||
nodes = cast(list[dict[str, object]], graph["nodes"])
|
||||
parts: list[str] = []
|
||||
for node in nodes:
|
||||
node_id = html.escape(cast(str, node["node_id"]))
|
||||
attribute_node_id = html.escape(cast(str, node["node_id"]), quote=True)
|
||||
title = html.escape(cast(str, node["title"]))
|
||||
family = html.escape(cast(str, node["family"]))
|
||||
status = html.escape(cast(str, node["status"]))
|
||||
summary = html.escape(cast(str, node["summary"]))
|
||||
parts.append(
|
||||
"<li><article>"
|
||||
f'<button type="button" data-node-id="{attribute_node_id}">'
|
||||
f"{title} ({node_id})</button>"
|
||||
f'<p class="muted">{family} · {status}</p>'
|
||||
f"<p>{summary}</p>"
|
||||
"</article></li>"
|
||||
)
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def _static_edge_markup(plan: dict[str, object]) -> str:
|
||||
graph = cast(dict[str, object], plan["graph"])
|
||||
edges = cast(list[dict[str, object]], graph["edges"])
|
||||
return "".join(
|
||||
"<tr>"
|
||||
f"<td>{html.escape(cast(str, edge['source_id']))}</td>"
|
||||
f"<td>{html.escape(cast(str, edge['relation']))}</td>"
|
||||
f"<td>{html.escape(cast(str, edge['target_id']))}</td>"
|
||||
"</tr>"
|
||||
for edge in edges
|
||||
)
|
||||
|
||||
|
||||
class PortableGraphHtmlRenderer:
|
||||
"""Render a validated graph package without querying or publishing project state."""
|
||||
|
||||
renderer_id = "portable_graph_html"
|
||||
renderer_version = "1"
|
||||
|
||||
def render(self, package: ProjectionPackageV1) -> ProjectionRenderResult:
|
||||
started = perf_counter_ns()
|
||||
package = ProjectionPackageV1.from_dict(package.as_dict())
|
||||
document = package.document
|
||||
if package.kind != "graph":
|
||||
raise DocForgeError("invalid_projection", "Graph renderer requires a graph package")
|
||||
renderer = cast(dict[str, object], document["renderer"])
|
||||
if renderer != {
|
||||
"renderer_id": self.renderer_id,
|
||||
"renderer_version": self.renderer_version,
|
||||
}:
|
||||
raise DocForgeError("unsupported_renderer", "Graph renderer identity is incompatible")
|
||||
if document["components"] != [
|
||||
{"component_id": "graph.portable-document@1"},
|
||||
{"component_id": "graph.accessible-list@1"},
|
||||
{"component_id": "graph.relationship-table@1"},
|
||||
]:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Portable graph renderer component declarations are incompatible",
|
||||
)
|
||||
if document["assets"] != []:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Portable graph renderer does not accept project-provided assets",
|
||||
)
|
||||
plan = cast(dict[str, object], document["plan"])
|
||||
view = cast(dict[str, object], plan["view"])
|
||||
initial_mode = view.get("initial_mode")
|
||||
policy = cast(dict[str, object], plan["policy"])
|
||||
if (
|
||||
initial_mode not in {"nodes", "flow", "web"}
|
||||
or policy.get("logic_requested") is not False
|
||||
):
|
||||
raise DocForgeError(
|
||||
"unsupported_renderer",
|
||||
"Portable graph renderer version 1 does not render Logic projections",
|
||||
)
|
||||
title = html.escape(cast(str, view["title"]))
|
||||
project = cast(dict[str, object], plan["project"])
|
||||
embedded = _embedded_json(plan)
|
||||
static_nodes = _static_node_markup(plan)
|
||||
static_edges = _static_edge_markup(plan)
|
||||
csp = (
|
||||
"default-src 'none'; "
|
||||
f"style-src 'sha256-{_csp_hash(PORTABLE_GRAPH_CSS)}'; "
|
||||
f"script-src 'sha256-{_csp_hash(PORTABLE_GRAPH_JAVASCRIPT)}'; "
|
||||
"img-src 'none'; connect-src 'none'; object-src 'none'; base-uri 'none'; "
|
||||
"form-action 'none'; frame-ancestors 'none'"
|
||||
)
|
||||
output = (
|
||||
"<!DOCTYPE html>\n"
|
||||
'<html lang="en">\n'
|
||||
"<head>\n"
|
||||
'<meta charset="utf-8">\n'
|
||||
'<meta name="viewport" content="width=device-width, initial-scale=1">\n'
|
||||
'<meta http-equiv="Content-Security-Policy" '
|
||||
f'content="{html.escape(csp, quote=True)}">\n'
|
||||
f"<title>{title} · DocForge graph</title>\n"
|
||||
f"<style>{PORTABLE_GRAPH_CSS}</style>\n"
|
||||
"</head>\n"
|
||||
"<body>\n"
|
||||
'<a class="skip" href="#main">Skip to graph content</a>\n'
|
||||
"<header>\n"
|
||||
f"<h1>{title}</h1>\n"
|
||||
f'<p class="muted">Generation {html.escape(cast(str, project["source_hash"]))}</p>\n'
|
||||
'<div class="controls" role="group" aria-label="Graph controls">\n'
|
||||
'<label>Filter nodes<input id="filter" type="search" autocomplete="off"></label>\n'
|
||||
'<label>View mode<select id="mode">'
|
||||
f'<option value="nodes"{" selected" if initial_mode == "nodes" else ""}>Nodes</option>'
|
||||
f'<option value="flow"{" selected" if initial_mode == "flow" else ""}>Flow</option>'
|
||||
f'<option value="web"{" selected" if initial_mode == "web" else ""}>Web</option>'
|
||||
"</select></label>\n"
|
||||
"</div>\n"
|
||||
'<p id="status" class="summary" role="status" aria-live="polite"></p>\n'
|
||||
"</header>\n"
|
||||
f'<main id="main" tabindex="-1" data-mode="{initial_mode}">\n'
|
||||
'<div class="layout">\n'
|
||||
'<section class="panel" data-panel="nodes" aria-labelledby="nodes-title">'
|
||||
'<h2 id="nodes-title">Nodes</h2>'
|
||||
f'<ul id="node-list" class="node-list">{static_nodes}</ul></section>\n'
|
||||
'<section class="panel" data-panel="relationships" aria-labelledby="relations-title">'
|
||||
'<h2 id="relations-title">Relationships</h2>'
|
||||
"<table><caption>Selected graph facts</caption><thead><tr>"
|
||||
'<th scope="col">Source</th><th scope="col">Relation</th>'
|
||||
f'<th scope="col">Target</th></tr></thead><tbody id="edge-rows">{static_edges}</tbody>'
|
||||
"</table>"
|
||||
"</section>\n"
|
||||
"</div>\n"
|
||||
"</main>\n"
|
||||
'<dialog id="node-dialog" aria-labelledby="node-dialog-title">'
|
||||
'<div id="node-detail"><h2 id="node-dialog-title">Node details</h2></div>'
|
||||
'<button id="close-dialog" type="button">Close</button></dialog>\n'
|
||||
f'<script id="docforge-graph-plan" type="application/json">{embedded}</script>\n'
|
||||
f"<script>{PORTABLE_GRAPH_JAVASCRIPT}</script>\n"
|
||||
"</body>\n"
|
||||
"</html>\n"
|
||||
).encode()
|
||||
policy = cast(dict[str, object], document["output_policy"])
|
||||
maximum = policy.get("max_total_bytes")
|
||||
if set(policy) != {"artifact_ids", "max_total_bytes"} or policy.get("artifact_ids") != [
|
||||
"portable-graph.html"
|
||||
]:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Portable graph output policy is incompatible",
|
||||
)
|
||||
if type(maximum) is not int or maximum < 1 or len(output) > maximum:
|
||||
raise DocForgeError("render_too_large", "Rendered output exceeds the configured limit")
|
||||
artifact = ProjectionArtifact(
|
||||
artifact_id="portable-graph.html",
|
||||
media_type="text/html; charset=utf-8",
|
||||
content=output,
|
||||
)
|
||||
receipt = ProjectionReceiptV1.create(
|
||||
kind="graph",
|
||||
package_id=package.package_id,
|
||||
plan_id=cast(str, document["plan_id"]),
|
||||
renderer=dict(renderer),
|
||||
artifacts=[artifact.evidence()],
|
||||
diagnostics={
|
||||
"warnings": [],
|
||||
},
|
||||
timing={"elapsed_ns": perf_counter_ns() - started},
|
||||
peak_memory_bytes=None,
|
||||
)
|
||||
return ProjectionRenderResult((artifact,), receipt)
|
||||
|
|
@ -17,6 +17,10 @@ from docforge.projection_contract import (
|
|||
ProjectionReceiptV1,
|
||||
ProjectionRenderResult,
|
||||
)
|
||||
from docforge.projection_fragments import (
|
||||
FragmentRecord,
|
||||
fragment_semantic_hash,
|
||||
)
|
||||
|
||||
_TEMPLATE_TOKEN = re.compile(r"{{\s*([a-z_][a-z0-9_]*)\s*}}")
|
||||
_ALLOWED_TOKENS = frozenset(
|
||||
|
|
@ -35,6 +39,7 @@ _ACTIVE_TEMPLATE_CONTENT = re.compile(
|
|||
r"|<\s*meta\b[^>]*\bhttp-equiv\s*=\s*[\"']?\s*refresh\b",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_MANUAL_PAGE_COMPONENT = "manual.page@1"
|
||||
|
||||
|
||||
class ManualHtmlRenderer:
|
||||
|
|
@ -65,11 +70,15 @@ class ManualHtmlRenderer:
|
|||
raise DocForgeError("unsupported_renderer", "Manual renderer identity is incompatible")
|
||||
plan = cast(dict[str, object], document["plan"])
|
||||
assets = cast(list[object], document["assets"])
|
||||
if len(assets) != 1 or not isinstance(assets[0], dict):
|
||||
if len(assets) not in {1, 2} or not isinstance(assets[0], dict):
|
||||
raise DocForgeError("invalid_projection", "Manual template asset is invalid")
|
||||
asset = cast(dict[str, object], assets[0])
|
||||
if (
|
||||
set(asset) != {"asset_id", "media_type", "sha256", "text"}
|
||||
set(asset)
|
||||
not in (
|
||||
{"asset_id", "media_type", "sha256", "text"},
|
||||
{"asset_id", "media_type", "sha256", "text", "render_identity"},
|
||||
)
|
||||
or asset.get("asset_id") != "manual.template"
|
||||
or asset.get("media_type") != "text/html; charset=utf-8"
|
||||
or not isinstance(asset.get("text"), str)
|
||||
|
|
@ -98,11 +107,27 @@ class ManualHtmlRenderer:
|
|||
"invalid_template",
|
||||
"Render template must contain docforge_content exactly once",
|
||||
)
|
||||
identity = render_identity or cast(str, plan["plan_id"])
|
||||
packaged_identity = asset.get("render_identity")
|
||||
if packaged_identity is not None and (
|
||||
not isinstance(packaged_identity, str)
|
||||
or len(packaged_identity) != 64
|
||||
or any(character not in "0123456789abcdef" for character in packaged_identity)
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Manual render identity is invalid",
|
||||
)
|
||||
if render_identity is not None and packaged_identity not in {None, render_identity}:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Manual render identity is inconsistent",
|
||||
)
|
||||
identity = render_identity or packaged_identity or cast(str, plan["plan_id"])
|
||||
fragments = self._fragments(plan, assets[1:] if len(assets) == 2 else [])
|
||||
project = cast(dict[str, object], plan["project"])
|
||||
view = cast(dict[str, object], plan["view"])
|
||||
replacements = {
|
||||
"docforge_content": self._content(plan),
|
||||
"docforge_content": self._content(plan, fragments),
|
||||
"docforge_project_id": html.escape(cast(str, project["project_id"]), quote=True),
|
||||
"docforge_render_identity": identity,
|
||||
"docforge_title": html.escape(cast(str, view["title"]), quote=True),
|
||||
|
|
@ -131,7 +156,98 @@ class ManualHtmlRenderer:
|
|||
)
|
||||
return ProjectionRenderResult((artifact,), receipt)
|
||||
|
||||
def _content(self, plan: dict[str, object]) -> str:
|
||||
def _fragments(
|
||||
self,
|
||||
plan: dict[str, object],
|
||||
assets: list[object],
|
||||
) -> dict[str, str]:
|
||||
if not assets:
|
||||
return {}
|
||||
asset = assets[0]
|
||||
if not isinstance(asset, dict):
|
||||
raise DocForgeError("invalid_projection", "Manual fragment asset is invalid")
|
||||
document = cast(dict[str, object], asset)
|
||||
if (
|
||||
set(document) != {"asset_id", "media_type", "records"}
|
||||
or document.get("asset_id") != "manual.fragments"
|
||||
or document.get("media_type") != "application/vnd.docforge.projection-fragments.v1+json"
|
||||
or not isinstance(document.get("records"), list)
|
||||
):
|
||||
raise DocForgeError("invalid_projection", "Manual fragment asset is invalid")
|
||||
pages = cast(list[object], plan["pages"])
|
||||
records = cast(list[object], document["records"])
|
||||
if len(records) != len(pages):
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Manual fragment inventory is incomplete",
|
||||
)
|
||||
fragments: dict[str, str] = {}
|
||||
for page_value, record_value in zip(pages, records, strict=True):
|
||||
if not isinstance(page_value, dict):
|
||||
raise DocForgeError("invalid_projection", "Manual page is invalid")
|
||||
page = cast(dict[str, object], page_value)
|
||||
record = FragmentRecord.from_dict(record_value)
|
||||
if (
|
||||
record.key.projection_kind != "manual"
|
||||
or record.key.renderer_id != self.renderer_id
|
||||
or record.key.renderer_version != self.renderer_version
|
||||
or record.key.component_version != _MANUAL_PAGE_COMPONENT
|
||||
or record.key.semantic_input_hash != fragment_semantic_hash(page)
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Manual fragment identity does not match its page semantics",
|
||||
)
|
||||
try:
|
||||
fragment = record.content.decode("utf-8")
|
||||
except UnicodeDecodeError as error:
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Manual fragment content is not valid UTF-8",
|
||||
) from error
|
||||
node_id = cast(str, page["node_id"])
|
||||
if node_id in fragments or fragment != self.render_page_fragment(page):
|
||||
raise DocForgeError(
|
||||
"invalid_projection",
|
||||
"Manual fragment content does not match its page semantics",
|
||||
)
|
||||
fragments[node_id] = fragment
|
||||
return fragments
|
||||
|
||||
def render_page_fragment(self, page: dict[str, object]) -> str:
|
||||
"""Render one page from complete plan semantics without project authority."""
|
||||
|
||||
node_id = cast(str, page["node_id"])
|
||||
sections = [
|
||||
f'<section id="node-{html.escape(node_id, quote=True)}">',
|
||||
f"<h2>{html.escape(cast(str, page['title']))}</h2>",
|
||||
'<dl class="docforge-node-meta">',
|
||||
f"<dt>ID</dt><dd>{html.escape(node_id)}</dd>",
|
||||
f"<dt>Family</dt><dd>{html.escape(cast(str, page['family']))}</dd>",
|
||||
f"<dt>Status</dt><dd>{html.escape(cast(str, page['status']))}</dd>",
|
||||
f"<dt>Authority</dt><dd>{html.escape(cast(str, page['authority']))}</dd>",
|
||||
"</dl>",
|
||||
f'<p class="docforge-summary">{html.escape(cast(str, page["summary"]))}</p>',
|
||||
self.markdown.render(cast(str, page["content"])).rstrip(),
|
||||
]
|
||||
relationships = cast(list[object], page["cross_references"])
|
||||
if relationships:
|
||||
sections.append('<ul class="docforge-relationships">')
|
||||
for relationship_value in relationships:
|
||||
relationship = cast(dict[str, object], relationship_value)
|
||||
sections.append(
|
||||
f"<li>{html.escape(cast(str, relationship['relation']))}: "
|
||||
f"{html.escape(cast(str, relationship['target_id']))}</li>"
|
||||
)
|
||||
sections.append("</ul>")
|
||||
sections.append("</section>")
|
||||
return "\n".join(sections)
|
||||
|
||||
def _content(
|
||||
self,
|
||||
plan: dict[str, object],
|
||||
fragments: dict[str, str],
|
||||
) -> str:
|
||||
navigation = ['<nav aria-label="Documentation"><ul>']
|
||||
for value in cast(list[object], plan["navigation"]):
|
||||
item = cast(dict[str, object], value)
|
||||
|
|
@ -144,29 +260,5 @@ class ManualHtmlRenderer:
|
|||
for value in cast(list[object], plan["pages"]):
|
||||
page = cast(dict[str, object], value)
|
||||
node_id = cast(str, page["node_id"])
|
||||
sections.extend(
|
||||
[
|
||||
f'<section id="node-{html.escape(node_id, quote=True)}">',
|
||||
f"<h2>{html.escape(cast(str, page['title']))}</h2>",
|
||||
'<dl class="docforge-node-meta">',
|
||||
f"<dt>ID</dt><dd>{html.escape(node_id)}</dd>",
|
||||
f"<dt>Family</dt><dd>{html.escape(cast(str, page['family']))}</dd>",
|
||||
f"<dt>Status</dt><dd>{html.escape(cast(str, page['status']))}</dd>",
|
||||
f"<dt>Authority</dt><dd>{html.escape(cast(str, page['authority']))}</dd>",
|
||||
"</dl>",
|
||||
f'<p class="docforge-summary">{html.escape(cast(str, page["summary"]))}</p>',
|
||||
self.markdown.render(cast(str, page["content"])).rstrip(),
|
||||
]
|
||||
)
|
||||
relationships = cast(list[object], page["cross_references"])
|
||||
if relationships:
|
||||
sections.append('<ul class="docforge-relationships">')
|
||||
for relationship_value in relationships:
|
||||
relationship = cast(dict[str, object], relationship_value)
|
||||
sections.append(
|
||||
f"<li>{html.escape(cast(str, relationship['relation']))}: "
|
||||
f"{html.escape(cast(str, relationship['target_id']))}</li>"
|
||||
)
|
||||
sections.append("</ul>")
|
||||
sections.append("</section>")
|
||||
sections.append(fragments.get(node_id, self.render_page_fragment(page)))
|
||||
return "\n".join(sections)
|
||||
|
|
|
|||
187
tests/accessibility.spec.mjs
Normal file
187
tests/accessibility.spec.mjs
Normal file
|
|
@ -0,0 +1,187 @@
|
|||
import AxeBuilder from "@axe-core/playwright";
|
||||
import { expect, test } from "@playwright/test";
|
||||
import { spawn } from "node:child_process";
|
||||
import { createInterface } from "node:readline";
|
||||
|
||||
const AXE_TAGS = [
|
||||
"wcag2a",
|
||||
"wcag2aa",
|
||||
"wcag21a",
|
||||
"wcag21aa",
|
||||
"wcag22a",
|
||||
"wcag22aa",
|
||||
];
|
||||
const MANUAL_AXE_TAGS = AXE_TAGS.filter((tag) => !tag.startsWith("wcag22"));
|
||||
|
||||
let fixtureProcess;
|
||||
let surfaces;
|
||||
|
||||
function startFixture() {
|
||||
const python = process.env.DOCFORGE_PYTHON || ".venv/bin/python";
|
||||
const child = spawn(python, ["tools/accessibility_fixture.py"], {
|
||||
cwd: process.cwd(),
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
});
|
||||
let stderr = "";
|
||||
child.stderr.setEncoding("utf8");
|
||||
child.stderr.on("data", (chunk) => {
|
||||
stderr += chunk;
|
||||
});
|
||||
const lines = createInterface({ input: child.stdout });
|
||||
const ready = new Promise((resolve, reject) => {
|
||||
let settled = false;
|
||||
lines.once("line", (line) => {
|
||||
settled = true;
|
||||
try {
|
||||
const payload = JSON.parse(line);
|
||||
if (
|
||||
payload.schema_version !== 1
|
||||
|| typeof payload.manual_html !== "string"
|
||||
|| typeof payload.portable_html !== "string"
|
||||
|| typeof payload.live_url !== "string"
|
||||
) {
|
||||
throw new Error("Accessibility fixture returned an invalid payload");
|
||||
}
|
||||
resolve(payload);
|
||||
} catch (error) {
|
||||
reject(error);
|
||||
} finally {
|
||||
lines.close();
|
||||
}
|
||||
});
|
||||
child.once("exit", (code, signal) => {
|
||||
if (!settled) {
|
||||
reject(
|
||||
new Error(
|
||||
`Accessibility fixture exited before readiness `
|
||||
+ `(code=${code}, signal=${signal}):\n${stderr}`,
|
||||
),
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
return { child, ready };
|
||||
}
|
||||
|
||||
async function stopFixture(child) {
|
||||
if (child.exitCode !== null || child.signalCode !== null) {
|
||||
return;
|
||||
}
|
||||
const exited = new Promise((resolve) => child.once("exit", resolve));
|
||||
child.stdin.end();
|
||||
await Promise.race([
|
||||
exited,
|
||||
new Promise((_, reject) => {
|
||||
setTimeout(() => reject(new Error("Accessibility fixture did not stop")), 5_000);
|
||||
}),
|
||||
]);
|
||||
}
|
||||
|
||||
function violationReport(violations) {
|
||||
return violations.map((violation) => {
|
||||
const targets = violation.nodes
|
||||
.flatMap((node) => node.target)
|
||||
.join(", ");
|
||||
return `${violation.id} (${violation.impact}): ${violation.help}\n ${targets}`;
|
||||
}).join("\n");
|
||||
}
|
||||
|
||||
async function expectNoAxeViolations(page, tags = AXE_TAGS) {
|
||||
const results = await new AxeBuilder({ page }).withTags(tags).analyze();
|
||||
expect(results.violations, violationReport(results.violations)).toEqual([]);
|
||||
}
|
||||
|
||||
async function tabTo(page, selector, maximumTabs = 40) {
|
||||
for (let count = 0; count < maximumTabs; count += 1) {
|
||||
await page.keyboard.press("Tab");
|
||||
if (await page.evaluate((target) => document.activeElement?.matches(target), selector)) {
|
||||
return page.locator(selector).filter({ visible: true }).first();
|
||||
}
|
||||
}
|
||||
throw new Error(`Keyboard focus did not reach ${selector}`);
|
||||
}
|
||||
|
||||
test.beforeAll(async () => {
|
||||
const fixture = startFixture();
|
||||
fixtureProcess = fixture.child;
|
||||
surfaces = await fixture.ready;
|
||||
});
|
||||
|
||||
test.afterAll(async () => {
|
||||
await stopFixture(fixtureProcess);
|
||||
});
|
||||
|
||||
test("generated manual has no axe violations and its navigation works by keyboard", async ({
|
||||
page,
|
||||
}) => {
|
||||
await page.setContent(surfaces.manual_html, { waitUntil: "load" });
|
||||
await expect(page.locator("main section")).not.toHaveCount(0);
|
||||
// The generic renderer owns structure, while this frozen project template owns target sizing.
|
||||
await expectNoAxeViolations(page, MANUAL_AXE_TAGS);
|
||||
|
||||
await page.keyboard.press("Tab");
|
||||
const firstNavigationLink = page.locator("nav[aria-label='Documentation'] a").first();
|
||||
await expect(firstNavigationLink).toBeFocused();
|
||||
const target = await firstNavigationLink.getAttribute("href");
|
||||
expect(target).toMatch(/^#[A-Za-z0-9_.-]+$/);
|
||||
await page.keyboard.press("Enter");
|
||||
await expect.poll(() => page.evaluate(() => window.location.hash)).toBe(target);
|
||||
});
|
||||
|
||||
test("portable graph supports skip, filter, view, and dialog keyboard flows", async ({ page }) => {
|
||||
await page.goto("about:blank");
|
||||
await page.setContent(surfaces.portable_html, { waitUntil: "load" });
|
||||
await expect(page.locator("#status")).toContainText("nodes and");
|
||||
await expectNoAxeViolations(page);
|
||||
|
||||
await page.keyboard.press("Tab");
|
||||
await expect(page.locator("a.skip")).toBeFocused();
|
||||
await page.keyboard.press("Enter");
|
||||
await expect(page.locator("main#main")).toBeFocused();
|
||||
|
||||
await page.goto("about:blank");
|
||||
await page.setContent(surfaces.portable_html, { waitUntil: "load" });
|
||||
await page.keyboard.press("Tab");
|
||||
await page.keyboard.press("Tab");
|
||||
await expect(page.locator("#filter")).toBeFocused();
|
||||
await page.keyboard.type("guide.workflow");
|
||||
await expect(page.locator("#status")).toContainText("1 nodes and");
|
||||
|
||||
await page.keyboard.press("Tab");
|
||||
await expect(page.locator("#mode")).toBeFocused();
|
||||
await page.keyboard.press("ArrowDown");
|
||||
await expect(page.locator("main#main")).toHaveAttribute("data-mode", "flow");
|
||||
await page.keyboard.press("ArrowUp");
|
||||
await expect(page.locator("main#main")).toHaveAttribute("data-mode", "nodes");
|
||||
|
||||
await page.keyboard.press("Tab");
|
||||
const nodeButton = page.locator("#node-list button").first();
|
||||
await expect(nodeButton).toBeFocused();
|
||||
await page.keyboard.press("Enter");
|
||||
await expect(page.locator("#node-dialog")).toHaveAttribute("open", "");
|
||||
await expect(page.locator("#close-dialog")).toBeFocused();
|
||||
await expectNoAxeViolations(page);
|
||||
await page.keyboard.press("Escape");
|
||||
await expect(page.locator("#node-dialog")).not.toHaveAttribute("open", "");
|
||||
await expect(nodeButton).toBeFocused();
|
||||
});
|
||||
|
||||
test("live viewer passes axe and exposes keyboard graph and resize controls", async ({ page }) => {
|
||||
await page.goto(surfaces.live_url);
|
||||
await expect(page.locator("#status")).toContainText("nodes ·");
|
||||
await expect(page.locator("#graph g.node[role='button']").first()).toBeVisible();
|
||||
await expectNoAxeViolations(page);
|
||||
|
||||
const resizer = await tabTo(page, "#left-resizer");
|
||||
const originalWidth = Number(await resizer.getAttribute("aria-valuenow"));
|
||||
await page.keyboard.press("ArrowRight");
|
||||
await expect(resizer).toHaveAttribute("aria-valuenow", String(originalWidth + 16));
|
||||
|
||||
await tabTo(page, "#graph g.node[role='button']");
|
||||
await page.keyboard.press("Shift+Enter");
|
||||
await expect(page.locator("#node-dialog")).toHaveAttribute("open", "");
|
||||
await expect(page.locator("#close-node-dialog")).toBeFocused();
|
||||
await expectNoAxeViolations(page);
|
||||
await page.keyboard.press("Escape");
|
||||
await expect(page.locator("#node-dialog")).not.toHaveAttribute("open", "");
|
||||
});
|
||||
|
|
@ -17,6 +17,25 @@ from docforge.viewer_manager import ViewerManager
|
|||
ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURES = ROOT / "tests" / "fixtures"
|
||||
|
||||
GRAPH_CONFIG = """
|
||||
|
||||
[graph_render]
|
||||
output_root = ".docforge/portable-graph"
|
||||
|
||||
[[graph_render.views]]
|
||||
id = "architecture"
|
||||
renderer = "portable_graph_html"
|
||||
output = "architecture.html"
|
||||
title = "Alpha architecture"
|
||||
root = "guide.workflow"
|
||||
initial_mode = "nodes"
|
||||
depth = 2
|
||||
max_nodes = 20
|
||||
max_edges = 40
|
||||
max_work = 1000
|
||||
include_logic = false
|
||||
"""
|
||||
|
||||
|
||||
class DocForgeCliTests(unittest.TestCase):
|
||||
def copy_fixture(self, destination: Path) -> Path:
|
||||
|
|
@ -134,6 +153,30 @@ class DocForgeCliTests(unittest.TestCase):
|
|||
else:
|
||||
os.environ["DOCFORGE_VIEWER_MANAGER_STATE"] = previous
|
||||
|
||||
def test_portable_graph_plan_render_and_status_are_self_service(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture(Path(directory))
|
||||
descriptor = root / ".docforge/project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
|
||||
encoding="utf-8",
|
||||
)
|
||||
parser = _parser()
|
||||
planned = _run(
|
||||
parser.parse_args(["--project-root", str(root), "graph-plan", "architecture"])
|
||||
)
|
||||
self.assertEqual("architecture", planned["view_id"])
|
||||
rendered = _run(
|
||||
parser.parse_args(["--project-root", str(root), "graph-render", "architecture"])
|
||||
)
|
||||
self.assertEqual("current", rendered["state"])
|
||||
status = _run(
|
||||
parser.parse_args(
|
||||
["--project-root", str(root), "graph-render-status", "architecture"]
|
||||
)
|
||||
)
|
||||
self.assertEqual("current", status["state"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
|
|
@ -18,6 +19,7 @@ from jsonschema import Draft202012Validator
|
|||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
|
||||
from docforge.changeset_contract import document_hash
|
||||
from docforge.cli import _parser, _run, main
|
||||
from docforge.client_config import (
|
||||
_read_existing,
|
||||
|
|
@ -39,6 +41,19 @@ CONFIGURATION_SCHEMA = json.loads(
|
|||
DOCTOR_SCHEMA = json.loads((SCHEMAS / "doctor-result.schema.json").read_text(encoding="utf-8"))
|
||||
POLICY_SCHEMA = json.loads((SCHEMAS / "policy.schema.json").read_text(encoding="utf-8"))
|
||||
|
||||
GRAPH_RENDER_CONFIG = """
|
||||
|
||||
[graph_render]
|
||||
output_root = ".docforge/portable-graph"
|
||||
|
||||
[[graph_render.views]]
|
||||
id = "architecture"
|
||||
renderer = "portable_graph_html"
|
||||
output = "architecture.html"
|
||||
title = "Alpha architecture"
|
||||
root = "guide.workflow"
|
||||
"""
|
||||
|
||||
|
||||
class ClientIntegrationTests(unittest.TestCase):
|
||||
def copy_fixture(self, destination: Path) -> Path:
|
||||
|
|
@ -156,6 +171,256 @@ class ClientIntegrationTests(unittest.TestCase):
|
|||
)
|
||||
self.assertEqual("invalid_capability_binding", escalated.exception.code)
|
||||
|
||||
def test_nondefault_projection_policy_selectors_serialize_and_validate(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture(Path(directory))
|
||||
descriptor = root / ".docforge" / "project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8") + GRAPH_RENDER_CONFIG,
|
||||
encoding="utf-8",
|
||||
)
|
||||
project = Project.open(root)
|
||||
expected_arguments = [
|
||||
"-I",
|
||||
"-m",
|
||||
"docforge.mcp_server",
|
||||
"--project-root",
|
||||
str(root),
|
||||
"--capability-mode",
|
||||
"read",
|
||||
"--manual-render-policy",
|
||||
"disabled",
|
||||
"--portable-graph-policy",
|
||||
"disabled",
|
||||
"--live-viewer-policy",
|
||||
"disabled",
|
||||
]
|
||||
for client in ("codex", "claude", "openclaw"):
|
||||
with self.subTest(client=client):
|
||||
result = generate_client_configuration(
|
||||
project,
|
||||
client,
|
||||
manual_render_policy="disabled",
|
||||
portable_graph_policy="disabled",
|
||||
live_viewer_policy="disabled",
|
||||
)
|
||||
Draft202012Validator(CONFIGURATION_SCHEMA).validate(result)
|
||||
_validate_configuration_result(result)
|
||||
self.assertEqual(
|
||||
{
|
||||
"schema_version": 2,
|
||||
"manual": "disabled",
|
||||
"portable_graph": "disabled",
|
||||
"live_viewer": "disabled",
|
||||
},
|
||||
result["projection_policy"],
|
||||
)
|
||||
self.assertEqual(expected_arguments, result["binding"]["args"])
|
||||
content = result["artifact"]["content"]
|
||||
if client == "codex":
|
||||
document = tomllib.loads(content)
|
||||
serialized = document["mcp_servers"][result["server_name"]]["args"]
|
||||
elif client == "claude":
|
||||
document = json.loads(content)
|
||||
serialized = document["mcpServers"][result["server_name"]]["args"]
|
||||
else:
|
||||
document = json.loads(content)
|
||||
serialized = document["mcp"]["servers"][result["server_name"]]["args"]
|
||||
self.assertEqual(expected_arguments, serialized)
|
||||
canonical = json.dumps(
|
||||
result["projection_policy"],
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
).encode("utf-8")
|
||||
self.assertEqual(
|
||||
hashlib.sha256(canonical).hexdigest(),
|
||||
result["projection_policy_hash"],
|
||||
)
|
||||
|
||||
def test_projection_policy_schema_and_configuration_validation_reject_drift(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
project = Project.open(self.copy_fixture(Path(directory)))
|
||||
result = generate_client_configuration(
|
||||
project,
|
||||
"codex",
|
||||
manual_render_policy="disabled",
|
||||
live_viewer_policy="disabled",
|
||||
)
|
||||
validator = Draft202012Validator(CONFIGURATION_SCHEMA)
|
||||
validator.validate(result)
|
||||
_validate_configuration_result(result)
|
||||
|
||||
for field, value in (
|
||||
("schema_version", 1),
|
||||
("manual", "on-demand"),
|
||||
("portable_graph", "auto"),
|
||||
("live_viewer", "explicit"),
|
||||
):
|
||||
with self.subTest(field=field):
|
||||
drifted = json.loads(json.dumps(result))
|
||||
drifted["projection_policy"][field] = value
|
||||
self.assertTrue(list(validator.iter_errors(drifted)))
|
||||
|
||||
missing = json.loads(json.dumps(result))
|
||||
missing.pop("projection_policy")
|
||||
self.assertTrue(list(validator.iter_errors(missing)))
|
||||
extra = json.loads(json.dumps(result))
|
||||
extra["projection_policy"]["project_path"] = "/private/project"
|
||||
self.assertTrue(list(validator.iter_errors(extra)))
|
||||
|
||||
mismatched = json.loads(json.dumps(result))
|
||||
mismatched["projection_policy"]["manual"] = "explicit"
|
||||
canonical = json.dumps(
|
||||
mismatched["projection_policy"],
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
).encode("utf-8")
|
||||
mismatched["projection_policy_hash"] = hashlib.sha256(canonical).hexdigest()
|
||||
with self.assertRaises(AssertionError):
|
||||
_validate_configuration_result(mismatched)
|
||||
|
||||
bad_hash = json.loads(json.dumps(result))
|
||||
bad_hash["projection_policy_hash"] = "0" * 64
|
||||
with self.assertRaises(AssertionError):
|
||||
_validate_configuration_result(bad_hash)
|
||||
|
||||
defaulted = generate_client_configuration(project, "codex")
|
||||
self.assertNotIn(
|
||||
"--manual-render-policy",
|
||||
defaulted["binding"]["args"],
|
||||
)
|
||||
defaulted["projection_policy"]["manual"] = "disabled"
|
||||
canonical = json.dumps(
|
||||
defaulted["projection_policy"],
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
).encode("utf-8")
|
||||
defaulted["projection_policy_hash"] = hashlib.sha256(canonical).hexdigest()
|
||||
with self.assertRaises(AssertionError):
|
||||
_validate_configuration_result(defaulted)
|
||||
|
||||
unavailable = json.loads(json.dumps(result))
|
||||
unavailable["projection_availability"]["manual_configured"] = False
|
||||
with self.assertRaises(AssertionError):
|
||||
_validate_configuration_result(unavailable)
|
||||
|
||||
descriptor = project.descriptor.descriptor_path
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8") + GRAPH_RENDER_CONFIG,
|
||||
encoding="utf-8",
|
||||
)
|
||||
graph_project = Project.open(project.descriptor.root)
|
||||
graph_defaulted = generate_client_configuration(graph_project, "codex")
|
||||
self.assertNotIn(
|
||||
"--portable-graph-policy",
|
||||
graph_defaulted["binding"]["args"],
|
||||
)
|
||||
coordinated_graph_drift = json.loads(json.dumps(graph_defaulted))
|
||||
coordinated_graph_drift["projection_policy"]["portable_graph"] = "disabled"
|
||||
coordinated_graph_drift["projection_availability"]["portable_graph_configured"] = False
|
||||
canonical = json.dumps(
|
||||
coordinated_graph_drift["projection_policy"],
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
).encode("utf-8")
|
||||
coordinated_graph_drift["projection_policy_hash"] = hashlib.sha256(
|
||||
canonical
|
||||
).hexdigest()
|
||||
coordinated_graph_drift["configuration_hash"] = document_hash(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"client": coordinated_graph_drift["client"],
|
||||
"server_name": coordinated_graph_drift["server_name"],
|
||||
"project": coordinated_graph_drift["project"],
|
||||
"binding": coordinated_graph_drift["binding"],
|
||||
"effective_policy": coordinated_graph_drift["effective_policy"],
|
||||
"projection_policy": coordinated_graph_drift["projection_policy"],
|
||||
"projection_policy_hash": coordinated_graph_drift["projection_policy_hash"],
|
||||
"projection_availability": coordinated_graph_drift["projection_availability"],
|
||||
"artifact_format": coordinated_graph_drift["artifact"]["format"],
|
||||
"artifact_content_sha256": coordinated_graph_drift["artifact"][
|
||||
"content_sha256"
|
||||
],
|
||||
}
|
||||
)
|
||||
with self.assertRaises(AssertionError):
|
||||
_validate_configuration_result(coordinated_graph_drift)
|
||||
|
||||
invalid_selections = (
|
||||
{"manual_render_policy": "sometimes"},
|
||||
{"portable_graph_policy": "auto"},
|
||||
{"live_viewer_policy": "always"},
|
||||
)
|
||||
for selection in invalid_selections:
|
||||
with self.subTest(selection=selection):
|
||||
with self.assertRaises(DocForgeError) as raised:
|
||||
generate_client_configuration(graph_project, "codex", **selection)
|
||||
self.assertEqual("invalid_projection_policy", raised.exception.code)
|
||||
|
||||
def test_doctor_round_trips_projection_selectors_and_rejects_invalid_modes(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
parent = Path(directory)
|
||||
project = Project.open(self.copy_fixture(parent))
|
||||
ProjectIndex(project).build()
|
||||
config = parent / "openclaw.json"
|
||||
generated = generate_client_configuration(
|
||||
project,
|
||||
"openclaw",
|
||||
manual_render_policy="disabled",
|
||||
live_viewer_policy="disabled",
|
||||
output=config,
|
||||
)
|
||||
healthy = run_doctor(
|
||||
project,
|
||||
"openclaw",
|
||||
config_path=config,
|
||||
server_name=generated["server_name"],
|
||||
)
|
||||
Draft202012Validator(DOCTOR_SCHEMA).validate(healthy)
|
||||
self.assertEqual("healthy", healthy["doctor_state"])
|
||||
self.assertIn(
|
||||
"effective_policy_valid",
|
||||
[check["code"] for check in healthy["checks"]],
|
||||
)
|
||||
|
||||
document = json.loads(config.read_text(encoding="utf-8"))
|
||||
entry = document["mcp"]["servers"][generated["server_name"]]
|
||||
arguments = entry["args"]
|
||||
manual_position = arguments.index("--manual-render-policy") + 1
|
||||
arguments[manual_position] = "sometimes"
|
||||
config.write_text(json.dumps(document, sort_keys=True), encoding="utf-8")
|
||||
invalid = run_doctor(
|
||||
project,
|
||||
"openclaw",
|
||||
config_path=config,
|
||||
server_name=generated["server_name"],
|
||||
)
|
||||
Draft202012Validator(DOCTOR_SCHEMA).validate(invalid)
|
||||
self.assertEqual("unhealthy", invalid["doctor_state"])
|
||||
policy_check = next(
|
||||
check for check in invalid["checks"] if check["check_id"] == "policy.effective"
|
||||
)
|
||||
self.assertEqual("invalid_projection_policy", policy_check["code"])
|
||||
|
||||
arguments[manual_position] = "auto"
|
||||
config.write_text(json.dumps(document, sort_keys=True), encoding="utf-8")
|
||||
unavailable = run_doctor(
|
||||
project,
|
||||
"openclaw",
|
||||
config_path=config,
|
||||
server_name=generated["server_name"],
|
||||
)
|
||||
Draft202012Validator(DOCTOR_SCHEMA).validate(unavailable)
|
||||
self.assertEqual("unhealthy", unavailable["doctor_state"])
|
||||
policy_check = next(
|
||||
check for check in unavailable["checks"] if check["check_id"] == "policy.effective"
|
||||
)
|
||||
self.assertEqual("projection_policy_unavailable", policy_check["code"])
|
||||
|
||||
def test_explicit_fragment_write_is_atomic_conflict_aware_and_private(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
parent = Path(directory)
|
||||
|
|
|
|||
344
tests/test_graph_publication.py
Normal file
344
tests/test_graph_publication.py
Normal file
|
|
@ -0,0 +1,344 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import tempfile
|
||||
import tomllib
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from jsonschema import Draft202012Validator
|
||||
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.graph_rendering import GraphRenderService
|
||||
from docforge.models import ProjectState
|
||||
from docforge.project import Project
|
||||
from docforge.projection_contract import projection_hash
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURES = ROOT / "tests" / "fixtures"
|
||||
PROJECT_SCHEMA = json.loads((ROOT / "schemas/project.schema.json").read_text(encoding="utf-8"))
|
||||
|
||||
GRAPH_CONFIG = """
|
||||
|
||||
[graph_render]
|
||||
output_root = ".docforge/portable-graph"
|
||||
|
||||
[[graph_render.views]]
|
||||
id = "architecture"
|
||||
renderer = "portable_graph_html"
|
||||
output = "architecture.html"
|
||||
title = "Alpha architecture"
|
||||
root = "guide.workflow"
|
||||
initial_mode = "nodes"
|
||||
depth = 2
|
||||
max_nodes = 20
|
||||
max_edges = 40
|
||||
max_work = 1000
|
||||
families = ["guide", "proof"]
|
||||
relations = ["depends_on", "proves"]
|
||||
authorities = []
|
||||
statuses = []
|
||||
tags = []
|
||||
include_logic = false
|
||||
"""
|
||||
|
||||
|
||||
class GraphPublicationTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temporary.cleanup)
|
||||
self.root = Path(self.temporary.name) / "alpha"
|
||||
shutil.copytree(FIXTURES / "alpha", self.root)
|
||||
descriptor = self.root / ".docforge/project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
|
||||
encoding="utf-8",
|
||||
)
|
||||
self.project = Project.open(self.root)
|
||||
self.service = GraphRenderService(self.project)
|
||||
|
||||
def test_descriptor_schema_and_runtime_accept_the_separate_graph_view(self) -> None:
|
||||
document = tomllib.loads((self.root / ".docforge/project.toml").read_text(encoding="utf-8"))
|
||||
Draft202012Validator(PROJECT_SCHEMA).validate(document)
|
||||
config = self.project.descriptor.graph_render
|
||||
assert config is not None
|
||||
self.assertEqual(self.root / ".docforge/portable-graph", config.output_root)
|
||||
self.assertEqual("architecture", config.views[0].view_id)
|
||||
self.assertEqual("guide.workflow", config.views[0].root_node_id)
|
||||
self.assertIsNone(config.views[0].query)
|
||||
|
||||
def test_render_publication_is_deterministic_durable_and_unchanged_on_reuse(self) -> None:
|
||||
missing = self.service.status("architecture")
|
||||
self.assertEqual("stale", missing["state"])
|
||||
self.assertEqual("missing", missing["outputs"][0]["state"])
|
||||
|
||||
first = self.service.render("architecture")
|
||||
output = self.root / ".docforge/portable-graph/architecture.html"
|
||||
manifest = self.root / ".docforge/cache/projection-publications/graph/architecture.json"
|
||||
artifact_root = self.root / ".docforge/cache/projection-artifacts"
|
||||
self.assertEqual("current", first["state"])
|
||||
self.assertEqual("published", first["publication"])
|
||||
self.assertTrue(output.is_file())
|
||||
self.assertTrue(manifest.is_file())
|
||||
self.assertEqual(1, len(tuple(artifact_root.glob("*.html"))))
|
||||
before = output.stat()
|
||||
before_bytes = output.read_bytes()
|
||||
|
||||
current = self.service.status("architecture")
|
||||
self.assertEqual("current", current["state"])
|
||||
self.assertEqual("manifest", current["outputs"][0]["verification"])
|
||||
second = self.service.render("architecture")
|
||||
after = output.stat()
|
||||
self.assertEqual("unchanged", second["publication"])
|
||||
self.assertEqual(before_bytes, output.read_bytes())
|
||||
self.assertEqual((before.st_dev, before.st_ino), (after.st_dev, after.st_ino))
|
||||
|
||||
def test_status_is_manifest_only_and_detects_source_output_and_manifest_changes(self) -> None:
|
||||
self.service.render("architecture")
|
||||
output = self.root / ".docforge/portable-graph/architecture.html"
|
||||
manifest = self.root / ".docforge/cache/projection-publications/graph/architecture.json"
|
||||
with mock.patch.object(
|
||||
self.project,
|
||||
"load",
|
||||
side_effect=AssertionError("status must not load or plan"),
|
||||
):
|
||||
self.assertEqual("current", self.service.status("architecture")["state"])
|
||||
|
||||
output.write_bytes(output.read_bytes() + b"\n")
|
||||
changed_output = self.service.status("architecture")
|
||||
self.assertEqual("stale", changed_output["state"])
|
||||
self.assertEqual("output_changed", changed_output["outputs"][0]["reason"])
|
||||
|
||||
self.service.render("architecture")
|
||||
source = self.root / "docs/content/workflow.md"
|
||||
source.write_text(
|
||||
source.read_text(encoding="utf-8") + "\nChanged after publication.\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
changed_source = self.service.status("architecture")
|
||||
self.assertEqual("stale", changed_source["state"])
|
||||
self.assertEqual("stale", changed_source["outputs"][0]["state"])
|
||||
self.assertEqual(
|
||||
"source_generation_changed",
|
||||
changed_source["outputs"][0]["reason"],
|
||||
)
|
||||
|
||||
self.service.render("architecture")
|
||||
manifest.write_text("{bad-json", encoding="utf-8")
|
||||
corrupt = self.service.status("architecture")
|
||||
self.assertEqual("missing", corrupt["outputs"][0]["state"])
|
||||
|
||||
def test_status_validates_nested_manifest_and_artifact_store_evidence(self) -> None:
|
||||
self.service.render("architecture")
|
||||
manifest_path = (
|
||||
self.root / ".docforge/cache/projection-publications/graph/architecture.json"
|
||||
)
|
||||
original = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
mutations = {
|
||||
"bad_project": lambda value: value.__setitem__("project", "bad"),
|
||||
"forged_artifact": lambda value: value.__setitem__(
|
||||
"artifact",
|
||||
{
|
||||
"artifact_id": "portable-graph.html",
|
||||
"media_type": "text/html; charset=utf-8",
|
||||
"sha256": "0" * 64,
|
||||
"bytes": 1,
|
||||
},
|
||||
),
|
||||
"bad_store": lambda value: value["store"].__setitem__("size", -1),
|
||||
}
|
||||
for name, mutate in mutations.items():
|
||||
with self.subTest(name=name):
|
||||
value = json.loads(json.dumps(original))
|
||||
mutate(value)
|
||||
value.pop("publication_id")
|
||||
value["publication_id"] = projection_hash(value)
|
||||
manifest_path.write_text(
|
||||
json.dumps(value, sort_keys=True, indent=2) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
status = self.service.status("architecture")
|
||||
self.assertEqual("unverified", status["outputs"][0]["state"])
|
||||
self.assertEqual("manifest_invalid", status["outputs"][0]["reason"])
|
||||
manifest_path.write_text(
|
||||
json.dumps(original, sort_keys=True, indent=2) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
artifact = next((self.root / ".docforge/cache/projection-artifacts").glob("*.html"))
|
||||
artifact.unlink()
|
||||
missing = self.service.status("architecture")
|
||||
self.assertEqual("stale", missing["outputs"][0]["state"])
|
||||
self.assertEqual("artifact_store_missing", missing["outputs"][0]["reason"])
|
||||
repaired = self.service.render("architecture")
|
||||
self.assertEqual("current", repaired["state"])
|
||||
self.assertTrue(artifact.is_file())
|
||||
|
||||
def test_status_detects_source_and_publication_races(self) -> None:
|
||||
self.service.render("architecture")
|
||||
current = self.project.incremental_state()
|
||||
assert current is not None
|
||||
changed = ProjectState(source_hash="0" * 64, revision="changed")
|
||||
with mock.patch.object(
|
||||
self.project,
|
||||
"incremental_state",
|
||||
side_effect=(current, changed),
|
||||
):
|
||||
raced_source = self.service.status("architecture")
|
||||
self.assertEqual("stale", raced_source["state"])
|
||||
self.assertEqual(
|
||||
"source_changed_during_status",
|
||||
raced_source["outputs"][0]["reason"],
|
||||
)
|
||||
|
||||
baseline = self.service._manifest_status(
|
||||
self.project.descriptor.graph_render.views[0], # type: ignore[union-attr]
|
||||
current,
|
||||
)
|
||||
replaced = dict(baseline)
|
||||
replaced["publication_id"] = "f" * 64
|
||||
with mock.patch.object(
|
||||
self.service,
|
||||
"_manifest_status",
|
||||
side_effect=(baseline, replaced),
|
||||
):
|
||||
raced_publication = self.service.status("architecture")
|
||||
self.assertEqual("stale", raced_publication["state"])
|
||||
self.assertEqual(
|
||||
"publication_changed_during_status",
|
||||
raced_publication["outputs"][0]["reason"],
|
||||
)
|
||||
|
||||
def test_manifest_failure_after_output_is_degraded_success(self) -> None:
|
||||
with mock.patch.object(
|
||||
self.service,
|
||||
"_publish_manifest",
|
||||
side_effect=DocForgeError(
|
||||
"publication_failure",
|
||||
"Synthetic manifest failure",
|
||||
),
|
||||
):
|
||||
result = self.service.render("architecture")
|
||||
self.assertEqual("degraded", result["state"])
|
||||
self.assertEqual("published", result["publication"])
|
||||
self.assertEqual("manifest", result["committed_stage"])
|
||||
self.assertTrue((self.root / ".docforge/portable-graph/architecture.html").is_file())
|
||||
self.assertEqual("failed", result["manifest"]["state"])
|
||||
|
||||
def test_post_commit_stage_failures_are_degraded_and_precommit_failures_raise(self) -> None:
|
||||
committed = DocForgeError(
|
||||
"publication_failure",
|
||||
"Synthetic committed failure",
|
||||
mutation_committed=True,
|
||||
)
|
||||
cases = (
|
||||
("_publish_artifact", "artifact_store", "partial", False),
|
||||
("_publish_output", "output", "published", None),
|
||||
)
|
||||
for method, stage, publication, output_exists in cases:
|
||||
with self.subTest(method=method):
|
||||
root = Path(self.temporary.name) / method
|
||||
shutil.copytree(FIXTURES / "alpha", root)
|
||||
descriptor = root / ".docforge/project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
|
||||
encoding="utf-8",
|
||||
)
|
||||
service = GraphRenderService(Project.open(root))
|
||||
with mock.patch.object(service, method, side_effect=committed):
|
||||
result = service.render("architecture")
|
||||
self.assertEqual("degraded", result["state"])
|
||||
self.assertEqual(stage, result["committed_stage"])
|
||||
self.assertEqual(publication, result["publication"])
|
||||
if output_exists is not None:
|
||||
self.assertEqual(
|
||||
output_exists,
|
||||
(root / ".docforge/portable-graph/architecture.html").exists(),
|
||||
)
|
||||
|
||||
uncommitted = DocForgeError(
|
||||
"publication_failure",
|
||||
"Synthetic precommit failure",
|
||||
mutation_committed=False,
|
||||
)
|
||||
with (
|
||||
mock.patch.object(
|
||||
self.service,
|
||||
"_publish_output",
|
||||
side_effect=uncommitted,
|
||||
),
|
||||
self.assertRaises(DocForgeError),
|
||||
):
|
||||
self.service.render("architecture")
|
||||
|
||||
def test_configuration_rejects_unsafe_ambiguous_and_overlapping_views(self) -> None:
|
||||
cases = {
|
||||
"both_scope": GRAPH_CONFIG.replace(
|
||||
'root = "guide.workflow"',
|
||||
'root = "guide.workflow"\nquery = "workflow"',
|
||||
),
|
||||
"output_overlap": GRAPH_CONFIG.replace(
|
||||
'output_root = ".docforge/portable-graph"',
|
||||
'output_root = "docs/content"',
|
||||
),
|
||||
"active_renderer": GRAPH_CONFIG.replace(
|
||||
'renderer = "portable_graph_html"',
|
||||
'renderer = "shell"',
|
||||
),
|
||||
"oversized": GRAPH_CONFIG.replace("max_nodes = 20", "max_nodes = 100000"),
|
||||
"logic_mode": GRAPH_CONFIG.replace('initial_mode = "nodes"', 'initial_mode = "logic"'),
|
||||
"logic_projection": GRAPH_CONFIG.replace(
|
||||
"include_logic = false",
|
||||
"include_logic = true",
|
||||
),
|
||||
"long_title": GRAPH_CONFIG.replace(
|
||||
'title = "Alpha architecture"',
|
||||
f'title = "{"x" * 1025}"',
|
||||
),
|
||||
"long_query": GRAPH_CONFIG.replace(
|
||||
'root = "guide.workflow"',
|
||||
f'query = "{"x" * 10001}"',
|
||||
),
|
||||
"too_many_filters": GRAPH_CONFIG.replace(
|
||||
'families = ["guide", "proof"]',
|
||||
"families = [" + ", ".join(f'"family-{index}"' for index in range(65)) + "]",
|
||||
),
|
||||
}
|
||||
for name, graph_config in cases.items():
|
||||
with self.subTest(name=name):
|
||||
root = Path(self.temporary.name) / name
|
||||
shutil.copytree(FIXTURES / "alpha", root)
|
||||
descriptor = root / ".docforge/project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8") + graph_config,
|
||||
encoding="utf-8",
|
||||
)
|
||||
parsed = tomllib.loads(descriptor.read_text(encoding="utf-8"))
|
||||
if name != "output_overlap":
|
||||
self.assertFalse(Draft202012Validator(PROJECT_SCHEMA).is_valid(parsed))
|
||||
with self.assertRaises(DocForgeError):
|
||||
Project.open(root)
|
||||
|
||||
relocated = Path(self.temporary.name) / "descriptor-overlap"
|
||||
shutil.copytree(FIXTURES / "alpha", relocated)
|
||||
descriptor = relocated / ".docforge/project.toml"
|
||||
base = descriptor.read_text(encoding="utf-8").replace(
|
||||
'cache_root = ".docforge/cache"\nindex = ".docforge/cache/index.sqlite3"',
|
||||
'cache_root = "var/cache"\nindex = "var/cache/index.sqlite3"',
|
||||
)
|
||||
descriptor.write_text(
|
||||
base
|
||||
+ GRAPH_CONFIG.replace(
|
||||
'output_root = ".docforge/portable-graph"',
|
||||
'output_root = ".docforge"',
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
with self.assertRaises(DocForgeError):
|
||||
Project.open(relocated)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
284
tests/test_graph_rendering.py
Normal file
284
tests/test_graph_rendering.py
Normal file
|
|
@ -0,0 +1,284 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sqlite3
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
from unittest import mock
|
||||
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.graph_projection import (
|
||||
GraphViewRequestV1,
|
||||
build_graph_projection_package,
|
||||
build_graph_view_plan,
|
||||
)
|
||||
from docforge.project import Project
|
||||
from docforge.projection_contract import ProjectionPackageV1
|
||||
from docforge_renderers.graph import PortableGraphHtmlRenderer
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURES = ROOT / "tests" / "fixtures"
|
||||
|
||||
|
||||
class PortableGraphRenderingTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temporary.cleanup)
|
||||
self.root = Path(self.temporary.name) / "alpha"
|
||||
shutil.copytree(FIXTURES / "alpha", self.root)
|
||||
self.project = Project.open(self.root)
|
||||
self.snapshot = self.project.load()
|
||||
self.request = GraphViewRequestV1(
|
||||
view_id="architecture",
|
||||
title="Alpha architecture",
|
||||
root_node_id="guide.workflow",
|
||||
depth=2,
|
||||
max_nodes=20,
|
||||
max_edges=40,
|
||||
max_work=1_000,
|
||||
)
|
||||
self.plan = build_graph_view_plan(self.snapshot, self.request, False)
|
||||
self.package = build_graph_projection_package(
|
||||
self.plan,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=1_000_000,
|
||||
)
|
||||
|
||||
def test_portable_artifact_is_deterministic_self_contained_and_generation_bound(self) -> None:
|
||||
renderer = PortableGraphHtmlRenderer()
|
||||
first = renderer.render(self.package)
|
||||
second = renderer.render(self.package)
|
||||
self.assertEqual(first.artifacts, second.artifacts)
|
||||
self.assertEqual(1, len(first.artifacts))
|
||||
artifact = first.artifacts[0]
|
||||
self.assertEqual("portable-graph.html", artifact.artifact_id)
|
||||
self.assertEqual("text/html; charset=utf-8", artifact.media_type)
|
||||
receipt = first.receipt.as_dict()
|
||||
evidence = cast(list[dict[str, object]], receipt["artifacts"])
|
||||
self.assertEqual(
|
||||
hashlib.sha256(artifact.content).hexdigest(),
|
||||
evidence[0]["sha256"],
|
||||
)
|
||||
rendered = artifact.content.decode("utf-8")
|
||||
self.assertIn("Content-Security-Policy", rendered)
|
||||
self.assertIn("default-src 'none'", rendered)
|
||||
self.assertIn('type="application/json"', rendered)
|
||||
self.assertNotIn(str(self.root), rendered)
|
||||
self.assertNotIn("docs/content/", rendered)
|
||||
self.assertNotIn("Editors change canonical nodes", rendered)
|
||||
self.assertNotIn("fetch(", rendered)
|
||||
self.assertNotIn("XMLHttpRequest", rendered)
|
||||
self.assertNotIn("WebSocket", rendered)
|
||||
self.assertIn(self.snapshot.source_hash, rendered)
|
||||
|
||||
def test_embedded_plan_is_exact_and_script_breakout_is_inert(self) -> None:
|
||||
plan = copy.deepcopy(self.plan.as_dict())
|
||||
view = plan["view"]
|
||||
assert isinstance(view, dict)
|
||||
view["title"] = '</script><script id="injected">alert(1)</script>'
|
||||
plan.pop("plan_id")
|
||||
from docforge.projection_contract import GraphViewPlanV1
|
||||
|
||||
malicious = GraphViewPlanV1.create(plan)
|
||||
package = build_graph_projection_package(
|
||||
malicious,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=1_000_000,
|
||||
)
|
||||
rendered = PortableGraphHtmlRenderer().render(package).artifacts[0].content.decode("utf-8")
|
||||
self.assertNotIn('<script id="injected">', rendered)
|
||||
self.assertIn("</script>", rendered)
|
||||
prefix = '<script id="docforge-graph-plan" type="application/json">'
|
||||
embedded = rendered.split(prefix, 1)[1].split("</script>", 1)[0]
|
||||
self.assertEqual(malicious.as_dict(), json.loads(embedded))
|
||||
|
||||
def test_renderer_has_no_project_database_or_filesystem_write_capability(self) -> None:
|
||||
forbidden = AssertionError("portable graph renderer crossed its capability boundary")
|
||||
with (
|
||||
mock.patch.object(Project, "open", side_effect=forbidden),
|
||||
mock.patch.object(Project, "load", side_effect=forbidden),
|
||||
mock.patch.object(sqlite3, "connect", side_effect=forbidden),
|
||||
mock.patch.object(Path, "write_bytes", side_effect=forbidden),
|
||||
mock.patch.object(Path, "write_text", side_effect=forbidden),
|
||||
mock.patch.object(Path, "mkdir", side_effect=forbidden),
|
||||
mock.patch.object(os, "replace", side_effect=forbidden),
|
||||
mock.patch.object(os, "rename", side_effect=forbidden),
|
||||
mock.patch.object(os, "unlink", side_effect=forbidden),
|
||||
):
|
||||
result = PortableGraphHtmlRenderer().render(self.package)
|
||||
self.assertEqual(1, len(result.artifacts))
|
||||
|
||||
def test_renderer_rejects_wrong_kind_identity_assets_and_size(self) -> None:
|
||||
wrong = ProjectionPackageV1.create(
|
||||
kind="graph",
|
||||
plan=self.plan,
|
||||
renderer={
|
||||
"renderer_id": PortableGraphHtmlRenderer.renderer_id,
|
||||
"renderer_version": "other",
|
||||
},
|
||||
components=[],
|
||||
assets=[],
|
||||
output_policy={
|
||||
"artifact_ids": ["portable-graph.html"],
|
||||
"max_total_bytes": 1_000_000,
|
||||
},
|
||||
)
|
||||
with self.assertRaises(DocForgeError) as unsupported:
|
||||
PortableGraphHtmlRenderer().render(wrong)
|
||||
self.assertEqual("unsupported_renderer", unsupported.exception.code)
|
||||
|
||||
with_asset = ProjectionPackageV1.create(
|
||||
kind="graph",
|
||||
plan=self.plan,
|
||||
renderer={
|
||||
"renderer_id": PortableGraphHtmlRenderer.renderer_id,
|
||||
"renderer_version": PortableGraphHtmlRenderer.renderer_version,
|
||||
},
|
||||
components=[],
|
||||
assets=[
|
||||
{
|
||||
"asset_id": "project-script",
|
||||
"media_type": "text/javascript",
|
||||
"sha256": "0" * 64,
|
||||
"text": "alert(1)",
|
||||
}
|
||||
],
|
||||
output_policy={"artifact_ids": ["portable-graph.html"], "max_total_bytes": 1_000_000},
|
||||
)
|
||||
with self.assertRaises(DocForgeError) as assets:
|
||||
PortableGraphHtmlRenderer().render(with_asset)
|
||||
self.assertEqual("invalid_projection", assets.exception.code)
|
||||
|
||||
wrong_components = ProjectionPackageV1.create(
|
||||
kind="graph",
|
||||
plan=self.plan,
|
||||
renderer={
|
||||
"renderer_id": PortableGraphHtmlRenderer.renderer_id,
|
||||
"renderer_version": PortableGraphHtmlRenderer.renderer_version,
|
||||
},
|
||||
components=[],
|
||||
assets=[],
|
||||
output_policy={
|
||||
"artifact_ids": ["portable-graph.html"],
|
||||
"max_total_bytes": 1_000_000,
|
||||
},
|
||||
)
|
||||
with self.assertRaises(DocForgeError) as components:
|
||||
PortableGraphHtmlRenderer().render(wrong_components)
|
||||
self.assertEqual("invalid_projection", components.exception.code)
|
||||
|
||||
wrong_artifact = ProjectionPackageV1.create(
|
||||
kind="graph",
|
||||
plan=self.plan,
|
||||
renderer={
|
||||
"renderer_id": PortableGraphHtmlRenderer.renderer_id,
|
||||
"renderer_version": PortableGraphHtmlRenderer.renderer_version,
|
||||
},
|
||||
components=self.package.document["components"], # type: ignore[arg-type]
|
||||
assets=[],
|
||||
output_policy={
|
||||
"artifact_ids": ["unexpected.html"],
|
||||
"max_total_bytes": 1_000_000,
|
||||
},
|
||||
)
|
||||
with self.assertRaises(DocForgeError) as artifact:
|
||||
PortableGraphHtmlRenderer().render(wrong_artifact)
|
||||
self.assertEqual("invalid_projection", artifact.exception.code)
|
||||
|
||||
tiny = build_graph_projection_package(
|
||||
self.plan,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=1,
|
||||
)
|
||||
with self.assertRaises(DocForgeError) as too_large:
|
||||
PortableGraphHtmlRenderer().render(tiny)
|
||||
self.assertEqual("render_too_large", too_large.exception.code)
|
||||
|
||||
def test_initial_mode_is_honored_and_logic_is_rejected(self) -> None:
|
||||
for mode in ("nodes", "flow", "web"):
|
||||
with self.subTest(mode=mode):
|
||||
plan = build_graph_view_plan(
|
||||
self.snapshot,
|
||||
GraphViewRequestV1(
|
||||
view_id=f"{mode}-view",
|
||||
title=f"{mode.title()} view",
|
||||
root_node_id="guide.workflow",
|
||||
initial_mode=mode, # type: ignore[arg-type]
|
||||
depth=2,
|
||||
max_nodes=20,
|
||||
max_edges=40,
|
||||
max_work=1_000,
|
||||
),
|
||||
False,
|
||||
)
|
||||
package = build_graph_projection_package(
|
||||
plan,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=1_000_000,
|
||||
)
|
||||
rendered = (
|
||||
PortableGraphHtmlRenderer().render(package).artifacts[0].content.decode("utf-8")
|
||||
)
|
||||
self.assertIn(f'data-mode="{mode}"', rendered)
|
||||
self.assertIn(f'<option value="{mode}" selected>', rendered)
|
||||
|
||||
logic_plan = build_graph_view_plan(
|
||||
self.snapshot,
|
||||
GraphViewRequestV1(
|
||||
view_id="logic-view",
|
||||
title="Logic view",
|
||||
root_node_id="guide.workflow",
|
||||
initial_mode="logic",
|
||||
include_logic=True,
|
||||
),
|
||||
True,
|
||||
)
|
||||
logic_package = build_graph_projection_package(
|
||||
logic_plan,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=1_000_000,
|
||||
)
|
||||
with self.assertRaises(DocForgeError) as logic:
|
||||
PortableGraphHtmlRenderer().render(logic_package)
|
||||
self.assertEqual("unsupported_renderer", logic.exception.code)
|
||||
|
||||
def test_artifact_exposes_accessible_static_and_interactive_structure(self) -> None:
|
||||
rendered = (
|
||||
PortableGraphHtmlRenderer().render(self.package).artifacts[0].content.decode("utf-8")
|
||||
)
|
||||
static = rendered.split('<script id="docforge-graph-plan"', 1)[0]
|
||||
self.assertIn("Editing workflow (guide.workflow)", static)
|
||||
self.assertIn("<td>guide.workflow</td>", static)
|
||||
self.assertIn('data-panel="nodes"', static)
|
||||
self.assertIn('data-panel="relationships"', static)
|
||||
for expected in (
|
||||
'href="#main"',
|
||||
'<main id="main" tabindex="-1" data-mode="nodes">',
|
||||
'role="group" aria-label="Graph controls"',
|
||||
'role="status" aria-live="polite"',
|
||||
'<th scope="col">Source</th>',
|
||||
'<th scope="col">Relation</th>',
|
||||
'<th scope="col">Target</th>',
|
||||
'<dialog id="node-dialog"',
|
||||
'aria-labelledby="node-dialog-title"',
|
||||
'dialog.addEventListener("close", () => opener?.focus())',
|
||||
"@media (prefers-reduced-motion: reduce)",
|
||||
"@media (forced-colors: active)",
|
||||
):
|
||||
with self.subTest(expected=expected):
|
||||
self.assertIn(expected, rendered)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -3,6 +3,7 @@ from __future__ import annotations
|
|||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from docforge.cli import _parser, _run
|
||||
from docforge.errors import DocForgeError
|
||||
|
|
@ -10,6 +11,28 @@ from docforge.project import Project
|
|||
|
||||
|
||||
class DocForgeOnboardingTests(unittest.TestCase):
|
||||
def test_scaffold_honors_disabled_manual_projection_policy(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory) / "disabled-render"
|
||||
root.mkdir()
|
||||
arguments = _parser().parse_args(
|
||||
[
|
||||
"--project-root",
|
||||
str(root),
|
||||
"--manual-render-policy",
|
||||
"disabled",
|
||||
"onboard",
|
||||
"--scaffold",
|
||||
]
|
||||
)
|
||||
with mock.patch(
|
||||
"docforge.cli.RenderService.render",
|
||||
side_effect=AssertionError("disabled onboarding must not render"),
|
||||
):
|
||||
result = _run(arguments)
|
||||
self.assertEqual("skipped", result["render"]["state"])
|
||||
self.assertFalse((root / ".docforge/rendered/manual.html").exists())
|
||||
|
||||
def test_assessment_detects_multiple_languages_without_writing(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory) / "polyglot"
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ from unittest import mock
|
|||
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.manual_projection import (
|
||||
_cycles,
|
||||
build_manual_projection_package,
|
||||
build_manual_render_plan,
|
||||
)
|
||||
|
|
@ -30,6 +31,8 @@ from docforge.projection_contract import (
|
|||
canonical_projection_bytes,
|
||||
projection_hash,
|
||||
)
|
||||
from docforge.projection_fragments import FRAGMENT_CACHE_DIRECTORY, FragmentRecord
|
||||
from docforge.projection_worker import render_projection_in_worker
|
||||
from docforge.render_contract import GenericHtmlRenderer
|
||||
from docforge_renderers.manual import ManualHtmlRenderer
|
||||
|
||||
|
|
@ -388,6 +391,21 @@ class ProjectionContractTests(unittest.TestCase):
|
|||
workflow["cross_references"],
|
||||
)
|
||||
|
||||
def test_cycle_detection_handles_the_maximum_deep_graph_iteratively(self) -> None:
|
||||
node_ids = tuple(f"node.{index:05d}" for index in range(10_000))
|
||||
chain = tuple(
|
||||
Edge(node_ids[index], "depends_on", node_ids[index + 1])
|
||||
for index in range(len(node_ids) - 1)
|
||||
)
|
||||
self.assertEqual([], _cycles(node_ids, chain))
|
||||
self.assertEqual(
|
||||
[list(node_ids)],
|
||||
_cycles(
|
||||
node_ids,
|
||||
(*chain, Edge(node_ids[-1], "depends_on", node_ids[0])),
|
||||
),
|
||||
)
|
||||
|
||||
def test_alpha_compatibility_shim_preserves_legacy_identity_and_bytes(self) -> None:
|
||||
renderer = GenericHtmlRenderer()
|
||||
self.assertEqual(ALPHA_RENDERER_VERSION, renderer.renderer_version)
|
||||
|
|
@ -412,6 +430,171 @@ class ProjectionContractTests(unittest.TestCase):
|
|||
prepared.output,
|
||||
)
|
||||
|
||||
def test_manual_fragments_are_incremental_and_full_worker_output_is_the_oracle(
|
||||
self,
|
||||
) -> None:
|
||||
renderer = GenericHtmlRenderer()
|
||||
with mock.patch(
|
||||
"docforge.render_contract.render_projection_in_worker",
|
||||
wraps=render_projection_in_worker,
|
||||
) as detached:
|
||||
cold = renderer.prepare(
|
||||
self.snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.assertEqual(2, detached.call_count)
|
||||
fragment_root = self.snapshot.descriptor.cache_root / FRAGMENT_CACHE_DIRECTORY
|
||||
self.assertEqual(
|
||||
len(self.plan.document["pages"]),
|
||||
len(list(fragment_root.glob("*.json"))),
|
||||
)
|
||||
|
||||
with mock.patch(
|
||||
"docforge.render_contract.render_projection_in_worker",
|
||||
wraps=render_projection_in_worker,
|
||||
) as detached:
|
||||
warm = renderer.prepare(
|
||||
self.snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.assertEqual(1, detached.call_count)
|
||||
self.assertEqual(cold.output, warm.output)
|
||||
|
||||
with mock.patch(
|
||||
"docforge.render_contract.render_projection_in_worker",
|
||||
wraps=render_projection_in_worker,
|
||||
) as detached:
|
||||
full = GenericHtmlRenderer(incremental=False).prepare(
|
||||
self.snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.assertEqual(1, detached.call_count)
|
||||
self.assertEqual(full.output, warm.output)
|
||||
|
||||
first_fragment = sorted(fragment_root.glob("*.json"))[0]
|
||||
first_fragment.write_bytes(b"{corrupt")
|
||||
with mock.patch(
|
||||
"docforge.render_contract.render_projection_in_worker",
|
||||
wraps=render_projection_in_worker,
|
||||
) as detached:
|
||||
recovered = renderer.prepare(
|
||||
self.snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.assertEqual(2, detached.call_count)
|
||||
self.assertEqual(full.output, recovered.output)
|
||||
|
||||
original_record = FragmentRecord.from_bytes(first_fragment.read_bytes())
|
||||
first_fragment.write_bytes(
|
||||
FragmentRecord.create(
|
||||
original_record.key,
|
||||
b"<script>forged fragment</script>",
|
||||
).to_bytes()
|
||||
)
|
||||
with mock.patch(
|
||||
"docforge.render_contract.render_projection_in_worker",
|
||||
wraps=render_projection_in_worker,
|
||||
) as detached:
|
||||
forged = renderer.prepare(
|
||||
self.snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.assertEqual(2, detached.call_count)
|
||||
self.assertEqual(full.output, forged.output)
|
||||
self.assertNotIn(b"forged fragment", forged.output)
|
||||
|
||||
def test_oversized_manual_fragment_bypasses_cache_and_uses_full_worker(self) -> None:
|
||||
content = "x" * 4_100_000
|
||||
node = replace(
|
||||
self.snapshot.nodes[0],
|
||||
content=content,
|
||||
content_hash=hashlib.sha256(content.encode()).hexdigest(),
|
||||
)
|
||||
snapshot = replace(
|
||||
self.snapshot,
|
||||
descriptor=replace(
|
||||
self.snapshot.descriptor,
|
||||
limits=replace(
|
||||
self.snapshot.descriptor.limits,
|
||||
max_render_bytes=20_000_000,
|
||||
),
|
||||
),
|
||||
nodes=(node,),
|
||||
edges=(),
|
||||
source_hash=hashlib.sha256(b"oversized-fragment").hexdigest(),
|
||||
)
|
||||
with mock.patch(
|
||||
"docforge.render_contract.render_projection_in_worker",
|
||||
wraps=render_projection_in_worker,
|
||||
) as detached:
|
||||
prepared = GenericHtmlRenderer().prepare(
|
||||
snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.assertEqual(1, detached.call_count)
|
||||
full = GenericHtmlRenderer(incremental=False).prepare(
|
||||
snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.assertEqual(full.output, prepared.output)
|
||||
|
||||
def test_fragment_package_overflow_falls_back_to_full_worker(self) -> None:
|
||||
content = "x" * 2_700_000
|
||||
base = self.snapshot.nodes[0]
|
||||
nodes = tuple(
|
||||
replace(
|
||||
base,
|
||||
node_id=f"guide.large-{index}",
|
||||
title=f"Large {index}",
|
||||
content=content,
|
||||
source_path=f"docs/content/large-{index}.md",
|
||||
content_hash=hashlib.sha256(f"{index}:{content}".encode()).hexdigest(),
|
||||
)
|
||||
for index in range(4)
|
||||
)
|
||||
descriptor = replace(
|
||||
self.snapshot.descriptor,
|
||||
limits=replace(
|
||||
self.snapshot.descriptor.limits,
|
||||
max_render_bytes=20_000_000,
|
||||
),
|
||||
)
|
||||
snapshot = replace(
|
||||
self.snapshot,
|
||||
descriptor=descriptor,
|
||||
nodes=nodes,
|
||||
edges=(),
|
||||
source_hash=hashlib.sha256(b"large-fragment-fixture").hexdigest(),
|
||||
)
|
||||
incremental = GenericHtmlRenderer().prepare(
|
||||
snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
full = GenericHtmlRenderer(incremental=False).prepare(
|
||||
snapshot,
|
||||
self.view,
|
||||
self.template,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.assertGreater(len(full.output), 10_000_000)
|
||||
self.assertEqual(full.output, incremental.output)
|
||||
|
||||
def test_manual_renderer_rejects_project_provided_active_content(self) -> None:
|
||||
for active in (
|
||||
"<script>alert(1)</script>{{ docforge_content }}",
|
||||
|
|
|
|||
205
tests/test_projection_fragments.py
Normal file
205
tests/test_projection_fragments.py
Normal file
|
|
@ -0,0 +1,205 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from docforge.projection_fragments import (
|
||||
FRAGMENT_CACHE_DIRECTORY,
|
||||
FRAGMENT_KEY_CONTRACT,
|
||||
FRAGMENT_RECORD_CONTRACT,
|
||||
FRAGMENT_SCHEMA_VERSION,
|
||||
FragmentKey,
|
||||
FragmentRecord,
|
||||
ProjectionFragmentCache,
|
||||
fragment_semantic_hash,
|
||||
)
|
||||
|
||||
|
||||
class ProjectionFragmentCacheTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temporary.cleanup)
|
||||
self.root = Path(self.temporary.name) / "project"
|
||||
self.root.mkdir()
|
||||
self.cache_root = self.root / ".docforge" / "cache"
|
||||
self.cache = ProjectionFragmentCache(
|
||||
self.root,
|
||||
self.cache_root,
|
||||
maximum_content_bytes=1_000,
|
||||
)
|
||||
self.key = self.make_key({"node": "guide.alpha", "summary": "Alpha"})
|
||||
|
||||
@staticmethod
|
||||
def make_key(
|
||||
semantics: object,
|
||||
*,
|
||||
renderer_version: str = "1",
|
||||
component_version: str = "graph.node@1",
|
||||
) -> FragmentKey:
|
||||
return FragmentKey.create(
|
||||
projection_kind="graph",
|
||||
renderer_id="portable_graph_html",
|
||||
renderer_version=renderer_version,
|
||||
component_version=component_version,
|
||||
semantic_input_hash=fragment_semantic_hash(semantics),
|
||||
)
|
||||
|
||||
@property
|
||||
def entry_path(self) -> Path:
|
||||
return self.cache_root / FRAGMENT_CACHE_DIRECTORY / f"{self.key.key_id}.json"
|
||||
|
||||
def test_miss_hit_and_unchanged_reuse_are_exact(self) -> None:
|
||||
self.assertIsNone(self.cache.get(self.key))
|
||||
with mock.patch(
|
||||
"docforge.projection_fragments.atomic_replace_bytes_at",
|
||||
wraps=__import__(
|
||||
"docforge.projection_fragments",
|
||||
fromlist=["atomic_replace_bytes_at"],
|
||||
).atomic_replace_bytes_at,
|
||||
) as atomic:
|
||||
stored = self.cache.put(self.key, b"<li>Alpha</li>")
|
||||
self.assertIsNotNone(stored)
|
||||
self.assertEqual(1, atomic.call_count)
|
||||
before = self.entry_path.stat()
|
||||
repeated = self.cache.put(self.key, b"<li>Alpha</li>")
|
||||
self.assertEqual(1, atomic.call_count)
|
||||
self.assertEqual(stored, repeated)
|
||||
self.assertEqual(stored, self.cache.get(self.key))
|
||||
after = self.entry_path.stat()
|
||||
self.assertEqual((before.st_dev, before.st_ino), (after.st_dev, after.st_ino))
|
||||
|
||||
def test_key_versions_and_complete_semantics_invalidate_independently(self) -> None:
|
||||
self.assertIsNotNone(self.cache.put(self.key, b"alpha"))
|
||||
variants = (
|
||||
self.make_key(
|
||||
{"node": "guide.alpha", "summary": "Alpha"},
|
||||
renderer_version="2",
|
||||
),
|
||||
self.make_key(
|
||||
{"node": "guide.alpha", "summary": "Alpha"},
|
||||
component_version="graph.node@2",
|
||||
),
|
||||
self.make_key({"node": "guide.alpha", "summary": "Changed"}),
|
||||
)
|
||||
for variant in variants:
|
||||
with self.subTest(key=variant):
|
||||
self.assertNotEqual(self.key.key_id, variant.key_id)
|
||||
self.assertIsNone(self.cache.get(variant))
|
||||
|
||||
def test_add_delete_and_reorder_change_sequence_semantics(self) -> None:
|
||||
base = ["a", "b"]
|
||||
added = ["a", "b", "c"]
|
||||
deleted = ["a"]
|
||||
reordered = ["b", "a"]
|
||||
keys = [self.make_key(value) for value in (base, added, deleted, reordered)]
|
||||
self.assertEqual(4, len({key.key_id for key in keys}))
|
||||
self.assertEqual(
|
||||
fragment_semantic_hash({"a": 1, "b": 2}),
|
||||
fragment_semantic_hash({"b": 2, "a": 1}),
|
||||
)
|
||||
|
||||
def test_corrupt_oversized_incompatible_and_foreign_entries_are_misses(self) -> None:
|
||||
self.assertIsNotNone(self.cache.put(self.key, b"alpha"))
|
||||
self.entry_path.write_bytes(b"{bad-json")
|
||||
self.assertIsNone(self.cache.get(self.key))
|
||||
self.assertIsNotNone(self.cache.put(self.key, b"repaired"))
|
||||
repaired = self.cache.get(self.key)
|
||||
self.assertIsNotNone(repaired)
|
||||
assert repaired is not None
|
||||
self.assertEqual(b"repaired", repaired.content)
|
||||
|
||||
self.entry_path.write_bytes(b"x" * (self.cache.maximum_record_bytes + 1))
|
||||
self.assertIsNone(self.cache.get(self.key))
|
||||
|
||||
incompatible = FragmentRecord.create(self.key, b"alpha").as_dict()
|
||||
incompatible["schema_version"] = FRAGMENT_SCHEMA_VERSION + 1
|
||||
self.entry_path.write_bytes(
|
||||
json.dumps(incompatible, sort_keys=True, separators=(",", ":")).encode()
|
||||
)
|
||||
self.assertIsNone(self.cache.get(self.key))
|
||||
|
||||
foreign_key = self.make_key({"node": "foreign"})
|
||||
foreign = FragmentRecord.create(foreign_key, b"foreign").to_bytes()
|
||||
self.entry_path.write_bytes(foreign)
|
||||
self.assertIsNone(self.cache.get(self.key))
|
||||
|
||||
def test_symlinked_entry_and_cache_root_fail_closed_without_outside_writes(self) -> None:
|
||||
self.entry_path.parent.mkdir(parents=True)
|
||||
outside = Path(self.temporary.name) / "outside.json"
|
||||
outside.write_bytes(FragmentRecord.create(self.key, b"outside").to_bytes())
|
||||
self.entry_path.symlink_to(outside)
|
||||
self.assertIsNone(self.cache.get(self.key))
|
||||
self.assertIsNone(self.cache.put(self.key, b"replacement"))
|
||||
self.assertEqual(b"outside", FragmentRecord.from_bytes(outside.read_bytes()).content)
|
||||
|
||||
outside_cache = Path(self.temporary.name) / "outside-cache"
|
||||
escaped = ProjectionFragmentCache(self.root, outside_cache)
|
||||
self.assertIsNone(escaped.put(self.key, b"escaped"))
|
||||
self.assertFalse(outside_cache.exists())
|
||||
|
||||
def test_keys_and_records_have_deterministic_path_free_serialization(self) -> None:
|
||||
same_key = self.make_key({"summary": "Alpha", "node": "guide.alpha"})
|
||||
self.assertEqual(self.key, same_key)
|
||||
first = FragmentRecord.create(self.key, b"\x00fragment\xff")
|
||||
second = FragmentRecord.create(same_key, b"\x00fragment\xff")
|
||||
self.assertEqual(first, second)
|
||||
self.assertEqual(first.to_bytes(), second.to_bytes())
|
||||
self.assertEqual(first, FragmentRecord.from_bytes(first.to_bytes()))
|
||||
document = json.loads(first.to_bytes())
|
||||
self.assertEqual(FRAGMENT_SCHEMA_VERSION, document["schema_version"])
|
||||
self.assertEqual(FRAGMENT_RECORD_CONTRACT, document["contract"])
|
||||
self.assertEqual(FRAGMENT_KEY_CONTRACT, document["key"]["contract"])
|
||||
self.assertNotIn(str(self.root), first.to_bytes().decode("utf-8"))
|
||||
self.assertEqual(first.byte_count, len(first.content))
|
||||
self.assertEqual(64, len(first.content_sha256))
|
||||
|
||||
def test_cache_write_failure_is_a_miss_and_does_not_mutate_canonical_files(self) -> None:
|
||||
canonical = self.root / "canonical.md"
|
||||
canonical.write_text("canonical", encoding="utf-8")
|
||||
with mock.patch(
|
||||
"docforge.projection_fragments.atomic_replace_bytes_at",
|
||||
side_effect=OSError("synthetic cache failure"),
|
||||
):
|
||||
self.assertIsNone(self.cache.put(self.key, b"fragment"))
|
||||
self.assertIsNone(self.cache.get(self.key))
|
||||
self.assertEqual("canonical", canonical.read_text(encoding="utf-8"))
|
||||
|
||||
def test_record_rejects_noncanonical_serialization_and_invalid_evidence(self) -> None:
|
||||
record = FragmentRecord.create(self.key, b"alpha")
|
||||
pretty = json.dumps(record.as_dict(), sort_keys=True, indent=2).encode()
|
||||
with self.assertRaisesRegex(Exception, "canonically serialized"):
|
||||
FragmentRecord.from_bytes(pretty)
|
||||
tampered = record.as_dict()
|
||||
tampered["byte_count"] = record.byte_count + 1
|
||||
canonical = json.dumps(tampered, sort_keys=True, separators=(",", ":")).encode()
|
||||
with self.assertRaisesRegex(Exception, "byte evidence"):
|
||||
FragmentRecord.from_bytes(canonical)
|
||||
|
||||
def test_prune_retains_only_the_exact_bounded_active_inventory(self) -> None:
|
||||
retained = self.key
|
||||
stale = self.make_key({"node": "stale"})
|
||||
self.assertIsNotNone(self.cache.put(retained, b"retained"))
|
||||
self.assertIsNotNone(self.cache.put(stale, b"stale"))
|
||||
foreign = self.entry_path.parent / "foreign.tmp"
|
||||
foreign.write_bytes(b"foreign")
|
||||
|
||||
self.assertTrue(self.cache.prune((retained,)))
|
||||
retained_record = self.cache.get(retained)
|
||||
self.assertIsNotNone(retained_record)
|
||||
assert retained_record is not None
|
||||
self.assertEqual(b"retained", retained_record.content)
|
||||
self.assertIsNone(self.cache.get(stale))
|
||||
self.assertFalse(foreign.exists())
|
||||
|
||||
with mock.patch(
|
||||
"docforge.projection_fragments.MAX_FRAGMENT_CACHE_BYTES",
|
||||
1,
|
||||
):
|
||||
self.assertFalse(self.cache.prune((retained,)))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
266
tests/test_projection_policy.py
Normal file
266
tests/test_projection_policy.py
Normal file
|
|
@ -0,0 +1,266 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
import unittest
|
||||
from dataclasses import FrozenInstanceError
|
||||
from pathlib import Path
|
||||
|
||||
from jsonschema import Draft202012Validator
|
||||
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.projection_policy import (
|
||||
ProjectionPolicyV2,
|
||||
compose_projection_policy,
|
||||
)
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
POLICY_SCHEMA = json.loads(
|
||||
(ROOT / "schemas" / "projection-policy.schema.json").read_text(encoding="utf-8")
|
||||
)
|
||||
|
||||
|
||||
class ProjectionPolicyV2Tests(unittest.TestCase):
|
||||
def test_compatible_defaults_are_independent_and_schema_valid(self) -> None:
|
||||
cases = (
|
||||
(
|
||||
{
|
||||
"manual_configured": False,
|
||||
"portable_graph_configured": False,
|
||||
"application_enabled": False,
|
||||
},
|
||||
("disabled", "disabled", "on-demand"),
|
||||
),
|
||||
(
|
||||
{
|
||||
"manual_configured": True,
|
||||
"portable_graph_configured": False,
|
||||
"application_enabled": False,
|
||||
},
|
||||
("explicit", "disabled", "on-demand"),
|
||||
),
|
||||
(
|
||||
{
|
||||
"manual_configured": True,
|
||||
"portable_graph_configured": True,
|
||||
"application_enabled": True,
|
||||
},
|
||||
("auto", "explicit", "on-demand"),
|
||||
),
|
||||
(
|
||||
{
|
||||
"manual_configured": True,
|
||||
"portable_graph_configured": True,
|
||||
"application_enabled": False,
|
||||
"live_viewer_available": False,
|
||||
},
|
||||
("explicit", "explicit", "disabled"),
|
||||
),
|
||||
)
|
||||
validator = Draft202012Validator(POLICY_SCHEMA)
|
||||
for arguments, expected in cases:
|
||||
with self.subTest(arguments=arguments):
|
||||
policy = compose_projection_policy(**arguments)
|
||||
self.assertEqual(
|
||||
expected, (policy.manual, policy.portable_graph, policy.live_viewer)
|
||||
)
|
||||
validator.validate(policy.as_dict())
|
||||
|
||||
def test_explicit_selections_can_narrow_each_consumer_independently(self) -> None:
|
||||
policy = compose_projection_policy(
|
||||
manual="explicit",
|
||||
portable_graph="disabled",
|
||||
live_viewer="disabled",
|
||||
manual_configured=True,
|
||||
portable_graph_configured=True,
|
||||
application_enabled=True,
|
||||
)
|
||||
self.assertEqual(
|
||||
{
|
||||
"schema_version": 2,
|
||||
"manual": "explicit",
|
||||
"portable_graph": "disabled",
|
||||
"live_viewer": "disabled",
|
||||
},
|
||||
policy.as_dict(),
|
||||
)
|
||||
|
||||
fully_disabled = compose_projection_policy(
|
||||
manual="disabled",
|
||||
portable_graph="disabled",
|
||||
live_viewer="disabled",
|
||||
manual_configured=False,
|
||||
portable_graph_configured=False,
|
||||
application_enabled=False,
|
||||
live_viewer_available=False,
|
||||
)
|
||||
self.assertEqual("disabled", fully_disabled.manual)
|
||||
self.assertEqual("disabled", fully_disabled.portable_graph)
|
||||
self.assertEqual("disabled", fully_disabled.live_viewer)
|
||||
|
||||
def test_policy_is_frozen_and_hashes_exact_canonical_payload(self) -> None:
|
||||
first = compose_projection_policy(
|
||||
manual_configured=True,
|
||||
portable_graph_configured=True,
|
||||
application_enabled=False,
|
||||
)
|
||||
second = compose_projection_policy(
|
||||
manual_configured=True,
|
||||
portable_graph_configured=True,
|
||||
application_enabled=False,
|
||||
)
|
||||
canonical = json.dumps(
|
||||
first.as_dict(),
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
).encode("utf-8")
|
||||
self.assertEqual(first.as_dict(), second.as_dict())
|
||||
self.assertEqual(hashlib.sha256(canonical).hexdigest(), first.policy_hash)
|
||||
self.assertEqual(first.policy_hash, second.policy_hash)
|
||||
|
||||
changed = compose_projection_policy(
|
||||
manual="disabled",
|
||||
manual_configured=True,
|
||||
portable_graph_configured=True,
|
||||
application_enabled=False,
|
||||
)
|
||||
self.assertNotEqual(first.policy_hash, changed.policy_hash)
|
||||
with self.assertRaises(FrozenInstanceError):
|
||||
first.manual = "disabled" # type: ignore[misc]
|
||||
|
||||
def test_invalid_modes_and_direct_construction_fail_closed(self) -> None:
|
||||
cases = (
|
||||
("manual", {"manual": "sometimes"}),
|
||||
("portable_graph", {"portable_graph": "auto"}),
|
||||
("live_viewer", {"live_viewer": "always"}),
|
||||
("manual", {"manual": 1}),
|
||||
)
|
||||
for expected_projection, selection in cases:
|
||||
with self.subTest(selection=selection):
|
||||
with self.assertRaises(DocForgeError) as raised:
|
||||
compose_projection_policy(
|
||||
**selection,
|
||||
manual_configured=True,
|
||||
portable_graph_configured=True,
|
||||
application_enabled=True,
|
||||
)
|
||||
self.assertEqual("invalid_projection_policy", raised.exception.code)
|
||||
self.assertEqual(
|
||||
expected_projection,
|
||||
raised.exception.details["projection"],
|
||||
)
|
||||
|
||||
with self.assertRaises(DocForgeError) as direct:
|
||||
ProjectionPolicyV2(
|
||||
manual="automatic", # type: ignore[arg-type]
|
||||
portable_graph="explicit",
|
||||
live_viewer="on-demand",
|
||||
)
|
||||
self.assertEqual("invalid_projection_policy", direct.exception.code)
|
||||
|
||||
def test_resource_and_capability_unavailability_are_distinct(self) -> None:
|
||||
cases = (
|
||||
(
|
||||
{"manual": "explicit"},
|
||||
{
|
||||
"manual_configured": False,
|
||||
"portable_graph_configured": False,
|
||||
"application_enabled": False,
|
||||
},
|
||||
"manual",
|
||||
"manual_render_config",
|
||||
),
|
||||
(
|
||||
{"manual": "auto"},
|
||||
{
|
||||
"manual_configured": True,
|
||||
"portable_graph_configured": False,
|
||||
"application_enabled": False,
|
||||
},
|
||||
"manual",
|
||||
"canonical_application",
|
||||
),
|
||||
(
|
||||
{"portable_graph": "explicit"},
|
||||
{
|
||||
"manual_configured": False,
|
||||
"portable_graph_configured": False,
|
||||
"application_enabled": False,
|
||||
},
|
||||
"portable_graph",
|
||||
"portable_graph_render_config",
|
||||
),
|
||||
(
|
||||
{"live_viewer": "on-demand"},
|
||||
{
|
||||
"manual_configured": False,
|
||||
"portable_graph_configured": False,
|
||||
"application_enabled": False,
|
||||
"live_viewer_available": False,
|
||||
},
|
||||
"live_viewer",
|
||||
"live_viewer_runtime",
|
||||
),
|
||||
)
|
||||
for selection, availability, projection, required in cases:
|
||||
with self.subTest(selection=selection):
|
||||
with self.assertRaises(DocForgeError) as raised:
|
||||
compose_projection_policy(**selection, **availability)
|
||||
self.assertEqual("projection_policy_unavailable", raised.exception.code)
|
||||
self.assertEqual(projection, raised.exception.details["projection"])
|
||||
self.assertEqual(required, raised.exception.details["required"])
|
||||
|
||||
def test_availability_inputs_must_be_real_booleans(self) -> None:
|
||||
cases = (
|
||||
{"manual_configured": 1},
|
||||
{"portable_graph_configured": 0},
|
||||
{"application_enabled": "yes"},
|
||||
{"live_viewer_available": None},
|
||||
)
|
||||
for replacement in cases:
|
||||
arguments = {
|
||||
"manual_configured": False,
|
||||
"portable_graph_configured": False,
|
||||
"application_enabled": False,
|
||||
"live_viewer_available": True,
|
||||
**replacement,
|
||||
}
|
||||
with self.subTest(replacement=replacement):
|
||||
with self.assertRaises(DocForgeError) as raised:
|
||||
compose_projection_policy(**arguments)
|
||||
self.assertEqual("invalid_projection_policy", raised.exception.code)
|
||||
|
||||
def test_schema_rejects_every_runtime_contract_drift(self) -> None:
|
||||
validator = Draft202012Validator(POLICY_SCHEMA)
|
||||
valid = compose_projection_policy(
|
||||
manual_configured=True,
|
||||
portable_graph_configured=True,
|
||||
application_enabled=True,
|
||||
).as_dict()
|
||||
invalid_documents: list[dict[str, object]] = []
|
||||
for field in ("schema_version", "manual", "portable_graph", "live_viewer"):
|
||||
missing = copy.deepcopy(valid)
|
||||
missing.pop(field)
|
||||
invalid_documents.append(missing)
|
||||
for field, value in (
|
||||
("schema_version", 1),
|
||||
("manual", "on-demand"),
|
||||
("portable_graph", "auto"),
|
||||
("live_viewer", "explicit"),
|
||||
):
|
||||
changed = copy.deepcopy(valid)
|
||||
changed[field] = value
|
||||
invalid_documents.append(changed)
|
||||
extra = copy.deepcopy(valid)
|
||||
extra["project_path"] = "/private/project"
|
||||
invalid_documents.append(extra)
|
||||
|
||||
for document in invalid_documents:
|
||||
with self.subTest(document=document):
|
||||
self.assertFalse(validator.is_valid(document))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
239
tests/test_projection_policy_integration.py
Normal file
239
tests/test_projection_policy_integration.py
Normal file
|
|
@ -0,0 +1,239 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from docforge.application import CanonicalApplicationService
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.graph_rendering import GraphRenderService
|
||||
from docforge.index import ProjectIndex
|
||||
from docforge.mcp_server import DocForgeService
|
||||
from docforge.project import Project
|
||||
from docforge.rendering import RenderService
|
||||
from docforge.viewer_manager import ViewerManagerClient
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURES = ROOT / "tests" / "fixtures"
|
||||
|
||||
GRAPH_CONFIG = """
|
||||
|
||||
[graph_render]
|
||||
output_root = ".docforge/portable-graph"
|
||||
|
||||
[[graph_render.views]]
|
||||
id = "architecture"
|
||||
renderer = "portable_graph_html"
|
||||
output = "architecture.html"
|
||||
title = "Alpha architecture"
|
||||
root = "guide.workflow"
|
||||
initial_mode = "nodes"
|
||||
depth = 2
|
||||
max_nodes = 20
|
||||
max_edges = 40
|
||||
max_work = 1000
|
||||
include_logic = false
|
||||
"""
|
||||
|
||||
|
||||
class ProjectionPolicyIntegrationTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temporary.cleanup)
|
||||
self.root = Path(self.temporary.name) / "alpha"
|
||||
shutil.copytree(FIXTURES / "alpha", self.root)
|
||||
|
||||
def project(self, *, graph: bool = False) -> Project:
|
||||
if graph:
|
||||
descriptor = self.root / ".docforge/project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
|
||||
encoding="utf-8",
|
||||
)
|
||||
return Project.open(self.root)
|
||||
|
||||
def test_disabled_manual_blocks_work_before_load_but_status_remains_receipt_only(self) -> None:
|
||||
project = self.project()
|
||||
service = RenderService(project, manual_policy="disabled")
|
||||
with mock.patch.object(
|
||||
project,
|
||||
"load",
|
||||
side_effect=AssertionError("disabled manual policy must fail before loading"),
|
||||
):
|
||||
status = service.status("manual")
|
||||
self.assertEqual("stale", status["state"])
|
||||
for operation in (
|
||||
lambda: service.render("manual"),
|
||||
lambda: service.deep_status("manual"),
|
||||
lambda: service.preview("proposal", "manual"),
|
||||
):
|
||||
with self.subTest(operation=operation), self.assertRaises(DocForgeError) as raised:
|
||||
operation()
|
||||
self.assertEqual(
|
||||
"projection_policy_forbids_operation",
|
||||
raised.exception.code,
|
||||
)
|
||||
|
||||
def test_disabled_graph_blocks_plan_and_render_but_status_remains_available(self) -> None:
|
||||
project = self.project(graph=True)
|
||||
service = GraphRenderService(project, portable_graph_policy="disabled")
|
||||
with mock.patch.object(
|
||||
project,
|
||||
"load",
|
||||
side_effect=AssertionError("disabled graph policy must fail before loading"),
|
||||
):
|
||||
status = service.status("architecture")
|
||||
self.assertEqual("stale", status["state"])
|
||||
for operation in (
|
||||
lambda: service.plan("architecture"),
|
||||
lambda: service.render("architecture"),
|
||||
):
|
||||
with self.subTest(operation=operation), self.assertRaises(DocForgeError) as raised:
|
||||
operation()
|
||||
self.assertEqual(
|
||||
"projection_policy_forbids_operation",
|
||||
raised.exception.code,
|
||||
)
|
||||
|
||||
def test_live_viewer_disabled_blocks_start_before_index_work_but_allows_lifecycle(self) -> None:
|
||||
project = self.project()
|
||||
index = ProjectIndex(project)
|
||||
client = ViewerManagerClient(index, live_viewer_policy="disabled")
|
||||
with (
|
||||
mock.patch.object(
|
||||
index,
|
||||
"check",
|
||||
side_effect=AssertionError("disabled viewer must fail before index work"),
|
||||
),
|
||||
self.assertRaises(DocForgeError) as raised,
|
||||
):
|
||||
client.start(node_id="guide.workflow")
|
||||
self.assertEqual("projection_policy_forbids_operation", raised.exception.code)
|
||||
|
||||
with mock.patch.object(
|
||||
client,
|
||||
"_lifecycle_request",
|
||||
side_effect=(
|
||||
{"status": "ok", "state": "not_running"},
|
||||
{"status": "ok", "state": "stopped"},
|
||||
),
|
||||
):
|
||||
self.assertEqual("not_running", client.status()["state"])
|
||||
self.assertEqual("stopped", client.stop()["state"])
|
||||
|
||||
def test_application_auto_renders_and_explicit_or_disabled_skip_cleanly(self) -> None:
|
||||
project = self.project()
|
||||
for mode, expected_action, expected_calls in (
|
||||
("auto", "rendered", 1),
|
||||
("explicit", "skipped_explicit", 0),
|
||||
("disabled", "skipped_disabled", 0),
|
||||
):
|
||||
with self.subTest(mode=mode):
|
||||
service = CanonicalApplicationService(
|
||||
project,
|
||||
applier_id="alpha-editor",
|
||||
applier=mock.Mock(),
|
||||
manual_policy=mode, # type: ignore[arg-type]
|
||||
)
|
||||
with (
|
||||
mock.patch.object(
|
||||
service.changesets,
|
||||
"apply",
|
||||
return_value={"status": "ok", "applied": True},
|
||||
),
|
||||
mock.patch.object(
|
||||
service.index,
|
||||
"build",
|
||||
return_value={"status": "ok"},
|
||||
),
|
||||
mock.patch.object(
|
||||
service.index,
|
||||
"check",
|
||||
return_value={"status": "ok"},
|
||||
),
|
||||
mock.patch.object(
|
||||
service.rendering,
|
||||
"render",
|
||||
return_value={"status": "ok", "state": "current"},
|
||||
) as rendered,
|
||||
):
|
||||
result = service.apply("policy-application", "a" * 64)
|
||||
refresh = result["derived_refresh"]
|
||||
assert isinstance(refresh, dict)
|
||||
self.assertEqual(
|
||||
{"mode": mode, "action": expected_action},
|
||||
refresh["render_policy"],
|
||||
)
|
||||
self.assertEqual(expected_calls, rendered.call_count)
|
||||
self.assertEqual("ok", refresh["status"])
|
||||
|
||||
def test_mcp_exposes_v2_without_changing_v1_policy_or_legacy_render_projection(self) -> None:
|
||||
project = self.project(graph=True)
|
||||
baseline = DocForgeService(project, capability_mode_name="read")
|
||||
narrowed = DocForgeService(
|
||||
project,
|
||||
capability_mode_name="read",
|
||||
manual_projection_policy="disabled",
|
||||
portable_graph_policy="disabled",
|
||||
live_viewer_policy="disabled",
|
||||
)
|
||||
self.assertEqual(baseline.policy.as_dict(), narrowed.policy.as_dict())
|
||||
result = narrowed.bootstrap()
|
||||
self.assertEqual(narrowed.policy.as_dict(), result["effective_policy"])
|
||||
self.assertEqual(
|
||||
narrowed.projection_policy.as_dict(),
|
||||
result["projection_policy"],
|
||||
)
|
||||
self.assertEqual(
|
||||
narrowed.projection_policy.policy_hash,
|
||||
result["projection_policy_hash"],
|
||||
)
|
||||
session = result["session_contract"]
|
||||
assert isinstance(session, dict)
|
||||
self.assertEqual(
|
||||
{
|
||||
"manual": narrowed.policy.manual_render,
|
||||
"graph": narrowed.policy.graph_render,
|
||||
"live_viewer": narrowed.policy.live_viewer,
|
||||
},
|
||||
session["render_policies"],
|
||||
)
|
||||
self.assertEqual(
|
||||
"stale",
|
||||
narrowed.graph_render_status("architecture")["state"],
|
||||
)
|
||||
blocked_plan = narrowed.graph_plan("architecture")
|
||||
self.assertEqual(
|
||||
"projection_policy_forbids_operation",
|
||||
blocked_plan["error"]["code"],
|
||||
)
|
||||
|
||||
def test_direct_service_policy_values_are_runtime_validated(self) -> None:
|
||||
project = self.project(graph=True)
|
||||
factories = (
|
||||
lambda: RenderService(project, manual_policy="bogus"), # type: ignore[arg-type]
|
||||
lambda: GraphRenderService(
|
||||
project,
|
||||
portable_graph_policy="bogus", # type: ignore[arg-type]
|
||||
),
|
||||
lambda: ViewerManagerClient(
|
||||
ProjectIndex(project),
|
||||
live_viewer_policy="bogus", # type: ignore[arg-type]
|
||||
),
|
||||
lambda: CanonicalApplicationService(
|
||||
project,
|
||||
applier_id=None,
|
||||
applier=None,
|
||||
manual_policy="bogus", # type: ignore[arg-type]
|
||||
),
|
||||
)
|
||||
for factory in factories:
|
||||
with self.subTest(factory=factory), self.assertRaises(DocForgeError) as raised:
|
||||
factory()
|
||||
self.assertEqual("invalid_projection_policy", raised.exception.code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -8,12 +8,14 @@ from pathlib import Path
|
|||
from jsonschema import Draft202012Validator
|
||||
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.graph_projection import build_graph_projection_package
|
||||
from docforge.projection_contract import (
|
||||
GraphViewPlanV1,
|
||||
ManualRenderPlanV1,
|
||||
ProjectionPackageV1,
|
||||
ProjectionReceiptV1,
|
||||
)
|
||||
from docforge_renderers.graph import PortableGraphHtmlRenderer
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SCHEMAS = ROOT / "schemas"
|
||||
|
|
@ -220,6 +222,14 @@ class ProjectionSchemaTests(unittest.TestCase):
|
|||
|
||||
graph_package = self.package(self.graph_plan()).as_dict()
|
||||
self.validator("projection-package.schema.json").validate(graph_package)
|
||||
typed_graph_package = build_graph_projection_package(
|
||||
self.graph_plan(),
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=1_000_000,
|
||||
)
|
||||
graph_receipt = PortableGraphHtmlRenderer().render(typed_graph_package).receipt.as_dict()
|
||||
self.validator("projection-receipt.schema.json").validate(graph_receipt)
|
||||
|
||||
def test_unknown_fields_are_rejected_at_contract_boundaries(self) -> None:
|
||||
cases = (
|
||||
|
|
|
|||
388
tests/test_projection_worker.py
Normal file
388
tests/test_projection_worker.py
Normal file
|
|
@ -0,0 +1,388 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import contextlib
|
||||
import copy
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.graph_projection import (
|
||||
GraphViewRequestV1,
|
||||
build_graph_projection_package,
|
||||
build_graph_view_plan,
|
||||
)
|
||||
from docforge.manual_projection import (
|
||||
build_manual_projection_package,
|
||||
build_manual_render_plan,
|
||||
)
|
||||
from docforge.project import Project
|
||||
from docforge.projection_contract import (
|
||||
ProjectionPackageV1,
|
||||
ProjectionReceiptV1,
|
||||
canonical_projection_bytes,
|
||||
)
|
||||
from docforge.projection_worker import (
|
||||
MAX_WORKER_ARTIFACT_BYTES,
|
||||
MAX_WORKER_RESPONSE_BYTES,
|
||||
_child_response,
|
||||
render_projection_in_worker,
|
||||
)
|
||||
from docforge.render_contract import GenericHtmlRenderer
|
||||
from docforge_renderers.graph import PortableGraphHtmlRenderer
|
||||
from docforge_renderers.manual import ManualHtmlRenderer
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURES = ROOT / "tests" / "fixtures"
|
||||
|
||||
|
||||
class ProjectionWorkerTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temporary.cleanup)
|
||||
self.root = Path(self.temporary.name) / "alpha"
|
||||
shutil.copytree(FIXTURES / "alpha", self.root)
|
||||
self.project = Project.open(self.root)
|
||||
self.snapshot = self.project.load()
|
||||
render = self.snapshot.descriptor.render
|
||||
assert render is not None
|
||||
self.manual_view = render.views[0]
|
||||
self.manual_version = GenericHtmlRenderer().renderer_version
|
||||
self.manual_plan = build_manual_render_plan(
|
||||
self.snapshot,
|
||||
self.manual_view,
|
||||
changeset_hash=None,
|
||||
)
|
||||
self.manual_package = build_manual_projection_package(
|
||||
self.manual_plan,
|
||||
self.manual_view.template_path.read_bytes(),
|
||||
renderer_id="generic_html",
|
||||
renderer_version=self.manual_version,
|
||||
max_output_bytes=self.snapshot.descriptor.limits.max_render_bytes,
|
||||
)
|
||||
self.graph_plan = build_graph_view_plan(
|
||||
self.snapshot,
|
||||
GraphViewRequestV1(
|
||||
view_id="architecture",
|
||||
title="Alpha architecture",
|
||||
root_node_id="guide.workflow",
|
||||
depth=2,
|
||||
max_nodes=20,
|
||||
max_edges=40,
|
||||
max_work=1_000,
|
||||
),
|
||||
False,
|
||||
)
|
||||
self.graph_package = build_graph_projection_package(
|
||||
self.graph_plan,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=self.snapshot.descriptor.limits.max_render_bytes,
|
||||
)
|
||||
|
||||
def test_manual_and_graph_workers_match_the_in_process_renderers(self) -> None:
|
||||
cases = (
|
||||
(
|
||||
self.manual_package,
|
||||
ManualHtmlRenderer(self.manual_version).render(self.manual_package),
|
||||
),
|
||||
(
|
||||
self.graph_package,
|
||||
PortableGraphHtmlRenderer().render(self.graph_package),
|
||||
),
|
||||
)
|
||||
for package, expected in cases:
|
||||
with self.subTest(kind=package.kind):
|
||||
result = render_projection_in_worker(package)
|
||||
self.assertEqual(expected.artifacts, result.artifacts)
|
||||
receipt = result.receipt.as_dict()
|
||||
self.assertEqual(package.package_id, receipt["package_id"])
|
||||
self.assertEqual(package.document["plan_id"], receipt["plan_id"])
|
||||
self.assertIs(type(receipt["peak_memory_bytes"]), int)
|
||||
self.assertGreater(receipt["peak_memory_bytes"], 0)
|
||||
|
||||
def test_parent_launch_is_fixed_and_request_contains_no_runtime_authority(self) -> None:
|
||||
original = subprocess.run
|
||||
with mock.patch(
|
||||
"docforge.projection_worker.subprocess.run",
|
||||
wraps=original,
|
||||
) as launched:
|
||||
render_projection_in_worker(self.manual_package)
|
||||
command = launched.call_args.args[0]
|
||||
options = launched.call_args.kwargs
|
||||
self.assertEqual(
|
||||
[sys.executable, "-I", "-m", "docforge._projection_worker_main"],
|
||||
command,
|
||||
)
|
||||
self.assertIs(options["shell"], False)
|
||||
self.assertEqual(sys.prefix, options["cwd"])
|
||||
self.assertEqual(
|
||||
{"PYTHONIOENCODING": "utf-8", "PYTHONUTF8": "1"},
|
||||
options["env"],
|
||||
)
|
||||
request = options["input"]
|
||||
self.assertEqual(
|
||||
canonical_projection_bytes(self.manual_package.as_dict()) + b"\n",
|
||||
request,
|
||||
)
|
||||
for forbidden in (
|
||||
str(self.root).encode(),
|
||||
b"project_path",
|
||||
b"index_path",
|
||||
b"database_path",
|
||||
b'"command"',
|
||||
b'"module"',
|
||||
b'"shell"',
|
||||
b'"sql"',
|
||||
):
|
||||
self.assertNotIn(forbidden, request)
|
||||
|
||||
def test_worker_ignores_hostile_cwd_and_pythonpath_import_shadows(self) -> None:
|
||||
shadow_root = Path(self.temporary.name) / "shadow"
|
||||
shadow_package = shadow_root / "docforge"
|
||||
shadow_package.mkdir(parents=True)
|
||||
sentinel = shadow_root / "executed"
|
||||
shadow_package.joinpath("__init__.py").write_text(
|
||||
"from pathlib import Path\n"
|
||||
f"Path({str(sentinel)!r}).write_text('executed', encoding='utf-8')\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
with (
|
||||
contextlib.chdir(shadow_root),
|
||||
mock.patch.dict(
|
||||
os.environ,
|
||||
{"PYTHONPATH": str(shadow_root)},
|
||||
clear=False,
|
||||
),
|
||||
):
|
||||
result = render_projection_in_worker(self.manual_package)
|
||||
self.assertEqual("manual", result.receipt.document["kind"])
|
||||
self.assertFalse(sentinel.exists())
|
||||
|
||||
def test_parent_accepts_only_valid_packages_and_closed_renderer_versions(self) -> None:
|
||||
with self.assertRaises(TypeError):
|
||||
render_projection_in_worker({}) # type: ignore[arg-type]
|
||||
|
||||
unsupported = ProjectionPackageV1.create(
|
||||
kind="manual",
|
||||
plan=self.manual_plan,
|
||||
renderer={"renderer_id": "generic_html", "renderer_version": "other"},
|
||||
components=[{"component_id": "manual.document@1"}],
|
||||
assets=copy.deepcopy(self.manual_package.document["assets"]),
|
||||
output_policy=copy.deepcopy(self.manual_package.document["output_policy"]),
|
||||
)
|
||||
with (
|
||||
mock.patch("docforge.projection_worker._invoke_worker") as invoke,
|
||||
self.assertRaises(DocForgeError) as raised,
|
||||
):
|
||||
render_projection_in_worker(unsupported)
|
||||
self.assertEqual("unsupported_renderer", raised.exception.code)
|
||||
invoke.assert_not_called()
|
||||
|
||||
permissive_allowance = ProjectionPackageV1.create(
|
||||
kind="manual",
|
||||
plan=self.manual_plan,
|
||||
renderer={
|
||||
"renderer_id": "generic_html",
|
||||
"renderer_version": self.manual_version,
|
||||
},
|
||||
components=[{"component_id": "manual.document@1"}],
|
||||
assets=copy.deepcopy(self.manual_package.document["assets"]),
|
||||
output_policy={
|
||||
"artifact_ids": ["manual.html"],
|
||||
"max_total_bytes": MAX_WORKER_ARTIFACT_BYTES + 1,
|
||||
},
|
||||
)
|
||||
rendered = render_projection_in_worker(permissive_allowance)
|
||||
self.assertEqual(
|
||||
ManualHtmlRenderer(self.manual_version)
|
||||
.render(permissive_allowance)
|
||||
.artifacts[0]
|
||||
.content,
|
||||
rendered.artifacts[0].content,
|
||||
)
|
||||
|
||||
def test_process_timeout_exit_and_signal_fail_closed(self) -> None:
|
||||
failures = (
|
||||
(
|
||||
subprocess.TimeoutExpired(["worker"], 30),
|
||||
"projection_worker_timeout",
|
||||
),
|
||||
(
|
||||
subprocess.CompletedProcess(["worker"], 2, stdout=b""),
|
||||
"projection_worker_failure",
|
||||
),
|
||||
(
|
||||
subprocess.CompletedProcess(["worker"], -9, stdout=b""),
|
||||
"projection_worker_failure",
|
||||
),
|
||||
)
|
||||
for outcome, code in failures:
|
||||
with self.subTest(outcome=type(outcome).__name__):
|
||||
patch = (
|
||||
mock.patch(
|
||||
"docforge.projection_worker._invoke_worker",
|
||||
side_effect=outcome,
|
||||
)
|
||||
if isinstance(outcome, BaseException)
|
||||
else mock.patch(
|
||||
"docforge.projection_worker._invoke_worker",
|
||||
return_value=outcome,
|
||||
)
|
||||
)
|
||||
with patch, self.assertRaises(DocForgeError) as raised:
|
||||
render_projection_in_worker(self.manual_package)
|
||||
self.assertEqual(code, raised.exception.code)
|
||||
|
||||
def test_malformed_trailing_and_oversized_responses_fail_closed(self) -> None:
|
||||
valid = _child_response(self.manual_package)
|
||||
malformed = (
|
||||
b"",
|
||||
b"{}",
|
||||
b"not-json\n",
|
||||
b'{ "schema_version": 1 }\n',
|
||||
valid + b"{}\n",
|
||||
b"x" * (MAX_WORKER_RESPONSE_BYTES + 1),
|
||||
)
|
||||
for response in malformed:
|
||||
with (
|
||||
self.subTest(length=len(response)),
|
||||
mock.patch(
|
||||
"docforge.projection_worker._invoke_worker",
|
||||
return_value=subprocess.CompletedProcess(
|
||||
["worker"],
|
||||
0,
|
||||
stdout=response,
|
||||
),
|
||||
),
|
||||
self.assertRaises(DocForgeError) as raised,
|
||||
):
|
||||
render_projection_in_worker(self.manual_package)
|
||||
self.assertEqual("projection_worker_failure", raised.exception.code)
|
||||
|
||||
def test_wrong_artifact_and_receipt_evidence_fail_closed(self) -> None:
|
||||
valid = json.loads(_child_response(self.manual_package))
|
||||
cases: dict[str, dict[str, object]] = {}
|
||||
|
||||
wrong_content = copy.deepcopy(valid)
|
||||
wrong_content["artifacts"][0]["content_base64"] = base64.b64encode(b"changed").decode()
|
||||
cases["content_hash"] = wrong_content
|
||||
|
||||
wrong_id = copy.deepcopy(valid)
|
||||
wrong_id["artifacts"][0]["artifact_id"] = "other.html"
|
||||
cases["artifact_id"] = wrong_id
|
||||
|
||||
missing_artifact = copy.deepcopy(valid)
|
||||
missing_artifact["artifacts"] = []
|
||||
cases["artifact_count"] = missing_artifact
|
||||
|
||||
wrong_media_type = copy.deepcopy(valid)
|
||||
wrong_media_type["artifacts"][0]["media_type"] = "application/octet-stream"
|
||||
cases["media_type"] = wrong_media_type
|
||||
|
||||
bad_base64 = copy.deepcopy(valid)
|
||||
bad_base64["artifacts"][0]["content_base64"] = "***"
|
||||
cases["base64"] = bad_base64
|
||||
|
||||
wrong_package = copy.deepcopy(valid)
|
||||
receipt = wrong_package["receipt"]
|
||||
replacement = ProjectionReceiptV1.create(
|
||||
kind="manual",
|
||||
package_id="0" * 64,
|
||||
plan_id=receipt["plan_id"],
|
||||
renderer=receipt["renderer"],
|
||||
artifacts=receipt["artifacts"],
|
||||
diagnostics=receipt["diagnostics"],
|
||||
timing=receipt["timing"],
|
||||
peak_memory_bytes=receipt["peak_memory_bytes"],
|
||||
)
|
||||
wrong_package["receipt"] = replacement.as_dict()
|
||||
cases["package_id"] = wrong_package
|
||||
|
||||
zero_peak = copy.deepcopy(valid)
|
||||
receipt = zero_peak["receipt"]
|
||||
replacement = ProjectionReceiptV1.create(
|
||||
kind="manual",
|
||||
package_id=receipt["package_id"],
|
||||
plan_id=receipt["plan_id"],
|
||||
renderer=receipt["renderer"],
|
||||
artifacts=receipt["artifacts"],
|
||||
diagnostics=receipt["diagnostics"],
|
||||
timing=receipt["timing"],
|
||||
peak_memory_bytes=0,
|
||||
)
|
||||
zero_peak["receipt"] = replacement.as_dict()
|
||||
cases["peak_memory"] = zero_peak
|
||||
|
||||
wrong_size = copy.deepcopy(valid)
|
||||
receipt = wrong_size["receipt"]
|
||||
artifacts = copy.deepcopy(receipt["artifacts"])
|
||||
artifacts[0]["bytes"] += 1
|
||||
replacement = ProjectionReceiptV1.create(
|
||||
kind="manual",
|
||||
package_id=receipt["package_id"],
|
||||
plan_id=receipt["plan_id"],
|
||||
renderer=receipt["renderer"],
|
||||
artifacts=artifacts,
|
||||
diagnostics=receipt["diagnostics"],
|
||||
timing=receipt["timing"],
|
||||
peak_memory_bytes=receipt["peak_memory_bytes"],
|
||||
)
|
||||
wrong_size["receipt"] = replacement.as_dict()
|
||||
cases["artifact_size"] = wrong_size
|
||||
|
||||
configured_oversize = copy.deepcopy(valid)
|
||||
maximum = self.manual_package.document["output_policy"]["max_total_bytes"]
|
||||
configured_oversize["artifacts"][0]["content_base64"] = base64.b64encode(
|
||||
b"x" * (maximum + 1)
|
||||
).decode()
|
||||
cases["configured_aggregate"] = configured_oversize
|
||||
|
||||
for name, document in cases.items():
|
||||
response = canonical_projection_bytes(document) + b"\n"
|
||||
with (
|
||||
self.subTest(name=name),
|
||||
mock.patch(
|
||||
"docforge.projection_worker._invoke_worker",
|
||||
return_value=subprocess.CompletedProcess(
|
||||
["worker"],
|
||||
0,
|
||||
stdout=response,
|
||||
),
|
||||
),
|
||||
self.assertRaises(DocForgeError) as raised,
|
||||
):
|
||||
render_projection_in_worker(self.manual_package)
|
||||
self.assertEqual("projection_worker_failure", raised.exception.code)
|
||||
|
||||
def test_child_rejects_noncanonical_invalid_and_trailing_requests(self) -> None:
|
||||
valid = canonical_projection_bytes(self.manual_package.as_dict()) + b"\n"
|
||||
requests = (
|
||||
b"{}\n",
|
||||
b'{ "schema_version": 1 }\n',
|
||||
valid + b"{}\n",
|
||||
)
|
||||
for request in requests:
|
||||
with self.subTest(length=len(request)):
|
||||
completed = subprocess.run(
|
||||
[sys.executable, "-I", "-m", "docforge._projection_worker_main"],
|
||||
cwd=ROOT,
|
||||
input=request,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
timeout=10,
|
||||
)
|
||||
self.assertEqual(2, completed.returncode)
|
||||
self.assertEqual(b"", completed.stdout)
|
||||
self.assertEqual(b"", completed.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -66,8 +66,10 @@ PUBLIC_IMPORTS = {
|
|||
"docforge.index": ("ProjectIndex",),
|
||||
"docforge.graph_projection": (
|
||||
"GraphViewRequestV1",
|
||||
"build_graph_projection_package",
|
||||
"build_graph_view_plan",
|
||||
),
|
||||
"docforge.graph_rendering": ("GraphRenderService",),
|
||||
"docforge.manual_projection": (
|
||||
"build_manual_projection_package",
|
||||
"build_manual_render_plan",
|
||||
|
|
@ -92,6 +94,19 @@ PUBLIC_IMPORTS = {
|
|||
"capability_mode",
|
||||
"compose_effective_policy",
|
||||
),
|
||||
"docforge.projection_policy": (
|
||||
"ProjectionPolicyV2",
|
||||
"compose_projection_policy",
|
||||
"validate_live_viewer_projection_mode",
|
||||
"validate_manual_projection_mode",
|
||||
"validate_portable_graph_projection_mode",
|
||||
),
|
||||
"docforge.projection_fragments": (
|
||||
"FragmentKey",
|
||||
"FragmentRecord",
|
||||
"ProjectionFragmentCache",
|
||||
"fragment_semantic_hash",
|
||||
),
|
||||
"docforge.projection_contract": (
|
||||
"GraphViewPlanV1",
|
||||
"ManualRenderPlanV1",
|
||||
|
|
@ -102,6 +117,7 @@ PUBLIC_IMPORTS = {
|
|||
"canonical_projection_bytes",
|
||||
"projection_hash",
|
||||
),
|
||||
"docforge.projection_worker": ("render_projection_in_worker",),
|
||||
"docforge.retrieval": (
|
||||
"ContextCapsuleV1",
|
||||
"RetrievalPlanV1",
|
||||
|
|
@ -116,6 +132,7 @@ PUBLIC_IMPORTS = {
|
|||
"renderer_for",
|
||||
),
|
||||
"docforge_renderers.manual": ("ManualHtmlRenderer",),
|
||||
"docforge_renderers.graph": ("PortableGraphHtmlRenderer",),
|
||||
}
|
||||
|
||||
EXPECTED_ENTRY_POINTS = {
|
||||
|
|
@ -135,6 +152,9 @@ EXPECTED_CLI_COMMANDS = {
|
|||
"doctor",
|
||||
"filter",
|
||||
"generation-diff",
|
||||
"graph-plan",
|
||||
"graph-render",
|
||||
"graph-render-status",
|
||||
"impact",
|
||||
"info",
|
||||
"onboard",
|
||||
|
|
@ -180,6 +200,8 @@ EXPECTED_MCP_TOOLS = {
|
|||
"docforge_rebase_changeset",
|
||||
"docforge_register_changes",
|
||||
"docforge_render_status",
|
||||
"docforge_graph_plan",
|
||||
"docforge_graph_render_status",
|
||||
"docforge_search",
|
||||
"docforge_stop_visualization",
|
||||
"docforge_sync",
|
||||
|
|
@ -236,6 +258,9 @@ class PublicContractTests(unittest.TestCase):
|
|||
)
|
||||
self.assertIn("--project-root", completed.stdout)
|
||||
self.assertIn("--no-ast", completed.stdout)
|
||||
self.assertIn("--manual-render-policy", completed.stdout)
|
||||
self.assertIn("--portable-graph-policy", completed.stdout)
|
||||
self.assertIn("--live-viewer-policy", completed.stdout)
|
||||
|
||||
def test_published_schemas_validate_their_current_contract_examples(self) -> None:
|
||||
for path in sorted(SCHEMAS.glob("*.json")):
|
||||
|
|
|
|||
|
|
@ -55,6 +55,14 @@ class DocForgeRenderingTests(unittest.TestCase):
|
|||
self.assertEqual("missing", missing["outputs"][0]["state"])
|
||||
|
||||
first = service.render("manual")
|
||||
projection_receipt = first["output"]["projection_receipt"]
|
||||
self.assertIsInstance(projection_receipt, dict)
|
||||
assert isinstance(projection_receipt, dict)
|
||||
self.assertEqual("manual", projection_receipt["kind"])
|
||||
self.assertEqual(
|
||||
first["output"]["actual_output_hash"],
|
||||
projection_receipt["artifacts"][0]["sha256"],
|
||||
)
|
||||
output = root / ".docforge/rendered/manual.html"
|
||||
first_bytes = output.read_bytes()
|
||||
second = service.render("manual")
|
||||
|
|
@ -191,7 +199,12 @@ class DocForgeRenderingTests(unittest.TestCase):
|
|||
self.assertEqual("receipt_corrupt", corrupt["outputs"][0]["reason"])
|
||||
|
||||
def test_render_receipt_schema_and_renderer_version_fail_closed(self) -> None:
|
||||
for mutation in ("missing_hash", "renderer_version", "file_identity"):
|
||||
for mutation in (
|
||||
"missing_hash",
|
||||
"renderer_version",
|
||||
"file_identity",
|
||||
"projection_artifact",
|
||||
):
|
||||
with self.subTest(mutation=mutation), tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
service = RenderService(Project.open(root))
|
||||
|
|
@ -202,6 +215,8 @@ class DocForgeRenderingTests(unittest.TestCase):
|
|||
receipt.pop("output_hash")
|
||||
elif mutation == "renderer_version":
|
||||
receipt["renderer_version"] = "obsolete"
|
||||
elif mutation == "projection_artifact":
|
||||
receipt["projection_receipt"]["artifacts"][0]["sha256"] = "0" * 64
|
||||
else:
|
||||
receipt["output_file"].pop("ctime_ns")
|
||||
receipt_path.write_text(
|
||||
|
|
@ -401,6 +416,19 @@ class DocForgeRenderingTests(unittest.TestCase):
|
|||
with self.assertRaisesRegex(DocForgeError, "configured limit") as limit_error:
|
||||
limit_service.render("manual")
|
||||
self.assertEqual("render_too_large", limit_error.exception.code)
|
||||
|
||||
excessive = self.copy_fixture("alpha", Path(directory) / "excessive")
|
||||
excessive_descriptor = excessive / ".docforge/project.toml"
|
||||
excessive_descriptor.write_text(
|
||||
excessive_descriptor.read_text(encoding="utf-8").replace(
|
||||
"max_changeset_bytes = 100000",
|
||||
"max_changeset_bytes = 100000\nmax_render_bytes = 20000001",
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
permissive_limit = RenderService(Project.open(excessive)).render("manual")
|
||||
self.assertEqual("current", permissive_limit["receipt"]["state"])
|
||||
self.assertTrue((excessive / ".docforge/rendered/manual.html").is_file())
|
||||
self.assertFalse((limit_root / ".docforge/rendered/manual.html").exists())
|
||||
|
||||
template_limit_root = self.copy_fixture("alpha", parent / "template-limit")
|
||||
|
|
|
|||
99
tools/accessibility_fixture.py
Normal file
99
tools/accessibility_fixture.py
Normal file
|
|
@ -0,0 +1,99 @@
|
|||
"""Serve exact DocForge accessibility fixtures to the Playwright gate."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from docforge.graph_projection import (
|
||||
GraphViewRequestV1,
|
||||
build_graph_projection_package,
|
||||
build_graph_view_plan,
|
||||
)
|
||||
from docforge.index import ProjectIndex
|
||||
from docforge.project import Project
|
||||
from docforge.rendering import RenderService
|
||||
from docforge.visualization import VisualizationRunner
|
||||
from docforge_renderers.graph import PortableGraphHtmlRenderer
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def _manual_html(project: Project) -> str:
|
||||
result = RenderService(project).render("manual")
|
||||
output = result.get("output")
|
||||
if not isinstance(output, dict):
|
||||
raise RuntimeError("Manual render did not return output identity")
|
||||
relative_path = cast(dict[str, object], output).get("path")
|
||||
if not isinstance(relative_path, str):
|
||||
raise RuntimeError("Manual render did not return an output path")
|
||||
return (project.descriptor.root / relative_path).read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _portable_graph_html(project: Project) -> str:
|
||||
plan = build_graph_view_plan(
|
||||
project.load(),
|
||||
GraphViewRequestV1(
|
||||
view_id="accessibility-gate",
|
||||
title="Portable graph accessibility fixture",
|
||||
root_node_id="guide.workflow",
|
||||
depth=2,
|
||||
max_nodes=20,
|
||||
max_edges=40,
|
||||
max_work=1_000,
|
||||
),
|
||||
False,
|
||||
)
|
||||
package = build_graph_projection_package(
|
||||
plan,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=1_000_000,
|
||||
)
|
||||
artifact = PortableGraphHtmlRenderer().render(package).artifacts[0]
|
||||
return artifact.content.decode("utf-8")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
with tempfile.TemporaryDirectory(prefix="docforge-accessibility-") as directory:
|
||||
project_root = Path(directory) / "alpha"
|
||||
shutil.copytree(ROOT / "tests" / "fixtures" / "alpha", project_root)
|
||||
project = Project.open(project_root)
|
||||
index = ProjectIndex(project)
|
||||
index.build()
|
||||
portable_html = _portable_graph_html(project)
|
||||
manual_html = _manual_html(project)
|
||||
runner = VisualizationRunner(
|
||||
index,
|
||||
register_atexit=False,
|
||||
persistent=True,
|
||||
)
|
||||
try:
|
||||
visualization = runner.start(node_id="guide.workflow", depth=2)
|
||||
live_url = visualization.get("url")
|
||||
if not isinstance(live_url, str):
|
||||
raise RuntimeError("Live viewer did not return a URL")
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"manual_html": manual_html,
|
||||
"portable_html": portable_html,
|
||||
"live_url": live_url,
|
||||
},
|
||||
separators=(",", ":"),
|
||||
),
|
||||
flush=True,
|
||||
)
|
||||
sys.stdin.buffer.read()
|
||||
finally:
|
||||
runner.stop()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
|
@ -10,9 +10,19 @@ import tempfile
|
|||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from docforge.graph_projection import (
|
||||
GraphViewRequestV1,
|
||||
build_graph_projection_package,
|
||||
build_graph_view_plan,
|
||||
)
|
||||
from docforge.index import ProjectIndex
|
||||
from docforge.project import Project
|
||||
from docforge.rendering import RenderService
|
||||
from docforge_renderers.graph import (
|
||||
PORTABLE_GRAPH_CSS,
|
||||
PORTABLE_GRAPH_JAVASCRIPT,
|
||||
PortableGraphHtmlRenderer,
|
||||
)
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
ASSET_ROOT = ROOT / "src" / "docforge" / "assets"
|
||||
|
|
@ -39,6 +49,30 @@ def _rendered_manual_html() -> str:
|
|||
return (project_root / relative_path).read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _portable_graph_html() -> str:
|
||||
project = Project.open(ROOT / "tests" / "fixtures" / "alpha")
|
||||
plan = build_graph_view_plan(
|
||||
project.load(),
|
||||
GraphViewRequestV1(
|
||||
view_id="web-quality",
|
||||
title="Portable graph quality fixture",
|
||||
root_node_id="guide.workflow",
|
||||
depth=2,
|
||||
max_nodes=20,
|
||||
max_edges=40,
|
||||
max_work=1_000,
|
||||
),
|
||||
False,
|
||||
)
|
||||
package = build_graph_projection_package(
|
||||
plan,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=1_000_000,
|
||||
)
|
||||
return PortableGraphHtmlRenderer().render(package).artifacts[0].content.decode("utf-8")
|
||||
|
||||
|
||||
def _tool(name: str) -> str:
|
||||
suffix = ".cmd" if os.name == "nt" else ""
|
||||
path = ROOT / "node_modules" / ".bin" / f"{name}{suffix}"
|
||||
|
|
@ -82,6 +116,16 @@ def main() -> int:
|
|||
],
|
||||
_rendered_manual_html(),
|
||||
),
|
||||
(
|
||||
"portable graph HTML",
|
||||
[
|
||||
_tool("html-validate"),
|
||||
"--stdin",
|
||||
"--stdin-filename=portable-graph.html",
|
||||
"--max-warnings=0",
|
||||
],
|
||||
_portable_graph_html(),
|
||||
),
|
||||
(
|
||||
"CSS",
|
||||
[
|
||||
|
|
@ -102,6 +146,26 @@ def main() -> int:
|
|||
],
|
||||
(ASSET_ROOT / "graph.js").read_text(encoding="utf-8"),
|
||||
),
|
||||
(
|
||||
"portable graph CSS",
|
||||
[
|
||||
_tool("stylelint"),
|
||||
"--stdin",
|
||||
"--stdin-filename=portable-graph.css",
|
||||
"--max-warnings=0",
|
||||
],
|
||||
PORTABLE_GRAPH_CSS,
|
||||
),
|
||||
(
|
||||
"portable graph JavaScript",
|
||||
[
|
||||
_tool("eslint"),
|
||||
"--stdin",
|
||||
"--stdin-filename=portable-graph.js",
|
||||
"--max-warnings=0",
|
||||
],
|
||||
PORTABLE_GRAPH_JAVASCRIPT,
|
||||
),
|
||||
)
|
||||
results = [_run(label, command, content) for label, command, content in checks]
|
||||
return 0 if all(results) else 1
|
||||
|
|
|
|||
893
tools/milestone3_benchmark.py
Normal file
893
tools/milestone3_benchmark.py
Normal file
|
|
@ -0,0 +1,893 @@
|
|||
"""Milestone 3 projection, worker, fragment, and publication benchmark gates."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import gc
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import platform
|
||||
import resource
|
||||
import statistics
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import tracemalloc
|
||||
from collections.abc import Callable, Mapping, Sequence
|
||||
from dataclasses import dataclass, replace
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from milestone0_baseline import write_synthetic_project
|
||||
|
||||
from docforge.graph_projection import (
|
||||
GraphViewRequestV1,
|
||||
build_graph_projection_package,
|
||||
build_graph_view_plan,
|
||||
)
|
||||
from docforge.graph_rendering import GraphRenderService
|
||||
from docforge.manual_projection import (
|
||||
build_manual_projection_package,
|
||||
build_manual_render_plan,
|
||||
)
|
||||
from docforge.models import ProjectSnapshot
|
||||
from docforge.project import Project
|
||||
from docforge.projection_contract import (
|
||||
MAX_PACKAGE_BYTES,
|
||||
MAX_PLAN_BYTES,
|
||||
MAX_RECEIPT_BYTES,
|
||||
GraphViewPlanV1,
|
||||
ManualRenderPlanV1,
|
||||
ProjectionPackageV1,
|
||||
ProjectionRenderResult,
|
||||
canonical_projection_bytes,
|
||||
projection_hash,
|
||||
)
|
||||
from docforge.projection_fragments import (
|
||||
FragmentKey,
|
||||
FragmentRecord,
|
||||
ProjectionFragmentCache,
|
||||
fragment_semantic_hash,
|
||||
)
|
||||
from docforge.projection_worker import (
|
||||
MAX_WORKER_ARTIFACT_BYTES,
|
||||
render_projection_in_worker,
|
||||
)
|
||||
from docforge.render_contract import GenericHtmlRenderer, PreparedRender
|
||||
from docforge.rendering import RenderService
|
||||
from docforge.telemetry import COUNTER_NAMES, request
|
||||
from docforge_renderers.graph import PortableGraphHtmlRenderer
|
||||
from docforge_renderers.manual import ManualHtmlRenderer
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
FULL_NODE_COUNT = 1_000
|
||||
SMOKE_NODE_COUNT = 25
|
||||
DEFAULT_FULL_SAMPLES = 3
|
||||
DEFAULT_SMOKE_SAMPLES = 1
|
||||
MAX_STATUS_RESPONSE_BYTES = 256_000
|
||||
MAX_TRACED_PEAK_BYTES = 256 * 1024 * 1024
|
||||
MAX_CHILD_PEAK_BYTES = 256 * 1024 * 1024
|
||||
|
||||
ZERO_WORK_COUNTERS = tuple(
|
||||
counter for counter in COUNTER_NAMES if counter != "source_generation_checks"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _ProjectionSample:
|
||||
plan_id: str
|
||||
package_id: str
|
||||
artifact: bytes
|
||||
receipt: Mapping[str, object]
|
||||
plan_bytes: int
|
||||
package_bytes: int
|
||||
child_peak_memory_bytes: int | None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _StatusSample:
|
||||
response: Mapping[str, object]
|
||||
counters: Mapping[str, object]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _FragmentSweep:
|
||||
fragment_count: int
|
||||
aggregate_content_bytes: int
|
||||
ordered_record_hash: str
|
||||
|
||||
|
||||
def _parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Gate DocForge2 Milestone 3 projection behavior on a disposable project."
|
||||
)
|
||||
parser.add_argument("--mode", choices=("smoke", "full"), default="full")
|
||||
parser.add_argument("--nodes", type=int)
|
||||
parser.add_argument("--samples", type=int)
|
||||
parser.add_argument("--output", type=Path)
|
||||
return parser
|
||||
|
||||
|
||||
def encode_report(value: object) -> str:
|
||||
"""Serialize one report deterministically for files, CI logs, and comparisons."""
|
||||
|
||||
return json.dumps(value, sort_keys=True, indent=2, ensure_ascii=False) + "\n"
|
||||
|
||||
|
||||
def _git(arguments: list[str]) -> str:
|
||||
return subprocess.run(
|
||||
["git", *arguments],
|
||||
cwd=ROOT,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout.strip()
|
||||
|
||||
|
||||
def _compact_size(value: object) -> int:
|
||||
return len(canonical_projection_bytes(value))
|
||||
|
||||
|
||||
def _sha256(content: bytes) -> str:
|
||||
return hashlib.sha256(content).hexdigest()
|
||||
|
||||
|
||||
def _prepare_fixture(root: Path, node_count: int) -> None:
|
||||
write_synthetic_project(root, node_count)
|
||||
descriptor = root / ".docforge" / "project.toml"
|
||||
maximum_edges = node_count - 1
|
||||
maximum_work = max(100, node_count * 4)
|
||||
original = descriptor.read_text(encoding="utf-8")
|
||||
original = original.replace(
|
||||
"max_render_bytes = 20000000",
|
||||
"max_render_bytes = 4000000",
|
||||
)
|
||||
descriptor.write_text(
|
||||
original
|
||||
+ f"""
|
||||
|
||||
[graph_render]
|
||||
output_root = ".docforge/portable-graph"
|
||||
|
||||
[[graph_render.views]]
|
||||
id = "architecture"
|
||||
renderer = "portable_graph_html"
|
||||
output = "architecture.html"
|
||||
title = "Synthetic architecture"
|
||||
query = "synthetic measurement"
|
||||
initial_mode = "web"
|
||||
depth = 1
|
||||
max_nodes = {node_count}
|
||||
max_edges = {maximum_edges}
|
||||
max_work = {maximum_work}
|
||||
families = ["guide"]
|
||||
relations = ["depends_on"]
|
||||
authorities = []
|
||||
statuses = ["active"]
|
||||
tags = []
|
||||
include_logic = false
|
||||
""",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
|
||||
def _manual_package(
|
||||
snapshot: ProjectSnapshot,
|
||||
) -> tuple[ManualRenderPlanV1, ProjectionPackageV1, str]:
|
||||
descriptor = snapshot.descriptor
|
||||
render = descriptor.render
|
||||
if render is None:
|
||||
raise RuntimeError("Milestone 3 fixture has no manual render configuration")
|
||||
view = render.views[0]
|
||||
renderer_version = GenericHtmlRenderer(incremental=False).renderer_version
|
||||
plan = build_manual_render_plan(snapshot, view, changeset_hash=None)
|
||||
package = build_manual_projection_package(
|
||||
plan,
|
||||
view.template_path.read_bytes(),
|
||||
renderer_id=ManualHtmlRenderer.renderer_id,
|
||||
renderer_version=renderer_version,
|
||||
max_output_bytes=descriptor.limits.max_render_bytes,
|
||||
)
|
||||
return plan, package, renderer_version
|
||||
|
||||
|
||||
def _graph_package(
|
||||
snapshot: ProjectSnapshot,
|
||||
node_count: int,
|
||||
) -> tuple[GraphViewPlanV1, ProjectionPackageV1]:
|
||||
descriptor = snapshot.descriptor
|
||||
plan = build_graph_view_plan(
|
||||
snapshot,
|
||||
GraphViewRequestV1(
|
||||
view_id="architecture",
|
||||
title="Synthetic architecture",
|
||||
query="synthetic measurement",
|
||||
initial_mode="web",
|
||||
depth=1,
|
||||
max_nodes=node_count,
|
||||
max_edges=node_count - 1,
|
||||
max_work=max(100, node_count * 4),
|
||||
families=("guide",),
|
||||
relations=("depends_on",),
|
||||
statuses=("active",),
|
||||
),
|
||||
False,
|
||||
)
|
||||
package = build_graph_projection_package(
|
||||
plan,
|
||||
renderer_id=PortableGraphHtmlRenderer.renderer_id,
|
||||
renderer_version=PortableGraphHtmlRenderer.renderer_version,
|
||||
max_output_bytes=descriptor.limits.max_render_bytes,
|
||||
)
|
||||
return plan, package
|
||||
|
||||
|
||||
def _projection_sample(
|
||||
plan: ManualRenderPlanV1 | GraphViewPlanV1,
|
||||
package: ProjectionPackageV1,
|
||||
result: ProjectionRenderResult,
|
||||
) -> _ProjectionSample:
|
||||
if len(result.artifacts) != 1:
|
||||
raise RuntimeError("Projection benchmark expected exactly one artifact")
|
||||
artifact = result.artifacts[0].content
|
||||
peak = result.receipt.document.get("peak_memory_bytes")
|
||||
if peak is not None and (type(peak) is not int or peak < 1):
|
||||
raise RuntimeError("Detached worker did not report valid peak memory")
|
||||
return _ProjectionSample(
|
||||
plan_id=plan.plan_id,
|
||||
package_id=package.package_id,
|
||||
artifact=artifact,
|
||||
receipt=result.receipt.as_dict(),
|
||||
plan_bytes=_compact_size(plan.as_dict()),
|
||||
package_bytes=_compact_size(package.as_dict()),
|
||||
child_peak_memory_bytes=peak,
|
||||
)
|
||||
|
||||
|
||||
def _projection_summary(sample: _ProjectionSample) -> dict[str, object]:
|
||||
return {
|
||||
"plan_id": sample.plan_id,
|
||||
"package_id": sample.package_id,
|
||||
"artifact_sha256": _sha256(sample.artifact),
|
||||
"artifact_bytes": len(sample.artifact),
|
||||
"plan_bytes": sample.plan_bytes,
|
||||
"package_bytes": sample.package_bytes,
|
||||
}
|
||||
|
||||
|
||||
def _measure(
|
||||
operation: Callable[[], object],
|
||||
*,
|
||||
samples: int,
|
||||
p95_limit_ms: float,
|
||||
response_limit_bytes: int,
|
||||
summary: Callable[[object], Mapping[str, object]],
|
||||
response_size: Callable[[object], int],
|
||||
warmups: int = 0,
|
||||
) -> tuple[dict[str, object], object]:
|
||||
for _ in range(warmups):
|
||||
operation()
|
||||
durations: list[float] = []
|
||||
traced_peaks: list[int] = []
|
||||
response_sizes: list[int] = []
|
||||
stable_summary: Mapping[str, object] | None = None
|
||||
last: object = None
|
||||
for _ in range(samples):
|
||||
gc.collect()
|
||||
tracemalloc.start()
|
||||
started = time.perf_counter_ns()
|
||||
try:
|
||||
value = operation()
|
||||
elapsed_ms = (time.perf_counter_ns() - started) / 1_000_000
|
||||
_, traced_peak = tracemalloc.get_traced_memory()
|
||||
finally:
|
||||
tracemalloc.stop()
|
||||
current_summary = dict(summary(value))
|
||||
if stable_summary is None:
|
||||
stable_summary = current_summary
|
||||
elif current_summary != stable_summary:
|
||||
raise RuntimeError("Milestone 3 operation changed deterministic result across samples")
|
||||
current_response_size = response_size(value)
|
||||
if current_response_size > response_limit_bytes:
|
||||
raise RuntimeError(
|
||||
"Milestone 3 response exceeded its fixed benchmark boundary: "
|
||||
f"{current_response_size} > {response_limit_bytes}"
|
||||
)
|
||||
if traced_peak > MAX_TRACED_PEAK_BYTES:
|
||||
raise RuntimeError(
|
||||
"Milestone 3 operation exceeded its traced-memory boundary: "
|
||||
f"{traced_peak} > {MAX_TRACED_PEAK_BYTES}"
|
||||
)
|
||||
durations.append(elapsed_ms)
|
||||
traced_peaks.append(traced_peak)
|
||||
response_sizes.append(current_response_size)
|
||||
last = value
|
||||
ordered = sorted(durations)
|
||||
p95_index = max(0, math.ceil(len(ordered) * 0.95) - 1)
|
||||
p95 = ordered[p95_index]
|
||||
if p95 > p95_limit_ms:
|
||||
raise RuntimeError(f"Milestone 3 operation p95 {p95:.3f} ms exceeds {p95_limit_ms:.3f} ms")
|
||||
assert stable_summary is not None
|
||||
return (
|
||||
{
|
||||
"samples": samples,
|
||||
"median_ms": round(statistics.median(ordered), 3),
|
||||
"p95_ms": round(p95, 3),
|
||||
"min_ms": round(ordered[0], 3),
|
||||
"max_ms": round(ordered[-1], 3),
|
||||
"p95_limit_ms": p95_limit_ms,
|
||||
"maximum_response_bytes": max(response_sizes),
|
||||
"response_limit_bytes": response_limit_bytes,
|
||||
"maximum_traced_peak_bytes": max(traced_peaks),
|
||||
"traced_peak_limit_bytes": MAX_TRACED_PEAK_BYTES,
|
||||
"stable_result": dict(stable_summary),
|
||||
},
|
||||
last,
|
||||
)
|
||||
|
||||
|
||||
def _projection_measurement(
|
||||
operation: Callable[[], _ProjectionSample],
|
||||
*,
|
||||
samples: int,
|
||||
p95_limit_ms: float,
|
||||
) -> tuple[dict[str, object], _ProjectionSample]:
|
||||
child_peaks: list[int] = []
|
||||
|
||||
def observed_operation() -> _ProjectionSample:
|
||||
sample = operation()
|
||||
if sample.child_peak_memory_bytes is not None:
|
||||
if sample.child_peak_memory_bytes > MAX_CHILD_PEAK_BYTES:
|
||||
raise RuntimeError(
|
||||
"Detached worker peak memory "
|
||||
f"{sample.child_peak_memory_bytes} exceeds "
|
||||
f"{MAX_CHILD_PEAK_BYTES} bytes"
|
||||
)
|
||||
child_peaks.append(sample.child_peak_memory_bytes)
|
||||
return sample
|
||||
|
||||
measurement, value = _measure(
|
||||
observed_operation,
|
||||
samples=samples,
|
||||
p95_limit_ms=p95_limit_ms,
|
||||
response_limit_bytes=MAX_RECEIPT_BYTES,
|
||||
summary=lambda item: _projection_summary(cast(_ProjectionSample, item)),
|
||||
response_size=lambda item: _compact_size(cast(_ProjectionSample, item).receipt),
|
||||
)
|
||||
sample = cast(_ProjectionSample, value)
|
||||
if sample.plan_bytes > MAX_PLAN_BYTES or sample.package_bytes > MAX_PACKAGE_BYTES:
|
||||
raise RuntimeError("Projection plan or package exceeded its protocol boundary")
|
||||
if child_peaks:
|
||||
measurement["maximum_child_peak_bytes"] = max(child_peaks)
|
||||
measurement["child_peak_limit_bytes"] = MAX_CHILD_PEAK_BYTES
|
||||
return measurement, sample
|
||||
|
||||
|
||||
def _profiled_status(operation: Callable[[], Mapping[str, object]]) -> _StatusSample:
|
||||
with request("benchmark.m3", enabled=True) as collector:
|
||||
response = operation()
|
||||
if collector is None:
|
||||
raise RuntimeError("Milestone 3 telemetry collector was not created")
|
||||
counters = collector.as_dict(outcome="ok")["counters"]
|
||||
typed_counters = cast(Mapping[str, object], counters)
|
||||
for counter in ZERO_WORK_COUNTERS:
|
||||
if typed_counters[counter] != 0:
|
||||
raise RuntimeError(f"Receipt-only status performed forbidden work: {counter}")
|
||||
if typed_counters["source_generation_checks"] != 2:
|
||||
raise RuntimeError("Receipt-only status did not perform its two race-safe source checks")
|
||||
if response.get("status") != "ok" or response.get("state") != "current":
|
||||
raise RuntimeError("Receipt-only status did not report a current publication")
|
||||
return _StatusSample(response=response, counters=typed_counters)
|
||||
|
||||
|
||||
def _status_summary(value: object) -> Mapping[str, object]:
|
||||
sample = cast(_StatusSample, value)
|
||||
return {
|
||||
"response": dict(sample.response),
|
||||
"counters": dict(sample.counters),
|
||||
}
|
||||
|
||||
|
||||
def _fragment_summary(value: object) -> Mapping[str, object]:
|
||||
sample = cast(_FragmentSweep, value)
|
||||
return {
|
||||
"fragment_count": sample.fragment_count,
|
||||
"aggregate_content_bytes": sample.aggregate_content_bytes,
|
||||
"ordered_record_hash": sample.ordered_record_hash,
|
||||
}
|
||||
|
||||
|
||||
def _fragment_sweep(records: Sequence[FragmentRecord]) -> _FragmentSweep:
|
||||
return _FragmentSweep(
|
||||
fragment_count=len(records),
|
||||
aggregate_content_bytes=sum(record.byte_count for record in records),
|
||||
ordered_record_hash=projection_hash([record.record_id for record in records]),
|
||||
)
|
||||
|
||||
|
||||
def _prepared_summary(value: object) -> Mapping[str, object]:
|
||||
prepared = cast(PreparedRender, value)
|
||||
return {
|
||||
"render_identity": prepared.render_identity,
|
||||
"output_sha256": prepared.output_hash,
|
||||
"output_bytes": len(prepared.output),
|
||||
}
|
||||
|
||||
|
||||
def _prepared_child_peak(prepared: PreparedRender) -> int:
|
||||
receipt = prepared.projection_receipt
|
||||
if not isinstance(receipt, Mapping):
|
||||
raise RuntimeError("Production detached render did not return a projection receipt")
|
||||
peak = receipt.get("peak_memory_bytes")
|
||||
if type(peak) is not int or peak < 1:
|
||||
raise RuntimeError("Production detached render did not report valid child peak memory")
|
||||
if peak > MAX_CHILD_PEAK_BYTES:
|
||||
raise RuntimeError(
|
||||
f"Production detached worker peak memory {peak} exceeds {MAX_CHILD_PEAK_BYTES} bytes"
|
||||
)
|
||||
return peak
|
||||
|
||||
|
||||
def _prepared_response_size(value: object) -> int:
|
||||
prepared = cast(PreparedRender, value)
|
||||
return _compact_size(prepared.projection_receipt)
|
||||
|
||||
|
||||
def _prepared_measurement(
|
||||
operation: Callable[[], PreparedRender],
|
||||
*,
|
||||
samples: int,
|
||||
p95_limit_ms: float,
|
||||
) -> tuple[dict[str, object], PreparedRender]:
|
||||
child_peaks: list[int] = []
|
||||
|
||||
def observed_operation() -> PreparedRender:
|
||||
prepared = operation()
|
||||
child_peaks.append(_prepared_child_peak(prepared))
|
||||
return prepared
|
||||
|
||||
measurement, value = _measure(
|
||||
observed_operation,
|
||||
samples=samples,
|
||||
p95_limit_ms=p95_limit_ms,
|
||||
response_limit_bytes=MAX_RECEIPT_BYTES,
|
||||
summary=_prepared_summary,
|
||||
response_size=_prepared_response_size,
|
||||
)
|
||||
measurement["maximum_child_peak_bytes"] = max(child_peaks)
|
||||
measurement["child_peak_limit_bytes"] = MAX_CHILD_PEAK_BYTES
|
||||
return measurement, cast(PreparedRender, value)
|
||||
|
||||
|
||||
def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
|
||||
project = Project.open(root)
|
||||
snapshot = project.load()
|
||||
if len(snapshot.nodes) != node_count or len(snapshot.edges) != node_count - 1:
|
||||
raise RuntimeError("Milestone 3 fixture does not have full synthetic coverage")
|
||||
|
||||
operations: dict[str, object] = {}
|
||||
|
||||
def manual_full() -> _ProjectionSample:
|
||||
plan, package, renderer_version = _manual_package(snapshot)
|
||||
result = ManualHtmlRenderer(renderer_version).render(package)
|
||||
return _projection_sample(plan, package, result)
|
||||
|
||||
operations["manual_full_render"], manual_result = _projection_measurement(
|
||||
manual_full,
|
||||
samples=samples,
|
||||
p95_limit_ms=15_000,
|
||||
)
|
||||
|
||||
def graph_full() -> _ProjectionSample:
|
||||
plan, package = _graph_package(snapshot, node_count)
|
||||
result = PortableGraphHtmlRenderer().render(package)
|
||||
return _projection_sample(plan, package, result)
|
||||
|
||||
operations["portable_graph_full_render"], graph_result = _projection_measurement(
|
||||
graph_full,
|
||||
samples=samples,
|
||||
p95_limit_ms=10_000,
|
||||
)
|
||||
|
||||
manual_plan, manual_package, manual_renderer_version = _manual_package(snapshot)
|
||||
graph_plan, graph_package = _graph_package(snapshot, node_count)
|
||||
graph_diagnostics = cast(Mapping[str, object], graph_plan.document["diagnostics"])
|
||||
manual_pages = cast(list[dict[str, object]], manual_plan.document["pages"])
|
||||
if (
|
||||
len(manual_pages) != node_count
|
||||
or graph_diagnostics["returned_nodes"] != node_count
|
||||
or graph_diagnostics["returned_edges"] != node_count - 1
|
||||
):
|
||||
raise RuntimeError("Projection plans did not retain every synthetic node and edge")
|
||||
|
||||
manual_worker_measurement, manual_worker = _projection_measurement(
|
||||
lambda: _projection_sample(
|
||||
manual_plan,
|
||||
manual_package,
|
||||
render_projection_in_worker(manual_package),
|
||||
),
|
||||
samples=samples,
|
||||
p95_limit_ms=20_000,
|
||||
)
|
||||
operations["manual_detached_worker"] = manual_worker_measurement
|
||||
manual_worker_peak = cast(
|
||||
int,
|
||||
manual_worker_measurement["maximum_child_peak_bytes"],
|
||||
)
|
||||
|
||||
graph_worker_measurement, graph_worker = _projection_measurement(
|
||||
lambda: _projection_sample(
|
||||
graph_plan,
|
||||
graph_package,
|
||||
render_projection_in_worker(graph_package),
|
||||
),
|
||||
samples=samples,
|
||||
p95_limit_ms=20_000,
|
||||
)
|
||||
operations["portable_graph_detached_worker"] = graph_worker_measurement
|
||||
graph_worker_peak = cast(
|
||||
int,
|
||||
graph_worker_measurement["maximum_child_peak_bytes"],
|
||||
)
|
||||
if (
|
||||
manual_worker.artifact != manual_result.artifact
|
||||
or graph_worker.artifact != graph_result.artifact
|
||||
):
|
||||
raise RuntimeError("Detached worker output is not byte-equivalent to in-process output")
|
||||
|
||||
manual_renderer = ManualHtmlRenderer(manual_renderer_version)
|
||||
fragment_records = [
|
||||
FragmentRecord.create(
|
||||
FragmentKey.create(
|
||||
projection_kind="manual",
|
||||
renderer_id=ManualHtmlRenderer.renderer_id,
|
||||
renderer_version=manual_renderer_version,
|
||||
component_version=GenericHtmlRenderer.page_component_version,
|
||||
semantic_input_hash=fragment_semantic_hash(page),
|
||||
),
|
||||
manual_renderer.render_page_fragment(page).encode("utf-8"),
|
||||
)
|
||||
for page in manual_pages
|
||||
]
|
||||
fragment_cache = ProjectionFragmentCache(
|
||||
root,
|
||||
root / ".docforge" / "milestone3-fragment-cache",
|
||||
)
|
||||
|
||||
def fragment_misses() -> _FragmentSweep:
|
||||
if any(fragment_cache.get(record.key) is not None for record in fragment_records):
|
||||
raise RuntimeError("Cold fragment-cache lookup unexpectedly hit")
|
||||
return _fragment_sweep(fragment_records)
|
||||
|
||||
operations["fragment_cache_miss_sweep"], _ = _measure(
|
||||
fragment_misses,
|
||||
samples=samples,
|
||||
p95_limit_ms=5_000,
|
||||
response_limit_bytes=32_768,
|
||||
summary=_fragment_summary,
|
||||
response_size=lambda item: _compact_size(_fragment_summary(item)),
|
||||
)
|
||||
|
||||
def fragment_puts() -> _FragmentSweep:
|
||||
published: list[FragmentRecord] = []
|
||||
for record in fragment_records:
|
||||
stored = fragment_cache.put(record.key, record.content)
|
||||
if stored != record:
|
||||
raise RuntimeError("Fragment cache did not publish an exact record")
|
||||
assert stored is not None
|
||||
published.append(stored)
|
||||
return _fragment_sweep(published)
|
||||
|
||||
operations["fragment_cache_put_sweep"], _ = _measure(
|
||||
fragment_puts,
|
||||
samples=1,
|
||||
p95_limit_ms=10_000,
|
||||
response_limit_bytes=32_768,
|
||||
summary=_fragment_summary,
|
||||
response_size=lambda item: _compact_size(_fragment_summary(item)),
|
||||
)
|
||||
|
||||
def fragment_hits() -> _FragmentSweep:
|
||||
loaded: list[FragmentRecord] = []
|
||||
for expected in fragment_records:
|
||||
record = fragment_cache.get(expected.key)
|
||||
if record != expected:
|
||||
raise RuntimeError("Fragment cache hit was not byte-exact")
|
||||
assert record is not None
|
||||
loaded.append(record)
|
||||
return _fragment_sweep(loaded)
|
||||
|
||||
operations["fragment_cache_hit_sweep"], _ = _measure(
|
||||
fragment_hits,
|
||||
samples=samples,
|
||||
p95_limit_ms=5_000,
|
||||
response_limit_bytes=32_768,
|
||||
summary=_fragment_summary,
|
||||
response_size=lambda item: _compact_size(_fragment_summary(item)),
|
||||
)
|
||||
|
||||
render_config = snapshot.descriptor.render
|
||||
assert render_config is not None
|
||||
incremental_package = build_manual_projection_package(
|
||||
manual_plan,
|
||||
render_config.views[0].template_path.read_bytes(),
|
||||
renderer_id=ManualHtmlRenderer.renderer_id,
|
||||
renderer_version=manual_renderer_version,
|
||||
max_output_bytes=snapshot.descriptor.limits.max_render_bytes,
|
||||
fragment_records=[record.as_dict() for record in fragment_records],
|
||||
)
|
||||
|
||||
def fragment_equivalence() -> _ProjectionSample:
|
||||
sample = _projection_sample(
|
||||
manual_plan,
|
||||
incremental_package,
|
||||
ManualHtmlRenderer(manual_renderer_version).render(incremental_package),
|
||||
)
|
||||
if sample.artifact != manual_result.artifact:
|
||||
raise RuntimeError("Fragment-assisted manual output is not byte-equivalent")
|
||||
return sample
|
||||
|
||||
operations["fragment_assisted_equivalence"], fragment_result = _projection_measurement(
|
||||
fragment_equivalence,
|
||||
samples=samples,
|
||||
p95_limit_ms=15_000,
|
||||
)
|
||||
|
||||
render_config = snapshot.descriptor.render
|
||||
assert render_config is not None
|
||||
manual_view = render_config.views[0]
|
||||
template_bytes = manual_view.template_path.read_bytes()
|
||||
production_renderer = GenericHtmlRenderer()
|
||||
operations["manual_incremental_cold"], cold_prepared = _prepared_measurement(
|
||||
lambda: production_renderer.prepare(
|
||||
snapshot,
|
||||
manual_view,
|
||||
template_bytes,
|
||||
changeset_hash=None,
|
||||
),
|
||||
samples=1,
|
||||
p95_limit_ms=20_000,
|
||||
)
|
||||
operations["manual_incremental_warm"], warm_prepared = _prepared_measurement(
|
||||
lambda: production_renderer.prepare(
|
||||
snapshot,
|
||||
manual_view,
|
||||
template_bytes,
|
||||
changeset_hash=None,
|
||||
),
|
||||
samples=samples,
|
||||
p95_limit_ms=20_000,
|
||||
)
|
||||
operations["manual_forced_full"], forced_prepared = _prepared_measurement(
|
||||
lambda: GenericHtmlRenderer(incremental=False).prepare(
|
||||
snapshot,
|
||||
manual_view,
|
||||
template_bytes,
|
||||
changeset_hash=None,
|
||||
),
|
||||
samples=samples,
|
||||
p95_limit_ms=20_000,
|
||||
)
|
||||
production_values = (cold_prepared, warm_prepared, forced_prepared)
|
||||
if len({value.output for value in production_values}) != 1:
|
||||
raise RuntimeError("Production cold, warm, and forced-full manual output differs")
|
||||
|
||||
changed_node = replace(
|
||||
snapshot.nodes[0],
|
||||
content=snapshot.nodes[0].content + "\nChanged projection content.\n",
|
||||
content_hash=_sha256(
|
||||
(snapshot.nodes[0].content + "\nChanged projection content.\n").encode()
|
||||
),
|
||||
)
|
||||
added_node = replace(
|
||||
snapshot.nodes[-1],
|
||||
node_id="guide.synthetic-added",
|
||||
title="Synthetic added",
|
||||
source_path="docs/content/synthetic-added.md",
|
||||
content_hash=_sha256(b"synthetic-added"),
|
||||
)
|
||||
variants = {
|
||||
"change": replace(
|
||||
snapshot,
|
||||
nodes=(changed_node, *snapshot.nodes[1:]),
|
||||
source_hash=_sha256(b"manual-change-variant"),
|
||||
),
|
||||
"add": replace(
|
||||
snapshot,
|
||||
nodes=(*snapshot.nodes, added_node),
|
||||
source_hash=_sha256(b"manual-add-variant"),
|
||||
),
|
||||
"delete": replace(
|
||||
snapshot,
|
||||
nodes=snapshot.nodes[:-1],
|
||||
edges=tuple(
|
||||
edge
|
||||
for edge in snapshot.edges
|
||||
if edge.source_id != snapshot.nodes[-1].node_id
|
||||
and edge.target_id != snapshot.nodes[-1].node_id
|
||||
),
|
||||
source_hash=_sha256(b"manual-delete-variant"),
|
||||
),
|
||||
"reorder": replace(
|
||||
snapshot,
|
||||
nodes=tuple(reversed(snapshot.nodes)),
|
||||
source_hash=_sha256(b"manual-reorder-variant"),
|
||||
),
|
||||
}
|
||||
variant_equivalence: dict[str, bool] = {}
|
||||
variant_child_peaks: list[int] = []
|
||||
for name, variant in variants.items():
|
||||
incremental = GenericHtmlRenderer().prepare(
|
||||
variant,
|
||||
manual_view,
|
||||
template_bytes,
|
||||
changeset_hash=None,
|
||||
)
|
||||
full = GenericHtmlRenderer(incremental=False).prepare(
|
||||
variant,
|
||||
manual_view,
|
||||
template_bytes,
|
||||
changeset_hash=None,
|
||||
)
|
||||
variant_child_peaks.extend(
|
||||
(
|
||||
_prepared_child_peak(incremental),
|
||||
_prepared_child_peak(full),
|
||||
)
|
||||
)
|
||||
variant_equivalence[name] = incremental.output == full.output
|
||||
if not all(variant_equivalence.values()):
|
||||
raise RuntimeError("Production incremental mutation output differs from forced full")
|
||||
|
||||
manual_service = RenderService(project)
|
||||
graph_service = GraphRenderService(project)
|
||||
manual_service.render("manual")
|
||||
graph_service.render("architecture")
|
||||
operations["manual_status_no_work"], manual_status = _measure(
|
||||
lambda: _profiled_status(lambda: manual_service.status("manual")),
|
||||
samples=samples,
|
||||
p95_limit_ms=500,
|
||||
response_limit_bytes=MAX_STATUS_RESPONSE_BYTES,
|
||||
summary=_status_summary,
|
||||
response_size=lambda item: _compact_size(cast(_StatusSample, item).response),
|
||||
)
|
||||
operations["portable_graph_status_no_work"], graph_status = _measure(
|
||||
lambda: _profiled_status(lambda: graph_service.status("architecture")),
|
||||
samples=samples,
|
||||
p95_limit_ms=500,
|
||||
response_limit_bytes=MAX_STATUS_RESPONSE_BYTES,
|
||||
summary=_status_summary,
|
||||
response_size=lambda item: _compact_size(cast(_StatusSample, item).response),
|
||||
)
|
||||
|
||||
return {
|
||||
"fixture": {
|
||||
"kind": "synthetic_generic_projection",
|
||||
"node_count": node_count,
|
||||
"edge_count": node_count - 1,
|
||||
"manual_page_count": len(manual_pages),
|
||||
"portable_graph_node_count": graph_diagnostics["returned_nodes"],
|
||||
"portable_graph_edge_count": graph_diagnostics["returned_edges"],
|
||||
"full_coverage": (
|
||||
len(manual_pages) == node_count
|
||||
and graph_diagnostics["returned_nodes"] == node_count
|
||||
and graph_diagnostics["returned_edges"] == node_count - 1
|
||||
),
|
||||
},
|
||||
"operations": operations,
|
||||
"equivalence": {
|
||||
"manual_in_process_vs_detached": manual_worker.artifact == manual_result.artifact,
|
||||
"portable_graph_in_process_vs_detached": graph_worker.artifact == graph_result.artifact,
|
||||
"manual_full_vs_fragment_assisted": fragment_result.artifact == manual_result.artifact,
|
||||
"manual_production_cold_warm_full": len({value.output for value in production_values})
|
||||
== 1,
|
||||
"manual_production_variants": variant_equivalence,
|
||||
},
|
||||
"sizes": {
|
||||
"manual": {
|
||||
**_projection_summary(manual_result),
|
||||
"receipt_bytes": _compact_size(manual_result.receipt),
|
||||
},
|
||||
"portable_graph": {
|
||||
**_projection_summary(graph_result),
|
||||
"receipt_bytes": _compact_size(graph_result.receipt),
|
||||
},
|
||||
"fragment_assisted_manual": {
|
||||
**_projection_summary(fragment_result),
|
||||
"receipt_bytes": _compact_size(fragment_result.receipt),
|
||||
"fragment_count": len(fragment_records),
|
||||
"aggregate_fragment_content_bytes": sum(
|
||||
record.byte_count for record in fragment_records
|
||||
),
|
||||
},
|
||||
"manual_status_response_bytes": _compact_size(
|
||||
cast(_StatusSample, manual_status).response
|
||||
),
|
||||
"portable_graph_status_response_bytes": _compact_size(
|
||||
cast(_StatusSample, graph_status).response
|
||||
),
|
||||
},
|
||||
"memory": {
|
||||
"process_peak_rss_kib": int(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss),
|
||||
"manual_worker_peak_bytes": manual_worker_peak,
|
||||
"manual_production_worker_peak_bytes": max(
|
||||
cast(
|
||||
int,
|
||||
operations["manual_incremental_cold"]["maximum_child_peak_bytes"],
|
||||
),
|
||||
cast(
|
||||
int,
|
||||
operations["manual_incremental_warm"]["maximum_child_peak_bytes"],
|
||||
),
|
||||
cast(
|
||||
int,
|
||||
operations["manual_forced_full"]["maximum_child_peak_bytes"],
|
||||
),
|
||||
*variant_child_peaks,
|
||||
),
|
||||
"portable_graph_worker_peak_bytes": graph_worker_peak,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
arguments = _parser().parse_args()
|
||||
default_nodes = FULL_NODE_COUNT if arguments.mode == "full" else SMOKE_NODE_COUNT
|
||||
default_samples = DEFAULT_FULL_SAMPLES if arguments.mode == "full" else DEFAULT_SMOKE_SAMPLES
|
||||
node_count = default_nodes if arguments.nodes is None else arguments.nodes
|
||||
samples = default_samples if arguments.samples is None else arguments.samples
|
||||
if not 2 <= node_count <= FULL_NODE_COUNT:
|
||||
raise SystemExit("--nodes must be between 2 and 1000")
|
||||
if arguments.mode == "full" and node_count != FULL_NODE_COUNT:
|
||||
raise SystemExit("--mode full requires exactly 1000 nodes")
|
||||
if samples < 1:
|
||||
raise SystemExit("--samples must be positive")
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="docforge-milestone3-") as directory:
|
||||
benchmark_root = Path(directory).resolve()
|
||||
_prepare_fixture(benchmark_root, node_count)
|
||||
measurement = _benchmark(benchmark_root, node_count, samples)
|
||||
|
||||
status = _git(["status", "--porcelain"])
|
||||
report: dict[str, object] = {
|
||||
"schema_version": 1,
|
||||
"benchmark": "docforge2_milestone3",
|
||||
"mode": arguments.mode,
|
||||
"source": {
|
||||
"revision": _git(["rev-parse", "HEAD"]),
|
||||
"dirty": bool(status),
|
||||
},
|
||||
"environment": {
|
||||
"platform": platform.platform(),
|
||||
"machine": platform.machine(),
|
||||
"python": platform.python_version(),
|
||||
"implementation": platform.python_implementation(),
|
||||
},
|
||||
"method": {
|
||||
"clock": "time.perf_counter_ns",
|
||||
"in_process_peak_memory": "tracemalloc per measured invocation",
|
||||
"detached_peak_memory": "worker receipt resource peak RSS",
|
||||
"process_peak_memory": "resource.getrusage(RUSAGE_SELF).ru_maxrss",
|
||||
"response_size": "UTF-8 bytes of canonical compact sorted JSON",
|
||||
"samples": samples,
|
||||
"full_mode_node_requirement": FULL_NODE_COUNT,
|
||||
"determinism": (
|
||||
"stable semantic summaries must match across samples; report JSON uses sorted keys"
|
||||
),
|
||||
"maximum_worker_artifact_bytes": MAX_WORKER_ARTIFACT_BYTES,
|
||||
},
|
||||
**measurement,
|
||||
}
|
||||
encoded = encode_report(report)
|
||||
if arguments.output is not None:
|
||||
output = arguments.output.resolve()
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
output.write_text(encoded, encoding="utf-8")
|
||||
sys.stdout.write(encoded)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Loading…
Add table
Add a link
Reference in a new issue