administrator/DocForge2
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

Compare commits

base: administrator:d6d9f47672c513c4927c5c42ad97a175ed651a74
Branches Tags
administrator:main
administrator:dev
administrator:codex/cross-identity-acceptance
administrator:v2.0.0
administrator:v1.4.0
administrator:v1.0.0
..
compare: administrator:4c5773c8652aedfeab82cf87ee7da1b7bc01eddc
Branches Tags
administrator:dev
administrator:main
administrator:codex/cross-identity-acceptance
administrator:v2.0.0
administrator:v1.4.0
administrator:v1.0.0

No commits in common. "d6d9f47672c513c4927c5c42ad97a175ed651a74" and "4c5773c8652aedfeab82cf87ee7da1b7bc01eddc" have entirely different histories.
d6d9f47672 ... 4c5773c865

72 changed files with 193 additions and 12118 deletions
Download patch file Download diff file Expand all files Collapse all files

24
ACTIVE_SLICE.md
View file

@ -1,13 +1,19 @@
# Milestone state
# Active milestone
```text
Last completed milestone: 3 — independent projections
Outcome: Manual output, portable graph artifacts, and the live viewer are independent generation-pinned consumers of the validated graph.
Evidence: Clean candidate f5dccb5e1c312121f1af63780162f593d9363b98; 281 tests and 272 subtests; 3 accessibility flows; clean 1,000-node ten-sample benchmark; isolated wheel proof; no secret-scan findings.
Active milestone: None.
Next directional milestone: 4 — adapter SDK and product documentation.
Status: Milestone 3 is closed. Milestone 4 has not started.
Milestone: 2 — agent retrieval and MCP experience
Goal: Let one project-bound server return compact, task-shaped, explainable context under an explicit effective policy.
In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks.
Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass.
Status: Complete. Effective policy, versioned task retrieval, latest-generation diff receipts,
logarithmic bounded page packing, deterministic client configuration, and the read-only integration
doctor are implemented and contract-tested. The complete repository gate passes with 205 tests and
120 subtests. Three independent adversarial audits found no remaining implementation blocker. The
clean 1,000-node baseline is recorded against candidate commit
`fb0df5e4a1c591c2a84788fd4814d98550f11863`, including task/generation reconstruction,
response-size behavior, zero-hidden-work counters, and isolated memory. No tag or release was
created, no production integration was repointed, and self-hosting remains out of scope.
```
Milestones 4–5 remain directional context. Do not begin Milestone 4 without a new active-slice
contract.
Milestones 3–5 remain directional context and are not active.

176
DEVELOPMENT_NOTES.md
View file

@ -345,7 +345,7 @@ These are notes, not commitments:
than bounded positions, they will need a different versioned security contract and persisted key
lifecycle.
## Milestone 2 — complete: agent retrieval and MCP experience
## Milestone 2 — active: agent retrieval and MCP experience
### Audit reconciliation
@ -610,177 +610,3 @@ Milestone 2 is complete. Follow-up ideas stay explicitly later-scope: avoid reco
task-shaped capsule for every continuation page, add authenticated continuation when the threat
model requires it, verify a native Claude timeout representation, and introduce adapter-owned
launcher metadata before generating configurations for custom adapters.
## Milestone 3 — complete: independent projections
Milestone 3 began only after `main` and `dev` were aligned at the verified Milestone 2 closeout.
Three read-only audits ran before source changes:
- Manual planning, immutable packages, renderer isolation, receipts, preview/application
integration, and full/incremental equivalence.
- Portable graph planning, static artifacts, the live viewer boundary, worker protocol, and static
plus interactive accessibility.
- Packaging, optional dependencies, public contracts, projection policies, performance,
incremental fragments, and maintained gates.
The active design constraints are unchanged: renderers consume one validated immutable generation;
manual and graph plans remain separate; the live viewer is not retrieval authority; core remains
usable without rendering; status performs no hidden rendering; full rendering remains the recovery
and equivalence oracle; no storage rewrite is assumed.
### Milestone 3 architecture decision
The three audits converged on one compatibility-first boundary:
- The existing `docforge.render_contract` names, `GenericHtmlRenderer.prepare()` signature,
`generic_html` renderer identity, and byte output remain the version-1 compatibility surface.
They become adapters over the new manual-planning path rather than being changed in place.
- New `ManualRenderPlanV1`, `GraphViewPlanV1`, `ProjectionPackageV1`, and
`ProjectionReceiptV1` contracts use strict canonical JSON, deterministic ordering, independent
item and byte bounds, exact generation and policy binding, and content-derived identities.
- Plans and packages contain selected graph facts and bounded content. They never contain a
project object, SQLite handle, absolute project or index path, arbitrary query, command, or
project-provided executable code.
- The planner owns graph selection and meaning. A renderer may transform only a validated package
into declared artifacts and cannot select nodes, invent relationships, crawl the project, choose
publication paths, or mutate canonical sources.
- Manual and portable graph renderers live behind independent import boundaries. Renderer
dependencies load lazily. Default installation behavior remains compatible during the initial
migration; optional dependency changes require their own verified packaging decision.
- Portable graph rendering is additive. It does not replace or silently change
`docforge_visualize`, `graph-browser@17`, the viewer-manager protocol, or the query-backed live
viewer.
- Effective policy version 1 remains frozen. Milestone 3 introduces a version-2 projection-policy
view for manual `auto|explicit|disabled`, portable graph `explicit|disabled`, and live viewer
`on-demand|disabled` enforcement, while retaining the version-1 projection for existing clients.
- Publication commits content-addressed artifacts first, renderer evidence second, and a bounded
generation/view manifest last. Status remains receipt-only. Failures after artifact replacement
report committed degraded success rather than an ordinary failed mutation.
- Full planning and rendering remain the recovery and equivalence oracle. Incremental fragments
are disposable, keyed from complete plan semantics, and may be reused only when byte-exact
artifact equivalence is proven.
- The live source endpoint must stop reading mutable canonical files behind a pinned graph
snapshot. Portable artifacts never inherit that path-bearing behavior.
The first implementation slice freezes existing golden output, adds the four versioned contracts
and validators, introduces pure manual and graph planners, and makes the legacy manual renderer a
compatibility wrapper. Publication hardening, detached rendering, incremental fragments, portable
graph publication, independent policy enforcement, accessibility, and maintained performance
gates follow on top of that frozen boundary.
### Milestone 3 contract slice
The first slice now implements:
- Strict Draft 2020-12 schemas and runtime canonical-hash validation for manual plans, graph plans,
projection packages, and projection receipts.
- A deterministic manual planner that owns page selection, navigation, cross-references,
backlinks, search documents, component assignments, orphan diagnostics, and cycle diagnostics.
- A deterministic graph planner with exact-root or metadata-only lexical scope, closed filters,
explicit node/edge/work bounds, deterministic omissions, path/source-body exclusion, and
no-AST Logic exclusion.
- A separate `docforge_renderers.manual` package. Its renderer accepts only a validated package and
has no project, SQLite, publication-path, or filesystem-write API.
- The frozen `GenericHtmlRenderer` compatibility shim over the new planner/package/renderer
pipeline. The alpha artifact remains exactly 2,043 bytes with output SHA-256
`81656bb89debc7ad1fbe8bc290e9a3ba90664442b17a6d57e908d30d20c47f77` and legacy render identity
`1c0a49c28ba3b0dabf94be36e75def197dee1be3cb73ac405b09875383c8dc5f`.
- Rejection of project-template scripts, inline event handlers, `javascript:` URLs, embedded
browsing contexts, and refresh redirects.
- Wheel inclusion for both typed packages and every published JSON schema. Importing `docforge`
no longer imports `markdown_it` or the manual renderer package.
- A live-viewer correction: source evidence now comes from the pinned index generation. The
viewer no longer reopens mutable canonical files behind an older graph snapshot.
The new repository-native contract target passed 91 tests and 120 subtests at the slice boundary.
The combined projection, rendering, and live-viewer focus passed with byte-exact compatibility and
no hidden source/path authority.
### Durable portable graph publication
The portable graph path now has its own declared `graph_render` views, pure plans, fixed
`portable_graph_html` renderer, content-addressed artifact store, renderer receipts, and one bounded
generation/view manifest as the publication commit. It supports Nodes, Flow, and Web without
including Logic. Static HTML contains the complete pre-rendered graph and treats JavaScript as
progressive enhancement.
Publication revalidates source, view, artifact, receipt, and output identities across replacement.
Status reads only bounded manifest and receipt evidence. It never plans or renders. Repair may
restore a declared output from its content-addressed artifact. A post-artifact failure that cannot
be rolled back returns explicit degraded committed evidence rather than reporting an ordinary
failed mutation.
### Detached workers and incremental fragments
Manual and portable graph packages execute through one fixed one-request child protocol. The
parent launches isolated Python from a trusted working directory with a sanitized environment,
spools stdout to disk, reads one bounded canonical response, and validates the complete artifact
and receipt identity. The worker accepts only the two built-in renderer identities. Requests are
bounded by the 24,000,000-byte package contract, actual artifact transfer by 20,000,000 bytes, and
execution by a 30-second timeout.
Manual fragment records are semantic, versioned, canonical, hash-bound, and stored below a
dedicated confined cache. The worker independently recomputes the expected page fragment before
using a record. Corrupt, forged, oversized, stale, or aggregate-oversized records fall back to the
full detached render. Cold fragment creation is compared byte-for-byte with that full oracle before
cache publication. The cache retains only the current inventory and is capped at 10,000 entries
and 64,000,000 bytes.
### Independent policies and accessibility
Projection policy version 2 independently composes manual `auto|explicit|disabled`, portable graph
`explicit|disabled`, and live viewer `on-demand|disabled`. CLI, MCP, generated client
configuration, doctor, render services, canonical application, onboarding, and viewer-manager
entry points enforce their relevant policy. Status remains available when an active operation is
disabled.
Generated client evidence binds the projection policy, its hash, projection availability, and the
current descriptor hash into the configuration hash. Validation cross-checks omitted default
selectors against the bound descriptor so coordinated policy and availability drift fails closed.
The version-1 effective-policy payload remains unchanged for existing clients.
Pinned Playwright 1.62.0 and axe-core 4.12.1 gates exercise the frozen manual, portable graph, and
live viewer with selected WCAG A/AA axe tags and keyboard interaction flows. Portable and live
graph presentation received only the minimal contrast and nested-role corrections needed by those
gates.
### Scale and runtime hardening
The first 1,000-node full benchmark exposed recursive strongly connected-component traversal in
manual planning. Cycle detection now uses an iterative two-pass traversal. A regression covers the
descriptor maximum of 10,000 nodes as both a deep acyclic chain and one strongly connected
component.
The isolated wheel proof also exposed a Python `runpy` warning when the worker module was imported
during package initialization before `-m` execution. A private fixed module entrypoint now owns
child startup. Malformed child input returns code 2 with empty stdout and stderr.
Configured render ceilings above 20,000,000 bytes remain accepted for compatibility, and small
actual artifacts render normally. The detached protocol still rejects an actual transfer beyond
its fixed 20,000,000-byte boundary.
### Milestone 3 closeout
Candidate `f5dccb5e1c312121f1af63780162f593d9363b98` passed the complete repository gate: formatting,
Python and web lint, strict types, compilation, 281 tests and 272 subtests, three accessibility
flows, lock and dependency checks, package builds, and all milestone smoke benchmarks. The
maintained projection contract subset passed 142 tests and 236 subtests.
The clean ten-sample 1,000-node benchmark passed every latency, memory, response-size, no-work, and
equivalence gate. Manual full rendering measured 810.490 ms p95, portable graph full rendering
323.690 ms p95, and receipt-only status 111.381 ms and 59.331 ms p95 respectively. Direct detached
worker peaks were 88,580,096 and 89,583,616 bytes. The separately gated production manual worker
peak was 104,771,584 bytes. Production cold, warm, forced-full, add, change, delete, and reorder
outputs were byte-identical.
Production warm fragment rendering measured 2,206.540 ms p95 versus 978.870 ms for forced full.
Milestone 3 therefore closes the fragment isolation, invalidation, equivalence, and recovery
contract without claiming a throughput win. Later optimization must begin from that evidence.
The exact method and measurements are recorded in `docs/MILESTONE_3_BASELINE.md` and
`benchmarks/milestone3-2026-07-29.json`. The candidate passed an isolated wheel CLI/MCP/worker
proof. Gitleaks 8.30.1 found no findings across the six Milestone 3 commits or candidate tree.
Milestone 3 is complete. No tag, release, production integration repointing, WorldForge change,
ScrapeStation change, storage rewrite, or self-hosting dependency was introduced. Milestone 4
remains directional and has not started.

25
Makefile
View file

@ -5,10 +5,7 @@ NPM := npm
PYTHONPYCACHEPREFIX := /tmp/docforge-quality-pycache
PYTEST_BASETEMP := /tmp/docforge-quality-pytest
.PHONY: accessibility benchmark benchmark-m1 benchmark-m1-smoke benchmark-m2 benchmark-m2-smoke benchmark-m3 benchmark-m3-full benchmark-m3-smoke benchmark-smoke build compile contract dependencies format-check gate lint lock test type
accessibility:
$(NPM) run test:accessibility
.PHONY: benchmark benchmark-m1 benchmark-m1-smoke benchmark-m2 benchmark-m2-smoke benchmark-smoke build compile contract dependencies format-check gate lint lock test type
format-check:
$(PYTHON) -m ruff format --check src tests tools
@ -28,18 +25,9 @@ contract:
-p no:cacheprovider --basetemp=$(PYTEST_BASETEMP) \
tests/test_public_contract.py \
tests/test_policy.py \
tests/test_projection_policy.py \
tests/test_projection_policy_integration.py \
tests/test_projection_worker.py \
tests/test_projection_fragments.py \
tests/test_retrieval.py \
tests/test_generation_diff.py \
tests/test_client_integration.py \
tests/test_projection_contract.py \
tests/test_projection_schemas.py \
tests/test_graph_projection.py \
tests/test_graph_rendering.py \
tests/test_graph_publication.py \
tests/test_observability.py::TelemetryContractTests::test_schema_fixed_names_match_the_implementation \
tests/test_adapter_contract.py::AdapterContractTests::test_no_ast_index_policy_rejects_logic_publication \
tests/test_adapter_contract.py::AdapterContractTests::test_no_ast_index_accepts_legacy_and_non_logic_incremental_adapters \
@ -80,13 +68,4 @@ benchmark-m2-smoke:
benchmark-m2:
$(PYTHON) tools/milestone2_benchmark.py --nodes 1000 --samples 10
benchmark-m3-smoke:
$(PYTHON) tools/milestone3_benchmark.py --mode smoke \
--output /tmp/docforge-milestone3-smoke.json > /dev/null
benchmark-m3:
$(PYTHON) tools/milestone3_benchmark.py --mode full
benchmark-m3-full: benchmark-m3
gate: format-check lint type compile contract test accessibility lock dependencies build benchmark-smoke benchmark-m1-smoke benchmark-m2-smoke benchmark-m3-smoke
gate: format-check lint type compile contract test lock dependencies build benchmark-smoke benchmark-m1-smoke benchmark-m2-smoke

73
README.md
View file

@ -26,12 +26,6 @@ declared manuals, visualizes project structure, and manages reviewable documenta
- Detects project-local adapter implementation and configuration changes and requires a fresh
project-bound process before any further MCP work.
- Keeps function-scoped control-flow projections separate from the primary architecture graph.
- Compiles manuals and portable graph artifacts from separate versioned, generation-pinned plans
and immutable packages.
- Runs built-in manual and graph renderers in fixed detached workers with validated receipts,
bounded transfer, and no project-path authority.
- Enforces independent manual, portable-graph, and live-viewer policy while keeping status
receipt-only.
- Runs a managed loopback graph browser with neighborhood, semantic Flow, convergence Web,
function-scoped Logic, source inspection, and branch-aware node hiding.
- Supports generic documentation projects and project-owned source adapters.
@ -79,63 +73,6 @@ details, and explicit truncation. Paged results use one top-level cursor and a v
`receipt_header`; `stored_receipt_hash` identifies the complete persisted receipt. The read never
exposes Logic details, loads canonical source, repairs derived state, or invents history.
## Independent projections
Manual compilation, portable graph rendering, and the live viewer consume the same validated graph
generation through separate boundaries:
```text
validated generation
├── ManualRenderPlanV1 → immutable package → detached manual renderer
├── GraphViewPlanV1 → immutable package → detached portable graph renderer
└── pinned index → managed read-only live viewer
```
Plans, packages, and receipts are canonical, versioned, hash-identified, bounded, and contain no
project object, SQLite handle, absolute project path, command, or caller-selected renderer module.
Renderers cannot select graph facts, crawl canonical sources, choose publication paths, or mutate
project state.
Declare portable graph output separately from manual views:
```toml
[graph_render]
output_root = ".docforge/portable-graph"
[[graph_render.views]]
id = "architecture"
renderer = "portable_graph_html"
output = "architecture.html"
title = "Architecture"
root = "architecture.overview"
initial_mode = "web"
depth = 3
max_nodes = 250
max_edges = 1000
max_work = 100000
include_logic = false
```
Plan, publish, and inspect it explicitly:
```bash
.venv/bin/docforge --project-root "$PROJECT" graph-plan architecture
.venv/bin/docforge --project-root "$PROJECT" graph-render architecture
.venv/bin/docforge --project-root "$PROJECT" graph-render-status architecture
```
The version-2 projection policy independently selects manual
`auto|explicit|disabled`, portable graph `explicit|disabled`, and live viewer
`on-demand|disabled`. Use `--manual-render-policy`, `--portable-graph-policy`, and
`--live-viewer-policy` on CLI/MCP startup or generated client configuration. Status remains
available when the corresponding active operation is disabled.
Manual fragment reuse is disposable. Cold record publication is guarded by byte-exact comparison
with a full detached render; warm records are independently recomputed and validated inside the
worker. Full rendering remains the recovery and equivalence oracle. Portable artifacts commit
content-addressed output and renderer evidence before one bounded generation/view manifest; status
does not plan or render.
## Graph views
The browser presents the primary architecture graph through three complementary views and loads a
@ -233,18 +170,10 @@ DocForge describes them as a source graph.
performance, memory, rendering and response sizes, bottlenecks, and missing coverage.
- [Milestone 0 closeout](docs/MILESTONE_0_CLOSEOUT.md) — lineage, migration, security scan,
repository state, and fresh-clone proof.
- [Milestone 1 baseline](docs/MILESTONE_1_BASELINE.md) — warm operation latency, structured work,
status, retrieval, and memory measurements.
- [Milestone 1 closeout](docs/MILESTONE_1_CLOSEOUT.md) — fast-core contracts, adversarial
validation, compatibility boundaries, and exact candidate evidence.
- [Milestone 2 baseline](docs/MILESTONE_2_BASELINE.md) — task context, generation diff, client
configuration, doctor, response-size, counter, and memory measurements.
- [Milestone 2 closeout](docs/MILESTONE_2_CLOSEOUT.md) — implemented contracts, adversarial
validation, exclusions, and exact candidate evidence.
- [Milestone 3 baseline](docs/MILESTONE_3_BASELINE.md) — manual, fragment, worker, portable graph,
status, equivalence, response-size, and memory measurements.
- [Milestone 3 closeout](docs/MILESTONE_3_CLOSEOUT.md) — independent projection contracts,
adversarial validation, compatibility boundaries, and exact candidate evidence.
- [MCP contract](docs/MCP_CONTRACT.md) — exact tool and process boundary.
- [Viewer manager](docs/VIEWER_MANAGER.md) — native service setup and lifecycle.
- [Adapter decision](docs/APPLICATION_DECISION.md) — why custom adapters own canonical
@ -268,8 +197,6 @@ make gate
Focused entry points are available as `make contract`, `make test`, `make type`,
`make benchmark-smoke`, `make benchmark`, `make benchmark-m1-smoke`, and
`make benchmark-m1`. Milestone 2 adds `make benchmark-m2-smoke` and `make benchmark-m2`.
Milestone 3 adds `make accessibility`, `make benchmark-m3-smoke`, `make benchmark-m3`, and
`make benchmark-m3-full`.
The committed 1,000-node baseline and its measurement method are under `benchmarks/`.

46
SLICE_HISTORY.md
View file

@ -1,51 +1,5 @@
# Completed slices
## DocForge2 Milestone 3 independent projections
### Changed
- Added strict version-1 manual plan, graph plan, projection package, and projection receipt
contracts with canonical identities and packaged schemas.
- Kept the legacy manual API and exact alpha bytes as a compatibility wrapper over a pure planner,
immutable package, and independent renderer.
- Added generation-pinned portable graph planning, detached rendering, content-addressed durable
publication, repair, and receipt-only status.
- Added a fixed isolated worker protocol with bounded request, response, artifact, timeout,
environment, and renderer inventory.
- Added semantic manual fragments with independent worker validation, corruption recovery,
full-render equivalence, and bounded cache retention.
- Added version-2 independent manual, portable-graph, and live-viewer policy with descriptor-bound
generated-client evidence while retaining effective policy version 1.
- Corrected live source reads to use the pinned index generation.
- Added static and interactive accessibility gates for the manual, portable graph, and live viewer.
- Replaced recursive cycle planning with an iterative traversal proven at 10,000 nodes.
### Verification
- The complete gate passed 281 tests, 272 subtests, three accessibility flows, dependency checks,
builds, and all milestone smoke benchmarks.
- The maintained projection contract subset passed 142 tests and 236 subtests.
- The clean ten-sample 1,000-node benchmark passed every latency, memory, response-size, no-work,
deterministic-output, and full/incremental equivalence threshold.
- An isolated wheel passed CLI, MCP, detached manual rendering, and malformed worker startup.
- Three adversarial review tracks were reconciled, and Gitleaks found no findings in the six
milestone commits or candidate tree.
### Limits
- Production fragment reuse is correct and recoverable but is slower than forced full rendering at
the maintained 1,000-page fixture. No speedup is claimed.
- Actual detached artifact transfer is capped at 20,000,000 bytes.
- Portable graph publication is CLI-owned. MCP exposes read-only plan and status tools.
- Remote render services, render farms, third-party renderer ecosystems, storage replacement,
self-hosting, adapter SDK expansion, and production integration changes remain out of scope.
- No tag or release was created.
### Next gate
Milestone 4 remains directional and is not active. Create a new active-slice contract before
starting adapter SDK or product-documentation implementation.
## DocForge2 Milestone 0 successor foundation
### Changed

24
benchmarks/README.md
View file

@ -34,14 +34,6 @@ make benchmark-m2-smoke
make benchmark-m2
```
Run the Milestone 3 independent-projection smoke and maintained full gates:
```bash
make benchmark-m3-smoke
make benchmark-m3
make benchmark-m3-full
```
The benchmark creates canonical sources, derived state, changesets, rendered output, and caches
only in a disposable temporary directory. It does not read another project, self-host DocForge, or
mutate repository content.
@ -64,16 +56,9 @@ records whether diagnostics were dropped for response budget, checks all hidden-
and measures isolated-process peak RSS. Its interpretation is in
[`docs/MILESTONE_2_BASELINE.md`](../docs/MILESTONE_2_BASELINE.md).
`milestone3-2026-07-29.json` is the clean-tree independent-projection baseline captured from commit
`f5dccb5e1c312121f1af63780162f593d9363b98`. It measures versioned manual and graph planning,
in-process and detached rendering, production cold/warm/forced-full behavior, fragment cache
sweeps, add/change/delete/reorder equivalence, portable publication, receipt-only status, traced
memory, detached worker peak memory, and response size. Its interpretation is in
[`docs/MILESTONE_3_BASELINE.md`](../docs/MILESTONE_3_BASELINE.md).
The generic fixtures expose whole-source and projection scaling. They do not replace incremental
adapter equivalence tests. Milestone 3's portable fixture contains 1,000 Nodes/Flow/Web nodes and
999 edges; portable version 1 deliberately excludes Logic.
The generic fixture exposes whole-source scaling. It does not replace the incremental adapter
equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has
no portable graph-planning or graph-rendering contract.
The Milestone 1 harness treats wall time and structured work counters as separate gates. Warm
operations fail if they load a complete project, parse source files, reconstruct an adapter
@ -88,5 +73,4 @@ status. The 1,000-node run records bounded semantic summaries for exact errors,
backlinks, both traversal directions, paged context, render receipt states, and visualization
freshness. The reported p95 uses the nearest-rank method; with ten samples it is the maximum.
`process_peak_rss_kib` is the cumulative main-process `RUSAGE_SELF` high-water mark and excludes the
detached viewer worker. It is diagnostic and not operation-local. Milestone 3 memory gates use
per-operation `tracemalloc` peaks and detached worker receipt peaks instead.
detached viewer worker.

442
benchmarks/milestone3-2026-07-29.json
View file

@ -1,442 +0,0 @@
{
"benchmark": "docforge2_milestone3",
"environment": {
"implementation": "CPython",
"machine": "x86_64",
"platform": "Linux-7.1.3-200.nobara.fc44.x86_64-x86_64-with-glibc2.43",
"python": "3.14.6"
},
"equivalence": {
"manual_full_vs_fragment_assisted": true,
"manual_in_process_vs_detached": true,
"manual_production_cold_warm_full": true,
"manual_production_variants": {
"add": true,
"change": true,
"delete": true,
"reorder": true
},
"portable_graph_in_process_vs_detached": true
},
"fixture": {
"edge_count": 999,
"full_coverage": true,
"kind": "synthetic_generic_projection",
"manual_page_count": 1000,
"node_count": 1000,
"portable_graph_edge_count": 999,
"portable_graph_node_count": 1000
},
"memory": {
"manual_production_worker_peak_bytes": 104771584,
"manual_worker_peak_bytes": 88580096,
"portable_graph_worker_peak_bytes": 89583616,
"process_peak_rss_kib": 102692
},
"method": {
"clock": "time.perf_counter_ns",
"detached_peak_memory": "worker receipt resource peak RSS",
"determinism": "stable semantic summaries must match across samples; report JSON uses sorted keys",
"full_mode_node_requirement": 1000,
"in_process_peak_memory": "tracemalloc per measured invocation",
"maximum_worker_artifact_bytes": 20000000,
"process_peak_memory": "resource.getrusage(RUSAGE_SELF).ru_maxrss",
"response_size": "UTF-8 bytes of canonical compact sorted JSON",
"samples": 10
},
"mode": "full",
"operations": {
"fragment_assisted_equivalence": {
"max_ms": 666.119,
"maximum_response_bytes": 664,
"maximum_traced_peak_bytes": 5332792,
"median_ms": 638.999,
"min_ms": 633.157,
"p95_limit_ms": 15000,
"p95_ms": 666.119,
"response_limit_bytes": 128000,
"samples": 10,
"stable_result": {
"artifact_bytes": 583149,
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"package_bytes": 2364302,
"package_id": "ecc21e58106c420a9f78ffbba997778ca98b61586ecfdb23a15b08f431a52c42",
"plan_bytes": 1006393,
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
},
"traced_peak_limit_bytes": 268435456
},
"fragment_cache_hit_sweep": {
"max_ms": 514.324,
"maximum_response_bytes": 145,
"maximum_traced_peak_bytes": 6941174,
"median_ms": 498.911,
"min_ms": 495.447,
"p95_limit_ms": 5000,
"p95_ms": 514.324,
"response_limit_bytes": 32768,
"samples": 10,
"stable_result": {
"aggregate_content_bytes": 516921,
"fragment_count": 1000,
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
},
"traced_peak_limit_bytes": 268435456
},
"fragment_cache_miss_sweep": {
"max_ms": 106.888,
"maximum_response_bytes": 145,
"maximum_traced_peak_bytes": 146245,
"median_ms": 106.203,
"min_ms": 105.383,
"p95_limit_ms": 5000,
"p95_ms": 106.888,
"response_limit_bytes": 32768,
"samples": 10,
"stable_result": {
"aggregate_content_bytes": 516921,
"fragment_count": 1000,
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
},
"traced_peak_limit_bytes": 268435456
},
"fragment_cache_put_sweep": {
"max_ms": 539.212,
"maximum_response_bytes": 145,
"maximum_traced_peak_bytes": 509548,
"median_ms": 539.212,
"min_ms": 539.212,
"p95_limit_ms": 10000,
"p95_ms": 539.212,
"response_limit_bytes": 32768,
"samples": 1,
"stable_result": {
"aggregate_content_bytes": 516921,
"fragment_count": 1000,
"ordered_record_hash": "99a95d575504238233a6367c884fa1d50231855f2f2ffe3dae90fa18c4136f72"
},
"traced_peak_limit_bytes": 268435456
},
"manual_detached_worker": {
"child_peak_limit_bytes": 268435456,
"max_ms": 599.394,
"maximum_child_peak_bytes": 88580096,
"maximum_response_bytes": 667,
"maximum_traced_peak_bytes": 28203708,
"median_ms": 591.317,
"min_ms": 585.123,
"p95_limit_ms": 20000,
"p95_ms": 599.394,
"response_limit_bytes": 128000,
"samples": 10,
"stable_result": {
"artifact_bytes": 583149,
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"package_bytes": 1007297,
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
"plan_bytes": 1006393,
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
},
"traced_peak_limit_bytes": 268435456
},
"manual_forced_full": {
"child_peak_limit_bytes": 268435456,
"max_ms": 978.87,
"maximum_child_peak_bytes": 104771584,
"maximum_response_bytes": 668,
"maximum_traced_peak_bytes": 30425714,
"median_ms": 944.135,
"min_ms": 934.773,
"p95_limit_ms": 20000,
"p95_ms": 978.87,
"response_limit_bytes": 128000,
"samples": 10,
"stable_result": {
"output_bytes": 583149,
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
},
"traced_peak_limit_bytes": 268435456
},
"manual_full_render": {
"max_ms": 810.49,
"maximum_response_bytes": 664,
"maximum_traced_peak_bytes": 4412754,
"median_ms": 801.948,
"min_ms": 773.7,
"p95_limit_ms": 15000,
"p95_ms": 810.49,
"response_limit_bytes": 128000,
"samples": 10,
"stable_result": {
"artifact_bytes": 583149,
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"package_bytes": 1007297,
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
"plan_bytes": 1006393,
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de"
},
"traced_peak_limit_bytes": 268435456
},
"manual_incremental_cold": {
"child_peak_limit_bytes": 268435456,
"max_ms": 2827.152,
"maximum_child_peak_bytes": 99454976,
"maximum_response_bytes": 668,
"maximum_traced_peak_bytes": 35160716,
"median_ms": 2827.152,
"min_ms": 2827.152,
"p95_limit_ms": 20000,
"p95_ms": 2827.152,
"response_limit_bytes": 128000,
"samples": 1,
"stable_result": {
"output_bytes": 583149,
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
},
"traced_peak_limit_bytes": 268435456
},
"manual_incremental_warm": {
"child_peak_limit_bytes": 268435456,
"max_ms": 2206.54,
"maximum_child_peak_bytes": 104767488,
"maximum_response_bytes": 669,
"maximum_traced_peak_bytes": 34676043,
"median_ms": 2143.388,
"min_ms": 2125.466,
"p95_limit_ms": 20000,
"p95_ms": 2206.54,
"response_limit_bytes": 128000,
"samples": 10,
"stable_result": {
"output_bytes": 583149,
"output_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0"
},
"traced_peak_limit_bytes": 268435456
},
"manual_status_no_work": {
"max_ms": 111.381,
"maximum_response_bytes": 1542,
"maximum_traced_peak_bytes": 1792025,
"median_ms": 62.881,
"min_ms": 60.111,
"p95_limit_ms": 500,
"p95_ms": 111.381,
"response_limit_bytes": 256000,
"samples": 10,
"stable_result": {
"counters": {
"adapter_projection_loads": 0,
"adapter_source_extractions": 0,
"index_builds": 0,
"index_checks": 0,
"index_synchronizations": 0,
"project_loads": 0,
"render_output_bytes_built": 0,
"render_output_bytes_hashed": 0,
"render_prepare_calls": 0,
"source_bytes_parsed": 0,
"source_files_parsed": 0,
"source_generation_checks": 2,
"viewer_manager_requests": 0
},
"response": {
"adapter": "generic",
"configured": true,
"outputs": [
{
"actual_output_hash": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"expected_output_hash": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"path": ".docforge/rendered/manual.html",
"projection_receipt": {
"artifacts": [
{
"artifact_id": "manual.html",
"bytes": 583149,
"media_type": "text/html; charset=utf-8",
"sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c"
}
],
"contract": "docforge.projection-receipt",
"diagnostics": {
"warnings": []
},
"kind": "manual",
"package_id": "b30c65390570537a16ce03afe6a992599c6ca0c4ef5ea2f06d14b33e248bc9aa",
"peak_memory_bytes": 104771584,
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
"receipt_id": "ead4c25b34ada8df674903a592e25dd6bbf9817df2b04e5ebc66cde9e41c1ed1",
"renderer": {
"renderer_id": "generic_html",
"renderer_version": "1+markdown-it-py-4.2.0"
},
"schema_version": 1,
"timing": {
"elapsed_ns": 114544865
}
},
"reason": null,
"receipt_schema_version": 1,
"render_identity": "73f9a175afd7bfea5af70d4c7df902c982f042c413d533817dc2bade9e54d4b0",
"renderer": "generic_html",
"renderer_version": "1+markdown-it-py-4.2.0",
"state": "current",
"template_hash": "ae0ebb3eadeb530e9d033c8fbd21321d406e29d04c0ae7d07a9a0f929f0ffa9f",
"verification": "receipt",
"view_id": "manual"
}
],
"project_id": "synthetic-1000",
"project_root_fingerprint": "0747acfd975703c0",
"revision": "unversioned",
"source_hash": "a314da4ffa8fcf291ef7a7b0fc87737caeaa89fa3c558ea442afeb0e5ae49c2d",
"state": "current",
"status": "ok",
"verification": "receipt"
}
},
"traced_peak_limit_bytes": 268435456
},
"portable_graph_detached_worker": {
"child_peak_limit_bytes": 268435456,
"max_ms": 268.428,
"maximum_child_peak_bytes": 89583616,
"maximum_response_bytes": 660,
"maximum_traced_peak_bytes": 27587227,
"median_ms": 265.448,
"min_ms": 263.761,
"p95_limit_ms": 20000,
"p95_ms": 268.428,
"response_limit_bytes": 128000,
"samples": 10,
"stable_result": {
"artifact_bytes": 718383,
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
"package_bytes": 398715,
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
"plan_bytes": 398158,
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b"
},
"traced_peak_limit_bytes": 268435456
},
"portable_graph_full_render": {
"max_ms": 323.69,
"maximum_response_bytes": 657,
"maximum_traced_peak_bytes": 3490976,
"median_ms": 303.736,
"min_ms": 301.215,
"p95_limit_ms": 10000,
"p95_ms": 323.69,
"response_limit_bytes": 128000,
"samples": 10,
"stable_result": {
"artifact_bytes": 718383,
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
"package_bytes": 398715,
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
"plan_bytes": 398158,
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b"
},
"traced_peak_limit_bytes": 268435456
},
"portable_graph_status_no_work": {
"max_ms": 59.331,
"maximum_response_bytes": 879,
"maximum_traced_peak_bytes": 685012,
"median_ms": 58.467,
"min_ms": 58.052,
"p95_limit_ms": 500,
"p95_ms": 59.331,
"response_limit_bytes": 256000,
"samples": 10,
"stable_result": {
"counters": {
"adapter_projection_loads": 0,
"adapter_source_extractions": 0,
"index_builds": 0,
"index_checks": 0,
"index_synchronizations": 0,
"project_loads": 0,
"render_output_bytes_built": 0,
"render_output_bytes_hashed": 0,
"render_prepare_calls": 0,
"source_bytes_parsed": 0,
"source_files_parsed": 0,
"source_generation_checks": 2,
"viewer_manager_requests": 0
},
"response": {
"adapter": "generic",
"configured": true,
"outputs": [
{
"artifact": {
"artifact_id": "portable-graph.html",
"bytes": 718383,
"media_type": "text/html; charset=utf-8",
"sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30"
},
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
"path": ".docforge/portable-graph/architecture.html",
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b",
"publication_id": "2620ce106593d5499931b145a42fc1eccd31f5e356b19bf1d76687f8d45bcad2",
"reason": null,
"renderer": "portable_graph_html",
"renderer_version": "1",
"state": "current",
"verification": "manifest",
"view_id": "architecture"
}
],
"project_id": "synthetic-1000",
"project_root_fingerprint": "0747acfd975703c0",
"revision": "unversioned",
"source_hash": "a314da4ffa8fcf291ef7a7b0fc87737caeaa89fa3c558ea442afeb0e5ae49c2d",
"state": "current",
"status": "ok"
}
},
"traced_peak_limit_bytes": 268435456
}
},
"schema_version": 1,
"sizes": {
"fragment_assisted_manual": {
"aggregate_fragment_content_bytes": 516921,
"artifact_bytes": 583149,
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"fragment_count": 1000,
"package_bytes": 2364302,
"package_id": "ecc21e58106c420a9f78ffbba997778ca98b61586ecfdb23a15b08f431a52c42",
"plan_bytes": 1006393,
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
"receipt_bytes": 664
},
"manual": {
"artifact_bytes": 583149,
"artifact_sha256": "278ff8cd6b092e7d8114527dba88051917aa827b22d4aed61e57acf004e5870c",
"package_bytes": 1007297,
"package_id": "5fec20ace40b6db7e89083704af371245a5b5b73247f71edb6eab0cb2bd5e2c1",
"plan_bytes": 1006393,
"plan_id": "47008f5177230e3ad5abbfee8bccaff0498963106cb33ed7cdd6f8bbc3c495de",
"receipt_bytes": 664
},
"manual_status_response_bytes": 1542,
"portable_graph": {
"artifact_bytes": 718383,
"artifact_sha256": "91dd5bb31679225e5af40a1f94a9e75f72921e36416e3d8b00c45f46a37a3a30",
"package_bytes": 398715,
"package_id": "72f96d7ab92c1d6b59f4f32539558232ecf35deda099fc4887c77890da1d4309",
"plan_bytes": 398158,
"plan_id": "6dac35937d855f72c6670f87aace3c0cb1dbff67bb842c55370429584064f76b",
"receipt_bytes": 657
},
"portable_graph_status_response_bytes": 879
},
"source": {
"dirty": false,
"revision": "f5dccb5e1c312121f1af63780162f593d9363b98"
}
}

49
docs/COMPATIBILITY.md
View file

@ -71,19 +71,6 @@ Existing hand-written client configurations remain valid and are never rewritten
Doctor is inspection-only and does not become a hidden bootstrap, synchronization, or migration
path.
The following Milestone 3 CLI additions are also additive:
- `docforge graph-plan VIEW_ID`
- `docforge graph-render VIEW_ID`
- `docforge graph-render-status [VIEW_ID]`
- `--manual-render-policy auto|explicit|disabled`
- `--portable-graph-policy explicit|disabled`
- `--live-viewer-policy on-demand|disabled`
MCP adds the read-only `docforge_graph_plan` and `docforge_graph_render_status` tools. Portable
graph publication remains an explicit local CLI integration action. Existing manual render,
preview, visualization, and status names remain supported.
## Versioned data contracts
Milestone 0 preserves:
@ -109,11 +96,6 @@ Milestone 0 preserves:
- Latest-generation-diff page schema version 1. Pages use one top-level pagination object and a
nested `receipt_header`. `stored_receipt_hash` names the complete stored receipt. Opaque cursors
may be restarted after a server or receipt change and are not durable public identifiers.
- Manual render-plan schema version 1.
- Graph view-plan schema version 1.
- Projection-package schema version 1.
- Projection-receipt schema version 1.
- Independent projection-policy schema version 2. Effective policy version 1 remains frozen.
Indexes, attestations, extraction caches, previews, and rendered artifacts are disposable. A schema
change may rebuild them. Canonical project content and stored proposals may not be silently
@ -179,24 +161,12 @@ The following guarantees remain stable:
## Rendering and visualization
The `generic_html` renderer remains the supported version-1 manual projection. Its public
`GenericHtmlRenderer.prepare()` signature, renderer identity, frozen alpha bytes, confined paths,
raw-HTML suppression, fixed template tokens, deterministic identities, atomic replacement, and
side-effect-free status remain compatible. It now delegates through a versioned manual plan,
immutable package, and independent renderer.
The `portable_graph_html` renderer and `graph_render` descriptor table are additive. Manual and
portable graph declarations, plans, policies, publication receipts, and status remain separate.
The portable renderer does not replace the existing live viewer or `docforge_visualize`.
Existing project descriptors may retain any positive `max_render_bytes` accepted by schema version
1. A value above 20,000,000 bytes does not make the descriptor invalid, and a smaller actual
artifact still renders. Actual detached worker transfer is a separate fixed 20,000,000-byte
runtime boundary.
The `generic_html` renderer remains the supported version-1 manual projection. It retains confined
paths, raw-HTML suppression, fixed template tokens, deterministic identities, atomic replacement,
and side-effect-free status.
The live graph viewer remains a read-only consumer of a generation-pinned validated index. It does
not become project authority or MCP retrieval authority. Source reads use the pinned index
generation instead of reopening mutable canonical files behind that generation.
not become project authority or MCP retrieval authority.
## Task-context compatibility
@ -229,9 +199,8 @@ The exact version-1 relation aliases are frozen by the MCP contract and reposito
Changing an alias category requires a new planner version; it is not a silent implementation
detail.
Milestone 3 adds `ManualRenderPlanV1`, `GraphViewPlanV1`, projection package and receipt version 1,
and projection policy version 2. These are additive submodule and schema contracts. They do not
change the legacy task-context, adapter, changeset, or effective-policy contracts described above.
`ManualRenderPlan`, `GraphViewPlan`, a portable graph renderer, and independently packaged
renderers are later-milestone direction. Milestone 0 does not claim that those contracts exist.
## Safety boundary
@ -252,8 +221,6 @@ Milestone 0 records rather than redesigns these areas:
callers use targeted retrieval for that node.
- One individually oversized changeset diff is transported as reconstructable canonical-JSON
chunks. Cursors are corruption-detecting read tokens, not authenticated authorization tokens.
- Production fragment validation is currently slower than forced-full rendering at the maintained
1,000-page fixture. Full rendering remains the equivalence and recovery oracle.
- Remote render services, render farms, third-party renderer ecosystems, and a separate render MCP
remain deferred.
- Manual planning is not separated from rendering.
- There is no portable graph-planning or graph-rendering contract.
- DocForge2 does not self-host its bootstrap documentation.

67
docs/CONTRACT.md
View file

@ -24,11 +24,6 @@ commit when Git is available; it cannot change repository state.
- Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1.
- Generated client configuration: `schemas/client-configuration.schema.json`, version 1.
- Client doctor result: `schemas/doctor-result.schema.json`, version 1.
- Manual render plan: `schemas/manual-render-plan.schema.json`, version 1.
- Graph view plan: `schemas/graph-view-plan.schema.json`, version 1.
- Projection package: `schemas/projection-package.schema.json`, version 1.
- Projection receipt: `schemas/projection-receipt.schema.json`, version 1.
- Independent projection policy: `schemas/projection-policy.schema.json`, version 2.
- Index schema: version 3, disposable and reproducible.
- Index attestation: schema version 1, disposable and reproducible.
- Core, CLI, and MCP server: version 1.3.0.dev0.
@ -87,9 +82,7 @@ the header alone. One top-level pagination object carries the only continuation
Generated Codex, Claude, and OpenClaw fragments are machine-local projections. They are not
canonical project content. Version 1 binds the selected project, exact isolated Python
interpreter, canonical argument layout, effective policy, no-AST projection, render policy,
timeouts, artifact bytes, and configuration hash. Milestone 3 adds the version-2 projection policy,
its hash, projection availability, and the exact descriptor hash to that attested configuration
evidence. Omitted default selectors are recomposed against the bound descriptor.
timeouts, artifact bytes, and configuration hash.
Preview is side-effect free. Explicit publication creates only one new private standalone
fragment in an existing real directory. It never merges or replaces different content. Descriptor,
@ -142,12 +135,10 @@ preview root, and one or more stable view IDs. Each view names a built-in render
derived output file, title, and optional family filter. Paths are resolved under the project root
and may not overlap canonical content, authority files, changesets, templates, or previews.
The `generic_html` renderer uses pinned CommonMark parsing with raw HTML disabled. Templates are
UTF-8 files with a fixed token vocabulary; they cannot name commands, modules, or executable
The initial `generic_html` renderer uses pinned CommonMark parsing with raw HTML disabled. Templates
are UTF-8 files with a fixed token vocabulary; they cannot name commands, modules, or executable
renderers. Render identity covers the canonical source hash, optional changeset hash, selected node
and edge identities, view configuration, template hash, renderer contract, and exact parser
version. The frozen version-1 API and alpha bytes are preserved by a compatibility wrapper over the
manual plan/package/renderer path.
and edge identities, view configuration, template hash, renderer contract, and exact parser version.
An explicit CLI render atomically replaces one declared derived output. MCP can render a validated
changeset only to its isolated preview path. Normal status verifies bounded source, configuration,
@ -155,56 +146,6 @@ template, output, renderer, and publication-receipt identities without reconstru
Explicit deep status remains the side-effect-free full-render oracle. Input changes detected before
atomic replacement fail without publishing a current receipt for stale output.
## Independent projection boundary
Manual and portable graph plans are separate version-1 contracts over one immutable validated
generation. They use canonical JSON, deterministic ordering, fixed structural and serialized-size
bounds, and content-derived identities. Plans contain selected graph facts and bounded content.
They contain no live project object, database handle, absolute project or index path, arbitrary
query, command, executable path, or caller-selected module.
Projection packages bind one plan to inert assets, a closed built-in renderer identity, declared
component versions, and an artifact inventory with a byte allowance. Receipts bind the exact
package, plan, renderer, artifact hashes and sizes, diagnostics, timing, and detached peak memory.
Manual and graph renderer modules accept only validated packages. They cannot select nodes, invent
relationships, read project state, choose publication paths, or write canonical files.
Detached execution uses one fixed private Python module, isolated mode, a trusted working
directory, a sanitized environment, exactly one canonical newline-terminated JSON request and
response, a closed renderer allowlist, a 30-second timeout, disk-spooled stdout, and bounded reads. The package
contract is capped at 24,000,000 bytes and actual detached artifact transfer at 20,000,000 bytes.
Project descriptors may retain a larger `max_render_bytes` compatibility allowance, but an actual
detached transfer above the fixed worker boundary fails closed.
Portable graph configuration is independent of manual render configuration. One view selects
either an exact root or a bounded metadata-only lexical query plus closed filters and node, edge,
depth, and work limits. Logic is excluded. The renderer emits a complete static Nodes, Flow, or Web
artifact and uses JavaScript only as progressive enhancement.
Portable publication commits a content-addressed artifact, renderer receipt, and one bounded
generation/view manifest in that order. The manifest is the publication commit. Status reads only
bounded manifest and receipt evidence and never plans or renders. Repair restores declared output
only from validated content-addressed evidence. A failure after a replacement that cannot be
proven rolled back returns explicit degraded committed evidence.
Manual fragment records are disposable semantic cache entries. Their keys bind the renderer,
component version, and complete page semantics. The detached renderer recomputes the expected page
fragment before accepting cached bytes. Cold creation is compared with a full detached render
before cache publication. Invalid, corrupt, forged, stale, individually oversized, or
aggregate-oversized records fall back to the full oracle. The dedicated cache retains only current
keys and is capped at 10,000 entries and 64,000,000 bytes.
Projection policy version 2 composes manual `auto|explicit|disabled`, portable graph
`explicit|disabled`, and live viewer `on-demand|disabled` independently. Active plan, render,
application, onboarding, and viewer-start operations enforce the relevant policy before hidden
work. Receipt-only status and explicit viewer stop remain available. Effective policy version 1
and its legacy projections remain unchanged.
The live viewer remains separate from portable graph publication. It consumes one
generation-pinned validated index through the viewer manager. Source reads come from that pinned
generation and do not reopen mutable canonical files behind an older snapshot. Neither live nor
portable visualization is retrieval or canonical authority.
Normal MCP access does not expose canonical application. An explicitly configured canonical
applier registers one hash-bound application tool. No MCP mode exposes arbitrary renderer
execution, arbitrary file writes, shell commands, Git mutation, build commands, deployment, or

90
docs/MILESTONE_3_BASELINE.md
View file

@ -1,90 +0,0 @@
# Milestone 3 baseline
## Scope and method
This baseline records the independent-projection behavior completed in Milestone 3. It was
captured on 2026-07-29 from clean candidate commit
`f5dccb5e1c312121f1af63780162f593d9363b98`.
The maintained command was:
```bash
.venv/bin/python tools/milestone3_benchmark.py \
--mode full \
--nodes 1000 \
--samples 10 \
--output benchmarks/milestone3-2026-07-29.json
```
The synthetic generic fixture contains 1,000 manual pages, 1,000 portable-graph nodes, and 999
edges. Durations use `time.perf_counter_ns()` and nearest-rank p95. In-process peak memory uses
`tracemalloc`; detached worker peak memory comes from the worker receipt and `RUSAGE_SELF`.
Every measured result is checked for deterministic semantic identity and bounded response size.
Environment:
- Linux 7.1.3-200.nobara.fc44.x86_64.
- CPython 3.14.6.
- x86_64.
- Ten samples except the one-time production cold render and fragment-cache population.
- In-process and detached-worker memory ceiling: 268,435,456 bytes.
- Detached artifact-transfer ceiling: 20,000,000 bytes.
The complete machine-readable result is
[`benchmarks/milestone3-2026-07-29.json`](../benchmarks/milestone3-2026-07-29.json).
## Results
| Operation | Median | p95 | Limit | Maximum response |
|---|---:|---:|---:|---:|
| Manual full plan/package/render | 801.948 ms | 810.490 ms | 15,000 ms | 664 B |
| Manual detached worker | 591.317 ms | 599.394 ms | 20,000 ms | 667 B |
| Fragment-assisted equivalence | 638.999 ms | 666.119 ms | 15,000 ms | 664 B |
| Production incremental cold | 2,827.152 ms | 2,827.152 ms | 20,000 ms | 668 B |
| Production incremental warm | 2,143.388 ms | 2,206.540 ms | 20,000 ms | 669 B |
| Production forced full | 944.135 ms | 978.870 ms | 20,000 ms | 668 B |
| Fragment cache miss sweep | 106.203 ms | 106.888 ms | 5,000 ms | 145 B |
| Fragment cache hit sweep | 498.911 ms | 514.324 ms | 5,000 ms | 145 B |
| Portable graph full plan/package/render | 303.736 ms | 323.690 ms | 10,000 ms | 657 B |
| Portable graph detached worker | 265.448 ms | 268.428 ms | 20,000 ms | 660 B |
| Manual receipt-only status | 62.881 ms | 111.381 ms | 500 ms | 1,542 B |
| Portable graph receipt-only status | 58.467 ms | 59.331 ms | 500 ms | 879 B |
The largest traced in-process peak was 35,160,716 bytes. The direct manual worker track peaked at
88,580,096 bytes and the portable graph worker at 89,583,616 bytes. The production manual paths,
including cold, warm, forced-full, and mutation variants, peaked at 104,771,584 bytes. Every child
peak was validated from its projection receipt against the 268,435,456-byte gate.
The manual artifact was 583,149 bytes. The portable graph artifact was 718,383 bytes. The manual
plan was 1,006,393 bytes and its ordinary package was 1,007,297 bytes. The graph plan was 398,158
bytes and its package was 398,715 bytes.
## Equivalence and no-work gates
The benchmark proved exact output equivalence for:
- Manual in-process and detached rendering.
- Manual full and fragment-assisted rendering.
- Production cold, warm, and forced-full rendering.
- Production add, change, delete, and reorder variants.
- Portable graph in-process and detached rendering.
Manual and portable-graph status each performed zero project loads, source parses, adapter
projection loads, adapter extraction, index checks, synchronization, index builds, render
preparation, output construction, output hashing, and viewer-manager requests. Each status path
performed only two cheap source-generation checks and verified committed receipt or manifest
evidence.
## Measured limits and future notes
- Fragment reuse is a correctness, isolation, and recovery boundary in this milestone. At 1,000
pages, production warm fragment validation is slower than the forced-full path. Later
optimization must start from this measurement and preserve byte equivalence.
- Full rendering remains the oracle and recovery path. Invalid, corrupt, oversized, stale, or
mismatched fragment records fall back without changing canonical facts.
- The benchmark main-process `ru_maxrss` value was 102,692 KiB. It is cumulative across all
main-process operations and is recorded only as diagnostic context. Detached child peaks are
measured separately. Per-operation traced peaks and every detached receipt peak own the memory
gates.
- The results do not justify a storage rewrite, render farm, remote renderer, or separate render
MCP.

89
docs/MILESTONE_3_CLOSEOUT.md
View file

@ -1,89 +0,0 @@
# Milestone 3 closeout
## Outcome
Milestone 3 is complete. Manual compilation, portable graph rendering, and the live viewer are
separate generation-pinned consumers of the validated graph. They cannot become canonical or
retrieval authority.
Implemented contracts:
- Version-1 `ManualRenderPlan`, `GraphViewPlan`, projection package, and projection receipt.
- Strict canonical JSON identities and packaged Draft 2020-12 schemas.
- Independent manual and portable-graph renderer import boundaries.
- One isolated, fixed, one-request detached worker protocol with bounded request, response,
artifact, timeout, environment, and renderer inventory.
- Content-addressed portable graph artifacts, renderer receipts, generation/view manifests,
receipt-only status, repair, and degraded committed-publication evidence.
- Disposable semantic fragment records with bounded cache inventory, corruption recovery, and
full-render equivalence.
- Version-2 independent projection policy while preserving version-1 effective-policy behavior.
- Generation-pinned live source reads and a separate read-only viewer-manager lifecycle.
- Automated axe-tag and keyboard gates for the manual, portable graph, and live viewer.
- Repository-native contract, smoke, scale, response-size, memory, and equivalence gates.
## Candidate evidence
The frozen implementation candidate is
`f5dccb5e1c312121f1af63780162f593d9363b98`.
The complete repository gate passed:
- Ruff formatting and lint.
- HTML, rendered-manual HTML, portable-graph HTML, CSS, and JavaScript checks.
- Pyright with zero diagnostics.
- Warning-strict compilation and tests.
- 281 tests and 272 subtests.
- Three Playwright and axe accessibility flows. The alpha manual is checked with WCAG 2.0/2.1
A/AA axe tags; portable and live graph flows add WCAG 2.2 A/AA tags and keyboard interaction.
- Lock and npm dependency-tree checks.
- Wheel and source-distribution builds.
- Milestone 0, 1, 2, and 3 smoke benchmarks.
The maintained projection contract subset passed 142 tests and 236 subtests. A 10,000-node deep
chain and one 10,000-node strongly connected component prove that manual cycle planning has no
recursion-depth failure.
An isolated wheel installation passed CLI and MCP startup, a real detached manual render, and the
closed malformed-worker-request contract. Six Milestone 3 commits and the complete candidate tree
passed Gitleaks 8.30.1 with no findings.
Three independent adversarial review tracks covered manual isolation and fragment integrity,
portable publication and policy binding, and worker/accessibility/benchmark gates. Reproduced
project import, hostile environment, unbounded stdout, fragment forgery, cache growth, aggregate
overflow, coordinated policy drift, render-limit compatibility, deep-graph, and module-startup
defects were fixed and regression-tested before closeout.
The clean ten-sample 1,000-node benchmark passed every threshold. Exact measurements, equivalence
results, memory peaks, and response sizes are recorded in
[`MILESTONE_3_BASELINE.md`](MILESTONE_3_BASELINE.md) and
[`benchmarks/milestone3-2026-07-29.json`](../benchmarks/milestone3-2026-07-29.json).
## Preserved boundaries
- The `docforge` distribution, package, CLI, MCP executable, and existing tool names remain.
- The frozen alpha manual remains exactly 2,043 bytes with its legacy output hash and render
identity.
- Legacy one-method `load_projection()` adapters remain supported.
- Effective policy version 1, no-AST behavior, and existing client bindings remain compatible.
- Project descriptor schema version 1 and SQLite index schema version 3 remain unchanged.
- Configured `max_render_bytes` values above the detached transfer ceiling still load; a small
actual artifact renders normally. Actual detached transfer remains capped at 20,000,000 bytes.
- No storage replacement or self-hosting dependency was introduced.
- WorldForge and ScrapeStation were not touched.
- No production MCP integration was repointed.
- The legacy Forgejo repository and `legacy` remote were not changed.
- No tag, release, release announcement, or visibility change was created.
## Known follow-up work
Milestone 4 remains directional and is not active. Its adapter SDK and product-documentation work
must not silently absorb these separate future ideas:
- Optimize production fragment reuse only from measured profiles while preserving the forced-full
oracle.
- Add authenticated cursors only if a stronger threat model requires them.
- Verify Claude's native timeout representation.
- Add versioned adapter-owned launcher metadata before generating custom-adapter configurations.
- Keep remote render services, shared render farms, third-party renderers, storage replacement,
and self-hosting deferred until their own evidence justifies them.

158
docs/USER_MANUAL.md
View file

@ -27,10 +27,6 @@ incremental methods while retaining the full loader as a fallback.
- Opt-in incremental adapter extraction with reverse-dependency invalidation.
- Lazy function-scoped logic projections that do not densify the primary graph.
- Declared HTML render views. Arbitrary templates, render commands, and output paths are rejected.
- Separate versioned manual and portable graph plans, immutable packages, detached built-in
renderers, and validated receipts.
- Content-addressed portable Nodes/Flow/Web artifacts with receipt-only status and repair.
- Independent manual, portable-graph, and live-viewer policy.
- A loopback-only graph browser with Nodes, semantic Flow, and convergence Web views,
relationship keys, source inspection, branch-aware node hiding, panel resizing, zooming, and
managed idle shutdown.
@ -51,11 +47,7 @@ Canonical files own facts:
canonical Markdown/TOML or adapter sources
↓ validate
disposable SQLite graph
├── query / compile context
├── ManualRenderPlanV1 → detached manual renderer → declared manual
├── GraphViewPlanV1 → detached graph renderer → portable Nodes/Flow/Web artifact
└── pinned index → managed live Nodes/Flow/Web/Logic viewer
↓
↓ query / visualize / compile context
people and agents
↓ propose
isolated changeset + preview
@ -85,8 +77,8 @@ source format.
Clone and verify DocForge:
```bash
git clone forgejo@repo.andraxion.net:administrator/DocForge2.git /absolute/path/DocForge2
cd /absolute/path/DocForge2
git clone forgejo@repo.andraxion.net:administrator/DocForge.git /absolute/path/DocForge
cd /absolute/path/DocForge
uv sync --group dev
npm ci
@ -167,27 +159,6 @@ output = "Docs/Rendered/Manual.html"
title = "My Project Manual"
families = ["architecture", "system", "operations", "roadmap"]
[graph_render]
output_root = ".docforge/portable-graph"
[[graph_render.views]]
id = "architecture"
renderer = "portable_graph_html"
output = "architecture.html"
title = "Architecture"
root = "architecture.overview"
initial_mode = "web"
depth = 3
max_nodes = 250
max_edges = 1000
max_work = 100000
families = ["architecture", "system"]
relations = ["depends_on", "owns", "calls", "reads", "writes", "tested_by", "relates_to"]
authorities = []
statuses = ["current", "active", "verified"]
tags = []
include_logic = false
[graph]
allowed_relations = ["depends_on", "owns", "calls", "reads", "writes", "tested_by", "relates_to"]
@ -521,25 +492,13 @@ builds or repairs the index.
```text
render-status [VIEW_ID] [--deep]
render VIEW_ID
graph-plan VIEW_ID
graph-render VIEW_ID
graph-render-status [VIEW_ID]
preview CHANGESET_ID VIEW_ID
apply CHANGESET_ID --changeset-hash SHA256 --applier WRITER_ID
```
`graph-plan` validates and returns one declared `GraphViewPlanV1` without publishing. A portable
view must select exactly one stable `root` or metadata-only lexical `query`. It may use only Nodes,
Flow, or Web as `initial_mode`; portable version 1 excludes function-scoped Logic.
`graph-render` explicitly publishes the declared static artifact, content-addressed renderer
evidence, and generation/view manifest. `graph-render-status` verifies only bounded committed
evidence and never plans or renders. Portable publication is a local CLI action.
The CLI apply command supports the generic adapter. It verifies that the configured writer owns the
changeset, applies the exact reviewed hash, rebuilds the index, checks it, and regenerates every
declared manual render when manual policy is `auto`. It does not publish portable graphs, commit,
or push the result.
declared render. It does not commit or push the result.
### Viewer commands
@ -590,20 +549,6 @@ declares the named writer, and application requires the same writer/applier iden
`--no-ast` to preserve the no-AST binding. Generic CLI generation refuses project-owned adapters
because it cannot safely reconstruct their composition.
Select projection behavior independently:
```bash
docforge configure codex \
--project /absolute/path/MyProject \
--manual-render-policy explicit \
--portable-graph-policy disabled \
--live-viewer-policy on-demand
```
The generated version-1 configuration result carries an additive version-2 `projection_policy`,
its hash, projection availability, and the exact descriptor hash. Omitted default selectors are
validated against that descriptor rather than trusted as self-reported output.
Inspect one configured client binding:
```bash
@ -624,54 +569,6 @@ starts MCP, executes the configured command, synchronizes, builds, renders, star
writes configuration. Claude timeout representation and client filtering that cannot be proved
locally remain explicit warnings.
## Independent projection behavior
Manual and portable graph renderers consume immutable, path-free packages. A package binds one
generation-pinned plan, inert assets, fixed component versions, a built-in renderer identity, and
an exact artifact inventory. The detached child cannot select nodes, open the project or index,
choose a publication path, execute project code, or mutate canonical facts.
Child startup is fixed to isolated Python, a private module entrypoint, a trusted working
directory, and a sanitized environment. One request and response use canonical newline-terminated
JSON. The request, response, receipt, execution time, and disk-spooled stdout are bounded. Actual
artifact transfer is capped at 20,000,000 bytes even when the descriptor retains a larger
`max_render_bytes` compatibility value.
Manual fragment records are disposable semantic cache entries. On a cold miss, DocForge performs a
trusted full detached render, extracts candidate page fragments, and compares fragment-assisted
output byte-for-byte before publishing records. On a warm hit, the worker recomputes each expected
page fragment before accepting cached bytes. Corrupt, forged, stale, individually oversized, or
aggregate-oversized records fall back to the full oracle. Fragment reuse is currently a correctness
and recovery boundary, not a promised speedup.
Projection policy version 2 is:
```text
manual: auto | explicit | disabled
portable_graph: explicit | disabled
live_viewer: on-demand | disabled
```
For ordinary CLI commands, place the corresponding global flag before the subcommand:
```bash
docforge --project-root "$PROJECT" --manual-render-policy disabled render manual
docforge --project-root "$PROJECT" --portable-graph-policy disabled graph-plan architecture
docforge --project-root "$PROJECT" --live-viewer-policy disabled visualize
```
An active operation blocked by policy returns `projection_policy_forbids_operation` before hidden
work. Manual and portable receipt-only status remain available. Viewer status and explicit stop
remain available when viewer start is disabled.
A non-disabled projection also requires its declared configuration or runtime. Manual `explicit`
requires manual render configuration. Manual `auto` additionally requires canonical application in
the current operation or server capability. Portable graph `explicit` requires portable graph
render configuration, and live viewer `on-demand` requires its runtime. An unavailable selection
returns `projection_policy_unavailable` before work begins. In particular, ordinary CLI `render`
operations cannot select manual `auto`; use `explicit`, or let a configured canonical `apply`
operation own automatic regeneration.
## MCP usage
Run one MCP server per project with absolute paths:
@ -689,10 +586,7 @@ Select the session's declared surface explicitly when useful:
```bash
docforge-mcp \
--project-root /absolute/path/MyProject \
--capability-mode read \
--manual-render-policy explicit \
--portable-graph-policy explicit \
--live-viewer-policy on-demand
--capability-mode read
```
Supported modes are `read`, `proposal`, `application`, and `operator`. Existing startup defaults
@ -719,9 +613,8 @@ identity, not a command. The changeset creator, configured writer, and canonical
Call `docforge_bootstrap` first. Its version-1 `session_contract` contains the fixed binding,
current graph generation, effective policy, actual capabilities, render policies, prohibitions,
and a recommended first operation. The result also carries the independently composed version-2
`projection_policy` and hash. Workflow guidance does not recommend registration or application
when those startup capabilities are unavailable.
and a recommended first operation. Workflow guidance does not recommend registration or
application when those startup capabilities are unavailable.
Example MCP client configuration:
@ -760,16 +653,11 @@ Example MCP client configuration:
- `docforge_get_task_context`
- `docforge_validate_project`
- `docforge_render_status`
- `docforge_graph_plan`
- `docforge_graph_render_status`
- `docforge_visualize`
- `docforge_visualization_status`
- `docforge_stop_visualization`
- `docforge_get_generation_diff`
MCP graph plan and status are read-only. MCP does not expose portable graph publication; use the
explicit local `graph-render` CLI command.
### Proposal tools
- `docforge_create_changeset`
@ -1088,29 +976,8 @@ docforge --project-root "$PROJECT" render-status
docforge --project-root "$PROJECT" render VIEW_ID
```
Successful canonical apply regenerates declared manual views only when manual policy is `auto`.
A manual canonical edit requires reindexing and explicit rendering. Portable graph publication
always remains a separate explicit CLI action.
Portable graph publication has separate status and policy:
```bash
docforge --project-root "$PROJECT" graph-render-status
docforge --project-root "$PROJECT" graph-render architecture
```
### `projection_policy_forbids_operation`
The process was deliberately started with the relevant manual, portable-graph, or live-viewer
operation disabled. Restart with an allowed selector after confirming that the integration should
receive that capability. Status and explicit stop operations remain available as described above.
### `projection_policy_unavailable`
The selected non-disabled projection has no matching project configuration or runtime. Add the
declared manual or portable graph render configuration, or make the live viewer runtime available,
before selecting that mode. Manual `auto` also requires an operation or MCP server with canonical
application enabled. Use manual `explicit` for a standalone CLI render.
Successful canonical apply regenerates all declared views automatically. A manual canonical edit
requires reindexing and rendering.
### Descriptor changed after startup
@ -1125,11 +992,8 @@ Run the complete release gate from the DocForge repository:
make gate
```
Use `make benchmark` for the historical Milestone 0 baseline, `make benchmark-m1` for the
counter-gated warm-operation benchmark, `make benchmark-m2` for agent workflow gates, and
`make benchmark-m3-full` for the ten-sample 1,000-node projection, worker, fragment, status,
equivalence, response-size, and memory gates. `make accessibility` runs the generated manual,
portable graph, and live viewer axe and keyboard flows.
Use `make benchmark` for the historical Milestone 0 baseline and `make benchmark-m1` for the
counter-gated 1,000-node warm-operation benchmark.
Project-specific vocabulary, extraction rules, and serialization belong in the project adapter.
Generic core behavior must remain deterministic, project-bound, and recoverable.

22
eslint.config.mjs
View file

@ -21,26 +21,4 @@ export default [
"prefer-const": "error",
},
},
{
files: ["playwright.accessibility.config.mjs", "tests/accessibility.spec.mjs"],
...js.configs.recommended,
languageOptions: {
ecmaVersion: 2024,
sourceType: "module",
globals: {
...globals.browser,
...globals.node,
},
},
linterOptions: {
reportUnusedDisableDirectives: "error",
},
rules: {
...js.configs.recommended.rules,
eqeqeq: "error",
"no-implicit-coercion": "error",
"no-var": "error",
"prefer-const": "error",
},
},
];

88
package-lock.json generated
View file

@ -8,9 +8,7 @@
"name": "docforge-web-quality",
"version": "0.0.0",
"devDependencies": {
"@axe-core/playwright": "4.12.1",
"@eslint/js": "10.0.1",
"@playwright/test": "1.62.0",
"eslint": "10.8.0",
"globals": "17.7.0",
"html-validate": "11.5.6",
@ -20,19 +18,6 @@
"stylelint-csstree-validator": "4.0.0"
}
},
"node_modules/@axe-core/playwright": {
"version": "4.12.1",
"resolved": "https://registry.npmjs.org/@axe-core/playwright/-/playwright-4.12.1.tgz",
"integrity": "sha512-rMd7xriptqKpP+w5265i4Hdkv2X5kbu6uiBi/B2I7uf3hieRBM3qDCfaKPtxfiYb2mKXfF+yLODJwIx+Jv1GDw==",
"dev": true,
"license": "MPL-2.0",
"dependencies": {
"axe-core": "~4.12.1"
},
"peerDependencies": {
"playwright-core": ">= 1.0.0"
}
},
"node_modules/@babel/code-frame": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.29.7.tgz",
@ -530,22 +515,6 @@
"node": ">= 8"
}
},
"node_modules/@playwright/test": {
"version": "1.62.0",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.62.0.tgz",
"integrity": "sha512-9zOJ6ZQRAena31MpOH9VSzIz8Ou3YJ/wtY/eQm5T2uhfhG7/U3COrMS8xOtUrZrp9OgdmzEnIYODye3nY1VqzA==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.62.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/@sindresorhus/merge-streams": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/@sindresorhus/merge-streams/-/merge-streams-4.0.0.tgz",
@ -666,16 +635,6 @@
"node": ">=8"
}
},
"node_modules/axe-core": {
"version": "4.12.1",
"resolved": "https://registry.npmjs.org/axe-core/-/axe-core-4.12.1.tgz",
"integrity": "sha512-s7iGf5GaVMxEG0ENN9x+xTr7GFZCb1ZP/1uATUpCEK2X78nDB3RwbtFCo9pGAf9ru+VwoQ464DkaLEeRM08wJA==",
"dev": true,
"license": "MPL-2.0",
"engines": {
"node": ">=4"
}
},
"node_modules/balanced-match": {
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz",
@ -1984,53 +1943,6 @@
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/playwright": {
"version": "1.62.0",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.0.tgz",
"integrity": "sha512-Z14dG305dgaLu6foB1TXQagFiW8JfSUIUaUuPaKQ6NtBPKF1P/qXcqfh6c6K/icPqdy37JmjbiBXf6JNg6Sylw==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.62.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=20"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.62.0",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.0.tgz",
"integrity": "sha512-nsNRyq0r2zsG8AcRHWknc9QRA5XCueC7gWMrs+Gx2tlZn9hcl8zudfh00lhJPY1DE7NmZ6bDsT9g2yey8mXljA==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/playwright/node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/postcss": {
"version": "8.5.23",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.23.tgz",

6
package.json
View file

@ -4,14 +4,10 @@
"private": true,
"packageManager": "npm@10.9.7",
"scripts": {
"install:accessibility-browser": "playwright install chromium",
"lint:web": "uv run python tools/check_web_assets.py && eslint --max-warnings=0 playwright.accessibility.config.mjs tests/accessibility.spec.mjs",
"test:accessibility": "npm run install:accessibility-browser && playwright test --config=playwright.accessibility.config.mjs"
"lint:web": "uv run python tools/check_web_assets.py"
},
"devDependencies": {
"@axe-core/playwright": "4.12.1",
"@eslint/js": "10.0.1",
"@playwright/test": "1.62.0",
"eslint": "10.8.0",
"globals": "17.7.0",
"html-validate": "11.5.6",

24
playwright.accessibility.config.mjs
View file

@ -1,24 +0,0 @@
import { defineConfig } from "@playwright/test";
export default defineConfig({
testDir: "./tests",
testMatch: "accessibility.spec.mjs",
fullyParallel: false,
workers: 1,
retries: 0,
reporter: "line",
outputDir: "/tmp/docforge-playwright-accessibility",
timeout: 30_000,
expect: {
timeout: 5_000,
},
use: {
browserName: "chromium",
bypassCSP: true,
headless: true,
viewport: {
width: 1440,
height: 1000,
},
},
});

5
pyproject.toml
View file

@ -31,10 +31,7 @@ docforge-mcp = "docforge.mcp_server:main"
docforge-viewer-manager = "docforge.viewer_manager:main"
[tool.hatch.build.targets.wheel]
packages = ["src/docforge", "src/docforge_renderers"]
[tool.hatch.build.targets.wheel.force-include]
schemas = "docforge/schemas"
packages = ["src/docforge"]
[tool.ruff]
line-length = 100

38
schemas/client-configuration.schema.json
View file

@ -127,17 +127,6 @@
},
"additionalProperties": false
},
"projection_policy": {
"type": "object",
"required": ["schema_version", "manual", "portable_graph", "live_viewer"],
"properties": {
"schema_version": { "const": 2 },
"manual": { "enum": ["auto", "explicit", "disabled"] },
"portable_graph": { "enum": ["explicit", "disabled"] },
"live_viewer": { "enum": ["on-demand", "disabled"] }
},
"additionalProperties": false
},
"diagnostics": {
"type": "object",
"required": [
@ -222,9 +211,6 @@
"project",
"binding",
"effective_policy",
"projection_policy",
"projection_policy_hash",
"projection_availability",
"artifact",
"configuration_hash",
"warnings"
@ -245,8 +231,7 @@
"project_id",
"project_root",
"project_root_fingerprint",
"adapter",
"descriptor_hash"
"adapter"
],
"properties": {
"project_id": { "type": "string", "minLength": 1 },
@ -255,8 +240,7 @@
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1 },
"descriptor_hash": { "$ref": "#/$defs/sha256" }
"adapter": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
@ -333,24 +317,6 @@
"additionalProperties": false
},
"effective_policy": { "$ref": "#/$defs/effective_policy" },
"projection_policy": { "$ref": "#/$defs/projection_policy" },
"projection_policy_hash": { "$ref": "#/$defs/sha256" },
"projection_availability": {
"type": "object",
"required": [
"manual_configured",
"portable_graph_configured",
"application_enabled",
"live_viewer_available"
],
"properties": {
"manual_configured": { "type": "boolean" },
"portable_graph_configured": { "type": "boolean" },
"application_enabled": { "type": "boolean" },
"live_viewer_available": { "const": true }
},
"additionalProperties": false
},
"artifact": {
"type": "object",
"required": [

321
schemas/graph-view-plan.schema.json
View file

@ -1,321 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/graph-view-plan-v1.json",
"title": "DocForge immutable portable graph view plan",
"$comment": "plan_id is the SHA-256 of canonical JSON without plan_id and is verified by the runtime contract validator. The version-1 nested graph vocabulary remains planner-owned.",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"project": {
"type": "object",
"required": [
"project_id",
"project_root_fingerprint",
"adapter",
"revision",
"source_hash"
],
"properties": {
"project_id": { "type": "string", "minLength": 1 },
"project_root_fingerprint": {
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1 },
"revision": { "type": "string", "minLength": 1 },
"source_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"filter": {
"type": "array",
"maxItems": 64,
"items": {
"type": "string",
"minLength": 1,
"maxLength": 1024
},
"uniqueItems": true
},
"edge": {
"type": "object",
"required": ["source_id", "relation", "target_id"],
"properties": {
"source_id": { "type": "string", "minLength": 1 },
"relation": { "type": "string", "minLength": 1 },
"target_id": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
"node": {
"type": "object",
"required": [
"node_id",
"title",
"family",
"authority",
"status",
"tags",
"summary",
"content_hash"
],
"properties": {
"node_id": { "type": "string", "minLength": 1 },
"title": { "type": "string", "minLength": 1 },
"family": { "type": "string", "minLength": 1 },
"authority": { "type": "string", "minLength": 1 },
"status": { "type": "string", "minLength": 1 },
"tags": { "$ref": "#/$defs/filter" },
"summary": { "type": "string" },
"content_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"omission": {
"oneOf": [
{
"type": "object",
"required": ["code", "subject", "limit", "minimum_omitted"],
"properties": {
"code": { "const": "node_result_limit" },
"subject": { "const": "nodes" },
"limit": { "type": "integer", "minimum": 1, "maximum": 1000 },
"minimum_omitted": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
{
"type": "object",
"required": ["code", "subject", "limit", "minimum_omitted"],
"properties": {
"code": { "const": "edge_result_limit" },
"subject": { "const": "edges" },
"limit": { "type": "integer", "minimum": 0, "maximum": 4000 },
"minimum_omitted": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"code",
"subject",
"limit",
"examined",
"minimum_omitted"
],
"properties": {
"code": { "const": "work_limit" },
"subject": { "const": "selection" },
"limit": { "type": "integer", "minimum": 1, "maximum": 1000000 },
"examined": { "type": "integer", "minimum": 0 },
"minimum_omitted": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
{
"type": "object",
"required": ["code", "subject", "minimum_omitted"],
"properties": {
"code": { "const": "logic_forbidden" },
"subject": { "const": "logic" },
"minimum_omitted": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
}
]
}
},
"type": "object",
"required": [
"schema_version",
"contract",
"plan_id",
"project",
"view",
"bounds",
"policy",
"graph",
"omissions",
"diagnostics"
],
"properties": {
"schema_version": { "const": 1 },
"contract": { "const": "docforge.graph-view-plan" },
"plan_id": { "$ref": "#/$defs/sha256" },
"project": { "$ref": "#/$defs/project" },
"view": {
"type": "object",
"required": [
"view_id",
"title",
"initial_mode",
"scope",
"filters",
"detail_fields"
],
"properties": {
"view_id": {
"type": "string",
"minLength": 1,
"maxLength": 1024
},
"title": {
"type": "string",
"minLength": 1,
"maxLength": 1024
},
"initial_mode": { "enum": ["nodes", "flow", "web", "logic"] },
"scope": {
"oneOf": [
{
"type": "object",
"required": ["kind", "root_node_id", "depth"],
"properties": {
"kind": { "const": "exact_root" },
"root_node_id": {
"type": "string",
"minLength": 1,
"maxLength": 1024
},
"depth": { "type": "integer", "minimum": 1, "maximum": 32 }
},
"additionalProperties": false
},
{
"type": "object",
"required": ["kind", "query"],
"properties": {
"kind": { "const": "lexical" },
"query": {
"type": "string",
"minLength": 1,
"maxLength": 10000
}
},
"additionalProperties": false
}
]
},
"filters": {
"type": "object",
"required": [
"families",
"relations",
"authorities",
"statuses",
"tags"
],
"properties": {
"families": { "$ref": "#/$defs/filter" },
"relations": { "$ref": "#/$defs/filter" },
"authorities": { "$ref": "#/$defs/filter" },
"statuses": { "$ref": "#/$defs/filter" },
"tags": { "$ref": "#/$defs/filter" }
},
"additionalProperties": false
},
"detail_fields": {
"const": [
"node_id",
"title",
"family",
"authority",
"status",
"tags",
"summary",
"content_hash"
]
}
},
"additionalProperties": false
},
"bounds": {
"type": "object",
"required": ["depth", "max_nodes", "max_edges", "max_work"],
"properties": {
"depth": { "type": "integer", "minimum": 1, "maximum": 32 },
"max_nodes": { "type": "integer", "minimum": 1, "maximum": 1000 },
"max_edges": { "type": "integer", "minimum": 0, "maximum": 4000 },
"max_work": { "type": "integer", "minimum": 1, "maximum": 1000000 }
},
"additionalProperties": false
},
"policy": {
"type": "object",
"required": [
"visibility",
"source_paths",
"source_bodies",
"database_queries",
"executable_content",
"logic",
"logic_requested"
],
"properties": {
"visibility": { "const": "selected_graph_only" },
"source_paths": { "const": "excluded" },
"source_bodies": { "const": "excluded" },
"database_queries": { "const": "forbidden" },
"executable_content": { "const": "forbidden" },
"logic": { "enum": ["allowed", "forbidden"] },
"logic_requested": { "type": "boolean" }
},
"additionalProperties": false
},
"graph": {
"type": "object",
"required": ["root_node_id", "nodes", "edges", "logic_projections"],
"properties": {
"root_node_id": {
"type": ["string", "null"],
"minLength": 1,
"maxLength": 1024
},
"nodes": {
"type": "array",
"maxItems": 1000,
"items": { "$ref": "#/$defs/node" }
},
"edges": {
"type": "array",
"maxItems": 4000,
"items": { "$ref": "#/$defs/edge" }
},
"logic_projections": {
"type": "array",
"maxItems": 0
}
},
"additionalProperties": false
},
"omissions": {
"type": "array",
"maxItems": 4,
"items": { "$ref": "#/$defs/omission" }
},
"diagnostics": {
"type": "object",
"required": [
"selection",
"returned_nodes",
"returned_edges",
"examined_work_units",
"truncated",
"ordering"
],
"properties": {
"selection": { "enum": ["exact_root", "lexical"] },
"returned_nodes": { "type": "integer", "minimum": 0, "maximum": 1000 },
"returned_edges": { "type": "integer", "minimum": 0, "maximum": 4000 },
"examined_work_units": { "type": "integer", "minimum": 0 },
"truncated": { "type": "boolean" },
"ordering": { "const": "node_id;source_id,relation,target_id" }
},
"additionalProperties": false
}
},
"additionalProperties": false
}

214
schemas/manual-render-plan.schema.json
View file

@ -1,214 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/manual-render-plan-v1.json",
"title": "DocForge immutable manual render plan",
"$comment": "plan_id is the SHA-256 of canonical JSON without plan_id and is verified by the runtime contract validator.",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"project": {
"type": "object",
"required": [
"project_id",
"project_root_fingerprint",
"adapter",
"revision",
"source_hash"
],
"properties": {
"project_id": { "type": "string", "minLength": 1 },
"project_root_fingerprint": {
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1 },
"revision": { "type": "string", "minLength": 1 },
"source_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"edge": {
"type": "object",
"required": ["source_id", "relation", "target_id"],
"properties": {
"source_id": { "type": "string", "minLength": 1 },
"relation": { "type": "string", "minLength": 1 },
"target_id": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
"page": {
"type": "object",
"required": [
"node_id",
"title",
"family",
"authority",
"status",
"tags",
"summary",
"content",
"content_hash",
"components",
"breadcrumbs",
"cross_references",
"backlinks"
],
"properties": {
"node_id": { "type": "string", "minLength": 1 },
"title": { "type": "string", "minLength": 1 },
"family": { "type": "string", "minLength": 1 },
"authority": { "type": "string", "minLength": 1 },
"status": { "type": "string", "minLength": 1 },
"tags": {
"type": "array",
"maxItems": 10000,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
},
"summary": { "type": "string" },
"content": { "type": "string" },
"content_hash": { "$ref": "#/$defs/sha256" },
"components": {
"type": "array",
"maxItems": 32,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
},
"breadcrumbs": {
"type": "array",
"maxItems": 10000,
"items": { "type": "string", "minLength": 1 }
},
"cross_references": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/edge" }
},
"backlinks": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/edge" }
}
},
"additionalProperties": false
},
"navigation_item": {
"type": "object",
"required": ["node_id", "title"],
"properties": {
"node_id": { "type": "string", "minLength": 1 },
"title": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
"search_document": {
"type": "object",
"required": [
"node_id",
"title",
"summary",
"family",
"status",
"tags"
],
"properties": {
"node_id": { "type": "string", "minLength": 1 },
"title": { "type": "string", "minLength": 1 },
"summary": { "type": "string" },
"family": { "type": "string", "minLength": 1 },
"status": { "type": "string", "minLength": 1 },
"tags": {
"type": "array",
"maxItems": 10000,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
}
},
"additionalProperties": false
}
},
"type": "object",
"required": [
"schema_version",
"contract",
"plan_id",
"project",
"view",
"changeset_hash",
"pages",
"navigation",
"search_documents",
"diagnostics"
],
"properties": {
"schema_version": { "const": 1 },
"contract": { "const": "docforge.manual-render-plan" },
"plan_id": { "$ref": "#/$defs/sha256" },
"project": { "$ref": "#/$defs/project" },
"view": {
"type": "object",
"required": ["view_id", "title", "families", "renderer"],
"properties": {
"view_id": { "type": "string", "minLength": 1 },
"title": { "type": "string", "minLength": 1 },
"families": {
"type": "array",
"maxItems": 10000,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
},
"renderer": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
"changeset_hash": {
"oneOf": [
{ "$ref": "#/$defs/sha256" },
{ "type": "null" }
]
},
"pages": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/page" }
},
"navigation": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/navigation_item" }
},
"search_documents": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/search_document" }
},
"diagnostics": {
"type": "object",
"required": ["orphans", "cycles"],
"properties": {
"orphans": {
"type": "array",
"maxItems": 10000,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
},
"cycles": {
"type": "array",
"maxItems": 10000,
"items": {
"type": "array",
"minItems": 1,
"maxItems": 10000,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
}
}
},
"additionalProperties": false
}
},
"additionalProperties": false
}

76
schemas/project.schema.json
View file

@ -74,82 +74,6 @@
},
"additionalProperties": false
},
"graph_render": {
"type": "object",
"required": ["output_root", "views"],
"properties": {
"output_root": { "$ref": "#/$defs/relativePath" },
"views": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"required": ["id", "renderer", "output", "title"],
"properties": {
"id": { "type": "string", "pattern": "^[a-z0-9][a-z0-9._-]{1,127}$" },
"renderer": { "const": "portable_graph_html" },
"output": {
"allOf": [
{ "$ref": "#/$defs/relativePath" },
{ "pattern": "\\.html$" }
]
},
"title": { "type": "string", "minLength": 1, "maxLength": 1024 },
"root": { "type": "string", "minLength": 1, "maxLength": 1024 },
"query": { "type": "string", "minLength": 1, "maxLength": 10000 },
"initial_mode": { "enum": ["nodes", "flow", "web"] },
"depth": { "type": "integer", "minimum": 1, "maximum": 32 },
"max_nodes": { "type": "integer", "minimum": 1, "maximum": 1000 },
"max_edges": { "type": "integer", "minimum": 0, "maximum": 4000 },
"max_work": { "type": "integer", "minimum": 1, "maximum": 1000000 },
"families": {
"type": "array",
"maxItems": 64,
"uniqueItems": true,
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
},
"relations": {
"type": "array",
"maxItems": 64,
"uniqueItems": true,
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
},
"authorities": {
"type": "array",
"maxItems": 64,
"uniqueItems": true,
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
},
"statuses": {
"type": "array",
"maxItems": 64,
"uniqueItems": true,
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
},
"tags": {
"type": "array",
"maxItems": 64,
"uniqueItems": true,
"items": { "type": "string", "minLength": 1, "maxLength": 1024 }
},
"include_logic": { "const": false }
},
"oneOf": [
{
"required": ["root"],
"not": { "required": ["query"] }
},
{
"required": ["query"],
"not": { "required": ["root"] }
}
],
"additionalProperties": false
}
}
},
"additionalProperties": false
},
"graph": {
"type": "object",
"required": ["allowed_relations"],

190
schemas/projection-package.schema.json
View file

@ -1,190 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/projection-package-v1.json",
"title": "DocForge immutable renderer projection package",
"$comment": "package_id and embedded plan identity equality are verified by the runtime contract validator.",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"manual_plan": {
"type": "object",
"required": [
"schema_version",
"contract",
"plan_id",
"project",
"view",
"changeset_hash",
"pages",
"navigation",
"search_documents",
"diagnostics"
],
"properties": {
"schema_version": { "const": 1 },
"contract": { "const": "docforge.manual-render-plan" },
"plan_id": { "$ref": "#/$defs/sha256" },
"project": { "type": "object" },
"view": { "type": "object" },
"changeset_hash": {
"oneOf": [
{ "$ref": "#/$defs/sha256" },
{ "type": "null" }
]
},
"pages": { "type": "array", "maxItems": 10000 },
"navigation": { "type": "array", "maxItems": 10000 },
"search_documents": { "type": "array", "maxItems": 10000 },
"diagnostics": { "type": "object" }
},
"additionalProperties": false
},
"graph_plan": {
"type": "object",
"required": [
"schema_version",
"contract",
"plan_id",
"project",
"view",
"bounds",
"policy",
"graph",
"omissions",
"diagnostics"
],
"properties": {
"schema_version": { "const": 1 },
"contract": { "const": "docforge.graph-view-plan" },
"plan_id": { "$ref": "#/$defs/sha256" },
"project": { "type": "object" },
"view": { "type": "object" },
"bounds": { "type": "object" },
"policy": { "type": "object" },
"graph": { "type": "object" },
"omissions": { "type": "array", "maxItems": 10000 },
"diagnostics": { "type": "object" }
},
"additionalProperties": false
},
"renderer": {
"type": "object",
"required": ["renderer_id", "renderer_version"],
"properties": {
"renderer_id": { "type": "string", "minLength": 1 },
"renderer_version": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
"component": {
"type": "object",
"required": ["component_id"],
"properties": {
"component_id": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
"asset": {
"type": "object",
"required": ["asset_id", "media_type", "sha256", "text"],
"properties": {
"asset_id": {
"type": "string",
"minLength": 1,
"pattern": "^[^/]+$"
},
"media_type": { "type": "string", "minLength": 1 },
"sha256": { "$ref": "#/$defs/sha256" },
"text": { "type": "string" }
},
"additionalProperties": false
}
},
"type": "object",
"required": [
"schema_version",
"contract",
"package_id",
"kind",
"plan_id",
"plan",
"renderer",
"components",
"assets",
"output_policy"
],
"properties": {
"schema_version": { "const": 1 },
"contract": { "const": "docforge.projection-package" },
"package_id": { "$ref": "#/$defs/sha256" },
"kind": { "enum": ["manual", "graph"] },
"plan_id": { "$ref": "#/$defs/sha256" },
"plan": {
"oneOf": [
{ "$ref": "#/$defs/manual_plan" },
{ "$ref": "#/$defs/graph_plan" }
]
},
"renderer": { "$ref": "#/$defs/renderer" },
"components": {
"type": "array",
"maxItems": 32,
"items": { "$ref": "#/$defs/component" },
"uniqueItems": true
},
"assets": {
"type": "array",
"maxItems": 32,
"items": { "$ref": "#/$defs/asset" }
},
"output_policy": {
"type": "object",
"required": ["artifact_ids", "max_total_bytes"],
"properties": {
"artifact_ids": {
"type": "array",
"minItems": 1,
"maxItems": 32,
"items": {
"type": "string",
"minLength": 1,
"pattern": "^[^/]+$"
},
"uniqueItems": true
},
"max_total_bytes": {
"type": "integer",
"minimum": 1
}
},
"additionalProperties": false
}
},
"allOf": [
{
"if": {
"properties": { "kind": { "const": "manual" } },
"required": ["kind"]
},
"then": {
"properties": {
"plan": { "$ref": "#/$defs/manual_plan" }
}
}
},
{
"if": {
"properties": { "kind": { "const": "graph" } },
"required": ["kind"]
},
"then": {
"properties": {
"plan": { "$ref": "#/$defs/graph_plan" }
}
}
}
],
"additionalProperties": false
}

19
schemas/projection-policy.schema.json
View file

@ -1,19 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/projection-policy-v2.json",
"title": "DocForge independent projection policy",
"type": "object",
"required": [
"schema_version",
"manual",
"portable_graph",
"live_viewer"
],
"properties": {
"schema_version": { "const": 2 },
"manual": { "enum": ["auto", "explicit", "disabled"] },
"portable_graph": { "enum": ["explicit", "disabled"] },
"live_viewer": { "enum": ["on-demand", "disabled"] }
},
"additionalProperties": false
}

89
schemas/projection-receipt.schema.json
View file

@ -1,89 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/projection-receipt-v1.json",
"title": "DocForge projection renderer receipt",
"$comment": "receipt_id is the SHA-256 of canonical JSON without receipt_id and is verified by the runtime contract validator.",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"renderer": {
"type": "object",
"required": ["renderer_id", "renderer_version"],
"properties": {
"renderer_id": { "type": "string", "minLength": 1 },
"renderer_version": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
"artifact": {
"type": "object",
"required": ["artifact_id", "media_type", "sha256", "bytes"],
"properties": {
"artifact_id": {
"type": "string",
"minLength": 1,
"pattern": "^[^/]+$"
},
"media_type": { "type": "string", "minLength": 1 },
"sha256": { "$ref": "#/$defs/sha256" },
"bytes": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
}
},
"type": "object",
"required": [
"schema_version",
"contract",
"receipt_id",
"kind",
"package_id",
"plan_id",
"renderer",
"artifacts",
"diagnostics",
"timing",
"peak_memory_bytes"
],
"properties": {
"schema_version": { "const": 1 },
"contract": { "const": "docforge.projection-receipt" },
"receipt_id": { "$ref": "#/$defs/sha256" },
"kind": { "enum": ["manual", "graph"] },
"package_id": { "$ref": "#/$defs/sha256" },
"plan_id": { "$ref": "#/$defs/sha256" },
"renderer": { "$ref": "#/$defs/renderer" },
"artifacts": {
"type": "array",
"maxItems": 32,
"items": { "$ref": "#/$defs/artifact" }
},
"diagnostics": {
"type": "object",
"required": ["warnings"],
"properties": {
"warnings": {
"type": "array",
"maxItems": 10000,
"items": { "type": "string" }
}
},
"additionalProperties": false
},
"timing": {
"type": "object",
"required": ["elapsed_ns"],
"properties": {
"elapsed_ns": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"peak_memory_bytes": {
"type": ["integer", "null"],
"minimum": 0
}
},
"additionalProperties": false
}

6
schemas/result.schema.json
View file

@ -20,7 +20,6 @@
"test",
"benchmark.m1",
"benchmark.m2",
"benchmark.m3",
"mcp.invoke",
"mcp.bootstrap",
"mcp.sync",
@ -38,8 +37,6 @@
"mcp.generation_diff",
"mcp.validate_project",
"mcp.render_status",
"mcp.graph_plan",
"mcp.graph_render_status",
"mcp.visualize",
"mcp.visualization_status",
"mcp.stop_visualization",
@ -61,9 +58,6 @@
"cli.impact",
"cli.context",
"cli.generation-diff",
"cli.graph-plan",
"cli.graph-render",
"cli.graph-render-status",
"cli.configure",
"cli.doctor",
"cli.render",

177
src/docforge/_fs_safety.py
View file

@ -3,10 +3,7 @@
from __future__ import annotations
import os
import secrets
import stat
from collections.abc import Callable
from contextlib import suppress
from pathlib import Path
from .errors import DocForgeError
@ -72,177 +69,3 @@ def require_bound_directory(path: Path, directory_fd: int) -> None:
"path_escape",
"Derived cache root disappeared during publication",
) from error
def open_confined_directory(root: Path, path: Path, *, create: bool) -> int:
"""Open a descendant directory through stable no-follow directory descriptors."""
try:
unsafe = (
root.is_symlink()
or root.resolve(strict=True) != root
or not path.is_relative_to(root)
or path == root
)
except OSError as error:
raise DocForgeError("path_escape", "Project root cannot be resolved safely") from error
if unsafe:
raise DocForgeError("path_escape", "Derived output directory is not confined")
relative = path.relative_to(root)
try:
descriptor = os.open(root, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
except OSError as error:
raise DocForgeError("path_escape", "Project root cannot be opened safely") from error
try:
for part in relative.parts:
if part in {"", ".", ".."}:
raise DocForgeError("path_escape", "Derived output directory is not confined")
if create:
try:
os.mkdir(part, mode=0o700, dir_fd=descriptor)
except FileExistsError:
pass
except OSError as error:
raise DocForgeError(
"publication_failure",
"Derived output directory could not be created",
) from error
try:
next_descriptor = os.open(
part,
os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW,
dir_fd=descriptor,
)
except OSError as error:
raise DocForgeError(
"path_escape",
"Derived output directory is missing or unsafe",
) from error
os.close(descriptor)
descriptor = next_descriptor
require_bound_directory(path, descriptor)
return descriptor
except Exception:
os.close(descriptor)
raise
def safe_file_identity_at(
directory: Path,
directory_fd: int,
name: str,
) -> dict[str, object] | None:
"""Return one no-follow regular-file identity relative to a bound directory."""
del directory
if not name or "/" in name or name in {".", ".."}:
raise DocForgeError("path_escape", "Derived artifact name is unsafe")
try:
current = os.stat(name, dir_fd=directory_fd, follow_symlinks=False)
except FileNotFoundError:
return None
except OSError as error:
raise DocForgeError("path_escape", "Derived artifact cannot be inspected") from error
if not stat.S_ISREG(current.st_mode):
raise DocForgeError("path_escape", "Derived artifact is not a safe regular file")
return {
"path": name,
"device": current.st_dev,
"inode": current.st_ino,
"mode": current.st_mode,
"size": current.st_size,
"mtime_ns": current.st_mtime_ns,
"ctime_ns": current.st_ctime_ns,
}
def read_bounded_file_at(
directory_fd: int,
name: str,
maximum_bytes: int,
) -> bytes | None:
"""Read one regular file through a bound directory without following links."""
try:
descriptor = os.open(name, os.O_RDONLY | os.O_NOFOLLOW, dir_fd=directory_fd)
except FileNotFoundError:
return None
except OSError as error:
raise DocForgeError("path_escape", "Derived artifact cannot be opened safely") from error
with os.fdopen(descriptor, "rb") as handle:
current = os.fstat(handle.fileno())
if not stat.S_ISREG(current.st_mode) or current.st_size > maximum_bytes:
raise DocForgeError("invalid_projection", "Derived artifact is invalid or oversized")
content = handle.read(maximum_bytes + 1)
if len(content) > maximum_bytes:
raise DocForgeError("invalid_projection", "Derived artifact is oversized")
return content
def atomic_replace_bytes_at(
path: Path,
directory_fd: int,
name: str,
content: bytes,
*,
verify: Callable[[], None],
) -> dict[str, object]:
"""Durably replace one file inside an already bound directory."""
if not name or "/" in name or name in {".", ".."}:
raise DocForgeError("path_escape", "Derived artifact name is unsafe")
existing = safe_file_identity_at(path, directory_fd, name)
del existing
temporary = f".docforge-projection-{secrets.token_hex(12)}"
descriptor: int | None = None
committed = False
try:
descriptor = os.open(
temporary,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW,
0o600,
dir_fd=directory_fd,
)
with os.fdopen(descriptor, "wb") as handle:
descriptor = None
handle.write(content)
handle.flush()
os.fsync(handle.fileno())
verify()
require_bound_directory(path, directory_fd)
os.replace(
temporary,
name,
src_dir_fd=directory_fd,
dst_dir_fd=directory_fd,
)
committed = True
os.fsync(directory_fd)
identity = safe_file_identity_at(path, directory_fd, name)
if identity is None:
raise DocForgeError(
"publication_failure",
"Derived artifact disappeared after publication",
mutation_committed=True,
)
return identity
except DocForgeError as error:
if committed:
raise DocForgeError(
"publication_failure",
"Derived artifact was replaced but final publication verification failed",
mutation_committed=True,
cause=error.code,
) from error
raise
except OSError as error:
raise DocForgeError(
"publication_failure",
"Derived artifact publication failed",
mutation_committed=committed,
) from error
finally:
if descriptor is not None:
os.close(descriptor)
with suppress(OSError):
os.unlink(temporary, dir_fd=directory_fd)

8
src/docforge/_projection_worker_main.py
View file

@ -1,8 +0,0 @@
"""Private module entry point for the detached projection worker."""
from __future__ import annotations
from .projection_worker import main
if __name__ == "__main__":
raise SystemExit(main())

21
src/docforge/application.py
View file

@ -15,7 +15,6 @@ from .changesets import ChangesetStore
from .errors import DocForgeError
from .index import ProjectIndex
from .models import Node, ProjectService, ProjectSnapshot
from .projection_policy import ManualProjectionMode, validate_manual_projection_mode
from .rendering import RenderService
@ -354,19 +353,13 @@ class CanonicalApplicationService:
applier_id: str | None,
applier: CanonicalApplier | None,
index: ProjectIndex | None = None,
manual_policy: ManualProjectionMode = "auto",
) -> None:
self.project = project
self.applier_id = applier_id
self.applier = applier
self.changesets = ChangesetStore(project, applier_id)
self.index = index or ProjectIndex(project)
self.manual_policy = validate_manual_projection_mode(manual_policy)
self.rendering = RenderService(
project,
self.changesets,
manual_policy=self.manual_policy,
)
self.rendering = RenderService(project, self.changesets)
@property
def enabled(self) -> bool:
@ -409,9 +402,7 @@ class CanonicalApplicationService:
)
renders: list[dict[str, object]] = []
config = self.project.descriptor.render
render_action = "not_configured"
if config is not None and self.manual_policy == "auto":
render_action = "rendered"
if config is not None:
for view in config.views:
try:
rendered = self.rendering.render(view.view_id)
@ -444,10 +435,6 @@ class CanonicalApplicationService:
"error": error.as_dict(),
}
)
elif config is not None:
render_action = (
"skipped_explicit" if self.manual_policy == "explicit" else "skipped_disabled"
)
return {
**applied,
"derived_refresh": {
@ -455,10 +442,6 @@ class CanonicalApplicationService:
"index": index_result,
"check": index_check,
"renders": renders,
"render_policy": {
"mode": self.manual_policy,
"action": render_action,
},
"errors": refresh_errors,
},
}

2
src/docforge/assets/graph.html
View file

@ -95,7 +95,7 @@
aria-label="Visible relationship color and symbol key"></ul>
</details>
<svg id="graph" viewBox="-600 -410 1200 820"
role="group" aria-label="Interactive node neighborhood"></svg>
role="img" aria-label="Node neighborhood"></svg>
<div class="empty" id="empty">Search for a node to inspect its neighborhood.</div>
<div class="connection-state" id="connection-state" role="alert" hidden>
<strong>Visualization disconnected</strong>

113
src/docforge/cli.py
View file

@ -13,11 +13,9 @@ from .client_config import CLIENT_NAMES, generate_client_configuration
from .context import compile_context
from .doctor import run_doctor
from .errors import DocForgeError
from .graph_rendering import GraphRenderService
from .index import ProjectIndex
from .onboarding import assess_project, scaffold_project
from .project import Project, project_root_fingerprint
from .projection_policy import compose_projection_policy
from .rendering import RenderService
from .telemetry import request
from .viewer_manager import ViewerManagerClient
@ -31,18 +29,6 @@ def _parser() -> argparse.ArgumentParser:
action="store_true",
help="Attach bounded request-local stage timings and counters",
)
parser.add_argument(
"--manual-render-policy",
choices=("auto", "explicit", "disabled"),
)
parser.add_argument(
"--portable-graph-policy",
choices=("explicit", "disabled"),
)
parser.add_argument(
"--live-viewer-policy",
choices=("on-demand", "disabled"),
)
commands = parser.add_subparsers(dest="command", required=True)
configure = commands.add_parser("configure")
configure.add_argument("client", choices=CLIENT_NAMES)
@ -56,21 +42,6 @@ def _parser() -> argparse.ArgumentParser:
configure.add_argument("--proposal-writer")
configure.add_argument("--canonical-applier")
configure.add_argument("--no-ast", action="store_true")
configure.add_argument(
"--manual-render-policy",
choices=("auto", "explicit", "disabled"),
default=argparse.SUPPRESS,
)
configure.add_argument(
"--portable-graph-policy",
choices=("explicit", "disabled"),
default=argparse.SUPPRESS,
)
configure.add_argument(
"--live-viewer-policy",
choices=("on-demand", "disabled"),
default=argparse.SUPPRESS,
)
configure.add_argument("--startup-timeout", type=int, default=30)
configure.add_argument("--tool-timeout", type=int, default=300)
configure.add_argument("--output", type=Path)
@ -124,12 +95,6 @@ def _parser() -> argparse.ArgumentParser:
render_status = commands.add_parser("render-status")
render_status.add_argument("view_id", nargs="?")
render_status.add_argument("--deep", action="store_true")
graph_plan = commands.add_parser("graph-plan")
graph_plan.add_argument("view_id")
graph_render = commands.add_parser("graph-render")
graph_render.add_argument("view_id")
graph_render_status = commands.add_parser("graph-render-status")
graph_render_status.add_argument("view_id", nargs="?")
preview = commands.add_parser("preview")
preview.add_argument("changeset_id")
preview.add_argument("view_id")
@ -159,9 +124,6 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
proposal_writer=arguments.proposal_writer,
canonical_applier=arguments.canonical_applier,
no_ast=arguments.no_ast,
manual_render_policy=arguments.manual_render_policy,
portable_graph_policy=arguments.portable_graph_policy,
live_viewer_policy=arguments.live_viewer_policy,
startup_timeout=arguments.startup_timeout,
tool_timeout=arguments.tool_timeout,
output=arguments.output,
@ -190,39 +152,12 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
content_root=arguments.content_root,
)
project = Project.open(arguments.project_root)
projection_policy = compose_projection_policy(
manual=arguments.manual_render_policy,
portable_graph=arguments.portable_graph_policy,
live_viewer=arguments.live_viewer_policy,
manual_configured=project.descriptor.render is not None,
portable_graph_configured=project.descriptor.graph_render is not None,
application_enabled=False,
)
build = ProjectIndex(project).build()
render = (
{
"status": "ok",
"state": "skipped",
"reason": "projection_policy_disabled",
}
if projection_policy.manual == "disabled"
else RenderService(
project,
manual_policy=projection_policy.manual,
).render("manual")
)
render = RenderService(project).render("manual")
return {**scaffold, "build": build, "render": render}
return assess_project(arguments.project_root, requested_languages=languages)
project = Project.open(arguments.project_root)
index = ProjectIndex(project)
projection_policy = compose_projection_policy(
manual=arguments.manual_render_policy,
portable_graph=arguments.portable_graph_policy,
live_viewer=arguments.live_viewer_policy,
manual_configured=project.descriptor.render is not None,
portable_graph_configured=project.descriptor.graph_render is not None,
application_enabled=arguments.command == "apply",
)
if arguments.command == "info":
snapshot = project.load()
return {
@ -315,52 +250,24 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
cursor=arguments.cursor,
)
if arguments.command == "render":
return RenderService(
project,
manual_policy=projection_policy.manual,
).render(arguments.view_id)
return RenderService(project).render(arguments.view_id)
if arguments.command == "render-status":
rendering = RenderService(
project,
manual_policy=projection_policy.manual,
)
rendering = RenderService(project)
return (
rendering.deep_status(arguments.view_id)
if arguments.deep
else rendering.status(arguments.view_id)
)
if arguments.command == "graph-plan":
return GraphRenderService(
project,
portable_graph_policy=projection_policy.portable_graph,
).plan(arguments.view_id)
if arguments.command == "graph-render":
return GraphRenderService(
project,
portable_graph_policy=projection_policy.portable_graph,
).render(arguments.view_id)
if arguments.command == "graph-render-status":
return GraphRenderService(
project,
portable_graph_policy=projection_policy.portable_graph,
).status(arguments.view_id)
if arguments.command == "preview":
return RenderService(
project,
manual_policy=projection_policy.manual,
).preview(arguments.changeset_id, arguments.view_id)
return RenderService(project).preview(arguments.changeset_id, arguments.view_id)
if arguments.command == "apply":
return CanonicalApplicationService(
project,
applier_id=arguments.applier,
applier=GenericCanonicalApplier(project),
manual_policy=projection_policy.manual,
).apply(arguments.changeset_id, arguments.changeset_hash)
if arguments.command == "visualize":
visualization = ViewerManagerClient(
index,
live_viewer_policy=projection_policy.live_viewer,
).start(
visualization = ViewerManagerClient(index).start(
node_id=arguments.node,
query=arguments.query,
depth=arguments.depth,
@ -377,15 +284,9 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
"visualization": visualization,
}
if arguments.command == "visualization-status":
return ViewerManagerClient(
index,
live_viewer_policy=projection_policy.live_viewer,
).status()
return ViewerManagerClient(index).status()
if arguments.command == "visualization-stop":
return ViewerManagerClient(
index,
live_viewer_policy=projection_policy.live_viewer,
).stop()
return ViewerManagerClient(index).stop()
raise DocForgeError("invalid_command", "Unknown command")

158
src/docforge/client_config.py
View file

@ -18,10 +18,9 @@ from typing import Literal, cast
from .changeset_contract import document_hash
from .errors import DocForgeError
from .models import ProjectDescriptor, ProjectService
from .models import ProjectService
from .policy import CapabilityMode, compose_effective_policy
from .project import Project, project_root_fingerprint, validate_descriptor_binding
from .projection_policy import compose_projection_policy
from .project import project_root_fingerprint, validate_descriptor_binding
ClientName = Literal["codex", "claude", "openclaw"]
CLIENT_NAMES: tuple[ClientName, ...] = ("codex", "claude", "openclaw")
@ -606,37 +605,11 @@ def _atomic_write(
os.close(directory_fd)
def _validate_configuration_result(
result: dict[str, object],
*,
trusted_descriptor: ProjectDescriptor | None = None,
) -> None:
def _validate_configuration_result(result: dict[str, object]) -> None:
artifact = cast(dict[str, object], result["artifact"])
binding = cast(dict[str, object], result["binding"])
policy = cast(dict[str, object], result["effective_policy"])
projection_policy = cast(dict[str, object], result["projection_policy"])
projection_availability = cast(
dict[str, object],
result["projection_availability"],
)
project = cast(dict[str, object], result["project"])
if trusted_descriptor is None:
try:
bound_descriptor = Project.open(cast(str, project["project_root"])).descriptor
except (DocForgeError, KeyError, TypeError) as error:
raise AssertionError(
"Generated client project binding cannot be independently validated"
) from error
else:
bound_descriptor = trusted_descriptor
if (
project["project_id"] != bound_descriptor.project_id
or project["project_root"] != str(bound_descriptor.root)
or project["project_root_fingerprint"] != project_root_fingerprint(bound_descriptor.root)
or project["adapter"] != bound_descriptor.adapter
or project["descriptor_hash"] != bound_descriptor.descriptor_hash
):
raise AssertionError("Generated client project binding drifted")
content = cast(str, artifact["content"])
if artifact["content_sha256"] != hashlib.sha256(content.encode("utf-8")).hexdigest():
raise AssertionError("Generated client content hash drifted")
@ -671,29 +644,6 @@ def _validate_configuration_result(
if remaining[-1:] != ["--no-ast"] or arguments.count("--no-ast") != 1:
raise AssertionError("Generated no-AST argument layout drifted")
remaining = remaining[:-1]
projection_arguments: dict[str, str] = {}
authority_arguments: list[str] = []
position = 0
projection_options = {
"--manual-render-policy": "manual",
"--portable-graph-policy": "portable_graph",
"--live-viewer-policy": "live_viewer",
}
while position < len(remaining):
option = remaining[position]
field = projection_options.get(option)
if field is None:
authority_arguments.append(option)
position += 1
continue
if position + 1 >= len(remaining) or option in projection_arguments:
raise AssertionError("Generated projection policy argument layout drifted")
value = remaining[position + 1]
projection_arguments[option] = value
if projection_policy[field] != value:
raise AssertionError("Generated projection policy argument drifted")
position += 2
remaining = authority_arguments
mode = binding["capability_mode"]
if (
(mode == "read" and remaining)
@ -713,34 +663,6 @@ def _validate_configuration_result(
)
):
raise AssertionError("Generated authority argument layout drifted")
expected_projection_policy = compose_projection_policy(
manual=projection_arguments.get("--manual-render-policy"),
portable_graph=projection_arguments.get("--portable-graph-policy"),
live_viewer=projection_arguments.get("--live-viewer-policy"),
manual_configured=cast(bool, projection_availability["manual_configured"]),
portable_graph_configured=cast(
bool,
projection_availability["portable_graph_configured"],
),
application_enabled=cast(
bool,
projection_availability["application_enabled"],
),
live_viewer_available=cast(
bool,
projection_availability["live_viewer_available"],
),
)
if (
projection_policy != expected_projection_policy.as_dict()
or projection_availability["manual_configured"] != (render_policy["manual"] != "disabled")
or projection_availability["manual_configured"] != (bound_descriptor.render is not None)
or projection_availability["portable_graph_configured"]
!= (bound_descriptor.graph_render is not None)
or projection_availability["application_enabled"] != (mode == "application")
or projection_availability["live_viewer_available"] is not True
):
raise AssertionError("Generated projection policy drifted from its availability")
composed_policy = compose_effective_policy(
selected_mode=cast(CapabilityMode, mode),
capability_source="explicit",
@ -776,18 +698,6 @@ def _validate_configuration_result(
)
):
raise AssertionError("Generated client policy drifted from its binding")
if (
result["projection_policy_hash"]
!= hashlib.sha256(
json.dumps(
projection_policy,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
).hexdigest()
):
raise AssertionError("Generated projection policy hash drifted")
expected_hash = document_hash(
{
"schema_version": 1,
@ -796,9 +706,6 @@ def _validate_configuration_result(
"project": project,
"binding": binding,
"effective_policy": policy,
"projection_policy": projection_policy,
"projection_policy_hash": result["projection_policy_hash"],
"projection_availability": projection_availability,
"artifact_format": artifact["format"],
"artifact_content_sha256": artifact["content_sha256"],
}
@ -816,9 +723,6 @@ def generate_client_configuration(
proposal_writer: str | None = None,
canonical_applier: str | None = None,
no_ast: bool = False,
manual_render_policy: str | None = None,
portable_graph_policy: str | None = None,
live_viewer_policy: str | None = None,
startup_timeout: int = 30,
tool_timeout: int = 300,
output: Path | None = None,
@ -935,6 +839,9 @@ def generate_client_configuration(
arguments.extend(("--proposal-writer", proposal_writer))
if canonical_applier is not None:
arguments.extend(("--canonical-applier", canonical_applier))
if no_ast:
arguments.append("--no-ast")
policy = compose_effective_policy(
selected_mode=selected_mode,
capability_source="explicit",
@ -943,40 +850,6 @@ def generate_client_configuration(
render_configured=descriptor.render is not None,
application_enabled=canonical_applier is not None,
)
projection_policy = compose_projection_policy(
manual=manual_render_policy,
portable_graph=portable_graph_policy,
live_viewer=live_viewer_policy,
manual_configured=descriptor.render is not None,
portable_graph_configured=descriptor.graph_render is not None,
application_enabled=canonical_applier is not None,
)
default_projection_policy = compose_projection_policy(
manual_configured=descriptor.render is not None,
portable_graph_configured=descriptor.graph_render is not None,
application_enabled=canonical_applier is not None,
)
for option, selected, default in (
(
"--manual-render-policy",
projection_policy.manual,
default_projection_policy.manual,
),
(
"--portable-graph-policy",
projection_policy.portable_graph,
default_projection_policy.portable_graph,
),
(
"--live-viewer-policy",
projection_policy.live_viewer,
default_projection_policy.live_viewer,
),
):
if selected != default:
arguments.extend((option, selected))
if no_ast:
arguments.append("--no-ast")
artifact_format, content, warning = _artifact(
selected_client,
server_name=selected_name,
@ -1029,7 +902,6 @@ def generate_client_configuration(
"project_root": str(descriptor.root),
"project_root_fingerprint": fingerprint,
"adapter": descriptor.adapter,
"descriptor_hash": descriptor.descriptor_hash,
}
policy_payload = policy.as_dict()
plan_hash = document_hash(
@ -1040,14 +912,6 @@ def generate_client_configuration(
"project": project_binding,
"binding": binding,
"effective_policy": policy_payload,
"projection_policy": projection_policy.as_dict(),
"projection_policy_hash": projection_policy.policy_hash,
"projection_availability": {
"manual_configured": descriptor.render is not None,
"portable_graph_configured": descriptor.graph_render is not None,
"application_enabled": canonical_applier is not None,
"live_viewer_available": True,
},
"artifact_format": artifact_format,
"artifact_content_sha256": artifact["content_sha256"],
}
@ -1062,14 +926,6 @@ def generate_client_configuration(
"project": project_binding,
"binding": binding,
"effective_policy": policy_payload,
"projection_policy": projection_policy.as_dict(),
"projection_policy_hash": projection_policy.policy_hash,
"projection_availability": {
"manual_configured": descriptor.render is not None,
"portable_graph_configured": descriptor.graph_render is not None,
"application_enabled": canonical_applier is not None,
"live_viewer_available": True,
},
"artifact": artifact,
"configuration_hash": plan_hash,
"warnings": [
@ -1077,5 +933,5 @@ def generate_client_configuration(
*([] if publication_warning is None else [{"code": publication_warning}]),
],
}
_validate_configuration_result(result, trusted_descriptor=descriptor)
_validate_configuration_result(result)
return result

17
src/docforge/doctor.py
View file

@ -20,7 +20,6 @@ from .project import (
project_root_fingerprint,
validate_descriptor_binding,
)
from .projection_policy import compose_projection_policy
MAX_CLIENT_CONFIG_BYTES = 1_000_000
MAX_CLIENT_SERVERS = 256
@ -615,9 +614,6 @@ def _parse_binding(arguments: list[str]) -> dict[str, object]:
"--proposal-writer",
"--canonical-applier",
"--capability-mode",
"--manual-render-policy",
"--portable-graph-policy",
"--live-viewer-policy",
}
flag_options = {"--no-ast", "--diagnostics"}
position = 0
@ -666,9 +662,6 @@ def _parse_binding(arguments: list[str]) -> dict[str, object]:
"capability_mode_implicit": implicit,
"no_ast": "--no-ast" in flags,
"diagnostics": "--diagnostics" in flags,
"manual_render_policy": values.get("--manual-render-policy"),
"portable_graph_policy": values.get("--portable-graph-policy"),
"live_viewer_policy": values.get("--live-viewer-policy"),
}
@ -756,16 +749,6 @@ def _runtime_policy_check(
canonical_applier is not None and selected_mode in {"application", "operator"}
),
)
compose_projection_policy(
manual=cast(str | None, binding["manual_render_policy"]),
portable_graph=cast(str | None, binding["portable_graph_policy"]),
live_viewer=cast(str | None, binding["live_viewer_policy"]),
manual_configured=project.descriptor.render is not None,
portable_graph_configured=project.descriptor.graph_render is not None,
application_enabled=(
canonical_applier is not None and selected_mode in {"application", "operator"}
),
)
if cast(bool, binding["capability_mode_implicit"]):
return (
"warning",

525
src/docforge/graph_projection.py
View file

@ -1,525 +0,0 @@
"""Pure, bounded portable-graph planning over one immutable graph generation."""
from __future__ import annotations
import re
from dataclasses import dataclass
from typing import Literal, cast
from .errors import DocForgeError
from .models import Edge, Node, ProjectSnapshot
from .project import project_root_fingerprint
from .projection_contract import (
MAX_GRAPH_VIEW_DEPTH,
MAX_GRAPH_VIEW_EDGES,
MAX_GRAPH_VIEW_FILTERS,
MAX_GRAPH_VIEW_NODES,
MAX_GRAPH_VIEW_QUERY_CHARS,
MAX_GRAPH_VIEW_STRING_CHARS,
MAX_GRAPH_VIEW_WORK,
GraphViewPlanV1,
ProjectionPackageV1,
)
GraphViewMode = Literal["nodes", "flow", "web", "logic"]
_QUERY_TOKEN = re.compile(r"\w+", re.UNICODE)
_DETAIL_FIELDS = (
"node_id",
"title",
"family",
"authority",
"status",
"tags",
"summary",
"content_hash",
)
@dataclass(frozen=True)
class GraphViewRequestV1:
"""One closed, inert graph selection request."""
view_id: str
title: str
root_node_id: str | None = None
query: str | None = None
initial_mode: GraphViewMode = "nodes"
depth: int = 1
max_nodes: int = 100
max_edges: int = 400
max_work: int = 100_000
families: tuple[str, ...] = ()
relations: tuple[str, ...] = ()
authorities: tuple[str, ...] = ()
statuses: tuple[str, ...] = ()
tags: tuple[str, ...] = ()
include_logic: bool = False
@dataclass(frozen=True)
class _ValidatedRequest:
view_id: str
title: str
root_node_id: str | None
query: str | None
initial_mode: GraphViewMode
depth: int
max_nodes: int
max_edges: int
max_work: int
families: tuple[str, ...]
relations: tuple[str, ...]
authorities: tuple[str, ...]
statuses: tuple[str, ...]
tags: tuple[str, ...]
include_logic: bool
def _invalid(message: str, **details: object) -> DocForgeError:
return DocForgeError("invalid_graph_view_request", message, **details)
def _string(value: object, *, field: str, maximum: int = MAX_GRAPH_VIEW_STRING_CHARS) -> str:
if not isinstance(value, str) or not value.strip() or len(value) > maximum or "\0" in value:
raise _invalid("Graph view request string is invalid", field=field)
return value.strip()
def _filter_values(values: object, *, field: str) -> tuple[str, ...]:
if not isinstance(values, tuple):
raise _invalid("Graph view filter is invalid", field=field)
tuple_values = cast(tuple[object, ...], values)
if not all(isinstance(value, str) for value in tuple_values):
raise _invalid("Graph view filter is invalid", field=field)
if len(tuple_values) > MAX_GRAPH_VIEW_FILTERS:
raise _invalid("Graph view filter is invalid", field=field)
normalized = tuple(_string(value, field=field) for value in cast(tuple[str, ...], tuple_values))
if len(normalized) != len(set(normalized)):
raise _invalid("Graph view filter contains duplicates", field=field)
return tuple(sorted(normalized))
def _validated_request(request: GraphViewRequestV1) -> _ValidatedRequest:
view_id = _string(request.view_id, field="view_id")
title = _string(request.title, field="title")
if (request.root_node_id is None) == (request.query is None):
raise _invalid("Choose exactly one exact root or lexical query")
root_node_id = (
None
if request.root_node_id is None
else _string(request.root_node_id, field="root_node_id")
)
query = (
None
if request.query is None
else _string(
request.query,
field="query",
maximum=MAX_GRAPH_VIEW_QUERY_CHARS,
)
)
if request.initial_mode not in {"nodes", "flow", "web", "logic"}:
raise _invalid("Graph view initial mode is unsupported")
if type(request.depth) is not int or not 1 <= request.depth <= MAX_GRAPH_VIEW_DEPTH:
raise _invalid(
"Graph view depth is outside the fixed boundary",
maximum=MAX_GRAPH_VIEW_DEPTH,
)
for field, value, minimum, maximum in (
("max_nodes", request.max_nodes, 1, MAX_GRAPH_VIEW_NODES),
("max_edges", request.max_edges, 0, MAX_GRAPH_VIEW_EDGES),
("max_work", request.max_work, 1, MAX_GRAPH_VIEW_WORK),
):
if type(value) is not int or not minimum <= value <= maximum:
raise _invalid(
"Graph view bound is outside the fixed boundary",
field=field,
minimum=minimum,
maximum=maximum,
)
if type(request.include_logic) is not bool:
raise _invalid("Graph view Logic selection must be Boolean")
return _ValidatedRequest(
view_id=view_id,
title=title,
root_node_id=root_node_id,
query=query,
initial_mode=request.initial_mode,
depth=request.depth,
max_nodes=request.max_nodes,
max_edges=request.max_edges,
max_work=request.max_work,
families=_filter_values(request.families, field="families"),
relations=_filter_values(request.relations, field="relations"),
authorities=_filter_values(request.authorities, field="authorities"),
statuses=_filter_values(request.statuses, field="statuses"),
tags=_filter_values(request.tags, field="tags"),
include_logic=request.include_logic,
)
def _validated_graph(
snapshot: ProjectSnapshot,
) -> tuple[tuple[Node, ...], tuple[Edge, ...], dict[str, Node]]:
nodes = tuple(sorted(snapshot.nodes, key=lambda node: node.node_id))
node_by_id = {node.node_id: node for node in nodes}
if len(node_by_id) != len(nodes):
raise DocForgeError(
"invalid_projection",
"Graph view snapshot contains duplicate node identities",
)
edges = tuple(
sorted(
snapshot.edges,
key=lambda edge: (edge.source_id, edge.relation, edge.target_id),
)
)
edge_keys = {(edge.source_id, edge.relation, edge.target_id) for edge in edges}
if len(edge_keys) != len(edges) or any(
edge.source_id not in node_by_id or edge.target_id not in node_by_id for edge in edges
):
raise DocForgeError(
"invalid_projection",
"Graph view snapshot contains invalid relationships",
)
return nodes, edges, node_by_id
def _eligible(node: Node, request: _ValidatedRequest) -> bool:
return (
(not request.families or node.family in request.families)
and (not request.authorities or node.authority in request.authorities)
and (not request.statuses or node.status in request.statuses)
and (not request.tags or set(request.tags).issubset(node.tags))
)
def _relation_allowed(edge: Edge, request: _ValidatedRequest) -> bool:
return not request.relations or edge.relation in request.relations
def _lexical_text(node: Node) -> str:
return " ".join(
(
node.node_id,
node.title,
node.summary,
node.family,
node.authority,
node.status,
*node.tags,
)
).casefold()
def _lexical_nodes(
nodes: tuple[Node, ...],
request: _ValidatedRequest,
omissions: list[dict[str, object]],
) -> tuple[set[str], int, bool]:
query = request.query
maximum_nodes = request.max_nodes
maximum_work = request.max_work
assert query is not None
terms = tuple(dict.fromkeys(_QUERY_TOKEN.findall(query.casefold())))
if not terms:
raise _invalid("Lexical graph scope contains no searchable text")
selected: set[str] = set()
work = 0
work_limited = False
for node in nodes:
if work >= maximum_work:
work_limited = True
break
work += 1
if not _eligible(node, request) or not all(term in _lexical_text(node) for term in terms):
continue
if len(selected) >= maximum_nodes:
omissions.append(
{
"code": "node_result_limit",
"subject": "nodes",
"limit": maximum_nodes,
"minimum_omitted": 1,
}
)
break
selected.add(node.node_id)
return selected, work, work_limited
def _root_nodes(
edges: tuple[Edge, ...],
node_by_id: dict[str, Node],
request: _ValidatedRequest,
omissions: list[dict[str, object]],
) -> tuple[set[str], int, bool]:
root_node_id = request.root_node_id
maximum_nodes = request.max_nodes
maximum_work = request.max_work
depth = request.depth
assert root_node_id is not None
root = node_by_id.get(root_node_id)
if root is None:
raise DocForgeError(
"missing_node",
"No node has the requested stable ID",
node_id=root_node_id,
)
if not _eligible(root, request):
raise _invalid("Exact graph root is excluded by the closed node filters")
selected = {root_node_id}
frontier = {root_node_id}
work = 0
work_limited = False
node_limited = False
for _ in range(depth):
if not frontier:
break
next_frontier: set[str] = set()
for edge in edges:
if work >= maximum_work:
work_limited = True
break
work += 1
if not _relation_allowed(edge, request):
continue
candidate: str | None = None
if edge.source_id in frontier:
candidate = edge.target_id
elif edge.target_id in frontier:
candidate = edge.source_id
if candidate is None or candidate in selected:
continue
node = node_by_id[candidate]
if not _eligible(node, request):
continue
if len(selected) >= maximum_nodes:
node_limited = True
continue
selected.add(candidate)
next_frontier.add(candidate)
if work_limited:
break
frontier = next_frontier
if node_limited:
omissions.append(
{
"code": "node_result_limit",
"subject": "nodes",
"limit": maximum_nodes,
"minimum_omitted": 1,
}
)
return selected, work, work_limited
def _selected_edges(
edges: tuple[Edge, ...],
selected_ids: set[str],
request: _ValidatedRequest,
*,
initial_work: int,
omissions: list[dict[str, object]],
) -> tuple[list[Edge], int, bool]:
maximum_edges = request.max_edges
maximum_work = request.max_work
selected: list[Edge] = []
work = initial_work
work_limited = False
edge_limited = False
for edge in edges:
if work >= maximum_work:
work_limited = True
break
work += 1
if (
edge.source_id not in selected_ids
or edge.target_id not in selected_ids
or not _relation_allowed(edge, request)
):
continue
if len(selected) >= maximum_edges:
edge_limited = True
break
selected.append(edge)
if edge_limited:
omissions.append(
{
"code": "edge_result_limit",
"subject": "edges",
"limit": maximum_edges,
"minimum_omitted": 1,
}
)
return selected, work, work_limited
def _node_payload(node: Node) -> dict[str, object]:
return {
"node_id": node.node_id,
"title": node.title,
"family": node.family,
"authority": node.authority,
"status": node.status,
"tags": sorted(node.tags),
"summary": node.summary,
"content_hash": node.content_hash,
}
def _edge_payload(edge: Edge) -> dict[str, str]:
return {
"source_id": edge.source_id,
"relation": edge.relation,
"target_id": edge.target_id,
}
def build_graph_view_plan(
snapshot: ProjectSnapshot,
request: GraphViewRequestV1,
allow_logic: bool,
) -> GraphViewPlanV1:
"""Build one deterministic, path-free graph plan without rendering or storage access."""
if type(allow_logic) is not bool:
raise _invalid("Graph view Logic policy must be Boolean")
normalized = _validated_request(request)
nodes, edges, node_by_id = _validated_graph(snapshot)
omissions: list[dict[str, object]] = []
if normalized.root_node_id is not None:
selected_ids, work, work_limited = _root_nodes(
edges,
node_by_id,
normalized,
omissions,
)
scope: dict[str, object] = {
"kind": "exact_root",
"root_node_id": normalized.root_node_id,
"depth": normalized.depth,
}
else:
selected_ids, work, work_limited = _lexical_nodes(
nodes,
normalized,
omissions,
)
scope = {
"kind": "lexical",
"query": normalized.query,
}
selected_edges, work, edge_work_limited = _selected_edges(
edges,
selected_ids,
normalized,
initial_work=work,
omissions=omissions,
)
work_limited = work_limited or edge_work_limited
if work_limited:
omissions.append(
{
"code": "work_limit",
"subject": "selection",
"limit": normalized.max_work,
"examined": work,
"minimum_omitted": 1,
}
)
logic_requested = normalized.include_logic
if logic_requested and not allow_logic:
omissions.append(
{
"code": "logic_forbidden",
"subject": "logic",
"minimum_omitted": 1,
}
)
omissions.sort(key=lambda item: (str(item["code"]), str(item["subject"])))
selected_nodes = [node_by_id[node_id] for node_id in sorted(selected_ids)]
filters: dict[str, list[str]] = {
"families": list(normalized.families),
"relations": list(normalized.relations),
"authorities": list(normalized.authorities),
"statuses": list(normalized.statuses),
"tags": list(normalized.tags),
}
return GraphViewPlanV1.create(
{
"project": {
"project_id": snapshot.descriptor.project_id,
"project_root_fingerprint": project_root_fingerprint(snapshot.descriptor.root),
"adapter": snapshot.descriptor.adapter,
"revision": snapshot.revision,
"source_hash": snapshot.source_hash,
},
"view": {
"view_id": normalized.view_id,
"title": normalized.title,
"initial_mode": normalized.initial_mode,
"scope": scope,
"filters": filters,
"detail_fields": list(_DETAIL_FIELDS),
},
"bounds": {
"depth": normalized.depth,
"max_nodes": normalized.max_nodes,
"max_edges": normalized.max_edges,
"max_work": normalized.max_work,
},
"policy": {
"visibility": "selected_graph_only",
"source_paths": "excluded",
"source_bodies": "excluded",
"database_queries": "forbidden",
"executable_content": "forbidden",
"logic": "allowed" if allow_logic else "forbidden",
"logic_requested": logic_requested,
},
"graph": {
"root_node_id": normalized.root_node_id,
"nodes": [_node_payload(node) for node in selected_nodes],
"edges": [_edge_payload(edge) for edge in selected_edges],
"logic_projections": [],
},
"omissions": omissions,
"diagnostics": {
"selection": scope["kind"],
"returned_nodes": len(selected_nodes),
"returned_edges": len(selected_edges),
"examined_work_units": work,
"truncated": bool(omissions),
"ordering": "node_id;source_id,relation,target_id",
},
}
)
def build_graph_projection_package(
plan: GraphViewPlanV1,
*,
renderer_id: str,
renderer_version: str,
max_output_bytes: int,
) -> ProjectionPackageV1:
"""Bind a graph plan to the fixed portable renderer without adding runtime authority."""
return ProjectionPackageV1.create(
kind="graph",
plan=plan,
renderer={"renderer_id": renderer_id, "renderer_version": renderer_version},
components=[
{"component_id": "graph.portable-document@1"},
{"component_id": "graph.accessible-list@1"},
{"component_id": "graph.relationship-table@1"},
],
assets=[],
output_policy={
"artifact_ids": ["portable-graph.html"],
"max_total_bytes": max_output_bytes,
},
)

285
src/docforge/graph_render_config.py
View file

@ -1,285 +0,0 @@
"""Strict parsing and confinement for optional portable graph artifacts."""
from __future__ import annotations
from pathlib import Path
from typing import Literal, cast
from .config_validation import (
ID_PATTERN,
confined_path,
positive_int,
require_string,
string_list,
)
from .errors import DocForgeError
from .models import GraphRenderConfig, GraphRenderView, Limits, RenderConfig
from .projection_contract import (
MAX_GRAPH_VIEW_DEPTH,
MAX_GRAPH_VIEW_EDGES,
MAX_GRAPH_VIEW_FILTERS,
MAX_GRAPH_VIEW_NODES,
MAX_GRAPH_VIEW_QUERY_CHARS,
MAX_GRAPH_VIEW_STRING_CHARS,
MAX_GRAPH_VIEW_WORK,
)
_CONFIG_KEYS = frozenset({"output_root", "views"})
_VIEW_KEYS = frozenset(
{
"id",
"renderer",
"output",
"title",
"root",
"query",
"initial_mode",
"depth",
"max_nodes",
"max_edges",
"max_work",
"families",
"relations",
"authorities",
"statuses",
"tags",
"include_logic",
}
)
_MODES = frozenset({"nodes", "flow", "web"})
def _overlaps(first: Path, second: Path) -> bool:
return first == second or first.is_relative_to(second) or second.is_relative_to(first)
def _optional_string(document: dict[str, object], key: str, source: Path) -> str | None:
if key not in document:
return None
return require_string(document, key, source)
def _bounded_string(
document: dict[str, object],
key: str,
source: Path,
*,
maximum: int,
) -> str:
value = require_string(document, key, source)
if len(value) > maximum:
raise DocForgeError("invalid_config", f"{key} exceeds its fixed character limit")
return value
def _bounded_strings(
value: object,
*,
key: str,
source: Path,
) -> tuple[str, ...]:
values = string_list(value, key=key, source=source)
if len(values) > MAX_GRAPH_VIEW_FILTERS or any(
len(item) > MAX_GRAPH_VIEW_STRING_CHARS for item in values
):
raise DocForgeError("invalid_config", f"{key} exceeds its fixed bounds")
return values
def load_graph_render_config(
root: Path,
document: object,
*,
descriptor_path: Path,
content_roots: tuple[Path, ...],
authority_files: tuple[Path, ...],
cache_root: Path,
index_path: Path,
changeset_root: Path,
manual_render: RenderConfig | None,
limits: Limits,
) -> GraphRenderConfig | None:
if document is None:
return None
if not isinstance(document, dict):
raise DocForgeError("invalid_config", "graph_render must be a table")
document = cast(dict[str, object], document)
unknown = sorted(set(document) - _CONFIG_KEYS)
if unknown:
raise DocForgeError("invalid_config", "graph_render has unknown fields", fields=unknown)
output_root = confined_path(
root,
document.get("output_root"),
field="graph_render.output_root",
must_exist=False,
)
protected = [*content_roots, cache_root, changeset_root]
if manual_render is not None:
protected.extend((manual_render.template_root, manual_render.preview_root))
protected.extend(view.output_path for view in manual_render.views)
if any(_overlaps(output_root, path) for path in protected):
raise DocForgeError(
"invalid_config",
"Portable graph output must not overlap canonical or other derived roots",
)
protected_files = (descriptor_path, index_path, *authority_files)
if any(path == output_root or path.is_relative_to(output_root) for path in protected_files):
raise DocForgeError(
"invalid_config",
"Portable graph output overlaps a protected project path",
)
view_values_value = document.get("views")
if not isinstance(view_values_value, list) or not view_values_value:
raise DocForgeError(
"invalid_config",
"graph_render.views must contain at least one view",
)
view_values = cast(list[object], view_values_value)
if len(view_values) > limits.max_render_views:
raise DocForgeError("invalid_config", "graph_render.views exceeds the configured limit")
views: list[GraphRenderView] = []
view_ids: set[str] = set()
outputs: set[Path] = set()
for value in view_values:
if not isinstance(value, dict):
raise DocForgeError("invalid_config", "Each portable graph view must be a table")
view = cast(dict[str, object], value)
unknown_view = sorted(set(view) - _VIEW_KEYS)
if unknown_view:
raise DocForgeError(
"invalid_config",
"Portable graph view has unknown fields",
fields=unknown_view,
)
view_id = require_string(view, "id", descriptor_path)
if ID_PATTERN.fullmatch(view_id) is None or view_id in view_ids:
raise DocForgeError(
"invalid_config",
"Portable graph view ID is invalid or duplicated",
id=view_id,
)
view_ids.add(view_id)
renderer = require_string(view, "renderer", descriptor_path)
if renderer != "portable_graph_html":
raise DocForgeError(
"unsupported_renderer",
"Portable graph view names an unsupported built-in renderer",
renderer=renderer,
)
output = confined_path(
output_root,
view.get("output"),
field="graph_render.view.output",
must_exist=False,
)
if output.suffix != ".html" or output in outputs:
raise DocForgeError(
"invalid_config",
"Portable graph outputs must be unique HTML files",
)
outputs.add(output)
root_node_id = _optional_string(view, "root", descriptor_path)
query = _optional_string(view, "query", descriptor_path)
if (root_node_id is None) == (query is None):
raise DocForgeError(
"invalid_config",
"Portable graph view requires exactly one root or query",
)
if root_node_id is not None and len(root_node_id) > MAX_GRAPH_VIEW_STRING_CHARS:
raise DocForgeError("invalid_config", "Portable graph root exceeds its fixed limit")
if query is not None and len(query) > MAX_GRAPH_VIEW_QUERY_CHARS:
raise DocForgeError("invalid_config", "Portable graph query exceeds its fixed limit")
initial_mode_value = view.get("initial_mode", "nodes")
if not isinstance(initial_mode_value, str):
raise DocForgeError(
"invalid_config",
"Portable graph initial mode is unsupported",
)
initial_mode = cast(
Literal["nodes", "flow", "web", "logic"],
initial_mode_value,
)
if initial_mode not in _MODES:
raise DocForgeError(
"invalid_config",
"Portable graph initial mode is unsupported",
)
depth = positive_int(view.get("depth", 1), "graph_render.view.depth")
max_nodes = positive_int(view.get("max_nodes", 100), "graph_render.view.max_nodes")
max_edges = positive_int(
view.get("max_edges", 400),
"graph_render.view.max_edges",
allow_zero=True,
)
max_work = positive_int(view.get("max_work", 100_000), "graph_render.view.max_work")
if (
depth > min(limits.max_traversal_depth, MAX_GRAPH_VIEW_DEPTH)
or max_nodes > min(limits.max_nodes, MAX_GRAPH_VIEW_NODES)
or max_edges > MAX_GRAPH_VIEW_EDGES
or max_work > MAX_GRAPH_VIEW_WORK
):
raise DocForgeError(
"invalid_config",
"Portable graph view exceeds project or fixed safety limits",
)
include_logic = view.get("include_logic", False)
if type(include_logic) is not bool:
raise DocForgeError(
"invalid_config",
"Portable graph include_logic must be Boolean",
)
if include_logic:
raise DocForgeError(
"unsupported_renderer",
"Portable graph renderer version 1 does not support Logic projections",
)
views.append(
GraphRenderView(
view_id=view_id,
renderer=renderer,
output_path=output,
title=_bounded_string(
view,
"title",
descriptor_path,
maximum=MAX_GRAPH_VIEW_STRING_CHARS,
),
root_node_id=root_node_id,
query=query,
initial_mode=initial_mode,
depth=depth,
max_nodes=max_nodes,
max_edges=max_edges,
max_work=max_work,
families=_bounded_strings(
view.get("families", []),
key="graph_render.view.families",
source=descriptor_path,
),
relations=_bounded_strings(
view.get("relations", []),
key="graph_render.view.relations",
source=descriptor_path,
),
authorities=_bounded_strings(
view.get("authorities", []),
key="graph_render.view.authorities",
source=descriptor_path,
),
statuses=_bounded_strings(
view.get("statuses", []),
key="graph_render.view.statuses",
source=descriptor_path,
),
tags=_bounded_strings(
view.get("tags", []),
key="graph_render.view.tags",
source=descriptor_path,
),
include_logic=include_logic,
)
)
return GraphRenderConfig(
output_root=output_root,
views=tuple(sorted(views, key=lambda item: item.view_id)),
)

815
src/docforge/graph_rendering.py
View file

@ -1,815 +0,0 @@
"""Declared portable graph planning, publication, and receipt-only status."""
from __future__ import annotations
import fcntl
import json
import os
from collections.abc import Callable, Generator
from contextlib import contextmanager
from pathlib import Path
from typing import cast
from ._fs_safety import (
atomic_replace_bytes_at,
open_confined_directory,
read_bounded_file_at,
require_bound_directory,
safe_file_identity_at,
)
from .errors import DocForgeError
from .graph_projection import (
GraphViewRequestV1,
build_graph_projection_package,
build_graph_view_plan,
)
from .models import (
GenerationRecordingProject,
GraphRenderConfig,
GraphRenderView,
IncrementalStateProject,
ProjectService,
ProjectSnapshot,
ProjectState,
)
from .project import project_root_fingerprint
from .projection_contract import GraphViewPlanV1, ProjectionReceiptV1, projection_hash
from .projection_policy import (
PortableGraphProjectionMode,
validate_portable_graph_projection_mode,
)
from .projection_worker import render_projection_in_worker
GRAPH_RENDERER_ID = "portable_graph_html"
GRAPH_RENDERER_VERSION = "1"
GRAPH_PUBLICATION_MANIFEST_VERSION = 1
GRAPH_PUBLICATION_CONTRACT = "docforge.graph-publication"
MAX_GRAPH_PUBLICATION_BYTES = 256_000
class GraphRenderService:
"""Publish one declared artifact while keeping planning and rendering independent."""
def __init__(
self,
project: ProjectService,
*,
allow_logic: bool = False,
portable_graph_policy: PortableGraphProjectionMode = "explicit",
) -> None:
self.project = project
self.allow_logic = allow_logic
self.portable_graph_policy = validate_portable_graph_projection_mode(portable_graph_policy)
def _require_rendering(self, operation: str) -> None:
if self.portable_graph_policy == "disabled":
raise DocForgeError(
"projection_policy_forbids_operation",
"Portable graph projection policy disables rendering work",
projection="portable_graph",
mode=self.portable_graph_policy,
operation=operation,
)
def plan(self, view_id: str) -> dict[str, object]:
self._require_rendering("plan")
snapshot = self.project.load()
view = self._view(self._config(snapshot), view_id)
plan = self._plan(snapshot, view)
return {
"status": "ok",
**self._identity(snapshot),
"view_id": view.view_id,
"plan": plan.as_dict(),
}
def status(self, view_id: str | None = None) -> dict[str, object]:
config = self.project.descriptor.graph_render
current = self._current_state()
if config is None:
return self._status_result(
current,
configured=False,
state="not_configured",
outputs=[],
)
views = config.views if view_id is None else (self._view(config, view_id),)
first_outputs = [self._manifest_status(view, current) for view in views]
outputs = [self._manifest_status(view, current) for view in views]
if outputs != first_outputs:
for output in outputs:
if output["state"] == "current":
output["state"] = "stale"
output["reason"] = "publication_changed_during_status"
final = self._current_state()
if final != current:
for output in outputs:
if output["state"] == "current":
output["state"] = "stale"
output["reason"] = "source_changed_during_status"
identity = final if final is not None else current
return self._status_result(
identity,
configured=True,
state="current" if all(item["state"] == "current" for item in outputs) else "stale",
outputs=outputs,
)
def render(self, view_id: str) -> dict[str, object]:
self._require_rendering("render")
with self._lock():
current_status = self.status(view_id)
current_outputs = cast(list[dict[str, object]], current_status["outputs"])
if current_status["state"] == "current" and current_outputs:
return {
**current_status,
"publication": "unchanged",
"output": current_outputs[0],
}
snapshot = self.project.load()
view = self._view(self._config(snapshot), view_id)
plan = self._plan(snapshot, view)
package = build_graph_projection_package(
plan,
renderer_id=GRAPH_RENDERER_ID,
renderer_version=GRAPH_RENDERER_VERSION,
max_output_bytes=snapshot.descriptor.limits.max_render_bytes,
)
result = render_projection_in_worker(package)
if len(result.artifacts) != 1:
raise DocForgeError(
"invalid_projection",
"Portable graph renderer returned an unsupported artifact set",
)
artifact = result.artifacts[0]
def verify() -> None:
current = self.project.load()
if (
current.revision != snapshot.revision
or current.source_hash != snapshot.source_hash
):
raise DocForgeError(
"render_input_changed",
"Canonical input changed during portable graph rendering",
)
verify()
if isinstance(self.project, GenerationRecordingProject):
self.project.record_generation(snapshot)
artifact_evidence = artifact.evidence()
try:
store_identity = self._publish_artifact(
snapshot,
artifact_evidence["sha256"],
artifact.content,
verify=verify,
)
except DocForgeError as error:
if self._mutation_committed(error):
return self._degraded_publication(
snapshot,
view,
plan,
package.package_id,
result.receipt.as_dict(),
artifact_evidence,
stage="artifact_store",
error=error,
output_published=False,
)
raise
try:
output_identity = self._publish_output(
snapshot,
view,
artifact.content,
verify=verify,
)
except DocForgeError as error:
if self._mutation_committed(error):
return self._degraded_publication(
snapshot,
view,
plan,
package.package_id,
result.receipt.as_dict(),
artifact_evidence,
stage="output",
error=error,
output_published=True,
)
raise
manifest = self._manifest(
snapshot,
view,
plan,
package.package_id,
result.receipt.as_dict(),
artifact_evidence,
store_identity,
output_identity,
)
try:
self._publish_manifest(snapshot, view, manifest, verify=verify)
except DocForgeError as error:
return self._degraded_publication(
snapshot,
view,
plan,
package.package_id,
result.receipt.as_dict(),
artifact_evidence,
stage="manifest",
error=error,
output_published=True,
)
return {
"status": "ok",
**self._identity(snapshot),
"view_id": view.view_id,
"state": "current",
"publication": "published",
"plan_id": plan.plan_id,
"package_id": package.package_id,
"output": {
**artifact.evidence(),
"path": view.output_path.relative_to(snapshot.descriptor.root).as_posix(),
},
"receipt": result.receipt.as_dict(),
"manifest": {
"state": "current",
"publication_id": manifest["publication_id"],
},
}
@staticmethod
def _mutation_committed(error: DocForgeError) -> bool:
return error.details.get("mutation_committed") is True
def _degraded_publication(
self,
snapshot: ProjectSnapshot,
view: GraphRenderView,
plan: GraphViewPlanV1,
package_id: str,
receipt: dict[str, object],
artifact: dict[str, object],
*,
stage: str,
error: DocForgeError,
output_published: bool,
) -> dict[str, object]:
return {
"status": "ok",
**self._identity(snapshot),
"view_id": view.view_id,
"state": "degraded",
"publication": "published" if output_published else "partial",
"committed_stage": stage,
"plan_id": plan.plan_id,
"package_id": package_id,
"artifact": artifact,
"output": {
**artifact,
"path": view.output_path.relative_to(snapshot.descriptor.root).as_posix(),
"state": "unverified" if output_published else "not_published",
},
"receipt": receipt,
"manifest": {
"state": "failed",
"error": error.as_dict(),
},
}
def _manifest_status(
self,
view: GraphRenderView,
current: ProjectState | None,
) -> dict[str, object]:
manifest = self._read_manifest(view)
base = {
"view_id": view.view_id,
"renderer": GRAPH_RENDERER_ID,
"renderer_version": GRAPH_RENDERER_VERSION,
"path": view.output_path.relative_to(self.project.descriptor.root).as_posix(),
"verification": "manifest",
}
if manifest is None:
return {**base, "state": "missing", "reason": "manifest_missing"}
if not self._valid_manifest(view, manifest):
return {**base, "state": "unverified", "reason": "manifest_invalid"}
if current is None:
reason = (
"source_generation_changed"
if isinstance(self.project, GenerationRecordingProject)
else "source_generation_unavailable"
)
return {
**base,
"state": (
"stale"
if isinstance(self.project, GenerationRecordingProject)
else "unverified"
),
"reason": reason,
"plan_id": manifest.get("plan_id"),
"package_id": manifest.get("package_id"),
}
project = cast(dict[str, object], manifest["project"])
if project["revision"] != current.revision or project["source_hash"] != current.source_hash:
return {
**base,
"state": "stale",
"reason": "source_generation_changed",
"plan_id": manifest["plan_id"],
"package_id": manifest["package_id"],
}
artifact = cast(dict[str, object], manifest["artifact"])
store = cast(dict[str, object], manifest["store"])
artifact_root = self.project.descriptor.cache_root / "projection-artifacts"
if not artifact_root.exists():
return {
**base,
"state": "stale",
"reason": "artifact_store_missing",
"plan_id": manifest["plan_id"],
"package_id": manifest["package_id"],
}
if artifact_root.is_symlink() or not artifact_root.is_dir():
return {
**base,
"state": "unsafe",
"reason": "artifact_store_unsafe",
"plan_id": manifest["plan_id"],
"package_id": manifest["package_id"],
}
artifact_directory: int | None = None
try:
artifact_directory = open_confined_directory(
self.project.descriptor.root,
artifact_root,
create=False,
)
artifact_identity = safe_file_identity_at(
artifact_root,
artifact_directory,
f"{artifact['sha256']}.html",
)
except DocForgeError:
return {
**base,
"state": "unsafe",
"reason": "artifact_store_unsafe",
"plan_id": manifest["plan_id"],
"package_id": manifest["package_id"],
}
finally:
if artifact_directory is not None:
os.close(artifact_directory)
if artifact_identity is None:
return {
**base,
"state": "stale",
"reason": "artifact_store_missing",
"plan_id": manifest["plan_id"],
"package_id": manifest["package_id"],
}
if artifact_identity != store:
return {
**base,
"state": "stale",
"reason": "artifact_store_changed",
"plan_id": manifest["plan_id"],
"package_id": manifest["package_id"],
}
try:
directory = open_confined_directory(
self.project.descriptor.root,
view.output_path.parent,
create=False,
)
except DocForgeError:
return {**base, "state": "unsafe", "reason": "output_root_unsafe"}
try:
identity = safe_file_identity_at(
view.output_path.parent, directory, view.output_path.name
)
except DocForgeError:
return {**base, "state": "unsafe", "reason": "output_unsafe"}
finally:
os.close(directory)
expected = cast(dict[str, object], manifest["output"])
if identity != expected:
return {
**base,
"state": "stale",
"reason": "output_changed",
"plan_id": manifest["plan_id"],
"package_id": manifest["package_id"],
}
return {
**base,
"state": "current",
"reason": None,
"plan_id": manifest["plan_id"],
"package_id": manifest["package_id"],
"publication_id": manifest["publication_id"],
"artifact": manifest["artifact"],
}
def _read_manifest(self, view: GraphRenderView) -> dict[str, object] | None:
root = self._manifest_root()
if not root.is_dir() or root.is_symlink():
return None
try:
descriptor = open_confined_directory(
self.project.descriptor.root,
root,
create=False,
)
except DocForgeError:
return None
try:
raw = read_bounded_file_at(
descriptor,
f"{view.view_id}.json",
MAX_GRAPH_PUBLICATION_BYTES,
)
except DocForgeError:
return None
finally:
os.close(descriptor)
if raw is None:
return None
try:
value: object = json.loads(raw)
except (UnicodeDecodeError, json.JSONDecodeError):
return None
return cast(dict[str, object], value) if isinstance(value, dict) else None
def _valid_manifest(self, view: GraphRenderView, manifest: dict[str, object]) -> bool:
required = {
"schema_version",
"contract",
"publication_id",
"project",
"view_id",
"view_config_hash",
"plan_id",
"package_id",
"renderer",
"receipt",
"artifact",
"store",
"output",
}
try:
if (
set(manifest) != required
or manifest.get("schema_version") != GRAPH_PUBLICATION_MANIFEST_VERSION
or manifest.get("contract") != GRAPH_PUBLICATION_CONTRACT
or manifest.get("view_id") != view.view_id
or manifest.get("view_config_hash") != self._view_hash(view)
or not self._hash(manifest.get("plan_id"))
or not self._hash(manifest.get("package_id"))
):
return False
project = manifest.get("project")
descriptor = self.project.descriptor
if not isinstance(project, dict):
return False
project_document = cast(dict[str, object], project)
if (
set(project_document)
!= {
"project_id",
"project_root_fingerprint",
"adapter",
"revision",
"source_hash",
}
or project_document.get("project_id") != descriptor.project_id
or project_document.get("project_root_fingerprint")
!= project_root_fingerprint(descriptor.root)
or project_document.get("adapter") != descriptor.adapter
or not isinstance(project_document.get("revision"), str)
or not project_document["revision"]
or not self._hash(project_document.get("source_hash"))
):
return False
renderer = manifest.get("renderer")
if renderer != {
"renderer_id": GRAPH_RENDERER_ID,
"renderer_version": GRAPH_RENDERER_VERSION,
}:
return False
receipt_value = manifest.get("receipt")
if not isinstance(receipt_value, dict):
return False
receipt = ProjectionReceiptV1.from_dict(
dict(cast(dict[str, object], receipt_value))
).as_dict()
artifacts = receipt.get("artifacts")
if not isinstance(artifacts, list):
return False
artifact_values = cast(list[object], artifacts)
if (
receipt.get("kind") != "graph"
or receipt.get("plan_id") != manifest["plan_id"]
or receipt.get("package_id") != manifest["package_id"]
or receipt.get("renderer") != renderer
or len(artifact_values) != 1
or manifest.get("artifact") != artifact_values[0]
):
return False
artifact = artifact_values[0]
if not isinstance(artifact, dict):
return False
artifact_document = cast(dict[str, object], artifact)
if (
artifact_document.get("artifact_id") != "portable-graph.html"
or artifact_document.get("media_type") != "text/html; charset=utf-8"
):
return False
artifact_hash = artifact_document.get("sha256")
artifact_bytes = artifact_document.get("bytes")
store = manifest.get("store")
output = manifest.get("output")
if (
not self._file_identity(store, expected_name=f"{artifact_hash}.html")
or not self._file_identity(output, expected_name=view.output_path.name)
or type(artifact_bytes) is not int
or cast(dict[str, object], store)["size"] != artifact_bytes
or cast(dict[str, object], output)["size"] != artifact_bytes
):
return False
body = dict(manifest)
publication_id = body.pop("publication_id", None)
return self._hash(publication_id) and publication_id == projection_hash(body)
except (DocForgeError, KeyError, TypeError, ValueError):
return False
@staticmethod
def _hash(value: object) -> bool:
return (
isinstance(value, str)
and len(value) == 64
and all(character in "0123456789abcdef" for character in value)
)
@staticmethod
def _file_identity(value: object, *, expected_name: str) -> bool:
if not isinstance(value, dict):
return False
document = cast(dict[str, object], value)
required = {"path", "device", "inode", "mode", "size", "mtime_ns", "ctime_ns"}
return (
set(document) == required
and document.get("path") == expected_name
and all(
type(document.get(field)) is int and cast(int, document[field]) >= 0
for field in required - {"path"}
)
)
def _manifest(
self,
snapshot: ProjectSnapshot,
view: GraphRenderView,
plan: GraphViewPlanV1,
package_id: str,
receipt: dict[str, object],
artifact: dict[str, object],
store: dict[str, object],
output: dict[str, object],
) -> dict[str, object]:
body: dict[str, object] = {
"schema_version": GRAPH_PUBLICATION_MANIFEST_VERSION,
"contract": GRAPH_PUBLICATION_CONTRACT,
"project": self._identity(snapshot),
"view_id": view.view_id,
"view_config_hash": self._view_hash(view),
"plan_id": plan.plan_id,
"package_id": package_id,
"renderer": {
"renderer_id": GRAPH_RENDERER_ID,
"renderer_version": GRAPH_RENDERER_VERSION,
},
"receipt": receipt,
"artifact": artifact,
"store": store,
"output": output,
}
return {**body, "publication_id": projection_hash(body)}
def _publish_artifact(
self,
snapshot: ProjectSnapshot,
artifact_hash: object,
content: bytes,
*,
verify: Callable[[], None],
) -> dict[str, object]:
if not isinstance(artifact_hash, str):
raise DocForgeError("invalid_projection", "Artifact hash is invalid")
root = snapshot.descriptor.cache_root / "projection-artifacts"
descriptor = open_confined_directory(snapshot.descriptor.root, root, create=True)
name = f"{artifact_hash}.html"
try:
try:
existing = read_bounded_file_at(descriptor, name, len(content))
except DocForgeError as error:
if error.code != "invalid_projection":
raise
existing = None
if existing == content:
identity = safe_file_identity_at(root, descriptor, name)
assert identity is not None
return identity
return atomic_replace_bytes_at(root, descriptor, name, content, verify=verify)
finally:
os.close(descriptor)
def _publish_output(
self,
snapshot: ProjectSnapshot,
view: GraphRenderView,
content: bytes,
*,
verify: Callable[[], None],
) -> dict[str, object]:
root = view.output_path.parent
descriptor = open_confined_directory(snapshot.descriptor.root, root, create=True)
try:
try:
existing = read_bounded_file_at(
descriptor,
view.output_path.name,
len(content),
)
except DocForgeError as error:
if error.code != "invalid_projection":
raise
existing = None
if existing == content:
identity = safe_file_identity_at(root, descriptor, view.output_path.name)
assert identity is not None
return identity
return atomic_replace_bytes_at(
root,
descriptor,
view.output_path.name,
content,
verify=verify,
)
finally:
os.close(descriptor)
def _publish_manifest(
self,
snapshot: ProjectSnapshot,
view: GraphRenderView,
manifest: dict[str, object],
*,
verify: Callable[[], None],
) -> None:
raw = json.dumps(manifest, sort_keys=True, indent=2).encode() + b"\n"
if len(raw) > MAX_GRAPH_PUBLICATION_BYTES:
raise DocForgeError(
"projection_too_large",
"Portable graph publication manifest exceeds its fixed limit",
)
root = self._manifest_root()
descriptor = open_confined_directory(snapshot.descriptor.root, root, create=True)
try:
atomic_replace_bytes_at(
root,
descriptor,
f"{view.view_id}.json",
raw,
verify=verify,
)
finally:
os.close(descriptor)
@contextmanager
def _lock(self) -> Generator[None]:
root = self.project.descriptor.cache_root
descriptor = open_confined_directory(self.project.descriptor.root, root, create=True)
lock_descriptor: int | None = None
try:
lock_descriptor = os.open(
"graph-render.lock",
os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW,
0o600,
dir_fd=descriptor,
)
fcntl.flock(lock_descriptor, fcntl.LOCK_EX)
require_bound_directory(root, descriptor)
yield
except OSError as error:
raise DocForgeError(
"publication_failure",
"Portable graph render lock is unavailable",
) from error
finally:
if lock_descriptor is not None:
os.close(lock_descriptor)
os.close(descriptor)
def _plan(self, snapshot: ProjectSnapshot, view: GraphRenderView) -> GraphViewPlanV1:
return build_graph_view_plan(
snapshot,
GraphViewRequestV1(
view_id=view.view_id,
title=view.title,
root_node_id=view.root_node_id,
query=view.query,
initial_mode=view.initial_mode,
depth=view.depth,
max_nodes=view.max_nodes,
max_edges=view.max_edges,
max_work=view.max_work,
families=view.families,
relations=view.relations,
authorities=view.authorities,
statuses=view.statuses,
tags=view.tags,
include_logic=view.include_logic,
),
self.allow_logic,
)
def _current_state(self) -> ProjectState | None:
if isinstance(self.project, IncrementalStateProject):
return self.project.incremental_state()
return None
def _config(self, snapshot: ProjectSnapshot) -> GraphRenderConfig:
config = snapshot.descriptor.graph_render
if config is None:
raise DocForgeError(
"graph_render_not_configured",
"Project has no portable graph render configuration",
)
return config
@staticmethod
def _view(config: GraphRenderConfig, view_id: str) -> GraphRenderView:
for view in config.views:
if view.view_id == view_id:
return view
raise DocForgeError(
"unknown_graph_render_view",
"Portable graph view is not declared",
view_id=view_id,
)
def _manifest_root(self) -> Path:
return self.project.descriptor.cache_root / "projection-publications" / "graph"
@staticmethod
def _view_hash(view: GraphRenderView) -> str:
return projection_hash(
{
"view_id": view.view_id,
"renderer": view.renderer,
"title": view.title,
"root_node_id": view.root_node_id,
"query": view.query,
"initial_mode": view.initial_mode,
"depth": view.depth,
"max_nodes": view.max_nodes,
"max_edges": view.max_edges,
"max_work": view.max_work,
"families": list(view.families),
"relations": list(view.relations),
"authorities": list(view.authorities),
"statuses": list(view.statuses),
"tags": list(view.tags),
"include_logic": view.include_logic,
}
)
def _status_result(self, current: ProjectState | None, **payload: object) -> dict[str, object]:
descriptor = self.project.descriptor
return {
"status": "ok",
"project_id": descriptor.project_id,
"project_root_fingerprint": project_root_fingerprint(descriptor.root),
"adapter": descriptor.adapter,
"revision": current.revision if current is not None else "unknown",
"source_hash": current.source_hash if current is not None else None,
**payload,
}
@staticmethod
def _identity(snapshot: ProjectSnapshot) -> dict[str, object]:
return {
"project_id": snapshot.descriptor.project_id,
"project_root_fingerprint": project_root_fingerprint(snapshot.descriptor.root),
"adapter": snapshot.descriptor.adapter,
"revision": snapshot.revision,
"source_hash": snapshot.source_hash,
}

208
src/docforge/manual_projection.py
View file

@ -1,208 +0,0 @@
"""Pure manual planning over one immutable validated graph generation."""
from __future__ import annotations
import hashlib
from collections import defaultdict
from collections.abc import Sequence
from .errors import DocForgeError
from .models import Edge, ProjectSnapshot, RenderView
from .project import project_root_fingerprint
from .projection_contract import ManualRenderPlanV1, ProjectionPackageV1
def _edge_dict(edge: Edge) -> dict[str, str]:
return {
"source_id": edge.source_id,
"relation": edge.relation,
"target_id": edge.target_id,
}
def _cycles(node_ids: tuple[str, ...], edges: tuple[Edge, ...]) -> list[list[str]]:
"""Return deterministic strongly connected components that represent cycles."""
adjacency: dict[str, list[str]] = {node_id: [] for node_id in node_ids}
reverse_adjacency: dict[str, list[str]] = {node_id: [] for node_id in node_ids}
for edge in edges:
adjacency[edge.source_id].append(edge.target_id)
reverse_adjacency[edge.target_id].append(edge.source_id)
for targets in (*adjacency.values(), *reverse_adjacency.values()):
targets.sort()
visited: set[str] = set()
finished: list[str] = []
for node_id in node_ids:
if node_id in visited:
continue
visited.add(node_id)
traversal: list[tuple[str, int]] = [(node_id, 0)]
while traversal:
current, position = traversal[-1]
targets = adjacency[current]
if position < len(targets):
target = targets[position]
traversal[-1] = (current, position + 1)
if target not in visited:
visited.add(target)
traversal.append((target, 0))
continue
finished.append(current)
traversal.pop()
assigned: set[str] = set()
components: list[list[str]] = []
for node_id in reversed(finished):
if node_id in assigned:
continue
assigned.add(node_id)
component: list[str] = []
component_stack = [node_id]
while component_stack:
current = component_stack.pop()
component.append(current)
for target in reversed(reverse_adjacency[current]):
if target not in assigned:
assigned.add(target)
component_stack.append(target)
component.sort()
if len(component) > 1 or component[0] in adjacency[component[0]]:
components.append(component)
return sorted(components)
def build_manual_render_plan(
snapshot: ProjectSnapshot,
view: RenderView,
*,
changeset_hash: str | None,
) -> ManualRenderPlanV1:
"""Select and describe a complete manual without rendering markup."""
selected = tuple(
node for node in snapshot.nodes if not view.families or node.family in view.families
)
selected_ids = {node.node_id for node in selected}
edges = tuple(
edge
for edge in snapshot.edges
if edge.source_id in selected_ids and edge.target_id in selected_ids
)
outgoing: dict[str, list[Edge]] = defaultdict(list)
incoming: dict[str, list[Edge]] = defaultdict(list)
for edge in edges:
outgoing[edge.source_id].append(edge)
incoming[edge.target_id].append(edge)
for values in (*outgoing.values(), *incoming.values()):
values.sort(key=lambda edge: (edge.source_id, edge.relation, edge.target_id))
pages = [
{
"node_id": node.node_id,
"title": node.title,
"family": node.family,
"authority": node.authority,
"status": node.status,
"tags": list(node.tags),
"summary": node.summary,
"content": node.content,
"content_hash": node.content_hash,
"components": [
"manual.node-metadata@1",
"manual.summary@1",
"manual.commonmark@1",
"manual.relationships@1",
],
"breadcrumbs": [],
"cross_references": [_edge_dict(edge) for edge in outgoing[node.node_id]],
"backlinks": [_edge_dict(edge) for edge in incoming[node.node_id]],
}
for node in selected
]
node_ids = tuple(node.node_id for node in selected)
connected = {endpoint for edge in edges for endpoint in (edge.source_id, edge.target_id)}
return ManualRenderPlanV1.create(
{
"project": {
"project_id": snapshot.descriptor.project_id,
"project_root_fingerprint": project_root_fingerprint(snapshot.descriptor.root),
"adapter": snapshot.descriptor.adapter,
"revision": snapshot.revision,
"source_hash": snapshot.source_hash,
},
"view": {
"view_id": view.view_id,
"title": view.title,
"families": list(view.families),
"renderer": view.renderer,
},
"changeset_hash": changeset_hash,
"pages": pages,
"navigation": [{"node_id": node.node_id, "title": node.title} for node in selected],
"search_documents": [
{
"node_id": node.node_id,
"title": node.title,
"summary": node.summary,
"family": node.family,
"status": node.status,
"tags": list(node.tags),
}
for node in selected
],
"diagnostics": {
"orphans": [node_id for node_id in node_ids if node_id not in connected],
"cycles": _cycles(node_ids, edges),
},
}
)
def build_manual_projection_package(
plan: ManualRenderPlanV1,
template_bytes: bytes,
*,
renderer_id: str,
renderer_version: str,
max_output_bytes: int,
render_identity: str | None = None,
fragment_records: Sequence[dict[str, object]] = (),
) -> ProjectionPackageV1:
"""Bind one plan and inert template asset for a path-free manual renderer."""
try:
template = template_bytes.decode("utf-8")
except UnicodeDecodeError as error:
raise DocForgeError("invalid_template", "Render template is not valid UTF-8") from error
template_asset: dict[str, object] = {
"asset_id": "manual.template",
"media_type": "text/html; charset=utf-8",
"sha256": hashlib.sha256(template_bytes).hexdigest(),
"text": template,
}
if render_identity is not None:
template_asset["render_identity"] = render_identity
assets = [template_asset]
if fragment_records:
assets.append(
{
"asset_id": "manual.fragments",
"media_type": "application/vnd.docforge.projection-fragments.v1+json",
"records": list(fragment_records),
}
)
return ProjectionPackageV1.create(
kind="manual",
plan=plan,
renderer={"renderer_id": renderer_id, "renderer_version": renderer_version},
components=[
{"component_id": "manual.document@1"},
{"component_id": "manual.commonmark@1"},
],
assets=assets,
output_policy={
"artifact_ids": ["manual.html"],
"max_total_bytes": max_output_bytes,
},
)

109
src/docforge/mcp_server.py
View file

@ -15,13 +15,11 @@ from .application import CanonicalApplicationService, CanonicalApplier, GenericC
from .changesets import ChangesetStore
from .context import compile_context
from .errors import DocForgeError
from .graph_rendering import GraphRenderService
from .index import ProjectIndex
from .models import IncrementalStateProject, ProjectService, RuntimeValidatedProject
from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt
from .policy import CapabilityMode, capability_mode, compose_effective_policy
from .project import Project, project_root_fingerprint
from .projection_policy import compose_projection_policy
from .rendering import RenderService
from .retrieval import MAX_TASK_EVIDENCE, TaskKind, build_retrieval_plan
from .telemetry import request, stage
@ -49,8 +47,6 @@ READ_TOOLS = (
"docforge_get_task_context",
"docforge_validate_project",
"docforge_render_status",
"docforge_graph_plan",
"docforge_graph_render_status",
"docforge_visualize",
"docforge_stop_visualization",
"docforge_visualization_status",
@ -138,9 +134,6 @@ class DocForgeService:
no_ast: bool = False,
diagnostics: bool = False,
capability_mode_name: str | None = None,
manual_projection_policy: str | None = None,
portable_graph_policy: str | None = None,
live_viewer_policy: str | None = None,
) -> None:
self.project = project
default_mode: CapabilityMode = (
@ -160,40 +153,19 @@ class DocForgeService:
render_configured=project.descriptor.render is not None,
application_enabled=application_enabled,
)
self.projection_policy = compose_projection_policy(
manual=manual_projection_policy,
portable_graph=portable_graph_policy,
live_viewer=live_viewer_policy,
manual_configured=project.descriptor.render is not None,
portable_graph_configured=project.descriptor.graph_render is not None,
application_enabled=application_enabled,
)
self.index = ProjectIndex(self.project, allow_logic=not self.policy.no_ast)
self.changesets = ChangesetStore(
self.project,
proposal_writer if selected_mode != "read" else None,
)
self.rendering = RenderService(
self.project,
self.changesets,
manual_policy=self.projection_policy.manual,
)
self.graph_rendering = GraphRenderService(
self.project,
allow_logic=not self.policy.no_ast,
portable_graph_policy=self.projection_policy.portable_graph,
)
self.rendering = RenderService(self.project, self.changesets)
self.application = CanonicalApplicationService(
self.project,
applier_id=canonical_applier_id if application_enabled else None,
applier=canonical_applier if application_enabled else None,
index=self.index,
manual_policy=self.projection_policy.manual,
)
self.visualization = ViewerManagerClient(
self.index,
live_viewer_policy=self.projection_policy.live_viewer,
)
self.visualization = ViewerManagerClient(self.index)
self.context_provider = context_provider
self.task_context_available = context_provider is compile_context
self.binding_metadata = dict(binding_metadata or {})
@ -649,7 +621,6 @@ class DocForgeService:
}
capabilities = self.capabilities()
effective_policy = self.policy.as_dict()
projection_policy = self.projection_policy.as_dict()
session_contract: dict[str, object] = {
"schema_version": 1,
"binding": binding,
@ -659,8 +630,6 @@ class DocForgeService:
"freshness": "current",
},
"effective_policy": effective_policy,
"projection_policy": projection_policy,
"projection_policy_hash": self.projection_policy.policy_hash,
"capabilities": capabilities,
"render_policies": {
"manual": effective_policy["manual_render"],
@ -683,8 +652,6 @@ class DocForgeService:
"canonical_paths": [str(path) for path in descriptor.content_roots],
"adapter_policy": self.adapter_policy(),
"effective_policy": effective_policy,
"projection_policy": projection_policy,
"projection_policy_hash": self.projection_policy.policy_hash,
"capabilities": capabilities,
"session_contract": session_contract,
"proposal_access": proposal_access,
@ -747,8 +714,6 @@ class DocForgeService:
),
"adapter_policy": self.adapter_policy(),
"effective_policy": self.policy.as_dict(),
"projection_policy": self.projection_policy.as_dict(),
"projection_policy_hash": self.projection_policy.policy_hash,
"capabilities": self.capabilities(),
"canonical_paths": [
*(relative(path) for path in snapshot.descriptor.content_roots),
@ -864,29 +829,6 @@ class DocForgeService:
operation_name="mcp.render_status",
)
def graph_plan(self, view_id: str) -> dict[str, object]:
"""Plan one declared portable graph without publishing derived output."""
return self.invoke(
lambda: self.graph_rendering.plan(view_id),
synchronize=False,
load_error_identity=False,
operation_name="mcp.graph_plan",
)
def graph_render_status(
self,
view_id: str | None = None,
) -> dict[str, object]:
"""Report portable-graph publication state without planning or rendering."""
return self.invoke(
lambda: self.graph_rendering.status(view_id),
synchronize=False,
load_error_identity=False,
operation_name="mcp.graph_render_status",
)
def context(
self,
profile: str,
@ -1633,18 +1575,6 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
return service.render_status(view_id, deep=deep)
@server.tool(name="docforge_graph_plan")
def graph_plan(view_id: str) -> dict[str, Any]:
"""Plan one declared portable graph without publishing derived output."""
return service.graph_plan(view_id)
@server.tool(name="docforge_graph_render_status")
def graph_render_status(view_id: str | None = None) -> dict[str, Any]:
"""Report portable-graph publication state without rendering."""
return service.graph_render_status(view_id)
@server.tool(name="docforge_visualize")
def visualize(
node_id: str | None = None,
@ -1692,8 +1622,6 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
get_task_context,
validate_project,
render_status,
graph_plan,
graph_render_status,
visualize,
stop_visualization,
visualization_status,
@ -2076,9 +2004,6 @@ def create_server(
no_ast: bool = False,
diagnostics: bool = False,
capability_mode: str | None = None,
manual_projection_policy: str | None = None,
portable_graph_policy: str | None = None,
live_viewer_policy: str | None = None,
) -> FastMCP:
project = Project.open(project_root)
return create_project_server(
@ -2095,9 +2020,6 @@ def create_server(
no_ast=no_ast,
diagnostics=diagnostics,
capability_mode=capability_mode,
manual_projection_policy=manual_projection_policy,
portable_graph_policy=portable_graph_policy,
live_viewer_policy=live_viewer_policy,
)
@ -2112,9 +2034,6 @@ def create_project_server(
no_ast: bool = False,
diagnostics: bool = False,
capability_mode: str | None = None,
manual_projection_policy: str | None = None,
portable_graph_policy: str | None = None,
live_viewer_policy: str | None = None,
) -> FastMCP:
"""Create the full fixed MCP surface for one explicitly configured project service."""
@ -2128,9 +2047,6 @@ def create_project_server(
no_ast=no_ast,
diagnostics=diagnostics,
capability_mode_name=capability_mode,
manual_projection_policy=manual_projection_policy,
portable_graph_policy=portable_graph_policy,
live_viewer_policy=live_viewer_policy,
)
return _create_bound_server(
service,
@ -2146,9 +2062,6 @@ def create_read_only_server(
no_ast: bool = False,
diagnostics: bool = False,
capability_mode: str | None = None,
manual_projection_policy: str | None = None,
portable_graph_policy: str | None = None,
live_viewer_policy: str | None = None,
) -> FastMCP:
"""Create an adapter-capable MCP server exposing only the fixed read tool surface."""
@ -2160,9 +2073,6 @@ def create_read_only_server(
no_ast=no_ast,
diagnostics=diagnostics,
capability_mode_name="read" if capability_mode is None else capability_mode,
manual_projection_policy=manual_projection_policy,
portable_graph_policy=portable_graph_policy,
live_viewer_policy=live_viewer_policy,
)
if service.policy.capability_mode != "read":
raise DocForgeError(
@ -2196,18 +2106,6 @@ def main() -> None:
choices=("read", "proposal", "application", "operator"),
help="Expose the versioned project-bound capability surface",
)
parser.add_argument(
"--manual-render-policy",
choices=("auto", "explicit", "disabled"),
)
parser.add_argument(
"--portable-graph-policy",
choices=("explicit", "disabled"),
)
parser.add_argument(
"--live-viewer-policy",
choices=("on-demand", "disabled"),
)
arguments = parser.parse_args()
create_server(
arguments.project_root,
@ -2216,9 +2114,6 @@ def main() -> None:
no_ast=arguments.no_ast,
diagnostics=arguments.diagnostics,
capability_mode=arguments.capability_mode,
manual_projection_policy=arguments.manual_render_policy,
portable_graph_policy=arguments.portable_graph_policy,
live_viewer_policy=arguments.live_viewer_policy,
).run(transport="stdio")

30
src/docforge/models.py
View file

@ -5,7 +5,7 @@ from __future__ import annotations
from collections.abc import Mapping
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Literal, Protocol, runtime_checkable
from typing import Protocol, runtime_checkable
@dataclass(frozen=True)
@ -49,33 +49,6 @@ class RenderConfig:
views: tuple[RenderView, ...]
@dataclass(frozen=True)
class GraphRenderView:
view_id: str
renderer: str
output_path: Path
title: str
root_node_id: str | None
query: str | None
initial_mode: Literal["nodes", "flow", "web", "logic"]
depth: int
max_nodes: int
max_edges: int
max_work: int
families: tuple[str, ...]
relations: tuple[str, ...]
authorities: tuple[str, ...]
statuses: tuple[str, ...]
tags: tuple[str, ...]
include_logic: bool
@dataclass(frozen=True)
class GraphRenderConfig:
output_root: Path
views: tuple[GraphRenderView, ...]
@dataclass(frozen=True)
class ContextProfile:
profile_id: str
@ -105,7 +78,6 @@ class ProjectDescriptor:
allowed_relations: tuple[str, ...]
profiles: tuple[ContextProfile, ...]
limits: Limits
graph_render: GraphRenderConfig | None = None
@dataclass(frozen=True)

16
src/docforge/project.py
View file

@ -25,7 +25,6 @@ from .config_validation import (
string_list,
)
from .errors import DocForgeError
from .graph_render_config import load_graph_render_config
from .models import (
ContextProfile,
Edge,
@ -67,7 +66,6 @@ _DESCRIPTOR_KEYS = frozenset(
"derived",
"changesets",
"render",
"graph_render",
"graph",
"limits",
"profiles",
@ -550,6 +548,7 @@ def _load_descriptor(root: Path) -> ProjectDescriptor:
for field in defaults.__dataclass_fields__
}
)
render = load_render_config(
root,
document.get("render"),
@ -561,18 +560,6 @@ def _load_descriptor(root: Path) -> ProjectDescriptor:
changeset_root=changeset_root,
limits=limits,
)
graph_render = load_graph_render_config(
root,
document.get("graph_render"),
descriptor_path=descriptor_path,
content_roots=content_roots,
authority_files=authority_files,
cache_root=cache_root,
index_path=index_path,
changeset_root=changeset_root,
manual_render=render,
limits=limits,
)
profile_documents = document.get("profiles", [])
if not isinstance(profile_documents, list):
@ -636,7 +623,6 @@ def _load_descriptor(root: Path) -> ProjectDescriptor:
changeset_root=changeset_root,
proposal_writers=tuple(sorted(proposal_writers, key=lambda writer: writer.writer_id)),
render=render,
graph_render=graph_render,
allowed_relations=allowed_relations,
profiles=tuple(profiles),
limits=limits,

536
src/docforge/projection_contract.py
View file

@ -1,536 +0,0 @@
"""Versioned, canonical contracts shared by independent projection renderers."""
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from typing import Literal, cast
from .errors import DocForgeError
MANUAL_RENDER_PLAN_CONTRACT = "docforge.manual-render-plan"
GRAPH_VIEW_PLAN_CONTRACT = "docforge.graph-view-plan"
PROJECTION_PACKAGE_CONTRACT = "docforge.projection-package"
PROJECTION_RECEIPT_CONTRACT = "docforge.projection-receipt"
PROJECTION_SCHEMA_VERSION = 1
MAX_PLAN_BYTES = 16_000_000
MAX_PACKAGE_BYTES = 24_000_000
MAX_RECEIPT_BYTES = 128_000
MAX_PROJECTION_ARTIFACTS = 32
MAX_GRAPH_VIEW_DEPTH = 32
MAX_GRAPH_VIEW_NODES = 1_000
MAX_GRAPH_VIEW_EDGES = 4_000
MAX_GRAPH_VIEW_WORK = 1_000_000
MAX_GRAPH_VIEW_FILTERS = 64
MAX_GRAPH_VIEW_STRING_CHARS = 1_024
MAX_GRAPH_VIEW_QUERY_CHARS = 10_000
ProjectionKind = Literal["manual", "graph"]
def canonical_projection_bytes(value: object) -> bytes:
"""Return the one canonical UTF-8 representation used for projection identities."""
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode(
"utf-8"
)
def projection_hash(value: object) -> str:
return hashlib.sha256(canonical_projection_bytes(value)).hexdigest()
def _is_hash(value: object) -> bool:
return (
isinstance(value, str)
and len(value) == 64
and all(character in "0123456789abcdef" for character in value)
)
def _bounded(document: dict[str, object], maximum: int, *, kind: str) -> None:
size = len(canonical_projection_bytes(document))
if size > maximum:
raise DocForgeError(
"projection_too_large",
f"{kind} exceeds its fixed serialized-size limit",
maximum_bytes=maximum,
actual_bytes=size,
)
def _validated_identity(
document: dict[str, object],
*,
identity_field: str,
maximum: int,
kind: str,
) -> dict[str, object]:
_bounded(document, maximum, kind=kind)
identity = document.get(identity_field)
if not _is_hash(identity):
raise DocForgeError("invalid_projection", f"{kind} identity is invalid")
body = dict(document)
body.pop(identity_field)
if projection_hash(body) != identity:
raise DocForgeError("invalid_projection", f"{kind} identity does not match its content")
return document
def _reject_runtime_authority(value: object) -> None:
"""Reject structural capabilities while treating selected content as inert data."""
forbidden_keys = {
"command",
"database",
"database_path",
"index_path",
"project_root",
"project_path",
"sql",
}
if isinstance(value, dict):
payload = cast(dict[object, object], value)
for key, item in payload.items():
if isinstance(key, str) and key in forbidden_keys:
raise DocForgeError(
"invalid_projection",
"Projection package contains forbidden runtime authority",
field=key,
)
if (
isinstance(key, str)
and (key == "path" or key.endswith("_path"))
and isinstance(item, str)
and item.startswith("/")
):
raise DocForgeError(
"invalid_projection",
"Projection package contains an absolute runtime path",
field=key,
)
_reject_runtime_authority(item)
elif isinstance(value, list):
for item in cast(list[object], value):
_reject_runtime_authority(item)
@dataclass(frozen=True)
class ManualRenderPlanV1:
"""One immutable, bounded manual plan prepared from a validated graph generation."""
document: dict[str, object]
@property
def plan_id(self) -> str:
return cast(str, self.document["plan_id"])
def as_dict(self) -> dict[str, object]:
return dict(self.document)
@classmethod
def create(cls, payload: dict[str, object]) -> ManualRenderPlanV1:
body = {
"schema_version": PROJECTION_SCHEMA_VERSION,
"contract": MANUAL_RENDER_PLAN_CONTRACT,
**payload,
}
document = {**body, "plan_id": projection_hash(body)}
return cls(validate_manual_render_plan(document))
@classmethod
def from_dict(cls, document: dict[str, object]) -> ManualRenderPlanV1:
return cls(validate_manual_render_plan(dict(document)))
@dataclass(frozen=True)
class GraphViewPlanV1:
"""One immutable, bounded portable-graph plan."""
document: dict[str, object]
@property
def plan_id(self) -> str:
return cast(str, self.document["plan_id"])
def as_dict(self) -> dict[str, object]:
return dict(self.document)
@classmethod
def create(cls, payload: dict[str, object]) -> GraphViewPlanV1:
body = {
"schema_version": PROJECTION_SCHEMA_VERSION,
"contract": GRAPH_VIEW_PLAN_CONTRACT,
**payload,
}
document = {**body, "plan_id": projection_hash(body)}
return cls(validate_graph_view_plan(document))
@classmethod
def from_dict(cls, document: dict[str, object]) -> GraphViewPlanV1:
return cls(validate_graph_view_plan(dict(document)))
@dataclass(frozen=True)
class ProjectionPackageV1:
"""Path-free package supplied to one capability-isolated renderer."""
document: dict[str, object]
@property
def package_id(self) -> str:
return cast(str, self.document["package_id"])
@property
def kind(self) -> ProjectionKind:
return cast(ProjectionKind, self.document["kind"])
def as_dict(self) -> dict[str, object]:
return dict(self.document)
@classmethod
def create(
cls,
*,
kind: ProjectionKind,
plan: ManualRenderPlanV1 | GraphViewPlanV1,
renderer: dict[str, object],
components: list[dict[str, object]],
assets: list[dict[str, object]],
output_policy: dict[str, object],
) -> ProjectionPackageV1:
body: dict[str, object] = {
"schema_version": PROJECTION_SCHEMA_VERSION,
"contract": PROJECTION_PACKAGE_CONTRACT,
"kind": kind,
"plan_id": plan.plan_id,
"plan": plan.as_dict(),
"renderer": renderer,
"components": components,
"assets": assets,
"output_policy": output_policy,
}
document = {**body, "package_id": projection_hash(body)}
return cls(validate_projection_package(document))
@classmethod
def from_dict(cls, document: dict[str, object]) -> ProjectionPackageV1:
return cls(validate_projection_package(dict(document)))
@dataclass(frozen=True)
class ProjectionReceiptV1:
"""Renderer evidence that contains identities and sizes, never artifact bytes."""
document: dict[str, object]
@property
def receipt_id(self) -> str:
return cast(str, self.document["receipt_id"])
def as_dict(self) -> dict[str, object]:
return dict(self.document)
@classmethod
def create(
cls,
*,
kind: ProjectionKind,
package_id: str,
plan_id: str,
renderer: dict[str, object],
artifacts: list[dict[str, object]],
diagnostics: dict[str, object],
timing: dict[str, object],
peak_memory_bytes: int | None,
) -> ProjectionReceiptV1:
body: dict[str, object] = {
"schema_version": PROJECTION_SCHEMA_VERSION,
"contract": PROJECTION_RECEIPT_CONTRACT,
"kind": kind,
"package_id": package_id,
"plan_id": plan_id,
"renderer": renderer,
"artifacts": artifacts,
"diagnostics": diagnostics,
"timing": timing,
"peak_memory_bytes": peak_memory_bytes,
}
document = {**body, "receipt_id": projection_hash(body)}
return cls(validate_projection_receipt(document))
@classmethod
def from_dict(cls, document: dict[str, object]) -> ProjectionReceiptV1:
return cls(validate_projection_receipt(dict(document)))
@dataclass(frozen=True)
class ProjectionArtifact:
"""One renderer-produced artifact addressed by a logical identifier."""
artifact_id: str
media_type: str
content: bytes
def evidence(self) -> dict[str, object]:
return {
"artifact_id": self.artifact_id,
"media_type": self.media_type,
"sha256": hashlib.sha256(self.content).hexdigest(),
"bytes": len(self.content),
}
@dataclass(frozen=True)
class ProjectionRenderResult:
"""Artifact bytes plus the bounded renderer receipt that attests them."""
artifacts: tuple[ProjectionArtifact, ...]
receipt: ProjectionReceiptV1
def _validate_project_identity(value: object) -> None:
if not isinstance(value, dict):
raise DocForgeError("invalid_projection", "Projection project identity is invalid")
project = cast(dict[str, object], value)
if set(project) != {
"project_id",
"project_root_fingerprint",
"adapter",
"revision",
"source_hash",
}:
raise DocForgeError("invalid_projection", "Projection project identity is invalid")
if not all(
isinstance(project.get(key), str) and bool(project[key])
for key in ("project_id", "project_root_fingerprint", "adapter", "revision")
) or not _is_hash(project.get("source_hash")):
raise DocForgeError("invalid_projection", "Projection project identity is invalid")
def validate_manual_render_plan(document: dict[str, object]) -> dict[str, object]:
required = {
"schema_version",
"contract",
"plan_id",
"project",
"view",
"changeset_hash",
"pages",
"navigation",
"search_documents",
"diagnostics",
}
if set(document) != required:
raise DocForgeError("invalid_projection", "Manual render plan fields are invalid")
if (
document.get("schema_version") != PROJECTION_SCHEMA_VERSION
or document.get("contract") != MANUAL_RENDER_PLAN_CONTRACT
):
raise DocForgeError("invalid_projection", "Manual render plan version is unsupported")
_validate_project_identity(document.get("project"))
if not all(
isinstance(document.get(key), expected)
for key, expected in (
("view", dict),
("pages", list),
("navigation", list),
("search_documents", list),
("diagnostics", dict),
)
):
raise DocForgeError("invalid_projection", "Manual render plan structure is invalid")
changeset_hash = document.get("changeset_hash")
if changeset_hash is not None and not _is_hash(changeset_hash):
raise DocForgeError("invalid_projection", "Manual render plan changeset hash is invalid")
return _validated_identity(
document,
identity_field="plan_id",
maximum=MAX_PLAN_BYTES,
kind="Manual render plan",
)
def validate_graph_view_plan(document: dict[str, object]) -> dict[str, object]:
required = {
"schema_version",
"contract",
"plan_id",
"project",
"view",
"bounds",
"policy",
"graph",
"omissions",
"diagnostics",
}
if set(document) != required:
raise DocForgeError("invalid_projection", "Graph view plan fields are invalid")
if (
document.get("schema_version") != PROJECTION_SCHEMA_VERSION
or document.get("contract") != GRAPH_VIEW_PLAN_CONTRACT
):
raise DocForgeError("invalid_projection", "Graph view plan version is unsupported")
_validate_project_identity(document.get("project"))
if not all(
isinstance(document.get(key), expected)
for key, expected in (
("view", dict),
("bounds", dict),
("policy", dict),
("graph", dict),
("omissions", list),
("diagnostics", dict),
)
):
raise DocForgeError("invalid_projection", "Graph view plan structure is invalid")
return _validated_identity(
document,
identity_field="plan_id",
maximum=MAX_PLAN_BYTES,
kind="Graph view plan",
)
def validate_projection_package(document: dict[str, object]) -> dict[str, object]:
required = {
"schema_version",
"contract",
"package_id",
"kind",
"plan_id",
"plan",
"renderer",
"components",
"assets",
"output_policy",
}
if set(document) != required:
raise DocForgeError("invalid_projection", "Projection package fields are invalid")
kind = document.get("kind")
if (
document.get("schema_version") != PROJECTION_SCHEMA_VERSION
or document.get("contract") != PROJECTION_PACKAGE_CONTRACT
or kind not in {"manual", "graph"}
):
raise DocForgeError("invalid_projection", "Projection package version or kind is invalid")
plan = document.get("plan")
if not isinstance(plan, dict):
raise DocForgeError("invalid_projection", "Projection package plan is invalid")
validated_plan = (
validate_manual_render_plan(cast(dict[str, object], plan))
if kind == "manual"
else validate_graph_view_plan(cast(dict[str, object], plan))
)
if document.get("plan_id") != validated_plan.get("plan_id"):
raise DocForgeError("invalid_projection", "Projection package plan identity is invalid")
if not all(
isinstance(document.get(key), expected)
for key, expected in (
("renderer", dict),
("components", list),
("assets", list),
("output_policy", dict),
)
):
raise DocForgeError("invalid_projection", "Projection package structure is invalid")
if len(cast(list[object], document["assets"])) > MAX_PROJECTION_ARTIFACTS:
raise DocForgeError("projection_too_large", "Projection package has too many assets")
_reject_runtime_authority(document)
return _validated_identity(
document,
identity_field="package_id",
maximum=MAX_PACKAGE_BYTES,
kind="Projection package",
)
def validate_projection_receipt(document: dict[str, object]) -> dict[str, object]:
required = {
"schema_version",
"contract",
"receipt_id",
"kind",
"package_id",
"plan_id",
"renderer",
"artifacts",
"diagnostics",
"timing",
"peak_memory_bytes",
}
if set(document) != required:
raise DocForgeError("invalid_projection", "Projection receipt fields are invalid")
if (
document.get("schema_version") != PROJECTION_SCHEMA_VERSION
or document.get("contract") != PROJECTION_RECEIPT_CONTRACT
or document.get("kind") not in {"manual", "graph"}
or not _is_hash(document.get("package_id"))
or not _is_hash(document.get("plan_id"))
):
raise DocForgeError("invalid_projection", "Projection receipt identity is invalid")
artifacts_value = document.get("artifacts")
if not isinstance(artifacts_value, list):
raise DocForgeError("invalid_projection", "Projection receipt structure is invalid")
artifacts = cast(list[object], artifacts_value)
renderer = document.get("renderer")
diagnostics = document.get("diagnostics")
timing = document.get("timing")
if (
not isinstance(renderer, dict)
or not isinstance(diagnostics, dict)
or not isinstance(timing, dict)
):
raise DocForgeError("invalid_projection", "Projection receipt structure is invalid")
renderer_document = cast(dict[str, object], renderer)
diagnostics_document = cast(dict[str, object], diagnostics)
timing_document = cast(dict[str, object], timing)
warnings = diagnostics_document.get("warnings")
if (
len(artifacts) > MAX_PROJECTION_ARTIFACTS
or set(renderer_document) != {"renderer_id", "renderer_version"}
or not all(
isinstance(renderer_document.get(field), str) and renderer_document[field]
for field in ("renderer_id", "renderer_version")
)
or set(diagnostics_document) != {"warnings"}
or not isinstance(warnings, list)
or len(cast(list[object], warnings)) > 10_000
or not all(isinstance(item, str) for item in cast(list[object], warnings))
or set(timing_document) != {"elapsed_ns"}
or type(timing_document.get("elapsed_ns")) is not int
or cast(int, timing_document["elapsed_ns"]) < 0
):
raise DocForgeError("invalid_projection", "Projection receipt structure is invalid")
peak = document.get("peak_memory_bytes")
if peak is not None and (type(peak) is not int or peak < 0):
raise DocForgeError("invalid_projection", "Projection receipt memory value is invalid")
artifact_ids: set[str] = set()
for artifact in artifacts:
if not isinstance(artifact, dict):
raise DocForgeError("invalid_projection", "Projection receipt artifact is invalid")
item = cast(dict[str, object], artifact)
if (
set(item) != {"artifact_id", "media_type", "sha256", "bytes"}
or not isinstance(item.get("artifact_id"), str)
or not item["artifact_id"]
or "/" in cast(str, item["artifact_id"])
or not isinstance(item.get("media_type"), str)
or not item["media_type"]
or not _is_hash(item.get("sha256"))
or type(item.get("bytes")) is not int
or cast(int, item["bytes"]) < 0
):
raise DocForgeError("invalid_projection", "Projection receipt artifact is invalid")
artifact_id = cast(str, item["artifact_id"])
if artifact_id in artifact_ids:
raise DocForgeError("invalid_projection", "Projection receipt artifacts are duplicated")
artifact_ids.add(artifact_id)
return _validated_identity(
document,
identity_field="receipt_id",
maximum=MAX_RECEIPT_BYTES,
kind="Projection receipt",
)

489
src/docforge/projection_fragments.py
View file

@ -1,489 +0,0 @@
"""Bounded, path-free, disposable projection fragment caching."""
from __future__ import annotations
import base64
import hashlib
import json
import os
from dataclasses import dataclass
from pathlib import Path
from typing import Literal, cast
from ._fs_safety import (
atomic_replace_bytes_at,
open_confined_directory,
read_bounded_file_at,
require_bound_directory,
)
from .errors import DocForgeError
from .projection_contract import canonical_projection_bytes, projection_hash
FRAGMENT_SCHEMA_VERSION = 1
FRAGMENT_KEY_CONTRACT = "docforge.projection-fragment-key"
FRAGMENT_RECORD_CONTRACT = "docforge.projection-fragment-record"
FRAGMENT_CACHE_DIRECTORY = "projection-fragments-v1"
MAX_FRAGMENT_CONTENT_BYTES = 4_000_000
MAX_FRAGMENT_ID_CHARS = 256
MAX_FRAGMENT_CACHE_ENTRIES = 10_000
MAX_FRAGMENT_CACHE_BYTES = 64_000_000
_MAX_RECORD_OVERHEAD_BYTES = 8_192
ProjectionKind = Literal["manual", "graph"]
def _is_hash(value: object) -> bool:
return (
isinstance(value, str)
and len(value) == 64
and all(character in "0123456789abcdef" for character in value)
)
def _version_string(value: object, *, field: str) -> str:
if (
not isinstance(value, str)
or not value
or value != value.strip()
or len(value) > MAX_FRAGMENT_ID_CHARS
or any(ord(character) < 32 for character in value)
):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment identity is invalid",
field=field,
)
return value
def fragment_semantic_hash(value: object) -> str:
"""Hash one complete semantic input using the projection canonical JSON form."""
try:
return hashlib.sha256(canonical_projection_bytes(value)).hexdigest()
except (TypeError, ValueError) as error:
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment semantic input is not canonical JSON",
) from error
@dataclass(frozen=True)
class FragmentKey:
"""Versioned identity for one renderer component's complete semantics."""
projection_kind: ProjectionKind
renderer_id: str
renderer_version: str
component_version: str
semantic_input_hash: str
key_id: str
@classmethod
def create(
cls,
*,
projection_kind: ProjectionKind,
renderer_id: str,
renderer_version: str,
component_version: str,
semantic_input_hash: str,
) -> FragmentKey:
body = cls._body(
projection_kind=projection_kind,
renderer_id=renderer_id,
renderer_version=renderer_version,
component_version=component_version,
semantic_input_hash=semantic_input_hash,
)
return cls._from_validated({**body, "key_id": projection_hash(body)})
@classmethod
def from_dict(cls, value: object) -> FragmentKey:
if not isinstance(value, dict):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment key is invalid",
)
return cls._from_validated(dict(cast(dict[str, object], value)))
@staticmethod
def _body(
*,
projection_kind: object,
renderer_id: object,
renderer_version: object,
component_version: object,
semantic_input_hash: object,
) -> dict[str, object]:
if projection_kind not in {"manual", "graph"}:
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment kind is invalid",
)
if not _is_hash(semantic_input_hash):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment semantic input hash is invalid",
)
return {
"schema_version": FRAGMENT_SCHEMA_VERSION,
"contract": FRAGMENT_KEY_CONTRACT,
"projection_kind": projection_kind,
"renderer_id": _version_string(renderer_id, field="renderer_id"),
"renderer_version": _version_string(
renderer_version,
field="renderer_version",
),
"component_version": _version_string(
component_version,
field="component_version",
),
"semantic_input_hash": semantic_input_hash,
}
@classmethod
def _from_validated(cls, value: dict[str, object]) -> FragmentKey:
required = {
"schema_version",
"contract",
"projection_kind",
"renderer_id",
"renderer_version",
"component_version",
"semantic_input_hash",
"key_id",
}
if (
set(value) != required
or value.get("schema_version") != FRAGMENT_SCHEMA_VERSION
or value.get("contract") != FRAGMENT_KEY_CONTRACT
):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment key contract is incompatible",
)
body = cls._body(
projection_kind=value.get("projection_kind"),
renderer_id=value.get("renderer_id"),
renderer_version=value.get("renderer_version"),
component_version=value.get("component_version"),
semantic_input_hash=value.get("semantic_input_hash"),
)
key_id = value.get("key_id")
if not _is_hash(key_id) or key_id != projection_hash(body):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment key does not match its semantics",
)
return cls(
projection_kind=cast(ProjectionKind, body["projection_kind"]),
renderer_id=cast(str, body["renderer_id"]),
renderer_version=cast(str, body["renderer_version"]),
component_version=cast(str, body["component_version"]),
semantic_input_hash=cast(str, body["semantic_input_hash"]),
key_id=cast(str, key_id),
)
def as_dict(self) -> dict[str, object]:
return {
"schema_version": FRAGMENT_SCHEMA_VERSION,
"contract": FRAGMENT_KEY_CONTRACT,
"projection_kind": self.projection_kind,
"renderer_id": self.renderer_id,
"renderer_version": self.renderer_version,
"component_version": self.component_version,
"semantic_input_hash": self.semantic_input_hash,
"key_id": self.key_id,
}
@dataclass(frozen=True)
class FragmentRecord:
"""One path-free fragment payload with complete byte evidence."""
key: FragmentKey
content: bytes
byte_count: int
content_sha256: str
record_id: str
@classmethod
def create(cls, key: FragmentKey, content: bytes) -> FragmentRecord:
if len(content) > MAX_FRAGMENT_CONTENT_BYTES:
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment content is invalid or oversized",
maximum_bytes=MAX_FRAGMENT_CONTENT_BYTES,
)
body = cls._body(key, content)
return cls(
key=key,
content=content,
byte_count=len(content),
content_sha256=hashlib.sha256(content).hexdigest(),
record_id=projection_hash(body),
)
@classmethod
def from_dict(cls, value: object) -> FragmentRecord:
if not isinstance(value, dict):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment record is invalid",
)
document = dict(cast(dict[str, object], value))
required = {
"schema_version",
"contract",
"record_id",
"key",
"content_encoding",
"content",
"byte_count",
"content_sha256",
}
if (
set(document) != required
or document.get("schema_version") != FRAGMENT_SCHEMA_VERSION
or document.get("contract") != FRAGMENT_RECORD_CONTRACT
or document.get("content_encoding") != "base64"
):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment record contract is incompatible",
)
encoded = document.get("content")
if not isinstance(encoded, str):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment content encoding is invalid",
)
try:
content = base64.b64decode(encoded.encode("ascii"), validate=True)
except (UnicodeEncodeError, ValueError) as error:
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment content encoding is invalid",
) from error
if len(content) > MAX_FRAGMENT_CONTENT_BYTES:
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment content is oversized",
maximum_bytes=MAX_FRAGMENT_CONTENT_BYTES,
)
key = FragmentKey.from_dict(document.get("key"))
body = cls._body(key, content)
record_id = document.get("record_id")
if (
type(document.get("byte_count")) is not int
or document.get("byte_count") != len(content)
or document.get("content_sha256") != hashlib.sha256(content).hexdigest()
or not _is_hash(record_id)
or record_id != projection_hash(body)
):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment byte evidence is invalid",
)
return cls(
key=key,
content=content,
byte_count=len(content),
content_sha256=hashlib.sha256(content).hexdigest(),
record_id=cast(str, record_id),
)
@classmethod
def from_bytes(cls, raw: bytes) -> FragmentRecord:
try:
value: object = json.loads(raw)
except (UnicodeDecodeError, json.JSONDecodeError) as error:
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment record is not valid JSON",
) from error
record = cls.from_dict(value)
if record.to_bytes() != raw:
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment record is not canonically serialized",
)
return record
@staticmethod
def _body(key: FragmentKey, content: bytes) -> dict[str, object]:
return {
"schema_version": FRAGMENT_SCHEMA_VERSION,
"contract": FRAGMENT_RECORD_CONTRACT,
"key": key.as_dict(),
"content_encoding": "base64",
"content": base64.b64encode(content).decode("ascii"),
"byte_count": len(content),
"content_sha256": hashlib.sha256(content).hexdigest(),
}
def as_dict(self) -> dict[str, object]:
return {
**self._body(self.key, self.content),
"record_id": self.record_id,
}
def to_bytes(self) -> bytes:
return canonical_projection_bytes(self.as_dict())
class ProjectionFragmentCache:
"""Confined best-effort storage for immutable projection fragments."""
def __init__(
self,
project_root: Path,
cache_root: Path,
*,
maximum_content_bytes: int = MAX_FRAGMENT_CONTENT_BYTES,
) -> None:
if (
type(maximum_content_bytes) is not int
or not 1 <= maximum_content_bytes <= MAX_FRAGMENT_CONTENT_BYTES
):
raise DocForgeError(
"invalid_projection_fragment",
"Projection fragment cache byte limit is invalid",
maximum_bytes=MAX_FRAGMENT_CONTENT_BYTES,
)
self.project_root = project_root
self.cache_root = cache_root
self.fragment_root = cache_root / FRAGMENT_CACHE_DIRECTORY
self.maximum_content_bytes = maximum_content_bytes
@property
def maximum_record_bytes(self) -> int:
encoded = ((self.maximum_content_bytes + 2) // 3) * 4
return encoded + _MAX_RECORD_OVERHEAD_BYTES
def get(self, key: FragmentKey) -> FragmentRecord | None:
"""Return one exact compatible fragment, treating every cache defect as a miss."""
descriptor: int | None = None
try:
descriptor = self._open(create=False)
raw = read_bounded_file_at(
descriptor,
f"{key.key_id}.json",
self.maximum_record_bytes,
)
if raw is None:
return None
record = FragmentRecord.from_bytes(raw)
if record.key != key or record.byte_count > self.maximum_content_bytes:
return None
require_bound_directory(self.fragment_root, descriptor)
return record
except (DocForgeError, OSError):
return None
finally:
if descriptor is not None:
os.close(descriptor)
def put(self, key: FragmentKey, content: bytes) -> FragmentRecord | None:
"""Durably publish one fragment, returning ``None`` on disposable cache failure."""
if len(content) > self.maximum_content_bytes:
return None
try:
record = FragmentRecord.create(key, content)
except DocForgeError:
return None
raw = record.to_bytes()
if len(raw) > self.maximum_record_bytes:
return None
descriptor: int | None = None
try:
descriptor = self._open(create=True)
name = f"{key.key_id}.json"
try:
existing = read_bounded_file_at(
descriptor,
name,
self.maximum_record_bytes,
)
except DocForgeError as error:
if error.code == "path_escape":
return None
existing = None
if existing == raw:
require_bound_directory(self.fragment_root, descriptor)
return record
atomic_replace_bytes_at(
self.fragment_root,
descriptor,
name,
raw,
verify=lambda: require_bound_directory(self.fragment_root, descriptor),
)
return record
except (DocForgeError, OSError):
return None
finally:
if descriptor is not None:
os.close(descriptor)
def prune(self, keep: tuple[FragmentKey, ...]) -> bool:
"""Remove every stale entry and prove the retained inventory is bounded."""
keep_ids = {key.key_id for key in keep}
if len(keep_ids) > MAX_FRAGMENT_CACHE_ENTRIES:
return False
descriptor: int | None = None
try:
descriptor = self._open(create=False)
retained_entries = 0
retained_bytes = 0
with os.scandir(descriptor) as entries:
for entry in entries:
name = entry.name
retained = (
len(name) == 69
and name.endswith(".json")
and name[:-5] in keep_ids
and all(character in "0123456789abcdef" for character in name[:-5])
)
if retained:
identity = entry.stat(follow_symlinks=False)
if not entry.is_file(follow_symlinks=False):
return False
retained_entries += 1
retained_bytes += identity.st_size
continue
try:
os.unlink(name, dir_fd=descriptor)
except OSError:
return False
if (
retained_entries > MAX_FRAGMENT_CACHE_ENTRIES
or retained_bytes > MAX_FRAGMENT_CACHE_BYTES
):
return False
require_bound_directory(self.fragment_root, descriptor)
os.fsync(descriptor)
return True
except (DocForgeError, OSError):
return False
finally:
if descriptor is not None:
os.close(descriptor)
def _open(self, *, create: bool) -> int:
if self.cache_root == self.project_root or not self.cache_root.is_relative_to(
self.project_root
):
raise DocForgeError(
"path_escape",
"Projection fragment cache is not confined to a derived project root",
)
return open_confined_directory(
self.project_root,
self.fragment_root,
create=create,
)

234
src/docforge/projection_policy.py
View file

@ -1,234 +0,0 @@
"""Independent version-2 policy for manual, portable graph, and live projections."""
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from typing import Literal, cast
from .errors import DocForgeError
ManualProjectionMode = Literal["auto", "explicit", "disabled"]
PortableGraphProjectionMode = Literal["explicit", "disabled"]
LiveViewerProjectionMode = Literal["on-demand", "disabled"]
MANUAL_PROJECTION_MODES: tuple[ManualProjectionMode, ...] = (
"auto",
"explicit",
"disabled",
)
PORTABLE_GRAPH_PROJECTION_MODES: tuple[PortableGraphProjectionMode, ...] = (
"explicit",
"disabled",
)
LIVE_VIEWER_PROJECTION_MODES: tuple[LiveViewerProjectionMode, ...] = (
"on-demand",
"disabled",
)
def _invalid_mode(field: str, value: object, allowed: tuple[str, ...]) -> DocForgeError:
return DocForgeError(
"invalid_projection_policy",
"Projection policy mode is unsupported",
projection=field,
mode=value,
allowed=list(allowed),
)
def _select_mode(
value: object | None,
*,
field: str,
default: str,
allowed: tuple[str, ...],
) -> str:
selected: object = default if value is None else value
if not isinstance(selected, str) or selected not in allowed:
raise _invalid_mode(field, selected, allowed)
return selected
def _require_boolean(field: str, value: object) -> bool:
if type(value) is not bool:
raise DocForgeError(
"invalid_projection_policy",
"Projection policy availability must be Boolean",
field=field,
)
return value
def _unavailable(field: str, mode: str, required: str) -> DocForgeError:
return DocForgeError(
"projection_policy_unavailable",
"Projection policy mode is unavailable",
projection=field,
mode=mode,
required=required,
)
def validate_manual_projection_mode(value: object) -> ManualProjectionMode:
"""Validate one direct manual-service policy selection."""
return cast(
ManualProjectionMode,
_select_mode(
value,
field="manual",
default="explicit",
allowed=MANUAL_PROJECTION_MODES,
),
)
def validate_portable_graph_projection_mode(
value: object,
) -> PortableGraphProjectionMode:
"""Validate one direct portable-graph service policy selection."""
return cast(
PortableGraphProjectionMode,
_select_mode(
value,
field="portable_graph",
default="explicit",
allowed=PORTABLE_GRAPH_PROJECTION_MODES,
),
)
def validate_live_viewer_projection_mode(value: object) -> LiveViewerProjectionMode:
"""Validate one direct live-viewer service policy selection."""
return cast(
LiveViewerProjectionMode,
_select_mode(
value,
field="live_viewer",
default="on-demand",
allowed=LIVE_VIEWER_PROJECTION_MODES,
),
)
@dataclass(frozen=True)
class ProjectionPolicyV2:
"""One immutable policy for three independent projection consumers."""
manual: ManualProjectionMode
portable_graph: PortableGraphProjectionMode
live_viewer: LiveViewerProjectionMode
def __post_init__(self) -> None:
if self.manual not in MANUAL_PROJECTION_MODES:
raise _invalid_mode("manual", self.manual, MANUAL_PROJECTION_MODES)
if self.portable_graph not in PORTABLE_GRAPH_PROJECTION_MODES:
raise _invalid_mode(
"portable_graph",
self.portable_graph,
PORTABLE_GRAPH_PROJECTION_MODES,
)
if self.live_viewer not in LIVE_VIEWER_PROJECTION_MODES:
raise _invalid_mode(
"live_viewer",
self.live_viewer,
LIVE_VIEWER_PROJECTION_MODES,
)
def as_dict(self) -> dict[str, object]:
return {
"schema_version": 2,
"manual": self.manual,
"portable_graph": self.portable_graph,
"live_viewer": self.live_viewer,
}
@property
def policy_hash(self) -> str:
raw = json.dumps(
self.as_dict(),
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
return hashlib.sha256(raw).hexdigest()
def compose_projection_policy(
*,
manual: str | None = None,
portable_graph: str | None = None,
live_viewer: str | None = None,
manual_configured: bool,
portable_graph_configured: bool,
application_enabled: bool,
live_viewer_available: bool = True,
) -> ProjectionPolicyV2:
"""Compose compatible defaults with explicit availability-checked selections."""
manual_available = _require_boolean("manual_configured", manual_configured)
graph_available = _require_boolean(
"portable_graph_configured",
portable_graph_configured,
)
application_available = _require_boolean("application_enabled", application_enabled)
viewer_available = _require_boolean("live_viewer_available", live_viewer_available)
default_manual = (
"auto"
if manual_available and application_available
else ("explicit" if manual_available else "disabled")
)
default_graph = "explicit" if graph_available else "disabled"
default_viewer = "on-demand" if viewer_available else "disabled"
manual_mode = cast(
ManualProjectionMode,
_select_mode(
manual,
field="manual",
default=default_manual,
allowed=MANUAL_PROJECTION_MODES,
),
)
graph_mode = cast(
PortableGraphProjectionMode,
_select_mode(
portable_graph,
field="portable_graph",
default=default_graph,
allowed=PORTABLE_GRAPH_PROJECTION_MODES,
),
)
viewer_mode = cast(
LiveViewerProjectionMode,
_select_mode(
live_viewer,
field="live_viewer",
default=default_viewer,
allowed=LIVE_VIEWER_PROJECTION_MODES,
),
)
if manual_mode != "disabled" and not manual_available:
raise _unavailable("manual", manual_mode, "manual_render_config")
if manual_mode == "auto" and not application_available:
raise _unavailable("manual", manual_mode, "canonical_application")
if graph_mode == "explicit" and not graph_available:
raise _unavailable(
"portable_graph",
graph_mode,
"portable_graph_render_config",
)
if viewer_mode == "on-demand" and not viewer_available:
raise _unavailable("live_viewer", viewer_mode, "live_viewer_runtime")
return ProjectionPolicyV2(
manual=manual_mode,
portable_graph=graph_mode,
live_viewer=viewer_mode,
)

436
src/docforge/projection_worker.py
View file

@ -1,436 +0,0 @@
"""One-shot detached execution for the fixed built-in projection renderers."""
from __future__ import annotations
import base64
import binascii
import json
import os
import resource
import subprocess
import sys
import tempfile
from importlib.metadata import version
from typing import cast
from .errors import DocForgeError
from .projection_contract import (
MAX_PACKAGE_BYTES,
MAX_PROJECTION_ARTIFACTS,
MAX_RECEIPT_BYTES,
ProjectionArtifact,
ProjectionPackageV1,
ProjectionReceiptV1,
ProjectionRenderResult,
canonical_projection_bytes,
)
WORKER_PROTOCOL_VERSION = 1
MAX_WORKER_ARTIFACT_BYTES = 20_000_000
MAX_WORKER_REQUEST_BYTES = MAX_PACKAGE_BYTES + 1
MAX_WORKER_RESPONSE_BYTES = 4 * ((MAX_WORKER_ARTIFACT_BYTES + 2) // 3) + MAX_RECEIPT_BYTES + 256_000
WORKER_TIMEOUT_SECONDS = 30
_GENERIC_HTML_RENDERER_ID = "generic_html"
_PORTABLE_GRAPH_RENDERER_ID = "portable_graph_html"
_PORTABLE_GRAPH_RENDERER_VERSION = "1"
def _generic_html_renderer_version() -> str:
return f"1+markdown-it-py-{version('markdown-it-py')}"
def _worker_failure(message: str, **details: object) -> DocForgeError:
return DocForgeError("projection_worker_failure", message, **details)
def _renderer_identity(package: ProjectionPackageV1) -> dict[str, object]:
renderer_value = package.document.get("renderer")
if not isinstance(renderer_value, dict):
raise DocForgeError("unsupported_renderer", "Projection renderer identity is invalid")
renderer = cast(dict[str, object], renderer_value)
if set(renderer) != {"renderer_id", "renderer_version"}:
raise DocForgeError("unsupported_renderer", "Projection renderer identity is invalid")
renderer_id = renderer.get("renderer_id")
renderer_version = renderer.get("renderer_version")
if not isinstance(renderer_id, str) or not isinstance(renderer_version, str):
raise DocForgeError("unsupported_renderer", "Projection renderer identity is invalid")
supported = (
package.kind == "manual"
and renderer_id == _GENERIC_HTML_RENDERER_ID
and renderer_version == _generic_html_renderer_version()
) or (
package.kind == "graph"
and renderer_id == _PORTABLE_GRAPH_RENDERER_ID
and renderer_version == _PORTABLE_GRAPH_RENDERER_VERSION
)
if not supported:
raise DocForgeError(
"unsupported_renderer",
"Projection worker supports only the fixed built-in renderer versions",
)
return dict(renderer)
def _output_policy(package: ProjectionPackageV1) -> tuple[tuple[str, ...], int]:
policy_value = package.document.get("output_policy")
if not isinstance(policy_value, dict):
raise DocForgeError("invalid_projection", "Projection output policy is invalid")
policy = cast(dict[str, object], policy_value)
if set(policy) != {"artifact_ids", "max_total_bytes"}:
raise DocForgeError("invalid_projection", "Projection output policy is invalid")
artifact_ids_value = policy.get("artifact_ids")
maximum = policy.get("max_total_bytes")
if not isinstance(artifact_ids_value, list):
raise DocForgeError("invalid_projection", "Projection artifact inventory is invalid")
artifact_ids_objects = cast(list[object], artifact_ids_value)
if (
not artifact_ids_objects
or len(artifact_ids_objects) > MAX_PROJECTION_ARTIFACTS
or not all(
isinstance(artifact_id, str)
and bool(artifact_id)
and "/" not in artifact_id
and artifact_id not in {".", ".."}
for artifact_id in artifact_ids_objects
)
):
raise DocForgeError("invalid_projection", "Projection artifact inventory is invalid")
artifact_ids = tuple(cast(list[str], artifact_ids_objects))
if len(set(artifact_ids)) != len(artifact_ids):
raise DocForgeError("invalid_projection", "Projection artifact inventory is duplicated")
if type(maximum) is not int or maximum < 1:
raise DocForgeError(
"invalid_projection",
"Projection output byte allowance is invalid",
)
return artifact_ids, maximum
def _validated_package(package: object) -> ProjectionPackageV1:
if not isinstance(package, ProjectionPackageV1):
raise TypeError("Projection worker requires ProjectionPackageV1")
validated = ProjectionPackageV1.from_dict(package.as_dict())
_renderer_identity(validated)
_output_policy(validated)
return validated
def _validate_result(
package: ProjectionPackageV1,
result: ProjectionRenderResult,
*,
require_peak_memory: bool,
) -> ProjectionRenderResult:
artifact_ids, maximum = _output_policy(package)
if (
len(result.artifacts) != len(artifact_ids)
or tuple(artifact.artifact_id for artifact in result.artifacts) != artifact_ids
):
raise _worker_failure("Projection worker returned an invalid artifact inventory")
total_bytes = 0
evidence: list[dict[str, object]] = []
for artifact in result.artifacts:
if not artifact.media_type or type(artifact.content) is not bytes:
raise _worker_failure("Projection worker returned an invalid artifact")
total_bytes += len(artifact.content)
if total_bytes > maximum or total_bytes > MAX_WORKER_ARTIFACT_BYTES:
raise _worker_failure("Projection worker artifact transfer exceeded its fixed boundary")
evidence.append(artifact.evidence())
try:
receipt = ProjectionReceiptV1.from_dict(result.receipt.as_dict())
except DocForgeError as error:
raise _worker_failure("Projection worker receipt is invalid") from error
receipt_document = receipt.document
renderer = _renderer_identity(package)
timing_value = receipt_document.get("timing")
if not isinstance(timing_value, dict):
raise _worker_failure("Projection worker receipt timing is invalid")
timing = cast(dict[str, object], timing_value)
elapsed = timing.get("elapsed_ns")
peak_memory = receipt_document.get("peak_memory_bytes")
if (
receipt_document.get("kind") != package.kind
or receipt_document.get("package_id") != package.package_id
or receipt_document.get("plan_id") != package.document.get("plan_id")
or receipt_document.get("renderer") != renderer
or receipt_document.get("artifacts") != evidence
or type(elapsed) is not int
or elapsed < 0
or (require_peak_memory and (type(peak_memory) is not int or peak_memory <= 0))
):
raise _worker_failure("Projection worker receipt does not attest the requested package")
return ProjectionRenderResult(tuple(result.artifacts), receipt)
def _decode_canonical_line(raw: bytes, *, maximum: int, label: str) -> dict[str, object]:
if type(raw) is not bytes or len(raw) > maximum:
raise _worker_failure(f"{label} exceeded its fixed boundary", maximum_bytes=maximum)
if not raw or not raw.endswith(b"\n") or raw.count(b"\n") != 1:
raise _worker_failure(f"{label} framing is invalid")
payload = raw[:-1]
try:
value: object = json.loads(payload)
except (UnicodeDecodeError, json.JSONDecodeError) as error:
raise _worker_failure(f"{label} is not valid JSON") from error
if not isinstance(value, dict):
raise _worker_failure(f"{label} must be one JSON object")
document = cast(dict[str, object], value)
if canonical_projection_bytes(document) != payload:
raise _worker_failure(f"{label} is not canonical JSON")
return document
def _encode_request(package: ProjectionPackageV1) -> bytes:
encoded = canonical_projection_bytes(package.as_dict()) + b"\n"
if len(encoded) > MAX_WORKER_REQUEST_BYTES:
raise DocForgeError(
"projection_too_large",
"Projection worker request exceeds its fixed boundary",
maximum_bytes=MAX_WORKER_REQUEST_BYTES,
)
return encoded
def _invoke_worker(request: bytes) -> subprocess.CompletedProcess[bytes]:
environment = {key: os.environ[key] for key in ("SYSTEMROOT", "WINDIR") if key in os.environ}
environment.update(
{
"PYTHONIOENCODING": "utf-8",
"PYTHONUTF8": "1",
}
)
command = [sys.executable, "-I", "-m", "docforge._projection_worker_main"]
with tempfile.TemporaryFile() as output:
completed = subprocess.run(
command,
input=request,
stdout=output,
stderr=subprocess.DEVNULL,
check=False,
timeout=WORKER_TIMEOUT_SECONDS,
shell=False,
cwd=sys.prefix,
env=environment,
)
output.seek(0)
stdout = output.read(MAX_WORKER_RESPONSE_BYTES + 1)
return subprocess.CompletedProcess(
command,
completed.returncode,
stdout=stdout,
)
def _decode_response(package: ProjectionPackageV1, raw: bytes) -> ProjectionRenderResult:
document = _decode_canonical_line(
raw,
maximum=MAX_WORKER_RESPONSE_BYTES,
label="Projection worker response",
)
if (
set(document) != {"schema_version", "artifacts", "receipt"}
or document.get("schema_version") != WORKER_PROTOCOL_VERSION
):
raise _worker_failure("Projection worker response contract is invalid")
artifact_values = document.get("artifacts")
receipt_value = document.get("receipt")
if not isinstance(artifact_values, list) or not isinstance(receipt_value, dict):
raise _worker_failure("Projection worker response structure is invalid")
artifacts: list[ProjectionArtifact] = []
total_bytes = 0
for value in cast(list[object], artifact_values):
if not isinstance(value, dict):
raise _worker_failure("Projection worker artifact envelope is invalid")
artifact = cast(dict[str, object], value)
if set(artifact) != {"artifact_id", "media_type", "content_base64"}:
raise _worker_failure("Projection worker artifact envelope is invalid")
artifact_id = artifact.get("artifact_id")
media_type = artifact.get("media_type")
encoded = artifact.get("content_base64")
if (
not isinstance(artifact_id, str)
or not isinstance(media_type, str)
or not isinstance(encoded, str)
):
raise _worker_failure("Projection worker artifact envelope is invalid")
try:
content = base64.b64decode(encoded.encode("ascii"), validate=True)
except (UnicodeEncodeError, binascii.Error, ValueError) as error:
raise _worker_failure("Projection worker artifact encoding is invalid") from error
total_bytes += len(content)
if total_bytes > MAX_WORKER_ARTIFACT_BYTES:
raise _worker_failure("Projection worker artifact transfer exceeded its fixed boundary")
artifacts.append(ProjectionArtifact(artifact_id, media_type, content))
try:
receipt = ProjectionReceiptV1.from_dict(cast(dict[str, object], receipt_value))
except DocForgeError as error:
raise _worker_failure("Projection worker receipt is invalid") from error
return _validate_result(
package,
ProjectionRenderResult(tuple(artifacts), receipt),
require_peak_memory=True,
)
def render_projection_in_worker(package: ProjectionPackageV1) -> ProjectionRenderResult:
"""Render one validated path-free package in a fixed one-shot child process."""
validated = _validated_package(package)
request = _encode_request(validated)
try:
completed = _invoke_worker(request)
except subprocess.TimeoutExpired as error:
raise DocForgeError(
"projection_worker_timeout",
"Detached projection worker exceeded its fixed timeout",
timeout_seconds=WORKER_TIMEOUT_SECONDS,
) from error
except OSError as error:
raise _worker_failure("Detached projection worker could not be launched") from error
if completed.returncode != 0:
if completed.returncode == 3 and completed.stdout:
try:
failure = _decode_canonical_line(
completed.stdout,
maximum=MAX_RECEIPT_BYTES,
label="Projection worker error response",
)
error = failure.get("error")
error_document = cast(dict[str, object], error) if isinstance(error, dict) else None
if (
set(failure) == {"schema_version", "error"}
and failure.get("schema_version") == WORKER_PROTOCOL_VERSION
and error_document is not None
and set(error_document) == {"code", "message", "details"}
and isinstance(error_document.get("code"), str)
and bool(error_document["code"])
and isinstance(error_document.get("message"), str)
and bool(error_document["message"])
and isinstance(error_document.get("details"), dict)
):
raise DocForgeError(
cast(str, error_document["code"]),
cast(str, error_document["message"]),
**cast(dict[str, object], error_document["details"]),
)
except DocForgeError as error:
if error.code != "projection_worker_failure":
raise
if completed.returncode < 0:
raise _worker_failure(
"Detached projection worker terminated by signal",
signal=-completed.returncode,
)
raise _worker_failure(
"Detached projection worker exited unsuccessfully",
exit_code=completed.returncode,
)
if type(completed.stdout) is not bytes:
raise _worker_failure("Detached projection worker returned invalid output")
return _decode_response(validated, completed.stdout)
def _render_package(package: ProjectionPackageV1) -> ProjectionRenderResult:
renderer = _renderer_identity(package)
if renderer["renderer_id"] == _GENERIC_HTML_RENDERER_ID:
from docforge_renderers.manual import ManualHtmlRenderer
result = ManualHtmlRenderer(cast(str, renderer["renderer_version"])).render(package)
else:
from docforge_renderers.graph import PortableGraphHtmlRenderer
result = PortableGraphHtmlRenderer().render(package)
return _validate_result(package, result, require_peak_memory=False)
def _peak_memory_bytes() -> int:
peak = int(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
return max(1, peak if sys.platform == "darwin" else peak * 1024)
def _child_response(package: ProjectionPackageV1) -> bytes:
result = _render_package(package)
original = result.receipt.document
receipt = ProjectionReceiptV1.create(
kind=package.kind,
package_id=package.package_id,
plan_id=cast(str, package.document["plan_id"]),
renderer=cast(dict[str, object], original["renderer"]),
artifacts=[artifact.evidence() for artifact in result.artifacts],
diagnostics=cast(dict[str, object], original["diagnostics"]),
timing=cast(dict[str, object], original["timing"]),
peak_memory_bytes=_peak_memory_bytes(),
)
validated = _validate_result(
package,
ProjectionRenderResult(result.artifacts, receipt),
require_peak_memory=True,
)
document: dict[str, object] = {
"schema_version": WORKER_PROTOCOL_VERSION,
"artifacts": [
{
"artifact_id": artifact.artifact_id,
"media_type": artifact.media_type,
"content_base64": base64.b64encode(artifact.content).decode("ascii"),
}
for artifact in validated.artifacts
],
"receipt": validated.receipt.as_dict(),
}
encoded = canonical_projection_bytes(document) + b"\n"
if len(encoded) > MAX_WORKER_RESPONSE_BYTES:
raise _worker_failure(
"Projection worker response exceeded its fixed boundary",
maximum_bytes=MAX_WORKER_RESPONSE_BYTES,
)
return encoded
def _read_child_request() -> ProjectionPackageV1:
raw = sys.stdin.buffer.read(MAX_WORKER_REQUEST_BYTES + 1)
document = _decode_canonical_line(
raw,
maximum=MAX_WORKER_REQUEST_BYTES,
label="Projection worker request",
)
return _validated_package(ProjectionPackageV1.from_dict(document))
def main(argv: list[str] | None = None) -> int:
"""Run the closed one-request child protocol."""
arguments = sys.argv[1:] if argv is None else argv
if arguments:
return 2
try:
package = _read_child_request()
except Exception:
return 2
try:
response = _child_response(package)
sys.stdout.buffer.write(response)
sys.stdout.buffer.flush()
except DocForgeError as error:
response = (
canonical_projection_bytes(
{
"schema_version": WORKER_PROTOCOL_VERSION,
"error": error.as_dict(),
}
)
+ b"\n"
)
if len(response) <= MAX_RECEIPT_BYTES:
sys.stdout.buffer.write(response)
sys.stdout.buffer.flush()
return 3
except Exception:
return 2
return 0
if __name__ == "__main__":
raise SystemExit(main())

1
src/docforge/py.typed
View file

@ -1 +0,0 @@
# PEP 561 marker for the typed DocForge public package.

234
src/docforge/render_contract.py
View file

@ -1,30 +1,31 @@
"""Compatibility shim over the versioned manual projection boundary."""
"""Deterministic built-in renderer contract and safe template primitives."""
from __future__ import annotations
import hashlib
import html
import json
import re
from dataclasses import dataclass
from importlib.metadata import version
from pathlib import Path
from typing import Protocol, cast
from typing import Protocol
from markdown_it import MarkdownIt
from .errors import DocForgeError
from .manual_projection import build_manual_projection_package, build_manual_render_plan
from .models import ProjectSnapshot, RenderView
from .projection_contract import (
ManualRenderPlanV1,
ProjectionPackageV1,
ProjectionRenderResult,
from .models import Edge, Node, ProjectSnapshot, RenderView
_TEMPLATE_TOKEN = re.compile(r"{{\s*([a-z_][a-z0-9_]*)\s*}}")
_ALLOWED_TOKENS = frozenset(
{
"docforge_content",
"docforge_project_id",
"docforge_render_identity",
"docforge_title",
"docforge_view_id",
}
)
from .projection_fragments import (
FragmentKey,
FragmentRecord,
ProjectionFragmentCache,
fragment_semantic_hash,
)
from .projection_worker import render_projection_in_worker
@dataclass(frozen=True)
@ -35,7 +36,6 @@ class PreparedRender:
renderer: str
renderer_version: str
template_hash: str
projection_receipt: dict[str, object] | None = None
class Renderer(Protocol):
@ -55,18 +55,16 @@ class Renderer(Protocol):
class GenericHtmlRenderer:
"""Preserve the public v1 renderer API over the plan-only manual renderer."""
"""Render validated nodes through escaped CommonMark and a strict token template."""
renderer_id = "generic_html"
contract_version = "1"
page_component_version = "manual.page@1"
def __init__(self, *, incremental: bool = True) -> None:
def __init__(self) -> None:
self.markdown = MarkdownIt("commonmark", {"html": False, "typographer": False})
self.renderer_version = (
f"{self.contract_version}+markdown-it-py-{version('markdown-it-py')}"
)
self.incremental = incremental
def prepare(
self,
@ -76,6 +74,22 @@ class GenericHtmlRenderer:
*,
changeset_hash: str | None,
) -> PreparedRender:
try:
template = template_bytes.decode("utf-8")
except UnicodeDecodeError as error:
raise DocForgeError("invalid_template", "Render template is not valid UTF-8") from error
tokens = _TEMPLATE_TOKEN.findall(template)
unknown = sorted(set(tokens) - _ALLOWED_TOKENS)
remainder = _TEMPLATE_TOKEN.sub("", template)
if unknown or "{{" in remainder or "}}" in remainder:
raise DocForgeError(
"invalid_template", "Render template contains unsupported tokens", tokens=unknown
)
if tokens.count("docforge_content") != 1:
raise DocForgeError(
"invalid_template", "Render template must contain docforge_content exactly once"
)
selected = tuple(
node for node in snapshot.nodes if not view.families or node.family in view.families
)
@ -114,31 +128,16 @@ class GenericHtmlRenderer:
render_identity = hashlib.sha256(
json.dumps(identity_payload, sort_keys=True, separators=(",", ":")).encode("utf-8")
).hexdigest()
plan = build_manual_render_plan(snapshot, view, changeset_hash=changeset_hash)
full_package = build_manual_projection_package(
plan,
template_bytes,
renderer_id=self.renderer_id,
renderer_version=self.renderer_version,
max_output_bytes=snapshot.descriptor.limits.max_render_bytes,
render_identity=render_identity,
)
if self.incremental:
result = self._incremental_result(
snapshot,
plan,
template_bytes,
full_package=full_package,
render_identity=render_identity,
)
else:
result = render_projection_in_worker(full_package)
if len(result.artifacts) != 1:
raise DocForgeError(
"invalid_projection",
"Manual renderer returned an unsupported artifact set",
)
output = result.artifacts[0].content
content = self._content(selected, selected_edges)
replacements = {
"docforge_content": content,
"docforge_project_id": html.escape(snapshot.descriptor.project_id, quote=True),
"docforge_render_identity": render_identity,
"docforge_title": html.escape(view.title, quote=True),
"docforge_view_id": html.escape(view.view_id, quote=True),
}
rendered = _TEMPLATE_TOKEN.sub(lambda match: replacements[match.group(1)], template)
output = rendered.rstrip().encode("utf-8") + b"\n"
return PreparedRender(
render_identity=render_identity,
output_hash=hashlib.sha256(output).hexdigest(),
@ -146,114 +145,45 @@ class GenericHtmlRenderer:
renderer=self.renderer_id,
renderer_version=self.renderer_version,
template_hash=template_hash,
projection_receipt=result.receipt.as_dict(),
)
def _incremental_result(
self,
snapshot: ProjectSnapshot,
plan: ManualRenderPlanV1,
template_bytes: bytes,
*,
full_package: ProjectionPackageV1,
render_identity: str,
) -> ProjectionRenderResult:
cache = ProjectionFragmentCache(
snapshot.descriptor.root,
snapshot.descriptor.cache_root,
)
pages = cast(list[object], plan.document["pages"])
records: list[FragmentRecord | None] = []
missing: list[tuple[int, FragmentKey]] = []
for value in pages:
page = cast(dict[str, object], value)
key = FragmentKey.create(
projection_kind="manual",
renderer_id=self.renderer_id,
renderer_version=self.renderer_version,
component_version=self.page_component_version,
semantic_input_hash=fragment_semantic_hash(page),
def _content(self, nodes: tuple[Node, ...], edges: tuple[Edge, ...]) -> str:
navigation = ['<nav aria-label="Documentation"><ul>']
for node in nodes:
navigation.append(
f'<li><a href="#node-{html.escape(node.node_id, quote=True)}">'
f"{html.escape(node.title)}</a></li>"
)
record = cache.get(key)
if record is None:
missing.append((len(records), key))
records.append(record)
if not records:
return render_projection_in_worker(full_package)
full_result: ProjectionRenderResult | None = None
new_records: list[FragmentRecord] = []
if missing:
full_result = render_projection_in_worker(full_package)
fragments = self._extract_page_fragments(
pages,
full_result.artifacts[0].content,
navigation.append("</ul></nav>")
sections = [*navigation]
edge_map: dict[str, list[Edge]] = {}
for edge in edges:
edge_map.setdefault(edge.source_id, []).append(edge)
for node in nodes:
sections.extend(
[
f'<section id="node-{html.escape(node.node_id, quote=True)}">',
f"<h2>{html.escape(node.title)}</h2>",
'<dl class="docforge-node-meta">',
f"<dt>ID</dt><dd>{html.escape(node.node_id)}</dd>",
f"<dt>Family</dt><dd>{html.escape(node.family)}</dd>",
f"<dt>Status</dt><dd>{html.escape(node.status)}</dd>",
f"<dt>Authority</dt><dd>{html.escape(node.authority)}</dd>",
"</dl>",
f'<p class="docforge-summary">{html.escape(node.summary)}</p>',
self.markdown.render(node.content).rstrip(),
]
)
try:
for position, key in missing:
record = FragmentRecord.create(key, fragments[position])
records[position] = record
new_records.append(record)
except DocForgeError:
return full_result
complete_records = [record for record in records if record is not None]
if len(complete_records) != len(records):
return full_result or render_projection_in_worker(full_package)
try:
incremental_package = build_manual_projection_package(
plan,
template_bytes,
fragment_records=[record.as_dict() for record in complete_records],
renderer_id=self.renderer_id,
renderer_version=self.renderer_version,
max_output_bytes=snapshot.descriptor.limits.max_render_bytes,
render_identity=render_identity,
)
incremental_result = render_projection_in_worker(incremental_package)
except DocForgeError:
cache.prune(())
return full_result or render_projection_in_worker(full_package)
if full_result is None:
cache.prune(tuple(record.key for record in complete_records))
return incremental_result
if tuple(
(artifact.artifact_id, artifact.media_type, artifact.content)
for artifact in incremental_result.artifacts
) != tuple(
(artifact.artifact_id, artifact.media_type, artifact.content)
for artifact in full_result.artifacts
):
cache.prune(())
return full_result
for record in new_records:
cache.put(record.key, record.content)
cache.prune(tuple(record.key for record in complete_records))
return incremental_result
@staticmethod
def _extract_page_fragments(
pages: list[object],
output: bytes,
) -> list[bytes]:
"""Extract exact deterministic page sections from one trusted full artifact."""
fragments: list[bytes] = []
cursor = 0
closing = b"</section>"
for value in pages:
page = cast(dict[str, object], value)
node_id = cast(str, page["node_id"])
marker = f'<section id="node-{html.escape(node_id, quote=True)}">'.encode()
start = output.find(marker, cursor)
end = output.find(closing, start + len(marker)) if start >= 0 else -1
if start < 0 or end < 0:
raise DocForgeError(
"invalid_projection",
"Full manual artifact does not contain its planned page fragments",
)
end += len(closing)
fragments.append(output[start:end])
cursor = end
return fragments
relationships = edge_map.get(node.node_id, [])
if relationships:
sections.append('<ul class="docforge-relationships">')
for edge in relationships:
sections.append(
f"<li>{html.escape(edge.relation)}: {html.escape(edge.target_id)}</li>"
)
sections.append("</ul>")
sections.append("</section>")
return "\n".join(sections)
_RENDERERS: dict[str, type[GenericHtmlRenderer]] = {
@ -261,13 +191,13 @@ _RENDERERS: dict[str, type[GenericHtmlRenderer]] = {
}
def renderer_for(view: RenderView, *, incremental: bool = True) -> Renderer:
def renderer_for(view: RenderView) -> Renderer:
factory = _RENDERERS.get(view.renderer)
if factory is None:
raise DocForgeError(
"unsupported_renderer", "View does not name a supported built-in renderer"
)
return factory(incremental=incremental)
return factory()
def relative_output(snapshot: ProjectSnapshot, path: Path) -> str:

86
src/docforge/rendering.py
View file

@ -26,8 +26,6 @@ from .models import (
RenderView,
)
from .project import project_root_fingerprint
from .projection_contract import ProjectionReceiptV1
from .projection_policy import ManualProjectionMode, validate_manual_projection_mode
from .render_contract import PreparedRender, relative_output, renderer_for
from .telemetry import increment, stage
@ -38,26 +36,9 @@ MAX_RENDER_RECEIPT_BYTES = 64_000
class RenderService:
"""Render only declared views through fixed built-in renderer implementations."""
def __init__(
self,
project: ProjectService,
changesets: ChangesetStore | None = None,
*,
manual_policy: ManualProjectionMode = "explicit",
) -> None:
def __init__(self, project: ProjectService, changesets: ChangesetStore | None = None) -> None:
self.project = project
self.changesets = changesets or ChangesetStore(project)
self.manual_policy = validate_manual_projection_mode(manual_policy)
def _require_rendering(self, operation: str) -> None:
if self.manual_policy == "disabled":
raise DocForgeError(
"projection_policy_forbids_operation",
"Manual projection policy disables rendering work",
projection="manual",
mode=self.manual_policy,
operation=operation,
)
def status(self, view_id: str | None = None) -> dict[str, object]:
"""Report publication state from bounded receipts without rendering canonical content."""
@ -105,7 +86,6 @@ class RenderService:
def deep_status(self, view_id: str | None = None) -> dict[str, object]:
"""Recompute render output as the explicit side-effect-free equivalence oracle."""
self._require_rendering("deep_status")
snapshot = self.project.load()
config = snapshot.descriptor.render
if config is None:
@ -127,12 +107,7 @@ class RenderService:
snapshot.descriptor.root,
view.output_path,
)
prepared, _ = self._prepare(
snapshot,
view,
changeset_hash=None,
incremental=False,
)
prepared, _ = self._prepare(snapshot, view, changeset_hash=None)
state = "missing"
actual_hash: str | None = None
output = view.output_path
@ -176,7 +151,6 @@ class RenderService:
)
def render(self, view_id: str) -> dict[str, object]:
self._require_rendering("render")
with self._lock():
snapshot = self.project.load()
config = self._config(snapshot)
@ -399,8 +373,6 @@ class RenderService:
"template_file": final_template_file,
"output_file": final_output_file,
}
if prepared.projection_receipt is not None:
payload["projection_receipt"] = prepared.projection_receipt
raw = json.dumps(payload, sort_keys=True, indent=2).encode("utf-8") + b"\n"
if len(raw) > MAX_RENDER_RECEIPT_BYTES:
raise DocForgeError(
@ -600,10 +572,8 @@ class RenderService:
renderer = renderer_for(view)
template_file = receipt.get("template_file")
output_file = receipt.get("output_file")
fields = set(receipt)
projection_receipt = receipt.get("projection_receipt")
return (
fields in (required, required | {"projection_receipt"})
set(receipt) == required
and receipt.get("schema_version") == RENDER_RECEIPT_SCHEMA_VERSION
and receipt.get("project_id") == descriptor.project_id
and receipt.get("project_root_fingerprint") == project_root_fingerprint(descriptor.root)
@ -632,50 +602,6 @@ class RenderService:
descriptor.limits.max_render_bytes,
)
and cast(dict[str, object], output_file)["size"] == receipt.get("output_bytes")
and (
projection_receipt is None
or self._valid_projection_receipt(
projection_receipt,
renderer_id=renderer.renderer_id,
renderer_version=renderer.renderer_version,
output_hash=cast(str, receipt["output_hash"]),
output_bytes=cast(int, receipt["output_bytes"]),
)
)
)
@staticmethod
def _valid_projection_receipt(
value: object,
*,
renderer_id: str,
renderer_version: str,
output_hash: str,
output_bytes: int,
) -> bool:
if not isinstance(value, dict):
return False
try:
receipt = ProjectionReceiptV1.from_dict(cast(dict[str, object], value))
except DocForgeError:
return False
document = receipt.document
return (
document.get("kind") == "manual"
and document.get("renderer")
== {
"renderer_id": renderer_id,
"renderer_version": renderer_version,
}
and document.get("artifacts")
== [
{
"artifact_id": "manual.html",
"media_type": "text/html; charset=utf-8",
"sha256": output_hash,
"bytes": output_bytes,
}
]
)
@staticmethod
@ -737,7 +663,6 @@ class RenderService:
"reason": reason,
"verification": "receipt",
"receipt_schema_version": payload.get("schema_version"),
"projection_receipt": payload.get("projection_receipt"),
}
def _current_state(self) -> ProjectState | None:
@ -762,7 +687,6 @@ class RenderService:
}
def preview(self, changeset_id: str, view_id: str) -> dict[str, object]:
self._require_rendering("preview")
with self._lock():
snapshot, changeset_hash = self.changesets.projected_snapshot(changeset_id)
config = self._config(snapshot)
@ -807,12 +731,11 @@ class RenderService:
view: RenderView,
*,
changeset_hash: str | None,
incremental: bool = True,
) -> tuple[PreparedRender, bytes]:
increment("render_prepare_calls")
template = self._template_bytes(snapshot, view)
with stage("render.prepare"):
prepared = renderer_for(view, incremental=incremental).prepare(
prepared = renderer_for(view).prepare(
snapshot,
view,
template,
@ -927,7 +850,6 @@ class RenderService:
"expected_output_hash": prepared.output_hash,
"actual_output_hash": actual_hash,
"template_hash": prepared.template_hash,
"projection_receipt": prepared.projection_receipt,
"path": relative_output(snapshot, view.output_path),
"state": state,
}

6
src/docforge/telemetry.py
View file

@ -79,7 +79,6 @@ OPERATION_NAMES = frozenset(
"test",
"benchmark.m1",
"benchmark.m2",
"benchmark.m3",
"mcp.invoke",
"mcp.bootstrap",
"mcp.sync",
@ -97,8 +96,6 @@ OPERATION_NAMES = frozenset(
"mcp.generation_diff",
"mcp.validate_project",
"mcp.render_status",
"mcp.graph_plan",
"mcp.graph_render_status",
"mcp.visualize",
"mcp.visualization_status",
"mcp.stop_visualization",
@ -120,9 +117,6 @@ OPERATION_NAMES = frozenset(
"cli.impact",
"cli.context",
"cli.generation-diff",
"cli.graph-plan",
"cli.graph-render",
"cli.graph-render-status",
"cli.configure",
"cli.doctor",
"cli.render",

21
src/docforge/viewer_manager.py
View file

@ -28,10 +28,6 @@ from .errors import DocForgeError
from .index import ProjectIndex
from .models import IncrementalStateProject
from .project import project_root_fingerprint
from .projection_policy import (
LiveViewerProjectionMode,
validate_live_viewer_projection_mode,
)
from .telemetry import increment, stage
from .visualization import VISUALIZATION_TEMPLATE, VisualizationIndexSnapshot
@ -627,16 +623,9 @@ class ViewerManager:
class ViewerManagerClient:
"""Project-bound MCP-side client for the separately supervised manager service."""
def __init__(
self,
index: ProjectIndex,
*,
state_path: Path | None = None,
live_viewer_policy: LiveViewerProjectionMode = "on-demand",
) -> None:
def __init__(self, index: ProjectIndex, *, state_path: Path | None = None) -> None:
self.index = index
self.state_path = state_path or default_state_path()
self.live_viewer_policy = validate_live_viewer_projection_mode(live_viewer_policy)
def start(
self,
@ -645,14 +634,6 @@ class ViewerManagerClient:
query: str | None = None,
depth: int = 1,
) -> dict[str, object]:
if self.live_viewer_policy == "disabled":
raise DocForgeError(
"projection_policy_forbids_operation",
"Live viewer projection policy disables viewer startup",
projection="live_viewer",
mode=self.live_viewer_policy,
operation="start",
)
if node_id is not None and query is not None:
raise DocForgeError(
"invalid_visualization_target",

46
src/docforge/visualization.py
View file

@ -506,11 +506,11 @@ class VisualizationIndexSnapshot:
)
def source(self, node_id: str) -> dict[str, object]:
"""Return bounded source evidence stored in the pinned index generation."""
"""Return one node's bounded, project-confined UTF-8 source file."""
with self._connection() as connection:
row = connection.execute(
"SELECT node_id, source_path, source_anchor, content FROM nodes WHERE node_id = ?",
"SELECT node_id, source_path, source_anchor FROM nodes WHERE node_id = ?",
(node_id,),
).fetchone()
if row is None:
@ -518,8 +518,7 @@ class VisualizationIndexSnapshot:
"""
SELECT logic.logic_id AS node_id,
owner.source_path AS source_path,
logic.source_anchor AS source_anchor,
owner.content AS content
logic.source_anchor AS source_anchor
FROM logic_nodes AS logic
JOIN nodes AS owner ON owner.node_id = logic.owner_node_id
WHERE logic.logic_id = ?
@ -534,26 +533,51 @@ class VisualizationIndexSnapshot:
"No node has the requested stable ID",
node_id=node_id,
)
content = row["content"]
if not isinstance(content, str):
relative = Path(row["source_path"])
if relative.is_absolute() or ".." in relative.parts or not relative.parts:
raise DocForgeError("path_escape", "Node source path is unsafe", node_id=node_id)
source = self.project_root / relative
try:
resolved = source.resolve(strict=True)
except OSError as error:
raise DocForgeError(
"invalid_index",
"Pinned source evidence is invalid",
"missing_source",
"Node source file is unavailable",
node_id=node_id,
) from error
if (
source.is_symlink()
or resolved != source
or not source.is_relative_to(self.project_root)
or not source.is_file()
):
raise DocForgeError("path_escape", "Node source file is unsafe", node_id=node_id)
if source.stat().st_size > self.max_source_bytes:
raise DocForgeError(
"source_too_large",
"Node source exceeds the configured source limit",
node_id=node_id,
)
raw = content.encode("utf-8")
raw = source.read_bytes()
if len(raw) > self.max_source_bytes:
raise DocForgeError(
"source_too_large",
"Pinned source evidence exceeds the configured source limit",
"Node source exceeds the configured source limit",
node_id=node_id,
)
try:
content = raw.decode("utf-8")
except UnicodeDecodeError as error:
raise DocForgeError(
"invalid_source",
"Node source is not UTF-8",
node_id=node_id,
) from error
return self._result(
node_id=node_id,
source_path=row["source_path"],
source_anchor=row["source_anchor"],
content=content,
source_provenance="index_snapshot",
snapshot=True,
)

1
src/docforge_renderers/__init__.py
View file

@ -1 +0,0 @@
"""Capability-isolated renderer implementations for DocForge projection packages."""

321
src/docforge_renderers/graph.py
View file

@ -1,321 +0,0 @@
"""Deterministic self-contained renderer for one portable graph package."""
from __future__ import annotations
import base64
import hashlib
import html
import json
from time import perf_counter_ns
from typing import cast
from docforge.errors import DocForgeError
from docforge.projection_contract import (
ProjectionArtifact,
ProjectionPackageV1,
ProjectionReceiptV1,
ProjectionRenderResult,
)
PORTABLE_GRAPH_CSS = """
:root { color-scheme: light dark; font-family: system-ui, sans-serif; }
* { box-sizing: border-box; }
body { margin: 0; background: Canvas; color: CanvasText; }
.skip { position: absolute; left: -10000px; top: auto; }
.skip:focus { left: 1rem; top: 1rem; z-index: 2; padding: .5rem; background: Canvas; }
header, main { width: min(96%, 1100px); margin: 0 auto; }
header { padding: 1rem 0; }
.controls { display: flex; flex-wrap: wrap; gap: .75rem; align-items: end; }
label { display: grid; gap: .25rem; font-weight: 600; }
input, select, button { font: inherit; min-height: 2.75rem; padding: .45rem .65rem; }
button { cursor: pointer; }
button:focus-visible, input:focus-visible, select:focus-visible { outline: .2rem solid Highlight; }
.summary { margin: 1rem 0; }
.layout { display: grid; grid-template-columns: minmax(16rem, 1fr) minmax(20rem, 2fr); gap: 1rem; }
.panel { border: 1px solid GrayText; border-radius: .5rem; padding: 1rem; overflow: auto; }
html[data-enhanced="true"] main[data-mode="nodes"] .layout,
html[data-enhanced="true"] main[data-mode="flow"] .layout { grid-template-columns: 1fr; }
html[data-enhanced="true"] main[data-mode="nodes"] [data-panel="relationships"] { display: none; }
html[data-enhanced="true"] main[data-mode="flow"] [data-panel="nodes"] { display: none; }
.node-list { list-style: none; padding: 0; margin: 0; display: grid; gap: .5rem; }
.node-list button {
width: 100%; text-align: left; border: 1px solid GrayText; border-radius: .35rem;
}
.node-list button[aria-current="true"] { border-width: .2rem; }
table { border-collapse: collapse; width: 100%; }
th, td { text-align: left; border-bottom: 1px solid GrayText; padding: .5rem; vertical-align: top; }
caption { text-align: left; font-weight: 700; margin-bottom: .5rem; }
.muted { color: CanvasText; }
dialog {
max-width: min(42rem, calc(100% - 2rem));
border: 1px solid GrayText; border-radius: .5rem;
}
dialog::backdrop { background: rgb(0 0 0 / 55%); }
@media (max-width: 48rem) { .layout { grid-template-columns: 1fr; } }
@media (prefers-reduced-motion: reduce) {
*, *::before, *::after { scroll-behavior: auto !important; }
}
@media (forced-colors: active) {
.panel, .node-list button, dialog { border: 2px solid CanvasText; }
}
""".strip()
PORTABLE_GRAPH_JAVASCRIPT = r"""
(() => {
"use strict";
const plan = JSON.parse(document.getElementById("docforge-graph-plan").textContent);
const nodes = plan.graph.nodes;
const edges = plan.graph.edges;
const list = document.getElementById("node-list");
const rows = document.getElementById("edge-rows");
const filter = document.getElementById("filter");
const mode = document.getElementById("mode");
const main = document.getElementById("main");
const status = document.getElementById("status");
const dialog = document.getElementById("node-dialog");
const detail = document.getElementById("node-detail");
const close = document.getElementById("close-dialog");
let opener = null;
document.documentElement.dataset.enhanced = "true";
const matches = (node) => {
const query = filter.value.trim().toLocaleLowerCase();
const fields = [
node.node_id, node.title, node.summary, node.family, node.status, ...node.tags
];
return !query || fields
.join(" ").toLocaleLowerCase().includes(query);
};
const selectedIds = () => new Set(nodes.filter(matches).map((node) => node.node_id));
const render = () => {
main.dataset.mode = mode.value;
const visible = nodes.filter(matches);
const ids = selectedIds();
list.replaceChildren(...visible.map((node) => {
const item = document.createElement("li");
const button = document.createElement("button");
button.type = "button";
button.textContent = `${node.title} (${node.node_id})`;
button.dataset.nodeId = node.node_id;
button.addEventListener("click", () => inspect(node, button));
item.append(button);
return item;
}));
const visibleEdges = edges.filter((edge) => ids.has(edge.source_id) && ids.has(edge.target_id));
rows.replaceChildren(...visibleEdges.map((edge) => {
const row = document.createElement("tr");
[edge.source_id, edge.relation, edge.target_id].forEach((value) => {
const cell = document.createElement("td");
cell.textContent = value;
row.append(cell);
});
return row;
}));
status.textContent = `${visible.length} nodes and ${visibleEdges.length} relationships `
+ `shown in ${mode.value} mode.`;
};
const inspect = (node, button) => {
opener = button;
detail.replaceChildren();
const heading = document.createElement("h2");
heading.id = "node-dialog-title";
heading.textContent = node.title;
const identity = document.createElement("p");
identity.textContent = `${node.node_id} · ${node.family} · ${node.status}`;
const summary = document.createElement("p");
summary.textContent = node.summary;
detail.append(heading, identity, summary);
dialog.showModal();
close.focus();
};
close.addEventListener("click", () => dialog.close());
dialog.addEventListener("close", () => opener?.focus());
filter.addEventListener("input", render);
mode.addEventListener("change", render);
render();
})();
""".strip()
def _csp_hash(content: str) -> str:
digest = hashlib.sha256(content.encode("utf-8")).digest()
return base64.b64encode(digest).decode("ascii")
def _embedded_json(value: object) -> str:
return (
json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
.replace("&", "\\u0026")
.replace("<", "\\u003c")
.replace(">", "\\u003e")
)
def _static_node_markup(plan: dict[str, object]) -> str:
graph = cast(dict[str, object], plan["graph"])
nodes = cast(list[dict[str, object]], graph["nodes"])
parts: list[str] = []
for node in nodes:
node_id = html.escape(cast(str, node["node_id"]))
attribute_node_id = html.escape(cast(str, node["node_id"]), quote=True)
title = html.escape(cast(str, node["title"]))
family = html.escape(cast(str, node["family"]))
status = html.escape(cast(str, node["status"]))
summary = html.escape(cast(str, node["summary"]))
parts.append(
"<li><article>"
f'<button type="button" data-node-id="{attribute_node_id}">'
f"{title} ({node_id})</button>"
f'<p class="muted">{family} · {status}</p>'
f"<p>{summary}</p>"
"</article></li>"
)
return "".join(parts)
def _static_edge_markup(plan: dict[str, object]) -> str:
graph = cast(dict[str, object], plan["graph"])
edges = cast(list[dict[str, object]], graph["edges"])
return "".join(
"<tr>"
f"<td>{html.escape(cast(str, edge['source_id']))}</td>"
f"<td>{html.escape(cast(str, edge['relation']))}</td>"
f"<td>{html.escape(cast(str, edge['target_id']))}</td>"
"</tr>"
for edge in edges
)
class PortableGraphHtmlRenderer:
"""Render a validated graph package without querying or publishing project state."""
renderer_id = "portable_graph_html"
renderer_version = "1"
def render(self, package: ProjectionPackageV1) -> ProjectionRenderResult:
started = perf_counter_ns()
package = ProjectionPackageV1.from_dict(package.as_dict())
document = package.document
if package.kind != "graph":
raise DocForgeError("invalid_projection", "Graph renderer requires a graph package")
renderer = cast(dict[str, object], document["renderer"])
if renderer != {
"renderer_id": self.renderer_id,
"renderer_version": self.renderer_version,
}:
raise DocForgeError("unsupported_renderer", "Graph renderer identity is incompatible")
if document["components"] != [
{"component_id": "graph.portable-document@1"},
{"component_id": "graph.accessible-list@1"},
{"component_id": "graph.relationship-table@1"},
]:
raise DocForgeError(
"invalid_projection",
"Portable graph renderer component declarations are incompatible",
)
if document["assets"] != []:
raise DocForgeError(
"invalid_projection",
"Portable graph renderer does not accept project-provided assets",
)
plan = cast(dict[str, object], document["plan"])
view = cast(dict[str, object], plan["view"])
initial_mode = view.get("initial_mode")
policy = cast(dict[str, object], plan["policy"])
if (
initial_mode not in {"nodes", "flow", "web"}
or policy.get("logic_requested") is not False
):
raise DocForgeError(
"unsupported_renderer",
"Portable graph renderer version 1 does not render Logic projections",
)
title = html.escape(cast(str, view["title"]))
project = cast(dict[str, object], plan["project"])
embedded = _embedded_json(plan)
static_nodes = _static_node_markup(plan)
static_edges = _static_edge_markup(plan)
csp = (
"default-src 'none'; "
f"style-src 'sha256-{_csp_hash(PORTABLE_GRAPH_CSS)}'; "
f"script-src 'sha256-{_csp_hash(PORTABLE_GRAPH_JAVASCRIPT)}'; "
"img-src 'none'; connect-src 'none'; object-src 'none'; base-uri 'none'; "
"form-action 'none'; frame-ancestors 'none'"
)
output = (
"<!DOCTYPE html>\n"
'<html lang="en">\n'
"<head>\n"
'<meta charset="utf-8">\n'
'<meta name="viewport" content="width=device-width, initial-scale=1">\n'
'<meta http-equiv="Content-Security-Policy" '
f'content="{html.escape(csp, quote=True)}">\n'
f"<title>{title} · DocForge graph</title>\n"
f"<style>{PORTABLE_GRAPH_CSS}</style>\n"
"</head>\n"
"<body>\n"
'<a class="skip" href="#main">Skip to graph content</a>\n'
"<header>\n"
f"<h1>{title}</h1>\n"
f'<p class="muted">Generation {html.escape(cast(str, project["source_hash"]))}</p>\n'
'<div class="controls" role="group" aria-label="Graph controls">\n'
'<label>Filter nodes<input id="filter" type="search" autocomplete="off"></label>\n'
'<label>View mode<select id="mode">'
f'<option value="nodes"{" selected" if initial_mode == "nodes" else ""}>Nodes</option>'
f'<option value="flow"{" selected" if initial_mode == "flow" else ""}>Flow</option>'
f'<option value="web"{" selected" if initial_mode == "web" else ""}>Web</option>'
"</select></label>\n"
"</div>\n"
'<p id="status" class="summary" role="status" aria-live="polite"></p>\n'
"</header>\n"
f'<main id="main" tabindex="-1" data-mode="{initial_mode}">\n'
'<div class="layout">\n'
'<section class="panel" data-panel="nodes" aria-labelledby="nodes-title">'
'<h2 id="nodes-title">Nodes</h2>'
f'<ul id="node-list" class="node-list">{static_nodes}</ul></section>\n'
'<section class="panel" data-panel="relationships" aria-labelledby="relations-title">'
'<h2 id="relations-title">Relationships</h2>'
"<table><caption>Selected graph facts</caption><thead><tr>"
'<th scope="col">Source</th><th scope="col">Relation</th>'
f'<th scope="col">Target</th></tr></thead><tbody id="edge-rows">{static_edges}</tbody>'
"</table>"
"</section>\n"
"</div>\n"
"</main>\n"
'<dialog id="node-dialog" aria-labelledby="node-dialog-title">'
'<div id="node-detail"><h2 id="node-dialog-title">Node details</h2></div>'
'<button id="close-dialog" type="button">Close</button></dialog>\n'
f'<script id="docforge-graph-plan" type="application/json">{embedded}</script>\n'
f"<script>{PORTABLE_GRAPH_JAVASCRIPT}</script>\n"
"</body>\n"
"</html>\n"
).encode()
policy = cast(dict[str, object], document["output_policy"])
maximum = policy.get("max_total_bytes")
if set(policy) != {"artifact_ids", "max_total_bytes"} or policy.get("artifact_ids") != [
"portable-graph.html"
]:
raise DocForgeError(
"invalid_projection",
"Portable graph output policy is incompatible",
)
if type(maximum) is not int or maximum < 1 or len(output) > maximum:
raise DocForgeError("render_too_large", "Rendered output exceeds the configured limit")
artifact = ProjectionArtifact(
artifact_id="portable-graph.html",
media_type="text/html; charset=utf-8",
content=output,
)
receipt = ProjectionReceiptV1.create(
kind="graph",
package_id=package.package_id,
plan_id=cast(str, document["plan_id"]),
renderer=dict(renderer),
artifacts=[artifact.evidence()],
diagnostics={
"warnings": [],
},
timing={"elapsed_ns": perf_counter_ns() - started},
peak_memory_bytes=None,
)
return ProjectionRenderResult((artifact,), receipt)

264
src/docforge_renderers/manual.py
View file

@ -1,264 +0,0 @@
"""Plan-only renderer for the built-in DocForge manual artifact."""
from __future__ import annotations
import hashlib
import html
import re
from time import perf_counter_ns
from typing import cast
from markdown_it import MarkdownIt
from docforge.errors import DocForgeError
from docforge.projection_contract import (
ProjectionArtifact,
ProjectionPackageV1,
ProjectionReceiptV1,
ProjectionRenderResult,
)
from docforge.projection_fragments import (
FragmentRecord,
fragment_semantic_hash,
)
_TEMPLATE_TOKEN = re.compile(r"{{\s*([a-z_][a-z0-9_]*)\s*}}")
_ALLOWED_TOKENS = frozenset(
{
"docforge_content",
"docforge_project_id",
"docforge_render_identity",
"docforge_title",
"docforge_view_id",
}
)
_ACTIVE_TEMPLATE_CONTENT = re.compile(
r"<\s*(?:script|iframe|object|embed)\b"
r"|\son[a-z0-9_-]+\s*="
r"|javascript\s*:"
r"|<\s*meta\b[^>]*\bhttp-equiv\s*=\s*[\"']?\s*refresh\b",
re.IGNORECASE,
)
_MANUAL_PAGE_COMPONENT = "manual.page@1"
class ManualHtmlRenderer:
"""Transform one validated path-free package without graph-selection authority."""
renderer_id = "generic_html"
def __init__(self, renderer_version: str) -> None:
self.renderer_version = renderer_version
self.markdown = MarkdownIt("commonmark", {"html": False, "typographer": False})
def render(
self,
package: ProjectionPackageV1,
*,
render_identity: str | None = None,
) -> ProjectionRenderResult:
started = perf_counter_ns()
package = ProjectionPackageV1.from_dict(package.as_dict())
document = package.document
if package.kind != "manual":
raise DocForgeError("invalid_projection", "Manual renderer requires a manual package")
renderer = cast(dict[str, object], document["renderer"])
if renderer != {
"renderer_id": self.renderer_id,
"renderer_version": self.renderer_version,
}:
raise DocForgeError("unsupported_renderer", "Manual renderer identity is incompatible")
plan = cast(dict[str, object], document["plan"])
assets = cast(list[object], document["assets"])
if len(assets) not in {1, 2} or not isinstance(assets[0], dict):
raise DocForgeError("invalid_projection", "Manual template asset is invalid")
asset = cast(dict[str, object], assets[0])
if (
set(asset)
not in (
{"asset_id", "media_type", "sha256", "text"},
{"asset_id", "media_type", "sha256", "text", "render_identity"},
)
or asset.get("asset_id") != "manual.template"
or asset.get("media_type") != "text/html; charset=utf-8"
or not isinstance(asset.get("text"), str)
):
raise DocForgeError("invalid_projection", "Manual template asset is invalid")
template = cast(str, asset["text"])
template_bytes = template.encode("utf-8")
if hashlib.sha256(template_bytes).hexdigest() != asset.get("sha256"):
raise DocForgeError("invalid_projection", "Manual template asset hash is invalid")
if _ACTIVE_TEMPLATE_CONTENT.search(template):
raise DocForgeError(
"invalid_template",
"Render template contains active or executable content",
)
tokens = _TEMPLATE_TOKEN.findall(template)
unknown = sorted(set(tokens) - _ALLOWED_TOKENS)
remainder = _TEMPLATE_TOKEN.sub("", template)
if unknown or "{{" in remainder or "}}" in remainder:
raise DocForgeError(
"invalid_template",
"Render template contains unsupported tokens",
tokens=unknown,
)
if tokens.count("docforge_content") != 1:
raise DocForgeError(
"invalid_template",
"Render template must contain docforge_content exactly once",
)
packaged_identity = asset.get("render_identity")
if packaged_identity is not None and (
not isinstance(packaged_identity, str)
or len(packaged_identity) != 64
or any(character not in "0123456789abcdef" for character in packaged_identity)
):
raise DocForgeError(
"invalid_projection",
"Manual render identity is invalid",
)
if render_identity is not None and packaged_identity not in {None, render_identity}:
raise DocForgeError(
"invalid_projection",
"Manual render identity is inconsistent",
)
identity = render_identity or packaged_identity or cast(str, plan["plan_id"])
fragments = self._fragments(plan, assets[1:] if len(assets) == 2 else [])
project = cast(dict[str, object], plan["project"])
view = cast(dict[str, object], plan["view"])
replacements = {
"docforge_content": self._content(plan, fragments),
"docforge_project_id": html.escape(cast(str, project["project_id"]), quote=True),
"docforge_render_identity": identity,
"docforge_title": html.escape(cast(str, view["title"]), quote=True),
"docforge_view_id": html.escape(cast(str, view["view_id"]), quote=True),
}
rendered = _TEMPLATE_TOKEN.sub(lambda match: replacements[match.group(1)], template)
output = rendered.rstrip().encode("utf-8") + b"\n"
policy = cast(dict[str, object], document["output_policy"])
maximum = policy.get("max_total_bytes")
if type(maximum) is not int or maximum < 1 or len(output) > maximum:
raise DocForgeError("render_too_large", "Rendered output exceeds the configured limit")
artifact = ProjectionArtifact(
artifact_id="manual.html",
media_type="text/html; charset=utf-8",
content=output,
)
receipt = ProjectionReceiptV1.create(
kind="manual",
package_id=package.package_id,
plan_id=cast(str, document["plan_id"]),
renderer=dict(renderer),
artifacts=[artifact.evidence()],
diagnostics={"warnings": []},
timing={"elapsed_ns": perf_counter_ns() - started},
peak_memory_bytes=None,
)
return ProjectionRenderResult((artifact,), receipt)
def _fragments(
self,
plan: dict[str, object],
assets: list[object],
) -> dict[str, str]:
if not assets:
return {}
asset = assets[0]
if not isinstance(asset, dict):
raise DocForgeError("invalid_projection", "Manual fragment asset is invalid")
document = cast(dict[str, object], asset)
if (
set(document) != {"asset_id", "media_type", "records"}
or document.get("asset_id") != "manual.fragments"
or document.get("media_type") != "application/vnd.docforge.projection-fragments.v1+json"
or not isinstance(document.get("records"), list)
):
raise DocForgeError("invalid_projection", "Manual fragment asset is invalid")
pages = cast(list[object], plan["pages"])
records = cast(list[object], document["records"])
if len(records) != len(pages):
raise DocForgeError(
"invalid_projection",
"Manual fragment inventory is incomplete",
)
fragments: dict[str, str] = {}
for page_value, record_value in zip(pages, records, strict=True):
if not isinstance(page_value, dict):
raise DocForgeError("invalid_projection", "Manual page is invalid")
page = cast(dict[str, object], page_value)
record = FragmentRecord.from_dict(record_value)
if (
record.key.projection_kind != "manual"
or record.key.renderer_id != self.renderer_id
or record.key.renderer_version != self.renderer_version
or record.key.component_version != _MANUAL_PAGE_COMPONENT
or record.key.semantic_input_hash != fragment_semantic_hash(page)
):
raise DocForgeError(
"invalid_projection",
"Manual fragment identity does not match its page semantics",
)
try:
fragment = record.content.decode("utf-8")
except UnicodeDecodeError as error:
raise DocForgeError(
"invalid_projection",
"Manual fragment content is not valid UTF-8",
) from error
node_id = cast(str, page["node_id"])
if node_id in fragments or fragment != self.render_page_fragment(page):
raise DocForgeError(
"invalid_projection",
"Manual fragment content does not match its page semantics",
)
fragments[node_id] = fragment
return fragments
def render_page_fragment(self, page: dict[str, object]) -> str:
"""Render one page from complete plan semantics without project authority."""
node_id = cast(str, page["node_id"])
sections = [
f'<section id="node-{html.escape(node_id, quote=True)}">',
f"<h2>{html.escape(cast(str, page['title']))}</h2>",
'<dl class="docforge-node-meta">',
f"<dt>ID</dt><dd>{html.escape(node_id)}</dd>",
f"<dt>Family</dt><dd>{html.escape(cast(str, page['family']))}</dd>",
f"<dt>Status</dt><dd>{html.escape(cast(str, page['status']))}</dd>",
f"<dt>Authority</dt><dd>{html.escape(cast(str, page['authority']))}</dd>",
"</dl>",
f'<p class="docforge-summary">{html.escape(cast(str, page["summary"]))}</p>',
self.markdown.render(cast(str, page["content"])).rstrip(),
]
relationships = cast(list[object], page["cross_references"])
if relationships:
sections.append('<ul class="docforge-relationships">')
for relationship_value in relationships:
relationship = cast(dict[str, object], relationship_value)
sections.append(
f"<li>{html.escape(cast(str, relationship['relation']))}: "
f"{html.escape(cast(str, relationship['target_id']))}</li>"
)
sections.append("</ul>")
sections.append("</section>")
return "\n".join(sections)
def _content(
self,
plan: dict[str, object],
fragments: dict[str, str],
) -> str:
navigation = ['<nav aria-label="Documentation"><ul>']
for value in cast(list[object], plan["navigation"]):
item = cast(dict[str, object], value)
navigation.append(
f'<li><a href="#node-{html.escape(cast(str, item["node_id"]), quote=True)}">'
f"{html.escape(cast(str, item['title']))}</a></li>"
)
navigation.append("</ul></nav>")
sections = [*navigation]
for value in cast(list[object], plan["pages"]):
page = cast(dict[str, object], value)
node_id = cast(str, page["node_id"])
sections.append(fragments.get(node_id, self.render_page_fragment(page)))
return "\n".join(sections)

1
src/docforge_renderers/py.typed
View file

@ -1 +0,0 @@
# PEP 561 marker for the typed DocForge renderer package.

187
tests/accessibility.spec.mjs
View file

@ -1,187 +0,0 @@
import AxeBuilder from "@axe-core/playwright";
import { expect, test } from "@playwright/test";
import { spawn } from "node:child_process";
import { createInterface } from "node:readline";
const AXE_TAGS = [
"wcag2a",
"wcag2aa",
"wcag21a",
"wcag21aa",
"wcag22a",
"wcag22aa",
];
const MANUAL_AXE_TAGS = AXE_TAGS.filter((tag) => !tag.startsWith("wcag22"));
let fixtureProcess;
let surfaces;
function startFixture() {
const python = process.env.DOCFORGE_PYTHON || ".venv/bin/python";
const child = spawn(python, ["tools/accessibility_fixture.py"], {
cwd: process.cwd(),
stdio: ["pipe", "pipe", "pipe"],
});
let stderr = "";
child.stderr.setEncoding("utf8");
child.stderr.on("data", (chunk) => {
stderr += chunk;
});
const lines = createInterface({ input: child.stdout });
const ready = new Promise((resolve, reject) => {
let settled = false;
lines.once("line", (line) => {
settled = true;
try {
const payload = JSON.parse(line);
if (
payload.schema_version !== 1
|| typeof payload.manual_html !== "string"
|| typeof payload.portable_html !== "string"
|| typeof payload.live_url !== "string"
) {
throw new Error("Accessibility fixture returned an invalid payload");
}
resolve(payload);
} catch (error) {
reject(error);
} finally {
lines.close();
}
});
child.once("exit", (code, signal) => {
if (!settled) {
reject(
new Error(
`Accessibility fixture exited before readiness `
+ `(code=${code}, signal=${signal}):\n${stderr}`,
),
);
}
});
});
return { child, ready };
}
async function stopFixture(child) {
if (child.exitCode !== null || child.signalCode !== null) {
return;
}
const exited = new Promise((resolve) => child.once("exit", resolve));
child.stdin.end();
await Promise.race([
exited,
new Promise((_, reject) => {
setTimeout(() => reject(new Error("Accessibility fixture did not stop")), 5_000);
}),
]);
}
function violationReport(violations) {
return violations.map((violation) => {
const targets = violation.nodes
.flatMap((node) => node.target)
.join(", ");
return `${violation.id} (${violation.impact}): ${violation.help}\n ${targets}`;
}).join("\n");
}
async function expectNoAxeViolations(page, tags = AXE_TAGS) {
const results = await new AxeBuilder({ page }).withTags(tags).analyze();
expect(results.violations, violationReport(results.violations)).toEqual([]);
}
async function tabTo(page, selector, maximumTabs = 40) {
for (let count = 0; count < maximumTabs; count += 1) {
await page.keyboard.press("Tab");
if (await page.evaluate((target) => document.activeElement?.matches(target), selector)) {
return page.locator(selector).filter({ visible: true }).first();
}
}
throw new Error(`Keyboard focus did not reach ${selector}`);
}
test.beforeAll(async () => {
const fixture = startFixture();
fixtureProcess = fixture.child;
surfaces = await fixture.ready;
});
test.afterAll(async () => {
await stopFixture(fixtureProcess);
});
test("generated manual has no axe violations and its navigation works by keyboard", async ({
page,
}) => {
await page.setContent(surfaces.manual_html, { waitUntil: "load" });
await expect(page.locator("main section")).not.toHaveCount(0);
// The generic renderer owns structure, while this frozen project template owns target sizing.
await expectNoAxeViolations(page, MANUAL_AXE_TAGS);
await page.keyboard.press("Tab");
const firstNavigationLink = page.locator("nav[aria-label='Documentation'] a").first();
await expect(firstNavigationLink).toBeFocused();
const target = await firstNavigationLink.getAttribute("href");
expect(target).toMatch(/^#[A-Za-z0-9_.-]+$/);
await page.keyboard.press("Enter");
await expect.poll(() => page.evaluate(() => window.location.hash)).toBe(target);
});
test("portable graph supports skip, filter, view, and dialog keyboard flows", async ({ page }) => {
await page.goto("about:blank");
await page.setContent(surfaces.portable_html, { waitUntil: "load" });
await expect(page.locator("#status")).toContainText("nodes and");
await expectNoAxeViolations(page);
await page.keyboard.press("Tab");
await expect(page.locator("a.skip")).toBeFocused();
await page.keyboard.press("Enter");
await expect(page.locator("main#main")).toBeFocused();
await page.goto("about:blank");
await page.setContent(surfaces.portable_html, { waitUntil: "load" });
await page.keyboard.press("Tab");
await page.keyboard.press("Tab");
await expect(page.locator("#filter")).toBeFocused();
await page.keyboard.type("guide.workflow");
await expect(page.locator("#status")).toContainText("1 nodes and");
await page.keyboard.press("Tab");
await expect(page.locator("#mode")).toBeFocused();
await page.keyboard.press("ArrowDown");
await expect(page.locator("main#main")).toHaveAttribute("data-mode", "flow");
await page.keyboard.press("ArrowUp");
await expect(page.locator("main#main")).toHaveAttribute("data-mode", "nodes");
await page.keyboard.press("Tab");
const nodeButton = page.locator("#node-list button").first();
await expect(nodeButton).toBeFocused();
await page.keyboard.press("Enter");
await expect(page.locator("#node-dialog")).toHaveAttribute("open", "");
await expect(page.locator("#close-dialog")).toBeFocused();
await expectNoAxeViolations(page);
await page.keyboard.press("Escape");
await expect(page.locator("#node-dialog")).not.toHaveAttribute("open", "");
await expect(nodeButton).toBeFocused();
});
test("live viewer passes axe and exposes keyboard graph and resize controls", async ({ page }) => {
await page.goto(surfaces.live_url);
await expect(page.locator("#status")).toContainText("nodes ·");
await expect(page.locator("#graph g.node[role='button']").first()).toBeVisible();
await expectNoAxeViolations(page);
const resizer = await tabTo(page, "#left-resizer");
const originalWidth = Number(await resizer.getAttribute("aria-valuenow"));
await page.keyboard.press("ArrowRight");
await expect(resizer).toHaveAttribute("aria-valuenow", String(originalWidth + 16));
await tabTo(page, "#graph g.node[role='button']");
await page.keyboard.press("Shift+Enter");
await expect(page.locator("#node-dialog")).toHaveAttribute("open", "");
await expect(page.locator("#close-node-dialog")).toBeFocused();
await expectNoAxeViolations(page);
await page.keyboard.press("Escape");
await expect(page.locator("#node-dialog")).not.toHaveAttribute("open", "");
});

43
tests/test_cli.py
View file

@ -17,25 +17,6 @@ from docforge.viewer_manager import ViewerManager
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
GRAPH_CONFIG = """
[graph_render]
output_root = ".docforge/portable-graph"
[[graph_render.views]]
id = "architecture"
renderer = "portable_graph_html"
output = "architecture.html"
title = "Alpha architecture"
root = "guide.workflow"
initial_mode = "nodes"
depth = 2
max_nodes = 20
max_edges = 40
max_work = 1000
include_logic = false
"""
class DocForgeCliTests(unittest.TestCase):
def copy_fixture(self, destination: Path) -> Path:
@ -153,30 +134,6 @@ class DocForgeCliTests(unittest.TestCase):
else:
os.environ["DOCFORGE_VIEWER_MANAGER_STATE"] = previous
def test_portable_graph_plan_render_and_status_are_self_service(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
descriptor = root / ".docforge/project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
encoding="utf-8",
)
parser = _parser()
planned = _run(
parser.parse_args(["--project-root", str(root), "graph-plan", "architecture"])
)
self.assertEqual("architecture", planned["view_id"])
rendered = _run(
parser.parse_args(["--project-root", str(root), "graph-render", "architecture"])
)
self.assertEqual("current", rendered["state"])
status = _run(
parser.parse_args(
["--project-root", str(root), "graph-render-status", "architecture"]
)
)
self.assertEqual("current", status["state"])
if __name__ == "__main__":
unittest.main()

265
tests/test_client_integration.py
View file

@ -1,7 +1,6 @@
from __future__ import annotations
import contextlib
import hashlib
import io
import json
import os
@ -19,7 +18,6 @@ from jsonschema import Draft202012Validator
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from docforge.changeset_contract import document_hash
from docforge.cli import _parser, _run, main
from docforge.client_config import (
_read_existing,
@ -41,19 +39,6 @@ CONFIGURATION_SCHEMA = json.loads(
DOCTOR_SCHEMA = json.loads((SCHEMAS / "doctor-result.schema.json").read_text(encoding="utf-8"))
POLICY_SCHEMA = json.loads((SCHEMAS / "policy.schema.json").read_text(encoding="utf-8"))
GRAPH_RENDER_CONFIG = """
[graph_render]
output_root = ".docforge/portable-graph"
[[graph_render.views]]
id = "architecture"
renderer = "portable_graph_html"
output = "architecture.html"
title = "Alpha architecture"
root = "guide.workflow"
"""
class ClientIntegrationTests(unittest.TestCase):
def copy_fixture(self, destination: Path) -> Path:
@ -171,256 +156,6 @@ class ClientIntegrationTests(unittest.TestCase):
)
self.assertEqual("invalid_capability_binding", escalated.exception.code)
def test_nondefault_projection_policy_selectors_serialize_and_validate(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8") + GRAPH_RENDER_CONFIG,
encoding="utf-8",
)
project = Project.open(root)
expected_arguments = [
"-I",
"-m",
"docforge.mcp_server",
"--project-root",
str(root),
"--capability-mode",
"read",
"--manual-render-policy",
"disabled",
"--portable-graph-policy",
"disabled",
"--live-viewer-policy",
"disabled",
]
for client in ("codex", "claude", "openclaw"):
with self.subTest(client=client):
result = generate_client_configuration(
project,
client,
manual_render_policy="disabled",
portable_graph_policy="disabled",
live_viewer_policy="disabled",
)
Draft202012Validator(CONFIGURATION_SCHEMA).validate(result)
_validate_configuration_result(result)
self.assertEqual(
{
"schema_version": 2,
"manual": "disabled",
"portable_graph": "disabled",
"live_viewer": "disabled",
},
result["projection_policy"],
)
self.assertEqual(expected_arguments, result["binding"]["args"])
content = result["artifact"]["content"]
if client == "codex":
document = tomllib.loads(content)
serialized = document["mcp_servers"][result["server_name"]]["args"]
elif client == "claude":
document = json.loads(content)
serialized = document["mcpServers"][result["server_name"]]["args"]
else:
document = json.loads(content)
serialized = document["mcp"]["servers"][result["server_name"]]["args"]
self.assertEqual(expected_arguments, serialized)
canonical = json.dumps(
result["projection_policy"],
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
self.assertEqual(
hashlib.sha256(canonical).hexdigest(),
result["projection_policy_hash"],
)
def test_projection_policy_schema_and_configuration_validation_reject_drift(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture(Path(directory)))
result = generate_client_configuration(
project,
"codex",
manual_render_policy="disabled",
live_viewer_policy="disabled",
)
validator = Draft202012Validator(CONFIGURATION_SCHEMA)
validator.validate(result)
_validate_configuration_result(result)
for field, value in (
("schema_version", 1),
("manual", "on-demand"),
("portable_graph", "auto"),
("live_viewer", "explicit"),
):
with self.subTest(field=field):
drifted = json.loads(json.dumps(result))
drifted["projection_policy"][field] = value
self.assertTrue(list(validator.iter_errors(drifted)))
missing = json.loads(json.dumps(result))
missing.pop("projection_policy")
self.assertTrue(list(validator.iter_errors(missing)))
extra = json.loads(json.dumps(result))
extra["projection_policy"]["project_path"] = "/private/project"
self.assertTrue(list(validator.iter_errors(extra)))
mismatched = json.loads(json.dumps(result))
mismatched["projection_policy"]["manual"] = "explicit"
canonical = json.dumps(
mismatched["projection_policy"],
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
mismatched["projection_policy_hash"] = hashlib.sha256(canonical).hexdigest()
with self.assertRaises(AssertionError):
_validate_configuration_result(mismatched)
bad_hash = json.loads(json.dumps(result))
bad_hash["projection_policy_hash"] = "0" * 64
with self.assertRaises(AssertionError):
_validate_configuration_result(bad_hash)
defaulted = generate_client_configuration(project, "codex")
self.assertNotIn(
"--manual-render-policy",
defaulted["binding"]["args"],
)
defaulted["projection_policy"]["manual"] = "disabled"
canonical = json.dumps(
defaulted["projection_policy"],
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
defaulted["projection_policy_hash"] = hashlib.sha256(canonical).hexdigest()
with self.assertRaises(AssertionError):
_validate_configuration_result(defaulted)
unavailable = json.loads(json.dumps(result))
unavailable["projection_availability"]["manual_configured"] = False
with self.assertRaises(AssertionError):
_validate_configuration_result(unavailable)
descriptor = project.descriptor.descriptor_path
descriptor.write_text(
descriptor.read_text(encoding="utf-8") + GRAPH_RENDER_CONFIG,
encoding="utf-8",
)
graph_project = Project.open(project.descriptor.root)
graph_defaulted = generate_client_configuration(graph_project, "codex")
self.assertNotIn(
"--portable-graph-policy",
graph_defaulted["binding"]["args"],
)
coordinated_graph_drift = json.loads(json.dumps(graph_defaulted))
coordinated_graph_drift["projection_policy"]["portable_graph"] = "disabled"
coordinated_graph_drift["projection_availability"]["portable_graph_configured"] = False
canonical = json.dumps(
coordinated_graph_drift["projection_policy"],
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
coordinated_graph_drift["projection_policy_hash"] = hashlib.sha256(
canonical
).hexdigest()
coordinated_graph_drift["configuration_hash"] = document_hash(
{
"schema_version": 1,
"client": coordinated_graph_drift["client"],
"server_name": coordinated_graph_drift["server_name"],
"project": coordinated_graph_drift["project"],
"binding": coordinated_graph_drift["binding"],
"effective_policy": coordinated_graph_drift["effective_policy"],
"projection_policy": coordinated_graph_drift["projection_policy"],
"projection_policy_hash": coordinated_graph_drift["projection_policy_hash"],
"projection_availability": coordinated_graph_drift["projection_availability"],
"artifact_format": coordinated_graph_drift["artifact"]["format"],
"artifact_content_sha256": coordinated_graph_drift["artifact"][
"content_sha256"
],
}
)
with self.assertRaises(AssertionError):
_validate_configuration_result(coordinated_graph_drift)
invalid_selections = (
{"manual_render_policy": "sometimes"},
{"portable_graph_policy": "auto"},
{"live_viewer_policy": "always"},
)
for selection in invalid_selections:
with self.subTest(selection=selection):
with self.assertRaises(DocForgeError) as raised:
generate_client_configuration(graph_project, "codex", **selection)
self.assertEqual("invalid_projection_policy", raised.exception.code)
def test_doctor_round_trips_projection_selectors_and_rejects_invalid_modes(self) -> None:
with tempfile.TemporaryDirectory() as directory:
parent = Path(directory)
project = Project.open(self.copy_fixture(parent))
ProjectIndex(project).build()
config = parent / "openclaw.json"
generated = generate_client_configuration(
project,
"openclaw",
manual_render_policy="disabled",
live_viewer_policy="disabled",
output=config,
)
healthy = run_doctor(
project,
"openclaw",
config_path=config,
server_name=generated["server_name"],
)
Draft202012Validator(DOCTOR_SCHEMA).validate(healthy)
self.assertEqual("healthy", healthy["doctor_state"])
self.assertIn(
"effective_policy_valid",
[check["code"] for check in healthy["checks"]],
)
document = json.loads(config.read_text(encoding="utf-8"))
entry = document["mcp"]["servers"][generated["server_name"]]
arguments = entry["args"]
manual_position = arguments.index("--manual-render-policy") + 1
arguments[manual_position] = "sometimes"
config.write_text(json.dumps(document, sort_keys=True), encoding="utf-8")
invalid = run_doctor(
project,
"openclaw",
config_path=config,
server_name=generated["server_name"],
)
Draft202012Validator(DOCTOR_SCHEMA).validate(invalid)
self.assertEqual("unhealthy", invalid["doctor_state"])
policy_check = next(
check for check in invalid["checks"] if check["check_id"] == "policy.effective"
)
self.assertEqual("invalid_projection_policy", policy_check["code"])
arguments[manual_position] = "auto"
config.write_text(json.dumps(document, sort_keys=True), encoding="utf-8")
unavailable = run_doctor(
project,
"openclaw",
config_path=config,
server_name=generated["server_name"],
)
Draft202012Validator(DOCTOR_SCHEMA).validate(unavailable)
self.assertEqual("unhealthy", unavailable["doctor_state"])
policy_check = next(
check for check in unavailable["checks"] if check["check_id"] == "policy.effective"
)
self.assertEqual("projection_policy_unavailable", policy_check["code"])
def test_explicit_fragment_write_is_atomic_conflict_aware_and_private(self) -> None:
with tempfile.TemporaryDirectory() as directory:
parent = Path(directory)

404
tests/test_graph_projection.py
View file

@ -1,404 +0,0 @@
from __future__ import annotations
import json
import tempfile
import unittest
from dataclasses import replace
from pathlib import Path
from typing import Any, cast
from docforge.errors import DocForgeError
from docforge.graph_projection import (
GraphViewRequestV1,
build_graph_view_plan,
)
from docforge.models import Edge, Limits, Node, ProjectDescriptor, ProjectSnapshot
from docforge.projection_contract import GraphViewPlanV1
def _document(plan: GraphViewPlanV1) -> dict[str, Any]:
return cast(dict[str, Any], plan.as_dict())
def _node(
node_id: str,
*,
title: str | None = None,
family: str = "code",
authority: str = "derived",
status: str = "active",
tags: tuple[str, ...] = (),
) -> Node:
return Node(
node_id=node_id,
title=title or node_id,
family=family,
authority=authority,
status=status,
tags=tags,
summary=f"Summary for {node_id}",
content=f"SECRET SOURCE BODY {node_id}",
source_path=f"/private/source/{node_id}.py",
source_anchor=f"line-{len(node_id)}",
content_hash=(node_id.encode("utf-8").hex() + "0" * 64)[:64],
)
def _snapshot(
root: Path,
nodes: tuple[Node, ...],
edges: tuple[Edge, ...],
) -> ProjectSnapshot:
descriptor = ProjectDescriptor(
schema_version=1,
project_id="graph-project",
title="Graph project",
adapter="generic",
root=root,
descriptor_path=root / ".docforge" / "project.toml",
descriptor_hash="d" * 64,
content_roots=(root / "docs",),
authority_files=(),
cache_root=root / ".docforge" / "cache",
index_path=root / ".docforge" / "cache" / "index.sqlite3",
changeset_root=root / ".docforge" / "changesets",
proposal_writers=(),
render=None,
allowed_relations=tuple(sorted({edge.relation for edge in edges})),
profiles=(),
limits=Limits(),
)
return ProjectSnapshot(
descriptor=descriptor,
nodes=nodes,
edges=edges,
revision="revision-1",
source_hash="a" * 64,
)
class GraphProjectionTests(unittest.TestCase):
def test_exact_root_plan_is_deterministic_sorted_and_path_free(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
nodes = (
_node("c", family="docs", tags=("python",)),
_node("a", tags=("python", "callable")),
_node("b", tags=("python",)),
_node("unrelated"),
)
edges = (
Edge("b", "calls", "c"),
Edge("a", "calls", "b"),
Edge("unrelated", "calls", "c"),
)
request = GraphViewRequestV1(
view_id="architecture",
title="Architecture",
root_node_id="a",
depth=2,
max_nodes=10,
max_edges=10,
max_work=100,
)
first = build_graph_view_plan(_snapshot(root, nodes, edges), request, True)
second = build_graph_view_plan(
_snapshot(root, tuple(reversed(nodes)), tuple(reversed(edges))),
request,
True,
)
self.assertEqual(first.as_dict(), second.as_dict())
GraphViewPlanV1.from_dict(first.as_dict())
document = _document(first)
self.assertEqual(
["a", "b", "c"], [node["node_id"] for node in document["graph"]["nodes"]]
)
self.assertEqual(
[
{"source_id": "a", "relation": "calls", "target_id": "b"},
{"source_id": "b", "relation": "calls", "target_id": "c"},
],
document["graph"]["edges"],
)
encoded = json.dumps(document, sort_keys=True)
self.assertNotIn(str(root), encoded)
self.assertNotIn("SECRET SOURCE BODY", encoded)
self.assertNotIn("/private/source/", encoded)
self.assertNotIn("line-1", encoded)
self.assertEqual("excluded", document["policy"]["source_paths"])
self.assertEqual("excluded", document["policy"]["source_bodies"])
self.assertEqual("allowed", document["policy"]["logic"])
self.assertEqual("exact_root", document["view"]["scope"]["kind"])
def test_lexical_scope_uses_metadata_only_and_closed_filters(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
nodes = (
_node(
"api.handler",
title="Request handler",
family="code",
authority="derived",
tags=("python", "route"),
),
_node(
"api.test",
title="Handler proof",
family="test",
authority="approved_plan",
tags=("python", "test"),
),
replace(
_node("hidden.body", family="code", tags=("python",)),
content="request handler appears only in the forbidden source body",
),
)
edges = (
Edge("api.handler", "tested_by", "api.test"),
Edge("hidden.body", "relates_to", "api.handler"),
)
request = GraphViewRequestV1(
view_id="routes",
title="Routes",
query="request handler",
families=("code",),
authorities=("derived",),
tags=("python", "route"),
relations=("tested_by",),
max_nodes=10,
max_edges=10,
max_work=100,
)
plan = build_graph_view_plan(_snapshot(root, nodes, edges), request, False)
document = _document(plan)
self.assertEqual(
["api.handler"],
[node["node_id"] for node in document["graph"]["nodes"]],
)
self.assertEqual([], document["graph"]["edges"])
self.assertEqual(
{
"families": ["code"],
"relations": ["tested_by"],
"authorities": ["derived"],
"statuses": [],
"tags": ["python", "route"],
},
document["view"]["filters"],
)
self.assertEqual("lexical", document["view"]["scope"]["kind"])
def test_result_and_work_limits_emit_explicit_omissions(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
nodes = tuple(_node(value) for value in ("a", "b", "c", "d"))
edges = (
Edge("a", "calls", "b"),
Edge("a", "calls", "c"),
Edge("a", "calls", "d"),
Edge("b", "calls", "c"),
)
result_limited = build_graph_view_plan(
_snapshot(root, nodes, edges),
GraphViewRequestV1(
view_id="limited",
title="Limited",
root_node_id="a",
max_nodes=2,
max_edges=0,
max_work=100,
),
False,
)
result_limited = _document(result_limited)
self.assertEqual(
["a", "b"], [node["node_id"] for node in result_limited["graph"]["nodes"]]
)
self.assertEqual([], result_limited["graph"]["edges"])
self.assertEqual(
["edge_result_limit", "node_result_limit"],
[item["code"] for item in result_limited["omissions"]],
)
work_limited = build_graph_view_plan(
_snapshot(root, nodes, edges),
GraphViewRequestV1(
view_id="work",
title="Work",
root_node_id="a",
max_nodes=10,
max_edges=10,
max_work=1,
),
False,
)
work_limited = _document(work_limited)
self.assertIn(
"work_limit",
[item["code"] for item in work_limited["omissions"]],
)
self.assertEqual(
1,
work_limited["diagnostics"]["examined_work_units"],
)
self.assertTrue(work_limited["diagnostics"]["truncated"])
def test_no_ast_policy_excludes_requested_logic(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
snapshot = _snapshot(root, (_node("a"),), ())
request = GraphViewRequestV1(
view_id="logic",
title="Logic",
root_node_id="a",
initial_mode="logic",
include_logic=True,
)
blocked = _document(build_graph_view_plan(snapshot, request, False))
self.assertEqual("forbidden", blocked["policy"]["logic"])
self.assertEqual([], blocked["graph"]["logic_projections"])
self.assertIn(
"logic_forbidden",
[item["code"] for item in blocked["omissions"]],
)
allowed = _document(build_graph_view_plan(snapshot, request, True))
self.assertEqual("allowed", allowed["policy"]["logic"])
self.assertNotIn(
"logic_forbidden",
[item["code"] for item in allowed["omissions"]],
)
def test_edge_and_node_filters_constrain_exact_root_bfs(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
nodes = (
_node("root", family="code", status="active"),
_node("code-child", family="code", status="active"),
_node("doc-child", family="docs", status="active"),
_node("old-child", family="code", status="historical"),
)
edges = (
Edge("root", "calls", "code-child"),
Edge("root", "documents", "doc-child"),
Edge("root", "calls", "old-child"),
)
plan = build_graph_view_plan(
_snapshot(root, nodes, edges),
GraphViewRequestV1(
view_id="filtered",
title="Filtered",
root_node_id="root",
families=("code",),
statuses=("active",),
relations=("calls",),
max_work=100,
),
False,
)
plan = _document(plan)
self.assertEqual(
["code-child", "root"], [node["node_id"] for node in plan["graph"]["nodes"]]
)
self.assertEqual(
[{"source_id": "root", "relation": "calls", "target_id": "code-child"}],
plan["graph"]["edges"],
)
def test_invalid_requests_and_graphs_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
snapshot = _snapshot(root, (_node("a"),), ())
invalid_requests = (
GraphViewRequestV1(view_id="v", title="V"),
GraphViewRequestV1(
view_id="v",
title="V",
root_node_id="a",
query="a",
),
GraphViewRequestV1(
view_id="v",
title="V",
root_node_id="a",
max_nodes=0,
),
GraphViewRequestV1(
view_id="v",
title="V",
query="***",
),
GraphViewRequestV1(
view_id="v",
title="V",
root_node_id="a",
families=("code", "code"),
),
)
for request in invalid_requests:
with self.subTest(request=request), self.assertRaises(DocForgeError) as error:
build_graph_view_plan(snapshot, request, False)
self.assertEqual("invalid_graph_view_request", error.exception.code)
with self.assertRaises(DocForgeError) as missing:
build_graph_view_plan(
snapshot,
GraphViewRequestV1(
view_id="v",
title="V",
root_node_id="missing",
),
False,
)
self.assertEqual("missing_node", missing.exception.code)
duplicate = _snapshot(root, (_node("a"), _node("a")), ())
with self.assertRaises(DocForgeError) as invalid:
build_graph_view_plan(
duplicate,
GraphViewRequestV1(
view_id="v",
title="V",
root_node_id="a",
),
False,
)
self.assertEqual("invalid_projection", invalid.exception.code)
def test_plan_identity_changes_with_generation_request_and_policy(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
snapshot = _snapshot(root, (_node("a"),), ())
request = GraphViewRequestV1(
view_id="v",
title="V",
root_node_id="a",
)
base = build_graph_view_plan(snapshot, request, False)
self.assertEqual(
base.plan_id,
build_graph_view_plan(snapshot, request, False).plan_id,
)
self.assertNotEqual(
base.plan_id,
build_graph_view_plan(
replace(snapshot, source_hash="b" * 64),
request,
False,
).plan_id,
)
self.assertNotEqual(
base.plan_id,
build_graph_view_plan(
snapshot,
replace(request, title="Other"),
False,
).plan_id,
)
self.assertNotEqual(
base.plan_id,
build_graph_view_plan(snapshot, request, True).plan_id,
)
if __name__ == "__main__":
unittest.main()

344
tests/test_graph_publication.py
View file

@ -1,344 +0,0 @@
from __future__ import annotations
import json
import shutil
import tempfile
import tomllib
import unittest
from pathlib import Path
from unittest import mock
from jsonschema import Draft202012Validator
from docforge.errors import DocForgeError
from docforge.graph_rendering import GraphRenderService
from docforge.models import ProjectState
from docforge.project import Project
from docforge.projection_contract import projection_hash
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
PROJECT_SCHEMA = json.loads((ROOT / "schemas/project.schema.json").read_text(encoding="utf-8"))
GRAPH_CONFIG = """
[graph_render]
output_root = ".docforge/portable-graph"
[[graph_render.views]]
id = "architecture"
renderer = "portable_graph_html"
output = "architecture.html"
title = "Alpha architecture"
root = "guide.workflow"
initial_mode = "nodes"
depth = 2
max_nodes = 20
max_edges = 40
max_work = 1000
families = ["guide", "proof"]
relations = ["depends_on", "proves"]
authorities = []
statuses = []
tags = []
include_logic = false
"""
class GraphPublicationTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name) / "alpha"
shutil.copytree(FIXTURES / "alpha", self.root)
descriptor = self.root / ".docforge/project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
encoding="utf-8",
)
self.project = Project.open(self.root)
self.service = GraphRenderService(self.project)
def test_descriptor_schema_and_runtime_accept_the_separate_graph_view(self) -> None:
document = tomllib.loads((self.root / ".docforge/project.toml").read_text(encoding="utf-8"))
Draft202012Validator(PROJECT_SCHEMA).validate(document)
config = self.project.descriptor.graph_render
assert config is not None
self.assertEqual(self.root / ".docforge/portable-graph", config.output_root)
self.assertEqual("architecture", config.views[0].view_id)
self.assertEqual("guide.workflow", config.views[0].root_node_id)
self.assertIsNone(config.views[0].query)
def test_render_publication_is_deterministic_durable_and_unchanged_on_reuse(self) -> None:
missing = self.service.status("architecture")
self.assertEqual("stale", missing["state"])
self.assertEqual("missing", missing["outputs"][0]["state"])
first = self.service.render("architecture")
output = self.root / ".docforge/portable-graph/architecture.html"
manifest = self.root / ".docforge/cache/projection-publications/graph/architecture.json"
artifact_root = self.root / ".docforge/cache/projection-artifacts"
self.assertEqual("current", first["state"])
self.assertEqual("published", first["publication"])
self.assertTrue(output.is_file())
self.assertTrue(manifest.is_file())
self.assertEqual(1, len(tuple(artifact_root.glob("*.html"))))
before = output.stat()
before_bytes = output.read_bytes()
current = self.service.status("architecture")
self.assertEqual("current", current["state"])
self.assertEqual("manifest", current["outputs"][0]["verification"])
second = self.service.render("architecture")
after = output.stat()
self.assertEqual("unchanged", second["publication"])
self.assertEqual(before_bytes, output.read_bytes())
self.assertEqual((before.st_dev, before.st_ino), (after.st_dev, after.st_ino))
def test_status_is_manifest_only_and_detects_source_output_and_manifest_changes(self) -> None:
self.service.render("architecture")
output = self.root / ".docforge/portable-graph/architecture.html"
manifest = self.root / ".docforge/cache/projection-publications/graph/architecture.json"
with mock.patch.object(
self.project,
"load",
side_effect=AssertionError("status must not load or plan"),
):
self.assertEqual("current", self.service.status("architecture")["state"])
output.write_bytes(output.read_bytes() + b"\n")
changed_output = self.service.status("architecture")
self.assertEqual("stale", changed_output["state"])
self.assertEqual("output_changed", changed_output["outputs"][0]["reason"])
self.service.render("architecture")
source = self.root / "docs/content/workflow.md"
source.write_text(
source.read_text(encoding="utf-8") + "\nChanged after publication.\n",
encoding="utf-8",
)
changed_source = self.service.status("architecture")
self.assertEqual("stale", changed_source["state"])
self.assertEqual("stale", changed_source["outputs"][0]["state"])
self.assertEqual(
"source_generation_changed",
changed_source["outputs"][0]["reason"],
)
self.service.render("architecture")
manifest.write_text("{bad-json", encoding="utf-8")
corrupt = self.service.status("architecture")
self.assertEqual("missing", corrupt["outputs"][0]["state"])
def test_status_validates_nested_manifest_and_artifact_store_evidence(self) -> None:
self.service.render("architecture")
manifest_path = (
self.root / ".docforge/cache/projection-publications/graph/architecture.json"
)
original = json.loads(manifest_path.read_text(encoding="utf-8"))
mutations = {
"bad_project": lambda value: value.__setitem__("project", "bad"),
"forged_artifact": lambda value: value.__setitem__(
"artifact",
{
"artifact_id": "portable-graph.html",
"media_type": "text/html; charset=utf-8",
"sha256": "0" * 64,
"bytes": 1,
},
),
"bad_store": lambda value: value["store"].__setitem__("size", -1),
}
for name, mutate in mutations.items():
with self.subTest(name=name):
value = json.loads(json.dumps(original))
mutate(value)
value.pop("publication_id")
value["publication_id"] = projection_hash(value)
manifest_path.write_text(
json.dumps(value, sort_keys=True, indent=2) + "\n",
encoding="utf-8",
)
status = self.service.status("architecture")
self.assertEqual("unverified", status["outputs"][0]["state"])
self.assertEqual("manifest_invalid", status["outputs"][0]["reason"])
manifest_path.write_text(
json.dumps(original, sort_keys=True, indent=2) + "\n",
encoding="utf-8",
)
artifact = next((self.root / ".docforge/cache/projection-artifacts").glob("*.html"))
artifact.unlink()
missing = self.service.status("architecture")
self.assertEqual("stale", missing["outputs"][0]["state"])
self.assertEqual("artifact_store_missing", missing["outputs"][0]["reason"])
repaired = self.service.render("architecture")
self.assertEqual("current", repaired["state"])
self.assertTrue(artifact.is_file())
def test_status_detects_source_and_publication_races(self) -> None:
self.service.render("architecture")
current = self.project.incremental_state()
assert current is not None
changed = ProjectState(source_hash="0" * 64, revision="changed")
with mock.patch.object(
self.project,
"incremental_state",
side_effect=(current, changed),
):
raced_source = self.service.status("architecture")
self.assertEqual("stale", raced_source["state"])
self.assertEqual(
"source_changed_during_status",
raced_source["outputs"][0]["reason"],
)
baseline = self.service._manifest_status(
self.project.descriptor.graph_render.views[0], # type: ignore[union-attr]
current,
)
replaced = dict(baseline)
replaced["publication_id"] = "f" * 64
with mock.patch.object(
self.service,
"_manifest_status",
side_effect=(baseline, replaced),
):
raced_publication = self.service.status("architecture")
self.assertEqual("stale", raced_publication["state"])
self.assertEqual(
"publication_changed_during_status",
raced_publication["outputs"][0]["reason"],
)
def test_manifest_failure_after_output_is_degraded_success(self) -> None:
with mock.patch.object(
self.service,
"_publish_manifest",
side_effect=DocForgeError(
"publication_failure",
"Synthetic manifest failure",
),
):
result = self.service.render("architecture")
self.assertEqual("degraded", result["state"])
self.assertEqual("published", result["publication"])
self.assertEqual("manifest", result["committed_stage"])
self.assertTrue((self.root / ".docforge/portable-graph/architecture.html").is_file())
self.assertEqual("failed", result["manifest"]["state"])
def test_post_commit_stage_failures_are_degraded_and_precommit_failures_raise(self) -> None:
committed = DocForgeError(
"publication_failure",
"Synthetic committed failure",
mutation_committed=True,
)
cases = (
("_publish_artifact", "artifact_store", "partial", False),
("_publish_output", "output", "published", None),
)
for method, stage, publication, output_exists in cases:
with self.subTest(method=method):
root = Path(self.temporary.name) / method
shutil.copytree(FIXTURES / "alpha", root)
descriptor = root / ".docforge/project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
encoding="utf-8",
)
service = GraphRenderService(Project.open(root))
with mock.patch.object(service, method, side_effect=committed):
result = service.render("architecture")
self.assertEqual("degraded", result["state"])
self.assertEqual(stage, result["committed_stage"])
self.assertEqual(publication, result["publication"])
if output_exists is not None:
self.assertEqual(
output_exists,
(root / ".docforge/portable-graph/architecture.html").exists(),
)
uncommitted = DocForgeError(
"publication_failure",
"Synthetic precommit failure",
mutation_committed=False,
)
with (
mock.patch.object(
self.service,
"_publish_output",
side_effect=uncommitted,
),
self.assertRaises(DocForgeError),
):
self.service.render("architecture")
def test_configuration_rejects_unsafe_ambiguous_and_overlapping_views(self) -> None:
cases = {
"both_scope": GRAPH_CONFIG.replace(
'root = "guide.workflow"',
'root = "guide.workflow"\nquery = "workflow"',
),
"output_overlap": GRAPH_CONFIG.replace(
'output_root = ".docforge/portable-graph"',
'output_root = "docs/content"',
),
"active_renderer": GRAPH_CONFIG.replace(
'renderer = "portable_graph_html"',
'renderer = "shell"',
),
"oversized": GRAPH_CONFIG.replace("max_nodes = 20", "max_nodes = 100000"),
"logic_mode": GRAPH_CONFIG.replace('initial_mode = "nodes"', 'initial_mode = "logic"'),
"logic_projection": GRAPH_CONFIG.replace(
"include_logic = false",
"include_logic = true",
),
"long_title": GRAPH_CONFIG.replace(
'title = "Alpha architecture"',
f'title = "{"x" * 1025}"',
),
"long_query": GRAPH_CONFIG.replace(
'root = "guide.workflow"',
f'query = "{"x" * 10001}"',
),
"too_many_filters": GRAPH_CONFIG.replace(
'families = ["guide", "proof"]',
"families = [" + ", ".join(f'"family-{index}"' for index in range(65)) + "]",
),
}
for name, graph_config in cases.items():
with self.subTest(name=name):
root = Path(self.temporary.name) / name
shutil.copytree(FIXTURES / "alpha", root)
descriptor = root / ".docforge/project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8") + graph_config,
encoding="utf-8",
)
parsed = tomllib.loads(descriptor.read_text(encoding="utf-8"))
if name != "output_overlap":
self.assertFalse(Draft202012Validator(PROJECT_SCHEMA).is_valid(parsed))
with self.assertRaises(DocForgeError):
Project.open(root)
relocated = Path(self.temporary.name) / "descriptor-overlap"
shutil.copytree(FIXTURES / "alpha", relocated)
descriptor = relocated / ".docforge/project.toml"
base = descriptor.read_text(encoding="utf-8").replace(
'cache_root = ".docforge/cache"\nindex = ".docforge/cache/index.sqlite3"',
'cache_root = "var/cache"\nindex = "var/cache/index.sqlite3"',
)
descriptor.write_text(
base
+ GRAPH_CONFIG.replace(
'output_root = ".docforge/portable-graph"',
'output_root = ".docforge"',
),
encoding="utf-8",
)
with self.assertRaises(DocForgeError):
Project.open(relocated)
if __name__ == "__main__":
unittest.main()

284
tests/test_graph_rendering.py
View file

@ -1,284 +0,0 @@
from __future__ import annotations
import copy
import hashlib
import json
import os
import shutil
import sqlite3
import tempfile
import unittest
from pathlib import Path
from typing import cast
from unittest import mock
from docforge.errors import DocForgeError
from docforge.graph_projection import (
GraphViewRequestV1,
build_graph_projection_package,
build_graph_view_plan,
)
from docforge.project import Project
from docforge.projection_contract import ProjectionPackageV1
from docforge_renderers.graph import PortableGraphHtmlRenderer
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
class PortableGraphRenderingTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name) / "alpha"
shutil.copytree(FIXTURES / "alpha", self.root)
self.project = Project.open(self.root)
self.snapshot = self.project.load()
self.request = GraphViewRequestV1(
view_id="architecture",
title="Alpha architecture",
root_node_id="guide.workflow",
depth=2,
max_nodes=20,
max_edges=40,
max_work=1_000,
)
self.plan = build_graph_view_plan(self.snapshot, self.request, False)
self.package = build_graph_projection_package(
self.plan,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=1_000_000,
)
def test_portable_artifact_is_deterministic_self_contained_and_generation_bound(self) -> None:
renderer = PortableGraphHtmlRenderer()
first = renderer.render(self.package)
second = renderer.render(self.package)
self.assertEqual(first.artifacts, second.artifacts)
self.assertEqual(1, len(first.artifacts))
artifact = first.artifacts[0]
self.assertEqual("portable-graph.html", artifact.artifact_id)
self.assertEqual("text/html; charset=utf-8", artifact.media_type)
receipt = first.receipt.as_dict()
evidence = cast(list[dict[str, object]], receipt["artifacts"])
self.assertEqual(
hashlib.sha256(artifact.content).hexdigest(),
evidence[0]["sha256"],
)
rendered = artifact.content.decode("utf-8")
self.assertIn("Content-Security-Policy", rendered)
self.assertIn("default-src &#x27;none&#x27;", rendered)
self.assertIn('type="application/json"', rendered)
self.assertNotIn(str(self.root), rendered)
self.assertNotIn("docs/content/", rendered)
self.assertNotIn("Editors change canonical nodes", rendered)
self.assertNotIn("fetch(", rendered)
self.assertNotIn("XMLHttpRequest", rendered)
self.assertNotIn("WebSocket", rendered)
self.assertIn(self.snapshot.source_hash, rendered)
def test_embedded_plan_is_exact_and_script_breakout_is_inert(self) -> None:
plan = copy.deepcopy(self.plan.as_dict())
view = plan["view"]
assert isinstance(view, dict)
view["title"] = '</script><script id="injected">alert(1)</script>'
plan.pop("plan_id")
from docforge.projection_contract import GraphViewPlanV1
malicious = GraphViewPlanV1.create(plan)
package = build_graph_projection_package(
malicious,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=1_000_000,
)
rendered = PortableGraphHtmlRenderer().render(package).artifacts[0].content.decode("utf-8")
self.assertNotIn('<script id="injected">', rendered)
self.assertIn("&lt;/script&gt;", rendered)
prefix = '<script id="docforge-graph-plan" type="application/json">'
embedded = rendered.split(prefix, 1)[1].split("</script>", 1)[0]
self.assertEqual(malicious.as_dict(), json.loads(embedded))
def test_renderer_has_no_project_database_or_filesystem_write_capability(self) -> None:
forbidden = AssertionError("portable graph renderer crossed its capability boundary")
with (
mock.patch.object(Project, "open", side_effect=forbidden),
mock.patch.object(Project, "load", side_effect=forbidden),
mock.patch.object(sqlite3, "connect", side_effect=forbidden),
mock.patch.object(Path, "write_bytes", side_effect=forbidden),
mock.patch.object(Path, "write_text", side_effect=forbidden),
mock.patch.object(Path, "mkdir", side_effect=forbidden),
mock.patch.object(os, "replace", side_effect=forbidden),
mock.patch.object(os, "rename", side_effect=forbidden),
mock.patch.object(os, "unlink", side_effect=forbidden),
):
result = PortableGraphHtmlRenderer().render(self.package)
self.assertEqual(1, len(result.artifacts))
def test_renderer_rejects_wrong_kind_identity_assets_and_size(self) -> None:
wrong = ProjectionPackageV1.create(
kind="graph",
plan=self.plan,
renderer={
"renderer_id": PortableGraphHtmlRenderer.renderer_id,
"renderer_version": "other",
},
components=[],
assets=[],
output_policy={
"artifact_ids": ["portable-graph.html"],
"max_total_bytes": 1_000_000,
},
)
with self.assertRaises(DocForgeError) as unsupported:
PortableGraphHtmlRenderer().render(wrong)
self.assertEqual("unsupported_renderer", unsupported.exception.code)
with_asset = ProjectionPackageV1.create(
kind="graph",
plan=self.plan,
renderer={
"renderer_id": PortableGraphHtmlRenderer.renderer_id,
"renderer_version": PortableGraphHtmlRenderer.renderer_version,
},
components=[],
assets=[
{
"asset_id": "project-script",
"media_type": "text/javascript",
"sha256": "0" * 64,
"text": "alert(1)",
}
],
output_policy={"artifact_ids": ["portable-graph.html"], "max_total_bytes": 1_000_000},
)
with self.assertRaises(DocForgeError) as assets:
PortableGraphHtmlRenderer().render(with_asset)
self.assertEqual("invalid_projection", assets.exception.code)
wrong_components = ProjectionPackageV1.create(
kind="graph",
plan=self.plan,
renderer={
"renderer_id": PortableGraphHtmlRenderer.renderer_id,
"renderer_version": PortableGraphHtmlRenderer.renderer_version,
},
components=[],
assets=[],
output_policy={
"artifact_ids": ["portable-graph.html"],
"max_total_bytes": 1_000_000,
},
)
with self.assertRaises(DocForgeError) as components:
PortableGraphHtmlRenderer().render(wrong_components)
self.assertEqual("invalid_projection", components.exception.code)
wrong_artifact = ProjectionPackageV1.create(
kind="graph",
plan=self.plan,
renderer={
"renderer_id": PortableGraphHtmlRenderer.renderer_id,
"renderer_version": PortableGraphHtmlRenderer.renderer_version,
},
components=self.package.document["components"], # type: ignore[arg-type]
assets=[],
output_policy={
"artifact_ids": ["unexpected.html"],
"max_total_bytes": 1_000_000,
},
)
with self.assertRaises(DocForgeError) as artifact:
PortableGraphHtmlRenderer().render(wrong_artifact)
self.assertEqual("invalid_projection", artifact.exception.code)
tiny = build_graph_projection_package(
self.plan,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=1,
)
with self.assertRaises(DocForgeError) as too_large:
PortableGraphHtmlRenderer().render(tiny)
self.assertEqual("render_too_large", too_large.exception.code)
def test_initial_mode_is_honored_and_logic_is_rejected(self) -> None:
for mode in ("nodes", "flow", "web"):
with self.subTest(mode=mode):
plan = build_graph_view_plan(
self.snapshot,
GraphViewRequestV1(
view_id=f"{mode}-view",
title=f"{mode.title()} view",
root_node_id="guide.workflow",
initial_mode=mode, # type: ignore[arg-type]
depth=2,
max_nodes=20,
max_edges=40,
max_work=1_000,
),
False,
)
package = build_graph_projection_package(
plan,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=1_000_000,
)
rendered = (
PortableGraphHtmlRenderer().render(package).artifacts[0].content.decode("utf-8")
)
self.assertIn(f'data-mode="{mode}"', rendered)
self.assertIn(f'<option value="{mode}" selected>', rendered)
logic_plan = build_graph_view_plan(
self.snapshot,
GraphViewRequestV1(
view_id="logic-view",
title="Logic view",
root_node_id="guide.workflow",
initial_mode="logic",
include_logic=True,
),
True,
)
logic_package = build_graph_projection_package(
logic_plan,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=1_000_000,
)
with self.assertRaises(DocForgeError) as logic:
PortableGraphHtmlRenderer().render(logic_package)
self.assertEqual("unsupported_renderer", logic.exception.code)
def test_artifact_exposes_accessible_static_and_interactive_structure(self) -> None:
rendered = (
PortableGraphHtmlRenderer().render(self.package).artifacts[0].content.decode("utf-8")
)
static = rendered.split('<script id="docforge-graph-plan"', 1)[0]
self.assertIn("Editing workflow (guide.workflow)", static)
self.assertIn("<td>guide.workflow</td>", static)
self.assertIn('data-panel="nodes"', static)
self.assertIn('data-panel="relationships"', static)
for expected in (
'href="#main"',
'<main id="main" tabindex="-1" data-mode="nodes">',
'role="group" aria-label="Graph controls"',
'role="status" aria-live="polite"',
'<th scope="col">Source</th>',
'<th scope="col">Relation</th>',
'<th scope="col">Target</th>',
'<dialog id="node-dialog"',
'aria-labelledby="node-dialog-title"',
'dialog.addEventListener("close", () => opener?.focus())',
"@media (prefers-reduced-motion: reduce)",
"@media (forced-colors: active)",
):
with self.subTest(expected=expected):
self.assertIn(expected, rendered)
if __name__ == "__main__":
unittest.main()

23
tests/test_onboarding.py
View file

@ -3,7 +3,6 @@ from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from docforge.cli import _parser, _run
from docforge.errors import DocForgeError
@ -11,28 +10,6 @@ from docforge.project import Project
class DocForgeOnboardingTests(unittest.TestCase):
def test_scaffold_honors_disabled_manual_projection_policy(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory) / "disabled-render"
root.mkdir()
arguments = _parser().parse_args(
[
"--project-root",
str(root),
"--manual-render-policy",
"disabled",
"onboard",
"--scaffold",
]
)
with mock.patch(
"docforge.cli.RenderService.render",
side_effect=AssertionError("disabled onboarding must not render"),
):
result = _run(arguments)
self.assertEqual("skipped", result["render"]["state"])
self.assertFalse((root / ".docforge/rendered/manual.html").exists())
def test_assessment_detects_multiple_languages_without_writing(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory) / "polyglot"

669
tests/test_projection_contract.py
View file

@ -1,669 +0,0 @@
from __future__ import annotations
import copy
import hashlib
import json
import os
import shutil
import sqlite3
import tempfile
import unittest
from dataclasses import replace
from pathlib import Path
from unittest import mock
from docforge.errors import DocForgeError
from docforge.manual_projection import (
_cycles,
build_manual_projection_package,
build_manual_render_plan,
)
from docforge.models import Edge
from docforge.project import Project
from docforge.projection_contract import (
MANUAL_RENDER_PLAN_CONTRACT,
PROJECTION_PACKAGE_CONTRACT,
PROJECTION_RECEIPT_CONTRACT,
ManualRenderPlanV1,
ProjectionArtifact,
ProjectionPackageV1,
ProjectionReceiptV1,
canonical_projection_bytes,
projection_hash,
)
from docforge.projection_fragments import FRAGMENT_CACHE_DIRECTORY, FragmentRecord
from docforge.projection_worker import render_projection_in_worker
from docforge.render_contract import GenericHtmlRenderer
from docforge_renderers.manual import ManualHtmlRenderer
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
ALPHA_RENDERER_VERSION = "1+markdown-it-py-4.2.0"
ALPHA_RENDER_IDENTITY = "1c0a49c28ba3b0dabf94be36e75def197dee1be3cb73ac405b09875383c8dc5f"
ALPHA_OUTPUT_HASH = "81656bb89debc7ad1fbe8bc290e9a3ba90664442b17a6d57e908d30d20c47f77"
ALPHA_OUTPUT_BYTES = 2043
class ProjectionContractTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name) / "alpha"
shutil.copytree(FIXTURES / "alpha", self.root)
self.project = Project.open(self.root)
self.snapshot = self.project.load()
assert self.snapshot.descriptor.render is not None
self.view = self.snapshot.descriptor.render.views[0]
self.template = self.view.template_path.read_bytes()
self.plan = build_manual_render_plan(
self.snapshot,
self.view,
changeset_hash=None,
)
self.package = build_manual_projection_package(
self.plan,
self.template,
renderer_id="generic_html",
renderer_version=ALPHA_RENDERER_VERSION,
max_output_bytes=self.snapshot.descriptor.limits.max_render_bytes,
)
def test_canonical_identity_is_stable_and_tampering_is_rejected(self) -> None:
self.assertEqual(
b'{"a":"\xc3\xa9","b":1}',
canonical_projection_bytes({"b": 1, "a": "\N{LATIN SMALL LETTER E WITH ACUTE}"}),
)
self.assertEqual(
projection_hash(
{key: value for key, value in self.plan.as_dict().items() if key != "plan_id"}
),
self.plan.plan_id,
)
self.assertEqual(
self.plan.plan_id,
build_manual_render_plan(
self.snapshot,
self.view,
changeset_hash=None,
).plan_id,
)
self.assertEqual(
self.package.package_id,
ProjectionPackageV1.from_dict(
json.loads(json.dumps(self.package.as_dict()))
).package_id,
)
tampered_plan = copy.deepcopy(self.plan.as_dict())
pages = tampered_plan["pages"]
assert isinstance(pages, list)
assert isinstance(pages[0], dict)
pages[0]["title"] = "Tampered title"
with self.assertRaises(DocForgeError) as plan_error:
ManualRenderPlanV1.from_dict(tampered_plan)
self.assertEqual("invalid_projection", plan_error.exception.code)
tampered_package = copy.deepcopy(self.package.as_dict())
assets = tampered_package["assets"]
assert isinstance(assets, list)
assert isinstance(assets[0], dict)
assets[0]["text"] = f"{assets[0]['text']}\nTampered"
with self.assertRaises(DocForgeError) as package_error:
ProjectionPackageV1.from_dict(tampered_package)
self.assertEqual("invalid_projection", package_error.exception.code)
def test_contract_documents_reject_unknown_or_malformed_fields(self) -> None:
plan_with_extra = copy.deepcopy(self.plan.as_dict())
plan_with_extra["unexpected"] = True
with self.assertRaises(DocForgeError) as extra_plan:
ManualRenderPlanV1.from_dict(plan_with_extra)
self.assertEqual("invalid_projection", extra_plan.exception.code)
plan_with_foreign_project = copy.deepcopy(self.plan.as_dict())
project = plan_with_foreign_project["project"]
assert isinstance(project, dict)
project["absolute_root"] = str(self.root)
with self.assertRaises(DocForgeError) as foreign_project:
ManualRenderPlanV1.from_dict(plan_with_foreign_project)
self.assertEqual("invalid_projection", foreign_project.exception.code)
package_with_extra = copy.deepcopy(self.package.as_dict())
package_with_extra["unexpected"] = []
with self.assertRaises(DocForgeError) as extra_package:
ProjectionPackageV1.from_dict(package_with_extra)
self.assertEqual("invalid_projection", extra_package.exception.code)
result = ManualHtmlRenderer(ALPHA_RENDERER_VERSION).render(self.package)
receipt_with_extra = copy.deepcopy(result.receipt.as_dict())
receipt_with_extra["artifact_bytes"] = "forbidden"
with self.assertRaises(DocForgeError) as extra_receipt:
ProjectionReceiptV1.from_dict(receipt_with_extra)
self.assertEqual("invalid_projection", extra_receipt.exception.code)
with self.assertRaises(DocForgeError) as path_artifact:
ProjectionReceiptV1.create(
kind="manual",
package_id=self.package.package_id,
plan_id=self.plan.plan_id,
renderer={
"renderer_id": "generic_html",
"renderer_version": ALPHA_RENDERER_VERSION,
},
artifacts=[
{
"artifact_id": "../manual.html",
"media_type": "text/html",
"sha256": "0" * 64,
"bytes": 1,
}
],
diagnostics={},
timing={"elapsed_ns": 0},
peak_memory_bytes=None,
)
self.assertEqual("invalid_projection", path_artifact.exception.code)
def test_projection_package_is_path_free_and_rejects_runtime_references(self) -> None:
serialized = canonical_projection_bytes(self.package.as_dict())
self.assertNotIn(str(self.root).encode("utf-8"), serialized)
self.assertNotIn(b"source_path", serialized)
self.assertNotIn(b"sqlite", serialized.lower())
with self.assertRaises(DocForgeError) as absolute_path:
ProjectionPackageV1.create(
kind="manual",
plan=self.plan,
renderer={
"renderer_id": "generic_html",
"renderer_version": ALPHA_RENDERER_VERSION,
},
components=[],
assets=[],
output_policy={
"artifact_ids": ["manual.html"],
"max_total_bytes": 1000,
"template_path": "/home/example/private-template.html",
},
)
self.assertEqual("invalid_projection", absolute_path.exception.code)
with self.assertRaises(DocForgeError) as database_reference:
ProjectionPackageV1.create(
kind="manual",
plan=self.plan,
renderer={
"renderer_id": "generic_html",
"renderer_version": ALPHA_RENDERER_VERSION,
},
components=[],
assets=[],
output_policy={
"artifact_ids": ["manual.html"],
"max_total_bytes": 1000,
"database": "index.sqlite3",
},
)
self.assertEqual("invalid_projection", database_reference.exception.code)
def test_receipt_attests_artifacts_without_embedding_content(self) -> None:
result = ManualHtmlRenderer(ALPHA_RENDERER_VERSION).render(
self.package,
render_identity=ALPHA_RENDER_IDENTITY,
)
self.assertEqual(1, len(result.artifacts))
artifact = result.artifacts[0]
evidence = artifact.evidence()
receipt = result.receipt.as_dict()
self.assertEqual(self.package.package_id, receipt["package_id"])
self.assertEqual(self.plan.plan_id, receipt["plan_id"])
self.assertEqual([evidence], receipt["artifacts"])
self.assertEqual(PROJECTION_RECEIPT_CONTRACT, receipt["contract"])
self.assertEqual(
{
"renderer_id": "generic_html",
"renderer_version": ALPHA_RENDERER_VERSION,
},
receipt["renderer"],
)
self.assertEqual({"warnings": []}, receipt["diagnostics"])
self.assertIsNone(receipt["peak_memory_bytes"])
timing = receipt["timing"]
assert isinstance(timing, dict)
self.assertGreaterEqual(timing["elapsed_ns"], 0)
self.assertNotIn("content", evidence)
self.assertNotIn(artifact.content, canonical_projection_bytes(receipt))
self.assertEqual(
receipt["receipt_id"],
ProjectionReceiptV1.from_dict(copy.deepcopy(receipt)).receipt_id,
)
tampered_receipt = copy.deepcopy(receipt)
artifacts = tampered_receipt["artifacts"]
assert isinstance(artifacts, list)
assert isinstance(artifacts[0], dict)
artifacts[0]["bytes"] = int(artifacts[0]["bytes"]) + 1
with self.assertRaises(DocForgeError) as tampered:
ProjectionReceiptV1.from_dict(tampered_receipt)
self.assertEqual("invalid_projection", tampered.exception.code)
def test_manual_plan_is_deterministic_and_preserves_alpha_semantics(self) -> None:
document = self.plan.as_dict()
self.assertEqual(MANUAL_RENDER_PLAN_CONTRACT, document["contract"])
self.assertIsNone(document["changeset_hash"])
pages = document["pages"]
navigation = document["navigation"]
search_documents = document["search_documents"]
diagnostics = document["diagnostics"]
assert isinstance(pages, list)
assert isinstance(navigation, list)
assert isinstance(search_documents, list)
assert isinstance(diagnostics, dict)
self.assertEqual(
["guide.foundation", "guide.workflow", "proof.validation"],
[page["node_id"] for page in pages],
)
self.assertEqual(
["guide.foundation", "guide.workflow", "proof.validation"],
[item["node_id"] for item in navigation],
)
self.assertEqual(
["guide.foundation", "guide.workflow", "proof.validation"],
[item["node_id"] for item in search_documents],
)
self.assertEqual([], diagnostics["orphans"])
self.assertEqual([], diagnostics["cycles"])
page_by_id = {page["node_id"]: page for page in pages}
self.assertEqual(
[
{
"source_id": "guide.workflow",
"relation": "depends_on",
"target_id": "guide.foundation",
}
],
page_by_id["guide.foundation"]["backlinks"],
)
self.assertEqual(
[
{
"source_id": "guide.workflow",
"relation": "depends_on",
"target_id": "guide.foundation",
}
],
page_by_id["guide.workflow"]["cross_references"],
)
self.assertEqual(
[
{
"source_id": "proof.validation",
"relation": "proves",
"target_id": "guide.workflow",
}
],
page_by_id["guide.workflow"]["backlinks"],
)
self.assertEqual(
[
{
"source_id": "proof.validation",
"relation": "proves",
"target_id": "guide.workflow",
}
],
page_by_id["proof.validation"]["cross_references"],
)
self.assertTrue(
all(
page["components"]
== [
"manual.node-metadata@1",
"manual.summary@1",
"manual.commonmark@1",
"manual.relationships@1",
]
for page in pages
)
)
proposed = build_manual_render_plan(
self.snapshot,
self.view,
changeset_hash="a" * 64,
)
self.assertNotEqual(self.plan.plan_id, proposed.plan_id)
self.assertEqual("a" * 64, proposed.as_dict()["changeset_hash"])
def test_cycle_orphan_backlink_and_cross_reference_planning(self) -> None:
edges = (
Edge("guide.foundation", "relates_to", "guide.workflow"),
Edge("guide.workflow", "returns_to", "guide.foundation"),
)
snapshot = replace(self.snapshot, edges=edges)
first = build_manual_render_plan(snapshot, self.view, changeset_hash=None)
second = build_manual_render_plan(snapshot, self.view, changeset_hash=None)
self.assertEqual(first.plan_id, second.plan_id)
document = first.as_dict()
diagnostics = document["diagnostics"]
pages = document["pages"]
assert isinstance(diagnostics, dict)
assert isinstance(pages, list)
self.assertEqual(["proof.validation"], diagnostics["orphans"])
self.assertEqual(
[["guide.foundation", "guide.workflow"]],
diagnostics["cycles"],
)
page_by_id = {page["node_id"]: page for page in pages}
foundation = page_by_id["guide.foundation"]
workflow = page_by_id["guide.workflow"]
self.assertEqual(
[
{
"source_id": "guide.foundation",
"relation": "relates_to",
"target_id": "guide.workflow",
}
],
foundation["cross_references"],
)
self.assertEqual(
[
{
"source_id": "guide.workflow",
"relation": "returns_to",
"target_id": "guide.foundation",
}
],
foundation["backlinks"],
)
self.assertEqual(
foundation["cross_references"],
workflow["backlinks"],
)
self.assertEqual(
foundation["backlinks"],
workflow["cross_references"],
)
def test_cycle_detection_handles_the_maximum_deep_graph_iteratively(self) -> None:
node_ids = tuple(f"node.{index:05d}" for index in range(10_000))
chain = tuple(
Edge(node_ids[index], "depends_on", node_ids[index + 1])
for index in range(len(node_ids) - 1)
)
self.assertEqual([], _cycles(node_ids, chain))
self.assertEqual(
[list(node_ids)],
_cycles(
node_ids,
(*chain, Edge(node_ids[-1], "depends_on", node_ids[0])),
),
)
def test_alpha_compatibility_shim_preserves_legacy_identity_and_bytes(self) -> None:
renderer = GenericHtmlRenderer()
self.assertEqual(ALPHA_RENDERER_VERSION, renderer.renderer_version)
prepared = renderer.prepare(
self.snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertEqual(ALPHA_RENDER_IDENTITY, prepared.render_identity)
self.assertEqual(ALPHA_OUTPUT_HASH, prepared.output_hash)
self.assertEqual(ALPHA_OUTPUT_BYTES, len(prepared.output))
self.assertEqual(
ALPHA_OUTPUT_HASH,
hashlib.sha256(prepared.output).hexdigest(),
)
self.assertEqual(b"<!DOCTYPE html>", prepared.output.splitlines()[0])
self.assertTrue(prepared.output.endswith(b"</html>\n"))
self.assertIn(
f'content="{ALPHA_RENDER_IDENTITY}"'.encode(),
prepared.output,
)
def test_manual_fragments_are_incremental_and_full_worker_output_is_the_oracle(
self,
) -> None:
renderer = GenericHtmlRenderer()
with mock.patch(
"docforge.render_contract.render_projection_in_worker",
wraps=render_projection_in_worker,
) as detached:
cold = renderer.prepare(
self.snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertEqual(2, detached.call_count)
fragment_root = self.snapshot.descriptor.cache_root / FRAGMENT_CACHE_DIRECTORY
self.assertEqual(
len(self.plan.document["pages"]),
len(list(fragment_root.glob("*.json"))),
)
with mock.patch(
"docforge.render_contract.render_projection_in_worker",
wraps=render_projection_in_worker,
) as detached:
warm = renderer.prepare(
self.snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertEqual(1, detached.call_count)
self.assertEqual(cold.output, warm.output)
with mock.patch(
"docforge.render_contract.render_projection_in_worker",
wraps=render_projection_in_worker,
) as detached:
full = GenericHtmlRenderer(incremental=False).prepare(
self.snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertEqual(1, detached.call_count)
self.assertEqual(full.output, warm.output)
first_fragment = sorted(fragment_root.glob("*.json"))[0]
first_fragment.write_bytes(b"{corrupt")
with mock.patch(
"docforge.render_contract.render_projection_in_worker",
wraps=render_projection_in_worker,
) as detached:
recovered = renderer.prepare(
self.snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertEqual(2, detached.call_count)
self.assertEqual(full.output, recovered.output)
original_record = FragmentRecord.from_bytes(first_fragment.read_bytes())
first_fragment.write_bytes(
FragmentRecord.create(
original_record.key,
b"<script>forged fragment</script>",
).to_bytes()
)
with mock.patch(
"docforge.render_contract.render_projection_in_worker",
wraps=render_projection_in_worker,
) as detached:
forged = renderer.prepare(
self.snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertEqual(2, detached.call_count)
self.assertEqual(full.output, forged.output)
self.assertNotIn(b"forged fragment", forged.output)
def test_oversized_manual_fragment_bypasses_cache_and_uses_full_worker(self) -> None:
content = "x" * 4_100_000
node = replace(
self.snapshot.nodes[0],
content=content,
content_hash=hashlib.sha256(content.encode()).hexdigest(),
)
snapshot = replace(
self.snapshot,
descriptor=replace(
self.snapshot.descriptor,
limits=replace(
self.snapshot.descriptor.limits,
max_render_bytes=20_000_000,
),
),
nodes=(node,),
edges=(),
source_hash=hashlib.sha256(b"oversized-fragment").hexdigest(),
)
with mock.patch(
"docforge.render_contract.render_projection_in_worker",
wraps=render_projection_in_worker,
) as detached:
prepared = GenericHtmlRenderer().prepare(
snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertEqual(1, detached.call_count)
full = GenericHtmlRenderer(incremental=False).prepare(
snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertEqual(full.output, prepared.output)
def test_fragment_package_overflow_falls_back_to_full_worker(self) -> None:
content = "x" * 2_700_000
base = self.snapshot.nodes[0]
nodes = tuple(
replace(
base,
node_id=f"guide.large-{index}",
title=f"Large {index}",
content=content,
source_path=f"docs/content/large-{index}.md",
content_hash=hashlib.sha256(f"{index}:{content}".encode()).hexdigest(),
)
for index in range(4)
)
descriptor = replace(
self.snapshot.descriptor,
limits=replace(
self.snapshot.descriptor.limits,
max_render_bytes=20_000_000,
),
)
snapshot = replace(
self.snapshot,
descriptor=descriptor,
nodes=nodes,
edges=(),
source_hash=hashlib.sha256(b"large-fragment-fixture").hexdigest(),
)
incremental = GenericHtmlRenderer().prepare(
snapshot,
self.view,
self.template,
changeset_hash=None,
)
full = GenericHtmlRenderer(incremental=False).prepare(
snapshot,
self.view,
self.template,
changeset_hash=None,
)
self.assertGreater(len(full.output), 10_000_000)
self.assertEqual(full.output, incremental.output)
def test_manual_renderer_rejects_project_provided_active_content(self) -> None:
for active in (
"<script>alert(1)</script>{{ docforge_content }}",
'<main onclick="alert(1)">{{ docforge_content }}</main>',
'<a href="javascript:alert(1)">{{ docforge_content }}</a>',
'<iframe src="https://example.invalid"></iframe>{{ docforge_content }}',
'<meta http-equiv="refresh" content="0">{{ docforge_content }}',
):
with self.subTest(active=active):
package = build_manual_projection_package(
self.plan,
active.encode("utf-8"),
renderer_id="generic_html",
renderer_version=ALPHA_RENDERER_VERSION,
max_output_bytes=self.snapshot.descriptor.limits.max_render_bytes,
)
with self.assertRaises(DocForgeError) as rejected:
ManualHtmlRenderer(ALPHA_RENDERER_VERSION).render(package)
self.assertEqual("invalid_template", rejected.exception.code)
def test_manual_renderer_has_no_project_sqlite_or_path_write_capability(self) -> None:
renderer = ManualHtmlRenderer(ALPHA_RENDERER_VERSION)
forbidden = AssertionError("manual renderer crossed its capability boundary")
with (
mock.patch.object(Project, "open", side_effect=forbidden),
mock.patch.object(Project, "load", side_effect=forbidden),
mock.patch.object(sqlite3, "connect", side_effect=forbidden),
mock.patch.object(Path, "write_bytes", side_effect=forbidden),
mock.patch.object(Path, "write_text", side_effect=forbidden),
mock.patch.object(Path, "mkdir", side_effect=forbidden),
mock.patch.object(Path, "touch", side_effect=forbidden),
mock.patch.object(Path, "unlink", side_effect=forbidden),
mock.patch.object(Path, "rename", side_effect=forbidden),
mock.patch.object(Path, "replace", side_effect=forbidden),
mock.patch.object(os, "mkdir", side_effect=forbidden),
mock.patch.object(os, "makedirs", side_effect=forbidden),
mock.patch.object(os, "rename", side_effect=forbidden),
mock.patch.object(os, "replace", side_effect=forbidden),
mock.patch.object(os, "unlink", side_effect=forbidden),
):
result = renderer.render(
self.package,
render_identity=ALPHA_RENDER_IDENTITY,
)
self.assertEqual(1, len(result.artifacts))
self.assertEqual("manual.html", result.artifacts[0].artifact_id)
self.assertEqual(ALPHA_OUTPUT_HASH, result.artifacts[0].evidence()["sha256"])
def test_projection_artifact_evidence_is_canonical_and_content_free(self) -> None:
artifact = ProjectionArtifact(
artifact_id="manual.html",
media_type="text/html; charset=utf-8",
content=b"manual bytes",
)
self.assertEqual(
{
"artifact_id": "manual.html",
"media_type": "text/html; charset=utf-8",
"sha256": hashlib.sha256(b"manual bytes").hexdigest(),
"bytes": len(b"manual bytes"),
},
artifact.evidence(),
)
self.assertEqual(
PROJECTION_PACKAGE_CONTRACT,
self.package.as_dict()["contract"],
)
if __name__ == "__main__":
unittest.main()

205
tests/test_projection_fragments.py
View file

@ -1,205 +0,0 @@
from __future__ import annotations
import json
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from docforge.projection_fragments import (
FRAGMENT_CACHE_DIRECTORY,
FRAGMENT_KEY_CONTRACT,
FRAGMENT_RECORD_CONTRACT,
FRAGMENT_SCHEMA_VERSION,
FragmentKey,
FragmentRecord,
ProjectionFragmentCache,
fragment_semantic_hash,
)
class ProjectionFragmentCacheTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name) / "project"
self.root.mkdir()
self.cache_root = self.root / ".docforge" / "cache"
self.cache = ProjectionFragmentCache(
self.root,
self.cache_root,
maximum_content_bytes=1_000,
)
self.key = self.make_key({"node": "guide.alpha", "summary": "Alpha"})
@staticmethod
def make_key(
semantics: object,
*,
renderer_version: str = "1",
component_version: str = "graph.node@1",
) -> FragmentKey:
return FragmentKey.create(
projection_kind="graph",
renderer_id="portable_graph_html",
renderer_version=renderer_version,
component_version=component_version,
semantic_input_hash=fragment_semantic_hash(semantics),
)
@property
def entry_path(self) -> Path:
return self.cache_root / FRAGMENT_CACHE_DIRECTORY / f"{self.key.key_id}.json"
def test_miss_hit_and_unchanged_reuse_are_exact(self) -> None:
self.assertIsNone(self.cache.get(self.key))
with mock.patch(
"docforge.projection_fragments.atomic_replace_bytes_at",
wraps=__import__(
"docforge.projection_fragments",
fromlist=["atomic_replace_bytes_at"],
).atomic_replace_bytes_at,
) as atomic:
stored = self.cache.put(self.key, b"<li>Alpha</li>")
self.assertIsNotNone(stored)
self.assertEqual(1, atomic.call_count)
before = self.entry_path.stat()
repeated = self.cache.put(self.key, b"<li>Alpha</li>")
self.assertEqual(1, atomic.call_count)
self.assertEqual(stored, repeated)
self.assertEqual(stored, self.cache.get(self.key))
after = self.entry_path.stat()
self.assertEqual((before.st_dev, before.st_ino), (after.st_dev, after.st_ino))
def test_key_versions_and_complete_semantics_invalidate_independently(self) -> None:
self.assertIsNotNone(self.cache.put(self.key, b"alpha"))
variants = (
self.make_key(
{"node": "guide.alpha", "summary": "Alpha"},
renderer_version="2",
),
self.make_key(
{"node": "guide.alpha", "summary": "Alpha"},
component_version="graph.node@2",
),
self.make_key({"node": "guide.alpha", "summary": "Changed"}),
)
for variant in variants:
with self.subTest(key=variant):
self.assertNotEqual(self.key.key_id, variant.key_id)
self.assertIsNone(self.cache.get(variant))
def test_add_delete_and_reorder_change_sequence_semantics(self) -> None:
base = ["a", "b"]
added = ["a", "b", "c"]
deleted = ["a"]
reordered = ["b", "a"]
keys = [self.make_key(value) for value in (base, added, deleted, reordered)]
self.assertEqual(4, len({key.key_id for key in keys}))
self.assertEqual(
fragment_semantic_hash({"a": 1, "b": 2}),
fragment_semantic_hash({"b": 2, "a": 1}),
)
def test_corrupt_oversized_incompatible_and_foreign_entries_are_misses(self) -> None:
self.assertIsNotNone(self.cache.put(self.key, b"alpha"))
self.entry_path.write_bytes(b"{bad-json")
self.assertIsNone(self.cache.get(self.key))
self.assertIsNotNone(self.cache.put(self.key, b"repaired"))
repaired = self.cache.get(self.key)
self.assertIsNotNone(repaired)
assert repaired is not None
self.assertEqual(b"repaired", repaired.content)
self.entry_path.write_bytes(b"x" * (self.cache.maximum_record_bytes + 1))
self.assertIsNone(self.cache.get(self.key))
incompatible = FragmentRecord.create(self.key, b"alpha").as_dict()
incompatible["schema_version"] = FRAGMENT_SCHEMA_VERSION + 1
self.entry_path.write_bytes(
json.dumps(incompatible, sort_keys=True, separators=(",", ":")).encode()
)
self.assertIsNone(self.cache.get(self.key))
foreign_key = self.make_key({"node": "foreign"})
foreign = FragmentRecord.create(foreign_key, b"foreign").to_bytes()
self.entry_path.write_bytes(foreign)
self.assertIsNone(self.cache.get(self.key))
def test_symlinked_entry_and_cache_root_fail_closed_without_outside_writes(self) -> None:
self.entry_path.parent.mkdir(parents=True)
outside = Path(self.temporary.name) / "outside.json"
outside.write_bytes(FragmentRecord.create(self.key, b"outside").to_bytes())
self.entry_path.symlink_to(outside)
self.assertIsNone(self.cache.get(self.key))
self.assertIsNone(self.cache.put(self.key, b"replacement"))
self.assertEqual(b"outside", FragmentRecord.from_bytes(outside.read_bytes()).content)
outside_cache = Path(self.temporary.name) / "outside-cache"
escaped = ProjectionFragmentCache(self.root, outside_cache)
self.assertIsNone(escaped.put(self.key, b"escaped"))
self.assertFalse(outside_cache.exists())
def test_keys_and_records_have_deterministic_path_free_serialization(self) -> None:
same_key = self.make_key({"summary": "Alpha", "node": "guide.alpha"})
self.assertEqual(self.key, same_key)
first = FragmentRecord.create(self.key, b"\x00fragment\xff")
second = FragmentRecord.create(same_key, b"\x00fragment\xff")
self.assertEqual(first, second)
self.assertEqual(first.to_bytes(), second.to_bytes())
self.assertEqual(first, FragmentRecord.from_bytes(first.to_bytes()))
document = json.loads(first.to_bytes())
self.assertEqual(FRAGMENT_SCHEMA_VERSION, document["schema_version"])
self.assertEqual(FRAGMENT_RECORD_CONTRACT, document["contract"])
self.assertEqual(FRAGMENT_KEY_CONTRACT, document["key"]["contract"])
self.assertNotIn(str(self.root), first.to_bytes().decode("utf-8"))
self.assertEqual(first.byte_count, len(first.content))
self.assertEqual(64, len(first.content_sha256))
def test_cache_write_failure_is_a_miss_and_does_not_mutate_canonical_files(self) -> None:
canonical = self.root / "canonical.md"
canonical.write_text("canonical", encoding="utf-8")
with mock.patch(
"docforge.projection_fragments.atomic_replace_bytes_at",
side_effect=OSError("synthetic cache failure"),
):
self.assertIsNone(self.cache.put(self.key, b"fragment"))
self.assertIsNone(self.cache.get(self.key))
self.assertEqual("canonical", canonical.read_text(encoding="utf-8"))
def test_record_rejects_noncanonical_serialization_and_invalid_evidence(self) -> None:
record = FragmentRecord.create(self.key, b"alpha")
pretty = json.dumps(record.as_dict(), sort_keys=True, indent=2).encode()
with self.assertRaisesRegex(Exception, "canonically serialized"):
FragmentRecord.from_bytes(pretty)
tampered = record.as_dict()
tampered["byte_count"] = record.byte_count + 1
canonical = json.dumps(tampered, sort_keys=True, separators=(",", ":")).encode()
with self.assertRaisesRegex(Exception, "byte evidence"):
FragmentRecord.from_bytes(canonical)
def test_prune_retains_only_the_exact_bounded_active_inventory(self) -> None:
retained = self.key
stale = self.make_key({"node": "stale"})
self.assertIsNotNone(self.cache.put(retained, b"retained"))
self.assertIsNotNone(self.cache.put(stale, b"stale"))
foreign = self.entry_path.parent / "foreign.tmp"
foreign.write_bytes(b"foreign")
self.assertTrue(self.cache.prune((retained,)))
retained_record = self.cache.get(retained)
self.assertIsNotNone(retained_record)
assert retained_record is not None
self.assertEqual(b"retained", retained_record.content)
self.assertIsNone(self.cache.get(stale))
self.assertFalse(foreign.exists())
with mock.patch(
"docforge.projection_fragments.MAX_FRAGMENT_CACHE_BYTES",
1,
):
self.assertFalse(self.cache.prune((retained,)))
if __name__ == "__main__":
unittest.main()

266
tests/test_projection_policy.py
View file

@ -1,266 +0,0 @@
from __future__ import annotations
import copy
import hashlib
import json
import unittest
from dataclasses import FrozenInstanceError
from pathlib import Path
from jsonschema import Draft202012Validator
from docforge.errors import DocForgeError
from docforge.projection_policy import (
ProjectionPolicyV2,
compose_projection_policy,
)
ROOT = Path(__file__).resolve().parents[1]
POLICY_SCHEMA = json.loads(
(ROOT / "schemas" / "projection-policy.schema.json").read_text(encoding="utf-8")
)
class ProjectionPolicyV2Tests(unittest.TestCase):
def test_compatible_defaults_are_independent_and_schema_valid(self) -> None:
cases = (
(
{
"manual_configured": False,
"portable_graph_configured": False,
"application_enabled": False,
},
("disabled", "disabled", "on-demand"),
),
(
{
"manual_configured": True,
"portable_graph_configured": False,
"application_enabled": False,
},
("explicit", "disabled", "on-demand"),
),
(
{
"manual_configured": True,
"portable_graph_configured": True,
"application_enabled": True,
},
("auto", "explicit", "on-demand"),
),
(
{
"manual_configured": True,
"portable_graph_configured": True,
"application_enabled": False,
"live_viewer_available": False,
},
("explicit", "explicit", "disabled"),
),
)
validator = Draft202012Validator(POLICY_SCHEMA)
for arguments, expected in cases:
with self.subTest(arguments=arguments):
policy = compose_projection_policy(**arguments)
self.assertEqual(
expected, (policy.manual, policy.portable_graph, policy.live_viewer)
)
validator.validate(policy.as_dict())
def test_explicit_selections_can_narrow_each_consumer_independently(self) -> None:
policy = compose_projection_policy(
manual="explicit",
portable_graph="disabled",
live_viewer="disabled",
manual_configured=True,
portable_graph_configured=True,
application_enabled=True,
)
self.assertEqual(
{
"schema_version": 2,
"manual": "explicit",
"portable_graph": "disabled",
"live_viewer": "disabled",
},
policy.as_dict(),
)
fully_disabled = compose_projection_policy(
manual="disabled",
portable_graph="disabled",
live_viewer="disabled",
manual_configured=False,
portable_graph_configured=False,
application_enabled=False,
live_viewer_available=False,
)
self.assertEqual("disabled", fully_disabled.manual)
self.assertEqual("disabled", fully_disabled.portable_graph)
self.assertEqual("disabled", fully_disabled.live_viewer)
def test_policy_is_frozen_and_hashes_exact_canonical_payload(self) -> None:
first = compose_projection_policy(
manual_configured=True,
portable_graph_configured=True,
application_enabled=False,
)
second = compose_projection_policy(
manual_configured=True,
portable_graph_configured=True,
application_enabled=False,
)
canonical = json.dumps(
first.as_dict(),
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
self.assertEqual(first.as_dict(), second.as_dict())
self.assertEqual(hashlib.sha256(canonical).hexdigest(), first.policy_hash)
self.assertEqual(first.policy_hash, second.policy_hash)
changed = compose_projection_policy(
manual="disabled",
manual_configured=True,
portable_graph_configured=True,
application_enabled=False,
)
self.assertNotEqual(first.policy_hash, changed.policy_hash)
with self.assertRaises(FrozenInstanceError):
first.manual = "disabled" # type: ignore[misc]
def test_invalid_modes_and_direct_construction_fail_closed(self) -> None:
cases = (
("manual", {"manual": "sometimes"}),
("portable_graph", {"portable_graph": "auto"}),
("live_viewer", {"live_viewer": "always"}),
("manual", {"manual": 1}),
)
for expected_projection, selection in cases:
with self.subTest(selection=selection):
with self.assertRaises(DocForgeError) as raised:
compose_projection_policy(
**selection,
manual_configured=True,
portable_graph_configured=True,
application_enabled=True,
)
self.assertEqual("invalid_projection_policy", raised.exception.code)
self.assertEqual(
expected_projection,
raised.exception.details["projection"],
)
with self.assertRaises(DocForgeError) as direct:
ProjectionPolicyV2(
manual="automatic", # type: ignore[arg-type]
portable_graph="explicit",
live_viewer="on-demand",
)
self.assertEqual("invalid_projection_policy", direct.exception.code)
def test_resource_and_capability_unavailability_are_distinct(self) -> None:
cases = (
(
{"manual": "explicit"},
{
"manual_configured": False,
"portable_graph_configured": False,
"application_enabled": False,
},
"manual",
"manual_render_config",
),
(
{"manual": "auto"},
{
"manual_configured": True,
"portable_graph_configured": False,
"application_enabled": False,
},
"manual",
"canonical_application",
),
(
{"portable_graph": "explicit"},
{
"manual_configured": False,
"portable_graph_configured": False,
"application_enabled": False,
},
"portable_graph",
"portable_graph_render_config",
),
(
{"live_viewer": "on-demand"},
{
"manual_configured": False,
"portable_graph_configured": False,
"application_enabled": False,
"live_viewer_available": False,
},
"live_viewer",
"live_viewer_runtime",
),
)
for selection, availability, projection, required in cases:
with self.subTest(selection=selection):
with self.assertRaises(DocForgeError) as raised:
compose_projection_policy(**selection, **availability)
self.assertEqual("projection_policy_unavailable", raised.exception.code)
self.assertEqual(projection, raised.exception.details["projection"])
self.assertEqual(required, raised.exception.details["required"])
def test_availability_inputs_must_be_real_booleans(self) -> None:
cases = (
{"manual_configured": 1},
{"portable_graph_configured": 0},
{"application_enabled": "yes"},
{"live_viewer_available": None},
)
for replacement in cases:
arguments = {
"manual_configured": False,
"portable_graph_configured": False,
"application_enabled": False,
"live_viewer_available": True,
**replacement,
}
with self.subTest(replacement=replacement):
with self.assertRaises(DocForgeError) as raised:
compose_projection_policy(**arguments)
self.assertEqual("invalid_projection_policy", raised.exception.code)
def test_schema_rejects_every_runtime_contract_drift(self) -> None:
validator = Draft202012Validator(POLICY_SCHEMA)
valid = compose_projection_policy(
manual_configured=True,
portable_graph_configured=True,
application_enabled=True,
).as_dict()
invalid_documents: list[dict[str, object]] = []
for field in ("schema_version", "manual", "portable_graph", "live_viewer"):
missing = copy.deepcopy(valid)
missing.pop(field)
invalid_documents.append(missing)
for field, value in (
("schema_version", 1),
("manual", "on-demand"),
("portable_graph", "auto"),
("live_viewer", "explicit"),
):
changed = copy.deepcopy(valid)
changed[field] = value
invalid_documents.append(changed)
extra = copy.deepcopy(valid)
extra["project_path"] = "/private/project"
invalid_documents.append(extra)
for document in invalid_documents:
with self.subTest(document=document):
self.assertFalse(validator.is_valid(document))
if __name__ == "__main__":
unittest.main()

239
tests/test_projection_policy_integration.py
View file

@ -1,239 +0,0 @@
from __future__ import annotations
import shutil
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from docforge.application import CanonicalApplicationService
from docforge.errors import DocForgeError
from docforge.graph_rendering import GraphRenderService
from docforge.index import ProjectIndex
from docforge.mcp_server import DocForgeService
from docforge.project import Project
from docforge.rendering import RenderService
from docforge.viewer_manager import ViewerManagerClient
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
GRAPH_CONFIG = """
[graph_render]
output_root = ".docforge/portable-graph"
[[graph_render.views]]
id = "architecture"
renderer = "portable_graph_html"
output = "architecture.html"
title = "Alpha architecture"
root = "guide.workflow"
initial_mode = "nodes"
depth = 2
max_nodes = 20
max_edges = 40
max_work = 1000
include_logic = false
"""
class ProjectionPolicyIntegrationTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name) / "alpha"
shutil.copytree(FIXTURES / "alpha", self.root)
def project(self, *, graph: bool = False) -> Project:
if graph:
descriptor = self.root / ".docforge/project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
encoding="utf-8",
)
return Project.open(self.root)
def test_disabled_manual_blocks_work_before_load_but_status_remains_receipt_only(self) -> None:
project = self.project()
service = RenderService(project, manual_policy="disabled")
with mock.patch.object(
project,
"load",
side_effect=AssertionError("disabled manual policy must fail before loading"),
):
status = service.status("manual")
self.assertEqual("stale", status["state"])
for operation in (
lambda: service.render("manual"),
lambda: service.deep_status("manual"),
lambda: service.preview("proposal", "manual"),
):
with self.subTest(operation=operation), self.assertRaises(DocForgeError) as raised:
operation()
self.assertEqual(
"projection_policy_forbids_operation",
raised.exception.code,
)
def test_disabled_graph_blocks_plan_and_render_but_status_remains_available(self) -> None:
project = self.project(graph=True)
service = GraphRenderService(project, portable_graph_policy="disabled")
with mock.patch.object(
project,
"load",
side_effect=AssertionError("disabled graph policy must fail before loading"),
):
status = service.status("architecture")
self.assertEqual("stale", status["state"])
for operation in (
lambda: service.plan("architecture"),
lambda: service.render("architecture"),
):
with self.subTest(operation=operation), self.assertRaises(DocForgeError) as raised:
operation()
self.assertEqual(
"projection_policy_forbids_operation",
raised.exception.code,
)
def test_live_viewer_disabled_blocks_start_before_index_work_but_allows_lifecycle(self) -> None:
project = self.project()
index = ProjectIndex(project)
client = ViewerManagerClient(index, live_viewer_policy="disabled")
with (
mock.patch.object(
index,
"check",
side_effect=AssertionError("disabled viewer must fail before index work"),
),
self.assertRaises(DocForgeError) as raised,
):
client.start(node_id="guide.workflow")
self.assertEqual("projection_policy_forbids_operation", raised.exception.code)
with mock.patch.object(
client,
"_lifecycle_request",
side_effect=(
{"status": "ok", "state": "not_running"},
{"status": "ok", "state": "stopped"},
),
):
self.assertEqual("not_running", client.status()["state"])
self.assertEqual("stopped", client.stop()["state"])
def test_application_auto_renders_and_explicit_or_disabled_skip_cleanly(self) -> None:
project = self.project()
for mode, expected_action, expected_calls in (
("auto", "rendered", 1),
("explicit", "skipped_explicit", 0),
("disabled", "skipped_disabled", 0),
):
with self.subTest(mode=mode):
service = CanonicalApplicationService(
project,
applier_id="alpha-editor",
applier=mock.Mock(),
manual_policy=mode, # type: ignore[arg-type]
)
with (
mock.patch.object(
service.changesets,
"apply",
return_value={"status": "ok", "applied": True},
),
mock.patch.object(
service.index,
"build",
return_value={"status": "ok"},
),
mock.patch.object(
service.index,
"check",
return_value={"status": "ok"},
),
mock.patch.object(
service.rendering,
"render",
return_value={"status": "ok", "state": "current"},
) as rendered,
):
result = service.apply("policy-application", "a" * 64)
refresh = result["derived_refresh"]
assert isinstance(refresh, dict)
self.assertEqual(
{"mode": mode, "action": expected_action},
refresh["render_policy"],
)
self.assertEqual(expected_calls, rendered.call_count)
self.assertEqual("ok", refresh["status"])
def test_mcp_exposes_v2_without_changing_v1_policy_or_legacy_render_projection(self) -> None:
project = self.project(graph=True)
baseline = DocForgeService(project, capability_mode_name="read")
narrowed = DocForgeService(
project,
capability_mode_name="read",
manual_projection_policy="disabled",
portable_graph_policy="disabled",
live_viewer_policy="disabled",
)
self.assertEqual(baseline.policy.as_dict(), narrowed.policy.as_dict())
result = narrowed.bootstrap()
self.assertEqual(narrowed.policy.as_dict(), result["effective_policy"])
self.assertEqual(
narrowed.projection_policy.as_dict(),
result["projection_policy"],
)
self.assertEqual(
narrowed.projection_policy.policy_hash,
result["projection_policy_hash"],
)
session = result["session_contract"]
assert isinstance(session, dict)
self.assertEqual(
{
"manual": narrowed.policy.manual_render,
"graph": narrowed.policy.graph_render,
"live_viewer": narrowed.policy.live_viewer,
},
session["render_policies"],
)
self.assertEqual(
"stale",
narrowed.graph_render_status("architecture")["state"],
)
blocked_plan = narrowed.graph_plan("architecture")
self.assertEqual(
"projection_policy_forbids_operation",
blocked_plan["error"]["code"],
)
def test_direct_service_policy_values_are_runtime_validated(self) -> None:
project = self.project(graph=True)
factories = (
lambda: RenderService(project, manual_policy="bogus"), # type: ignore[arg-type]
lambda: GraphRenderService(
project,
portable_graph_policy="bogus", # type: ignore[arg-type]
),
lambda: ViewerManagerClient(
ProjectIndex(project),
live_viewer_policy="bogus", # type: ignore[arg-type]
),
lambda: CanonicalApplicationService(
project,
applier_id=None,
applier=None,
manual_policy="bogus", # type: ignore[arg-type]
),
)
for factory in factories:
with self.subTest(factory=factory), self.assertRaises(DocForgeError) as raised:
factory()
self.assertEqual("invalid_projection_policy", raised.exception.code)
if __name__ == "__main__":
unittest.main()

324
tests/test_projection_schemas.py
View file

@ -1,324 +0,0 @@
from __future__ import annotations
import copy
import json
import unittest
from pathlib import Path
from jsonschema import Draft202012Validator
from docforge.errors import DocForgeError
from docforge.graph_projection import build_graph_projection_package
from docforge.projection_contract import (
GraphViewPlanV1,
ManualRenderPlanV1,
ProjectionPackageV1,
ProjectionReceiptV1,
)
from docforge_renderers.graph import PortableGraphHtmlRenderer
ROOT = Path(__file__).resolve().parents[1]
SCHEMAS = ROOT / "schemas"
class ProjectionSchemaTests(unittest.TestCase):
@staticmethod
def schema(name: str) -> dict[str, object]:
return json.loads((SCHEMAS / name).read_text(encoding="utf-8"))
@classmethod
def validator(cls, name: str) -> Draft202012Validator:
return Draft202012Validator(cls.schema(name))
@staticmethod
def project_identity() -> dict[str, object]:
return {
"project_id": "schema-fixture",
"project_root_fingerprint": "a" * 16,
"adapter": "generic",
"revision": "fixture-revision",
"source_hash": "b" * 64,
}
@classmethod
def manual_plan(cls) -> ManualRenderPlanV1:
return ManualRenderPlanV1.create(
{
"project": cls.project_identity(),
"view": {
"view_id": "manual",
"title": "Schema Manual",
"families": ["guide"],
"renderer": "generic_html",
},
"changeset_hash": None,
"pages": [
{
"node_id": "guide.schema",
"title": "Projection schema",
"family": "guide",
"authority": "authoritative",
"status": "approved",
"tags": ["schema"],
"summary": "Defines the projection schema fixture.",
"content": "The schema fixture is deterministic.",
"content_hash": "c" * 64,
"components": ["manual.commonmark@1"],
"breadcrumbs": [],
"cross_references": [],
"backlinks": [],
}
],
"navigation": [
{
"node_id": "guide.schema",
"title": "Projection schema",
}
],
"search_documents": [
{
"node_id": "guide.schema",
"title": "Projection schema",
"summary": "Defines the projection schema fixture.",
"family": "guide",
"status": "approved",
"tags": ["schema"],
}
],
"diagnostics": {"orphans": ["guide.schema"], "cycles": []},
}
)
@classmethod
def graph_plan(cls) -> GraphViewPlanV1:
return GraphViewPlanV1.create(
{
"project": cls.project_identity(),
"view": {
"view_id": "portable",
"title": "Portable graph",
"initial_mode": "nodes",
"scope": {
"kind": "exact_root",
"root_node_id": "guide.schema",
"depth": 2,
},
"filters": {
"families": [],
"relations": [],
"authorities": [],
"statuses": [],
"tags": [],
},
"detail_fields": [
"node_id",
"title",
"family",
"authority",
"status",
"tags",
"summary",
"content_hash",
],
},
"bounds": {
"depth": 2,
"max_nodes": 100,
"max_edges": 400,
"max_work": 100000,
},
"policy": {
"visibility": "selected_graph_only",
"source_paths": "excluded",
"source_bodies": "excluded",
"database_queries": "forbidden",
"executable_content": "forbidden",
"logic": "forbidden",
"logic_requested": False,
},
"graph": {
"root_node_id": "guide.schema",
"nodes": [],
"edges": [],
"logic_projections": [],
},
"omissions": [],
"diagnostics": {
"selection": "exact_root",
"returned_nodes": 0,
"returned_edges": 0,
"examined_work_units": 0,
"truncated": False,
"ordering": "node_id;source_id,relation,target_id",
},
}
)
@classmethod
def package(
cls,
plan: ManualRenderPlanV1 | GraphViewPlanV1 | None = None,
) -> ProjectionPackageV1:
selected = plan or cls.manual_plan()
kind = "manual" if isinstance(selected, ManualRenderPlanV1) else "graph"
return ProjectionPackageV1.create(
kind=kind,
plan=selected,
renderer={
"renderer_id": "generic_html",
"renderer_version": "1",
},
components=[{"component_id": "projection.document@1"}],
assets=[
{
"asset_id": "projection.template",
"media_type": "text/plain; charset=utf-8",
"sha256": "d" * 64,
"text": "fixture",
}
],
output_policy={
"artifact_ids": ["projection.html"],
"max_total_bytes": 1000000,
},
)
@classmethod
def receipt(cls) -> ProjectionReceiptV1:
package = cls.package()
return ProjectionReceiptV1.create(
kind="manual",
package_id=package.package_id,
plan_id=package.document["plan_id"], # type: ignore[arg-type]
renderer={
"renderer_id": "generic_html",
"renderer_version": "1",
},
artifacts=[
{
"artifact_id": "manual.html",
"media_type": "text/html; charset=utf-8",
"sha256": "e" * 64,
"bytes": 123,
}
],
diagnostics={"warnings": []},
timing={"elapsed_ns": 123456},
peak_memory_bytes=None,
)
def test_schemas_are_valid_and_accept_current_documents(self) -> None:
documents = {
"manual-render-plan.schema.json": self.manual_plan().as_dict(),
"graph-view-plan.schema.json": self.graph_plan().as_dict(),
"projection-package.schema.json": self.package().as_dict(),
"projection-receipt.schema.json": self.receipt().as_dict(),
}
for name, document in documents.items():
with self.subTest(schema=name):
schema = self.schema(name)
Draft202012Validator.check_schema(schema)
Draft202012Validator(schema).validate(document)
graph_package = self.package(self.graph_plan()).as_dict()
self.validator("projection-package.schema.json").validate(graph_package)
typed_graph_package = build_graph_projection_package(
self.graph_plan(),
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=1_000_000,
)
graph_receipt = PortableGraphHtmlRenderer().render(typed_graph_package).receipt.as_dict()
self.validator("projection-receipt.schema.json").validate(graph_receipt)
def test_unknown_fields_are_rejected_at_contract_boundaries(self) -> None:
cases = (
(
"manual-render-plan.schema.json",
self.manual_plan().as_dict(),
),
(
"graph-view-plan.schema.json",
self.graph_plan().as_dict(),
),
(
"projection-package.schema.json",
self.package().as_dict(),
),
(
"projection-receipt.schema.json",
self.receipt().as_dict(),
),
)
for name, document in cases:
with self.subTest(schema=name):
document["unexpected"] = True
self.assertFalse(self.validator(name).is_valid(document))
manual = self.manual_plan().as_dict()
manual["pages"][0]["unexpected"] = True # type: ignore[index]
self.assertFalse(self.validator("manual-render-plan.schema.json").is_valid(manual))
package = self.package().as_dict()
package["assets"][0]["path"] = "/tmp/escape" # type: ignore[index]
self.assertFalse(self.validator("projection-package.schema.json").is_valid(package))
receipt = self.receipt().as_dict()
receipt["artifacts"][0]["content"] = "not receipt evidence" # type: ignore[index]
self.assertFalse(self.validator("projection-receipt.schema.json").is_valid(receipt))
def test_obviously_malformed_identities_and_structures_are_rejected(self) -> None:
manual = self.manual_plan().as_dict()
manual["plan_id"] = "not-a-sha256"
self.assertFalse(self.validator("manual-render-plan.schema.json").is_valid(manual))
graph = self.graph_plan().as_dict()
graph["project"]["project_root_fingerprint"] = "wrong" # type: ignore[index]
self.assertFalse(self.validator("graph-view-plan.schema.json").is_valid(graph))
graph = self.graph_plan().as_dict()
graph["bounds"] = -1
self.assertFalse(self.validator("graph-view-plan.schema.json").is_valid(graph))
package = self.package().as_dict()
package["assets"][0].pop("sha256") # type: ignore[index]
self.assertFalse(self.validator("projection-package.schema.json").is_valid(package))
package = self.package().as_dict()
package["output_policy"]["max_total_bytes"] = 0 # type: ignore[index]
self.assertFalse(self.validator("projection-package.schema.json").is_valid(package))
package = self.package().as_dict()
package["kind"] = "graph"
self.assertFalse(self.validator("projection-package.schema.json").is_valid(package))
receipt = self.receipt().as_dict()
receipt["artifacts"][0]["artifact_id"] = "../manual.html" # type: ignore[index]
self.assertFalse(self.validator("projection-receipt.schema.json").is_valid(receipt))
receipt = self.receipt().as_dict()
receipt["artifacts"][0]["bytes"] = -1 # type: ignore[index]
self.assertFalse(self.validator("projection-receipt.schema.json").is_valid(receipt))
receipt = self.receipt().as_dict()
receipt["peak_memory_bytes"] = True
self.assertFalse(self.validator("projection-receipt.schema.json").is_valid(receipt))
def test_assets_and_artifacts_enforce_fixed_collection_bounds(self) -> None:
package = self.package().as_dict()
package["assets"] = [copy.deepcopy(package["assets"][0]) for _ in range(33)] # type: ignore[index]
self.assertFalse(self.validator("projection-package.schema.json").is_valid(package))
receipt = self.receipt().as_dict()
receipt["artifacts"] = [
copy.deepcopy(receipt["artifacts"][0])
for _ in range(33) # type: ignore[index]
]
self.assertFalse(self.validator("projection-receipt.schema.json").is_valid(receipt))
def test_canonical_identity_equality_remains_a_runtime_check(self) -> None:
document = self.manual_plan().as_dict()
document["plan_id"] = "f" * 64
self.validator("manual-render-plan.schema.json").validate(document)
with self.assertRaises(DocForgeError) as raised:
ManualRenderPlanV1.from_dict(document)
self.assertEqual("invalid_projection", raised.exception.code)
if __name__ == "__main__":
unittest.main()

388
tests/test_projection_worker.py
View file

@ -1,388 +0,0 @@
from __future__ import annotations
import base64
import contextlib
import copy
import json
import os
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from docforge.errors import DocForgeError
from docforge.graph_projection import (
GraphViewRequestV1,
build_graph_projection_package,
build_graph_view_plan,
)
from docforge.manual_projection import (
build_manual_projection_package,
build_manual_render_plan,
)
from docforge.project import Project
from docforge.projection_contract import (
ProjectionPackageV1,
ProjectionReceiptV1,
canonical_projection_bytes,
)
from docforge.projection_worker import (
MAX_WORKER_ARTIFACT_BYTES,
MAX_WORKER_RESPONSE_BYTES,
_child_response,
render_projection_in_worker,
)
from docforge.render_contract import GenericHtmlRenderer
from docforge_renderers.graph import PortableGraphHtmlRenderer
from docforge_renderers.manual import ManualHtmlRenderer
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
class ProjectionWorkerTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name) / "alpha"
shutil.copytree(FIXTURES / "alpha", self.root)
self.project = Project.open(self.root)
self.snapshot = self.project.load()
render = self.snapshot.descriptor.render
assert render is not None
self.manual_view = render.views[0]
self.manual_version = GenericHtmlRenderer().renderer_version
self.manual_plan = build_manual_render_plan(
self.snapshot,
self.manual_view,
changeset_hash=None,
)
self.manual_package = build_manual_projection_package(
self.manual_plan,
self.manual_view.template_path.read_bytes(),
renderer_id="generic_html",
renderer_version=self.manual_version,
max_output_bytes=self.snapshot.descriptor.limits.max_render_bytes,
)
self.graph_plan = build_graph_view_plan(
self.snapshot,
GraphViewRequestV1(
view_id="architecture",
title="Alpha architecture",
root_node_id="guide.workflow",
depth=2,
max_nodes=20,
max_edges=40,
max_work=1_000,
),
False,
)
self.graph_package = build_graph_projection_package(
self.graph_plan,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=self.snapshot.descriptor.limits.max_render_bytes,
)
def test_manual_and_graph_workers_match_the_in_process_renderers(self) -> None:
cases = (
(
self.manual_package,
ManualHtmlRenderer(self.manual_version).render(self.manual_package),
),
(
self.graph_package,
PortableGraphHtmlRenderer().render(self.graph_package),
),
)
for package, expected in cases:
with self.subTest(kind=package.kind):
result = render_projection_in_worker(package)
self.assertEqual(expected.artifacts, result.artifacts)
receipt = result.receipt.as_dict()
self.assertEqual(package.package_id, receipt["package_id"])
self.assertEqual(package.document["plan_id"], receipt["plan_id"])
self.assertIs(type(receipt["peak_memory_bytes"]), int)
self.assertGreater(receipt["peak_memory_bytes"], 0)
def test_parent_launch_is_fixed_and_request_contains_no_runtime_authority(self) -> None:
original = subprocess.run
with mock.patch(
"docforge.projection_worker.subprocess.run",
wraps=original,
) as launched:
render_projection_in_worker(self.manual_package)
command = launched.call_args.args[0]
options = launched.call_args.kwargs
self.assertEqual(
[sys.executable, "-I", "-m", "docforge._projection_worker_main"],
command,
)
self.assertIs(options["shell"], False)
self.assertEqual(sys.prefix, options["cwd"])
self.assertEqual(
{"PYTHONIOENCODING": "utf-8", "PYTHONUTF8": "1"},
options["env"],
)
request = options["input"]
self.assertEqual(
canonical_projection_bytes(self.manual_package.as_dict()) + b"\n",
request,
)
for forbidden in (
str(self.root).encode(),
b"project_path",
b"index_path",
b"database_path",
b'"command"',
b'"module"',
b'"shell"',
b'"sql"',
):
self.assertNotIn(forbidden, request)
def test_worker_ignores_hostile_cwd_and_pythonpath_import_shadows(self) -> None:
shadow_root = Path(self.temporary.name) / "shadow"
shadow_package = shadow_root / "docforge"
shadow_package.mkdir(parents=True)
sentinel = shadow_root / "executed"
shadow_package.joinpath("__init__.py").write_text(
"from pathlib import Path\n"
f"Path({str(sentinel)!r}).write_text('executed', encoding='utf-8')\n",
encoding="utf-8",
)
with (
contextlib.chdir(shadow_root),
mock.patch.dict(
os.environ,
{"PYTHONPATH": str(shadow_root)},
clear=False,
),
):
result = render_projection_in_worker(self.manual_package)
self.assertEqual("manual", result.receipt.document["kind"])
self.assertFalse(sentinel.exists())
def test_parent_accepts_only_valid_packages_and_closed_renderer_versions(self) -> None:
with self.assertRaises(TypeError):
render_projection_in_worker({}) # type: ignore[arg-type]
unsupported = ProjectionPackageV1.create(
kind="manual",
plan=self.manual_plan,
renderer={"renderer_id": "generic_html", "renderer_version": "other"},
components=[{"component_id": "manual.document@1"}],
assets=copy.deepcopy(self.manual_package.document["assets"]),
output_policy=copy.deepcopy(self.manual_package.document["output_policy"]),
)
with (
mock.patch("docforge.projection_worker._invoke_worker") as invoke,
self.assertRaises(DocForgeError) as raised,
):
render_projection_in_worker(unsupported)
self.assertEqual("unsupported_renderer", raised.exception.code)
invoke.assert_not_called()
permissive_allowance = ProjectionPackageV1.create(
kind="manual",
plan=self.manual_plan,
renderer={
"renderer_id": "generic_html",
"renderer_version": self.manual_version,
},
components=[{"component_id": "manual.document@1"}],
assets=copy.deepcopy(self.manual_package.document["assets"]),
output_policy={
"artifact_ids": ["manual.html"],
"max_total_bytes": MAX_WORKER_ARTIFACT_BYTES + 1,
},
)
rendered = render_projection_in_worker(permissive_allowance)
self.assertEqual(
ManualHtmlRenderer(self.manual_version)
.render(permissive_allowance)
.artifacts[0]
.content,
rendered.artifacts[0].content,
)
def test_process_timeout_exit_and_signal_fail_closed(self) -> None:
failures = (
(
subprocess.TimeoutExpired(["worker"], 30),
"projection_worker_timeout",
),
(
subprocess.CompletedProcess(["worker"], 2, stdout=b""),
"projection_worker_failure",
),
(
subprocess.CompletedProcess(["worker"], -9, stdout=b""),
"projection_worker_failure",
),
)
for outcome, code in failures:
with self.subTest(outcome=type(outcome).__name__):
patch = (
mock.patch(
"docforge.projection_worker._invoke_worker",
side_effect=outcome,
)
if isinstance(outcome, BaseException)
else mock.patch(
"docforge.projection_worker._invoke_worker",
return_value=outcome,
)
)
with patch, self.assertRaises(DocForgeError) as raised:
render_projection_in_worker(self.manual_package)
self.assertEqual(code, raised.exception.code)
def test_malformed_trailing_and_oversized_responses_fail_closed(self) -> None:
valid = _child_response(self.manual_package)
malformed = (
b"",
b"{}",
b"not-json\n",
b'{ "schema_version": 1 }\n',
valid + b"{}\n",
b"x" * (MAX_WORKER_RESPONSE_BYTES + 1),
)
for response in malformed:
with (
self.subTest(length=len(response)),
mock.patch(
"docforge.projection_worker._invoke_worker",
return_value=subprocess.CompletedProcess(
["worker"],
0,
stdout=response,
),
),
self.assertRaises(DocForgeError) as raised,
):
render_projection_in_worker(self.manual_package)
self.assertEqual("projection_worker_failure", raised.exception.code)
def test_wrong_artifact_and_receipt_evidence_fail_closed(self) -> None:
valid = json.loads(_child_response(self.manual_package))
cases: dict[str, dict[str, object]] = {}
wrong_content = copy.deepcopy(valid)
wrong_content["artifacts"][0]["content_base64"] = base64.b64encode(b"changed").decode()
cases["content_hash"] = wrong_content
wrong_id = copy.deepcopy(valid)
wrong_id["artifacts"][0]["artifact_id"] = "other.html"
cases["artifact_id"] = wrong_id
missing_artifact = copy.deepcopy(valid)
missing_artifact["artifacts"] = []
cases["artifact_count"] = missing_artifact
wrong_media_type = copy.deepcopy(valid)
wrong_media_type["artifacts"][0]["media_type"] = "application/octet-stream"
cases["media_type"] = wrong_media_type
bad_base64 = copy.deepcopy(valid)
bad_base64["artifacts"][0]["content_base64"] = "***"
cases["base64"] = bad_base64
wrong_package = copy.deepcopy(valid)
receipt = wrong_package["receipt"]
replacement = ProjectionReceiptV1.create(
kind="manual",
package_id="0" * 64,
plan_id=receipt["plan_id"],
renderer=receipt["renderer"],
artifacts=receipt["artifacts"],
diagnostics=receipt["diagnostics"],
timing=receipt["timing"],
peak_memory_bytes=receipt["peak_memory_bytes"],
)
wrong_package["receipt"] = replacement.as_dict()
cases["package_id"] = wrong_package
zero_peak = copy.deepcopy(valid)
receipt = zero_peak["receipt"]
replacement = ProjectionReceiptV1.create(
kind="manual",
package_id=receipt["package_id"],
plan_id=receipt["plan_id"],
renderer=receipt["renderer"],
artifacts=receipt["artifacts"],
diagnostics=receipt["diagnostics"],
timing=receipt["timing"],
peak_memory_bytes=0,
)
zero_peak["receipt"] = replacement.as_dict()
cases["peak_memory"] = zero_peak
wrong_size = copy.deepcopy(valid)
receipt = wrong_size["receipt"]
artifacts = copy.deepcopy(receipt["artifacts"])
artifacts[0]["bytes"] += 1
replacement = ProjectionReceiptV1.create(
kind="manual",
package_id=receipt["package_id"],
plan_id=receipt["plan_id"],
renderer=receipt["renderer"],
artifacts=artifacts,
diagnostics=receipt["diagnostics"],
timing=receipt["timing"],
peak_memory_bytes=receipt["peak_memory_bytes"],
)
wrong_size["receipt"] = replacement.as_dict()
cases["artifact_size"] = wrong_size
configured_oversize = copy.deepcopy(valid)
maximum = self.manual_package.document["output_policy"]["max_total_bytes"]
configured_oversize["artifacts"][0]["content_base64"] = base64.b64encode(
b"x" * (maximum + 1)
).decode()
cases["configured_aggregate"] = configured_oversize
for name, document in cases.items():
response = canonical_projection_bytes(document) + b"\n"
with (
self.subTest(name=name),
mock.patch(
"docforge.projection_worker._invoke_worker",
return_value=subprocess.CompletedProcess(
["worker"],
0,
stdout=response,
),
),
self.assertRaises(DocForgeError) as raised,
):
render_projection_in_worker(self.manual_package)
self.assertEqual("projection_worker_failure", raised.exception.code)
def test_child_rejects_noncanonical_invalid_and_trailing_requests(self) -> None:
valid = canonical_projection_bytes(self.manual_package.as_dict()) + b"\n"
requests = (
b"{}\n",
b'{ "schema_version": 1 }\n',
valid + b"{}\n",
)
for request in requests:
with self.subTest(length=len(request)):
completed = subprocess.run(
[sys.executable, "-I", "-m", "docforge._projection_worker_main"],
cwd=ROOT,
input=request,
capture_output=True,
check=False,
timeout=10,
)
self.assertEqual(2, completed.returncode)
self.assertEqual(b"", completed.stdout)
self.assertEqual(b"", completed.stderr)
if __name__ == "__main__":
unittest.main()

44
tests/test_public_contract.py
View file

@ -64,16 +64,6 @@ PUBLIC_IMPORTS = {
"docforge.client_config": ("generate_client_configuration",),
"docforge.doctor": ("run_doctor",),
"docforge.index": ("ProjectIndex",),
"docforge.graph_projection": (
"GraphViewRequestV1",
"build_graph_projection_package",
"build_graph_view_plan",
),
"docforge.graph_rendering": ("GraphRenderService",),
"docforge.manual_projection": (
"build_manual_projection_package",
"build_manual_render_plan",
),
"docforge.mcp_server": (
"create_project_server",
"create_read_only_server",
@ -94,30 +84,6 @@ PUBLIC_IMPORTS = {
"capability_mode",
"compose_effective_policy",
),
"docforge.projection_policy": (
"ProjectionPolicyV2",
"compose_projection_policy",
"validate_live_viewer_projection_mode",
"validate_manual_projection_mode",
"validate_portable_graph_projection_mode",
),
"docforge.projection_fragments": (
"FragmentKey",
"FragmentRecord",
"ProjectionFragmentCache",
"fragment_semantic_hash",
),
"docforge.projection_contract": (
"GraphViewPlanV1",
"ManualRenderPlanV1",
"ProjectionArtifact",
"ProjectionPackageV1",
"ProjectionReceiptV1",
"ProjectionRenderResult",
"canonical_projection_bytes",
"projection_hash",
),
"docforge.projection_worker": ("render_projection_in_worker",),
"docforge.retrieval": (
"ContextCapsuleV1",
"RetrievalPlanV1",
@ -131,8 +97,6 @@ PUBLIC_IMPORTS = {
"Renderer",
"renderer_for",
),
"docforge_renderers.manual": ("ManualHtmlRenderer",),
"docforge_renderers.graph": ("PortableGraphHtmlRenderer",),
}
EXPECTED_ENTRY_POINTS = {
@ -152,9 +116,6 @@ EXPECTED_CLI_COMMANDS = {
"doctor",
"filter",
"generation-diff",
"graph-plan",
"graph-render",
"graph-render-status",
"impact",
"info",
"onboard",
@ -200,8 +161,6 @@ EXPECTED_MCP_TOOLS = {
"docforge_rebase_changeset",
"docforge_register_changes",
"docforge_render_status",
"docforge_graph_plan",
"docforge_graph_render_status",
"docforge_search",
"docforge_stop_visualization",
"docforge_sync",
@ -258,9 +217,6 @@ class PublicContractTests(unittest.TestCase):
)
self.assertIn("--project-root", completed.stdout)
self.assertIn("--no-ast", completed.stdout)
self.assertIn("--manual-render-policy", completed.stdout)
self.assertIn("--portable-graph-policy", completed.stdout)
self.assertIn("--live-viewer-policy", completed.stdout)
def test_published_schemas_validate_their_current_contract_examples(self) -> None:
for path in sorted(SCHEMAS.glob("*.json")):

30
tests/test_rendering.py
View file

@ -55,14 +55,6 @@ class DocForgeRenderingTests(unittest.TestCase):
self.assertEqual("missing", missing["outputs"][0]["state"])
first = service.render("manual")
projection_receipt = first["output"]["projection_receipt"]
self.assertIsInstance(projection_receipt, dict)
assert isinstance(projection_receipt, dict)
self.assertEqual("manual", projection_receipt["kind"])
self.assertEqual(
first["output"]["actual_output_hash"],
projection_receipt["artifacts"][0]["sha256"],
)
output = root / ".docforge/rendered/manual.html"
first_bytes = output.read_bytes()
second = service.render("manual")
@ -199,12 +191,7 @@ class DocForgeRenderingTests(unittest.TestCase):
self.assertEqual("receipt_corrupt", corrupt["outputs"][0]["reason"])
def test_render_receipt_schema_and_renderer_version_fail_closed(self) -> None:
for mutation in (
"missing_hash",
"renderer_version",
"file_identity",
"projection_artifact",
):
for mutation in ("missing_hash", "renderer_version", "file_identity"):
with self.subTest(mutation=mutation), tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
service = RenderService(Project.open(root))
@ -215,8 +202,6 @@ class DocForgeRenderingTests(unittest.TestCase):
receipt.pop("output_hash")
elif mutation == "renderer_version":
receipt["renderer_version"] = "obsolete"
elif mutation == "projection_artifact":
receipt["projection_receipt"]["artifacts"][0]["sha256"] = "0" * 64
else:
receipt["output_file"].pop("ctime_ns")
receipt_path.write_text(
@ -416,19 +401,6 @@ class DocForgeRenderingTests(unittest.TestCase):
with self.assertRaisesRegex(DocForgeError, "configured limit") as limit_error:
limit_service.render("manual")
self.assertEqual("render_too_large", limit_error.exception.code)
excessive = self.copy_fixture("alpha", Path(directory) / "excessive")
excessive_descriptor = excessive / ".docforge/project.toml"
excessive_descriptor.write_text(
excessive_descriptor.read_text(encoding="utf-8").replace(
"max_changeset_bytes = 100000",
"max_changeset_bytes = 100000\nmax_render_bytes = 20000001",
),
encoding="utf-8",
)
permissive_limit = RenderService(Project.open(excessive)).render("manual")
self.assertEqual("current", permissive_limit["receipt"]["state"])
self.assertTrue((excessive / ".docforge/rendered/manual.html").is_file())
self.assertFalse((limit_root / ".docforge/rendered/manual.html").exists())
template_limit_root = self.copy_fixture("alpha", parent / "template-limit")

19
tests/test_visualization.py
View file

@ -386,25 +386,6 @@ class VisualizationTests(unittest.TestCase):
with self.assertRaisesRegex(DocForgeError, "category is unsupported"):
snapshot.filter_nodes(category="relation", value="depends_on", limit=2)
def test_snapshot_source_never_mixes_pinned_graph_with_newer_canonical_text(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
index = ProjectIndex(Project.open(root))
index.build()
snapshot = VisualizationIndexSnapshot(index, index.check())
before = snapshot.source("guide.workflow")
source = root / "docs/content/workflow.md"
source.write_text(
source.read_text(encoding="utf-8") + "\nNewer unindexed source text.\n",
encoding="utf-8",
)
after = snapshot.source("guide.workflow")
self.assertEqual(before["content"], after["content"])
self.assertNotIn("Newer unindexed source text", after["content"])
self.assertEqual("index_snapshot", after["source_provenance"])
def test_flow_reverses_imports_into_a_complete_structural_path(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))

99
tools/accessibility_fixture.py
View file

@ -1,99 +0,0 @@
"""Serve exact DocForge accessibility fixtures to the Playwright gate."""
from __future__ import annotations
import json
import shutil
import sys
import tempfile
from pathlib import Path
from typing import cast
from docforge.graph_projection import (
GraphViewRequestV1,
build_graph_projection_package,
build_graph_view_plan,
)
from docforge.index import ProjectIndex
from docforge.project import Project
from docforge.rendering import RenderService
from docforge.visualization import VisualizationRunner
from docforge_renderers.graph import PortableGraphHtmlRenderer
ROOT = Path(__file__).resolve().parents[1]
def _manual_html(project: Project) -> str:
result = RenderService(project).render("manual")
output = result.get("output")
if not isinstance(output, dict):
raise RuntimeError("Manual render did not return output identity")
relative_path = cast(dict[str, object], output).get("path")
if not isinstance(relative_path, str):
raise RuntimeError("Manual render did not return an output path")
return (project.descriptor.root / relative_path).read_text(encoding="utf-8")
def _portable_graph_html(project: Project) -> str:
plan = build_graph_view_plan(
project.load(),
GraphViewRequestV1(
view_id="accessibility-gate",
title="Portable graph accessibility fixture",
root_node_id="guide.workflow",
depth=2,
max_nodes=20,
max_edges=40,
max_work=1_000,
),
False,
)
package = build_graph_projection_package(
plan,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=1_000_000,
)
artifact = PortableGraphHtmlRenderer().render(package).artifacts[0]
return artifact.content.decode("utf-8")
def main() -> int:
with tempfile.TemporaryDirectory(prefix="docforge-accessibility-") as directory:
project_root = Path(directory) / "alpha"
shutil.copytree(ROOT / "tests" / "fixtures" / "alpha", project_root)
project = Project.open(project_root)
index = ProjectIndex(project)
index.build()
portable_html = _portable_graph_html(project)
manual_html = _manual_html(project)
runner = VisualizationRunner(
index,
register_atexit=False,
persistent=True,
)
try:
visualization = runner.start(node_id="guide.workflow", depth=2)
live_url = visualization.get("url")
if not isinstance(live_url, str):
raise RuntimeError("Live viewer did not return a URL")
print(
json.dumps(
{
"schema_version": 1,
"manual_html": manual_html,
"portable_html": portable_html,
"live_url": live_url,
},
separators=(",", ":"),
),
flush=True,
)
sys.stdin.buffer.read()
finally:
runner.stop()
return 0
if __name__ == "__main__":
sys.exit(main())

64
tools/check_web_assets.py
View file

@ -10,19 +10,9 @@ import tempfile
from pathlib import Path
from typing import cast
from docforge.graph_projection import (
GraphViewRequestV1,
build_graph_projection_package,
build_graph_view_plan,
)
from docforge.index import ProjectIndex
from docforge.project import Project
from docforge.rendering import RenderService
from docforge_renderers.graph import (
PORTABLE_GRAPH_CSS,
PORTABLE_GRAPH_JAVASCRIPT,
PortableGraphHtmlRenderer,
)
ROOT = Path(__file__).resolve().parents[1]
ASSET_ROOT = ROOT / "src" / "docforge" / "assets"
@ -49,30 +39,6 @@ def _rendered_manual_html() -> str:
return (project_root / relative_path).read_text(encoding="utf-8")
def _portable_graph_html() -> str:
project = Project.open(ROOT / "tests" / "fixtures" / "alpha")
plan = build_graph_view_plan(
project.load(),
GraphViewRequestV1(
view_id="web-quality",
title="Portable graph quality fixture",
root_node_id="guide.workflow",
depth=2,
max_nodes=20,
max_edges=40,
max_work=1_000,
),
False,
)
package = build_graph_projection_package(
plan,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=1_000_000,
)
return PortableGraphHtmlRenderer().render(package).artifacts[0].content.decode("utf-8")
def _tool(name: str) -> str:
suffix = ".cmd" if os.name == "nt" else ""
path = ROOT / "node_modules" / ".bin" / f"{name}{suffix}"
@ -116,16 +82,6 @@ def main() -> int:
],
_rendered_manual_html(),
),
(
"portable graph HTML",
[
_tool("html-validate"),
"--stdin",
"--stdin-filename=portable-graph.html",
"--max-warnings=0",
],
_portable_graph_html(),
),
(
"CSS",
[
@ -146,26 +102,6 @@ def main() -> int:
],
(ASSET_ROOT / "graph.js").read_text(encoding="utf-8"),
),
(
"portable graph CSS",
[
_tool("stylelint"),
"--stdin",
"--stdin-filename=portable-graph.css",
"--max-warnings=0",
],
PORTABLE_GRAPH_CSS,
),
(
"portable graph JavaScript",
[
_tool("eslint"),
"--stdin",
"--stdin-filename=portable-graph.js",
"--max-warnings=0",
],
PORTABLE_GRAPH_JAVASCRIPT,
),
)
results = [_run(label, command, content) for label, command, content in checks]
return 0 if all(results) else 1

893
tools/milestone3_benchmark.py
View file

@ -1,893 +0,0 @@
"""Milestone 3 projection, worker, fragment, and publication benchmark gates."""
from __future__ import annotations
import argparse
import gc
import hashlib
import json
import math
import platform
import resource
import statistics
import subprocess
import sys
import tempfile
import time
import tracemalloc
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass, replace
from pathlib import Path
from typing import cast
from milestone0_baseline import write_synthetic_project
from docforge.graph_projection import (
GraphViewRequestV1,
build_graph_projection_package,
build_graph_view_plan,
)
from docforge.graph_rendering import GraphRenderService
from docforge.manual_projection import (
build_manual_projection_package,
build_manual_render_plan,
)
from docforge.models import ProjectSnapshot
from docforge.project import Project
from docforge.projection_contract import (
MAX_PACKAGE_BYTES,
MAX_PLAN_BYTES,
MAX_RECEIPT_BYTES,
GraphViewPlanV1,
ManualRenderPlanV1,
ProjectionPackageV1,
ProjectionRenderResult,
canonical_projection_bytes,
projection_hash,
)
from docforge.projection_fragments import (
FragmentKey,
FragmentRecord,
ProjectionFragmentCache,
fragment_semantic_hash,
)
from docforge.projection_worker import (
MAX_WORKER_ARTIFACT_BYTES,
render_projection_in_worker,
)
from docforge.render_contract import GenericHtmlRenderer, PreparedRender
from docforge.rendering import RenderService
from docforge.telemetry import COUNTER_NAMES, request
from docforge_renderers.graph import PortableGraphHtmlRenderer
from docforge_renderers.manual import ManualHtmlRenderer
ROOT = Path(__file__).resolve().parents[1]
FULL_NODE_COUNT = 1_000
SMOKE_NODE_COUNT = 25
DEFAULT_FULL_SAMPLES = 3
DEFAULT_SMOKE_SAMPLES = 1
MAX_STATUS_RESPONSE_BYTES = 256_000
MAX_TRACED_PEAK_BYTES = 256 * 1024 * 1024
MAX_CHILD_PEAK_BYTES = 256 * 1024 * 1024
ZERO_WORK_COUNTERS = tuple(
counter for counter in COUNTER_NAMES if counter != "source_generation_checks"
)
@dataclass(frozen=True)
class _ProjectionSample:
plan_id: str
package_id: str
artifact: bytes
receipt: Mapping[str, object]
plan_bytes: int
package_bytes: int
child_peak_memory_bytes: int | None
@dataclass(frozen=True)
class _StatusSample:
response: Mapping[str, object]
counters: Mapping[str, object]
@dataclass(frozen=True)
class _FragmentSweep:
fragment_count: int
aggregate_content_bytes: int
ordered_record_hash: str
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="Gate DocForge2 Milestone 3 projection behavior on a disposable project."
)
parser.add_argument("--mode", choices=("smoke", "full"), default="full")
parser.add_argument("--nodes", type=int)
parser.add_argument("--samples", type=int)
parser.add_argument("--output", type=Path)
return parser
def encode_report(value: object) -> str:
"""Serialize one report deterministically for files, CI logs, and comparisons."""
return json.dumps(value, sort_keys=True, indent=2, ensure_ascii=False) + "\n"
def _git(arguments: list[str]) -> str:
return subprocess.run(
["git", *arguments],
cwd=ROOT,
check=True,
capture_output=True,
text=True,
).stdout.strip()
def _compact_size(value: object) -> int:
return len(canonical_projection_bytes(value))
def _sha256(content: bytes) -> str:
return hashlib.sha256(content).hexdigest()
def _prepare_fixture(root: Path, node_count: int) -> None:
write_synthetic_project(root, node_count)
descriptor = root / ".docforge" / "project.toml"
maximum_edges = node_count - 1
maximum_work = max(100, node_count * 4)
original = descriptor.read_text(encoding="utf-8")
original = original.replace(
"max_render_bytes = 20000000",
"max_render_bytes = 4000000",
)
descriptor.write_text(
original
+ f"""
[graph_render]
output_root = ".docforge/portable-graph"
[[graph_render.views]]
id = "architecture"
renderer = "portable_graph_html"
output = "architecture.html"
title = "Synthetic architecture"
query = "synthetic measurement"
initial_mode = "web"
depth = 1
max_nodes = {node_count}
max_edges = {maximum_edges}
max_work = {maximum_work}
families = ["guide"]
relations = ["depends_on"]
authorities = []
statuses = ["active"]
tags = []
include_logic = false
""",
encoding="utf-8",
)
def _manual_package(
snapshot: ProjectSnapshot,
) -> tuple[ManualRenderPlanV1, ProjectionPackageV1, str]:
descriptor = snapshot.descriptor
render = descriptor.render
if render is None:
raise RuntimeError("Milestone 3 fixture has no manual render configuration")
view = render.views[0]
renderer_version = GenericHtmlRenderer(incremental=False).renderer_version
plan = build_manual_render_plan(snapshot, view, changeset_hash=None)
package = build_manual_projection_package(
plan,
view.template_path.read_bytes(),
renderer_id=ManualHtmlRenderer.renderer_id,
renderer_version=renderer_version,
max_output_bytes=descriptor.limits.max_render_bytes,
)
return plan, package, renderer_version
def _graph_package(
snapshot: ProjectSnapshot,
node_count: int,
) -> tuple[GraphViewPlanV1, ProjectionPackageV1]:
descriptor = snapshot.descriptor
plan = build_graph_view_plan(
snapshot,
GraphViewRequestV1(
view_id="architecture",
title="Synthetic architecture",
query="synthetic measurement",
initial_mode="web",
depth=1,
max_nodes=node_count,
max_edges=node_count - 1,
max_work=max(100, node_count * 4),
families=("guide",),
relations=("depends_on",),
statuses=("active",),
),
False,
)
package = build_graph_projection_package(
plan,
renderer_id=PortableGraphHtmlRenderer.renderer_id,
renderer_version=PortableGraphHtmlRenderer.renderer_version,
max_output_bytes=descriptor.limits.max_render_bytes,
)
return plan, package
def _projection_sample(
plan: ManualRenderPlanV1 | GraphViewPlanV1,
package: ProjectionPackageV1,
result: ProjectionRenderResult,
) -> _ProjectionSample:
if len(result.artifacts) != 1:
raise RuntimeError("Projection benchmark expected exactly one artifact")
artifact = result.artifacts[0].content
peak = result.receipt.document.get("peak_memory_bytes")
if peak is not None and (type(peak) is not int or peak < 1):
raise RuntimeError("Detached worker did not report valid peak memory")
return _ProjectionSample(
plan_id=plan.plan_id,
package_id=package.package_id,
artifact=artifact,
receipt=result.receipt.as_dict(),
plan_bytes=_compact_size(plan.as_dict()),
package_bytes=_compact_size(package.as_dict()),
child_peak_memory_bytes=peak,
)
def _projection_summary(sample: _ProjectionSample) -> dict[str, object]:
return {
"plan_id": sample.plan_id,
"package_id": sample.package_id,
"artifact_sha256": _sha256(sample.artifact),
"artifact_bytes": len(sample.artifact),
"plan_bytes": sample.plan_bytes,
"package_bytes": sample.package_bytes,
}
def _measure(
operation: Callable[[], object],
*,
samples: int,
p95_limit_ms: float,
response_limit_bytes: int,
summary: Callable[[object], Mapping[str, object]],
response_size: Callable[[object], int],
warmups: int = 0,
) -> tuple[dict[str, object], object]:
for _ in range(warmups):
operation()
durations: list[float] = []
traced_peaks: list[int] = []
response_sizes: list[int] = []
stable_summary: Mapping[str, object] | None = None
last: object = None
for _ in range(samples):
gc.collect()
tracemalloc.start()
started = time.perf_counter_ns()
try:
value = operation()
elapsed_ms = (time.perf_counter_ns() - started) / 1_000_000
_, traced_peak = tracemalloc.get_traced_memory()
finally:
tracemalloc.stop()
current_summary = dict(summary(value))
if stable_summary is None:
stable_summary = current_summary
elif current_summary != stable_summary:
raise RuntimeError("Milestone 3 operation changed deterministic result across samples")
current_response_size = response_size(value)
if current_response_size > response_limit_bytes:
raise RuntimeError(
"Milestone 3 response exceeded its fixed benchmark boundary: "
f"{current_response_size} > {response_limit_bytes}"
)
if traced_peak > MAX_TRACED_PEAK_BYTES:
raise RuntimeError(
"Milestone 3 operation exceeded its traced-memory boundary: "
f"{traced_peak} > {MAX_TRACED_PEAK_BYTES}"
)
durations.append(elapsed_ms)
traced_peaks.append(traced_peak)
response_sizes.append(current_response_size)
last = value
ordered = sorted(durations)
p95_index = max(0, math.ceil(len(ordered) * 0.95) - 1)
p95 = ordered[p95_index]
if p95 > p95_limit_ms:
raise RuntimeError(f"Milestone 3 operation p95 {p95:.3f} ms exceeds {p95_limit_ms:.3f} ms")
assert stable_summary is not None
return (
{
"samples": samples,
"median_ms": round(statistics.median(ordered), 3),
"p95_ms": round(p95, 3),
"min_ms": round(ordered[0], 3),
"max_ms": round(ordered[-1], 3),
"p95_limit_ms": p95_limit_ms,
"maximum_response_bytes": max(response_sizes),
"response_limit_bytes": response_limit_bytes,
"maximum_traced_peak_bytes": max(traced_peaks),
"traced_peak_limit_bytes": MAX_TRACED_PEAK_BYTES,
"stable_result": dict(stable_summary),
},
last,
)
def _projection_measurement(
operation: Callable[[], _ProjectionSample],
*,
samples: int,
p95_limit_ms: float,
) -> tuple[dict[str, object], _ProjectionSample]:
child_peaks: list[int] = []
def observed_operation() -> _ProjectionSample:
sample = operation()
if sample.child_peak_memory_bytes is not None:
if sample.child_peak_memory_bytes > MAX_CHILD_PEAK_BYTES:
raise RuntimeError(
"Detached worker peak memory "
f"{sample.child_peak_memory_bytes} exceeds "
f"{MAX_CHILD_PEAK_BYTES} bytes"
)
child_peaks.append(sample.child_peak_memory_bytes)
return sample
measurement, value = _measure(
observed_operation,
samples=samples,
p95_limit_ms=p95_limit_ms,
response_limit_bytes=MAX_RECEIPT_BYTES,
summary=lambda item: _projection_summary(cast(_ProjectionSample, item)),
response_size=lambda item: _compact_size(cast(_ProjectionSample, item).receipt),
)
sample = cast(_ProjectionSample, value)
if sample.plan_bytes > MAX_PLAN_BYTES or sample.package_bytes > MAX_PACKAGE_BYTES:
raise RuntimeError("Projection plan or package exceeded its protocol boundary")
if child_peaks:
measurement["maximum_child_peak_bytes"] = max(child_peaks)
measurement["child_peak_limit_bytes"] = MAX_CHILD_PEAK_BYTES
return measurement, sample
def _profiled_status(operation: Callable[[], Mapping[str, object]]) -> _StatusSample:
with request("benchmark.m3", enabled=True) as collector:
response = operation()
if collector is None:
raise RuntimeError("Milestone 3 telemetry collector was not created")
counters = collector.as_dict(outcome="ok")["counters"]
typed_counters = cast(Mapping[str, object], counters)
for counter in ZERO_WORK_COUNTERS:
if typed_counters[counter] != 0:
raise RuntimeError(f"Receipt-only status performed forbidden work: {counter}")
if typed_counters["source_generation_checks"] != 2:
raise RuntimeError("Receipt-only status did not perform its two race-safe source checks")
if response.get("status") != "ok" or response.get("state") != "current":
raise RuntimeError("Receipt-only status did not report a current publication")
return _StatusSample(response=response, counters=typed_counters)
def _status_summary(value: object) -> Mapping[str, object]:
sample = cast(_StatusSample, value)
return {
"response": dict(sample.response),
"counters": dict(sample.counters),
}
def _fragment_summary(value: object) -> Mapping[str, object]:
sample = cast(_FragmentSweep, value)
return {
"fragment_count": sample.fragment_count,
"aggregate_content_bytes": sample.aggregate_content_bytes,
"ordered_record_hash": sample.ordered_record_hash,
}
def _fragment_sweep(records: Sequence[FragmentRecord]) -> _FragmentSweep:
return _FragmentSweep(
fragment_count=len(records),
aggregate_content_bytes=sum(record.byte_count for record in records),
ordered_record_hash=projection_hash([record.record_id for record in records]),
)
def _prepared_summary(value: object) -> Mapping[str, object]:
prepared = cast(PreparedRender, value)
return {
"render_identity": prepared.render_identity,
"output_sha256": prepared.output_hash,
"output_bytes": len(prepared.output),
}
def _prepared_child_peak(prepared: PreparedRender) -> int:
receipt = prepared.projection_receipt
if not isinstance(receipt, Mapping):
raise RuntimeError("Production detached render did not return a projection receipt")
peak = receipt.get("peak_memory_bytes")
if type(peak) is not int or peak < 1:
raise RuntimeError("Production detached render did not report valid child peak memory")
if peak > MAX_CHILD_PEAK_BYTES:
raise RuntimeError(
f"Production detached worker peak memory {peak} exceeds {MAX_CHILD_PEAK_BYTES} bytes"
)
return peak
def _prepared_response_size(value: object) -> int:
prepared = cast(PreparedRender, value)
return _compact_size(prepared.projection_receipt)
def _prepared_measurement(
operation: Callable[[], PreparedRender],
*,
samples: int,
p95_limit_ms: float,
) -> tuple[dict[str, object], PreparedRender]:
child_peaks: list[int] = []
def observed_operation() -> PreparedRender:
prepared = operation()
child_peaks.append(_prepared_child_peak(prepared))
return prepared
measurement, value = _measure(
observed_operation,
samples=samples,
p95_limit_ms=p95_limit_ms,
response_limit_bytes=MAX_RECEIPT_BYTES,
summary=_prepared_summary,
response_size=_prepared_response_size,
)
measurement["maximum_child_peak_bytes"] = max(child_peaks)
measurement["child_peak_limit_bytes"] = MAX_CHILD_PEAK_BYTES
return measurement, cast(PreparedRender, value)
def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
project = Project.open(root)
snapshot = project.load()
if len(snapshot.nodes) != node_count or len(snapshot.edges) != node_count - 1:
raise RuntimeError("Milestone 3 fixture does not have full synthetic coverage")
operations: dict[str, object] = {}
def manual_full() -> _ProjectionSample:
plan, package, renderer_version = _manual_package(snapshot)
result = ManualHtmlRenderer(renderer_version).render(package)
return _projection_sample(plan, package, result)
operations["manual_full_render"], manual_result = _projection_measurement(
manual_full,
samples=samples,
p95_limit_ms=15_000,
)
def graph_full() -> _ProjectionSample:
plan, package = _graph_package(snapshot, node_count)
result = PortableGraphHtmlRenderer().render(package)
return _projection_sample(plan, package, result)
operations["portable_graph_full_render"], graph_result = _projection_measurement(
graph_full,
samples=samples,
p95_limit_ms=10_000,
)
manual_plan, manual_package, manual_renderer_version = _manual_package(snapshot)
graph_plan, graph_package = _graph_package(snapshot, node_count)
graph_diagnostics = cast(Mapping[str, object], graph_plan.document["diagnostics"])
manual_pages = cast(list[dict[str, object]], manual_plan.document["pages"])
if (
len(manual_pages) != node_count
or graph_diagnostics["returned_nodes"] != node_count
or graph_diagnostics["returned_edges"] != node_count - 1
):
raise RuntimeError("Projection plans did not retain every synthetic node and edge")
manual_worker_measurement, manual_worker = _projection_measurement(
lambda: _projection_sample(
manual_plan,
manual_package,
render_projection_in_worker(manual_package),
),
samples=samples,
p95_limit_ms=20_000,
)
operations["manual_detached_worker"] = manual_worker_measurement
manual_worker_peak = cast(
int,
manual_worker_measurement["maximum_child_peak_bytes"],
)
graph_worker_measurement, graph_worker = _projection_measurement(
lambda: _projection_sample(
graph_plan,
graph_package,
render_projection_in_worker(graph_package),
),
samples=samples,
p95_limit_ms=20_000,
)
operations["portable_graph_detached_worker"] = graph_worker_measurement
graph_worker_peak = cast(
int,
graph_worker_measurement["maximum_child_peak_bytes"],
)
if (
manual_worker.artifact != manual_result.artifact
or graph_worker.artifact != graph_result.artifact
):
raise RuntimeError("Detached worker output is not byte-equivalent to in-process output")
manual_renderer = ManualHtmlRenderer(manual_renderer_version)
fragment_records = [
FragmentRecord.create(
FragmentKey.create(
projection_kind="manual",
renderer_id=ManualHtmlRenderer.renderer_id,
renderer_version=manual_renderer_version,
component_version=GenericHtmlRenderer.page_component_version,
semantic_input_hash=fragment_semantic_hash(page),
),
manual_renderer.render_page_fragment(page).encode("utf-8"),
)
for page in manual_pages
]
fragment_cache = ProjectionFragmentCache(
root,
root / ".docforge" / "milestone3-fragment-cache",
)
def fragment_misses() -> _FragmentSweep:
if any(fragment_cache.get(record.key) is not None for record in fragment_records):
raise RuntimeError("Cold fragment-cache lookup unexpectedly hit")
return _fragment_sweep(fragment_records)
operations["fragment_cache_miss_sweep"], _ = _measure(
fragment_misses,
samples=samples,
p95_limit_ms=5_000,
response_limit_bytes=32_768,
summary=_fragment_summary,
response_size=lambda item: _compact_size(_fragment_summary(item)),
)
def fragment_puts() -> _FragmentSweep:
published: list[FragmentRecord] = []
for record in fragment_records:
stored = fragment_cache.put(record.key, record.content)
if stored != record:
raise RuntimeError("Fragment cache did not publish an exact record")
assert stored is not None
published.append(stored)
return _fragment_sweep(published)
operations["fragment_cache_put_sweep"], _ = _measure(
fragment_puts,
samples=1,
p95_limit_ms=10_000,
response_limit_bytes=32_768,
summary=_fragment_summary,
response_size=lambda item: _compact_size(_fragment_summary(item)),
)
def fragment_hits() -> _FragmentSweep:
loaded: list[FragmentRecord] = []
for expected in fragment_records:
record = fragment_cache.get(expected.key)
if record != expected:
raise RuntimeError("Fragment cache hit was not byte-exact")
assert record is not None
loaded.append(record)
return _fragment_sweep(loaded)
operations["fragment_cache_hit_sweep"], _ = _measure(
fragment_hits,
samples=samples,
p95_limit_ms=5_000,
response_limit_bytes=32_768,
summary=_fragment_summary,
response_size=lambda item: _compact_size(_fragment_summary(item)),
)
render_config = snapshot.descriptor.render
assert render_config is not None
incremental_package = build_manual_projection_package(
manual_plan,
render_config.views[0].template_path.read_bytes(),
renderer_id=ManualHtmlRenderer.renderer_id,
renderer_version=manual_renderer_version,
max_output_bytes=snapshot.descriptor.limits.max_render_bytes,
fragment_records=[record.as_dict() for record in fragment_records],
)
def fragment_equivalence() -> _ProjectionSample:
sample = _projection_sample(
manual_plan,
incremental_package,
ManualHtmlRenderer(manual_renderer_version).render(incremental_package),
)
if sample.artifact != manual_result.artifact:
raise RuntimeError("Fragment-assisted manual output is not byte-equivalent")
return sample
operations["fragment_assisted_equivalence"], fragment_result = _projection_measurement(
fragment_equivalence,
samples=samples,
p95_limit_ms=15_000,
)
render_config = snapshot.descriptor.render
assert render_config is not None
manual_view = render_config.views[0]
template_bytes = manual_view.template_path.read_bytes()
production_renderer = GenericHtmlRenderer()
operations["manual_incremental_cold"], cold_prepared = _prepared_measurement(
lambda: production_renderer.prepare(
snapshot,
manual_view,
template_bytes,
changeset_hash=None,
),
samples=1,
p95_limit_ms=20_000,
)
operations["manual_incremental_warm"], warm_prepared = _prepared_measurement(
lambda: production_renderer.prepare(
snapshot,
manual_view,
template_bytes,
changeset_hash=None,
),
samples=samples,
p95_limit_ms=20_000,
)
operations["manual_forced_full"], forced_prepared = _prepared_measurement(
lambda: GenericHtmlRenderer(incremental=False).prepare(
snapshot,
manual_view,
template_bytes,
changeset_hash=None,
),
samples=samples,
p95_limit_ms=20_000,
)
production_values = (cold_prepared, warm_prepared, forced_prepared)
if len({value.output for value in production_values}) != 1:
raise RuntimeError("Production cold, warm, and forced-full manual output differs")
changed_node = replace(
snapshot.nodes[0],
content=snapshot.nodes[0].content + "\nChanged projection content.\n",
content_hash=_sha256(
(snapshot.nodes[0].content + "\nChanged projection content.\n").encode()
),
)
added_node = replace(
snapshot.nodes[-1],
node_id="guide.synthetic-added",
title="Synthetic added",
source_path="docs/content/synthetic-added.md",
content_hash=_sha256(b"synthetic-added"),
)
variants = {
"change": replace(
snapshot,
nodes=(changed_node, *snapshot.nodes[1:]),
source_hash=_sha256(b"manual-change-variant"),
),
"add": replace(
snapshot,
nodes=(*snapshot.nodes, added_node),
source_hash=_sha256(b"manual-add-variant"),
),
"delete": replace(
snapshot,
nodes=snapshot.nodes[:-1],
edges=tuple(
edge
for edge in snapshot.edges
if edge.source_id != snapshot.nodes[-1].node_id
and edge.target_id != snapshot.nodes[-1].node_id
),
source_hash=_sha256(b"manual-delete-variant"),
),
"reorder": replace(
snapshot,
nodes=tuple(reversed(snapshot.nodes)),
source_hash=_sha256(b"manual-reorder-variant"),
),
}
variant_equivalence: dict[str, bool] = {}
variant_child_peaks: list[int] = []
for name, variant in variants.items():
incremental = GenericHtmlRenderer().prepare(
variant,
manual_view,
template_bytes,
changeset_hash=None,
)
full = GenericHtmlRenderer(incremental=False).prepare(
variant,
manual_view,
template_bytes,
changeset_hash=None,
)
variant_child_peaks.extend(
(
_prepared_child_peak(incremental),
_prepared_child_peak(full),
)
)
variant_equivalence[name] = incremental.output == full.output
if not all(variant_equivalence.values()):
raise RuntimeError("Production incremental mutation output differs from forced full")
manual_service = RenderService(project)
graph_service = GraphRenderService(project)
manual_service.render("manual")
graph_service.render("architecture")
operations["manual_status_no_work"], manual_status = _measure(
lambda: _profiled_status(lambda: manual_service.status("manual")),
samples=samples,
p95_limit_ms=500,
response_limit_bytes=MAX_STATUS_RESPONSE_BYTES,
summary=_status_summary,
response_size=lambda item: _compact_size(cast(_StatusSample, item).response),
)
operations["portable_graph_status_no_work"], graph_status = _measure(
lambda: _profiled_status(lambda: graph_service.status("architecture")),
samples=samples,
p95_limit_ms=500,
response_limit_bytes=MAX_STATUS_RESPONSE_BYTES,
summary=_status_summary,
response_size=lambda item: _compact_size(cast(_StatusSample, item).response),
)
return {
"fixture": {
"kind": "synthetic_generic_projection",
"node_count": node_count,
"edge_count": node_count - 1,
"manual_page_count": len(manual_pages),
"portable_graph_node_count": graph_diagnostics["returned_nodes"],
"portable_graph_edge_count": graph_diagnostics["returned_edges"],
"full_coverage": (
len(manual_pages) == node_count
and graph_diagnostics["returned_nodes"] == node_count
and graph_diagnostics["returned_edges"] == node_count - 1
),
},
"operations": operations,
"equivalence": {
"manual_in_process_vs_detached": manual_worker.artifact == manual_result.artifact,
"portable_graph_in_process_vs_detached": graph_worker.artifact == graph_result.artifact,
"manual_full_vs_fragment_assisted": fragment_result.artifact == manual_result.artifact,
"manual_production_cold_warm_full": len({value.output for value in production_values})
== 1,
"manual_production_variants": variant_equivalence,
},
"sizes": {
"manual": {
**_projection_summary(manual_result),
"receipt_bytes": _compact_size(manual_result.receipt),
},
"portable_graph": {
**_projection_summary(graph_result),
"receipt_bytes": _compact_size(graph_result.receipt),
},
"fragment_assisted_manual": {
**_projection_summary(fragment_result),
"receipt_bytes": _compact_size(fragment_result.receipt),
"fragment_count": len(fragment_records),
"aggregate_fragment_content_bytes": sum(
record.byte_count for record in fragment_records
),
},
"manual_status_response_bytes": _compact_size(
cast(_StatusSample, manual_status).response
),
"portable_graph_status_response_bytes": _compact_size(
cast(_StatusSample, graph_status).response
),
},
"memory": {
"process_peak_rss_kib": int(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss),
"manual_worker_peak_bytes": manual_worker_peak,
"manual_production_worker_peak_bytes": max(
cast(
int,
operations["manual_incremental_cold"]["maximum_child_peak_bytes"],
),
cast(
int,
operations["manual_incremental_warm"]["maximum_child_peak_bytes"],
),
cast(
int,
operations["manual_forced_full"]["maximum_child_peak_bytes"],
),
*variant_child_peaks,
),
"portable_graph_worker_peak_bytes": graph_worker_peak,
},
}
def main() -> int:
arguments = _parser().parse_args()
default_nodes = FULL_NODE_COUNT if arguments.mode == "full" else SMOKE_NODE_COUNT
default_samples = DEFAULT_FULL_SAMPLES if arguments.mode == "full" else DEFAULT_SMOKE_SAMPLES
node_count = default_nodes if arguments.nodes is None else arguments.nodes
samples = default_samples if arguments.samples is None else arguments.samples
if not 2 <= node_count <= FULL_NODE_COUNT:
raise SystemExit("--nodes must be between 2 and 1000")
if arguments.mode == "full" and node_count != FULL_NODE_COUNT:
raise SystemExit("--mode full requires exactly 1000 nodes")
if samples < 1:
raise SystemExit("--samples must be positive")
with tempfile.TemporaryDirectory(prefix="docforge-milestone3-") as directory:
benchmark_root = Path(directory).resolve()
_prepare_fixture(benchmark_root, node_count)
measurement = _benchmark(benchmark_root, node_count, samples)
status = _git(["status", "--porcelain"])
report: dict[str, object] = {
"schema_version": 1,
"benchmark": "docforge2_milestone3",
"mode": arguments.mode,
"source": {
"revision": _git(["rev-parse", "HEAD"]),
"dirty": bool(status),
},
"environment": {
"platform": platform.platform(),
"machine": platform.machine(),
"python": platform.python_version(),
"implementation": platform.python_implementation(),
},
"method": {
"clock": "time.perf_counter_ns",
"in_process_peak_memory": "tracemalloc per measured invocation",
"detached_peak_memory": "worker receipt resource peak RSS",
"process_peak_memory": "resource.getrusage(RUSAGE_SELF).ru_maxrss",
"response_size": "UTF-8 bytes of canonical compact sorted JSON",
"samples": samples,
"full_mode_node_requirement": FULL_NODE_COUNT,
"determinism": (
"stable semantic summaries must match across samples; report JSON uses sorted keys"
),
"maximum_worker_artifact_bytes": MAX_WORKER_ARTIFACT_BYTES,
},
**measurement,
}
encoded = encode_report(report)
if arguments.output is not None:
output = arguments.output.resolve()
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(encoded, encoding="utf-8")
sys.stdout.write(encoded)
return 0
if __name__ == "__main__":
raise SystemExit(main())