administrator/DocForge2
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

Compare commits

base: administrator:a9a75c5c274f29673b50b1d1b6d100912ea31e27
Branches Tags
administrator:main
administrator:dev
administrator:codex/cross-identity-acceptance
administrator:v2.0.0
administrator:v1.4.0
administrator:v1.0.0
...
compare: administrator:4c5773c8652aedfeab82cf87ee7da1b7bc01eddc
Branches Tags
administrator:dev
administrator:main
administrator:codex/cross-identity-acceptance
administrator:v2.0.0
administrator:v1.4.0
administrator:v1.0.0

6 commits
a9a75c5c27 ... 4c5773c865

Author SHA1 Message Date
Andraxion
4c5773c865 Close Milestone 2 with measured evidence 2026-07-29 10:22:56 -04:00
Andraxion
fb0df5e4a1 Add deterministic client integration diagnostics 2026-07-29 10:15:27 -04:00
Andraxion
eb9355b003 Make task context page packing logarithmic 2026-07-29 08:51:22 -04:00
Andraxion
9a48233983 Add bounded generation transition receipts 2026-07-29 08:23:04 -04:00
Andraxion
4cc6277054 Add versioned task context capsules 2026-07-29 07:10:18 -04:00
Andraxion
34cd5f74c1 Add versioned effective policy 2026-07-29 06:26:40 -04:00
40 changed files with 14412 additions and 153 deletions
Download patch file Download diff file Expand all files Collapse all files

9
ACTIVE_SLICE.md
View file

@ -6,7 +6,14 @@ Goal: Let one project-bound server return compact, task-shaped, explainable cont
In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks.
Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass.
Status: Active. Read-only contract audits begin from the verified Milestone 1 boundary.
Status: Complete. Effective policy, versioned task retrieval, latest-generation diff receipts,
logarithmic bounded page packing, deterministic client configuration, and the read-only integration
doctor are implemented and contract-tested. The complete repository gate passes with 205 tests and
120 subtests. Three independent adversarial audits found no remaining implementation blocker. The
clean 1,000-node baseline is recorded against candidate commit
`fb0df5e4a1c591c2a84788fd4814d98550f11863`, including task/generation reconstruction,
response-size behavior, zero-hidden-work counters, and isolated memory. No tag or release was
created, no production integration was repointed, and self-hosting remains out of scope.
```
Milestones 3–5 remain directional context and are not active.

266
DEVELOPMENT_NOTES.md
View file

@ -344,3 +344,269 @@ These are notes, not commitments:
- Cursor authentication remains deliberately absent. If read cursors ever carry authority rather
than bounded positions, they will need a different versioned security contract and persisted key
lifecycle.
## Milestone 2 — active: agent retrieval and MCP experience
### Audit reconciliation
Three independent read-only audits covered effective policy and bootstrap, task-shaped retrieval
and context capsules, and generation diffs plus client configuration and doctor checks.
They agreed on these boundaries:
- Keep the project descriptor at schema version 1. Process capability and client configuration are
machine-specific bindings, not canonical project content.
- Preserve the legacy adapter-policy payload, no-AST shorthand, tool names, default tool ordering,
one-method adapters, and custom context provider.
- Add one versioned effective-policy authority and derive bootstrap, contract, instructions, and
access reporting from it.
- Add one task-context operation with a closed task-kind vocabulary and one immutable,
generation-pinned retrieval plan. Do not create a tool for every task kind.
- Produce evidence gaps only from declared plan requirements and completed bounded checks. Never
infer missing facts from arbitrary project naming.
- Record only the latest bounded generation transition as disposable evidence. Do not add a
history database.
- Preview client configuration by default. Any write must be explicit, atomic, merge-preserving,
and backed by a verified client-format driver.
- Keep doctor strictly read-only. It must not bootstrap, synchronize, build, render, start a
viewer, or rewrite client configuration.
### Versioned effective policy and session contract
The binding now composes an immutable version-1 policy containing capability mode, adapter
evolution, AST and Logic behavior, synchronization and integrity levels, render and viewer
behavior, profiling, blocked tools, prohibitions, and explicit precedence. `--no-ast` is a
restrictive override. The exact legacy `adapter_policy` response remains a projection of the new
object.
Bootstrap reuses the identity already proven by synchronization and no longer reloads the complete
project. Its additive version-1 session contract reports binding, generation, effective policy,
actual registered surfaces and mutation access, render policies, first operation, filtered
workflow, and prohibitions. Read mode does not recommend proposals. Proposal mode recommends
registration and review only with writer access. Application is recommended only when the
exact-hash applier is enabled.
Existing factory defaults and tool order remain unchanged. Explicit application mode fails closed
without an applier. Operator mode is reserved and currently adds no tools.
### Versioned task retrieval and context capsules
The first Milestone 2 retrieval slice adds one `docforge_get_task_context` read tool rather than a
family of task-specific tools. Its closed task kinds are change, implementation, failure,
ownership, test, operation, and release. One immutable `RetrievalPlanV1` derives exact or lexical
focus, bounded bidirectional graph traversal, metadata hydration, required evidence categories,
and fixed work budgets from the project descriptor and effective policy.
The public executor re-derives every submitted plan before opening SQLite. It rejects modified
steps, task identity, requirements, category order, bounds, policy identity, or hashes as
`invalid_retrieval_plan`. Traversal binds the project relation set by canonical hash and queries
the already-validated edge table by endpoint, avoiding relation-sized SQL parameter lists.
Version-1 internal ceilings are 1,000 evidence items, 100,000 candidate edges, and 10,000 task
query characters.
The executor uses one immutable SQLite read generation. It rejects missing explicit focus, blocks
unresolved or tied lexical focus, stops at deterministic evidence and candidate-edge limits, and
checks source identity again when the transaction closes. `ContextCapsuleV1` binds the generation,
policy, request, plan, evidence collection, and complete capsule with canonical hashes.
Project relation names remain authoritative. The core recognizes only a versioned alias map for
structure, implementation, dependency, execution, data, evidence, and context. Unknown allowed
relations stay visible under their raw names as `unclassified`. Required evidence diagnostics
distinguish categories the project never declared, completed bounded checks with no selected
evidence, and incomplete checks caused by a result, work, token, or response limit.
Each evidence item carries a stable content hash, confined source identity, shortest selected graph
path, every additional qualifying relationship reason observed during traversal, and explicit
limitations where the current graph cannot prove evidence type, extractor identity, relationship
source provenance, or observation time. The planner contains no Logic operation, so no-AST
bindings can use task context without weakening their existing Logic prohibition.
Path direction is relative to the preceding traversal node. Additional relationship reasons use
the evidence node as their direction subject. Candidate-edge and unclassified-relation ceilings
produce explicit omissions and bounded summaries.
MCP pagination preserves the complete plan, collection, and capsule hashes while returning bounded
pages. Its cursor additionally binds the effective policy and task request. An individually
oversized item advances once as a hash-identified omission. A later generation or policy change
fails closed as `stale_cursor`.
The legacy profile-context contract remains intact. A custom context provider does not silently
gain core task planning. Version 1 defines no custom task-planner extension, so the additive tool
returns `task_context_unavailable` without synchronization or a complete projection load.
Two independent pre-commit audits reproduced and closed plan-forgery, relation-sized SQL,
SQLite-parameter portability, ambiguous relationship-direction, missing work-limit evidence,
schema/runtime drift, incomplete page hashing, and custom-provider hidden-load defects. Regression
coverage includes 33,005 valid relation names, fixed extreme project limits, tampered plans,
evidence-relative directions, edge and unclassified limits, schema-valid pages, changed cursor
semantics, oversized evidence advancement, no-AST retrieval, and legacy complete-projection
adapters.
The complete repository gate passes with 158 tests and 101 subtests, zero Pyright diagnostics,
warning-strict execution, package builds, public-contract validation, and the maintained Milestone
0 and Milestone 1 smoke benchmarks. Gitleaks 8.30.1 reports no secret findings in the working tree.
### Latest-generation diff receipt
Three read-only audits reconciled the index publication, public transport, compatibility, and
no-AST boundaries before implementation. The selected design stores one disposable
`generation-diff.json` receipt. It does not add a history database, arbitrary generation
selectors, source text, rendered content, or Logic details.
Before a build loads current source, it accepts an existing index only when its exact main-file
inode has a matching stable whole-file attestation and no WAL, journal, or shared-memory sidecar.
It then captures that predecessor through an immutable main-file transaction. The capture validates
the SQLite application and schema IDs, project/root/adapter binding, integrity, complete node and
edge rows, Logic aggregate identity, FTS count, metadata hashes and counts, and final file
signature. It never calls normal check or synchronization and never repairs predecessor evidence.
The final source revalidation now compares exact nodes and edges in addition to source hash,
revision, and Logic. A verified predecessor that maps the same source identity to different graph
content fails before publication as `generation_collision`. This closes a pre-existing adapter
determinism gap found during the generation-diff audit.
SQLite replacement is now the explicit derived mutation commit point. Whole-file attestation,
cheap source-generation, and generation-diff receipts publish independently afterward. Any
post-commit receipt failure returns `status = ok`, `index = published`, a bounded degraded
publication record, and receipt-stage names. It never rolls back the new index or reports a false failed
mutation. Attestation hashing checks the exact index signature before, during, and immediately
before receipt publication.
Version-1 diff semantics compare every core `Node` field by stable node ID and exact edge triples.
Node renames are removal plus addition. Edge changes are removal plus addition. Exact summary
counts and a full ordered item-hash collection cover every change. Retained details are
deterministically ordered and independently capped at 1,000 items and 1 MiB with explicit item- or
byte-limit evidence. A first build or untrusted predecessor is a baseline with no fabricated
all-added result. A same-generation reindex republishes the existing meaningful transition against
the new index file identity instead of erasing it with an empty diff.
The additive public surfaces are:
- CLI `generation-diff [--limit N] [--cursor OPAQUE]`.
- MCP `docforge_get_generation_diff(limit=None, cursor=None)`.
- Telemetry operations `cli.generation-diff` and `mcp.generation_diff`.
Public reads do not open SQLite, call `project.load()`, extract an adapter projection, parse source,
check, synchronize, build, or repair. They strictly validate the bounded receipt, compare stable
receipt and index file identities, require two matching cheap source-generation checks, and report
unknown for legacy adapters without that capability. Missing, corrupt, foreign, oversized,
symlinked, stale, or concurrently changed evidence remains a read-only status outcome.
Generation-diff pagination binds the complete stored receipt hash and effective policy. That hash
already covers project, generation, graph, collection, and committed-index identity. Page size may
change. A replaced receipt returns `stale_cursor`. One top-level pagination object owns the only
cursor. The nested version-1 page uses `receipt_header.stored_receipt_hash` so it never
misrepresents the complete receipt hash as the hash of a partial header. The summary distinguishes
additional retained pages from details permanently omitted by the fixed publication limits.
Adversarial coverage now includes strict runtime/schema rejection, predecessor attestation and
generation identity, live and synthetic SQLite sidecars, cache-root symlink substitution,
source/sidecar changes during diff preparation, independent receipt failures, and degraded
post-commit identity and durability failures. Focused verification passes the direct, CLI, MCP,
schema, pagination, legacy, incremental no-AST, and zero-work suites. The complete repository gate
passes with 176 tests and 113 subtests, zero Pyright diagnostics, package builds, web checks, and
the maintained Milestone 0 and Milestone 1 smoke benchmarks. Final independent re-audit is in
progress before this slice is committed.
The final dense benchmark uses a 1,000-node transition with 1,000 changed details. Its receipt is
775,663 bytes. Direct status is 37.659 ms median and 39.108 ms p95. A maximum-size MCP request
returns 307 items in 199,754 bytes at 54.037 ms median and 56.617 ms p95. Four pages reconstruct
all 1,000 retained details in 652,798 bytes at 201.55 ms median. Peak RSS is 79,096 KiB.
Every hidden-work counter remains zero; the read performs exactly two cheap source-generation
checks.
Measurement found and removed two avoidable costs before commit. Receipt loading had repeated the
complete 1,000-item validator solely to check project identity; it now validates once and compares
the three binding fields directly. Page fitting had encoded every growing prefix; it now uses an
exact logarithmic search and retains the hash-only oversized-item omission path. The maximum page
fell from 272.06 ms p95 to 56.617 ms p95, while full traversal fell from roughly 859 ms to
203.55 ms p95. Regression tests require one receipt validation and at most 15 response encodes for
1,000 page candidates. Final independent publication, contract, and performance audits approve
the slice for commit.
### Deterministic client configuration and read-only doctor
Client integration remains an explicit machine-local boundary rather than canonical project
content. `docforge configure {codex,claude,openclaw} --project PATH` previews a deterministic
version-1 fragment by default. An optional output path publishes only a standalone fragment into
an existing real directory. Publication is create-only, private-mode, no-follow, bounded, and
conflict-aware. Existing differing client configuration is never merged, replaced, or silently
overwritten.
Generated commands use the exact current virtual-environment Python executable with isolated
module startup. The binding records explicit read, proposal, or application mode, no-AST policy,
render policy, empty environment, and bounded timeouts. Proposal and application generation fail
closed unless the descriptor declares the required writer and matching applier identity. A generic
CLI cannot reconstruct project-owned adapter composition, so custom adapters return an explicit
unavailable result instead of generating a misleading command.
Codex and OpenClaw fragments include their verified timeout fields. Claude JSON fragment syntax is
supported, while its timeout representation remains an explicit warning. The configuration result
has a strict JSON schema and canonical plan hash. Diagnostics are additive and remain disabled by
default.
`docforge doctor --client CLIENT` performs bounded, non-mutating inspection only. It reads the
project descriptor and selected client file through stable, directory-bound, no-follow handles;
parses at most 1 MiB and 256 server entries; selects at most one exact project binding; validates
the closed server argument set; checks executable, capability, declared authority, no-AST,
timeouts, environment-key names, and tool-filter presence; and performs only a stat-level index
presence check. It never loads canonical sources, opens SQLite, starts MCP, executes the configured
command, synchronizes, builds, renders, starts a viewer, or writes client configuration.
Doctor reports healthy, degraded, or unhealthy with stable process exit codes 0, 1, and 2. Secret
environment values are parsed only to enforce bounded string limits and are never returned.
Unknown or unverified client tool filtering, Claude timeout representation, implicit legacy
capability mode, shadowed authority, and missing disposable indexes are warnings. Unsafe paths,
malformed matching entries, unexpected executables, wrong project roots, invalid authorities, and
missing configuration are failures.
The first benchmark smoke failed for the correct product reason: its disposable doctor fragment
used the shared path `/tmp/doctor-codex.toml`, where a previous run had left different content. The
harness now creates a project subdirectory inside one unique temporary root and places the client
fragment beside it. This preserves create-only conflict safety and makes every run disposable.
The final pre-commit 1,000-node audit sample passes every provisional Milestone 2 gate. Task
context reconstructs 1,000 candidates as 108 cited evidence records and 892 explicit bounded
omissions across 11 pages in 703.808 ms. Generation diff reconstructs 1,000 changed details across
10 pages in 427.450 ms. Maximum pages remain below the 200,000-byte MCP budget; generation diff
uses 199,566 bytes and proves that diagnostics are discarded before the primary result. Isolated
peak RSS is 86,168 KiB.
Configuration preview now includes a bounded real import probe of the exact isolated interpreter,
so its provisional single-sample latency is about 315 ms rather than the earlier sub-millisecond
derivation-only figure. Doctor remains below 1 ms on generated disposable configurations.
Every configuration and doctor hidden-work counter is zero.
The aggregate `make gate` includes the Milestone 2 smoke benchmark. The frozen candidate passes
205 tests and 120 schema subtests, strict warnings, Ruff, formatting, Pyright, web checks,
compilation, lock and dependency checks, package builds, and all three milestone smoke benchmarks.
Three independent final audits approve client publication and policy binding, doctor fail-closed
behavior, and benchmark/contract coverage. Clean-revision benchmark evidence is still required
before closeout.
### Milestone 2 closeout
Candidate commit `fb0df5e4a1c591c2a84788fd4814d98550f11863` passed the clean ten-sample
Milestone 2 benchmark. Task-context complete traversal measured 703.561 ms median and 721.847 ms
p95 across 11 bounded pages. It reconstructed the exact 1,000-candidate collection from 108 cited
evidence records, 891 original token-budget omissions, and one hash-attested response-limit
surrogate. Generation-diff complete traversal measured 418.607 ms median and 425.315 ms p95 across
10 pages.
Read and no-AST bootstrap remained below 10 ms p95. The maximum generation page used 199,566 bytes
of the 200,000-byte budget and correctly discarded diagnostics before primary evidence.
Configuration preview measured about 314 ms median and 365 ms p95 because it proves the real
isolated interpreter import on every invocation. Codex and OpenClaw doctor checks remained below
0.6 ms p95; Claude remained explicitly degraded because its timeout format is unverified.
Isolated-process peak RSS was 86,448 KiB against the 262,144 KiB gate.
All measured configuration and doctor counters were zero. Task-context pages performed one index
check and two cheap generation checks with no loads, parses, synchronization, builds, extraction,
rendering, or viewer work. Generation-diff pages performed two cheap generation checks and no
index check. The canonical machine-readable result is
`benchmarks/milestone2-2026-07-29.json`.
Milestone 2 is complete. Follow-up ideas stay explicitly later-scope: avoid recomputing the
task-shaped capsule for every continuation page, add authenticated continuation when the threat
model requires it, verify a native Claude timeout representation, and introduce adapter-owned
launcher metadata before generating configurations for custom adapters.

16
Makefile
View file

@ -5,7 +5,7 @@ NPM := npm
PYTHONPYCACHEPREFIX := /tmp/docforge-quality-pycache
PYTEST_BASETEMP := /tmp/docforge-quality-pytest
.PHONY: benchmark benchmark-m1 benchmark-m1-smoke benchmark-smoke build compile contract dependencies format-check gate lint lock test type
.PHONY: benchmark benchmark-m1 benchmark-m1-smoke benchmark-m2 benchmark-m2-smoke benchmark-smoke build compile contract dependencies format-check gate lint lock test type
format-check:
$(PYTHON) -m ruff format --check src tests tools
@ -24,6 +24,11 @@ contract:
PYTHONPYCACHEPREFIX=$(PYTHONPYCACHEPREFIX) $(PYTHON) -m pytest -q \
-p no:cacheprovider --basetemp=$(PYTEST_BASETEMP) \
tests/test_public_contract.py \
tests/test_policy.py \
tests/test_retrieval.py \
tests/test_generation_diff.py \
tests/test_client_integration.py \
tests/test_observability.py::TelemetryContractTests::test_schema_fixed_names_match_the_implementation \
tests/test_adapter_contract.py::AdapterContractTests::test_no_ast_index_policy_rejects_logic_publication \
tests/test_adapter_contract.py::AdapterContractTests::test_no_ast_index_accepts_legacy_and_non_logic_incremental_adapters \
tests/test_adapter_contract.py::AdapterContractTests::test_no_ast_rejects_preexisting_logic_index_and_viewer_snapshot \
@ -56,4 +61,11 @@ benchmark-m1-smoke:
benchmark-m1:
$(PYTHON) tools/milestone1_benchmark.py --nodes 1000 --samples 10
gate: format-check lint type compile contract test lock dependencies build benchmark-smoke benchmark-m1-smoke
benchmark-m2-smoke:
$(PYTHON) tools/milestone2_benchmark.py --nodes 25 --samples 1 \
--output /tmp/docforge-milestone2-smoke.json > /dev/null
benchmark-m2:
$(PYTHON) tools/milestone2_benchmark.py --nodes 1000 --samples 10
gate: format-check lint type compile contract test lock dependencies build benchmark-smoke benchmark-m1-smoke benchmark-m2-smoke

41
README.md
View file

@ -9,6 +9,14 @@ declared manuals, visualizes project structure, and manages reviewable documenta
- Validates stable Markdown/TOML nodes and typed relationships.
- Builds a deterministic SQLite search and graph index.
- Exposes project-bound CLI and MCP query surfaces.
- Compiles versioned, generation-bound task context with cited evidence, explicit gaps, and bounded
continuation.
- Records one bounded, versioned latest-generation graph transition without creating a history
database.
- Generates deterministic project-bound Codex, Claude, and OpenClaw client fragments without
copying ambient secrets.
- Diagnoses one client binding through bounded read-only checks without starting MCP or rebuilding
project state.
- Automatically synchronizes disposable indexes before MCP work.
- Creates, validates, diffs, and previews isolated changesets.
- Registers complete proposals atomically without caller-managed hash chaining.
@ -48,6 +56,23 @@ AST, Tree-sitter, compiler-AST, or function-Logic extraction, blocks the Logic t
nonempty Logic publication. Complete-projection adapters continue unchanged, and non-AST
incremental fingerprinting and caching remain allowed.
DocForge2 bindings may also declare
`--capability-mode read|proposal|application|operator`. Bootstrap returns one versioned effective
policy and the actual startup-gated capabilities. Existing tool surfaces and the legacy no-AST
payload remain compatible.
`docforge_get_task_context` is an additive read tool for `change`, `implementation`, `failure`,
`ownership`, `test`, `operation`, and `release` work. It derives a closed version-1 retrieval plan,
executes it against one immutable index generation, and returns a hash-bound context capsule.
Project relation names remain authoritative. DocForge classifies only its versioned alias set and
preserves every unknown relation as `unclassified` instead of guessing semantics.
`docforge_get_generation_diff` reports the latest verified primary-graph transition through one
bounded disposable receipt. It includes exact node and edge change counts, hash-bound retained
details, and explicit truncation. Paged results use one top-level cursor and a versioned
`receipt_header`; `stored_receipt_hash` identifies the complete persisted receipt. The read never
exposes Logic details, loads canonical source, repairs derived state, or invents history.
## Graph views
The browser presents the primary architecture graph through three complementary views and loads a
@ -114,6 +139,16 @@ Start an MCP server for one project:
Add `--canonical-applier project-editor` only when that MCP integration should expose the
hash-bound `docforge_apply_changeset` tool.
Preview a read-only Codex fragment and diagnose an installed binding:
```bash
.venv/bin/docforge configure codex --project "$PROJECT"
.venv/bin/docforge doctor --client codex --project "$PROJECT"
```
Pass `--output /absolute/path/docforge.toml` only when creating a standalone fragment. DocForge
never replaces or merges an existing different client file.
For an unconfigured codebase, begin with a read-only language and documentation assessment:
```bash
@ -135,6 +170,10 @@ DocForge describes them as a source graph.
performance, memory, rendering and response sizes, bottlenecks, and missing coverage.
- [Milestone 0 closeout](docs/MILESTONE_0_CLOSEOUT.md) — lineage, migration, security scan,
repository state, and fresh-clone proof.
- [Milestone 2 baseline](docs/MILESTONE_2_BASELINE.md) — task context, generation diff, client
configuration, doctor, response-size, counter, and memory measurements.
- [Milestone 2 closeout](docs/MILESTONE_2_CLOSEOUT.md) — implemented contracts, adversarial
validation, exclusions, and exact candidate evidence.
- [MCP contract](docs/MCP_CONTRACT.md) — exact tool and process boundary.
- [Viewer manager](docs/VIEWER_MANAGER.md) — native service setup and lifecycle.
- [Adapter decision](docs/APPLICATION_DECISION.md) — why custom adapters own canonical
@ -157,7 +196,7 @@ make gate
Focused entry points are available as `make contract`, `make test`, `make type`,
`make benchmark-smoke`, `make benchmark`, `make benchmark-m1-smoke`, and
`make benchmark-m1`.
`make benchmark-m1`. Milestone 2 adds `make benchmark-m2-smoke` and `make benchmark-m2`.
The committed 1,000-node baseline and its measurement method are under `benchmarks/`.

14
benchmarks/README.md
View file

@ -27,6 +27,13 @@ Run the maintained 1,000-node Milestone 1 benchmark:
make benchmark-m1
```
Run the Milestone 2 agent-workflow smoke and full gates:
```bash
make benchmark-m2-smoke
make benchmark-m2
```
The benchmark creates canonical sources, derived state, changesets, rendered output, and caches
only in a disposable temporary directory. It does not read another project, self-host DocForge, or
mutate repository content.
@ -42,6 +49,13 @@ harness enforces operation-specific p95 ceilings and fixed zero-work counter inv
human-readable interpretation is in
[`docs/MILESTONE_1_BASELINE.md`](../docs/MILESTONE_1_BASELINE.md).
`milestone2-2026-07-29.json` is the clean-tree agent-retrieval and client-integration baseline
captured from commit `fb0df5e4a1c591c2a84788fd4814d98550f11863`. It gates every warmup and
sample, reconstructs complete task-context and generation-diff collections across bounded pages,
records whether diagnostics were dropped for response budget, checks all hidden-work counters,
and measures isolated-process peak RSS. Its interpretation is in
[`docs/MILESTONE_2_BASELINE.md`](../docs/MILESTONE_2_BASELINE.md).
The generic fixture exposes whole-source scaling. It does not replace the incremental adapter
equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has
no portable graph-planning or graph-rendering contract.

457
benchmarks/milestone2-2026-07-29.json Normal file
View file

@ -0,0 +1,457 @@
{
"benchmark": "docforge2_milestone2",
"environment": {
"implementation": "CPython",
"machine": "x86_64",
"platform": "Linux-7.1.3-200.nobara.fc44.x86_64-x86_64-with-glibc2.43",
"python": "3.14.6"
},
"fixture": {
"edge_count": 999,
"kind": "synthetic_generic_focus_fan_in",
"max_tool_output_chars": 200000,
"node_count": 1000,
"source_file_count": 1000
},
"isolated_process_peak_rss_kib": 86448,
"method": {
"clock": "time.perf_counter_ns",
"memory": "isolated child-process resource.getrusage(RUSAGE_SELF).ru_maxrss",
"memory_limit_kib": 262144,
"memory_probe_samples": 10,
"percentile": "nearest-rank",
"response_size": "UTF-8 bytes of compact sorted JSON",
"samples": 10,
"warmups": 1,
"zero_work_counters": [
"project_loads",
"source_files_parsed",
"source_bytes_parsed",
"adapter_projection_loads",
"adapter_source_extractions",
"index_synchronizations",
"index_builds",
"render_prepare_calls",
"render_output_bytes_built",
"render_output_bytes_hashed",
"viewer_manager_requests"
]
},
"operations": {
"bootstrap_no_ast": {
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 1, "minimum": 1},
"index_synchronizations": {"maximum": 1, "minimum": 1},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 1, "minimum": 1},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"max_ms": 9.379,
"maximum_response_bytes": 8796,
"median_ms": 9.153,
"min_ms": 9.026,
"p95_limit_ms": 100,
"p95_ms": 9.379,
"response_bytes": 8795,
"response_limit_bytes": 32768,
"samples": 10,
"validated_invocations": 11
},
"bootstrap_read": {
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 1, "minimum": 1},
"index_synchronizations": {"maximum": 1, "minimum": 1},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 1, "minimum": 1},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"max_ms": 9.884,
"maximum_response_bytes": 7495,
"median_ms": 9.406,
"min_ms": 9.053,
"p95_limit_ms": 100,
"p95_ms": 9.884,
"response_bytes": 7491,
"response_limit_bytes": 32768,
"samples": 10,
"validated_invocations": 11
},
"configuration_preview": {
"claude": {
"artifact_format": "claude-json-fragment-v1",
"configuration_hash": "4dbb4ed0f38264fdba350de8904cc898493d620194b5105b7889c54bd5913c9c",
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 0, "minimum": 0},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 0, "minimum": 0},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"max_ms": 364.365,
"maximum_response_bytes": 2748,
"median_ms": 314.326,
"min_ms": 314.278,
"p95_limit_ms": 500,
"p95_ms": 364.365,
"response_bytes": 2748,
"response_limit_bytes": 32768,
"samples": 10,
"validated_invocations": 11
},
"codex": {
"artifact_format": "codex-toml-fragment-v1",
"configuration_hash": "3e1dd5191021da1778cd1c4f4658768537775e5e16252e42d4c80e328841145b",
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 0, "minimum": 0},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 0, "minimum": 0},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"max_ms": 364.383,
"maximum_response_bytes": 2627,
"median_ms": 314.365,
"min_ms": 314.248,
"p95_limit_ms": 500,
"p95_ms": 364.383,
"response_bytes": 2627,
"response_limit_bytes": 32768,
"samples": 10,
"validated_invocations": 11
},
"openclaw": {
"artifact_format": "openclaw-json-fragment-v1",
"configuration_hash": "6f269e90a55088c5d517f91761c53a3b90036d62fd80b5b9d094668267257b99",
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 0, "minimum": 0},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 0, "minimum": 0},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"max_ms": 364.532,
"maximum_response_bytes": 2869,
"median_ms": 314.401,
"min_ms": 314.251,
"p95_limit_ms": 500,
"p95_ms": 364.532,
"response_bytes": 2869,
"response_limit_bytes": 32768,
"samples": 10,
"validated_invocations": 11
}
},
"doctor": {
"claude": {
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 0, "minimum": 0},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 0, "minimum": 0},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"doctor_state": "degraded",
"max_ms": 0.446,
"maximum_response_bytes": 3669,
"median_ms": 0.364,
"min_ms": 0.352,
"p95_limit_ms": 100,
"p95_ms": 0.446,
"response_bytes": 3669,
"response_limit_bytes": 32768,
"samples": 10,
"summary": {"failed": 0, "passed": 11, "skipped": 1, "warning": 2},
"validated_invocations": 11
},
"codex": {
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 0, "minimum": 0},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 0, "minimum": 0},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"doctor_state": "healthy",
"max_ms": 0.556,
"maximum_response_bytes": 3595,
"median_ms": 0.421,
"min_ms": 0.404,
"p95_limit_ms": 100,
"p95_ms": 0.556,
"response_bytes": 3595,
"response_limit_bytes": 32768,
"samples": 10,
"summary": {"failed": 0, "passed": 13, "skipped": 1, "warning": 0},
"validated_invocations": 11
},
"openclaw": {
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 0, "minimum": 0},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 0, "minimum": 0},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"doctor_state": "healthy",
"max_ms": 0.484,
"maximum_response_bytes": 3602,
"median_ms": 0.384,
"min_ms": 0.353,
"p95_limit_ms": 100,
"p95_ms": 0.484,
"response_bytes": 3602,
"response_limit_bytes": 32768,
"samples": 10,
"summary": {"failed": 0, "passed": 13, "skipped": 1, "warning": 0},
"validated_invocations": 11
}
},
"generation_diff_complete": {
"max_ms": 425.315,
"maximum_response_bytes": 984,
"median_ms": 418.607,
"min_ms": 410.974,
"p95_limit_ms": 500,
"p95_ms": 425.315,
"response_bytes": 983,
"response_limit_bytes": 32768,
"result_summary": {
"aggregate_page_bytes": 664715,
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 0, "minimum": 0},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 2, "minimum": 2},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"elapsed_ms": 419.04,
"item_count": 1000,
"maximum_cursor_bytes": 448,
"maximum_page_bytes": 66516,
"ordered_item_hash": "1ac48cc72532809ef5d3e949756e536eec819f348eaf06338c9b39b14e63b2c7",
"page_count": 10,
"receipt_hash": "6913c962972d8255f56966e5cfab5ac8293e41bdfb5f39ef91a4f33a3b092f88",
"status": "ok"
},
"samples": 10,
"validated_invocations": 11
},
"generation_diff_diagnostic_page": {
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 0, "minimum": 0},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 2, "minimum": 2},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"max_ms": 41.65,
"maximum_response_bytes": 66516,
"median_ms": 40.961,
"min_ms": 40.239,
"p95_limit_ms": 100,
"p95_ms": 41.65,
"response_bytes": 66516,
"response_limit_bytes": 200000,
"samples": 10,
"validated_invocations": 11
},
"generation_diff_maximum_page": {
"diagnostics_dropped_for_budget": true,
"max_ms": 59.184,
"maximum_response_bytes": 199566,
"median_ms": 55.565,
"min_ms": 54.68,
"p95_limit_ms": 100,
"p95_ms": 59.184,
"response_bytes": 199566,
"response_limit_bytes": 200000,
"samples": 10,
"validated_invocations": 11
},
"task_context_complete": {
"max_ms": 721.847,
"maximum_response_bytes": 1325,
"median_ms": 703.561,
"min_ms": 688.975,
"p95_limit_ms": 2500,
"p95_ms": 721.847,
"response_bytes": 1324,
"response_limit_bytes": 32768,
"result_summary": {
"aggregate_page_bytes": 348845,
"capsule_hash": "20663ed685a255f7cb8a0e8d78262bf7bf26863d728f0159e0b29000f6a52b0a",
"collection_hash": "9dbc46eb5b1ac8c8340bb149205d3599fffaf67e3b363de2d035475da274857c",
"collection_hash_reconstructed": true,
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 1, "minimum": 1},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 2, "minimum": 2},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"elapsed_ms": 719.55,
"evidence_count": 108,
"item_count": 1000,
"maximum_cursor_bytes": 1066,
"maximum_page_bytes": 151172,
"omission_count": 892,
"ordered_candidate_hash": "bdeb3a8f4018000f72a5ff1891aa800b6edf98070b9814f443c6bac4e52c38f3",
"ordered_evidence_hash": "ec07bac7f528f5a3afbc083ea5ad60541335c8789d42980fe7bae0476e0a5331",
"page_count": 11,
"plan_hash": "84dbe60267e5f8359adcf30d4d395f1ac5d5bea957beaa3888d2938120860a9c",
"status": "ok"
},
"samples": 10,
"validated_invocations": 11
},
"task_context_diagnostic_page": {
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 1, "minimum": 1},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 2, "minimum": 2},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"max_ms": 93.115,
"maximum_response_bytes": 7190,
"median_ms": 73.326,
"min_ms": 72.264,
"p95_limit_ms": 500,
"p95_ms": 93.115,
"response_bytes": 7190,
"response_limit_bytes": 200000,
"samples": 10,
"validated_invocations": 11
},
"task_context_maximum_page": {
"counter_ranges": {
"adapter_projection_loads": {"maximum": 0, "minimum": 0},
"adapter_source_extractions": {"maximum": 0, "minimum": 0},
"index_builds": {"maximum": 0, "minimum": 0},
"index_checks": {"maximum": 1, "minimum": 1},
"index_synchronizations": {"maximum": 0, "minimum": 0},
"project_loads": {"maximum": 0, "minimum": 0},
"render_output_bytes_built": {"maximum": 0, "minimum": 0},
"render_output_bytes_hashed": {"maximum": 0, "minimum": 0},
"render_prepare_calls": {"maximum": 0, "minimum": 0},
"source_bytes_parsed": {"maximum": 0, "minimum": 0},
"source_files_parsed": {"maximum": 0, "minimum": 0},
"source_generation_checks": {"maximum": 2, "minimum": 2},
"viewer_manager_requests": {"maximum": 0, "minimum": 0}
},
"diagnostics_dropped_for_budget": false,
"max_ms": 87.372,
"maximum_response_bytes": 7192,
"median_ms": 84.933,
"min_ms": 83.652,
"p95_limit_ms": 500,
"p95_ms": 87.372,
"response_bytes": 7192,
"response_limit_bytes": 200000,
"samples": 10,
"validated_invocations": 11
}
},
"process_peak_rss_kib": 86072,
"schema_version": 1,
"source": {
"dirty": false,
"revision": "fb0df5e4a1c591c2a84788fd4814d98550f11863"
}
}

59
docs/COMPATIBILITY.md
View file

@ -35,6 +35,7 @@ remain supported:
- `docforge.index`
- `docforge.mcp_server`
- `docforge.models`
- `docforge.policy`
- `docforge.render_contract`
Names beginning with an underscore are implementation details. New public names may be added
@ -59,6 +60,17 @@ MCP results retain:
The result schema describes the common envelope. Operation-specific fields are additive and remain
bounded by the configured tool-output limit.
The following Milestone 2 CLI additions do not change existing command signatures:
- `docforge configure codex|claude|openclaw --project ROOT`
- `docforge doctor --client codex|claude|openclaw`
Configuration output is a new version-1 machine-local contract. It preserves the `docforge`
package and executable names and emits the existing `docforge.mcp_server` module entrypoint.
Existing hand-written client configurations remain valid and are never rewritten automatically.
Doctor is inspection-only and does not become a hidden bootstrap, synchronization, or migration
path.
## Versioned data contracts
Milestone 0 preserves:
@ -72,10 +84,18 @@ Milestone 0 preserves:
version 3 adds a source-ordered incoming-edge index for bounded impact traversal.
- Index-attestation schema version 1.
- Incremental extraction-cache schema version 1.
- Effective process-policy schema version 1. The project descriptor remains schema version 1;
machine-specific capability selection is a startup binding, not canonical project content.
- Read-pagination schema version 1. Existing tool names and required arguments are unchanged.
Context and changeset MCP reads accept optional limits and opaque generation-bound cursors.
Direct Python changeset methods and the ordinary CLI context command retain full legacy results
when pagination is not requested.
- Latest-generation-diff receipt schema version 1. The additive `generation-diff` CLI command and
`docforge_get_generation_diff` MCP read accept only optional pagination fields. They record one
primary-graph transition and do not create a history store or expose Logic details.
- Latest-generation-diff page schema version 1. Pages use one top-level pagination object and a
nested `receipt_header`. `stored_receipt_hash` names the complete stored receipt. Opaque cursors
may be restarted after a server or receipt change and are not durable public identifiers.
Indexes, attestations, extraction caches, previews, and rendered artifacts are disposable. A schema
change may rebuild them. Canonical project content and stored proposals may not be silently
@ -96,6 +116,10 @@ must retain `load_projection()` as their independent clean-build and equivalence
Project adapters remain explicitly composed. Generic DocForge does not discover arbitrary adapter
modules or choose a project globally.
The supported generation-diff Python boundary is `ProjectIndex.generation_diff()`. Helpers in the
`docforge.generation_diff` module implement the disposable publication contract and are internal;
they are not frozen as adapter-authoring imports.
## Preserved no-AST binding
`docforge-mcp --project-root /project --no-ast` is a stable shorthand for the
@ -112,6 +136,10 @@ The binding:
- Applies the same restriction during hash-bound canonical-application refresh.
- Reports the effective policy through bootstrap and contract results.
The legacy `adapter_policy` payload and error codes remain unchanged. The version-1
`effective_policy` is additive and makes precedence, capability mode, render behavior, blocked
tools, and prohibitions machine-readable.
DocForge does not inspect arbitrary adapter source to prove which parser implementation it uses.
The no-AST binding is an owner-selected process policy backed by Logic publication and retrieval
enforcement. It is not a filesystem sandbox and cannot stop an unrelated process with repository
@ -140,6 +168,37 @@ and side-effect-free status.
The live graph viewer remains a read-only consumer of a generation-pinned validated index. It does
not become project authority or MCP retrieval authority.
## Task-context compatibility
`docforge_get_task_context` is an additive MCP read tool. The legacy `docforge_get_context`
signature, profile compiler, direct Python results, and custom three-argument context-provider
contract remain unchanged.
The new `ContextCapsuleV1` and `RetrievalPlanV1` types live in the public
`docforge.retrieval` submodule. Version 1 guarantees:
- A closed task-kind vocabulary and core-derived plan. Callers cannot inject arbitrary operations,
SQL, paths, relations, or Logic requests.
- One immutable index transaction and one exact project, adapter, revision, source, policy,
request, plan, collection, and capsule identity.
- Deterministic bounded focus, traversal, hydration, token accounting, response packing, and
continuation, with fixed version-1 ceilings of 1,000 evidence items, 100,000 candidate edges, and
10,000 task-query characters.
- Raw preservation of project-owned relation names. Only the documented versioned alias map gains
task semantics; all other relations remain `unclassified`.
- Separate missing, incomplete, blocked, and provenance-limitation evidence.
- No-AST bindings retain task context but never add a Logic retrieval step or weaken the existing
Logic prohibition.
An integration that replaces the legacy context provider does not silently receive the core task
planner. Version 1 has no custom task-planner protocol. The task-context tool remains registered
for additive name compatibility but returns `task_context_unavailable` without loading or
synchronizing the custom projection.
The exact version-1 relation aliases are frozen by the MCP contract and repository contract tests.
Changing an alias category requires a new planner version; it is not a silent implementation
detail.
`ManualRenderPlan`, `GraphViewPlan`, a portable graph renderer, and independently packaged
renderers are later-milestone direction. Milestone 0 does not claim that those contracts exist.

56
docs/CONTRACT.md
View file

@ -18,7 +18,13 @@ commit when Git is available; it cannot change repository state.
- Edge schema: `schemas/edge.schema.json`, version 1.
- Result envelope: `schemas/result.schema.json`, version 1.
- Changeset schema: `schemas/changeset.schema.json`, version 1.
- Index schema: version 2, disposable and reproducible.
- Effective policy: `schemas/policy.schema.json`, version 1.
- Task context capsule: `schemas/context-capsule.schema.json`, version 1.
- Latest generation diff: `schemas/generation-diff.schema.json`, version 1.
- Latest generation-diff page: `schemas/generation-diff-page.schema.json`, version 1.
- Generated client configuration: `schemas/client-configuration.schema.json`, version 1.
- Client doctor result: `schemas/doctor-result.schema.json`, version 1.
- Index schema: version 3, disposable and reproducible.
- Index attestation: schema version 1, disposable and reproducible.
- Core, CLI, and MCP server: version 1.3.0.dev0.
- Incremental extraction cache: version 1, disposable and reproducible.
@ -44,11 +50,52 @@ source hash. Errors use a stable code, direct message, structured details, and a
tool when recovery is safe. MCP operations synchronize disposable index state under a project lock
before reading or proposing. Canonical source validation remains fail-closed.
Task-context retrieval derives a closed version-1 plan from a bounded task kind and the effective
process policy. It executes against one immutable index transaction and returns generation-bound,
hash-identified evidence, gaps, omissions, and provenance limitations. Project relation names
remain authoritative. The core applies task semantics only to its versioned alias set and preserves
every other allowed relation as unclassified.
An atomic index build writes a whole-file SHA-256 attestation after complete graph, row, FTS, and
SQLite integrity verification. A fresh process may use that receipt to verify an unchanged index
without reconstructing all graph rows. A missing, malformed, or mismatched receipt falls back to
complete verification and is repaired only after that verification succeeds.
Index replacement is the derived publication commit point. Attestation, cheap source-generation,
and latest-generation-diff receipts are independent post-commit evidence. Their failure produces
bounded degraded success and never falsely reports that a committed index mutation failed.
Before replacement, a build accepts a predecessor only when its exact main-file inode has a
matching whole-file attestation, has no WAL, journal, or shared-memory sidecar, and passes the
published SQLite identity, row, hash, FTS, integrity, and policy checks. It uses an immutable
main-file read and never repairs predecessor evidence. The build then revalidates the new source
snapshot including exact node and edge equality and rejects a stable source identity that produces
different graph content as `generation_collision`.
The version-1 generation-diff receipt stores one bounded latest primary-graph transition. It is
not history and contains no Logic details or source text. Public pages carry one
`receipt_header`; its `stored_receipt_hash` identifies the complete persisted receipt rather than
the header alone. One top-level pagination object carries the only continuation cursor.
## Machine-local client integration
Generated Codex, Claude, and OpenClaw fragments are machine-local projections. They are not
canonical project content. Version 1 binds the selected project, exact isolated Python
interpreter, canonical argument layout, effective policy, no-AST projection, render policy,
timeouts, artifact bytes, and configuration hash.
Preview is side-effect free. Explicit publication creates only one new private standalone
fragment in an existing real directory. It never merges or replaces different content. Descriptor,
parent, target, content, ownership, permission, and link identities are checked before and after
the directory durability boundary. A failure rolls back when that can be proven and otherwise
returns bounded unconfirmed publication evidence.
Doctor is a bounded read-only inspector with one fixed check inventory. It uses stable no-follow
descriptor and configuration reads plus stat-only derived-index evidence. It never loads a
complete projection, opens SQLite, starts MCP, executes the configured command, synchronizes,
builds, renders, starts a viewer, or writes configuration. Unprovable client behavior is a warning,
not an invented success.
## Isolated proposal model
Create, update, move, and delete are ordered node operations inside an isolated changeset. Every
@ -94,9 +141,10 @@ renderers. Render identity covers the canonical source hash, optional changeset
and edge identities, view configuration, template hash, renderer contract, and exact parser version.
An explicit CLI render atomically replaces one declared derived output. MCP can render a validated
changeset only to its isolated preview path. Status recomputes expected output without writing and
reports `current`, `stale`, `missing`, `unsafe`, or `oversized`. Input changes detected before atomic
replacement fail without replacing the prior output.
changeset only to its isolated preview path. Normal status verifies bounded source, configuration,
template, output, renderer, and publication-receipt identities without reconstructing the output.
Explicit deep status remains the side-effect-free full-render oracle. Input changes detected before
atomic replacement fail without publishing a current receipt for stale output.
Normal MCP access does not expose canonical application. An explicitly configured canonical
applier registers one hash-bound application tool. No MCP mode exposes arbitrary renderer

98
docs/MCP_CONTRACT.md
View file

@ -16,6 +16,18 @@ Canonical application is a second independent startup gate. The generic server a
`--canonical-applier WRITER_ID`. A project adapter must also supply a compatible project-owned
canonical applier implementation.
The additive `--capability-mode read|proposal|application|operator` option selects a versioned
effective process policy. Existing factory defaults and tool ordering remain unchanged: the
read-only factory exposes the read surface, the ordinary project factory exposes the proposal
surface, and an application-enabled factory adds exact-hash application. `application` mode fails
closed unless a canonical applier is bound. `operator` is reserved for explicitly selected
operator-only tools and adds none in the current contract.
Bootstrap and contract results include `effective_policy` schema version 1 plus a separate
`capabilities` record. Policy states the requested process behavior. Capabilities state the actual
registered surface and startup-bound proposal/application access. The project descriptor remains
schema version 1 and does not silently acquire machine-specific process policy.
## Read tools
- `docforge_bootstrap`
@ -30,19 +42,24 @@ canonical applier implementation.
- `docforge_dependencies`
- `docforge_impact`
- `docforge_get_context`
- `docforge_get_task_context`
- `docforge_validate_project`
- `docforge_render_status`
- `docforge_visualize`
- `docforge_stop_visualization`
- `docforge_visualization_status`
- `docforge_get_generation_diff`
Each response states that document text is project content, not higher-priority instructions. Each
response includes project identity, revision, source hash, adapter version, and staleness state.
Every normal tool call first checks current source identity and atomically rebuilds disposable index
state when it is missing, stale, or invalid. `docforge_bootstrap` performs that synchronization and
returns the complete fixed binding, active index path, proposal and application capabilities, and
recommended workflow. `docforge_sync` exposes the same idempotent synchronization explicitly.
Neither operation changes canonical sources.
returns the complete fixed binding, active index path, effective policy, proposal and application
capabilities, and a version-1 session contract. Bootstrap reuses the identity proven by
synchronization instead of loading the project again. Its first operation and workflow guidance
mention proposal or application tools only when the corresponding startup access is enabled.
`docforge_sync` exposes the same idempotent synchronization explicitly. Neither operation changes
canonical sources.
Search, filter, backlinks, dependencies, and impact accept explicit result limits bounded by the
project `max_results` policy. Omitted limits are still capped. Collection responses report whether
@ -57,6 +74,77 @@ oversized entry advances as a hash-identified `response size limit` omission so
loop; targeted retrieval remains available for that node. The existing three-argument custom
context-provider contract is unchanged because pagination is applied after provider selection.
`docforge_get_task_context` accepts one closed task kind (`change`, `implementation`, `failure`,
`ownership`, `test`, `operation`, or `release`), a bounded task description, and optional
`focus_node_id`, token `budget`, page `limit`, and opaque `cursor`. It derives, rather than accepts,
a version-1 retrieval plan. The plan contains only exact or lexical focus, bounded outgoing and
incoming graph traversal, and metadata hydration. It cannot request arbitrary SQL, paths, relation
names, or Logic extraction. Task context applies fixed internal ceilings of 1,000 evidence items,
100,000 examined candidate edges, and 10,000 task-query characters even when broader project
limits are configured. Traversal steps bind the complete project-owned relation vocabulary by hash
rather than copying an unbounded name list into every response.
The plan and returned context capsule are bound to the effective policy and one immutable index
generation. Every evidence item identifies its indexed source path, content hash, graph path,
additional qualifying relationship reasons, and the provenance facts that the current graph
cannot prove. Required evidence gaps distinguish an undeclared relation category, a completed
bounded search with no selected evidence, and an incomplete proof caused by a work, result, token,
or response limit. Unknown project relations remain present with their raw names and an
`unclassified_relation` limitation; DocForge never infers semantics from spelling outside the
versioned alias map.
Path relationship direction is relative to the preceding traversal node. Additional
`relationship_reasons` direction is relative to the evidence item itself: `outgoing` when that
evidence node is the stored source and `incoming` when it is the stored target.
The exact version-1 aliases are: structure (`contains`, `defined_in`, `defines`, `owns`);
implementation (`implemented_by`, `implements`, `inherits`, `inherits_from`); dependency
(`depends_on`, `imports`); execution (`activates`, `calls`, `dispatches_to`, `launches`); data
(`reads`, `writes`); evidence (`documents`, `governs`, `proves`, `tested_by`, `verifies`); and
context (`relates_to`). Every other allowed relation is `unclassified`.
Task-context continuation partitions the immutable evidence stream without changing its
`request_hash`, `plan_hash`, `collection_hash`, or `capsule_hash`. Its cursor additionally binds
the effective policy and task request. One evidence item that cannot fit advances exactly once as
a hash-identified `response_limit` omission. A changed generation, policy, plan, or collection
fails as `stale_cursor`.
`docforge_get_generation_diff` accepts only optional `limit` and `cursor` fields. It reads the one
latest version-1 primary-graph transition receipt; it does not accept arbitrary generations,
paths, or history selectors. Exact summary counts and the full item-collection hash cover the
complete transition. Pagination covers only the deterministically ordered retained details and
states separately when the fixed 1,000-item or 1 MiB publication limit permanently omitted
details.
The receipt binds project, root, adapter, index schema, from/to source identity, node and edge
hashes and counts, the committed index file identity, retained and full collection hashes, and its
own canonical hash. Node changes compare every core `Node` field. Edge identity is the exact
`(source_id, relation, target_id)` triple. Logic is excluded from public diff details.
Current pages use `page_schema_version = 1`. The nested `receipt_header` contains every stored
receipt field except `items`; its `stored_receipt_hash` is the hash of the complete stored receipt,
not of the header. Page items and hash-identified response-limit omissions are siblings of that
header. The only pagination object is at the top level, and its `next_cursor` is the only cursor
copy. The page hash covers the complete header, page items, omissions, receipt state, and
pagination receipt.
Receipt states are fail-closed: `current` is proven against cheap source identity and exact index
and receipt inodes; `stale` is a proven generation mismatch; `missing` means no receipt;
`unsafe` means confinement or file-type checks failed; `unverified` covers corrupt, foreign,
oversized, or concurrently changed evidence; and `unknown` means the project cannot provide a
cheap generation identity. Only `current` returns a page.
This status boundary is non-repairing. It never opens SQLite, loads or extracts an adapter
projection, parses source, synchronizes, builds, or writes a receipt. Cheap source identity and
stable receipt/index file identities can establish `current`; legacy projects without cheap
identity report `unknown`. Invalid or unavailable disposable evidence remains an explicit status
instead of triggering hidden recovery.
The legacy `docforge_get_context` tool and its custom three-argument provider contract remain
unchanged. A server with a custom context provider does not silently inherit the core task planner;
version 1 exposes no custom task-planner extension point. `docforge_get_task_context` returns
`task_context_unavailable` without synchronizing or loading the custom projection.
Version-1 cursors are canonical JSON encoded as base64url with a domain-separated SHA-256
corruption checksum. They are opaque and fail closed, but are not authenticated authorization
tokens. Cursors bind the project, adapter, source generation, operation parameters, collection
@ -257,3 +345,7 @@ and indexed Logic, but it does not inspect arbitrary adapter source to prove whi
implementation the adapter uses. DocForge still exposes no filesystem sandbox and cannot prevent
an unrelated process with direct repository write access from editing adapter files. Repository
permissions and project instructions remain responsible for that broader boundary.
The legacy `adapter_policy` object remains byte-compatible. It is now a projection of the
versioned `effective_policy`; `--no-ast` restrictively overrides adapter evolution, AST analysis,
and Logic indexing without widening any other capability.

85
docs/MILESTONE_2_BASELINE.md Normal file
View file

@ -0,0 +1,85 @@
# Milestone 2 baseline
## Scope and method
This baseline records the agent-retrieval and client-integration behavior added in Milestone 2.
It was captured on 2026-07-29 from clean candidate commit
`fb0df5e4a1c591c2a84788fd4814d98550f11863`.
The maintained command was:
```bash
.venv/bin/python tools/milestone2_benchmark.py \
--nodes 1000 \
--samples 10 \
--output /tmp/docforge-milestone2-final.json
```
The fixture contains 1,000 Markdown nodes and 999 edges in a direct fan-in around one focus node.
The configured MCP response limit is 200,000 characters. Durations use
`time.perf_counter_ns()` and nearest-rank p95. Peak memory uses an isolated child process and
`RUSAGE_SELF`. Every warmup and measured invocation is validated.
Environment:
- Linux 7.1.3-200.nobara.fc44.x86_64.
- CPython 3.14.6.
- x86_64.
- Ten warm samples after one warmup.
- Isolated memory ceiling: 262,144 KiB.
The complete machine-readable result is
[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json).
## Results
| Operation | Median | p95 | Limit | Maximum response |
|---|---:|---:|---:|---:|
| Read bootstrap | 9.406 ms | 9.884 ms | 100 ms | 7,495 B |
| No-AST bootstrap | 9.153 ms | 9.379 ms | 100 ms | 8,796 B |
| Task diagnostic page | 73.326 ms | 93.115 ms | 500 ms | 7,190 B |
| Task complete traversal | 703.561 ms | 721.847 ms | 2,500 ms | 151,172 B/page |
| Generation diagnostic page | 40.961 ms | 41.650 ms | 100 ms | 66,516 B |
| Generation maximum page | 55.565 ms | 59.184 ms | 100 ms | 199,566 B |
| Generation complete traversal | 418.607 ms | 425.315 ms | 500 ms | 66,516 B/page |
| Codex configuration preview | 314.365 ms | 364.383 ms | 500 ms | 2,627 B |
| Claude configuration preview | 314.326 ms | 364.365 ms | 500 ms | 2,748 B |
| OpenClaw configuration preview | 314.401 ms | 364.532 ms | 500 ms | 2,869 B |
| Codex doctor | 0.421 ms | 0.556 ms | 100 ms | 3,595 B |
| Claude doctor | 0.364 ms | 0.446 ms | 100 ms | 3,669 B |
| OpenClaw doctor | 0.384 ms | 0.484 ms | 100 ms | 3,602 B |
Isolated peak RSS was 86,448 KiB.
Task traversal returned 108 evidence records and 892 explicit omissions across 11 pages. One
individually oversized focus record became a response-limit surrogate bound to the original record
hash. The remaining omissions were token-budget evidence. The benchmark verified every unique
subject, reconstructed the original collection hash, and matched the exact 1,000-node fixture.
Generation traversal returned all 1,000 changed-node details across 10 pages. It reconstructed the
stored retained-collection hash. The maximum generation page approached the response limit and
proved that optional diagnostics were dropped before the primary result.
## Structured-work gates
Configuration preview and doctor performed zero project loads, source parses, adapter projection
loads, adapter extraction, index checks, synchronization, index builds, render preparation,
rendered-byte construction or hashing, and viewer-manager requests.
Task-context pages performed exactly one index check and two cheap source-generation checks. They
performed none of the hidden work above. Generation-diff pages performed exactly two cheap
source-generation checks and no index check or hidden work.
## Measured limits and future notes
- Continuation is stateless and regenerates the task capsule for each page. The complete
11-page traversal remains within its gate, but later work can avoid repeated planning without
weakening generation binding.
- Configuration preview deliberately spends about 314 ms proving that the exact isolated
interpreter can import the MCP module. Discovery-only checks were rejected as unsafe.
- Claude configuration syntax is supported, but its timeout representation remains unverified.
Doctor therefore reports degraded rather than healthy.
- Doctor is a configuration inspector, not an MCP connection or SQLite integrity test.
- Legacy adapters without cheap source-generation identity report unknown for generation-diff
freshness.
- The results do not justify a storage rewrite. SQLite remains fast after one generation is pinned.

68
docs/MILESTONE_2_CLOSEOUT.md Normal file
View file

@ -0,0 +1,68 @@
# Milestone 2 closeout
## Outcome
Milestone 2 is complete. One project-bound server can expose an explicit effective policy and
return compact, task-shaped, explainable context. Users can generate deterministic client
fragments and inspect their bindings without hidden runtime work.
Implemented contracts:
- Version-1 effective policy and capability-aware bootstrap.
- Version-1 retrieval plans and context capsules.
- Bounded task-context pagination with evidence gaps and explicit omissions.
- One disposable latest-generation transition receipt and paged read surface.
- Deterministic Codex, Claude, and OpenClaw standalone configuration fragments.
- Fixed-inventory read-only doctor results.
- Dedicated configuration and doctor JSON schemas.
- Repository-native Milestone 2 contract, smoke, scale, response-size, counter, and memory gates.
## Candidate evidence
The frozen implementation candidate is
`fb0df5e4a1c591c2a84788fd4814d98550f11863`.
The complete repository gate passed:
- Ruff formatting and lint.
- HTML, rendered-manual HTML, CSS, and JavaScript checks.
- Pyright with zero diagnostics.
- Warning-strict compilation and tests.
- 205 tests and 120 subtests.
- Lock and npm dependency-tree checks.
- Wheel and source-distribution builds.
- Milestone 0, 1, and 2 smoke benchmarks.
Three independent read-only adversarial audits covered client publication and policy binding,
doctor race and malformed-input behavior, and benchmark/contract evidence. Reproduced descriptor,
parent, target, filesystem, policy, secret-redaction, ambiguity, parser, response-size, and hidden
work defects were fixed and regression-tested before the candidate was frozen.
The clean ten-sample 1,000-node benchmark passed every threshold. Exact measurements and counter
ranges are recorded in
[`MILESTONE_2_BASELINE.md`](MILESTONE_2_BASELINE.md) and
[`benchmarks/milestone2-2026-07-29.json`](../benchmarks/milestone2-2026-07-29.json).
## Preserved boundaries
- The `docforge` package, imports, CLI executable, MCP executable, and existing tool names remain.
- Legacy one-method `load_projection()` adapters remain supported.
- The no-AST shorthand and legacy adapter-policy payload remain compatible.
- Project descriptor schema version 1 remains unchanged.
- No storage replacement was introduced.
- No legacy DocForge MCP or DocForge2 self-hosting was used.
- WorldForge and ScrapeStation were not touched.
- No production MCP integration was repointed.
- The legacy Forgejo repository and `legacy` remote were not changed.
- No tag, release, release announcement, or visibility change was created.
## Known follow-up work
The next active milestone may improve projection independence. It must not silently absorb these
separate future ideas:
- Avoid recomputing a complete task capsule for every continuation page.
- Add authenticated cursors only if a stronger threat model requires them.
- Verify Claude's native timeout representation.
- Add versioned adapter-owned launcher metadata before generating custom-adapter configurations.
- Keep doctor read-only; a live connection test must be an explicit separate operation.

146
docs/USER_MANUAL.md
View file

@ -478,8 +478,15 @@ backlinks NODE_ID [--relation RELATION] [--limit N]
dependencies NODE_ID [--depth N] [--limit N]
impact NODE_ID [--depth N] [--limit N]
context PROFILE [--budget N] [--limit N] [--cursor OPAQUE]
generation-diff [--limit N] [--cursor OPAQUE]
```
`generation-diff` returns the latest verified primary-graph transition. It is not a history query.
Current results carry a version-1 page, a `receipt_header` bound to the complete stored receipt by
`stored_receipt_hash`, and one top-level pagination cursor. Missing, unsafe, stale, corrupt, or
unprovable disposable evidence is reported as a non-repairing receipt status. The command never
builds or repairs the index.
### Render and proposal commands
```text
@ -501,6 +508,67 @@ visualization-status
visualization-stop
```
### Client configuration and doctor
Preview one deterministic standalone client fragment:
```bash
docforge configure codex --project /absolute/path/MyProject
docforge configure claude --project /absolute/path/MyProject
docforge configure openclaw --project /absolute/path/MyProject
```
Preview is the default. Add `--output /absolute/path/fragment` to create a new private fragment in
an existing real directory. Publication is create-only. DocForge accepts an identical existing
private single-link file as unchanged, but it never merges, replaces, broadens permissions, or
follows a symlink. Descriptor, parent, and target identities are revalidated across the
publication commit.
The generated command uses the exact current Python interpreter with isolated module startup.
Generation first proves that this interpreter can import `docforge.mcp_server`. The result binds
the project root, effective policy, arguments, artifact bytes, and all hashes. It copies no ambient
environment values.
Select authority explicitly:
```bash
docforge configure codex \
--project /absolute/path/MyProject \
--capability-mode proposal \
--proposal-writer project-editor
docforge configure codex \
--project /absolute/path/MyProject \
--capability-mode application \
--proposal-writer project-editor \
--canonical-applier project-editor
```
Read mode is the default. Proposal and application modes fail closed unless the descriptor
declares the named writer, and application requires the same writer/applier identity. Add
`--no-ast` to preserve the no-AST binding. Generic CLI generation refuses project-owned adapters
because it cannot safely reconstruct their composition.
Inspect one configured client binding:
```bash
docforge doctor --client codex --project /absolute/path/MyProject
docforge doctor --client codex \
--project /absolute/path/MyProject \
--config /absolute/path/config.toml \
--server-name my-project-docforge
```
Doctor returns `healthy`, `degraded`, or `unhealthy` with exit codes 0, 1, or 2. Its fixed
version-1 inventory checks project and descriptor binding, the client driver and entry, executable
and arguments, project root, effective policy, no-AST state, timeouts, environment-key names,
tool-filter representation, and stat-only index presence.
Doctor is intentionally not a connection test. It never loads canonical sources, opens SQLite,
starts MCP, executes the configured command, synchronizes, builds, renders, starts a viewer, or
writes configuration. Claude timeout representation and client filtering that cannot be proved
locally remain explicit warnings.
## MCP usage
Run one MCP server per project with absolute paths:
@ -513,6 +581,19 @@ docforge-mcp \
Omit `--proposal-writer` when the MCP client should not create or append proposals.
Select the session's declared surface explicitly when useful:
```bash
docforge-mcp \
--project-root /absolute/path/MyProject \
--capability-mode read
```
Supported modes are `read`, `proposal`, `application`, and `operator`. Existing startup defaults
remain compatible. Capability mode describes the registered surface; bootstrap separately reports
whether a configured writer or applier actually grants mutation access. Application mode refuses
startup without a canonical applier. Operator mode is reserved and currently adds no tools.
Add `--diagnostics` when profiling a development or benchmark session. Each MCP response then
includes bounded stage timings and compiler-work counters. The same flag is available on
`docforge`. Diagnostics are disabled by default, record no project content or paths, and never
@ -530,6 +611,11 @@ docforge-mcp \
Without `--canonical-applier`, `docforge_apply_changeset` is not registered. The flag is an
identity, not a command. The changeset creator, configured writer, and canonical applier must agree.
Call `docforge_bootstrap` first. Its version-1 `session_contract` contains the fixed binding,
current graph generation, effective policy, actual capabilities, render policies, prohibitions,
and a recommended first operation. Workflow guidance does not recommend registration or
application when those startup capabilities are unavailable.
Example MCP client configuration:
```json
@ -564,11 +650,13 @@ Example MCP client configuration:
- `docforge_dependencies`
- `docforge_impact`
- `docforge_get_context`
- `docforge_get_task_context`
- `docforge_validate_project`
- `docforge_render_status`
- `docforge_visualize`
- `docforge_visualization_status`
- `docforge_stop_visualization`
- `docforge_get_generation_diff`
### Proposal tools
@ -595,6 +683,54 @@ The application call requires `changeset_id` and `expected_changeset_hash`. Alwa
inspect the final diff after the last proposal mutation. Apply that exact hash. A proposal mutation
creates a new hash, so an earlier approval cannot silently apply later content.
Use `docforge_get_task_context` when an agent needs one bounded task-shaped intake instead of a
named profile. Choose `task_kind` from `change`, `implementation`, `failure`, `ownership`, `test`,
`operation`, or `release`. Supply `focus_node_id` when the stable node is known. Without it,
DocForge performs a bounded lexical focus search and refuses a tied best match instead of silently
choosing one.
The returned version-1 capsule includes:
- The exact project, adapter, source generation, effective policy, request, and retrieval-plan
hashes.
- Ordered focus and related evidence with source paths, content hashes, graph paths, and all
qualifying relationship reasons observed during the bounded traversal.
- Explicit evidence gaps and omissions, including whether a check completed.
- Provenance limitations for facts that the current graph does not carry, such as extractor
identity, observation time, and source provenance for relationships.
Project descriptors still own the valid relation vocabulary. The planner recognizes a fixed alias
map for structure, implementation, dependency, execution, data, evidence, and context. Any other
valid project relation is returned unchanged as `unclassified`; it is never assigned guessed task
semantics.
A relationship inside `relationship_path` describes the direction traveled from the preceding
node. A relationship inside `relationship_reasons` describes direction from the evidence item
itself. This keeps stored source and target identity exact while making each evidence explanation
locally readable.
Task context never exceeds 1,000 evidence items, 100,000 examined candidate edges, or 10,000 task
query characters, even when a project configures broader general limits. An edge-work or
unclassified-relation ceiling appears as an explicit omission rather than an unbounded response.
Use `docforge_get_generation_diff` after synchronization or a completed implementation slice to
inspect the one latest verified primary-graph transition. The version-1 receipt reports exact
added, removed, and changed node counts plus added and removed edge counts. Retained node details
identify changed fields and before/after hashes and source paths. Edge details retain the exact raw
relation triple. The receipt stores no source text, rendered content, Logic identities, or
historical sequence.
The first successful publication is an explicit baseline and does not claim every current node was
added. A corrupt, foreign, unsafe, or unavailable predecessor produces an unavailable comparison
rather than fabricated removals. A same-generation reindex preserves the latest meaningful
transition. Each later real transition atomically replaces the single disposable receipt.
Generation-diff reads use only the bounded receipt, stable file identities, and an adapter's cheap
source-generation proof. They do not open SQLite, load a complete adapter projection, parse source,
synchronize, build, or repair. Legacy adapters without cheap identity report `unknown`. Missing,
corrupt, foreign, oversized, or concurrently changed receipts report an explicit receipt state and
do not trigger hidden recovery.
Recommended release-candidate sequence:
1. Call `docforge_bootstrap`. It synchronizes derived state and reports the exact fixed binding.
@ -657,6 +793,16 @@ inspection pages may use hash summaries. A large diff may return `result_mode =
"canonical_json_chunk"`; concatenate the chunks in order and verify `payload_hash` before decoding
the reconstructed `operations` and `changes` object.
`docforge_get_task_context` uses the same opaque continuation discipline over capsule evidence
followed by capsule omissions. Keep the semantic task arguments unchanged while paging. Page size
may change. Every page retains the same plan, collection, and capsule hashes. A `stale_cursor`
means that the generation, policy, plan, or collection changed; discard earlier pages and restart.
`docforge_get_generation_diff` paginates only the details retained in the latest bounded receipt.
Its summary counts and full collection hash still cover permanently truncated details. The cursor
binds the exact receipt, target generation, retained and full collection hashes, receipt state, and
effective policy. A replacement receipt returns `stale_cursor`; restart from its first page.
Canonical application records its terminal receipt immediately after the project-owned serializer
verifies the new canonical state. A later index or render refresh failure is reported as degraded
derived state with remediation, not as permission to apply the same canonical change again.

894
schemas/client-configuration.schema.json Normal file
View file

@ -0,0 +1,894 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/client-configuration-v1.json",
"title": "DocForge deterministic client configuration plan",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"adapter_policy": {
"oneOf": [
{
"type": "object",
"required": [
"mode",
"ast_analysis",
"logic_projection",
"incremental_extraction",
"adapter_rewrite"
],
"properties": {
"mode": { "const": "standard" },
"ast_analysis": { "const": "allowed" },
"logic_projection": { "const": "allowed" },
"incremental_extraction": { "const": "allowed" },
"adapter_rewrite": { "const": "not_requested" }
},
"additionalProperties": false
},
{
"type": "object",
"required": [
"mode",
"ast_analysis",
"logic_projection",
"incremental_extraction",
"adapter_rewrite",
"blocked_tools",
"instruction"
],
"properties": {
"mode": { "const": "preserve-no-ast" },
"ast_analysis": { "const": "forbidden" },
"logic_projection": { "const": "forbidden" },
"incremental_extraction": { "const": "allowed" },
"adapter_rewrite": { "const": "forbidden" },
"blocked_tools": {
"const": ["docforge_get_logic"]
},
"instruction": {
"const": "Preserve the existing adapter extraction strategy. Do not add Python AST, Tree-sitter, compiler-AST, or function-Logic extraction. Non-AST incremental fingerprinting and caching remain allowed."
}
},
"additionalProperties": false
}
]
},
"effective_policy": {
"type": "object",
"required": [
"schema_version",
"capability_mode",
"capability_source",
"adapter_evolution",
"ast_analysis",
"logic_indexing",
"synchronization",
"integrity",
"manual_render",
"graph_render",
"live_viewer",
"profiling",
"blocked_tools",
"prohibitions",
"precedence"
],
"properties": {
"schema_version": { "const": 1 },
"capability_mode": {
"enum": ["read", "proposal", "application"]
},
"capability_source": { "const": "explicit" },
"adapter_evolution": { "enum": ["allowed", "preserve"] },
"ast_analysis": { "enum": ["allowed", "forbidden"] },
"logic_indexing": { "enum": ["full", "off"] },
"synchronization": { "const": "automatic" },
"integrity": { "const": "validated" },
"manual_render": { "enum": ["auto", "explicit", "disabled"] },
"graph_render": { "const": "disabled" },
"live_viewer": { "const": "on-demand" },
"profiling": { "const": "disabled" },
"blocked_tools": {
"type": "array",
"maxItems": 1,
"items": { "const": "docforge_get_logic" },
"uniqueItems": true
},
"prohibitions": {
"type": "array",
"minItems": 7,
"maxItems": 11,
"items": {
"enum": [
"arbitrary_file_access",
"arbitrary_renderer_execution",
"shell_execution",
"git_mutation",
"deployment",
"publication",
"project_switching",
"adapter_ast_upgrade",
"tree_sitter_upgrade",
"compiler_ast_upgrade",
"function_logic_extraction"
]
},
"uniqueItems": true
},
"precedence": {
"const": [
"core_safety",
"explicit_binding",
"no_ast_shorthand",
"resource_availability"
]
}
},
"additionalProperties": false
},
"diagnostics": {
"type": "object",
"required": [
"schema_version",
"operation",
"outcome",
"elapsed_ns",
"stages",
"counters"
],
"properties": {
"schema_version": { "const": 1 },
"operation": { "const": "cli.configure" },
"outcome": { "const": "ok" },
"elapsed_ns": { "type": "integer", "minimum": 0 },
"stages": {
"type": "object",
"maxProperties": 14,
"propertyNames": {
"enum": [
"source.generation",
"source.parse",
"adapter.projection",
"adapter.extract",
"index.check",
"index.synchronize",
"index.build",
"index.read",
"render.status",
"render.prepare",
"render.output_hash",
"visualization.status",
"viewer.manager",
"mcp.runtime_validation"
]
},
"additionalProperties": {
"type": "object",
"required": ["calls", "elapsed_ns"],
"properties": {
"calls": { "type": "integer", "minimum": 1 },
"elapsed_ns": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
}
},
"counters": {
"type": "object",
"required": [
"project_loads",
"source_files_parsed",
"source_bytes_parsed",
"adapter_projection_loads",
"adapter_source_extractions",
"source_generation_checks",
"index_checks",
"index_synchronizations",
"index_builds",
"render_prepare_calls",
"render_output_bytes_built",
"render_output_bytes_hashed",
"viewer_manager_requests"
],
"additionalProperties": {
"type": "integer",
"minimum": 0
},
"maxProperties": 13
}
},
"additionalProperties": false
}
},
"type": "object",
"required": [
"status",
"schema_version",
"operation",
"action",
"client",
"server_name",
"project",
"binding",
"effective_policy",
"artifact",
"configuration_hash",
"warnings"
],
"properties": {
"status": { "const": "ok" },
"schema_version": { "const": 1 },
"operation": { "const": "client.configure" },
"action": { "enum": ["preview", "write"] },
"client": { "enum": ["codex", "claude", "openclaw"] },
"server_name": {
"type": "string",
"pattern": "^[a-z0-9][a-z0-9_-]{0,63}$"
},
"project": {
"type": "object",
"required": [
"project_id",
"project_root",
"project_root_fingerprint",
"adapter"
],
"properties": {
"project_id": { "type": "string", "minLength": 1 },
"project_root": { "type": "string", "minLength": 1 },
"project_root_fingerprint": {
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1 }
},
"additionalProperties": false
},
"binding": {
"type": "object",
"required": [
"transport",
"capability_mode",
"adapter_policy",
"render_policy",
"command",
"args",
"environment",
"timeouts"
],
"properties": {
"transport": { "const": "stdio" },
"capability_mode": {
"enum": ["read", "proposal", "application"]
},
"adapter_policy": { "$ref": "#/$defs/adapter_policy" },
"render_policy": {
"type": "object",
"required": ["manual", "graph", "live_viewer"],
"properties": {
"manual": { "enum": ["auto", "explicit", "disabled"] },
"graph": { "const": "disabled" },
"live_viewer": { "const": "on-demand" }
},
"additionalProperties": false
},
"command": { "type": "string", "minLength": 1 },
"args": {
"type": "array",
"minItems": 7,
"maxItems": 64,
"prefixItems": [
{ "const": "-I" },
{ "const": "-m" },
{ "const": "docforge.mcp_server" },
{ "const": "--project-root" },
{ "type": "string", "minLength": 1, "maxLength": 4096 },
{ "const": "--capability-mode" },
{ "enum": ["read", "proposal", "application"] }
],
"items": {
"type": "string",
"minLength": 1,
"maxLength": 4096
}
},
"environment": {
"type": "object",
"maxProperties": 0
},
"timeouts": {
"type": "object",
"required": ["startup_seconds", "tool_seconds"],
"properties": {
"startup_seconds": {
"type": "integer",
"minimum": 1,
"maximum": 3600
},
"tool_seconds": {
"type": "integer",
"minimum": 1,
"maximum": 86400
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
"effective_policy": { "$ref": "#/$defs/effective_policy" },
"artifact": {
"type": "object",
"required": [
"format",
"content",
"content_sha256",
"output_path",
"write_state",
"durability"
],
"properties": {
"format": {
"enum": [
"codex-toml-fragment-v1",
"claude-json-fragment-v1",
"openclaw-json-fragment-v1"
]
},
"content": {
"type": "string",
"minLength": 1,
"maxLength": 65536
},
"content_sha256": { "$ref": "#/$defs/sha256" },
"output_path": {
"type": ["string", "null"],
"minLength": 1
},
"write_state": {
"enum": ["not_requested", "created", "unchanged"]
},
"durability": {
"enum": ["not_applicable", "confirmed", "unconfirmed"]
}
},
"additionalProperties": false
},
"configuration_hash": { "$ref": "#/$defs/sha256" },
"warnings": {
"type": "array",
"maxItems": 8,
"items": {
"type": "object",
"required": ["code"],
"properties": {
"code": {
"enum": [
"timeout_format_unverified",
"publication_durability_unconfirmed",
"publication_location_unconfirmed",
"publication_binding_unconfirmed"
]
}
},
"additionalProperties": false
}
},
"diagnostics": { "$ref": "#/$defs/diagnostics" }
},
"allOf": [
{
"if": {
"properties": {
"warnings": {
"contains": {
"properties": {
"code": { "const": "publication_binding_unconfirmed" }
},
"required": ["code"]
}
}
},
"required": ["warnings"]
},
"then": {
"properties": {
"artifact": {
"properties": {
"output_path": { "type": "null" },
"write_state": { "const": "created" },
"durability": { "const": "unconfirmed" }
}
}
}
}
},
{
"if": {
"properties": { "client": { "const": "codex" } },
"required": ["client"]
},
"then": {
"properties": {
"artifact": {
"properties": {
"format": { "const": "codex-toml-fragment-v1" }
}
},
"warnings": {
"not": {
"contains": {
"properties": {
"code": { "const": "timeout_format_unverified" }
},
"required": ["code"]
}
}
}
}
}
},
{
"if": {
"properties": { "client": { "const": "openclaw" } },
"required": ["client"]
},
"then": {
"properties": {
"artifact": {
"properties": {
"format": { "const": "openclaw-json-fragment-v1" }
}
},
"warnings": {
"not": {
"contains": {
"properties": {
"code": { "const": "timeout_format_unverified" }
},
"required": ["code"]
}
}
}
}
}
},
{
"if": {
"properties": { "client": { "const": "claude" } },
"required": ["client"]
},
"then": {
"properties": {
"artifact": {
"properties": {
"format": { "const": "claude-json-fragment-v1" }
}
},
"warnings": {
"contains": {
"properties": {
"code": { "const": "timeout_format_unverified" }
},
"required": ["code"]
}
}
}
}
},
{
"if": {
"properties": { "action": { "const": "preview" } },
"required": ["action"]
},
"then": {
"properties": {
"artifact": {
"properties": {
"output_path": { "type": "null" },
"write_state": { "const": "not_requested" },
"durability": { "const": "not_applicable" }
}
}
}
},
"else": {
"properties": {
"artifact": {
"properties": {
"output_path": {
"type": ["string", "null"],
"minLength": 1
},
"write_state": { "enum": ["created", "unchanged"] }
},
"allOf": [
{
"if": {
"properties": { "write_state": { "const": "created" } },
"required": ["write_state"]
},
"then": {
"properties": {
"durability": { "enum": ["confirmed", "unconfirmed"] }
}
},
"else": {
"properties": {
"durability": { "const": "not_applicable" }
}
}
}
]
}
}
}
},
{
"if": {
"properties": {
"binding": {
"properties": { "capability_mode": { "const": "read" } },
"required": ["capability_mode"]
}
},
"required": ["binding"]
},
"then": {
"properties": {
"binding": {
"properties": {
"args": {
"prefixItems": [{}, {}, {}, {}, {}, {}, { "const": "read" }]
}
}
},
"effective_policy": {
"properties": { "capability_mode": { "const": "read" } }
}
}
}
},
{
"if": {
"properties": {
"binding": {
"properties": { "capability_mode": { "const": "proposal" } },
"required": ["capability_mode"]
}
},
"required": ["binding"]
},
"then": {
"properties": {
"binding": {
"properties": {
"args": {
"prefixItems": [{}, {}, {}, {}, {}, {}, { "const": "proposal" }]
}
}
},
"effective_policy": {
"properties": { "capability_mode": { "const": "proposal" } }
}
}
}
},
{
"if": {
"properties": {
"binding": {
"properties": { "capability_mode": { "const": "application" } },
"required": ["capability_mode"]
}
},
"required": ["binding"]
},
"then": {
"properties": {
"binding": {
"properties": {
"args": {
"prefixItems": [{}, {}, {}, {}, {}, {}, { "const": "application" }]
}
}
},
"effective_policy": {
"properties": { "capability_mode": { "const": "application" } }
}
}
}
},
{
"if": {
"properties": {
"binding": {
"properties": {
"adapter_policy": {
"properties": { "mode": { "const": "preserve-no-ast" } },
"required": ["mode"]
}
},
"required": ["adapter_policy"]
}
},
"required": ["binding"]
},
"then": {
"properties": {
"binding": {
"properties": {
"args": {
"contains": { "const": "--no-ast" },
"minContains": 1,
"maxContains": 1
}
}
},
"effective_policy": {
"properties": {
"adapter_evolution": { "const": "preserve" },
"ast_analysis": { "const": "forbidden" },
"logic_indexing": { "const": "off" },
"blocked_tools": { "const": ["docforge_get_logic"] },
"prohibitions": {
"const": [
"arbitrary_file_access",
"arbitrary_renderer_execution",
"shell_execution",
"git_mutation",
"deployment",
"publication",
"project_switching",
"adapter_ast_upgrade",
"tree_sitter_upgrade",
"compiler_ast_upgrade",
"function_logic_extraction"
]
}
}
}
}
},
"else": {
"properties": {
"binding": {
"properties": {
"args": {
"not": {
"contains": { "const": "--no-ast" }
}
}
}
},
"effective_policy": {
"properties": {
"adapter_evolution": { "const": "allowed" },
"ast_analysis": { "const": "allowed" },
"logic_indexing": { "const": "full" },
"blocked_tools": { "const": [] },
"prohibitions": {
"const": [
"arbitrary_file_access",
"arbitrary_renderer_execution",
"shell_execution",
"git_mutation",
"deployment",
"publication",
"project_switching"
]
}
}
}
}
}
},
{
"if": {
"properties": {
"binding": {
"properties": {
"render_policy": {
"properties": { "manual": { "const": "auto" } },
"required": ["manual"]
}
},
"required": ["render_policy"]
}
},
"required": ["binding"]
},
"then": {
"properties": {
"effective_policy": {
"properties": { "manual_render": { "const": "auto" } }
}
}
}
},
{
"if": {
"properties": {
"binding": {
"properties": {
"render_policy": {
"properties": { "manual": { "const": "explicit" } },
"required": ["manual"]
}
},
"required": ["render_policy"]
}
},
"required": ["binding"]
},
"then": {
"properties": {
"effective_policy": {
"properties": { "manual_render": { "const": "explicit" } }
}
}
}
},
{
"if": {
"properties": {
"binding": {
"properties": {
"render_policy": {
"properties": { "manual": { "const": "disabled" } },
"required": ["manual"]
}
},
"required": ["render_policy"]
}
},
"required": ["binding"]
},
"then": {
"properties": {
"effective_policy": {
"properties": { "manual_render": { "const": "disabled" } }
}
}
}
},
{
"if": {
"properties": {
"artifact": {
"properties": {
"durability": { "const": "unconfirmed" }
},
"required": ["durability"]
}
},
"required": ["artifact"]
},
"then": {
"properties": {
"artifact": {
"properties": {
"write_state": { "const": "created" }
}
},
"warnings": {
"anyOf": [
{
"contains": {
"properties": {
"code": { "const": "publication_durability_unconfirmed" }
},
"required": ["code"]
}
},
{
"contains": {
"properties": {
"code": { "const": "publication_location_unconfirmed" }
},
"required": ["code"]
}
},
{
"contains": {
"properties": {
"code": { "const": "publication_binding_unconfirmed" }
},
"required": ["code"]
}
}
]
}
}
}
},
{
"if": {
"properties": {
"warnings": {
"contains": {
"properties": {
"code": { "const": "publication_location_unconfirmed" }
},
"required": ["code"]
}
}
},
"required": ["warnings"]
},
"then": {
"properties": {
"artifact": {
"properties": {
"output_path": { "type": "null" },
"write_state": { "const": "created" },
"durability": { "const": "unconfirmed" }
}
}
}
}
},
{
"if": {
"properties": {
"action": { "const": "write" },
"artifact": {
"properties": { "output_path": { "type": "null" } },
"required": ["output_path"]
}
},
"required": ["action", "artifact"]
},
"then": {
"properties": {
"warnings": {
"anyOf": [
{
"contains": {
"properties": {
"code": { "const": "publication_location_unconfirmed" }
},
"required": ["code"]
}
},
{
"contains": {
"properties": {
"code": { "const": "publication_binding_unconfirmed" }
},
"required": ["code"]
}
}
]
}
}
}
},
{
"if": {
"properties": {
"warnings": {
"contains": {
"properties": {
"code": { "const": "publication_durability_unconfirmed" }
},
"required": ["code"]
}
}
},
"required": ["warnings"]
},
"then": {
"properties": {
"artifact": {
"properties": {
"write_state": { "const": "created" },
"durability": { "const": "unconfirmed" }
}
}
}
}
}
],
"additionalProperties": false
}

435
schemas/context-capsule.schema.json Normal file
View file

@ -0,0 +1,435 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/context-capsule-v1.json",
"title": "DocForge task context capsule",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"pagination": {
"type": "object",
"required": [
"schema_version",
"kind",
"returned_count",
"limit",
"total_count",
"has_more",
"next_cursor"
],
"properties": {
"schema_version": { "const": 1 },
"kind": { "const": "task-context.items" },
"returned_count": { "type": "integer", "minimum": 0 },
"limit": { "type": "integer", "minimum": 1 },
"total_count": { "type": "integer", "minimum": 0 },
"has_more": { "type": "boolean" },
"next_cursor": {
"type": ["string", "null"],
"minLength": 1,
"maxLength": 8192
}
},
"additionalProperties": false
},
"category": {
"enum": [
"structure",
"implementation",
"dependency",
"execution",
"data",
"evidence",
"context",
"unclassified"
]
},
"step": {
"type": "object",
"required": [
"step_id",
"operation",
"relation_scope",
"relation_set_hash",
"direction",
"depth",
"limit",
"required",
"evidence_role"
],
"properties": {
"step_id": { "type": "string", "minLength": 1, "maxLength": 64 },
"operation": {
"enum": ["exact", "search", "outgoing", "incoming", "metadata"]
},
"relation_scope": { "enum": ["none", "project_allowed"] },
"relation_set_hash": {
"oneOf": [
{ "$ref": "#/$defs/sha256" },
{ "type": "null" }
]
},
"direction": { "enum": ["none", "outgoing", "incoming"] },
"depth": { "type": "integer", "minimum": 0 },
"limit": { "type": "integer", "minimum": 1 },
"required": { "type": "boolean" },
"evidence_role": { "type": "string", "minLength": 1, "maxLength": 64 }
},
"additionalProperties": false
},
"requirement": {
"type": "object",
"required": ["requirement_id", "check", "category", "required"],
"properties": {
"requirement_id": { "type": "string", "minLength": 1, "maxLength": 128 },
"check": { "const": "selected_relation_category" },
"category": { "$ref": "#/$defs/category" },
"required": { "const": true }
},
"additionalProperties": false
},
"plan": {
"type": "object",
"required": [
"schema_version",
"planner",
"task_kind",
"request_hash",
"effective_policy_hash",
"focus_node_id",
"limits",
"category_order",
"steps",
"requirements",
"plan_hash"
],
"properties": {
"schema_version": { "const": 1 },
"planner": {
"type": "object",
"required": ["id", "version"],
"properties": {
"id": { "const": "docforge.core.task-context" },
"version": { "const": 1 }
},
"additionalProperties": false
},
"task_kind": {
"enum": [
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release"
]
},
"request_hash": { "$ref": "#/$defs/sha256" },
"effective_policy_hash": { "$ref": "#/$defs/sha256" },
"focus_node_id": { "type": ["string", "null"], "maxLength": 256 },
"limits": {
"type": "object",
"required": [
"max_evidence",
"max_tokens",
"max_depth",
"max_candidate_edges"
],
"properties": {
"max_evidence": { "type": "integer", "minimum": 1, "maximum": 1000 },
"max_tokens": { "type": "integer", "minimum": 1 },
"max_depth": { "type": "integer", "minimum": 0 },
"max_candidate_edges": {
"type": "integer",
"minimum": 1,
"maximum": 100000
}
},
"additionalProperties": false
},
"category_order": {
"type": "array",
"minItems": 8,
"maxItems": 8,
"items": { "$ref": "#/$defs/category" },
"uniqueItems": true
},
"steps": {
"type": "array",
"minItems": 4,
"maxItems": 4,
"items": { "$ref": "#/$defs/step" }
},
"requirements": {
"type": "array",
"minItems": 1,
"maxItems": 8,
"items": { "$ref": "#/$defs/requirement" }
},
"plan_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"relationship": {
"type": "object",
"required": [
"source_id",
"relation",
"target_id",
"direction",
"category",
"provenance"
],
"properties": {
"source_id": { "type": "string", "minLength": 1 },
"relation": { "type": "string", "minLength": 1 },
"target_id": { "type": "string", "minLength": 1 },
"direction": { "enum": ["outgoing", "incoming"] },
"category": { "$ref": "#/$defs/category" },
"provenance": {
"const": "validated_graph_edge_without_source_provenance"
}
},
"additionalProperties": false
},
"evidence": {
"type": "object",
"required": [
"evidence_hash",
"role",
"reason_code",
"node_id",
"title",
"family",
"authority",
"status",
"tags",
"summary",
"text",
"estimated_tokens",
"source",
"depth",
"relationship_path",
"relationship_reasons",
"provenance_limitations"
],
"properties": {
"evidence_hash": { "$ref": "#/$defs/sha256" },
"role": { "enum": ["focus", "related"] },
"reason_code": {
"enum": ["exact_focus", "lexical_focus", "relationship_path"]
},
"node_id": { "type": "string", "minLength": 1 },
"title": { "type": "string" },
"family": { "type": "string" },
"authority": { "type": "string" },
"status": { "type": "string" },
"tags": {
"type": "array",
"items": { "type": "string" }
},
"summary": { "type": "string" },
"text": { "type": "string" },
"estimated_tokens": { "type": "integer", "minimum": 1 },
"source": {
"type": "object",
"required": ["path", "anchor", "content_hash"],
"properties": {
"path": { "type": "string", "minLength": 1 },
"anchor": { "type": ["string", "null"] },
"content_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"depth": { "type": "integer", "minimum": 0 },
"relationship_path": {
"type": "array",
"items": { "$ref": "#/$defs/relationship" }
},
"relationship_reasons": {
"type": "array",
"items": { "$ref": "#/$defs/relationship" },
"uniqueItems": true
},
"provenance_limitations": {
"const": [
"evidence_type_unavailable",
"extractor_identity_unavailable",
"relationship_provenance_unavailable",
"observation_time_unavailable"
]
}
},
"additionalProperties": false
},
"gap": {
"type": "object",
"required": [
"code",
"requirement_id",
"category",
"state",
"check_complete",
"detail"
],
"properties": {
"code": {
"enum": [
"focus_not_found",
"focus_ambiguous",
"category_not_declared",
"no_selected_evidence",
"evidence_incomplete",
"unclassified_relation"
]
},
"requirement_id": { "type": "string", "minLength": 1 },
"category": {
"oneOf": [
{ "$ref": "#/$defs/category" },
{ "type": "null" }
]
},
"state": { "enum": ["missing", "incomplete", "blocked", "limitation"] },
"check_complete": { "type": "boolean" },
"detail": { "type": "string", "minLength": 1, "maxLength": 1000 }
},
"additionalProperties": false
},
"omission": {
"type": "object",
"required": ["code", "subject", "detail_hash"],
"properties": {
"code": {
"enum": [
"result_limit",
"token_budget",
"response_limit",
"edge_examination_limit",
"unclassified_relation_limit"
]
},
"subject": { "type": "string", "minLength": 1, "maxLength": 256 },
"detail_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
}
},
"type": "object",
"required": [
"schema_version",
"state",
"task_kind",
"generation",
"plan",
"focus",
"evidence",
"gaps",
"omissions",
"summary",
"collection_hash",
"capsule_hash"
],
"properties": {
"schema_version": { "const": 1 },
"state": { "enum": ["complete", "incomplete", "blocked"] },
"task_kind": {
"enum": [
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release"
]
},
"generation": {
"type": "object",
"required": [
"project_id",
"project_root_fingerprint",
"adapter",
"revision",
"source_hash",
"index_schema_version"
],
"properties": {
"project_id": { "type": "string", "minLength": 1 },
"project_root_fingerprint": {
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1 },
"revision": { "type": "string", "minLength": 1 },
"source_hash": { "$ref": "#/$defs/sha256" },
"index_schema_version": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
"plan": { "$ref": "#/$defs/plan" },
"focus": {
"type": "object",
"required": ["state", "node_id", "candidate_count"],
"properties": {
"state": { "enum": ["resolved", "not_found", "ambiguous"] },
"node_id": { "type": ["string", "null"] },
"candidate_count": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"evidence": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/evidence" }
},
"gaps": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/gap" }
},
"omissions": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/omission" }
},
"summary": {
"type": "object",
"required": [
"evidence_count",
"gap_count",
"omission_count",
"selected_count",
"examined_edge_count",
"estimated_tokens",
"unclassified_relations"
],
"properties": {
"evidence_count": { "type": "integer", "minimum": 0 },
"gap_count": { "type": "integer", "minimum": 0 },
"omission_count": { "type": "integer", "minimum": 0 },
"selected_count": { "type": "integer", "minimum": 0 },
"examined_edge_count": { "type": "integer", "minimum": 0 },
"estimated_tokens": { "type": "integer", "minimum": 0 },
"unclassified_relations": {
"type": "array",
"maxItems": 10000,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
},
"page_evidence_count": { "type": "integer", "minimum": 0 },
"page_omission_count": { "type": "integer", "minimum": 0 },
"page_item_count": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"collection_hash": { "$ref": "#/$defs/sha256" },
"capsule_hash": { "$ref": "#/$defs/sha256" },
"page_state": { "enum": ["complete", "incomplete", "blocked"] },
"page_hash": { "$ref": "#/$defs/sha256" },
"pagination": { "$ref": "#/$defs/pagination" }
},
"additionalProperties": false
}

466
schemas/doctor-result.schema.json Normal file
View file

@ -0,0 +1,466 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/doctor-result-v1.json",
"title": "DocForge bounded read-only integration doctor result",
"$defs": {
"diagnostics": {
"type": "object",
"required": [
"schema_version",
"operation",
"outcome",
"elapsed_ns",
"stages",
"counters"
],
"properties": {
"schema_version": { "const": 1 },
"operation": { "const": "cli.doctor" },
"outcome": { "const": "ok" },
"elapsed_ns": { "type": "integer", "minimum": 0 },
"stages": {
"type": "object",
"maxProperties": 14,
"propertyNames": {
"enum": [
"source.generation",
"source.parse",
"adapter.projection",
"adapter.extract",
"index.check",
"index.synchronize",
"index.build",
"index.read",
"render.status",
"render.prepare",
"render.output_hash",
"visualization.status",
"viewer.manager",
"mcp.runtime_validation"
]
},
"additionalProperties": {
"type": "object",
"required": ["calls", "elapsed_ns"],
"properties": {
"calls": { "type": "integer", "minimum": 1 },
"elapsed_ns": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
}
},
"counters": {
"type": "object",
"required": [
"project_loads",
"source_files_parsed",
"source_bytes_parsed",
"adapter_projection_loads",
"adapter_source_extractions",
"source_generation_checks",
"index_checks",
"index_synchronizations",
"index_builds",
"render_prepare_calls",
"render_output_bytes_built",
"render_output_bytes_hashed",
"viewer_manager_requests"
],
"additionalProperties": {
"type": "integer",
"minimum": 0
},
"maxProperties": 13
}
},
"additionalProperties": false
}
},
"type": "object",
"required": [
"status",
"schema_version",
"doctor_state",
"client",
"project",
"config",
"summary",
"guarantees",
"checks"
],
"properties": {
"status": { "const": "ok" },
"schema_version": { "const": 1 },
"doctor_state": { "enum": ["healthy", "degraded", "unhealthy"] },
"client": { "enum": ["codex", "claude", "openclaw"] },
"project": {
"type": "object",
"required": [
"project_id",
"project_root",
"project_root_fingerprint",
"adapter"
],
"properties": {
"project_id": { "type": "string", "minLength": 1, "maxLength": 128 },
"project_root": { "type": "string", "minLength": 1, "maxLength": 4096 },
"project_root_fingerprint": {
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1, "maxLength": 256 }
},
"additionalProperties": false
},
"config": {
"type": "object",
"required": ["path", "server_name"],
"properties": {
"path": { "type": "string", "minLength": 1, "maxLength": 4096 },
"server_name": {
"type": ["string", "null"],
"maxLength": 256
}
},
"additionalProperties": false
},
"summary": {
"type": "object",
"required": ["passed", "warning", "failed", "skipped"],
"properties": {
"passed": { "type": "integer", "minimum": 0, "maximum": 14 },
"warning": { "type": "integer", "minimum": 0, "maximum": 14 },
"failed": { "type": "integer", "minimum": 0, "maximum": 14 },
"skipped": { "type": "integer", "minimum": 0, "maximum": 14 }
},
"additionalProperties": false
},
"guarantees": {
"type": "object",
"required": [
"read_only",
"project_loads",
"adapter_projection_loads",
"adapter_source_extractions",
"sqlite_opens",
"index_checks",
"index_synchronizations",
"index_builds",
"renders",
"viewer_operations",
"client_config_writes",
"configured_command_executions"
],
"properties": {
"read_only": { "const": true },
"project_loads": { "const": 0 },
"adapter_projection_loads": { "const": 0 },
"adapter_source_extractions": { "const": 0 },
"sqlite_opens": { "const": 0 },
"index_checks": { "const": 0 },
"index_synchronizations": { "const": 0 },
"index_builds": { "const": 0 },
"renders": { "const": 0 },
"viewer_operations": { "const": 0 },
"client_config_writes": { "const": 0 },
"configured_command_executions": { "const": 0 }
},
"additionalProperties": false
},
"checks": {
"type": "array",
"minItems": 14,
"maxItems": 14,
"items": {
"type": "object",
"required": ["check_id", "state", "code", "message", "details"],
"properties": {
"check_id": {
"enum": [
"project.binding",
"project.canonical_validation",
"client.driver",
"client.config",
"client.entry",
"server.executable",
"server.arguments",
"server.project_binding",
"policy.effective",
"policy.no_ast",
"client.timeouts",
"client.environment",
"client.tool_filter",
"derived.index"
]
},
"state": {
"enum": ["passed", "warning", "failed", "skipped"]
},
"code": {
"type": "string",
"minLength": 1,
"maxLength": 128
},
"message": {
"type": "string",
"minLength": 1,
"maxLength": 512
},
"details": {
"type": "object",
"maxProperties": 16,
"propertyNames": {
"type": "string",
"minLength": 1,
"maxLength": 128
},
"additionalProperties": {
"oneOf": [
{ "type": "string", "maxLength": 512 },
{ "type": "integer" },
{ "type": "boolean" },
{ "type": "null" },
{
"type": "array",
"maxItems": 16,
"items": {
"oneOf": [
{ "type": "string", "maxLength": 256 },
{ "type": "integer" },
{ "type": "boolean" },
{ "type": "null" }
]
}
}
]
}
}
},
"additionalProperties": false
}
},
"diagnostics": { "$ref": "#/$defs/diagnostics" }
},
"allOf": [
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "project.binding" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": {
"check_id": { "const": "project.canonical_validation" }
},
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "client.driver" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "client.config" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "client.entry" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "server.executable" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "server.arguments" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": {
"check_id": { "const": "server.project_binding" }
},
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "policy.effective" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "policy.no_ast" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "client.timeouts" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "client.environment" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "client.tool_filter" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"properties": {
"checks": {
"contains": {
"properties": { "check_id": { "const": "derived.index" } },
"required": ["check_id"]
},
"minContains": 1,
"maxContains": 1
}
}
},
{
"if": {
"properties": { "doctor_state": { "const": "healthy" } },
"required": ["doctor_state"]
},
"then": {
"properties": {
"summary": {
"properties": {
"warning": { "const": 0 },
"failed": { "const": 0 }
}
}
}
}
},
{
"if": {
"properties": { "doctor_state": { "const": "degraded" } },
"required": ["doctor_state"]
},
"then": {
"properties": {
"summary": {
"properties": {
"warning": { "minimum": 1 },
"failed": { "const": 0 }
}
}
}
}
},
{
"if": {
"properties": { "doctor_state": { "const": "unhealthy" } },
"required": ["doctor_state"]
},
"then": {
"properties": {
"summary": {
"properties": {
"failed": { "minimum": 1 }
}
}
}
}
}
],
"additionalProperties": false
}

430
schemas/generation-diff-page.schema.json Normal file
View file

@ -0,0 +1,430 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/generation-diff-page-v1.json",
"title": "DocForge bounded latest-generation diff page",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"generation": {
"type": "object",
"required": [
"revision",
"source_hash",
"node_count",
"node_hash",
"edge_count",
"edge_hash",
"index_schema_version"
],
"properties": {
"revision": { "type": "string", "minLength": 1 },
"source_hash": { "$ref": "#/$defs/sha256" },
"node_count": { "type": "integer", "minimum": 0 },
"node_hash": { "$ref": "#/$defs/sha256" },
"edge_count": { "type": "integer", "minimum": 0 },
"edge_hash": { "$ref": "#/$defs/sha256" },
"index_schema_version": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
"summary": {
"type": "object",
"required": [
"nodes_added",
"nodes_removed",
"nodes_changed",
"edges_added",
"edges_removed",
"total_changes"
],
"properties": {
"nodes_added": { "type": "integer", "minimum": 0 },
"nodes_removed": { "type": "integer", "minimum": 0 },
"nodes_changed": { "type": "integer", "minimum": 0 },
"edges_added": { "type": "integer", "minimum": 0 },
"edges_removed": { "type": "integer", "minimum": 0 },
"total_changes": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"node_item": {
"type": "object",
"required": [
"entity",
"change",
"node_id",
"before_content_hash",
"after_content_hash",
"before_source_path",
"after_source_path",
"before_node_hash",
"after_node_hash",
"changed_fields",
"item_hash"
],
"properties": {
"entity": { "const": "node" },
"change": { "enum": ["added", "removed", "changed"] },
"node_id": { "type": "string", "minLength": 1 },
"before_content_hash": {
"oneOf": [{ "$ref": "#/$defs/sha256" }, { "type": "null" }]
},
"after_content_hash": {
"oneOf": [{ "$ref": "#/$defs/sha256" }, { "type": "null" }]
},
"before_source_path": { "type": ["string", "null"] },
"after_source_path": { "type": ["string", "null"] },
"before_node_hash": {
"oneOf": [{ "$ref": "#/$defs/sha256" }, { "type": "null" }]
},
"after_node_hash": {
"oneOf": [{ "$ref": "#/$defs/sha256" }, { "type": "null" }]
},
"changed_fields": {
"type": "array",
"uniqueItems": true,
"items": {
"enum": [
"title",
"family",
"authority",
"status",
"tags",
"summary",
"content",
"source_path",
"source_anchor",
"content_hash"
]
}
},
"item_hash": { "$ref": "#/$defs/sha256" }
},
"allOf": [
{
"if": {
"properties": { "change": { "const": "added" } },
"required": ["change"]
},
"then": {
"properties": {
"before_content_hash": { "type": "null" },
"before_source_path": { "type": "null" },
"before_node_hash": { "type": "null" },
"after_content_hash": { "$ref": "#/$defs/sha256" },
"after_source_path": { "type": "string", "minLength": 1 },
"after_node_hash": { "$ref": "#/$defs/sha256" },
"changed_fields": { "maxItems": 0 }
}
}
},
{
"if": {
"properties": { "change": { "const": "removed" } },
"required": ["change"]
},
"then": {
"properties": {
"before_content_hash": { "$ref": "#/$defs/sha256" },
"before_source_path": { "type": "string", "minLength": 1 },
"before_node_hash": { "$ref": "#/$defs/sha256" },
"after_content_hash": { "type": "null" },
"after_source_path": { "type": "null" },
"after_node_hash": { "type": "null" },
"changed_fields": { "maxItems": 0 }
}
}
},
{
"if": {
"properties": { "change": { "const": "changed" } },
"required": ["change"]
},
"then": {
"properties": {
"before_content_hash": { "$ref": "#/$defs/sha256" },
"before_source_path": { "type": "string", "minLength": 1 },
"before_node_hash": { "$ref": "#/$defs/sha256" },
"after_content_hash": { "$ref": "#/$defs/sha256" },
"after_source_path": { "type": "string", "minLength": 1 },
"after_node_hash": { "$ref": "#/$defs/sha256" },
"changed_fields": { "minItems": 1 }
}
}
}
],
"additionalProperties": false
},
"edge_item": {
"type": "object",
"required": [
"entity",
"change",
"source_id",
"relation",
"target_id",
"item_hash"
],
"properties": {
"entity": { "const": "edge" },
"change": { "enum": ["added", "removed"] },
"source_id": { "type": "string", "minLength": 1 },
"relation": { "type": "string", "minLength": 1 },
"target_id": { "type": "string", "minLength": 1 },
"item_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"receipt_header": {
"type": "object",
"required": [
"schema_version",
"diff_semantics_version",
"project_id",
"project_root_fingerprint",
"adapter",
"kind",
"reason",
"from_generation",
"to_generation",
"summary",
"full_item_count",
"retained_item_count",
"details_truncated",
"truncation_reason",
"full_collection_hash",
"retained_collection_hash",
"index_signature",
"stored_receipt_hash"
],
"properties": {
"schema_version": { "const": 1 },
"diff_semantics_version": { "const": 1 },
"project_id": { "type": "string", "minLength": 1 },
"project_root_fingerprint": {
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1 },
"kind": { "enum": ["baseline", "transition"] },
"reason": {
"enum": [
null,
"no_predecessor",
"predecessor_unsafe",
"predecessor_unsupported_schema",
"predecessor_foreign",
"predecessor_policy_incompatible",
"predecessor_corrupt",
"predecessor_unattested",
"predecessor_changed",
"no_meaningful_transition"
]
},
"from_generation": {
"oneOf": [{ "$ref": "#/$defs/generation" }, { "type": "null" }]
},
"to_generation": { "$ref": "#/$defs/generation" },
"summary": { "$ref": "#/$defs/summary" },
"full_item_count": { "type": "integer", "minimum": 0 },
"retained_item_count": {
"type": "integer",
"minimum": 0,
"maximum": 1000
},
"details_truncated": { "type": "boolean" },
"truncation_reason": {
"enum": [null, "receipt_item_limit", "receipt_byte_limit"]
},
"full_collection_hash": { "$ref": "#/$defs/sha256" },
"retained_collection_hash": { "$ref": "#/$defs/sha256" },
"index_signature": {
"type": "object",
"required": ["device", "inode", "size", "mtime_ns", "ctime_ns"],
"properties": {
"device": { "type": "integer", "minimum": 0 },
"inode": { "type": "integer", "minimum": 0 },
"size": { "type": "integer", "minimum": 0 },
"mtime_ns": { "type": "integer", "minimum": 0 },
"ctime_ns": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"stored_receipt_hash": { "$ref": "#/$defs/sha256" }
},
"allOf": [
{
"if": {
"properties": { "kind": { "const": "baseline" } },
"required": ["kind"]
},
"then": {
"properties": {
"from_generation": { "type": "null" },
"reason": { "not": { "type": "null" } },
"summary": {
"properties": {
"nodes_added": { "const": 0 },
"nodes_removed": { "const": 0 },
"nodes_changed": { "const": 0 },
"edges_added": { "const": 0 },
"edges_removed": { "const": 0 },
"total_changes": { "const": 0 }
}
},
"full_item_count": { "const": 0 },
"retained_item_count": { "const": 0 },
"details_truncated": { "const": false },
"truncation_reason": { "const": null },
"full_collection_hash": {
"const": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945"
},
"retained_collection_hash": {
"const": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945"
}
}
},
"else": {
"properties": {
"from_generation": { "$ref": "#/$defs/generation" },
"reason": { "type": "null" }
}
}
},
{
"if": {
"properties": { "truncation_reason": { "const": null } },
"required": ["truncation_reason"]
},
"then": {
"properties": {
"details_truncated": { "const": false },
"full_item_count": { "maximum": 1000 }
}
}
},
{
"if": {
"properties": {
"truncation_reason": { "const": "receipt_item_limit" }
},
"required": ["truncation_reason"]
},
"then": {
"properties": {
"details_truncated": { "const": true },
"full_item_count": { "minimum": 1001 },
"retained_item_count": { "const": 1000 }
}
}
},
{
"if": {
"properties": {
"truncation_reason": { "const": "receipt_byte_limit" }
},
"required": ["truncation_reason"]
},
"then": {
"properties": {
"details_truncated": { "const": true }
}
}
}
],
"additionalProperties": false
},
"page_body": {
"type": "object",
"required": [
"page_schema_version",
"receipt_header",
"items",
"omissions",
"page_hash"
],
"properties": {
"page_schema_version": { "const": 1 },
"receipt_header": { "$ref": "#/$defs/receipt_header" },
"items": {
"type": "array",
"maxItems": 1000,
"items": {
"oneOf": [
{ "$ref": "#/$defs/node_item" },
{ "$ref": "#/$defs/edge_item" }
]
}
},
"omissions": {
"type": "array",
"maxItems": 1,
"items": {
"type": "object",
"required": ["code", "item_hash"],
"properties": {
"code": { "const": "response_limit" },
"item_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
}
},
"page_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"pagination": {
"type": "object",
"required": [
"schema_version",
"kind",
"returned_count",
"limit",
"total_count",
"has_more",
"next_cursor"
],
"properties": {
"schema_version": { "const": 1 },
"kind": { "const": "generation-diff.items" },
"returned_count": { "type": "integer", "minimum": 0 },
"limit": { "type": "integer", "minimum": 1, "maximum": 1000 },
"total_count": { "type": "integer", "minimum": 0, "maximum": 1000 },
"has_more": { "type": "boolean" },
"next_cursor": {
"type": ["string", "null"],
"minLength": 1,
"maxLength": 8192
}
},
"allOf": [
{
"if": {
"properties": { "has_more": { "const": true } },
"required": ["has_more"]
},
"then": {
"properties": {
"next_cursor": { "type": "string", "minLength": 1 }
}
},
"else": {
"properties": {
"next_cursor": { "type": "null" }
}
}
}
],
"additionalProperties": false
}
},
"type": "object",
"required": ["generation_diff", "pagination"],
"properties": {
"generation_diff": { "$ref": "#/$defs/page_body" },
"pagination": { "$ref": "#/$defs/pagination" }
},
"additionalProperties": false
}

363
schemas/generation-diff.schema.json Normal file
View file

@ -0,0 +1,363 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/generation-diff-v1.json",
"title": "DocForge latest primary-graph generation diff receipt",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"generation": {
"type": "object",
"required": [
"revision",
"source_hash",
"node_count",
"node_hash",
"edge_count",
"edge_hash",
"index_schema_version"
],
"properties": {
"revision": { "type": "string", "minLength": 1 },
"source_hash": { "$ref": "#/$defs/sha256" },
"node_count": { "type": "integer", "minimum": 0 },
"node_hash": { "$ref": "#/$defs/sha256" },
"edge_count": { "type": "integer", "minimum": 0 },
"edge_hash": { "$ref": "#/$defs/sha256" },
"index_schema_version": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
"node_item": {
"type": "object",
"required": [
"entity",
"change",
"node_id",
"before_content_hash",
"after_content_hash",
"before_source_path",
"after_source_path",
"before_node_hash",
"after_node_hash",
"changed_fields",
"item_hash"
],
"properties": {
"entity": { "const": "node" },
"change": { "enum": ["added", "removed", "changed"] },
"node_id": { "type": "string", "minLength": 1 },
"before_content_hash": {
"oneOf": [
{ "$ref": "#/$defs/sha256" },
{ "type": "null" }
]
},
"after_content_hash": {
"oneOf": [
{ "$ref": "#/$defs/sha256" },
{ "type": "null" }
]
},
"before_source_path": { "type": ["string", "null"] },
"after_source_path": { "type": ["string", "null"] },
"before_node_hash": {
"oneOf": [
{ "$ref": "#/$defs/sha256" },
{ "type": "null" }
]
},
"after_node_hash": {
"oneOf": [
{ "$ref": "#/$defs/sha256" },
{ "type": "null" }
]
},
"changed_fields": {
"type": "array",
"items": {
"enum": [
"title",
"family",
"authority",
"status",
"tags",
"summary",
"content",
"source_path",
"source_anchor",
"content_hash"
]
},
"uniqueItems": true
},
"item_hash": { "$ref": "#/$defs/sha256" }
},
"allOf": [
{
"if": {
"properties": { "change": { "const": "added" } },
"required": ["change"]
},
"then": {
"properties": {
"before_content_hash": { "type": "null" },
"before_source_path": { "type": "null" },
"before_node_hash": { "type": "null" },
"after_content_hash": { "$ref": "#/$defs/sha256" },
"after_source_path": { "type": "string", "minLength": 1 },
"after_node_hash": { "$ref": "#/$defs/sha256" },
"changed_fields": { "maxItems": 0 }
}
}
},
{
"if": {
"properties": { "change": { "const": "removed" } },
"required": ["change"]
},
"then": {
"properties": {
"before_content_hash": { "$ref": "#/$defs/sha256" },
"before_source_path": { "type": "string", "minLength": 1 },
"before_node_hash": { "$ref": "#/$defs/sha256" },
"after_content_hash": { "type": "null" },
"after_source_path": { "type": "null" },
"after_node_hash": { "type": "null" },
"changed_fields": { "maxItems": 0 }
}
}
},
{
"if": {
"properties": { "change": { "const": "changed" } },
"required": ["change"]
},
"then": {
"properties": {
"before_content_hash": { "$ref": "#/$defs/sha256" },
"before_source_path": { "type": "string", "minLength": 1 },
"before_node_hash": { "$ref": "#/$defs/sha256" },
"after_content_hash": { "$ref": "#/$defs/sha256" },
"after_source_path": { "type": "string", "minLength": 1 },
"after_node_hash": { "$ref": "#/$defs/sha256" },
"changed_fields": { "minItems": 1 }
}
}
}
],
"additionalProperties": false
},
"edge_item": {
"type": "object",
"required": [
"entity",
"change",
"source_id",
"relation",
"target_id",
"item_hash"
],
"properties": {
"entity": { "const": "edge" },
"change": { "enum": ["added", "removed"] },
"source_id": { "type": "string", "minLength": 1 },
"relation": { "type": "string", "minLength": 1 },
"target_id": { "type": "string", "minLength": 1 },
"item_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
}
},
"type": "object",
"required": [
"schema_version",
"diff_semantics_version",
"project_id",
"project_root_fingerprint",
"adapter",
"kind",
"reason",
"from_generation",
"to_generation",
"summary",
"items",
"full_item_count",
"retained_item_count",
"details_truncated",
"truncation_reason",
"full_collection_hash",
"retained_collection_hash",
"index_signature",
"receipt_hash"
],
"properties": {
"schema_version": { "const": 1 },
"diff_semantics_version": { "const": 1 },
"project_id": { "type": "string", "minLength": 1 },
"project_root_fingerprint": {
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1 },
"kind": { "enum": ["baseline", "transition"] },
"reason": {
"enum": [
null,
"no_predecessor",
"predecessor_unsafe",
"predecessor_unsupported_schema",
"predecessor_foreign",
"predecessor_policy_incompatible",
"predecessor_corrupt",
"predecessor_unattested",
"predecessor_changed",
"no_meaningful_transition"
]
},
"from_generation": {
"oneOf": [
{ "$ref": "#/$defs/generation" },
{ "type": "null" }
]
},
"to_generation": { "$ref": "#/$defs/generation" },
"summary": {
"type": "object",
"required": [
"nodes_added",
"nodes_removed",
"nodes_changed",
"edges_added",
"edges_removed",
"total_changes"
],
"properties": {
"nodes_added": { "type": "integer", "minimum": 0 },
"nodes_removed": { "type": "integer", "minimum": 0 },
"nodes_changed": { "type": "integer", "minimum": 0 },
"edges_added": { "type": "integer", "minimum": 0 },
"edges_removed": { "type": "integer", "minimum": 0 },
"total_changes": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"items": {
"type": "array",
"maxItems": 1000,
"items": {
"oneOf": [
{ "$ref": "#/$defs/node_item" },
{ "$ref": "#/$defs/edge_item" }
]
}
},
"full_item_count": { "type": "integer", "minimum": 0 },
"retained_item_count": {
"type": "integer",
"minimum": 0,
"maximum": 1000
},
"details_truncated": { "type": "boolean" },
"truncation_reason": {
"enum": [null, "receipt_item_limit", "receipt_byte_limit"]
},
"full_collection_hash": { "$ref": "#/$defs/sha256" },
"retained_collection_hash": { "$ref": "#/$defs/sha256" },
"index_signature": {
"type": "object",
"required": ["device", "inode", "size", "mtime_ns", "ctime_ns"],
"properties": {
"device": { "type": "integer", "minimum": 0 },
"inode": { "type": "integer", "minimum": 0 },
"size": { "type": "integer", "minimum": 0 },
"mtime_ns": { "type": "integer", "minimum": 0 },
"ctime_ns": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"receipt_hash": { "$ref": "#/$defs/sha256" }
},
"allOf": [
{
"if": {
"properties": { "kind": { "const": "baseline" } },
"required": ["kind"]
},
"then": {
"properties": {
"from_generation": { "type": "null" },
"reason": { "not": { "type": "null" } },
"summary": {
"properties": {
"nodes_added": { "const": 0 },
"nodes_removed": { "const": 0 },
"nodes_changed": { "const": 0 },
"edges_added": { "const": 0 },
"edges_removed": { "const": 0 },
"total_changes": { "const": 0 }
}
},
"full_item_count": { "const": 0 },
"retained_item_count": { "const": 0 },
"details_truncated": { "const": false },
"truncation_reason": { "const": null },
"full_collection_hash": {
"const": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945"
},
"retained_collection_hash": {
"const": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945"
}
}
},
"else": {
"properties": {
"from_generation": { "$ref": "#/$defs/generation" },
"reason": { "type": "null" }
}
}
},
{
"if": {
"properties": { "truncation_reason": { "const": null } },
"required": ["truncation_reason"]
},
"then": {
"properties": {
"details_truncated": { "const": false },
"full_item_count": { "maximum": 1000 }
}
}
},
{
"if": {
"properties": {
"truncation_reason": { "const": "receipt_item_limit" }
},
"required": ["truncation_reason"]
},
"then": {
"properties": {
"details_truncated": { "const": true },
"full_item_count": { "minimum": 1001 },
"retained_item_count": { "const": 1000 }
}
}
},
{
"if": {
"properties": {
"truncation_reason": { "const": "receipt_byte_limit" }
},
"required": ["truncation_reason"]
},
"then": {
"properties": {
"details_truncated": { "const": true }
}
}
}
],
"additionalProperties": false
}

62
schemas/policy.schema.json Normal file
View file

@ -0,0 +1,62 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/policy-v1.json",
"title": "DocForge effective process policy",
"type": "object",
"required": [
"schema_version",
"capability_mode",
"capability_source",
"adapter_evolution",
"ast_analysis",
"logic_indexing",
"synchronization",
"integrity",
"manual_render",
"graph_render",
"live_viewer",
"profiling",
"blocked_tools",
"prohibitions",
"precedence"
],
"properties": {
"schema_version": { "const": 1 },
"capability_mode": {
"enum": ["read", "proposal", "application", "operator"]
},
"capability_source": {
"enum": ["factory_default", "explicit"]
},
"adapter_evolution": { "enum": ["allowed", "preserve"] },
"ast_analysis": { "enum": ["allowed", "forbidden"] },
"logic_indexing": { "enum": ["full", "off"] },
"synchronization": { "const": "automatic" },
"integrity": { "const": "validated" },
"manual_render": { "enum": ["auto", "explicit", "disabled"] },
"graph_render": { "const": "disabled" },
"live_viewer": { "const": "on-demand" },
"profiling": { "enum": ["enabled", "disabled"] },
"blocked_tools": {
"type": "array",
"maxItems": 64,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
},
"prohibitions": {
"type": "array",
"maxItems": 64,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
},
"precedence": {
"const": [
"core_safety",
"explicit_binding",
"no_ast_shorthand",
"resource_availability"
]
}
},
"additionalProperties": false
}

21
schemas/result.schema.json
View file

@ -19,6 +19,7 @@
"enum": [
"test",
"benchmark.m1",
"benchmark.m2",
"mcp.invoke",
"mcp.bootstrap",
"mcp.sync",
@ -32,6 +33,8 @@
"mcp.dependencies",
"mcp.impact",
"mcp.context",
"mcp.task_context",
"mcp.generation_diff",
"mcp.validate_project",
"mcp.render_status",
"mcp.visualize",
@ -54,6 +57,9 @@
"cli.dependencies",
"cli.impact",
"cli.context",
"cli.generation-diff",
"cli.configure",
"cli.doctor",
"cli.render",
"cli.render-status",
"cli.preview",
@ -149,6 +155,8 @@
"kind": {
"enum": [
"context.items",
"task-context.items",
"generation-diff.items",
"changeset.list",
"changeset.inspect",
"changeset.validate",
@ -214,7 +222,18 @@
"properties": {
"code": { "type": "string", "minLength": 1 },
"message": { "type": "string", "minLength": 1 },
"details": { "type": "object" }
"details": { "type": "object" },
"remediation": {
"type": "object",
"required": ["retryable"],
"properties": {
"retryable": { "type": "boolean" },
"action": { "type": "string", "minLength": 1 },
"tool": { "type": "string", "minLength": 1 },
"arguments": { "type": "object" }
},
"additionalProperties": false
}
},
"additionalProperties": false
}

71
src/docforge/_fs_safety.py Normal file
View file

@ -0,0 +1,71 @@
"""Internal directory binding helpers for disposable publication paths."""
from __future__ import annotations
import os
import stat
from pathlib import Path
from .errors import DocForgeError
def open_bound_directory(path: Path) -> int:
"""Open one real directory and bind its current inode for later operations."""
try:
path_status = path.lstat()
if (
stat.S_ISLNK(path_status.st_mode)
or not stat.S_ISDIR(path_status.st_mode)
or path.resolve(strict=True) != path
):
raise DocForgeError(
"path_escape",
"Derived cache root is not a safe real directory",
)
directory_fd = os.open(path, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
except FileNotFoundError as error:
raise DocForgeError(
"missing_index",
"Derived cache root does not exist",
) from error
except OSError as error:
raise DocForgeError(
"path_escape",
"Derived cache root cannot be opened safely",
) from error
try:
opened_status = os.fstat(directory_fd)
if opened_status.st_dev != path_status.st_dev or opened_status.st_ino != path_status.st_ino:
raise DocForgeError(
"path_escape",
"Derived cache root changed while opening",
)
except Exception:
os.close(directory_fd)
raise
return directory_fd
def require_bound_directory(path: Path, directory_fd: int) -> None:
"""Require a path to still name the exact opened real directory."""
try:
path_status = path.lstat()
opened_status = os.fstat(directory_fd)
if (
stat.S_ISLNK(path_status.st_mode)
or not stat.S_ISDIR(path_status.st_mode)
or path.resolve(strict=True) != path
or opened_status.st_dev != path_status.st_dev
or opened_status.st_ino != path_status.st_ino
):
raise DocForgeError(
"path_escape",
"Derived cache root changed during publication",
)
except FileNotFoundError as error:
raise DocForgeError(
"path_escape",
"Derived cache root disappeared during publication",
) from error

69
src/docforge/cli.py
View file

@ -9,7 +9,9 @@ import webbrowser
from pathlib import Path
from .application import CanonicalApplicationService, GenericCanonicalApplier
from .client_config import CLIENT_NAMES, generate_client_configuration
from .context import compile_context
from .doctor import run_doctor
from .errors import DocForgeError
from .index import ProjectIndex
from .onboarding import assess_project, scaffold_project
@ -21,13 +23,33 @@ from .viewer_manager import ViewerManagerClient
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="docforge")
parser.add_argument("--project-root", type=Path, required=True)
parser.add_argument("--project-root", type=Path)
parser.add_argument(
"--diagnostics",
action="store_true",
help="Attach bounded request-local stage timings and counters",
)
commands = parser.add_subparsers(dest="command", required=True)
configure = commands.add_parser("configure")
configure.add_argument("client", choices=CLIENT_NAMES)
configure.add_argument("--project", type=Path, required=True)
configure.add_argument("--name")
configure.add_argument(
"--capability-mode",
choices=("read", "proposal", "application"),
default="read",
)
configure.add_argument("--proposal-writer")
configure.add_argument("--canonical-applier")
configure.add_argument("--no-ast", action="store_true")
configure.add_argument("--startup-timeout", type=int, default=30)
configure.add_argument("--tool-timeout", type=int, default=300)
configure.add_argument("--output", type=Path)
doctor = commands.add_parser("doctor")
doctor.add_argument("--client", choices=CLIENT_NAMES, required=True)
doctor.add_argument("--project", type=Path)
doctor.add_argument("--config", type=Path)
doctor.add_argument("--server-name")
onboard = commands.add_parser("onboard")
onboard.add_argument("--language", action="append", default=[])
onboard.add_argument("--scaffold", action="store_true")
@ -65,6 +87,9 @@ def _parser() -> argparse.ArgumentParser:
context.add_argument("--budget", type=int)
context.add_argument("--limit", type=int)
context.add_argument("--cursor")
generation_diff = commands.add_parser("generation-diff")
generation_diff.add_argument("--limit", type=int)
generation_diff.add_argument("--cursor")
render = commands.add_parser("render")
render.add_argument("view_id")
render_status = commands.add_parser("render-status")
@ -89,6 +114,33 @@ def _parser() -> argparse.ArgumentParser:
def _run(arguments: argparse.Namespace) -> dict[str, object]:
if arguments.command == "configure":
project = Project.open(arguments.project)
return generate_client_configuration(
project,
arguments.client,
server_name=arguments.name,
capability_mode=arguments.capability_mode,
proposal_writer=arguments.proposal_writer,
canonical_applier=arguments.canonical_applier,
no_ast=arguments.no_ast,
startup_timeout=arguments.startup_timeout,
tool_timeout=arguments.tool_timeout,
output=arguments.output,
)
if arguments.command == "doctor":
root = arguments.project or arguments.project_root or Path.cwd()
return run_doctor(
Project.open(root),
arguments.client,
config_path=arguments.config,
server_name=arguments.server_name,
)
if arguments.project_root is None:
raise DocForgeError(
"missing_project_root",
"This command requires --project-root",
)
if arguments.command == "onboard":
languages = tuple(arguments.language)
if arguments.scaffold:
@ -187,6 +239,16 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
cursor=arguments.cursor,
)
return compile_context(index, arguments.profile, arguments.budget)
if arguments.command == "generation-diff":
from .mcp_server import DocForgeService
return DocForgeService(
project,
capability_mode_name="read",
).generation_diff(
limit=arguments.limit,
cursor=arguments.cursor,
)
if arguments.command == "render":
return RenderService(project).render(arguments.view_id)
if arguments.command == "render-status":
@ -237,13 +299,14 @@ def main(argv: list[str] | None = None) -> int:
) as collector:
try:
result = _run(arguments)
code = 0
doctor_state = result.get("doctor_state")
code = 2 if doctor_state == "unhealthy" else (1 if doctor_state == "degraded" else 0)
except DocForgeError as error:
result = {"status": "error", "error": error.as_dict()}
code = 2
if collector is not None:
result["diagnostics"] = collector.as_dict(
outcome="ok" if code == 0 else "error",
outcome="ok" if result.get("status") == "ok" else "error",
)
print(json.dumps(result, sort_keys=True, indent=2))
return code

937
src/docforge/client_config.py Normal file
View file

@ -0,0 +1,937 @@
"""Deterministic, explicit client-configuration plans for DocForge MCP."""
from __future__ import annotations
import hashlib
import json
import os
import re
import secrets
import stat
import subprocess
import sys
from collections.abc import Callable
from contextlib import suppress
from dataclasses import dataclass
from pathlib import Path
from typing import Literal, cast
from .changeset_contract import document_hash
from .errors import DocForgeError
from .models import ProjectService
from .policy import CapabilityMode, compose_effective_policy
from .project import project_root_fingerprint, validate_descriptor_binding
ClientName = Literal["codex", "claude", "openclaw"]
CLIENT_NAMES: tuple[ClientName, ...] = ("codex", "claude", "openclaw")
MAX_CLIENT_FRAGMENT_BYTES = 1_000_000
GENERATED_CAPABILITY_MODES: tuple[CapabilityMode, ...] = (
"read",
"proposal",
"application",
)
_SERVER_NAME = re.compile(r"[a-z0-9][a-z0-9_-]{0,63}")
@dataclass(frozen=True)
class _FileIdentity:
device: int
inode: int
mode: int
size: int
mtime_ns: int
ctime_ns: int
uid: int
link_count: int
def _file_identity(status: os.stat_result) -> _FileIdentity:
return _FileIdentity(
device=status.st_dev,
inode=status.st_ino,
mode=status.st_mode,
size=status.st_size,
mtime_ns=status.st_mtime_ns,
ctime_ns=status.st_ctime_ns,
uid=status.st_uid,
link_count=status.st_nlink,
)
def _client_name(value: str) -> ClientName:
if value not in CLIENT_NAMES:
raise DocForgeError(
"unsupported_client",
"Client configuration target is unsupported",
client=value,
allowed=list(CLIENT_NAMES),
)
return value
def _capability_mode(value: str) -> CapabilityMode:
if value not in GENERATED_CAPABILITY_MODES:
raise DocForgeError(
"invalid_capability_mode",
"Generated configuration supports read, proposal, or application mode",
capability_mode=value,
allowed=list(GENERATED_CAPABILITY_MODES),
)
return value
def _bounded_seconds(value: int, *, field: str, maximum: int) -> int:
if type(value) is not int or value < 1 or value > maximum:
raise DocForgeError(
"invalid_timeout",
"Client timeout is outside the supported range",
field=field,
minimum=1,
maximum=maximum,
)
return value
def _default_server_name(project_id: str, fingerprint: str) -> str:
prefix = re.sub(r"[^a-z0-9_-]+", "-", project_id.lower()).strip("-_")
prefix = prefix or "project"
suffix = f"-{fingerprint}"
available = 64 - len("docforge-") - len(suffix)
return f"docforge-{prefix[:available]}{suffix}"
def _validated_server_name(value: str | None, *, project_id: str, fingerprint: str) -> str:
selected = value or _default_server_name(project_id, fingerprint)
if _SERVER_NAME.fullmatch(selected) is None:
raise DocForgeError(
"invalid_server_name",
"Generated server name must be a stable lowercase client identifier",
pattern=_SERVER_NAME.pattern,
maximum_length=64,
)
return selected
def _toml_string(value: str) -> str:
return json.dumps(value, ensure_ascii=False)
def _toml_array(values: list[str]) -> str:
return "[" + ", ".join(_toml_string(value) for value in values) + "]"
def _artifact(
client: ClientName,
*,
server_name: str,
command: str,
arguments: list[str],
startup_timeout: int,
tool_timeout: int,
) -> tuple[str, str, str | None]:
if client == "codex":
content = "\n".join(
(
f'[mcp_servers."{server_name}"]',
f"command = {_toml_string(command)}",
f"args = {_toml_array(arguments)}",
"env = {}",
f"startup_timeout_sec = {startup_timeout}",
f"tool_timeout_sec = {tool_timeout}",
"",
)
)
return "codex-toml-fragment-v1", content, None
if client == "openclaw":
content = (
json.dumps(
{
"mcp": {
"servers": {
server_name: {
"args": arguments,
"command": command,
"connectTimeout": startup_timeout,
"env": {},
"supportsParallelToolCalls": False,
"timeout": tool_timeout,
}
}
}
},
ensure_ascii=False,
indent=2,
sort_keys=True,
)
+ "\n"
)
return "openclaw-json-fragment-v1", content, None
content = (
json.dumps(
{
"mcpServers": {
server_name: {
"args": arguments,
"command": command,
"env": {},
}
}
},
ensure_ascii=False,
indent=2,
sort_keys=True,
)
+ "\n"
)
return (
"claude-json-fragment-v1",
content,
"Claude per-server timeout representation is not yet verified.",
)
def _signature(
directory_fd: int,
name: str,
) -> _FileIdentity | None:
try:
status = os.stat(name, dir_fd=directory_fd, follow_symlinks=False)
except FileNotFoundError:
return None
except OSError as error:
raise DocForgeError(
"unsafe_output",
"Configuration output cannot be inspected safely",
) from error
if stat.S_ISLNK(status.st_mode) or not stat.S_ISREG(status.st_mode):
raise DocForgeError(
"unsafe_output",
"Configuration output must be a regular file and not a symbolic link",
)
return _file_identity(status)
def _parent_binding_current(path: Path, directory_fd: int) -> bool:
try:
before = path.lstat()
resolved = path.resolve(strict=True)
after = path.lstat()
opened = os.fstat(directory_fd)
return (
not stat.S_ISLNK(before.st_mode)
and stat.S_ISDIR(before.st_mode)
and resolved == path
and (before.st_dev, before.st_ino, before.st_mode)
== (after.st_dev, after.st_ino, after.st_mode)
== (opened.st_dev, opened.st_ino, opened.st_mode)
)
except OSError:
return False
def _require_parent_binding(path: Path, directory_fd: int) -> None:
if not _parent_binding_current(path, directory_fd):
raise DocForgeError(
"output_changed",
"Configuration output parent changed during publication",
)
def _bound_parent(path: Path) -> tuple[Path, int]:
absolute = Path(os.path.abspath(path.expanduser()))
parent = absolute.parent
try:
parent_status = parent.lstat()
resolved = parent.resolve(strict=True)
except OSError as error:
raise DocForgeError(
"invalid_output",
"Configuration output parent does not exist",
) from error
if (
stat.S_ISLNK(parent_status.st_mode)
or not stat.S_ISDIR(parent_status.st_mode)
or resolved != parent
or absolute.name in {"", ".", ".."}
):
raise DocForgeError(
"unsafe_output",
"Configuration output parent must be one real non-symlinked directory",
)
try:
directory_fd = os.open(
parent,
os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW,
)
except OSError as error:
raise DocForgeError(
"unsafe_output",
"Configuration output parent cannot be opened safely",
) from error
opened = os.fstat(directory_fd)
if opened.st_dev != parent_status.st_dev or opened.st_ino != parent_status.st_ino:
with suppress(OSError):
os.close(directory_fd)
raise DocForgeError(
"output_changed",
"Configuration output parent changed while it was opened",
)
return absolute, directory_fd
def _read_existing(
directory_fd: int,
name: str,
signature: _FileIdentity,
) -> bytes:
if signature.size > MAX_CLIENT_FRAGMENT_BYTES:
raise DocForgeError(
"output_oversized",
"Existing configuration output exceeds the bounded fragment limit",
maximum_bytes=MAX_CLIENT_FRAGMENT_BYTES,
)
try:
descriptor = os.open(
name,
os.O_RDONLY | os.O_NOFOLLOW,
dir_fd=directory_fd,
)
except OSError as error:
raise DocForgeError(
"unsafe_output",
"Configuration output cannot be opened safely",
) from error
try:
opened = os.fstat(descriptor)
opened_signature = _file_identity(opened)
if opened_signature != signature:
raise DocForgeError(
"output_changed",
"Configuration output changed while it was opened",
)
remaining = MAX_CLIENT_FRAGMENT_BYTES + 1
chunks: list[bytes] = []
while remaining:
chunk = os.read(descriptor, min(65_536, remaining))
if not chunk:
break
chunks.append(chunk)
remaining -= len(chunk)
raw = b"".join(chunks)
finally:
with suppress(OSError):
os.close(descriptor)
if len(raw) > MAX_CLIENT_FRAGMENT_BYTES or _signature(directory_fd, name) != signature:
raise DocForgeError(
"output_changed",
"Configuration output changed while it was read",
)
return raw
def _private_existing(identity: _FileIdentity) -> bool:
return (
identity.uid == os.geteuid()
and stat.S_IMODE(identity.mode) & 0o077 == 0
and identity.link_count == 1
)
def _rollback_link(
directory_fd: int,
name: str,
expected: _FileIdentity,
) -> bool:
try:
current = _signature(directory_fd, name)
if current is None:
return True
if current.device != expected.device or current.inode != expected.inode:
return True
os.unlink(name, dir_fd=directory_fd)
return True
except (DocForgeError, OSError):
return False
def _rollback_and_sync(
directory_fd: int,
name: str,
expected: _FileIdentity,
) -> bool:
if not _rollback_link(directory_fd, name, expected):
return False
try:
os.fsync(directory_fd)
except OSError:
return False
return True
def _atomic_write(
path: Path,
content: str,
*,
validate_binding: Callable[[], None],
) -> tuple[str, str, str | None, Path | None]:
target, directory_fd = _bound_parent(path)
encoded = content.encode("utf-8")
if len(encoded) > MAX_CLIENT_FRAGMENT_BYTES:
with suppress(OSError):
os.close(directory_fd)
raise DocForgeError(
"output_oversized",
"Generated configuration fragment exceeds the bounded limit",
maximum_bytes=MAX_CLIENT_FRAGMENT_BYTES,
)
temporary_name = f".docforge-client-{secrets.token_hex(12)}"
temporary_created = False
committed = False
linked_identity: _FileIdentity | None = None
try:
_require_parent_binding(target.parent, directory_fd)
before = _signature(directory_fd, target.name)
if before is not None:
if not _private_existing(before):
raise DocForgeError(
"unsafe_output",
(
"Existing configuration fragment must be owned by the current user, "
"private, and singly linked"
),
)
existing = _read_existing(directory_fd, target.name, before)
if existing == encoded:
validate_binding()
_require_parent_binding(target.parent, directory_fd)
current = _signature(directory_fd, target.name)
if (
current != before
or current is None
or not _private_existing(current)
or _read_existing(directory_fd, target.name, current) != encoded
):
raise DocForgeError(
"output_changed",
"Configuration output changed before unchanged publication was confirmed",
)
validate_binding()
_require_parent_binding(target.parent, directory_fd)
return "unchanged", "not_applicable", None, target
raise DocForgeError(
"output_conflict",
"Configuration fragment already exists with different content",
existing_sha256=hashlib.sha256(existing).hexdigest(),
generated_sha256=hashlib.sha256(encoded).hexdigest(),
)
try:
temporary_fd = os.open(
temporary_name,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW,
0o600,
dir_fd=directory_fd,
)
except OSError as error:
raise DocForgeError(
"output_publication_failed",
"Configuration fragment temporary file could not be created",
) from error
temporary_created = True
try:
with os.fdopen(temporary_fd, "wb", closefd=True) as handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
except OSError as error:
raise DocForgeError(
"output_publication_failed",
"Configuration fragment temporary file could not be written durably",
) from error
temporary_identity = _signature(directory_fd, temporary_name)
if temporary_identity is None:
raise DocForgeError(
"output_changed",
"Configuration fragment temporary file disappeared before publication",
)
if _signature(directory_fd, target.name) is not None:
raise DocForgeError(
"output_changed",
"Configuration output appeared before atomic publication",
)
_require_parent_binding(target.parent, directory_fd)
validate_binding()
try:
os.link(
temporary_name,
target.name,
src_dir_fd=directory_fd,
dst_dir_fd=directory_fd,
follow_symlinks=False,
)
except FileExistsError as error:
raise DocForgeError(
"output_changed",
"Configuration output appeared during atomic publication",
) from error
except OSError as error:
raise DocForgeError(
"output_publication_failed",
"Configuration fragment could not be published atomically",
) from error
linked_identity = temporary_identity
try:
validate_binding()
except DocForgeError:
if _rollback_link(directory_fd, target.name, temporary_identity):
raise
committed = True
return (
"created",
"unconfirmed",
"publication_binding_unconfirmed",
None,
)
linked = _signature(directory_fd, target.name)
if (
linked is None
or linked.device != temporary_identity.device
or linked.inode != temporary_identity.inode
or _read_existing(directory_fd, target.name, linked) != encoded
or not _parent_binding_current(target.parent, directory_fd)
):
if _rollback_link(directory_fd, target.name, temporary_identity):
raise DocForgeError(
"output_changed",
"Configuration output changed during atomic publication",
)
committed = True
return (
"created",
"unconfirmed",
"publication_location_unconfirmed",
None,
)
durability = "confirmed"
warning: str | None = None
try:
os.unlink(temporary_name, dir_fd=directory_fd)
temporary_created = False
published = _signature(directory_fd, target.name)
if (
published is None
or not _private_existing(published)
or published.device != temporary_identity.device
or published.inode != temporary_identity.inode
or _read_existing(directory_fd, target.name, published) != encoded
or not _parent_binding_current(target.parent, directory_fd)
):
if _rollback_link(directory_fd, target.name, temporary_identity):
raise DocForgeError(
"output_changed",
"Configuration output changed after atomic publication",
)
committed = True
return (
"created",
"unconfirmed",
"publication_location_unconfirmed",
None,
)
try:
validate_binding()
except DocForgeError:
if _rollback_link(directory_fd, target.name, temporary_identity):
raise
committed = True
return (
"created",
"unconfirmed",
"publication_binding_unconfirmed",
None,
)
os.fsync(directory_fd)
except OSError:
durability = "unconfirmed"
warning = "publication_durability_unconfirmed"
try:
validate_binding()
except DocForgeError:
if _rollback_and_sync(directory_fd, target.name, temporary_identity):
raise
committed = True
return (
"created",
"unconfirmed",
"publication_binding_unconfirmed",
None,
)
try:
published = _signature(directory_fd, target.name)
publication_current = (
published is not None
and _private_existing(published)
and published.device == temporary_identity.device
and published.inode == temporary_identity.inode
and _read_existing(directory_fd, target.name, published) == encoded
and _parent_binding_current(target.parent, directory_fd)
)
except DocForgeError:
publication_current = False
if not publication_current:
if _rollback_and_sync(directory_fd, target.name, temporary_identity):
raise DocForgeError(
"output_changed",
"Configuration output changed before publication was finalized",
)
committed = True
return (
"created",
"unconfirmed",
"publication_location_unconfirmed",
None,
)
committed = True
return "created", durability, warning, target
except Exception:
if not committed and linked_identity is not None:
_rollback_link(directory_fd, target.name, linked_identity)
if not committed and temporary_created:
with suppress(OSError):
os.unlink(temporary_name, dir_fd=directory_fd)
raise
finally:
with suppress(OSError):
os.close(directory_fd)
def _validate_configuration_result(result: dict[str, object]) -> None:
artifact = cast(dict[str, object], result["artifact"])
binding = cast(dict[str, object], result["binding"])
policy = cast(dict[str, object], result["effective_policy"])
project = cast(dict[str, object], result["project"])
content = cast(str, artifact["content"])
if artifact["content_sha256"] != hashlib.sha256(content.encode("utf-8")).hexdigest():
raise AssertionError("Generated client content hash drifted")
timeouts = cast(dict[str, object], binding["timeouts"])
expected_format, expected_content, _ = _artifact(
cast(ClientName, result["client"]),
server_name=cast(str, result["server_name"]),
command=cast(str, binding["command"]),
arguments=cast(list[str], binding["args"]),
startup_timeout=cast(int, timeouts["startup_seconds"]),
tool_timeout=cast(int, timeouts["tool_seconds"]),
)
if artifact["format"] != expected_format or content != expected_content:
raise AssertionError("Generated client artifact drifted from its binding")
adapter_policy = cast(dict[str, object], binding["adapter_policy"])
render_policy = cast(dict[str, object], binding["render_policy"])
arguments = cast(list[str], binding["args"])
prefix = [
"-I",
"-m",
"docforge.mcp_server",
"--project-root",
cast(str, project["project_root"]),
"--capability-mode",
cast(str, binding["capability_mode"]),
]
if arguments[:7] != prefix:
raise AssertionError("Generated client arguments drifted from their binding")
remaining = arguments[7:]
no_ast_argument = "--no-ast" in arguments
if no_ast_argument:
if remaining[-1:] != ["--no-ast"] or arguments.count("--no-ast") != 1:
raise AssertionError("Generated no-AST argument layout drifted")
remaining = remaining[:-1]
mode = binding["capability_mode"]
if (
(mode == "read" and remaining)
or (
mode == "proposal"
and (len(remaining) != 2 or remaining[0] != "--proposal-writer" or not remaining[1])
)
or (
mode == "application"
and (
len(remaining) != 4
or remaining[0] != "--proposal-writer"
or remaining[2] != "--canonical-applier"
or not remaining[1]
or remaining[1] != remaining[3]
)
)
):
raise AssertionError("Generated authority argument layout drifted")
composed_policy = compose_effective_policy(
selected_mode=cast(CapabilityMode, mode),
capability_source="explicit",
no_ast=adapter_policy["mode"] == "preserve-no-ast",
diagnostics=False,
render_configured=render_policy["manual"] != "disabled",
application_enabled=mode == "application",
)
expected_policy = composed_policy.as_dict()
if (
policy != expected_policy
or adapter_policy != composed_policy.adapter_policy()
or binding["capability_mode"] != policy["capability_mode"]
or no_ast_argument != (adapter_policy["mode"] == "preserve-no-ast")
or render_policy["manual"] != policy["manual_render"]
or render_policy["graph"] != policy["graph_render"]
or render_policy["live_viewer"] != policy["live_viewer"]
or (
adapter_policy["mode"] == "preserve-no-ast"
and (
policy["adapter_evolution"] != "preserve"
or policy["ast_analysis"] != "forbidden"
or policy["logic_indexing"] != "off"
)
)
or (
adapter_policy["mode"] == "standard"
and (
policy["adapter_evolution"] != "allowed"
or policy["ast_analysis"] != "allowed"
or policy["logic_indexing"] != "full"
)
)
):
raise AssertionError("Generated client policy drifted from its binding")
expected_hash = document_hash(
{
"schema_version": 1,
"client": result["client"],
"server_name": result["server_name"],
"project": project,
"binding": binding,
"effective_policy": policy,
"artifact_format": artifact["format"],
"artifact_content_sha256": artifact["content_sha256"],
}
)
if result["configuration_hash"] != expected_hash:
raise AssertionError("Generated client configuration hash drifted")
def generate_client_configuration(
project: ProjectService,
client: str,
*,
server_name: str | None = None,
capability_mode: str = "read",
proposal_writer: str | None = None,
canonical_applier: str | None = None,
no_ast: bool = False,
startup_timeout: int = 30,
tool_timeout: int = 300,
output: Path | None = None,
) -> dict[str, object]:
"""Build one deterministic client fragment and optionally publish it explicitly."""
validate_descriptor_binding(project.descriptor)
selected_client = _client_name(client)
selected_mode = _capability_mode(capability_mode)
startup_seconds = _bounded_seconds(
startup_timeout,
field="startup_timeout",
maximum=3_600,
)
tool_seconds = _bounded_seconds(
tool_timeout,
field="tool_timeout",
maximum=86_400,
)
descriptor = project.descriptor
if descriptor.adapter != "generic":
raise DocForgeError(
"client_configuration_unavailable",
"Generic CLI configuration cannot reconstruct a project-owned adapter",
adapter=descriptor.adapter,
)
writer_ids = {writer.writer_id for writer in descriptor.proposal_writers}
if selected_mode == "read":
if proposal_writer is not None or canonical_applier is not None:
raise DocForgeError(
"invalid_capability_binding",
"Read configuration cannot bind proposal or application authority",
)
elif selected_mode == "proposal":
if proposal_writer is None or proposal_writer not in writer_ids:
raise DocForgeError(
"capability_unavailable",
"Proposal configuration requires a descriptor-declared writer",
required="proposal_writer",
)
if canonical_applier is not None:
raise DocForgeError(
"invalid_capability_binding",
"Proposal configuration cannot bind a canonical applier",
)
else:
if (
proposal_writer is None
or canonical_applier is None
or proposal_writer != canonical_applier
or proposal_writer not in writer_ids
):
raise DocForgeError(
"capability_unavailable",
"Application configuration requires one declared writer/applier identity",
required="matching_declared_writer_and_applier",
)
fingerprint = project_root_fingerprint(descriptor.root)
selected_name = _validated_server_name(
server_name,
project_id=descriptor.project_id,
fingerprint=fingerprint,
)
executable = Path(os.path.abspath(sys.executable))
try:
executable_status = executable.stat()
except OSError as error:
raise DocForgeError(
"client_configuration_unavailable",
"Current Python executable cannot be inspected",
) from error
if not stat.S_ISREG(executable_status.st_mode) or not os.access(executable, os.X_OK):
raise DocForgeError(
"client_configuration_unavailable",
"Current Python executable is not a runnable regular file",
)
try:
probe = subprocess.run(
[
str(executable),
"-I",
"-B",
"-c",
"import docforge.mcp_server",
],
check=False,
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=10,
env=os.environ.copy(),
)
except (OSError, subprocess.SubprocessError) as error:
raise DocForgeError(
"client_configuration_unavailable",
"Current isolated Python executable could not be probed safely",
) from error
if probe.returncode != 0:
raise DocForgeError(
"client_configuration_unavailable",
"Current isolated Python executable cannot import docforge.mcp_server",
)
arguments = [
"-I",
"-m",
"docforge.mcp_server",
"--project-root",
str(descriptor.root),
"--capability-mode",
selected_mode,
]
if proposal_writer is not None:
arguments.extend(("--proposal-writer", proposal_writer))
if canonical_applier is not None:
arguments.extend(("--canonical-applier", canonical_applier))
if no_ast:
arguments.append("--no-ast")
policy = compose_effective_policy(
selected_mode=selected_mode,
capability_source="explicit",
no_ast=no_ast,
diagnostics=False,
render_configured=descriptor.render is not None,
application_enabled=canonical_applier is not None,
)
artifact_format, content, warning = _artifact(
selected_client,
server_name=selected_name,
command=str(executable),
arguments=arguments,
startup_timeout=startup_seconds,
tool_timeout=tool_seconds,
)
if output is None:
validate_descriptor_binding(descriptor)
write_state = "not_requested"
durability = "not_applicable"
publication_warning = None
output_path = None
else:
validate_descriptor_binding(descriptor)
write_state, durability, publication_warning, published_path = _atomic_write(
output,
content,
validate_binding=lambda: validate_descriptor_binding(descriptor),
)
output_path = str(published_path) if published_path is not None else None
artifact = {
"format": artifact_format,
"content": content,
"content_sha256": hashlib.sha256(content.encode("utf-8")).hexdigest(),
"output_path": output_path,
"write_state": write_state,
"durability": durability,
}
binding = {
"transport": "stdio",
"capability_mode": selected_mode,
"adapter_policy": policy.adapter_policy(),
"render_policy": {
"manual": policy.manual_render,
"graph": policy.graph_render,
"live_viewer": policy.live_viewer,
},
"command": str(executable),
"args": arguments,
"environment": {},
"timeouts": {
"startup_seconds": startup_seconds,
"tool_seconds": tool_seconds,
},
}
project_binding = {
"project_id": descriptor.project_id,
"project_root": str(descriptor.root),
"project_root_fingerprint": fingerprint,
"adapter": descriptor.adapter,
}
policy_payload = policy.as_dict()
plan_hash = document_hash(
{
"schema_version": 1,
"client": selected_client,
"server_name": selected_name,
"project": project_binding,
"binding": binding,
"effective_policy": policy_payload,
"artifact_format": artifact_format,
"artifact_content_sha256": artifact["content_sha256"],
}
)
result: dict[str, object] = {
"status": "ok",
"schema_version": 1,
"operation": "client.configure",
"action": "write" if output is not None else "preview",
"client": selected_client,
"server_name": selected_name,
"project": project_binding,
"binding": binding,
"effective_policy": policy_payload,
"artifact": artifact,
"configuration_hash": plan_hash,
"warnings": [
*([] if warning is None else [{"code": "timeout_format_unverified"}]),
*([] if publication_warning is None else [{"code": publication_warning}]),
],
}
_validate_configuration_result(result)
return result

1287
src/docforge/doctor.py Normal file
View file

File diff suppressed because it is too large Load diff

911
src/docforge/generation_diff.py Normal file
View file

@ -0,0 +1,911 @@
"""Bounded latest-generation transition receipts for disposable graph indexes."""
from __future__ import annotations
import json
import os
import secrets
import stat
from collections.abc import Mapping
from contextlib import suppress
from dataclasses import dataclass
from pathlib import Path
from typing import Literal, cast
from ._fs_safety import open_bound_directory, require_bound_directory
from .errors import DocForgeError
from .models import Edge, Node, ProjectDescriptor, ProjectSnapshot
from .pagination import canonical_hash
GENERATION_DIFF_SCHEMA_VERSION = 1
GENERATION_DIFF_SEMANTICS_VERSION = 1
MAX_GENERATION_DIFF_ITEMS = 1_000
MAX_GENERATION_DIFF_BYTES = 1_048_576
GENERATION_DIFF_FILENAME = "generation-diff.json"
IndexSignature = tuple[int, int, int, int, int]
PredecessorReason = Literal[
"no_predecessor",
"predecessor_unsafe",
"predecessor_unsupported_schema",
"predecessor_foreign",
"predecessor_policy_incompatible",
"predecessor_corrupt",
"predecessor_unattested",
"predecessor_changed",
"no_meaningful_transition",
]
_GENERATION_KEYS = frozenset(
{
"revision",
"source_hash",
"node_count",
"node_hash",
"edge_count",
"edge_hash",
"index_schema_version",
}
)
_SUMMARY_KEYS = frozenset(
{
"nodes_added",
"nodes_removed",
"nodes_changed",
"edges_added",
"edges_removed",
"total_changes",
}
)
_SIGNATURE_KEYS = frozenset({"device", "inode", "size", "mtime_ns", "ctime_ns"})
_RECEIPT_KEYS = frozenset(
{
"schema_version",
"diff_semantics_version",
"project_id",
"project_root_fingerprint",
"adapter",
"kind",
"reason",
"from_generation",
"to_generation",
"summary",
"items",
"full_item_count",
"retained_item_count",
"details_truncated",
"truncation_reason",
"full_collection_hash",
"retained_collection_hash",
"index_signature",
"receipt_hash",
}
)
_BASELINE_REASONS = frozenset(
{
"no_predecessor",
"predecessor_unsafe",
"predecessor_unsupported_schema",
"predecessor_foreign",
"predecessor_policy_incompatible",
"predecessor_corrupt",
"predecessor_unattested",
"predecessor_changed",
"no_meaningful_transition",
}
)
_NODE_ITEM_KEYS = frozenset(
{
"entity",
"change",
"node_id",
"before_content_hash",
"after_content_hash",
"before_source_path",
"after_source_path",
"before_node_hash",
"after_node_hash",
"changed_fields",
"item_hash",
}
)
_NODE_CHANGED_FIELDS = frozenset(
{
"title",
"family",
"authority",
"status",
"tags",
"summary",
"content",
"source_path",
"source_anchor",
"content_hash",
}
)
_EDGE_ITEM_KEYS = frozenset(
{
"entity",
"change",
"source_id",
"relation",
"target_id",
"item_hash",
}
)
@dataclass(frozen=True)
class PublishedGraph:
"""One completely verified predecessor publication."""
generation: dict[str, object]
nodes: tuple[Node, ...]
edges: tuple[Edge, ...]
logic_hash: str
signature: IndexSignature
@dataclass(frozen=True)
class GenerationDiffDraft:
"""A receipt body awaiting the committed index file identity."""
fields: dict[str, object]
items: tuple[dict[str, object], ...]
def generation_diff_path(descriptor: ProjectDescriptor) -> Path:
"""Return the fixed project-confined latest-transition receipt path."""
return descriptor.cache_root / GENERATION_DIFF_FILENAME
def index_signature(path: Path) -> IndexSignature:
"""Return the exact identity of one safe regular index publication."""
try:
status = path.lstat()
except OSError as error:
raise DocForgeError("missing_index", "Derived index does not exist") from error
if stat.S_ISLNK(status.st_mode) or not stat.S_ISREG(status.st_mode):
raise DocForgeError("path_escape", "Derived index path is not a safe regular file")
return (
status.st_dev,
status.st_ino,
status.st_size,
status.st_mtime_ns,
status.st_ctime_ns,
)
def signature_payload(signature: IndexSignature) -> dict[str, int]:
"""Convert one stat identity to its versioned JSON representation."""
return {
"device": signature[0],
"inode": signature[1],
"size": signature[2],
"mtime_ns": signature[3],
"ctime_ns": signature[4],
}
def generation_identity(status: Mapping[str, object]) -> dict[str, object]:
"""Select the primary-graph identity stored in a public diff receipt."""
return {
"revision": status["revision"],
"source_hash": status["source_hash"],
"node_count": status["node_count"],
"node_hash": status["node_hash"],
"edge_count": status["edge_count"],
"edge_hash": status["edge_hash"],
"index_schema_version": status["index_schema_version"],
}
def prepare_generation_diff(
descriptor: ProjectDescriptor,
*,
predecessor: PublishedGraph | None,
predecessor_reason: PredecessorReason | None,
current_snapshot: ProjectSnapshot,
current_status: Mapping[str, object],
preserved_receipt: Mapping[str, object] | None = None,
) -> GenerationDiffDraft:
"""Prepare one deterministic latest transition without publishing it."""
current_generation = generation_identity(current_status)
if predecessor is None:
return _baseline_draft(
descriptor,
current_generation,
predecessor_reason or "no_predecessor",
)
if predecessor.generation == current_generation:
if preserved_receipt is not None and _receipt_targets(
preserved_receipt,
descriptor,
current_generation,
predecessor.signature,
):
fields = {
key: value
for key, value in preserved_receipt.items()
if key not in {"index_signature", "receipt_hash", "items"}
}
preserved_items = preserved_receipt.get("items")
if not isinstance(preserved_items, list):
raise DocForgeError(
"invalid_generation_diff",
"Preserved generation diff has no item collection",
)
items = tuple(
cast(dict[str, object], item)
for item in cast(list[object], preserved_items)
if isinstance(item, dict)
)
return GenerationDiffDraft(fields=fields, items=items)
return _baseline_draft(
descriptor,
current_generation,
"no_meaningful_transition",
)
items = _change_items(
predecessor.nodes,
predecessor.edges,
current_snapshot.nodes,
current_snapshot.edges,
)
summary = _summary(items)
return GenerationDiffDraft(
fields={
"schema_version": GENERATION_DIFF_SCHEMA_VERSION,
"diff_semantics_version": GENERATION_DIFF_SEMANTICS_VERSION,
"project_id": descriptor.project_id,
"project_root_fingerprint": _root_fingerprint(descriptor),
"adapter": descriptor.adapter,
"kind": "transition",
"reason": None,
"from_generation": predecessor.generation,
"to_generation": current_generation,
"summary": summary,
"full_item_count": len(items),
"full_collection_hash": canonical_hash([item["item_hash"] for item in items]),
},
items=items,
)
def finalize_generation_diff(
draft: GenerationDiffDraft,
*,
signature: IndexSignature,
) -> dict[str, object]:
"""Bind a draft to the committed index and enforce fixed receipt ceilings."""
full_count = cast(int, draft.fields["full_item_count"])
retained = list(draft.items[:MAX_GENERATION_DIFF_ITEMS])
item_limited = full_count > len(retained)
byte_limited = draft.fields.get("truncation_reason") == "receipt_byte_limit"
while True:
receipt = _final_receipt(
draft.fields,
retained,
signature=signature,
item_limited=item_limited,
byte_limited=byte_limited,
)
if len(_receipt_bytes(receipt)) <= MAX_GENERATION_DIFF_BYTES:
return receipt
if not retained:
raise DocForgeError(
"generation_diff_failure",
"Generation diff identity exceeds the fixed receipt byte limit",
)
retained.pop()
byte_limited = True
def publish_generation_diff(
descriptor: ProjectDescriptor,
receipt: Mapping[str, object],
) -> None:
"""Atomically replace the single latest-generation receipt."""
root = descriptor.cache_root
if generation_diff_path(descriptor).parent != root:
raise DocForgeError("path_escape", "Generation diff path is not confined")
raw = _receipt_bytes(receipt)
if len(raw) > MAX_GENERATION_DIFF_BYTES:
raise DocForgeError(
"generation_diff_failure",
"Generation diff receipt exceeds the fixed byte limit",
)
root_fd = open_bound_directory(root)
temporary_name = f".generation-diff-{secrets.token_hex(12)}"
temporary_created = False
try:
try:
status = os.stat(
GENERATION_DIFF_FILENAME,
dir_fd=root_fd,
follow_symlinks=False,
)
except FileNotFoundError:
status = None
if status is not None and (
stat.S_ISLNK(status.st_mode) or not stat.S_ISREG(status.st_mode)
):
raise DocForgeError(
"path_escape",
"Generation diff receipt path is not a safe regular file",
)
descriptor_fd = os.open(
temporary_name,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW,
0o600,
dir_fd=root_fd,
)
temporary_created = True
with os.fdopen(descriptor_fd, "wb") as handle:
handle.write(raw)
handle.flush()
os.fsync(handle.fileno())
require_bound_directory(root, root_fd)
os.replace(
temporary_name,
GENERATION_DIFF_FILENAME,
src_dir_fd=root_fd,
dst_dir_fd=root_fd,
)
temporary_created = False
os.fsync(root_fd)
require_bound_directory(root, root_fd)
except Exception:
if temporary_created:
with suppress(OSError):
os.unlink(temporary_name, dir_fd=root_fd)
raise
finally:
os.close(root_fd)
def load_generation_diff(
descriptor: ProjectDescriptor,
) -> tuple[dict[str, object] | None, str | None, IndexSignature | None]:
"""Read and validate one receipt without repairing any derived state."""
try:
root_fd = open_bound_directory(descriptor.cache_root)
except DocForgeError as error:
reason = "missing_receipt" if error.code == "missing_index" else "unsafe_receipt"
return None, reason, None
try:
try:
descriptor_fd = os.open(
GENERATION_DIFF_FILENAME,
os.O_RDONLY | os.O_NOFOLLOW,
dir_fd=root_fd,
)
except FileNotFoundError:
return None, "missing_receipt", None
except OSError:
return None, "unsafe_receipt", None
with os.fdopen(descriptor_fd, "rb") as handle:
status_before = os.fstat(handle.fileno())
if stat.S_ISLNK(status_before.st_mode) or not stat.S_ISREG(status_before.st_mode):
return None, "unsafe_receipt", None
before: IndexSignature = (
status_before.st_dev,
status_before.st_ino,
status_before.st_size,
status_before.st_mtime_ns,
status_before.st_ctime_ns,
)
if before[2] > MAX_GENERATION_DIFF_BYTES:
return None, "oversized_receipt", before
raw = handle.read(MAX_GENERATION_DIFF_BYTES + 1)
status_after = os.fstat(handle.fileno())
after: IndexSignature = (
status_after.st_dev,
status_after.st_ino,
status_after.st_size,
status_after.st_mtime_ns,
status_after.st_ctime_ns,
)
if len(raw) > MAX_GENERATION_DIFF_BYTES:
return None, "oversized_receipt", before
try:
path_status = os.stat(
GENERATION_DIFF_FILENAME,
dir_fd=root_fd,
follow_symlinks=False,
)
except OSError:
return None, "receipt_changed", before
path_signature: IndexSignature = (
path_status.st_dev,
path_status.st_ino,
path_status.st_size,
path_status.st_mtime_ns,
path_status.st_ctime_ns,
)
try:
require_bound_directory(descriptor.cache_root, root_fd)
except DocForgeError:
return None, "unsafe_receipt", before
finally:
os.close(root_fd)
if before != after or before != path_signature or len(raw) != before[2]:
return None, "receipt_changed", before
try:
parsed: object = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError):
return None, "corrupt_receipt", before
if not isinstance(parsed, dict):
return None, "corrupt_receipt", before
receipt = cast(dict[str, object], parsed)
if not validate_generation_diff_receipt(receipt):
return None, "corrupt_receipt", before
if (
receipt.get("project_id") != descriptor.project_id
or receipt.get("project_root_fingerprint") != _root_fingerprint(descriptor)
or receipt.get("adapter") != descriptor.adapter
):
return None, "foreign_receipt", before
return receipt, None, before
def valid_generation_identity(value: object) -> bool:
"""Return whether a primary-graph generation has the exact version-1 shape."""
return _valid_generation(value)
def validate_generation_diff_receipt(
receipt: Mapping[str, object],
*,
descriptor: ProjectDescriptor | None = None,
) -> bool:
"""Strictly validate the complete version-1 receipt and its hashes."""
if frozenset(receipt) != _RECEIPT_KEYS:
return False
if (
receipt.get("schema_version") != GENERATION_DIFF_SCHEMA_VERSION
or receipt.get("diff_semantics_version") != GENERATION_DIFF_SEMANTICS_VERSION
or not _nonempty(receipt.get("project_id"))
or not _fingerprint(receipt.get("project_root_fingerprint"))
or not _nonempty(receipt.get("adapter"))
):
return False
if descriptor is not None and (
receipt["project_id"] != descriptor.project_id
or receipt["project_root_fingerprint"] != _root_fingerprint(descriptor)
or receipt["adapter"] != descriptor.adapter
):
return False
kind = receipt.get("kind")
reason = receipt.get("reason")
from_generation = receipt.get("from_generation")
if kind == "baseline":
if reason not in _BASELINE_REASONS or from_generation is not None:
return False
elif kind == "transition":
if (
reason is not None
or not _valid_generation(from_generation)
or from_generation == receipt.get("to_generation")
):
return False
else:
return False
if not _valid_generation(receipt.get("to_generation")):
return False
summary_value = receipt.get("summary")
items_value = receipt.get("items")
if not isinstance(summary_value, dict) or not isinstance(items_value, list):
return False
summary = cast(dict[str, object], summary_value)
items = cast(list[object], items_value)
if frozenset(summary) != _SUMMARY_KEYS or any(
type(value) is not int or value < 0 for value in summary.values()
):
return False
total = sum(
cast(int, summary[key])
for key in (
"nodes_added",
"nodes_removed",
"nodes_changed",
"edges_added",
"edges_removed",
)
)
if summary.get("total_changes") != total:
return False
if kind == "baseline" and (total != 0 or items):
return False
if len(items) > MAX_GENERATION_DIFF_ITEMS or not all(_valid_item(item) for item in items):
return False
typed_items = tuple(cast(dict[str, object], item) for item in items)
if list(typed_items) != sorted(typed_items, key=_item_sort_key):
return False
identities = tuple(_item_identity(item) for item in typed_items)
if len(identities) != len(set(identities)):
return False
retained_summary = _summary(typed_items)
if any(
retained_summary[key] > cast(int, summary[key])
for key in (
"nodes_added",
"nodes_removed",
"nodes_changed",
"edges_added",
"edges_removed",
)
):
return False
full_count = receipt.get("full_item_count")
retained_count = receipt.get("retained_item_count")
truncated = receipt.get("details_truncated")
truncation_reason = receipt.get("truncation_reason")
if (
type(full_count) is not int
or type(retained_count) is not int
or full_count != total
or retained_count != len(items)
or retained_count > full_count
or type(truncated) is not bool
or truncated != (retained_count < full_count)
):
return False
if truncation_reason is None:
if truncated or full_count > MAX_GENERATION_DIFF_ITEMS:
return False
elif truncation_reason == "receipt_item_limit":
if (
not truncated
or full_count <= MAX_GENERATION_DIFF_ITEMS
or retained_count != MAX_GENERATION_DIFF_ITEMS
):
return False
elif truncation_reason == "receipt_byte_limit":
if not truncated or retained_count >= min(full_count, MAX_GENERATION_DIFF_ITEMS):
return False
else:
return False
item_hashes = [cast(dict[str, object], item)["item_hash"] for item in items]
retained_hash = canonical_hash(item_hashes)
if receipt.get("retained_collection_hash") != retained_hash:
return False
full_hash = receipt.get("full_collection_hash")
if not _sha256(full_hash):
return False
if not truncated and full_hash != retained_hash:
return False
signature_value = receipt.get("index_signature")
if not isinstance(signature_value, dict):
return False
signature = cast(dict[str, object], signature_value)
if frozenset(signature) != _SIGNATURE_KEYS:
return False
if any(type(value) is not int or value < 0 for value in signature.values()):
return False
receipt_hash = receipt.get("receipt_hash")
if not _sha256(receipt_hash):
return False
unhashed = {key: value for key, value in receipt.items() if key != "receipt_hash"}
return receipt_hash == canonical_hash(unhashed)
def _baseline_draft(
descriptor: ProjectDescriptor,
current_generation: Mapping[str, object],
reason: PredecessorReason,
) -> GenerationDiffDraft:
empty_hash = canonical_hash([])
return GenerationDiffDraft(
fields={
"schema_version": GENERATION_DIFF_SCHEMA_VERSION,
"diff_semantics_version": GENERATION_DIFF_SEMANTICS_VERSION,
"project_id": descriptor.project_id,
"project_root_fingerprint": _root_fingerprint(descriptor),
"adapter": descriptor.adapter,
"kind": "baseline",
"reason": reason,
"from_generation": None,
"to_generation": dict(current_generation),
"summary": {
"nodes_added": 0,
"nodes_removed": 0,
"nodes_changed": 0,
"edges_added": 0,
"edges_removed": 0,
"total_changes": 0,
},
"full_item_count": 0,
"full_collection_hash": empty_hash,
},
items=(),
)
def _change_items(
before_nodes: tuple[Node, ...],
before_edges: tuple[Edge, ...],
after_nodes: tuple[Node, ...],
after_edges: tuple[Edge, ...],
) -> tuple[dict[str, object], ...]:
before_by_id = {node.node_id: node for node in before_nodes}
after_by_id = {node.node_id: node for node in after_nodes}
items: list[dict[str, object]] = []
for node_id in sorted(before_by_id.keys() | after_by_id.keys()):
before = before_by_id.get(node_id)
after = after_by_id.get(node_id)
if before == after:
continue
if before is None:
change = "added"
elif after is None:
change = "removed"
else:
change = "changed"
before_payload = before.as_dict() if before is not None else None
after_payload = after.as_dict() if after is not None else None
changed_fields = (
[]
if before_payload is None or after_payload is None
else sorted(key for key in before_payload if before_payload[key] != after_payload[key])
)
payload: dict[str, object] = {
"entity": "node",
"change": change,
"node_id": node_id,
"before_content_hash": None if before is None else before.content_hash,
"after_content_hash": None if after is None else after.content_hash,
"before_source_path": None if before is None else before.source_path,
"after_source_path": None if after is None else after.source_path,
"before_node_hash": (
None if before_payload is None else canonical_hash(before_payload)
),
"after_node_hash": (None if after_payload is None else canonical_hash(after_payload)),
"changed_fields": changed_fields,
}
payload["item_hash"] = canonical_hash(payload)
items.append(payload)
before_edge_set = {(edge.source_id, edge.relation, edge.target_id) for edge in before_edges}
after_edge_set = {(edge.source_id, edge.relation, edge.target_id) for edge in after_edges}
for change, values in (
("removed", sorted(before_edge_set - after_edge_set)),
("added", sorted(after_edge_set - before_edge_set)),
):
for source_id, relation, target_id in values:
payload = {
"entity": "edge",
"change": change,
"source_id": source_id,
"relation": relation,
"target_id": target_id,
}
payload["item_hash"] = canonical_hash(payload)
items.append(payload)
return tuple(sorted(items, key=_item_sort_key))
def _item_sort_key(item: Mapping[str, object]) -> tuple[str, str, str, str, str]:
return (
cast(str, item["entity"]),
cast(str, item.get("node_id", item.get("source_id", ""))),
cast(str, item.get("relation", "")),
cast(str, item.get("target_id", "")),
cast(str, item["change"]),
)
def _item_identity(item: Mapping[str, object]) -> tuple[str, ...]:
if item["entity"] == "node":
return ("node", cast(str, item["node_id"]))
return (
"edge",
cast(str, item["source_id"]),
cast(str, item["relation"]),
cast(str, item["target_id"]),
)
def _summary(items: tuple[dict[str, object], ...]) -> dict[str, int]:
result = {
"nodes_added": 0,
"nodes_removed": 0,
"nodes_changed": 0,
"edges_added": 0,
"edges_removed": 0,
"total_changes": len(items),
}
for item in items:
entity = cast(str, item["entity"])
change = cast(str, item["change"])
key = f"{entity}s_{change}"
result[key] += 1
return result
def _final_receipt(
fields: Mapping[str, object],
retained: list[dict[str, object]],
*,
signature: IndexSignature,
item_limited: bool,
byte_limited: bool,
) -> dict[str, object]:
full_count = cast(int, fields["full_item_count"])
truncated = len(retained) < full_count
if byte_limited:
reason: str | None = "receipt_byte_limit"
elif item_limited:
reason = "receipt_item_limit"
else:
reason = None
receipt = {
**fields,
"items": retained,
"retained_item_count": len(retained),
"details_truncated": truncated,
"truncation_reason": reason,
"retained_collection_hash": canonical_hash([item["item_hash"] for item in retained]),
"index_signature": signature_payload(signature),
}
receipt["receipt_hash"] = canonical_hash(receipt)
return receipt
def _receipt_targets(
receipt: Mapping[str, object],
descriptor: ProjectDescriptor,
generation: Mapping[str, object],
signature: IndexSignature,
) -> bool:
return (
validate_generation_diff_receipt(receipt, descriptor=descriptor)
and receipt.get("to_generation") == dict(generation)
and receipt.get("index_signature") == signature_payload(signature)
)
def _valid_generation(value: object) -> bool:
if not isinstance(value, dict):
return False
generation = cast(dict[str, object], value)
if frozenset(generation) != _GENERATION_KEYS:
return False
source_hash = generation.get("source_hash")
node_hash = generation.get("node_hash")
edge_hash = generation.get("edge_hash")
return (
_nonempty(generation.get("revision"))
and _sha256(source_hash)
and _sha256(node_hash)
and _sha256(edge_hash)
and type(generation.get("node_count")) is int
and cast(int, generation["node_count"]) >= 0
and type(generation.get("edge_count")) is int
and cast(int, generation["edge_count"]) >= 0
and type(generation.get("index_schema_version")) is int
and cast(int, generation["index_schema_version"]) >= 1
)
def _valid_item(value: object) -> bool:
if not isinstance(value, dict):
return False
payload = cast(dict[str, object], value)
entity = payload.get("entity")
if entity == "node":
if frozenset(payload) != _NODE_ITEM_KEYS:
return False
changed_fields_value = payload.get("changed_fields")
if not isinstance(changed_fields_value, list):
return False
changed_fields = cast(list[object], changed_fields_value)
if (
payload.get("change") not in {"added", "removed", "changed"}
or not _nonempty(payload.get("node_id"))
or any(
not _nonempty(field) or field not in _NODE_CHANGED_FIELDS
for field in changed_fields
)
or changed_fields != sorted(set(cast(list[str], changed_fields)))
):
return False
for key in (
"before_content_hash",
"after_content_hash",
"before_node_hash",
"after_node_hash",
):
candidate = payload.get(key)
if candidate is not None and not _sha256(candidate):
return False
for key in ("before_source_path", "after_source_path"):
candidate = payload.get(key)
if candidate is not None and not _nonempty(candidate):
return False
change = payload["change"]
before_values = (
payload["before_content_hash"],
payload["before_source_path"],
payload["before_node_hash"],
)
after_values = (
payload["after_content_hash"],
payload["after_source_path"],
payload["after_node_hash"],
)
if change == "added":
if (
any(value is not None for value in before_values)
or not all(value is not None for value in after_values)
or changed_fields
):
return False
elif change == "removed":
if (
not all(value is not None for value in before_values)
or any(value is not None for value in after_values)
or changed_fields
):
return False
elif (
not all(value is not None for value in (*before_values, *after_values))
or not changed_fields
or payload["before_node_hash"] == payload["after_node_hash"]
):
return False
elif entity == "edge":
if (
frozenset(payload) != _EDGE_ITEM_KEYS
or payload.get("change") not in {"added", "removed"}
or not _nonempty(payload.get("source_id"))
or not _nonempty(payload.get("relation"))
or not _nonempty(payload.get("target_id"))
):
return False
else:
return False
item_hash = payload.get("item_hash")
unhashed = {key: item for key, item in payload.items() if key != "item_hash"}
return _sha256(item_hash) and item_hash == canonical_hash(unhashed)
def _receipt_bytes(receipt: Mapping[str, object]) -> bytes:
return json.dumps(receipt, sort_keys=True, indent=2, ensure_ascii=True).encode("utf-8") + b"\n"
def _root_fingerprint(descriptor: ProjectDescriptor) -> str:
from .project import project_root_fingerprint
return project_root_fingerprint(descriptor.root)
def _nonempty(value: object) -> bool:
return isinstance(value, str) and bool(value)
def _sha256(value: object) -> bool:
return (
isinstance(value, str)
and len(value) == 64
and all(character in "0123456789abcdef" for character in value)
)
def _fingerprint(value: object) -> bool:
return (
isinstance(value, str)
and len(value) == 16
and all(character in "0123456789abcdef" for character in value)
)

1152
src/docforge/index.py
View file

File diff suppressed because it is too large Load diff

670
src/docforge/mcp_server.py
View file

@ -18,8 +18,10 @@ from .errors import DocForgeError
from .index import ProjectIndex
from .models import IncrementalStateProject, ProjectService, RuntimeValidatedProject
from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt
from .policy import CapabilityMode, capability_mode, compose_effective_policy
from .project import Project, project_root_fingerprint
from .rendering import RenderService
from .retrieval import MAX_TASK_EVIDENCE, TaskKind, build_retrieval_plan
from .telemetry import request, stage
from .viewer_manager import ViewerManagerClient
@ -42,11 +44,13 @@ READ_TOOLS = (
"docforge_dependencies",
"docforge_impact",
"docforge_get_context",
"docforge_get_task_context",
"docforge_validate_project",
"docforge_render_status",
"docforge_visualize",
"docforge_stop_visualization",
"docforge_visualization_status",
"docforge_get_generation_diff",
)
PROPOSAL_TOOLS = (
"docforge_create_changeset",
@ -129,50 +133,96 @@ class DocForgeService:
binding_metadata: Mapping[str, object] | None = None,
no_ast: bool = False,
diagnostics: bool = False,
capability_mode_name: str | None = None,
) -> None:
self.project = project
self.index = ProjectIndex(self.project, allow_logic=not no_ast)
self.changesets = ChangesetStore(self.project, proposal_writer)
default_mode: CapabilityMode = (
"application" if canonical_applier is not None else "proposal"
)
selected_mode = capability_mode(capability_mode_name, default=default_mode)
application_enabled = (
canonical_applier_id is not None
and canonical_applier is not None
and selected_mode in {"application", "operator"}
)
self.policy = compose_effective_policy(
selected_mode=selected_mode,
capability_source=("factory_default" if capability_mode_name is None else "explicit"),
no_ast=no_ast,
diagnostics=diagnostics,
render_configured=project.descriptor.render is not None,
application_enabled=application_enabled,
)
self.index = ProjectIndex(self.project, allow_logic=not self.policy.no_ast)
self.changesets = ChangesetStore(
self.project,
proposal_writer if selected_mode != "read" else None,
)
self.rendering = RenderService(self.project, self.changesets)
self.application = CanonicalApplicationService(
self.project,
applier_id=canonical_applier_id,
applier=canonical_applier,
applier_id=canonical_applier_id if application_enabled else None,
applier=canonical_applier if application_enabled else None,
index=self.index,
)
self.visualization = ViewerManagerClient(self.index)
self.context_provider = context_provider
self.task_context_available = context_provider is compile_context
self.binding_metadata = dict(binding_metadata or {})
self.no_ast = no_ast
self.no_ast = self.policy.no_ast
self.diagnostics = diagnostics
self.tool_surface = tool_surface or (
*ALL_TOOLS,
*(APPLICATION_TOOLS if self.application.enabled else ()),
default_surface = (
READ_TOOLS
if selected_mode == "read"
else (
*ALL_TOOLS,
*(
APPLICATION_TOOLS
if self.application.enabled and selected_mode in {"application", "operator"}
else ()
),
)
)
self.tool_surface = tool_surface or default_surface
def adapter_policy(self) -> dict[str, object]:
"""Return the immutable adapter-evolution policy for this MCP binding."""
if not self.no_ast:
return {
"mode": "standard",
"ast_analysis": "allowed",
"logic_projection": "allowed",
"incremental_extraction": "allowed",
"adapter_rewrite": "not_requested",
}
return self.policy.adapter_policy()
def capabilities(self) -> dict[str, object]:
"""Return the registered surfaces separately from startup-bound authority."""
proposal_access = self.changesets.access()
application_access = self.application.access()
return {
"mode": "preserve-no-ast",
"ast_analysis": "forbidden",
"logic_projection": "forbidden",
"incremental_extraction": "allowed",
"adapter_rewrite": "forbidden",
"blocked_tools": ["docforge_get_logic"],
"instruction": (
"Preserve the existing adapter extraction strategy. Do not add Python AST, "
"Tree-sitter, compiler-AST, or function-Logic extraction. Non-AST incremental "
"fingerprinting and caching remain allowed."
),
"schema_version": 1,
"mode": self.policy.capability_mode,
"registered_tools": list(self.tool_surface),
"read": {
"enabled": True,
"tools": [tool for tool in READ_TOOLS if tool in self.tool_surface],
},
"task_context": {
"enabled": self.task_context_available,
"reason": (
None
if self.task_context_available
else "custom_context_policy_not_supported_by_task_context_v1"
),
},
"proposal": {
"surface_enabled": any(tool in self.tool_surface for tool in PROPOSAL_TOOLS),
"mutation_access": proposal_access,
},
"application": {
"surface_enabled": any(tool in self.tool_surface for tool in APPLICATION_TOOLS),
"mutation_access": application_access,
},
"operator": {
"enabled": self.policy.capability_mode == "operator",
"tools": [],
},
}
def invoke(
@ -507,27 +557,39 @@ class DocForgeService:
def bootstrap(self) -> dict[str, object]:
def operation() -> dict[str, object]:
synchronized = self.index.synchronize()
snapshot = self.project.load()
root = snapshot.descriptor.root
descriptor = self.project.descriptor
root = descriptor.root
binding = {
"project_root": str(root),
"descriptor_path": str(snapshot.descriptor.descriptor_path),
"adapter": snapshot.descriptor.adapter,
"cache_root": str(snapshot.descriptor.cache_root),
"index_path": str(snapshot.descriptor.index_path),
"changeset_root": str(snapshot.descriptor.changeset_root),
"descriptor_path": str(descriptor.descriptor_path),
"adapter": descriptor.adapter,
"cache_root": str(descriptor.cache_root),
"index_path": str(descriptor.index_path),
"changeset_root": str(descriptor.changeset_root),
**self.binding_metadata,
"adapter_policy": self.adapter_policy(),
}
recommended_workflow = [
"docforge_get_context or targeted read tools",
(
"docforge_get_task_context, docforge_get_context, or targeted read tools"
if self.task_context_available
else "docforge_get_context or targeted read tools"
),
"make and verify one coherent implementation slice",
"docforge_sync",
"docforge_register_changes",
"docforge_get_changeset_diff",
"docforge_apply_changeset",
"docforge_bootstrap",
]
proposal_access = self.changesets.access()
application_access = self.application.access()
if proposal_access["enabled"] and "docforge_register_changes" in self.tool_surface:
recommended_workflow.extend(
(
"docforge_register_changes",
"docforge_get_changeset_diff",
)
)
if application_access["enabled"] and "docforge_apply_changeset" in self.tool_surface:
recommended_workflow.append("docforge_apply_changeset")
recommended_workflow.append("docforge_bootstrap")
if self.no_ast:
recommended_workflow.insert(
1,
@ -536,19 +598,64 @@ class DocForgeService:
"compiler-AST, or function-Logic extraction"
),
)
if self.task_context_available:
recommended_first_operation: dict[str, object] = {
"tool": "docforge_get_task_context",
"arguments": {
"task_kind": "implementation",
"task": "<describe the current task>",
},
"reason": "Begin with one bounded task-shaped context capsule.",
}
elif descriptor.profiles:
recommended_first_operation = {
"tool": "docforge_get_context",
"arguments": {"profile": descriptor.profiles[0].profile_id},
"reason": "Begin with one configured bounded context profile.",
}
else:
recommended_first_operation = {
"tool": "docforge_project_info",
"arguments": dict[str, object](),
"reason": "Confirm the fixed binding before targeted retrieval.",
}
capabilities = self.capabilities()
effective_policy = self.policy.as_dict()
session_contract: dict[str, object] = {
"schema_version": 1,
"binding": binding,
"generation": {
"revision": synchronized["revision"],
"source_hash": synchronized["source_hash"],
"freshness": "current",
},
"effective_policy": effective_policy,
"capabilities": capabilities,
"render_policies": {
"manual": effective_policy["manual_render"],
"graph": effective_policy["graph_render"],
"live_viewer": effective_policy["live_viewer"],
},
"recommended_first_operation": recommended_first_operation,
"recommended_workflow": recommended_workflow,
"prohibitions": effective_policy["prohibitions"],
}
return {
"status": "ok",
"project_id": snapshot.descriptor.project_id,
"project_id": descriptor.project_id,
"project_root_fingerprint": project_root_fingerprint(root),
"title": snapshot.descriptor.title,
"adapter": snapshot.descriptor.adapter,
"revision": snapshot.revision,
"source_hash": snapshot.source_hash,
"title": descriptor.title,
"adapter": descriptor.adapter,
"revision": synchronized["revision"],
"source_hash": synchronized["source_hash"],
"binding": binding,
"canonical_paths": [str(path) for path in snapshot.descriptor.content_roots],
"canonical_paths": [str(path) for path in descriptor.content_roots],
"adapter_policy": self.adapter_policy(),
"proposal_access": self.changesets.access(),
"canonical_application_access": self.application.access(),
"effective_policy": effective_policy,
"capabilities": capabilities,
"session_contract": session_contract,
"proposal_access": proposal_access,
"canonical_application_access": application_access,
"synchronization": synchronized["synchronization"],
"recommended_workflow": recommended_workflow,
}
@ -606,6 +713,8 @@ class DocForgeService:
"Canonical project files own facts; DocForge results are derived."
),
"adapter_policy": self.adapter_policy(),
"effective_policy": self.policy.as_dict(),
"capabilities": self.capabilities(),
"canonical_paths": [
*(relative(path) for path in snapshot.descriptor.content_roots),
*(relative(path) for path in snapshot.descriptor.authority_files),
@ -638,14 +747,20 @@ class DocForgeService:
],
"allowed_tools": list(self.tool_surface),
"excluded_operations": list(
EXCLUDED_OPERATIONS
+ (
("canonical_writes", "canonical_changeset_application")
if not self.application.enabled
else ()
dict.fromkeys(
EXCLUDED_OPERATIONS
+ (
("canonical_writes", "canonical_changeset_application")
if not self.application.enabled
else ()
)
+ (
READ_ONLY_EXCLUDED_OPERATIONS
if self.policy.capability_mode == "read"
else ()
)
+ tuple(self.policy.prohibitions)
)
+ (READ_ONLY_EXCLUDED_OPERATIONS if self.tool_surface == READ_TOOLS else ())
+ (("adapter_ast_upgrade", "function_logic_extraction") if self.no_ast else ())
),
"proposal_access": self.changesets.access(),
"canonical_application_access": self.application.access(),
@ -741,6 +856,370 @@ class DocForgeService:
operation_name="mcp.context",
)
def task_context(
self,
task_kind: TaskKind,
task: str,
*,
focus_node_id: str | None = None,
budget: int | None = None,
limit: int | None = None,
cursor: str | None = None,
) -> dict[str, Any]:
"""Return one task-shaped capsule from a single immutable graph generation."""
if not self.task_context_available:
def unavailable() -> dict[str, object]:
raise DocForgeError(
"task_context_unavailable",
(
"This binding uses a custom context provider; "
"core task planning is unavailable"
),
)
return self.invoke(
unavailable,
synchronize=False,
load_error_identity=False,
operation_name="mcp.task_context",
)
def operation() -> dict[str, object]:
maximum_evidence = min(
self.project.descriptor.limits.max_results,
MAX_TASK_EVIDENCE,
)
selected_limit = page_limit(
limit,
default=min(20, maximum_evidence),
maximum=maximum_evidence,
)
plan = build_retrieval_plan(
self.project.descriptor,
task_kind=task_kind,
task=task,
focus_node_id=focus_node_id,
budget=budget,
limit=maximum_evidence,
effective_policy=self.policy.as_dict(),
)
result = self.index.task_context(plan)
return self._page_task_context_result(
result,
selected_limit=selected_limit,
cursor=cursor,
)
return self.invoke(operation, operation_name="mcp.task_context")
def generation_diff(
self,
*,
limit: int | None = None,
cursor: str | None = None,
) -> dict[str, Any]:
"""Return one bounded page from the latest verified graph transition."""
def operation() -> dict[str, object]:
maximum_items = min(
self.project.descriptor.limits.max_results,
1_000,
)
selected_limit = page_limit(
limit,
default=min(20, maximum_items),
maximum=maximum_items,
)
result = self.index.generation_diff()
receipt_value = result.get("generation_diff")
if receipt_value is None:
if cursor is not None:
decode_cursor(
cursor,
kind="generation-diff.items",
binding={
"project_id": result.get("project_id"),
"receipt_state": result.get("receipt_state"),
"effective_policy_hash": canonical_hash(self.policy.as_dict()),
},
total_count=0,
)
return result
if not isinstance(receipt_value, Mapping):
raise DocForgeError(
"invalid_generation_diff",
"Generation diff receipt is malformed",
)
receipt = dict(cast(Mapping[str, object], receipt_value))
items_value = receipt.pop("items", None)
if not isinstance(items_value, list):
raise DocForgeError(
"invalid_generation_diff",
"Generation diff receipt has no bounded item collection",
)
items = cast(list[object], items_value)
stored_receipt_hash = receipt.pop("receipt_hash", None)
if not isinstance(stored_receipt_hash, str):
raise DocForgeError(
"invalid_generation_diff",
"Generation diff receipt has no stable identity",
)
receipt_header = {
**receipt,
"stored_receipt_hash": stored_receipt_hash,
}
binding = {
"stored_receipt_hash": stored_receipt_hash,
"effective_policy_hash": canonical_hash(self.policy.as_dict()),
}
position = decode_cursor(
cursor,
kind="generation-diff.items",
binding=binding,
total_count=len(items),
)
page_items: list[object] = []
page_omissions: list[dict[str, object]] = []
consumed = 0
maximum_chars = self.project.descriptor.limits.max_tool_output_chars
def page_result() -> dict[str, object]:
pagination = page_receipt(
kind="generation-diff.items",
binding=binding,
position=position,
count=consumed,
limit=selected_limit,
total_count=len(items),
)
page_hash = canonical_hash(
{
"page_schema_version": 1,
"receipt_state": result["receipt_state"],
"receipt_header": receipt_header,
"pagination": pagination,
"items": page_items,
"omissions": page_omissions,
}
)
return {
**result,
"generation_diff": {
"page_schema_version": 1,
"receipt_header": receipt_header,
"items": page_items,
"omissions": page_omissions,
"page_hash": page_hash,
},
"pagination": pagination,
}
candidates = items[position : position + selected_limit]
def fits(candidate_count: int) -> bool:
nonlocal consumed
page_items[:] = candidates[:candidate_count]
consumed = candidate_count
decorated = {
**page_result(),
"server_version": SERVER_VERSION,
"content_warning": CONTENT_WARNING,
}
return self._encoded_length(decorated) <= maximum_chars
lower = 0
upper = len(candidates)
while lower < upper:
midpoint = (lower + upper + 1) // 2
if fits(midpoint):
lower = midpoint
else:
upper = midpoint - 1
fits(lower)
if lower == 0 and candidates:
item = candidates[0]
item_payload: Mapping[str, object] = (
cast(Mapping[str, object], item) if isinstance(item, Mapping) else {}
)
page_omissions.append(
{
"code": "response_limit",
"item_hash": item_payload.get("item_hash"),
}
)
consumed = 1
return page_result()
return self.invoke(
operation,
synchronize=False,
load_error_identity=False,
operation_name="mcp.generation_diff",
)
def _page_task_context_result(
self,
result: dict[str, object],
*,
selected_limit: int,
cursor: str | None,
) -> dict[str, object]:
capsule_value = result.get("capsule")
if not isinstance(capsule_value, Mapping):
raise DocForgeError(
"invalid_task_context_result",
"Task context did not return a versioned capsule",
)
capsule = dict(cast(Mapping[str, object], capsule_value))
plan_value = capsule.get("plan")
generation_value = capsule.get("generation")
evidence_value = capsule.get("evidence")
omissions_value = capsule.get("omissions")
gaps_value = capsule.get("gaps")
if (
capsule.get("schema_version") != 1
or not isinstance(plan_value, Mapping)
or not isinstance(generation_value, Mapping)
or not isinstance(evidence_value, list)
or not isinstance(omissions_value, list)
or not isinstance(gaps_value, list)
or not isinstance(capsule.get("collection_hash"), str)
or not isinstance(capsule.get("capsule_hash"), str)
):
raise DocForgeError(
"invalid_task_context_result",
"Task context capsule is malformed",
)
plan_payload = cast(Mapping[str, object], plan_value)
generation = cast(Mapping[str, object], generation_value)
evidence = cast(list[object], evidence_value)
omissions = cast(list[object], omissions_value)
gaps = cast(list[object], gaps_value)
binding = {
"project_id": result.get("project_id"),
"project_root_fingerprint": result.get("project_root_fingerprint"),
"adapter": result.get("adapter"),
"revision": result.get("revision"),
"source_hash": result.get("source_hash"),
"index_schema_version": generation.get("index_schema_version"),
"effective_policy_hash": plan_payload.get("effective_policy_hash"),
"request_hash": plan_payload.get("request_hash"),
"plan_hash": plan_payload.get("plan_hash"),
"collection_hash": capsule["collection_hash"],
"capsule_hash": capsule["capsule_hash"],
}
items = [
*(("evidence", item) for item in evidence),
*(("omission", item) for item in omissions),
]
position = decode_cursor(
cursor,
kind="task-context.items",
binding=binding,
total_count=len(items),
)
page_evidence: list[object] = []
page_omissions: list[object] = []
consumed = 0
response_limited = False
maximum = self.project.descriptor.limits.max_tool_output_chars
def page_result() -> dict[str, object]:
pagination = page_receipt(
kind="task-context.items",
binding=binding,
position=position,
count=consumed,
limit=selected_limit,
total_count=len(items),
)
page_state = "incomplete" if response_limited else capsule.get("state")
page_summary = {
**cast(dict[str, object], capsule.get("summary", {})),
"page_evidence_count": len(page_evidence),
"page_omission_count": len(page_omissions),
"page_item_count": consumed,
}
page_hash = canonical_hash(
{
"capsule_hash": capsule["capsule_hash"],
"position": position,
"page_state": page_state,
"pagination": pagination,
"summary": page_summary,
"evidence": page_evidence,
"gaps": gaps,
"omissions": page_omissions,
}
)
page_capsule = {
**capsule,
"evidence": page_evidence,
"omissions": page_omissions,
"page_state": page_state,
"page_hash": page_hash,
"pagination": pagination,
"summary": page_summary,
}
return {
**result,
"capsule": page_capsule,
"next_cursor": pagination["next_cursor"],
"pagination": pagination,
}
candidates = items[position : position + selected_limit]
def populate(candidate_count: int) -> None:
nonlocal consumed, response_limited
page_evidence.clear()
page_omissions.clear()
for kind, item in candidates[:candidate_count]:
destination = page_evidence if kind == "evidence" else page_omissions
destination.append(item)
consumed = candidate_count
response_limited = candidate_count < len(candidates)
def fits(candidate_count: int) -> bool:
populate(candidate_count)
decorated = {
**page_result(),
"server_version": SERVER_VERSION,
"content_warning": CONTENT_WARNING,
"staleness": "current",
}
return self._encoded_length(decorated) <= maximum
lower = 0
upper = len(candidates)
while lower < upper:
midpoint = (lower + upper + 1) // 2
if fits(midpoint):
lower = midpoint
else:
upper = midpoint - 1
populate(lower)
if lower == 0 and candidates:
_, item = candidates[0]
subject = "unknown"
if isinstance(item, Mapping):
item_payload = cast(Mapping[str, object], item)
candidate = item_payload.get("node_id") or item_payload.get("subject")
if isinstance(candidate, str) and candidate:
subject = candidate[:256]
page_omissions.append(
{
"code": "response_limit",
"subject": subject,
"detail_hash": canonical_hash(cast(object, item)),
}
)
consumed = 1
response_limited = True
return page_result()
def _page_context_result(
self,
result: dict[str, object],
@ -907,19 +1386,20 @@ class DocForgeService:
def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMCP:
capability = (
"Read validated documentation for exactly one configured project."
if read_only
else (
"Read validated documentation and write isolated proposal changesets and previews for "
"exactly one configured project"
+ (
", with hash-bound canonical application enabled."
if service.application.enabled
else "."
)
)
)
capability = {
"read": "Read validated documentation for exactly one configured project.",
"proposal": (
"Read validated documentation and use startup-gated isolated proposal changesets and "
"previews for exactly one configured project."
),
"application": (
"Read validated documentation, use startup-gated isolated proposals, and apply one "
"exact validated changeset hash for exactly one configured project."
),
"operator": (
"Operate the fixed validated documentation binding for exactly one configured project."
),
}[service.policy.capability_mode]
server = FastMCP(
"DocForge",
instructions=(
@ -1060,6 +1540,26 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
return service.context(profile, budget, limit=limit, cursor=cursor)
@server.tool(name="docforge_get_task_context")
def get_task_context(
task_kind: TaskKind,
task: str,
focus_node_id: str | None = None,
budget: int | None = None,
limit: int | None = None,
cursor: str | None = None,
) -> dict[str, Any]:
"""Return one bounded task-shaped context capsule with explicit evidence gaps."""
return service.task_context(
task_kind,
task,
focus_node_id=focus_node_id,
budget=budget,
limit=limit,
cursor=cursor,
)
@server.tool(name="docforge_validate_project")
def validate_project() -> dict[str, Any]:
"""Validate current canonical sources and graph without writing any project file."""
@ -1097,6 +1597,15 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
return service.visualization_status()
@server.tool(name="docforge_get_generation_diff")
def get_generation_diff(
limit: int | None = None,
cursor: str | None = None,
) -> dict[str, Any]:
"""Return the latest bounded primary-graph generation transition."""
return service.generation_diff(limit=limit, cursor=cursor)
_registered_read_tools = (
bootstrap,
synchronize,
@ -1110,11 +1619,13 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
dependencies,
impact,
get_context,
get_task_context,
validate_project,
render_status,
visualize,
visualization_status,
stop_visualization,
visualization_status,
get_generation_diff,
)
if read_only:
return server
@ -1492,6 +2003,7 @@ def create_server(
canonical_applier_id: str | None = None,
no_ast: bool = False,
diagnostics: bool = False,
capability_mode: str | None = None,
) -> FastMCP:
project = Project.open(project_root)
return create_project_server(
@ -1507,6 +2019,7 @@ def create_server(
},
no_ast=no_ast,
diagnostics=diagnostics,
capability_mode=capability_mode,
)
@ -1520,6 +2033,7 @@ def create_project_server(
binding_metadata: Mapping[str, object] | None = None,
no_ast: bool = False,
diagnostics: bool = False,
capability_mode: str | None = None,
) -> FastMCP:
"""Create the full fixed MCP surface for one explicitly configured project service."""
@ -1532,8 +2046,12 @@ def create_project_server(
binding_metadata=binding_metadata,
no_ast=no_ast,
diagnostics=diagnostics,
capability_mode_name=capability_mode,
)
return _create_bound_server(
service,
read_only=service.policy.capability_mode == "read",
)
return _create_bound_server(service, read_only=False)
def create_read_only_server(
@ -1543,6 +2061,7 @@ def create_read_only_server(
binding_metadata: Mapping[str, object] | None = None,
no_ast: bool = False,
diagnostics: bool = False,
capability_mode: str | None = None,
) -> FastMCP:
"""Create an adapter-capable MCP server exposing only the fixed read tool surface."""
@ -1553,7 +2072,14 @@ def create_read_only_server(
binding_metadata=binding_metadata,
no_ast=no_ast,
diagnostics=diagnostics,
capability_mode_name="read" if capability_mode is None else capability_mode,
)
if service.policy.capability_mode != "read":
raise DocForgeError(
"invalid_capability_mode",
"Read-only server factory accepts only read capability mode",
capability_mode=service.policy.capability_mode,
)
return _create_bound_server(service, read_only=True)
@ -1575,6 +2101,11 @@ def main() -> None:
action="store_true",
help="Attach bounded request-local stage timings and counters",
)
parser.add_argument(
"--capability-mode",
choices=("read", "proposal", "application", "operator"),
help="Expose the versioned project-bound capability surface",
)
arguments = parser.parse_args()
create_server(
arguments.project_root,
@ -1582,6 +2113,7 @@ def main() -> None:
canonical_applier_id=arguments.canonical_applier,
no_ast=arguments.no_ast,
diagnostics=arguments.diagnostics,
capability_mode=arguments.capability_mode,
).run(transport="stdio")

3
src/docforge/pagination.py
View file

@ -103,7 +103,6 @@ def decode_cursor(
or not isinstance(stored_binding, dict)
or type(position) is not int
or position < 0
or position >= total_count
or not isinstance(checksum, str)
or len(checksum) != 64
):
@ -117,6 +116,8 @@ def decode_cursor(
"stale_cursor",
"Pagination cursor does not match the current result generation",
)
if position >= total_count:
raise _invalid_cursor()
return position

166
src/docforge/policy.py Normal file
View file

@ -0,0 +1,166 @@
"""Versioned immutable policy composition for one project-bound server."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Literal
from .errors import DocForgeError
CapabilityMode = Literal["read", "proposal", "application", "operator"]
CAPABILITY_MODES: tuple[CapabilityMode, ...] = (
"read",
"proposal",
"application",
"operator",
)
POLICY_PRECEDENCE = (
"core_safety",
"explicit_binding",
"no_ast_shorthand",
"resource_availability",
)
def capability_mode(value: str | None, *, default: CapabilityMode) -> CapabilityMode:
"""Validate one additive capability-mode selection."""
selected = default if value is None else value
if selected not in CAPABILITY_MODES:
raise DocForgeError(
"invalid_capability_mode",
"Capability mode is unsupported",
capability_mode=selected,
allowed=list(CAPABILITY_MODES),
)
return selected # type: ignore[return-value]
@dataclass(frozen=True)
class EffectivePolicyV1:
"""One fully composed process policy shared by every public projection."""
capability_mode: CapabilityMode
capability_source: Literal["factory_default", "explicit"]
adapter_evolution: Literal["allowed", "preserve"]
ast_analysis: Literal["allowed", "forbidden"]
logic_indexing: Literal["full", "off"]
synchronization: Literal["automatic"]
integrity: Literal["validated"]
manual_render: Literal["auto", "explicit", "disabled"]
graph_render: Literal["disabled"]
live_viewer: Literal["on-demand"]
profiling: Literal["enabled", "disabled"]
blocked_tools: tuple[str, ...]
prohibitions: tuple[str, ...]
@property
def no_ast(self) -> bool:
return self.ast_analysis == "forbidden"
def as_dict(self) -> dict[str, object]:
return {
"schema_version": 1,
"capability_mode": self.capability_mode,
"capability_source": self.capability_source,
"adapter_evolution": self.adapter_evolution,
"ast_analysis": self.ast_analysis,
"logic_indexing": self.logic_indexing,
"synchronization": self.synchronization,
"integrity": self.integrity,
"manual_render": self.manual_render,
"graph_render": self.graph_render,
"live_viewer": self.live_viewer,
"profiling": self.profiling,
"blocked_tools": list(self.blocked_tools),
"prohibitions": list(self.prohibitions),
"precedence": list(POLICY_PRECEDENCE),
}
def adapter_policy(self) -> dict[str, object]:
"""Preserve the exact legacy adapter-policy projection."""
if not self.no_ast:
return {
"mode": "standard",
"ast_analysis": "allowed",
"logic_projection": "allowed",
"incremental_extraction": "allowed",
"adapter_rewrite": "not_requested",
}
return {
"mode": "preserve-no-ast",
"ast_analysis": "forbidden",
"logic_projection": "forbidden",
"incremental_extraction": "allowed",
"adapter_rewrite": "forbidden",
"blocked_tools": ["docforge_get_logic"],
"instruction": (
"Preserve the existing adapter extraction strategy. Do not add Python AST, "
"Tree-sitter, compiler-AST, or function-Logic extraction. Non-AST incremental "
"fingerprinting and caching remain allowed."
),
}
def compose_effective_policy(
*,
selected_mode: CapabilityMode,
capability_source: Literal["factory_default", "explicit"],
no_ast: bool,
diagnostics: bool,
render_configured: bool,
application_enabled: bool,
) -> EffectivePolicyV1:
"""Compose fixed defaults with restrictive compatibility shorthands."""
if selected_mode == "application" and not application_enabled:
raise DocForgeError(
"capability_unavailable",
"Application capability requires a startup-bound canonical applier",
capability_mode=selected_mode,
required="canonical_applier",
)
prohibitions = [
"arbitrary_file_access",
"arbitrary_renderer_execution",
"shell_execution",
"git_mutation",
"deployment",
"publication",
"project_switching",
]
blocked_tools: tuple[str, ...] = ()
if no_ast:
prohibitions.extend(
(
"adapter_ast_upgrade",
"tree_sitter_upgrade",
"compiler_ast_upgrade",
"function_logic_extraction",
)
)
blocked_tools = ("docforge_get_logic",)
manual_render: Literal["auto", "explicit", "disabled"]
if not render_configured:
manual_render = "disabled"
elif application_enabled and selected_mode in {"application", "operator"}:
manual_render = "auto"
else:
manual_render = "explicit"
return EffectivePolicyV1(
capability_mode=selected_mode,
capability_source=capability_source,
adapter_evolution="preserve" if no_ast else "allowed",
ast_analysis="forbidden" if no_ast else "allowed",
logic_indexing="off" if no_ast else "full",
synchronization="automatic",
integrity="validated",
manual_render=manual_render,
graph_render="disabled",
live_viewer="on-demand",
profiling="enabled" if diagnostics else "disabled",
blocked_tools=blocked_tools,
prohibitions=tuple(prohibitions),
)

180
src/docforge/project.py
View file

@ -11,6 +11,7 @@ import tempfile
import tomllib
from collections import Counter
from collections.abc import Mapping
from contextlib import suppress
from dataclasses import dataclass, replace
from pathlib import Path, PurePosixPath
from typing import Any, cast
@ -39,6 +40,7 @@ from .telemetry import increment, stage
SOURCE_GENERATION_SCHEMA_VERSION = 1
GENERIC_SOURCE_CONTRACT = "docforge-core:0.7.1:index:1"
MAX_PROJECT_DESCRIPTOR_BYTES = 1_000_000
_CORE_METADATA = frozenset(
{
@ -210,12 +212,167 @@ def _receipt_signature(path: Path) -> tuple[int, int, int, int, int] | None:
)
def _read_descriptor(descriptor_path: Path) -> bytes:
try:
parent = descriptor_path.parent
parent_status = parent.lstat()
if (
stat.S_ISLNK(parent_status.st_mode)
or not stat.S_ISDIR(parent_status.st_mode)
or parent.resolve(strict=True) != parent
):
raise DocForgeError(
"project_descriptor_unsafe",
"Project descriptor parent must be one real confined directory",
)
directory_fd = os.open(
parent,
os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW,
)
except FileNotFoundError as error:
raise DocForgeError("missing_config", "Missing .docforge/project.toml") from error
except DocForgeError:
raise
except OSError as error:
raise DocForgeError(
"project_descriptor_unsafe",
"Project descriptor cannot be inspected safely",
) from error
opened_parent = os.fstat(directory_fd)
if opened_parent.st_dev != parent_status.st_dev or opened_parent.st_ino != parent_status.st_ino:
with suppress(OSError):
os.close(directory_fd)
raise DocForgeError(
"project_descriptor_changed",
"Project descriptor parent changed while it was opened",
)
try:
def parent_current() -> bool:
try:
before = parent.lstat()
resolved = parent.resolve(strict=True)
after = parent.lstat()
opened = os.fstat(directory_fd)
return (
not stat.S_ISLNK(before.st_mode)
and stat.S_ISDIR(before.st_mode)
and resolved == parent
and (before.st_dev, before.st_ino, before.st_mode)
== (after.st_dev, after.st_ino, after.st_mode)
== (opened.st_dev, opened.st_ino, opened.st_mode)
)
except OSError:
return False
if not parent_current():
raise DocForgeError(
"project_descriptor_changed",
"Project descriptor parent changed before it was read",
)
try:
before = os.stat(
descriptor_path.name,
dir_fd=directory_fd,
follow_symlinks=False,
)
except FileNotFoundError as error:
raise DocForgeError("missing_config", "Missing .docforge/project.toml") from error
except OSError as error:
raise DocForgeError(
"project_descriptor_unsafe",
"Project descriptor cannot be inspected safely",
) from error
if stat.S_ISLNK(before.st_mode) or not stat.S_ISREG(before.st_mode):
raise DocForgeError(
"project_descriptor_unsafe",
"Project descriptor must be a regular file and not a symbolic link",
)
if before.st_size > MAX_PROJECT_DESCRIPTOR_BYTES:
raise DocForgeError(
"project_descriptor_oversized",
"Project descriptor exceeds the bounded configuration limit",
maximum_bytes=MAX_PROJECT_DESCRIPTOR_BYTES,
)
try:
descriptor = os.open(
descriptor_path.name,
os.O_RDONLY | os.O_NOFOLLOW,
dir_fd=directory_fd,
)
except OSError as error:
raise DocForgeError(
"project_descriptor_unsafe",
"Project descriptor cannot be opened safely",
) from error
try:
opened = os.fstat(descriptor)
if opened.st_dev != before.st_dev or opened.st_ino != before.st_ino:
raise DocForgeError(
"project_descriptor_changed",
"Project descriptor changed while it was opened",
)
chunks: list[bytes] = []
remaining = MAX_PROJECT_DESCRIPTOR_BYTES + 1
while remaining:
chunk = os.read(descriptor, min(65_536, remaining))
if not chunk:
break
chunks.append(chunk)
remaining -= len(chunk)
raw = b"".join(chunks)
finally:
with suppress(OSError):
os.close(descriptor)
if len(raw) > MAX_PROJECT_DESCRIPTOR_BYTES:
raise DocForgeError(
"project_descriptor_oversized",
"Project descriptor exceeds the bounded configuration limit",
maximum_bytes=MAX_PROJECT_DESCRIPTOR_BYTES,
)
try:
after = os.stat(
descriptor_path.name,
dir_fd=directory_fd,
follow_symlinks=False,
)
except OSError as error:
raise DocForgeError(
"project_descriptor_changed",
"Project descriptor changed while it was read",
) from error
if (
before.st_dev,
before.st_ino,
before.st_size,
before.st_mtime_ns,
before.st_ctime_ns,
) != (
after.st_dev,
after.st_ino,
after.st_size,
after.st_mtime_ns,
after.st_ctime_ns,
):
raise DocForgeError(
"project_descriptor_changed",
"Project descriptor changed while it was read",
)
if not parent_current():
raise DocForgeError(
"project_descriptor_changed",
"Project descriptor parent changed while it was read",
)
return raw
finally:
with suppress(OSError):
os.close(directory_fd)
def _load_descriptor(root: Path) -> ProjectDescriptor:
descriptor_path = root / ".docforge" / "project.toml"
if not descriptor_path.is_file():
raise DocForgeError("missing_config", "Missing .docforge/project.toml")
descriptor_bytes = _read_descriptor(descriptor_path)
try:
descriptor_bytes = descriptor_path.read_bytes()
document = cast(dict[str, object], tomllib.loads(descriptor_bytes.decode("utf-8")))
except UnicodeDecodeError as error:
raise DocForgeError("invalid_config", "Project descriptor is not UTF-8") from error
@ -472,6 +629,17 @@ def _load_descriptor(root: Path) -> ProjectDescriptor:
)
def validate_descriptor_binding(descriptor: ProjectDescriptor) -> None:
"""Require the bounded descriptor bytes to match one opened project binding."""
descriptor_bytes = _read_descriptor(descriptor.descriptor_path)
if hashlib.sha256(descriptor_bytes).hexdigest() != descriptor.descriptor_hash:
raise DocForgeError(
"source_changed",
"Project descriptor changed after the project was opened",
)
def _markdown_record(path: Path, text: str) -> tuple[dict[str, Any], str]:
lines = text.splitlines()
if not lines or lines[0] != "+++":
@ -739,11 +907,7 @@ class Project:
def load(self) -> ProjectSnapshot:
increment("project_loads")
descriptor_bytes = self.descriptor.descriptor_path.read_bytes()
if hashlib.sha256(descriptor_bytes).hexdigest() != self.descriptor.descriptor_hash:
raise DocForgeError(
"source_changed", "Project descriptor changed after the project was opened"
)
validate_descriptor_binding(self.descriptor)
ordered_sources, ordered_directories = self._canonical_inventory()
generation_paths = (
self.descriptor.descriptor_path,

671
src/docforge/retrieval.py Normal file
View file

@ -0,0 +1,671 @@
"""Versioned task-shaped retrieval plans and immutable context capsules."""
from __future__ import annotations
from dataclasses import dataclass, replace
from typing import Literal, cast
from .errors import DocForgeError
from .models import Edge, Node, ProjectDescriptor
from .pagination import canonical_hash
TaskKind = Literal[
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release",
]
TASK_KINDS: tuple[TaskKind, ...] = (
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release",
)
RelationCategory = Literal[
"structure",
"implementation",
"dependency",
"execution",
"data",
"evidence",
"context",
"unclassified",
]
BASE_RELATION_CATEGORIES: tuple[RelationCategory, ...] = (
"structure",
"implementation",
"dependency",
"execution",
"data",
"evidence",
"context",
)
RELATION_CATEGORIES: dict[RelationCategory, tuple[str, ...]] = {
"structure": ("contains", "defined_in", "defines", "owns"),
"implementation": (
"implemented_by",
"implements",
"inherits",
"inherits_from",
),
"dependency": ("depends_on", "imports"),
"execution": ("activates", "calls", "dispatches_to", "launches"),
"data": ("reads", "writes"),
"evidence": ("documents", "governs", "proves", "tested_by", "verifies"),
"context": ("relates_to",),
"unclassified": (),
}
TASK_REQUIREMENTS: dict[TaskKind, tuple[RelationCategory, ...]] = {
"change": ("dependency",),
"implementation": ("implementation",),
"failure": ("execution",),
"ownership": ("structure",),
"test": ("evidence",),
"operation": ("execution",),
"release": ("evidence",),
}
PLANNER_ID = "docforge.core.task-context"
PLANNER_VERSION = 1
MAX_TASK_EVIDENCE = 1_000
MAX_TASK_CANDIDATE_EDGES = 100_000
MAX_TASK_QUERY_CHARS = 10_000
PROVENANCE_LIMITATIONS = (
"evidence_type_unavailable",
"extractor_identity_unavailable",
"relationship_provenance_unavailable",
"observation_time_unavailable",
)
_RELATION_TO_CATEGORY = {
relation: category
for category, relations in RELATION_CATEGORIES.items()
for relation in relations
}
def relation_category(relation: str) -> RelationCategory:
"""Classify only versioned known aliases; preserve every other relation."""
return cast(RelationCategory, _RELATION_TO_CATEGORY.get(relation, "unclassified"))
@dataclass(frozen=True)
class RetrievalStepV1:
step_id: str
operation: Literal["exact", "search", "outgoing", "incoming", "metadata"]
relation_scope: Literal["none", "project_allowed"]
relation_set_hash: str | None
direction: Literal["none", "outgoing", "incoming"]
depth: int
limit: int
required: bool
evidence_role: str
def as_dict(self) -> dict[str, object]:
return {
"step_id": self.step_id,
"operation": self.operation,
"relation_scope": self.relation_scope,
"relation_set_hash": self.relation_set_hash,
"direction": self.direction,
"depth": self.depth,
"limit": self.limit,
"required": self.required,
"evidence_role": self.evidence_role,
}
@dataclass(frozen=True)
class RetrievalRequirementV1:
requirement_id: str
category: RelationCategory
def as_dict(self) -> dict[str, object]:
return {
"requirement_id": self.requirement_id,
"check": "selected_relation_category",
"category": self.category,
"required": True,
}
@dataclass(frozen=True)
class RetrievalPlanV1:
schema_version: Literal[1]
planner_id: str
planner_version: int
task_kind: TaskKind
task_query: str
focus_node_id: str | None
request_hash: str
effective_policy_hash: str
max_evidence: int
max_tokens: int
max_depth: int
max_candidate_edges: int
category_order: tuple[RelationCategory, ...]
steps: tuple[RetrievalStepV1, ...]
requirements: tuple[RetrievalRequirementV1, ...]
plan_hash: str
def as_dict(self) -> dict[str, object]:
return self.payload(include_hash=True)
def payload(self, *, include_hash: bool) -> dict[str, object]:
result: dict[str, object] = {
"schema_version": self.schema_version,
"planner": {
"id": self.planner_id,
"version": self.planner_version,
},
"task_kind": self.task_kind,
"request_hash": self.request_hash,
"effective_policy_hash": self.effective_policy_hash,
"focus_node_id": self.focus_node_id,
"limits": {
"max_evidence": self.max_evidence,
"max_tokens": self.max_tokens,
"max_depth": self.max_depth,
"max_candidate_edges": self.max_candidate_edges,
},
"category_order": list(self.category_order),
"steps": [step.as_dict() for step in self.steps],
"requirements": [requirement.as_dict() for requirement in self.requirements],
}
if include_hash:
result["plan_hash"] = self.plan_hash
return result
def build_retrieval_plan(
descriptor: ProjectDescriptor,
*,
task_kind: str,
task: str,
focus_node_id: str | None,
budget: int | None,
limit: int | None,
effective_policy: dict[str, object],
) -> RetrievalPlanV1:
"""Derive one fixed plan from bounded inputs rather than accepting caller operations."""
return _build_retrieval_plan(
descriptor,
task_kind=task_kind,
task=task,
focus_node_id=focus_node_id,
budget=budget,
limit=limit,
effective_policy_hash=canonical_hash(effective_policy),
)
def validate_retrieval_plan(
plan: RetrievalPlanV1,
descriptor: ProjectDescriptor,
) -> RetrievalPlanV1:
"""Reject forged, stale-shape, or internally inconsistent public plan objects."""
try:
expected = _build_retrieval_plan(
descriptor,
task_kind=plan.task_kind,
task=plan.task_query,
focus_node_id=plan.focus_node_id,
budget=plan.max_tokens,
limit=plan.max_evidence,
effective_policy_hash=plan.effective_policy_hash,
)
except (AttributeError, TypeError, DocForgeError) as error:
raise DocForgeError(
"invalid_retrieval_plan",
"Task retrieval plan is malformed or outside the fixed version-1 contract",
) from error
if plan != expected:
raise DocForgeError(
"invalid_retrieval_plan",
"Task retrieval plan does not match its fixed version-1 derivation",
)
return plan
def _build_retrieval_plan(
descriptor: ProjectDescriptor,
*,
task_kind: str,
task: str,
focus_node_id: str | None,
budget: int | None,
limit: int | None,
effective_policy_hash: str,
) -> RetrievalPlanV1:
if task_kind not in TASK_KINDS:
raise DocForgeError(
"invalid_task_kind",
"Task context kind is unsupported",
task_kind=task_kind,
allowed=list(TASK_KINDS),
)
selected_kind: TaskKind = task_kind # type: ignore[assignment]
normalized_task = task.strip()
if not normalized_task or len(normalized_task) > min(
descriptor.limits.max_query_chars,
MAX_TASK_QUERY_CHARS,
):
raise DocForgeError(
"invalid_task_focus",
"Task description is empty or exceeds the configured query limit",
)
if focus_node_id is not None and (not focus_node_id or len(focus_node_id) > 256):
raise DocForgeError("invalid_task_focus", "Task focus node ID is invalid")
selected_budget = _bounded_value(
budget,
default=min(8_000, descriptor.limits.max_context_tokens),
maximum=descriptor.limits.max_context_tokens,
code="invalid_budget",
)
selected_limit = _bounded_value(
limit,
default=min(20, descriptor.limits.max_results),
maximum=min(descriptor.limits.max_results, MAX_TASK_EVIDENCE),
code="invalid_limit",
)
if not _is_sha256(effective_policy_hash):
raise DocForgeError(
"invalid_retrieval_plan",
"Effective policy identity is not a SHA-256 value",
)
selected_depth = min(2, descriptor.limits.max_traversal_depth)
requirements = tuple(
RetrievalRequirementV1(
requirement_id=f"{selected_kind}.{category}",
category=category,
)
for category in TASK_REQUIREMENTS[selected_kind]
)
category_order: tuple[RelationCategory, ...] = (
*TASK_REQUIREMENTS[selected_kind],
*(
category
for category in BASE_RELATION_CATEGORIES
if category not in TASK_REQUIREMENTS[selected_kind]
),
"unclassified",
)
relation_set_hash = canonical_hash(sorted(descriptor.allowed_relations))
focus_operation: Literal["exact", "search"] = "exact" if focus_node_id else "search"
steps = (
RetrievalStepV1(
step_id="focus",
operation=focus_operation,
relation_scope="none",
relation_set_hash=None,
direction="none",
depth=0,
limit=1,
required=True,
evidence_role="focus",
),
RetrievalStepV1(
step_id="outgoing",
operation="outgoing",
relation_scope="project_allowed",
relation_set_hash=relation_set_hash,
direction="outgoing",
depth=selected_depth,
limit=selected_limit,
required=False,
evidence_role="related",
),
RetrievalStepV1(
step_id="incoming",
operation="incoming",
relation_scope="project_allowed",
relation_set_hash=relation_set_hash,
direction="incoming",
depth=selected_depth,
limit=selected_limit,
required=False,
evidence_role="related",
),
RetrievalStepV1(
step_id="metadata",
operation="metadata",
relation_scope="none",
relation_set_hash=None,
direction="none",
depth=0,
limit=selected_limit,
required=True,
evidence_role="provenance",
),
)
request_hash = canonical_hash(
{
"task_kind": selected_kind,
"task": normalized_task,
"focus_node_id": focus_node_id,
"budget": selected_budget,
"limit": selected_limit,
}
)
placeholder = RetrievalPlanV1(
schema_version=1,
planner_id=PLANNER_ID,
planner_version=PLANNER_VERSION,
task_kind=selected_kind,
task_query=normalized_task,
focus_node_id=focus_node_id,
request_hash=request_hash,
effective_policy_hash=effective_policy_hash,
max_evidence=selected_limit,
max_tokens=selected_budget,
max_depth=selected_depth,
max_candidate_edges=min(
(selected_limit + 1) ** 2,
MAX_TASK_CANDIDATE_EDGES,
),
category_order=category_order,
steps=steps,
requirements=requirements,
plan_hash="",
)
return replace(
placeholder,
plan_hash=canonical_hash(placeholder.payload(include_hash=False)),
)
@dataclass(frozen=True)
class CapsuleRelationshipV1:
source_id: str
relation: str
target_id: str
direction: Literal["outgoing", "incoming"]
category: RelationCategory
def as_dict(self) -> dict[str, object]:
return {
"source_id": self.source_id,
"relation": self.relation,
"target_id": self.target_id,
"direction": self.direction,
"category": self.category,
"provenance": "validated_graph_edge_without_source_provenance",
}
@dataclass(frozen=True)
class CapsuleEvidenceV1:
evidence_hash: str
role: Literal["focus", "related"]
reason_code: Literal["exact_focus", "lexical_focus", "relationship_path"]
node: Node
depth: int
relationship_path: tuple[CapsuleRelationshipV1, ...]
relationship_reasons: tuple[CapsuleRelationshipV1, ...]
estimated_tokens: int
def as_dict(self) -> dict[str, object]:
result = self.payload()
return {"evidence_hash": self.evidence_hash, **result}
def payload(self) -> dict[str, object]:
result: dict[str, object] = {
"role": self.role,
"reason_code": self.reason_code,
"node_id": self.node.node_id,
"title": self.node.title,
"family": self.node.family,
"authority": self.node.authority,
"status": self.node.status,
"tags": list(self.node.tags),
"summary": self.node.summary,
"text": _node_text(self.node),
"estimated_tokens": self.estimated_tokens,
"source": {
"path": self.node.source_path,
"anchor": self.node.source_anchor,
"content_hash": self.node.content_hash,
},
"depth": self.depth,
"relationship_path": [
relationship.as_dict() for relationship in self.relationship_path
],
"relationship_reasons": [
relationship.as_dict() for relationship in self.relationship_reasons
],
"provenance_limitations": list(PROVENANCE_LIMITATIONS),
}
return result
def capsule_evidence(
*,
role: Literal["focus", "related"],
reason_code: Literal["exact_focus", "lexical_focus", "relationship_path"],
node: Node,
depth: int,
relationship_path: tuple[CapsuleRelationshipV1, ...],
relationship_reasons: tuple[CapsuleRelationshipV1, ...],
) -> CapsuleEvidenceV1:
tokens = estimate_tokens(_node_text(node))
placeholder = CapsuleEvidenceV1(
evidence_hash="",
role=role,
reason_code=reason_code,
node=node,
depth=depth,
relationship_path=relationship_path,
relationship_reasons=relationship_reasons,
estimated_tokens=tokens,
)
return replace(
placeholder,
evidence_hash=canonical_hash(placeholder.payload()),
)
@dataclass(frozen=True)
class EvidenceGapV1:
code: Literal[
"focus_not_found",
"focus_ambiguous",
"category_not_declared",
"no_selected_evidence",
"evidence_incomplete",
"unclassified_relation",
]
requirement_id: str
category: RelationCategory | None
state: Literal["missing", "incomplete", "blocked", "limitation"]
check_complete: bool
detail: str
def as_dict(self) -> dict[str, object]:
return {
"code": self.code,
"requirement_id": self.requirement_id,
"category": self.category,
"state": self.state,
"check_complete": self.check_complete,
"detail": self.detail,
}
@dataclass(frozen=True)
class CapsuleOmissionV1:
code: Literal[
"result_limit",
"token_budget",
"response_limit",
"edge_examination_limit",
"unclassified_relation_limit",
]
subject: str
detail_hash: str
def as_dict(self) -> dict[str, str]:
return {
"code": self.code,
"subject": self.subject,
"detail_hash": self.detail_hash,
}
@dataclass(frozen=True)
class ContextCapsuleV1:
schema_version: Literal[1]
state: Literal["complete", "incomplete", "blocked"]
task_kind: TaskKind
generation: tuple[tuple[str, object], ...]
plan: RetrievalPlanV1
focus_state: Literal["resolved", "not_found", "ambiguous"]
focus_node_id: str | None
focus_candidate_count: int
evidence: tuple[CapsuleEvidenceV1, ...]
gaps: tuple[EvidenceGapV1, ...]
omissions: tuple[CapsuleOmissionV1, ...]
selected_count: int
examined_edge_count: int
estimated_tokens: int
unclassified_relations: tuple[str, ...]
collection_hash: str
capsule_hash: str
def as_dict(self) -> dict[str, object]:
return self.payload(include_hashes=True)
def payload(self, *, include_hashes: bool) -> dict[str, object]:
evidence = [item.as_dict() for item in self.evidence]
gaps = [gap.as_dict() for gap in self.gaps]
omissions = [omission.as_dict() for omission in self.omissions]
result: dict[str, object] = {
"schema_version": self.schema_version,
"state": self.state,
"task_kind": self.task_kind,
"generation": dict(self.generation),
"plan": self.plan.as_dict(),
"focus": {
"state": self.focus_state,
"node_id": self.focus_node_id,
"candidate_count": self.focus_candidate_count,
},
"evidence": evidence,
"gaps": gaps,
"omissions": omissions,
"summary": {
"evidence_count": len(evidence),
"gap_count": len(gaps),
"omission_count": len(omissions),
"selected_count": self.selected_count,
"examined_edge_count": self.examined_edge_count,
"estimated_tokens": self.estimated_tokens,
"unclassified_relations": list(self.unclassified_relations),
},
}
if include_hashes:
result["collection_hash"] = self.collection_hash
result["capsule_hash"] = self.capsule_hash
return result
def finalize_capsule(
*,
state: Literal["complete", "incomplete", "blocked"],
task_kind: TaskKind,
generation: dict[str, object],
plan: RetrievalPlanV1,
focus_state: Literal["resolved", "not_found", "ambiguous"],
focus_node_id: str | None,
focus_candidate_count: int,
evidence: tuple[CapsuleEvidenceV1, ...],
gaps: tuple[EvidenceGapV1, ...],
omissions: tuple[CapsuleOmissionV1, ...],
selected_count: int,
examined_edge_count: int,
estimated_tokens: int,
unclassified_relations: tuple[str, ...],
) -> ContextCapsuleV1:
collection_hash = canonical_hash(
{
"generation": generation,
"plan_hash": plan.plan_hash,
"evidence": [item.evidence_hash for item in evidence],
"gaps": [gap.as_dict() for gap in gaps],
"omissions": [omission.as_dict() for omission in omissions],
}
)
placeholder = ContextCapsuleV1(
schema_version=1,
state=state,
task_kind=task_kind,
generation=tuple(generation.items()),
plan=plan,
focus_state=focus_state,
focus_node_id=focus_node_id,
focus_candidate_count=focus_candidate_count,
evidence=evidence,
gaps=gaps,
omissions=omissions,
selected_count=selected_count,
examined_edge_count=examined_edge_count,
estimated_tokens=estimated_tokens,
unclassified_relations=unclassified_relations,
collection_hash=collection_hash,
capsule_hash="",
)
capsule_hash = canonical_hash(
{
**placeholder.payload(include_hashes=False),
"collection_hash": collection_hash,
}
)
return replace(placeholder, capsule_hash=capsule_hash)
def estimate_tokens(text: str) -> int:
return max(1, (len(text) + 3) // 4)
def edge_tuple(edge: Edge) -> tuple[str, str, str]:
return edge.source_id, edge.relation, edge.target_id
def _node_text(node: Node) -> str:
return (
f"ID: {node.node_id}\nTitle: {node.title}\nFamily: {node.family}\n"
f"Authority: {node.authority}\nStatus: {node.status}\nSource: {node.source_path}\n"
f"Summary: {node.summary}\n\n{node.content}"
)
def _bounded_value(
value: int | None,
*,
default: int,
maximum: int,
code: str,
) -> int:
selected = default if value is None else value
if type(selected) is not int or selected < 1 or selected > maximum:
raise DocForgeError(code, "Task context limit is outside the configured range")
return selected
def _is_sha256(value: object) -> bool:
return (
isinstance(value, str)
and len(value) == 64
and all(character in "0123456789abcdef" for character in value)
)

6
src/docforge/telemetry.py
View file

@ -78,6 +78,7 @@ OPERATION_NAMES = frozenset(
{
"test",
"benchmark.m1",
"benchmark.m2",
"mcp.invoke",
"mcp.bootstrap",
"mcp.sync",
@ -91,6 +92,8 @@ OPERATION_NAMES = frozenset(
"mcp.dependencies",
"mcp.impact",
"mcp.context",
"mcp.task_context",
"mcp.generation_diff",
"mcp.validate_project",
"mcp.render_status",
"mcp.visualize",
@ -113,6 +116,9 @@ OPERATION_NAMES = frozenset(
"cli.dependencies",
"cli.impact",
"cli.context",
"cli.generation-diff",
"cli.configure",
"cli.doctor",
"cli.render",
"cli.render-status",
"cli.preview",

33
tests/test_adapter_contract.py
View file

@ -2,6 +2,7 @@ from __future__ import annotations
import hashlib
import importlib
import json
import sqlite3
import sys
import tempfile
@ -55,8 +56,10 @@ from docforge.visualization import VisualizationIndexSnapshot
class Loader:
def __init__(self, projection: AdapterProjection) -> None:
self.projection = projection
self.load_calls = 0
def load_projection(self) -> AdapterProjection:
self.load_calls += 1
return self.projection
@ -643,12 +646,17 @@ class AdapterContractTests(unittest.TestCase):
def test_no_ast_index_accepts_legacy_and_non_logic_incremental_adapters(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory).resolve()
legacy_loader = Loader(self.projection(root))
legacy = AdapterProject(
Loader(self.projection(root)),
legacy_loader,
cache_root=root / ".cache" / "legacy-no-ast",
)
legacy_index = ProjectIndex(legacy, allow_logic=False)
self.assertEqual(2, legacy_index.build()["node_count"])
calls_after_build = legacy_loader.load_calls
legacy_diff = legacy_index.generation_diff()
self.assertEqual("unknown", legacy_diff["receipt_state"])
self.assertEqual(calls_after_build, legacy_loader.load_calls)
self.assertEqual(
"guide.workflow",
legacy_index.get_node("guide.workflow")["node"]["node_id"],
@ -672,6 +680,12 @@ class AdapterContractTests(unittest.TestCase):
self.assertEqual(2, second["build"]["cache_hits"])
self.assertEqual(0, second["build"]["reparsed_sources"])
self.assertEqual(0, second["logic_projection_count"])
generation_diff = index.generation_diff()
self.assertEqual("current", generation_diff["receipt_state"])
self.assertNotIn(
"logic",
json.dumps(generation_diff["generation_diff"]).casefold(),
)
def test_no_ast_rejects_preexisting_logic_index_and_viewer_snapshot(self) -> None:
with tempfile.TemporaryDirectory() as directory:
@ -866,8 +880,9 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory).resolve()
fixture = AdapterContractTests()
loader = Loader(fixture.projection(root))
project = AdapterProject(
Loader(fixture.projection(root)),
loader,
cache_root=root / ".cache" / "adapter-read-only",
)
index = ProjectIndex(project)
@ -903,6 +918,14 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
context = await session.call_tool(
"docforge_get_context", {"profile": "fixture", "budget": 321}
)
load_calls = loader.load_calls
task_context = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Do not load the legacy projection for this capability error",
},
)
self.assertEqual(READ_TOOLS, tuple(tool.name for tool in tools.tools))
self.assertEqual("adapter-fixture", info.structuredContent["project_id"])
@ -915,6 +938,12 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertFalse(contract.structuredContent["isolated_changeset_writes_allowed"])
self.assertEqual("fixture", context.structuredContent["profile"])
self.assertEqual([("fixture", 321)], calls)
self.assertEqual("error", task_context.structuredContent["status"])
self.assertEqual(
"task_context_unavailable",
task_context.structuredContent["error"]["code"],
)
self.assertEqual(load_calls, loader.load_calls)
self.assertFalse(project.descriptor.changeset_root.exists())
async def test_adapter_project_proposals_require_explicit_policy_and_stay_isolated(

1264
tests/test_client_integration.py Normal file
View file

File diff suppressed because it is too large Load diff

993
tests/test_generation_diff.py Normal file
View file

@ -0,0 +1,993 @@
from __future__ import annotations
import contextlib
import io
import json
import shutil
import sqlite3
import tempfile
import unittest
from collections.abc import Callable
from dataclasses import replace
from pathlib import Path
from unittest import mock
from jsonschema import Draft202012Validator
from mcp.shared.memory import create_connected_server_and_client_session
import docforge.generation_diff as generation_diff_module
from docforge.cli import main
from docforge.errors import DocForgeError
from docforge.generation_diff import (
GenerationDiffDraft,
finalize_generation_diff,
generation_diff_path,
publish_generation_diff,
validate_generation_diff_receipt,
)
from docforge.index import ProjectIndex
from docforge.mcp_server import DocForgeService, create_server
from docforge.models import ProjectDescriptor, ProjectSnapshot
from docforge.pagination import canonical_hash
from docforge.project import Project, project_root_fingerprint
from docforge.telemetry import request
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
GENERATION_DIFF_SCHEMA = json.loads(
(ROOT / "schemas" / "generation-diff.schema.json").read_text(encoding="utf-8")
)
GENERATION_DIFF_PAGE_SCHEMA = json.loads(
(ROOT / "schemas" / "generation-diff-page.schema.json").read_text(encoding="utf-8")
)
RESULT_SCHEMA = json.loads((ROOT / "schemas" / "result.schema.json").read_text(encoding="utf-8"))
class StaticProject:
"""Small legacy one-method project used to prove load-free status behavior."""
def __init__(self, snapshot: ProjectSnapshot) -> None:
self.descriptor: ProjectDescriptor = snapshot.descriptor
self.snapshot = snapshot
self.load_calls = 0
def load(self) -> ProjectSnapshot:
self.load_calls += 1
return self.snapshot
def canonical_source_paths(self) -> tuple[Path, ...]:
return ()
def validate_proposal(
self,
base: ProjectSnapshot,
projected: ProjectSnapshot,
operations: tuple[object, ...],
) -> None:
del base, projected, operations
class GenerationDiffTests(unittest.TestCase):
def copy_fixture(self, destination: Path) -> Path:
root = destination / "alpha"
shutil.copytree(FIXTURES / "alpha", root)
shutil.rmtree(root / ".docforge" / "cache", ignore_errors=True)
return root
@staticmethod
def change_graph(root: Path, suffix: str = "changed") -> None:
foundation = root / "docs" / "content" / "foundation.md"
foundation.write_text(
foundation.read_text(encoding="utf-8").replace(
"Defines which Alpha files own documentation facts.",
f"Defines which Alpha files own documentation facts. {suffix}",
),
encoding="utf-8",
)
proof = root / "docs" / "content" / "proof.toml"
proof.write_text(
proof.read_text(encoding="utf-8").replace(
'proves = ["guide.workflow"]',
'proves = ["guide.foundation"]',
),
encoding="utf-8",
)
@staticmethod
def rehash_receipt(receipt: dict[str, object]) -> dict[str, object]:
items = receipt["items"]
if not isinstance(items, list):
raise AssertionError("receipt items are not a list")
for item in items:
if not isinstance(item, dict):
raise AssertionError("receipt item is not an object")
item["item_hash"] = canonical_hash(
{key: value for key, value in item.items() if key != "item_hash"}
)
retained_hash = canonical_hash([item["item_hash"] for item in items])
receipt["retained_collection_hash"] = retained_hash
if not receipt["details_truncated"]:
receipt["full_collection_hash"] = retained_hash
receipt["receipt_hash"] = canonical_hash(
{key: value for key, value in receipt.items() if key != "receipt_hash"}
)
return receipt
def transition_receipt(self, root: Path) -> tuple[Project, ProjectIndex, dict[str, object]]:
project = Project.open(root)
index = ProjectIndex(project)
index.build()
self.change_graph(root)
index.build()
receipt = json.loads(generation_diff_path(project.descriptor).read_text(encoding="utf-8"))
return project, index, receipt
def test_first_build_and_exact_transition_are_schema_valid(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
first = index.build()
first_receipt = json.loads(
generation_diff_path(project.descriptor).read_text(encoding="utf-8")
)
Draft202012Validator(GENERATION_DIFF_SCHEMA).validate(first_receipt)
self.assertTrue(
validate_generation_diff_receipt(
first_receipt,
descriptor=project.descriptor,
)
)
self.assertEqual("baseline", first_receipt["kind"])
self.assertEqual("no_predecessor", first_receipt["reason"])
self.assertEqual(0, first_receipt["full_item_count"])
self.assertEqual("ok", first["status"])
self.change_graph(root)
second = index.build()
receipt = json.loads(
generation_diff_path(project.descriptor).read_text(encoding="utf-8")
)
Draft202012Validator(GENERATION_DIFF_SCHEMA).validate(receipt)
self.assertEqual("transition", receipt["kind"])
self.assertEqual(
{
"nodes_added": 0,
"nodes_removed": 0,
"nodes_changed": 2,
"edges_added": 1,
"edges_removed": 1,
"total_changes": 4,
},
receipt["summary"],
)
self.assertEqual(
[
("edge", "added"),
("edge", "removed"),
("node", "changed"),
("node", "changed"),
],
[(item["entity"], item["change"]) for item in receipt["items"]],
)
self.assertNotIn("logic_hash", json.dumps(receipt, sort_keys=True).casefold())
self.assertEqual("ok", second["status"])
status = ProjectIndex(project).generation_diff()
self.assertEqual("current", status["receipt_state"])
self.assertEqual(receipt["receipt_hash"], status["generation_diff"]["receipt_hash"])
def test_same_generation_reindex_preserves_latest_meaningful_transition(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
self.change_graph(root)
index.build()
before = json.loads(
generation_diff_path(project.descriptor).read_text(encoding="utf-8")
)
index.build()
after = json.loads(generation_diff_path(project.descriptor).read_text(encoding="utf-8"))
self.assertEqual("transition", after["kind"])
self.assertEqual(before["from_generation"], after["from_generation"])
self.assertEqual(before["to_generation"], after["to_generation"])
self.assertEqual(before["summary"], after["summary"])
self.assertEqual(before["full_collection_hash"], after["full_collection_hash"])
self.assertNotEqual(before["index_signature"], after["index_signature"])
self.assertEqual(
["generation-diff.json"],
[path.name for path in project.descriptor.cache_root.glob("*generation-diff*")],
)
def test_receipt_item_and_byte_limits_preserve_exact_summary_hashes(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
descriptor = Project.open(root).descriptor
generation = {
"revision": "unversioned",
"source_hash": "1" * 64,
"node_count": 0,
"node_hash": "2" * 64,
"edge_count": 0,
"edge_hash": "3" * 64,
"index_schema_version": 3,
}
items: list[dict[str, object]] = []
for index in range(1_002):
item: dict[str, object] = {
"entity": "edge",
"change": "added",
"source_id": f"source-{index:04d}",
"relation": "relates_to",
"target_id": f"target-{index:04d}",
}
item["item_hash"] = canonical_hash(item)
items.append(item)
fields: dict[str, object] = {
"schema_version": 1,
"diff_semantics_version": 1,
"project_id": descriptor.project_id,
"project_root_fingerprint": project_root_fingerprint(descriptor.root),
"adapter": descriptor.adapter,
"kind": "transition",
"reason": None,
"from_generation": generation,
"to_generation": {**generation, "source_hash": "4" * 64},
"summary": {
"nodes_added": 0,
"nodes_removed": 0,
"nodes_changed": 0,
"edges_added": len(items),
"edges_removed": 0,
"total_changes": len(items),
},
"full_item_count": len(items),
"full_collection_hash": canonical_hash([item["item_hash"] for item in items]),
}
receipt = finalize_generation_diff(
GenerationDiffDraft(fields=fields, items=tuple(items)),
signature=(1, 2, 3, 4, 5),
)
Draft202012Validator(GENERATION_DIFF_SCHEMA).validate(receipt)
self.assertEqual(1_002, receipt["full_item_count"])
self.assertEqual(1_000, receipt["retained_item_count"])
self.assertEqual("receipt_item_limit", receipt["truncation_reason"])
self.assertNotEqual(
receipt["full_collection_hash"],
receipt["retained_collection_hash"],
)
huge = dict(items[0])
huge["source_id"] = "source-" + ("x" * 1_100_000)
huge["item_hash"] = canonical_hash(
{key: value for key, value in huge.items() if key != "item_hash"}
)
huge_fields = {
**fields,
"summary": {
**fields["summary"],
"edges_added": 1,
"total_changes": 1,
},
"full_item_count": 1,
"full_collection_hash": canonical_hash([huge["item_hash"]]),
}
byte_limited = finalize_generation_diff(
GenerationDiffDraft(fields=huge_fields, items=(huge,)),
signature=(1, 2, 3, 4, 5),
)
self.assertEqual(0, byte_limited["retained_item_count"])
self.assertEqual("receipt_byte_limit", byte_limited["truncation_reason"])
def test_receipt_runtime_and_schema_reject_malformed_semantics(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project, _, receipt = self.transition_receipt(root)
validator = Draft202012Validator(GENERATION_DIFF_SCHEMA)
malformed_fields = json.loads(json.dumps(receipt))
changed = next(
item
for item in malformed_fields["items"]
if item["entity"] == "node" and item["change"] == "changed"
)
changed["changed_fields"] = 1
self.rehash_receipt(malformed_fields)
self.assertFalse(
validate_generation_diff_receipt(
malformed_fields,
descriptor=project.descriptor,
)
)
self.assertFalse(validator.is_valid(malformed_fields))
duplicate_fields = json.loads(json.dumps(receipt))
changed = next(
item
for item in duplicate_fields["items"]
if item["entity"] == "node" and item["change"] == "changed"
)
changed["changed_fields"] = ["content", "content"]
self.rehash_receipt(duplicate_fields)
self.assertFalse(validate_generation_diff_receipt(duplicate_fields))
self.assertFalse(validator.is_valid(duplicate_fields))
impossible_added = json.loads(json.dumps(receipt))
changed = next(
item
for item in impossible_added["items"]
if item["entity"] == "node" and item["change"] == "changed"
)
changed["change"] = "added"
changed["changed_fields"] = []
self.rehash_receipt(impossible_added)
self.assertFalse(validate_generation_diff_receipt(impossible_added))
self.assertFalse(validator.is_valid(impossible_added))
reordered = json.loads(json.dumps(receipt))
reordered["items"].reverse()
self.rehash_receipt(reordered)
self.assertFalse(validate_generation_diff_receipt(reordered))
same_generation = json.loads(json.dumps(receipt))
same_generation["from_generation"] = same_generation["to_generation"]
self.rehash_receipt(same_generation)
self.assertFalse(validate_generation_diff_receipt(same_generation))
impossible_truncation = json.loads(json.dumps(receipt))
impossible_truncation["items"].pop()
impossible_truncation["retained_item_count"] = len(impossible_truncation["items"])
impossible_truncation["details_truncated"] = True
impossible_truncation["truncation_reason"] = "receipt_item_limit"
self.rehash_receipt(impossible_truncation)
self.assertFalse(validate_generation_diff_receipt(impossible_truncation))
self.assertFalse(validator.is_valid(impossible_truncation))
baseline_with_changes = json.loads(json.dumps(receipt))
baseline_with_changes["kind"] = "baseline"
baseline_with_changes["reason"] = "no_meaningful_transition"
baseline_with_changes["from_generation"] = None
self.rehash_receipt(baseline_with_changes)
self.assertFalse(validate_generation_diff_receipt(baseline_with_changes))
self.assertFalse(validator.is_valid(baseline_with_changes))
def test_generation_collision_and_nondeterministic_build_fail_precommit(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
base = Project.open(root).load()
static = StaticProject(base)
index = ProjectIndex(static)
index.build()
original_index = index.path.read_bytes()
original_receipt = generation_diff_path(static.descriptor).read_bytes()
changed_node = replace(
base.nodes[0],
title="Different graph under the same generation",
)
static.snapshot = replace(
base,
nodes=(changed_node, *base.nodes[1:]),
)
with self.assertRaises(DocForgeError) as collision:
index.build()
self.assertEqual("generation_collision", collision.exception.code)
self.assertEqual(original_index, index.path.read_bytes())
self.assertEqual(
original_receipt,
generation_diff_path(static.descriptor).read_bytes(),
)
calls = 0
def unstable() -> ProjectSnapshot:
nonlocal calls
calls += 1
return (
base
if calls == 1
else replace(
base,
nodes=(changed_node, *base.nodes[1:]),
)
)
static.snapshot = base
with (
mock.patch.object(static, "load", side_effect=unstable),
self.assertRaises(DocForgeError) as changed,
):
ProjectIndex(static).build()
self.assertEqual("source_changed", changed.exception.code)
def test_post_commit_receipt_failures_report_degraded_success(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
self.change_graph(root)
with (
mock.patch.object(
ProjectIndex,
"_write_attestation",
side_effect=OSError("attestation failed"),
),
mock.patch.object(
Project,
"record_generation",
side_effect=OSError("generation failed"),
),
mock.patch(
"docforge.index.publish_generation_diff",
side_effect=OSError("diff failed"),
),
):
result = index.build()
self.assertEqual("ok", result["status"])
self.assertEqual("degraded", result["publication"]["state"])
self.assertEqual("published", result["publication"]["index"])
self.assertEqual(
{"attestation", "source_generation", "generation_diff"},
{error["stage"] for error in result["publication"]["errors"]},
)
with contextlib.closing(__import__("sqlite3").connect(index.path)) as connection:
metadata = dict(connection.execute("SELECT key, value FROM metadata"))
self.assertEqual(project.load().source_hash, metadata["source_hash"])
def test_predecessor_requires_attestation_and_valid_generation_identity(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
index.attestation_path.unlink()
self.change_graph(root)
result = index.build()
receipt = json.loads(
generation_diff_path(project.descriptor).read_text(encoding="utf-8")
)
self.assertEqual("ok", result["status"])
self.assertEqual("baseline", receipt["kind"])
self.assertEqual("predecessor_unattested", receipt["reason"])
self.assertTrue(validate_generation_diff_receipt(receipt))
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
with contextlib.closing(sqlite3.connect(index.path)) as connection:
connection.execute("UPDATE metadata SET value = 'bad' WHERE key = 'source_hash'")
connection.commit()
index._write_attestation()
self.change_graph(root)
index.build()
receipt = json.loads(
generation_diff_path(project.descriptor).read_text(encoding="utf-8")
)
self.assertEqual("baseline", receipt["kind"])
self.assertEqual("predecessor_corrupt", receipt["reason"])
self.assertTrue(validate_generation_diff_receipt(receipt))
def test_sqlite_sidecars_refuse_precommit_publication(self) -> None:
for suffix in ("-wal", "-journal", "-shm"):
with self.subTest(suffix=suffix), tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
original_index = index.path.read_bytes()
original_receipt = generation_diff_path(project.descriptor).read_bytes()
Path(f"{index.path}{suffix}").write_bytes(b"unproven-sidecar")
self.change_graph(root)
with self.assertRaises(DocForgeError) as blocked:
index.build()
self.assertEqual("index_busy", blocked.exception.code)
self.assertEqual(original_index, index.path.read_bytes())
self.assertEqual(
original_receipt,
generation_diff_path(project.descriptor).read_bytes(),
)
def test_live_wal_state_cannot_bypass_main_index_identity(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
original_receipt = generation_diff_path(project.descriptor).read_bytes()
connection = sqlite3.connect(index.path)
try:
self.assertEqual(
"wal",
connection.execute("PRAGMA journal_mode=WAL").fetchone()[0],
)
connection.execute(
"UPDATE metadata SET value = ? WHERE key = 'source_hash'",
("f" * 64,),
)
connection.commit()
self.assertTrue(Path(f"{index.path}-wal").exists())
main_file_after_wal = index.path.read_bytes()
with self.assertRaises(DocForgeError) as blocked:
index.build()
self.assertEqual("index_busy", blocked.exception.code)
self.assertEqual(main_file_after_wal, index.path.read_bytes())
self.assertEqual(
original_receipt,
generation_diff_path(project.descriptor).read_bytes(),
)
finally:
connection.close()
def test_sidecar_or_source_change_during_diff_preparation_aborts_precommit(self) -> None:
for mutation in ("source", "sidecar"):
with self.subTest(mutation=mutation), tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
original_index = index.path.read_bytes()
original_receipt = generation_diff_path(project.descriptor).read_bytes()
self.change_graph(root)
from docforge import index as index_module
real_prepare = index_module.prepare_generation_diff
def mutate_after_prepare(
*args: object,
_prepare: Callable[..., object] = real_prepare,
_mutation: str = mutation,
_root: Path = root,
_index: ProjectIndex = index,
**kwargs: object,
) -> object:
draft = _prepare(*args, **kwargs)
if _mutation == "source":
source = _root / "docs" / "content" / "workflow.md"
source.write_text(
source.read_text(encoding="utf-8") + "\nConcurrent change.\n",
encoding="utf-8",
)
else:
Path(f"{_index.path}-wal").write_bytes(b"appeared")
return draft
with (
mock.patch(
"docforge.index.prepare_generation_diff",
side_effect=mutate_after_prepare,
),
self.assertRaises(DocForgeError) as blocked,
):
index.build()
self.assertEqual(
"source_changed" if mutation == "source" else "index_busy",
blocked.exception.code,
)
self.assertEqual(original_index, index.path.read_bytes())
self.assertEqual(
original_receipt,
generation_diff_path(project.descriptor).read_bytes(),
)
def test_cache_root_symlink_cannot_redirect_receipt_publication(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
receipt = json.loads(
generation_diff_path(project.descriptor).read_text(encoding="utf-8")
)
cache_root = project.descriptor.cache_root
preserved = cache_root.with_name("preserved-cache")
outside = root / "outside-cache"
outside.mkdir()
cache_root.rename(preserved)
cache_root.symlink_to(outside, target_is_directory=True)
try:
with self.assertRaises(DocForgeError) as blocked:
publish_generation_diff(project.descriptor, receipt)
self.assertEqual("path_escape", blocked.exception.code)
self.assertFalse((outside / "generation-diff.json").exists())
finally:
cache_root.unlink()
preserved.rename(cache_root)
def test_post_commit_identity_and_durability_failures_are_degraded(self) -> None:
cases = (
("_fsync_cache_directory", OSError("fsync failed"), "index_directory_sync"),
(
"_published_index_signature",
OSError("signature failed"),
"index_identity",
),
)
for method, failure, stage in cases:
with self.subTest(method=method), tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
self.change_graph(root)
with mock.patch.object(ProjectIndex, method, side_effect=failure):
result = index.build()
self.assertEqual("ok", result["status"])
self.assertEqual("degraded", result["publication"]["state"])
self.assertIn(
stage,
{error["stage"] for error in result["publication"]["errors"]},
)
with contextlib.closing(sqlite3.connect(index.path)) as connection:
metadata = dict(connection.execute("SELECT key, value FROM metadata"))
self.assertEqual(project.load().source_hash, metadata["source_hash"])
def test_post_commit_receipts_fail_independently(self) -> None:
cases = (
("attestation", "docforge.index.ProjectIndex._write_attestation"),
("source_generation", "docforge.project.Project.record_generation"),
("generation_diff", "docforge.index.publish_generation_diff"),
)
for failed_receipt, target in cases:
with self.subTest(receipt=failed_receipt), tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
self.change_graph(root)
with mock.patch(target, side_effect=OSError("failed independently")):
result = index.build()
receipts = result["publication"]["receipts"]
self.assertEqual("unavailable", receipts[failed_receipt]["state"])
for name in {"attestation", "source_generation", "generation_diff"} - {
failed_receipt
}:
self.assertEqual("published", receipts[name]["state"])
def test_read_is_bounded_read_only_and_legacy_projects_do_not_load(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
receipt_path = generation_diff_path(project.descriptor)
original_receipt = json.loads(receipt_path.read_text(encoding="utf-8"))
before = {
path: path.stat().st_mtime_ns
for path in project.descriptor.cache_root.iterdir()
if path.is_file()
}
with (
request("test", enabled=True) as collector,
mock.patch.object(project, "load", side_effect=AssertionError("loaded")),
mock.patch.object(index, "check", side_effect=AssertionError("checked")),
mock.patch.object(index, "build", side_effect=AssertionError("built")),
mock.patch.object(index, "synchronize", side_effect=AssertionError("synced")),
):
result = index.generation_diff()
self.assertEqual("current", result["receipt_state"])
self.assertIsNotNone(collector)
diagnostics = collector.as_dict(outcome="ok")
for counter in (
"project_loads",
"source_files_parsed",
"adapter_projection_loads",
"adapter_source_extractions",
"index_checks",
"index_synchronizations",
"index_builds",
):
self.assertEqual(0, diagnostics["counters"][counter])
self.assertGreaterEqual(diagnostics["counters"]["source_generation_checks"], 2)
self.assertEqual(
before,
{
path: path.stat().st_mtime_ns
for path in project.descriptor.cache_root.iterdir()
if path.is_file()
},
)
with mock.patch.object(
generation_diff_module,
"validate_generation_diff_receipt",
wraps=generation_diff_module.validate_generation_diff_receipt,
) as validated:
self.assertEqual("current", index.generation_diff()["receipt_state"])
self.assertEqual(1, validated.call_count)
receipt_path.write_text("{broken", encoding="utf-8")
corrupt_before = receipt_path.read_bytes()
corrupt = index.generation_diff()
self.assertEqual("unverified", corrupt["receipt_state"])
self.assertEqual("corrupt_receipt", corrupt["receipt_reason"])
self.assertEqual(corrupt_before, receipt_path.read_bytes())
foreign = {**original_receipt, "project_id": "foreign-project"}
foreign["receipt_hash"] = canonical_hash(
{key: value for key, value in foreign.items() if key != "receipt_hash"}
)
receipt_path.write_text(
json.dumps(foreign, sort_keys=True, indent=2) + "\n",
encoding="utf-8",
)
foreign_result = index.generation_diff()
self.assertEqual("unverified", foreign_result["receipt_state"])
self.assertEqual("foreign_receipt", foreign_result["receipt_reason"])
outside = root / "foreign-generation-diff.json"
outside.write_text(
json.dumps(original_receipt, sort_keys=True, indent=2) + "\n",
encoding="utf-8",
)
receipt_path.unlink()
receipt_path.symlink_to(outside)
unsafe = index.generation_diff()
self.assertEqual("unsafe", unsafe["receipt_state"])
self.assertEqual("unsafe_receipt", unsafe["receipt_reason"])
snapshot = project.load()
legacy = StaticProject(snapshot)
calls = legacy.load_calls
unknown = ProjectIndex(legacy).generation_diff()
self.assertEqual("unknown", unknown["receipt_state"])
self.assertEqual(calls, legacy.load_calls)
def test_cli_generation_diff_is_additive_and_paged(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
self.change_graph(root)
index.build()
output = io.StringIO()
with contextlib.redirect_stdout(output):
code = main(
[
"--project-root",
str(root),
"generation-diff",
"--limit",
"1",
]
)
result = json.loads(output.getvalue())
self.assertEqual(0, code)
Draft202012Validator(RESULT_SCHEMA).validate(result)
Draft202012Validator(GENERATION_DIFF_PAGE_SCHEMA).validate(
{
"generation_diff": result["generation_diff"],
"pagination": result["pagination"],
}
)
self.assertEqual("generation-diff.items", result["pagination"]["kind"])
self.assertEqual(1, result["pagination"]["returned_count"])
self.assertTrue(result["pagination"]["has_more"])
self.assertNotIn("next_cursor", result)
self.assertNotIn("pagination", result["generation_diff"])
self.assertEqual(
{
"page_schema_version",
"receipt_header",
"items",
"omissions",
"page_hash",
},
set(result["generation_diff"]),
)
page = result["generation_diff"]
self.assertIn("stored_receipt_hash", page["receipt_header"])
self.assertNotIn("receipt_hash", page["receipt_header"])
self.assertEqual(
page["page_hash"],
canonical_hash(
{
"page_schema_version": 1,
"receipt_state": result["receipt_state"],
"receipt_header": page["receipt_header"],
"pagination": result["pagination"],
"items": page["items"],
"omissions": page["omissions"],
}
),
)
self.assertEqual(
result["pagination"]["returned_count"],
len(page["items"]) + len(page["omissions"]),
)
self.assertEqual(
result["pagination"]["total_count"],
page["receipt_header"]["retained_item_count"],
)
malformed_page = json.loads(json.dumps(result))
stored_receipt = json.loads(
generation_diff_path(project.descriptor).read_text(encoding="utf-8")
)
node = next(item for item in stored_receipt["items"] if item["entity"] == "node")
node["change"] = "added"
malformed_page["generation_diff"]["items"] = [node]
self.assertFalse(
Draft202012Validator(GENERATION_DIFF_PAGE_SCHEMA).is_valid(
{
"generation_diff": malformed_page["generation_diff"],
"pagination": malformed_page["pagination"],
}
)
)
malformed_header = json.loads(json.dumps(result))
header = malformed_header["generation_diff"]["receipt_header"]
header["kind"] = "baseline"
header["reason"] = "no_meaningful_transition"
header["from_generation"] = None
self.assertFalse(
Draft202012Validator(GENERATION_DIFF_PAGE_SCHEMA).is_valid(
{
"generation_diff": malformed_header["generation_diff"],
"pagination": malformed_header["pagination"],
}
)
)
contradictory_pagination = json.loads(json.dumps(result))
contradictory_pagination["pagination"]["has_more"] = False
self.assertFalse(
Draft202012Validator(GENERATION_DIFF_PAGE_SCHEMA).is_valid(
{
"generation_diff": contradictory_pagination["generation_diff"],
"pagination": contradictory_pagination["pagination"],
}
)
)
missing_cursor = json.loads(json.dumps(result))
missing_cursor["pagination"]["next_cursor"] = None
self.assertFalse(
Draft202012Validator(GENERATION_DIFF_PAGE_SCHEMA).is_valid(
{
"generation_diff": missing_cursor["generation_diff"],
"pagination": missing_cursor["pagination"],
}
)
)
class GenerationDiffMcpTests(unittest.IsolatedAsyncioTestCase):
def copy_fixture(self, destination: Path) -> Path:
root = destination / "alpha"
shutil.copytree(FIXTURES / "alpha", root)
shutil.rmtree(root / ".docforge" / "cache", ignore_errors=True)
return root
async def test_mcp_surface_paginates_and_cursors_bind_the_receipt(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
GenerationDiffTests.change_graph(root)
index.build()
async with create_connected_server_and_client_session(
create_server(root),
raise_exceptions=True,
) as session:
tools = {tool.name: tool for tool in (await session.list_tools()).tools}
schema = tools["docforge_get_generation_diff"].inputSchema
self.assertEqual({"limit", "cursor"}, set(schema["properties"]))
self.assertEqual([], schema.get("required", []))
first = await session.call_tool(
"docforge_get_generation_diff",
{"limit": 1},
)
first_result = first.structuredContent
Draft202012Validator(RESULT_SCHEMA).validate(first_result)
Draft202012Validator(GENERATION_DIFF_PAGE_SCHEMA).validate(
{
"generation_diff": first_result["generation_diff"],
"pagination": first_result["pagination"],
}
)
self.assertEqual("current", first_result["receipt_state"])
self.assertEqual(1, first_result["pagination"]["returned_count"])
cursor = first_result["pagination"]["next_cursor"]
self.assertIsInstance(cursor, str)
self.assertLess(len(cursor), 1_000)
self.assertNotIn("next_cursor", first_result)
second = await session.call_tool(
"docforge_get_generation_diff",
{"limit": 2, "cursor": cursor},
)
self.assertEqual(2, second.structuredContent["pagination"]["returned_count"])
foundation = root / "docs" / "content" / "foundation.md"
foundation.write_text(
foundation.read_text(encoding="utf-8") + "\nAnother transition.\n",
encoding="utf-8",
)
ProjectIndex(Project.open(root)).build()
stale = await session.call_tool(
"docforge_get_generation_diff",
{"limit": 1, "cursor": cursor},
)
self.assertEqual("stale_cursor", stale.structuredContent["error"]["code"])
Draft202012Validator(RESULT_SCHEMA).validate(stale.structuredContent)
def test_service_diagnostics_drop_before_primary_page(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
descriptor_path = root / ".docforge" / "project.toml"
descriptor_path.write_text(
descriptor_path.read_text(encoding="utf-8").replace(
"max_results = 20",
"max_results = 20\nmax_tool_output_chars = 2100",
),
encoding="utf-8",
)
project = Project.open(root)
ProjectIndex(project).build()
service = DocForgeService(
project,
diagnostics=True,
capability_mode_name="read",
)
result = service.generation_diff()
self.assertEqual("ok", result["status"])
self.assertNotIn("diagnostics", result)
self.assertLessEqual(service._encoded_length(result), 2_100)
def test_page_sizing_uses_logarithmic_response_encodes(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
descriptor_path = root / ".docforge" / "project.toml"
descriptor_path.write_text(
descriptor_path.read_text(encoding="utf-8").replace(
"max_results = 20",
"max_results = 1000",
),
encoding="utf-8",
)
project = Project.open(root)
index = ProjectIndex(project)
index.build()
stored = index.generation_diff()
receipt = dict(stored["generation_diff"])
receipt["items"] = [
{
"item_hash": canonical_hash({"ordinal": ordinal}),
"payload": "x" * 500,
}
for ordinal in range(1_000)
]
service = DocForgeService(
project,
capability_mode_name="read",
)
with (
mock.patch.object(
service.index,
"generation_diff",
return_value={**stored, "generation_diff": receipt},
),
mock.patch.object(
service,
"_encoded_length",
wraps=service._encoded_length,
) as encoded,
):
result = service.generation_diff(limit=1_000)
self.assertEqual("ok", result["status"])
self.assertGreater(result["pagination"]["returned_count"], 0)
self.assertLess(result["pagination"]["returned_count"], 1_000)
self.assertLessEqual(encoded.call_count, 15)

486
tests/test_mcp_server.py
View file

@ -12,26 +12,33 @@ from contextlib import contextmanager
from pathlib import Path
from unittest import mock
from jsonschema import Draft202012Validator
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from mcp.shared.memory import create_connected_server_and_client_session
from docforge.changesets import ChangesetStore
from docforge.errors import DocForgeError
from docforge.index import ProjectIndex
from docforge.mcp_server import (
ALL_TOOLS,
APPLICATION_TOOLS,
CONTENT_WARNING,
PROPOSAL_TOOLS,
READ_TOOLS,
SERVER_VERSION,
DocForgeService,
_create_bound_server,
create_server,
)
from docforge.project import Project
from docforge.project import Project, project_root_fingerprint
from docforge.viewer_manager import ViewerManager
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
CAPSULE_SCHEMA = json.loads(
(ROOT / "schemas" / "context-capsule.schema.json").read_text(encoding="utf-8")
)
class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
@ -83,6 +90,8 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertNotIn("limit", tools[name].inputSchema.get("required", []))
for name in (
"docforge_get_context",
"docforge_get_task_context",
"docforge_get_generation_diff",
"docforge_list_changesets",
"docforge_get_changeset",
"docforge_validate_changeset",
@ -91,6 +100,22 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
for field in ("limit", "cursor"):
self.assertIn(field, tools[name].inputSchema["properties"])
self.assertNotIn(field, tools[name].inputSchema.get("required", []))
self.assertEqual(
{"task_kind", "task"},
set(tools["docforge_get_task_context"].inputSchema["required"]),
)
self.assertEqual(
[
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release",
],
tools["docforge_get_task_context"].inputSchema["properties"]["task_kind"]["enum"],
)
self.assertIn(
"deep",
tools["docforge_render_status"].inputSchema["properties"],
@ -103,6 +128,43 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
)
)
async def test_explicit_capability_modes_preserve_surfaces_and_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
ProjectIndex(Project.open(root)).build()
async with create_connected_server_and_client_session(
create_server(root, capability_mode="read"),
raise_exceptions=True,
) as session:
read_names = tuple(tool.name for tool in (await session.list_tools()).tools)
read_bootstrap = await session.call_tool("docforge_bootstrap", {})
self.assertEqual(READ_TOOLS, read_names)
self.assertEqual(
"read",
read_bootstrap.structuredContent["effective_policy"]["capability_mode"],
)
self.assertNotIn(
"docforge_register_changes",
read_bootstrap.structuredContent["recommended_workflow"],
)
async with create_connected_server_and_client_session(
create_server(
root,
"alpha-editor",
canonical_applier_id="alpha-editor",
capability_mode="application",
),
raise_exceptions=True,
) as session:
application_names = tuple(tool.name for tool in (await session.list_tools()).tools)
self.assertEqual((*ALL_TOOLS, *APPLICATION_TOOLS), application_names)
with self.assertRaises(DocForgeError) as unavailable:
create_server(root, capability_mode="application")
self.assertEqual("capability_unavailable", unavailable.exception.code)
async def test_factory_diagnostics_are_additive_through_real_mcp(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
@ -121,6 +183,403 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertEqual(0, diagnostics["counters"]["project_loads"])
self.assertEqual(0, diagnostics["counters"]["source_files_parsed"])
async def test_task_context_is_hash_stable_paged_and_generation_bound(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
ProjectIndex(Project.open(root)).build()
async with create_connected_server_and_client_session(
create_server(root, capability_mode="read", diagnostics=True),
raise_exceptions=True,
) as session:
first = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"limit": 1,
},
)
first_capsule = first.structuredContent["capsule"]
Draft202012Validator(CAPSULE_SCHEMA).validate(first_capsule)
cursor = first_capsule["pagination"]["next_cursor"]
hashes = {
first_capsule["capsule_hash"],
first_capsule["collection_hash"],
first_capsule["plan"]["plan_hash"],
}
evidence_ids = [item["node_id"] for item in first_capsule["evidence"]]
while cursor is not None:
page = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"limit": 2,
"cursor": cursor,
},
)
capsule = page.structuredContent["capsule"]
self.assertEqual(first_capsule["capsule_hash"], capsule["capsule_hash"])
self.assertEqual(
first_capsule["collection_hash"],
capsule["collection_hash"],
)
self.assertEqual(
first_capsule["plan"]["plan_hash"],
capsule["plan"]["plan_hash"],
)
evidence_ids.extend(item["node_id"] for item in capsule["evidence"])
cursor = capsule["pagination"]["next_cursor"]
self.assertEqual(
["guide.workflow", "guide.foundation", "proof.validation"],
evidence_ids,
)
self.assertEqual(3, len(hashes))
self.assertEqual(
"mcp.task_context",
first.structuredContent["diagnostics"]["operation"],
)
counters = first.structuredContent["diagnostics"]["counters"]
self.assertEqual(0, counters["project_loads"])
self.assertEqual(0, counters["source_files_parsed"])
self.assertEqual(0, counters["adapter_projection_loads"])
self.assertEqual(0, counters["adapter_source_extractions"])
self.assertEqual(0, counters["index_builds"])
self.assertLessEqual(
len(json.dumps(first.structuredContent, separators=(",", ":"))),
Project.open(root).descriptor.limits.max_tool_output_chars,
)
for changed_arguments in (
{"task": "A different task"},
{"task_kind": "failure"},
{"focus_node_id": "guide.foundation"},
{"budget": 100},
):
arguments = {
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"limit": 1,
"cursor": first_capsule["pagination"]["next_cursor"],
**changed_arguments,
}
changed_cursor = await session.call_tool(
"docforge_get_task_context",
arguments,
)
self.assertEqual(
"stale_cursor",
changed_cursor.structuredContent["error"]["code"],
)
different_policy = DocForgeService(
Project.open(root),
capability_mode_name="proposal",
).task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
limit=1,
cursor=first_capsule["pagination"]["next_cursor"],
)
self.assertEqual("stale_cursor", different_policy["error"]["code"])
changed = root / "docs/content/foundation.md"
changed.write_text(
changed.read_text(encoding="utf-8") + "\nNew generation.\n",
encoding="utf-8",
)
stale = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"limit": 1,
"cursor": first_capsule["pagination"]["next_cursor"],
},
)
self.assertEqual("error", stale.structuredContent["status"])
self.assertEqual("stale_cursor", stale.structuredContent["error"]["code"])
async def test_custom_context_policy_does_not_silently_gain_task_planning(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
ProjectIndex(project).build()
service = DocForgeService(
project,
context_provider=lambda index, profile, budget: {
"profile": profile,
"budget": budget,
"entries": [],
"omissions": [],
},
capability_mode_name="read",
)
async with create_connected_server_and_client_session(
_create_bound_server(service, read_only=True),
raise_exceptions=True,
) as session:
bootstrap = await session.call_tool("docforge_bootstrap", {})
with (
mock.patch.object(
service.project,
"load",
side_effect=AssertionError("capability errors must not load"),
),
mock.patch.object(
service.index,
"check",
side_effect=AssertionError("capability errors must not check"),
),
mock.patch.object(
service.index,
"build",
side_effect=AssertionError("capability errors must not build"),
),
mock.patch.object(
service.index,
"synchronize",
side_effect=AssertionError("capability errors must not synchronize"),
),
):
result = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Do not widen the adapter context policy",
},
)
self.assertEqual(
"docforge_get_context",
bootstrap.structuredContent["session_contract"]["recommended_first_operation"][
"tool"
],
)
self.assertFalse(bootstrap.structuredContent["capabilities"]["task_context"]["enabled"])
self.assertEqual("error", result.structuredContent["status"])
self.assertEqual(
"task_context_unavailable",
result.structuredContent["error"]["code"],
)
def test_task_context_page_hash_binds_final_page_envelope(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8").replace(
"max_context_tokens = 2000",
"max_context_tokens = 2000\nmax_tool_output_chars = 8000",
),
encoding="utf-8",
)
project = Project.open(root)
ProjectIndex(project).build()
service = DocForgeService(project, capability_mode_name="read")
one = service.task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
limit=1,
)
two = service.task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
limit=2,
)
one_capsule = one["capsule"]
two_capsule = two["capsule"]
self.assertEqual(
["guide.workflow"],
[item["node_id"] for item in one_capsule["evidence"]],
)
self.assertEqual(
["guide.workflow"],
[item["node_id"] for item in two_capsule["evidence"]],
)
self.assertEqual("complete", one_capsule["page_state"])
self.assertEqual("incomplete", two_capsule["page_state"])
self.assertNotEqual(one_capsule["page_hash"], two_capsule["page_hash"])
self.assertNotEqual(one_capsule["pagination"], two_capsule["pagination"])
self.assertLessEqual(
len(json.dumps(one, sort_keys=True, separators=(",", ":"))),
8_000,
)
self.assertLessEqual(
len(json.dumps(two, sort_keys=True, separators=(",", ":"))),
8_000,
)
def test_dense_task_context_page_packing_is_logarithmic(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8")
.replace("max_results = 20", "max_results = 1000")
.replace(
"max_context_tokens = 2000",
"max_context_tokens = 2000\nmax_tool_output_chars = 20000",
),
encoding="utf-8",
)
project = Project.open(root)
service = DocForgeService(project, capability_mode_name="read")
capsule = {
"schema_version": 1,
"plan": {
"effective_policy_hash": "1" * 64,
"request_hash": "2" * 64,
"plan_hash": "3" * 64,
},
"generation": {"index_schema_version": 3},
"evidence": [
{
"node_id": f"node.{index:04d}",
"content": "bounded evidence " * 40,
}
for index in range(1_000)
],
"gaps": [],
"omissions": [],
"collection_hash": "4" * 64,
"capsule_hash": "5" * 64,
"state": "complete",
"summary": {"evidence_count": 1_000},
}
result = {
"status": "ok",
"project_id": project.descriptor.project_id,
"project_root_fingerprint": project_root_fingerprint(project.descriptor.root),
"adapter": project.descriptor.adapter,
"revision": "test-revision",
"source_hash": "6" * 64,
"capsule": capsule,
}
with mock.patch.object(
service,
"_encoded_length",
wraps=service._encoded_length,
) as encoded_length:
page = service._page_task_context_result(
result,
selected_limit=1_000,
cursor=None,
)
pagination = page["pagination"]
self.assertGreater(pagination["returned_count"], 0)
self.assertLess(pagination["returned_count"], 1_000)
returned_count = pagination["returned_count"]
self.assertEqual(
capsule["evidence"][:returned_count],
page["capsule"]["evidence"],
)
self.assertEqual([], page["capsule"]["omissions"])
self.assertEqual(
pagination["next_cursor"],
page["capsule"]["pagination"]["next_cursor"],
)
self.assertEqual(
returned_count,
page["capsule"]["summary"]["page_item_count"],
)
self.assertLessEqual(encoded_length.call_count, 11)
decorated = {
**page,
"server_version": SERVER_VERSION,
"content_warning": CONTENT_WARNING,
"staleness": "current",
}
self.assertLessEqual(
len(json.dumps(decorated, sort_keys=True, separators=(",", ":"))),
20_000,
)
def test_task_context_default_page_clamps_to_small_project_limit(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8").replace(
"max_results = 20",
"max_results = 2",
),
encoding="utf-8",
)
project = Project.open(root)
ProjectIndex(project).build()
result = DocForgeService(
project,
capability_mode_name="read",
).task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
)
self.assertEqual(2, result["pagination"]["limit"])
self.assertLessEqual(result["pagination"]["returned_count"], 2)
def test_oversized_task_evidence_advances_once_as_an_omission(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8").replace(
"max_context_tokens = 2000",
"max_context_tokens = 50000\nmax_tool_output_chars = 8000",
),
encoding="utf-8",
)
workflow = root / "docs" / "content" / "workflow.md"
workflow.write_text(
workflow.read_text(encoding="utf-8") + "\n" + ("large evidence " * 2_000),
encoding="utf-8",
)
project = Project.open(root)
ProjectIndex(project).build()
service = DocForgeService(project, capability_mode_name="read")
first = service.task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
budget=50_000,
limit=1,
)
first_capsule = first["capsule"]
self.assertEqual([], first_capsule["evidence"])
self.assertEqual("response_limit", first_capsule["omissions"][0]["code"])
self.assertEqual("guide.workflow", first_capsule["omissions"][0]["subject"])
self.assertEqual(1, first_capsule["pagination"]["returned_count"])
self.assertTrue(first_capsule["pagination"]["has_more"])
second = service.task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
budget=50_000,
limit=1,
cursor=first_capsule["pagination"]["next_cursor"],
)
self.assertNotEqual(first_capsule["page_hash"], second["capsule"]["page_hash"])
self.assertNotIn(
"guide.workflow",
[
item.get("node_id", item.get("subject"))
for item in (
*second["capsule"]["evidence"],
*second["capsule"]["omissions"],
)
],
)
async def test_context_pagination_is_complete_and_stale_cursors_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
@ -265,6 +724,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
logic = await session.call_tool(
"docforge_get_logic", {"owner_node_id": "guide.workflow"}
)
task_context = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow without AST analysis",
"focus_node_id": "guide.workflow",
},
)
policy = bootstrap.structuredContent["adapter_policy"]
self.assertEqual("preserve-no-ast", policy["mode"])
@ -290,6 +757,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
"adapter_policy_forbids_logic",
logic.structuredContent["error"]["code"],
)
self.assertEqual("ok", task_context.structuredContent["status"])
self.assertNotIn(
"logic",
{
step["operation"]
for step in task_context.structuredContent["capsule"]["plan"]["steps"]
},
)
async def test_every_read_tool_returns_scoped_structured_results(self) -> None:
with tempfile.TemporaryDirectory() as directory:
@ -319,6 +794,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
("docforge_visualization_status", {}),
("docforge_bootstrap", {}),
("docforge_sync", {}),
(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
},
),
)
with self.running_manager(Path(directory) / "viewer-manager.json"):
service = DocForgeService(Project.open(root))
@ -374,6 +857,7 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
context = results[9].structuredContent
self.assertLessEqual(context["estimated_tokens"], 180)
self.assertTrue(context["omissions"])
self.assertEqual("complete", results[17].structuredContent["capsule"]["state"])
async def test_invalid_traversal_limit_is_a_structured_domain_error(self) -> None:
with tempfile.TemporaryDirectory() as directory:

14
tests/test_observability.py
View file

@ -34,6 +34,10 @@ from docforge.viewer_manager import ViewerManagerClient
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
RESULT_SCHEMA = json.loads((ROOT / "schemas" / "result.schema.json").read_text())
CLIENT_CONFIGURATION_SCHEMA = json.loads(
(ROOT / "schemas" / "client-configuration.schema.json").read_text()
)
DOCTOR_RESULT_SCHEMA = json.loads((ROOT / "schemas" / "doctor-result.schema.json").read_text())
ZERO_WORK_COUNTERS = (
"project_loads",
"source_files_parsed",
@ -134,6 +138,16 @@ class TelemetryContractTests(unittest.TestCase):
set(COUNTER_NAMES),
set(properties["counters"]["properties"]),
)
for schema in (CLIENT_CONFIGURATION_SCHEMA, DOCTOR_RESULT_SCHEMA):
dedicated = schema["$defs"]["diagnostics"]["properties"]
self.assertEqual(
set(STAGE_NAMES),
set(dedicated["stages"]["propertyNames"]["enum"]),
)
self.assertEqual(
set(COUNTER_NAMES),
set(dedicated["counters"]["required"]),
)
def test_thread_and_async_request_contexts_are_isolated(self) -> None:
barrier = threading.Barrier(2)

161
tests/test_policy.py Normal file
View file

@ -0,0 +1,161 @@
from __future__ import annotations
import json
import shutil
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from jsonschema import Draft202012Validator
from docforge.application import GenericCanonicalApplier
from docforge.errors import DocForgeError
from docforge.index import ProjectIndex
from docforge.mcp_server import (
ALL_TOOLS,
APPLICATION_TOOLS,
READ_TOOLS,
DocForgeService,
)
from docforge.policy import (
POLICY_PRECEDENCE,
capability_mode,
compose_effective_policy,
)
from docforge.project import Project
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
POLICY_SCHEMA = json.loads((ROOT / "schemas" / "policy.schema.json").read_text(encoding="utf-8"))
class EffectivePolicyTests(unittest.TestCase):
def copy_fixture(self, destination: Path) -> Path:
root = destination / "alpha"
shutil.copytree(FIXTURES / "alpha", root)
return root
def test_policy_schema_and_legacy_adapter_projection_are_exact(self) -> None:
standard = compose_effective_policy(
selected_mode="proposal",
capability_source="factory_default",
no_ast=False,
diagnostics=False,
render_configured=True,
application_enabled=False,
)
preserve = compose_effective_policy(
selected_mode="read",
capability_source="explicit",
no_ast=True,
diagnostics=True,
render_configured=False,
application_enabled=False,
)
validator = Draft202012Validator(POLICY_SCHEMA)
validator.validate(standard.as_dict())
validator.validate(preserve.as_dict())
self.assertEqual(list(POLICY_PRECEDENCE), preserve.as_dict()["precedence"])
self.assertEqual(
{
"mode": "standard",
"ast_analysis": "allowed",
"logic_projection": "allowed",
"incremental_extraction": "allowed",
"adapter_rewrite": "not_requested",
},
standard.adapter_policy(),
)
self.assertEqual("preserve-no-ast", preserve.adapter_policy()["mode"])
self.assertEqual(["docforge_get_logic"], preserve.adapter_policy()["blocked_tools"])
self.assertEqual("off", preserve.as_dict()["logic_indexing"])
self.assertEqual("enabled", preserve.as_dict()["profiling"])
def test_invalid_or_unavailable_capability_fails_closed(self) -> None:
with self.assertRaises(DocForgeError) as invalid:
capability_mode("admin", default="read")
self.assertEqual("invalid_capability_mode", invalid.exception.code)
with self.assertRaises(DocForgeError) as unavailable:
compose_effective_policy(
selected_mode="application",
capability_source="explicit",
no_ast=False,
diagnostics=False,
render_configured=True,
application_enabled=False,
)
self.assertEqual("capability_unavailable", unavailable.exception.code)
def test_modes_preserve_default_surfaces_and_narrow_authority(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
read = DocForgeService(
project,
proposal_writer="alpha-editor",
capability_mode_name="read",
)
proposal = DocForgeService(
project,
proposal_writer="alpha-editor",
capability_mode_name="proposal",
)
application = DocForgeService(
project,
proposal_writer="alpha-editor",
canonical_applier_id="alpha-editor",
canonical_applier=GenericCanonicalApplier(project),
capability_mode_name="application",
)
operator = DocForgeService(
project,
proposal_writer="alpha-editor",
capability_mode_name="operator",
)
self.assertEqual(READ_TOOLS, read.tool_surface)
self.assertFalse(read.changesets.access()["enabled"])
self.assertEqual(ALL_TOOLS, proposal.tool_surface)
self.assertTrue(proposal.changesets.access()["enabled"])
self.assertFalse(proposal.application.enabled)
self.assertEqual((*ALL_TOOLS, *APPLICATION_TOOLS), application.tool_surface)
self.assertTrue(application.application.enabled)
self.assertEqual(ALL_TOOLS, operator.tool_surface)
self.assertTrue(operator.capabilities()["operator"]["enabled"])
def test_bootstrap_reuses_synchronized_identity_and_filters_workflow(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
ProjectIndex(project).build()
read = DocForgeService(project, capability_mode_name="read")
with mock.patch.object(
project,
"load",
side_effect=AssertionError("bootstrap must not reload after synchronization"),
):
result = read.bootstrap()
self.assertEqual("ok", result["status"])
self.assertEqual(1, result["session_contract"]["schema_version"])
self.assertEqual("read", result["effective_policy"]["capability_mode"])
self.assertEqual(
result["effective_policy"], result["session_contract"]["effective_policy"]
)
self.assertNotIn("docforge_register_changes", result["recommended_workflow"])
self.assertNotIn("docforge_apply_changeset", result["recommended_workflow"])
self.assertEqual(
"docforge_get_task_context",
result["session_contract"]["recommended_first_operation"]["tool"],
)
proposal = DocForgeService(
project,
proposal_writer="alpha-editor",
capability_mode_name="proposal",
).bootstrap()
self.assertIn("docforge_register_changes", proposal["recommended_workflow"])
self.assertIn("docforge_get_changeset_diff", proposal["recommended_workflow"])
self.assertNotIn("docforge_apply_changeset", proposal["recommended_workflow"])

27
tests/test_public_contract.py
View file

@ -61,6 +61,8 @@ PUBLIC_IMPORTS = {
"CanonicalApplicationService",
"GenericCanonicalApplier",
),
"docforge.client_config": ("generate_client_configuration",),
"docforge.doctor": ("run_doctor",),
"docforge.index": ("ProjectIndex",),
"docforge.mcp_server": (
"create_project_server",
@ -77,6 +79,18 @@ PUBLIC_IMPORTS = {
"ProjectService",
"ProjectSnapshot",
),
"docforge.policy": (
"EffectivePolicyV1",
"capability_mode",
"compose_effective_policy",
),
"docforge.retrieval": (
"ContextCapsuleV1",
"RetrievalPlanV1",
"build_retrieval_plan",
"relation_category",
"validate_retrieval_plan",
),
"docforge.render_contract": (
"GenericHtmlRenderer",
"PreparedRender",
@ -96,9 +110,12 @@ EXPECTED_CLI_COMMANDS = {
"backlinks",
"build",
"check",
"configure",
"context",
"dependencies",
"doctor",
"filter",
"generation-diff",
"impact",
"info",
"onboard",
@ -127,6 +144,8 @@ EXPECTED_MCP_TOOLS = {
"docforge_get_changeset",
"docforge_get_changeset_diff",
"docforge_get_context",
"docforge_get_generation_diff",
"docforge_get_task_context",
"docforge_get_contract",
"docforge_get_logic",
"docforge_get_node",
@ -250,6 +269,14 @@ class PublicContractTests(unittest.TestCase):
result_validator = Draft202012Validator(self.schema("result.schema.json"))
result_validator.validate(success)
result_validator.validate(error)
generation_diff = service.generation_diff(limit=1)
result_validator.validate(generation_diff)
Draft202012Validator(self.schema("generation-diff-page.schema.json")).validate(
{
"generation_diff": generation_diff["generation_diff"],
"pagination": generation_diff["pagination"],
}
)
def test_changeset_hash_is_exact_canonical_json_sha256(self) -> None:
document = {

464
tests/test_retrieval.py Normal file
View file

@ -0,0 +1,464 @@
from __future__ import annotations
import json
import shutil
import tempfile
import unittest
from dataclasses import replace
from pathlib import Path
from unittest import mock
from jsonschema import Draft202012Validator
from docforge.errors import DocForgeError
from docforge.index import ProjectIndex
from docforge.models import ProjectState
from docforge.policy import compose_effective_policy
from docforge.project import Project
from docforge.retrieval import (
BASE_RELATION_CATEGORIES,
MAX_TASK_CANDIDATE_EDGES,
MAX_TASK_EVIDENCE,
RELATION_CATEGORIES,
TASK_KINDS,
TASK_REQUIREMENTS,
build_retrieval_plan,
relation_category,
)
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
CAPSULE_SCHEMA = json.loads(
(ROOT / "schemas" / "context-capsule.schema.json").read_text(encoding="utf-8")
)
class TaskRetrievalTests(unittest.TestCase):
def copy_fixture(self, name: str, destination: Path) -> Path:
root = destination / name
shutil.copytree(FIXTURES / name, root)
return root
@staticmethod
def effective_policy() -> dict[str, object]:
return compose_effective_policy(
selected_mode="read",
capability_source="explicit",
no_ast=False,
diagnostics=False,
render_configured=True,
application_enabled=False,
).as_dict()
def plan(
self,
project: Project,
*,
task_kind: str = "change",
task: str = "Change the editing workflow",
focus_node_id: str | None = "guide.workflow",
budget: int | None = None,
limit: int | None = None,
):
return build_retrieval_plan(
project.descriptor,
task_kind=task_kind,
task=task,
focus_node_id=focus_node_id,
budget=budget,
limit=limit,
effective_policy=self.effective_policy(),
)
def test_every_task_plan_is_closed_deterministic_and_hash_bound(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
hashes = {}
relation_hashes = set()
for task_kind in TASK_KINDS:
first = self.plan(project, task_kind=task_kind)
second = self.plan(project, task_kind=task_kind)
self.assertEqual(first, second)
self.assertEqual(TASK_REQUIREMENTS[task_kind], first.category_order[:1])
self.assertEqual(
{
*BASE_RELATION_CATEGORIES,
"unclassified",
},
set(first.category_order),
)
self.assertEqual(64, len(first.plan_hash))
relation_hashes.update(
step.relation_set_hash
for step in first.steps
if step.relation_scope == "project_allowed"
)
hashes[task_kind] = first.plan_hash
self.assertEqual(len(TASK_KINDS), len(set(hashes.values())))
self.assertEqual(1, len(relation_hashes))
self.assertEqual(
{
"change": ("dependency",),
"implementation": ("implementation",),
"failure": ("execution",),
"ownership": ("structure",),
"test": ("evidence",),
"operation": ("execution",),
"release": ("evidence",),
},
TASK_REQUIREMENTS,
)
mapped = [
relation for relations in RELATION_CATEGORIES.values() for relation in relations
]
self.assertEqual(len(mapped), len(set(mapped)))
self.assertEqual(
{
"structure": ("contains", "defined_in", "defines", "owns"),
"implementation": (
"implemented_by",
"implements",
"inherits",
"inherits_from",
),
"dependency": ("depends_on", "imports"),
"execution": (
"activates",
"calls",
"dispatches_to",
"launches",
),
"data": ("reads", "writes"),
"evidence": (
"documents",
"governs",
"proves",
"tested_by",
"verifies",
),
"context": ("relates_to",),
"unclassified": (),
},
RELATION_CATEGORIES,
)
self.assertEqual("dependency", relation_category("depends_on"))
self.assertEqual("evidence", relation_category("proves"))
self.assertEqual("unclassified", relation_category("owns_database"))
self.assertEqual("unclassified", relation_category("when_true"))
def test_exact_capsule_is_schema_valid_stable_and_explainable(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
index = ProjectIndex(project)
index.build()
plan = self.plan(project)
first = index.task_context(plan)
second = index.task_context(plan)
self.assertEqual(first, second)
capsule = first["capsule"]
Draft202012Validator(CAPSULE_SCHEMA).validate(capsule)
self.assertEqual("complete", capsule["state"])
self.assertEqual("resolved", capsule["focus"]["state"])
self.assertEqual("guide.workflow", capsule["focus"]["node_id"])
self.assertEqual(
["guide.workflow", "guide.foundation", "proof.validation"],
[item["node_id"] for item in capsule["evidence"]],
)
dependency = capsule["evidence"][1]["relationship_path"][0]
self.assertEqual("depends_on", dependency["relation"])
self.assertEqual("dependency", dependency["category"])
self.assertEqual("outgoing", dependency["direction"])
workflow_reasons = capsule["evidence"][0]["relationship_reasons"]
self.assertIn(
("depends_on", "outgoing"),
{
(relationship["relation"], relationship["direction"])
for relationship in workflow_reasons
},
)
self.assertIn(
("proves", "incoming"),
{
(relationship["relation"], relationship["direction"])
for relationship in workflow_reasons
},
)
foundation_reason = capsule["evidence"][1]["relationship_reasons"][0]
self.assertEqual("depends_on", foundation_reason["relation"])
self.assertEqual("incoming", foundation_reason["direction"])
self.assertNotIn(
"no_selected_evidence",
[gap["code"] for gap in capsule["gaps"]],
)
self.assertEqual(64, len(capsule["collection_hash"]))
self.assertEqual(64, len(capsule["capsule_hash"]))
def test_gaps_distinguish_undeclared_complete_and_incomplete_checks(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
undeclared = index.task_context(self.plan(project, task_kind="implementation"))[
"capsule"
]
self.assertIn(
"category_not_declared",
[gap["code"] for gap in undeclared["gaps"]],
)
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8").replace(
'"returns_to"]',
'"returns_to", "calls"]',
),
encoding="utf-8",
)
declared_project = Project.open(root)
declared_index = ProjectIndex(declared_project)
declared_index.build()
missing = declared_index.task_context(
self.plan(
declared_project,
task_kind="failure",
focus_node_id="guide.foundation",
)
)["capsule"]
self.assertIn(
"no_selected_evidence",
[gap["code"] for gap in missing["gaps"]],
)
incomplete = declared_index.task_context(self.plan(declared_project, budget=1))[
"capsule"
]
self.assertEqual("incomplete", incomplete["state"])
self.assertTrue(incomplete["omissions"])
self.assertIn(
"evidence_incomplete",
[gap["code"] for gap in incomplete["gaps"]],
)
self.assertNotIn(
"no_selected_evidence",
[gap["code"] for gap in incomplete["gaps"]],
)
def test_unclassified_relation_is_preserved_without_guessed_semantics(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("awesome-ski-game", Path(directory)))
index = ProjectIndex(project)
index.build()
plan = build_retrieval_plan(
project.descriptor,
task_kind="change",
task="Change the first descent session",
focus_node_id="session.first-descent",
budget=None,
limit=None,
effective_policy=self.effective_policy(),
)
capsule = index.task_context(plan)["capsule"]
self.assertIn("informs", capsule["summary"]["unclassified_relations"])
relation = next(
relationship
for item in capsule["evidence"]
for relationship in item["relationship_reasons"]
if relationship["relation"] == "informs"
)
self.assertEqual("unclassified", relation["category"])
self.assertIn(
"unclassified_relation",
[gap["code"] for gap in capsule["gaps"]],
)
def test_plan_is_compact_at_large_valid_relation_scale_and_limits_are_fixed(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
relation_names = tuple(f"relation-{position:05d}" for position in range(33_005))
descriptor = replace(
project.descriptor,
allowed_relations=relation_names,
limits=replace(
project.descriptor.limits,
max_results=2**63,
),
)
plan = build_retrieval_plan(
descriptor,
task_kind="change",
task="Exercise a very large valid relation policy",
focus_node_id="guide.workflow",
budget=None,
limit=None,
effective_policy=self.effective_policy(),
)
self.assertLessEqual(plan.max_evidence, MAX_TASK_EVIDENCE)
self.assertLessEqual(plan.max_candidate_edges, MAX_TASK_CANDIDATE_EDGES)
traversal = [step for step in plan.steps if step.operation in {"outgoing", "incoming"}]
self.assertEqual(2, len(traversal))
self.assertTrue(all(step.relation_scope == "project_allowed" for step in traversal))
self.assertTrue(all(len(step.relation_set_hash or "") == 64 for step in traversal))
self.assertLess(len(json.dumps(plan.as_dict())), 5_000)
def test_executor_rejects_tampered_public_plan_objects(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
index = ProjectIndex(project)
index.build()
plan = self.plan(project)
tampered = (
replace(plan, steps=plan.steps[:-1]),
replace(plan, requirements=()),
replace(plan, task_query="Different task"),
replace(plan, category_order=tuple(reversed(plan.category_order))),
replace(plan, max_evidence=plan.max_evidence + 1),
replace(plan, effective_policy_hash="0" * 64),
replace(plan, plan_hash="0" * 64),
)
for candidate in tampered:
with self.subTest(candidate=candidate):
with self.assertRaises(DocForgeError) as invalid:
index.task_context(candidate)
self.assertEqual("invalid_retrieval_plan", invalid.exception.code)
def test_work_and_unclassified_limits_are_explicit_and_bounded(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
relation_names = [f"relation-{position:03d}" for position in range(450)]
descriptor = root / ".docforge" / "project.toml"
raw_descriptor = descriptor.read_text(encoding="utf-8")
raw_relations = ", ".join(
json.dumps(relation)
for relation in (
"depends_on",
"proves",
"supersedes",
"relates_to",
"returns_to",
*relation_names,
)
)
descriptor.write_text(
raw_descriptor.replace(
'"depends_on", "proves", "supersedes", "relates_to", "returns_to"',
raw_relations,
),
encoding="utf-8",
)
workflow = root / "docs" / "content" / "workflow.md"
raw_workflow = workflow.read_text(encoding="utf-8")
relationships = "".join(
f'{relation} = ["guide.foundation"]\n' for relation in relation_names
)
workflow.write_text(
raw_workflow.replace("+++\n\nEditors", f"{relationships}+++\n\nEditors", 1),
encoding="utf-8",
)
project = Project.open(root)
index = ProjectIndex(project)
index.build()
capsule = index.task_context(self.plan(project))["capsule"]
Draft202012Validator(CAPSULE_SCHEMA).validate(capsule)
omission_codes = {omission["code"] for omission in capsule["omissions"]}
self.assertIn("edge_examination_limit", omission_codes)
self.assertIn("unclassified_relation_limit", omission_codes)
self.assertEqual(
project.descriptor.limits.max_results,
len(capsule["summary"]["unclassified_relations"]),
)
self.assertLessEqual(
capsule["summary"]["examined_edge_count"],
MAX_TASK_CANDIDATE_EDGES,
)
def test_lexical_focus_blocks_missing_and_ambiguous_selection(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
missing = index.task_context(
self.plan(
project,
task="Words absent from every indexed node",
focus_node_id=None,
)
)["capsule"]
self.assertEqual("blocked", missing["state"])
self.assertEqual("not_found", missing["focus"]["state"])
self.assertEqual("focus_not_found", missing["gaps"][0]["code"])
duplicate = root / "docs" / "content" / "workflow-copy.md"
duplicate.write_text(
(root / "docs" / "content" / "workflow.md")
.read_text(encoding="utf-8")
.replace('id = "guide.workflow"', 'id = "guide.workflow-copy"'),
encoding="utf-8",
)
duplicate_project = Project.open(root)
duplicate_index = ProjectIndex(duplicate_project)
duplicate_index.build()
ambiguous = duplicate_index.task_context(
self.plan(
duplicate_project,
task="Editing workflow",
focus_node_id=None,
)
)["capsule"]
self.assertEqual("blocked", ambiguous["state"])
self.assertEqual("ambiguous", ambiguous["focus"]["state"])
self.assertEqual("focus_ambiguous", ambiguous["gaps"][0]["code"])
def test_plan_rejects_unbounded_or_untyped_inputs(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
cases = (
({"task_kind": "unknown"}, "invalid_task_kind"),
({"task": " "}, "invalid_task_focus"),
({"focus_node_id": ""}, "invalid_task_focus"),
({"budget": True}, "invalid_budget"),
({"budget": 0}, "invalid_budget"),
({"limit": True}, "invalid_limit"),
({"limit": project.descriptor.limits.max_results + 1}, "invalid_limit"),
)
for overrides, code in cases:
arguments = {
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"budget": None,
"limit": None,
**overrides,
}
with self.subTest(arguments=overrides):
with self.assertRaises(DocForgeError) as invalid:
self.plan(project, **arguments)
self.assertEqual(code, invalid.exception.code)
def test_final_generation_change_rejects_the_whole_capsule(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
index = ProjectIndex(project)
built = index.build()
current = ProjectState(
revision=built["revision"],
source_hash=built["source_hash"],
)
changed = ProjectState(
revision=current.revision,
source_hash="0" * 64,
)
with (
mock.patch.object(
project,
"incremental_state",
side_effect=(current, changed),
),
self.assertRaises(DocForgeError) as stale,
):
index.task_context(self.plan(project))
self.assertEqual("source_changed", stale.exception.code)

1014
tools/milestone2_benchmark.py Normal file
View file

File diff suppressed because it is too large Load diff