administrator/DocForge2
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

Add versioned task context capsules

Browse source
This commit is contained in:
Andraxion 2026-07-29 07:10:18 -04:00
parent 34cd5f74c1
commit 4cc6277054
18 changed files with 2834 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

4
ACTIVE_SLICE.md
View file

@ -6,7 +6,9 @@ Goal: Let one project-bound server return compact, task-shaped, explainable cont
In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks. In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks.
Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases. Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass. Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass.
Status: Active. Read-only contract audits begin from the verified Milestone 1 boundary. Status: Active. Effective policy is committed. Versioned task retrieval and context-capsule
transport pass independent audit and the complete repository gate. The latest-generation diff
receipt is the next slice.
``` ```
Milestones 3–5 remain directional context and are not active. Milestones 3–5 remain directional context and are not active.

57
DEVELOPMENT_NOTES.md
View file

@ -388,3 +388,60 @@ exact-hash applier is enabled.
Existing factory defaults and tool order remain unchanged. Explicit application mode fails closed Existing factory defaults and tool order remain unchanged. Explicit application mode fails closed
without an applier. Operator mode is reserved and currently adds no tools. without an applier. Operator mode is reserved and currently adds no tools.
### Versioned task retrieval and context capsules
The first Milestone 2 retrieval slice adds one `docforge_get_task_context` read tool rather than a
family of task-specific tools. Its closed task kinds are change, implementation, failure,
ownership, test, operation, and release. One immutable `RetrievalPlanV1` derives exact or lexical
focus, bounded bidirectional graph traversal, metadata hydration, required evidence categories,
and fixed work budgets from the project descriptor and effective policy.
The public executor re-derives every submitted plan before opening SQLite. It rejects modified
steps, task identity, requirements, category order, bounds, policy identity, or hashes as
`invalid_retrieval_plan`. Traversal binds the project relation set by canonical hash and queries
the already-validated edge table by endpoint, avoiding relation-sized SQL parameter lists.
Version-1 internal ceilings are 1,000 evidence items, 100,000 candidate edges, and 10,000 task
query characters.
The executor uses one immutable SQLite read generation. It rejects missing explicit focus, blocks
unresolved or tied lexical focus, stops at deterministic evidence and candidate-edge limits, and
checks source identity again when the transaction closes. `ContextCapsuleV1` binds the generation,
policy, request, plan, evidence collection, and complete capsule with canonical hashes.
Project relation names remain authoritative. The core recognizes only a versioned alias map for
structure, implementation, dependency, execution, data, evidence, and context. Unknown allowed
relations stay visible under their raw names as `unclassified`. Required evidence diagnostics
distinguish categories the project never declared, completed bounded checks with no selected
evidence, and incomplete checks caused by a result, work, token, or response limit.
Each evidence item carries a stable content hash, confined source identity, shortest selected graph
path, every additional qualifying relationship reason observed during traversal, and explicit
limitations where the current graph cannot prove evidence type, extractor identity, relationship
source provenance, or observation time. The planner contains no Logic operation, so no-AST
bindings can use task context without weakening their existing Logic prohibition.
Path direction is relative to the preceding traversal node. Additional relationship reasons use
the evidence node as their direction subject. Candidate-edge and unclassified-relation ceilings
produce explicit omissions and bounded summaries.
MCP pagination preserves the complete plan, collection, and capsule hashes while returning bounded
pages. Its cursor additionally binds the effective policy and task request. An individually
oversized item advances once as a hash-identified omission. A later generation or policy change
fails closed as `stale_cursor`.
The legacy profile-context contract remains intact. A custom context provider does not silently
gain core task planning. Version 1 defines no custom task-planner extension, so the additive tool
returns `task_context_unavailable` without synchronization or a complete projection load.
Two independent pre-commit audits reproduced and closed plan-forgery, relation-sized SQL,
SQLite-parameter portability, ambiguous relationship-direction, missing work-limit evidence,
schema/runtime drift, incomplete page hashing, and custom-provider hidden-load defects. Regression
coverage includes 33,005 valid relation names, fixed extreme project limits, tampered plans,
evidence-relative directions, edge and unclassified limits, schema-valid pages, changed cursor
semantics, oversized evidence advancement, no-AST retrieval, and legacy complete-projection
adapters.
The complete repository gate passes with 158 tests and 101 subtests, zero Pyright diagnostics,
warning-strict execution, package builds, public-contract validation, and the maintained Milestone
0 and Milestone 1 smoke benchmarks. Gitleaks 8.30.1 reports no secret findings in the working tree.

8
README.md
View file

@ -9,6 +9,8 @@ declared manuals, visualizes project structure, and manages reviewable documenta
- Validates stable Markdown/TOML nodes and typed relationships. - Validates stable Markdown/TOML nodes and typed relationships.
- Builds a deterministic SQLite search and graph index. - Builds a deterministic SQLite search and graph index.
- Exposes project-bound CLI and MCP query surfaces. - Exposes project-bound CLI and MCP query surfaces.
- Compiles versioned, generation-bound task context with cited evidence, explicit gaps, and bounded
continuation.
- Automatically synchronizes disposable indexes before MCP work. - Automatically synchronizes disposable indexes before MCP work.
- Creates, validates, diffs, and previews isolated changesets. - Creates, validates, diffs, and previews isolated changesets.
- Registers complete proposals atomically without caller-managed hash chaining. - Registers complete proposals atomically without caller-managed hash chaining.
@ -53,6 +55,12 @@ DocForge2 bindings may also declare
policy and the actual startup-gated capabilities. Existing tool surfaces and the legacy no-AST policy and the actual startup-gated capabilities. Existing tool surfaces and the legacy no-AST
payload remain compatible. payload remain compatible.
`docforge_get_task_context` is an additive read tool for `change`, `implementation`, `failure`,
`ownership`, `test`, `operation`, and `release` work. It derives a closed version-1 retrieval plan,
executes it against one immutable index generation, and returns a hash-bound context capsule.
Project relation names remain authoritative. DocForge classifies only its versioned alias set and
preserves every unknown relation as `unclassified` instead of guessing semantics.
## Graph views ## Graph views
The browser presents the primary architecture graph through three complementary views and loads a The browser presents the primary architecture graph through three complementary views and loads a

31
docs/COMPATIBILITY.md
View file

@ -147,6 +147,37 @@ and side-effect-free status.
The live graph viewer remains a read-only consumer of a generation-pinned validated index. It does The live graph viewer remains a read-only consumer of a generation-pinned validated index. It does
not become project authority or MCP retrieval authority. not become project authority or MCP retrieval authority.
## Task-context compatibility
`docforge_get_task_context` is an additive MCP read tool. The legacy `docforge_get_context`
signature, profile compiler, direct Python results, and custom three-argument context-provider
contract remain unchanged.
The new `ContextCapsuleV1` and `RetrievalPlanV1` types live in the public
`docforge.retrieval` submodule. Version 1 guarantees:
- A closed task-kind vocabulary and core-derived plan. Callers cannot inject arbitrary operations,
SQL, paths, relations, or Logic requests.
- One immutable index transaction and one exact project, adapter, revision, source, policy,
request, plan, collection, and capsule identity.
- Deterministic bounded focus, traversal, hydration, token accounting, response packing, and
continuation, with fixed version-1 ceilings of 1,000 evidence items, 100,000 candidate edges, and
10,000 task-query characters.
- Raw preservation of project-owned relation names. Only the documented versioned alias map gains
task semantics; all other relations remain `unclassified`.
- Separate missing, incomplete, blocked, and provenance-limitation evidence.
- No-AST bindings retain task context but never add a Logic retrieval step or weaken the existing
Logic prohibition.
An integration that replaces the legacy context provider does not silently receive the core task
planner. Version 1 has no custom task-planner protocol. The task-context tool remains registered
for additive name compatibility but returns `task_context_unavailable` without loading or
synchronizing the custom projection.
The exact version-1 relation aliases are frozen by the MCP contract and repository contract tests.
Changing an alias category requires a new planner version; it is not a silent implementation
detail.
`ManualRenderPlan`, `GraphViewPlan`, a portable graph renderer, and independently packaged `ManualRenderPlan`, `GraphViewPlan`, a portable graph renderer, and independently packaged
renderers are later-milestone direction. Milestone 0 does not claim that those contracts exist. renderers are later-milestone direction. Milestone 0 does not claim that those contracts exist.

17
docs/CONTRACT.md
View file

@ -18,7 +18,9 @@ commit when Git is available; it cannot change repository state.
- Edge schema: `schemas/edge.schema.json`, version 1. - Edge schema: `schemas/edge.schema.json`, version 1.
- Result envelope: `schemas/result.schema.json`, version 1. - Result envelope: `schemas/result.schema.json`, version 1.
- Changeset schema: `schemas/changeset.schema.json`, version 1. - Changeset schema: `schemas/changeset.schema.json`, version 1.
- Index schema: version 2, disposable and reproducible. - Effective policy: `schemas/policy.schema.json`, version 1.
- Task context capsule: `schemas/context-capsule.schema.json`, version 1.
- Index schema: version 3, disposable and reproducible.
- Index attestation: schema version 1, disposable and reproducible. - Index attestation: schema version 1, disposable and reproducible.
- Core, CLI, and MCP server: version 1.3.0.dev0. - Core, CLI, and MCP server: version 1.3.0.dev0.
- Incremental extraction cache: version 1, disposable and reproducible. - Incremental extraction cache: version 1, disposable and reproducible.
@ -44,6 +46,12 @@ source hash. Errors use a stable code, direct message, structured details, and a
tool when recovery is safe. MCP operations synchronize disposable index state under a project lock tool when recovery is safe. MCP operations synchronize disposable index state under a project lock
before reading or proposing. Canonical source validation remains fail-closed. before reading or proposing. Canonical source validation remains fail-closed.
Task-context retrieval derives a closed version-1 plan from a bounded task kind and the effective
process policy. It executes against one immutable index transaction and returns generation-bound,
hash-identified evidence, gaps, omissions, and provenance limitations. Project relation names
remain authoritative. The core applies task semantics only to its versioned alias set and preserves
every other allowed relation as unclassified.
An atomic index build writes a whole-file SHA-256 attestation after complete graph, row, FTS, and An atomic index build writes a whole-file SHA-256 attestation after complete graph, row, FTS, and
SQLite integrity verification. A fresh process may use that receipt to verify an unchanged index SQLite integrity verification. A fresh process may use that receipt to verify an unchanged index
without reconstructing all graph rows. A missing, malformed, or mismatched receipt falls back to without reconstructing all graph rows. A missing, malformed, or mismatched receipt falls back to
@ -94,9 +102,10 @@ renderers. Render identity covers the canonical source hash, optional changeset
and edge identities, view configuration, template hash, renderer contract, and exact parser version. and edge identities, view configuration, template hash, renderer contract, and exact parser version.
An explicit CLI render atomically replaces one declared derived output. MCP can render a validated An explicit CLI render atomically replaces one declared derived output. MCP can render a validated
changeset only to its isolated preview path. Status recomputes expected output without writing and changeset only to its isolated preview path. Normal status verifies bounded source, configuration,
reports `current`, `stale`, `missing`, `unsafe`, or `oversized`. Input changes detected before atomic template, output, renderer, and publication-receipt identities without reconstructing the output.
replacement fail without replacing the prior output. Explicit deep status remains the side-effect-free full-render oracle. Input changes detected before
atomic replacement fail without publishing a current receipt for stale output.
Normal MCP access does not expose canonical application. An explicitly configured canonical Normal MCP access does not expose canonical application. An explicitly configured canonical
applier registers one hash-bound application tool. No MCP mode exposes arbitrary renderer applier registers one hash-bound application tool. No MCP mode exposes arbitrary renderer

41
docs/MCP_CONTRACT.md
View file

@ -42,6 +42,7 @@ schema version 1 and does not silently acquire machine-specific process policy.
- `docforge_dependencies` - `docforge_dependencies`
- `docforge_impact` - `docforge_impact`
- `docforge_get_context` - `docforge_get_context`
- `docforge_get_task_context`
- `docforge_validate_project` - `docforge_validate_project`
- `docforge_render_status` - `docforge_render_status`
- `docforge_visualize` - `docforge_visualize`
@ -72,6 +73,46 @@ oversized entry advances as a hash-identified `response size limit` omission so
loop; targeted retrieval remains available for that node. The existing three-argument custom loop; targeted retrieval remains available for that node. The existing three-argument custom
context-provider contract is unchanged because pagination is applied after provider selection. context-provider contract is unchanged because pagination is applied after provider selection.
`docforge_get_task_context` accepts one closed task kind (`change`, `implementation`, `failure`,
`ownership`, `test`, `operation`, or `release`), a bounded task description, and optional
`focus_node_id`, token `budget`, page `limit`, and opaque `cursor`. It derives, rather than accepts,
a version-1 retrieval plan. The plan contains only exact or lexical focus, bounded outgoing and
incoming graph traversal, and metadata hydration. It cannot request arbitrary SQL, paths, relation
names, or Logic extraction. Task context applies fixed internal ceilings of 1,000 evidence items,
100,000 examined candidate edges, and 10,000 task-query characters even when broader project
limits are configured. Traversal steps bind the complete project-owned relation vocabulary by hash
rather than copying an unbounded name list into every response.
The plan and returned context capsule are bound to the effective policy and one immutable index
generation. Every evidence item identifies its indexed source path, content hash, graph path,
additional qualifying relationship reasons, and the provenance facts that the current graph
cannot prove. Required evidence gaps distinguish an undeclared relation category, a completed
bounded search with no selected evidence, and an incomplete proof caused by a work, result, token,
or response limit. Unknown project relations remain present with their raw names and an
`unclassified_relation` limitation; DocForge never infers semantics from spelling outside the
versioned alias map.
Path relationship direction is relative to the preceding traversal node. Additional
`relationship_reasons` direction is relative to the evidence item itself: `outgoing` when that
evidence node is the stored source and `incoming` when it is the stored target.
The exact version-1 aliases are: structure (`contains`, `defined_in`, `defines`, `owns`);
implementation (`implemented_by`, `implements`, `inherits`, `inherits_from`); dependency
(`depends_on`, `imports`); execution (`activates`, `calls`, `dispatches_to`, `launches`); data
(`reads`, `writes`); evidence (`documents`, `governs`, `proves`, `tested_by`, `verifies`); and
context (`relates_to`). Every other allowed relation is `unclassified`.
Task-context continuation partitions the immutable evidence stream without changing its
`request_hash`, `plan_hash`, `collection_hash`, or `capsule_hash`. Its cursor additionally binds
the effective policy and task request. One evidence item that cannot fit advances exactly once as
a hash-identified `response_limit` omission. A changed generation, policy, plan, or collection
fails as `stale_cursor`.
The legacy `docforge_get_context` tool and its custom three-argument provider contract remain
unchanged. A server with a custom context provider does not silently inherit the core task planner;
version 1 exposes no custom task-planner extension point. `docforge_get_task_context` returns
`task_context_unavailable` without synchronizing or loading the custom projection.
Version-1 cursors are canonical JSON encoded as base64url with a domain-separated SHA-256 Version-1 cursors are canonical JSON encoded as base64url with a domain-separated SHA-256
corruption checksum. They are opaque and fail closed, but are not authenticated authorization corruption checksum. They are opaque and fail closed, but are not authenticated authorization
tokens. Cursors bind the project, adapter, source generation, operation parameters, collection tokens. Cursors bind the project, adapter, source generation, operation parameters, collection

36
docs/USER_MANUAL.md
View file

@ -582,6 +582,7 @@ Example MCP client configuration:
- `docforge_dependencies` - `docforge_dependencies`
- `docforge_impact` - `docforge_impact`
- `docforge_get_context` - `docforge_get_context`
- `docforge_get_task_context`
- `docforge_validate_project` - `docforge_validate_project`
- `docforge_render_status` - `docforge_render_status`
- `docforge_visualize` - `docforge_visualize`
@ -613,6 +614,36 @@ The application call requires `changeset_id` and `expected_changeset_hash`. Alwa
inspect the final diff after the last proposal mutation. Apply that exact hash. A proposal mutation inspect the final diff after the last proposal mutation. Apply that exact hash. A proposal mutation
creates a new hash, so an earlier approval cannot silently apply later content. creates a new hash, so an earlier approval cannot silently apply later content.
Use `docforge_get_task_context` when an agent needs one bounded task-shaped intake instead of a
named profile. Choose `task_kind` from `change`, `implementation`, `failure`, `ownership`, `test`,
`operation`, or `release`. Supply `focus_node_id` when the stable node is known. Without it,
DocForge performs a bounded lexical focus search and refuses a tied best match instead of silently
choosing one.
The returned version-1 capsule includes:
- The exact project, adapter, source generation, effective policy, request, and retrieval-plan
hashes.
- Ordered focus and related evidence with source paths, content hashes, graph paths, and all
qualifying relationship reasons observed during the bounded traversal.
- Explicit evidence gaps and omissions, including whether a check completed.
- Provenance limitations for facts that the current graph does not carry, such as extractor
identity, observation time, and source provenance for relationships.
Project descriptors still own the valid relation vocabulary. The planner recognizes a fixed alias
map for structure, implementation, dependency, execution, data, evidence, and context. Any other
valid project relation is returned unchanged as `unclassified`; it is never assigned guessed task
semantics.
A relationship inside `relationship_path` describes the direction traveled from the preceding
node. A relationship inside `relationship_reasons` describes direction from the evidence item
itself. This keeps stored source and target identity exact while making each evidence explanation
locally readable.
Task context never exceeds 1,000 evidence items, 100,000 examined candidate edges, or 10,000 task
query characters, even when a project configures broader general limits. An edge-work or
unclassified-relation ceiling appears as an explicit omission rather than an unbounded response.
Recommended release-candidate sequence: Recommended release-candidate sequence:
1. Call `docforge_bootstrap`. It synchronizes derived state and reports the exact fixed binding. 1. Call `docforge_bootstrap`. It synchronizes derived state and reports the exact fixed binding.
@ -675,6 +706,11 @@ inspection pages may use hash summaries. A large diff may return `result_mode =
"canonical_json_chunk"`; concatenate the chunks in order and verify `payload_hash` before decoding "canonical_json_chunk"`; concatenate the chunks in order and verify `payload_hash` before decoding
the reconstructed `operations` and `changes` object. the reconstructed `operations` and `changes` object.
`docforge_get_task_context` uses the same opaque continuation discipline over capsule evidence
followed by capsule omissions. Keep the semantic task arguments unchanged while paging. Page size
may change. Every page retains the same plan, collection, and capsule hashes. A `stale_cursor`
means that the generation, policy, plan, or collection changed; discard earlier pages and restart.
Canonical application records its terminal receipt immediately after the project-owned serializer Canonical application records its terminal receipt immediately after the project-owned serializer
verifies the new canonical state. A later index or render refresh failure is reported as degraded verifies the new canonical state. A later index or render refresh failure is reported as degraded
derived state with remediation, not as permission to apply the same canonical change again. derived state with remediation, not as permission to apply the same canonical change again.

435
schemas/context-capsule.schema.json Normal file
View file

@ -0,0 +1,435 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://docforge.local/schema/context-capsule-v1.json",
"title": "DocForge task context capsule",
"$defs": {
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"pagination": {
"type": "object",
"required": [
"schema_version",
"kind",
"returned_count",
"limit",
"total_count",
"has_more",
"next_cursor"
],
"properties": {
"schema_version": { "const": 1 },
"kind": { "const": "task-context.items" },
"returned_count": { "type": "integer", "minimum": 0 },
"limit": { "type": "integer", "minimum": 1 },
"total_count": { "type": "integer", "minimum": 0 },
"has_more": { "type": "boolean" },
"next_cursor": {
"type": ["string", "null"],
"minLength": 1,
"maxLength": 8192
}
},
"additionalProperties": false
},
"category": {
"enum": [
"structure",
"implementation",
"dependency",
"execution",
"data",
"evidence",
"context",
"unclassified"
]
},
"step": {
"type": "object",
"required": [
"step_id",
"operation",
"relation_scope",
"relation_set_hash",
"direction",
"depth",
"limit",
"required",
"evidence_role"
],
"properties": {
"step_id": { "type": "string", "minLength": 1, "maxLength": 64 },
"operation": {
"enum": ["exact", "search", "outgoing", "incoming", "metadata"]
},
"relation_scope": { "enum": ["none", "project_allowed"] },
"relation_set_hash": {
"oneOf": [
{ "$ref": "#/$defs/sha256" },
{ "type": "null" }
]
},
"direction": { "enum": ["none", "outgoing", "incoming"] },
"depth": { "type": "integer", "minimum": 0 },
"limit": { "type": "integer", "minimum": 1 },
"required": { "type": "boolean" },
"evidence_role": { "type": "string", "minLength": 1, "maxLength": 64 }
},
"additionalProperties": false
},
"requirement": {
"type": "object",
"required": ["requirement_id", "check", "category", "required"],
"properties": {
"requirement_id": { "type": "string", "minLength": 1, "maxLength": 128 },
"check": { "const": "selected_relation_category" },
"category": { "$ref": "#/$defs/category" },
"required": { "const": true }
},
"additionalProperties": false
},
"plan": {
"type": "object",
"required": [
"schema_version",
"planner",
"task_kind",
"request_hash",
"effective_policy_hash",
"focus_node_id",
"limits",
"category_order",
"steps",
"requirements",
"plan_hash"
],
"properties": {
"schema_version": { "const": 1 },
"planner": {
"type": "object",
"required": ["id", "version"],
"properties": {
"id": { "const": "docforge.core.task-context" },
"version": { "const": 1 }
},
"additionalProperties": false
},
"task_kind": {
"enum": [
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release"
]
},
"request_hash": { "$ref": "#/$defs/sha256" },
"effective_policy_hash": { "$ref": "#/$defs/sha256" },
"focus_node_id": { "type": ["string", "null"], "maxLength": 256 },
"limits": {
"type": "object",
"required": [
"max_evidence",
"max_tokens",
"max_depth",
"max_candidate_edges"
],
"properties": {
"max_evidence": { "type": "integer", "minimum": 1, "maximum": 1000 },
"max_tokens": { "type": "integer", "minimum": 1 },
"max_depth": { "type": "integer", "minimum": 0 },
"max_candidate_edges": {
"type": "integer",
"minimum": 1,
"maximum": 100000
}
},
"additionalProperties": false
},
"category_order": {
"type": "array",
"minItems": 8,
"maxItems": 8,
"items": { "$ref": "#/$defs/category" },
"uniqueItems": true
},
"steps": {
"type": "array",
"minItems": 4,
"maxItems": 4,
"items": { "$ref": "#/$defs/step" }
},
"requirements": {
"type": "array",
"minItems": 1,
"maxItems": 8,
"items": { "$ref": "#/$defs/requirement" }
},
"plan_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"relationship": {
"type": "object",
"required": [
"source_id",
"relation",
"target_id",
"direction",
"category",
"provenance"
],
"properties": {
"source_id": { "type": "string", "minLength": 1 },
"relation": { "type": "string", "minLength": 1 },
"target_id": { "type": "string", "minLength": 1 },
"direction": { "enum": ["outgoing", "incoming"] },
"category": { "$ref": "#/$defs/category" },
"provenance": {
"const": "validated_graph_edge_without_source_provenance"
}
},
"additionalProperties": false
},
"evidence": {
"type": "object",
"required": [
"evidence_hash",
"role",
"reason_code",
"node_id",
"title",
"family",
"authority",
"status",
"tags",
"summary",
"text",
"estimated_tokens",
"source",
"depth",
"relationship_path",
"relationship_reasons",
"provenance_limitations"
],
"properties": {
"evidence_hash": { "$ref": "#/$defs/sha256" },
"role": { "enum": ["focus", "related"] },
"reason_code": {
"enum": ["exact_focus", "lexical_focus", "relationship_path"]
},
"node_id": { "type": "string", "minLength": 1 },
"title": { "type": "string" },
"family": { "type": "string" },
"authority": { "type": "string" },
"status": { "type": "string" },
"tags": {
"type": "array",
"items": { "type": "string" }
},
"summary": { "type": "string" },
"text": { "type": "string" },
"estimated_tokens": { "type": "integer", "minimum": 1 },
"source": {
"type": "object",
"required": ["path", "anchor", "content_hash"],
"properties": {
"path": { "type": "string", "minLength": 1 },
"anchor": { "type": ["string", "null"] },
"content_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
},
"depth": { "type": "integer", "minimum": 0 },
"relationship_path": {
"type": "array",
"items": { "$ref": "#/$defs/relationship" }
},
"relationship_reasons": {
"type": "array",
"items": { "$ref": "#/$defs/relationship" },
"uniqueItems": true
},
"provenance_limitations": {
"const": [
"evidence_type_unavailable",
"extractor_identity_unavailable",
"relationship_provenance_unavailable",
"observation_time_unavailable"
]
}
},
"additionalProperties": false
},
"gap": {
"type": "object",
"required": [
"code",
"requirement_id",
"category",
"state",
"check_complete",
"detail"
],
"properties": {
"code": {
"enum": [
"focus_not_found",
"focus_ambiguous",
"category_not_declared",
"no_selected_evidence",
"evidence_incomplete",
"unclassified_relation"
]
},
"requirement_id": { "type": "string", "minLength": 1 },
"category": {
"oneOf": [
{ "$ref": "#/$defs/category" },
{ "type": "null" }
]
},
"state": { "enum": ["missing", "incomplete", "blocked", "limitation"] },
"check_complete": { "type": "boolean" },
"detail": { "type": "string", "minLength": 1, "maxLength": 1000 }
},
"additionalProperties": false
},
"omission": {
"type": "object",
"required": ["code", "subject", "detail_hash"],
"properties": {
"code": {
"enum": [
"result_limit",
"token_budget",
"response_limit",
"edge_examination_limit",
"unclassified_relation_limit"
]
},
"subject": { "type": "string", "minLength": 1, "maxLength": 256 },
"detail_hash": { "$ref": "#/$defs/sha256" }
},
"additionalProperties": false
}
},
"type": "object",
"required": [
"schema_version",
"state",
"task_kind",
"generation",
"plan",
"focus",
"evidence",
"gaps",
"omissions",
"summary",
"collection_hash",
"capsule_hash"
],
"properties": {
"schema_version": { "const": 1 },
"state": { "enum": ["complete", "incomplete", "blocked"] },
"task_kind": {
"enum": [
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release"
]
},
"generation": {
"type": "object",
"required": [
"project_id",
"project_root_fingerprint",
"adapter",
"revision",
"source_hash",
"index_schema_version"
],
"properties": {
"project_id": { "type": "string", "minLength": 1 },
"project_root_fingerprint": {
"type": "string",
"pattern": "^[0-9a-f]{16}$"
},
"adapter": { "type": "string", "minLength": 1 },
"revision": { "type": "string", "minLength": 1 },
"source_hash": { "$ref": "#/$defs/sha256" },
"index_schema_version": { "type": "integer", "minimum": 1 }
},
"additionalProperties": false
},
"plan": { "$ref": "#/$defs/plan" },
"focus": {
"type": "object",
"required": ["state", "node_id", "candidate_count"],
"properties": {
"state": { "enum": ["resolved", "not_found", "ambiguous"] },
"node_id": { "type": ["string", "null"] },
"candidate_count": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"evidence": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/evidence" }
},
"gaps": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/gap" }
},
"omissions": {
"type": "array",
"maxItems": 10000,
"items": { "$ref": "#/$defs/omission" }
},
"summary": {
"type": "object",
"required": [
"evidence_count",
"gap_count",
"omission_count",
"selected_count",
"examined_edge_count",
"estimated_tokens",
"unclassified_relations"
],
"properties": {
"evidence_count": { "type": "integer", "minimum": 0 },
"gap_count": { "type": "integer", "minimum": 0 },
"omission_count": { "type": "integer", "minimum": 0 },
"selected_count": { "type": "integer", "minimum": 0 },
"examined_edge_count": { "type": "integer", "minimum": 0 },
"estimated_tokens": { "type": "integer", "minimum": 0 },
"unclassified_relations": {
"type": "array",
"maxItems": 10000,
"items": { "type": "string", "minLength": 1 },
"uniqueItems": true
},
"page_evidence_count": { "type": "integer", "minimum": 0 },
"page_omission_count": { "type": "integer", "minimum": 0 },
"page_item_count": { "type": "integer", "minimum": 0 }
},
"additionalProperties": false
},
"collection_hash": { "$ref": "#/$defs/sha256" },
"capsule_hash": { "$ref": "#/$defs/sha256" },
"page_state": { "enum": ["complete", "incomplete", "blocked"] },
"page_hash": { "$ref": "#/$defs/sha256" },
"pagination": { "$ref": "#/$defs/pagination" }
},
"additionalProperties": false
}

2
schemas/result.schema.json
View file

@ -32,6 +32,7 @@
"mcp.dependencies", "mcp.dependencies",
"mcp.impact", "mcp.impact",
"mcp.context", "mcp.context",
"mcp.task_context",
"mcp.validate_project", "mcp.validate_project",
"mcp.render_status", "mcp.render_status",
"mcp.visualize", "mcp.visualize",
@ -149,6 +150,7 @@
"kind": { "kind": {
"enum": [ "enum": [
"context.items", "context.items",
"task-context.items",
"changeset.list", "changeset.list",
"changeset.inspect", "changeset.inspect",
"changeset.validate", "changeset.validate",

439
src/docforge/index.py
View file

@ -14,7 +14,7 @@ from collections.abc import Callable, Generator
from contextlib import contextmanager from contextlib import contextmanager
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import cast from typing import Literal, cast
from .errors import DocForgeError from .errors import DocForgeError
from .models import ( from .models import (
@ -32,11 +32,25 @@ from .models import (
ProjectSnapshot, ProjectSnapshot,
ProjectState, ProjectState,
) )
from .pagination import canonical_hash
from .project import project_root_fingerprint from .project import project_root_fingerprint
from .retrieval import (
CapsuleEvidenceV1,
CapsuleOmissionV1,
CapsuleRelationshipV1,
ContextCapsuleV1,
EvidenceGapV1,
RetrievalPlanV1,
capsule_evidence,
finalize_capsule,
relation_category,
validate_retrieval_plan,
)
from .telemetry import increment, stage from .telemetry import increment, stage
INDEX_SCHEMA_VERSION = 3 INDEX_SCHEMA_VERSION = 3
APPLICATION_ID = 1_146_683_778 APPLICATION_ID = 1_146_683_778
_SQLITE_PARAMETER_CHUNK = 500
def _node_hash(nodes: tuple[Node, ...]) -> str: def _node_hash(nodes: tuple[Node, ...]) -> str:
@ -147,6 +161,17 @@ class _IndexReadSnapshot:
} }
@dataclass(frozen=True)
class _TaskSelection:
node_id: str
role: str
reason_code: str
depth: int
category_rank: int
first_edge: tuple[str, str, str]
relationship_path: tuple[CapsuleRelationshipV1, ...]
class ProjectIndex: class ProjectIndex:
"""A disposable index that always checks current canonical source before queries.""" """A disposable index that always checks current canonical source before queries."""
@ -798,6 +823,418 @@ class ProjectIndex:
status.st_ctime_ns, status.st_ctime_ns,
) )
def task_context(self, plan: RetrievalPlanV1) -> dict[str, object]:
"""Execute one fixed task plan inside one immutable index generation."""
validate_retrieval_plan(plan, self.project.descriptor)
with self._read_snapshot() as snapshot:
capsule = self._task_context_capsule(snapshot, plan)
return snapshot.result(capsule=capsule.as_dict())
def _task_context_capsule(
self,
snapshot: _IndexReadSnapshot,
plan: RetrievalPlanV1,
) -> ContextCapsuleV1:
generation = {
"project_id": snapshot.checked["project_id"],
"project_root_fingerprint": snapshot.checked["project_root_fingerprint"],
"adapter": snapshot.checked["adapter"],
"revision": snapshot.checked["revision"],
"source_hash": snapshot.checked["source_hash"],
"index_schema_version": INDEX_SCHEMA_VERSION,
}
focus_rows: list[sqlite3.Row]
focus_reason = "exact_focus"
if plan.focus_node_id is not None:
row = snapshot.connection.execute(
"SELECT * FROM nodes WHERE node_id = ?",
(plan.focus_node_id,),
).fetchone()
if row is None:
raise DocForgeError(
"missing_node",
"No node has the requested stable ID",
node_id=plan.focus_node_id,
)
focus_rows = [row]
else:
terms = re_tokenize(plan.task_query)
if not terms:
raise DocForgeError(
"invalid_task_focus",
"Task description contains no searchable text",
)
expression = " AND ".join(f'"{term.replace(chr(34), chr(34) * 2)}"' for term in terms)
focus_rows = snapshot.connection.execute(
"""
SELECT nodes.*, bm25(node_fts) AS rank
FROM node_fts JOIN nodes USING(node_id)
WHERE node_fts MATCH ?
ORDER BY rank, nodes.node_id
LIMIT ?
""",
(expression, plan.max_evidence + 1),
).fetchall()
focus_reason = "lexical_focus"
if not focus_rows:
return finalize_capsule(
state="blocked",
task_kind=plan.task_kind,
generation=generation,
plan=plan,
focus_state="not_found",
focus_node_id=None,
focus_candidate_count=0,
evidence=(),
gaps=(
EvidenceGapV1(
code="focus_not_found",
requirement_id="focus",
category=None,
state="blocked",
check_complete=True,
detail="No indexed node matched the bounded lexical focus.",
),
),
omissions=(),
selected_count=0,
examined_edge_count=0,
estimated_tokens=0,
unclassified_relations=(),
)
if len(focus_rows) > 1 and focus_rows[0]["rank"] == focus_rows[1]["rank"]:
return finalize_capsule(
state="blocked",
task_kind=plan.task_kind,
generation=generation,
plan=plan,
focus_state="ambiguous",
focus_node_id=None,
focus_candidate_count=len(focus_rows),
evidence=(),
gaps=(
EvidenceGapV1(
code="focus_ambiguous",
requirement_id="focus",
category=None,
state="blocked",
check_complete=True,
detail=(
"The highest-ranked lexical focus is tied; provide focus_node_id."
),
),
),
omissions=(),
selected_count=0,
examined_edge_count=0,
estimated_tokens=0,
unclassified_relations=(),
)
focus_rows = focus_rows[:1]
focus_node_id = cast(str, focus_rows[0]["node_id"])
selections: dict[str, _TaskSelection] = {
focus_node_id: _TaskSelection(
node_id=focus_node_id,
role="focus",
reason_code=focus_reason,
depth=0,
category_rank=-1,
first_edge=("", "", ""),
relationship_path=(),
)
}
relationship_reasons: dict[str, set[CapsuleRelationshipV1]] = {focus_node_id: set()}
queue: deque[str] = deque((focus_node_id,))
examined_edge_count = 0
traversal_incomplete = False
edge_examination_limit_reached = False
first_omitted_node: str | None = None
unclassified_relations: set[str] = set()
while queue and not traversal_incomplete:
current = queue.popleft()
current_selection = selections[current]
if current_selection.depth >= plan.max_depth:
continue
remaining = plan.max_candidate_edges - examined_edge_count
if remaining <= 0:
traversal_incomplete = True
edge_examination_limit_reached = True
break
outgoing = snapshot.connection.execute(
"SELECT source_id, relation, target_id FROM edges "
"WHERE source_id = ? ORDER BY source_id, relation, target_id LIMIT ?",
(current, remaining + 1),
).fetchall()
incoming = snapshot.connection.execute(
"SELECT source_id, relation, target_id FROM edges "
"WHERE target_id = ? ORDER BY source_id, relation, target_id LIMIT ?",
(current, remaining + 1),
).fetchall()
candidates = {
(
cast(str, row["source_id"]),
cast(str, row["relation"]),
cast(str, row["target_id"]),
"outgoing" if row["source_id"] == current else "incoming",
)
for row in (*outgoing, *incoming)
}
ordered = sorted(
candidates,
key=lambda item: (
plan.category_order.index(relation_category(item[1])),
item[0],
item[1],
item[2],
item[3],
),
)
if len(ordered) > remaining:
traversal_incomplete = True
edge_examination_limit_reached = True
ordered = ordered[:remaining]
for source_id, relation, target_id, direction in ordered:
examined_edge_count += 1
category = relation_category(relation)
if category == "unclassified":
unclassified_relations.add(relation)
neighbor = target_id if source_id == current else source_id
relationship = CapsuleRelationshipV1(
source_id=source_id,
relation=relation,
target_id=target_id,
direction=cast(Literal["outgoing", "incoming"], direction),
category=category,
)
evidence_reason = CapsuleRelationshipV1(
source_id=source_id,
relation=relation,
target_id=target_id,
direction="outgoing" if source_id == neighbor else "incoming",
category=category,
)
if neighbor in selections:
relationship_reasons.setdefault(neighbor, set()).add(evidence_reason)
continue
if len(selections) >= plan.max_evidence:
traversal_incomplete = True
first_omitted_node = neighbor
break
selections[neighbor] = _TaskSelection(
node_id=neighbor,
role="related",
reason_code="relationship_path",
depth=current_selection.depth + 1,
category_rank=plan.category_order.index(category),
first_edge=(source_id, relation, target_id),
relationship_path=(
*current_selection.relationship_path,
relationship,
),
)
relationship_reasons[neighbor] = {evidence_reason}
queue.append(neighbor)
ordered_selections = sorted(
selections.values(),
key=lambda selection: (
0 if selection.role == "focus" else 1,
selection.category_rank,
selection.depth,
selection.first_edge,
selection.node_id,
),
)
selected_node_ids = tuple(selection.node_id for selection in ordered_selections)
rows: list[sqlite3.Row] = []
for position in range(0, len(selected_node_ids), _SQLITE_PARAMETER_CHUNK):
node_id_chunk = selected_node_ids[position : position + _SQLITE_PARAMETER_CHUNK]
placeholders = ",".join("?" for _ in node_id_chunk)
rows.extend(
snapshot.connection.execute(
f"SELECT * FROM nodes WHERE node_id IN ({placeholders}) ORDER BY node_id",
node_id_chunk,
).fetchall()
)
nodes = {cast(str, row["node_id"]): _row_to_node(row) for row in rows}
evidence: list[CapsuleEvidenceV1] = []
omissions: list[CapsuleOmissionV1] = []
used_tokens = 0
returned_categories: set[str] = set()
for selection in ordered_selections:
node = nodes[selection.node_id]
item = capsule_evidence(
role=cast(Literal["focus", "related"], selection.role),
reason_code=cast(
Literal["exact_focus", "lexical_focus", "relationship_path"],
selection.reason_code,
),
node=node,
depth=selection.depth,
relationship_path=selection.relationship_path,
relationship_reasons=tuple(
sorted(
relationship_reasons.get(selection.node_id, set()),
key=lambda relationship: (
relationship.source_id,
relationship.relation,
relationship.target_id,
relationship.direction,
),
)
),
)
if used_tokens + item.estimated_tokens > plan.max_tokens:
omissions.append(
CapsuleOmissionV1(
code="token_budget",
subject=node.node_id,
detail_hash=canonical_hash(
{
"node_id": node.node_id,
"content_hash": node.content_hash,
"estimated_tokens": item.estimated_tokens,
}
),
)
)
continue
evidence.append(item)
used_tokens += item.estimated_tokens
returned_categories.update(
relationship.category for relationship in item.relationship_reasons
)
if first_omitted_node is not None:
omissions.append(
CapsuleOmissionV1(
code="result_limit",
subject=first_omitted_node,
detail_hash=canonical_hash(
{
"node_id": first_omitted_node,
"max_evidence": plan.max_evidence,
}
),
)
)
if edge_examination_limit_reached:
omissions.append(
CapsuleOmissionV1(
code="edge_examination_limit",
subject=focus_node_id,
detail_hash=canonical_hash(
{
"focus_node_id": focus_node_id,
"examined_edge_count": examined_edge_count,
"max_candidate_edges": plan.max_candidate_edges,
}
),
)
)
ordered_unclassified_relations = tuple(sorted(unclassified_relations))
bounded_unclassified_relations = ordered_unclassified_relations[: plan.max_evidence]
if len(ordered_unclassified_relations) > len(bounded_unclassified_relations):
omissions.append(
CapsuleOmissionV1(
code="unclassified_relation_limit",
subject=focus_node_id,
detail_hash=canonical_hash(
{
"unclassified_relation_count": len(ordered_unclassified_relations),
"returned_count": len(bounded_unclassified_relations),
}
),
)
)
declared_categories = {
relation_category(relation) for relation in self.project.descriptor.allowed_relations
}
gaps: list[EvidenceGapV1] = []
incomplete = traversal_incomplete or bool(omissions)
for requirement in plan.requirements:
category = requirement.category
if category not in declared_categories:
gaps.append(
EvidenceGapV1(
code="category_not_declared",
requirement_id=requirement.requirement_id,
category=category,
state="missing",
check_complete=True,
detail=(
"No declared project relation maps to this required plan category."
),
)
)
elif category in returned_categories:
continue
elif incomplete:
gaps.append(
EvidenceGapV1(
code="evidence_incomplete",
requirement_id=requirement.requirement_id,
category=category,
state="incomplete",
check_complete=False,
detail=(
"A bounded work, result, or token limit prevented a complete "
"returned-evidence proof."
),
)
)
else:
gaps.append(
EvidenceGapV1(
code="no_selected_evidence",
requirement_id=requirement.requirement_id,
category=category,
state="missing",
check_complete=True,
detail=(
"The complete bounded check found no selected graph evidence in "
"this category."
),
)
)
for relation in bounded_unclassified_relations:
gaps.append(
EvidenceGapV1(
code="unclassified_relation",
requirement_id=f"relation.{relation}",
category="unclassified",
state="limitation",
check_complete=True,
detail="The project relation is preserved without inferred task semantics.",
)
)
gaps.sort(
key=lambda gap: (
gap.requirement_id,
gap.code,
"" if gap.category is None else gap.category,
)
)
return finalize_capsule(
state="incomplete" if incomplete else "complete",
task_kind=plan.task_kind,
generation=generation,
plan=plan,
focus_state="resolved",
focus_node_id=focus_node_id,
focus_candidate_count=len(focus_rows),
evidence=tuple(evidence),
gaps=tuple(gaps),
omissions=tuple(omissions),
selected_count=len(selections),
examined_edge_count=examined_edge_count,
estimated_tokens=used_tokens,
unclassified_relations=bounded_unclassified_relations,
)
def get_node(self, node_id: str) -> dict[str, object]: def get_node(self, node_id: str) -> dict[str, object]:
with self._read_snapshot() as snapshot: with self._read_snapshot() as snapshot:
row = snapshot.connection.execute( row = snapshot.connection.execute(

253
src/docforge/mcp_server.py
View file

@ -21,6 +21,7 @@ from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt
from .policy import CapabilityMode, capability_mode, compose_effective_policy from .policy import CapabilityMode, capability_mode, compose_effective_policy
from .project import Project, project_root_fingerprint from .project import Project, project_root_fingerprint
from .rendering import RenderService from .rendering import RenderService
from .retrieval import MAX_TASK_EVIDENCE, TaskKind, build_retrieval_plan
from .telemetry import request, stage from .telemetry import request, stage
from .viewer_manager import ViewerManagerClient from .viewer_manager import ViewerManagerClient
@ -43,6 +44,7 @@ READ_TOOLS = (
"docforge_dependencies", "docforge_dependencies",
"docforge_impact", "docforge_impact",
"docforge_get_context", "docforge_get_context",
"docforge_get_task_context",
"docforge_validate_project", "docforge_validate_project",
"docforge_render_status", "docforge_render_status",
"docforge_visualize", "docforge_visualize",
@ -164,6 +166,7 @@ class DocForgeService:
) )
self.visualization = ViewerManagerClient(self.index) self.visualization = ViewerManagerClient(self.index)
self.context_provider = context_provider self.context_provider = context_provider
self.task_context_available = context_provider is compile_context
self.binding_metadata = dict(binding_metadata or {}) self.binding_metadata = dict(binding_metadata or {})
self.no_ast = self.policy.no_ast self.no_ast = self.policy.no_ast
self.diagnostics = diagnostics self.diagnostics = diagnostics
@ -199,6 +202,14 @@ class DocForgeService:
"enabled": True, "enabled": True,
"tools": [tool for tool in READ_TOOLS if tool in self.tool_surface], "tools": [tool for tool in READ_TOOLS if tool in self.tool_surface],
}, },
"task_context": {
"enabled": self.task_context_available,
"reason": (
None
if self.task_context_available
else "custom_context_policy_not_supported_by_task_context_v1"
),
},
"proposal": { "proposal": {
"surface_enabled": any(tool in self.tool_surface for tool in PROPOSAL_TOOLS), "surface_enabled": any(tool in self.tool_surface for tool in PROPOSAL_TOOLS),
"mutation_access": proposal_access, "mutation_access": proposal_access,
@ -558,7 +569,11 @@ class DocForgeService:
"adapter_policy": self.adapter_policy(), "adapter_policy": self.adapter_policy(),
} }
recommended_workflow = [ recommended_workflow = [
"docforge_get_context or targeted read tools", (
"docforge_get_task_context, docforge_get_context, or targeted read tools"
if self.task_context_available
else "docforge_get_context or targeted read tools"
),
"make and verify one coherent implementation slice", "make and verify one coherent implementation slice",
"docforge_sync", "docforge_sync",
] ]
@ -582,8 +597,17 @@ class DocForgeService:
"compiler-AST, or function-Logic extraction" "compiler-AST, or function-Logic extraction"
), ),
) )
if descriptor.profiles: if self.task_context_available:
recommended_first_operation: dict[str, object] = { recommended_first_operation: dict[str, object] = {
"tool": "docforge_get_task_context",
"arguments": {
"task_kind": "implementation",
"task": "<describe the current task>",
},
"reason": "Begin with one bounded task-shaped context capsule.",
}
elif descriptor.profiles:
recommended_first_operation = {
"tool": "docforge_get_context", "tool": "docforge_get_context",
"arguments": {"profile": descriptor.profiles[0].profile_id}, "arguments": {"profile": descriptor.profiles[0].profile_id},
"reason": "Begin with one configured bounded context profile.", "reason": "Begin with one configured bounded context profile.",
@ -831,6 +855,210 @@ class DocForgeService:
operation_name="mcp.context", operation_name="mcp.context",
) )
def task_context(
self,
task_kind: TaskKind,
task: str,
*,
focus_node_id: str | None = None,
budget: int | None = None,
limit: int | None = None,
cursor: str | None = None,
) -> dict[str, Any]:
"""Return one task-shaped capsule from a single immutable graph generation."""
if not self.task_context_available:
def unavailable() -> dict[str, object]:
raise DocForgeError(
"task_context_unavailable",
(
"This binding uses a custom context provider; "
"core task planning is unavailable"
),
)
return self.invoke(
unavailable,
synchronize=False,
load_error_identity=False,
operation_name="mcp.task_context",
)
def operation() -> dict[str, object]:
maximum_evidence = min(
self.project.descriptor.limits.max_results,
MAX_TASK_EVIDENCE,
)
selected_limit = page_limit(
limit,
default=min(20, maximum_evidence),
maximum=maximum_evidence,
)
plan = build_retrieval_plan(
self.project.descriptor,
task_kind=task_kind,
task=task,
focus_node_id=focus_node_id,
budget=budget,
limit=maximum_evidence,
effective_policy=self.policy.as_dict(),
)
result = self.index.task_context(plan)
return self._page_task_context_result(
result,
selected_limit=selected_limit,
cursor=cursor,
)
return self.invoke(operation, operation_name="mcp.task_context")
def _page_task_context_result(
self,
result: dict[str, object],
*,
selected_limit: int,
cursor: str | None,
) -> dict[str, object]:
capsule_value = result.get("capsule")
if not isinstance(capsule_value, Mapping):
raise DocForgeError(
"invalid_task_context_result",
"Task context did not return a versioned capsule",
)
capsule = dict(cast(Mapping[str, object], capsule_value))
plan_value = capsule.get("plan")
generation_value = capsule.get("generation")
evidence_value = capsule.get("evidence")
omissions_value = capsule.get("omissions")
gaps_value = capsule.get("gaps")
if (
capsule.get("schema_version") != 1
or not isinstance(plan_value, Mapping)
or not isinstance(generation_value, Mapping)
or not isinstance(evidence_value, list)
or not isinstance(omissions_value, list)
or not isinstance(gaps_value, list)
or not isinstance(capsule.get("collection_hash"), str)
or not isinstance(capsule.get("capsule_hash"), str)
):
raise DocForgeError(
"invalid_task_context_result",
"Task context capsule is malformed",
)
plan_payload = cast(Mapping[str, object], plan_value)
generation = cast(Mapping[str, object], generation_value)
evidence = cast(list[object], evidence_value)
omissions = cast(list[object], omissions_value)
gaps = cast(list[object], gaps_value)
binding = {
"project_id": result.get("project_id"),
"project_root_fingerprint": result.get("project_root_fingerprint"),
"adapter": result.get("adapter"),
"revision": result.get("revision"),
"source_hash": result.get("source_hash"),
"index_schema_version": generation.get("index_schema_version"),
"effective_policy_hash": plan_payload.get("effective_policy_hash"),
"request_hash": plan_payload.get("request_hash"),
"plan_hash": plan_payload.get("plan_hash"),
"collection_hash": capsule["collection_hash"],
"capsule_hash": capsule["capsule_hash"],
}
items = [
*(("evidence", item) for item in evidence),
*(("omission", item) for item in omissions),
]
position = decode_cursor(
cursor,
kind="task-context.items",
binding=binding,
total_count=len(items),
)
page_evidence: list[object] = []
page_omissions: list[object] = []
consumed = 0
response_limited = False
maximum = self.project.descriptor.limits.max_tool_output_chars
def page_result() -> dict[str, object]:
pagination = page_receipt(
kind="task-context.items",
binding=binding,
position=position,
count=consumed,
limit=selected_limit,
total_count=len(items),
)
page_state = "incomplete" if response_limited else capsule.get("state")
page_summary = {
**cast(dict[str, object], capsule.get("summary", {})),
"page_evidence_count": len(page_evidence),
"page_omission_count": len(page_omissions),
"page_item_count": consumed,
}
page_hash = canonical_hash(
{
"capsule_hash": capsule["capsule_hash"],
"position": position,
"page_state": page_state,
"pagination": pagination,
"summary": page_summary,
"evidence": page_evidence,
"gaps": gaps,
"omissions": page_omissions,
}
)
page_capsule = {
**capsule,
"evidence": page_evidence,
"omissions": page_omissions,
"page_state": page_state,
"page_hash": page_hash,
"pagination": pagination,
"summary": page_summary,
}
return {
**result,
"capsule": page_capsule,
"next_cursor": pagination["next_cursor"],
"pagination": pagination,
}
for kind, item in items[position:]:
if consumed >= selected_limit:
break
destination = page_evidence if kind == "evidence" else page_omissions
destination.append(item)
consumed += 1
decorated = {
**page_result(),
"server_version": SERVER_VERSION,
"content_warning": CONTENT_WARNING,
"staleness": "current",
}
if self._encoded_length(decorated) <= maximum:
continue
destination.pop()
consumed -= 1
response_limited = True
if consumed == 0:
subject = "unknown"
if isinstance(item, Mapping):
item_payload = cast(Mapping[str, object], item)
candidate = item_payload.get("node_id") or item_payload.get("subject")
if isinstance(candidate, str) and candidate:
subject = candidate[:256]
page_omissions.append(
{
"code": "response_limit",
"subject": subject,
"detail_hash": canonical_hash(cast(object, item)),
}
)
consumed = 1
break
return page_result()
def _page_context_result( def _page_context_result(
self, self,
result: dict[str, object], result: dict[str, object],
@ -1151,6 +1379,26 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
return service.context(profile, budget, limit=limit, cursor=cursor) return service.context(profile, budget, limit=limit, cursor=cursor)
@server.tool(name="docforge_get_task_context")
def get_task_context(
task_kind: TaskKind,
task: str,
focus_node_id: str | None = None,
budget: int | None = None,
limit: int | None = None,
cursor: str | None = None,
) -> dict[str, Any]:
"""Return one bounded task-shaped context capsule with explicit evidence gaps."""
return service.task_context(
task_kind,
task,
focus_node_id=focus_node_id,
budget=budget,
limit=limit,
cursor=cursor,
)
@server.tool(name="docforge_validate_project") @server.tool(name="docforge_validate_project")
def validate_project() -> dict[str, Any]: def validate_project() -> dict[str, Any]:
"""Validate current canonical sources and graph without writing any project file.""" """Validate current canonical sources and graph without writing any project file."""
@ -1201,6 +1449,7 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
dependencies, dependencies,
impact, impact,
get_context, get_context,
get_task_context,
validate_project, validate_project,
render_status, render_status,
visualize, visualize,

671
src/docforge/retrieval.py Normal file
View file

@ -0,0 +1,671 @@
"""Versioned task-shaped retrieval plans and immutable context capsules."""
from __future__ import annotations
from dataclasses import dataclass, replace
from typing import Literal, cast
from .errors import DocForgeError
from .models import Edge, Node, ProjectDescriptor
from .pagination import canonical_hash
TaskKind = Literal[
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release",
]
TASK_KINDS: tuple[TaskKind, ...] = (
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release",
)
RelationCategory = Literal[
"structure",
"implementation",
"dependency",
"execution",
"data",
"evidence",
"context",
"unclassified",
]
BASE_RELATION_CATEGORIES: tuple[RelationCategory, ...] = (
"structure",
"implementation",
"dependency",
"execution",
"data",
"evidence",
"context",
)
RELATION_CATEGORIES: dict[RelationCategory, tuple[str, ...]] = {
"structure": ("contains", "defined_in", "defines", "owns"),
"implementation": (
"implemented_by",
"implements",
"inherits",
"inherits_from",
),
"dependency": ("depends_on", "imports"),
"execution": ("activates", "calls", "dispatches_to", "launches"),
"data": ("reads", "writes"),
"evidence": ("documents", "governs", "proves", "tested_by", "verifies"),
"context": ("relates_to",),
"unclassified": (),
}
TASK_REQUIREMENTS: dict[TaskKind, tuple[RelationCategory, ...]] = {
"change": ("dependency",),
"implementation": ("implementation",),
"failure": ("execution",),
"ownership": ("structure",),
"test": ("evidence",),
"operation": ("execution",),
"release": ("evidence",),
}
PLANNER_ID = "docforge.core.task-context"
PLANNER_VERSION = 1
MAX_TASK_EVIDENCE = 1_000
MAX_TASK_CANDIDATE_EDGES = 100_000
MAX_TASK_QUERY_CHARS = 10_000
PROVENANCE_LIMITATIONS = (
"evidence_type_unavailable",
"extractor_identity_unavailable",
"relationship_provenance_unavailable",
"observation_time_unavailable",
)
_RELATION_TO_CATEGORY = {
relation: category
for category, relations in RELATION_CATEGORIES.items()
for relation in relations
}
def relation_category(relation: str) -> RelationCategory:
"""Classify only versioned known aliases; preserve every other relation."""
return cast(RelationCategory, _RELATION_TO_CATEGORY.get(relation, "unclassified"))
@dataclass(frozen=True)
class RetrievalStepV1:
step_id: str
operation: Literal["exact", "search", "outgoing", "incoming", "metadata"]
relation_scope: Literal["none", "project_allowed"]
relation_set_hash: str | None
direction: Literal["none", "outgoing", "incoming"]
depth: int
limit: int
required: bool
evidence_role: str
def as_dict(self) -> dict[str, object]:
return {
"step_id": self.step_id,
"operation": self.operation,
"relation_scope": self.relation_scope,
"relation_set_hash": self.relation_set_hash,
"direction": self.direction,
"depth": self.depth,
"limit": self.limit,
"required": self.required,
"evidence_role": self.evidence_role,
}
@dataclass(frozen=True)
class RetrievalRequirementV1:
requirement_id: str
category: RelationCategory
def as_dict(self) -> dict[str, object]:
return {
"requirement_id": self.requirement_id,
"check": "selected_relation_category",
"category": self.category,
"required": True,
}
@dataclass(frozen=True)
class RetrievalPlanV1:
schema_version: Literal[1]
planner_id: str
planner_version: int
task_kind: TaskKind
task_query: str
focus_node_id: str | None
request_hash: str
effective_policy_hash: str
max_evidence: int
max_tokens: int
max_depth: int
max_candidate_edges: int
category_order: tuple[RelationCategory, ...]
steps: tuple[RetrievalStepV1, ...]
requirements: tuple[RetrievalRequirementV1, ...]
plan_hash: str
def as_dict(self) -> dict[str, object]:
return self.payload(include_hash=True)
def payload(self, *, include_hash: bool) -> dict[str, object]:
result: dict[str, object] = {
"schema_version": self.schema_version,
"planner": {
"id": self.planner_id,
"version": self.planner_version,
},
"task_kind": self.task_kind,
"request_hash": self.request_hash,
"effective_policy_hash": self.effective_policy_hash,
"focus_node_id": self.focus_node_id,
"limits": {
"max_evidence": self.max_evidence,
"max_tokens": self.max_tokens,
"max_depth": self.max_depth,
"max_candidate_edges": self.max_candidate_edges,
},
"category_order": list(self.category_order),
"steps": [step.as_dict() for step in self.steps],
"requirements": [requirement.as_dict() for requirement in self.requirements],
}
if include_hash:
result["plan_hash"] = self.plan_hash
return result
def build_retrieval_plan(
descriptor: ProjectDescriptor,
*,
task_kind: str,
task: str,
focus_node_id: str | None,
budget: int | None,
limit: int | None,
effective_policy: dict[str, object],
) -> RetrievalPlanV1:
"""Derive one fixed plan from bounded inputs rather than accepting caller operations."""
return _build_retrieval_plan(
descriptor,
task_kind=task_kind,
task=task,
focus_node_id=focus_node_id,
budget=budget,
limit=limit,
effective_policy_hash=canonical_hash(effective_policy),
)
def validate_retrieval_plan(
plan: RetrievalPlanV1,
descriptor: ProjectDescriptor,
) -> RetrievalPlanV1:
"""Reject forged, stale-shape, or internally inconsistent public plan objects."""
try:
expected = _build_retrieval_plan(
descriptor,
task_kind=plan.task_kind,
task=plan.task_query,
focus_node_id=plan.focus_node_id,
budget=plan.max_tokens,
limit=plan.max_evidence,
effective_policy_hash=plan.effective_policy_hash,
)
except (AttributeError, TypeError, DocForgeError) as error:
raise DocForgeError(
"invalid_retrieval_plan",
"Task retrieval plan is malformed or outside the fixed version-1 contract",
) from error
if plan != expected:
raise DocForgeError(
"invalid_retrieval_plan",
"Task retrieval plan does not match its fixed version-1 derivation",
)
return plan
def _build_retrieval_plan(
descriptor: ProjectDescriptor,
*,
task_kind: str,
task: str,
focus_node_id: str | None,
budget: int | None,
limit: int | None,
effective_policy_hash: str,
) -> RetrievalPlanV1:
if task_kind not in TASK_KINDS:
raise DocForgeError(
"invalid_task_kind",
"Task context kind is unsupported",
task_kind=task_kind,
allowed=list(TASK_KINDS),
)
selected_kind: TaskKind = task_kind # type: ignore[assignment]
normalized_task = task.strip()
if not normalized_task or len(normalized_task) > min(
descriptor.limits.max_query_chars,
MAX_TASK_QUERY_CHARS,
):
raise DocForgeError(
"invalid_task_focus",
"Task description is empty or exceeds the configured query limit",
)
if focus_node_id is not None and (not focus_node_id or len(focus_node_id) > 256):
raise DocForgeError("invalid_task_focus", "Task focus node ID is invalid")
selected_budget = _bounded_value(
budget,
default=min(8_000, descriptor.limits.max_context_tokens),
maximum=descriptor.limits.max_context_tokens,
code="invalid_budget",
)
selected_limit = _bounded_value(
limit,
default=min(20, descriptor.limits.max_results),
maximum=min(descriptor.limits.max_results, MAX_TASK_EVIDENCE),
code="invalid_limit",
)
if not _is_sha256(effective_policy_hash):
raise DocForgeError(
"invalid_retrieval_plan",
"Effective policy identity is not a SHA-256 value",
)
selected_depth = min(2, descriptor.limits.max_traversal_depth)
requirements = tuple(
RetrievalRequirementV1(
requirement_id=f"{selected_kind}.{category}",
category=category,
)
for category in TASK_REQUIREMENTS[selected_kind]
)
category_order: tuple[RelationCategory, ...] = (
*TASK_REQUIREMENTS[selected_kind],
*(
category
for category in BASE_RELATION_CATEGORIES
if category not in TASK_REQUIREMENTS[selected_kind]
),
"unclassified",
)
relation_set_hash = canonical_hash(sorted(descriptor.allowed_relations))
focus_operation: Literal["exact", "search"] = "exact" if focus_node_id else "search"
steps = (
RetrievalStepV1(
step_id="focus",
operation=focus_operation,
relation_scope="none",
relation_set_hash=None,
direction="none",
depth=0,
limit=1,
required=True,
evidence_role="focus",
),
RetrievalStepV1(
step_id="outgoing",
operation="outgoing",
relation_scope="project_allowed",
relation_set_hash=relation_set_hash,
direction="outgoing",
depth=selected_depth,
limit=selected_limit,
required=False,
evidence_role="related",
),
RetrievalStepV1(
step_id="incoming",
operation="incoming",
relation_scope="project_allowed",
relation_set_hash=relation_set_hash,
direction="incoming",
depth=selected_depth,
limit=selected_limit,
required=False,
evidence_role="related",
),
RetrievalStepV1(
step_id="metadata",
operation="metadata",
relation_scope="none",
relation_set_hash=None,
direction="none",
depth=0,
limit=selected_limit,
required=True,
evidence_role="provenance",
),
)
request_hash = canonical_hash(
{
"task_kind": selected_kind,
"task": normalized_task,
"focus_node_id": focus_node_id,
"budget": selected_budget,
"limit": selected_limit,
}
)
placeholder = RetrievalPlanV1(
schema_version=1,
planner_id=PLANNER_ID,
planner_version=PLANNER_VERSION,
task_kind=selected_kind,
task_query=normalized_task,
focus_node_id=focus_node_id,
request_hash=request_hash,
effective_policy_hash=effective_policy_hash,
max_evidence=selected_limit,
max_tokens=selected_budget,
max_depth=selected_depth,
max_candidate_edges=min(
(selected_limit + 1) ** 2,
MAX_TASK_CANDIDATE_EDGES,
),
category_order=category_order,
steps=steps,
requirements=requirements,
plan_hash="",
)
return replace(
placeholder,
plan_hash=canonical_hash(placeholder.payload(include_hash=False)),
)
@dataclass(frozen=True)
class CapsuleRelationshipV1:
source_id: str
relation: str
target_id: str
direction: Literal["outgoing", "incoming"]
category: RelationCategory
def as_dict(self) -> dict[str, object]:
return {
"source_id": self.source_id,
"relation": self.relation,
"target_id": self.target_id,
"direction": self.direction,
"category": self.category,
"provenance": "validated_graph_edge_without_source_provenance",
}
@dataclass(frozen=True)
class CapsuleEvidenceV1:
evidence_hash: str
role: Literal["focus", "related"]
reason_code: Literal["exact_focus", "lexical_focus", "relationship_path"]
node: Node
depth: int
relationship_path: tuple[CapsuleRelationshipV1, ...]
relationship_reasons: tuple[CapsuleRelationshipV1, ...]
estimated_tokens: int
def as_dict(self) -> dict[str, object]:
result = self.payload()
return {"evidence_hash": self.evidence_hash, **result}
def payload(self) -> dict[str, object]:
result: dict[str, object] = {
"role": self.role,
"reason_code": self.reason_code,
"node_id": self.node.node_id,
"title": self.node.title,
"family": self.node.family,
"authority": self.node.authority,
"status": self.node.status,
"tags": list(self.node.tags),
"summary": self.node.summary,
"text": _node_text(self.node),
"estimated_tokens": self.estimated_tokens,
"source": {
"path": self.node.source_path,
"anchor": self.node.source_anchor,
"content_hash": self.node.content_hash,
},
"depth": self.depth,
"relationship_path": [
relationship.as_dict() for relationship in self.relationship_path
],
"relationship_reasons": [
relationship.as_dict() for relationship in self.relationship_reasons
],
"provenance_limitations": list(PROVENANCE_LIMITATIONS),
}
return result
def capsule_evidence(
*,
role: Literal["focus", "related"],
reason_code: Literal["exact_focus", "lexical_focus", "relationship_path"],
node: Node,
depth: int,
relationship_path: tuple[CapsuleRelationshipV1, ...],
relationship_reasons: tuple[CapsuleRelationshipV1, ...],
) -> CapsuleEvidenceV1:
tokens = estimate_tokens(_node_text(node))
placeholder = CapsuleEvidenceV1(
evidence_hash="",
role=role,
reason_code=reason_code,
node=node,
depth=depth,
relationship_path=relationship_path,
relationship_reasons=relationship_reasons,
estimated_tokens=tokens,
)
return replace(
placeholder,
evidence_hash=canonical_hash(placeholder.payload()),
)
@dataclass(frozen=True)
class EvidenceGapV1:
code: Literal[
"focus_not_found",
"focus_ambiguous",
"category_not_declared",
"no_selected_evidence",
"evidence_incomplete",
"unclassified_relation",
]
requirement_id: str
category: RelationCategory | None
state: Literal["missing", "incomplete", "blocked", "limitation"]
check_complete: bool
detail: str
def as_dict(self) -> dict[str, object]:
return {
"code": self.code,
"requirement_id": self.requirement_id,
"category": self.category,
"state": self.state,
"check_complete": self.check_complete,
"detail": self.detail,
}
@dataclass(frozen=True)
class CapsuleOmissionV1:
code: Literal[
"result_limit",
"token_budget",
"response_limit",
"edge_examination_limit",
"unclassified_relation_limit",
]
subject: str
detail_hash: str
def as_dict(self) -> dict[str, str]:
return {
"code": self.code,
"subject": self.subject,
"detail_hash": self.detail_hash,
}
@dataclass(frozen=True)
class ContextCapsuleV1:
schema_version: Literal[1]
state: Literal["complete", "incomplete", "blocked"]
task_kind: TaskKind
generation: tuple[tuple[str, object], ...]
plan: RetrievalPlanV1
focus_state: Literal["resolved", "not_found", "ambiguous"]
focus_node_id: str | None
focus_candidate_count: int
evidence: tuple[CapsuleEvidenceV1, ...]
gaps: tuple[EvidenceGapV1, ...]
omissions: tuple[CapsuleOmissionV1, ...]
selected_count: int
examined_edge_count: int
estimated_tokens: int
unclassified_relations: tuple[str, ...]
collection_hash: str
capsule_hash: str
def as_dict(self) -> dict[str, object]:
return self.payload(include_hashes=True)
def payload(self, *, include_hashes: bool) -> dict[str, object]:
evidence = [item.as_dict() for item in self.evidence]
gaps = [gap.as_dict() for gap in self.gaps]
omissions = [omission.as_dict() for omission in self.omissions]
result: dict[str, object] = {
"schema_version": self.schema_version,
"state": self.state,
"task_kind": self.task_kind,
"generation": dict(self.generation),
"plan": self.plan.as_dict(),
"focus": {
"state": self.focus_state,
"node_id": self.focus_node_id,
"candidate_count": self.focus_candidate_count,
},
"evidence": evidence,
"gaps": gaps,
"omissions": omissions,
"summary": {
"evidence_count": len(evidence),
"gap_count": len(gaps),
"omission_count": len(omissions),
"selected_count": self.selected_count,
"examined_edge_count": self.examined_edge_count,
"estimated_tokens": self.estimated_tokens,
"unclassified_relations": list(self.unclassified_relations),
},
}
if include_hashes:
result["collection_hash"] = self.collection_hash
result["capsule_hash"] = self.capsule_hash
return result
def finalize_capsule(
*,
state: Literal["complete", "incomplete", "blocked"],
task_kind: TaskKind,
generation: dict[str, object],
plan: RetrievalPlanV1,
focus_state: Literal["resolved", "not_found", "ambiguous"],
focus_node_id: str | None,
focus_candidate_count: int,
evidence: tuple[CapsuleEvidenceV1, ...],
gaps: tuple[EvidenceGapV1, ...],
omissions: tuple[CapsuleOmissionV1, ...],
selected_count: int,
examined_edge_count: int,
estimated_tokens: int,
unclassified_relations: tuple[str, ...],
) -> ContextCapsuleV1:
collection_hash = canonical_hash(
{
"generation": generation,
"plan_hash": plan.plan_hash,
"evidence": [item.evidence_hash for item in evidence],
"gaps": [gap.as_dict() for gap in gaps],
"omissions": [omission.as_dict() for omission in omissions],
}
)
placeholder = ContextCapsuleV1(
schema_version=1,
state=state,
task_kind=task_kind,
generation=tuple(generation.items()),
plan=plan,
focus_state=focus_state,
focus_node_id=focus_node_id,
focus_candidate_count=focus_candidate_count,
evidence=evidence,
gaps=gaps,
omissions=omissions,
selected_count=selected_count,
examined_edge_count=examined_edge_count,
estimated_tokens=estimated_tokens,
unclassified_relations=unclassified_relations,
collection_hash=collection_hash,
capsule_hash="",
)
capsule_hash = canonical_hash(
{
**placeholder.payload(include_hashes=False),
"collection_hash": collection_hash,
}
)
return replace(placeholder, capsule_hash=capsule_hash)
def estimate_tokens(text: str) -> int:
return max(1, (len(text) + 3) // 4)
def edge_tuple(edge: Edge) -> tuple[str, str, str]:
return edge.source_id, edge.relation, edge.target_id
def _node_text(node: Node) -> str:
return (
f"ID: {node.node_id}\nTitle: {node.title}\nFamily: {node.family}\n"
f"Authority: {node.authority}\nStatus: {node.status}\nSource: {node.source_path}\n"
f"Summary: {node.summary}\n\n{node.content}"
)
def _bounded_value(
value: int | None,
*,
default: int,
maximum: int,
code: str,
) -> int:
selected = default if value is None else value
if type(selected) is not int or selected < 1 or selected > maximum:
raise DocForgeError(code, "Task context limit is outside the configured range")
return selected
def _is_sha256(value: object) -> bool:
return (
isinstance(value, str)
and len(value) == 64
and all(character in "0123456789abcdef" for character in value)
)

1
src/docforge/telemetry.py
View file

@ -91,6 +91,7 @@ OPERATION_NAMES = frozenset(
"mcp.dependencies", "mcp.dependencies",
"mcp.impact", "mcp.impact",
"mcp.context", "mcp.context",
"mcp.task_context",
"mcp.validate_project", "mcp.validate_project",
"mcp.render_status", "mcp.render_status",
"mcp.visualize", "mcp.visualize",

19
tests/test_adapter_contract.py
View file

@ -55,8 +55,10 @@ from docforge.visualization import VisualizationIndexSnapshot
class Loader: class Loader:
def __init__(self, projection: AdapterProjection) -> None: def __init__(self, projection: AdapterProjection) -> None:
self.projection = projection self.projection = projection
self.load_calls = 0
def load_projection(self) -> AdapterProjection: def load_projection(self) -> AdapterProjection:
self.load_calls += 1
return self.projection return self.projection
@ -866,8 +868,9 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
root = Path(directory).resolve() root = Path(directory).resolve()
fixture = AdapterContractTests() fixture = AdapterContractTests()
loader = Loader(fixture.projection(root))
project = AdapterProject( project = AdapterProject(
Loader(fixture.projection(root)), loader,
cache_root=root / ".cache" / "adapter-read-only", cache_root=root / ".cache" / "adapter-read-only",
) )
index = ProjectIndex(project) index = ProjectIndex(project)
@ -903,6 +906,14 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
context = await session.call_tool( context = await session.call_tool(
"docforge_get_context", {"profile": "fixture", "budget": 321} "docforge_get_context", {"profile": "fixture", "budget": 321}
) )
load_calls = loader.load_calls
task_context = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Do not load the legacy projection for this capability error",
},
)
self.assertEqual(READ_TOOLS, tuple(tool.name for tool in tools.tools)) self.assertEqual(READ_TOOLS, tuple(tool.name for tool in tools.tools))
self.assertEqual("adapter-fixture", info.structuredContent["project_id"]) self.assertEqual("adapter-fixture", info.structuredContent["project_id"])
@ -915,6 +926,12 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertFalse(contract.structuredContent["isolated_changeset_writes_allowed"]) self.assertFalse(contract.structuredContent["isolated_changeset_writes_allowed"])
self.assertEqual("fixture", context.structuredContent["profile"]) self.assertEqual("fixture", context.structuredContent["profile"])
self.assertEqual([("fixture", 321)], calls) self.assertEqual([("fixture", 321)], calls)
self.assertEqual("error", task_context.structuredContent["status"])
self.assertEqual(
"task_context_unavailable",
task_context.structuredContent["error"]["code"],
)
self.assertEqual(load_calls, loader.load_calls)
self.assertFalse(project.descriptor.changeset_root.exists()) self.assertFalse(project.descriptor.changeset_root.exists())
async def test_adapter_project_proposals_require_explicit_policy_and_stay_isolated( async def test_adapter_project_proposals_require_explicit_policy_and_stay_isolated(

356
tests/test_mcp_server.py
View file

@ -12,6 +12,7 @@ from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from unittest import mock from unittest import mock
from jsonschema import Draft202012Validator
from mcp import ClientSession, StdioServerParameters from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client from mcp.client.stdio import stdio_client
from mcp.shared.memory import create_connected_server_and_client_session from mcp.shared.memory import create_connected_server_and_client_session
@ -34,6 +35,9 @@ from docforge.viewer_manager import ViewerManager
ROOT = Path(__file__).resolve().parents[1] ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures" FIXTURES = ROOT / "tests" / "fixtures"
CAPSULE_SCHEMA = json.loads(
(ROOT / "schemas" / "context-capsule.schema.json").read_text(encoding="utf-8")
)
class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase): class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
@ -85,6 +89,7 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertNotIn("limit", tools[name].inputSchema.get("required", [])) self.assertNotIn("limit", tools[name].inputSchema.get("required", []))
for name in ( for name in (
"docforge_get_context", "docforge_get_context",
"docforge_get_task_context",
"docforge_list_changesets", "docforge_list_changesets",
"docforge_get_changeset", "docforge_get_changeset",
"docforge_validate_changeset", "docforge_validate_changeset",
@ -93,6 +98,22 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
for field in ("limit", "cursor"): for field in ("limit", "cursor"):
self.assertIn(field, tools[name].inputSchema["properties"]) self.assertIn(field, tools[name].inputSchema["properties"])
self.assertNotIn(field, tools[name].inputSchema.get("required", [])) self.assertNotIn(field, tools[name].inputSchema.get("required", []))
self.assertEqual(
{"task_kind", "task"},
set(tools["docforge_get_task_context"].inputSchema["required"]),
)
self.assertEqual(
[
"change",
"implementation",
"failure",
"ownership",
"test",
"operation",
"release",
],
tools["docforge_get_task_context"].inputSchema["properties"]["task_kind"]["enum"],
)
self.assertIn( self.assertIn(
"deep", "deep",
tools["docforge_render_status"].inputSchema["properties"], tools["docforge_render_status"].inputSchema["properties"],
@ -160,6 +181,316 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertEqual(0, diagnostics["counters"]["project_loads"]) self.assertEqual(0, diagnostics["counters"]["project_loads"])
self.assertEqual(0, diagnostics["counters"]["source_files_parsed"]) self.assertEqual(0, diagnostics["counters"]["source_files_parsed"])
async def test_task_context_is_hash_stable_paged_and_generation_bound(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
ProjectIndex(Project.open(root)).build()
async with create_connected_server_and_client_session(
create_server(root, capability_mode="read", diagnostics=True),
raise_exceptions=True,
) as session:
first = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"limit": 1,
},
)
first_capsule = first.structuredContent["capsule"]
Draft202012Validator(CAPSULE_SCHEMA).validate(first_capsule)
cursor = first_capsule["pagination"]["next_cursor"]
hashes = {
first_capsule["capsule_hash"],
first_capsule["collection_hash"],
first_capsule["plan"]["plan_hash"],
}
evidence_ids = [item["node_id"] for item in first_capsule["evidence"]]
while cursor is not None:
page = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"limit": 2,
"cursor": cursor,
},
)
capsule = page.structuredContent["capsule"]
self.assertEqual(first_capsule["capsule_hash"], capsule["capsule_hash"])
self.assertEqual(
first_capsule["collection_hash"],
capsule["collection_hash"],
)
self.assertEqual(
first_capsule["plan"]["plan_hash"],
capsule["plan"]["plan_hash"],
)
evidence_ids.extend(item["node_id"] for item in capsule["evidence"])
cursor = capsule["pagination"]["next_cursor"]
self.assertEqual(
["guide.workflow", "guide.foundation", "proof.validation"],
evidence_ids,
)
self.assertEqual(3, len(hashes))
self.assertEqual(
"mcp.task_context",
first.structuredContent["diagnostics"]["operation"],
)
counters = first.structuredContent["diagnostics"]["counters"]
self.assertEqual(0, counters["project_loads"])
self.assertEqual(0, counters["source_files_parsed"])
self.assertEqual(0, counters["adapter_projection_loads"])
self.assertEqual(0, counters["adapter_source_extractions"])
self.assertEqual(0, counters["index_builds"])
self.assertLessEqual(
len(json.dumps(first.structuredContent, separators=(",", ":"))),
Project.open(root).descriptor.limits.max_tool_output_chars,
)
for changed_arguments in (
{"task": "A different task"},
{"task_kind": "failure"},
{"focus_node_id": "guide.foundation"},
{"budget": 100},
):
arguments = {
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"limit": 1,
"cursor": first_capsule["pagination"]["next_cursor"],
**changed_arguments,
}
changed_cursor = await session.call_tool(
"docforge_get_task_context",
arguments,
)
self.assertEqual(
"stale_cursor",
changed_cursor.structuredContent["error"]["code"],
)
different_policy = DocForgeService(
Project.open(root),
capability_mode_name="proposal",
).task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
limit=1,
cursor=first_capsule["pagination"]["next_cursor"],
)
self.assertEqual("stale_cursor", different_policy["error"]["code"])
changed = root / "docs/content/foundation.md"
changed.write_text(
changed.read_text(encoding="utf-8") + "\nNew generation.\n",
encoding="utf-8",
)
stale = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"limit": 1,
"cursor": first_capsule["pagination"]["next_cursor"],
},
)
self.assertEqual("error", stale.structuredContent["status"])
self.assertEqual("stale_cursor", stale.structuredContent["error"]["code"])
async def test_custom_context_policy_does_not_silently_gain_task_planning(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
ProjectIndex(project).build()
service = DocForgeService(
project,
context_provider=lambda index, profile, budget: {
"profile": profile,
"budget": budget,
"entries": [],
"omissions": [],
},
capability_mode_name="read",
)
async with create_connected_server_and_client_session(
_create_bound_server(service, read_only=True),
raise_exceptions=True,
) as session:
bootstrap = await session.call_tool("docforge_bootstrap", {})
with (
mock.patch.object(
service.project,
"load",
side_effect=AssertionError("capability errors must not load"),
),
mock.patch.object(
service.index,
"check",
side_effect=AssertionError("capability errors must not check"),
),
mock.patch.object(
service.index,
"build",
side_effect=AssertionError("capability errors must not build"),
),
mock.patch.object(
service.index,
"synchronize",
side_effect=AssertionError("capability errors must not synchronize"),
),
):
result = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Do not widen the adapter context policy",
},
)
self.assertEqual(
"docforge_get_context",
bootstrap.structuredContent["session_contract"]["recommended_first_operation"][
"tool"
],
)
self.assertFalse(bootstrap.structuredContent["capabilities"]["task_context"]["enabled"])
self.assertEqual("error", result.structuredContent["status"])
self.assertEqual(
"task_context_unavailable",
result.structuredContent["error"]["code"],
)
def test_task_context_page_hash_binds_final_page_envelope(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8").replace(
"max_context_tokens = 2000",
"max_context_tokens = 2000\nmax_tool_output_chars = 8000",
),
encoding="utf-8",
)
project = Project.open(root)
ProjectIndex(project).build()
service = DocForgeService(project, capability_mode_name="read")
one = service.task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
limit=1,
)
two = service.task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
limit=2,
)
one_capsule = one["capsule"]
two_capsule = two["capsule"]
self.assertEqual(
["guide.workflow"],
[item["node_id"] for item in one_capsule["evidence"]],
)
self.assertEqual(
["guide.workflow"],
[item["node_id"] for item in two_capsule["evidence"]],
)
self.assertEqual("complete", one_capsule["page_state"])
self.assertEqual("incomplete", two_capsule["page_state"])
self.assertNotEqual(one_capsule["page_hash"], two_capsule["page_hash"])
self.assertNotEqual(one_capsule["pagination"], two_capsule["pagination"])
self.assertLessEqual(
len(json.dumps(one, sort_keys=True, separators=(",", ":"))),
8_000,
)
self.assertLessEqual(
len(json.dumps(two, sort_keys=True, separators=(",", ":"))),
8_000,
)
def test_task_context_default_page_clamps_to_small_project_limit(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8").replace(
"max_results = 20",
"max_results = 2",
),
encoding="utf-8",
)
project = Project.open(root)
ProjectIndex(project).build()
result = DocForgeService(
project,
capability_mode_name="read",
).task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
)
self.assertEqual(2, result["pagination"]["limit"])
self.assertLessEqual(result["pagination"]["returned_count"], 2)
def test_oversized_task_evidence_advances_once_as_an_omission(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8").replace(
"max_context_tokens = 2000",
"max_context_tokens = 50000\nmax_tool_output_chars = 8000",
),
encoding="utf-8",
)
workflow = root / "docs" / "content" / "workflow.md"
workflow.write_text(
workflow.read_text(encoding="utf-8") + "\n" + ("large evidence " * 2_000),
encoding="utf-8",
)
project = Project.open(root)
ProjectIndex(project).build()
service = DocForgeService(project, capability_mode_name="read")
first = service.task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
budget=50_000,
limit=1,
)
first_capsule = first["capsule"]
self.assertEqual([], first_capsule["evidence"])
self.assertEqual("response_limit", first_capsule["omissions"][0]["code"])
self.assertEqual("guide.workflow", first_capsule["omissions"][0]["subject"])
self.assertEqual(1, first_capsule["pagination"]["returned_count"])
self.assertTrue(first_capsule["pagination"]["has_more"])
second = service.task_context(
"change",
"Change the editing workflow",
focus_node_id="guide.workflow",
budget=50_000,
limit=1,
cursor=first_capsule["pagination"]["next_cursor"],
)
self.assertNotEqual(first_capsule["page_hash"], second["capsule"]["page_hash"])
self.assertNotIn(
"guide.workflow",
[
item.get("node_id", item.get("subject"))
for item in (
*second["capsule"]["evidence"],
*second["capsule"]["omissions"],
)
],
)
async def test_context_pagination_is_complete_and_stale_cursors_fail_closed(self) -> None: async def test_context_pagination_is_complete_and_stale_cursors_fail_closed(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory)) root = self.copy_fixture("alpha", Path(directory))
@ -304,6 +635,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
logic = await session.call_tool( logic = await session.call_tool(
"docforge_get_logic", {"owner_node_id": "guide.workflow"} "docforge_get_logic", {"owner_node_id": "guide.workflow"}
) )
task_context = await session.call_tool(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow without AST analysis",
"focus_node_id": "guide.workflow",
},
)
policy = bootstrap.structuredContent["adapter_policy"] policy = bootstrap.structuredContent["adapter_policy"]
self.assertEqual("preserve-no-ast", policy["mode"]) self.assertEqual("preserve-no-ast", policy["mode"])
@ -329,6 +668,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
"adapter_policy_forbids_logic", "adapter_policy_forbids_logic",
logic.structuredContent["error"]["code"], logic.structuredContent["error"]["code"],
) )
self.assertEqual("ok", task_context.structuredContent["status"])
self.assertNotIn(
"logic",
{
step["operation"]
for step in task_context.structuredContent["capsule"]["plan"]["steps"]
},
)
async def test_every_read_tool_returns_scoped_structured_results(self) -> None: async def test_every_read_tool_returns_scoped_structured_results(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
@ -358,6 +705,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
("docforge_visualization_status", {}), ("docforge_visualization_status", {}),
("docforge_bootstrap", {}), ("docforge_bootstrap", {}),
("docforge_sync", {}), ("docforge_sync", {}),
(
"docforge_get_task_context",
{
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
},
),
) )
with self.running_manager(Path(directory) / "viewer-manager.json"): with self.running_manager(Path(directory) / "viewer-manager.json"):
service = DocForgeService(Project.open(root)) service = DocForgeService(Project.open(root))
@ -413,6 +768,7 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
context = results[9].structuredContent context = results[9].structuredContent
self.assertLessEqual(context["estimated_tokens"], 180) self.assertLessEqual(context["estimated_tokens"], 180)
self.assertTrue(context["omissions"]) self.assertTrue(context["omissions"])
self.assertEqual("complete", results[17].structuredContent["capsule"]["state"])
async def test_invalid_traversal_limit_is_a_structured_domain_error(self) -> None: async def test_invalid_traversal_limit_is_a_structured_domain_error(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:

2
tests/test_policy.py
View file

@ -147,7 +147,7 @@ class EffectivePolicyTests(unittest.TestCase):
self.assertNotIn("docforge_register_changes", result["recommended_workflow"]) self.assertNotIn("docforge_register_changes", result["recommended_workflow"])
self.assertNotIn("docforge_apply_changeset", result["recommended_workflow"]) self.assertNotIn("docforge_apply_changeset", result["recommended_workflow"])
self.assertEqual( self.assertEqual(
"docforge_get_context", "docforge_get_task_context",
result["session_contract"]["recommended_first_operation"]["tool"], result["session_contract"]["recommended_first_operation"]["tool"],
) )

8
tests/test_public_contract.py
View file

@ -82,6 +82,13 @@ PUBLIC_IMPORTS = {
"capability_mode", "capability_mode",
"compose_effective_policy", "compose_effective_policy",
), ),
"docforge.retrieval": (
"ContextCapsuleV1",
"RetrievalPlanV1",
"build_retrieval_plan",
"relation_category",
"validate_retrieval_plan",
),
"docforge.render_contract": ( "docforge.render_contract": (
"GenericHtmlRenderer", "GenericHtmlRenderer",
"PreparedRender", "PreparedRender",
@ -132,6 +139,7 @@ EXPECTED_MCP_TOOLS = {
"docforge_get_changeset", "docforge_get_changeset",
"docforge_get_changeset_diff", "docforge_get_changeset_diff",
"docforge_get_context", "docforge_get_context",
"docforge_get_task_context",
"docforge_get_contract", "docforge_get_contract",
"docforge_get_logic", "docforge_get_logic",
"docforge_get_node", "docforge_get_node",

464
tests/test_retrieval.py Normal file
View file

@ -0,0 +1,464 @@
from __future__ import annotations
import json
import shutil
import tempfile
import unittest
from dataclasses import replace
from pathlib import Path
from unittest import mock
from jsonschema import Draft202012Validator
from docforge.errors import DocForgeError
from docforge.index import ProjectIndex
from docforge.models import ProjectState
from docforge.policy import compose_effective_policy
from docforge.project import Project
from docforge.retrieval import (
BASE_RELATION_CATEGORIES,
MAX_TASK_CANDIDATE_EDGES,
MAX_TASK_EVIDENCE,
RELATION_CATEGORIES,
TASK_KINDS,
TASK_REQUIREMENTS,
build_retrieval_plan,
relation_category,
)
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
CAPSULE_SCHEMA = json.loads(
(ROOT / "schemas" / "context-capsule.schema.json").read_text(encoding="utf-8")
)
class TaskRetrievalTests(unittest.TestCase):
def copy_fixture(self, name: str, destination: Path) -> Path:
root = destination / name
shutil.copytree(FIXTURES / name, root)
return root
@staticmethod
def effective_policy() -> dict[str, object]:
return compose_effective_policy(
selected_mode="read",
capability_source="explicit",
no_ast=False,
diagnostics=False,
render_configured=True,
application_enabled=False,
).as_dict()
def plan(
self,
project: Project,
*,
task_kind: str = "change",
task: str = "Change the editing workflow",
focus_node_id: str | None = "guide.workflow",
budget: int | None = None,
limit: int | None = None,
):
return build_retrieval_plan(
project.descriptor,
task_kind=task_kind,
task=task,
focus_node_id=focus_node_id,
budget=budget,
limit=limit,
effective_policy=self.effective_policy(),
)
def test_every_task_plan_is_closed_deterministic_and_hash_bound(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
hashes = {}
relation_hashes = set()
for task_kind in TASK_KINDS:
first = self.plan(project, task_kind=task_kind)
second = self.plan(project, task_kind=task_kind)
self.assertEqual(first, second)
self.assertEqual(TASK_REQUIREMENTS[task_kind], first.category_order[:1])
self.assertEqual(
{
*BASE_RELATION_CATEGORIES,
"unclassified",
},
set(first.category_order),
)
self.assertEqual(64, len(first.plan_hash))
relation_hashes.update(
step.relation_set_hash
for step in first.steps
if step.relation_scope == "project_allowed"
)
hashes[task_kind] = first.plan_hash
self.assertEqual(len(TASK_KINDS), len(set(hashes.values())))
self.assertEqual(1, len(relation_hashes))
self.assertEqual(
{
"change": ("dependency",),
"implementation": ("implementation",),
"failure": ("execution",),
"ownership": ("structure",),
"test": ("evidence",),
"operation": ("execution",),
"release": ("evidence",),
},
TASK_REQUIREMENTS,
)
mapped = [
relation for relations in RELATION_CATEGORIES.values() for relation in relations
]
self.assertEqual(len(mapped), len(set(mapped)))
self.assertEqual(
{
"structure": ("contains", "defined_in", "defines", "owns"),
"implementation": (
"implemented_by",
"implements",
"inherits",
"inherits_from",
),
"dependency": ("depends_on", "imports"),
"execution": (
"activates",
"calls",
"dispatches_to",
"launches",
),
"data": ("reads", "writes"),
"evidence": (
"documents",
"governs",
"proves",
"tested_by",
"verifies",
),
"context": ("relates_to",),
"unclassified": (),
},
RELATION_CATEGORIES,
)
self.assertEqual("dependency", relation_category("depends_on"))
self.assertEqual("evidence", relation_category("proves"))
self.assertEqual("unclassified", relation_category("owns_database"))
self.assertEqual("unclassified", relation_category("when_true"))
def test_exact_capsule_is_schema_valid_stable_and_explainable(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
index = ProjectIndex(project)
index.build()
plan = self.plan(project)
first = index.task_context(plan)
second = index.task_context(plan)
self.assertEqual(first, second)
capsule = first["capsule"]
Draft202012Validator(CAPSULE_SCHEMA).validate(capsule)
self.assertEqual("complete", capsule["state"])
self.assertEqual("resolved", capsule["focus"]["state"])
self.assertEqual("guide.workflow", capsule["focus"]["node_id"])
self.assertEqual(
["guide.workflow", "guide.foundation", "proof.validation"],
[item["node_id"] for item in capsule["evidence"]],
)
dependency = capsule["evidence"][1]["relationship_path"][0]
self.assertEqual("depends_on", dependency["relation"])
self.assertEqual("dependency", dependency["category"])
self.assertEqual("outgoing", dependency["direction"])
workflow_reasons = capsule["evidence"][0]["relationship_reasons"]
self.assertIn(
("depends_on", "outgoing"),
{
(relationship["relation"], relationship["direction"])
for relationship in workflow_reasons
},
)
self.assertIn(
("proves", "incoming"),
{
(relationship["relation"], relationship["direction"])
for relationship in workflow_reasons
},
)
foundation_reason = capsule["evidence"][1]["relationship_reasons"][0]
self.assertEqual("depends_on", foundation_reason["relation"])
self.assertEqual("incoming", foundation_reason["direction"])
self.assertNotIn(
"no_selected_evidence",
[gap["code"] for gap in capsule["gaps"]],
)
self.assertEqual(64, len(capsule["collection_hash"]))
self.assertEqual(64, len(capsule["capsule_hash"]))
def test_gaps_distinguish_undeclared_complete_and_incomplete_checks(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
undeclared = index.task_context(self.plan(project, task_kind="implementation"))[
"capsule"
]
self.assertIn(
"category_not_declared",
[gap["code"] for gap in undeclared["gaps"]],
)
descriptor = root / ".docforge" / "project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8").replace(
'"returns_to"]',
'"returns_to", "calls"]',
),
encoding="utf-8",
)
declared_project = Project.open(root)
declared_index = ProjectIndex(declared_project)
declared_index.build()
missing = declared_index.task_context(
self.plan(
declared_project,
task_kind="failure",
focus_node_id="guide.foundation",
)
)["capsule"]
self.assertIn(
"no_selected_evidence",
[gap["code"] for gap in missing["gaps"]],
)
incomplete = declared_index.task_context(self.plan(declared_project, budget=1))[
"capsule"
]
self.assertEqual("incomplete", incomplete["state"])
self.assertTrue(incomplete["omissions"])
self.assertIn(
"evidence_incomplete",
[gap["code"] for gap in incomplete["gaps"]],
)
self.assertNotIn(
"no_selected_evidence",
[gap["code"] for gap in incomplete["gaps"]],
)
def test_unclassified_relation_is_preserved_without_guessed_semantics(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("awesome-ski-game", Path(directory)))
index = ProjectIndex(project)
index.build()
plan = build_retrieval_plan(
project.descriptor,
task_kind="change",
task="Change the first descent session",
focus_node_id="session.first-descent",
budget=None,
limit=None,
effective_policy=self.effective_policy(),
)
capsule = index.task_context(plan)["capsule"]
self.assertIn("informs", capsule["summary"]["unclassified_relations"])
relation = next(
relationship
for item in capsule["evidence"]
for relationship in item["relationship_reasons"]
if relationship["relation"] == "informs"
)
self.assertEqual("unclassified", relation["category"])
self.assertIn(
"unclassified_relation",
[gap["code"] for gap in capsule["gaps"]],
)
def test_plan_is_compact_at_large_valid_relation_scale_and_limits_are_fixed(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
relation_names = tuple(f"relation-{position:05d}" for position in range(33_005))
descriptor = replace(
project.descriptor,
allowed_relations=relation_names,
limits=replace(
project.descriptor.limits,
max_results=2**63,
),
)
plan = build_retrieval_plan(
descriptor,
task_kind="change",
task="Exercise a very large valid relation policy",
focus_node_id="guide.workflow",
budget=None,
limit=None,
effective_policy=self.effective_policy(),
)
self.assertLessEqual(plan.max_evidence, MAX_TASK_EVIDENCE)
self.assertLessEqual(plan.max_candidate_edges, MAX_TASK_CANDIDATE_EDGES)
traversal = [step for step in plan.steps if step.operation in {"outgoing", "incoming"}]
self.assertEqual(2, len(traversal))
self.assertTrue(all(step.relation_scope == "project_allowed" for step in traversal))
self.assertTrue(all(len(step.relation_set_hash or "") == 64 for step in traversal))
self.assertLess(len(json.dumps(plan.as_dict())), 5_000)
def test_executor_rejects_tampered_public_plan_objects(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
index = ProjectIndex(project)
index.build()
plan = self.plan(project)
tampered = (
replace(plan, steps=plan.steps[:-1]),
replace(plan, requirements=()),
replace(plan, task_query="Different task"),
replace(plan, category_order=tuple(reversed(plan.category_order))),
replace(plan, max_evidence=plan.max_evidence + 1),
replace(plan, effective_policy_hash="0" * 64),
replace(plan, plan_hash="0" * 64),
)
for candidate in tampered:
with self.subTest(candidate=candidate):
with self.assertRaises(DocForgeError) as invalid:
index.task_context(candidate)
self.assertEqual("invalid_retrieval_plan", invalid.exception.code)
def test_work_and_unclassified_limits_are_explicit_and_bounded(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
relation_names = [f"relation-{position:03d}" for position in range(450)]
descriptor = root / ".docforge" / "project.toml"
raw_descriptor = descriptor.read_text(encoding="utf-8")
raw_relations = ", ".join(
json.dumps(relation)
for relation in (
"depends_on",
"proves",
"supersedes",
"relates_to",
"returns_to",
*relation_names,
)
)
descriptor.write_text(
raw_descriptor.replace(
'"depends_on", "proves", "supersedes", "relates_to", "returns_to"',
raw_relations,
),
encoding="utf-8",
)
workflow = root / "docs" / "content" / "workflow.md"
raw_workflow = workflow.read_text(encoding="utf-8")
relationships = "".join(
f'{relation} = ["guide.foundation"]\n' for relation in relation_names
)
workflow.write_text(
raw_workflow.replace("+++\n\nEditors", f"{relationships}+++\n\nEditors", 1),
encoding="utf-8",
)
project = Project.open(root)
index = ProjectIndex(project)
index.build()
capsule = index.task_context(self.plan(project))["capsule"]
Draft202012Validator(CAPSULE_SCHEMA).validate(capsule)
omission_codes = {omission["code"] for omission in capsule["omissions"]}
self.assertIn("edge_examination_limit", omission_codes)
self.assertIn("unclassified_relation_limit", omission_codes)
self.assertEqual(
project.descriptor.limits.max_results,
len(capsule["summary"]["unclassified_relations"]),
)
self.assertLessEqual(
capsule["summary"]["examined_edge_count"],
MAX_TASK_CANDIDATE_EDGES,
)
def test_lexical_focus_blocks_missing_and_ambiguous_selection(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
missing = index.task_context(
self.plan(
project,
task="Words absent from every indexed node",
focus_node_id=None,
)
)["capsule"]
self.assertEqual("blocked", missing["state"])
self.assertEqual("not_found", missing["focus"]["state"])
self.assertEqual("focus_not_found", missing["gaps"][0]["code"])
duplicate = root / "docs" / "content" / "workflow-copy.md"
duplicate.write_text(
(root / "docs" / "content" / "workflow.md")
.read_text(encoding="utf-8")
.replace('id = "guide.workflow"', 'id = "guide.workflow-copy"'),
encoding="utf-8",
)
duplicate_project = Project.open(root)
duplicate_index = ProjectIndex(duplicate_project)
duplicate_index.build()
ambiguous = duplicate_index.task_context(
self.plan(
duplicate_project,
task="Editing workflow",
focus_node_id=None,
)
)["capsule"]
self.assertEqual("blocked", ambiguous["state"])
self.assertEqual("ambiguous", ambiguous["focus"]["state"])
self.assertEqual("focus_ambiguous", ambiguous["gaps"][0]["code"])
def test_plan_rejects_unbounded_or_untyped_inputs(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
cases = (
({"task_kind": "unknown"}, "invalid_task_kind"),
({"task": " "}, "invalid_task_focus"),
({"focus_node_id": ""}, "invalid_task_focus"),
({"budget": True}, "invalid_budget"),
({"budget": 0}, "invalid_budget"),
({"limit": True}, "invalid_limit"),
({"limit": project.descriptor.limits.max_results + 1}, "invalid_limit"),
)
for overrides, code in cases:
arguments = {
"task_kind": "change",
"task": "Change the editing workflow",
"focus_node_id": "guide.workflow",
"budget": None,
"limit": None,
**overrides,
}
with self.subTest(arguments=overrides):
with self.assertRaises(DocForgeError) as invalid:
self.plan(project, **arguments)
self.assertEqual(code, invalid.exception.code)
def test_final_generation_change_rejects_the_whole_capsule(self) -> None:
with tempfile.TemporaryDirectory() as directory:
project = Project.open(self.copy_fixture("alpha", Path(directory)))
index = ProjectIndex(project)
built = index.build()
current = ProjectState(
revision=built["revision"],
source_hash=built["source_hash"],
)
changed = ProjectState(
revision=current.revision,
source_hash="0" * 64,
)
with (
mock.patch.object(
project,
"incremental_state",
side_effect=(current, changed),
),
self.assertRaises(DocForgeError) as stale,
):
index.task_context(self.plan(project))
self.assertEqual("source_changed", stale.exception.code)