Add versioned task context capsules
This commit is contained in:
parent
34cd5f74c1
commit
4cc6277054
18 changed files with 2834 additions and 10 deletions
|
|
@ -6,7 +6,9 @@ Goal: Let one project-bound server return compact, task-shaped, explainable cont
|
|||
In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks.
|
||||
Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
|
||||
Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass.
|
||||
Status: Active. Read-only contract audits begin from the verified Milestone 1 boundary.
|
||||
Status: Active. Effective policy is committed. Versioned task retrieval and context-capsule
|
||||
transport pass independent audit and the complete repository gate. The latest-generation diff
|
||||
receipt is the next slice.
|
||||
```
|
||||
|
||||
Milestones 3–5 remain directional context and are not active.
|
||||
|
|
|
|||
|
|
@ -388,3 +388,60 @@ exact-hash applier is enabled.
|
|||
|
||||
Existing factory defaults and tool order remain unchanged. Explicit application mode fails closed
|
||||
without an applier. Operator mode is reserved and currently adds no tools.
|
||||
|
||||
### Versioned task retrieval and context capsules
|
||||
|
||||
The first Milestone 2 retrieval slice adds one `docforge_get_task_context` read tool rather than a
|
||||
family of task-specific tools. Its closed task kinds are change, implementation, failure,
|
||||
ownership, test, operation, and release. One immutable `RetrievalPlanV1` derives exact or lexical
|
||||
focus, bounded bidirectional graph traversal, metadata hydration, required evidence categories,
|
||||
and fixed work budgets from the project descriptor and effective policy.
|
||||
|
||||
The public executor re-derives every submitted plan before opening SQLite. It rejects modified
|
||||
steps, task identity, requirements, category order, bounds, policy identity, or hashes as
|
||||
`invalid_retrieval_plan`. Traversal binds the project relation set by canonical hash and queries
|
||||
the already-validated edge table by endpoint, avoiding relation-sized SQL parameter lists.
|
||||
Version-1 internal ceilings are 1,000 evidence items, 100,000 candidate edges, and 10,000 task
|
||||
query characters.
|
||||
|
||||
The executor uses one immutable SQLite read generation. It rejects missing explicit focus, blocks
|
||||
unresolved or tied lexical focus, stops at deterministic evidence and candidate-edge limits, and
|
||||
checks source identity again when the transaction closes. `ContextCapsuleV1` binds the generation,
|
||||
policy, request, plan, evidence collection, and complete capsule with canonical hashes.
|
||||
|
||||
Project relation names remain authoritative. The core recognizes only a versioned alias map for
|
||||
structure, implementation, dependency, execution, data, evidence, and context. Unknown allowed
|
||||
relations stay visible under their raw names as `unclassified`. Required evidence diagnostics
|
||||
distinguish categories the project never declared, completed bounded checks with no selected
|
||||
evidence, and incomplete checks caused by a result, work, token, or response limit.
|
||||
|
||||
Each evidence item carries a stable content hash, confined source identity, shortest selected graph
|
||||
path, every additional qualifying relationship reason observed during traversal, and explicit
|
||||
limitations where the current graph cannot prove evidence type, extractor identity, relationship
|
||||
source provenance, or observation time. The planner contains no Logic operation, so no-AST
|
||||
bindings can use task context without weakening their existing Logic prohibition.
|
||||
|
||||
Path direction is relative to the preceding traversal node. Additional relationship reasons use
|
||||
the evidence node as their direction subject. Candidate-edge and unclassified-relation ceilings
|
||||
produce explicit omissions and bounded summaries.
|
||||
|
||||
MCP pagination preserves the complete plan, collection, and capsule hashes while returning bounded
|
||||
pages. Its cursor additionally binds the effective policy and task request. An individually
|
||||
oversized item advances once as a hash-identified omission. A later generation or policy change
|
||||
fails closed as `stale_cursor`.
|
||||
|
||||
The legacy profile-context contract remains intact. A custom context provider does not silently
|
||||
gain core task planning. Version 1 defines no custom task-planner extension, so the additive tool
|
||||
returns `task_context_unavailable` without synchronization or a complete projection load.
|
||||
|
||||
Two independent pre-commit audits reproduced and closed plan-forgery, relation-sized SQL,
|
||||
SQLite-parameter portability, ambiguous relationship-direction, missing work-limit evidence,
|
||||
schema/runtime drift, incomplete page hashing, and custom-provider hidden-load defects. Regression
|
||||
coverage includes 33,005 valid relation names, fixed extreme project limits, tampered plans,
|
||||
evidence-relative directions, edge and unclassified limits, schema-valid pages, changed cursor
|
||||
semantics, oversized evidence advancement, no-AST retrieval, and legacy complete-projection
|
||||
adapters.
|
||||
|
||||
The complete repository gate passes with 158 tests and 101 subtests, zero Pyright diagnostics,
|
||||
warning-strict execution, package builds, public-contract validation, and the maintained Milestone
|
||||
0 and Milestone 1 smoke benchmarks. Gitleaks 8.30.1 reports no secret findings in the working tree.
|
||||
|
|
|
|||
|
|
@ -9,6 +9,8 @@ declared manuals, visualizes project structure, and manages reviewable documenta
|
|||
- Validates stable Markdown/TOML nodes and typed relationships.
|
||||
- Builds a deterministic SQLite search and graph index.
|
||||
- Exposes project-bound CLI and MCP query surfaces.
|
||||
- Compiles versioned, generation-bound task context with cited evidence, explicit gaps, and bounded
|
||||
continuation.
|
||||
- Automatically synchronizes disposable indexes before MCP work.
|
||||
- Creates, validates, diffs, and previews isolated changesets.
|
||||
- Registers complete proposals atomically without caller-managed hash chaining.
|
||||
|
|
@ -53,6 +55,12 @@ DocForge2 bindings may also declare
|
|||
policy and the actual startup-gated capabilities. Existing tool surfaces and the legacy no-AST
|
||||
payload remain compatible.
|
||||
|
||||
`docforge_get_task_context` is an additive read tool for `change`, `implementation`, `failure`,
|
||||
`ownership`, `test`, `operation`, and `release` work. It derives a closed version-1 retrieval plan,
|
||||
executes it against one immutable index generation, and returns a hash-bound context capsule.
|
||||
Project relation names remain authoritative. DocForge classifies only its versioned alias set and
|
||||
preserves every unknown relation as `unclassified` instead of guessing semantics.
|
||||
|
||||
## Graph views
|
||||
|
||||
The browser presents the primary architecture graph through three complementary views and loads a
|
||||
|
|
|
|||
|
|
@ -147,6 +147,37 @@ and side-effect-free status.
|
|||
The live graph viewer remains a read-only consumer of a generation-pinned validated index. It does
|
||||
not become project authority or MCP retrieval authority.
|
||||
|
||||
## Task-context compatibility
|
||||
|
||||
`docforge_get_task_context` is an additive MCP read tool. The legacy `docforge_get_context`
|
||||
signature, profile compiler, direct Python results, and custom three-argument context-provider
|
||||
contract remain unchanged.
|
||||
|
||||
The new `ContextCapsuleV1` and `RetrievalPlanV1` types live in the public
|
||||
`docforge.retrieval` submodule. Version 1 guarantees:
|
||||
|
||||
- A closed task-kind vocabulary and core-derived plan. Callers cannot inject arbitrary operations,
|
||||
SQL, paths, relations, or Logic requests.
|
||||
- One immutable index transaction and one exact project, adapter, revision, source, policy,
|
||||
request, plan, collection, and capsule identity.
|
||||
- Deterministic bounded focus, traversal, hydration, token accounting, response packing, and
|
||||
continuation, with fixed version-1 ceilings of 1,000 evidence items, 100,000 candidate edges, and
|
||||
10,000 task-query characters.
|
||||
- Raw preservation of project-owned relation names. Only the documented versioned alias map gains
|
||||
task semantics; all other relations remain `unclassified`.
|
||||
- Separate missing, incomplete, blocked, and provenance-limitation evidence.
|
||||
- No-AST bindings retain task context but never add a Logic retrieval step or weaken the existing
|
||||
Logic prohibition.
|
||||
|
||||
An integration that replaces the legacy context provider does not silently receive the core task
|
||||
planner. Version 1 has no custom task-planner protocol. The task-context tool remains registered
|
||||
for additive name compatibility but returns `task_context_unavailable` without loading or
|
||||
synchronizing the custom projection.
|
||||
|
||||
The exact version-1 relation aliases are frozen by the MCP contract and repository contract tests.
|
||||
Changing an alias category requires a new planner version; it is not a silent implementation
|
||||
detail.
|
||||
|
||||
`ManualRenderPlan`, `GraphViewPlan`, a portable graph renderer, and independently packaged
|
||||
renderers are later-milestone direction. Milestone 0 does not claim that those contracts exist.
|
||||
|
||||
|
|
|
|||
|
|
@ -18,7 +18,9 @@ commit when Git is available; it cannot change repository state.
|
|||
- Edge schema: `schemas/edge.schema.json`, version 1.
|
||||
- Result envelope: `schemas/result.schema.json`, version 1.
|
||||
- Changeset schema: `schemas/changeset.schema.json`, version 1.
|
||||
- Index schema: version 2, disposable and reproducible.
|
||||
- Effective policy: `schemas/policy.schema.json`, version 1.
|
||||
- Task context capsule: `schemas/context-capsule.schema.json`, version 1.
|
||||
- Index schema: version 3, disposable and reproducible.
|
||||
- Index attestation: schema version 1, disposable and reproducible.
|
||||
- Core, CLI, and MCP server: version 1.3.0.dev0.
|
||||
- Incremental extraction cache: version 1, disposable and reproducible.
|
||||
|
|
@ -44,6 +46,12 @@ source hash. Errors use a stable code, direct message, structured details, and a
|
|||
tool when recovery is safe. MCP operations synchronize disposable index state under a project lock
|
||||
before reading or proposing. Canonical source validation remains fail-closed.
|
||||
|
||||
Task-context retrieval derives a closed version-1 plan from a bounded task kind and the effective
|
||||
process policy. It executes against one immutable index transaction and returns generation-bound,
|
||||
hash-identified evidence, gaps, omissions, and provenance limitations. Project relation names
|
||||
remain authoritative. The core applies task semantics only to its versioned alias set and preserves
|
||||
every other allowed relation as unclassified.
|
||||
|
||||
An atomic index build writes a whole-file SHA-256 attestation after complete graph, row, FTS, and
|
||||
SQLite integrity verification. A fresh process may use that receipt to verify an unchanged index
|
||||
without reconstructing all graph rows. A missing, malformed, or mismatched receipt falls back to
|
||||
|
|
@ -94,9 +102,10 @@ renderers. Render identity covers the canonical source hash, optional changeset
|
|||
and edge identities, view configuration, template hash, renderer contract, and exact parser version.
|
||||
|
||||
An explicit CLI render atomically replaces one declared derived output. MCP can render a validated
|
||||
changeset only to its isolated preview path. Status recomputes expected output without writing and
|
||||
reports `current`, `stale`, `missing`, `unsafe`, or `oversized`. Input changes detected before atomic
|
||||
replacement fail without replacing the prior output.
|
||||
changeset only to its isolated preview path. Normal status verifies bounded source, configuration,
|
||||
template, output, renderer, and publication-receipt identities without reconstructing the output.
|
||||
Explicit deep status remains the side-effect-free full-render oracle. Input changes detected before
|
||||
atomic replacement fail without publishing a current receipt for stale output.
|
||||
|
||||
Normal MCP access does not expose canonical application. An explicitly configured canonical
|
||||
applier registers one hash-bound application tool. No MCP mode exposes arbitrary renderer
|
||||
|
|
|
|||
|
|
@ -42,6 +42,7 @@ schema version 1 and does not silently acquire machine-specific process policy.
|
|||
- `docforge_dependencies`
|
||||
- `docforge_impact`
|
||||
- `docforge_get_context`
|
||||
- `docforge_get_task_context`
|
||||
- `docforge_validate_project`
|
||||
- `docforge_render_status`
|
||||
- `docforge_visualize`
|
||||
|
|
@ -72,6 +73,46 @@ oversized entry advances as a hash-identified `response size limit` omission so
|
|||
loop; targeted retrieval remains available for that node. The existing three-argument custom
|
||||
context-provider contract is unchanged because pagination is applied after provider selection.
|
||||
|
||||
`docforge_get_task_context` accepts one closed task kind (`change`, `implementation`, `failure`,
|
||||
`ownership`, `test`, `operation`, or `release`), a bounded task description, and optional
|
||||
`focus_node_id`, token `budget`, page `limit`, and opaque `cursor`. It derives, rather than accepts,
|
||||
a version-1 retrieval plan. The plan contains only exact or lexical focus, bounded outgoing and
|
||||
incoming graph traversal, and metadata hydration. It cannot request arbitrary SQL, paths, relation
|
||||
names, or Logic extraction. Task context applies fixed internal ceilings of 1,000 evidence items,
|
||||
100,000 examined candidate edges, and 10,000 task-query characters even when broader project
|
||||
limits are configured. Traversal steps bind the complete project-owned relation vocabulary by hash
|
||||
rather than copying an unbounded name list into every response.
|
||||
|
||||
The plan and returned context capsule are bound to the effective policy and one immutable index
|
||||
generation. Every evidence item identifies its indexed source path, content hash, graph path,
|
||||
additional qualifying relationship reasons, and the provenance facts that the current graph
|
||||
cannot prove. Required evidence gaps distinguish an undeclared relation category, a completed
|
||||
bounded search with no selected evidence, and an incomplete proof caused by a work, result, token,
|
||||
or response limit. Unknown project relations remain present with their raw names and an
|
||||
`unclassified_relation` limitation; DocForge never infers semantics from spelling outside the
|
||||
versioned alias map.
|
||||
|
||||
Path relationship direction is relative to the preceding traversal node. Additional
|
||||
`relationship_reasons` direction is relative to the evidence item itself: `outgoing` when that
|
||||
evidence node is the stored source and `incoming` when it is the stored target.
|
||||
|
||||
The exact version-1 aliases are: structure (`contains`, `defined_in`, `defines`, `owns`);
|
||||
implementation (`implemented_by`, `implements`, `inherits`, `inherits_from`); dependency
|
||||
(`depends_on`, `imports`); execution (`activates`, `calls`, `dispatches_to`, `launches`); data
|
||||
(`reads`, `writes`); evidence (`documents`, `governs`, `proves`, `tested_by`, `verifies`); and
|
||||
context (`relates_to`). Every other allowed relation is `unclassified`.
|
||||
|
||||
Task-context continuation partitions the immutable evidence stream without changing its
|
||||
`request_hash`, `plan_hash`, `collection_hash`, or `capsule_hash`. Its cursor additionally binds
|
||||
the effective policy and task request. One evidence item that cannot fit advances exactly once as
|
||||
a hash-identified `response_limit` omission. A changed generation, policy, plan, or collection
|
||||
fails as `stale_cursor`.
|
||||
|
||||
The legacy `docforge_get_context` tool and its custom three-argument provider contract remain
|
||||
unchanged. A server with a custom context provider does not silently inherit the core task planner;
|
||||
version 1 exposes no custom task-planner extension point. `docforge_get_task_context` returns
|
||||
`task_context_unavailable` without synchronizing or loading the custom projection.
|
||||
|
||||
Version-1 cursors are canonical JSON encoded as base64url with a domain-separated SHA-256
|
||||
corruption checksum. They are opaque and fail closed, but are not authenticated authorization
|
||||
tokens. Cursors bind the project, adapter, source generation, operation parameters, collection
|
||||
|
|
|
|||
|
|
@ -582,6 +582,7 @@ Example MCP client configuration:
|
|||
- `docforge_dependencies`
|
||||
- `docforge_impact`
|
||||
- `docforge_get_context`
|
||||
- `docforge_get_task_context`
|
||||
- `docforge_validate_project`
|
||||
- `docforge_render_status`
|
||||
- `docforge_visualize`
|
||||
|
|
@ -613,6 +614,36 @@ The application call requires `changeset_id` and `expected_changeset_hash`. Alwa
|
|||
inspect the final diff after the last proposal mutation. Apply that exact hash. A proposal mutation
|
||||
creates a new hash, so an earlier approval cannot silently apply later content.
|
||||
|
||||
Use `docforge_get_task_context` when an agent needs one bounded task-shaped intake instead of a
|
||||
named profile. Choose `task_kind` from `change`, `implementation`, `failure`, `ownership`, `test`,
|
||||
`operation`, or `release`. Supply `focus_node_id` when the stable node is known. Without it,
|
||||
DocForge performs a bounded lexical focus search and refuses a tied best match instead of silently
|
||||
choosing one.
|
||||
|
||||
The returned version-1 capsule includes:
|
||||
|
||||
- The exact project, adapter, source generation, effective policy, request, and retrieval-plan
|
||||
hashes.
|
||||
- Ordered focus and related evidence with source paths, content hashes, graph paths, and all
|
||||
qualifying relationship reasons observed during the bounded traversal.
|
||||
- Explicit evidence gaps and omissions, including whether a check completed.
|
||||
- Provenance limitations for facts that the current graph does not carry, such as extractor
|
||||
identity, observation time, and source provenance for relationships.
|
||||
|
||||
Project descriptors still own the valid relation vocabulary. The planner recognizes a fixed alias
|
||||
map for structure, implementation, dependency, execution, data, evidence, and context. Any other
|
||||
valid project relation is returned unchanged as `unclassified`; it is never assigned guessed task
|
||||
semantics.
|
||||
|
||||
A relationship inside `relationship_path` describes the direction traveled from the preceding
|
||||
node. A relationship inside `relationship_reasons` describes direction from the evidence item
|
||||
itself. This keeps stored source and target identity exact while making each evidence explanation
|
||||
locally readable.
|
||||
|
||||
Task context never exceeds 1,000 evidence items, 100,000 examined candidate edges, or 10,000 task
|
||||
query characters, even when a project configures broader general limits. An edge-work or
|
||||
unclassified-relation ceiling appears as an explicit omission rather than an unbounded response.
|
||||
|
||||
Recommended release-candidate sequence:
|
||||
|
||||
1. Call `docforge_bootstrap`. It synchronizes derived state and reports the exact fixed binding.
|
||||
|
|
@ -675,6 +706,11 @@ inspection pages may use hash summaries. A large diff may return `result_mode =
|
|||
"canonical_json_chunk"`; concatenate the chunks in order and verify `payload_hash` before decoding
|
||||
the reconstructed `operations` and `changes` object.
|
||||
|
||||
`docforge_get_task_context` uses the same opaque continuation discipline over capsule evidence
|
||||
followed by capsule omissions. Keep the semantic task arguments unchanged while paging. Page size
|
||||
may change. Every page retains the same plan, collection, and capsule hashes. A `stale_cursor`
|
||||
means that the generation, policy, plan, or collection changed; discard earlier pages and restart.
|
||||
|
||||
Canonical application records its terminal receipt immediately after the project-owned serializer
|
||||
verifies the new canonical state. A later index or render refresh failure is reported as degraded
|
||||
derived state with remediation, not as permission to apply the same canonical change again.
|
||||
|
|
|
|||
435
schemas/context-capsule.schema.json
Normal file
435
schemas/context-capsule.schema.json
Normal file
|
|
@ -0,0 +1,435 @@
|
|||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://docforge.local/schema/context-capsule-v1.json",
|
||||
"title": "DocForge task context capsule",
|
||||
"$defs": {
|
||||
"sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
},
|
||||
"pagination": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schema_version",
|
||||
"kind",
|
||||
"returned_count",
|
||||
"limit",
|
||||
"total_count",
|
||||
"has_more",
|
||||
"next_cursor"
|
||||
],
|
||||
"properties": {
|
||||
"schema_version": { "const": 1 },
|
||||
"kind": { "const": "task-context.items" },
|
||||
"returned_count": { "type": "integer", "minimum": 0 },
|
||||
"limit": { "type": "integer", "minimum": 1 },
|
||||
"total_count": { "type": "integer", "minimum": 0 },
|
||||
"has_more": { "type": "boolean" },
|
||||
"next_cursor": {
|
||||
"type": ["string", "null"],
|
||||
"minLength": 1,
|
||||
"maxLength": 8192
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"category": {
|
||||
"enum": [
|
||||
"structure",
|
||||
"implementation",
|
||||
"dependency",
|
||||
"execution",
|
||||
"data",
|
||||
"evidence",
|
||||
"context",
|
||||
"unclassified"
|
||||
]
|
||||
},
|
||||
"step": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"step_id",
|
||||
"operation",
|
||||
"relation_scope",
|
||||
"relation_set_hash",
|
||||
"direction",
|
||||
"depth",
|
||||
"limit",
|
||||
"required",
|
||||
"evidence_role"
|
||||
],
|
||||
"properties": {
|
||||
"step_id": { "type": "string", "minLength": 1, "maxLength": 64 },
|
||||
"operation": {
|
||||
"enum": ["exact", "search", "outgoing", "incoming", "metadata"]
|
||||
},
|
||||
"relation_scope": { "enum": ["none", "project_allowed"] },
|
||||
"relation_set_hash": {
|
||||
"oneOf": [
|
||||
{ "$ref": "#/$defs/sha256" },
|
||||
{ "type": "null" }
|
||||
]
|
||||
},
|
||||
"direction": { "enum": ["none", "outgoing", "incoming"] },
|
||||
"depth": { "type": "integer", "minimum": 0 },
|
||||
"limit": { "type": "integer", "minimum": 1 },
|
||||
"required": { "type": "boolean" },
|
||||
"evidence_role": { "type": "string", "minLength": 1, "maxLength": 64 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"requirement": {
|
||||
"type": "object",
|
||||
"required": ["requirement_id", "check", "category", "required"],
|
||||
"properties": {
|
||||
"requirement_id": { "type": "string", "minLength": 1, "maxLength": 128 },
|
||||
"check": { "const": "selected_relation_category" },
|
||||
"category": { "$ref": "#/$defs/category" },
|
||||
"required": { "const": true }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"plan": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schema_version",
|
||||
"planner",
|
||||
"task_kind",
|
||||
"request_hash",
|
||||
"effective_policy_hash",
|
||||
"focus_node_id",
|
||||
"limits",
|
||||
"category_order",
|
||||
"steps",
|
||||
"requirements",
|
||||
"plan_hash"
|
||||
],
|
||||
"properties": {
|
||||
"schema_version": { "const": 1 },
|
||||
"planner": {
|
||||
"type": "object",
|
||||
"required": ["id", "version"],
|
||||
"properties": {
|
||||
"id": { "const": "docforge.core.task-context" },
|
||||
"version": { "const": 1 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"task_kind": {
|
||||
"enum": [
|
||||
"change",
|
||||
"implementation",
|
||||
"failure",
|
||||
"ownership",
|
||||
"test",
|
||||
"operation",
|
||||
"release"
|
||||
]
|
||||
},
|
||||
"request_hash": { "$ref": "#/$defs/sha256" },
|
||||
"effective_policy_hash": { "$ref": "#/$defs/sha256" },
|
||||
"focus_node_id": { "type": ["string", "null"], "maxLength": 256 },
|
||||
"limits": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"max_evidence",
|
||||
"max_tokens",
|
||||
"max_depth",
|
||||
"max_candidate_edges"
|
||||
],
|
||||
"properties": {
|
||||
"max_evidence": { "type": "integer", "minimum": 1, "maximum": 1000 },
|
||||
"max_tokens": { "type": "integer", "minimum": 1 },
|
||||
"max_depth": { "type": "integer", "minimum": 0 },
|
||||
"max_candidate_edges": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 100000
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"category_order": {
|
||||
"type": "array",
|
||||
"minItems": 8,
|
||||
"maxItems": 8,
|
||||
"items": { "$ref": "#/$defs/category" },
|
||||
"uniqueItems": true
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"minItems": 4,
|
||||
"maxItems": 4,
|
||||
"items": { "$ref": "#/$defs/step" }
|
||||
},
|
||||
"requirements": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"maxItems": 8,
|
||||
"items": { "$ref": "#/$defs/requirement" }
|
||||
},
|
||||
"plan_hash": { "$ref": "#/$defs/sha256" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"relationship": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"source_id",
|
||||
"relation",
|
||||
"target_id",
|
||||
"direction",
|
||||
"category",
|
||||
"provenance"
|
||||
],
|
||||
"properties": {
|
||||
"source_id": { "type": "string", "minLength": 1 },
|
||||
"relation": { "type": "string", "minLength": 1 },
|
||||
"target_id": { "type": "string", "minLength": 1 },
|
||||
"direction": { "enum": ["outgoing", "incoming"] },
|
||||
"category": { "$ref": "#/$defs/category" },
|
||||
"provenance": {
|
||||
"const": "validated_graph_edge_without_source_provenance"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"evidence": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"evidence_hash",
|
||||
"role",
|
||||
"reason_code",
|
||||
"node_id",
|
||||
"title",
|
||||
"family",
|
||||
"authority",
|
||||
"status",
|
||||
"tags",
|
||||
"summary",
|
||||
"text",
|
||||
"estimated_tokens",
|
||||
"source",
|
||||
"depth",
|
||||
"relationship_path",
|
||||
"relationship_reasons",
|
||||
"provenance_limitations"
|
||||
],
|
||||
"properties": {
|
||||
"evidence_hash": { "$ref": "#/$defs/sha256" },
|
||||
"role": { "enum": ["focus", "related"] },
|
||||
"reason_code": {
|
||||
"enum": ["exact_focus", "lexical_focus", "relationship_path"]
|
||||
},
|
||||
"node_id": { "type": "string", "minLength": 1 },
|
||||
"title": { "type": "string" },
|
||||
"family": { "type": "string" },
|
||||
"authority": { "type": "string" },
|
||||
"status": { "type": "string" },
|
||||
"tags": {
|
||||
"type": "array",
|
||||
"items": { "type": "string" }
|
||||
},
|
||||
"summary": { "type": "string" },
|
||||
"text": { "type": "string" },
|
||||
"estimated_tokens": { "type": "integer", "minimum": 1 },
|
||||
"source": {
|
||||
"type": "object",
|
||||
"required": ["path", "anchor", "content_hash"],
|
||||
"properties": {
|
||||
"path": { "type": "string", "minLength": 1 },
|
||||
"anchor": { "type": ["string", "null"] },
|
||||
"content_hash": { "$ref": "#/$defs/sha256" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"depth": { "type": "integer", "minimum": 0 },
|
||||
"relationship_path": {
|
||||
"type": "array",
|
||||
"items": { "$ref": "#/$defs/relationship" }
|
||||
},
|
||||
"relationship_reasons": {
|
||||
"type": "array",
|
||||
"items": { "$ref": "#/$defs/relationship" },
|
||||
"uniqueItems": true
|
||||
},
|
||||
"provenance_limitations": {
|
||||
"const": [
|
||||
"evidence_type_unavailable",
|
||||
"extractor_identity_unavailable",
|
||||
"relationship_provenance_unavailable",
|
||||
"observation_time_unavailable"
|
||||
]
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"gap": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"code",
|
||||
"requirement_id",
|
||||
"category",
|
||||
"state",
|
||||
"check_complete",
|
||||
"detail"
|
||||
],
|
||||
"properties": {
|
||||
"code": {
|
||||
"enum": [
|
||||
"focus_not_found",
|
||||
"focus_ambiguous",
|
||||
"category_not_declared",
|
||||
"no_selected_evidence",
|
||||
"evidence_incomplete",
|
||||
"unclassified_relation"
|
||||
]
|
||||
},
|
||||
"requirement_id": { "type": "string", "minLength": 1 },
|
||||
"category": {
|
||||
"oneOf": [
|
||||
{ "$ref": "#/$defs/category" },
|
||||
{ "type": "null" }
|
||||
]
|
||||
},
|
||||
"state": { "enum": ["missing", "incomplete", "blocked", "limitation"] },
|
||||
"check_complete": { "type": "boolean" },
|
||||
"detail": { "type": "string", "minLength": 1, "maxLength": 1000 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"omission": {
|
||||
"type": "object",
|
||||
"required": ["code", "subject", "detail_hash"],
|
||||
"properties": {
|
||||
"code": {
|
||||
"enum": [
|
||||
"result_limit",
|
||||
"token_budget",
|
||||
"response_limit",
|
||||
"edge_examination_limit",
|
||||
"unclassified_relation_limit"
|
||||
]
|
||||
},
|
||||
"subject": { "type": "string", "minLength": 1, "maxLength": 256 },
|
||||
"detail_hash": { "$ref": "#/$defs/sha256" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schema_version",
|
||||
"state",
|
||||
"task_kind",
|
||||
"generation",
|
||||
"plan",
|
||||
"focus",
|
||||
"evidence",
|
||||
"gaps",
|
||||
"omissions",
|
||||
"summary",
|
||||
"collection_hash",
|
||||
"capsule_hash"
|
||||
],
|
||||
"properties": {
|
||||
"schema_version": { "const": 1 },
|
||||
"state": { "enum": ["complete", "incomplete", "blocked"] },
|
||||
"task_kind": {
|
||||
"enum": [
|
||||
"change",
|
||||
"implementation",
|
||||
"failure",
|
||||
"ownership",
|
||||
"test",
|
||||
"operation",
|
||||
"release"
|
||||
]
|
||||
},
|
||||
"generation": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"project_id",
|
||||
"project_root_fingerprint",
|
||||
"adapter",
|
||||
"revision",
|
||||
"source_hash",
|
||||
"index_schema_version"
|
||||
],
|
||||
"properties": {
|
||||
"project_id": { "type": "string", "minLength": 1 },
|
||||
"project_root_fingerprint": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{16}$"
|
||||
},
|
||||
"adapter": { "type": "string", "minLength": 1 },
|
||||
"revision": { "type": "string", "minLength": 1 },
|
||||
"source_hash": { "$ref": "#/$defs/sha256" },
|
||||
"index_schema_version": { "type": "integer", "minimum": 1 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"plan": { "$ref": "#/$defs/plan" },
|
||||
"focus": {
|
||||
"type": "object",
|
||||
"required": ["state", "node_id", "candidate_count"],
|
||||
"properties": {
|
||||
"state": { "enum": ["resolved", "not_found", "ambiguous"] },
|
||||
"node_id": { "type": ["string", "null"] },
|
||||
"candidate_count": { "type": "integer", "minimum": 0 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"evidence": {
|
||||
"type": "array",
|
||||
"maxItems": 10000,
|
||||
"items": { "$ref": "#/$defs/evidence" }
|
||||
},
|
||||
"gaps": {
|
||||
"type": "array",
|
||||
"maxItems": 10000,
|
||||
"items": { "$ref": "#/$defs/gap" }
|
||||
},
|
||||
"omissions": {
|
||||
"type": "array",
|
||||
"maxItems": 10000,
|
||||
"items": { "$ref": "#/$defs/omission" }
|
||||
},
|
||||
"summary": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"evidence_count",
|
||||
"gap_count",
|
||||
"omission_count",
|
||||
"selected_count",
|
||||
"examined_edge_count",
|
||||
"estimated_tokens",
|
||||
"unclassified_relations"
|
||||
],
|
||||
"properties": {
|
||||
"evidence_count": { "type": "integer", "minimum": 0 },
|
||||
"gap_count": { "type": "integer", "minimum": 0 },
|
||||
"omission_count": { "type": "integer", "minimum": 0 },
|
||||
"selected_count": { "type": "integer", "minimum": 0 },
|
||||
"examined_edge_count": { "type": "integer", "minimum": 0 },
|
||||
"estimated_tokens": { "type": "integer", "minimum": 0 },
|
||||
"unclassified_relations": {
|
||||
"type": "array",
|
||||
"maxItems": 10000,
|
||||
"items": { "type": "string", "minLength": 1 },
|
||||
"uniqueItems": true
|
||||
},
|
||||
"page_evidence_count": { "type": "integer", "minimum": 0 },
|
||||
"page_omission_count": { "type": "integer", "minimum": 0 },
|
||||
"page_item_count": { "type": "integer", "minimum": 0 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"collection_hash": { "$ref": "#/$defs/sha256" },
|
||||
"capsule_hash": { "$ref": "#/$defs/sha256" },
|
||||
"page_state": { "enum": ["complete", "incomplete", "blocked"] },
|
||||
"page_hash": { "$ref": "#/$defs/sha256" },
|
||||
"pagination": { "$ref": "#/$defs/pagination" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
|
|
@ -32,6 +32,7 @@
|
|||
"mcp.dependencies",
|
||||
"mcp.impact",
|
||||
"mcp.context",
|
||||
"mcp.task_context",
|
||||
"mcp.validate_project",
|
||||
"mcp.render_status",
|
||||
"mcp.visualize",
|
||||
|
|
@ -149,6 +150,7 @@
|
|||
"kind": {
|
||||
"enum": [
|
||||
"context.items",
|
||||
"task-context.items",
|
||||
"changeset.list",
|
||||
"changeset.inspect",
|
||||
"changeset.validate",
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ from collections.abc import Callable, Generator
|
|||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
from typing import Literal, cast
|
||||
|
||||
from .errors import DocForgeError
|
||||
from .models import (
|
||||
|
|
@ -32,11 +32,25 @@ from .models import (
|
|||
ProjectSnapshot,
|
||||
ProjectState,
|
||||
)
|
||||
from .pagination import canonical_hash
|
||||
from .project import project_root_fingerprint
|
||||
from .retrieval import (
|
||||
CapsuleEvidenceV1,
|
||||
CapsuleOmissionV1,
|
||||
CapsuleRelationshipV1,
|
||||
ContextCapsuleV1,
|
||||
EvidenceGapV1,
|
||||
RetrievalPlanV1,
|
||||
capsule_evidence,
|
||||
finalize_capsule,
|
||||
relation_category,
|
||||
validate_retrieval_plan,
|
||||
)
|
||||
from .telemetry import increment, stage
|
||||
|
||||
INDEX_SCHEMA_VERSION = 3
|
||||
APPLICATION_ID = 1_146_683_778
|
||||
_SQLITE_PARAMETER_CHUNK = 500
|
||||
|
||||
|
||||
def _node_hash(nodes: tuple[Node, ...]) -> str:
|
||||
|
|
@ -147,6 +161,17 @@ class _IndexReadSnapshot:
|
|||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _TaskSelection:
|
||||
node_id: str
|
||||
role: str
|
||||
reason_code: str
|
||||
depth: int
|
||||
category_rank: int
|
||||
first_edge: tuple[str, str, str]
|
||||
relationship_path: tuple[CapsuleRelationshipV1, ...]
|
||||
|
||||
|
||||
class ProjectIndex:
|
||||
"""A disposable index that always checks current canonical source before queries."""
|
||||
|
||||
|
|
@ -798,6 +823,418 @@ class ProjectIndex:
|
|||
status.st_ctime_ns,
|
||||
)
|
||||
|
||||
def task_context(self, plan: RetrievalPlanV1) -> dict[str, object]:
|
||||
"""Execute one fixed task plan inside one immutable index generation."""
|
||||
|
||||
validate_retrieval_plan(plan, self.project.descriptor)
|
||||
with self._read_snapshot() as snapshot:
|
||||
capsule = self._task_context_capsule(snapshot, plan)
|
||||
return snapshot.result(capsule=capsule.as_dict())
|
||||
|
||||
def _task_context_capsule(
|
||||
self,
|
||||
snapshot: _IndexReadSnapshot,
|
||||
plan: RetrievalPlanV1,
|
||||
) -> ContextCapsuleV1:
|
||||
generation = {
|
||||
"project_id": snapshot.checked["project_id"],
|
||||
"project_root_fingerprint": snapshot.checked["project_root_fingerprint"],
|
||||
"adapter": snapshot.checked["adapter"],
|
||||
"revision": snapshot.checked["revision"],
|
||||
"source_hash": snapshot.checked["source_hash"],
|
||||
"index_schema_version": INDEX_SCHEMA_VERSION,
|
||||
}
|
||||
focus_rows: list[sqlite3.Row]
|
||||
focus_reason = "exact_focus"
|
||||
if plan.focus_node_id is not None:
|
||||
row = snapshot.connection.execute(
|
||||
"SELECT * FROM nodes WHERE node_id = ?",
|
||||
(plan.focus_node_id,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise DocForgeError(
|
||||
"missing_node",
|
||||
"No node has the requested stable ID",
|
||||
node_id=plan.focus_node_id,
|
||||
)
|
||||
focus_rows = [row]
|
||||
else:
|
||||
terms = re_tokenize(plan.task_query)
|
||||
if not terms:
|
||||
raise DocForgeError(
|
||||
"invalid_task_focus",
|
||||
"Task description contains no searchable text",
|
||||
)
|
||||
expression = " AND ".join(f'"{term.replace(chr(34), chr(34) * 2)}"' for term in terms)
|
||||
focus_rows = snapshot.connection.execute(
|
||||
"""
|
||||
SELECT nodes.*, bm25(node_fts) AS rank
|
||||
FROM node_fts JOIN nodes USING(node_id)
|
||||
WHERE node_fts MATCH ?
|
||||
ORDER BY rank, nodes.node_id
|
||||
LIMIT ?
|
||||
""",
|
||||
(expression, plan.max_evidence + 1),
|
||||
).fetchall()
|
||||
focus_reason = "lexical_focus"
|
||||
if not focus_rows:
|
||||
return finalize_capsule(
|
||||
state="blocked",
|
||||
task_kind=plan.task_kind,
|
||||
generation=generation,
|
||||
plan=plan,
|
||||
focus_state="not_found",
|
||||
focus_node_id=None,
|
||||
focus_candidate_count=0,
|
||||
evidence=(),
|
||||
gaps=(
|
||||
EvidenceGapV1(
|
||||
code="focus_not_found",
|
||||
requirement_id="focus",
|
||||
category=None,
|
||||
state="blocked",
|
||||
check_complete=True,
|
||||
detail="No indexed node matched the bounded lexical focus.",
|
||||
),
|
||||
),
|
||||
omissions=(),
|
||||
selected_count=0,
|
||||
examined_edge_count=0,
|
||||
estimated_tokens=0,
|
||||
unclassified_relations=(),
|
||||
)
|
||||
if len(focus_rows) > 1 and focus_rows[0]["rank"] == focus_rows[1]["rank"]:
|
||||
return finalize_capsule(
|
||||
state="blocked",
|
||||
task_kind=plan.task_kind,
|
||||
generation=generation,
|
||||
plan=plan,
|
||||
focus_state="ambiguous",
|
||||
focus_node_id=None,
|
||||
focus_candidate_count=len(focus_rows),
|
||||
evidence=(),
|
||||
gaps=(
|
||||
EvidenceGapV1(
|
||||
code="focus_ambiguous",
|
||||
requirement_id="focus",
|
||||
category=None,
|
||||
state="blocked",
|
||||
check_complete=True,
|
||||
detail=(
|
||||
"The highest-ranked lexical focus is tied; provide focus_node_id."
|
||||
),
|
||||
),
|
||||
),
|
||||
omissions=(),
|
||||
selected_count=0,
|
||||
examined_edge_count=0,
|
||||
estimated_tokens=0,
|
||||
unclassified_relations=(),
|
||||
)
|
||||
focus_rows = focus_rows[:1]
|
||||
|
||||
focus_node_id = cast(str, focus_rows[0]["node_id"])
|
||||
selections: dict[str, _TaskSelection] = {
|
||||
focus_node_id: _TaskSelection(
|
||||
node_id=focus_node_id,
|
||||
role="focus",
|
||||
reason_code=focus_reason,
|
||||
depth=0,
|
||||
category_rank=-1,
|
||||
first_edge=("", "", ""),
|
||||
relationship_path=(),
|
||||
)
|
||||
}
|
||||
relationship_reasons: dict[str, set[CapsuleRelationshipV1]] = {focus_node_id: set()}
|
||||
queue: deque[str] = deque((focus_node_id,))
|
||||
examined_edge_count = 0
|
||||
traversal_incomplete = False
|
||||
edge_examination_limit_reached = False
|
||||
first_omitted_node: str | None = None
|
||||
unclassified_relations: set[str] = set()
|
||||
while queue and not traversal_incomplete:
|
||||
current = queue.popleft()
|
||||
current_selection = selections[current]
|
||||
if current_selection.depth >= plan.max_depth:
|
||||
continue
|
||||
remaining = plan.max_candidate_edges - examined_edge_count
|
||||
if remaining <= 0:
|
||||
traversal_incomplete = True
|
||||
edge_examination_limit_reached = True
|
||||
break
|
||||
outgoing = snapshot.connection.execute(
|
||||
"SELECT source_id, relation, target_id FROM edges "
|
||||
"WHERE source_id = ? ORDER BY source_id, relation, target_id LIMIT ?",
|
||||
(current, remaining + 1),
|
||||
).fetchall()
|
||||
incoming = snapshot.connection.execute(
|
||||
"SELECT source_id, relation, target_id FROM edges "
|
||||
"WHERE target_id = ? ORDER BY source_id, relation, target_id LIMIT ?",
|
||||
(current, remaining + 1),
|
||||
).fetchall()
|
||||
candidates = {
|
||||
(
|
||||
cast(str, row["source_id"]),
|
||||
cast(str, row["relation"]),
|
||||
cast(str, row["target_id"]),
|
||||
"outgoing" if row["source_id"] == current else "incoming",
|
||||
)
|
||||
for row in (*outgoing, *incoming)
|
||||
}
|
||||
ordered = sorted(
|
||||
candidates,
|
||||
key=lambda item: (
|
||||
plan.category_order.index(relation_category(item[1])),
|
||||
item[0],
|
||||
item[1],
|
||||
item[2],
|
||||
item[3],
|
||||
),
|
||||
)
|
||||
if len(ordered) > remaining:
|
||||
traversal_incomplete = True
|
||||
edge_examination_limit_reached = True
|
||||
ordered = ordered[:remaining]
|
||||
for source_id, relation, target_id, direction in ordered:
|
||||
examined_edge_count += 1
|
||||
category = relation_category(relation)
|
||||
if category == "unclassified":
|
||||
unclassified_relations.add(relation)
|
||||
neighbor = target_id if source_id == current else source_id
|
||||
relationship = CapsuleRelationshipV1(
|
||||
source_id=source_id,
|
||||
relation=relation,
|
||||
target_id=target_id,
|
||||
direction=cast(Literal["outgoing", "incoming"], direction),
|
||||
category=category,
|
||||
)
|
||||
evidence_reason = CapsuleRelationshipV1(
|
||||
source_id=source_id,
|
||||
relation=relation,
|
||||
target_id=target_id,
|
||||
direction="outgoing" if source_id == neighbor else "incoming",
|
||||
category=category,
|
||||
)
|
||||
if neighbor in selections:
|
||||
relationship_reasons.setdefault(neighbor, set()).add(evidence_reason)
|
||||
continue
|
||||
if len(selections) >= plan.max_evidence:
|
||||
traversal_incomplete = True
|
||||
first_omitted_node = neighbor
|
||||
break
|
||||
selections[neighbor] = _TaskSelection(
|
||||
node_id=neighbor,
|
||||
role="related",
|
||||
reason_code="relationship_path",
|
||||
depth=current_selection.depth + 1,
|
||||
category_rank=plan.category_order.index(category),
|
||||
first_edge=(source_id, relation, target_id),
|
||||
relationship_path=(
|
||||
*current_selection.relationship_path,
|
||||
relationship,
|
||||
),
|
||||
)
|
||||
relationship_reasons[neighbor] = {evidence_reason}
|
||||
queue.append(neighbor)
|
||||
|
||||
ordered_selections = sorted(
|
||||
selections.values(),
|
||||
key=lambda selection: (
|
||||
0 if selection.role == "focus" else 1,
|
||||
selection.category_rank,
|
||||
selection.depth,
|
||||
selection.first_edge,
|
||||
selection.node_id,
|
||||
),
|
||||
)
|
||||
selected_node_ids = tuple(selection.node_id for selection in ordered_selections)
|
||||
rows: list[sqlite3.Row] = []
|
||||
for position in range(0, len(selected_node_ids), _SQLITE_PARAMETER_CHUNK):
|
||||
node_id_chunk = selected_node_ids[position : position + _SQLITE_PARAMETER_CHUNK]
|
||||
placeholders = ",".join("?" for _ in node_id_chunk)
|
||||
rows.extend(
|
||||
snapshot.connection.execute(
|
||||
f"SELECT * FROM nodes WHERE node_id IN ({placeholders}) ORDER BY node_id",
|
||||
node_id_chunk,
|
||||
).fetchall()
|
||||
)
|
||||
nodes = {cast(str, row["node_id"]): _row_to_node(row) for row in rows}
|
||||
evidence: list[CapsuleEvidenceV1] = []
|
||||
omissions: list[CapsuleOmissionV1] = []
|
||||
used_tokens = 0
|
||||
returned_categories: set[str] = set()
|
||||
for selection in ordered_selections:
|
||||
node = nodes[selection.node_id]
|
||||
item = capsule_evidence(
|
||||
role=cast(Literal["focus", "related"], selection.role),
|
||||
reason_code=cast(
|
||||
Literal["exact_focus", "lexical_focus", "relationship_path"],
|
||||
selection.reason_code,
|
||||
),
|
||||
node=node,
|
||||
depth=selection.depth,
|
||||
relationship_path=selection.relationship_path,
|
||||
relationship_reasons=tuple(
|
||||
sorted(
|
||||
relationship_reasons.get(selection.node_id, set()),
|
||||
key=lambda relationship: (
|
||||
relationship.source_id,
|
||||
relationship.relation,
|
||||
relationship.target_id,
|
||||
relationship.direction,
|
||||
),
|
||||
)
|
||||
),
|
||||
)
|
||||
if used_tokens + item.estimated_tokens > plan.max_tokens:
|
||||
omissions.append(
|
||||
CapsuleOmissionV1(
|
||||
code="token_budget",
|
||||
subject=node.node_id,
|
||||
detail_hash=canonical_hash(
|
||||
{
|
||||
"node_id": node.node_id,
|
||||
"content_hash": node.content_hash,
|
||||
"estimated_tokens": item.estimated_tokens,
|
||||
}
|
||||
),
|
||||
)
|
||||
)
|
||||
continue
|
||||
evidence.append(item)
|
||||
used_tokens += item.estimated_tokens
|
||||
returned_categories.update(
|
||||
relationship.category for relationship in item.relationship_reasons
|
||||
)
|
||||
if first_omitted_node is not None:
|
||||
omissions.append(
|
||||
CapsuleOmissionV1(
|
||||
code="result_limit",
|
||||
subject=first_omitted_node,
|
||||
detail_hash=canonical_hash(
|
||||
{
|
||||
"node_id": first_omitted_node,
|
||||
"max_evidence": plan.max_evidence,
|
||||
}
|
||||
),
|
||||
)
|
||||
)
|
||||
if edge_examination_limit_reached:
|
||||
omissions.append(
|
||||
CapsuleOmissionV1(
|
||||
code="edge_examination_limit",
|
||||
subject=focus_node_id,
|
||||
detail_hash=canonical_hash(
|
||||
{
|
||||
"focus_node_id": focus_node_id,
|
||||
"examined_edge_count": examined_edge_count,
|
||||
"max_candidate_edges": plan.max_candidate_edges,
|
||||
}
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
ordered_unclassified_relations = tuple(sorted(unclassified_relations))
|
||||
bounded_unclassified_relations = ordered_unclassified_relations[: plan.max_evidence]
|
||||
if len(ordered_unclassified_relations) > len(bounded_unclassified_relations):
|
||||
omissions.append(
|
||||
CapsuleOmissionV1(
|
||||
code="unclassified_relation_limit",
|
||||
subject=focus_node_id,
|
||||
detail_hash=canonical_hash(
|
||||
{
|
||||
"unclassified_relation_count": len(ordered_unclassified_relations),
|
||||
"returned_count": len(bounded_unclassified_relations),
|
||||
}
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
declared_categories = {
|
||||
relation_category(relation) for relation in self.project.descriptor.allowed_relations
|
||||
}
|
||||
gaps: list[EvidenceGapV1] = []
|
||||
incomplete = traversal_incomplete or bool(omissions)
|
||||
for requirement in plan.requirements:
|
||||
category = requirement.category
|
||||
if category not in declared_categories:
|
||||
gaps.append(
|
||||
EvidenceGapV1(
|
||||
code="category_not_declared",
|
||||
requirement_id=requirement.requirement_id,
|
||||
category=category,
|
||||
state="missing",
|
||||
check_complete=True,
|
||||
detail=(
|
||||
"No declared project relation maps to this required plan category."
|
||||
),
|
||||
)
|
||||
)
|
||||
elif category in returned_categories:
|
||||
continue
|
||||
elif incomplete:
|
||||
gaps.append(
|
||||
EvidenceGapV1(
|
||||
code="evidence_incomplete",
|
||||
requirement_id=requirement.requirement_id,
|
||||
category=category,
|
||||
state="incomplete",
|
||||
check_complete=False,
|
||||
detail=(
|
||||
"A bounded work, result, or token limit prevented a complete "
|
||||
"returned-evidence proof."
|
||||
),
|
||||
)
|
||||
)
|
||||
else:
|
||||
gaps.append(
|
||||
EvidenceGapV1(
|
||||
code="no_selected_evidence",
|
||||
requirement_id=requirement.requirement_id,
|
||||
category=category,
|
||||
state="missing",
|
||||
check_complete=True,
|
||||
detail=(
|
||||
"The complete bounded check found no selected graph evidence in "
|
||||
"this category."
|
||||
),
|
||||
)
|
||||
)
|
||||
for relation in bounded_unclassified_relations:
|
||||
gaps.append(
|
||||
EvidenceGapV1(
|
||||
code="unclassified_relation",
|
||||
requirement_id=f"relation.{relation}",
|
||||
category="unclassified",
|
||||
state="limitation",
|
||||
check_complete=True,
|
||||
detail="The project relation is preserved without inferred task semantics.",
|
||||
)
|
||||
)
|
||||
gaps.sort(
|
||||
key=lambda gap: (
|
||||
gap.requirement_id,
|
||||
gap.code,
|
||||
"" if gap.category is None else gap.category,
|
||||
)
|
||||
)
|
||||
return finalize_capsule(
|
||||
state="incomplete" if incomplete else "complete",
|
||||
task_kind=plan.task_kind,
|
||||
generation=generation,
|
||||
plan=plan,
|
||||
focus_state="resolved",
|
||||
focus_node_id=focus_node_id,
|
||||
focus_candidate_count=len(focus_rows),
|
||||
evidence=tuple(evidence),
|
||||
gaps=tuple(gaps),
|
||||
omissions=tuple(omissions),
|
||||
selected_count=len(selections),
|
||||
examined_edge_count=examined_edge_count,
|
||||
estimated_tokens=used_tokens,
|
||||
unclassified_relations=bounded_unclassified_relations,
|
||||
)
|
||||
|
||||
def get_node(self, node_id: str) -> dict[str, object]:
|
||||
with self._read_snapshot() as snapshot:
|
||||
row = snapshot.connection.execute(
|
||||
|
|
|
|||
|
|
@ -21,6 +21,7 @@ from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt
|
|||
from .policy import CapabilityMode, capability_mode, compose_effective_policy
|
||||
from .project import Project, project_root_fingerprint
|
||||
from .rendering import RenderService
|
||||
from .retrieval import MAX_TASK_EVIDENCE, TaskKind, build_retrieval_plan
|
||||
from .telemetry import request, stage
|
||||
from .viewer_manager import ViewerManagerClient
|
||||
|
||||
|
|
@ -43,6 +44,7 @@ READ_TOOLS = (
|
|||
"docforge_dependencies",
|
||||
"docforge_impact",
|
||||
"docforge_get_context",
|
||||
"docforge_get_task_context",
|
||||
"docforge_validate_project",
|
||||
"docforge_render_status",
|
||||
"docforge_visualize",
|
||||
|
|
@ -164,6 +166,7 @@ class DocForgeService:
|
|||
)
|
||||
self.visualization = ViewerManagerClient(self.index)
|
||||
self.context_provider = context_provider
|
||||
self.task_context_available = context_provider is compile_context
|
||||
self.binding_metadata = dict(binding_metadata or {})
|
||||
self.no_ast = self.policy.no_ast
|
||||
self.diagnostics = diagnostics
|
||||
|
|
@ -199,6 +202,14 @@ class DocForgeService:
|
|||
"enabled": True,
|
||||
"tools": [tool for tool in READ_TOOLS if tool in self.tool_surface],
|
||||
},
|
||||
"task_context": {
|
||||
"enabled": self.task_context_available,
|
||||
"reason": (
|
||||
None
|
||||
if self.task_context_available
|
||||
else "custom_context_policy_not_supported_by_task_context_v1"
|
||||
),
|
||||
},
|
||||
"proposal": {
|
||||
"surface_enabled": any(tool in self.tool_surface for tool in PROPOSAL_TOOLS),
|
||||
"mutation_access": proposal_access,
|
||||
|
|
@ -558,7 +569,11 @@ class DocForgeService:
|
|||
"adapter_policy": self.adapter_policy(),
|
||||
}
|
||||
recommended_workflow = [
|
||||
"docforge_get_context or targeted read tools",
|
||||
(
|
||||
"docforge_get_task_context, docforge_get_context, or targeted read tools"
|
||||
if self.task_context_available
|
||||
else "docforge_get_context or targeted read tools"
|
||||
),
|
||||
"make and verify one coherent implementation slice",
|
||||
"docforge_sync",
|
||||
]
|
||||
|
|
@ -582,8 +597,17 @@ class DocForgeService:
|
|||
"compiler-AST, or function-Logic extraction"
|
||||
),
|
||||
)
|
||||
if descriptor.profiles:
|
||||
if self.task_context_available:
|
||||
recommended_first_operation: dict[str, object] = {
|
||||
"tool": "docforge_get_task_context",
|
||||
"arguments": {
|
||||
"task_kind": "implementation",
|
||||
"task": "<describe the current task>",
|
||||
},
|
||||
"reason": "Begin with one bounded task-shaped context capsule.",
|
||||
}
|
||||
elif descriptor.profiles:
|
||||
recommended_first_operation = {
|
||||
"tool": "docforge_get_context",
|
||||
"arguments": {"profile": descriptor.profiles[0].profile_id},
|
||||
"reason": "Begin with one configured bounded context profile.",
|
||||
|
|
@ -831,6 +855,210 @@ class DocForgeService:
|
|||
operation_name="mcp.context",
|
||||
)
|
||||
|
||||
def task_context(
|
||||
self,
|
||||
task_kind: TaskKind,
|
||||
task: str,
|
||||
*,
|
||||
focus_node_id: str | None = None,
|
||||
budget: int | None = None,
|
||||
limit: int | None = None,
|
||||
cursor: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Return one task-shaped capsule from a single immutable graph generation."""
|
||||
|
||||
if not self.task_context_available:
|
||||
|
||||
def unavailable() -> dict[str, object]:
|
||||
raise DocForgeError(
|
||||
"task_context_unavailable",
|
||||
(
|
||||
"This binding uses a custom context provider; "
|
||||
"core task planning is unavailable"
|
||||
),
|
||||
)
|
||||
|
||||
return self.invoke(
|
||||
unavailable,
|
||||
synchronize=False,
|
||||
load_error_identity=False,
|
||||
operation_name="mcp.task_context",
|
||||
)
|
||||
|
||||
def operation() -> dict[str, object]:
|
||||
maximum_evidence = min(
|
||||
self.project.descriptor.limits.max_results,
|
||||
MAX_TASK_EVIDENCE,
|
||||
)
|
||||
selected_limit = page_limit(
|
||||
limit,
|
||||
default=min(20, maximum_evidence),
|
||||
maximum=maximum_evidence,
|
||||
)
|
||||
plan = build_retrieval_plan(
|
||||
self.project.descriptor,
|
||||
task_kind=task_kind,
|
||||
task=task,
|
||||
focus_node_id=focus_node_id,
|
||||
budget=budget,
|
||||
limit=maximum_evidence,
|
||||
effective_policy=self.policy.as_dict(),
|
||||
)
|
||||
result = self.index.task_context(plan)
|
||||
return self._page_task_context_result(
|
||||
result,
|
||||
selected_limit=selected_limit,
|
||||
cursor=cursor,
|
||||
)
|
||||
|
||||
return self.invoke(operation, operation_name="mcp.task_context")
|
||||
|
||||
def _page_task_context_result(
|
||||
self,
|
||||
result: dict[str, object],
|
||||
*,
|
||||
selected_limit: int,
|
||||
cursor: str | None,
|
||||
) -> dict[str, object]:
|
||||
capsule_value = result.get("capsule")
|
||||
if not isinstance(capsule_value, Mapping):
|
||||
raise DocForgeError(
|
||||
"invalid_task_context_result",
|
||||
"Task context did not return a versioned capsule",
|
||||
)
|
||||
capsule = dict(cast(Mapping[str, object], capsule_value))
|
||||
plan_value = capsule.get("plan")
|
||||
generation_value = capsule.get("generation")
|
||||
evidence_value = capsule.get("evidence")
|
||||
omissions_value = capsule.get("omissions")
|
||||
gaps_value = capsule.get("gaps")
|
||||
if (
|
||||
capsule.get("schema_version") != 1
|
||||
or not isinstance(plan_value, Mapping)
|
||||
or not isinstance(generation_value, Mapping)
|
||||
or not isinstance(evidence_value, list)
|
||||
or not isinstance(omissions_value, list)
|
||||
or not isinstance(gaps_value, list)
|
||||
or not isinstance(capsule.get("collection_hash"), str)
|
||||
or not isinstance(capsule.get("capsule_hash"), str)
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_task_context_result",
|
||||
"Task context capsule is malformed",
|
||||
)
|
||||
plan_payload = cast(Mapping[str, object], plan_value)
|
||||
generation = cast(Mapping[str, object], generation_value)
|
||||
evidence = cast(list[object], evidence_value)
|
||||
omissions = cast(list[object], omissions_value)
|
||||
gaps = cast(list[object], gaps_value)
|
||||
binding = {
|
||||
"project_id": result.get("project_id"),
|
||||
"project_root_fingerprint": result.get("project_root_fingerprint"),
|
||||
"adapter": result.get("adapter"),
|
||||
"revision": result.get("revision"),
|
||||
"source_hash": result.get("source_hash"),
|
||||
"index_schema_version": generation.get("index_schema_version"),
|
||||
"effective_policy_hash": plan_payload.get("effective_policy_hash"),
|
||||
"request_hash": plan_payload.get("request_hash"),
|
||||
"plan_hash": plan_payload.get("plan_hash"),
|
||||
"collection_hash": capsule["collection_hash"],
|
||||
"capsule_hash": capsule["capsule_hash"],
|
||||
}
|
||||
items = [
|
||||
*(("evidence", item) for item in evidence),
|
||||
*(("omission", item) for item in omissions),
|
||||
]
|
||||
position = decode_cursor(
|
||||
cursor,
|
||||
kind="task-context.items",
|
||||
binding=binding,
|
||||
total_count=len(items),
|
||||
)
|
||||
page_evidence: list[object] = []
|
||||
page_omissions: list[object] = []
|
||||
consumed = 0
|
||||
response_limited = False
|
||||
maximum = self.project.descriptor.limits.max_tool_output_chars
|
||||
|
||||
def page_result() -> dict[str, object]:
|
||||
pagination = page_receipt(
|
||||
kind="task-context.items",
|
||||
binding=binding,
|
||||
position=position,
|
||||
count=consumed,
|
||||
limit=selected_limit,
|
||||
total_count=len(items),
|
||||
)
|
||||
page_state = "incomplete" if response_limited else capsule.get("state")
|
||||
page_summary = {
|
||||
**cast(dict[str, object], capsule.get("summary", {})),
|
||||
"page_evidence_count": len(page_evidence),
|
||||
"page_omission_count": len(page_omissions),
|
||||
"page_item_count": consumed,
|
||||
}
|
||||
page_hash = canonical_hash(
|
||||
{
|
||||
"capsule_hash": capsule["capsule_hash"],
|
||||
"position": position,
|
||||
"page_state": page_state,
|
||||
"pagination": pagination,
|
||||
"summary": page_summary,
|
||||
"evidence": page_evidence,
|
||||
"gaps": gaps,
|
||||
"omissions": page_omissions,
|
||||
}
|
||||
)
|
||||
page_capsule = {
|
||||
**capsule,
|
||||
"evidence": page_evidence,
|
||||
"omissions": page_omissions,
|
||||
"page_state": page_state,
|
||||
"page_hash": page_hash,
|
||||
"pagination": pagination,
|
||||
"summary": page_summary,
|
||||
}
|
||||
return {
|
||||
**result,
|
||||
"capsule": page_capsule,
|
||||
"next_cursor": pagination["next_cursor"],
|
||||
"pagination": pagination,
|
||||
}
|
||||
|
||||
for kind, item in items[position:]:
|
||||
if consumed >= selected_limit:
|
||||
break
|
||||
destination = page_evidence if kind == "evidence" else page_omissions
|
||||
destination.append(item)
|
||||
consumed += 1
|
||||
decorated = {
|
||||
**page_result(),
|
||||
"server_version": SERVER_VERSION,
|
||||
"content_warning": CONTENT_WARNING,
|
||||
"staleness": "current",
|
||||
}
|
||||
if self._encoded_length(decorated) <= maximum:
|
||||
continue
|
||||
destination.pop()
|
||||
consumed -= 1
|
||||
response_limited = True
|
||||
if consumed == 0:
|
||||
subject = "unknown"
|
||||
if isinstance(item, Mapping):
|
||||
item_payload = cast(Mapping[str, object], item)
|
||||
candidate = item_payload.get("node_id") or item_payload.get("subject")
|
||||
if isinstance(candidate, str) and candidate:
|
||||
subject = candidate[:256]
|
||||
page_omissions.append(
|
||||
{
|
||||
"code": "response_limit",
|
||||
"subject": subject,
|
||||
"detail_hash": canonical_hash(cast(object, item)),
|
||||
}
|
||||
)
|
||||
consumed = 1
|
||||
break
|
||||
return page_result()
|
||||
|
||||
def _page_context_result(
|
||||
self,
|
||||
result: dict[str, object],
|
||||
|
|
@ -1151,6 +1379,26 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
|
|||
|
||||
return service.context(profile, budget, limit=limit, cursor=cursor)
|
||||
|
||||
@server.tool(name="docforge_get_task_context")
|
||||
def get_task_context(
|
||||
task_kind: TaskKind,
|
||||
task: str,
|
||||
focus_node_id: str | None = None,
|
||||
budget: int | None = None,
|
||||
limit: int | None = None,
|
||||
cursor: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Return one bounded task-shaped context capsule with explicit evidence gaps."""
|
||||
|
||||
return service.task_context(
|
||||
task_kind,
|
||||
task,
|
||||
focus_node_id=focus_node_id,
|
||||
budget=budget,
|
||||
limit=limit,
|
||||
cursor=cursor,
|
||||
)
|
||||
|
||||
@server.tool(name="docforge_validate_project")
|
||||
def validate_project() -> dict[str, Any]:
|
||||
"""Validate current canonical sources and graph without writing any project file."""
|
||||
|
|
@ -1201,6 +1449,7 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
|
|||
dependencies,
|
||||
impact,
|
||||
get_context,
|
||||
get_task_context,
|
||||
validate_project,
|
||||
render_status,
|
||||
visualize,
|
||||
|
|
|
|||
671
src/docforge/retrieval.py
Normal file
671
src/docforge/retrieval.py
Normal file
|
|
@ -0,0 +1,671 @@
|
|||
"""Versioned task-shaped retrieval plans and immutable context capsules."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, replace
|
||||
from typing import Literal, cast
|
||||
|
||||
from .errors import DocForgeError
|
||||
from .models import Edge, Node, ProjectDescriptor
|
||||
from .pagination import canonical_hash
|
||||
|
||||
TaskKind = Literal[
|
||||
"change",
|
||||
"implementation",
|
||||
"failure",
|
||||
"ownership",
|
||||
"test",
|
||||
"operation",
|
||||
"release",
|
||||
]
|
||||
TASK_KINDS: tuple[TaskKind, ...] = (
|
||||
"change",
|
||||
"implementation",
|
||||
"failure",
|
||||
"ownership",
|
||||
"test",
|
||||
"operation",
|
||||
"release",
|
||||
)
|
||||
|
||||
RelationCategory = Literal[
|
||||
"structure",
|
||||
"implementation",
|
||||
"dependency",
|
||||
"execution",
|
||||
"data",
|
||||
"evidence",
|
||||
"context",
|
||||
"unclassified",
|
||||
]
|
||||
BASE_RELATION_CATEGORIES: tuple[RelationCategory, ...] = (
|
||||
"structure",
|
||||
"implementation",
|
||||
"dependency",
|
||||
"execution",
|
||||
"data",
|
||||
"evidence",
|
||||
"context",
|
||||
)
|
||||
|
||||
RELATION_CATEGORIES: dict[RelationCategory, tuple[str, ...]] = {
|
||||
"structure": ("contains", "defined_in", "defines", "owns"),
|
||||
"implementation": (
|
||||
"implemented_by",
|
||||
"implements",
|
||||
"inherits",
|
||||
"inherits_from",
|
||||
),
|
||||
"dependency": ("depends_on", "imports"),
|
||||
"execution": ("activates", "calls", "dispatches_to", "launches"),
|
||||
"data": ("reads", "writes"),
|
||||
"evidence": ("documents", "governs", "proves", "tested_by", "verifies"),
|
||||
"context": ("relates_to",),
|
||||
"unclassified": (),
|
||||
}
|
||||
|
||||
TASK_REQUIREMENTS: dict[TaskKind, tuple[RelationCategory, ...]] = {
|
||||
"change": ("dependency",),
|
||||
"implementation": ("implementation",),
|
||||
"failure": ("execution",),
|
||||
"ownership": ("structure",),
|
||||
"test": ("evidence",),
|
||||
"operation": ("execution",),
|
||||
"release": ("evidence",),
|
||||
}
|
||||
|
||||
PLANNER_ID = "docforge.core.task-context"
|
||||
PLANNER_VERSION = 1
|
||||
MAX_TASK_EVIDENCE = 1_000
|
||||
MAX_TASK_CANDIDATE_EDGES = 100_000
|
||||
MAX_TASK_QUERY_CHARS = 10_000
|
||||
PROVENANCE_LIMITATIONS = (
|
||||
"evidence_type_unavailable",
|
||||
"extractor_identity_unavailable",
|
||||
"relationship_provenance_unavailable",
|
||||
"observation_time_unavailable",
|
||||
)
|
||||
|
||||
_RELATION_TO_CATEGORY = {
|
||||
relation: category
|
||||
for category, relations in RELATION_CATEGORIES.items()
|
||||
for relation in relations
|
||||
}
|
||||
|
||||
|
||||
def relation_category(relation: str) -> RelationCategory:
|
||||
"""Classify only versioned known aliases; preserve every other relation."""
|
||||
|
||||
return cast(RelationCategory, _RELATION_TO_CATEGORY.get(relation, "unclassified"))
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RetrievalStepV1:
|
||||
step_id: str
|
||||
operation: Literal["exact", "search", "outgoing", "incoming", "metadata"]
|
||||
relation_scope: Literal["none", "project_allowed"]
|
||||
relation_set_hash: str | None
|
||||
direction: Literal["none", "outgoing", "incoming"]
|
||||
depth: int
|
||||
limit: int
|
||||
required: bool
|
||||
evidence_role: str
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"step_id": self.step_id,
|
||||
"operation": self.operation,
|
||||
"relation_scope": self.relation_scope,
|
||||
"relation_set_hash": self.relation_set_hash,
|
||||
"direction": self.direction,
|
||||
"depth": self.depth,
|
||||
"limit": self.limit,
|
||||
"required": self.required,
|
||||
"evidence_role": self.evidence_role,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RetrievalRequirementV1:
|
||||
requirement_id: str
|
||||
category: RelationCategory
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"requirement_id": self.requirement_id,
|
||||
"check": "selected_relation_category",
|
||||
"category": self.category,
|
||||
"required": True,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RetrievalPlanV1:
|
||||
schema_version: Literal[1]
|
||||
planner_id: str
|
||||
planner_version: int
|
||||
task_kind: TaskKind
|
||||
task_query: str
|
||||
focus_node_id: str | None
|
||||
request_hash: str
|
||||
effective_policy_hash: str
|
||||
max_evidence: int
|
||||
max_tokens: int
|
||||
max_depth: int
|
||||
max_candidate_edges: int
|
||||
category_order: tuple[RelationCategory, ...]
|
||||
steps: tuple[RetrievalStepV1, ...]
|
||||
requirements: tuple[RetrievalRequirementV1, ...]
|
||||
plan_hash: str
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return self.payload(include_hash=True)
|
||||
|
||||
def payload(self, *, include_hash: bool) -> dict[str, object]:
|
||||
result: dict[str, object] = {
|
||||
"schema_version": self.schema_version,
|
||||
"planner": {
|
||||
"id": self.planner_id,
|
||||
"version": self.planner_version,
|
||||
},
|
||||
"task_kind": self.task_kind,
|
||||
"request_hash": self.request_hash,
|
||||
"effective_policy_hash": self.effective_policy_hash,
|
||||
"focus_node_id": self.focus_node_id,
|
||||
"limits": {
|
||||
"max_evidence": self.max_evidence,
|
||||
"max_tokens": self.max_tokens,
|
||||
"max_depth": self.max_depth,
|
||||
"max_candidate_edges": self.max_candidate_edges,
|
||||
},
|
||||
"category_order": list(self.category_order),
|
||||
"steps": [step.as_dict() for step in self.steps],
|
||||
"requirements": [requirement.as_dict() for requirement in self.requirements],
|
||||
}
|
||||
if include_hash:
|
||||
result["plan_hash"] = self.plan_hash
|
||||
return result
|
||||
|
||||
|
||||
def build_retrieval_plan(
|
||||
descriptor: ProjectDescriptor,
|
||||
*,
|
||||
task_kind: str,
|
||||
task: str,
|
||||
focus_node_id: str | None,
|
||||
budget: int | None,
|
||||
limit: int | None,
|
||||
effective_policy: dict[str, object],
|
||||
) -> RetrievalPlanV1:
|
||||
"""Derive one fixed plan from bounded inputs rather than accepting caller operations."""
|
||||
|
||||
return _build_retrieval_plan(
|
||||
descriptor,
|
||||
task_kind=task_kind,
|
||||
task=task,
|
||||
focus_node_id=focus_node_id,
|
||||
budget=budget,
|
||||
limit=limit,
|
||||
effective_policy_hash=canonical_hash(effective_policy),
|
||||
)
|
||||
|
||||
|
||||
def validate_retrieval_plan(
|
||||
plan: RetrievalPlanV1,
|
||||
descriptor: ProjectDescriptor,
|
||||
) -> RetrievalPlanV1:
|
||||
"""Reject forged, stale-shape, or internally inconsistent public plan objects."""
|
||||
|
||||
try:
|
||||
expected = _build_retrieval_plan(
|
||||
descriptor,
|
||||
task_kind=plan.task_kind,
|
||||
task=plan.task_query,
|
||||
focus_node_id=plan.focus_node_id,
|
||||
budget=plan.max_tokens,
|
||||
limit=plan.max_evidence,
|
||||
effective_policy_hash=plan.effective_policy_hash,
|
||||
)
|
||||
except (AttributeError, TypeError, DocForgeError) as error:
|
||||
raise DocForgeError(
|
||||
"invalid_retrieval_plan",
|
||||
"Task retrieval plan is malformed or outside the fixed version-1 contract",
|
||||
) from error
|
||||
if plan != expected:
|
||||
raise DocForgeError(
|
||||
"invalid_retrieval_plan",
|
||||
"Task retrieval plan does not match its fixed version-1 derivation",
|
||||
)
|
||||
return plan
|
||||
|
||||
|
||||
def _build_retrieval_plan(
|
||||
descriptor: ProjectDescriptor,
|
||||
*,
|
||||
task_kind: str,
|
||||
task: str,
|
||||
focus_node_id: str | None,
|
||||
budget: int | None,
|
||||
limit: int | None,
|
||||
effective_policy_hash: str,
|
||||
) -> RetrievalPlanV1:
|
||||
if task_kind not in TASK_KINDS:
|
||||
raise DocForgeError(
|
||||
"invalid_task_kind",
|
||||
"Task context kind is unsupported",
|
||||
task_kind=task_kind,
|
||||
allowed=list(TASK_KINDS),
|
||||
)
|
||||
selected_kind: TaskKind = task_kind # type: ignore[assignment]
|
||||
normalized_task = task.strip()
|
||||
if not normalized_task or len(normalized_task) > min(
|
||||
descriptor.limits.max_query_chars,
|
||||
MAX_TASK_QUERY_CHARS,
|
||||
):
|
||||
raise DocForgeError(
|
||||
"invalid_task_focus",
|
||||
"Task description is empty or exceeds the configured query limit",
|
||||
)
|
||||
if focus_node_id is not None and (not focus_node_id or len(focus_node_id) > 256):
|
||||
raise DocForgeError("invalid_task_focus", "Task focus node ID is invalid")
|
||||
selected_budget = _bounded_value(
|
||||
budget,
|
||||
default=min(8_000, descriptor.limits.max_context_tokens),
|
||||
maximum=descriptor.limits.max_context_tokens,
|
||||
code="invalid_budget",
|
||||
)
|
||||
selected_limit = _bounded_value(
|
||||
limit,
|
||||
default=min(20, descriptor.limits.max_results),
|
||||
maximum=min(descriptor.limits.max_results, MAX_TASK_EVIDENCE),
|
||||
code="invalid_limit",
|
||||
)
|
||||
if not _is_sha256(effective_policy_hash):
|
||||
raise DocForgeError(
|
||||
"invalid_retrieval_plan",
|
||||
"Effective policy identity is not a SHA-256 value",
|
||||
)
|
||||
selected_depth = min(2, descriptor.limits.max_traversal_depth)
|
||||
requirements = tuple(
|
||||
RetrievalRequirementV1(
|
||||
requirement_id=f"{selected_kind}.{category}",
|
||||
category=category,
|
||||
)
|
||||
for category in TASK_REQUIREMENTS[selected_kind]
|
||||
)
|
||||
category_order: tuple[RelationCategory, ...] = (
|
||||
*TASK_REQUIREMENTS[selected_kind],
|
||||
*(
|
||||
category
|
||||
for category in BASE_RELATION_CATEGORIES
|
||||
if category not in TASK_REQUIREMENTS[selected_kind]
|
||||
),
|
||||
"unclassified",
|
||||
)
|
||||
relation_set_hash = canonical_hash(sorted(descriptor.allowed_relations))
|
||||
focus_operation: Literal["exact", "search"] = "exact" if focus_node_id else "search"
|
||||
steps = (
|
||||
RetrievalStepV1(
|
||||
step_id="focus",
|
||||
operation=focus_operation,
|
||||
relation_scope="none",
|
||||
relation_set_hash=None,
|
||||
direction="none",
|
||||
depth=0,
|
||||
limit=1,
|
||||
required=True,
|
||||
evidence_role="focus",
|
||||
),
|
||||
RetrievalStepV1(
|
||||
step_id="outgoing",
|
||||
operation="outgoing",
|
||||
relation_scope="project_allowed",
|
||||
relation_set_hash=relation_set_hash,
|
||||
direction="outgoing",
|
||||
depth=selected_depth,
|
||||
limit=selected_limit,
|
||||
required=False,
|
||||
evidence_role="related",
|
||||
),
|
||||
RetrievalStepV1(
|
||||
step_id="incoming",
|
||||
operation="incoming",
|
||||
relation_scope="project_allowed",
|
||||
relation_set_hash=relation_set_hash,
|
||||
direction="incoming",
|
||||
depth=selected_depth,
|
||||
limit=selected_limit,
|
||||
required=False,
|
||||
evidence_role="related",
|
||||
),
|
||||
RetrievalStepV1(
|
||||
step_id="metadata",
|
||||
operation="metadata",
|
||||
relation_scope="none",
|
||||
relation_set_hash=None,
|
||||
direction="none",
|
||||
depth=0,
|
||||
limit=selected_limit,
|
||||
required=True,
|
||||
evidence_role="provenance",
|
||||
),
|
||||
)
|
||||
request_hash = canonical_hash(
|
||||
{
|
||||
"task_kind": selected_kind,
|
||||
"task": normalized_task,
|
||||
"focus_node_id": focus_node_id,
|
||||
"budget": selected_budget,
|
||||
"limit": selected_limit,
|
||||
}
|
||||
)
|
||||
placeholder = RetrievalPlanV1(
|
||||
schema_version=1,
|
||||
planner_id=PLANNER_ID,
|
||||
planner_version=PLANNER_VERSION,
|
||||
task_kind=selected_kind,
|
||||
task_query=normalized_task,
|
||||
focus_node_id=focus_node_id,
|
||||
request_hash=request_hash,
|
||||
effective_policy_hash=effective_policy_hash,
|
||||
max_evidence=selected_limit,
|
||||
max_tokens=selected_budget,
|
||||
max_depth=selected_depth,
|
||||
max_candidate_edges=min(
|
||||
(selected_limit + 1) ** 2,
|
||||
MAX_TASK_CANDIDATE_EDGES,
|
||||
),
|
||||
category_order=category_order,
|
||||
steps=steps,
|
||||
requirements=requirements,
|
||||
plan_hash="",
|
||||
)
|
||||
return replace(
|
||||
placeholder,
|
||||
plan_hash=canonical_hash(placeholder.payload(include_hash=False)),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CapsuleRelationshipV1:
|
||||
source_id: str
|
||||
relation: str
|
||||
target_id: str
|
||||
direction: Literal["outgoing", "incoming"]
|
||||
category: RelationCategory
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"source_id": self.source_id,
|
||||
"relation": self.relation,
|
||||
"target_id": self.target_id,
|
||||
"direction": self.direction,
|
||||
"category": self.category,
|
||||
"provenance": "validated_graph_edge_without_source_provenance",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CapsuleEvidenceV1:
|
||||
evidence_hash: str
|
||||
role: Literal["focus", "related"]
|
||||
reason_code: Literal["exact_focus", "lexical_focus", "relationship_path"]
|
||||
node: Node
|
||||
depth: int
|
||||
relationship_path: tuple[CapsuleRelationshipV1, ...]
|
||||
relationship_reasons: tuple[CapsuleRelationshipV1, ...]
|
||||
estimated_tokens: int
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
result = self.payload()
|
||||
return {"evidence_hash": self.evidence_hash, **result}
|
||||
|
||||
def payload(self) -> dict[str, object]:
|
||||
result: dict[str, object] = {
|
||||
"role": self.role,
|
||||
"reason_code": self.reason_code,
|
||||
"node_id": self.node.node_id,
|
||||
"title": self.node.title,
|
||||
"family": self.node.family,
|
||||
"authority": self.node.authority,
|
||||
"status": self.node.status,
|
||||
"tags": list(self.node.tags),
|
||||
"summary": self.node.summary,
|
||||
"text": _node_text(self.node),
|
||||
"estimated_tokens": self.estimated_tokens,
|
||||
"source": {
|
||||
"path": self.node.source_path,
|
||||
"anchor": self.node.source_anchor,
|
||||
"content_hash": self.node.content_hash,
|
||||
},
|
||||
"depth": self.depth,
|
||||
"relationship_path": [
|
||||
relationship.as_dict() for relationship in self.relationship_path
|
||||
],
|
||||
"relationship_reasons": [
|
||||
relationship.as_dict() for relationship in self.relationship_reasons
|
||||
],
|
||||
"provenance_limitations": list(PROVENANCE_LIMITATIONS),
|
||||
}
|
||||
return result
|
||||
|
||||
|
||||
def capsule_evidence(
|
||||
*,
|
||||
role: Literal["focus", "related"],
|
||||
reason_code: Literal["exact_focus", "lexical_focus", "relationship_path"],
|
||||
node: Node,
|
||||
depth: int,
|
||||
relationship_path: tuple[CapsuleRelationshipV1, ...],
|
||||
relationship_reasons: tuple[CapsuleRelationshipV1, ...],
|
||||
) -> CapsuleEvidenceV1:
|
||||
tokens = estimate_tokens(_node_text(node))
|
||||
placeholder = CapsuleEvidenceV1(
|
||||
evidence_hash="",
|
||||
role=role,
|
||||
reason_code=reason_code,
|
||||
node=node,
|
||||
depth=depth,
|
||||
relationship_path=relationship_path,
|
||||
relationship_reasons=relationship_reasons,
|
||||
estimated_tokens=tokens,
|
||||
)
|
||||
return replace(
|
||||
placeholder,
|
||||
evidence_hash=canonical_hash(placeholder.payload()),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EvidenceGapV1:
|
||||
code: Literal[
|
||||
"focus_not_found",
|
||||
"focus_ambiguous",
|
||||
"category_not_declared",
|
||||
"no_selected_evidence",
|
||||
"evidence_incomplete",
|
||||
"unclassified_relation",
|
||||
]
|
||||
requirement_id: str
|
||||
category: RelationCategory | None
|
||||
state: Literal["missing", "incomplete", "blocked", "limitation"]
|
||||
check_complete: bool
|
||||
detail: str
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"code": self.code,
|
||||
"requirement_id": self.requirement_id,
|
||||
"category": self.category,
|
||||
"state": self.state,
|
||||
"check_complete": self.check_complete,
|
||||
"detail": self.detail,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CapsuleOmissionV1:
|
||||
code: Literal[
|
||||
"result_limit",
|
||||
"token_budget",
|
||||
"response_limit",
|
||||
"edge_examination_limit",
|
||||
"unclassified_relation_limit",
|
||||
]
|
||||
subject: str
|
||||
detail_hash: str
|
||||
|
||||
def as_dict(self) -> dict[str, str]:
|
||||
return {
|
||||
"code": self.code,
|
||||
"subject": self.subject,
|
||||
"detail_hash": self.detail_hash,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContextCapsuleV1:
|
||||
schema_version: Literal[1]
|
||||
state: Literal["complete", "incomplete", "blocked"]
|
||||
task_kind: TaskKind
|
||||
generation: tuple[tuple[str, object], ...]
|
||||
plan: RetrievalPlanV1
|
||||
focus_state: Literal["resolved", "not_found", "ambiguous"]
|
||||
focus_node_id: str | None
|
||||
focus_candidate_count: int
|
||||
evidence: tuple[CapsuleEvidenceV1, ...]
|
||||
gaps: tuple[EvidenceGapV1, ...]
|
||||
omissions: tuple[CapsuleOmissionV1, ...]
|
||||
selected_count: int
|
||||
examined_edge_count: int
|
||||
estimated_tokens: int
|
||||
unclassified_relations: tuple[str, ...]
|
||||
collection_hash: str
|
||||
capsule_hash: str
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return self.payload(include_hashes=True)
|
||||
|
||||
def payload(self, *, include_hashes: bool) -> dict[str, object]:
|
||||
evidence = [item.as_dict() for item in self.evidence]
|
||||
gaps = [gap.as_dict() for gap in self.gaps]
|
||||
omissions = [omission.as_dict() for omission in self.omissions]
|
||||
result: dict[str, object] = {
|
||||
"schema_version": self.schema_version,
|
||||
"state": self.state,
|
||||
"task_kind": self.task_kind,
|
||||
"generation": dict(self.generation),
|
||||
"plan": self.plan.as_dict(),
|
||||
"focus": {
|
||||
"state": self.focus_state,
|
||||
"node_id": self.focus_node_id,
|
||||
"candidate_count": self.focus_candidate_count,
|
||||
},
|
||||
"evidence": evidence,
|
||||
"gaps": gaps,
|
||||
"omissions": omissions,
|
||||
"summary": {
|
||||
"evidence_count": len(evidence),
|
||||
"gap_count": len(gaps),
|
||||
"omission_count": len(omissions),
|
||||
"selected_count": self.selected_count,
|
||||
"examined_edge_count": self.examined_edge_count,
|
||||
"estimated_tokens": self.estimated_tokens,
|
||||
"unclassified_relations": list(self.unclassified_relations),
|
||||
},
|
||||
}
|
||||
if include_hashes:
|
||||
result["collection_hash"] = self.collection_hash
|
||||
result["capsule_hash"] = self.capsule_hash
|
||||
return result
|
||||
|
||||
|
||||
def finalize_capsule(
|
||||
*,
|
||||
state: Literal["complete", "incomplete", "blocked"],
|
||||
task_kind: TaskKind,
|
||||
generation: dict[str, object],
|
||||
plan: RetrievalPlanV1,
|
||||
focus_state: Literal["resolved", "not_found", "ambiguous"],
|
||||
focus_node_id: str | None,
|
||||
focus_candidate_count: int,
|
||||
evidence: tuple[CapsuleEvidenceV1, ...],
|
||||
gaps: tuple[EvidenceGapV1, ...],
|
||||
omissions: tuple[CapsuleOmissionV1, ...],
|
||||
selected_count: int,
|
||||
examined_edge_count: int,
|
||||
estimated_tokens: int,
|
||||
unclassified_relations: tuple[str, ...],
|
||||
) -> ContextCapsuleV1:
|
||||
collection_hash = canonical_hash(
|
||||
{
|
||||
"generation": generation,
|
||||
"plan_hash": plan.plan_hash,
|
||||
"evidence": [item.evidence_hash for item in evidence],
|
||||
"gaps": [gap.as_dict() for gap in gaps],
|
||||
"omissions": [omission.as_dict() for omission in omissions],
|
||||
}
|
||||
)
|
||||
placeholder = ContextCapsuleV1(
|
||||
schema_version=1,
|
||||
state=state,
|
||||
task_kind=task_kind,
|
||||
generation=tuple(generation.items()),
|
||||
plan=plan,
|
||||
focus_state=focus_state,
|
||||
focus_node_id=focus_node_id,
|
||||
focus_candidate_count=focus_candidate_count,
|
||||
evidence=evidence,
|
||||
gaps=gaps,
|
||||
omissions=omissions,
|
||||
selected_count=selected_count,
|
||||
examined_edge_count=examined_edge_count,
|
||||
estimated_tokens=estimated_tokens,
|
||||
unclassified_relations=unclassified_relations,
|
||||
collection_hash=collection_hash,
|
||||
capsule_hash="",
|
||||
)
|
||||
capsule_hash = canonical_hash(
|
||||
{
|
||||
**placeholder.payload(include_hashes=False),
|
||||
"collection_hash": collection_hash,
|
||||
}
|
||||
)
|
||||
return replace(placeholder, capsule_hash=capsule_hash)
|
||||
|
||||
|
||||
def estimate_tokens(text: str) -> int:
|
||||
return max(1, (len(text) + 3) // 4)
|
||||
|
||||
|
||||
def edge_tuple(edge: Edge) -> tuple[str, str, str]:
|
||||
return edge.source_id, edge.relation, edge.target_id
|
||||
|
||||
|
||||
def _node_text(node: Node) -> str:
|
||||
return (
|
||||
f"ID: {node.node_id}\nTitle: {node.title}\nFamily: {node.family}\n"
|
||||
f"Authority: {node.authority}\nStatus: {node.status}\nSource: {node.source_path}\n"
|
||||
f"Summary: {node.summary}\n\n{node.content}"
|
||||
)
|
||||
|
||||
|
||||
def _bounded_value(
|
||||
value: int | None,
|
||||
*,
|
||||
default: int,
|
||||
maximum: int,
|
||||
code: str,
|
||||
) -> int:
|
||||
selected = default if value is None else value
|
||||
if type(selected) is not int or selected < 1 or selected > maximum:
|
||||
raise DocForgeError(code, "Task context limit is outside the configured range")
|
||||
return selected
|
||||
|
||||
|
||||
def _is_sha256(value: object) -> bool:
|
||||
return (
|
||||
isinstance(value, str)
|
||||
and len(value) == 64
|
||||
and all(character in "0123456789abcdef" for character in value)
|
||||
)
|
||||
|
|
@ -91,6 +91,7 @@ OPERATION_NAMES = frozenset(
|
|||
"mcp.dependencies",
|
||||
"mcp.impact",
|
||||
"mcp.context",
|
||||
"mcp.task_context",
|
||||
"mcp.validate_project",
|
||||
"mcp.render_status",
|
||||
"mcp.visualize",
|
||||
|
|
|
|||
|
|
@ -55,8 +55,10 @@ from docforge.visualization import VisualizationIndexSnapshot
|
|||
class Loader:
|
||||
def __init__(self, projection: AdapterProjection) -> None:
|
||||
self.projection = projection
|
||||
self.load_calls = 0
|
||||
|
||||
def load_projection(self) -> AdapterProjection:
|
||||
self.load_calls += 1
|
||||
return self.projection
|
||||
|
||||
|
||||
|
|
@ -866,8 +868,9 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory).resolve()
|
||||
fixture = AdapterContractTests()
|
||||
loader = Loader(fixture.projection(root))
|
||||
project = AdapterProject(
|
||||
Loader(fixture.projection(root)),
|
||||
loader,
|
||||
cache_root=root / ".cache" / "adapter-read-only",
|
||||
)
|
||||
index = ProjectIndex(project)
|
||||
|
|
@ -903,6 +906,14 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
context = await session.call_tool(
|
||||
"docforge_get_context", {"profile": "fixture", "budget": 321}
|
||||
)
|
||||
load_calls = loader.load_calls
|
||||
task_context = await session.call_tool(
|
||||
"docforge_get_task_context",
|
||||
{
|
||||
"task_kind": "change",
|
||||
"task": "Do not load the legacy projection for this capability error",
|
||||
},
|
||||
)
|
||||
|
||||
self.assertEqual(READ_TOOLS, tuple(tool.name for tool in tools.tools))
|
||||
self.assertEqual("adapter-fixture", info.structuredContent["project_id"])
|
||||
|
|
@ -915,6 +926,12 @@ class AdapterReadOnlyMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertFalse(contract.structuredContent["isolated_changeset_writes_allowed"])
|
||||
self.assertEqual("fixture", context.structuredContent["profile"])
|
||||
self.assertEqual([("fixture", 321)], calls)
|
||||
self.assertEqual("error", task_context.structuredContent["status"])
|
||||
self.assertEqual(
|
||||
"task_context_unavailable",
|
||||
task_context.structuredContent["error"]["code"],
|
||||
)
|
||||
self.assertEqual(load_calls, loader.load_calls)
|
||||
self.assertFalse(project.descriptor.changeset_root.exists())
|
||||
|
||||
async def test_adapter_project_proposals_require_explicit_policy_and_stay_isolated(
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ from contextlib import contextmanager
|
|||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from jsonschema import Draft202012Validator
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
from mcp.shared.memory import create_connected_server_and_client_session
|
||||
|
|
@ -34,6 +35,9 @@ from docforge.viewer_manager import ViewerManager
|
|||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURES = ROOT / "tests" / "fixtures"
|
||||
CAPSULE_SCHEMA = json.loads(
|
||||
(ROOT / "schemas" / "context-capsule.schema.json").read_text(encoding="utf-8")
|
||||
)
|
||||
|
||||
|
||||
class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
|
||||
|
|
@ -85,6 +89,7 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertNotIn("limit", tools[name].inputSchema.get("required", []))
|
||||
for name in (
|
||||
"docforge_get_context",
|
||||
"docforge_get_task_context",
|
||||
"docforge_list_changesets",
|
||||
"docforge_get_changeset",
|
||||
"docforge_validate_changeset",
|
||||
|
|
@ -93,6 +98,22 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
for field in ("limit", "cursor"):
|
||||
self.assertIn(field, tools[name].inputSchema["properties"])
|
||||
self.assertNotIn(field, tools[name].inputSchema.get("required", []))
|
||||
self.assertEqual(
|
||||
{"task_kind", "task"},
|
||||
set(tools["docforge_get_task_context"].inputSchema["required"]),
|
||||
)
|
||||
self.assertEqual(
|
||||
[
|
||||
"change",
|
||||
"implementation",
|
||||
"failure",
|
||||
"ownership",
|
||||
"test",
|
||||
"operation",
|
||||
"release",
|
||||
],
|
||||
tools["docforge_get_task_context"].inputSchema["properties"]["task_kind"]["enum"],
|
||||
)
|
||||
self.assertIn(
|
||||
"deep",
|
||||
tools["docforge_render_status"].inputSchema["properties"],
|
||||
|
|
@ -160,6 +181,316 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
self.assertEqual(0, diagnostics["counters"]["project_loads"])
|
||||
self.assertEqual(0, diagnostics["counters"]["source_files_parsed"])
|
||||
|
||||
async def test_task_context_is_hash_stable_paged_and_generation_bound(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
ProjectIndex(Project.open(root)).build()
|
||||
async with create_connected_server_and_client_session(
|
||||
create_server(root, capability_mode="read", diagnostics=True),
|
||||
raise_exceptions=True,
|
||||
) as session:
|
||||
first = await session.call_tool(
|
||||
"docforge_get_task_context",
|
||||
{
|
||||
"task_kind": "change",
|
||||
"task": "Change the editing workflow",
|
||||
"focus_node_id": "guide.workflow",
|
||||
"limit": 1,
|
||||
},
|
||||
)
|
||||
first_capsule = first.structuredContent["capsule"]
|
||||
Draft202012Validator(CAPSULE_SCHEMA).validate(first_capsule)
|
||||
cursor = first_capsule["pagination"]["next_cursor"]
|
||||
hashes = {
|
||||
first_capsule["capsule_hash"],
|
||||
first_capsule["collection_hash"],
|
||||
first_capsule["plan"]["plan_hash"],
|
||||
}
|
||||
evidence_ids = [item["node_id"] for item in first_capsule["evidence"]]
|
||||
while cursor is not None:
|
||||
page = await session.call_tool(
|
||||
"docforge_get_task_context",
|
||||
{
|
||||
"task_kind": "change",
|
||||
"task": "Change the editing workflow",
|
||||
"focus_node_id": "guide.workflow",
|
||||
"limit": 2,
|
||||
"cursor": cursor,
|
||||
},
|
||||
)
|
||||
capsule = page.structuredContent["capsule"]
|
||||
self.assertEqual(first_capsule["capsule_hash"], capsule["capsule_hash"])
|
||||
self.assertEqual(
|
||||
first_capsule["collection_hash"],
|
||||
capsule["collection_hash"],
|
||||
)
|
||||
self.assertEqual(
|
||||
first_capsule["plan"]["plan_hash"],
|
||||
capsule["plan"]["plan_hash"],
|
||||
)
|
||||
evidence_ids.extend(item["node_id"] for item in capsule["evidence"])
|
||||
cursor = capsule["pagination"]["next_cursor"]
|
||||
|
||||
self.assertEqual(
|
||||
["guide.workflow", "guide.foundation", "proof.validation"],
|
||||
evidence_ids,
|
||||
)
|
||||
self.assertEqual(3, len(hashes))
|
||||
self.assertEqual(
|
||||
"mcp.task_context",
|
||||
first.structuredContent["diagnostics"]["operation"],
|
||||
)
|
||||
counters = first.structuredContent["diagnostics"]["counters"]
|
||||
self.assertEqual(0, counters["project_loads"])
|
||||
self.assertEqual(0, counters["source_files_parsed"])
|
||||
self.assertEqual(0, counters["adapter_projection_loads"])
|
||||
self.assertEqual(0, counters["adapter_source_extractions"])
|
||||
self.assertEqual(0, counters["index_builds"])
|
||||
self.assertLessEqual(
|
||||
len(json.dumps(first.structuredContent, separators=(",", ":"))),
|
||||
Project.open(root).descriptor.limits.max_tool_output_chars,
|
||||
)
|
||||
|
||||
for changed_arguments in (
|
||||
{"task": "A different task"},
|
||||
{"task_kind": "failure"},
|
||||
{"focus_node_id": "guide.foundation"},
|
||||
{"budget": 100},
|
||||
):
|
||||
arguments = {
|
||||
"task_kind": "change",
|
||||
"task": "Change the editing workflow",
|
||||
"focus_node_id": "guide.workflow",
|
||||
"limit": 1,
|
||||
"cursor": first_capsule["pagination"]["next_cursor"],
|
||||
**changed_arguments,
|
||||
}
|
||||
changed_cursor = await session.call_tool(
|
||||
"docforge_get_task_context",
|
||||
arguments,
|
||||
)
|
||||
self.assertEqual(
|
||||
"stale_cursor",
|
||||
changed_cursor.structuredContent["error"]["code"],
|
||||
)
|
||||
|
||||
different_policy = DocForgeService(
|
||||
Project.open(root),
|
||||
capability_mode_name="proposal",
|
||||
).task_context(
|
||||
"change",
|
||||
"Change the editing workflow",
|
||||
focus_node_id="guide.workflow",
|
||||
limit=1,
|
||||
cursor=first_capsule["pagination"]["next_cursor"],
|
||||
)
|
||||
self.assertEqual("stale_cursor", different_policy["error"]["code"])
|
||||
|
||||
changed = root / "docs/content/foundation.md"
|
||||
changed.write_text(
|
||||
changed.read_text(encoding="utf-8") + "\nNew generation.\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
stale = await session.call_tool(
|
||||
"docforge_get_task_context",
|
||||
{
|
||||
"task_kind": "change",
|
||||
"task": "Change the editing workflow",
|
||||
"focus_node_id": "guide.workflow",
|
||||
"limit": 1,
|
||||
"cursor": first_capsule["pagination"]["next_cursor"],
|
||||
},
|
||||
)
|
||||
self.assertEqual("error", stale.structuredContent["status"])
|
||||
self.assertEqual("stale_cursor", stale.structuredContent["error"]["code"])
|
||||
|
||||
async def test_custom_context_policy_does_not_silently_gain_task_planning(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
project = Project.open(root)
|
||||
ProjectIndex(project).build()
|
||||
service = DocForgeService(
|
||||
project,
|
||||
context_provider=lambda index, profile, budget: {
|
||||
"profile": profile,
|
||||
"budget": budget,
|
||||
"entries": [],
|
||||
"omissions": [],
|
||||
},
|
||||
capability_mode_name="read",
|
||||
)
|
||||
async with create_connected_server_and_client_session(
|
||||
_create_bound_server(service, read_only=True),
|
||||
raise_exceptions=True,
|
||||
) as session:
|
||||
bootstrap = await session.call_tool("docforge_bootstrap", {})
|
||||
with (
|
||||
mock.patch.object(
|
||||
service.project,
|
||||
"load",
|
||||
side_effect=AssertionError("capability errors must not load"),
|
||||
),
|
||||
mock.patch.object(
|
||||
service.index,
|
||||
"check",
|
||||
side_effect=AssertionError("capability errors must not check"),
|
||||
),
|
||||
mock.patch.object(
|
||||
service.index,
|
||||
"build",
|
||||
side_effect=AssertionError("capability errors must not build"),
|
||||
),
|
||||
mock.patch.object(
|
||||
service.index,
|
||||
"synchronize",
|
||||
side_effect=AssertionError("capability errors must not synchronize"),
|
||||
),
|
||||
):
|
||||
result = await session.call_tool(
|
||||
"docforge_get_task_context",
|
||||
{
|
||||
"task_kind": "change",
|
||||
"task": "Do not widen the adapter context policy",
|
||||
},
|
||||
)
|
||||
self.assertEqual(
|
||||
"docforge_get_context",
|
||||
bootstrap.structuredContent["session_contract"]["recommended_first_operation"][
|
||||
"tool"
|
||||
],
|
||||
)
|
||||
self.assertFalse(bootstrap.structuredContent["capabilities"]["task_context"]["enabled"])
|
||||
self.assertEqual("error", result.structuredContent["status"])
|
||||
self.assertEqual(
|
||||
"task_context_unavailable",
|
||||
result.structuredContent["error"]["code"],
|
||||
)
|
||||
|
||||
def test_task_context_page_hash_binds_final_page_envelope(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
descriptor = root / ".docforge" / "project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8").replace(
|
||||
"max_context_tokens = 2000",
|
||||
"max_context_tokens = 2000\nmax_tool_output_chars = 8000",
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
project = Project.open(root)
|
||||
ProjectIndex(project).build()
|
||||
service = DocForgeService(project, capability_mode_name="read")
|
||||
one = service.task_context(
|
||||
"change",
|
||||
"Change the editing workflow",
|
||||
focus_node_id="guide.workflow",
|
||||
limit=1,
|
||||
)
|
||||
two = service.task_context(
|
||||
"change",
|
||||
"Change the editing workflow",
|
||||
focus_node_id="guide.workflow",
|
||||
limit=2,
|
||||
)
|
||||
one_capsule = one["capsule"]
|
||||
two_capsule = two["capsule"]
|
||||
self.assertEqual(
|
||||
["guide.workflow"],
|
||||
[item["node_id"] for item in one_capsule["evidence"]],
|
||||
)
|
||||
self.assertEqual(
|
||||
["guide.workflow"],
|
||||
[item["node_id"] for item in two_capsule["evidence"]],
|
||||
)
|
||||
self.assertEqual("complete", one_capsule["page_state"])
|
||||
self.assertEqual("incomplete", two_capsule["page_state"])
|
||||
self.assertNotEqual(one_capsule["page_hash"], two_capsule["page_hash"])
|
||||
self.assertNotEqual(one_capsule["pagination"], two_capsule["pagination"])
|
||||
self.assertLessEqual(
|
||||
len(json.dumps(one, sort_keys=True, separators=(",", ":"))),
|
||||
8_000,
|
||||
)
|
||||
self.assertLessEqual(
|
||||
len(json.dumps(two, sort_keys=True, separators=(",", ":"))),
|
||||
8_000,
|
||||
)
|
||||
|
||||
def test_task_context_default_page_clamps_to_small_project_limit(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
descriptor = root / ".docforge" / "project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8").replace(
|
||||
"max_results = 20",
|
||||
"max_results = 2",
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
project = Project.open(root)
|
||||
ProjectIndex(project).build()
|
||||
result = DocForgeService(
|
||||
project,
|
||||
capability_mode_name="read",
|
||||
).task_context(
|
||||
"change",
|
||||
"Change the editing workflow",
|
||||
focus_node_id="guide.workflow",
|
||||
)
|
||||
self.assertEqual(2, result["pagination"]["limit"])
|
||||
self.assertLessEqual(result["pagination"]["returned_count"], 2)
|
||||
|
||||
def test_oversized_task_evidence_advances_once_as_an_omission(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
descriptor = root / ".docforge" / "project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8").replace(
|
||||
"max_context_tokens = 2000",
|
||||
"max_context_tokens = 50000\nmax_tool_output_chars = 8000",
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
workflow = root / "docs" / "content" / "workflow.md"
|
||||
workflow.write_text(
|
||||
workflow.read_text(encoding="utf-8") + "\n" + ("large evidence " * 2_000),
|
||||
encoding="utf-8",
|
||||
)
|
||||
project = Project.open(root)
|
||||
ProjectIndex(project).build()
|
||||
service = DocForgeService(project, capability_mode_name="read")
|
||||
first = service.task_context(
|
||||
"change",
|
||||
"Change the editing workflow",
|
||||
focus_node_id="guide.workflow",
|
||||
budget=50_000,
|
||||
limit=1,
|
||||
)
|
||||
first_capsule = first["capsule"]
|
||||
self.assertEqual([], first_capsule["evidence"])
|
||||
self.assertEqual("response_limit", first_capsule["omissions"][0]["code"])
|
||||
self.assertEqual("guide.workflow", first_capsule["omissions"][0]["subject"])
|
||||
self.assertEqual(1, first_capsule["pagination"]["returned_count"])
|
||||
self.assertTrue(first_capsule["pagination"]["has_more"])
|
||||
second = service.task_context(
|
||||
"change",
|
||||
"Change the editing workflow",
|
||||
focus_node_id="guide.workflow",
|
||||
budget=50_000,
|
||||
limit=1,
|
||||
cursor=first_capsule["pagination"]["next_cursor"],
|
||||
)
|
||||
self.assertNotEqual(first_capsule["page_hash"], second["capsule"]["page_hash"])
|
||||
self.assertNotIn(
|
||||
"guide.workflow",
|
||||
[
|
||||
item.get("node_id", item.get("subject"))
|
||||
for item in (
|
||||
*second["capsule"]["evidence"],
|
||||
*second["capsule"]["omissions"],
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
async def test_context_pagination_is_complete_and_stale_cursors_fail_closed(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
|
|
@ -304,6 +635,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
logic = await session.call_tool(
|
||||
"docforge_get_logic", {"owner_node_id": "guide.workflow"}
|
||||
)
|
||||
task_context = await session.call_tool(
|
||||
"docforge_get_task_context",
|
||||
{
|
||||
"task_kind": "change",
|
||||
"task": "Change the editing workflow without AST analysis",
|
||||
"focus_node_id": "guide.workflow",
|
||||
},
|
||||
)
|
||||
|
||||
policy = bootstrap.structuredContent["adapter_policy"]
|
||||
self.assertEqual("preserve-no-ast", policy["mode"])
|
||||
|
|
@ -329,6 +668,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
"adapter_policy_forbids_logic",
|
||||
logic.structuredContent["error"]["code"],
|
||||
)
|
||||
self.assertEqual("ok", task_context.structuredContent["status"])
|
||||
self.assertNotIn(
|
||||
"logic",
|
||||
{
|
||||
step["operation"]
|
||||
for step in task_context.structuredContent["capsule"]["plan"]["steps"]
|
||||
},
|
||||
)
|
||||
|
||||
async def test_every_read_tool_returns_scoped_structured_results(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
|
|
@ -358,6 +705,14 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
("docforge_visualization_status", {}),
|
||||
("docforge_bootstrap", {}),
|
||||
("docforge_sync", {}),
|
||||
(
|
||||
"docforge_get_task_context",
|
||||
{
|
||||
"task_kind": "change",
|
||||
"task": "Change the editing workflow",
|
||||
"focus_node_id": "guide.workflow",
|
||||
},
|
||||
),
|
||||
)
|
||||
with self.running_manager(Path(directory) / "viewer-manager.json"):
|
||||
service = DocForgeService(Project.open(root))
|
||||
|
|
@ -413,6 +768,7 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
|
|||
context = results[9].structuredContent
|
||||
self.assertLessEqual(context["estimated_tokens"], 180)
|
||||
self.assertTrue(context["omissions"])
|
||||
self.assertEqual("complete", results[17].structuredContent["capsule"]["state"])
|
||||
|
||||
async def test_invalid_traversal_limit_is_a_structured_domain_error(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
|
|
|
|||
|
|
@ -147,7 +147,7 @@ class EffectivePolicyTests(unittest.TestCase):
|
|||
self.assertNotIn("docforge_register_changes", result["recommended_workflow"])
|
||||
self.assertNotIn("docforge_apply_changeset", result["recommended_workflow"])
|
||||
self.assertEqual(
|
||||
"docforge_get_context",
|
||||
"docforge_get_task_context",
|
||||
result["session_contract"]["recommended_first_operation"]["tool"],
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -82,6 +82,13 @@ PUBLIC_IMPORTS = {
|
|||
"capability_mode",
|
||||
"compose_effective_policy",
|
||||
),
|
||||
"docforge.retrieval": (
|
||||
"ContextCapsuleV1",
|
||||
"RetrievalPlanV1",
|
||||
"build_retrieval_plan",
|
||||
"relation_category",
|
||||
"validate_retrieval_plan",
|
||||
),
|
||||
"docforge.render_contract": (
|
||||
"GenericHtmlRenderer",
|
||||
"PreparedRender",
|
||||
|
|
@ -132,6 +139,7 @@ EXPECTED_MCP_TOOLS = {
|
|||
"docforge_get_changeset",
|
||||
"docforge_get_changeset_diff",
|
||||
"docforge_get_context",
|
||||
"docforge_get_task_context",
|
||||
"docforge_get_contract",
|
||||
"docforge_get_logic",
|
||||
"docforge_get_node",
|
||||
|
|
|
|||
464
tests/test_retrieval.py
Normal file
464
tests/test_retrieval.py
Normal file
|
|
@ -0,0 +1,464 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import tempfile
|
||||
import unittest
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from jsonschema import Draft202012Validator
|
||||
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.index import ProjectIndex
|
||||
from docforge.models import ProjectState
|
||||
from docforge.policy import compose_effective_policy
|
||||
from docforge.project import Project
|
||||
from docforge.retrieval import (
|
||||
BASE_RELATION_CATEGORIES,
|
||||
MAX_TASK_CANDIDATE_EDGES,
|
||||
MAX_TASK_EVIDENCE,
|
||||
RELATION_CATEGORIES,
|
||||
TASK_KINDS,
|
||||
TASK_REQUIREMENTS,
|
||||
build_retrieval_plan,
|
||||
relation_category,
|
||||
)
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
FIXTURES = ROOT / "tests" / "fixtures"
|
||||
CAPSULE_SCHEMA = json.loads(
|
||||
(ROOT / "schemas" / "context-capsule.schema.json").read_text(encoding="utf-8")
|
||||
)
|
||||
|
||||
|
||||
class TaskRetrievalTests(unittest.TestCase):
|
||||
def copy_fixture(self, name: str, destination: Path) -> Path:
|
||||
root = destination / name
|
||||
shutil.copytree(FIXTURES / name, root)
|
||||
return root
|
||||
|
||||
@staticmethod
|
||||
def effective_policy() -> dict[str, object]:
|
||||
return compose_effective_policy(
|
||||
selected_mode="read",
|
||||
capability_source="explicit",
|
||||
no_ast=False,
|
||||
diagnostics=False,
|
||||
render_configured=True,
|
||||
application_enabled=False,
|
||||
).as_dict()
|
||||
|
||||
def plan(
|
||||
self,
|
||||
project: Project,
|
||||
*,
|
||||
task_kind: str = "change",
|
||||
task: str = "Change the editing workflow",
|
||||
focus_node_id: str | None = "guide.workflow",
|
||||
budget: int | None = None,
|
||||
limit: int | None = None,
|
||||
):
|
||||
return build_retrieval_plan(
|
||||
project.descriptor,
|
||||
task_kind=task_kind,
|
||||
task=task,
|
||||
focus_node_id=focus_node_id,
|
||||
budget=budget,
|
||||
limit=limit,
|
||||
effective_policy=self.effective_policy(),
|
||||
)
|
||||
|
||||
def test_every_task_plan_is_closed_deterministic_and_hash_bound(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
project = Project.open(self.copy_fixture("alpha", Path(directory)))
|
||||
hashes = {}
|
||||
relation_hashes = set()
|
||||
for task_kind in TASK_KINDS:
|
||||
first = self.plan(project, task_kind=task_kind)
|
||||
second = self.plan(project, task_kind=task_kind)
|
||||
self.assertEqual(first, second)
|
||||
self.assertEqual(TASK_REQUIREMENTS[task_kind], first.category_order[:1])
|
||||
self.assertEqual(
|
||||
{
|
||||
*BASE_RELATION_CATEGORIES,
|
||||
"unclassified",
|
||||
},
|
||||
set(first.category_order),
|
||||
)
|
||||
self.assertEqual(64, len(first.plan_hash))
|
||||
relation_hashes.update(
|
||||
step.relation_set_hash
|
||||
for step in first.steps
|
||||
if step.relation_scope == "project_allowed"
|
||||
)
|
||||
hashes[task_kind] = first.plan_hash
|
||||
self.assertEqual(len(TASK_KINDS), len(set(hashes.values())))
|
||||
self.assertEqual(1, len(relation_hashes))
|
||||
self.assertEqual(
|
||||
{
|
||||
"change": ("dependency",),
|
||||
"implementation": ("implementation",),
|
||||
"failure": ("execution",),
|
||||
"ownership": ("structure",),
|
||||
"test": ("evidence",),
|
||||
"operation": ("execution",),
|
||||
"release": ("evidence",),
|
||||
},
|
||||
TASK_REQUIREMENTS,
|
||||
)
|
||||
|
||||
mapped = [
|
||||
relation for relations in RELATION_CATEGORIES.values() for relation in relations
|
||||
]
|
||||
self.assertEqual(len(mapped), len(set(mapped)))
|
||||
self.assertEqual(
|
||||
{
|
||||
"structure": ("contains", "defined_in", "defines", "owns"),
|
||||
"implementation": (
|
||||
"implemented_by",
|
||||
"implements",
|
||||
"inherits",
|
||||
"inherits_from",
|
||||
),
|
||||
"dependency": ("depends_on", "imports"),
|
||||
"execution": (
|
||||
"activates",
|
||||
"calls",
|
||||
"dispatches_to",
|
||||
"launches",
|
||||
),
|
||||
"data": ("reads", "writes"),
|
||||
"evidence": (
|
||||
"documents",
|
||||
"governs",
|
||||
"proves",
|
||||
"tested_by",
|
||||
"verifies",
|
||||
),
|
||||
"context": ("relates_to",),
|
||||
"unclassified": (),
|
||||
},
|
||||
RELATION_CATEGORIES,
|
||||
)
|
||||
self.assertEqual("dependency", relation_category("depends_on"))
|
||||
self.assertEqual("evidence", relation_category("proves"))
|
||||
self.assertEqual("unclassified", relation_category("owns_database"))
|
||||
self.assertEqual("unclassified", relation_category("when_true"))
|
||||
|
||||
def test_exact_capsule_is_schema_valid_stable_and_explainable(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
project = Project.open(self.copy_fixture("alpha", Path(directory)))
|
||||
index = ProjectIndex(project)
|
||||
index.build()
|
||||
plan = self.plan(project)
|
||||
first = index.task_context(plan)
|
||||
second = index.task_context(plan)
|
||||
|
||||
self.assertEqual(first, second)
|
||||
capsule = first["capsule"]
|
||||
Draft202012Validator(CAPSULE_SCHEMA).validate(capsule)
|
||||
self.assertEqual("complete", capsule["state"])
|
||||
self.assertEqual("resolved", capsule["focus"]["state"])
|
||||
self.assertEqual("guide.workflow", capsule["focus"]["node_id"])
|
||||
self.assertEqual(
|
||||
["guide.workflow", "guide.foundation", "proof.validation"],
|
||||
[item["node_id"] for item in capsule["evidence"]],
|
||||
)
|
||||
dependency = capsule["evidence"][1]["relationship_path"][0]
|
||||
self.assertEqual("depends_on", dependency["relation"])
|
||||
self.assertEqual("dependency", dependency["category"])
|
||||
self.assertEqual("outgoing", dependency["direction"])
|
||||
workflow_reasons = capsule["evidence"][0]["relationship_reasons"]
|
||||
self.assertIn(
|
||||
("depends_on", "outgoing"),
|
||||
{
|
||||
(relationship["relation"], relationship["direction"])
|
||||
for relationship in workflow_reasons
|
||||
},
|
||||
)
|
||||
self.assertIn(
|
||||
("proves", "incoming"),
|
||||
{
|
||||
(relationship["relation"], relationship["direction"])
|
||||
for relationship in workflow_reasons
|
||||
},
|
||||
)
|
||||
foundation_reason = capsule["evidence"][1]["relationship_reasons"][0]
|
||||
self.assertEqual("depends_on", foundation_reason["relation"])
|
||||
self.assertEqual("incoming", foundation_reason["direction"])
|
||||
self.assertNotIn(
|
||||
"no_selected_evidence",
|
||||
[gap["code"] for gap in capsule["gaps"]],
|
||||
)
|
||||
self.assertEqual(64, len(capsule["collection_hash"]))
|
||||
self.assertEqual(64, len(capsule["capsule_hash"]))
|
||||
|
||||
def test_gaps_distinguish_undeclared_complete_and_incomplete_checks(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
project = Project.open(root)
|
||||
index = ProjectIndex(project)
|
||||
index.build()
|
||||
|
||||
undeclared = index.task_context(self.plan(project, task_kind="implementation"))[
|
||||
"capsule"
|
||||
]
|
||||
self.assertIn(
|
||||
"category_not_declared",
|
||||
[gap["code"] for gap in undeclared["gaps"]],
|
||||
)
|
||||
|
||||
descriptor = root / ".docforge" / "project.toml"
|
||||
descriptor.write_text(
|
||||
descriptor.read_text(encoding="utf-8").replace(
|
||||
'"returns_to"]',
|
||||
'"returns_to", "calls"]',
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
declared_project = Project.open(root)
|
||||
declared_index = ProjectIndex(declared_project)
|
||||
declared_index.build()
|
||||
missing = declared_index.task_context(
|
||||
self.plan(
|
||||
declared_project,
|
||||
task_kind="failure",
|
||||
focus_node_id="guide.foundation",
|
||||
)
|
||||
)["capsule"]
|
||||
self.assertIn(
|
||||
"no_selected_evidence",
|
||||
[gap["code"] for gap in missing["gaps"]],
|
||||
)
|
||||
|
||||
incomplete = declared_index.task_context(self.plan(declared_project, budget=1))[
|
||||
"capsule"
|
||||
]
|
||||
self.assertEqual("incomplete", incomplete["state"])
|
||||
self.assertTrue(incomplete["omissions"])
|
||||
self.assertIn(
|
||||
"evidence_incomplete",
|
||||
[gap["code"] for gap in incomplete["gaps"]],
|
||||
)
|
||||
self.assertNotIn(
|
||||
"no_selected_evidence",
|
||||
[gap["code"] for gap in incomplete["gaps"]],
|
||||
)
|
||||
|
||||
def test_unclassified_relation_is_preserved_without_guessed_semantics(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
project = Project.open(self.copy_fixture("awesome-ski-game", Path(directory)))
|
||||
index = ProjectIndex(project)
|
||||
index.build()
|
||||
plan = build_retrieval_plan(
|
||||
project.descriptor,
|
||||
task_kind="change",
|
||||
task="Change the first descent session",
|
||||
focus_node_id="session.first-descent",
|
||||
budget=None,
|
||||
limit=None,
|
||||
effective_policy=self.effective_policy(),
|
||||
)
|
||||
capsule = index.task_context(plan)["capsule"]
|
||||
self.assertIn("informs", capsule["summary"]["unclassified_relations"])
|
||||
relation = next(
|
||||
relationship
|
||||
for item in capsule["evidence"]
|
||||
for relationship in item["relationship_reasons"]
|
||||
if relationship["relation"] == "informs"
|
||||
)
|
||||
self.assertEqual("unclassified", relation["category"])
|
||||
self.assertIn(
|
||||
"unclassified_relation",
|
||||
[gap["code"] for gap in capsule["gaps"]],
|
||||
)
|
||||
|
||||
def test_plan_is_compact_at_large_valid_relation_scale_and_limits_are_fixed(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
project = Project.open(self.copy_fixture("alpha", Path(directory)))
|
||||
relation_names = tuple(f"relation-{position:05d}" for position in range(33_005))
|
||||
descriptor = replace(
|
||||
project.descriptor,
|
||||
allowed_relations=relation_names,
|
||||
limits=replace(
|
||||
project.descriptor.limits,
|
||||
max_results=2**63,
|
||||
),
|
||||
)
|
||||
plan = build_retrieval_plan(
|
||||
descriptor,
|
||||
task_kind="change",
|
||||
task="Exercise a very large valid relation policy",
|
||||
focus_node_id="guide.workflow",
|
||||
budget=None,
|
||||
limit=None,
|
||||
effective_policy=self.effective_policy(),
|
||||
)
|
||||
self.assertLessEqual(plan.max_evidence, MAX_TASK_EVIDENCE)
|
||||
self.assertLessEqual(plan.max_candidate_edges, MAX_TASK_CANDIDATE_EDGES)
|
||||
traversal = [step for step in plan.steps if step.operation in {"outgoing", "incoming"}]
|
||||
self.assertEqual(2, len(traversal))
|
||||
self.assertTrue(all(step.relation_scope == "project_allowed" for step in traversal))
|
||||
self.assertTrue(all(len(step.relation_set_hash or "") == 64 for step in traversal))
|
||||
self.assertLess(len(json.dumps(plan.as_dict())), 5_000)
|
||||
|
||||
def test_executor_rejects_tampered_public_plan_objects(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
project = Project.open(self.copy_fixture("alpha", Path(directory)))
|
||||
index = ProjectIndex(project)
|
||||
index.build()
|
||||
plan = self.plan(project)
|
||||
tampered = (
|
||||
replace(plan, steps=plan.steps[:-1]),
|
||||
replace(plan, requirements=()),
|
||||
replace(plan, task_query="Different task"),
|
||||
replace(plan, category_order=tuple(reversed(plan.category_order))),
|
||||
replace(plan, max_evidence=plan.max_evidence + 1),
|
||||
replace(plan, effective_policy_hash="0" * 64),
|
||||
replace(plan, plan_hash="0" * 64),
|
||||
)
|
||||
for candidate in tampered:
|
||||
with self.subTest(candidate=candidate):
|
||||
with self.assertRaises(DocForgeError) as invalid:
|
||||
index.task_context(candidate)
|
||||
self.assertEqual("invalid_retrieval_plan", invalid.exception.code)
|
||||
|
||||
def test_work_and_unclassified_limits_are_explicit_and_bounded(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
relation_names = [f"relation-{position:03d}" for position in range(450)]
|
||||
descriptor = root / ".docforge" / "project.toml"
|
||||
raw_descriptor = descriptor.read_text(encoding="utf-8")
|
||||
raw_relations = ", ".join(
|
||||
json.dumps(relation)
|
||||
for relation in (
|
||||
"depends_on",
|
||||
"proves",
|
||||
"supersedes",
|
||||
"relates_to",
|
||||
"returns_to",
|
||||
*relation_names,
|
||||
)
|
||||
)
|
||||
descriptor.write_text(
|
||||
raw_descriptor.replace(
|
||||
'"depends_on", "proves", "supersedes", "relates_to", "returns_to"',
|
||||
raw_relations,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
workflow = root / "docs" / "content" / "workflow.md"
|
||||
raw_workflow = workflow.read_text(encoding="utf-8")
|
||||
relationships = "".join(
|
||||
f'{relation} = ["guide.foundation"]\n' for relation in relation_names
|
||||
)
|
||||
workflow.write_text(
|
||||
raw_workflow.replace("+++\n\nEditors", f"{relationships}+++\n\nEditors", 1),
|
||||
encoding="utf-8",
|
||||
)
|
||||
project = Project.open(root)
|
||||
index = ProjectIndex(project)
|
||||
index.build()
|
||||
capsule = index.task_context(self.plan(project))["capsule"]
|
||||
Draft202012Validator(CAPSULE_SCHEMA).validate(capsule)
|
||||
omission_codes = {omission["code"] for omission in capsule["omissions"]}
|
||||
self.assertIn("edge_examination_limit", omission_codes)
|
||||
self.assertIn("unclassified_relation_limit", omission_codes)
|
||||
self.assertEqual(
|
||||
project.descriptor.limits.max_results,
|
||||
len(capsule["summary"]["unclassified_relations"]),
|
||||
)
|
||||
self.assertLessEqual(
|
||||
capsule["summary"]["examined_edge_count"],
|
||||
MAX_TASK_CANDIDATE_EDGES,
|
||||
)
|
||||
|
||||
def test_lexical_focus_blocks_missing_and_ambiguous_selection(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = self.copy_fixture("alpha", Path(directory))
|
||||
project = Project.open(root)
|
||||
index = ProjectIndex(project)
|
||||
index.build()
|
||||
missing = index.task_context(
|
||||
self.plan(
|
||||
project,
|
||||
task="Words absent from every indexed node",
|
||||
focus_node_id=None,
|
||||
)
|
||||
)["capsule"]
|
||||
self.assertEqual("blocked", missing["state"])
|
||||
self.assertEqual("not_found", missing["focus"]["state"])
|
||||
self.assertEqual("focus_not_found", missing["gaps"][0]["code"])
|
||||
|
||||
duplicate = root / "docs" / "content" / "workflow-copy.md"
|
||||
duplicate.write_text(
|
||||
(root / "docs" / "content" / "workflow.md")
|
||||
.read_text(encoding="utf-8")
|
||||
.replace('id = "guide.workflow"', 'id = "guide.workflow-copy"'),
|
||||
encoding="utf-8",
|
||||
)
|
||||
duplicate_project = Project.open(root)
|
||||
duplicate_index = ProjectIndex(duplicate_project)
|
||||
duplicate_index.build()
|
||||
ambiguous = duplicate_index.task_context(
|
||||
self.plan(
|
||||
duplicate_project,
|
||||
task="Editing workflow",
|
||||
focus_node_id=None,
|
||||
)
|
||||
)["capsule"]
|
||||
self.assertEqual("blocked", ambiguous["state"])
|
||||
self.assertEqual("ambiguous", ambiguous["focus"]["state"])
|
||||
self.assertEqual("focus_ambiguous", ambiguous["gaps"][0]["code"])
|
||||
|
||||
def test_plan_rejects_unbounded_or_untyped_inputs(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
project = Project.open(self.copy_fixture("alpha", Path(directory)))
|
||||
cases = (
|
||||
({"task_kind": "unknown"}, "invalid_task_kind"),
|
||||
({"task": " "}, "invalid_task_focus"),
|
||||
({"focus_node_id": ""}, "invalid_task_focus"),
|
||||
({"budget": True}, "invalid_budget"),
|
||||
({"budget": 0}, "invalid_budget"),
|
||||
({"limit": True}, "invalid_limit"),
|
||||
({"limit": project.descriptor.limits.max_results + 1}, "invalid_limit"),
|
||||
)
|
||||
for overrides, code in cases:
|
||||
arguments = {
|
||||
"task_kind": "change",
|
||||
"task": "Change the editing workflow",
|
||||
"focus_node_id": "guide.workflow",
|
||||
"budget": None,
|
||||
"limit": None,
|
||||
**overrides,
|
||||
}
|
||||
with self.subTest(arguments=overrides):
|
||||
with self.assertRaises(DocForgeError) as invalid:
|
||||
self.plan(project, **arguments)
|
||||
self.assertEqual(code, invalid.exception.code)
|
||||
|
||||
def test_final_generation_change_rejects_the_whole_capsule(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
project = Project.open(self.copy_fixture("alpha", Path(directory)))
|
||||
index = ProjectIndex(project)
|
||||
built = index.build()
|
||||
current = ProjectState(
|
||||
revision=built["revision"],
|
||||
source_hash=built["source_hash"],
|
||||
)
|
||||
changed = ProjectState(
|
||||
revision=current.revision,
|
||||
source_hash="0" * 64,
|
||||
)
|
||||
with (
|
||||
mock.patch.object(
|
||||
project,
|
||||
"incremental_state",
|
||||
side_effect=(current, changed),
|
||||
),
|
||||
self.assertRaises(DocForgeError) as stale,
|
||||
):
|
||||
index.task_context(self.plan(project))
|
||||
self.assertEqual("source_changed", stale.exception.code)
|
||||
Loading…
Add table
Add a link
Reference in a new issue