administrator/DocForge2
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

feat: add project adapter shadow contract

Browse source
This commit is contained in:
Andraxion 2026-07-22 04:17:05 -04:00
parent 411f417670
commit 561d98f1f8
10 changed files with 532 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

12
ACTIVE_SLICE.md
View file

@ -1,11 +1,11 @@
# Active slice
```text
Slice: DFG-5 Worldforge shadow adapter
Goal: Prove that a project-specific adapter can reproduce Worldforge documentation semantics and generated output without changing the live Worldforge workflow.
In scope: Worldforge source mapping, authority and graph validation, active and phase context profiles, render models, parallel index comparisons, and byte-for-byte shadow output checks.
Out of scope: Canonical changeset application, live workflow replacement, public output changes, deployment, arbitrary commands, Git mutation, accounts, HTTP transport, and a web UI.
Done when: The shadow adapter matches current Worldforge nodes, edges, bounded contexts, validation results, and rendered outputs while existing Worldforge manual tests remain green.
Slice: DFG-5B Worldforge full-family shadow completion
Goal: Complete the Worldforge shadow proof after the independently managed AssetForge family is explicitly placed in scope.
In scope: AssetForge graph projection, its context profile, the combined manual render, and a full 532-node and 830-edge comparison.
Out of scope: Reading or rebuilding AssetForge before explicit authorization; canonical changeset application; live workflow replacement; public output changes; deployment; arbitrary commands; Git mutation; accounts; HTTP transport; and a web UI.
Done when: The complete adapter matches all Worldforge nodes, edges, bounded contexts, validation results, and 32 rendered outputs while existing Worldforge manual tests remain green.
Owners: DocForge core retains generic mechanics. The Worldforge adapter owns Worldforge-specific semantics. Existing Worldforge sources and builders remain authoritative during shadow adoption.
Proof: Pending implementation and verification.
Proof: DFG-5A mapped and validated 522 non-AssetForge nodes and 805 edges through a generic adapter contract, matched exact lookup, weighted search, filtering, backlinks, dependencies, three deterministic context packs, the current active context cache, and 31 byte-identical outputs. It wrote no canonical or public files. The remaining 10 nodes, 25 edges, AssetForge context, and manual/manual.html are intentionally excluded.
```

14
README.md
View file

@ -10,9 +10,11 @@ isolated previews through the explicit render boundary.
## Current gate
DFG-1 through DFG-4 are complete. DFG-5 is the active gate: a shadow-only Worldforge adapter that
must reproduce current semantics and output without changing the live workflow. Canonical
application remains external and closed to the normal MCP server.
DFG-1 through DFG-4 are complete. DFG-5 is the active gate. Its first shadow proof maps the
non-AssetForge Worldforge graph through the standard index boundary and verifies project-specific
queries, active and phase contexts, and 31 unaffected generated outputs. The independently managed
AssetForge family and the combined manual page remain excluded until that content is explicitly in
scope. Canonical application remains external and closed to the normal MCP server.
## Development
@ -37,3 +39,9 @@ Projects may also declare render views with confined template, preview, and outp
built-in renderer converts CommonMark to escaped HTML through strict template tokens. MCP may render
validated changesets only into isolated preview paths. Declared project output is generated through
the explicit local CLI command and is never an MCP operation.
Project adapters implement `AdapterLoader` and return one ordered, immutable `AdapterProjection`.
`AdapterProject` validates the projection and exposes it through the same disposable index used by
generic projects. Project-specific context, query ordering, and render-model policy remain in the
adapter. Shadow adapters are local integration tools; the normal MCP server does not discover or
execute them.

38
SLICE_HISTORY.md
View file

@ -176,3 +176,41 @@ DFG-4: add deterministic previews and confined renderer orchestration without ca
DFG-5: reproduce Worldforge semantics and generated output through a shadow-only adapter without
changing the live workflow.
## DFG-5A Worldforge non-AssetForge shadow proof
### Changed
- Added a reusable adapter contract with ordered project projections, adapter metadata, root and
identity validation, a standard read-index bridge, and byte-exact artifact comparison.
- Generalized the derived index boundary to accept any immutable project service without changing
generic project loading, proposals, rendering, or MCP behavior.
- Added a Worldforge-local shadow adapter that translates the existing normalized manual index into
core nodes and edges while retaining acceptance and relationship provenance as adapter metadata.
- Kept Worldforge-specific weighted search, backlink ordering, context profiles, and render-model
composition in the Worldforge adapter.
- Excluded the independently managed AssetForge family and combined manual output from this subgate.
### Verification
- The shadow graph matched 522 nodes and 805 edges exactly and built through DocForge's standard
disposable index.
- Exact lookup, three weighted searches, active-development filtering, Phase 5 backlinks, and Phase
5 dependency traversal matched the current Worldforge index.
- Active, Phase 3, and Phase 5 context packs were byte-repeatable. Active also matched the current
derived context cache.
- All 31 generated outputs that do not require AssetForge matched committed bytes. The proof wrote
only temporary derived files and removed them afterward.
- DocForge adapter-contract tests cover standard index use, graph and metadata rejection, identity
changes, cache confinement, and complete byte-exact artifact comparison.
### Limits
- The remaining 10 AssetForge nodes, 25 incident edges, AssetForge context profile, and combined
`manual/manual.html` output are not read or rebuilt by this proof.
- The shadow adapter is an explicit local command. It is not discoverable or executable through the
normal MCP server.
### Next gate
DFG-5B: complete the full-family shadow proof when AssetForge is explicitly authorized.

12
docs/CONTRACT.md
View file

@ -78,3 +78,15 @@ replacement fail without replacing the prior output.
Normal MCP access does not expose canonical application, declared project-output rendering,
arbitrary renderer execution, arbitrary file writes, shell commands, Git mutation, build commands,
deployment, or publication.
## Project adapter boundary
An adapter supplies one deterministically ordered `AdapterProjection` containing core nodes and
edges plus ordered adapter metadata. The core validates root identity, graph integrity, stable
ordering, and metadata-key uniqueness before exposing the projection through the standard derived
index. The loader is called again during an operation so identity or source changes fail closed.
Adapters own stricter project semantics such as authority precedence, phase rules, context
selection, query ordering, and render-model composition. They may not weaken root confinement,
canonical authority, graph validation, hashing, stale-state checks, or derived-output boundaries.
Shadow adapters are explicit local integrations and are not loaded by the normal MCP process.

271
src/docforge/adapter_contract.py Normal file
View file

@ -0,0 +1,271 @@
"""Reusable contracts for explicit project adapters and shadow verification."""
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol
from .config_validation import AUTHORITIES, ID_PATTERN
from .errors import DocForgeError
from .models import (
Edge,
Limits,
Node,
ProjectDescriptor,
ProjectSnapshot,
)
from .project import validate_graph
@dataclass(frozen=True)
class AdapterNode:
"""One core node plus deterministic adapter-owned metadata."""
node: Node
metadata: tuple[tuple[str, str], ...] = ()
def as_dict(self) -> dict[str, object]:
return {"node": self.node.as_dict(), "metadata": dict(self.metadata)}
@dataclass(frozen=True)
class AdapterEdge:
"""One core edge plus deterministic adapter-owned metadata."""
edge: Edge
metadata: tuple[tuple[str, str], ...] = ()
def as_dict(self) -> dict[str, object]:
return {"edge": self.edge.as_dict(), "metadata": dict(self.metadata)}
@dataclass(frozen=True)
class AdapterProjection:
"""A complete immutable graph projection supplied by one project adapter."""
project_id: str
title: str
adapter_id: str
adapter_version: str
root: Path
revision: str
source_hash: str
nodes: tuple[AdapterNode, ...]
edges: tuple[AdapterEdge, ...]
def core_nodes(self) -> tuple[Node, ...]:
return tuple(item.node for item in self.nodes)
def core_edges(self) -> tuple[Edge, ...]:
return tuple(item.edge for item in self.edges)
def identity(self) -> str:
payload = {
"project_id": self.project_id,
"adapter_id": self.adapter_id,
"adapter_version": self.adapter_version,
"revision": self.revision,
"source_hash": self.source_hash,
"nodes": [item.as_dict() for item in self.nodes],
"edges": [item.as_dict() for item in self.edges],
}
encoded = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
return hashlib.sha256(encoded).hexdigest()
class AdapterLoader(Protocol):
"""Load one current, deterministic, project-confined adapter projection."""
def load_projection(self) -> AdapterProjection: ...
def validate_projection(projection: AdapterProjection) -> None:
"""Validate generic invariants without interpreting adapter metadata."""
root = projection.root.resolve(strict=True)
if not root.is_dir() or projection.root != root:
raise DocForgeError("invalid_adapter", "Adapter root must be a resolved directory")
for label, value in (
("project_id", projection.project_id),
("title", projection.title),
("adapter_id", projection.adapter_id),
("adapter_version", projection.adapter_version),
("revision", projection.revision),
("source_hash", projection.source_hash),
):
if not value.strip():
raise DocForgeError("invalid_adapter", f"Adapter {label} must not be empty")
if ID_PATTERN.fullmatch(projection.project_id) is None:
raise DocForgeError("invalid_adapter", "Adapter project ID is invalid")
if len(projection.source_hash) != 64 or any(
character not in "0123456789abcdef" for character in projection.source_hash
):
raise DocForgeError("invalid_adapter", "Adapter source hash must be lowercase SHA-256")
ordered_nodes = tuple(sorted(projection.nodes, key=lambda item: item.node.node_id))
ordered_edges = tuple(
sorted(
projection.edges,
key=lambda item: (
item.edge.source_id,
item.edge.relation,
item.edge.target_id,
),
)
)
if projection.nodes != ordered_nodes or projection.edges != ordered_edges:
raise DocForgeError(
"invalid_adapter", "Adapter projection must be deterministically ordered"
)
for item in (*projection.nodes, *projection.edges):
keys = [key for key, _ in item.metadata]
if keys != sorted(keys) or len(keys) != len(set(keys)):
raise DocForgeError(
"invalid_adapter", "Adapter metadata keys must be unique and ordered"
)
for item in projection.nodes:
node = item.node
source = Path(node.source_path)
if ID_PATTERN.fullmatch(node.node_id) is None:
raise DocForgeError("invalid_adapter", "Adapter node ID is invalid", id=node.node_id)
if node.authority not in AUTHORITIES:
raise DocForgeError(
"invalid_adapter", "Adapter node authority is invalid", id=node.node_id
)
if (
not node.title.strip()
or not node.family.strip()
or not node.status.strip()
or not node.summary.strip()
or not node.content.strip()
):
raise DocForgeError(
"invalid_adapter", "Adapter node has empty required content", id=node.node_id
)
if source.is_absolute() or ".." in source.parts or not node.source_path:
raise DocForgeError(
"invalid_adapter", "Adapter node source path is unsafe", id=node.node_id
)
if len(node.tags) != len(set(node.tags)) or any(not tag for tag in node.tags):
raise DocForgeError("invalid_adapter", "Adapter node tags are invalid", id=node.node_id)
if len(node.content_hash) != 64 or any(
character not in "0123456789abcdef" for character in node.content_hash
):
raise DocForgeError(
"invalid_adapter", "Adapter node content hash is invalid", id=node.node_id
)
for item in projection.edges:
if ID_PATTERN.fullmatch(item.edge.relation) is None:
raise DocForgeError("invalid_adapter", "Adapter relationship type is invalid")
validate_graph(projection.core_nodes(), projection.core_edges())
class AdapterProject:
"""Expose a validated adapter projection through the standard index boundary."""
def __init__(self, loader: AdapterLoader, *, cache_root: Path) -> None:
self.loader = loader
initial = loader.load_projection()
validate_projection(initial)
root = initial.root
resolved_cache = cache_root.resolve(strict=False)
if (
resolved_cache == root
or not resolved_cache.is_relative_to(root)
or resolved_cache.is_symlink()
):
raise DocForgeError(
"path_escape", "Adapter cache must be a confined project subdirectory"
)
self._identity = (
initial.project_id,
initial.adapter_id,
initial.adapter_version,
initial.root,
)
self.descriptor = ProjectDescriptor(
schema_version=1,
project_id=initial.project_id,
title=initial.title,
adapter=f"{initial.adapter_id}@{initial.adapter_version}",
root=root,
descriptor_path=root / ".docforge" / "shadow-adapter.toml",
descriptor_hash=initial.identity(),
content_roots=(),
authority_files=(),
cache_root=resolved_cache,
index_path=resolved_cache / "index.sqlite3",
changeset_root=resolved_cache / "changesets-disabled",
proposal_writers=(),
render=None,
allowed_relations=tuple(sorted({item.edge.relation for item in initial.edges})),
profiles=(),
limits=Limits(
max_nodes=max(10_000, len(initial.nodes)),
max_context_tokens=64_000,
),
)
def load(self) -> ProjectSnapshot:
projection = self.loader.load_projection()
validate_projection(projection)
identity = (
projection.project_id,
projection.adapter_id,
projection.adapter_version,
projection.root,
)
if identity != self._identity:
raise DocForgeError("adapter_changed", "Adapter identity changed during the operation")
return ProjectSnapshot(
descriptor=self.descriptor,
nodes=projection.core_nodes(),
edges=projection.core_edges(),
source_hash=projection.source_hash,
revision=projection.revision,
)
@dataclass(frozen=True)
class ShadowArtifact:
"""One named deterministic byte artifact used by a shadow comparison."""
artifact_id: str
content: bytes
@property
def sha256(self) -> str:
return hashlib.sha256(self.content).hexdigest()
def compare_artifacts(
reference: tuple[ShadowArtifact, ...], candidate: tuple[ShadowArtifact, ...]
) -> dict[str, object]:
"""Compare complete artifact sets without writing either side."""
reference_by_id = {item.artifact_id: item for item in reference}
candidate_by_id = {item.artifact_id: item for item in candidate}
if len(reference_by_id) != len(reference) or len(candidate_by_id) != len(candidate):
raise DocForgeError("duplicate_artifact", "Shadow artifact IDs must be unique")
missing = sorted(set(reference_by_id) - set(candidate_by_id))
unexpected = sorted(set(candidate_by_id) - set(reference_by_id))
changed = sorted(
artifact_id
for artifact_id in set(reference_by_id) & set(candidate_by_id)
if reference_by_id[artifact_id].content != candidate_by_id[artifact_id].content
)
return {
"status": "ok" if not missing and not unexpected and not changed else "mismatch",
"count": len(reference),
"missing": missing,
"unexpected": unexpected,
"changed": changed,
"hashes": {
artifact_id: reference_by_id[artifact_id].sha256
for artifact_id in sorted(reference_by_id)
if artifact_id in candidate_by_id
and reference_by_id[artifact_id].content == candidate_by_id[artifact_id].content
},
}

1
src/docforge/config_validation.py
View file

@ -9,6 +9,7 @@ from typing import Any
from .errors import DocForgeError
ID_PATTERN = re.compile(r"[a-z0-9][a-z0-9._-]{1,127}")
AUTHORITIES = frozenset({"authoritative", "approved_plan", "derived", "proposal", "historical"})
_SECRET_PARTS = frozenset({".git", ".ssh", ".gnupg", "secrets", "credentials"})

18
src/docforge/index.py
View file

@ -8,13 +8,13 @@ import os
import sqlite3
import tempfile
from collections import deque
from collections.abc import Iterator
from collections.abc import Generator
from contextlib import contextmanager
from pathlib import Path
from .errors import DocForgeError
from .models import Edge, Node, ProjectSnapshot
from .project import Project, project_root_fingerprint
from .models import Edge, Node, ProjectService, ProjectSnapshot
from .project import project_root_fingerprint
INDEX_SCHEMA_VERSION = 1
APPLICATION_ID = 1_146_683_778
@ -43,7 +43,7 @@ def _connect_read_only(path: Path) -> sqlite3.Connection:
@contextmanager
def _read_connection(path: Path) -> Iterator[sqlite3.Connection]:
def _read_connection(path: Path) -> Generator[sqlite3.Connection, None, None]:
connection: sqlite3.Connection | None = None
try:
connection = _connect_read_only(path)
@ -76,7 +76,7 @@ def _status(snapshot: ProjectSnapshot) -> dict[str, object]:
class ProjectIndex:
"""A disposable index that always checks current canonical source before queries."""
def __init__(self, project: Project) -> None:
def __init__(self, project: ProjectService) -> None:
self.project = project
@property
@ -265,7 +265,7 @@ class ProjectIndex:
""",
(expression, bounded),
).fetchall()
results = []
results: list[dict[str, object]] = []
for row in rows:
payload = _row_to_node(row).as_dict(include_content=False)
payload.update({"rank": row["rank"], "snippet": row["snippet"]})
@ -330,7 +330,7 @@ class ProjectIndex:
checked = self.check()
self._require_node(node_id)
maximum = self.project.descriptor.limits.max_traversal_depth
if not isinstance(depth, int) or isinstance(depth, bool) or depth < 0 or depth > maximum:
if type(depth) is not int or depth < 0 or depth > maximum:
raise DocForgeError("invalid_depth", "Traversal depth is outside the configured limit")
with _read_connection(self.path) as connection:
edges = tuple(
@ -340,7 +340,7 @@ class ProjectIndex:
"ORDER BY source_id, relation, target_id"
)
)
queue = deque([(node_id, 0, (node_id,))])
queue: deque[tuple[str, int, tuple[str, ...]]] = deque([(node_id, 0, (node_id,))])
seen = {node_id}
results: list[dict[str, object]] = []
while queue:
@ -417,7 +417,7 @@ def _row_to_node(row: sqlite3.Row) -> Node:
def _bounded_limit(value: int | None, maximum: int, *, default: int) -> int:
if value is None:
return min(default, maximum)
if not isinstance(value, int) or isinstance(value, bool) or value < 1 or value > maximum:
if type(value) is not int or value < 1 or value > maximum:
raise DocForgeError("invalid_limit", "Result limit is outside the configured range")
return value

9
src/docforge/models.py
View file

@ -4,6 +4,7 @@ from __future__ import annotations
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Protocol
@dataclass(frozen=True)
@ -118,6 +119,14 @@ class ProjectSnapshot:
revision: str
class ProjectService(Protocol):
"""Minimum immutable project boundary required by derived read services."""
descriptor: ProjectDescriptor
def load(self) -> ProjectSnapshot: ...
@dataclass(frozen=True)
class ContextEntry:
node_id: str

12
src/docforge/project.py
View file

@ -11,7 +11,14 @@ from dataclasses import replace
from pathlib import Path
from typing import Any
from .config_validation import ID_PATTERN, confined_path, positive_int, require_string, string_list
from .config_validation import (
AUTHORITIES,
ID_PATTERN,
confined_path,
positive_int,
require_string,
string_list,
)
from .errors import DocForgeError
from .models import (
ContextProfile,
@ -24,7 +31,6 @@ from .models import (
)
from .render_config import load_render_config
_AUTHORITIES = frozenset({"authoritative", "approved_plan", "derived", "proposal", "historical"})
_CORE_METADATA = frozenset(
{
"schema_version",
@ -363,7 +369,7 @@ def validated_node_from_record(
if ID_PATTERN.fullmatch(node_id) is None:
raise DocForgeError("invalid_source", f"{source.name}: node ID is invalid", id=node_id)
authority = require_string(record, "authority", source)
if authority not in _AUTHORITIES:
if authority not in AUTHORITIES:
raise DocForgeError(
"invalid_source", f"{source.name}: authority is invalid", authority=authority
)

166
tests/test_adapter_contract.py Normal file
View file

@ -0,0 +1,166 @@
from __future__ import annotations
import tempfile
import unittest
from dataclasses import replace
from pathlib import Path
from docforge.adapter_contract import (
AdapterEdge,
AdapterNode,
AdapterProject,
AdapterProjection,
ShadowArtifact,
compare_artifacts,
validate_projection,
)
from docforge.errors import DocForgeError
from docforge.index import ProjectIndex
from docforge.models import Edge, Node
class Loader:
def __init__(self, projection: AdapterProjection) -> None:
self.projection = projection
def load_projection(self) -> AdapterProjection:
return self.projection
class AdapterContractTests(unittest.TestCase):
def projection(self, root: Path) -> AdapterProjection:
foundation = Node(
node_id="guide.foundation",
title="Foundation",
family="guide",
authority="authoritative",
status="active",
tags=("guide",),
summary="The base contract.",
content="Foundation content.",
source_path="docs/foundation.md",
source_anchor=None,
content_hash="1" * 64,
)
workflow = Node(
node_id="guide.workflow",
title="Workflow",
family="guide",
authority="approved_plan",
status="planned",
tags=("guide", "workflow"),
summary="The editing workflow.",
content="Workflow content.",
source_path="docs/workflow.md",
source_anchor=None,
content_hash="2" * 64,
)
return AdapterProjection(
project_id="adapter-fixture",
title="Adapter fixture",
adapter_id="fixture-shadow",
adapter_version="1",
root=root,
revision="fixture-revision",
source_hash="3" * 64,
nodes=(
AdapterNode(foundation, (("acceptance", "proven"),)),
AdapterNode(workflow, (("acceptance", "pending"),)),
),
edges=(
AdapterEdge(
Edge("guide.workflow", "depends_on", "guide.foundation"),
(("source", "docs/workflow.md"),),
),
),
)
def test_adapter_projection_builds_and_checks_through_standard_index(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory).resolve()
projection = self.projection(root)
project = AdapterProject(Loader(projection), cache_root=root / ".cache" / "shadow")
index = ProjectIndex(project)
built = index.build()
checked = index.check()
self.assertEqual("fixture-shadow@1", built["adapter"])
self.assertEqual(2, checked["node_count"])
self.assertEqual("Workflow", index.get_node("guide.workflow")["node"]["title"])
self.assertEqual(projection.identity(), projection.identity())
def test_projection_rejects_unsorted_metadata_graph_and_identity_changes(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory).resolve()
projection = self.projection(root)
invalid_metadata = replace(
projection,
nodes=(
replace(
projection.nodes[0],
metadata=(("z", "last"), ("a", "first")),
),
projection.nodes[1],
),
)
with self.assertRaisesRegex(DocForgeError, "metadata keys"):
validate_projection(invalid_metadata)
invalid_source = replace(
projection,
nodes=(
replace(
projection.nodes[0],
node=replace(projection.nodes[0].node, source_path="../outside.md"),
),
projection.nodes[1],
),
)
with self.assertRaisesRegex(DocForgeError, "source path"):
validate_projection(invalid_source)
broken = replace(
projection,
edges=(AdapterEdge(Edge("guide.workflow", "depends_on", "missing.node")),),
)
with self.assertRaisesRegex(DocForgeError, "missing nodes"):
validate_projection(broken)
loader = Loader(projection)
project = AdapterProject(loader, cache_root=root / ".cache" / "shadow")
loader.projection = replace(projection, adapter_version="2")
with self.assertRaisesRegex(DocForgeError, "identity changed"):
project.load()
def test_adapter_cache_must_remain_inside_project(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory).resolve()
outside = root.parent / "outside-adapter-cache"
with self.assertRaisesRegex(DocForgeError, "confined"):
AdapterProject(Loader(self.projection(root)), cache_root=outside)
def test_artifact_comparison_is_complete_and_byte_exact(self) -> None:
reference = (
ShadowArtifact("manual", b"same"),
ShadowArtifact("timeline", b"old"),
)
exact = compare_artifacts(reference, reference)
self.assertEqual("ok", exact["status"])
self.assertEqual(2, exact["count"])
mismatch = compare_artifacts(
reference,
(
ShadowArtifact("timeline", b"new"),
ShadowArtifact("extra", b"extra"),
),
)
self.assertEqual("mismatch", mismatch["status"])
self.assertEqual(["manual"], mismatch["missing"])
self.assertEqual(["extra"], mismatch["unexpected"])
self.assertEqual(["timeline"], mismatch["changed"])
if __name__ == "__main__":
unittest.main()