1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

test: prove unrelated project isolation

This commit is contained in:
Andraxion 2026-07-22 05:01:07 -04:00
parent 1c4aa79d1c
commit 1943c36ef3
12 changed files with 492 additions and 12 deletions

View file

@ -1,11 +1,11 @@
# Active slice
```text
Slice: DFG-6 unrelated-project proof
Goal: Prove that DocForge's core works for a project with different families and no Worldforge phase assumptions.
In scope: An Awesome Ski Game fixture, generic loading and indexing, search, bounded context, changesets, preview rendering, two simultaneous project-bound servers, and cross-project isolation checks.
Out of scope: Worldforge read-only adoption; AssetForge proposal adoption; canonical changeset application; live workflow replacement; deployment; arbitrary commands; Git mutation; accounts; HTTP transport; and a web UI.
Done when: The second project works without the Worldforge adapter or vocabulary, all generic operations pass, and neither server can access the other project's sources, indexes, changesets, previews, or outputs.
Owners: DocForge core owns generic mechanics and isolation. Fixtures own their project-specific vocabularies. Worldforge-specific semantics remain in the Worldforge adapter.
Proof: DFG-5 mapped and validated all 532 Worldforge nodes and 830 edges, matched exact lookup, weighted search, filters, backlinks, dependencies, four deterministic bounded contexts, and all 32 generated outputs. The full AssetForge profile includes all ten nodes, reduced budgets record omissions, invalid budgets fail, and the proof writes no canonical or public files.
Slice: DFG-7 Worldforge read-only adoption
Goal: Use DocForge for routine Worldforge retrieval while the established builder and integration workflow remain authoritative.
In scope: A Worldforge project descriptor, separate Worldforge and AssetForge read-only sessions, real-task retrieval comparisons, context size and lookup measurements, omission and stale-state checks, and a documented rollback path.
Out of scope: AssetForge proposal adoption; canonical changeset application; replacing the Worldforge builder; deployment changes; arbitrary commands; Git mutation through MCP; accounts; HTTP transport; and a web UI.
Done when: Multiple real Worldforge tasks use project-bound DocForge retrieval without authority errors, phase drift, cross-family leakage, worse context quality, or loss of the current local-command rollback path.
Owners: Worldforge canonical sources and builders retain facts and output ownership. DocForge owns bounded read-only retrieval. Worldforge-specific profile and ordering policy remains in the Worldforge adapter.
Proof: DFG-6 loaded Awesome Ski Game through the generic path with five nodes, six edges, five unrelated families, and no project-specific core vocabulary. Indexing, search, filters, traversal, bounded context, isolated proposals, validation, and preview passed. Two simultaneous real MCP server processes remained bound to separate projects and could not cross-read nodes or mix indexes, changesets, previews, or output paths.
```

View file

@ -10,11 +10,11 @@ isolated previews through the explicit render boundary.
## Current gate
DFG-1 through DFG-5 are complete. The full Worldforge shadow proof maps all 532 nodes and 830 edges
through the standard index boundary and verifies project-specific queries, four bounded context
profiles, and all 32 generated outputs without changing canonical or public files. DFG-6 is the
active gate: prove the core against an unrelated project and simultaneous isolated servers.
Canonical application remains external and closed to the normal MCP server.
DFG-1 through DFG-6 are complete. Awesome Ski Game proves that the generic core can index, retrieve,
compile context, accept isolated proposals, and render previews for unrelated project families. Two
simultaneous MCP server processes remain project-bound and cannot cross-read or mix derived state.
DFG-7 is the active gate: adopt DocForge for read-only Worldforge retrieval while retaining the
existing builder and rollback path. Canonical application remains external and closed to MCP.
## Development

View file

@ -250,3 +250,41 @@ DFG-5B: complete the full-family shadow proof when AssetForge is explicitly auth
### Next gate
DFG-6: prove the generic core with an unrelated project and simultaneous project-isolated servers.
## DFG-6 unrelated-project proof
### Changed
- Added an Awesome Ski Game fixture using the generic project descriptor, five unrelated node
families, six relationships, a bounded `ride-day` context, one proposal writer, and one declared
field-guide view.
- Added an end-to-end proof covering generic loading, indexing, exact graph counts, search, filters,
dependency traversal, deterministic bounded context, isolated updates, validation, diffs, and
escaped preview rendering.
- Added a source guard that rejects Worldforge, AssetForge, phase, or villager vocabulary in the
generic core.
- Added a live isolation proof with two simultaneous MCP server subprocesses bound to Awesome Ski
Game and Alpha Documentation.
### Verification
- Awesome Ski Game loaded through `adapter = "generic"` with five nodes, six edges, and trail,
riding, safety, session, and proof families.
- The 300-token context retained its required session node, stayed within budget, recorded
omissions, and reproduced exactly.
- The proposal changed only its isolated changeset and preview. Canonical sources and declared
output remained unchanged, and raw HTML was escaped.
- Both live servers returned their own project identity and nodes, rejected the other project's
stable IDs, and wrote same-named changesets and previews only under their bound roots.
- The complete DocForge suite passes with warnings treated as errors.
### Limits
- This proof does not adopt DocForge inside Worldforge or enable any canonical write path.
- The Worldforge adapter and generic Awesome Ski Game fixture remain separate ownership paths.
- HTTP transport, accounts, and web administration remain unopened.
### Next gate
DFG-7: adopt project-bound DocForge retrieval for real Worldforge read-only tasks with measured
quality and a documented rollback path.

View file

@ -0,0 +1,54 @@
schema_version = 1
project_id = "awesome-ski-game"
title = "Awesome Ski Game"
adapter = "generic"
[sources]
content_roots = ["handbook"]
authority_files = ["DOCUMENTATION_POLICY.md"]
[derived]
cache_root = ".docforge/cache"
index = ".docforge/cache/index.sqlite3"
[changesets]
root = ".docforge/changesets"
[[changesets.writers]]
id = "ski-editor"
families = ["proof", "riding", "safety", "session", "trail"]
operations = ["create", "update", "move", "delete"]
[render]
template_root = "site/templates"
preview_root = ".docforge/previews"
[[render.views]]
id = "field-guide"
renderer = "generic_html"
template = "field-guide.html"
output = ".docforge/rendered/field-guide.html"
title = "Awesome Ski Game Field Guide"
families = ["proof", "riding", "safety", "session", "trail"]
[graph]
allowed_relations = ["depends_on", "informs", "proves", "relates_to"]
[limits]
max_source_bytes = 100000
max_nodes = 100
max_query_chars = 200
max_results = 20
max_traversal_depth = 4
max_context_tokens = 2000
max_changesets = 20
max_changeset_operations = 20
max_changeset_bytes = 100000
[[profiles]]
id = "ride-day"
families = ["proof", "riding", "safety", "session", "trail"]
statuses = ["active", "current", "open", "ready", "verified"]
required_nodes = ["session.first-descent"]
token_budget = 900
dependency_depth = 3

View file

@ -0,0 +1,4 @@
# Documentation policy
Files under `handbook/` own Awesome Ski Game documentation facts. DocForge indexes, contexts,
changesets, previews, and rendered pages are derived and may be rebuilt.

View file

@ -0,0 +1,11 @@
[[nodes]]
schema_version = 1
id = "proof.carve-playtest"
title = "Carve playtest proof"
family = "proof"
authority = "derived"
status = "verified"
tags = ["carving", "playtest", "proof"]
summary = "Records that the edge-control fixture has a bounded test target."
content = "The test target records grip, carve angle, skid state, and the route segment for each sample."
proves = ["riding.edge-control"]

View file

@ -0,0 +1,14 @@
+++
schema_version = 1
id = "riding.edge-control"
title = "Edge control"
family = "riding"
authority = "approved_plan"
status = "ready"
tags = ["carving", "controls", "snow"]
summary = "Defines how player input produces grip, carve angle, and controlled skidding."
depends_on = ["trail.glacier-bowl"]
+++
Edge input changes ski grip continuously. Higher edge angles create tighter carved turns when the
surface supports them; excessive angle or poor snow contact produces a controlled skid.

View file

@ -0,0 +1,14 @@
+++
schema_version = 1
id = "safety.avalanche-window"
title = "Avalanche window"
family = "safety"
authority = "authoritative"
status = "current"
tags = ["conditions", "risk", "weather"]
summary = "Defines the conditions that close exposed terrain during the test session."
informs = ["session.first-descent"]
+++
The session closes exposed terrain when wind loading crosses the configured safety threshold. A
closure changes route availability but does not rewrite the terrain definition.

View file

@ -0,0 +1,14 @@
+++
schema_version = 1
id = "session.first-descent"
title = "First descent session"
family = "session"
authority = "approved_plan"
status = "active"
tags = ["playtest", "route", "session"]
summary = "Combines one route, one riding mechanic, and current safety conditions for a playtest."
depends_on = ["riding.edge-control", "safety.avalanche-window", "trail.glacier-bowl"]
+++
The first descent starts on the sheltered practice face, crosses the bowl, and exits through the
marked lower route. The session records completion time, falls, controlled skids, and route closure.

View file

@ -0,0 +1,13 @@
+++
schema_version = 1
id = "trail.glacier-bowl"
title = "Glacier Bowl"
family = "trail"
authority = "authoritative"
status = "open"
tags = ["alpine", "route", "snow"]
summary = "Defines the terrain and route constraints for the first playable descent."
+++
Glacier Bowl is a wide alpine route with one narrow entry, a sheltered practice face, and a marked
exit. The playable route must keep the entry and exit readable in flat light.

View file

@ -0,0 +1,12 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="docforge-render" content="{{ docforge_render_identity }}">
<title>{{ docforge_title }}</title>
</head>
<body data-project="{{ docforge_project_id }}" data-view="{{ docforge_view_id }}">
<header><h1>{{ docforge_title }}</h1></header>
<main>{{ docforge_content }}</main>
</body>
</html>

View file

@ -0,0 +1,306 @@
from __future__ import annotations
import asyncio
import hashlib
import re
import shutil
import sys
import tempfile
import unittest
from contextlib import AsyncExitStack
from pathlib import Path
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from docforge.changesets import ChangesetStore
from docforge.context import compile_context
from docforge.index import ProjectIndex
from docforge.project import Project
from docforge.rendering import RenderService
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
def copy_fixture(name: str, destination: Path) -> Path:
root = destination / name
shutil.copytree(FIXTURES / name, root)
return root
def canonical_digest(root: Path) -> str:
paths = [
root / ".docforge/project.toml",
*(path for path in root.rglob("*") if path.is_file() and ".docforge" not in path.parts),
]
digest = hashlib.sha256()
for path in sorted(set(paths), key=lambda item: item.relative_to(root).as_posix()):
digest.update(path.relative_to(root).as_posix().encode())
digest.update(path.read_bytes())
return digest.hexdigest()
def node_hash(project: Project, node_id: str) -> str:
return next(node.content_hash for node in project.load().nodes if node.node_id == node_id)
class UnrelatedProjectProofTests(unittest.TestCase):
def test_generic_ski_project_completes_the_full_workflow(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = copy_fixture("awesome-ski-game", Path(directory))
before = canonical_digest(root)
project = Project.open(root)
snapshot = project.load()
self.assertEqual("generic", project.descriptor.adapter)
self.assertEqual("awesome-ski-game", snapshot.descriptor.project_id)
self.assertEqual(
{"proof", "riding", "safety", "session", "trail"},
{node.family for node in snapshot.nodes},
)
self.assertEqual(5, len(snapshot.nodes))
self.assertEqual(6, len(snapshot.edges))
index = ProjectIndex(project)
built = index.build()
self.assertEqual(built, index.check())
self.assertEqual(
["proof.carve-playtest", "riding.edge-control"],
[item["node_id"] for item in index.search("grip carve", limit=5)["results"]],
)
self.assertEqual(
["safety.avalanche-window"],
[
item["node_id"]
for item in index.filter_nodes(family="safety", status="current")["results"]
],
)
self.assertEqual(
{
"riding.edge-control",
"safety.avalanche-window",
"trail.glacier-bowl",
},
{
item["node_id"]
for item in index.dependencies("session.first-descent", depth=3)["results"]
},
)
first_context = compile_context(index, "ride-day", budget=300)
second_context = compile_context(index, "ride-day", budget=300)
self.assertEqual(first_context, second_context)
self.assertLessEqual(first_context["estimated_tokens"], 300)
self.assertEqual("session.first-descent", first_context["entries"][0]["node_id"])
self.assertTrue(first_context["omissions"])
changesets = ChangesetStore(project, "ski-editor")
created = changesets.create("edge-tuning")
updated = changesets.propose_update(
changeset_id="edge-tuning",
expected_changeset_hash=created["changeset_hash"],
node_id="riding.edge-control",
expected_content_hash=node_hash(project, "riding.edge-control"),
metadata={"summary": "Proposes a narrower grip transition for review."},
content="<script>unsafe()</script>\n\n**Previewed edge-tuning proposal.**",
relationship_changes=[],
rationale="Exercise an unrelated project's isolated proposal path.",
)
validated = changesets.validate("edge-tuning")
diff = changesets.diff("edge-tuning")
self.assertTrue(validated["valid"])
self.assertEqual(updated["changeset_hash"], validated["changeset_hash"])
self.assertEqual("update", diff["changes"][0]["operation"])
rendering = RenderService(project, changesets)
first_preview = rendering.preview("edge-tuning", "field-guide")
preview_path = root / ".docforge/previews/edge-tuning/field-guide.html"
preview_bytes = preview_path.read_bytes()
second_preview = rendering.preview("edge-tuning", "field-guide")
self.assertEqual(first_preview["preview_identity"], second_preview["preview_identity"])
self.assertEqual(preview_bytes, preview_path.read_bytes())
preview = preview_bytes.decode()
self.assertIn("&lt;script&gt;unsafe()&lt;/script&gt;", preview)
self.assertIn("<strong>Previewed edge-tuning proposal.</strong>", preview)
self.assertFalse((root / ".docforge/rendered/field-guide.html").exists())
self.assertEqual(before, canonical_digest(root))
def test_generic_core_contains_no_project_specific_vocabulary(self) -> None:
banned = re.compile(r"\b(worldforge|assetforge|phase|villager)\b", re.IGNORECASE)
leaks = []
for path in sorted((ROOT / "src/docforge").rglob("*.py")):
for line_number, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
if banned.search(line):
leaks.append(f"{path.name}:{line_number}:{line.strip()}")
self.assertEqual([], leaks)
class UnrelatedProjectIsolationTests(unittest.IsolatedAsyncioTestCase):
async def test_two_live_servers_remain_project_isolated(self) -> None:
with tempfile.TemporaryDirectory() as directory:
parent = Path(directory)
ski_root = copy_fixture("awesome-ski-game", parent / "ski")
alpha_root = copy_fixture("alpha", parent / "alpha")
ski_before = canonical_digest(ski_root)
alpha_before = canonical_digest(alpha_root)
ski_project = Project.open(ski_root)
alpha_project = Project.open(alpha_root)
ProjectIndex(ski_project).build()
ProjectIndex(alpha_project).build()
ski_parameters = StdioServerParameters(
command=sys.executable,
args=[
"-m",
"docforge.mcp_server",
"--project-root",
str(ski_root),
"--proposal-writer",
"ski-editor",
],
)
alpha_parameters = StdioServerParameters(
command=sys.executable,
args=[
"-m",
"docforge.mcp_server",
"--project-root",
str(alpha_root),
"--proposal-writer",
"alpha-editor",
],
)
async with AsyncExitStack() as stack:
ski_read, ski_write = await stack.enter_async_context(stdio_client(ski_parameters))
alpha_read, alpha_write = await stack.enter_async_context(
stdio_client(alpha_parameters)
)
ski_session = await stack.enter_async_context(ClientSession(ski_read, ski_write))
alpha_session = await stack.enter_async_context(
ClientSession(alpha_read, alpha_write)
)
await asyncio.gather(ski_session.initialize(), alpha_session.initialize())
ski_info, alpha_info = await asyncio.gather(
ski_session.call_tool("docforge_project_info", {}),
alpha_session.call_tool("docforge_project_info", {}),
)
ski_own, alpha_own, ski_cross, alpha_cross = await asyncio.gather(
ski_session.call_tool(
"docforge_get_node", {"node_id": "session.first-descent"}
),
alpha_session.call_tool("docforge_get_node", {"node_id": "guide.workflow"}),
ski_session.call_tool("docforge_get_node", {"node_id": "guide.workflow"}),
alpha_session.call_tool(
"docforge_get_node", {"node_id": "session.first-descent"}
),
)
ski_result, alpha_result = await asyncio.gather(
self._propose_and_preview(
ski_session,
node_id="riding.edge-control",
content_hash=node_hash(ski_project, "riding.edge-control"),
view_id="field-guide",
summary="A ski-only proposal from its bound server.",
),
self._propose_and_preview(
alpha_session,
node_id="guide.workflow",
content_hash=node_hash(alpha_project, "guide.workflow"),
view_id="manual",
summary="An Alpha-only proposal from its bound server.",
),
)
self.assertEqual("awesome-ski-game", ski_info.structuredContent["project_id"])
self.assertEqual("alpha-docs", alpha_info.structuredContent["project_id"])
self.assertEqual("current", ski_info.structuredContent["index_health"]["state"])
self.assertEqual("current", alpha_info.structuredContent["index_health"]["state"])
self.assertNotEqual(
ski_info.structuredContent["project_root_fingerprint"],
alpha_info.structuredContent["project_root_fingerprint"],
)
self.assertNotEqual(
ski_info.structuredContent["source_hash"],
alpha_info.structuredContent["source_hash"],
)
self.assertEqual("session.first-descent", ski_own.structuredContent["node"]["node_id"])
self.assertEqual("guide.workflow", alpha_own.structuredContent["node"]["node_id"])
self.assertEqual("missing_node", ski_cross.structuredContent["error"]["code"])
self.assertEqual("awesome-ski-game", ski_cross.structuredContent["project_id"])
self.assertEqual("missing_node", alpha_cross.structuredContent["error"]["code"])
self.assertEqual("alpha-docs", alpha_cross.structuredContent["project_id"])
self.assertEqual("ski-editor", ski_result["creator"])
self.assertEqual("alpha-editor", alpha_result["creator"])
ski_changeset = ski_root / ".docforge/changesets/parallel-edit.json"
alpha_changeset = alpha_root / ".docforge/changesets/parallel-edit.json"
ski_preview = ski_root / ".docforge/previews/parallel-edit/field-guide.html"
alpha_preview = alpha_root / ".docforge/previews/parallel-edit/manual.html"
ski_index = ski_root / ".docforge/cache/index.sqlite3"
alpha_index = alpha_root / ".docforge/cache/index.sqlite3"
self.assertTrue(ski_index.is_file())
self.assertTrue(alpha_index.is_file())
self.assertTrue(ski_index.resolve().is_relative_to(ski_root))
self.assertTrue(alpha_index.resolve().is_relative_to(alpha_root))
self.assertTrue(ski_changeset.is_file())
self.assertTrue(alpha_changeset.is_file())
self.assertTrue(ski_preview.is_file())
self.assertTrue(alpha_preview.is_file())
ski_html = ski_preview.read_text(encoding="utf-8")
alpha_html = alpha_preview.read_text(encoding="utf-8")
self.assertIn('data-project="awesome-ski-game"', ski_html)
self.assertNotIn('data-project="alpha-docs"', ski_html)
self.assertIn('data-project="alpha-docs"', alpha_html)
self.assertNotIn('data-project="awesome-ski-game"', alpha_html)
self.assertFalse((ski_root / ".docforge/rendered/field-guide.html").exists())
self.assertFalse((alpha_root / ".docforge/rendered/manual.html").exists())
self.assertEqual(ski_before, canonical_digest(ski_root))
self.assertEqual(alpha_before, canonical_digest(alpha_root))
async def _propose_and_preview(
self,
session: ClientSession,
*,
node_id: str,
content_hash: str,
view_id: str,
summary: str,
) -> dict[str, object]:
created = await session.call_tool(
"docforge_create_changeset", {"changeset_id": "parallel-edit"}
)
updated = await session.call_tool(
"docforge_propose_node_update",
{
"changeset_id": "parallel-edit",
"expected_changeset_hash": created.structuredContent["changeset_hash"],
"node_id": node_id,
"expected_content_hash": content_hash,
"metadata": {"summary": summary},
"content": None,
"relationship_changes": [],
"rationale": "Prove simultaneous project-bound proposal isolation.",
},
)
validated = await session.call_tool(
"docforge_validate_changeset", {"changeset_id": "parallel-edit"}
)
preview = await session.call_tool(
"docforge_preview_changeset",
{"changeset_id": "parallel-edit", "view_id": view_id},
)
self.assertEqual("ok", updated.structuredContent["status"])
self.assertTrue(validated.structuredContent["valid"])
self.assertEqual("current", preview.structuredContent["state"])
return {
"creator": created.structuredContent["creator"],
"changeset_hash": updated.structuredContent["changeset_hash"],
"preview_identity": preview.structuredContent["preview_identity"],
}
if __name__ == "__main__":
unittest.main()