1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

Bind visualization status to snapshot freshness

This commit is contained in:
Andraxion 2026-07-29 05:23:52 -04:00
parent 24bd13f9d9
commit 176b2d2784
9 changed files with 627 additions and 43 deletions

View file

@ -241,6 +241,32 @@ will be recorded only from a clean committed revision. The historical Milestone
behaviorally unchanged as comparison evidence; it only exposes shared fixture and measurement behaviorally unchanged as comparison evidence; it only exposes shared fixture and measurement
helpers to the Milestone 1 harness. helpers to the Milestone 1 harness.
#### Visualization snapshot freshness
Visualization workers now receive a version-1 snapshot specification containing the exact
validated index publication signature: device, inode, size, modification time, and change time.
Both the manager and worker reject a launch if that publication changes before startup. The
transmitted project root, root fingerprint, source identity, adapter, counts, limits, and confined
index path are strictly validated before the worker may serve source or graph data.
Worker health reports index freshness through stat-only comparison. It does not open SQLite and
does not renew the browser activity lease. The version-2 viewer-manager protocol validates the
complete worker identity and distinguishes an unreachable worker from a live stale worker. A stale
worker stays lifecycle `running` for accurate diagnosis, but the next visualize request stops it
and launches a newly validated snapshot instead of reusing it.
Client status separately compares the worker's pinned source identity with
`IncrementalStateProject.incremental_state()`. The composite snapshot is stale if either proof is
stale, current only when both proofs are current, and unknown otherwise. A stopped worker has
unknown snapshot identity. MCP preserves this state at the top-level `staleness` field and disables
recovery synchronization and full-load error decoration.
Tests cover signature mutation before worker startup, malformed identity, missing and symlinked
indexes, stat-only health, unchanged activity, current/unknown/stale source states, live stale
workers, non-reuse, and zero-load status. The Milestone 1 benchmark now measures current, stale,
not-running, and unavailable visualization status separately with the same zero-work and 50 ms p95
gates as other receipt status operations.
### Initial design constraints ### Initial design constraints
- Full rebuild remains the recovery and equivalence oracle. - Full rebuild remains the recovery and equivalence oracle.
@ -262,7 +288,7 @@ These are notes, not commitments:
- A durable telemetry exporter remains deliberately deferred. Request-local bounded aggregates are - A durable telemetry exporter remains deliberately deferred. Request-local bounded aggregates are
enough to prove compiler work in Milestone 1 without adding persistence, cardinality, or privacy enough to prove compiler work in Milestone 1 without adding persistence, cardinality, or privacy
risks. risks.
- Visualization freshness needs a separate source/index snapshot contract. Lifecycle health alone - The stat identity is a cheap publication proof, not a cryptographic integrity scan. Full index
must not be relabeled as current documentation state. validation remains the launch and query oracle.
- Large context and changeset payloads may need cursor pagination or compact immutable receipts. - Large context and changeset payloads may need cursor pagination or compact immutable receipts.
The choice should follow actual client workflows rather than generic pagination machinery. The choice should follow actual client workflows rather than generic pagination machinery.

View file

@ -182,9 +182,16 @@ bridges its retained predecessors and successors with an explicit omitted path.
a project-bound worker owned by a project-bound worker owned by
the separately supervised per-user viewer manager. Standard-input transaction completion and MCP the separately supervised per-user viewer manager. Standard-input transaction completion and MCP
host exit do not close the listener. Repeated visualization requests reuse the current worker while host exit do not close the listener. Repeated visualization requests reuse the current worker while
its exact snapshot remains valid. `docforge_visualization_status` reports lifecycle state, and its exact snapshot remains valid. The version-2 manager protocol binds each worker to the exact
`docforge_stop_visualization` explicitly stops the current project's worker. The manager reclaims a validated five-field index publication signature. Health checks compare that signature without
worker only after one hour with no browser activity. opening SQLite. A stale worker remains `state = running` but is never reused.
`docforge_visualization_status` reports lifecycle and freshness independently. `snapshot_state` and
top-level `staleness` are `stale` when either the index or cheap source identity is proven stale,
`current` only when both are proven current, and `unknown` otherwise. The `freshness` object exposes
the separate index and source states. Status never checks, synchronizes, or rebuilds the index and
never performs a complete project load. `docforge_stop_visualization` explicitly stops the current
project's worker. The manager reclaims a worker only after one hour with no browser activity.
## Excluded tools ## Excluded tools

View file

@ -254,6 +254,17 @@ docforge --project-root "$PROJECT" visualize --query persistence
The command opens the default browser. Add `--no-open` when a script only needs the returned JSON The command opens the default browser. Add `--no-open` when a script only needs the returned JSON
URL. Use `visualization-status` and `visualization-stop` to inspect or stop the project viewer. URL. Use `visualization-status` and `visualization-stop` to inspect or stop the project viewer.
Status separates the worker lifecycle from snapshot freshness. A worker may remain `running` while
`snapshot_state` is `stale`; it will not be reused by the next `visualize` call. `freshness.index`
checks the exact pinned index publication with file identity only. `freshness.source` compares the
cheap project generation when the project can prove one. Unavailable proof is `unknown`, never
silently `current`. Status does not load project content, open SQLite, rebuild the index, or renew
browser activity.
The freshness protocol requires viewer manager version 2. After upgrading an already running
installation, rerun `docforge-viewer-manager install-user-service` or restart the foreground
manager before requesting status.
## Visualization usage ## Visualization usage
- Left-click a node for its compact descriptor. - Left-click a node for its compact descriptor.

View file

@ -4,6 +4,7 @@ from __future__ import annotations
import argparse import argparse
import json import json
import math
import os import os
import plistlib import plistlib
import secrets import secrets
@ -25,15 +26,27 @@ from typing import BinaryIO, cast
from .errors import DocForgeError from .errors import DocForgeError
from .index import ProjectIndex from .index import ProjectIndex
from .models import IncrementalStateProject
from .project import project_root_fingerprint from .project import project_root_fingerprint
from .telemetry import increment, stage from .telemetry import increment, stage
from .visualization import VISUALIZATION_TEMPLATE, VisualizationIndexSnapshot from .visualization import VISUALIZATION_TEMPLATE, VisualizationIndexSnapshot
MANAGER_PROTOCOL = "docforge-viewer-manager@1" MANAGER_PROTOCOL = "docforge-viewer-manager@2"
MANAGER_RUNTIME = "viewer-manager@1" MANAGER_RUNTIME = "viewer-manager@2"
DEFAULT_IDLE_TIMEOUT_SECONDS = 3600.0 DEFAULT_IDLE_TIMEOUT_SECONDS = 3600.0
DEFAULT_CHECK_INTERVAL_SECONDS = 30.0 DEFAULT_CHECK_INTERVAL_SECONDS = 30.0
MAX_MESSAGE_BYTES = 1_000_000 MAX_MESSAGE_BYTES = 1_000_000
WORKER_SNAPSHOT_KEYS = frozenset(
{
"project_id",
"project_root_fingerprint",
"revision",
"source_hash",
"adapter",
"node_count",
"edge_count",
}
)
def default_runtime_root() -> Path: def default_runtime_root() -> Path:
@ -225,6 +238,12 @@ class _ManagedWorker:
last_activity_at: float last_activity_at: float
@dataclass(frozen=True)
class _WorkerHealth:
last_activity_at: float
index_state: str
class ViewerManager: class ViewerManager:
"""Own project viewer processes behind an authenticated loopback control API.""" """Own project viewer processes behind an authenticated loopback control API."""
@ -426,12 +445,13 @@ class ViewerManager:
self._workers.pop(key, None) self._workers.pop(key, None)
self._stop_worker(worker) self._stop_worker(worker)
return {"status": "ok", "state": "not_running"} return {"status": "ok", "state": "not_running"}
worker.last_activity_at = activity worker.last_activity_at = activity.last_activity_at
return { return {
"status": "ok", "status": "ok",
"state": "running", "state": "running",
"snapshot": dict(worker.snapshot), "snapshot": dict(worker.snapshot),
"idle_seconds": max(0, int(time.time() - activity)), "index_state": activity.index_state,
"idle_seconds": max(0, int(time.time() - activity.last_activity_at)),
"idle_timeout_seconds": self.idle_timeout_seconds, "idle_timeout_seconds": self.idle_timeout_seconds,
} }
@ -501,9 +521,9 @@ class ViewerManager:
if worker.snapshot != snapshot.identity or worker.process.poll() is not None: if worker.snapshot != snapshot.identity or worker.process.poll() is not None:
return False return False
activity = self._health(worker) activity = self._health(worker)
if activity is None: if activity is None or activity.index_state != "current":
return False return False
worker.last_activity_at = activity worker.last_activity_at = activity.last_activity_at
return True return True
@staticmethod @staticmethod
@ -518,7 +538,7 @@ class ViewerManager:
worker.process.wait(timeout=2) worker.process.wait(timeout=2)
@staticmethod @staticmethod
def _health(worker: _ManagedWorker) -> float | None: def _health(worker: _ManagedWorker) -> _WorkerHealth | None:
request = urllib.request.Request( request = urllib.request.Request(
f"http://127.0.0.1:{worker.port}/{worker.token}/api/health", f"http://127.0.0.1:{worker.port}/{worker.token}/api/health",
headers={"Accept": "application/json"}, headers={"Accept": "application/json"},
@ -531,10 +551,25 @@ class ViewerManager:
if not isinstance(payload, dict): if not isinstance(payload, dict):
return None return None
payload = cast(dict[str, object], payload) payload = cast(dict[str, object], payload)
if payload.get("viewer") != "alive": if payload.get("status") != "ok" or payload.get("viewer") != "alive":
return None
if frozenset(worker.snapshot) != WORKER_SNAPSHOT_KEYS:
return None
if any(payload.get(key) != value for key, value in worker.snapshot.items()):
return None return None
activity = payload.get("last_activity_at") activity = payload.get("last_activity_at")
return float(activity) if isinstance(activity, int | float) else None index_state = payload.get("index_state")
if (
isinstance(activity, bool)
or not isinstance(activity, int | float)
or not math.isfinite(activity)
or index_state not in {"current", "stale"}
):
return None
return _WorkerHealth(
last_activity_at=float(activity),
index_state=cast(str, index_state),
)
def _result( def _result(
self, self,
@ -575,11 +610,11 @@ class ViewerManager:
with self._lock: with self._lock:
expired: list[tuple[str, _ManagedWorker]] = [] expired: list[tuple[str, _ManagedWorker]] = []
for key, worker in self._workers.items(): for key, worker in self._workers.items():
activity = self._health(worker) health = self._health(worker)
if activity is None or activity < cutoff: if health is None or health.last_activity_at < cutoff:
expired.append((key, worker)) expired.append((key, worker))
else: else:
worker.last_activity_at = activity worker.last_activity_at = health.last_activity_at
for key, worker in expired: for key, worker in expired:
self._workers.pop(key, None) self._workers.pop(key, None)
self._stop_worker(worker) self._stop_worker(worker)
@ -637,7 +672,77 @@ class ViewerManagerClient:
def status(self) -> dict[str, object]: def status(self) -> dict[str, object]:
with stage("visualization.status"): with stage("visualization.status"):
return self._lifecycle_request("status") response = self._lifecycle_request("status")
lifecycle = response.get("state")
if lifecycle not in {"running", "not_running"}:
raise DocForgeError(
"visualization_unavailable",
"Viewer manager returned an invalid lifecycle state",
)
if lifecycle != "running":
return {
**response,
"revision": "unknown",
"source_hash": None,
"snapshot_state": "unknown",
"staleness": "unknown",
"freshness": {
"index": "unknown",
"source": "unknown",
},
}
index_value = response.get("index_state")
index_state = (
cast(str, index_value) if index_value in {"current", "stale"} else "unknown"
)
snapshot_value = response.get("snapshot")
snapshot_payload: dict[str, object] = (
cast(dict[str, object], snapshot_value) if isinstance(snapshot_value, dict) else {}
)
source_state = self._source_state(snapshot_payload)
revision = snapshot_payload.get("revision")
source_hash = snapshot_payload.get("source_hash")
snapshot_state = (
"stale"
if "stale" in {index_state, source_state}
else (
"current"
if index_state == "current" and source_state == "current"
else "unknown"
)
)
return {
**response,
"revision": revision if isinstance(revision, str) else "unknown",
"source_hash": source_hash if isinstance(source_hash, str) else None,
"snapshot_state": snapshot_state,
"staleness": snapshot_state,
"freshness": {
"index": index_state,
"source": source_state,
},
}
def _source_state(self, snapshot: object) -> str:
project = self.index.project
if not isinstance(project, IncrementalStateProject) or not isinstance(snapshot, dict):
return "unknown"
snapshot = cast(dict[str, object], snapshot)
revision = snapshot.get("revision")
source_hash = snapshot.get("source_hash")
if not isinstance(revision, str) or not isinstance(source_hash, str):
return "unknown"
try:
state = project.incremental_state()
except (DocForgeError, OSError, RuntimeError, TypeError, ValueError):
return "unknown"
if state is None:
return "unknown"
return (
"current"
if state.revision == revision and state.source_hash == source_hash
else "stale"
)
def _lifecycle_request(self, action: str) -> dict[str, object]: def _lifecycle_request(self, action: str) -> dict[str, object]:
descriptor = self.index.project.descriptor descriptor = self.index.project.descriptor

View file

@ -9,6 +9,7 @@ import secrets
import signal import signal
import socket import socket
import sqlite3 import sqlite3
import stat
import sys import sys
import tempfile import tempfile
import threading import threading
@ -73,6 +74,9 @@ LEASE_MONITOR_INTERVAL_SECONDS = 1.0
VISUALIZATION_REGISTRY_NAME = ".visualization.json" VISUALIZATION_REGISTRY_NAME = ".visualization.json"
VISUALIZATION_LOCK_NAME = ".visualization.lock" VISUALIZATION_LOCK_NAME = ".visualization.lock"
VISUALIZATION_RUNTIME = "persistent-worker@1" VISUALIZATION_RUNTIME = "persistent-worker@1"
VISUALIZATION_SNAPSHOT_SCHEMA_VERSION = 1
IndexSignature = tuple[int, int, int, int, int]
class _VisualizationHttpServer(ThreadingHTTPServer): class _VisualizationHttpServer(ThreadingHTTPServer):
@ -103,10 +107,13 @@ class VisualizationIndexSnapshot:
self.max_depth = index.project.descriptor.limits.max_traversal_depth self.max_depth = index.project.descriptor.limits.max_traversal_depth
self.identity: dict[str, object] = {key: checked[key] for key in self._IDENTITY_KEYS} self.identity: dict[str, object] = {key: checked[key] for key in self._IDENTITY_KEYS}
self._stat = self._safe_stat() self._stat = self._safe_stat()
self._validate_snapshot()
@classmethod @classmethod
def from_spec(cls, spec: dict[str, object]) -> VisualizationIndexSnapshot: def from_spec(cls, spec: dict[str, object]) -> VisualizationIndexSnapshot:
snapshot = cls.__new__(cls) snapshot = cls.__new__(cls)
if spec.get("schema_version") != VISUALIZATION_SNAPSHOT_SCHEMA_VERSION:
raise DocForgeError("invalid_index", "Visualization snapshot version is invalid")
path = spec["path"] path = spec["path"]
title = spec["title"] title = spec["title"]
project_root = spec["project_root"] project_root = spec["project_root"]
@ -117,11 +124,16 @@ class VisualizationIndexSnapshot:
if ( if (
not isinstance(path, str) not isinstance(path, str)
or not isinstance(title, str) or not isinstance(title, str)
or not title
or not isinstance(project_root, str) or not isinstance(project_root, str)
or type(max_source_bytes) is not int or type(max_source_bytes) is not int
or max_source_bytes < 1
or type(max_query_chars) is not int or type(max_query_chars) is not int
or max_query_chars < 1
or type(max_results) is not int or type(max_results) is not int
or max_results < 1
or type(max_depth) is not int or type(max_depth) is not int
or max_depth < 1
): ):
raise DocForgeError("invalid_index", "Visualization snapshot is invalid") raise DocForgeError("invalid_index", "Visualization snapshot is invalid")
snapshot.path = Path(path) snapshot.path = Path(path)
@ -131,6 +143,12 @@ class VisualizationIndexSnapshot:
raise DocForgeError("invalid_index", "Visualization project root is invalid") from error raise DocForgeError("invalid_index", "Visualization project root is invalid") from error
if not snapshot.project_root.is_dir(): if not snapshot.project_root.is_dir():
raise DocForgeError("invalid_index", "Visualization project root is invalid") raise DocForgeError("invalid_index", "Visualization project root is invalid")
if not snapshot.path.is_absolute():
raise DocForgeError("invalid_index", "Visualization index path is invalid")
try:
snapshot.path.relative_to(snapshot.project_root)
except ValueError as error:
raise DocForgeError("invalid_index", "Visualization index path is invalid") from error
snapshot.title = title snapshot.title = title
snapshot.max_source_bytes = max_source_bytes snapshot.max_source_bytes = max_source_bytes
snapshot.max_query_chars = max_query_chars snapshot.max_query_chars = max_query_chars
@ -139,13 +157,59 @@ class VisualizationIndexSnapshot:
identity = spec["identity"] identity = spec["identity"]
if not isinstance(identity, dict): if not isinstance(identity, dict):
raise DocForgeError("invalid_index", "Visualization identity is invalid") raise DocForgeError("invalid_index", "Visualization identity is invalid")
typed_identity = cast(dict[str, object], identity) snapshot.identity = cls._parse_identity(
snapshot.identity = {key: typed_identity[key] for key in cls._IDENTITY_KEYS} cast(dict[str, object], identity),
snapshot._stat = snapshot._safe_stat() snapshot.project_root,
)
snapshot._stat = cls._parse_index_signature(spec.get("index_signature"))
if snapshot.index_state() != "current":
raise DocForgeError(
"visualization_stale",
"The validated index changed before the visualization worker started",
)
snapshot._validate_snapshot()
return snapshot return snapshot
@classmethod
def _parse_identity(
cls,
value: dict[str, object],
project_root: Path,
) -> dict[str, object]:
if set(value) != set(cls._IDENTITY_KEYS):
raise DocForgeError("invalid_index", "Visualization identity is invalid")
project_id = value.get("project_id")
fingerprint = value.get("project_root_fingerprint")
revision = value.get("revision")
source_hash = value.get("source_hash")
adapter = value.get("adapter")
node_count = value.get("node_count")
edge_count = value.get("edge_count")
if (
not isinstance(project_id, str)
or not project_id
or not isinstance(fingerprint, str)
or len(fingerprint) != 16
or any(character not in "0123456789abcdef" for character in fingerprint)
or fingerprint != project_root_fingerprint(project_root)
or not isinstance(revision, str)
or not revision
or not isinstance(source_hash, str)
or len(source_hash) != 64
or any(character not in "0123456789abcdef" for character in source_hash)
or not isinstance(adapter, str)
or not adapter
or type(node_count) is not int
or node_count < 0
or type(edge_count) is not int
or edge_count < 0
):
raise DocForgeError("invalid_index", "Visualization identity is invalid")
return {key: value[key] for key in cls._IDENTITY_KEYS}
def spec(self) -> dict[str, object]: def spec(self) -> dict[str, object]:
return { return {
"schema_version": VISUALIZATION_SNAPSHOT_SCHEMA_VERSION,
"path": str(self.path), "path": str(self.path),
"project_root": str(self.project_root), "project_root": str(self.project_root),
"title": self.title, "title": self.title,
@ -154,8 +218,44 @@ class VisualizationIndexSnapshot:
"max_results": self.max_results, "max_results": self.max_results,
"max_depth": self.max_depth, "max_depth": self.max_depth,
"identity": dict(self.identity), "identity": dict(self.identity),
"index_signature": {
"schema_version": 1,
"device": self._stat[0],
"inode": self._stat[1],
"size": self._stat[2],
"mtime_ns": self._stat[3],
"ctime_ns": self._stat[4],
},
} }
@staticmethod
def _parse_index_signature(value: object) -> IndexSignature:
if not isinstance(value, dict):
raise DocForgeError("invalid_index", "Visualization index signature is invalid")
payload = cast(dict[str, object], value)
if payload.get("schema_version") != 1:
raise DocForgeError("invalid_index", "Visualization index signature is invalid")
fields = ("device", "inode", "size", "mtime_ns", "ctime_ns")
values: list[int] = []
for field in fields:
item = payload.get(field)
if type(item) is not int or item < 0:
raise DocForgeError("invalid_index", "Visualization index signature is invalid")
values.append(item)
return cast(IndexSignature, tuple(values))
def index_state(self) -> str:
"""Return cheap publication freshness without opening SQLite."""
try:
return "current" if self._safe_stat() == self._stat else "stale"
except DocForgeError:
return "stale"
def _validate_snapshot(self) -> None:
with self._connection():
pass
def overview(self) -> dict[str, object]: def overview(self) -> dict[str, object]:
with self._connection() as connection: with self._connection() as connection:
return self._result( return self._result(
@ -749,15 +849,26 @@ class VisualizationIndexSnapshot:
raise DocForgeError("invalid_limit", "Result limit is outside the configured range") raise DocForgeError("invalid_limit", "Result limit is outside the configured range")
return value return value
def _safe_stat(self) -> tuple[int, int, int, int]: def _safe_stat(self) -> IndexSignature:
if ( try:
self.path.is_symlink() status = self.path.lstat()
or not self.path.is_file() if not stat.S_ISREG(status.st_mode) or self.path.resolve(strict=True) != self.path:
or self.path.resolve(strict=True) != self.path raise DocForgeError(
): "missing_index",
raise DocForgeError("missing_index", "Validated visualization index is unavailable") "Validated visualization index is unavailable",
stat = self.path.stat() )
return (stat.st_dev, stat.st_ino, stat.st_size, stat.st_mtime_ns) except OSError as error:
raise DocForgeError(
"missing_index",
"Validated visualization index is unavailable",
) from error
return (
status.st_dev,
status.st_ino,
status.st_size,
status.st_mtime_ns,
status.st_ctime_ns,
)
@contextmanager @contextmanager
def _connection(self) -> Generator[sqlite3.Connection, None, None]: def _connection(self) -> Generator[sqlite3.Connection, None, None]:
@ -1074,7 +1185,11 @@ class VisualizationRunner:
if parsed.path == f"{prefix}/api/health": if parsed.path == f"{prefix}/api/health":
with self._lock: with self._lock:
last_activity = self._activity_last_seen last_activity = self._activity_last_seen
payload = reader.result(viewer="alive", last_activity_at=last_activity) payload = reader.result(
viewer="alive",
last_activity_at=last_activity,
index_state=reader.index_state(),
)
elif parsed.path == f"{prefix}/api/overview": elif parsed.path == f"{prefix}/api/overview":
self._touch_lease() self._touch_lease()
payload = reader.overview() payload = reader.overview()

View file

@ -533,6 +533,20 @@ class AdapterContractTests(unittest.TestCase):
self.assertEqual(0, read_counters["adapter_source_extractions"]) self.assertEqual(0, read_counters["adapter_source_extractions"])
self.assertEqual(0, read_counters["index_builds"]) self.assertEqual(0, read_counters["index_builds"])
pinned_state = project.incremental_state()
assert pinned_state is not None
loader.sources["guide.foundation"] = "Changed after viewer pin."
client = ViewerManagerClient(index)
self.assertEqual(
"stale",
client._source_state(
{
"revision": pinned_state.revision,
"source_hash": pinned_state.source_hash,
}
),
)
legacy = AdapterProject( legacy = AdapterProject(
Loader(self.projection(root)), Loader(self.projection(root)),
cache_root=root / ".cache" / "legacy", cache_root=root / ".cache" / "legacy",
@ -544,6 +558,15 @@ class AdapterContractTests(unittest.TestCase):
self.assertEqual(1, legacy_counters["project_loads"]) self.assertEqual(1, legacy_counters["project_loads"])
self.assertEqual(1, legacy_counters["adapter_projection_loads"]) self.assertEqual(1, legacy_counters["adapter_projection_loads"])
self.assertEqual(0, legacy_counters["adapter_source_extractions"]) self.assertEqual(0, legacy_counters["adapter_source_extractions"])
self.assertEqual(
"unknown",
ViewerManagerClient(ProjectIndex(legacy))._source_state(
{
"revision": "legacy",
"source_hash": "0" * 64,
}
),
)
def test_incremental_adapter_reuses_sources_and_invalidates_reverse_dependencies(self) -> None: def test_incremental_adapter_reuses_sources_and_invalidates_reverse_dependencies(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:

View file

@ -193,7 +193,7 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
finally: finally:
service.visualization.stop() service.visualization.stop()
for result in results: for position, result in enumerate(results):
self.assertFalse(result.isError) self.assertFalse(result.isError)
self.assertIsNotNone(result.structuredContent) self.assertIsNotNone(result.structuredContent)
payload = result.structuredContent payload = result.structuredContent
@ -201,7 +201,10 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertEqual("alpha-docs", payload["project_id"]) self.assertEqual("alpha-docs", payload["project_id"])
self.assertEqual(CONTENT_WARNING, payload["content_warning"]) self.assertEqual(CONTENT_WARNING, payload["content_warning"])
self.assertTrue(payload["project_root_fingerprint"]) self.assertTrue(payload["project_root_fingerprint"])
self.assertEqual("current", payload["staleness"]) self.assertEqual(
"unknown" if position == 14 else "current",
payload["staleness"],
)
contract = results[1].structuredContent contract = results[1].structuredContent
self.assertFalse(contract["canonical_writes_allowed"]) self.assertFalse(contract["canonical_writes_allowed"])
self.assertFalse(contract["project_switching_allowed"]) self.assertFalse(contract["project_switching_allowed"])
@ -222,6 +225,13 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertTrue(visualization["url"].startswith("http://127.0.0.1:")) self.assertTrue(visualization["url"].startswith("http://127.0.0.1:"))
self.assertEqual("stopped", results[13].structuredContent["state"]) self.assertEqual("stopped", results[13].structuredContent["state"])
self.assertEqual("not_running", results[14].structuredContent["state"]) self.assertEqual("not_running", results[14].structuredContent["state"])
self.assertEqual("unknown", results[14].structuredContent["revision"])
self.assertIsNone(results[14].structuredContent["source_hash"])
self.assertEqual("unknown", results[14].structuredContent["snapshot_state"])
self.assertEqual(
{"index": "unknown", "source": "unknown"},
results[14].structuredContent["freshness"],
)
context = results[9].structuredContent context = results[9].structuredContent
self.assertLessEqual(context["estimated_tokens"], 180) self.assertLessEqual(context["estimated_tokens"], 180)
self.assertTrue(context["omissions"]) self.assertTrue(context["omissions"])

View file

@ -12,11 +12,14 @@ import urllib.parse
import urllib.request import urllib.request
from contextlib import contextmanager from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from unittest import mock
from docforge.errors import DocForgeError from docforge.errors import DocForgeError
from docforge.index import ProjectIndex from docforge.index import ProjectIndex
from docforge.mcp_server import DocForgeService
from docforge.models import ProjectState
from docforge.project import Project from docforge.project import Project
from docforge.viewer_manager import ViewerManager, ViewerManagerClient from docforge.viewer_manager import ViewerManager, ViewerManagerClient, _ManagedWorker
from docforge.visualization import ( from docforge.visualization import (
_GRAPH_BROWSER_CSS, _GRAPH_BROWSER_CSS,
_GRAPH_BROWSER_HTML, _GRAPH_BROWSER_HTML,
@ -61,6 +64,256 @@ class VisualizationTests(unittest.TestCase):
manager.shutdown() manager.shutdown()
thread.join(timeout=2) thread.join(timeout=2)
def test_snapshot_spec_binds_the_exact_validated_index_publication(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
index = ProjectIndex(Project.open(root))
index.build()
snapshot = VisualizationIndexSnapshot(index, index.check())
spec = snapshot.spec()
self.assertEqual(1, spec["schema_version"])
self.assertEqual(1, spec["index_signature"]["schema_version"])
self.assertEqual("current", VisualizationIndexSnapshot.from_spec(spec).index_state())
malformed = {**spec, "index_signature": {"schema_version": 1}}
with self.assertRaises(DocForgeError) as invalid:
VisualizationIndexSnapshot.from_spec(malformed)
self.assertEqual("invalid_index", invalid.exception.code)
wrong_fingerprint = {
**spec,
"identity": {
**spec["identity"],
"project_root_fingerprint": "0" * 16,
},
}
with self.assertRaises(DocForgeError) as invalid_fingerprint:
VisualizationIndexSnapshot.from_spec(wrong_fingerprint)
self.assertEqual("invalid_index", invalid_fingerprint.exception.code)
string_count = {
**spec,
"identity": {
**spec["identity"],
"node_count": str(spec["identity"]["node_count"]),
},
}
with self.assertRaises(DocForgeError) as invalid_count:
VisualizationIndexSnapshot.from_spec(string_count)
self.assertEqual("invalid_index", invalid_count.exception.code)
with index.path.open("ab") as stream:
stream.write(b"\n")
self.assertEqual("stale", snapshot.index_state())
with self.assertRaises(DocForgeError) as stale:
VisualizationIndexSnapshot.from_spec(spec)
self.assertEqual("visualization_stale", stale.exception.code)
def test_snapshot_index_state_rejects_missing_and_symlinked_publications(self) -> None:
for mutation in ("delete", "replace", "symlink"):
with self.subTest(mutation=mutation), tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
index = ProjectIndex(Project.open(root))
index.build()
snapshot = VisualizationIndexSnapshot(index, index.check())
if mutation == "delete":
index.path.unlink()
elif mutation == "replace":
replacement = index.path.with_suffix(".replacement")
shutil.copy2(index.path, replacement)
replacement.replace(index.path)
else:
backup = index.path.with_suffix(".backup")
index.path.rename(backup)
index.path.symlink_to(backup)
self.assertEqual("stale", snapshot.index_state())
def test_health_is_stat_only_and_reports_stale_without_renewing_activity(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
index = ProjectIndex(Project.open(root))
index.build()
runner = VisualizationRunner(index)
try:
result = runner.start()
url = str(result["url"]).split("?", 1)[0] + "api/health"
with (
mock.patch(
"docforge.visualization.sqlite3.connect",
side_effect=AssertionError("health opened SQLite"),
),
urllib.request.urlopen(url, timeout=2) as response,
):
current = json.load(response)
self.assertEqual("current", current["index_state"])
with index.path.open("ab") as stream:
stream.write(b"\n")
with (
mock.patch(
"docforge.visualization.sqlite3.connect",
side_effect=AssertionError("health opened SQLite"),
),
urllib.request.urlopen(url, timeout=2) as response,
):
stale = json.load(response)
self.assertEqual("stale", stale["index_state"])
self.assertEqual(current["last_activity_at"], stale["last_activity_at"])
finally:
runner.stop()
def test_manager_health_rejects_malformed_and_identity_mismatched_payloads(self) -> None:
snapshot = {
"project_id": "alpha-docs",
"project_root_fingerprint": "0" * 16,
"revision": "revision",
"source_hash": "a" * 64,
"adapter": "generic",
"node_count": 3,
"edge_count": 2,
}
worker = _ManagedWorker(
process=mock.Mock(),
port=12345,
token="token",
snapshot=snapshot,
last_activity_at=1.0,
)
valid = {
"status": "ok",
"viewer": "alive",
"last_activity_at": 1.0,
"index_state": "current",
**snapshot,
}
invalid_payloads = (
{key: value for key, value in valid.items() if key != "status"},
{**valid, "project_id": "other"},
{**valid, "last_activity_at": True},
{**valid, "last_activity_at": float("nan")},
{key: value for key, value in valid.items() if key != "index_state"},
)
for payload in invalid_payloads:
with self.subTest(payload=payload):
response = mock.MagicMock()
response.__enter__.return_value.read.return_value = json.dumps(payload).encode()
with mock.patch(
"docforge.viewer_manager.urllib.request.urlopen",
return_value=response,
):
self.assertIsNone(ViewerManager._health(worker))
def test_manager_status_separates_lifecycle_index_and_source_freshness(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
index.build()
state_path = Path(directory) / "viewer-manager.json"
with self.running_manager(state_path) as manager:
client = ViewerManagerClient(index, state_path=state_path)
first = client.start()
with (
mock.patch.object(
project,
"load",
side_effect=AssertionError("status loaded the project"),
),
mock.patch.object(
index,
"check",
side_effect=AssertionError("status checked the index"),
),
mock.patch.object(
index,
"build",
side_effect=AssertionError("status built the index"),
),
mock.patch.object(
index,
"synchronize",
side_effect=AssertionError("status synchronized the index"),
),
):
current = client.status()
self.assertEqual("running", current["state"])
self.assertEqual("current", current["snapshot_state"])
self.assertEqual(
{"index": "current", "source": "current"},
current["freshness"],
)
self.assertEqual(first["snapshot"]["source_hash"], current["source_hash"])
service = DocForgeService(project, diagnostics=True)
service.visualization = ViewerManagerClient(
service.index,
state_path=state_path,
)
mcp_current = service.visualization_status()
counters = mcp_current["diagnostics"]["counters"]
self.assertEqual(0, counters["project_loads"])
self.assertEqual(0, counters["source_files_parsed"])
self.assertEqual(0, counters["index_checks"])
self.assertEqual(0, counters["index_synchronizations"])
self.assertEqual(0, counters["index_builds"])
self.assertEqual(1, counters["viewer_manager_requests"])
project.generation_path.unlink()
unknown = client.status()
self.assertEqual("running", unknown["state"])
self.assertEqual("unknown", unknown["snapshot_state"])
self.assertEqual("unknown", unknown["freshness"]["source"])
index.build()
stale = client.status()
self.assertEqual("running", stale["state"])
self.assertEqual("stale", stale["snapshot_state"])
self.assertEqual("stale", stale["freshness"]["index"])
time.sleep(0.06)
self.assertEqual(1, len(manager._workers))
restarted = client.start()
self.assertFalse(restarted["reused"])
self.assertNotEqual(
str(first["url"]).split("?", 1)[0],
str(restarted["url"]).split("?", 1)[0],
)
self.assertEqual("current", client.status()["snapshot_state"])
def test_client_source_freshness_distinguishes_mismatch_and_unknown(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
index = ProjectIndex(project)
checked = index.build()
client = ViewerManagerClient(index)
response = {
"status": "ok",
"state": "running",
"index_state": "current",
"snapshot": {
"revision": checked["revision"],
"source_hash": checked["source_hash"],
},
"project_id": project.descriptor.project_id,
"project_root_fingerprint": "test",
"adapter": project.descriptor.adapter,
}
with mock.patch.object(client, "_lifecycle_request", return_value=response):
with mock.patch.object(
project,
"incremental_state",
return_value=ProjectState(source_hash="f" * 64, revision="changed"),
):
stale = client.status()
self.assertEqual("stale", stale["freshness"]["source"])
self.assertEqual("stale", stale["snapshot_state"])
with mock.patch.object(project, "incremental_state", return_value=None):
unknown = client.status()
self.assertEqual("unknown", unknown["freshness"]["source"])
self.assertEqual("unknown", unknown["snapshot_state"])
def test_overview_and_neighborhood_are_deterministic_and_bounded(self) -> None: def test_overview_and_neighborhood_are_deterministic_and_bounded(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory)) root = self.copy_fixture("alpha", Path(directory))

View file

@ -9,6 +9,8 @@ import resource
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import threading
import time
from collections.abc import Callable, Mapping from collections.abc import Callable, Mapping
from pathlib import Path from pathlib import Path
from typing import cast from typing import cast
@ -24,7 +26,7 @@ from docforge.index import ProjectIndex
from docforge.mcp_server import DocForgeService from docforge.mcp_server import DocForgeService
from docforge.project import Project from docforge.project import Project
from docforge.rendering import RenderService from docforge.rendering import RenderService
from docforge.viewer_manager import ViewerManagerClient from docforge.viewer_manager import ViewerManager, ViewerManagerClient
ROOT = Path(__file__).resolve().parents[1] ROOT = Path(__file__).resolve().parents[1]
ZERO_WORK_COUNTERS = ( ZERO_WORK_COUNTERS = (
@ -107,10 +109,6 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
ProjectIndex(project).build() ProjectIndex(project).build()
RenderService(project).render("manual") RenderService(project).render("manual")
service = DocForgeService(project, diagnostics=True) service = DocForgeService(project, diagnostics=True)
service.visualization = ViewerManagerClient(
service.index,
state_path=root / ".docforge" / "missing-viewer-manager.json",
)
target = synthetic_node_id(node_count - 1) target = synthetic_node_id(node_count - 1)
operations = { operations = {
"warm_no_change_synchronize": _operation( "warm_no_change_synchronize": _operation(
@ -164,13 +162,49 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
samples=samples, samples=samples,
p95_limit_ms=50, p95_limit_ms=50,
), ),
"visualization_unavailable_status": _operation( }
state_path = root / ".docforge" / "benchmark-viewer-manager.json"
manager = ViewerManager(state_path, check_interval_seconds=0.02)
manager_thread = threading.Thread(target=manager.serve_forever, daemon=True)
manager_thread.start()
deadline = time.monotonic() + 2
while not state_path.exists() and time.monotonic() < deadline:
time.sleep(0.01)
if not state_path.exists():
manager.shutdown()
manager_thread.join(timeout=2)
raise RuntimeError("Viewer manager did not start")
service.visualization = ViewerManagerClient(service.index, state_path=state_path)
try:
service.visualization.start()
operations["visualization_current_status"] = _operation(
service.visualization_status,
samples=samples,
p95_limit_ms=50,
)
with service.index.path.open("ab") as stream:
stream.write(b"\n")
operations["visualization_stale_status"] = _operation(
service.visualization_status,
samples=samples,
p95_limit_ms=50,
)
service.stop_visualization()
operations["visualization_not_running_status"] = _operation(
service.visualization_status,
samples=samples,
p95_limit_ms=50,
)
finally:
manager.shutdown()
manager_thread.join(timeout=2)
service.visualization = ViewerManagerClient(service.index, state_path=state_path)
operations["visualization_unavailable_status"] = _operation(
service.visualization_status, service.visualization_status,
samples=samples, samples=samples,
p95_limit_ms=50, p95_limit_ms=50,
expected_status="error", expected_status="error",
), )
}
return { return {
"fixture": { "fixture": {
"kind": "synthetic_generic", "kind": "synthetic_generic",