1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

Manage graph viewers with a supervised local service

This commit is contained in:
Andraxion 2026-07-25 00:17:21 -04:00
parent eb48ba1a51
commit 7c87536167
14 changed files with 1027 additions and 103 deletions

View file

@ -46,9 +46,9 @@ relation-specific colors, line patterns, directional symbols, and an exact visib
constructs a bounded upstream lineage with semantic direction for execution, data, and dependency
relations while excluding structural and evidence context.
DFG-18 makes the viewer a truly persistent project-bound local service. It survives MCP process
turnover and browser inactivity, reuses the current snapshot URL, and stops only when
`docforge_stop_visualization` explicitly requests it.
DFG-18 makes the viewer a managed project-bound local service. A native per-user supervisor owns
the viewer manager, which survives MCP process turnover, reuses the current snapshot URL, exposes
status and explicit stop tools, and reclaims only genuinely idle workers.
## Development
@ -100,8 +100,9 @@ surface.
Both generic and explicit adapter MCP servers expose the same visualization tools because they read
the validated `ProjectIndex` supplied by the project binding. `docforge_visualize` reuses its
unguessable loopback URL when the current snapshot remains valid, or replaces the worker after a
complete index check when it does not. The project-bound worker remains available until
`docforge_stop_visualization` explicitly stops it.
complete index check when it does not. The per-user viewer manager retains active workers across
MCP process turnover, reports state through `docforge_visualization_status`, and reclaims idle
workers after one hour. See [`docs/VIEWER_MANAGER.md`](docs/VIEWER_MANAGER.md) for installation.
See [`docs/NEW_PROJECT_QUICKSTART.md`](docs/NEW_PROJECT_QUICKSTART.md) for a complete generic MCP
setup, continuous-agent policy, visualization instructions, and a project-adapter checklist.

View file

@ -1,5 +1,22 @@
# Completed slices
## DFG-19 cross-platform supervised viewer manager
### Changed
- Released DocForge 0.12.0 with an authenticated loopback viewer-manager protocol.
- Moved viewer process ownership out of the MCP stdio host and into an OS-supervised per-user
manager. Linux uses systemd user services, macOS LaunchAgents, and Windows Task Scheduler.
- Added `docforge_visualization_status` and changed worker lifetime to a one-hour browser-activity
policy with explicit per-project stop.
- Replaced Unix-socket and inherited-file-descriptor assumptions with loopback TCP and standard
input/output worker control, so the manager protocol works on Windows as well as Unix platforms.
### Verification
- Lifecycle tests cover manager worker reuse, browser activity renewal, idle reclamation, explicit
stop, strict MCP surface registration, and manager-mediated visualization startup.
## DFG-18 persistent visualization lifecycle
### Changed

View file

@ -1,4 +1,4 @@
# DocForge 0.11 contract
# DocForge 0.12 contract
## Authority boundary
@ -23,7 +23,7 @@ approved contract and measured cross-project evidence; it is not an unimplemente
- Result envelope: `schemas/result.schema.json`, version 1.
- Changeset schema: `schemas/changeset.schema.json`, version 1.
- Index schema: version 1, disposable and reproducible.
- Core, CLI, and MCP server: version 0.11.0.
- Core, CLI, and MCP server: version 0.12.0.
Schema files describe the generic interchange contract. Runtime validation remains responsible for
path confinement, source hashing, relationship resolution, dependency cycles, project limits, stale
@ -147,16 +147,19 @@ shortest-hop calculation places Nodes on distance rings; Flow uses left-to-right
with the destination on the right. Each role palette darkens progressively by distance, capped at
fifty percent.
Each invocation creates or reuses one persistent local worker for the validated snapshot. The worker
is detached from the short-lived MCP transport and has no browser inactivity or owner-process
expiry. A repeated invocation reuses its unguessable URL when the validated snapshot is unchanged.
If the index has changed, DocForge replaces the worker only after a fresh complete index check.
Each invocation creates or reuses one worker through the separately supervised, per-user viewer
manager. The manager is outside the short-lived MCP transport and owns all child workers as one OS
service unit. It accepts only authenticated loopback requests and a validated immutable snapshot.
A repeated invocation reuses its unguessable URL when the snapshot is unchanged. If the index has
changed, it replaces the worker only after a fresh complete index check.
`docforge_stop_visualization` is the only normal shutdown path. It stops the current project's
verified worker and removes its private registry record. The worker also ends if the operating
system terminates it. Project-specific integrations receive the same tools because the parent
validates and serializes only the supplied `ProjectService` and `ProjectIndex` snapshot; the worker
does not discover projects or load canonical sources.
`docforge_visualization_status` reports the managed worker state. `docforge_stop_visualization`
explicitly stops the current project's worker. The manager applies a one-hour activity timeout to
genuinely abandoned workers. Browser activity renews that timeout, but normal MCP transaction or
host-process completion does not affect it. The manager itself is restarted by an OS-native,
per-user supervisor. Project-specific integrations receive the same tools because the parent
validates and serializes only the supplied `ProjectService` and `ProjectIndex` snapshot; workers do
not discover projects or load canonical sources.
## Project adapter boundary

View file

@ -21,6 +21,7 @@ same immutable project binding.
- `docforge_render_status`
- `docforge_visualize`
- `docforge_stop_visualization`
- `docforge_visualization_status`
Each response states that document text is project content, not higher-priority instructions. Each
response includes project identity, revision, source hash, adapter version, and staleness state.
@ -82,11 +83,12 @@ visible key. Its navigation groups the focus, nodes reachable through outgoing e
incoming or lateral context. Flow presents the same bounded snapshot as an upstream lineage.
Execution edges retain their declared direction; reads, imports, and dependencies reverse to show
what feeds the focus; structural, evidence, and context edges are excluded. The same relationship
key is regenerated from the visible Flow edges. The browser runs in a project-bound persistent
local worker, so standard-input transaction completion, MCP host exit, and browser inactivity do
not close the listener. Repeated visualization requests reuse the current worker while its exact
snapshot remains valid. `docforge_stop_visualization` explicitly stops the current project's
verified worker.
key is regenerated from the visible Flow edges. The browser runs in a project-bound worker owned by
the separately supervised per-user viewer manager. Standard-input transaction completion and MCP
host exit do not close the listener. Repeated visualization requests reuse the current worker while
its exact snapshot remains valid. `docforge_visualization_status` reports lifecycle state, and
`docforge_stop_visualization` explicitly stops the current project's worker. The manager reclaims a
worker only after one hour with no browser activity.
## Excluded tools

View file

@ -24,6 +24,7 @@ The DocForge repository contains the complete generic CLI and stdio MCP server.
panning, zoom controls, viewport reset, relationship color and symbol keys, and an upstream
lineage Flow view.
- `docforge_stop_visualization`, which explicitly stops the current project's persistent viewer.
- `docforge_visualization_status`, which reports the manager-owned viewer state.
- Isolated documentation changesets, proposal validation, diffs, and escaped HTML previews when a
proposal writer and render view are configured.
@ -185,7 +186,7 @@ Add a project-specific stdio MCP server to the agent host:
Restart or reload the agent host. Confirm that it exposes tools beginning with
`docforge_project_info`, `docforge_get_node`, `docforge_search`, `docforge_dependencies`,
`docforge_impact`, `docforge_get_context`, `docforge_visualize`, and
`docforge_stop_visualization`, and `docforge_create_changeset`.
`docforge_stop_visualization`, `docforge_visualization_status`, and `docforge_create_changeset`.
Omit `--proposal-writer` for a read-only integration.
@ -339,7 +340,9 @@ Place this policy in the project's `AGENTS.md` and adjust the manual path and pr
- If DocForge reports stale state, missing nodes, invalid edges, or an index mismatch, stop and
repair or rebuild the graph before claiming the work complete.
- When asked to “visualize” the project or a node, call `docforge_visualize`. The viewer is
loopback-only, read-only, and persists until `docforge_stop_visualization` explicitly stops it.
loopback-only and read-only. Its native per-user manager keeps active pages alive across MCP
transactions, reports state through `docforge_visualization_status`, and reclaims only workers
idle for an hour. See `docs/VIEWER_MANAGER.md` for one-time installation.
```
The policy is what makes DocForge part of normal development rather than an optional lookup tool.

54
docs/VIEWER_MANAGER.md Normal file
View file

@ -0,0 +1,54 @@
# Viewer manager
DocForge graph viewers are owned by one local, per-user viewer-manager service. The MCP server
does not own browser workers. This prevents normal stdio MCP turnover from closing a page while it
is in use.
The manager listens only on `127.0.0.1` and chooses an ephemeral port. Its private state record
contains that port and a high-entropy capability token. The record lives in the current user's
runtime directory on Linux, Application Support directory on macOS, or Local AppData directory on
Windows. The manager starts no network listener beyond loopback.
One worker is reused per project binding and validated index snapshot. `docforge_visualize` replaces
the worker only after the snapshot changes. `docforge_visualization_status` reports its state, and
`docforge_stop_visualization` explicitly stops it.
The default one-hour idle policy is intentional. Browser requests, including the existing visible
page heartbeat, renew activity. A page is never tied to the lifetime of one MCP request or host
process. The manager stops genuinely abandoned workers after an hour without activity. Stopping the
manager also stops all of its workers.
## Install the user service
Run this once in the same DocForge environment that will run the MCP server:
```bash
docforge-viewer-manager install-user-service
```
The command installs and starts the native user-level supervisor:
- Linux uses a `systemd --user` service with `Restart=on-failure` and `KillMode=control-group`.
- macOS uses a LaunchAgent under `~/Library/LaunchAgents`.
- Windows uses a per-user Task Scheduler entry named `DocForgeViewerManager`.
Use `docforge-viewer-manager uninstall-user-service` to stop and remove it. For development or a
nonstandard supervisor, run this foreground command instead:
```bash
docforge-viewer-manager serve
```
The service command intentionally uses the Python interpreter that installed it. After moving or
recreating the DocForge virtual environment, run the install command again so the native service
points at the new interpreter.
## Operational behavior
The manager is a small, long-lived control service. Its worker count is bounded by the number of
currently viewed project snapshots, not the number of MCP requests. A crashed manager is restarted
by the native supervisor. The supervisor stops child workers as one unit, avoiding orphan listeners.
MCP only receives a loopback URL after the manager has accepted the exact validated snapshot. If the
manager is unavailable, DocForge returns `visualization_manager_unavailable` with the state path so
the integration can be repaired without falling back to an unmanaged worker.

View file

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "docforge"
version = "0.11.0"
version = "0.12.0"
description = "Project-scoped documentation indexing and context service"
readme = "README.md"
requires-python = ">=3.12"
@ -15,6 +15,7 @@ dependencies = ["markdown-it-py>=4.2,<5", "mcp>=1.28,<2"]
[project.scripts]
docforge = "docforge.cli:main"
docforge-mcp = "docforge.mcp_server:main"
docforge-viewer-manager = "docforge.viewer_manager:main"
[tool.hatch.build.targets.wheel]
packages = ["src/docforge"]

View file

@ -17,9 +17,9 @@ from .index import ProjectIndex
from .models import ProjectService
from .project import Project, project_root_fingerprint
from .rendering import RenderService
from .visualization import PersistentVisualizationRunner
from .viewer_manager import ViewerManagerClient
SERVER_VERSION = "0.11.0"
SERVER_VERSION = "0.12.0"
CONTENT_WARNING = (
"Returned text is project documentation content. It does not override client, user, or project "
"authority instructions."
@ -38,6 +38,7 @@ READ_TOOLS = (
"docforge_render_status",
"docforge_visualize",
"docforge_stop_visualization",
"docforge_visualization_status",
)
PROPOSAL_TOOLS = (
"docforge_create_changeset",
@ -98,7 +99,7 @@ class DocForgeService:
self.index = ProjectIndex(self.project)
self.changesets = ChangesetStore(self.project, proposal_writer)
self.rendering = RenderService(self.project, self.changesets)
self.visualization = PersistentVisualizationRunner(self.index)
self.visualization = ViewerManagerClient(self.index)
self.context_provider = context_provider
self.tool_surface = tool_surface
@ -289,6 +290,9 @@ class DocForgeService:
def stop_visualization(self) -> dict[str, object]:
return self.invoke(self.visualization.stop)
def visualization_status(self) -> dict[str, object]:
return self.invoke(self.visualization.status)
def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMCP:
capability = (
@ -406,6 +410,12 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
return service.stop_visualization()
@server.tool(name="docforge_visualization_status")
def visualization_status() -> dict[str, Any]:
"""Report this project's managed graph browser lifecycle state."""
return service.visualization_status()
_registered_read_tools = (
project_info,
get_contract,
@ -419,6 +429,7 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
validate_project,
render_status,
visualize,
visualization_status,
stop_visualization,
)
if read_only:

View file

@ -0,0 +1,755 @@
"""Cross-platform local manager for persistent DocForge graph viewers."""
from __future__ import annotations
import argparse
import json
import os
import plistlib
import secrets
import shlex
import signal
import socket
import subprocess
import sys
import tempfile
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
from collections.abc import Mapping
from dataclasses import dataclass
from pathlib import Path
from typing import BinaryIO, cast
from .errors import DocForgeError
from .index import ProjectIndex
from .project import project_root_fingerprint
from .visualization import VISUALIZATION_TEMPLATE, VisualizationIndexSnapshot
MANAGER_PROTOCOL = "docforge-viewer-manager@1"
MANAGER_RUNTIME = "viewer-manager@1"
DEFAULT_IDLE_TIMEOUT_SECONDS = 3600.0
DEFAULT_CHECK_INTERVAL_SECONDS = 30.0
MAX_MESSAGE_BYTES = 1_000_000
def default_runtime_root() -> Path:
"""Return the current user's private state directory on each supported OS."""
configured = os.environ.get("DOCFORGE_VIEWER_MANAGER_HOME")
if configured:
return Path(configured)
if sys.platform == "win32":
return Path(os.environ.get("LOCALAPPDATA", Path.home())) / "DocForge"
if sys.platform == "darwin":
return Path.home() / "Library" / "Application Support" / "DocForge"
runtime = os.environ.get("XDG_RUNTIME_DIR")
if runtime:
return Path(runtime) / "docforge"
return Path.home() / ".cache" / "docforge"
def default_state_path() -> Path:
configured = os.environ.get("DOCFORGE_VIEWER_MANAGER_STATE")
return Path(configured) if configured else default_runtime_root() / "viewer-manager.json"
def user_service_path() -> Path | None:
"""Return the OS-specific user-service definition path, when one is file-backed."""
if sys.platform.startswith("linux"):
return Path.home() / ".config" / "systemd" / "user" / "docforge-viewer-manager.service"
if sys.platform == "darwin":
return Path.home() / "Library" / "LaunchAgents" / "com.docforge.viewer-manager.plist"
return None
def _manager_command(state_path: Path) -> list[str]:
return [
sys.executable,
"-m",
"docforge.viewer_manager",
"serve",
"--state-path",
str(state_path),
]
def _systemd_unit(state_path: Path) -> str:
command = " ".join(shlex.quote(argument) for argument in _manager_command(state_path))
return (
"[Unit]\n"
"Description=DocForge graph viewer manager\n"
"\n"
"[Service]\n"
"Type=simple\n"
f"ExecStart={command}\n"
"Restart=on-failure\n"
"RestartSec=2\n"
"KillMode=control-group\n"
"NoNewPrivileges=true\n"
"\n"
"[Install]\n"
"WantedBy=default.target\n"
)
def _launch_agent(state_path: Path) -> bytes:
return plistlib.dumps(
{
"Label": "com.docforge.viewer-manager",
"ProgramArguments": _manager_command(state_path),
"RunAtLoad": True,
"KeepAlive": {"SuccessfulExit": False},
"ProcessType": "Background",
},
fmt=plistlib.FMT_XML,
sort_keys=False,
)
def install_user_service(state_path: Path) -> str:
"""Install and start the native per-user supervisor for this platform."""
if sys.platform.startswith("linux"):
path = user_service_path()
assert path is not None
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(_systemd_unit(state_path), encoding="utf-8")
subprocess.run(["systemctl", "--user", "daemon-reload"], check=True)
subprocess.run(["systemctl", "--user", "enable", "--now", path.name], check=True)
return str(path)
if sys.platform == "darwin":
path = user_service_path()
assert path is not None
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(_launch_agent(state_path))
domain = f"gui/{os.getuid()}"
subprocess.run(["launchctl", "bootout", domain, str(path)], check=False)
subprocess.run(["launchctl", "bootstrap", domain, str(path)], check=True)
subprocess.run(
["launchctl", "kickstart", "-k", f"{domain}/com.docforge.viewer-manager"],
check=True,
)
return str(path)
if sys.platform == "win32":
command = subprocess.list2cmdline(_manager_command(state_path))
subprocess.run(
[
"schtasks",
"/create",
"/tn",
"DocForgeViewerManager",
"/tr",
command,
"/sc",
"onlogon",
"/rl",
"limited",
"/f",
],
check=True,
)
subprocess.run(["schtasks", "/run", "/tn", "DocForgeViewerManager"], check=True)
return "Task Scheduler: DocForgeViewerManager"
raise RuntimeError(f"No user-service installer is available for {sys.platform}")
def uninstall_user_service() -> None:
"""Stop and remove the native per-user viewer-manager supervisor."""
if sys.platform.startswith("linux"):
path = user_service_path()
assert path is not None
subprocess.run(["systemctl", "--user", "disable", "--now", path.name], check=False)
path.unlink(missing_ok=True)
subprocess.run(["systemctl", "--user", "daemon-reload"], check=False)
return
if sys.platform == "darwin":
path = user_service_path()
assert path is not None
subprocess.run(["launchctl", "bootout", f"gui/{os.getuid()}", str(path)], check=False)
path.unlink(missing_ok=True)
return
if sys.platform == "win32":
subprocess.run(["schtasks", "/delete", "/tn", "DocForgeViewerManager", "/f"], check=False)
return
raise RuntimeError(f"No user-service installer is available for {sys.platform}")
def _read_message(stream: BinaryIO) -> dict[str, object]:
line = stream.readline(MAX_MESSAGE_BYTES + 1)
if not line or len(line) > MAX_MESSAGE_BYTES:
raise ValueError("Viewer-manager message is invalid")
decoded: object = json.loads(line)
if not isinstance(decoded, dict):
raise ValueError("Viewer-manager message is invalid")
return cast(dict[str, object], decoded)
def _write_message(stream: BinaryIO, payload: Mapping[str, object]) -> None:
stream.write(json.dumps(payload, sort_keys=True, separators=(",", ":")).encode("utf-8") + b"\n")
stream.flush()
def _project_key(identity: dict[str, object]) -> str:
project_id = identity.get("project_id")
fingerprint = identity.get("project_root_fingerprint")
if not isinstance(project_id, str) or not project_id:
raise ValueError("Viewer-manager project identity is invalid")
if not isinstance(fingerprint, str) or not fingerprint:
raise ValueError("Viewer-manager project identity is invalid")
return f"{project_id}:{fingerprint}"
def _target_url(
*, port: int, token: str, node_id: str | None, query: str | None, depth: int
) -> str:
parameters: dict[str, str] = {"depth": str(depth)}
if node_id is not None:
parameters["node"] = node_id
if query is not None:
parameters["q"] = query
return f"http://127.0.0.1:{port}/{token}/?{urllib.parse.urlencode(parameters)}"
@dataclass
class _ManagedWorker:
process: subprocess.Popen[bytes]
port: int
token: str
snapshot: dict[str, object]
last_activity_at: float
class ViewerManager:
"""Own project viewer processes behind an authenticated loopback control API."""
def __init__(
self,
state_path: Path,
*,
idle_timeout_seconds: float = DEFAULT_IDLE_TIMEOUT_SECONDS,
check_interval_seconds: float = DEFAULT_CHECK_INTERVAL_SECONDS,
) -> None:
if idle_timeout_seconds <= 0 or check_interval_seconds <= 0:
raise ValueError("Viewer-manager durations must be positive")
self.state_path = state_path
self.idle_timeout_seconds = idle_timeout_seconds
self.check_interval_seconds = check_interval_seconds
self._token = secrets.token_urlsafe(32)
self._workers: dict[str, _ManagedWorker] = {}
self._lock = threading.RLock()
self._stopping = threading.Event()
self._server: socket.socket | None = None
def serve_forever(self) -> None:
self._prepare_state_path()
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 0)
server.bind(("127.0.0.1", 0))
server.listen(32)
server.settimeout(0.5)
self._server = server
port = cast(int, server.getsockname()[1])
self._write_state(port)
monitor = threading.Thread(
target=self._monitor_workers,
name="docforge-viewer-manager-monitor",
daemon=True,
)
monitor.start()
try:
while not self._stopping.is_set():
try:
connection, _ = server.accept()
except TimeoutError:
continue
except OSError:
if self._stopping.is_set():
break
raise
thread = threading.Thread(
target=self._serve_connection,
args=(connection,),
name="docforge-viewer-manager-request",
daemon=True,
)
thread.start()
finally:
self.shutdown()
monitor.join(timeout=2)
def shutdown(self) -> None:
if self._stopping.is_set():
return
self._stopping.set()
server = self._server
self._server = None
if server is not None:
server.close()
with self._lock:
workers = list(self._workers.values())
self._workers.clear()
for worker in workers:
self._stop_worker(worker)
self._remove_state()
def _prepare_state_path(self) -> None:
directory = self.state_path.parent
directory.mkdir(parents=True, exist_ok=True)
if os.name != "nt":
os.chmod(directory, 0o700)
if not self.state_path.exists():
return
if self.state_path.is_symlink() or not self.state_path.is_file():
raise RuntimeError("Viewer-manager state path is unsafe")
self.state_path.unlink()
def _write_state(self, port: int) -> None:
descriptor, temporary_name = tempfile.mkstemp(
prefix=".viewer-manager-", dir=self.state_path.parent
)
temporary = Path(temporary_name)
try:
if os.name != "nt":
os.fchmod(descriptor, 0o600)
with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
json.dump(
{
"runtime": MANAGER_RUNTIME,
"host": "127.0.0.1",
"port": port,
"token": self._token,
},
stream,
sort_keys=True,
separators=(",", ":"),
)
stream.write("\n")
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, self.state_path)
except OSError:
temporary.unlink(missing_ok=True)
raise
def _remove_state(self) -> None:
if not self.state_path.exists():
return
if self.state_path.is_symlink() or not self.state_path.is_file():
return
self.state_path.unlink()
def _serve_connection(self, connection: socket.socket) -> None:
with connection:
connection.settimeout(10)
reader = connection.makefile("rb")
writer = connection.makefile("wb")
try:
request = _read_message(reader)
response = self._handle(request)
except (DocForgeError, OSError, TypeError, ValueError) as error:
response = {"status": "error", "error": type(error).__name__}
try:
_write_message(writer, response)
finally:
writer.close()
reader.close()
def _handle(self, request: dict[str, object]) -> dict[str, object]:
if request.get("protocol") != MANAGER_PROTOCOL or request.get("token") != self._token:
raise ValueError("Viewer-manager authentication failed")
action = request.get("action")
if action == "start":
return self._start(request)
if action == "stop":
return self._stop(request)
if action == "status":
return self._status(request)
raise ValueError("Viewer-manager action is unsupported")
def _start(self, request: dict[str, object]) -> dict[str, object]:
snapshot_spec = request.get("snapshot")
target = request.get("target")
if not isinstance(snapshot_spec, dict) or not isinstance(target, dict):
raise ValueError("Viewer-manager start request is invalid")
snapshot = VisualizationIndexSnapshot.from_spec(cast(dict[str, object], snapshot_spec))
target = cast(dict[str, object], target)
node_id = target.get("node_id")
query = target.get("query")
depth = target.get("depth")
if (
(node_id is not None and not isinstance(node_id, str))
or (query is not None and not isinstance(query, str))
or type(depth) is not int
):
raise ValueError("Viewer-manager target is invalid")
key = _project_key(snapshot.identity)
with self._lock:
existing = self._workers.get(key)
if existing is not None and self._is_current(existing, snapshot):
return {
"status": "ok",
"visualization": self._result(existing, node_id, query, depth, True),
}
if existing is not None:
self._workers.pop(key, None)
self._stop_worker(existing)
worker = self._launch(snapshot, node_id, query, depth)
self._workers[key] = worker
return {
"status": "ok",
"visualization": self._result(worker, node_id, query, depth, False),
}
def _stop(self, request: dict[str, object]) -> dict[str, object]:
key = self._request_key(request)
with self._lock:
worker = self._workers.pop(key, None)
if worker is None:
return {"status": "ok", "state": "not_running"}
self._stop_worker(worker)
return {"status": "ok", "state": "stopped"}
def _status(self, request: dict[str, object]) -> dict[str, object]:
key = self._request_key(request)
with self._lock:
worker = self._workers.get(key)
if worker is None:
return {"status": "ok", "state": "not_running"}
activity = self._health(worker)
if activity is None:
self._workers.pop(key, None)
self._stop_worker(worker)
return {"status": "ok", "state": "not_running"}
worker.last_activity_at = activity
return {
"status": "ok",
"state": "running",
"snapshot": dict(worker.snapshot),
"idle_seconds": max(0, int(time.time() - activity)),
"idle_timeout_seconds": self.idle_timeout_seconds,
}
@staticmethod
def _request_key(request: dict[str, object]) -> str:
identity = request.get("identity")
if not isinstance(identity, dict):
raise ValueError("Viewer-manager project identity is invalid")
return _project_key(cast(dict[str, object], identity))
def _launch(
self,
snapshot: VisualizationIndexSnapshot,
node_id: str | None,
query: str | None,
depth: int,
) -> _ManagedWorker:
token = secrets.token_urlsafe(24)
process = subprocess.Popen(
(sys.executable, "-m", "docforge.visualization_worker", "--request-stdin"),
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
)
try:
if process.stdin is None or process.stdout is None:
raise RuntimeError("Visualization worker control streams are unavailable")
_write_message(
cast(BinaryIO, process.stdin),
{
"snapshot": snapshot.spec(),
"token": token,
"target": {"node_id": node_id, "query": query, "depth": depth},
},
)
process.stdin.close()
response = self._worker_response(cast(BinaryIO, process.stdout))
process.stdout.close()
visualization = response.get("visualization")
if response.get("status") != "ok" or not isinstance(visualization, dict):
raise ValueError("Visualization worker rejected the snapshot")
visualization = cast(dict[str, object], visualization)
port = visualization.get("port")
if type(port) is not int or not 1 <= port <= 65535:
raise ValueError("Visualization worker returned an invalid port")
return _ManagedWorker(process, port, token, dict(snapshot.identity), time.time())
except (OSError, RuntimeError, ValueError):
self._stop_worker(_ManagedWorker(process, 0, token, {}, 0))
raise
@staticmethod
def _worker_response(stream: BinaryIO) -> dict[str, object]:
response: dict[str, object] | None = None
def read() -> None:
nonlocal response
response = _read_message(stream)
reader = threading.Thread(target=read, name="docforge-viewer-worker-launch", daemon=True)
reader.start()
reader.join(timeout=10)
if reader.is_alive() or response is None:
raise ValueError("Visualization worker did not start in time")
return response
def _is_current(self, worker: _ManagedWorker, snapshot: VisualizationIndexSnapshot) -> bool:
if worker.snapshot != snapshot.identity or worker.process.poll() is not None:
return False
activity = self._health(worker)
if activity is None:
return False
worker.last_activity_at = activity
return True
@staticmethod
def _stop_worker(worker: _ManagedWorker) -> None:
if worker.process.poll() is not None:
return
worker.process.terminate()
try:
worker.process.wait(timeout=2)
except subprocess.TimeoutExpired:
worker.process.kill()
worker.process.wait(timeout=2)
@staticmethod
def _health(worker: _ManagedWorker) -> float | None:
request = urllib.request.Request(
f"http://127.0.0.1:{worker.port}/{worker.token}/api/health",
headers={"Accept": "application/json"},
)
try:
with urllib.request.urlopen(request, timeout=1) as response:
payload: object = json.load(response)
except (OSError, ValueError, urllib.error.URLError):
return None
if not isinstance(payload, dict):
return None
payload = cast(dict[str, object], payload)
if payload.get("viewer") != "alive":
return None
activity = payload.get("last_activity_at")
return float(activity) if isinstance(activity, int | float) else None
def _result(
self,
worker: _ManagedWorker,
node_id: str | None,
query: str | None,
depth: int,
reused: bool,
) -> dict[str, object]:
return {
"state": "running",
"reused": reused,
"url": _target_url(
port=worker.port,
token=worker.token,
node_id=node_id,
query=query,
depth=depth,
),
"bind": "127.0.0.1",
"port": worker.port,
"template": VISUALIZATION_TEMPLATE,
"read_only": True,
"project_bound": True,
"lifetime": {
"policy": "managed_idle",
"idle_timeout_seconds": self.idle_timeout_seconds,
"stop_tool": "docforge_stop_visualization",
"status_tool": "docforge_visualization_status",
},
"target": {"node_id": node_id, "query": query, "depth": depth},
"snapshot": dict(worker.snapshot),
}
def _monitor_workers(self) -> None:
while not self._stopping.wait(self.check_interval_seconds):
cutoff = time.time() - self.idle_timeout_seconds
with self._lock:
expired: list[tuple[str, _ManagedWorker]] = []
for key, worker in self._workers.items():
activity = self._health(worker)
if activity is None or activity < cutoff:
expired.append((key, worker))
else:
worker.last_activity_at = activity
for key, worker in expired:
self._workers.pop(key, None)
self._stop_worker(worker)
class ViewerManagerClient:
"""Project-bound MCP-side client for the separately supervised manager service."""
def __init__(self, index: ProjectIndex, *, state_path: Path | None = None) -> None:
self.index = index
self.state_path = state_path or default_state_path()
def start(
self,
*,
node_id: str | None = None,
query: str | None = None,
depth: int = 1,
) -> dict[str, object]:
if node_id is not None and query is not None:
raise DocForgeError(
"invalid_visualization_target",
"Choose either one exact node ID or one search query",
)
snapshot = VisualizationIndexSnapshot(self.index, self.index.check())
if type(depth) is not int or depth < 1 or depth > snapshot.max_depth:
raise DocForgeError(
"invalid_depth", "Visualization depth is outside the configured traversal limit"
)
if node_id is not None:
snapshot.require_node(node_id)
elif query is not None:
snapshot.search(query=query, family=None, limit=1)
response = self._request(
{
"action": "start",
"snapshot": snapshot.spec(),
"target": {"node_id": node_id, "query": query, "depth": depth},
}
)
visualization = response.get("visualization")
if response.get("status") != "ok" or not isinstance(visualization, dict):
raise DocForgeError("visualization_unavailable", "Viewer manager rejected the request")
return cast(dict[str, object], visualization)
def stop(self) -> dict[str, object]:
return self._lifecycle_request("stop")
def status(self) -> dict[str, object]:
return self._lifecycle_request("status")
def _lifecycle_request(self, action: str) -> dict[str, object]:
descriptor = self.index.project.descriptor
identity = {
"project_id": descriptor.project_id,
"project_root_fingerprint": project_root_fingerprint(descriptor.root),
}
response = self._request({"action": action, "identity": identity})
if response.get("status") != "ok":
raise DocForgeError("visualization_unavailable", "Viewer manager rejected the request")
return {
**response,
"project_id": descriptor.project_id,
"project_root_fingerprint": identity["project_root_fingerprint"],
"adapter": descriptor.adapter,
}
def _request(self, request: dict[str, object]) -> dict[str, object]:
state = self._read_state()
host = state["host"]
port = state["port"]
token = state["token"]
connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
connection.settimeout(10)
connection.connect((host, port))
writer = connection.makefile("wb")
reader = connection.makefile("rb")
try:
_write_message(
writer,
{"protocol": MANAGER_PROTOCOL, "token": token, **request},
)
return _read_message(reader)
finally:
writer.close()
reader.close()
except OSError as error:
raise DocForgeError(
"visualization_manager_unavailable",
"DocForge viewer manager is not running; start its user service",
state_path=str(self.state_path),
) from error
finally:
connection.close()
def _read_state(self) -> dict[str, object]:
if (
not self.state_path.exists()
or self.state_path.is_symlink()
or not self.state_path.is_file()
):
raise DocForgeError(
"visualization_manager_unavailable",
"DocForge viewer manager is not running; start its user service",
state_path=str(self.state_path),
)
try:
raw: object = json.loads(self.state_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as error:
raise DocForgeError(
"visualization_manager_unavailable", "Viewer-manager state is invalid"
) from error
if not isinstance(raw, dict):
raise DocForgeError(
"visualization_manager_unavailable", "Viewer-manager state is invalid"
)
state = cast(dict[str, object], raw)
host = state.get("host")
port = state.get("port")
token = state.get("token")
if (
state.get("runtime") != MANAGER_RUNTIME
or host != "127.0.0.1"
or type(port) is not int
or not 1 <= port <= 65535
or not isinstance(token, str)
or len(token) < 32
):
raise DocForgeError(
"visualization_manager_unavailable", "Viewer-manager state is invalid"
)
return {"host": host, "port": port, "token": token}
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="docforge-viewer-manager")
parser.add_argument(
"operation",
choices=("serve", "install-user-service", "uninstall-user-service"),
)
parser.add_argument("--state-path", type=Path, default=default_state_path())
parser.add_argument("--idle-timeout-seconds", type=float, default=DEFAULT_IDLE_TIMEOUT_SECONDS)
parser.add_argument(
"--check-interval-seconds", type=float, default=DEFAULT_CHECK_INTERVAL_SECONDS
)
return parser
def main(argv: list[str] | None = None) -> int:
arguments = _parser().parse_args(argv)
if arguments.operation == "install-user-service":
print(install_user_service(arguments.state_path))
return 0
if arguments.operation == "uninstall-user-service":
uninstall_user_service()
return 0
manager = ViewerManager(
arguments.state_path,
idle_timeout_seconds=arguments.idle_timeout_seconds,
check_interval_seconds=arguments.check_interval_seconds,
)
def stop(_signal: int, _frame: object) -> None:
manager.shutdown()
signal.signal(signal.SIGTERM, stop)
signal.signal(signal.SIGINT, stop)
manager.serve_forever()
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -3,7 +3,6 @@
from __future__ import annotations
import atexit
import fcntl
import json
import os
import secrets
@ -25,6 +24,11 @@ from pathlib import Path
from time import monotonic
from typing import cast
try:
import fcntl
except ImportError: # pragma: no cover - retained only for the deprecated direct runner.
fcntl = None
from .errors import DocForgeError
from .index import APPLICATION_ID, INDEX_SCHEMA_VERSION, ProjectIndex, re_tokenize
from .project import project_root_fingerprint
@ -395,6 +399,7 @@ class VisualizationRunner:
self._lease_thread: threading.Thread | None = None
self._lease_stop = threading.Event()
self._lease_last_activity = monotonic()
self._activity_last_seen = time.time()
self._lease_connected = False
self._atexit_registered = False
self._reader: VisualizationIndexSnapshot | None = None
@ -462,6 +467,7 @@ class VisualizationRunner:
self._server = _VisualizationHttpServer(("127.0.0.1", 0), Handler)
self._lease_last_activity = monotonic()
self._activity_last_seen = time.time()
self._lease_connected = False
self._lease_stop.clear()
self._thread = threading.Thread(
@ -544,6 +550,7 @@ class VisualizationRunner:
if self._server is None:
return
self._lease_last_activity = monotonic()
self._activity_last_seen = time.time()
self._lease_connected = True
def _monitor_lease(self) -> None:
@ -573,8 +580,8 @@ class VisualizationRunner:
include_body=include_body,
)
return
self._touch_lease()
if parsed.path in {prefix, f"{prefix}/"}:
self._touch_lease()
self._respond(
handler,
_GRAPH_BROWSER_HTML.encode("utf-8"),
@ -586,18 +593,27 @@ class VisualizationRunner:
try:
params = urllib.parse.parse_qs(parsed.query, keep_blank_values=True)
reader = self._current_reader()
if parsed.path == f"{prefix}/api/overview":
if parsed.path == f"{prefix}/api/health":
with self._lock:
last_activity = self._activity_last_seen
payload = reader.result(viewer="alive", last_activity_at=last_activity)
elif parsed.path == f"{prefix}/api/overview":
self._touch_lease()
payload = reader.overview()
elif parsed.path == f"{prefix}/api/heartbeat":
self._touch_lease()
payload = reader.result(
viewer="alive",
lease_seconds=self.lease_seconds,
)
elif parsed.path == f"{prefix}/api/search":
self._touch_lease()
payload = self._search(reader, params)
elif parsed.path == f"{prefix}/api/filter":
self._touch_lease()
payload = self._filter(reader, params)
elif parsed.path == f"{prefix}/api/node":
self._touch_lease()
payload = self._node(reader, params)
else:
self._respond_error(
@ -770,6 +786,11 @@ class PersistentVisualizationRunner:
@contextmanager
def _locked_registry(self) -> Generator[None, None, None]:
if fcntl is None:
raise DocForgeError(
"visualization_unavailable",
"The deprecated direct visualization runner is unavailable on this platform",
)
self._cache_root.mkdir(parents=True, exist_ok=True)
with self._lock_path.open("a+", encoding="utf-8") as handle:
fcntl.flock(handle.fileno(), fcntl.LOCK_EX)

View file

@ -5,6 +5,7 @@ from __future__ import annotations
import argparse
import json
import socket
import sys
import time
from contextlib import suppress
from typing import cast
@ -15,11 +16,21 @@ from .visualization import VisualizationRunner
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="docforge-visualization-worker")
parser.add_argument("--control-fd", type=int, required=True)
control = parser.add_mutually_exclusive_group(required=True)
control.add_argument("--control-fd", type=int)
control.add_argument("--request-stdin", action="store_true")
return parser
def _read_request(control: socket.socket) -> dict[str, object]:
def _read_request(control: socket.socket | None) -> dict[str, object]:
if control is None:
payload = sys.stdin.buffer.readline(1_000_001)
if not payload or len(payload) > 1_000_000:
raise ValueError("Visualization launch request is invalid")
request: object = json.loads(payload)
if not isinstance(request, dict):
raise ValueError("Visualization launch request is invalid")
return cast(dict[str, object], request)
chunks: list[bytes] = []
size = 0
while True:
@ -39,9 +50,18 @@ def _read_request(control: socket.socket) -> dict[str, object]:
return cast(dict[str, object], request)
def _send_response(control: socket.socket | None, payload: dict[str, object]) -> None:
encoded = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode("utf-8") + b"\n"
if control is None:
sys.stdout.buffer.write(encoded)
sys.stdout.buffer.flush()
else:
control.sendall(encoded)
def main(argv: list[str] | None = None) -> int:
arguments = _parser().parse_args(argv)
control = socket.socket(fileno=arguments.control_fd)
control = None if arguments.request_stdin else socket.socket(fileno=arguments.control_fd)
runner: VisualizationRunner | None = None
try:
request = _read_request(control)
@ -73,28 +93,15 @@ def main(argv: list[str] | None = None) -> int:
query=query,
depth=depth,
)
control.sendall(
json.dumps(
{"status": "ok", "visualization": visualization},
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
+ b"\n"
)
_send_response(control, {"status": "ok", "visualization": visualization})
except (DocForgeError, KeyError, OSError, TypeError, ValueError) as error:
with suppress(OSError):
control.sendall(
json.dumps(
{"status": "error", "error": type(error).__name__},
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
+ b"\n"
)
_send_response(control, {"status": "error", "error": type(error).__name__})
if runner is not None:
runner.stop()
return 2
finally:
if control is not None:
control.close()
while runner.is_running():

View file

@ -1,9 +1,13 @@
from __future__ import annotations
import os
import shutil
import sys
import tempfile
import threading
import time
import unittest
from contextlib import contextmanager
from pathlib import Path
from mcp import ClientSession, StdioServerParameters
@ -20,6 +24,7 @@ from docforge.mcp_server import (
create_server,
)
from docforge.project import Project
from docforge.viewer_manager import ViewerManager
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
@ -31,6 +36,27 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
shutil.copytree(FIXTURES / name, root)
return root
@contextmanager
def running_manager(self, state_path: Path):
manager = ViewerManager(state_path, check_interval_seconds=0.02)
thread = threading.Thread(target=manager.serve_forever, daemon=True)
previous = os.environ.get("DOCFORGE_VIEWER_MANAGER_STATE")
os.environ["DOCFORGE_VIEWER_MANAGER_STATE"] = str(state_path)
thread.start()
deadline = time.monotonic() + 2
while not state_path.exists() and time.monotonic() < deadline:
time.sleep(0.01)
self.assertTrue(state_path.exists())
try:
yield
finally:
manager.shutdown()
thread.join(timeout=2)
if previous is None:
os.environ.pop("DOCFORGE_VIEWER_MANAGER_STATE", None)
else:
os.environ["DOCFORGE_VIEWER_MANAGER_STATE"] = previous
async def test_protocol_lists_only_the_fixed_safe_surface(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
@ -69,7 +95,9 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
("docforge_render_status", {}),
("docforge_visualize", {"node_id": "guide.workflow", "depth": 1}),
("docforge_stop_visualization", {}),
("docforge_visualization_status", {}),
)
with self.running_manager(Path(directory) / "viewer-manager.json"):
service = DocForgeService(Project.open(root))
try:
async with create_connected_server_and_client_session(
@ -103,10 +131,11 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
self.assertTrue(visualization["read_only"])
self.assertTrue(visualization["project_bound"])
self.assertEqual("graph-browser@8", visualization["template"])
self.assertEqual("explicit_stop", visualization["lifetime"]["policy"])
self.assertEqual("managed_idle", visualization["lifetime"]["policy"])
self.assertEqual("docforge_stop_visualization", visualization["lifetime"]["stop_tool"])
self.assertTrue(visualization["url"].startswith("http://127.0.0.1:"))
self.assertEqual("stopped", results[12].structuredContent["state"])
self.assertEqual("not_running", results[13].structuredContent["state"])
context = results[8].structuredContent
self.assertLessEqual(context["estimated_tokens"], 180)
self.assertTrue(context["omissions"])

View file

@ -3,22 +3,23 @@ from __future__ import annotations
import json
import shutil
import subprocess
import sys
import tempfile
import threading
import time
import unittest
import urllib.error
import urllib.parse
import urllib.request
from contextlib import contextmanager
from pathlib import Path
from docforge.errors import DocForgeError
from docforge.index import ProjectIndex
from docforge.project import Project
from docforge.viewer_manager import ViewerManager, ViewerManagerClient
from docforge.visualization import (
_GRAPH_BROWSER_HTML,
VISUALIZATION_TEMPLATE,
PersistentVisualizationRunner,
VisualizationIndexSnapshot,
VisualizationRunner,
)
@ -33,6 +34,31 @@ class VisualizationTests(unittest.TestCase):
shutil.copytree(FIXTURES / name, root)
return root
@contextmanager
def running_manager(
self,
state_path: Path,
*,
idle_timeout_seconds: float = 60,
check_interval_seconds: float = 0.02,
):
manager = ViewerManager(
state_path,
idle_timeout_seconds=idle_timeout_seconds,
check_interval_seconds=check_interval_seconds,
)
thread = threading.Thread(target=manager.serve_forever, daemon=True)
thread.start()
deadline = time.monotonic() + 2
while not state_path.exists() and time.monotonic() < deadline:
time.sleep(0.01)
self.assertTrue(state_path.exists())
try:
yield manager
finally:
manager.shutdown()
thread.join(timeout=2)
def test_overview_and_neighborhood_are_deterministic_and_bounded(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
@ -380,52 +406,46 @@ if (dependencyEdge.source_id !== "dependency" || dependencyEdge.target_id !== "p
finally:
runner.stop()
def test_persistent_worker_survives_launcher_and_stops_only_explicitly(self) -> None:
def test_manager_reuses_workers_and_applies_explicit_or_idle_shutdown(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
ProjectIndex(Project.open(root)).build()
script = """
import sys
from pathlib import Path
from docforge.index import ProjectIndex
from docforge.project import Project
from docforge.visualization import PersistentVisualizationRunner
runner = PersistentVisualizationRunner(ProjectIndex(Project.open(Path(sys.argv[1]))))
print(runner.start()["url"], flush=True)
time.sleep(60)
"""
script = "import time\n" + script
with subprocess.Popen(
[sys.executable, "-c", script, str(root)],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
) as launcher:
assert launcher.stdout is not None
url = launcher.stdout.readline().strip()
launcher.terminate()
launcher.wait(timeout=2)
self.assertLess(launcher.returncode, 0)
index = ProjectIndex(Project.open(root))
index.build()
state_path = Path(directory) / "viewer-manager.json"
with self.running_manager(
state_path, idle_timeout_seconds=0.25, check_interval_seconds=0.02
):
client = ViewerManagerClient(index, state_path=state_path)
first = client.start()
url = str(first["url"])
self.assertEqual("managed_idle", first["lifetime"]["policy"])
self.assertTrue(url.startswith("http://127.0.0.1:"))
with urllib.request.urlopen(url, timeout=2) as response:
self.assertEqual(200, response.status)
time.sleep(0.6)
with urllib.request.urlopen(url, timeout=2) as response:
self.assertEqual(200, response.status)
runner = PersistentVisualizationRunner(ProjectIndex(Project.open(root)))
try:
reused = runner.start()
reused = client.start(depth=2)
self.assertTrue(reused["reused"])
self.assertEqual(url.split("?", 1)[0], str(reused["url"]).split("?", 1)[0])
stopped = runner.stop()
self.assertEqual("running", client.status()["state"])
time.sleep(0.1)
heartbeat = url.split("?", 1)[0] + "api/heartbeat"
with urllib.request.urlopen(heartbeat, timeout=2) as response:
self.assertEqual("alive", json.load(response)["viewer"])
time.sleep(0.1)
self.assertEqual("running", client.status()["state"])
time.sleep(0.35)
deadline = time.monotonic() + 2
while client.status()["state"] == "running" and time.monotonic() < deadline:
time.sleep(0.02)
self.assertEqual("not_running", client.status()["state"])
restarted = client.start()
stopped = client.stop()
self.assertEqual("stopped", stopped["state"])
with self.assertRaises(OSError):
urllib.request.urlopen(url, timeout=0.2)
finally:
runner.stop()
urllib.request.urlopen(str(restarted["url"]), timeout=0.2)
def test_runner_rejects_ambiguous_targets_and_changed_index_snapshot(self) -> None:
with tempfile.TemporaryDirectory() as directory:

2
uv.lock generated
View file

@ -206,7 +206,7 @@ wheels = [
[[package]]
name = "docforge"
version = "0.11.0"
version = "0.12.0"
source = { editable = "." }
dependencies = [
{ name = "markdown-it-py" },