Consolidate DocForge setup and command reference

2026-07-25 01:16:05 -04:00 · 2026-07-25 01:16:05 -04:00 · 3c15e26283
commit 3c15e26283
parent e405326334
4 changed files with 337 additions and 445 deletions
--- a/README.md
+++ b/README.md
@ -1,108 +1,356 @@
 # DocForge

-DocForge is a project-scoped documentation service for people and AI agents. It loads canonical
-Markdown and TOML from one repository, validates stable nodes and relationships, builds a disposable
-search index, and returns bounded context with source provenance.
+DocForge is a project-scoped documentation service for people and AI agents. It reads canonical
+Markdown and TOML from one repository, validates stable nodes and relationships, builds a
+disposable graph/search index, and returns bounded context with source provenance.

-DocForge does not own project facts. It does not select project work, apply proposals, run project
-commands, or perform Git and deployment operations. It may write only configured derived output and
-isolated previews through the explicit render boundary.
+The canonical manual remains the project’s source of truth. DocForge does not apply canonical
+changes, select project work, run project commands, commit, push, deploy, or publish. It may write
+only configured derived output and isolated proposal previews through explicit boundaries.

-## Current state
+This README is the complete setup and command reference. `docs/NEW_PROJECT_QUICKSTART.md` is kept
+only as a compatibility link for existing bookmarks.

-DFG-0 through DFG-14 are complete. Worldforge uses separate read-only sessions and an optional
-AssetForge-only proposal process. OpenClaw can propose updates to existing AssetForge chapter prose
-through isolated, validated changesets and escaped previews. Canonical integration remains a
-developer review step through Worldforge's established builder. DFG-9 found no measured need for an
-application command, so manual canonical integration is the permanent DocForge 0.x policy.
-Canonical application remains external and closed to the library, CLI, and MCP server. See
-[`docs/APPLICATION_DECISION.md`](docs/APPLICATION_DECISION.md).
+## What DocForge provides

-DFG-10 adds one generic `docforge_visualize` MCP tool. It validates the configured project and
-derived index, then starts a token-protected read-only graph browser on `127.0.0.1`. The browser
-supports project counts, family filtering, lexical search, exact node inspection, and bounded
-incoming-and-outgoing neighborhoods. It accepts no project path, database path, SQL, external bind
-address, or write operation.
+- A project-bound CLI for validation, indexing, graph queries, context, and declared renders.
+- A stdio MCP server with a fixed, project-scoped read surface.
+- Optional isolated documentation proposal changesets. Canonical application stays in the owning
+  project’s normal editing and review workflow.
+- A loopback-only graph browser with a native per-user viewer manager.
+- A generic manual adapter and a contract for project-owned source-code adapters.

-DFG-11 upgrades the fixed browser template with pointer-centered mouse-wheel zoom, left-button drag
-pan, zoom buttons, a reset control, and a live zoom percentage. These controls operate only on the
-client-side SVG viewport and do not broaden the read-only HTTP or project authority boundary.
+The generic adapter reads only declared Markdown and TOML nodes. It does not infer application
+modules, functions, calls, routes, tables, tests, or ownership. Projects that need those facts
+provide a deterministic, project-owned source adapter.

-DFG-12 makes graph-node activation open a modal inspector without replacing the current
-neighborhood. The dialog exposes the node's complete validated content and offers a separate
-Explore neighborhood action when the user wants to recenter the graph.
+## Install and verify

-DFG-13 makes pointer activation reliable by delaying SVG pointer capture until an actual drag
-crosses the movement threshold. It also ensures the empty-canvas instruction disappears whenever a
-neighborhood is rendered.
-
-DFG-14 makes the viewer useful as a durable project-manual navigator. Resizable side panels and a
-draggable, resizable inspector support dense material. Neighborhoods are grouped generically by
-topology into the focus node, outgoing paths, and incoming or lateral context, with distinct
-palettes and progressive hop-distance shading.
-
-DFG-17 makes relationship meaning visible without changing project graph facts. Nodes displays
-relation-specific colors, line patterns, directional symbols, and an exact visible key. Flow
-constructs a bounded upstream lineage with semantic direction for execution, data, and dependency
-relations while excluding structural and evidence context.
-
-DFG-18 makes the viewer a managed project-bound local service. A native per-user supervisor owns
-the viewer manager, which survives MCP process turnover, reuses the current snapshot URL, exposes
-status and explicit stop tools, and reclaims only genuinely idle workers.
-
-## Development
-
-Install Pyright once with `npm install -g pyright`. DocForge configures it to use the repository
-virtual environment and treats a clean strict run as a required development gate.
+Requirements: Python 3.12+, [`uv`](https://docs.astral.sh/uv/), and Node/npm for browser-asset
+validation. Install Pyright once globally.

 ```bash
+git clone forgejo@repo.andraxion.net:administrator/DocForge.git /absolute/path/DocForge
+cd /absolute/path/DocForge
 uv sync
 npm ci
-pyright
+npm install -g pyright
+
+pyright --pythonpath .venv/bin/python
 npm run lint:web
 uv run ruff check src tests tools
 uv run ruff format --check src tests tools
 uv run python -m unittest discover -s tests -v
-uv run docforge --project-root tests/fixtures/alpha validate
-uv run docforge --project-root tests/fixtures/alpha render-status manual
-uv run docforge --project-root tests/fixtures/alpha render manual
-uv run docforge-mcp --project-root tests/fixtures/alpha --proposal-writer alpha-editor
 ```

-The web gate validates the exact graph-browser HTML, CSS, and JavaScript served by DocForge. It also
-renders the fixture manual into a temporary project and validates that final HTML document.
+`npm run lint:web` validates the exact HTML, CSS, and JavaScript served by the graph browser, plus
+a freshly rendered fixture manual.

-The command prints deterministic JSON. Derived indexes live under each project's configured cache
-directory and are never canonical input.
+## Connect a generic project

-Proposal writers are declared by ID in `.docforge/project.toml`. Each writer receives explicit node
-families and operation types. The MCP process binds to one writer at startup; tools cannot select or
-impersonate another writer. Create, update, move, and delete tools write only isolated JSON
-changesets below the configured changeset root.
+### 1. Create the project descriptor

-Projects may also declare render views with confined template, preview, and output paths. The first
-built-in renderer converts CommonMark to escaped HTML through strict template tokens. MCP may render
-validated changesets only into isolated preview paths. Declared project output is generated through
-the explicit local CLI command and is never an MCP operation.
+Create `/absolute/path/MyProject/.docforge/project.toml`:

-Project adapters implement `AdapterLoader` and return one ordered, immutable `AdapterProjection`.
-`AdapterProject` validates the projection and exposes it through the same disposable index used by
-generic projects. Project-specific context, query ordering, and render-model policy remain in the
-adapter. Shadow adapters are local integration tools; the normal MCP server does not discover or
-execute them. An explicit integration may bind a validated adapter project to DocForge's read-only
-MCP surface without enabling proposal tools.
+```toml
+schema_version = 1
+project_id = "my-project"
+title = "My Project"
+adapter = "generic"

-Proposal-enabled adapters supply confined project settings and a project-owned validator. The core
-still owns hashes, permissions, changeset storage, conflict checks, graph validation, diffs, and
-preview confinement. The adapter owns source-format rules and may only narrow the allowed proposal
-surface.
+[sources]
+content_roots = ["Docs/Manual"]
+authority_files = []

-Both generic and explicit adapter MCP servers expose the same visualization tools because they read
-the validated `ProjectIndex` supplied by the project binding. `docforge_visualize` reuses its
-unguessable loopback URL when the current snapshot remains valid, or replaces the worker after a
-complete index check when it does not. The per-user viewer manager retains active workers across
-MCP process turnover, reports state through `docforge_visualization_status`, and reclaims idle
-workers after one hour. See [`docs/VIEWER_MANAGER.md`](docs/VIEWER_MANAGER.md) for installation.
+[derived]
+cache_root = ".docforge/cache"
+index = ".docforge/cache/index.sqlite3"

-See [`docs/NEW_PROJECT_QUICKSTART.md`](docs/NEW_PROJECT_QUICKSTART.md) for a complete generic MCP
-setup, continuous-agent policy, visualization instructions, and a project-adapter checklist.
+[changesets]
+root = ".docforge/changesets"
+
+[[changesets.writers]]
+id = "project-editor"
+families = ["architecture", "core", "system", "function", "operations", "roadmap"]
+operations = ["create", "update", "move", "delete"]
+
+[graph]
+allowed_relations = ["depends_on", "owns", "calls", "reads", "writes", "tested_by", "relates_to"]
+
+[limits]
+max_source_bytes = 500000
+max_nodes = 10000
+max_query_chars = 500
+max_results = 100
+max_traversal_depth = 6
+max_context_tokens = 12000
+max_changesets = 100
+max_changeset_operations = 100
+max_changeset_bytes = 1000000
+
+[[profiles]]
+id = "development"
+families = ["architecture", "core", "system", "function", "operations", "roadmap"]
+statuses = ["current", "active", "open", "verified"]
+required_nodes = ["architecture.overview"]
+token_budget = 8000
+dependency_depth = 3
+```
+
+Choose only family names, relations, limits, and required root nodes that the actual project can
+support truthfully.
+
+### 2. Add canonical nodes
+
+Create `/absolute/path/MyProject/Docs/Manual/architecture-overview.md`:
+
+```markdown
+++
+schema_version = 1
+id = "architecture.overview"
+title = "Architecture overview"
+family = "architecture"
+authority = "authoritative"
+status = "current"
+tags = ["architecture", "ownership"]
+summary = "Defines the top-level systems and ownership boundaries."
+++
+
+# Architecture overview
+
+Describe the project’s core authorities, system boundaries, persistence owners, runtime flow,
+failure behavior, tests, and operational entry points.
+```
+
+Every node needs a stable, unique `id`. Declare relationships in the same front matter:
+
+```toml
+depends_on = ["core.database"]
+calls = ["system.metadata"]
+tested_by = ["function.test-metadata-publication"]
+```
+
+Targets must already exist in the graph. Keep each node focused enough that an agent can retrieve
+relevant facts without loading the entire manual.
+
+### 3. Build and check the derived graph
+
+```bash
+DOCFORGE=/absolute/path/DocForge/.venv/bin/docforge
+PROJECT=/absolute/path/MyProject
+
+"$DOCFORGE" --project-root "$PROJECT" validate
+"$DOCFORGE" --project-root "$PROJECT" build
+"$DOCFORGE" --project-root "$PROJECT" check
+"$DOCFORGE" --project-root "$PROJECT" context development
+```
+
+`build` creates the disposable SQLite index in the configured cache path. Canonical Markdown and
+TOML remain authoritative. Rebuild after canonical documentation changes. A stale or altered index
+fails closed.
+
+## CLI command reference
+
+Every command emits deterministic JSON and begins with:
+
+```bash
+docforge --project-root /absolute/path/MyProject <command>
+```
+
+| Command | Purpose |
+| --- | --- |
+| `info` | Report the bound project and descriptor facts. |
+| `validate` | Validate the descriptor and canonical source graph. |
+| `build` | Build the disposable index. |
+| `check` | Validate source and confirm the index matches it. |
+| `validate-index` | Validate the existing derived index’s schema and identity. |
+| `show NODE_ID` | Return one complete node. |
+| `search QUERY [--limit N]` | Lexically search nodes. |
+| `filter [--family X] [--authority X] [--status X] [--tag X] [--limit N]` | Filter nodes by declared fields. |
+| `backlinks NODE_ID [--relation RELATION]` | Return incoming relationships. |
+| `dependencies NODE_ID [--depth N]` | Traverse declared dependencies. |
+| `impact NODE_ID [--depth N]` | Traverse likely downstream impact. |
+| `context PROFILE [--budget N]` | Return bounded, cited context for one configured profile. |
+| `render-status [VIEW_ID]` | Report declared render output state. |
+| `render VIEW_ID` | Generate one configured canonical render output. |
+| `preview CHANGESET_ID VIEW_ID` | Render one isolated changeset preview. |
+
+Examples:
+
+```bash
+docforge --project-root "$PROJECT" show architecture.overview
+docforge --project-root "$PROJECT" search metadata --limit 20
+docforge --project-root "$PROJECT" filter --family system --status current
+docforge --project-root "$PROJECT" backlinks system.metadata --relation calls
+docforge --project-root "$PROJECT" dependencies system.metadata --depth 3
+docforge --project-root "$PROJECT" impact system.metadata --depth 3
+docforge --project-root "$PROJECT" render-status manual
+docforge --project-root "$PROJECT" render manual
+```
+
+## MCP server
+
+Run one MCP server per project. Use absolute paths.
+
+```bash
+docforge-mcp --project-root /absolute/path/MyProject --proposal-writer project-editor
+```
+
+Omit `--proposal-writer` for a read-only integration. The writer value is a configured writer ID,
+not a shell command.
+
+For an MCP client that uses JSON configuration:
+
+```json
+{
+  "mcpServers": {
+    "my-project-docforge": {
+      "command": "/absolute/path/DocForge/.venv/bin/docforge-mcp",
+      "args": [
+        "--project-root",
+        "/absolute/path/MyProject",
+        "--proposal-writer",
+        "project-editor"
+      ]
+    }
+  }
+}
+```
+
+### Fixed read tools
+
+| MCP tool | Purpose |
+| --- | --- |
+| `docforge_project_info` | Project identity, index health, and capabilities. |
+| `docforge_get_contract` | Bound project contract and excluded operations. |
+| `docforge_get_node` | One exact node with full validated content. |
+| `docforge_search` | Lexical node search. |
+| `docforge_filter_nodes` | Filter by family, authority, status, or tag. |
+| `docforge_backlinks` | Incoming graph relationships. |
+| `docforge_dependencies` | Declared dependency traversal. |
+| `docforge_impact` | Downstream impact traversal. |
+| `docforge_get_context` | Bounded, cited context for a configured profile. |
+| `docforge_validate_project` | Validate source and derived index state. |
+| `docforge_render_status` | Declared render-output state. |
+| `docforge_visualize` | Start or reuse the project’s managed, read-only graph viewer. |
+| `docforge_visualization_status` | Viewer state, URL, and lifecycle information. |
+| `docforge_stop_visualization` | Explicitly stop the project viewer. |
+
+When a proposal writer is configured, the server additionally exposes:
+
+```text
+docforge_create_changeset
+docforge_list_changesets
+docforge_get_changeset
+docforge_propose_node_create
+docforge_propose_node_update
+docforge_propose_node_move
+docforge_propose_node_delete
+docforge_validate_changeset
+docforge_get_changeset_diff
+docforge_preview_changeset
+```
+
+MCP proposals write only isolated changesets. Review the diff, apply the approved text through the
+project’s normal workflow, then run `validate`, `build`, and `check`.
+
+## Managed graph viewer
+
+Install the per-user manager once:
+
+```bash
+docforge-viewer-manager install-user-service
+```
+
+Its lifecycle commands are:
+
+```bash
+docforge-viewer-manager serve
+docforge-viewer-manager uninstall-user-service
+```
+
+The native supervisor is systemd on Linux, a LaunchAgent on macOS, and Task Scheduler on Windows.
+The manager owns one loopback-only viewer worker per project snapshot. `docforge_visualize` starts
+or reuses it. Active browser requests renew its one-hour idle timeout. Stop it explicitly through
+`docforge_stop_visualization` when it is no longer wanted.
+
+The viewer is read-only, token-protected, and bound to `127.0.0.1`. It shows both:
+
+- **Nodes:** the bounded local graph with original stored relationship direction.
+- **Flow:** the complete bounded directed ancestry of the selected node. It follows every incoming
+  stored edge recursively, preserving `source → target`; it does not infer or reverse arrows based
+  on relationship names.
+
+## Project-owned source adapters
+
+Use the generic adapter when a declared manual graph is sufficient. Build a project-owned adapter
+when the graph must include source files, modules, functions, routes, tables, tests, or other
+code-derived facts.
+
+An adapter must:
+
+1. Read source as data. Never import or execute the application to discover facts.
+2. Use a deterministic, project-confined source set. For Git repositories, start with
+   `git ls-files` and exclude generated output, caches, secrets, and binaries.
+3. Emit only evidence-backed nodes and edges with stable IDs, safe relative anchors, and hashes.
+4. Sort nodes, edges, and metadata deterministically.
+5. Return one immutable `AdapterProjection` through `AdapterLoader.load_projection()`.
+6. Keep its cache within the project and build/check through `ProjectIndex`.
+7. Bind the projection to `create_read_only_server()` unless an explicit proposal policy exists.
+8. Reject proposal operations against derived nodes and adapter-created edges.
+
+The project owns its adapter and launches it rather than generic `docforge-mcp`:
+
+```json
+{
+  "mcpServers": {
+    "my-project-docforge": {
+      "command": "/absolute/path/MyProject/.venv/bin/python",
+      "args": [
+        "-m",
+        "docforge_adapter.server",
+        "serve",
+        "--project-root",
+        "/absolute/path/MyProject"
+      ]
+    }
+  }
+}
+```
+
+Require deterministic build, staleness, malformed-input, project-isolation, MCP-protocol, and
+project full-gate tests before relying on an adapter.
+
+## Agent policy
+
+Add a project-specific version of this to `AGENTS.md`:
+
+```markdown
+## Canonical documentation and DocForge
+
+- `/Docs/Manual/` is the canonical systems manual and architecture source of truth.
+- Before a systems-level, persistence, API, worker, plugin, operational, or mainline change, call
+  `docforge_project_info` or `docforge_validate_project`, then retrieve relevant nodes with
+  `docforge_search`, `docforge_get_context`, or `docforge_get_node`.
+- Inspect dependencies, backlinks, and impact at ownership boundaries. Read implementation and
+  tests as well as the manual. Treat a code/manual disagreement as a defect.
+- After implementation, create and validate an isolated DocForge changeset. Review its diff.
+- Apply approved text through the normal editing workflow, then run `validate`, `build`, and
+  `check`. Do not commit derived caches, changesets, or previews unless the project explicitly says
+  otherwise.
+- When asked to visualize a project or node, call `docforge_visualize`.
+```
+
+## Normal development loop
+
+1. Validate the DocForge project.
+2. Retrieve relevant context, backlinks, dependencies, and impact.
+3. Inspect the corresponding implementation and tests.
+4. Implement and test the change.
+5. Create and validate a DocForge proposal when manual changes are needed.
+6. Review the proposal diff and apply approved text through the normal project workflow.
+7. Run `validate`, `build`, and `check`.
+8. Run the project’s complete test gate.
+9. Commit code, tests, and canonical manual updates together.
+
+DocForge never applies, commits, pushes, builds, deploys, or publishes on the project’s behalf.