diff --git a/README.md b/README.md index 2fcdd38..b756563 100644 --- a/README.md +++ b/README.md @@ -68,3 +68,6 @@ Both generic and explicit adapter MCP servers expose the same visualization tool the validated `ProjectIndex` supplied by the project binding. Invoking it again refreshes the browser only after a complete index check. The ephemeral listener and its unguessable URL token live only for the MCP process lifetime. + +See [`docs/NEW_PROJECT_QUICKSTART.md`](docs/NEW_PROJECT_QUICKSTART.md) for a complete generic MCP +setup, continuous-agent policy, visualization instructions, and a project-adapter checklist. diff --git a/docs/NEW_PROJECT_QUICKSTART.md b/docs/NEW_PROJECT_QUICKSTART.md new file mode 100644 index 0000000..f78e65e --- /dev/null +++ b/docs/NEW_PROJECT_QUICKSTART.md @@ -0,0 +1,354 @@ +# DocForge: New Project Quickstart + +This guide connects one project to DocForge so an AI agent can read its documentation graph before +development work, propose documentation updates during development, inspect impact, and keep the +canonical manual synchronized with the code. + +DocForge is project-bound. Run one MCP server per project. Its MCP tools can read the validated +graph and write isolated proposals, but they cannot directly edit canonical documentation, run +builds, use Git, or deploy software. Canonical changes must be applied through the project's normal +editing and review workflow. + +## What this installs + +The DocForge repository contains the complete generic CLI and stdio MCP server. Launching +`docforge-mcp` as shown below exposes: + +- Project identity, validation, exact-node retrieval, lexical search, and filtering. +- Backlinks, dependency traversal, impact traversal, and bounded context profiles. +- `docforge_visualize`, which starts the token-protected, loopback-only `graph-browser@2` viewer. + The viewer supports search, family filtering, exact-node inspection, bounded neighborhoods, + mouse-wheel zoom, left-button drag panning, zoom controls, and viewport reset. +- Isolated documentation changesets, proposal validation, diffs, and escaped HTML previews when a + proposal writer and render view are configured. + +The generic adapter graphs the Markdown and TOML nodes and relationships declared by the project. +It does **not** inspect arbitrary source code or automatically infer modules, functions, calls, +routes, database tables, tests, or ownership. + +A project that needs automatic source-code graph extraction must provide a project-specific +DocForge adapter. The adapter deterministically projects those code facts into DocForge nodes and +edges, then binds that projection to DocForge's standard index and MCP tools. Ani-web's large code +graph is an example of a custom adapter; it is not behavior supplied by the generic quickstart. + +## 1. Install DocForge + +Requirements: + +- Python 3.12 or newer. +- `uv`. +- Access to the DocForge repository. + +```bash +git clone forgejo@repo.andraxion.net:administrator/DocForge.git /absolute/path/DocForge +cd /absolute/path/DocForge +uv sync +uv run python -m unittest discover -s tests -v +``` + +Use absolute paths in all MCP configuration. + +## 2. Configure the project + +Create `/absolute/path/MyProject/.docforge/project.toml`: + +```toml +schema_version = 1 +project_id = "my-project" +title = "My Project" +adapter = "generic" + +[sources] +content_roots = ["Docs/Manual"] +authority_files = [] + +[derived] +cache_root = ".docforge/cache" +index = ".docforge/cache/index.sqlite3" + +[changesets] +root = ".docforge/changesets" + +[[changesets.writers]] +id = "project-editor" +families = ["architecture", "core", "system", "function", "operations", "roadmap"] +operations = ["create", "update", "move", "delete"] + +[graph] +allowed_relations = [ + "depends_on", + "owns", + "calls", + "reads", + "writes", + "tested_by", + "relates_to", +] + +[limits] +max_source_bytes = 500000 +max_nodes = 10000 +max_query_chars = 500 +max_results = 100 +max_traversal_depth = 6 +max_context_tokens = 12000 +max_changesets = 100 +max_changeset_operations = 100 +max_changeset_bytes = 1000000 + +[[profiles]] +id = "development" +families = ["architecture", "core", "system", "function", "operations", "roadmap"] +statuses = ["current", "active", "open", "verified"] +required_nodes = ["architecture.overview"] +token_budget = 8000 +dependency_depth = 3 +``` + +Choose family names, relationship names, limits, and the required root node for the actual project. +Do not copy relationships that the project cannot support truthfully. + +## 3. Create canonical nodes + +Create `/absolute/path/MyProject/Docs/Manual/architecture-overview.md`: + +```markdown ++++ +schema_version = 1 +id = "architecture.overview" +title = "Architecture overview" +family = "architecture" +authority = "authoritative" +status = "current" +tags = ["architecture", "ownership"] +summary = "Defines the top-level systems and ownership boundaries." ++++ + +# Architecture overview + +Describe the project, its core authorities, system boundaries, persistence owners, runtime flow, +failure behavior, tests, and operational entry points. +``` + +Every node needs a stable, unique `id`. Relationships are declared in the same front matter: + +```toml +depends_on = ["core.database"] +calls = ["system.metadata"] +tested_by = ["function.test-metadata-publication"] +``` + +The relationship target must already exist in the project graph. Keep nodes focused enough that an +agent can retrieve the relevant facts without loading the whole manual. + +## 4. Validate and build the graph + +```bash +DOCFORGE=/absolute/path/DocForge/.venv/bin/docforge +PROJECT=/absolute/path/MyProject + +"$DOCFORGE" --project-root "$PROJECT" validate +"$DOCFORGE" --project-root "$PROJECT" build +"$DOCFORGE" --project-root "$PROJECT" check +"$DOCFORGE" --project-root "$PROJECT" context development +``` + +`build` creates the disposable SQLite index under `.docforge/cache/`. Canonical Markdown remains the +source of truth. Rebuild after canonical documentation changes. A stale or altered index fails +closed. + +## 5. Register the MCP server + +Add a project-specific stdio MCP server to the agent host: + +```json +{ + "mcpServers": { + "my-project-docforge": { + "command": "/absolute/path/DocForge/.venv/bin/docforge-mcp", + "args": [ + "--project-root", + "/absolute/path/MyProject", + "--proposal-writer", + "project-editor" + ] + } + } +} +``` + +Restart or reload the agent host. Confirm that it exposes tools beginning with +`docforge_project_info`, `docforge_get_node`, `docforge_search`, `docforge_dependencies`, +`docforge_impact`, `docforge_get_context`, `docforge_visualize`, and +`docforge_create_changeset`. + +Omit `--proposal-writer` for a read-only integration. + +## 5A. Optional source-code adapter + +The generic setup above is complete when the project only needs a graph of its declared manual +nodes. Build a project-specific adapter when the graph must also contain source files, modules, +functions, routes, tables, tests, services, plugins, or other facts derived from the codebase. + +Tell Codex to keep the adapter inside the owning project, for example: + +```text +MyProject/ + docforge_adapter/ + __init__.py + graph.py # Parse tracked source as data and produce project facts. + integration.py # Translate facts into DocForge nodes, edges, and policy. + server.py # Build, check, report, and serve the bound MCP process. + tests/ + test_docforge_adapter.py +``` + +The adapter must: + +1. Read source files as data. It must not import or execute the application to discover facts. +2. Select a deterministic, project-confined source set. A Git project should normally begin with + `git ls-files`; exclude generated manuals, caches, build output, secrets, and binary artifacts. +3. Parse each supported language or format with deterministic parsers. Emit only relationships + backed by direct evidence. Omit uncertain calls, ownership, or persistence edges instead of + guessing. +4. Give every node a stable ID, family, authority, status, summary, safe relative source path, + optional source anchor, and SHA-256 content hash. +5. Give every edge stable source and target IDs, a declared relation, and adapter metadata that + records its evidence and origin. +6. Sort nodes by node ID, edges by source/relation/target, and metadata by key. +7. Implement `AdapterLoader.load_projection()` and return one immutable `AdapterProjection` with a + resolved project root, revision, adapter ID/version, and lowercase SHA-256 source hash. +8. Wrap the loader in `AdapterProject`, place its cache inside the project, and build/check it + through `ProjectIndex`. +9. Bind the project to `create_read_only_server()`. Use `create_project_server()` only after adding + explicit `AdapterProjectSettings`, a startup-bound proposal writer, confined canonical sources, + and a validator that prevents proposals from changing derived source nodes or adapter edges. +10. Supply a project context provider if `docforge_get_context` needs project-specific profiles. + Search, node retrieval, traversal, impact, and visualization work directly from the standard + index. + +The essential integration shape is: + +```python +from pathlib import Path + +from docforge.adapter_contract import AdapterProject +from docforge.index import ProjectIndex +from docforge.mcp_server import create_read_only_server + +from .graph import ProjectAdapterLoader + + +def open_project(root: Path): + loader = ProjectAdapterLoader(root.resolve(strict=True)) + project = AdapterProject( + loader, + cache_root=loader.root / ".docforge" / "cache" / "project-adapter", + ) + return project, ProjectIndex(project) + + +def serve(root: Path) -> None: + project, index = open_project(root) + index.check() # Build explicitly before serving; never hide stale state. + create_read_only_server(project).run(transport="stdio") +``` + +The project-owned `server.py` should provide explicit `build`, `check`, and `serve` operations. The +MCP host then launches that module instead of the generic `docforge-mcp` command: + +```json +{ + "mcpServers": { + "my-project-docforge": { + "command": "/absolute/path/MyProject/.venv/bin/python", + "args": [ + "-m", + "docforge_adapter.server", + "serve", + "--project-root", + "/absolute/path/MyProject" + ] + } + } +} +``` + +If the adapter combines canonical manual nodes with derived code nodes, load the generic manual +through `Project.open(project_root)`, translate its canonical nodes and edges into `AdapterNode` and +`AdapterEdge`, merge them with the derived projection, and reject proposal operations against all +derived node IDs and adapter-created relationships. + +Require these adapter acceptance checks: + +- Two unchanged builds produce the same ordered nodes, edges, source hash, and projection identity. +- A tracked source change changes the source hash and makes the old index stale. +- Missing targets, duplicate IDs, unsafe paths, unsorted metadata, and invalid hashes fail closed. +- Building the graph does not import the application or cause runtime, network, database, or + filesystem side effects. +- Read-only MCP exposes only the fixed DocForge read surface, including `docforge_visualize`. +- Proposal-enabled MCP cannot modify derived source facts or write outside confined changeset and + preview roots. +- Build, check, MCP protocol tests, adapter tests, and the owning project's full test gate pass. + +### Ready-to-give Codex setup request + +```text +Install and configure DocForge for this repository using the new-project quickstart. Use the +generic adapter for the canonical manual, and create a project-owned source adapter if source-code +graph extraction is required. Inspect the repository languages and formats before choosing +parsers. Read tracked source as data; never import or execute the application for discovery. +Produce deterministic, evidence-backed nodes and edges with stable IDs and safe source anchors. +Omit relationships that cannot be proven. Bind one project-scoped MCP server with visualization, +build and check the index explicitly, add adapter contract and staleness tests, and add the supplied +DocForge policy to AGENTS.md. Keep canonical manual writes outside MCP: use isolated proposals, +review their diffs, apply them through the normal project workflow, then rebuild and check the +index. Do not report completion until DocForge validation, adapter tests, MCP protocol checks, and +the project's complete test gate pass. +``` + +## 6. Tell the agent to use DocForge automatically + +Place this policy in the project's `AGENTS.md` and adjust the manual path and profile if needed: + +```markdown +## Canonical documentation and DocForge + +- `/Docs/Manual/` is the canonical systems manual and architecture source of truth. +- Use the project-bound DocForge MCP server for every systems-level, architectural, persistence, + plugin, worker, API, operational, or mainline change. +- Before editing, call `docforge_project_info` or `docforge_validate_project`. Retrieve the relevant + nodes with `docforge_search`, `docforge_get_context`, or `docforge_get_node`. Inspect dependencies, + backlinks, and impact when changing an owned boundary. +- Read the implementation and tests as well as the manual. Treat a code/manual disagreement as a + defect. Do not silently choose one side. +- During implementation, preserve explicit ownership, inputs, outputs, state writes, failure + behavior, callers, tests, and operational consequences. +- After implementation, create an isolated DocForge changeset. Propose every required manual + update, validate the changeset, and inspect its diff. +- DocForge proposals do not edit canonical files. Apply the reviewed proposal through the project's + normal file-editing workflow, then run DocForge `validate`, `build`, and `check`. +- Commit code, tests, and canonical documentation together. Never commit `.docforge/cache/`, + `.docforge/changesets/`, or previews unless the project explicitly declares otherwise. +- If DocForge reports stale state, missing nodes, invalid edges, or an index mismatch, stop and + repair or rebuild the graph before claiming the work complete. +- When asked to “visualize” the project or a node, call `docforge_visualize`. The viewer is + loopback-only, read-only, and lives only while the MCP process runs. +``` + +The policy is what makes DocForge part of normal development rather than an optional lookup tool. + +## 7. Normal development loop + +1. Validate the DocForge project. +2. Retrieve the relevant context and impact graph. +3. Inspect the corresponding code and tests. +4. Implement and test the change. +5. Create and validate a DocForge proposal. +6. Review the proposal diff. +7. Apply the approved text through normal project editing. +8. Run `validate`, `build`, and `check`. +9. Run the project's complete test gate. +10. Commit code, tests, and canonical manual updates together. + +DocForge never applies, commits, pushes, builds, deploys, or publishes on the project's behalf.