DocForge2/README.md

# DocForge

DocForge is a project-scoped documentation and source graph for people and AI agents. Canonical
project files remain authoritative; DocForge validates them, builds disposable search and
relationship indexes, compiles bounded context, manages reviewable changesets, renders declared
manuals and portable graph views, and serves one project-bound MCP surface.

DocForge never treats indexed text as instructions. It does not run project build commands,
compilers, Git operations, deployments, or arbitrary renderers, and it does not select projects
globally.

DocForge 1.0.0 remains the latest published stable release in this product line. This checkout is
the `1.4.0` release candidate and contains the complete Milestone 4 adapter platform plus
Milestone 5 stabilization evidence. The `v1.4.0` tag and Forgejo release are still pending; this
candidate is not published yet.

## Start here

- New installation or first project: [New-project quickstart](docs/NEW_PROJECT_QUICKSTART.md)
- Mental model and authority: [Core concepts and authority](docs/CORE_CONCEPTS_AND_AUTHORITY.md)
- Complete configuration shape: [Project descriptor](docs/PROJECT_DESCRIPTOR.md)
- Fixed runnable examples: [Reference adapters](docs/REFERENCE_ADAPTERS.md)
- Agent and client setup: [Agent integration](docs/AGENT_INTEGRATION.md)
- Exact live command and tool inventory: [Generated command reference](docs/COMMAND_REFERENCE.md)
- Task-oriented operating guide: [User manual](docs/USER_MANUAL.md)
- Release-candidate changes and evidence: [Changelog](CHANGELOG.md) and [Milestone 5
  baseline](docs/MILESTONE_5_BASELINE.md)

## Current capabilities

- Validates Markdown and TOML nodes, stable IDs, typed relationships, project limits, and confined
  paths.
- Builds a deterministic, disposable SQLite graph and search index with integrity and generation
  evidence.
- Exposes project-bound CLI and MCP read, proposal, application, rendering, and visualization
  surfaces according to the startup policy.
- Compiles versioned, generation-bound task context with cited evidence, explicit gaps, bounded
  output, and deterministic continuation.
- Creates isolated documentation changesets, validates complete projected graphs, and applies only
  one explicitly approved changeset hash through a separately bound canonical applier.
- Supports complete-projection adapters and opt-in incremental adapters with reverse-dependency
  invalidation, bounded extraction caches, and a clean full-build equivalence oracle.
- Publishes function-scoped Logic separately from the primary graph for Python, JavaScript,
  TypeScript, and C++ integrations that provide it.
- Compiles manual and portable graph plans into immutable packages for fixed detached renderers,
  then records bounded receipts.
- Runs a managed loopback graph viewer with Nodes, Flow, Web, lazy Logic, source inspection, and
  branch-aware hiding.
- Generates deterministic Codex, Claude, and OpenClaw client fragments without copying ambient
  environment values or secrets.

## Adapter platform

Adapter authors use the public `docforge.adapter_sdk` surface for graph types, complete and
incremental contracts, validation helpers, and `verify_adapter_conformance()`. Existing adapters
that implement only `load_projection()` remain supported. Incremental adapters add a manifest and
source extraction while retaining `load_projection()` as the complete graph oracle. An adapter
that publishes Logic incrementally also supplies `load_complete_assembly()` so the complete oracle
covers both graph and Logic.

The repository includes bounded reference integrations for:

- Python, using the standard-library AST and publishing only project-local imports;
- JavaScript and TypeScript, using their distinct optional Tree-sitter grammars and publishing only
  project-local static relative imports and re-exports;
- C++, using `compile_commands.json` as translation-unit inventory and fingerprint evidence, without
  executing its commands or a compiler, and publishing only directly resolvable project-local
  quoted includes.

These integrations demonstrate the adapter contract; they do not claim resolved calls,
inheritance, types, runtime behavior, macro expansion, compiler include semantics, or semantic
ownership. Python works from the base wheel. Install the `javascript`, `typescript`, or `cpp`
extra for the corresponding grammar, or `languages` for all three:

```bash
uv pip install "/absolute/path/DocForge[javascript]"
uv pip install "/absolute/path/DocForge[typescript]"
uv pip install "/absolute/path/DocForge[cpp]"
uv pip install "/absolute/path/DocForge[languages]"
```

Reference projects use the fixed `.docforge/reference-adapter.toml` descriptor and the installed
`docforge.reference_mcp` module. That server is read-only and exposes no proposal or application
surface.

Project-owned adapters use `AdapterLauncherV1` and
`generate_adapter_client_configuration()` to produce a client fragment from an explicitly
constructed adapter project. The launcher is immutable and contains no command, caller arguments,
working directory, environment, discovery rule, or callable selector. It resolves through isolated
Python to one installed, project-owned top-level module; the only trusted dotted exception is the
fixed `docforge.reference_mcp` binding. Generic `docforge configure` intentionally refuses custom
adapters because it cannot safely reconstruct project-owned composition.

See the [Adapter authoring guide](docs/ADAPTER_AUTHORING_GUIDE.md), [Reference
adapters](docs/REFERENCE_ADAPTERS.md), and [Agent integration](docs/AGENT_INTEGRATION.md) for the
supported routes.

## Authority and projections

Canonical Markdown, TOML, adapter-declared sources, and descriptor files own project facts. The
SQLite index, extraction cache, render packages, previews, portable artifacts, receipts, client
fragments, and viewer processes are derived and replaceable.

The primary graph contains project nodes and relationships. Logic is a separate, lazy,
function-scoped control-flow projection. Manual output, portable graph output, and the live viewer
are independent consumers of one validated generation:

```text
validated generation
    ├── ManualRenderPlanV1 → immutable package → detached manual renderer
    ├── GraphViewPlanV1    → immutable package → detached portable graph renderer
    └── pinned index       → managed read-only live viewer
```

The version-2 projection policy selects each consumer independently:

```text
manual:         auto | explicit | disabled
portable_graph: explicit | disabled
live_viewer:    on-demand | disabled
```

The process capability policy is separate. Capability mode controls the registered read,
proposal, application, or reserved operator surface; descriptor writers and the startup-bound
canonical applier determine whether mutations are actually authorized. `--no-ast` is a restrictive
binding policy over adapter evolution, Logic publication, and Logic retrieval. It is not a parser
inspection mechanism or a filesystem sandbox, and the AST/Tree-sitter reference integrations
should not be presented as no-AST adapters.

Read [Policy precedence](docs/POLICY_PRECEDENCE.md), [Rendering and
visualization](docs/RENDERING_AND_VISUALIZATION.md), and [Legacy and no-AST
operation](docs/LEGACY_AND_NO_AST.md) before changing those boundaries.

## Five-minute generic project

Requirements are Python 3.12 or newer and an installed DocForge environment. From a development
checkout, `uv sync --group dev` creates `.venv`:

```bash
git clone <repository-url> /absolute/path/DocForge
cd /absolute/path/DocForge
uv sync --group dev

PROJECT=/absolute/path/MyProject
.venv/bin/docforge --project-root "$PROJECT" onboard
.venv/bin/docforge --project-root "$PROJECT" onboard \
  --scaffold \
  --project-id my-project \
  --title "My Project"
```

The first command is read-only. Scaffolding is explicit and create-only: it writes a generic
descriptor, one canonical overview node, and a built-in manual template, then indexes and renders
them. Detected source languages remain `adapter_required` until a validated frontend is selected.

Operate the configured project:

```bash
.venv/bin/docforge --project-root "$PROJECT" validate
.venv/bin/docforge --project-root "$PROJECT" reindex
.venv/bin/docforge --project-root "$PROJECT" search architecture
.venv/bin/docforge --project-root "$PROJECT" visualize
```

Install the persistent per-user graph viewer manager once when using the live viewer:

```bash
.venv/bin/docforge-viewer-manager install-user-service
```

Start one generic, read-only MCP server:

```bash
.venv/bin/docforge-mcp \
  --project-root "$PROJECT" \
  --capability-mode read
```

Add a descriptor-authorized proposal writer only when the client should create proposals. Add a
matching `--canonical-applier` only when that integration should expose exact-hash application.
See the [quickstart](docs/NEW_PROJECT_QUICKSTART.md) for the reference-adapter and generated-client
paths.

## Documentation

### Learn and operate

- [New-project quickstart](docs/NEW_PROJECT_QUICKSTART.md)
- [Core concepts and authority](docs/CORE_CONCEPTS_AND_AUTHORITY.md)
- [Project descriptor](docs/PROJECT_DESCRIPTOR.md)
- [Policy precedence](docs/POLICY_PRECEDENCE.md)
- [Reference adapters](docs/REFERENCE_ADAPTERS.md)
- [Agent integration](docs/AGENT_INTEGRATION.md)
- [User manual](docs/USER_MANUAL.md)
- [Generated command reference](docs/COMMAND_REFERENCE.md)
- [Project onboarding](docs/PROJECT_ONBOARDING.md)
- [Rendering and visualization](docs/RENDERING_AND_VISUALIZATION.md)
- [Recovery and performance](docs/RECOVERY_AND_PERFORMANCE.md)
- [Security](docs/SECURITY.md)

### Contracts and compatibility

- [Core contract](docs/CONTRACT.md)
- [MCP contract](docs/MCP_CONTRACT.md)
- [Compatibility contract](docs/COMPATIBILITY.md)
- [Adapter authoring guide](docs/ADAPTER_AUTHORING_GUIDE.md)
- [Incremental indexing](docs/INCREMENTAL_INDEXING.md)
- [Adapter application decision](docs/APPLICATION_DECISION.md)
- [Legacy and no-AST operation](docs/LEGACY_AND_NO_AST.md)
- [Migrating from version 1](docs/MIGRATING_FROM_V1.md)
- [Viewer manager](docs/VIEWER_MANAGER.md)

### Milestone evidence

- [Milestone 5 baseline](docs/MILESTONE_5_BASELINE.md)
- [Milestone 5 release-candidate closeout](docs/MILESTONE_5_CLOSEOUT.md)
- [Milestone 4 baseline](docs/MILESTONE_4_BASELINE.md)
- [Milestone 4 closeout](docs/MILESTONE_4_CLOSEOUT.md)
- [Milestone 3 baseline](docs/MILESTONE_3_BASELINE.md) and [closeout](docs/MILESTONE_3_CLOSEOUT.md)
- [Milestone 2 baseline](docs/MILESTONE_2_BASELINE.md) and [closeout](docs/MILESTONE_2_CLOSEOUT.md)
- [Milestone 1 baseline](docs/MILESTONE_1_BASELINE.md) and [closeout](docs/MILESTONE_1_CLOSEOUT.md)
- [Milestone 0 baseline](docs/MILESTONE_0_BASELINE.md) and [closeout](docs/MILESTONE_0_CLOSEOUT.md)

Historical milestone records preserve the facts and dependency observations of their frozen
candidates. Use the current guides and contracts for present behavior.

## Development

Run the repository-native gate:

```bash
make gate
```

Milestone 4 maintenance entry points include:

```bash
make adoption-m4
make benchmark-m4-smoke
make benchmark-m4
make benchmark-m4-full
make command-reference-check
make docs-check
```

`adoption-m4` builds and exercises a fresh base wheel without Tree-sitter packages and proves the
Python reference plus a real isolated read-only MCP retrieval. The optional-language integrations
have their own focused tests and extras. `benchmark-m4` runs the maintained full adapter workload;
the smoke target is for routine gate coverage, not final performance evidence.

Milestone 5 adds focused and aggregate release-candidate gates:

```bash
make compatibility-m5
make migration-m5
make concurrency-m5
make recovery-m5
make task-evidence-m5
make release-gate
make fresh-clone-m5
```

`release-gate` includes the complete repository, accessibility, adoption, artifact-reproducibility,
secret-scan, and full maintained benchmark sequence. `fresh-clone-m5` is the final remote-commit
rehearsal and must pass before the annotated tag and Forgejo release are created.

Pass `--diagnostics` to `docforge` or `docforge-mcp` for bounded request-local timings and compiler
work counters. Diagnostics are disabled by default and do not displace a primary result when the
configured output budget is tight.

Read [AGENTS.md](AGENTS.md) before changing core boundaries.