DocForge2/docs/RECOVERY_AND_PERFORMANCE.md

# Recovery and performance

DocForge keeps canonical project evidence separate from disposable indexes, caches, receipts, and
rendered projections. Recovery rebuilds derived state from current authority; it does not rewrite
canonical content to make a cache look valid.

For the underlying boundaries, see [core authority](CORE_CONCEPTS_AND_AUTHORITY.md) and the
[security model](SECURITY.md).

## Recovery order

Use the narrowest verified recovery:

1. Run `docforge check` or the corresponding MCP status operation.
2. Run `docforge sync` to repair a missing or stale disposable index when safe.
3. Run `docforge reindex` for an explicit complete rebuild.
4. Restart a long-running MCP or viewer binding after descriptor or adapter implementation drift.
5. Regenerate a derived client fragment, command reference, preview, manual, or portable graph
   from current project evidence.

Never delete or rewrite canonical sources, active changesets, or reviewed hashes as cache cleanup.

## Extraction-cache recovery

Incremental adapter contributions are stored in a version-1 disposable extraction cache. Reads are
no-follow, regular-file-only, identity-checked, bounded to 64,000,000 bytes and 10,000 source
records, and fail to a cache miss on malformed or incompatible data. Publication is atomic.

After a miss, current manifest sources are extracted again and the complete assembly is validated.
The active SQLite generation remains authoritative for reads until a new verified index is
published. An extraction cache may therefore be safely ahead of the last index; the two files do
not pretend to be one transaction.

## Index and receipt recovery

A validated SQLite index is a generation-pinned derived snapshot. Missing, corrupt, unattested, or
stale indexes rebuild from the complete project or adapter oracle. Generation receipts and render
receipts are post-publication evidence. Failure to write a receipt after a committed artifact is
reported as degraded committed success, not as permission to repeat a mutation.

The live viewer pins one validated index identity. Index replacement makes the running snapshot
stale and causes a later visualize request to start a fresh worker.

Milestone 5 maintains exact-oracle recovery proofs for four independently disposable artifacts:

- A corrupt index attestation is rejected, then `synchronize()` recreates the exact attestation
  after complete verification while preserving canonical bytes, snapshot hash, and index identity.
- A corrupt manual-render receipt reports `unverified/receipt_corrupt`; explicit rendering restores
  the exact output bytes and semantic receipt, and normal and deep status return `current`.
- A corrupt generation-diff receipt reports `unverified/corrupt_receipt`; a complete index build
  recreates the exact current-graph baseline with `baseline/no_meaningful_transition`.
- A corrupt portable-graph manifest reports missing publication evidence; explicit rendering
  restores the exact artifact bytes and semantic manifest, and status returns `current`.

These proofs are maintained by `tests/test_milestone5_recovery.py`. They do not promote receipts,
attestations, or manifests to canonical authority.

## Proposal and application recovery

Hash or base conflicts are not cache failures. Retrieve the current changeset and diff, then
review the new exact hash. Rebase is allowed only when every touched node, relationship, source,
permission, and graph invariant still matches. A content conflict requires a new proposal.

Generic canonical create, update, and delete publication compares exact target identity at the
commit boundary. Concurrent target mutation fails closed. In-process failures roll back only when
the exact displaced state remains provable; otherwise DocForge preserves foreign data and returns
`application_recovery_required` with retained evidence.

Canonical application is not journaled across several files. Each file publication is atomic, but
process or host death between publications can leave a partial canonical application. Inspect the
named canonical targets, the active proposal, and `.docforge/application/transaction-*` before
deciding whether to restore or create a new proposal. Do not retry the old approved hash merely
because a process ended.

If semantic application committed but private transaction cleanup degraded, the result remains
`applied`. The proposal is closed and its compact lifecycle receipt records
`application_recovery.status = "cleanup_required"`, retained paths, and remediation. Preserve and
inspect those files. Remove only artifacts proven to be DocForge-owned. If a canonical serializer
fails its round-trip check before success, use its reported rollback state. See
[migrating from v1](MIGRATING_FROM_V1.md) for rollback planning.

## Milestone 4 scale evidence

The maintained Python reference benchmark creates 334 Python source files and produces:

- 1,002 primary nodes and 1,001 primary edges.
- 334 Logic projections with 2,338 Logic nodes and 2,338 Logic edges.
- Exact complete/incremental primary graph and Logic equality.
- Exact output after corrupt extraction-cache recovery and corrupt-index recovery.
- Zero `ast.parse` calls and zero `extract_source` calls during a warm build.

On the frozen Milestone 4 candidate, operation p95 values were 1.266 to 1.898 seconds. Per-operation
traced peaks were about 61 to 67 MiB, and process high-water was 78,798,848 bytes. These are
regression measurements from one machine, not universal latency promises. The machine-readable
record is `../benchmarks/milestone4-2026-07-29.json`; detailed method and hashes are in
[the Milestone 4 baseline](MILESTONE_4_BASELINE.md).

The zero-parser claim is deliberately Python-only. Focused JavaScript and TypeScript tests prove
their manifests avoid Tree-sitter. The C++ reference manifest currently uses Tree-sitter while
discovering bounded quoted includes, so a warm C++ cache hit is not evidence of zero parser work.

## Maintained gates

Run:

```bash
make gate
make adoption-m4
make benchmark-m4-full
make docs-check
make compatibility-m5
make migration-m5
make concurrency-m5
make recovery-m5
make task-evidence-m5
make release-gate
make fresh-clone-m5
```

The main gate includes smoke benchmarks. Full milestone evidence is recorded separately from a
clean candidate so smoke or dirty-tree results cannot become release claims. `release-gate`
aggregates the complete quality, compatibility, migration, concurrency, recovery, task-evidence,
fresh-wheel, identity, reproducible-artifact, secret-scan, and full benchmark proofs.
`fresh-clone-m5` repeats that aggregate gate from anonymous HTTPS at one exact published commit
after verifying the frozen annotated `v1.0.0` migration tag.