1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

Compare commits

...

16 commits

46 changed files with 6108 additions and 136 deletions

View file

@ -1,15 +1,134 @@
# Milestone state
```text
Last completed milestone: 4 — adapter SDK and product documentation.
Evidence: Frozen executable candidate 95271dcf2e48045b9d3aed9b9ea09c7fc155692c; 347 tests and 402 subtests; 3 accessibility flows; offline fresh-wheel adoption; clean 1,002-node graph-plus-Logic benchmark; no secret-scan findings.
Active milestone: None.
Outcome: New projects can adopt DocForge incrementally, and users can understand the CLI, MCP server, policies, adapters, renderers, and agent integration without reading implementation code.
Status: Milestone 4 is closed. Milestone 5 has not started.
Next directional milestone: 5 — stabilization and first DocForge2 release.
Last completed milestone: 5 — stabilization and first DocForge2 release.
Baseline: annotated v1.4.0 release commit on main, dev, origin/main, and origin/dev.
Active milestone: none.
Outcome: Compatibility, determinism, recovery, security, performance, and representative task advantage are proven for the first successor release.
Status: Complete.
Release: 1.4.0.
```
Milestone 5 remains directional context. Do not begin release stabilization, versioning, tagging,
or publication without a new active-slice contract based on the Milestone 4 evidence.
## Authority
The completed Milestone 4 contract and its exclusions are preserved in `SLICE_HISTORY.md`.
This contract activated Milestone 5 from the clean, merged, and pushed Milestone 4 closeout. The
roadmap in `/home/andraxion/.openclaw/workspace/DocForgeOutline.md` supplies direction; this file
freezes the executable scope and acceptance criteria.
The completed release synchronizes `main` and `dev` at the documentation-bearing commit identified
by annotated tag `v1.4.0`.
## Required release evidence
The release candidate must prove all of the following from maintained, reproducible gates:
1. Legacy one-method adapter compatibility and the frozen package, CLI, MCP, schema, rendering,
descriptor, result-envelope, and no-AST surfaces.
2. Exact complete/incremental primary-graph and Logic equivalence for maintained incremental
adapters.
3. Deterministic adapter, worker, renderer, command-reference, configuration, and publication
output.
4. Detection of concurrent source mutation without publishing or serving a mixed or stale
generation.
5. Atomic, crash-safe derived publication and exact recovery from interrupted publication.
6. Corrupt extraction-cache, index, attestation, receipt, fragment, and projection recovery using
canonical sources as the authority.
7. No stale reads after source or policy change, including generation-pinned retrieval and viewer
behavior.
8. Exact-hash canonical application with project-owned serialization, stale-proposal rejection,
and post-application resynchronization.
9. Closed policy precedence across process capability, descriptor policy, `--no-ast`, projection
policy, worker enforcement, and viewer enforcement.
10. Manual and portable-graph isolation, immutable package verification, bounded detached workers,
rendering-policy enforcement, viewer-policy enforcement, and accessibility.
11. Comparative representative task evidence with fixed questions, answer keys, provenance,
latency, and response-size accounting for both graph-assisted and source-only workflows.
12. A fresh-wheel and fresh-clone release rehearsal, full quality and browser gates, maintained
benchmarks, secret scans, exact version identity, signed-off release notes, and reproducible
release artifacts.
## Deliverables
- A maintained Milestone 5 compatibility matrix and aggregate release gate.
- Reproducible migration, recovery, concurrency, and comparative-task evidence.
- One authoritative package version shared by package metadata, Python, CLI, MCP, viewer manager,
reference MCP, generated configuration, and release documentation.
- A standard project license file, changelog or release notes, release baseline, closeout record,
and machine-readable evidence.
- A clean `main` merge, synchronized `main` and `dev`, annotated `v1.4.0` tag, and Forgejo release
only after the final documentation-bearing commit passes a fresh-clone gate.
## Fixed boundaries
- Preserve the `docforge` package, `docforge` CLI, `docforge-mcp`, MCP tool names, schema version 1
surfaces, effective policy version 1, projection policy version 2, and legacy adapter entry point.
- Preserve canonical project sources. Migration may rebuild disposable state but may not rewrite
canonical content merely to satisfy the release.
- Full rebuild remains the recovery and equivalence oracle.
- Derived artifacts must fail closed on malformed, foreign, stale, oversized, or incompatible
state.
- Version `1.4.0` is additive relative to `1.0.0`; breaking a frozen contract requires a separately
justified major-version decision.
## Exclusions
- No WorldForge change or benchmark.
- No ScrapeStation change, production binding change, or production migration.
- No legacy-repository mutation.
- No production MCP repointing.
- No remote render service, render farm, third-party renderer ecosystem, graph federation, or
dedicated graph database.
- No arbitrary adapter command execution, compiler execution, remote execution, or expanded
launcher authority.
- No PyPI publication unless it is separately verified as an intended existing release channel.
## Release sequence
1. Freeze and implement the compatibility, migration, recovery, concurrency, and task-evidence
gates.
2. Stabilize version identity, packaging, license, security, and release automation.
3. Freeze one clean executable candidate and run the full repository, browser, benchmark,
fresh-wheel, fresh-clone, and secret-scan gates.
4. Close documentation atomically against that candidate and rerun documentation-only validation.
5. Merge and push the final candidate.
6. Rehearse from a fresh anonymous clone at the exact commit.
7. Create and push the annotated `v1.4.0` tag and publish the Forgejo release from the verified
artifacts.
The completed Milestone 4 contract and exclusions remain preserved in `SLICE_HISTORY.md`.
## Release-candidate evidence — 2026-07-29
The executable implementation is frozen at
`d2bb95fe6190e659cf66ba57c78be53b63b53240`. The proof-bearing candidate base is
`2b98059b44f4d46b4d4cce776f163e893c647c76`; it includes exact legacy-tag verification for the
fresh-clone gate and four maintained aggregate derived-recovery tests.
The clean executable release gate passed formatting, Python and web lint, strict Pyright,
compilation, lock and dependency checks, builds, generated documentation, three accessibility
flows, fresh-wheel adoption, artifact reproducibility, secret scans, and the full Milestone 0
through Milestone 4 benchmark sequence. Its exact test evidence was:
- 142 contract tests plus 272 subtests.
- 371 complete tests plus 419 subtests.
- 116 compatibility tests plus 263 subtests.
- 29 concurrency tests plus 2 subtests.
- 68 recovery tests plus 62 subtests.
The real migration gate preserves exact canonical and proposal bytes from the annotated `v1.0.0`
lineage while rebuilding the disposable index from schema 1 to schema 3. The real-package task
gate uses the lock-pinned `markdown-it-py 4.2.0` tree and proves exact graph-assisted and
source-only answers for all reviewed tasks.
The later proof-only recovery commit adds four maintained tests without changing executable
product code. The current recovery aggregate passes 72 tests plus 62 subtests.
The documentation-bearing candidate `49e1a87c138cdc63fb5abb85fc6eb2cf9f4a9d73` passed the complete
local release gate with 378 tests plus 422 subtests, all three accessibility flows, and every
focused and full gate above. Its final documentation-only descendant is the commit identified by
annotated tag `v1.4.0`; that exact remote commit passes the anonymous fresh-clone rehearsal before
tagging.
The public Forgejo release publishes the reproducible wheel, source distribution, and
machine-readable release-identity evidence from the tagged commit. `main`, `dev`, `origin/main`,
and `origin/dev` resolve to that same commit. No PyPI publication was performed.

56
CHANGELOG.md Normal file
View file

@ -0,0 +1,56 @@
# Changelog
Notable changes in the DocForge product line are recorded here. Historical milestone evidence
remains in `docs/MILESTONE_*_BASELINE.md` and `docs/MILESTONE_*_CLOSEOUT.md`.
## Unreleased
## 1.4.0 - 2026-07-29
Version `1.4.0` is the first additive DocForge2 successor release. Annotated tag `v1.4.0`
identifies the exact synchronized documentation-bearing commit that passed the fresh-clone gate.
Added:
- One authoritative version shared by package metadata, Python, CLI, generic MCP, reference MCP,
viewer manager, generated client bindings, and release checks.
- Public adapter SDK and complete graph-plus-Logic conformance checks.
- Base Python and optional JavaScript, TypeScript, and C++ reference adapters.
- Task-shaped retrieval, generation diffs, generated client configuration, doctor checks, and
bounded response continuation.
- Independent manual, portable-graph, and live-viewer projections with immutable packages,
receipts, policy enforcement, and accessibility gates.
- Maintained compatibility, migration, concurrency, recovery, representative-task,
fresh-wheel, artifact-reproducibility, secret-scan, and benchmark gates.
- MIT licensing and Forgejo repository metadata.
Changed:
- Warm graph reads use generation-bound SQLite state without reparsing canonical project sources.
- Incremental adapters use bounded extraction caches while retaining a complete-build equivalence
oracle.
- Generated client configuration is bound to the exact DocForge version.
- Derived publication uses durable atomic replacement and fails closed on malformed, foreign,
stale, oversized, or incompatible state.
- Generic canonical application uses project-owned locking, compare-and-swap publication, and
exact-hash proposal checks while preserving raced source data.
Compatibility:
- The `docforge` distribution and Python package, `docforge` CLI, `docforge-mcp`, MCP tool names,
legacy one-method adapters, generic projects, and descriptor schema version 1 remain supported.
- Version `1.4.0` rebuilds disposable version-1 indexes as version 3 without rewriting canonical
source or active proposal bytes.
- The base wheel has no Tree-sitter dependency. Optional language extras remain explicit.
Release channels:
- The intended release channel is the public Forgejo repository and its release artifacts.
- PyPI publication is not planned because the `docforge` name is occupied by an unrelated
project.
## 1.0.0 - 2026-07-25
The first stable release established the project-scoped graph, generic Markdown/TOML adapter,
SQLite index, project-bound CLI and MCP server, reviewable exact-hash changesets, declared manual
rendering, visualization, and the original versioned compatibility surface.

View file

@ -15,6 +15,74 @@ deferred. Stable user and compatibility contracts still belong in dedicated docu
- DocForge2 does not self-host during this program.
- Release tags and Forgejo releases require Rob's explicit approval.
Milestone 5 was explicitly authorized on 2026-07-29 through the end of the roadmap, including
evidence-based delegation and the milestone release actions frozen in `ACTIVE_SLICE.md`. That
authorization does not widen the project, production-binding, or external-publication boundaries
recorded there.
## Milestone 5 — complete: stabilization and first DocForge2 release
Milestone 5 starts from the clean, merged, and pushed Milestone 4 closeout
`6d06195950d33bcd2d712f8819bbfb3d6652ad03`. The candidate version is `1.4.0`, subject to the full
release contract rather than version work alone.
The release work is evidence-first. Maintained compatibility, migration, recovery, concurrency,
policy, projection, task-comparison, package, clone, browser, benchmark, and secret-scan gates must
pass before documentation closeout, tag creation, or publication. `main` remains the Milestone 4
baseline while implementation proceeds on `dev`.
### Release-candidate freeze — 2026-07-29
The executable implementation froze at
`d2bb95fe6190e659cf66ba57c78be53b63b53240`. It centralizes version `1.4.0`, binds generated
clients to that version, adds MIT license and package metadata, proves reproducible artifacts,
migrates a real archived `v1.0.0` project, maintains comparative real-task evidence, and hardens
derived and canonical publication against measured race and durability failures.
The clean executable release gate passed:
- Ruff formatting and lint, web lint, strict Pyright, compilation, lock, dependency, build,
generated-reference, and documentation checks.
- 142 contract tests plus 272 subtests and 371 complete tests plus 419 subtests.
- Three manual, portable-graph, and live-viewer accessibility flows.
- 116 compatibility tests plus 263 subtests.
- 29 concurrency tests plus 2 subtests.
- Offline fresh-wheel adoption, reproducible artifacts, version checks, and secret scans.
- Every maintained full Milestone 0 through Milestone 4 benchmark.
Commit `97f3b6b` made the fresh-clone rehearsal require and verify the exact annotated legacy tag.
Commit `2b98059b44f4d46b4d4cce776f163e893c647c76` added four maintained recovery proofs without
changing product code. The recovery aggregate now passes 72 tests plus 62 subtests. Those tests
record exact canonical bytes, canonical collection hash, complete snapshot hash, and index
graph-plus-Logic identity before corrupting and repairing the index attestation, manual render
receipt, generation-diff receipt, and portable-graph manifest through normal public work paths.
The real migration evidence uses tag object
`2d7d306a37da89f1c860c7f0be161c45386acf61`, peeled commit
`593c173b453236a6872d0a4e88e7a51a67a21cde`, and a real schema-1 index plus active proposal.
Canonical bytes, the graph snapshot, proposal hash, and proposal file bytes remain exact after the
schema-3 rebuild. The proof retains the inherited version-1 metadata/runtime mismatch: package
metadata is `1.0.0`, while the module and server report `0.15.0`.
The representative real-task track pins `markdown-it-py 4.2.0` at 66 Python files and 225,945
bytes. Both workflows answer all three reviewed questions exactly. Graph-assisted work inspects
349 versus 10,628 bytes, 419 versus 225,945 bytes, and 212 versus 225,945 bytes, with medians of
0.008 versus 1.341 ms, 0.009 versus 32.108 ms, and 0.013 versus 32.142 ms. Source-only final
responses are smaller, so the evidence claims reduced inspected source and faster maintained task
latency rather than a universal response-size advantage.
### Release closeout
Documentation candidate `49e1a87c138cdc63fb5abb85fc6eb2cf9f4a9d73` passed the complete
release gate with 378 tests plus 422 subtests, all three accessibility flows, every focused
Milestone 5 gate, fresh-wheel adoption, reproducible artifacts, both secret scans, and every full
Milestone 0 through Milestone 4 benchmark.
The final documentation-only descendant synchronizes `main` and `dev`, passes the anonymous exact-
commit clone rehearsal, and is identified by annotated tag `v1.4.0`. The public Forgejo release
attaches the reproducible wheel, source distribution, and machine-readable release-identity
evidence. PyPI remains excluded.
## Milestone 0 — complete
Milestone 0 established the public successor, preserved the complete lineage and v1 tag, integrated

21
LICENSE Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Worldforge contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View file

@ -2,10 +2,11 @@ PYTHON := .venv/bin/python
PYRIGHT := pyright
UV := uv
NPM := npm
GITLEAKS := gitleaks
PYTHONPYCACHEPREFIX := /tmp/docforge-quality-pycache
PYTEST_BASETEMP := /tmp/docforge-quality-pytest
.PHONY: accessibility adoption-m4 benchmark benchmark-m1 benchmark-m1-smoke benchmark-m2 benchmark-m2-smoke benchmark-m3 benchmark-m3-full benchmark-m3-smoke benchmark-m4 benchmark-m4-full benchmark-m4-smoke benchmark-smoke build command-reference-check compile contract dependencies docs-check format-check gate lint lock test type
.PHONY: accessibility adoption-m4 benchmark benchmark-full benchmark-m1 benchmark-m1-smoke benchmark-m2 benchmark-m2-smoke benchmark-m3 benchmark-m3-full benchmark-m3-smoke benchmark-m4 benchmark-m4-full benchmark-m4-smoke benchmark-smoke build command-reference-check compatibility-m5 compile concurrency-m5 contract dependencies docs-check format-check fresh-clone-m5 gate lint lock migration-m5 recovery-m5 release-artifacts release-gate release-posttag release-pretag secret-scan task-evidence-m5 task-evidence-m5-smoke test type version-check
accessibility:
$(NPM) run test:accessibility
@ -62,6 +63,74 @@ build:
adoption-m4:
$(PYTHON) tools/milestone4_adoption.py
version-check:
$(PYTHON) tools/check_release_identity.py --mode smoke --tag-state ignore > /dev/null
release-artifacts:
$(PYTHON) tools/check_release_identity.py --mode full --require-clean \
--tag-state ignore --output /tmp/docforge-milestone5-release-identity.json > /dev/null
secret-scan:
$(GITLEAKS) git . --no-banner --redact
$(GITLEAKS) dir . --no-banner --redact
migration-m5:
$(PYTHON) tools/milestone5_migration.py \
--output /tmp/docforge-milestone5-migration.json > /dev/null
task-evidence-m5-smoke:
$(PYTHON) tools/milestone5_task_evidence.py --mode smoke \
--output /tmp/docforge-milestone5-task-evidence-smoke.json > /dev/null
task-evidence-m5:
$(PYTHON) tools/milestone5_task_evidence.py --mode full \
--output /tmp/docforge-milestone5-task-evidence.json > /dev/null
compatibility-m5:
PYTHONPYCACHEPREFIX=$(PYTHONPYCACHEPREFIX) $(PYTHON) -m pytest -q \
-p no:cacheprovider --basetemp=$(PYTEST_BASETEMP)-compatibility \
tests/test_public_contract.py \
tests/test_adapter_contract.py \
tests/test_adapter_sdk.py \
tests/test_policy.py \
tests/test_projection_policy.py \
tests/test_projection_policy_integration.py \
tests/test_projection_contract.py \
tests/test_projection_schemas.py \
tests/test_projection_worker.py \
tests/test_projection_fragments.py \
tests/test_retrieval.py \
tests/test_graph_projection.py \
tests/test_graph_rendering.py \
tests/test_graph_publication.py
concurrency-m5:
PYTHONPYCACHEPREFIX=$(PYTHONPYCACHEPREFIX) $(PYTHON) -m pytest -q \
-p no:cacheprovider --basetemp=$(PYTEST_BASETEMP)-concurrency \
tests/test_core.py::DocForgeCoreTests::test_stale_source_fails_closed_and_failed_rebuild_preserves_index \
tests/test_core.py::DocForgeCoreTests::test_query_rechecks_source_identity_before_returning \
tests/test_core.py::DocForgeCoreTests::test_source_set_change_during_load_fails_closed \
tests/test_adapter_contract.py::AdapterContractTests::test_fast_incremental_reads_reverify_a_changed_index_file \
tests/test_retrieval.py::TaskRetrievalTests::test_final_generation_change_rejects_the_whole_capsule \
tests/test_visualization.py::VisualizationTests::test_snapshot_source_never_mixes_pinned_graph_with_newer_canonical_text \
tests/test_graph_publication.py::GraphPublicationTests::test_status_detects_source_and_publication_races \
tests/test_changesets.py
recovery-m5:
PYTHONPYCACHEPREFIX=$(PYTHONPYCACHEPREFIX) $(PYTHON) -m pytest -q \
-p no:cacheprovider --basetemp=$(PYTEST_BASETEMP)-recovery \
tests/test_milestone5_recovery.py \
tests/test_core.py::DocForgeCoreTests::test_missing_or_corrupt_generation_falls_back_and_repairs \
tests/test_core.py::DocForgeCoreTests::test_stale_source_fails_closed_and_failed_rebuild_preserves_index \
tests/test_incremental_cache.py \
tests/test_python_reference_adapter.py \
tests/test_javascript_reference_adapter.py \
tests/test_cpp_reference_adapter.py \
tests/test_generation_diff.py \
tests/test_rendering.py \
tests/test_graph_publication.py \
tests/test_projection_fragments.py
command-reference-check:
$(PYTHON) tools/generate_command_reference.py \
--output docs/COMMAND_REFERENCE.md --check > /dev/null
@ -109,4 +178,20 @@ benchmark-m4:
benchmark-m4-full: benchmark-m4
benchmark-full: benchmark benchmark-m1 benchmark-m2 benchmark-m3-full benchmark-m4-full
gate: format-check lint type compile contract test accessibility lock dependencies build docs-check benchmark-smoke benchmark-m1-smoke benchmark-m2-smoke benchmark-m3-smoke benchmark-m4-smoke
release-gate: gate compatibility-m5 migration-m5 concurrency-m5 recovery-m5 task-evidence-m5 adoption-m4 version-check release-artifacts secret-scan benchmark-full
fresh-clone-m5:
$(PYTHON) tools/milestone5_fresh_clone.py \
--output /tmp/docforge-milestone5-fresh-clone.json > /dev/null
release-pretag: release-gate fresh-clone-m5
$(PYTHON) tools/check_release_identity.py --mode smoke --require-clean \
--tag-state absent > /dev/null
release-posttag:
$(PYTHON) tools/check_release_identity.py --mode smoke --require-clean \
--tag-state head > /dev/null

View file

@ -9,10 +9,9 @@ DocForge never treats indexed text as instructions. It does not run project buil
compilers, Git operations, deployments, or arbitrary renderers, and it does not select projects
globally.
DocForge 1.0.0 remains the first stable release in this product line. This checkout is the
post-1.0 development line (`1.3.0.dev0`) and contains the Milestone 4 adapter-platform work.
Milestone 4 is not a new product release, and Milestone 5 release and publication work remains
separate.
DocForge 1.4.0 is the first stable DocForge2 successor release. It contains the complete adapter
platform, stabilization and recovery evidence, representative real-task proof, and reproducible
release artifacts. Annotated tag `v1.4.0` identifies the synchronized release commit.
## Start here
@ -23,6 +22,8 @@ separate.
- Agent and client setup: [Agent integration](docs/AGENT_INTEGRATION.md)
- Exact live command and tool inventory: [Generated command reference](docs/COMMAND_REFERENCE.md)
- Task-oriented operating guide: [User manual](docs/USER_MANUAL.md)
- Release changes and evidence: [Changelog](CHANGELOG.md) and [Milestone 5
baseline](docs/MILESTONE_5_BASELINE.md)
## Current capabilities
@ -210,6 +211,8 @@ paths.
### Milestone evidence
- [Milestone 5 baseline](docs/MILESTONE_5_BASELINE.md)
- [Milestone 5 closeout](docs/MILESTONE_5_CLOSEOUT.md)
- [Milestone 4 baseline](docs/MILESTONE_4_BASELINE.md)
- [Milestone 4 closeout](docs/MILESTONE_4_CLOSEOUT.md)
- [Milestone 3 baseline](docs/MILESTONE_3_BASELINE.md) and [closeout](docs/MILESTONE_3_CLOSEOUT.md)
@ -244,6 +247,22 @@ Python reference plus a real isolated read-only MCP retrieval. The optional-lang
have their own focused tests and extras. `benchmark-m4` runs the maintained full adapter workload;
the smoke target is for routine gate coverage, not final performance evidence.
Milestone 5 adds focused and aggregate release gates:
```bash
make compatibility-m5
make migration-m5
make concurrency-m5
make recovery-m5
make task-evidence-m5
make release-gate
make fresh-clone-m5
```
`release-gate` includes the complete repository, accessibility, adoption, artifact-reproducibility,
secret-scan, and full maintained benchmark sequence. `fresh-clone-m5` is the final remote-commit
rehearsal and must pass before the annotated tag and Forgejo release are created.
Pass `--diagnostics` to `docforge` or `docforge-mcp` for bounded request-local timings and compiler
work counters. Diagnostics are disabled by default and do not displace a primary result when the
configured output budget is tight.

View file

@ -1,5 +1,46 @@
# Completed slices
## DocForge2 Milestone 5 stabilization and first release
### Changed
- Centralized version `1.4.0` across package metadata, Python, executable surfaces, generated
clients, and release validation.
- Added MIT licensing, public Forgejo metadata, reproducible artifact checks, a real archived-v1
migration proof, focused compatibility/concurrency/recovery gates, and representative
graph-assisted versus source-only task evidence.
- Hardened durable derived publication and generic exact-hash canonical application against
measured race windows.
- Added a fresh-clone rehearsal that requires the exact annotated legacy tag.
### Verification
- The documentation-bearing candidate passed 378 tests and 422 subtests, 142 contract tests and
272 subtests, three accessibility flows, lint, strict types, compilation, dependencies, builds,
adoption, reproducible artifacts, secret scans, and every maintained full benchmark.
- Focused compatibility passed 116 tests plus 263 subtests. Concurrency passed 29 tests plus 2
subtests.
- Four later proof-only tests brought the recovery gate to 72 tests plus 62 subtests without
changing executable product code.
- Real `v1.0.0` migration preserved canonical, graph, and proposal evidence while rebuilding the
disposable index from schema 1 to schema 3.
- The lock-pinned real-package comparison returned exact answers from both workflows and measured
substantially less inspected source for graph-assisted work.
### Limits
- Multi-file canonical application does not claim process-death atomicity.
- Source-only task responses were smaller even where graph-assisted work inspected fewer bytes and
ran faster.
- No WorldForge, ScrapeStation, legacy repository, production binding, self-hosting, or PyPI
publication was changed.
### Release
The exact synchronized release commit passes the fresh anonymous clone gate and is identified by
annotated tag `v1.4.0`. Its public Forgejo release carries the reproducible wheel, source
distribution, and machine-readable release-identity evidence.
## DocForge2 Milestone 4 adapter SDK and product documentation
### Changed

View file

@ -1,8 +1,9 @@
# DocForge2 Milestone 0 compatibility contract
# DocForge compatibility contract
Milestone 0 establishes DocForge2 as the successor repository without renaming or replacing the
working DocForge interfaces. Compatibility changes require an explicit decision, a contract-test
update, and migration guidance.
update, and migration guidance. DocForge 1.4.0 preserves that baseline and adds the adapter,
rendering, recovery, and release surfaces recorded below.
The compatibility gate is:
@ -16,6 +17,11 @@ The complete repository gate is:
make gate
```
Milestone 5 also maintains `make compatibility-m5` for the frozen public, adapter, policy,
projection, rendering, and retrieval matrix. `make release-gate` aggregates that matrix with
migration, concurrency, recovery, task-evidence, adoption, version, artifact, secret-scan, browser,
and benchmark proofs.
## Distribution and Python imports
The Python distribution and import package remain `docforge`.
@ -63,6 +69,12 @@ names and arguments remain supported. Additive commands, tools, and response fie
Removing or changing an existing name, required argument, stable error code, or safety boundary
requires an explicit compatibility decision.
Version `1.4.0` comes from one `docforge._version` authority. The four maintained executable
surfaces report `docforge 1.4.0`, `docforge-mcp 1.4.0`,
`python -m docforge.reference_mcp 1.4.0`, and `docforge-viewer-manager 1.4.0` for `--version`.
Generated generic and adapter client configurations include and hash-bind the same
`docforge_version`.
MCP results retain:
- A structured `status`.
@ -191,6 +203,14 @@ The following guarantees remain stable:
6. Canonical application is absent unless one startup-bound applier is configured.
7. Derived refresh failures produce an explicit degraded receipt after canonical application. They
do not make an applied proposal safe to apply twice.
8. Generic canonical publication compares exact target identity at the commit boundary. Concurrent
create, update, and delete mutations fail closed, roll back when exact state remains provable, or
retain recovery evidence without overwriting foreign data.
9. Per-file publication is atomic and in-process rollback covers earlier publications, but
canonical application has no process-death journal and does not promise multi-file crash
atomicity.
10. Cleanup degradation after semantic commit closes the proposal as `applied` and persists bounded
`application_recovery` lifecycle metadata instead of returning a retryable ordinary failure.
## Rendering and visualization
@ -260,6 +280,15 @@ symbolic-link escapes. Documentation text remains untrusted data. Normal MCP ope
arbitrary filesystem access, renderer execution, shell command, Git mutation, deployment,
publication, or project switching.
DocForge is not a filesystem sandbox. Mode-0700 canonical transaction directories protect against
other users and ordinary path access; deliberate arbitrary tampering by another process with the
same operating-system UID is outside the compatibility boundary.
The historical `v1.0.0` release carried distribution metadata `1.0.0` while its module and MCP
runtime reported `0.15.0`. Version 1.4.0 records that inherited mismatch in its maintained
migration proof and resolves current identity through one authority. See
[migrating from v1](MIGRATING_FROM_V1.md).
## Recorded weaknesses, not compatibility promises
Milestone 0 records rather than redesigns these areas:
@ -268,7 +297,6 @@ Milestone 0 records rather than redesigns these areas:
- The base wheel intentionally omits Tree-sitter. JavaScript, TypeScript, and C++ syntax evidence
requires the matching `docforge[javascript]`, `docforge[typescript]`, or `docforge[cpp]` extra.
Python reference evidence uses the standard library and remains available in the base wheel.
- Several version strings and defaults remain duplicated.
- One individually oversized context entry is represented as explicit bounded omission evidence;
callers use targeted retrieval for that node.
- One individually oversized changeset diff is transported as reconstructable canonical-JSON

View file

@ -1,4 +1,4 @@
# DocForge post-1.0 development contract
# DocForge 1.4 contract
## Authority boundary
@ -34,13 +34,19 @@ commit when Git is available; it cannot change repository state.
- Reference adapter configuration: `schemas/reference-adapter.schema.json`, version 1.
- Index schema: version 3, disposable and reproducible.
- Index attestation: schema version 1, disposable and reproducible.
- Core, CLI, and MCP server: version 1.3.0.dev0.
- Distribution, Python package, CLI, generic MCP, reference MCP, and viewer manager: version 1.4.0.
- Incremental extraction cache: version 1, disposable and reproducible.
Schema files describe the generic interchange contract. Runtime validation remains responsible for
path confinement, source hashing, relationship resolution, dependency cycles, project limits, stale
state, and adapter-specific rules that JSON Schema cannot prove by itself.
`src/docforge/_version.py` is the sole package-version authority. The maintained executable
surfaces report exactly `docforge 1.4.0`, `docforge-mcp 1.4.0`,
`python -m docforge.reference_mcp 1.4.0`, and `docforge-viewer-manager 1.4.0` for `--version`.
Generated generic and adapter client configurations bind `docforge_version` into their validated
hashes.
## Generic node storage
Markdown nodes begin with a TOML metadata block delimited by `+++`. The remaining Markdown is the
@ -73,6 +79,13 @@ Index replacement is the derived publication commit point. Attestation, cheap so
and latest-generation-diff receipts are independent post-commit evidence. Their failure produces
bounded degraded success and never falsely reports that a committed index mutation failed.
Derived publication stages and flushes one complete bounded artifact before atomic replacement or
no-clobber publication, then flushes the containing directory. An interruption therefore leaves
the preceding complete artifact, the new complete artifact, or explicit degraded post-commit
evidence. Index attestations, manual render receipts, generation-diff baselines, and portable-graph
manifests have maintained exact-oracle corruption-and-repair proofs. Canonical project files remain
unchanged throughout those recoveries.
Before replacement, a build accepts a predecessor only when its exact main-file inode has a
matching whole-file attestation, has no WAL, journal, or shared-memory sidecar, and passes the
published SQLite identity, row, hash, FTS, integrity, and policy checks. It uses an immutable
@ -174,6 +187,30 @@ default active listing contains only draft and ready work. Stale, applied, and a
remain queryable by explicit status or history request. Terminal proposals do not block new
proposals.
## Canonical application durability
Generic canonical application stages replacements and backups in a mode-0700
`.docforge/application/transaction-*` directory. Before each canonical create, update, or delete,
it compares exact file identity at the publication boundary. Creates use no-clobber publication.
Updates and deletes use atomic exchange and no-replace detachment. A concurrent canonical-target
mutation fails closed, is rolled back only when exact displaced state remains provable, or is
retained without overwriting foreign data.
This is per-file compare-and-swap publication, not multi-file crash atomicity. In-process failures
run rollback across already published files, but there is no process-death journal. Process or host
death between publications may leave a partial canonical application and requires operator
inspection before a new proposal or restoration.
The private transaction namespace is integrity-confined against ordinary path access. It is not a
filesystem sandbox, and deliberate arbitrary tampering by another process with the same operating-
system UID is outside the contract. DocForge identity-checks private files before consuming or
removing them.
After the serializer reproduces the approved graph, canonical success is final. If private cleanup
then degrades, application still returns `applied`, closes the proposal, and persists bounded
`application_recovery` lifecycle metadata with status `cleanup_required`, retained paths, and
remediation. The reviewed changeset must not be applied twice.
## Declared rendering and previews
Render configuration is optional. A configured project declares one template root, one isolated

View file

@ -19,6 +19,36 @@ Read [compatibility](COMPATIBILITY.md), [legacy and no-AST operation](LEGACY_AND
Disposable index and cache schemas may change. Rebuild them rather than copying them as authority.
## Maintained v1.0.0 migration evidence
`make migration-m5` archives and executes the actual annotated `v1.0.0` release, then opens its
fixture, index, and active proposal through DocForge 1.4.0. The frozen tag object is
`2d7d306a37da89f1c860c7f0be161c45386acf61`; it identifies commit
`593c173b453236a6872d0a4e88e7a51a67a21cde`.
The tagged release contains an inherited identity mismatch that the migration proof records
rather than hiding: distribution metadata says `1.0.0`, while `docforge.__version__` and the MCP
server report `0.15.0`. DocForge 1.4.0 replaces that duplicated state with one authoritative
version and requires its package and server values to agree.
The maintained fixture evidence is exact:
- Canonical collection hash:
`9fde91b6b08669177d690cdf9f91b162120baee1f7e24b05fec67f56617f286a`.
- Graph snapshot hash:
`45bef8b0e1a4dac976e096dcf8f9048e1268211cd7f7638cdf99951428ec0500`.
- Source hash: `0aa6ad13a95355102300a69b2f9d06883c301d63e2624b083e45f15102dab504`.
- Active proposal hash:
`2c055dfae45443b4a4d9d4087ef70959e7293beae8d27acb14ffffe52eca7111`.
- Proposal-file hash:
`4658d494b43bd7c6cc3e3f5933a2c878c817b52bc4566e9e8429e7b8e43ca007`.
Current loading preserves all five identities and every canonical byte. It rebuilds the disposable
index from schema 1 to schema 3 without changing the graph or active proposal. All 20 tagged CLI
commands remain in the current 28-command surface, and all 25 tagged MCP tools remain in the
current 36-tool application-enabled surface. These counts describe the maintained rehearsal, not
a promise that every additive current command belongs in a legacy binding.
## Recommended migration
1. Record the v1 package version, adapter identity, project descriptor, canonical source hash,

View file

@ -0,0 +1,189 @@
# Milestone 5 baseline
## Status and method
This baseline records the frozen DocForge `1.4.0` release lineage and the evidence used to publish
annotated tag `v1.4.0`.
The executable implementation freeze is
`d2bb95fe6190e659cf66ba57c78be53b63b53240`. Fresh-clone legacy-tag verification was added in
`97f3b6b1ae303c972387508557a3d54ea621a702`, and the maintained aggregate recovery proofs were
completed in
`2b98059b44f4d46b4d4cce776f163e893c647c76`. Documentation-bearing candidate
`49e1a87c138cdc63fb5abb85fc6eb2cf9f4a9d73` passed the complete clean local release gate. The
annotated tag points to its final documentation-only descendant after the exact fresh-clone gate.
The clean executable candidate used:
- Linux 7.1.3-200.nobara.fc44.x86_64 with glibc 2.43.
- CPython 3.14.6 on x86_64.
- The repository lockfile and offline wheel inputs.
- Maintained repository-native gates rather than a self-hosted DocForge development loop.
No WorldForge, ScrapeStation, legacy repository, production binding, or production MCP
configuration was changed.
## Quality and compatibility gates
The clean executable candidate passed:
- Ruff formatting and lint, web lint, strict Pyright with zero diagnostics, and compilation.
- 142 contract tests plus 272 subtests.
- 371 complete tests plus 419 subtests.
- Three Playwright and axe accessibility flows covering the manual, portable graph, and live
viewer.
- Lockfile, dependency-tree, wheel, source-distribution, generated-reference, and documentation
checks.
- 116 compatibility tests plus 263 subtests.
- 29 concurrency tests plus 2 subtests.
- Offline fresh-wheel adoption, exact version identity, reproducible artifact, and secret-scan
gates.
- The complete maintained Milestone 0 through Milestone 4 benchmark sequence.
The later maintained recovery proof adds four tests without changing executable product code. Its
aggregate recovery gate passes 72 tests plus 62 subtests. The final clean clone reruns the complete
totals from the documentation-bearing commit before tagging.
Compatibility remains additive:
- The `docforge` distribution and imports, CLI, generic MCP executable, MCP tool names, generic
project descriptor, result envelopes, legacy one-method adapter, and no-AST behavior remain.
- Effective policy version 1, projection policy version 2, descriptor schema version 1, and index
schema version 3 remain the current authorities.
- Complete and incremental maintained adapters produce exact primary-graph and Logic equivalence.
## Version and artifact identity
One source file owns version `1.4.0`. The following executable surfaces reported that exact
version:
- `docforge`
- `docforge-mcp`
- `docforge-viewer-manager`
- `python -m docforge.reference_mcp`
The package metadata also reported `docforge 1.4.0`, Python 3.12 or newer, the MIT license
expression, the public Forgejo repository, and its issues URL. Generated client bindings include
the version in their validated, hash-bound identity.
Two independent builds from the executable candidate and the same source-date epoch produced
identical artifacts:
| Artifact | Bytes | Executable-candidate SHA-256 |
|---|---:|---|
| Wheel | 315,811 | `c3b9bfa320d00d827154e0f858a6b970459ae0eba6ab14e1cd4f6ba470e33b6a` |
| Source distribution | 685,891 | `bb5f9333e5fa2365cf0f8ac12d7920122315b4835c9f81b7aa457eb05f1702f5` |
Both contained the license and version authority. These identify the executable freeze only.
Documentation changes alter the final release artifacts, so final checksums must be generated from
the exact tagged documentation-bearing commit.
The release channel is Forgejo only. PyPI is excluded because the `docforge` name is occupied by an
unrelated project.
## Exact migration evidence
The migration gate archives and executes the real annotated `v1.0.0` lineage:
- Tag object: `2d7d306a37da89f1c860c7f0be161c45386acf61`.
- Peeled commit: `593c173b453236a6872d0a4e88e7a51a67a21cde`.
- Canonical byte hash before and after:
`9fde91b6b08669177d690cdf9f91b162120baee1f7e24b05fec67f56617f286a`.
- Snapshot hash before and after:
`45bef8b0e1a4dac976e096dcf8f9048e1268211cd7f7638cdf99951428ec0500`.
- Active proposal hash:
`2c055dfae45443b4a4d9d4087ef70959e7293beae8d27acb14ffffe52eca7111`.
- Active proposal file hash:
`4658d494b43bd7c6cc3e3f5933a2c878c817b52bc4566e9e8429e7b8e43ca007`.
The version-1 disposable index rebuilt from schema 1 to schema 3. Canonical bytes, graph meaning,
and the active proposal remained exact. The current CLI is a 28-command superset of the version-1
20-command surface. The current generic MCP surface is a 36-tool superset of the version-1
25-tool surface.
The proof also preserves a real inherited version-1 inconsistency: package metadata reports
`1.0.0`, while its Python module and server report `0.15.0`. Migration evidence records that fact;
it does not rewrite history to make the old identities agree.
## Recovery and concurrency evidence
The maintained gates prove:
- Concurrent canonical-source changes fail closed without publishing or serving mixed
generations.
- Exact-hash canonical application uses project-owned serialization and compare-and-swap
publication, rejects stale proposals, and preserves raced source data.
- Extraction caches and SQLite indexes rebuild from canonical sources or complete adapter
projections.
- Corrupt index attestations, manual render receipts, generation-diff receipts, and portable-graph
manifests recover through their normal synchronize, rebuild, or render entry points.
- Recovery preserves exact canonical bytes, the canonical collection hash, the complete snapshot
hash, and primary-graph-plus-Logic index identity.
- Derived publication uses durable atomic replacement and directory synchronization.
The release does not claim that a multi-file canonical application is process-death atomic. A
malicious same-UID process deliberately modifying the private mode-0700 transaction directory is
also outside the application race contract. Those limits do not weaken the maintained
concurrent-canonical-source compare-and-swap proof.
## Representative real-task evidence
The real-package track uses installed, lock-pinned `markdown-it-py 4.2.0`:
- 66 Python files.
- 225,945 source bytes.
- Source-tree SHA-256:
`bd57c9f332fcf6507282ec2023e6804fce0cf844631696336ee17cbe46e63aad`.
- No network, production binding, installed-source mutation, or self-hosting.
- Python reference-adapter local-import projection for graph-assisted work.
- Independent standard-library AST import inspection for source-only work.
Both workflows returned the exact reviewed answer for all three tasks:
| Task | Graph inspected | Source inspected | Graph median | Source median |
|---|---:|---:|---:|---:|
| Renderer direct dependencies | 349 B | 10,628 B | 0.008 ms | 1.341 ms |
| Three-level HTML-block impact | 419 B | 225,945 B | 0.009 ms | 32.108 ms |
| CLI-to-core-state path | 212 B | 225,945 B | 0.013 ms | 32.142 ms |
Per-task latency excludes the separately reported one-time graph preparation. Graph-assisted
answers inspected substantially fewer agent-visible bytes and carried generation-bound
provenance. The source-only answers were smaller on the final response-size metric. The comparison
therefore records the measured tradeoff rather than claiming that every metric favors the graph.
The real-package semantic evidence SHA-256 is
`c8c19b2900fa47abdaff94f5b13fd9ca537241089edc97b30d32d9db8809a2c1`.
The combined real and generated-track evidence SHA-256 is
`45952883b47449eb4fd862b51854aa2001749928e0d37c5a6e58c2d3292fbee4`.
## Machine-readable evidence
The maintained evidence generators emit canonical compact JSON and can write it atomically with
their `--output` option:
```bash
.venv/bin/python tools/milestone5_migration.py --output /tmp/docforge-m5-migration.json
.venv/bin/python tools/milestone5_task_evidence.py \
--mode full --output /tmp/docforge-m5-task-evidence.json
.venv/bin/python tools/check_release_identity.py \
--mode full --require-clean --tag-state absent \
--output /tmp/docforge-m5-release-identity.json
.venv/bin/python tools/milestone5_fresh_clone.py \
--commit "$(git rev-parse HEAD)" \
--output /tmp/docforge-m5-fresh-clone.json
```
Migration, task, release identity, artifact hashes, and the final anonymous-clone result therefore
remain reproducible machine evidence rather than prose-only claims. The final Forgejo release
attaches the release-identity JSON beside the wheel and source distribution.
## Final release evidence
The documentation-bearing candidate passed 378 tests plus 422 subtests and all three accessibility
flows. The final documentation-only descendant is synchronized across `main`, `dev`,
`origin/main`, and `origin/dev`, then passes the maintained anonymous fresh-clone rehearsal at that
exact commit. Annotated tag `v1.4.0` and the Forgejo release identify that commit.
Final artifact checksums are generated from the tagged source and recorded in the attached
machine-readable release-identity evidence. The release includes the wheel and source
distribution. It does not publish to PyPI.

View file

@ -0,0 +1,97 @@
# Milestone 5 closeout
## Outcome
DocForge `1.4.0` completes the first DocForge2 successor release. Compatibility, migration,
determinism, concurrency, recovery, policy, projection isolation, accessibility, performance,
adoption, artifact reproducibility, and representative real-task evidence are maintained and
passing.
The documentation-bearing candidate passed its complete local release gate. Its final
documentation-only descendant passes the anonymous exact-commit clone gate and is identified by
the annotated `v1.4.0` tag and public Forgejo release.
## Candidate lineage
- Merged Milestone 4 baseline:
`6d06195950d33bcd2d712f8819bbfb3d6652ad03`.
- Frozen executable implementation:
`d2bb95fe6190e659cf66ba57c78be53b63b53240`.
- Fresh-clone legacy-tag verification:
`97f3b6b1ae303c972387508557a3d54ea621a702`.
- Maintained aggregate recovery proof:
`2b98059b44f4d46b4d4cce776f163e893c647c76`.
- Release version and tag: `1.4.0` and `v1.4.0`.
- Publication channel: public Forgejo release only.
The tag resolves to the final documentation-bearing descendant of this lineage, never to the
earlier executable-only commit.
## Closed release evidence
The clean executable release gate passed:
- Formatting, Python and web lint, strict types, compilation, lock, dependency, package-build,
generated-reference, and documentation checks.
- 142 contract tests plus 272 subtests.
- 378 complete tests plus 422 subtests.
- Three interactive accessibility flows.
- 116 compatibility tests plus 263 subtests.
- 29 concurrency tests plus 2 subtests.
- Offline fresh-wheel adoption.
- Reproducible wheel and source-distribution builds with exact version and MIT license identity.
- Gitleaks scans of reachable history and the candidate directory with no findings.
- Full maintained Milestone 0, 1, 2, 3, and 4 benchmarks.
The proof-only recovery commit adds four maintained tests. The aggregate recovery gate passes 72
tests plus 62 subtests.
Detailed migration identities, artifact evidence, recovery boundaries, and task measurements are
in the [Milestone 5 baseline](MILESTONE_5_BASELINE.md).
## Compatibility and migration result
Version `1.4.0` preserves the established distribution, imports, CLI, MCP, schema-1 descriptor,
generic-project, one-method adapter, exact-hash changeset, rendering, result-envelope, and no-AST
surfaces. New adapter, retrieval, projection, and release capabilities are additive.
The real annotated `v1.0.0` archive migrates without canonical or proposal changes. Its schema-1
index rebuilds as schema 3, and its graph identity remains exact. The current CLI and MCP
registrations are supersets of the version-1 surfaces. The proof reports the inherited version-1
metadata/runtime mismatch instead of hiding it.
## Representative task result
The pinned real-package comparison uses `markdown-it-py 4.2.0`, 66 Python files, and 225,945 bytes.
Graph-assisted and source-only workflows both return exact reviewed answers for direct
dependencies, bounded reverse impact, and a dependency path.
Graph-assisted medians were 0.008, 0.009, and 0.013 ms after one-time preparation, versus 1.341,
32.108, and 32.142 ms for source-only inspection. Graph-assisted work also reduced inspected bytes
from 10,628 to 349, from 225,945 to 419, and from 225,945 to 212. Source-only final responses were
smaller, so no universal response-size advantage is claimed.
## Preserved boundaries
- Canonical project files remain authoritative. Indexes, caches, receipts, render output, graph
output, client fragments, and viewer state remain disposable.
- Full rebuild remains the recovery and equivalence oracle.
- Derived publication is durable and atomic. Multi-file canonical application does not claim
process-death atomicity.
- `--no-ast` remains a restrictive binding policy, not a parser detector or filesystem sandbox.
- Reference adapters publish narrow static evidence and do not claim resolved calls, types,
inheritance, runtime behavior, compiler semantics, or semantic ownership.
- Reference MCP remains read-only.
- No WorldForge, ScrapeStation, legacy repository, production binding, storage, or self-hosting
change belongs to this release.
- No PyPI publication belongs to this release.
## Publication
`main`, `dev`, `origin/main`, and `origin/dev` resolve to the exact documentation-bearing release
commit. An anonymous HTTPS clone of that commit verifies the frozen annotated `v1.0.0` migration
tag and repeats the complete release gate before `v1.4.0` is created.
The public Forgejo release attaches the reproducible wheel, source distribution, and
machine-readable release-identity evidence containing their exact SHA-256 checksums. No PyPI
publication, production binding change, or legacy-repository mutation is part of this release.

View file

@ -41,14 +41,42 @@ reported as degraded committed success, not as permission to repeat a mutation.
The live viewer pins one validated index identity. Index replacement makes the running snapshot
stale and causes a later visualize request to start a fresh worker.
Milestone 5 maintains exact-oracle recovery proofs for four independently disposable artifacts:
- A corrupt index attestation is rejected, then `synchronize()` recreates the exact attestation
after complete verification while preserving canonical bytes, snapshot hash, and index identity.
- A corrupt manual-render receipt reports `unverified/receipt_corrupt`; explicit rendering restores
the exact output bytes and semantic receipt, and normal and deep status return `current`.
- A corrupt generation-diff receipt reports `unverified/corrupt_receipt`; a complete index build
recreates the exact current-graph baseline with `baseline/no_meaningful_transition`.
- A corrupt portable-graph manifest reports missing publication evidence; explicit rendering
restores the exact artifact bytes and semantic manifest, and status returns `current`.
These proofs are maintained by `tests/test_milestone5_recovery.py`. They do not promote receipts,
attestations, or manifests to canonical authority.
## Proposal and application recovery
Hash or base conflicts are not cache failures. Retrieve the current changeset and diff, then
review the new exact hash. Rebase is allowed only when every touched node, relationship, source,
permission, and graph invariant still matches. A content conflict requires a new proposal.
If a canonical serializer fails its round-trip check, use its reported rollback state. Do not
reapply a changeset whose application may already have committed. See
Generic canonical create, update, and delete publication compares exact target identity at the
commit boundary. Concurrent target mutation fails closed. In-process failures roll back only when
the exact displaced state remains provable; otherwise DocForge preserves foreign data and returns
`application_recovery_required` with retained evidence.
Canonical application is not journaled across several files. Each file publication is atomic, but
process or host death between publications can leave a partial canonical application. Inspect the
named canonical targets, the active proposal, and `.docforge/application/transaction-*` before
deciding whether to restore or create a new proposal. Do not retry the old approved hash merely
because a process ended.
If semantic application committed but private transaction cleanup degraded, the result remains
`applied`. The proposal is closed and its compact lifecycle receipt records
`application_recovery.status = "cleanup_required"`, retained paths, and remediation. Preserve and
inspect those files. Remove only artifacts proven to be DocForge-owned. If a canonical serializer
fails its round-trip check before success, use its reported rollback state. See
[migrating from v1](MIGRATING_FROM_V1.md) for rollback planning.
## Milestone 4 scale evidence
@ -80,7 +108,18 @@ make gate
make adoption-m4
make benchmark-m4-full
make docs-check
make compatibility-m5
make migration-m5
make concurrency-m5
make recovery-m5
make task-evidence-m5
make release-gate
make fresh-clone-m5
```
The main gate includes smoke benchmarks. Full milestone evidence is recorded separately from a
clean candidate so smoke or dirty-tree results cannot become release claims.
clean candidate so smoke or dirty-tree results cannot become release claims. `release-gate`
aggregates the complete quality, compatibility, migration, concurrency, recovery, task-evidence,
fresh-wheel, identity, reproducible-artifact, secret-scan, and full benchmark proofs.
`fresh-clone-m5` repeats that aggregate gate from anonymous HTTPS at one exact published commit
after verifying the frozen annotated `v1.0.0` migration tag.

View file

@ -1,7 +1,8 @@
# Security model
DocForge is a project-bound knowledge compiler. Its security boundary is an explicit project root,
closed configuration, bounded data, and exact identities. It is not a general process sandbox.
closed configuration, bounded data, and exact identities. It is not a general process or
filesystem sandbox.
Start with [core authority](CORE_CONCEPTS_AND_AUTHORITY.md), then use
[policy precedence](POLICY_PRECEDENCE.md) to decide which capabilities a server should expose.
@ -18,6 +19,13 @@ Confinement protects DocForge operations. It does not stop another process with
from changing files. Long-running bindings revalidate descriptor and adapter implementation
identity and require a restart after drift.
Generic canonical application stages backups and replacements below
`.docforge/application/transaction-*` in mode-0700 directories. That private namespace confines
ordinary path access and prevents access by other users. Deliberate arbitrary tampering by another
process running as the same operating-system user is outside this boundary. DocForge still
identity-checks private files before using or removing them, but mode `0700` is not isolation from
the same UID.
## Untrusted project content
Documentation, source text, templates, adapter metadata, compiler-database entries, and changeset
@ -56,16 +64,33 @@ the exact changeset hash that was reviewed. Source identity, content hashes, per
conflicts, graph validity, and serializer round trips are checked before success. DocForge never
turns prose approval into a fuzzy merge.
The generic applier compares exact canonical file identity immediately before each publication.
Create uses no-clobber publication. Update and delete use atomic exchange and no-replace moves.
Concurrent canonical-target mutation therefore fails closed, rolls back when the exact displaced
state is still provable, or retains recovery evidence without overwriting foreign data.
This compare-and-swap protection is not a process-death journal. One file publication is atomic,
and an in-process failure runs exact rollback, but an application spanning several canonical files
does not promise crash atomicity if the process or host dies between publications. Operators must
inspect canonical state and retained transaction evidence before retrying after such an
interruption.
## Derived state and publication
SQLite indexes, source-generation receipts, extraction caches, render fragments, previews, and
portable artifacts are disposable. Corrupt, stale, foreign, oversized, or mismatched derived
state is rejected or rebuilt from current project evidence.
Generated command-reference publication serializes cooperating writers and uses no-clobber or
compare-and-swap publication. A raced target is restored or retained for recovery instead of being
silently discarded. Projection publication records when an artifact was committed but later
receipt verification degraded, so a completed mutation is never reported as an ordinary failure.
Derived publication stages complete bounded output, flushes file and directory state, and commits
with atomic replacement or no-clobber compare-and-swap. It is crash-safe: an interruption leaves
the prior verified artifact, the complete new artifact, or explicit degraded post-commit evidence,
not a mixed publication. Generated command-reference publication also serializes cooperating
writers. A raced target is restored or retained for recovery instead of being silently discarded.
Projection and canonical-application lifecycles record when semantic content committed but later
private cleanup or receipt verification degraded. Canonical success closes the applied proposal
and persists compact `application_recovery` metadata with retained paths and remediation. A
completed mutation is never reported as an ordinary retryable failure.
## Limits and denial-of-service resistance

View file

@ -5,9 +5,11 @@ people and AI agents can search, inspect, visualize, and change through reviewab
Canonical project files remain authoritative. The SQLite graph, previews, rendered manuals, and
viewer processes are derived and can be rebuilt.
DocForge 1.0.0 is the first stable product release. It includes the project-scoped graph,
CLI and MCP query surfaces, hash-approved proposal application, generic and project-owned adapters,
declared rendering, and the Nodes/Flow/Web visualization model documented below.
This manual describes the DocForge 1.4.0 release. The tagged `v1.0.0` baseline was the first stable
product release. Version 1.4.0 preserves its project-scoped graph, CLI and MCP query
surfaces, hash-approved proposal application, generic and project-owned adapters, declared
rendering, and Nodes/Flow/Web model while adding the maintained incremental, projection, adapter
SDK, recovery, and release proofs documented below.
Later incremental-compiler capabilities are additive. A Release 1 adapter with only
`load_projection()` remains valid and follows the same complete-rebuild path. No existing project
@ -76,6 +78,12 @@ The generic adapter can serialize its Markdown and TOML nodes directly. A custom
provide its own canonical applier because only that project knows how a graph node maps back to its
source format.
Generic canonical application compare-and-swaps each target against its exact expected identity.
A concurrent create, update, or delete fails closed, rolls back when the exact displaced state is
still provable, or preserves recovery evidence without overwriting foreign data. Per-file
publication is atomic, but an application spanning several canonical files has no process-death
journal and does not claim crash atomicity across the group.
## Setup
### Requirements
@ -115,6 +123,20 @@ The base wheel contains the Python reference adapter and no Tree-sitter distribu
TypeScript, and C++ require their matching optional extras. `docforge[languages]` installs all
three optional frontend groups.
Verify the four executable surfaces from the exact installed environment:
```bash
python -m docforge.cli --version
python -m docforge.mcp_server --version
python -m docforge.reference_mcp --version
python -m docforge.viewer_manager --version
```
For version 1.4.0 these report `docforge 1.4.0`, `docforge-mcp 1.4.0`,
`python -m docforge.reference_mcp 1.4.0`, and `docforge-viewer-manager 1.4.0`. Package metadata,
Python imports, generated generic and adapter configurations, and these commands share the same
version authority.
### Configure a reference source project
Reference adapters are a narrow alternative to the generic documentation descriptor. Create
@ -996,6 +1018,10 @@ effective policy. A replacement receipt returns `stale_cursor`; restart from its
Canonical application records its terminal receipt immediately after the project-owned serializer
verifies the new canonical state. A later index or render refresh failure is reported as degraded
derived state with remediation, not as permission to apply the same canonical change again.
Likewise, failure to remove a private transaction artifact after semantic commit returns
`applied`, closes the proposal, and persists compact `application_recovery` lifecycle metadata
with `cleanup_required`, retained paths, and remediation. Inspect and remove only files proven to
be DocForge-owned.
Every successful declared render publishes a bounded version-1 receipt below the disposable cache.
Normal `render-status` compares cheap source-generation, view-configuration, template-file, and
@ -1115,6 +1141,13 @@ process verifies the unchanged database against that receipt instead of reconstr
row. Missing or mismatched receipts fall back to complete verification and are recreated only after
the full check succeeds.
Milestone 5 maintains exact recovery for four corrupt derived artifacts. Synchronization restores
a corrupt index attestation after complete verification. Explicit `render` restores a corrupt
manual receipt to the exact output and receipt semantics. A complete `reindex` recreates a corrupt
generation-diff baseline against the exact current graph. Explicit `graph-render` recreates a
corrupt portable-graph manifest and exact artifact. Status operations diagnose these conditions
without hidden repair.
### `visualization_manager_unavailable`
The per-user manager is not installed, is stopped, or points to an old virtual environment.
@ -1159,9 +1192,28 @@ content or relationship conflict remains fail-closed and requires a newly review
### `application_mismatch`
The written sources did not reproduce the validated projection. DocForge rolls the generic
canonical files back. For a custom adapter, fix its serializer or node-to-source mapping before
retrying.
The written sources did not reproduce the validated projection. During an ordinary in-process
failure, DocForge rolls generic canonical files back when their exact publication identities are
still provable. If another process raced a target, DocForge preserves foreign and displaced data
and returns `application_recovery_required` rather than overwriting either. For a custom adapter,
fix its serializer or node-to-source mapping before retrying.
### `application_recovery_required` or `cleanup_required`
`application_recovery_required` means canonical publication or rollback encountered concurrent or
unprovable state. Preserve every retained file named in the error. Compare it with the canonical
target and resolve the project before creating a newly reviewed proposal. Do not retry the old
approved hash.
`cleanup_required` means semantic application already committed. The proposal is closed as
`applied`, and its lifecycle receipt names private transaction artifacts that could not be removed.
Inspect those files and remove only confirmed DocForge-owned artifacts. The canonical change must
not be applied again.
Generic application uses mode-0700 transaction directories, but DocForge is not a filesystem
sandbox. Deliberate arbitrary tampering by another process running as the same operating-system
user is outside that integrity boundary. A process or host death can also interrupt a multi-file
application because canonical application has no process-death journal.
### `path_escape`, `unsafe_template`, or missing source
@ -1178,7 +1230,7 @@ ambiguous adapter evidence.
### Full inspector content does not fit
DocForge 1.0 uses a fixed header and footer with a scrollable inspector body. If an older page is
DocForge 1.4 uses a fixed header and footer with a scrollable inspector body. If an older page is
still open, stop and reopen the visualization so it loads the current `graph-browser@17` template.
### Render output is stale
@ -1219,7 +1271,7 @@ the process so it binds the new descriptor deliberately.
## Development and verification
Run the complete release gate from the DocForge repository:
Run the ordinary repository gate:
```bash
make gate
@ -1234,5 +1286,25 @@ and recovery benchmark. `make adoption-m4` performs the offline fresh-wheel proo
the maintained documentation graph. `make accessibility` runs the generated manual, portable
graph, and live viewer axe and keyboard flows.
Milestone 5 adds maintained compatibility, migration, concurrency, recovery, comparative-task,
release-identity, reproducible-artifact, secret-scan, and fresh-clone gates:
```bash
make compatibility-m5
make migration-m5
make concurrency-m5
make recovery-m5
make task-evidence-m5
make release-gate
make fresh-clone-m5
```
`release-gate` aggregates the full quality, browser, compatibility, migration, concurrency,
recovery, task-evidence, fresh-wheel, version, artifact, secret-scan, and benchmark suite.
`fresh-clone-m5` anonymously clones the exact published candidate over HTTPS, fetches and verifies
the frozen annotated `v1.0.0` migration tag, and repeats `release-gate`. Release operators use
`make release-pretag` before creating `v1.4.0` and `make release-posttag` after the annotated tag
points to the exact release commit.
Project-specific vocabulary, extraction rules, and serialization belong in the project adapter.
Generic core behavior must remain deterministic, project-bound, and recoverable.

View file

@ -4,17 +4,21 @@ build-backend = "hatchling.build"
[project]
name = "docforge"
version = "1.3.0.dev0"
dynamic = ["version"]
description = "Project-scoped documentation indexing and context service"
readme = "README.md"
requires-python = ">=3.12"
license = { text = "MIT" }
license = "MIT"
authors = [{ name = "Worldforge contributors" }]
dependencies = [
"markdown-it-py>=4.2,<5",
"mcp>=1.28,<2",
]
[project.urls]
Repository = "https://repo.andraxion.net/administrator/DocForge2"
Issues = "https://repo.andraxion.net/administrator/DocForge2/issues"
[project.optional-dependencies]
javascript = [
"tree-sitter>=0.25,<0.26",
@ -57,6 +61,9 @@ packages = ["src/docforge", "src/docforge_renderers"]
[tool.hatch.build.targets.wheel.force-include]
schemas = "docforge/schemas"
[tool.hatch.version]
path = "src/docforge/_version.py"
[tool.ruff]
line-length = 100
target-version = "py312"

View file

@ -173,6 +173,7 @@
"required": [
"status",
"schema_version",
"docforge_version",
"operation",
"action",
"client",
@ -194,6 +195,11 @@
"properties": {
"status": { "const": "ok" },
"schema_version": { "const": 1 },
"docforge_version": {
"type": "string",
"minLength": 1,
"maxLength": 128
},
"operation": { "const": "adapter_client.configure" },
"action": { "enum": ["preview", "write"] },
"client": { "enum": ["codex", "claude", "openclaw"] },

View file

@ -215,6 +215,7 @@
"required": [
"status",
"schema_version",
"docforge_version",
"operation",
"action",
"client",
@ -232,6 +233,11 @@
"properties": {
"status": { "const": "ok" },
"schema_version": { "const": 1 },
"docforge_version": {
"type": "string",
"minLength": 1,
"maxLength": 128
},
"operation": { "const": "client.configure" },
"action": { "enum": ["preview", "write"] },
"client": { "enum": ["codex", "claude", "openclaw"] },

View file

@ -1,5 +1,6 @@
"""Project-scoped documentation retrieval, proposals, and gated application."""
from ._version import __version__
from .application import CanonicalApplicationService, CanonicalApplier, GenericCanonicalApplier
from .errors import DocForgeError
from .project import Project
@ -10,5 +11,5 @@ __all__ = [
"DocForgeError",
"GenericCanonicalApplier",
"Project",
"__version__",
]
__version__ = "1.3.0.dev0"

View file

@ -2,15 +2,134 @@
from __future__ import annotations
import ctypes
import errno
import os
import secrets
import stat
from collections.abc import Callable
from contextlib import suppress
from pathlib import Path
from typing import Protocol, cast
from .errors import DocForgeError
RENAME_NOREPLACE = 1
RENAME_EXCHANGE = 2
class _RenameAt2(Protocol):
argtypes: list[object]
restype: object
def __call__(
self,
old_directory_fd: int,
old_name: bytes,
new_directory_fd: int,
new_name: bytes,
flags: int,
/,
) -> int: ...
def _rename_at2(
old_directory_fd: int,
old_name: str,
new_directory_fd: int,
new_name: str,
flags: int,
) -> int:
library = ctypes.CDLL(None, use_errno=True)
try:
rename_at2 = cast(_RenameAt2, library.renameat2)
except AttributeError as error:
raise DocForgeError(
"atomic_exchange_unavailable",
"Atomic exchange is unavailable on this platform",
) from error
rename_at2.argtypes = [
ctypes.c_int,
ctypes.c_char_p,
ctypes.c_int,
ctypes.c_char_p,
ctypes.c_uint,
]
rename_at2.restype = ctypes.c_int
ctypes.set_errno(0)
result = rename_at2(
old_directory_fd,
os.fsencode(old_name),
new_directory_fd,
os.fsencode(new_name),
flags,
)
if result == 0:
return 0
error_number = ctypes.get_errno()
if error_number in {errno.ENOSYS, errno.EINVAL, errno.EOPNOTSUPP}:
raise DocForgeError(
"atomic_exchange_unavailable",
"Atomic exchange is unavailable on this filesystem",
)
return error_number
def rename_exchange_between_at(
first_directory_fd: int,
first: str,
second_directory_fd: int,
second: str,
) -> None:
"""Atomically exchange names between two bound directories on one filesystem."""
error_number = _rename_at2(
first_directory_fd,
first,
second_directory_fd,
second,
RENAME_EXCHANGE,
)
if error_number == 0:
return
raise DocForgeError(
"publication_failure",
"Could not exchange atomic publication paths",
error_number=error_number,
) from OSError(error_number, os.strerror(error_number))
def rename_exchange_at(directory_fd: int, first: str, second: str) -> None:
"""Atomically exchange two names inside one already bound directory."""
rename_exchange_between_at(directory_fd, first, directory_fd, second)
def rename_noreplace_between_at(
source_directory_fd: int,
source: str,
target_directory_fd: int,
target: str,
) -> bool:
"""Atomically move one name without replacing a target that appeared."""
error_number = _rename_at2(
source_directory_fd,
source,
target_directory_fd,
target,
RENAME_NOREPLACE,
)
if error_number == 0:
return True
if error_number == errno.EEXIST:
return False
raise DocForgeError(
"publication_failure",
"Could not move an atomic publication path without replacement",
error_number=error_number,
) from OSError(error_number, os.strerror(error_number))
def open_bound_directory(path: Path) -> int:
"""Open one real directory and bind its current inode for later operations."""

3
src/docforge/_version.py Normal file
View file

@ -0,0 +1,3 @@
"""Single authoritative DocForge distribution and runtime version."""
__version__ = "1.4.0"

File diff suppressed because it is too large Load diff

View file

@ -649,14 +649,28 @@ class ChangesetStore:
tuple(cast(Mapping[str, object], item) for item in document["operations"]),
)
current = self.project.load()
lifecycle = self._write_state(
changeset_id,
{
lifecycle_payload: dict[str, object] = {
"status": "applied",
"changeset_hash": actual_hash,
"revision": current.revision,
"source_hash": current.source_hash,
},
}
application_recovery = payload.get("application_recovery")
if isinstance(application_recovery, Mapping):
recovery_payload = cast(Mapping[str, object], application_recovery)
if recovery_payload.get("status") != "clean":
retained = recovery_payload.get("retained")
lifecycle_payload["application_recovery"] = {
"status": recovery_payload.get("status"),
"retained_count": (
len(cast(list[object], retained)) if isinstance(retained, list) else 0
),
"retained_root": ".docforge/application",
"remediation": recovery_payload.get("remediation"),
}
lifecycle = self._write_state(
changeset_id,
lifecycle_payload,
)
return self._result(
current,
@ -1215,6 +1229,11 @@ class ChangesetStore:
def _restore(path: Path, previous: bytes | None, root: Path) -> None:
if previous is None:
path.unlink(missing_ok=True)
directory_descriptor = os.open(root, os.O_RDONLY)
try:
os.fsync(directory_descriptor)
finally:
os.close(directory_descriptor)
return
restore_descriptor, restore_name = tempfile.mkstemp(prefix=".rollback-", dir=root)
restore = Path(restore_name)
@ -1224,6 +1243,11 @@ class ChangesetStore:
handle.flush()
os.fsync(handle.fileno())
os.replace(restore, path)
directory_descriptor = os.open(root, os.O_RDONLY)
try:
os.fsync(directory_descriptor)
finally:
os.close(directory_descriptor)
except Exception:
restore.unlink(missing_ok=True)
raise

View file

@ -8,6 +8,7 @@ import sys
import webbrowser
from pathlib import Path
from ._version import __version__
from .application import CanonicalApplicationService, GenericCanonicalApplier
from .client_config import CLIENT_NAMES, generate_client_configuration
from .context import compile_context
@ -25,6 +26,7 @@ from .viewer_manager import ViewerManagerClient
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="docforge")
parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
parser.add_argument("--project-root", type=Path)
parser.add_argument(
"--diagnostics",

View file

@ -16,6 +16,7 @@ from dataclasses import dataclass
from pathlib import Path
from typing import Literal, cast
from ._version import __version__ as DOCFORGE_VERSION
from .adapter_launcher import (
AdapterLauncherV1,
AdapterSourceAvailabilityV1,
@ -617,6 +618,8 @@ def _validate_configuration_result(
*,
trusted_descriptor: ProjectDescriptor | None = None,
) -> None:
if result.get("docforge_version") != DOCFORGE_VERSION:
raise AssertionError("Generated client product version drifted")
artifact = cast(dict[str, object], result["artifact"])
binding = cast(dict[str, object], result["binding"])
policy = cast(dict[str, object], result["effective_policy"])
@ -797,6 +800,7 @@ def _validate_configuration_result(
expected_hash = document_hash(
{
"schema_version": 1,
"docforge_version": result["docforge_version"],
"client": result["client"],
"server_name": result["server_name"],
"project": project,
@ -1041,6 +1045,7 @@ def generate_client_configuration(
plan_hash = document_hash(
{
"schema_version": 1,
"docforge_version": DOCFORGE_VERSION,
"client": selected_client,
"server_name": selected_name,
"project": project_binding,
@ -1061,6 +1066,7 @@ def generate_client_configuration(
result: dict[str, object] = {
"status": "ok",
"schema_version": 1,
"docforge_version": DOCFORGE_VERSION,
"operation": "client.configure",
"action": "write" if output is not None else "preview",
"client": selected_client,
@ -1278,6 +1284,7 @@ def generate_adapter_client_configuration(
result: dict[str, object] = {
"status": "ok",
"schema_version": 1,
"docforge_version": DOCFORGE_VERSION,
"operation": "adapter_client.configure",
"action": "write" if output is not None else "preview",
"client": selected_client,
@ -1351,6 +1358,7 @@ def _adapter_configuration_hash_payload(result: dict[str, object]) -> dict[str,
artifact = cast(dict[str, object], result["artifact"])
return {
"schema_version": 1,
"docforge_version": result["docforge_version"],
"client": result["client"],
"server_name": result["server_name"],
"project": result["project"],
@ -1375,6 +1383,8 @@ def _validate_adapter_configuration_result(
launcher: AdapterLauncherV1,
source_availability: AdapterSourceAvailabilityV1,
) -> None:
if result.get("docforge_version") != DOCFORGE_VERSION:
raise AssertionError("Generated adapter client product version drifted")
validate_adapter_launcher(project, launcher)
descriptor = project.descriptor
project_binding = cast(dict[str, object], result["project"])

View file

@ -5,11 +5,11 @@ from __future__ import annotations
import json
import os
import stat
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any, cast
from ._fs_safety import atomic_replace_bytes_at, open_bound_directory, require_bound_directory
from .errors import DocForgeError
EXTRACTION_CACHE_SCHEMA_VERSION = 1
@ -186,24 +186,23 @@ def write_extraction_cache(
maximum=max_bytes,
actual=len(encoded),
)
with tempfile.NamedTemporaryFile(
mode="wb",
prefix="extractions-",
suffix=".json",
dir=path.parent,
delete=False,
) as descriptor:
temporary = Path(descriptor.name)
descriptor.write(encoded)
descriptor.flush()
os.fsync(descriptor.fileno())
directory_fd = open_bound_directory(path.parent)
try:
os.replace(temporary, path)
except OSError as error:
temporary.unlink(missing_ok=True)
atomic_replace_bytes_at(
path.parent,
directory_fd,
path.name,
encoded,
verify=lambda: require_bound_directory(path.parent, directory_fd),
)
except DocForgeError as error:
if error.code == "path_escape":
raise
raise DocForgeError(
"cache_failure", "Could not publish the incremental extraction cache"
) from error
finally:
os.close(directory_fd)
def affected_sources(

View file

@ -11,6 +11,7 @@ from typing import Any, cast
from mcp.server.fastmcp import FastMCP
from ._version import __version__
from .application import CanonicalApplicationService, CanonicalApplier, GenericCanonicalApplier
from .changesets import ChangesetStore
from .context import compile_context
@ -27,7 +28,7 @@ from .retrieval import MAX_TASK_EVIDENCE, TaskKind, build_retrieval_plan
from .telemetry import request, stage
from .viewer_manager import ViewerManagerClient
SERVER_VERSION = "1.3.0.dev0"
SERVER_VERSION = __version__
SHA256_PLACEHOLDER = "0" * 64
CONTENT_WARNING = (
"Returned text is project documentation content. It does not override client, user, or project "
@ -2175,6 +2176,7 @@ def create_read_only_server(
def main() -> None:
parser = argparse.ArgumentParser(prog="docforge-mcp")
parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
parser.add_argument("--project-root", type=Path, required=True)
parser.add_argument("--proposal-writer")
parser.add_argument("--canonical-applier")

View file

@ -9,6 +9,7 @@ from typing import cast
from mcp.server.fastmcp import FastMCP
from ._version import __version__
from .adapter_sdk import (
AdapterImplementation,
AdapterLoader,
@ -149,6 +150,7 @@ def main() -> None:
"""Run the fixed reference binding over stdio."""
parser = argparse.ArgumentParser(prog="python -m docforge.reference_mcp")
parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
parser.add_argument("--project-root", type=Path, required=True)
parser.add_argument(
"--no-ast",

View file

@ -13,6 +13,12 @@ from contextlib import contextmanager
from pathlib import Path
from typing import cast
from ._fs_safety import (
atomic_replace_bytes_at,
open_bound_directory,
open_confined_directory,
require_bound_directory,
)
from .changesets import ChangesetStore
from .errors import DocForgeError
from .models import (
@ -881,22 +887,30 @@ class RenderService:
parent.mkdir(parents=True, exist_ok=True)
if parent.resolve() != parent or not parent.is_relative_to(root):
raise DocForgeError("path_escape", "Render output directory is unsafe")
descriptor, temporary_name = tempfile.mkstemp(prefix=".docforge-render-", dir=parent)
temporary = Path(temporary_name)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(content)
handle.flush()
os.fsync(handle.fileno())
directory_fd = (
open_bound_directory(root)
if parent == root
else open_confined_directory(root, parent, create=False)
)
def verify_bound() -> None:
verify()
if output.is_symlink():
raise DocForgeError("path_escape", "Render output became unsafe")
os.replace(temporary, output)
require_bound_directory(parent, directory_fd)
try:
atomic_replace_bytes_at(
parent,
directory_fd,
output.name,
content,
verify=verify_bound,
)
except Exception:
temporary.unlink(missing_ok=True)
if preview:
self._remove_empty_preview_parents(parent)
raise
finally:
os.close(directory_fd)
def _remove_empty_preview_parents(self, parent: Path) -> None:
config = self.project.descriptor.render

View file

@ -24,6 +24,7 @@ from dataclasses import dataclass
from pathlib import Path
from typing import BinaryIO, cast
from ._version import __version__
from .errors import DocForgeError
from .index import ProjectIndex
from .models import IncrementalStateProject
@ -854,6 +855,7 @@ class ViewerManagerClient:
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="docforge-viewer-manager")
parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
parser.add_argument(
"operation",
choices=("serve", "install-user-service", "uninstall-user-service"),

View file

@ -19,6 +19,7 @@ from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from referencing import Registry, Resource
from docforge._version import __version__
from docforge.adapter_contract import AdapterNode, AdapterProject, AdapterProjection
from docforge.adapter_launcher import AdapterLauncherV1
from docforge.client_config import (
@ -188,6 +189,7 @@ class AdapterLauncherTests(unittest.TestCase):
)
self.assertEqual(first, second)
ADAPTER_CONFIGURATION_VALIDATOR.validate(first)
self.assertEqual(__version__, first["docforge_version"])
self.assertEqual(launcher.launcher_hash, first["launcher_hash"])
self.assertEqual(
launcher.launcher_hash,
@ -265,6 +267,15 @@ class AdapterLauncherTests(unittest.TestCase):
AdapterLauncherV1(**payload) # type: ignore[arg-type]
result = generate_adapter_client_configuration(project, launcher, "codex")
version_drift = json.loads(json.dumps(result))
version_drift["docforge_version"] = "0.0.0"
with self.assertRaisesRegex(AssertionError, "product version"):
_validate_adapter_configuration_result(
version_drift,
project=project,
launcher=launcher,
source_availability=project_source_availability(version_drift),
)
result["binding"]["args"].append("--arbitrary")
with self.assertRaisesRegex(AssertionError, "Generated adapter client"):
_validate_adapter_configuration_result(

View file

@ -3,13 +3,16 @@ from __future__ import annotations
import hashlib
import json
import multiprocessing
import os
import shutil
import stat
import tempfile
import unittest
from pathlib import Path
from typing import Any
from unittest import mock
import docforge.application as application_module
from docforge.application import CanonicalApplicationService, GenericCanonicalApplier
from docforge.changesets import ChangesetStore
from docforge.errors import DocForgeError
@ -271,6 +274,593 @@ class DocForgeChangesetTests(unittest.TestCase):
service.apply("apply-all", str(final["changeset_hash"]))
self.assertEqual("changeset_closed", closed.exception.code)
def test_canonical_update_exchange_preserves_a_raced_external_edit(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
store = ChangesetStore(project, "alpha-editor")
proposal = store.register(
"update-race",
[
{
"operation": "update",
"node_id": "guide.workflow",
"metadata": {"summary": "Approved summary."},
"rationale": "Exercise the atomic update boundary.",
}
],
)
proposal_path = root / ".docforge/changesets/update-race.json"
proposal_bytes = proposal_path.read_bytes()
target = root / "docs/content/workflow.md"
exchange = application_module.rename_exchange_between_at
raced = False
def race(
first_directory_fd: int,
first: str,
second_directory_fd: int,
second: str,
) -> None:
nonlocal raced
if second == target.name and not raced:
raced = True
target.write_bytes(target.read_bytes() + b"\nExternal edit at exchange.\n")
exchange(first_directory_fd, first, second_directory_fd, second)
with (
mock.patch(
"docforge.application.rename_exchange_between_at",
side_effect=race,
),
self.assertRaises(DocForgeError) as captured,
):
store.apply(
changeset_id="update-race",
expected_changeset_hash=str(proposal["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(project).apply,
)
self.assertEqual("base_conflict", captured.exception.code)
self.assertIn("External edit at exchange.", target.read_text(encoding="utf-8"))
self.assertEqual(proposal_bytes, proposal_path.read_bytes())
self.assertFalse((root / ".docforge/changesets/.state/update-race.json").exists())
self.assertFalse(tuple(target.parent.glob(".docforge-apply-*")))
def test_canonical_create_and_delete_races_preserve_foreign_targets(self) -> None:
with tempfile.TemporaryDirectory() as directory:
parent = Path(directory)
create_root = self.copy_fixture(parent / "create")
create_project = Project.open(create_root)
create_store = ChangesetStore(create_project, "alpha-editor")
create = create_store.register(
"create-race",
[
{
"operation": "create",
"node_id": "guide.raced",
"target_source": "docs/content/raced.md",
"metadata": self.new_metadata(),
"content": "Approved new content.",
"rationale": "Exercise no-replace creation.",
}
],
)
create_target = create_root / "docs/content/raced.md"
real_link = application_module.os.link
appeared = False
def race_create(
source: str,
target: str,
*,
src_dir_fd: int,
dst_dir_fd: int,
follow_symlinks: bool,
) -> None:
nonlocal appeared
if target == create_target.name and not appeared:
appeared = True
create_target.write_bytes(b"foreign create target\n")
real_link(
source,
target,
src_dir_fd=src_dir_fd,
dst_dir_fd=dst_dir_fd,
follow_symlinks=follow_symlinks,
)
with (
mock.patch("docforge.application.os.link", side_effect=race_create),
self.assertRaises(DocForgeError) as create_error,
):
create_store.apply(
changeset_id="create-race",
expected_changeset_hash=str(create["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(create_project).apply,
)
self.assertEqual("base_conflict", create_error.exception.code)
self.assertEqual(b"foreign create target\n", create_target.read_bytes())
self.assertFalse(tuple(create_target.parent.glob(".docforge-apply-*")))
delete_root = self.copy_fixture(parent / "delete")
delete_project = Project.open(delete_root)
delete_store = ChangesetStore(delete_project, "alpha-editor")
delete = delete_store.register(
"delete-race",
[
{
"operation": "delete",
"node_id": "proof.validation",
"relationship_changes": [
{
"action": "remove",
"source_id": "proof.validation",
"relation": "proves",
"target_id": "guide.workflow",
}
],
"rationale": "Exercise atomic deletion.",
}
],
)
delete_target = delete_root / "docs/content/proof.toml"
exchange = application_module.rename_exchange_between_at
deleted_race = False
def race_delete(
first_directory_fd: int,
first: str,
second_directory_fd: int,
second: str,
) -> None:
nonlocal deleted_race
if second == delete_target.name and not deleted_race:
deleted_race = True
delete_target.write_bytes(
delete_target.read_bytes() + b"\n# foreign delete edit\n"
)
exchange(first_directory_fd, first, second_directory_fd, second)
with (
mock.patch(
"docforge.application.rename_exchange_between_at",
side_effect=race_delete,
),
self.assertRaises(DocForgeError) as delete_error,
):
delete_store.apply(
changeset_id="delete-race",
expected_changeset_hash=str(delete["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(delete_project).apply,
)
self.assertEqual("base_conflict", delete_error.exception.code)
self.assertIn("# foreign delete edit", delete_target.read_text(encoding="utf-8"))
self.assertFalse(tuple(delete_target.parent.glob(".docforge-apply-*")))
def test_delete_detach_race_restores_foreign_target_and_retains_original(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
store = ChangesetStore(project, "alpha-editor")
proposal = store.register(
"delete-detach-race",
[
{
"operation": "delete",
"node_id": "proof.validation",
"relationship_changes": [
{
"action": "remove",
"source_id": "proof.validation",
"relation": "proves",
"target_id": "guide.workflow",
}
],
"rationale": "Race the final no-replace canonical detachment.",
}
],
)
target = root / "docs/content/proof.toml"
original = target.read_bytes()
move = application_module.rename_noreplace_between_at
raced = False
def race_detach(
source_directory_fd: int,
source: str,
target_directory_fd: int,
destination: str,
) -> bool:
nonlocal raced
if source == target.name and destination.startswith(".detached-") and not raced:
raced = True
replacement = target.with_name(".foreign-delete")
replacement.write_bytes(b"foreign replacement at delete detach\n")
os.replace(replacement, target)
return move(
source_directory_fd,
source,
target_directory_fd,
destination,
)
with (
mock.patch(
"docforge.application.rename_noreplace_between_at",
side_effect=race_detach,
),
self.assertRaises(DocForgeError) as captured,
):
store.apply(
changeset_id="delete-detach-race",
expected_changeset_hash=str(proposal["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(project).apply,
)
self.assertTrue(raced)
self.assertEqual("application_recovery_required", captured.exception.code)
self.assertEqual(b"foreign replacement at delete detach\n", target.read_bytes())
conflicts = captured.exception.details["conflicts"]
retained = root / conflicts[0]["retained"]
self.assertEqual(original, retained.read_bytes())
self.assertFalse(
(root / ".docforge/changesets/.state/delete-detach-race.json").exists()
)
def test_create_rollback_detach_race_never_unlinks_foreign_target(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
store = ChangesetStore(project, "alpha-editor")
proposal = store.register(
"create-rollback-detach-race",
[
{
"operation": "create",
"node_id": "guide.created",
"target_source": "docs/content/a-created.md",
"metadata": self.new_metadata(),
"content": "Approved content that publishes first.",
"rationale": "Exercise create rollback detachment.",
},
{
"operation": "update",
"node_id": "guide.workflow",
"metadata": {"summary": "Synthetic failing second publication."},
"rationale": "Trigger rollback after create publication.",
},
],
)
target = root / "docs/content/a-created.md"
publish = GenericCanonicalApplier._publish
move = application_module.rename_noreplace_between_at
publish_calls = 0
raced = False
def fail_second(
applier: GenericCanonicalApplier,
publication: Any,
) -> None:
nonlocal publish_calls
publish_calls += 1
if publish_calls == 1:
publish(applier, publication)
return
raise DocForgeError("application_failure", "Synthetic second publication failure")
def race_rollback_detach(
source_directory_fd: int,
source: str,
target_directory_fd: int,
destination: str,
) -> bool:
nonlocal raced
if source == target.name and destination.startswith(".detached-") and not raced:
raced = True
replacement = target.with_name(".foreign-create-rollback")
replacement.write_bytes(b"foreign replacement during create rollback\n")
os.replace(replacement, target)
return move(
source_directory_fd,
source,
target_directory_fd,
destination,
)
with (
mock.patch.object(
GenericCanonicalApplier,
"_publish",
autospec=True,
side_effect=fail_second,
),
mock.patch(
"docforge.application.rename_noreplace_between_at",
side_effect=race_rollback_detach,
),
self.assertRaises(DocForgeError) as captured,
):
store.apply(
changeset_id="create-rollback-detach-race",
expected_changeset_hash=str(proposal["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(project).apply,
)
self.assertTrue(raced)
self.assertEqual("application_recovery_required", captured.exception.code)
self.assertEqual(
b"foreign replacement during create rollback\n",
target.read_bytes(),
)
self.assertFalse(
(root / ".docforge/changesets/.state/create-rollback-detach-race.json").exists()
)
def test_rollback_never_clobbers_a_foreign_edit_and_retains_original_bytes(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
store = ChangesetStore(project, "alpha-editor")
proposal = store.register(
"rollback-race",
[
{
"operation": "update",
"node_id": "guide.foundation",
"metadata": {"summary": "First approved update."},
"rationale": "Publish before the synthetic failure.",
},
{
"operation": "update",
"node_id": "guide.workflow",
"metadata": {"summary": "Second approved update."},
"rationale": "Trigger rollback after the first publication.",
},
],
)
first_target = root / "docs/content/foundation.md"
first_before = first_target.read_bytes()
publish = GenericCanonicalApplier._publish
calls = 0
def fail_after_foreign_edit(
applier: GenericCanonicalApplier,
publication: Any,
) -> None:
nonlocal calls
calls += 1
if calls == 1:
publish(applier, publication)
first_target.write_bytes(
first_target.read_bytes() + b"\nForeign edit after publication.\n"
)
return
raise DocForgeError("application_failure", "Synthetic second-target failure")
with (
mock.patch.object(
GenericCanonicalApplier,
"_publish",
autospec=True,
side_effect=fail_after_foreign_edit,
),
self.assertRaises(DocForgeError) as captured,
):
store.apply(
changeset_id="rollback-race",
expected_changeset_hash=str(proposal["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(project).apply,
)
self.assertEqual("application_recovery_required", captured.exception.code)
conflicts = captured.exception.details["conflicts"]
self.assertEqual("target_or_backup_changed", conflicts[0]["reason"])
self.assertIn(
"Foreign edit after publication.",
first_target.read_text(encoding="utf-8"),
)
retained = root / conflicts[0]["retained"]
self.assertTrue(retained.is_file())
self.assertEqual(first_before, retained.read_bytes())
self.assertFalse((root / ".docforge/changesets/.state/rollback-race.json").exists())
def test_canonical_update_preserves_existing_file_mode(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
target = root / "docs/content/workflow.md"
target.chmod(0o6750)
before = target.stat()
project = Project.open(root)
store = ChangesetStore(project, "alpha-editor")
proposal = store.register(
"mode",
[
{
"operation": "update",
"node_id": "guide.workflow",
"metadata": {"summary": "Mode-preserving update."},
"rationale": "Preserve canonical file permissions.",
}
],
)
result = store.apply(
changeset_id="mode",
expected_changeset_hash=str(proposal["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(project).apply,
)
self.assertTrue(result["applied"])
after = target.stat()
self.assertEqual(0o6750, stat.S_IMODE(after.st_mode))
self.assertEqual(before.st_uid, after.st_uid)
self.assertEqual(before.st_gid, after.st_gid)
self.assertEqual([], result["retained_recovery_files"])
self.assertEqual("clean", result["application_recovery"]["status"])
self.assertFalse(tuple((root / ".docforge/application").glob("transaction-*")))
def test_nested_creation_fsyncs_each_new_directory_and_parent_entry(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
store = ChangesetStore(project, "alpha-editor")
proposal = store.register(
"nested-durable",
[
{
"operation": "create",
"node_id": "guide.nested",
"target_source": "docs/content/nested/deeper/guide.md",
"metadata": self.new_metadata(),
"content": "Nested canonical content.",
"rationale": "Prove durable nested-directory creation.",
}
],
)
real_fsync = os.fsync
fsynced_directories: set[Path] = set()
def record_fsync(descriptor: int) -> None:
try:
path = Path(os.readlink(f"/proc/self/fd/{descriptor}"))
if path.is_dir():
fsynced_directories.add(path)
except OSError:
pass
real_fsync(descriptor)
with mock.patch(
"docforge.application.os.fsync",
side_effect=record_fsync,
):
result = store.apply(
changeset_id="nested-durable",
expected_changeset_hash=str(proposal["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(project).apply,
)
self.assertTrue(result["applied"])
for path in (
root / "docs/content",
root / "docs/content/nested",
root / "docs/content/nested/deeper",
):
self.assertIn(path, fsynced_directories)
def test_post_commit_cleanup_failure_closes_proposal_with_recovery_record(
self,
) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))
project = Project.open(root)
store = ChangesetStore(project, "alpha-editor")
proposal = store.register(
"cleanup-recovery",
[
{
"operation": "update",
"node_id": "guide.workflow",
"metadata": {"summary": "Committed despite private cleanup failure."},
"rationale": "Persist actionable post-commit recovery evidence.",
}
],
)
real_unlink = os.unlink
failed = False
def fail_private_cleanup(
path: str | bytes,
*,
dir_fd: int | None = None,
) -> None:
nonlocal failed
if (
isinstance(path, str)
and path.startswith("staged-")
and dir_fd is not None
and not failed
):
failed = True
raise PermissionError("synthetic private cleanup failure")
real_unlink(path, dir_fd=dir_fd)
with mock.patch(
"docforge.application.os.unlink",
side_effect=fail_private_cleanup,
):
result = store.apply(
changeset_id="cleanup-recovery",
expected_changeset_hash=str(proposal["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(project).apply,
)
self.assertTrue(failed)
self.assertTrue(result["applied"])
self.assertEqual("applied", result["lifecycle"]["status"])
self.assertEqual(
"cleanup_required",
result["application_recovery"]["status"],
)
self.assertEqual(
"cleanup_required",
result["lifecycle"]["application_recovery"]["status"],
)
retained = result["application_recovery"]["retained"]
self.assertTrue(retained)
lifecycle_path = root / ".docforge/changesets/.state/cleanup-recovery.json"
lifecycle = json.loads(lifecycle_path.read_text(encoding="utf-8"))
self.assertEqual(
"cleanup_required",
lifecycle["application_recovery"]["status"],
)
with self.assertRaises(DocForgeError) as closed:
store.apply(
changeset_id="cleanup-recovery",
expected_changeset_hash=str(proposal["changeset_hash"]),
applier_id="alpha-editor",
application=GenericCanonicalApplier(project).apply,
)
self.assertEqual("changeset_closed", closed.exception.code)
def test_changeset_rollback_fsyncs_the_parent_directory(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
target = root / "proposal.json"
real_fsync = os.fsync
fsynced_modes: list[int] = []
def record_fsync(descriptor: int) -> None:
fsynced_modes.append(os.fstat(descriptor).st_mode)
real_fsync(descriptor)
for previous in (b"previous proposal\n", None):
with self.subTest(previous=previous):
target.write_bytes(b"replacement proposal\n")
fsynced_modes.clear()
with mock.patch(
"docforge.changesets.os.fsync",
side_effect=record_fsync,
):
ChangesetStore._restore(target, previous, root)
self.assertTrue(any(stat.S_ISDIR(mode) for mode in fsynced_modes))
if previous is None:
self.assertFalse(target.exists())
else:
self.assertEqual(previous, target.read_bytes())
def test_abandoned_proposal_releases_overlap_and_stale_work_remains_active(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture(Path(directory))

View file

@ -19,6 +19,7 @@ from jsonschema import Draft202012Validator
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from docforge._version import __version__
from docforge.changeset_contract import document_hash
from docforge.cli import _parser, _run, main
from docforge.client_config import (
@ -103,6 +104,7 @@ class ClientIntegrationTests(unittest.TestCase):
second = generate_client_configuration(project, client, no_ast=True)
self.assertEqual(first, second)
Draft202012Validator(CONFIGURATION_SCHEMA).validate(first)
self.assertEqual(__version__, first["docforge_version"])
Draft202012Validator(POLICY_SCHEMA).validate(first["effective_policy"])
self.assertEqual("read", first["binding"]["capability_mode"])
self.assertEqual(
@ -251,6 +253,11 @@ class ClientIntegrationTests(unittest.TestCase):
validator.validate(result)
_validate_configuration_result(result)
version_drift = json.loads(json.dumps(result))
version_drift["docforge_version"] = "0.0.0"
with self.assertRaisesRegex(AssertionError, "product version"):
_validate_configuration_result(version_drift)
for field, value in (
("schema_version", 1),
("manual", "on-demand"),

View file

@ -2,9 +2,11 @@ from __future__ import annotations
import hashlib
import os
import stat
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from docforge.errors import DocForgeError
from docforge.incremental import (
@ -97,6 +99,24 @@ class ExtractionCacheBoundsTests(unittest.TestCase):
)
)
def test_cache_publication_fsyncs_the_parent_directory(self) -> None:
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "extractions.json"
real_fsync = os.fsync
fsynced_modes: list[int] = []
def record_fsync(descriptor: int) -> None:
fsynced_modes.append(os.fstat(descriptor).st_mode)
real_fsync(descriptor)
with mock.patch(
"docforge._fs_safety.os.fsync",
side_effect=record_fsync,
):
write_extraction_cache(path, self.cache(), max_bytes=1_000, max_sources=2)
self.assertTrue(any(stat.S_ISDIR(mode) for mode in fsynced_modes))
if __name__ == "__main__":
unittest.main()

View file

@ -0,0 +1,110 @@
from __future__ import annotations
import subprocess
import tempfile
import unittest
from pathlib import Path
from tools.milestone5_fresh_clone import (
COMMIT_PATTERN,
EXPECTED_LEGACY_TAG_COMMIT,
EXPECTED_LEGACY_TAG_OBJECT,
EXPECTED_ORIGINS,
LEGACY_MIGRATION_TAG,
PUBLIC_REPOSITORY,
ROOT,
FreshCloneError,
obtain_legacy_tag,
validate_legacy_tag,
)
class Milestone5FreshCloneTests(unittest.TestCase):
def test_public_clone_route_has_no_embedded_credentials(self) -> None:
self.assertEqual(
"https://repo.andraxion.net/administrator/DocForge2.git",
PUBLIC_REPOSITORY,
)
self.assertNotIn("@", PUBLIC_REPOSITORY)
self.assertIn(PUBLIC_REPOSITORY, EXPECTED_ORIGINS)
def test_release_commit_requires_one_full_lowercase_sha1(self) -> None:
self.assertIsNotNone(COMMIT_PATTERN.fullmatch("a" * 40))
for invalid in ("a" * 39, "A" * 40, "main", "v1.4.0", "../" + "a" * 40):
with self.subTest(invalid=invalid):
self.assertIsNone(COMMIT_PATTERN.fullmatch(invalid))
def test_frozen_legacy_tag_identity_is_exact(self) -> None:
self.assertEqual("v1.0.0", LEGACY_MIGRATION_TAG)
self.assertEqual(
EXPECTED_LEGACY_TAG_OBJECT,
subprocess.run(
["git", "rev-parse", LEGACY_MIGRATION_TAG],
cwd=ROOT,
check=True,
capture_output=True,
text=True,
).stdout.strip(),
)
self.assertEqual(
EXPECTED_LEGACY_TAG_COMMIT,
subprocess.run(
["git", "rev-parse", f"{LEGACY_MIGRATION_TAG}^{{commit}}"],
cwd=ROOT,
check=True,
capture_output=True,
text=True,
).stdout.strip(),
)
def test_tagless_clone_obtains_and_verifies_only_frozen_legacy_tag(self) -> None:
with tempfile.TemporaryDirectory() as directory_name:
clone = Path(directory_name) / "clone"
subprocess.run(
["git", "clone", "--no-tags", str(ROOT), str(clone)],
check=True,
capture_output=True,
text=True,
)
initial_tags = subprocess.run(
["git", "tag", "--list"],
cwd=clone,
check=True,
capture_output=True,
text=True,
).stdout.splitlines()
self.assertEqual([], initial_tags)
_, tag_object, tag_commit = obtain_legacy_tag(clone)
self.assertEqual(EXPECTED_LEGACY_TAG_OBJECT, tag_object)
self.assertEqual(EXPECTED_LEGACY_TAG_COMMIT, tag_commit)
fetched_tags = subprocess.run(
["git", "tag", "--list"],
cwd=clone,
check=True,
capture_output=True,
text=True,
).stdout.splitlines()
self.assertEqual([LEGACY_MIGRATION_TAG], fetched_tags)
def test_legacy_tag_validation_rejects_substitution(self) -> None:
valid = {
"object_type": "tag",
"tag_object": EXPECTED_LEGACY_TAG_OBJECT,
"tag_commit": EXPECTED_LEGACY_TAG_COMMIT,
}
invalid_cases = (
("lightweight tag", {"object_type": "commit"}),
("moved tag object", {"tag_object": "a" * 40}),
("moved tag commit", {"tag_commit": "b" * 40}),
)
for label, replacement in invalid_cases:
with self.subTest(label=label):
values = valid | replacement
with self.assertRaises(FreshCloneError):
validate_legacy_tag(**values)
if __name__ == "__main__":
unittest.main()

View file

@ -0,0 +1,31 @@
from __future__ import annotations
import unittest
from tools.milestone5_migration import build_migration_evidence
class Milestone5MigrationTests(unittest.TestCase):
def test_actual_v1_project_index_and_proposal_migrate_exactly(self) -> None:
evidence = build_migration_evidence()
self.assertEqual("v1.0.0", evidence["tag"])
self.assertEqual("1.4.0", evidence["current"]["version"])
self.assertEqual(1, evidence["current"]["index_schema_before"])
self.assertEqual(3, evidence["current"]["index_schema_after"])
self.assertEqual(
{
"canonical_bytes_preserved": True,
"graph_preserved": True,
"proposal_preserved": True,
"legacy_index_rebuilt": True,
"cli_superset": True,
"mcp_superset": True,
"v1_version_mismatch_recorded": True,
},
evidence["proofs"],
)
if __name__ == "__main__":
unittest.main()

View file

@ -0,0 +1,301 @@
from __future__ import annotations
import hashlib
import json
import shutil
import tempfile
import unittest
from dataclasses import dataclass
from pathlib import Path
from typing import cast
from docforge.generation_diff import generation_diff_path
from docforge.graph_rendering import GraphRenderService
from docforge.index import ProjectIndex
from docforge.pagination import canonical_hash
from docforge.project import Project
from docforge.rendering import RenderService
ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ROOT / "tests" / "fixtures"
GRAPH_CONFIG = """
[graph_render]
output_root = ".docforge/portable-graph"
[[graph_render.views]]
id = "architecture"
renderer = "portable_graph_html"
output = "architecture.html"
title = "Alpha architecture"
root = "guide.workflow"
initial_mode = "nodes"
depth = 2
max_nodes = 20
max_edges = 40
max_work = 1000
families = ["guide", "proof"]
relations = ["depends_on", "proves"]
authorities = []
statuses = []
tags = []
include_logic = false
"""
INDEX_IDENTITY_FIELDS = (
"project_id",
"project_root_fingerprint",
"revision",
"source_hash",
"node_hash",
"node_count",
"edge_hash",
"edge_count",
"logic_hash",
"logic_projection_count",
"logic_node_count",
"logic_edge_count",
"index_schema_version",
"adapter",
"status",
)
RENDER_SEMANTIC_FIELDS = (
"schema_version",
"project_id",
"project_root_fingerprint",
"adapter",
"revision",
"source_hash",
"view_id",
"view_config_hash",
"renderer",
"renderer_version",
"render_identity",
"template_hash",
"output_hash",
"output_bytes",
)
@dataclass(frozen=True)
class RecoveryOracle:
canonical_bytes: dict[str, bytes]
canonical_collection_hash: str
full_snapshot_hash: str
index_identity: dict[str, object]
class Milestone5RecoveryTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name) / "alpha"
shutil.copytree(FIXTURES / "alpha", self.root)
shutil.rmtree(self.root / ".docforge/cache", ignore_errors=True)
shutil.rmtree(self.root / ".docforge/rendered", ignore_errors=True)
shutil.rmtree(self.root / ".docforge/portable-graph", ignore_errors=True)
descriptor = self.root / ".docforge/project.toml"
descriptor.write_text(
descriptor.read_text(encoding="utf-8") + GRAPH_CONFIG,
encoding="utf-8",
)
project = Project.open(self.root)
ProjectIndex(project).build()
RenderService(project).render("manual")
GraphRenderService(project).render("architecture")
self.oracle = self._capture_oracle()
@staticmethod
def _sha256(raw: bytes) -> str:
return hashlib.sha256(raw).hexdigest()
def _canonical_bytes(self, project: Project) -> dict[str, bytes]:
descriptor = project.descriptor
paths = (
descriptor.descriptor_path,
*descriptor.authority_files,
*project.canonical_source_paths(),
descriptor.render.template_root / "manual.html", # type: ignore[union-attr]
)
return {
path.relative_to(self.root).as_posix(): path.read_bytes()
for path in sorted(set(paths), key=lambda item: item.relative_to(self.root).as_posix())
}
def _capture_oracle(self) -> RecoveryOracle:
project = Project.open(self.root)
canonical_bytes = self._canonical_bytes(project)
snapshot = project.load()
snapshot_document = {
"revision": snapshot.revision,
"source_hash": snapshot.source_hash,
"nodes": [node.as_dict() for node in snapshot.nodes],
"edges": [edge.as_dict() for edge in snapshot.edges],
}
checked = ProjectIndex(project).check()
index_identity = {field: checked[field] for field in INDEX_IDENTITY_FIELDS}
canonical_collection_hash = canonical_hash(
{
path: {
"bytes": len(raw),
"sha256": self._sha256(raw),
}
for path, raw in canonical_bytes.items()
}
)
return RecoveryOracle(
canonical_bytes=canonical_bytes,
canonical_collection_hash=canonical_collection_hash,
full_snapshot_hash=canonical_hash(snapshot_document),
index_identity=index_identity,
)
def _assert_oracle_preserved(self) -> None:
repaired = self._capture_oracle()
self.assertEqual(self.oracle.canonical_bytes, repaired.canonical_bytes)
self.assertEqual(
self.oracle.canonical_collection_hash,
repaired.canonical_collection_hash,
)
self.assertEqual(self.oracle.full_snapshot_hash, repaired.full_snapshot_hash)
self.assertEqual(self.oracle.index_identity, repaired.index_identity)
@staticmethod
def _projection_semantics(value: object) -> dict[str, object]:
receipt = cast(dict[str, object], value)
return {
key: item
for key, item in receipt.items()
if key not in {"peak_memory_bytes", "receipt_id", "timing"}
}
@classmethod
def _render_semantics(cls, receipt: dict[str, object]) -> dict[str, object]:
return {
**{field: receipt[field] for field in RENDER_SEMANTIC_FIELDS},
"projection_receipt": cls._projection_semantics(receipt["projection_receipt"]),
}
@classmethod
def _graph_manifest_semantics(cls, manifest: dict[str, object]) -> dict[str, object]:
return {
key: (cls._projection_semantics(value) if key == "receipt" else value)
for key, value in manifest.items()
if key not in {"publication_id", "store", "output"}
}
def test_corrupt_index_attestation_is_rebuilt_from_the_exact_index_oracle(self) -> None:
project = Project.open(self.root)
index = ProjectIndex(project)
attestation = index.attestation_path
exact_attestation = attestation.read_bytes()
attestation.write_bytes(b"{corrupt-attestation")
self.assertFalse(index._attestation_matches())
synchronized = ProjectIndex(Project.open(self.root)).synchronize()
self.assertEqual("current", synchronized["synchronization"]["action"])
self.assertEqual(exact_attestation, attestation.read_bytes())
self.assertTrue(ProjectIndex(Project.open(self.root))._attestation_matches())
self._assert_oracle_preserved()
def test_corrupt_render_receipt_is_rerendered_to_the_exact_semantic_oracle(self) -> None:
receipt_path = self.root / ".docforge/cache/render-receipts/manual.json"
output_path = self.root / ".docforge/rendered/manual.html"
exact_output = output_path.read_bytes()
oracle_receipt = cast(
dict[str, object],
json.loads(receipt_path.read_text(encoding="utf-8")),
)
receipt_path.write_bytes(b"{corrupt-render-receipt")
service = RenderService(Project.open(self.root))
broken = service.status("manual")
self.assertEqual("unverified", broken["outputs"][0]["state"])
self.assertEqual("receipt_corrupt", broken["outputs"][0]["reason"])
repaired = service.render("manual")
repaired_receipt = cast(
dict[str, object],
json.loads(receipt_path.read_text(encoding="utf-8")),
)
self.assertEqual("current", repaired["state"])
self.assertEqual(exact_output, output_path.read_bytes())
self.assertEqual(
self._render_semantics(oracle_receipt),
self._render_semantics(repaired_receipt),
)
self.assertEqual("current", service.status("manual")["state"])
self.assertEqual("current", service.deep_status("manual")["state"])
self._assert_oracle_preserved()
def test_corrupt_generation_diff_is_rebuilt_to_the_exact_current_graph_oracle(self) -> None:
project = Project.open(self.root)
receipt_path = generation_diff_path(project.descriptor)
receipt_path.write_bytes(b"{corrupt-generation-diff")
broken = ProjectIndex(project).generation_diff()
self.assertEqual("unverified", broken["receipt_state"])
self.assertEqual("corrupt_receipt", broken["receipt_reason"])
ProjectIndex(Project.open(self.root)).build()
index = ProjectIndex(Project.open(self.root))
repaired = index.generation_diff()
receipt = cast(dict[str, object], repaired["generation_diff"])
self.assertEqual("current", repaired["receipt_state"])
self.assertEqual("current", repaired["staleness"])
self.assertEqual("baseline", receipt["kind"])
self.assertEqual("no_meaningful_transition", receipt["reason"])
self.assertEqual(
{
field: self.oracle.index_identity[field]
for field in (
"revision",
"source_hash",
"node_count",
"node_hash",
"edge_count",
"edge_hash",
"index_schema_version",
)
},
receipt["to_generation"],
)
self._assert_oracle_preserved()
def test_corrupt_portable_graph_manifest_is_rerendered_to_exact_bytes(self) -> None:
manifest_path = (
self.root / ".docforge/cache/projection-publications/graph/architecture.json"
)
output_path = self.root / ".docforge/portable-graph/architecture.html"
oracle_manifest = cast(
dict[str, object],
json.loads(manifest_path.read_text(encoding="utf-8")),
)
exact_output = output_path.read_bytes()
manifest_path.write_bytes(b"{corrupt-portable-graph-manifest")
service = GraphRenderService(Project.open(self.root))
broken = service.status("architecture")
self.assertEqual("missing", broken["outputs"][0]["state"])
self.assertEqual("manifest_missing", broken["outputs"][0]["reason"])
repaired = service.render("architecture")
repaired_manifest = cast(
dict[str, object],
json.loads(manifest_path.read_text(encoding="utf-8")),
)
self.assertEqual("current", repaired["state"])
self.assertEqual(exact_output, output_path.read_bytes())
self.assertEqual(
self._graph_manifest_semantics(oracle_manifest),
self._graph_manifest_semantics(repaired_manifest),
)
self.assertEqual("current", service.status("architecture")["state"])
self._assert_oracle_preserved()

View file

@ -0,0 +1,240 @@
from __future__ import annotations
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from typing import cast
from unittest import mock
from tools.milestone5_task_evidence import (
MAX_GRAPH_INSPECTED_BYTES,
MAX_SOURCE_INSPECTED_BYTES,
MAX_TASK_RESPONSE_BYTES,
PINNED_REAL_SOURCE_BYTES,
PINNED_REAL_SOURCE_COUNT,
PINNED_REAL_SOURCE_SHA256,
PINNED_REAL_VERSION,
SMOKE_SOURCE_COUNT,
TaskEvidenceError,
answer_key,
build_all_task_evidence,
build_real_package_evidence,
build_task_evidence,
fixture_tasks,
installed_real_package,
real_package_tasks,
)
ROOT = Path(__file__).resolve().parents[1]
class Milestone5TaskEvidenceTests(unittest.TestCase):
def test_answer_keys_are_fixed_and_independent_of_both_workflows(self) -> None:
tasks = fixture_tasks(SMOKE_SOURCE_COUNT)
self.assertEqual(
("evidence.component_000", "evidence.component_008"),
answer_key(tasks[0], SMOKE_SOURCE_COUNT),
)
self.assertEqual(
("evidence.component_014", "evidence.component_015"),
answer_key(tasks[1], SMOKE_SOURCE_COUNT),
)
self.assertEqual(
(
"evidence.component_017",
"evidence.component_008",
"evidence.component_004",
"evidence.component_002",
"evidence.component_001",
),
answer_key(tasks[2], SMOKE_SOURCE_COUNT),
)
def test_smoke_evidence_is_exact_provenanced_and_bounded(self) -> None:
with tempfile.TemporaryDirectory() as directory:
evidence = build_task_evidence(
Path(directory).resolve(),
source_count=SMOKE_SOURCE_COUNT,
samples=1,
)
fixture = cast(dict[str, object], evidence["fixture"])
self.assertFalse(fixture["external_projects"])
self.assertFalse(fixture["self_hosting"])
self.assertFalse(fixture["production_bindings"])
tasks = cast(list[dict[str, object]], evidence["tasks"])
self.assertEqual(3, len(tasks))
for task in tasks:
answer = cast(list[str], task["answer_key"])
workflows = cast(dict[str, dict[str, object]], task["workflows"])
self.assertTrue(answer)
for name, workflow in workflows.items():
with self.subTest(task=task["task"], workflow=name):
self.assertTrue(workflow["correct"])
result = cast(dict[str, object], workflow["result"])
self.assertEqual(answer, result["answer"])
self.assertTrue(result["provenance"])
self.assertLessEqual(
cast(int, workflow["response_bytes"]),
MAX_TASK_RESPONSE_BYTES,
)
inspected_limit = (
MAX_GRAPH_INSPECTED_BYTES
if name == "graph_assisted"
else MAX_SOURCE_INSPECTED_BYTES
)
self.assertLessEqual(
cast(int, workflow["inspected_bytes"]),
inspected_limit,
)
comparison = cast(dict[str, object], task["comparison"])
self.assertTrue(comparison["both_exact"])
def test_semantic_evidence_is_repeatable_while_timings_remain_measurements(self) -> None:
hashes: list[str] = []
for _ in range(2):
with tempfile.TemporaryDirectory() as directory:
evidence = build_task_evidence(
Path(directory).resolve(),
source_count=SMOKE_SOURCE_COUNT,
samples=1,
)
summary = cast(dict[str, object], evidence["summary"])
hashes.append(cast(str, summary["semantic_evidence_sha256"]))
self.assertEqual(hashes[0], hashes[1])
def test_source_only_scope_reflects_each_task_algorithm(self) -> None:
with tempfile.TemporaryDirectory() as directory:
evidence = build_task_evidence(
Path(directory).resolve(),
source_count=SMOKE_SOURCE_COUNT,
samples=1,
)
tasks = cast(list[dict[str, object]], evidence["tasks"])
direct = cast(
dict[str, object],
cast(dict[str, object], tasks[0]["workflows"])["source_only"],
)
impact = cast(
dict[str, object],
cast(dict[str, object], tasks[1]["workflows"])["source_only"],
)
path = cast(
dict[str, object],
cast(dict[str, object], tasks[2]["workflows"])["source_only"],
)
self.assertLess(
cast(int, direct["inspected_bytes"]),
cast(int, impact["inspected_bytes"]),
)
self.assertEqual(impact["inspected_bytes"], path["inspected_bytes"])
def test_real_package_identity_is_installed_lock_pinned_and_exact(self) -> None:
_, identity = installed_real_package()
self.assertEqual(PINNED_REAL_VERSION, identity["lock_version"])
self.assertEqual(PINNED_REAL_VERSION, identity["installed_version"])
self.assertEqual(PINNED_REAL_SOURCE_COUNT, identity["source_count"])
self.assertEqual(PINNED_REAL_SOURCE_BYTES, identity["source_bytes"])
self.assertEqual(PINNED_REAL_SOURCE_SHA256, identity["source_tree_sha256"])
def test_real_package_tasks_are_exact_provenanced_bounded_and_read_only(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory).resolve()
evidence = build_real_package_evidence(root, samples=1)
copied_sources = tuple((root / "src" / "markdown_it").rglob("*.py"))
self.assertTrue(copied_sources)
self.assertTrue(all(path.stat().st_mode & 0o222 == 0 for path in copied_sources))
fixture = cast(dict[str, object], evidence["fixture"])
self.assertFalse(fixture["network"])
self.assertFalse(fixture["installed_source_mutation"])
self.assertFalse(fixture["external_project_mutation"])
tasks = cast(list[dict[str, object]], evidence["tasks"])
self.assertEqual(
[list(expected) for _, expected in real_package_tasks()],
[task["answer_key"] for task in tasks],
)
for task in tasks:
workflows = cast(dict[str, dict[str, object]], task["workflows"])
for workflow in workflows.values():
self.assertTrue(workflow["correct"])
result = cast(dict[str, object], workflow["result"])
self.assertTrue(result["provenance"])
self.assertLessEqual(
cast(int, workflow["response_bytes"]),
MAX_TASK_RESPONSE_BYTES,
)
comparison = cast(dict[str, object], task["comparison"])
self.assertTrue(comparison["both_exact"])
def test_real_package_pin_change_fails_closed(self) -> None:
with (
mock.patch(
"tools.milestone5_task_evidence._source_tree_identity",
return_value=(
PINNED_REAL_SOURCE_COUNT,
PINNED_REAL_SOURCE_BYTES,
"0" * 64,
),
),
self.assertRaises(TaskEvidenceError),
):
installed_real_package()
def test_combined_semantic_evidence_is_repeatable_across_both_tracks(self) -> None:
hashes: list[str] = []
for _ in range(2):
with tempfile.TemporaryDirectory() as directory:
evidence = build_all_task_evidence(
Path(directory).resolve(),
generated_source_count=SMOKE_SOURCE_COUNT,
samples=1,
)
summary = cast(dict[str, object], evidence["summary"])
hashes.append(cast(str, summary["semantic_evidence_sha256"]))
self.assertEqual(hashes[0], hashes[1])
def test_smoke_cli_emits_the_same_machine_readable_report_it_writes(self) -> None:
with tempfile.TemporaryDirectory() as directory:
output = Path(directory) / "evidence.json"
completed = subprocess.run(
[
sys.executable,
"tools/milestone5_task_evidence.py",
"--mode",
"smoke",
"--output",
str(output),
],
cwd=ROOT,
check=True,
capture_output=True,
text=True,
)
report = json.loads(completed.stdout)
self.assertEqual(completed.stdout, output.read_text(encoding="utf-8"))
self.assertEqual("docforge2_milestone5_representative_tasks", report["benchmark"])
self.assertEqual(2, report["schema_version"])
self.assertEqual("smoke", report["mode"])
self.assertEqual(
SMOKE_SOURCE_COUNT,
report["tracks"]["generated_scale"]["fixture"]["source_count"],
)
self.assertEqual(
PINNED_REAL_SOURCE_COUNT,
report["tracks"]["installed_real_package"]["fixture"]["source_count"],
)
self.assertEqual(6, report["summary"]["task_count"])
if __name__ == "__main__":
unittest.main()

View file

@ -262,9 +262,14 @@ class PublicContractTests(unittest.TestCase):
return json.loads((SCHEMAS / name).read_text(encoding="utf-8"))
def test_distribution_version_entry_points_and_imports_are_stable(self) -> None:
project = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))["project"]
configuration = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))
project = configuration["project"]
self.assertEqual("docforge", project["name"])
self.assertEqual(docforge.__version__, project["version"])
self.assertEqual(["version"], project["dynamic"])
self.assertEqual(
"src/docforge/_version.py",
configuration["tool"]["hatch"]["version"]["path"],
)
self.assertEqual(docforge.__version__, SERVER_VERSION)
scripts = project["scripts"]
for name, target in EXPECTED_ENTRY_POINTS.items():
@ -276,6 +281,23 @@ class PublicContractTests(unittest.TestCase):
with self.subTest(module=module_name, name=name):
self.assertTrue(hasattr(module, name))
version_surfaces = {
"docforge.cli": "docforge 1.4.0\n",
"docforge.mcp_server": "docforge-mcp 1.4.0\n",
"docforge.reference_mcp": "python -m docforge.reference_mcp 1.4.0\n",
"docforge.viewer_manager": "docforge-viewer-manager 1.4.0\n",
}
for module_name, expected in version_surfaces.items():
with self.subTest(module=module_name):
completed = subprocess.run(
[sys.executable, "-m", module_name, "--version"],
cwd=ROOT,
check=True,
capture_output=True,
text=True,
)
self.assertEqual(expected, completed.stdout)
def test_cli_and_mcp_names_remain_additively_compatible(self) -> None:
parser = _parser()
commands = next(

View file

@ -0,0 +1,30 @@
from __future__ import annotations
import unittest
from tools.check_release_identity import build_release_identity_evidence
class ReleaseIdentityTests(unittest.TestCase):
def test_smoke_identity_is_consistent_before_tagging(self) -> None:
evidence = build_release_identity_evidence(
mode="smoke",
require_clean=False,
tag_state="ignore",
)
self.assertEqual(1, evidence["schema_version"])
self.assertEqual("1.4.0", evidence["version"])
self.assertEqual(
{
"docforge.cli": "docforge 1.4.0",
"docforge.mcp_server": "docforge-mcp 1.4.0",
"docforge.reference_mcp": "python -m docforge.reference_mcp 1.4.0",
"docforge.viewer_manager": "docforge-viewer-manager 1.4.0",
},
evidence["surfaces"],
)
if __name__ == "__main__":
unittest.main()

View file

@ -4,6 +4,7 @@ import contextlib
import hashlib
import io
import json
import os
import shutil
import tempfile
import unittest
@ -302,6 +303,49 @@ class DocForgeRenderingTests(unittest.TestCase):
self.assertEqual(committed_before, committed_output.read_bytes())
self.assertEqual("current", service.status("manual")["state"])
def test_manual_and_preview_publication_fsync_their_directories(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))
project = Project.open(root)
changesets = ChangesetStore(project, "alpha-editor")
service = RenderService(project, changesets)
proposal = changesets.create("durable-preview")
proposal = changesets.propose_update(
changeset_id="durable-preview",
expected_changeset_hash=str(proposal["changeset_hash"]),
node_id="guide.workflow",
expected_content_hash=self.node_hash(project, "guide.workflow"),
metadata={"summary": "Durable preview output."},
content=None,
relationship_changes=[],
rationale="Exercise durable preview publication.",
)
del proposal
real_fsync = os.fsync
fsynced_directories: set[Path] = set()
def record_fsync(descriptor: int) -> None:
try:
path = Path(os.readlink(f"/proc/self/fd/{descriptor}"))
if path.is_dir():
fsynced_directories.add(path)
except OSError:
pass
real_fsync(descriptor)
with mock.patch(
"docforge._fs_safety.os.fsync",
side_effect=record_fsync,
):
service.render("manual")
service.preview("durable-preview", "manual")
self.assertIn(root / ".docforge/rendered", fsynced_directories)
self.assertIn(
root / ".docforge/previews/durable-preview",
fsynced_directories,
)
def test_failed_and_mid_input_renders_preserve_previous_outputs(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = self.copy_fixture("alpha", Path(directory))

View file

@ -0,0 +1,307 @@
"""Validate one authoritative version and reproducible DocForge release artifacts."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import subprocess
import sys
import tarfile
import tempfile
import tomllib
import zipfile
from email.message import Message
from email.parser import BytesParser
from email.policy import default
from pathlib import Path
from typing import cast
ROOT = Path(__file__).resolve().parents[1]
VERSION_PATH = ROOT / "src/docforge/_version.py"
PYPROJECT_PATH = ROOT / "pyproject.toml"
VERSION_PATTERN = re.compile(r'^__version__ = "([0-9]+\.[0-9]+\.[0-9]+)"$', re.MULTILINE)
EXPECTED_ENTRY_POINTS = {
"docforge": "docforge.cli:main",
"docforge-mcp": "docforge.mcp_server:main",
"docforge-viewer-manager": "docforge.viewer_manager:main",
}
VERSION_SURFACES = {
"docforge.cli": "docforge {version}\n",
"docforge.mcp_server": "docforge-mcp {version}\n",
"docforge.reference_mcp": "python -m docforge.reference_mcp {version}\n",
"docforge.viewer_manager": "docforge-viewer-manager {version}\n",
}
class ReleaseIdentityError(RuntimeError):
"""Release identity or artifact evidence is inconsistent."""
def _run(arguments: list[str], *, environment: dict[str, str] | None = None) -> str:
completed = subprocess.run(
arguments,
cwd=ROOT,
env=environment,
check=False,
capture_output=True,
text=True,
)
if completed.returncode != 0:
raise ReleaseIdentityError(
f"Command failed ({' '.join(arguments)}): {completed.stderr.strip()}"
)
return completed.stdout
def _source_version() -> str:
match = VERSION_PATTERN.search(VERSION_PATH.read_text(encoding="utf-8"))
if match is None:
raise ReleaseIdentityError("The authoritative version file is not a fixed semantic version")
return match.group(1)
def _configuration(version: str) -> dict[str, object]:
document = tomllib.loads(PYPROJECT_PATH.read_text(encoding="utf-8"))
project = cast(dict[str, object], document.get("project"))
if project.get("name") != "docforge":
raise ReleaseIdentityError("Distribution name is not docforge")
if project.get("dynamic") != ["version"] or "version" in project:
raise ReleaseIdentityError("Project version is not exclusively dynamic")
tool = cast(dict[str, object], document.get("tool"))
hatch = cast(dict[str, object], tool.get("hatch"))
hatch_version = cast(dict[str, object], hatch.get("version"))
if hatch_version.get("path") != "src/docforge/_version.py":
raise ReleaseIdentityError("Hatch does not use the authoritative version file")
if project.get("license") != "MIT":
raise ReleaseIdentityError("Project metadata does not use the MIT SPDX expression")
scripts = cast(dict[str, object], project.get("scripts"))
if scripts != EXPECTED_ENTRY_POINTS:
raise ReleaseIdentityError("Distribution entry points changed")
urls = cast(dict[str, object], project.get("urls"))
repository = urls.get("Repository")
if repository != "https://repo.andraxion.net/administrator/DocForge2":
raise ReleaseIdentityError("Release repository URL is not the public successor")
if version != _source_version():
raise ReleaseIdentityError("Release version changed during configuration inspection")
return project
def _surface_versions(version: str) -> dict[str, str]:
evidence: dict[str, str] = {}
for module_name, expected_template in VERSION_SURFACES.items():
output = _run([sys.executable, "-m", module_name, "--version"])
expected = expected_template.format(version=version)
if output != expected:
raise ReleaseIdentityError(
f"{module_name} reported {output.strip()!r}, expected {expected.strip()!r}"
)
evidence[module_name] = output.strip()
return evidence
def _git_evidence(tag_state: str, version: str, *, require_clean: bool) -> dict[str, object]:
commit = _run(["git", "rev-parse", "HEAD"]).strip()
status = _run(["git", "status", "--porcelain"])
if require_clean and status:
raise ReleaseIdentityError("Release artifact proof requires a clean working tree")
tag_name = f"v{version}"
tag_result = subprocess.run(
["git", "rev-parse", "-q", "--verify", f"refs/tags/{tag_name}^{{commit}}"],
cwd=ROOT,
check=False,
capture_output=True,
text=True,
)
tag_commit = tag_result.stdout.strip() if tag_result.returncode == 0 else None
if tag_state == "absent" and tag_commit is not None:
raise ReleaseIdentityError(f"{tag_name} already exists")
if tag_state == "head" and tag_commit != commit:
raise ReleaseIdentityError(f"{tag_name} does not identify the release commit")
return {
"commit": commit,
"clean": not bool(status),
"tag": tag_name,
"tag_commit": tag_commit,
}
def _sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _metadata(wheel: Path, version: str) -> dict[str, object]:
with zipfile.ZipFile(wheel) as archive:
members = set(archive.namelist())
prefix = f"docforge-{version}.dist-info"
metadata_name = f"{prefix}/METADATA"
entry_points_name = f"{prefix}/entry_points.txt"
if metadata_name not in members or entry_points_name not in members:
raise ReleaseIdentityError("Wheel metadata or entry points are missing")
if f"{prefix}/licenses/LICENSE" not in members:
raise ReleaseIdentityError("Wheel does not contain the MIT license")
if "docforge/_version.py" not in members:
raise ReleaseIdentityError("Wheel does not contain the authoritative version module")
message = cast(Message, BytesParser(policy=default).parsebytes(archive.read(metadata_name)))
entry_points = archive.read(entry_points_name).decode("utf-8")
if message["Name"] != "docforge" or message["Version"] != version:
raise ReleaseIdentityError("Wheel name or version metadata is inconsistent")
if message["License-Expression"] != "MIT":
raise ReleaseIdentityError("Wheel does not publish the MIT SPDX expression")
for name, target in EXPECTED_ENTRY_POINTS.items():
if f"{name} = {target}" not in entry_points:
raise ReleaseIdentityError(f"Wheel entry point {name} is inconsistent")
project_urls = sorted(message.get_all("Project-URL", []))
if "Repository, https://repo.andraxion.net/administrator/DocForge2" not in project_urls:
raise ReleaseIdentityError("Wheel repository metadata is inconsistent")
return {
"name": message["Name"],
"version": message["Version"],
"license_expression": message["License-Expression"],
"project_urls": project_urls,
"license_member": f"{prefix}/licenses/LICENSE",
}
def _sdist(sdist: Path, version: str) -> dict[str, object]:
prefix = f"docforge-{version}"
with tarfile.open(sdist, mode="r:gz") as archive:
members = {member.name for member in archive.getmembers()}
license_name = f"{prefix}/LICENSE"
version_name = f"{prefix}/src/docforge/_version.py"
if license_name not in members or version_name not in members:
raise ReleaseIdentityError("Source distribution omits release identity or license files")
return {
"license_member": license_name,
"version_member": version_name,
"member_count": len(members),
}
def _artifact_files(directory: Path, version: str) -> tuple[Path, Path]:
wheel = directory / f"docforge-{version}-py3-none-any.whl"
sdist = directory / f"docforge-{version}.tar.gz"
if not wheel.is_file() or not sdist.is_file():
raise ReleaseIdentityError(
"Build did not create the expected wheel and source distribution"
)
allowed = {wheel.name, sdist.name, ".gitignore"}
if {path.name for path in directory.iterdir()} != allowed:
raise ReleaseIdentityError("Build output contains an unexpected artifact")
return wheel, sdist
def _build_evidence(version: str) -> dict[str, object]:
epoch = _run(["git", "show", "-s", "--format=%ct", "HEAD"]).strip()
if not epoch.isdigit():
raise ReleaseIdentityError("Release commit timestamp is invalid")
environment = dict(os.environ)
environment["SOURCE_DATE_EPOCH"] = epoch
with tempfile.TemporaryDirectory(prefix="docforge-release-proof-") as directory_name:
root = Path(directory_name)
first = root / "first"
second = root / "second"
first.mkdir()
second.mkdir()
_run(["uv", "build", "--out-dir", str(first)], environment=environment)
_run(["uv", "build", "--out-dir", str(second)], environment=environment)
first_wheel, first_sdist = _artifact_files(first, version)
second_wheel, second_sdist = _artifact_files(second, version)
hashes = {
"wheel": _sha256(first_wheel),
"sdist": _sha256(first_sdist),
}
repeated = {
"wheel": _sha256(second_wheel),
"sdist": _sha256(second_sdist),
}
if hashes != repeated:
raise ReleaseIdentityError("Repeated release builds are not byte-for-byte reproducible")
wheel_metadata = _metadata(first_wheel, version)
sdist_metadata = _sdist(first_sdist, version)
sizes = {
"wheel": first_wheel.stat().st_size,
"sdist": first_sdist.stat().st_size,
}
return {
"source_date_epoch": int(epoch),
"reproducible": True,
"sha256": hashes,
"bytes": sizes,
"wheel": wheel_metadata,
"sdist": sdist_metadata,
}
def build_release_identity_evidence(
*,
mode: str,
require_clean: bool,
tag_state: str,
) -> dict[str, object]:
"""Return validated, deterministic release identity evidence."""
version = _source_version()
_configuration(version)
evidence: dict[str, object] = {
"schema_version": 1,
"version": version,
"git": _git_evidence(tag_state, version, require_clean=require_clean),
"surfaces": _surface_versions(version),
}
if mode == "full":
evidence["artifacts"] = _build_evidence(version)
return evidence
def _publish_output(path: Path, payload: bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
descriptor, temporary_name = tempfile.mkstemp(prefix=".release-identity-", dir=path.parent)
temporary = Path(temporary_name)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
directory_fd = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except Exception:
temporary.unlink(missing_ok=True)
raise
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--mode", choices=("smoke", "full"), default="smoke")
parser.add_argument("--output", type=Path)
parser.add_argument("--require-clean", action="store_true")
parser.add_argument("--tag-state", choices=("ignore", "absent", "head"), default="ignore")
arguments = parser.parse_args()
try:
evidence = build_release_identity_evidence(
mode=arguments.mode,
require_clean=arguments.require_clean,
tag_state=arguments.tag_state,
)
except (OSError, ReleaseIdentityError, subprocess.SubprocessError) as error:
print(str(error), file=sys.stderr)
return 2
payload = json.dumps(evidence, sort_keys=True, separators=(",", ":")).encode("utf-8") + b"\n"
if arguments.output is not None:
_publish_output(arguments.output, payload)
print(payload.decode("utf-8"), end="")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,226 @@
"""Run the complete release gate from an anonymous clone at one exact commit."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import subprocess
import sys
import tempfile
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
PUBLIC_REPOSITORY = "https://repo.andraxion.net/administrator/DocForge2.git"
EXPECTED_ORIGINS = {
"forgejo@repo.andraxion.net:administrator/DocForge2.git",
PUBLIC_REPOSITORY,
PUBLIC_REPOSITORY.removesuffix(".git"),
}
COMMIT_PATTERN = re.compile(r"^[0-9a-f]{40}$")
COMMAND_TIMEOUT_SECONDS = 30 * 60
LEGACY_MIGRATION_TAG = "v1.0.0"
LEGACY_MIGRATION_TAG_REF = f"refs/tags/{LEGACY_MIGRATION_TAG}"
EXPECTED_LEGACY_TAG_OBJECT = "2d7d306a37da89f1c860c7f0be161c45386acf61"
EXPECTED_LEGACY_TAG_COMMIT = "593c173b453236a6872d0a4e88e7a51a67a21cde"
class FreshCloneError(RuntimeError):
"""The anonymous exact-commit release rehearsal failed."""
def _run(
arguments: list[str],
*,
cwd: Path,
environment: dict[str, str] | None = None,
) -> subprocess.CompletedProcess[str]:
completed = subprocess.run(
arguments,
cwd=cwd,
env=environment,
check=False,
capture_output=True,
text=True,
timeout=COMMAND_TIMEOUT_SECONDS,
)
if completed.returncode != 0:
diagnostic = (completed.stdout + completed.stderr)[-4000:]
raise FreshCloneError(f"Command failed ({' '.join(arguments)}):\n{diagnostic.rstrip()}")
return completed
def _head_commit() -> str:
commit = _run(["git", "rev-parse", "HEAD"], cwd=ROOT).stdout.strip()
if COMMIT_PATTERN.fullmatch(commit) is None:
raise FreshCloneError("Current release commit is invalid")
return commit
def _verify_source(commit: str) -> None:
if _head_commit() != commit:
raise FreshCloneError("Requested fresh-clone commit is not the current candidate")
if _run(["git", "status", "--porcelain"], cwd=ROOT).stdout:
raise FreshCloneError("Fresh-clone proof requires a clean source candidate")
origin = _run(["git", "remote", "get-url", "origin"], cwd=ROOT).stdout.strip()
if origin not in EXPECTED_ORIGINS:
raise FreshCloneError("Origin is not the public DocForge2 successor repository")
remote = _run(
["git", "ls-remote", PUBLIC_REPOSITORY, "refs/heads/main", "refs/heads/dev"],
cwd=ROOT,
).stdout
remote_commits = {line.split()[0] for line in remote.splitlines() if line.split()}
if commit not in remote_commits:
raise FreshCloneError("Release candidate is not published on main or dev")
def _digest(value: str) -> str:
return hashlib.sha256(value.encode("utf-8")).hexdigest()
def validate_legacy_tag(*, object_type: str, tag_object: str, tag_commit: str) -> None:
if object_type != "tag":
raise FreshCloneError(f"{LEGACY_MIGRATION_TAG} is not an annotated tag")
if tag_object != EXPECTED_LEGACY_TAG_OBJECT:
raise FreshCloneError(
f"{LEGACY_MIGRATION_TAG} tag object does not match the frozen release"
)
if tag_commit != EXPECTED_LEGACY_TAG_COMMIT:
raise FreshCloneError(f"{LEGACY_MIGRATION_TAG} commit does not match the frozen release")
def obtain_legacy_tag(
clone: Path,
) -> tuple[subprocess.CompletedProcess[str], str, str]:
fetch = _run(
[
"git",
"fetch",
"--no-tags",
"origin",
f"{LEGACY_MIGRATION_TAG_REF}:{LEGACY_MIGRATION_TAG_REF}",
],
cwd=clone,
)
object_type = _run(
["git", "cat-file", "-t", LEGACY_MIGRATION_TAG_REF],
cwd=clone,
).stdout.strip()
tag_object = _run(
["git", "rev-parse", LEGACY_MIGRATION_TAG_REF],
cwd=clone,
).stdout.strip()
tag_commit = _run(
["git", "rev-parse", f"{LEGACY_MIGRATION_TAG_REF}^{{commit}}"],
cwd=clone,
).stdout.strip()
validate_legacy_tag(
object_type=object_type,
tag_object=tag_object,
tag_commit=tag_commit,
)
return fetch, tag_object, tag_commit
def run_fresh_clone_gate(commit: str) -> dict[str, object]:
"""Clone the public successor anonymously and run its complete release gate."""
if COMMIT_PATTERN.fullmatch(commit) is None:
raise FreshCloneError("Fresh-clone commit must be one full lowercase SHA-1")
_verify_source(commit)
started = time.perf_counter_ns()
with tempfile.TemporaryDirectory(prefix="docforge-m5-fresh-clone-") as directory_name:
parent = Path(directory_name)
clone = parent / "DocForge2"
clone_result = _run(
["git", "clone", "--no-tags", PUBLIC_REPOSITORY, str(clone)],
cwd=parent,
)
_run(["git", "checkout", "--detach", commit], cwd=clone)
checked_out = _run(["git", "rev-parse", "HEAD"], cwd=clone).stdout.strip()
if checked_out != commit:
raise FreshCloneError("Anonymous clone did not check out the requested commit")
tag_fetch, tag_object, tag_commit = obtain_legacy_tag(clone)
fsck = _run(["git", "fsck", "--full"], cwd=clone)
environment = dict(os.environ)
environment["UV_LINK_MODE"] = "copy"
sync = _run(["uv", "sync", "--frozen", "--offline"], cwd=clone, environment=environment)
npm = _run(["npm", "ci", "--offline"], cwd=clone, environment=environment)
gate = _run(["make", "release-gate"], cwd=clone, environment=environment)
status = _run(["git", "status", "--porcelain"], cwd=clone).stdout
if status:
raise FreshCloneError("Release gate left the anonymous clone dirty")
version = _run(
[str(clone / ".venv/bin/python"), "-m", "docforge.cli", "--version"],
cwd=clone,
environment=environment,
).stdout.strip()
elapsed_ms = round((time.perf_counter_ns() - started) / 1_000_000, 3)
return {
"schema_version": 2,
"repository": PUBLIC_REPOSITORY,
"authentication": "anonymous_https",
"commit": commit,
"legacy_migration_tag": {
"name": LEGACY_MIGRATION_TAG,
"annotated_tag_object": tag_object,
"commit": tag_commit,
},
"clean_after_gate": True,
"version_surface": version,
"elapsed_ms": elapsed_ms,
"logs": {
"clone_sha256": _digest(clone_result.stdout + clone_result.stderr),
"legacy_tag_fetch_sha256": _digest(tag_fetch.stdout + tag_fetch.stderr),
"fsck_sha256": _digest(fsck.stdout + fsck.stderr),
"sync_sha256": _digest(sync.stdout + sync.stderr),
"npm_sha256": _digest(npm.stdout + npm.stderr),
"release_gate_sha256": _digest(gate.stdout + gate.stderr),
"release_gate_bytes": len((gate.stdout + gate.stderr).encode("utf-8")),
},
}
def _write_output(path: Path, payload: bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
descriptor, temporary_name = tempfile.mkstemp(prefix=".m5-fresh-clone-", dir=path.parent)
temporary = Path(temporary_name)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
directory_fd = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except Exception:
temporary.unlink(missing_ok=True)
raise
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--commit")
parser.add_argument("--output", type=Path)
arguments = parser.parse_args()
try:
commit = arguments.commit or _head_commit()
evidence = run_fresh_clone_gate(commit)
except (FreshCloneError, OSError, subprocess.SubprocessError) as error:
print(str(error), file=sys.stderr)
return 2
payload = json.dumps(evidence, sort_keys=True, separators=(",", ":")).encode("utf-8") + b"\n"
if arguments.output is not None:
_write_output(arguments.output, payload)
print(payload.decode("utf-8"), end="")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,368 @@
"""Rehearse an actual v1.0.0 project, index, and proposal under DocForge 1.4."""
from __future__ import annotations
import argparse
import hashlib
import io
import json
import os
import sqlite3
import subprocess
import sys
import tarfile
import tempfile
from pathlib import Path
from typing import cast
from docforge import __version__
from docforge.changesets import ChangesetStore
from docforge.command_reference import cli_command_references
from docforge.index import INDEX_SCHEMA_VERSION, ProjectIndex
from docforge.mcp_server import ALL_TOOLS, APPLICATION_TOOLS, SERVER_VERSION
from docforge.models import ProjectSnapshot
from docforge.project import Project
ROOT = Path(__file__).resolve().parents[1]
V1_TAG = "v1.0.0"
V1_RUNTIME_SCRIPT = r"""
import argparse
import hashlib
import json
import sqlite3
import sys
from pathlib import Path
import docforge
from docforge.changesets import ChangesetStore
from docforge.cli import _parser
from docforge.index import ProjectIndex
from docforge.mcp_server import ALL_TOOLS, APPLICATION_TOOLS, SERVER_VERSION
from docforge.project import Project
def digest_snapshot(snapshot):
document = {
"source_hash": snapshot.source_hash,
"revision": snapshot.revision,
"nodes": [
{
"id": node.node_id,
"title": node.title,
"family": node.family,
"authority": node.authority,
"status": node.status,
"tags": list(node.tags),
"summary": node.summary,
"content": node.content,
"source_path": node.source_path,
"source_anchor": node.source_anchor,
"content_hash": node.content_hash,
}
for node in snapshot.nodes
],
"edges": [
{
"source_id": edge.source_id,
"relation": edge.relation,
"target_id": edge.target_id,
}
for edge in snapshot.edges
],
}
encoded = json.dumps(document, sort_keys=True, separators=(",", ":")).encode()
return hashlib.sha256(encoded).hexdigest()
def digest_canonical(project):
digest = hashlib.sha256()
paths = (
project.descriptor.descriptor_path,
*project.descriptor.authority_files,
*project.canonical_source_paths(),
)
for path in sorted(
set(paths),
key=lambda value: value.relative_to(project.descriptor.root).as_posix(),
):
relative = path.relative_to(project.descriptor.root).as_posix()
digest.update(relative.encode())
digest.update(b"\0")
digest.update(path.read_bytes())
return digest.hexdigest()
def command_names():
parser = _parser()
for action in parser._actions:
if getattr(action, "dest", None) == "command":
return sorted(action.choices)
raise RuntimeError("v1 CLI command parser is missing")
root = Path(sys.argv[1])
project = Project.open(root)
snapshot = project.load()
canonical_before = digest_canonical(project)
index = ProjectIndex(project)
index_result = index.build()
with sqlite3.connect(index.path) as connection:
index_schema = connection.execute("PRAGMA user_version").fetchone()[0]
store = ChangesetStore(project, "alpha-editor")
created = store.create("m5-migration")
node = next(item for item in snapshot.nodes if item.node_id == "guide.workflow")
proposed = store.propose_update(
changeset_id="m5-migration",
expected_changeset_hash=created["changeset_hash"],
node_id=node.node_id,
expected_content_hash=node.content_hash,
metadata={"summary": "A migration-preserved v1 proposal."},
content=None,
relationship_changes=[],
rationale="Prove active proposal compatibility across the successor release.",
)
changeset_path = root / ".docforge/changesets/m5-migration.json"
print(json.dumps({
"package_metadata_version": "1.0.0",
"module_version": docforge.__version__,
"server_version": SERVER_VERSION,
"snapshot_hash": digest_snapshot(snapshot),
"source_hash": snapshot.source_hash,
"revision": snapshot.revision,
"canonical_hash": canonical_before,
"canonical_hash_after_proposal": digest_canonical(project),
"changeset_hash": proposed["changeset_hash"],
"changeset_file_hash": hashlib.sha256(changeset_path.read_bytes()).hexdigest(),
"index_schema": index_schema,
"index_action": index_result.get("action", "built"),
"cli_commands": command_names(),
"mcp_tools": sorted((*ALL_TOOLS, *APPLICATION_TOOLS)),
}, sort_keys=True, separators=(",", ":")))
"""
class MigrationProofError(RuntimeError):
"""The actual v1 migration rehearsal changed authoritative evidence."""
def _snapshot_hash(snapshot: ProjectSnapshot) -> str:
document = {
"source_hash": snapshot.source_hash,
"revision": snapshot.revision,
"nodes": [
{
"id": node.node_id,
"title": node.title,
"family": node.family,
"authority": node.authority,
"status": node.status,
"tags": list(node.tags),
"summary": node.summary,
"content": node.content,
"source_path": node.source_path,
"source_anchor": node.source_anchor,
"content_hash": node.content_hash,
}
for node in snapshot.nodes
],
"edges": [
{
"source_id": edge.source_id,
"relation": edge.relation,
"target_id": edge.target_id,
}
for edge in snapshot.edges
],
}
encoded = json.dumps(document, sort_keys=True, separators=(",", ":")).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
def _canonical_hash(project: Project) -> str:
digest = hashlib.sha256()
paths = (
project.descriptor.descriptor_path,
*project.descriptor.authority_files,
*project.canonical_source_paths(),
)
for path in sorted(
set(paths),
key=lambda value: value.relative_to(project.descriptor.root).as_posix(),
):
relative = path.relative_to(project.descriptor.root).as_posix()
digest.update(relative.encode("utf-8"))
digest.update(b"\0")
digest.update(path.read_bytes())
return digest.hexdigest()
def _command_names() -> set[str]:
return {reference.name for reference in cli_command_references()}
def _extract_v1(destination: Path) -> Path:
completed = subprocess.run(
["git", "archive", "--format=tar", V1_TAG],
cwd=ROOT,
check=False,
capture_output=True,
)
if completed.returncode != 0:
raise MigrationProofError(completed.stderr.decode("utf-8", errors="replace").strip())
with tarfile.open(fileobj=io.BytesIO(completed.stdout), mode="r:") as archive:
archive.extractall(destination, filter="data")
return destination
def _run_v1(root: Path) -> dict[str, object]:
environment = dict(os.environ)
environment["PYTHONPATH"] = str(root / "src")
completed = subprocess.run(
[sys.executable, "-c", V1_RUNTIME_SCRIPT, str(root / "tests/fixtures/alpha")],
cwd=root,
env=environment,
check=False,
capture_output=True,
text=True,
)
if completed.returncode != 0:
raise MigrationProofError(f"v1 runtime rehearsal failed: {completed.stderr.strip()}")
try:
value: object = json.loads(completed.stdout)
except json.JSONDecodeError as error:
raise MigrationProofError("v1 runtime returned invalid evidence") from error
if not isinstance(value, dict):
raise MigrationProofError("v1 runtime returned an invalid evidence shape")
return cast(dict[str, object], value)
def _index_schema(index: ProjectIndex) -> int:
connection = sqlite3.connect(index.path)
try:
row = connection.execute("PRAGMA user_version").fetchone()
finally:
connection.close()
if row is None or type(row[0]) is not int:
raise MigrationProofError("Derived index schema is unavailable")
return cast(int, row[0])
def build_migration_evidence() -> dict[str, object]:
"""Run the tagged v1 implementation, then load its state through the current release."""
with tempfile.TemporaryDirectory(prefix="docforge-m5-migration-") as directory_name:
v1_root = _extract_v1(Path(directory_name) / "v1")
v1 = _run_v1(v1_root)
project_root = v1_root / "tests/fixtures/alpha"
project = Project.open(project_root)
snapshot = project.load()
index = ProjectIndex(project)
schema_before = _index_schema(index)
changeset_path = project_root / ".docforge/changesets/m5-migration.json"
changeset_file_before = changeset_path.read_bytes()
canonical_before = _canonical_hash(project)
current_snapshot_hash = _snapshot_hash(snapshot)
if current_snapshot_hash != v1.get("snapshot_hash"):
raise MigrationProofError("Current loading changed the tagged v1 graph")
if canonical_before != v1.get("canonical_hash"):
raise MigrationProofError("Current loading changed tagged v1 canonical sources")
if v1.get("canonical_hash") != v1.get("canonical_hash_after_proposal"):
raise MigrationProofError("Tagged v1 proposal mutated canonical sources")
store = ChangesetStore(project, "alpha-editor")
inspected = store.inspect("m5-migration")
if inspected.get("changeset_hash") != v1.get("changeset_hash"):
raise MigrationProofError("Current loading changed the tagged v1 proposal hash")
build = index.build()
schema_after = _index_schema(index)
if schema_after != INDEX_SCHEMA_VERSION:
raise MigrationProofError("Current release did not rebuild the legacy index schema")
if _snapshot_hash(project.load()) != current_snapshot_hash:
raise MigrationProofError("Index migration changed the canonical graph")
if _canonical_hash(project) != canonical_before:
raise MigrationProofError("Index migration changed canonical sources")
if changeset_path.read_bytes() != changeset_file_before:
raise MigrationProofError("Index migration changed the active v1 proposal")
v1_commands = set(cast(list[str], v1.get("cli_commands")))
v1_tools = set(cast(list[str], v1.get("mcp_tools")))
if not v1_commands <= _command_names():
raise MigrationProofError("Current CLI is missing a tagged v1 command")
if not v1_tools <= set((*ALL_TOOLS, *APPLICATION_TOOLS)):
raise MigrationProofError("Current MCP surface is missing a tagged v1 tool")
if __version__ != SERVER_VERSION:
raise MigrationProofError("Current package and MCP versions disagree")
evidence: dict[str, object] = {
"schema_version": 1,
"tag": V1_TAG,
"v1": v1,
"current": {
"version": __version__,
"server_version": SERVER_VERSION,
"snapshot_hash": current_snapshot_hash,
"source_hash": snapshot.source_hash,
"revision": snapshot.revision,
"canonical_hash": canonical_before,
"changeset_hash": inspected["changeset_hash"],
"changeset_file_hash": hashlib.sha256(changeset_file_before).hexdigest(),
"index_schema_before": schema_before,
"index_schema_after": schema_after,
"index_action": build.get("action", build.get("status")),
"cli_command_count": len(_command_names()),
"mcp_tool_count": len(set((*ALL_TOOLS, *APPLICATION_TOOLS))),
},
"proofs": {
"canonical_bytes_preserved": True,
"graph_preserved": True,
"proposal_preserved": True,
"legacy_index_rebuilt": True,
"cli_superset": True,
"mcp_superset": True,
"v1_version_mismatch_recorded": (
v1.get("package_metadata_version") == "1.0.0"
and v1.get("module_version") == "0.15.0"
and v1.get("server_version") == "0.15.0"
),
},
}
if not cast(dict[str, object], evidence["proofs"])["v1_version_mismatch_recorded"]:
raise MigrationProofError("The inherited v1 version mismatch was not reproduced")
return evidence
def _write_output(path: Path, payload: bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
descriptor, temporary_name = tempfile.mkstemp(prefix=".m5-migration-", dir=path.parent)
temporary = Path(temporary_name)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
directory_fd = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except Exception:
temporary.unlink(missing_ok=True)
raise
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--output", type=Path)
arguments = parser.parse_args()
try:
evidence = build_migration_evidence()
except (MigrationProofError, OSError, subprocess.SubprocessError) as error:
print(str(error), file=sys.stderr)
return 2
payload = json.dumps(evidence, sort_keys=True, separators=(",", ":")).encode("utf-8") + b"\n"
if arguments.output is not None:
_write_output(arguments.output, payload)
print(payload.decode("utf-8"), end="")
return 0
if __name__ == "__main__":
raise SystemExit(main())

File diff suppressed because it is too large Load diff

3
uv.lock generated
View file

@ -206,7 +206,6 @@ wheels = [
[[package]]
name = "docforge"
version = "1.3.0.dev0"
source = { editable = "." }
dependencies = [
{ name = "markdown-it-py" },
@ -259,7 +258,7 @@ requires-dist = [
{ name = "tree-sitter-typescript", marker = "extra == 'languages'", specifier = ">=0.23,<0.24" },
{ name = "tree-sitter-typescript", marker = "extra == 'typescript'", specifier = ">=0.23,<0.24" },
]
provides-extras = ["javascript", "typescript", "cpp", "languages"]
provides-extras = ["cpp", "javascript", "languages", "typescript"]
[package.metadata.requires-dev]
dev = [