92 lines
4.5 KiB
Markdown
92 lines
4.5 KiB
Markdown
# DocForge2 development notes
|
||
|
||
This is the running implementation record for DocForge2. It records what is active, what was
|
||
measured, what changed, what failed, why architectural decisions were made, and which ideas were
|
||
deferred. Stable user and compatibility contracts still belong in dedicated documentation.
|
||
|
||
## Working rules
|
||
|
||
- Only one milestone is active at a time.
|
||
- `main` remains the last fully verified milestone.
|
||
- Active implementation occurs on `dev`.
|
||
- Every milestone begins from direct repository evidence and ends with focused tests, the complete
|
||
repository gate, updated measurements, documentation closeout, and a clean pushed state.
|
||
- WorldForge, ScrapeStation, legacy DocForge, and production MCP bindings remain out of scope.
|
||
- DocForge2 does not self-host during this program.
|
||
- Release tags and Forgejo releases require Rob's explicit approval.
|
||
|
||
## Milestone 0 — complete
|
||
|
||
Milestone 0 established the public successor, preserved the complete lineage and v1 tag, integrated
|
||
the no-AST and adapter-lifecycle work, froze compatibility guarantees, added repository-native
|
||
quality and contract gates, and recorded cold/warm performance, memory, rendering, and response
|
||
sizes.
|
||
|
||
The central measurement was decisive: a 1,000-node warm exact lookup took about 286 ms while the
|
||
generation-pinned SQLite query path took about 0.4–1.4 ms. Repeated whole-source loading and
|
||
validation, not SQLite, is the first optimization target.
|
||
|
||
## Milestone 1 — active: fast, observable core
|
||
|
||
### Outcome
|
||
|
||
Warm retrieval should disappear into normal tool overhead. Routine reads must not parse project
|
||
sources. Status must not render or rebuild hidden work. Results must remain bounded independently
|
||
of project size.
|
||
|
||
### Starting evidence
|
||
|
||
- Generic `Project.load()` walks, captures, parses, rereads, validates, hashes, and checks Git for
|
||
the complete source set.
|
||
- Exact retrieval validates twice around one bounded SQLite query.
|
||
- Context compilation performs three full project loads.
|
||
- Render status recompiles the complete manual.
|
||
- Incremental adapters already prove that manifest attestation can make no-change synchronization
|
||
and exact retrieval sub-millisecond on a tiny fixture.
|
||
- Pinned viewer queries prove the current SQLite schema can serve bounded reads quickly.
|
||
|
||
### Current work
|
||
|
||
1. Audit request-scoped immutable snapshot and persistent-generation options.
|
||
2. Audit result receipts, pagination, bounded response contracts, and side-effect-free status.
|
||
3. Audit graph validation complexity, indexed traversal, profiling, and zero-source-parse proofs.
|
||
4. Reconcile the audits into the smallest additive design that preserves v1 behavior.
|
||
5. Implement and measure coherent slices, committing only after their gates pass.
|
||
|
||
### Work log
|
||
|
||
#### Linear dependency validation
|
||
|
||
The inherited dependency-cycle preparation scanned every edge once for every node. The graph
|
||
validator now constructs dependency adjacency in one edge pass and sorts each adjacency list before
|
||
the existing deterministic depth-first cycle check.
|
||
|
||
A 2,000-node regression test counts complete edge-collection iteration passes and caps them at
|
||
four. The focused correctness and bounded-pass tests pass, and the configured strict source type
|
||
gate is clean.
|
||
|
||
One validation command initially included `tests/test_core.py` in a direct Pyright invocation.
|
||
Repository Pyright intentionally covers `src` and `tools`, so that command reported existing
|
||
untyped test-result indexing rather than a source defect. Rerunning the repository-configured type
|
||
gate produced zero diagnostics.
|
||
|
||
### Initial design constraints
|
||
|
||
- Full rebuild remains the recovery and equivalence oracle.
|
||
- Canonical content remains authoritative.
|
||
- Existing one-method `load_projection()` adapters remain unchanged.
|
||
- No-AST adapters remain first-class.
|
||
- Indexes, source-generation receipts, and caches remain disposable.
|
||
- Cheap reads may trust only identity-bound, versioned, corruption-checked receipts.
|
||
- Any optimization must fail closed on source mutation and must preserve stale-read refusal.
|
||
|
||
### Future ideas and suggestions
|
||
|
||
These are notes, not commitments:
|
||
|
||
- A stable source-generation provider may deserve a public adapter capability only after both the
|
||
generic project and one incremental adapter prove the same boundary.
|
||
- Profiling receipts could eventually feed the human-facing project control panel, but Milestone 1
|
||
should expose structured data before adding UI.
|
||
- Large context and changeset payloads may need cursor pagination or compact immutable receipts.
|
||
The choice should follow actual client workflows rather than generic pagination machinery.
|