93 lines
4.5 KiB
Markdown
93 lines
4.5 KiB
Markdown
|
# DocForge2 development notes
|
|||
|
|
|
|||
|
|
This is the running implementation record for DocForge2. It records what is active, what was
|
|||
|
|
measured, what changed, what failed, why architectural decisions were made, and which ideas were
|
|||
|
|
deferred. Stable user and compatibility contracts still belong in dedicated documentation.
|
|||
|
|
|
|||
|
|
## Working rules
|
|||
|
|
|
|||
|
|
- Only one milestone is active at a time.
|
|||
|
|
- `main` remains the last fully verified milestone.
|
|||
|
|
- Active implementation occurs on `dev`.
|
|||
|
|
- Every milestone begins from direct repository evidence and ends with focused tests, the complete
|
|||
|
|
repository gate, updated measurements, documentation closeout, and a clean pushed state.
|
|||
|
|
- WorldForge, ScrapeStation, legacy DocForge, and production MCP bindings remain out of scope.
|
|||
|
|
- DocForge2 does not self-host during this program.
|
|||
|
|
- Release tags and Forgejo releases require Rob's explicit approval.
|
|||
|
|
|
|||
|
|
## Milestone 0 — complete
|
|||
|
|
|
|||
|
|
Milestone 0 established the public successor, preserved the complete lineage and v1 tag, integrated
|
|||
|
|
the no-AST and adapter-lifecycle work, froze compatibility guarantees, added repository-native
|
|||
|
|
quality and contract gates, and recorded cold/warm performance, memory, rendering, and response
|
|||
|
|
sizes.
|
|||
|
|
|
|||
|
|
The central measurement was decisive: a 1,000-node warm exact lookup took about 286 ms while the
|
|||
|
|
generation-pinned SQLite query path took about 0.4–1.4 ms. Repeated whole-source loading and
|
|||
|
|
validation, not SQLite, is the first optimization target.
|
|||
|
|
|
|||
|
|
## Milestone 1 — active: fast, observable core
|
|||
|
|
|
|||
|
|
### Outcome
|
|||
|
|
|
|||
|
|
Warm retrieval should disappear into normal tool overhead. Routine reads must not parse project
|
|||
|
|
sources. Status must not render or rebuild hidden work. Results must remain bounded independently
|
|||
|
|
of project size.
|
|||
|
|
|
|||
|
|
### Starting evidence
|
|||
|
|
|
|||
|
|
- Generic `Project.load()` walks, captures, parses, rereads, validates, hashes, and checks Git for
|
|||
|
|
the complete source set.
|
|||
|
|
- Exact retrieval validates twice around one bounded SQLite query.
|
|||
|
|
- Context compilation performs three full project loads.
|
|||
|
|
- Render status recompiles the complete manual.
|
|||
|
|
- Incremental adapters already prove that manifest attestation can make no-change synchronization
|
|||
|
|
and exact retrieval sub-millisecond on a tiny fixture.
|
|||
|
|
- Pinned viewer queries prove the current SQLite schema can serve bounded reads quickly.
|
|||
|
|
|
|||
|
|
### Current work
|
|||
|
|
|
|||
|
|
1. Audit request-scoped immutable snapshot and persistent-generation options.
|
|||
|
|
2. Audit result receipts, pagination, bounded response contracts, and side-effect-free status.
|
|||
|
|
3. Audit graph validation complexity, indexed traversal, profiling, and zero-source-parse proofs.
|
|||
|
|
4. Reconcile the audits into the smallest additive design that preserves v1 behavior.
|
|||
|
|
5. Implement and measure coherent slices, committing only after their gates pass.
|
|||
|
|
|
|||
|
|
### Work log
|
|||
|
|
|
|||
|
|
#### Linear dependency validation
|
|||
|
|
|
|||
|
|
The inherited dependency-cycle preparation scanned every edge once for every node. The graph
|
|||
|
|
validator now constructs dependency adjacency in one edge pass and sorts each adjacency list before
|
|||
|
|
the existing deterministic depth-first cycle check.
|
|||
|
|
|
|||
|
|
A 2,000-node regression test counts complete edge-collection iteration passes and caps them at
|
|||
|
|
four. The focused correctness and bounded-pass tests pass, and the configured strict source type
|
|||
|
|
gate is clean.
|
|||
|
|
|
|||
|
|
One validation command initially included `tests/test_core.py` in a direct Pyright invocation.
|
|||
|
|
Repository Pyright intentionally covers `src` and `tools`, so that command reported existing
|
|||
|
|
untyped test-result indexing rather than a source defect. Rerunning the repository-configured type
|
|||
|
|
gate produced zero diagnostics.
|
|||
|
|
|
|||
|
|
### Initial design constraints
|
|||
|
|
|
|||
|
|
- Full rebuild remains the recovery and equivalence oracle.
|
|||
|
|
- Canonical content remains authoritative.
|
|||
|
|
- Existing one-method `load_projection()` adapters remain unchanged.
|
|||
|
|
- No-AST adapters remain first-class.
|
|||
|
|
- Indexes, source-generation receipts, and caches remain disposable.
|
|||
|
|
- Cheap reads may trust only identity-bound, versioned, corruption-checked receipts.
|
|||
|
|
- Any optimization must fail closed on source mutation and must preserve stale-read refusal.
|
|||
|
|
|
|||
|
|
### Future ideas and suggestions
|
|||
|
|
|
|||
|
|
These are notes, not commitments:
|
|||
|
|
|
|||
|
|
- A stable source-generation provider may deserve a public adapter capability only after both the
|
|||
|
|
generic project and one incremental adapter prove the same boundary.
|
|||
|
|
- Profiling receipts could eventually feed the human-facing project control panel, but Milestone 1
|
|||
|
|
should expose structured data before adding UI.
|
|||
|
|
- Large context and changeset payloads may need cursor pagination or compact immutable receipts.
|
|||
|
|
The choice should follow actual client workflows rather than generic pagination machinery.
|