Make dependency validation linear

ACTIVE_SLICE.md

@@ -1,13 +1,12 @@
-# Milestone status
+# Active milestone
 
 ```text
-Milestone: 0 — successor foundation and measured baseline
+Milestone: 1 — fast, observable core
-Goal: Seed DocForge2 from the most advanced local lineage without breaking DocForge v1 contracts.
+Goal: Make warm retrieval immediate by removing repeated whole-project work without changing graph meaning.
-In scope: Complete Git lineage; verified no-AST work; adapter lifecycle safeguards; compatibility guarantees; repository-native contract and quality gates; cold/warm, memory, rendering, and response-size baselines; public successor migration; fresh-clone verification.
+In scope: Structured profiling; immutable request snapshots; duplicate-check elimination; linear validation; indexed traversal; compact receipts and bounded pagination where measurements require them; receipt-based status; persistent source generations; cheap no-change detection.
-Out of scope: Storage redesign; compiler or renderer redesign; portable render plans; self-hosting; production MCP repointing; WorldForge or ScrapeStation changes; tags and releases.
+Out of scope: Speculative storage replacement; task-shaped agent retrieval; independent render-plan packages; adapter SDK expansion; self-hosting; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases.
-Done when: administrator/DocForge2 is public and seeded from the advanced clean tree; administrator/DocForge remains intact; origin and legacy identify the successor and v1 remotes; all gates and a fresh-clone proof pass; the recorded baseline identifies measured bottlenecks without speculative optimization.
+Done when: Routine warm reads parse zero canonical sources; exact, search, traversal, context, synchronization, and status paths are bounded and measured; stale and corrupt state still fail closed or recover safely; legacy and no-AST adapters remain compatible; the complete repository gate passes.
-Status: Complete. DocForge2 is public and seeded from the complete advanced lineage. Compatibility and quality gates pass locally and from an anonymous fresh clone. The legacy repository and production MCP bindings remain unchanged.
+Status: Active. Repository audits and design reconciliation are in progress.
 ```
 
-No later milestone is active. `main` is the verified Milestone 0 state. `dev` begins at the same
+Milestones 2–5 remain directional context and are not active.
-commit and remains inactive until the next milestone is explicitly opened.

DEVELOPMENT_NOTES.md

@@ -0,0 +1,92 @@
+# DocForge2 development notes
+
+This is the running implementation record for DocForge2. It records what is active, what was
+measured, what changed, what failed, why architectural decisions were made, and which ideas were
+deferred. Stable user and compatibility contracts still belong in dedicated documentation.
+
+## Working rules
+
+- Only one milestone is active at a time.
+- `main` remains the last fully verified milestone.
+- Active implementation occurs on `dev`.
+- Every milestone begins from direct repository evidence and ends with focused tests, the complete
+  repository gate, updated measurements, documentation closeout, and a clean pushed state.
+- WorldForge, ScrapeStation, legacy DocForge, and production MCP bindings remain out of scope.
+- DocForge2 does not self-host during this program.
+- Release tags and Forgejo releases require Rob's explicit approval.
+
+## Milestone 0 — complete
+
+Milestone 0 established the public successor, preserved the complete lineage and v1 tag, integrated
+the no-AST and adapter-lifecycle work, froze compatibility guarantees, added repository-native
+quality and contract gates, and recorded cold/warm performance, memory, rendering, and response
+sizes.
+
+The central measurement was decisive: a 1,000-node warm exact lookup took about 286 ms while the
+generation-pinned SQLite query path took about 0.4–1.4 ms. Repeated whole-source loading and
+validation, not SQLite, is the first optimization target.
+
+## Milestone 1 — active: fast, observable core
+
+### Outcome
+
+Warm retrieval should disappear into normal tool overhead. Routine reads must not parse project
+sources. Status must not render or rebuild hidden work. Results must remain bounded independently
+of project size.
+
+### Starting evidence
+
+- Generic `Project.load()` walks, captures, parses, rereads, validates, hashes, and checks Git for
+  the complete source set.
+- Exact retrieval validates twice around one bounded SQLite query.
+- Context compilation performs three full project loads.
+- Render status recompiles the complete manual.
+- Incremental adapters already prove that manifest attestation can make no-change synchronization
+  and exact retrieval sub-millisecond on a tiny fixture.
+- Pinned viewer queries prove the current SQLite schema can serve bounded reads quickly.
+
+### Current work
+
+1. Audit request-scoped immutable snapshot and persistent-generation options.
+2. Audit result receipts, pagination, bounded response contracts, and side-effect-free status.
+3. Audit graph validation complexity, indexed traversal, profiling, and zero-source-parse proofs.
+4. Reconcile the audits into the smallest additive design that preserves v1 behavior.
+5. Implement and measure coherent slices, committing only after their gates pass.
+
+### Work log
+
+#### Linear dependency validation
+
+The inherited dependency-cycle preparation scanned every edge once for every node. The graph
+validator now constructs dependency adjacency in one edge pass and sorts each adjacency list before
+the existing deterministic depth-first cycle check.
+
+A 2,000-node regression test counts complete edge-collection iteration passes and caps them at
+four. The focused correctness and bounded-pass tests pass, and the configured strict source type
+gate is clean.
+
+One validation command initially included `tests/test_core.py` in a direct Pyright invocation.
+Repository Pyright intentionally covers `src` and `tools`, so that command reported existing
+untyped test-result indexing rather than a source defect. Rerunning the repository-configured type
+gate produced zero diagnostics.
+
+### Initial design constraints
+
+- Full rebuild remains the recovery and equivalence oracle.
+- Canonical content remains authoritative.
+- Existing one-method `load_projection()` adapters remain unchanged.
+- No-AST adapters remain first-class.
+- Indexes, source-generation receipts, and caches remain disposable.
+- Cheap reads may trust only identity-bound, versioned, corruption-checked receipts.
+- Any optimization must fail closed on source mutation and must preserve stale-read refusal.
+
+### Future ideas and suggestions
+
+These are notes, not commitments:
+
+- A stable source-generation provider may deserve a public adapter capability only after both the
+  generic project and one incremental adapter prove the same boundary.
+- Profiling receipts could eventually feed the human-facing project control panel, but Milestone 1
+  should expose structured data before adding UI.
+- Large context and changeset payloads may need cursor pagination or compact immutable receipts.
+  The choice should follow actual client workflows rather than generic pagination machinery.

src/docforge/project.py

@@ -479,14 +479,12 @@ def validate_graph(nodes: tuple[Node, ...], edges: tuple[Edge, ...]) -> None:
     if missing:
         raise DocForgeError("broken_edge", "Relationships target missing nodes", targets=missing)
 
-    dependencies = {
+    dependencies: dict[str, list[str]] = {node_id: [] for node_id in node_ids}
-        node_id: sorted(
+    for edge in edges:
-            edge.target_id
+        if edge.relation == "depends_on":
-            for edge in edges
+            dependencies[edge.source_id].append(edge.target_id)
-            if edge.source_id == node_id and edge.relation == "depends_on"
+    for targets in dependencies.values():
-        )
+        targets.sort()
-        for node_id in sorted(node_ids)
-    }
     visiting: set[str] = set()
     visited: set[str] = set()
 

tests/test_core.py

@@ -8,7 +8,10 @@ import sqlite3
 import sys
 import tempfile
 import unittest
+from collections.abc import Iterator
+from dataclasses import replace
 from pathlib import Path
+from typing import TypeVar, cast
 from unittest import mock
 
 ROOT = Path(__file__).resolve().parents[1]
@@ -18,9 +21,26 @@ from docforge.cli import main  # noqa: E402
 from docforge.context import compile_context  # noqa: E402
 from docforge.errors import DocForgeError  # noqa: E402
 from docforge.index import ProjectIndex  # noqa: E402
-from docforge.project import Project  # noqa: E402
+from docforge.models import Edge  # noqa: E402
+from docforge.project import Project, validate_graph  # noqa: E402
 
 FIXTURES = ROOT / "tests" / "fixtures"
+T = TypeVar("T")
+
+
+class CountingTuple(tuple[T, ...]):
+    """Count complete iteration passes without changing tuple behavior."""
+
+    iterations: int
+
+    def __new__(cls, values: tuple[T, ...]) -> CountingTuple[T]:
+        instance = super().__new__(cls, values)
+        instance.iterations = 0
+        return instance
+
+    def __iter__(self) -> Iterator[T]:
+        self.iterations += 1
+        return super().__iter__()
 
 
 class DocForgeCoreTests(unittest.TestCase):
@@ -149,6 +169,31 @@ class DocForgeCoreTests(unittest.TestCase):
             with self.assertRaisesRegex(DocForgeError, "cycle"):
                 Project.open(root).load()
 
+    def test_graph_validation_uses_a_bounded_number_of_edge_passes(self) -> None:
+        snapshot = Project.open(FIXTURES / "alpha").load()
+        nodes = tuple(
+            replace(
+                snapshot.nodes[0],
+                node_id=f"linear.node-{index:05d}",
+                source_path=f"docs/node-{index:05d}.md",
+            )
+            for index in range(2_000)
+        )
+        edges = CountingTuple(
+            tuple(
+                Edge(
+                    f"linear.node-{index:05d}",
+                    "depends_on",
+                    f"linear.node-{index - 1:05d}",
+                )
+                for index in range(1, len(nodes))
+            )
+        )
+
+        validate_graph(nodes, cast(tuple[Edge, ...], edges))
+
+        self.assertLessEqual(edges.iterations, 4)
+
     def test_index_build_is_repeatable_and_validates_every_row(self) -> None:
         with tempfile.TemporaryDirectory() as directory:
             root = self.copy_fixture("alpha", Path(directory))