From a9a75c5c274f29673b50b1d1b6d100912ea31e27 Mon Sep 17 00:00:00 2001 From: Andraxion Date: Wed, 29 Jul 2026 06:11:22 -0400 Subject: [PATCH] Close Milestone 1 fast core --- ACTIVE_SLICE.md | 14 +- DEVELOPMENT_NOTES.md | 52 +- benchmarks/README.md | 6 + benchmarks/milestone1-2026-07-29.json | 1381 +++++++++++++++++++++++++ docs/MILESTONE_1_BASELINE.md | 102 ++ docs/MILESTONE_1_CLOSEOUT.md | 99 ++ 6 files changed, 1626 insertions(+), 28 deletions(-) create mode 100644 benchmarks/milestone1-2026-07-29.json create mode 100644 docs/MILESTONE_1_BASELINE.md create mode 100644 docs/MILESTONE_1_CLOSEOUT.md diff --git a/ACTIVE_SLICE.md b/ACTIVE_SLICE.md index 782de62..3dbf6b3 100644 --- a/ACTIVE_SLICE.md +++ b/ACTIVE_SLICE.md @@ -1,12 +1,12 @@ # Active milestone ```text -Milestone: 1 — fast, observable core -Goal: Make warm retrieval immediate by removing repeated whole-project work without changing graph meaning. -In scope: Structured profiling; immutable request snapshots; duplicate-check elimination; linear validation; indexed traversal; compact receipts and bounded pagination where measurements require them; receipt-based status; persistent source generations; cheap no-change detection. -Out of scope: Speculative storage replacement; task-shaped agent retrieval; independent render-plan packages; adapter SDK expansion; self-hosting; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases. -Done when: Routine warm reads parse zero canonical sources; exact, search, traversal, context, synchronization, and status paths are bounded and measured; stale and corrupt state still fail closed or recover safely; legacy and no-AST adapters remain compatible; the complete repository gate passes. -Status: Active. Repository audits and design reconciliation are in progress. +Milestone: 2 — agent retrieval and MCP experience +Goal: Let one project-bound server return compact, task-shaped, explainable context under an explicit effective policy. +In scope: Capability modes; capability-aware bootstrap; versioned retrieval plans and context capsules; task-shaped context; generation diffs; evidence-gap diagnostics; generated client configuration; doctor checks. +Out of scope: Independent render-plan packages; adapter SDK expansion; self-hosting; storage replacement; embeddings; WorldForge or ScrapeStation changes; production MCP repointing; tags and releases. +Done when: Policy and capabilities are explicit; bootstrap recommends only available actions; task context is compact, deterministic, provenance-bearing, and bounded; generation and evidence gaps are explainable; generated configuration and doctor checks are safe and tested; the complete repository gate and Milestone 2 benchmark pass. +Status: Active. Read-only contract audits begin from the verified Milestone 1 boundary. ``` -Milestones 2–5 remain directional context and are not active. +Milestones 3–5 remain directional context and are not active. diff --git a/DEVELOPMENT_NOTES.md b/DEVELOPMENT_NOTES.md index 92a6029..d57fc78 100644 --- a/DEVELOPMENT_NOTES.md +++ b/DEVELOPMENT_NOTES.md @@ -26,7 +26,7 @@ The central measurement was decisive: a 1,000-node warm exact lookup took about generation-pinned SQLite query path took about 0.4–1.4 ms. Repeated whole-source loading and validation, not SQLite, is the first optimization target. -## Milestone 1 — active: fast, observable core +## Milestone 1 — complete: fast, observable core ### Outcome @@ -45,13 +45,12 @@ of project size. and exact retrieval sub-millisecond on a tiny fixture. - Pinned viewer queries prove the current SQLite schema can serve bounded reads quickly. -### Current work +### Final outcome -1. Audit request-scoped immutable snapshot and persistent-generation options. -2. Audit result receipts, pagination, bounded response contracts, and side-effect-free status. -3. Audit graph validation complexity, indexed traversal, profiling, and zero-source-parse proofs. -4. Reconcile the audits into the smallest additive design that preserves v1 behavior. -5. Implement and measure coherent slices, committing only after their gates pass. +Routine generic reads now parse zero canonical source files, use one generation-pinned SQLite +snapshot, and return bounded results. Status checks perform no hidden rendering or rebuilding. +Structured counters prove those invariants independently of machine timing. Complete loading and +deep validation remain recovery and equivalence oracles. ### Work log @@ -116,20 +115,20 @@ backlinks, dependency, impact, context, and no-change synchronization operations `Project.load()` is forbidden. They also prove a final source-generation change is rejected before return and missing/corrupt receipts fall back and repair. -On the maintained 1,000-file fixture, the current work-in-progress measurements are: +The final clean 1,000-file run recorded: -| Operation | Milestone 0 median | Milestone 1 WIP median | -|---|---:|---:| -| Warm no-change synchronize | 142.479 ms | 20.007 ms | -| Exact node | 286.306 ms | 40.277 ms | -| Search, limit 20 | 288.793 ms | 41.455 ms | -| Dependencies, depth 8 | 287.791 ms | 41.551 ms | -| Context, 32k | 436.897 ms | 46.245 ms | -| MCP exact node | 287.094 ms | 40.051 ms | -| MCP context, 32k | 434.853 ms | 46.454 ms | +| Operation | Milestone 0 median | Milestone 1 median | Milestone 1 p95 | +|---|---:|---:|---:| +| Warm no-change synchronize | 142.479 ms | 9.192 ms | 9.324 ms | +| Exact node | 286.306 ms | 17.887 ms | 18.577 ms | +| Search, limit 20 | 288.793 ms | 19.518 ms | 19.884 ms | +| Dependencies, depth 8 | 287.791 ms | 18.117 ms | 19.061 ms | +| Context, 32k, page 20 | 436.897 ms | 25.395 ms | 25.867 ms | +| Render status | 150.591 ms | 18.969 ms | 19.613 ms | +| Visualization status | — | 9.875 ms | 10.836 ms | -The three-sample WIP run is directional, not the final Milestone 1 baseline. The final evidence run -will use the maintained sample counts and committed clean-tree revision. +The recorded run came from clean commit `6253c45a5eca01efa8c73ea3dfe4d85c55878ada`. +Every measured operation passed its p95 threshold and work-counter contract. #### Read-only audit reconciliation @@ -245,8 +244,8 @@ The result JSON schema contains the same closed operation, stage, and counter se implementation. The maintained `tools/milestone1_benchmark.py` harness adds hard counter and p95 latency gates to a -disposable generic project. The smoke target is part of `make gate`; the 1,000-node evidence run -will be recorded only from a clean committed revision. The historical Milestone 0 harness remains +disposable generic project. The smoke target is part of `make gate`; the final 1,000-node evidence +is recorded in `benchmarks/milestone1-2026-07-29.json`. The historical Milestone 0 harness remains behaviorally unchanged as comparison evidence; it only exposes shared fixture and measurement helpers to the Milestone 1 harness. @@ -308,6 +307,17 @@ incoming traversal, and measures current/stale/missing/corrupt render receipts p visualization lifecycle states. A maintained query-plan test prevents the incoming traversal temporary sort from returning. +#### Milestone closeout + +The complete repository gate passed with 138 tests and 77 subtests, zero Pyright diagnostics, +warning-strict execution, package builds, contract checks, and both benchmark smoke gates. The +clean 1,000-node benchmark passed all maintained thresholds and is interpreted in +`docs/MILESTONE_1_BASELINE.md`. Compatibility, measured decisions, limitations, and scope evidence +are frozen in `docs/MILESTONE_1_CLOSEOUT.md`. + +Milestone 1 made no storage rewrite, self-hosting change, production integration change, tag, or +release. + ### Initial design constraints - Full rebuild remains the recovery and equivalence oracle. diff --git a/benchmarks/README.md b/benchmarks/README.md index 744cfa2..05f5c55 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -36,6 +36,12 @@ mutate repository content. `time.perf_counter_ns()` for durations. The file is data, not a performance threshold. Later work must explain fixture or environment changes before comparing results. +`milestone1-2026-07-29.json` is the clean-tree fast-core baseline captured from commit +`6253c45a5eca01efa8c73ea3dfe4d85c55878ada`. Unlike the historical baseline, the Milestone 1 +harness enforces operation-specific p95 ceilings and fixed zero-work counter invariants. Its +human-readable interpretation is in +[`docs/MILESTONE_1_BASELINE.md`](../docs/MILESTONE_1_BASELINE.md). + The generic fixture exposes whole-source scaling. It does not replace the incremental adapter equivalence tests and does not claim to measure a portable graph renderer, because Milestone 0 has no portable graph-planning or graph-rendering contract. diff --git a/benchmarks/milestone1-2026-07-29.json b/benchmarks/milestone1-2026-07-29.json new file mode 100644 index 0000000..8dd6947 --- /dev/null +++ b/benchmarks/milestone1-2026-07-29.json @@ -0,0 +1,1381 @@ +{ + "benchmark": "docforge2_milestone1", + "environment": { + "implementation": "CPython", + "machine": "x86_64", + "platform": "Linux-7.1.3-200.nobara.fc44.x86_64-x86_64-with-glibc2.43", + "python": "3.14.6" + }, + "fixture": { + "edge_count": 999, + "kind": "synthetic_generic", + "node_count": 1000, + "source_file_count": 1000 + }, + "method": { + "clock": "time.perf_counter_ns", + "memory": "resource.getrusage(RUSAGE_SELF).ru_maxrss", + "memory_scope": "cumulative main-process high-water mark; detached viewer-worker memory excluded", + "percentile": "nearest-rank", + "response_size": "UTF-8 bytes of compact sorted JSON", + "samples": 10, + "warmups": 1, + "zero_work_counters": [ + "project_loads", + "source_files_parsed", + "source_bytes_parsed", + "adapter_projection_loads", + "adapter_source_extractions", + "index_builds", + "render_prepare_calls", + "render_output_bytes_built", + "render_output_bytes_hashed" + ] + }, + "operations": { + "backlinks_limit_20": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 18.455, + "median_ms": 18.039, + "min_ms": 17.683, + "p95_limit_ms": 100, + "p95_ms": 18.455, + "response_bytes": 1210, + "result_summary": { + "count": 1, + "limit": 20, + "staleness": "current", + "status": "ok", + "truncated": false, + "truncation_reason": null + }, + "samples": 10, + "validated_invocations": 11 + }, + "context_32k": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 25.867, + "median_ms": 25.395, + "min_ms": 25.038, + "p95_limit_ms": 250, + "p95_ms": 25.867, + "response_bytes": 13152, + "result_summary": { + "budget": 32000, + "entry_count": 20, + "estimated_tokens": 31974, + "omission_count": 0, + "pagination": { + "has_more": true, + "kind": "context.items", + "limit": 20, + "returned_count": 20, + "total_count": 1000 + }, + "staleness": "current", + "status": "ok", + "truncation_reason": "result_limit" + }, + "samples": 10, + "validated_invocations": 11 + }, + "dependencies_depth_8": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 19.061, + "median_ms": 18.117, + "min_ms": 17.939, + "p95_limit_ms": 100, + "p95_ms": 19.061, + "response_bytes": 2559, + "result_summary": { + "candidate_edges_consumed": 8, + "candidate_edges_limit": 10201, + "count": 8, + "limit": 100, + "staleness": "current", + "status": "ok", + "truncated": false, + "truncation_reason": null + }, + "samples": 10, + "validated_invocations": 11 + }, + "exact_node": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 18.577, + "median_ms": 17.887, + "min_ms": 17.597, + "p95_limit_ms": 50, + "p95_ms": 18.577, + "response_bytes": 1487, + "result_summary": { + "staleness": "current", + "status": "ok" + }, + "samples": 10, + "validated_invocations": 11 + }, + "filter_limit_20": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 19.534, + "median_ms": 18.274, + "min_ms": 18.152, + "p95_limit_ms": 100, + "p95_ms": 19.534, + "response_bytes": 8572, + "result_summary": { + "count": 20, + "limit": 20, + "staleness": "current", + "status": "ok", + "truncated": true, + "truncation_reason": "result_limit" + }, + "samples": 10, + "validated_invocations": 11 + }, + "impact_depth_8": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 18.716, + "median_ms": 18.059, + "min_ms": 17.757, + "p95_limit_ms": 100, + "p95_ms": 18.716, + "response_bytes": 2553, + "result_summary": { + "candidate_edges_consumed": 8, + "candidate_edges_limit": 10201, + "count": 8, + "limit": 100, + "staleness": "current", + "status": "ok", + "truncated": false, + "truncation_reason": null + }, + "samples": 10, + "validated_invocations": 11 + }, + "missing_node_error": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 18.32, + "median_ms": 18.17, + "min_ms": 17.866, + "p95_limit_ms": 50, + "p95_ms": 18.32, + "response_bytes": 1130, + "result_summary": { + "error_code": "missing_node", + "staleness": "current", + "status": "error" + }, + "samples": 10, + "validated_invocations": 11 + }, + "render_corrupt_receipt_status": { + "counter_expectations": { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 0, + "minimum": 0 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 18.511, + "median_ms": 18.007, + "min_ms": 17.804, + "p95_limit_ms": 50, + "p95_ms": 18.511, + "response_bytes": 1352, + "result_summary": { + "configured": true, + "outputs": [ + { + "reason": "receipt_corrupt", + "state": "unverified", + "view_id": "manual" + } + ], + "staleness": "current", + "state": "stale", + "status": "ok", + "verification": "receipt" + }, + "samples": 10, + "validated_invocations": 11 + }, + "render_missing_receipt_status": { + "counter_expectations": { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 0, + "minimum": 0 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 18.746, + "median_ms": 17.986, + "min_ms": 17.661, + "p95_limit_ms": 50, + "p95_ms": 18.746, + "response_bytes": 1352, + "result_summary": { + "configured": true, + "outputs": [ + { + "reason": "receipt_missing", + "state": "unverified", + "view_id": "manual" + } + ], + "staleness": "current", + "state": "stale", + "status": "ok", + "verification": "receipt" + }, + "samples": 10, + "validated_invocations": 11 + }, + "render_receipt_status": { + "counter_expectations": { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 0, + "minimum": 0 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 19.613, + "median_ms": 18.969, + "min_ms": 18.695, + "p95_limit_ms": 50, + "p95_ms": 19.613, + "response_bytes": 1603, + "result_summary": { + "configured": true, + "outputs": [ + { + "reason": null, + "state": "current", + "view_id": "manual" + } + ], + "staleness": "current", + "state": "current", + "status": "ok", + "verification": "receipt" + }, + "samples": 10, + "validated_invocations": 11 + }, + "render_stale_status": { + "counter_expectations": { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 0, + "minimum": 0 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 19.087, + "median_ms": 18.91, + "min_ms": 18.449, + "p95_limit_ms": 50, + "p95_ms": 19.087, + "response_bytes": 1551, + "result_summary": { + "configured": true, + "outputs": [ + { + "reason": "template_changed", + "state": "stale", + "view_id": "manual" + } + ], + "staleness": "current", + "state": "stale", + "status": "ok", + "verification": "receipt" + }, + "samples": 10, + "validated_invocations": 11 + }, + "search_limit_20": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 2, + "minimum": 2 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 19.884, + "median_ms": 19.518, + "min_ms": 19.359, + "p95_limit_ms": 100, + "p95_ms": 19.884, + "response_bytes": 11164, + "result_summary": { + "count": 20, + "limit": 20, + "staleness": "current", + "status": "ok", + "truncated": true, + "truncation_reason": "result_limit" + }, + "samples": 10, + "validated_invocations": 11 + }, + "visualization_current_status": { + "counter_expectations": { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 1 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 0, + "minimum": 0 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 1, + "minimum": 1 + }, + "viewer_manager_requests": { + "maximum": 1, + "minimum": 1 + } + }, + "max_ms": 10.836, + "median_ms": 9.875, + "min_ms": 9.456, + "p95_limit_ms": 50, + "p95_ms": 10.836, + "response_bytes": 1446, + "result_summary": { + "freshness": { + "index": "current", + "source": "current" + }, + "snapshot_state": "current", + "staleness": "current", + "state": "running", + "status": "ok" + }, + "samples": 10, + "validated_invocations": 11 + }, + "visualization_not_running_status": { + "counter_expectations": { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 1 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 0, + "minimum": 0 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 0, + "minimum": 0 + }, + "viewer_manager_requests": { + "maximum": 1, + "minimum": 1 + } + }, + "max_ms": 0.288, + "median_ms": 0.258, + "min_ms": 0.24, + "p95_limit_ms": 50, + "p95_ms": 0.288, + "response_bytes": 1008, + "result_summary": { + "freshness": { + "index": "unknown", + "source": "unknown" + }, + "snapshot_state": "unknown", + "staleness": "unknown", + "state": "not_running", + "status": "ok" + }, + "samples": 10, + "validated_invocations": 11 + }, + "visualization_stale_status": { + "counter_expectations": { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 1 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 0, + "minimum": 0 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 1, + "minimum": 1 + }, + "viewer_manager_requests": { + "maximum": 1, + "minimum": 1 + } + }, + "max_ms": 10.472, + "median_ms": 10.05, + "min_ms": 9.543, + "p95_limit_ms": 50, + "p95_ms": 10.472, + "response_bytes": 1438, + "result_summary": { + "freshness": { + "index": "stale", + "source": "current" + }, + "snapshot_state": "stale", + "staleness": "stale", + "state": "running", + "status": "ok" + }, + "samples": 10, + "validated_invocations": 11 + }, + "visualization_unavailable_status": { + "counter_expectations": { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 1 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 0, + "minimum": 0 + }, + "index_synchronizations": { + "maximum": 0, + "minimum": 0 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 0, + "minimum": 0 + }, + "viewer_manager_requests": { + "maximum": 1, + "minimum": 1 + } + }, + "max_ms": 0.081, + "median_ms": 0.053, + "min_ms": 0.032, + "p95_limit_ms": 50, + "p95_ms": 0.081, + "response_bytes": 1138, + "result_summary": { + "error_code": "visualization_manager_unavailable", + "staleness": "unknown", + "status": "error" + }, + "samples": 10, + "validated_invocations": 11 + }, + "warm_no_change_synchronize": { + "counter_expectations": { + "index_checks": 1, + "index_synchronizations": 1, + "viewer_manager_requests": 0 + }, + "counter_ranges": { + "adapter_projection_loads": { + "maximum": 0, + "minimum": 0 + }, + "adapter_source_extractions": { + "maximum": 0, + "minimum": 0 + }, + "index_builds": { + "maximum": 0, + "minimum": 0 + }, + "index_checks": { + "maximum": 1, + "minimum": 1 + }, + "index_synchronizations": { + "maximum": 1, + "minimum": 1 + }, + "project_loads": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_built": { + "maximum": 0, + "minimum": 0 + }, + "render_output_bytes_hashed": { + "maximum": 0, + "minimum": 0 + }, + "render_prepare_calls": { + "maximum": 0, + "minimum": 0 + }, + "source_bytes_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_files_parsed": { + "maximum": 0, + "minimum": 0 + }, + "source_generation_checks": { + "maximum": 1, + "minimum": 1 + }, + "viewer_manager_requests": { + "maximum": 0, + "minimum": 0 + } + }, + "max_ms": 9.324, + "median_ms": 9.192, + "min_ms": 9.067, + "p95_limit_ms": 100, + "p95_ms": 9.324, + "response_bytes": 1570, + "result_summary": { + "staleness": "current", + "status": "ok", + "synchronization_action": "current" + }, + "samples": 10, + "validated_invocations": 11 + } + }, + "process_peak_rss_kib": 940116, + "schema_version": 1, + "source": { + "dirty": false, + "revision": "6253c45a5eca01efa8c73ea3dfe4d85c55878ada" + } +} diff --git a/docs/MILESTONE_1_BASELINE.md b/docs/MILESTONE_1_BASELINE.md new file mode 100644 index 0000000..da8f151 --- /dev/null +++ b/docs/MILESTONE_1_BASELINE.md @@ -0,0 +1,102 @@ +# DocForge2 Milestone 1 baseline + +Milestone 1 removes repeated whole-project work from routine warm reads while retaining complete +loading and deep validation as recovery and equivalence oracles. The maintained machine-readable +evidence is +[`benchmarks/milestone1-2026-07-29.json`](../benchmarks/milestone1-2026-07-29.json), captured from +clean commit `6253c45a5eca01efa8c73ea3dfe4d85c55878ada`. + +## Environment and method + +- Platform: x86-64 Linux 7.1.3 with glibc 2.43. +- Python: CPython 3.14.6. +- Fixture: 1,000 Markdown files, 1,000 nodes, and 999 dependency edges. +- Samples: ten measured invocations after validated warmups. +- Duration clock: `time.perf_counter_ns()`. +- Percentile: nearest rank, so p95 is the maximum with ten samples. +- Response size: UTF-8 bytes of compact, sorted JSON. +- Process memory: cumulative main-process `RUSAGE_SELF` high-water mark. + +All canonical sources, caches, indexes, changesets, renders, and viewer state were created in a +disposable temporary directory. The run did not read WorldForge, ScrapeStation, legacy DocForge +indexes, or production MCP state. + +The benchmark validates every warmup and measured result. It also fails when a routine warm +operation performs a project load, parses canonical source, rebuilds an adapter projection, +extracts adapter sources, builds an index, prepares a render, constructs rendered output, or hashes +complete rendered output. + +## Maintained 1,000-node results + +| Operation | Median | p95 | Gate | Response | +|---|---:|---:|---:|---:| +| Warm no-change synchronization | 9.192 ms | 9.324 ms | 100 ms | 1,570 B | +| Exact node | 17.887 ms | 18.577 ms | 50 ms | 1,487 B | +| Missing-node error | 18.170 ms | 18.320 ms | 50 ms | 1,130 B | +| Search, limit 20 | 19.518 ms | 19.884 ms | 100 ms | 11,164 B | +| Filter, limit 20 | 18.274 ms | 19.534 ms | 100 ms | 8,572 B | +| Backlinks, limit 20 | 18.039 ms | 18.455 ms | 100 ms | 1,210 B | +| Dependencies, depth 8 | 18.117 ms | 19.061 ms | 100 ms | 2,559 B | +| Impact, depth 8 | 18.059 ms | 18.716 ms | 100 ms | 2,553 B | +| Context, 32,000-token budget, page 20 | 25.395 ms | 25.867 ms | 250 ms | 13,152 B | +| Current render receipt status | 18.969 ms | 19.613 ms | 50 ms | 1,603 B | +| Stale render receipt status | 18.910 ms | 19.087 ms | 50 ms | 1,551 B | +| Missing render receipt status | 17.986 ms | 18.746 ms | 50 ms | 1,352 B | +| Corrupt render receipt status | 18.007 ms | 18.511 ms | 50 ms | 1,352 B | +| Current visualization status | 9.875 ms | 10.836 ms | 50 ms | 1,446 B | +| Stale visualization status | 10.050 ms | 10.472 ms | 50 ms | 1,438 B | +| Not-running visualization status | 0.258 ms | 0.288 ms | 50 ms | 1,008 B | +| Unavailable visualization status | 0.053 ms | 0.081 ms | 50 ms | 1,138 B | + +Every measured p95 passed its maintained ceiling. Exact retrieval is 15.4 times faster than the +Milestone 0 median. Warm synchronization is 15.5 times faster. The paged context response is 17.2 +times faster and 19.6 times smaller than the inherited full response. + +The cumulative process peak was 940,116 KiB. This is not an operation-local steady-state value. It +includes fixture construction, all benchmark phases, and Python allocator high-water behavior. It +excludes the detached visualization worker. Milestone 0's isolated subprocess measurements remain +the better evidence for per-operation steady-state memory until a maintained operation-local memory +harness is added. + +## Work-proof counters + +Routine retrieval and status operations recorded: + +- Zero complete project loads. +- Zero canonical files or bytes parsed. +- Zero adapter projection loads and source extractions. +- Zero index builds. +- Zero render preparations, output bytes constructed, or complete output bytes hashed. +- One index check and two cheap source-generation checks for each pinned retrieval. +- One viewer-manager request for each running visualization-status query. + +No-change synchronization recorded one synchronization and no build. Receipt and visualization +status recorded no hidden synchronization. The counter contract is fixed, schema-validated, and +executed by the repository gate. + +## Meaning of the result + +The Milestone 0 evidence showed that SQLite queries were already fast after a generation was +pinned. Milestone 1 confirms that repeated source discovery, parsing, and validation were the +dominant cost. A versioned source-generation receipt, immutable SQLite read snapshot, and bounded +indexed operations remove that cost without changing graph authority or storage. + +The evidence still does not justify replacing SQLite. Complete project loading, complete index +checking, full adapter projection, and deep render validation remain independent truth and recovery +oracles. + +## Known limits + +- The context compiler still materializes its bounded selected graph before transport pagination. + A streaming planner requires separate scale evidence. +- Generic stat identities are cheap publication proofs, not cryptographic integrity scans. +- Legacy non-incremental adapters may not provide a cheap generation identity. +- Incremental adapter manifest, invalidation, extraction, and assembly are not yet measured at + 1,000-source scale. +- Pagination cursors detect corruption and stale generations. They are not authenticated + authorization tokens. +- An individually oversized context entry is returned as explicit hash-identified omission + evidence. Targeted retrieval is required for its content. +- The maintained process peak is cumulative and excludes detached worker memory. +- Manual and graph render plans do not exist until Milestone 3. + diff --git a/docs/MILESTONE_1_CLOSEOUT.md b/docs/MILESTONE_1_CLOSEOUT.md new file mode 100644 index 0000000..2c3b98c --- /dev/null +++ b/docs/MILESTONE_1_CLOSEOUT.md @@ -0,0 +1,99 @@ +# DocForge2 Milestone 1 closeout + +Milestone 1 establishes a fast, observable core without changing graph meaning, canonical +authority, the supported `docforge` identity, or the legacy adapter boundary. + +## Completed contracts + +- Generic projects publish a versioned source-generation receipt only after complete stable + verification. +- Routine reads validate file and membership-directory identities without parsing canonical + sources. +- Every indexed read uses one immutable read-only SQLite transaction pinned between source and + index identity checks. +- Dependency validation is linear in nodes and edges and uses an iterative deterministic cycle + check. +- Search, filtering, backlinks, dependency, impact, context, and changeset review results are + bounded independently of project size. +- Version-1 cursors bind the project, adapter, canonical generation, query, collection, and + position. Corrupt and stale cursors fail closed. +- Mutation tools preflight their minimum receipt and never report `result_too_large` after a + committed operation. +- Render status uses a publication receipt and performs no hidden render, source parse, index + rebuild, or repair. +- Visualization status separates lifecycle from source and index freshness and performs no hidden + SQLite validation or source parse. +- Request-local diagnostics expose fixed bounded stage timings and work counters without recording + source text, paths, node IDs, queries, or SQL. + +Complete loading, deep index checking, deep render status, and full adapter projection remain the +recovery and equivalence oracles. + +## Compatibility + +The distribution, import package, three executable names, existing CLI commands, existing MCP tool +names, and required arguments remain supported. New limits, cursors, deep-status switches, and +diagnostics are additive. + +A one-method `load_projection()` adapter remains supported. Incremental behavior remains optional. +The no-AST binding continues to reject Logic publication and every Logic retrieval surface, +including application refresh and live visualization. + +The disposable SQLite index schema is version 3. Version 2 indexes rebuild automatically. No +canonical source or stored proposal is migrated to satisfy the new index. + +## Verification + +The complete repository gate passed at clean commit +`6253c45a5eca01efa8c73ea3dfe4d85c55878ada`: + +- Ruff formatting and lint. +- HTML, rendered-manual HTML, CSS, and JavaScript lint. +- Pyright with zero diagnostics. +- Python compilation. +- Public-contract and no-AST checks. +- 138 tests and 77 subtests under warnings-as-errors. +- Lock and JavaScript dependency-tree checks. +- Wheel and source-distribution builds. +- Milestone 0 benchmark smoke. +- Milestone 1 counter and latency smoke. + +The maintained clean 1,000-node benchmark passed every latency and work-counter threshold. Exact +retrieval measured 18.577 ms p95. Paged 32,000-token context measured 25.867 ms p95. Warm no-change +synchronization measured 9.324 ms p95. Render receipt status measured 19.613 ms p95. +Visualization status measured 10.836 ms p95. + +Detailed evidence is in +[`MILESTONE_1_BASELINE.md`](MILESTONE_1_BASELINE.md) and +[`benchmarks/milestone1-2026-07-29.json`](../benchmarks/milestone1-2026-07-29.json). + +## Measured decisions + +SQLite remains the derived retrieval store. The benchmark demonstrates that whole-source +validation around SQLite, not SQLite retrieval itself, caused the inherited latency. No speculative +storage rewrite was made. + +Receipt caches remain disposable and fail closed. A missing, corrupt, incompatible, foreign, or +changed receipt falls back to the complete oracle or reports an explicit unverified state according +to the operation's safety contract. + +Pagination is transport state, not project authority. It adds no database and grants no +authorization. + +## Remaining weaknesses + +- Context selection is bounded but not yet streaming internally. +- Scaled incremental-adapter performance remains unmeasured. +- Generic cheap generation proof uses filesystem identity rather than content hashing on every + read. +- Legacy adapters without incremental state cannot always prove current identity cheaply. +- One oversized context entry requires targeted retrieval after an explicit omission. +- Operation-local and detached-worker memory need a maintained isolated harness. +- Tree-sitter and its JavaScript and C++ grammars remain mandatory package dependencies. +- `ManualRenderPlan`, `GraphViewPlan`, and independent renderer packages remain Milestone 3 work. + +## Scope confirmation + +Milestone 1 did not self-host DocForge2, change WorldForge or ScrapeStation, repoint a production +MCP integration, modify the legacy Forgejo repository, create a tag, or create a release. +