diff --git a/DEVELOPMENT_NOTES.md b/DEVELOPMENT_NOTES.md index 62e0c89..7c92773 100644 --- a/DEVELOPMENT_NOTES.md +++ b/DEVELOPMENT_NOTES.md @@ -267,6 +267,38 @@ workers, non-reuse, and zero-load status. The Milestone 1 benchmark now measures not-running, and unavailable visualization status separately with the same zero-work and 50 ms p95 gates as other receipt status operations. +#### Bounded pagination and exact large-result review + +The final Milestone 1 contract audit found that count limits and the global MCP output ceiling were +not sufficient. A 1,000-node context response already exceeded the normal 200,000-character tool +limit, and one allowed changeset operation can be larger than that limit. Returning +`result_too_large` kept transport bounded but stranded useful evidence. + +Version-1 pagination now uses canonical, base64url cursors with a domain-separated SHA-256 +corruption checksum. Cursors bind the project, adapter, canonical generation, semantic query, +collection hash, and position. They are deliberately unkeyed read tokens rather than authorization +credentials. Corrupt tokens fail as `invalid_cursor`; changed generations or collections fail as +`stale_cursor` with explicit pagination-restart remediation. + +Context transport flattens the compiler's deterministic selected entries followed by all explicit +omissions, then partitions each page back into the existing arrays. Both item count and exact +compact-JSON response size constrain packing. An individually oversized entry becomes a bounded, +hash-identified omission and advances the cursor, avoiding an infinite retry while preserving the +fact that evidence was excluded. + +Changeset list, inspection, validation, and diff reads preserve direct full-result defaults while +MCP uses bounded pages. Pages retain exact changeset identity and hash. Large operation pages +compact content-bearing fields into hashes and character counts. A single oversized structured +diff is serialized once as canonical ASCII JSON and returned through hash-bound chunks that +reconstruct the exact legacy `operations` and `changes` arrays. This solves transport growth +without lowering canonical changeset limits or adding cursor storage. + +The benchmark now validates zero-work and operation-specific counters for every warmup and measured +sample, records bounded semantic response summaries, covers filter, backlinks, outgoing and +incoming traversal, and measures current/stale/missing/corrupt render receipts plus all +visualization lifecycle states. A maintained query-plan test prevents the incoming traversal +temporary sort from returning. + ### Initial design constraints - Full rebuild remains the recovery and equivalence oracle. @@ -290,5 +322,6 @@ These are notes, not commitments: risks. - The stat identity is a cheap publication proof, not a cryptographic integrity scan. Full index validation remains the launch and query oracle. -- Large context and changeset payloads may need cursor pagination or compact immutable receipts. - The choice should follow actual client workflows rather than generic pagination machinery. +- Cursor authentication remains deliberately absent. If read cursors ever carry authority rather + than bounded positions, they will need a different versioned security contract and persisted key + lifecycle. diff --git a/benchmarks/README.md b/benchmarks/README.md index d6ec101..744cfa2 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -45,3 +45,12 @@ operations fail if they load a complete project, parse source files, reconstruct projection, extract adapter sources, build an index, prepare a render, construct rendered output, or hash complete rendered output. Its latency ceilings are the Milestone 1 targets, not claims about all hardware. + +Every warmup and measured invocation is validated. The recorded counter ranges also require one +index synchronization for the synchronization operation, no hidden synchronization for reads and +status, one index check for each retrieval snapshot, and exactly one manager request for viewer +status. The 1,000-node run records bounded semantic summaries for exact errors, search, filtering, +backlinks, both traversal directions, paged context, render receipt states, and visualization +freshness. The reported p95 uses the nearest-rank method; with ten samples it is the maximum. +`process_peak_rss_kib` is the cumulative main-process `RUSAGE_SELF` high-water mark and excludes the +detached viewer worker. diff --git a/docs/COMPATIBILITY.md b/docs/COMPATIBILITY.md index be98b17..c92004f 100644 --- a/docs/COMPATIBILITY.md +++ b/docs/COMPATIBILITY.md @@ -72,6 +72,10 @@ Milestone 0 preserves: version 3 adds a source-ordered incoming-edge index for bounded impact traversal. - Index-attestation schema version 1. - Incremental extraction-cache schema version 1. +- Read-pagination schema version 1. Existing tool names and required arguments are unchanged. + Context and changeset MCP reads accept optional limits and opaque generation-bound cursors. + Direct Python changeset methods and the ordinary CLI context command retain full legacy results + when pagination is not requested. Indexes, attestations, extraction caches, previews, and rendered artifacts are disposable. A schema change may rebuild them. Canonical project content and stored proposals may not be silently @@ -154,7 +158,10 @@ Milestone 0 records rather than redesigns these areas: - Tree-sitter and the JavaScript and C++ grammars remain mandatory installation dependencies even when their runtime modules are unused. - Several version strings and defaults remain duplicated. -- Large changeset results and context responses need compact receipt or pagination contracts. +- One individually oversized context entry is represented as explicit bounded omission evidence; + callers use targeted retrieval for that node. +- One individually oversized changeset diff is transported as reconstructable canonical-JSON + chunks. Cursors are corruption-detecting read tokens, not authenticated authorization tokens. - Manual planning is not separated from rendering. - There is no portable graph-planning or graph-rendering contract. - DocForge2 does not self-host its bootstrap documentation. diff --git a/docs/MCP_CONTRACT.md b/docs/MCP_CONTRACT.md index c3474a3..f81199d 100644 --- a/docs/MCP_CONTRACT.md +++ b/docs/MCP_CONTRACT.md @@ -49,6 +49,20 @@ project `max_results` policy. Omitted limits are still capped. Collection respon they were truncated. Traversal also reports whether truncation came from the result limit or its deterministic candidate-edge work budget; it does not scan or materialize the complete edge table. +`docforge_get_context` accepts optional `limit` and `cursor` arguments. Its page is one deterministic +stream containing selected entries first and explicit omission evidence second. The page receipt +reports the returned count, total evidence count, whether another page exists, and an opaque +generation-bound cursor. Packing also observes the configured MCP response limit. An individually +oversized entry advances as a hash-identified `response size limit` omission so pagination cannot +loop; targeted retrieval remains available for that node. The existing three-argument custom +context-provider contract is unchanged because pagination is applied after provider selection. + +Version-1 cursors are canonical JSON encoded as base64url with a domain-separated SHA-256 +corruption checksum. They are opaque and fail closed, but are not authenticated authorization +tokens. Cursors bind the project, adapter, source generation, operation parameters, collection +hash, and position. A changed generation or collection returns `stale_cursor` with +`restart_pagination`; DocForge never silently restarts at page one or combines generations. + Adapter-backed servers also validate their process-start implementation fingerprint before every tool. `adapter_restart_required` is stale but not synchronizable. Its remediation is `restart_project_server`; the current process does not reload project code, update Git staging, or @@ -101,6 +115,15 @@ Changeset listing returns draft and ready work by default. Stale, applied, and a remain available through an explicit status or history request. Applied and abandoned proposals no longer participate in overlap conflict detection. +Changeset list, inspection, validation, and diff reads accept optional `limit` and `cursor` +arguments. Direct Python and CLI methods still return their complete legacy result when pagination +is not requested. MCP defaults to bounded pages while preserving the exact changeset hash and +ordered operation sequence. Pages may contain fewer records than requested to remain inside the +response policy. Oversized inspection or validation pages return deterministic operation summaries +with hashes and character counts. An individually oversized diff becomes a sequence of +`canonical_json_chunk` pages; concatenating the ASCII chunks, decoding the JSON, and verifying its +payload hash reconstructs the exact `operations` and `changes` arrays without duplication. + Successful mutations return their existing full result while it fits the configured output limit. Before any proposal, preview, or canonical mutation, the server verifies that a minimum exact success receipt can fit. An impossible receipt fails with `result_too_large`, diff --git a/docs/USER_MANUAL.md b/docs/USER_MANUAL.md index 5de38a5..f161c56 100644 --- a/docs/USER_MANUAL.md +++ b/docs/USER_MANUAL.md @@ -477,7 +477,7 @@ filter [--family X] [--authority X] [--status X] [--tag X] [--limit N] backlinks NODE_ID [--relation RELATION] [--limit N] dependencies NODE_ID [--depth N] [--limit N] impact NODE_ID [--depth N] [--limit N] -context PROFILE [--budget N] +context PROFILE [--budget N] [--limit N] [--cursor OPAQUE] ``` ### Render and proposal commands @@ -641,6 +641,22 @@ an explicit `status="stale"` query for rebase decisions. Applied and abandoned p terminal history, remain available by status or history request, and no longer block new proposals against the same canonical base. +Context and changeset reads use version-1 continuation receipts when their evidence exceeds one +page. Follow `pagination.next_cursor` with the same tool and semantic arguments until +`pagination.has_more` is false. Page size may change between calls. Treat the cursor as opaque. +It is bound to the project, adapter, source generation, query, exact changeset hash, and collection +identity. `stale_cursor` means evidence changed between pages; discard prior pages and restart the +read instead of mixing generations. + +`docforge_get_context` paginates one ordered evidence stream: selected entries followed by explicit +omissions. An entry too large for one MCP response is represented by a bounded omission carrying +its node ID and detail hash, and the cursor advances. `docforge_list_changesets`, +`docforge_get_changeset`, `docforge_validate_changeset`, and `docforge_get_changeset_diff` accept +the same optional `limit` and `cursor` fields. Small results keep their familiar fields. Large +inspection pages may use hash summaries. A large diff may return `result_mode = +"canonical_json_chunk"`; concatenate the chunks in order and verify `payload_hash` before decoding +the reconstructed `operations` and `changes` object. + Canonical application records its terminal receipt immediately after the project-owned serializer verifies the new canonical state. A later index or render refresh failure is reported as degraded derived state with remediation, not as permission to apply the same canonical change again. diff --git a/schemas/result.schema.json b/schemas/result.schema.json index c8fcdc0..ef298a3 100644 --- a/schemas/result.schema.json +++ b/schemas/result.schema.json @@ -132,6 +132,41 @@ } }, "additionalProperties": false + }, + "pagination": { + "type": "object", + "required": [ + "schema_version", + "kind", + "returned_count", + "limit", + "total_count", + "has_more", + "next_cursor" + ], + "properties": { + "schema_version": { "const": 1 }, + "kind": { + "enum": [ + "context.items", + "changeset.list", + "changeset.inspect", + "changeset.validate", + "changeset.diff", + "changeset.diff-chunks" + ] + }, + "returned_count": { "type": "integer", "minimum": 0 }, + "limit": { "type": "integer", "minimum": 1 }, + "total_count": { "type": "integer", "minimum": 0 }, + "has_more": { "type": "boolean" }, + "next_cursor": { + "type": ["string", "null"], + "minLength": 1, + "maxLength": 8192 + } + }, + "additionalProperties": false } }, "oneOf": [ @@ -147,6 +182,7 @@ "pattern": "^[0-9a-f]{64}$" }, "adapter": { "type": "string" }, + "pagination": { "$ref": "#/$defs/pagination" }, "diagnostics": { "$ref": "#/$defs/diagnostics" } }, "additionalProperties": true diff --git a/src/docforge/changesets.py b/src/docforge/changesets.py index 20fb44b..74412ce 100644 --- a/src/docforge/changesets.py +++ b/src/docforge/changesets.py @@ -20,9 +20,12 @@ from .changeset_contract import ( ) from .errors import DocForgeError from .models import Edge, Node, ProjectService, ProjectSnapshot, ProposalWriter +from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt from .project import project_root_fingerprint from .proposal_projection import ProposalProjector +MAX_ABANDON_REASON_CHARS = 2_000 + class ChangesetStore: """One project-bound proposal store with an optional immutable writer identity.""" @@ -277,23 +280,37 @@ class ChangesetStore: }, ) - def validate(self, changeset_id: str) -> dict[str, object]: + def validate( + self, + changeset_id: str, + *, + limit: int | None = None, + cursor: str | None = None, + ) -> dict[str, object]: validate_id(changeset_id, "changeset_id") with self._lock(): snapshot, document, nodes, edges = self._validate_locked(changeset_id) - return self._result( + result = self._result( snapshot, document, valid=True, projected_node_count=len(nodes), projected_edge_count=len(edges), ) + return self._page_document_result( + result, + kind="changeset.validate", + limit=limit, + cursor=cursor, + ) def list_changesets( self, *, include_history: bool = True, status: str | None = None, + limit: int | None = None, + cursor: str | None = None, ) -> dict[str, object]: with self._lock(): snapshot = self.project.load() @@ -322,14 +339,93 @@ class ChangesetStore: "operation_count": len(document["operations"]), } ) - return self._base_result(snapshot, count=len(records), changesets=records) + result = self._base_result(snapshot, count=len(records), changesets=records) + if limit is None and cursor is None: + return result + selected_limit = page_limit( + limit, + default=20, + maximum=self.project.descriptor.limits.max_results, + ) + binding = { + "project_id": snapshot.descriptor.project_id, + "project_root_fingerprint": project_root_fingerprint(snapshot.descriptor.root), + "adapter": snapshot.descriptor.adapter, + "revision": snapshot.revision, + "source_hash": snapshot.source_hash, + "include_history": include_history, + "status": status, + "collection_hash": canonical_hash(records), + } + position = decode_cursor( + cursor, + kind="changeset.list", + binding=binding, + total_count=len(records), + ) + page = records[position : position + selected_limit] + while page: + candidate = { + **result, + "count": len(page), + "total_count": len(records), + "changesets": page, + "pagination": page_receipt( + kind="changeset.list", + binding=binding, + position=position, + count=len(page), + limit=selected_limit, + total_count=len(records), + ), + } + if self._encoded_length(candidate) <= self._safe_page_chars(): + return candidate + page.pop() + if position < len(records): + compact_record = self._compact_list_record(records[position]) + return { + **result, + "count": 1, + "total_count": len(records), + "changesets": [compact_record], + "result_mode": "changeset_summaries", + "pagination": page_receipt( + kind="changeset.list", + binding=binding, + position=position, + count=1, + limit=selected_limit, + total_count=len(records), + ), + } + return { + **result, + "count": 0, + "total_count": len(records), + "changesets": [], + "pagination": page_receipt( + kind="changeset.list", + binding=binding, + position=position, + count=0, + limit=selected_limit, + total_count=len(records), + ), + } - def inspect(self, changeset_id: str) -> dict[str, object]: + def inspect( + self, + changeset_id: str, + *, + limit: int | None = None, + cursor: str | None = None, + ) -> dict[str, object]: validate_id(changeset_id, "changeset_id") with self._lock(): document = self._read(self._path(changeset_id)) snapshot = self.project.load() - return self._result( + result = self._result( snapshot, document, base_state=self._base_state(document, snapshot), @@ -338,6 +434,12 @@ class ChangesetStore: self._base_state(document, snapshot), ), ) + return self._page_document_result( + result, + kind="changeset.inspect", + limit=limit, + cursor=cursor, + ) def rebase( self, @@ -398,8 +500,15 @@ class ChangesetStore: validate_id(changeset_id, "changeset_id") validate_hash(expected_changeset_hash, "expected_changeset_hash") - if not reason.strip(): + normalized_reason = reason.strip() + if not normalized_reason: raise DocForgeError("invalid_operation", "Abandon reason must be non-empty") + if len(normalized_reason) > MAX_ABANDON_REASON_CHARS: + raise DocForgeError( + "changeset_too_large", + "Abandon reason exceeds its character limit", + maximum=MAX_ABANDON_REASON_CHARS, + ) with self._lock(): document = self._read(self._path(changeset_id)) actual_hash = document_hash(document) @@ -418,7 +527,7 @@ class ChangesetStore: { "status": "abandoned", "changeset_hash": actual_hash, - "reason": reason.strip(), + "reason": normalized_reason, "revision": snapshot.revision, "source_hash": snapshot.source_hash, }, @@ -430,7 +539,13 @@ class ChangesetStore: lifecycle=receipt, ) - def diff(self, changeset_id: str) -> dict[str, object]: + def diff( + self, + changeset_id: str, + *, + limit: int | None = None, + cursor: str | None = None, + ) -> dict[str, object]: validate_id(changeset_id, "changeset_id") with self._lock(): snapshot, document, _, _ = self._validate_locked(changeset_id) @@ -453,7 +568,14 @@ class ChangesetStore: sorted(before_edges - edges), ) ) - return self._result(snapshot, document, valid=True, changes=changes) + result = self._result(snapshot, document, valid=True, changes=changes) + return self._page_document_result( + result, + kind="changeset.diff", + limit=limit, + cursor=cursor, + parallel_key="changes", + ) def projected_snapshot(self, changeset_id: str) -> tuple[ProjectSnapshot, str]: """Return a validated in-memory proposal projection for derived preview use.""" @@ -790,6 +912,219 @@ class ChangesetStore: **payload, ) + def _page_document_result( + self, + result: dict[str, object], + *, + kind: str, + limit: int | None, + cursor: str | None, + parallel_key: str | None = None, + ) -> dict[str, object]: + """Page bulky operation-aligned payloads while preserving direct full defaults.""" + + if limit is None and cursor is None: + return result + selected_limit = page_limit( + limit, + default=20, + maximum=self.project.descriptor.limits.max_results, + ) + operations_value = result.get("operations") + if not isinstance(operations_value, list): + raise DocForgeError( + "invalid_pagination_source", + "Changeset result does not contain a deterministic operation list", + ) + operations = cast(list[object], operations_value) + parallel: list[object] | None = None + if parallel_key is not None: + parallel_value = result.get(parallel_key) + if not isinstance(parallel_value, list): + raise DocForgeError( + "invalid_pagination_source", + "Changeset result does not contain an aligned detail list", + ) + parallel = cast(list[object], parallel_value) + if len(parallel) != len(operations): + raise DocForgeError( + "invalid_pagination_source", + "Changeset detail list is not aligned with its operations", + ) + binding = { + "project_id": result["project_id"], + "project_root_fingerprint": result["project_root_fingerprint"], + "revision": result["revision"], + "source_hash": result["source_hash"], + "changeset_id": result["changeset_id"], + "changeset_hash": result["changeset_hash"], + "adapter": result["adapter"], + "result_hash": canonical_hash( + { + "operations": operations, + **({parallel_key: parallel} if parallel_key is not None else {}), + } + ), + } + if ( + kind == "changeset.diff" + and parallel_key is not None + and parallel is not None + and self._encoded_length(result) > self._safe_page_chars() + ): + return self._page_json_chunks( + result, + operations=operations, + changes=parallel, + binding=binding, + cursor=cursor, + ) + position = decode_cursor( + cursor, + kind=kind, + binding=binding, + total_count=len(operations), + ) + page = operations[position : position + selected_limit] + paged = { + **result, + "operations": page, + "returned_operation_count": len(page), + "pagination": page_receipt( + kind=kind, + binding=binding, + position=position, + count=len(page), + limit=selected_limit, + total_count=len(operations), + ), + } + if parallel_key is not None and parallel is not None: + paged[parallel_key] = parallel[position : position + selected_limit] + if self._encoded_length(paged) > self._safe_page_chars(): + paged["operations"] = [ + self._operation_summary(item) + for item in operations[position : position + selected_limit] + ] + paged["result_mode"] = "operation_summaries" + paged["detail_tool"] = "docforge_get_changeset_diff" + return paged + + def _page_json_chunks( + self, + result: dict[str, object], + *, + operations: list[object], + changes: list[object], + binding: Mapping[str, object], + cursor: str | None, + ) -> dict[str, object]: + payload = {"operations": operations, "changes": changes} + encoded = json.dumps( + payload, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=True, + allow_nan=False, + ) + chunk_chars = max(512, min(64_000, self._safe_page_chars() // 2)) + chunks = [ + encoded[offset : offset + chunk_chars] for offset in range(0, len(encoded), chunk_chars) + ] or [""] + chunk_binding = { + **binding, + "payload_hash": canonical_hash(payload), + "chunk_chars": chunk_chars, + } + position = decode_cursor( + cursor, + kind="changeset.diff-chunks", + binding=chunk_binding, + total_count=len(chunks), + ) + compact = { + key: value for key, value in result.items() if key not in {"operations", "changes"} + } + return { + **compact, + "result_mode": "canonical_json_chunk", + "payload": "changeset_diff", + "payload_hash": chunk_binding["payload_hash"], + "payload_characters": len(encoded), + "chunk": { + "index": position, + "characters": len(chunks[position]), + "content": chunks[position], + }, + "pagination": page_receipt( + kind="changeset.diff-chunks", + binding=chunk_binding, + position=position, + count=1, + limit=1, + total_count=len(chunks), + ), + } + + @staticmethod + def _operation_summary(operation: object) -> dict[str, object]: + if not isinstance(operation, Mapping): + raise DocForgeError( + "invalid_pagination_source", + "Changeset operation is not a deterministic object", + ) + payload = cast(Mapping[str, object], operation) + summary = { + key: payload.get(key) + for key in ( + "sequence", + "operation", + "node_id", + "expected_content_hash", + "target_source", + ) + } + for key in ("metadata", "content", "relationship_changes", "rationale"): + value = payload.get(key) + summary[f"{key}_hash"] = canonical_hash(value) + summary[f"{key}_characters"] = len( + json.dumps( + value, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=True, + allow_nan=False, + ) + ) + return summary + + @staticmethod + def _compact_list_record(record: dict[str, object]) -> dict[str, object]: + lifecycle = record.get("lifecycle") + if not isinstance(lifecycle, Mapping): + return record + lifecycle_payload = cast(Mapping[str, object], lifecycle) + reason = lifecycle_payload.get("reason") + if not isinstance(reason, str): + return record + return { + **record, + "lifecycle": { + **lifecycle_payload, + "reason": { + "characters": len(reason), + "sha256": canonical_hash(reason), + }, + }, + } + + def _safe_page_chars(self) -> int: + return max(1_024, self.project.descriptor.limits.max_tool_output_chars - 2_048) + + @staticmethod + def _encoded_length(value: Mapping[str, object]) -> int: + return len(json.dumps(value, sort_keys=True, separators=(",", ":"))) + @staticmethod def _base_result(snapshot: ProjectSnapshot, **payload: object) -> dict[str, object]: return { diff --git a/src/docforge/cli.py b/src/docforge/cli.py index 2f35d10..ef4d749 100644 --- a/src/docforge/cli.py +++ b/src/docforge/cli.py @@ -63,6 +63,8 @@ def _parser() -> argparse.ArgumentParser: context = commands.add_parser("context") context.add_argument("profile") context.add_argument("--budget", type=int) + context.add_argument("--limit", type=int) + context.add_argument("--cursor") render = commands.add_parser("render") render.add_argument("view_id") render_status = commands.add_parser("render-status") @@ -175,6 +177,15 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]: limit=arguments.limit, ) if arguments.command == "context": + if arguments.limit is not None or arguments.cursor is not None: + from .mcp_server import DocForgeService + + return DocForgeService(project).context( + arguments.profile, + arguments.budget, + limit=arguments.limit, + cursor=arguments.cursor, + ) return compile_context(index, arguments.profile, arguments.budget) if arguments.command == "render": return RenderService(project).render(arguments.view_id) diff --git a/src/docforge/mcp_server.py b/src/docforge/mcp_server.py index 4d80a51..e4c721a 100644 --- a/src/docforge/mcp_server.py +++ b/src/docforge/mcp_server.py @@ -17,6 +17,7 @@ from .context import compile_context from .errors import DocForgeError from .index import ProjectIndex from .models import IncrementalStateProject, ProjectService, RuntimeValidatedProject +from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt from .project import Project, project_root_fingerprint from .rendering import RenderService from .telemetry import request, stage @@ -87,6 +88,7 @@ STALE_ERROR_CODES = frozenset( "content_conflict", "source_changed", "stale_adapter_source", + "stale_cursor", "stale_index", } ) @@ -488,6 +490,11 @@ class DocForgeService: "tool": "docforge_get_changeset", "arguments": {"changeset_id": ""}, } + if error.code == "stale_cursor": + return { + "retryable": True, + "action": "restart_pagination", + } return None def synchronize(self) -> dict[str, object]: @@ -707,12 +714,158 @@ class DocForgeService: operation_name="mcp.render_status", ) - def context(self, profile: str, budget: int | None = None) -> dict[str, Any]: + def context( + self, + profile: str, + budget: int | None = None, + *, + limit: int | None = None, + cursor: str | None = None, + ) -> dict[str, Any]: + def operation() -> dict[str, object]: + selected_limit = page_limit( + limit, + default=20, + maximum=self.project.descriptor.limits.max_results, + ) + result = self.context_provider(self.index, profile, budget) + return self._page_context_result( + result, + profile=profile, + selected_limit=selected_limit, + cursor=cursor, + ) + return self.invoke( - lambda: self.context_provider(self.index, profile, budget), + operation, operation_name="mcp.context", ) + def _page_context_result( + self, + result: dict[str, object], + *, + profile: str, + selected_limit: int, + cursor: str | None, + ) -> dict[str, object]: + entries_value = result.get("entries") + omissions_value = result.get("omissions") + if not isinstance(entries_value, list) or not isinstance(omissions_value, list): + raise DocForgeError( + "invalid_context_result", + "Context provider did not return deterministic entries and omissions", + ) + entries = cast(list[object], entries_value) + omissions = cast(list[object], omissions_value) + binding = { + "project_id": result.get("project_id"), + "project_root_fingerprint": result.get("project_root_fingerprint"), + "adapter": result.get("adapter"), + "revision": result.get("revision"), + "source_hash": result.get("source_hash"), + "profile": profile, + "budget": result.get("budget"), + "selection_hash": canonical_hash( + { + "entries": entries, + "omissions": omissions, + } + ), + } + evidence = [ + *(("entry", item) for item in entries), + *(("omission", item) for item in omissions), + ] + position = decode_cursor( + cursor, + kind="context.items", + binding=binding, + total_count=len(evidence), + ) + page_entries: list[object] = [] + page_omissions: list[object] = [] + consumed = 0 + truncation_reason: str | None = None + maximum = self.project.descriptor.limits.max_tool_output_chars + + def page_result() -> dict[str, object]: + pagination = page_receipt( + kind="context.items", + binding=binding, + position=position, + count=consumed, + limit=selected_limit, + total_count=len(evidence), + ) + return { + **result, + "entries": page_entries, + "omissions": page_omissions, + "entry_count": len(page_entries), + "omission_count": len(page_omissions), + "page_count": consumed, + "page_estimated_tokens": sum( + cast(int, cast(Mapping[str, object], item).get("estimated_tokens", 0)) + for item in page_entries + if isinstance(item, Mapping) + ), + "summary": { + "entry_count": len(entries), + "omission_count": len(omissions), + "evidence_count": len(evidence), + "estimated_tokens": result.get("estimated_tokens"), + }, + "truncation_reason": truncation_reason, + "next_cursor": pagination["next_cursor"], + "pagination": pagination, + } + + for kind, item in evidence[position:]: + if consumed >= selected_limit: + truncation_reason = "result_limit" + break + destination = page_entries if kind == "entry" else page_omissions + destination.append(item) + consumed += 1 + candidate = page_result() + decorated = { + **candidate, + "server_version": SERVER_VERSION, + "content_warning": CONTENT_WARNING, + "staleness": "current", + } + if self._encoded_length(decorated) <= maximum: + continue + destination.pop() + consumed -= 1 + truncation_reason = "response_limit" + if consumed == 0: + compact = self._oversized_context_omission(kind, item) + page_omissions.append(compact) + consumed = 1 + break + if position + consumed < len(evidence) and truncation_reason is None: + truncation_reason = "result_limit" + return page_result() + + @staticmethod + def _oversized_context_omission(kind: str, item: object) -> dict[str, object]: + node_id = "unknown" + hash_source = item + if isinstance(item, Mapping): + item_payload = cast(Mapping[str, object], item) + candidate = item_payload.get("node_id") + if isinstance(candidate, str) and candidate: + node_id = candidate[:256] + hash_source = dict(item_payload) + return { + "node_id": node_id, + "reason": "response size limit", + "original_evidence": kind, + "detail_hash": canonical_hash(hash_source), + } + def visualize( self, node_id: str | None = None, @@ -897,10 +1050,15 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC ) @server.tool(name="docforge_get_context") - def get_context(profile: str, budget: int | None = None) -> dict[str, Any]: + def get_context( + profile: str, + budget: int | None = None, + limit: int | None = None, + cursor: str | None = None, + ) -> dict[str, Any]: """Compile bounded cited context from one configured profile with explicit omissions.""" - return service.context(profile, budget) + return service.context(profile, budget, limit=limit, cursor=cursor) @server.tool(name="docforge_validate_project") def validate_project() -> dict[str, Any]: @@ -1000,6 +1158,8 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC def list_changesets( include_history: bool = False, status: str | None = None, + limit: int | None = 20, + cursor: str | None = None, ) -> dict[str, Any]: """List active proposals by default, with optional lifecycle history.""" @@ -1007,16 +1167,26 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC lambda: service.changesets.list_changesets( include_history=include_history, status=status, + limit=20 if limit is None else limit, + cursor=cursor, ), operation_name="mcp.changeset", ) @server.tool(name="docforge_get_changeset") - def get_changeset(changeset_id: str) -> dict[str, Any]: + def get_changeset( + changeset_id: str, + limit: int | None = 20, + cursor: str | None = None, + ) -> dict[str, Any]: """Inspect a stored proposal even when its canonical base has become stale.""" return service.invoke( - lambda: service.changesets.inspect(changeset_id), + lambda: service.changesets.inspect( + changeset_id, + limit=20 if limit is None else limit, + cursor=cursor, + ), operation_name="mcp.changeset", ) @@ -1225,20 +1395,36 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC ) @server.tool(name="docforge_validate_changeset") - def validate_changeset(changeset_id: str) -> dict[str, Any]: + def validate_changeset( + changeset_id: str, + limit: int | None = 20, + cursor: str | None = None, + ) -> dict[str, Any]: """Validate a proposal against its exact canonical base and other active proposals.""" return service.invoke( - lambda: service.changesets.validate(changeset_id), + lambda: service.changesets.validate( + changeset_id, + limit=20 if limit is None else limit, + cursor=cursor, + ), operation_name="mcp.changeset", ) @server.tool(name="docforge_get_changeset_diff") - def get_changeset_diff(changeset_id: str) -> dict[str, Any]: + def get_changeset_diff( + changeset_id: str, + limit: int | None = 20, + cursor: str | None = None, + ) -> dict[str, Any]: """Return a deterministic structured and textual diff without applying the proposal.""" return service.invoke( - lambda: service.changesets.diff(changeset_id), + lambda: service.changesets.diff( + changeset_id, + limit=20 if limit is None else limit, + cursor=cursor, + ), operation_name="mcp.changeset", ) diff --git a/src/docforge/pagination.py b/src/docforge/pagination.py new file mode 100644 index 0000000..a90916c --- /dev/null +++ b/src/docforge/pagination.py @@ -0,0 +1,163 @@ +"""Deterministic, generation-bound pagination cursors for bounded public results.""" + +from __future__ import annotations + +import base64 +import binascii +import hashlib +import hmac +import json +from collections.abc import Mapping +from typing import cast + +from .errors import DocForgeError + +CURSOR_SCHEMA_VERSION = 1 +MAX_CURSOR_CHARS = 8_192 +_CURSOR_DOMAIN = b"docforge-page-cursor-v1\0" +_CURSOR_KEYS = frozenset({"schema_version", "kind", "binding", "position", "checksum"}) + + +def canonical_hash(value: object) -> str: + """Hash one JSON-compatible value using DocForge's deterministic JSON form.""" + + try: + encoded = _canonical_bytes(value) + except (TypeError, ValueError) as error: + raise DocForgeError( + "invalid_pagination_source", + "Pagination source data is not deterministic JSON", + ) from error + return hashlib.sha256(encoded).hexdigest() + + +def page_limit(limit: int | None, *, default: int, maximum: int) -> int: + """Validate one additive page size against the project result policy.""" + + selected = default if limit is None else limit + if type(selected) is not int or selected < 1 or selected > maximum: + raise DocForgeError( + "invalid_limit", + "Page limit is outside the configured result limit", + maximum=maximum, + ) + return selected + + +def encode_cursor( + *, + kind: str, + binding: Mapping[str, object], + position: int, +) -> str: + """Encode a corruption-detecting cursor bound to an immutable result identity.""" + + if not kind or type(position) is not int or position < 0: + raise ValueError("Cursor kind and position must be valid") + body: dict[str, object] = { + "schema_version": CURSOR_SCHEMA_VERSION, + "kind": kind, + "binding": dict(binding), + "position": position, + } + checksum = hashlib.sha256(_CURSOR_DOMAIN + _canonical_bytes(body)).hexdigest() + envelope = {**body, "checksum": checksum} + return base64.urlsafe_b64encode(_canonical_bytes(envelope)).decode("ascii").rstrip("=") + + +def decode_cursor( + cursor: str | None, + *, + kind: str, + binding: Mapping[str, object], + total_count: int, +) -> int: + """Return a validated position, rejecting corrupt, foreign, or stale cursors.""" + + if cursor is None: + return 0 + if not cursor or len(cursor) > MAX_CURSOR_CHARS or not cursor.isascii(): + raise _invalid_cursor() + padding = "=" * (-len(cursor) % 4) + try: + raw = base64.b64decode( + (cursor + padding).encode("ascii"), + altchars=b"-_", + validate=True, + ) + parsed: object = json.loads(raw.decode("utf-8")) + except (binascii.Error, UnicodeDecodeError, json.JSONDecodeError): + raise _invalid_cursor() from None + if base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=") != cursor or not isinstance( + parsed, dict + ): + raise _invalid_cursor() + payload = cast(dict[str, object], parsed) + checksum = payload.get("checksum") + position = payload.get("position") + stored_binding = payload.get("binding") + if ( + frozenset(payload) != _CURSOR_KEYS + or payload.get("schema_version") != CURSOR_SCHEMA_VERSION + or payload.get("kind") != kind + or not isinstance(stored_binding, dict) + or type(position) is not int + or position < 0 + or position >= total_count + or not isinstance(checksum, str) + or len(checksum) != 64 + ): + raise _invalid_cursor() + body = {key: payload[key] for key in payload if key != "checksum"} + expected = hashlib.sha256(_CURSOR_DOMAIN + _canonical_bytes(body)).hexdigest() + if not hmac.compare_digest(checksum, expected): + raise _invalid_cursor() + if stored_binding != dict(binding): + raise DocForgeError( + "stale_cursor", + "Pagination cursor does not match the current result generation", + ) + return position + + +def page_receipt( + *, + kind: str, + binding: Mapping[str, object], + position: int, + count: int, + limit: int, + total_count: int, +) -> dict[str, object]: + """Return one bounded page receipt and the next generation-bound cursor.""" + + next_position = position + count + has_more = next_position < total_count + return { + "schema_version": CURSOR_SCHEMA_VERSION, + "kind": kind, + "returned_count": count, + "limit": limit, + "total_count": total_count, + "has_more": has_more, + "next_cursor": ( + encode_cursor(kind=kind, binding=binding, position=next_position) if has_more else None + ), + } + + +def _canonical_bytes(value: object) -> bytes: + return json.dumps( + value, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=True, + allow_nan=False, + ).encode("utf-8") + + +def _invalid_cursor() -> DocForgeError: + return DocForgeError( + "invalid_cursor", + "Pagination cursor is malformed or does not match its operation", + ) diff --git a/tests/test_cli.py b/tests/test_cli.py index 2338435..92e3cda 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -39,6 +39,20 @@ class DocForgeCliTests(unittest.TestCase): ] ) self.assertEqual(7, arguments.limit) + context = parser.parse_args( + [ + "--project-root", + "/tmp/project", + "context", + "active", + "--limit", + "7", + "--cursor", + "opaque", + ] + ) + self.assertEqual(7, context.limit) + self.assertEqual("opaque", context.cursor) def test_reindex_apply_and_visualization_commands_are_self_service(self) -> None: with tempfile.TemporaryDirectory() as directory: diff --git a/tests/test_core.py b/tests/test_core.py index 5c61b23..445e69b 100644 --- a/tests/test_core.py +++ b/tests/test_core.py @@ -378,6 +378,27 @@ class DocForgeCoreTests(unittest.TestCase): operation() self.assertEqual("invalid_limit", invalid.exception.code) + def test_incoming_traversal_uses_source_ordered_covering_index(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture("alpha", Path(directory)) + index = ProjectIndex(Project.open(root)) + index.build() + + connection = sqlite3.connect(index.path) + try: + plan = connection.execute( + "EXPLAIN QUERY PLAN " + "SELECT source_id, relation, target_id FROM edges " + "WHERE target_id = ? " + "ORDER BY source_id, relation, target_id LIMIT ?", + ("guide.foundation", 101), + ).fetchall() + finally: + connection.close() + details = " ".join(str(row[3]) for row in plan) + self.assertIn("edges_target_source", details) + self.assertNotIn("USE TEMP B-TREE", details) + def test_query_rechecks_source_identity_before_returning(self) -> None: with tempfile.TemporaryDirectory() as directory: root = self.copy_fixture("alpha", Path(directory)) diff --git a/tests/test_mcp_server.py b/tests/test_mcp_server.py index a9cd1d7..b945b29 100644 --- a/tests/test_mcp_server.py +++ b/tests/test_mcp_server.py @@ -16,6 +16,7 @@ from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import stdio_client from mcp.shared.memory import create_connected_server_and_client_session +from docforge.changesets import ChangesetStore from docforge.index import ProjectIndex from docforge.mcp_server import ( ALL_TOOLS, @@ -80,6 +81,16 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase): ): self.assertIn("limit", tools[name].inputSchema["properties"]) self.assertNotIn("limit", tools[name].inputSchema.get("required", [])) + for name in ( + "docforge_get_context", + "docforge_list_changesets", + "docforge_get_changeset", + "docforge_validate_changeset", + "docforge_get_changeset_diff", + ): + for field in ("limit", "cursor"): + self.assertIn(field, tools[name].inputSchema["properties"]) + self.assertNotIn(field, tools[name].inputSchema.get("required", [])) self.assertIn( "deep", tools["docforge_render_status"].inputSchema["properties"], @@ -110,6 +121,134 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase): self.assertEqual(0, diagnostics["counters"]["project_loads"]) self.assertEqual(0, diagnostics["counters"]["source_files_parsed"]) + async def test_context_pagination_is_complete_and_stale_cursors_fail_closed(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture("alpha", Path(directory)) + ProjectIndex(Project.open(root)).build() + async with create_connected_server_and_client_session( + create_server(root), raise_exceptions=True + ) as session: + first = await session.call_tool( + "docforge_get_context", + {"profile": "active", "limit": 1}, + ) + cursor = first.structuredContent["pagination"]["next_cursor"] + pages = [first.structuredContent] + while cursor is not None: + page = await session.call_tool( + "docforge_get_context", + { + "profile": "active", + "limit": 2, + "cursor": cursor, + }, + ) + pages.append(page.structuredContent) + cursor = page.structuredContent["pagination"]["next_cursor"] + + stale_cursor = first.structuredContent["pagination"]["next_cursor"] + workflow = root / "docs" / "content" / "workflow.md" + workflow.write_text( + workflow.read_text(encoding="utf-8") + "\nChanged between pages.\n", + encoding="utf-8", + ) + stale = await session.call_tool( + "docforge_get_context", + { + "profile": "active", + "limit": 1, + "cursor": stale_cursor, + }, + ) + + evidence = [ + *(("entry", item["node_id"]) for page in pages for item in page["entries"]), + *(("omission", item["node_id"]) for page in pages for item in page["omissions"]), + ] + self.assertEqual(len(evidence), pages[0]["summary"]["evidence_count"]) + self.assertEqual(len(evidence), len(set(evidence))) + self.assertEqual("stale_cursor", stale.structuredContent["error"]["code"]) + self.assertEqual("stale", stale.structuredContent["staleness"]) + self.assertEqual( + "restart_pagination", + stale.structuredContent["error"]["remediation"]["action"], + ) + + async def test_oversized_changeset_reads_return_exact_pages_and_chunks(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture("alpha", Path(directory)) + descriptor = root / ".docforge" / "project.toml" + descriptor.write_text( + descriptor.read_text(encoding="utf-8").replace( + "max_changeset_bytes = 100000", + "max_changeset_bytes = 100000\nmax_tool_output_chars = 20000", + ), + encoding="utf-8", + ) + project = Project.open(root) + ProjectIndex(project).build() + store = ChangesetStore(project, "alpha-editor") + created = store.create("mcp-chunked-diff") + foundation = next( + node for node in project.load().nodes if node.node_id == "guide.foundation" + ) + proposed = store.propose_update( + changeset_id="mcp-chunked-diff", + expected_changeset_hash=str(created["changeset_hash"]), + node_id=foundation.node_id, + expected_content_hash=foundation.content_hash, + metadata=None, + content="replacement " * 4_000, + relationship_changes=[], + rationale="Exercise bounded MCP diff reconstruction.", + ) + direct = store.diff("mcp-chunked-diff") + + async with create_connected_server_and_client_session( + create_server(root), raise_exceptions=True + ) as session: + inspected = await session.call_tool( + "docforge_get_changeset", + {"changeset_id": "mcp-chunked-diff"}, + ) + validated = await session.call_tool( + "docforge_validate_changeset", + {"changeset_id": "mcp-chunked-diff"}, + ) + cursor: str | None = None + chunks: list[str] = [] + while True: + page = await session.call_tool( + "docforge_get_changeset_diff", + { + "changeset_id": "mcp-chunked-diff", + "cursor": cursor, + }, + ) + self.assertLessEqual( + len(json.dumps(page.structuredContent, separators=(",", ":"))), + 20_000, + ) + chunks.append(page.structuredContent["chunk"]["content"]) + cursor = page.structuredContent["pagination"]["next_cursor"] + if cursor is None: + break + + self.assertEqual("operation_summaries", inspected.structuredContent["result_mode"]) + self.assertEqual("operation_summaries", validated.structuredContent["result_mode"]) + self.assertTrue(validated.structuredContent["valid"]) + self.assertEqual( + proposed["changeset_hash"], + validated.structuredContent["changeset_hash"], + ) + self.assertEqual( + { + "changes": direct["changes"], + "operations": direct["operations"], + }, + json.loads("".join(chunks)), + ) + async def test_no_ast_binding_preserves_adapter_and_blocks_logic(self) -> None: with tempfile.TemporaryDirectory() as directory: root = self.copy_fixture("alpha", Path(directory)) diff --git a/tests/test_pagination.py b/tests/test_pagination.py new file mode 100644 index 0000000..6705e43 --- /dev/null +++ b/tests/test_pagination.py @@ -0,0 +1,338 @@ +from __future__ import annotations + +import json +import shutil +import tempfile +import unittest +from pathlib import Path + +from jsonschema import Draft202012Validator + +from docforge.changesets import ChangesetStore +from docforge.errors import DocForgeError +from docforge.mcp_server import DocForgeService +from docforge.pagination import decode_cursor, encode_cursor +from docforge.project import Project, project_root_fingerprint + +ROOT = Path(__file__).resolve().parents[1] +FIXTURES = ROOT / "tests" / "fixtures" + + +class PaginationTests(unittest.TestCase): + def copy_fixture(self, destination: Path, *, max_tool_chars: int | None = None) -> Path: + root = destination / "alpha" + shutil.copytree(FIXTURES / "alpha", root) + if max_tool_chars is not None: + descriptor = root / ".docforge" / "project.toml" + text = descriptor.read_text(encoding="utf-8") + text = text.replace( + "max_changeset_bytes = 100000", + (f"max_changeset_bytes = 100000\nmax_tool_output_chars = {max_tool_chars}"), + ) + descriptor.write_text(text, encoding="utf-8") + return root + + def test_cursor_is_canonical_corruption_detecting_and_binding_bound(self) -> None: + binding = {"project_id": "alpha", "source_hash": "a" * 64} + cursor = encode_cursor(kind="context.items", binding=binding, position=2) + + self.assertEqual( + 2, + decode_cursor( + cursor, + kind="context.items", + binding=binding, + total_count=4, + ), + ) + with self.assertRaises(DocForgeError) as corrupt: + decode_cursor( + f"{cursor[:-1]}{'A' if cursor[-1] != 'A' else 'B'}", + kind="context.items", + binding=binding, + total_count=4, + ) + self.assertEqual("invalid_cursor", corrupt.exception.code) + with self.assertRaises(DocForgeError) as foreign: + decode_cursor( + cursor, + kind="context.items", + binding={**binding, "source_hash": "b" * 64}, + total_count=4, + ) + self.assertEqual("stale_cursor", foreign.exception.code) + with self.assertRaises(DocForgeError) as wrong_operation: + decode_cursor( + cursor, + kind="changeset.list", + binding=binding, + total_count=4, + ) + self.assertEqual("invalid_cursor", wrong_operation.exception.code) + + def test_context_pages_entries_then_omissions_without_loss(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture(Path(directory)) + project = Project.open(root) + source_hash = "a" * 64 + + def context_provider( + _index: object, profile: str, budget: int | None + ) -> dict[str, object]: + return { + "status": "ok", + "project_id": project.descriptor.project_id, + "project_root_fingerprint": project_root_fingerprint(root), + "adapter": project.descriptor.adapter, + "revision": "test-revision", + "source_hash": source_hash, + "profile": profile, + "budget": budget, + "estimated_tokens": 3, + "entries": [ + { + "node_id": "guide.one", + "reason": "required", + "estimated_tokens": 1, + "text": "one", + }, + { + "node_id": "guide.two", + "reason": "eligible", + "estimated_tokens": 2, + "text": "two", + }, + ], + "omissions": [ + {"node_id": "guide.three", "reason": "token budget"}, + {"node_id": "guide.four", "reason": "token budget"}, + ], + } + + service = DocForgeService(project, context_provider=context_provider) + result_validator = Draft202012Validator( + json.loads((ROOT / "schemas" / "result.schema.json").read_text(encoding="utf-8")) + ) + cursor: str | None = None + evidence: list[tuple[str, str]] = [] + limits = [1, 2, 1] + page_index = 0 + while True: + page = service.context( + "active", + 600, + limit=limits[min(page_index, len(limits) - 1)], + cursor=cursor, + ) + result_validator.validate(page) + evidence.extend( + ("entry", str(item["node_id"])) + for item in page["entries"] + if isinstance(item, dict) + ) + evidence.extend( + ("omission", str(item["node_id"])) + for item in page["omissions"] + if isinstance(item, dict) + ) + pagination = page["pagination"] + self.assertIsInstance(pagination, dict) + cursor = pagination["next_cursor"] + page_index += 1 + if cursor is None: + break + + self.assertEqual( + [ + ("entry", "guide.one"), + ("entry", "guide.two"), + ("omission", "guide.three"), + ("omission", "guide.four"), + ], + evidence, + ) + + def test_context_oversized_item_advances_as_a_bounded_omission(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture(Path(directory), max_tool_chars=4_000) + project = Project.open(root) + + def context_provider( + _index: object, profile: str, budget: int | None + ) -> dict[str, object]: + return { + "status": "ok", + "project_id": project.descriptor.project_id, + "project_root_fingerprint": project_root_fingerprint(root), + "adapter": project.descriptor.adapter, + "revision": "test-revision", + "source_hash": "a" * 64, + "profile": profile, + "budget": budget, + "estimated_tokens": 10_000, + "entries": [ + { + "node_id": "guide.oversized", + "reason": "required", + "estimated_tokens": 10_000, + "text": "x" * 20_000, + } + ], + "omissions": [], + } + + result = DocForgeService(project, context_provider=context_provider).context( + "active", + 600, + limit=1, + ) + + self.assertEqual("ok", result["status"]) + self.assertEqual([], result["entries"]) + self.assertEqual("response size limit", result["omissions"][0]["reason"]) + self.assertLess(len(json.dumps(result, separators=(",", ":"))), 4_000) + self.assertFalse(result["pagination"]["has_more"]) + + def test_page_limits_reject_boolean_zero_and_policy_overflow(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture(Path(directory)) + project = Project.open(root) + store = ChangesetStore(project, "alpha-editor") + store.create("bounded-page") + for limit in (True, 0, project.descriptor.limits.max_results + 1): + with self.subTest(limit=limit), self.assertRaises(DocForgeError) as invalid: + store.list_changesets(limit=limit) + self.assertEqual("invalid_limit", invalid.exception.code) + + def test_changeset_pages_preserve_full_direct_defaults_and_exact_order(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture(Path(directory)) + project = Project.open(root) + store = ChangesetStore(project, "alpha-editor") + created = store.create("paged-operations") + first = store.propose_update( + changeset_id="paged-operations", + expected_changeset_hash=str(created["changeset_hash"]), + node_id="guide.foundation", + expected_content_hash=next( + node.content_hash + for node in project.load().nodes + if node.node_id == "guide.foundation" + ), + metadata={"summary": "First paged update."}, + content=None, + relationship_changes=[], + rationale="First page.", + ) + store.propose_update( + changeset_id="paged-operations", + expected_changeset_hash=str(first["changeset_hash"]), + node_id="proof.validation", + expected_content_hash=next( + node.content_hash + for node in project.load().nodes + if node.node_id == "proof.validation" + ), + metadata={"summary": "Second paged update."}, + content=None, + relationship_changes=[], + rationale="Second page.", + ) + + full = store.inspect("paged-operations") + self.assertNotIn("pagination", full) + first_page = store.inspect("paged-operations", limit=1) + second_page = store.inspect( + "paged-operations", + limit=2, + cursor=str(first_page["pagination"]["next_cursor"]), + ) + combined = [*first_page["operations"], *second_page["operations"]] + self.assertEqual(full["operations"], combined) + self.assertEqual(full["changeset_hash"], second_page["changeset_hash"]) + + full_diff = store.diff("paged-operations") + diff_page = store.diff("paged-operations", limit=1) + diff_next = store.diff( + "paged-operations", + limit=1, + cursor=str(diff_page["pagination"]["next_cursor"]), + ) + self.assertEqual( + full_diff["changes"], + [*diff_page["changes"], *diff_next["changes"]], + ) + + def test_changeset_list_cursor_stales_when_collection_changes(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture(Path(directory)) + store = ChangesetStore(Project.open(root), "alpha-editor") + store.create("page-a") + store.create("page-b") + first = store.list_changesets(limit=1) + cursor = str(first["pagination"]["next_cursor"]) + second = store.list_changesets(limit=2, cursor=cursor) + self.assertEqual( + ["page-a", "page-b"], + [ + *( + item["changeset_id"] + for item in first["changesets"] + if isinstance(item, dict) + ), + *( + item["changeset_id"] + for item in second["changesets"] + if isinstance(item, dict) + ), + ], + ) + store.create("page-c") + + with self.assertRaises(DocForgeError) as stale: + store.list_changesets(limit=1, cursor=cursor) + self.assertEqual("stale_cursor", stale.exception.code) + + def test_oversized_diff_is_reconstructable_from_hash_bound_chunks(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = self.copy_fixture(Path(directory), max_tool_chars=20_000) + project = Project.open(root) + store = ChangesetStore(project, "alpha-editor") + created = store.create("chunked-diff") + store.propose_update( + changeset_id="chunked-diff", + expected_changeset_hash=str(created["changeset_hash"]), + node_id="guide.foundation", + expected_content_hash=next( + node.content_hash + for node in project.load().nodes + if node.node_id == "guide.foundation" + ), + metadata=None, + content="replacement " * 4_000, + relationship_changes=[], + rationale="Exercise deterministic chunk transport.", + ) + full = store.diff("chunked-diff") + cursor: str | None = None + chunks: list[str] = [] + while True: + page = store.diff("chunked-diff", limit=20, cursor=cursor) + self.assertEqual("canonical_json_chunk", page["result_mode"]) + chunks.append(page["chunk"]["content"]) + cursor = page["pagination"]["next_cursor"] + if cursor is None: + break + reconstructed = json.loads("".join(chunks)) + + self.assertEqual( + { + "changes": full["changes"], + "operations": full["operations"], + }, + reconstructed, + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/milestone1_benchmark.py b/tools/milestone1_benchmark.py index 6d72f31..4e1c46e 100644 --- a/tools/milestone1_benchmark.py +++ b/tools/milestone1_benchmark.py @@ -21,7 +21,6 @@ from milestone0_baseline import ( write_synthetic_project, ) -from docforge.context import compile_context from docforge.index import ProjectIndex from docforge.mcp_server import DocForgeService from docforge.project import Project @@ -80,27 +79,145 @@ def _diagnostics(result: object) -> Mapping[str, object]: return diagnostics +def _result_summary(result: Mapping[str, object]) -> dict[str, object]: + """Retain bounded semantic evidence without copying primary result payloads.""" + + summary: dict[str, object] = {} + for key in ( + "status", + "count", + "limit", + "truncated", + "truncation_reason", + "candidate_edges_consumed", + "candidate_edges_limit", + "budget", + "estimated_tokens", + "state", + "verification", + "configured", + "snapshot_state", + "staleness", + ): + if key in result: + summary[key] = result[key] + error = result.get("error") + if isinstance(error, Mapping): + error_payload = cast(Mapping[str, object], error) + if isinstance(error_payload.get("code"), str): + summary["error_code"] = error_payload["code"] + synchronization = result.get("synchronization") + if isinstance(synchronization, Mapping): + synchronization_payload = cast(Mapping[str, object], synchronization) + if isinstance(synchronization_payload.get("action"), str): + summary["synchronization_action"] = synchronization_payload["action"] + freshness = result.get("freshness") + if isinstance(freshness, Mapping): + freshness_payload = cast(Mapping[str, object], freshness) + summary["freshness"] = { + key: freshness_payload[key] + for key in ("index", "source") + if isinstance(freshness_payload.get(key), str) + } + outputs = result.get("outputs") + if isinstance(outputs, list): + summarized_outputs: list[dict[str, object]] = [] + for item in cast(list[object], outputs)[:10]: + if not isinstance(item, Mapping): + continue + item_payload = cast(Mapping[str, object], item) + summarized_outputs.append( + { + key: item_payload[key] + for key in ("view_id", "state", "reason") + if key in item_payload + } + ) + summary["outputs"] = summarized_outputs + entries = result.get("entries") + if isinstance(entries, list): + summary["entry_count"] = len(cast(list[object], entries)) + omissions = result.get("omissions") + if isinstance(omissions, list): + summary["omission_count"] = len(cast(list[object], omissions)) + pagination = result.get("pagination") + if isinstance(pagination, Mapping): + pagination_payload = cast(Mapping[str, object], pagination) + summary["pagination"] = { + key: pagination_payload[key] + for key in ("kind", "returned_count", "limit", "total_count", "has_more") + if key in pagination_payload + } + return summary + + def _operation( operation: Callable[[], dict[str, object]], *, samples: int, p95_limit_ms: float, expected_status: str = "ok", + expected_counters: Mapping[str, int], ) -> dict[str, object]: - measurement, last = measure_operation(operation, samples=samples) + diagnostics_records: list[Mapping[str, object]] = [] + result_summaries: list[dict[str, object]] = [] + + def validated_operation() -> dict[str, object]: + result = operation() + diagnostics = _diagnostics(result) + counters_value = diagnostics["counters"] + if not isinstance(counters_value, Mapping): + raise RuntimeError("Measured diagnostics did not return counters") + counters = cast(Mapping[str, object], counters_value) + for counter, expected in expected_counters.items(): + if counters.get(counter) != expected: + raise RuntimeError( + f"Warm operation expected {counter}={expected}, " + f"received {counters.get(counter)!r}" + ) + diagnostics_records.append(diagnostics) + result_summaries.append(_result_summary(result)) + return result + + measurement, last = measure_operation(validated_operation, samples=samples) if not isinstance(last, Mapping): raise RuntimeError(f"Measured operation did not return status={expected_status}") last_payload = cast(Mapping[str, object], last) if last_payload.get("status") != expected_status: raise RuntimeError(f"Measured operation did not return status={expected_status}") - diagnostics = _diagnostics(last_payload) + for summary in result_summaries: + if summary.get("status") != expected_status: + raise RuntimeError(f"Measured operation did not return status={expected_status}") p95_ms = float(cast(float, measurement["p95_ms"])) if p95_ms > p95_limit_ms: raise RuntimeError(f"Warm operation p95 {p95_ms:.3f} ms exceeds {p95_limit_ms:.3f} ms") + counter_names = sorted( + { + key + for diagnostics in diagnostics_records + for key in cast(Mapping[str, object], diagnostics["counters"]) + } + ) + counter_ranges = { + counter: { + "minimum": min( + cast(int, cast(Mapping[str, object], record["counters"])[counter]) + for record in diagnostics_records + ), + "maximum": max( + cast(int, cast(Mapping[str, object], record["counters"])[counter]) + for record in diagnostics_records + ), + } + for counter in counter_names + } return { **measurement, "p95_limit_ms": p95_limit_ms, - "diagnostics": diagnostics, + "validated_invocations": len(diagnostics_records), + "counter_expectations": dict(sorted(expected_counters.items())), + "counter_ranges": counter_ranges, + "result_summary": result_summaries[-1], } @@ -110,11 +227,33 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]: RenderService(project).render("manual") service = DocForgeService(project, diagnostics=True) target = synthetic_node_id(node_count - 1) + backlink_target = synthetic_node_id(node_count - 2) + first = synthetic_node_id(0) + read_counters = { + "index_checks": 1, + "index_synchronizations": 0, + "viewer_manager_requests": 0, + } + status_counters = { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 0, + } + visualization_counters = { + "index_checks": 0, + "index_synchronizations": 0, + "viewer_manager_requests": 1, + } operations = { "warm_no_change_synchronize": _operation( service.synchronize, samples=samples, p95_limit_ms=100, + expected_counters={ + "index_checks": 1, + "index_synchronizations": 1, + "viewer_manager_requests": 0, + }, ), "exact_node": _operation( lambda: service.invoke( @@ -123,6 +262,7 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]: ), samples=samples, p95_limit_ms=50, + expected_counters=read_counters, ), "missing_node_error": _operation( lambda: service.invoke( @@ -132,6 +272,7 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]: samples=samples, p95_limit_ms=50, expected_status="error", + expected_counters=read_counters, ), "search_limit_20": _operation( lambda: service.invoke( @@ -140,6 +281,25 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]: ), samples=samples, p95_limit_ms=100, + expected_counters=read_counters, + ), + "filter_limit_20": _operation( + lambda: service.invoke( + lambda: service.index.filter_nodes(family="guide", limit=20), + operation_name="mcp.filter", + ), + samples=samples, + p95_limit_ms=100, + expected_counters=read_counters, + ), + "backlinks_limit_20": _operation( + lambda: service.invoke( + lambda: service.index.backlinks(backlink_target, limit=20), + operation_name="mcp.backlinks", + ), + samples=samples, + p95_limit_ms=100, + expected_counters=read_counters, ), "dependencies_depth_8": _operation( lambda: service.invoke( @@ -148,21 +308,58 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]: ), samples=samples, p95_limit_ms=100, + expected_counters=read_counters, ), - "context_32k": _operation( + "impact_depth_8": _operation( lambda: service.invoke( - lambda: compile_context(service.index, "active", 32_000), - operation_name="mcp.context", + lambda: service.index.impact(first, depth=8, limit=100), + operation_name="mcp.impact", ), samples=samples, + p95_limit_ms=100, + expected_counters=read_counters, + ), + "context_32k": _operation( + lambda: service.context("active", 32_000, limit=20), + samples=samples, p95_limit_ms=250, + expected_counters=read_counters, ), "render_receipt_status": _operation( lambda: service.render_status("manual"), samples=samples, p95_limit_ms=50, + expected_counters=status_counters, ), } + template = root / "docs" / "templates" / "manual.html" + original_template = template.read_bytes() + template.write_bytes(original_template + b"\n") + operations["render_stale_status"] = _operation( + lambda: service.render_status("manual"), + samples=samples, + p95_limit_ms=50, + expected_counters=status_counters, + ) + template.write_bytes(original_template) + RenderService(project).render("manual") + receipt_path = root / ".docforge" / "cache" / "render-receipts" / "manual.json" + receipt_path.unlink() + operations["render_missing_receipt_status"] = _operation( + lambda: service.render_status("manual"), + samples=samples, + p95_limit_ms=50, + expected_counters=status_counters, + ) + RenderService(project).render("manual") + receipt_path.write_text("{", encoding="utf-8") + operations["render_corrupt_receipt_status"] = _operation( + lambda: service.render_status("manual"), + samples=samples, + p95_limit_ms=50, + expected_counters=status_counters, + ) + RenderService(project).render("manual") state_path = root / ".docforge" / "benchmark-viewer-manager.json" manager = ViewerManager(state_path, check_interval_seconds=0.02) manager_thread = threading.Thread(target=manager.serve_forever, daemon=True) @@ -181,6 +378,7 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]: service.visualization_status, samples=samples, p95_limit_ms=50, + expected_counters=visualization_counters, ) with service.index.path.open("ab") as stream: stream.write(b"\n") @@ -188,12 +386,14 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]: service.visualization_status, samples=samples, p95_limit_ms=50, + expected_counters=visualization_counters, ) service.stop_visualization() operations["visualization_not_running_status"] = _operation( service.visualization_status, samples=samples, p95_limit_ms=50, + expected_counters=visualization_counters, ) finally: manager.shutdown() @@ -204,6 +404,7 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]: samples=samples, p95_limit_ms=50, expected_status="error", + expected_counters=visualization_counters, ) return { "fixture": { @@ -243,8 +444,13 @@ def main() -> int: "method": { "clock": "time.perf_counter_ns", "memory": "resource.getrusage(RUSAGE_SELF).ru_maxrss", + "memory_scope": ( + "cumulative main-process high-water mark; detached viewer-worker memory excluded" + ), "response_size": "UTF-8 bytes of compact sorted JSON", "samples": arguments.samples, + "warmups": 1, + "percentile": "nearest-rank", "zero_work_counters": list(ZERO_WORK_COUNTERS), }, **measurement,