Bound paged retrieval responses

DEVELOPMENT_NOTES.md

@@ -267,6 +267,38 @@ workers, non-reuse, and zero-load status. The Milestone 1 benchmark now measures
 not-running, and unavailable visualization status separately with the same zero-work and 50 ms p95
 gates as other receipt status operations.
 
+#### Bounded pagination and exact large-result review
+
+The final Milestone 1 contract audit found that count limits and the global MCP output ceiling were
+not sufficient. A 1,000-node context response already exceeded the normal 200,000-character tool
+limit, and one allowed changeset operation can be larger than that limit. Returning
+`result_too_large` kept transport bounded but stranded useful evidence.
+
+Version-1 pagination now uses canonical, base64url cursors with a domain-separated SHA-256
+corruption checksum. Cursors bind the project, adapter, canonical generation, semantic query,
+collection hash, and position. They are deliberately unkeyed read tokens rather than authorization
+credentials. Corrupt tokens fail as `invalid_cursor`; changed generations or collections fail as
+`stale_cursor` with explicit pagination-restart remediation.
+
+Context transport flattens the compiler's deterministic selected entries followed by all explicit
+omissions, then partitions each page back into the existing arrays. Both item count and exact
+compact-JSON response size constrain packing. An individually oversized entry becomes a bounded,
+hash-identified omission and advances the cursor, avoiding an infinite retry while preserving the
+fact that evidence was excluded.
+
+Changeset list, inspection, validation, and diff reads preserve direct full-result defaults while
+MCP uses bounded pages. Pages retain exact changeset identity and hash. Large operation pages
+compact content-bearing fields into hashes and character counts. A single oversized structured
+diff is serialized once as canonical ASCII JSON and returned through hash-bound chunks that
+reconstruct the exact legacy `operations` and `changes` arrays. This solves transport growth
+without lowering canonical changeset limits or adding cursor storage.
+
+The benchmark now validates zero-work and operation-specific counters for every warmup and measured
+sample, records bounded semantic response summaries, covers filter, backlinks, outgoing and
+incoming traversal, and measures current/stale/missing/corrupt render receipts plus all
+visualization lifecycle states. A maintained query-plan test prevents the incoming traversal
+temporary sort from returning.
+
 ### Initial design constraints
 
 - Full rebuild remains the recovery and equivalence oracle.
@@ -290,5 +322,6 @@ These are notes, not commitments:
   risks.
 - The stat identity is a cheap publication proof, not a cryptographic integrity scan. Full index
   validation remains the launch and query oracle.
-- Large context and changeset payloads may need cursor pagination or compact immutable receipts.
+- Cursor authentication remains deliberately absent. If read cursors ever carry authority rather
-  The choice should follow actual client workflows rather than generic pagination machinery.
+  than bounded positions, they will need a different versioned security contract and persisted key
+  lifecycle.

benchmarks/README.md

@@ -45,3 +45,12 @@ operations fail if they load a complete project, parse source files, reconstruct
 projection, extract adapter sources, build an index, prepare a render, construct rendered output,
 or hash complete rendered output. Its latency ceilings are the Milestone 1 targets, not claims
 about all hardware.
+
+Every warmup and measured invocation is validated. The recorded counter ranges also require one
+index synchronization for the synchronization operation, no hidden synchronization for reads and
+status, one index check for each retrieval snapshot, and exactly one manager request for viewer
+status. The 1,000-node run records bounded semantic summaries for exact errors, search, filtering,
+backlinks, both traversal directions, paged context, render receipt states, and visualization
+freshness. The reported p95 uses the nearest-rank method; with ten samples it is the maximum.
+`process_peak_rss_kib` is the cumulative main-process `RUSAGE_SELF` high-water mark and excludes the
+detached viewer worker.

docs/COMPATIBILITY.md

@@ -72,6 +72,10 @@ Milestone 0 preserves:
   version 3 adds a source-ordered incoming-edge index for bounded impact traversal.
 - Index-attestation schema version 1.
 - Incremental extraction-cache schema version 1.
+- Read-pagination schema version 1. Existing tool names and required arguments are unchanged.
+  Context and changeset MCP reads accept optional limits and opaque generation-bound cursors.
+  Direct Python changeset methods and the ordinary CLI context command retain full legacy results
+  when pagination is not requested.
 
 Indexes, attestations, extraction caches, previews, and rendered artifacts are disposable. A schema
 change may rebuild them. Canonical project content and stored proposals may not be silently
@@ -154,7 +158,10 @@ Milestone 0 records rather than redesigns these areas:
 - Tree-sitter and the JavaScript and C++ grammars remain mandatory installation dependencies even
   when their runtime modules are unused.
 - Several version strings and defaults remain duplicated.
-- Large changeset results and context responses need compact receipt or pagination contracts.
+- One individually oversized context entry is represented as explicit bounded omission evidence;
+  callers use targeted retrieval for that node.
+- One individually oversized changeset diff is transported as reconstructable canonical-JSON
+  chunks. Cursors are corruption-detecting read tokens, not authenticated authorization tokens.
 - Manual planning is not separated from rendering.
 - There is no portable graph-planning or graph-rendering contract.
 - DocForge2 does not self-host its bootstrap documentation.

docs/MCP_CONTRACT.md

@@ -49,6 +49,20 @@ project `max_results` policy. Omitted limits are still capped. Collection respon
 they were truncated. Traversal also reports whether truncation came from the result limit or its
 deterministic candidate-edge work budget; it does not scan or materialize the complete edge table.
 
+`docforge_get_context` accepts optional `limit` and `cursor` arguments. Its page is one deterministic
+stream containing selected entries first and explicit omission evidence second. The page receipt
+reports the returned count, total evidence count, whether another page exists, and an opaque
+generation-bound cursor. Packing also observes the configured MCP response limit. An individually
+oversized entry advances as a hash-identified `response size limit` omission so pagination cannot
+loop; targeted retrieval remains available for that node. The existing three-argument custom
+context-provider contract is unchanged because pagination is applied after provider selection.
+
+Version-1 cursors are canonical JSON encoded as base64url with a domain-separated SHA-256
+corruption checksum. They are opaque and fail closed, but are not authenticated authorization
+tokens. Cursors bind the project, adapter, source generation, operation parameters, collection
+hash, and position. A changed generation or collection returns `stale_cursor` with
+`restart_pagination`; DocForge never silently restarts at page one or combines generations.
+
 Adapter-backed servers also validate their process-start implementation fingerprint before every
 tool. `adapter_restart_required` is stale but not synchronizable. Its remediation is
 `restart_project_server`; the current process does not reload project code, update Git staging, or
@@ -101,6 +115,15 @@ Changeset listing returns draft and ready work by default. Stale, applied, and a
 remain available through an explicit status or history request. Applied and abandoned proposals no
 longer participate in overlap conflict detection.
 
+Changeset list, inspection, validation, and diff reads accept optional `limit` and `cursor`
+arguments. Direct Python and CLI methods still return their complete legacy result when pagination
+is not requested. MCP defaults to bounded pages while preserving the exact changeset hash and
+ordered operation sequence. Pages may contain fewer records than requested to remain inside the
+response policy. Oversized inspection or validation pages return deterministic operation summaries
+with hashes and character counts. An individually oversized diff becomes a sequence of
+`canonical_json_chunk` pages; concatenating the ASCII chunks, decoding the JSON, and verifying its
+payload hash reconstructs the exact `operations` and `changes` arrays without duplication.
+
 Successful mutations return their existing full result while it fits the configured output limit.
 Before any proposal, preview, or canonical mutation, the server verifies that a minimum exact
 success receipt can fit. An impossible receipt fails with `result_too_large`,

docs/USER_MANUAL.md

@@ -477,7 +477,7 @@ filter [--family X] [--authority X] [--status X] [--tag X] [--limit N]
 backlinks NODE_ID [--relation RELATION] [--limit N]
 dependencies NODE_ID [--depth N] [--limit N]
 impact NODE_ID [--depth N] [--limit N]
-context PROFILE [--budget N]
+context PROFILE [--budget N] [--limit N] [--cursor OPAQUE]
 ```
 
 ### Render and proposal commands
@@ -641,6 +641,22 @@ an explicit `status="stale"` query for rebase decisions. Applied and abandoned p
 terminal history, remain available by status or history request, and no longer block new proposals
 against the same canonical base.
 
+Context and changeset reads use version-1 continuation receipts when their evidence exceeds one
+page. Follow `pagination.next_cursor` with the same tool and semantic arguments until
+`pagination.has_more` is false. Page size may change between calls. Treat the cursor as opaque.
+It is bound to the project, adapter, source generation, query, exact changeset hash, and collection
+identity. `stale_cursor` means evidence changed between pages; discard prior pages and restart the
+read instead of mixing generations.
+
+`docforge_get_context` paginates one ordered evidence stream: selected entries followed by explicit
+omissions. An entry too large for one MCP response is represented by a bounded omission carrying
+its node ID and detail hash, and the cursor advances. `docforge_list_changesets`,
+`docforge_get_changeset`, `docforge_validate_changeset`, and `docforge_get_changeset_diff` accept
+the same optional `limit` and `cursor` fields. Small results keep their familiar fields. Large
+inspection pages may use hash summaries. A large diff may return `result_mode =
+"canonical_json_chunk"`; concatenate the chunks in order and verify `payload_hash` before decoding
+the reconstructed `operations` and `changes` object.
+
 Canonical application records its terminal receipt immediately after the project-owned serializer
 verifies the new canonical state. A later index or render refresh failure is reported as degraded
 derived state with remediation, not as permission to apply the same canonical change again.

schemas/result.schema.json

@@ -132,6 +132,41 @@
         }
       },
       "additionalProperties": false
+    },
+    "pagination": {
+      "type": "object",
+      "required": [
+        "schema_version",
+        "kind",
+        "returned_count",
+        "limit",
+        "total_count",
+        "has_more",
+        "next_cursor"
+      ],
+      "properties": {
+        "schema_version": { "const": 1 },
+        "kind": {
+          "enum": [
+            "context.items",
+            "changeset.list",
+            "changeset.inspect",
+            "changeset.validate",
+            "changeset.diff",
+            "changeset.diff-chunks"
+          ]
+        },
+        "returned_count": { "type": "integer", "minimum": 0 },
+        "limit": { "type": "integer", "minimum": 1 },
+        "total_count": { "type": "integer", "minimum": 0 },
+        "has_more": { "type": "boolean" },
+        "next_cursor": {
+          "type": ["string", "null"],
+          "minLength": 1,
+          "maxLength": 8192
+        }
+      },
+      "additionalProperties": false
     }
   },
   "oneOf": [
@@ -147,6 +182,7 @@
           "pattern": "^[0-9a-f]{64}$"
         },
         "adapter": { "type": "string" },
+        "pagination": { "$ref": "#/$defs/pagination" },
         "diagnostics": { "$ref": "#/$defs/diagnostics" }
       },
       "additionalProperties": true

src/docforge/changesets.py

@@ -20,9 +20,12 @@ from .changeset_contract import (
 )
 from .errors import DocForgeError
 from .models import Edge, Node, ProjectService, ProjectSnapshot, ProposalWriter
+from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt
 from .project import project_root_fingerprint
 from .proposal_projection import ProposalProjector
 
+MAX_ABANDON_REASON_CHARS = 2_000
+
 
 class ChangesetStore:
     """One project-bound proposal store with an optional immutable writer identity."""
@@ -277,23 +280,37 @@ class ChangesetStore:
             },
         )
 
-    def validate(self, changeset_id: str) -> dict[str, object]:
+    def validate(
+        self,
+        changeset_id: str,
+        *,
+        limit: int | None = None,
+        cursor: str | None = None,
+    ) -> dict[str, object]:
         validate_id(changeset_id, "changeset_id")
         with self._lock():
             snapshot, document, nodes, edges = self._validate_locked(changeset_id)
-            return self._result(
+            result = self._result(
                 snapshot,
                 document,
                 valid=True,
                 projected_node_count=len(nodes),
                 projected_edge_count=len(edges),
             )
+            return self._page_document_result(
+                result,
+                kind="changeset.validate",
+                limit=limit,
+                cursor=cursor,
+            )
 
     def list_changesets(
         self,
         *,
         include_history: bool = True,
         status: str | None = None,
+        limit: int | None = None,
+        cursor: str | None = None,
     ) -> dict[str, object]:
         with self._lock():
             snapshot = self.project.load()
@@ -322,14 +339,93 @@ class ChangesetStore:
                         "operation_count": len(document["operations"]),
                     }
                 )
-            return self._base_result(snapshot, count=len(records), changesets=records)
+            result = self._base_result(snapshot, count=len(records), changesets=records)
+            if limit is None and cursor is None:
+                return result
+            selected_limit = page_limit(
+                limit,
+                default=20,
+                maximum=self.project.descriptor.limits.max_results,
+            )
+            binding = {
+                "project_id": snapshot.descriptor.project_id,
+                "project_root_fingerprint": project_root_fingerprint(snapshot.descriptor.root),
+                "adapter": snapshot.descriptor.adapter,
+                "revision": snapshot.revision,
+                "source_hash": snapshot.source_hash,
+                "include_history": include_history,
+                "status": status,
+                "collection_hash": canonical_hash(records),
+            }
+            position = decode_cursor(
+                cursor,
+                kind="changeset.list",
+                binding=binding,
+                total_count=len(records),
+            )
+            page = records[position : position + selected_limit]
+            while page:
+                candidate = {
+                    **result,
+                    "count": len(page),
+                    "total_count": len(records),
+                    "changesets": page,
+                    "pagination": page_receipt(
+                        kind="changeset.list",
+                        binding=binding,
+                        position=position,
+                        count=len(page),
+                        limit=selected_limit,
+                        total_count=len(records),
+                    ),
+                }
+                if self._encoded_length(candidate) <= self._safe_page_chars():
+                    return candidate
+                page.pop()
+            if position < len(records):
+                compact_record = self._compact_list_record(records[position])
+                return {
+                    **result,
+                    "count": 1,
+                    "total_count": len(records),
+                    "changesets": [compact_record],
+                    "result_mode": "changeset_summaries",
+                    "pagination": page_receipt(
+                        kind="changeset.list",
+                        binding=binding,
+                        position=position,
+                        count=1,
+                        limit=selected_limit,
+                        total_count=len(records),
+                    ),
+                }
+            return {
+                **result,
+                "count": 0,
+                "total_count": len(records),
+                "changesets": [],
+                "pagination": page_receipt(
+                    kind="changeset.list",
+                    binding=binding,
+                    position=position,
+                    count=0,
+                    limit=selected_limit,
+                    total_count=len(records),
+                ),
+            }
 
-    def inspect(self, changeset_id: str) -> dict[str, object]:
+    def inspect(
+        self,
+        changeset_id: str,
+        *,
+        limit: int | None = None,
+        cursor: str | None = None,
+    ) -> dict[str, object]:
         validate_id(changeset_id, "changeset_id")
         with self._lock():
             document = self._read(self._path(changeset_id))
             snapshot = self.project.load()
-            return self._result(
+            result = self._result(
                 snapshot,
                 document,
                 base_state=self._base_state(document, snapshot),
@@ -338,6 +434,12 @@ class ChangesetStore:
                     self._base_state(document, snapshot),
                 ),
             )
+            return self._page_document_result(
+                result,
+                kind="changeset.inspect",
+                limit=limit,
+                cursor=cursor,
+            )
 
     def rebase(
         self,
@@ -398,8 +500,15 @@ class ChangesetStore:
 
         validate_id(changeset_id, "changeset_id")
         validate_hash(expected_changeset_hash, "expected_changeset_hash")
-        if not reason.strip():
+        normalized_reason = reason.strip()
+        if not normalized_reason:
             raise DocForgeError("invalid_operation", "Abandon reason must be non-empty")
+        if len(normalized_reason) > MAX_ABANDON_REASON_CHARS:
+            raise DocForgeError(
+                "changeset_too_large",
+                "Abandon reason exceeds its character limit",
+                maximum=MAX_ABANDON_REASON_CHARS,
+            )
         with self._lock():
             document = self._read(self._path(changeset_id))
             actual_hash = document_hash(document)
@@ -418,7 +527,7 @@ class ChangesetStore:
                 {
                     "status": "abandoned",
                     "changeset_hash": actual_hash,
-                    "reason": reason.strip(),
+                    "reason": normalized_reason,
                     "revision": snapshot.revision,
                     "source_hash": snapshot.source_hash,
                 },
@@ -430,7 +539,13 @@ class ChangesetStore:
                 lifecycle=receipt,
             )
 
-    def diff(self, changeset_id: str) -> dict[str, object]:
+    def diff(
+        self,
+        changeset_id: str,
+        *,
+        limit: int | None = None,
+        cursor: str | None = None,
+    ) -> dict[str, object]:
         validate_id(changeset_id, "changeset_id")
         with self._lock():
             snapshot, document, _, _ = self._validate_locked(changeset_id)
@@ -453,7 +568,14 @@ class ChangesetStore:
                         sorted(before_edges - edges),
                     )
                 )
-            return self._result(snapshot, document, valid=True, changes=changes)
+            result = self._result(snapshot, document, valid=True, changes=changes)
+            return self._page_document_result(
+                result,
+                kind="changeset.diff",
+                limit=limit,
+                cursor=cursor,
+                parallel_key="changes",
+            )
 
     def projected_snapshot(self, changeset_id: str) -> tuple[ProjectSnapshot, str]:
         """Return a validated in-memory proposal projection for derived preview use."""
@@ -790,6 +912,219 @@ class ChangesetStore:
             **payload,
         )
 
+    def _page_document_result(
+        self,
+        result: dict[str, object],
+        *,
+        kind: str,
+        limit: int | None,
+        cursor: str | None,
+        parallel_key: str | None = None,
+    ) -> dict[str, object]:
+        """Page bulky operation-aligned payloads while preserving direct full defaults."""
+
+        if limit is None and cursor is None:
+            return result
+        selected_limit = page_limit(
+            limit,
+            default=20,
+            maximum=self.project.descriptor.limits.max_results,
+        )
+        operations_value = result.get("operations")
+        if not isinstance(operations_value, list):
+            raise DocForgeError(
+                "invalid_pagination_source",
+                "Changeset result does not contain a deterministic operation list",
+            )
+        operations = cast(list[object], operations_value)
+        parallel: list[object] | None = None
+        if parallel_key is not None:
+            parallel_value = result.get(parallel_key)
+            if not isinstance(parallel_value, list):
+                raise DocForgeError(
+                    "invalid_pagination_source",
+                    "Changeset result does not contain an aligned detail list",
+                )
+            parallel = cast(list[object], parallel_value)
+            if len(parallel) != len(operations):
+                raise DocForgeError(
+                    "invalid_pagination_source",
+                    "Changeset detail list is not aligned with its operations",
+                )
+        binding = {
+            "project_id": result["project_id"],
+            "project_root_fingerprint": result["project_root_fingerprint"],
+            "revision": result["revision"],
+            "source_hash": result["source_hash"],
+            "changeset_id": result["changeset_id"],
+            "changeset_hash": result["changeset_hash"],
+            "adapter": result["adapter"],
+            "result_hash": canonical_hash(
+                {
+                    "operations": operations,
+                    **({parallel_key: parallel} if parallel_key is not None else {}),
+                }
+            ),
+        }
+        if (
+            kind == "changeset.diff"
+            and parallel_key is not None
+            and parallel is not None
+            and self._encoded_length(result) > self._safe_page_chars()
+        ):
+            return self._page_json_chunks(
+                result,
+                operations=operations,
+                changes=parallel,
+                binding=binding,
+                cursor=cursor,
+            )
+        position = decode_cursor(
+            cursor,
+            kind=kind,
+            binding=binding,
+            total_count=len(operations),
+        )
+        page = operations[position : position + selected_limit]
+        paged = {
+            **result,
+            "operations": page,
+            "returned_operation_count": len(page),
+            "pagination": page_receipt(
+                kind=kind,
+                binding=binding,
+                position=position,
+                count=len(page),
+                limit=selected_limit,
+                total_count=len(operations),
+            ),
+        }
+        if parallel_key is not None and parallel is not None:
+            paged[parallel_key] = parallel[position : position + selected_limit]
+        if self._encoded_length(paged) > self._safe_page_chars():
+            paged["operations"] = [
+                self._operation_summary(item)
+                for item in operations[position : position + selected_limit]
+            ]
+            paged["result_mode"] = "operation_summaries"
+            paged["detail_tool"] = "docforge_get_changeset_diff"
+        return paged
+
+    def _page_json_chunks(
+        self,
+        result: dict[str, object],
+        *,
+        operations: list[object],
+        changes: list[object],
+        binding: Mapping[str, object],
+        cursor: str | None,
+    ) -> dict[str, object]:
+        payload = {"operations": operations, "changes": changes}
+        encoded = json.dumps(
+            payload,
+            sort_keys=True,
+            separators=(",", ":"),
+            ensure_ascii=True,
+            allow_nan=False,
+        )
+        chunk_chars = max(512, min(64_000, self._safe_page_chars() // 2))
+        chunks = [
+            encoded[offset : offset + chunk_chars] for offset in range(0, len(encoded), chunk_chars)
+        ] or [""]
+        chunk_binding = {
+            **binding,
+            "payload_hash": canonical_hash(payload),
+            "chunk_chars": chunk_chars,
+        }
+        position = decode_cursor(
+            cursor,
+            kind="changeset.diff-chunks",
+            binding=chunk_binding,
+            total_count=len(chunks),
+        )
+        compact = {
+            key: value for key, value in result.items() if key not in {"operations", "changes"}
+        }
+        return {
+            **compact,
+            "result_mode": "canonical_json_chunk",
+            "payload": "changeset_diff",
+            "payload_hash": chunk_binding["payload_hash"],
+            "payload_characters": len(encoded),
+            "chunk": {
+                "index": position,
+                "characters": len(chunks[position]),
+                "content": chunks[position],
+            },
+            "pagination": page_receipt(
+                kind="changeset.diff-chunks",
+                binding=chunk_binding,
+                position=position,
+                count=1,
+                limit=1,
+                total_count=len(chunks),
+            ),
+        }
+
+    @staticmethod
+    def _operation_summary(operation: object) -> dict[str, object]:
+        if not isinstance(operation, Mapping):
+            raise DocForgeError(
+                "invalid_pagination_source",
+                "Changeset operation is not a deterministic object",
+            )
+        payload = cast(Mapping[str, object], operation)
+        summary = {
+            key: payload.get(key)
+            for key in (
+                "sequence",
+                "operation",
+                "node_id",
+                "expected_content_hash",
+                "target_source",
+            )
+        }
+        for key in ("metadata", "content", "relationship_changes", "rationale"):
+            value = payload.get(key)
+            summary[f"{key}_hash"] = canonical_hash(value)
+            summary[f"{key}_characters"] = len(
+                json.dumps(
+                    value,
+                    sort_keys=True,
+                    separators=(",", ":"),
+                    ensure_ascii=True,
+                    allow_nan=False,
+                )
+            )
+        return summary
+
+    @staticmethod
+    def _compact_list_record(record: dict[str, object]) -> dict[str, object]:
+        lifecycle = record.get("lifecycle")
+        if not isinstance(lifecycle, Mapping):
+            return record
+        lifecycle_payload = cast(Mapping[str, object], lifecycle)
+        reason = lifecycle_payload.get("reason")
+        if not isinstance(reason, str):
+            return record
+        return {
+            **record,
+            "lifecycle": {
+                **lifecycle_payload,
+                "reason": {
+                    "characters": len(reason),
+                    "sha256": canonical_hash(reason),
+                },
+            },
+        }
+
+    def _safe_page_chars(self) -> int:
+        return max(1_024, self.project.descriptor.limits.max_tool_output_chars - 2_048)
+
+    @staticmethod
+    def _encoded_length(value: Mapping[str, object]) -> int:
+        return len(json.dumps(value, sort_keys=True, separators=(",", ":")))
+
     @staticmethod
     def _base_result(snapshot: ProjectSnapshot, **payload: object) -> dict[str, object]:
         return {

src/docforge/cli.py

@@ -63,6 +63,8 @@ def _parser() -> argparse.ArgumentParser:
     context = commands.add_parser("context")
     context.add_argument("profile")
     context.add_argument("--budget", type=int)
+    context.add_argument("--limit", type=int)
+    context.add_argument("--cursor")
     render = commands.add_parser("render")
     render.add_argument("view_id")
     render_status = commands.add_parser("render-status")
@@ -175,6 +177,15 @@ def _run(arguments: argparse.Namespace) -> dict[str, object]:
             limit=arguments.limit,
         )
     if arguments.command == "context":
+        if arguments.limit is not None or arguments.cursor is not None:
+            from .mcp_server import DocForgeService
+
+            return DocForgeService(project).context(
+                arguments.profile,
+                arguments.budget,
+                limit=arguments.limit,
+                cursor=arguments.cursor,
+            )
         return compile_context(index, arguments.profile, arguments.budget)
     if arguments.command == "render":
         return RenderService(project).render(arguments.view_id)

src/docforge/mcp_server.py

@@ -17,6 +17,7 @@ from .context import compile_context
 from .errors import DocForgeError
 from .index import ProjectIndex
 from .models import IncrementalStateProject, ProjectService, RuntimeValidatedProject
+from .pagination import canonical_hash, decode_cursor, page_limit, page_receipt
 from .project import Project, project_root_fingerprint
 from .rendering import RenderService
 from .telemetry import request, stage
@@ -87,6 +88,7 @@ STALE_ERROR_CODES = frozenset(
         "content_conflict",
         "source_changed",
         "stale_adapter_source",
+        "stale_cursor",
         "stale_index",
     }
 )
@@ -488,6 +490,11 @@ class DocForgeService:
                 "tool": "docforge_get_changeset",
                 "arguments": {"changeset_id": "<same>"},
             }
+        if error.code == "stale_cursor":
+            return {
+                "retryable": True,
+                "action": "restart_pagination",
+            }
         return None
 
     def synchronize(self) -> dict[str, object]:
@@ -707,12 +714,158 @@ class DocForgeService:
             operation_name="mcp.render_status",
         )
 
-    def context(self, profile: str, budget: int | None = None) -> dict[str, Any]:
+    def context(
+        self,
+        profile: str,
+        budget: int | None = None,
+        *,
+        limit: int | None = None,
+        cursor: str | None = None,
+    ) -> dict[str, Any]:
+        def operation() -> dict[str, object]:
+            selected_limit = page_limit(
+                limit,
+                default=20,
+                maximum=self.project.descriptor.limits.max_results,
+            )
+            result = self.context_provider(self.index, profile, budget)
+            return self._page_context_result(
+                result,
+                profile=profile,
+                selected_limit=selected_limit,
+                cursor=cursor,
+            )
+
         return self.invoke(
-            lambda: self.context_provider(self.index, profile, budget),
+            operation,
             operation_name="mcp.context",
         )
 
+    def _page_context_result(
+        self,
+        result: dict[str, object],
+        *,
+        profile: str,
+        selected_limit: int,
+        cursor: str | None,
+    ) -> dict[str, object]:
+        entries_value = result.get("entries")
+        omissions_value = result.get("omissions")
+        if not isinstance(entries_value, list) or not isinstance(omissions_value, list):
+            raise DocForgeError(
+                "invalid_context_result",
+                "Context provider did not return deterministic entries and omissions",
+            )
+        entries = cast(list[object], entries_value)
+        omissions = cast(list[object], omissions_value)
+        binding = {
+            "project_id": result.get("project_id"),
+            "project_root_fingerprint": result.get("project_root_fingerprint"),
+            "adapter": result.get("adapter"),
+            "revision": result.get("revision"),
+            "source_hash": result.get("source_hash"),
+            "profile": profile,
+            "budget": result.get("budget"),
+            "selection_hash": canonical_hash(
+                {
+                    "entries": entries,
+                    "omissions": omissions,
+                }
+            ),
+        }
+        evidence = [
+            *(("entry", item) for item in entries),
+            *(("omission", item) for item in omissions),
+        ]
+        position = decode_cursor(
+            cursor,
+            kind="context.items",
+            binding=binding,
+            total_count=len(evidence),
+        )
+        page_entries: list[object] = []
+        page_omissions: list[object] = []
+        consumed = 0
+        truncation_reason: str | None = None
+        maximum = self.project.descriptor.limits.max_tool_output_chars
+
+        def page_result() -> dict[str, object]:
+            pagination = page_receipt(
+                kind="context.items",
+                binding=binding,
+                position=position,
+                count=consumed,
+                limit=selected_limit,
+                total_count=len(evidence),
+            )
+            return {
+                **result,
+                "entries": page_entries,
+                "omissions": page_omissions,
+                "entry_count": len(page_entries),
+                "omission_count": len(page_omissions),
+                "page_count": consumed,
+                "page_estimated_tokens": sum(
+                    cast(int, cast(Mapping[str, object], item).get("estimated_tokens", 0))
+                    for item in page_entries
+                    if isinstance(item, Mapping)
+                ),
+                "summary": {
+                    "entry_count": len(entries),
+                    "omission_count": len(omissions),
+                    "evidence_count": len(evidence),
+                    "estimated_tokens": result.get("estimated_tokens"),
+                },
+                "truncation_reason": truncation_reason,
+                "next_cursor": pagination["next_cursor"],
+                "pagination": pagination,
+            }
+
+        for kind, item in evidence[position:]:
+            if consumed >= selected_limit:
+                truncation_reason = "result_limit"
+                break
+            destination = page_entries if kind == "entry" else page_omissions
+            destination.append(item)
+            consumed += 1
+            candidate = page_result()
+            decorated = {
+                **candidate,
+                "server_version": SERVER_VERSION,
+                "content_warning": CONTENT_WARNING,
+                "staleness": "current",
+            }
+            if self._encoded_length(decorated) <= maximum:
+                continue
+            destination.pop()
+            consumed -= 1
+            truncation_reason = "response_limit"
+            if consumed == 0:
+                compact = self._oversized_context_omission(kind, item)
+                page_omissions.append(compact)
+                consumed = 1
+            break
+        if position + consumed < len(evidence) and truncation_reason is None:
+            truncation_reason = "result_limit"
+        return page_result()
+
+    @staticmethod
+    def _oversized_context_omission(kind: str, item: object) -> dict[str, object]:
+        node_id = "unknown"
+        hash_source = item
+        if isinstance(item, Mapping):
+            item_payload = cast(Mapping[str, object], item)
+            candidate = item_payload.get("node_id")
+            if isinstance(candidate, str) and candidate:
+                node_id = candidate[:256]
+            hash_source = dict(item_payload)
+        return {
+            "node_id": node_id,
+            "reason": "response size limit",
+            "original_evidence": kind,
+            "detail_hash": canonical_hash(hash_source),
+        }
+
     def visualize(
         self,
         node_id: str | None = None,
@@ -897,10 +1050,15 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
         )
 
     @server.tool(name="docforge_get_context")
-    def get_context(profile: str, budget: int | None = None) -> dict[str, Any]:
+    def get_context(
+        profile: str,
+        budget: int | None = None,
+        limit: int | None = None,
+        cursor: str | None = None,
+    ) -> dict[str, Any]:
         """Compile bounded cited context from one configured profile with explicit omissions."""
 
-        return service.context(profile, budget)
+        return service.context(profile, budget, limit=limit, cursor=cursor)
 
     @server.tool(name="docforge_validate_project")
     def validate_project() -> dict[str, Any]:
@@ -1000,6 +1158,8 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
     def list_changesets(
         include_history: bool = False,
         status: str | None = None,
+        limit: int | None = 20,
+        cursor: str | None = None,
     ) -> dict[str, Any]:
         """List active proposals by default, with optional lifecycle history."""
 
@@ -1007,16 +1167,26 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
             lambda: service.changesets.list_changesets(
                 include_history=include_history,
                 status=status,
+                limit=20 if limit is None else limit,
+                cursor=cursor,
             ),
             operation_name="mcp.changeset",
         )
 
     @server.tool(name="docforge_get_changeset")
-    def get_changeset(changeset_id: str) -> dict[str, Any]:
+    def get_changeset(
+        changeset_id: str,
+        limit: int | None = 20,
+        cursor: str | None = None,
+    ) -> dict[str, Any]:
         """Inspect a stored proposal even when its canonical base has become stale."""
 
         return service.invoke(
-            lambda: service.changesets.inspect(changeset_id),
+            lambda: service.changesets.inspect(
+                changeset_id,
+                limit=20 if limit is None else limit,
+                cursor=cursor,
+            ),
             operation_name="mcp.changeset",
         )
 
@@ -1225,20 +1395,36 @@ def _create_bound_server(service: DocForgeService, *, read_only: bool) -> FastMC
         )
 
     @server.tool(name="docforge_validate_changeset")
-    def validate_changeset(changeset_id: str) -> dict[str, Any]:
+    def validate_changeset(
+        changeset_id: str,
+        limit: int | None = 20,
+        cursor: str | None = None,
+    ) -> dict[str, Any]:
         """Validate a proposal against its exact canonical base and other active proposals."""
 
         return service.invoke(
-            lambda: service.changesets.validate(changeset_id),
+            lambda: service.changesets.validate(
+                changeset_id,
+                limit=20 if limit is None else limit,
+                cursor=cursor,
+            ),
             operation_name="mcp.changeset",
         )
 
     @server.tool(name="docforge_get_changeset_diff")
-    def get_changeset_diff(changeset_id: str) -> dict[str, Any]:
+    def get_changeset_diff(
+        changeset_id: str,
+        limit: int | None = 20,
+        cursor: str | None = None,
+    ) -> dict[str, Any]:
         """Return a deterministic structured and textual diff without applying the proposal."""
 
         return service.invoke(
-            lambda: service.changesets.diff(changeset_id),
+            lambda: service.changesets.diff(
+                changeset_id,
+                limit=20 if limit is None else limit,
+                cursor=cursor,
+            ),
             operation_name="mcp.changeset",
         )
 

src/docforge/pagination.py

@@ -0,0 +1,163 @@
+"""Deterministic, generation-bound pagination cursors for bounded public results."""
+
+from __future__ import annotations
+
+import base64
+import binascii
+import hashlib
+import hmac
+import json
+from collections.abc import Mapping
+from typing import cast
+
+from .errors import DocForgeError
+
+CURSOR_SCHEMA_VERSION = 1
+MAX_CURSOR_CHARS = 8_192
+_CURSOR_DOMAIN = b"docforge-page-cursor-v1\0"
+_CURSOR_KEYS = frozenset({"schema_version", "kind", "binding", "position", "checksum"})
+
+
+def canonical_hash(value: object) -> str:
+    """Hash one JSON-compatible value using DocForge's deterministic JSON form."""
+
+    try:
+        encoded = _canonical_bytes(value)
+    except (TypeError, ValueError) as error:
+        raise DocForgeError(
+            "invalid_pagination_source",
+            "Pagination source data is not deterministic JSON",
+        ) from error
+    return hashlib.sha256(encoded).hexdigest()
+
+
+def page_limit(limit: int | None, *, default: int, maximum: int) -> int:
+    """Validate one additive page size against the project result policy."""
+
+    selected = default if limit is None else limit
+    if type(selected) is not int or selected < 1 or selected > maximum:
+        raise DocForgeError(
+            "invalid_limit",
+            "Page limit is outside the configured result limit",
+            maximum=maximum,
+        )
+    return selected
+
+
+def encode_cursor(
+    *,
+    kind: str,
+    binding: Mapping[str, object],
+    position: int,
+) -> str:
+    """Encode a corruption-detecting cursor bound to an immutable result identity."""
+
+    if not kind or type(position) is not int or position < 0:
+        raise ValueError("Cursor kind and position must be valid")
+    body: dict[str, object] = {
+        "schema_version": CURSOR_SCHEMA_VERSION,
+        "kind": kind,
+        "binding": dict(binding),
+        "position": position,
+    }
+    checksum = hashlib.sha256(_CURSOR_DOMAIN + _canonical_bytes(body)).hexdigest()
+    envelope = {**body, "checksum": checksum}
+    return base64.urlsafe_b64encode(_canonical_bytes(envelope)).decode("ascii").rstrip("=")
+
+
+def decode_cursor(
+    cursor: str | None,
+    *,
+    kind: str,
+    binding: Mapping[str, object],
+    total_count: int,
+) -> int:
+    """Return a validated position, rejecting corrupt, foreign, or stale cursors."""
+
+    if cursor is None:
+        return 0
+    if not cursor or len(cursor) > MAX_CURSOR_CHARS or not cursor.isascii():
+        raise _invalid_cursor()
+    padding = "=" * (-len(cursor) % 4)
+    try:
+        raw = base64.b64decode(
+            (cursor + padding).encode("ascii"),
+            altchars=b"-_",
+            validate=True,
+        )
+        parsed: object = json.loads(raw.decode("utf-8"))
+    except (binascii.Error, UnicodeDecodeError, json.JSONDecodeError):
+        raise _invalid_cursor() from None
+    if base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=") != cursor or not isinstance(
+        parsed, dict
+    ):
+        raise _invalid_cursor()
+    payload = cast(dict[str, object], parsed)
+    checksum = payload.get("checksum")
+    position = payload.get("position")
+    stored_binding = payload.get("binding")
+    if (
+        frozenset(payload) != _CURSOR_KEYS
+        or payload.get("schema_version") != CURSOR_SCHEMA_VERSION
+        or payload.get("kind") != kind
+        or not isinstance(stored_binding, dict)
+        or type(position) is not int
+        or position < 0
+        or position >= total_count
+        or not isinstance(checksum, str)
+        or len(checksum) != 64
+    ):
+        raise _invalid_cursor()
+    body = {key: payload[key] for key in payload if key != "checksum"}
+    expected = hashlib.sha256(_CURSOR_DOMAIN + _canonical_bytes(body)).hexdigest()
+    if not hmac.compare_digest(checksum, expected):
+        raise _invalid_cursor()
+    if stored_binding != dict(binding):
+        raise DocForgeError(
+            "stale_cursor",
+            "Pagination cursor does not match the current result generation",
+        )
+    return position
+
+
+def page_receipt(
+    *,
+    kind: str,
+    binding: Mapping[str, object],
+    position: int,
+    count: int,
+    limit: int,
+    total_count: int,
+) -> dict[str, object]:
+    """Return one bounded page receipt and the next generation-bound cursor."""
+
+    next_position = position + count
+    has_more = next_position < total_count
+    return {
+        "schema_version": CURSOR_SCHEMA_VERSION,
+        "kind": kind,
+        "returned_count": count,
+        "limit": limit,
+        "total_count": total_count,
+        "has_more": has_more,
+        "next_cursor": (
+            encode_cursor(kind=kind, binding=binding, position=next_position) if has_more else None
+        ),
+    }
+
+
+def _canonical_bytes(value: object) -> bytes:
+    return json.dumps(
+        value,
+        sort_keys=True,
+        separators=(",", ":"),
+        ensure_ascii=True,
+        allow_nan=False,
+    ).encode("utf-8")
+
+
+def _invalid_cursor() -> DocForgeError:
+    return DocForgeError(
+        "invalid_cursor",
+        "Pagination cursor is malformed or does not match its operation",
+    )

tests/test_cli.py

@@ -39,6 +39,20 @@ class DocForgeCliTests(unittest.TestCase):
                     ]
                 )
                 self.assertEqual(7, arguments.limit)
+        context = parser.parse_args(
+            [
+                "--project-root",
+                "/tmp/project",
+                "context",
+                "active",
+                "--limit",
+                "7",
+                "--cursor",
+                "opaque",
+            ]
+        )
+        self.assertEqual(7, context.limit)
+        self.assertEqual("opaque", context.cursor)
 
     def test_reindex_apply_and_visualization_commands_are_self_service(self) -> None:
         with tempfile.TemporaryDirectory() as directory:

tests/test_core.py

@@ -378,6 +378,27 @@ class DocForgeCoreTests(unittest.TestCase):
                     operation()
                 self.assertEqual("invalid_limit", invalid.exception.code)
 
+    def test_incoming_traversal_uses_source_ordered_covering_index(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture("alpha", Path(directory))
+            index = ProjectIndex(Project.open(root))
+            index.build()
+
+            connection = sqlite3.connect(index.path)
+            try:
+                plan = connection.execute(
+                    "EXPLAIN QUERY PLAN "
+                    "SELECT source_id, relation, target_id FROM edges "
+                    "WHERE target_id = ? "
+                    "ORDER BY source_id, relation, target_id LIMIT ?",
+                    ("guide.foundation", 101),
+                ).fetchall()
+            finally:
+                connection.close()
+            details = " ".join(str(row[3]) for row in plan)
+            self.assertIn("edges_target_source", details)
+            self.assertNotIn("USE TEMP B-TREE", details)
+
     def test_query_rechecks_source_identity_before_returning(self) -> None:
         with tempfile.TemporaryDirectory() as directory:
             root = self.copy_fixture("alpha", Path(directory))

tests/test_mcp_server.py

@@ -16,6 +16,7 @@ from mcp import ClientSession, StdioServerParameters
 from mcp.client.stdio import stdio_client
 from mcp.shared.memory import create_connected_server_and_client_session
 
+from docforge.changesets import ChangesetStore
 from docforge.index import ProjectIndex
 from docforge.mcp_server import (
     ALL_TOOLS,
@@ -80,6 +81,16 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
             ):
                 self.assertIn("limit", tools[name].inputSchema["properties"])
                 self.assertNotIn("limit", tools[name].inputSchema.get("required", []))
+            for name in (
+                "docforge_get_context",
+                "docforge_list_changesets",
+                "docforge_get_changeset",
+                "docforge_validate_changeset",
+                "docforge_get_changeset_diff",
+            ):
+                for field in ("limit", "cursor"):
+                    self.assertIn(field, tools[name].inputSchema["properties"])
+                    self.assertNotIn(field, tools[name].inputSchema.get("required", []))
             self.assertIn(
                 "deep",
                 tools["docforge_render_status"].inputSchema["properties"],
@@ -110,6 +121,134 @@ class DocForgeMcpTests(unittest.IsolatedAsyncioTestCase):
             self.assertEqual(0, diagnostics["counters"]["project_loads"])
             self.assertEqual(0, diagnostics["counters"]["source_files_parsed"])
 
+    async def test_context_pagination_is_complete_and_stale_cursors_fail_closed(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture("alpha", Path(directory))
+            ProjectIndex(Project.open(root)).build()
+            async with create_connected_server_and_client_session(
+                create_server(root), raise_exceptions=True
+            ) as session:
+                first = await session.call_tool(
+                    "docforge_get_context",
+                    {"profile": "active", "limit": 1},
+                )
+                cursor = first.structuredContent["pagination"]["next_cursor"]
+                pages = [first.structuredContent]
+                while cursor is not None:
+                    page = await session.call_tool(
+                        "docforge_get_context",
+                        {
+                            "profile": "active",
+                            "limit": 2,
+                            "cursor": cursor,
+                        },
+                    )
+                    pages.append(page.structuredContent)
+                    cursor = page.structuredContent["pagination"]["next_cursor"]
+
+                stale_cursor = first.structuredContent["pagination"]["next_cursor"]
+                workflow = root / "docs" / "content" / "workflow.md"
+                workflow.write_text(
+                    workflow.read_text(encoding="utf-8") + "\nChanged between pages.\n",
+                    encoding="utf-8",
+                )
+                stale = await session.call_tool(
+                    "docforge_get_context",
+                    {
+                        "profile": "active",
+                        "limit": 1,
+                        "cursor": stale_cursor,
+                    },
+                )
+
+            evidence = [
+                *(("entry", item["node_id"]) for page in pages for item in page["entries"]),
+                *(("omission", item["node_id"]) for page in pages for item in page["omissions"]),
+            ]
+            self.assertEqual(len(evidence), pages[0]["summary"]["evidence_count"])
+            self.assertEqual(len(evidence), len(set(evidence)))
+            self.assertEqual("stale_cursor", stale.structuredContent["error"]["code"])
+            self.assertEqual("stale", stale.structuredContent["staleness"])
+            self.assertEqual(
+                "restart_pagination",
+                stale.structuredContent["error"]["remediation"]["action"],
+            )
+
+    async def test_oversized_changeset_reads_return_exact_pages_and_chunks(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture("alpha", Path(directory))
+            descriptor = root / ".docforge" / "project.toml"
+            descriptor.write_text(
+                descriptor.read_text(encoding="utf-8").replace(
+                    "max_changeset_bytes = 100000",
+                    "max_changeset_bytes = 100000\nmax_tool_output_chars = 20000",
+                ),
+                encoding="utf-8",
+            )
+            project = Project.open(root)
+            ProjectIndex(project).build()
+            store = ChangesetStore(project, "alpha-editor")
+            created = store.create("mcp-chunked-diff")
+            foundation = next(
+                node for node in project.load().nodes if node.node_id == "guide.foundation"
+            )
+            proposed = store.propose_update(
+                changeset_id="mcp-chunked-diff",
+                expected_changeset_hash=str(created["changeset_hash"]),
+                node_id=foundation.node_id,
+                expected_content_hash=foundation.content_hash,
+                metadata=None,
+                content="replacement " * 4_000,
+                relationship_changes=[],
+                rationale="Exercise bounded MCP diff reconstruction.",
+            )
+            direct = store.diff("mcp-chunked-diff")
+
+            async with create_connected_server_and_client_session(
+                create_server(root), raise_exceptions=True
+            ) as session:
+                inspected = await session.call_tool(
+                    "docforge_get_changeset",
+                    {"changeset_id": "mcp-chunked-diff"},
+                )
+                validated = await session.call_tool(
+                    "docforge_validate_changeset",
+                    {"changeset_id": "mcp-chunked-diff"},
+                )
+                cursor: str | None = None
+                chunks: list[str] = []
+                while True:
+                    page = await session.call_tool(
+                        "docforge_get_changeset_diff",
+                        {
+                            "changeset_id": "mcp-chunked-diff",
+                            "cursor": cursor,
+                        },
+                    )
+                    self.assertLessEqual(
+                        len(json.dumps(page.structuredContent, separators=(",", ":"))),
+                        20_000,
+                    )
+                    chunks.append(page.structuredContent["chunk"]["content"])
+                    cursor = page.structuredContent["pagination"]["next_cursor"]
+                    if cursor is None:
+                        break
+
+            self.assertEqual("operation_summaries", inspected.structuredContent["result_mode"])
+            self.assertEqual("operation_summaries", validated.structuredContent["result_mode"])
+            self.assertTrue(validated.structuredContent["valid"])
+            self.assertEqual(
+                proposed["changeset_hash"],
+                validated.structuredContent["changeset_hash"],
+            )
+            self.assertEqual(
+                {
+                    "changes": direct["changes"],
+                    "operations": direct["operations"],
+                },
+                json.loads("".join(chunks)),
+            )
+
     async def test_no_ast_binding_preserves_adapter_and_blocks_logic(self) -> None:
         with tempfile.TemporaryDirectory() as directory:
             root = self.copy_fixture("alpha", Path(directory))

tests/test_pagination.py

@@ -0,0 +1,338 @@
+from __future__ import annotations
+
+import json
+import shutil
+import tempfile
+import unittest
+from pathlib import Path
+
+from jsonschema import Draft202012Validator
+
+from docforge.changesets import ChangesetStore
+from docforge.errors import DocForgeError
+from docforge.mcp_server import DocForgeService
+from docforge.pagination import decode_cursor, encode_cursor
+from docforge.project import Project, project_root_fingerprint
+
+ROOT = Path(__file__).resolve().parents[1]
+FIXTURES = ROOT / "tests" / "fixtures"
+
+
+class PaginationTests(unittest.TestCase):
+    def copy_fixture(self, destination: Path, *, max_tool_chars: int | None = None) -> Path:
+        root = destination / "alpha"
+        shutil.copytree(FIXTURES / "alpha", root)
+        if max_tool_chars is not None:
+            descriptor = root / ".docforge" / "project.toml"
+            text = descriptor.read_text(encoding="utf-8")
+            text = text.replace(
+                "max_changeset_bytes = 100000",
+                (f"max_changeset_bytes = 100000\nmax_tool_output_chars = {max_tool_chars}"),
+            )
+            descriptor.write_text(text, encoding="utf-8")
+        return root
+
+    def test_cursor_is_canonical_corruption_detecting_and_binding_bound(self) -> None:
+        binding = {"project_id": "alpha", "source_hash": "a" * 64}
+        cursor = encode_cursor(kind="context.items", binding=binding, position=2)
+
+        self.assertEqual(
+            2,
+            decode_cursor(
+                cursor,
+                kind="context.items",
+                binding=binding,
+                total_count=4,
+            ),
+        )
+        with self.assertRaises(DocForgeError) as corrupt:
+            decode_cursor(
+                f"{cursor[:-1]}{'A' if cursor[-1] != 'A' else 'B'}",
+                kind="context.items",
+                binding=binding,
+                total_count=4,
+            )
+        self.assertEqual("invalid_cursor", corrupt.exception.code)
+        with self.assertRaises(DocForgeError) as foreign:
+            decode_cursor(
+                cursor,
+                kind="context.items",
+                binding={**binding, "source_hash": "b" * 64},
+                total_count=4,
+            )
+        self.assertEqual("stale_cursor", foreign.exception.code)
+        with self.assertRaises(DocForgeError) as wrong_operation:
+            decode_cursor(
+                cursor,
+                kind="changeset.list",
+                binding=binding,
+                total_count=4,
+            )
+        self.assertEqual("invalid_cursor", wrong_operation.exception.code)
+
+    def test_context_pages_entries_then_omissions_without_loss(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture(Path(directory))
+            project = Project.open(root)
+            source_hash = "a" * 64
+
+            def context_provider(
+                _index: object, profile: str, budget: int | None
+            ) -> dict[str, object]:
+                return {
+                    "status": "ok",
+                    "project_id": project.descriptor.project_id,
+                    "project_root_fingerprint": project_root_fingerprint(root),
+                    "adapter": project.descriptor.adapter,
+                    "revision": "test-revision",
+                    "source_hash": source_hash,
+                    "profile": profile,
+                    "budget": budget,
+                    "estimated_tokens": 3,
+                    "entries": [
+                        {
+                            "node_id": "guide.one",
+                            "reason": "required",
+                            "estimated_tokens": 1,
+                            "text": "one",
+                        },
+                        {
+                            "node_id": "guide.two",
+                            "reason": "eligible",
+                            "estimated_tokens": 2,
+                            "text": "two",
+                        },
+                    ],
+                    "omissions": [
+                        {"node_id": "guide.three", "reason": "token budget"},
+                        {"node_id": "guide.four", "reason": "token budget"},
+                    ],
+                }
+
+            service = DocForgeService(project, context_provider=context_provider)
+            result_validator = Draft202012Validator(
+                json.loads((ROOT / "schemas" / "result.schema.json").read_text(encoding="utf-8"))
+            )
+            cursor: str | None = None
+            evidence: list[tuple[str, str]] = []
+            limits = [1, 2, 1]
+            page_index = 0
+            while True:
+                page = service.context(
+                    "active",
+                    600,
+                    limit=limits[min(page_index, len(limits) - 1)],
+                    cursor=cursor,
+                )
+                result_validator.validate(page)
+                evidence.extend(
+                    ("entry", str(item["node_id"]))
+                    for item in page["entries"]
+                    if isinstance(item, dict)
+                )
+                evidence.extend(
+                    ("omission", str(item["node_id"]))
+                    for item in page["omissions"]
+                    if isinstance(item, dict)
+                )
+                pagination = page["pagination"]
+                self.assertIsInstance(pagination, dict)
+                cursor = pagination["next_cursor"]
+                page_index += 1
+                if cursor is None:
+                    break
+
+            self.assertEqual(
+                [
+                    ("entry", "guide.one"),
+                    ("entry", "guide.two"),
+                    ("omission", "guide.three"),
+                    ("omission", "guide.four"),
+                ],
+                evidence,
+            )
+
+    def test_context_oversized_item_advances_as_a_bounded_omission(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture(Path(directory), max_tool_chars=4_000)
+            project = Project.open(root)
+
+            def context_provider(
+                _index: object, profile: str, budget: int | None
+            ) -> dict[str, object]:
+                return {
+                    "status": "ok",
+                    "project_id": project.descriptor.project_id,
+                    "project_root_fingerprint": project_root_fingerprint(root),
+                    "adapter": project.descriptor.adapter,
+                    "revision": "test-revision",
+                    "source_hash": "a" * 64,
+                    "profile": profile,
+                    "budget": budget,
+                    "estimated_tokens": 10_000,
+                    "entries": [
+                        {
+                            "node_id": "guide.oversized",
+                            "reason": "required",
+                            "estimated_tokens": 10_000,
+                            "text": "x" * 20_000,
+                        }
+                    ],
+                    "omissions": [],
+                }
+
+            result = DocForgeService(project, context_provider=context_provider).context(
+                "active",
+                600,
+                limit=1,
+            )
+
+            self.assertEqual("ok", result["status"])
+            self.assertEqual([], result["entries"])
+            self.assertEqual("response size limit", result["omissions"][0]["reason"])
+            self.assertLess(len(json.dumps(result, separators=(",", ":"))), 4_000)
+            self.assertFalse(result["pagination"]["has_more"])
+
+    def test_page_limits_reject_boolean_zero_and_policy_overflow(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture(Path(directory))
+            project = Project.open(root)
+            store = ChangesetStore(project, "alpha-editor")
+            store.create("bounded-page")
+            for limit in (True, 0, project.descriptor.limits.max_results + 1):
+                with self.subTest(limit=limit), self.assertRaises(DocForgeError) as invalid:
+                    store.list_changesets(limit=limit)
+                self.assertEqual("invalid_limit", invalid.exception.code)
+
+    def test_changeset_pages_preserve_full_direct_defaults_and_exact_order(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture(Path(directory))
+            project = Project.open(root)
+            store = ChangesetStore(project, "alpha-editor")
+            created = store.create("paged-operations")
+            first = store.propose_update(
+                changeset_id="paged-operations",
+                expected_changeset_hash=str(created["changeset_hash"]),
+                node_id="guide.foundation",
+                expected_content_hash=next(
+                    node.content_hash
+                    for node in project.load().nodes
+                    if node.node_id == "guide.foundation"
+                ),
+                metadata={"summary": "First paged update."},
+                content=None,
+                relationship_changes=[],
+                rationale="First page.",
+            )
+            store.propose_update(
+                changeset_id="paged-operations",
+                expected_changeset_hash=str(first["changeset_hash"]),
+                node_id="proof.validation",
+                expected_content_hash=next(
+                    node.content_hash
+                    for node in project.load().nodes
+                    if node.node_id == "proof.validation"
+                ),
+                metadata={"summary": "Second paged update."},
+                content=None,
+                relationship_changes=[],
+                rationale="Second page.",
+            )
+
+            full = store.inspect("paged-operations")
+            self.assertNotIn("pagination", full)
+            first_page = store.inspect("paged-operations", limit=1)
+            second_page = store.inspect(
+                "paged-operations",
+                limit=2,
+                cursor=str(first_page["pagination"]["next_cursor"]),
+            )
+            combined = [*first_page["operations"], *second_page["operations"]]
+            self.assertEqual(full["operations"], combined)
+            self.assertEqual(full["changeset_hash"], second_page["changeset_hash"])
+
+            full_diff = store.diff("paged-operations")
+            diff_page = store.diff("paged-operations", limit=1)
+            diff_next = store.diff(
+                "paged-operations",
+                limit=1,
+                cursor=str(diff_page["pagination"]["next_cursor"]),
+            )
+            self.assertEqual(
+                full_diff["changes"],
+                [*diff_page["changes"], *diff_next["changes"]],
+            )
+
+    def test_changeset_list_cursor_stales_when_collection_changes(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture(Path(directory))
+            store = ChangesetStore(Project.open(root), "alpha-editor")
+            store.create("page-a")
+            store.create("page-b")
+            first = store.list_changesets(limit=1)
+            cursor = str(first["pagination"]["next_cursor"])
+            second = store.list_changesets(limit=2, cursor=cursor)
+            self.assertEqual(
+                ["page-a", "page-b"],
+                [
+                    *(
+                        item["changeset_id"]
+                        for item in first["changesets"]
+                        if isinstance(item, dict)
+                    ),
+                    *(
+                        item["changeset_id"]
+                        for item in second["changesets"]
+                        if isinstance(item, dict)
+                    ),
+                ],
+            )
+            store.create("page-c")
+
+            with self.assertRaises(DocForgeError) as stale:
+                store.list_changesets(limit=1, cursor=cursor)
+            self.assertEqual("stale_cursor", stale.exception.code)
+
+    def test_oversized_diff_is_reconstructable_from_hash_bound_chunks(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            root = self.copy_fixture(Path(directory), max_tool_chars=20_000)
+            project = Project.open(root)
+            store = ChangesetStore(project, "alpha-editor")
+            created = store.create("chunked-diff")
+            store.propose_update(
+                changeset_id="chunked-diff",
+                expected_changeset_hash=str(created["changeset_hash"]),
+                node_id="guide.foundation",
+                expected_content_hash=next(
+                    node.content_hash
+                    for node in project.load().nodes
+                    if node.node_id == "guide.foundation"
+                ),
+                metadata=None,
+                content="replacement " * 4_000,
+                relationship_changes=[],
+                rationale="Exercise deterministic chunk transport.",
+            )
+            full = store.diff("chunked-diff")
+            cursor: str | None = None
+            chunks: list[str] = []
+            while True:
+                page = store.diff("chunked-diff", limit=20, cursor=cursor)
+                self.assertEqual("canonical_json_chunk", page["result_mode"])
+                chunks.append(page["chunk"]["content"])
+                cursor = page["pagination"]["next_cursor"]
+                if cursor is None:
+                    break
+            reconstructed = json.loads("".join(chunks))
+
+            self.assertEqual(
+                {
+                    "changes": full["changes"],
+                    "operations": full["operations"],
+                },
+                reconstructed,
+            )
+
+
+if __name__ == "__main__":
+    unittest.main()

tools/milestone1_benchmark.py

@@ -21,7 +21,6 @@ from milestone0_baseline import (
     write_synthetic_project,
 )
 
-from docforge.context import compile_context
 from docforge.index import ProjectIndex
 from docforge.mcp_server import DocForgeService
 from docforge.project import Project
@@ -80,27 +79,145 @@ def _diagnostics(result: object) -> Mapping[str, object]:
     return diagnostics
 
 
+def _result_summary(result: Mapping[str, object]) -> dict[str, object]:
+    """Retain bounded semantic evidence without copying primary result payloads."""
+
+    summary: dict[str, object] = {}
+    for key in (
+        "status",
+        "count",
+        "limit",
+        "truncated",
+        "truncation_reason",
+        "candidate_edges_consumed",
+        "candidate_edges_limit",
+        "budget",
+        "estimated_tokens",
+        "state",
+        "verification",
+        "configured",
+        "snapshot_state",
+        "staleness",
+    ):
+        if key in result:
+            summary[key] = result[key]
+    error = result.get("error")
+    if isinstance(error, Mapping):
+        error_payload = cast(Mapping[str, object], error)
+        if isinstance(error_payload.get("code"), str):
+            summary["error_code"] = error_payload["code"]
+    synchronization = result.get("synchronization")
+    if isinstance(synchronization, Mapping):
+        synchronization_payload = cast(Mapping[str, object], synchronization)
+        if isinstance(synchronization_payload.get("action"), str):
+            summary["synchronization_action"] = synchronization_payload["action"]
+    freshness = result.get("freshness")
+    if isinstance(freshness, Mapping):
+        freshness_payload = cast(Mapping[str, object], freshness)
+        summary["freshness"] = {
+            key: freshness_payload[key]
+            for key in ("index", "source")
+            if isinstance(freshness_payload.get(key), str)
+        }
+    outputs = result.get("outputs")
+    if isinstance(outputs, list):
+        summarized_outputs: list[dict[str, object]] = []
+        for item in cast(list[object], outputs)[:10]:
+            if not isinstance(item, Mapping):
+                continue
+            item_payload = cast(Mapping[str, object], item)
+            summarized_outputs.append(
+                {
+                    key: item_payload[key]
+                    for key in ("view_id", "state", "reason")
+                    if key in item_payload
+                }
+            )
+        summary["outputs"] = summarized_outputs
+    entries = result.get("entries")
+    if isinstance(entries, list):
+        summary["entry_count"] = len(cast(list[object], entries))
+    omissions = result.get("omissions")
+    if isinstance(omissions, list):
+        summary["omission_count"] = len(cast(list[object], omissions))
+    pagination = result.get("pagination")
+    if isinstance(pagination, Mapping):
+        pagination_payload = cast(Mapping[str, object], pagination)
+        summary["pagination"] = {
+            key: pagination_payload[key]
+            for key in ("kind", "returned_count", "limit", "total_count", "has_more")
+            if key in pagination_payload
+        }
+    return summary
+
+
 def _operation(
     operation: Callable[[], dict[str, object]],
     *,
     samples: int,
     p95_limit_ms: float,
     expected_status: str = "ok",
+    expected_counters: Mapping[str, int],
 ) -> dict[str, object]:
-    measurement, last = measure_operation(operation, samples=samples)
+    diagnostics_records: list[Mapping[str, object]] = []
+    result_summaries: list[dict[str, object]] = []
+
+    def validated_operation() -> dict[str, object]:
+        result = operation()
+        diagnostics = _diagnostics(result)
+        counters_value = diagnostics["counters"]
+        if not isinstance(counters_value, Mapping):
+            raise RuntimeError("Measured diagnostics did not return counters")
+        counters = cast(Mapping[str, object], counters_value)
+        for counter, expected in expected_counters.items():
+            if counters.get(counter) != expected:
+                raise RuntimeError(
+                    f"Warm operation expected {counter}={expected}, "
+                    f"received {counters.get(counter)!r}"
+                )
+        diagnostics_records.append(diagnostics)
+        result_summaries.append(_result_summary(result))
+        return result
+
+    measurement, last = measure_operation(validated_operation, samples=samples)
     if not isinstance(last, Mapping):
         raise RuntimeError(f"Measured operation did not return status={expected_status}")
     last_payload = cast(Mapping[str, object], last)
     if last_payload.get("status") != expected_status:
         raise RuntimeError(f"Measured operation did not return status={expected_status}")
-    diagnostics = _diagnostics(last_payload)
+    for summary in result_summaries:
+        if summary.get("status") != expected_status:
+            raise RuntimeError(f"Measured operation did not return status={expected_status}")
     p95_ms = float(cast(float, measurement["p95_ms"]))
     if p95_ms > p95_limit_ms:
         raise RuntimeError(f"Warm operation p95 {p95_ms:.3f} ms exceeds {p95_limit_ms:.3f} ms")
+    counter_names = sorted(
+        {
+            key
+            for diagnostics in diagnostics_records
+            for key in cast(Mapping[str, object], diagnostics["counters"])
+        }
+    )
+    counter_ranges = {
+        counter: {
+            "minimum": min(
+                cast(int, cast(Mapping[str, object], record["counters"])[counter])
+                for record in diagnostics_records
+            ),
+            "maximum": max(
+                cast(int, cast(Mapping[str, object], record["counters"])[counter])
+                for record in diagnostics_records
+            ),
+        }
+        for counter in counter_names
+    }
     return {
         **measurement,
         "p95_limit_ms": p95_limit_ms,
-        "diagnostics": diagnostics,
+        "validated_invocations": len(diagnostics_records),
+        "counter_expectations": dict(sorted(expected_counters.items())),
+        "counter_ranges": counter_ranges,
+        "result_summary": result_summaries[-1],
     }
 
 
@@ -110,11 +227,33 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
     RenderService(project).render("manual")
     service = DocForgeService(project, diagnostics=True)
     target = synthetic_node_id(node_count - 1)
+    backlink_target = synthetic_node_id(node_count - 2)
+    first = synthetic_node_id(0)
+    read_counters = {
+        "index_checks": 1,
+        "index_synchronizations": 0,
+        "viewer_manager_requests": 0,
+    }
+    status_counters = {
+        "index_checks": 0,
+        "index_synchronizations": 0,
+        "viewer_manager_requests": 0,
+    }
+    visualization_counters = {
+        "index_checks": 0,
+        "index_synchronizations": 0,
+        "viewer_manager_requests": 1,
+    }
     operations = {
         "warm_no_change_synchronize": _operation(
             service.synchronize,
             samples=samples,
             p95_limit_ms=100,
+            expected_counters={
+                "index_checks": 1,
+                "index_synchronizations": 1,
+                "viewer_manager_requests": 0,
+            },
         ),
         "exact_node": _operation(
             lambda: service.invoke(
@@ -123,6 +262,7 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
             ),
             samples=samples,
             p95_limit_ms=50,
+            expected_counters=read_counters,
         ),
         "missing_node_error": _operation(
             lambda: service.invoke(
@@ -132,6 +272,7 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
             samples=samples,
             p95_limit_ms=50,
             expected_status="error",
+            expected_counters=read_counters,
         ),
         "search_limit_20": _operation(
             lambda: service.invoke(
@@ -140,6 +281,25 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
             ),
             samples=samples,
             p95_limit_ms=100,
+            expected_counters=read_counters,
+        ),
+        "filter_limit_20": _operation(
+            lambda: service.invoke(
+                lambda: service.index.filter_nodes(family="guide", limit=20),
+                operation_name="mcp.filter",
+            ),
+            samples=samples,
+            p95_limit_ms=100,
+            expected_counters=read_counters,
+        ),
+        "backlinks_limit_20": _operation(
+            lambda: service.invoke(
+                lambda: service.index.backlinks(backlink_target, limit=20),
+                operation_name="mcp.backlinks",
+            ),
+            samples=samples,
+            p95_limit_ms=100,
+            expected_counters=read_counters,
         ),
         "dependencies_depth_8": _operation(
             lambda: service.invoke(
@@ -148,21 +308,58 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
             ),
             samples=samples,
             p95_limit_ms=100,
+            expected_counters=read_counters,
         ),
-        "context_32k": _operation(
+        "impact_depth_8": _operation(
             lambda: service.invoke(
-                lambda: compile_context(service.index, "active", 32_000),
+                lambda: service.index.impact(first, depth=8, limit=100),
-                operation_name="mcp.context",
+                operation_name="mcp.impact",
             ),
             samples=samples,
+            p95_limit_ms=100,
+            expected_counters=read_counters,
+        ),
+        "context_32k": _operation(
+            lambda: service.context("active", 32_000, limit=20),
+            samples=samples,
             p95_limit_ms=250,
+            expected_counters=read_counters,
         ),
         "render_receipt_status": _operation(
             lambda: service.render_status("manual"),
             samples=samples,
             p95_limit_ms=50,
+            expected_counters=status_counters,
         ),
     }
+    template = root / "docs" / "templates" / "manual.html"
+    original_template = template.read_bytes()
+    template.write_bytes(original_template + b"\n")
+    operations["render_stale_status"] = _operation(
+        lambda: service.render_status("manual"),
+        samples=samples,
+        p95_limit_ms=50,
+        expected_counters=status_counters,
+    )
+    template.write_bytes(original_template)
+    RenderService(project).render("manual")
+    receipt_path = root / ".docforge" / "cache" / "render-receipts" / "manual.json"
+    receipt_path.unlink()
+    operations["render_missing_receipt_status"] = _operation(
+        lambda: service.render_status("manual"),
+        samples=samples,
+        p95_limit_ms=50,
+        expected_counters=status_counters,
+    )
+    RenderService(project).render("manual")
+    receipt_path.write_text("{", encoding="utf-8")
+    operations["render_corrupt_receipt_status"] = _operation(
+        lambda: service.render_status("manual"),
+        samples=samples,
+        p95_limit_ms=50,
+        expected_counters=status_counters,
+    )
+    RenderService(project).render("manual")
     state_path = root / ".docforge" / "benchmark-viewer-manager.json"
     manager = ViewerManager(state_path, check_interval_seconds=0.02)
     manager_thread = threading.Thread(target=manager.serve_forever, daemon=True)
@@ -181,6 +378,7 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
             service.visualization_status,
             samples=samples,
             p95_limit_ms=50,
+            expected_counters=visualization_counters,
         )
         with service.index.path.open("ab") as stream:
             stream.write(b"\n")
@@ -188,12 +386,14 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
             service.visualization_status,
             samples=samples,
             p95_limit_ms=50,
+            expected_counters=visualization_counters,
         )
         service.stop_visualization()
         operations["visualization_not_running_status"] = _operation(
             service.visualization_status,
             samples=samples,
             p95_limit_ms=50,
+            expected_counters=visualization_counters,
         )
     finally:
         manager.shutdown()
@@ -204,6 +404,7 @@ def _benchmark(root: Path, node_count: int, samples: int) -> dict[str, object]:
         samples=samples,
         p95_limit_ms=50,
         expected_status="error",
+        expected_counters=visualization_counters,
     )
     return {
         "fixture": {
@@ -243,8 +444,13 @@ def main() -> int:
         "method": {
             "clock": "time.perf_counter_ns",
             "memory": "resource.getrusage(RUSAGE_SELF).ru_maxrss",
+            "memory_scope": (
+                "cumulative main-process high-water mark; detached viewer-worker memory excluded"
+            ),
             "response_size": "UTF-8 bytes of compact sorted JSON",
             "samples": arguments.samples,
+            "warmups": 1,
+            "percentile": "nearest-rank",
             "zero_work_counters": list(ZERO_WORK_COUNTERS),
         },
         **measurement,