administrator/DocForge2
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions Pages

Close Milestone 4 with adapter adoption evidence

Browse source
This commit is contained in:
Andraxion 2026-07-29 15:34:25 -04:00
parent 95271dcf2e
commit 6d06195950
27 changed files with 2870 additions and 325 deletions
Download patch file Download diff file Expand all files Collapse all files

48
docs/INCREMENTAL_INDEXING.md
View file

@ -4,6 +4,10 @@ DocForge Release 1 adapters return one complete immutable projection. That contr
supported. The incremental compiler adds an opt-in source-scoped contract that avoids reparsing
unchanged files while preserving the same validated, atomically published graph.
Import these contracts from the public `docforge.adapter_sdk` facade. See
[Legacy Adapters and No-AST Policy](LEGACY_AND_NO_AST.md) for the preserved one-method contract and
[Reference Adapters](REFERENCE_ADAPTERS.md) for the four maintained implementations.
## Release 1 compatibility
The incremental interface is additive:
@ -34,18 +38,26 @@ Incremental indexing is an extraction optimization. It does not weaken publicati
An interrupted extraction never replaces the last validated SQLite index. A malformed,
incompatible, or missing cache is a cache miss, not a partial graph.
One cache generation is bounded to 10,000 source contributions and 64,000,000 encoded bytes.
Aggregate graph and Logic assembly limits are described in the
[Language Adapter Authoring Guide](ADAPTER_AUTHORING_GUIDE.md#current-aggregate-bounds).
## Adapter contract
An incremental loader implements all three methods:
An incremental loader implements the first, second, and fourth methods. It implements
`load_complete_assembly()` as well when it publishes Logic:
```python
class MyAdapter:
def load_manifest(self) -> AdapterManifest: ...
def extract_source(self, source: AdapterSource) -> AdapterSourceProjection: ...
def load_complete_assembly(self) -> AdapterAssembly: ...
def load_projection(self) -> AdapterProjection: ...
```
`load_projection()` remains the deterministic full-rebuild fallback and equivalence oracle.
`load_projection()` remains the deterministic full-rebuild primary-graph oracle. An incremental
adapter that publishes Logic must additionally implement `load_complete_assembly()` as the
cache-independent complete graph-plus-Logic oracle.
Each `AdapterSource` declares:
@ -137,7 +149,8 @@ fresh process must import and validate the current adapter.
```
Adapters can call `AdapterProject.verify_incremental_equivalence()` in release and contract tests.
The check compares project identity, revision, source hash, nodes, and relationships against
The check compares project identity, revision, source hash, nodes, relationships, and Logic against
the independent complete assembly. It also requires the complete assembly's primary graph to match
`load_projection()`.
## Manual changes and relationships
@ -170,12 +183,29 @@ logic empty until they implement a language analyzer.
This boundary prevents thousands of boolean expressions and basic blocks from polluting Nodes,
Flow, Web, ordinary search, or architectural traversal. The Logic tab and `docforge_get_logic`
request one function-scoped projection on demand. The built-in analyzers cover Python,
JavaScript, and C++. Python uses the standard-library AST. JavaScript and C++ share pinned
Tree-sitter infrastructure with thin language-aware control-flow profiles. Parsers run only while
extracting a changed source contribution; ordinary graph reads do not load or execute them. A
grammar alone supplies syntax, not control-flow meaning, so each new language still needs a small
semantic profile for its branch, loop, case, exception, and termination constructs. All analyzers
report possible static paths; they do not claim runtime branch outcomes.
JavaScript, TypeScript, and C++. Python uses the standard-library AST. JavaScript, TypeScript, and
C++ use distinct optional Tree-sitter grammars with thin language-aware control-flow profiles.
Ordinary graph reads use stored projections and do not load or execute these parsers. A grammar
alone supplies syntax, not control-flow meaning, so each new language still needs a profile for
its branch, loop, case, exception, and termination constructs. All analyzers report possible
static paths; they do not claim runtime branch outcomes.
## Manifest and warm-parser scope
Parser work is language-specific and must be measured at the correct boundary:
- Python manifest construction fingerprints source and tokenizes local imports without calling
`ast.parse`.
- JavaScript and TypeScript manifest construction lexes static relative module specifiers without
invoking their distinct Tree-sitter extraction parsers. Focused tests prove this behavior on an
unchanged warm build.
- The C++ reference manifest parses inventoried sources with `tree-sitter-cpp` to discover quoted
include dependencies. A warm C++ extraction-cache hit is not a zero-parser claim.
The maintained Python benchmark instruments the unchanged warm path and requires zero
`ast.parse` calls and zero `extract_source` calls. That exact zero-parser benchmark claim is Python
only. JavaScript and TypeScript retain focused parser-free-manifest tests; C++ deliberately does
not.
## Full rebuilds