96 lines
4.3 KiB
Markdown
96 lines
4.3 KiB
Markdown
|
# Milestone 4 baseline
|
||
|
|
|
||
|
|
## Scope and method
|
||
|
|
|
||
|
|
This baseline records the adapter SDK, Python reference adapter, incremental equivalence, and
|
||
|
|
recovery behavior completed in Milestone 4. It was captured on 2026-07-29 from clean executable
|
||
|
|
candidate `95271dcf2e48045b9d3aed9b9ea09c7fc155692c`.
|
||
|
|
|
||
|
|
The maintained command was:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
.venv/bin/python tools/milestone4_benchmark.py \
|
||
|
|
--mode full \
|
||
|
|
--output benchmarks/milestone4-2026-07-29.json
|
||
|
|
```
|
||
|
|
|
||
|
|
The synthetic project contains 334 Python files. Each file contributes one module, one function,
|
||
|
|
and one argument node, for 1,002 primary nodes. Imports form a deterministic chain. Durations use
|
||
|
|
`time.perf_counter_ns()` and nearest-rank p95. Each ordinary result is serialized as compact sorted
|
||
|
|
JSON for its response-size gate. Per-operation memory uses `tracemalloc`; cumulative process
|
||
|
|
high-water uses `RUSAGE_SELF`.
|
||
|
|
|
||
|
|
Environment:
|
||
|
|
|
||
|
|
- Linux 7.1.3-200.nobara.fc44.x86_64 with glibc 2.43.
|
||
|
|
- CPython 3.14.6.
|
||
|
|
- x86_64.
|
||
|
|
- Three warm samples; cold, equivalence, and recovery operations run once.
|
||
|
|
- Per-operation traced-memory ceiling: 268,435,456 bytes.
|
||
|
|
- Process high-water ceiling: 536,870,912 bytes.
|
||
|
|
- Response ceiling: 524,288 bytes.
|
||
|
|
|
||
|
|
The complete machine-readable result is
|
||
|
|
[`benchmarks/milestone4-2026-07-29.json`](../benchmarks/milestone4-2026-07-29.json).
|
||
|
|
Its SHA-256 is `b6a871dde730a119fc0a138c47bd25f2c533ed3171a9a3d098075aef08173600`.
|
||
|
|
The stable evidence payload SHA-256 is
|
||
|
|
`4a8db461a311b5df4abd5aa00063e9a347d8b9dba19b30e35684f561d5271549`.
|
||
|
|
|
||
|
|
## Results
|
||
|
|
|
||
|
|
| Operation | Median | p95 | Limit | Traced peak | Response |
|
||
|
|
|---|---:|---:|---:|---:|---:|
|
||
|
|
| Cold incremental build | 1,599.760 ms | 1,599.760 ms | 20,000 ms | 68,573,540 B | 14,580 B |
|
||
|
|
| Warm incremental build | 1,244.427 ms | 1,265.387 ms | 20,000 ms | 69,997,166 B | 14,578 B |
|
||
|
|
| Complete/incremental equivalence | 1,897.919 ms | 1,897.919 ms | 30,000 ms | 64,305,919 B | 230 B |
|
||
|
|
| Corrupt extraction-cache recovery | 1,695.406 ms | 1,695.406 ms | 30,000 ms | 69,776,923 B | 14,580 B |
|
||
|
|
| Corrupt index recovery | 1,707.879 ms | 1,707.879 ms | 30,000 ms | 68,561,924 B | 14,779 B |
|
||
|
|
|
||
|
|
Process high-water was 78,798,848 bytes. The regression limits intentionally leave multiple times
|
||
|
|
the measured headroom; they are tripwires, not performance promises.
|
||
|
|
|
||
|
|
## Deterministic graph and Logic evidence
|
||
|
|
|
||
|
|
The candidate produced:
|
||
|
|
|
||
|
|
- 1,002 primary nodes with hash
|
||
|
|
`f7705dedf8dd388857a20d11f459dc74de797bcd12d3ed36e7f3aa75d67c328f`.
|
||
|
|
- 1,001 primary edges with hash
|
||
|
|
`00cc90998b6783afc8c9d1fd900409e5e3ba352868c4ba30e9bb2cbd59c52d35`.
|
||
|
|
- 334 Logic projections containing 2,338 Logic nodes and 2,338 Logic edges, with hash
|
||
|
|
`c86a3ae74777c2cec3a82c83e6e5bcca0196772ccea63cb13340fa9141593b0c`.
|
||
|
|
- Complete assembly hash
|
||
|
|
`2888182ab765fbffe3ba873c1613345640e8e6d89be74ddfca7452c0a5056345`.
|
||
|
|
- Source hash
|
||
|
|
`30ef23aa41061de2d4a7c995fe109d7a41518d9ee5493d805dd256581b47dae2`.
|
||
|
|
|
||
|
|
The independently loaded complete assembly and the incremental assembly matched exactly across
|
||
|
|
project identity, revision, source, nodes, edges, and Logic.
|
||
|
|
|
||
|
|
## Work and recovery gates
|
||
|
|
|
||
|
|
The warm build recorded all 334 sources as cache hits, zero reparsed sources, zero `ast.parse`
|
||
|
|
calls, and zero `extract_source` calls. Corrupting the extraction cache forced all sources through
|
||
|
|
extraction and reproduced the same graph and Logic hashes. Corrupting SQLite rebuilt the index
|
||
|
|
entirely from cache hits with zero parsing or extraction and reproduced those hashes.
|
||
|
|
|
||
|
|
This zero-parser evidence applies to the Python benchmark. Focused JavaScript and TypeScript tests
|
||
|
|
prove parser-free manifests. The C++ reference manifest uses Tree-sitter for bounded quoted-include
|
||
|
|
discovery and does not make a zero-warm-parser claim.
|
||
|
|
|
||
|
|
## Fresh-wheel and reference evidence
|
||
|
|
|
||
|
|
The offline adoption proof built and installed the base wheel without Tree-sitter packages,
|
||
|
|
built and checked a real Python reference project, started the isolated reference MCP server, and
|
||
|
|
performed bootstrap, search, and exact retrieval over its 21 read tools. Selecting C++ without its
|
||
|
|
extra failed with `optional_dependency_missing` and `docforge[cpp]` remediation.
|
||
|
|
|
||
|
|
Focused fixtures produced:
|
||
|
|
|
||
|
|
- Python: 14 nodes, 13 edges, 5 Logic projections.
|
||
|
|
- JavaScript: 14 nodes, 14 edges, 5 Logic projections.
|
||
|
|
- TypeScript: 13 nodes, 14 edges, 4 Logic projections.
|
||
|
|
- C++: 17 nodes, 16 edges, 5 Logic projections.
|
||
|
|
|
||
|
|
These fixture counts verify implementations; they are not language-wide completeness claims.
|