Add language-neutral project onboarding
This commit is contained in:
parent
73165c9f51
commit
09c09300b1
8 changed files with 924 additions and 9 deletions
|
|
@ -1,14 +1,15 @@
|
|||
# Active slice
|
||||
|
||||
```text
|
||||
Slice: DFG-14 durable graph navigation (complete)
|
||||
Goal: Make the generic graph browser durable across short MCP transactions and efficient for navigating dense project manuals.
|
||||
In scope: A browser-renewed listener lease; bounded abandoned-viewer shutdown; explicit disconnected state; resizable side panels; a draggable and resizable unblurred modal; topology-derived primary, child, and edge/context navigation sections; hop-ring layout; role palettes; progressive distance shading; keyboard-operable panel resizing; deterministic interaction checks; and complete regression verification.
|
||||
Out of scope: Graph mutation; source editing; persisted UI layout; project-specific relationship vocabulary; arbitrary templates; external hosting; canonical writes; unbounded listener lifetime; or non-loopback binding.
|
||||
Done when: An open viewer survives MCP transport completion, closes after its browser lease disappears or its process is terminated, all requested panels can be resized, the modal can be moved and resized without backdrop blur, every neighborhood exposes generic role sections, hop distance is visually encoded up to fifty-percent darkening, and the complete DocForge gate passes.
|
||||
Owners: DocForge owns viewer lease and generic presentation behavior. The configured project continues to own graph facts and relationship semantics. The process owner retains explicit termination authority.
|
||||
Slice: DFG-21 language-neutral project onboarding
|
||||
Goal: Let an unfamiliar codebase assess DocForge readiness and create a valid generic manual without implying that detected source languages already have semantic extraction.
|
||||
In scope: Read-only repository assessment; deterministic language and build-evidence detection; explicit multi-language selection; safe generic manual scaffolding; immediate index and render; a detailed language-neutral onboarding and adapter checklist; focused and complete quality proof.
|
||||
Out of scope: Bundled C++, Rust, Java, or other source frontends; dependency installation; project builds; Git mutation through DocForge; MCP self-installation; automatic canonical-document import; inferred source-to-manual relationships; deployment or publication.
|
||||
Done when: Assessment writes nothing, excluded trees and symlinks are ignored, mixed-language evidence is reported deterministically, scaffolding refuses conflicts and unsafe paths, a new project validates, indexes, and renders immediately, source graph status remains adapter_required, documentation explains the complete frontend and proof path, and the full DocForge quality gate passes.
|
||||
Owners: DocForge owns generic assessment, scaffolding, graph contracts, incremental compilation, rendering, visualization, and MCP boundaries. Each project or reusable frontend owns language-specific source discovery and semantic extraction. Canonical project files retain authority.
|
||||
Proof: The focused onboarding, CLI, and core suite passed 19 tests and 2 subtests. Strict Pyright passed with no errors or warnings. Ruff lint and formatting, Python compilation, and the HTML/CSS/JavaScript quality gate passed. The complete warning-strict suite passed 81 tests and 2 subtests. The scaffold test creates a Rust project, writes a confined generic configuration without replacement, builds one authoritative node, renders the manual, reopens the project, and retains source_graph_status=adapter_required.
|
||||
```
|
||||
|
||||
**Next gate:** None planned. Measure actual graph-browser use before extending layout, export,
|
||||
minimap, or remote-access policy. Canonical application remains permanently out of scope under
|
||||
`docs/APPLICATION_DECISION.md`.
|
||||
**Next gate:** Prove Worldforge's C++ integration against the generic incremental contract.
|
||||
Extract a reusable language frontend only after a second consumer demonstrates which behavior is
|
||||
genuinely shared.
|
||||
|
|
|
|||
12
README.md
12
README.md
|
|
@ -101,6 +101,16 @@ Start an MCP server for one project:
|
|||
Add `--canonical-applier project-editor` only when that MCP integration should expose the
|
||||
hash-bound `docforge_apply_changeset` tool.
|
||||
|
||||
For an unconfigured codebase, begin with a read-only language and documentation assessment:
|
||||
|
||||
```bash
|
||||
.venv/bin/docforge --project-root /absolute/path/MyProject onboard
|
||||
```
|
||||
|
||||
Add `--scaffold`, a stable project ID, and a title to create, index, and render a generic starter
|
||||
manual. Source files are reported separately and require a validated language frontend before
|
||||
DocForge describes them as a source graph.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [User manual](docs/USER_MANUAL.md) — features, setup, visualization, CLI, MCP, apply, adapters,
|
||||
|
|
@ -112,6 +122,8 @@ hash-bound `docforge_apply_changeset` tool.
|
|||
serialization.
|
||||
- [Incremental adapter indexing](docs/INCREMENTAL_INDEXING.md) — source-scoped extraction,
|
||||
invalidation, equivalence, relationship changes, and the lazy Logic boundary.
|
||||
- [Project onboarding](docs/PROJECT_ONBOARDING.md) — repository assessment, safe manual
|
||||
scaffolding, language frontends, source/manual integration, proof, and MCP activation.
|
||||
|
||||
## Development
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,28 @@
|
|||
# Completed slices
|
||||
|
||||
## DFG-21 language-neutral project onboarding
|
||||
|
||||
### Changed
|
||||
|
||||
- Added a read-only onboarding assessment that detects common source languages, build evidence,
|
||||
likely documentation, existing configuration, and capability readiness without writing files.
|
||||
- Added explicit language selection for C, C++, C#, Go, Java, JavaScript, Kotlin, Lua, PHP,
|
||||
Python, Ruby, Rust, Scala, Swift, and TypeScript.
|
||||
- Added conflict-safe generic scaffolding that creates project configuration, one authoritative
|
||||
overview, a built-in manual template, the derived index, and the rendered starter manual.
|
||||
- Kept language detection separate from semantic extraction. Every detected source language
|
||||
remains `adapter_required` until a language frontend passes the adapter contract.
|
||||
- Added the complete language-neutral onboarding checklist covering authority, manual import,
|
||||
frontend ownership, C++, Rust, and Java build evidence, incremental compilation, source/manual
|
||||
links, views, MCP activation, and maintenance.
|
||||
|
||||
### Verification
|
||||
|
||||
- Focused onboarding, CLI, and core tests passed 19 tests and 2 subtests.
|
||||
- Strict Pyright, Ruff lint and formatting, Python compilation, and the HTML/CSS/JavaScript quality
|
||||
gate passed.
|
||||
- The complete warning-strict suite passed 81 tests and 2 subtests.
|
||||
|
||||
## Dev-Rewrite multi-language Logic and traceable browser
|
||||
|
||||
### Changed
|
||||
|
|
|
|||
266
docs/PROJECT_ONBOARDING.md
Normal file
266
docs/PROJECT_ONBOARDING.md
Normal file
|
|
@ -0,0 +1,266 @@
|
|||
# Project onboarding
|
||||
|
||||
DocForge onboarding has two separate outcomes:
|
||||
|
||||
1. A generic manual can be configured, indexed, rendered, visualized, and exposed through the MCP.
|
||||
2. A source graph additionally requires one validated language frontend per source language.
|
||||
|
||||
The onboarding command never claims that source semantics exist merely because it found source
|
||||
files. It reports each detected language as `adapter_required` until a project integration supplies
|
||||
and proves that frontend.
|
||||
|
||||
## Start with a read-only assessment
|
||||
|
||||
```bash
|
||||
docforge --project-root /absolute/path/MyProject onboard
|
||||
```
|
||||
|
||||
The assessment:
|
||||
|
||||
- detects common source languages and build-system evidence;
|
||||
- excludes version-control, dependency, generated, cache, and build directories;
|
||||
- inventories likely documentation;
|
||||
- reports whether the project is already configured;
|
||||
- states which capabilities are ready and which still need an adapter;
|
||||
- does not create or modify files.
|
||||
|
||||
Limit detection to one or more known profiles when automatic discovery is not appropriate:
|
||||
|
||||
```bash
|
||||
docforge --project-root /absolute/path/MyProject onboard --language rust
|
||||
docforge --project-root /absolute/path/MyProject onboard --language java
|
||||
docforge --project-root /absolute/path/MyProject onboard \
|
||||
--language cpp \
|
||||
--language typescript
|
||||
```
|
||||
|
||||
Current profile IDs are `c`, `cpp`, `csharp`, `go`, `java`, `javascript`, `kotlin`, `lua`, `php`,
|
||||
`python`, `ruby`, `rust`, `scala`, `swift`, and `typescript`. A profile recognizes project
|
||||
evidence. It is not itself a parser.
|
||||
|
||||
## Scaffold a generic manual
|
||||
|
||||
After reviewing the assessment:
|
||||
|
||||
```bash
|
||||
docforge --project-root /absolute/path/MyProject onboard \
|
||||
--scaffold \
|
||||
--project-id my-project \
|
||||
--title "My Project"
|
||||
```
|
||||
|
||||
Scaffolding creates:
|
||||
|
||||
- `.docforge/project.toml`;
|
||||
- `.docforge/templates/manual.html`;
|
||||
- `docs/docforge/content/project-overview.md`;
|
||||
- the derived SQLite index;
|
||||
- the rendered starter manual.
|
||||
|
||||
The command refuses to replace any existing target. Canonical files are written before the
|
||||
descriptor, and a failed write removes files created by that attempt. The configured manual is
|
||||
immediately usable through the generic CLI, viewer, and MCP.
|
||||
|
||||
The starter overview records detected languages and states that the source graph is unavailable
|
||||
until a language frontend passes the adapter proof. That limitation is deliberate.
|
||||
|
||||
## Complete onboarding checklist
|
||||
|
||||
### 1. Repository assessment
|
||||
|
||||
- [ ] Resolve one explicit project root.
|
||||
- [ ] Detect version-control and worktree boundaries.
|
||||
- [ ] Detect source languages and build systems.
|
||||
- [ ] Find existing manuals, design notes, API references, plans, and proof records.
|
||||
- [ ] Exclude vendored, generated, dependency, cache, and build trees.
|
||||
- [ ] Estimate source, documentation, and expected graph size.
|
||||
- [ ] Report missing tools without changing the repository.
|
||||
- [ ] Review the assessment before scaffolding.
|
||||
|
||||
Done when authored source is distinguishable from disposable and external files.
|
||||
|
||||
### 2. Identity and authority
|
||||
|
||||
- [ ] Assign a stable project ID and title.
|
||||
- [ ] Declare canonical content roots.
|
||||
- [ ] Declare authority files.
|
||||
- [ ] Declare derived cache, changeset, preview, template, and render roots.
|
||||
- [ ] Define documentation families and allowed relationships.
|
||||
- [ ] Define proposal writers and operations.
|
||||
- [ ] Decide which views are public, internal, or restricted.
|
||||
- [ ] Keep source mutation disabled unless separately designed and authorized.
|
||||
|
||||
Done when every durable documentation fact has one authoritative source and every derived output
|
||||
can be deleted without losing that fact.
|
||||
|
||||
### 3. Manual foundation
|
||||
|
||||
- [ ] Scaffold or adapt `.docforge/project.toml`.
|
||||
- [ ] Create at least one authoritative overview node.
|
||||
- [ ] Assign stable node IDs, families, authorities, statuses, tags, and summaries.
|
||||
- [ ] Import existing documents without silently changing their meaning.
|
||||
- [ ] Separate current implementation, approved plans, proposals, and history.
|
||||
- [ ] Define bounded context profiles for common development tasks.
|
||||
- [ ] Validate, index, render, and visualize the manual.
|
||||
|
||||
Done when every rendered passage can be traced to a canonical source.
|
||||
|
||||
### 4. Language frontend selection
|
||||
|
||||
For every source language:
|
||||
|
||||
- [ ] Select or implement one frontend.
|
||||
- [ ] Record its frontend and extractor versions.
|
||||
- [ ] Define source discovery from authoritative build information.
|
||||
- [ ] Define stable symbol identities.
|
||||
- [ ] Define ownership for shared or generated declarations.
|
||||
- [ ] Define supported node kinds and relationships.
|
||||
- [ ] Define dependency discovery.
|
||||
- [ ] State unsupported semantic facts explicitly.
|
||||
|
||||
All frontends emit the same DocForge contracts:
|
||||
|
||||
- `AdapterManifest` inventories fingerprinted extraction units and dependencies.
|
||||
- `AdapterSourceProjection` owns nodes, relationships, and optional function Logic for one unit.
|
||||
- `AdapterProjection` provides the deterministic complete rebuild.
|
||||
|
||||
Language metadata may differ. Graph publication, indexing, querying, visualization, and MCP
|
||||
behavior do not.
|
||||
|
||||
Done when repeated extraction produces the same stable identities without inferred or guessed
|
||||
facts.
|
||||
|
||||
### 5. Build-system evidence
|
||||
|
||||
#### C and C++
|
||||
|
||||
- [ ] Use an authoritative compilation database.
|
||||
- [ ] Preserve target flags, definitions, language standards, and include paths.
|
||||
- [ ] Resolve headers shared by multiple translation units.
|
||||
- [ ] Assign shared symbols to one deterministic source contribution.
|
||||
- [ ] Record compiler-derived project include dependencies.
|
||||
|
||||
#### Rust
|
||||
|
||||
- [ ] Read the Cargo workspace and package graph.
|
||||
- [ ] Respect packages, targets, features, and conditional compilation.
|
||||
- [ ] Model crates, modules, traits, implementations, functions, and supported macros.
|
||||
- [ ] Treat expanded macro output as derived evidence.
|
||||
- [ ] Record the exact toolchain and extraction backend.
|
||||
|
||||
#### Java
|
||||
|
||||
- [ ] Read Gradle, Maven, or explicit source-root configuration.
|
||||
- [ ] Respect modules, source sets, language level, and classpath.
|
||||
- [ ] Model packages, classes, interfaces, records, methods, fields, and supported annotations.
|
||||
- [ ] Separate authored source from generated and annotation-processor output.
|
||||
- [ ] Record inheritance and interface implementation.
|
||||
|
||||
Other languages follow the same rule: the language frontend translates authoritative build and
|
||||
source evidence into the common adapter contract.
|
||||
|
||||
Done when a clean machine can reproduce the same source inventory from declared configuration.
|
||||
|
||||
### 6. Complete reference projection
|
||||
|
||||
- [ ] Extract the complete supported source tree.
|
||||
- [ ] Generate stable source and symbol nodes.
|
||||
- [ ] Generate only evidence-backed relationships.
|
||||
- [ ] Generate optional function-scoped Logic separately from the primary graph.
|
||||
- [ ] Reject duplicate node or Logic ownership.
|
||||
- [ ] Reject missing relationship endpoints.
|
||||
- [ ] Reject unsafe source paths.
|
||||
- [ ] Record project identity, source hash, counts, and duration.
|
||||
- [ ] Repeat the build and compare exact output.
|
||||
|
||||
Done when two unchanged complete builds are identical.
|
||||
|
||||
### 7. Incremental compilation
|
||||
|
||||
- [ ] Fingerprint each extraction unit.
|
||||
- [ ] Record extractor versions.
|
||||
- [ ] Record direct source dependencies.
|
||||
- [ ] Invalidate reverse dependents.
|
||||
- [ ] Remove deleted-source contributions.
|
||||
- [ ] Treat missing or corrupt caches as cache misses.
|
||||
- [ ] Publish cache and graph generations atomically.
|
||||
- [ ] Keep the complete projection as the equivalence oracle.
|
||||
|
||||
Required proof:
|
||||
|
||||
- [ ] cold build;
|
||||
- [ ] unchanged warm build;
|
||||
- [ ] implementation-file change;
|
||||
- [ ] shared-header or shared-module change;
|
||||
- [ ] added, renamed, and deleted source;
|
||||
- [ ] build-feature or compiler-setting change;
|
||||
- [ ] corrupt cache;
|
||||
- [ ] interrupted extraction;
|
||||
- [ ] complete-versus-incremental equivalence.
|
||||
|
||||
Done when incremental extraction produces exactly the complete projection.
|
||||
|
||||
### 8. Source and manual integration
|
||||
|
||||
- [ ] Link documented systems to their implementation.
|
||||
- [ ] Link API reference nodes to extracted symbols.
|
||||
- [ ] Link roadmap work to affected systems.
|
||||
- [ ] Link relevant tests and proof artifacts.
|
||||
- [ ] Report implemented but undocumented systems.
|
||||
- [ ] Report documented systems without implementation.
|
||||
- [ ] Keep uncertain links as proposals.
|
||||
- [ ] Keep source and manual projections independently rebuildable.
|
||||
|
||||
Done when a developer can navigate from a decision to implementation and back without guessing
|
||||
from filenames.
|
||||
|
||||
### 9. Views and MCP
|
||||
|
||||
- [ ] Configure the generic graph browser.
|
||||
- [ ] Configure manual, source, API, roadmap, and proof views as needed.
|
||||
- [ ] Verify search, filters, backlinks, dependencies, impact, and Logic.
|
||||
- [ ] Generate the exact project-bound MCP command.
|
||||
- [ ] Select read, proposal, and application capabilities explicitly.
|
||||
- [ ] Register and reload the client.
|
||||
- [ ] Call `docforge_bootstrap`.
|
||||
- [ ] Verify project ID, root fingerprint, adapter version, revision, source hash, and index health.
|
||||
- [ ] Verify the MCP cannot switch projects or weaken project authority.
|
||||
|
||||
Done when a new session can identify and retrieve the correct project without being told its file
|
||||
layout.
|
||||
|
||||
### 10. Operating guide and maintenance
|
||||
|
||||
- [ ] Record the authority and progressive-reading order.
|
||||
- [ ] Explain exact lookup, search, context, backlinks, and impact analysis.
|
||||
- [ ] Explain proposal, review, approval, and application.
|
||||
- [ ] Explain cache invalidation and recovery.
|
||||
- [ ] Explain frontend and adapter version changes.
|
||||
- [ ] Run complete/incremental equivalence in continuous integration.
|
||||
- [ ] Add contract tests for newly supported language features.
|
||||
- [ ] Never hand-resolve generated-output conflicts.
|
||||
- [ ] Never convert an inferred relationship into canonical truth silently.
|
||||
|
||||
Done when a developer unfamiliar with the repository can use DocForge without loading the entire
|
||||
manual or inventing another documentation workflow.
|
||||
|
||||
## CLI and MCP boundary
|
||||
|
||||
Initial assessment and scaffolding belong to the CLI because an MCP server cannot be registered
|
||||
until the project exists. The MCP begins at `docforge_bootstrap`, after its process has been fixed
|
||||
to one configured project root.
|
||||
|
||||
DocForge does not let an MCP call install dependencies, run project builds, modify Git, deploy, or
|
||||
publish. A project integration may use its own normal development workflow for those actions.
|
||||
|
||||
## Frontend packaging direction
|
||||
|
||||
Reusable language frontends should be separate packages or project-owned adapters over the public
|
||||
DocForge contracts. They must not put language-specific rules into the graph, index, viewer, or MCP
|
||||
core.
|
||||
|
||||
Worldforge is the first complete C++ reference integration. A reusable C++ package should be
|
||||
extracted only after that integration proves stable ownership, compiler dependency invalidation,
|
||||
and complete/incremental equivalence. Rust and Java frontends should then implement the same
|
||||
contract using their authoritative build and language tooling rather than copying C++ extraction
|
||||
rules.
|
||||
|
|
@ -92,6 +92,32 @@ uv run pytest -q
|
|||
Use the executables under `/absolute/path/DocForge/.venv/bin/` when DocForge is not installed into
|
||||
the active shell environment.
|
||||
|
||||
### Assess and onboard an unconfigured project
|
||||
|
||||
Run a read-only assessment before writing configuration:
|
||||
|
||||
```bash
|
||||
docforge --project-root /absolute/path/MyProject onboard
|
||||
```
|
||||
|
||||
The result reports detected languages, build evidence, likely documentation, existing
|
||||
configuration, and capability status. Detection does not claim that a language frontend exists.
|
||||
Limit the assessment with one or more `--language` options when needed.
|
||||
|
||||
Create, index, and render a generic starter manual explicitly:
|
||||
|
||||
```bash
|
||||
docforge --project-root /absolute/path/MyProject onboard \
|
||||
--language rust \
|
||||
--scaffold \
|
||||
--project-id my-project \
|
||||
--title "My Project"
|
||||
```
|
||||
|
||||
Scaffolding refuses to replace existing target files. It leaves source-graph status at
|
||||
`adapter_required` until a project integration implements and proves the adapter contract.
|
||||
See [Project onboarding](PROJECT_ONBOARDING.md) for the complete language-neutral checklist.
|
||||
|
||||
### Configure a generic project
|
||||
|
||||
Create `/absolute/path/MyProject/.docforge/project.toml`:
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ from .application import CanonicalApplicationService, GenericCanonicalApplier
|
|||
from .context import compile_context
|
||||
from .errors import DocForgeError
|
||||
from .index import ProjectIndex
|
||||
from .onboarding import assess_project, scaffold_project
|
||||
from .project import Project, project_root_fingerprint
|
||||
from .rendering import RenderService
|
||||
from .viewer_manager import ViewerManagerClient
|
||||
|
|
@ -21,6 +22,12 @@ def _parser() -> argparse.ArgumentParser:
|
|||
parser = argparse.ArgumentParser(prog="docforge")
|
||||
parser.add_argument("--project-root", type=Path, required=True)
|
||||
commands = parser.add_subparsers(dest="command", required=True)
|
||||
onboard = commands.add_parser("onboard")
|
||||
onboard.add_argument("--language", action="append", default=[])
|
||||
onboard.add_argument("--scaffold", action="store_true")
|
||||
onboard.add_argument("--project-id")
|
||||
onboard.add_argument("--title")
|
||||
onboard.add_argument("--content-root", default="docs/docforge/content")
|
||||
commands.add_parser("info")
|
||||
commands.add_parser("validate")
|
||||
commands.add_parser("build")
|
||||
|
|
@ -72,6 +79,21 @@ def _parser() -> argparse.ArgumentParser:
|
|||
|
||||
|
||||
def _run(arguments: argparse.Namespace) -> dict[str, object]:
|
||||
if arguments.command == "onboard":
|
||||
languages = tuple(arguments.language)
|
||||
if arguments.scaffold:
|
||||
scaffold = scaffold_project(
|
||||
arguments.project_root,
|
||||
requested_languages=languages,
|
||||
project_id=arguments.project_id,
|
||||
title=arguments.title,
|
||||
content_root=arguments.content_root,
|
||||
)
|
||||
project = Project.open(arguments.project_root)
|
||||
build = ProjectIndex(project).build()
|
||||
render = RenderService(project).render("manual")
|
||||
return {**scaffold, "build": build, "render": render}
|
||||
return assess_project(arguments.project_root, requested_languages=languages)
|
||||
project = Project.open(arguments.project_root)
|
||||
index = ProjectIndex(project)
|
||||
if arguments.command == "info":
|
||||
|
|
|
|||
450
src/docforge/onboarding.py
Normal file
450
src/docforge/onboarding.py
Normal file
|
|
@ -0,0 +1,450 @@
|
|||
"""Language-neutral project assessment and safe generic DocForge scaffolding."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from .errors import DocForgeError
|
||||
|
||||
_EXCLUDED_DIRECTORIES = frozenset(
|
||||
{
|
||||
".cache",
|
||||
".docforge",
|
||||
".git",
|
||||
".gradle",
|
||||
".idea",
|
||||
".mypy_cache",
|
||||
".pytest_cache",
|
||||
".ruff_cache",
|
||||
".tox",
|
||||
".venv",
|
||||
".vscode",
|
||||
"__pycache__",
|
||||
"_deps",
|
||||
"bin",
|
||||
"build",
|
||||
"coverage",
|
||||
"dist",
|
||||
"external",
|
||||
"generated",
|
||||
"node_modules",
|
||||
"obj",
|
||||
"out",
|
||||
"target",
|
||||
"third_party",
|
||||
"vendor",
|
||||
"venv",
|
||||
}
|
||||
)
|
||||
_PROTECTED_PARTS = frozenset({".git", ".ssh", ".gnupg", "secrets", "credentials"})
|
||||
_PROJECT_ID_PATTERN = re.compile(r"[a-z0-9][a-z0-9._-]{1,127}")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LanguageProfile:
|
||||
language_id: str
|
||||
title: str
|
||||
suffixes: tuple[str, ...]
|
||||
build_markers: tuple[str, ...]
|
||||
|
||||
|
||||
_LANGUAGE_PROFILES = (
|
||||
LanguageProfile(
|
||||
"c",
|
||||
"C",
|
||||
(".c",),
|
||||
("CMakeLists.txt", "meson.build", "Makefile", "configure.ac"),
|
||||
),
|
||||
LanguageProfile(
|
||||
"cpp",
|
||||
"C++",
|
||||
(".cc", ".cpp", ".cxx", ".hh", ".hpp", ".hxx"),
|
||||
("CMakeLists.txt", "meson.build", "Makefile", "conanfile.py", "vcpkg.json"),
|
||||
),
|
||||
LanguageProfile("csharp", "C#", (".cs",), (".sln", ".csproj", "global.json")),
|
||||
LanguageProfile("go", "Go", (".go",), ("go.mod", "go.work")),
|
||||
LanguageProfile(
|
||||
"java",
|
||||
"Java",
|
||||
(".java",),
|
||||
("build.gradle", "build.gradle.kts", "pom.xml", "settings.gradle"),
|
||||
),
|
||||
LanguageProfile(
|
||||
"javascript",
|
||||
"JavaScript",
|
||||
(".cjs", ".js", ".jsx", ".mjs"),
|
||||
("package.json",),
|
||||
),
|
||||
LanguageProfile(
|
||||
"kotlin",
|
||||
"Kotlin",
|
||||
(".kt", ".kts"),
|
||||
("build.gradle", "build.gradle.kts", "settings.gradle.kts"),
|
||||
),
|
||||
LanguageProfile("lua", "Lua", (".lua",), (".luacheckrc",)),
|
||||
LanguageProfile("php", "PHP", (".php",), ("composer.json",)),
|
||||
LanguageProfile(
|
||||
"python",
|
||||
"Python",
|
||||
(".py",),
|
||||
("pyproject.toml", "requirements.txt", "setup.py", "setup.cfg"),
|
||||
),
|
||||
LanguageProfile("ruby", "Ruby", (".rb",), ("Gemfile", ".ruby-version")),
|
||||
LanguageProfile(
|
||||
"rust",
|
||||
"Rust",
|
||||
(".rs",),
|
||||
("Cargo.toml", "Cargo.lock", "rust-toolchain.toml"),
|
||||
),
|
||||
LanguageProfile("scala", "Scala", (".scala",), ("build.sbt",)),
|
||||
LanguageProfile("swift", "Swift", (".swift",), ("Package.swift",)),
|
||||
LanguageProfile(
|
||||
"typescript",
|
||||
"TypeScript",
|
||||
(".ts", ".tsx"),
|
||||
("package.json", "tsconfig.json"),
|
||||
),
|
||||
)
|
||||
_PROFILES_BY_ID = {profile.language_id: profile for profile in _LANGUAGE_PROFILES}
|
||||
|
||||
|
||||
def _relative_project_path(root: Path, raw: str, *, field: str) -> Path:
|
||||
candidate = Path(raw)
|
||||
if candidate.is_absolute() or ".." in candidate.parts or not candidate.parts:
|
||||
raise DocForgeError("path_escape", f"{field} must stay inside the project root", path=raw)
|
||||
if any(part.lower() in _PROTECTED_PARTS for part in candidate.parts):
|
||||
raise DocForgeError("secret_path", f"{field} may not reference a protected path", path=raw)
|
||||
resolved = (root / candidate).resolve(strict=False)
|
||||
if not resolved.is_relative_to(root):
|
||||
raise DocForgeError("path_escape", f"{field} resolves outside the project root", path=raw)
|
||||
return candidate
|
||||
|
||||
|
||||
def _walk_project_files(root: Path) -> tuple[Path, ...]:
|
||||
files: list[Path] = []
|
||||
for directory, directory_names, file_names in os.walk(root, followlinks=False):
|
||||
current = Path(directory)
|
||||
directory_names[:] = sorted(
|
||||
name
|
||||
for name in directory_names
|
||||
if name not in _EXCLUDED_DIRECTORIES and not (current / name).is_symlink()
|
||||
)
|
||||
for name in sorted(file_names):
|
||||
path = current / name
|
||||
if not path.is_symlink():
|
||||
files.append(path.relative_to(root))
|
||||
return tuple(files)
|
||||
|
||||
|
||||
def _normalize_requested_languages(requested: tuple[str, ...]) -> tuple[str, ...]:
|
||||
if not requested or requested == ("auto",):
|
||||
return ()
|
||||
values = tuple(sorted(set(item.strip().lower() for item in requested if item.strip())))
|
||||
if "auto" in values:
|
||||
raise DocForgeError(
|
||||
"invalid_onboarding",
|
||||
"language auto cannot be combined with explicit language profiles",
|
||||
)
|
||||
unknown = tuple(item for item in values if item not in _PROFILES_BY_ID)
|
||||
if unknown:
|
||||
raise DocForgeError(
|
||||
"unsupported_language_profile",
|
||||
"One or more language profiles are not recognized",
|
||||
languages=list(unknown),
|
||||
supported=sorted(_PROFILES_BY_ID),
|
||||
)
|
||||
return values
|
||||
|
||||
|
||||
def _language_inventory(
|
||||
root: Path, files: tuple[Path, ...], requested: tuple[str, ...]
|
||||
) -> tuple[dict[str, object], ...]:
|
||||
explicit = _normalize_requested_languages(requested)
|
||||
profiles = tuple(_PROFILES_BY_ID[item] for item in explicit) if explicit else _LANGUAGE_PROFILES
|
||||
names = {path.name for path in files}
|
||||
inventory: list[dict[str, object]] = []
|
||||
for profile in profiles:
|
||||
source_count = sum(path.suffix.lower() in profile.suffixes for path in files)
|
||||
markers = sorted(marker for marker in profile.build_markers if marker in names)
|
||||
if source_count or explicit:
|
||||
inventory.append(
|
||||
{
|
||||
"id": profile.language_id,
|
||||
"title": profile.title,
|
||||
"source_files": source_count,
|
||||
"build_evidence": markers,
|
||||
"frontend_status": "adapter_required",
|
||||
}
|
||||
)
|
||||
return tuple(sorted(inventory, key=lambda item: str(item["id"])))
|
||||
|
||||
|
||||
def _documentation_inventory(files: tuple[Path, ...]) -> tuple[str, ...]:
|
||||
candidates = {
|
||||
path.as_posix()
|
||||
for path in files
|
||||
if path.suffix.lower() in {".md", ".mdx", ".rst", ".toml"}
|
||||
and (
|
||||
path.name.lower().startswith(("readme", "architecture", "design", "manual"))
|
||||
or any(part.lower() in {"doc", "docs", "manual"} for part in path.parts[:-1])
|
||||
)
|
||||
}
|
||||
return tuple(sorted(candidates))
|
||||
|
||||
|
||||
def _default_project_id(root: Path) -> str:
|
||||
value = re.sub(r"[^a-z0-9._-]+", "-", root.name.lower()).strip("-._")
|
||||
if len(value) < 2:
|
||||
value = f"{value or 'project'}-docs"
|
||||
return value[:128]
|
||||
|
||||
|
||||
def assess_project(root: Path, *, requested_languages: tuple[str, ...] = ()) -> dict[str, object]:
|
||||
"""Return a deterministic, read-only onboarding assessment."""
|
||||
|
||||
resolved = root.resolve(strict=True)
|
||||
if not resolved.is_dir():
|
||||
raise DocForgeError("invalid_project_root", "Project root must be a directory")
|
||||
files = _walk_project_files(resolved)
|
||||
languages = _language_inventory(resolved, files, requested_languages)
|
||||
existing_descriptor = resolved / ".docforge" / "project.toml"
|
||||
documentation = _documentation_inventory(files)
|
||||
return {
|
||||
"status": "ok",
|
||||
"mode": "assessment",
|
||||
"project_root": str(resolved),
|
||||
"project_id_suggestion": _default_project_id(resolved),
|
||||
"file_count": len(files),
|
||||
"languages": list(languages),
|
||||
"documentation_candidates": list(documentation),
|
||||
"configured": existing_descriptor.is_file(),
|
||||
"capabilities": {
|
||||
"manual_scaffold": "available" if not existing_descriptor.exists() else "configured",
|
||||
"source_graph": ("adapter_required" if languages else "no_supported_source_detected"),
|
||||
"incremental_compilation": "available_after_adapter",
|
||||
"mcp": "available_after_configuration",
|
||||
"viewer": "available_after_index",
|
||||
},
|
||||
"next_actions": [
|
||||
"Review detected languages and documentation authority.",
|
||||
(
|
||||
"Run onboard with --scaffold to create a generic manual when the project is "
|
||||
"unconfigured."
|
||||
),
|
||||
"Implement or select one language frontend per source language.",
|
||||
"Prove full and incremental projection equivalence.",
|
||||
"Generate and register the fixed project MCP command.",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _toml_string(value: str) -> str:
|
||||
return json.dumps(value, ensure_ascii=False)
|
||||
|
||||
|
||||
def _descriptor(project_id: str, title: str, content_root: Path) -> str:
|
||||
content = content_root.as_posix()
|
||||
return f"""schema_version = 1
|
||||
project_id = {_toml_string(project_id)}
|
||||
title = {_toml_string(title)}
|
||||
adapter = "generic"
|
||||
|
||||
[sources]
|
||||
content_roots = [{_toml_string(content)}]
|
||||
authority_files = []
|
||||
|
||||
[derived]
|
||||
cache_root = ".docforge/cache"
|
||||
index = ".docforge/cache/index.sqlite3"
|
||||
|
||||
[changesets]
|
||||
root = ".docforge/changesets"
|
||||
|
||||
[[changesets.writers]]
|
||||
id = "project-editor"
|
||||
families = ["api", "architecture", "operations", "proof", "roadmap", "system"]
|
||||
operations = ["create", "update", "move", "delete"]
|
||||
|
||||
[render]
|
||||
template_root = ".docforge/templates"
|
||||
preview_root = ".docforge/previews"
|
||||
|
||||
[[render.views]]
|
||||
id = "manual"
|
||||
renderer = "generic_html"
|
||||
template = "manual.html"
|
||||
output = ".docforge/rendered/manual.html"
|
||||
title = {_toml_string(f"{title} Manual")}
|
||||
families = ["api", "architecture", "operations", "proof", "roadmap", "system"]
|
||||
|
||||
[graph]
|
||||
allowed_relations = [
|
||||
"calls",
|
||||
"defines",
|
||||
"depends_on",
|
||||
"implements",
|
||||
"inherits_from",
|
||||
"owns",
|
||||
"reads",
|
||||
"relates_to",
|
||||
"tested_by",
|
||||
"writes",
|
||||
]
|
||||
|
||||
[limits]
|
||||
max_source_bytes = 500000
|
||||
max_nodes = 10000
|
||||
max_query_chars = 500
|
||||
max_results = 100
|
||||
max_traversal_depth = 6
|
||||
max_context_tokens = 12000
|
||||
max_changesets = 100
|
||||
max_changeset_operations = 100
|
||||
max_changeset_bytes = 1000000
|
||||
|
||||
[[profiles]]
|
||||
id = "development"
|
||||
families = ["api", "architecture", "operations", "proof", "roadmap", "system"]
|
||||
statuses = ["active", "current", "verified"]
|
||||
required_nodes = ["architecture.overview"]
|
||||
token_budget = 8000
|
||||
dependency_depth = 3
|
||||
"""
|
||||
|
||||
|
||||
def _overview(title: str, languages: tuple[dict[str, object], ...]) -> str:
|
||||
language_titles = [str(item["title"]) for item in languages]
|
||||
tags = ["architecture", "onboarding", *[str(item["id"]) for item in languages]]
|
||||
language_text = ", ".join(language_titles) if language_titles else "No source language selected"
|
||||
return f"""+++
|
||||
schema_version = 1
|
||||
id = "architecture.overview"
|
||||
title = "Project architecture"
|
||||
family = "architecture"
|
||||
authority = "authoritative"
|
||||
status = "current"
|
||||
tags = {json.dumps(tags)}
|
||||
summary = "Introduces the project and its documentation authority."
|
||||
+++
|
||||
|
||||
# {title}
|
||||
|
||||
DocForge indexes the canonical documentation under this directory. Derived indexes, rendered
|
||||
pages, previews, and extraction caches may be deleted and rebuilt.
|
||||
|
||||
Detected or selected source languages: {language_text}.
|
||||
|
||||
Source-code facts require a language frontend that implements DocForge's adapter contract. Until
|
||||
that frontend passes full and incremental equivalence checks, this manual remains authoritative
|
||||
and the source graph remains explicitly unavailable.
|
||||
"""
|
||||
|
||||
|
||||
_TEMPLATE = """<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="docforge-render" content="{{ docforge_render_identity }}">
|
||||
<title>{{ docforge_title }}</title>
|
||||
<style>
|
||||
:root { color-scheme: dark; font-family: system-ui, sans-serif; }
|
||||
body { margin: 0; background: #0e1518; color: #dce9e6; }
|
||||
header, main { width: min(70rem, calc(100% - 3rem)); margin: auto; }
|
||||
header { padding: 3rem 0 1rem; border-bottom: 1px solid #31504c; }
|
||||
main { padding: 2rem 0 5rem; line-height: 1.65; }
|
||||
article { margin: 0 0 2rem; padding: 1.5rem; background: #14211f; border: 1px solid #31504c; }
|
||||
a { color: #78ddd0; }
|
||||
code { color: #f1c779; }
|
||||
</style>
|
||||
</head>
|
||||
<body data-project="{{ docforge_project_id }}" data-view="{{ docforge_view_id }}">
|
||||
<header><h1>{{ docforge_title }}</h1></header>
|
||||
<main>{{ docforge_content }}</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
|
||||
def _write_new(path: Path, content: str) -> None:
|
||||
if path.exists() or path.is_symlink():
|
||||
raise DocForgeError(
|
||||
"onboarding_conflict",
|
||||
"Onboarding will not replace an existing path",
|
||||
path=str(path),
|
||||
)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_name(f".{path.name}.docforge-new")
|
||||
try:
|
||||
temporary.write_text(content, encoding="utf-8")
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
temporary.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def scaffold_project(
|
||||
root: Path,
|
||||
*,
|
||||
requested_languages: tuple[str, ...] = (),
|
||||
project_id: str | None = None,
|
||||
title: str | None = None,
|
||||
content_root: str = "docs/docforge/content",
|
||||
) -> dict[str, object]:
|
||||
"""Create a minimal generic manual without pretending a source adapter exists."""
|
||||
|
||||
resolved = root.resolve(strict=True)
|
||||
assessment = assess_project(resolved, requested_languages=requested_languages)
|
||||
language_values = cast(list[object], assessment["languages"])
|
||||
selected_languages = tuple(
|
||||
cast(dict[str, object], item) for item in language_values if isinstance(item, dict)
|
||||
)
|
||||
resolved_id = project_id or str(assessment["project_id_suggestion"])
|
||||
if _PROJECT_ID_PATTERN.fullmatch(resolved_id) is None:
|
||||
raise DocForgeError(
|
||||
"invalid_onboarding", "project_id is not a stable DocForge ID", project_id=resolved_id
|
||||
)
|
||||
resolved_title = title.strip() if title and title.strip() else resolved.name
|
||||
relative_content = _relative_project_path(resolved, content_root, field="onboard.content_root")
|
||||
targets = (
|
||||
resolved / ".docforge" / "project.toml",
|
||||
resolved / ".docforge" / "templates" / "manual.html",
|
||||
resolved / relative_content / "project-overview.md",
|
||||
)
|
||||
conflicts = [str(path) for path in targets if path.exists() or path.is_symlink()]
|
||||
if conflicts:
|
||||
raise DocForgeError(
|
||||
"onboarding_conflict",
|
||||
"Onboarding will not replace existing project files",
|
||||
paths=conflicts,
|
||||
)
|
||||
|
||||
created: list[Path] = []
|
||||
try:
|
||||
template = targets[1]
|
||||
_write_new(template, _TEMPLATE)
|
||||
created.append(template)
|
||||
overview = targets[2]
|
||||
_write_new(overview, _overview(resolved_title, selected_languages))
|
||||
created.append(overview)
|
||||
descriptor = targets[0]
|
||||
_write_new(descriptor, _descriptor(resolved_id, resolved_title, relative_content))
|
||||
created.append(descriptor)
|
||||
except Exception:
|
||||
for path in reversed(created):
|
||||
path.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
return {
|
||||
**assessment,
|
||||
"mode": "scaffold",
|
||||
"configured": True,
|
||||
"project_id": resolved_id,
|
||||
"title": resolved_title,
|
||||
"created": [str(path.relative_to(resolved)) for path in created],
|
||||
"source_graph_status": "adapter_required",
|
||||
}
|
||||
115
tests/test_onboarding.py
Normal file
115
tests/test_onboarding.py
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from docforge.cli import _parser, _run
|
||||
from docforge.errors import DocForgeError
|
||||
from docforge.project import Project
|
||||
|
||||
|
||||
class DocForgeOnboardingTests(unittest.TestCase):
|
||||
def test_assessment_detects_multiple_languages_without_writing(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory) / "polyglot"
|
||||
(root / "src").mkdir(parents=True)
|
||||
(root / "src/main.rs").write_text("fn main() {}\n", encoding="utf-8")
|
||||
(root / "src/App.java").write_text("class App {}\n", encoding="utf-8")
|
||||
(root / "Cargo.toml").write_text("[package]\nname='polyglot'\n", encoding="utf-8")
|
||||
(root / "pom.xml").write_text("<project/>\n", encoding="utf-8")
|
||||
(root / "target/generated").mkdir(parents=True)
|
||||
(root / "target/generated/ignored.rs").write_text("", encoding="utf-8")
|
||||
|
||||
result = assess = _run(_parser().parse_args(["--project-root", str(root), "onboard"]))
|
||||
|
||||
self.assertEqual("assessment", result["mode"])
|
||||
self.assertFalse(result["configured"])
|
||||
self.assertFalse((root / ".docforge").exists())
|
||||
languages = {item["id"]: item for item in assess["languages"]}
|
||||
self.assertEqual(1, languages["rust"]["source_files"])
|
||||
self.assertEqual(["Cargo.toml"], languages["rust"]["build_evidence"])
|
||||
self.assertEqual(1, languages["java"]["source_files"])
|
||||
self.assertEqual(["pom.xml"], languages["java"]["build_evidence"])
|
||||
|
||||
def test_explicit_language_keeps_zero_source_profile_visible(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
result = _run(
|
||||
_parser().parse_args(["--project-root", str(root), "onboard", "--language", "rust"])
|
||||
)
|
||||
self.assertEqual(
|
||||
[
|
||||
{
|
||||
"id": "rust",
|
||||
"title": "Rust",
|
||||
"source_files": 0,
|
||||
"build_evidence": [],
|
||||
"frontend_status": "adapter_required",
|
||||
}
|
||||
],
|
||||
result["languages"],
|
||||
)
|
||||
|
||||
def test_scaffold_builds_and_renders_a_valid_generic_manual(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory) / "ski-game"
|
||||
(root / "src").mkdir(parents=True)
|
||||
(root / "src/lib.rs").write_text("pub fn carve() {}\n", encoding="utf-8")
|
||||
result = _run(
|
||||
_parser().parse_args(
|
||||
[
|
||||
"--project-root",
|
||||
str(root),
|
||||
"onboard",
|
||||
"--language",
|
||||
"rust",
|
||||
"--scaffold",
|
||||
"--project-id",
|
||||
"awesome-ski-game",
|
||||
"--title",
|
||||
"Awesome Ski Game",
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
self.assertEqual("scaffold", result["mode"])
|
||||
self.assertEqual("adapter_required", result["source_graph_status"])
|
||||
self.assertEqual(1, result["build"]["node_count"])
|
||||
self.assertTrue((root / ".docforge/rendered/manual.html").is_file())
|
||||
snapshot = Project.open(root).load()
|
||||
self.assertEqual("awesome-ski-game", snapshot.descriptor.project_id)
|
||||
self.assertEqual("architecture.overview", snapshot.nodes[0].node_id)
|
||||
self.assertIn("Rust", snapshot.nodes[0].content)
|
||||
|
||||
def test_scaffold_refuses_existing_files_and_unsafe_paths(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
(root / ".docforge").mkdir()
|
||||
(root / ".docforge/project.toml").write_text("existing\n", encoding="utf-8")
|
||||
arguments = _parser().parse_args(["--project-root", str(root), "onboard", "--scaffold"])
|
||||
with self.assertRaisesRegex(DocForgeError, "replace existing"):
|
||||
_run(arguments)
|
||||
self.assertEqual(
|
||||
"existing\n",
|
||||
(root / ".docforge/project.toml").read_text(encoding="utf-8"),
|
||||
)
|
||||
|
||||
clean = Path(directory) / "clean"
|
||||
clean.mkdir()
|
||||
escaped = _parser().parse_args(
|
||||
[
|
||||
"--project-root",
|
||||
str(clean),
|
||||
"onboard",
|
||||
"--scaffold",
|
||||
"--content-root",
|
||||
"../outside",
|
||||
]
|
||||
)
|
||||
with self.assertRaisesRegex(DocForgeError, "inside the project root"):
|
||||
_run(escaped)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Loading…
Add table
Add a link
Reference in a new issue