**ontoref** is a protocol specification and tooling layer for structured self-knowledge in software projects. It provides schemas, Nushell automation, and Rust crates so that projects can describe what they are, record architectural decisions, track operational state, and execute formalized procedures — all as typed, queryable artifacts. --- ## Axioms | Axiom | Meaning | | --- | --- | | **Protocol, Not Runtime** | Never a runtime dependency. Projects adopt the protocol; ontoref provides the schemas and modules to do so. | | **Self-Describing** | Consumes its own protocol: `.ontology/`, `adrs/`, `reflection/` in this repo ARE ontoref running against itself. | | **No Enforcement** | ontoref defines contracts. There is no enforcement mechanism. Coherence is voluntary and emerges from justified adoption. | | **DAG-Formalized Knowledge** | Concepts, tensions, decisions, state — encoded as DAGs. Enables transversal queries and impact analysis. | ## Layers ```text ontology/ Protocol specification — Nickel schemas for nodes, edges, ADRs, state, gates adrs/ Architecture Decision Records — typed NCL with constraints and ontology checks reflection/ Operational tooling — Nushell modules, DAG modes, forms, and schemas crates/ Rust implementation — typed struct loaders and mode executors .ontology/ Self-description — ontoref's own ontology, state, gate, and manifest ``` ## Crates | Crate | Purpose | | --- | --- | | `ontoref-ontology` | `.ontology/` NCL → typed Rust structs: Node, Edge, Dimension, Gate, Membrane. `Node` carries `artifact_paths` and `adrs` (`Vec`, both `serde(default)`). Graph traversal, invariant queries. Zero deps. | | `ontoref-reflection` | NCL DAG contract executor with guards (pre-flight Block/Warn checks) and convergence loops (RetryFailed/RetryAll). ADR lifecycle, step dep resolution, config seal. `stratum-graph` + `stratum-state` required. | | `ontoref-daemon` | HTTP UI (11 pages), actor registry, notification barrier, MCP (34 tools), search engine, search bookmarks, SurrealDB, NCL export cache, per-file ontology versioning, annotated API catalog, Agent Task Composer. | | `ontoref-derive` | Proc-macro crate. `#[onto_api(...)]` annotates HTTP handlers — `description` is optional when a `///` doc comment exists (first line used as fallback). `#[onto_mcp_tool(name, description, input_schema)]` registers MCP tool unit-structs in the catalog at link time via `inventory::submit!(McpToolEntry{...})`; the annotated item is emitted unchanged and `ToolBase`/`AsyncTool` impls remain on the struct. `#[derive(OntologyNode)]` + `#[onto(id, name, paths, description, adrs)]` auto-registers nodes via `inventory::submit!` at link time, merged into `Core` by `merge_contributors()`. `#[derive(ConfigFields)]` + `#[config_section(id, ncl_file)]` registers config struct fields. All four aggregate via `inventory::collect!`. | `ontoref-daemon` caches `nickel export` results (keyed by path + mtime), reducing full sync scans from ~2m42s to <30s. The daemon is always optional — every module falls back to direct subprocess when unavailable. ### Daemon Capabilities **Unified Auth Model** — all surfaces (CLI, UI, MCP) exchange a key for a UUID v4 session token via `POST /sessions`. Token lifetime: 30 days, O(1) lookup. Project keys carry `role` (admin|viewer) and `label` for audit trail. Daemon-level admin via `ONTOREF_ADMIN_TOKEN_FILE`. `GET /sessions` and `DELETE /sessions/{id}` for session visibility and revocation. Key rotation invalidates all sessions for the rotated project. CLI injects `ONTOREF_TOKEN` as Bearer automatically. **Q&A Knowledge Store** — accumulated Q&A entries persist to `reflection/qa.ncl` (typed NCL, git-versioned). Not localStorage. Any actor — developer, agent, CI — reads the same store. **MCP Server** — 34 tools over stdio and streamable-HTTP, all registered at link time via `#[onto_mcp_tool]` (no manual catalog wiring). Categories: discovery, retrieval, project state, ontology, backlog, validation, Q&A, bookmarks, API surface. Representative subset: | Tool | What it does | | --- | --- | | `ontoref_guides` | Full project context on cold start: axioms, practices, gate, actor policy | | `ontoref_api_catalog` | Annotated HTTP surface — all routes with auth, actors, params, tags | | `ontoref_file_versions` | Per-file reload counters — detect which ontology files changed | | `ontoref_validate_adrs` | Run typed ADR constraint checks; returns pass/fail per constraint | | `ontoref_validate` | Full project validation: ADRs, content assets, connections, gate consistency, manifest coverage | | `ontoref_impact` | BFS impact graph from a node, optionally across project connections | | `ontoref_qa_list` | List Q&A entries with optional filter | | `ontoref_qa_add` | Append a new Q&A entry to `reflection/qa.ncl` | | `ontoref_action_list` | List quick actions from `.ontoref/config.ncl` | | `ontoref_action_add` | Create a reflection mode + register as a quick action | **Search Bookmarks** — search results persist to `reflection/search_bookmarks.ncl` (typed NCL, `BookmarkEntry` schema). Same atomic-write pattern as Q&A. IDs are sequential `sb-NNN`. Concurrency-safe via `NclWriteLock`. Add and remove from the daemon search UI. **Personal Ontology Schemas** — `ontology/schemas/career.ncl`, `personal.ncl`, `project-card.ncl` provide typed contract layers for career and content artifacts (Skills, WorkExperience, Talks, Content lifecycle, Opportunities, PublicationCards). All types carry `linked_nodes` referencing core ontology node IDs — bridging career artifacts into the DAG. Five content/career reflection modes (`draft-application`, `draft-email`, `generate-article`, `update-cv`, `write-cfp`) query these schemas to ground output in declared project artifacts rather than free-form prose. **API Catalog** — every HTTP handler carries `#[onto_api(method, path, auth, actors, params, tags)]`. `description` is sourced from the first `///` doc line above the handler — no duplication with doc comments. At link time `inventory::submit!` registers each route. `GET /api/catalog` returns the full annotated surface as JSON. The `/ui/{slug}/api` page renders it with client-side filtering (method, auth, path). `describe api [--actor] [--tag] [--fmt]` renders the catalog in the CLI. `ontoref_api_catalog` exposes it to MCP agents. **Semantic Diff** — `describe diff [--file ] [--fmt json|text]` computes a node- and edge-level diff of `.ontology/` files against the last git commit. Reports added/removed/changed nodes by id and edges by `from→to[kind]` key — not a text diff. **Per-File Versioning** — each ontology file tracked in `ProjectContext.file_versions: DashMap`. Counter increments on every watcher-triggered reload. `GET /projects/{slug}/ontology/versions` and `ontoref_file_versions` MCP tool expose the map. Dashboard surfaces the counters. **ADR–Node Linkage** — nodes declare which ADRs validate them via `adrs: Array String`. `describe` surfaces a **Validated by** section per node (CLI and `--fmt md`). The graph UI renders each ADR as a clickable link that opens the full ADR content via `GET /api/adr/{id}`. **Browser-Style Panel Navigation** — graph, search, and api_catalog UI pages carry a back/forward history stack (cursor-into-array model). Clicking nodes, ADRs, or search results pushes to history; clicking artifacts opens the source file in the configured repository or docs. `card.repo` in `card.ncl` resolves to `{repo}/src/branch/main/{path}` (Gitea format). For `.rs` files, `card.docs` redirects to the cargo docs URL instead. `insert_brand_ctx` injects both as `card_repo`/`card_docs` into every Tera template. **Passive Drift Observation** — background file watcher that detects divergence between Yang code artifacts and Yin ontology. Watches `crates/`, `.ontology/`, `adrs/`, `reflection/modes/`. After a 15s debounce runs `sync scan + sync diff`; emits an `ontology_drift` notification when MISSING/STALE/DRIFT/BROKEN items are found. Never applies changes — `apply` is always deliberate. **Quick Actions** — runnable shortcuts over reflection modes, configured as `quick_actions` in `.ontoref/config.ncl`. Accessible from HTTP (`/actions`), CLI (`ontoref`), and MCP (`ontoref_action_list/add`). **Config Surface** — per-project config introspection, coherence verification, and documented mutation. Rust structs annotated with `#[derive(ConfigFields)]` + `#[config_section(id, ncl_file)]` register their field names at link time via `inventory::submit!(ConfigFieldsEntry{...})`. The daemon queries `inventory::iter::()` at startup to build a zero-maintenance registry of which Rust fields each struct reads from each NCL section. Multi-consumer coherence (`GET /projects/{slug}/config/coherence`) compares the inventory registry against NCL export keys, Nu script accessor patterns, and CI fields declared in `manifest.ncl` — any NCL field claimed by no consumer is flagged unclaimed. `GET /projects/{slug}/config/quickref` generates living documentation (rationales, override history, coherence status) on demand. Config mutation never modifies source NCL files. `PUT /projects/{slug}/config/{section}` writes a `{section}.overrides.ncl` file with only the changed fields plus a `_overrides_meta` audit record (actor, reason, timestamp, previous value), then appends a single idempotent import line to the entry-point NCL using the `&` merge operator. `nickel export` validates the merged result against the section's declared contract before committing; contract violations revert the override file and return the nickel error verbatim. NCL contracts (`std.contract.from_validator`) are the single validation gate — Rust structs are contract-trusted readers with `#[serde(default)]`. Ontoref demonstrates the pattern on itself: `.ontoref/contracts.ncl` applies `LogConfig` and `DaemonConfig` contracts to `.ontoref/config.ncl`. ([ADR-008](adrs/adr-008-ncl-first-config-validation-and-override-layer.ncl)) **Protocol Migration System** — protocol upgrades for consumer projects expressed as ordered NCL files in `reflection/migrations/NNN-slug.ncl`. Each migration declares a typed check (`FileExists | Grep | NuCmd`) whose result IS the applied state — no state file, fully idempotent. `migrate list` shows all migrations with applied/pending status; `migrate pending` lists only what is missing; `migrate show ` renders runtime-interpolated instructions (project_root and project_name auto-detected). NuCmd checks are valid Nushell (no bash `&&`, `$env.VAR` not `$VAR`). Grep checks targeting ADR files scope to `adr-[0-9][0-9][0-9]-*.ncl` to exclude schema/template infrastructure files. 12 migrations shipped; `0012-rust-doc-authoring-pattern` adds the `/// → //! → node description` three-layer doc convention and optional pre-commit hooks (`docs-links`, `docs-drift`) to consumer `CLAUDE.md`. ([ADR-010](adrs/adr-010-protocol-migration-system.ncl)) **Manifest Self-Interrogation** — `manifest_type` gains three typed arrays that answer self-knowledge queries agents and operators need on cold start: `capabilities[]` (what the project does, why it was built, how it works — with explicit `nodes[]` and `adrs[]` cross-references into the DAG), `requirements[]` (prerequisites classified by environment: `'Production | 'Development | 'Both` and kind: `'Tool | 'Service | 'EnvVar | 'Infrastructure`), `critical_deps[]` (external dependencies with required `failure_impact` and optional `mitigation`). `describe requirements` surfaces these; `describe guides` and `ontoref_guides` include all three arrays in their output. ([ADR-009](adrs/adr-009-manifest-self-interrogation-layer-three-semantic-axes.ncl)) **Domain Extension System** — CLI commands conditional on `repo_kind`. The ontoref bash wrapper resolves the first argument against `$ONTOREF_ROOT/domains/{id}/repo_kinds.txt` before delegating to the Nu dispatcher; if the project's `repo_kind` matches, the domain's `commands.nu` is dispatched directly. Each domain ships `domain.ncl` (typed contract: commands, pages, short_alias, repo_kinds), `commands.nu` (Nu entry point), and `repo_kinds.txt` (grep-readable, sub-1ms dispatch). Two domains shipped: `personal` (PersonalOntology — cfp, career, content, opportunities) and `provisioning` (DevWorkspace/Mixed — state, connections, gates, card, backlog). Short aliases (`personal`, `prov`) work as both `ore prov ` and standalone `prov `. `ore help` and `describe capabilities` surface the active domain automatically. New domains require only three files — no changes to the Nu dispatcher. ([ADR-012](adrs/adr-012-domain-extension-system.ncl)) **Mode Hierarchy Validation** — `validate modes [--check]` reads `reflection/defaults/workflow.ncl::level_hierarchy` and checks every `.ncl` mode file for: level declared, strategy declared, delegate chain coherent, compose `extends` references valid. `mode resolve ` prints which hierarchy level handles the given mode and why. `validate modes --self-test` generates synthetic fixtures in a temp dir for fast CI smoke-testing of the validator itself. ([ADR-018](adrs/adr-018-workflow-layer-model.ncl)) **Project Picker — vault and registry badges** — each project card surfaces OCI state inline: registry participant badge (`⟳ `) when `registry_provides` is declared; vault badge (`⛁ · N`) coloured green (declarative) or amber (legacy) when `sops.enabled` is true. Expanded project panel shows a collapsible **Registry** section with namespace, endpoint, and push/pull capability. The manage page adds **Runtime Services** toggles — MCP and GraphQL can be switched without a daemon restart via HTMX `POST /ui/manage/services/{service}/toggle`. **VCS Abstraction Layer** — `reflection/modules/vcs.nu` exposes a uniform API over jj and git: `detect`, `show-committed`, `restore-file`, `remote-url`, `current-branch`, `uncommitted-files`, `commit-count`. All ontoref modules consume `vcs.nu` — never hardcoded `^git`. Detection is filesystem-based (`.jj/` vs `.git/`), no config required. jj is opt-in: all operations degrade to git when `.jj/` is absent. `reflection/bin/jjw.nu` wraps jj workspaces, ontoref runs, and optional Radicle patch submission into a single `jjw agent create|step|publish|merge|discard` lifecycle for agent-driven development. `jjw-ncl-merge.nu` is a jj merge tool for `.ontology/` NCL conflicts, registered manually in `~/.config/jj/config.toml`. jj and Radicle are not protocol requirements — consumer projects use plain git without any configuration change. ## Install ```sh just install-daemon # build + install binary, bootstrapper, CLI, UI assets, config skeleton ontoref config-edit # browser form → ~/.config/ontoref/config.ncl ontoref-daemon-boot # NCL pipe bootstrap: nickel export config.ncl | daemon --config-stdin ontoref-daemon-boot --dry-run # preview composed JSON without starting ``` Installed layout (`~/.local/bin/`): | Binary | Role | | --- | --- | | `ontoref` | Global CLI dispatcher — all reflection modes, ADR lifecycle, daemon control | | `ontoref-daemon` | Bootstrapper (public entrypoint) — validates config via Nickel, pipes JSON to binary | | `ontoref-daemon.bin` | Compiled Rust binary — never called directly | Global config at `~/.config/ontoref/config.ncl` (type-checked Nickel). Global NATS stream topology at `~/.config/ontoref/streams.json`. Project-local topology override via `nats/streams.json` + `nats_events.streams_config` in `.ontoref/config.ncl`. ## Onboarding a project ```sh cd /path/to/my-project ontoref setup # idempotent; kind: Service by default ontoref setup --kind Library # Library | Service | DevWorkspace | PublishedCrate | AgentResource | Mixed | PersonalOntology ontoref setup --parent /path/to/fw # implementation child: adds framework layer + browse mode ontoref setup --gen-keys ["admin:dev" "viewer:ci"] # bootstrap auth keys (no-op if keys already exist) ``` `ontoref setup` creates `.ontoref/project.ncl`, `.ontoref/config.ncl` (with logo auto-detection), `.ontology/` scaffold, `adrs/`, `reflection/modes/`, `backlog.ncl`, `qa.ncl`, git hooks, and registers the project in `~/.config/ontoref/projects.ncl`. For existing projects that predate `setup`, or to bring an already-adopted project up to the current protocol version (adds `manifest.ncl` and `connections.ncl`): ```sh ontoref --actor developer adopt_ontoref # first-time adoption ontoref run update_ontoref # bring existing project to protocol v2 ``` The `update_ontoref` mode detects missing v2 files, adds them idempotently, validates both with `nickel export`, scans ADRs for deprecated `check_hint` fields, and prints a protocol update report. The reusable `reflection/templates/update-ontology-prompt.md` guides an agent through full ontology enrichment in 8 phases. `ONTOREF_PROJECT_ROOT` is set by the consumer wrapper — one ontoref checkout serves multiple projects. ## Credential vault and registry federation ontoref ships a credential model for projects publishing or consuming OCI artifacts (domain contracts and integration modes) on a self-hosted registry like ZOT. The model is layered, declarative, and avoids ambient docker config: - **Layer 0** — master age private key (`.kage`) per actor, declared in `~/.config/ontoref/config.ncl::vault.master_key_path` (override per-project in `/.ontoref/project.ncl::sops.master_key_path`) - **Layer 1** — `access.sops.yaml` per project, multi-recipient encrypted; carries `zot_username`, `zot_password`, `vault_key`, `cosign_password` - **Layer 2** — operation credentials (RO/RW per registry entry) under `src-vault/registry/`, referenced by `manifest.ncl::registry_provides[].credential_sops*` Tenant isolation within a single vault uses sops `creation_rules` driven by `sops.recipient_groups` + `sops.recipient_rules` in `project.ncl` — different clients/agents get disjoint recipient sets per file, all in one vault. Multi-vault is explicitly out of scope ([ADR-019](adrs/adr-019-per-file-recipient-routing-tenant-isolation.ncl)). **Adoption** — copy a template from `install/resources/templates/sops/`: | Template | When to use | |---|---| | `single-team/` | One team, no tenant separation | | `multi-tenant/` | Multiple clients with isolated credentials | | `agent-first/` | AI agents (MCP) read a single read-only credential | For integration artifacts (publishing domain contracts or consuming someone's mode), templates in `install/resources/templates/integration/`: `domain-producer/`, `mode-producer/`, `mode-consumer/`. **Day-to-day**: ```sh ore secrets bootstrap # create vault for a new project (admin only) ore secrets sync # pull latest src-vault from ZOT ore secrets open # acquire OCI lock + edit access.sops.yaml ore secrets close # impact report → push → release lock ore secrets describe # full inventory: groups, rules, scopes, ops ore secrets audit # 6 ADR-017 + ADR-019 constraint checks ``` See FAQ entries in `reflection/qa.ncl` for diagrams, troubleshooting, and the 15 named errors. ADRs: [017](adrs/adr-017-registry-credential-vault-model.ncl) (vault model) and [019](adrs/adr-019-per-file-recipient-routing-tenant-isolation.ncl) (per-file recipient routing). ## Prerequisites - [Nushell](https://www.nushell.sh/) >= 0.110.0 - [Nickel](https://nickel-lang.org/) (for schema evaluation) - Rust toolchain (for building crates) - [Just](https://just.systems/) (for CI recipes) - [age](https://github.com/FiloSottile/age) + [sops](https://github.com/getsops/sops) (credential vault, ADR-017/019) - [oras](https://oras.land/) + [cosign](https://github.com/sigstore/cosign) ≥ 2 (OCI artifact federation) - [restic](https://restic.net/) or [kopia](https://kopia.io/) (vault snapshots) To build `ontoref-daemon` and `ontoref-reflection` with NATS/SurrealDB support, the stratumiops repo must be checked out at `../../../stratumiops`. Without it, build without default features: ```sh cargo build -p ontoref-daemon --no-default-features cargo build -p ontoref-ontology # always standalone ``` ## Development ```sh cargo check-all # check all targets + features cargo test-all # run full test suite just ci-lint # clippy + TOML + Nickel + Markdown just ci-docs # rustdoc broken intra-doc link check just ci-full # all CI checks nu --ide-check 50 reflection/modules/.nu # validate a Nushell module ./ontoref --actor developer # run a reflection mode ./ontoref sync diff --docs # crate //! drift against ontology nodes ./ontoref describe workspace # per-crate doc coverage + drift status ``` ### Doc authoring convention Three canonical layers — no duplication across them: | Layer | Where | Read by | |-------|-------|---------| | `///` first line | handlers, structs, types | `#[onto_api]`, `#[derive(OntologyNode)]`, MCP | | `//!` first sentence | `lib.rs` | `describe features`, mdBook crates chapter, drift check | | node `description` | `.ontology/core.ncl` | UI graph, `describe project`, CLI | `sync diff --docs --fail-on-drift` (used by pre-commit `docs-drift` hook) enforces that `//!` first sentence stays aligned with the practice node description (Jaccard ≥ 0.20 threshold). ## License MIT OR Apache-2.0