ontoref/README.md

ontoref is a protocol specification and tooling layer for structured self-knowledge in software projects. It provides schemas, Nushell automation, and Rust crates so that projects can describe what they are, record architectural decisions, track operational state, and execute formalized procedures — all as typed, queryable artifacts.

---

Axioms

Axiom	Meaning
Protocol, Not Runtime	Never a runtime dependency. Projects adopt the protocol; ontoref provides the schemas and modules to do so.
Self-Describing	Consumes its own protocol: .ontology/, adrs/, reflection/ in this repo ARE ontoref running against itself.
No Enforcement	ontoref defines contracts. There is no enforcement mechanism. Coherence is voluntary and emerges from justified adoption.
DAG-Formalized Knowledge	Concepts, tensions, decisions, state — encoded as DAGs. Enables transversal queries and impact analysis.

Layers

ontology/          Protocol specification — Nickel schemas for nodes, edges, ADRs, state, gates
adrs/              Architecture Decision Records — typed NCL with constraints and ontology checks
reflection/        Operational tooling — Nushell modules, DAG modes, forms, and schemas
crates/            Rust implementation — typed struct loaders and mode executors
.ontology/         Self-description — ontoref's own ontology, state, gate, and manifest

Crates

Crate	Purpose
ontoref-ontology	.ontology/ NCL → typed Rust structs: Node, Edge, Dimension, Gate, Membrane. Node carries artifact_paths and adrs (Vec<String>, both serde(default)). Graph traversal, invariant queries. Zero deps.
ontoref-reflection	NCL DAG contract executor with guards (pre-flight Block/Warn checks) and convergence loops (RetryFailed/RetryAll). ADR lifecycle, step dep resolution, config seal. stratum-graph + stratum-state required.
ontoref-daemon	HTTP UI (11 pages), actor registry, notification barrier, MCP (34 tools), search engine, search bookmarks, SurrealDB, NCL export cache, per-file ontology versioning, annotated API catalog, Agent Task Composer.
ontoref-derive	Proc-macro crate. #[onto_api(...)] annotates HTTP handlers — description is optional when a /// doc comment exists (first line used as fallback). #[onto_mcp_tool(name, description, input_schema)] registers MCP tool unit-structs in the catalog at link time via inventory::submit!(McpToolEntry{...}); the annotated item is emitted unchanged and ToolBase/AsyncTool impls remain on the struct. #[derive(OntologyNode)] + #[onto(id, name, paths, description, adrs)] auto-registers nodes via inventory::submit! at link time, merged into Core by merge_contributors(). #[derive(ConfigFields)] + #[config_section(id, ncl_file)] registers config struct fields. All four aggregate via inventory::collect!.

ontoref-daemon caches nickel export results (keyed by path + mtime), reducing full sync scans from ~2m42s to <30s. The daemon is always optional — every module falls back to direct subprocess when unavailable.

Daemon Capabilities

Unified Auth Model — all surfaces (CLI, UI, MCP) exchange a key for a UUID v4 session token via POST /sessions. Token lifetime: 30 days, O(1) lookup. Project keys carry role (admin|viewer) and label for audit trail. Daemon-level admin via ONTOREF_ADMIN_TOKEN_FILE. GET /sessions and DELETE /sessions/{id} for session visibility and revocation. Key rotation invalidates all sessions for the rotated project. CLI injects ONTOREF_TOKEN as Bearer automatically.

Q&A Knowledge Store — accumulated Q&A entries persist to reflection/qa.ncl (typed NCL, git-versioned). Not localStorage. Any actor — developer, agent, CI — reads the same store.

MCP Server — 34 tools over stdio and streamable-HTTP, all registered at link time via #[onto_mcp_tool] (no manual catalog wiring). Categories: discovery, retrieval, project state, ontology, backlog, validation, Q&A, bookmarks, API surface. Representative subset:

Tool	What it does
ontoref_guides	Full project context on cold start: axioms, practices, gate, actor policy
ontoref_api_catalog	Annotated HTTP surface — all routes with auth, actors, params, tags
ontoref_file_versions	Per-file reload counters — detect which ontology files changed
ontoref_validate_adrs	Run typed ADR constraint checks; returns pass/fail per constraint
ontoref_validate	Full project validation: ADRs, content assets, connections, gate consistency, manifest coverage
ontoref_impact	BFS impact graph from a node, optionally across project connections
ontoref_qa_list	List Q&A entries with optional filter
ontoref_qa_add	Append a new Q&A entry to reflection/qa.ncl
ontoref_action_list	List quick actions from .ontoref/config.ncl
ontoref_action_add	Create a reflection mode + register as a quick action

Search Bookmarks — search results persist to reflection/search_bookmarks.ncl (typed NCL, BookmarkEntry schema). Same atomic-write pattern as Q&A. IDs are sequential sb-NNN. Concurrency-safe via NclWriteLock. Add and remove from the daemon search UI.

Personal Ontology Schemas — ontology/schemas/career.ncl, personal.ncl, project-card.ncl provide typed contract layers for career and content artifacts (Skills, WorkExperience, Talks, Content lifecycle, Opportunities, PublicationCards). All types carry linked_nodes referencing core ontology node IDs — bridging career artifacts into the DAG. Five content/career reflection modes (draft-application, draft-email, generate-article, update-cv, write-cfp) query these schemas to ground output in declared project artifacts rather than free-form prose.

API Catalog — every HTTP handler carries #[onto_api(method, path, auth, actors, params, tags)]. description is sourced from the first /// doc line above the handler — no duplication with doc comments. At link time inventory::submit! registers each route. GET /api/catalog returns the full annotated surface as JSON. The /ui/{slug}/api page renders it with client-side filtering (method, auth, path). describe api [--actor] [--tag] [--fmt] renders the catalog in the CLI. ontoref_api_catalog exposes it to MCP agents.

Semantic Diff — describe diff [--file <ncl>] [--fmt json|text] computes a node- and edge-level diff of .ontology/ files against the last git commit. Reports added/removed/changed nodes by id and edges by from→to[kind] key — not a text diff.

Per-File Versioning — each ontology file tracked in ProjectContext.file_versions: DashMap<PathBuf, u64>. Counter increments on every watcher-triggered reload. GET /projects/{slug}/ontology/versions and ontoref_file_versions MCP tool expose the map. Dashboard surfaces the counters.

ADR–Node Linkage — nodes declare which ADRs validate them via adrs: Array String. describe surfaces a Validated by section per node (CLI and --fmt md). The graph UI renders each ADR as a clickable link that opens the full ADR content via GET /api/adr/{id}.

Browser-Style Panel Navigation — graph, search, and api_catalog UI pages carry a back/forward history stack (cursor-into-array model). Clicking nodes, ADRs, or search results pushes to history; clicking artifacts opens the source file in the configured repository or docs. card.repo in card.ncl resolves to {repo}/src/branch/main/{path} (Gitea format). For .rs files, card.docs redirects to the cargo docs URL instead. insert_brand_ctx injects both as card_repo/card_docs into every Tera template.

Passive Drift Observation — background file watcher that detects divergence between Yang code artifacts and Yin ontology. Watches crates/, .ontology/, adrs/, reflection/modes/. After a 15s debounce runs sync scan + sync diff; emits an ontology_drift notification when MISSING/STALE/DRIFT/BROKEN items are found. Never applies changes — apply is always deliberate.

Quick Actions — runnable shortcuts over reflection modes, configured as quick_actions in .ontoref/config.ncl. Accessible from HTTP (/actions), CLI (ontoref), and MCP (ontoref_action_list/add).

Config Surface — per-project config introspection, coherence verification, and documented mutation. Rust structs annotated with #[derive(ConfigFields)] + #[config_section(id, ncl_file)] register their field names at link time via inventory::submit!(ConfigFieldsEntry{...}). The daemon queries inventory::iter::<ConfigFieldsEntry>() at startup to build a zero-maintenance registry of which Rust fields each struct reads from each NCL section. Multi-consumer coherence (GET /projects/{slug}/config/coherence) compares the inventory registry against NCL export keys, Nu script accessor patterns, and CI fields declared in manifest.ncl — any NCL field claimed by no consumer is flagged unclaimed. GET /projects/{slug}/config/quickref generates living documentation (rationales, override history, coherence status) on demand.

Config mutation never modifies source NCL files. PUT /projects/{slug}/config/{section} writes a {section}.overrides.ncl file with only the changed fields plus a _overrides_meta audit record (actor, reason, timestamp, previous value), then appends a single idempotent import line to the entry-point NCL using the & merge operator. nickel export validates the merged result against the section's declared contract before committing; contract violations revert the override file and return the nickel error verbatim. NCL contracts (std.contract.from_validator) are the single validation gate — Rust structs are contract-trusted readers with #[serde(default)].

Ontoref demonstrates the pattern on itself: .ontoref/contracts.ncl applies LogConfig and DaemonConfig contracts to .ontoref/config.ncl. (ADR-008)

Protocol Migration System — protocol upgrades for consumer projects expressed as ordered NCL files in reflection/migrations/NNN-slug.ncl. Each migration declares a typed check (FileExists | Grep | NuCmd) whose result IS the applied state — no state file, fully idempotent. migrate list shows all migrations with applied/pending status; migrate pending lists only what is missing; migrate show <id> renders runtime-interpolated instructions (project_root and project_name auto-detected). NuCmd checks are valid Nushell (no bash &&, $env.VAR not $VAR). Grep checks targeting ADR files scope to adr-[0-9][0-9][0-9]-*.ncl to exclude schema/template infrastructure files. 12 migrations shipped; 0012-rust-doc-authoring-pattern adds the /// → //! → node description three-layer doc convention and optional pre-commit hooks (docs-links, docs-drift) to consumer CLAUDE.md. (ADR-010)

Manifest Self-Interrogation — manifest_type gains three typed arrays that answer self-knowledge queries agents and operators need on cold start: capabilities[] (what the project does, why it was built, how it works — with explicit nodes[] and adrs[] cross-references into the DAG), requirements[] (prerequisites classified by environment: 'Production | 'Development | 'Both and kind: 'Tool | 'Service | 'EnvVar | 'Infrastructure), critical_deps[] (external dependencies with required failure_impact and optional mitigation). describe requirements surfaces these; describe guides and ontoref_guides include all three arrays in their output. (ADR-009)

Domain Extension System — CLI commands conditional on repo_kind. The ontoref bash wrapper resolves the first argument against $ONTOREF_ROOT/domains/{id}/repo_kinds.txt before delegating to the Nu dispatcher; if the project's repo_kind matches, the domain's commands.nu is dispatched directly. Each domain ships domain.ncl (typed contract: commands, pages, short_alias, repo_kinds), commands.nu (Nu entry point), and repo_kinds.txt (grep-readable, sub-1ms dispatch). Two domains shipped: personal (PersonalOntology — cfp, career, content, opportunities) and provisioning (DevWorkspace/Mixed — state, connections, gates, card, backlog). Short aliases (personal, prov) work as both ore prov <cmd> and standalone prov <cmd>. ore help and describe capabilities surface the active domain automatically. New domains require only three files — no changes to the Nu dispatcher. (ADR-012)

Mode Hierarchy Validation — validate modes [--check] reads reflection/defaults/workflow.ncl::level_hierarchy and checks every .ncl mode file for: level declared, strategy declared, delegate chain coherent, compose extends references valid. mode resolve <id> prints which hierarchy level handles the given mode and why. validate modes --self-test generates synthetic fixtures in a temp dir for fast CI smoke-testing of the validator itself. (ADR-018)

Project Picker — vault and registry badges — each project card surfaces OCI state inline: registry participant badge (⟳ <participant>) when registry_provides is declared; vault badge (⛁ <vault_id> · N) coloured green (declarative) or amber (legacy) when sops.enabled is true. Expanded project panel shows a collapsible Registry section with namespace, endpoint, and push/pull capability. The manage page adds Runtime Services toggles — MCP and GraphQL can be switched without a daemon restart via HTMX POST /ui/manage/services/{service}/toggle.

VCS Abstraction Layer — reflection/modules/vcs.nu exposes a uniform API over jj and git: detect, show-committed, restore-file, remote-url, current-branch, uncommitted-files, commit-count. All ontoref modules consume vcs.nu — never hardcoded ^git. Detection is filesystem-based (.jj/ vs .git/), no config required. jj is opt-in: all operations degrade to git when .jj/ is absent. reflection/bin/jjw.nu wraps jj workspaces, ontoref runs, and optional Radicle patch submission into a single jjw agent create|step|publish|merge|discard lifecycle for agent-driven development. jjw-ncl-merge.nu is a jj merge tool for .ontology/ NCL conflicts, registered manually in ~/.config/jj/config.toml. jj and Radicle are not protocol requirements — consumer projects use plain git without any configuration change.

Install

just install-daemon        # build + install binary, bootstrapper, CLI, UI assets, config skeleton
ontoref config-edit        # browser form → ~/.config/ontoref/config.ncl
ontoref-daemon-boot        # NCL pipe bootstrap: nickel export config.ncl | daemon --config-stdin
ontoref-daemon-boot --dry-run   # preview composed JSON without starting

Installed layout (~/.local/bin/):

Binary	Role
ontoref	Global CLI dispatcher — all reflection modes, ADR lifecycle, daemon control
ontoref-daemon	Bootstrapper (public entrypoint) — validates config via Nickel, pipes JSON to binary
ontoref-daemon.bin	Compiled Rust binary — never called directly

Global config at ~/.config/ontoref/config.ncl (type-checked Nickel). Global NATS stream topology at ~/.config/ontoref/streams.json. Project-local topology override via nats/streams.json + nats_events.streams_config in .ontoref/config.ncl.

Onboarding a project

cd /path/to/my-project
ontoref setup                          # idempotent; kind: Service by default
ontoref setup --kind Library           # Library | Service | DevWorkspace | PublishedCrate | AgentResource | Mixed | PersonalOntology
ontoref setup --parent /path/to/fw     # implementation child: adds framework layer + browse mode
ontoref setup --gen-keys ["admin:dev" "viewer:ci"]   # bootstrap auth keys (no-op if keys already exist)

ontoref setup creates .ontoref/project.ncl, .ontoref/config.ncl (with logo auto-detection), .ontology/ scaffold, adrs/, reflection/modes/, backlog.ncl, qa.ncl, git hooks, and registers the project in ~/.config/ontoref/projects.ncl.

For existing projects that predate setup, or to bring an already-adopted project up to the current protocol version (adds manifest.ncl and connections.ncl):

ontoref --actor developer adopt_ontoref    # first-time adoption
ontoref run update_ontoref                 # bring existing project to protocol v2

The update_ontoref mode detects missing v2 files, adds them idempotently, validates both with nickel export, scans ADRs for deprecated check_hint fields, and prints a protocol update report. The reusable reflection/templates/update-ontology-prompt.md guides an agent through full ontology enrichment in 8 phases.

ONTOREF_PROJECT_ROOT is set by the consumer wrapper — one ontoref checkout serves multiple projects.

Credential vault and registry federation

ontoref ships a credential model for projects publishing or consuming OCI artifacts (domain contracts and integration modes) on a self-hosted registry like ZOT. The model is layered, declarative, and avoids ambient docker config:

• Layer 0 — master age private key (.kage) per actor, declared in ~/.config/ontoref/config.ncl::vault.master_key_path (override per-project in <project>/.ontoref/project.ncl::sops.master_key_path)
• Layer 1 — access.sops.yaml per project, multi-recipient encrypted; carries zot_username, zot_password, vault_key, cosign_password
• Layer 2 — operation credentials (RO/RW per registry entry) under src-vault/registry/, referenced by manifest.ncl::registry_provides[].credential_sops*

Tenant isolation within a single vault uses sops creation_rules driven by sops.recipient_groups + sops.recipient_rules in project.ncl — different clients/agents get disjoint recipient sets per file, all in one vault. Multi-vault is explicitly out of scope (ADR-019).

Adoption — copy a template from install/resources/templates/sops/:

Template	When to use
single-team/	One team, no tenant separation
multi-tenant/	Multiple clients with isolated credentials
agent-first/	AI agents (MCP) read a single read-only credential

For integration artifacts (publishing domain contracts or consuming someone's mode), templates in install/resources/templates/integration/: domain-producer/, mode-producer/, mode-consumer/.

Day-to-day:

ore secrets bootstrap         # create vault for a new project (admin only)
ore secrets sync              # pull latest src-vault from ZOT
ore secrets open              # acquire OCI lock + edit access.sops.yaml
ore secrets close             # impact report → push → release lock
ore secrets describe          # full inventory: groups, rules, scopes, ops
ore secrets audit             # 6 ADR-017 + ADR-019 constraint checks

See FAQ entries in reflection/qa.ncl for diagrams, troubleshooting, and the 15 named errors. ADRs: 017 (vault model) and 019 (per-file recipient routing).

Prerequisites

• Nushell >= 0.110.0
• Nickel (for schema evaluation)
• Rust toolchain (for building crates)
• Just (for CI recipes)
• age + sops (credential vault, ADR-017/019)
• oras + cosign ≥ 2 (OCI artifact federation)
• restic or kopia (vault snapshots)

To build ontoref-daemon and ontoref-reflection with NATS/SurrealDB support, the stratumiops repo must be checked out at ../../../stratumiops. Without it, build without default features:

cargo build -p ontoref-daemon --no-default-features
cargo build -p ontoref-ontology   # always standalone

Development

cargo check-all                              # check all targets + features
cargo test-all                               # run full test suite
just ci-lint                                 # clippy + TOML + Nickel + Markdown
just ci-docs                                 # rustdoc broken intra-doc link check
just ci-full                                 # all CI checks
nu --ide-check 50 reflection/modules/<file>.nu  # validate a Nushell module
./ontoref --actor developer <mode>           # run a reflection mode
./ontoref sync diff --docs                   # crate //! drift against ontology nodes
./ontoref describe workspace                 # per-crate doc coverage + drift status

Doc authoring convention

Three canonical layers — no duplication across them:

Layer	Where	Read by
/// first line	handlers, structs, types	#[onto_api], #[derive(OntologyNode)], MCP
//! first sentence	lib.rs	describe features, mdBook crates chapter, drift check
node description	.ontology/core.ncl	UI graph, describe project, CLI

sync diff --docs --fail-on-drift (used by pre-commit docs-drift hook) enforces that //! first sentence stays aligned with the practice node description (Jaccard ≥ 0.20 threshold).

License

MIT OR Apache-2.0