Self-Describing Protocol for Evolving Codebases
Find a file
Jesús Pérez 6721daf440
Some checks failed
Nickel Type Check / Nickel Type Checking (push) Has been cancelled
Rust CI / Security Audit (push) Has been cancelled
Rust CI / Check + Test + Lint (nightly) (push) Has been cancelled
Rust CI / Check + Test + Lint (stable) (push) Has been cancelled
feat: workflow layer model — NCL-first CI/build/distribution generator
Adds a declarative workflow system where .ontology/workflow.ncl declares
  independent layers (commit-fast, ci-standard, ci-exhaustive), each with a
  trigger and a set of providers. The generator materialises Woodpecker YAML,
  justfile recipes, and pre-commit hook stubs from that single declaration.

  Layers form a set, not a chain — no layer depends on another at the
  declaration level. Woodpecker steps express fine-grained parallelism
  internally (lint → test/docs/build).

  Schema and catalog:
  - reflection/schemas/workflow.ncl — types, contracts, provider constructors
  - reflection/defaults/workflow.ncl — standard Rust/Nushell/Nickel catalog
    (14 validations, 4 builds)

  Generator (ore workflow generate):
  - PreCommit: prints hook fragments for manual merge into .pre-commit-config.yaml
  - Woodpecker: writes per-layer YAML with image selection, tool installs,
    depends_on chains, and CI-aware command translation
    (nu --ide-check → find+xargs; markdownlint-cli2 → node:lts image)
  - Justfile: single justfiles/workflow.just aggregated from all Justfile layers
    (hoisted out of per-provider loop — was writing N times, now once)

  Dispatcher: ore workflow validate|list|generate|diff + ore wf v|l|gen|diff

  Cargo/nextest profiles:
  - .cargo/config.toml: [profile.ci-test] (debug=0, no-incremental) and
    [profile.ci] (line-tables-only) — shared by test + docs hooks to reuse
    .rlib artifacts; CI retains actionable backtraces
  - .config/nextest.toml: ci-test (fail-fast, no retries) and ci (all failures,
    one retry)

  Pre-commit: nextest run with --profile ci-test --cargo-profile ci-test;
  docs-links hook shares same profile to reuse artifacts

  Self-application: ontoref declares its own workflow in .ontology/workflow.ncl
  and the generated .woodpecker/ files are the authoritative CI definition

  Migration 0014: checks workflow.ncl + both cargo and nextest profiles present

  Adoption: reflection/templates/ontology/workflow.ncl stub + install_workflow
  confirm in adopt_ontoref form + step 5 in adoption script template
2026-04-08 14:20:38 +01:00
.cargo feat: workflow layer model — NCL-first CI/build/distribution generator 2026-04-08 14:20:38 +01:00
.coder feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
.config feat: workflow layer model — NCL-first CI/build/distribution generator 2026-04-08 14:20:38 +01:00
.github/workflows chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
.ontology feat: workflow layer model — NCL-first CI/build/distribution generator 2026-04-08 14:20:38 +01:00
.ontoref feat: config surface, NCL contracts, override-layer mutation, on+re update 2026-03-26 20:20:22 +00:00
.vale chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
.woodpecker feat: workflow layer model — NCL-first CI/build/distribution generator 2026-04-08 14:20:38 +01:00
adrs feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
assets feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
crates chore: fix /Users/Akasha/Development/ontoref withoud .ontoref 2026-04-08 11:33:35 +01:00
domains feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
install feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
justfiles feat: workflow layer model — NCL-first CI/build/distribution generator 2026-04-08 14:20:38 +01:00
nats feat: unified auth model, project onboarding, install pipeline, config management 2026-03-13 20:56:31 +00:00
ontology feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
reflection feat: workflow layer model — NCL-first CI/build/distribution generator 2026-04-08 14:20:38 +01:00
templates --- 2026-03-23 00:58:27 +01:00
.clippy.toml chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
.gitignore feat: browser-style panel nav, repo file routing, migration 0007 2026-03-29 08:32:50 +01:00
.markdownlint-cli2.jsonc feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
.pre-commit-config.yaml feat: workflow layer model — NCL-first CI/build/distribution generator 2026-04-08 14:20:38 +01:00
.rustfmt.toml chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
.shellcheckrc chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
.taplo.toml chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
.vale.ini chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
.yamllint-ci.yml chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
card.ncl feat: browser-style panel nav, repo file routing, migration 0007 2026-03-29 08:32:50 +01:00
Cargo.lock feat: config surface, NCL contracts, override-layer mutation, on+re update 2026-03-26 20:20:22 +00:00
Cargo.toml --- 2026-03-23 00:58:27 +01:00
CHANGELOG.md feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
CODE_OF_CONDUCT.md chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
CONTRIBUTING.md chore: add ci and dev settings 2026-03-13 00:15:49 +00:00
deny.toml --- 2026-03-23 00:58:27 +01:00
justfile feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
ontoref feat: personal/career schemas, content modes, search bookmarks, Nu 0.111 compat (ADR-006), commit optimize 2026-03-16 01:48:17 +00:00
README.md feat: domain extension system, VCS abstraction, personal/provisioning domains, web subpages 2026-04-07 23:08:29 +01:00
SECURITY.md chore: add ci and dev settings 2026-03-13 00:15:49 +00:00

ontoref


ontoref is a protocol specification and tooling layer for structured self-knowledge in software projects. It provides schemas, Nushell automation, and Rust crates so that projects can describe what they are, record architectural decisions, track operational state, and execute formalized procedures — all as typed, queryable artifacts.


Axioms

Axiom Meaning
Protocol, Not Runtime Never a runtime dependency. Projects adopt the protocol; ontoref provides the schemas and modules to do so.
Self-Describing Consumes its own protocol: .ontology/, adrs/, reflection/ in this repo ARE ontoref running against itself.
No Enforcement ontoref defines contracts. There is no enforcement mechanism. Coherence is voluntary and emerges from justified adoption.
DAG-Formalized Knowledge Concepts, tensions, decisions, state — encoded as DAGs. Enables transversal queries and impact analysis.

Layers

ontology/          Protocol specification — Nickel schemas for nodes, edges, ADRs, state, gates
adrs/              Architecture Decision Records — typed NCL with constraints and ontology checks
reflection/        Operational tooling — Nushell modules, DAG modes, forms, and schemas
crates/            Rust implementation — typed struct loaders and mode executors
.ontology/         Self-description — ontoref's own ontology, state, gate, and manifest

Crates

Crate Purpose
ontoref-ontology .ontology/ NCL → typed Rust structs: Node, Edge, Dimension, Gate, Membrane. Node carries artifact_paths and adrs (Vec<String>, both serde(default)). Graph traversal, invariant queries. Zero deps.
ontoref-reflection NCL DAG contract executor with guards (pre-flight Block/Warn checks) and convergence loops (RetryFailed/RetryAll). ADR lifecycle, step dep resolution, config seal. stratum-graph + stratum-state required.
ontoref-daemon HTTP UI (11 pages), actor registry, notification barrier, MCP (33 tools), search engine, search bookmarks, SurrealDB, NCL export cache, per-file ontology versioning, annotated API catalog, Agent Task Composer.
ontoref-derive Proc-macro crate. #[onto_api(...)] annotates HTTP handlers — description is optional when a /// doc comment exists (first line used as fallback). #[derive(OntologyNode)] + #[onto(id, name, paths, description, adrs)] auto-registers nodes via inventory::submit! at link time, merged into Core by merge_contributors(). #[derive(ConfigFields)] + #[config_section(id, ncl_file)] registers config struct fields. All three aggregate via inventory::collect!.

ontoref-daemon caches nickel export results (keyed by path + mtime), reducing full sync scans from ~2m42s to <30s. The daemon is always optional — every module falls back to direct subprocess when unavailable.

Daemon Capabilities

Unified Auth Model — all surfaces (CLI, UI, MCP) exchange a key for a UUID v4 session token via POST /sessions. Token lifetime: 30 days, O(1) lookup. Project keys carry role (admin|viewer) and label for audit trail. Daemon-level admin via ONTOREF_ADMIN_TOKEN_FILE. GET /sessions and DELETE /sessions/{id} for session visibility and revocation. Key rotation invalidates all sessions for the rotated project. CLI injects ONTOREF_TOKEN as Bearer automatically.

Q&A Knowledge Store — accumulated Q&A entries persist to reflection/qa.ncl (typed NCL, git-versioned). Not localStorage. Any actor — developer, agent, CI — reads the same store.

MCP Server — 29 tools over stdio and streamable-HTTP. Categories: discovery, retrieval, project state, ontology, backlog, validation, Q&A, bookmarks, API surface. Representative subset:

Tool What it does
ontoref_guides Full project context on cold start: axioms, practices, gate, actor policy
ontoref_api_catalog Annotated HTTP surface — all routes with auth, actors, params, tags
ontoref_file_versions Per-file reload counters — detect which ontology files changed
ontoref_validate_adrs Run typed ADR constraint checks; returns pass/fail per constraint
ontoref_validate Full project validation: ADRs, content assets, connections, gate consistency, manifest coverage
ontoref_impact BFS impact graph from a node, optionally across project connections
ontoref_qa_list List Q&A entries with optional filter
ontoref_qa_add Append a new Q&A entry to reflection/qa.ncl
ontoref_action_list List quick actions from .ontoref/config.ncl
ontoref_action_add Create a reflection mode + register as a quick action

Search Bookmarks — search results persist to reflection/search_bookmarks.ncl (typed NCL, BookmarkEntry schema). Same atomic-write pattern as Q&A. IDs are sequential sb-NNN. Concurrency-safe via NclWriteLock. Add and remove from the daemon search UI.

Personal Ontology Schemasontology/schemas/career.ncl, personal.ncl, project-card.ncl provide typed contract layers for career and content artifacts (Skills, WorkExperience, Talks, Content lifecycle, Opportunities, PublicationCards). All types carry linked_nodes referencing core ontology node IDs — bridging career artifacts into the DAG. Five content/career reflection modes (draft-application, draft-email, generate-article, update-cv, write-cfp) query these schemas to ground output in declared project artifacts rather than free-form prose.

API Catalog — every HTTP handler carries #[onto_api(method, path, auth, actors, params, tags)]. description is sourced from the first /// doc line above the handler — no duplication with doc comments. At link time inventory::submit! registers each route. GET /api/catalog returns the full annotated surface as JSON. The /ui/{slug}/api page renders it with client-side filtering (method, auth, path). describe api [--actor] [--tag] [--fmt] renders the catalog in the CLI. ontoref_api_catalog exposes it to MCP agents.

Semantic Diffdescribe diff [--file <ncl>] [--fmt json|text] computes a node- and edge-level diff of .ontology/ files against the last git commit. Reports added/removed/changed nodes by id and edges by from→to[kind] key — not a text diff.

Per-File Versioning — each ontology file tracked in ProjectContext.file_versions: DashMap<PathBuf, u64>. Counter increments on every watcher-triggered reload. GET /projects/{slug}/ontology/versions and ontoref_file_versions MCP tool expose the map. Dashboard surfaces the counters.

ADRNode Linkage — nodes declare which ADRs validate them via adrs: Array String. describe surfaces a Validated by section per node (CLI and --fmt md). The graph UI renders each ADR as a clickable link that opens the full ADR content via GET /api/adr/{id}.

Browser-Style Panel Navigation — graph, search, and api_catalog UI pages carry a back/forward history stack (cursor-into-array model). Clicking nodes, ADRs, or search results pushes to history; clicking artifacts opens the source file in the configured repository or docs. card.repo in card.ncl resolves to {repo}/src/branch/main/{path} (Gitea format). For .rs files, card.docs redirects to the cargo docs URL instead. insert_brand_ctx injects both as card_repo/card_docs into every Tera template.

Passive Drift Observation — background file watcher that detects divergence between Yang code artifacts and Yin ontology. Watches crates/, .ontology/, adrs/, reflection/modes/. After a 15s debounce runs sync scan + sync diff; emits an ontology_drift notification when MISSING/STALE/DRIFT/BROKEN items are found. Never applies changes — apply is always deliberate.

Quick Actions — runnable shortcuts over reflection modes, configured as quick_actions in .ontoref/config.ncl. Accessible from HTTP (/actions), CLI (ontoref), and MCP (ontoref_action_list/add).

Config Surface — per-project config introspection, coherence verification, and documented mutation. Rust structs annotated with #[derive(ConfigFields)] + #[config_section(id, ncl_file)] register their field names at link time via inventory::submit!(ConfigFieldsEntry{...}). The daemon queries inventory::iter::<ConfigFieldsEntry>() at startup to build a zero-maintenance registry of which Rust fields each struct reads from each NCL section. Multi-consumer coherence (GET /projects/{slug}/config/coherence) compares the inventory registry against NCL export keys, Nu script accessor patterns, and CI fields declared in manifest.ncl — any NCL field claimed by no consumer is flagged unclaimed. GET /projects/{slug}/config/quickref generates living documentation (rationales, override history, coherence status) on demand.

Config mutation never modifies source NCL files. PUT /projects/{slug}/config/{section} writes a {section}.overrides.ncl file with only the changed fields plus a _overrides_meta audit record (actor, reason, timestamp, previous value), then appends a single idempotent import line to the entry-point NCL using the & merge operator. nickel export validates the merged result against the section's declared contract before committing; contract violations revert the override file and return the nickel error verbatim. NCL contracts (std.contract.from_validator) are the single validation gate — Rust structs are contract-trusted readers with #[serde(default)].

Ontoref demonstrates the pattern on itself: .ontoref/contracts.ncl applies LogConfig and DaemonConfig contracts to .ontoref/config.ncl. (ADR-008)

Protocol Migration System — protocol upgrades for consumer projects expressed as ordered NCL files in reflection/migrations/NNN-slug.ncl. Each migration declares a typed check (FileExists | Grep | NuCmd) whose result IS the applied state — no state file, fully idempotent. migrate list shows all migrations with applied/pending status; migrate pending lists only what is missing; migrate show <id> renders runtime-interpolated instructions (project_root and project_name auto-detected). NuCmd checks are valid Nushell (no bash &&, $env.VAR not $VAR). Grep checks targeting ADR files scope to adr-[0-9][0-9][0-9]-*.ncl to exclude schema/template infrastructure files. 12 migrations shipped; 0012-rust-doc-authoring-pattern adds the /// → //! → node description three-layer doc convention and optional pre-commit hooks (docs-links, docs-drift) to consumer CLAUDE.md. (ADR-010)

Manifest Self-Interrogationmanifest_type gains three typed arrays that answer self-knowledge queries agents and operators need on cold start: capabilities[] (what the project does, why it was built, how it works — with explicit nodes[] and adrs[] cross-references into the DAG), requirements[] (prerequisites classified by environment: 'Production | 'Development | 'Both and kind: 'Tool | 'Service | 'EnvVar | 'Infrastructure), critical_deps[] (external dependencies with required failure_impact and optional mitigation). describe requirements surfaces these; describe guides and ontoref_guides include all three arrays in their output. (ADR-009)

Domain Extension System — CLI commands conditional on repo_kind. The ontoref bash wrapper resolves the first argument against $ONTOREF_ROOT/domains/{id}/repo_kinds.txt before delegating to the Nu dispatcher; if the project's repo_kind matches, the domain's commands.nu is dispatched directly. Each domain ships domain.ncl (typed contract: commands, pages, short_alias, repo_kinds), commands.nu (Nu entry point), and repo_kinds.txt (grep-readable, sub-1ms dispatch). Two domains shipped: personal (PersonalOntology — cfp, career, content, opportunities) and provisioning (DevWorkspace/Mixed — state, connections, gates, card, backlog). Short aliases (personal, prov) work as both ore prov <cmd> and standalone prov <cmd>. ore help and describe capabilities surface the active domain automatically. New domains require only three files — no changes to the Nu dispatcher. (ADR-012)

VCS Abstraction Layerreflection/modules/vcs.nu exposes a uniform API over jj and git: detect, show-committed, restore-file, remote-url, current-branch, uncommitted-files, commit-count. All ontoref modules consume vcs.nu — never hardcoded ^git. Detection is filesystem-based (.jj/ vs .git/), no config required. jj is opt-in: all operations degrade to git when .jj/ is absent. reflection/bin/jjw.nu wraps jj workspaces, ontoref runs, and optional Radicle patch submission into a single jjw agent create|step|publish|merge|discard lifecycle for agent-driven development. jjw-ncl-merge.nu is a jj merge tool for .ontology/ NCL conflicts, registered manually in ~/.config/jj/config.toml. jj and Radicle are not protocol requirements — consumer projects use plain git without any configuration change.

Install

just install-daemon        # build + install binary, bootstrapper, CLI, UI assets, config skeleton
ontoref config-edit        # browser form → ~/.config/ontoref/config.ncl
ontoref-daemon-boot        # NCL pipe bootstrap: nickel export config.ncl | daemon --config-stdin
ontoref-daemon-boot --dry-run   # preview composed JSON without starting

Installed layout (~/.local/bin/):

Binary Role
ontoref Global CLI dispatcher — all reflection modes, ADR lifecycle, daemon control
ontoref-daemon Bootstrapper (public entrypoint) — validates config via Nickel, pipes JSON to binary
ontoref-daemon.bin Compiled Rust binary — never called directly

Global config at ~/.config/ontoref/config.ncl (type-checked Nickel). Global NATS stream topology at ~/.config/ontoref/streams.json. Project-local topology override via nats/streams.json + nats_events.streams_config in .ontoref/config.ncl.

Onboarding a project

cd /path/to/my-project
ontoref setup                          # idempotent; kind: Service by default
ontoref setup --kind Library           # Library | Service | DevWorkspace | PublishedCrate | AgentResource | Mixed | PersonalOntology
ontoref setup --parent /path/to/fw     # implementation child: adds framework layer + browse mode
ontoref setup --gen-keys ["admin:dev" "viewer:ci"]   # bootstrap auth keys (no-op if keys already exist)

ontoref setup creates .ontoref/project.ncl, .ontoref/config.ncl (with logo auto-detection), .ontology/ scaffold, adrs/, reflection/modes/, backlog.ncl, qa.ncl, git hooks, and registers the project in ~/.config/ontoref/projects.ncl.

For existing projects that predate setup, or to bring an already-adopted project up to the current protocol version (adds manifest.ncl and connections.ncl):

ontoref --actor developer adopt_ontoref    # first-time adoption
ontoref run update_ontoref                 # bring existing project to protocol v2

The update_ontoref mode detects missing v2 files, adds them idempotently, validates both with nickel export, scans ADRs for deprecated check_hint fields, and prints a protocol update report. The reusable reflection/templates/update-ontology-prompt.md guides an agent through full ontology enrichment in 8 phases.

ONTOREF_PROJECT_ROOT is set by the consumer wrapper — one ontoref checkout serves multiple projects.

Prerequisites

  • Nushell >= 0.110.0
  • Nickel (for schema evaluation)
  • Rust toolchain (for building crates)
  • Just (for CI recipes)

To build ontoref-daemon and ontoref-reflection with NATS/SurrealDB support, the stratumiops repo must be checked out at ../../../stratumiops. Without it, build without default features:

cargo build -p ontoref-daemon --no-default-features
cargo build -p ontoref-ontology   # always standalone

Development

cargo check-all                              # check all targets + features
cargo test-all                               # run full test suite
just ci-lint                                 # clippy + TOML + Nickel + Markdown
just ci-docs                                 # rustdoc broken intra-doc link check
just ci-full                                 # all CI checks
nu --ide-check 50 reflection/modules/<file>.nu  # validate a Nushell module
./ontoref --actor developer <mode>           # run a reflection mode
./ontoref sync diff --docs                   # crate //! drift against ontology nodes
./ontoref describe workspace                 # per-crate doc coverage + drift status

Doc authoring convention

Three canonical layers — no duplication across them:

Layer Where Read by
/// first line handlers, structs, types #[onto_api], #[derive(OntologyNode)], MCP
//! first sentence lib.rs describe features, mdBook crates chapter, drift check
node description .ontology/core.ncl UI graph, describe project, CLI

sync diff --docs --fail-on-drift (used by pre-commit docs-drift hook) enforces that //! first sentence stays aligned with the practice node description (Jaccard ≥ 0.20 threshold).

License

MIT OR Apache-2.0