085607130a
feat: API catalog surface, protocol v2 tooling, MCP expansion, on+re update
## Summary
Session 2026-03-23. Closes the loop between handler code and discoverability
across all three surfaces (browser, CLI, MCP agent) via compile-time inventory
registration. Adds protocol v2 update tooling, extends MCP from 21 to 29 tools,
and brings the self-description up to date.
## API Catalog Surface (#[onto_api] proc-macro)
- crates/ontoref-derive: new proc-macro crate; `#[onto_api(method, path,
description, auth, actors, params, tags)]` emits `inventory::submit!(ApiRouteEntry{...})`
at link time
- crates/ontoref-daemon/src/api_catalog.rs: `catalog()` — pure fn over
`inventory::iter::<ApiRouteEntry>()`, zero runtime allocation
- GET /api/catalog: returns full annotated HTTP surface as JSON
- templates/pages/api_catalog.html: new page with client-side filtering by
method, auth, path/description; detail panel per route (params table,
feature flag); linked from dashboard card and nav
- UI nav: "API" link (</> icon) added to mobile dropdown and desktop bar
- inventory = "0.3" added to workspace.dependencies (MIT, zero transitive deps)
## Protocol Update Mode
- reflection/modes/update_ontoref.ncl: 9-step DAG (5 detect parallel, 2 update
idempotent, 2 validate, 1 report) — brings any project from protocol v1 to v2
by adding manifest.ncl and connections.ncl if absent, scanning ADRs for
deprecated check_hint, validating with nickel export
- reflection/templates/update-ontology-prompt.md: 8-phase reusable prompt for
agent-driven ontology enrichment (infrastructure → audit → core.ncl →
state.ncl → manifest.ncl → connections.ncl → ADR migration → validation)
## CLI — describe group extensions
- reflection/bin/ontoref.nu: `describe diff [--fmt] [--file]` and
`describe api [--actor] [--tag] [--auth] [--fmt]` registered as canonical
subcommands with log-action; aliases `df` and `da` added; QUICK REFERENCE
and ALIASES sections updated
## MCP — two new tools (21 → 29 total)
- ontoref_api_catalog: filters catalog() output by actor/tag/auth; returns
{ routes, total } — no HTTP roundtrip, calls inventory directly
- ontoref_file_versions: reads ProjectContext.file_versions DashMap per slug;
returns BTreeMap<filename, u64> reload counters
- insert_mcp_ctx: audited and updated from 15 to 28 entries in 6 groups
- HelpTool JSON: 8 new entries (validate_adrs, validate, impact, guides,
bookmark_list, bookmark_add, api_catalog, file_versions)
- ServerHandler::get_info instructions updated to mention new tools
## Web UI — dashboard additions
- Dashboard: "API Catalog" card (9th); "Ontology File Versions" section showing
per-file reload counters from file_versions DashMap
- dashboard_mp: builds BTreeMap<String, u64> from ctx.file_versions and injects
into Tera context
## on+re update
- .ontology/core.ncl: describe-query-layer and adopt-ontoref-tooling descriptions
updated; ontoref-daemon updated ("11 pages", "29 tools", API catalog,
per-file versioning, #[onto_api]); new node api-catalog-surface (Yang/Practice)
with 3 edges; artifact_paths extended across 3 nodes
- .ontology/state.ncl: protocol-maturity blocker updated (protocol v2 complete);
self-description-coverage catalyst updated with session 2026-03-23 additions
- ADR-007: "API Surface Discoverability via #[onto_api] Proc-Macro" — Accepted
## Documentation
- README.md: crates table updated (11 pages, 29 MCP tools, ontoref-derive row);
MCP representative table expanded; API Catalog, Semantic Diff, Per-File
Versioning paragraphs added; update_ontoref onboarding section added
- CHANGELOG.md: [Unreleased] section with 4 change groups
- assets/web/src/index.html: tool counts 19→29 (EN+ES), page counts 12→11
(EN+ES), daemon description paragraph updated with API catalog + #[onto_api]
193 lines
10 KiB
Markdown
193 lines
10 KiB
Markdown
<p align="center">
|
||
<img src="assets/ontoref-logo.svg" alt="ontoref" width="320" />
|
||
</p>
|
||
|
||
<br>
|
||
|
||
**ontoref** is a protocol specification and tooling layer for structured self-knowledge in
|
||
software projects. It provides schemas, Nushell automation, and Rust crates so that projects
|
||
can describe what they are, record architectural decisions, track operational state, and
|
||
execute formalized procedures — all as typed, queryable artifacts.
|
||
|
||
---
|
||
|
||
## Axioms
|
||
|
||
| Axiom | Meaning |
|
||
| --- | --- |
|
||
| **Protocol, Not Runtime** | Never a runtime dependency. Projects adopt the protocol; ontoref provides the schemas and modules to do so. |
|
||
| **Self-Describing** | Consumes its own protocol: `.ontology/`, `adrs/`, `reflection/` in this repo ARE ontoref running against itself. |
|
||
| **No Enforcement** | ontoref defines contracts. There is no enforcement mechanism. Coherence is voluntary and emerges from justified adoption. |
|
||
| **DAG-Formalized Knowledge** | Concepts, tensions, decisions, state — encoded as DAGs. Enables transversal queries and impact analysis. |
|
||
|
||
## Layers
|
||
|
||
```text
|
||
ontology/ Protocol specification — Nickel schemas for nodes, edges, ADRs, state, gates
|
||
adrs/ Architecture Decision Records — typed NCL with constraints and ontology checks
|
||
reflection/ Operational tooling — Nushell modules, DAG modes, forms, and schemas
|
||
crates/ Rust implementation — typed struct loaders and mode executors
|
||
.ontology/ Self-description — ontoref's own ontology, state, gate, and manifest
|
||
```
|
||
|
||
## Crates
|
||
|
||
| Crate | Purpose |
|
||
| --- | --- |
|
||
| `ontoref-ontology` | `.ontology/` NCL → typed Rust structs: Node, Edge, Dimension, Gate, Membrane. `Node` carries `artifact_paths` and `adrs` (`Vec<String>`, both `serde(default)`). Graph traversal, invariant queries. Zero deps. |
|
||
| `ontoref-reflection` | NCL DAG contract executor: ADR lifecycle, step dep resolution, config seal. `stratum-graph` + `stratum-state` required. |
|
||
| `ontoref-daemon` | HTTP UI (11 pages), actor registry, notification barrier, MCP (29 tools), search engine, search bookmarks, SurrealDB, NCL export cache, per-file ontology versioning, annotated API catalog. |
|
||
| `ontoref-derive` | Proc-macro crate. `#[onto_api(...)]` annotates HTTP handlers; `inventory::submit!` emits route entries at link time. `GET /api/catalog` aggregates them via `inventory::collect!`. |
|
||
|
||
`ontoref-daemon` caches `nickel export` results (keyed by path + mtime), reducing full sync
|
||
scans from ~2m42s to <30s. The daemon is always optional — every module falls back to direct
|
||
subprocess when unavailable.
|
||
|
||
### Daemon Capabilities
|
||
|
||
**Unified Auth Model** — all surfaces (CLI, UI, MCP) exchange a key for a UUID v4 session token
|
||
via `POST /sessions`. Token lifetime: 30 days, O(1) lookup. Project keys carry `role`
|
||
(admin|viewer) and `label` for audit trail. Daemon-level admin via `ONTOREF_ADMIN_TOKEN_FILE`.
|
||
`GET /sessions` and `DELETE /sessions/{id}` for session visibility and revocation. Key rotation
|
||
invalidates all sessions for the rotated project. CLI injects `ONTOREF_TOKEN` as Bearer
|
||
automatically.
|
||
|
||
**Q&A Knowledge Store** — accumulated Q&A entries persist to `reflection/qa.ncl` (typed NCL,
|
||
git-versioned). Not localStorage. Any actor — developer, agent, CI — reads the same store.
|
||
|
||
**MCP Server** — 29 tools over stdio and streamable-HTTP. Categories: discovery, retrieval, project
|
||
state, ontology, backlog, validation, Q&A, bookmarks, API surface. Representative subset:
|
||
|
||
| Tool | What it does |
|
||
| --- | --- |
|
||
| `ontoref_guides` | Full project context on cold start: axioms, practices, gate, actor policy |
|
||
| `ontoref_api_catalog` | Annotated HTTP surface — all routes with auth, actors, params, tags |
|
||
| `ontoref_file_versions` | Per-file reload counters — detect which ontology files changed |
|
||
| `ontoref_validate_adrs` | Run typed ADR constraint checks; returns pass/fail per constraint |
|
||
| `ontoref_validate` | Full project validation: ADRs, content assets, connections, gate consistency |
|
||
| `ontoref_impact` | BFS impact graph from a node, optionally across project connections |
|
||
| `ontoref_qa_list` | List Q&A entries with optional filter |
|
||
| `ontoref_qa_add` | Append a new Q&A entry to `reflection/qa.ncl` |
|
||
| `ontoref_action_list` | List quick actions from `.ontoref/config.ncl` |
|
||
| `ontoref_action_add` | Create a reflection mode + register as a quick action |
|
||
|
||
**Search Bookmarks** — search results persist to `reflection/search_bookmarks.ncl` (typed NCL,
|
||
`BookmarkEntry` schema). Same atomic-write pattern as Q&A. IDs are sequential `sb-NNN`.
|
||
Concurrency-safe via `NclWriteLock`. Add and remove from the daemon search UI.
|
||
|
||
**Personal Ontology Schemas** — `ontology/schemas/career.ncl`, `personal.ncl`, `project-card.ncl`
|
||
provide typed contract layers for career and content artifacts (Skills, WorkExperience, Talks,
|
||
Content lifecycle, Opportunities, PublicationCards). All types carry `linked_nodes` referencing
|
||
core ontology node IDs — bridging career artifacts into the DAG. Five content/career reflection
|
||
modes (`draft-application`, `draft-email`, `generate-article`, `update-cv`, `write-cfp`) query
|
||
these schemas to ground output in declared project artifacts rather than free-form prose.
|
||
|
||
**API Catalog** — every HTTP handler carries `#[onto_api(method, path, description, auth, actors, params, tags)]`.
|
||
At link time `inventory::submit!` registers each route. `GET /api/catalog` returns the full annotated
|
||
surface as JSON. The `/ui/{slug}/api` page renders it with client-side filtering (method, auth, path).
|
||
`describe api [--actor] [--tag] [--fmt]` renders the catalog in the CLI. `ontoref_api_catalog` exposes
|
||
it to MCP agents.
|
||
|
||
**Semantic Diff** — `describe diff [--file <ncl>] [--fmt json|text]` computes a node- and edge-level
|
||
diff of `.ontology/` files against the last git commit. Reports added/removed/changed nodes by id and
|
||
edges by `from→to[kind]` key — not a text diff.
|
||
|
||
**Per-File Versioning** — each ontology file tracked in `ProjectContext.file_versions: DashMap<PathBuf, u64>`.
|
||
Counter increments on every watcher-triggered reload. `GET /projects/{slug}/ontology/versions` and
|
||
`ontoref_file_versions` MCP tool expose the map. Dashboard surfaces the counters.
|
||
|
||
**ADR–Node Linkage** — nodes declare which ADRs validate them via `adrs: Array String`.
|
||
`describe` surfaces a **Validated by** section per node (CLI and `--fmt md`). The graph UI
|
||
renders each ADR as a clickable link that opens the full ADR content in a modal via
|
||
`GET /api/adr/{id}`.
|
||
|
||
**Passive Drift Observation** — background file watcher that detects divergence between Yang
|
||
code artifacts and Yin ontology. Watches `crates/`, `.ontology/`, `adrs/`, `reflection/modes/`.
|
||
After a 15s debounce runs `sync scan + sync diff`; emits an `ontology_drift` notification when
|
||
MISSING/STALE/DRIFT/BROKEN items are found. Never applies changes — `apply` is always deliberate.
|
||
|
||
**Quick Actions** — runnable shortcuts over reflection modes, configured as `quick_actions` in
|
||
`.ontoref/config.ncl`. Accessible from HTTP (`/actions`), CLI (`ontoref`), and MCP
|
||
(`ontoref_action_list/add`).
|
||
|
||
## Install
|
||
|
||
```sh
|
||
just install-daemon # build + install binary, bootstrapper, CLI, UI assets, config skeleton
|
||
ontoref config-edit # browser form → ~/.config/ontoref/config.ncl
|
||
ontoref-daemon-boot # NCL pipe bootstrap: nickel export config.ncl | daemon --config-stdin
|
||
ontoref-daemon-boot --dry-run # preview composed JSON without starting
|
||
```
|
||
|
||
Installed layout (`~/.local/bin/`):
|
||
|
||
| Binary | Role |
|
||
| --- | --- |
|
||
| `ontoref` | Global CLI dispatcher — all reflection modes, ADR lifecycle, daemon control |
|
||
| `ontoref-daemon` | Bootstrapper (public entrypoint) — validates config via Nickel, pipes JSON to binary |
|
||
| `ontoref-daemon.bin` | Compiled Rust binary — never called directly |
|
||
|
||
Global config at `~/.config/ontoref/config.ncl` (type-checked Nickel). Global NATS stream topology at
|
||
`~/.config/ontoref/streams.json`. Project-local topology override via `nats/streams.json` +
|
||
`nats_events.streams_config` in `.ontoref/config.ncl`.
|
||
|
||
## Onboarding a project
|
||
|
||
```sh
|
||
cd /path/to/my-project
|
||
ontoref setup # idempotent; kind: Service by default
|
||
ontoref setup --kind Library # Library | DevWorkspace | PublishedCrate | AgentResource | Mixed
|
||
ontoref setup --parent /path/to/fw # implementation child: adds framework layer + browse mode
|
||
ontoref setup --gen-keys ["admin:dev" "viewer:ci"] # bootstrap auth keys (no-op if keys already exist)
|
||
```
|
||
|
||
`ontoref setup` creates `.ontoref/project.ncl`, `.ontoref/config.ncl` (with logo auto-detection),
|
||
`.ontology/` scaffold, `adrs/`, `reflection/modes/`, `backlog.ncl`, `qa.ncl`, git hooks, and
|
||
registers the project in `~/.config/ontoref/projects.ncl`.
|
||
|
||
For existing projects that predate `setup`, or to bring an already-adopted project up to the
|
||
current protocol version (adds `manifest.ncl` and `connections.ncl`):
|
||
|
||
```sh
|
||
ontoref --actor developer adopt_ontoref # first-time adoption
|
||
ontoref run update_ontoref # bring existing project to protocol v2
|
||
```
|
||
|
||
The `update_ontoref` mode detects missing v2 files, adds them idempotently, validates both with
|
||
`nickel export`, scans ADRs for deprecated `check_hint` fields, and prints a protocol update
|
||
report. The reusable `reflection/templates/update-ontology-prompt.md` guides an agent through
|
||
full ontology enrichment in 8 phases.
|
||
|
||
`ONTOREF_PROJECT_ROOT` is set by the consumer wrapper — one ontoref checkout serves multiple projects.
|
||
|
||
## Prerequisites
|
||
|
||
- [Nushell](https://www.nushell.sh/) >= 0.110.0
|
||
- [Nickel](https://nickel-lang.org/) (for schema evaluation)
|
||
- Rust toolchain (for building crates)
|
||
- [Just](https://just.systems/) (for CI recipes)
|
||
|
||
To build `ontoref-daemon` and `ontoref-reflection` with NATS/SurrealDB support, the
|
||
stratumiops repo must be checked out at `../../../stratumiops`. Without it, build without
|
||
default features:
|
||
|
||
```sh
|
||
cargo build -p ontoref-daemon --no-default-features
|
||
cargo build -p ontoref-ontology # always standalone
|
||
```
|
||
|
||
## Development
|
||
|
||
```sh
|
||
cargo check-all # check all targets + features
|
||
cargo test-all # run full test suite
|
||
just ci-lint # clippy + TOML + Nickel + Markdown
|
||
just ci-full # all CI checks
|
||
nu --ide-check 50 reflection/modules/<file>.nu # validate a Nushell module
|
||
./ontoref --actor developer <mode> # run a reflection mode
|
||
```
|
||
|
||
## License
|
||
|
||
MIT OR Apache-2.0
|