ontoref/adrs/adr-007-api-surface-discoverability-onto-api-proc-macro.ncl

85 lines
6.9 KiB
Plaintext
Raw Permalink Normal View History

--- feat: API catalog surface, protocol v2 tooling, MCP expansion, on+re update ## Summary Session 2026-03-23. Closes the loop between handler code and discoverability across all three surfaces (browser, CLI, MCP agent) via compile-time inventory registration. Adds protocol v2 update tooling, extends MCP from 21 to 29 tools, and brings the self-description up to date. ## API Catalog Surface (#[onto_api] proc-macro) - crates/ontoref-derive: new proc-macro crate; `#[onto_api(method, path, description, auth, actors, params, tags)]` emits `inventory::submit!(ApiRouteEntry{...})` at link time - crates/ontoref-daemon/src/api_catalog.rs: `catalog()` — pure fn over `inventory::iter::<ApiRouteEntry>()`, zero runtime allocation - GET /api/catalog: returns full annotated HTTP surface as JSON - templates/pages/api_catalog.html: new page with client-side filtering by method, auth, path/description; detail panel per route (params table, feature flag); linked from dashboard card and nav - UI nav: "API" link (</> icon) added to mobile dropdown and desktop bar - inventory = "0.3" added to workspace.dependencies (MIT, zero transitive deps) ## Protocol Update Mode - reflection/modes/update_ontoref.ncl: 9-step DAG (5 detect parallel, 2 update idempotent, 2 validate, 1 report) — brings any project from protocol v1 to v2 by adding manifest.ncl and connections.ncl if absent, scanning ADRs for deprecated check_hint, validating with nickel export - reflection/templates/update-ontology-prompt.md: 8-phase reusable prompt for agent-driven ontology enrichment (infrastructure → audit → core.ncl → state.ncl → manifest.ncl → connections.ncl → ADR migration → validation) ## CLI — describe group extensions - reflection/bin/ontoref.nu: `describe diff [--fmt] [--file]` and `describe api [--actor] [--tag] [--auth] [--fmt]` registered as canonical subcommands with log-action; aliases `df` and `da` added; QUICK REFERENCE and ALIASES sections updated ## MCP — two new tools (21 → 29 total) - ontoref_api_catalog: filters catalog() output by actor/tag/auth; returns { routes, total } — no HTTP roundtrip, calls inventory directly - ontoref_file_versions: reads ProjectContext.file_versions DashMap per slug; returns BTreeMap<filename, u64> reload counters - insert_mcp_ctx: audited and updated from 15 to 28 entries in 6 groups - HelpTool JSON: 8 new entries (validate_adrs, validate, impact, guides, bookmark_list, bookmark_add, api_catalog, file_versions) - ServerHandler::get_info instructions updated to mention new tools ## Web UI — dashboard additions - Dashboard: "API Catalog" card (9th); "Ontology File Versions" section showing per-file reload counters from file_versions DashMap - dashboard_mp: builds BTreeMap<String, u64> from ctx.file_versions and injects into Tera context ## on+re update - .ontology/core.ncl: describe-query-layer and adopt-ontoref-tooling descriptions updated; ontoref-daemon updated ("11 pages", "29 tools", API catalog, per-file versioning, #[onto_api]); new node api-catalog-surface (Yang/Practice) with 3 edges; artifact_paths extended across 3 nodes - .ontology/state.ncl: protocol-maturity blocker updated (protocol v2 complete); self-description-coverage catalyst updated with session 2026-03-23 additions - ADR-007: "API Surface Discoverability via #[onto_api] Proc-Macro" — Accepted ## Documentation - README.md: crates table updated (11 pages, 29 MCP tools, ontoref-derive row); MCP representative table expanded; API Catalog, Semantic Diff, Per-File Versioning paragraphs added; update_ontoref onboarding section added - CHANGELOG.md: [Unreleased] section with 4 change groups - assets/web/src/index.html: tool counts 19→29 (EN+ES), page counts 12→11 (EN+ES), daemon description paragraph updated with API catalog + #[onto_api]
2026-03-23 00:58:27 +01:00
let d = import "adr-defaults.ncl" in
d.make_adr {
id = "adr-007",
title = "API Surface Discoverability via #[onto_api] Proc-Macro",
status = 'Accepted,
date = "2026-03-23",
context = "ontoref-daemon exposes ~28 HTTP routes across api.rs, sync.rs, and other handler modules. Before this decision, the authoritative route list existed only in the axum Router definition — undiscoverable without reading source. MCP agents, CLI users, and the web UI had no machine-readable way to enumerate routes, their auth requirements, parameter shapes, or actor restrictions. OpenAPI was considered but rejected as a runtime dependency that would require schema maintenance separate from the handler code. The `#[onto_api]` proc-macro in `ontoref-derive` addresses this by making the handler annotation the single source of truth: the macro emits `inventory::submit!(ApiRouteEntry{...})` at link time, and `api_catalog::catalog()` collects them via `inventory::collect!`. No runtime registry, no startup allocation, no separate schema file.",
decision = "Every HTTP handler in ontoref-daemon must carry `#[onto_api(method, path, description, auth, actors, params, tags)]`. The proc-macro (in `crates/ontoref-derive`) emits `inventory::submit!(ApiRouteEntry{...})` at link time. `GET /api/catalog` calls `api_catalog::catalog()` — a pure function over `inventory::iter::<ApiRouteEntry>()` — and returns the annotated surface as JSON. The web UI at `/ui/{slug}/api` renders it with client-side filtering. `describe api [--actor] [--tag] [--auth] [--fmt]` queries this endpoint from the CLI. The MCP tool `ontoref_api_catalog` calls `catalog()` directly without HTTP. This surfaces the complete API to three actors (browser, CLI, MCP agent) from one annotation site per handler.",
rationale = [
{
claim = "Compile-time registration eliminates drift",
detail = "inventory uses linker sections (.init_array on ELF, __mod_init_func on Mach-O) to collect ApiRouteEntry items at link time. A handler that exists in the binary but lacks #[onto_api] is detectable — cargo test or a Grep constraint catches the gap. A handler that has #[onto_api] but is removed will automatically disappear from catalog(). The annotation and the implementation are co-located and co-deleted.",
},
{
claim = "Zero runtime overhead and zero startup allocation",
detail = "inventory::iter::<ApiRouteEntry>() walks a linked-list built by the linker — no HashMap, no Arc, no lazy_static. catalog() is a pure function that sorts and returns &'static references. This satisfies the ontoref axiom 'Protocol, Not Runtime': the catalog is available without daemon state, without DB, without cache warmup.",
},
{
claim = "Three-surface consistency without duplication",
detail = "Browser (api_catalog.html), CLI (describe api), and MCP (ontoref_api_catalog) all read the same inventory. A manual registry or OpenAPI spec would require three update sites per route change. With #[onto_api], changing a route's auth requirement is a one-line annotation edit that propagates to all surfaces on next build.",
},
],
consequences = {
positive = [
"API surface is always current: catalog() reflects exactly the handlers compiled into the binary",
"Agents (MCP) can call ontoref_api_catalog on cold start to understand the full HTTP surface without prior knowledge",
"describe api --actor agent filters to actor-appropriate routes; agents can self-serve their available endpoints",
"New handlers without #[onto_api] are caught by the Grep constraint before merge",
"inventory (MIT, 0.3.x) has no transitive deps — passes deny.toml audit",
],
negative = [
"#[onto_api] parameters are stringly-typed — a misspelled auth value is not caught at compile time (only at review/Grep)",
"inventory linker trick is platform-specific: supported on Linux (ELF), macOS (Mach-O), Windows (PE) but not on targets that lack .init_array equivalent",
"Proc-macro adds a new crate (ontoref-derive) to the workspace; ontoref-ontology users who only need zero-dep struct loading do not need it",
],
},
alternatives_considered = [
{
option = "OpenAPI / utoipa with generated JSON schema",
why_rejected = "Requires maintaining a separate schema artifact (openapi.json) and a runtime schema struct tree. The schema can drift from actual handler signatures. utoipa adds ~15 transitive deps including serde_yaml. Violates 'Protocol, Not Runtime' — the schema becomes a runtime artifact rather than a compile-time invariant.",
},
{
option = "Manual route registry (Vec<RouteInfo> in main.rs)",
why_rejected = "A manually maintained Vec has guaranteed drift: handlers are added, routes change, and the Vec is updated inconsistently. Proven failure mode in the previous session where insert_mcp_ctx listed 15 tools while the router had 27.",
},
{
option = "Runtime reflection via axum Router introspection",
why_rejected = "axum does not expose a stable introspection API for registered routes. Workarounds (tower_http trace layer capture, method_router hacks) are brittle across axum versions and cannot surface handler metadata (auth, actors, params).",
},
],
constraints = [
{
id = "onto-api-on-all-handlers",
claim = "Every public HTTP handler in ontoref-daemon must carry #[onto_api(...)]",
scope = "ontoref-daemon (crates/ontoref-daemon/src/api.rs, crates/ontoref-daemon/src/sync.rs)",
severity = 'Hard,
feat: config surface, NCL contracts, override-layer mutation, on+re update Config surface — per-project config introspection, coherence verification, and audited mutation without destroying NCL structure (ADR-008): - crates/ontoref-daemon/src/config.rs — typed DaemonNclConfig (parse-at-boundary pattern); all section structs derive ConfigFields + config_section(id, ncl_file) emitting inventory::submit!(ConfigFieldsEntry{...}) at link time - crates/ontoref-derive/src/lib.rs — #[derive(ConfigFields)] proc-macro; serde rename support; serde_rename_of() helper extracted to fix excessive_nesting - crates/ontoref-daemon/src/main.rs — 3-tuple bootstrap block (nickel_import_path, loaded_ncl_config: Option<DaemonNclConfig>, stdin_raw); apply_ui_config takes &UiConfig; NATS call site typed; resolve_asset_dir cfg(feature = "ui") - crates/ontoref-daemon/src/api.rs — config GET/PUT endpoints, quickref, coherence, cross-project comparison; index_section_fields() extracted (excessive_nesting) - crates/ontoref-daemon/src/config_coherence.rs — multi-consumer coherence; merge_meta_into_section() extracted; and() replaces unnecessary and_then NCL contracts for ontoref's own config: - .ontoref/contracts.ncl — LogConfig (LogLevel, LogRotation, PositiveInt) and DaemonConfig (Port, optional overrides); std.contract.from_validator throughout - .ontoref/config.ncl — log | C.LogConfig applied - .ontology/manifest.ncl — contracts_path, log/daemon contract refs, daemon section with DaemonRuntimeConfig consumer and 7 declared fields Protocol: - adrs/adr-008-ncl-first-config-validation-and-override-layer.ncl — NCL contracts as single validation gate; Rust structs are contract-trusted; override-layer mutation writes {section}.overrides.ncl + _overrides_meta, never touches source on+re update: - .ontology/core.ncl — config-surface node (28 practices); adr-lifecycle extended to adr-007 + adr-008; 6 new edges (ManifestsIn daemon, DependsOn ontology-crate, Complements api-catalog-surface/dag-formalized/self-describing/adopt-ontoref) - .ontology/state.ncl — protocol-maturity blocker and self-description-coverage catalyst updated for session 2026-03-26 - README.md / CHANGELOG.md updated
2026-03-26 20:20:22 +00:00
check = { tag = 'Grep, pattern = "#\\[onto_api", paths = ["crates/ontoref-daemon/src/api.rs", "crates/ontoref-daemon/src/sync.rs"], must_be_empty = false },
--- feat: API catalog surface, protocol v2 tooling, MCP expansion, on+re update ## Summary Session 2026-03-23. Closes the loop between handler code and discoverability across all three surfaces (browser, CLI, MCP agent) via compile-time inventory registration. Adds protocol v2 update tooling, extends MCP from 21 to 29 tools, and brings the self-description up to date. ## API Catalog Surface (#[onto_api] proc-macro) - crates/ontoref-derive: new proc-macro crate; `#[onto_api(method, path, description, auth, actors, params, tags)]` emits `inventory::submit!(ApiRouteEntry{...})` at link time - crates/ontoref-daemon/src/api_catalog.rs: `catalog()` — pure fn over `inventory::iter::<ApiRouteEntry>()`, zero runtime allocation - GET /api/catalog: returns full annotated HTTP surface as JSON - templates/pages/api_catalog.html: new page with client-side filtering by method, auth, path/description; detail panel per route (params table, feature flag); linked from dashboard card and nav - UI nav: "API" link (</> icon) added to mobile dropdown and desktop bar - inventory = "0.3" added to workspace.dependencies (MIT, zero transitive deps) ## Protocol Update Mode - reflection/modes/update_ontoref.ncl: 9-step DAG (5 detect parallel, 2 update idempotent, 2 validate, 1 report) — brings any project from protocol v1 to v2 by adding manifest.ncl and connections.ncl if absent, scanning ADRs for deprecated check_hint, validating with nickel export - reflection/templates/update-ontology-prompt.md: 8-phase reusable prompt for agent-driven ontology enrichment (infrastructure → audit → core.ncl → state.ncl → manifest.ncl → connections.ncl → ADR migration → validation) ## CLI — describe group extensions - reflection/bin/ontoref.nu: `describe diff [--fmt] [--file]` and `describe api [--actor] [--tag] [--auth] [--fmt]` registered as canonical subcommands with log-action; aliases `df` and `da` added; QUICK REFERENCE and ALIASES sections updated ## MCP — two new tools (21 → 29 total) - ontoref_api_catalog: filters catalog() output by actor/tag/auth; returns { routes, total } — no HTTP roundtrip, calls inventory directly - ontoref_file_versions: reads ProjectContext.file_versions DashMap per slug; returns BTreeMap<filename, u64> reload counters - insert_mcp_ctx: audited and updated from 15 to 28 entries in 6 groups - HelpTool JSON: 8 new entries (validate_adrs, validate, impact, guides, bookmark_list, bookmark_add, api_catalog, file_versions) - ServerHandler::get_info instructions updated to mention new tools ## Web UI — dashboard additions - Dashboard: "API Catalog" card (9th); "Ontology File Versions" section showing per-file reload counters from file_versions DashMap - dashboard_mp: builds BTreeMap<String, u64> from ctx.file_versions and injects into Tera context ## on+re update - .ontology/core.ncl: describe-query-layer and adopt-ontoref-tooling descriptions updated; ontoref-daemon updated ("11 pages", "29 tools", API catalog, per-file versioning, #[onto_api]); new node api-catalog-surface (Yang/Practice) with 3 edges; artifact_paths extended across 3 nodes - .ontology/state.ncl: protocol-maturity blocker updated (protocol v2 complete); self-description-coverage catalyst updated with session 2026-03-23 additions - ADR-007: "API Surface Discoverability via #[onto_api] Proc-Macro" — Accepted ## Documentation - README.md: crates table updated (11 pages, 29 MCP tools, ontoref-derive row); MCP representative table expanded; API Catalog, Semantic Diff, Per-File Versioning paragraphs added; update_ontoref onboarding section added - CHANGELOG.md: [Unreleased] section with 4 change groups - assets/web/src/index.html: tool counts 19→29 (EN+ES), page counts 12→11 (EN+ES), daemon description paragraph updated with API catalog + #[onto_api]
2026-03-23 00:58:27 +01:00
rationale = "catalog() is only as complete as the set of annotated handlers. Unannotated handlers are invisible to agents, CLI, and the web UI — equivalent to undocumented and unauditable routes.",
},
{
id = "inventory-feature-gate",
claim = "inventory must remain a workspace dependency gated behind the 'catalog' feature of ontoref-derive; ontoref-ontology must not depend on inventory",
scope = "ontoref-ontology (Cargo.toml), ontoref-derive (Cargo.toml)",
severity = 'Hard,
feat: config surface, NCL contracts, override-layer mutation, on+re update Config surface — per-project config introspection, coherence verification, and audited mutation without destroying NCL structure (ADR-008): - crates/ontoref-daemon/src/config.rs — typed DaemonNclConfig (parse-at-boundary pattern); all section structs derive ConfigFields + config_section(id, ncl_file) emitting inventory::submit!(ConfigFieldsEntry{...}) at link time - crates/ontoref-derive/src/lib.rs — #[derive(ConfigFields)] proc-macro; serde rename support; serde_rename_of() helper extracted to fix excessive_nesting - crates/ontoref-daemon/src/main.rs — 3-tuple bootstrap block (nickel_import_path, loaded_ncl_config: Option<DaemonNclConfig>, stdin_raw); apply_ui_config takes &UiConfig; NATS call site typed; resolve_asset_dir cfg(feature = "ui") - crates/ontoref-daemon/src/api.rs — config GET/PUT endpoints, quickref, coherence, cross-project comparison; index_section_fields() extracted (excessive_nesting) - crates/ontoref-daemon/src/config_coherence.rs — multi-consumer coherence; merge_meta_into_section() extracted; and() replaces unnecessary and_then NCL contracts for ontoref's own config: - .ontoref/contracts.ncl — LogConfig (LogLevel, LogRotation, PositiveInt) and DaemonConfig (Port, optional overrides); std.contract.from_validator throughout - .ontoref/config.ncl — log | C.LogConfig applied - .ontology/manifest.ncl — contracts_path, log/daemon contract refs, daemon section with DaemonRuntimeConfig consumer and 7 declared fields Protocol: - adrs/adr-008-ncl-first-config-validation-and-override-layer.ncl — NCL contracts as single validation gate; Rust structs are contract-trusted; override-layer mutation writes {section}.overrides.ncl + _overrides_meta, never touches source on+re update: - .ontology/core.ncl — config-surface node (28 practices); adr-lifecycle extended to adr-007 + adr-008; 6 new edges (ManifestsIn daemon, DependsOn ontology-crate, Complements api-catalog-surface/dag-formalized/self-describing/adopt-ontoref) - .ontology/state.ncl — protocol-maturity blocker and self-description-coverage catalyst updated for session 2026-03-26 - README.md / CHANGELOG.md updated
2026-03-26 20:20:22 +00:00
check = { tag = 'Grep, pattern = "inventory", paths = ["crates/ontoref-ontology/Cargo.toml"], must_be_empty = true },
--- feat: API catalog surface, protocol v2 tooling, MCP expansion, on+re update ## Summary Session 2026-03-23. Closes the loop between handler code and discoverability across all three surfaces (browser, CLI, MCP agent) via compile-time inventory registration. Adds protocol v2 update tooling, extends MCP from 21 to 29 tools, and brings the self-description up to date. ## API Catalog Surface (#[onto_api] proc-macro) - crates/ontoref-derive: new proc-macro crate; `#[onto_api(method, path, description, auth, actors, params, tags)]` emits `inventory::submit!(ApiRouteEntry{...})` at link time - crates/ontoref-daemon/src/api_catalog.rs: `catalog()` — pure fn over `inventory::iter::<ApiRouteEntry>()`, zero runtime allocation - GET /api/catalog: returns full annotated HTTP surface as JSON - templates/pages/api_catalog.html: new page with client-side filtering by method, auth, path/description; detail panel per route (params table, feature flag); linked from dashboard card and nav - UI nav: "API" link (</> icon) added to mobile dropdown and desktop bar - inventory = "0.3" added to workspace.dependencies (MIT, zero transitive deps) ## Protocol Update Mode - reflection/modes/update_ontoref.ncl: 9-step DAG (5 detect parallel, 2 update idempotent, 2 validate, 1 report) — brings any project from protocol v1 to v2 by adding manifest.ncl and connections.ncl if absent, scanning ADRs for deprecated check_hint, validating with nickel export - reflection/templates/update-ontology-prompt.md: 8-phase reusable prompt for agent-driven ontology enrichment (infrastructure → audit → core.ncl → state.ncl → manifest.ncl → connections.ncl → ADR migration → validation) ## CLI — describe group extensions - reflection/bin/ontoref.nu: `describe diff [--fmt] [--file]` and `describe api [--actor] [--tag] [--auth] [--fmt]` registered as canonical subcommands with log-action; aliases `df` and `da` added; QUICK REFERENCE and ALIASES sections updated ## MCP — two new tools (21 → 29 total) - ontoref_api_catalog: filters catalog() output by actor/tag/auth; returns { routes, total } — no HTTP roundtrip, calls inventory directly - ontoref_file_versions: reads ProjectContext.file_versions DashMap per slug; returns BTreeMap<filename, u64> reload counters - insert_mcp_ctx: audited and updated from 15 to 28 entries in 6 groups - HelpTool JSON: 8 new entries (validate_adrs, validate, impact, guides, bookmark_list, bookmark_add, api_catalog, file_versions) - ServerHandler::get_info instructions updated to mention new tools ## Web UI — dashboard additions - Dashboard: "API Catalog" card (9th); "Ontology File Versions" section showing per-file reload counters from file_versions DashMap - dashboard_mp: builds BTreeMap<String, u64> from ctx.file_versions and injects into Tera context ## on+re update - .ontology/core.ncl: describe-query-layer and adopt-ontoref-tooling descriptions updated; ontoref-daemon updated ("11 pages", "29 tools", API catalog, per-file versioning, #[onto_api]); new node api-catalog-surface (Yang/Practice) with 3 edges; artifact_paths extended across 3 nodes - .ontology/state.ncl: protocol-maturity blocker updated (protocol v2 complete); self-description-coverage catalyst updated with session 2026-03-23 additions - ADR-007: "API Surface Discoverability via #[onto_api] Proc-Macro" — Accepted ## Documentation - README.md: crates table updated (11 pages, 29 MCP tools, ontoref-derive row); MCP representative table expanded; API Catalog, Semantic Diff, Per-File Versioning paragraphs added; update_ontoref onboarding section added - CHANGELOG.md: [Unreleased] section with 4 change groups - assets/web/src/index.html: tool counts 19→29 (EN+ES), page counts 12→11 (EN+ES), daemon description paragraph updated with API catalog + #[onto_api]
2026-03-23 00:58:27 +01:00
rationale = "ontoref-ontology is the zero-dep adoption surface (ADR-001). Adding inventory — even as an optional dep — violates that contract and makes protocol adoption heavier for downstream crates that only need typed NCL loading.",
},
],
related_adrs = ["adr-001"],
ontology_check = {
decision_string = "Use #[onto_api] proc-macro + inventory linker registration as the single source of truth for the HTTP API surface; surface via GET /api/catalog, describe api CLI subcommand, and ontoref_api_catalog MCP tool",
invariants_at_risk = ["protocol-not-runtime"],
verdict = 'Safe,
},
}