jesus/prvng_platform
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: update README and CHANGELOG with ops control plane, audit-mirror, contract-tests

Browse source
This commit is contained in:
Jesús Pérez 2026-05-12 03:13:17 +01:00
parent 2431636064
commit a747d8a201
Signed by: jesus
GPG key ID: 9F243E355E0BC939
403 changed files with 37239 additions and 7018 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.cargo/config.toml Normal file
View file

@ -0,0 +1,2 @@
[build]
target-dir = "/Volumes/Devel/provisioning/platform/target"

138
CHANGELOG.md Normal file
View file

@ -0,0 +1,138 @@
# Platform Changelog
## [Unreleased]
### Added
#### ops control plane (`crates/ops-keeper`, `crates/ops-controller`)
New two-component ops control plane implementing secure, auditable operation dispatch.
**ops-keeper** — policy gate before any operation reaches the orchestrator:
- Glob-based `PolicyDef` matching on `op_type`, `image_patterns`, `target_patterns`
- `Signer` issues compact Ed25519 JWTs (`OpsClaims`) on policy approval
- NATS JetStream pull consumer on `ops.pending.*`
- `AuditEvent` emission to `ops.audit.*` on every terminal decision
**ops-controller** — durable executor with at-least-once delivery:
- Verifies keeper-signed JWT before dispatching to orchestrator HTTP API
- SurrealDB idempotency store; `reconcile_pending` reconciles stale ops on startup
- Structured `AckResult` (Ack / Nak / Term) per JetStream semantics
- Emits audit events to `ops.audit.*` on every terminal outcome
**ADR**: ADR-038 (ops control plane design)
#### audit-mirror (`crates/audit-mirror`)
Sidecar that consumes `ops.audit.*` and creates a signed git commit per event in a local Radicle repository, then announces the repo. JTI deduplication prevents double-commits on consumer redelivery.
**ADR**: ADR-038
#### contract-tests (`crates/contract-tests`)
G3 contract test suite verifying semantic equivalence across three tiers:
- **Tier A**: direct registry invocation (reference baseline)
- **Tier B**: axum HTTP daemon on ephemeral port
- **Tier C**: in-process MCP `handle_request`
Normaliser strips volatile fields; JSON schema validates every tier output against `listing_output_schema`. Regression guard for CLI↔HTTP↔MCP drift.
#### control-center: NATS→WebSocket bridge (`src/handlers/nats_bridge.rs`)
Eliminates long-polling between control-center UI and orchestrator. Durable JetStream consumer `cc-task-status-bridge` subscribes to `TASKS` stream and re-broadcasts each `TaskStatusPayload` to all connected WebSocket clients via `WebSocketManager`.
#### API catalog endpoints (`crates/ai-service`, `crates/control-center`)
`GET /api-catalog` added to ai-service and control-center. Each handler uses `inventory::iter::<ApiRouteEntry>()` to emit a sorted JSON catalog of all registered routes — self-describing API surface without a separate OpenAPI toolchain.
#### SOLID boundary pre-commit check (`.pre-commit-hooks/solid-boundary-check.sh`)
Shell hook enforced in `.pre-commit-config.yaml`. Fails the commit if a crate imports across declared SOLID layer boundaries, keeping dependency direction consistent before code reaches CI.
### Removed / Lifted
- **`crates/backup-manager`** — lifted to `cloudatasave` (LibreCloud/cloudDataSave). **ADR-041**.
- **`crates/buildkit-launcher`** — lifted to `lian-build`. **ADR-040**.
- **`crates/daemon`** — `Cargo.toml` removed; binary consolidated into `provisioning-daemon`.
- **`prov-ecosystem/crates/nu-daemon`** / **`daemon-cli`** — excluded from workspace; `rustls=0.23.28` pin conflicts with surrealdb@3 (`^0.23.36`). Build standalone with `cargo build -p nu-daemon` until `nu-command` relaxes the pin.
---
## [3.6.0] — 2026-04-17
### Added
#### ncl-sync daemon (`crates/ncl-sync/`)
New Rust daemon that eliminates `nickel export` latency from CLI commands by pre-compiling NCL files and maintaining a shared cache.
- **File watching**: `notify` watcher on workspace NCL directories; re-exports automatically on file change
- **Warm-up**: on `prvng platform start`, scans all NCL files in the workspace and exports stale entries concurrently (configurable concurrency)
- **Shared cache**: `~/.cache/provisioning/config-cache/` — same directory and key strategy as `nu_plugin_nickel`
- **Content-addressed keys**: `SHA256(file_content + sorted_import_paths + format)` — zero coordination overhead between daemon and plugin
- **Sync requests**: Nu processes write `.sync-<pid>.json` sidecars after mutations; daemon drains them within 500 ms
- **CLI subcommands**: `daemon`, `warm`, `invalidate`, `key`, `stats`
- **Config-driven** via `platform-config`: `platform/config/ncl-sync.ncl` — idle timeout, concurrency, poll interval, extra import paths
- **No platform dependencies**: intentionally avoids NATS, SurrealDB, and orchestrator to prevent bootstrap circularity
#### Platform lifecycle integration
- `prvng platform start` now launches `ncl-sync daemon` (via `ncl-sync-start` in `service-manager.nu`)
- `prvng platform stop` stops ncl-sync (via PID file)
- `prvng platform status` shows ncl-sync running state
#### Nu cache layer
- `lib_provisioning/config/cache/core.nu`: implemented `cache-lookup`, `cache-write`, `write-sync-request` (previously no-ops)
- `lib_provisioning/config/cache/nickel.nu`: implemented `lookup-nickel-cache`, `derive-ncl-cache-key`, `request-ncl-sync`
#### nickel_processor wrappers
New `lib_provisioning/utils/nickel_processor.nu` functions:
- `ncl-eval`: drop-in for `^nickel export ... | from json` — checks plugin cache, propagates error on failure
- `ncl-eval-soft`: soft-failure variant with configurable fallback value
#### nu_plugin_nickel updates
- `nickel-eval` and `nickel-export` now accept `--import-path LIST` flag (`-I`)
- New command `nickel-cache-key`: prints the cache key for a file (parity testing)
- Cache key derivation updated to `SHA256(file_content + sorted_import_paths + format)` — aligned with ncl-sync
- Cache key now includes import paths: same file with different import paths produces different cache entries
#### Hot-path migration (C1)
Replaced `^nickel export ... | from json` with `ncl-eval`/`ncl-eval-soft` in the four highest-frequency call sites:
- `main_provisioning/dispatcher.nu` — commands-registry load
- `main_provisioning/components.nu` — workspace settings export
- `main_provisioning/workflow.nu` — workflow NCL exports
- `main_provisioning/extensions.nu` — per-extension metadata
#### Extended migration (C2 + C3)
Migrated remaining high-value call sites (~55 additional sites across 30+ files) including `config/export.nu`, `taskservs/discover.nu`, `servers/create.nu`, `ontoref-queries.nu`, `service-manager.nu`, `dag.nu`, and others.
#### ADRs
- `adrs/adr-022-ncl-sync-daemon.ncl`: daemon design decisions, key strategy rationale, constraint enforcement
- `adrs/adr-023-ncl-export-wrapper.ncl`: `ncl-eval`/`ncl-eval-soft` wrapper design, migration strategy
#### Tests
- `tests/cache/test_key_parity.nu`: validates that ncl-sync and nu_plugin_nickel produce identical keys for the same `(file, import_paths, format)` triple
### Performance
| Command | Before | After (warm cache) |
|---------|--------|--------------------|
| `prvng component list` | 3–7 s | ~1.5 s |
| `prvng workflow list` | 3–5 s | ~1.5 s |
| `prvng deploy` | 15–30 s | ~3–5 s |
| Multi-export commands (ontoref) | 12–30 s | ~1.5 s |
Nu module parse startup (~1.2 s) is unaffected — separate concern.
---
## [3.5.0] — 2025-10-07
Initial platform version with orchestrator, control center, installer, MCP server, vault service, and extension registry.

81
Cargo.toml
View file

@ -11,24 +11,39 @@ members = [
"crates/control-center",
"crates/control-center-ui",
"crates/vault-service",
"crates/detector",
"crates/mcp-server",
"crates/daemon",
"prov-ecosystem/crates/daemon-cli",
# lifted: "crates/backup-manager" → cloudatasave (LibreCloud/cloudDataSave) — adr-041
# archived: "crates/detector" → archive/detector (no dependents, stale since Jan 2026)
"prov-ecosystem/crates/machines",
"prov-ecosystem/crates/encrypt",
"prov-ecosystem/crates/backup",
"prov-ecosystem/crates/observability",
"crates/ncl-sync",
"crates/prvng-cli",
"crates/provisioning-core",
"crates/provisioning-tool",
"crates/provisioning-daemon",
"crates/contract-tests",
"crates/extension-manager",
"crates/ops-keeper",
"crates/audit-mirror",
"crates/ops-controller",
# lifted: "crates/buildkit-launcher" → lian-build — adr-040
]
exclude = [
"syntaxis",
"syntaxis/core",
# archived: syntaxis/ → archive/syntaxis (Jan 2026, all refs commented out)
# archived: stratumiops/ → archive/stratumiops (Jan 2026, workspace uses canonical ../../../Development/stratumiops)
"prov-ecosystem/crates/syntaxis-integration",
"prov-ecosystem/crates/audit",
"prov-ecosystem/crates/valida",
"prov-ecosystem/crates/runtime",
"prov-ecosystem/crates/gitops",
# nu-daemon + daemon-cli are excluded: nu-command@0.110.0 (via nushell feature) pins
# rustls=0.23.28, hard conflict with surrealdb@3 (requires ^0.23.36). Not resolvable
# until nu-command relaxes its rustls pin. Build standalone: cargo build -p nu-daemon
"crates/nu-daemon",
"prov-ecosystem/crates/daemon-cli",
]
resolver = "2"
@ -97,12 +112,16 @@ resolver = "2"
# DATABASE AND STORAGE
# ============================================================================
sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "sqlite", "chrono", "uuid"] }
surrealdb = { version = "2.6", features = ["kv-mem", "protocol-ws", "protocol-http"] }
# kv-surrealkv: core relational/graph (orchestrator state, control-center)
# kv-rocksdb: hot data (embeddings cache, audit logs) — via platform-db embedded-rocksdb feature
# rustls excluded: nu-command@0.110.0 pins rustls=0.23.28, SurrealDB 3 requires ^0.23.36 (conflict)
# TLS for remote connections is handled at the proxy layer (nginx/Caddy) in production.
surrealdb = { version = "3", features = ["kv-mem", "kv-surrealkv", "protocol-ws"], default-features = false }
# ============================================================================
# MESSAGING (NATS)
# ============================================================================
async-nats = "0.40"
async-nats = "0.46"
# ============================================================================
# SECURITY AND CRYPTOGRAPHY
@ -112,7 +131,7 @@ resolver = "2"
base64 = "0.22"
git2 = { version = "0.20", default-features = false, features = ["https", "ssh"] }
hmac = "0.12"
jsonwebtoken = { version = "10.3", features = ["rust_crypto"] }
jsonwebtoken = { version = "10.3", default-features = false, features = ["aws_lc_rs"] }
rand = { version = "0.9", features = ["std_rng", "os_rng"] }
ring = "0.17"
sha2 = "0.10"
@ -127,12 +146,18 @@ resolver = "2"
# ============================================================================
regex = "1.12"
validator = { version = "0.20", features = ["derive"] }
globset = "0.4"
# ============================================================================
# GRAPH ALGORITHMS AND UTILITIES
# ============================================================================
petgraph = "0.8"
# ============================================================================
# CONCURRENT DATA STRUCTURES
# ============================================================================
dashmap = "6"
# ============================================================================
# ADDITIONAL SHARED DEPENDENCIES
# ============================================================================
@ -250,10 +275,8 @@ resolver = "2"
xxhash-rust = { version = "0.8", features = ["xxh3"] }
# ============================================================================
# RAG FRAMEWORK DEPENDENCIES (Rig)
# RAG AND TEXT PROCESSING
# ============================================================================
rig-core = "0.30"
rig-surrealdb = "0.1"
tokenizers = "0.22"
# ============================================================================
@ -261,8 +284,8 @@ resolver = "2"
# ============================================================================
moka = { version = "0.12", features = ["future"] }
sled = "0.34"
fastembed = "5.8"
lancedb = "0.23"
fastembed = "5.11"
lancedb = "0.26"
arrow = "=56"
# ============================================================================
@ -271,30 +294,38 @@ resolver = "2"
platform-config = { path = "./crates/platform-config" }
platform-nats = { path = "./crates/platform-nats" }
platform-db = { path = "./crates/platform-db" }
service-clients = { path = "./crates/service-clients" }
rag = { path = "./crates/rag" }
mcp-server = { path = "./crates/mcp-server" }
platform-clients = { path = "./crates/service-clients" }
platform-rag = { path = "./crates/rag" }
provisioning-mcp = { path = "./crates/mcp-server" }
ai-service = { path = "./crates/ai-service" }
# ============================================================================
# PROV-ECOSYSTEM (Now members of workspace)
# ============================================================================
daemon-cli = { path = "./prov-ecosystem/crates/daemon-cli" }
machines = { path = "./prov-ecosystem/crates/machines" }
encrypt = { path = "./prov-ecosystem/crates/encrypt" }
backup = { path = "./prov-ecosystem/crates/backup" }
observability = { path = "./prov-ecosystem/crates/observability" }
platform-machines = { path = "./prov-ecosystem/crates/machines" }
platform-encrypt = { path = "./prov-ecosystem/crates/encrypt" }
platform-backup = { path = "./prov-ecosystem/crates/backup" }
platform-observability = { path = "./prov-ecosystem/crates/observability" }
init-servs = { path = "./prov-ecosystem/crates/init-servs" }
# stratum-embeddings and stratum-llm are built in isolated Docker context for RAG
# See: crates/rag/docker/Dockerfile
stratum-embeddings = { path = "./stratumiops/crates/stratum-embeddings", features = ["openai-provider", "ollama-provider", "fastembed-provider", "memory-cache"] }
stratum-llm = { path = "./stratumiops/crates/stratum-llm", features = ["anthropic", "openai", "ollama"] }
# ============================================================================
# ONTOREF PROTOCOL ADOPTION (API catalog surface — Phase 4)
# ============================================================================
ontoref-ontology = { path = "../../ontoref/crates/ontoref-ontology", features = ["derive"] }
ontoref-derive = { path = "../../ontoref/crates/ontoref-derive" }
inventory = "0.3"
# Stratum ecosystem — sourced from canonical stratumiops repo (SurrealDB v3 throughout)
stratum-embeddings = { path = "../../stratumiops/crates/stratum-embeddings", features = ["openai-provider", "ollama-provider", "fastembed-provider", "memory-cache"] }
stratum-llm = { path = "../../stratumiops/crates/stratum-llm", features = ["anthropic", "openai", "ollama"] }
stratum-graph = { path = "../../stratumiops/crates/stratum-graph" }
stratum-state = { path = "../../stratumiops/crates/stratum-state" }
# ============================================================================
# SECRETUMVAULT (Enterprise Secrets Management - canonical source)
# ============================================================================
secretumvault = { path = "../../../Development/secretumvault", features = ["surrealdb-storage", "filesystem", "server", "cedar"] }
secretumvault = { path = "../../secretumvault", features = ["surrealdb-storage", "filesystem", "server", "cedar"] }
# ============================================================================
# WASM/WEB-SPECIFIC DEPENDENCIES

236
README.md
View file

@ -204,7 +204,71 @@ OCI-compliant registry for extension distribution and versioning.
---
### 7. **API Gateway** (`infrastructure/api-gateway/`)
### 7. **ops-keeper** (`crates/ops-keeper/`)
Policy-based operation gate — signs approved operations with Ed25519 keys before forwarding to the control plane.
**Language**: Rust
**Purpose**: Operation approval, policy enforcement, and keeper-signed JWT emission
**Key Features**:
- Glob-based `PolicyDef` matching against op type, image patterns, and target patterns
- `Signer` wraps an Ed25519 key pair; emits compact JWTs (`OpsClaims`) on approval
- `PendingOp` tracking with NATS JetStream durable consumer (`ops.pending.*`)
- `AuditEvent` emission to `ops.audit.*` stream on approval or rejection
- Nickel-driven policy config (`keeper_policy.ncl`)
**Status**: ✅ Active Development
---
### 8. **ops-controller** (`crates/ops-controller/`)
NATS JetStream consumer that processes keeper-approved operations, calls the orchestrator, and enforces idempotency via SurrealDB.
**Language**: Rust
**Purpose**: Durable control plane execution with at-least-once delivery guarantees
**Key Features**:
- Pull consumer on `ops.pending.*` JetStream stream
- Ed25519 JWT verification of keeper-signed claims before dispatch
- Idempotency check via SurrealDB; reconciles stale pending ops on startup
- Orchestrator HTTP dispatch with structured `AckResult` (Ack/Nak/Term)
- Audit emission (`ops.audit.*`) on every terminal outcome
**Status**: ✅ Active Development
**ADR**: ADR-038 (ops control plane design)
---
### 9. **audit-mirror** (`crates/audit-mirror/`)
Sidecar that consumes `ops.audit.*` NATS events and mirrors each event as a signed git commit into a Radicle repository.
**Language**: Rust
**Purpose**: Immutable, content-addressed audit trail via Radicle git storage
**Key Features**:
- NATS JetStream pull consumer on `ops.audit.*`
- JTI deduplication — skips already-committed event IDs via `git log` scan
- `commit_writer` creates signed commits with the audit payload as the blob
- `radicle_publish` announces the repo to the Radicle network after each commit
- Configurable via CLI flags (NATS URL, workspace, Radicle repo path, key path)
**Status**: ✅ Active Development
**ADR**: ADR-038
---
### 10. **API Gateway** (`infrastructure/api-gateway/`)
Unified REST API gateway for external integration.
@ -218,46 +282,113 @@ Unified REST API gateway for external integration.
- Authentication and authorization
- Rate limiting and throttling
- API versioning
- Request validation
- Metrics and monitoring
**Status**: 🔄 Planned
**Endpoints** (Planned):
---
- `/api/v1/servers/*` - Server management
- `/api/v1/taskservs/*` - Task service operations
- `/api/v1/clusters/*` - Cluster operations
- `/api/v1/workflows/*` - Workflow management
### 11. **Extension Registry** (`crates/extension-registry/`)
Registry and catalog for browsing, discovering, and distributing extensions.
**Language**: Rust
**Purpose**: Extension discovery, metadata management, and OCI/Forgejo-backed distribution
**Status**: ✅ Active Development
---
### 8. **Extension Registry** (`extension-registry/`)
### 12. **contract-tests** (`crates/contract-tests/`)
Registry and catalog for browsing and discovering extensions.
G3 contract test suite — verifies semantic equivalence across the CLI↔HTTP↔MCP tier stack.
**Purpose**: Extension discovery and metadata management
**Language**: Rust (test crate)
**Purpose**: Prevent drift between registry, HTTP daemon, and MCP server response shapes
**Key Features**:
- Extension catalog
- Search and filtering
- Version history
- Dependency information
- Documentation links
- Community ratings (future)
- Tier A: direct registry invocation (reference baseline)
- Tier B: axum HTTP server on `127.0.0.1:0` (ephemeral port)
- Tier C: in-process MCP `handle_request`
- Normaliser strips volatile fields (`trace_id`, `timestamp`) — asserts semantic, not byte-for-byte equality
- JSON schema validation against `listing_output_schema` on every tier
**Status**: 🔄 Planned
**Status**: ✅ Active Development
---
### 9. **Provisioning Server** (`provisioning-server/`)
### 13. **ncl-sync** (`crates/ncl-sync/`)
Alternative provisioning service implementation.
Nickel configuration sync daemon — compiles NCL to JSON proactively and maintains a shared cache for all Nu processes.
**Purpose**: Additional provisioning service capabilities
**Language**: Rust
**Status**: 🔄 In Development
**Purpose**: Eliminate `nickel export` latency (~2–5s per call) from CLI commands by pre-compiling NCL files and serving results from an in-memory-backed file cache.
**Key Features**:
- File watcher (`notify`) on workspace NCL directories — re-exports on change automatically
- Warm-up on `prvng platform start` — first command of the day already finds cache hot
- Shared cache at `~/.cache/provisioning/config-cache/` used by both this daemon and `nu_plugin_nickel`
- Content-addressed keys: `SHA256(file_content + sorted_import_paths + format)` — identical to plugin key strategy, zero coordination overhead
- Post-operation sync: Nu writes `.sync-<pid>.json` sidecar after mutations; daemon re-exports within 500 ms
- Configurable via `platform/config/ncl-sync.ncl` (idle timeout, concurrency, poll interval)
- No NATS, no SurrealDB, no platform service dependencies — intentional to avoid bootstrap circularity
**Status**: ✅ Production Ready
**Install**:
```bash
cargo build --release --package ncl-sync
install -m 0755 target/release/provisioning-ncl-sync ~/.local/bin/provisioning-ncl-sync
```
**Usage**:
```bash
# Start daemon for a workspace
ncl-sync daemon --workspace ~/workspaces/libre-daoshi
# One-shot warm-up
ncl-sync warm ~/workspaces/libre-daoshi
# Evict a specific file from cache
ncl-sync invalidate settings.ncl
# Print cache key (parity testing)
ncl-sync key settings.ncl --import-path /ws --import-path /prov
# Cache statistics
ncl-sync stats
```
**Lifecycle integration**: Started automatically by `prvng platform start`, stopped by `prvng platform stop`. Status visible in `prvng platform status`.
**Performance impact** (with warm cache):
| Command | Before | After |
|---------|--------|-------|
| `prvng component list` | ~3–7 s | ~1.5 s |
| `prvng workflow list` | ~3–5 s | ~1.5 s |
| `prvng deploy` | ~15–30 s | ~3–5 s |
**Configuration** (`platform/config/ncl-sync.ncl`):
```nickel
{
ncl_sync = {
idle_timeout_secs = 600, # daemon auto-shutdown after N seconds idle
sync_poll_interval_ms = 500, # how often to check for sync-request sidecars
warm_concurrency = 4, # max parallel nickel export during warm-up
extra_import_paths = [], # additional import paths beyond workspace + $PROVISIONING
}
}
```
**ADRs**: [ADR-022](../adrs/adr-022-ncl-sync-daemon.ncl) (daemon design), [ADR-023](../adrs/adr-023-ncl-export-wrapper.ncl) (Nu wrapper strategy)
---
@ -345,16 +476,27 @@ Systemd service units for platform services.
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Installer │ │ OCI Registry │ │ Extension │ │
│ │ (Rust/Nu) │ │ │ │ Registry │ │
│ │ Installer │ │ Extension │ │ ops-keeper │ │
│ │ (Rust/Nu) │ │ Registry │ │ (Rust) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ops-controller│ │ audit-mirror │ │
│ │ (Rust) │ │ (Rust) │ │
│ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ ncl-sync daemon (Rust) │ │
│ │ ~/.cache/provisioning/config-cache/ ←→ Nu procs │ │
│ └──────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────────────┐
│ Data & State Layer │
│ • SurrealDB (State Management) │
│ • NATS JetStream (ops.pending.*, ops.audit.*, TASKS) │
│ • SurrealDB (State Management, Idempotency) │
│ • Radicle (Immutable Audit Log via git) │
│ • File-based Persistence (Checkpoints) │
│ • Configuration Storage │
└──────────────────────────────────────────────────────────────┘
```
@ -373,11 +515,13 @@ Systemd service units for platform services.
### Key Dependencies
- **tokio** - Async runtime for Rust services
- **axum** / **actix-web** - Web frameworks
- **axum** - Web framework (control-center, orchestrator, provisioning-daemon)
- **async-nats** - NATS JetStream client (ops-keeper, ops-controller, audit-mirror, control-center)
- **surrealdb** - State management and idempotency store
- **serde** - Serialization/deserialization
- **bollard** - Docker API client (test environments)
- **ratatui** - Terminal UI framework (installer)
- **SurrealDB** - State management database
- **git2** - Radicle git integration (audit-mirror)
- **jsonwebtoken** - Ed25519 JWT signing/verification (ops-keeper, ops-controller)
---
@ -477,19 +621,23 @@ nu run.nu
```bash
platform/
├── orchestrator/ # Rust orchestrator service
├── control-center/ # Rust control center backend
├── control-center-ui/ # Web frontend
├── installer/ # Rust/Nushell installer
├── mcp-server/ # Nushell MCP server
├── infrastructure/api-gateway/ # Rust API gateway (planned)
├── infrastructure/oci-registry/ # OCI registry (planned)
├── extension-registry/ # Extension catalog (planned)
├── provisioning-server/# Alternative service
├── infrastructure/docker/ # Docker Compose configs
├── k8s/ # Kubernetes manifests
├── infrastructure/systemd/ # Systemd units
└── docs/ # Platform documentation
├── crates/
│ ├── orchestrator/ # Rust orchestrator service
│ ├── control-center/ # Rust control center backend
│ ├── control-center-ui/ # Web frontend
│ ├── mcp-server/ # Nushell MCP server
│ ├── ncl-sync/ # Nickel config sync daemon
│ └── ...
├── config/
│ ├── ncl-sync.ncl # ncl-sync daemon configuration
│ └── external-services.ncl
├── infrastructure/
│ ├── api-gateway/ # Rust API gateway (planned)
│ ├── oci-registry/ # OCI registry (planned)
│ ├── docker/ # Docker Compose configs
│ ├── systemd/ # Systemd units
│ └── ...
└── docs/ # Platform documentation
```
### Adding New Services
@ -569,5 +717,5 @@ For platform service issues:
---
**Maintained By**: Platform Team
**Last Updated**: 2025-10-07
**Platform Version**: 3.5.0
**Last Updated**: 2026-05-12
**Platform Version**: 3.6.0

11
crates/ai-service/Cargo.toml
View file

@ -20,6 +20,11 @@ axum = { workspace = true }
tower = { workspace = true, features = ["full"] }
tower-http = { workspace = true, features = ["cors", "trace"] }
# Ontoref API catalog
ontoref-ontology = { workspace = true }
ontoref-derive = { workspace = true }
inventory = { workspace = true }
# Serialization
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
@ -29,7 +34,7 @@ toml = { workspace = true }
platform-config = { workspace = true }
# Centralized observability (logging, metrics, health, tracing)
observability = { workspace = true, features = ["logging", "metrics-prometheus", "health"] }
platform-observability = { workspace = true, features = ["logging", "metrics-prometheus", "health"] }
# Error handling
anyhow = { workspace = true }
@ -47,10 +52,10 @@ uuid = { workspace = true, features = ["v4", "serde"] }
clap = { workspace = true, features = ["derive"] }
# RAG crate for AI capabilities
rag = { workspace = true }
platform-rag = { workspace = true }
# MCP server tools for real implementations
mcp-server = { workspace = true }
provisioning-mcp = { workspace = true }
# Graph operations for DAG
petgraph = { workspace = true }

12
crates/ai-service/src/api_catalog.rs Normal file
View file

@ -0,0 +1,12 @@
use axum::{extract::State, response::IntoResponse, Json};
use ontoref_ontology::api::ApiRouteEntry;
use serde_json::json;
use std::sync::Arc;
use crate::service::AiService;
pub async fn api_catalog(State(_state): State<Arc<AiService>>) -> impl IntoResponse {
let mut routes: Vec<&'static ApiRouteEntry> = inventory::iter::<ApiRouteEntry>().collect();
routes.sort_by(|a, b| a.path.cmp(b.path).then(a.method.cmp(b.method)));
Json(json!({ "service": "ai-service", "routes": routes }))
}

52
crates/ai-service/src/handlers.rs
View file

@ -9,6 +9,7 @@ use axum::{
routing::{get, post},
Json, Router,
};
use ontoref_derive::onto_api;
use serde_json::json;
use tracing::debug;
@ -27,10 +28,19 @@ pub fn create_routes(state: Arc<AiService>) -> Router {
get(get_best_practices_handler),
)
.route("/health", get(health_check_handler))
.route("/api/catalog", get(crate::api_catalog::api_catalog))
.with_state(state)
}
/// Call an MCP tool
#[onto_api(
method = "POST",
path = "/api/v1/ai/mcp/tool",
description = "Invoke an MCP tool by name with arguments",
auth = "bearer",
actors = "developer, agent",
tags = "ai, mcp",
feature = ""
)]
async fn call_mcp_tool_handler(
State(service): State<Arc<AiService>>,
Json(req): Json<McpToolRequest>,
@ -45,7 +55,15 @@ async fn call_mcp_tool_handler(
Ok(Json(response))
}
/// Ask AI a question (RAG-powered)
#[onto_api(
method = "POST",
path = "/api/v1/ai/ask",
description = "Ask a RAG-powered question grounded in Nickel schemas and deployment history",
auth = "bearer",
actors = "developer, agent",
tags = "ai, rag",
feature = ""
)]
async fn ask_handler(
State(service): State<Arc<AiService>>,
Json(req): Json<AskRequest>,
@ -57,7 +75,15 @@ async fn ask_handler(
Ok(Json(response))
}
/// Get extension dependency DAG
#[onto_api(
method = "GET",
path = "/api/v1/ai/dag/extensions",
description = "Get the extension dependency DAG used by AI for schema-aware config generation",
auth = "bearer",
actors = "developer, agent",
tags = "ai, dag, extensions",
feature = ""
)]
async fn get_extension_dag_handler(
State(service): State<Arc<AiService>>,
) -> Result<Json<DagResponse>, InternalError> {
@ -71,7 +97,15 @@ async fn get_extension_dag_handler(
Ok(Json(dag))
}
/// Get best practices for a category
#[onto_api(
method = "GET",
path = "/api/v1/ai/knowledge/best-practices",
description = "Get best practices for a given category from the knowledge base",
auth = "bearer",
actors = "developer, agent",
tags = "ai, knowledge",
feature = ""
)]
async fn get_best_practices_handler(
State(service): State<Arc<AiService>>,
axum::extract::Query(params): axum::extract::Query<std::collections::HashMap<String, String>>,
@ -91,7 +125,15 @@ async fn get_best_practices_handler(
Ok(Json(practices))
}
/// Health check endpoint
#[onto_api(
method = "GET",
path = "/health",
description = "AI service health check",
auth = "none",
actors = "developer, agent, ci",
tags = "health",
feature = ""
)]
async fn health_check_handler(
State(service): State<Arc<AiService>>,
) -> Result<StatusCode, InternalError> {

1
crates/ai-service/src/lib.rs
View file

@ -4,6 +4,7 @@
//! Exposes Claude-based question answering, MCP tool execution, extension
//! dependency graphs, and best practice recommendations via HTTP API.
pub mod api_catalog;
pub mod config;
pub mod dag;
pub mod handlers;

10
crates/ai-service/src/main.rs
View file

@ -31,6 +31,11 @@ struct Args {
/// Service bind port
#[arg(short = 'p', long, default_value_t = DEFAULT_PORT)]
port: u16,
/// Print all #[onto_api] registered routes as JSON and exit.
/// Pipe to api-catalog-ai-service.json: `just export-api-catalog`
#[arg(long)]
dump_api_catalog: bool,
}
#[tokio::main]
@ -38,6 +43,11 @@ async fn main() -> anyhow::Result<()> {
// Parse CLI arguments FIRST (so --help works before any other processing)
let args = Args::parse();
if args.dump_api_catalog {
println!("{}", ontoref_ontology::api::dump_catalog_json());
return Ok(());
}
// Initialize centralized observability (logging, metrics, health checks)
let _guard = observability::init_from_env("ai-service", env!("CARGO_PKG_VERSION"))?;

30
crates/audit-mirror/Cargo.toml Normal file
View file

@ -0,0 +1,30 @@
[package]
authors.workspace = true
edition.workspace = true
license.workspace = true
name = "audit-mirror"
version.workspace = true
description = "NATS ops.audit.* → Radicle git commit sidecar with jti idempotency (ADR-038)"
[[bin]]
name = "audit-mirror"
path = "src/main.rs"
[dependencies]
async-nats = { workspace = true }
tokio = { workspace = true, features = ["full"] }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
anyhow = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
chrono = { workspace = true }
clap = { workspace = true }
futures = { workspace = true }
git2 = { workspace = true }
bytes = { workspace = true }
reqwest = { workspace = true }
[dev-dependencies]
tempfile = { workspace = true }

99
crates/audit-mirror/src/commit_writer.rs Normal file
View file

@ -0,0 +1,99 @@
use git2::{Repository, Signature};
use crate::error::MirrorError;
/// Writes a single commit into the HEAD of `repo` embedding `jti` in the message.
///
/// The commit message format is:
/// audit: jti=<jti>
///
/// <json_payload>
///
/// This is the canonical format that `jti_check::already_committed` searches for.
/// An empty tree is used because the audit repo contains only commit-message blobs;
/// no working-tree files are modified.
pub fn write_audit_commit(
repo: &Repository,
jti: &str,
json_payload: &str,
author_name: &str,
author_email: &str,
) -> Result<git2::Oid, MirrorError> {
let sig = Signature::now(author_name, author_email)
.map_err(|e| MirrorError::Git(e.to_string()))?;
let tree_id = {
let mut index = repo.index().map_err(|e| MirrorError::Git(e.to_string()))?;
// Write tree from current index (may be empty — no working-tree blobs needed)
index.write_tree().map_err(|e| MirrorError::Git(e.to_string()))?
};
let tree = repo
.find_tree(tree_id)
.map_err(|e| MirrorError::Git(e.to_string()))?;
let message = format!("audit: jti={jti}\n\n{json_payload}");
// Parent is HEAD if it exists; first commit for an empty (unborn) branch has no parent.
let parent_commit;
let parents: Vec<&git2::Commit<'_>>;
match repo.head() {
Ok(head_ref) => {
parent_commit = head_ref
.peel_to_commit()
.map_err(|e| MirrorError::Git(e.to_string()))?;
parents = vec![&parent_commit];
}
Err(e) if e.code() == git2::ErrorCode::UnbornBranch => {
parents = vec![];
}
Err(e) => return Err(MirrorError::Git(e.to_string())),
}
let oid = repo
.commit(Some("HEAD"), &sig, &sig, &message, &tree, &parents)
.map_err(|e| MirrorError::Git(e.to_string()))?;
Ok(oid)
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::TempDir;
fn init_repo() -> (TempDir, Repository) {
let dir = TempDir::new().unwrap();
let repo = Repository::init(dir.path()).unwrap();
let mut cfg = repo.config().unwrap();
cfg.set_str("user.name", "test").unwrap();
cfg.set_str("user.email", "test@test.com").unwrap();
(dir, repo)
}
#[test]
fn writes_commit_to_empty_repo() {
let (_dir, repo) = init_repo();
let oid = write_audit_commit(&repo, "jti-001", r#"{"event":"test"}"#, "ci", "ci@ci.local").unwrap();
let commit = repo.find_commit(oid).unwrap();
assert!(commit.message().unwrap().contains("jti=jti-001"));
assert!(commit.message().unwrap().contains(r#"{"event":"test"}"#));
}
#[test]
fn writes_second_commit_with_correct_parent() {
let (_dir, repo) = init_repo();
let first = write_audit_commit(&repo, "jti-001", "{}", "ci", "ci@ci.local").unwrap();
let second = write_audit_commit(&repo, "jti-002", "{}", "ci", "ci@ci.local").unwrap();
let commit = repo.find_commit(second).unwrap();
assert_eq!(commit.parent_id(0).unwrap(), first);
}
#[test]
fn message_contains_jti_marker() {
let (_dir, repo) = init_repo();
let oid = write_audit_commit(&repo, "abc-xyz", r#"{"op":"deploy"}"#, "keeper", "keeper@ops").unwrap();
let msg = repo.find_commit(oid).unwrap().message().unwrap().to_string();
assert!(msg.starts_with("audit: jti=abc-xyz\n\n"));
}
}

19
crates/audit-mirror/src/error.rs Normal file
View file

@ -0,0 +1,19 @@
use thiserror::Error;
#[derive(Debug, Error)]
pub enum MirrorError {
#[error("git error: {0}")]
Git(String),
#[error("NATS error: {0}")]
Nats(String),
#[error("json error: {0}")]
Json(#[from] serde_json::Error),
#[error("io error: {0}")]
Io(#[from] std::io::Error),
}
impl From<git2::Error> for MirrorError {
fn from(e: git2::Error) -> Self {
MirrorError::Git(e.to_string())
}
}

92
crates/audit-mirror/src/jti_check.rs Normal file
View file

@ -0,0 +1,92 @@
use git2::Repository;
use crate::error::MirrorError;
/// Returns true if a commit with the given jti already exists in the HEAD ancestor chain.
/// Searches commit messages for the marker `jti=<value>` added by commit_writer.
///
/// This is the idempotency guard required by ADR-038 constraint audit-mirror-idempotent-on-jti.
pub fn already_committed(repo: &Repository, jti: &str) -> Result<bool, MirrorError> {
let head = match repo.head() {
Ok(r) => r,
Err(e) if e.code() == git2::ErrorCode::UnbornBranch => return Ok(false),
Err(e) => return Err(MirrorError::Git(e.to_string())),
};
let head_commit = head
.peel_to_commit()
.map_err(|e| MirrorError::Git(e.to_string()))?;
let marker = format!("jti={jti}");
let mut revwalk = repo
.revwalk()
.map_err(|e| MirrorError::Git(e.to_string()))?;
revwalk
.push(head_commit.id())
.map_err(|e| MirrorError::Git(e.to_string()))?;
revwalk.set_sorting(git2::Sort::TOPOLOGICAL | git2::Sort::TIME)
.map_err(|e| MirrorError::Git(e.to_string()))?;
for oid in revwalk {
let oid = oid.map_err(|e| MirrorError::Git(e.to_string()))?;
let commit = repo
.find_commit(oid)
.map_err(|e| MirrorError::Git(e.to_string()))?;
if commit.message().unwrap_or("").contains(&marker) {
return Ok(true);
}
}
Ok(false)
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::TempDir;
fn init_bare_repo_with_commit(message: &str) -> (TempDir, Repository) {
let dir = TempDir::new().unwrap();
let repo = Repository::init(dir.path()).unwrap();
{
let mut config = repo.config().unwrap();
config.set_str("user.name", "test").unwrap();
config.set_str("user.email", "test@test.com").unwrap();
}
let sig = git2::Signature::now("test", "test@test.com").unwrap();
{
let tree_id = {
let mut index = repo.index().unwrap();
index.write_tree().unwrap()
};
let tree = repo.find_tree(tree_id).unwrap();
repo.commit(Some("HEAD"), &sig, &sig, message, &tree, &[])
.unwrap();
// tree drops here so repo can be moved below
}
(dir, repo)
}
#[test]
fn detects_existing_jti_in_head_commit() {
let (_dir, repo) = init_bare_repo_with_commit("audit: jti=abc-123\n\npayload");
assert!(already_committed(&repo, "abc-123").unwrap());
}
#[test]
fn returns_false_for_unknown_jti() {
let (_dir, repo) = init_bare_repo_with_commit("audit: jti=abc-123\n\npayload");
assert!(!already_committed(&repo, "unknown-jti").unwrap());
}
#[test]
fn returns_false_for_empty_repo() {
let dir = TempDir::new().unwrap();
let repo = Repository::init(dir.path()).unwrap();
assert!(!already_committed(&repo, "any-jti").unwrap());
}
}

188
crates/audit-mirror/src/main.rs Normal file
View file

@ -0,0 +1,188 @@
mod commit_writer;
mod error;
mod jti_check;
mod radicle_publish;
use std::path::PathBuf;
use async_nats::jetstream::{self, consumer::PullConsumer};
use bytes::Bytes;
use clap::Parser;
use futures::StreamExt;
use git2::Repository;
use tracing::{error, info, warn};
use commit_writer::write_audit_commit;
use jti_check::already_committed;
use radicle_publish::announce_repo;
#[derive(Parser)]
#[command(name = "audit-mirror", about = "NATS ops.audit.* → Radicle git commit sidecar (ADR-038)")]
struct Cli {
/// NATS server URL
#[arg(long, default_value = "nats://127.0.0.1:4222")]
nats_url: String,
/// Workspace name (used to derive stream and consumer names)
#[arg(long)]
workspace: String,
/// Path to the local Radicle storage repo for this workspace's state
/// (e.g. ~/.radicle/storage/<rid>)
#[arg(long)]
state_repo_path: PathBuf,
/// Radicle Repository ID (RID) for gossip announce
#[arg(long)]
rid: String,
/// Author name embedded in git commits
#[arg(long, default_value = "audit-mirror")]
author_name: String,
/// Author email embedded in git commits
#[arg(long, default_value = "audit-mirror@ops.local")]
author_email: String,
/// Radicle local HTTP API base URL
#[arg(long, default_value = "http://127.0.0.1:8776")]
radicle_api_url: String,
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
tracing_subscriber::fmt()
.with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
.init();
let cli = Cli::parse();
let nats = async_nats::connect(&cli.nats_url).await?;
let js = jetstream::new(nats);
let stream_name = format!("OPS_AUDIT_{}", cli.workspace.to_uppercase());
let consumer_name = format!("{}-audit-mirror", cli.workspace);
// Ensure stream exists (idempotent — ADR-038 requires the stream be pre-provisioned
// by ops-controller, so we only access it here, not create it).
let stream = js.get_stream(&stream_name).await.map_err(|e| {
format!("stream '{stream_name}' not found — ensure ops-controller has provisioned it: {e}")
})?;
let consumer: PullConsumer = stream
.get_or_create_consumer(
&consumer_name,
jetstream::consumer::pull::Config {
durable_name: Some(consumer_name.clone()),
ack_policy: jetstream::consumer::AckPolicy::Explicit,
filter_subject: format!("ops.audit.{}.>", cli.workspace),
..Default::default()
},
)
.await?;
info!(
stream = stream_name,
consumer = consumer_name,
repo = ?cli.state_repo_path,
"audit-mirror ready"
);
let repo = Repository::open(&cli.state_repo_path).map_err(|e| {
format!("cannot open state repo at {:?}: {e}", cli.state_repo_path)
})?;
let mut messages = consumer.messages().await?;
while let Some(msg_result) = messages.next().await {
let msg = match msg_result {
Ok(m) => m,
Err(e) => {
error!("NATS message error: {e}");
continue;
}
};
let jti = match extract_jti(&msg.payload) {
Some(j) => j,
None => {
warn!(subject = %msg.subject, "no jti in audit payload; acking and skipping");
let _ = msg.ack().await;
continue;
}
};
match already_committed(&repo, &jti) {
Ok(true) => {
info!(jti, "jti already committed — idempotent skip");
let _ = msg.ack().await;
continue;
}
Ok(false) => {}
Err(e) => {
error!(jti, error = %e, "jti_check failed; leaving in queue for retry");
continue;
}
}
let payload_str = String::from_utf8_lossy(&msg.payload);
match write_audit_commit(
&repo,
&jti,
&payload_str,
&cli.author_name,
&cli.author_email,
) {
Ok(oid) => {
info!(jti, commit = %oid, "audit commit written");
}
Err(e) => {
error!(jti, error = %e, "commit_writer failed; leaving in queue for retry");
continue;
}
}
// Best-effort gossip announce — non-fatal, commit is durable.
if let Err(e) = announce_repo(&cli.rid, &cli.radicle_api_url).await {
warn!(jti, error = %e, "announce_repo failed (non-fatal)");
}
if let Err(e) = msg.ack().await {
warn!(jti, error = %e, "ack failed (JetStream will redeliver; jti_check will deduplicate)");
}
}
warn!("NATS message stream ended; audit-mirror exiting");
Ok(())
}
/// Extracts the `jti` field from a JSON payload.
fn extract_jti(payload: &Bytes) -> Option<String> {
let v: serde_json::Value = serde_json::from_slice(payload).ok()?;
v.get("jti")?.as_str().map(str::to_owned)
}
#[cfg(test)]
mod tests {
use super::*;
use bytes::Bytes;
#[test]
fn extracts_jti_from_payload() {
let payload = Bytes::from(r#"{"jti":"abc-123","event":"deploy"}"#);
assert_eq!(extract_jti(&payload), Some("abc-123".to_string()));
}
#[test]
fn returns_none_for_missing_jti() {
let payload = Bytes::from(r#"{"event":"deploy"}"#);
assert_eq!(extract_jti(&payload), None);
}
#[test]
fn returns_none_for_invalid_json() {
let payload = Bytes::from(b"not-json" as &[u8]);
assert_eq!(extract_jti(&payload), None);
}
}

35
crates/audit-mirror/src/radicle_publish.rs Normal file
View file

@ -0,0 +1,35 @@
use tracing::{info, warn};
/// Signals the local Radicle node to announce updated refs for `rid`.
///
/// Radicle Heartwood exposes an HTTP API at `http://127.0.0.1:8776`. After
/// writing a new audit commit to the Radicle storage repo, we POST to the node
/// sync endpoint so gossip propagates immediately rather than waiting for the
/// node's background scan interval.
///
/// Failure here is non-fatal: the commit is already durable in the local git
/// store and radicled will eventually pick it up via its storage-directory
/// watcher. We log a warning and return Ok(()) so the caller can ack the
/// JetStream message.
pub async fn announce_repo(rid: &str, node_api_url: &str) -> Result<(), crate::error::MirrorError> {
let url = format!("{node_api_url}/v1/repos/{rid}/sync");
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(5))
.build()
.map_err(|e| crate::error::MirrorError::Nats(format!("HTTP client init: {e}")))?;
match client.post(&url).send().await {
Ok(resp) if resp.status().is_success() => {
info!(rid, "Radicle repo sync triggered successfully");
}
Ok(resp) => {
warn!(rid, status = %resp.status(), "Radicle sync returned non-success; node will sync on next scan");
}
Err(e) => {
warn!(rid, error = %e, "Radicle sync request failed (non-fatal; local commit is durable)");
}
}
Ok(())
}

39
crates/backup-manager/Cargo.toml Normal file
View file

@ -0,0 +1,39 @@
[package]
authors.workspace = true
edition.workspace = true
license.workspace = true
name = "backup-manager"
repository.workspace = true
version.workspace = true
description = "Multi-context backup orchestrator: restic-first with kopia opt-in, multi-destination replication, ontoref-driven docs. One-shot, daemon, standalone, or coordinator modes. Distinct from platform-backup (../prov-ecosystem/crates/backup) which is a low-level multi-backend library; backup-manager is the orchestrator binary."
keywords = ["backup", "restic", "kopia", "orchestrator", "ontoref"]
categories = ["command-line-utilities", "development-tools"]
[lib]
name = "backup_manager"
path = "src/lib.rs"
[[bin]]
name = "prvng-backup"
path = "src/main.rs"
[dependencies]
platform-config = { workspace = true }
anyhow = { workspace = true }
async-trait = { workspace = true }
chrono = { workspace = true }
clap = { version = "4", features = ["derive", "env"] }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["full"] }
toml = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
uuid = { workspace = true }
[dev-dependencies]
tempfile = { workspace = true }
tokio-test = { workspace = true }

329
crates/backup-manager/src/config.rs Normal file
View file

@ -0,0 +1,329 @@
//! Configuration loading for the backup-manager via [`platform_config::ConfigLoader`].
//!
//! Configuration files are searched along the standard hierarchy
//! (`~/.config/provisioning/platform/config/backup-manager.ncl` on Linux,
//! `~/Library/Application Support/...` on macOS) and merged with environment
//! variable overrides (`BACKUP_*` mapped to JSON paths). Nickel contracts
//! validate the merged configuration BEFORE serde deserialisation, so type
//! errors surface as Nickel error messages, not serde "expected struct field".
//!
//! See ADR-015 (`adrs/adr-015-config-driven-platform-config.ncl`) for the
//! mandate that every platform crate uses this loader.
use std::path::Path;
use platform_config::ConfigLoader;
use serde::{Deserialize, Serialize};
/// Top-level configuration for the backup-manager binary across all four modes.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct BackupManagerConfig {
/// Daemon-mode runtime parameters. Ignored in `one-shot` and `standalone`.
#[serde(default)]
pub daemon: DaemonConfig,
/// Provider configuration (default provider, version pinning).
#[serde(default)]
pub providers: ProvidersConfig,
/// Path (or glob) to the directory holding policy Nickel files.
/// Resolved at runtime by invoking `nickel export` on each match.
#[serde(default)]
pub policies: PoliciesRef,
/// secretumvault client configuration.
#[serde(default)]
pub vault: VaultClientConfig,
/// platform-nats client configuration.
#[serde(default)]
pub nats: NatsClientConfig,
/// Prometheus metrics emission configuration.
#[serde(default)]
pub metrics: MetricsConfig,
}
/// Daemon-mode parameters: priority queue, concurrency, API endpoint.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DaemonConfig {
/// HTTP API listen address (e.g. `127.0.0.1:9099`).
#[serde(default = "default_daemon_http_addr")]
pub http_addr: String,
/// Unix socket path for local CLI communication.
#[serde(default = "default_daemon_unix_socket")]
pub unix_socket: String,
/// Maximum concurrent backup operations across all destinations.
#[serde(default = "default_max_parallel_backups")]
pub max_parallel_backups: usize,
/// Maximum concurrent operations per destination.
#[serde(default = "default_max_parallel_per_destination")]
pub max_parallel_per_destination: usize,
/// Path where the daemon persists its state (queue, last-run timestamps).
#[serde(default = "default_state_path")]
pub state_path: String,
}
impl Default for DaemonConfig {
fn default() -> Self {
Self {
http_addr: default_daemon_http_addr(),
unix_socket: default_daemon_unix_socket(),
max_parallel_backups: default_max_parallel_backups(),
max_parallel_per_destination: default_max_parallel_per_destination(),
state_path: default_state_path(),
}
}
}
fn default_daemon_http_addr() -> String { "127.0.0.1:9099".to_string() }
fn default_daemon_unix_socket() -> String { "/run/prvng-backup.sock".to_string() }
fn default_max_parallel_backups() -> usize { 4 }
fn default_max_parallel_per_destination() -> usize { 2 }
fn default_state_path() -> String { "/var/lib/prvng-backup/state".to_string() }
/// Provider defaults and version pinning.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ProvidersConfig {
/// Provider used when a `BackupPolicy` does not specify one explicitly.
#[serde(default = "default_provider_name")]
pub default: String,
/// Minimum acceptable restic CLI version. The daemon emits a warning if
/// the installed binary is older. Format: `semver` ("0.16.0").
#[serde(default)]
pub restic_min_version: Option<String>,
/// Minimum acceptable kopia CLI version, when kopia is referenced.
#[serde(default)]
pub kopia_min_version: Option<String>,
}
impl Default for ProvidersConfig {
fn default() -> Self {
Self {
default: default_provider_name(),
restic_min_version: None,
kopia_min_version: None,
}
}
}
fn default_provider_name() -> String { "restic".to_string() }
/// Reference to the directory tree where `BackupPolicy` / `BackupGroup` /
/// `SystemBackupDef` Nickel files are committed.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PoliciesRef {
/// Root directory containing component policies (`infra/<workspace>/components/`).
#[serde(default = "default_components_path")]
pub components_path: String,
/// Path to the BackupGroup declarations file.
#[serde(default = "default_groups_path")]
pub groups_path: String,
/// Path to the SystemBackupDef declarations file.
#[serde(default = "default_system_backups_path")]
pub system_backups_path: String,
/// Nickel `--import-path` value used when exporting policies.
#[serde(default = "default_nickel_import_path")]
pub nickel_import_path: String,
}
impl Default for PoliciesRef {
fn default() -> Self {
Self {
components_path: default_components_path(),
groups_path: default_groups_path(),
system_backups_path: default_system_backups_path(),
nickel_import_path: default_nickel_import_path(),
}
}
}
fn default_components_path() -> String {
"infra/libre-wuji/components".to_string()
}
fn default_groups_path() -> String {
"infra/libre-wuji/backup-groups.ncl".to_string()
}
fn default_system_backups_path() -> String {
"infra/libre-wuji/system-backups.ncl".to_string()
}
fn default_nickel_import_path() -> String { "provisioning".to_string() }
/// secretumvault connection parameters.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct VaultClientConfig {
/// Vault HTTP endpoint (e.g. `https://vault.libre-wuji.svc.cluster.local:8200`).
#[serde(default)]
pub endpoint: Option<String>,
/// Path inside vault where the manager's NKey/JWT lives for auth.
#[serde(default)]
pub auth_path: Option<String>,
/// Audit log target (subject prefix or file path).
#[serde(default)]
pub audit_target: Option<String>,
}
/// platform-nats connection parameters (mirrors `NatsConnectionConfig`).
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct NatsClientConfig {
/// NATS URL (e.g. `nats://nats.ops-system.svc:4222`).
#[serde(default)]
pub url: Option<String>,
/// Path inside vault where the NKey seed is stored.
#[serde(default)]
pub nkey_seed_vault_path: Option<String>,
/// Require signed messages on subscribed subjects.
#[serde(default)]
pub require_signed_messages: bool,
}
/// Prometheus metrics configuration.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MetricsConfig {
/// Metrics emission mode. Daemon mode uses HTTP; one-shot uses textfile.
#[serde(default)]
pub mode: MetricsMode,
/// HTTP listen address when `mode = "http"`.
#[serde(default = "default_metrics_http_addr")]
pub http_addr: String,
/// Textfile path when `mode = "textfile"` (consumed by node_exporter).
#[serde(default = "default_metrics_textfile_path")]
pub textfile_path: String,
}
impl Default for MetricsConfig {
fn default() -> Self {
Self {
mode: MetricsMode::default(),
http_addr: default_metrics_http_addr(),
textfile_path: default_metrics_textfile_path(),
}
}
}
/// Metrics emission mode.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, Default, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum MetricsMode {
/// HTTP endpoint scraped by Prometheus (used by daemon mode).
#[default]
Http,
/// Textfile written to a directory scraped by node_exporter (used by one-shot).
Textfile,
}
fn default_metrics_http_addr() -> String { "127.0.0.1:9100".to_string() }
fn default_metrics_textfile_path() -> String {
"/var/lib/node-exporter/textfile_collector/prvng-backup.prom".to_string()
}
impl ConfigLoader for BackupManagerConfig {
fn service_name() -> &'static str {
"backup-manager"
}
fn collect_env_overrides() -> serde_json::Value {
// Map `BACKUP_*` environment variables to nested JSON paths matching
// the structure of `BackupManagerConfig`. Only set keys are included;
// the merge happens inside `platform_config::format::load_config_with_overrides`.
let mut overrides = serde_json::Map::new();
let mut daemon = serde_json::Map::new();
if let Ok(v) = std::env::var("BACKUP_DAEMON_HTTP_ADDR") {
daemon.insert("http_addr".into(), serde_json::Value::String(v));
}
if let Ok(v) = std::env::var("BACKUP_DAEMON_UNIX_SOCKET") {
daemon.insert("unix_socket".into(), serde_json::Value::String(v));
}
if !daemon.is_empty() {
overrides.insert("daemon".into(), serde_json::Value::Object(daemon));
}
let mut vault = serde_json::Map::new();
if let Ok(v) = std::env::var("BACKUP_VAULT_ENDPOINT") {
vault.insert("endpoint".into(), serde_json::Value::String(v));
}
if !vault.is_empty() {
overrides.insert("vault".into(), serde_json::Value::Object(vault));
}
let mut nats = serde_json::Map::new();
if let Ok(v) = std::env::var("BACKUP_NATS_URL") {
nats.insert("url".into(), serde_json::Value::String(v));
}
if !nats.is_empty() {
overrides.insert("nats".into(), serde_json::Value::Object(nats));
}
let mut providers = serde_json::Map::new();
if let Ok(v) = std::env::var("BACKUP_PROVIDER_DEFAULT") {
providers.insert("default".into(), serde_json::Value::String(v));
}
if !providers.is_empty() {
overrides.insert("providers".into(), serde_json::Value::Object(providers));
}
serde_json::Value::Object(overrides)
}
fn from_path<P: AsRef<Path>>(
path: P,
) -> std::result::Result<Self, Box<dyn std::error::Error + Send + Sync>> {
let path = path.as_ref();
let overrides = Self::collect_env_overrides();
let json_value = platform_config::format::load_config_with_overrides(path, &overrides)
.map_err(|e| -> Box<dyn std::error::Error + Send + Sync> { Box::new(e) })?;
serde_json::from_value(json_value).map_err(|e| {
let err_msg = format!(
"Failed to deserialize backup-manager config from {:?}: {}",
path, e
);
Box::new(std::io::Error::new(std::io::ErrorKind::InvalidData, err_msg))
as Box<dyn std::error::Error + Send + Sync>
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn defaults_are_consistent() {
let cfg = BackupManagerConfig::default();
assert_eq!(cfg.daemon.http_addr, "127.0.0.1:9099");
assert_eq!(cfg.providers.default, "restic");
assert_eq!(cfg.metrics.mode, MetricsMode::Http);
assert!(!cfg.nats.require_signed_messages);
}
#[test]
fn service_name_matches_practice_node_artifact_path() {
assert_eq!(BackupManagerConfig::service_name(), "backup-manager");
}
#[test]
fn env_overrides_emit_only_set_keys() {
// Without env vars set, the override map should be empty.
// (The test runs in an isolated process; if BACKUP_* leaks from the
// host shell the assertion is loose: just ensure it's a JSON object.)
let v = BackupManagerConfig::collect_env_overrides();
assert!(v.is_object());
}
}

72
crates/backup-manager/src/error.rs Normal file
View file

@ -0,0 +1,72 @@
//! Error types for the backup-manager crate.
//!
//! All public APIs return [`Result<T>`] which is shorthand for
//! `std::result::Result<T, BackupError>`.
use thiserror::Error;
/// Top-level error type for the backup-manager.
#[derive(Debug, Error)]
pub enum BackupError {
/// Configuration could not be loaded or validated.
#[error("configuration error: {0}")]
Config(String),
/// A policy file could not be deserialised or violated a contract.
#[error("policy error: {0}")]
Policy(String),
/// A backup provider (restic, kopia) failed.
#[error("provider '{provider}' error: {message}")]
Provider {
/// Provider identifier (`restic`, `kopia`, ...).
provider: String,
/// Underlying error message from the provider invocation.
message: String,
},
/// A destination was unreachable or rejected the operation.
#[error("destination '{destination}' error: {message}")]
Destination {
/// Destination identifier (matches `BackupPolicy.destinations[].name`).
destination: String,
/// Underlying error.
message: String,
},
/// The vault (secretumvault) returned an error or was unreachable.
#[error("vault error: {0}")]
Vault(String),
/// NATS publish/subscribe error.
#[error("nats error: {0}")]
Nats(String),
/// A Kubernetes API call failed.
#[error("kubernetes error: {0}")]
Kubernetes(String),
/// Filesystem I/O error.
#[error("io error: {0}")]
Io(#[from] std::io::Error),
/// JSON (de)serialisation error.
#[error("json error: {0}")]
Json(#[from] serde_json::Error),
/// A pre/post hook exited non-zero.
#[error("hook '{hook}' failed: exit {exit_code}")]
HookFailed {
/// Hook identifier (`pre`, `post`, or a named hook).
hook: String,
/// Process exit code.
exit_code: i32,
},
/// Any other anyhow-flavoured failure that does not fit the categories above.
#[error("{0}")]
Other(#[from] anyhow::Error),
}
/// Crate-local result alias.
pub type Result<T> = std::result::Result<T, BackupError>;

60
crates/backup-manager/src/lib.rs Normal file
View file

@ -0,0 +1,60 @@
//! Multi-context backup orchestrator: restic-first with kopia opt-in, multi-destination replication, ontoref-driven docs.
//!
//! `backup-manager` runs in four modes that share a single binary, configuration
//! loader, schema set, and event surface:
//!
//! - **one-shot** — invoked by a scheduler (Kubernetes CronJob, system cron,
//! systemd timer). Loads the policy, executes the requested operation
//! (`backup`, `restore`, `verify`, `prune`, `list`), reports through metrics
//! and NATS events, terminates.
//! - **daemon** — long-running orchestrator. Holds a priority queue, enforces
//! concurrency budgets, reloads policies via [`platform_config`] watcher,
//! exposes a local HTTP/Unix socket API, and emits Prometheus metrics on a
//! dedicated endpoint.
//! - **standalone** — workstation mode. Connects to a destination directly with
//! the operator's age key and runs `mount`, `restore`, or `list` without
//! needing the cluster, the daemon, or even a Nickel runtime (the binary
//! embeds the schema subset required to deserialise policies).
//! - **coordinator** — orchestrates `BackupGroup` consistent cuts across
//! members. Tags every member's snapshot with the same `group_id` and
//! `group_ts` so a single timestamp recovers the whole group.
//!
//! ## Schema integration
//!
//! Policies are declared in Nickel (`provisioning/schemas/lib/backup_policy.ncl`,
//! `backup_group.ncl`, `system_backup.ncl`, `verify_policy.ncl`, `concerns.ncl`).
//! The Nickel layer enforces three invariants at export time:
//!
//! - `EncryptionRequired` — every [`BackupProvider`](policy::BackupProvider)
//! must declare `features.encryption = true`.
//! - `MultiDestinationRequired` — every enabled
//! [`BackupPolicy`](policy::BackupPolicy) must declare ≥2 destinations,
//! at least one with `role = 'primary`.
//! - `NonEmptyScopes` — every enabled `BackupPolicy` must list at least one
//! `BackupScope`.
//!
//! ## Custody model (ADR-011)
//!
//! Four custody boxes, never co-dependent: keys live in `secretumvault`,
//! definitions in git, data in ≥2 destinations, production in the cluster.
//! Standalone restore needs only one custody box plus the bootstrap key.
//!
//! ## ontoref integration
//!
//! This crate is registered as the `backup-manager` Practice node in
//! `.ontology/core.ncl`. `ontoref generate-mdbook` extracts this `//!` and
//! produces architecture documentation automatically. The first sentence above
//! is Jaccard-aligned with the node `description` and validated by
//! `ontoref sync diff --docs`.
#![forbid(unsafe_code)]
#![warn(missing_docs)]
#![warn(rust_2018_idioms)]
pub mod config;
pub mod error;
pub mod modes;
pub mod policy;
pub use config::BackupManagerConfig;
pub use error::{BackupError, Result};

348
crates/backup-manager/src/main.rs Normal file
View file

@ -0,0 +1,348 @@
//! `prvng-backup` — entry point for the backup-manager binary.
//!
//! Subcommands map to the four modes described in the crate-level docs.
//! This first cut implements the `policy` subcommand end-to-end (validate
//! and render Nickel-exported policies); the other modes are stubs whose
//! signatures and CLI surface are wired so the contract is visible to users
//! and downstream code while the implementations land in subsequent commits.
use std::path::{Path, PathBuf};
use std::process::ExitCode;
use clap::{Args, Parser, Subcommand};
use backup_manager::modes::policy_cmds;
/// CLI for the backup-manager.
#[derive(Debug, Parser)]
#[command(
name = "prvng-backup",
version,
about = "Multi-context backup orchestrator (restic/kopia, ontoref-integrated)"
)]
struct Cli {
/// Path to a configuration file. When absent, the standard hierarchy
/// (`~/.config/provisioning/platform/config/backup-manager.ncl`) is used.
#[arg(short, long, env = "BACKUP_CONFIG")]
config: Option<PathBuf>,
/// Subcommand selecting the mode of operation.
#[command(subcommand)]
command: Mode,
}
/// Top-level mode dispatch.
#[derive(Debug, Subcommand)]
enum Mode {
/// Render and validate Nickel policy declarations.
Policy(PolicyArgs),
/// One-shot operation invoked by a scheduler (cron, systemd timer).
OneShot(OneShotArgs),
/// Long-running daemon orchestrator.
Daemon(DaemonArgs),
/// Standalone workstation mode (mount/restore/list without cluster).
Standalone(StandaloneArgs),
/// BackupGroup coordinator.
Coordinator(CoordinatorArgs),
}
// ── policy ──────────────────────────────────────────────────────────────────
#[derive(Debug, Args)]
struct PolicyArgs {
/// Sub-action to perform.
#[command(subcommand)]
action: PolicyAction,
}
#[derive(Debug, Subcommand)]
enum PolicyAction {
/// Validate one or more Nickel policy files (export + deserialise).
Validate {
/// Files or directories to validate.
#[arg(required = true)]
paths: Vec<PathBuf>,
/// Nickel `--import-path` value.
#[arg(long, default_value = "provisioning")]
import_path: String,
},
/// Render a policy file to JSON on stdout.
Render {
/// Policy file to render.
path: PathBuf,
/// Nickel `--import-path` value.
#[arg(long, default_value = "provisioning")]
import_path: String,
},
}
// ── one-shot ────────────────────────────────────────────────────────────────
#[derive(Debug, Args)]
struct OneShotArgs {
/// Operation to perform.
#[command(subcommand)]
op: OneShotOp,
}
#[derive(Debug, Subcommand)]
enum OneShotOp {
/// Run a backup for a component, group, or system target.
Backup {
/// Target identifier.
target: String,
/// Optional scope filter.
#[arg(long)]
scope: Option<String>,
},
/// Restore a snapshot to a directory.
Restore {
/// Target identifier.
target: String,
/// Snapshot ID or `latest`.
#[arg(long)]
snapshot: String,
/// Restore destination directory.
#[arg(long)]
to: PathBuf,
},
/// Verify snapshots (provider-level integrity checks).
Verify {
/// Target identifier.
target: String,
/// Verify level.
#[arg(long, default_value = "quick")]
level: String,
},
/// Apply retention policy and prune old snapshots.
Prune {
/// Target identifier.
target: String,
/// Dry run (do not delete).
#[arg(long)]
dry_run: bool,
},
/// List snapshots.
List {
/// Target identifier.
target: String,
/// Optional scope filter.
#[arg(long)]
scope: Option<String>,
},
}
// ── daemon ──────────────────────────────────────────────────────────────────
#[derive(Debug, Args)]
struct DaemonArgs {
/// Daemon control action.
#[command(subcommand)]
action: DaemonAction,
}
#[derive(Debug, Subcommand)]
enum DaemonAction {
/// Start the daemon (foreground).
Start,
/// Reload policies (sends SIGHUP equivalent through the local API).
Reload,
/// Drain the queue and shut down cleanly.
Drain,
/// Print status as JSON.
Status,
}
// ── standalone ──────────────────────────────────────────────────────────────
#[derive(Debug, Args)]
struct StandaloneArgs {
/// Standalone operation.
#[command(subcommand)]
op: StandaloneOp,
}
#[derive(Debug, Subcommand)]
enum StandaloneOp {
/// Mount a snapshot via FUSE.
Mount {
/// Component name.
component: String,
/// Snapshot ID or `latest`.
#[arg(long)]
snapshot: String,
/// Mountpoint directory.
#[arg(long)]
at: PathBuf,
/// Tag filters (`key=value`).
#[arg(long)]
tag: Vec<String>,
},
/// Restore directly to a directory without going through the daemon.
Restore {
/// Component name.
component: String,
/// Snapshot ID or `latest`.
#[arg(long)]
snapshot: String,
/// Restore destination directory.
#[arg(long)]
to: PathBuf,
},
/// List snapshots (no daemon).
List {
/// Component name.
component: String,
/// Optional scope filter.
#[arg(long)]
scope: Option<String>,
},
}
// ── coordinator ─────────────────────────────────────────────────────────────
#[derive(Debug, Args)]
struct CoordinatorArgs {
/// Coordinator action.
#[command(subcommand)]
action: CoordinatorAction,
}
#[derive(Debug, Subcommand)]
enum CoordinatorAction {
/// Run a BackupGroup.
GroupRun {
/// Group name.
name: String,
},
/// Restore a BackupGroup at a given timestamp.
GroupRestore {
/// Group name.
name: String,
/// RFC3339 timestamp.
#[arg(long)]
at: String,
/// Restore destination directory.
#[arg(long)]
to: PathBuf,
},
}
#[tokio::main]
async fn main() -> ExitCode {
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_env("BACKUP_LOG")
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info")),
)
.init();
let cli = Cli::parse();
match cli.command {
Mode::Policy(args) => run_policy(args),
Mode::OneShot(_) | Mode::Daemon(_) | Mode::Standalone(_) | Mode::Coordinator(_) => {
eprintln!(
"this mode is wired in the CLI surface but its implementation \
lands in a subsequent commit. The `policy` subcommand is fully \
functional today (validate / render Nickel-exported policies)."
);
ExitCode::from(2)
}
}
}
fn run_policy(args: PolicyArgs) -> ExitCode {
match args.action {
PolicyAction::Validate { paths, import_path } => run_policy_validate(&paths, &import_path),
PolicyAction::Render { path, import_path } => run_policy_render(&path, &import_path),
}
}
fn run_policy_validate(paths: &[PathBuf], import_path: &str) -> ExitCode {
let mut total_failures = 0usize;
for path in paths {
total_failures += validate_path(path, import_path);
}
if total_failures == 0 {
ExitCode::SUCCESS
} else {
eprintln!("{} validation failure(s)", total_failures);
ExitCode::from(1)
}
}
fn validate_path(path: &Path, import_path: &str) -> usize {
if path.is_dir() {
validate_directory(path, import_path)
} else {
let v = policy_cmds::validate_policy(path, import_path);
print_validation(&v);
usize::from(!v.ok)
}
}
fn validate_directory(path: &Path, import_path: &str) -> usize {
let entries = match std::fs::read_dir(path) {
Ok(e) => e,
Err(e) => {
eprintln!("could not read directory {:?}: {}", path, e);
return 1;
}
};
entries
.flatten()
.filter_map(|entry| {
let p = entry.path();
if p.extension().is_some_and(|e| e == "ncl") {
Some(p)
} else {
None
}
})
.map(|p| {
let v = policy_cmds::validate_policy(&p, import_path);
print_validation(&v);
usize::from(!v.ok)
})
.sum()
}
fn run_policy_render(path: &Path, import_path: &str) -> ExitCode {
let value = match policy_cmds::nickel_export_to_json(path, import_path) {
Ok(v) => v,
Err(e) => {
eprintln!("{}", e);
return ExitCode::from(1);
}
};
match serde_json::to_string_pretty(&value) {
Ok(rendered) => {
println!("{}", rendered);
ExitCode::SUCCESS
}
Err(e) => {
eprintln!("failed to render JSON: {}", e);
ExitCode::from(1)
}
}
}
fn print_validation(v: &policy_cmds::PolicyValidation) {
if v.ok {
println!(
"OK {} components={} backup-enabled={}",
v.source, v.component_count, v.enabled_backup_components.len()
);
} else {
println!(
"FAIL {} error={}",
v.source,
v.error.as_deref().unwrap_or("unknown")
);
}
}

7
crates/backup-manager/src/modes/mod.rs Normal file
View file

@ -0,0 +1,7 @@
//! Execution modes — `one-shot`, `daemon`, `standalone`, `coordinator`.
//!
//! Each mode is a distinct subcommand of the [`prvng-backup`](../../bin/main.rs)
//! binary. The mode is selected at process start; the configuration loaded
//! from [`crate::BackupManagerConfig`] is shared across all modes.
pub mod policy_cmds;

170
crates/backup-manager/src/modes/policy_cmds.rs Normal file
View file

@ -0,0 +1,170 @@
//! `policy` subcommand — render and validate `BackupPolicy` declarations.
//!
//! This is the first end-to-end mode wired into the binary. It exercises:
//!
//! - Reading workspace declarations via `nickel export --import-path …`.
//! - Deserialising the resulting JSON into [`crate::policy`] structs.
//! - Reporting structured information about policies, groups, and system
//! backup definitions without touching destinations or vault.
//!
//! Subsequent modes (`backup`, `restore`, `mount`, etc.) build on the same
//! deserialisation pipeline implemented here.
use std::path::Path;
use std::process::Command;
use crate::error::{BackupError, Result};
use crate::policy::BackupPolicy;
/// Render a Nickel file to JSON via the `nickel` CLI.
///
/// Honours the `nickel_import_path` provided in [`crate::config::PoliciesRef`].
pub fn nickel_export_to_json(path: &Path, import_path: &str) -> Result<serde_json::Value> {
let output = Command::new("nickel")
.args(["export", "--format", "json", "--import-path", import_path])
.arg(path)
.output()
.map_err(|e| BackupError::Policy(format!(
"failed to invoke `nickel export` on {:?}: {}", path, e
)))?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(BackupError::Policy(format!(
"nickel export failed for {:?}: {}", path, stderr
)));
}
serde_json::from_slice::<serde_json::Value>(&output.stdout).map_err(BackupError::from)
}
/// Result of validating a single policy file.
#[derive(Debug)]
pub struct PolicyValidation {
/// Path to the source Nickel file.
pub source: String,
/// Whether the file successfully exported and deserialised.
pub ok: bool,
/// Error message if validation failed.
pub error: Option<String>,
/// Component count (top-level keys in the JSON object).
pub component_count: usize,
/// Names of components for which `concerns.backup.kind == "enabled"`.
pub enabled_backup_components: Vec<String>,
}
/// Validate a single Nickel policy file by exporting and deserialising it.
pub fn validate_policy(path: &Path, import_path: &str) -> PolicyValidation {
let source = path.display().to_string();
let json = match nickel_export_to_json(path, import_path) {
Ok(v) => v,
Err(e) => {
return PolicyValidation {
source,
ok: false,
error: Some(e.to_string()),
component_count: 0,
enabled_backup_components: Vec::new(),
};
}
};
let object = match json.as_object() {
Some(map) => map,
None => {
return PolicyValidation {
source,
ok: false,
error: Some("policy file did not export a JSON object".to_string()),
component_count: 0,
enabled_backup_components: Vec::new(),
};
}
};
let mut enabled = Vec::new();
for (component_name, component_value) in object {
// Best-effort: locate concerns.backup.kind inside the component.
// Components produced by `ext.make_X` helpers may nest the policy in
// different ways; we don't fail on missing concerns here, we just
// skip when not found. Strict enforcement is the Nickel layer's job.
if let Some(kind) = component_value
.pointer("/concerns/backup/kind")
.and_then(serde_json::Value::as_str)
{
if kind == "enabled" {
enabled.push(component_name.clone());
}
}
}
PolicyValidation {
source,
ok: true,
error: None,
component_count: object.len(),
enabled_backup_components: enabled,
}
}
/// Try to deserialise a `BackupPolicy` directly from its JSON representation.
/// Used by tests and by future scope-expansion code; not exposed via CLI yet.
pub fn deserialize_backup_policy(value: serde_json::Value) -> Result<BackupPolicy> {
serde_json::from_value(value).map_err(BackupError::from)
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn deserialize_minimal_backup_policy() {
let value = json!({
"provider": { "name": "restic" },
"destinations": [
{
"name": "primary",
"kind": "s3",
"uri": "s3:fsn1.hetzner/test",
"cred_ref": { "path": "creds/test", "kind": "s3" },
"role": "primary"
},
{
"name": "replica",
"kind": "b2",
"uri": "b2:test-replica",
"cred_ref": { "path": "creds/replica", "kind": "b2" },
"role": "replica"
}
],
"encryption": {
"path": "backup-manager/master-key",
"algorithm": "age_x25519"
},
"schedule": { "kind": "cron", "cron_expr": "0 2 * * *" },
"retention": {
"keep_last": 7,
"keep_daily": 7,
"keep_weekly": 4,
"keep_monthly": 6,
"keep_yearly": 0
},
"scopes": [
{
"kind": "service_full",
"name": "main",
"paths": ["/data"]
}
],
"tag_strategy": {
"component_label": "test-component"
}
});
let policy = deserialize_backup_policy(value).expect("deserialise");
assert_eq!(policy.destinations.len(), 2);
assert_eq!(policy.scopes.len(), 1);
assert_eq!(policy.provider.name, "restic");
}
}

324
crates/backup-manager/src/policy/component.rs Normal file
View file

@ -0,0 +1,324 @@
//! Component-level policy: `BackupPolicy`, `Destination`, `RetentionPolicy`,
//! `Schedule`, `BackupProviderRef`, plus the surrounding `ServiceConcerns`
//! umbrella.
use std::collections::BTreeMap;
use serde::{Deserialize, Serialize};
use super::scope::BackupScope;
// ============================================================================
// ServiceConcerns umbrella (ADR-008)
// ============================================================================
/// One of `enabled`, `disabled`, `pending`, `inherited`.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum ConcernStateKind {
/// Concern is implemented; the impl payload field is populated.
Enabled,
/// Concern is explicitly opted out.
Disabled,
/// Concern is deferred; `backlog_ref` points to the tracking item.
Pending,
/// Concern is inherited from a parent component.
Inherited,
}
/// State of one concern within [`ServiceConcerns`].
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ConcernState {
/// Discriminator.
pub kind: ConcernStateKind,
/// `disabled` reason.
#[serde(default)]
pub reason: Option<String>,
/// `disabled` since (ISO date).
#[serde(default)]
pub since: Option<String>,
/// `pending` backlog reference.
#[serde(default)]
pub backlog_ref: Option<String>,
/// `pending` target iteration.
#[serde(default)]
pub target_iteration: Option<String>,
/// `inherited` parent component.
#[serde(default)]
pub from: Option<String>,
/// `enabled` payload — only one of these is populated based on which
/// concern this state belongs to.
#[serde(default)]
pub backup_impl: Option<BackupPolicy>,
/// `enabled` payload for non-backup concerns kept opaque at this layer
/// (TLS/DNS/certs/observability/security details are not consumed by the
/// backup-manager binary).
#[serde(default, flatten)]
pub other_impl: BTreeMap<String, serde_json::Value>,
}
/// Mandatory umbrella in `ComponentDef.concerns`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ServiceConcerns {
/// TLS concern state (opaque to the backup binary).
pub tls: ConcernState,
/// DNS concern state.
pub dns: ConcernState,
/// Certificates concern state.
pub certs: ConcernState,
/// Backup concern state — when `kind = Enabled`, `backup_impl` is set.
pub backup: ConcernState,
/// Observability concern state.
pub observability: ConcernState,
/// Security concern state.
pub security: ConcernState,
}
// ============================================================================
// BackupPolicy
// ============================================================================
/// Reference to a [`BackupProvider`] declared under
/// `provisioning/extensions/providers/backup/<name>/`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BackupProviderRef {
/// Provider directory name (e.g. `"restic"`, `"kopia"`).
pub name: String,
/// Pinned CLI version. The daemon emits a warning if the installed
/// binary is older.
#[serde(default)]
pub version: Option<String>,
}
/// Schedule discriminated union (cron / interval / NATS-event).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ScheduleSpec {
/// `cron`, `interval`, or `on_event`.
pub kind: String,
/// Cron expression when `kind = "cron"`.
#[serde(default)]
pub cron_expr: Option<String>,
/// Jitter seconds applied to cron firing.
#[serde(default)]
pub jitter_sec: Option<u64>,
/// Period when `kind = "interval"`.
#[serde(default)]
pub every: Option<String>,
/// Random jitter for interval scheduling.
#[serde(default)]
pub jitter: Option<String>,
/// NATS subject when `kind = "on_event"`.
#[serde(default)]
pub subject: Option<String>,
/// Debounce duration for event-driven schedules.
#[serde(default)]
pub debounce: Option<String>,
}
/// Retention preset. Mirrors restic's `--keep-*` flags.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct RetentionPolicy {
/// `--keep-last`.
#[serde(default)]
pub keep_last: u32,
/// `--keep-daily`.
#[serde(default)]
pub keep_daily: u32,
/// `--keep-weekly`.
#[serde(default)]
pub keep_weekly: u32,
/// `--keep-monthly`.
#[serde(default)]
pub keep_monthly: u32,
/// `--keep-yearly`.
#[serde(default)]
pub keep_yearly: u32,
/// Hard upper bound on snapshot age (deleted regardless of keep_*).
#[serde(default)]
pub prune_after: Option<String>,
}
/// Destination protocol kind.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum DestinationKind {
/// S3-compatible object storage (Hetzner, MinIO, AWS S3, etc.).
S3,
/// Backblaze B2.
B2,
/// Local filesystem (only valid for archive role).
Local,
/// SFTP target.
Sftp,
/// restic REST server.
RestServer,
}
/// Role of a destination within a policy. The `MultiDestinationRequired`
/// Nickel contract requires at least one `Primary` and a total of ≥2 entries.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum DestinationRole {
/// First-write target. Backups land here directly.
Primary,
/// Replicated copy of `Primary` for off-site durability.
Replica,
/// Cold archive (typically slower or cheaper storage).
Archive,
}
/// Backup destination specification.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Destination {
/// Stable identifier used in metric labels and tags.
pub name: String,
/// Protocol kind.
pub kind: DestinationKind,
/// Provider URI (e.g. `s3:fsn1.hetzner/libre-wuji-backups`).
pub uri: String,
/// Reference to credentials in `secretumvault`.
pub cred_ref: VaultCredRef,
/// Role within the policy.
pub role: DestinationRole,
/// Region (for cloud providers that support it).
#[serde(default)]
pub region: Option<String>,
}
/// Reference to a credentials entry in vault.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VaultCredRef {
/// Path under vault (e.g. `backup-manager/destinations/hetzner-primary`).
pub path: String,
/// Type of credential payload at the path.
pub kind: String,
}
/// Reference to an encryption key in vault.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VaultKeyRef {
/// Path under vault.
pub path: String,
/// Algorithm (`age_x25519`, `aes_gcm_256`, etc.).
#[serde(default)]
pub algorithm: String,
/// HKDF derivation parameters when applicable.
#[serde(default)]
pub derivation: Option<HkdfDerivation>,
}
/// HKDF derivation parameters.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HkdfDerivation {
/// Method (`none` or `hkdf_sha256`).
pub method: String,
/// HKDF info parameter.
pub info: String,
}
/// Tag generation strategy for snapshots.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TagStrategy {
/// Used as the `component=<value>` tag.
pub component_label: String,
/// Additional static tags as `k=v` strings.
#[serde(default)]
pub extra: Vec<String>,
}
/// Pre/post hooks executed by the manager around the backup run.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct Hooks {
/// Commands run before backup; non-zero exit aborts when `abort_on_failure = true`.
#[serde(default)]
pub pre: Vec<String>,
/// Commands run after backup, regardless of outcome.
#[serde(default)]
pub post: Vec<String>,
/// Hook timeout (e.g. `"5m"`).
#[serde(default = "default_hook_timeout")]
pub timeout: String,
/// Whether a non-zero pre hook aborts the backup.
#[serde(default = "default_abort_on_failure")]
pub abort_on_failure: bool,
}
fn default_hook_timeout() -> String { "5m".to_string() }
fn default_abort_on_failure() -> bool { true }
/// Bandwidth throttle (passed to provider as `--limit-upload`/`--limit-download`).
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct Throttle {
/// Upload limit in KB/s.
#[serde(default)]
pub upload_kbps: Option<u32>,
/// Download limit in KB/s.
#[serde(default)]
pub download_kbps: Option<u32>,
}
/// Reference to a verify policy (drill spec lives separately to keep policies
/// focused).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VerifyPolicyRef {
/// Verify schedule.
#[serde(default)]
pub schedule: Option<ScheduleSpec>,
/// Level: `quick`, `deep`, `restore_drill`, `full_dr`.
#[serde(default = "default_verify_level")]
pub level: String,
/// Reference to a `DrillSpec` by name.
#[serde(default)]
pub drill_ref: Option<String>,
}
fn default_verify_level() -> String { "quick".to_string() }
/// Component-level backup policy.
///
/// The Nickel contracts `EncryptionRequired`, `MultiDestinationRequired`, and
/// `NonEmptyScopes` are enforced at policy export time, so values reaching
/// this struct already satisfy the invariants.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BackupPolicy {
/// Provider reference.
pub provider: BackupProviderRef,
/// ≥2 destinations, at least one with `role = Primary`.
pub destinations: Vec<Destination>,
/// Encryption key reference.
pub encryption: VaultKeyRef,
/// Schedule.
pub schedule: ScheduleSpec,
/// Retention policy.
pub retention: RetentionPolicy,
/// Scopes (≥1).
pub scopes: Vec<BackupScope>,
/// Tag strategy.
pub tag_strategy: TagStrategy,
/// Optional hooks.
#[serde(default)]
pub hooks: Option<Hooks>,
/// Optional verify reference.
#[serde(default)]
pub verify: Option<VerifyPolicyRef>,
/// Optional throttle.
#[serde(default)]
pub throttle: Option<Throttle>,
/// Optional consistency-group name (this policy participates in a `BackupGroup`).
#[serde(default)]
pub consistency_group: Option<String>,
}

68
crates/backup-manager/src/policy/group.rs Normal file
View file

@ -0,0 +1,68 @@
//! BackupGroup — multi-component consistency points.
use serde::{Deserialize, Serialize};
use super::component::{Destination, RetentionPolicy, ScheduleSpec, TagStrategy, VaultKeyRef,
VerifyPolicyRef};
/// Member of a [`BackupGroup`].
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GroupMember {
/// Component name.
pub component: String,
/// Optional scope filter; omitted means all scopes of the component.
#[serde(default)]
pub scope: Option<String>,
}
/// Coordination strategy discriminator.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum CoordinationKind {
/// Snapshots fired in parallel; consistency is tag-based (group_id).
BestEffort,
/// Ordered pre-hooks pause writers, snapshot, unquiesce. Bounded downtime.
QuiesceWindow,
/// Atomic at CSI layer via Longhorn VolumeSnapshotGroup.
CsiConsistentGroup,
}
/// Coordination strategy applied during a group run.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CoordinationStrategy {
/// Variant.
pub kind: CoordinationKind,
/// Quiesce sequence (`QuiesceWindow`).
#[serde(default)]
pub quiesce_seq: Vec<String>,
/// Maximum downtime for `QuiesceWindow`.
#[serde(default)]
pub max_downtime: Option<String>,
/// CSI VolumeSnapshotClass for `CsiConsistentGroup`.
#[serde(default)]
pub snapshot_class: Option<String>,
}
/// A group of components/scopes captured atomically (à la Chandy-Lamport).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BackupGroup {
/// Group identifier (used in CLI: `--group <name>`).
pub name: String,
/// Members participating in the consistent cut.
pub members: Vec<GroupMember>,
/// Group schedule (independent of per-member policies).
pub schedule: ScheduleSpec,
/// Coordination strategy.
pub coordination: CoordinationStrategy,
/// Retention shared across all members for the group's snapshots.
pub retention: RetentionPolicy,
/// Destinations (same `MultiDestinationRequired` invariant).
pub destinations: Vec<Destination>,
/// Encryption key.
pub encryption: VaultKeyRef,
/// Tag strategy.
pub tag_strategy: TagStrategy,
/// Optional verify reference.
#[serde(default)]
pub verify: Option<VerifyPolicyRef>,
}

26
crates/backup-manager/src/policy/mod.rs Normal file
View file

@ -0,0 +1,26 @@
//! Policy types deserialised from Nickel-exported JSON.
//!
//! Nickel files under `provisioning/schemas/lib/` define the canonical schema
//! (BackupPolicy, BackupGroup, SystemBackupDef, ServiceConcerns, etc.).
//! The manager invokes `nickel export` on workspace declarations and parses
//! the resulting JSON into these structs. The JSON tag conventions mirror the
//! Nickel discriminated unions: `kind` field plus payload variants.
//!
//! Only the shape needed at runtime is materialised here; the full Nickel
//! schema includes optional fields (e.g. extended `dns_records.extra`) that
//! we treat as `serde_json::Value` to avoid coupling the binary to every
//! schema evolution.
pub mod component;
pub mod group;
pub mod scope;
pub mod system;
pub use component::{
BackupPolicy, BackupProviderRef, ConcernState, Destination, DestinationKind,
DestinationRole, RetentionPolicy, ScheduleSpec, ServiceConcerns,
};
pub use group::{BackupGroup, CoordinationKind, CoordinationStrategy, GroupMember};
pub use scope::{BackupScope, DumpStrategy, DumpStrategyKind, ScopeKind};
pub use system::{HostSelector, HostSelectorKind, SystemBackupDef, SystemBackupTarget,
SystemBackupTargetKind};

175
crates/backup-manager/src/policy/scope.rs Normal file
View file

@ -0,0 +1,175 @@
//! BackupScope and DumpStrategy types — the unit of backup work.
//!
//! Each scope expands to one provider invocation with deterministic tags.
//! The discriminator is `kind`; the rest of the fields are populated only
//! for variants that use them (Nickel emits `default = []`/`default = {}`).
use std::collections::BTreeMap;
use serde::{Deserialize, Serialize};
/// Discriminator for [`BackupScope`].
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum ScopeKind {
/// Snapshot a fixed list of paths.
ServiceFull,
/// Snapshot per domain (one tag set per domain).
PerDomain,
/// Snapshot per mailbox (selector-driven enumeration).
PerMailbox,
/// Snapshot a database via a [`DumpStrategy`].
Database,
/// Snapshot a CSI volume by name.
VolumeSnapshot,
/// Archive logs from a source (loki / journald / files).
LogsArchive,
/// Export a subset of a key-value store (etcd / consul).
KvExport,
}
/// Database engine, when scope kind is [`ScopeKind::Database`].
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum DbEngine {
/// PostgreSQL — supports streaming pg_dump and WAL archiving externally.
Postgresql,
/// MariaDB.
Mariadb,
/// MySQL.
Mysql,
/// Redis (RDB snapshot or AOF).
Redis,
/// MongoDB (mongodump).
Mongodb,
/// SurrealDB.
Surrealdb,
/// etcd (etcdctl snapshot).
Etcd,
/// SQLite (file-level).
Sqlite,
}
/// Discriminator for [`DumpStrategy`].
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum DumpStrategyKind {
/// Stream the dump straight to the provider's stdin.
StreamToStdin,
/// Write the dump to a path, then snapshot the path.
DumpToPath,
/// Like `DumpToPath` but with a custom dump command.
PreDumpThenPath,
/// CSI VolumeSnapshot CRD (crash-consistent, no app coordination).
CsiVolumeSnapshot,
/// Application quiesce hook + snapshot + unquiesce.
AppQuiesceThenSnapshot,
}
/// Strategy for capturing a database scope. See ADR-013 for the consistency
/// trade-offs across variants.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DumpStrategy {
/// Variant discriminator.
pub kind: DumpStrategyKind,
/// Custom dump command (optional, used by `PreDumpThenPath`).
#[serde(default)]
pub dump_command: Option<String>,
/// Path where the dump file lives (`DumpToPath`, `PreDumpThenPath`).
#[serde(default)]
pub path: Option<String>,
/// Whether to remove the dump file after the snapshot completes.
#[serde(default = "default_true")]
pub cleanup: bool,
/// CSI volume name (`CsiVolumeSnapshot`, `AppQuiesceThenSnapshot`).
#[serde(default)]
pub volume: Option<String>,
/// CSI VolumeSnapshotClass.
#[serde(default)]
pub snapshot_class: Option<String>,
/// Quiesce command (`AppQuiesceThenSnapshot`).
#[serde(default)]
pub quiesce_cmd: Option<String>,
/// Unquiesce command (`AppQuiesceThenSnapshot`).
#[serde(default)]
pub unquiesce_cmd: Option<String>,
}
fn default_true() -> bool { true }
/// One unit of backup work. The provider receives this via [`crate::policy::component::BackupPolicy`].
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BackupScope {
/// Discriminator.
pub kind: ScopeKind,
/// Identifier for the scope (used in CLI: `--scope <name>`).
pub name: String,
/// Path list (`ServiceFull`, `LogsArchive`).
#[serde(default)]
pub paths: Vec<String>,
/// Exclusion globs.
#[serde(default)]
pub exclude: Vec<String>,
/// Domain list (`PerDomain`).
#[serde(default)]
pub domains: Vec<String>,
/// Base path for per-domain / per-mailbox scopes.
#[serde(default)]
pub base_path: String,
/// Mailbox selector (`PerMailbox`).
#[serde(default)]
pub selector: Option<String>,
/// Database engine (`Database`).
#[serde(default)]
pub engine: Option<DbEngine>,
/// Dump strategy (`Database`).
#[serde(default)]
pub dump_strategy: Option<DumpStrategy>,
/// Volume names (`VolumeSnapshot`).
#[serde(default)]
pub volumes: Vec<String>,
/// CSI VolumeSnapshotClass (`VolumeSnapshot`).
#[serde(default)]
pub snapshot_class: Option<String>,
/// Log sources (`LogsArchive`).
#[serde(default)]
pub sources: Vec<String>,
/// Log archive format (`LogsArchive`).
#[serde(default)]
pub format: Option<String>,
/// Rotation duration (`LogsArchive`).
#[serde(default)]
pub rotation: Option<String>,
/// KV source kind (`KvExport`).
#[serde(default)]
pub source: Option<String>,
/// Tag prefix prepended to determinístico tags.
#[serde(default)]
pub tag_prefix: String,
/// Static extra tags (key-value).
#[serde(default)]
pub tags: BTreeMap<String, String>,
}

196
crates/backup-manager/src/policy/system.rs Normal file
View file

@ -0,0 +1,196 @@
//! SystemBackupDef — backup specifications for artefacts outside the cluster.
//!
//! Targets include etcd, k8s certs, host configs, external DNS configurations,
//! builder environment tools, provisioning state, log archives, SOPS/Age key
//! bundles, and vault state. Each target is a discriminated variant; the
//! manager dispatches based on `kind` (see ADR-011 for vault state custody).
use serde::{Deserialize, Serialize};
use super::component::{
BackupProviderRef, Destination, RetentionPolicy, ScheduleSpec, TagStrategy, Throttle,
VaultCredRef, VaultKeyRef, VerifyPolicyRef, Hooks,
};
/// Discriminator for [`HostSelector`].
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum HostSelectorKind {
/// Run on a single control-plane host (any).
CpOnly,
/// Run on the first control-plane host (deterministic by name sort).
CpFirst,
/// Run on all control-plane hosts.
ControlPlanes,
/// Run on all worker hosts.
Workers,
/// Run on every server in the workspace.
AllServers,
/// Run on the listed hosts.
List,
}
/// Selects which hosts execute a system backup. Mirrors the Nickel `HostSelector`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HostSelector {
/// Variant.
pub kind: HostSelectorKind,
/// Hostname list when `kind = List`.
#[serde(default)]
pub members: Vec<String>,
}
/// Discriminator for [`SystemBackupTarget`].
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum SystemBackupTargetKind {
/// etcd snapshot via etcdctl.
Etcd,
/// Kubernetes PKI certificates (`/etc/kubernetes/pki`).
K8sCerts,
/// Cluster resources (Secret/ConfigMap/CRD content).
ClusterResources,
/// Longhorn engine state.
LonghornEngine,
/// Host configuration files.
HostConfigs,
/// External DNS server configuration and zone files.
ExternalDns,
/// Builder environment tools and caches.
BuilderEnv,
/// Provisioning state (definitions, state, locks).
ProvisioningState,
/// Log archive (loki / journald / files).
LogsArchive,
/// SOPS/Age key bundle (separate custody chain).
SopsKeys,
/// secretumvault state (with bootstrap-key encryption per ADR-011).
VaultState,
}
/// Discriminated target description.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SystemBackupTarget {
/// Variant.
pub kind: SystemBackupTargetKind,
// ── 'etcd
/// etcd endpoints.
#[serde(default)]
pub endpoints: Vec<String>,
/// CA reference.
#[serde(default)]
pub ca_ref: Option<VaultCredRef>,
/// Client cert reference.
#[serde(default)]
pub cert_ref: Option<VaultCredRef>,
/// Client key reference.
#[serde(default)]
pub key_ref: Option<VaultCredRef>,
// ── 'k8s_certs / 'host_configs / 'logs_archive (paths)
/// Path list.
#[serde(default)]
pub paths: Vec<String>,
/// Exclusion globs.
#[serde(default)]
pub exclude: Vec<String>,
// ── 'cluster_resources
/// Namespace list.
#[serde(default)]
pub namespaces: Vec<String>,
/// Kind list (`secret`, `certificate`, ...).
#[serde(default)]
pub kinds: Vec<String>,
// ── 'longhorn_engine
/// Component list (`volumes`, `engines`, `replicas`, `settings`).
#[serde(default)]
pub components: Vec<String>,
// ── 'external_dns
/// Source kind (`coredns`, `powerdns`, `unbound`, ...).
#[serde(default)]
pub source_kind: Option<String>,
/// Config file paths.
#[serde(default)]
pub config_paths: Vec<String>,
/// Zone file directories.
#[serde(default)]
pub zones_paths: Vec<String>,
// ── 'builder_env
/// Tool names (informational tags).
#[serde(default)]
pub tools: Vec<String>,
/// Secret names that must accompany the artefact.
#[serde(default)]
pub secrets: Vec<String>,
// ── 'provisioning_state
/// Definitions directory.
#[serde(default)]
pub definitions_path: Option<String>,
/// State directory.
#[serde(default)]
pub state_path: Option<String>,
/// Lock directory.
#[serde(default)]
pub lock_path: Option<String>,
// ── 'logs_archive
/// Loki/journald selector.
#[serde(default)]
pub selector: Option<String>,
/// Archive format.
#[serde(default)]
pub format: Option<String>,
// ── 'sops_keys / 'vault_state
/// Age key file paths.
#[serde(default)]
pub age_keys: Vec<String>,
/// Recipient public keys.
#[serde(default)]
pub recipients: Vec<String>,
/// Vault HTTP endpoint to back up.
#[serde(default)]
pub vault_endpoint: Option<String>,
/// Specific vault paths to capture (omitted = full state).
#[serde(default)]
pub vault_paths: Vec<String>,
}
/// Top-level system backup definition.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SystemBackupDef {
/// Identifier (used in CLI: `prvng-backup one-shot backup <name>`).
pub name: String,
/// Target description.
pub target: SystemBackupTarget,
/// Host selector.
pub host_selector: HostSelector,
/// Provider.
pub provider: BackupProviderRef,
/// Schedule.
pub schedule: ScheduleSpec,
/// Retention.
pub retention: RetentionPolicy,
/// Destinations.
pub destinations: Vec<Destination>,
/// Encryption key. For `VaultState`, ADR-011 requires a bootstrap key
/// stored OUTSIDE vault (offline custody).
pub encryption: VaultKeyRef,
/// Tag strategy.
pub tag_strategy: TagStrategy,
/// Optional verify reference.
#[serde(default)]
pub verify: Option<VerifyPolicyRef>,
/// Optional hooks.
#[serde(default)]
pub hooks: Option<Hooks>,
/// Optional throttle.
#[serde(default)]
pub throttle: Option<Throttle>,
}

21
crates/buildkit-launcher/Cargo.toml Normal file
View file

@ -0,0 +1,21 @@
[package]
authors.workspace = true
description = "Ephemeral buildkit runner launcher — spawns hcloud VMs, runs buildctl, emits metrics (ADR-039)"
edition.workspace = true
name = "buildkit-launcher"
version.workspace = true
[dependencies]
anyhow = { workspace = true }
clap = { workspace = true }
reqwest = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "process", "time", "io-util"] }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
uuid = { workspace = true }
[[bin]]
name = "buildkit-launcher"
path = "src/main.rs"

19
crates/buildkit-launcher/Cargo.workspace.toml Normal file
View file

@ -0,0 +1,19 @@
[workspace]
members = ["crates/buildkit-launcher"]
resolver = "2"
[workspace.package]
authors = ["Jesus Perez <jesus@librecloud.online>"]
edition = "2021"
version = "1.0.11"
[workspace.dependencies]
anyhow = "1.0"
clap = { version = "4.5", features = ["derive", "env"] }
reqwest = { version = "0.13", features = ["json", "rustls"], default-features = false }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
tokio = { version = "1.49", features = ["rt-multi-thread", "process", "time", "io-util", "macros"] }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
uuid = { version = "1.20", features = ["v4", "serde"] }

38
crates/buildkit-launcher/Dockerfile Normal file
View file

@ -0,0 +1,38 @@
# syntax=docker/dockerfile:1.7
# Build context: provisioning/ (pass --local context=. from provisioning/ root)
# Dockerfile path (buildctl): platform/crates/buildkit-launcher/Dockerfile
# Target architecture: aarch64 (CAX/CCX Woodpecker agents are ARM64)
# Cache: --export-cache / --import-cache against zot /cache/buildkit-launcher
FROM rust:bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN rustup target add aarch64-unknown-linux-gnu
WORKDIR /workspace
# Cargo.workspace.toml is a self-contained workspace manifest: it omits all
# path deps that point outside this build context (ontoref, stratumiops, etc.).
# It must define every key that the crate's Cargo.toml inherits via .workspace = true.
COPY platform/crates/buildkit-launcher/Cargo.workspace.toml Cargo.toml
COPY platform/crates/buildkit-launcher/ crates/buildkit-launcher/
RUN --mount=type=cache,target=/root/.cargo/registry,sharing=locked \
--mount=type=cache,target=/root/.cargo/git,sharing=locked \
--mount=type=cache,target=/workspace/target,sharing=locked \
cargo build --release --target aarch64-unknown-linux-gnu \
--package buildkit-launcher && \
cp target/aarch64-unknown-linux-gnu/release/buildkit-launcher /buildkit-launcher-bin
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates openssh-client \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /buildkit-launcher-bin /buildkit-launcher
ENTRYPOINT ["/buildkit-launcher"]

137
crates/buildkit-launcher/src/buildctl_runner.rs Normal file
View file

@ -0,0 +1,137 @@
use anyhow::{Context, Result};
use std::path::Path;
use tokio::process::Command;
use tracing::{debug, info};
pub const OOM_EXIT_CODE: i32 = 137;
pub struct BuildOutput {
pub cpu_secs: f64,
pub mem_peak_mb: f64,
pub duration_secs: f64,
}
pub async fn rsync_context(
ssh_host: &str,
ssh_port: u16,
context_path: &Path,
ssh_key: &Path,
) -> Result<()> {
info!(host = %ssh_host, port = ssh_port, "rsyncing build context");
let status = Command::new("rsync")
.args([
"-az",
"--delete",
"-e",
&format!(
"ssh -p {} -i {} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null",
ssh_port,
ssh_key.display()
),
])
.arg(format!("{}/", context_path.display()))
.arg(format!("root@{}:/build/context/", ssh_host))
.status()
.await
.context("rsync failed to start")?;
if !status.success() {
anyhow::bail!("rsync exited {}", status);
}
Ok(())
}
pub async fn run_buildctl(
ssh_host: &str,
ssh_port: u16,
ssh_key: &Path,
image_ref: &str,
dockerfile: &str,
cache_from: Option<&str>,
cache_to: Option<&str>,
) -> Result<BuildOutput> {
let mut buildctl_args = vec![
"build".to_string(),
"--frontend".to_string(),
"dockerfile.v0".to_string(),
"--opt".to_string(),
format!("filename={}", dockerfile),
"--local".to_string(),
"context=/build/context".to_string(),
"--local".to_string(),
"dockerfile=/build/context".to_string(),
"--output".to_string(),
format!("type=image,name={},push=true", image_ref),
];
if let Some(from) = cache_from {
buildctl_args.push("--import-cache".to_string());
buildctl_args.push(from.to_string());
}
if let Some(to) = cache_to {
buildctl_args.push("--export-cache".to_string());
buildctl_args.push(to.to_string());
}
let remote_cmd = format!("buildctl {}", buildctl_args.join(" "));
debug!(cmd = %remote_cmd, "running buildctl on runner");
let start = std::time::Instant::now();
let output = Command::new("ssh")
.args([
"-p",
&ssh_port.to_string(),
"-i",
&ssh_key.to_string_lossy(),
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
&format!("root@{}", ssh_host),
&remote_cmd,
])
.output()
.await
.context("ssh buildctl failed to start")?;
let duration_secs = start.elapsed().as_secs_f64();
if !output.status.success() {
let exit_code = output.status.code().unwrap_or(-1);
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!(
"buildctl exit={}: {}",
exit_code,
stderr.lines().last().unwrap_or("(no stderr)")
);
}
// Parse /proc/self/status peak memory from buildkitd if available in stdout
// Fallback: estimate from image size (not accurate; real metrics come from cgroups)
let mem_peak_mb = parse_mem_peak_from_output(&String::from_utf8_lossy(&output.stdout))
.unwrap_or(0.0);
// cpu_secs: rough estimate — not from cgroups; orchestrator records real P95 after multiple runs
let cpu_secs = duration_secs;
Ok(BuildOutput { cpu_secs, mem_peak_mb, duration_secs })
}
fn parse_mem_peak_from_output(stdout: &str) -> Option<f64> {
for line in stdout.lines() {
if let Some(val) = line.strip_prefix("MEM_PEAK_MB=") {
return val.parse().ok();
}
}
None
}
pub fn is_oom(err: &anyhow::Error) -> bool {
let msg = err.to_string();
msg.contains(&format!("exit={}", OOM_EXIT_CODE))
|| msg.contains("exit=137")
|| msg.contains("OOM")
|| msg.contains("Killed")
}

188
crates/buildkit-launcher/src/main.rs Normal file
View file

@ -0,0 +1,188 @@
mod buildctl_runner;
mod orchestrator_client;
mod retry;
mod sizing;
use anyhow::{Context, Result};
use clap::Parser;
use orchestrator_client::OrchestratorClient;
use std::path::PathBuf;
use tracing::{error, info, warn};
#[derive(Parser)]
#[command(name = "buildkit-launcher", about = "Ephemeral buildkit runner — ADR-039")]
struct Args {
/// Workspace name (used for P95 lookup and runner labelling)
#[arg(long, env = "BUILDKIT_WORKSPACE")]
workspace: String,
/// Build context directory (must be reachable from this host)
#[arg(long)]
context: PathBuf,
/// Fully-qualified image reference to push after build
#[arg(long)]
image: String,
/// Dockerfile path relative to context
#[arg(long, default_value = "Dockerfile")]
dockerfile: String,
/// Cache-from reference (optional, e.g. type=registry,ref=zot.example/cache)
#[arg(long)]
cache_from: Option<String>,
/// Cache-to reference (optional)
#[arg(long)]
cache_to: Option<String>,
/// SSH key to use when connecting to the runner
#[arg(long, env = "BUILDKIT_SSH_KEY")]
ssh_key: PathBuf,
/// Language hint for size defaults (rust, go, java, etc.)
#[arg(long)]
language: Option<String>,
/// Golden image snapshot ID or name to boot the runner from
#[arg(long, env = "BUILDKIT_RUNNER_IMAGE", default_value = "buildkit-runner-latest")]
runner_image: String,
/// Orchestrator base URL
#[arg(long, env = "ORCHESTRATOR_URL", default_value = "http://localhost:9011")]
orchestrator_url: String,
}
#[tokio::main]
async fn main() -> Result<()> {
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::from_default_env()
.add_directive("buildkit_launcher=info".parse()?),
)
.init();
let args = Args::parse();
let client = OrchestratorClient::new(args.orchestrator_url.clone());
// Resolve runner size
let p95 = client.get_p95(&args.workspace).await.unwrap_or(None);
let (p95_cpu, p95_mem) = p95
.as_ref()
.map(|s| (Some(s.cpu_p95), Some(s.memory_mb_p95)))
.unwrap_or((None, None));
let size = sizing::resolve(
&args.context,
p95_cpu,
p95_mem,
args.language.as_deref(),
)
.context("failed to resolve runner size")?;
info!(
cpu = size.cpu,
memory_gb = size.memory_gb,
workspace = %args.workspace,
"resolved runner size"
);
let result = run_build(&args, &client, size.clone()).await;
match result {
Ok(()) => Ok(()),
Err(e) if buildctl_runner::is_oom(&e) && retry::oom_retry_allowed(0) => {
warn!(
error = %e,
limit = retry::MAX_OOM_RETRIES,
"OOM detected — retrying at next size tier"
);
let retry_size = retry::next_size_tier(&size)
.context("OOM on largest tier — no retry possible")?;
info!(
cpu = retry_size.cpu,
memory_gb = retry_size.memory_gb,
"retrying build at next tier"
);
run_build(&args, &client, retry_size).await
}
Err(e) => Err(e),
}
}
async fn run_build(args: &Args, client: &OrchestratorClient, size: sizing::RunnerSize) -> Result<()> {
let runner = client
.spawn_runner(&size, &args.workspace, &args.runner_image)
.await
.context("failed to spawn runner")?;
info!(
lease_id = %runner.lease_id,
ssh_host = %runner.ssh_host,
ssh_port = runner.ssh_port,
expires_at = %runner.expires_at,
"runner spawned"
);
let destroy = |lease_id: String| {
let client = client.clone();
tokio::spawn(async move {
if let Err(e) = client.destroy_runner(&lease_id).await {
error!(lease_id = %lease_id, error = %e, "failed to destroy runner");
}
})
};
// Rsync build context to runner
if let Err(e) = buildctl_runner::rsync_context(
&runner.ssh_host,
runner.ssh_port,
&args.context,
&args.ssh_key,
)
.await
{
destroy(runner.lease_id.clone());
return Err(e.context("rsync build context failed"));
}
// Run buildctl
let build_result = buildctl_runner::run_buildctl(
&runner.ssh_host,
runner.ssh_port,
&args.ssh_key,
&args.image,
&args.dockerfile,
args.cache_from.as_deref(),
args.cache_to.as_deref(),
)
.await;
// Always destroy the runner, even on build failure
destroy(runner.lease_id.clone());
let output = build_result.context("buildctl failed")?;
info!(
duration_secs = output.duration_secs,
mem_peak_mb = output.mem_peak_mb,
"build complete — recording metrics"
);
if let Err(e) = client
.record_metrics(
&args.workspace,
output.cpu_secs,
output.mem_peak_mb,
output.duration_secs,
)
.await
{
warn!(error = %e, "failed to record build metrics (non-fatal)");
}
Ok(())
}

158
crates/buildkit-launcher/src/orchestrator_client.rs Normal file
View file

@ -0,0 +1,158 @@
use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};
use crate::sizing::RunnerSize;
#[derive(Debug, Serialize)]
pub struct SpawnRequest {
pub cpu: u32,
pub memory_gb: u32,
pub disk_gb: u32,
pub time_budget_min: u32,
pub workspace: String,
pub image: String,
}
#[derive(Debug, Deserialize)]
pub struct SpawnResponse {
pub lease_id: String,
pub ssh_host: String,
pub ssh_port: u16,
pub expires_at: String,
}
#[derive(Debug, Deserialize)]
pub struct P95Stats {
pub cpu_p95: f64,
pub memory_mb_p95: f64,
#[allow(dead_code)]
pub duration_secs_p95: f64,
#[allow(dead_code)]
pub sample_count: u64,
}
#[derive(Debug, Deserialize)]
struct ApiResponse<T> {
success: bool,
data: Option<T>,
error: Option<String>,
}
#[derive(Clone)]
pub struct OrchestratorClient {
base_url: String,
http: reqwest::Client,
}
impl OrchestratorClient {
pub fn new(base_url: String) -> Self {
Self {
base_url,
http: reqwest::Client::new(),
}
}
pub async fn spawn_runner(
&self,
size: &RunnerSize,
workspace: &str,
image: &str,
) -> Result<SpawnResponse> {
let req = SpawnRequest {
cpu: size.cpu,
memory_gb: size.memory_gb,
disk_gb: size.disk_gb,
time_budget_min: size.time_budget_min,
workspace: workspace.to_string(),
image: image.to_string(),
};
let resp: ApiResponse<SpawnResponse> = self
.http
.post(format!("{}/api/v1/vm-pool/spawn", self.base_url))
.json(&req)
.send()
.await
.context("POST /api/v1/vm-pool/spawn failed")?
.error_for_status()
.context("orchestrator returned error for spawn")?
.json()
.await
.context("failed to deserialize SpawnResponse")?;
if !resp.success {
anyhow::bail!(
"spawn_runner: {}",
resp.error.as_deref().unwrap_or("unknown error")
);
}
resp.data.context("spawn_runner: missing data in response")
}
pub async fn destroy_runner(&self, lease_id: &str) -> Result<()> {
let resp = self
.http
.delete(format!("{}/api/v1/vm-pool/{}", self.base_url, lease_id))
.send()
.await
.context("DELETE /api/v1/vm-pool/{lease_id} failed")?;
if !resp.status().is_success() {
anyhow::bail!("destroy_runner HTTP {}", resp.status());
}
Ok(())
}
pub async fn get_p95(&self, workspace: &str) -> Result<Option<P95Stats>> {
let resp = self
.http
.get(format!(
"{}/api/v1/vm-pool/p95/{}",
self.base_url, workspace
))
.send()
.await
.context("GET /api/v1/vm-pool/p95/{workspace} failed")?;
if resp.status() == reqwest::StatusCode::NOT_FOUND {
return Ok(None);
}
let body: ApiResponse<P95Stats> = resp
.error_for_status()
.context("p95 endpoint error")?
.json()
.await
.context("failed to deserialize P95Stats")?;
Ok(body.data)
}
pub async fn record_metrics(
&self,
workspace: &str,
cpu_secs: f64,
mem_peak_mb: f64,
duration_secs: f64,
) -> Result<()> {
let payload = serde_json::json!({
"workspace": workspace,
"cpu_secs": cpu_secs,
"mem_peak_mb": mem_peak_mb,
"duration_secs": duration_secs,
});
let resp = self
.http
.post(format!("{}/api/v1/vm-pool/metrics", self.base_url))
.json(&payload)
.send()
.await
.context("POST /api/v1/vm-pool/metrics failed")?;
if !resp.status().is_success() {
anyhow::bail!("record_metrics HTTP {}", resp.status());
}
Ok(())
}
}

31
crates/buildkit-launcher/src/retry.rs Normal file
View file

@ -0,0 +1,31 @@
use crate::sizing::RunnerSize;
/// Hard bound on OOM auto-retries per build — ADR-039 constraint oom-retry-bounded.
/// Repeated OOM after one retry indicates misconfiguration; surface to developer.
pub const MAX_OOM_RETRIES: u8 = 1;
/// Returns whether an OOM retry is permitted given the count of retries already consumed.
pub fn oom_retry_allowed(retries_used: u8) -> bool {
retries_used < MAX_OOM_RETRIES
}
const SIZE_TIERS: &[(&str, u32, u32)] = &[
("cx22", 2, 4),
("cx32", 4, 8),
("cx42", 8, 16),
("cx52", 16, 32),
];
pub fn next_size_tier(current: &RunnerSize) -> Option<RunnerSize> {
// Find current tier by matching cpu/memory_gb
let current_idx = SIZE_TIERS.iter().position(|(_, cpu, mem)| {
*cpu >= current.cpu && *mem >= current.memory_gb
})?;
SIZE_TIERS.get(current_idx + 1).map(|(_, cpu, mem)| RunnerSize {
cpu: *cpu,
memory_gb: *mem,
disk_gb: current.disk_gb,
time_budget_min: current.time_budget_min,
})
}

92
crates/buildkit-launcher/src/sizing.rs Normal file
View file

@ -0,0 +1,92 @@
use anyhow::Result;
use serde::Deserialize;
use std::path::Path;
#[derive(Debug, Clone)]
pub struct RunnerSize {
pub cpu: u32,
pub memory_gb: u32,
pub disk_gb: u32,
pub time_budget_min: u32,
}
#[derive(Deserialize, Default)]
struct BuildSpec {
cpu: Option<u32>,
memory_gb: Option<u32>,
disk_gb: Option<u32>,
time_budget_min: Option<u32>,
language: Option<String>,
}
impl RunnerSize {
fn language_default(language: &str) -> Self {
match language {
"rust" => Self { cpu: 4, memory_gb: 8, disk_gb: 50, time_budget_min: 60 },
"go" => Self { cpu: 2, memory_gb: 4, disk_gb: 30, time_budget_min: 30 },
"java" | "kotlin" | "scala" => Self { cpu: 4, memory_gb: 8, disk_gb: 40, time_budget_min: 45 },
_ => Self { cpu: 2, memory_gb: 4, disk_gb: 30, time_budget_min: 30 },
}
}
pub fn from_p95(cpu_p95: f64, memory_mb_p95: f64) -> Self {
let cpu = ((cpu_p95 * 1.2).ceil() as u32).max(2);
let memory_gb = (((memory_mb_p95 * 1.2) / 1024.0).ceil() as u32).max(4);
Self { cpu, memory_gb, disk_gb: 30, time_budget_min: 60 }
}
}
fn parse_build_spec(context_path: &Path) -> Option<BuildSpec> {
let spec_path = context_path.join(".build-spec.ncl");
if !spec_path.exists() {
return None;
}
// Extract JSON from nickel export — buildkit-launcher shells out to nickel
let output = std::process::Command::new("nickel")
.args(["export", "--format", "json"])
.arg(&spec_path)
.output()
.ok()?;
if !output.status.success() {
return None;
}
serde_json::from_slice(&output.stdout).ok()
}
pub fn resolve(
context_path: &Path,
p95_cpu: Option<f64>,
p95_mem_mb: Option<f64>,
language_hint: Option<&str>,
) -> Result<RunnerSize> {
let spec = parse_build_spec(context_path).unwrap_or_default();
// Tier 1: explicit declaration in .build-spec.ncl
if spec.cpu.is_some() || spec.memory_gb.is_some() {
let base = language_hint
.or(spec.language.as_deref())
.map(RunnerSize::language_default)
.unwrap_or_else(|| RunnerSize::language_default("default"));
return Ok(RunnerSize {
cpu: spec.cpu.unwrap_or(base.cpu),
memory_gb: spec.memory_gb.unwrap_or(base.memory_gb),
disk_gb: spec.disk_gb.unwrap_or(base.disk_gb),
time_budget_min: spec.time_budget_min.unwrap_or(base.time_budget_min),
});
}
// Tier 2: P95 historical × 1.2
if let (Some(cpu_p95), Some(mem_p95)) = (p95_cpu, p95_mem_mb) {
let mut size = RunnerSize::from_p95(cpu_p95, mem_p95);
if let Some(budget) = spec.time_budget_min {
size.time_budget_min = budget;
}
return Ok(size);
}
// Tier 3: language defaults
let lang = language_hint
.or(spec.language.as_deref())
.unwrap_or("default");
Ok(RunnerSize::language_default(lang))
}

29
crates/contract-tests/Cargo.toml Normal file
View file

@ -0,0 +1,29 @@
[package]
authors.workspace = true
edition.workspace = true
license.workspace = true
name = "contract-tests"
repository.workspace = true
version.workspace = true
description = "G3 — CLI↔HTTP↔MCP contract tests for the provisioning-core tool Registry"
publish = false
[lib]
path = "src/lib.rs"
[dependencies]
provisioning-core = { path = "../provisioning-core" }
provisioning-daemon = { path = "../provisioning-daemon" }
provisioning-mcp = { path = "../mcp-server" }
tokio = { workspace = true, features = ["full"] }
axum = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
anyhow = { workspace = true }
async-trait = { workspace = true }
reqwest = { workspace = true }
jsonschema = "0.28"
[dev-dependencies]
rustls = { version = "0.23", features = ["aws_lc_rs"] }

257
crates/contract-tests/src/lib.rs Normal file
View file

@ -0,0 +1,257 @@
//! G3 — CLI↔HTTP↔MCP contract-test harness.
//!
//! Provides fixture tools, a three-tier dispatcher, and envelope-normalisation
//! helpers so the invariant "same tool + same params → same payload through all
//! three surfaces" can be asserted in an integration test without touching
//! filesystem, orchestrator, or provider CLIs.
use async_trait::async_trait;
use provisioning_core::{
Environment, Registry, ToolError,
protocol::ToolCategory,
tool::{Context, Tool},
};
use serde_json::{Value, json};
use std::sync::Arc;
// ── Fixture tools ─────────────────────────────────────────────────────────────
/// Returns the input params verbatim — schema allows any object.
pub struct EchoTool;
#[async_trait]
impl Tool for EchoTool {
fn name(&self) -> &'static str { "ct_echo" }
fn description(&self) -> &'static str { "Echo the input parameters verbatim" }
fn schema(&self) -> Value {
json!({
"type": "object",
"properties": {
"message": { "type": "string" }
},
"required": ["message"],
"additionalProperties": true
})
}
fn category(&self) -> ToolCategory { ToolCategory::Read }
async fn invoke(&self, params: Value, _ctx: &Context) -> Result<Value, ToolError> {
let message = params.get("message").and_then(|v| v.as_str()).ok_or_else(|| {
ToolError::invalid_param("message", "required string field missing")
})?;
Ok(json!({
"items": [{ "message": message }],
"total": 1
}))
}
}
/// Deterministic three-item listing — same output regardless of params.
pub struct ListingTool;
#[async_trait]
impl Tool for ListingTool {
fn name(&self) -> &'static str { "ct_listing" }
fn description(&self) -> &'static str { "Return a canonical three-item listing" }
fn schema(&self) -> Value {
json!({ "type": "object", "properties": {}, "additionalProperties": false })
}
fn category(&self) -> ToolCategory { ToolCategory::Read }
async fn invoke(&self, _params: Value, _ctx: &Context) -> Result<Value, ToolError> {
Ok(json!({
"items": [
{ "id": "a", "value": 1 },
{ "id": "b", "value": 2 },
{ "id": "c", "value": 3 }
],
"total": 3
}))
}
}
/// Always fails with `InvalidParam` — used to validate error-path contract.
pub struct FailingTool;
#[async_trait]
impl Tool for FailingTool {
fn name(&self) -> &'static str { "ct_fail" }
fn description(&self) -> &'static str { "Always fails with InvalidParam" }
fn schema(&self) -> Value {
json!({ "type": "object", "properties": {}, "additionalProperties": true })
}
fn category(&self) -> ToolCategory { ToolCategory::Read }
async fn invoke(&self, _params: Value, _ctx: &Context) -> Result<Value, ToolError> {
Err(ToolError::invalid_param("expected", "fixture always rejects"))
}
}
/// Build a Registry populated with the three fixture tools.
pub fn make_fixture_registry() -> Registry {
let mut reg = Registry::new();
reg.register(Arc::new(EchoTool)).expect("register echo");
reg.register(Arc::new(ListingTool)).expect("register listing");
reg.register(Arc::new(FailingTool)).expect("register fail");
reg
}
// ── Output-shape schema (the contract) ────────────────────────────────────────
/// JSON Schema that all `Read`-category list-style tools must satisfy.
/// Generated once at build time so each contract-test case can validate
/// without re-parsing.
pub fn listing_output_schema() -> Value {
json!({
"type": "object",
"required": ["items", "total"],
"properties": {
"items": { "type": "array" },
"total": { "type": "integer", "minimum": 0 }
},
"additionalProperties": true
})
}
// ── Three-tier dispatcher ─────────────────────────────────────────────────────
pub struct TierOutcome {
/// Inner payload after envelope unwrap. `None` on error.
pub payload: Option<Value>,
/// Error code on failure, `None` on success. Each tier maps ToolError
/// to its own code namespace — the normaliser strips that difference.
pub error_code: Option<i32>,
/// Error message (trimmed of tier-specific prefixes).
pub error_message: Option<String>,
}
impl TierOutcome {
pub fn ok(payload: Value) -> Self {
Self { payload: Some(payload), error_code: None, error_message: None }
}
pub fn err(code: i32, message: impl Into<String>) -> Self {
Self { payload: None, error_code: Some(code), error_message: Some(message.into()) }
}
}
/// Tier A — direct Registry invocation. The reference tier: what the other
/// two surfaces must agree with after envelope unwrapping.
pub async fn tier_registry(
registry: &Registry,
env: Arc<Environment>,
tool: &str,
params: Value,
) -> TierOutcome {
let ctx = Context::new(env);
match registry.invoke(tool, params, &ctx).await {
Ok(v) => TierOutcome::ok(v),
Err(e) => {
let code = match &e {
ToolError::NotFound(_) => -32001,
ToolError::InvalidParam { .. } => -32602,
ToolError::Unauthorized(_) => -32003,
_ => -32000,
};
TierOutcome::err(code, e.to_string())
}
}
}
/// Tier B — HTTP daemon. The axum router is reused directly on a TcpListener
/// bound to 127.0.0.1:0 for deterministic port isolation.
pub async fn tier_http(base_url: &str, tool: &str, params: Value) -> TierOutcome {
let client = reqwest::Client::new();
let url = format!("{}/api/v1/tools/{}", base_url, tool);
let body = json!({ "params": params });
let resp = match client.post(&url).json(&body).send().await {
Ok(r) => r,
Err(e) => return TierOutcome::err(-32000, format!("http transport: {e}")),
};
let status = resp.status();
let body: Value = match resp.json().await {
Ok(v) => v,
Err(e) => return TierOutcome::err(-32000, format!("http body: {e}")),
};
if status.is_success() {
match body.get("result").cloned() {
Some(v) => TierOutcome::ok(v),
None => TierOutcome::err(-32000, "missing result field in http response"),
}
} else {
let code = body
.get("code")
.and_then(|v| v.as_i64())
.map(|c| c as i32)
.unwrap_or(-32000);
let msg = body
.get("error")
.and_then(|v| v.as_str())
.unwrap_or("<missing error>")
.to_string();
TierOutcome::err(code, msg)
}
}
/// Tier C — MCP surface via McpServer::handle_request.
/// MCP wraps success payloads in `{content: [{type:"text", text: "<json>"}]}`;
/// the unwrap parses the text field back into JSON.
pub async fn tier_mcp(server: &provisioning_mcp_server::registry_server::McpServer, tool: &str, params: Value) -> TierOutcome {
let rpc = json!({
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": { "name": tool, "arguments": params }
});
let resp = server.handle_request(rpc).await;
if let Some(err) = resp.get("error") {
let code = err.get("code").and_then(|v| v.as_i64()).map(|c| c as i32).unwrap_or(-32000);
let msg = err.get("message").and_then(|v| v.as_str()).unwrap_or("<missing>").to_string();
return TierOutcome::err(code, msg);
}
let content = match resp.get("result").and_then(|r| r.get("content")) {
Some(c) => c,
None => return TierOutcome::err(-32000, "mcp response missing result.content"),
};
let text = content
.get(0)
.and_then(|c| c.get("text"))
.and_then(|t| t.as_str());
let text = match text {
Some(t) => t,
None => return TierOutcome::err(-32000, "mcp content[0].text missing"),
};
match serde_json::from_str::<Value>(text) {
Ok(v) => TierOutcome::ok(v),
Err(e) => TierOutcome::err(-32000, format!("mcp text parse: {e}")),
}
}
// ── Payload normalisation ─────────────────────────────────────────────────────
/// Strip fields that are allowed to diverge between tiers: trace ids, timestamps,
/// uuids. The contract is on semantic payload, not byte equality.
pub fn normalise(v: &Value) -> Value {
match v {
Value::Object(map) => {
let mut out = serde_json::Map::new();
for (k, val) in map {
if matches!(k.as_str(), "trace_id" | "span_id" | "timestamp" | "generated_at" | "request_id")
{
continue;
}
out.insert(k.clone(), normalise(val));
}
Value::Object(out)
}
Value::Array(arr) => Value::Array(arr.iter().map(normalise).collect()),
_ => v.clone(),
}
}

194
crates/contract-tests/tests/g3_contract.rs Normal file
View file

@ -0,0 +1,194 @@
//! G3 — CLI↔HTTP↔MCP contract test.
//!
//! For each fixture tool, invoke via:
//! Tier A — Registry directly (reference)
//! Tier B — HTTP daemon (axum::serve on 127.0.0.1:0)
//! Tier C — MCP server (handle_request in-process)
//!
//! Assert:
//! 1. All tiers that succeed produce equal normalised payloads
//! 2. The payload validates against the listing output schema
//! 3. All tiers that fail produce the same error code
//!
//! The normaliser strips volatile fields (trace_id, timestamp, …) so the
//! contract is on semantic equivalence, not byte-for-byte equality.
use contract_tests::{
listing_output_schema, make_fixture_registry, normalise,
tier_http, tier_mcp, tier_registry,
};
use jsonschema::Validator;
use provisioning_mcp_server::registry_server::McpServer;
use provisioning_core::Environment;
use provisioning_daemon::{AppState, http::router};
use serde_json::{Value, json};
use std::sync::{Arc, OnceLock};
// ── Test harness ──────────────────────────────────────────────────────────────
static CRYPTO: OnceLock<()> = OnceLock::new();
fn init_crypto() {
CRYPTO.get_or_init(|| {
let _ = rustls::crypto::aws_lc_rs::default_provider().install_default();
});
}
struct Harness {
env: Arc<Environment>,
registry: Arc<provisioning_core::Registry>,
mcp: McpServer,
http_base: String,
_daemon_task: tokio::task::JoinHandle<()>,
}
async fn boot() -> Harness {
init_crypto();
// Shared fixture registry for all tiers.
let registry = Arc::new(make_fixture_registry());
let env = Arc::new(Environment::default());
// Tier B — bind daemon on ephemeral port.
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.expect("bind");
let addr = listener.local_addr().expect("local_addr");
let http_base = format!("http://{addr}");
// AppState owns its own Arc<Registry>; we build it from a fresh fixture
// registry so the daemon and the reference tier see identical tool sets.
let daemon_registry = make_fixture_registry();
let app = AppState::new(daemon_registry, Environment::default());
let app_router = router(app);
let task = tokio::spawn(async move {
axum::serve(listener, app_router).await.expect("daemon serve");
});
// Wait for the daemon to accept connections (fast — local TCP).
for _ in 0..40 {
if reqwest::get(format!("{}/health", http_base)).await.is_ok() {
break;
}
tokio::time::sleep(std::time::Duration::from_millis(25)).await;
}
// Tier C — in-process MCP server with its own fixture registry.
let mcp = McpServer::new(make_fixture_registry(), Environment::default(), "g3-test", "0");
Harness { env, registry, mcp, http_base, _daemon_task: task }
}
fn validate_listing(v: &Value) {
let schema = listing_output_schema();
let validator = Validator::new(&schema).expect("compile schema");
validator
.validate(v)
.unwrap_or_else(|errors| panic!("listing schema validation failed: {errors}"));
}
// ── Success contract ──────────────────────────────────────────────────────────
#[tokio::test]
async fn listing_tool_agrees_across_three_tiers() {
let h = boot().await;
let params = json!({});
let a = tier_registry(&h.registry, h.env.clone(), "ct_listing", params.clone()).await;
let b = tier_http(&h.http_base, "ct_listing", params.clone()).await;
let c = tier_mcp(&h.mcp, "ct_listing", params.clone()).await;
let a_payload = a.payload.expect("tier A registry must succeed");
let b_payload = b.payload.expect("tier B http must succeed");
let c_payload = c.payload.expect("tier C mcp must succeed");
validate_listing(&a_payload);
validate_listing(&b_payload);
validate_listing(&c_payload);
assert_eq!(normalise(&a_payload), normalise(&b_payload), "A↔B diverge");
assert_eq!(normalise(&a_payload), normalise(&c_payload), "A↔C diverge");
assert_eq!(normalise(&b_payload), normalise(&c_payload), "B↔C diverge");
}
#[tokio::test]
async fn echo_tool_agrees_across_three_tiers() {
let h = boot().await;
let params = json!({ "message": "contract" });
let a = tier_registry(&h.registry, h.env.clone(), "ct_echo", params.clone()).await;
let b = tier_http(&h.http_base, "ct_echo", params.clone()).await;
let c = tier_mcp(&h.mcp, "ct_echo", params.clone()).await;
let a_payload = a.payload.expect("A must succeed");
let b_payload = b.payload.expect("B must succeed");
let c_payload = c.payload.expect("C must succeed");
validate_listing(&a_payload);
validate_listing(&b_payload);
validate_listing(&c_payload);
assert_eq!(normalise(&a_payload), normalise(&b_payload));
assert_eq!(normalise(&a_payload), normalise(&c_payload));
// Exact content check — echo should round-trip the message
assert_eq!(a_payload["items"][0]["message"], "contract");
assert_eq!(b_payload["items"][0]["message"], "contract");
assert_eq!(c_payload["items"][0]["message"], "contract");
}
// ── Failure contract ──────────────────────────────────────────────────────────
#[tokio::test]
async fn invalid_param_error_code_agrees_across_tiers() {
let h = boot().await;
let params = json!({}); // echo requires "message" → InvalidParam
let a = tier_registry(&h.registry, h.env.clone(), "ct_echo", params.clone()).await;
let b = tier_http(&h.http_base, "ct_echo", params.clone()).await;
let c = tier_mcp(&h.mcp, "ct_echo", params.clone()).await;
// All three must classify as InvalidParam → -32602
assert_eq!(a.error_code, Some(-32602), "A: {:?}", a.error_message);
assert_eq!(b.error_code, Some(-32602), "B: {:?}", b.error_message);
assert_eq!(c.error_code, Some(-32602), "C: {:?}", c.error_message);
}
#[tokio::test]
async fn failing_tool_error_code_agrees_across_tiers() {
let h = boot().await;
let params = json!({});
let a = tier_registry(&h.registry, h.env.clone(), "ct_fail", params.clone()).await;
let b = tier_http(&h.http_base, "ct_fail", params.clone()).await;
let c = tier_mcp(&h.mcp, "ct_fail", params.clone()).await;
assert_eq!(a.error_code, Some(-32602));
assert_eq!(b.error_code, Some(-32602));
assert_eq!(c.error_code, Some(-32602));
}
// ── tools/list contract ───────────────────────────────────────────────────────
#[tokio::test]
async fn tools_list_count_agrees_across_surfaces() {
let h = boot().await;
// Registry baseline
let reg_count = h.registry.len();
// HTTP
let http_body: Value = reqwest::get(format!("{}/api/v1/tools", h.http_base))
.await
.expect("http list")
.json()
.await
.expect("parse");
let http_count = http_body["tools"].as_array().expect("tools array").len();
// MCP
let rpc = json!({"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}});
let mcp_resp = h.mcp.handle_request(rpc).await;
let mcp_count = mcp_resp["result"]["tools"].as_array().expect("mcp tools").len();
assert_eq!(reg_count, http_count, "HTTP count diverges from registry");
assert_eq!(reg_count, mcp_count, "MCP count diverges from registry");
}

13
crates/control-center-ui/src/api/orchestrator_client.rs
View file

@ -224,6 +224,19 @@ impl OrchestratorClient {
}
}
/// Submit a unified component lifecycle operation.
pub async fn deploy_component(&self, operation: &str, workflow: &ComponentWorkflow) -> Result<String, OrchestratorError> {
let path = format!("/api/v1/workflows/component/{operation}");
let request = self.build_request("POST", &path)?;
let response = self.execute_json_request::<String, _>(request, workflow).await?;
if response.success {
Ok(response.data.unwrap_or_default())
} else {
Err(OrchestratorError::Api(response.error.unwrap_or_default()))
}
}
/// Create cluster workflow
pub async fn create_cluster_workflow(&self, workflow: &ClusterWorkflow) -> Result<String, OrchestratorError> {
let request = self.build_request("POST", "/workflows/cluster/create")?;

24
crates/control-center-ui/src/api/orchestrator_types.rs
View file

@ -84,6 +84,30 @@ pub struct ClusterWorkflow {
pub wait: bool,
}
/// Unified component lifecycle workflow request.
/// Submitted to `/api/v1/workflows/component/{op}`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ComponentWorkflow {
pub workspace: String,
pub infra: String,
pub component: String,
pub server: String,
#[serde(default)]
pub namespace: Option<String>,
#[serde(default = "default_ssh_user")]
pub ssh_user: String,
#[serde(default)]
pub ssh_key_path: Option<String>,
pub settings: String,
#[serde(default)]
pub check_mode: bool,
pub provisioning: String,
}
fn default_ssh_user() -> String {
"root".to_string()
}
/// Batch operation request
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BatchOperationRequest {

9
crates/control-center/Cargo.toml
View file

@ -18,6 +18,11 @@ tokio = { workspace = true }
# Web server and API
axum = { workspace = true }
hyper = { workspace = true }
# Ontoref API catalog
ontoref-ontology = { workspace = true }
ontoref-derive = { workspace = true }
inventory = { workspace = true }
tower = { workspace = true }
tower-http = { workspace = true }
@ -40,7 +45,7 @@ clap = { workspace = true }
config = { workspace = true }
# Centralized observability (logging, metrics, health, tracing)
observability = { workspace = true, features = ["logging", "metrics-prometheus", "health"] }
platform-observability = { workspace = true, features = ["logging", "metrics-prometheus", "health"] }
# Error handling
anyhow = { workspace = true }
@ -58,7 +63,7 @@ validator = { workspace = true }
reqwest = { workspace = true }
# HTTP service clients (machines, init, AI) - enables remote service calls
service-clients = { workspace = true }
platform-clients = { workspace = true }
# Platform configuration management
platform-config = { workspace = true }

12
crates/control-center/src/api_catalog.rs Normal file
View file

@ -0,0 +1,12 @@
use std::sync::Arc;
use axum::{extract::State, response::IntoResponse, Json};
use control_center::AppState;
use ontoref_ontology::api::ApiRouteEntry;
use serde_json::json;
pub async fn api_catalog(State(_state): State<Arc<AppState>>) -> impl IntoResponse {
let mut routes: Vec<&'static ApiRouteEntry> = inventory::iter::<ApiRouteEntry>().collect();
routes.sort_by(|a, b| a.path.cmp(b.path).then(a.method.cmp(b.method)));
Json(json!({ "service": "control-center", "routes": routes }))
}

391
crates/control-center/src/error.rs.old
View file

@ -1,391 +0,0 @@
use axum::{
http::StatusCode,
response::{IntoResponse, Response},
Json,
};
use thiserror::Error;
use tracing::error;
/// Result type alias for the control center
pub type Result<T> = std::result::Result<T, ControlCenterError>;
/// Main error type for the control center service
#[derive(Error, Debug)]
pub enum ControlCenterError {
/// Database-related errors
#[error("Database error: {0}")]
Database(String),
/// Authentication errors
#[error("Authentication error: {0}")]
Authentication(String),
/// Authorization errors
#[error("Authorization error: {0}")]
Authorization(String),
/// JWT token errors
#[error("Token error: {0}")]
Token(String),
/// Validation errors
#[error("Validation error: {0}")]
Validation(String),
/// User not found
#[error("User not found: {0}")]
UserNotFound(String),
/// Role not found
#[error("Role not found: {0}")]
RoleNotFound(String),
/// Permission not found
#[error("Permission not found: {0}")]
PermissionNotFound(String),
/// Session not found
#[error("Session not found: {0}")]
SessionNotFound(String),
/// Session expired
#[error("Session expired: {0}")]
SessionExpired(String),
/// Resource not found
#[error("Not found: {0}")]
NotFound(String),
/// Unauthorized access
#[error("Unauthorized: {0}")]
Unauthorized(String),
/// Forbidden access
#[error("Forbidden: {0}")]
Forbidden(String),
/// Conflict (e.g., duplicate resources)
#[error("Conflict: {0}")]
Conflict(String),
/// Bad request
#[error("Bad request: {0}")]
BadRequest(String),
/// Internal server error
#[error("Internal server error: {0}")]
Internal(String),
/// Configuration errors
#[error("Configuration error: {0}")]
Configuration(String),
/// External service errors
#[error("External service error: {0}")]
ExternalService(String),
/// Rate limit exceeded
#[error("Rate limit exceeded: {0}")]
RateLimit(String),
/// WebSocket errors
#[error("WebSocket error: {0}")]
WebSocket(String),
/// Serialization errors
#[error("Serialization error: {0}")]
Serialization(String),
/// Policy evaluation error
#[error("Policy evaluation error: {0}")]
PolicyEvaluation(String),
/// Policy parsing error
#[error("Policy parsing error: {0}")]
PolicyParsing(String),
/// Cedar policy error
#[error("Cedar policy error: {0}")]
Cedar(String),
/// Compliance check failed
#[error("Compliance check failed: {0}")]
Compliance(String),
/// Anomaly detection error
#[error("Anomaly detection error: {0}")]
Anomaly(String),
/// Analysis failed error
#[error("Analysis failed: {0}")]
AnalysisFailed(String),
/// Generic anyhow errors
#[error(transparent)]
Other(#[from] anyhow::Error),
}
impl ControlCenterError {
/// Get the HTTP status code for this error
pub fn status_code(&self) -> StatusCode {
match self {
Self::Database(_) => StatusCode::INTERNAL_SERVER_ERROR,
Self::Authentication(_) => StatusCode::UNAUTHORIZED,
Self::Authorization(_) => StatusCode::FORBIDDEN,
Self::Token(_) => StatusCode::UNAUTHORIZED,
Self::Validation(_) => StatusCode::BAD_REQUEST,
Self::UserNotFound(_) => StatusCode::NOT_FOUND,
Self::RoleNotFound(_) => StatusCode::NOT_FOUND,
Self::PermissionNotFound(_) => StatusCode::NOT_FOUND,
Self::SessionNotFound(_) => StatusCode::NOT_FOUND,
Self::SessionExpired(_) => StatusCode::UNAUTHORIZED,
Self::NotFound(_) => StatusCode::NOT_FOUND,
Self::Unauthorized(_) => StatusCode::UNAUTHORIZED,
Self::Forbidden(_) => StatusCode::FORBIDDEN,
Self::Conflict(_) => StatusCode::CONFLICT,
Self::BadRequest(_) => StatusCode::BAD_REQUEST,
Self::Internal(_) => StatusCode::INTERNAL_SERVER_ERROR,
Self::Configuration(_) => StatusCode::INTERNAL_SERVER_ERROR,
Self::ExternalService(_) => StatusCode::BAD_GATEWAY,
Self::RateLimit(_) => StatusCode::TOO_MANY_REQUESTS,
Self::WebSocket(_) => StatusCode::BAD_REQUEST,
Self::Serialization(_) => StatusCode::INTERNAL_SERVER_ERROR,
Self::PolicyEvaluation(_) => StatusCode::INTERNAL_SERVER_ERROR,
Self::PolicyParsing(_) => StatusCode::BAD_REQUEST,
Self::Cedar(_) => StatusCode::INTERNAL_SERVER_ERROR,
Self::Compliance(_) => StatusCode::FORBIDDEN,
Self::Anomaly(_) => StatusCode::INTERNAL_SERVER_ERROR,
Self::AnalysisFailed(_) => StatusCode::BAD_REQUEST,
Self::Other(_) => StatusCode::INTERNAL_SERVER_ERROR,
}
}
/// Get the error code string for API responses
pub fn error_code(&self) -> &'static str {
match self {
Self::Database(_) => "DATABASE_ERROR",
Self::Authentication(_) => "AUTHENTICATION_ERROR",
Self::Authorization(_) => "AUTHORIZATION_ERROR",
Self::Token(_) => "TOKEN_ERROR",
Self::Validation(_) => "VALIDATION_ERROR",
Self::UserNotFound(_) => "USER_NOT_FOUND",
Self::RoleNotFound(_) => "ROLE_NOT_FOUND",
Self::PermissionNotFound(_) => "PERMISSION_NOT_FOUND",
Self::SessionNotFound(_) => "SESSION_NOT_FOUND",
Self::SessionExpired(_) => "SESSION_EXPIRED",
Self::NotFound(_) => "NOT_FOUND",
Self::Unauthorized(_) => "UNAUTHORIZED",
Self::Forbidden(_) => "FORBIDDEN",
Self::Conflict(_) => "CONFLICT",
Self::BadRequest(_) => "BAD_REQUEST",
Self::Internal(_) => "INTERNAL_ERROR",
Self::Configuration(_) => "CONFIGURATION_ERROR",
Self::ExternalService(_) => "EXTERNAL_SERVICE_ERROR",
Self::RateLimit(_) => "RATE_LIMIT_EXCEEDED",
Self::WebSocket(_) => "WEBSOCKET_ERROR",
Self::Serialization(_) => "SERIALIZATION_ERROR",
Self::PolicyEvaluation(_) => "POLICY_EVALUATION_ERROR",
Self::PolicyParsing(_) => "POLICY_PARSING_ERROR",
Self::Cedar(_) => "CEDAR_ERROR",
Self::Compliance(_) => "COMPLIANCE_ERROR",
Self::Anomaly(_) => "ANOMALY_ERROR",
Self::AnalysisFailed(_) => "ANALYSIS_FAILED",
Self::Other(_) => "INTERNAL_ERROR",
}
}
/// Check if this error should be logged at ERROR level
pub fn should_log_error(&self) -> bool {
match self {
Self::Database(_) => true,
Self::Internal(_) => true,
Self::Configuration(_) => true,
Self::Serialization(_) => true,
Self::PolicyEvaluation(_) => true,
Self::Cedar(_) => true,
Self::Anomaly(_) => true,
Self::AnalysisFailed(_) => true,
Self::Other(_) => true,
_ => false, // Client errors are logged at WARN level
}
}
}
/// Error response structure for API
#[derive(serde::Serialize)]
pub struct ErrorResponse {
pub error: String,
pub error_code: String,
pub message: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub details: Option<serde_json::Value>,
pub timestamp: chrono::DateTime<chrono::Utc>,
}
impl ErrorResponse {
pub fn new(error: &ControlCenterError) -> Self {
Self {
error: error.error_code().to_string(),
error_code: error.error_code().to_string(),
message: error.to_string(),
details: None,
timestamp: chrono::Utc::now(),
}
}
pub fn with_details(mut self, details: serde_json::Value) -> Self {
self.details = Some(details);
self
}
}
impl IntoResponse for ControlCenterError {
fn into_response(self) -> Response {
let status_code = self.status_code();
// Log the error appropriately
if self.should_log_error() {
error!("Error occurred: {} (status: {})", self, status_code);
} else {
tracing::warn!("Client error: {} (status: {})", self, status_code);
}
let error_response = ErrorResponse::new(&self);
(status_code, Json(error_response)).into_response()
}
}
// Implement conversions for common error types
impl From<surrealdb::Error> for ControlCenterError {
fn from(error: surrealdb::Error) -> Self {
Self::Database(error.to_string())
}
}
impl From<jsonwebtoken::errors::Error> for ControlCenterError {
fn from(error: jsonwebtoken::errors::Error) -> Self {
Self::Token(error.to_string())
}
}
impl From<validator::ValidationErrors> for ControlCenterError {
fn from(errors: validator::ValidationErrors) -> Self {
let details = errors
.field_errors()
.iter()
.map(|(field, errors)| {
let messages: Vec<String> = errors
.iter()
.filter_map(|e| e.message.as_ref().map(|m| m.to_string()))
.collect();
(field.to_string(), messages)
})
.collect::<std::collections::HashMap<_, _>>();
Self::Validation(format!("Validation failed: {:?}", details))
}
}
impl From<serde_json::Error> for ControlCenterError {
fn from(error: serde_json::Error) -> Self {
Self::Serialization(error.to_string())
}
}
impl From<std::io::Error> for ControlCenterError {
fn from(error: std::io::Error) -> Self {
Self::Internal(format!("IO error: {}", error))
}
}
impl From<toml::de::Error> for ControlCenterError {
fn from(error: toml::de::Error) -> Self {
Self::Configuration(format!("TOML parsing error: {}", error))
}
}
impl From<argon2::Error> for ControlCenterError {
fn from(error: argon2::Error) -> Self {
Self::Authentication(format!("Password hashing error: {}", error))
}
}
impl From<reqwest::Error> for ControlCenterError {
fn from(error: reqwest::Error) -> Self {
Self::ExternalService(error.to_string())
}
}
impl From<cedar_policy::ParseError> for ControlCenterError {
fn from(err: cedar_policy::ParseError) -> Self {
ControlCenterError::Cedar(err.to_string())
}
}
impl From<cedar_policy::PolicySetError> for ControlCenterError {
fn from(err: cedar_policy::PolicySetError) -> Self {
ControlCenterError::Cedar(err.to_string())
}
}
impl From<crate::kms::kms_service_client::KmsClientError> for ControlCenterError {
fn from(error: crate::kms::kms_service_client::KmsClientError) -> Self {
Self::ExternalService(format!("KMS error: {}", error))
}
}
impl From<crate::kms::error::KmsError> for ControlCenterError {
fn from(error: crate::kms::error::KmsError) -> Self {
Self::Internal(format!("KMS error: {}", error))
}
}
/// Helper macro for creating specific error types
#[macro_export]
macro_rules! control_center_error {
($variant:ident, $msg:expr) => {
$crate::error::ControlCenterError::$variant($msg.to_string())
};
($variant:ident, $fmt:expr, $($arg:tt)*) => {
$crate::error::ControlCenterError::$variant(format!($fmt, $($arg)*))
};
}
/// Helper functions for creating common errors
pub fn user_not_found(identifier: &str) -> ControlCenterError {
ControlCenterError::UserNotFound(identifier.to_string())
}
pub fn session_not_found(session_id: &str) -> ControlCenterError {
ControlCenterError::SessionNotFound(session_id.to_string())
}
pub fn session_expired(session_id: &str) -> ControlCenterError {
ControlCenterError::SessionExpired(session_id.to_string())
}
pub fn unauthorized(message: &str) -> ControlCenterError {
ControlCenterError::Unauthorized(message.to_string())
}
pub fn forbidden(message: &str) -> ControlCenterError {
ControlCenterError::Forbidden(message.to_string())
}
pub fn bad_request(message: &str) -> ControlCenterError {
ControlCenterError::BadRequest(message.to_string())
}
pub fn conflict(message: &str) -> ControlCenterError {
ControlCenterError::Conflict(message.to_string())
}
pub fn internal_error(message: &str) -> ControlCenterError {
ControlCenterError::Internal(message.to_string())
}
pub fn not_found(message: &str) -> ControlCenterError {
ControlCenterError::NotFound(message.to_string())
}

37
crates/control-center/src/handlers/auth.rs
View file

@ -4,6 +4,7 @@ use axum::{
extract::{Request, State},
response::Json,
};
use ontoref_derive::onto_api;
use serde::{Deserialize, Serialize};
use tracing::info;
@ -14,6 +15,15 @@ use crate::services::AuthService;
use crate::AppState;
/// Login endpoint
#[onto_api(
method = "POST",
path = "/auth/login",
description = "Authenticate and obtain JWT tokens",
auth = "none",
actors = "developer, agent",
tags = "auth",
feature = ""
)]
pub async fn login(
State(app_state): State<Arc<AppState>>,
Json(request): Json<LoginRequest>,
@ -32,6 +42,15 @@ pub async fn login(
}
/// Refresh token endpoint
#[onto_api(
method = "POST",
path = "/auth/refresh",
description = "Refresh JWT access token",
auth = "none",
actors = "developer, agent",
tags = "auth",
feature = ""
)]
pub async fn refresh_token(
State(app_state): State<Arc<AppState>>,
Json(request): Json<RefreshTokenRequest>,
@ -42,6 +61,15 @@ pub async fn refresh_token(
}
/// Logout endpoint
#[onto_api(
method = "POST",
path = "/auth/logout",
description = "Invalidate current session and logout",
auth = "bearer",
actors = "developer, agent",
tags = "auth",
feature = ""
)]
pub async fn logout(
State(app_state): State<Arc<AppState>>,
Json(logout_request): Json<LogoutRequest>,
@ -126,6 +154,15 @@ pub async fn invalidate_all_sessions(
}
/// Health check endpoint (no auth required)
#[onto_api(
method = "GET",
path = "/health",
description = "Service health check",
auth = "none",
actors = "developer, agent, ci",
tags = "health",
feature = ""
)]
pub async fn health_check() -> Json<ApiResponse<HealthCheckResponse>> {
Json(ApiResponse::success(HealthCheckResponse {
status: "healthy".to_string(),

55
crates/control-center/src/handlers/iac_deployment.rs
View file

@ -7,6 +7,7 @@ use axum::{
http::StatusCode,
Json,
};
use ontoref_derive::onto_api;
use serde::{Deserialize, Serialize};
// Re-export service types to avoid duplication
@ -32,6 +33,15 @@ pub struct DeploymentQuery {
}
/// List deployment plans
#[onto_api(
method = "GET",
path = "/deployments",
description = "List all deployment plans",
auth = "bearer",
actors = "developer, agent",
tags = "deployments",
feature = ""
)]
pub async fn list_deployments(
State(state): State<Arc<AppState>>,
Query(params): Query<DeploymentQuery>,
@ -53,6 +63,15 @@ pub async fn list_deployments(
}
/// Get deployment plan
#[onto_api(
method = "GET",
path = "/deployments/{id}",
description = "Get a deployment plan by ID",
auth = "bearer",
actors = "developer, agent",
tags = "deployments",
feature = ""
)]
pub async fn get_deployment(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
@ -65,6 +84,15 @@ pub async fn get_deployment(
}
/// Create deployment plan
#[onto_api(
method = "POST",
path = "/deployments",
description = "Create a new deployment plan",
auth = "bearer",
actors = "developer, agent",
tags = "deployments",
feature = ""
)]
pub async fn create_deployment(
State(state): State<Arc<AppState>>,
Json(plan): Json<DeploymentPlan>,
@ -74,6 +102,15 @@ pub async fn create_deployment(
}
/// Update deployment plan
#[onto_api(
method = "PUT",
path = "/deployments/{id}",
description = "Update a deployment plan",
auth = "bearer",
actors = "developer, agent",
tags = "deployments",
feature = ""
)]
pub async fn update_deployment(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
@ -95,6 +132,15 @@ pub struct SubmissionResult {
}
/// Submit deployment plan to orchestrator
#[onto_api(
method = "POST",
path = "/deployments/{id}/submit",
description = "Submit a deployment plan to the orchestrator",
auth = "bearer",
actors = "developer, agent",
tags = "deployments",
feature = ""
)]
pub async fn submit_deployment(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
@ -171,6 +217,15 @@ pub struct DeploymentStatus {
}
/// Get deployment execution status
#[onto_api(
method = "GET",
path = "/deployments/{id}/status",
description = "Get deployment execution status",
auth = "bearer",
actors = "developer, agent",
tags = "deployments",
feature = ""
)]
pub async fn get_deployment_status(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,

28
crates/control-center/src/handlers/iac_detection.rs
View file

@ -10,6 +10,7 @@ use axum::{
http::StatusCode,
Json,
};
use ontoref_derive::onto_api;
use serde::{Deserialize, Serialize};
// Re-export service types
@ -45,6 +46,15 @@ pub struct PagedResponse<T> {
}
/// List all detection results
#[onto_api(
method = "GET",
path = "/detections",
description = "List all infrastructure detections",
auth = "bearer",
actors = "developer, agent",
tags = "detections",
feature = ""
)]
pub async fn list_detections(
State(state): State<Arc<AppState>>,
Query(params): Query<DetectionQuery>,
@ -79,6 +89,15 @@ pub async fn list_detections(
}
/// Get single detection by ID
#[onto_api(
method = "GET",
path = "/detections/{id}",
description = "Get infrastructure detection by ID",
auth = "bearer",
actors = "developer, agent",
tags = "detections",
feature = ""
)]
pub async fn get_detection(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
@ -104,6 +123,15 @@ pub struct AnalyzeProjectRequest {
pub organization: Option<String>,
}
#[onto_api(
method = "POST",
path = "/detections/analyze",
description = "Analyze a project for infrastructure technologies",
auth = "bearer",
actors = "developer, agent",
tags = "detections",
feature = ""
)]
pub async fn analyze_project(
State(state): State<Arc<AppState>>,
Json(req): Json<AnalyzeProjectRequest>,

64
crates/control-center/src/handlers/iac_rules.rs
View file

@ -7,6 +7,7 @@ use axum::{
http::StatusCode,
Json,
};
use ontoref_derive::onto_api;
use serde::{Deserialize, Serialize};
// Re-export service types to avoid duplication
@ -32,6 +33,15 @@ pub struct RulesQuery {
}
/// List all inference rules
#[onto_api(
method = "GET",
path = "/rules",
description = "List all inference rules",
auth = "bearer",
actors = "developer",
tags = "rules",
feature = ""
)]
pub async fn list_rules(
State(state): State<Arc<AppState>>,
Query(params): Query<RulesQuery>,
@ -53,6 +63,15 @@ pub async fn list_rules(
}
/// List rules for specific organization
#[onto_api(
method = "GET",
path = "/rules/org/{org}",
description = "List inference rules for a specific organization",
auth = "bearer",
actors = "developer",
tags = "rules",
feature = ""
)]
pub async fn list_org_rules(
State(state): State<Arc<AppState>>,
Path(org): Path<String>,
@ -75,6 +94,15 @@ pub async fn list_org_rules(
}
/// Get single rule
#[onto_api(
method = "GET",
path = "/rules/{id}",
description = "Get an inference rule by ID",
auth = "bearer",
actors = "developer",
tags = "rules",
feature = ""
)]
pub async fn get_rule(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
@ -83,6 +111,15 @@ pub async fn get_rule(
}
/// Create new rule
#[onto_api(
method = "POST",
path = "/rules",
description = "Create a new inference rule",
auth = "bearer",
actors = "developer",
tags = "rules",
feature = ""
)]
pub async fn create_rule(
State(state): State<Arc<AppState>>,
Json(rule): Json<InferenceRule>,
@ -92,6 +129,15 @@ pub async fn create_rule(
}
/// Update existing rule
#[onto_api(
method = "PUT",
path = "/rules/{id}",
description = "Update an inference rule",
auth = "bearer",
actors = "developer",
tags = "rules",
feature = ""
)]
pub async fn update_rule(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
@ -102,6 +148,15 @@ pub async fn update_rule(
}
/// Delete rule
#[onto_api(
method = "DELETE",
path = "/rules/{id}",
description = "Delete an inference rule",
auth = "bearer",
actors = "developer",
tags = "rules",
feature = ""
)]
pub async fn delete_rule(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,
@ -124,6 +179,15 @@ pub struct TestRuleResult {
pub confidence_score: f32,
}
#[onto_api(
method = "POST",
path = "/rules/{id}/test",
description = "Test an inference rule against a project",
auth = "bearer",
actors = "developer",
tags = "rules",
feature = ""
)]
pub async fn test_rule(
State(state): State<Arc<AppState>>,
Path(id): Path<String>,

10
crates/control-center/src/handlers/permission.rs
View file

@ -4,6 +4,7 @@ use axum::{
extract::{Query, Request, State},
response::Json,
};
use ontoref_derive::onto_api;
use serde::Deserialize;
use crate::error::{auth, http, ControlCenterError, Result};
@ -13,6 +14,15 @@ use crate::models::PermissionResponse;
use crate::AppState;
/// List permissions
#[onto_api(
method = "GET",
path = "/permissions",
description = "List all permissions",
auth = "bearer",
actors = "developer",
tags = "permissions",
feature = ""
)]
pub async fn list_permissions(
State(app_state): State<Arc<AppState>>,
Query(params): Query<ListPermissionsParams>,

127
crates/control-center/src/handlers/secrets.rs
View file

@ -6,6 +6,7 @@ use axum::{
response::IntoResponse,
Json,
};
use ontoref_derive::onto_api;
use serde::{Deserialize, Serialize};
use tracing::{error, info};
@ -181,6 +182,15 @@ fn default_limit() -> usize {
}
/// Handler: Create a new secret
#[onto_api(
method = "POST",
path = "/secrets",
description = "Create a new secret",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn create_secret(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -269,6 +279,15 @@ pub async fn create_secret(
}
/// Handler: Get a secret value (decrypted)
#[onto_api(
method = "GET",
path = "/secrets/{path}",
description = "Get a secret value by path",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn get_secret(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -356,6 +375,15 @@ pub struct GetSecretQuery {
}
/// Handler: List secrets
#[onto_api(
method = "GET",
path = "/secrets",
description = "List all secrets",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn list_secrets(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -392,6 +420,15 @@ pub async fn list_secrets(
}
/// Handler: Update a secret (creates new version)
#[onto_api(
method = "PUT",
path = "/secrets/{path}",
description = "Update a secret, creating a new version",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn update_secret(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -497,6 +534,15 @@ pub async fn update_secret(
}
/// Handler: Delete a secret
#[onto_api(
method = "DELETE",
path = "/secrets/{path}",
description = "Delete a secret",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn delete_secret(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -579,6 +625,15 @@ pub async fn delete_secret(
}
/// Handler: Get secret history/versions
#[onto_api(
method = "GET",
path = "/secrets/{path}/history",
description = "Get version history for a secret",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn get_secret_history(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -606,6 +661,15 @@ pub async fn get_secret_history(
}
/// Handler: Restore a specific version of a secret
#[onto_api(
method = "POST",
path = "/secrets/{path}/restore/{version}",
description = "Restore a secret to a specific version",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn restore_secret_version(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -741,6 +805,15 @@ pub struct RotationStatusResponse {
}
/// Handler: Force rotate a secret
#[onto_api(
method = "POST",
path = "/secrets/{path}/rotate",
description = "Force rotate a secret",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn force_rotate_secret(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -845,6 +918,15 @@ pub async fn force_rotate_secret(
}
/// Handler: Get rotation status for a secret
#[onto_api(
method = "GET",
path = "/secrets/{path}/rotation-status",
description = "Get rotation status for a secret",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn get_rotation_status(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -922,6 +1004,15 @@ pub struct GrantResponse {
}
/// Handler: Create a secret grant (sharing)
#[onto_api(
method = "POST",
path = "/secrets/{path}/grant",
description = "Create a grant to share a secret",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn create_grant(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -998,6 +1089,15 @@ pub struct RevokeGrantRequest {
}
/// Handler: Revoke a secret grant
#[onto_api(
method = "POST",
path = "/secrets/grant/{grant_id}/revoke",
description = "Revoke a secret grant",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn revoke_grant(
State(app_state): State<Arc<AppState>>,
Extension(security_ctx): Extension<SecurityContext>,
@ -1035,6 +1135,15 @@ pub async fn revoke_grant(
// ============== PHASE 3.4: MONITORING HANDLERS ==============
/// Handler: Get dashboard metrics
#[onto_api(
method = "GET",
path = "/secrets/monitoring/dashboard",
description = "Get secrets monitoring dashboard metrics",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn get_dashboard_metrics(
State(app_state): State<Arc<AppState>>,
Extension(_security_ctx): Extension<SecurityContext>,
@ -1055,6 +1164,15 @@ pub async fn get_dashboard_metrics(
}
/// Handler: Get alert summary
#[onto_api(
method = "GET",
path = "/secrets/monitoring/alerts",
description = "Get secrets monitoring alert summary",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn get_alert_summary(
State(app_state): State<Arc<AppState>>,
Extension(_security_ctx): Extension<SecurityContext>,
@ -1072,6 +1190,15 @@ pub async fn get_alert_summary(
}
/// Handler: Get expiring secrets
#[onto_api(
method = "GET",
path = "/secrets/monitoring/expiring",
description = "Get list of expiring secrets",
auth = "admin",
actors = "developer",
tags = "secrets",
feature = ""
)]
pub async fn get_expiring_secrets(
State(app_state): State<Arc<AppState>>,
Extension(_security_ctx): Extension<SecurityContext>,

1
crates/control-center/src/lib.rs
View file

@ -23,6 +23,7 @@ pub mod error;
pub mod handlers;
pub mod middleware;
pub mod models;
pub mod ncl_config;
pub mod services;
pub mod simple_config;
pub mod storage;

80
crates/control-center/src/main.rs
View file

@ -11,6 +11,8 @@ use axum::{
routing::{get, post},
Router,
};
mod api_catalog;
use clap::Parser;
use control_center::handlers::{
auth::*,
@ -94,12 +96,22 @@ struct Cli {
/// Generate default configuration file
#[arg(long)]
generate_config: bool,
/// Print all #[onto_api] registered routes as JSON and exit.
/// Pipe to api-catalog-control-center.json: `just export-api-catalog`
#[arg(long)]
dump_api_catalog: bool,
}
#[tokio::main]
async fn main() -> Result<()> {
let cli = Cli::parse();
if cli.dump_api_catalog {
println!("{}", ontoref_ontology::api::dump_catalog_json());
return Ok(());
}
// Generate default config if requested
if cli.generate_config {
let config_path = cli.config.unwrap_or_else(|| PathBuf::from("config.toml"));
@ -114,7 +126,7 @@ async fn main() -> Result<()> {
// Check if control-center is enabled in deployment-mode.ncl
if let Ok(deployment) = platform_config::load_deployment_mode() {
if let Ok(enabled) = deployment.is_service_enabled("control-center") {
if let Ok(enabled) = deployment.is_service_enabled("control_center") {
if !enabled {
warn!("⚠ Control Center is DISABLED in deployment-mode.ncl");
std::process::exit(1);
@ -123,25 +135,28 @@ async fn main() -> Result<()> {
}
}
// Try to load control-center.ncl
if let Ok(config) = platform_config::load_service_config_from_ncl("control-center") {
info!("✓ Loaded control-center configuration from NCL");
tracing::debug!("Config: {:?}", config);
// Load configuration from NCL using the same pattern as orchestrator and vault-service:
// ControlCenterNclConfig implements ConfigLoader which reads the NCL via
// PROVISIONING_CONFIG_DIR, exports to JSON, and deserializes into typed structs.
use control_center::ncl_config::ControlCenterNclConfig;
use platform_config::format::ConfigLoader as _;
let ncl = ControlCenterNclConfig::load().map_err(|e| {
control_center::ControlCenterError::Infrastructure(
control_center::error::infrastructure::InfrastructureError::Configuration(format!(
"Failed to load control-center NCL config: {}",
e
)),
)
})?;
let mut config = ncl.to_runtime_config()?;
// Apply explicit CLI overrides on top of NCL values
if let Some(path) = &cli.config {
// If a TOML config file was explicitly given, merge it on top of NCL
let toml_config = Config::load_from_file(path)?;
config = toml_config;
}
// Resolve config file path using new resolver
let resolver = platform_config::ConfigResolver::new()
.with_cli_config(cli.config.clone())
.with_cli_config_dir(cli.config_dir.clone())
.with_cli_mode(cli.mode.clone());
// Load configuration
let mut config = if let Some(path) = resolver.resolve("control-center") {
Config::load_from_file(path)?
} else {
Config::load()?
};
// Apply CLI overrides
if let Some(port) = cli.port {
config.server.port = port;
@ -207,6 +222,7 @@ async fn create_router(app_state: Arc<AppState>) -> Result<Router> {
.route("/health", get(health_check))
.route("/auth/login", post(login))
.route("/auth/refresh", post(refresh_token))
.route("/api/catalog", get(api_catalog::api_catalog))
.layer(auth_rate_limit);
// Protected routes (authentication required)
@ -239,44 +255,44 @@ async fn create_router(app_state: Arc<AppState>) -> Result<Router> {
// .route("/permissions/actions", get(get_actions))
// Detection routes (Infrastructure-from-Code)
.route("/detections", get(list_detections))
.route("/detections/:id", get(get_detection))
.route("/detections/{id}", get(get_detection))
.route("/detections/analyze", post(analyze_project))
// Rules routes (Inference rules)
.route("/rules", get(list_rules).post(create_rule))
.route("/rules/org/:org", get(list_org_rules))
.route("/rules/org/{org}", get(list_org_rules))
.route(
"/rules/:id",
"/rules/{id}",
get(get_rule).put(update_rule).delete(delete_rule),
)
.route("/rules/:id/test", post(test_rule))
.route("/rules/{id}/test", post(test_rule))
// Deployment routes
.route(
"/deployments",
get(list_deployments).post(create_deployment),
)
.route(
"/deployments/:id",
"/deployments/{id}",
get(get_deployment).put(update_deployment),
)
.route("/deployments/:id/submit", post(submit_deployment))
.route("/deployments/:id/status", get(get_deployment_status))
.route("/deployments/{id}/submit", post(submit_deployment))
.route("/deployments/{id}/status", get(get_deployment_status))
// Secrets routes (Phase 1.5 - Now active with SecretsService state initialization)
.route("/secrets", post(create_secret).get(list_secrets))
.route(
"/secrets/:path",
"/secrets/{path}",
get(get_secret).put(update_secret).delete(delete_secret),
)
.route("/secrets/:path/history", get(get_secret_history))
.route("/secrets/{path}/history", get(get_secret_history))
.route(
"/secrets/:path/restore/:version",
"/secrets/{path}/restore/{version}",
post(restore_secret_version),
)
// Secrets Phase 3.1: Rotation routes
.route("/secrets/:path/rotate", post(force_rotate_secret))
.route("/secrets/:path/rotation-status", get(get_rotation_status))
.route("/secrets/{path}/rotate", post(force_rotate_secret))
.route("/secrets/{path}/rotation-status", get(get_rotation_status))
// Secrets Phase 3.2: Sharing routes
.route("/secrets/:path/grant", post(create_grant))
.route("/secrets/grant/:grant_id/revoke", post(revoke_grant))
.route("/secrets/{path}/grant", post(create_grant))
.route("/secrets/grant/{grant_id}/revoke", post(revoke_grant))
// Secrets Phase 3.4: Monitoring routes
.route("/secrets/monitoring/dashboard", get(get_dashboard_metrics))
.route("/secrets/monitoring/alerts", get(get_alert_summary))

7
crates/control-center/src/models/permission.rs
View file

@ -1,13 +1,14 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use surrealdb::sql::Thing;
use surrealdb::types::{RecordId, SurrealValue};
use uuid::Uuid;
use validator::Validate;
/// Permission model for fine-grained access control
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct Permission {
pub id: Option<Thing>,
pub id: Option<RecordId>,
pub permission_id: Uuid,
pub name: String,
pub resource: String,

7
crates/control-center/src/models/role.rs
View file

@ -1,13 +1,14 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use surrealdb::sql::Thing;
use surrealdb::types::{RecordId, SurrealValue};
use uuid::Uuid;
use validator::Validate;
/// Role model for permission-based access control
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct Role {
pub id: Option<Thing>,
pub id: Option<RecordId>,
pub role_id: Uuid,
pub name: String,
pub description: Option<String>,

10
crates/control-center/src/models/session.rs
View file

@ -1,12 +1,13 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use surrealdb::sql::Thing;
use surrealdb::types::{RecordId, SurrealValue};
use uuid::Uuid;
/// Session model for managing user sessions and refresh tokens
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct Session {
pub id: Option<Thing>,
pub id: Option<RecordId>,
pub session_id: Uuid,
pub user_id: Uuid,
pub refresh_token: String,
@ -18,7 +19,8 @@ pub struct Session {
}
/// Client information for session tracking
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct ClientInfo {
pub user_agent: Option<String>,
pub ip_address: Option<String>,

7
crates/control-center/src/models/user.rs
View file

@ -1,13 +1,14 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use surrealdb::sql::Thing;
use surrealdb::types::{RecordId, SurrealValue};
use uuid::Uuid;
use validator::Validate;
/// User model for SurrealDB storage
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct User {
pub id: Option<Thing>,
pub id: Option<RecordId>,
pub user_id: Uuid,
pub email: String,
pub username: String,

404
crates/control-center/src/ncl_config.rs Normal file
View file

@ -0,0 +1,404 @@
use platform_config::format::ConfigLoader;
use serde::{Deserialize, Serialize};
use crate::error::{infrastructure, ControlCenterError, Result};
use crate::simple_config::{
Config, CorsConfig, DatabaseConfig, JwtConfig, LoggingConfig, RateLimitConfig, SecurityConfig,
ServerConfig,
};
/// Root wrapper matching the NCL export key `control_center`.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct ControlCenterNclConfig {
#[serde(default)]
pub control_center: NclSettings,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct NclSettings {
#[serde(default)]
pub server: NclServer,
#[serde(default)]
pub database: NclDatabase,
#[serde(default)]
pub security: NclSecurity,
#[serde(default)]
pub logging: NclLogging,
#[serde(default)]
pub users: NclUsers,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NclServer {
#[serde(default = "default_host")]
pub host: String,
#[serde(default = "default_port")]
pub port: u16,
pub workers: Option<usize>,
pub keep_alive: Option<u64>,
pub max_connections: Option<usize>,
}
impl Default for NclServer {
fn default() -> Self {
Self {
host: default_host(),
port: default_port(),
workers: Some(2),
keep_alive: Some(75),
max_connections: Some(100),
}
}
}
fn default_host() -> String {
"0.0.0.0".to_string()
}
fn default_port() -> u16 {
9012
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct NclDatabase {
/// Internal storage backend of the SurrealDB server (e.g. "rocksdb").
/// Does NOT determine how the Rust client connects — always WebSocket for Docker.
#[serde(default)]
pub backend: Option<String>,
#[serde(default)]
pub mode: Option<String>,
/// RocksDB data path (server-side configuration, not used by Rust client)
#[serde(default)]
pub path: Option<String>,
/// WebSocket connection string for remote SurrealDB (Docker).
/// Falls back to "127.0.0.1:8000" if unset.
#[serde(default)]
pub connection_string: Option<String>,
#[serde(default)]
pub namespace: Option<String>,
/// SurrealDB authentication credentials (required when server runs with --user/--pass)
#[serde(default)]
pub credentials: Option<NclDbCredentials>,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct NclDbCredentials {
pub username: Option<String>,
pub password: Option<String>,
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct NclSecurity {
#[serde(default)]
pub jwt: NclJwt,
#[serde(default)]
pub rate_limiting: NclRateLimiting,
#[serde(default)]
pub cors: NclCors,
#[serde(default)]
pub session: NclSession,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NclJwt {
#[serde(default = "default_issuer")]
pub issuer: String,
#[serde(default = "default_audience")]
pub audience: String,
/// Access token lifetime in seconds
#[serde(default = "default_expiration")]
pub expiration: i64,
/// Refresh token lifetime in seconds
#[serde(default = "default_refresh_expiration")]
pub refresh_expiration: i64,
}
impl Default for NclJwt {
fn default() -> Self {
Self {
issuer: default_issuer(),
audience: default_audience(),
expiration: default_expiration(),
refresh_expiration: default_refresh_expiration(),
}
}
}
fn default_issuer() -> String {
"control-center".to_string()
}
fn default_audience() -> String {
"provisioning".to_string()
}
fn default_expiration() -> i64 {
3600
}
fn default_refresh_expiration() -> i64 {
86400
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NclRateLimiting {
#[serde(default)]
pub enabled: bool,
/// NCL schema types this field as String ("1000"), so we use Value to handle both.
#[serde(default = "default_max_requests_json")]
pub max_requests: serde_json::Value,
#[serde(default = "default_window_secs")]
pub window_seconds: u64,
}
impl Default for NclRateLimiting {
fn default() -> Self {
Self {
enabled: false,
max_requests: serde_json::Value::Number(100.into()),
window_seconds: 60,
}
}
}
impl NclRateLimiting {
fn max_requests_u32(&self) -> u32 {
match &self.max_requests {
serde_json::Value::Number(n) => n.as_u64().unwrap_or(100) as u32,
serde_json::Value::String(s) => s.parse().unwrap_or(100),
_ => 100,
}
}
}
fn default_max_requests_json() -> serde_json::Value {
serde_json::Value::Number(100.into())
}
fn default_window_secs() -> u64 {
60
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct NclCors {
#[serde(default)]
pub enabled: bool,
#[serde(default)]
pub allow_credentials: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NclSession {
/// Idle timeout in seconds — drives session_cleanup_interval_minutes
#[serde(default = "default_idle_timeout")]
pub idle_timeout: u64,
}
impl Default for NclSession {
fn default() -> Self {
Self {
idle_timeout: default_idle_timeout(),
}
}
}
fn default_idle_timeout() -> u64 {
3600
}
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct NclUsers {
#[serde(default)]
pub sessions: NclUserSessions,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NclUserSessions {
pub max_active: Option<usize>,
#[serde(default = "default_idle_timeout")]
pub idle_timeout: u64,
}
impl Default for NclUserSessions {
fn default() -> Self {
Self {
max_active: Some(5),
idle_timeout: 3600,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NclLogging {
#[serde(default = "default_log_level")]
pub level: String,
#[serde(default = "default_log_format")]
pub format: String,
}
impl Default for NclLogging {
fn default() -> Self {
Self {
level: default_log_level(),
format: default_log_format(),
}
}
}
fn default_log_level() -> String {
"info".to_string()
}
fn default_log_format() -> String {
"json".to_string()
}
impl ConfigLoader for ControlCenterNclConfig {
fn service_name() -> &'static str {
"control-center"
}
fn load_from_hierarchy(
) -> std::result::Result<Self, Box<dyn std::error::Error + Send + Sync>> {
let service = Self::service_name();
if let Some(path) = platform_config::resolve_config_path(service) {
return Self::from_path(&path);
}
Ok(Self::default())
}
fn from_path<P: AsRef<std::path::Path>>(
path: P,
) -> std::result::Result<Self, Box<dyn std::error::Error + Send + Sync>> {
let path = path.as_ref();
let json_value = platform_config::format::load_config(path).map_err(|e| {
Box::new(e) as Box<dyn std::error::Error + Send + Sync>
})?;
serde_json::from_value(json_value).map_err(|e| {
Box::new(std::io::Error::new(
std::io::ErrorKind::InvalidData,
format!(
"Failed to deserialize control-center NCL config from {:?}: {}",
path, e
),
)) as Box<dyn std::error::Error + Send + Sync>
})
}
}
impl ControlCenterNclConfig {
/// Convert the NCL platform config into the runtime `Config` used by services.
///
/// JWT RSA keys are generated fresh on each startup (NCL uses HS256 which
/// is incompatible with the service's RS256 implementation). For solo mode,
/// sessions do not persist across restarts, so ephemeral keys are acceptable.
pub fn to_runtime_config(&self) -> Result<Config> {
let cc = &self.control_center;
let key_pair =
crate::services::jwt::generate_rsa_key_pair().map_err(|e| {
ControlCenterError::Infrastructure(
infrastructure::InfrastructureError::Configuration(format!(
"Failed to generate RSA key pair: {}",
e
)),
)
})?;
let db_url = cc
.database
.connection_string
.as_deref()
.unwrap_or("127.0.0.1:8000")
.to_string();
let namespace = cc
.database
.namespace
.as_deref()
.unwrap_or("control_center")
.to_string();
// Credentials: NCL → env vars SURREALDB_USER / SURREALDB_PASS → None
let db_username = cc
.database
.credentials
.as_ref()
.and_then(|c| c.username.clone())
.or_else(|| std::env::var("SURREALDB_USER").ok());
let db_password = cc
.database
.credentials
.as_ref()
.and_then(|c| c.password.clone())
.or_else(|| std::env::var("SURREALDB_PASS").ok());
Ok(Config {
server: ServerConfig {
host: cc.server.host.clone(),
port: cc.server.port,
workers: cc.server.workers,
keep_alive: cc.server.keep_alive,
max_connections: cc.server.max_connections,
},
database: DatabaseConfig {
url: db_url,
namespace,
database: "main".to_string(),
username: db_username,
password: db_password,
},
jwt: JwtConfig {
issuer: cc.security.jwt.issuer.clone(),
audience: cc.security.jwt.audience.clone(),
access_token_expiration_hours: (cc.security.jwt.expiration / 3600).max(1),
refresh_token_expiration_hours: (cc.security.jwt.refresh_expiration / 3600).max(1),
private_key_pem: key_pair.private_key_pem,
public_key_pem: key_pair.public_key_pem,
},
rate_limiting: RateLimitConfig {
max_requests: cc.security.rate_limiting.max_requests_u32(),
window_seconds: cc.security.rate_limiting.window_seconds,
per_ip: true,
global: false,
},
cors: CorsConfig {
allowed_origins: vec!["*".to_string()],
allowed_methods: vec![
"GET".to_string(),
"POST".to_string(),
"PUT".to_string(),
"DELETE".to_string(),
"PATCH".to_string(),
"OPTIONS".to_string(),
],
allowed_headers: vec![
"content-type".to_string(),
"authorization".to_string(),
"accept".to_string(),
"x-requested-with".to_string(),
"x-session-id".to_string(),
],
expose_headers: vec![
"x-total-count".to_string(),
"x-rate-limit-remaining".to_string(),
"x-rate-limit-limit".to_string(),
"x-rate-limit-reset".to_string(),
],
max_age: 86400,
allow_credentials: cc.security.cors.allow_credentials,
},
security: SecurityConfig {
session_cleanup_interval_minutes: (cc.security.session.idle_timeout / 60).max(1),
max_sessions_per_user: cc.users.sessions.max_active,
password_min_length: 8,
password_require_special_chars: false,
password_require_numbers: false,
password_require_uppercase: false,
failed_login_lockout_attempts: Some(5),
failed_login_lockout_duration_minutes: Some(15),
},
logging: LoggingConfig {
level: cc.logging.level.clone(),
format: cc.logging.format.clone(),
file_path: None,
max_file_size: Some("100MB".to_string()),
max_files: Some(10),
},
})
}
}

19
crates/control-center/src/services/database.rs
View file

@ -1,6 +1,7 @@
use anyhow::Context;
use surrealdb::engine::local::Mem;
use surrealdb::engine::remote::ws::{Client, Ws};
use surrealdb::opt::auth::Root;
use surrealdb::types::SurrealValue;
use surrealdb::Surreal;
use tracing::{info, warn};
@ -9,7 +10,7 @@ use crate::error::{auth, ControlCenterError, Result};
/// Database service for SurrealDB operations
#[derive(Clone)]
pub struct DatabaseService {
pub db: Surreal<surrealdb::engine::local::Db>,
pub db: Surreal<Client>,
}
// Use the configuration from simple_config
@ -18,15 +19,18 @@ use crate::simple_config::DatabaseConfig;
impl DatabaseService {
/// Create a new database service and connect
pub async fn new(config: DatabaseConfig) -> Result<Self> {
info!("Connecting to SurrealDB (in-memory) at {}", config.url);
info!("Connecting to SurrealDB at ws://{}", config.url);
let db = Surreal::new::<Mem>(())
let db = Surreal::new::<Ws>(&*config.url)
.await
.context("Failed to connect to SurrealDB")?;
// Sign in with root credentials if provided
// Sign in only when credentials are explicitly configured
if let (Some(username), Some(password)) = (&config.username, &config.password) {
db.signin(Root { username, password })
db.signin(Root {
username: username.clone(),
password: password.clone(),
})
.await
.context("Failed to sign in to SurrealDB")?;
}
@ -414,7 +418,8 @@ impl DatabaseService {
}
/// Database statistics
#[derive(Debug, serde::Deserialize, serde::Serialize)]
#[derive(Debug, serde::Deserialize, serde::Serialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct DatabaseStatistics {
pub users_count: i64,
pub roles_count: i64,

13
crates/control-center/src/services/iac_deployment.rs
View file

@ -6,13 +6,15 @@ use std::sync::Arc;
use anyhow::Context;
use serde::{Deserialize, Serialize};
use surrealdb::types::SurrealValue;
use uuid::Uuid;
use super::DatabaseService;
use crate::error::{http, ControlCenterError, Result};
/// Deployment task status
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Default)]
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Default, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
#[serde(rename_all = "lowercase")]
pub enum TaskStatus {
#[default]
@ -34,7 +36,8 @@ impl std::fmt::Display for TaskStatus {
}
/// Deployment task
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct DeploymentTask {
pub id: String,
pub task_type: String,
@ -47,7 +50,8 @@ pub struct DeploymentTask {
}
/// Deployment plan
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct DeploymentPlan {
pub id: String,
pub name: String,
@ -59,7 +63,8 @@ pub struct DeploymentPlan {
}
/// Deployment execution status
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct DeploymentExecution {
pub id: String,
pub plan_id: String,

8
crates/control-center/src/services/iac_detection.rs
View file

@ -4,6 +4,8 @@
use std::sync::Arc;
use surrealdb::types::SurrealValue;
use anyhow::Context;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
@ -12,7 +14,8 @@ use super::DatabaseService;
use crate::error::{http, ControlCenterError, Result};
/// Detected technology information
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct DetectedTechnology {
pub technology: String,
pub confidence: f32,
@ -21,7 +24,8 @@ pub struct DetectedTechnology {
}
/// Detection result with analysis metadata
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct DetectionResult {
pub id: String,
pub project_path: String,

8
crates/control-center/src/services/iac_rules.rs
View file

@ -4,6 +4,8 @@
use std::sync::Arc;
use surrealdb::types::SurrealValue;
use anyhow::Context;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
@ -12,7 +14,8 @@ use super::DatabaseService;
use crate::error::{http, ControlCenterError, Result};
/// Single inference in a rule
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct RuleInference {
pub technology: String,
pub reason: String,
@ -21,7 +24,8 @@ pub struct RuleInference {
}
/// Inference rule for infrastructure completion
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct InferenceRule {
pub id: String,
pub name: String,

10
crates/control-center/src/services/secrets.rs
View file

@ -3,6 +3,7 @@ use std::sync::Arc;
use base64::Engine;
use chrono::Utc;
use serde::{Deserialize, Serialize};
use surrealdb::types::SurrealValue;
use tracing::{debug, error, info};
use crate::audit::AuditLogger;
@ -14,7 +15,8 @@ use crate::kms::kms_service_client::KmsServiceClient;
use crate::storage::surrealdb_storage::SurrealDbStorage;
/// Database connection information
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct DatabaseConnection {
pub host: String,
pub port: u16,
@ -32,7 +34,8 @@ pub enum SecretLifecycle {
}
/// Secret type classification
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub enum SecretType {
/// Database credentials
Database {
@ -58,7 +61,8 @@ pub enum SecretType {
}
/// Vault secret metadata stored in SurrealDB
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct VaultSecret {
// Existing fields
pub id: String,

2
crates/control-center/src/simple_config.rs
View file

@ -126,7 +126,7 @@ impl Default for ServerConfig {
fn default() -> Self {
Self {
host: "0.0.0.0".to_string(),
port: 9080,
port: 9012,
workers: None,
keep_alive: Some(75),
max_connections: Some(1000),

7
crates/control-center/src/storage/mod.rs
View file

@ -11,6 +11,7 @@ use std::sync::Arc;
use async_trait::async_trait;
pub use database::{Database, DatabaseConfig};
use serde::{Deserialize, Serialize};
use surrealdb::types::SurrealValue;
// TODO: Re-enable when policies module is fixed
// use crate::policies::{PolicyMetadata, PolicyVersion};
// use crate::policies::versioning::RollbackResult;
@ -145,7 +146,8 @@ impl Default for PolicySearchQuery {
}
/// Policy evaluation event for audit trail
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct PolicyEvaluationEvent {
pub id: String,
pub policy_id: String,
@ -174,7 +176,8 @@ pub struct PolicyMetrics {
}
/// Compliance check result
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct ComplianceCheckResult {
pub id: String,
pub framework: String,

34
crates/control-center/src/storage/surrealdb_storage.rs
View file

@ -17,7 +17,8 @@ use crate::error::{auth, policy, ControlCenterError, Result};
use crate::services::secrets::SecretType;
use crate::simple_config::Config;
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct PolicyMetadata {
pub id: String,
pub name: String,
@ -32,7 +33,8 @@ pub struct PolicyMetadata {
pub enabled: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
pub struct PolicyVersion {
pub version_id: String,
pub policy_id: String,
@ -54,12 +56,14 @@ pub struct RollbackResult {
use async_trait::async_trait;
use surrealdb::engine::local::Mem;
use surrealdb::engine::remote::ws::{Client, Ws};
use surrealdb::{RecordId, Surreal};
use surrealdb::types::{RecordId, SurrealValue};
use surrealdb::Surreal;
use tracing::{debug, info};
use uuid::Uuid;
/// SurrealDB record for policies
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
struct PolicyRecord {
pub id: RecordId,
pub policy_id: String,
@ -70,21 +74,24 @@ struct PolicyRecord {
}
/// SurrealDB record for policy versions
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
struct PolicyVersionRecord {
pub id: RecordId,
pub version: PolicyVersion,
}
/// SurrealDB record for policy evaluations
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
struct PolicyEvaluationRecord {
pub id: RecordId,
pub evaluation: PolicyEvaluationEvent,
}
/// SurrealDB record for compliance checks
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize, SurrealValue)]
#[surreal(crate = "surrealdb::types")]
struct ComplianceCheckRecord {
pub id: RecordId,
pub result: ComplianceCheckResult,
@ -110,8 +117,11 @@ impl SurrealDbPolicyStorage<Client> {
if let (Some(username), Some(password)) =
(&config.database.username, &config.database.password)
{
db.signin(surrealdb::opt::auth::Root { username, password })
.await?;
db.signin(surrealdb::opt::auth::Root {
username: username.clone(),
password: password.clone(),
})
.await?;
}
// Use namespace and database
@ -285,7 +295,7 @@ where
{
/// Generate record ID for table
fn generate_record_id(&self, table: &str) -> RecordId {
RecordId::from_table_key(table, Uuid::new_v4().to_string())
RecordId::new(table, Uuid::new_v4().to_string())
}
}
@ -701,7 +711,7 @@ where
pub async fn create_secret(&self, secret: &VaultSecret) -> Result<()> {
let _: Option<VaultSecret> = self
.db
.create(("vault_secrets", &secret.id))
.create(("vault_secrets", secret.id.as_str()))
.content(secret.clone())
.await?;
@ -866,7 +876,7 @@ where
// Update current version in vault_secrets
let _: Option<VaultSecret> = self
.db
.update(("vault_secrets", &secret.id))
.update(("vault_secrets", secret.id.as_str()))
.content(secret.clone())
.await?;

48
crates/daemon/Cargo.toml
View file

@ -1,48 +0,0 @@
[package]
authors.workspace = true
edition.workspace = true
license.workspace = true
name = "daemon"
repository.workspace = true
version.workspace = true
[[bin]]
name = "provisioning-daemon"
path = "src/main.rs"
[dependencies]
# Core daemon library from prov-ecosystem
daemon-cli = { workspace = true }
# Async runtime and networking
axum = { workspace = true }
tokio = { workspace = true }
tower = { workspace = true }
tower-http = { workspace = true }
# Serialization
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
toml = { workspace = true }
# Platform configuration
platform-config = { workspace = true }
# Centralized observability (logging, metrics, health, tracing)
observability = { workspace = true, features = ["logging", "metrics-prometheus", "health"] }
# Error handling
anyhow = { workspace = true }
thiserror = { workspace = true }
# Logging
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
# CLI
clap = { workspace = true, features = ["derive"] }
# Utilities
chrono = { workspace = true }
dirs = { workspace = true }
uuid = { workspace = true }

23
crates/data/tasks/00b7d705-2ac2-4532-9ce9-d161dd320011.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "00b7d705-2ac2-4532-9ce9-d161dd320011",
"name": "component_install_democratic_csi",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_00b7d705-2ac2-4532-9ce9-d161dd320011.tar.gz.b64' > /tmp/democratic_csi.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/democratic_csi.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/democratic_csi && mkdir -p /tmp/democratic_csi && tar xzf /tmp/democratic_csi.tar.gz -C /tmp/democratic_csi && cd /tmp/democratic_csi && sudo bash install-democratic_csi.sh install ; rc=$?; rm -f /tmp/democratic_csi.tar.gz && rm -rf /tmp/democratic_csi; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_00b7d705-2ac2-4532-9ce9-d161dd320011.tar.gz.b64' /tmp/democratic_csi.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T03:15:03.134943Z",
"started_at": "2026-04-21T03:15:03.377192Z",
"completed_at": "2026-04-21T03:15:30.771379Z",
"output": "=== democratic-csi: v0.14.6 ===\n=== democratic-csi: deploying nfs-client driver (server=10.0.8.20) ===\nnamespace/democratic-csi unchanged\n\"democratic-csi\" has been added to your repositories\nHang tight while we grab the latest from your chart repositories...\n...Successfully got an update from the \"democratic-csi\" chart repository\nUpdate Complete. ⎈Happy Helming!⎈\nserviceaccount/democratic-csi-controller-sa created\nserviceaccount/democratic-csi-node-sa created\nsecret/democratic-csi-driver-config created\nconfigmap/democratic-csi created\nstorageclass.storage.k8s.io/nfs-shared created\nclusterrole.rbac.authorization.k8s.io/democratic-csi-controller-cr created\nclusterrole.rbac.authorization.k8s.io/democratic-csi-node-cr created\nclusterrolebinding.rbac.authorization.k8s.io/democratic-csi-controller-rb created\nclusterrolebinding.rbac.authorization.k8s.io/democratic-csi-node-rb created\ndaemonset.apps/democratic-csi-node created\ndeployment.apps/democratic-csi-controller created\ncsidriver.storage.k8s.io/org.democratic-csi.nfs-client created\n=== democratic-csi: waiting for controller ===\nWaiting for deployment \"democratic-csi-controller\" rollout to finish: 0 of 1 updated replicas are available...\ndeployment \"democratic-csi-controller\" successfully rolled out\n=== democratic-csi: ready (nfs=10.0.8.20/var/lib/data/shared, class=nfs-shared) ===\n",
"error": null,
"tags": {
"type": "component",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"component": "democratic_csi",
"operation": "install",
"server": "libre-wuji-cp-0"
}
}

23
crates/data/tasks/056539f0-0782-456f-b3e0-3a3e4efc3bb6.json Normal file
View file

File diff suppressed because one or more lines are too long

23
crates/data/tasks/059e4edd-7f6b-4766-85d8-379c9fc92694.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "059e4edd-7f6b-4766-85d8-379c9fc92694",
"name": "component_install_os",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_059e4edd-7f6b-4766-85d8-379c9fc92694.tar.gz.b64' > /tmp/os.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/os.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/os && mkdir -p /tmp/os && tar xzf /tmp/os.tar.gz -C /tmp/os && cd /tmp/os && sudo bash install-os.sh install ; rc=$?; rm -f /tmp/os.tar.gz && rm -rf /tmp/os; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_059e4edd-7f6b-4766-85d8-379c9fc92694.tar.gz.b64' /tmp/os.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-21T01:37:11.770013Z",
"started_at": "2026-04-21T01:37:11.879953Z",
"completed_at": "2026-04-21T01:37:12.257125Z",
"output": null,
"error": "Command execution failed: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\n@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @\n@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\nIT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!\nSomeone could be eavesdropping on you right now (man-in-the-middle attack)!\nIt is also possible that a host key has just been changed.\nThe fingerprint for the ED25519 key sent by the remote host is\nSHA256:n47ih/YVibQVEAIYpId7Q5YYswXe0aBwPp/FrJrtESY.\nPlease contact your system administrator.\nAdd correct host key in /Users/jesusperezlorenzo/.ssh/known_hosts to get rid of this message.\nOffending ECDSA key in /Users/jesusperezlorenzo/.ssh/known_hosts:228\nHost key for libre-wuji-cp-0 has changed and you have requested strict checking.\nHost key verification failed.\nscp: Connection closed\n",
"tags": {
"type": "component",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"component": "os",
"server": "libre-wuji-cp-0",
"operation": "install"
}
}

23
crates/data/tasks/0b72338e-1d92-4370-8ac3-f0c725940c82.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "0b72338e-1d92-4370-8ac3-f0c725940c82",
"name": "component_install_democratic_csi",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_0b72338e-1d92-4370-8ac3-f0c725940c82.tar.gz.b64' > /tmp/democratic_csi.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/democratic_csi.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/democratic_csi && mkdir -p /tmp/democratic_csi && tar xzf /tmp/democratic_csi.tar.gz -C /tmp/democratic_csi && cd /tmp/democratic_csi && sudo bash install-democratic_csi.sh install ; rc=$?; rm -f /tmp/democratic_csi.tar.gz && rm -rf /tmp/democratic_csi; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_0b72338e-1d92-4370-8ac3-f0c725940c82.tar.gz.b64' /tmp/democratic_csi.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T02:44:06.586118Z",
"started_at": "2026-04-21T02:44:06.727424Z",
"completed_at": "2026-04-21T02:44:10.649693Z",
"output": "=== democratic-csi: v0.14.6 ===\n=== democratic-csi: installing helm ===\nDownloading https://get.helm.sh/helm-v3.20.2-linux-arm64.tar.gz\nVerifying checksum... Done.\nPreparing to install helm into /usr/local/bin\nhelm installed into /usr/local/bin/helm\n=== democratic-csi: nfs_server empty — deferred (namespace only) ===\nnamespace/democratic-csi created\n Set nfs_server to the private IP of the external_nfs node and re-run.\n",
"error": null,
"tags": {
"component": "democratic_csi",
"server": "libre-wuji-cp-0",
"operation": "install",
"type": "component",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji"
}
}

17
crates/data/tasks/0c6642ff-9cfc-4123-9c2d-b185029f02bf.json Normal file
View file

@ -0,0 +1,17 @@
{
"id": "0c6642ff-9cfc-4123-9c2d-b185029f02bf",
"name": "execute_servers_script_libre-wuji-cp-0",
"command": "bash",
"args": [
"-c",
"base64 -d < /tmp/orchestrator_script_0c6642ff-9cfc-4123-9c2d-b185029f02bf.tar.gz.b64 | gunzip | tar -xOf - script.sh | bash +x"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T00:47:28.933303Z",
"started_at": "2026-04-21T00:47:29.292291Z",
"completed_at": "2026-04-21T00:47:56.758101Z",
"output": "✓ Prerequisites: HCLOUD_TOKEN set, hcloud CLI available\n\n=== Managing SSH Keys ===\n✓ SSH public key found: /Users/jesusperezlorenzo/.ssh/htz_ops.pub\nChecking if SSH key 'htz_ops' exists in Hetzner...\n✓ SSH key htz_ops already exists with ID: 106168627\n\n=== SSH Key Management Complete ===\nSSH_KEY_ID: 106168627\nSSH_KEY_NAME: htz_ops\nEnvironment exported to: /tmp/provisioning-wuji-20260421-014729/.env\n\n=== Managing Network ===\n✓ Network config validated: 10.0.8.0/22 with subnet 10.0.8.0/24 in zone eu-central\nChecking if network 'wuwei' exists...\n✓ Network 'wuwei' already exists with ID: 11875940\n ✓ Subnet 10.0.8.0/24 already present\n\n=== Network Management Complete ===\nNETWORK_ID: 11875940\nNETWORK_NAME: wuwei\nEnvironment exported to: /tmp/provisioning-wuji-20260421-014729/.env\n\u001b[0;34m[INFO]\u001b[0m Step 1/3: Checking server 'libre-wuji-cp-0'...\n\u001b[0;34m[INFO]\u001b[0m Creating server 'libre-wuji-cp-0'...\n\u001b[0;32m[✓]\u001b[0m 127585130 libre-wuji-cp-0 createdIPv4: \n\u001b[0;32m[✓]\u001b[0m Server created: ID=127585130\n\u001b[0;34m[INFO]\u001b[0m Waiting for server 'libre-wuji-cp-0' to reach running state...\n\u001b[0;32m[✓]\u001b[0m Server 'libre-wuji-cp-0' is running\nServer 127585130 attached to Network 11875940\n\u001b[0;32m[✓]\u001b[0m Attached to network wuwei with IP 10.0.8.20\nResource protection enabled for Server 127585130\n\u001b[0;32m[✓]\u001b[0m Protection enabled: delete rebuild\n\u001b[0;34m[INFO]\u001b[0m Step 2/2: Attaching server to firewall 'librecloud-fw'...\nFirewall 10819279 applied to resource\n\u001b[0;32m[✓]\u001b[0m Server attached to firewall 'librecloud-fw'\n\u001b[0;32m[✓]\u001b[0m ✓ Server provisioning complete\n\u001b[0;34m[INFO]\u001b[0m Summary:\n\u001b[0;34m[INFO]\u001b[0m • Server: libre-wuji-cp-0 (ID: 127585130)\n\u001b[0;34m[INFO]\u001b[0m • Firewall: librecloud-fw\n\u001b[0;34m[INFO]\u001b[0m • Status: Ready for deployment\n",
"error": null,
"tags": {}
}

23
crates/data/tasks/0db48326-6bd1-4015-a0b9-5146e842c318.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "0db48326-6bd1-4015-a0b9-5146e842c318",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_0db48326-6bd1-4015-a0b9-5146e842c318.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_0db48326-6bd1-4015-a0b9-5146e842c318.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T04:42:26.575558Z",
"started_at": "2026-04-21T04:42:26.914686Z",
"completed_at": "2026-04-21T04:42:39.104765Z",
"output": "",
"error": null,
"tags": {
"server": "libre-wuji-wrk-0",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"type": "component",
"operation": "install",
"component": "kubernetes"
}
}

23
crates/data/tasks/1055cc82-4c97-45ab-9095-871d5a5bf254.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "1055cc82-4c97-45ab-9095-871d5a5bf254",
"name": "component_install_cilium",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_1055cc82-4c97-45ab-9095-871d5a5bf254.tar.gz.b64' > /tmp/cilium.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/cilium.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/cilium && mkdir -p /tmp/cilium && tar xzf /tmp/cilium.tar.gz -C /tmp/cilium && cd /tmp/cilium && sudo bash install-cilium.sh install ; rc=$?; rm -f /tmp/cilium.tar.gz && rm -rf /tmp/cilium; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_1055cc82-4c97-45ab-9095-871d5a5bf254.tar.gz.b64' /tmp/cilium.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-20T11:14:59.966046Z",
"started_at": "2026-04-20T11:15:00.300158Z",
"completed_at": "2026-04-20T11:15:06.758918Z",
"output": null,
"error": "Command execution failed: \nError: Unable to install Cilium: Kubernetes cluster unreachable: Get \"https://10.0.8.20:6443/version\": dial tcp 10.0.8.20:6443: connect: connection refused\n",
"tags": {
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"component": "cilium",
"server": "libre-wuji-cp-0",
"operation": "install",
"type": "component"
}
}

23
crates/data/tasks/12e0f651-c7ea-403d-b82f-84be04f899bd.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "12e0f651-c7ea-403d-b82f-84be04f899bd",
"name": "component_install_resolv",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_12e0f651-c7ea-403d-b82f-84be04f899bd.tar.gz.b64' > /tmp/resolv.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/resolv.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/resolv && mkdir -p /tmp/resolv && tar xzf /tmp/resolv.tar.gz -C /tmp/resolv && cd /tmp/resolv && sudo bash install-resolv.sh install ; rc=$?; rm -f /tmp/resolv.tar.gz && rm -rf /tmp/resolv; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_12e0f651-c7ea-403d-b82f-84be04f899bd.tar.gz.b64' /tmp/resolv.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T02:14:25.735155Z",
"started_at": "2026-04-21T02:14:25.900745Z",
"completed_at": "2026-04-21T02:14:28.703065Z",
"output": "",
"error": null,
"tags": {
"type": "component",
"server": "libre-wuji-wrk-0",
"component": "resolv",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"operation": "install"
}
}

23
crates/data/tasks/147f51f7-cf97-4952-84da-4e32c8473e8d.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "147f51f7-cf97-4952-84da-4e32c8473e8d",
"name": "component_install_hetzner_csi",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_147f51f7-cf97-4952-84da-4e32c8473e8d.tar.gz.b64' > /tmp/hetzner_csi.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/hetzner_csi.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/hetzner_csi && mkdir -p /tmp/hetzner_csi && tar xzf /tmp/hetzner_csi.tar.gz -C /tmp/hetzner_csi && cd /tmp/hetzner_csi && sudo bash install-hetzner_csi.sh install ; rc=$?; rm -f /tmp/hetzner_csi.tar.gz && rm -rf /tmp/hetzner_csi; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_147f51f7-cf97-4952-84da-4e32c8473e8d.tar.gz.b64' /tmp/hetzner_csi.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T02:26:38.339315Z",
"started_at": "2026-04-21T02:26:38.692203Z",
"completed_at": "2026-04-21T02:26:52.261401Z",
"output": "=== hetzner-csi: checking prerequisites ===\n=== hetzner-csi: deploying v2.9.0 ===\nserviceaccount/hcloud-csi-controller created\nclusterrole.rbac.authorization.k8s.io/hcloud-csi-controller created\nclusterrolebinding.rbac.authorization.k8s.io/hcloud-csi-controller created\nservice/hcloud-csi-controller-metrics created\nservice/hcloud-csi-node-metrics created\ndaemonset.apps/hcloud-csi-node created\ndeployment.apps/hcloud-csi-controller created\ncsidriver.storage.k8s.io/csi.hetzner.cloud created\n=== hetzner-csi: creating StorageClass hcloud-volumes (reclaimPolicy=Retain) ===\nstorageclass.storage.k8s.io/hcloud-volumes created\nstorageclass.storage.k8s.io/hcloud-volumes annotated\n=== hetzner-csi: waiting for controller pod ===\nWaiting for deployment \"hcloud-csi-controller\" rollout to finish: 0 of 1 updated replicas are available...\ndeployment \"hcloud-csi-controller\" successfully rolled out\n=== hetzner-csi: ready (StorageClass=hcloud-volumes, policy=Retain, fs=ext4) ===\n",
"error": null,
"tags": {
"server": "libre-wuji-cp-0",
"operation": "install",
"type": "component",
"component": "hetzner_csi",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji"
}
}

23
crates/data/tasks/17ebdbae-5a51-4bc0-881b-24c168c70396.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "17ebdbae-5a51-4bc0-881b-24c168c70396",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_17ebdbae-5a51-4bc0-881b-24c168c70396.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_17ebdbae-5a51-4bc0-881b-24c168c70396.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T05:10:57.138024Z",
"started_at": "2026-04-21T05:10:57.371587Z",
"completed_at": "2026-04-21T05:11:09.302706Z",
"output": "",
"error": null,
"tags": {
"operation": "install",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"type": "component",
"component": "kubernetes",
"server": "libre-wuji-wrk-0"
}
}

23
crates/data/tasks/18c5c3f1-82c7-460c-a7c4-463e1c17017f.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "18c5c3f1-82c7-460c-a7c4-463e1c17017f",
"name": "component_install_containerd",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_18c5c3f1-82c7-460c-a7c4-463e1c17017f.tar.gz.b64' > /tmp/containerd.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/containerd.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/containerd && mkdir -p /tmp/containerd && tar xzf /tmp/containerd.tar.gz -C /tmp/containerd && cd /tmp/containerd && sudo bash install-containerd.sh install ; rc=$?; rm -f /tmp/containerd.tar.gz && rm -rf /tmp/containerd; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_18c5c3f1-82c7-460c-a7c4-463e1c17017f.tar.gz.b64' /tmp/containerd.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-20T10:28:47.849117Z",
"started_at": "2026-04-20T10:28:48.291233Z",
"completed_at": "2026-04-20T10:28:55.043273Z",
"output": null,
"error": "Command execution failed: tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'\ntar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'\ntar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'\ntar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'\ntar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'\ntar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'\ntar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'\ninstall-containerd.sh: line 42: _common_service_stop: command not found\ninstall-containerd.sh: line 93: _common_service_start: command not found\n",
"tags": {
"type": "component",
"server": "libre-wuji-cp-0",
"component": "containerd",
"operation": "install",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji"
}
}

23
crates/data/tasks/1de586af-a397-47c3-bff4-88e77d138496.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "1de586af-a397-47c3-bff4-88e77d138496",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_1de586af-a397-47c3-bff4-88e77d138496.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_1de586af-a397-47c3-bff4-88e77d138496.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-20T11:01:10.252268Z",
"started_at": "2026-04-20T11:01:10.486569Z",
"completed_at": "2026-04-20T11:01:13.593520Z",
"output": null,
"error": "Command execution failed: resources/kubeadm-config.yaml not found\n",
"tags": {
"component": "kubernetes",
"server": "libre-wuji-cp-0",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"operation": "install",
"type": "component"
}
}

23
crates/data/tasks/1fd7f217-ea89-442b-8983-e21c1f07b887.json Normal file
View file

File diff suppressed because one or more lines are too long

23
crates/data/tasks/21a41f63-1a83-4178-8e4a-82211f07883c.json Normal file
View file

File diff suppressed because one or more lines are too long

23
crates/data/tasks/21a7ed02-b4f0-4f09-98cc-8406a98cd0de.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "21a7ed02-b4f0-4f09-98cc-8406a98cd0de",
"name": "component_install_external_nfs",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_21a7ed02-b4f0-4f09-98cc-8406a98cd0de.tar.gz.b64' > /tmp/external_nfs.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/external_nfs.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/external_nfs && mkdir -p /tmp/external_nfs && tar xzf /tmp/external_nfs.tar.gz -C /tmp/external_nfs && cd /tmp/external_nfs && sudo bash install-external_nfs.sh install ; rc=$?; rm -f /tmp/external_nfs.tar.gz && rm -rf /tmp/external_nfs; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_21a7ed02-b4f0-4f09-98cc-8406a98cd0de.tar.gz.b64' /tmp/external_nfs.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-21T02:43:42.534304Z",
"started_at": "2026-04-21T02:43:42.637603Z",
"completed_at": "2026-04-21T02:43:45.247118Z",
"output": null,
"error": "Command execution failed: bash: install-external_nfs.sh: No such file or directory\n",
"tags": {
"server": "libre-wuji-cp-0",
"type": "component",
"operation": "install",
"component": "external_nfs",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji"
}
}

23
crates/data/tasks/21d299bc-3605-467b-bc88-54694ce92eca.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "21d299bc-3605-467b-bc88-54694ce92eca",
"name": "component_install_os",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_21d299bc-3605-467b-bc88-54694ce92eca.tar.gz.b64' > /tmp/os.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/os.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/os && mkdir -p /tmp/os && tar xzf /tmp/os.tar.gz -C /tmp/os && cd /tmp/os && sudo bash install-os.sh install ; rc=$?; rm -f /tmp/os.tar.gz && rm -rf /tmp/os; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_21d299bc-3605-467b-bc88-54694ce92eca.tar.gz.b64' /tmp/os.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T01:55:04.220548Z",
"started_at": "2026-04-21T01:55:04.481214Z",
"completed_at": "2026-04-21T01:55:11.473780Z",
"output": "APT::Get::Update::SourceListWarnings::NonFreeFirmware \"false\";\nHit:1 http://mirror.hetzner.com/debian/packages bookworm InRelease\nHit:2 http://mirror.hetzner.com/debian/packages bookworm-backports InRelease\nHit:3 http://mirror.hetzner.com/debian/packages bookworm-updates InRelease\nHit:4 http://mirror.hetzner.com/debian/security bookworm-security InRelease\nHit:5 http://deb.debian.org/debian bookworm InRelease\nHit:6 http://security.debian.org/debian-security bookworm-security InRelease\nHit:7 http://deb.debian.org/debian bookworm-updates InRelease\nReading package lists...\nReading package lists...\nBuilding dependency tree...\nReading state information...\nCalculating upgrade...\nThe following packages have been kept back:\n linux-image-arm64\n0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.\nReading package lists...\nBuilding dependency tree...\nReading state information...\n0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.\n",
"error": null,
"tags": {
"component": "os",
"server": "libre-wuji-wrk-0",
"type": "component",
"operation": "install",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji"
}
}

17
crates/data/tasks/24c4352d-ab7c-4756-95a4-8b9c23712d71.json Normal file
View file

@ -0,0 +1,17 @@
{
"id": "24c4352d-ab7c-4756-95a4-8b9c23712d71",
"name": "execute_servers_script_libre-wuji-wrk-0",
"command": "bash",
"args": [
"-c",
"base64 -d < /tmp/orchestrator_script_24c4352d-ab7c-4756-95a4-8b9c23712d71.tar.gz.b64 | gunzip | tar -xOf - script.sh | bash +x"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T00:42:51.560222Z",
"started_at": "2026-04-21T00:42:51.712428Z",
"completed_at": "2026-04-21T00:43:34.296729Z",
"output": "✓ Prerequisites: HCLOUD_TOKEN set, hcloud CLI available\n\n=== Managing SSH Keys ===\n✓ SSH public key found: /Users/jesusperezlorenzo/.ssh/htz_ops.pub\nChecking if SSH key 'htz_ops' exists in Hetzner...\n✓ SSH key htz_ops already exists with ID: 106168627\n\n=== SSH Key Management Complete ===\nSSH_KEY_ID: 106168627\nSSH_KEY_NAME: htz_ops\nEnvironment exported to: /tmp/provisioning-wuji-20260421-014251/.env\n\n=== Managing Network ===\n✓ Network config validated: 10.0.8.0/22 with subnet 10.0.8.0/24 in zone eu-central\nChecking if network 'wuwei' exists...\n✓ Network 'wuwei' already exists with ID: 11875940\n ✓ Subnet 10.0.8.0/24 already present\n\n=== Network Management Complete ===\nNETWORK_ID: 11875940\nNETWORK_NAME: wuwei\nEnvironment exported to: /tmp/provisioning-wuji-20260421-014251/.env\n\u001b[0;34m[INFO]\u001b[0m Step 1/3: Checking server 'libre-wuji-wrk-0'...\n\u001b[0;34m[INFO]\u001b[0m Creating server 'libre-wuji-wrk-0'...\n\u001b[0;32m[✓]\u001b[0m 127584497 libre-wuji-wrk-0 createdIPv4: \n\u001b[0;32m[✓]\u001b[0m Server created: ID=127584497\n\u001b[0;34m[INFO]\u001b[0m Waiting for server 'libre-wuji-wrk-0' to reach running state...\n\u001b[0;32m[✓]\u001b[0m Server 'libre-wuji-wrk-0' is running\nServer 127584497 attached to Network 11875940\n\u001b[0;32m[✓]\u001b[0m Attached to network wuwei with IP 10.0.8.24\nResource protection enabled for Server 127584497\n\u001b[0;32m[✓]\u001b[0m Protection enabled: delete rebuild\n\u001b[0;34m[INFO]\u001b[0m Step 2/2: Attaching server to firewall 'librecloud-fw'...\nFirewall 10819279 applied to resource\n\u001b[0;32m[✓]\u001b[0m Server attached to firewall 'librecloud-fw'\n\u001b[0;34m[INFO]\u001b[0m Step 3/3: Assigning floating IP 'librecloud-fip-mail' to 'libre-wuji-wrk-0'...\nFloating IP 127388996 assigned to Server 127584497\n\u001b[0;32m[✓]\u001b[0m Assigned floating IP 'librecloud-fip-mail' (id=127388996) → libre-wuji-wrk-0 (id=127584497)\n\u001b[0;32m[✓]\u001b[0m ✓ Server provisioning complete\n\u001b[0;34m[INFO]\u001b[0m Summary:\n\u001b[0;34m[INFO]\u001b[0m • Server: libre-wuji-wrk-0 (ID: 127584497)\n\u001b[0;34m[INFO]\u001b[0m • Firewall: librecloud-fw\n\u001b[0;34m[INFO]\u001b[0m • Float IP: librecloud-fip-mail (id=127388996)\n\u001b[0;34m[INFO]\u001b[0m • Status: Ready for deployment\n",
"error": null,
"tags": {}
}

23
crates/data/tasks/29a4a214-c961-4b14-8256-afaf1541ca81.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "29a4a214-c961-4b14-8256-afaf1541ca81",
"name": "component_install_hetzner_csi",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_29a4a214-c961-4b14-8256-afaf1541ca81.tar.gz.b64' > /tmp/hetzner_csi.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/hetzner_csi.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/hetzner_csi && mkdir -p /tmp/hetzner_csi && tar xzf /tmp/hetzner_csi.tar.gz -C /tmp/hetzner_csi && cd /tmp/hetzner_csi && sudo bash install-hetzner_csi.sh install ; rc=$?; rm -f /tmp/hetzner_csi.tar.gz && rm -rf /tmp/hetzner_csi; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_29a4a214-c961-4b14-8256-afaf1541ca81.tar.gz.b64' /tmp/hetzner_csi.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-21T02:16:01.781982Z",
"started_at": "2026-04-21T02:16:02.265091Z",
"completed_at": "2026-04-21T02:16:05.102310Z",
"output": null,
"error": "Command execution failed: ERROR: Secret 'hcloud' not found in kube-system.\n Create it first: kubectl -n kube-system create secret generic hcloud --from-literal=token=<HCLOUD_TOKEN>\n",
"tags": {
"operation": "install",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"type": "component",
"component": "hetzner_csi",
"server": "libre-wuji-cp-0"
}
}

23
crates/data/tasks/2af968fa-1f9e-44b7-85d6-ea913de2142e.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "2af968fa-1f9e-44b7-85d6-ea913de2142e",
"name": "component_install_crun",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_2af968fa-1f9e-44b7-85d6-ea913de2142e.tar.gz.b64' > /tmp/crun.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/crun.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/crun && mkdir -p /tmp/crun && tar xzf /tmp/crun.tar.gz -C /tmp/crun && cd /tmp/crun && sudo bash install-crun.sh install ; rc=$?; rm -f /tmp/crun.tar.gz && rm -rf /tmp/crun; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_2af968fa-1f9e-44b7-85d6-ea913de2142e.tar.gz.b64' /tmp/crun.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T01:44:12.211853Z",
"started_at": "2026-04-21T01:44:12.491759Z",
"completed_at": "2026-04-21T01:44:15.870915Z",
"output": "",
"error": null,
"tags": {
"operation": "install",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"server": "libre-wuji-cp-0",
"component": "crun",
"type": "component"
}
}

23
crates/data/tasks/3bc38fc5-9d51-4c44-a761-7c06948606f6.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "3bc38fc5-9d51-4c44-a761-7c06948606f6",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_3bc38fc5-9d51-4c44-a761-7c06948606f6.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp /tmp/kubernetes.tar.gz 'root@libre-wuji-wkr-0:/tmp/' && ssh 'root@libre-wuji-wkr-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_3bc38fc5-9d51-4c44-a761-7c06948606f6.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-21T03:12:56.664977Z",
"started_at": "2026-04-21T03:12:56.887653Z",
"completed_at": "2026-04-21T03:12:56.927657Z",
"output": null,
"error": "Command execution failed: ssh: Could not resolve hostname libre-wuji-wkr-0: nodename nor servname provided, or not known\nscp: Connection closed\n",
"tags": {
"operation": "install",
"type": "component",
"server": "libre-wuji-wkr-0",
"component": "kubernetes",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji"
}
}

23
crates/data/tasks/3c4b73ce-4ec0-4c63-b8a0-f04408f82c67.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "3c4b73ce-4ec0-4c63-b8a0-f04408f82c67",
"name": "component_install_hetzner_csi",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_3c4b73ce-4ec0-4c63-b8a0-f04408f82c67.tar.gz.b64' > /tmp/hetzner_csi.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/hetzner_csi.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/hetzner_csi && mkdir -p /tmp/hetzner_csi && tar xzf /tmp/hetzner_csi.tar.gz -C /tmp/hetzner_csi && cd /tmp/hetzner_csi && sudo bash install-hetzner_csi.sh install ; rc=$?; rm -f /tmp/hetzner_csi.tar.gz && rm -rf /tmp/hetzner_csi; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_3c4b73ce-4ec0-4c63-b8a0-f04408f82c67.tar.gz.b64' /tmp/hetzner_csi.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-21T02:23:24.029018Z",
"started_at": "2026-04-21T02:23:24.447791Z",
"completed_at": "2026-04-21T02:23:27.237129Z",
"output": null,
"error": "Command execution failed: ERROR: Secret 'hcloud' not found in kube-system.\n Create it first: kubectl -n kube-system create secret generic hcloud --from-literal=token=<HCLOUD_TOKEN>\n",
"tags": {
"component": "hetzner_csi",
"type": "component",
"server": "libre-wuji-cp-0",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"operation": "install"
}
}

23
crates/data/tasks/3c7369a3-b249-43e3-8559-f7de35dc7fe6.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "3c7369a3-b249-43e3-8559-f7de35dc7fe6",
"name": "component_install_youki",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_3c7369a3-b249-43e3-8559-f7de35dc7fe6.tar.gz.b64' > /tmp/youki.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/youki.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/youki && mkdir -p /tmp/youki && tar xzf /tmp/youki.tar.gz -C /tmp/youki && cd /tmp/youki && sudo bash install-youki.sh install ; rc=$?; rm -f /tmp/youki.tar.gz && rm -rf /tmp/youki; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_3c7369a3-b249-43e3-8559-f7de35dc7fe6.tar.gz.b64' /tmp/youki.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-20T10:14:12.601474Z",
"started_at": "2026-04-20T10:14:12.933804Z",
"completed_at": "2026-04-20T10:14:16.513877Z",
"output": "",
"error": null,
"tags": {
"component": "youki",
"server": "libre-wuji-wrk-0",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"operation": "install",
"type": "component"
}
}

23
crates/data/tasks/3cb42c41-866d-428a-887f-341fb816a1b2.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "3cb42c41-866d-428a-887f-341fb816a1b2",
"name": "component_install_crun",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_3cb42c41-866d-428a-887f-341fb816a1b2.tar.gz.b64' > /tmp/crun.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/crun.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/crun && mkdir -p /tmp/crun && tar xzf /tmp/crun.tar.gz -C /tmp/crun && cd /tmp/crun && sudo bash install-crun.sh install ; rc=$?; rm -f /tmp/crun.tar.gz && rm -rf /tmp/crun; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_3cb42c41-866d-428a-887f-341fb816a1b2.tar.gz.b64' /tmp/crun.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-20T10:13:45.058247Z",
"started_at": "2026-04-20T10:13:45.327751Z",
"completed_at": "2026-04-20T10:13:48.708291Z",
"output": "",
"error": null,
"tags": {
"component": "crun",
"type": "component",
"operation": "install",
"server": "libre-wuji-wrk-0",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji"
}
}

23
crates/data/tasks/3ed38c0f-9920-4dc8-b746-0b783cbaab0e.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "3ed38c0f-9920-4dc8-b746-0b783cbaab0e",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_3ed38c0f-9920-4dc8-b746-0b783cbaab0e.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_3ed38c0f-9920-4dc8-b746-0b783cbaab0e.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T03:28:27.448853Z",
"started_at": "2026-04-21T03:28:27.459746Z",
"completed_at": "2026-04-21T03:28:32.081653Z",
"output": "No k8s_join.sh found\n",
"error": null,
"tags": {
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"server": "libre-wuji-wrk-0",
"type": "component",
"component": "kubernetes",
"operation": "install"
}
}

23
crates/data/tasks/3f27a39a-303d-4ef3-a1fb-fdff3b69d530.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "3f27a39a-303d-4ef3-a1fb-fdff3b69d530",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_3f27a39a-303d-4ef3-a1fb-fdff3b69d530.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_3f27a39a-303d-4ef3-a1fb-fdff3b69d530.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T03:54:35.730627Z",
"started_at": "2026-04-21T03:54:35.923566Z",
"completed_at": "2026-04-21T03:54:42.449188Z",
"output": "",
"error": null,
"tags": {
"component": "kubernetes",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"operation": "install",
"server": "libre-wuji-wrk-0",
"type": "component"
}
}

23
crates/data/tasks/43a62145-7402-4a4b-8aa4-76befe884ebd.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "43a62145-7402-4a4b-8aa4-76befe884ebd",
"name": "component_reinstall_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_43a62145-7402-4a4b-8aa4-76befe884ebd.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh reinstall ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_43a62145-7402-4a4b-8aa4-76befe884ebd.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T03:04:23.522971Z",
"started_at": "2026-04-21T03:04:23.925109Z",
"completed_at": "2026-04-21T03:04:27.724227Z",
"output": "Warning: probe patch failed\n2026_04_21_030427 | apiserver probes patched: startup=300s liveness=120s readiness=15s\n/etc/kubernetes/admin.conf not found\n",
"error": null,
"tags": {
"component": "kubernetes",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"type": "component",
"server": "libre-wuji-cp-0",
"operation": "reinstall"
}
}

23
crates/data/tasks/44a67063-a05a-4126-9aac-5bb52fc850f9.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "44a67063-a05a-4126-9aac-5bb52fc850f9",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_44a67063-a05a-4126-9aac-5bb52fc850f9.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_44a67063-a05a-4126-9aac-5bb52fc850f9.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T03:17:07.185465Z",
"started_at": "2026-04-21T03:17:07.354693Z",
"completed_at": "2026-04-21T03:17:16.032753Z",
"output": "No k8s_join.sh found\n",
"error": null,
"tags": {
"server": "libre-wuji-wrk-0",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"operation": "install",
"type": "component",
"component": "kubernetes"
}
}

23
crates/data/tasks/44cf7c11-770c-4f78-a810-06d4ae105a8e.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "44cf7c11-770c-4f78-a810-06d4ae105a8e",
"name": "component_install_democratic_csi",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_44cf7c11-770c-4f78-a810-06d4ae105a8e.tar.gz.b64' > /tmp/democratic_csi.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/democratic_csi.tar.gz 'root@libre-wuji-cp-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-cp-0' 'rm -rf /tmp/democratic_csi && mkdir -p /tmp/democratic_csi && tar xzf /tmp/democratic_csi.tar.gz -C /tmp/democratic_csi && cd /tmp/democratic_csi && sudo bash install-democratic_csi.sh install ; rc=$?; rm -f /tmp/democratic_csi.tar.gz && rm -rf /tmp/democratic_csi; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_44cf7c11-770c-4f78-a810-06d4ae105a8e.tar.gz.b64' /tmp/democratic_csi.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T03:13:05.041471Z",
"started_at": "2026-04-21T03:13:05.419506Z",
"completed_at": "2026-04-21T03:13:08.431367Z",
"output": "=== democratic-csi: v0.14.6 ===\n=== democratic-csi: nfs_server empty — deferred (namespace only) ===\nnamespace/democratic-csi unchanged\n Set nfs_server to the private IP of the external_nfs node and re-run.\n",
"error": null,
"tags": {
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"component": "democratic_csi",
"operation": "install",
"type": "component",
"server": "libre-wuji-cp-0"
}
}

23
crates/data/tasks/49b985ef-68a6-4f56-ab2d-20a523411fe8.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "49b985ef-68a6-4f56-ab2d-20a523411fe8",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_49b985ef-68a6-4f56-ab2d-20a523411fe8.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_49b985ef-68a6-4f56-ab2d-20a523411fe8.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T02:39:51.849617Z",
"started_at": "2026-04-21T02:39:52.264364Z",
"completed_at": "2026-04-21T02:39:55.544779Z",
"output": "No k8s_join.sh found\n",
"error": null,
"tags": {
"component": "kubernetes",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"type": "component",
"server": "libre-wuji-wrk-0",
"operation": "install"
}
}

23
crates/data/tasks/4a072d77-e98e-4776-a7cc-2d64c511162b.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "4a072d77-e98e-4776-a7cc-2d64c511162b",
"name": "component_install_fip",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_4a072d77-e98e-4776-a7cc-2d64c511162b.tar.gz.b64' > /tmp/fip.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/fip.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/fip && mkdir -p /tmp/fip && tar xzf /tmp/fip.tar.gz -C /tmp/fip && cd /tmp/fip && sudo bash install-fip.sh install ; rc=$?; rm -f /tmp/fip.tar.gz && rm -rf /tmp/fip; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_4a072d77-e98e-4776-a7cc-2d64c511162b.tar.gz.b64' /tmp/fip.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-20T10:10:19.412561Z",
"started_at": "2026-04-20T10:10:19.538669Z",
"completed_at": "2026-04-20T10:10:23.400232Z",
"output": "Written /etc/network/interfaces.d/60-floating-ip\nFloating IP 49.12.115.133 active on eth0\n",
"error": null,
"tags": {
"server": "libre-wuji-wrk-0",
"component": "fip",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"operation": "install",
"type": "component"
}
}

23
crates/data/tasks/4df21b80-697c-45af-9d00-c13d8a8ae975.json Normal file
View file

File diff suppressed because one or more lines are too long

17
crates/data/tasks/5006edf3-c986-443f-9610-5b5cb9224269.json Normal file
View file

@ -0,0 +1,17 @@
{
"id": "5006edf3-c986-443f-9610-5b5cb9224269",
"name": "execute_servers_script_libre-wuji-cp-0",
"command": "bash",
"args": [
"-c",
"base64 -d < /tmp/orchestrator_script_5006edf3-c986-443f-9610-5b5cb9224269.tar.gz.b64 | gunzip | tar -xOf - script.sh | bash +x"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T00:42:51.560218Z",
"started_at": "2026-04-21T00:42:51.712434Z",
"completed_at": "2026-04-21T00:43:20.037945Z",
"output": "✓ Prerequisites: HCLOUD_TOKEN set, hcloud CLI available\n\n=== Managing SSH Keys ===\n✓ SSH public key found: /Users/jesusperezlorenzo/.ssh/htz_ops.pub\nChecking if SSH key 'htz_ops' exists in Hetzner...\n✓ SSH key htz_ops already exists with ID: 106168627\n\n=== SSH Key Management Complete ===\nSSH_KEY_ID: 106168627\nSSH_KEY_NAME: htz_ops\nEnvironment exported to: /tmp/provisioning-wuji-20260421-014251/.env\n\n=== Managing Network ===\n✓ Network config validated: 10.0.8.0/22 with subnet 10.0.8.0/24 in zone eu-central\nChecking if network 'wuwei' exists...\n✓ Network 'wuwei' already exists with ID: 11875940\n ✓ Subnet 10.0.8.0/24 already present\n\n=== Network Management Complete ===\nNETWORK_ID: 11875940\nNETWORK_NAME: wuwei\nEnvironment exported to: /tmp/provisioning-wuji-20260421-014251/.env\n\u001b[0;34m[INFO]\u001b[0m Step 1/3: Checking server 'libre-wuji-cp-0'...\n\u001b[0;34m[INFO]\u001b[0m Creating server 'libre-wuji-cp-0'...\n\u001b[0;32m[✓]\u001b[0m 127584498 libre-wuji-cp-0 createdIPv4: \n\u001b[0;32m[✓]\u001b[0m Server created: ID=127584498\n\u001b[0;34m[INFO]\u001b[0m Waiting for server 'libre-wuji-cp-0' to reach running state...\n\u001b[0;32m[✓]\u001b[0m Server 'libre-wuji-cp-0' is running\nServer 127584498 attached to Network 11875940\n\u001b[0;32m[✓]\u001b[0m Attached to network wuwei with IP 10.0.8.20\nResource protection enabled for Server 127584498\n\u001b[0;32m[✓]\u001b[0m Protection enabled: delete rebuild\n\u001b[0;34m[INFO]\u001b[0m Step 2/2: Attaching server to firewall 'librecloud-fw'...\nFirewall 10819279 applied to resource\n\u001b[0;32m[✓]\u001b[0m Server attached to firewall 'librecloud-fw'\n\u001b[0;32m[✓]\u001b[0m ✓ Server provisioning complete\n\u001b[0;34m[INFO]\u001b[0m Summary:\n\u001b[0;34m[INFO]\u001b[0m • Server: libre-wuji-cp-0 (ID: 127584498)\n\u001b[0;34m[INFO]\u001b[0m • Firewall: librecloud-fw\n\u001b[0;34m[INFO]\u001b[0m • Status: Ready for deployment\n",
"error": null,
"tags": {}
}

23
crates/data/tasks/50f27d6d-65a0-4400-88e6-06ffa9bf5ecc.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "50f27d6d-65a0-4400-88e6-06ffa9bf5ecc",
"name": "component_install_kubernetes",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_50f27d6d-65a0-4400-88e6-06ffa9bf5ecc.tar.gz.b64' > /tmp/kubernetes.tar.gz && scp -i '/Users/jesusperezlorenzo/.ssh/htz_ops' /tmp/kubernetes.tar.gz 'root@libre-wuji-wrk-0:/tmp/' && ssh -i '/Users/jesusperezlorenzo/.ssh/htz_ops' 'root@libre-wuji-wrk-0' 'rm -rf /tmp/kubernetes && mkdir -p /tmp/kubernetes && tar xzf /tmp/kubernetes.tar.gz -C /tmp/kubernetes && cd /tmp/kubernetes && sudo bash install-kubernetes.sh install ; rc=$?; rm -f /tmp/kubernetes.tar.gz && rm -rf /tmp/kubernetes; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_50f27d6d-65a0-4400-88e6-06ffa9bf5ecc.tar.gz.b64' /tmp/kubernetes.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T03:52:48.460149Z",
"started_at": "2026-04-21T03:52:48.513324Z",
"completed_at": "2026-04-21T03:52:59.984673Z",
"output": "",
"error": null,
"tags": {
"type": "component",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"operation": "install",
"component": "kubernetes",
"server": "libre-wuji-wrk-0"
}
}

17
crates/data/tasks/52c80f88-1fa5-4c3f-8e15-b645d0b7abdd.json Normal file
View file

@ -0,0 +1,17 @@
{
"id": "52c80f88-1fa5-4c3f-8e15-b645d0b7abdd",
"name": "execute_servers_script_libre-wuji-cp-0",
"command": "bash",
"args": [
"-c",
"base64 -d < /tmp/orchestrator_script_52c80f88-1fa5-4c3f-8e15-b645d0b7abdd.tar.gz.b64 | gunzip | tar -xOf - script.sh | bash +x"
],
"dependencies": [],
"status": "Completed",
"created_at": "2026-04-21T01:07:42.324101Z",
"started_at": "2026-04-21T01:07:42.517244Z",
"completed_at": "2026-04-21T01:08:10.615549Z",
"output": "✓ Prerequisites: HCLOUD_TOKEN set, hcloud CLI available\n\n=== Managing SSH Keys ===\n✓ SSH public key found: /Users/jesusperezlorenzo/.ssh/htz_ops.pub\nChecking if SSH key 'htz_ops' exists in Hetzner...\n✓ SSH key htz_ops already exists with ID: 106168627\n\n=== SSH Key Management Complete ===\nSSH_KEY_ID: 106168627\nSSH_KEY_NAME: htz_ops\nEnvironment exported to: /tmp/provisioning-wuji-20260421-020742/.env\n\n=== Managing Network ===\n✓ Network config validated: 10.0.8.0/22 with subnet 10.0.8.0/24 in zone eu-central\nChecking if network 'wuwei' exists...\n✓ Network 'wuwei' already exists with ID: 11875940\n ✓ Subnet 10.0.8.0/24 already present\n\n=== Network Management Complete ===\nNETWORK_ID: 11875940\nNETWORK_NAME: wuwei\nEnvironment exported to: /tmp/provisioning-wuji-20260421-020742/.env\n\u001b[0;34m[INFO]\u001b[0m Step 1/3: Checking server 'libre-wuji-cp-0'...\n\u001b[0;34m[INFO]\u001b[0m Creating server 'libre-wuji-cp-0'...\n\u001b[0;32m[✓]\u001b[0m 127586862 libre-wuji-cp-0 createdIPv4: \n\u001b[0;32m[✓]\u001b[0m Server created: ID=127586862\n\u001b[0;34m[INFO]\u001b[0m Waiting for server 'libre-wuji-cp-0' to reach running state...\n\u001b[0;32m[✓]\u001b[0m Server 'libre-wuji-cp-0' is running\nServer 127586862 attached to Network 11875940\n\u001b[0;32m[✓]\u001b[0m Attached to network wuwei with IP 10.0.8.20\nResource protection enabled for Server 127586862\n\u001b[0;32m[✓]\u001b[0m Protection enabled: delete rebuild\n\u001b[0;34m[INFO]\u001b[0m Step 2/2: Attaching server to firewall 'librecloud-fw'...\nFirewall 10819279 applied to resource\n\u001b[0;32m[✓]\u001b[0m Server attached to firewall 'librecloud-fw'\n\u001b[0;32m[✓]\u001b[0m ✓ Server provisioning complete\n\u001b[0;34m[INFO]\u001b[0m Summary:\n\u001b[0;34m[INFO]\u001b[0m • Server: libre-wuji-cp-0 (ID: 127586862)\n\u001b[0;34m[INFO]\u001b[0m • Firewall: librecloud-fw\n\u001b[0;34m[INFO]\u001b[0m • Status: Ready for deployment\n",
"error": null,
"tags": {}
}

23
crates/data/tasks/5406f205-24f1-467c-90cc-7e917b7e3293.json Normal file
View file

@ -0,0 +1,23 @@
{
"id": "5406f205-24f1-467c-90cc-7e917b7e3293",
"name": "component_install_containerd",
"command": "bash",
"args": [
"-c",
"base64 -d < '/tmp/orchestrator_comp_5406f205-24f1-467c-90cc-7e917b7e3293.tar.gz.b64' > /tmp/containerd.tar.gz && scp /tmp/containerd.tar.gz 'root@libre-wuji-wk-0:/tmp/' && ssh 'root@libre-wuji-wk-0' 'rm -rf /tmp/containerd && mkdir -p /tmp/containerd && tar xzf /tmp/containerd.tar.gz -C /tmp/containerd && cd /tmp/containerd && sudo bash install-containerd.sh install ; rc=$?; rm -f /tmp/containerd.tar.gz && rm -rf /tmp/containerd; exit $rc' ; rc=$?; rm -f '/tmp/orchestrator_comp_5406f205-24f1-467c-90cc-7e917b7e3293.tar.gz.b64' /tmp/containerd.tar.gz; exit $rc"
],
"dependencies": [],
"status": "Failed",
"created_at": "2026-04-20T10:29:18.910354Z",
"started_at": "2026-04-20T10:29:19.413672Z",
"completed_at": "2026-04-20T10:29:19.627169Z",
"output": null,
"error": "Command execution failed: ssh: Could not resolve hostname libre-wuji-wk-0: nodename nor servname provided, or not known\nscp: Connection closed\n",
"tags": {
"component": "containerd",
"workspace": "/Users/Akasha/project-provisioning/workspaces/libre-wuji",
"type": "component",
"server": "libre-wuji-wk-0",
"operation": "install"
}
}

Some files were not shown because too many files have changed in this diff Show more