# Daemon-CLI Phases 3-7 Implementation Roadmap ## Current Status: Phases 1-2 Complete ✅ All foundation is in place. The API server is running with stub implementations ready for ecosystem crate integration. --- ## Phase 3: Dual-Mode CLI Client ### Files to Create/Modify - `src/cli/mod.rs` - CLI module structure - `src/cli/client.rs` - HTTP client for daemon communication - `src/cli/offline.rs` - Direct library call mode - `src/cli/output.rs` - Output formatting (table, JSON, YAML) - `src/cli/commands/daemon.rs` - Daemon management commands - `src/cli/commands/valida.rs` - Validation commands - `src/cli/commands/runtime.rs` - Runtime commands - `src/cli/commands/encrypt.rs` - Encryption commands - `src/bin/provctl.rs` - Update with real implementations ### Implementation Pattern ```rust // HTTP Mode: Connect to daemon async fn execute_validation(args: ValidaArgs) -> Result<()> { let client = DaemonClient::new("http://localhost:9090")?; let result = client.valida().validate(&args).await?; print_result(result)?; Ok(()) } // Offline Mode: Direct library calls async fn execute_validation_offline(args: ValidaArgs) -> Result<()> { use valida::RuleSet; let rules = RuleSet::load_from_file(&args.rules_file)?; let result = rules.validate(&args.config, args.phase)?; print_result(result)?; Ok(()) } // Auto-detection async fn execute(cmd: Command) -> Result<()> { if DaemonClient::check_available().await { execute_with_daemon(cmd).await } else { execute_offline(cmd).await } } ``` ### Key Features - HTTP client using reqwest - Daemon availability detection via health check - Graceful fallback to offline mode - Consistent CLI interface regardless of mode - JSON output for scripting --- ## Phase 4: Ecosystem Crate Integration ### Files to Create/Modify - `src/orchestration/mod.rs` - Operation registry - `src/orchestration/operation.rs` - Operation trait - `src/orchestration/valida.rs` - Validation operations - `src/orchestration/runtime.rs` - Runtime operations - `src/orchestration/encrypt.rs` - Encryption operations - `src/orchestration/init_servs.rs` - Service management - `src/api/handlers/valida.rs` - Replace stubs - `src/api/handlers/runtime.rs` - Replace stubs - `src/api/handlers/encrypt.rs` - Replace stubs ### Pattern Example: Valida Integration ```rust // src/orchestration/valida.rs #[async_trait] impl EcosystemOperation for ValidaOperation { async fn execute(&self, params: Value) -> Result { let config_file = params["config_file"].as_str()?; let phase = params.get("phase").and_then(|p| p.as_str()).unwrap_or("deploy"); // Call valida crate let rules = valida::RuleSet::from_file(config_file)?; let validation_result = rules.validate_with_phase(config_file, phase)?; // Cache result let cache_key = format!("valida:{}:{}", config_file, phase); self.cache.set_command(cache_key, serde_json::to_string(&validation_result)?).await?; Ok(OperationResult { status: if validation_result.passed { "success" } else { "failed" }, data: validation_result, }) } } // src/api/handlers/valida.rs pub async fn validate_config( State(state): State, Json(req): Json, ) -> impl IntoResponse { let operation = ValidaOperation::new(&state.cache); let params = json!({ "config_file": req.config_file, "phase": req.phase, }); match operation.execute(params).await { Ok(result) => (StatusCode::OK, Json(result)).into_response(), Err(e) => error_response(e), } } ``` ### Crate-by-Crate Integration #### Valida ```toml [dependencies] valida = { path = "../valida", optional = true } [features] valida = ["dep:valida"] ``` Validate configurations and emit validation events. #### Runtime ```toml [dependencies] runtime = { path = "../runtime", optional = true } ``` Detect container runtime, cache results. #### Encrypt ```toml [dependencies] encrypt = { path = "../encrypt", optional = true } ``` Support multiple encryption backends (SOPS, KMS, etc.). #### Init-Servs ```toml [dependencies] init-servs = { path = "../init-servs", optional = true } ``` Service installation and lifecycle management. #### Observability ```toml [dependencies] observability = { path = "../observability", optional = true } ``` Health checks and metrics collection. --- ## Phase 5: Event Bus & Syntaxis Integration ### Files to Create - `src/events/mod.rs` - Event bus core - `src/events/types.rs` - Event definitions - `src/events/bus.rs` - Event publishing/subscription - `src/events/publisher.rs` - Broadcast sender - `src/events/subscriber.rs` - Broadcast receiver - `src/orchestration/syntaxis.rs` - Syntaxis event handler ### Event Types ```rust #[derive(Debug, Clone, Serialize)] pub enum EcosystemEvent { ValidationCompleted { rule_id: String, passed: bool, timestamp: DateTime, }, RuntimeDetected { runtime: String, version: String, timestamp: DateTime, }, EncryptionCompleted { backend: String, success: bool, timestamp: DateTime, }, ServiceInstalled { service_name: String, timestamp: DateTime, }, // ... more events } ``` ### Integration Pattern ```rust // Publish event after operation pub async fn validate_config( State(state): State, Json(req): Json, ) -> impl IntoResponse { let operation = ValidaOperation::new(&state.cache); let result = operation.execute(params).await?; // Publish event state.event_bus.publish(EcosystemEvent::ValidationCompleted { passed: result.passed, timestamp: Utc::now(), }).await; (StatusCode::OK, Json(result)) } // Subscribe to events in Syntaxis handler pub async fn handle_validation_event(event: ValidationCompleted) { if !event.passed { // Create task in Syntaxis syntaxis::create_task(Task { title: format!("Validation failed: {}", event.rule_id), phase: "devel", status: "blocked", }).await; } } ``` --- ## Phase 6: Observability & GitOps Integration ### Files to Create/Modify - `src/integration/mod.rs` - Integration module - `src/integration/health_server.rs` - Health server integration - `src/integration/gitops_webhooks.rs` - GitOps webhook integration ### Health Server Integration ```rust pub fn create_health_router() -> Router { Router::new() .route("/livez", get(liveness)) .route("/readyz", get(readiness)) .route("/healthz", get(startup)) } // Integrate into main router let app = Router::new() .nest("/api/v1", api_routes(state)) .nest("/health", health_routes(state)); ``` ### GitOps Webhook Integration ```rust pub async fn handle_git_webhook( State(state): State, Json(event): Json, ) -> impl IntoResponse { // Publish to event bus state.event_bus.publish(EcosystemEvent::GitEventReceived { event_type: event.event_type, timestamp: Utc::now(), }).await; (StatusCode::ACCEPTED, Json(json!({"status": "received"}))) } ``` --- ## Phase 7: Configuration Rendering & Nushell Integration ### Files to Create - `src/rendering/mod.rs` - Rendering module - `src/rendering/tera.rs` - Tera template engine - `src/rendering/kcl.rs` - KCL renderer (optional feature) - `src/rendering/nickel.rs` - Nickel renderer (optional feature) - `src/rendering/fluent.rs` - Fluent i18n (optional feature) - `scripts/ecosystem_env.nu` - Pre-loaded Nushell environment - `scripts/daemon/valida_daemon.nu` - Valida functions - `scripts/daemon/runtime_daemon.nu` - Runtime functions - etc. ### Tera Template Rendering ```rust pub async fn render_template( State(state): State, Json(req): Json, ) -> impl IntoResponse { // Check cache if let Some(cached) = state.cache.get_config(&req.template).await { return (StatusCode::OK, Json(json!({"result": cached}))); } // Render let mut tera = Tera::new("templates/**/*.tera")?; let result = tera.render(&req.template, &req.context)?; // Cache state.cache.set_config(req.template, result.clone()).await?; (StatusCode::OK, Json(json!({"result": result}))) } ``` ### Nushell Integration ```nushell # scripts/ecosystem_env.nu # Pre-loaded functions for fast execution export def valida-validate [file: string, phase: string = "deploy"] { http post "http://localhost:9090/api/v1/valida/validate" { config_file: $file, phase: $phase } } export def runtime-detect [] { http get "http://localhost:9090/api/v1/runtime/detect" } export def cache-stats [] { http get "http://localhost:9090/api/v1/cache/stats" } ``` ### Feature Gating ```toml [features] kcl = [] nickel = [] tera-default = [] fluent-default = [] [dependencies] tera = { version = "1.20", optional = true } fluent = { version = "0.17", optional = true } ``` --- ## Implementation Order & Dependencies ``` Phase 4: Ecosystem Crates ◄─── Phase 3: CLI Client ▼ Phase 5: Event Bus ◄───────── Phase 4: Crate Integration ▼ Phase 6: Health/GitOps ◄───── Phase 5: Events ▼ Phase 7: Rendering/Nushell ◄─ Phase 6: Integration ``` ### Recommended Implementation Sequence 1. **Phase 4 First**: Implement crate integration so API handlers work 2. **Phase 3 Second**: Implement CLI that calls Phase 4 operations 3. **Phase 5 Third**: Add event bus for coordination 4. **Phase 6 Fourth**: Integrate with existing services 5. **Phase 7 Last**: Add advanced features (rendering, Nushell) --- ## Testing Strategy for Phases 3-7 ### Phase 3: CLI Testing ```bash # Test daemon mode provctl daemon status provctl valida validate --file test.yaml # Test offline mode (daemon down) PROV_OFFLINE=1 provctl valida validate --file test.yaml # Test JSON output provctl valida validate --file test.yaml --json ``` ### Phase 4: Crate Integration Testing ```bash # Test each integrated crate curl -X POST http://localhost:9090/api/v1/valida/validate \ -H "Content-Type: application/json" \ -d '{"config_file": "test.yaml", "phase": "deploy"}' curl http://localhost:9090/api/v1/runtime/detect curl -X POST http://localhost:9090/api/v1/encrypt/encrypt \ -H "Content-Type: application/json" \ -d '{"file": "secrets.env", "backend": "sops"}' ``` ### Phase 5: Event Testing ```bash # Subscribe to events in separate terminal provctl events subscribe --type validation_completed # Trigger operation to emit events provctl valida validate --file test.yaml # Watch for events in first terminal ``` ### Phase 6 & 7: Integration Testing ```bash # Health checks curl http://localhost:9090/health/livez curl http://localhost:9090/health/readyz # Rendering curl -X POST http://localhost:9090/api/v1/render/tera \ -H "Content-Type: application/json" \ -d '{"template": "config.tera", "context": {}}' # Nushell functions nu -c " source scripts/ecosystem_env.nu valida-validate 'test.yaml' 'deploy' " ``` --- ## Success Criteria ### Phase 3 ✓ - [ ] `provctl daemon status` shows daemon status - [ ] `provctl valida validate` works in both modes - [ ] Graceful fallback when daemon unavailable - [ ] JSON output works correctly ### Phase 4 ✓ - [ ] All ecosystem crates integrated - [ ] Real validation, runtime detection, encryption working - [ ] 80%+ cache hit ratio - [ ] <100ms end-to-end execution ### Phase 5 ✓ - [ ] Events publish/subscribe working - [ ] Syntaxis tasks created from events - [ ] 10000+ events/sec throughput - [ ] Optional persistence layer working ### Phase 6 ✓ - [ ] Health probes respond correctly - [ ] GitOps webhooks handled - [ ] Unified logging across all services - [ ] Metrics collected and accessible ### Phase 7 ✓ - [ ] KCL/Nickel/Tera rendering works - [ ] Fluent i18n translation working - [ ] Nushell functions execute <5ms - [ ] Scripts integrate with daemon --- ## Conclusion The roadmap is clear. Phases 1-2 provide a rock-solid foundation. Phases 3-7 build incrementally on that, each phase being independent enough to implement in parallel but dependent on Phase 1-2 as the base. Key principles: - ✅ Progressive enhancement - ✅ No breaking changes - ✅ Each phase is self-contained - ✅ Feature-gated for flexibility - ✅ Comprehensive testing at each stage Ready to begin implementation when you are! 🚀