provisioning-platform/daemon-cli/ROADMAP_PHASES_3-7.md

# Daemon-CLI Phases 3-7 Implementation Roadmap

## Current Status: Phases 1-2 Complete ✅

All foundation is in place. The API server is running with stub implementations ready for ecosystem crate integration.

---

## Phase 3: Dual-Mode CLI Client

### Files to Create/Modify
- `src/cli/mod.rs` - CLI module structure
- `src/cli/client.rs` - HTTP client for daemon communication
- `src/cli/offline.rs` - Direct library call mode
- `src/cli/output.rs` - Output formatting (table, JSON, YAML)
- `src/cli/commands/daemon.rs` - Daemon management commands
- `src/cli/commands/valida.rs` - Validation commands
- `src/cli/commands/runtime.rs` - Runtime commands
- `src/cli/commands/encrypt.rs` - Encryption commands
- `src/bin/provctl.rs` - Update with real implementations

### Implementation Pattern
```rust
// HTTP Mode: Connect to daemon
async fn execute_validation(args: ValidaArgs) -> Result<()> {
    let client = DaemonClient::new("http://localhost:9090")?;
    let result = client.valida().validate(&args).await?;
    print_result(result)?;
    Ok(())
}

// Offline Mode: Direct library calls
async fn execute_validation_offline(args: ValidaArgs) -> Result<()> {
    use valida::RuleSet;
    let rules = RuleSet::load_from_file(&args.rules_file)?;
    let result = rules.validate(&args.config, args.phase)?;
    print_result(result)?;
    Ok(())
}

// Auto-detection
async fn execute(cmd: Command) -> Result<()> {
    if DaemonClient::check_available().await {
        execute_with_daemon(cmd).await
    } else {
        execute_offline(cmd).await
    }
}
```

### Key Features
- HTTP client using reqwest
- Daemon availability detection via health check
- Graceful fallback to offline mode
- Consistent CLI interface regardless of mode
- JSON output for scripting

---

## Phase 4: Ecosystem Crate Integration

### Files to Create/Modify
- `src/orchestration/mod.rs` - Operation registry
- `src/orchestration/operation.rs` - Operation trait
- `src/orchestration/valida.rs` - Validation operations
- `src/orchestration/runtime.rs` - Runtime operations
- `src/orchestration/encrypt.rs` - Encryption operations
- `src/orchestration/init_servs.rs` - Service management
- `src/api/handlers/valida.rs` - Replace stubs
- `src/api/handlers/runtime.rs` - Replace stubs
- `src/api/handlers/encrypt.rs` - Replace stubs

### Pattern Example: Valida Integration
```rust
// src/orchestration/valida.rs
#[async_trait]
impl EcosystemOperation for ValidaOperation {
    async fn execute(&self, params: Value) -> Result<OperationResult> {
        let config_file = params["config_file"].as_str()?;
        let phase = params.get("phase").and_then(|p| p.as_str()).unwrap_or("deploy");

        // Call valida crate
        let rules = valida::RuleSet::from_file(config_file)?;
        let validation_result = rules.validate_with_phase(config_file, phase)?;

        // Cache result
        let cache_key = format!("valida:{}:{}", config_file, phase);
        self.cache.set_command(cache_key, serde_json::to_string(&validation_result)?).await?;

        Ok(OperationResult {
            status: if validation_result.passed { "success" } else { "failed" },
            data: validation_result,
        })
    }
}

// src/api/handlers/valida.rs
pub async fn validate_config(
    State(state): State<AppState>,
    Json(req): Json<ValidateRequest>,
) -> impl IntoResponse {
    let operation = ValidaOperation::new(&state.cache);
    let params = json!({
        "config_file": req.config_file,
        "phase": req.phase,
    });

    match operation.execute(params).await {
        Ok(result) => (StatusCode::OK, Json(result)).into_response(),
        Err(e) => error_response(e),
    }
}
```

### Crate-by-Crate Integration

#### Valida
```toml
[dependencies]
valida = { path = "../valida", optional = true }

[features]
valida = ["dep:valida"]
```

Validate configurations and emit validation events.

#### Runtime
```toml
[dependencies]
runtime = { path = "../runtime", optional = true }
```

Detect container runtime, cache results.

#### Encrypt
```toml
[dependencies]
encrypt = { path = "../encrypt", optional = true }
```

Support multiple encryption backends (SOPS, KMS, etc.).

#### Init-Servs
```toml
[dependencies]
init-servs = { path = "../init-servs", optional = true }
```

Service installation and lifecycle management.

#### Observability
```toml
[dependencies]
observability = { path = "../observability", optional = true }
```

Health checks and metrics collection.

---

## Phase 5: Event Bus & Syntaxis Integration

### Files to Create
- `src/events/mod.rs` - Event bus core
- `src/events/types.rs` - Event definitions
- `src/events/bus.rs` - Event publishing/subscription
- `src/events/publisher.rs` - Broadcast sender
- `src/events/subscriber.rs` - Broadcast receiver
- `src/orchestration/syntaxis.rs` - Syntaxis event handler

### Event Types
```rust
#[derive(Debug, Clone, Serialize)]
pub enum EcosystemEvent {
    ValidationCompleted {
        rule_id: String,
        passed: bool,
        timestamp: DateTime<Utc>,
    },
    RuntimeDetected {
        runtime: String,
        version: String,
        timestamp: DateTime<Utc>,
    },
    EncryptionCompleted {
        backend: String,
        success: bool,
        timestamp: DateTime<Utc>,
    },
    ServiceInstalled {
        service_name: String,
        timestamp: DateTime<Utc>,
    },
    // ... more events
}
```

### Integration Pattern
```rust
// Publish event after operation
pub async fn validate_config(
    State(state): State<AppState>,
    Json(req): Json<ValidateRequest>,
) -> impl IntoResponse {
    let operation = ValidaOperation::new(&state.cache);
    let result = operation.execute(params).await?;

    // Publish event
    state.event_bus.publish(EcosystemEvent::ValidationCompleted {
        passed: result.passed,
        timestamp: Utc::now(),
    }).await;

    (StatusCode::OK, Json(result))
}

// Subscribe to events in Syntaxis handler
pub async fn handle_validation_event(event: ValidationCompleted) {
    if !event.passed {
        // Create task in Syntaxis
        syntaxis::create_task(Task {
            title: format!("Validation failed: {}", event.rule_id),
            phase: "devel",
            status: "blocked",
        }).await;
    }
}
```

---

## Phase 6: Observability & GitOps Integration

### Files to Create/Modify
- `src/integration/mod.rs` - Integration module
- `src/integration/health_server.rs` - Health server integration
- `src/integration/gitops_webhooks.rs` - GitOps webhook integration

### Health Server Integration
```rust
pub fn create_health_router() -> Router {
    Router::new()
        .route("/livez", get(liveness))
        .route("/readyz", get(readiness))
        .route("/healthz", get(startup))
}

// Integrate into main router
let app = Router::new()
    .nest("/api/v1", api_routes(state))
    .nest("/health", health_routes(state));
```

### GitOps Webhook Integration
```rust
pub async fn handle_git_webhook(
    State(state): State<AppState>,
    Json(event): Json<GitEvent>,
) -> impl IntoResponse {
    // Publish to event bus
    state.event_bus.publish(EcosystemEvent::GitEventReceived {
        event_type: event.event_type,
        timestamp: Utc::now(),
    }).await;

    (StatusCode::ACCEPTED, Json(json!({"status": "received"})))
}
```

---

## Phase 7: Configuration Rendering & Nushell Integration

### Files to Create
- `src/rendering/mod.rs` - Rendering module
- `src/rendering/tera.rs` - Tera template engine
- `src/rendering/kcl.rs` - KCL renderer (optional feature)
- `src/rendering/nickel.rs` - Nickel renderer (optional feature)
- `src/rendering/fluent.rs` - Fluent i18n (optional feature)
- `scripts/ecosystem_env.nu` - Pre-loaded Nushell environment
- `scripts/daemon/valida_daemon.nu` - Valida functions
- `scripts/daemon/runtime_daemon.nu` - Runtime functions
- etc.

### Tera Template Rendering
```rust
pub async fn render_template(
    State(state): State<AppState>,
    Json(req): Json<RenderRequest>,
) -> impl IntoResponse {
    // Check cache
    if let Some(cached) = state.cache.get_config(&req.template).await {
        return (StatusCode::OK, Json(json!({"result": cached})));
    }

    // Render
    let mut tera = Tera::new("templates/**/*.tera")?;
    let result = tera.render(&req.template, &req.context)?;

    // Cache
    state.cache.set_config(req.template, result.clone()).await?;

    (StatusCode::OK, Json(json!({"result": result})))
}
```

### Nushell Integration
```nushell
# scripts/ecosystem_env.nu

# Pre-loaded functions for fast execution
export def valida-validate [file: string, phase: string = "deploy"] {
    http post "http://localhost:9090/api/v1/valida/validate" {
        config_file: $file,
        phase: $phase
    }
}

export def runtime-detect [] {
    http get "http://localhost:9090/api/v1/runtime/detect"
}

export def cache-stats [] {
    http get "http://localhost:9090/api/v1/cache/stats"
}
```

### Feature Gating
```toml
[features]
kcl = []
nickel = []
tera-default = []
fluent-default = []

[dependencies]
tera = { version = "1.20", optional = true }
fluent = { version = "0.17", optional = true }
```

---

## Implementation Order & Dependencies

```
Phase 4: Ecosystem Crates ◄─── Phase 3: CLI Client
    ▼
Phase 5: Event Bus ◄───────── Phase 4: Crate Integration
    ▼
Phase 6: Health/GitOps ◄───── Phase 5: Events
    ▼
Phase 7: Rendering/Nushell ◄─ Phase 6: Integration
```

### Recommended Implementation Sequence
1. **Phase 4 First**: Implement crate integration so API handlers work
2. **Phase 3 Second**: Implement CLI that calls Phase 4 operations
3. **Phase 5 Third**: Add event bus for coordination
4. **Phase 6 Fourth**: Integrate with existing services
5. **Phase 7 Last**: Add advanced features (rendering, Nushell)

---

## Testing Strategy for Phases 3-7

### Phase 3: CLI Testing
```bash
# Test daemon mode
provctl daemon status
provctl valida validate --file test.yaml

# Test offline mode (daemon down)
PROV_OFFLINE=1 provctl valida validate --file test.yaml

# Test JSON output
provctl valida validate --file test.yaml --json
```

### Phase 4: Crate Integration Testing
```bash
# Test each integrated crate
curl -X POST http://localhost:9090/api/v1/valida/validate \
  -H "Content-Type: application/json" \
  -d '{"config_file": "test.yaml", "phase": "deploy"}'

curl http://localhost:9090/api/v1/runtime/detect

curl -X POST http://localhost:9090/api/v1/encrypt/encrypt \
  -H "Content-Type: application/json" \
  -d '{"file": "secrets.env", "backend": "sops"}'
```

### Phase 5: Event Testing
```bash
# Subscribe to events in separate terminal
provctl events subscribe --type validation_completed

# Trigger operation to emit events
provctl valida validate --file test.yaml

# Watch for events in first terminal
```

### Phase 6 & 7: Integration Testing
```bash
# Health checks
curl http://localhost:9090/health/livez
curl http://localhost:9090/health/readyz

# Rendering
curl -X POST http://localhost:9090/api/v1/render/tera \
  -H "Content-Type: application/json" \
  -d '{"template": "config.tera", "context": {}}'

# Nushell functions
nu -c "
  source scripts/ecosystem_env.nu
  valida-validate 'test.yaml' 'deploy'
"
```

---

## Success Criteria

### Phase 3 ✓
- [ ] `provctl daemon status` shows daemon status
- [ ] `provctl valida validate` works in both modes
- [ ] Graceful fallback when daemon unavailable
- [ ] JSON output works correctly

### Phase 4 ✓
- [ ] All ecosystem crates integrated
- [ ] Real validation, runtime detection, encryption working
- [ ] 80%+ cache hit ratio
- [ ] <100ms end-to-end execution

### Phase 5 ✓
- [ ] Events publish/subscribe working
- [ ] Syntaxis tasks created from events
- [ ] 10000+ events/sec throughput
- [ ] Optional persistence layer working

### Phase 6 ✓
- [ ] Health probes respond correctly
- [ ] GitOps webhooks handled
- [ ] Unified logging across all services
- [ ] Metrics collected and accessible

### Phase 7 ✓
- [ ] KCL/Nickel/Tera rendering works
- [ ] Fluent i18n translation working
- [ ] Nushell functions execute <5ms
- [ ] Scripts integrate with daemon

---

## Conclusion

The roadmap is clear. Phases 1-2 provide a rock-solid foundation. Phases 3-7 build incrementally on that, each phase being independent enough to implement in parallel but dependent on Phase 1-2 as the base.

Key principles:
- ✅ Progressive enhancement
- ✅ No breaking changes
- ✅ Each phase is self-contained
- ✅ Feature-gated for flexibility
- ✅ Comprehensive testing at each stage

Ready to begin implementation when you are! 🚀