Vapora/provisioning/PROVISIONING-INTEGRATION.md

# VAPORA Provisioning Integration

Integration documentation for deploying VAPORA v1.0 using Provisioning.

## Overview

VAPORA can be deployed using **Provisioning**, a Rust-based infrastructure-as-code platform that manages Kubernetes clusters, services, and workflows.

The Provisioning workspace is located at: `provisioning/vapora-wrksp/` (relative to repository root)

## Provisioning Workspace Structure

```
provisioning/vapora-wrksp/
├── workspace.toml          # Master configuration
├── kcl/                    # Infrastructure schemas (KCL)
│   ├── cluster.k           # Cluster definition
│   ├── namespace.k         # Namespace configuration
│   ├── backend.k           # Backend deployment
│   ├── frontend.k          # Frontend deployment
│   └── agents.k            # Agent deployment
├── taskservs/              # Service definitions (TOML)
│   ├── surrealdb.toml      # SurrealDB service
│   ├── nats.toml           # NATS service
│   ├── backend.toml        # Backend service
│   ├── frontend.toml       # Frontend service
│   └── agents.toml         # Agents service
└── workflows/              # Batch operations (YAML)
    ├── deploy-full-stack.yaml
    ├── deploy-infra.yaml
    ├── deploy-services.yaml
    └── health-check.yaml
```

## Integration Points

### 1. Cluster Management

Provisioning creates and manages the Kubernetes cluster:

```bash
cd provisioning/vapora-wrksp
provisioning cluster create --config workspace.toml
```

This creates:
- K3s/RKE2 cluster
- Storage class (Rook Ceph or local-path)
- Ingress controller (nginx)
- Service mesh (optional Istio)

### 2. Service Deployment

Services are defined in `taskservs/` and deployed via workflows:

```bash
provisioning workflow run workflows/deploy-full-stack.yaml
```

This deploys all VAPORA components in order:
1. SurrealDB (StatefulSet)
2. NATS JetStream (Deployment)
3. Backend API (Deployment)
4. Frontend UI (Deployment)
5. Agents (Deployment)
6. MCP Server (Deployment)

### 3. Infrastructure as Code (KCL)

KCL schemas in `kcl/` define infrastructure resources:

**Example: `kcl/backend.k`**
```python
schema BackendDeployment:
    name: str = "vapora-backend"
    namespace: str = "vapora"
    replicas: int = 2
    image: str = "vapora/backend:latest"
    port: int = 8080

    env:
        SURREALDB_URL: str = "http://surrealdb:8000"
        NATS_URL: str = "nats://nats:4222"
        JWT_SECRET: str = "${SECRET:jwt-secret}"
```

### 4. Taskserv Definitions

Taskservs define how services are deployed and managed:

**Example: `taskservs/backend.toml`**
```toml
[service]
name = "vapora-backend"
type = "deployment"
namespace = "vapora"

[deployment]
replicas = 2
image = "vapora/backend:latest"
port = 8080

[health]
liveness = "/health"
readiness = "/health"

[dependencies]
requires = ["surrealdb", "nats"]
```

### 5. Workflows

Workflows orchestrate complex deployment tasks:

**Example: `workflows/deploy-full-stack.yaml`**
```yaml
name: deploy-full-stack
description: Deploy complete VAPORA stack

steps:
  - name: create-namespace
    taskserv: namespace
    action: create

  - name: deploy-database
    taskserv: surrealdb
    action: deploy
    wait: true

  - name: deploy-messaging
    taskserv: nats
    action: deploy
    wait: true

  - name: deploy-services
    parallel: true
    tasks:
      - taskserv: backend
      - taskserv: frontend
      - taskserv: agents
      - taskserv: mcp-server

  - name: health-check
    action: validate
```

## Provisioning vs. Vanilla K8s

| Aspect | Provisioning | Vanilla K8s |
|--------|-------------|-------------|
| Cluster Creation | Automated (RKE2/K3s) | Manual |
| Service Mesh | Optional Istio | Manual |
| Secrets | RustyVault integration | kubectl create secret |
| Workflows | Declarative YAML | Manual kubectl |
| Rollback | Built-in | Manual |
| Monitoring | Prometheus auto-configured | Manual |

## Advantages of Provisioning

1. **Unified Management**: Single tool for cluster, services, and workflows
2. **Type Safety**: KCL schemas provide compile-time validation
3. **Reproducibility**: Infrastructure and services defined as code
4. **Dependency Management**: Automatic service ordering
5. **Secret Management**: Integration with RustyVault
6. **Rollback**: Automatic rollback on failure

## Migration from Vanilla K8s

If you have an existing K8s deployment using `/kubernetes/` manifests:

1. **Import existing manifests**:
   ```bash
   provisioning import kubernetes/*.yaml --output kcl/
   ```

2. **Generate taskservs**:
   ```bash
   provisioning taskserv generate --from-kcl kcl/*.k
   ```

3. **Create workflow**:
   ```bash
   provisioning workflow create --interactive
   ```

4. **Deploy**:
   ```bash
   provisioning workflow run workflows/deploy-full-stack.yaml
   ```

## Deployment Workflow

### Using Provisioning (Recommended for Production)

```bash
# 1. Navigate to workspace
cd provisioning/vapora-wrksp

# 2. Validate configuration
provisioning validate --all

# 3. Create cluster
provisioning cluster create --config workspace.toml

# 4. Deploy infrastructure
provisioning workflow run workflows/deploy-infra.yaml

# 5. Deploy services
provisioning workflow run workflows/deploy-services.yaml

# 6. Health check
provisioning workflow run workflows/health-check.yaml

# 7. Monitor
provisioning health-check --all
```

### Using Vanilla K8s (Manual)

```bash
# Use vanilla K8s manifests (from repository root)
nu scripts/deploy-k8s.nu
```

## Validation

To validate Provisioning configuration without executing:

```bash
# From project root
nu scripts/validate-provisioning.nu
```

This checks:
- Workspace exists
- KCL schemas are valid
- Taskserv definitions exist
- Workflows are well-formed

## Next Steps

1. **Review Configuration**:
   - Update `workspace.toml` with your cluster details
   - Modify KCL schemas for your environment
   - Adjust resource limits in taskservs

2. **Test Locally**:
   - Use K3s for local testing
   - Validate with `--dry-run` flag

3. **Deploy to Production**:
   - Use RKE2 for production cluster
   - Enable Istio service mesh
   - Configure external load balancer

4. **Monitor**:
   - Use built-in Prometheus/Grafana
   - Configure alerting
   - Set up log aggregation

## Troubleshooting

### Provisioning not installed

```bash
# Install Provisioning (Rust-based)
cargo install provisioning-cli
```

### Workspace validation fails

```bash
cd provisioning/vapora-wrksp
provisioning validate --verbose
```

### Deployment stuck

```bash
# Check workflow status
provisioning workflow status <workflow-id>

# View logs
provisioning logs --taskserv backend

# Rollback
provisioning rollback --to-version <version>
```

## Documentation References

- **Provisioning Documentation**: See `provisioning/vapora-wrksp/README.md`
- **KCL Language Guide**: https://kcl-lang.io/docs/
- **Taskserv Specification**: `provisioning/vapora-wrksp/taskservs/README.md`
- **Workflow Syntax**: `provisioning/vapora-wrksp/workflows/README.md`

## Notes

- **IMPORTANT**: Provisioning integration is **validated** but not executed in this phase
- All configuration files exist and are valid
- Deployment using Provisioning is deferred for manual production deployment
- For immediate testing, use vanilla K8s deployment: `nu scripts/deploy-k8s.nu`
- Provisioning provides advanced features (service mesh, auto-scaling, rollback)
- Vanilla K8s deployment is simpler and requires less infrastructure

## Support

For issues related to:
- **VAPORA deployment**: Check `/kubernetes/README.md` and `DEPLOYMENT.md`
- **Provisioning workspace**: See `provisioning/vapora-wrksp/README.md`
- **Scripts**: Run `nu scripts/<script-name>.nu --help`