# Database and Configuration Architecture\n\n**Date**: 2025-10-07\n**Status**: ACTIVE DOCUMENTATION\n\n---\n\n## Control-Center Database (DBS)\n\n### Database Type: **SurrealDB** (In-Memory Backend)\n\nControl-Center uses **SurrealDB with kv-mem backend**, an embedded in-memory database - **no separate database server required**.\n\n### Database Configuration\n\n```\n[database]\nurl = "memory"  # In-memory backend\nnamespace = "control_center"\ndatabase = "main"\n```\n\n**Storage**: In-memory (data persists during process lifetime)\n\n**Production Alternative**: Switch to remote WebSocket connection for persistent storage:\n\n```\n[database]\nurl = "ws://localhost:8000"\nnamespace = "control_center"\ndatabase = "main"\nusername = "root"\npassword = "secret"\n```\n\n### Why SurrealDB kv-mem\n\n| Feature | SurrealDB kv-mem | RocksDB | PostgreSQL |\n| --------- | ------------------ | --------- | ------------ |\n| **Deployment** | Embedded (no server) | Embedded | Server only |\n| **Build Deps** | None | libclang, bzip2 | Many |\n| **Docker** | Simple | Complex | External service |\n| **Performance** | Very fast (memory) | Very fast (disk) | Network latency |\n| **Use Case** | Dev/test, graphs | Production K/V | Relational data |\n| **GraphQL** | Built-in | None | External |\n\n**Control-Center choice**: SurrealDB kv-mem for **zero-dependency embedded storage**, perfect for:\n\n- Policy engine state\n- Session management\n- Configuration cache\n- Audit logs\n- User credentials\n- Graph-based policy relationships\n\n### Additional Database Support\n\nControl-Center also supports (via Cargo.toml dependencies):\n\n1. **SurrealDB (WebSocket)** - For production persistent storage\n\n   ```toml\n   surrealdb = { version = "2.3", features = ["kv-mem", "protocol-ws", "protocol-http"] }\n   ```\n\n1. **SQLx** - For SQL database backends (optional)\n\n   ```toml\n   sqlx = { workspace = true }\n   ```\n\n**Default**: SurrealDB kv-mem (embedded, no extra setup, no build dependencies)\n\n---\n\n## Orchestrator Database\n\n### Storage Type: **Filesystem** (File-based Queue)\n\nOrchestrator uses simple file-based storage by default:\n\n```\n[orchestrator.storage]\ntype = "filesystem"  # Default\nbackend_path = "{{orchestrator.paths.data_dir}}/queue.rkvs"\n```\n\n**Resolved Path**:\n\n```\n{{workspace.path}}/.orchestrator/data/queue.rkvs\n```\n\n### Optional: SurrealDB Backend\n\nFor production deployments, switch to SurrealDB:\n\n```\n[orchestrator.storage]\ntype = "surrealdb-server"  # or surrealdb-embedded\n\n[orchestrator.storage.surrealdb]\nurl = "ws://localhost:8000"\nnamespace = "orchestrator"\ndatabase = "tasks"\nusername = "root"\npassword = "secret"\n```\n\n---\n\n## Configuration Loading Architecture\n\n### Hierarchical Configuration System\n\nAll services load configuration in this order (priority: low → high):\n\n```\n1. System Defaults       provisioning/config/config.defaults.toml\n2. Service Defaults      provisioning/platform/{service}/config.defaults.toml\n3. Workspace Config      workspace/{name}/config/provisioning.yaml\n4. User Config           ~/Library/Application Support/provisioning/user_config.yaml\n5. Environment Variables PROVISIONING_*, CONTROL_CENTER_*, ORCHESTRATOR_*\n6. Runtime Overrides     --config flag or API updates\n```\n\n### Variable Interpolation\n\nConfigs support dynamic variable interpolation:\n\n```\n[paths]\nbase = "/Users/Akasha/project-provisioning/provisioning"\ndata_dir = "{{paths.base}}/data"  # Resolves to: /Users/.../data\n\n[database]\nurl = "rocksdb://{{paths.data_dir}}/control-center.db"\n# Resolves to: rocksdb:///Users/.../data/control-center.db\n```\n\n**Supported Variables**:\n\n- `{{paths.*}}` - Path variables from config\n- `{{workspace.path}}` - Current workspace path\n- `{{env.HOME}}` - Environment variables\n- `{{now.date}}` - Current date/time\n- `{{git.branch}}` - Git branch name\n\n### Service-Specific Config Files\n\nEach platform service has its own `config.defaults.toml`:\n\n| Service | Config File | Purpose |\n| --------- | ------------- | --------- |\n| **Orchestrator** | `provisioning/platform/orchestrator/config.defaults.toml` | Workflow management, queue settings |\n| **Control-Center** | `provisioning/platform/control-center/config.defaults.toml` | Web UI, auth, database |\n| **MCP Server** | `provisioning/platform/mcp-server/config.defaults.toml` | AI integration settings |\n| **KMS** | `provisioning/core/services/kms/config.defaults.toml` | Key management |\n\n### Central Configuration\n\n**Master config**: `provisioning/config/config.defaults.toml`\n\nContains:\n\n- Global paths\n- Provider configurations\n- Cache settings\n- Debug flags\n- Environment-specific overrides\n\n### Workspace-Aware Paths\n\nAll services use workspace-aware paths:\n\n**Orchestrator**:\n\n```\n[orchestrator.paths]\nbase = "{{workspace.path}}/.orchestrator"\ndata_dir = "{{orchestrator.paths.base}}/data"\nlogs_dir = "{{orchestrator.paths.base}}/logs"\nqueue_dir = "{{orchestrator.paths.data_dir}}/queue"\n```\n\n**Control-Center**:\n\n```\n[paths]\nbase = "{{workspace.path}}/.control-center"\ndata_dir = "{{paths.base}}/data"\nlogs_dir = "{{paths.base}}/logs"\n```\n\n**Result** (workspace: `workspace-librecloud`):\n\n```\nworkspace-librecloud/\n├── .orchestrator/\n│   ├── data/\n│   │   └── queue.rkvs\n│   └── logs/\n└── .control-center/\n    ├── data/\n    │   └── control-center.db\n    └── logs/\n```\n\n---\n\n## Environment Variable Overrides\n\nAny config value can be overridden via environment variables:\n\n### Control-Center\n\n```\n# Override server port\nexport CONTROL_CENTER_SERVER_PORT=8081\n\n# Override database URL\nexport CONTROL_CENTER_DATABASE_URL="rocksdb:///custom/path/db"\n\n# Override JWT secret\nexport CONTROL_CENTER_JWT_ISSUER="my-issuer"\n```\n\n### Orchestrator\n\n```\n# Override orchestrator port\nexport ORCHESTRATOR_SERVER_PORT=8080\n\n# Override storage backend\nexport ORCHESTRATOR_STORAGE_TYPE="surrealdb-server"\nexport ORCHESTRATOR_STORAGE_SURREALDB_URL="ws://localhost:8000"\n\n# Override concurrency\nexport ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS=10\n```\n\n### Naming Convention\n\n```\n{SERVICE}_{SECTION}_{KEY} = value\n```\n\n**Examples**:\n\n- `CONTROL_CENTER_SERVER_PORT` → `[server] port`\n- `ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS` → `[queue] max_concurrent_tasks`\n- `PROVISIONING_DEBUG_ENABLED` → `[debug] enabled`\n\n---\n\n## Docker vs Native Configuration\n\n### Docker Deployment\n\n**Container paths** (resolved inside container):\n\n```\n[paths]\nbase = "/app/provisioning"\ndata_dir = "/data"  # Mounted volume\nlogs_dir = "/var/log/orchestrator"  # Mounted volume\n```\n\n**Docker Compose volumes**:\n\n```\nservices:\n  orchestrator:\n    volumes:\n      - orchestrator-data:/data\n      - orchestrator-logs:/var/log/orchestrator\n\n  control-center:\n    volumes:\n      - control-center-data:/data\n\nvolumes:\n  orchestrator-data:\n  orchestrator-logs:\n  control-center-data:\n```\n\n### Native Deployment\n\n**Host paths** (macOS/Linux):\n\n```\n[paths]\nbase = "/Users/Akasha/project-provisioning/provisioning"\ndata_dir = "{{workspace.path}}/.orchestrator/data"\nlogs_dir = "{{workspace.path}}/.orchestrator/logs"\n```\n\n---\n\n## Configuration Validation\n\nCheck current configuration:\n\n```\n# Show effective configuration\nprovisioning env\n\n# Show all config and environment\nprovisioning allenv\n\n# Validate configuration\nprovisioning validate config\n\n# Show service-specific config\nPROVISIONING_DEBUG=true ./orchestrator --show-config\n```\n\n---\n\n## KMS Database\n\n**Cosmian KMS** uses its own database (when deployed):\n\n```\n# KMS database location (Docker)\n/data/kms.db  # SQLite database inside KMS container\n\n# KMS database location (Native)\n{{workspace.path}}/.kms/data/kms.db\n```\n\nKMS also integrates with Control-Center's KMS hybrid backend (local + remote):\n\n```\n[kms]\nmode = "hybrid"  # local, remote, or hybrid\n\n[kms.local]\ndatabase_path = "{{paths.data_dir}}/kms.db"\n\n[kms.remote]\nserver_url = "http://localhost:9998"  # Cosmian KMS server\n```\n\n---\n\n## Summary\n\n### Control-Center Database\n\n- **Type**: RocksDB (embedded)\n- **Location**: `{{workspace.path}}/.control-center/data/control-center.db`\n- **No server required**: Embedded in control-center process\n\n### Orchestrator Database\n\n- **Type**: Filesystem (default) or SurrealDB (production)\n- **Location**: `{{workspace.path}}/.orchestrator/data/queue.rkvs`\n- **Optional server**: SurrealDB for production\n\n### Configuration Loading\n\n1. System defaults (provisioning/config/)\n2. Service defaults (platform/{service}/)\n3. Workspace config\n4. User config\n5. Environment variables\n6. Runtime overrides\n\n### Best Practices\n\n- ✅ Use workspace-aware paths\n- ✅ Override via environment variables in Docker\n- ✅ Keep secrets in KMS, not config files\n- ✅ Use RocksDB for single-node deployments\n- ✅ Use SurrealDB for distributed/production deployments\n\n---\n\n**Related Documentation**:\n\n- [Configuration System](../infrastructure/configuration-guide.md)\n- [KMS Architecture](../security/kms-architecture.md)\n- [Workspace Switching](../infrastructure/workspace-switching-guide.md)
