# Database and Configuration Architecture

**Date**: 2025-10-07
**Status**: ACTIVE DOCUMENTATION

---

## Control-Center Database (DBS)

### Database Type: **SurrealDB** (In-Memory Backend)

Control-Center uses **SurrealDB with kv-mem backend**, an embedded in-memory database - **no separate database server required**.

### Database Configuration

```toml
[database]
url = "memory"  # In-memory backend
namespace = "control_center"
database = "main"
```plaintext

**Storage**: In-memory (data persists during process lifetime)

**Production Alternative**: Switch to remote WebSocket connection for persistent storage:

```toml
[database]
url = "ws://localhost:8000"
namespace = "control_center"
database = "main"
username = "root"
password = "secret"
```plaintext

### Why SurrealDB kv-mem

| Feature | SurrealDB kv-mem | RocksDB | PostgreSQL |
|---------|------------------|---------|------------|
| **Deployment** | Embedded (no server) | Embedded | Server only |
| **Build Deps** | None | libclang, bzip2 | Many |
| **Docker** | Simple | Complex | External service |
| **Performance** | Very fast (memory) | Very fast (disk) | Network latency |
| **Use Case** | Dev/test, graphs | Production K/V | Relational data |
| **GraphQL** | Built-in | None | External |

**Control-Center choice**: SurrealDB kv-mem for **zero-dependency embedded storage**, perfect for:

- Policy engine state
- Session management
- Configuration cache
- Audit logs
- User credentials
- Graph-based policy relationships

### Additional Database Support

Control-Center also supports (via Cargo.toml dependencies):

1. **SurrealDB (WebSocket)** - For production persistent storage

   ```toml
   surrealdb = { version = "2.3", features = ["kv-mem", "protocol-ws", "protocol-http"] }
   ```

1. **SQLx** - For SQL database backends (optional)

   ```toml
   sqlx = { workspace = true }
   ```

**Default**: SurrealDB kv-mem (embedded, no extra setup, no build dependencies)

---

## Orchestrator Database

### Storage Type: **Filesystem** (File-based Queue)

Orchestrator uses simple file-based storage by default:

```toml
[orchestrator.storage]
type = "filesystem"  # Default
backend_path = "{{orchestrator.paths.data_dir}}/queue.rkvs"
```plaintext

**Resolved Path**:

```plaintext
{{workspace.path}}/.orchestrator/data/queue.rkvs
```plaintext

### Optional: SurrealDB Backend

For production deployments, switch to SurrealDB:

```toml
[orchestrator.storage]
type = "surrealdb-server"  # or surrealdb-embedded

[orchestrator.storage.surrealdb]
url = "ws://localhost:8000"
namespace = "orchestrator"
database = "tasks"
username = "root"
password = "secret"
```plaintext

---

## Configuration Loading Architecture

### Hierarchical Configuration System

All services load configuration in this order (priority: low → high):

```plaintext
1. System Defaults       provisioning/config/config.defaults.toml
2. Service Defaults      provisioning/platform/{service}/config.defaults.toml
3. Workspace Config      workspace/{name}/config/provisioning.yaml
4. User Config           ~/Library/Application Support/provisioning/user_config.yaml
5. Environment Variables PROVISIONING_*, CONTROL_CENTER_*, ORCHESTRATOR_*
6. Runtime Overrides     --config flag or API updates
```plaintext

### Variable Interpolation

Configs support dynamic variable interpolation:

```toml
[paths]
base = "/Users/Akasha/project-provisioning/provisioning"
data_dir = "{{paths.base}}/data"  # Resolves to: /Users/.../data

[database]
url = "rocksdb://{{paths.data_dir}}/control-center.db"
# Resolves to: rocksdb:///Users/.../data/control-center.db
```plaintext

**Supported Variables**:

- `{{paths.*}}` - Path variables from config
- `{{workspace.path}}` - Current workspace path
- `{{env.HOME}}` - Environment variables
- `{{now.date}}` - Current date/time
- `{{git.branch}}` - Git branch name

### Service-Specific Config Files

Each platform service has its own `config.defaults.toml`:

| Service | Config File | Purpose |
|---------|-------------|---------|
| **Orchestrator** | `provisioning/platform/orchestrator/config.defaults.toml` | Workflow management, queue settings |
| **Control-Center** | `provisioning/platform/control-center/config.defaults.toml` | Web UI, auth, database |
| **MCP Server** | `provisioning/platform/mcp-server/config.defaults.toml` | AI integration settings |
| **KMS** | `provisioning/core/services/kms/config.defaults.toml` | Key management |

### Central Configuration

**Master config**: `provisioning/config/config.defaults.toml`

Contains:

- Global paths
- Provider configurations
- Cache settings
- Debug flags
- Environment-specific overrides

### Workspace-Aware Paths

All services use workspace-aware paths:

**Orchestrator**:

```toml
[orchestrator.paths]
base = "{{workspace.path}}/.orchestrator"
data_dir = "{{orchestrator.paths.base}}/data"
logs_dir = "{{orchestrator.paths.base}}/logs"
queue_dir = "{{orchestrator.paths.data_dir}}/queue"
```plaintext

**Control-Center**:

```toml
[paths]
base = "{{workspace.path}}/.control-center"
data_dir = "{{paths.base}}/data"
logs_dir = "{{paths.base}}/logs"
```plaintext

**Result** (workspace: `workspace-librecloud`):

```plaintext
workspace-librecloud/
├── .orchestrator/
│   ├── data/
│   │   └── queue.rkvs
│   └── logs/
└── .control-center/
    ├── data/
    │   └── control-center.db
    └── logs/
```plaintext

---

## Environment Variable Overrides

Any config value can be overridden via environment variables:

### Control-Center

```bash
# Override server port
export CONTROL_CENTER_SERVER_PORT=8081

# Override database URL
export CONTROL_CENTER_DATABASE_URL="rocksdb:///custom/path/db"

# Override JWT secret
export CONTROL_CENTER_JWT_ISSUER="my-issuer"
```plaintext

### Orchestrator

```bash
# Override orchestrator port
export ORCHESTRATOR_SERVER_PORT=8080

# Override storage backend
export ORCHESTRATOR_STORAGE_TYPE="surrealdb-server"
export ORCHESTRATOR_STORAGE_SURREALDB_URL="ws://localhost:8000"

# Override concurrency
export ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS=10
```plaintext

### Naming Convention

```plaintext
{SERVICE}_{SECTION}_{KEY} = value
```plaintext

**Examples**:

- `CONTROL_CENTER_SERVER_PORT` → `[server] port`
- `ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS` → `[queue] max_concurrent_tasks`
- `PROVISIONING_DEBUG_ENABLED` → `[debug] enabled`

---

## Docker vs Native Configuration

### Docker Deployment

**Container paths** (resolved inside container):

```toml
[paths]
base = "/app/provisioning"
data_dir = "/data"  # Mounted volume
logs_dir = "/var/log/orchestrator"  # Mounted volume
```plaintext

**Docker Compose volumes**:

```yaml
services:
  orchestrator:
    volumes:
      - orchestrator-data:/data
      - orchestrator-logs:/var/log/orchestrator

  control-center:
    volumes:
      - control-center-data:/data

volumes:
  orchestrator-data:
  orchestrator-logs:
  control-center-data:
```plaintext

### Native Deployment

**Host paths** (macOS/Linux):

```toml
[paths]
base = "/Users/Akasha/project-provisioning/provisioning"
data_dir = "{{workspace.path}}/.orchestrator/data"
logs_dir = "{{workspace.path}}/.orchestrator/logs"
```plaintext

---

## Configuration Validation

Check current configuration:

```bash
# Show effective configuration
provisioning env

# Show all config and environment
provisioning allenv

# Validate configuration
provisioning validate config

# Show service-specific config
PROVISIONING_DEBUG=true ./orchestrator --show-config
```plaintext

---

## KMS Database

**Cosmian KMS** uses its own database (when deployed):

```bash
# KMS database location (Docker)
/data/kms.db  # SQLite database inside KMS container

# KMS database location (Native)
{{workspace.path}}/.kms/data/kms.db
```plaintext

KMS also integrates with Control-Center's KMS hybrid backend (local + remote):

```toml
[kms]
mode = "hybrid"  # local, remote, or hybrid

[kms.local]
database_path = "{{paths.data_dir}}/kms.db"

[kms.remote]
server_url = "http://localhost:9998"  # Cosmian KMS server
```plaintext

---

## Summary

### Control-Center Database

- **Type**: RocksDB (embedded)
- **Location**: `{{workspace.path}}/.control-center/data/control-center.db`
- **No server required**: Embedded in control-center process

### Orchestrator Database

- **Type**: Filesystem (default) or SurrealDB (production)
- **Location**: `{{workspace.path}}/.orchestrator/data/queue.rkvs`
- **Optional server**: SurrealDB for production

### Configuration Loading

1. System defaults (provisioning/config/)
2. Service defaults (platform/{service}/)
3. Workspace config
4. User config
5. Environment variables
6. Runtime overrides

### Best Practices

- ✅ Use workspace-aware paths
- ✅ Override via environment variables in Docker
- ✅ Keep secrets in KMS, not config files
- ✅ Use RocksDB for single-node deployments
- ✅ Use SurrealDB for distributed/production deployments

---

**Related Documentation**:

- [Configuration System](../infrastructure/configuration-guide.md)
- [KMS Architecture](../security/kms-architecture.md)
- [Workspace Switching](../infrastructure/workspace-switching-guide.md)
