Database and Configuration Architecture

Date: 2025-10-07 Status: ACTIVE DOCUMENTATION

Control-Center Database (DBS)

Database Type: SurrealDB (In-Memory Backend)

Control-Center uses SurrealDB with kv-mem backend, an embedded in-memory database - no separate database server required.

Database Configuration

[database]
url = "memory"  # In-memory backend
namespace = "control_center"
database = "main"

Storage: In-memory (data persists during process lifetime)

Production Alternative: Switch to remote WebSocket connection for persistent storage:

[database]
url = "ws://localhost:8000"
namespace = "control_center"
database = "main"
username = "root"
password = "secret"

Why SurrealDB kv-mem?

Feature	SurrealDB kv-mem	RocksDB	PostgreSQL
Deployment	Embedded (no server)	Embedded	Server only
Build Deps	None	libclang, bzip2	Many
Docker	Simple	Complex	External service
Performance	Very fast (memory)	Very fast (disk)	Network latency
Use Case	Dev/test, graphs	Production K/V	Relational data
GraphQL	Built-in	None	External

Control-Center choice: SurrealDB kv-mem for zero-dependency embedded storage, perfect for:

Policy engine state
Session management
Configuration cache
Audit logs
User credentials
Graph-based policy relationships

Additional Database Support

Control-Center also supports (via Cargo.toml dependencies):

SurrealDB (WebSocket) - For production persistent storage

surrealdb = { version = "2.3", features = ["kv-mem", "protocol-ws", "protocol-http"] }

SQLx - For SQL database backends (optional)
```
sqlx = { workspace = true }
```

Default: SurrealDB kv-mem (embedded, no extra setup, no build dependencies)

Orchestrator Database

Storage Type: Filesystem (File-based Queue)

Orchestrator uses simple file-based storage by default:

[orchestrator.storage]
type = "filesystem"  # Default
backend_path = "{{orchestrator.paths.data_dir}}/queue.rkvs"

Resolved Path:

{{workspace.path}}/.orchestrator/data/queue.rkvs

Optional: SurrealDB Backend

For production deployments, switch to SurrealDB:

[orchestrator.storage]
type = "surrealdb-server"  # or surrealdb-embedded

[orchestrator.storage.surrealdb]
url = "ws://localhost:8000"
namespace = "orchestrator"
database = "tasks"
username = "root"
password = "secret"

Configuration Loading Architecture

Hierarchical Configuration System

All services load configuration in this order (priority: low → high):

1. System Defaults       provisioning/config/config.defaults.toml
2. Service Defaults      provisioning/platform/{service}/config.defaults.toml
3. Workspace Config      workspace/{name}/config/provisioning.yaml
4. User Config           ~/Library/Application Support/provisioning/user_config.yaml
5. Environment Variables PROVISIONING_*, CONTROL_CENTER_*, ORCHESTRATOR_*
6. Runtime Overrides     --config flag or API updates

Variable Interpolation

Configs support dynamic variable interpolation:

[paths]
base = "/Users/Akasha/project-provisioning/provisioning"
data_dir = "{{paths.base}}/data"  # Resolves to: /Users/.../data

[database]
url = "rocksdb://{{paths.data_dir}}/control-center.db"
# Resolves to: rocksdb:///Users/.../data/control-center.db

Supported Variables:

{{paths.*}} - Path variables from config
{{workspace.path}} - Current workspace path
{{env.HOME}} - Environment variables
{{now.date}} - Current date/time
{{git.branch}} - Git branch name

Service-Specific Config Files

Each platform service has its own config.defaults.toml:

Service	Config File	Purpose
Orchestrator	`provisioning/platform/orchestrator/config.defaults.toml`	Workflow management, queue settings
Control-Center	`provisioning/platform/control-center/config.defaults.toml`	Web UI, auth, database
MCP Server	`provisioning/platform/mcp-server/config.defaults.toml`	AI integration settings
KMS	`provisioning/core/services/kms/config.defaults.toml`	Key management

Central Configuration

Master config: provisioning/config/config.defaults.toml

Contains:

Global paths
Provider configurations
Cache settings
Debug flags
Environment-specific overrides

Workspace-Aware Paths

All services use workspace-aware paths:

Orchestrator:

[orchestrator.paths]
base = "{{workspace.path}}/.orchestrator"
data_dir = "{{orchestrator.paths.base}}/data"
logs_dir = "{{orchestrator.paths.base}}/logs"
queue_dir = "{{orchestrator.paths.data_dir}}/queue"

Control-Center:

[paths]
base = "{{workspace.path}}/.control-center"
data_dir = "{{paths.base}}/data"
logs_dir = "{{paths.base}}/logs"

Result (workspace: workspace-librecloud):

workspace-librecloud/
├── .orchestrator/
│   ├── data/
│   │   └── queue.rkvs
│   └── logs/
└── .control-center/
    ├── data/
    │   └── control-center.db
    └── logs/

Environment Variable Overrides

Any config value can be overridden via environment variables:

Control-Center

# Override server port
export CONTROL_CENTER_SERVER_PORT=8081

# Override database URL
export CONTROL_CENTER_DATABASE_URL="rocksdb:///custom/path/db"

# Override JWT secret
export CONTROL_CENTER_JWT_ISSUER="my-issuer"

Orchestrator

# Override orchestrator port
export ORCHESTRATOR_SERVER_PORT=8080

# Override storage backend
export ORCHESTRATOR_STORAGE_TYPE="surrealdb-server"
export ORCHESTRATOR_STORAGE_SURREALDB_URL="ws://localhost:8000"

# Override concurrency
export ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS=10

Naming Convention

{SERVICE}_{SECTION}_{KEY} = value

Examples:

CONTROL_CENTER_SERVER_PORT → [server] port
ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS → [queue] max_concurrent_tasks
PROVISIONING_DEBUG_ENABLED → [debug] enabled

Docker vs Native Configuration

Docker Deployment

Container paths (resolved inside container):

[paths]
base = "/app/provisioning"
data_dir = "/data"  # Mounted volume
logs_dir = "/var/log/orchestrator"  # Mounted volume

Docker Compose volumes:

services:
  orchestrator:
    volumes:
      - orchestrator-data:/data
      - orchestrator-logs:/var/log/orchestrator

  control-center:
    volumes:
      - control-center-data:/data

volumes:
  orchestrator-data:
  orchestrator-logs:
  control-center-data:

Native Deployment

Host paths (macOS/Linux):

[paths]
base = "/Users/Akasha/project-provisioning/provisioning"
data_dir = "{{workspace.path}}/.orchestrator/data"
logs_dir = "{{workspace.path}}/.orchestrator/logs"

Configuration Validation

Check current configuration:

# Show effective configuration
provisioning env

# Show all config and environment
provisioning allenv

# Validate configuration
provisioning validate config

# Show service-specific config
PROVISIONING_DEBUG=true ./orchestrator --show-config

KMS Database

Cosmian KMS uses its own database (when deployed):

# KMS database location (Docker)
/data/kms.db  # SQLite database inside KMS container

# KMS database location (Native)
{{workspace.path}}/.kms/data/kms.db

KMS also integrates with Control-Center’s KMS hybrid backend (local + remote):

[kms]
mode = "hybrid"  # local, remote, or hybrid

[kms.local]
database_path = "{{paths.data_dir}}/kms.db"

[kms.remote]
server_url = "http://localhost:9998"  # Cosmian KMS server

Summary

Control-Center Database

Type: RocksDB (embedded)
Location: {{workspace.path}}/.control-center/data/control-center.db
No server required: Embedded in control-center process

Orchestrator Database

Type: Filesystem (default) or SurrealDB (production)
Location: {{workspace.path}}/.orchestrator/data/queue.rkvs
Optional server: SurrealDB for production

Configuration Loading

System defaults (provisioning/config/)
Service defaults (platform/{service}/)
Workspace config
User config
Environment variables
Runtime overrides

Best Practices

✅ Use workspace-aware paths
✅ Override via environment variables in Docker
✅ Keep secrets in KMS, not config files
✅ Use RocksDB for single-node deployments
✅ Use SurrealDB for distributed/production deployments

Related Documentation:

Configuration System: .claude/features/configuration-system.md
KMS Architecture: provisioning/platform/control-center/src/kms/README.md
Workspace Switching: .claude/features/workspace-switching.md

Keyboard shortcuts

Provisioning Platform Documentation