syntaxis/docs/surrealdb-setup-guide.md

SurrealDB 2.3 Setup & Deployment Guide

Version: 2.3 Status: Production Ready Last Updated: November 15, 2025

---

Overview

This guide covers deploying and configuring SurrealDB 2.3 for the syntaxis project in three deployment contexts:

1. Local Development - Server mode for development/testing
2. Docker Container - Production containerized deployment
3. Kubernetes - Enterprise orchestration with HA/scaling

---

Part 1: Local Development Setup

1.1 Install SurrealDB

macOS (via Homebrew)

brew install surrealdb
# Verify installation
surreal version
# Output: surrealdb version X.X.X

Linux (via Cargo)

cargo install surreal
# Or download binary from https://github.com/surrealdb/surrealdb/releases

Windows (via Chocolatey)

choco install surrealdb

1.2 Start SurrealDB Server (Local Development)

Basic In-Memory Server

# Start SurrealDB listening on localhost:8000
surreal start --bind 127.0.0.1:8000 memory

# Surreal outputs:
# 2025-11-15T10:00:00.000Z INFO  surreal::cli: Starting SurrealDB server...
# 2025-11-15T10:00:00.123Z INFO  surreal::net: Database server now listening on 0.0.0.0:8000

File-Based RocksDB Server

# Persistent storage in local directory
surreal start --bind 127.0.0.1:8000 file:///tmp/surrealdb.db

# Or with expanded home path
surreal start --bind 127.0.0.1:8000 file://~/.local/share/surrealdb/workspace.db

With Authentication

# Start with credentials (recommended for production-like testing)
surreal start \
  --bind 127.0.0.1:8000 \
  --username admin \
  --password secure_password_123 \
  file:///tmp/surrealdb.db

1.3 Configure Application to Connect

Configuration File: configs/database.toml

# Database Configuration - syntaxis
engine = "surrealdb"  # or "sqlite" for testing

# SQLite Configuration (Default for development)
[sqlite]
path = "~/.local/share/core/workspace.db"
max_connections = 5
timeout_secs = 30
wal_mode = true
pragma_synchronous = "NORMAL"
pragma_cache_size = 2000

# SurrealDB Configuration (Optional - requires server running)
[surrealdb]
# Connection modes (choose one)
# In-Memory (embedded, no server needed):
url = "mem://"

# File-based (embedded RocksDB, no server needed):
# url = "file:///tmp/surrealdb.db"

# Remote Server (requires surreal start):
url = "ws://localhost:8000"

# Optional: ws:// for WebSocket or http:// for HTTP
# url = "http://localhost:8000"

# Namespace and Database selection
namespace = "syntaxis"
database = "projects"

# Authentication (if server started with --username/--password)
# username = "admin"
# password = "secure_password_123"

# Connection pooling
max_connections = 10
timeout_secs = 30

# TLS Configuration (for production)
# tls_enabled = true
# tls_ca_cert = "/path/to/ca.pem"
# tls_client_cert = "/path/to/client.pem"
# tls_client_key = "/path/to/client.key"

1.4 Enable SurrealDB in Application

In Rust Code (syntaxis-core):

use workspace_core::persistence::{Database, SurrealDatabase};

#[tokio::main]
async fn main() -> Result<()> {
    // Embedded mode (no server required)
    let db = SurrealDatabase::new_memory().await?;
    // or
    let db = SurrealDatabase::new_file("/tmp/surrealdb.db").await?;

    // Server mode (requires `surreal start` running)
    // let db = SurrealDatabase::new_server(
    //     "ws://localhost:8000",
    //     "syntaxis",
    //     "projects",
    //     None,  // username
    //     None,  // password
    // ).await?;

    Ok(())
}

1.5 Verify Connection

# Test if server is running
curl http://localhost:8000/health
# Expected: {"status":"ok"}

# Check with surreal CLI
surreal sql --endpoint ws://localhost:8000
# Then at prompt:
# > SELECT 1;

---

Part 2: Docker Container Deployment

2.1 Create Dockerfile

File: Dockerfile.surrealdb

# Multi-stage build for minimal image size
FROM surrealdb/surrealdb:latest AS surrealdb

# Production runtime image
FROM debian:bookworm-slim

# Install minimal dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Copy SurrealDB binary from builder
COPY --from=surrealdb /usr/local/bin/surreal /usr/local/bin/surreal

# Create data directory
RUN mkdir -p /data && chmod 755 /data

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# Expose port
EXPOSE 8000

# Labels for container metadata
LABEL maintainer="syntaxis"
LABEL version="2.3"
LABEL description="SurrealDB 2.3 for syntaxis"

# Run SurrealDB
ENTRYPOINT ["surreal", "start"]
CMD ["--bind", "0.0.0.0:8000", "file:///data/surrealdb.db"]

# Volume for data persistence
VOLUME ["/data"]

2.2 Docker Compose Configuration

File: docker-compose.surrealdb.yml

version: "3.9"

services:
  surrealdb:
    # Option 1: Use official SurrealDB image
    image: surrealdb/surrealdb:latest

    # Option 2: Build from local Dockerfile
    # build:
    #   context: .
    #   dockerfile: Dockerfile.surrealdb

    container_name: workspace-surrealdb

    # Port mapping
    ports:
      - "8000:8000"

    # Volume for persistent data
    volumes:
      - surrealdb-data:/data

    # Environment variables
    environment:
      # SURREAL_LOG: debug  # Enable debug logging
      TZ: UTC

    # Command override
    command:
      - start
      - --bind
      - "0.0.0.0:8000"
      - --username
      - "admin"
      - --password
      - "workspace_password_123"
      - file:///data/surrealdb.db

    # Resource limits
    deploy:
      resources:
        limits:
          cpus: "2"
          memory: 2G
        reservations:
          cpus: "1"
          memory: 1G

    # Restart policy
    restart: unless-stopped

    # Health check
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 5s

    # Network
    networks:
      - workspace-net

  # Optional: Redis cache layer (for performance)
  redis:
    image: redis:7-alpine
    container_name: workspace-redis
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data
    restart: unless-stopped
    networks:
      - workspace-net

volumes:
  surrealdb-data:
    driver: local
  redis-data:
    driver: local

networks:
  workspace-net:
    driver: bridge

2.3 Docker Compose Usage

# Start SurrealDB
docker-compose -f docker-compose.surrealdb.yml up -d

# View logs
docker-compose -f docker-compose.surrealdb.yml logs -f surrealdb

# Connect to database
docker-compose -f docker-compose.surrealdb.yml exec surrealdb \
  surreal sql --endpoint ws://localhost:8000 \
  --username admin \
  --password workspace_password_123

# Stop services
docker-compose -f docker-compose.surrealdb.yml down

# Remove volumes (caution: deletes data)
docker-compose -f docker-compose.surrealdb.yml down -v

2.4 Multi-Environment Compose Override

File: docker-compose.override.yml (local development)

version: "3.9"

services:
  surrealdb:
    # Expose debug port
    ports:
      - "8000:8000"
      - "8001:8001"  # Debug port

    # Use memory backend for faster development
    command:
      - start
      - --bind
      - "0.0.0.0:8000"
      - memory

    # Less resource restriction for dev
    deploy:
      resources:
        limits:
          cpus: "4"
          memory: 4G

File: docker-compose.prod.yml (production)

version: "3.9"

services:
  surrealdb:
    image: surrealdb/surrealdb:2.3.0  # Pin version for prod

    # Stricter resource limits
    deploy:
      resources:
        limits:
          cpus: "1"
          memory: 1.5G
        reservations:
          cpus: "0.5"
          memory: 1G

    # Stricter restart policy
    restart: on-failure:5

---

Part 3: Kubernetes Deployment

3.1 Kubernetes ConfigMap

File: k8s/surrealdb-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: surrealdb-config
  namespace: workspace
spec:
  database.toml: |
    engine = "surrealdb"

    [surrealdb]
    url = "ws://surrealdb:8000"
    namespace = "syntaxis"
    database = "projects"
    max_connections = 20
    timeout_secs = 60

  init-schema.sql: |
    -- SurrealDB schema initialization
    DEFINE TABLE projects SCHEMAFULL;
    DEFINE TABLE checklist_items SCHEMAFULL;
    DEFINE TABLE phase_transitions SCHEMAFULL;
    DEFINE TABLE security_assessments SCHEMAFULL;
    DEFINE TABLE phase_history SCHEMAFULL;
    DEFINE TABLE tool_configurations SCHEMAFULL;
    DEFINE TABLE tool_dependencies SCHEMAFULL;
    DEFINE TABLE security_assessment_details SCHEMAFULL;
    DEFINE TABLE team_members SCHEMAFULL;

3.2 Kubernetes Secret

File: k8s/surrealdb-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: surrealdb-credentials
  namespace: workspace
type: Opaque
stringData:
  username: "admin"
  password: "secure_kubernetes_password_here_at_least_32_chars_minimum"
  # Note: In production, use external secret management (Vault, AWS Secrets Manager, etc)

3.3 SurrealDB StatefulSet

File: k8s/surrealdb-statefulset.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: surrealdb
  namespace: workspace
  labels:
    app: surrealdb
    version: "2.3"
spec:
  serviceName: surrealdb
  replicas: 1  # For HA, scale to 3 with clustering (future)
  selector:
    matchLabels:
      app: surrealdb
  template:
    metadata:
      labels:
        app: surrealdb
        version: "2.3"
    spec:
      # Node affinity for stability
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              preference:
                matchExpressions:
                  - key: node-type
                    operator: In
                    values: ["database"]

      # Security context
      securityContext:
        runAsNonRoot: true
        runAsUser: 999
        fsGroup: 999

      # Init containers for schema setup
      initContainers:
        - name: init-schema
          image: surrealdb/surrealdb:2.3
          command:
            - /bin/sh
            - -c
            - |
              # Wait for SurrealDB to be ready
              surreal sql \
                --endpoint ws://localhost:8000 \
                --username admin \
                --password $SURREALDB_PASSWORD \
                < /config/init-schema.sql
          env:
            - name: SURREALDB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: surrealdb-credentials
                  key: password
          volumeMounts:
            - name: config
              mountPath: /config

      containers:
        - name: surrealdb
          image: surrealdb/surrealdb:2.3
          imagePullPolicy: IfNotPresent

          # Resource requests and limits
          resources:
            requests:
              cpu: "500m"
              memory: "512Mi"
            limits:
              cpu: "2000m"
              memory: "2Gi"

          # Port exposure
          ports:
            - name: http
              containerPort: 8000
              protocol: TCP

          # Environment variables
          env:
            - name: SURREALDB_USER
              valueFrom:
                secretKeyRef:
                  name: surrealdb-credentials
                  key: username
            - name: SURREALDB_PASS
              valueFrom:
                secretKeyRef:
                  name: surrealdb-credentials
                  key: password

          # Startup probe (for slow startup)
          startupProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 10
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 30

          # Liveness probe
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 10
            periodSeconds: 30
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 3

          # Readiness probe
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 10
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 3

          # Volume mounts
          volumeMounts:
            - name: config
              mountPath: /etc/surrealdb
            - name: data
              mountPath: /data

          # Security context for container
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL

          # Startup command
          args:
            - start
            - --bind
            - 0.0.0.0:8000
            - --username
            - $(SURREALDB_USER)
            - --password
            - $(SURREALDB_PASS)
            - file:///data/surrealdb.db

      # Volumes
      volumes:
        - name: config
          configMap:
            name: surrealdb-config

  # Persistent volume claim
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: standard  # Change to appropriate storage class
        resources:
          requests:
            storage: 10Gi  # Adjust based on data needs

3.4 Kubernetes Service

File: k8s/surrealdb-service.yaml

---
# Headless service for StatefulSet
apiVersion: v1
kind: Service
metadata:
  name: surrealdb
  namespace: workspace
  labels:
    app: surrealdb
spec:
  clusterIP: None  # Headless service
  selector:
    app: surrealdb
  ports:
    - name: http
      port: 8000
      targetPort: 8000
      protocol: TCP

---
# Load balancer service for external access (optional)
apiVersion: v1
kind: Service
metadata:
  name: surrealdb-lb
  namespace: workspace
  labels:
    app: surrealdb
spec:
  type: LoadBalancer
  selector:
    app: surrealdb
  ports:
    - name: http
      port: 8000
      targetPort: 8000
      protocol: TCP

---
# ClusterIP for internal access
apiVersion: v1
kind: Service
metadata:
  name: surrealdb-internal
  namespace: workspace
  labels:
    app: surrealdb
spec:
  type: ClusterIP
  selector:
    app: surrealdb
  ports:
    - name: http
      port: 8000
      targetPort: 8000
      protocol: TCP

3.5 Kubernetes Deployment Script

File: k8s/deploy.sh

#!/bin/bash
set -e

NAMESPACE="workspace"
CONTEXT="kubernetes-cluster"  # Change to your context

echo "📦 Deploying SurrealDB to Kubernetes..."

# Switch context
kubectl config use-context $CONTEXT
echo "✓ Using context: $CONTEXT"

# Create namespace if needed
kubectl create namespace $NAMESPACE --dry-run=client -o yaml | kubectl apply -f -
echo "✓ Namespace: $NAMESPACE"

# Apply resources
echo "📝 Applying ConfigMap..."
kubectl apply -f k8s/surrealdb-config.yaml

echo "🔐 Applying Secrets..."
kubectl apply -f k8s/surrealdb-secret.yaml

echo "📊 Applying StatefulSet..."
kubectl apply -f k8s/surrealdb-statefulset.yaml

echo "🌐 Applying Services..."
kubectl apply -f k8s/surrealdb-service.yaml

# Wait for rollout
echo "⏳ Waiting for SurrealDB to be ready..."
kubectl rollout status statefulset/surrealdb -n $NAMESPACE --timeout=5m

# Verify
echo "✓ Checking pod status..."
kubectl get pods -n $NAMESPACE -l app=surrealdb

echo "✓ Checking services..."
kubectl get svc -n $NAMESPACE -l app=surrealdb

echo ""
echo "🎉 SurrealDB deployed successfully!"
echo ""
echo "Access information:"
echo "  Internal: ws://surrealdb:8000"
echo "  External: kubectl port-forward -n $NAMESPACE svc/surrealdb-lb 8000:8000"
echo ""
echo "To connect:"
echo "  kubectl -n $NAMESPACE exec -it surrealdb-0 -- surreal sql"

3.6 Kubernetes Values for Helm (Optional)

File: k8s/helm-values.yaml (if using Helm)

# SurrealDB Helm Chart Values
# Install: helm install surrealdb surrealdb/surrealdb -f helm-values.yaml

replicaCount: 1

image:
  repository: surrealdb/surrealdb
  pullPolicy: IfNotPresent
  tag: "2.3"

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  create: true
  annotations: {}
  name: "surrealdb"

podAnnotations: {}

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 999
  fsGroup: 999

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  capabilities:
    drop:
      - ALL

service:
  type: ClusterIP
  port: 8000

resources:
  limits:
    cpu: 2000m
    memory: 2Gi
  requests:
    cpu: 500m
    memory: 512Mi

persistence:
  enabled: true
  storageClassName: standard
  size: 10Gi
  path: /data

auth:
  username: admin
  password: "secure_password_here"

config:
  namespace: "syntaxis"
  database: "projects"
  logLevel: info

---

Part 4: Production Considerations

4.1 High Availability Setup

# For HA, scale replicas to 3 and configure clustering
# k8s/surrealdb-statefulset.yaml (updated for HA)
spec:
  replicas: 3  # Scale to 3 nodes

  # In container args, add clustering:
  args:
    - start
    - --bind
    - 0.0.0.0:8000
    - --auth
    - enable
    - --log
    - debug
    - --clock
    - "2ms"  # Clock precision for clustering
    # Enable clustering for multi-node setup
    # Note: Requires SurrealDB enterprise or cloud version

4.2 Backup and Recovery

# Backup SurrealDB data
kubectl -n workspace exec surrealdb-0 -- \
  tar czf /tmp/backup.tar.gz /data/

# Extract backup
kubectl -n workspace cp surrealdb-0:/tmp/backup.tar.gz ./backup.tar.gz

# Restore from backup
kubectl -n workspace cp backup.tar.gz surrealdb-0:/tmp/
kubectl -n workspace exec surrealdb-0 -- \
  tar xzf /tmp/backup.tar.gz -C /

4.3 Monitoring and Logging

# Prometheus ServiceMonitor (if using Prometheus)
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: surrealdb
  namespace: workspace
spec:
  selector:
    matchLabels:
      app: surrealdb
  endpoints:
    - port: http
      interval: 30s
      path: /metrics

4.4 Network Security

# NetworkPolicy to restrict access
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: surrealdb-netpol
  namespace: workspace
spec:
  podSelector:
    matchLabels:
      app: surrealdb
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: workspace-app
      ports:
        - protocol: TCP
          port: 8000
  egress:
    - to:
        - podSelector: {}
      ports:
        - protocol: TCP
          port: 53  # DNS

---

Part 5: Troubleshooting

Issue: Connection Refused

Error: connection refused when connecting to localhost:8000

Solution:

# Check if SurrealDB is running
ps aux | grep surreal

# If not running, start it
surreal start --bind 127.0.0.1:8000 memory

# Test connection
curl http://localhost:8000/health

Issue: Permission Denied in Docker

Error: permission denied when accessing /data

Solution:

# Fix file permissions
docker-compose exec surrealdb chmod 755 /data
docker-compose exec surrealdb chown -R 999:999 /data

Issue: Kubernetes Pod CrashLoopBackOff

Error: Pod keeps restarting

Solution:

# Check logs
kubectl logs -n workspace surrealdb-0 --previous

# Check events
kubectl describe pod -n workspace surrealdb-0

# Common causes:
# 1. Credentials incorrect in secret
# 2. PVC not available
# 3. Insufficient resources

Issue: Slow Queries

Solution:

# Enable query logging
surreal start --log debug --bind 127.0.0.1:8000 file:///tmp/surrealdb.db

# Check query performance in logs
# Optimize indexes if needed

---

Summary

Quick Start

Local Development:

# Terminal 1: Start SurrealDB
surreal start --bind 127.0.0.1:8000 memory

# Terminal 2: Run your application
cargo run -p syntaxis-cli

Docker Development:

docker-compose -f docker-compose.surrealdb.yml up -d
# Update configs/database.toml to use:
# url = "ws://localhost:8000"

Kubernetes Production:

cd k8s
./deploy.sh
# Access: kubectl port-forward -n workspace svc/surrealdb 8000:8000

---

References

• SurrealDB Official Docs
• SurrealDB Docker Hub
• Kubernetes Documentation
• syntaxis PERSISTENCE_QUICK_START.md
• SURREALDB_2_3_MIGRATION.md

---

Last Updated: November 15, 2025 Version: SurrealDB 2.3 Status: Production Ready