kogral/docs/architecture/adrs/006-surrealdb-v3-engine-abstraction.md

ADR-006: SurrealDB 3.0 Engine Abstraction

Status: Accepted

Date: 2026-02-21

Deciders: Architecture Team

Supersedes: ADR-003: Hybrid Storage Strategy (SurrealDB connection model)

---

Context

SurrealDB 2.x required a concrete connection type at compile time (Surreal<Ws>, Surreal<Mem>, etc.), forcing a single engine choice per binary. This created three problems:

1. No runtime engine selection: switching from embedded to remote required recompilation
2. Tight test coupling: tests depended on whichever engine was compiled in
3. No dual-layout support: using SurrealKV for graph data and RocksDB for hot data required separate crates or unsafe casting

SurrealDB 3.0 introduces surrealdb::engine::any::connect(url) returning Surreal<Any> — a type-erased connection dispatched at runtime by URL scheme:

URL scheme	Engine	Characteristics
mem://	In-memory	Ephemeral, test isolation
surrealkv://path	SurrealKV (B-tree)	Embedded, relational/graph data
rocksdb://path	RocksDB (LSM)	Embedded, append-heavy hot data
ws://host:port	WebSocket	Remote, team/shared deployment

This makes the engine a pure config concern.

---

Decision

Use Surreal<Any> throughout SurrealDbStorage, selecting the engine from SurrealEngineConfig at runtime via URL dispatch.

Config Schema

SurrealEngineConfig is a serde-tagged enum serialized with tag = "engine":

#[serde(tag = "engine", rename_all = "snake_case")]
pub enum SurrealEngineConfig {
    Mem,
    SurrealKv { path: String },
    RocksDb   { path: String },
    Ws        { url: String },
}

to_url() maps variants to the URL scheme engine::any::connect expects:

impl SurrealEngineConfig {
    pub fn to_url(&self) -> String {
        match self {
            Self::Mem               => "mem://".to_string(),
            Self::SurrealKv { path } => format!("surrealkv://{path}"),
            Self::RocksDb   { path } => format!("rocksdb://{path}"),
            Self::Ws        { url }  => url.clone(),
        }
    }
}

Dual Hot/Cold Layout

SurrealDbStorage holds two independent Surreal<Any> connections:

pub struct SurrealDbStorage {
    graph_db: Surreal<Any>,   // SurrealKV default — nodes, edges, metadata
    hot_db:   Surreal<Any>,   // RocksDB default  — embeddings, session logs
    namespace: String,
}

Default production layout:

secondary.surrealdb = {
  graph = { engine = "surreal_kv", path = ".kogral/db/graph" },
  hot   = { engine = "rocks_db",   path = ".kogral/db/hot"   },
  namespace = "kogral",
}

Test layout (in-memory, no filesystem side effects):

secondary.surrealdb = {
  graph = { engine = "mem" },
  hot   = { engine = "mem" },
  namespace = "test",
}

Remote team deployment (single SurrealDB instance, two databases):

secondary.surrealdb = {
  graph = { engine = "ws", url = "ws://kb.company.com:8000" },
  hot   = { engine = "ws", url = "ws://kb.company.com:8000" },
  namespace = "engineering",
}

Storage Factory

storage::factory::build() performs the runtime dispatch, keeping SurrealDB-specific logic behind the surrealdb-backend feature gate:

#[allow(clippy::unused_async)]
pub async fn build(config: &StorageConfig, base_path: PathBuf) -> Result<Box<dyn Storage>> {
    #[cfg(feature = "surrealdb-backend")]
    if config.secondary.enabled && config.secondary.storage_type == SecondaryStorageType::Surrealdb {
        let db = SurrealDbStorage::from_config(&config.secondary.surrealdb).await?;
        return Ok(Box::new(db));
    }
    Ok(match config.primary {
        StorageType::Memory     => Box::new(MemoryStorage::new()),
        StorageType::Filesystem => Box::new(FilesystemStorage::new(base_path)),
    })
}

Type Erasure Composition

impl Storage for Box<dyn Storage> enables EventingStorage<Box<dyn Storage>> without knowing the concrete backend at compile time:

#[async_trait]
impl Storage for Box<dyn Storage> {
    async fn save_graph(&mut self, graph: &Graph) -> Result<()> {
        (**self).save_graph(graph).await
    }
    // ... full delegation for all methods
}

CRUD via serde_json::Value

SurrealDB 3.0 removed the IntoSurrealValue/SurrealValue traits. All CRUD goes through serde_json::Value as the intermediary:

// save_node
let row = serde_json::to_value(node)
    .map_err(|e| KbError::Storage(e.to_string()))?;
let _: Option<serde_json::Value> = self.graph_db
    .upsert(("nodes", format!("{graph_name}__{}", node.id)))
    .content(row)
    .await
    .map_err(|e| KbError::Storage(e.to_string()))?;

.bind() parameters require 'static values — &str arguments must be .to_string():

let nodes: Vec<Node> = self.graph_db
    .query("SELECT * FROM nodes WHERE project = $g")
    .bind(("g", graph_name.to_string()))   // .to_string() required
    .await...

Error Handling

KbError::Database(#[from] surrealdb::Error) was removed. All SurrealDB errors convert via map_err(|e| KbError::Storage(e.to_string())), avoiding #[from] coupling to a feature-gated type that would break compilation on default features.

---

Consequences

Positive

• Engine selection is a config value, not a compile-time decision
• Tests run fully in-memory (engine = "mem") with zero filesystem side effects
• Dual layout (SurrealKV + RocksDB) is the embedded production default
• Remote deployment (WebSocket) requires only a config change
• Storage trait consumers never see SurrealDB types — Box<dyn Storage> is the boundary

Negative

• Surreal<Any> has slightly higher dispatch overhead than concrete types (negligible vs. I/O)
• serde_json::Value intermediary adds one extra allocation per CRUD call
• engine::any requires all four engine feature flags compiled in when surrealdb-backend is enabled (larger binary)

---

Feature Matrix

[features]
default           = ["filesystem"]
filesystem        = []
surrealdb-backend = ["dep:surrealdb"]
nats-events       = ["dep:platform-nats", "dep:bytes"]
orchestration     = ["nats-events", "dep:stratum-orchestrator"]
full              = ["surrealdb-backend", "nats-events", "orchestration"]

SurrealDB dependency activates all four engines:

surrealdb = { workspace = true, optional = true,
              features = ["kv-surrealkv", "kv-rocksdb", "protocol-ws", "rustls"] }

---

References

• SurrealDB engine::any docs
• SurrealDbStorage
• Storage Factory
• Config Schema
• Nickel Defaults

---

Revision History

Date	Author	Change
2026-02-21	Architecture Team	Initial decision — SurrealDB 3.0 + engine::any

---

Previous ADR: ADR-005: MCP Protocol

Next ADR: ADR-007: NATS Event Publishing