## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Config-Driven Architecture

The KOGRAL follows a config-driven architecture where all behavior is defined through Nickel configuration files rather than hardcoded in Rust.

## Philosophy

"Configuration, not code, defines behavior"

Instead of hardcoding storage backends, embedding providers, or query parameters, KB uses a layered configuration system that composes settings from multiple sources:

1. Schema contracts (type definitions)
2. Defaults (base values)
3. Mode overlays (dev/prod/test optimizations)
4. User customizations (project-specific overrides)

This approach provides:

- ✅ Type safety - Nickel contracts validate configuration before runtime
- ✅ Composability - Mix and match configurations for different environments
- ✅ Discoverability - Self-documenting schemas with inline documentation
- ✅ Hot-reload - Change behavior without recompiling Rust code
- ✅ Double validation - Nickel contracts + serde ensure correctness

## Configuration Composition Flow

The configuration system uses a four-layer composition pattern:

### Layer 1: Schema Contracts

Location: `schemas/kb/contracts.ncl`

Purpose: Define types and validation rules using Nickel contracts.

Example:

    {
      StorageType = [| 'filesystem, 'memory, 'surrealdb |],

      StorageConfig = {
        primary | StorageType
          | doc "Primary storage backend"
          | default = 'filesystem,

        secondary | SecondaryStorageConfig
          | doc "Optional secondary storage"
          | default = { enabled = false },
      },
    }

Benefits:

- Enum validation (only valid storage types accepted)
- Required vs optional fields
- Default values for optional fields
- Documentation attached to types

### Layer 2: Defaults

Location: `schemas/kb/defaults.ncl`

Purpose: Provide sensible base values for all configuration options.

Example:

    {
      base = {
        storage = {
          primary = 'filesystem,
          secondary = {
            enabled = false,
            type = 'surrealdb,
            url = "ws://localhost:8000",
          },
        },
        embeddings = {
          enabled = true,
          provider = 'fastembed,
          model = "BAAI/bge-small-en-v1.5",
          dimensions = 384,
        },
      } | contracts.KbConfig,
    }

Validated by: `contracts.KbConfig` contract ensures defaults are valid.

### Layer 3: Mode Overlays

Location: `schemas/kb/modes/{dev,prod,test}.ncl`

Purpose: Environment-specific optimizations and tuning.

#### Development Mode (`dev.ncl`)

Optimized for: Fast iteration, local development, debugging

    {
      storage = {
        primary = 'filesystem,
        secondary = { enabled = false },  # No database overhead
      },
      embeddings = {
        provider = 'fastembed,  # Local, no API costs
      },
      sync = {
        auto_index = false,  # Manual control
      },
    }

#### Production Mode (`prod.ncl`)

Optimized for: Performance, reliability, scalability

    {
      storage = {
        secondary = { enabled = true },  # SurrealDB for scale
      },
      embeddings = {
        provider = 'openai,  # High-quality cloud embeddings
        model = "text-embedding-3-small",
        dimensions = 1536,
      },
      sync = {
        auto_index = true,
        debounce_ms = 300,  # Fast response
      },
    }

#### Test Mode (`test.ncl`)

Optimized for: Fast tests, isolation, determinism

    {
      storage = {
        primary = 'memory,  # Ephemeral, no disk I/O
      },
      embeddings = {
        enabled = false,  # Disable for speed
      },
      sync = {
        auto_index = false,
        debounce_ms = 0,  # No delays in tests
      },
    }

### Layer 4: User Customizations

Location: `.kb-config/core/kb.ncl` or `.kb-config/platform/{dev,prod,test}.ncl`

Purpose: Project-specific or deployment-specific overrides.

Example (user project config):

    let mode = import "../../schemas/kb/modes/dev.ncl" in

    let user_custom = {
      graph = {
        name = "my-project",
      },
      embeddings = {
        provider = 'claude,  # Override to use Claude
        model = "claude-3-haiku-20240307",
      },
      query = {
        similarity_threshold = 0.7,  # Stricter threshold
      },
    } in

    helpers.compose_config defaults.base mode user_custom
      | contracts.KbConfig

## Composition Mechanism

The `helpers.ncl` module provides the composition function:

    {
      # Recursively merge with override precedence
      merge_with_override = fun base override => /* ... */,

      # Compose three layers
      compose_config = fun defaults mode_config user_custom =>
        let with_mode = merge_with_override defaults mode_config in
        merge_with_override with_mode user_custom,
    }

Merge behavior:

- Records are merged recursively
- Override values take precedence over base values
- Arrays are not merged, override replaces base
- Null in override keeps base value

Example merge:

    base = { storage = { primary = 'filesystem }, embeddings = { enabled = true } }
    override = { storage = { primary = 'memory } }
    # Result: { storage = { primary = 'memory }, embeddings = { enabled = true } }

## Export to JSON

Once composed, the Nickel configuration is exported to JSON for Rust consumption:

    nickel export --format json .kb-config/core/kb.ncl > .kb-config/targets/kb-core.json

Output (`.kb-config/targets/kb-core.json`):

    {
      "graph": {
        "name": "my-project",
        "version": "1.0.0"
      },
      "storage": {
        "primary": "memory",
        "secondary": {
          "enabled": false
        }
      },
      "embeddings": {
        "enabled": true,
        "provider": "claude",
        "model": "claude-3-haiku-20240307",
        "dimensions": 768
      }
    }

## Rust Integration

The Rust code deserializes the JSON into typed structs using serde:

    #![allow(unused)]
    fn main() {
    use serde::{Deserialize, Serialize};

    #[derive(Debug, Deserialize, Serialize)]
    pub struct KbConfig {
        pub graph: GraphConfig,
        pub storage: StorageConfig,
        pub embeddings: EmbeddingConfig,
        pub templates: TemplateConfig,
        pub query: QueryConfig,
        pub mcp: McpConfig,
        pub sync: SyncConfig,
    }

    impl KbConfig {
        pub fn from_file(path: &Path) -> Result<Self> {
            let json = std::fs::read_to_string(path)?;
            let config: KbConfig = serde_json::from_str(&json)?;
            Ok(config)
        }
    }
    }

Usage in kb-core:

    #![allow(unused)]
    fn main() {
    let config = KbConfig::from_file(".kb-config/targets/kb-core.json")?;

    // Config drives behavior
    let storage: Box<dyn Storage> = match config.storage.primary {
        StorageType::Filesystem => Box::new(FilesystemStorage::new(&config)?),
        StorageType::Memory => Box::new(MemoryStorage::new()),
        StorageType::SurrealDb => Box::new(SurrealDbStorage::new(&config).await?),
    };

    let embeddings: Box<dyn EmbeddingProvider> = match config.embeddings.provider {
        EmbeddingProviderType::FastEmbed => Box::new(FastEmbedProvider::new()?),
        EmbeddingProviderType::OpenAI => Box::new(RigEmbeddingProvider::openai(&config)?),
        EmbeddingProviderType::Claude => Box::new(RigEmbeddingProvider::claude(&config)?),
        EmbeddingProviderType::Ollama => Box::new(RigEmbeddingProvider::ollama(&config)?),
    };
    }

## Double Validation

Configuration is validated twice:

### 1. Nickel Contract Validation

At export time, Nickel validates:

- ✅ Types match contracts (e.g., `primary | StorageType`)
- ✅ Required fields are present
- ✅ Enums have valid values
- ✅ Nested structure is correct

Error example:

    error: contract broken by a value
      ┌─ .kb-config/core/kb.ncl:15:5
      │
    15│     primary = 'invalid,
      │     ^^^^^^^^^^^^^^^^^^^ applied to this expression
      │
      = This value is not in the enum ['filesystem, 'memory, 'surrealdb]

### 2. Serde Deserialization Validation

At runtime, serde validates:

- ✅ JSON structure matches Rust types
- ✅ Field names match (with rename support)
- ✅ Values can be converted to Rust types
- ✅ Required fields are not null

Error example:

    #![allow(unused)]
    fn main() {
    Error: missing field `graph` at line 1 column 123
    }

## Benefits of Config-Driven Architecture

### 1. Zero Hardcoding

Bad (hardcoded):

    #![allow(unused)]
    fn main() {
    // Hardcoded - requires recompilation to change
    let storage = FilesystemStorage::new("/fixed/path");
    let threshold = 0.6;
    }

Good (config-driven):

    #![allow(unused)]
    fn main() {
    // Config-driven - change via .ncl file
    let storage = create_storage(&config)?;
    let threshold = config.query.similarity_threshold;
    }

### 2. Environment Flexibility

Same codebase, different behavior:

    # Development
    nickel export .kb-config/platform/dev.ncl > targets/kb-core.json
    # → Filesystem storage, fastembed, no auto-sync

    # Production
    nickel export .kb-config/platform/prod.ncl > targets/kb-core.json
    # → SurrealDB enabled, OpenAI embeddings, auto-sync

    # Testing
    nickel export .kb-config/platform/test.ncl > targets/kb-core.json
    # → In-memory storage, no embeddings, isolated

### 3. Self-Documenting

Nickel contracts include inline documentation:

    StorageType = [| 'filesystem, 'memory, 'surrealdb |]
      | doc "Storage backend type: filesystem (git-tracked), memory (ephemeral), surrealdb (scalable)",

IDEs can show this documentation when editing `.ncl` files.

### 4. Type-Safe Evolution

When adding new features:

1. Update contract in `contracts.ncl`
2. Add default in `defaults.ncl`
3. Export validates existing configs
4. Rust compilation validates deserialization

Breaking changes are caught before runtime.

### 5. Testability

Different test scenarios without code changes:

    # test-semantic-search.ncl
    let test_config = defaults.base & {
      embeddings = { enabled = true, provider = 'fastembed },
      query = { similarity_threshold = 0.3 },
    } in test_config

    #![allow(unused)]
    fn main() {
    #[test]
    fn test_semantic_search() {
        let config = KbConfig::from_file("test-semantic-search.json")?;
        // Config drives test behavior
    }
    }

## Configuration Discovery

KB tools automatically discover configuration:

1. Check `.kb-config/targets/kb-core.json` (pre-exported)
2. Check `.kb-config/core/kb.ncl` (export on-demand)
3. Check environment variable `KB_CONFIG`
4. Fall back to embedded defaults

    #![allow(unused)]
    fn main() {
    impl KbConfig {
        pub fn discover() -> Result<Self> {
            if let Ok(config) = Self::from_file(".kb-config/targets/kb-core.json") {
                return Ok(config);
            }

            if Path::new(".kb-config/core/kb.ncl").exists() {
                // Export and load
                let output = Command::new("nickel")
                    .args(["export", "--format", "json", ".kb-config/core/kb.ncl"])
                    .output()?;
                return serde_json::from_slice(&output.stdout)?;
            }

            if let Ok(path) = std::env::var("KB_CONFIG") {
                return Self::from_file(&path);
            }

            Ok(Self::default())  // Embedded defaults
        }
    }
    }

## Integration with justfile

The `justfile` integrates configuration validation:

    # Validate all Nickel configs
    nickel-validate-all:
        @echo "Validating Nickel schemas..."
        nickel typecheck schemas/kb/contracts.ncl
        nickel typecheck schemas/kb/defaults.ncl
        nickel typecheck schemas/kb/helpers.ncl
        nickel typecheck schemas/kb/modes/dev.ncl
        nickel typecheck schemas/kb/modes/prod.ncl
        nickel typecheck schemas/kb/modes/test.ncl
        nickel typecheck .kb-config/core/kb.ncl

    # Export all platform configs
    nickel-export-all:
        @echo "Exporting platform configs to JSON..."
        @mkdir -p .kb-config/targets
        nickel export --format json .kb-config/platform/dev.ncl > .kb-config/targets/kb-dev.json
        nickel export --format json .kb-config/platform/prod.ncl > .kb-config/targets/kb-prod.json
        nickel export --format json .kb-config/platform/test.ncl > .kb-config/targets/kb-test.json
        @echo "Exported 3 configurations to .kb-config/targets/"

Usage:

    just nickel::validate-all  # Check configs are valid
    just nickel::export-all    # Generate JSON for all environments

## Best Practices

### 1. Never Hardcode

If it's behavior, it's config:

- Storage paths
- API endpoints
- Thresholds
- Timeouts
- Feature flags
- Provider selection

### 2. Use Modes for Environment Differences

Don't put environment-specific values in user config:

Bad:

    # user config with env-specific values
    {
      storage = {
        url = if env == "prod" then "prod-url" else "dev-url"  # Don't do this
      }
    }

Good:

    # modes/prod.ncl
    { storage = { url = "prod-url" } }

    # modes/dev.ncl
    { storage = { url = "dev-url" } }

    # user config is environment-agnostic
    { graph = { name = "my-project" } }

### 3. Document Complex Fields

Use Nickel's `doc` metadata:

    similarity_threshold | Number
      | doc "Minimum cosine similarity (0-1) for semantic search matches. Higher = stricter."
      | default = 0.6,

### 4. Validate Early

Run `nickel typecheck` in CI/CD before building Rust code.

### 5. Version Configs

Track `.ncl` files in git, ignore `.kb-config/targets/*.json` (generated).

## See Also

- Schema Reference: Configuration Schema
- User Guide: Configuration Guide
- ADR: Why Nickel vs TOML
- Examples: `.kb-config/core/kb.ncl`, `.kb-config/platform/*.ncl`