provisioning/docs/src/infrastructure/nickel-guide.md

# Nickel Guide

Comprehensive guide to using Nickel as the infrastructure-as-code language for the Provisioning platform.

## Critical Principle: Nickel is Source of Truth

**TYPE-SAFETY ALWAYS REQUIRED**: ALL configurations MUST be type-safe and validated via Nickel.
TOML is NOT acceptable as source of truth. Validation is NOT optional, NOT "progressive",
NOT "production-only". This applies to ALL profiles (developer, production, cicd).

Nickel is the PRIMARY IaC language. TOML files are GENERATED OUTPUT ONLY, never the source.

## Why Nickel

Nickel provides:

- **Type Safety**: Static type checking catches errors before deployment
- **Lazy Evaluation**: Efficient configuration composition and merging
- **Contract System**: Schema validation with gradual typing
- **Record Merging**: Powerful composition without duplication
- **LSP Support**: IDE integration for autocomplete and validation
- **Human-Readable**: Clear syntax for infrastructure definition

## Installation

```bash
# macOS (Homebrew)
brew install nickel

# Linux (Cargo)
cargo install nickel-lang-cli

# Verify installation
nickel --version  # 1.15.1+
```

## Core Concepts

### Records and Fields

Records are the fundamental data structure in Nickel:

```nickel
{
  name = "my-server"
  plan = "medium"
  zone = "de-fra1"
}
```

### Type Annotations

Add type safety with contracts:

```nickel
{
  name : String = "my-server"
  plan : String = "medium"
  cpu_count : Number = 4
  enabled : Bool = true
}
```

### Record Merging

Compose configurations by merging records:

```nickel
let base_config = {
  provider = "upcloud"
  region = "de-fra1"
} in

let server_config = base_config & {
  name = "web-01"
  plan = "medium"
} in

server_config
```

Result:

```nickel
{
  provider = "upcloud"
  region = "de-fra1"
  name = "web-01"
  plan = "medium"
}
```

### Contracts (Schema Validation)

Define contracts to validate structure:

```nickel
let ServerContract = {
  name | String
  plan | String | default = "small"
  zone | String | default = "de-fra1"
  cpu | Number | optional
} in

{
  name = "my-server"
  plan = "large"
} | ServerContract
```

## Three-File Pattern (Provisioning Standard)

The platform uses a standardized three-file pattern for all schemas:

### 1. `contracts.ncl` - Type Definitions

Defines the schema contracts:

```nickel
# contracts.ncl
{
  Server = {
    name | String
    plan | String | default = "small"
    zone | String | default = "de-fra1"
    disk_size_gb | Number | default = 25
    backup_enabled | Bool | default = false
    role | | [ 'control, 'worker, 'standalone | ] | optional
  }

  Infrastructure = {
    servers | Array Server
    provider | String
    environment | | [ 'development, 'staging, 'production | ]
  }
}
```

### 2. `defaults.ncl` - Default Values

Provides sensible defaults:

```nickel
# defaults.ncl
{
  server = {
    name = "unnamed-server"
    plan = "small"
    zone = "de-fra1"
    disk_size_gb = 25
    backup_enabled = false
  }

  infrastructure = {
    servers = []
    provider = "local"
    environment = 'development
  }
}
```

### 3. `main.ncl` - Entry Point

Combines contracts and defaults, provides makers:

```nickel
# main.ncl
let contracts_lib = import "./contracts.ncl" in
let defaults_lib = import "./defaults.ncl" in

{
  # Direct access to defaults (for inspection)
  defaults = defaults_lib

  # Convenience makers (90% of use cases)
  make_server | not_exported = fun overrides =>
    defaults_lib.server & overrides

  make_infrastructure | not_exported = fun overrides =>
    defaults_lib.infrastructure & overrides

  # Default instances (bare defaults)
  DefaultServer = defaults_lib.server
  DefaultInfrastructure = defaults_lib.infrastructure
}
```

### Usage Example

```nickel
# user-infra.ncl
let infra_lib = import "provisioning/schemas/infrastructure/main.ncl" in

infra_lib.make_infrastructure {
  provider = "upcloud"
  environment = 'production
  servers = [
    infra_lib.make_server {
      name = "web-01"
      plan = "medium"
      backup_enabled = true
    }
    infra_lib.make_server {
      name = "web-02"
      plan = "medium"
      backup_enabled = true
    }
  ]
}
```

## Hybrid Interface Pattern

Records can be used both as functions (makers) and as plain data:

```nickel
let config_lib = import "./config.ncl" in

# Use as function (with overrides)
let custom_config = config_lib.make_server { name = "custom" } in

# Use as plain data (defaults)
let default_config = config_lib.DefaultServer in

{
  custom = custom_config
  default = default_config
}
```

## Record Merging Strategies

### Priority Merging (Default)

```nickel
let base = { a = 1, b = 2 } in
let override = { b = 3, c = 4 } in
base & override
# Result: { a = 1, b = 3, c = 4 }
```

### Recursive Merging

```nickel
let base = {
  server = { cpu = 2, ram = 4 }
} in

let override = {
  server = { ram = 8, disk = 100 }
} in

std.record.merge_all [base, override]
# Result: { server = { cpu = 2, ram = 8, disk = 100 } }
```

## Lazy Evaluation

Nickel evaluates expressions lazily, only when needed:

```nickel
let expensive_computation = std.string.join " " ["a", "b", "c"] in

{
  # Only evaluated when accessed
  computed_field = expensive_computation

  # Conditional evaluation
  conditional = if environment == 'production then
    expensive_computation
  else
    "dev-value"
}
```

## Schema Organization

The platform organizes Nickel schemas by domain:

```text
provisioning/schemas/
├── main.ncl                  # Top-level entry point
├── config/                   # Configuration schemas
│   ├── settings/
│   │   ├── main.ncl
│   │   ├── contracts.ncl
│   │   └── defaults.ncl
│   └── defaults/
│       ├── main.ncl
│       ├── contracts.ncl
│       └── defaults.ncl
├── infrastructure/           # Infrastructure definitions
│   ├── servers/
│   ├── networks/
│   └── storage/
├── deployment/               # Deployment schemas
├── services/                 # Service configurations
├── operations/               # Operational schemas
└── generator/                # Runtime schema generation
```

## Type System

### Primitive Types

```nickel
{
  string_field : String = "text"
  number_field : Number = 42
  bool_field : Bool = true
}
```

### Array Types

```nickel
{
  names : Array String = ["alice", "bob", "charlie"]
  ports : Array Number = [80, 443, 8080]
}
```

### Enum Types

```nickel
{
  environment : | [ 'development, 'staging, 'production | ] = 'production
  role : | [ 'control, 'worker, 'standalone | ] = 'worker
}
```

### Optional Fields

```nickel
{
  required_field : String = "value"
  optional_field : String | optional
}
```

### Default Values

```nickel
{
  with_default : String | default = "default-value"
}
```

## Validation Patterns

### Runtime Validation

```nickel
let validate_plan = fun plan =>
  if plan == "small" | | plan == "medium" | | plan == "large" then
    plan
  else
    std.fail "Invalid plan: must be small, medium, or large"
in

{
  plan = validate_plan "medium"
}
```

### Contract-Based Validation

```nickel
let PlanContract = | [ 'small, 'medium, 'large | ] in

{
  plan | PlanContract = 'medium
}
```

## Real-World Examples

### Simple Server Configuration

```nickel
{
  metadata = {
    name = "demo-server"
    provider = "upcloud"
    environment = 'development
  }

  infrastructure = {
    servers = [
      {
        name = "web-01"
        plan = "medium"
        zone = "de-fra1"
        disk_size_gb = 50
        backup_enabled = true
        role = 'standalone
      }
    ]
  }

  services = {
    taskservs = ["containerd", "docker"]
  }
}
```

### Kubernetes Cluster Configuration

```nickel
{
  metadata = {
    name = "k8s-prod"
    provider = "upcloud"
    environment = 'production
  }

  infrastructure = {
    servers = [
      {
        name = "k8s-control-01"
        plan = "medium"
        role = 'control
        zone = "de-fra1"
        disk_size_gb = 50
        backup_enabled = true
      }
      {
        name = "k8s-worker-01"
        plan = "large"
        role = 'worker
        zone = "de-fra1"
        disk_size_gb = 100
        backup_enabled = true
      }
      {
        name = "k8s-worker-02"
        plan = "large"
        role = 'worker
        zone = "de-fra1"
        disk_size_gb = 100
        backup_enabled = true
      }
    ]
  }

  services = {
    taskservs = ["containerd", "etcd", "kubernetes", "cilium", "rook-ceph"]
  }

  kubernetes = {
    version = "1.28.0"
    pod_cidr = "10.244.0.0/16"
    service_cidr = "10.96.0.0/12"
    container_runtime = "containerd"
    cri_socket = "/run/containerd/containerd.sock"
  }
}
```

### Multi-Provider Batch Workflow

```nickel
{
  batch_workflow = {
    operations = [
      {
        id = "aws-cluster"
        provider = "aws"
        region = "us-east-1"
        servers = [
          { name = "aws-web-01", plan = "t3.medium" }
        ]
      }
      {
        id = "upcloud-cluster"
        provider = "upcloud"
        region = "de-fra1"
        servers = [
          { name = "upcloud-web-01", plan = "medium" }
        ]
        dependencies = ["aws-cluster"]
      }
    ]
    parallel_limit = 2
  }
}
```

## Validation Workflow

### Type-Check Schema

```bash
# Check syntax and types
nickel typecheck infra/my-cluster.ncl

# Export to JSON (validates during export)
nickel export infra/my-cluster.ncl

# Export to TOML (generated output only)
nickel export --format toml infra/my-cluster.ncl > config.toml
```

### Platform Validation

```bash
# Validate against platform contracts
provisioning validate config --infra my-cluster

# Verbose validation
provisioning validate config --verbose
```

## IDE Integration

### Language Server (nickel-lang-lsp)

Install LSP for IDE support:

```bash
# Install LSP server
cargo install nickel-lang-lsp

# Configure your editor (VS Code example)
# Install "Nickel" extension from marketplace
```

Features:

- Syntax highlighting
- Type checking on save
- Autocomplete
- Hover documentation
- Go to definition

### VS Code Configuration

```json
{
  "nickel.lsp.command": "nickel-lang-lsp",
  "nickel.lsp.args": ["--stdio"],
  "nickel.format.onSave": true
}
```

## Common Patterns

### Environment-Specific Configuration

```nickel
let env_configs = {
  development = {
    plan = "small"
    backup_enabled = false
  }
  production = {
    plan = "large"
    backup_enabled = true
  }
} in

let environment = 'production in

{
  servers = [
    env_configs.%{std.string.from_enum environment} & {
      name = "server-01"
    }
  ]
}
```

### Configuration Composition

```nickel
let base_server = {
  zone = "de-fra1"
  backup_enabled = false
} in

let prod_overrides = {
  backup_enabled = true
  disk_size_gb = 100
} in

{
  servers = [
    base_server & { name = "dev-01" }
    base_server & prod_overrides & { name = "prod-01" }
  ]
}
```

## Migration from TOML

TOML is ONLY for generated output. Source is always Nickel.

```bash
# Generate TOML from Nickel (if needed for external tools)
nickel export --format toml infra/cluster.ncl > cluster.toml

# NEVER edit cluster.toml directly - edit cluster.ncl instead
```

## Best Practices

1. **Use Three-File Pattern**: Separate contracts, defaults, and main entry
2. **Type Everything**: Add type annotations for all fields
3. **Validate Early**: Run `nickel typecheck` before deployment
4. **Use Makers**: Leverage maker functions for composition
5. **Document Contracts**: Add comments explaining schema requirements
6. **Avoid Duplication**: Use record merging and defaults
7. **Test Locally**: Export and verify before deploying
8. **Version Schemas**: Track schema changes in version control

## Debugging

### Type Errors

```bash
# Detailed type error messages
nickel typecheck --color always infra/cluster.ncl
```

### Schema Inspection

```bash
# Export to JSON for inspection
nickel export infra/cluster.ncl | jq '.'

# Check specific field
nickel export infra/cluster.ncl | jq '.metadata'
```

### Format Code

```bash
# Auto-format Nickel files
nickel fmt infra/cluster.ncl

# Check formatting without modifying
nickel fmt --check infra/cluster.ncl
```

## Next Steps

- [Schemas Reference](schemas-reference.md) - Platform schema organization
- [Configuration System](configuration-system.md) - Hierarchical configuration
- [Providers](providers.md) - Cloud provider schemas
- [Batch Workflows](batch-workflows.md) - Multi-cloud orchestration with Nickel