provisioning/docs/src/infrastructure/config-rendering-guide.md

# Configuration Rendering Guide

This guide covers the unified configuration rendering system in the CLI daemon that supports Nickel and Tera template engines. KCL support is deprecated.

## Overview

The CLI daemon (`cli-daemon`) provides a high-performance REST API for rendering configurations in multiple formats:

- **Nickel**: Functional configuration language with lazy evaluation and type safety (primary choice)
- **Tera**: Jinja2-compatible template engine (simple templating)
- **KCL**: Type-safe infrastructure configuration language (legacy - deprecated)

All renderers are accessible through a single unified API endpoint with intelligent caching to minimize latency.

## Quick Start

### Starting the Daemon

The daemon runs on port 9091 by default:

```bash
# Start in background
./target/release/cli-daemon &

# Check it's running
curl http://localhost:9091/health
```

### Simple Nickel Rendering

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "nickel",
    "content": "{ name = \"my-server\", cpu = 4, memory = 8192 }",
    "name": "server-config"
  }'
```

**Response**:

```json
{
  "rendered": "{ name = \"my-server\", cpu = 4, memory = 8192 }",
  "error": null,
  "language": "nickel",
  "execution_time_ms": 23
}
```

## REST API Reference

### POST /config/render

Render a configuration in any supported language.

**Request Headers**:

```plaintext
Content-Type: application/json
```

**Request Body**:

```json
{
  "language": "nickel|tera|kcl",
  "content": "...configuration content...",
  "context": {
    "key1": "value1",
    "key2": 123
  },
  "name": "optional-config-name"
}
```

**Parameters**:

| Parameter | Type | Required | Description |
| ----------- | ------ | ---------- | ------------- |
| `language` | string | Yes | One of: `nickel`, `tera`, `kcl` (deprecated) |
| `content` | string | Yes | The configuration or template content to render |
| `context` | object | No | Variables to pass to the configuration (JSON object) |
| `name` | string | No | Optional name for logging purposes |

**Response** (Success):

```json
{
  "rendered": "...rendered output...",
  "error": null,
  "language": "kcl",
  "execution_time_ms": 23
}
```

**Response** (Error):

```json
{
  "rendered": null,
  "error": "KCL evaluation failed: undefined variable 'name'",
  "language": "kcl",
  "execution_time_ms": 18
}
```

**Status Codes**:

- `200 OK` - Rendering completed (check `error` field in body for evaluation errors)
- `400 Bad Request` - Invalid request format
- `500 Internal Server Error` - Daemon error

### GET /config/stats

Get rendering statistics across all languages.

**Response**:

```json
{
  "total_renders": 156,
  "successful_renders": 154,
  "failed_renders": 2,
  "average_time_ms": 28,
  "kcl_renders": 78,
  "nickel_renders": 52,
  "tera_renders": 26,
  "kcl_cache_hits": 68,
  "nickel_cache_hits": 35,
  "tera_cache_hits": 18
}
```

### POST /config/stats/reset

Reset all rendering statistics.

**Response**:

```json
{
  "status": "success",
  "message": "Configuration rendering statistics reset"
}
```

## KCL Rendering (Deprecated)

**Note**: KCL is deprecated. Use Nickel for new configurations.

### Basic KCL Configuration

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "kcl",
    "content": "
name = \"production-server\"
type = \"web\"
cpu = 4
memory = 8192
disk = 50

tags = {
    environment = \"production\"
    team = \"platform\"
}
",
    "name": "prod-server-config"
  }'
```

### KCL with Context Variables

Pass context variables using the `-D` flag syntax internally:

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "kcl",
    "content": "
name = option(\"server_name\", default=\"default-server\")
environment = option(\"env\", default=\"dev\")
cpu = option(\"cpu_count\", default=2)
memory = option(\"memory_mb\", default=2048)
",
    "context": {
      "server_name": "app-server-01",
      "env": "production",
      "cpu_count": 8,
      "memory_mb": 16384
    },
    "name": "server-with-context"
  }'
```

### Expected KCL Rendering Time

- **First render (cache miss)**: 20-50 ms
- **Cached render (same content)**: 1-5 ms
- **Large configs (100+ variables)**: 50-100 ms

## Nickel Rendering

### Basic Nickel Configuration

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "nickel",
    "content": "{
  name = \"production-server\",
  type = \"web\",
  cpu = 4,
  memory = 8192,
  disk = 50,
  tags = {
    environment = \"production\",
    team = \"platform\"
  }
}",
    "name": "nickel-server-config"
  }'
```

### Nickel with Lazy Evaluation

Nickel excels at evaluating only what's needed:

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "nickel",
    "content": "{
  server = {
    name = \"db-01\",
    # Expensive computation - only computed if accessed
    health_check = std.array.fold
      (fun acc x => acc + x)
      0
      [1, 2, 3, 4, 5]
  },
  networking = {
    dns_servers = [\"8.8.8.8\", \"8.8.4.4\"],
    firewall_rules = [\"allow_ssh\", \"allow_https\"]
  }
}",
    "context": {
      "only_server": true
    }
  }'
```

### Expected Nickel Rendering Time

- **First render (cache miss)**: 30-60 ms
- **Cached render (same content)**: 1-5 ms
- **Large configs with lazy evaluation**: 40-80 ms

**Advantage**: Nickel only computes fields that are actually used in the output

## Tera Template Rendering

### Basic Tera Template

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "tera",
    "content": "
Server Configuration
====================

Name: {{ server_name }}
Environment: {{ environment | default(value=\"development\") }}
Type: {{ server_type }}

Assigned Tasks:
{% for task in tasks %}
  - {{ task }}
{% endfor %}

{% if enable_monitoring %}
Monitoring: ENABLED
  - Prometheus: true
  - Grafana: true
{% else %}
Monitoring: DISABLED
{% endif %}
",
    "context": {
      "server_name": "prod-web-01",
      "environment": "production",
      "server_type": "web",
      "tasks": ["kubernetes", "prometheus", "cilium"],
      "enable_monitoring": true
    },
    "name": "server-template"
  }'
```

### Tera Filters and Functions

Tera supports Jinja2-compatible filters and functions:

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "tera",
    "content": "
Configuration for {{ environment | upper }}
Servers: {{ server_count | default(value=1) }}
Cost estimate: \${{ monthly_cost | round(precision=2) }}

{% for server in servers | reverse %}
- {{ server.name }}: {{ server.cpu }} CPUs
{% endfor %}
",
    "context": {
      "environment": "production",
      "server_count": 5,
      "monthly_cost": 1234.567,
      "servers": [
        {"name": "web-01", "cpu": 4},
        {"name": "db-01", "cpu": 8},
        {"name": "cache-01", "cpu": 2}
      ]
    }
  }'
```

### Expected Tera Rendering Time

- **Simple templates**: 4-10 ms
- **Complex templates with loops**: 10-20 ms
- **Always fast** (template is pre-compiled)

## Performance Characteristics

### Caching Strategy

All three renderers use LRU (Least Recently Used) caching:

- **Cache Size**: 100 entries per renderer
- **Cache Key**: SHA256 hash of (content + context)
- **Cache Hit**: Typically < 5 ms
- **Cache Miss**: Language-dependent (20-60 ms)

**To maximize cache hits**:

1. Render the same config multiple times → hits after first render
2. Use static content when possible → better cache reuse
3. Monitor cache hit ratio via `/config/stats`

### Benchmarks

Comparison of rendering times (on commodity hardware):

| Scenario | KCL | Nickel | Tera |
| ---------- | ----- | -------- | ------ |
| Simple config (10 vars) | 20 ms | 30 ms | 5 ms |
| Medium config (50 vars) | 35 ms | 45 ms | 8 ms |
| Large config (100+ vars) | 50-100 ms | 50-80 ms | 10 ms |
| Cached render | 1-5 ms | 1-5 ms | 1-5 ms |

### Memory Usage

- Each renderer keeps 100 cached entries in memory
- Average config size in cache: ~5 KB
- Maximum memory per renderer: ~500 KB + overhead

## Error Handling

### Common Errors

#### KCL Binary Not Found

**Error Response**:

```json
{
  "rendered": null,
  "error": "KCL binary not found in PATH. Install KCL or set KCL_PATH environment variable",
  "language": "kcl",
  "execution_time_ms": 0
}
```

**Solution**:

```bash
# Install KCL
kcl version

# Or set explicit path
export KCL_PATH=/usr/local/bin/kcl
```

#### Invalid KCL Syntax

**Error Response**:

```json
{
  "rendered": null,
  "error": "KCL evaluation failed: Parse error at line 3: expected '='",
  "language": "kcl",
  "execution_time_ms": 12
}
```

**Solution**: Verify Nickel syntax. Run `nickel eval file.ncl` directly for better error messages.

#### Missing Context Variable

**Error Response**:

```json
{
  "rendered": null,
  "error": "KCL evaluation failed: undefined variable 'required_var'",
  "language": "kcl",
  "execution_time_ms": 8
}
```

**Solution**: Provide required context variables or use `option()` with defaults.

#### Invalid JSON in Context

**HTTP Status**: `400 Bad Request`
**Body**: Error message about invalid JSON

**Solution**: Ensure context is valid JSON.

## Integration Examples

### Using with Nushell

```nushell
# Render a Nickel config from Nushell
let config = open workspace/config/provisioning.ncl | into string
let response = curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d $"{{ language: \"nickel\", content: $config }}" | from json

print $response.rendered
```

### Using with Python

```python
import requests
import json

def render_config(language, content, context=None, name=None):
    payload = {
        "language": language,
        "content": content,
        "context": context or {},
        "name": name
    }

    response = requests.post(
        "http://localhost:9091/config/render",
        json=payload
    )

    return response.json()

# Example usage
result = render_config(
    "nickel",
    '{name = "server", cpu = 4}',
    {"name": "prod-server"},
    "my-config"
)

if result["error"]:
    print(f"Error: {result['error']}")
else:
    print(f"Rendered in {result['execution_time_ms']}ms")
    print(result["rendered"])
```

### Using with Curl

```bash
#!/bin/bash

# Function to render config
render_config() {
    local language=$1
    local content=$2
    local name=${3:-"unnamed"}

    curl -X POST http://localhost:9091/config/render \
        -H "Content-Type: application/json" \
        -d @- << EOF
{
  "language": "$language",
  "content": $(echo "$content" | jq -Rs .),
  "name": "$name"
}
EOF
}

# Usage
render_config "nickel" "{name = \"my-server\"}"  "server-config"
```

## Troubleshooting

### Daemon Won't Start

**Check log level**:

```bash
PROVISIONING_LOG_LEVEL=debug ./target/release/cli-daemon
```

**Verify Nushell binary**:

```bash
which nu
# or set explicit path
NUSHELL_PATH=/usr/local/bin/nu ./target/release/cli-daemon
```

### Very Slow Rendering

**Check cache hit rate**:

```bash
curl http://localhost:9091/config/stats | jq '.nickel_cache_hits / .nickel_renders'
```

**If low cache hit rate**: Rendering same configs repeatedly?

**Monitor execution time**:

```bash
curl http://localhost:9091/config/render ... | jq '.execution_time_ms'
```

### Rendering Hangs

**Set timeout** (depends on client):

```bash
curl --max-time 10 -X POST http://localhost:9091/config/render ...
```

**Check daemon logs** for stuck processes.

### Out of Memory

**Reduce cache size** (rebuild with modified config) or restart daemon.

## Best Practices

1. **Choose right language for task**:
   - KCL: Familiar, type-safe, use if already in ecosystem
   - Nickel: Large configs with lazy evaluation needs
   - Tera: Simple templating, fastest

2. **Use context variables** instead of hardcoding values:

   ```json
   "context": {
     "environment": "production",
     "replica_count": 3
   }
   ```

1. **Monitor statistics** to understand performance:

   ```bash
   watch -n 1 'curl -s http://localhost:9091/config/stats | jq'
   ```

2. **Cache warming**: Pre-render common configs on startup

3. **Error handling**: Always check `error` field in response

## See Also

- [KCL Documentation](https://www.kcl-lang.io/docs/user_docs/getting-started/intro)
- [Nickel User Manual](https://nickel-lang.org/user-manual/introduction/)
- [Tera Template Engine](https://keats.github.io/tera/)
- CLI Daemon Architecture: `provisioning/platform/cli-daemon/README.md`

---

## Quick Reference

### API Endpoint

```plaintext
POST http://localhost:9091/config/render
```

### Request Template

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "kcl|nickel|tera",
    "content": "...",
    "context": {...},
    "name": "optional-name"
  }'
```

### Quick Examples

#### KCL - Simple Config

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "kcl",
    "content": "name = \"server\"\ncpu = 4\nmemory = 8192"
  }'
```

#### KCL - With Context

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "kcl",
    "content": "name = option(\"server_name\")\nenvironment = option(\"env\", default=\"dev\")",
    "context": {"server_name": "prod-01", "env": "production"}
  }'
```

#### Nickel - Simple Config

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "nickel",
    "content": "{name = \"server\", cpu = 4, memory = 8192}"
  }'
```

#### Tera - Template with Loops

```bash
curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d '{
    "language": "tera",
    "content": "{% for task in tasks %}{{ task }}\n{% endfor %}",
    "context": {"tasks": ["kubernetes", "postgres", "redis"]}
  }'
```

### Statistics

```bash
# Get stats
curl http://localhost:9091/config/stats

# Reset stats
curl -X POST http://localhost:9091/config/stats/reset

# Watch stats in real-time
watch -n 1 'curl -s http://localhost:9091/config/stats | jq'
```

### Performance Guide

| Language | Cold | Cached | Use Case |
| ---------- | ------ | -------- | ---------- |
| **KCL** | 20-50 ms | 1-5 ms | Type-safe infrastructure configs |
| **Nickel** | 30-60 ms | 1-5 ms | Large configs, lazy evaluation |
| **Tera** | 5-20 ms | 1-5 ms | Simple templating |

### Status Codes

| Code | Meaning |
| ------ | --------- |
| 200 | Success (check `error` field for evaluation errors) |
| 400 | Invalid request |
| 500 | Daemon error |

### Response Fields

```json
{
  "rendered": "...output or null on error",
  "error": "...error message or null on success",
  "language": "kcl|nickel|tera",
  "execution_time_ms": 23
}
```

### Languages Comparison

#### KCL

```kcl
name = "server"
type = "web"
cpu = 4
memory = 8192

tags = {
    env = "prod"
    team = "platform"
}
```

**Pros**: Familiar syntax, type-safe, existing patterns
**Cons**: Eager evaluation, verbose for simple cases

#### Nickel

```nickel
{
  name = "server",
  type = "web",
  cpu = 4,
  memory = 8192,
  tags = {
    env = "prod",
    team = "platform"
  }
}
```

**Pros**: Lazy evaluation, functional style, compact
**Cons**: Different paradigm, smaller ecosystem

#### Tera

```jinja2
Server: {{ name }}
Type: {{ type | upper }}
{% for tag_name, tag_value in tags %}
- {{ tag_name }}: {{ tag_value }}
{% endfor %}
```

**Pros**: Fast, simple, familiar template syntax
**Cons**: No validation, template-only

### Caching

**How it works**: SHA256(content + context) → cached result

**Cache hit**: < 5 ms
**Cache miss**: 20-60 ms (language dependent)
**Cache size**: 100 entries per language

**Cache stats**:

```bash
curl -s http://localhost:9091/config/stats | jq '{
  kcl_cache_hits: .kcl_cache_hits,
  kcl_renders: .kcl_renders,
  kcl_hit_ratio: (.kcl_cache_hits / .kcl_renders * 100)
}'
```

### Common Tasks

#### Batch Rendering

```bash
#!/bin/bash
for config in configs/*.ncl; do
  curl -X POST http://localhost:9091/config/render \
    -H "Content-Type: application/json" \
    -d "$(jq -n --arg content \"$(cat $config)\" \
      '{language: "nickel", content: $content}')"
done
```

#### Validate Before Rendering

```bash
# Nickel validation
nickel typecheck my-config.ncl

# Daemon validation (via first render)
curl ... # catches errors in response
```

#### Monitor Cache Performance

```bash
#!/bin/bash
while true; do
  STATS=$(curl -s http://localhost:9091/config/stats)
  HIT_RATIO=$( echo "$STATS" | jq '.nickel_cache_hits / .nickel_renders * 100')
  echo "Cache hit ratio: ${HIT_RATIO}%"
  sleep 5
done
```

### Error Examples

#### Missing Binary

```json
{
  "error": "Nickel binary not found. Install Nickel or set NICKEL_PATH",
  "rendered": null
}
```

**Fix**: `export NICKEL_PATH=/path/to/nickel` or install Nickel

#### Syntax Error

```json
{
  "error": "Nickel type checking failed: Type mismatch at line 3",
  "rendered": null
}
```

**Fix**: Check Nickel syntax, run `nickel typecheck file.ncl` directly

#### Missing Variable

```json
{
  "error": "Nickel evaluation failed: undefined variable 'name'",
  "rendered": null
}
```

**Fix**: Provide in `context` or define as optional field with default

### Integration Quick Start

#### Nushell

```nushell
use lib_provisioning

let config = open server.ncl | into string
let result = (curl -X POST http://localhost:9091/config/render \
  -H "Content-Type: application/json" \
  -d {language: "nickel", content: $config} | from json)

if ($result.error != null) {
  error $result.error
} else {
  print $result.rendered
}
```

#### Python

```python
import requests

resp = requests.post("http://localhost:9091/config/render", json={
    "language": "nickel",
    "content": '{name = "server"}',
    "context": {}
})
result = resp.json()
print(result["rendered"] if not result["error"] else f"Error: {result['error']}")
```

#### Bash

```bash
render() {
  curl -s -X POST http://localhost:9091/config/render \
    -H "Content-Type: application/json" \
    -d "$1" | jq '.'
}

# Usage
render '{"language":"nickel","content":"{name = \"server\"}"}'
```

### Environment Variables

```bash
# Daemon configuration
PROVISIONING_LOG_LEVEL=debug        # Log level
DAEMON_BIND=127.0.0.1:9091         # Bind address
NUSHELL_PATH=/usr/local/bin/nu      # Nushell binary
NICKEL_PATH=/usr/local/bin/nickel   # Nickel binary
```

### Useful Commands

```bash
# Health check
curl http://localhost:9091/health

# Daemon info
curl http://localhost:9091/info

# View stats
curl http://localhost:9091/config/stats | jq '.'

# Pretty print stats
curl -s http://localhost:9091/config/stats | jq '{
  total: .total_renders,
  success_rate: (.successful_renders / .total_renders * 100),
  avg_time: .average_time_ms,
  cache_hit_rate: ((.nickel_cache_hits + .tera_cache_hits) / (.nickel_renders + .tera_renders) * 100)
}'
```

### Troubleshooting Checklist

- [ ] Daemon running? `curl http://localhost:9091/health`
- [ ] Correct content for language?
- [ ] Valid JSON in context?
- [ ] Binary available? (KCL/Nickel)
- [ ] Check log level? `PROVISIONING_LOG_LEVEL=debug`
- [ ] Cache hit rate? `/config/stats`
- [ ] Error in response? Check `error` field