# TypeDialog + Nickel Configuration System for Platform Services

Complete configuration system for provisioning platform services (orchestrator, control-center, mcp-server, vault-service,
extension-registry, rag, ai-service, provisioning-daemon) across multiple deployment modes (solo, multiuser, cicd, enterprise).

## Architecture Overview

This system implements a **TypeDialog + Nickel configuration workflow** that provides:

- **Type-safe configuration** via Nickel schemas with validation
- **Interactive configuration** via TypeDialog forms with real-time constraint validation
- **Multi-mode deployment** (solo/multiuser/cicd/enterprise) with mode-specific defaults
- **Configuration composition** (base defaults + mode overlays + user customization + validation)
- **Automated TOML export** for Rust service consumption
- **Docker Compose + Kubernetes templates** for infrastructure deployment

## Directory Structure

```bash
provisioning/.typedialog/provisioning/platform/
├── constraints/                    # Single source of truth for validation limits
├── schemas/                        # Nickel type contracts (services + common + deployment modes)
├── defaults/                       # Default configuration values (services + common + deployment modes)
├── validators/                     # Validation logic (constraints, ranges, business rules)
├── configs/                        # Generated mode-specific Nickel configurations (4 services × 4 modes = 16 configs)
├── forms/                          # TypeDialog form definitions (4 main forms + flat fragments)
│   └── fragments/                  # Reusable form fragments (workspace, server, database, etc.)
├── templates/                      # Jinja2 + Nickel templates for config/deployment generation
│   ├── docker-compose/             # Docker Compose templates (solo/multiuser/cicd/enterprise)
│   ├── kubernetes/                 # Kubernetes deployment templates
│   └── configs/                    # Service configuration templates (TOML generation)
├── scripts/                        # Nushell orchestration scripts (configure, generate, validate, deploy)
├── examples/                       # Example configurations for different deployment scenarios
└── values/                         # User configuration files (gitignored *.ncl)
```

## Configuration Workflow

### 1. User Interaction (TypeDialog)

```nushell
nu scripts/configure.nu orchestrator solo --backend web
```

- Launches interactive form (web/tui/cli)
- Loads existing config as default values (if exists)
- Validates user input against constraints
- Generates updated Nickel config

### 2. Configuration Composition

```toml
Base Defaults (defaults/*.ncl)
  ↓
+ Mode Overlay (defaults/deployment/{mode}-defaults.ncl)
  ↓
+ User Customization (values/{service}.{mode}.ncl)
  ↓
+ Schema Validation (schemas/*.ncl)
  ↓
+ Constraint Validation (validators/*.ncl)
  ↓
= Final Configuration (configs/{service}.{mode}.ncl)
```

### 3. TOML Export

```toml
nu scripts/generate-configs.nu orchestrator solo
```

Exports Nickel config to TOML:
- `provisioning/platform/config/orchestrator.solo.toml` (consumed by Rust services)

## Deployment Modes

### Solo (2 CPU, 4GB RAM)
- Single developer/testing
- Filesystem or embedded database
- Minimal security
- All services enabled

### MultiUser (4 CPU, 8GB RAM)
- Team collaboration, staging
- PostgreSQL or SurrealDB server
- RBAC enabled
- Gitea integration

### CI/CD (8 CPU, 16GB RAM)
- Automated pipelines, ephemeral
- API-driven configuration
- Fast cleanup, minimal storage

### Enterprise (16+ CPU, 32+ GB RAM)
- Production high availability
- SurrealDB cluster with replication
- MFA required, KMS integration
- Compliance (SOC2/HIPAA)

## Key Components

### Constraints (constraints/constraints.toml)
Single source of truth for validation limits across all services. Used for:
- Form field validation (min/max values)
- Constraint interpolation in TypeDialog forms
- Nickel validator bounds checking

### Schemas (schemas/*.ncl)
Type-safe configuration contracts defining:
- Required/optional fields
- Valid value types and enums
- Default values
- Input/output type signatures

**Organization**:
- `schemas/common/` - HTTP server, database, security, monitoring, logging
- `schemas/{orchestrator,control-center,mcp-server,vault-service,extension-registry,rag,ai-service,provisioning-daemon}.ncl` - Service-specific schemas
- `schemas/deployment/{solo,multiuser,cicd,enterprise}.ncl` - Mode-specific schemas

### Defaults (defaults/*.ncl)
Configuration base values composed with mode overlays:
- `defaults/{service}-defaults.ncl` - Service base defaults
- `defaults/common/` - Shared defaults (server, database, security)
- `defaults/deployment/{mode}-defaults.ncl` - Mode-specific value overrides

### Validators (validators/*.ncl)
Business logic validation using constraints:
- Port range validation (1024-65535)
- Resource allocation validation (CPU, memory)
- Workflow/policy validation (service-specific)
- Cross-field validation

### Configurations (configs/*.ncl)
Generated mode-specific Nickel configs (NOT manually edited):
- `orchestrator.{solo,multiuser,cicd,enterprise}.ncl`
- `control-center.{solo,multiuser,cicd,enterprise}.ncl`
- `mcp-server.{solo,multiuser,cicd,enterprise}.ncl`
- `vault-service.{solo,multiuser,cicd,enterprise}.ncl`
- `extension-registry.{solo,multiuser,cicd,enterprise}.ncl`
- `rag.{solo,multiuser,cicd,enterprise}.ncl`
- `ai-service.{solo,multiuser,cicd,enterprise}.ncl`
- `provisioning-daemon.{solo,multiuser,cicd,enterprise}.ncl`

### Forms (forms/*.toml)
TypeDialog form definitions with **flat fragments** referenced by paths:
- 4 main forms: `{service}-form.toml`
- Fragments: `fragments/{name}-section.toml` (workspace, server, database, security, monitoring, etc.)
- CRITICAL: Every form element has `nickel_path` for Nickel structure mapping

**Fragment Organization** (FLAT, referenced by paths):
- `workspace-section.toml`
- `server-section.toml`
- `database-rocksdb-section.toml`
- `database-surrealdb-section.toml`
- `database-postgres-section.toml`
- `security-section.toml`
- `monitoring-section.toml`
- `logging-section.toml`
- `orchestrator-queue-section.toml`
- `orchestrator-workflow-section.toml`
- ... (service-specific and mode-specific fragments)

### Templates (templates/)
Jinja2 + Nickel templates for automated generation:
- `{service}-config.ncl.j2` - Nickel output template (critical for TypeDialog nickel-roundtrip)
- `docker-compose/platform-stack.{mode}.yml.ncl` - Docker Compose templates
- `kubernetes/{service}-deployment.yaml.ncl` - Kubernetes templates

### Scripts (scripts/)
Nushell orchestration (NuShell 0.109+):
- `configure.nu` - Interactive TypeDialog wizard (nickel-roundtrip workflow)
- `generate-configs.nu` - Export Nickel → TOML
- `validate-config.nu` - Typecheck Nickel configs
- `render-docker-compose.nu` - Generate Docker Compose files
- `render-kubernetes.nu` - Generate Kubernetes manifests
- `install-services.nu` - Deploy platform services
- `detect-services.nu` - Auto-detect running services

### Examples (examples/)
Reference configurations for different scenarios:
- `orchestrator-solo.ncl` - Simple development setup
- `orchestrator-enterprise.ncl` - Complex production setup
- `full-platform-enterprise.ncl` - Complete enterprise stack

### Values (values/)
User configuration directory (gitignored):
- `{service}.{mode}.ncl` - User customizations (loaded in compose)
- `.gitignore` - Ignores `*.ncl` files
- `orchestrator.example.ncl` - Documented example template

## TypeDialog nickel-roundtrip Workflow

CRITICAL: Forms use Jinja2 templates for Nickel generation:

```nickel
# Command pattern
typedialog-web nickel-roundtrip "$CONFIG_FILE" "$FORM_FILE" --output "$CONFIG_FILE" --template "$NCL_TEMPLATE"

# Example
typedialog-web nickel-roundtrip 
    "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl" 
    "provisioning/.typedialog/provisioning/platform/forms/orchestrator-form.toml" 
    --output "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl" 
    --template "provisioning/.typedialog/provisioning/platform/templates/orchestrator-config.ncl.j2"
```

**Key Requirements**:
1. **Jinja2 template** (`config.ncl.j2`) - Defines Nickel output structure with conditional `{% if %}` blocks
2. **nickel_path** in form elements - Maps form fields to Nickel structure paths (e.g., `["orchestrator", "queue", "max_concurrent_tasks"]`)
3. **Constraint interpolation** - Form limits reference constraints (e.g., `${constraint.orchestrator.queue.concurrent_tasks.max}`)
4. **Base + overlay composition** - Nickel imports merge defaults + mode overlays + validators

## Usage Workflow

### 1. Configure Service (Interactive)

```toml
# Start TypeDialog wizard for orchestrator in solo mode
nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo --backend web
```

Wizard:
1. Loads existing config (if exists) as defaults
2. Shows form with validated constraints
3. User edits configuration
4. Generates updated Nickel config to `provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl`

### 2. Validate Configuration

```toml
# Typecheck Nickel config
nu provisioning/.typedialog/provisioning/platform/scripts/validate-config.nu provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl
```

### 3. Generate TOML for Rust Services

```toml
# Export Nickel → TOML
nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo
```

Output: `provisioning/platform/config/orchestrator.solo.toml`

### 4. Deploy Services

```bash
# Install services (Docker Compose or Kubernetes)
nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo
```

## Configuration Loading Hierarchy (Rust Services)

```rust
1. Environment variables (ORCHESTRATOR_*)
2. User config (values/{service}.{mode}.ncl → TOML)
3. Mode-specific defaults (configs/{service}.{mode}.toml)
4. Service defaults (config/orchestrator.defaults.toml)
```

## Constraint Interpolation Example

**constraints.toml**:

```toml
[orchestrator.queue.concurrent_tasks]
min = 1
max = 100
```

**Form element** (fragments/orchestrator-queue-section.toml):

```toml
[[elements]]
name = "max_concurrent_tasks"
type = "number"
min = "${constraint.orchestrator.queue.concurrent_tasks.min}"
max = "${constraint.orchestrator.queue.concurrent_tasks.max}"
nickel_path = ["orchestrator", "queue", "max_concurrent_tasks"]
```

**Jinja2 template** (orchestrator-config.ncl.j2):

```nickel
orchestrator = {
  queue = {
    {%- if max_concurrent_tasks %}
    max_concurrent_tasks = {{ max_concurrent_tasks }},
    {%- endif %}
  },
}
```

## Getting Started

1. **Run configuration wizard**:

   ```bash
   nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo
   ```

2. **Generate TOML configs**:

   ```bash
   nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo
   ```

3. **Deploy services**:

   ```bash
   nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo
   ```

## Documentation

- `constraints/README.md` - How to modify validation constraints
- `schemas/README.md` - Schema patterns and imports
- `defaults/README.md` - Defaults composition and merging strategy
- `validators/README.md` - Validator patterns and error handling
- `forms/README.md` - Form structure and fragment organization
- `forms/fragments/README.md` - Fragment usage and nickel_path mapping
- `scripts/README.md` - Script usage and dependencies
- `examples/README.md` - Example deployment scenarios
- `templates/README.md` - Template patterns and interpolation

## Key Files

| File | Purpose |
| ------ | --------- |
| `constraints/constraints.toml` | Single source of truth for validation limits |
| `schemas/orchestrator.ncl` | Orchestrator type schema |
| `defaults/orchestrator-defaults.ncl` | Orchestrator default values |
| `validators/orchestrator-validator.ncl` | Orchestrator validation logic |
| `configs/orchestrator.solo.ncl` | Generated solo mode config |
| `forms/orchestrator-form.toml` | Orchestrator form definition |
| `templates/orchestrator-config.ncl.j2` | Nickel output template |
| `scripts/configure.nu` | Interactive configuration wizard |
| `scripts/generate-configs.nu` | Nickel → TOML export |
| `values/orchestrator.solo.ncl` | User configuration (gitignored) |

## Tools Required

- **Nickel** (0.10+) - Configuration language
- **TypeDialog** - Interactive form backend
- **NuShell** (0.109+) - Script orchestration
- **Jinja2/tera** - Template rendering (via nu_plugin_tera)
- **TOML** - Config file format (for Rust services)

## Notes

- Configuration files in `values/` are **gitignored** (user-specific)
- Generated configs in `configs/` are composed automatically (not hand-edited)
- Each mode (solo/multiuser/cicd/enterprise) has different resource defaults
- Fragments are **flat** in `forms/fragments/` and referenced by paths in form definitions
- All form elements must have `nickel_path` for proper Nickel structure mapping
- Constraint interpolation enables dynamic form validation based on service requirements

---

**Version**: 1.0.0
**Created**: 2025-01-05
**Last Updated**: 2025-01-05