provisioning/schemas/platform/README.md

TypeDialog + Nickel Configuration System for Platform Services

Complete configuration system for provisioning platform services (orchestrator, control-center, mcp-server, vault-service, extension-registry, rag, ai-service, provisioning-daemon) across multiple deployment modes (solo, multiuser, cicd, enterprise).

Architecture Overview

This system implements a TypeDialog + Nickel configuration workflow that provides:

• Type-safe configuration via Nickel schemas with validation
• Interactive configuration via TypeDialog forms with real-time constraint validation
• Multi-mode deployment (solo/multiuser/cicd/enterprise) with mode-specific defaults
• Configuration composition (base defaults + mode overlays + user customization + validation)
• Automated TOML export for Rust service consumption
• Docker Compose + Kubernetes templates for infrastructure deployment

Directory Structure

provisioning/.typedialog/provisioning/platform/
├── constraints/                    # Single source of truth for validation limits
├── schemas/                        # Nickel type contracts (services + common + deployment modes)
├── defaults/                       # Default configuration values (services + common + deployment modes)
├── validators/                     # Validation logic (constraints, ranges, business rules)
├── configs/                        # Generated mode-specific Nickel configurations (4 services × 4 modes = 16 configs)
├── forms/                          # TypeDialog form definitions (4 main forms + flat fragments)
│   └── fragments/                  # Reusable form fragments (workspace, server, database, etc.)
├── templates/                      # Jinja2 + Nickel templates for config/deployment generation
│   ├── docker-compose/             # Docker Compose templates (solo/multiuser/cicd/enterprise)
│   ├── kubernetes/                 # Kubernetes deployment templates
│   └── configs/                    # Service configuration templates (TOML generation)
├── scripts/                        # Nushell orchestration scripts (configure, generate, validate, deploy)
├── examples/                       # Example configurations for different deployment scenarios
└── values/                         # User configuration files (gitignored *.ncl)

Configuration Workflow

1. User Interaction (TypeDialog)

nu scripts/configure.nu orchestrator solo --backend web

• Launches interactive form (web/tui/cli)
• Loads existing config as default values (if exists)
• Validates user input against constraints
• Generates updated Nickel config

2. Configuration Composition

Base Defaults (defaults/*.ncl)
  ↓
+ Mode Overlay (defaults/deployment/{mode}-defaults.ncl)
  ↓
+ User Customization (values/{service}.{mode}.ncl)
  ↓
+ Schema Validation (schemas/*.ncl)
  ↓
+ Constraint Validation (validators/*.ncl)
  ↓
= Final Configuration (configs/{service}.{mode}.ncl)

3. TOML Export

nu scripts/generate-configs.nu orchestrator solo

Exports Nickel config to TOML:

• provisioning/platform/config/orchestrator.solo.toml (consumed by Rust services)

Deployment Modes

Solo (2 CPU, 4GB RAM)

• Single developer/testing
• Filesystem or embedded database
• Minimal security
• All services enabled

MultiUser (4 CPU, 8GB RAM)

• Team collaboration, staging
• PostgreSQL or SurrealDB server
• RBAC enabled
• Gitea integration

CI/CD (8 CPU, 16GB RAM)

• Automated pipelines, ephemeral
• API-driven configuration
• Fast cleanup, minimal storage

Enterprise (16+ CPU, 32+ GB RAM)

• Production high availability
• SurrealDB cluster with replication
• MFA required, KMS integration
• Compliance (SOC2/HIPAA)

Key Components

Constraints (constraints/constraints.toml)

Single source of truth for validation limits across all services. Used for:

• Form field validation (min/max values)
• Constraint interpolation in TypeDialog forms
• Nickel validator bounds checking

Schemas (schemas/*.ncl)

Type-safe configuration contracts defining:

• Required/optional fields
• Valid value types and enums
• Default values
• Input/output type signatures

Organization:

• schemas/common/ - HTTP server, database, security, monitoring, logging
• schemas/{orchestrator,control-center,mcp-server,vault-service,extension-registry,rag,ai-service,provisioning-daemon}.ncl - Service-specific schemas
• schemas/deployment/{solo,multiuser,cicd,enterprise}.ncl - Mode-specific schemas

Defaults (defaults/*.ncl)

Configuration base values composed with mode overlays:

• defaults/{service}-defaults.ncl - Service base defaults
• defaults/common/ - Shared defaults (server, database, security)
• defaults/deployment/{mode}-defaults.ncl - Mode-specific value overrides

Validators (validators/*.ncl)

Business logic validation using constraints:

• Port range validation (1024-65535)
• Resource allocation validation (CPU, memory)
• Workflow/policy validation (service-specific)
• Cross-field validation

Configurations (configs/*.ncl)

Generated mode-specific Nickel configs (NOT manually edited):

• orchestrator.{solo,multiuser,cicd,enterprise}.ncl
• control-center.{solo,multiuser,cicd,enterprise}.ncl
• mcp-server.{solo,multiuser,cicd,enterprise}.ncl
• vault-service.{solo,multiuser,cicd,enterprise}.ncl
• extension-registry.{solo,multiuser,cicd,enterprise}.ncl
• rag.{solo,multiuser,cicd,enterprise}.ncl
• ai-service.{solo,multiuser,cicd,enterprise}.ncl
• provisioning-daemon.{solo,multiuser,cicd,enterprise}.ncl

Forms (forms/*.toml)

TypeDialog form definitions with flat fragments referenced by paths:

• 4 main forms: {service}-form.toml
• Fragments: fragments/{name}-section.toml (workspace, server, database, security, monitoring, etc.)
• CRITICAL: Every form element has nickel_path for Nickel structure mapping

Fragment Organization (FLAT, referenced by paths):

• workspace-section.toml
• server-section.toml
• database-rocksdb-section.toml
• database-surrealdb-section.toml
• database-postgres-section.toml
• security-section.toml
• monitoring-section.toml
• logging-section.toml
• orchestrator-queue-section.toml
• orchestrator-workflow-section.toml
• ... (service-specific and mode-specific fragments)

Templates (templates/)

Jinja2 + Nickel templates for automated generation:

• {service}-config.ncl.j2 - Nickel output template (critical for TypeDialog nickel-roundtrip)
• docker-compose/platform-stack.{mode}.yml.ncl - Docker Compose templates
• kubernetes/{service}-deployment.yaml.ncl - Kubernetes templates

Scripts (scripts/)

Nushell orchestration (NuShell 0.109+):

• configure.nu - Interactive TypeDialog wizard (nickel-roundtrip workflow)
• generate-configs.nu - Export Nickel → TOML
• validate-config.nu - Typecheck Nickel configs
• render-docker-compose.nu - Generate Docker Compose files
• render-kubernetes.nu - Generate Kubernetes manifests
• install-services.nu - Deploy platform services
• detect-services.nu - Auto-detect running services

Examples (examples/)

Reference configurations for different scenarios:

• orchestrator-solo.ncl - Simple development setup
• orchestrator-enterprise.ncl - Complex production setup
• full-platform-enterprise.ncl - Complete enterprise stack

Values (values/)

User configuration directory (gitignored):

• {service}.{mode}.ncl - User customizations (loaded in compose)
• .gitignore - Ignores *.ncl files
• orchestrator.example.ncl - Documented example template

TypeDialog nickel-roundtrip Workflow

CRITICAL: Forms use Jinja2 templates for Nickel generation:

# Command pattern
typedialog-web nickel-roundtrip "$CONFIG_FILE" "$FORM_FILE" --output "$CONFIG_FILE" --template "$NCL_TEMPLATE"

# Example
typedialog-web nickel-roundtrip 
    "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl" 
    "provisioning/.typedialog/provisioning/platform/forms/orchestrator-form.toml" 
    --output "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl" 
    --template "provisioning/.typedialog/provisioning/platform/templates/orchestrator-config.ncl.j2"

Key Requirements:

1. Jinja2 template (config.ncl.j2) - Defines Nickel output structure with conditional {% if %} blocks
2. nickel_path in form elements - Maps form fields to Nickel structure paths (e.g., ["orchestrator", "queue", "max_concurrent_tasks"])
3. Constraint interpolation - Form limits reference constraints (e.g., ${constraint.orchestrator.queue.concurrent_tasks.max})
4. Base + overlay composition - Nickel imports merge defaults + mode overlays + validators

Usage Workflow

1. Configure Service (Interactive)

# Start TypeDialog wizard for orchestrator in solo mode
nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo --backend web

Wizard:

1. Loads existing config (if exists) as defaults
2. Shows form with validated constraints
3. User edits configuration
4. Generates updated Nickel config to provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl

2. Validate Configuration

# Typecheck Nickel config
nu provisioning/.typedialog/provisioning/platform/scripts/validate-config.nu provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl

3. Generate TOML for Rust Services

# Export Nickel → TOML
nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo

Output: provisioning/platform/config/orchestrator.solo.toml

4. Deploy Services

# Install services (Docker Compose or Kubernetes)
nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo

Configuration Loading Hierarchy (Rust Services)

1. Environment variables (ORCHESTRATOR_*)
2. User config (values/{service}.{mode}.ncl → TOML)
3. Mode-specific defaults (configs/{service}.{mode}.toml)
4. Service defaults (config/orchestrator.defaults.toml)

Constraint Interpolation Example

constraints.toml:

[orchestrator.queue.concurrent_tasks]
min = 1
max = 100

Form element (fragments/orchestrator-queue-section.toml):

[[elements]]
name = "max_concurrent_tasks"
type = "number"
min = "${constraint.orchestrator.queue.concurrent_tasks.min}"
max = "${constraint.orchestrator.queue.concurrent_tasks.max}"
nickel_path = ["orchestrator", "queue", "max_concurrent_tasks"]

Jinja2 template (orchestrator-config.ncl.j2):

orchestrator = {
  queue = {
    {%- if max_concurrent_tasks %}
    max_concurrent_tasks = {{ max_concurrent_tasks }},
    {%- endif %}
  },
}

Getting Started

1. Run configuration wizard:

   nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo
   

2. Generate TOML configs:

   nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo
   

3. Deploy services:

   nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo
   

Documentation

• constraints/README.md - How to modify validation constraints
• schemas/README.md - Schema patterns and imports
• defaults/README.md - Defaults composition and merging strategy
• validators/README.md - Validator patterns and error handling
• forms/README.md - Form structure and fragment organization
• forms/fragments/README.md - Fragment usage and nickel_path mapping
• scripts/README.md - Script usage and dependencies
• examples/README.md - Example deployment scenarios
• templates/README.md - Template patterns and interpolation

Key Files

File	Purpose
constraints/constraints.toml	Single source of truth for validation limits
schemas/orchestrator.ncl	Orchestrator type schema
defaults/orchestrator-defaults.ncl	Orchestrator default values
validators/orchestrator-validator.ncl	Orchestrator validation logic
configs/orchestrator.solo.ncl	Generated solo mode config
forms/orchestrator-form.toml	Orchestrator form definition
templates/orchestrator-config.ncl.j2	Nickel output template
scripts/configure.nu	Interactive configuration wizard
scripts/generate-configs.nu	Nickel → TOML export
values/orchestrator.solo.ncl	User configuration (gitignored)

Tools Required

• Nickel (0.10+) - Configuration language
• TypeDialog - Interactive form backend
• NuShell (0.109+) - Script orchestration
• Jinja2/tera - Template rendering (via nu_plugin_tera)
• TOML - Config file format (for Rust services)

Notes

• Configuration files in values/ are gitignored (user-specific)
• Generated configs in configs/ are composed automatically (not hand-edited)
• Each mode (solo/multiuser/cicd/enterprise) has different resource defaults
• Fragments are flat in forms/fragments/ and referenced by paths in form definitions
• All form elements must have nickel_path for proper Nickel structure mapping
• Constraint interpolation enables dynamic form validation based on service requirements

---

Version: 1.0.0 Created: 2025-01-05 Last Updated: 2025-01-05