provisioning/schemas/platform

Platform Configuration System - Schemas & Defaults

Source of truth for platform service configurations (orchestrator, control-center, mcp-server, vault-service, extension-registry, rag, ai-service, provisioning-daemon) across all deployment modes.

This directory contains the active, production configuration system that powers:

• Type-safe configuration via Nickel schemas
• Constraint-based validation
• Multi-mode deployment (solo/multiuser/cicd/enterprise)
• Interactive TypeDialog forms
• TOML export for Rust service consumption

Directory Structure (Flat)

provisioning/schemas/platform/
├── orchestrator.ncl                # Service schemas (flat, one file per service)
├── control-center.ncl
├── mcp-server.ncl
├── vault-service.ncl
├── extension-registry.ncl
├── ai-service.ncl
├── rag.ncl
├── provisioning-daemon.ncl
│
├── common/                         # Shared schemas and utilities
│   ├── constraints.ncl             # GENERATED - Nickel constraint validators (do NOT edit)
│   ├── helpers.ncl                 # Configuration composition helpers
│   ├── server.ncl                  # HTTP server schema
│   ├── database.ncl                # Database backend schema
│   ├── security.ncl                # Authentication/encryption
│   ├── monitoring.ncl              # Metrics and health checks
│   ├── logging.ncl                 # Log configuration
│   ├── network.ncl                 # Network binding and TLS
│   ├── storage.ncl                 # Storage backend
│   └── workspace.ncl               # Workspace configuration
│
├── deployment/                     # Mode-specific schemas (flat)
│   ├── solo.ncl                    # 2 CPU, 4GB RAM
│   ├── multiuser.ncl               # 4 CPU, 8GB RAM
│   ├── cicd.ncl                    # 8 CPU, 16GB RAM
│   └── enterprise.ncl              # 16+ CPU, 32+ GB RAM
│
├── defaults/                       # Default values by service and mode
│   ├── orchestrator-defaults.ncl
│   ├── control-center-defaults.ncl
│   ├── mcp-server-defaults.ncl
│   ├── vault-service-defaults.ncl
│   ├── extension-registry-defaults.ncl
│   ├── ai-service-defaults.ncl
│   ├── rag-defaults.ncl
│   ├── provisioning-daemon-defaults.ncl
│   ├── common/                     # Shared defaults (6 files)
│   │   ├── server-defaults.ncl
│   │   ├── database-defaults.ncl
│   │   ├── security-defaults.ncl
│   │   ├── monitoring-defaults.ncl
│   │   └── logging-defaults.ncl
│   └── deployment/                 # Mode defaults (4 files)
│       ├── solo-defaults.ncl
│       ├── multiuser-defaults.ncl
│       ├── cicd-defaults.ncl
│       └── enterprise-defaults.ncl
│
├── constraints/                    # Single source of truth
│   ├── constraints.toml            # MASTER FILE - All validation limits
│   └── README.md
│
├── configs/                        # Generated intermediate configs
│   ├── orchestrator.solo.ncl
│   ├── orchestrator.multiuser.ncl
│   ├── orchestrator.cicd.ncl
│   ├── orchestrator.enterprise.ncl
│   └── ... (8 services × 4 modes = 32 configs)
│
├── templates/                      # Output generation templates
│   ├── configs/                    # TOML export templates (Jinja2)
│   ├── docker-compose/             # Docker Compose templates
│   ├── kubernetes/                 # Kubernetes manifests
│   └── service-config-template.ncl
│
├── examples/                       # Reference configurations
│   ├── orchestrator-solo.ncl
│   ├── orchestrator-enterprise.ncl
│   ├── control-center-multiuser.ncl
│   └── full-platform-enterprise.ncl
│
└── values/                         # User customizations (gitignored)
    ├── .gitignore                  # Ignores *.ncl, *.toml
    └── (user configs populated at runtime)

Key Changes:

• ✅ Flat structure (no nested schemas/ or defaults/)
• ✅ Archived validators/ (88KB dead code)
• ✅ Generated constraints.ncl (gitignored)
• ✅ Master constraints.toml (single source)

Configuration Workflow

1. User Interaction (TypeDialog)

nu scripts/configure.nu orchestrator solo --backend web

• Launches interactive form (web/tui/cli)
• Loads existing config as default values (if exists)
• Validates user input against constraints
• Generates updated Nickel config

2. Configuration Composition

Base Defaults (defaults/*.ncl)
  ↓
+ Mode Overlay (defaults/deployment/{mode}-defaults.ncl)
  ↓
+ User Customization (values/{service}.{mode}.ncl)
  ↓
+ Schema Validation (schemas/*.ncl)
  ↓
+ Constraint Validation (validators/*.ncl)
  ↓
= Final Configuration (configs/{service}.{mode}.ncl)

3. TOML Export

nu scripts/generate-configs.nu orchestrator solo

Exports Nickel config to TOML:

• provisioning/platform/config/orchestrator.solo.toml (consumed by Rust services)

Deployment Modes

Solo (2 CPU, 4GB RAM)

• Single developer/testing
• Filesystem or embedded database
• Minimal security
• All services enabled

MultiUser (4 CPU, 8GB RAM)

• Team collaboration, staging
• PostgreSQL or SurrealDB server
• RBAC enabled
• Gitea integration

CI/CD (8 CPU, 16GB RAM)

• Automated pipelines, ephemeral
• API-driven configuration
• Fast cleanup, minimal storage

Enterprise (16+ CPU, 32+ GB RAM)

• Production high availability
• SurrealDB cluster with replication
• MFA required, KMS integration
• Compliance (SOC2/HIPAA)

Key Components

Constraints (constraints/constraints.toml)

Single source of truth for validation limits across all services. Used for:

• Form field validation (min/max values)
• Constraint interpolation in TypeDialog forms
• Nickel validator bounds checking

Schemas (schemas/*.ncl)

Type-safe configuration contracts defining:

• Required/optional fields
• Valid value types and enums
• Default values
• Input/output type signatures

Organization:

• schemas/common/ - HTTP server, database, security, monitoring, logging
• schemas/{orchestrator,control-center,mcp-server,vault-service,extension-registry,rag,ai-service,provisioning-daemon}.ncl - Service-specific schemas
• schemas/deployment/{solo,multiuser,cicd,enterprise}.ncl - Mode-specific schemas

Defaults (defaults/*.ncl)

Configuration base values composed with mode overlays:

• defaults/{service}-defaults.ncl - Service base defaults
• defaults/common/ - Shared defaults (server, database, security)
• defaults/deployment/{mode}-defaults.ncl - Mode-specific value overrides

Validators (validators/*.ncl)

Business logic validation using constraints:

• Port range validation (1024-65535)
• Resource allocation validation (CPU, memory)
• Workflow/policy validation (service-specific)
• Cross-field validation

Configurations (configs/*.ncl)

Generated mode-specific Nickel configs (NOT manually edited):

• orchestrator.{solo,multiuser,cicd,enterprise}.ncl
• control-center.{solo,multiuser,cicd,enterprise}.ncl
• mcp-server.{solo,multiuser,cicd,enterprise}.ncl
• vault-service.{solo,multiuser,cicd,enterprise}.ncl
• extension-registry.{solo,multiuser,cicd,enterprise}.ncl
• rag.{solo,multiuser,cicd,enterprise}.ncl
• ai-service.{solo,multiuser,cicd,enterprise}.ncl
• provisioning-daemon.{solo,multiuser,cicd,enterprise}.ncl

Forms (forms/*.toml)

TypeDialog form definitions with flat fragments referenced by paths:

• 4 main forms: {service}-form.toml
• Fragments: fragments/{name}-section.toml (workspace, server, database, security, monitoring, etc.)
• CRITICAL: Every form element has nickel_path for Nickel structure mapping

Fragment Organization (FLAT, referenced by paths):

• workspace-section.toml
• server-section.toml
• database-rocksdb-section.toml
• database-surrealdb-section.toml
• database-postgres-section.toml
• security-section.toml
• monitoring-section.toml
• logging-section.toml
• orchestrator-queue-section.toml
• orchestrator-workflow-section.toml
• ... (service-specific and mode-specific fragments)

Templates (templates/)

Jinja2 + Nickel templates for automated generation:

• {service}-config.ncl.j2 - Nickel output template (critical for TypeDialog nickel-roundtrip)
• docker-compose/platform-stack.{mode}.yml.ncl - Docker Compose templates
• kubernetes/{service}-deployment.yaml.ncl - Kubernetes templates

Scripts (scripts/)

Nushell orchestration (NuShell 0.109+):

• configure.nu - Interactive TypeDialog wizard (nickel-roundtrip workflow)
• generate-configs.nu - Export Nickel → TOML
• validate-config.nu - Typecheck Nickel configs
• render-docker-compose.nu - Generate Docker Compose files
• render-kubernetes.nu - Generate Kubernetes manifests
• install-services.nu - Deploy platform services
• detect-services.nu - Auto-detect running services

Examples (examples/)

Reference configurations for different scenarios:

• orchestrator-solo.ncl - Simple development setup
• orchestrator-enterprise.ncl - Complex production setup
• full-platform-enterprise.ncl - Complete enterprise stack

Values (values/)

User configuration directory (gitignored):

• {service}.{mode}.ncl - User customizations (loaded in compose)
• .gitignore - Ignores *.ncl files
• orchestrator.example.ncl - Documented example template

TypeDialog nickel-roundtrip Workflow

CRITICAL: Forms use Jinja2 templates for Nickel generation:

# Command pattern
typedialog-web nickel-roundtrip "$CONFIG_FILE" "$FORM_FILE" --output "$CONFIG_FILE" --template "$NCL_TEMPLATE"

# Example
typedialog-web nickel-roundtrip 
    "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl" 
    "provisioning/.typedialog/provisioning/platform/forms/orchestrator-form.toml" 
    --output "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl" 
    --template "provisioning/.typedialog/provisioning/platform/templates/orchestrator-config.ncl.j2"

Key Requirements:

1. Jinja2 template (config.ncl.j2) - Defines Nickel output structure with conditional {% if %} blocks
2. nickel_path in form elements - Maps form fields to Nickel structure paths (e.g., ["orchestrator", "queue", "max_concurrent_tasks"])
3. Constraint interpolation - Form limits reference constraints (e.g., ${constraint.orchestrator.queue.concurrent_tasks.max})
4. Base + overlay composition - Nickel imports merge defaults + mode overlays + validators

Usage Workflow

1. Configure Service (Interactive)

# Start TypeDialog wizard for orchestrator in solo mode
nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo --backend web

Wizard:

1. Loads existing config (if exists) as defaults
2. Shows form with validated constraints
3. User edits configuration
4. Generates updated Nickel config to provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl

2. Validate Configuration

# Typecheck Nickel config
nu provisioning/.typedialog/provisioning/platform/scripts/validate-config.nu provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl

3. Generate TOML for Rust Services

# Export Nickel → TOML
nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo

Output: provisioning/platform/config/orchestrator.solo.toml

4. Deploy Services

# Install services (Docker Compose or Kubernetes)
nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo

Configuration Loading Hierarchy (Rust Services)

1. Environment variables (ORCHESTRATOR_*)
2. User config (values/{service}.{mode}.ncl → TOML)
3. Mode-specific defaults (configs/{service}.{mode}.toml)
4. Service defaults (config/orchestrator.defaults.toml)

Constraint Interpolation Example

constraints.toml:

[orchestrator.queue.concurrent_tasks]
min = 1
max = 100

Form element (fragments/orchestrator-queue-section.toml):

[[elements]]
name = "max_concurrent_tasks"
type = "number"
min = "${constraint.orchestrator.queue.concurrent_tasks.min}"
max = "${constraint.orchestrator.queue.concurrent_tasks.max}"
nickel_path = ["orchestrator", "queue", "max_concurrent_tasks"]

Jinja2 template (orchestrator-config.ncl.j2):

orchestrator = {
  queue = {
    {%- if max_concurrent_tasks %}
    max_concurrent_tasks = {{ max_concurrent_tasks }},
    {%- endif %}
  },
}

Getting Started

1. Run configuration wizard:

   nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo
   

2. Generate TOML configs:

   nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo
   

3. Deploy services:

   nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo
   

Documentation

• constraints/README.md - How to modify validation constraints
• schemas/README.md - Schema patterns and imports
• defaults/README.md - Defaults composition and merging strategy
• validators/README.md - Validator patterns and error handling
• forms/README.md - Form structure and fragment organization
• forms/fragments/README.md - Fragment usage and nickel_path mapping
• scripts/README.md - Script usage and dependencies
• examples/README.md - Example deployment scenarios
• templates/README.md - Template patterns and interpolation

Key Files

File	Purpose
constraints/constraints.toml	Single source of truth for validation limits
schemas/orchestrator.ncl	Orchestrator type schema
defaults/orchestrator-defaults.ncl	Orchestrator default values
validators/orchestrator-validator.ncl	Orchestrator validation logic
configs/orchestrator.solo.ncl	Generated solo mode config
forms/orchestrator-form.toml	Orchestrator form definition
templates/orchestrator-config.ncl.j2	Nickel output template
scripts/configure.nu	Interactive configuration wizard
scripts/generate-configs.nu	Nickel → TOML export
values/orchestrator.solo.ncl	User configuration (gitignored)

Tools Required

• Nickel (0.10+) - Configuration language
• TypeDialog - Interactive form backend
• NuShell (0.109+) - Script orchestration
• Jinja2/tera - Template rendering (via nu_plugin_tera)
• TOML - Config file format (for Rust services)

Notes

• Configuration files in values/ are gitignored (user-specific)
• Generated configs in configs/ are composed automatically (not hand-edited)
• Each mode (solo/multiuser/cicd/enterprise) has different resource defaults
• Fragments are flat in forms/fragments/ and referenced by paths in form definitions
• All form elements must have nickel_path for proper Nickel structure mapping
• Constraint interpolation enables dynamic form validation based on service requirements

---

Version: 1.0.0 Created: 2025-01-05 Last Updated: 2025-01-05