2026-01-14 04:56:20 +00:00
# TypeDialog + Nickel Configuration System for Platform Services
Complete configuration system for provisioning platform services (orchestrator, control-center, mcp-server, vault-service,
extension-registry, rag, ai-service, provisioning-daemon) across multiple deployment modes (solo, multiuser, cicd, enterprise).
## Architecture Overview
This system implements a **TypeDialog + Nickel configuration workflow** that provides:
- **Type-safe configuration** via Nickel schemas with validation
- **Interactive configuration** via TypeDialog forms with real-time constraint validation
- **Multi-mode deployment** (solo/multiuser/cicd/enterprise) with mode-specific defaults
- **Configuration composition** (base defaults + mode overlays + user customization + validation)
- **Automated TOML export** for Rust service consumption
- **Docker Compose + Kubernetes templates** for infrastructure deployment
## Directory Structure
```bash
provisioning/.typedialog/provisioning/platform/
├── constraints/ # Single source of truth for validation limits
├── schemas/ # Nickel type contracts (services + common + deployment modes)
├── defaults/ # Default configuration values (services + common + deployment modes)
├── validators/ # Validation logic (constraints, ranges, business rules)
├── configs/ # Generated mode-specific Nickel configurations (4 services ×
├── forms/ # TypeDialog form definitions (4 main forms + flat fragments)
│ └── fragments/ # Reusable form fragments (workspace, server, database, etc.)
├── templates/ # Jinja2 + Nickel templates for config/deployment generation
│ ├── docker-compose/ # Docker Compose templates (solo/multiuser/cicd/enterprise)
│ ├── kubernetes/ # Kubernetes deployment templates
│ └── configs/ # Service configuration templates (TOML generation)
├── scripts/ # Nushell orchestration scripts (configure, generate, validate, deploy)
├── examples/ # Example configurations for different deployment scenarios
└── values/ # User configuration files (gitignored *.ncl)
```
## Configuration Workflow
### 1. User Interaction (TypeDialog)
```nushell
nu scripts/configure.nu orchestrator solo --backend web
```
- Launches interactive form (web/tui/cli)
- Loads existing config as default values (if exists)
- Validates user input against constraints
- Generates updated Nickel config
### 2. Configuration Composition
```toml
Base Defaults (defaults/*.ncl)
↓
+ Mode Overlay (defaults/deployment/{mode}-defaults.ncl)
↓
+ User Customization (values/{service}.{mode}.ncl)
↓
+ Schema Validation (schemas/*.ncl)
↓
+ Constraint Validation (validators/*.ncl)
↓
= Final Configuration (configs/{service}.{mode}.ncl)
```
### 3. TOML Export
```toml
nu scripts/generate-configs.nu orchestrator solo
```
Exports Nickel config to TOML:
- `provisioning/platform/config/orchestrator.solo.toml` (consumed by Rust services)
## Deployment Modes
### Solo (2 CPU, 4GB RAM)
- Single developer/testing
- Filesystem or embedded database
- Minimal security
- All services enabled
### MultiUser (4 CPU, 8GB RAM)
- Team collaboration, staging
- PostgreSQL or SurrealDB server
- RBAC enabled
- Gitea integration
### CI/CD (8 CPU, 16GB RAM)
- Automated pipelines, ephemeral
- API-driven configuration
- Fast cleanup, minimal storage
### Enterprise (16+ CPU, 32+ GB RAM)
- Production high availability
- SurrealDB cluster with replication
- MFA required, KMS integration
- Compliance (SOC2/HIPAA)
## Key Components
### Constraints (constraints/constraints.toml)
Single source of truth for validation limits across all services. Used for:
- Form field validation (min/max values)
- Constraint interpolation in TypeDialog forms
- Nickel validator bounds checking
### Schemas (schemas/*.ncl)
Type-safe configuration contracts defining:
- Required/optional fields
- Valid value types and enums
- Default values
- Input/output type signatures
**Organization**:
- `schemas/common/` - HTTP server, database, security, monitoring, logging
- `schemas/{orchestrator,control-center,mcp-server,vault-service,extension-registry,rag,ai-service,provisioning-daemon}.ncl` - Service-specific schemas
- `schemas/deployment/{solo,multiuser,cicd,enterprise}.ncl` - Mode-specific schemas
### Defaults (defaults/*.ncl)
Configuration base values composed with mode overlays:
- `defaults/{service}-defaults.ncl` - Service base defaults
- `defaults/common/` - Shared defaults (server, database, security)
- `defaults/deployment/{mode}-defaults.ncl` - Mode-specific value overrides
### Validators (validators/*.ncl)
Business logic validation using constraints:
- Port range validation (1024-65535)
- Resource allocation validation (CPU, memory)
- Workflow/policy validation (service-specific)
- Cross-field validation
### Configurations (configs/*.ncl)
Generated mode-specific Nickel configs (NOT manually edited):
- `orchestrator.{solo,multiuser,cicd,enterprise}.ncl`
- `control-center.{solo,multiuser,cicd,enterprise}.ncl`
- `mcp-server.{solo,multiuser,cicd,enterprise}.ncl`
- `vault-service.{solo,multiuser,cicd,enterprise}.ncl`
- `extension-registry.{solo,multiuser,cicd,enterprise}.ncl`
- `rag.{solo,multiuser,cicd,enterprise}.ncl`
- `ai-service.{solo,multiuser,cicd,enterprise}.ncl`
- `provisioning-daemon.{solo,multiuser,cicd,enterprise}.ncl`
### Forms (forms/*.toml)
TypeDialog form definitions with **flat fragments** referenced by paths:
- 4 main forms: `{service}-form.toml`
- Fragments: `fragments/{name}-section.toml` (workspace, server, database, security, monitoring, etc.)
- CRITICAL: Every form element has `nickel_path` for Nickel structure mapping
**Fragment Organization** (FLAT, referenced by paths):
- `workspace-section.toml`
- `server-section.toml`
- `database-rocksdb-section.toml`
- `database-surrealdb-section.toml`
- `database-postgres-section.toml`
- `security-section.toml`
- `monitoring-section.toml`
- `logging-section.toml`
- `orchestrator-queue-section.toml`
- `orchestrator-workflow-section.toml`
- ... (service-specific and mode-specific fragments)
### Templates (templates/)
Jinja2 + Nickel templates for automated generation:
- `{service}-config.ncl.j2` - Nickel output template (critical for TypeDialog nickel-roundtrip)
- `docker-compose/platform-stack.{mode}.yml.ncl` - Docker Compose templates
- `kubernetes/{service}-deployment.yaml.ncl` - Kubernetes templates
### Scripts (scripts/)
Nushell orchestration (NuShell 0.109+):
- `configure.nu` - Interactive TypeDialog wizard (nickel-roundtrip workflow)
- `generate-configs.nu` - Export Nickel → TOML
- `validate-config.nu` - Typecheck Nickel configs
- `render-docker-compose.nu` - Generate Docker Compose files
- `render-kubernetes.nu` - Generate Kubernetes manifests
- `install-services.nu` - Deploy platform services
- `detect-services.nu` - Auto-detect running services
### Examples (examples/)
Reference configurations for different scenarios:
- `orchestrator-solo.ncl` - Simple development setup
- `orchestrator-enterprise.ncl` - Complex production setup
- `full-platform-enterprise.ncl` - Complete enterprise stack
### Values (values/)
User configuration directory (gitignored):
- `{service}.{mode}.ncl` - User customizations (loaded in compose)
- `.gitignore` - Ignores `*.ncl` files
- `orchestrator.example.ncl` - Documented example template
## TypeDialog nickel-roundtrip Workflow
CRITICAL: Forms use Jinja2 templates for Nickel generation:
```nickel
# Command pattern
typedialog-web nickel-roundtrip "$CONFIG_FILE" "$FORM_FILE" --output "$CONFIG_FILE" --template "$NCL_TEMPLATE"
# Example
typedialog-web nickel-roundtrip
"provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl"
"provisioning/.typedialog/provisioning/platform/forms/orchestrator-form.toml"
--output "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl"
--template "provisioning/.typedialog/provisioning/platform/templates/orchestrator-config.ncl.j2"
```
**Key Requirements**:
1. **Jinja2 template** (`config.ncl.j2` ) - Defines Nickel output structure with conditional `{% if %}` blocks
2. **nickel_path** in form elements - Maps form fields to Nickel structure paths (e.g., `["orchestrator", "queue", "max_concurrent_tasks"]` )
3. **Constraint interpolation** - Form limits reference constraints (e.g., `${constraint.orchestrator.queue.concurrent_tasks.max}` )
4. **Base + overlay composition** - Nickel imports merge defaults + mode overlays + validators
## Usage Workflow
### 1. Configure Service (Interactive)
```toml
# Start TypeDialog wizard for orchestrator in solo mode
nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo --backend web
```
Wizard:
1. Loads existing config (if exists) as defaults
2. Shows form with validated constraints
3. User edits configuration
4. Generates updated Nickel config to `provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl`
### 2. Validate Configuration
```toml
# Typecheck Nickel config
nu provisioning/.typedialog/provisioning/platform/scripts/validate-config.nu provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl
```
### 3. Generate TOML for Rust Services
```toml
# Export Nickel → TOML
nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo
```
Output: `provisioning/platform/config/orchestrator.solo.toml`
### 4. Deploy Services
```bash
# Install services (Docker Compose or Kubernetes)
nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo
```
## Configuration Loading Hierarchy (Rust Services)
```rust
1. Environment variables (ORCHESTRATOR_*)
2. User config (values/{service}.{mode}.ncl → TOML)
3. Mode-specific defaults (configs/{service}.{mode}.toml)
4. Service defaults (config/orchestrator.defaults.toml)
```
## Constraint Interpolation Example
**constraints.toml**:
```toml
[orchestrator.queue.concurrent_tasks]
min = 1
max = 100
```
**Form element** (fragments/orchestrator-queue-section.toml):
```toml
[[elements]]
name = "max_concurrent_tasks"
type = "number"
min = "${constraint.orchestrator.queue.concurrent_tasks.min}"
max = "${constraint.orchestrator.queue.concurrent_tasks.max}"
nickel_path = ["orchestrator", "queue", "max_concurrent_tasks"]
```
**Jinja2 template** (orchestrator-config.ncl.j2):
```nickel
orchestrator = {
queue = {
{%- if max_concurrent_tasks %}
max_concurrent_tasks = {{ max_concurrent_tasks }},
{%- endif %}
},
}
```
## Getting Started
1. **Run configuration wizard** :
```bash
nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo
```
2. **Generate TOML configs** :
```bash
nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo
```
3. **Deploy services** :
```bash
nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo
```
## Documentation
- `constraints/README.md` - How to modify validation constraints
- `schemas/README.md` - Schema patterns and imports
- `defaults/README.md` - Defaults composition and merging strategy
- `validators/README.md` - Validator patterns and error handling
- `forms/README.md` - Form structure and fragment organization
- `forms/fragments/README.md` - Fragment usage and nickel_path mapping
- `scripts/README.md` - Script usage and dependencies
- `examples/README.md` - Example deployment scenarios
- `templates/README.md` - Template patterns and interpolation
## Key Files
| File | Purpose |
| ------ | --------- |
| `constraints/constraints.toml` | Single source of truth for validation limits |
| `schemas/orchestrator.ncl` | Orchestrator type schema |
| `defaults/orchestrator-defaults.ncl` | Orchestrator default values |
| `validators/orchestrator-validator.ncl` | Orchestrator validation logic |
| `configs/orchestrator.solo.ncl` | Generated solo mode config |
| `forms/orchestrator-form.toml` | Orchestrator form definition |
| `templates/orchestrator-config.ncl.j2` | Nickel output template |
| `scripts/configure.nu` | Interactive configuration wizard |
| `scripts/generate-configs.nu` | Nickel → TOML export |
| `values/orchestrator.solo.ncl` | User configuration (gitignored) |
## Tools Required
- **Nickel** (0.10+) - Configuration language
- **TypeDialog** - Interactive form backend
- **NuShell** (0.109+) - Script orchestration
- **Jinja2/tera** - Template rendering (via nu_plugin_tera)
- **TOML** - Config file format (for Rust services)
## Notes
- Configuration files in `values/` are **gitignored** (user-specific)
- Generated configs in `configs/` are composed automatically (not hand-edited)
- Each mode (solo/multiuser/cicd/enterprise) has different resource defaults
- Fragments are **flat** in `forms/fragments/` and referenced by paths in form definitions
- All form elements must have `nickel_path` for proper Nickel structure mapping
- Constraint interpolation enables dynamic form validation based on service requirements
---
**Version**: 1.0.0
**Created**: 2025-01-05
**Last Updated**: 2025-01-05
