13 KiB
13 KiB
\nprovisioning/.typedialog/provisioning/platform/\n├── constraints/ # Single source of truth for validation limits\n├── schemas/ # Nickel type contracts (services + common + deployment modes)\n├── defaults/ # Default configuration values (services + common + deployment modes)\n├── validators/ # Validation logic (constraints, ranges, business rules)\n├── configs/ # Generated mode-specific Nickel configurations (4 services × 4 modes = 16 configs)\n├── forms/ # TypeDialog form definitions (4 main forms + flat fragments)\n│ └── fragments/ # Reusable form fragments (workspace, server, database, etc.)\n├── templates/ # Jinja2 + Nickel templates for config/deployment generation\n│ ├── docker-compose/ # Docker Compose templates (solo/multiuser/cicd/enterprise)\n│ ├── kubernetes/ # Kubernetes deployment templates\n│ └── configs/ # Service configuration templates (TOML generation)\n├── scripts/ # Nushell orchestration scripts (configure, generate, validate, deploy)\n├── examples/ # Example configurations for different deployment scenarios\n└── values/ # User configuration files (gitignored *.ncl)\n\n\n## Configuration Workflow\n\n### 1. User Interaction (TypeDialog)\n\n\nnu scripts/configure.nu orchestrator solo --backend web\n\n\n- Launches interactive form (web/tui/cli)\n- Loads existing config as default values (if exists)\n- Validates user input against constraints\n- Generates updated Nickel config\n\n### 2. Configuration Composition\n\n\nBase Defaults (defaults/*.ncl)\n ↓\n+ Mode Overlay (defaults/deployment/{mode}-defaults.ncl)\n ↓\n+ User Customization (values/{service}.{mode}.ncl)\n ↓\n+ Schema Validation (schemas/*.ncl)\n ↓\n+ Constraint Validation (validators/*.ncl)\n ↓\n= Final Configuration (configs/{service}.{mode}.ncl)\n\n\n### 3. TOML Export\n\n\nnu scripts/generate-configs.nu orchestrator solo\n\n\nExports Nickel config to TOML:\n- provisioning/platform/config/orchestrator.solo.toml (consumed by Rust services)\n\n## Deployment Modes\n\n### Solo (2 CPU, 4GB RAM)\n- Single developer/testing\n- Filesystem or embedded database\n- Minimal security\n- All services enabled\n\n### MultiUser (4 CPU, 8GB RAM)\n- Team collaboration, staging\n- PostgreSQL or SurrealDB server\n- RBAC enabled\n- Gitea integration\n\n### CI/CD (8 CPU, 16GB RAM)\n- Automated pipelines, ephemeral\n- API-driven configuration\n- Fast cleanup, minimal storage\n\n### Enterprise (16+ CPU, 32+ GB RAM)\n- Production high availability\n- SurrealDB cluster with replication\n- MFA required, KMS integration\n- Compliance (SOC2/HIPAA)\n\n## Key Components\n\n### Constraints (constraints/constraints.toml)\nSingle source of truth for validation limits across all services. Used for:\n- Form field validation (min/max values)\n- Constraint interpolation in TypeDialog forms\n- Nickel validator bounds checking\n\n### Schemas (schemas/.ncl)\nType-safe configuration contracts defining:\n- Required/optional fields\n- Valid value types and enums\n- Default values\n- Input/output type signatures\n\nOrganization:\n- schemas/common/ - HTTP server, database, security, monitoring, logging\n- schemas/{orchestrator,control-center,mcp-server,vault-service,extension-registry,rag,ai-service,provisioning-daemon}.ncl - Service-specific schemas\n- schemas/deployment/{solo,multiuser,cicd,enterprise}.ncl - Mode-specific schemas\n\n### Defaults (defaults/.ncl)\nConfiguration base values composed with mode overlays:\n- defaults/{service}-defaults.ncl - Service base defaults\n- defaults/common/ - Shared defaults (server, database, security)\n- defaults/deployment/{mode}-defaults.ncl - Mode-specific value overrides\n\n### Validators (validators/.ncl)\nBusiness logic validation using constraints:\n- Port range validation (1024-65535)\n- Resource allocation validation (CPU, memory)\n- Workflow/policy validation (service-specific)\n- Cross-field validation\n\n### Configurations (configs/.ncl)\nGenerated mode-specific Nickel configs (NOT manually edited):\n- orchestrator.{solo,multiuser,cicd,enterprise}.ncl\n- control-center.{solo,multiuser,cicd,enterprise}.ncl\n- mcp-server.{solo,multiuser,cicd,enterprise}.ncl\n- vault-service.{solo,multiuser,cicd,enterprise}.ncl\n- extension-registry.{solo,multiuser,cicd,enterprise}.ncl\n- rag.{solo,multiuser,cicd,enterprise}.ncl\n- ai-service.{solo,multiuser,cicd,enterprise}.ncl\n- provisioning-daemon.{solo,multiuser,cicd,enterprise}.ncl\n\n### Forms (forms/*.toml)\nTypeDialog form definitions with flat fragments referenced by paths:\n- 4 main forms: {service}-form.toml\n- Fragments: fragments/{name}-section.toml (workspace, server, database, security, monitoring, etc.)\n- CRITICAL: Every form element has nickel_path for Nickel structure mapping\n\nFragment Organization (FLAT, referenced by paths):\n- workspace-section.toml\n- server-section.toml\n- database-rocksdb-section.toml\n- database-surrealdb-section.toml\n- database-postgres-section.toml\n- security-section.toml\n- monitoring-section.toml\n- logging-section.toml\n- orchestrator-queue-section.toml\n- orchestrator-workflow-section.toml\n- ... (service-specific and mode-specific fragments)\n\n### Templates (templates/)\nJinja2 + Nickel templates for automated generation:\n- {service}-config.ncl.j2 - Nickel output template (critical for TypeDialog nickel-roundtrip)\n- docker-compose/platform-stack.{mode}.yml.ncl - Docker Compose templates\n- kubernetes/{service}-deployment.yaml.ncl - Kubernetes templates\n\n### Scripts (scripts/)\nNushell orchestration (NuShell 0.109+):\n- configure.nu - Interactive TypeDialog wizard (nickel-roundtrip workflow)\n- generate-configs.nu - Export Nickel → TOML\n- validate-config.nu - Typecheck Nickel configs\n- render-docker-compose.nu - Generate Docker Compose files\n- render-kubernetes.nu - Generate Kubernetes manifests\n- install-services.nu - Deploy platform services\n- detect-services.nu - Auto-detect running services\n\n### Examples (examples/)\nReference configurations for different scenarios:\n- orchestrator-solo.ncl - Simple development setup\n- orchestrator-enterprise.ncl - Complex production setup\n- full-platform-enterprise.ncl - Complete enterprise stack\n\n### Values (values/)\nUser configuration directory (gitignored):\n- {service}.{mode}.ncl - User customizations (loaded in compose)\n- .gitignore - Ignores *.ncl files\n- orchestrator.example.ncl - Documented example template\n\n## TypeDialog nickel-roundtrip Workflow\n\nCRITICAL: Forms use Jinja2 templates for Nickel generation:\n\n\n# Command pattern\ntypedialog-web nickel-roundtrip "$CONFIG_FILE" "$FORM_FILE" --output "$CONFIG_FILE" --template "$NCL_TEMPLATE"\n\n# Example\ntypedialog-web nickel-roundtrip \n "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl" \n "provisioning/.typedialog/provisioning/platform/forms/orchestrator-form.toml" \n --output "provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl" \n --template "provisioning/.typedialog/provisioning/platform/templates/orchestrator-config.ncl.j2"\n\n\nKey Requirements:\n1. Jinja2 template (config.ncl.j2) - Defines Nickel output structure with conditional {% if %} blocks\n2. nickel_path in form elements - Maps form fields to Nickel structure paths (e.g., ["orchestrator", "queue", "max_concurrent_tasks"])\n3. Constraint interpolation - Form limits reference constraints (e.g., ${constraint.orchestrator.queue.concurrent_tasks.max})\n4. Base + overlay composition - Nickel imports merge defaults + mode overlays + validators\n\n## Usage Workflow\n\n### 1. Configure Service (Interactive)\n\n\n# Start TypeDialog wizard for orchestrator in solo mode\nnu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo --backend web\n\n\nWizard:\n1. Loads existing config (if exists) as defaults\n2. Shows form with validated constraints\n3. User edits configuration\n4. Generates updated Nickel config to provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl\n\n### 2. Validate Configuration\n\n\n# Typecheck Nickel config\nnu provisioning/.typedialog/provisioning/platform/scripts/validate-config.nu provisioning/.typedialog/provisioning/platform/values/orchestrator.solo.ncl\n\n\n### 3. Generate TOML for Rust Services\n\n\n# Export Nickel → TOML\nnu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo\n\n\nOutput: provisioning/platform/config/orchestrator.solo.toml\n\n### 4. Deploy Services\n\n\n# Install services (Docker Compose or Kubernetes)\nnu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo\n\n\n## Configuration Loading Hierarchy (Rust Services)\n\n\n1. Environment variables (ORCHESTRATOR_*)\n2. User config (values/{service}.{mode}.ncl → TOML)\n3. Mode-specific defaults (configs/{service}.{mode}.toml)\n4. Service defaults (config/orchestrator.defaults.toml)\n\n\n## Constraint Interpolation Example\n\nconstraints.toml:\n\n\n[orchestrator.queue.concurrent_tasks]\nmin = 1\nmax = 100\n\n\nForm element (fragments/orchestrator-queue-section.toml):\n\n\n[[elements]]\nname = "max_concurrent_tasks"\ntype = "number"\nmin = "${constraint.orchestrator.queue.concurrent_tasks.min}"\nmax = "${constraint.orchestrator.queue.concurrent_tasks.max}"\nnickel_path = ["orchestrator", "queue", "max_concurrent_tasks"]\n\n\nJinja2 template (orchestrator-config.ncl.j2):\n\n\norchestrator = {\n queue = {\n {%- if max_concurrent_tasks %}\n max_concurrent_tasks = {{ max_concurrent_tasks }},\n {%- endif %}\n },\n}\n\n\n## Getting Started\n\n1. Run configuration wizard:\n\n bash\n nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo\n \n\n2. Generate TOML configs:\n\n bash\n nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo\n \n\n3. Deploy services:\n\n bash\n nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo\n \n\n## Documentation\n\n- constraints/README.md - How to modify validation constraints\n- schemas/README.md - Schema patterns and imports\n- defaults/README.md - Defaults composition and merging strategy\n- validators/README.md - Validator patterns and error handling\n- forms/README.md - Form structure and fragment organization\n- forms/fragments/README.md - Fragment usage and nickel_path mapping\n- scripts/README.md - Script usage and dependencies\n- examples/README.md - Example deployment scenarios\n- templates/README.md - Template patterns and interpolation\n\n## Key Files\n\n| File | Purpose |\n| ------ | --------- |\n| constraints/constraints.toml | Single source of truth for validation limits |\n| schemas/orchestrator.ncl | Orchestrator type schema |\n| defaults/orchestrator-defaults.ncl | Orchestrator default values |\n| validators/orchestrator-validator.ncl | Orchestrator validation logic |\n| configs/orchestrator.solo.ncl | Generated solo mode config |\n| forms/orchestrator-form.toml | Orchestrator form definition |\n| templates/orchestrator-config.ncl.j2 | Nickel output template |\n| scripts/configure.nu | Interactive configuration wizard |\n| scripts/generate-configs.nu | Nickel → TOML export |\n| values/orchestrator.solo.ncl | User configuration (gitignored) |\n\n## Tools Required\n\n- Nickel (0.10+) - Configuration language\n- TypeDialog - Interactive form backend\n- NuShell (0.109+) - Script orchestration\n- Jinja2/tera - Template rendering (via nu_plugin_tera)\n- TOML - Config file format (for Rust services)\n\n## Notes\n\n- Configuration files in values/ are gitignored (user-specific)\n- Generated configs in configs/ are composed automatically (not hand-edited)\n- Each mode (solo/multiuser/cicd/enterprise) has different resource defaults\n- Fragments are flat in forms/fragments/ and referenced by paths in form definitions\n- All form elements must have nickel_path for proper Nickel structure mapping\n- Constraint interpolation enables dynamic form validation based on service requirements\n\n---\n\nVersion: 1.0.0\nCreated: 2025-01-05\nLast Updated: 2025-01-05