# ADR-010: Configuration File Format Strategy\n\n**Status**: Accepted\n**Date**: 2025-12-03\n**Decision Makers**: Architecture Team\n**Implementation**: Multi-phase migration (KCL workspace configs + template reorganization)\n\n---\n\n## Context\n\nThe provisioning project historically used a single configuration format (YAML/TOML environment variables) for all purposes. As the system evolved,\ndifferent parts naturally adopted different formats:\n\n- **TOML** for modular provider and platform configurations (`providers/*.toml`, `platform/*.toml`)\n- **KCL** for infrastructure-as-code definitions with type safety\n- **YAML** for workspace metadata\n\nHowever, the workspace configuration remained in **YAML** (`provisioning.yaml`),\ncreating inconsistency and leaving type-unsafe configuration handling. Meanwhile,\ncomplete KCL schemas for workspace configuration were designed but unused.\n\n**Problem**: Three different formats in the same system without documented rationale or consistent patterns.\n\n---\n\n## Decision\n\nAdopt a **three-format strategy** with clear separation of concerns:\n\n| Format | Purpose | Use Cases |\n| -------- | --------- | ----------- |\n| **KCL** | Infrastructure as Code & Schemas | Workspace config, infrastructure definitions, type-safe validation |\n| **TOML** | Application Configuration & Settings | System defaults, provider settings, user preferences, interpolation |\n| **YAML** | Metadata & Kubernetes Resources | K8s manifests, tool metadata, version tracking, CI/CD resources |\n\n---\n\n## Implementation Strategy\n\n### Phase 1: Documentation (Complete)\n\nDefine and document the three-format approach through:\n\n1. **ADR-010** (this document) - Rationale and strategy\n2. **CLAUDE.md updates** - Quick reference for developers\n3. **Configuration hierarchy** - Explicit precedence rules\n\n### Phase 2: Workspace Config Migration (In Progress)\n\n**Migrate workspace configuration from YAML to KCL**:\n\n1. Create comprehensive workspace configuration schema in KCL\n2. Implement backward-compatible config loader (KCL first, fallback to YAML)\n3. Provide migration script to convert existing workspaces\n4. Update workspace initialization to generate KCL configs\n\n**Expected Outcome**:\n\n- `workspace/config/provisioning.ncl` (KCL, type-safe, validated)\n- Full schema validation with semantic versioning checks\n- Automatic validation at config load time\n\n### Phase 3: Template File Reorganization (In Progress)\n\n**Move template files to proper directory structure and correct extensions**:\n\n```\nPrevious (KCL):\n  provisioning/kcl/templates/*.k  (had Nushell/Jinja2 code, not KCL)\n\nCurrent (Nickel):\n  provisioning/templates/\n    ├── nushell/*.nu.j2\n    ├── config/*.toml.j2\n    ├── nickel/*.ncl.j2\n    └── README.md\n```\n\n**Expected Outcome**:\n\n- Templates properly classified and discoverable\n- KCL validation passes (15/16 errors eliminated)\n- Template system clean and maintainable\n\n---\n\n## Rationale for Each Format\n\n### KCL for Workspace Configuration\n\n**Why KCL over YAML or TOML?**\n\n1. **Type Safety**: Catch configuration errors at schema validation time, not runtime\n\n   ```kcl\n   schema WorkspaceDeclaration:\n       metadata: Metadata\n       check:\n           regex.match(metadata.version, r"^\d+\.\d+\.\d+$"), \\n               "Version must be semantic versioning"\n   ```\n\n1. **Schema-First Development**: Schemas are first-class citizens\n   - Document expected structure upfront\n   - IDE support for auto-completion\n   - Enforce required fields and value ranges\n\n2. **Immutable by Default**: Infrastructure configurations are immutable\n   - Prevents accidental mutations\n   - Better for reproducible deployments\n   - Aligns with PAP principle: "configuration-driven, not hardcoded"\n\n3. **Complex Validation**: KCL supports sophisticated validation rules\n   - Semantic versioning validation\n   - Dependency checking\n   - Cross-field validation\n   - Range constraints on numeric values\n\n4. **Ecosystem Consistency**: KCL is already used for infrastructure definitions\n   - Server configurations use KCL\n   - Cluster definitions use KCL\n   - Taskserv definitions use KCL\n   - Using KCL for workspace config maintains consistency\n\n5. **Existing Schemas**: `provisioning/kcl/generator/declaration.ncl` already defines complete workspace schemas\n   - No design work needed\n   - Production-ready schemas\n   - Well-tested patterns\n\n### TOML for Application Configuration\n\n**Why TOML for settings?**\n\n1. **Hierarchical Structure**: Native support for nested configurations\n\n   ```toml\n   [http]\n   use_curl = false\n   timeout = 30\n\n   [debug]\n   enabled = false\n   log_level = "info"\n   ```\n\n2. **Interpolation Support**: Dynamic variable substitution\n\n   ```toml\n   base_path = "/Users/home/provisioning"\n   cache_path = "{{base_path}}/.cache"\n   ```\n\n3. **Industry Standard**: Widely used for application configuration (Rust, Python, Go)\n\n4. **Human Readable**: Clear, explicit, easy to edit\n\n5. **Validation Support**: Schema files (`.schema.toml`) for validation\n\n**Use Cases**:\n\n- System defaults: `provisioning/config/config.defaults.toml`\n- Provider settings: `workspace/config/providers/*.toml`\n- Platform services: `workspace/config/platform/*.toml`\n- User preferences: User config files\n\n### YAML for Metadata and Kubernetes Resources\n\n**Why YAML for metadata?**\n\n1. **Kubernetes Compatibility**: YAML is K8s standard\n   - K8s manifests use YAML\n   - Consistent with ecosystem\n   - Familiar to DevOps engineers\n\n2. **Lightweight**: Good for simple data structures\n\n   ```yaml\n   workspace:\n     name: "librecloud"\n     version: "1.0.0"\n     created: "2025-10-06T12:29:43Z"\n   ```\n\n3. **Version Control**: Human-readable format\n   - Diffs are clear and meaningful\n   - Git-friendly\n   - Comments supported\n\n**Use Cases**:\n\n- K8s resource definitions\n- Tool metadata (versions, sources, tags)\n- CI/CD configuration files\n- User workspace metadata (during transition)\n\n---\n\n## Configuration Hierarchy (Priority)\n\n**When loading configuration, use this precedence (highest to lowest)**:\n\n1. **Runtime Arguments** (highest priority)\n   - CLI flags passed to commands\n   - Explicit user input\n\n2. **Environment Variables** (PROVISIONING_*)\n   - Override system settings\n   - Deployment-specific overrides\n   - Secrets via env vars\n\n3. **User Configuration** (Centralized)\n   - User preferences: `~/.config/provisioning/user_config.yaml`\n   - User workspace overrides: `workspace/config/local-overrides.toml`\n\n4. **Infrastructure Configuration**\n   - Workspace KCL config: `workspace/config/provisioning.ncl`\n   - Platform services: `workspace/config/platform/*.toml`\n   - Provider configs: `workspace/config/providers/*.toml`\n\n5. **System Defaults** (lowest priority)\n   - System config: `provisioning/config/config.defaults.toml`\n   - Schema defaults: defined in KCL schemas\n\n---\n\n## Migration Path\n\n### For Existing Workspaces\n\n1. **Migration Path**: Config loader checks for `.ncl` first, then falls back to `.yaml` for legacy systems\n\n   ```nushell\n   # Try Nickel first (current)\n   if ($config_nickel | path exists) {\n       let config = (load_nickel_workspace_config $config_nickel)\n   } else if ($config_yaml | path exists) {\n       # Legacy YAML support (from pre-migration)\n       let config = (open $config_yaml)\n   }\n   ```\n\n2. **Automatic Migration**: Migration script converts YAML/KCL → Nickel\n\n   ```bash\n   provisioning workspace migrate-config --all\n   ```\n\n3. **Validation**: New KCL configs validated against schemas\n\n### For New Workspaces\n\n1. **Generate KCL**: Workspace initialization creates `.k` files\n\n   ```bash\n   provisioning workspace create my-workspace\n   # Creates: workspace/my-workspace/config/provisioning.ncl\n   ```\n\n2. **Use Existing Schemas**: Leverage `provisioning/kcl/generator/declaration.ncl`\n\n3. **Schema Validation**: Automatic validation during config load\n\n---\n\n## File Format Guidelines for Developers\n\n### When to Use Each Format\n\n**Use KCL for**:\n\n- Infrastructure definitions (servers, clusters, taskservs)\n- Configuration with type requirements\n- Schema definitions\n- Any config that needs validation rules\n- Workspace configuration\n\n**Use TOML for**:\n\n- Application settings (HTTP client, logging, timeouts)\n- Provider-specific settings\n- Platform service configuration\n- User preferences and overrides\n- System defaults with interpolation\n\n**Use YAML for**:\n\n- Kubernetes manifests\n- CI/CD configuration (GitHub Actions, GitLab CI)\n- Tool metadata\n- Human-readable documentation files\n- Version control metadata\n\n---\n\n## Consequences\n\n### Benefits\n\n✅ **Type Safety**: KCL schema validation catches config errors early\n✅ **Consistency**: Infrastructure definitions and configs use same language\n✅ **Maintainability**: Clear separation of concerns (IaC vs settings vs metadata)\n✅ **Validation**: Semantic versioning, required fields, range checks\n✅ **Tooling**: IDE support for KCL auto-completion\n✅ **Documentation**: Self-documenting schemas with descriptions\n✅ **Ecosystem Alignment**: TOML for settings (Rust standard), YAML for K8s\n\n### Trade-offs\n\n⚠️ **Learning Curve**: Developers must understand three formats\n⚠️ **Migration Effort**: Existing YAML configs need conversion\n⚠️ **Tooling Requirements**: KCL compiler needed (already a dependency)\n\n### Risk Mitigation\n\n1. **Documentation**: Clear guidelines in CLAUDE.md\n2. **Backward Compatibility**: YAML support maintained during transition\n3. **Automation**: Migration scripts for existing workspaces\n4. **Gradual Migration**: No hard cutoff, both formats supported for extended period\n\n---\n\n## Template File Reorganization\n\n### Problem\n\nCurrently, 15/16 files in `provisioning/kcl/templates/` have `.k` extension but contain Nushell/Jinja2 code, not KCL:\n\n```\nprovisioning/kcl/templates/\n├── server.ncl          # Actually Nushell/Jinja2 template\n├── taskserv.ncl        # Actually Nushell/Jinja2 template\n└── ...               # 15 more template files\n```\n\nThis causes:\n\n- KCL validation failures (96.6% of errors)\n- Misclassification (templates in KCL directory)\n- Confusing directory structure\n\n### Solution\n\nReorganize into type-specific directories:\n\n```\nprovisioning/templates/\n├── nushell/           # Nushell code generation (*.nu.j2)\n│   ├── server.nu.j2\n│   ├── taskserv.nu.j2\n│   └── ...\n├── config/            # Config file generation (*.toml.j2, *.yaml.j2)\n│   ├── provider.toml.j2\n│   └── ...\n├── kcl/               # KCL file generation (*.k.j2)\n│   ├── workspace.ncl.j2\n│   └── ...\n└── README.md\n```\n\n### Outcome\n\n✅ Correct file classification\n✅ KCL validation passes completely\n✅ Clear template organization\n✅ Easier to discover and maintain templates\n\n---\n\n## References\n\n### Existing KCL Schemas\n\n1. **Workspace Declaration**: `provisioning/kcl/generator/declaration.ncl`\n   - `WorkspaceDeclaration` - Complete workspace specification\n   - `Metadata` - Name, version, author, timestamps\n   - `DeploymentConfig` - Deployment modes, servers, HA settings\n   - Includes validation rules and semantic versioning\n\n2. **Workspace Layer**: `provisioning/workspace/layers/workspace.layer.ncl`\n   - `WorkspaceLayer` - Template paths, priorities, metadata\n\n3. **Core Settings**: `provisioning/kcl/settings.ncl`\n   - `Settings` - Main provisioning settings\n   - `SecretProvider` - SOPS/KMS configuration\n   - `AIProvider` - AI provider configuration\n\n### Related ADRs\n\n- **ADR-001**: Project Structure\n- **ADR-005**: Extension Framework\n- **ADR-006**: Provisioning CLI Refactoring\n- **ADR-009**: Security System Complete\n\n---\n\n## Decision Status\n\n**Status**: Accepted\n\n**Next Steps**:\n\n1. ✅ Document strategy (this ADR)\n2. ⏳ Create workspace configuration KCL schema\n3. ⏳ Implement backward-compatible config loader\n4. ⏳ Create migration script for YAML → KCL\n5. ⏳ Move template files to proper directories\n6. ⏳ Update documentation with examples\n7. ⏳ Migrate workspace_librecloud to KCL\n\n---\n\n**Last Updated**: 2025-12-03