Layered Template Architecture System\n\nThis workspace provides a combined Layered Extension Architecture with Override System and Template-Based Infrastructure Pattern Library that maintains PAP principles while enabling maximum reusability of infrastructure configurations.\n\n## 🏗️ Architecture Overview\n\n### Layer Hierarchy\n\nThe system resolves configurations through a three-tier layer system:\n\n1. Core Layer (Priority 100) - provisioning/extensions/\n - Base provisioning system extensions\n - Core taskservs, providers, and clusters\n\n2. Workspace Layer (Priority 200) - provisioning/workspace/templates/\n - Shared templates extracted from proven infrastructure patterns\n - Reusable configurations across multiple infrastructures\n\n3. Infrastructure Layer (Priority 300) - workspace/infra/{name}/\n - Infrastructure-specific configurations and overrides\n - Custom implementations per infrastructure\n\n### Directory Structure\n\n\nprovisioning/workspace/\n├── templates/ # Template library\n│ ├── taskservs/ # Taskserv configuration templates\n│ │ ├── kubernetes/ # Kubernetes templates\n│ │ │ ├── base.k # Base configuration\n│ │ │ └── variants/ # HA, single-node variants\n│ │ ├── storage/ # Storage system templates\n│ │ ├── networking/ # Network configuration templates\n│ │ └── container-runtime/ # Container runtime templates\n│ ├── providers/ # Provider templates\n│ │ ├── upcloud/ # UpCloud provider templates\n│ │ └── aws/ # AWS provider templates\n│ ├── servers/ # Server configuration patterns\n│ └── clusters/ # Complete cluster templates\n├── layers/ # Layer definitions\n│ ├── core.layer.k # Core layer definition\n│ ├── workspace.layer.k # Workspace layer definition\n│ └── infra.layer.k # Infrastructure layer definition\n├── registry/ # Extension registry\n│ ├── manifest.yaml # Template catalog and metadata\n│ └── imports.k # Central import aliases\n├── templates/lib/ # Composition utilities\n│ ├── compose.k # KCL composition functions\n│ └── override.k # Override and layer utilities\n└── tools/ # Migration and management tools\n └── migrate-infra.nu # Infrastructure migration tool\n\n\n## 🚀 Getting Started\n\n### 1. Extract Existing Patterns\n\nExtract patterns from existing infrastructure (e.g., wuji) to create reusable templates:\n\n\n# Extract all patterns from wuji infrastructure\ncd provisioning/workspace/tools\n./migrate-infra.nu extract wuji\n\n# Extract specific types only\n./migrate-infra.nu extract wuji --type taskservs\n\n\n### 2. Use Enhanced Module Loader\n\nThe enhanced module loader provides template and layer management:\n\n\n# List available templates\nprovisioning/core/cli/module-loader-enhanced template list\n\n# Show layer resolution order\nprovisioning/core/cli/module-loader-enhanced layer show\n\n# Test layer resolution for a specific module\nprovisioning/core/cli/module-loader-enhanced layer test kubernetes --infra wuji\n\n\n### 3. Apply Templates to New Infrastructure\n\n\n# Apply kubernetes template to new infrastructure\nprovisioning/core/cli/module-loader-enhanced template apply kubernetes-base new-infra --provider upcloud\n\n# Load taskservs using templates\nprovisioning/core/cli/module-loader-enhanced load enhanced taskservs workspace/infra/new-infra [kubernetes, cilium] --layer workspace\n\n\n## 📋 Usage Examples\n\n### Creating a New Infrastructure from Templates\n\n\n# 1. Create directory structure\nmkdir -p workspace/infra/my-new-infra/{taskservs,defs,overrides}\n\n# 2. Apply base templates\ncd provisioning\n./core/cli/module-loader-enhanced template apply kubernetes-base my-new-infra\n\n# 3. Customize for your needs\n# Edit workspace/infra/my-new-infra/taskservs/kubernetes.k\n\n# 4. Test layer resolution\n./core/cli/module-loader-enhanced layer test kubernetes --infra my-new-infra\n\n\n### Converting Existing Infrastructure\n\n\n# 1. Extract patterns to templates\ncd provisioning/workspace/tools\n./migrate-infra.nu extract existing-infra\n\n# 2. Convert infrastructure to use templates\n./migrate-infra.nu convert existing-infra\n\n# 3. Validate conversion\n./migrate-infra.nu validate existing-infra\n\n\n### Template Development\n\n\n# Create a new taskserv template\n# provisioning/workspace/templates/taskservs/my-service/base.k\n\nimport taskservs.my_service.kcl.my_service as service_core\nimport ../../../workspace/templates/lib/compose as comp\n\nschema MyServiceBase {\n version: str = "1.0.0"\n cluster_name: str\n # ... configuration options\n}\n\ndef create_my_service [cluster_name: str, overrides: any = {}] -> any {\n let base_config = MyServiceBase { cluster_name = $cluster_name }\n let final_config = comp.deep_merge $base_config $overrides\n\n service_core.MyService $final_config\n}\n\n\n## 🔧 Configuration Composition\n\n### Using Templates in Infrastructure\n\n\n# workspace/infra/my-infra/taskservs/kubernetes.k\n\nimport provisioning.workspace.registry.imports as reg\nimport provisioning.workspace.templates.lib.override as ovr\n\n# Use base template with infrastructure-specific overrides\n_taskserv = ovr.infrastructure_overrides.taskserv_override(\n reg.tpl_kubernetes_base.kubernetes_base,\n "my-infra",\n "kubernetes",\n {\n cluster_name: "my-infra"\n version: "1.31.0" # Override version\n cri: "crio" # Override container runtime\n # Custom network configuration\n network_config: {\n pod_cidr: "10.244.0.0/16"\n service_cidr: "10.96.0.0/12"\n }\n }\n)\n\n\n### Layer Composition\n\n\n# Compose configuration through all layers\nimport provisioning.workspace.templates.lib.compose as comp\n\n# Manual layer composition\nfinal_config = comp.compose_templates(\n $core_config, # From provisioning/extensions\n $workspace_config, # From provisioning/workspace/templates\n $infra_config # From workspace/infra/{name}\n)\n\n\n## 🛠️ Advanced Features\n\n### Provider-Aware Composition\n\n\nimport provisioning.workspace.templates.lib.override as ovr\n\n# Apply provider-specific configurations\nconfig = ovr.override_patterns.env_override(\n $base_config,\n "upcloud",\n {\n upcloud: { zone: "es-mad1", plan: "2xCPU-4GB" },\n aws: { region: "us-west-2", instance_type: "t3.medium" },\n local: { memory: "4GB", cpus: 2 }\n }\n)\n\n\n### Conditional Overrides\n\n\n# Infrastructure-specific conditional overrides\nconfig = ovr.layer_resolution.infra_conditional(\n $base_config,\n $infra_name,\n {\n "production": { ha: true, replicas: 3 },\n "development": { ha: false, replicas: 1 },\n "default": { ha: false, replicas: 1 }\n }\n)\n\n\n## 📚 Benefits\n\n### ✅ Maintains PAP Principles\n\n- Configuration-driven: No hardcoded values\n- Modular: Clear separation of concerns\n- Declarative: Infrastructure as code\n- Reusable: DRY principle throughout\n\n### ✅ Flexible Override System\n\n- Layer-based resolution: Clean precedence order\n- Selective overrides: Override only what's needed\n- Provider-agnostic: Works across all providers\n- Environment-aware: Dev/test/prod configurations\n\n### ✅ Template Reusability\n\n- Pattern extraction: Learn from existing infrastructures\n- Template versioning: Track evolution over time\n- Composition utilities: Rich KCL composition functions\n- Migration tools: Easy conversion process\n\n### ✅ No Core Changes\n\n- Non-invasive: Core provisioning system unchanged\n- Backward compatible: Existing infrastructures continue working\n- Progressive adoption: Migrate at your own pace\n- Extensible: Add new templates and layers easily\n\n## 🔄 Migration Path\n\n1. Extract patterns from existing infrastructures using migrate-infra.nu extract\n2. Create templates in provisioning/workspace/templates/\n3. Convert infrastructures to use templates with migrate-infra.nu convert\n4. Validate the conversion with migrate-infra.nu validate\n5. Test layer resolution with enhanced module loader\n6. Iterate and improve templates based on usage\n\n## 📖 Further Reading\n\n- Layer Definitions: See layers/*.layer.k for layer configuration\n- Template Examples: Browse templates/ for real-world patterns\n- Composition Utilities: Check templates/lib/ for KCL utilities\n- Migration Tools: Use tools/migrate-infra.nu for infrastructure conversion\n- Registry System: Explore registry/ for template metadata and imports\n\nThis system provides the perfect balance of flexibility, reusability, and maintainability while preserving the core provisioning system's integrity.
