2 lines
9.1 KiB
Markdown
2 lines
9.1 KiB
Markdown
# Layered Template Architecture System\n\nThis workspace provides a combined **Layered Extension Architecture with Override System** and **Template-Based Infrastructure Pattern Library** that maintains PAP principles while enabling maximum reusability of infrastructure configurations.\n\n## 🏗️ Architecture Overview\n\n### Layer Hierarchy\n\nThe system resolves configurations through a three-tier layer system:\n\n1. **Core Layer (Priority 100)** - `provisioning/extensions/`\n - Base provisioning system extensions\n - Core taskservs, providers, and clusters\n\n2. **Workspace Layer (Priority 200)** - `provisioning/workspace/templates/`\n - Shared templates extracted from proven infrastructure patterns\n - Reusable configurations across multiple infrastructures\n\n3. **Infrastructure Layer (Priority 300)** - `workspace/infra/{name}/`\n - Infrastructure-specific configurations and overrides\n - Custom implementations per infrastructure\n\n### Directory Structure\n\n```\nprovisioning/workspace/\n├── templates/ # Template library\n│ ├── taskservs/ # Taskserv configuration templates\n│ │ ├── kubernetes/ # Kubernetes templates\n│ │ │ ├── base.k # Base configuration\n│ │ │ └── variants/ # HA, single-node variants\n│ │ ├── storage/ # Storage system templates\n│ │ ├── networking/ # Network configuration templates\n│ │ └── container-runtime/ # Container runtime templates\n│ ├── providers/ # Provider templates\n│ │ ├── upcloud/ # UpCloud provider templates\n│ │ └── aws/ # AWS provider templates\n│ ├── servers/ # Server configuration patterns\n│ └── clusters/ # Complete cluster templates\n├── layers/ # Layer definitions\n│ ├── core.layer.k # Core layer definition\n│ ├── workspace.layer.k # Workspace layer definition\n│ └── infra.layer.k # Infrastructure layer definition\n├── registry/ # Extension registry\n│ ├── manifest.yaml # Template catalog and metadata\n│ └── imports.k # Central import aliases\n├── templates/lib/ # Composition utilities\n│ ├── compose.k # KCL composition functions\n│ └── override.k # Override and layer utilities\n└── tools/ # Migration and management tools\n └── migrate-infra.nu # Infrastructure migration tool\n```\n\n## 🚀 Getting Started\n\n### 1. Extract Existing Patterns\n\nExtract patterns from existing infrastructure (e.g., wuji) to create reusable templates:\n\n```\n# Extract all patterns from wuji infrastructure\ncd provisioning/workspace/tools\n./migrate-infra.nu extract wuji\n\n# Extract specific types only\n./migrate-infra.nu extract wuji --type taskservs\n```\n\n### 2. Use Enhanced Module Loader\n\nThe enhanced module loader provides template and layer management:\n\n```\n# List available templates\nprovisioning/core/cli/module-loader-enhanced template list\n\n# Show layer resolution order\nprovisioning/core/cli/module-loader-enhanced layer show\n\n# Test layer resolution for a specific module\nprovisioning/core/cli/module-loader-enhanced layer test kubernetes --infra wuji\n```\n\n### 3. Apply Templates to New Infrastructure\n\n```\n# Apply kubernetes template to new infrastructure\nprovisioning/core/cli/module-loader-enhanced template apply kubernetes-base new-infra --provider upcloud\n\n# Load taskservs using templates\nprovisioning/core/cli/module-loader-enhanced load enhanced taskservs workspace/infra/new-infra [kubernetes, cilium] --layer workspace\n```\n\n## 📋 Usage Examples\n\n### Creating a New Infrastructure from Templates\n\n```\n# 1. Create directory structure\nmkdir -p workspace/infra/my-new-infra/{taskservs,defs,overrides}\n\n# 2. Apply base templates\ncd provisioning\n./core/cli/module-loader-enhanced template apply kubernetes-base my-new-infra\n\n# 3. Customize for your needs\n# Edit workspace/infra/my-new-infra/taskservs/kubernetes.k\n\n# 4. Test layer resolution\n./core/cli/module-loader-enhanced layer test kubernetes --infra my-new-infra\n```\n\n### Converting Existing Infrastructure\n\n```\n# 1. Extract patterns to templates\ncd provisioning/workspace/tools\n./migrate-infra.nu extract existing-infra\n\n# 2. Convert infrastructure to use templates\n./migrate-infra.nu convert existing-infra\n\n# 3. Validate conversion\n./migrate-infra.nu validate existing-infra\n```\n\n### Template Development\n\n```\n# Create a new taskserv template\n# provisioning/workspace/templates/taskservs/my-service/base.k\n\nimport taskservs.my_service.kcl.my_service as service_core\nimport ../../../workspace/templates/lib/compose as comp\n\nschema MyServiceBase {\n version: str = "1.0.0"\n cluster_name: str\n # ... configuration options\n}\n\ndef create_my_service [cluster_name: str, overrides: any = {}] -> any {\n let base_config = MyServiceBase { cluster_name = $cluster_name }\n let final_config = comp.deep_merge $base_config $overrides\n\n service_core.MyService $final_config\n}\n```\n\n## 🔧 Configuration Composition\n\n### Using Templates in Infrastructure\n\n```\n# workspace/infra/my-infra/taskservs/kubernetes.k\n\nimport provisioning.workspace.registry.imports as reg\nimport provisioning.workspace.templates.lib.override as ovr\n\n# Use base template with infrastructure-specific overrides\n_taskserv = ovr.infrastructure_overrides.taskserv_override(\n reg.tpl_kubernetes_base.kubernetes_base,\n "my-infra",\n "kubernetes",\n {\n cluster_name: "my-infra"\n version: "1.31.0" # Override version\n cri: "crio" # Override container runtime\n # Custom network configuration\n network_config: {\n pod_cidr: "10.244.0.0/16"\n service_cidr: "10.96.0.0/12"\n }\n }\n)\n```\n\n### Layer Composition\n\n```\n# Compose configuration through all layers\nimport provisioning.workspace.templates.lib.compose as comp\n\n# Manual layer composition\nfinal_config = comp.compose_templates(\n $core_config, # From provisioning/extensions\n $workspace_config, # From provisioning/workspace/templates\n $infra_config # From workspace/infra/{name}\n)\n```\n\n## 🛠️ Advanced Features\n\n### Provider-Aware Composition\n\n```\nimport provisioning.workspace.templates.lib.override as ovr\n\n# Apply provider-specific configurations\nconfig = ovr.override_patterns.env_override(\n $base_config,\n "upcloud",\n {\n upcloud: { zone: "es-mad1", plan: "2xCPU-4GB" },\n aws: { region: "us-west-2", instance_type: "t3.medium" },\n local: { memory: "4GB", cpus: 2 }\n }\n)\n```\n\n### Conditional Overrides\n\n```\n# Infrastructure-specific conditional overrides\nconfig = ovr.layer_resolution.infra_conditional(\n $base_config,\n $infra_name,\n {\n "production": { ha: true, replicas: 3 },\n "development": { ha: false, replicas: 1 },\n "default": { ha: false, replicas: 1 }\n }\n)\n```\n\n## 📚 Benefits\n\n### ✅ Maintains PAP Principles\n\n- **Configuration-driven**: No hardcoded values\n- **Modular**: Clear separation of concerns\n- **Declarative**: Infrastructure as code\n- **Reusable**: DRY principle throughout\n\n### ✅ Flexible Override System\n\n- **Layer-based resolution**: Clean precedence order\n- **Selective overrides**: Override only what's needed\n- **Provider-agnostic**: Works across all providers\n- **Environment-aware**: Dev/test/prod configurations\n\n### ✅ Template Reusability\n\n- **Pattern extraction**: Learn from existing infrastructures\n- **Template versioning**: Track evolution over time\n- **Composition utilities**: Rich KCL composition functions\n- **Migration tools**: Easy conversion process\n\n### ✅ No Core Changes\n\n- **Non-invasive**: Core provisioning system unchanged\n- **Backward compatible**: Existing infrastructures continue working\n- **Progressive adoption**: Migrate at your own pace\n- **Extensible**: Add new templates and layers easily\n\n## 🔄 Migration Path\n\n1. **Extract patterns** from existing infrastructures using `migrate-infra.nu extract`\n2. **Create templates** in `provisioning/workspace/templates/`\n3. **Convert infrastructures** to use templates with `migrate-infra.nu convert`\n4. **Validate** the conversion with `migrate-infra.nu validate`\n5. **Test** layer resolution with enhanced module loader\n6. **Iterate** and improve templates based on usage\n\n## 📖 Further Reading\n\n- **Layer Definitions**: See `layers/*.layer.k` for layer configuration\n- **Template Examples**: Browse `templates/` for real-world patterns\n- **Composition Utilities**: Check `templates/lib/` for KCL utilities\n- **Migration Tools**: Use `tools/migrate-infra.nu` for infrastructure conversion\n- **Registry System**: Explore `registry/` for template metadata and imports\n\nThis system provides the perfect balance of flexibility, reusability, and maintainability while preserving the core provisioning system's integrity.
|