prvng_platform/crates/orchestrator/docs/extension-loading.md

8.5 KiB

Extension Loading Guide\n\n## Overview\n\nThe extension loading module provides dynamic loading of providers, taskservs, and clusters through Nushell script integration.\n\n## Architecture\n\n{$detected_lang}\n┌──────────────────┐\n│ Orchestrator │\n│ (Rust) │\n└────────┬─────────┘\n │\n ▼\n┌──────────────────┐\n│ Extension Manager│\n│ │\n│ - Caching │\n│ - Type safety │\n│ - Validation │\n└────────┬─────────┘\n │\n ▼\n┌──────────────────┐\n│Extension Loader │\n│ (Nushell Call) │\n└────────┬─────────┘\n │\n ▼\n┌──────────────────┐\n│ Nushell Scripts │\n│ (module load) │\n└──────────────────┘\n\n\n## Extension Types\n\n### 1. Providers\n\nCloud provider implementations (AWS, UpCloud, Local):\n\n{$detected_lang}\nlet provider = extension_manager.load_extension(\n ExtensionType::Provider,\n "aws".to_string(),\n Some("2.0.0".to_string())\n).await?;\n\n\n### 2. Taskservs\n\nInfrastructure service definitions (Kubernetes, PostgreSQL, etc.):\n\n{$detected_lang}\nlet taskserv = extension_manager.load_extension(\n ExtensionType::Taskserv,\n "kubernetes".to_string(),\n None // Load latest version\n).await?;\n\n\n### 3. Clusters\n\nComplete cluster configurations (Buildkit, CI/CD, etc.):\n\n{$detected_lang}\nlet cluster = extension_manager.load_extension(\n ExtensionType::Cluster,\n "buildkit".to_string(),\n Some("1.0.0".to_string())\n).await?;\n\n\n## Features\n\n### LRU Caching\n\nExtensions are cached using LRU (Least Recently Used) strategy:\n\n- Cache size: 100 extensions\n- Cache key: {type}:{name}:{version}\n- Automatic eviction: Oldest entries removed when full\n\n### Type Safety\n\nAll extensions are strongly typed:\n\n{$detected_lang}\npub struct Extension {\n pub metadata: ExtensionMetadata,\n pub path: String,\n pub loaded_at: chrono::DateTime<chrono::Utc>,\n}\n\npub struct ExtensionMetadata {\n pub name: String,\n pub version: String,\n pub description: String,\n pub extension_type: ExtensionType,\n pub dependencies: Vec<String>,\n pub author: Option<String>,\n pub repository: Option<String>,\n}\n\n\n### Version Management\n\nLoad specific versions or use latest:\n\n{$detected_lang}\n// Load specific version\nlet ext = extension_manager.load_extension(\n ExtensionType::Taskserv,\n "kubernetes".to_string(),\n Some("1.28.0".to_string())\n).await?;\n\n// Load latest version\nlet ext = extension_manager.load_extension(\n ExtensionType::Taskserv,\n "kubernetes".to_string(),\n None\n).await?;\n\n\n## Configuration\n\nExtension settings in config.defaults.toml:\n\n{$detected_lang}\n[orchestrator.extensions]\nauto_load = true\ncache_dir = "{{orchestrator.paths.data_dir}}/extensions"\n\n\n### Configuration Options\n\n- auto_load: Enable automatic extension loading (default: true)\n- cache_dir: Directory for caching extension artifacts\n\n## API Endpoints\n\n### List Loaded Extensions\n\n{$detected_lang}\nGET /api/v1/extensions/loaded\n\n\nResponse:\n\n{$detected_lang}\n{\n "success": true,\n "data": [\n {\n "metadata": {\n "name": "kubernetes",\n "version": "1.28.0",\n "description": "Kubernetes container orchestrator",\n "extension_type": "Taskserv",\n "dependencies": ["containerd", "etcd"],\n "author": "provisioning-team",\n "repository": null\n },\n "path": "extensions/taskservs/kubernetes",\n "loaded_at": "2025-10-06T12:30:00Z"\n }\n ]\n}\n\n\n### Reload Extension\n\n{$detected_lang}\nPOST /api/v1/extensions/reload\nContent-Type: application/json\n\n{\n "extension_type": "taskserv",\n "name": "kubernetes"\n}\n\n\nResponse:\n\n{$detected_lang}\n{\n "success": true,\n "data": "Extension kubernetes reloaded"\n}\n\n\n## Usage Examples\n\n### Load Extension\n\n{$detected_lang}\nuse provisioning_orchestrator::extensions::{ExtensionManager, ExtensionType};\n\nlet manager = ExtensionManager::new(\n "/usr/local/bin/nu".to_string(),\n "/usr/local/bin/provisioning".to_string(),\n);\n\nlet extension = manager.load_extension(\n ExtensionType::Taskserv,\n "kubernetes".to_string(),\n Some("1.28.0".to_string())\n).await?;\n\nprintln!("Loaded: {} v{}", extension.metadata.name, extension.metadata.version);\n\n\n### List Loaded Extensions\n\n{$detected_lang}\nlet extensions = manager.list_loaded_extensions().await;\nfor ext in extensions {\n println!("{} ({}) - loaded at {}",\n ext.metadata.name,\n ext.metadata.extension_type,\n ext.loaded_at\n );\n}\n\n\n### Reload Extension\n\n{$detected_lang}\nlet extension = manager.reload_extension(\n ExtensionType::Taskserv,\n "kubernetes".to_string()\n).await?;\n\n\n### Check if Loaded\n\n{$detected_lang}\nlet is_loaded = manager.is_extension_loaded(\n ExtensionType::Taskserv,\n "kubernetes"\n).await;\n\nif !is_loaded {\n // Load the extension\n manager.load_extension(\n ExtensionType::Taskserv,\n "kubernetes".to_string(),\n None\n ).await?;\n}\n\n\n### Clear Cache\n\n{$detected_lang}\nmanager.clear_cache().await;\n\n\n## Integration with Workflows\n\n### Taskserv Installation Workflow\n\n1. Load extension (before installation)\n2. Validate dependencies\n3. Generate configuration\n4. Execute installation\n5. Verify installation\n\n{$detected_lang}\n// Step 1: Load extension\nlet extension = extension_manager.load_extension(\n ExtensionType::Taskserv,\n "kubernetes".to_string(),\n Some("1.28.0".to_string())\n).await?;\n\n// Step 2: Validate dependencies\nfor dep in &extension.metadata.dependencies {\n ensure_dependency_installed(dep).await?;\n}\n\n// Continue with installation...\n\n\n## Nushell Integration\n\nThe extension loader calls Nushell commands:\n\n{$detected_lang}\n# Load taskserv extension\nprovisioning module load taskserv kubernetes --version 1.28.0\n\n# Discover available extensions\nprovisioning module discover taskserv --output json\n\n# Get extension metadata\nprovisioning module discover taskserv --name kubernetes --output json\n\n\n## Error Handling\n\nThe extension loader handles errors gracefully:\n\n- Extension not found: Returns clear error message\n- Version mismatch: Reports available versions\n- Dependency errors: Lists missing dependencies\n- Load failures: Logs detailed error information\n\n## Testing\n\nRun extension loading tests:\n\n{$detected_lang}\ncd provisioning/platform/orchestrator\ncargo test test_extension_loading\n\n\n## Troubleshooting\n\n### Extension load fails\n\n1. Check Nushell is installed and accessible\n2. Verify extension exists in expected location\n3. Check provisioning path configuration\n4. Review orchestrator logs\n\n### Cache issues\n\n1. Clear cache manually: manager.clear_cache().await\n2. Check cache directory permissions\n3. Verify disk space availability\n\n## Best Practices\n\n1. Use versioning: Always specify version for production\n2. Cache management: Clear cache periodically in dev environments\n3. Dependency validation: Check dependencies before loading\n4. Error handling: Always handle load failures gracefully\n\n## Security Considerations\n\n1. Code execution: Extensions execute Nushell code\n2. Validation: Verify extension metadata\n3. Sandboxing: Consider sandboxed execution\n4. Audit: Log all extension loading operations\n\n## Performance\n\n### Cache Hit Ratio\n\nMonitor cache effectiveness:\n\n{$detected_lang}\nlet total_loads = metrics.total_extension_loads;\nlet cache_hits = metrics.cache_hits;\nlet hit_ratio = cache_hits as f64 / total_loads as f64;\nprintln!("Cache hit ratio: {:.2}%", hit_ratio * 100.0);\n\n\n### Loading Time\n\nExtension loading is optimized:\n\n- Cached: < 1ms\n- Cold load: 100-500ms (depends on extension size)\n- With dependencies: Variable (depends on dependency count)\n\n## Future Enhancements\n\n- [ ] Extension hot-reload without cache clear\n- [ ] Dependency graph visualization\n- [ ] Extension marketplace integration\n- [ ] Automatic version updates\n- [ ] Extension sandboxing