prvng_platform/orchestrator/docs/EXTENSION_LOADING.md

Extension Loading Guide

Overview

The extension loading module provides dynamic loading of providers, taskservs, and clusters through Nushell script integration.

Architecture

┌──────────────────┐
│   Orchestrator   │
│     (Rust)       │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ Extension Manager│
│                  │
│  - Caching       │
│  - Type safety   │
│  - Validation    │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│Extension Loader  │
│  (Nushell Call)  │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  Nushell Scripts │
│  (module load)   │
└──────────────────┘

Extension Types

1. Providers

Cloud provider implementations (AWS, UpCloud, Local):

let provider = extension_manager.load_extension(
    ExtensionType::Provider,
    "aws".to_string(),
    Some("2.0.0".to_string())
).await?;

2. Taskservs

Infrastructure service definitions (Kubernetes, PostgreSQL, etc.):

let taskserv = extension_manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    None  // Load latest version
).await?;

3. Clusters

Complete cluster configurations (Buildkit, CI/CD, etc.):

let cluster = extension_manager.load_extension(
    ExtensionType::Cluster,
    "buildkit".to_string(),
    Some("1.0.0".to_string())
).await?;

Features

LRU Caching

Extensions are cached using LRU (Least Recently Used) strategy:

• Cache size: 100 extensions
• Cache key: {type}:{name}:{version}
• Automatic eviction: Oldest entries removed when full

Type Safety

All extensions are strongly typed:

pub struct Extension {
    pub metadata: ExtensionMetadata,
    pub path: String,
    pub loaded_at: chrono::DateTime<chrono::Utc>,
}

pub struct ExtensionMetadata {
    pub name: String,
    pub version: String,
    pub description: String,
    pub extension_type: ExtensionType,
    pub dependencies: Vec<String>,
    pub author: Option<String>,
    pub repository: Option<String>,
}

Version Management

Load specific versions or use latest:

// Load specific version
let ext = extension_manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    Some("1.28.0".to_string())
).await?;

// Load latest version
let ext = extension_manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    None
).await?;

Configuration

Extension settings in config.defaults.toml:

[orchestrator.extensions]
auto_load = true
cache_dir = "{{orchestrator.paths.data_dir}}/extensions"

Configuration Options

• auto_load: Enable automatic extension loading (default: true)
• cache_dir: Directory for caching extension artifacts

API Endpoints

List Loaded Extensions

GET /api/v1/extensions/loaded

Response:

{
  "success": true,
  "data": [
    {
      "metadata": {
        "name": "kubernetes",
        "version": "1.28.0",
        "description": "Kubernetes container orchestrator",
        "extension_type": "Taskserv",
        "dependencies": ["containerd", "etcd"],
        "author": "provisioning-team",
        "repository": null
      },
      "path": "extensions/taskservs/kubernetes",
      "loaded_at": "2025-10-06T12:30:00Z"
    }
  ]
}

Reload Extension

POST /api/v1/extensions/reload
Content-Type: application/json

{
  "extension_type": "taskserv",
  "name": "kubernetes"
}

Response:

{
  "success": true,
  "data": "Extension kubernetes reloaded"
}

Usage Examples

Load Extension

use provisioning_orchestrator::extensions::{ExtensionManager, ExtensionType};

let manager = ExtensionManager::new(
    "/usr/local/bin/nu".to_string(),
    "/usr/local/bin/provisioning".to_string(),
);

let extension = manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    Some("1.28.0".to_string())
).await?;

println!("Loaded: {} v{}", extension.metadata.name, extension.metadata.version);

List Loaded Extensions

let extensions = manager.list_loaded_extensions().await;
for ext in extensions {
    println!("{} ({}) - loaded at {}",
        ext.metadata.name,
        ext.metadata.extension_type,
        ext.loaded_at
    );
}

Reload Extension

let extension = manager.reload_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string()
).await?;

Check if Loaded

let is_loaded = manager.is_extension_loaded(
    ExtensionType::Taskserv,
    "kubernetes"
).await;

if !is_loaded {
    // Load the extension
    manager.load_extension(
        ExtensionType::Taskserv,
        "kubernetes".to_string(),
        None
    ).await?;
}

Clear Cache

manager.clear_cache().await;

Integration with Workflows

Taskserv Installation Workflow

1. Load extension (before installation)
2. Validate dependencies
3. Generate configuration
4. Execute installation
5. Verify installation

// Step 1: Load extension
let extension = extension_manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    Some("1.28.0".to_string())
).await?;

// Step 2: Validate dependencies
for dep in &extension.metadata.dependencies {
    ensure_dependency_installed(dep).await?;
}

// Continue with installation...

Nushell Integration

The extension loader calls Nushell commands:

# Load taskserv extension
provisioning module load taskserv kubernetes --version 1.28.0

# Discover available extensions
provisioning module discover taskserv --output json

# Get extension metadata
provisioning module discover taskserv --name kubernetes --output json

Error Handling

The extension loader handles errors gracefully:

• Extension not found: Returns clear error message
• Version mismatch: Reports available versions
• Dependency errors: Lists missing dependencies
• Load failures: Logs detailed error information

Testing

Run extension loading tests:

cd provisioning/platform/orchestrator
cargo test test_extension_loading

Troubleshooting

Extension load fails

1. Check Nushell is installed and accessible
2. Verify extension exists in expected location
3. Check provisioning path configuration
4. Review orchestrator logs

Cache issues

1. Clear cache manually: manager.clear_cache().await
2. Check cache directory permissions
3. Verify disk space availability

Best Practices

1. Use versioning: Always specify version for production
2. Cache management: Clear cache periodically in dev environments
3. Dependency validation: Check dependencies before loading
4. Error handling: Always handle load failures gracefully

Security Considerations

1. Code execution: Extensions execute Nushell code
2. Validation: Verify extension metadata
3. Sandboxing: Consider sandboxed execution
4. Audit: Log all extension loading operations

Performance

Cache Hit Ratio

Monitor cache effectiveness:

let total_loads = metrics.total_extension_loads;
let cache_hits = metrics.cache_hits;
let hit_ratio = cache_hits as f64 / total_loads as f64;
println!("Cache hit ratio: {:.2}%", hit_ratio * 100.0);

Loading Time

Extension loading is optimized:

• Cached: < 1ms
• Cold load: 100-500ms (depends on extension size)
• With dependencies: Variable (depends on dependency count)

Future Enhancements

• Extension hot-reload without cache clear
• Dependency graph visualization
• Extension marketplace integration
• Automatic version updates
• Extension sandboxing