prvng_platform/crates/orchestrator/docs/EXTENSION_LOADING.md

Extension Loading Guide

Overview

The extension loading module provides dynamic loading of providers, taskservs, and clusters through Nushell script integration.

Architecture

┌──────────────────┐
│   Orchestrator   │
│     (Rust)       │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ Extension Manager│
│                  │
│  - Caching       │
│  - Type safety   │
│  - Validation    │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│Extension Loader  │
│  (Nushell Call)  │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  Nushell Scripts │
│  (module load)   │
└──────────────────┘
```plaintext

## Extension Types

### 1. Providers

Cloud provider implementations (AWS, UpCloud, Local):

```rust
let provider = extension_manager.load_extension(
    ExtensionType::Provider,
    "aws".to_string(),
    Some("2.0.0".to_string())
).await?;
```plaintext

### 2. Taskservs

Infrastructure service definitions (Kubernetes, PostgreSQL, etc.):

```rust
let taskserv = extension_manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    None  // Load latest version
).await?;
```plaintext

### 3. Clusters

Complete cluster configurations (Buildkit, CI/CD, etc.):

```rust
let cluster = extension_manager.load_extension(
    ExtensionType::Cluster,
    "buildkit".to_string(),
    Some("1.0.0".to_string())
).await?;
```plaintext

## Features

### LRU Caching

Extensions are cached using LRU (Least Recently Used) strategy:

- **Cache size**: 100 extensions
- **Cache key**: `{type}:{name}:{version}`
- **Automatic eviction**: Oldest entries removed when full

### Type Safety

All extensions are strongly typed:

```rust
pub struct Extension {
    pub metadata: ExtensionMetadata,
    pub path: String,
    pub loaded_at: chrono::DateTime<chrono::Utc>,
}

pub struct ExtensionMetadata {
    pub name: String,
    pub version: String,
    pub description: String,
    pub extension_type: ExtensionType,
    pub dependencies: Vec<String>,
    pub author: Option<String>,
    pub repository: Option<String>,
}
```plaintext

### Version Management

Load specific versions or use latest:

```rust
// Load specific version
let ext = extension_manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    Some("1.28.0".to_string())
).await?;

// Load latest version
let ext = extension_manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    None
).await?;
```plaintext

## Configuration

Extension settings in `config.defaults.toml`:

```toml
[orchestrator.extensions]
auto_load = true
cache_dir = "{{orchestrator.paths.data_dir}}/extensions"
```plaintext

### Configuration Options

- **auto_load**: Enable automatic extension loading (default: true)
- **cache_dir**: Directory for caching extension artifacts

## API Endpoints

### List Loaded Extensions

```http
GET /api/v1/extensions/loaded
```plaintext

**Response:**

```json
{
  "success": true,
  "data": [
    {
      "metadata": {
        "name": "kubernetes",
        "version": "1.28.0",
        "description": "Kubernetes container orchestrator",
        "extension_type": "Taskserv",
        "dependencies": ["containerd", "etcd"],
        "author": "provisioning-team",
        "repository": null
      },
      "path": "extensions/taskservs/kubernetes",
      "loaded_at": "2025-10-06T12:30:00Z"
    }
  ]
}
```plaintext

### Reload Extension

```http
POST /api/v1/extensions/reload
Content-Type: application/json

{
  "extension_type": "taskserv",
  "name": "kubernetes"
}
```plaintext

**Response:**

```json
{
  "success": true,
  "data": "Extension kubernetes reloaded"
}
```plaintext

## Usage Examples

### Load Extension

```rust
use provisioning_orchestrator::extensions::{ExtensionManager, ExtensionType};

let manager = ExtensionManager::new(
    "/usr/local/bin/nu".to_string(),
    "/usr/local/bin/provisioning".to_string(),
);

let extension = manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    Some("1.28.0".to_string())
).await?;

println!("Loaded: {} v{}", extension.metadata.name, extension.metadata.version);
```plaintext

### List Loaded Extensions

```rust
let extensions = manager.list_loaded_extensions().await;
for ext in extensions {
    println!("{} ({}) - loaded at {}",
        ext.metadata.name,
        ext.metadata.extension_type,
        ext.loaded_at
    );
}
```plaintext

### Reload Extension

```rust
let extension = manager.reload_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string()
).await?;
```plaintext

### Check if Loaded

```rust
let is_loaded = manager.is_extension_loaded(
    ExtensionType::Taskserv,
    "kubernetes"
).await;

if !is_loaded {
    // Load the extension
    manager.load_extension(
        ExtensionType::Taskserv,
        "kubernetes".to_string(),
        None
    ).await?;
}
```plaintext

### Clear Cache

```rust
manager.clear_cache().await;
```plaintext

## Integration with Workflows

### Taskserv Installation Workflow

1. **Load extension** (before installation)
2. Validate dependencies
3. Generate configuration
4. Execute installation
5. Verify installation

```rust
// Step 1: Load extension
let extension = extension_manager.load_extension(
    ExtensionType::Taskserv,
    "kubernetes".to_string(),
    Some("1.28.0".to_string())
).await?;

// Step 2: Validate dependencies
for dep in &extension.metadata.dependencies {
    ensure_dependency_installed(dep).await?;
}

// Continue with installation...
```plaintext

## Nushell Integration

The extension loader calls Nushell commands:

```bash
# Load taskserv extension
provisioning module load taskserv kubernetes --version 1.28.0

# Discover available extensions
provisioning module discover taskserv --output json

# Get extension metadata
provisioning module discover taskserv --name kubernetes --output json
```plaintext

## Error Handling

The extension loader handles errors gracefully:

- **Extension not found**: Returns clear error message
- **Version mismatch**: Reports available versions
- **Dependency errors**: Lists missing dependencies
- **Load failures**: Logs detailed error information

## Testing

Run extension loading tests:

```bash
cd provisioning/platform/orchestrator
cargo test test_extension_loading
```plaintext

## Troubleshooting

### Extension load fails

1. Check Nushell is installed and accessible
2. Verify extension exists in expected location
3. Check provisioning path configuration
4. Review orchestrator logs

### Cache issues

1. Clear cache manually: `manager.clear_cache().await`
2. Check cache directory permissions
3. Verify disk space availability

## Best Practices

1. **Use versioning**: Always specify version for production
2. **Cache management**: Clear cache periodically in dev environments
3. **Dependency validation**: Check dependencies before loading
4. **Error handling**: Always handle load failures gracefully

## Security Considerations

1. **Code execution**: Extensions execute Nushell code
2. **Validation**: Verify extension metadata
3. **Sandboxing**: Consider sandboxed execution
4. **Audit**: Log all extension loading operations

## Performance

### Cache Hit Ratio

Monitor cache effectiveness:

```rust
let total_loads = metrics.total_extension_loads;
let cache_hits = metrics.cache_hits;
let hit_ratio = cache_hits as f64 / total_loads as f64;
println!("Cache hit ratio: {:.2}%", hit_ratio * 100.0);
```plaintext

### Loading Time

Extension loading is optimized:

- **Cached**: < 1ms
- **Cold load**: 100-500ms (depends on extension size)
- **With dependencies**: Variable (depends on dependency count)

## Future Enhancements

- [ ] Extension hot-reload without cache clear
- [ ] Dependency graph visualization
- [ ] Extension marketplace integration
- [ ] Automatic version updates
- [ ] Extension sandboxing