Provider-Agnostic Architecture Documentation

Overview

The new provider-agnostic architecture eliminates hardcoded provider dependencies and enables true multi-provider infrastructure deployments. This addresses two critical limitations of the previous middleware:

Hardcoded provider dependencies - No longer requires importing specific provider modules
Single-provider limitation - Now supports mixing multiple providers in the same deployment (e.g., AWS compute + Cloudflare DNS + UpCloud backup)

Architecture Components

1. Provider Interface (`interface.nu`)

Defines the contract that all providers must implement:

# Standard interface functions
- query_servers
- server_info
- server_exists
- create_server
- delete_server
- server_state
- get_ip
# ... and 20+ other functions

Key Features:

Type-safe function signatures
Comprehensive validation
Provider capability flags
Interface versioning

2. Provider Registry (`registry.nu`)

Manages provider discovery and registration:

# Initialize registry
init-provider-registry

# List available providers
list-providers --available-only

# Check provider availability
is-provider-available "aws"

Features:

Automatic provider discovery
Core and extension provider support
Caching for performance
Provider capability tracking

3. Provider Loader (`loader.nu`)

Handles dynamic provider loading and validation:

# Load provider dynamically
load-provider "aws"

# Get provider with auto-loading
get-provider "upcloud"

# Call provider function
call-provider-function "aws" "query_servers" $find $cols

Features:

Lazy loading (load only when needed)
Interface compliance validation
Error handling and recovery
Provider health checking

4. Provider Adapters

Each provider implements a standard adapter:

provisioning/extensions/providers/
├── aws/provider.nu        # AWS adapter
├── upcloud/provider.nu    # UpCloud adapter
├── local/provider.nu      # Local adapter
└── {custom}/provider.nu   # Custom providers

Adapter Structure:

# AWS Provider Adapter
export def query_servers [find?: string, cols?: string] {
    aws_query_servers $find $cols
}

export def create_server [settings: record, server: record, check: bool, wait: bool] {
    # AWS-specific implementation
}

5. Provider-Agnostic Middleware (`middleware_provider_agnostic.nu`)

The new middleware that uses dynamic dispatch:

# No hardcoded imports!
export def mw_query_servers [settings: record, find?: string, cols?: string] {
    $settings.data.servers | each { |server|
        # Dynamic provider loading and dispatch
        dispatch_provider_function $server.provider "query_servers" $find $cols
    }
}

Multi-Provider Support

Example: Mixed Provider Infrastructure

servers = [
    aws.Server {
        hostname = "compute-01"
        provider = "aws"
        # AWS-specific config
    }
    upcloud.Server {
        hostname = "backup-01"
        provider = "upcloud"
        # UpCloud-specific config
    }
    cloudflare.DNS {
        hostname = "api.example.com"
        provider = "cloudflare"
        # DNS-specific config
    }
]

Multi-Provider Deployment

# Deploy across multiple providers automatically
mw_deploy_multi_provider_infra $settings $deployment_plan

# Get deployment strategy recommendations
mw_suggest_deployment_strategy {
    regions: ["us-east-1", "eu-west-1"]
    high_availability: true
    cost_optimization: true
}

Provider Capabilities

Providers declare their capabilities:

capabilities: {
    server_management: true
    network_management: true
    auto_scaling: true        # AWS: yes, Local: no
    multi_region: true        # AWS: yes, Local: no
    serverless: true          # AWS: yes, UpCloud: no
    compliance_certifications: ["SOC2", "HIPAA"]
}

Migration Guide

From Old Middleware

Before (hardcoded):

# middleware.nu
use ../aws/nulib/aws/servers.nu *
use ../upcloud/nulib/upcloud/servers.nu *

match $server.provider {
    "aws" => { aws_query_servers $find $cols }
    "upcloud" => { upcloud_query_servers $find $cols }
}

After (provider-agnostic):

# middleware_provider_agnostic.nu
# No hardcoded imports!

# Dynamic dispatch
dispatch_provider_function $server.provider "query_servers" $find $cols

Migration Steps

Replace middleware file:

cp provisioning/extensions/providers/prov_lib/middleware.nu \
   provisioning/extensions/providers/prov_lib/middleware_legacy.backup

cp provisioning/extensions/providers/prov_lib/middleware_provider_agnostic.nu \
   provisioning/extensions/providers/prov_lib/middleware.nu

Test with existing infrastructure:

./provisioning/tools/test-provider-agnostic.nu run-all-tests

Update any custom code that directly imported provider modules

Adding New Providers

1. Create Provider Adapter

Create provisioning/extensions/providers/{name}/provider.nu:

# Digital Ocean Provider Example
export def get-provider-metadata [] {
    {
        name: "digitalocean"
        version: "1.0.0"
        capabilities: {
            server_management: true
            # ... other capabilities
        }
    }
}

# Implement required interface functions
export def query_servers [find?: string, cols?: string] {
    # DigitalOcean-specific implementation
}

export def create_server [settings: record, server: record, check: bool, wait: bool] {
    # DigitalOcean-specific implementation
}

# ... implement all required functions

2. Provider Discovery

The registry will automatically discover the new provider on next initialization.

3. Test New Provider

# Check if discovered
is-provider-available "digitalocean"

# Load and test
load-provider "digitalocean"
check-provider-health "digitalocean"

Best Practices

Provider Development

Implement full interface - All functions must be implemented
Handle errors gracefully - Return appropriate error values
Follow naming conventions - Use consistent function naming
Document capabilities - Accurately declare what your provider supports
Test thoroughly - Validate against the interface specification

Multi-Provider Deployments

Use capability-based selection - Choose providers based on required features
Handle provider failures - Design for provider unavailability
Optimize for cost/performance - Mix providers strategically
Monitor cross-provider dependencies - Understand inter-provider communication

Profile-Based Security

# Environment profiles can restrict providers
PROVISIONING_PROFILE=production  # Only allows certified providers
PROVISIONING_PROFILE=development # Allows all providers including local

Troubleshooting

Common Issues

Provider not found
- Check provider is in correct directory
- Verify provider.nu exists and implements interface
- Run init-provider-registry to refresh
Interface validation failed
- Use validate-provider-interface to check compliance
- Ensure all required functions are implemented
- Check function signatures match interface
Provider loading errors
- Check Nushell module syntax
- Verify import paths are correct
- Use check-provider-health for diagnostics

Debug Commands

# Registry diagnostics
get-provider-stats
list-providers --verbose

# Provider diagnostics
check-provider-health "aws"
check-all-providers-health

# Loader diagnostics
get-loader-stats

Performance Benefits

Lazy Loading - Providers loaded only when needed
Caching - Provider registry cached to disk
Reduced Memory - No hardcoded imports reducing memory usage
Parallel Operations - Multi-provider operations can run in parallel

Future Enhancements

Provider Plugins - Support for external provider plugins
Provider Versioning - Multiple versions of same provider
Provider Composition - Compose providers for complex scenarios
Provider Marketplace - Community provider sharing

API Reference

See the interface specification for complete function documentation:

get-provider-interface-docs | table

This returns the complete API with signatures and descriptions for all provider interface functions.

Keyboard shortcuts

Provisioning Platform Documentation