# Custom Provider Development

Develop custom cloud providers and infrastructure adapters for the Provisioning platform.

## Overview

A provider is an extension that interfaces with a cloud platform or infrastructure system.
Providers enable Provisioning to manage resources across AWS, UpCloud, Hetzner, Kubernetes,
on-premise systems, or custom platforms.

## Provider Framework

```text
┌──────────────────┐
│  Your Provider   │
├──────────────────┤
│ Operations:      │
│ - create         │
│ - delete         │
│ - list           │
│ - describe       │
├──────────────────┤
│ Features:        │
│ - Async Rust     │
│ - Error handling │
│ - State tracking │
├──────────────────┤
│ Output:          │
│ - Resources      │
│ - Metrics        │
│ - Events         │
└──────────────────┘
```

## Minimal Provider Template

```rust
// custom-providers/my-provider/src/lib.rs
use async_trait::async_trait;
use provisioning_sdk::{Provider, ProviderConfig, Resource, Error as ProviderError};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use tokio::sync::RwLock;
use std::sync::Arc;

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MyProviderConfig {
    pub region: String,
    pub api_endpoint: String,
    pub credentials_source: String,  // env, file, iam, etc.
}

pub struct MyProvider {
    config: MyProviderConfig,
    state: Arc<RwLock<ProviderState>>,
}

#[derive(Debug, Clone, Default)]
struct ProviderState {
    resources: HashMap<String, Resource>,
    operations_count: u64,
}

impl MyProvider {
    pub fn new(config: MyProviderConfig) -> Self {
        Self {
            config,
            state: Arc::new(RwLock::new(ProviderState::default())),
        }
    }

    async fn load_credentials(&self) -> Result<String, ProviderError> {
        match self.config.credentials_source.as_str() {
            "env" => {
                std::env::var("MY_PROVIDER_TOKEN")
                    .map_err| ( | e ProviderError::AuthenticationFailed(e.to_string()))
            }
            "iam" => {
                // Load from IAM role metadata
                self.load_iam_credentials().await
            }
            "file" => {
                // Load from credentials file
                self.load_credentials_file().await
            }
            _ => Err(ProviderError::AuthenticationFailed("Unknown credentials source".to_string())),
        }
    }

    async fn load_iam_credentials(&self) -> Result<String, ProviderError> {
        // Implementation would call IAM metadata service
        // This is a placeholder showing the pattern
        Ok("iam-token".to_string())
    }

    async fn load_credentials_file(&self) -> Result<String, ProviderError> {
        // Implementation would read from ~/.credentials/my-provider
        // This is a placeholder showing the pattern
        Ok("file-token".to_string())
    }
}

#[async_trait]
impl Provider for MyProvider {
    fn name(&self) -> &'static str {
        "my-provider"
    }

    fn version(&self) -> &'static str {
        "1.0.0"
    }

    fn capabilities(&self) -> Vec<String> {
        vec![
            "compute".to_string(),
            "storage".to_string(),
            "networking".to_string(),
        ]
    }

    async fn create(
        &self,
        resource_type: &str,
        config: serde_json::Value,
    ) -> Result<Resource, ProviderError> {
        let credentials = self.load_credentials().await?;

        match resource_type {
            "instance" => self.create_instance(&credentials, config).await,
            "storage" => self.create_storage(&credentials, config).await,
            "network" => self.create_network(&credentials, config).await,
            _ => Err(ProviderError::UnsupportedResourceType(
                resource_type.to_string(),
            )),
        }
    }

    async fn delete(&self, resource_type: &str, resource_id: &str)
        -> Result<(), ProviderError> {
        let credentials = self.load_credentials().await?;

        match resource_type {
            "instance" => self.delete_instance(&credentials, resource_id).await,
            "storage" => self.delete_storage(&credentials, resource_id).await,
            "network" => self.delete_network(&credentials, resource_id).await,
            _ => Err(ProviderError::UnsupportedResourceType(
                resource_type.to_string(),
            )),
        }
    }

    async fn list(&self, resource_type: &str) -> Result<Vec<Resource>, ProviderError> {
        let credentials = self.load_credentials().await?;

        match resource_type {
            "instance" => self.list_instances(&credentials).await,
            "storage" => self.list_storage(&credentials).await,
            "network" => self.list_networks(&credentials).await,
            _ => Err(ProviderError::UnsupportedResourceType(
                resource_type.to_string(),
            )),
        }
    }

    async fn describe(
        &self,
        resource_type: &str,
        resource_id: &str,
    ) -> Result<Resource, ProviderError> {
        let credentials = self.load_credentials().await?;

        match resource_type {
            "instance" => self.describe_instance(&credentials, resource_id).await,
            "storage" => self.describe_storage(&credentials, resource_id).await,
            "network" => self.describe_network(&credentials, resource_id).await,
            _ => Err(ProviderError::UnsupportedResourceType(
                resource_type.to_string(),
            )),
        }
    }
}

impl MyProvider {
    async fn create_instance(
        &self,
        _credentials: &str,
        config: serde_json::Value,
    ) -> Result<Resource, ProviderError> {
        let name = config.get("name")
            .and_then| ( | v v.as_str())
            .ok_or_else| ( | ProviderError::InvalidConfig("name required".to_string()))?;

        let size = config.get("size")
            .and_then| ( | v v.as_str())
            .ok_or_else| ( | ProviderError::InvalidConfig("size required".to_string()))?;

        let instance_id = format!("inst-{}", uuid::Uuid::new_v4());

        let resource = Resource {
            id: instance_id.clone(),
            resource_type: "instance".to_string(),
            name: name.to_string(),
            status: "running".to_string(),
            metadata: serde_json::json!({
                "size": size,
                "region": self.config.region,
            }),
        };

        let mut state = self.state.write().await;
        state.resources.insert(instance_id.clone(), resource.clone());
        state.operations_count += 1;

        Ok(resource)
    }

    async fn list_instances(&self, _credentials: &str) -> Result<Vec<Resource>, ProviderError> {
        let state = self.state.read().await;
        let instances: Vec<Resource> = state.resources
            .values()
            .filter| ( | r r.resource_type == "instance")
            .cloned()
            .collect();
        Ok(instances)
    }

    async fn describe_instance(
        &self,
        _credentials: &str,
        resource_id: &str,
    ) -> Result<Resource, ProviderError> {
        let state = self.state.read().await;
        state.resources
            .get(resource_id)
            .cloned()
            .ok_or_else| ( | ProviderError::ResourceNotFound(resource_id.to_string()))
    }

    async fn delete_instance(&self, _credentials: &str, resource_id: &str)
        -> Result<(), ProviderError> {
        let mut state = self.state.write().await;
        state.resources
            .remove(resource_id)
            .ok_or_else| ( | ProviderError::ResourceNotFound(resource_id.to_string()))?;
        state.operations_count += 1;
        Ok(())
    }

    async fn create_storage(
        &self,
        _credentials: &str,
        config: serde_json::Value,
    ) -> Result<Resource, ProviderError> {
        let name = config.get("name")
            .and_then| ( | v v.as_str())
            .ok_or_else| ( | ProviderError::InvalidConfig("name required".to_string()))?;

        let bucket_id = format!("bucket-{}", uuid::Uuid::new_v4());
        let resource = Resource {
            id: bucket_id.clone(),
            resource_type: "storage".to_string(),
            name: name.to_string(),
            status: "available".to_string(),
            metadata: serde_json::json!({
                "region": self.config.region,
            }),
        };

        let mut state = self.state.write().await;
        state.resources.insert(bucket_id.clone(), resource.clone());
        state.operations_count += 1;

        Ok(resource)
    }

    async fn list_storage(&self, _credentials: &str) -> Result<Vec<Resource>, ProviderError> {
        let state = self.state.read().await;
        let storage: Vec<Resource> = state.resources
            .values()
            .filter| ( | r r.resource_type == "storage")
            .cloned()
            .collect();
        Ok(storage)
    }

    async fn describe_storage(
        &self,
        _credentials: &str,
        resource_id: &str,
    ) -> Result<Resource, ProviderError> {
        let state = self.state.read().await;
        state.resources
            .get(resource_id)
            .cloned()
            .ok_or_else| ( | ProviderError::ResourceNotFound(resource_id.to_string()))
    }

    async fn delete_storage(&self, _credentials: &str, resource_id: &str)
        -> Result<(), ProviderError> {
        let mut state = self.state.write().await;
        state.resources
            .remove(resource_id)
            .ok_or_else| ( | ProviderError::ResourceNotFound(resource_id.to_string()))?;
        state.operations_count += 1;
        Ok(())
    }

    async fn create_network(
        &self,
        _credentials: &str,
        config: serde_json::Value,
    ) -> Result<Resource, ProviderError> {
        let name = config.get("name")
            .and_then| ( | v v.as_str())
            .ok_or_else| ( | ProviderError::InvalidConfig("name required".to_string()))?;

        let net_id = format!("net-{}", uuid::Uuid::new_v4());
        let resource = Resource {
            id: net_id.clone(),
            resource_type: "network".to_string(),
            name: name.to_string(),
            status: "active".to_string(),
            metadata: serde_json::json!({
                "region": self.config.region,
            }),
        };

        let mut state = self.state.write().await;
        state.resources.insert(net_id.clone(), resource.clone());
        state.operations_count += 1;

        Ok(resource)
    }

    async fn list_networks(&self, _credentials: &str) -> Result<Vec<Resource>, ProviderError> {
        let state = self.state.read().await;
        let networks: Vec<Resource> = state.resources
            .values()
            .filter| ( | r r.resource_type == "network")
            .cloned()
            .collect();
        Ok(networks)
    }

    async fn describe_network(
        &self,
        _credentials: &str,
        resource_id: &str,
    ) -> Result<Resource, ProviderError> {
        let state = self.state.read().await;
        state.resources
            .get(resource_id)
            .cloned()
            .ok_or_else| ( | ProviderError::ResourceNotFound(resource_id.to_string()))
    }

    async fn delete_network(&self, _credentials: &str, resource_id: &str)
        -> Result<(), ProviderError> {
        let mut state = self.state.write().await;
        state.resources
            .remove(resource_id)
            .ok_or_else| ( | ProviderError::ResourceNotFound(resource_id.to_string()))?;
        state.operations_count += 1;
        Ok(())
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_create_instance() {
        let config = MyProviderConfig {
            region: "us-west-1".to_string(),
            api_endpoint: " [https://api.example.com".to_string](https://api.example.com".to_string)(),
            credentials_source: "env".to_string(),
        };
        let provider = MyProvider::new(config);

        let instance_config = serde_json::json!({
            "name": "test-instance",
            "size": "medium",
        });

        let result = provider.create("instance", instance_config).await;
        assert!(result.is_ok());

        let resource = result.unwrap();
        assert_eq!(resource.resource_type, "instance");
        assert_eq!(resource.name, "test-instance");
    }

    #[tokio::test]
    async fn test_list_instances() {
        let config = MyProviderConfig {
            region: "us-west-1".to_string(),
            api_endpoint: " [https://api.example.com".to_string](https://api.example.com".to_string)(),
            credentials_source: "env".to_string(),
        };
        let provider = MyProvider::new(config);

        let config1 = serde_json::json!({"name": "inst1", "size": "small"});
        let config2 = serde_json::json!({"name": "inst2", "size": "large"});

        provider.create("instance", config1).await.unwrap();
        provider.create("instance", config2).await.unwrap();

        let instances = provider.list("instance").await.unwrap();
        assert_eq!(instances.len(), 2);
    }

    #[tokio::test]
    async fn test_delete_instance() {
        let config = MyProviderConfig {
            region: "us-west-1".to_string(),
            api_endpoint: " [https://api.example.com".to_string](https://api.example.com".to_string)(),
            credentials_source: "env".to_string(),
        };
        let provider = MyProvider::new(config);

        let instance_config = serde_json::json!({"name": "test", "size": "small"});
        let resource = provider.create("instance", instance_config).await.unwrap();

        let result = provider.delete("instance", &resource.id).await;
        assert!(result.is_ok());

        let instances = provider.list("instance").await.unwrap();
        assert_eq!(instances.len(), 0);
    }

    #[tokio::test]
    async fn test_mixed_resource_types() {
        let config = MyProviderConfig {
            region: "us-west-1".to_string(),
            api_endpoint: " [https://api.example.com".to_string](https://api.example.com".to_string)(),
            credentials_source: "env".to_string(),
        };
        let provider = MyProvider::new(config);

        let inst_cfg = serde_json::json!({"name": "inst1", "size": "small"});
        let stor_cfg = serde_json::json!({"name": "bucket1"});
        let net_cfg = serde_json::json!({"name": "net1"});

        provider.create("instance", inst_cfg).await.unwrap();
        provider.create("storage", stor_cfg).await.unwrap();
        provider.create("network", net_cfg).await.unwrap();

        let instances = provider.list("instance").await.unwrap();
        let storage = provider.list("storage").await.unwrap();
        let networks = provider.list("network").await.unwrap();

        assert_eq!(instances.len(), 1);
        assert_eq!(storage.len(), 1);
        assert_eq!(networks.len(), 1);
    }
}
```

## Error Handling with Retries

```rust
pub enum CustomError {
    Api(String),
    Network(String),
    Auth(String),
    NotFound(String),
}

impl From<CustomError> for ProviderError {
    fn from(err: CustomError) -> Self {
        match err {
            CustomError::Api(msg) => ProviderError::OperationFailed(msg),
            CustomError::Network(msg) => ProviderError::Timeout(msg),
            CustomError::Auth(msg) => ProviderError::AuthenticationFailed(msg),
            CustomError::NotFound(msg) => ProviderError::ResourceNotFound(msg),
        }
    }
}

pub async fn execute_with_retry<F, T>(
    mut operation: F,
    max_retries: usize,
) -> Result<T, ProviderError>
where
    F: FnMut() -> std::pin::Pin<
        Box<dyn std::future::Future<Output = Result<T, ProviderError>>>,
    >,
{
    let mut retries = 0;
    let mut backoff = std::time::Duration::from_millis(100);

    loop {
        match operation().await {
            Ok(result) => return Ok(result),
            Err(_) if retries < max_retries => {
                retries += 1;
                tokio::time::sleep(backoff).await;
                backoff = std::time::Duration::from_millis(
                    (backoff.as_millis() as u64 * 2).min(5000),
                );
            }
            Err(e) => return Err(e),
        }
    }
}
```

## Metrics and Observability

```rust
use prometheus::{Counter, Histogram, Registry};

pub struct ProviderMetrics {
    operations_total: Counter,
    operations_duration: Histogram,
    errors: Counter,
}

impl ProviderMetrics {
    pub fn new(registry: &Registry) -> Result<Self, Box<dyn std::error::Error>> {
        Ok(Self {
            operations_total: Counter::new(
                "provider_operations_total",
                "Total operations",
            )?,
            operations_duration: Histogram::new(
                "provider_operations_duration",
                "Operation duration",
            )?,
            errors: Counter::new("provider_errors_total", "Total errors")?,
        })
    }

    pub fn record_operation(&self, duration: std::time::Duration) {
        self.operations_total.inc();
        self.operations_duration.observe(duration.as_secs_f64());
    }

    pub fn record_error(&self) {
        self.errors.inc();
    }
}
```

## Provider Configuration (extension.yaml)

```yaml
name: my-provider
version: 1.0.0
edition: 2021

metadata:
  description: Custom cloud provider for Provisioning
  author: Your Organization
  license: Apache-2.0
  repository:  [https://github.com/your-org/my-provider](https://github.com/your-org/my-provider)
  documentation:  [https://docs.example.com/my-provider](https://docs.example.com/my-provider)

capabilities:
  - compute
  - storage
  - networking

resources:
  - name: instance
    operations: [create, delete, list, describe]
  - name: storage
    operations: [create, delete, list, describe]
  - name: network
    operations: [create, delete, list, describe]

dependencies:
  - name: provisioning-sdk
    version: ">=0.1.0"
  - name: tokio
    version: ">=1.0"
  - name: serde
    version: ">=1.0"

tests:
  unit: true
  integration: true

signing:
  key: your-gpg-key-id
```

## Building and Publishing

```bash
# Build provider
cargo build --release

# Run tests
cargo test --all

# Package
provisioning extension package \
    --path ./custom-providers/my-provider \
    --name "my-provider" \
    --version "1.0.0"

# Publish
provisioning extension publish \
    --file my-provider-1.0.0.tar.gz \
    --name "my-provider" \
    --version "1.0.0" \
    --description "Custom cloud provider" \
    --author "Your Organization"
```

## Using Custom Providers

```nickel
{
  extensions = {
    providers = [
      {
        name = "my-provider",
        version = "1.0.0",
        enabled = true,
        config = {
          region = "us-west-1",
          api_endpoint = " [https://api.example.com",](https://api.example.com",)
          credentials_source = "env"
        }
      }
    ]
  },

  infrastructure = {
    instances = [
      {
        name = "web-server",
        provider = "my-provider",
        resource_type = "instance",
        config = {
          size = "large",
          image = "ubuntu-22.04"
        }
      }
    ]
  }
}
```

## Best Practices

1. **Always use async/await** - Rust async allows concurrent operations
2. **Implement proper error handling** - Convert provider-specific errors to ProviderError
3. **Use state locks carefully** - Minimize lock hold time
4. **Log operations extensively** - Include timestamps and operation details
5. **Test error scenarios** - Network failures, timeouts, invalid configs
6. **Implement metrics** - Track operations, failures, latencies
7. **Support graceful degradation** - Partial failures shouldn't cascade
8. **Document resource lifecycle** - Creation, configuration, deletion states

## See Also

- [Extension Registry](../features/extension-registry.md)
- [Custom Task Services](./custom-task-services.md)
- [Custom Detector Development](./custom-detector-development.md)
