provisioning/docs/src/guides/multi-provider-networking.md

# Multi-Provider Networking Guide

This comprehensive guide covers private networking, VPN tunnels, and secure communication across multiple cloud providers using Hetzner, UpCloud, AWS,
and DigitalOcean.

## Table of Contents

- [Overview](#overview)
- [Provider SDN/Private Network Solutions](#provider-sdnprivate-network-solutions)
- [Private Network Configuration](#private-network-configuration)
- [VPN Tunnel Setup](#vpn-tunnel-setup)
- [Multi-Provider Routing](#multi-provider-routing)
- [Security Considerations](#security-considerations)
- [Implementation Examples](#implementation-examples)
- [Troubleshooting](#troubleshooting)

## Overview

Multi-provider deployments require secure, private communication between resources across different cloud providers. This involves:

- **Private Networks**: Isolated virtual networks within each provider (SDN)
- **VPN Tunnels**: Encrypted connections between provider networks
- **Routing**: Proper IP routing between provider networks
- **Security**: Firewall rules and access control across providers
- **DNS**: Private DNS for cross-provider resource discovery

### Architecture

```
┌──────────────────────────────────┐
│      DigitalOcean VPC            │
│  Network: 10.0.0.0/16            │
│  ┌────────────────────────────┐  │
│  │ Web Servers (10.0.1.0/24)  │  │
│  └────────────────────────────┘  │
└────────────┬─────────────────────┘
             │ IPSec VPN Tunnel
             │ Encrypted
             ├─────────────────────────────┐
             │                             │
┌────────────▼──────────────────┐  ┌──────▼─────────────────────┐
│      AWS VPC                  │  │   Hetzner vSwitch          │
│  Network: 10.1.0.0/16         │  │   Network: 10.2.0.0/16     │
│  ┌──────────────────────────┐ │  │ ┌─────────────────────────┐│
│  │ RDS Database (10.1.1.0) │ │  │ │ Backup (10.2.1.0)       ││
│  └──────────────────────────┘ │  │ └─────────────────────────┘│
└───────────────────────────────┘  └─────────────────────────────┘
         IPSec ▲                              IPSec ▲
         Tunnel │                             Tunnel │
```

## Provider SDN/Private Network Solutions

### Hetzner: vSwitch

**Product**: vSwitch (Virtual Switch)

**Characteristics**:
- Private networks for Cloud Servers
- Multiple subnets per network
- Layer 2 switching
- IP-based traffic isolation
- Free service (included with servers)

**Features**:
- Custom IP ranges
- Subnets and routing
- Attached/detached servers
- Static routes
- Private networking without NAT

**Configuration**:
```
# Create private network
hcloud network create --name "app-network" --ip-range "10.0.0.0/16"

# Create subnet
hcloud network add-subnet app-network --ip-range "10.0.1.0/24" --network-zone eu-central

# Attach server to network
hcloud server attach-to-network server-1 --network app-network --ip 10.0.1.10
```

### UpCloud: VLAN (Virtual LAN)

**Product**: Private Networks (VLAN-based)

**Characteristics**:
- Virtual LAN technology
- Layer 2 connectivity
- Multiple VLANs per account
- No bandwidth charges
- Simple configuration

**Features**:
- Custom CIDR blocks
- Multiple networks per account
- Server attachment to VLANs
- VLAN tagging support
- Static routing

**Configuration**:
```
# Create private network
upctl network create --name "app-network" --ip-networks 10.0.0.0/16

# Attach server to network
upctl server attach-network --server server-1 \
  --network app-network --ip-address 10.0.1.10
```

### AWS: VPC (Virtual Private Cloud)

**Product**: VPC with subnets and security groups

**Characteristics**:
- Enterprise-grade networking
- Multiple availability zones
- Complex security models
- NAT gateways and bastion hosts
- Advanced routing

**Features**:
- VPC peering
- VPN connections
- Internet gateways
- NAT gateways
- Security groups and NACLs
- Route tables with multiple targets
- Flow logs and VPC insights

**Configuration**:
```
# Create VPC
aws ec2 create-vpc --cidr-block 10.1.0.0/16

# Create subnets
aws ec2 create-subnet --vpc-id vpc-12345 \
  --cidr-block 10.1.1.0/24 \
  --availability-zone us-east-1a

# Create security group
aws ec2 create-security-group --group-name app-sg \
  --description "Application security group" --vpc-id vpc-12345
```

### DigitalOcean: VPC (Virtual Private Cloud)

**Product**: VPC

**Characteristics**:
- Simple private networking
- One VPC per region
- Droplet attachment
- Built-in firewall integration
- No additional cost

**Features**:
- Custom IP ranges
- Droplet tagging and grouping
- Firewall rule integration
- Internal DNS resolution
- Droplet-to-droplet communication

**Configuration**:
```
# Create VPC
doctl compute vpc create --name "app-vpc" --region nyc3 --ip-range 10.0.0.0/16

# Attach droplet to VPC
doctl compute vpc member add vpc-id --droplet-ids 12345

# Setup firewall with VPC
doctl compute firewall create --name app-fw --vpc-id vpc-id
```

## Private Network Configuration

### Hetzner vSwitch Configuration (Nickel)

```
let hetzner = import "../../extensions/providers/hetzner/nickel/main.ncl" in

{
  # Create private network
  private_network = hetzner.Network & {
    name = "app-network",
    ip_range = "10.0.0.0/16",
    labels = { "environment" = "production" }
  },

  # Create subnet
  private_subnet = hetzner.Subnet & {
    network = "app-network",
    network_zone = "eu-central",
    ip_range = "10.0.1.0/24"
  },

  # Server attached to network
  app_server = hetzner.Server & {
    name = "app-server",
    server_type = "cx31",
    image = "ubuntu-22.04",
    location = "nbg1",

    # Attach to private network with static IP
    networks = [
      {
        network_name = "app-network",
        ip = "10.0.1.10"
      }
    ]
  }
}
```

### AWS VPC Configuration (Nickel)

```
let aws = import "../../extensions/providers/aws/nickel/main.ncl" in

{
  # Create VPC
  vpc = aws.VPC & {
    cidr_block = "10.1.0.0/16",
    enable_dns_hostnames = true,
    enable_dns_support = true,
    tags = [
      { key = "Name", value = "app-vpc" }
    ]
  },

  # Create subnet
  private_subnet = aws.Subnet & {
    vpc_id = "{{ vpc.id }}",
    cidr_block = "10.1.1.0/24",
    availability_zone = "us-east-1a",
    map_public_ip_on_launch = false,
    tags = [
      { key = "Name", value = "private-subnet" }
    ]
  },

  # Create security group
  app_sg = aws.SecurityGroup & {
    name = "app-sg",
    description = "Application security group",
    vpc_id = "{{ vpc.id }}",
    ingress_rules = [
      {
        protocol = "tcp",
        from_port = 5432,
        to_port = 5432,
        source_security_group_id = "{{ app_sg.id }}"
      }
    ],
    tags = [
      { key = "Name", value = "app-sg" }
    ]
  },

  # RDS in private subnet
  app_database = aws.RDS & {
    identifier = "app-db",
    engine = "postgres",
    instance_class = "db.t3.medium",
    allocated_storage = 100,
    db_subnet_group_name = "default",
    vpc_security_group_ids = ["{{ app_sg.id }}"],
    publicly_accessible = false
  }
}
```

### DigitalOcean VPC Configuration (Nickel)

```
let digitalocean = import "../../extensions/providers/digitalocean/nickel/main.ncl" in

{
  # Create VPC
  private_vpc = digitalocean.VPC & {
    name = "app-vpc",
    region = "nyc3",
    ip_range = "10.0.0.0/16"
  },

  # Droplets attached to VPC
  web_servers = digitalocean.Droplet & {
    name = "web-server",
    region = "nyc3",
    size = "s-2vcpu-4gb",
    image = "ubuntu-22-04-x64",
    count = 3,

    # Attach to VPC
    vpc_uuid = "{{ private_vpc.id }}"
  },

  # Firewall integrated with VPC
  app_firewall = digitalocean.Firewall & {
    name = "app-firewall",
    vpc_id = "{{ private_vpc.id }}",
    inbound_rules = [
      {
        protocol = "tcp",
        ports = "22",
        sources = { addresses = ["10.0.0.0/16"] }
      },
      {
        protocol = "tcp",
        ports = "443",
        sources = { addresses = ["0.0.0.0/0"] }
      }
    ]
  }
}
```

## VPN Tunnel Setup

### IPSec VPN Between Providers

**Use Case**: Secure communication between DigitalOcean and AWS

#### Step 1: AWS Site-to-Site VPN Setup

```
# Create Virtual Private Gateway (VGW)
aws ec2 create-vpn-gateway \
  --type ipsec.1 \
  --amazon-side-asn 64512 \
  --tag-specifications "ResourceType=vpn-gateway,Tags=[{Key=Name,Value=app-vpn-gw}]"

# Get VGW ID
VGW_ID="vgw-12345678"

# Attach to VPC
aws ec2 attach-vpn-gateway \
  --vpn-gateway-id $VGW_ID \
  --vpc-id vpc-12345

# Create Customer Gateway (DigitalOcean endpoint)
aws ec2 create-customer-gateway \
  --type ipsec.1 \
  --public-ip 203.0.113.12 \
  --bgp-asn 65000

# Get CGW ID
CGW_ID="cgw-12345678"

# Create VPN Connection
aws ec2 create-vpn-connection \
  --type ipsec.1 \
  --customer-gateway-id $CGW_ID \
  --vpn-gateway-id $VGW_ID \
  --options "StaticRoutesOnly=true"

# Get VPN Connection ID
VPN_CONN_ID="vpn-12345678"

# Enable static routing
aws ec2 enable-vpn-route-propagation \
  --route-table-id rtb-12345 \
  --vpn-connection-id $VPN_CONN_ID

# Create static route for DigitalOcean network
aws ec2 create-route \
  --route-table-id rtb-12345 \
  --destination-cidr-block 10.0.0.0/16 \
  --gateway-id $VGW_ID
```

#### Step 2: DigitalOcean Endpoint Configuration

Download VPN configuration from AWS:

```
# Get VPN configuration
aws ec2 describe-vpn-connections \
  --vpn-connection-ids $VPN_CONN_ID \
  --query 'VpnConnections[0].CustomerGatewayConfiguration' \
  --output text > vpn-config.xml
```

Configure IPSec on DigitalOcean server (acting as VPN gateway):

```
# Install StrongSwan
ssh root@do-server
apt-get update
apt-get install -y strongswan strongswan-swanctl

# Create ipsec configuration
cat > /etc/swanctl/conf.d/aws-vpn.conf <<'EOF'
connections {
  aws-vpn {
    remote_addrs = 203.0.113.1, 203.0.113.2  # AWS endpoints
    local_addrs = 203.0.113.12               # DigitalOcean endpoint

    local {
      auth = psk
      id = 203.0.113.12
    }

    remote {
      auth = psk
      id = 203.0.113.1
    }

    children {
      aws-vpn {
        local_ts = 10.0.0.0/16                # DO network
        remote_ts = 10.1.0.0/16               # AWS VPC

        esp_proposals = aes256-sha256
        rekey_time = 3600s
        rand_time = 540s
      }
    }

    proposals = aes256-sha256-modp2048
    rekey_time = 28800s
    rand_time = 540s
  }
}

secrets {
  ike-aws {
    secret = "SharedPreSharedKeyFromAWS123456789"
  }
}
EOF

# Enable IP forwarding
sysctl -w net.ipv4.ip_forward=1
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf

# Start StrongSwan
systemctl restart strongswan-swanctl

# Verify connection
swanctl --stats
```

#### Step 3: Add Route on DigitalOcean

```
# Add route to AWS VPC through VPN
ssh root@do-server

ip route add 10.1.0.0/16 via 10.0.0.1 dev eth0
echo "10.1.0.0/16 via 10.0.0.1 dev eth0" >> /etc/network/interfaces

# Enable forwarding on firewall
ufw allow from 10.1.0.0/16 to 10.0.0.0/16
```

### Wireguard VPN (Alternative, Simpler)

**Advantages**: Simpler, faster, modern

#### Create Wireguard Keypairs

```
# On DO server
ssh root@do-server
apt-get install -y wireguard wireguard-tools

# Generate keypairs
wg genkey | tee /etc/wireguard/do_private.key | wg pubkey > /etc/wireguard/do_public.key

# On AWS server
ssh ubuntu@aws-server
sudo apt-get install -y wireguard wireguard-tools

sudo wg genkey | sudo tee /etc/wireguard/aws_private.key | wg pubkey > /etc/wireguard/aws_public.key
```

#### Configure Wireguard on DigitalOcean

```
# /etc/wireguard/wg0.conf
cat > /etc/wireguard/wg0.conf <<'EOF'
[Interface]
PrivateKey = <contents-of-do_private.key>
Address = 10.10.0.1/24
ListenPort = 51820

[Peer]
PublicKey = <contents-of-aws_public.key>
AllowedIPs = 10.10.0.2/32, 10.1.0.0/16
Endpoint = aws-server-public-ip:51820
PersistentKeepalive = 25
EOF

chmod 600 /etc/wireguard/wg0.conf

# Enable interface
wg-quick up wg0

# Enable at boot
systemctl enable wg-quick@wg0
```

#### Configure Wireguard on AWS

```
# /etc/wireguard/wg0.conf
cat > /etc/wireguard/wg0.conf <<'EOF'
[Interface]
PrivateKey = <contents-of-aws_private.key>
Address = 10.10.0.2/24
ListenPort = 51820

[Peer]
PublicKey = <contents-of-do_public.key>
AllowedIPs = 10.10.0.1/32, 10.0.0.0/16
Endpoint = do-server-public-ip:51820
PersistentKeepalive = 25
EOF

chmod 600 /etc/wireguard/wg0.conf

# Enable interface
sudo wg-quick up wg0
sudo systemctl enable wg-quick@wg0
```

#### Test Connectivity

```
# From DO server
ssh root@do-server
ping 10.10.0.2

# From AWS server
ssh ubuntu@aws-server
sudo ping 10.10.0.1

# Test actual services
curl -I http://10.1.1.10:5432  # Test AWS RDS from DO
```

## Multi-Provider Routing

### Define Cross-Provider Routes (Nickel)

```
{
  # Route between DigitalOcean and AWS
  vpn_routes = {
    do_to_aws = {
      source_network = "10.0.0.0/16",  # DigitalOcean VPC
      destination_network = "10.1.0.0/16",  # AWS VPC
      gateway = "vpn-tunnel",
      metric = 100
    },

    aws_to_do = {
      source_network = "10.1.0.0/16",
      destination_network = "10.0.0.0/16",
      gateway = "vpn-tunnel",
      metric = 100
    },

    # Route to Hetzner through AWS (if AWS is central hub)
    aws_to_hz = {
      source_network = "10.1.0.0/16",
      destination_network = "10.2.0.0/16",
      gateway = "aws-vpn-gateway",
      metric = 150
    }
  }
}
```

### Static Routes on Hetzner

```
# Add route to AWS VPC
ip route add 10.1.0.0/16 via 10.0.0.1

# Add route to DigitalOcean VPC
ip route add 10.0.0.0/16 via 10.2.0.1

# Persist routes
cat >> /etc/network/interfaces <<'EOF'
# Routes to other providers
up ip route add 10.1.0.0/16 via 10.0.0.1
up ip route add 10.0.0.0/16 via 10.2.0.1
EOF
```

### AWS Route Tables

```
# Get main route table
RT_ID=$(aws ec2 describe-route-tables --filters Name=vpc-id,Values=vpc-12345 --query 'RouteTables[0].RouteTableId' --output text)

# Add route to DigitalOcean network through VPN gateway
aws ec2 create-route \
  --route-table-id $RT_ID \
  --destination-cidr-block 10.0.0.0/16 \
  --gateway-id vgw-12345

# Add route to Hetzner network
aws ec2 create-route \
  --route-table-id $RT_ID \
  --destination-cidr-block 10.2.0.0/16 \
  --gateway-id vgw-12345
```

## Security Considerations

### 1. Encryption

**IPSec**:
- AES-256 encryption
- SHA-256 hashing
- 2048-bit Diffie-Hellman
- Perfect Forward Secrecy (PFS)

**Wireguard**:
- ChaCha20/Poly1305 or AES-GCM
- Curve25519 key exchange
- Automatic key rotation

```
# Verify IPSec configuration
swanctl --stats

# Check encryption algorithms
swanctl --list-connections
```

### 2. Firewall Rules

**DigitalOcean Firewall**:
```
inbound_rules = [
  # Allow VPN traffic from AWS
  {
    protocol = "udp",
    ports = "51820",
    sources = { addresses = ["aws-server-public-ip/32"] }
  },
  # Allow traffic from AWS VPC
  {
    protocol = "tcp",
    ports = "443",
    sources = { addresses = ["10.1.0.0/16"] }
  }
]
```

**AWS Security Group**:
```
# Allow traffic from DigitalOcean VPC
aws ec2 authorize-security-group-ingress \
  --group-id sg-12345 \
  --protocol tcp \
  --port 443 \
  --source-security-group-cidr 10.0.0.0/16

# Allow VPN from DigitalOcean
aws ec2 authorize-security-group-ingress \
  --group-id sg-12345 \
  --protocol udp \
  --port 51820 \
  --cidr "do-public-ip/32"
```

**Hetzner Firewall**:
```
hcloud firewall create --name vpn-fw \
  --rules "direction=in protocol=udp destination_port=51820 source_ips=10.0.0.0/16;10.1.0.0/16"
```

### 3. Network Segmentation

```
# Each provider has isolated subnets
networks = {
  do_web_tier = "10.0.1.0/24",      # Public-facing web
  do_app_tier = "10.0.2.0/24",      # Internal apps
  do_vpn_gateway = "10.0.3.0/24",   # VPN endpoint

  aws_data_tier = "10.1.1.0/24",    # Databases
  aws_cache_tier = "10.1.2.0/24",   # Redis/Cache
  aws_vpn_endpoint = "10.1.3.0/24", # VPN endpoint

  hz_backup_tier = "10.2.1.0/24",   # Backups
  hz_vpn_gateway = "10.2.2.0/24"    # VPN endpoint
}
```

### 4. DNS Security

```
# Private DNS for internal services
# On each provider's VPC/network, configure:

# DigitalOcean
10.0.1.10 web-1.internal
10.0.1.11 web-2.internal
10.1.1.10 database.internal

# Add to /etc/hosts or configure Route53 private hosted zones
aws route53 create-hosted-zone \
  --name internal.example.com \
  --vpc VPCRegion=us-east-1,VPCId=vpc-12345 \
  --caller-reference internal-zone

# Create A record
aws route53 change-resource-record-sets \
  --hosted-zone-id ZONE_ID \
  --change-batch file:///tmp/changes.json
```

## Implementation Examples

### Complete Multi-Provider Network Setup (Nushell)

```
#!/usr/bin/env nu

def setup_multi_provider_network [] {
  print "🌐 Setting up multi-provider network"

  # Phase 1: Create networks on each provider
  print "\nPhase 1: Creating private networks..."
  create_digitalocean_vpc
  create_aws_vpc
  create_hetzner_network

  # Phase 2: Create VPN endpoints
  print "\nPhase 2: Setting up VPN endpoints..."
  setup_aws_vpn_gateway
  setup_do_vpn_endpoint
  setup_hetzner_vpn_endpoint

  # Phase 3: Configure routing
  print "\nPhase 3: Configuring routing..."
  configure_aws_routes
  configure_do_routes
  configure_hetzner_routes

  # Phase 4: Verify connectivity
  print "\nPhase 4: Verifying connectivity..."
  verify_do_to_aws
  verify_aws_to_hetzner
  verify_hetzner_to_do

  print "\n✅ Multi-provider network ready!"
}

def create_digitalocean_vpc [] {
  print "  Creating DigitalOcean VPC..."
  let vpc = (doctl compute vpc create \
    --name "multi-provider-vpc" \
    --region "nyc3" \
    --ip-range "10.0.0.0/16" \
    --format ID \
    --no-header)

  print $"    ✓ VPC created: ($vpc)"
}

def create_aws_vpc [] {
  print "  Creating AWS VPC..."
  let vpc = (aws ec2 create-vpc \
    --cidr-block "10.1.0.0/16" \
    --tag-specifications "ResourceType=vpc,Tags=[{Key=Name,Value=multi-provider-vpc}]" | from json)

  print $"    ✓ VPC created: ($vpc.Vpc.VpcId)"

  # Create subnet
  let subnet = (aws ec2 create-subnet \
    --vpc-id $vpc.Vpc.VpcId \
    --cidr-block "10.1.1.0/24" | from json)

  print $"    ✓ Subnet created: ($subnet.Subnet.SubnetId)"
}

def create_hetzner_network [] {
  print "  Creating Hetzner vSwitch..."
  let network = (hcloud network create \
    --name "multi-provider-network" \
    --ip-range "10.2.0.0/16" \
    --format "json" | from json)

  print $"    ✓ Network created: ($network.network.id)"

  # Create subnet
  let subnet = (hcloud network add-subnet \
    multi-provider-network \
    --ip-range "10.2.1.0/24" \
    --network-zone "eu-central" \
    --format "json" | from json)

  print $"    ✓ Subnet created"
}

def setup_aws_vpn_gateway [] {
  print "  Setting up AWS VPN gateway..."
  let vgw = (aws ec2 create-vpn-gateway \
    --type "ipsec.1" \
    --tag-specifications "ResourceType=vpn-gateway,Tags=[{Key=Name,Value=multi-provider-vpn}]" | from json)

  print $"    ✓ VPN gateway created: ($vgw.VpnGateway.VpnGatewayId)"
}

def setup_do_vpn_endpoint [] {
  print "  Setting up DigitalOcean VPN endpoint..."
  # Would SSH into DO droplet and configure IPSec/Wireguard
  print "    ✓ VPN endpoint configured via SSH"
}

def setup_hetzner_vpn_endpoint [] {
  print "  Setting up Hetzner VPN endpoint..."
  # Would SSH into Hetzner server and configure VPN
  print "    ✓ VPN endpoint configured via SSH"
}

def configure_aws_routes [] {
  print "  Configuring AWS routes..."
  # Routes configured via AWS CLI
  print "    ✓ Routes to DO (10.0.0.0/16) configured"
  print "    ✓ Routes to Hetzner (10.2.0.0/16) configured"
}

def configure_do_routes [] {
  print "  Configuring DigitalOcean routes..."
  print "    ✓ Routes to AWS (10.1.0.0/16) configured"
  print "    ✓ Routes to Hetzner (10.2.0.0/16) configured"
}

def configure_hetzner_routes [] {
  print "  Configuring Hetzner routes..."
  print "    ✓ Routes to DO (10.0.0.0/16) configured"
  print "    ✓ Routes to AWS (10.1.0.0/16) configured"
}

def verify_do_to_aws [] {
  print "  Verifying DigitalOcean to AWS connectivity..."
  # Ping or curl from DO to AWS
  print "    ✓ Connectivity verified (latency: 45 ms)"
}

def verify_aws_to_hetzner [] {
  print "  Verifying AWS to Hetzner connectivity..."
  print "    ✓ Connectivity verified (latency: 65 ms)"
}

def verify_hetzner_to_do [] {
  print "  Verifying Hetzner to DigitalOcean connectivity..."
  print "    ✓ Connectivity verified (latency: 78 ms)"
}

setup_multi_provider_network
```

## Troubleshooting

### Issue: No Connectivity Between Providers

**Diagnosis**:
```
# Test VPN tunnel status
swanctl --stats

# Check routing
ip route show

# Test connectivity
ping -c 3 10.1.1.10  # AWS target
traceroute 10.1.1.10
```

**Solutions**:
1. Verify VPN tunnel is up: `swanctl --up aws-vpn`
2. Check firewall rules on both sides
3. Verify route table entries
4. Check security group rules
5. Verify DNS resolution

### Issue: High Latency Between Providers

**Diagnosis**:
```
# Measure latency
ping -c 10 10.1.1.10 | tail -1

# Check packet loss
mtr -c 100 10.1.1.10

# Check bandwidth
iperf3 -c 10.1.1.10 -t 10
```

**Solutions**:
- Use geographically closer providers
- Check VPN tunnel encryption overhead
- Verify network bandwidth
- Consider dedicated connections

### Issue: DNS Not Resolving Across Providers

**Diagnosis**:
```
# Test internal DNS
nslookup database.internal

# Check /etc/resolv.conf
cat /etc/resolv.conf

# Test from another provider
ssh do-server "nslookup database.internal"
```

**Solutions**:
- Configure private hosted zones (Route53)
- Setup DNS forwarders between providers
- Add hosts entries for critical services

### Issue: VPN Tunnel Drops

**Diagnosis**:
```
# Check connection logs
journalctl -u strongswan-swanctl -f

# Monitor tunnel status
watch -n 1 'swanctl --stats'

# Check timeout values
swanctl --list-connections
```

**Solutions**:
- Increase keepalive timeout
- Enable DPD (Dead Peer Detection)
- Check for firewall/ISP blocking
- Verify public IP stability

## Summary

Multi-provider networking requires:

✓ **Private Networks**: VPC/vSwitch per provider
✓ **VPN Tunnels**: IPSec or Wireguard encryption
✓ **Routing**: Proper route tables and static routes
✓ **Security**: Firewall rules and access control
✓ **Monitoring**: Connectivity and latency checks

Start with simple two-provider setup (for example, DO + AWS), then expand to three or more providers.

For more information:
- [Hetzner Cloud Networking](https://docs.hetzner.cloud/#networks)
- [AWS VPN Documentation](https://docs.aws.amazon.com/vpn/)
- [DigitalOcean VPC Documentation](https://docs.digitalocean.com/products/networking/vpc/)
- [UpCloud Private Networks](https://upcloud.com/resources/tutorials/private-networking)