14 KiB
14 KiB
\nCost-Optimized Architecture:\n├── Hetzner compute: €72.70/month (~$78)\n├── AWS managed services: $115/month\n└── DigitalOcean CDN: $64/month\nTotal: ~$280/month\n\nAll-AWS Equivalent:\n├── EC2 instances: ~$200+\n├── RDS database: ~$150+\n├── ElastiCache: ~$50+\n├── CloudFront CDN: ~$100+\n└── Other services: ~$50+\nTotal: ~$600+/month\n\nSavings: ~$320/month (53% reduction)\n\n\n### Architecture Benefits\n\nHetzner Advantages:\n- Best price/performance for compute (€20.90/month for 4 vCPU/8GB)\n- Powerful Load Balancer (€10/month)\n- Fast networking (10Gbps)\n- EU data residency (GDPR compliant)\n\nAWS Advantages:\n- Managed RDS: Automatic backups, failover, patching\n- ElastiCache: Redis cluster with automatic failover\n- SQS: Scalable message queue (pay per message)\n- CloudWatch: Comprehensive monitoring\n\nDigitalOcean Advantages:\n- CDN: Cost-effective content delivery ($25/month)\n- Spaces: Object storage at scale ($15/month)\n- Simple pricing and management\n- Edge nodes for regional distribution\n\n## Architecture Overview\n\n\n┌────────────────────────────────────────────────┐\n│ Client Requests │\n└─────────────────┬────────────────────────────────┘\n │ HTTPS/HTTP\n ┌────────▼─────────┐\n │ DigitalOcean │\n │ CDN / Spaces │\n └────────┬─────────┘\n │\n ┌────────────┼────────────┐\n │ │ │\n┌────▼──────┐ ┌──▼────────┐ ┌─▼──────┐\n│ Hetzner │ │ AWS │ │ DO │\n│ Compute │ │ Managed │ │ CDN │\n│ (Load LB) │ │ Services │ │ │\n└────┬──────┘ └──┬────────┘ └────────┘\n │VPN Tunnel │\n┌────▼──────────▼────┐\n│ Hetzner Network │ AWS VPC DO Spaces\n│ 10.0.0.0/16 ◄──► 10.1.0.0/16 ◄──► nyc3\n│ 3x CPX21 Servers │ RDS + Cache CDN +\n│ │ + SQS Backups\n└────────────────────┘\n\n\n## Prerequisites\n\n### 1. Cloud Accounts\n\n- Hetzner: Account with API token\n- AWS: Account with access keys\n- DigitalOcean: Account with API token\n\n### 2. Environment Variables\n\n\nexport HCLOUD_TOKEN="MC4wNTI1YmE1M2E4YmE0YTQzMTQyZTdlODYy"\nexport AWS_ACCESS_KEY_ID="AKIA1234567890ABCDEF"\nexport AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG+j/zI0m1234567890ab"\nexport DIGITALOCEAN_TOKEN="dop_v1_abc123def456ghi789jkl012mno"\n\n\n### 3. CLI Tools\n\n\n# Install and verify\nwhich hcloud && hcloud version\nwhich aws && aws --version\nwhich doctl && doctl version\nwhich nickel && nickel --version\n\n\n### 4. SSH Keys\n\n\n# Hetzner\nhcloud ssh-key create --name provisioning-key \\n --public-key-from-file ~/.ssh/id_rsa.pub\n\n# AWS\naws ec2 create-key-pair --key-name provisioning-key \\n --query 'KeyMaterial' --output text > provisioning-key.pem\nchmod 600 provisioning-key.pem\n\n# DigitalOcean\ndoctl compute ssh-key create provisioning-key \\n --public-key-from-file ~/.ssh/id_rsa.pub\n\n\n## Deployment\n\n### Step 1: Configure the Workspace\n\nEdit workspace.ncl:\n\n\n# Update networking if needed\ncompute_tier.primary_servers = hetzner.Server & {\n server_type = "cpx21",\n count = 3,\n location = "nbg1"\n}\n\n# Update AWS region if needed\nmanaged_services.database = aws.RDS & {\n instance_class = "db.t3.small",\n region = "us-east-1"\n}\n\n# Update CDN endpoints\ncdn_tier.cdn.endpoints = [{\n name = "app-cdn",\n origin = "content.example.com"\n}]\n\n\nEdit config.toml:\n\n\n[cost_tracking]\nmonthly_budget = 300\nbudget_alert_threshold = 280\n\n[application.cache]\nmax_memory = "250MB"\n\n\n### Step 2: Validate Configuration\n\n\n# Validate Nickel syntax\nnickel export workspace.ncl | jq . > /dev/null\n\n# Verify provider access\nhcloud context use default\naws sts get-caller-identity\ndoctl account get\n\n\n### Step 3: Deploy\n\n\nchmod +x deploy.nu\n./deploy.nu\n\n# Or with debug output\n./deploy.nu --debug\n\n\n### Step 4: Verify Deployment\n\n\n# Hetzner compute resources\nhcloud server list\nhcloud load-balancer list\n\n# AWS managed services\naws rds describe-db-instances --region us-east-1\naws elasticache describe-cache-clusters --region us-east-1\naws sqs list-queues --region us-east-1\n\n# DigitalOcean CDN\ndoctl compute cdn list\ndoctl compute spaces list\n\n\n## Post-Deployment Configuration\n\n### 1. Connect Hetzner Compute to AWS Database\n\n\n# Get Hetzner server IPs\nhcloud server list --format ID,PublicIPv4\n\n# Get RDS endpoint\naws rds describe-db-instances --region us-east-1 \\n --query 'DBInstances[0].Endpoint.Address'\n\n# On Hetzner server, install PostgreSQL client\nssh root@hetzner-server\napt-get update && apt-get install postgresql-client\n\n# Test connection to RDS\npsql -h app-db.abc123.us-east-1.rds.amazonaws.com \\n -U admin -d postgres -c "SELECT now();"\n\n\n### 2. Configure Application for Services\n\n\n# Application configuration file\ncat > /var/www/app/.env << EOF\nDATABASE_HOST=app-db.abc123.us-east-1.rds.amazonaws.com\nDATABASE_PORT=5432\nDATABASE_USER=admin\nDATABASE_PASSWORD=your_password\nDATABASE_NAME=app_db\n\nREDIS_HOST=app-cache.abc123.ng.0001.euc1.cache.amazonaws.com\nREDIS_PORT=6379\n\nSQS_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789/app-queue\n\nCDN_ENDPOINT=https://content.example.com\nSPACES_ENDPOINT=https://app-content.nyc3.digitaloceanspaces.com\nSPACES_KEY=your_spaces_key\nSPACES_SECRET=your_spaces_secret\n\nENVIRONMENT=production\nEOF\n\n\n### 3. Setup CDN and Object Storage\n\n\n# Configure Spaces bucket\ndoctl compute spaces create app-content --region nyc3\n\n# Get Spaces endpoint\ndoctl compute spaces list\n\n# Configure CDN endpoint\ndoctl compute cdn create --origin content.example.com\n\n# Upload test file\naws s3 cp test.html s3://app-content/\n\n\n### 4. Configure Application Queue\n\n\n# Get SQS queue URL\naws sqs list-queues --region us-east-1\n\n# Create queue if needed\naws sqs create-queue --queue-name app-queue --region us-east-1\n\n# Test queue\naws sqs send-message --queue-url https://sqs.us-east-1.amazonaws.com/123456789/app-queue \\n --message-body "test message" --region us-east-1\n\n\n### 5. Deploy Application\n\nSSH to Hetzner servers:\n\n\n# Get server IPs\nSERVERS=$(hcloud server list --format PublicIPv4 --no-header)\n\n# Deploy to each server\nfor server in $SERVERS; do\n ssh -o StrictHostKeyChecking=no root@$server << 'DEPLOY'\n cd /var/www\n git clone https://github.com/your-org/app.git\n cd app\n cp .env.example .env\n ./deploy.sh\n DEPLOY\ndone\n\n\n## Monitoring and Cost Control\n\n### Cost Monitoring\n\n\n# Hetzner billing\n# Manual via console: https://console.hetzner.cloud/billing\n\n# AWS cost tracking\naws ce get-cost-and-usage \\n --time-period Start=2024-01-01,End=2024-01-31 \\n --granularity MONTHLY \\n --metrics BlendedCost \\n --group-by Type=DIMENSION,Key=SERVICE\n\n# DigitalOcean billing\ndoctl billing get\n\n# Real-time cost status\naws ce get-cost-and-usage \\n --time-period Start=$(date -d '1 day ago' +%Y-%m-%d),End=$(date +%Y-%m-%d) \\n --granularity DAILY \\n --metrics BlendedCost\n\n\n### Application Performance Monitoring\n\n\n# RDS performance insights\naws pi describe-dimension-keys \\n --service-type RDS \\n --identifier arn:aws:rds:us-east-1:123456789:db:app-db \\n --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \\n --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \\n --period-in-seconds 60 \\n --metric db.load.avg \\n --partition-by Dimension \\n --dimension-group.group-by WAIT_EVENT\n\n# ElastiCache monitoring\naws cloudwatch get-metric-statistics \\n --namespace AWS/ElastiCache \\n --metric-name CPUUtilization \\n --dimensions Name=CacheClusterId,Value=app-cache \\n --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \\n --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \\n --period 300 \\n --statistics Average\n\n# SQS monitoring\naws sqs get-queue-attributes \\n --queue-url https://sqs.us-east-1.amazonaws.com/123456789/app-queue \\n --attribute-names All\n\n\n### Alerts Configuration\n\n\n# CPU threshold alert\naws cloudwatch put-metric-alarm \\n --alarm-name hetzner-cpu-high \\n --alarm-description "Alert when Hetzner CPU > 80%" \\n --metric-name CPUUtilization \\n --threshold 80 \\n --comparison-operator GreaterThanThreshold\n\n# Queue depth alert\naws cloudwatch put-metric-alarm \\n --alarm-name sqs-queue-depth-high \\n --alarm-description "Alert when SQS queue depth > 1000" \\n --metric-name ApproximateNumberOfMessagesVisible \\n --threshold 1000 \\n --comparison-operator GreaterThanThreshold\n\n# Cache eviction alert\naws cloudwatch put-metric-alarm \\n --alarm-name elasticache-eviction-rate-high \\n --alarm-description "Alert when cache eviction rate > 10%" \\n --metric-name EvictionRate \\n --namespace AWS/ElastiCache \\n --threshold 10 \\n --comparison-operator GreaterThanThreshold\n\n\n## Scaling and Optimization\n\n### Scale Hetzner Compute\n\nEdit workspace.ncl:\n\n\ncompute_tier.primary_servers = hetzner.Server & {\n count = 5, # Increase from 3\n server_type = "cpx21"\n}\n\n\nRedeploy:\n\n\n./deploy.nu\n\n\n### Upgrade Database\n\n\n# Modify RDS instance class\naws rds modify-db-instance \\n --db-instance-identifier app-db \\n --db-instance-class db.t3.medium \\n --apply-immediately \\n --region us-east-1\n\n\n### Add Caching Layer\n\nAlready configured with ElastiCache. Optimize by adjusting:\n\n\n[application.cache]\nmax_memory = "512MB"\neviction_policy = "allkeys-lru"\n\n\n### Increase Queue Throughput\n\nSQS automatically scales. Monitor with:\n\n\naws sqs get-queue-attributes \\n --queue-url https://sqs.us-east-1.amazonaws.com/123456789/app-queue \\n --attribute-names ApproximateNumberOfMessages\n\n\n## Cost Optimization Tips\n\n1. Hetzner Compute: CPX21 is sweet spot. Consider CX21 for lower workloads\n2. AWS RDS: Use t3.small for dev, t3.medium for prod with burst capability\n3. ElastiCache: 2 nodes with auto-failover. Monitor eviction rates\n4. SQS: Pay per request, no fixed costs. Good for variable load\n5. DigitalOcean CDN: Cache more aggressively (86400s TTL for assets)\n6. Spaces: Use lifecycle rules to delete old files automatically\n\n### Cost Reduction Checklist\n\n- Reduce Hetzner servers from 3 to 2 (saves ~€21/month)\n- Downgrade RDS to db.t3.micro for dev (saves ~$40/month)\n- Reduce ElastiCache nodes from 2 to 1 (saves ~$12/month)\n- Archive old CDN content (savings from Spaces storage)\n- Use reserved capacity on AWS (20-30% discount)\n\nPotential total savings: ~$100+/month with right-sizing.\n\n## Troubleshooting\n\n### Issue: Hetzner Can't Connect to RDS\n\nDiagnosis:\n\n# SSH to Hetzner server\nssh root@hetzner-server\n\n# Test connectivity\nnc -zv app-db.abc123.us-east-1.rds.amazonaws.com 5432\n\n\nSolution:\n- Check VPN tunnel is active\n- Verify RDS security group allows port 5432 from Hetzner network\n- Check route table on both sides\n\n### Issue: High Database Latency\n\nDiagnosis:\n\n# Check RDS performance\naws pi describe-dimension-keys --service-type RDS ...\n\n# Check network latency\nping -c 5 app-db.abc123.us-east-1.rds.amazonaws.com\n\n\nSolution:\n- Upgrade RDS instance class\n- Increase ElastiCache size to reduce database queries\n- Check network bandwidth between providers\n\n### Issue: Queue Processing Slow\n\nDiagnosis:\n\n# Check queue depth and age\naws sqs get-queue-attributes \\n --queue-url <queue-url> \\n --attribute-names All\n\n\nSolution:\n- Scale up application servers processing queue\n- Reduce visibility timeout if messages are timing out\n- Check application logs for processing errors\n\n## Cleanup\n\n\n# Hetzner\nhcloud server delete hetzner-app-1 hetzner-app-2 hetzner-app-3\nhcloud load-balancer delete app-lb\n\n# AWS\naws rds delete-db-instance --db-instance-identifier app-db --skip-final-snapshot\naws elasticache delete-cache-cluster --cache-cluster-id app-cache\naws sqs delete-queue --queue-url https://sqs.us-east-1.amazonaws.com/123456789/app-queue\n\n# DigitalOcean\ndoctl compute spaces delete app-content\ndoctl compute cdn delete cdn-app\ndoctl compute droplet delete edge-node-1 edge-node-2 edge-node-3\n\n\n## Next Steps\n\n1. Implement application logging to CloudWatch\n2. Set up Hetzner monitoring dashboard\n3. Configure auto-scaling based on queue depth\n4. Implement database read replicas for read-heavy workloads\n5. Add WAF protection to Hetzner load balancer\n6. Implement cross-region backups to Spaces\n7. Set up cost anomaly detection alerts\n\n## Support\n\nFor issues or questions:\n\n- Review the cost-optimized deployment guide\n- Check provider-specific documentation\n- Monitor costs with: aws ce get-cost-and-usage ...\n- Review deployment logs: ./deploy.nu --debug\n\n## Files\n\n- workspace.ncl: Infrastructure definition (Nickel)\n- config.toml: Provider credentials and settings\n- deploy.nu: Deployment orchestration (Nushell)\n- README.md: This file