Rustelo/info/deployment.md

# Deployment Guide

This guide covers the deployment of the Rustelo application with comprehensive monitoring, health checks, and CI/CD pipeline setup.

## Table of Contents

1. [Overview](#overview)
2. [Prerequisites](#prerequisites)
3. [Docker Setup](#docker-setup)
4. [Health Checks](#health-checks)
5. [Metrics and Monitoring](#metrics-and-monitoring)
6. [CI/CD Pipeline](#cicd-pipeline)
7. [Deployment Commands](#deployment-commands)
8. [Production Deployment](#production-deployment)
9. [Troubleshooting](#troubleshooting)

## Overview

The Rustelo application includes the following deployment features:

- **Docker Containerization**: Multi-stage builds with optimized production images
- **Health Check Endpoints**: Kubernetes-compatible liveness and readiness probes
- **Prometheus Metrics**: Comprehensive application and system metrics
- **GitHub Actions CI/CD**: Automated testing, building, and deployment
- **Grafana Dashboards**: Pre-configured monitoring dashboards

## Prerequisites

### System Requirements

- **Docker**: Version 20.0+ with Docker Compose
- **Node.js**: Version 18+ (for frontend builds)
- **Git**: For version control and CI/CD
- **curl**: For health checks and API testing

### Optional (for monitoring)

- **Prometheus**: For metrics collection
- **Grafana**: For visualization
- **PostgreSQL**: For production database
- **Redis**: For caching and sessions

## Docker Setup

### 1. Basic Development Setup

```bash
# Clone the repository
git clone <repository-url>
cd rustelo

# Start development environment
docker-compose up -d

# View logs
docker-compose logs -f
```

### 2. Production Setup

```bash
# Build and deploy production
./deploy.sh deploy -e production --migrate --backup

# Check deployment status
./deploy.sh status

# View application logs
./deploy.sh logs -f
```

### 3. Environment-Specific Configurations

#### Development
```bash
# Use development profile
docker-compose --profile dev up -d

# Or use the deploy script
./deploy.sh deploy -e development
```

#### Staging
```bash
# Deploy to staging
./deploy.sh deploy -e staging --migrate
```

#### Production
```bash
# Deploy to production with full monitoring
./deploy.sh deploy -e production --migrate --backup
```

## Health Checks

The application provides comprehensive health check endpoints:

### Endpoints

| Endpoint | Purpose | Description |
|----------|---------|-------------|
| `/health` | Comprehensive health check | Checks all components (database, auth, content, email, system) |
| `/health/live` | Liveness probe | Simple check if application is running |
| `/health/ready` | Readiness probe | Checks if application can handle traffic |

### Example Health Check Response

```json
{
  "status": "healthy",
  "timestamp": "2024-01-15T10:30:00Z",
  "version": "0.1.0",
  "environment": "production",
  "uptime_seconds": 3600,
  "components": [
    {
      "name": "database",
      "status": "healthy",
      "message": "Database connection successful",
      "response_time_ms": 25,
      "metadata": {
        "pool_size": 10,
        "idle_connections": 8
      }
    },
    {
      "name": "auth_service",
      "status": "healthy",
      "message": "Authentication service operational",
      "response_time_ms": 12,
      "metadata": {}
    }
  ],
  "summary": {
    "healthy": 5,
    "degraded": 0,
    "unhealthy": 0
  }
}
```

### Kubernetes Health Check Configuration

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rustelo-app
spec:
  template:
    spec:
      containers:
      - name: rustelo
        image: rustelo:latest
        ports:
        - containerPort: 3030
        livenessProbe:
          httpGet:
            path: /health/live
            port: 3030
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 3
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 3030
          initialDelaySeconds: 5
          periodSeconds: 5
          timeoutSeconds: 3
          failureThreshold: 3
```

## Metrics and Monitoring

### Prometheus Metrics

The application exposes metrics at `/metrics` endpoint with the following categories:

#### HTTP Metrics
- `rustelo_http_requests_total`: Total HTTP requests by method, path, status
- `rustelo_http_request_duration_seconds`: Request duration histogram
- `rustelo_http_requests_in_flight`: Current number of active requests

#### Database Metrics
- `rustelo_db_connections_active`: Active database connections
- `rustelo_db_connections_idle`: Idle database connections
- `rustelo_db_queries_total`: Total database queries by operation and table
- `rustelo_db_query_duration_seconds`: Database query duration histogram

#### Authentication Metrics
- `rustelo_auth_requests_total`: Authentication requests by type and status
- `rustelo_auth_failures_total`: Authentication failures by type and reason
- `rustelo_auth_sessions_active`: Number of active sessions
- `rustelo_auth_token_generations_total`: Total tokens generated

#### Content Metrics
- `rustelo_content_requests_total`: Content requests by type and status
- `rustelo_content_cache_hits_total`: Content cache hits
- `rustelo_content_cache_misses_total`: Content cache misses
- `rustelo_content_processing_duration_seconds`: Content processing time

#### System Metrics
- `rustelo_memory_usage_bytes`: Memory usage in bytes
- `rustelo_cpu_usage_percent`: CPU usage percentage
- `rustelo_disk_usage_bytes`: Disk usage by path
- `rustelo_uptime_seconds`: Application uptime

### Grafana Setup

#### 1. Start Monitoring Stack

```bash
# Start with monitoring services
docker-compose --profile monitoring up -d

# Access Grafana at http://localhost:3000
# Default credentials: admin/admin
```

#### 2. Pre-configured Dashboards

- **Rustelo Application Overview**: Main application metrics
- **System Resources**: CPU, memory, disk usage
- **Database Performance**: Connection pool, query metrics
- **Authentication Analytics**: Login patterns, failures
- **Content Management**: Cache performance, processing times

#### 3. Custom Metrics

You can add custom business metrics in your application:

```rust
// Record custom events
metrics.record_user_registration();
metrics.record_content_view();
metrics.record_rate_limit_hit();
```

### Alerting Rules

Example Prometheus alerting rules:

```yaml
groups:
  - name: rustelo
    rules:
      - alert: HighErrorRate
        expr: rate(rustelo_http_requests_total{status_code=~"5.."}[5m]) > 0.1
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "High error rate detected"
          description: "Error rate is {{ $value }} requests per second"

      - alert: DatabaseConnectionPoolExhausted
        expr: rustelo_db_connections_idle == 0
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Database connection pool exhausted"
          description: "No idle database connections available"

      - alert: HighMemoryUsage
        expr: rustelo_memory_usage_bytes > 1000000000  # 1GB
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High memory usage"
          description: "Memory usage is {{ $value }} bytes"
```

## CI/CD Pipeline

### GitHub Actions Workflow

The CI/CD pipeline includes:

1. **Test Suite**: Runs on every push and PR
2. **Security Audit**: Vulnerability scanning
3. **Docker Build**: Multi-platform image building
4. **Deployment**: Automated deployment to staging/production

### Pipeline Stages

#### 1. Testing Stage

```yaml
- name: Run tests
  run: cargo test --all-features
  env:
    DATABASE_URL: postgresql://postgres:postgres@localhost:5432/rustelo_test
    REDIS_URL: redis://localhost:6379
```

#### 2. Security Stage

```yaml
- name: Run security audit
  run: cargo audit

- name: Run cargo-deny
  uses: EmbarkStudios/cargo-deny-action@v1
```

#### 3. Build Stage

```yaml
- name: Build and push Docker image
  uses: docker/build-push-action@v5
  with:
    context: .
    platforms: linux/amd64,linux/arm64
    push: ${{ github.event_name == 'release' }}
    tags: ${{ steps.meta.outputs.tags }}
    cache-from: type=gha
    cache-to: type=gha,mode=max
```

#### 4. Deployment Stage

```yaml
- name: Deploy to production
  run: |
    ./deploy.sh deploy -e production --migrate --backup
    ./deploy.sh health
```

### Required Secrets

Set these secrets in your GitHub repository:

```bash
DOCKER_USERNAME=your_docker_username
DOCKER_PASSWORD=your_docker_password
PRODUCTION_SSH_KEY=your_production_server_ssh_key
DATABASE_URL=your_production_database_url
```

## Deployment Commands

### Using the Deploy Script

```bash
# Basic deployment
./deploy.sh deploy

# Deploy with options
./deploy.sh deploy -e staging --migrate --backup

# Scale application
./deploy.sh scale -s 3

# Check status
./deploy.sh status

# View logs
./deploy.sh logs -f

# Health check
./deploy.sh health

# Update to latest
./deploy.sh update

# Stop application
./deploy.sh stop

# Clean up
./deploy.sh clean
```

### Manual Docker Compose Commands

```bash
# Build and start services
docker-compose up -d --build

# Scale specific service
docker-compose up -d --scale app=3

# View logs
docker-compose logs -f app

# Check service status
docker-compose ps

# Stop all services
docker-compose down

# Remove volumes (WARNING: destroys data)
docker-compose down -v
```

## Production Deployment

### 1. Server Preparation

```bash
# Update system
sudo apt update && sudo apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/download/v2.23.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

# Create application directory
sudo mkdir -p /opt/rustelo
cd /opt/rustelo
```

### 2. Environment Configuration

```bash
# Create production environment file
cat > .env.production << EOF
ENVIRONMENT=production
DATABASE_URL=postgresql://username:password@localhost:5432/rustelo_prod
REDIS_URL=redis://localhost:6379
RUST_LOG=info
ENABLE_METRICS=true
ENABLE_HEALTH_CHECK=true
EOF
```

### 3. SSL/TLS Setup

```bash
# Generate certificates (using Let's Encrypt)
sudo apt install certbot
sudo certbot certonly --standalone -d yourdomain.com

# Update docker-compose.yml to use certificates
# Mount certificates in nginx service
```

### 4. Database Setup

```bash
# Create production database
sudo -u postgres createdb rustelo_prod
sudo -u postgres createuser rustelo_user

# Run migrations
./deploy.sh migrate -e production
```

### 5. Monitoring Setup

```bash
# Start monitoring stack
docker-compose --profile monitoring up -d

# Configure Grafana
# 1. Open http://your-server:3000
# 2. Login with admin/admin
# 3. Import dashboards from monitoring/grafana/dashboards/
```

### 6. Backup Strategy

```bash
# Create backup script
cat > backup.sh << 'EOF'
#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)
docker-compose exec -T db pg_dump -U postgres rustelo_prod > /opt/backups/rustelo_backup_$DATE.sql
find /opt/backups -name "rustelo_backup_*.sql" -mtime +7 -delete
EOF

# Add to crontab
crontab -e
# Add: 0 2 * * * /opt/rustelo/backup.sh
```

## Troubleshooting

### Common Issues

#### 1. Application Won't Start

```bash
# Check container logs
docker-compose logs app

# Check system resources
docker stats

# Verify environment variables
docker-compose config
```

#### 2. Database Connection Issues

```bash
# Test database connectivity
docker-compose exec app psql $DATABASE_URL -c "SELECT 1"

# Check database logs
docker-compose logs db

# Verify connection pool settings
curl http://localhost:3030/health | jq '.components[] | select(.name == "database")'
```

#### 3. Health Check Failures

```bash
# Check detailed health status
curl -s http://localhost:3030/health | jq .

# Test individual components
curl -s http://localhost:3030/health/live
curl -s http://localhost:3030/health/ready
```

#### 4. Performance Issues

```bash
# Check metrics
curl -s http://localhost:3030/metrics | grep rustelo_

# Monitor resource usage
docker stats --no-stream

# Check for slow queries
docker-compose exec db psql -U postgres -c "SELECT query, calls, total_time FROM pg_stat_statements ORDER BY total_time DESC LIMIT 10;"
```

### Debugging Commands

```bash
# Enter container shell
docker-compose exec app bash

# Check application logs
docker-compose logs -f app

# Test endpoints
curl -v http://localhost:3030/health
curl -v http://localhost:3030/metrics

# Check database
docker-compose exec db psql -U postgres rustelo_prod

# Monitor in real-time
watch -n 5 'docker stats --no-stream'
```

### Recovery Procedures

#### 1. Database Recovery

```bash
# Restore from backup
docker-compose exec -T db psql -U postgres rustelo_prod < backup_file.sql

# Reset database (WARNING: destroys data)
docker-compose down
docker volume rm rustelo_postgres_data
docker-compose up -d db
./deploy.sh migrate -e production
```

#### 2. Application Recovery

```bash
# Restart application
docker-compose restart app

# Full restart
docker-compose down
docker-compose up -d

# Rollback to previous version
docker-compose down
docker-compose pull
docker-compose up -d
```

## Security Considerations

### 1. Container Security

- Use non-root user in containers
- Scan images for vulnerabilities
- Keep base images updated
- Use multi-stage builds to reduce attack surface

### 2. Network Security

- Use internal networks for service communication
- Expose only necessary ports
- Implement rate limiting
- Use TLS for all external communications

### 3. Data Protection

- Encrypt sensitive data at rest
- Use encrypted database connections
- Implement proper backup encryption
- Regular security audits

### 4. Access Control

- Use strong authentication
- Implement role-based access control
- Regular password rotation
- Monitor access logs

## Performance Tuning

### 1. Application Optimization

```toml
# config.toml optimizations
[database]
max_connections = 20
min_connections = 5
connect_timeout = 10

[server]
worker_threads = 4
max_request_size = 1048576

[app]
enable_compression = true
cache_size = 1000
```

### 2. Database Optimization

```sql
-- Create indexes for frequently queried columns
CREATE INDEX idx_users_email ON users(email);
CREATE INDEX idx_content_created_at ON content(created_at);

-- Analyze query performance
EXPLAIN ANALYZE SELECT * FROM users WHERE email = 'user@example.com';
```

### 3. Container Resource Limits

```yaml
services:
  app:
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 1G
        reservations:
          cpus: '1.0'
          memory: 512M
```

## Maintenance

### Regular Tasks

1. **Daily**: Check application logs and health status
2. **Weekly**: Review metrics and performance
3. **Monthly**: Update dependencies and security patches
4. **Quarterly**: Review and update monitoring alerts

### Maintenance Commands

```bash
# Update system packages
sudo apt update && sudo apt upgrade -y

# Update Docker images
docker-compose pull
docker-compose up -d

# Clean up unused resources
docker system prune -f

# Backup database
./deploy.sh backup

# Check for security updates
cargo audit
```

For more detailed information, refer to the individual documentation files:

- [Configuration Guide](CONFIG_README.md)
- [Email Configuration](templates/email/README.md)
- [Security Guide](docs/SECURITY.md)
- [API Documentation](docs/API.md)