Rustelo/info/migration_guide.md

# Migration Guide: Environment Variables to TOML Configuration

This guide helps you migrate from the old environment variable-only configuration system to the new TOML-based configuration system with environment variable overrides.

## Overview

The new configuration system provides:
- **TOML files** for structured configuration
- **Environment variable overrides** for sensitive data
- **Environment-specific configs** (dev, prod, etc.)
- **Validation and error handling**
- **Better organization** of settings

## Migration Steps

### Step 1: Identify Current Configuration

First, identify all environment variables currently used in your application:

```bash
# List all environment variables starting with common prefixes
env | grep -E "^(SERVER_|DATABASE_|SESSION_|CORS_|TLS_|OAUTH_|SMTP_|REDIS_|LOG_)" | sort
```

### Step 2: Create Base Configuration File

Create a `config.toml` file with your current settings:

```toml
# config.toml
[server]
protocol = "http"
host = "127.0.0.1"
port = 3030
environment = "development"
log_level = "info"

[database]
url = "postgresql://localhost:5432/myapp"
max_connections = 10
min_connections = 1
connect_timeout = 30
idle_timeout = 600
max_lifetime = 1800

[session]
secret = "change-this-in-production"
cookie_name = "session_id"
cookie_secure = false
cookie_http_only = true
cookie_same_site = "lax"
max_age = 3600

[cors]
allowed_origins = ["http://localhost:3030"]
allowed_methods = ["GET", "POST", "PUT", "DELETE", "OPTIONS"]
allowed_headers = ["Content-Type", "Authorization"]
allow_credentials = true
max_age = 3600

[security]
enable_csrf = true
csrf_token_name = "csrf_token"
rate_limit_requests = 100
rate_limit_window = 60
bcrypt_cost = 12

[static]
assets_dir = "public"
site_root = "target/site"
site_pkg_dir = "pkg"

[oauth]
enabled = false

[email]
enabled = false
smtp_host = "localhost"
smtp_port = 587
smtp_username = ""
smtp_password = ""
from_email = "noreply@example.com"
from_name = "My App"

[redis]
enabled = false
url = "redis://localhost:6379"
pool_size = 10
connection_timeout = 5
command_timeout = 5

[app]
name = "My Rust App"
version = "0.1.0"
debug = true
enable_metrics = false
enable_health_check = true
enable_compression = true
max_request_size = 10485760

[logging]
format = "text"
level = "info"
file_path = "logs/app.log"
max_file_size = 10485760
max_files = 5
enable_console = true
enable_file = false

[content]
enabled = false
content_dir = "content"
cache_enabled = true
cache_ttl = 3600
max_file_size = 5242880

[features]
auth = true
tls = false
content_db = true
two_factor_auth = false
```

### Step 3: Update Code to Use New Configuration

Replace the old configuration loading:

```rust
// OLD: Environment-only configuration
use config::ServerConfig;

let server_config = ServerConfig::from_env()?;
let addr = server_config.server_address();
let log_level = server_config.log_level;
```

With the new configuration system:

```rust
// NEW: TOML + Environment configuration
use config::Config;

let config = Config::load()?;
let addr = config.server_address();
let log_level = config.server.log_level;
```

### Step 4: Environment Variable Mapping

Map your existing environment variables to the new system:

| Old Environment Variable | New TOML Location | Environment Override |
|-------------------------|-------------------|---------------------|
| `SERVER_HOST` | `server.host` | `SERVER_HOST` |
| `SERVER_PORT` | `server.port` | `SERVER_PORT` |
| `SERVER_PROTOCOL` | `server.protocol` | `SERVER_PROTOCOL` |
| `DATABASE_URL` | `database.url` | `DATABASE_URL` |
| `SESSION_SECRET` | `session.secret` | `SESSION_SECRET` |
| `LOG_LEVEL` | `server.log_level` | `LOG_LEVEL` |
| `ENVIRONMENT` | `server.environment` | `ENVIRONMENT` |
| `TLS_CERT_PATH` | `server.tls.cert_path` | `TLS_CERT_PATH` |
| `TLS_KEY_PATH` | `server.tls.key_path` | `TLS_KEY_PATH` |

### Step 5: Handle Sensitive Data

Move sensitive data to environment variables and use substitution:

```toml
# config.toml - Use environment variable substitution
[database]
url = "postgresql://user:${DATABASE_PASSWORD}@localhost:5432/myapp"

[session]
secret = "${SESSION_SECRET}"

[oauth.google]
client_id = "${GOOGLE_CLIENT_ID}"
client_secret = "${GOOGLE_CLIENT_SECRET}"

[email]
smtp_username = "${SMTP_USERNAME}"
smtp_password = "${SMTP_PASSWORD}"
```

### Step 6: Create Environment-Specific Configurations

Create separate configuration files for different environments:

**config.dev.toml:**
```toml
[server]
protocol = "http"
host = "127.0.0.1"
port = 3030
environment = "development"
log_level = "debug"

[database]
url = "postgresql://dev:dev@localhost:5432/myapp_dev"
max_connections = 5

[security]
enable_csrf = false
rate_limit_requests = 1000
bcrypt_cost = 4

[session]
cookie_secure = false
max_age = 7200
```

**config.prod.toml:**
```toml
[server]
protocol = "https"
host = "0.0.0.0"
port = 443
environment = "production"
log_level = "info"

[server.tls]
cert_path = "/etc/ssl/certs/app.crt"
key_path = "/etc/ssl/private/app.key"

[database]
url = "postgresql://prod:${DATABASE_PASSWORD}@db.example.com:5432/myapp_prod"
max_connections = 20

[security]
enable_csrf = true
rate_limit_requests = 50
bcrypt_cost = 12

[session]
secret = "${SESSION_SECRET}"
cookie_secure = true
cookie_same_site = "strict"
max_age = 3600
```

### Step 7: Update Deployment Scripts

Update your deployment scripts to use the new configuration system:

**Docker:**
```dockerfile
# OLD
ENV SERVER_HOST=0.0.0.0
ENV SERVER_PORT=8080
ENV DATABASE_URL=postgresql://...
ENV SESSION_SECRET=...

# NEW
COPY config.prod.toml /app/config.toml
ENV ENVIRONMENT=production
ENV DATABASE_PASSWORD=...
ENV SESSION_SECRET=...
```

**Kubernetes:**
```yaml
# OLD
env:
  - name: SERVER_HOST
    value: "0.0.0.0"
  - name: SERVER_PORT
    value: "8080"
  - name: DATABASE_URL
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: database-url

# NEW
env:
  - name: ENVIRONMENT
    value: "production"
  - name: DATABASE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: database-password
  - name: SESSION_SECRET
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: session-secret
```

### Step 8: Update Environment Files

Update your `.env` files to work with the new system:

**.env.development:**
```bash
# Environment
ENVIRONMENT=development

# Database
DATABASE_URL=postgresql://dev:dev@localhost:5432/myapp_dev

# Session
SESSION_SECRET=dev-secret-not-for-production

# OAuth (optional)
GOOGLE_CLIENT_ID=your-dev-google-client-id
GOOGLE_CLIENT_SECRET=your-dev-google-client-secret
```

**.env.production:**
```bash
# Environment
ENVIRONMENT=production

# Database
DATABASE_PASSWORD=your-production-database-password

# Session
SESSION_SECRET=your-super-secret-production-key

# OAuth
GOOGLE_CLIENT_ID=your-production-google-client-id
GOOGLE_CLIENT_SECRET=your-production-google-client-secret
```

### Step 9: Test the Migration

1. **Validate configuration:**
   ```bash
   cargo run --bin config_tool -- validate
   ```

2. **Show current configuration:**
   ```bash
   cargo run --bin config_tool -- show
   ```

3. **Check environment variables:**
   ```bash
   cargo run --bin config_tool -- check-env
   ```

4. **Run your application:**
   ```bash
   cargo run
   ```

### Step 10: Update Documentation

Update your project documentation to reflect the new configuration system:

1. Update README.md with configuration instructions
2. Document required environment variables
3. Provide example configuration files
4. Update deployment guides

## Common Migration Issues

### Issue 1: Configuration Not Found

**Error:**
```
Configuration file not found: config.toml
```

**Solution:**
Create a configuration file or set the `CONFIG_FILE` environment variable:
```bash
cp config.dev.toml config.toml
# or
export CONFIG_FILE=/path/to/your/config.toml
```

### Issue 2: Environment Variable Substitution

**Error:**
```
Environment variable 'DATABASE_PASSWORD' not found
```

**Solution:**
Set the required environment variable:
```bash
export DATABASE_PASSWORD=your-password
```

### Issue 3: TLS Configuration

**Error:**
```
TLS certificate path is required when using HTTPS
```

**Solution:**
Either disable HTTPS or provide certificate paths:
```toml
[server]
protocol = "http"  # Disable HTTPS
# or
protocol = "https"
[server.tls]
cert_path = "/path/to/cert.crt"
key_path = "/path/to/key.key"
```

### Issue 4: Database Connection

**Error:**
```
Failed to connect to database
```

**Solution:**
Check your database URL format and ensure the database is running:
```toml
[database]
url = "postgresql://username:password@host:port/database"
```

## Migration Checklist

- [ ] Identify all current environment variables
- [ ] Create base `config.toml` file
- [ ] Update code to use `Config::load()`
- [ ] Create environment-specific config files
- [ ] Move sensitive data to environment variables
- [ ] Update deployment scripts
- [ ] Update `.env` files
- [ ] Test configuration loading
- [ ] Validate configuration
- [ ] Update documentation
- [ ] Update CI/CD pipelines
- [ ] Train team on new configuration system

## Rollback Plan

If you need to rollback to the old system:

1. Keep the old configuration loading code in a separate branch
2. Maintain both systems during transition period
3. Use feature flags to switch between systems
4. Document the rollback process

```rust
// Rollback configuration loading
#[cfg(feature = "legacy-config")]
let config = ServerConfig::from_env()?;

#[cfg(not(feature = "legacy-config"))]
let config = Config::load()?;
```

## Best Practices After Migration

1. **Version control:** Keep configuration files in version control (except sensitive production configs)
2. **Environment parity:** Ensure dev/staging/prod configurations are consistent
3. **Documentation:** Keep configuration documentation up to date
4. **Validation:** Regularly validate configuration files
5. **Secrets management:** Use proper secrets management for production
6. **Monitoring:** Monitor configuration changes in production
7. **Testing:** Test configuration loading in CI/CD
8. **Backup:** Backup configuration files regularly

## Getting Help

If you encounter issues during migration:

1. Run the configuration tool: `cargo run --bin config_tool -- help`
2. Check the configuration examples in the repository
3. Review the CONFIG_README.md for detailed documentation
4. Open an issue on the project repository