Rustelo/info/migration_guide.md

Migration Guide: Environment Variables to TOML Configuration

This guide helps you migrate from the old environment variable-only configuration system to the new TOML-based configuration system with environment variable overrides.

Overview

The new configuration system provides:

• TOML files for structured configuration
• Environment variable overrides for sensitive data
• Environment-specific configs (dev, prod, etc.)
• Validation and error handling
• Better organization of settings

Migration Steps

Step 1: Identify Current Configuration

First, identify all environment variables currently used in your application:

# List all environment variables starting with common prefixes
env | grep -E "^(SERVER_|DATABASE_|SESSION_|CORS_|TLS_|OAUTH_|SMTP_|REDIS_|LOG_)" | sort

Step 2: Create Base Configuration File

Create a config.toml file with your current settings:

# config.toml
[server]
protocol = "http"
host = "127.0.0.1"
port = 3030
environment = "development"
log_level = "info"

[database]
url = "postgresql://localhost:5432/myapp"
max_connections = 10
min_connections = 1
connect_timeout = 30
idle_timeout = 600
max_lifetime = 1800

[session]
secret = "change-this-in-production"
cookie_name = "session_id"
cookie_secure = false
cookie_http_only = true
cookie_same_site = "lax"
max_age = 3600

[cors]
allowed_origins = ["http://localhost:3030"]
allowed_methods = ["GET", "POST", "PUT", "DELETE", "OPTIONS"]
allowed_headers = ["Content-Type", "Authorization"]
allow_credentials = true
max_age = 3600

[security]
enable_csrf = true
csrf_token_name = "csrf_token"
rate_limit_requests = 100
rate_limit_window = 60
bcrypt_cost = 12

[static]
assets_dir = "public"
site_root = "target/site"
site_pkg_dir = "pkg"

[oauth]
enabled = false

[email]
enabled = false
smtp_host = "localhost"
smtp_port = 587
smtp_username = ""
smtp_password = ""
from_email = "noreply@example.com"
from_name = "My App"

[redis]
enabled = false
url = "redis://localhost:6379"
pool_size = 10
connection_timeout = 5
command_timeout = 5

[app]
name = "My Rust App"
version = "0.1.0"
debug = true
enable_metrics = false
enable_health_check = true
enable_compression = true
max_request_size = 10485760

[logging]
format = "text"
level = "info"
file_path = "logs/app.log"
max_file_size = 10485760
max_files = 5
enable_console = true
enable_file = false

[content]
enabled = false
content_dir = "content"
cache_enabled = true
cache_ttl = 3600
max_file_size = 5242880

[features]
auth = true
tls = false
content_db = true
two_factor_auth = false

Step 3: Update Code to Use New Configuration

Replace the old configuration loading:

// OLD: Environment-only configuration
use config::ServerConfig;

let server_config = ServerConfig::from_env()?;
let addr = server_config.server_address();
let log_level = server_config.log_level;

With the new configuration system:

// NEW: TOML + Environment configuration
use config::Config;

let config = Config::load()?;
let addr = config.server_address();
let log_level = config.server.log_level;

Step 4: Environment Variable Mapping

Map your existing environment variables to the new system:

Old Environment Variable	New TOML Location	Environment Override
SERVER_HOST	server.host	SERVER_HOST
SERVER_PORT	server.port	SERVER_PORT
SERVER_PROTOCOL	server.protocol	SERVER_PROTOCOL
DATABASE_URL	database.url	DATABASE_URL
SESSION_SECRET	session.secret	SESSION_SECRET
LOG_LEVEL	server.log_level	LOG_LEVEL
ENVIRONMENT	server.environment	ENVIRONMENT
TLS_CERT_PATH	server.tls.cert_path	TLS_CERT_PATH
TLS_KEY_PATH	server.tls.key_path	TLS_KEY_PATH

Step 5: Handle Sensitive Data

Move sensitive data to environment variables and use substitution:

# config.toml - Use environment variable substitution
[database]
url = "postgresql://user:${DATABASE_PASSWORD}@localhost:5432/myapp"

[session]
secret = "${SESSION_SECRET}"

[oauth.google]
client_id = "${GOOGLE_CLIENT_ID}"
client_secret = "${GOOGLE_CLIENT_SECRET}"

[email]
smtp_username = "${SMTP_USERNAME}"
smtp_password = "${SMTP_PASSWORD}"

Step 6: Create Environment-Specific Configurations

Create separate configuration files for different environments:

config.dev.toml:

[server]
protocol = "http"
host = "127.0.0.1"
port = 3030
environment = "development"
log_level = "debug"

[database]
url = "postgresql://dev:dev@localhost:5432/myapp_dev"
max_connections = 5

[security]
enable_csrf = false
rate_limit_requests = 1000
bcrypt_cost = 4

[session]
cookie_secure = false
max_age = 7200

config.prod.toml:

[server]
protocol = "https"
host = "0.0.0.0"
port = 443
environment = "production"
log_level = "info"

[server.tls]
cert_path = "/etc/ssl/certs/app.crt"
key_path = "/etc/ssl/private/app.key"

[database]
url = "postgresql://prod:${DATABASE_PASSWORD}@db.example.com:5432/myapp_prod"
max_connections = 20

[security]
enable_csrf = true
rate_limit_requests = 50
bcrypt_cost = 12

[session]
secret = "${SESSION_SECRET}"
cookie_secure = true
cookie_same_site = "strict"
max_age = 3600

Step 7: Update Deployment Scripts

Update your deployment scripts to use the new configuration system:

Docker:

# OLD
ENV SERVER_HOST=0.0.0.0
ENV SERVER_PORT=8080
ENV DATABASE_URL=postgresql://...
ENV SESSION_SECRET=...

# NEW
COPY config.prod.toml /app/config.toml
ENV ENVIRONMENT=production
ENV DATABASE_PASSWORD=...
ENV SESSION_SECRET=...

Kubernetes:

# OLD
env:
  - name: SERVER_HOST
    value: "0.0.0.0"
  - name: SERVER_PORT
    value: "8080"
  - name: DATABASE_URL
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: database-url

# NEW
env:
  - name: ENVIRONMENT
    value: "production"
  - name: DATABASE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: database-password
  - name: SESSION_SECRET
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: session-secret

Step 8: Update Environment Files

Update your .env files to work with the new system:

.env.development:

# Environment
ENVIRONMENT=development

# Database
DATABASE_URL=postgresql://dev:dev@localhost:5432/myapp_dev

# Session
SESSION_SECRET=dev-secret-not-for-production

# OAuth (optional)
GOOGLE_CLIENT_ID=your-dev-google-client-id
GOOGLE_CLIENT_SECRET=your-dev-google-client-secret

.env.production:

# Environment
ENVIRONMENT=production

# Database
DATABASE_PASSWORD=your-production-database-password

# Session
SESSION_SECRET=your-super-secret-production-key

# OAuth
GOOGLE_CLIENT_ID=your-production-google-client-id
GOOGLE_CLIENT_SECRET=your-production-google-client-secret

Step 9: Test the Migration

1. Validate configuration:

   cargo run --bin config_tool -- validate
   

2. Show current configuration:

   cargo run --bin config_tool -- show
   

3. Check environment variables:

   cargo run --bin config_tool -- check-env
   

4. Run your application:

   cargo run
   

Step 10: Update Documentation

Update your project documentation to reflect the new configuration system:

1. Update README.md with configuration instructions
2. Document required environment variables
3. Provide example configuration files
4. Update deployment guides

Common Migration Issues

Issue 1: Configuration Not Found

Error:

Configuration file not found: config.toml

Solution: Create a configuration file or set the CONFIG_FILE environment variable:

cp config.dev.toml config.toml
# or
export CONFIG_FILE=/path/to/your/config.toml

Issue 2: Environment Variable Substitution

Error:

Environment variable 'DATABASE_PASSWORD' not found

Solution: Set the required environment variable:

export DATABASE_PASSWORD=your-password

Issue 3: TLS Configuration

Error:

TLS certificate path is required when using HTTPS

Solution: Either disable HTTPS or provide certificate paths:

[server]
protocol = "http"  # Disable HTTPS
# or
protocol = "https"
[server.tls]
cert_path = "/path/to/cert.crt"
key_path = "/path/to/key.key"

Issue 4: Database Connection

Error:

Failed to connect to database

Solution: Check your database URL format and ensure the database is running:

[database]
url = "postgresql://username:password@host:port/database"

Migration Checklist

• Identify all current environment variables
• Create base config.toml file
• Update code to use Config::load()
• Create environment-specific config files
• Move sensitive data to environment variables
• Update deployment scripts
• Update .env files
• Test configuration loading
• Validate configuration
• Update documentation
• Update CI/CD pipelines
• Train team on new configuration system

Rollback Plan

If you need to rollback to the old system:

1. Keep the old configuration loading code in a separate branch
2. Maintain both systems during transition period
3. Use feature flags to switch between systems
4. Document the rollback process

// Rollback configuration loading
#[cfg(feature = "legacy-config")]
let config = ServerConfig::from_env()?;

#[cfg(not(feature = "legacy-config"))]
let config = Config::load()?;

Best Practices After Migration

1. Version control: Keep configuration files in version control (except sensitive production configs)
2. Environment parity: Ensure dev/staging/prod configurations are consistent
3. Documentation: Keep configuration documentation up to date
4. Validation: Regularly validate configuration files
5. Secrets management: Use proper secrets management for production
6. Monitoring: Monitor configuration changes in production
7. Testing: Test configuration loading in CI/CD
8. Backup: Backup configuration files regularly

Getting Help

If you encounter issues during migration:

1. Run the configuration tool: cargo run --bin config_tool -- help
2. Check the configuration examples in the repository
3. Review the CONFIG_README.md for detailed documentation
4. Open an issue on the project repository