Rustelo/config/MIGRATION.md

Configuration System Migration Guide

This guide explains how to migrate from the old monolithic configuration system to the new modular, environment-aware configuration system.

Overview of Changes

The configuration system has been completely restructured to provide:

• Environment-specific configurations (dev, prod, example)
• Feature-based modularity with separate configuration files
• Automatic configuration building and validation
• Backup and management utilities
• Better maintainability and separation of concerns

Before and After

Old Structure (Before)

config/
├── config.toml                 # Monolithic config file
├── config.dev.toml            # Development overrides
├── config.prod.toml           # Production overrides
├── config.example.toml        # Example configuration
└── features/
    ├── auth.toml              # Feature configs (not environment-specific)
    ├── email.toml
    ├── tls.toml
    └── ...

New Structure (After)

config/
├── base/                      # Base configurations by environment
│   ├── dev.toml               # Development base settings
│   ├── prod.toml              # Production base settings
│   └── example.toml           # Example base settings
├── features/                  # Feature configurations by environment
│   ├── auth/
│   │   ├── dev.toml           # Auth settings for dev
│   │   ├── prod.toml          # Auth settings for prod
│   │   └── example.toml       # Auth example settings
│   ├── email/
│   │   ├── dev.toml
│   │   ├── prod.toml
│   │   └── example.toml
│   └── ...
├── scripts/                   # Configuration management tools
│   ├── build-config.sh        # Build complete configurations
│   ├── manage-config.sh       # Management utilities
│   └── test-config.sh         # Test suite
└── README.md                  # Documentation

Migration Steps

Step 1: Backup Current Configuration

First, backup your existing configuration files:

# Create backup directory
mkdir -p config/backup_$(date +%Y%m%d)

# Backup existing files
cp config.toml config/backup_$(date +%Y%m%d)/
cp config.*.toml config/backup_$(date +%Y%m%d)/ 2>/dev/null || true
cp -r config/features config/backup_$(date +%Y%m%d)/ 2>/dev/null || true

Step 2: Analyze Current Configuration

Review your current config.toml to understand which features are enabled:

# Check feature flags
grep -E "^\[features\]" -A 20 config.toml

# Check enabled features
grep -E "^[a-zA-Z_]+ = true" config.toml | grep -v "#"

Step 3: Create Base Configurations

Split your monolithic configuration into environment-specific base files:

For Development (config/base/dev.toml)

Extract development-specific settings:

• Debug mode enabled
• Relaxed security settings
• Local database connections
• Verbose logging
• Development-friendly timeouts

For Production (config/base/prod.toml)

Extract production-specific settings:

• Debug mode disabled
• Strict security settings
• Production database connections
• Minimal logging
• Production-optimized timeouts

For Example (config/base/example.toml)

Create comprehensive example with:

• All available options documented
• Best practice configurations
• Commented examples

Step 4: Create Feature Configurations

For each feature in your system, create environment-specific configurations:

Example: Authentication Feature

Development (config/features/auth/dev.toml):

[features]
auth = true

[auth.security]
max_login_attempts = 10
lockout_duration = 300
require_email_verification = false

[auth.password]
min_length = 6
require_uppercase = false

Production (config/features/auth/prod.toml):

[features]
auth = true

[auth.security]
max_login_attempts = 3
lockout_duration = 1800
require_email_verification = true

[auth.password]
min_length = 12
require_uppercase = true

Step 5: Set Up Build Scripts

Make the build scripts executable:

chmod +x config/scripts/build-config.sh
chmod +x config/scripts/manage-config.sh

Step 6: Test the New System

Build and validate configurations for each environment:

# Test development configuration
./config/scripts/build-config.sh dev config.dev.toml

# Test production configuration
./config/scripts/build-config.sh prod config.prod.toml

# Validate configurations
./config/scripts/manage-config.sh validate dev
./config/scripts/manage-config.sh validate prod

Step 7: Update Build Process

Update your build/deployment scripts to use the new configuration system:

Development

# Build development config
./config/scripts/build-config.sh dev config.toml

# Start application
cargo run

Production

# Build production config
./config/scripts/build-config.sh prod config.toml

# Deploy application
./deploy.sh

Step 8: Clean Up Old Files

Once the new system is working, remove old configuration files:

# Remove old monolithic configs (after backing up)
rm config.dev.toml config.prod.toml config.example.toml

# Remove old feature configs
rm config/features/*.toml

# Keep only the new structure

Feature Migration Examples

Migrating Authentication Feature

Old (config/features/auth.toml):

[features]
auth = true

[oauth]
enabled = false

[auth.jwt]
secret = "development-secret"
expiration = 86400

[auth.password]
min_length = 8
require_uppercase = true

New - Split into environments:

Development (config/features/auth/dev.toml):

[features]
auth = true

[oauth]
enabled = false

[auth.jwt]
secret = "dev-jwt-secret"
expiration = 86400

[auth.password]
min_length = 6
require_uppercase = false

Production (config/features/auth/prod.toml):

[features]
auth = true

[oauth]
enabled = true

[auth.jwt]
secret = "${JWT_SECRET}"
expiration = 3600

[auth.password]
min_length = 12
require_uppercase = true

Migrating Email Feature

Old (config/features/email.toml):

[features]
email = true

[email]
enabled = true
default_provider = "console"

[email.smtp]
host = "smtp.gmail.com"
port = 587

New - Split into environments:

Development (config/features/email/dev.toml):

[features]
email = true

[email]
enabled = true
default_provider = "console"

[email.console]
enabled = true
save_to_file = true

Production (config/features/email/prod.toml):

[features]
email = true

[email]
enabled = true
default_provider = "sendgrid"

[email.sendgrid]
api_key = "${SENDGRID_API_KEY}"

Environment Variables

The new system makes extensive use of environment variables for sensitive data:

Development

# Database
export DATABASE_URL="sqlite://dev_database.db"

# Session
export SESSION_SECRET="dev-session-secret"

# JWT
export JWT_SECRET="dev-jwt-secret"

Production

# Database
export DATABASE_URL="postgresql://user:pass@localhost/prod_db"

# Session
export SESSION_SECRET="$(openssl rand -base64 32)"

# JWT
export JWT_SECRET="$(openssl rand -base64 32)"

# Email
export SENDGRID_API_KEY="your-sendgrid-api-key"

# TLS
export TLS_CERT_PATH="/path/to/cert.pem"
export TLS_KEY_PATH="/path/to/key.pem"

Validation and Testing

Automated Testing

# Run the test suite
./config/scripts/test-config.sh

# Test specific environment
./config/scripts/manage-config.sh validate dev

Manual Validation

# Compare configurations
./config/scripts/manage-config.sh diff dev prod

# Check configuration status
./config/scripts/manage-config.sh status

Rollback Procedure

If you need to rollback to the old system:

1. Restore backup files:

cp config/backup_*/config.toml .
cp config/backup_*/config.*.toml .

2. Revert build scripts: Update your build process to use the old configuration files directly.

3. Update application code: If you made changes to configuration loading, revert those changes.

Benefits of the New System

1. Environment Separation: Clear separation between development and production settings
2. Feature Modularity: Each feature can be configured independently
3. Maintainability: Easier to maintain and update specific features
4. Validation: Built-in validation ensures configuration correctness
5. Backup: Automatic backup of existing configurations
6. Extensibility: Easy to add new features and environments

Troubleshooting

Common Issues

Missing Environment Variables

# Check for undefined variables
grep -r "\${" config/features/*/prod.toml

Invalid TOML Syntax

# Install TOML validator
cargo install toml-cli

# Validate files
toml get config.toml > /dev/null

Build Failures

# Enable debug mode
CONFIG_DEBUG=1 ./config/scripts/build-config.sh dev

Getting Help

1. Check the config/README.md for detailed documentation
2. Run ./config/scripts/manage-config.sh help for command usage
3. Use ./config/scripts/demo-config.sh to see the system in action
4. Review the test suite in ./config/scripts/test-config.sh

Next Steps

After successful migration:

1. Update Documentation: Update your project documentation to reflect the new configuration system
2. Train Team: Ensure all team members understand the new system
3. CI/CD Integration: Update your CI/CD pipelines to use the new build process
4. Monitoring: Set up monitoring to ensure configurations are built correctly in deployment

The new configuration system provides a much more maintainable and scalable approach to managing application settings across different environments.