Rustelo/config/MIGRATION.md

# Configuration System Migration Guide

This guide explains how to migrate from the old monolithic configuration system to the new modular, environment-aware configuration system.

## Overview of Changes

The configuration system has been completely restructured to provide:

- **Environment-specific configurations** (dev, prod, example)
- **Feature-based modularity** with separate configuration files
- **Automatic configuration building** and validation
- **Backup and management utilities**
- **Better maintainability** and separation of concerns

## Before and After

### Old Structure (Before)
```
config/
├── config.toml                 # Monolithic config file
├── config.dev.toml            # Development overrides
├── config.prod.toml           # Production overrides
├── config.example.toml        # Example configuration
└── features/
    ├── auth.toml              # Feature configs (not environment-specific)
    ├── email.toml
    ├── tls.toml
    └── ...
```

### New Structure (After)
```
config/
├── base/                      # Base configurations by environment
│   ├── dev.toml               # Development base settings
│   ├── prod.toml              # Production base settings
│   └── example.toml           # Example base settings
├── features/                  # Feature configurations by environment
│   ├── auth/
│   │   ├── dev.toml           # Auth settings for dev
│   │   ├── prod.toml          # Auth settings for prod
│   │   └── example.toml       # Auth example settings
│   ├── email/
│   │   ├── dev.toml
│   │   ├── prod.toml
│   │   └── example.toml
│   └── ...
├── scripts/                   # Configuration management tools
│   ├── build-config.sh        # Build complete configurations
│   ├── manage-config.sh       # Management utilities
│   └── test-config.sh         # Test suite
└── README.md                  # Documentation
```

## Migration Steps

### Step 1: Backup Current Configuration

First, backup your existing configuration files:

```bash
# Create backup directory
mkdir -p config/backup_$(date +%Y%m%d)

# Backup existing files
cp config.toml config/backup_$(date +%Y%m%d)/
cp config.*.toml config/backup_$(date +%Y%m%d)/ 2>/dev/null || true
cp -r config/features config/backup_$(date +%Y%m%d)/ 2>/dev/null || true
```

### Step 2: Analyze Current Configuration

Review your current `config.toml` to understand which features are enabled:

```bash
# Check feature flags
grep -E "^\[features\]" -A 20 config.toml

# Check enabled features
grep -E "^[a-zA-Z_]+ = true" config.toml | grep -v "#"
```

### Step 3: Create Base Configurations

Split your monolithic configuration into environment-specific base files:

#### For Development (`config/base/dev.toml`)
Extract development-specific settings:
- Debug mode enabled
- Relaxed security settings
- Local database connections
- Verbose logging
- Development-friendly timeouts

#### For Production (`config/base/prod.toml`)
Extract production-specific settings:
- Debug mode disabled
- Strict security settings
- Production database connections
- Minimal logging
- Production-optimized timeouts

#### For Example (`config/base/example.toml`)
Create comprehensive example with:
- All available options documented
- Best practice configurations
- Commented examples

### Step 4: Create Feature Configurations

For each feature in your system, create environment-specific configurations:

#### Example: Authentication Feature

**Development** (`config/features/auth/dev.toml`):
```toml
[features]
auth = true

[auth.security]
max_login_attempts = 10
lockout_duration = 300
require_email_verification = false

[auth.password]
min_length = 6
require_uppercase = false
```

**Production** (`config/features/auth/prod.toml`):
```toml
[features]
auth = true

[auth.security]
max_login_attempts = 3
lockout_duration = 1800
require_email_verification = true

[auth.password]
min_length = 12
require_uppercase = true
```

### Step 5: Set Up Build Scripts

Make the build scripts executable:

```bash
chmod +x config/scripts/build-config.sh
chmod +x config/scripts/manage-config.sh
```

### Step 6: Test the New System

Build and validate configurations for each environment:

```bash
# Test development configuration
./config/scripts/build-config.sh dev config.dev.toml

# Test production configuration
./config/scripts/build-config.sh prod config.prod.toml

# Validate configurations
./config/scripts/manage-config.sh validate dev
./config/scripts/manage-config.sh validate prod
```

### Step 7: Update Build Process

Update your build/deployment scripts to use the new configuration system:

#### Development
```bash
# Build development config
./config/scripts/build-config.sh dev config.toml

# Start application
cargo run
```

#### Production
```bash
# Build production config
./config/scripts/build-config.sh prod config.toml

# Deploy application
./deploy.sh
```

### Step 8: Clean Up Old Files

Once the new system is working, remove old configuration files:

```bash
# Remove old monolithic configs (after backing up)
rm config.dev.toml config.prod.toml config.example.toml

# Remove old feature configs
rm config/features/*.toml

# Keep only the new structure
```

## Feature Migration Examples

### Migrating Authentication Feature

**Old** (`config/features/auth.toml`):
```toml
[features]
auth = true

[oauth]
enabled = false

[auth.jwt]
secret = "development-secret"
expiration = 86400

[auth.password]
min_length = 8
require_uppercase = true
```

**New** - Split into environments:

**Development** (`config/features/auth/dev.toml`):
```toml
[features]
auth = true

[oauth]
enabled = false

[auth.jwt]
secret = "dev-jwt-secret"
expiration = 86400

[auth.password]
min_length = 6
require_uppercase = false
```

**Production** (`config/features/auth/prod.toml`):
```toml
[features]
auth = true

[oauth]
enabled = true

[auth.jwt]
secret = "${JWT_SECRET}"
expiration = 3600

[auth.password]
min_length = 12
require_uppercase = true
```

### Migrating Email Feature

**Old** (`config/features/email.toml`):
```toml
[features]
email = true

[email]
enabled = true
default_provider = "console"

[email.smtp]
host = "smtp.gmail.com"
port = 587
```

**New** - Split into environments:

**Development** (`config/features/email/dev.toml`):
```toml
[features]
email = true

[email]
enabled = true
default_provider = "console"

[email.console]
enabled = true
save_to_file = true
```

**Production** (`config/features/email/prod.toml`):
```toml
[features]
email = true

[email]
enabled = true
default_provider = "sendgrid"

[email.sendgrid]
api_key = "${SENDGRID_API_KEY}"
```

## Environment Variables

The new system makes extensive use of environment variables for sensitive data:

### Development
```bash
# Database
export DATABASE_URL="sqlite://dev_database.db"

# Session
export SESSION_SECRET="dev-session-secret"

# JWT
export JWT_SECRET="dev-jwt-secret"
```

### Production
```bash
# Database
export DATABASE_URL="postgresql://user:pass@localhost/prod_db"

# Session
export SESSION_SECRET="$(openssl rand -base64 32)"

# JWT
export JWT_SECRET="$(openssl rand -base64 32)"

# Email
export SENDGRID_API_KEY="your-sendgrid-api-key"

# TLS
export TLS_CERT_PATH="/path/to/cert.pem"
export TLS_KEY_PATH="/path/to/key.pem"
```

## Validation and Testing

### Automated Testing
```bash
# Run the test suite
./config/scripts/test-config.sh

# Test specific environment
./config/scripts/manage-config.sh validate dev
```

### Manual Validation
```bash
# Compare configurations
./config/scripts/manage-config.sh diff dev prod

# Check configuration status
./config/scripts/manage-config.sh status
```

## Rollback Procedure

If you need to rollback to the old system:

1. **Restore backup files**:
```bash
cp config/backup_*/config.toml .
cp config/backup_*/config.*.toml .
```

2. **Revert build scripts**: Update your build process to use the old configuration files directly.

3. **Update application code**: If you made changes to configuration loading, revert those changes.

## Benefits of the New System

1. **Environment Separation**: Clear separation between development and production settings
2. **Feature Modularity**: Each feature can be configured independently
3. **Maintainability**: Easier to maintain and update specific features
4. **Validation**: Built-in validation ensures configuration correctness
5. **Backup**: Automatic backup of existing configurations
6. **Extensibility**: Easy to add new features and environments

## Troubleshooting

### Common Issues

#### Missing Environment Variables
```bash
# Check for undefined variables
grep -r "\${" config/features/*/prod.toml
```

#### Invalid TOML Syntax
```bash
# Install TOML validator
cargo install toml-cli

# Validate files
toml get config.toml > /dev/null
```

#### Build Failures
```bash
# Enable debug mode
CONFIG_DEBUG=1 ./config/scripts/build-config.sh dev
```

### Getting Help

1. Check the `config/README.md` for detailed documentation
2. Run `./config/scripts/manage-config.sh help` for command usage
3. Use `./config/scripts/demo-config.sh` to see the system in action
4. Review the test suite in `./config/scripts/test-config.sh`

## Next Steps

After successful migration:

1. **Update Documentation**: Update your project documentation to reflect the new configuration system
2. **Train Team**: Ensure all team members understand the new system
3. **CI/CD Integration**: Update your CI/CD pipelines to use the new build process
4. **Monitoring**: Set up monitoring to ensure configurations are built correctly in deployment

The new configuration system provides a much more maintainable and scalable approach to managing application settings across different environments.