provisioning/docs/MIGRATION_GUIDE.md

# Migration Guide: Target-Based Configuration System

## Overview

This guide walks through migrating from the old `config.defaults.toml` system to the new workspace-based target configuration system.

## Migration Path

```
Old System                          New System
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
config.defaults.toml          →     ~/workspaces/{name}/config/provisioning.yaml
config.user.toml              →     ~/Library/Application Support/provisioning/ws_{name}.yaml
providers/{name}/config       →     ~/workspaces/{name}/config/providers/{name}.toml
                              →     ~/workspaces/{name}/config/platform/{service}.toml
```

## Step-by-Step Migration

### 1. Pre-Migration Check

```bash
# Check current configuration
provisioning env

# Backup current configuration
cp -r provisioning/config provisioning/config.backup.$(date +%Y%m%d)
```

### 2. Run Migration Script (Dry Run)

```bash
# Preview what will be done
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "my-project" \
  --dry-run
```

### 3. Execute Migration

```bash
# Run with backup
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "my-project" \
  --backup

# Or specify custom workspace path
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "my-project" \
  --workspace-path "$HOME/my-custom-path" \
  --backup
```

### 4. Verify Migration

```bash
# Validate workspace configuration
provisioning workspace config validate

# Check workspace status
provisioning workspace info

# List all workspaces
provisioning workspace list
```

### 5. Test Configuration

```bash
# Test with new configuration
provisioning --check server list

# Test provider configuration
provisioning provider validate aws

# Test platform configuration
provisioning platform orchestrator status
```

### 6. Update Environment Variables (if any)

```bash
# Old approach (no longer needed)
# export PROVISIONING_CONFIG_PATH="/path/to/config.defaults.toml"

# New approach - workspace is auto-detected from context
# Or set explicitly:
export PROVISIONING_WORKSPACE="my-project"
```

### 7. Clean Up Old Configuration

```bash
# After verifying everything works
rm provisioning/config/config.defaults.toml
rm provisioning/config/config.user.toml

# Keep backup for reference
# provisioning/config.backup.YYYYMMDD/
```

## Migration Script Options

### Required Arguments
- `--workspace-name`: Name for the new workspace (default: "default")

### Optional Arguments
- `--workspace-path`: Custom path for workspace (default: `~/workspaces/{name}`)
- `--dry-run`: Preview migration without making changes
- `--backup`: Create backup of old configuration files

### Examples

```bash
# Basic migration with default workspace
./provisioning/scripts/migrate-to-target-configs.nu --backup

# Custom workspace name
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "production" \
  --backup

# Custom workspace path
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "staging" \
  --workspace-path "/opt/workspaces/staging" \
  --backup

# Dry run first
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "production" \
  --dry-run
```

## New Workspace Structure

After migration, your workspace will look like:

```
~/workspaces/{name}/
├── config/
│   ├── provisioning.yaml          # Main workspace config
│   ├── providers/
│   │   ├── aws.toml               # AWS provider config
│   │   ├── upcloud.toml           # UpCloud provider config
│   │   └── local.toml             # Local provider config
│   └── platform/
│       ├── orchestrator.toml      # Orchestrator config
│       ├── control-center.toml    # Control center config
│       └── kms.toml               # KMS config
├── infra/
│   └── {infra-name}/              # Infrastructure definitions
├── .cache/                        # Cache directory
└── .runtime/                      # Runtime data
```

User context stored at:
```
~/Library/Application Support/provisioning/
└── ws_{name}.yaml                 # User workspace context
```

## Configuration Schema Validation

### Validate Workspace Config

```bash
# Validate main workspace configuration
provisioning workspace config validate

# Validate specific provider
provisioning provider validate aws

# Validate platform service
provisioning platform validate orchestrator
```

### Manual Validation

```nushell
use provisioning/core/nulib/lib_provisioning/config/schema_validator.nu *

# Validate workspace config
let config = (open ~/workspaces/my-project/config/provisioning.yaml | from yaml)
let result = (validate-workspace-config $config)
print-validation-results $result

# Validate provider config
let aws_config = (open ~/workspaces/my-project/config/providers/aws.toml | from toml)
let result = (validate-provider-config "aws" $aws_config)
print-validation-results $result
```

## Troubleshooting

### Migration Fails

**Problem**: Migration script fails with "workspace path already exists"

**Solution**:
```bash
# Use merge mode
# The script will prompt for confirmation
./provisioning/scripts/migrate-to-target-configs.nu --workspace-name "existing"

# Or choose different workspace name
./provisioning/scripts/migrate-to-target-configs.nu --workspace-name "existing-v2"
```

### Config Not Found

**Problem**: Commands can't find configuration after migration

**Solution**:
```bash
# Check workspace context
provisioning workspace info

# Ensure workspace is active
provisioning workspace activate my-project

# Manually set workspace
export PROVISIONING_WORKSPACE="my-project"
```

### Validation Errors

**Problem**: Configuration validation fails after migration

**Solution**:
```bash
# Check validation output
provisioning workspace config validate

# Review and fix errors in config files
vim ~/workspaces/my-project/config/provisioning.yaml

# Validate again
provisioning workspace config validate
```

### Provider Configuration Issues

**Problem**: Provider authentication fails after migration

**Solution**:
```bash
# Check provider configuration
cat ~/workspaces/my-project/config/providers/aws.toml

# Update credentials
vim ~/workspaces/my-project/config/providers/aws.toml

# Validate provider config
provisioning provider validate aws
```

## Testing Migration

Run the test suite to verify migration:

```bash
# Run configuration validation tests
nu provisioning/tests/config_validation_tests.nu

# Run integration tests
provisioning test --workspace my-project

# Test specific functionality
provisioning --check server list
provisioning --check taskserv list
```

## Rollback Procedure

If migration causes issues, rollback:

```bash
# Restore old configuration
cp -r provisioning/config.backup.YYYYMMDD/* provisioning/config/

# Remove new workspace
rm -rf ~/workspaces/my-project
rm ~/Library/Application\ Support/provisioning/ws_my-project.yaml

# Unset workspace environment variable
unset PROVISIONING_WORKSPACE

# Verify old config works
provisioning env
```

## Migration Checklist

- [ ] Backup current configuration
- [ ] Run migration script in dry-run mode
- [ ] Review dry-run output
- [ ] Execute migration with backup
- [ ] Verify workspace structure created
- [ ] Validate all configurations
- [ ] Test provider authentication
- [ ] Test platform services
- [ ] Run test suite
- [ ] Update documentation/scripts if needed
- [ ] Clean up old configuration files
- [ ] Document any custom changes

## Next Steps

After successful migration:

1. **Review Workspace Configuration**: Customize `provisioning.yaml` for your needs
2. **Configure Providers**: Update provider configs in `config/providers/`
3. **Configure Platform Services**: Update platform configs in `config/platform/`
4. **Test Operations**: Run `--check` mode commands to verify
5. **Update CI/CD**: Update pipelines to use new workspace system
6. **Document Changes**: Update team documentation

## Additional Resources

- [Workspace Configuration Schema](../config/workspace.schema.toml)
- [Provider Configuration Schemas](../extensions/providers/*/config.schema.toml)
- [Platform Configuration Schemas](../platform/*/config.schema.toml)
- [Configuration Validation Guide](./CONFIG_VALIDATION.md)
- [Workspace Management Guide](./WORKSPACE_GUIDE.md)