provisioning/docs/MIGRATION_GUIDE.md

Migration Guide: Target-Based Configuration System

Overview

This guide walks through migrating from the old config.defaults.toml system to the new workspace-based target configuration system.

Migration Path

Old System                          New System
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
config.defaults.toml          →     ~/workspaces/{name}/config/provisioning.yaml
config.user.toml              →     ~/Library/Application Support/provisioning/ws_{name}.yaml
providers/{name}/config       →     ~/workspaces/{name}/config/providers/{name}.toml
                              →     ~/workspaces/{name}/config/platform/{service}.toml

Step-by-Step Migration

1. Pre-Migration Check

# Check current configuration
provisioning env

# Backup current configuration
cp -r provisioning/config provisioning/config.backup.$(date +%Y%m%d)

2. Run Migration Script (Dry Run)

# Preview what will be done
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "my-project" \
  --dry-run

3. Execute Migration

# Run with backup
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "my-project" \
  --backup

# Or specify custom workspace path
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "my-project" \
  --workspace-path "$HOME/my-custom-path" \
  --backup

4. Verify Migration

# Validate workspace configuration
provisioning workspace config validate

# Check workspace status
provisioning workspace info

# List all workspaces
provisioning workspace list

5. Test Configuration

# Test with new configuration
provisioning --check server list

# Test provider configuration
provisioning provider validate aws

# Test platform configuration
provisioning platform orchestrator status

6. Update Environment Variables (if any)

# Old approach (no longer needed)
# export PROVISIONING_CONFIG_PATH="/path/to/config.defaults.toml"

# New approach - workspace is auto-detected from context
# Or set explicitly:
export PROVISIONING_WORKSPACE="my-project"

7. Clean Up Old Configuration

# After verifying everything works
rm provisioning/config/config.defaults.toml
rm provisioning/config/config.user.toml

# Keep backup for reference
# provisioning/config.backup.YYYYMMDD/

Migration Script Options

Required Arguments

• --workspace-name: Name for the new workspace (default: "default")

Optional Arguments

• --workspace-path: Custom path for workspace (default: ~/workspaces/{name})
• --dry-run: Preview migration without making changes
• --backup: Create backup of old configuration files

Examples

# Basic migration with default workspace
./provisioning/scripts/migrate-to-target-configs.nu --backup

# Custom workspace name
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "production" \
  --backup

# Custom workspace path
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "staging" \
  --workspace-path "/opt/workspaces/staging" \
  --backup

# Dry run first
./provisioning/scripts/migrate-to-target-configs.nu \
  --workspace-name "production" \
  --dry-run

New Workspace Structure

After migration, your workspace will look like:

~/workspaces/{name}/
├── config/
│   ├── provisioning.yaml          # Main workspace config
│   ├── providers/
│   │   ├── aws.toml               # AWS provider config
│   │   ├── upcloud.toml           # UpCloud provider config
│   │   └── local.toml             # Local provider config
│   └── platform/
│       ├── orchestrator.toml      # Orchestrator config
│       ├── control-center.toml    # Control center config
│       └── kms.toml               # KMS config
├── infra/
│   └── {infra-name}/              # Infrastructure definitions
├── .cache/                        # Cache directory
└── .runtime/                      # Runtime data

User context stored at:

~/Library/Application Support/provisioning/
└── ws_{name}.yaml                 # User workspace context

Configuration Schema Validation

Validate Workspace Config

# Validate main workspace configuration
provisioning workspace config validate

# Validate specific provider
provisioning provider validate aws

# Validate platform service
provisioning platform validate orchestrator

Manual Validation

use provisioning/core/nulib/lib_provisioning/config/schema_validator.nu *

# Validate workspace config
let config = (open ~/workspaces/my-project/config/provisioning.yaml | from yaml)
let result = (validate-workspace-config $config)
print-validation-results $result

# Validate provider config
let aws_config = (open ~/workspaces/my-project/config/providers/aws.toml | from toml)
let result = (validate-provider-config "aws" $aws_config)
print-validation-results $result

Troubleshooting

Migration Fails

Problem: Migration script fails with "workspace path already exists"

Solution:

# Use merge mode
# The script will prompt for confirmation
./provisioning/scripts/migrate-to-target-configs.nu --workspace-name "existing"

# Or choose different workspace name
./provisioning/scripts/migrate-to-target-configs.nu --workspace-name "existing-v2"

Config Not Found

Problem: Commands can't find configuration after migration

Solution:

# Check workspace context
provisioning workspace info

# Ensure workspace is active
provisioning workspace activate my-project

# Manually set workspace
export PROVISIONING_WORKSPACE="my-project"

Validation Errors

Problem: Configuration validation fails after migration

Solution:

# Check validation output
provisioning workspace config validate

# Review and fix errors in config files
vim ~/workspaces/my-project/config/provisioning.yaml

# Validate again
provisioning workspace config validate

Provider Configuration Issues

Problem: Provider authentication fails after migration

Solution:

# Check provider configuration
cat ~/workspaces/my-project/config/providers/aws.toml

# Update credentials
vim ~/workspaces/my-project/config/providers/aws.toml

# Validate provider config
provisioning provider validate aws

Testing Migration

Run the test suite to verify migration:

# Run configuration validation tests
nu provisioning/tests/config_validation_tests.nu

# Run integration tests
provisioning test --workspace my-project

# Test specific functionality
provisioning --check server list
provisioning --check taskserv list

Rollback Procedure

If migration causes issues, rollback:

# Restore old configuration
cp -r provisioning/config.backup.YYYYMMDD/* provisioning/config/

# Remove new workspace
rm -rf ~/workspaces/my-project
rm ~/Library/Application\ Support/provisioning/ws_my-project.yaml

# Unset workspace environment variable
unset PROVISIONING_WORKSPACE

# Verify old config works
provisioning env

Migration Checklist

• Backup current configuration
• Run migration script in dry-run mode
• Review dry-run output
• Execute migration with backup
• Verify workspace structure created
• Validate all configurations
• Test provider authentication
• Test platform services
• Run test suite
• Update documentation/scripts if needed
• Clean up old configuration files
• Document any custom changes

Next Steps

After successful migration:

1. Review Workspace Configuration: Customize provisioning.yaml for your needs
2. Configure Providers: Update provider configs in config/providers/
3. Configure Platform Services: Update platform configs in config/platform/
4. Test Operations: Run --check mode commands to verify
5. Update CI/CD: Update pipelines to use new workspace system
6. Document Changes: Update team documentation

Additional Resources

• Workspace Configuration Schema
• Provider Configuration Schemas
• Platform Configuration Schemas
• Configuration Validation Guide
• Workspace Management Guide