# Configuration Guide\n\nThis comprehensive guide explains the configuration system of the Infrastructure Automation platform, helping you understand, customize, and manage\nall configuration aspects.\n\n## What You'll Learn\n\n- Understanding the configuration hierarchy and precedence\n- Working with different configuration file types\n- Configuration interpolation and templating\n- Environment-specific configurations\n- User customization and overrides\n- Validation and troubleshooting\n- Advanced configuration patterns\n\n## Configuration Architecture\n\n### Configuration Hierarchy\n\nThe system uses a layered configuration approach with clear precedence rules:\n\n```\nRuntime CLI arguments (highest precedence)\n    ↓ (overrides)\nEnvironment Variables\n    ↓ (overrides)\nInfrastructure Config (./.provisioning.toml)\n    ↓ (overrides)\nProject Config (./provisioning.toml)\n    ↓ (overrides)\nUser Config (~/.config/provisioning/config.toml)\n    ↓ (overrides)\nSystem Defaults (config.defaults.toml) (lowest precedence)\n```\n\n### Configuration File Types\n\n| File Type | Purpose | Location | Format |\n| ----------- | --------- | ---------- | -------- |\n| **System Defaults** | Base system configuration | `config.defaults.toml` | TOML |\n| **User Config** | Personal preferences | `~/.config/provisioning/config.toml` | TOML |\n| **Project Config** | Project-wide settings | `./provisioning.toml` | TOML |\n| **Infrastructure Config** | Infra-specific settings | `./.provisioning.toml` | TOML |\n| **Environment Config** | Environment overrides | `config.{env}.toml` | TOML |\n| **Infrastructure Definitions** | Infrastructure as Code | `main.ncl`, `*.ncl` | Nickel |\n\n## Understanding Configuration Sections\n\n### Core System Configuration\n\n```\n[core]\nversion = "1.0.0"           # System version\nname = "provisioning"       # System identifier\n```\n\n### Path Configuration\n\nThe most critical configuration section that defines where everything is located:\n\n```\n[paths]\n# Base directory - all other paths derive from this\nbase = "/usr/local/provisioning"\n\n# Derived paths (usually don't need to change these)\nkloud = "{{paths.base}}/infra"\nproviders = "{{paths.base}}/providers"\ntaskservs = "{{paths.base}}/taskservs"\nclusters = "{{paths.base}}/cluster"\nresources = "{{paths.base}}/resources"\ntemplates = "{{paths.base}}/templates"\ntools = "{{paths.base}}/tools"\ncore = "{{paths.base}}/core"\n\n[paths.files]\n# Important file locations\nsettings_file = "settings.ncl"\nkeys = "{{paths.base}}/keys.yaml"\nrequirements = "{{paths.base}}/requirements.yaml"\n```\n\n### Debug and Logging\n\n```\n[debug]\nenabled = false             # Enable debug mode\nmetadata = false           # Show internal metadata\ncheck = false              # Default to check mode (dry run)\nremote = false            # Enable remote debugging\nlog_level = "info"        # Logging verbosity\nno_terminal = false       # Disable terminal features\n```\n\n### Output Configuration\n\n```\n[output]\nfile_viewer = "less"       # File viewer command\nformat = "yaml"           # Default output format (json, yaml, toml, text)\n```\n\n### Provider Configuration\n\n```\n[providers]\ndefault = "local"         # Default provider\n\n[providers.aws]\napi_url = ""              # AWS API endpoint (blank = default)\nauth = ""                 # Authentication method\ninterface = "CLI"         # Interface type (CLI or API)\n\n[providers.upcloud]\napi_url = "https://api.upcloud.com/1.3"\nauth = ""\ninterface = "CLI"\n\n[providers.local]\napi_url = ""\nauth = ""\ninterface = "CLI"\n```\n\n### Encryption (SOPS) Configuration\n\n```\n[sops]\nuse_sops = true           # Enable SOPS encryption\nconfig_path = "{{paths.base}}/.sops.yaml"\n\n# Search paths for Age encryption keys\nkey_search_paths = [\n    "{{paths.base}}/keys/age.txt",\n    "~/.config/sops/age/keys.txt"\n]\n```\n\n## Configuration Interpolation\n\nThe system supports powerful interpolation patterns for dynamic configuration values.\n\n### Basic Interpolation Patterns\n\n#### Path Interpolation\n\n```\n# Reference other path values\ntemplates = "{{paths.base}}/my-templates"\ncustom_path = "{{paths.providers}}/custom"\n```\n\n#### Environment Variable Interpolation\n\n```\n# Access environment variables\nuser_home = "{{env.HOME}}"\ncurrent_user = "{{env.USER}}"\ncustom_path = "{{env.CUSTOM_PATH || /default/path}}"  # With fallback\n```\n\n#### Date/Time Interpolation\n\n```\n# Dynamic date/time values\nlog_file = "{{paths.base}}/logs/app-{{now.date}}.log"\nbackup_dir = "{{paths.base}}/backups/{{now.timestamp}}"\n```\n\n#### Git Information Interpolation\n\n```\n# Git repository information\ndeployment_branch = "{{git.branch}}"\nversion_tag = "{{git.tag}}"\ncommit_hash = "{{git.commit}}"\n```\n\n#### Cross-Section References\n\n```\n# Reference values from other sections\ndatabase_host = "{{providers.aws.database_endpoint}}"\napi_key = "{{sops.decrypted_key}}"\n```\n\n### Advanced Interpolation\n\n#### Function Calls\n\n```\n# Built-in functions\nconfig_path = "{{path.join(env.HOME, .config, provisioning)}}"\nsafe_name = "{{str.lower(str.replace(project.name, ' ', '-'))}}"\n```\n\n#### Conditional Expressions\n\n```\n# Conditional logic\ndebug_level = "{{debug.enabled && 'debug' || 'info'}}"\nstorage_path = "{{env.STORAGE_PATH || path.join(paths.base, 'storage')}}"\n```\n\n### Interpolation Examples\n\n```\n[paths]\nbase = "/opt/provisioning"\nworkspace = "{{env.HOME}}/provisioning-workspace"\ncurrent_project = "{{paths.workspace}}/{{env.PROJECT_NAME || 'default'}}"\n\n[deployment]\nenvironment = "{{env.DEPLOY_ENV || 'development'}}"\ntimestamp = "{{now.iso8601}}"\nversion = "{{git.tag || git.commit}}"\n\n[database]\nconnection_string = "postgresql://{{env.DB_USER}}:{{env.DB_PASS}}@{{env.DB_HOST || 'localhost'}}/{{env.DB_NAME}}"\n\n[notifications]\nslack_channel = "#{{env.TEAM_NAME || 'general'}}-notifications"\nemail_subject = "Deployment {{deployment.environment}} - {{deployment.timestamp}}"\n```\n\n## Environment-Specific Configuration\n\n### Environment Detection\n\nThe system automatically detects the environment using:\n\n1. **PROVISIONING_ENV** environment variable\n2. **Git branch patterns** (dev, staging, main/master)\n3. **Directory patterns** (development, staging, production)\n4. **Explicit configuration**\n\n### Environment Configuration Files\n\nCreate environment-specific configurations:\n\n#### Development Environment (`config.dev.toml`)\n\n```\n[core]\nname = "provisioning-dev"\n\n[debug]\nenabled = true\nlog_level = "debug"\nmetadata = true\n\n[providers]\ndefault = "local"\n\n[cache]\nenabled = false  # Disable caching for development\n\n[notifications]\nenabled = false  # No notifications in dev\n```\n\n#### Testing Environment (`config.test.toml`)\n\n```\n[core]\nname = "provisioning-test"\n\n[debug]\nenabled = true\ncheck = true     # Default to check mode in testing\nlog_level = "info"\n\n[providers]\ndefault = "local"\n\n[infrastructure]\nauto_cleanup = true  # Clean up test resources\nresource_prefix = "test-{{git.branch}}-"\n```\n\n#### Production Environment (`config.prod.toml`)\n\n```\n[core]\nname = "provisioning-prod"\n\n[debug]\nenabled = false\nlog_level = "warn"\n\n[providers]\ndefault = "aws"\n\n[security]\nrequire_approval = true\naudit_logging = true\nencrypt_backups = true\n\n[notifications]\nenabled = true\ncritical_only = true\n```\n\n### Environment Switching\n\n```\n# Set environment for session\nexport PROVISIONING_ENV=dev\nprovisioning env\n\n# Use environment for single command\nprovisioning --environment prod server create\n\n# Switch environment permanently\nprovisioning env set prod\n```\n\n## User Configuration Customization\n\n### Creating Your User Configuration\n\n```\n# Initialize user configuration from template\nprovisioning init config\n\n# Or copy and customize\ncp config-examples/config.user.toml ~/.config/provisioning/config.toml\n```\n\n### Common User Customizations\n\n#### Developer Setup\n\n```\n[paths]\nbase = "/Users/alice/dev/provisioning"\n\n[debug]\nenabled = true\nlog_level = "debug"\n\n[providers]\ndefault = "local"\n\n[output]\nformat = "json"\nfile_viewer = "code"\n\n[sops]\nkey_search_paths = [\n    "/Users/alice/.config/sops/age/keys.txt"\n]\n```\n\n#### Operations Engineer Setup\n\n```\n[paths]\nbase = "/opt/provisioning"\n\n[debug]\nenabled = false\nlog_level = "info"\n\n[providers]\ndefault = "aws"\n\n[output]\nformat = "yaml"\n\n[notifications]\nenabled = true\nemail = "ops-team@company.com"\n```\n\n#### Team Lead Setup\n\n```\n[paths]\nbase = "/home/teamlead/provisioning"\n\n[debug]\nenabled = true\nmetadata = true\nlog_level = "info"\n\n[providers]\ndefault = "upcloud"\n\n[security]\nrequire_confirmation = true\naudit_logging = true\n\n[sops]\nkey_search_paths = [\n    "/secure/keys/team-lead.txt",\n    "~/.config/sops/age/keys.txt"\n]\n```\n\n## Project-Specific Configuration\n\n### Project Configuration File (`provisioning.toml`)\n\n```\n[project]\nname = "web-application"\ndescription = "Main web application infrastructure"\nversion = "2.1.0"\nteam = "platform-team"\n\n[paths]\n# Project-specific path overrides\ninfra = "./infrastructure"\ntemplates = "./custom-templates"\n\n[defaults]\n# Project defaults\nprovider = "aws"\nregion = "us-west-2"\nenvironment = "development"\n\n[cost_controls]\nmax_monthly_budget = 5000.00\nalert_threshold = 0.8\n\n[compliance]\nrequired_tags = ["team", "environment", "cost-center"]\nencryption_required = true\nbackup_required = true\n\n[notifications]\nslack_webhook = "https://hooks.slack.com/services/..."\nteam_email = "platform-team@company.com"\n```\n\n### Infrastructure-Specific Configuration (`.provisioning.toml`)\n\n```\n[infrastructure]\nname = "production-web-app"\nenvironment = "production"\nregion = "us-west-2"\n\n[overrides]\n# Infrastructure-specific overrides\ndebug.enabled = false\ndebug.log_level = "error"\ncache.enabled = true\n\n[scaling]\nauto_scaling_enabled = true\nmin_instances = 3\nmax_instances = 20\n\n[security]\nvpc_id = "vpc-12345678"\nsubnet_ids = ["subnet-12345678", "subnet-87654321"]\nsecurity_group_id = "sg-12345678"\n\n[monitoring]\nenabled = true\nretention_days = 90\nalerting_enabled = true\n```\n\n## Configuration Validation\n\n### Built-in Validation\n\n```\n# Validate current configuration\nprovisioning validate config\n\n# Detailed validation with warnings\nprovisioning validate config --detailed\n\n# Strict validation mode\nprovisioning validate config strict\n\n# Validate specific environment\nprovisioning validate config --environment prod\n```\n\n### Custom Validation Rules\n\nCreate custom validation in your configuration:\n\n```\n[validation]\n# Custom validation rules\nrequired_sections = ["paths", "providers", "debug"]\nrequired_env_vars = ["AWS_REGION", "PROJECT_NAME"]\nforbidden_values = ["password123", "admin"]\n\n[validation.paths]\n# Path validation rules\nbase_must_exist = true\nwritable_required = ["paths.base", "paths.cache"]\n\n[validation.security]\n# Security validation\nrequire_encryption = true\nmin_key_length = 32\n```\n\n## Troubleshooting Configuration\n\n### Common Configuration Issues\n\n#### Issue 1: Path Not Found Errors\n\n```\n# Problem: Base path doesn't exist\n# Check current configuration\nprovisioning env | grep paths.base\n\n# Verify path exists\nls -la /path/shown/above\n\n# Fix: Update user config\nnano ~/.config/provisioning/config.toml\n# Set correct paths.base = "/correct/path"\n```\n\n#### Issue 2: Interpolation Failures\n\n```\n# Problem: {{env.VARIABLE}} not resolving\n# Check environment variables\nenv | grep VARIABLE\n\n# Check interpolation\nprovisioning validate interpolation test\n\n# Debug interpolation\nprovisioning --debug validate interpolation validate\n```\n\n#### Issue 3: SOPS Encryption Errors\n\n```\n# Problem: Cannot decrypt SOPS files\n# Check SOPS configuration\nprovisioning sops config\n\n# Verify key files\nls -la ~/.config/sops/age/keys.txt\n\n# Test decryption\nsops -d encrypted-file.ncl\n```\n\n#### Issue 4: Provider Authentication\n\n```\n# Problem: Provider authentication failed\n# Check provider configuration\nprovisioning show providers\n\n# Test provider connection\nprovisioning provider test aws\n\n# Verify credentials\naws configure list  # For AWS\n```\n\n### Configuration Debugging\n\n```\n# Show current configuration hierarchy\nprovisioning config show --hierarchy\n\n# Show configuration sources\nprovisioning config sources\n\n# Show interpolated values\nprovisioning config interpolated\n\n# Debug specific section\nprovisioning config debug paths\nprovisioning config debug providers\n```\n\n### Configuration Reset\n\n```\n# Reset to defaults\nprovisioning config reset\n\n# Reset specific section\nprovisioning config reset providers\n\n# Backup current config before reset\nprovisioning config backup\n```\n\n## Advanced Configuration Patterns\n\n### Dynamic Configuration Loading\n\n```\n[dynamic]\n# Load configuration from external sources\nconfig_urls = [\n    "https://config.company.com/provisioning/base.toml",\n    "file:///etc/provisioning/shared.toml"\n]\n\n# Conditional configuration loading\nload_if_exists = [\n    "./local-overrides.toml",\n    "../shared/team-config.toml"\n]\n```\n\n### Configuration Templating\n\n```\n[templates]\n# Template-based configuration\nbase_template = "aws-web-app"\ntemplate_vars = {\n    region = "us-west-2"\n    instance_type = "t3.medium"\n    team_name = "platform"\n}\n\n# Template inheritance\nextends = ["base-web", "monitoring", "security"]\n```\n\n### Multi-Region Configuration\n\n```\n[regions]\nprimary = "us-west-2"\nsecondary = "us-east-1"\n\n[regions.us-west-2]\nproviders.aws.region = "us-west-2"\navailability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]\n\n[regions.us-east-1]\nproviders.aws.region = "us-east-1"\navailability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]\n```\n\n### Configuration Profiles\n\n```\n[profiles]\nactive = "development"\n\n[profiles.development]\ndebug.enabled = true\nproviders.default = "local"\ncost_controls.enabled = false\n\n[profiles.staging]\ndebug.enabled = true\nproviders.default = "aws"\ncost_controls.max_budget = 1000.00\n\n[profiles.production]\ndebug.enabled = false\nproviders.default = "aws"\nsecurity.strict_mode = true\n```\n\n## Configuration Management Best Practices\n\n### 1. Version Control\n\n```\n# Track configuration changes\ngit add provisioning.toml\ngit commit -m "feat(config): add production settings"\n\n# Use branches for configuration experiments\ngit checkout -b config/new-provider\n```\n\n### 2. Documentation\n\n```\n# Document your configuration choices\n[paths]\n# Using custom base path for team shared installation\nbase = "/opt/team-provisioning"\n\n[debug]\n# Debug enabled for troubleshooting infrastructure issues\nenabled = true\nlog_level = "debug"  # Temporary while debugging network problems\n```\n\n### 3. Validation\n\n```\n# Always validate before committing\nprovisioning validate config\ngit add . && git commit -m "update config"\n```\n\n### 4. Backup\n\n```\n# Regular configuration backups\nprovisioning config export --format yaml > config-backup-$(date +%Y%m%d).yaml\n\n# Automated backup script\necho '0 2 * * * provisioning config export > ~/backups/config-$(date +\%Y\%m\%d).yaml' | crontab -\n```\n\n### 5. Security\n\n- Never commit sensitive values in plain text\n- Use SOPS for encrypting secrets\n- Rotate encryption keys regularly\n- Audit configuration access\n\n```\n# Encrypt sensitive configuration\nsops -e settings.ncl > settings.encrypted.ncl\n\n# Audit configuration changes\ngit log -p -- provisioning.toml\n```\n\n## Configuration Migration\n\n### Migrating from Environment Variables\n\n```\n# Old: Environment variables\nexport PROVISIONING_DEBUG=true\nexport PROVISIONING_PROVIDER=aws\n\n# New: Configuration file\n[debug]\nenabled = true\n\n[providers]\ndefault = "aws"\n```\n\n### Upgrading Configuration Format\n\n```\n# Check for configuration updates needed\nprovisioning config check-version\n\n# Migrate to new format\nprovisioning config migrate --from 1.0 --to 2.0\n\n# Validate migrated configuration\nprovisioning validate config\n```\n\n## Next Steps\n\nNow that you understand the configuration system:\n\n1. **Create your user configuration**: `provisioning init config`\n2. **Set up environment-specific configs** for your workflow\n3. **Learn CLI commands**: [CLI Reference](cli-reference.md)\n4. **Practice with examples**: [Examples and Tutorials](examples/)\n5. **Troubleshoot issues**: [Troubleshooting Guide](troubleshooting-guide.md)\n\nYou now have complete control over how provisioning behaves in your environment!
