# Complete Deployment Guide: From Scratch to Production\n\n**Version**: 3.5.0\n**Last Updated**: 2025-10-09\n**Estimated Time**: 30-60 minutes\n**Difficulty**: Beginner to Intermediate\n\n---\n\n## Table of Contents\n\n1. [Prerequisites](#prerequisites)\n2. [Step 1: Install Nushell](#step-1-install-nushell)\n3. [Step 2: Install Nushell Plugins (Recommended)](#step-2-install-nushell-plugins-recommended)\n4. [Step 3: Install Required Tools](#step-3-install-required-tools)\n5. [Step 4: Clone and Setup Project](#step-4-clone-and-setup-project)\n6. [Step 5: Initialize Workspace](#step-5-initialize-workspace)\n7. [Step 6: Configure Environment](#step-6-configure-environment)\n8. [Step 7: Discover and Load Modules](#step-7-discover-and-load-modules)\n9. [Step 8: Validate Configuration](#step-8-validate-configuration)\n10. [Step 9: Deploy Servers](#step-9-deploy-servers)\n11. [Step 10: Install Task Services](#step-10-install-task-services)\n12. [Step 11: Create Clusters](#step-11-create-clusters)\n13. [Step 12: Verify Deployment](#step-12-verify-deployment)\n14. [Step 13: Post-Deployment](#step-13-post-deployment)\n15. [Troubleshooting](#troubleshooting)\n16. [Next Steps](#next-steps)\n\n---\n\n## Prerequisites\n\nBefore starting, ensure you have:\n\n- ✅ **Operating System**: macOS, Linux, or Windows (WSL2 recommended)\n- ✅ **Administrator Access**: Ability to install software and configure system\n- ✅ **Internet Connection**: For downloading dependencies and accessing cloud providers\n- ✅ **Cloud Provider Credentials**: UpCloud, Hetzner, AWS, or local development environment\n- ✅ **Basic Terminal Knowledge**: Comfortable running shell commands\n- ✅ **Text Editor**: vim, nano, Zed, VSCode, or your preferred editor\n\n### Recommended Hardware\n\n- **CPU**: 2+ cores\n- **RAM**: 8 GB minimum, 16 GB recommended\n- **Disk**: 20 GB free space minimum\n\n---\n\n## Step 1: Install Nushell\n\nNushell 0.109.1+ is the primary shell and scripting language for the provisioning platform.\n\n### macOS (via Homebrew)\n\n```\n# Install Nushell\nbrew install nushell\n\n# Verify installation\nnu --version\n# Expected: 0.109.1 or higher\n```\n\n### Linux (via Package Manager)\n\n**Ubuntu/Debian:**\n\n```\n# Add Nushell repository\ncurl -fsSL https://starship.rs/install.sh | bash\n\n# Install Nushell\nsudo apt update\nsudo apt install nushell\n\n# Verify installation\nnu --version\n```\n\n**Fedora:**\n\n```\nsudo dnf install nushell\nnu --version\n```\n\n**Arch Linux:**\n\n```\nsudo pacman -S nushell\nnu --version\n```\n\n### Linux/macOS (via Cargo)\n\n```\n# Install Rust (if not already installed)\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\nsource $HOME/.cargo/env\n\n# Install Nushell\ncargo install nu --locked\n\n# Verify installation\nnu --version\n```\n\n### Windows (via Winget)\n\n```\n# Install Nushell\nwinget install nushell\n\n# Verify installation\nnu --version\n```\n\n### Configure Nushell\n\n```\n# Start Nushell\nnu\n\n# Configure (creates default config if not exists)\nconfig nu\n```\n\n---\n\n## Step 2: Install Nushell Plugins (Recommended)\n\nNative plugins provide **10-50x performance improvement** for authentication, KMS, and orchestrator operations.\n\n### Why Install Plugins\n\n**Performance Gains:**\n\n- 🚀 **KMS operations**: ~5 ms vs ~50 ms (10x faster)\n- 🚀 **Orchestrator queries**: ~1 ms vs ~30 ms (30x faster)\n- 🚀 **Batch encryption**: 100 files in 0.5s vs 5s (10x faster)\n\n**Benefits:**\n\n- ✅ Native Nushell integration (pipelines, data structures)\n- ✅ OS keyring for secure token storage\n- ✅ Offline capability (Age encryption, local orchestrator)\n- ✅ Graceful fallback to HTTP if not installed\n\n### Prerequisites for Building Plugins\n\n```\n# Install Rust toolchain (if not already installed)\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\nsource $HOME/.cargo/env\nrustc --version\n# Expected: rustc 1.75+ or higher\n\n# Linux only: Install development packages\nsudo apt install libssl-dev pkg-config  # Ubuntu/Debian\nsudo dnf install openssl-devel          # Fedora\n\n# Linux only: Install keyring service (required for auth plugin)\nsudo apt install gnome-keyring          # Ubuntu/Debian (GNOME)\nsudo apt install kwalletmanager         # Ubuntu/Debian (KDE)\n```\n\n### Build Plugins\n\n```\n# Navigate to plugins directory\ncd provisioning/core/plugins/nushell-plugins\n\n# Build all three plugins in release mode (optimized)\ncargo build --release --all\n\n# Expected output:\n#    Compiling nu_plugin_auth v0.1.0\n#    Compiling nu_plugin_kms v0.1.0\n#    Compiling nu_plugin_orchestrator v0.1.0\n#     Finished release [optimized] target(s) in 2m 15s\n```\n\n**Build time**: ~2-5 minutes depending on hardware\n\n### Register Plugins with Nushell\n\n```\n# Register all three plugins (full paths recommended)\nplugin add $PWD/target/release/nu_plugin_auth\nplugin add $PWD/target/release/nu_plugin_kms\nplugin add $PWD/target/release/nu_plugin_orchestrator\n\n# Alternative (from plugins directory)\nplugin add target/release/nu_plugin_auth\nplugin add target/release/nu_plugin_kms\nplugin add target/release/nu_plugin_orchestrator\n```\n\n### Verify Plugin Installation\n\n```\n# List registered plugins\nplugin list | where name =~ "auth|kms|orch"\n\n# Expected output:\n# ╭───┬─────────────────────────┬─────────┬───────────────────────────────────╮\n# │ # │          name           │ version │           filename                │\n# ├───┼─────────────────────────┼─────────┼───────────────────────────────────┤\n# │ 0 │ nu_plugin_auth          │ 0.1.0   │ .../nu_plugin_auth                │\n# │ 1 │ nu_plugin_kms           │ 0.1.0   │ .../nu_plugin_kms                 │\n# │ 2 │ nu_plugin_orchestrator  │ 0.1.0   │ .../nu_plugin_orchestrator        │\n# ╰───┴─────────────────────────┴─────────┴───────────────────────────────────╯\n\n# Test each plugin\nauth --help       # Should show auth commands\nkms --help        # Should show kms commands\norch --help       # Should show orch commands\n```\n\n### Configure Plugin Environments\n\n```\n# Add to ~/.config/nushell/env.nu\n$env.CONTROL_CENTER_URL = "http://localhost:3000"\n$env.RUSTYVAULT_ADDR = "http://localhost:8200"\n$env.RUSTYVAULT_TOKEN = "your-vault-token-here"\n$env.ORCHESTRATOR_DATA_DIR = "provisioning/platform/orchestrator/data"\n\n# For Age encryption (local development)\n$env.AGE_IDENTITY = $"($env.HOME)/.age/key.txt"\n$env.AGE_RECIPIENT = "age1xxxxxxxxx"  # Replace with your public key\n```\n\n### Test Plugins (Quick Smoke Test)\n\n```\n# Test KMS plugin (requires backend configured)\nkms status\n# Expected: { backend: "rustyvault", status: "healthy", ... }\n# Or: Error if backend not configured (OK for now)\n\n# Test orchestrator plugin (reads local files)\norch status\n# Expected: { active_tasks: 0, completed_tasks: 0, health: "healthy" }\n# Or: Error if orchestrator not started yet (OK for now)\n\n# Test auth plugin (requires control center)\nauth verify\n# Expected: { active: false }\n# Or: Error if control center not running (OK for now)\n```\n\n**Note**: It's OK if plugins show errors at this stage. We'll configure backends and services later.\n\n### Skip Plugins (Not Recommended)\n\nIf you want to skip plugin installation for now:\n\n- ✅ All features work via HTTP API (slower but functional)\n- ⚠️ You'll miss 10-50x performance improvements\n- ⚠️ No offline capability for KMS/orchestrator\n- ℹ️ You can install plugins later anytime\n\nTo use HTTP fallback:\n\n```\n# System automatically uses HTTP if plugins not available\n# No configuration changes needed\n```\n\n---\n\n## Step 3: Install Required Tools\n\n### Essential Tools\n\n**SOPS (Secrets Management)**\n\n```\n# macOS\nbrew install sops\n\n# Linux\nwget https://github.com/mozilla/sops/releases/download/v3.10.2/sops-v3.10.2.linux.amd64\nsudo mv sops-v3.10.2.linux.amd64 /usr/local/bin/sops\nsudo chmod +x /usr/local/bin/sops\n\n# Verify\nsops --version\n# Expected: 3.10.2 or higher\n```\n\n**Age (Encryption Tool)**\n\n```\n# macOS\nbrew install age\n\n# Linux\nsudo apt install age  # Ubuntu/Debian\nsudo dnf install age  # Fedora\n\n# Or from source\ngo install filippo.io/age/cmd/...@latest\n\n# Verify\nage --version\n# Expected: 1.2.1 or higher\n\n# Generate Age key (for local encryption)\nage-keygen -o ~/.age/key.txt\ncat ~/.age/key.txt\n# Save the public key (age1...) for later\n```\n\n### Optional but Recommended Tools\n\n**K9s (Kubernetes Management)**\n\n```\n# macOS\nbrew install k9s\n\n# Linux\ncurl -sS https://webinstall.dev/k9s | bash\n\n# Verify\nk9s version\n# Expected: 0.50.6 or higher\n```\n\n**glow (Markdown Renderer)**\n\n```\n# macOS\nbrew install glow\n\n# Linux\nsudo apt install glow  # Ubuntu/Debian\nsudo dnf install glow  # Fedora\n\n# Verify\nglow --version\n```\n\n---\n\n## Step 4: Clone and Setup Project\n\n### Clone Repository\n\n```\n# Clone project\ngit clone https://github.com/your-org/project-provisioning.git\ncd project-provisioning\n\n# Or if already cloned, update to latest\ngit pull origin main\n```\n\n### Add CLI to PATH (Optional)\n\n```\n# Add to ~/.bashrc or ~/.zshrc\nexport PATH="$PATH:/Users/Akasha/project-provisioning/provisioning/core/cli"\n\n# Or create symlink\nsudo ln -s /Users/Akasha/project-provisioning/provisioning/core/cli/provisioning /usr/local/bin/provisioning\n\n# Verify\nprovisioning version\n# Expected: 3.5.0\n```\n\n---\n\n## Step 5: Initialize Workspace\n\nA workspace is a self-contained environment for managing infrastructure.\n\n### Create New Workspace\n\n```\n# Initialize new workspace\nprovisioning workspace init --name production\n\n# Or use interactive mode\nprovisioning workspace init\n# Name: production\n# Description: Production infrastructure\n# Provider: upcloud\n```\n\n**What this creates:**\n\nThe new workspace initialization now generates **Nickel configuration files** for type-safe, schema-validated infrastructure definitions:\n\n```\nworkspace/\n├── config/\n│   ├── config.ncl               # Master Nickel configuration (type-safe)\n│   ├── providers/\n│   │   └── upcloud.toml         # Provider-specific settings\n│   ├── platform/                # Platform service configs\n│   └── kms.toml                 # Key management settings\n├── infra/\n│   └── default/\n│       ├── main.ncl             # Infrastructure entry point\n│       └── servers.ncl          # Server definitions\n├── docs/                        # Auto-generated guides\n└── workspace.nu                 # Workspace utility scripts\n```\n\n### Workspace Configuration Format\n\nThe workspace configuration uses **Nickel (type-safe, validated)**. This provides:\n\n- ✅ **Type Safety**: Schema validation catches errors at load time\n- ✅ **Lazy Evaluation**: Only computes what's needed\n- ✅ **Validation**: Record merging, required fields, constraints\n- ✅ **Documentation**: Self-documenting with records\n\n**Example Nickel config** (`config.ncl`):\n\n```\n{\n  workspace = {\n    name = "production",\n    version = "1.0.0",\n    created = "2025-12-03T14:30:00Z",\n  },\n\n  paths = {\n    base = "/opt/workspaces/production",\n    infra = "/opt/workspaces/production/infra",\n    cache = "/opt/workspaces/production/.cache",\n  },\n\n  providers = {\n    active = ["upcloud"],\n    default = "upcloud",\n  },\n}\n```\n\n### Verify Workspace\n\n```\n# Show workspace info\nprovisioning workspace info\n\n# List all workspaces\nprovisioning workspace list\n\n# Show active workspace\nprovisioning workspace active\n# Expected: production\n```\n\n### View and Validate Workspace Configuration\n\nNow you can inspect and validate your Nickel workspace configuration:\n\n```\n# View complete workspace configuration\nprovisioning workspace config show\n\n# Show specific workspace\nprovisioning workspace config show production\n\n# View configuration in different formats\nprovisioning workspace config show --format=json\nprovisioning workspace config show --format=yaml\nprovisioning workspace config show --format=nickel  # Raw Nickel file\n\n# Validate workspace configuration\nprovisioning workspace config validate\n# Output: ✅ Validation complete - all configs are valid\n\n# Show configuration hierarchy (priority order)\nprovisioning workspace config hierarchy\n```\n\n**Configuration Validation**: The Nickel schema automatically validates:\n\n- ✅ Semantic versioning format (for example, "1.0.0")\n- ✅ Required sections present (workspace, paths, provisioning, etc.)\n- ✅ Valid file paths and types\n- ✅ Provider configuration exists for active providers\n- ✅ KMS and SOPS settings properly configured\n\n---\n\n## Step 6: Configure Environment\n\n### Set Provider Credentials\n\n**UpCloud Provider:**\n\n```\n# Create provider config\nvim workspace/config/providers/upcloud.toml\n```\n\n```\n[upcloud]\nusername = "your-upcloud-username"\npassword = "your-upcloud-password"  # Will be encrypted\n\n# Default settings\ndefault_zone = "de-fra1"\ndefault_plan = "2xCPU-4 GB"\n```\n\n**AWS Provider:**\n\n```\n# Create AWS config\nvim workspace/config/providers/aws.toml\n```\n\n```\n[aws]\nregion = "us-east-1"\naccess_key_id = "AKIAXXXXX"\nsecret_access_key = "xxxxx"  # Will be encrypted\n\n# Default settings\ndefault_instance_type = "t3.medium"\ndefault_region = "us-east-1"\n```\n\n### Encrypt Sensitive Data\n\n```\n# Generate Age key if not done already\nage-keygen -o ~/.age/key.txt\n\n# Encrypt provider configs\nkms encrypt (open workspace/config/providers/upcloud.toml) --backend age \\n    | save workspace/config/providers/upcloud.toml.enc\n\n# Or use SOPS\nsops --encrypt --age $(cat ~/.age/key.txt | grep "public key:" | cut -d: -f2) \\n    workspace/config/providers/upcloud.toml > workspace/config/providers/upcloud.toml.enc\n\n# Remove plaintext\nrm workspace/config/providers/upcloud.toml\n```\n\n### Configure Local Overrides\n\n```\n# Edit user-specific settings\nvim workspace/config/local-overrides.toml\n```\n\n```\n[user]\nname = "admin"\nemail = "admin@example.com"\n\n[preferences]\neditor = "vim"\noutput_format = "yaml"\nconfirm_delete = true\nconfirm_deploy = true\n\n[http]\nuse_curl = true  # Use curl instead of ureq\n\n[paths]\nssh_key = "~/.ssh/id_ed25519"\n```\n\n---\n\n## Step 7: Discover and Load Modules\n\n### Discover Available Modules\n\n```\n# Discover task services\nprovisioning module discover taskserv\n# Shows: kubernetes, containerd, etcd, cilium, helm, etc.\n\n# Discover providers\nprovisioning module discover provider\n# Shows: upcloud, aws, local\n\n# Discover clusters\nprovisioning module discover cluster\n# Shows: buildkit, registry, monitoring, etc.\n```\n\n### Load Modules into Workspace\n\n```\n# Load Kubernetes taskserv\nprovisioning module load taskserv production kubernetes\n\n# Load multiple modules\nprovisioning module load taskserv production kubernetes containerd cilium\n\n# Load cluster configuration\nprovisioning module load cluster production buildkit\n\n# Verify loaded modules\nprovisioning module list taskserv production\nprovisioning module list cluster production\n```\n\n---\n\n## Step 8: Validate Configuration\n\nBefore deploying, validate all configuration:\n\n```\n# Validate workspace configuration\nprovisioning workspace validate\n\n# Validate infrastructure configuration\nprovisioning validate config\n\n# Validate specific infrastructure\nprovisioning infra validate --infra production\n\n# Check environment variables\nprovisioning env\n\n# Show all configuration and environment\nprovisioning allenv\n```\n\n**Expected output:**\n\n```\n✓ Configuration valid\n✓ Provider credentials configured\n✓ Workspace initialized\n✓ Modules loaded: 3 taskservs, 1 cluster\n✓ SSH key configured\n✓ Age encryption key available\n```\n\n**Fix any errors** before proceeding to deployment.\n\n---\n\n## Step 9: Deploy Servers\n\n### Preview Server Creation (Dry Run)\n\n```\n# Check what would be created (no actual changes)\nprovisioning server create --infra production --check\n\n# With debug output for details\nprovisioning server create --infra production --check --debug\n```\n\n**Review the output:**\n\n- Server names and configurations\n- Zones and regions\n- CPU, memory, disk specifications\n- Estimated costs\n- Network settings\n\n### Create Servers\n\n```\n# Create servers (with confirmation prompt)\nprovisioning server create --infra production\n\n# Or auto-confirm (skip prompt)\nprovisioning server create --infra production --yes\n\n# Wait for completion\nprovisioning server create --infra production --wait\n```\n\n**Expected output:**\n\n```\nCreating servers for infrastructure: production\n\n  ● Creating server: k8s-master-01 (de-fra1, 4xCPU-8 GB)\n  ● Creating server: k8s-worker-01 (de-fra1, 4xCPU-8 GB)\n  ● Creating server: k8s-worker-02 (de-fra1, 4xCPU-8 GB)\n\n✓ Created 3 servers in 120 seconds\n\nServers:\n  • k8s-master-01: 192.168.1.10 (Running)\n  • k8s-worker-01: 192.168.1.11 (Running)\n  • k8s-worker-02: 192.168.1.12 (Running)\n```\n\n### Verify Server Creation\n\n```\n# List all servers\nprovisioning server list --infra production\n\n# Show detailed server info\nprovisioning server list --infra production --out yaml\n\n# SSH to server (test connectivity)\nprovisioning server ssh k8s-master-01\n# Type 'exit' to return\n```\n\n---\n\n## Step 10: Install Task Services\n\nTask services are infrastructure components like Kubernetes, databases, monitoring, etc.\n\n### Install Kubernetes (Check Mode First)\n\n```\n# Preview Kubernetes installation\nprovisioning taskserv create kubernetes --infra production --check\n\n# Shows:\n# - Dependencies required (containerd, etcd)\n# - Configuration to be applied\n# - Resources needed\n# - Estimated installation time\n```\n\n### Install Kubernetes\n\n```\n# Install Kubernetes (with dependencies)\nprovisioning taskserv create kubernetes --infra production\n\n# Or install dependencies first\nprovisioning taskserv create containerd --infra production\nprovisioning taskserv create etcd --infra production\nprovisioning taskserv create kubernetes --infra production\n\n# Monitor progress\nprovisioning workflow monitor <task_id>\n```\n\n**Expected output:**\n\n```\nInstalling taskserv: kubernetes\n\n  ● Installing containerd on k8s-master-01\n  ● Installing containerd on k8s-worker-01\n  ● Installing containerd on k8s-worker-02\n  ✓ Containerd installed (30s)\n\n  ● Installing etcd on k8s-master-01\n  ✓ etcd installed (20s)\n\n  ● Installing Kubernetes control plane on k8s-master-01\n  ✓ Kubernetes control plane ready (45s)\n\n  ● Joining worker nodes\n  ✓ k8s-worker-01 joined (15s)\n  ✓ k8s-worker-02 joined (15s)\n\n✓ Kubernetes installation complete (125 seconds)\n\nCluster Info:\n  • Version: 1.28.0\n  • Nodes: 3 (1 control-plane, 2 workers)\n  • API Server: https://192.168.1.10:6443\n```\n\n### Install Additional Services\n\n```\n# Install Cilium (CNI)\nprovisioning taskserv create cilium --infra production\n\n# Install Helm\nprovisioning taskserv create helm --infra production\n\n# Verify all taskservs\nprovisioning taskserv list --infra production\n```\n\n---\n\n## Step 11: Create Clusters\n\nClusters are complete application stacks (for example, BuildKit, OCI Registry, Monitoring).\n\n### Create BuildKit Cluster (Check Mode)\n\n```\n# Preview cluster creation\nprovisioning cluster create buildkit --infra production --check\n\n# Shows:\n# - Components to be deployed\n# - Dependencies required\n# - Configuration values\n# - Resource requirements\n```\n\n### Create BuildKit Cluster\n\n```\n# Create BuildKit cluster\nprovisioning cluster create buildkit --infra production\n\n# Monitor deployment\nprovisioning workflow monitor <task_id>\n\n# Or use plugin for faster monitoring\norch tasks --status running\n```\n\n**Expected output:**\n\n```\nCreating cluster: buildkit\n\n  ● Deploying BuildKit daemon\n  ● Deploying BuildKit worker\n  ● Configuring BuildKit cache\n  ● Setting up BuildKit registry integration\n\n✓ BuildKit cluster ready (60 seconds)\n\nCluster Info:\n  • BuildKit version: 0.12.0\n  • Workers: 2\n  • Cache: 50 GB\n  • Registry: registry.production.local\n```\n\n### Verify Cluster\n\n```\n# List all clusters\nprovisioning cluster list --infra production\n\n# Show cluster details\nprovisioning cluster list --infra production --out yaml\n\n# Check cluster health\nkubectl get pods -n buildkit\n```\n\n---\n\n## Step 12: Verify Deployment\n\n### Comprehensive Health Check\n\n```\n# Check orchestrator status\norch status\n# or\nprovisioning orchestrator status\n\n# Check all servers\nprovisioning server list --infra production\n\n# Check all taskservs\nprovisioning taskserv list --infra production\n\n# Check all clusters\nprovisioning cluster list --infra production\n\n# Verify Kubernetes cluster\nkubectl get nodes\nkubectl get pods --all-namespaces\n```\n\n### Run Validation Tests\n\n```\n# Validate infrastructure\nprovisioning infra validate --infra production\n\n# Test connectivity\nprovisioning server ssh k8s-master-01 "kubectl get nodes"\n\n# Test BuildKit\nkubectl exec -it -n buildkit buildkit-0 -- buildctl --version\n```\n\n### Expected Results\n\nAll checks should show:\n\n- ✅ Servers: Running\n- ✅ Taskservs: Installed and healthy\n- ✅ Clusters: Deployed and operational\n- ✅ Kubernetes: 3/3 nodes ready\n- ✅ BuildKit: 2/2 workers ready\n\n---\n\n## Step 13: Post-Deployment\n\n### Configure kubectl Access\n\n```\n# Get kubeconfig from master node\nprovisioning server ssh k8s-master-01 "cat ~/.kube/config" > ~/.kube/config-production\n\n# Set KUBECONFIG\nexport KUBECONFIG=~/.kube/config-production\n\n# Verify access\nkubectl get nodes\nkubectl get pods --all-namespaces\n```\n\n### Set Up Monitoring (Optional)\n\n```\n# Deploy monitoring stack\nprovisioning cluster create monitoring --infra production\n\n# Access Grafana\nkubectl port-forward -n monitoring svc/grafana 3000:80\n# Open: http://localhost:3000\n```\n\n### Configure CI/CD Integration (Optional)\n\n```\n# Generate CI/CD credentials\nprovisioning secrets generate aws --ttl 12h\n\n# Create CI/CD kubeconfig\nkubectl create serviceaccount ci-cd -n default\nkubectl create clusterrolebinding ci-cd --clusterrole=admin --serviceaccount=default:ci-cd\n```\n\n### Backup Configuration\n\n```\n# Backup workspace configuration\ntar -czf workspace-production-backup.tar.gz workspace/\n\n# Encrypt backup\nkms encrypt (open workspace-production-backup.tar.gz | encode base64) --backend age \\n    | save workspace-production-backup.tar.gz.enc\n\n# Store securely (S3, Vault, etc.)\n```\n\n---\n\n## Troubleshooting\n\n### Server Creation Fails\n\n**Problem**: Server creation times out or fails\n\n```\n# Check provider credentials\nprovisioning validate config\n\n# Check provider API status\ncurl -u username:password https://api.upcloud.com/1.3/account\n\n# Try with debug mode\nprovisioning server create --infra production --check --debug\n```\n\n### Taskserv Installation Fails\n\n**Problem**: Kubernetes installation fails\n\n```\n# Check server connectivity\nprovisioning server ssh k8s-master-01\n\n# Check logs\nprovisioning orchestrator logs | grep kubernetes\n\n# Check dependencies\nprovisioning taskserv list --infra production | where status == "failed"\n\n# Retry installation\nprovisioning taskserv delete kubernetes --infra production\nprovisioning taskserv create kubernetes --infra production\n```\n\n### Plugin Commands Don't Work\n\n**Problem**: `auth`, `kms`, or `orch` commands not found\n\n```\n# Check plugin registration\nplugin list | where name =~ "auth|kms|orch"\n\n# Re-register if missing\ncd provisioning/core/plugins/nushell-plugins\nplugin add target/release/nu_plugin_auth\nplugin add target/release/nu_plugin_kms\nplugin add target/release/nu_plugin_orchestrator\n\n# Restart Nushell\nexit\nnu\n```\n\n### KMS Encryption Fails\n\n**Problem**: `kms encrypt` returns error\n\n```\n# Check backend status\nkms status\n\n# Check RustyVault running\ncurl http://localhost:8200/v1/sys/health\n\n# Use Age backend instead (local)\nkms encrypt "data" --backend age --key age1xxxxxxxxx\n\n# Check Age key\ncat ~/.age/key.txt\n```\n\n### Orchestrator Not Running\n\n**Problem**: `orch status` returns error\n\n```\n# Check orchestrator status\nps aux | grep orchestrator\n\n# Start orchestrator\ncd provisioning/platform/orchestrator\n./scripts/start-orchestrator.nu --background\n\n# Check logs\ntail -f provisioning/platform/orchestrator/data/orchestrator.log\n```\n\n### Configuration Validation Errors\n\n**Problem**: `provisioning validate config` shows errors\n\n```\n# Show detailed errors\nprovisioning validate config --debug\n\n# Check configuration files\nprovisioning allenv\n\n# Fix missing settings\nvim workspace/config/local-overrides.toml\n```\n\n---\n\n## Next Steps\n\n### Explore Advanced Features\n\n1. **Multi-Environment Deployment**\n\n   ```bash\n   # Create dev and staging workspaces\n   provisioning workspace create dev\n   provisioning workspace create staging\n   provisioning workspace switch dev\n   ```\n\n1. **Batch Operations**\n\n   ```bash\n   # Deploy to multiple clouds\n   provisioning batch submit workflows/multi-cloud-deploy.ncl\n   ```\n\n2. **Security Features**\n\n   ```bash\n   # Enable MFA\n   auth mfa enroll totp\n\n   # Set up break-glass\n   provisioning break-glass request "Emergency access"\n   ```\n\n3. **Compliance and Audit**\n\n   ```bash\n   # Generate compliance report\n   provisioning compliance report --standard soc2\n   ```\n\n### Learn More\n\n- **Quick Reference**: `provisioning sc` or `docs/guides/quickstart-cheatsheet.md`\n- **Update Guide**: `docs/guides/update-infrastructure.md`\n- **Customize Guide**: `docs/guides/customize-infrastructure.md`\n- **Plugin Guide**: `docs/user/PLUGIN_INTEGRATION_GUIDE.md`\n- **Security System**: `docs/architecture/adr-009-security-system-complete.md`\n\n### Get Help\n\n```\n# Show help for any command\nprovisioning help\nprovisioning help server\nprovisioning help taskserv\n\n# Check version\nprovisioning version\n\n# Start Nushell session with provisioning library\nprovisioning nu\n```\n\n---\n\n## Summary\n\nYou've successfully:\n\n✅ Installed Nushell and essential tools\n✅ Built and registered native plugins (10-50x faster operations)\n✅ Cloned and configured the project\n✅ Initialized a production workspace\n✅ Configured provider credentials\n✅ Deployed servers\n✅ Installed Kubernetes and task services\n✅ Created application clusters\n✅ Verified complete deployment\n\n**Your infrastructure is now ready for production use!**\n\n---\n\n**Estimated Total Time**: 30-60 minutes\n**Next Guide**: [Update Infrastructure](update-infrastructure.md)\n**Questions?**: Open an issue or contact <platform-team@example.com>\n\n**Last Updated**: 2025-10-09\n**Version**: 3.5.0