2025-12-11 21:50:42 +00:00
|
|
|
|
# Complete Deployment Guide: From Scratch to Production
|
|
|
|
|
|
|
|
|
|
|
|
**Version**: 3.5.0
|
|
|
|
|
|
**Last Updated**: 2025-10-09
|
|
|
|
|
|
**Estimated Time**: 30-60 minutes
|
|
|
|
|
|
**Difficulty**: Beginner to Intermediate
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Table of Contents
|
|
|
|
|
|
|
|
|
|
|
|
1. [Prerequisites](#prerequisites)
|
|
|
|
|
|
2. [Step 1: Install Nushell](#step-1-install-nushell)
|
|
|
|
|
|
3. [Step 2: Install Nushell Plugins (Recommended)](#step-2-install-nushell-plugins-recommended)
|
|
|
|
|
|
4. [Step 3: Install Required Tools](#step-3-install-required-tools)
|
|
|
|
|
|
5. [Step 4: Clone and Setup Project](#step-4-clone-and-setup-project)
|
|
|
|
|
|
6. [Step 5: Initialize Workspace](#step-5-initialize-workspace)
|
|
|
|
|
|
7. [Step 6: Configure Environment](#step-6-configure-environment)
|
|
|
|
|
|
8. [Step 7: Discover and Load Modules](#step-7-discover-and-load-modules)
|
|
|
|
|
|
9. [Step 8: Validate Configuration](#step-8-validate-configuration)
|
|
|
|
|
|
10. [Step 9: Deploy Servers](#step-9-deploy-servers)
|
|
|
|
|
|
11. [Step 10: Install Task Services](#step-10-install-task-services)
|
|
|
|
|
|
12. [Step 11: Create Clusters](#step-11-create-clusters)
|
|
|
|
|
|
13. [Step 12: Verify Deployment](#step-12-verify-deployment)
|
|
|
|
|
|
14. [Step 13: Post-Deployment](#step-13-post-deployment)
|
|
|
|
|
|
15. [Troubleshooting](#troubleshooting)
|
|
|
|
|
|
16. [Next Steps](#next-steps)
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Prerequisites
|
|
|
|
|
|
|
|
|
|
|
|
Before starting, ensure you have:
|
|
|
|
|
|
|
|
|
|
|
|
- ✅ **Operating System**: macOS, Linux, or Windows (WSL2 recommended)
|
|
|
|
|
|
- ✅ **Administrator Access**: Ability to install software and configure system
|
|
|
|
|
|
- ✅ **Internet Connection**: For downloading dependencies and accessing cloud providers
|
|
|
|
|
|
- ✅ **Cloud Provider Credentials**: UpCloud, AWS, or local development environment
|
|
|
|
|
|
- ✅ **Basic Terminal Knowledge**: Comfortable running shell commands
|
|
|
|
|
|
- ✅ **Text Editor**: vim, nano, VSCode, or your preferred editor
|
|
|
|
|
|
|
|
|
|
|
|
### Recommended Hardware
|
|
|
|
|
|
|
|
|
|
|
|
- **CPU**: 2+ cores
|
2026-01-08 09:55:37 +00:00
|
|
|
|
- **RAM**: 8 GB minimum, 16 GB recommended
|
|
|
|
|
|
- **Disk**: 20 GB free space minimum
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 1: Install Nushell
|
|
|
|
|
|
|
|
|
|
|
|
Nushell 0.107.1+ is the primary shell and scripting language for the provisioning platform.
|
|
|
|
|
|
|
|
|
|
|
|
### macOS (via Homebrew)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Install Nushell
|
|
|
|
|
|
brew install nushell
|
|
|
|
|
|
|
|
|
|
|
|
# Verify installation
|
|
|
|
|
|
nu --version
|
|
|
|
|
|
# Expected: 0.107.1 or higher
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Linux (via Package Manager)
|
|
|
|
|
|
|
|
|
|
|
|
**Ubuntu/Debian:**
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```bash
|
|
|
|
|
|
# Add Nushell repository
|
|
|
|
|
|
curl -fsSL https://starship.rs/install.sh | bash
|
|
|
|
|
|
|
|
|
|
|
|
# Install Nushell
|
|
|
|
|
|
sudo apt update
|
|
|
|
|
|
sudo apt install nushell
|
|
|
|
|
|
|
|
|
|
|
|
# Verify installation
|
|
|
|
|
|
nu --version
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Fedora:**
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```bash
|
|
|
|
|
|
sudo dnf install nushell
|
|
|
|
|
|
nu --version
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Arch Linux:**
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```bash
|
|
|
|
|
|
sudo pacman -S nushell
|
|
|
|
|
|
nu --version
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Linux/macOS (via Cargo)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Install Rust (if not already installed)
|
|
|
|
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
|
|
|
|
|
source $HOME/.cargo/env
|
|
|
|
|
|
|
|
|
|
|
|
# Install Nushell
|
|
|
|
|
|
cargo install nu --locked
|
|
|
|
|
|
|
|
|
|
|
|
# Verify installation
|
|
|
|
|
|
nu --version
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Windows (via Winget)
|
|
|
|
|
|
|
|
|
|
|
|
```powershell
|
|
|
|
|
|
# Install Nushell
|
|
|
|
|
|
winget install nushell
|
|
|
|
|
|
|
|
|
|
|
|
# Verify installation
|
|
|
|
|
|
nu --version
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Configure Nushell
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Start Nushell
|
|
|
|
|
|
nu
|
|
|
|
|
|
|
|
|
|
|
|
# Configure (creates default config if not exists)
|
|
|
|
|
|
config nu
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 2: Install Nushell Plugins (Recommended)
|
|
|
|
|
|
|
|
|
|
|
|
Native plugins provide **10-50x performance improvement** for authentication, KMS, and orchestrator operations.
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
### Why Install Plugins
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Performance Gains:**
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
|
|
|
|
|
- 🚀 **KMS operations**: ~5 ms vs ~50 ms (10x faster)
|
|
|
|
|
|
- 🚀 **Orchestrator queries**: ~1 ms vs ~30 ms (30x faster)
|
2025-12-11 21:50:42 +00:00
|
|
|
|
- 🚀 **Batch encryption**: 100 files in 0.5s vs 5s (10x faster)
|
|
|
|
|
|
|
|
|
|
|
|
**Benefits:**
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
- ✅ Native Nushell integration (pipelines, data structures)
|
|
|
|
|
|
- ✅ OS keyring for secure token storage
|
|
|
|
|
|
- ✅ Offline capability (Age encryption, local orchestrator)
|
|
|
|
|
|
- ✅ Graceful fallback to HTTP if not installed
|
|
|
|
|
|
|
|
|
|
|
|
### Prerequisites for Building Plugins
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Install Rust toolchain (if not already installed)
|
|
|
|
|
|
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
|
|
|
|
|
source $HOME/.cargo/env
|
|
|
|
|
|
rustc --version
|
|
|
|
|
|
# Expected: rustc 1.75+ or higher
|
|
|
|
|
|
|
|
|
|
|
|
# Linux only: Install development packages
|
|
|
|
|
|
sudo apt install libssl-dev pkg-config # Ubuntu/Debian
|
|
|
|
|
|
sudo dnf install openssl-devel # Fedora
|
|
|
|
|
|
|
|
|
|
|
|
# Linux only: Install keyring service (required for auth plugin)
|
|
|
|
|
|
sudo apt install gnome-keyring # Ubuntu/Debian (GNOME)
|
|
|
|
|
|
sudo apt install kwalletmanager # Ubuntu/Debian (KDE)
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Build Plugins
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Navigate to plugins directory
|
|
|
|
|
|
cd provisioning/core/plugins/nushell-plugins
|
|
|
|
|
|
|
|
|
|
|
|
# Build all three plugins in release mode (optimized)
|
|
|
|
|
|
cargo build --release --all
|
|
|
|
|
|
|
|
|
|
|
|
# Expected output:
|
|
|
|
|
|
# Compiling nu_plugin_auth v0.1.0
|
|
|
|
|
|
# Compiling nu_plugin_kms v0.1.0
|
|
|
|
|
|
# Compiling nu_plugin_orchestrator v0.1.0
|
|
|
|
|
|
# Finished release [optimized] target(s) in 2m 15s
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Build time**: ~2-5 minutes depending on hardware
|
|
|
|
|
|
|
|
|
|
|
|
### Register Plugins with Nushell
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Register all three plugins (full paths recommended)
|
|
|
|
|
|
plugin add $PWD/target/release/nu_plugin_auth
|
|
|
|
|
|
plugin add $PWD/target/release/nu_plugin_kms
|
|
|
|
|
|
plugin add $PWD/target/release/nu_plugin_orchestrator
|
|
|
|
|
|
|
|
|
|
|
|
# Alternative (from plugins directory)
|
|
|
|
|
|
plugin add target/release/nu_plugin_auth
|
|
|
|
|
|
plugin add target/release/nu_plugin_kms
|
|
|
|
|
|
plugin add target/release/nu_plugin_orchestrator
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Verify Plugin Installation
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# List registered plugins
|
|
|
|
|
|
plugin list | where name =~ "auth|kms|orch"
|
|
|
|
|
|
|
|
|
|
|
|
# Expected output:
|
|
|
|
|
|
# ╭───┬─────────────────────────┬─────────┬───────────────────────────────────╮
|
|
|
|
|
|
# │ # │ name │ version │ filename │
|
|
|
|
|
|
# ├───┼─────────────────────────┼─────────┼───────────────────────────────────┤
|
|
|
|
|
|
# │ 0 │ nu_plugin_auth │ 0.1.0 │ .../nu_plugin_auth │
|
|
|
|
|
|
# │ 1 │ nu_plugin_kms │ 0.1.0 │ .../nu_plugin_kms │
|
|
|
|
|
|
# │ 2 │ nu_plugin_orchestrator │ 0.1.0 │ .../nu_plugin_orchestrator │
|
|
|
|
|
|
# ╰───┴─────────────────────────┴─────────┴───────────────────────────────────╯
|
|
|
|
|
|
|
|
|
|
|
|
# Test each plugin
|
|
|
|
|
|
auth --help # Should show auth commands
|
|
|
|
|
|
kms --help # Should show kms commands
|
|
|
|
|
|
orch --help # Should show orch commands
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Configure Plugin Environments
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Add to ~/.config/nushell/env.nu
|
|
|
|
|
|
$env.CONTROL_CENTER_URL = "http://localhost:3000"
|
|
|
|
|
|
$env.RUSTYVAULT_ADDR = "http://localhost:8200"
|
|
|
|
|
|
$env.RUSTYVAULT_TOKEN = "your-vault-token-here"
|
|
|
|
|
|
$env.ORCHESTRATOR_DATA_DIR = "provisioning/platform/orchestrator/data"
|
|
|
|
|
|
|
|
|
|
|
|
# For Age encryption (local development)
|
|
|
|
|
|
$env.AGE_IDENTITY = $"($env.HOME)/.age/key.txt"
|
|
|
|
|
|
$env.AGE_RECIPIENT = "age1xxxxxxxxx" # Replace with your public key
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Test Plugins (Quick Smoke Test)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Test KMS plugin (requires backend configured)
|
|
|
|
|
|
kms status
|
|
|
|
|
|
# Expected: { backend: "rustyvault", status: "healthy", ... }
|
|
|
|
|
|
# Or: Error if backend not configured (OK for now)
|
|
|
|
|
|
|
|
|
|
|
|
# Test orchestrator plugin (reads local files)
|
|
|
|
|
|
orch status
|
|
|
|
|
|
# Expected: { active_tasks: 0, completed_tasks: 0, health: "healthy" }
|
|
|
|
|
|
# Or: Error if orchestrator not started yet (OK for now)
|
|
|
|
|
|
|
|
|
|
|
|
# Test auth plugin (requires control center)
|
|
|
|
|
|
auth verify
|
|
|
|
|
|
# Expected: { active: false }
|
|
|
|
|
|
# Or: Error if control center not running (OK for now)
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Note**: It's OK if plugins show errors at this stage. We'll configure backends and services later.
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
### Skip Plugins (Not Recommended)
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
If you want to skip plugin installation for now:
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
- ✅ All features work via HTTP API (slower but functional)
|
|
|
|
|
|
- ⚠️ You'll miss 10-50x performance improvements
|
|
|
|
|
|
- ⚠️ No offline capability for KMS/orchestrator
|
|
|
|
|
|
- ℹ️ You can install plugins later anytime
|
|
|
|
|
|
|
|
|
|
|
|
To use HTTP fallback:
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```bash
|
|
|
|
|
|
# System automatically uses HTTP if plugins not available
|
|
|
|
|
|
# No configuration changes needed
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 3: Install Required Tools
|
|
|
|
|
|
|
|
|
|
|
|
### Essential Tools
|
|
|
|
|
|
|
|
|
|
|
|
**SOPS (Secrets Management)**
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# macOS
|
|
|
|
|
|
brew install sops
|
|
|
|
|
|
|
|
|
|
|
|
# Linux
|
|
|
|
|
|
wget https://github.com/mozilla/sops/releases/download/v3.10.2/sops-v3.10.2.linux.amd64
|
|
|
|
|
|
sudo mv sops-v3.10.2.linux.amd64 /usr/local/bin/sops
|
|
|
|
|
|
sudo chmod +x /usr/local/bin/sops
|
|
|
|
|
|
|
|
|
|
|
|
# Verify
|
|
|
|
|
|
sops --version
|
|
|
|
|
|
# Expected: 3.10.2 or higher
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Age (Encryption Tool)**
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# macOS
|
|
|
|
|
|
brew install age
|
|
|
|
|
|
|
|
|
|
|
|
# Linux
|
|
|
|
|
|
sudo apt install age # Ubuntu/Debian
|
|
|
|
|
|
sudo dnf install age # Fedora
|
|
|
|
|
|
|
|
|
|
|
|
# Or from source
|
|
|
|
|
|
go install filippo.io/age/cmd/...@latest
|
|
|
|
|
|
|
|
|
|
|
|
# Verify
|
|
|
|
|
|
age --version
|
|
|
|
|
|
# Expected: 1.2.1 or higher
|
|
|
|
|
|
|
|
|
|
|
|
# Generate Age key (for local encryption)
|
|
|
|
|
|
age-keygen -o ~/.age/key.txt
|
|
|
|
|
|
cat ~/.age/key.txt
|
|
|
|
|
|
# Save the public key (age1...) for later
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Optional but Recommended Tools
|
|
|
|
|
|
|
|
|
|
|
|
**K9s (Kubernetes Management)**
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# macOS
|
|
|
|
|
|
brew install k9s
|
|
|
|
|
|
|
|
|
|
|
|
# Linux
|
|
|
|
|
|
curl -sS https://webinstall.dev/k9s | bash
|
|
|
|
|
|
|
|
|
|
|
|
# Verify
|
|
|
|
|
|
k9s version
|
|
|
|
|
|
# Expected: 0.50.6 or higher
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**glow (Markdown Renderer)**
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# macOS
|
|
|
|
|
|
brew install glow
|
|
|
|
|
|
|
|
|
|
|
|
# Linux
|
|
|
|
|
|
sudo apt install glow # Ubuntu/Debian
|
|
|
|
|
|
sudo dnf install glow # Fedora
|
|
|
|
|
|
|
|
|
|
|
|
# Verify
|
|
|
|
|
|
glow --version
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 4: Clone and Setup Project
|
|
|
|
|
|
|
|
|
|
|
|
### Clone Repository
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Clone project
|
|
|
|
|
|
git clone https://github.com/your-org/project-provisioning.git
|
|
|
|
|
|
cd project-provisioning
|
|
|
|
|
|
|
|
|
|
|
|
# Or if already cloned, update to latest
|
|
|
|
|
|
git pull origin main
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Add CLI to PATH (Optional)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Add to ~/.bashrc or ~/.zshrc
|
|
|
|
|
|
export PATH="$PATH:/Users/Akasha/project-provisioning/provisioning/core/cli"
|
|
|
|
|
|
|
|
|
|
|
|
# Or create symlink
|
|
|
|
|
|
sudo ln -s /Users/Akasha/project-provisioning/provisioning/core/cli/provisioning /usr/local/bin/provisioning
|
|
|
|
|
|
|
|
|
|
|
|
# Verify
|
|
|
|
|
|
provisioning version
|
|
|
|
|
|
# Expected: 3.5.0
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 5: Initialize Workspace
|
|
|
|
|
|
|
|
|
|
|
|
A workspace is a self-contained environment for managing infrastructure.
|
|
|
|
|
|
|
|
|
|
|
|
### Create New Workspace
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Initialize new workspace
|
|
|
|
|
|
provisioning workspace init --name production
|
|
|
|
|
|
|
|
|
|
|
|
# Or use interactive mode
|
|
|
|
|
|
provisioning workspace init
|
|
|
|
|
|
# Name: production
|
|
|
|
|
|
# Description: Production infrastructure
|
|
|
|
|
|
# Provider: upcloud
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**What this creates:**
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
The new workspace initialization now generates **Nickel configuration files** for type-safe, schema-validated infrastructure definitions:
|
|
|
|
|
|
|
|
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
workspace/
|
|
|
|
|
|
├── config/
|
2026-01-08 09:55:37 +00:00
|
|
|
|
│ ├── config.ncl # Master Nickel configuration (type-safe)
|
|
|
|
|
|
│ ├── providers/
|
|
|
|
|
|
│ │ └── upcloud.toml # Provider-specific settings
|
|
|
|
|
|
│ ├── platform/ # Platform service configs
|
|
|
|
|
|
│ └── kms.toml # Key management settings
|
|
|
|
|
|
├── infra/
|
|
|
|
|
|
│ └── default/
|
|
|
|
|
|
│ ├── main.ncl # Infrastructure entry point
|
|
|
|
|
|
│ └── servers.ncl # Server definitions
|
|
|
|
|
|
├── docs/ # Auto-generated guides
|
|
|
|
|
|
└── workspace.nu # Workspace utility scripts
|
|
|
|
|
|
```plaintext
|
|
|
|
|
|
|
|
|
|
|
|
### Workspace Configuration Format
|
|
|
|
|
|
|
|
|
|
|
|
The workspace configuration uses **Nickel (type-safe, validated)**. This provides:
|
|
|
|
|
|
|
|
|
|
|
|
- ✅ **Type Safety**: Schema validation catches errors at load time
|
|
|
|
|
|
- ✅ **Lazy Evaluation**: Only computes what's needed
|
|
|
|
|
|
- ✅ **Validation**: Record merging, required fields, constraints
|
|
|
|
|
|
- ✅ **Documentation**: Self-documenting with records
|
|
|
|
|
|
|
|
|
|
|
|
**Example Nickel config** (`config.ncl`):
|
|
|
|
|
|
|
|
|
|
|
|
```nickel
|
|
|
|
|
|
{
|
|
|
|
|
|
workspace = {
|
|
|
|
|
|
name = "production",
|
|
|
|
|
|
version = "1.0.0",
|
|
|
|
|
|
created = "2025-12-03T14:30:00Z",
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
|
|
paths = {
|
|
|
|
|
|
base = "/opt/workspaces/production",
|
|
|
|
|
|
infra = "/opt/workspaces/production/infra",
|
|
|
|
|
|
cache = "/opt/workspaces/production/.cache",
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
|
|
providers = {
|
|
|
|
|
|
active = ["upcloud"],
|
|
|
|
|
|
default = "upcloud",
|
|
|
|
|
|
},
|
|
|
|
|
|
}
|
|
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Verify Workspace
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Show workspace info
|
|
|
|
|
|
provisioning workspace info
|
|
|
|
|
|
|
|
|
|
|
|
# List all workspaces
|
|
|
|
|
|
provisioning workspace list
|
|
|
|
|
|
|
|
|
|
|
|
# Show active workspace
|
|
|
|
|
|
provisioning workspace active
|
|
|
|
|
|
# Expected: production
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
|
|
|
|
|
|
|
|
|
|
|
### View and Validate Workspace Configuration
|
|
|
|
|
|
|
|
|
|
|
|
Now you can inspect and validate your Nickel workspace configuration:
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# View complete workspace configuration
|
|
|
|
|
|
provisioning workspace config show
|
|
|
|
|
|
|
|
|
|
|
|
# Show specific workspace
|
|
|
|
|
|
provisioning workspace config show production
|
|
|
|
|
|
|
|
|
|
|
|
# View configuration in different formats
|
|
|
|
|
|
provisioning workspace config show --format=json
|
|
|
|
|
|
provisioning workspace config show --format=yaml
|
|
|
|
|
|
provisioning workspace config show --format=nickel # Raw Nickel file
|
|
|
|
|
|
|
|
|
|
|
|
# Validate workspace configuration
|
|
|
|
|
|
provisioning workspace config validate
|
|
|
|
|
|
# Output: ✅ Validation complete - all configs are valid
|
|
|
|
|
|
|
|
|
|
|
|
# Show configuration hierarchy (priority order)
|
|
|
|
|
|
provisioning workspace config hierarchy
|
|
|
|
|
|
```plaintext
|
|
|
|
|
|
|
|
|
|
|
|
**Configuration Validation**: The Nickel schema automatically validates:
|
|
|
|
|
|
|
|
|
|
|
|
- ✅ Semantic versioning format (for example, "1.0.0")
|
|
|
|
|
|
- ✅ Required sections present (workspace, paths, provisioning, etc.)
|
|
|
|
|
|
- ✅ Valid file paths and types
|
|
|
|
|
|
- ✅ Provider configuration exists for active providers
|
|
|
|
|
|
- ✅ KMS and SOPS settings properly configured
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 6: Configure Environment
|
|
|
|
|
|
|
|
|
|
|
|
### Set Provider Credentials
|
|
|
|
|
|
|
|
|
|
|
|
**UpCloud Provider:**
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Create provider config
|
|
|
|
|
|
vim workspace/config/providers/upcloud.toml
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
```toml
|
|
|
|
|
|
[upcloud]
|
|
|
|
|
|
username = "your-upcloud-username"
|
|
|
|
|
|
password = "your-upcloud-password" # Will be encrypted
|
|
|
|
|
|
|
|
|
|
|
|
# Default settings
|
|
|
|
|
|
default_zone = "de-fra1"
|
2026-01-08 09:55:37 +00:00
|
|
|
|
default_plan = "2xCPU-4 GB"
|
|
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**AWS Provider:**
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Create AWS config
|
|
|
|
|
|
vim workspace/config/providers/aws.toml
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
```toml
|
|
|
|
|
|
[aws]
|
|
|
|
|
|
region = "us-east-1"
|
|
|
|
|
|
access_key_id = "AKIAXXXXX"
|
|
|
|
|
|
secret_access_key = "xxxxx" # Will be encrypted
|
|
|
|
|
|
|
|
|
|
|
|
# Default settings
|
|
|
|
|
|
default_instance_type = "t3.medium"
|
|
|
|
|
|
default_region = "us-east-1"
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Encrypt Sensitive Data
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Generate Age key if not done already
|
|
|
|
|
|
age-keygen -o ~/.age/key.txt
|
|
|
|
|
|
|
|
|
|
|
|
# Encrypt provider configs
|
|
|
|
|
|
kms encrypt (open workspace/config/providers/upcloud.toml) --backend age \
|
|
|
|
|
|
| save workspace/config/providers/upcloud.toml.enc
|
|
|
|
|
|
|
|
|
|
|
|
# Or use SOPS
|
|
|
|
|
|
sops --encrypt --age $(cat ~/.age/key.txt | grep "public key:" | cut -d: -f2) \
|
|
|
|
|
|
workspace/config/providers/upcloud.toml > workspace/config/providers/upcloud.toml.enc
|
|
|
|
|
|
|
|
|
|
|
|
# Remove plaintext
|
|
|
|
|
|
rm workspace/config/providers/upcloud.toml
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Configure Local Overrides
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Edit user-specific settings
|
|
|
|
|
|
vim workspace/config/local-overrides.toml
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
```toml
|
|
|
|
|
|
[user]
|
|
|
|
|
|
name = "admin"
|
|
|
|
|
|
email = "admin@example.com"
|
|
|
|
|
|
|
|
|
|
|
|
[preferences]
|
|
|
|
|
|
editor = "vim"
|
|
|
|
|
|
output_format = "yaml"
|
|
|
|
|
|
confirm_delete = true
|
|
|
|
|
|
confirm_deploy = true
|
|
|
|
|
|
|
|
|
|
|
|
[http]
|
|
|
|
|
|
use_curl = true # Use curl instead of ureq
|
|
|
|
|
|
|
|
|
|
|
|
[paths]
|
|
|
|
|
|
ssh_key = "~/.ssh/id_ed25519"
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 7: Discover and Load Modules
|
|
|
|
|
|
|
|
|
|
|
|
### Discover Available Modules
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Discover task services
|
|
|
|
|
|
provisioning module discover taskserv
|
|
|
|
|
|
# Shows: kubernetes, containerd, etcd, cilium, helm, etc.
|
|
|
|
|
|
|
|
|
|
|
|
# Discover providers
|
|
|
|
|
|
provisioning module discover provider
|
|
|
|
|
|
# Shows: upcloud, aws, local
|
|
|
|
|
|
|
|
|
|
|
|
# Discover clusters
|
|
|
|
|
|
provisioning module discover cluster
|
|
|
|
|
|
# Shows: buildkit, registry, monitoring, etc.
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Load Modules into Workspace
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Load Kubernetes taskserv
|
|
|
|
|
|
provisioning module load taskserv production kubernetes
|
|
|
|
|
|
|
|
|
|
|
|
# Load multiple modules
|
|
|
|
|
|
provisioning module load taskserv production kubernetes containerd cilium
|
|
|
|
|
|
|
|
|
|
|
|
# Load cluster configuration
|
|
|
|
|
|
provisioning module load cluster production buildkit
|
|
|
|
|
|
|
|
|
|
|
|
# Verify loaded modules
|
|
|
|
|
|
provisioning module list taskserv production
|
|
|
|
|
|
provisioning module list cluster production
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 8: Validate Configuration
|
|
|
|
|
|
|
|
|
|
|
|
Before deploying, validate all configuration:
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Validate workspace configuration
|
|
|
|
|
|
provisioning workspace validate
|
|
|
|
|
|
|
|
|
|
|
|
# Validate infrastructure configuration
|
|
|
|
|
|
provisioning validate config
|
|
|
|
|
|
|
|
|
|
|
|
# Validate specific infrastructure
|
|
|
|
|
|
provisioning infra validate --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Check environment variables
|
|
|
|
|
|
provisioning env
|
|
|
|
|
|
|
|
|
|
|
|
# Show all configuration and environment
|
|
|
|
|
|
provisioning allenv
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Expected output:**
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
✓ Configuration valid
|
|
|
|
|
|
✓ Provider credentials configured
|
|
|
|
|
|
✓ Workspace initialized
|
|
|
|
|
|
✓ Modules loaded: 3 taskservs, 1 cluster
|
|
|
|
|
|
✓ SSH key configured
|
|
|
|
|
|
✓ Age encryption key available
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Fix any errors** before proceeding to deployment.
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 9: Deploy Servers
|
|
|
|
|
|
|
|
|
|
|
|
### Preview Server Creation (Dry Run)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Check what would be created (no actual changes)
|
|
|
|
|
|
provisioning server create --infra production --check
|
|
|
|
|
|
|
|
|
|
|
|
# With debug output for details
|
|
|
|
|
|
provisioning server create --infra production --check --debug
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Review the output:**
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
- Server names and configurations
|
|
|
|
|
|
- Zones and regions
|
|
|
|
|
|
- CPU, memory, disk specifications
|
|
|
|
|
|
- Estimated costs
|
|
|
|
|
|
- Network settings
|
|
|
|
|
|
|
|
|
|
|
|
### Create Servers
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Create servers (with confirmation prompt)
|
|
|
|
|
|
provisioning server create --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Or auto-confirm (skip prompt)
|
|
|
|
|
|
provisioning server create --infra production --yes
|
|
|
|
|
|
|
|
|
|
|
|
# Wait for completion
|
|
|
|
|
|
provisioning server create --infra production --wait
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Expected output:**
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
Creating servers for infrastructure: production
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
● Creating server: k8s-master-01 (de-fra1, 4xCPU-8 GB)
|
|
|
|
|
|
● Creating server: k8s-worker-01 (de-fra1, 4xCPU-8 GB)
|
|
|
|
|
|
● Creating server: k8s-worker-02 (de-fra1, 4xCPU-8 GB)
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
✓ Created 3 servers in 120 seconds
|
|
|
|
|
|
|
|
|
|
|
|
Servers:
|
|
|
|
|
|
• k8s-master-01: 192.168.1.10 (Running)
|
|
|
|
|
|
• k8s-worker-01: 192.168.1.11 (Running)
|
|
|
|
|
|
• k8s-worker-02: 192.168.1.12 (Running)
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Verify Server Creation
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# List all servers
|
|
|
|
|
|
provisioning server list --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Show detailed server info
|
|
|
|
|
|
provisioning server list --infra production --out yaml
|
|
|
|
|
|
|
|
|
|
|
|
# SSH to server (test connectivity)
|
|
|
|
|
|
provisioning server ssh k8s-master-01
|
|
|
|
|
|
# Type 'exit' to return
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 10: Install Task Services
|
|
|
|
|
|
|
|
|
|
|
|
Task services are infrastructure components like Kubernetes, databases, monitoring, etc.
|
|
|
|
|
|
|
|
|
|
|
|
### Install Kubernetes (Check Mode First)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Preview Kubernetes installation
|
|
|
|
|
|
provisioning taskserv create kubernetes --infra production --check
|
|
|
|
|
|
|
|
|
|
|
|
# Shows:
|
|
|
|
|
|
# - Dependencies required (containerd, etcd)
|
|
|
|
|
|
# - Configuration to be applied
|
|
|
|
|
|
# - Resources needed
|
|
|
|
|
|
# - Estimated installation time
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Install Kubernetes
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Install Kubernetes (with dependencies)
|
|
|
|
|
|
provisioning taskserv create kubernetes --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Or install dependencies first
|
|
|
|
|
|
provisioning taskserv create containerd --infra production
|
|
|
|
|
|
provisioning taskserv create etcd --infra production
|
|
|
|
|
|
provisioning taskserv create kubernetes --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Monitor progress
|
|
|
|
|
|
provisioning workflow monitor <task_id>
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Expected output:**
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
Installing taskserv: kubernetes
|
|
|
|
|
|
|
|
|
|
|
|
● Installing containerd on k8s-master-01
|
|
|
|
|
|
● Installing containerd on k8s-worker-01
|
|
|
|
|
|
● Installing containerd on k8s-worker-02
|
|
|
|
|
|
✓ Containerd installed (30s)
|
|
|
|
|
|
|
|
|
|
|
|
● Installing etcd on k8s-master-01
|
|
|
|
|
|
✓ etcd installed (20s)
|
|
|
|
|
|
|
|
|
|
|
|
● Installing Kubernetes control plane on k8s-master-01
|
|
|
|
|
|
✓ Kubernetes control plane ready (45s)
|
|
|
|
|
|
|
|
|
|
|
|
● Joining worker nodes
|
|
|
|
|
|
✓ k8s-worker-01 joined (15s)
|
|
|
|
|
|
✓ k8s-worker-02 joined (15s)
|
|
|
|
|
|
|
|
|
|
|
|
✓ Kubernetes installation complete (125 seconds)
|
|
|
|
|
|
|
|
|
|
|
|
Cluster Info:
|
|
|
|
|
|
• Version: 1.28.0
|
|
|
|
|
|
• Nodes: 3 (1 control-plane, 2 workers)
|
|
|
|
|
|
• API Server: https://192.168.1.10:6443
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Install Additional Services
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Install Cilium (CNI)
|
|
|
|
|
|
provisioning taskserv create cilium --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Install Helm
|
|
|
|
|
|
provisioning taskserv create helm --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Verify all taskservs
|
|
|
|
|
|
provisioning taskserv list --infra production
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 11: Create Clusters
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
Clusters are complete application stacks (for example, BuildKit, OCI Registry, Monitoring).
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Create BuildKit Cluster (Check Mode)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Preview cluster creation
|
|
|
|
|
|
provisioning cluster create buildkit --infra production --check
|
|
|
|
|
|
|
|
|
|
|
|
# Shows:
|
|
|
|
|
|
# - Components to be deployed
|
|
|
|
|
|
# - Dependencies required
|
|
|
|
|
|
# - Configuration values
|
|
|
|
|
|
# - Resource requirements
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Create BuildKit Cluster
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Create BuildKit cluster
|
|
|
|
|
|
provisioning cluster create buildkit --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Monitor deployment
|
|
|
|
|
|
provisioning workflow monitor <task_id>
|
|
|
|
|
|
|
|
|
|
|
|
# Or use plugin for faster monitoring
|
|
|
|
|
|
orch tasks --status running
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Expected output:**
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
Creating cluster: buildkit
|
|
|
|
|
|
|
|
|
|
|
|
● Deploying BuildKit daemon
|
|
|
|
|
|
● Deploying BuildKit worker
|
|
|
|
|
|
● Configuring BuildKit cache
|
|
|
|
|
|
● Setting up BuildKit registry integration
|
|
|
|
|
|
|
|
|
|
|
|
✓ BuildKit cluster ready (60 seconds)
|
|
|
|
|
|
|
|
|
|
|
|
Cluster Info:
|
|
|
|
|
|
• BuildKit version: 0.12.0
|
|
|
|
|
|
• Workers: 2
|
2026-01-08 09:55:37 +00:00
|
|
|
|
• Cache: 50 GB
|
2025-12-11 21:50:42 +00:00
|
|
|
|
• Registry: registry.production.local
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Verify Cluster
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# List all clusters
|
|
|
|
|
|
provisioning cluster list --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Show cluster details
|
|
|
|
|
|
provisioning cluster list --infra production --out yaml
|
|
|
|
|
|
|
|
|
|
|
|
# Check cluster health
|
|
|
|
|
|
kubectl get pods -n buildkit
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 12: Verify Deployment
|
|
|
|
|
|
|
|
|
|
|
|
### Comprehensive Health Check
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Check orchestrator status
|
|
|
|
|
|
orch status
|
|
|
|
|
|
# or
|
|
|
|
|
|
provisioning orchestrator status
|
|
|
|
|
|
|
|
|
|
|
|
# Check all servers
|
|
|
|
|
|
provisioning server list --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Check all taskservs
|
|
|
|
|
|
provisioning taskserv list --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Check all clusters
|
|
|
|
|
|
provisioning cluster list --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Verify Kubernetes cluster
|
|
|
|
|
|
kubectl get nodes
|
|
|
|
|
|
kubectl get pods --all-namespaces
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Run Validation Tests
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Validate infrastructure
|
|
|
|
|
|
provisioning infra validate --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Test connectivity
|
|
|
|
|
|
provisioning server ssh k8s-master-01 "kubectl get nodes"
|
|
|
|
|
|
|
|
|
|
|
|
# Test BuildKit
|
|
|
|
|
|
kubectl exec -it -n buildkit buildkit-0 -- buildctl --version
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Expected Results
|
|
|
|
|
|
|
|
|
|
|
|
All checks should show:
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
- ✅ Servers: Running
|
|
|
|
|
|
- ✅ Taskservs: Installed and healthy
|
|
|
|
|
|
- ✅ Clusters: Deployed and operational
|
|
|
|
|
|
- ✅ Kubernetes: 3/3 nodes ready
|
|
|
|
|
|
- ✅ BuildKit: 2/2 workers ready
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Step 13: Post-Deployment
|
|
|
|
|
|
|
|
|
|
|
|
### Configure kubectl Access
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Get kubeconfig from master node
|
|
|
|
|
|
provisioning server ssh k8s-master-01 "cat ~/.kube/config" > ~/.kube/config-production
|
|
|
|
|
|
|
|
|
|
|
|
# Set KUBECONFIG
|
|
|
|
|
|
export KUBECONFIG=~/.kube/config-production
|
|
|
|
|
|
|
|
|
|
|
|
# Verify access
|
|
|
|
|
|
kubectl get nodes
|
|
|
|
|
|
kubectl get pods --all-namespaces
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Set Up Monitoring (Optional)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Deploy monitoring stack
|
|
|
|
|
|
provisioning cluster create monitoring --infra production
|
|
|
|
|
|
|
|
|
|
|
|
# Access Grafana
|
|
|
|
|
|
kubectl port-forward -n monitoring svc/grafana 3000:80
|
|
|
|
|
|
# Open: http://localhost:3000
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Configure CI/CD Integration (Optional)
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Generate CI/CD credentials
|
|
|
|
|
|
provisioning secrets generate aws --ttl 12h
|
|
|
|
|
|
|
|
|
|
|
|
# Create CI/CD kubeconfig
|
|
|
|
|
|
kubectl create serviceaccount ci-cd -n default
|
|
|
|
|
|
kubectl create clusterrolebinding ci-cd --clusterrole=admin --serviceaccount=default:ci-cd
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Backup Configuration
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Backup workspace configuration
|
|
|
|
|
|
tar -czf workspace-production-backup.tar.gz workspace/
|
|
|
|
|
|
|
|
|
|
|
|
# Encrypt backup
|
|
|
|
|
|
kms encrypt (open workspace-production-backup.tar.gz | encode base64) --backend age \
|
|
|
|
|
|
| save workspace-production-backup.tar.gz.enc
|
|
|
|
|
|
|
|
|
|
|
|
# Store securely (S3, Vault, etc.)
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Troubleshooting
|
|
|
|
|
|
|
|
|
|
|
|
### Server Creation Fails
|
|
|
|
|
|
|
|
|
|
|
|
**Problem**: Server creation times out or fails
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Check provider credentials
|
|
|
|
|
|
provisioning validate config
|
|
|
|
|
|
|
|
|
|
|
|
# Check provider API status
|
|
|
|
|
|
curl -u username:password https://api.upcloud.com/1.3/account
|
|
|
|
|
|
|
|
|
|
|
|
# Try with debug mode
|
|
|
|
|
|
provisioning server create --infra production --check --debug
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Taskserv Installation Fails
|
|
|
|
|
|
|
|
|
|
|
|
**Problem**: Kubernetes installation fails
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Check server connectivity
|
|
|
|
|
|
provisioning server ssh k8s-master-01
|
|
|
|
|
|
|
|
|
|
|
|
# Check logs
|
|
|
|
|
|
provisioning orchestrator logs | grep kubernetes
|
|
|
|
|
|
|
|
|
|
|
|
# Check dependencies
|
|
|
|
|
|
provisioning taskserv list --infra production | where status == "failed"
|
|
|
|
|
|
|
|
|
|
|
|
# Retry installation
|
|
|
|
|
|
provisioning taskserv delete kubernetes --infra production
|
|
|
|
|
|
provisioning taskserv create kubernetes --infra production
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Plugin Commands Don't Work
|
|
|
|
|
|
|
|
|
|
|
|
**Problem**: `auth`, `kms`, or `orch` commands not found
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Check plugin registration
|
|
|
|
|
|
plugin list | where name =~ "auth|kms|orch"
|
|
|
|
|
|
|
|
|
|
|
|
# Re-register if missing
|
|
|
|
|
|
cd provisioning/core/plugins/nushell-plugins
|
|
|
|
|
|
plugin add target/release/nu_plugin_auth
|
|
|
|
|
|
plugin add target/release/nu_plugin_kms
|
|
|
|
|
|
plugin add target/release/nu_plugin_orchestrator
|
|
|
|
|
|
|
|
|
|
|
|
# Restart Nushell
|
|
|
|
|
|
exit
|
|
|
|
|
|
nu
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### KMS Encryption Fails
|
|
|
|
|
|
|
|
|
|
|
|
**Problem**: `kms encrypt` returns error
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Check backend status
|
|
|
|
|
|
kms status
|
|
|
|
|
|
|
|
|
|
|
|
# Check RustyVault running
|
|
|
|
|
|
curl http://localhost:8200/v1/sys/health
|
|
|
|
|
|
|
|
|
|
|
|
# Use Age backend instead (local)
|
|
|
|
|
|
kms encrypt "data" --backend age --key age1xxxxxxxxx
|
|
|
|
|
|
|
|
|
|
|
|
# Check Age key
|
|
|
|
|
|
cat ~/.age/key.txt
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Orchestrator Not Running
|
|
|
|
|
|
|
|
|
|
|
|
**Problem**: `orch status` returns error
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Check orchestrator status
|
|
|
|
|
|
ps aux | grep orchestrator
|
|
|
|
|
|
|
|
|
|
|
|
# Start orchestrator
|
|
|
|
|
|
cd provisioning/platform/orchestrator
|
|
|
|
|
|
./scripts/start-orchestrator.nu --background
|
|
|
|
|
|
|
|
|
|
|
|
# Check logs
|
|
|
|
|
|
tail -f provisioning/platform/orchestrator/data/orchestrator.log
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
### Configuration Validation Errors
|
|
|
|
|
|
|
|
|
|
|
|
**Problem**: `provisioning validate config` shows errors
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Show detailed errors
|
|
|
|
|
|
provisioning validate config --debug
|
|
|
|
|
|
|
|
|
|
|
|
# Check configuration files
|
|
|
|
|
|
provisioning allenv
|
|
|
|
|
|
|
|
|
|
|
|
# Fix missing settings
|
|
|
|
|
|
vim workspace/config/local-overrides.toml
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Next Steps
|
|
|
|
|
|
|
|
|
|
|
|
### Explore Advanced Features
|
|
|
|
|
|
|
|
|
|
|
|
1. **Multi-Environment Deployment**
|
2026-01-08 09:55:37 +00:00
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```bash
|
|
|
|
|
|
# Create dev and staging workspaces
|
|
|
|
|
|
provisioning workspace create dev
|
|
|
|
|
|
provisioning workspace create staging
|
|
|
|
|
|
provisioning workspace switch dev
|
|
|
|
|
|
```
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
1. **Batch Operations**
|
|
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```bash
|
|
|
|
|
|
# Deploy to multiple clouds
|
2026-01-08 09:55:37 +00:00
|
|
|
|
provisioning batch submit workflows/multi-cloud-deploy.ncl
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
2. **Security Features**
|
|
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```bash
|
|
|
|
|
|
# Enable MFA
|
|
|
|
|
|
auth mfa enroll totp
|
|
|
|
|
|
|
|
|
|
|
|
# Set up break-glass
|
|
|
|
|
|
provisioning break-glass request "Emergency access"
|
|
|
|
|
|
```
|
|
|
|
|
|
|
2026-01-08 09:55:37 +00:00
|
|
|
|
3. **Compliance and Audit**
|
|
|
|
|
|
|
2025-12-11 21:50:42 +00:00
|
|
|
|
```bash
|
|
|
|
|
|
# Generate compliance report
|
|
|
|
|
|
provisioning compliance report --standard soc2
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### Learn More
|
|
|
|
|
|
|
|
|
|
|
|
- **Quick Reference**: `provisioning sc` or `docs/guides/quickstart-cheatsheet.md`
|
|
|
|
|
|
- **Update Guide**: `docs/guides/update-infrastructure.md`
|
|
|
|
|
|
- **Customize Guide**: `docs/guides/customize-infrastructure.md`
|
|
|
|
|
|
- **Plugin Guide**: `docs/user/PLUGIN_INTEGRATION_GUIDE.md`
|
|
|
|
|
|
- **Security System**: `docs/architecture/ADR-009-security-system-complete.md`
|
|
|
|
|
|
|
|
|
|
|
|
### Get Help
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
# Show help for any command
|
|
|
|
|
|
provisioning help
|
|
|
|
|
|
provisioning help server
|
|
|
|
|
|
provisioning help taskserv
|
|
|
|
|
|
|
|
|
|
|
|
# Check version
|
|
|
|
|
|
provisioning version
|
|
|
|
|
|
|
|
|
|
|
|
# Start Nushell session with provisioning library
|
|
|
|
|
|
provisioning nu
|
2026-01-08 09:55:37 +00:00
|
|
|
|
```plaintext
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Summary
|
|
|
|
|
|
|
|
|
|
|
|
You've successfully:
|
|
|
|
|
|
|
|
|
|
|
|
✅ Installed Nushell and essential tools
|
|
|
|
|
|
✅ Built and registered native plugins (10-50x faster operations)
|
|
|
|
|
|
✅ Cloned and configured the project
|
|
|
|
|
|
✅ Initialized a production workspace
|
|
|
|
|
|
✅ Configured provider credentials
|
|
|
|
|
|
✅ Deployed servers
|
|
|
|
|
|
✅ Installed Kubernetes and task services
|
|
|
|
|
|
✅ Created application clusters
|
|
|
|
|
|
✅ Verified complete deployment
|
|
|
|
|
|
|
|
|
|
|
|
**Your infrastructure is now ready for production use!**
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
**Estimated Total Time**: 30-60 minutes
|
|
|
|
|
|
**Next Guide**: [Update Infrastructure](update-infrastructure.md)
|
2026-01-08 09:55:37 +00:00
|
|
|
|
**Questions?**: Open an issue or contact <platform-team@example.com>
|
2025-12-11 21:50:42 +00:00
|
|
|
|
|
|
|
|
|
|
**Last Updated**: 2025-10-09
|
|
|
|
|
|
**Version**: 3.5.0
|