## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# From Scratch Guide

Complete walkthrough from zero to production-ready infrastructure deployment using the Provisioning platform. This guide covers installation, configuration, workspace setup, infrastructure definition, and deployment workflows.

## Overview

This guide walks you through:

- Installing prerequisites and the Provisioning platform
- Configuring cloud provider credentials
- Creating your first workspace
- Defining infrastructure using Nickel
- Deploying servers and task services
- Setting up Kubernetes clusters
- Implementing security best practices
- Monitoring and maintaining infrastructure

Time commitment: 2-3 hours for complete setup Prerequisites: Linux or macOS, terminal access, cloud provider account (optional)

## Phase 1: Installation

### System Prerequisites

Ensure your system meets minimum requirements:

    # Check OS (Linux or macOS)
    uname -s

    # Verify available disk space (minimum 10GB recommended)
    df -h ~

    # Check internet connectivity
    ping -c 3 github.com

### Install Required Tools

#### Nushell (Required)

    # macOS
    brew install nushell

    # Linux
    cargo install nu

    # Verify installation
    nu --version  # Expected: 0.109.1+

#### Nickel (Required)

    # macOS
    brew install nickel

    # Linux
    cargo install nickel-lang-cli

    # Verify installation
    nickel --version  # Expected: 1.15.1+

#### Additional Tools

    # SOPS for secrets management
    brew install sops  # macOS
    # or download from  [https://github.com/getsops/sops/releases](https://github.com/getsops/sops/releases)

    # Age for encryption
    brew install age  # macOS
    cargo install age  # Linux

    # K9s for Kubernetes management (optional)
    brew install derailed/k9s/k9s

    # Verify installations
    sops --version    # Expected: 3.10.2+
    age --version     # Expected: 1.2.1+
    k9s version       # Expected: 0.50.6+

### Install Provisioning Platform

#### Option 1: Using Installer Script (Recommended)

    # Download and run installer
    INSTALL_URL="https://raw.githubusercontent.com/yourusername/provisioning/main/install.sh"
    curl -sSL "$INSTALL_URL" | bash

    # Follow prompts to configure installation directory and path
    # Default: ~/.local/bin/provisioning

Installer performs:

- Downloads latest platform binaries
- Installs CLI to system PATH
- Creates default configuration structure
- Validates dependencies
- Runs health check

#### Option 2: Build from Source

    # Clone repository
    git clone  [https://github.com/yourusername/provisioning.git](https://github.com/yourusername/provisioning.git)
    cd provisioning

    # Build core CLI
    cd provisioning/core
    cargo build --release

    # Install to local bin
    cp target/release/provisioning ~/.local/bin/

    # Add to PATH (add to ~/.bashrc or ~/.zshrc)
    export PATH="$HOME/.local/bin:$PATH"

    # Verify installation
    provisioning version

### Platform Health Check

    # Verify installation
    provisioning setup check

    # Expected output:
    # ✓ Nushell 0.109.1 installed
    # ✓ Nickel 1.15.1 installed
    # ✓ SOPS 3.10.2 installed
    # ✓ Age 1.2.1 installed
    # ✓ Provisioning CLI installed
    # ✓ Configuration directory created
    # Platform ready for use

## Phase 2: Initial Configuration

### Generate User Configuration

    # Create user configuration directory
    mkdir -p ~/.config/provisioning

    # Generate default user config
    provisioning setup init-user-config

Generated configuration structure:

    ~/.config/provisioning/
    ├── user_config.yaml      # User preferences and workspace registry
    ├── credentials/          # Provider credentials (encrypted)
    ├── age/                  # Age encryption keys
    └── cache/                # CLI cache

### Configure Encryption

    # Generate Age key pair for secrets
    age-keygen -o ~/.config/provisioning/age/provisioning.key

    # Store public key
    age-keygen -y ~/.config/provisioning/age/provisioning.key > ~/.config/provisioning/age/provisioning.pub

    # Configure SOPS to use Age
    cat > ~/.config/sops/config.yaml <<EOF
    creation_rules:
      - path_regex: \.secret\.(yam| l tom| l json)$
        age: $(cat ~/.config/provisioning/age/provisioning.pub)
    EOF

### Provider Credentials

Configure credentials for your chosen cloud provider.

#### UpCloud Configuration

    # Edit user config
    nano ~/.config/provisioning/user_config.yaml

    # Add provider credentials
    cat >> ~/.config/provisioning/user_config.yaml <<EOF
    providers:
      upcloud:
        username: "your-upcloud-username"
        password_env: "UPCLOUD_PASSWORD"  # Read from environment variable
        default_zone: "de-fra1"
    EOF

    # Set environment variable (add to ~/.bashrc or ~/.zshrc)
    export UPCLOUD_PASSWORD="your-upcloud-password"

#### AWS Configuration

    # Add AWS credentials to user config
    cat >> ~/.config/provisioning/user_config.yaml <<EOF
    providers:
      aws:
        access_key_id_env: "AWS_ACCESS_KEY_ID"
        secret_access_key_env: "AWS_SECRET_ACCESS_KEY"
        default_region: "eu-west-1"
    EOF

    # Set environment variables
    export AWS_ACCESS_KEY_ID="your-access-key-id"
    export AWS_SECRET_ACCESS_KEY="your-secret-access-key"

#### Local Provider (Development)

    # Configure local provider for testing
    cat >> ~/.config/provisioning/user_config.yaml <<EOF
    providers:
      local:
        backend: "docker"  # or "podman", "libvirt"
        storage_path: "$HOME/.local/share/provisioning/local"
    EOF

    # Ensure Docker is running
    docker info

### Validate Configuration

    # Validate user configuration
    provisioning validate config

    # Test provider connectivity
    provisioning providers

    # Expected output:
    # PROVIDER    STATUS     REGION/ZONE
    # upcloud     connected  de-fra1
    # local       ready      localhost

## Phase 3: Create First Workspace

### Initialize Workspace

    # Create workspace for first project
    provisioning workspace init my-first-project

    # Navigate to workspace
    cd workspace_my_first_project

    # Verify structure
    ls -la

Workspace structure created:

    workspace_my_first_project/
    ├── infra/                   # Infrastructure definitions (Nickel)
    ├── config/                  # Workspace configuration
    │   ├── provisioning.yaml    # Workspace metadata
    │   ├── dev-defaults.toml    # Development defaults
    │   ├── test-defaults.toml   # Testing defaults
    │   └── prod-defaults.toml   # Production defaults
    ├── extensions/              # Workspace-specific extensions
    │   ├── providers/
    │   ├── taskservs/
    │   └── workflows/
    └── runtime/                 # State and logs (gitignored)
        ├── state/
        ├── checkpoints/
        └── logs/

### Configure Workspace

    # Edit workspace metadata
    nano config/provisioning.yaml

Example workspace configuration:

    workspace:
      name: my-first-project
      description: Learning Provisioning platform
      environment: development
      created: 2026-01-16T10:00:00Z

    defaults:
      provider: local
      region: localhost
      confirmation_required: false

    versioning:
      nushell: "0.109.1"
      nickel: "1.15.1"
      kubernetes: "1.29.0"

## Phase 4: Define Infrastructure

### Simple Server Configuration

Create your first infrastructure definition using Nickel:

    # Create server definition
    cat > infra/simple-server.ncl <<'EOF'
    {
      metadata = {
        name = "simple-server"
        provider = "local"
        environment = 'development
      }

      infrastructure = {
        servers = [
          {
            name = "dev-web-01"
            plan = "small"
            zone = "localhost"
            disk_size_gb = 25
            backup_enabled = false
            role = 'standalone
          }
        ]
      }

      services = {
        taskservs = ["containerd"]
      }
    }
    EOF

### Validate Infrastructure Schema

    # Type-check Nickel schema
    nickel typecheck infra/simple-server.ncl

    # Validate against platform contracts
    provisioning validate config --infra simple-server

    # Preview deployment
    provisioning server create --check --infra simple-server

Expected output:

    Infrastructure Plan: simple-server
    Provider: local
    Environment: development

    Servers to create:
      - dev-web-01 (small, standalone)
        Disk: 25 GB
        Backup: disabled

    Task services:
      - containerd

    Estimated resources:
      CPU: 1 core
      RAM: 1 GB
      Disk: 25 GB

    Validation: PASSED

### Deploy Infrastructure

    # Create server
    provisioning server create --infra simple-server --yes

    # Monitor deployment
    provisioning server status dev-web-01

Deployment progress:

    Creating server: dev-web-01...
      [████████████████████████] 100% - Container created
      [████████████████████████] 100% - Network configured
      [████████████████████████] 100% - SSH ready

    Server dev-web-01 created successfully
    IP Address: 172.17.0.2
    Status: running
    Provider: local (docker)

### Install Task Service

    # Install containerd
    provisioning taskserv create containerd --infra simple-server

    # Verify installation
    provisioning taskserv status containerd

Installation output:

    Installing containerd on dev-web-01...
      [████████████████████████] 100% - Dependencies resolved
      [████████████████████████] 100% - Containerd installed
      [████████████████████████] 100% - Service started
      [████████████████████████] 100% - Health check passed

    Containerd installed successfully
    Version: 1.7.0
    Runtime: runc

### Verify Deployment

    # SSH into server
    provisioning server ssh dev-web-01

    # Inside server - verify containerd
    sudo systemctl status containerd
    sudo ctr version

    # Exit server
    exit

    # List all resources
    provisioning server list
    provisioning taskserv list

## Phase 5: Kubernetes Cluster Deployment

### Define Kubernetes Infrastructure

    # Create Kubernetes cluster definition
    cat > infra/k8s-cluster.ncl <<'EOF'
    {
      metadata = {
        name = "k8s-dev-cluster"
        provider = "local"
        environment = 'development
      }

      infrastructure = {
        servers = [
          {
            name = "k8s-control-01"
            plan = "medium"
            role = 'control
            zone = "localhost"
            disk_size_gb = 50
          }
          {
            name = "k8s-worker-01"
            plan = "medium"
            role = 'worker
            zone = "localhost"
            disk_size_gb = 50
          }
          {
            name = "k8s-worker-02"
            plan = "medium"
            role = 'worker
            zone = "localhost"
            disk_size_gb = 50
          }
        ]
      }

      services = {
        taskservs = ["containerd", "etcd", "kubernetes", "cilium"]
      }

      kubernetes = {
        version = "1.29.0"
        pod_cidr = "10.244.0.0/16"
        service_cidr = "10.96.0.0/12"
        container_runtime = "containerd"
        cri_socket = "/run/containerd/containerd.sock"
      }
    }
    EOF

### Validate Kubernetes Configuration

    # Type-check schema
    nickel typecheck infra/k8s-cluster.ncl

    # Validate configuration
    provisioning validate config --infra k8s-cluster

    # Preview deployment
    provisioning cluster create --check --infra k8s-cluster

### Deploy Kubernetes Cluster

    # Create cluster infrastructure
    provisioning cluster create --infra k8s-cluster --yes

    # Monitor cluster deployment
    provisioning cluster status k8s-dev-cluster

Cluster deployment phases:

    Phase 1: Creating servers...
      [████████████████████████] 100% - 3/3 servers created

    Phase 2: Installing containerd...
      [████████████████████████] 100% - 3/3 nodes ready

    Phase 3: Installing etcd...
      [████████████████████████] 100% - Control plane ready

    Phase 4: Installing Kubernetes...
      [████████████████████████] 100% - API server available
      [████████████████████████] 100% - Workers joined

    Phase 5: Installing Cilium CNI...
      [████████████████████████] 100% - Network ready

    Kubernetes cluster deployed successfully
    Cluster: k8s-dev-cluster
    Control plane: k8s-control-01
    Workers: k8s-worker-01, k8s-worker-02

### Access Kubernetes Cluster

    # Get kubeconfig
    provisioning cluster kubeconfig k8s-dev-cluster > ~/.kube/config-dev

    # Set KUBECONFIG
    export KUBECONFIG=~/.kube/config-dev

    # Verify cluster
    kubectl get nodes

    # Expected output:
    # NAME              STATUS   ROLES           AGE   VERSION
    # k8s-control-01    Ready    control-plane   5m    v1.29.0
    # k8s-worker-01     Ready    <none>          4m    v1.29.0
    # k8s-worker-02     Ready    <none>          4m    v1.29.0

    # Use K9s for interactive management
    k9s

## Phase 6: Security Configuration

### Enable Audit Logging

    # Configure audit logging
    cat > config/audit-config.toml <<EOF
    [audit]
    enabled = true
    log_path = "runtime/logs/audit"
    retention_days = 90
    level = "info"

    [audit.filters]
    include_commands = ["server create", "server delete", "cluster deploy"]
    exclude_users = []
    EOF

### Configure SOPS for Secrets

    # Create secrets file
    cat > config/secrets.secret.yaml <<EOF
    database:
      password: "changeme-db-password"
      admin_user: "admin"

    kubernetes:
      service_account_key: "changeme-sa-key"
    EOF

    # Encrypt secrets with SOPS
    sops -e -i config/secrets.secret.yaml

    # Verify encryption
    cat config/secrets.secret.yaml  # Should show encrypted content

    # Decrypt when needed
    sops -d config/secrets.secret.yaml

### Enable MFA (Optional)

    # Enable multi-factor authentication
    provisioning security mfa enable

    # Scan QR code with authenticator app
    # Enter verification code

### Configure RBAC

    # Create role definition
    cat > config/rbac-roles.yaml <<EOF
    roles:
      - name: developer
        permissions:
          - server:read
          - server:create
          - taskserv:read
          - taskserv:install
        deny:
          - cluster:delete
          - config:modify

      - name: operator
        permissions:
          - "*:read"
          - server:*
          - taskserv:*
          - cluster:read
          - cluster:deploy

      - name: admin
        permissions:
          - "*:*"
    EOF

## Phase 7: Multi-Cloud Deployment

### Define Multi-Cloud Infrastructure

    # Create multi-cloud definition
    cat > infra/multi-cloud.ncl <<'EOF'
    {
      batch_workflow = {
        operations = [
          {
            id = "upcloud-frontend"
            provider = "upcloud"
            region = "de-fra1"
            servers = [
              {name = "upcloud-web-01", plan = "medium", role = 'web}
            ]
            taskservs = ["containerd", "nginx"]
          }
          {
            id = "aws-backend"
            provider = "aws"
            region = "eu-west-1"
            servers = [
              {name = "aws-api-01", plan = "t3.medium", role = 'api}
            ]
            taskservs = ["containerd", "docker"]
            dependencies = ["upcloud-frontend"]
          }
          {
            id = "local-database"
            provider = "local"
            region = "localhost"
            servers = [
              {name = "local-db-01", plan = "large", role = 'database}
            ]
            taskservs = ["postgresql"]
          }
        ]
        parallel_limit = 2
      }
    }
    EOF

### Deploy Multi-Cloud Infrastructure

    # Submit batch workflow
    provisioning batch submit infra/multi-cloud.ncl

    # Monitor workflow progress
    provisioning batch status

    # View detailed operation status
    provisioning batch operations

## Phase 8: Monitoring and Maintenance

### Platform Health Monitoring

    # Check platform health
    provisioning health

    # View service status
    provisioning service status orchestrator
    provisioning service status control-center

    # View logs
    provisioning logs --service orchestrator --tail 100

### Infrastructure Monitoring

    # List all servers
    provisioning server list --all-workspaces

    # Show server details
    provisioning server info k8s-control-01

    # Check task service status
    provisioning taskserv list
    provisioning taskserv health containerd

### Backup Configuration

    # Create backup
    provisioning backup create --type full --output ~/backups/provisioning-$(date +%Y%m%d).tar.gz

    # Schedule automatic backups
    provisioning backup schedule daily --time "02:00" --retention 7

## Phase 9: Advanced Workflows

### Custom Workflow Creation

    # Create custom workflow
    cat > extensions/workflows/deploy-app.ncl <<'EOF'
    {
      workflow = {
        name = "deploy-application"
        description = "Deploy application to Kubernetes"

        steps = [
          {
            name = "build-image"
            action = "docker-build"
            params = {dockerfile = "Dockerfile", tag = "myapp:latest"}
          }
          {
            name = "push-image"
            action = "docker-push"
            params = {image = "myapp:latest", registry = "registry.example.com"}
            depends_on = ["build-image"]
          }
          {
            name = "deploy-k8s"
            action = "kubectl-apply"
            params = {manifest = "k8s/deployment.yaml"}
            depends_on = ["push-image"]
          }
          {
            name = "verify-deployment"
            action = "kubectl-rollout-status"
            params = {deployment = "myapp"}
            depends_on = ["deploy-k8s"]
          }
        ]
      }
    }
    EOF

### Execute Custom Workflow

    # Run workflow
    provisioning workflow run deploy-application

    # Monitor workflow
    provisioning workflow status deploy-application

    # View workflow history
    provisioning workflow history

## Troubleshooting

### Common Issues

#### Server Creation Fails

    # Enable debug logging
    provisioning --debug server create --infra simple-server

    # Check provider connectivity
    provisioning providers

    # Validate credentials
    provisioning validate config

#### Task Service Installation Fails

    # Check server connectivity
    provisioning server ssh dev-web-01

    # Verify dependencies
    provisioning taskserv check-deps containerd

    # Retry installation
    provisioning taskserv create containerd --force

#### Cluster Deployment Fails

    # Check cluster status
    provisioning cluster status k8s-dev-cluster

    # View cluster logs
    provisioning cluster logs k8s-dev-cluster

    # Reset and retry
    provisioning cluster reset k8s-dev-cluster
    provisioning cluster create --infra k8s-cluster

## Next Steps

### Production Deployment

- Review Security Best Practices
- Configure Backup & Recovery
- Set up Monitoring
- Implement Disaster Recovery

### Advanced Features

- Explore Batch Workflows
- Configure Orchestrator
- Use Interactive Guides
- Develop Custom Extensions

### Learning Resources

- Nickel Guide - Infrastructure as code
- Workspace Management - Advanced workspace usage
- Multi-Cloud Deployment - Multi-cloud strategies
- API Reference - Complete API documentation

## Summary

You’ve completed the from-scratch guide and learned:

- Platform installation and configuration
- Provider credential setup
- Workspace creation and management
- Infrastructure definition with Nickel
- Server and task service deployment
- Kubernetes cluster deployment
- Security configuration
- Multi-cloud deployment
- Monitoring and maintenance
- Custom workflow creation

Your Provisioning platform is now ready for production use.