provisioning/docs/src/guides/from-scratch.md

# From Scratch Guide

Complete walkthrough from zero to production-ready infrastructure deployment using the Provisioning platform. This guide covers installation, configuration,
workspace setup, infrastructure definition, and deployment workflows.

## Overview

This guide walks you through:

- Installing prerequisites and the Provisioning platform
- Configuring cloud provider credentials
- Creating your first workspace
- Defining infrastructure using Nickel
- Deploying servers and task services
- Setting up Kubernetes clusters
- Implementing security best practices
- Monitoring and maintaining infrastructure

**Time commitment**: 2-3 hours for complete setup
**Prerequisites**: Linux or macOS, terminal access, cloud provider account (optional)

## Phase 1: Installation

### System Prerequisites

Ensure your system meets minimum requirements:

```bash
# Check OS (Linux or macOS)
uname -s

# Verify available disk space (minimum 10GB recommended)
df -h ~

# Check internet connectivity
ping -c 3 github.com
```

### Install Required Tools

#### Nushell (Required)

```bash
# macOS
brew install nushell

# Linux
cargo install nu

# Verify installation
nu --version  # Expected: 0.109.1+
```

#### Nickel (Required)

```bash
# macOS
brew install nickel

# Linux
cargo install nickel-lang-cli

# Verify installation
nickel --version  # Expected: 1.15.1+
```

#### Additional Tools

```bash
# SOPS for secrets management
brew install sops  # macOS
# or download from  [https://github.com/getsops/sops/releases](https://github.com/getsops/sops/releases)

# Age for encryption
brew install age  # macOS
cargo install age  # Linux

# K9s for Kubernetes management (optional)
brew install derailed/k9s/k9s

# Verify installations
sops --version    # Expected: 3.10.2+
age --version     # Expected: 1.2.1+
k9s version       # Expected: 0.50.6+
```

### Install Provisioning Platform

#### Option 1: Using Installer Script (Recommended)

```bash
# Download and run installer
INSTALL_URL="https://raw.githubusercontent.com/yourusername/provisioning/main/install.sh"
curl -sSL "$INSTALL_URL" | bash

# Follow prompts to configure installation directory and path
# Default: ~/.local/bin/provisioning
```

Installer performs:

- Downloads latest platform binaries
- Installs CLI to system PATH
- Creates default configuration structure
- Validates dependencies
- Runs health check

#### Option 2: Build from Source

```bash
# Clone repository
git clone  [https://github.com/yourusername/provisioning.git](https://github.com/yourusername/provisioning.git)
cd provisioning

# Build core CLI
cd provisioning/core
cargo build --release

# Install to local bin
cp target/release/provisioning ~/.local/bin/

# Add to PATH (add to ~/.bashrc or ~/.zshrc)
export PATH="$HOME/.local/bin:$PATH"

# Verify installation
provisioning version
```

### Platform Health Check

```bash
# Verify installation
provisioning setup check

# Expected output:
# ✓ Nushell 0.109.1 installed
# ✓ Nickel 1.15.1 installed
# ✓ SOPS 3.10.2 installed
# ✓ Age 1.2.1 installed
# ✓ Provisioning CLI installed
# ✓ Configuration directory created
# Platform ready for use
```

## Phase 2: Initial Configuration

### Generate User Configuration

```bash
# Create user configuration directory
mkdir -p ~/.config/provisioning

# Generate default user config
provisioning setup init-user-config
```

Generated configuration structure:

```text
~/.config/provisioning/
├── user_config.yaml      # User preferences and workspace registry
├── credentials/          # Provider credentials (encrypted)
├── age/                  # Age encryption keys
└── cache/                # CLI cache
```

### Configure Encryption

```bash
# Generate Age key pair for secrets
age-keygen -o ~/.config/provisioning/age/provisioning.key

# Store public key
age-keygen -y ~/.config/provisioning/age/provisioning.key > ~/.config/provisioning/age/provisioning.pub

# Configure SOPS to use Age
cat > ~/.config/sops/config.yaml <<EOF
creation_rules:
  - path_regex: \.secret\.(yam| l tom| l json)$
    age: $(cat ~/.config/provisioning/age/provisioning.pub)
EOF
```

### Provider Credentials

Configure credentials for your chosen cloud provider.

#### UpCloud Configuration

```bash
# Edit user config
nano ~/.config/provisioning/user_config.yaml

# Add provider credentials
cat >> ~/.config/provisioning/user_config.yaml <<EOF
providers:
  upcloud:
    username: "your-upcloud-username"
    password_env: "UPCLOUD_PASSWORD"  # Read from environment variable
    default_zone: "de-fra1"
EOF

# Set environment variable (add to ~/.bashrc or ~/.zshrc)
export UPCLOUD_PASSWORD="your-upcloud-password"
```

#### AWS Configuration

```bash
# Add AWS credentials to user config
cat >> ~/.config/provisioning/user_config.yaml <<EOF
providers:
  aws:
    access_key_id_env: "AWS_ACCESS_KEY_ID"
    secret_access_key_env: "AWS_SECRET_ACCESS_KEY"
    default_region: "eu-west-1"
EOF

# Set environment variables
export AWS_ACCESS_KEY_ID="your-access-key-id"
export AWS_SECRET_ACCESS_KEY="your-secret-access-key"
```

#### Local Provider (Development)

```bash
# Configure local provider for testing
cat >> ~/.config/provisioning/user_config.yaml <<EOF
providers:
  local:
    backend: "docker"  # or "podman", "libvirt"
    storage_path: "$HOME/.local/share/provisioning/local"
EOF

# Ensure Docker is running
docker info
```

### Validate Configuration

```bash
# Validate user configuration
provisioning validate config

# Test provider connectivity
provisioning providers

# Expected output:
# PROVIDER    STATUS     REGION/ZONE
# upcloud     connected  de-fra1
# local       ready      localhost
```

## Phase 3: Create First Workspace

### Initialize Workspace

```bash
# Create workspace for first project
provisioning workspace init my-first-project

# Navigate to workspace
cd workspace_my_first_project

# Verify structure
ls -la
```

Workspace structure created:

```text
workspace_my_first_project/
├── infra/                   # Infrastructure definitions (Nickel)
├── config/                  # Workspace configuration
│   ├── provisioning.yaml    # Workspace metadata
│   ├── dev-defaults.toml    # Development defaults
│   ├── test-defaults.toml   # Testing defaults
│   └── prod-defaults.toml   # Production defaults
├── extensions/              # Workspace-specific extensions
│   ├── providers/
│   ├── taskservs/
│   └── workflows/
└── runtime/                 # State and logs (gitignored)
    ├── state/
    ├── checkpoints/
    └── logs/
```

### Configure Workspace

```bash
# Edit workspace metadata
nano config/provisioning.yaml
```

Example workspace configuration:

```yaml
workspace:
  name: my-first-project
  description: Learning Provisioning platform
  environment: development
  created: 2026-01-16T10:00:00Z

defaults:
  provider: local
  region: localhost
  confirmation_required: false

versioning:
  nushell: "0.109.1"
  nickel: "1.15.1"
  kubernetes: "1.29.0"
```

## Phase 4: Define Infrastructure

### Simple Server Configuration

Create your first infrastructure definition using Nickel:

```bash
# Create server definition
cat > infra/simple-server.ncl <<'EOF'
{
  metadata = {
    name = "simple-server"
    provider = "local"
    environment = 'development
  }

  infrastructure = {
    servers = [
      {
        name = "dev-web-01"
        plan = "small"
        zone = "localhost"
        disk_size_gb = 25
        backup_enabled = false
        role = 'standalone
      }
    ]
  }

  services = {
    taskservs = ["containerd"]
  }
}
EOF
```

### Validate Infrastructure Schema

```bash
# Type-check Nickel schema
nickel typecheck infra/simple-server.ncl

# Validate against platform contracts
provisioning validate config --infra simple-server

# Preview deployment
provisioning server create --check --infra simple-server
```

Expected output:

```text
Infrastructure Plan: simple-server
Provider: local
Environment: development

Servers to create:
  - dev-web-01 (small, standalone)
    Disk: 25 GB
    Backup: disabled

Task services:
  - containerd

Estimated resources:
  CPU: 1 core
  RAM: 1 GB
  Disk: 25 GB

Validation: PASSED
```

### Deploy Infrastructure

```bash
# Create server
provisioning server create --infra simple-server --yes

# Monitor deployment
provisioning server status dev-web-01
```

Deployment progress:

```text
Creating server: dev-web-01...
  [████████████████████████] 100% - Container created
  [████████████████████████] 100% - Network configured
  [████████████████████████] 100% - SSH ready

Server dev-web-01 created successfully
IP Address: 172.17.0.2
Status: running
Provider: local (docker)
```

### Install Task Service

```bash
# Install containerd
provisioning taskserv create containerd --infra simple-server

# Verify installation
provisioning taskserv status containerd
```

Installation output:

```text
Installing containerd on dev-web-01...
  [████████████████████████] 100% - Dependencies resolved
  [████████████████████████] 100% - Containerd installed
  [████████████████████████] 100% - Service started
  [████████████████████████] 100% - Health check passed

Containerd installed successfully
Version: 1.7.0
Runtime: runc
```

### Verify Deployment

```bash
# SSH into server
provisioning server ssh dev-web-01

# Inside server - verify containerd
sudo systemctl status containerd
sudo ctr version

# Exit server
exit

# List all resources
provisioning server list
provisioning taskserv list
```

## Phase 5: Kubernetes Cluster Deployment

### Define Kubernetes Infrastructure

```bash
# Create Kubernetes cluster definition
cat > infra/k8s-cluster.ncl <<'EOF'
{
  metadata = {
    name = "k8s-dev-cluster"
    provider = "local"
    environment = 'development
  }

  infrastructure = {
    servers = [
      {
        name = "k8s-control-01"
        plan = "medium"
        role = 'control
        zone = "localhost"
        disk_size_gb = 50
      }
      {
        name = "k8s-worker-01"
        plan = "medium"
        role = 'worker
        zone = "localhost"
        disk_size_gb = 50
      }
      {
        name = "k8s-worker-02"
        plan = "medium"
        role = 'worker
        zone = "localhost"
        disk_size_gb = 50
      }
    ]
  }

  services = {
    taskservs = ["containerd", "etcd", "kubernetes", "cilium"]
  }

  kubernetes = {
    version = "1.29.0"
    pod_cidr = "10.244.0.0/16"
    service_cidr = "10.96.0.0/12"
    container_runtime = "containerd"
    cri_socket = "/run/containerd/containerd.sock"
  }
}
EOF
```

### Validate Kubernetes Configuration

```bash
# Type-check schema
nickel typecheck infra/k8s-cluster.ncl

# Validate configuration
provisioning validate config --infra k8s-cluster

# Preview deployment
provisioning cluster create --check --infra k8s-cluster
```

### Deploy Kubernetes Cluster

```bash
# Create cluster infrastructure
provisioning cluster create --infra k8s-cluster --yes

# Monitor cluster deployment
provisioning cluster status k8s-dev-cluster
```

Cluster deployment phases:

```text
Phase 1: Creating servers...
  [████████████████████████] 100% - 3/3 servers created

Phase 2: Installing containerd...
  [████████████████████████] 100% - 3/3 nodes ready

Phase 3: Installing etcd...
  [████████████████████████] 100% - Control plane ready

Phase 4: Installing Kubernetes...
  [████████████████████████] 100% - API server available
  [████████████████████████] 100% - Workers joined

Phase 5: Installing Cilium CNI...
  [████████████████████████] 100% - Network ready

Kubernetes cluster deployed successfully
Cluster: k8s-dev-cluster
Control plane: k8s-control-01
Workers: k8s-worker-01, k8s-worker-02
```

### Access Kubernetes Cluster

```bash
# Get kubeconfig
provisioning cluster kubeconfig k8s-dev-cluster > ~/.kube/config-dev

# Set KUBECONFIG
export KUBECONFIG=~/.kube/config-dev

# Verify cluster
kubectl get nodes

# Expected output:
# NAME              STATUS   ROLES           AGE   VERSION
# k8s-control-01    Ready    control-plane   5m    v1.29.0
# k8s-worker-01     Ready    <none>          4m    v1.29.0
# k8s-worker-02     Ready    <none>          4m    v1.29.0

# Use K9s for interactive management
k9s
```

## Phase 6: Security Configuration

### Enable Audit Logging

```bash
# Configure audit logging
cat > config/audit-config.toml <<EOF
[audit]
enabled = true
log_path = "runtime/logs/audit"
retention_days = 90
level = "info"

[audit.filters]
include_commands = ["server create", "server delete", "cluster deploy"]
exclude_users = []
EOF
```

### Configure SOPS for Secrets

```bash
# Create secrets file
cat > config/secrets.secret.yaml <<EOF
database:
  password: "changeme-db-password"
  admin_user: "admin"

kubernetes:
  service_account_key: "changeme-sa-key"
EOF

# Encrypt secrets with SOPS
sops -e -i config/secrets.secret.yaml

# Verify encryption
cat config/secrets.secret.yaml  # Should show encrypted content

# Decrypt when needed
sops -d config/secrets.secret.yaml
```

### Enable MFA (Optional)

```bash
# Enable multi-factor authentication
provisioning security mfa enable

# Scan QR code with authenticator app
# Enter verification code
```

### Configure RBAC

```bash
# Create role definition
cat > config/rbac-roles.yaml <<EOF
roles:
  - name: developer
    permissions:
      - server:read
      - server:create
      - taskserv:read
      - taskserv:install
    deny:
      - cluster:delete
      - config:modify

  - name: operator
    permissions:
      - "*:read"
      - server:*
      - taskserv:*
      - cluster:read
      - cluster:deploy

  - name: admin
    permissions:
      - "*:*"
EOF
```

## Phase 7: Multi-Cloud Deployment

### Define Multi-Cloud Infrastructure

```bash
# Create multi-cloud definition
cat > infra/multi-cloud.ncl <<'EOF'
{
  batch_workflow = {
    operations = [
      {
        id = "upcloud-frontend"
        provider = "upcloud"
        region = "de-fra1"
        servers = [
          {name = "upcloud-web-01", plan = "medium", role = 'web}
        ]
        taskservs = ["containerd", "nginx"]
      }
      {
        id = "aws-backend"
        provider = "aws"
        region = "eu-west-1"
        servers = [
          {name = "aws-api-01", plan = "t3.medium", role = 'api}
        ]
        taskservs = ["containerd", "docker"]
        dependencies = ["upcloud-frontend"]
      }
      {
        id = "local-database"
        provider = "local"
        region = "localhost"
        servers = [
          {name = "local-db-01", plan = "large", role = 'database}
        ]
        taskservs = ["postgresql"]
      }
    ]
    parallel_limit = 2
  }
}
EOF
```

### Deploy Multi-Cloud Infrastructure

```bash
# Submit batch workflow
provisioning batch submit infra/multi-cloud.ncl

# Monitor workflow progress
provisioning batch status

# View detailed operation status
provisioning batch operations
```

## Phase 8: Monitoring and Maintenance

### Platform Health Monitoring

```bash
# Check platform health
provisioning health

# View service status
provisioning service status orchestrator
provisioning service status control-center

# View logs
provisioning logs --service orchestrator --tail 100
```

### Infrastructure Monitoring

```bash
# List all servers
provisioning server list --all-workspaces

# Show server details
provisioning server info k8s-control-01

# Check task service status
provisioning taskserv list
provisioning taskserv health containerd
```

### Backup Configuration

```bash
# Create backup
provisioning backup create --type full --output ~/backups/provisioning-$(date +%Y%m%d).tar.gz

# Schedule automatic backups
provisioning backup schedule daily --time "02:00" --retention 7
```

## Phase 9: Advanced Workflows

### Custom Workflow Creation

```bash
# Create custom workflow
cat > extensions/workflows/deploy-app.ncl <<'EOF'
{
  workflow = {
    name = "deploy-application"
    description = "Deploy application to Kubernetes"

    steps = [
      {
        name = "build-image"
        action = "docker-build"
        params = {dockerfile = "Dockerfile", tag = "myapp:latest"}
      }
      {
        name = "push-image"
        action = "docker-push"
        params = {image = "myapp:latest", registry = "registry.example.com"}
        depends_on = ["build-image"]
      }
      {
        name = "deploy-k8s"
        action = "kubectl-apply"
        params = {manifest = "k8s/deployment.yaml"}
        depends_on = ["push-image"]
      }
      {
        name = "verify-deployment"
        action = "kubectl-rollout-status"
        params = {deployment = "myapp"}
        depends_on = ["deploy-k8s"]
      }
    ]
  }
}
EOF
```

### Execute Custom Workflow

```bash
# Run workflow
provisioning workflow run deploy-application

# Monitor workflow
provisioning workflow status deploy-application

# View workflow history
provisioning workflow history
```

## Troubleshooting

### Common Issues

#### Server Creation Fails

```bash
# Enable debug logging
provisioning --debug server create --infra simple-server

# Check provider connectivity
provisioning providers

# Validate credentials
provisioning validate config
```

#### Task Service Installation Fails

```bash
# Check server connectivity
provisioning server ssh dev-web-01

# Verify dependencies
provisioning taskserv check-deps containerd

# Retry installation
provisioning taskserv create containerd --force
```

#### Cluster Deployment Fails

```bash
# Check cluster status
provisioning cluster status k8s-dev-cluster

# View cluster logs
provisioning cluster logs k8s-dev-cluster

# Reset and retry
provisioning cluster reset k8s-dev-cluster
provisioning cluster create --infra k8s-cluster
```

## Next Steps

### Production Deployment

- Review [Security Best Practices](../security/README.md)
- Configure [Backup & Recovery](../operations/backup-recovery.md)
- Set up [Monitoring](../operations/monitoring.md)
- Implement [Disaster Recovery](disaster-recovery.md)

### Advanced Features

- Explore [Batch Workflows](../infrastructure/batch-workflows.md)
- Configure [Orchestrator](../features/orchestrator.md)
- Use [Interactive Guides](../features/interactive-guides.md)
- Develop [Custom Extensions](custom-extensions.md)

### Learning Resources

- [Nickel Guide](../infrastructure/nickel-guide.md) - Infrastructure as code
- [Workspace Management](workspace-management.md) - Advanced workspace usage
- [Multi-Cloud Deployment](multi-cloud-deployment.md) - Multi-cloud strategies
- [API Reference](../api-reference/README.md) - Complete API documentation

## Summary

You've completed the from-scratch guide and learned:

- Platform installation and configuration
- Provider credential setup
- Workspace creation and management
- Infrastructure definition with Nickel
- Server and task service deployment
- Kubernetes cluster deployment
- Security configuration
- Multi-cloud deployment
- Monitoring and maintenance
- Custom workflow creation

Your Provisioning platform is now ready for production use.