provisioning/templates/workspaces/kubernetes/setup.md

Kubernetes Workspace Setup

This template provides a complete Kubernetes cluster configuration using the package-based provisioning system.

Prerequisites

1. Core provisioning package installed:

   kcl-packager.nu install --version latest
   

2. Module loader CLI available:

   module-loader --help
   

Setup Steps

1. Initialize Workspace

# Create workspace from template
cp -r provisioning/templates/workspaces/kubernetes ./my-k8s-cluster
cd my-k8s-cluster

# Initialize directory structure
workspace-init.nu . init

2. Load Required Taskservs

# Load Kubernetes components
module-loader load taskservs . [kubernetes, cilium, containerd]

# Verify loading
module-loader list taskservs .

3. Load Cloud Provider

# For UpCloud
module-loader load providers . [upcloud]

# For AWS
module-loader load providers . [aws]

# For local development
module-loader load providers . [local]

4. Configure Infrastructure

1. Edit servers.k to uncomment the import statements and taskserv configurations
2. Adjust server specifications, hostnames, and labels as needed
3. Configure provider-specific settings in the generated provider files

5. Validate Configuration

# Validate KCL configuration
kcl run servers.k

# Validate workspace
module-loader validate .

6. Deploy Cluster

# Create servers
provisioning server create --infra . --check

# Install taskservs
provisioning taskserv create kubernetes --infra .
provisioning taskserv create cilium --infra .
provisioning taskserv create containerd --infra .

# Verify cluster
kubectl get nodes

Configuration Details

Server Roles

• k8s-master-01: Control plane node running the Kubernetes API server, etcd, and scheduler
• k8s-worker-01/02: Worker nodes running kubelet and container runtime

Taskservs

• containerd: Container runtime for Kubernetes
• kubernetes: Core Kubernetes components (kubelet, kubeadm, kubectl)
• cilium: CNI (Container Network Interface) for pod networking

Network Configuration

• All nodes have public IPv4 for initial setup
• Cilium provides internal pod-to-pod networking
• SSH access on port 22 for management

Customization

Adding More Workers

Copy the worker node configuration in servers.k and modify:

• hostname
• title
• Any provider-specific settings

Different Container Runtime

Replace containerd taskserv with:

• crio: CRI-O runtime
• docker: Docker runtime (not recommended for production)

Different CNI

Replace cilium taskserv with:

• calico: Calico CNI
• flannel: Flannel CNI
• Built-in kubenet (remove CNI taskserv)

Storage

Add storage taskservs:

module-loader load taskservs . [rook-ceph, mayastor]

Then add to server taskserv configurations:

taskservs = [
    { name = "containerd", profile = "default" },
    { name = "kubernetes", profile = "worker" },
    { name = "cilium", profile = "worker" },
    { name = "rook-ceph", profile = "default" }
]

Troubleshooting

Module Import Errors

If you see import errors like "module not found":

1. Verify modules are loaded: module-loader list taskservs .
2. Check generated import files: ls .taskservs/
3. Reload modules if needed: module-loader load taskservs . [kubernetes, cilium, containerd]

Provider Configuration

Check provider-specific configuration in .providers/ directory after loading.

Kubernetes Setup Issues

1. Check taskserv installation logs in ./tmp/k8s-deployment/
2. Verify all nodes are reachable via SSH
3. Check firewall rules for Kubernetes ports (6443, 10250, etc.)