# Clusters

Clusters are coordinated groups of services deployed together. Provisioning provides cluster definitions for common deployment patterns.

## Available Clusters

### Web Cluster

Production-ready web application deployment with load balancing, TLS, and monitoring.

**Components**:
- Nginx load balancer
- Application servers (configurable count)
- PostgreSQL database
- Redis cache
- Prometheus monitoring
- Let's Encrypt TLS certificates

**Configuration**:

```nickel
{
  clusters = [{
    name = "web-production",
    type = 'web,
    config = {
      app_servers = 3,
      load_balancer = {
        public_ip = true,
        tls_enabled = true,
        domain = "example.com"
      },
      database = {
        size = 'medium,
        replicas = 2,
        backup_enabled = true
      },
      cache = {
        size = 'small,
        persistence = true
      }
    }
  }]
}
```

### OCI Registry Cluster

Private container registry with S3-compatible storage and authentication.

**Components**:
- Harbor registry
- MinIO object storage
- PostgreSQL database
- Redis cache
- TLS termination

**Configuration**:

```nickel
{
  clusters = [{
    name = "registry-private",
    type = 'oci_registry,
    config = {
      domain = "registry.example.com",
      storage = {
        backend = 'minio,
        size_gb = 500,
        replicas = 3
      },
      authentication = {
        method = 'ldap,  # or 'database, 'oidc
        admin_password = std.secret "REGISTRY_ADMIN_PASSWORD"
      }
    }
  }]
}
```

### Kubernetes Cluster

Multi-node Kubernetes cluster with networking, storage, and monitoring.

**Components**:
- Control plane nodes
- Worker node pools
- Cilium CNI
- Rook-Ceph storage
- Metrics server
- Ingress controller

**Configuration**:

```nickel
{
  clusters = [{
    name = "k8s-production",
    type = 'kubernetes,
    config = {
      control_plane = {
        nodes = 3,
        plan = 'medium,
        high_availability = true
      },
      node_pools = [
        {
          name = "general",
          nodes = 5,
          plan = 'large,
          labels = {workload = "general"}
        },
        {
          name = "gpu",
          nodes = 2,
          plan = 'xlarge,
          labels = {workload = "ml"}
        }
      ],
      networking = {
        cni = 'cilium,
        pod_cidr = "10.42.0.0/16",
        service_cidr = "10.43.0.0/16"
      },
      storage = {
        provider = 'rook-ceph,
        default_storage_class = "ceph-block"
      }
    }
  }]
}
```

## Cluster Deployment

### CLI Commands

```bash
# List available cluster types
provisioning cluster types

# Show cluster configuration template
provisioning cluster template web

# Deploy cluster
provisioning cluster deploy web-production

# Check cluster health
provisioning cluster health web-production

# Scale cluster
provisioning cluster scale web-production --app-servers 5

# Destroy cluster
provisioning cluster destroy web-production
```

### Deployment Lifecycle

1. **Validation** - Validate cluster configuration
2. **Infrastructure** - Provision servers, networks, storage
3. **Services** - Install and configure task services
4. **Integration** - Connect services together
5. **Health Check** - Verify cluster health
6. **Ready** - Cluster operational

## Cluster Orchestration

Clusters use dependency graphs for orchestration:

```plaintext
Web Cluster Dependency Graph:

servers ──┐
          ├──> database ──┐
networks ─┘               ├──> app_servers ──> load_balancer
                          │
                          ├──> cache ──────────┘
                          │
                          └──> monitoring
```

Services are deployed in dependency order with parallel execution where possible.

## Custom Cluster Definitions

Create custom cluster types:

```plaintext
provisioning/extensions/clusters/
└── my-cluster/
    ├── cluster.ncl           # Cluster definition
    ├── deploy.nu             # Deployment script
    ├── health-check.nu       # Health validation
    └── README.md
```

**cluster.ncl** schema:

```nickel
{
  name = "my-cluster",
  version = "1.0.0",
  description = "Custom cluster type",
  components = {
    servers = [{name = "app", count = 3, plan = 'medium}],
    services = ["nginx", "postgresql", "redis"],
  },
  config_schema = {
    domain | String,
    replicas | Number | default = 3,
  }
}
```

## Cluster Management

### Scaling

Scale cluster components:

```bash
# Scale application servers
provisioning cluster scale web-production --component app_servers --count 5

# Scale database replicas
provisioning cluster scale web-production --component database --replicas 3
```

### Updates

Rolling updates without downtime:

```bash
# Update application version
provisioning cluster update web-production --app-version 2.0.0

# Update infrastructure (e.g., server plans)
provisioning cluster update web-production --plan large
```

### Backup and Recovery

```bash
# Create cluster backup
provisioning cluster backup web-production

# Restore from backup
provisioning cluster restore web-production --backup 2024-01-15-snapshot

# List backups
provisioning cluster backups web-production
```

## Monitoring

Cluster health monitoring:

```bash
# Overall cluster health
provisioning cluster health web-production

# Component health
provisioning cluster health web-production --component database

# Metrics
provisioning cluster metrics web-production
```

Health checks validate:
- All services running
- Network connectivity
- Storage availability
- Resource utilization

## Best Practices

1. **Use predefined clusters** - Leverage built-in cluster types
2. **Define dependencies** - Explicit service dependencies
3. **Implement health checks** - Comprehensive validation
4. **Plan for scaling** - Design clusters for horizontal scaling
5. **Automate backups** - Regular backup schedules
6. **Monitor resources** - Track resource utilization
7. **Test disaster recovery** - Validate backup/restore procedures

## References

- [Task Services](task-services.md) - Service catalog
- [Batch Workflows](batch-workflows.md) - Multi-cluster orchestration
- [Providers](providers.md) - Infrastructure providers
