Test Environment Guide

Version: 1.0.0 Date: 2025-10-06 Status: Production Ready

Overview

The Test Environment Service provides automated containerized testing for taskservs, servers, and multi-node clusters. Built into the orchestrator, it eliminates manual Docker management and provides realistic test scenarios.

Architecture

┌─────────────────────────────────────────────────┐
│         Orchestrator (port 8080)                │
│  ┌──────────────────────────────────────────┐  │
│  │  Test Orchestrator                       │  │
│  │  • Container Manager (Docker API)        │  │
│  │  • Network Isolation                     │  │
│  │  • Multi-node Topologies                 │  │
│  │  • Test Execution                        │  │
│  └──────────────────────────────────────────┘  │
└─────────────────────────────────────────────────┘
                      ↓
         ┌────────────────────────┐
         │   Docker Containers    │
         │  • Isolated Networks   │
         │  • Resource Limits     │
         │  • Volume Mounts       │
         └────────────────────────┘

Test Environment Types

1. Single Taskserv Test

Test individual taskserv in isolated container.

# Basic test
provisioning test env single kubernetes

# With resource limits
provisioning test env single redis --cpu 2000 --memory 4096

# Auto-start and cleanup
provisioning test quick postgres

2. Server Simulation

Simulate complete server with multiple taskservs.

# Server with taskservs
provisioning test env server web-01 [containerd kubernetes cilium]

# With infrastructure context
provisioning test env server db-01 [postgres redis] --infra prod-stack

3. Cluster Topology

Multi-node cluster simulation from templates.

# 3-node Kubernetes cluster
provisioning test topology load kubernetes_3node | test env cluster kubernetes --auto-start

# etcd cluster
provisioning test topology load etcd_cluster | test env cluster etcd

Quick Start

Prerequisites

Docker running:

docker ps  # Should work without errors

Orchestrator running:

cd provisioning/platform/orchestrator
./scripts/start-orchestrator.nu --background

Basic Workflow

# 1. Quick test (fastest)
provisioning test quick kubernetes

# 2. Or step-by-step
# Create environment
provisioning test env single kubernetes --auto-start

# List environments
provisioning test env list

# Check status
provisioning test env status <env-id>

# View logs
provisioning test env logs <env-id>

# Cleanup
provisioning test env cleanup <env-id>

Topology Templates

Available Templates

# List templates
provisioning test topology list

Template	Description	Nodes
`kubernetes_3node`	K8s HA cluster	1 CP + 2 workers
`kubernetes_single`	All-in-one K8s	1 node
`etcd_cluster`	etcd cluster	3 members
`containerd_test`	Standalone containerd	1 node
`postgres_redis`	Database stack	2 nodes

Using Templates

# Load and use template
provisioning test topology load kubernetes_3node | test env cluster kubernetes

# View template
provisioning test topology load etcd_cluster

Custom Topology

Create my-topology.toml:

[my_cluster]
name = "My Custom Cluster"
cluster_type = "custom"

[[my_cluster.nodes]]
name = "node-01"
role = "primary"
taskservs = ["postgres", "redis"]
[my_cluster.nodes.resources]
cpu_millicores = 2000
memory_mb = 4096

[[my_cluster.nodes]]
name = "node-02"
role = "replica"
taskservs = ["postgres"]
[my_cluster.nodes.resources]
cpu_millicores = 1000
memory_mb = 2048

[my_cluster.network]
subnet = "172.30.0.0/16"

Commands Reference

Environment Management

# Create from config
provisioning test env create <config>

# Single taskserv
provisioning test env single <taskserv> [--cpu N] [--memory MB]

# Server simulation
provisioning test env server <name> <taskservs> [--infra NAME]

# Cluster topology
provisioning test env cluster <type> <topology>

# List environments
provisioning test env list

# Get details
provisioning test env get <env-id>

# Show status
provisioning test env status <env-id>

Test Execution

# Run tests
provisioning test env run <env-id> [--tests [test1, test2]]

# View logs
provisioning test env logs <env-id>

# Cleanup
provisioning test env cleanup <env-id>

Quick Test

# One-command test (create, run, cleanup)
provisioning test quick <taskserv> [--infra NAME]

REST API

Create Environment

curl -X POST http://localhost:9090/test/environments/create \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "type": "single_taskserv",
      "taskserv": "kubernetes",
      "base_image": "ubuntu:22.04",
      "environment": {},
      "resources": {
        "cpu_millicores": 2000,
        "memory_mb": 4096
      }
    },
    "infra": "my-project",
    "auto_start": true,
    "auto_cleanup": false
  }'

List Environments

curl http://localhost:9090/test/environments

Run Tests

curl -X POST http://localhost:9090/test/environments/{id}/run \
  -H "Content-Type: application/json" \
  -d '{
    "tests": [],
    "timeout_seconds": 300
  }'

Cleanup

curl -X DELETE http://localhost:9090/test/environments/{id}

Use Cases

1. Taskserv Development

Test taskserv before deployment:

# Test new taskserv version
provisioning test env single my-taskserv --auto-start

# Check logs
provisioning test env logs <env-id>

2. Multi-Taskserv Integration

Test taskserv combinations:

# Test kubernetes + cilium + containerd
provisioning test env server k8s-test [kubernetes cilium containerd] --auto-start

3. Cluster Validation

Test cluster configurations:

# Test 3-node etcd cluster
provisioning test topology load etcd_cluster | test env cluster etcd --auto-start

4. CI/CD Integration

# .gitlab-ci.yml
test-taskserv:
  stage: test
  script:
    - provisioning test quick kubernetes
    - provisioning test quick redis
    - provisioning test quick postgres

Advanced Features

Resource Limits

# Custom CPU and memory
provisioning test env single postgres \
  --cpu 4000 \
  --memory 8192

Network Isolation

Each environment gets isolated network:

Subnet: 172.20.0.0/16 (default)
DNS enabled
Container-to-container communication

Auto-Cleanup

# Auto-cleanup after tests
provisioning test env single redis --auto-start --auto-cleanup

Multiple Environments

Run tests in parallel:

# Create multiple environments
provisioning test env single kubernetes --auto-start &
provisioning test env single postgres --auto-start &
provisioning test env single redis --auto-start &

wait

# List all
provisioning test env list

Troubleshooting

Docker not running

Error: Failed to connect to Docker

Solution:

# Check Docker
docker ps

# Start Docker daemon
sudo systemctl start docker  # Linux
open -a Docker  # macOS

Orchestrator not running

Error: Connection refused (port 8080)

Solution:

cd provisioning/platform/orchestrator
./scripts/start-orchestrator.nu --background

Environment creation fails

Check logs:

provisioning test env logs <env-id>

Check Docker:

docker ps -a
docker logs <container-id>

Out of resources

Error: Cannot allocate memory

Solution:

# Cleanup old environments
provisioning test env list | each {|env| provisioning test env cleanup $env.id }

# Or cleanup Docker
docker system prune -af

Best Practices

1. Use Templates

Reuse topology templates instead of recreating:

provisioning test topology load kubernetes_3node | test env cluster kubernetes

2. Auto-Cleanup

Always use auto-cleanup in CI/CD:

provisioning test quick <taskserv>  # Includes auto-cleanup

3. Resource Planning

Adjust resources based on needs:

Development: 1-2 cores, 2GB RAM
Integration: 2-4 cores, 4-8GB RAM
Production-like: 4+ cores, 8+ GB RAM

4. Parallel Testing

Run independent tests in parallel:

for taskserv in [kubernetes postgres redis] {
    provisioning test quick $taskserv &
}
wait

Configuration

Default Settings

Base image: ubuntu:22.04
CPU: 1000 millicores (1 core)
Memory: 2048 MB (2GB)
Network: 172.20.0.0/16

Custom Config

# Override defaults
provisioning test env single postgres \
  --base-image debian:12 \
  --cpu 2000 \
  --memory 4096

Version History

Version	Date	Changes
1.0.0	2025-10-06	Initial test environment service

Maintained By: Infrastructure Team

Keyboard shortcuts

Provisioning Platform Documentation