# Test Environment Service (v3.4.0)\n\n## 🚀 Test Environment Service Completed (2025-10-06)\n\nA comprehensive containerized test environment service has been integrated into the orchestrator, enabling automated testing of taskservs, complete\nservers, and multi-node clusters without manual Docker management.\n\n## Key Features\n\n- **Automated Container Management**: No manual Docker operations required\n- **Three Test Environment Types**: Single taskserv, server simulation, multi-node clusters\n- **Multi-Node Support**: Test complex topologies (Kubernetes HA, etcd clusters)\n- **Network Isolation**: Each test environment gets dedicated Docker networks\n- **Resource Management**: Configurable CPU, memory, and disk limits\n- **Topology Templates**: Predefined cluster configurations for common scenarios\n- **Auto-Cleanup**: Optional automatic cleanup after tests complete\n- **CI/CD Integration**: Easy integration into automated pipelines\n\n## Test Environment Types\n\n### 1. Single Taskserv Testing\n\nTest individual taskserv in isolated container:\n\n```\n# Quick test (create, run, cleanup)\nprovisioning test quick kubernetes\n\n# With custom resources\nprovisioning test env single postgres --cpu 2000 --memory 4096 --auto-start --auto-cleanup\n\n# With infrastructure context\nprovisioning test env single redis --infra my-project\n```\n\n### 2. Server Simulation\n\nTest complete server configurations with multiple taskservs:\n\n```\n# Simulate web server\nprovisioning test env server web-01 [containerd kubernetes cilium] --auto-start\n\n# Simulate database server\nprovisioning test env server db-01 [postgres redis] --infra prod-stack --auto-start\n```\n\n### 3. Multi-Node Cluster Topology\n\nTest complex cluster configurations before deployment:\n\n```\n# 3-node Kubernetes HA cluster\nprovisioning test topology load kubernetes_3node | test env cluster kubernetes --auto-start\n\n# etcd cluster\nprovisioning test topology load etcd_cluster | test env cluster etcd --auto-start\n\n# Single-node Kubernetes\nprovisioning test topology load kubernetes_single | test env cluster kubernetes\n```\n\n## Test Environment Management\n\n```\n# List all test environments\nprovisioning test env list\n\n# Check environment status\nprovisioning test env status <env-id>\n\n# View environment logs\nprovisioning test env logs <env-id>\n\n# Run tests in environment\nprovisioning test env run <env-id>\n\n# Cleanup environment\nprovisioning test env cleanup <env-id>\n```\n\n## Available Topology Templates\n\nPredefined multi-node cluster templates in `provisioning/config/test-topologies.toml`:\n\n| Template | Description | Nodes | Use Case |\n| ---------- | ------------- | ------- | ---------- |\n| `kubernetes_3node` | K8s HA cluster | 1 CP + 2 workers | Production-like testing |\n| `kubernetes_single` | All-in-one K8s | 1 node | Development testing |\n| `etcd_cluster` | etcd cluster | 3 members | Distributed consensus |\n| `containerd_test` | Standalone containerd | 1 node | Container runtime |\n| `postgres_redis` | Database stack | 2 nodes | Database integration |\n\n## REST API Endpoints\n\nThe orchestrator exposes test environment endpoints:\n\n- **Create Environment**: `POST http://localhost:9090/v1/test/environments/create`\n- **List Environments**: `GET http://localhost:9090/v1/test/environments`\n- **Get Environment**: `GET http://localhost:9090/v1/test/environments/{id}`\n- **Run Tests**: `POST http://localhost:9090/v1/test/environments/{id}/run`\n- **Cleanup**: `DELETE http://localhost:9090/v1/test/environments/{id}`\n- **Get Logs**: `GET http://localhost:9090/v1/test/environments/{id}/logs`\n\n## Prerequisites\n\n1. **Docker Running**: Test environments require Docker daemon\n\n   ```bash\n   docker ps  # Should work without errors\n   ```\n\n1. **Orchestrator Running**: Start the orchestrator to manage test containers\n\n   ```bash\n   cd provisioning/platform/orchestrator\n   ./scripts/start-orchestrator.nu --background\n   ```\n\n## Architecture\n\n```\nUser Command (CLI/API)\n    ↓\nTest Orchestrator (Rust)\n    ↓\nContainer Manager (bollard)\n    ↓\nDocker API\n    ↓\nIsolated Test Containers\n    • Dedicated networks\n    • Resource limits\n    • Volume mounts\n    • Multi-node support\n```\n\n## Configuration\n\n- **Topology Templates**: `provisioning/config/test-topologies.toml`\n- **Default Resources**: 1000 millicores CPU, 2048 MB memory\n- **Network**: 172.20.0.0/16 (default subnet)\n- **Base Image**: ubuntu:22.04 (configurable)\n\n## Use Cases\n\n1. **Taskserv Development**: Test new taskservs before deployment\n2. **Integration Testing**: Validate taskserv combinations\n3. **Cluster Validation**: Test multi-node configurations\n4. **CI/CD Integration**: Automated infrastructure testing\n5. **Production Simulation**: Test production-like deployments safely\n\n## CI/CD Integration Example\n\n```\n# GitLab CI\ntest-infrastructure:\n  stage: test\n  script:\n    - ./scripts/start-orchestrator.nu --background\n    - provisioning test quick kubernetes\n    - provisioning test quick postgres\n    - provisioning test quick redis\n    - provisioning test topology load kubernetes_3node |\n        test env cluster kubernetes --auto-start\n  artifacts:\n    when: on_failure\n    paths:\n      - test-logs/\n```\n\n## Documentation\n\nComplete documentation available:\n\n- **User Guide**: [Test Environment Guide](../testing/test-environment-guide.md)\n- **Detailed Usage**: [Test Environment Usage](../testing/test-environment-usage.md)\n- **Orchestrator README**: [Orchestrator](../operations/orchestrator-system.md)\n\n## Command Shortcuts\n\nTest commands are integrated into the CLI with shortcuts:\n\n- `test` or `tst` - Test command prefix\n- `test quick <taskserv>` - One-command test\n- `test env single/server/cluster` - Create test environments\n- `test topology load/list` - Manage topology templates
