# Integration Testing Guide\n\n**Version**: 1.0.0\n**Last Updated**: 2025-10-06\n\nThis guide provides comprehensive documentation for the provisioning platform integration testing suite.\n\n## Table of Contents\n\n1. [Overview](#overview)\n2. [Test Infrastructure](#test-infrastructure)\n3. [Running Tests Locally](#running-tests-locally)\n4. [Running Tests on OrbStack](#running-tests-on-orbstack)\n5. [Writing New Tests](#writing-new-tests)\n6. [Test Organization](#test-organization)\n7. [CI/CD Integration](#cicd-integration)\n8. [Troubleshooting](#troubleshooting)\n\n---\n\n## Overview\n\nThe integration testing suite validates all four execution modes of the provisioning platform:\n\n- **Solo Mode**: Single-user, minimal services (orchestrator, CoreDNS, OCI registry)\n- **Multi-User Mode**: Multi-user support with Gitea, PostgreSQL, RBAC\n- **CI/CD Mode**: Automation mode with API server, service accounts\n- **Enterprise Mode**: Full enterprise features (Harbor, KMS, Prometheus, Grafana, ELK)\n\n### Key Features\n\n- ✅ **Comprehensive Coverage**: Tests for all 4 modes, 15+ services\n- ✅ **OrbStack Integration**: Tests deployable to OrbStack machine "provisioning"\n- ✅ **Parallel Execution**: Run independent tests in parallel for speed\n- ✅ **Automatic Cleanup**: Resources cleaned up automatically after tests\n- ✅ **Multiple Report Formats**: JUnit XML, HTML, JSON\n- ✅ **CI/CD Ready**: GitHub Actions and GitLab CI integration\n\n---\n\n## Test Infrastructure\n\n### Prerequisites\n\n1. **OrbStack Installed**:\n\n ```bash\n # Install OrbStack (macOS)\n brew install --cask orbstack\n ```\n\n2. **OrbStack Machine Named "provisioning"**:\n\n ```bash\n # Create OrbStack machine\n orb create provisioning\n\n # Verify machine is running\n orb status provisioning\n ```\n\n3. **Nushell 0.107.1+**:\n\n ```bash\n # Install Nushell\n brew install nushell\n ```\n\n4. **Docker CLI**:\n\n ```bash\n # Verify Docker is available\n docker version\n ```\n\n### Test Configuration\n\nThe test suite is configured via `provisioning/tests/integration/test_config.yaml`:\n\n```\n# OrbStack connection\norbstack:\n machine_name: "provisioning"\n connection:\n type: "docker"\n socket: "/var/run/docker.sock"\n\n# Service endpoints\nservices:\n orchestrator:\n host: "172.20.0.10"\n port: 8080\n\n coredns:\n host: "172.20.0.2"\n port: 53\n\n # ... more services\n```\n\n**Key Settings**:\n\n- `orbstack.machine_name`: Name of OrbStack machine to use\n- `services.*`: IP addresses and ports for deployed services\n- `test_execution.parallel.max_workers`: Number of parallel test workers\n- `test_execution.timeouts.*`: Timeout values for various operations\n\n---\n\n## Running Tests Locally\n\n### Quick Start\n\n1. **Setup Test Environment**:\n\n ```bash\n # Setup solo mode environment\n nu provisioning/tests/integration/setup_test_environment.nu --mode solo\n ```\n\n1. **Run Tests**:\n\n ```bash\n # Run all tests for solo mode\n nu provisioning/tests/integration/framework/test_runner.nu --mode solo\n\n # Run specific test file\n nu provisioning/tests/integration/modes/test_solo_mode.nu\n ```\n\n2. **Teardown Test Environment**:\n\n ```bash\n # Cleanup all resources\n nu provisioning/tests/integration/teardown_test_environment.nu --force\n ```\n\n### Test Runner Options\n\n```\nnu provisioning/tests/integration/framework/test_runner.nu \n --mode <mode> # Test specific mode (solo, multiuser, cicd, enterprise)\n --filter <pattern> # Filter tests by regex pattern\n --parallel <n> # Number of parallel workers (default: 1)\n --verbose # Detailed output\n --report <path> # Generate HTML report\n --skip-setup # Skip environment setup\n --skip-teardown # Skip environment teardown\n```\n\n**Examples**:\n\n```\n# Run all tests for all modes\nnu provisioning/tests/integration/framework/test_runner.nu\n\n# Run only solo mode tests\nnu provisioning/tests/integration/framework/test_ru
