28 KiB
28 KiB
\n┌─────────────────────────────────────────┐\n│ Service Management CLI │\n│ (platform/services commands) │\n└─────────────────┬───────────────────────┘\n │\n ┌──────────┴──────────┐\n │ │\n ▼ ▼\n┌──────────────┐ ┌───────────────┐\n│ Manager │ │ Lifecycle │\n│ (Core) │ │ (Start/Stop)│\n└──────┬───────┘ └───────┬───────┘\n │ │\n ▼ ▼\n┌──────────────┐ ┌───────────────┐\n│ Health │ │ Dependencies │\n│ (Checks) │ │ (Resolution) │\n└──────────────┘ └───────────────┘\n │ │\n └────────┬───────────┘\n │\n ▼\n ┌────────────────┐\n │ Pre-flight │\n │ (Validation) │\n └────────────────┘\n\n\n### Component Responsibilities\n\nManager (manager.nu)\n\n- Service registry loading\n- Service status tracking\n- State persistence\n\nLifecycle (lifecycle.nu)\n\n- Service start/stop operations\n- Deployment mode handling\n- Process management\n\nHealth (health.nu)\n\n- Health check execution\n- HTTP/TCP/Command/File checks\n- Continuous monitoring\n\nDependencies (dependencies.nu)\n\n- Dependency graph analysis\n- Topological sorting\n- Startup order calculation\n\nPre-flight (preflight.nu)\n\n- Prerequisite validation\n- Conflict detection\n- Auto-start orchestration\n\n---\n\n## Service Registry\n\n### Configuration File\n\nLocation: provisioning/config/services.toml\n\n### Service Definition Structure\n\n\n[services.<service-name>]\nname = "<service-name>"\ntype = "platform" | "infrastructure" | "utility"\ncategory = "orchestration" | "auth" | "dns" | "git" | "registry" | "api" | "ui"\ndescription = "Service description"\nrequired_for = ["operation1", "operation2"]\ndependencies = ["dependency1", "dependency2"]\nconflicts = ["conflicting-service"]\n\n[services.<service-name>.deployment]\nmode = "binary" | "docker" | "docker-compose" | "kubernetes" | "remote"\n\n# Mode-specific configuration\n[services.<service-name>.deployment.binary]\nbinary_path = "/path/to/binary"\nargs = ["--arg1", "value1"]\nworking_dir = "/working/directory"\nenv = { KEY = "value" }\n\n[services.<service-name>.health_check]\ntype = "http" | "tcp" | "command" | "file" | "none"\ninterval = 10\nretries = 3\ntimeout = 5\n\n[services.<service-name>.health_check.http]\nendpoint = "http://localhost:9090/health"\nexpected_status = 200\nmethod = "GET"\n\n[services.<service-name>.startup]\nauto_start = true\nstart_timeout = 30\nstart_order = 10\nrestart_on_failure = true\nmax_restarts = 3\n\n\n### Example: Orchestrator Service\n\n\n[services.orchestrator]\nname = "orchestrator"\ntype = "platform"\ncategory = "orchestration"\ndescription = "Rust-based orchestrator for workflow coordination"\nrequired_for = ["server", "taskserv", "cluster", "workflow", "batch"]\n\n[services.orchestrator.deployment]\nmode = "binary"\n\n[services.orchestrator.deployment.binary]\nbinary_path = "${HOME}/.provisioning/bin/provisioning-orchestrator"\nargs = ["--port", "8080", "--data-dir", "${HOME}/.provisioning/orchestrator/data"]\n\n[services.orchestrator.health_check]\ntype = "http"\n\n[services.orchestrator.health_check.http]\nendpoint = "http://localhost:9090/health"\nexpected_status = 200\n\n[services.orchestrator.startup]\nauto_start = true\nstart_timeout = 30\nstart_order = 10\n\n\n---\n\n## Platform Commands\n\nPlatform commands manage all services as a cohesive system.\n\n### Start Platform\n\nStart all auto-start services or specific services:\n\n\n# Start all auto-start services\nprovisioning platform start\n\n# Start specific services (with dependencies)\nprovisioning platform start orchestrator control-center\n\n# Force restart if already running\nprovisioning platform start --force orchestrator\n\n\nBehavior:\n\n1. Resolves dependencies\n2. Calculates startup order (topological sort)\n3. Starts services in correct order\n4. Waits for health checks\n5. Reports success/failure\n\n### Stop Platform\n\nStop all running services or specific services:\n\n\n# Stop all running services\nprovisioning platform stop\n\n# Stop specific services\nprovisioning platform stop orchestrator control-center\n\n# Force stop (kill -9)\nprovisioning platform stop --force orchestrator\n\n\nBehavior:\n\n1. Checks for dependent services\n2. Stops in reverse dependency order\n3. Updates service state\n4. Cleans up PID files\n\n### Restart Platform\n\nRestart running services:\n\n\n# Restart all running services\nprovisioning platform restart\n\n# Restart specific services\nprovisioning platform restart orchestrator\n\n\n### Platform Status\n\nShow status of all services:\n\n\nprovisioning platform status\n\n\nOutput:\n\n\nPlatform Services Status\n\nRunning: 3/7\n\n=== ORCHESTRATION ===\n 🟢 orchestrator - running (uptime: 3600s) ✅\n\n=== UI ===\n 🟢 control-center - running (uptime: 3550s) ✅\n\n=== DNS ===\n ⚪ coredns - stopped ❓\n\n=== GIT ===\n ⚪ gitea - stopped ❓\n\n=== REGISTRY ===\n ⚪ oci-registry - stopped ❓\n\n=== API ===\n 🟢 mcp-server - running (uptime: 3540s) ✅\n ⚪ api-gateway - stopped ❓\n\n\n### Platform Health\n\nCheck health of all running services:\n\n\nprovisioning platform health\n\n\nOutput:\n\n\nPlatform Health Check\n\n✅ orchestrator: Healthy - HTTP health check passed\n✅ control-center: Healthy - HTTP status 200 matches expected\n⚪ coredns: Not running\n✅ mcp-server: Healthy - HTTP health check passed\n\nSummary: 3 healthy, 0 unhealthy, 4 not running\n\n\n### Platform Logs\n\nView service logs:\n\n\n# View last 50 lines\nprovisioning platform logs orchestrator\n\n# View last 100 lines\nprovisioning platform logs orchestrator --lines 100\n\n# Follow logs in real-time\nprovisioning platform logs orchestrator --follow\n\n\n---\n\n## Service Commands\n\nIndividual service management commands.\n\n### List Services\n\n\n# List all services\nprovisioning services list\n\n# List only running services\nprovisioning services list --running\n\n# Filter by category\nprovisioning services list --category orchestration\n\n\nOutput:\n\n\nname type category status deployment_mode auto_start\norchestrator platform orchestration running binary true\ncontrol-center platform ui stopped binary false\ncoredns infrastructure dns stopped docker false\n\n\n### Service Status\n\nGet detailed status of a service:\n\n\nprovisioning services status orchestrator\n\n\nOutput:\n\n\nService: orchestrator\nType: platform\nCategory: orchestration\nStatus: running\nDeployment: binary\nHealth: healthy\nAuto-start: true\nPID: 12345\nUptime: 3600s\nDependencies: []\n\n\n### Start Service\n\n\n# Start service (with pre-flight checks)\nprovisioning services start orchestrator\n\n# Force start (skip checks)\nprovisioning services start orchestrator --force\n\n\nPre-flight Checks:\n\n1. Validate prerequisites (binary exists, Docker running, etc.)\n2. Check for conflicts\n3. Verify dependencies are running\n4. Auto-start dependencies if needed\n\n### Stop Service\n\n\n# Stop service (with dependency check)\nprovisioning services stop orchestrator\n\n# Force stop (ignore dependents)\nprovisioning services stop orchestrator --force\n\n\n### Restart Service\n\n\nprovisioning services restart orchestrator\n\n\n### Service Health\n\nCheck service health:\n\n\nprovisioning services health orchestrator\n\n\nOutput:\n\n\nService: orchestrator\nStatus: healthy\nHealthy: true\nMessage: HTTP health check passed\nCheck type: http\nCheck duration: 15 ms\n\n\n### Service Logs\n\n\n# View logs\nprovisioning services logs orchestrator\n\n# Follow logs\nprovisioning services logs orchestrator --follow\n\n# Custom line count\nprovisioning services logs orchestrator --lines 200\n\n\n### Check Required Services\n\nCheck which services are required for an operation:\n\n\nprovisioning services check server\n\n\nOutput:\n\n\nOperation: server\nRequired services: orchestrator\nAll running: true\n\n\n### Service Dependencies\n\nView dependency graph:\n\n\n# View all dependencies\nprovisioning services dependencies\n\n# View specific service dependencies\nprovisioning services dependencies control-center\n\n\n### Validate Services\n\nValidate all service configurations:\n\n\nprovisioning services validate\n\n\nOutput:\n\n\nTotal services: 7\nValid: 6\nInvalid: 1\n\nInvalid services:\n ❌ coredns:\n - Docker is not installed or not running\n\n\n### Readiness Report\n\nGet platform readiness report:\n\n\nprovisioning services readiness\n\n\nOutput:\n\n\nPlatform Readiness Report\n\nTotal services: 7\nRunning: 3\nReady to start: 6\n\nServices:\n 🟢 orchestrator - platform - orchestration\n 🟢 control-center - platform - ui\n 🔴 coredns - infrastructure - dns\n Issues: 1\n 🟡 gitea - infrastructure - git\n\n\n### Monitor Service\n\nContinuous health monitoring:\n\n\n# Monitor with default interval (30s)\nprovisioning services monitor orchestrator\n\n# Custom interval\nprovisioning services monitor orchestrator --interval 10\n\n\n---\n\n## Deployment Modes\n\n### Binary Deployment\n\nRun services as native binaries.\n\nConfiguration:\n\n\n[services.orchestrator.deployment]\nmode = "binary"\n\n[services.orchestrator.deployment.binary]\nbinary_path = "${HOME}/.provisioning/bin/provisioning-orchestrator"\nargs = ["--port", "8080"]\nworking_dir = "${HOME}/.provisioning/orchestrator"\nenv = { RUST_LOG = "info" }\n\n\nProcess Management:\n\n- PID tracking in ~/.provisioning/services/pids/\n- Log output to ~/.provisioning/services/logs/\n- State tracking in ~/.provisioning/services/state/\n\n### Docker Deployment\n\nRun services as Docker containers.\n\nConfiguration:\n\n\n[services.coredns.deployment]\nmode = "docker"\n\n[services.coredns.deployment.docker]\nimage = "coredns/coredns:1.11.1"\ncontainer_name = "provisioning-coredns"\nports = ["5353:53/udp"]\nvolumes = ["${HOME}/.provisioning/coredns/Corefile:/Corefile:ro"]\nrestart_policy = "unless-stopped"\n\n\nPrerequisites:\n\n- Docker daemon running\n- Docker CLI installed\n\n### Docker Compose Deployment\n\nRun services via Docker Compose.\n\nConfiguration:\n\n\n[services.platform.deployment]\nmode = "docker-compose"\n\n[services.platform.deployment.docker_compose]\ncompose_file = "${HOME}/.provisioning/platform/docker-compose.yaml"\nservice_name = "orchestrator"\nproject_name = "provisioning"\n\n\nFile: provisioning/platform/docker-compose.yaml\n\n### Kubernetes Deployment\n\nRun services on Kubernetes.\n\nConfiguration:\n\n\n[services.orchestrator.deployment]\nmode = "kubernetes"\n\n[services.orchestrator.deployment.kubernetes]\nnamespace = "provisioning"\ndeployment_name = "orchestrator"\nmanifests_path = "${HOME}/.provisioning/k8s/orchestrator/"\n\n\nPrerequisites:\n\n- kubectl installed and configured\n- Kubernetes cluster accessible\n\n### Remote Deployment\n\nConnect to remotely-running services.\n\nConfiguration:\n\n\n[services.orchestrator.deployment]\nmode = "remote"\n\n[services.orchestrator.deployment.remote]\nendpoint = "https://orchestrator.example.com"\ntls_enabled = true\nauth_token_path = "${HOME}/.provisioning/tokens/orchestrator.token"\n\n\n---\n\n## Health Monitoring\n\n### Health Check Types\n\n#### HTTP Health Check\n\n\n[services.orchestrator.health_check]\ntype = "http"\n\n[services.orchestrator.health_check.http]\nendpoint = "http://localhost:9090/health"\nexpected_status = 200\nmethod = "GET"\n\n\n#### TCP Health Check\n\n\n[services.coredns.health_check]\ntype = "tcp"\n\n[services.coredns.health_check.tcp]\nhost = "localhost"\nport = 5353\n\n\n#### Command Health Check\n\n\n[services.custom.health_check]\ntype = "command"\n\n[services.custom.health_check.command]\ncommand = "systemctl is-active myservice"\nexpected_exit_code = 0\n\n\n#### File Health Check\n\n\n[services.custom.health_check]\ntype = "file"\n\n[services.custom.health_check.file]\npath = "/var/run/myservice.pid"\nmust_exist = true\n\n\n### Health Check Configuration\n\n- interval: Seconds between checks (default: 10)\n- retries: Max retry attempts (default: 3)\n- timeout: Check timeout in seconds (default: 5)\n\n### Continuous Monitoring\n\n\nprovisioning services monitor orchestrator --interval 30\n\n\nOutput:\n\n\nStarting health monitoring for orchestrator (interval: 30s)\nPress Ctrl+C to stop\n2025-10-06 14:30:00 ✅ orchestrator: HTTP health check passed\n2025-10-06 14:30:30 ✅ orchestrator: HTTP health check passed\n2025-10-06 14:31:00 ✅ orchestrator: HTTP health check passed\n\n\n---\n\n## Dependency Management\n\n### Dependency Graph\n\nServices can depend on other services:\n\n\n[services.control-center]\ndependencies = ["orchestrator"]\n\n[services.api-gateway]\ndependencies = ["orchestrator", "control-center", "mcp-server"]\n\n\n### Startup Order\n\nServices start in topological order:\n\n\norchestrator (order: 10)\n └─> control-center (order: 20)\n └─> api-gateway (order: 45)\n\n\n### Dependency Resolution\n\nAutomatic dependency resolution when starting services:\n\n\n# Starting control-center automatically starts orchestrator first\nprovisioning services start control-center\n\n\nOutput:\n\n\nStarting dependency: orchestrator\n✅ Started orchestrator with PID 12345\nWaiting for orchestrator to become healthy...\n✅ Service orchestrator is healthy\nStarting service: control-center\n✅ Started control-center with PID 12346\n✅ Service control-center is healthy\n\n\n### Conflicts\n\nServices can conflict with each other:\n\n\n[services.coredns]\nconflicts = ["dnsmasq", "systemd-resolved"]\n\n\nAttempting to start a conflicting service will fail:\n\n\nprovisioning services start coredns\n\n\nOutput:\n\n\n❌ Pre-flight check failed: conflicts\nConflicting services running: dnsmasq\n\n\n### Reverse Dependencies\n\nCheck which services depend on a service:\n\n\nprovisioning services dependencies orchestrator\n\n\nOutput:\n\n\n## orchestrator\n- Type: platform\n- Category: orchestration\n- Required by:\n - control-center\n - mcp-server\n - api-gateway\n\n\n### Safe Stop\n\nSystem prevents stopping services with running dependents:\n\n\nprovisioning services stop orchestrator\n\n\nOutput:\n\n\n❌ Cannot stop orchestrator:\n Dependent services running: control-center, mcp-server, api-gateway\n Use --force to stop anyway\n\n\n---\n\n## Pre-flight Checks\n\n### Purpose\n\nPre-flight checks ensure services can start successfully before attempting to start them.\n\n### Check Types\n\n1. Prerequisites: Binary exists, Docker running, etc.\n2. Conflicts: No conflicting services running\n3. Dependencies: All dependencies available\n\n### Automatic Checks\n\nPre-flight checks run automatically when starting services:\n\n\nprovisioning services start orchestrator\n\n\nCheck Process:\n\n\nRunning pre-flight checks for orchestrator...\n✅ Binary found: /Users/user/.provisioning/bin/provisioning-orchestrator\n✅ No conflicts detected\n✅ All dependencies available\nStarting service: orchestrator\n\n\n### Manual Validation\n\nValidate all services:\n\n\nprovisioning services validate\n\n\nValidate specific service:\n\n\nprovisioning services status orchestrator\n\n\n### Auto-Start\n\nServices with auto_start = true can be started automatically when needed:\n\n\n# Orchestrator auto-starts if needed for server operations\nprovisioning server create\n\n\nOutput:\n\n\nStarting required services...\n✅ Orchestrator started\nCreating server...\n\n\n---\n\n## Troubleshooting\n\n### Service Won't Start\n\nCheck prerequisites:\n\n\nprovisioning services validate\nprovisioning services status <service>\n\n\nCommon issues:\n\n- Binary not found: Check binary_path in config\n- Docker not running: Start Docker daemon\n- Port already in use: Check for conflicting processes\n- Dependencies not running: Start dependencies first\n\n### Service Health Check Failing\n\nView health status:\n\n\nprovisioning services health <service>\n\n\nCheck logs:\n\n\nprovisioning services logs <service> --follow\n\n\nCommon issues:\n\n- Service not fully initialized: Wait longer or increase start_timeout\n- Wrong health check endpoint: Verify endpoint in config\n- Network issues: Check firewall, port bindings\n\n### Dependency Issues\n\nView dependency tree:\n\n\nprovisioning services dependencies <service>\n\n\nCheck dependency status:\n\n\nprovisioning services status <dependency>\n\n\nStart with dependencies:\n\n\nprovisioning platform start <service>\n\n\n### Circular Dependencies\n\nValidate dependency graph:\n\n\n# This is done automatically but you can check manually\nnu -c "use lib_provisioning/services/mod.nu *; validate-dependency-graph"\n\n\n### PID File Stale\n\nIf service reports running but isn't:\n\n\n# Manual cleanup\nrm ~/.provisioning/services/pids/<service>.pid\n\n# Force restart\nprovisioning services restart <service>\n\n\n### Port Conflicts\n\nFind process using port:\n\n\nlsof -i :9090\n\n\nKill conflicting process:\n\n\nkill <PID>\n\n\n### Docker Issues\n\nCheck Docker status:\n\n\ndocker ps\ndocker info\n\n\nView container logs:\n\n\ndocker logs provisioning-<service>\n\n\nRestart Docker daemon:\n\n\n# macOS\nkillall Docker && open /Applications/Docker.app\n\n# Linux\nsystemctl restart docker\n\n\n### Service Logs\n\nView recent logs:\n\n\ntail -f ~/.provisioning/services/logs/<service>.log\n\n\nSearch logs:\n\n\ngrep "ERROR" ~/.provisioning/services/logs/<service>.log\n\n\n---\n\n## Advanced Usage\n\n### Custom Service Registration\n\nAdd custom services by editing provisioning/config/services.toml.\n\n### Integration with Workflows\n\nServices automatically start when required by workflows:\n\n\n# Orchestrator starts automatically if not running\nprovisioning workflow submit my-workflow\n\n\n### CI/CD Integration\n\n\n# GitLab CI\nbefore_script:\n - provisioning platform start orchestrator\n - provisioning services health orchestrator\n\ntest:\n script:\n - provisioning test quick kubernetes\n\n\n### Monitoring Integration\n\nServices can integrate with monitoring systems via health endpoints.\n\n---\n\n## Related Documentation\n\n- Orchestrator README\n- Test Environment Guide\n- Workflow Management\n\n---\n\n## Quick Reference\n\nVersion: 1.0.0\n\n### Platform Commands (Manage All Services)\n\n\n# Start all auto-start services\nprovisioning platform start\n\n# Start specific services with dependencies\nprovisioning platform start control-center mcp-server\n\n# Stop all running services\nprovisioning platform stop\n\n# Stop specific services\nprovisioning platform stop orchestrator\n\n# Restart services\nprovisioning platform restart\n\n# Show platform status\nprovisioning platform status\n\n# Check platform health\nprovisioning platform health\n\n# View service logs\nprovisioning platform logs orchestrator --follow\n\n\n---\n\n### Service Commands (Individual Services)\n\n\n# List all services\nprovisioning services list\n\n# List only running services\nprovisioning services list --running\n\n# Filter by category\nprovisioning services list --category orchestration\n\n# Service status\nprovisioning services status orchestrator\n\n# Start service (with pre-flight checks)\nprovisioning services start orchestrator\n\n# Force start (skip checks)\nprovisioning services start orchestrator --force\n\n# Stop service\nprovisioning services stop orchestrator\n\n# Force stop (ignore dependents)\nprovisioning services stop orchestrator --force\n\n# Restart service\nprovisioning services restart orchestrator\n\n# Check health\nprovisioning services health orchestrator\n\n# View logs\nprovisioning services logs orchestrator --follow --lines 100\n\n# Monitor health continuously\nprovisioning services monitor orchestrator --interval 30\n\n\n---\n\n### Dependency & Validation\n\n\n# View dependency graph\nprovisioning services dependencies\n\n# View specific service dependencies\nprovisioning services dependencies control-center\n\n# Validate all services\nprovisioning services validate\n\n# Check readiness\nprovisioning services readiness\n\n# Check required services for operation\nprovisioning services check server\n\n\n---\n\n### Registered Services\n\n| Service | Port | Type | Auto-Start | Dependencies |\n| --------- | ------ | ------ | ------------ | -------------- |\n| orchestrator | 8080 | Platform | Yes | - |\n| control-center | 8081 | Platform | No | orchestrator |\n| coredns | 5353 | Infrastructure | No | - |\n| gitea | 3000, 222 | Infrastructure | No | - |\n| oci-registry | 5000 | Infrastructure | No | - |\n| mcp-server | 8082 | Platform | No | orchestrator |\n| api-gateway | 8083 | Platform | No | orchestrator, control-center, mcp-server |\n\n---\n\n### Docker Compose\n\n\n# Start all services\ncd provisioning/platform\ndocker-compose up -d\n\n# Start specific services\ndocker-compose up -d orchestrator control-center\n\n# Check status\ndocker-compose ps\n\n# View logs\ndocker-compose logs -f orchestrator\n\n# Stop all services\ndocker-compose down\n\n# Stop and remove volumes\ndocker-compose down -v\n\n\n---\n\n### Service State Directories\n\n\n~/.provisioning/services/\n├── pids/ # Process ID files\n├── state/ # Service state (JSON)\n└── logs/ # Service logs\n\n\n---\n\n### Health Check Endpoints\n\n| Service | Endpoint | Type |\n| --------- | ---------- | ------ |\n| orchestrator | http://localhost:9090/health | HTTP |\n| control-center | http://localhost:9080/health | HTTP |\n| coredns | localhost:5353 | TCP |\n| gitea | http://localhost:3000/api/healthz | HTTP |\n| oci-registry | http://localhost:5000/v2/ | HTTP |\n| mcp-server | http://localhost:8082/health | HTTP |\n| api-gateway | http://localhost:8083/health | HTTP |\n\n---\n\n### Common Workflows\n\n#### Start Platform for Development\n\n\n# Start core services\nprovisioning platform start orchestrator\n\n# Check status\nprovisioning platform status\n\n# Check health\nprovisioning platform health\n\n\n#### Start Full Platform Stack\n\n\n# Use Docker Compose\ncd provisioning/platform\ndocker-compose up -d\n\n# Verify\ndocker-compose ps\nprovisioning platform health\n\n\n#### Debug Service Issues\n\n\n# Check service status\nprovisioning services status <service>\n\n# View logs\nprovisioning services logs <service> --follow\n\n# Check health\nprovisioning services health <service>\n\n# Validate prerequisites\nprovisioning services validate\n\n# Restart service\nprovisioning services restart <service>\n\n\n#### Safe Service Shutdown\n\n\n# Check dependents\nnu -c "use lib_provisioning/services/mod.nu *; can-stop-service orchestrator"\n\n# Stop with dependency check\nprovisioning services stop orchestrator\n\n# Force stop if needed\nprovisioning services stop orchestrator --force\n\n\n---\n\n### Troubleshooting\n\n#### Service Won't Start\n\n\n# 1. Check prerequisites\nprovisioning services validate\n\n# 2. View detailed status\nprovisioning services status <service>\n\n# 3. Check logs\nprovisioning services logs <service>\n\n# 4. Verify binary/image exists\nls ~/.provisioning/bin/<service>\ndocker images | grep <service>\n\n\n#### Health Check Failing\n\n\n# Check endpoint manually\ncurl http://localhost:9090/health\n\n# View health details\nprovisioning services health <service>\n\n# Monitor continuously\nprovisioning services monitor <service> --interval 10\n\n\n#### PID File Stale\n\n\n# Remove stale PID file\nrm ~/.provisioning/services/pids/<service>.pid\n\n# Restart service\nprovisioning services restart <service>\n\n\n#### Port Already in Use\n\n\n# Find process using port\nlsof -i :9090\n\n# Kill process\nkill <PID>\n\n# Restart service\nprovisioning services start <service>\n\n\n---\n\n### Integration with Operations\n\n#### Server Operations\n\n\n# Orchestrator auto-starts if needed\nprovisioning server create\n\n# Manual check\nprovisioning services check server\n\n\n#### Workflow Operations\n\n\n# Orchestrator auto-starts\nprovisioning workflow submit my-workflow\n\n# Check status\nprovisioning services status orchestrator\n\n\n#### Test Operations\n\n\n# Orchestrator required for test environments\nprovisioning test quick kubernetes\n\n# Pre-flight check\nprovisioning services check test-env\n\n\n---\n\n### Advanced Usage\n\n#### Custom Service Startup Order\n\nServices start based on:\n\n1. Dependency order (topological sort)\n2. start_order field (lower = earlier)\n\n#### Auto-Start Configuration\n\nEdit provisioning/config/services.toml:\n\n\n[services.<service>.startup]\nauto_start = true # Enable auto-start\nstart_timeout = 30 # Timeout in seconds\nstart_order = 10 # Startup priority\n\n\n#### Health Check Configuration\n\n\n[services.<service>.health_check]\ntype = "http" # http, tcp, command, file\ninterval = 10 # Seconds between checks\nretries = 3 # Max retry attempts\ntimeout = 5 # Check timeout\n\n[services.<service>.health_check.http]\nendpoint = "http://localhost:9090/health"\nexpected_status = 200\n\n\n---\n\n### Key Files\n\n- Service Registry: provisioning/config/services.toml\n- KCL Schema: provisioning/kcl/services.k\n- Docker Compose: provisioning/platform/docker-compose.yaml\n- User Guide: docs/user/SERVICE_MANAGEMENT_GUIDE.md\n\n---\n\n### Getting Help\n\n\n# View documentation\ncat docs/user/SERVICE_MANAGEMENT_GUIDE.md | less\n\n# Run verification\nnu provisioning/core/nulib/tests/verify_services.nu\n\n# Check readiness\nprovisioning services readiness\n\n\n---\n\nQuick Tip: Use --help flag with any command for detailed usage information.\n\n---\n\nMaintained By: Platform Team\nSupport: GitHub Issues