From 5f857c618e018632adcfa7751a4bbe4967971ea9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jesu=CC=81s=20Pe=CC=81rez?= Date: Tue, 12 May 2026 03:23:48 +0100 Subject: [PATCH] chore: update README and CHANGELOG with ops control plane, audit-mirror, contract-tests --- docs/README.md | 9 - docs/deployment/deployment-guide.md | 757 ---------------------------- docs/deployment/guide.md | 468 ----------------- docs/deployment/known-issues.md | 97 ---- docs/guides/quick-start.md | 282 ----------- 5 files changed, 1613 deletions(-) delete mode 100644 docs/README.md delete mode 100644 docs/deployment/deployment-guide.md delete mode 100644 docs/deployment/guide.md delete mode 100644 docs/deployment/known-issues.md delete mode 100644 docs/guides/quick-start.md diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index e8d363e..0000000 --- a/docs/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# Provisioning Platform Documentation - -User guides, deployment documentation, and operational guides. - -## Structure - -- **deployment/** - Deployment guides, troubleshooting, and build documentation -- **guides/** - Quick start and user guides -- **architecture/** - Architecture and design documentation (if present) diff --git a/docs/deployment/deployment-guide.md b/docs/deployment/deployment-guide.md deleted file mode 100644 index e7f7828..0000000 --- a/docs/deployment/deployment-guide.md +++ /dev/null @@ -1,757 +0,0 @@ -# Provisioning Platform Deployment Guide - -**Version**: 3.0.0 -**Date**: 2025-10-06 -**Deployment Modes**: Solo, Multi-User, CI/CD, Enterprise - ---- - -## Table of Contents - -1. [Overview](#overview) -2. [Prerequisites](#prerequisites) -3. [Deployment Modes](#deployment-modes) -4. [Quick Start](#quick-start) -5. [Configuration](#configuration) -6. [Deployment Methods](#deployment-methods) -7. [Post-Deployment](#post-deployment) -8. [Troubleshooting](#troubleshooting) - ---- - -## Overview - -The Provisioning Platform is a comprehensive infrastructure automation system that can be deployed in four modes: - -- **Solo**: Single-user local development (minimal services) -- **Multi-User**: Team collaboration with source control -- **CI/CD**: Automated deployment pipelines -- **Enterprise**: Full production with monitoring, KMS, and audit logging - -### Architecture Components - -| Component | Solo | Multi-User | CI/CD | Enterprise | -| ----------- | ------ | ------------ | ------- | ------------ | -| Orchestrator | ✓ | ✓ | ✓ | ✓ | -| Control Center | ✓ | ✓ | ✓ | ✓ | -| CoreDNS | ✓ | ✓ | ✓ | ✓ | -| OCI Registry (Zot) | ✓ | ✓ | ✓ | ---- | -| Extension Registry | ✓ | ✓ | ✓ | ✓ | -| Gitea | ---- | ✓ | ✓ | ✓ | -| PostgreSQL | ---- | ✓ | ✓ | ✓ | -| API Server | ---- | - | ✓ | ✓ | -| Harbor | ---- | - | ---- | ✓ | -| Cosmian KMS | ---- | - | ---- | ✓ | -| Prometheus | ---- | - | ---- | ✓ | -| Grafana | ---- | - | ---- | ✓ | -| Loki + Promtail | ---- | - | ---- | ✓ | -| Elasticsearch + Kibana | ---- | - | ---- | ✓ | -| Nginx Reverse Proxy | ---- | - | ---- | ✓ | - ---- - -## Prerequisites - -### Required Software - -1. **Docker** (version 20.10+) - - ```bash - docker --version - # Docker version 20.10.0 or higher - ``` - -2. **Docker Compose** (version 2.0+) - - ```bash - docker-compose --version - # Docker Compose version 2.0.0 or higher - ``` - -3. **Nushell** (version 0.107.1+ for automation scripts) - - ```bash - nu --version - # 0.107.1 or higher - ``` - -### System Requirements - -#### Solo Mode - -- **CPU**: 2 cores -- **Memory**: 4GB RAM -- **Disk**: 20GB free space -- **Network**: Internet connection for pulling images - -#### Multi-User Mode - -- **CPU**: 4 cores -- **Memory**: 8GB RAM -- **Disk**: 50GB free space -- **Network**: Internet connection + internal network - -#### CI/CD Mode - -- **CPU**: 8 cores -- **Memory**: 16GB RAM -- **Disk**: 100GB free space -- **Network**: Internet + dedicated CI/CD network - -#### Enterprise Mode - -- **CPU**: 16 cores -- **Memory**: 32GB RAM -- **Disk**: 500GB free space (SSD recommended) -- **Network**: High-bandwidth, low-latency network - -### Optional Tools - -- **OpenSSL** (for generating secrets) -- **kubectl** (for Kubernetes deployment) -- **Helm** (for Kubernetes package management) - ---- - -## Deployment Modes - -### Solo Mode - -**Use Case**: Local development, testing, personal use - -**Features**: - -- Minimal resource usage -- No authentication required -- SQLite databases -- Local file storage - -**Limitations**: - -- Single user only -- No version control integration -- No audit logging - -### Multi-User Mode - -**Use Case**: Small team collaboration - -**Features**: - -- Multi-user authentication -- Gitea for source control -- PostgreSQL shared database -- User management - -**Limitations**: - -- No automated pipelines -- No advanced monitoring - -### CI/CD Mode - -**Use Case**: Automated deployment pipelines - -**Features**: - -- All Multi-User features -- Provisioning API Server -- Webhook support -- Jenkins/GitLab Runner integration - -**Limitations**: - -- Basic monitoring only - -### Enterprise Mode - -**Use Case**: Production deployments, compliance requirements - -**Features**: - -- All CI/CD features -- Harbor registry (enterprise OCI) -- Cosmian KMS (secret management) -- Full monitoring stack (Prometheus, Grafana) -- Log aggregation (Loki, Elasticsearch) -- Audit logging -- TLS/SSL encryption -- Nginx reverse proxy - ---- - -## Quick Start - -### 1. Clone Repository - -```bash -cd /opt -git clone https://github.com/your-org/project-provisioning.git -cd project-provisioning/provisioning/platform -``` - -### 2. Generate Secrets - -```bash -# Generate .env file with random secrets -./scripts/generate-secrets.nu - -# Or copy and edit manually -cp .env.example .env -nano .env -``` - -### 3. Choose Deployment Mode and Deploy - -#### Solo Mode - -```bash -./scripts/deploy-platform.nu --mode solo -``` - -#### Multi-User Mode - -```bash -# Generate secrets first -./scripts/generate-secrets.nu - -# Deploy -./scripts/deploy-platform.nu --mode multi-user -``` - -#### CI/CD Mode - -```bash -./scripts/deploy-platform.nu --mode cicd --build -``` - -#### Enterprise Mode - -```bash -# Full production deployment -./scripts/deploy-platform.nu --mode enterprise --build --wait 600 -``` - -### 4. Verify Deployment - -```bash -# Check all services -./scripts/health-check.nu - -# View logs -docker-compose logs -f -``` - -### 5. Access Services - -- **Orchestrator**: -- **Control Center**: -- **OCI Registry**: -- **Gitea** (Multi-User+): -- **Grafana** (Enterprise): - ---- - -## Configuration - -### Environment Variables - -The `.env` file controls all deployment settings. Key variables: - -#### Platform Configuration - -```toml -PROVISIONING_MODE=solo # solo, multi-user, cicd, enterprise -PLATFORM_ENVIRONMENT=development # development, staging, production -``` - -#### Service Ports - -```bash -ORCHESTRATOR_PORT=8080 -CONTROL_CENTER_PORT=8081 -GITEA_HTTP_PORT=3000 -OCI_REGISTRY_PORT=5000 -``` - -#### Security Settings - -```toml -# Generate with: openssl rand -base64 32 -CONTROL_CENTER_JWT_SECRET= -API_SERVER_JWT_SECRET= -POSTGRES_PASSWORD= -``` - -#### Resource Limits - -```bash -ORCHESTRATOR_CPU_LIMIT=2000m -ORCHESTRATOR_MEMORY_LIMIT=2048M -``` - -### Configuration Files - -#### Docker Compose - -- **Main**: `docker-compose.yaml` (base services) -- **Solo**: `infrastructure/docker/docker-compose.solo.yaml` -- **Multi-User**: `infrastructure/docker/docker-compose.multi-user.yaml` -- **CI/CD**: `infrastructure/docker/docker-compose.cicd.yaml` -- **Enterprise**: `infrastructure/docker/docker-compose.enterprise.yaml` - -#### Service Configurations - -- **Orchestrator**: `orchestrator/config.defaults.toml` -- **Control Center**: `control-center/config.defaults.toml` -- **CoreDNS**: `config/coredns/Corefile` -- **OCI Registry**: `infrastructure/oci-registry/config.json` -- **Nginx**: `infrastructure/nginx/nginx.conf` -- **Prometheus**: `infrastructure/monitoring/prometheus/prometheus.yml` - ---- - -## Deployment Methods - -### Method 1: Docker Compose (Recommended) - -#### Deploy - -```bash -# Solo mode -docker-compose -f docker-compose.yaml - -f infrastructure/docker/docker-compose.solo.yaml - up -d - -# Multi-user mode -docker-compose -f docker-compose.yaml - -f infrastructure/docker/docker-compose.multi-user.yaml - up -d - -# CI/CD mode -docker-compose -f docker-compose.yaml - -f infrastructure/docker/docker-compose.multi-user.yaml - -f infrastructure/docker/docker-compose.cicd.yaml - up -d - -# Enterprise mode -docker-compose -f docker-compose.yaml - -f infrastructure/docker/docker-compose.multi-user.yaml - -f infrastructure/docker/docker-compose.cicd.yaml - -f infrastructure/docker/docker-compose.enterprise.yaml - up -d -``` - -#### Manage Services - -```bash -# View logs -docker-compose logs -f [service-name] - -# Restart service -docker-compose restart orchestrator - -# Stop all services -docker-compose down - -# Stop and remove volumes (WARNING: data loss) -docker-compose down --volumes -``` - -### Method 2: Systemd (Linux Production) - -#### Install Services - -```bash -cd systemd -sudo ./install-services.sh -``` - -#### Manage via systemd - -```bash -# Start platform -sudo systemctl start provisioning-platform - -# Enable auto-start on boot -sudo systemctl enable provisioning-platform - -# Check status -sudo systemctl status provisioning-platform - -# View logs -sudo journalctl -u provisioning-platform -f - -# Restart -sudo systemctl restart provisioning-platform - -# Stop -sudo systemctl stop provisioning-platform -``` - -### Method 3: Kubernetes - -See [KUBERNETES_DEPLOYMENT.md](./KUBERNETES_DEPLOYMENT.md) for detailed instructions. - -#### Quick Deploy - -```bash -# Create namespace -kubectl apply -f k8s/base/namespace.yaml - -# Deploy services -kubectl apply -f k8s/deployments/ -kubectl apply -f k8s/services/ -kubectl apply -f k8s/ingress/ - -# Check status -kubectl get pods -n provisioning -``` - -### Method 4: Automation Script (Nushell) - -```nushell -# Deploy with options -./scripts/deploy-platform.nu --mode enterprise - --build - --wait 300 - -# Health check -./scripts/health-check.nu - -# Dry run (show what would be deployed) -./scripts/deploy-platform.nu --mode enterprise --dry-run -``` - ---- - -## Post-Deployment - -### 1. Verify Services - -```bash -# Quick health check -./scripts/health-check.nu - -# Detailed Docker status -docker-compose ps - -# Check individual service -curl http://localhost:9090/health -``` - -### 2. Initial Configuration - -#### Create Admin User (Multi-User+) - -Access Gitea at and complete setup wizard. - -#### Configure DNS (Optional) - -Add to `/etc/hosts` or configure local DNS: - -```toml -127.0.0.1 provisioning.local -127.0.0.1 gitea.provisioning.local -127.0.0.1 grafana.provisioning.local -``` - -#### Configure Monitoring (Enterprise) - -1. Access Grafana: -2. Login with credentials from `.env`: - - Username: `admin` - - Password: `${GRAFANA_ADMIN_PASSWORD}` -3. Dashboards are auto-provisioned from `infrastructure/monitoring/grafana/dashboards/` - -### 3. Load Extensions - -```bash -# List available extensions -curl http://localhost:8082/api/v1/extensions - -# Upload extension (example) -curl -X POST http://localhost:8082/api/v1/extensions/upload - -F "file=@my-extension.tar.gz" -``` - -### 4. Test Workflows - -```bash -# Create test server (via orchestrator API) -curl -X POST http://localhost:9090/workflows/servers/create - -H "Content-Type: application/json" - -d '{"name": "test-server", "plan": "1xCPU-2GB"}' - -# Check workflow status -curl http://localhost:9090/tasks/ -``` - ---- - -## Troubleshooting - -### Common Issues - -#### Services Not Starting - -**Symptom**: `docker-compose up` fails or services crash - -**Solutions**: - -1. Check Docker daemon: - - ```bash - systemctl status docker - ``` - -1. Check logs: - - ```bash - docker-compose logs orchestrator - ``` - -2. Check resource limits: - - ```bash - docker stats - ``` - -3. Increase Docker resources in Docker Desktop settings - -#### Port Conflicts - -**Symptom**: `Error: port is already allocated` - -**Solutions**: - -1. Find conflicting process: - - ```bash - lsof -i :9090 - ``` - -2. Change port in `.env`: - - ```bash - ORCHESTRATOR_PORT=9080 - ``` - -3. Restart deployment: - - ```bash - docker-compose down - docker-compose up -d - ``` - -#### Health Checks Failing - -**Symptom**: Health check script reports unhealthy services - -**Solutions**: - -1. Check service logs: - - ```bash - docker-compose logs -f - ``` - -2. Verify network connectivity: - - ```bash - docker network inspect provisioning-net - ``` - -3. Check firewall rules: - - ```bash - sudo ufw status - ``` - -4. Wait longer for services to start: - - ```bash - ./scripts/deploy-platform.nu --wait 600 - ``` - -#### Database Connection Errors - -**Symptom**: PostgreSQL connection refused - -**Solutions**: - -1. Check PostgreSQL health: - - ```bash - docker exec provisioning-postgres pg_isready - ``` - -2. Verify credentials in `.env`: - - ```bash - grep POSTGRES_ .env - ``` - -3. Check PostgreSQL logs: - - ```bash - docker-compose logs postgres - ``` - -4. Recreate database: - - ```bash - docker-compose down - docker volume rm provisioning_postgres-data - docker-compose up -d - ``` - -#### Out of Disk Space - -**Symptom**: No space left on device - -**Solutions**: - -1. Clean Docker volumes: - - ```bash - docker volume prune - ``` - -2. Clean Docker images: - - ```bash - docker image prune -a - ``` - -3. Check volume sizes: - - ```bash - docker system df -v - ``` - -### Getting Help - -- **Logs**: Always check logs first: `docker-compose logs -f` -- **Health**: Run health check: `./scripts/health-check.nu --json` -- **Documentation**: See `docs/` directory -- **Issues**: File bug reports at GitHub repository - ---- - -## Security Best Practices - -### 1. Secret Management - -- **Never commit** `.env` files to version control -- Use `./scripts/generate-secrets.nu` to generate strong secrets -- Rotate secrets regularly -- Use KMS in enterprise mode - -### 2. Network Security - -- Use TLS/SSL in production (enterprise mode) -- Configure firewall rules: - - ```bash - sudo ufw allow 80/tcp - sudo ufw allow 443/tcp - sudo ufw enable - ``` - -- Use private networks for backend services - -### 3. Access Control - -- Enable authentication in multi-user mode -- Use strong passwords (16+ characters) -- Configure API keys for CI/CD access -- Enable audit logging in enterprise mode - -### 4. Regular Updates - -```bash -# Pull latest images -docker-compose pull - -# Rebuild with updates -./scripts/deploy-platform.nu --pull --build -``` - ---- - -## Backup and Recovery - -### Backup - -```bash -# Backup volumes -docker run --rm -v provisioning_orchestrator-data:/data - -v $(pwd)/backup:/backup - alpine tar czf /backup/orchestrator-data.tar.gz -C /data . - -# Backup PostgreSQL -docker exec provisioning-postgres pg_dumpall -U provisioning > backup/postgres-backup.sql -``` - -### Restore - -```bash -# Restore volume -docker run --rm -v provisioning_orchestrator-data:/data - -v $(pwd)/backup:/backup - alpine tar xzf /backup/orchestrator-data.tar.gz -C /data - -# Restore PostgreSQL -docker exec -i provisioning-postgres psql -U provisioning < backup/postgres-backup.sql -``` - ---- - -## Maintenance - -### Updates - -```bash -# Pull latest images -docker-compose pull - -# Recreate containers -docker-compose up -d --force-recreate - -# Remove old images -docker image prune -``` - -### Monitoring - -- **Prometheus**: -- **Grafana**: -- **Logs**: `docker-compose logs -f` - -### Health Checks - -```bash -# Automated health check -./scripts/health-check.nu - -# Manual checks -curl http://localhost:9090/health -curl http://localhost:8081/health -``` - ---- - -## Next Steps - -- [Production Deployment Guide](./PRODUCTION_DEPLOYMENT.md) -- [Kubernetes Deployment Guide](./KUBERNETES_DEPLOYMENT.md) -- [Docker Compose Reference](./DOCKER_COMPOSE_REFERENCE.md) -- [Monitoring Setup](./MONITORING_SETUP.md) -- [Security Hardening](./SECURITY_HARDENING.md) - ---- - -**Documentation Version**: 1.0.0 -**Last Updated**: 2025-10-06 -**Maintained By**: Platform Team diff --git a/docs/deployment/guide.md b/docs/deployment/guide.md deleted file mode 100644 index 2f8bff9..0000000 --- a/docs/deployment/guide.md +++ /dev/null @@ -1,468 +0,0 @@ -# Provisioning Platform - Deployment Guide - -**Last Updated**: 2025-10-07 -**Platform**: macOS (Apple Silicon / Intel) + OrbStack/Docker - ---- - -## ✅ Fixed: Docker Builds - -Docker builds have been **fixed** to properly handle the Rust workspace structure. Both deployment methods (Native and Docker) are now fully -supported. - -**Note**: Docker builds use Rust nightly to support edition2024 (required by async-graphql 7.x from surrealdb). -RocksDB has been replaced with SurrealDB in-memory backend (kv-mem) to simplify Docker builds (no libclang requirement). - ---- - -## 📦 Quick Start - -### Prerequisites - -**For Native Deployment:** - -- Rust 1.75+: `brew install rust` -- Nushell 0.107+: `brew install nushell` - -**For Docker Deployment:** - -- OrbStack (recommended): -- Or Docker Desktop: `brew install --cask docker` - ---- - -## 🚀 Deployment Methods - -### Method 1: Native Execution (Recommended for Development) - -**Fastest startup, easiest debugging, direct access to logs** - -```bash -cd provisioning/platform/scripts - -# 1. Build all services -nu run-native.nu build - -# 2. Start all services in background -nu run-native.nu start-all --background - -# 3. Check status -nu run-native.nu status - -# 4. View logs -nu run-native.nu logs orchestrator --follow - -# 5. Stop all -nu run-native.nu stop-all -``` - -**Services will run on:** - -- Orchestrator: -- Control Center: - -**Data stored in:** - -- `~/.provisioning-platform/data/` -- `~/.provisioning-platform/logs/` - ---- - -### Method 2: Docker Execution (Recommended for Production-like Testing) - -**Isolated environments, easy cleanup, supports all deployment modes** - -```bash -cd provisioning/platform/scripts - -# 1. Build Docker images (Solo mode) -nu run-docker.nu build solo - -# 2. Start services in background -nu run-docker.nu start solo --detach - -# 3. Check status -nu run-docker.nu status - -# 4. View logs -nu run-docker.nu logs orchestrator --follow - -# 5. Stop all -nu run-docker.nu stop -``` - -**Deployment Modes:** - -- `solo` - 2 CPU / 4GB RAM (dev/test) -- `multiuser` - 4 CPU / 8GB RAM (team) -- `cicd` - 8 CPU / 16GB RAM (automation) -- `enterprise` - 16 CPU / 32GB RAM (production + KMS) - ---- - -## 📋 Complete Command Reference - -### Native Execution (`run-native.nu`) - -| Command | Description | -| --------- | ------------- | -| `build` | Build all services | -| `start ` | Start orchestrator or control_center | -| `start-all` | Start all services | -| `stop ` | Stop a specific service | -| `stop-all` | Stop all services | -| `status` | Show service status | -| `logs ` | Show logs (add `--follow`) | -| `health` | Check service health | - -**Examples:** - -```nushell -nu run-native.nu build -nu run-native.nu start orchestrator --background -nu run-native.nu start control_center --background -nu run-native.nu logs orchestrator --follow -nu run-native.nu health -nu run-native.nu stop-all -``` - ---- - -### Docker Execution (`run-docker.nu`) - -| Command | Description | -| --------- | ------------- | -| `build [mode]` | Build Docker images | -| `start [mode]` | Start services (add `--detach`) | -| `stop` | Stop all services (add `--volumes` to delete data) | -| `restart [mode]` | Restart services | -| `status` | Show container status | -| `logs ` | Show logs (add `--follow`) | -| `exec ` | Execute command in container | -| `stats` | Show resource usage | -| `health` | Check service health | -| `config [mode]` | Show docker-compose config | -| `clean` | Remove containers (add `--all` for images too) | - -**Examples:** - -```bash -# Solo mode (fastest) -nu run-docker.nu build solo -nu run-docker.nu start solo --detach - -# Enterprise mode (with KMS) -nu run-docker.nu build enterprise -nu run-docker.nu start enterprise --detach - -# Operations -nu run-docker.nu status -nu run-docker.nu logs control-center --follow -nu run-docker.nu exec orchestrator bash -nu run-docker.nu stats -nu run-docker.nu stop -``` - ---- - -## 🗄️ Database Information - -### Control-Center Database - -**Type**: SurrealDB with in-memory backend (kv-mem) -**Location**: In-memory (data persisted during container/process lifetime) -**Production Alternative**: SurrealDB with remote WebSocket connection for persistent storage - -**No separate database server required** - SurrealDB in-memory backend is embedded in the control-center process. - -### Orchestrator Storage - -**Type**: Filesystem queue (default) -**Location**: - -- Native: `~/.provisioning-platform/data/orchestrator/queue.rkvs` -- Docker: `/data/queue.rkvs` (inside container) - -**Production Option**: Switch to SurrealDB via config for distributed deployments. - ---- - -## ⚙️ Configuration Loading - -Services load configuration in this order (priority: low → high): - -1. **System Defaults** - `provisioning/config/config.defaults.toml` -2. **Service Defaults** - `provisioning/platform/{service}/config.defaults.toml` -3. **Workspace Config** - `workspace/{name}/config/provisioning.yaml` -4. **User Config** - `~/Library/Application Support/provisioning/user_config.yaml` -5. **Environment Variables** - `CONTROL_CENTER_*`, `ORCHESTRATOR_*` -6. **Runtime Overrides** - `--config` flag - -**See full documentation**: `docs/architecture/DATABASE_AND_CONFIG_ARCHITECTURE.md` - ---- - -## 🐛 Troubleshooting - -### Native Deployment Issues - -**Build fails:** - -```bash -# Clean and rebuild -cd provisioning/platform -cargo clean -cargo build --release -``` - -**Port already in use:** - -```bash -# Check what's using the port -lsof -i :8080 -lsof -i :8081 - -# Kill the process or use different ports via environment variables -export ORCHESTRATOR_SERVER_PORT=8090 -export CONTROL_CENTER_SERVER_PORT=8091 -``` - -**Service won't start:** - -```bash -# Check logs for errors -nu run-native.nu logs orchestrator - -# Run in foreground to see output -nu run-native.nu start orchestrator -``` - ---- - -### Docker Deployment Issues - -**Build fails with workspace errors:** - -- **Fixed!** Dockerfiles now properly handle workspace structure -- If still failing: `nu run-docker.nu build solo --no-cache` - -**Containers won't start:** - -```bash -# Check container logs -nu run-docker.nu logs orchestrator - -# Check Docker daemon -docker ps -docker info - -# Restart Docker/OrbStack -``` - -**Port conflicts:** - -```bash -# Check what's using ports -lsof -i :8080 -lsof -i :8081 - -# Stop conflicting services or modify docker-compose.yaml ports -``` - -**Out of resources:** - -```bash -# Check current usage -nu run-docker.nu stats - -# Clean up unused containers/images -docker system prune -a - -# Or use the script -nu run-docker.nu clean --all -``` - ---- - -## 🔐 KMS Integration (Enterprise Mode) - -Enterprise mode includes Cosmian KMS for production-grade secret management. - -**Start with KMS:** - -```nushell -nu run-docker.nu build enterprise -nu run-docker.nu start enterprise --detach -``` - -**Access KMS:** - -- KMS API: -- KMS Health: - -**KMS Features:** - -- SSL certificate lifecycle management -- SSH private key rotation -- Cloud credential auto-refresh -- Audit trails -- Automatic key rotation - -**See full KMS documentation**: `provisioning/platform/control-center/src/kms/README.md` - ---- - -## 📊 Monitoring - -### Health Checks - -**Native:** - -```nushell -nu run-native.nu health -``` - -**Docker:** - -```nushell -nu run-docker.nu health -``` - -**Manual:** - -```bash -curl http://localhost:8080/health # Orchestrator -curl http://localhost:8081/health # Control Center -curl http://localhost:9998/health # KMS (enterprise only) -``` - -### Resource Usage - -**Docker:** - -```nushell -nu run-docker.nu stats -``` - -**Native:** - -```bash -ps aux | grep -E "provisioning-orchestrator|control-center" -top -pid -``` - ---- - -## 🧪 Testing Both Methods - -### Test Native Deployment - -```bash -cd provisioning/platform/scripts - -# 1. Build -nu run-native.nu build - -# 2. Start services -nu run-native.nu start-all --background - -# 3. Verify -nu run-native.nu status -nu run-native.nu health - -# 4. Test API -curl http://localhost:8080/health -curl http://localhost:8081/health - -# 5. Clean up -nu run-native.nu stop-all -``` - -### Test Docker Deployment - -```bash -cd provisioning/platform/scripts - -# 1. Build -nu run-docker.nu build solo - -# 2. Start services -nu run-docker.nu start solo --detach - -# 3. Verify -nu run-docker.nu status -nu run-docker.nu health - -# 4. Test API -curl http://localhost:8080/health -curl http://localhost:8081/health - -# 5. Clean up -nu run-docker.nu stop --volumes -``` - ---- - -## 🎯 Best Practices - -### Development Workflow - -1. **Use Native for Active Development** - - Faster iteration (no Docker rebuild) - - Direct log access - - Easy debugging with IDE - -2. **Use Docker for Integration Testing** - - Test deployment configurations - - Verify Docker builds - - Simulate production environment - -### Production Deployment - -1. **Use Docker/Kubernetes** - - Isolated environments - - Easy scaling - - Standard deployment - -2. **Use Enterprise Mode** - - KMS for secret management - - Full monitoring stack - - High availability - ---- - -## 📚 Related Documentation - -- **Architecture Overview**: [System Overview](../../docs/src/architecture/system-overview.md) -- **KMS Integration**: `provisioning/platform/control-center/src/kms/README.md` -- **Configuration System**: [Configuration Reference](../../docs/src/infrastructure/configuration-system.md) -- **Workspace Management**: [Workspace Management](../../docs/src/features/workspace-management.md) -- **Orchestrator API**: [Orchestrator Endpoints](../../docs/src/api-reference/orchestrator-endpoints.md) - ---- - -## ✅ Summary - -### Native Execution - -- ✅ **Fixed**: Workspace builds work correctly -- ✅ **Fast**: No container overhead -- ✅ **Simple**: Direct binary execution -- ✅ **Best for**: Development, debugging - -### Docker Execution - -- ✅ **Fixed**: Dockerfiles now workspace-aware -- ✅ **Isolated**: Clean environments -- ✅ **Flexible**: Multiple deployment modes -- ✅ **Best for**: Testing, production-like deployments - -**Both methods fully supported and tested!** - ---- - -**Quick Links:** - -- Native Script: `provisioning/platform/scripts/run-native.nu` -- Docker Script: `provisioning/platform/scripts/run-docker.nu` -- Docker Files: `provisioning/platform/docker-compose.yaml` + mode-specific overrides diff --git a/docs/deployment/known-issues.md b/docs/deployment/known-issues.md deleted file mode 100644 index b9d49cf..0000000 --- a/docs/deployment/known-issues.md +++ /dev/null @@ -1,97 +0,0 @@ -# Known Issues - Provisioning Platform - -## Control-Center Requires Rust Nightly (Edition 2024 Dependency) - -**Status**: Resolved (using nightly) -**Severity**: Low -**Affects**: Docker deployment only -**Date Reported**: 2025-10-07 -**Date Resolved**: 2025-10-07 - -### Issue - -Control-center Docker builds fail with the following error: - -```bash -feature 'edition2024' is required -this Cargo does not support nightly features, but if you -switch to nightly channel you can add -`cargo-features = ["edition2024"]` to enable this feature -``` - -### Root Cause - -Dependency chain: - -```bash -control-center → surrealdb 2.3.10 → surrealdb-core 2.3.10 → async-graphql 7.0.17 -``` - -The `async-graphql-value` crate v7.0.17 requires Rust edition 2024, which is not yet stable in Rust 1.82. -Edition 2024 is currently only available in Rust nightly builds. - -### Resolution - -**Updated Dockerfiles to use Rust nightly** (2025-10-07): - -Both `orchestrator/Dockerfile` and `control-center/Dockerfile` now use: - -```bash -FROM rustlang/rust:nightly-bookworm AS builder -``` - -This provides edition2024 support required by the surrealdb dependency chain. - -### Production Considerations - -**Rust Nightly Stability**: - -- Nightly builds are generally stable for compilation -- The compiled binaries are production-ready -- Runtime behavior is not affected by nightly vs stable compilation -- Consider pinning to a specific nightly date for reproducible builds - -**Alternative**: Use native deployment with stable Rust if nightly is a concern: - -```rust -cd provisioning/platform/scripts -nu run-native.nu build -nu run-native.nu start-all --background -``` - -### Timeline - -- **Rust 1.85** (estimated Feb 2025): Expected to stabilize edition 2024 -- **SurrealDB 3.x**: May drop async-graphql dependency - -### Tracking - -- Rust Edition 2024 RFC: -- SurrealDB Issue: -- async-graphql Issue: - -### Related Files - -- `provisioning/platform/control-center/Dockerfile` -- `provisioning/platform/Cargo.toml` (workspace dependencies) -- `provisioning/platform/control-center/Cargo.toml` - ---- - -## RocksDB Build Takes Long Time - -**Status**: Known Limitation -**Severity**: Low -**Affects**: All builds - -### Issue - -RocksDB compilation takes 30-60 seconds during builds. - -### Workaround - -Use cached Docker layers or native builds with incremental compilation. - ---- - -**Last Updated**: 2025-10-07 diff --git a/docs/guides/quick-start.md b/docs/guides/quick-start.md deleted file mode 100644 index 73902f6..0000000 --- a/docs/guides/quick-start.md +++ /dev/null @@ -1,282 +0,0 @@ -# Provisioning Platform - Quick Start - -Fast deployment guide for all modes. - ---- - -## Prerequisites - -```bash -# Verify Docker is installed and running -docker --version # 20.10+ -docker-compose --version # 2.0+ -docker ps # Should work without errors -``` - ---- - -## 1. Solo Mode (Local Development) - -**Services**: Orchestrator, Control Center, CoreDNS, OCI Registry, Extension Registry - -**Resources**: 2 CPU cores, 4GB RAM, 20GB disk - -```bash -cd /Users/Akasha/project-provisioning/provisioning/platform - -# Generate secrets -./scripts/generate-secrets.nu - -# Deploy -./scripts/deploy-platform.nu --mode solo - -# Verify -./scripts/health-check.nu - -# Access -open http://localhost:8080 # Orchestrator -open http://localhost:8081 # Control Center -``` - -**Stop**: - -```bash -docker-compose down -``` - ---- - -## 2. Multi-User Mode (Team Collaboration) - -**Services**: Solo + Gitea, PostgreSQL - -**Resources**: 4 CPU cores, 8GB RAM, 50GB disk - -```bash -cd /Users/Akasha/project-provisioning/provisioning/platform - -# Generate secrets -./scripts/generate-secrets.nu - -# Deploy -./scripts/deploy-platform.nu --mode multi-user - -# Verify -./scripts/health-check.nu - -# Access -open http://localhost:3000 # Gitea -open http://localhost:8081 # Control Center -``` - -**Configure Gitea**: - -1. Visit -2. Complete initial setup wizard -3. Create admin account - ---- - -## 3. CI/CD Mode (Automated Pipelines) - -**Services**: Multi-User + API Server, Jenkins (optional), GitLab Runner (optional) - -**Resources**: 8 CPU cores, 16GB RAM, 100GB disk - -```bash -cd /Users/Akasha/project-provisioning/provisioning/platform - -# Generate secrets -./scripts/generate-secrets.nu - -# Deploy -./scripts/deploy-platform.nu --mode cicd --build - -# Verify -./scripts/health-check.nu - -# Access -open http://localhost:8083 # API Server -``` - ---- - -## 4. Enterprise Mode (Production) - -**Services**: Full stack (15+ services) - -**Resources**: 16 CPU cores, 32GB RAM, 500GB disk - -```bash -cd /Users/Akasha/project-provisioning/provisioning/platform - -# Generate production secrets -./scripts/generate-secrets.nu --output .env.production - -# Review and customize -nano .env.production - -# Deploy with build -./scripts/deploy-platform.nu --mode enterprise - --env-file .env.production - --build - --wait 600 - -# Verify -./scripts/health-check.nu - -# Access -open http://localhost:3001 # Grafana (admin / password from .env) -open http://localhost:9090 # Prometheus -open http://localhost:5601 # Kibana -``` - ---- - -## Common Commands - -### View Logs - -```bash -docker-compose logs -f -docker-compose logs -f orchestrator -docker-compose logs --tail=100 orchestrator -``` - -### Restart Services - -```bash -docker-compose restart orchestrator -docker-compose restart -``` - -### Update Platform - -```bash -docker-compose pull -./scripts/deploy-platform.nu --mode --pull -``` - -### Stop Platform - -```bash -docker-compose down -``` - -### Clean Everything (WARNING: data loss) - -```bash -docker-compose down --volumes -``` - ---- - -## Systemd (Linux Production) - -```bash -# Install services -cd systemd -sudo ./install-services.sh - -# Enable and start -sudo systemctl enable --now provisioning-platform - -# Check status -sudo systemctl status provisioning-platform - -# View logs -sudo journalctl -u provisioning-platform -f - -# Restart -sudo systemctl restart provisioning-platform - -# Stop -sudo systemctl stop provisioning-platform -``` - ---- - -## Troubleshooting - -### Services not starting - -```bash -# Check Docker -systemctl status docker - -# Check logs -docker-compose logs orchestrator - -# Check resources -docker stats -``` - -### Port conflicts - -```bash -# Find what's using port -lsof -i :8080 - -# Change port in .env -nano .env -# Set ORCHESTRATOR_PORT=9080 - -# Restart -docker-compose down && docker-compose up -d -``` - -### Health checks failing - -```bash -# Check individual service -curl http://localhost:8080/health - -# Wait longer -./scripts/deploy-platform.nu --wait 600 - -# Check networks -docker network inspect provisioning-net -``` - ---- - -## Access URLs - -### Solo Mode - -- Orchestrator: -- Control Center: -- OCI Registry: - -### Multi-User Mode - -- Gitea: -- PostgreSQL: localhost:5432 - -### CI/CD Mode - -- API Server: - -### Enterprise Mode - -- Prometheus: -- Grafana: -- Kibana: -- Nginx: - ---- - -## Next Steps - -- **Full Guide**: See `docs/deployment/deployment-guide.md` -- **Configuration**: Edit `.env` file for customization -- **Monitoring**: Access Grafana dashboards (enterprise mode) -- **API**: Use API Server for automation (CI/CD mode) - ---- - -**Need Help?** - -- Health Check: `./scripts/health-check.nu` -- Logs: `docker-compose logs -f` -- Documentation: `docs/deployment/`