chore: update README and CHANGELOG with ops control plane, audit-mirror, contract-tests
This commit is contained in:
Jesús Pérez
parent
a747d8a201
commit
5f857c618e
GPG key ID:
9F243E355E0BC939
5 changed files with 0 additions and 1613 deletions
|
|
@ -1,9 +0,0 @@
|
||||||
# Provisioning Platform Documentation
|
|
||||||
|
|
||||||
User guides, deployment documentation, and operational guides.
|
|
||||||
|
|
||||||
## Structure
|
|
||||||
|
|
||||||
- **deployment/** - Deployment guides, troubleshooting, and build documentation
|
|
||||||
- **guides/** - Quick start and user guides
|
|
||||||
- **architecture/** - Architecture and design documentation (if present)
|
|
||||||
|
|
@ -1,757 +0,0 @@
|
||||||
# Provisioning Platform Deployment Guide
|
|
||||||
|
|
||||||
**Version**: 3.0.0
|
|
||||||
**Date**: 2025-10-06
|
|
||||||
**Deployment Modes**: Solo, Multi-User, CI/CD, Enterprise
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Table of Contents
|
|
||||||
|
|
||||||
1. [Overview](#overview)
|
|
||||||
2. [Prerequisites](#prerequisites)
|
|
||||||
3. [Deployment Modes](#deployment-modes)
|
|
||||||
4. [Quick Start](#quick-start)
|
|
||||||
5. [Configuration](#configuration)
|
|
||||||
6. [Deployment Methods](#deployment-methods)
|
|
||||||
7. [Post-Deployment](#post-deployment)
|
|
||||||
8. [Troubleshooting](#troubleshooting)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The Provisioning Platform is a comprehensive infrastructure automation system that can be deployed in four modes:
|
|
||||||
|
|
||||||
- **Solo**: Single-user local development (minimal services)
|
|
||||||
- **Multi-User**: Team collaboration with source control
|
|
||||||
- **CI/CD**: Automated deployment pipelines
|
|
||||||
- **Enterprise**: Full production with monitoring, KMS, and audit logging
|
|
||||||
|
|
||||||
### Architecture Components
|
|
||||||
|
|
||||||
| Component | Solo | Multi-User | CI/CD | Enterprise |
|
|
||||||
| ----------- | ------ | ------------ | ------- | ------------ |
|
|
||||||
| Orchestrator | ✓ | ✓ | ✓ | ✓ |
|
|
||||||
| Control Center | ✓ | ✓ | ✓ | ✓ |
|
|
||||||
| CoreDNS | ✓ | ✓ | ✓ | ✓ |
|
|
||||||
| OCI Registry (Zot) | ✓ | ✓ | ✓ | ---- |
|
|
||||||
| Extension Registry | ✓ | ✓ | ✓ | ✓ |
|
|
||||||
| Gitea | ---- | ✓ | ✓ | ✓ |
|
|
||||||
| PostgreSQL | ---- | ✓ | ✓ | ✓ |
|
|
||||||
| API Server | ---- | - | ✓ | ✓ |
|
|
||||||
| Harbor | ---- | - | ---- | ✓ |
|
|
||||||
| Cosmian KMS | ---- | - | ---- | ✓ |
|
|
||||||
| Prometheus | ---- | - | ---- | ✓ |
|
|
||||||
| Grafana | ---- | - | ---- | ✓ |
|
|
||||||
| Loki + Promtail | ---- | - | ---- | ✓ |
|
|
||||||
| Elasticsearch + Kibana | ---- | - | ---- | ✓ |
|
|
||||||
| Nginx Reverse Proxy | ---- | - | ---- | ✓ |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
### Required Software
|
|
||||||
|
|
||||||
1. **Docker** (version 20.10+)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker --version
|
|
||||||
# Docker version 20.10.0 or higher
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Docker Compose** (version 2.0+)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose --version
|
|
||||||
# Docker Compose version 2.0.0 or higher
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Nushell** (version 0.107.1+ for automation scripts)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nu --version
|
|
||||||
# 0.107.1 or higher
|
|
||||||
```
|
|
||||||
|
|
||||||
### System Requirements
|
|
||||||
|
|
||||||
#### Solo Mode
|
|
||||||
|
|
||||||
- **CPU**: 2 cores
|
|
||||||
- **Memory**: 4GB RAM
|
|
||||||
- **Disk**: 20GB free space
|
|
||||||
- **Network**: Internet connection for pulling images
|
|
||||||
|
|
||||||
#### Multi-User Mode
|
|
||||||
|
|
||||||
- **CPU**: 4 cores
|
|
||||||
- **Memory**: 8GB RAM
|
|
||||||
- **Disk**: 50GB free space
|
|
||||||
- **Network**: Internet connection + internal network
|
|
||||||
|
|
||||||
#### CI/CD Mode
|
|
||||||
|
|
||||||
- **CPU**: 8 cores
|
|
||||||
- **Memory**: 16GB RAM
|
|
||||||
- **Disk**: 100GB free space
|
|
||||||
- **Network**: Internet + dedicated CI/CD network
|
|
||||||
|
|
||||||
#### Enterprise Mode
|
|
||||||
|
|
||||||
- **CPU**: 16 cores
|
|
||||||
- **Memory**: 32GB RAM
|
|
||||||
- **Disk**: 500GB free space (SSD recommended)
|
|
||||||
- **Network**: High-bandwidth, low-latency network
|
|
||||||
|
|
||||||
### Optional Tools
|
|
||||||
|
|
||||||
- **OpenSSL** (for generating secrets)
|
|
||||||
- **kubectl** (for Kubernetes deployment)
|
|
||||||
- **Helm** (for Kubernetes package management)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Deployment Modes
|
|
||||||
|
|
||||||
### Solo Mode
|
|
||||||
|
|
||||||
**Use Case**: Local development, testing, personal use
|
|
||||||
|
|
||||||
**Features**:
|
|
||||||
|
|
||||||
- Minimal resource usage
|
|
||||||
- No authentication required
|
|
||||||
- SQLite databases
|
|
||||||
- Local file storage
|
|
||||||
|
|
||||||
**Limitations**:
|
|
||||||
|
|
||||||
- Single user only
|
|
||||||
- No version control integration
|
|
||||||
- No audit logging
|
|
||||||
|
|
||||||
### Multi-User Mode
|
|
||||||
|
|
||||||
**Use Case**: Small team collaboration
|
|
||||||
|
|
||||||
**Features**:
|
|
||||||
|
|
||||||
- Multi-user authentication
|
|
||||||
- Gitea for source control
|
|
||||||
- PostgreSQL shared database
|
|
||||||
- User management
|
|
||||||
|
|
||||||
**Limitations**:
|
|
||||||
|
|
||||||
- No automated pipelines
|
|
||||||
- No advanced monitoring
|
|
||||||
|
|
||||||
### CI/CD Mode
|
|
||||||
|
|
||||||
**Use Case**: Automated deployment pipelines
|
|
||||||
|
|
||||||
**Features**:
|
|
||||||
|
|
||||||
- All Multi-User features
|
|
||||||
- Provisioning API Server
|
|
||||||
- Webhook support
|
|
||||||
- Jenkins/GitLab Runner integration
|
|
||||||
|
|
||||||
**Limitations**:
|
|
||||||
|
|
||||||
- Basic monitoring only
|
|
||||||
|
|
||||||
### Enterprise Mode
|
|
||||||
|
|
||||||
**Use Case**: Production deployments, compliance requirements
|
|
||||||
|
|
||||||
**Features**:
|
|
||||||
|
|
||||||
- All CI/CD features
|
|
||||||
- Harbor registry (enterprise OCI)
|
|
||||||
- Cosmian KMS (secret management)
|
|
||||||
- Full monitoring stack (Prometheus, Grafana)
|
|
||||||
- Log aggregation (Loki, Elasticsearch)
|
|
||||||
- Audit logging
|
|
||||||
- TLS/SSL encryption
|
|
||||||
- Nginx reverse proxy
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
### 1. Clone Repository
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /opt
|
|
||||||
git clone https://github.com/your-org/project-provisioning.git
|
|
||||||
cd project-provisioning/provisioning/platform
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Generate Secrets
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Generate .env file with random secrets
|
|
||||||
./scripts/generate-secrets.nu
|
|
||||||
|
|
||||||
# Or copy and edit manually
|
|
||||||
cp .env.example .env
|
|
||||||
nano .env
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Choose Deployment Mode and Deploy
|
|
||||||
|
|
||||||
#### Solo Mode
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/deploy-platform.nu --mode solo
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Multi-User Mode
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Generate secrets first
|
|
||||||
./scripts/generate-secrets.nu
|
|
||||||
|
|
||||||
# Deploy
|
|
||||||
./scripts/deploy-platform.nu --mode multi-user
|
|
||||||
```
|
|
||||||
|
|
||||||
#### CI/CD Mode
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/deploy-platform.nu --mode cicd --build
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Enterprise Mode
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Full production deployment
|
|
||||||
./scripts/deploy-platform.nu --mode enterprise --build --wait 600
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Verify Deployment
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check all services
|
|
||||||
./scripts/health-check.nu
|
|
||||||
|
|
||||||
# View logs
|
|
||||||
docker-compose logs -f
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. Access Services
|
|
||||||
|
|
||||||
- **Orchestrator**: <http://localhost:9090>
|
|
||||||
- **Control Center**: <http://localhost:8081>
|
|
||||||
- **OCI Registry**: <http://localhost:5000>
|
|
||||||
- **Gitea** (Multi-User+): <http://localhost:3000>
|
|
||||||
- **Grafana** (Enterprise): <http://localhost:3001>
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
The `.env` file controls all deployment settings. Key variables:
|
|
||||||
|
|
||||||
#### Platform Configuration
|
|
||||||
|
|
||||||
```toml
|
|
||||||
PROVISIONING_MODE=solo # solo, multi-user, cicd, enterprise
|
|
||||||
PLATFORM_ENVIRONMENT=development # development, staging, production
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Service Ports
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ORCHESTRATOR_PORT=8080
|
|
||||||
CONTROL_CENTER_PORT=8081
|
|
||||||
GITEA_HTTP_PORT=3000
|
|
||||||
OCI_REGISTRY_PORT=5000
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Security Settings
|
|
||||||
|
|
||||||
```toml
|
|
||||||
# Generate with: openssl rand -base64 32
|
|
||||||
CONTROL_CENTER_JWT_SECRET=<random-secret>
|
|
||||||
API_SERVER_JWT_SECRET=<random-secret>
|
|
||||||
POSTGRES_PASSWORD=<random-password>
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Resource Limits
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ORCHESTRATOR_CPU_LIMIT=2000m
|
|
||||||
ORCHESTRATOR_MEMORY_LIMIT=2048M
|
|
||||||
```
|
|
||||||
|
|
||||||
### Configuration Files
|
|
||||||
|
|
||||||
#### Docker Compose
|
|
||||||
|
|
||||||
- **Main**: `docker-compose.yaml` (base services)
|
|
||||||
- **Solo**: `infrastructure/docker/docker-compose.solo.yaml`
|
|
||||||
- **Multi-User**: `infrastructure/docker/docker-compose.multi-user.yaml`
|
|
||||||
- **CI/CD**: `infrastructure/docker/docker-compose.cicd.yaml`
|
|
||||||
- **Enterprise**: `infrastructure/docker/docker-compose.enterprise.yaml`
|
|
||||||
|
|
||||||
#### Service Configurations
|
|
||||||
|
|
||||||
- **Orchestrator**: `orchestrator/config.defaults.toml`
|
|
||||||
- **Control Center**: `control-center/config.defaults.toml`
|
|
||||||
- **CoreDNS**: `config/coredns/Corefile`
|
|
||||||
- **OCI Registry**: `infrastructure/oci-registry/config.json`
|
|
||||||
- **Nginx**: `infrastructure/nginx/nginx.conf`
|
|
||||||
- **Prometheus**: `infrastructure/monitoring/prometheus/prometheus.yml`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Deployment Methods
|
|
||||||
|
|
||||||
### Method 1: Docker Compose (Recommended)
|
|
||||||
|
|
||||||
#### Deploy
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Solo mode
|
|
||||||
docker-compose -f docker-compose.yaml
|
|
||||||
-f infrastructure/docker/docker-compose.solo.yaml
|
|
||||||
up -d
|
|
||||||
|
|
||||||
# Multi-user mode
|
|
||||||
docker-compose -f docker-compose.yaml
|
|
||||||
-f infrastructure/docker/docker-compose.multi-user.yaml
|
|
||||||
up -d
|
|
||||||
|
|
||||||
# CI/CD mode
|
|
||||||
docker-compose -f docker-compose.yaml
|
|
||||||
-f infrastructure/docker/docker-compose.multi-user.yaml
|
|
||||||
-f infrastructure/docker/docker-compose.cicd.yaml
|
|
||||||
up -d
|
|
||||||
|
|
||||||
# Enterprise mode
|
|
||||||
docker-compose -f docker-compose.yaml
|
|
||||||
-f infrastructure/docker/docker-compose.multi-user.yaml
|
|
||||||
-f infrastructure/docker/docker-compose.cicd.yaml
|
|
||||||
-f infrastructure/docker/docker-compose.enterprise.yaml
|
|
||||||
up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Manage Services
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# View logs
|
|
||||||
docker-compose logs -f [service-name]
|
|
||||||
|
|
||||||
# Restart service
|
|
||||||
docker-compose restart orchestrator
|
|
||||||
|
|
||||||
# Stop all services
|
|
||||||
docker-compose down
|
|
||||||
|
|
||||||
# Stop and remove volumes (WARNING: data loss)
|
|
||||||
docker-compose down --volumes
|
|
||||||
```
|
|
||||||
|
|
||||||
### Method 2: Systemd (Linux Production)
|
|
||||||
|
|
||||||
#### Install Services
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd systemd
|
|
||||||
sudo ./install-services.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Manage via systemd
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Start platform
|
|
||||||
sudo systemctl start provisioning-platform
|
|
||||||
|
|
||||||
# Enable auto-start on boot
|
|
||||||
sudo systemctl enable provisioning-platform
|
|
||||||
|
|
||||||
# Check status
|
|
||||||
sudo systemctl status provisioning-platform
|
|
||||||
|
|
||||||
# View logs
|
|
||||||
sudo journalctl -u provisioning-platform -f
|
|
||||||
|
|
||||||
# Restart
|
|
||||||
sudo systemctl restart provisioning-platform
|
|
||||||
|
|
||||||
# Stop
|
|
||||||
sudo systemctl stop provisioning-platform
|
|
||||||
```
|
|
||||||
|
|
||||||
### Method 3: Kubernetes
|
|
||||||
|
|
||||||
See [KUBERNETES_DEPLOYMENT.md](./KUBERNETES_DEPLOYMENT.md) for detailed instructions.
|
|
||||||
|
|
||||||
#### Quick Deploy
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create namespace
|
|
||||||
kubectl apply -f k8s/base/namespace.yaml
|
|
||||||
|
|
||||||
# Deploy services
|
|
||||||
kubectl apply -f k8s/deployments/
|
|
||||||
kubectl apply -f k8s/services/
|
|
||||||
kubectl apply -f k8s/ingress/
|
|
||||||
|
|
||||||
# Check status
|
|
||||||
kubectl get pods -n provisioning
|
|
||||||
```
|
|
||||||
|
|
||||||
### Method 4: Automation Script (Nushell)
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Deploy with options
|
|
||||||
./scripts/deploy-platform.nu --mode enterprise
|
|
||||||
--build
|
|
||||||
--wait 300
|
|
||||||
|
|
||||||
# Health check
|
|
||||||
./scripts/health-check.nu
|
|
||||||
|
|
||||||
# Dry run (show what would be deployed)
|
|
||||||
./scripts/deploy-platform.nu --mode enterprise --dry-run
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Post-Deployment
|
|
||||||
|
|
||||||
### 1. Verify Services
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Quick health check
|
|
||||||
./scripts/health-check.nu
|
|
||||||
|
|
||||||
# Detailed Docker status
|
|
||||||
docker-compose ps
|
|
||||||
|
|
||||||
# Check individual service
|
|
||||||
curl http://localhost:9090/health
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Initial Configuration
|
|
||||||
|
|
||||||
#### Create Admin User (Multi-User+)
|
|
||||||
|
|
||||||
Access Gitea at <http://localhost:3000> and complete setup wizard.
|
|
||||||
|
|
||||||
#### Configure DNS (Optional)
|
|
||||||
|
|
||||||
Add to `/etc/hosts` or configure local DNS:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
127.0.0.1 provisioning.local
|
|
||||||
127.0.0.1 gitea.provisioning.local
|
|
||||||
127.0.0.1 grafana.provisioning.local
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Configure Monitoring (Enterprise)
|
|
||||||
|
|
||||||
1. Access Grafana: <http://localhost:3001>
|
|
||||||
2. Login with credentials from `.env`:
|
|
||||||
- Username: `admin`
|
|
||||||
- Password: `${GRAFANA_ADMIN_PASSWORD}`
|
|
||||||
3. Dashboards are auto-provisioned from `infrastructure/monitoring/grafana/dashboards/`
|
|
||||||
|
|
||||||
### 3. Load Extensions
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# List available extensions
|
|
||||||
curl http://localhost:8082/api/v1/extensions
|
|
||||||
|
|
||||||
# Upload extension (example)
|
|
||||||
curl -X POST http://localhost:8082/api/v1/extensions/upload
|
|
||||||
-F "file=@my-extension.tar.gz"
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Test Workflows
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create test server (via orchestrator API)
|
|
||||||
curl -X POST http://localhost:9090/workflows/servers/create
|
|
||||||
-H "Content-Type: application/json"
|
|
||||||
-d '{"name": "test-server", "plan": "1xCPU-2GB"}'
|
|
||||||
|
|
||||||
# Check workflow status
|
|
||||||
curl http://localhost:9090/tasks/<task-id>
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Common Issues
|
|
||||||
|
|
||||||
#### Services Not Starting
|
|
||||||
|
|
||||||
**Symptom**: `docker-compose up` fails or services crash
|
|
||||||
|
|
||||||
**Solutions**:
|
|
||||||
|
|
||||||
1. Check Docker daemon:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
systemctl status docker
|
|
||||||
```
|
|
||||||
|
|
||||||
1. Check logs:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose logs orchestrator
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Check resource limits:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker stats
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Increase Docker resources in Docker Desktop settings
|
|
||||||
|
|
||||||
#### Port Conflicts
|
|
||||||
|
|
||||||
**Symptom**: `Error: port is already allocated`
|
|
||||||
|
|
||||||
**Solutions**:
|
|
||||||
|
|
||||||
1. Find conflicting process:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
lsof -i :9090
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Change port in `.env`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ORCHESTRATOR_PORT=9080
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Restart deployment:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose down
|
|
||||||
docker-compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Health Checks Failing
|
|
||||||
|
|
||||||
**Symptom**: Health check script reports unhealthy services
|
|
||||||
|
|
||||||
**Solutions**:
|
|
||||||
|
|
||||||
1. Check service logs:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose logs -f <service>
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Verify network connectivity:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker network inspect provisioning-net
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Check firewall rules:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo ufw status
|
|
||||||
```
|
|
||||||
|
|
||||||
4. Wait longer for services to start:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/deploy-platform.nu --wait 600
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Database Connection Errors
|
|
||||||
|
|
||||||
**Symptom**: PostgreSQL connection refused
|
|
||||||
|
|
||||||
**Solutions**:
|
|
||||||
|
|
||||||
1. Check PostgreSQL health:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker exec provisioning-postgres pg_isready
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Verify credentials in `.env`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
grep POSTGRES_ .env
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Check PostgreSQL logs:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose logs postgres
|
|
||||||
```
|
|
||||||
|
|
||||||
4. Recreate database:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose down
|
|
||||||
docker volume rm provisioning_postgres-data
|
|
||||||
docker-compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Out of Disk Space
|
|
||||||
|
|
||||||
**Symptom**: No space left on device
|
|
||||||
|
|
||||||
**Solutions**:
|
|
||||||
|
|
||||||
1. Clean Docker volumes:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker volume prune
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Clean Docker images:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker image prune -a
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Check volume sizes:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker system df -v
|
|
||||||
```
|
|
||||||
|
|
||||||
### Getting Help
|
|
||||||
|
|
||||||
- **Logs**: Always check logs first: `docker-compose logs -f`
|
|
||||||
- **Health**: Run health check: `./scripts/health-check.nu --json`
|
|
||||||
- **Documentation**: See `docs/` directory
|
|
||||||
- **Issues**: File bug reports at GitHub repository
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Security Best Practices
|
|
||||||
|
|
||||||
### 1. Secret Management
|
|
||||||
|
|
||||||
- **Never commit** `.env` files to version control
|
|
||||||
- Use `./scripts/generate-secrets.nu` to generate strong secrets
|
|
||||||
- Rotate secrets regularly
|
|
||||||
- Use KMS in enterprise mode
|
|
||||||
|
|
||||||
### 2. Network Security
|
|
||||||
|
|
||||||
- Use TLS/SSL in production (enterprise mode)
|
|
||||||
- Configure firewall rules:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo ufw allow 80/tcp
|
|
||||||
sudo ufw allow 443/tcp
|
|
||||||
sudo ufw enable
|
|
||||||
```
|
|
||||||
|
|
||||||
- Use private networks for backend services
|
|
||||||
|
|
||||||
### 3. Access Control
|
|
||||||
|
|
||||||
- Enable authentication in multi-user mode
|
|
||||||
- Use strong passwords (16+ characters)
|
|
||||||
- Configure API keys for CI/CD access
|
|
||||||
- Enable audit logging in enterprise mode
|
|
||||||
|
|
||||||
### 4. Regular Updates
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Pull latest images
|
|
||||||
docker-compose pull
|
|
||||||
|
|
||||||
# Rebuild with updates
|
|
||||||
./scripts/deploy-platform.nu --pull --build
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Backup and Recovery
|
|
||||||
|
|
||||||
### Backup
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Backup volumes
|
|
||||||
docker run --rm -v provisioning_orchestrator-data:/data
|
|
||||||
-v $(pwd)/backup:/backup
|
|
||||||
alpine tar czf /backup/orchestrator-data.tar.gz -C /data .
|
|
||||||
|
|
||||||
# Backup PostgreSQL
|
|
||||||
docker exec provisioning-postgres pg_dumpall -U provisioning > backup/postgres-backup.sql
|
|
||||||
```
|
|
||||||
|
|
||||||
### Restore
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Restore volume
|
|
||||||
docker run --rm -v provisioning_orchestrator-data:/data
|
|
||||||
-v $(pwd)/backup:/backup
|
|
||||||
alpine tar xzf /backup/orchestrator-data.tar.gz -C /data
|
|
||||||
|
|
||||||
# Restore PostgreSQL
|
|
||||||
docker exec -i provisioning-postgres psql -U provisioning < backup/postgres-backup.sql
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Maintenance
|
|
||||||
|
|
||||||
### Updates
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Pull latest images
|
|
||||||
docker-compose pull
|
|
||||||
|
|
||||||
# Recreate containers
|
|
||||||
docker-compose up -d --force-recreate
|
|
||||||
|
|
||||||
# Remove old images
|
|
||||||
docker image prune
|
|
||||||
```
|
|
||||||
|
|
||||||
### Monitoring
|
|
||||||
|
|
||||||
- **Prometheus**: <http://localhost:9090>
|
|
||||||
- **Grafana**: <http://localhost:3001>
|
|
||||||
- **Logs**: `docker-compose logs -f`
|
|
||||||
|
|
||||||
### Health Checks
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Automated health check
|
|
||||||
./scripts/health-check.nu
|
|
||||||
|
|
||||||
# Manual checks
|
|
||||||
curl http://localhost:9090/health
|
|
||||||
curl http://localhost:8081/health
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
- [Production Deployment Guide](./PRODUCTION_DEPLOYMENT.md)
|
|
||||||
- [Kubernetes Deployment Guide](./KUBERNETES_DEPLOYMENT.md)
|
|
||||||
- [Docker Compose Reference](./DOCKER_COMPOSE_REFERENCE.md)
|
|
||||||
- [Monitoring Setup](./MONITORING_SETUP.md)
|
|
||||||
- [Security Hardening](./SECURITY_HARDENING.md)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Documentation Version**: 1.0.0
|
|
||||||
**Last Updated**: 2025-10-06
|
|
||||||
**Maintained By**: Platform Team
|
|
||||||
|
|
@ -1,468 +0,0 @@
|
||||||
# Provisioning Platform - Deployment Guide
|
|
||||||
|
|
||||||
**Last Updated**: 2025-10-07
|
|
||||||
**Platform**: macOS (Apple Silicon / Intel) + OrbStack/Docker
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✅ Fixed: Docker Builds
|
|
||||||
|
|
||||||
Docker builds have been **fixed** to properly handle the Rust workspace structure. Both deployment methods (Native and Docker) are now fully
|
|
||||||
supported.
|
|
||||||
|
|
||||||
**Note**: Docker builds use Rust nightly to support edition2024 (required by async-graphql 7.x from surrealdb).
|
|
||||||
RocksDB has been replaced with SurrealDB in-memory backend (kv-mem) to simplify Docker builds (no libclang requirement).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📦 Quick Start
|
|
||||||
|
|
||||||
### Prerequisites
|
|
||||||
|
|
||||||
**For Native Deployment:**
|
|
||||||
|
|
||||||
- Rust 1.75+: `brew install rust`
|
|
||||||
- Nushell 0.107+: `brew install nushell`
|
|
||||||
|
|
||||||
**For Docker Deployment:**
|
|
||||||
|
|
||||||
- OrbStack (recommended): <https://orbstack.dev>
|
|
||||||
- Or Docker Desktop: `brew install --cask docker`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Deployment Methods
|
|
||||||
|
|
||||||
### Method 1: Native Execution (Recommended for Development)
|
|
||||||
|
|
||||||
**Fastest startup, easiest debugging, direct access to logs**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd provisioning/platform/scripts
|
|
||||||
|
|
||||||
# 1. Build all services
|
|
||||||
nu run-native.nu build
|
|
||||||
|
|
||||||
# 2. Start all services in background
|
|
||||||
nu run-native.nu start-all --background
|
|
||||||
|
|
||||||
# 3. Check status
|
|
||||||
nu run-native.nu status
|
|
||||||
|
|
||||||
# 4. View logs
|
|
||||||
nu run-native.nu logs orchestrator --follow
|
|
||||||
|
|
||||||
# 5. Stop all
|
|
||||||
nu run-native.nu stop-all
|
|
||||||
```
|
|
||||||
|
|
||||||
**Services will run on:**
|
|
||||||
|
|
||||||
- Orchestrator: <http://localhost:8080>
|
|
||||||
- Control Center: <http://localhost:8081>
|
|
||||||
|
|
||||||
**Data stored in:**
|
|
||||||
|
|
||||||
- `~/.provisioning-platform/data/`
|
|
||||||
- `~/.provisioning-platform/logs/`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Method 2: Docker Execution (Recommended for Production-like Testing)
|
|
||||||
|
|
||||||
**Isolated environments, easy cleanup, supports all deployment modes**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd provisioning/platform/scripts
|
|
||||||
|
|
||||||
# 1. Build Docker images (Solo mode)
|
|
||||||
nu run-docker.nu build solo
|
|
||||||
|
|
||||||
# 2. Start services in background
|
|
||||||
nu run-docker.nu start solo --detach
|
|
||||||
|
|
||||||
# 3. Check status
|
|
||||||
nu run-docker.nu status
|
|
||||||
|
|
||||||
# 4. View logs
|
|
||||||
nu run-docker.nu logs orchestrator --follow
|
|
||||||
|
|
||||||
# 5. Stop all
|
|
||||||
nu run-docker.nu stop
|
|
||||||
```
|
|
||||||
|
|
||||||
**Deployment Modes:**
|
|
||||||
|
|
||||||
- `solo` - 2 CPU / 4GB RAM (dev/test)
|
|
||||||
- `multiuser` - 4 CPU / 8GB RAM (team)
|
|
||||||
- `cicd` - 8 CPU / 16GB RAM (automation)
|
|
||||||
- `enterprise` - 16 CPU / 32GB RAM (production + KMS)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📋 Complete Command Reference
|
|
||||||
|
|
||||||
### Native Execution (`run-native.nu`)
|
|
||||||
|
|
||||||
| Command | Description |
|
|
||||||
| --------- | ------------- |
|
|
||||||
| `build` | Build all services |
|
|
||||||
| `start <service>` | Start orchestrator or control_center |
|
|
||||||
| `start-all` | Start all services |
|
|
||||||
| `stop <service>` | Stop a specific service |
|
|
||||||
| `stop-all` | Stop all services |
|
|
||||||
| `status` | Show service status |
|
|
||||||
| `logs <service>` | Show logs (add `--follow`) |
|
|
||||||
| `health` | Check service health |
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
nu run-native.nu build
|
|
||||||
nu run-native.nu start orchestrator --background
|
|
||||||
nu run-native.nu start control_center --background
|
|
||||||
nu run-native.nu logs orchestrator --follow
|
|
||||||
nu run-native.nu health
|
|
||||||
nu run-native.nu stop-all
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Docker Execution (`run-docker.nu`)
|
|
||||||
|
|
||||||
| Command | Description |
|
|
||||||
| --------- | ------------- |
|
|
||||||
| `build [mode]` | Build Docker images |
|
|
||||||
| `start [mode]` | Start services (add `--detach`) |
|
|
||||||
| `stop` | Stop all services (add `--volumes` to delete data) |
|
|
||||||
| `restart [mode]` | Restart services |
|
|
||||||
| `status` | Show container status |
|
|
||||||
| `logs <service>` | Show logs (add `--follow`) |
|
|
||||||
| `exec <service> <cmd>` | Execute command in container |
|
|
||||||
| `stats` | Show resource usage |
|
|
||||||
| `health` | Check service health |
|
|
||||||
| `config [mode]` | Show docker-compose config |
|
|
||||||
| `clean` | Remove containers (add `--all` for images too) |
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Solo mode (fastest)
|
|
||||||
nu run-docker.nu build solo
|
|
||||||
nu run-docker.nu start solo --detach
|
|
||||||
|
|
||||||
# Enterprise mode (with KMS)
|
|
||||||
nu run-docker.nu build enterprise
|
|
||||||
nu run-docker.nu start enterprise --detach
|
|
||||||
|
|
||||||
# Operations
|
|
||||||
nu run-docker.nu status
|
|
||||||
nu run-docker.nu logs control-center --follow
|
|
||||||
nu run-docker.nu exec orchestrator bash
|
|
||||||
nu run-docker.nu stats
|
|
||||||
nu run-docker.nu stop
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🗄️ Database Information
|
|
||||||
|
|
||||||
### Control-Center Database
|
|
||||||
|
|
||||||
**Type**: SurrealDB with in-memory backend (kv-mem)
|
|
||||||
**Location**: In-memory (data persisted during container/process lifetime)
|
|
||||||
**Production Alternative**: SurrealDB with remote WebSocket connection for persistent storage
|
|
||||||
|
|
||||||
**No separate database server required** - SurrealDB in-memory backend is embedded in the control-center process.
|
|
||||||
|
|
||||||
### Orchestrator Storage
|
|
||||||
|
|
||||||
**Type**: Filesystem queue (default)
|
|
||||||
**Location**:
|
|
||||||
|
|
||||||
- Native: `~/.provisioning-platform/data/orchestrator/queue.rkvs`
|
|
||||||
- Docker: `/data/queue.rkvs` (inside container)
|
|
||||||
|
|
||||||
**Production Option**: Switch to SurrealDB via config for distributed deployments.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ⚙️ Configuration Loading
|
|
||||||
|
|
||||||
Services load configuration in this order (priority: low → high):
|
|
||||||
|
|
||||||
1. **System Defaults** - `provisioning/config/config.defaults.toml`
|
|
||||||
2. **Service Defaults** - `provisioning/platform/{service}/config.defaults.toml`
|
|
||||||
3. **Workspace Config** - `workspace/{name}/config/provisioning.yaml`
|
|
||||||
4. **User Config** - `~/Library/Application Support/provisioning/user_config.yaml`
|
|
||||||
5. **Environment Variables** - `CONTROL_CENTER_*`, `ORCHESTRATOR_*`
|
|
||||||
6. **Runtime Overrides** - `--config` flag
|
|
||||||
|
|
||||||
**See full documentation**: `docs/architecture/DATABASE_AND_CONFIG_ARCHITECTURE.md`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🐛 Troubleshooting
|
|
||||||
|
|
||||||
### Native Deployment Issues
|
|
||||||
|
|
||||||
**Build fails:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Clean and rebuild
|
|
||||||
cd provisioning/platform
|
|
||||||
cargo clean
|
|
||||||
cargo build --release
|
|
||||||
```
|
|
||||||
|
|
||||||
**Port already in use:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check what's using the port
|
|
||||||
lsof -i :8080
|
|
||||||
lsof -i :8081
|
|
||||||
|
|
||||||
# Kill the process or use different ports via environment variables
|
|
||||||
export ORCHESTRATOR_SERVER_PORT=8090
|
|
||||||
export CONTROL_CENTER_SERVER_PORT=8091
|
|
||||||
```
|
|
||||||
|
|
||||||
**Service won't start:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check logs for errors
|
|
||||||
nu run-native.nu logs orchestrator
|
|
||||||
|
|
||||||
# Run in foreground to see output
|
|
||||||
nu run-native.nu start orchestrator
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Docker Deployment Issues
|
|
||||||
|
|
||||||
**Build fails with workspace errors:**
|
|
||||||
|
|
||||||
- **Fixed!** Dockerfiles now properly handle workspace structure
|
|
||||||
- If still failing: `nu run-docker.nu build solo --no-cache`
|
|
||||||
|
|
||||||
**Containers won't start:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check container logs
|
|
||||||
nu run-docker.nu logs orchestrator
|
|
||||||
|
|
||||||
# Check Docker daemon
|
|
||||||
docker ps
|
|
||||||
docker info
|
|
||||||
|
|
||||||
# Restart Docker/OrbStack
|
|
||||||
```
|
|
||||||
|
|
||||||
**Port conflicts:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check what's using ports
|
|
||||||
lsof -i :8080
|
|
||||||
lsof -i :8081
|
|
||||||
|
|
||||||
# Stop conflicting services or modify docker-compose.yaml ports
|
|
||||||
```
|
|
||||||
|
|
||||||
**Out of resources:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check current usage
|
|
||||||
nu run-docker.nu stats
|
|
||||||
|
|
||||||
# Clean up unused containers/images
|
|
||||||
docker system prune -a
|
|
||||||
|
|
||||||
# Or use the script
|
|
||||||
nu run-docker.nu clean --all
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔐 KMS Integration (Enterprise Mode)
|
|
||||||
|
|
||||||
Enterprise mode includes Cosmian KMS for production-grade secret management.
|
|
||||||
|
|
||||||
**Start with KMS:**
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
nu run-docker.nu build enterprise
|
|
||||||
nu run-docker.nu start enterprise --detach
|
|
||||||
```
|
|
||||||
|
|
||||||
**Access KMS:**
|
|
||||||
|
|
||||||
- KMS API: <http://localhost:9998>
|
|
||||||
- KMS Health: <http://localhost:9998/health>
|
|
||||||
|
|
||||||
**KMS Features:**
|
|
||||||
|
|
||||||
- SSL certificate lifecycle management
|
|
||||||
- SSH private key rotation
|
|
||||||
- Cloud credential auto-refresh
|
|
||||||
- Audit trails
|
|
||||||
- Automatic key rotation
|
|
||||||
|
|
||||||
**See full KMS documentation**: `provisioning/platform/control-center/src/kms/README.md`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Monitoring
|
|
||||||
|
|
||||||
### Health Checks
|
|
||||||
|
|
||||||
**Native:**
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
nu run-native.nu health
|
|
||||||
```
|
|
||||||
|
|
||||||
**Docker:**
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
nu run-docker.nu health
|
|
||||||
```
|
|
||||||
|
|
||||||
**Manual:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl http://localhost:8080/health # Orchestrator
|
|
||||||
curl http://localhost:8081/health # Control Center
|
|
||||||
curl http://localhost:9998/health # KMS (enterprise only)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Resource Usage
|
|
||||||
|
|
||||||
**Docker:**
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
nu run-docker.nu stats
|
|
||||||
```
|
|
||||||
|
|
||||||
**Native:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ps aux | grep -E "provisioning-orchestrator|control-center"
|
|
||||||
top -pid <pid>
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧪 Testing Both Methods
|
|
||||||
|
|
||||||
### Test Native Deployment
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd provisioning/platform/scripts
|
|
||||||
|
|
||||||
# 1. Build
|
|
||||||
nu run-native.nu build
|
|
||||||
|
|
||||||
# 2. Start services
|
|
||||||
nu run-native.nu start-all --background
|
|
||||||
|
|
||||||
# 3. Verify
|
|
||||||
nu run-native.nu status
|
|
||||||
nu run-native.nu health
|
|
||||||
|
|
||||||
# 4. Test API
|
|
||||||
curl http://localhost:8080/health
|
|
||||||
curl http://localhost:8081/health
|
|
||||||
|
|
||||||
# 5. Clean up
|
|
||||||
nu run-native.nu stop-all
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test Docker Deployment
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd provisioning/platform/scripts
|
|
||||||
|
|
||||||
# 1. Build
|
|
||||||
nu run-docker.nu build solo
|
|
||||||
|
|
||||||
# 2. Start services
|
|
||||||
nu run-docker.nu start solo --detach
|
|
||||||
|
|
||||||
# 3. Verify
|
|
||||||
nu run-docker.nu status
|
|
||||||
nu run-docker.nu health
|
|
||||||
|
|
||||||
# 4. Test API
|
|
||||||
curl http://localhost:8080/health
|
|
||||||
curl http://localhost:8081/health
|
|
||||||
|
|
||||||
# 5. Clean up
|
|
||||||
nu run-docker.nu stop --volumes
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 Best Practices
|
|
||||||
|
|
||||||
### Development Workflow
|
|
||||||
|
|
||||||
1. **Use Native for Active Development**
|
|
||||||
- Faster iteration (no Docker rebuild)
|
|
||||||
- Direct log access
|
|
||||||
- Easy debugging with IDE
|
|
||||||
|
|
||||||
2. **Use Docker for Integration Testing**
|
|
||||||
- Test deployment configurations
|
|
||||||
- Verify Docker builds
|
|
||||||
- Simulate production environment
|
|
||||||
|
|
||||||
### Production Deployment
|
|
||||||
|
|
||||||
1. **Use Docker/Kubernetes**
|
|
||||||
- Isolated environments
|
|
||||||
- Easy scaling
|
|
||||||
- Standard deployment
|
|
||||||
|
|
||||||
2. **Use Enterprise Mode**
|
|
||||||
- KMS for secret management
|
|
||||||
- Full monitoring stack
|
|
||||||
- High availability
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📚 Related Documentation
|
|
||||||
|
|
||||||
- **Architecture Overview**: [System Overview](../../docs/src/architecture/system-overview.md)
|
|
||||||
- **KMS Integration**: `provisioning/platform/control-center/src/kms/README.md`
|
|
||||||
- **Configuration System**: [Configuration Reference](../../docs/src/infrastructure/configuration-system.md)
|
|
||||||
- **Workspace Management**: [Workspace Management](../../docs/src/features/workspace-management.md)
|
|
||||||
- **Orchestrator API**: [Orchestrator Endpoints](../../docs/src/api-reference/orchestrator-endpoints.md)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✅ Summary
|
|
||||||
|
|
||||||
### Native Execution
|
|
||||||
|
|
||||||
- ✅ **Fixed**: Workspace builds work correctly
|
|
||||||
- ✅ **Fast**: No container overhead
|
|
||||||
- ✅ **Simple**: Direct binary execution
|
|
||||||
- ✅ **Best for**: Development, debugging
|
|
||||||
|
|
||||||
### Docker Execution
|
|
||||||
|
|
||||||
- ✅ **Fixed**: Dockerfiles now workspace-aware
|
|
||||||
- ✅ **Isolated**: Clean environments
|
|
||||||
- ✅ **Flexible**: Multiple deployment modes
|
|
||||||
- ✅ **Best for**: Testing, production-like deployments
|
|
||||||
|
|
||||||
**Both methods fully supported and tested!**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Quick Links:**
|
|
||||||
|
|
||||||
- Native Script: `provisioning/platform/scripts/run-native.nu`
|
|
||||||
- Docker Script: `provisioning/platform/scripts/run-docker.nu`
|
|
||||||
- Docker Files: `provisioning/platform/docker-compose.yaml` + mode-specific overrides
|
|
||||||
|
|
@ -1,97 +0,0 @@
|
||||||
# Known Issues - Provisioning Platform
|
|
||||||
|
|
||||||
## Control-Center Requires Rust Nightly (Edition 2024 Dependency)
|
|
||||||
|
|
||||||
**Status**: Resolved (using nightly)
|
|
||||||
**Severity**: Low
|
|
||||||
**Affects**: Docker deployment only
|
|
||||||
**Date Reported**: 2025-10-07
|
|
||||||
**Date Resolved**: 2025-10-07
|
|
||||||
|
|
||||||
### Issue
|
|
||||||
|
|
||||||
Control-center Docker builds fail with the following error:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
feature 'edition2024' is required
|
|
||||||
this Cargo does not support nightly features, but if you
|
|
||||||
switch to nightly channel you can add
|
|
||||||
`cargo-features = ["edition2024"]` to enable this feature
|
|
||||||
```
|
|
||||||
|
|
||||||
### Root Cause
|
|
||||||
|
|
||||||
Dependency chain:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
control-center → surrealdb 2.3.10 → surrealdb-core 2.3.10 → async-graphql 7.0.17
|
|
||||||
```
|
|
||||||
|
|
||||||
The `async-graphql-value` crate v7.0.17 requires Rust edition 2024, which is not yet stable in Rust 1.82.
|
|
||||||
Edition 2024 is currently only available in Rust nightly builds.
|
|
||||||
|
|
||||||
### Resolution
|
|
||||||
|
|
||||||
**Updated Dockerfiles to use Rust nightly** (2025-10-07):
|
|
||||||
|
|
||||||
Both `orchestrator/Dockerfile` and `control-center/Dockerfile` now use:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
FROM rustlang/rust:nightly-bookworm AS builder
|
|
||||||
```
|
|
||||||
|
|
||||||
This provides edition2024 support required by the surrealdb dependency chain.
|
|
||||||
|
|
||||||
### Production Considerations
|
|
||||||
|
|
||||||
**Rust Nightly Stability**:
|
|
||||||
|
|
||||||
- Nightly builds are generally stable for compilation
|
|
||||||
- The compiled binaries are production-ready
|
|
||||||
- Runtime behavior is not affected by nightly vs stable compilation
|
|
||||||
- Consider pinning to a specific nightly date for reproducible builds
|
|
||||||
|
|
||||||
**Alternative**: Use native deployment with stable Rust if nightly is a concern:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
cd provisioning/platform/scripts
|
|
||||||
nu run-native.nu build
|
|
||||||
nu run-native.nu start-all --background
|
|
||||||
```
|
|
||||||
|
|
||||||
### Timeline
|
|
||||||
|
|
||||||
- **Rust 1.85** (estimated Feb 2025): Expected to stabilize edition 2024
|
|
||||||
- **SurrealDB 3.x**: May drop async-graphql dependency
|
|
||||||
|
|
||||||
### Tracking
|
|
||||||
|
|
||||||
- Rust Edition 2024 RFC: <https://github.com/rust-lang/rfcs/pull/3501>
|
|
||||||
- SurrealDB Issue: <https://github.com/surrealdb/surrealdb/issues>
|
|
||||||
- async-graphql Issue: <https://github.com/async-graphql/async-graphql/issues>
|
|
||||||
|
|
||||||
### Related Files
|
|
||||||
|
|
||||||
- `provisioning/platform/control-center/Dockerfile`
|
|
||||||
- `provisioning/platform/Cargo.toml` (workspace dependencies)
|
|
||||||
- `provisioning/platform/control-center/Cargo.toml`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## RocksDB Build Takes Long Time
|
|
||||||
|
|
||||||
**Status**: Known Limitation
|
|
||||||
**Severity**: Low
|
|
||||||
**Affects**: All builds
|
|
||||||
|
|
||||||
### Issue
|
|
||||||
|
|
||||||
RocksDB compilation takes 30-60 seconds during builds.
|
|
||||||
|
|
||||||
### Workaround
|
|
||||||
|
|
||||||
Use cached Docker layers or native builds with incremental compilation.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Last Updated**: 2025-10-07
|
|
||||||
|
|
@ -1,282 +0,0 @@
|
||||||
# Provisioning Platform - Quick Start
|
|
||||||
|
|
||||||
Fast deployment guide for all modes.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Verify Docker is installed and running
|
|
||||||
docker --version # 20.10+
|
|
||||||
docker-compose --version # 2.0+
|
|
||||||
docker ps # Should work without errors
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Solo Mode (Local Development)
|
|
||||||
|
|
||||||
**Services**: Orchestrator, Control Center, CoreDNS, OCI Registry, Extension Registry
|
|
||||||
|
|
||||||
**Resources**: 2 CPU cores, 4GB RAM, 20GB disk
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /Users/Akasha/project-provisioning/provisioning/platform
|
|
||||||
|
|
||||||
# Generate secrets
|
|
||||||
./scripts/generate-secrets.nu
|
|
||||||
|
|
||||||
# Deploy
|
|
||||||
./scripts/deploy-platform.nu --mode solo
|
|
||||||
|
|
||||||
# Verify
|
|
||||||
./scripts/health-check.nu
|
|
||||||
|
|
||||||
# Access
|
|
||||||
open http://localhost:8080 # Orchestrator
|
|
||||||
open http://localhost:8081 # Control Center
|
|
||||||
```
|
|
||||||
|
|
||||||
**Stop**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose down
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Multi-User Mode (Team Collaboration)
|
|
||||||
|
|
||||||
**Services**: Solo + Gitea, PostgreSQL
|
|
||||||
|
|
||||||
**Resources**: 4 CPU cores, 8GB RAM, 50GB disk
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /Users/Akasha/project-provisioning/provisioning/platform
|
|
||||||
|
|
||||||
# Generate secrets
|
|
||||||
./scripts/generate-secrets.nu
|
|
||||||
|
|
||||||
# Deploy
|
|
||||||
./scripts/deploy-platform.nu --mode multi-user
|
|
||||||
|
|
||||||
# Verify
|
|
||||||
./scripts/health-check.nu
|
|
||||||
|
|
||||||
# Access
|
|
||||||
open http://localhost:3000 # Gitea
|
|
||||||
open http://localhost:8081 # Control Center
|
|
||||||
```
|
|
||||||
|
|
||||||
**Configure Gitea**:
|
|
||||||
|
|
||||||
1. Visit <http://localhost:3000>
|
|
||||||
2. Complete initial setup wizard
|
|
||||||
3. Create admin account
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. CI/CD Mode (Automated Pipelines)
|
|
||||||
|
|
||||||
**Services**: Multi-User + API Server, Jenkins (optional), GitLab Runner (optional)
|
|
||||||
|
|
||||||
**Resources**: 8 CPU cores, 16GB RAM, 100GB disk
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /Users/Akasha/project-provisioning/provisioning/platform
|
|
||||||
|
|
||||||
# Generate secrets
|
|
||||||
./scripts/generate-secrets.nu
|
|
||||||
|
|
||||||
# Deploy
|
|
||||||
./scripts/deploy-platform.nu --mode cicd --build
|
|
||||||
|
|
||||||
# Verify
|
|
||||||
./scripts/health-check.nu
|
|
||||||
|
|
||||||
# Access
|
|
||||||
open http://localhost:8083 # API Server
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Enterprise Mode (Production)
|
|
||||||
|
|
||||||
**Services**: Full stack (15+ services)
|
|
||||||
|
|
||||||
**Resources**: 16 CPU cores, 32GB RAM, 500GB disk
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /Users/Akasha/project-provisioning/provisioning/platform
|
|
||||||
|
|
||||||
# Generate production secrets
|
|
||||||
./scripts/generate-secrets.nu --output .env.production
|
|
||||||
|
|
||||||
# Review and customize
|
|
||||||
nano .env.production
|
|
||||||
|
|
||||||
# Deploy with build
|
|
||||||
./scripts/deploy-platform.nu --mode enterprise
|
|
||||||
--env-file .env.production
|
|
||||||
--build
|
|
||||||
--wait 600
|
|
||||||
|
|
||||||
# Verify
|
|
||||||
./scripts/health-check.nu
|
|
||||||
|
|
||||||
# Access
|
|
||||||
open http://localhost:3001 # Grafana (admin / password from .env)
|
|
||||||
open http://localhost:9090 # Prometheus
|
|
||||||
open http://localhost:5601 # Kibana
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Common Commands
|
|
||||||
|
|
||||||
### View Logs
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose logs -f
|
|
||||||
docker-compose logs -f orchestrator
|
|
||||||
docker-compose logs --tail=100 orchestrator
|
|
||||||
```
|
|
||||||
|
|
||||||
### Restart Services
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose restart orchestrator
|
|
||||||
docker-compose restart
|
|
||||||
```
|
|
||||||
|
|
||||||
### Update Platform
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose pull
|
|
||||||
./scripts/deploy-platform.nu --mode <your-mode> --pull
|
|
||||||
```
|
|
||||||
|
|
||||||
### Stop Platform
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose down
|
|
||||||
```
|
|
||||||
|
|
||||||
### Clean Everything (WARNING: data loss)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker-compose down --volumes
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Systemd (Linux Production)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Install services
|
|
||||||
cd systemd
|
|
||||||
sudo ./install-services.sh
|
|
||||||
|
|
||||||
# Enable and start
|
|
||||||
sudo systemctl enable --now provisioning-platform
|
|
||||||
|
|
||||||
# Check status
|
|
||||||
sudo systemctl status provisioning-platform
|
|
||||||
|
|
||||||
# View logs
|
|
||||||
sudo journalctl -u provisioning-platform -f
|
|
||||||
|
|
||||||
# Restart
|
|
||||||
sudo systemctl restart provisioning-platform
|
|
||||||
|
|
||||||
# Stop
|
|
||||||
sudo systemctl stop provisioning-platform
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Services not starting
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check Docker
|
|
||||||
systemctl status docker
|
|
||||||
|
|
||||||
# Check logs
|
|
||||||
docker-compose logs orchestrator
|
|
||||||
|
|
||||||
# Check resources
|
|
||||||
docker stats
|
|
||||||
```
|
|
||||||
|
|
||||||
### Port conflicts
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Find what's using port
|
|
||||||
lsof -i :8080
|
|
||||||
|
|
||||||
# Change port in .env
|
|
||||||
nano .env
|
|
||||||
# Set ORCHESTRATOR_PORT=9080
|
|
||||||
|
|
||||||
# Restart
|
|
||||||
docker-compose down && docker-compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
### Health checks failing
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check individual service
|
|
||||||
curl http://localhost:8080/health
|
|
||||||
|
|
||||||
# Wait longer
|
|
||||||
./scripts/deploy-platform.nu --wait 600
|
|
||||||
|
|
||||||
# Check networks
|
|
||||||
docker network inspect provisioning-net
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Access URLs
|
|
||||||
|
|
||||||
### Solo Mode
|
|
||||||
|
|
||||||
- Orchestrator: <http://localhost:8080>
|
|
||||||
- Control Center: <http://localhost:8081>
|
|
||||||
- OCI Registry: <http://localhost:5000>
|
|
||||||
|
|
||||||
### Multi-User Mode
|
|
||||||
|
|
||||||
- Gitea: <http://localhost:3000>
|
|
||||||
- PostgreSQL: localhost:5432
|
|
||||||
|
|
||||||
### CI/CD Mode
|
|
||||||
|
|
||||||
- API Server: <http://localhost:8083>
|
|
||||||
|
|
||||||
### Enterprise Mode
|
|
||||||
|
|
||||||
- Prometheus: <http://localhost:9090>
|
|
||||||
- Grafana: <http://localhost:3001>
|
|
||||||
- Kibana: <http://localhost:5601>
|
|
||||||
- Nginx: <http://localhost:80>
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
- **Full Guide**: See `docs/deployment/deployment-guide.md`
|
|
||||||
- **Configuration**: Edit `.env` file for customization
|
|
||||||
- **Monitoring**: Access Grafana dashboards (enterprise mode)
|
|
||||||
- **API**: Use API Server for automation (CI/CD mode)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Need Help?**
|
|
||||||
|
|
||||||
- Health Check: `./scripts/health-check.nu`
|
|
||||||
- Logs: `docker-compose logs -f`
|
|
||||||
- Documentation: `docs/deployment/`
|
|
||||||
Loading…
Add table
Reference in a new issue