prvng_platform/docs/deployment/DEPLOYMENT_GUIDE.md

Provisioning Platform Deployment Guide

Version: 3.0.0 Date: 2025-10-06 Deployment Modes: Solo, Multi-User, CI/CD, Enterprise

---

Table of Contents

1. Overview
2. Prerequisites
3. Deployment Modes
4. Quick Start
5. Configuration
6. Deployment Methods
7. Post-Deployment
8. Troubleshooting

---

Overview

The Provisioning Platform is a comprehensive infrastructure automation system that can be deployed in four modes:

• Solo: Single-user local development (minimal services)
• Multi-User: Team collaboration with source control
• CI/CD: Automated deployment pipelines
• Enterprise: Full production with monitoring, KMS, and audit logging

Architecture Components

Component	Solo	Multi-User	CI/CD	Enterprise
Orchestrator	✓	✓	✓	✓
Control Center	✓	✓	✓	✓
CoreDNS	✓	✓	✓	✓
OCI Registry (Zot)	✓	✓	✓	-
Extension Registry	✓	✓	✓	✓
Gitea	-	✓	✓	✓
PostgreSQL	-	✓	✓	✓
API Server	-	-	✓	✓
Harbor	-	-	-	✓
Cosmian KMS	-	-	-	✓
Prometheus	-	-	-	✓
Grafana	-	-	-	✓
Loki + Promtail	-	-	-	✓
Elasticsearch + Kibana	-	-	-	✓
Nginx Reverse Proxy	-	-	-	✓

---

Prerequisites

Required Software

1. Docker (version 20.10+)

   docker --version
   # Docker version 20.10.0 or higher
   

2. Docker Compose (version 2.0+)

   docker-compose --version
   # Docker Compose version 2.0.0 or higher
   

3. Nushell (version 0.107.1+ for automation scripts)

   nu --version
   # 0.107.1 or higher
   

System Requirements

Solo Mode

• CPU: 2 cores
• Memory: 4GB RAM
• Disk: 20GB free space
• Network: Internet connection for pulling images

Multi-User Mode

• CPU: 4 cores
• Memory: 8GB RAM
• Disk: 50GB free space
• Network: Internet connection + internal network

CI/CD Mode

• CPU: 8 cores
• Memory: 16GB RAM
• Disk: 100GB free space
• Network: Internet + dedicated CI/CD network

Enterprise Mode

• CPU: 16 cores
• Memory: 32GB RAM
• Disk: 500GB free space (SSD recommended)
• Network: High-bandwidth, low-latency network

Optional Tools

• OpenSSL (for generating secrets)
• kubectl (for Kubernetes deployment)
• Helm (for Kubernetes package management)

---

Deployment Modes

Solo Mode

Use Case: Local development, testing, personal use

Features:

• Minimal resource usage
• No authentication required
• SQLite databases
• Local file storage

Limitations:

• Single user only
• No version control integration
• No audit logging

Multi-User Mode

Use Case: Small team collaboration

Features:

• Multi-user authentication
• Gitea for source control
• PostgreSQL shared database
• User management

Limitations:

• No automated pipelines
• No advanced monitoring

CI/CD Mode

Use Case: Automated deployment pipelines

Features:

• All Multi-User features
• Provisioning API Server
• Webhook support
• Jenkins/GitLab Runner integration

Limitations:

• Basic monitoring only

Enterprise Mode

Use Case: Production deployments, compliance requirements

Features:

• All CI/CD features
• Harbor registry (enterprise OCI)
• Cosmian KMS (secret management)
• Full monitoring stack (Prometheus, Grafana)
• Log aggregation (Loki, Elasticsearch)
• Audit logging
• TLS/SSL encryption
• Nginx reverse proxy

---

Quick Start

1. Clone Repository

cd /opt
git clone https://github.com/your-org/project-provisioning.git
cd project-provisioning/provisioning/platform

2. Generate Secrets

# Generate .env file with random secrets
./scripts/generate-secrets.nu

# Or copy and edit manually
cp .env.example .env
nano .env

3. Choose Deployment Mode and Deploy

Solo Mode

./scripts/deploy-platform.nu --mode solo

Multi-User Mode

# Generate secrets first
./scripts/generate-secrets.nu

# Deploy
./scripts/deploy-platform.nu --mode multi-user

CI/CD Mode

./scripts/deploy-platform.nu --mode cicd --build

Enterprise Mode

# Full production deployment
./scripts/deploy-platform.nu --mode enterprise --build --wait 600

4. Verify Deployment

# Check all services
./scripts/health-check.nu

# View logs
docker-compose logs -f

5. Access Services

• Orchestrator: http://localhost:8080
• Control Center: http://localhost:8081
• OCI Registry: http://localhost:5000
• Gitea (Multi-User+): http://localhost:3000
• Grafana (Enterprise): http://localhost:3001

---

Configuration

Environment Variables

The .env file controls all deployment settings. Key variables:

Platform Configuration

PROVISIONING_MODE=solo           # solo, multi-user, cicd, enterprise
PLATFORM_ENVIRONMENT=development # development, staging, production

Service Ports

ORCHESTRATOR_PORT=8080
CONTROL_CENTER_PORT=8081
GITEA_HTTP_PORT=3000
OCI_REGISTRY_PORT=5000

Security Settings

# Generate with: openssl rand -base64 32
CONTROL_CENTER_JWT_SECRET=<random-secret>
API_SERVER_JWT_SECRET=<random-secret>
POSTGRES_PASSWORD=<random-password>

Resource Limits

ORCHESTRATOR_CPU_LIMIT=2000m
ORCHESTRATOR_MEMORY_LIMIT=2048M

Configuration Files

Docker Compose

• Main: docker-compose.yaml (base services)
• Solo: docker-compose/docker-compose.solo.yaml
• Multi-User: docker-compose/docker-compose.multi-user.yaml
• CI/CD: docker-compose/docker-compose.cicd.yaml
• Enterprise: docker-compose/docker-compose.enterprise.yaml

Service Configurations

• Orchestrator: orchestrator/config.defaults.toml
• Control Center: control-center/config.defaults.toml
• CoreDNS: coredns/Corefile
• OCI Registry: oci-registry/config.json
• Nginx: nginx/nginx.conf
• Prometheus: monitoring/prometheus/prometheus.yml

---

Deployment Methods

Method 1: Docker Compose (Recommended)

Deploy

# Solo mode
docker-compose -f docker-compose.yaml \
               -f docker-compose/docker-compose.solo.yaml \
               up -d

# Multi-user mode
docker-compose -f docker-compose.yaml \
               -f docker-compose/docker-compose.multi-user.yaml \
               up -d

# CI/CD mode
docker-compose -f docker-compose.yaml \
               -f docker-compose/docker-compose.multi-user.yaml \
               -f docker-compose/docker-compose.cicd.yaml \
               up -d

# Enterprise mode
docker-compose -f docker-compose.yaml \
               -f docker-compose/docker-compose.multi-user.yaml \
               -f docker-compose/docker-compose.cicd.yaml \
               -f docker-compose/docker-compose.enterprise.yaml \
               up -d

Manage Services

# View logs
docker-compose logs -f [service-name]

# Restart service
docker-compose restart orchestrator

# Stop all services
docker-compose down

# Stop and remove volumes (WARNING: data loss)
docker-compose down --volumes

Method 2: Systemd (Linux Production)

Install Services

cd systemd
sudo ./install-services.sh

Manage via systemd

# Start platform
sudo systemctl start provisioning-platform

# Enable auto-start on boot
sudo systemctl enable provisioning-platform

# Check status
sudo systemctl status provisioning-platform

# View logs
sudo journalctl -u provisioning-platform -f

# Restart
sudo systemctl restart provisioning-platform

# Stop
sudo systemctl stop provisioning-platform

Method 3: Kubernetes

See KUBERNETES_DEPLOYMENT.md for detailed instructions.

Quick Deploy

# Create namespace
kubectl apply -f k8s/base/namespace.yaml

# Deploy services
kubectl apply -f k8s/deployments/
kubectl apply -f k8s/services/
kubectl apply -f k8s/ingress/

# Check status
kubectl get pods -n provisioning

Method 4: Automation Script (Nushell)

# Deploy with options
./scripts/deploy-platform.nu --mode enterprise \
                              --build \
                              --wait 300

# Health check
./scripts/health-check.nu

# Dry run (show what would be deployed)
./scripts/deploy-platform.nu --mode enterprise --dry-run

---

Post-Deployment

1. Verify Services

# Quick health check
./scripts/health-check.nu

# Detailed Docker status
docker-compose ps

# Check individual service
curl http://localhost:8080/health

2. Initial Configuration

Create Admin User (Multi-User+)

Access Gitea at http://localhost:3000 and complete setup wizard.

Configure DNS (Optional)

Add to /etc/hosts or configure local DNS:

127.0.0.1  provisioning.local
127.0.0.1  gitea.provisioning.local
127.0.0.1  grafana.provisioning.local

Configure Monitoring (Enterprise)

1. Access Grafana: http://localhost:3001
2. Login with credentials from .env:
   • Username: admin
   • Password: ${GRAFANA_ADMIN_PASSWORD}
3. Dashboards are auto-provisioned from monitoring/grafana/dashboards/

3. Load Extensions

# List available extensions
curl http://localhost:8082/api/v1/extensions

# Upload extension (example)
curl -X POST http://localhost:8082/api/v1/extensions/upload \
     -F "file=@my-extension.tar.gz"

4. Test Workflows

# Create test server (via orchestrator API)
curl -X POST http://localhost:8080/workflows/servers/create \
     -H "Content-Type: application/json" \
     -d '{"name": "test-server", "plan": "1xCPU-2GB"}'

# Check workflow status
curl http://localhost:8080/tasks/<task-id>

---

Troubleshooting

Common Issues

Services Not Starting

Symptom: docker-compose up fails or services crash

Solutions:

1. Check Docker daemon:

   systemctl status docker
   

2. Check logs:

   docker-compose logs orchestrator
   

3. Check resource limits:

   docker stats
   

4. Increase Docker resources in Docker Desktop settings

Port Conflicts

Symptom: Error: port is already allocated

Solutions:

1. Find conflicting process:

   lsof -i :8080
   

2. Change port in .env:

   ORCHESTRATOR_PORT=9080
   

3. Restart deployment:

   docker-compose down
   docker-compose up -d
   

Health Checks Failing

Symptom: Health check script reports unhealthy services

Solutions:

1. Check service logs:

   docker-compose logs -f <service>
   

2. Verify network connectivity:

   docker network inspect provisioning-net
   

3. Check firewall rules:

   sudo ufw status
   

4. Wait longer for services to start:

   ./scripts/deploy-platform.nu --wait 600
   

Database Connection Errors

Symptom: PostgreSQL connection refused

Solutions:

1. Check PostgreSQL health:

   docker exec provisioning-postgres pg_isready
   

2. Verify credentials in .env:

   grep POSTGRES_ .env
   

3. Check PostgreSQL logs:

   docker-compose logs postgres
   

4. Recreate database:

   docker-compose down
   docker volume rm provisioning_postgres-data
   docker-compose up -d
   

Out of Disk Space

Symptom: No space left on device

Solutions:

1. Clean Docker volumes:

   docker volume prune
   

2. Clean Docker images:

   docker image prune -a
   

3. Check volume sizes:

   docker system df -v
   

Getting Help

• Logs: Always check logs first: docker-compose logs -f
• Health: Run health check: ./scripts/health-check.nu --json
• Documentation: See docs/ directory
• Issues: File bug reports at GitHub repository

---

Security Best Practices

1. Secret Management

• Never commit .env files to version control
• Use ./scripts/generate-secrets.nu to generate strong secrets
• Rotate secrets regularly
• Use KMS in enterprise mode

2. Network Security

• Use TLS/SSL in production (enterprise mode)
• Configure firewall rules:

  sudo ufw allow 80/tcp
  sudo ufw allow 443/tcp
  sudo ufw enable
  

• Use private networks for backend services

3. Access Control

• Enable authentication in multi-user mode
• Use strong passwords (16+ characters)
• Configure API keys for CI/CD access
• Enable audit logging in enterprise mode

4. Regular Updates

# Pull latest images
docker-compose pull

# Rebuild with updates
./scripts/deploy-platform.nu --pull --build

---

Backup and Recovery

Backup

# Backup volumes
docker run --rm -v provisioning_orchestrator-data:/data \
           -v $(pwd)/backup:/backup \
           alpine tar czf /backup/orchestrator-data.tar.gz -C /data .

# Backup PostgreSQL
docker exec provisioning-postgres pg_dumpall -U provisioning > backup/postgres-backup.sql

Restore

# Restore volume
docker run --rm -v provisioning_orchestrator-data:/data \
           -v $(pwd)/backup:/backup \
           alpine tar xzf /backup/orchestrator-data.tar.gz -C /data

# Restore PostgreSQL
docker exec -i provisioning-postgres psql -U provisioning < backup/postgres-backup.sql

---

Maintenance

Updates

# Pull latest images
docker-compose pull

# Recreate containers
docker-compose up -d --force-recreate

# Remove old images
docker image prune

Monitoring

• Prometheus: http://localhost:9090
• Grafana: http://localhost:3001
• Logs: docker-compose logs -f

Health Checks

# Automated health check
./scripts/health-check.nu

# Manual checks
curl http://localhost:8080/health
curl http://localhost:8081/health

---

Next Steps

• Production Deployment Guide
• Kubernetes Deployment Guide
• Docker Compose Reference
• Monitoring Setup
• Security Hardening

---

Documentation Version: 1.0.0 Last Updated: 2025-10-06 Maintained By: Platform Team