557 lines
17 KiB
Markdown
557 lines
17 KiB
Markdown
<p align="center">
|
|
<img src="https://repo.jesusperez.pro/jesus/provisioning/media/branch/main/resources/provisioning_logo.svg" alt="Provisioning Logo" width="300"/>
|
|
</p>
|
|
<p align="center">
|
|
<img src="https://repo.jesusperez.pro/jesus/provisioning/media/branch/main/resources/logo-text.svg" alt="Provisioning" width="500"/>
|
|
</p>
|
|
|
|
|
|
---
|
|
|
|
# Platform Services
|
|
|
|
Platform-level services for the [Provisioning project](https://repo.jesusperez.pro/jesus/provisioning) infrastructure automation platform. These services provide the high-performance execution layer, management interfaces, and supporting infrastructure for the entire provisioning system.
|
|
|
|
## Overview
|
|
|
|
The Platform layer consists of **production-ready services** built primarily in Rust, providing:
|
|
|
|
- **Workflow Execution** - High-performance orchestration and task coordination
|
|
- **Management Interfaces** - Web UI and REST APIs for infrastructure management
|
|
- **Security & Authorization** - Enterprise-grade access control and permissions
|
|
- **Installation & Distribution** - Multi-mode installer with TUI, CLI, and unattended modes
|
|
- **AI Integration** - Model Context Protocol (MCP) server for intelligent assistance
|
|
- **Extension Management** - OCI-based registry for distributing modules
|
|
|
|
---
|
|
|
|
## Core Platform Services
|
|
|
|
### 1. **Orchestrator** (`orchestrator/`)
|
|
|
|
High-performance Rust/Nushell hybrid orchestrator for workflow execution.
|
|
|
|
**Language**: Rust + Nushell integration
|
|
|
|
**Purpose**: Workflow execution, task scheduling, state management
|
|
|
|
**Key Features**:
|
|
- File-based persistence for reliability
|
|
- Priority processing with retry logic
|
|
- Checkpoint recovery and automatic rollback
|
|
- REST API endpoints for external integration
|
|
- Solves deep call stack limitations
|
|
- Parallel task execution with dependency resolution
|
|
|
|
**Status**: ✅ Production Ready (v3.0.0)
|
|
|
|
**Documentation**: See [.claude/features/orchestrator-architecture.md](../../.claude/features/orchestrator-architecture.md)
|
|
|
|
**Quick Start**:
|
|
```bash
|
|
cd orchestrator
|
|
./scripts/start-orchestrator.nu --background
|
|
```
|
|
|
|
**REST API**:
|
|
- `GET http://localhost:8080/health` - Health check
|
|
- `GET http://localhost:8080/tasks` - List all tasks
|
|
- `POST http://localhost:8080/workflows/servers/create` - Server workflow
|
|
- `POST http://localhost:8080/workflows/taskserv/create` - Taskserv workflow
|
|
|
|
---
|
|
|
|
### 2. **Control Center** (`control-center/`)
|
|
|
|
Backend control center service with authorization and permissions management.
|
|
|
|
**Language**: Rust
|
|
|
|
**Purpose**: Web-based infrastructure management with RBAC
|
|
|
|
**Key Features**:
|
|
- **Authorization and permissions control** (enterprise security)
|
|
- Role-Based Access Control (RBAC)
|
|
- Audit logging and compliance tracking
|
|
- System management APIs
|
|
- Configuration management
|
|
- Resource monitoring
|
|
|
|
**Status**: ✅ Active Development
|
|
|
|
**Security Features**:
|
|
- Fine-grained permissions system
|
|
- User authentication and session management
|
|
- API key management
|
|
- Activity audit logs
|
|
|
|
---
|
|
|
|
### 3. **Control Center UI** (`control-center-ui/`)
|
|
|
|
Frontend web interface for infrastructure management.
|
|
|
|
**Language**: Web (HTML/CSS/JavaScript)
|
|
|
|
**Purpose**: User-friendly dashboard and administration interface
|
|
|
|
**Key Features**:
|
|
- Dashboard with real-time monitoring
|
|
- Configuration management interface
|
|
- System administration tools
|
|
- Workflow visualization
|
|
- Log viewing and search
|
|
|
|
**Status**: ✅ Active Development
|
|
|
|
**Integration**: Communicates with Control Center backend and Orchestrator APIs
|
|
|
|
---
|
|
|
|
### 4. **Installer** (`installer/`)
|
|
|
|
Multi-mode platform installation system with interactive TUI, headless CLI, and unattended modes.
|
|
|
|
**Language**: Rust (Ratatui TUI) + Nushell scripts
|
|
|
|
**Purpose**: Platform installation and configuration generation
|
|
|
|
**Key Features**:
|
|
- **Interactive TUI Mode**: Beautiful terminal UI with 7 screens
|
|
- **Headless Mode**: CLI automation for scripted installations
|
|
- **Unattended Mode**: Zero-interaction CI/CD deployments
|
|
- **Deployment Modes**: Solo (2 CPU/4GB), MultiUser (4 CPU/8GB), CICD (8 CPU/16GB), Enterprise (16 CPU/32GB)
|
|
- **MCP Integration**: 7 AI-powered settings tools for intelligent configuration
|
|
- **Nushell Scripts**: Complete deployment automation for Docker, Podman, Kubernetes, OrbStack
|
|
|
|
**Status**: ✅ Production Ready (v3.5.0)
|
|
|
|
**Quick Start**:
|
|
```bash
|
|
# Interactive TUI
|
|
provisioning-installer
|
|
|
|
# Headless mode
|
|
provisioning-installer --headless --mode solo --yes
|
|
|
|
# Unattended CI/CD
|
|
provisioning-installer --unattended --config config.toml
|
|
```
|
|
|
|
**Documentation**: `installer/docs/` - Complete guides and references
|
|
|
|
---
|
|
|
|
### 5. **MCP Server** (`mcp-server/`)
|
|
|
|
Model Context Protocol server for AI-powered assistance.
|
|
|
|
**Language**: Nushell
|
|
|
|
**Purpose**: AI integration for intelligent configuration and assistance
|
|
|
|
**Key Features**:
|
|
- 7 AI-powered settings tools
|
|
- Intelligent config completion
|
|
- Natural language infrastructure queries
|
|
- Configuration validation and suggestions
|
|
- Context-aware help system
|
|
|
|
**Status**: ✅ Active Development
|
|
|
|
**MCP Tools**:
|
|
- Settings generation
|
|
- Configuration validation
|
|
- Best practice recommendations
|
|
- Infrastructure planning assistance
|
|
- Error diagnosis and resolution
|
|
|
|
---
|
|
|
|
### 6. **OCI Registry** (`oci-registry/`)
|
|
|
|
OCI-compliant registry for extension distribution and versioning.
|
|
|
|
**Purpose**: Distributing and managing extensions
|
|
|
|
**Key Features**:
|
|
- Task service packages
|
|
- Provider packages
|
|
- Cluster templates
|
|
- Workflow definitions
|
|
- Version management and updates
|
|
- Dependency resolution
|
|
|
|
**Status**: 🔄 Planned
|
|
|
|
**Benefits**:
|
|
- Centralized extension management
|
|
- Version control and rollback
|
|
- Dependency tracking
|
|
- Community marketplace ready
|
|
|
|
---
|
|
|
|
### 7. **API Gateway** (`api-gateway/`)
|
|
|
|
Unified REST API gateway for external integration.
|
|
|
|
**Language**: Rust
|
|
|
|
**Purpose**: API routing, authentication, and rate limiting
|
|
|
|
**Key Features**:
|
|
- Request routing to backend services
|
|
- Authentication and authorization
|
|
- Rate limiting and throttling
|
|
- API versioning
|
|
- Request validation
|
|
- Metrics and monitoring
|
|
|
|
**Status**: 🔄 Planned
|
|
|
|
**Endpoints** (Planned):
|
|
- `/api/v1/servers/*` - Server management
|
|
- `/api/v1/taskservs/*` - Task service operations
|
|
- `/api/v1/clusters/*` - Cluster operations
|
|
- `/api/v1/workflows/*` - Workflow management
|
|
|
|
---
|
|
|
|
### 8. **Extension Registry** (`extension-registry/`)
|
|
|
|
Registry and catalog for browsing and discovering extensions.
|
|
|
|
**Purpose**: Extension discovery and metadata management
|
|
|
|
**Key Features**:
|
|
- Extension catalog
|
|
- Search and filtering
|
|
- Version history
|
|
- Dependency information
|
|
- Documentation links
|
|
- Community ratings (future)
|
|
|
|
**Status**: 🔄 Planned
|
|
|
|
---
|
|
|
|
### 9. **Provisioning Server** (`provisioning-server/`)
|
|
|
|
Alternative provisioning service implementation.
|
|
|
|
**Purpose**: Additional provisioning service capabilities
|
|
|
|
**Status**: 🔄 In Development
|
|
|
|
---
|
|
|
|
## Supporting Services
|
|
|
|
### CoreDNS (`coredns/`)
|
|
|
|
DNS service configuration for cluster environments.
|
|
|
|
**Purpose**: Service discovery and DNS resolution
|
|
|
|
**Status**: ✅ Configuration Ready
|
|
|
|
---
|
|
|
|
### Monitoring (`monitoring/`)
|
|
|
|
Observability and monitoring infrastructure.
|
|
|
|
**Purpose**: Metrics, logging, and alerting
|
|
|
|
**Components**:
|
|
- Prometheus configuration
|
|
- Grafana dashboards
|
|
- Alert rules
|
|
|
|
**Status**: ✅ Configuration Ready
|
|
|
|
---
|
|
|
|
### Nginx (`nginx/`)
|
|
|
|
Reverse proxy and load balancer configurations.
|
|
|
|
**Purpose**: HTTP routing and SSL termination
|
|
|
|
**Status**: ✅ Configuration Ready
|
|
|
|
---
|
|
|
|
### Docker Compose (`docker-compose/`)
|
|
|
|
Docker Compose configurations for local development.
|
|
|
|
**Purpose**: Quick local platform deployment
|
|
|
|
**Status**: ✅ Ready for Development
|
|
|
|
---
|
|
|
|
### Systemd (`systemd/`)
|
|
|
|
Systemd service units for platform services.
|
|
|
|
**Purpose**: Production deployment with systemd
|
|
|
|
**Status**: ✅ Ready for Production
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ User Interfaces │
|
|
│ • CLI (provisioning command) │
|
|
│ • Web UI (Control Center UI) │
|
|
│ • API Clients │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ API Gateway │
|
|
│ • Request Routing │
|
|
│ • Authentication & Authorization │
|
|
│ • Rate Limiting │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Platform Services Layer │
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ Orchestrator │ │Control Center│ │ MCP Server │ │
|
|
│ │ (Rust) │ │ (Rust) │ │ (Nushell) │ │
|
|
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ Installer │ │ OCI Registry │ │ Extension │ │
|
|
│ │(Rust/Nushell)│ │ │ │ Registry │ │
|
|
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Data & State Layer │
|
|
│ • SurrealDB (State Management) │
|
|
│ • File-based Persistence (Checkpoints) │
|
|
│ • Configuration Storage │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Technology Stack
|
|
|
|
### Primary Languages
|
|
|
|
| Language | Usage | Services |
|
|
|----------|-------|----------|
|
|
| **Rust** | Platform services, performance layer | Orchestrator, Control Center, Installer, API Gateway |
|
|
| **Nushell** | Scripting, automation, MCP integration | MCP Server, Installer scripts |
|
|
| **Web** | Frontend interfaces | Control Center UI |
|
|
|
|
### Key Dependencies
|
|
|
|
- **tokio** - Async runtime for Rust services
|
|
- **axum** / **actix-web** - Web frameworks
|
|
- **serde** - Serialization/deserialization
|
|
- **bollard** - Docker API client (test environments)
|
|
- **ratatui** - Terminal UI framework (installer)
|
|
- **SurrealDB** - State management database
|
|
|
|
---
|
|
|
|
## Deployment Modes
|
|
|
|
### 1. **Development Mode**
|
|
|
|
```bash
|
|
# Docker Compose for local development
|
|
docker-compose -f docker-compose/dev.yml up
|
|
```
|
|
|
|
### 2. **Production Mode (Systemd)**
|
|
|
|
```bash
|
|
# Install systemd units
|
|
sudo cp systemd/*.service /etc/systemd/system/
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl enable --now provisioning-orchestrator
|
|
sudo systemctl enable --now provisioning-control-center
|
|
```
|
|
|
|
### 3. **Kubernetes Deployment**
|
|
|
|
```bash
|
|
# Deploy platform services to Kubernetes
|
|
kubectl apply -f k8s/
|
|
```
|
|
|
|
---
|
|
|
|
## Security Features
|
|
|
|
### Enterprise Security Stack
|
|
|
|
1. **Authorization & Permissions** (Control Center)
|
|
- Role-Based Access Control (RBAC)
|
|
- Fine-grained permissions
|
|
- Audit logging
|
|
|
|
2. **Authentication**
|
|
- API key management
|
|
- Session management
|
|
- Token-based auth (JWT)
|
|
|
|
3. **Secrets Management**
|
|
- Integration with SOPS/Age
|
|
- Cosmian KMS support
|
|
- Secure configuration storage
|
|
|
|
4. **Policy Enforcement**
|
|
- Cedar policy engine integration
|
|
- Compliance checking
|
|
- Anomaly detection
|
|
|
|
---
|
|
|
|
## Getting Started
|
|
|
|
### Prerequisites
|
|
|
|
- **Rust** - Latest stable (for building platform services)
|
|
- **Nushell 0.107.1+** - For MCP server and scripts
|
|
- **Docker** (optional) - For containerized deployment
|
|
- **Kubernetes** (optional) - For K8s deployment
|
|
|
|
### Building Platform Services
|
|
|
|
```bash
|
|
# Build all Rust services
|
|
cd orchestrator && cargo build --release
|
|
cd ../control-center && cargo build --release
|
|
cd ../installer && cargo build --release
|
|
```
|
|
|
|
### Running Services
|
|
|
|
```bash
|
|
# Start orchestrator
|
|
cd orchestrator
|
|
./scripts/start-orchestrator.nu --background
|
|
|
|
# Start control center
|
|
cd control-center
|
|
cargo run --release
|
|
|
|
# Start MCP server
|
|
cd mcp-server
|
|
nu run.nu
|
|
```
|
|
|
|
---
|
|
|
|
## Development
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
platform/
|
|
├── orchestrator/ # Rust orchestrator service
|
|
├── control-center/ # Rust control center backend
|
|
├── control-center-ui/ # Web frontend
|
|
├── installer/ # Rust/Nushell installer
|
|
├── mcp-server/ # Nushell MCP server
|
|
├── api-gateway/ # Rust API gateway (planned)
|
|
├── oci-registry/ # OCI registry (planned)
|
|
├── extension-registry/ # Extension catalog (planned)
|
|
├── provisioning-server/# Alternative service
|
|
├── docker-compose/ # Docker Compose configs
|
|
├── k8s/ # Kubernetes manifests
|
|
├── systemd/ # Systemd units
|
|
└── docs/ # Platform documentation
|
|
```
|
|
|
|
### Adding New Services
|
|
|
|
1. Create service directory in `platform/`
|
|
2. Add README.md with service description
|
|
3. Implement service following architecture patterns
|
|
4. Add tests and documentation
|
|
5. Update platform/README.md (this file)
|
|
6. Add deployment configurations (docker-compose, k8s, systemd)
|
|
|
|
---
|
|
|
|
## Integration with [Provisioning](../../PROVISIONING.md)
|
|
|
|
Platform services integrate seamlessly with the [Provisioning](../../PROVISIONING.md) system:
|
|
|
|
- **Core Engine** (`../core/`) provides CLI and libraries
|
|
- **Extensions** (`../extensions/`) provide providers, taskservs, clusters
|
|
- **Platform Services** (this directory) provide execution and management
|
|
- **Configuration** (`../kcl/`, `../config/`) defines infrastructure
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
### Platform Documentation
|
|
|
|
- **Orchestrator**: [.claude/features/orchestrator-architecture.md](../../.claude/features/orchestrator-architecture.md)
|
|
- **Installer**: `installer/docs/` directory
|
|
- **Test Environments**: [.claude/features/test-environment-service.md](../../.claude/features/test-environment-service.md)
|
|
|
|
### API Documentation
|
|
|
|
- **REST API Reference**: `docs/api/` (when orchestrator is running)
|
|
- **MCP Tools Reference**: `mcp-server/docs/`
|
|
|
|
### Architecture Documentation
|
|
|
|
- **Main Project**: [PROVISIONING.md](../../PROVISIONING.md)
|
|
- **Project Architecture**: [CLAUDE.md](../../CLAUDE.md)
|
|
|
|
---
|
|
|
|
## Contributing
|
|
|
|
When contributing to platform services:
|
|
|
|
1. **Follow Rust Best Practices** - Idiomatic Rust, proper error handling
|
|
2. **Security First** - Always consider security implications
|
|
3. **Performance Matters** - Platform services are performance-critical
|
|
4. **Document APIs** - All REST endpoints must be documented
|
|
5. **Add Tests** - Unit tests and integration tests required
|
|
6. **Update Docs** - Keep README and API docs current
|
|
|
|
---
|
|
|
|
## Status Legend
|
|
|
|
- ✅ **Production Ready** - Fully implemented and tested
|
|
- ✅ **Active Development** - Working implementation, ongoing improvements
|
|
- ✅ **Configuration Ready** - Configuration files ready for deployment
|
|
- 🔄 **Planned** - Design phase, implementation pending
|
|
- 🔄 **In Development** - Early implementation stage
|
|
|
|
---
|
|
|
|
## Support
|
|
|
|
For platform service issues:
|
|
- Check service-specific README in service directory
|
|
- Review logs: `journalctl -u provisioning-*` (systemd)
|
|
- API documentation: `http://localhost:8080/docs` (when running)
|
|
- See [PROVISIONING.md](../../PROVISIONING.md) for general support
|
|
|
|
---
|
|
|
|
**Maintained By**: Platform Team
|
|
**Last Updated**: 2025-10-07
|
|
**Platform Version**: 3.5.0
|