chore: add schemas for auto-config
This commit is contained in:
Jesús Pérez
parent
22244bd21a
commit
b4eee821f5
GPG Key ID:
9F243E355E0BC939
@ -61,7 +61,7 @@ nickel export --format toml provisioning/schemas/infrastructure/systemd.ncl
|
||||
|
||||
Define service configuration once, generate multiple infrastructure outputs:
|
||||
|
||||
```
|
||||
```plaintext
|
||||
orchestrator.ncl (Platform Service Schema)
|
||||
↓
|
||||
Infrastructure Schemas (Docker, Kubernetes, Nginx, etc.)
|
||||
@ -123,7 +123,7 @@ nickel export --format yaml provisioning/schemas/infrastructure/kubernetes.ncl \
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
```plaintext
|
||||
infrastructure/
|
||||
├── README.md # This file
|
||||
├── docker-compose.ncl # Docker Compose schema (232 lines)
|
||||
@ -273,6 +273,7 @@ oci_registry_config # Zot backend with filesystem storage
|
||||
- Other services: 0.25-0.5 CPU, 256-512M RAM
|
||||
|
||||
**Export to JSON**:
|
||||
|
||||
```bash
|
||||
nickel export --format json provisioning/schemas/infrastructure/examples-solo-deployment.ncl
|
||||
# Output: 198 lines of configuration
|
||||
@ -298,6 +299,7 @@ oci_registry_config # Harbor backend with S3 replication
|
||||
- Services scale appropriately for production load
|
||||
|
||||
**Export to JSON**:
|
||||
|
||||
```bash
|
||||
nickel export --format json provisioning/schemas/infrastructure/examples-enterprise-deployment.ncl
|
||||
# Output: 313 lines of configuration
|
||||
@ -306,7 +308,7 @@ nickel export --format json provisioning/schemas/infrastructure/examples-enterpr
|
||||
### Example Comparison
|
||||
|
||||
| Aspect | Solo | Enterprise |
|
||||
|--------|------|-----------|
|
||||
| -------- | ------ | ----------- |
|
||||
| **Services** | 5 | 6 |
|
||||
| **Orchestrator CPU** | 1.0 | 4.0 |
|
||||
| **Orchestrator Memory** | 1024M | 4096M |
|
||||
@ -395,7 +397,7 @@ Infrastructure schemas are independent from platform config schemas:
|
||||
|
||||
ConfigLoader automatically loads platform configs. Infrastructure configs are generated separately and deployed via infrastructure tools:
|
||||
|
||||
```
|
||||
```plaintext
|
||||
Platform Schema (Nickel)
|
||||
↓ nickel export → TOML
|
||||
↓ ConfigLoader → Service reads config
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
# TypeDialog + Nickel Configuration System for Platform Services
|
||||
|
||||
Complete configuration system for provisioning platform services (orchestrator, control-center, mcp-server, vault-service, extension-registry, rag, ai-service, provisioning-daemon) across multiple deployment modes (solo, multiuser, cicd, enterprise).
|
||||
Complete configuration system for provisioning platform services (orchestrator, control-center, mcp-server, vault-service,
|
||||
extension-registry, rag, ai-service, provisioning-daemon) across multiple deployment modes (solo, multiuser, cicd, enterprise).
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
@ -15,7 +16,7 @@ This system implements a **TypeDialog + Nickel configuration workflow** that pro
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
```plaintext
|
||||
provisioning/.typedialog/provisioning/platform/
|
||||
├── constraints/ # Single source of truth for validation limits
|
||||
├── schemas/ # Nickel type contracts (services + common + deployment modes)
|
||||
@ -48,7 +49,7 @@ nu scripts/configure.nu orchestrator solo --backend web
|
||||
|
||||
### 2. Configuration Composition
|
||||
|
||||
```
|
||||
```plaintext
|
||||
Base Defaults (defaults/*.ncl)
|
||||
↓
|
||||
+ Mode Overlay (defaults/deployment/{mode}-defaults.ncl)
|
||||
@ -249,7 +250,7 @@ nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu so
|
||||
|
||||
## Configuration Loading Hierarchy (Rust Services)
|
||||
|
||||
```
|
||||
```plaintext
|
||||
1. Environment variables (ORCHESTRATOR_*)
|
||||
2. User config (values/{service}.{mode}.ncl → TOML)
|
||||
3. Mode-specific defaults (configs/{service}.{mode}.toml)
|
||||
@ -259,6 +260,7 @@ nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu so
|
||||
## Constraint Interpolation Example
|
||||
|
||||
**constraints.toml**:
|
||||
|
||||
```toml
|
||||
[orchestrator.queue.concurrent_tasks]
|
||||
min = 1
|
||||
@ -266,6 +268,7 @@ max = 100
|
||||
```
|
||||
|
||||
**Form element** (fragments/orchestrator-queue-section.toml):
|
||||
|
||||
```toml
|
||||
[[elements]]
|
||||
name = "max_concurrent_tasks"
|
||||
@ -276,6 +279,7 @@ nickel_path = ["orchestrator", "queue", "max_concurrent_tasks"]
|
||||
```
|
||||
|
||||
**Jinja2 template** (orchestrator-config.ncl.j2):
|
||||
|
||||
```nickel
|
||||
orchestrator = {
|
||||
queue = {
|
||||
@ -289,16 +293,19 @@ orchestrator = {
|
||||
## Getting Started
|
||||
|
||||
1. **Run configuration wizard**:
|
||||
|
||||
```bash
|
||||
nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo
|
||||
```
|
||||
|
||||
2. **Generate TOML configs**:
|
||||
|
||||
```bash
|
||||
nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo
|
||||
```
|
||||
|
||||
3. **Deploy services**:
|
||||
|
||||
```bash
|
||||
nu provisioning/.typedialog/provisioning/platform/scripts/install-services.nu solo
|
||||
```
|
||||
@ -318,7 +325,7 @@ orchestrator = {
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| ------ | --------- |
|
||||
| `constraints/constraints.toml` | Single source of truth for validation limits |
|
||||
| `schemas/orchestrator.ncl` | Orchestrator type schema |
|
||||
| `defaults/orchestrator-defaults.ncl` | Orchestrator default values |
|
||||
|
||||
@ -13,7 +13,7 @@ Configurations are **automatically generated** by composing:
|
||||
|
||||
## File Organization
|
||||
|
||||
```
|
||||
```plaintext
|
||||
configs/
|
||||
├── README.md # This file
|
||||
├── orchestrator.solo.ncl # Orchestrator solo mode
|
||||
@ -56,6 +56,7 @@ let validators = import "../validators/orchestrator-validator.ncl" in
|
||||
## Example Configuration
|
||||
|
||||
### Base Defaults
|
||||
|
||||
```nickel
|
||||
# defaults/orchestrator-defaults.ncl
|
||||
orchestrator = {
|
||||
@ -76,6 +77,7 @@ orchestrator = {
|
||||
```
|
||||
|
||||
### Solo Mode Override
|
||||
|
||||
```nickel
|
||||
# defaults/deployment/solo-defaults.ncl
|
||||
services.orchestrator = {
|
||||
@ -86,6 +88,7 @@ services.orchestrator = {
|
||||
```
|
||||
|
||||
### Generated Config
|
||||
|
||||
```nickel
|
||||
# configs/orchestrator.solo.ncl (auto-generated)
|
||||
{
|
||||
@ -119,6 +122,7 @@ services.orchestrator = {
|
||||
### Regenerating Configs
|
||||
|
||||
#### Via TypeDialog (Recommended)
|
||||
|
||||
```bash
|
||||
nu provisioning/.typedialog/provisioning/platform/scripts/configure.nu orchestrator solo
|
||||
```
|
||||
@ -130,6 +134,7 @@ Automatically:
|
||||
4. Generates updated config
|
||||
|
||||
#### Manual Rebuild
|
||||
|
||||
```bash
|
||||
# (Future) Script to rebuild all configs from sources
|
||||
nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu orchestrator solo
|
||||
@ -166,7 +171,7 @@ nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu or
|
||||
|
||||
## Configuration Values Hierarchy
|
||||
|
||||
```
|
||||
```plaintext
|
||||
1. Explicit user customization (values/{service}.{mode}.ncl)
|
||||
2. Mode-specific defaults (defaults/deployment/{mode}-defaults.ncl)
|
||||
3. Service base defaults (defaults/{service}-defaults.ncl)
|
||||
@ -179,12 +184,14 @@ Configurations are validated at three levels:
|
||||
|
||||
### 1. Schema Validation
|
||||
Type checking when config is evaluated:
|
||||
```
|
||||
|
||||
```plaintext
|
||||
| schemas.OrchestratorConfig
|
||||
```
|
||||
|
||||
### 2. Constraint Validation
|
||||
Range checking via validators:
|
||||
|
||||
```nickel
|
||||
max_concurrent_tasks = validators.ValidConcurrentTasks 5
|
||||
```
|
||||
@ -204,6 +211,7 @@ nu provisioning/.typedialog/provisioning/platform/scripts/generate-configs.nu or
|
||||
```
|
||||
|
||||
Rust services load the TOML:
|
||||
|
||||
```rust
|
||||
let config_path = "provisioning/platform/config/orchestrator.solo.toml";
|
||||
let config = Config::from_file(config_path)?;
|
||||
|
||||
@ -4,9 +4,10 @@ Complete documentation of the configuration pipeline that transforms interactive
|
||||
|
||||
## Overview
|
||||
|
||||
The provisioning platform uses a **four-stage configuration workflow** that leverages TypeDialog for interactive configuration, Nickel for type-safe composition, and TOML for service consumption:
|
||||
The provisioning platform uses a **four-stage configuration workflow** that leverages TypeDialog for interactive configuration,
|
||||
Nickel for type-safe composition, and TOML for service consumption:
|
||||
|
||||
```
|
||||
```plaintext
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Stage 1: User Interaction (TypeDialog) │
|
||||
│ - Can use Nickel configuration as default values │
|
||||
@ -419,7 +420,7 @@ level = "debug"
|
||||
|
||||
### Output Location
|
||||
|
||||
```
|
||||
```plaintext
|
||||
provisioning/platform/config/
|
||||
├── orchestrator.solo.toml # Exported from configs/orchestrator.solo.ncl
|
||||
├── orchestrator.multiuser.toml # Exported from configs/orchestrator.multiuser.ncl
|
||||
@ -801,7 +802,7 @@ ORCHESTRATOR_CONFIG=provisioning/platform/config/orchestrator.solo.toml cargo ru
|
||||
|
||||
### Component Interactions
|
||||
|
||||
```
|
||||
```plaintext
|
||||
TypeDialog Forms Nickel Schemas
|
||||
(forms/*.toml) ←shares→ (schemas/*.ncl)
|
||||
│ │
|
||||
|
||||
@ -16,7 +16,7 @@ These constraints are used by:
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
```plaintext
|
||||
constraints/
|
||||
└── constraints.toml # All validation constraints in TOML format
|
||||
```
|
||||
@ -26,6 +26,7 @@ constraints/
|
||||
### 1. Define Constraint
|
||||
|
||||
**constraints.toml**:
|
||||
|
||||
```toml
|
||||
[orchestrator.queue.concurrent_tasks]
|
||||
min = 1
|
||||
@ -35,6 +36,7 @@ max = 100
|
||||
### 2. Reference in Validator
|
||||
|
||||
**validators/orchestrator-validator.ncl**:
|
||||
|
||||
```nickel
|
||||
let constraints = import "../constraints/constraints.toml" in
|
||||
|
||||
@ -52,6 +54,7 @@ let constraints = import "../constraints/constraints.toml" in
|
||||
### 3. Reference in Form
|
||||
|
||||
**forms/fragments/orchestrator-queue-section.toml**:
|
||||
|
||||
```toml
|
||||
[[elements]]
|
||||
name = "max_concurrent_tasks"
|
||||
@ -107,6 +110,7 @@ When changing constraint bounds:
|
||||
### Example: Increase Max Queue Tasks
|
||||
|
||||
**Before**:
|
||||
|
||||
```toml
|
||||
[orchestrator.queue.concurrent_tasks]
|
||||
min = 1
|
||||
@ -114,6 +118,7 @@ max = 100
|
||||
```
|
||||
|
||||
**After**:
|
||||
|
||||
```toml
|
||||
[orchestrator.queue.concurrent_tasks]
|
||||
min = 1
|
||||
|
||||
@ -12,7 +12,7 @@ Defaults provide:
|
||||
|
||||
## File Organization
|
||||
|
||||
```
|
||||
```plaintext
|
||||
defaults/
|
||||
├── README.md # This file
|
||||
├── common/ # Shared defaults
|
||||
@ -36,7 +36,7 @@ defaults/
|
||||
|
||||
Configuration is built from layers:
|
||||
|
||||
```
|
||||
```plaintext
|
||||
Base Defaults (service-defaults.ncl)
|
||||
↓
|
||||
+ Mode Overlay (deployment/{mode}-defaults.ncl)
|
||||
@ -49,6 +49,7 @@ Base Defaults (service-defaults.ncl)
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```nickel
|
||||
# configs/orchestrator.solo.ncl
|
||||
let defaults = import "../defaults/orchestrator-defaults.ncl" in
|
||||
@ -68,6 +69,7 @@ let solo_defaults = import "../defaults/deployment/solo-defaults.ncl" in
|
||||
### 1. Service Base Defaults
|
||||
|
||||
**orchestrator-defaults.ncl**:
|
||||
|
||||
```nickel
|
||||
{
|
||||
orchestrator = {
|
||||
@ -97,6 +99,7 @@ let solo_defaults = import "../defaults/deployment/solo-defaults.ncl" in
|
||||
### 2. Mode-Specific Overrides
|
||||
|
||||
**deployment/solo-defaults.ncl**:
|
||||
|
||||
```nickel
|
||||
{
|
||||
resources = {
|
||||
@ -114,6 +117,7 @@ let solo_defaults = import "../defaults/deployment/solo-defaults.ncl" in
|
||||
```
|
||||
|
||||
**deployment/enterprise-defaults.ncl**:
|
||||
|
||||
```nickel
|
||||
{
|
||||
resources = {
|
||||
@ -134,6 +138,7 @@ let solo_defaults = import "../defaults/deployment/solo-defaults.ncl" in
|
||||
## Common Defaults
|
||||
|
||||
### server-defaults.ncl
|
||||
|
||||
```nickel
|
||||
{
|
||||
server = {
|
||||
@ -147,6 +152,7 @@ let solo_defaults = import "../defaults/deployment/solo-defaults.ncl" in
|
||||
```
|
||||
|
||||
### database-defaults.ncl
|
||||
|
||||
```nickel
|
||||
{
|
||||
database = {
|
||||
@ -159,6 +165,7 @@ let solo_defaults = import "../defaults/deployment/solo-defaults.ncl" in
|
||||
```
|
||||
|
||||
### security-defaults.ncl
|
||||
|
||||
```nickel
|
||||
{
|
||||
security = {
|
||||
@ -172,6 +179,7 @@ let solo_defaults = import "../defaults/deployment/solo-defaults.ncl" in
|
||||
```
|
||||
|
||||
### monitoring-defaults.ncl
|
||||
|
||||
```nickel
|
||||
{
|
||||
monitoring = {
|
||||
@ -218,6 +226,7 @@ let solo_defaults = import "../defaults/deployment/solo-defaults.ncl" in
|
||||
### Changing a Base Default
|
||||
|
||||
**orchestrator-defaults.ncl**:
|
||||
|
||||
```nickel
|
||||
# Before
|
||||
queue = {
|
||||
@ -238,6 +247,7 @@ queue = {
|
||||
### Changing Mode Override
|
||||
|
||||
**deployment/solo-defaults.ncl**:
|
||||
|
||||
```nickel
|
||||
# Before
|
||||
orchestrator = {
|
||||
|
||||
@ -68,6 +68,7 @@ nickel export --format toml full-platform-enterprise.ncl > platform.toml
|
||||
- No audit logging
|
||||
|
||||
**Usage**:
|
||||
|
||||
```bash
|
||||
# Export to TOML and run
|
||||
nickel export --format toml orchestrator-solo.ncl > orchestrator.solo.toml
|
||||
@ -78,6 +79,7 @@ nu ../../scripts/configure.nu orchestrator solo --backend cli
|
||||
```
|
||||
|
||||
**Customization Examples**:
|
||||
|
||||
```nickel
|
||||
# Increase concurrency for testing (still development-friendly)
|
||||
queue.max_concurrent_tasks = 5
|
||||
@ -141,12 +143,14 @@ workspace.path = "/path/to/my/workspace"
|
||||
- Network isolation and load balancing setup required
|
||||
|
||||
**Environment Variables Required**:
|
||||
|
||||
```bash
|
||||
export JWT_SECRET="<secure-random-256-bit-string>"
|
||||
export SURREALDB_PASSWORD="<database-password>"
|
||||
```
|
||||
|
||||
**Usage**:
|
||||
|
||||
```bash
|
||||
# Deploy standalone with SurrealDB
|
||||
nickel export --format toml orchestrator-enterprise.ncl > orchestrator.enterprise.toml
|
||||
@ -158,6 +162,7 @@ kubectl apply -f provisioning/platform/infrastructure/kubernetes/*.yaml
|
||||
```
|
||||
|
||||
**Customization Examples**:
|
||||
|
||||
```nickel
|
||||
# Adjust concurrency for your specific infrastructure
|
||||
queue.max_concurrent_tasks = 50 # Scale down if resource-constrained
|
||||
@ -222,12 +227,14 @@ security.rate_limit.requests_per_second = 20000
|
||||
- RBAC requires careful role definition
|
||||
|
||||
**Environment Variables Required**:
|
||||
|
||||
```bash
|
||||
export DB_PASSWORD="<postgresql-password>"
|
||||
export JWT_SECRET="<secure-random-string>"
|
||||
```
|
||||
|
||||
**Usage**:
|
||||
|
||||
```bash
|
||||
# Generate and deploy
|
||||
nickel export --format toml control-center-multiuser.ncl > control-center.multiuser.toml
|
||||
@ -242,14 +249,16 @@ docker-compose -f docker-compose.multiuser.yml up -d
|
||||
```
|
||||
|
||||
**RBAC Quick Reference**:
|
||||
|
||||
| Role | Intended Users | Key Permissions |
|
||||
|------|----------------|-----------------|
|
||||
| ------ | ---------------- | ----------------- |
|
||||
| admin | Infrastructure leads | All operations: full access |
|
||||
| operator | Operations engineers | Execute workflows, manage tasks, view policies |
|
||||
| developer | Application developers | View workflows, view policies (read-only) |
|
||||
| viewer | Non-technical (PM, QA) | View workflows only (minimal read) |
|
||||
|
||||
**Customization Examples**:
|
||||
|
||||
```nickel
|
||||
# Require MFA globally for higher security
|
||||
mfa.required = true
|
||||
@ -312,6 +321,7 @@ policies.password = {
|
||||
- 100 RPS rate limiting with 20-request burst
|
||||
|
||||
**Global Configuration**:
|
||||
|
||||
```nickel
|
||||
let deployment_mode = "enterprise"
|
||||
let namespace = "provisioning"
|
||||
@ -356,6 +366,7 @@ let environment = "production"
|
||||
- Network setup complexity (TLS, CORS, rate limiting)
|
||||
|
||||
**Environment Variables Required**:
|
||||
|
||||
```bash
|
||||
# Database credentials
|
||||
export DB_PASSWORD="<postgresql-password>"
|
||||
@ -375,7 +386,8 @@ export TLS_KEY="<key-path>"
|
||||
```
|
||||
|
||||
**Architecture Diagram**:
|
||||
```
|
||||
|
||||
```plaintext
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ Nginx Load Balancer (TLS, CORS, RateLimit) │
|
||||
│ https://orchestrator.example.com │
|
||||
@ -404,6 +416,7 @@ export TLS_KEY="<key-path>"
|
||||
```
|
||||
|
||||
**Usage**:
|
||||
|
||||
```bash
|
||||
# Export complete configuration
|
||||
nickel export --format toml full-platform-enterprise.ncl > platform.toml
|
||||
@ -424,6 +437,7 @@ docker-compose -f docker-compose.enterprise.yml up -d
|
||||
```
|
||||
|
||||
**Customization Examples**:
|
||||
|
||||
```nickel
|
||||
# Adjust deployment domain
|
||||
let domain = "my-company.com"
|
||||
@ -450,7 +464,7 @@ mcp_server.performance.cache_ttl = 7200
|
||||
## Deployment Mode Comparison Matrix
|
||||
|
||||
| Feature | Solo | MultiUser | Enterprise |
|
||||
|---------|------|-----------|-----------|
|
||||
| --------- | ------ | ----------- | ----------- |
|
||||
| **Ideal For** | Dev | Team/Staging | Production |
|
||||
| **Storage** | Filesystem | PostgreSQL | SurrealDB Cluster |
|
||||
| **Replicas** | 1 | 1 | 3+ (HA) |
|
||||
@ -615,6 +629,7 @@ ORCHESTRATOR_CONFIG=orchestrator.solo.toml timeout 5 cargo run --bin orchestrato
|
||||
### Port Already in Use
|
||||
|
||||
**Fix**: Change the port in your configuration:
|
||||
|
||||
```nickel
|
||||
server.port = 9999 # Instead of 9090
|
||||
```
|
||||
@ -630,6 +645,7 @@ server.port = 9999 # Instead of 9090
|
||||
### Import Not Found
|
||||
|
||||
**Fix**: Ensure all relative paths in imports are correct:
|
||||
|
||||
```nickel
|
||||
# Correct (relative to examples/)
|
||||
let defaults = import "../defaults/orchestrator-defaults.ncl" in
|
||||
@ -696,11 +712,13 @@ nickel export --format toml orchestrator-custom.ncl > orchestrator.custom.toml
|
||||
## Using Examples
|
||||
|
||||
### View Example
|
||||
|
||||
```bash
|
||||
cat provisioning/.typedialog/provisioning/platform/examples/orchestrator-solo.ncl
|
||||
```
|
||||
|
||||
### Copy and Customize
|
||||
|
||||
```bash
|
||||
# Start with solo example
|
||||
cp examples/orchestrator-solo.ncl values/orchestrator.solo.ncl
|
||||
@ -713,6 +731,7 @@ nu scripts/validate-config.nu values/orchestrator.solo.ncl
|
||||
```
|
||||
|
||||
### Generate from Example
|
||||
|
||||
```bash
|
||||
# Use example as base, regenerate with TypeDialog
|
||||
nu scripts/configure.nu orchestrator solo --backend web
|
||||
@ -769,11 +788,13 @@ Each example is a complete Nickel configuration:
|
||||
## Creating New Examples
|
||||
|
||||
### 1. Start with Existing Example
|
||||
|
||||
```bash
|
||||
cp examples/orchestrator-solo.ncl examples/orchestrator-custom.ncl
|
||||
```
|
||||
|
||||
### 2. Modify for Your Use Case
|
||||
|
||||
```toml
|
||||
# Update configuration values
|
||||
orchestrator.server.workers = 8 # More workers
|
||||
@ -781,6 +802,7 @@ orchestrator.queue.max_concurrent_tasks = 20 # Higher concurrency
|
||||
```
|
||||
|
||||
### 3. Validate Configuration
|
||||
|
||||
```bash
|
||||
nickel typecheck examples/orchestrator-custom.ncl
|
||||
nickel eval examples/orchestrator-custom.ncl
|
||||
@ -794,6 +816,7 @@ Add comments explaining:
|
||||
- Customization needed
|
||||
|
||||
### 5. Save as Reference
|
||||
|
||||
```bash
|
||||
mv examples/orchestrator-custom.ncl examples/orchestrator-{scenario}.ncl
|
||||
```
|
||||
|
||||
@ -12,7 +12,7 @@ Schemas define:
|
||||
|
||||
## File Organization
|
||||
|
||||
```
|
||||
```plaintext
|
||||
schemas/
|
||||
├── README.md # This file
|
||||
├── common/ # Shared schemas (server, database, security, etc.)
|
||||
@ -138,6 +138,7 @@ Log configuration:
|
||||
|
||||
### orchestrator.ncl
|
||||
Workflow orchestration:
|
||||
|
||||
```nickel
|
||||
OrchestratorConfig = {
|
||||
workspace | WorkspaceConfig,
|
||||
@ -153,6 +154,7 @@ OrchestratorConfig = {
|
||||
|
||||
### control-center.ncl
|
||||
Policy and RBAC:
|
||||
|
||||
```nickel
|
||||
ControlCenterConfig = {
|
||||
workspace | WorkspaceConfig,
|
||||
@ -166,6 +168,7 @@ ControlCenterConfig = {
|
||||
|
||||
### mcp-server.ncl
|
||||
MCP protocol server:
|
||||
|
||||
```nickel
|
||||
MCPServerConfig = {
|
||||
workspace | WorkspaceConfig,
|
||||
@ -186,6 +189,7 @@ Deployment schemas define resource constraints for each mode:
|
||||
- **enterprise.ncl** - 16+ CPU, 32+ GB RAM, HA
|
||||
|
||||
Example:
|
||||
|
||||
```nickel
|
||||
# schemas/deployment/solo.ncl
|
||||
{
|
||||
|
||||
@ -12,7 +12,7 @@ Templates provide:
|
||||
|
||||
## File Organization
|
||||
|
||||
```
|
||||
```plaintext
|
||||
templates/
|
||||
├── README.md # This file
|
||||
├── orchestrator-config.ncl.j2 # Nickel output template (Jinja2)
|
||||
@ -133,6 +133,7 @@ typedialog-web nickel-roundtrip "$CONFIG" "forms/{service}-form.toml" --output "
|
||||
### Key Jinja2 Patterns
|
||||
|
||||
**Conditional blocks** (only include if field is set):
|
||||
|
||||
```nickel
|
||||
{%- if workspace_name %}
|
||||
name = "{{ workspace_name }}",
|
||||
@ -140,6 +141,7 @@ name = "{{ workspace_name }}",
|
||||
```
|
||||
|
||||
**String values** (with quotes):
|
||||
|
||||
```nickel
|
||||
{%- if storage_backend %}
|
||||
backend = '{{ storage_backend }}, # Enum (atom syntax)
|
||||
@ -147,6 +149,7 @@ backend = '{{ storage_backend }}, # Enum (atom syntax)
|
||||
```
|
||||
|
||||
**Numeric values** (no quotes):
|
||||
|
||||
```nickel
|
||||
{%- if server_port %}
|
||||
port = {{ server_port }}, # Number
|
||||
@ -154,6 +157,7 @@ port = {{ server_port }}, # Number
|
||||
```
|
||||
|
||||
**Boolean values** (lower case):
|
||||
|
||||
```nickel
|
||||
{%- if workspace_enabled is defined %}
|
||||
enabled = {{ workspace_enabled | lower }}, # Boolean (true/false)
|
||||
@ -161,6 +165,7 @@ enabled = {{ workspace_enabled | lower }}, # Boolean (true/false)
|
||||
```
|
||||
|
||||
**Comments** (for generated files):
|
||||
|
||||
```nickel
|
||||
# Auto-generated by provisioning TypeDialog
|
||||
# Edit via: nu scripts/configure.nu orchestrator {mode}
|
||||
@ -295,16 +300,19 @@ let config = import "../../values/orchestrator.solo.ncl" in
|
||||
## Rendering Templates
|
||||
|
||||
### Render to JSON
|
||||
|
||||
```bash
|
||||
nickel export --format json templates/orchestrator-config.ncl.j2 > config.json
|
||||
```
|
||||
|
||||
### Render to YAML (via yq)
|
||||
|
||||
```bash
|
||||
nickel export --format json templates/kubernetes/orchestrator-deployment.yaml.ncl | yq -P > deployment.yaml
|
||||
```
|
||||
|
||||
### Render to TOML
|
||||
|
||||
```bash
|
||||
nickel export --format toml templates/configs/orchestrator-config.toml.ncl > config.toml
|
||||
```
|
||||
|
||||
@ -158,6 +158,7 @@ max_connections = 512
|
||||
### Database Configuration (Control Center)
|
||||
|
||||
**RocksDB** (solo, cicd modes):
|
||||
|
||||
```toml
|
||||
[database]
|
||||
backend = "rocksdb"
|
||||
@ -170,6 +171,7 @@ compression = "snappy"
|
||||
```
|
||||
|
||||
**PostgreSQL** (multiuser, enterprise modes):
|
||||
|
||||
```toml
|
||||
[database]
|
||||
backend = "postgres"
|
||||
@ -186,6 +188,7 @@ ssl_mode = "require"
|
||||
### Storage Configuration (Orchestrator)
|
||||
|
||||
**Filesystem** (solo, cicd modes):
|
||||
|
||||
```toml
|
||||
[storage]
|
||||
backend = "filesystem"
|
||||
@ -193,6 +196,7 @@ path = "/var/lib/provisioning/orchestrator/data"
|
||||
```
|
||||
|
||||
**SurrealDB Server** (multiuser mode):
|
||||
|
||||
```toml
|
||||
[storage]
|
||||
backend = "surrealdb_server"
|
||||
@ -202,6 +206,7 @@ surrealdb_database = "orchestrator"
|
||||
```
|
||||
|
||||
**SurrealDB Cluster** (enterprise mode):
|
||||
|
||||
```toml
|
||||
[storage]
|
||||
backend = "surrealdb_cluster"
|
||||
|
||||
@ -4,7 +4,8 @@ Nickel-based Docker Compose templates for deploying platform services across all
|
||||
|
||||
## Overview
|
||||
|
||||
This directory contains Nickel templates that generate Docker Compose files for different deployment scenarios. Each template imports configuration from `values/*.ncl` and expands to valid Docker Compose YAML.
|
||||
This directory contains Nickel templates that generate Docker Compose files for different deployment scenarios.
|
||||
Each template imports configuration from `values/*.ncl` and expands to valid Docker Compose YAML.
|
||||
|
||||
**Key Pattern**: Templates use **Nickel composition** to build service definitions dynamically based on configuration, allowing parameterized infrastructure-as-code.
|
||||
|
||||
@ -36,6 +37,7 @@ This directory contains Nickel templates that generate Docker Compose files for
|
||||
- Restart Policy: `unless-stopped` (survives host reboot)
|
||||
|
||||
**Usage**:
|
||||
|
||||
```bash
|
||||
# Generate from Nickel template
|
||||
nickel export --format json platform-stack.solo.yml.ncl | yq -P > docker-compose.solo.yml
|
||||
@ -51,6 +53,7 @@ docker-compose -f docker-compose.solo.yml down
|
||||
```
|
||||
|
||||
**Environment Variables** (recommended in `.env` file):
|
||||
|
||||
```bash
|
||||
ORCHESTRATOR_LOG_LEVEL=debug
|
||||
CONTROL_CENTER_LOG_LEVEL=info
|
||||
@ -96,6 +99,7 @@ MCP_SERVER_LOG_LEVEL=info
|
||||
- Logging: JSON format with rotation
|
||||
|
||||
**Usage**:
|
||||
|
||||
```bash
|
||||
# Generate from Nickel template
|
||||
nickel export --format json platform-stack.multiuser.yml.ncl | yq -P > docker-compose.multiuser.yml
|
||||
@ -118,6 +122,7 @@ docker-compose exec postgres psql -U postgres -c "CREATE DATABASE provisioning;"
|
||||
```
|
||||
|
||||
**Database Initialization**:
|
||||
|
||||
```bash
|
||||
# Connect to PostgreSQL for schema creation
|
||||
docker-compose exec postgres psql -U provisioning -d provisioning
|
||||
@ -130,6 +135,7 @@ docker-compose exec surrealdb surreal sql --auth root:password
|
||||
```
|
||||
|
||||
**Environment Variables** (in `.env.multiuser`):
|
||||
|
||||
```bash
|
||||
# Database Credentials (CRITICAL - change before production)
|
||||
DB_PASSWORD=your-strong-password
|
||||
@ -178,6 +184,7 @@ GITEA_ROOT_URL=http://localhost:3000/
|
||||
- **Low Resource**: Minimal memory/CPU footprint
|
||||
|
||||
**Usage**:
|
||||
|
||||
```bash
|
||||
# Generate from Nickel template
|
||||
nickel export --format json platform-stack.cicd.yml.ncl | yq -P > docker-compose.cicd.yml
|
||||
@ -197,6 +204,7 @@ docker-compose -f docker-compose.cicd.yml down -v
|
||||
```
|
||||
|
||||
**CI/CD Integration Example**:
|
||||
|
||||
```bash
|
||||
# GitHub Actions workflow
|
||||
- name: Start Provisioning Stack
|
||||
@ -213,6 +221,7 @@ docker-compose -f docker-compose.cicd.yml down -v
|
||||
```
|
||||
|
||||
**Environment Variables** (minimal):
|
||||
|
||||
```bash
|
||||
# Logging (optional)
|
||||
ORCHESTRATOR_LOG_LEVEL=warn
|
||||
@ -265,7 +274,8 @@ ORCHESTRATOR_LOG_LEVEL=warn
|
||||
- Logging: JSON format with 500MB files, kept 30 versions
|
||||
|
||||
**Architecture**:
|
||||
```
|
||||
|
||||
```plaintext
|
||||
┌──────────────────────┐
|
||||
│ External Client │
|
||||
│ (HTTPS, Port 443) │
|
||||
@ -299,6 +309,7 @@ Observability Stack:
|
||||
```
|
||||
|
||||
**Usage**:
|
||||
|
||||
```bash
|
||||
# Generate from Nickel template
|
||||
nickel export --format json platform-stack.enterprise.yml.ncl | yq -P > docker-compose.enterprise.yml
|
||||
@ -352,6 +363,7 @@ curl -H "Host: orchestrator.example.com" http://localhost/health
|
||||
- [ ] Configure external log aggregation (ELK stack, Splunk, etc.)
|
||||
|
||||
**Environment Variables** (in `.env.enterprise`):
|
||||
|
||||
```bash
|
||||
# Database Credentials (CRITICAL)
|
||||
DB_PASSWORD=your-strong-password-32-chars-min
|
||||
@ -534,6 +546,7 @@ docker logs -f $(docker ps | grep orchestrator | awk '{print $1}')
|
||||
**Error**: `bind: address already in use`
|
||||
|
||||
**Fix**: Change port in template or stop conflicting container:
|
||||
|
||||
```bash
|
||||
# Find process using port
|
||||
lsof -i :9090
|
||||
@ -549,6 +562,7 @@ ports:
|
||||
### Service Fails to Start
|
||||
|
||||
**Check logs**:
|
||||
|
||||
```bash
|
||||
docker-compose logs orchestrator
|
||||
```
|
||||
@ -563,6 +577,7 @@ docker-compose logs orchestrator
|
||||
### Persistent Volume Issues
|
||||
|
||||
**Clean volumes** (WARNING: Deletes data):
|
||||
|
||||
```bash
|
||||
docker-compose down -v
|
||||
docker volume prune -f
|
||||
|
||||
@ -221,6 +221,7 @@ kubectl describe service orchestrator -n provisioning
|
||||
### Required ConfigMaps
|
||||
|
||||
#### orchestrator-config
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
@ -235,6 +236,7 @@ data:
|
||||
```
|
||||
|
||||
#### control-center-config
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
@ -251,6 +253,7 @@ data:
|
||||
```
|
||||
|
||||
#### mcp-server-config
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
@ -272,6 +275,7 @@ data:
|
||||
### Required Secrets
|
||||
|
||||
#### control-center-secrets
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
|
||||
@ -101,7 +101,8 @@ ORCHESTRATOR_CONFIG=provisioning/platform/config/orchestrator.solo.toml cargo ru
|
||||
```
|
||||
|
||||
**Expected Output**:
|
||||
```
|
||||
|
||||
```plaintext
|
||||
[INFO] Orchestrator starting...
|
||||
[INFO] Server listening on 127.0.0.1:9090
|
||||
[INFO] Storage backend: filesystem
|
||||
@ -127,6 +128,7 @@ curl -X POST http://localhost:9090/api/workflows \
|
||||
To change configuration:
|
||||
|
||||
**Option A: Re-run Interactive Form**
|
||||
|
||||
```bash
|
||||
cd provisioning/.typedialog/provisioning/platform
|
||||
nu scripts/configure.nu orchestrator solo --backend cli
|
||||
@ -136,6 +138,7 @@ nu scripts/generate-configs.nu orchestrator solo
|
||||
```
|
||||
|
||||
**Option B: Edit TOML Directly**
|
||||
|
||||
```bash
|
||||
# Edit the file directly
|
||||
vi provisioning/platform/config/orchestrator.solo.toml
|
||||
@ -144,6 +147,7 @@ vi provisioning/platform/config/orchestrator.solo.toml
|
||||
```
|
||||
|
||||
**Option C: Environment Variable Override**
|
||||
|
||||
```bash
|
||||
# No file changes needed
|
||||
export ORCHESTRATOR_SERVER_PORT=9999
|
||||
@ -215,6 +219,7 @@ docker-compose -f provisioning/platform/infrastructure/docker/docker-compose.mul
|
||||
```
|
||||
|
||||
**Verify Services**:
|
||||
|
||||
```bash
|
||||
# Check all services are running
|
||||
docker-compose -f provisioning/platform/infrastructure/docker/docker-compose.multiuser.yml ps
|
||||
@ -466,6 +471,7 @@ kubectl apply -f provisioning/platform/infrastructure/kubernetes/prometheus-aler
|
||||
### Change Configuration Value
|
||||
|
||||
**Without Service Restart** (Environment Variable):
|
||||
|
||||
```bash
|
||||
# Override specific value via environment variable
|
||||
export ORCHESTRATOR_LOG_LEVEL=debug
|
||||
@ -476,6 +482,7 @@ ORCHESTRATOR_CONFIG=config.toml cargo run --bin orchestrator
|
||||
```
|
||||
|
||||
**With Service Restart** (TOML Edit):
|
||||
|
||||
```bash
|
||||
# Edit TOML directly
|
||||
vi provisioning/platform/config/orchestrator.solo.toml
|
||||
@ -486,6 +493,7 @@ ORCHESTRATOR_CONFIG=config.toml cargo run --bin orchestrator
|
||||
```
|
||||
|
||||
**With Validation** (Regenerate from Form):
|
||||
|
||||
```bash
|
||||
# Re-run interactive form to regenerate
|
||||
cd provisioning/.typedialog/provisioning/platform
|
||||
@ -500,6 +508,7 @@ nu scripts/generate-configs.nu orchestrator solo
|
||||
### Add Team Member
|
||||
|
||||
**In Kubernetes PostgreSQL**:
|
||||
|
||||
```bash
|
||||
kubectl exec -it -n provisioning deployment/postgres -- psql -U provisioning -d provisioning
|
||||
|
||||
@ -515,6 +524,7 @@ INSERT INTO role_assignments (user_id, role, granted_by, granted_at) VALUES
|
||||
### Scale Service Replicas
|
||||
|
||||
**In Kubernetes**:
|
||||
|
||||
```bash
|
||||
# Scale orchestrator from 3 to 5 replicas
|
||||
kubectl scale deployment orchestrator -n provisioning --replicas=5
|
||||
@ -578,7 +588,7 @@ ORCHESTRATOR_CONFIG=config.toml cargo run --bin orchestrator 2>&1 | grep -i "con
|
||||
|
||||
## Configuration File Locations
|
||||
|
||||
```
|
||||
```plaintext
|
||||
provisioning/.typedialog/provisioning/platform/
|
||||
├── forms/ # User-facing interactive forms
|
||||
│ ├── orchestrator-form.toml
|
||||
@ -628,7 +638,8 @@ provisioning/.typedialog/provisioning/platform/
|
||||
```
|
||||
|
||||
TOML output location:
|
||||
```
|
||||
|
||||
```plaintext
|
||||
provisioning/platform/config/
|
||||
├── orchestrator.solo.toml # Consumed by orchestrator service
|
||||
├── control-center.enterprise.toml # Consumed by control-center service
|
||||
|
||||
@ -12,7 +12,7 @@ Validators provide:
|
||||
|
||||
## File Organization
|
||||
|
||||
```
|
||||
```plaintext
|
||||
validators/
|
||||
├── README.md # This file
|
||||
├── common-validator.ncl # Ports, positive numbers, strings
|
||||
@ -94,6 +94,7 @@ ValidRange = fun min max value =>
|
||||
## Common Validators
|
||||
|
||||
### common-validator.ncl
|
||||
|
||||
```nickel
|
||||
let constraints = import "../constraints/constraints.toml" in
|
||||
|
||||
@ -123,6 +124,7 @@ let constraints = import "../constraints/constraints.toml" in
|
||||
```
|
||||
|
||||
### resource-validator.ncl
|
||||
|
||||
```nickel
|
||||
let constraints = import "../constraints/constraints.toml" in
|
||||
let common = import "./common-validator.ncl" in
|
||||
@ -143,6 +145,7 @@ let common = import "./common-validator.ncl" in
|
||||
## Service-Specific Validators
|
||||
|
||||
### orchestrator-validator.ncl
|
||||
|
||||
```nickel
|
||||
let constraints = import "../constraints/constraints.toml" in
|
||||
let common = import "./common-validator.ncl" in
|
||||
@ -178,6 +181,7 @@ let common = import "./common-validator.ncl" in
|
||||
```
|
||||
|
||||
### control-center-validator.ncl
|
||||
|
||||
```nickel
|
||||
{
|
||||
# JWT token expiration
|
||||
@ -195,6 +199,7 @@ let common = import "./common-validator.ncl" in
|
||||
```
|
||||
|
||||
### mcp-server-validator.ncl
|
||||
|
||||
```nickel
|
||||
{
|
||||
# Max concurrent tool executions
|
||||
@ -281,6 +286,7 @@ nickel export --format toml provisioning/.typedialog/provisioning/platform/confi
|
||||
## Adding a New Validator
|
||||
|
||||
1. **Create validator function** in appropriate file:
|
||||
|
||||
```nickel
|
||||
ValidMyValue = fun value =>
|
||||
if value < minimum then error "Too low"
|
||||
@ -289,6 +295,7 @@ nickel export --format toml provisioning/.typedialog/provisioning/platform/confi
|
||||
```
|
||||
|
||||
2. **Add constraint** to constraints.toml if needed:
|
||||
|
||||
```toml
|
||||
[service.feature.my_value]
|
||||
min = 1
|
||||
@ -296,11 +303,13 @@ nickel export --format toml provisioning/.typedialog/provisioning/platform/confi
|
||||
```
|
||||
|
||||
3. **Use in config**:
|
||||
|
||||
```nickel
|
||||
my_value = validators.ValidMyValue 50,
|
||||
```
|
||||
|
||||
4. **Add form constraint** (if interactive):
|
||||
|
||||
```toml
|
||||
[[elements]]
|
||||
name = "my_value"
|
||||
@ -309,6 +318,7 @@ nickel export --format toml provisioning/.typedialog/provisioning/platform/confi
|
||||
```
|
||||
|
||||
5. **Test**:
|
||||
|
||||
```bash
|
||||
nickel typecheck configs/service.mode.ncl
|
||||
```
|
||||
|
||||
@ -12,7 +12,7 @@ The values directory stores:
|
||||
|
||||
## File Organization
|
||||
|
||||
```
|
||||
```plaintext
|
||||
values/
|
||||
├── .gitignore # Ignore *.ncl user configs
|
||||
├── README.md # This file
|
||||
@ -44,6 +44,7 @@ Each config file (`{service}.{mode}.ncl`) is:
|
||||
- **Runtime data** - Created/updated by scripts and forms
|
||||
|
||||
Example:
|
||||
|
||||
```nickel
|
||||
# values/orchestrator.solo.ncl (auto-generated, user-editable)
|
||||
{
|
||||
@ -68,7 +69,7 @@ Example:
|
||||
|
||||
## .gitignore Pattern
|
||||
|
||||
```
|
||||
```plaintext
|
||||
# values/.gitignore
|
||||
*.ncl # Ignore all Nickel config files (user-specific)
|
||||
!*.example.ncl # EXCEPT example files (tracked for documentation)
|
||||
@ -150,6 +151,7 @@ This ensures:
|
||||
## Configuration Workflow
|
||||
|
||||
### 1. Generate Initial Config
|
||||
|
||||
```bash
|
||||
nu scripts/configure.nu orchestrator solo
|
||||
```
|
||||
@ -157,6 +159,7 @@ nu scripts/configure.nu orchestrator solo
|
||||
Creates `values/orchestrator.solo.ncl` from form input.
|
||||
|
||||
### 2. Edit Configuration
|
||||
|
||||
```bash
|
||||
# Manually edit if needed
|
||||
vi values/orchestrator.solo.ncl
|
||||
@ -166,11 +169,13 @@ nu scripts/configure.nu orchestrator solo --backend web
|
||||
```
|
||||
|
||||
### 3. Validate Configuration
|
||||
|
||||
```bash
|
||||
nu scripts/validate-config.nu values/orchestrator.solo.ncl
|
||||
```
|
||||
|
||||
### 4. Generate TOML for Services
|
||||
|
||||
```bash
|
||||
nu scripts/generate-configs.nu orchestrator solo
|
||||
```
|
||||
@ -181,7 +186,7 @@ Exports to `provisioning/platform/config/orchestrator.solo.toml` (consumed by Ru
|
||||
|
||||
User configs are composed with defaults during generation:
|
||||
|
||||
```
|
||||
```plaintext
|
||||
defaults/orchestrator-defaults.ncl (base values)
|
||||
↓ &
|
||||
values/orchestrator.solo.ncl (user customizations)
|
||||
@ -205,6 +210,7 @@ provisioning/platform/config/orchestrator.solo.toml (Rust service config)
|
||||
Since user configs are gitignored, sharing requires:
|
||||
|
||||
### Option 1: Share via File
|
||||
|
||||
```bash
|
||||
# Export current config
|
||||
cat values/orchestrator.solo.ncl > /tmp/orchestrator-config.ncl
|
||||
@ -215,6 +221,7 @@ cp /tmp/orchestrator-config.ncl values/orchestrator.solo.ncl
|
||||
|
||||
### Option 2: Use Example Template
|
||||
Share setup instructions instead of raw config:
|
||||
|
||||
```bash
|
||||
# Document the setup steps
|
||||
cat > SETUP.md << EOF
|
||||
@ -227,6 +234,7 @@ EOF
|
||||
|
||||
### Option 3: Store in Separate Repo
|
||||
For team configs, use a separate private repository:
|
||||
|
||||
```bash
|
||||
# Clone team configs
|
||||
git clone private-repo/provisioning-configs values/
|
||||
@ -249,16 +257,19 @@ chmod 600 values/orchestrator.solo.ncl
|
||||
If you accidentally delete a user config:
|
||||
|
||||
### Option 1: Regenerate from TypeDialog
|
||||
|
||||
```bash
|
||||
nu scripts/configure.nu orchestrator solo
|
||||
```
|
||||
|
||||
### Option 2: Copy from Backup
|
||||
|
||||
```bash
|
||||
cp /backup/provisioning-values/orchestrator.solo.ncl values/
|
||||
```
|
||||
|
||||
### Option 3: Use Example as Base
|
||||
|
||||
```bash
|
||||
cp examples/orchestrator-solo.ncl values/orchestrator.solo.ncl
|
||||
# Customize as needed
|
||||
@ -268,12 +279,14 @@ nu scripts/configure.nu orchestrator solo --backend web
|
||||
## Troubleshooting
|
||||
|
||||
### Config File Missing
|
||||
|
||||
```bash
|
||||
# Regenerate from defaults
|
||||
nu scripts/configure.nu orchestrator solo
|
||||
```
|
||||
|
||||
### Config Won't Validate
|
||||
|
||||
```bash
|
||||
# Check for syntax errors
|
||||
nickel eval values/orchestrator.solo.ncl
|
||||
@ -283,6 +296,7 @@ diff examples/orchestrator-solo.ncl values/orchestrator.solo.ncl
|
||||
```
|
||||
|
||||
### Changes Not Taking Effect
|
||||
|
||||
```bash
|
||||
# Regenerate TOML from Nickel
|
||||
nu scripts/generate-configs.nu orchestrator solo
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user