Vapora/provisioning/README.md
Jesús Pérez a395bd972f
Some checks failed
Rust CI / Security Audit (push) Has been cancelled
Rust CI / Check + Test + Lint (nightly) (push) Has been cancelled
Rust CI / Check + Test + Lint (stable) (push) Has been cancelled
mdBook Build & Deploy / Build mdBook (push) Has been cancelled
Nickel Type Check / Nickel Type Checking (push) Has been cancelled
mdBook Build & Deploy / Documentation Quality Check (push) Has been cancelled
mdBook Build & Deploy / Deploy to GitHub Pages (push) Has been cancelled
mdBook Build & Deploy / Notification (push) Has been cancelled
chore: add cd/ci ops
2026-01-12 03:36:55 +00:00

517 lines
13 KiB
Markdown

# VAPORA Provisioning Configuration
Complete configuration system for deploying VAPORA using **typedialog** (interactive forms) and **nickel** (configuration generation).
## Quick Start
### Generate Configuration via Interactive Form
```bash
# Start interactive setup wizard
typedialog \
--form .typedialog/vapora/forms/vapora-main-form.toml \
--output config/runtime/vapora.custom.toml
```
This generates a customized TOML configuration based on your answers.
### Use Predefined Deployment Profiles
```bash
# Copy example for your deployment mode
cp config/examples/vapora.solo.example.toml config/runtime/vapora.toml
# Or use Nickel to generate
nickel export config/examples/vapora.solo.example.ncl > config/runtime/vapora.json
```
## Directory Structure
```plaintext
provisioning/
├── config/
│ ├── examples/ # Reference configurations for all modes
│ │ ├── vapora.solo.example.toml
│ │ ├── vapora.solo.example.ncl
│ │ ├── vapora.multiuser.example.toml
│ │ ├── vapora.multiuser.example.ncl
│ │ ├── vapora.enterprise.example.toml
│ │ └── vapora.enterprise.example.ncl
│ └── runtime/ # Active configuration (generate or copy here)
│ └── .gitkeep
├── schemas/
│ ├── vapora/ # VAPORA service schemas
│ │ ├── main.ncl # Main unified configuration
│ │ ├── backend.ncl # Backend (Axum REST API)
│ │ ├── agents.ncl # Agents with learning profiles
│ │ └── llm-router.ncl # LLM Router with cost tracking
│ │
│ └── platform/
│ ├── common/
│ │ └── helpers.ncl # Configuration composition utilities
│ └── defaults/
│ └── deployment/
│ ├── solo.ncl # Solo mode (dev)
│ ├── multiuser.ncl # Multiuser (team)
│ └── enterprise.ncl # Enterprise (production)
└── .typedialog/
└── vapora/
└── forms/
├── vapora-main-form.toml # Main form with all settings
└── fragments/ # Modular form fragments
├── backend/
│ └── auth.toml # Auth config fragment
├── agents/
│ └── learning-profiles.toml # Learning & KG config
├── llm-router/
│ └── budget-enforcement.toml # Budget config
└── frontend/
```
## Deployment Modes
### 1. Solo (Development)
Local development with minimal resources:
- **CPU**: 2 cores
- **Memory**: 2GB
- **Storage**: /tmp/vapora (ephemeral)
- **Database**: Local file-based SurrealDB
- **Coordination**: No NATS (single process)
- **Cost tracking**: Disabled
- **Security**: JWT only, no TLS
**Use for:**
- Local development
- Testing features
- PoC deployments
- Single-user testing
**Generate:**
```bash
cp config/examples/vapora.solo.example.toml config/runtime/vapora.toml
typedialog --form .typedialog/vapora/forms/vapora-main-form.toml --output config/runtime/vapora.toml
```
### 2. Multiuser (Team)
Team collaboration with shared infrastructure:
- **CPU**: 4-8 cores
- **Memory**: 8-16GB
- **Storage**: /var/lib/vapora (persistent)
- **Database**: Remote SurrealDB (WS protocol)
- **Coordination**: NATS JetStream cluster
- **Cost tracking**: Enabled (per-role budgets)
- **Security**: TLS, MFA, audit logging
**Features:**
- Multi-tenant support with workspaces
- Learning profiles for agent improvement
- Cost optimization with budget enforcement
- Swarm coordination for balanced task distribution
- Knowledge graph retention: 30 days
**Use for:**
- Team development environments
- Staging deployments
- Department-scale rollouts
- Cost-controlled production (small teams)
**Generate:**
```bash
cp config/examples/vapora.multiuser.example.toml config/runtime/vapora.toml
# Edit as needed for your infrastructure
```
### 3. Enterprise (Production)
Large-scale production with HA and observability:
- **CPU**: 16+ cores (distributed)
- **Memory**: 32GB+ (distributed)
- **Storage**: High-availability persistent storage
- **Database**: SurrealDB cluster with replication
- **Coordination**: NATS JetStream cluster
- **Cost tracking**: Aggressive (detailed per-token)
- **Security**: Full TLS, MFA, audit logging, RBAC
- **Observability**: Prometheus metrics, OpenTelemetry tracing
**Features:**
- Multi-region deployment support
- All LLM providers enabled (Claude, OpenAI, Gemini, Ollama)
- Aggressive cost optimization with multi-provider fallback
- 90-day knowledge retention for enterprise learning
- Enterprise-grade backup strategy (6-hour intervals)
- Full distributed tracing and metrics
**Use for:**
- Production deployments (any scale)
- Multi-region rollouts
- Enterprise customers
- Mission-critical systems
**Generate:**
```bash
cp config/examples/vapora.enterprise.example.toml config/runtime/vapora.toml
# Customize for your infrastructure (TLS certs, domains, etc.)
```
## Configuration Layers
### 1. Schema Layer (`.schemas/`)
Defines the structure and types for all configurations:
- **Main schema** (`vapora/main.ncl`) - Unified service configuration
- **Service schemas** - Backend, Agents, LLM Router specifics
- **Deployment schemas** - Mode-specific defaults (solo, multiuser, enterprise)
**Example:**
```nickel
# Backend schema defines structure
{
host | String | doc "Bind address" | default = "0.0.0.0",
port | Number | doc "Port" | default = 8001,
workers | Number | doc "Worker threads" | default = 4,
# ... more fields
}
```
### 2. Form Layer (`.typedialog/`)
Interactive forms for configuration generation:
- **Main form** - Complete VAPORA setup wizard
- **Fragment forms** - Modular forms for specific features (auth, budgets, learning)
**Example:**
```toml
[[elements]]
name = "backend_port"
nickel_path = ["vapora", "backend", "port"]
prompt = "Backend Port"
default = 8001
type = "number"
```
### 3. Configuration Layer (`config/`)
Generated or manually-customized configurations:
- **Examples** - Reference configs for all modes (TOML + Nickel)
- **Runtime** - Active configurations (generated from forms or copied from examples)
## Key Configuration Concepts
### Cost-Aware LLM Routing
Budget enforcement per role with automatic fallback:
```toml
[llm_router.budget_enforcement]
enabled = true
window = "monthly"
[llm_router.budget_enforcement.role_limits]
architect_cents = 500000 # $5000/month
developer_cents = 300000 # $3000/month
reviewer_cents = 200000 # $2000/month
testing_cents = 100000 # $1000/month
```
When budget is exceeded:
1. Alert threshold triggered (80% default)
2. Automatic fallback to cheaper provider
3. Cost report generated
4. Manual intervention available
### Learning-Based Agent Selection
Agents improve with execution history:
```toml
[agents.learning]
enabled = true
recency_window_days = 7 # Weight recent tasks 3x higher
recency_multiplier = 3.0
[agents.learning.scoring]
load_weight = 0.3 # 30% on agent load
expertise_weight = 0.5 # 50% on expertise profile
confidence_weight = 0.2 # 20% confidence (prevents overfitting)
```
### Knowledge Graph
Temporal execution history with learning curves:
```toml
[agents.knowledge_graph]
enabled = true
retention_days = 90 # Keep 90 days of history
causal_reasoning = true # Understand task relationships
similarity_search = true # Recommend past solutions
```
## Customization Guide
### Modify Backend Settings
Edit `schemas/vapora/backend.ncl`:
```nickel
# Change default port
backend = {
port | Number | default = 9001,
# ... other fields
}
```
Or in `config/runtime/vapora.toml`:
```toml
[backend]
port = 9001
```
### Add New Service
1. Create schema: `schemas/vapora/newservice.ncl`
2. Add to main schema: `schemas/vapora/main.ncl`
3. Create form: `.typedialog/vapora/forms/fragments/newservice/`
4. Update examples in `config/examples/`
### Override Mode Defaults
Use Nickel composition in `config/runtime/vapora.custom.ncl`:
```nickel
let defaults = import "../../schemas/vapora/main.ncl" in
let mode = import "../../schemas/platform/defaults/deployment/enterprise.ncl" in
let customizations = {
backend.port = 9001,
llm_router.budget_enforcement.window = "weekly",
} in
std.record.merge defaults (std.record.merge mode customizations)
```
## Deployment
### Via Docker Compose
Use generated config with docker-compose:
```bash
# Generate config
cp config/examples/vapora.multiuser.example.toml config/runtime/vapora.toml
# Start services (requires docker-compose.yml that reads this config)
docker compose up -d
```
### Via Kubernetes
Convert Nickel to Kubernetes manifests:
```bash
# Export config as JSON
nickel export config/runtime/vapora.multiuser.ncl > config/runtime/vapora.json
# Use ConfigMap in K8s
kubectl create configmap vapora-config --from-file=vapora.json
```
### Via Provisioning Script
Use Nushell scripts to apply configuration:
```bash
# Read config and validate
nu scripts/deploy-vapora.nu \
--config config/runtime/vapora.toml \
--mode multiuser
```
## Validation
### Validate Nickel Configuration
```bash
# Type check
nickel typecheck config/runtime/vapora.custom.ncl
# Export to JSON
nickel export config/runtime/vapora.custom.ncl > vapora.json
# Validate JSON structure
jq . vapora.json
```
### Validate TOML Configuration
```bash
# Use toml-cli or similar
toml-cli validate config/runtime/vapora.toml
# Or via Rust
cargo build -p vapora-backend --features config-validation
```
### Test Configuration
```bash
# Dry-run backend with config
cd ../../crates/vapora-backend
cargo run --features dry-run -- --config ../../provisioning/config/runtime/vapora.toml
```
## Environment Variables
Override configuration values with environment variables:
```bash
# Backend
export VAPORA_BACKEND_PORT=9001
export VAPORA_BACKEND_WORKERS=8
# Database
export SURREAL_URL=ws://surrealdb:8000
export SURREAL_USER=root
export SURREAL_PASS=secret
# Agents
export VAPORA_AGENTS_MAX_INSTANCES=20
# LLM Router
export VAPORA_ROUTER_BUDGET_ENABLED=true
# Providers
export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...
```
## Security Considerations
### Solo Mode
- ⚠️ No TLS (HTTP only)
- ⚠️ No MFA
- ⚠️ Local storage (not backed up)
- **Use only for local development**
### Multiuser Mode
- ✅ TLS enabled
- ✅ MFA available
- ✅ Audit logging
- ✅ JWT tokens with 1-hour TTL
- **Suitable for internal teams**
### Enterprise Mode
- ✅ Enforced TLS
- ✅ MFA required
- ✅ Full audit logging
- ✅ JWT + refresh tokens
- ✅ RBAC-ready (integrates with Cedar)
- ✅ Encrypted secrets in transit
- **Production-ready**
## Troubleshooting
### Configuration Not Applied
1. Check file is in `config/runtime/`
2. Verify TOML syntax: `toml-cli validate vapora.toml`
3. Check environment variables aren't overriding
4. Restart services after config changes
### Port Already in Use
Edit configuration:
```toml
[backend]
port = 9001 # Change from 8001
```
### Database Connection Timeout
Check URL and connectivity:
```bash
# Test SurrealDB
curl -i http://localhost:8000/health
# Update config
[database]
url = "ws://surrealdb.example.com:8000"
```
### Cost Tracking Not Working
Ensure provider credentials are set:
```bash
export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...
```
## Advanced Topics
### Custom Scoring Formula
Modify learning profile weights in agents schema:
```nickel
[agents.learning.scoring]
load_weight = 0.2 # Reduce load importance
expertise_weight = 0.6 # Increase expertise importance
confidence_weight = 0.2
```
### Multi-Region Deployment
Create regional config:
```nickel
let defaults = import "../../schemas/vapora/main.ncl" in
let enterprise = import "../../schemas/platform/defaults/deployment/enterprise.ncl" in
{
..enterprise,
frontend.api_url = "https://us-west.vapora.production",
database.url = "ws://surrealdb-us-west.internal:8000",
}
```
### Budget Alerts and Actions
Define custom budget thresholds in `llm_router`:
```toml
[llm_router.budget_enforcement]
near_threshold_percent = 70 # Alert at 70%
auto_fallback = true # Auto-fallback to cheaper
```
## References
- [VAPORA Architecture](../../docs/architecture.md)
- [Nickel Language](https://nickel-lang.org/)
- [typedialog Documentation](https://github.com/typedoc/typedialog)
- [SurrealDB Configuration](https://surrealdb.com/docs/deployment)
- [NATS JetStream](https://docs.nats.io/nats-concepts/jetstream)
---
**Generated**: January 12, 2026
**VAPORA Version**: 1.2.0
**Last Updated**: January 12, 2026