Vapora/docs/setup/setup-guide.md
Jesús Pérez d14150da75 feat: Phase 5.3 - Multi-Agent Learning Infrastructure
Implement intelligent agent learning from Knowledge Graph execution history
with per-task-type expertise tracking, recency bias, and learning curves.

## Phase 5.3 Implementation

### Learning Infrastructure ( Complete)
- LearningProfileService with per-task-type expertise metrics
- TaskTypeExpertise model tracking success_rate, confidence, learning curves
- Recency bias weighting: recent 7 days weighted 3x higher (exponential decay)
- Confidence scoring prevents overfitting: min(1.0, executions / 20)
- Learning curves computed from daily execution windows

### Agent Scoring Service ( Complete)
- Unified AgentScore combining SwarmCoordinator + learning profiles
- Scoring formula: 0.3*base + 0.5*expertise + 0.2*confidence
- Rank agents by combined score for intelligent assignment
- Support for recency-biased scoring (recent_success_rate)
- Methods: rank_agents, select_best, rank_agents_with_recency

### KG Integration ( Complete)
- KGPersistence::get_executions_for_task_type() - query by agent + task type
- KGPersistence::get_agent_executions() - all executions for agent
- Coordinator::load_learning_profile_from_kg() - core KG→Learning integration
- Coordinator::load_all_learning_profiles() - batch load for multiple agents
- Convert PersistedExecution → ExecutionData for learning calculations

### Agent Assignment Integration ( Complete)
- AgentCoordinator uses learning profiles for task assignment
- extract_task_type() infers task type from title/description
- assign_task() scores candidates using AgentScoringService
- Fallback to load-based selection if no learning data available
- Learning profiles stored in coordinator.learning_profiles RwLock

### Profile Adapter Enhancements ( Complete)
- create_learning_profile() - initialize empty profiles
- add_task_type_expertise() - set task-type expertise
- update_profile_with_learning() - update swarm profiles from learning

## Files Modified

### vapora-knowledge-graph/src/persistence.rs (+30 lines)
- get_executions_for_task_type(agent_id, task_type, limit)
- get_agent_executions(agent_id, limit)

### vapora-agents/src/coordinator.rs (+100 lines)
- load_learning_profile_from_kg() - core KG integration method
- load_all_learning_profiles() - batch loading for agents
- assign_task() already uses learning-based scoring via AgentScoringService

### Existing Complete Implementation
- vapora-knowledge-graph/src/learning.rs - calculation functions
- vapora-agents/src/learning_profile.rs - data structures and expertise
- vapora-agents/src/scoring.rs - unified scoring service
- vapora-agents/src/profile_adapter.rs - adapter methods

## Tests Passing
- learning_profile: 7 tests 
- scoring: 5 tests 
- profile_adapter: 6 tests 
- coordinator: learning-specific tests 

## Data Flow
1. Task arrives → AgentCoordinator::assign_task()
2. Extract task_type from description
3. Query KG for task-type executions (load_learning_profile_from_kg)
4. Calculate expertise with recency bias
5. Score candidates (SwarmCoordinator + learning)
6. Assign to top-scored agent
7. Execution result → KG → Update learning profiles

## Key Design Decisions
 Recency bias: 7-day half-life with 3x weight for recent performance
 Confidence scoring: min(1.0, total_executions / 20) prevents overfitting
 Hierarchical scoring: 30% base load, 50% expertise, 20% confidence
 KG query limit: 100 recent executions per task-type for performance
 Async loading: load_learning_profile_from_kg supports concurrent loads

## Next: Phase 5.4 - Cost Optimization
Ready to implement budget enforcement and cost-aware provider selection.
2026-01-11 13:03:53 +00:00

802 lines
16 KiB
Markdown

---
title: Vapora Project - Complete Setup Guide
date: 2025-11-10
version: 1.0
status: READY
---
# 🛠️ Vapora - Complete Setup Guide
**Complete step-by-step guide for setting up the entire Vapora project from scratch.**
---
## 📋 Table of Contents
1. [Prerequisites & Environment](#prerequisites--environment)
2. [Installation](#installation)
3. [Configuration](#configuration)
4. [Building & Testing](#building--testing)
5. [Development Setup](#development-setup)
6. [First Run](#first-run)
7. [Troubleshooting](#troubleshooting)
---
## Prerequisites & Environment
### System Requirements
| Requirement | Minimum | Recommended |
|------------|---------|-------------|
| OS | macOS 10.15+ | macOS 12+ (M1/M2 optimized) |
| CPU | 2-core | 4+ cores |
| RAM | 4GB | 8GB+ |
| Disk | 2GB | 5GB+ |
| Internet | Required | Required |
### Software Requirements
**Required:**
- Rust 1.75+ (install from https://rustup.rs)
- Cargo (comes with Rust)
- Git 2.20+
- NuShell 0.95+ (for scripts)
**Optional but Recommended:**
- Node.js 18+ (for frontend tooling)
- Docker (for containerization)
- Kubernetes tools (kubectl, k3s for deployment)
### Prerequisite Check Script
```bash
#!/bin/bash
echo "🔍 Checking Vapora prerequisites..."
echo "=================================="
# Check Rust
if ! command -v rustc &> /dev/null; then
echo "❌ Rust not found. Install from https://rustup.rs"
exit 1
fi
echo "✅ Rust $(rustc --version | awk '{print $2}')"
# Check Cargo
if ! command -v cargo &> /dev/null; then
echo "❌ Cargo not found"
exit 1
fi
echo "✅ Cargo $(cargo --version | awk '{print $2}')"
# Check Git
if ! command -v git &> /dev/null; then
echo "❌ Git not found. Install from https://git-scm.com"
exit 1
fi
echo "✅ Git $(git --version | awk '{print $3}')"
# Check NuShell (optional)
if command -v nu &> /dev/null; then
echo "✅ NuShell $(nu --version)"
else
echo "⚠️ NuShell not found (optional, needed for scripts)"
fi
echo "=================================="
echo "✅ All prerequisites satisfied!"
```
Save as `check-prerequisites.sh` and run:
```bash
chmod +x check-prerequisites.sh
./check-prerequisites.sh
```
---
## Installation
### Step 1: Prepare Your Environment
```bash
# Update Rust (if already installed)
rustup update stable
rustup component add rustfmt clippy
# Clone or verify Vapora repo
if [ ! -d vapora ]; then
git clone https://github.com/vapora/vapora.git
fi
cd vapora
```
### Step 2: Install NuShell (if needed)
```bash
# macOS with Homebrew
brew install nu
# Or from source (if on other OS)
# See https://www.nushell.sh/book/installation.html
```
Verify:
```bash
nu --version # Should be 0.95+
```
### Step 3: Install Frontend Tools (if building frontend)
```bash
# Install trunk for Leptos WASM building
cargo install trunk
# Install wasm-pack for WASM compilation
rustup target add wasm32-unknown-unknown
```
### Step 4: Download Workspace Dependencies
```bash
# Download all dependencies (no compilation yet)
cargo fetch
# This may take 2-3 minutes depending on internet speed
```
### Step 5: Verify Workspace Structure
```bash
# Verify all crates are present
ls -la crates/
# Expected output (8 crates):
# vapora-agents/
# vapora-backend/
# vapora-doc-lifecycle/
# vapora-frontend/
# vapora-llm-router/
# vapora-mcp-server/
# vapora-shared/
# vapora-tracking/
```
---
## Configuration
### Option 1: Default Configuration (Recommended)
The project works out of the box with sensible defaults:
```yaml
# Default Settings
Backend:
port: 3000
host: 127.0.0.1
env: development
Database:
type: SQLite (local)
path: ~/.vapora/data.db
Tracking:
database: ~/.tracking/database.sqlite
watch_dirs:
- .coder/
- ~/.claude/todos/
debounce_ms: 500
LLM Router:
default_providers:
- claude3-opus
- gpt-4
- gemini-2-pro
fallback_enabled: true
Frontend:
port: 8080
hot_reload: true
```
**No configuration needed to start developing!** Skip to [Building & Testing](#building--testing).
### Option 2: Environment Variables
Create `.env` file in project root:
```bash
# Backend Configuration
VAPORA_PORT=3000
VAPORA_HOST=127.0.0.1
RUST_ENV=development
RUST_LOG=debug
# Database
VAPORA_DATABASE_URL=sqlite:///Users/Akasha/.vapora/data.db
DATABASE_MAX_CONNECTIONS=5
# Tracking System
TRACKING_DATABASE_URL=sqlite:///Users/Akasha/.tracking/database.sqlite
TRACKING_API_PORT=3000
TRACKING_WATCH_DIRS=/Users/Akasha/.coder,/Users/Akasha/.claude/todos
TRACKING_DEBOUNCE_MS=500
# LLM Configuration
LLM_DEFAULT_PROVIDER=claude-opus
LLM_FALLBACK_ENABLED=true
OPENAI_API_KEY=your_key_here
ANTHROPIC_API_KEY=your_key_here
GOOGLE_API_KEY=your_key_here
# Frontend
FRONTEND_PORT=8080
FRONTEND_HOT_RELOAD=true
# Logging
LOG_LEVEL=debug
LOG_FORMAT=json
```
Load with:
```bash
export $(cat .env | xargs)
```
### Option 3: Configuration File
Create `~/.vapora/config.toml`:
```toml
[server]
port = 3000
host = "127.0.0.1"
environment = "development"
[database]
url = "sqlite:///Users/Akasha/.vapora/data.db"
max_connections = 5
timeout_seconds = 5
[tracking]
database_url = "sqlite:///Users/Akasha/.tracking/database.sqlite"
api_port = 3000
watch_dirs = [
"/Users/Akasha/.coder",
"/Users/Akasha/.claude/todos"
]
debounce_ms = 500
[llm_router]
default_provider = "claude-opus"
fallback_enabled = true
[frontend]
port = 8080
hot_reload = true
[logging]
level = "debug"
format = "json"
file = "/tmp/vapora.log"
```
---
## Building & Testing
### Phase 1: Build All Crates
```bash
# Build all crates in workspace (dev mode)
cargo build
# Build time: 3-8 minutes (first time)
# Subsequent builds: 10-30 seconds
# For optimized release build (slower to build, faster runtime)
cargo build --release
# Build time: 5-15 minutes (first time)
```
### Phase 2: Run Full Test Suite
```bash
# Run all tests
cargo test --lib
# Expected output:
# test result: ok. XXX passed; 0 failed; 0 ignored; 0 measured
# Run tests for specific crate
cargo test -p vapora-tracking --lib
cargo test -p vapora-backend --lib
cargo test -p vapora-agents --lib
# Run tests with output
cargo test --lib -- --nocapture --test-threads=1
# Run specific test
cargo test test_health_endpoint -- --exact
```
### Phase 3: Code Quality Checks
```bash
# Format code
cargo fmt
# Check formatting without modifying
cargo fmt -- --check
# Lint with clippy
cargo clippy --all-targets --all-features -- -W clippy::all
# Run both format and clippy
cargo fmt && cargo clippy --all-targets --all-features -- -W clippy::all
```
### Phase 4: Documentation
```bash
# Generate documentation for all crates
cargo doc --no-deps --open
# Generate for specific crate
cargo doc -p vapora-tracking --no-deps --open
# Check documentation coverage
cargo doc --document-private-items
```
### Verification Checklist
```bash
#!/bin/bash
set -e
echo "🔍 Running Vapora verification..."
echo "=================================="
# Build
echo "1. Building workspace..."
cargo build 2>&1 | tail -3
echo "✅ Build successful"
# Tests
echo "2. Running tests..."
cargo test --lib 2>&1 | grep "test result"
echo "✅ Tests passed"
# Clippy
echo "3. Running clippy..."
cargo clippy --all-targets --all-features 2>&1 | grep -v "warning:" | tail -1
echo "✅ Code quality checks passed"
# Format
echo "4. Checking format..."
cargo fmt -- --check 2>&1 && echo "✅ Code is properly formatted" || echo "⚠️ Code needs formatting"
echo "=================================="
echo "✅ Verification complete!"
```
---
## Development Setup
### IDE Setup
**VS Code (Recommended)**
```bash
# Install recommended extensions
# 1. rust-analyzer (rust-lang.rust-analyzer)
# 2. CodeLLDB (vadimcn.vscode-lldb)
# 3. Even Better TOML (tamasfe.even-better-toml)
# 4. Leptos (Leptos)
# .vscode/settings.json
{
"[rust]": {
"editor.formatOnSave": true,
"editor.defaultFormatter": "rust-lang.rust-analyzer"
},
"rust-analyzer.inlayHints.enable": true,
"rust-analyzer.lens.enable": true
}
```
**IntelliJ IDEA / CLion**
```bash
# Install Rust plugin
# Settings → Plugins → Rust → Install
# Recommended settings:
# Rust → Clippy → Use Clippy instead of Cargo check
# Rust → Macro Expansion → Expand experimental attribute macros
```
### Git Setup
```bash
# Clone pre-commit hooks (if available)
git clone https://github.com/vapora/hooks .git/hooks
# Or create basic hook:
cat > .git/hooks/pre-commit << 'EOF'
#!/bin/bash
cargo fmt --check && cargo clippy --all -- -W clippy::all
EOF
chmod +x .git/hooks/pre-commit
```
### Development Workflow
```bash
# 1. Create feature branch
git checkout -b feat/my-feature
# 2. Make changes and build
cargo build
# 3. Run tests
cargo test --lib
# 4. Check code quality
cargo fmt
cargo clippy --all -- -W clippy::all
# 5. Commit and push
git add .
git commit -m "feat: implement my-feature"
git push origin feat/my-feature
# 6. Create pull request
```
---
## First Run
### Run Backend Server
**Terminal 1: Backend**
```bash
# Run backend
cargo run -p vapora-backend
# With debug logging
RUST_LOG=debug cargo run -p vapora-backend
# Expected output:
# 🚀 Vapora Backend Server
# Listening on http://127.0.0.1:3000
# Available endpoints:
# GET /api/v1/health
# GET /api/v1/tracking/summary
# POST /api/v1/agents/orchestrate
```
### Run Frontend (Optional)
**Terminal 2: Frontend**
```bash
cd crates/vapora-frontend
# Install trunk (if not already)
cargo install trunk
# Run frontend with hot-reload
trunk serve
# Expected output:
# 🦕 Listening on http://127.0.0.1:8080
```
### Test Endpoints
**Terminal 3: Test**
```bash
# Health check
curl http://localhost:3000/api/v1/health
# Response: {"status":"ok","service":"vapora-backend",...}
# Tracking summary
curl http://localhost:3000/api/v1/tracking/summary
# Response: {"total_entries":0,"changes":0,"todos":0}
# Create tracking entry
curl -X POST http://localhost:3000/api/v1/tracking/entries \
-H "Content-Type: application/json" \
-d '{"summary":"First entry","impact":"backend"}'
```
### Using CLI Commands
```bash
# Start tracking service (if using local service)
./scripts/start-tracking-service.nu --verbose
# Log a change
/log-change "Completed setup" --impact infrastructure --files 1
# Create a TODO
/add-todo "Review tracking system" --priority H --estimate M
# Check status
/track-status --limit 10
# Export data
./scripts/export-tracking.nu json --output setup-report.json
```
---
## Troubleshooting
### Build Issues
**Error: "error[E0433]: failed to resolve: use of undeclared type"**
Solution:
```bash
# Update Rust
rustup update stable
# Clean cache
cargo clean
# Rebuild
cargo build
```
**Error: "could not compile ... due to X previous errors"**
Solution:
```bash
# Check Rust version (must be 1.75+)
rustc --version
# Update if needed
rustup install 1.75
rustup default 1.75
```
**Error: "linker 'cc' not found"**
Solution (macOS):
```bash
# Install Xcode command line tools
xcode-select --install
```
### Test Issues
**Tests fail with timeout**
Solution:
```bash
# Run with single thread
cargo test --lib -- --test-threads=1
# Increase timeout
RUST_TEST_TIME_UNIT=60000 cargo test --lib
```
**Tests panic with "thread 'main' panicked"**
Solution:
```bash
# Run with backtrace
RUST_BACKTRACE=1 cargo test --lib -- --nocapture
# Check logs for actual error
RUST_LOG=trace cargo test --lib -- --nocapture
```
### Database Issues
**Error: "database file not found"**
Solution:
```bash
# Create database directory
mkdir -p ~/.tracking
mkdir -p ~/.vapora
# Initialize databases
./scripts/start-tracking-service.nu
# Wait for init and stop with Ctrl+C
```
**Error: "Failed to acquire database lock"**
Solution:
```bash
# Ensure only one instance is running
lsof | grep database.sqlite
# Kill any lingering processes
pkill -f "vapora-backend"
pkill -f "tracking-service"
# Restart
cargo run -p vapora-backend
```
### Port Already in Use
**Error: "Address already in use"**
Solution:
```bash
# Find process using port 3000
lsof -i :3000
# Kill process
kill -9 <PID>
# Or use different port
VAPORA_PORT=3001 cargo run -p vapora-backend
```
### NuShell Script Issues
**Error: "command not found: nu"**
Solution:
```bash
# Install NuShell
brew install nu
# Or add to PATH
export PATH="/usr/local/bin:$PATH"
```
**Scripts not executable**
Solution:
```bash
# Make scripts executable
chmod +x scripts/*.nu
# Run with nu explicitly
nu scripts/start-tracking-service.nu
```
### Frontend Issues
**Error: "trunk: command not found"**
Solution:
```bash
# Install trunk
cargo install trunk
# Install WASM target
rustup target add wasm32-unknown-unknown
```
**Frontend won't load styles**
Solution:
```bash
# Clear build cache
rm -rf crates/vapora-frontend/target
rm -rf crates/vapora-frontend/dist
# Rebuild
cd crates/vapora-frontend && trunk serve
```
### Quick Troubleshooting Reference
| Problem | Quick Fix |
|---------|-----------|
| Build fails | `cargo clean && cargo build` |
| Tests fail | `rustup update && cargo test --lib` |
| Port in use | `lsof -i :3000 && kill -9 <PID>` |
| DB errors | `rm ~/.vapora/data.db && cargo run` |
| NuShell missing | `brew install nu` |
| Clippy warnings | `cargo clippy -- -W clippy::all` |
| Format issues | `cargo fmt` |
| Slow build | `export CARGO_INCREMENTAL=1` |
---
## Verification Steps
### Post-Installation Verification
```bash
#!/bin/bash
echo "🔍 Post-installation verification..."
echo "===================================="
# 1. Check Rust
echo "1. Checking Rust..."
rustc --version | grep -q "1\.[0-9]\+\.[0-9]\+" && echo "✅ Rust OK" || echo "❌ Rust issue"
# 2. Check build
echo "2. Building..."
cargo build 2>&1 | grep -q "Finished" && echo "✅ Build OK" || echo "❌ Build failed"
# 3. Check tests
echo "3. Testing..."
cargo test --lib 2>&1 | grep -q "test result: ok" && echo "✅ Tests OK" || echo "❌ Tests failed"
# 4. Check code quality
echo "4. Code quality..."
cargo clippy --all 2>&1 | grep -v "warning:" | tail -1 | grep -q "error" && echo "❌ Clippy issues" || echo "✅ Code quality OK"
# 5. Check structure
echo "5. Project structure..."
[ -f "Cargo.toml" ] && [ -d "crates" ] && echo "✅ Structure OK" || echo "❌ Structure issue"
echo "===================================="
echo "✅ Verification complete!"
```
---
## What's Next?
### Immediate Next Steps
1. Read [`QUICKSTART.md`](./QUICKSTART.md) for 15-minute quick start
2. Run backend: `cargo run -p vapora-backend`
3. Visit frontend: `http://localhost:8080`
4. Create first tracking entry: `/log-change "Setup complete"`
### Learning Resources
- API Documentation: `cargo doc --open`
- Crate READMEs: `crates/*/README.md`
- Tracking System: [`QUICKSTART_TRACKING.md`](./QUICKSTART_TRACKING.md)
- Architecture: [`.coder/`](./.coder/)
### Development Tips
- Use `cargo watch` for continuous building
- Set `RUST_LOG=debug` for detailed logs
- Use IDE debugging (VS Code + CodeLLDB)
- Join community for help
---
## Getting Help
**Issues not listed above?**
1. Check crate-specific documentation: `cargo doc --open`
2. Review `.coder/` documentation for architecture
3. Check inline code comments
4. Run with `RUST_LOG=trace` for detailed logs
5. See [`QUICKSTART.md`](./QUICKSTART.md) for quick reference
---
## ✅ Setup Completion Checklist
- [ ] Rust 1.75+ installed
- [ ] All prerequisites verified
- [ ] Repository cloned
- [ ] Dependencies downloaded (`cargo fetch`)
- [ ] Workspace builds successfully (`cargo build`)
- [ ] All tests pass (`cargo test --lib`)
- [ ] Code quality checks pass (`cargo clippy`)
- [ ] Backend runs (`cargo run -p vapora-backend`)
- [ ] Frontend loads (optional)
- [ ] Tracking system works (`/track-status`)
**All checked? ✅ Vapora is ready for development!**
---
**For quick 15-minute setup:** See [`QUICKSTART.md`](./QUICKSTART.md)
**For tracking system setup:** See [`SETUP_TRACKING.md`](./SETUP_TRACKING.md)