Vapora/docs/setup/tracking-setup.md
Jesús Pérez d14150da75 feat: Phase 5.3 - Multi-Agent Learning Infrastructure
Implement intelligent agent learning from Knowledge Graph execution history
with per-task-type expertise tracking, recency bias, and learning curves.

## Phase 5.3 Implementation

### Learning Infrastructure ( Complete)
- LearningProfileService with per-task-type expertise metrics
- TaskTypeExpertise model tracking success_rate, confidence, learning curves
- Recency bias weighting: recent 7 days weighted 3x higher (exponential decay)
- Confidence scoring prevents overfitting: min(1.0, executions / 20)
- Learning curves computed from daily execution windows

### Agent Scoring Service ( Complete)
- Unified AgentScore combining SwarmCoordinator + learning profiles
- Scoring formula: 0.3*base + 0.5*expertise + 0.2*confidence
- Rank agents by combined score for intelligent assignment
- Support for recency-biased scoring (recent_success_rate)
- Methods: rank_agents, select_best, rank_agents_with_recency

### KG Integration ( Complete)
- KGPersistence::get_executions_for_task_type() - query by agent + task type
- KGPersistence::get_agent_executions() - all executions for agent
- Coordinator::load_learning_profile_from_kg() - core KG→Learning integration
- Coordinator::load_all_learning_profiles() - batch load for multiple agents
- Convert PersistedExecution → ExecutionData for learning calculations

### Agent Assignment Integration ( Complete)
- AgentCoordinator uses learning profiles for task assignment
- extract_task_type() infers task type from title/description
- assign_task() scores candidates using AgentScoringService
- Fallback to load-based selection if no learning data available
- Learning profiles stored in coordinator.learning_profiles RwLock

### Profile Adapter Enhancements ( Complete)
- create_learning_profile() - initialize empty profiles
- add_task_type_expertise() - set task-type expertise
- update_profile_with_learning() - update swarm profiles from learning

## Files Modified

### vapora-knowledge-graph/src/persistence.rs (+30 lines)
- get_executions_for_task_type(agent_id, task_type, limit)
- get_agent_executions(agent_id, limit)

### vapora-agents/src/coordinator.rs (+100 lines)
- load_learning_profile_from_kg() - core KG integration method
- load_all_learning_profiles() - batch loading for agents
- assign_task() already uses learning-based scoring via AgentScoringService

### Existing Complete Implementation
- vapora-knowledge-graph/src/learning.rs - calculation functions
- vapora-agents/src/learning_profile.rs - data structures and expertise
- vapora-agents/src/scoring.rs - unified scoring service
- vapora-agents/src/profile_adapter.rs - adapter methods

## Tests Passing
- learning_profile: 7 tests 
- scoring: 5 tests 
- profile_adapter: 6 tests 
- coordinator: learning-specific tests 

## Data Flow
1. Task arrives → AgentCoordinator::assign_task()
2. Extract task_type from description
3. Query KG for task-type executions (load_learning_profile_from_kg)
4. Calculate expertise with recency bias
5. Score candidates (SwarmCoordinator + learning)
6. Assign to top-scored agent
7. Execution result → KG → Update learning profiles

## Key Design Decisions
 Recency bias: 7-day half-life with 3x weight for recent performance
 Confidence scoring: min(1.0, total_executions / 20) prevents overfitting
 Hierarchical scoring: 30% base load, 50% expertise, 20% confidence
 KG query limit: 100 recent executions per task-type for performance
 Async loading: load_learning_profile_from_kg supports concurrent loads

## Next: Phase 5.4 - Cost Optimization
Ready to implement budget enforcement and cost-aware provider selection.
2026-01-11 13:03:53 +00:00

13 KiB

title, date, version
title date version
Vapora Tracking System - Complete Setup Guide 2025-11-10 1.0

🛠️ Vapora Tracking System - Complete Setup Guide

This guide covers everything from zero to fully operational tracking system.


📋 Table of Contents

  1. Prerequisites
  2. Installation
  3. Configuration
  4. Verification
  5. First Use
  6. Troubleshooting

Prerequisites

System Requirements

Requirement Minimum Recommended
OS macOS 10.15+ macOS 12+
RAM 2GB 4GB+
Disk 1GB 2GB+
Internet Required for install Required

Software Requirements

# Check if installed
rustc --version    # Need 1.70+
cargo --version    # Comes with Rust
git --version      # Need 2.20+

Installation Check

#!/bin/bash
echo "Checking prerequisites..."

# Check Rust
if ! command -v rustc &> /dev/null; then
    echo "❌ Rust not found. Install from https://rustup.rs"
    exit 1
fi

# Check Cargo
if ! command -v cargo &> /dev/null; then
    echo "❌ Cargo not found. Install Rust from https://rustup.rs"
    exit 1
fi

# Check Rust version
RUST_VERSION=$(rustc --version | awk '{print $2}')
echo "✅ Rust $RUST_VERSION found"
echo "✅ All prerequisites met!"

Installation

Step 1: Install Rust (if needed)

# Download and install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Follow the prompts (default options are fine)

# Add Rust to PATH
source $HOME/.cargo/env

# Verify installation
rustc --version
cargo --version

Expected output:

rustc 1.XX.X (XXXXXXXXXXXX YYYY-MM-DD)
cargo 1.XX.X (XXXXXXXXXXXX YYYY-MM-DD)

Step 2: Clone or Navigate to Vapora

# Option A: Clone (if you don't have it)
git clone https://github.com/yourusername/vapora.git
cd vapora

# Option B: Navigate (if you already have it)
cd vapora

# Verify structure
ls crates/vapora-tracking/
# Should show: Cargo.toml, README.md, src/, benches/

Step 3: Build the Tracking System

# Build tracking crate
cargo build -p vapora-tracking

# Output should show:
# Compiling vapora-tracking v0.1.0
# Finished `dev` profile [unoptimized + debuginfo]

# For faster runtime (slower build):
cargo build -p vapora-tracking --release

# Output should show:
# Finished `release` profile [optimized]

Build time:

  • Debug: 30-60 seconds (first time)
  • Release: 2-5 minutes (first time)
  • Subsequent: 1-10 seconds (incremental)

Step 4: Verify Build

# List build artifacts
ls -lh target/debug/deps/ | grep vapora_tracking

# Should show several .rlib and other files

# Or with release build
ls -lh target/release/deps/ | grep vapora_tracking

Step 5: Install CLI Components

# Create commands directory if it doesn't exist
mkdir -p ~/.claude/commands
mkdir -p ~/.claude/skills

# Copy tracking commands
cp ~/.claude/commands/log-change.md ~/.claude/commands/
cp ~/.claude/commands/add-todo.md ~/.claude/commands/
cp ~/.claude/commands/track-status.md ~/.claude/commands/

# Copy tracking skill
cp ~/.claude/skills/tracking.md ~/.claude/skills/

# Verify installation
ls -la ~/.claude/commands/ | grep -E "log-change|add-todo|track-status"
ls -la ~/.claude/skills/ | grep tracking

Configuration

The system works out of the box with defaults:

# Default database location
~/.tracking/database.sqlite

# Default watch directories
~/.coder/
~/.claude/todos/

# Default API port
3000

No configuration needed! Skip to Verification.

Option 2: Custom Configuration

Create ~/.config/vapora-tracking.toml:

[database]
# SQLite database path (use your home directory)
url = "sqlite://~/.tracking/database.sqlite"

# Max connections to database
max_connections = 5

# Connection timeout (seconds)
timeout = 5

[watcher]
# Directories to watch for changes (customize for your setup)
watch_dirs = [
    "~/.coder",
    "~/.claude/todos"
]

# Debounce time (milliseconds)
debounce_ms = 500

[api]
# API server port
port = 3000

# API host
host = "127.0.0.1"

[logging]
# Log level: trace, debug, info, warn, error
level = "info"

# Log file path
file = "/tmp/vapora-tracking.log"

Option 3: Environment Variables

# Set database location (use your home directory)
export TRACKING_DATABASE_URL="sqlite://~/.tracking/database.sqlite"

# Set API port
export TRACKING_API_PORT=3000

# Set log level
export RUST_LOG=info

# These override config file values

Verification

Test 1: Build Success

# Clean rebuild to verify
cargo clean -p vapora-tracking
cargo build -p vapora-tracking

# Check for success
if [ $? -eq 0 ]; then
    echo "✅ Build successful"
else
    echo "❌ Build failed"
    exit 1
fi

Test 2: Tests Pass

# Run all unit tests
cargo test -p vapora-tracking --lib

# Expected output:
# running 20 tests
# ...
# test result: ok. 20 passed; 0 failed

If not 20 tests:

# Check what tests exist
cargo test -p vapora-tracking --lib -- --list

# Run with output for debugging
cargo test -p vapora-tracking --lib -- --nocapture

Test 3: Code Quality

# Run clippy checks
cargo clippy -p vapora-tracking --lib -- -W clippy::all

# Should show minimal warnings (profile warning is expected)

Test 4: CLI Commands

# Verify commands are installed
ls ~/.claude/commands/

# Output should include:
# add-todo.md
# log-change.md
# track-status.md

# Verify skill
ls ~/.claude/skills/tracking.md

Test 5: API Health

# Start service
./scripts/start-tracking-service.nu

# Wait 2 seconds
sleep 2

# Check health endpoint
curl http://localhost:3000/api/v1/tracking/health

# Expected output:
# {"status":"ok","service":"vapora-tracking","timestamp":"2025-11-10T..."}

Verification Checklist

# Run all verifications
echo "Running verification tests..."

# Test 1: Build
cargo build -p vapora-tracking 2>&1 | grep -q "Finished"
[ $? -eq 0 ] && echo "✅ Build" || echo "❌ Build"

# Test 2: Tests
cargo test -p vapora-tracking --lib 2>&1 | grep -q "20 passed"
[ $? -eq 0 ] && echo "✅ Tests" || echo "❌ Tests"

# Test 3: Clippy
cargo clippy -p vapora-tracking --lib 2>&1 | grep -q "error:" && echo "❌ Clippy" || echo "✅ Clippy"

# Test 4: Commands
[ -f ~/.claude/commands/log-change.md ] && echo "✅ Commands" || echo "❌ Commands"

# Test 5: Skills
[ -f ~/.claude/skills/tracking.md ] && echo "✅ Skills" || echo "❌ Skills"

echo "Verification complete!"

First Use

Your First Change Log

# Log your first change
/log-change "Set up Vapora tracking system" \
  --impact infrastructure \
  --files 1

# Expected response: Change logged successfully

Your First TODO

# Create your first TODO
/add-todo "Review tracking system documentation" \
  --priority M \
  --estimate S \
  --due 2025-11-12

# Expected response: TODO created successfully

Your First Status Check

# Check current status
/track-status --limit 5

# Expected output shows your 1 change and 1 TODO

Your First Export

# Export to JSON
./scripts/export-tracking.nu json --output tracking-report

# Export to Markdown
./scripts/export-tracking.nu markdown --project vapora > report.md

# Check results
ls -la tracking-report.json
cat report.md

Troubleshooting

Build Issues

Issue: "error[E0433]: failed to resolve"

Solution:

# Update Rust
rustup update

# Clean and rebuild
cargo clean -p vapora-tracking
cargo build -p vapora-tracking

Issue: "could not compile ... due to X previous errors"

Solution:

# Check Rust version
rustc --version  # Must be 1.70+

# Update if needed
rustup install stable

# Verify dependencies
cargo tree -p vapora-tracking

Database Issues

Issue: "Database file not found"

Solution:

# Create database directory
mkdir -p ~/.tracking

# Start service to initialize
./scripts/start-tracking-service.nu

# Check database was created
ls -la ~/.tracking/

Issue: "Failed to initialize database"

Solution:

# Reset database
rm ~/.tracking/database.sqlite

# Service will recreate on next start
./scripts/start-tracking-service.nu

CLI Issues

Issue: "/log-change not found" in Claude Code

Solution:

# Verify commands are copied
ls ~/.claude/commands/log-change.md

# If missing, copy them
cp ~/.claude/commands/log-change.md ~/.claude/commands/

# Restart Claude Code

Issue: "Command not recognized"

Solution:

# Check command format
/log-change --help

# Should show usage information
# If not, commands aren't properly installed

API Issues

Issue: "Connection refused" when calling API

Solution:

# Start the service
./scripts/start-tracking-service.nu --verbose

# Wait for startup
sleep 2

# Check health
curl -v http://localhost:3000/api/v1/tracking/health

# Check logs
tail -f /tmp/vapora-tracking.log

Issue: "Port 3000 already in use"

Solution:

# Use different port
./scripts/start-tracking-service.nu --port 3001

# Or kill existing service
lsof -i :3000
kill -9 <PID>

Performance Issues

Issue: Build is very slow

Solution:

# Use incremental compilation
export CARGO_INCREMENTAL=1

# Use faster linker (if available)
export RUSTFLAGS="-C link-arg=-fuse-ld=lld"

# Or just use release build once
cargo build -p vapora-tracking --release

Issue: High memory usage

Solution:

# Limit parallel jobs during build
cargo build -p vapora-tracking -j 2

# Or reduce connection pool
# Edit storage.rs: max_connections = 2

Quick Troubleshooting Reference

Problem Quick Fix
Build fails cargo clean && cargo build
Tests fail rustup update then rebuild
Commands missing cp ~/.claude/commands/*
API won't start ./scripts/start-tracking-service.nu --verbose
Database errors rm ~/.tracking/database.sqlite
Port in use ./scripts/start-tracking-service.nu --port 3001
Slow build export CARGO_INCREMENTAL=1

System Validation Script

Save this as validate-tracking.sh:

#!/bin/bash

set -e

echo "🔍 Validating Vapora Tracking System Installation"
echo "=================================================="

# Check Rust
echo "Checking Rust..."
if ! command -v rustc &> /dev/null; then
    echo "❌ Rust not found"
    exit 1
fi
echo "✅ Rust $(rustc --version | awk '{print $2}')"

# Check build
echo "Building tracking crate..."
cargo build -p vapora-tracking 2>&1 | tail -3
echo "✅ Build successful"

# Check tests
echo "Running tests..."
TEST_OUTPUT=$(cargo test -p vapora-tracking --lib 2>&1)
if echo "$TEST_OUTPUT" | grep -q "test result: ok. 20 passed"; then
    echo "✅ All 20 tests passed"
else
    echo "❌ Tests failed"
    exit 1
fi

# Check CLI
echo "Checking CLI components..."
[ -f ~/.claude/commands/log-change.md ] && echo "✅ Commands installed" || echo "⚠️ Commands not found"
[ -f ~/.claude/skills/tracking.md ] && echo "✅ Skill installed" || echo "⚠️ Skill not found"

# Check scripts
echo "Checking scripts..."
[ -f ./scripts/start-tracking-service.nu ] && echo "✅ Scripts available" || echo "⚠️ Scripts not found"

echo ""
echo "=================================================="
echo "✅ Validation Complete - System Ready!"
echo "=================================================="

echo ""
echo "Next steps:"
echo "1. /log-change \"Your first change\""
echo "2. /add-todo \"Your first task\""
echo "3. /track-status"

Run it:

chmod +x validate-tracking.sh
./validate-tracking.sh

What's Next After Setup?

Immediate (Today)

  • Complete all verification tests
  • Create your first change log
  • Create your first TODO
  • Run your first status check

Short Term (This Week)

  • Use /log-change for actual changes
  • Use /add-todo for tasks
  • Explore /track-status filters
  • Try exporting to different formats

Medium Term (This Month)

  • Set up automated syncing
  • Create custom dashboard queries
  • Integrate with your workflows
  • Set up reports

Getting Help

Issue not listed above?

  1. Check the TROUBLESHOOTING section in INTEGRATION.md
  2. Review TRACKING_SYSTEM_STATUS.md
  3. Check logs: tail -f /tmp/vapora-tracking.log
  4. Read inline code documentation: cargo doc -p vapora-tracking --open

Summary

You've successfully set up the Vapora tracking system!

What you now have:

Built: vapora-tracking crate compiled and tested Verified: All 20 tests passing Installed: CLI commands and skill Running: Tracking service ready Configured: Database and API ready

Start using it:

/log-change "Example change" --impact backend
/add-todo "Example task" --priority H
/track-status

Happy tracking! 🚀