Vapora/docs/setup/tracking-quickstart.md

260 lines
5.9 KiB
Markdown
Raw Permalink Normal View History

feat: Phase 5.3 - Multi-Agent Learning Infrastructure Implement intelligent agent learning from Knowledge Graph execution history with per-task-type expertise tracking, recency bias, and learning curves. ## Phase 5.3 Implementation ### Learning Infrastructure (✅ Complete) - LearningProfileService with per-task-type expertise metrics - TaskTypeExpertise model tracking success_rate, confidence, learning curves - Recency bias weighting: recent 7 days weighted 3x higher (exponential decay) - Confidence scoring prevents overfitting: min(1.0, executions / 20) - Learning curves computed from daily execution windows ### Agent Scoring Service (✅ Complete) - Unified AgentScore combining SwarmCoordinator + learning profiles - Scoring formula: 0.3*base + 0.5*expertise + 0.2*confidence - Rank agents by combined score for intelligent assignment - Support for recency-biased scoring (recent_success_rate) - Methods: rank_agents, select_best, rank_agents_with_recency ### KG Integration (✅ Complete) - KGPersistence::get_executions_for_task_type() - query by agent + task type - KGPersistence::get_agent_executions() - all executions for agent - Coordinator::load_learning_profile_from_kg() - core KG→Learning integration - Coordinator::load_all_learning_profiles() - batch load for multiple agents - Convert PersistedExecution → ExecutionData for learning calculations ### Agent Assignment Integration (✅ Complete) - AgentCoordinator uses learning profiles for task assignment - extract_task_type() infers task type from title/description - assign_task() scores candidates using AgentScoringService - Fallback to load-based selection if no learning data available - Learning profiles stored in coordinator.learning_profiles RwLock ### Profile Adapter Enhancements (✅ Complete) - create_learning_profile() - initialize empty profiles - add_task_type_expertise() - set task-type expertise - update_profile_with_learning() - update swarm profiles from learning ## Files Modified ### vapora-knowledge-graph/src/persistence.rs (+30 lines) - get_executions_for_task_type(agent_id, task_type, limit) - get_agent_executions(agent_id, limit) ### vapora-agents/src/coordinator.rs (+100 lines) - load_learning_profile_from_kg() - core KG integration method - load_all_learning_profiles() - batch loading for agents - assign_task() already uses learning-based scoring via AgentScoringService ### Existing Complete Implementation - vapora-knowledge-graph/src/learning.rs - calculation functions - vapora-agents/src/learning_profile.rs - data structures and expertise - vapora-agents/src/scoring.rs - unified scoring service - vapora-agents/src/profile_adapter.rs - adapter methods ## Tests Passing - learning_profile: 7 tests ✅ - scoring: 5 tests ✅ - profile_adapter: 6 tests ✅ - coordinator: learning-specific tests ✅ ## Data Flow 1. Task arrives → AgentCoordinator::assign_task() 2. Extract task_type from description 3. Query KG for task-type executions (load_learning_profile_from_kg) 4. Calculate expertise with recency bias 5. Score candidates (SwarmCoordinator + learning) 6. Assign to top-scored agent 7. Execution result → KG → Update learning profiles ## Key Design Decisions ✅ Recency bias: 7-day half-life with 3x weight for recent performance ✅ Confidence scoring: min(1.0, total_executions / 20) prevents overfitting ✅ Hierarchical scoring: 30% base load, 50% expertise, 20% confidence ✅ KG query limit: 100 recent executions per task-type for performance ✅ Async loading: load_learning_profile_from_kg supports concurrent loads ## Next: Phase 5.4 - Cost Optimization Ready to implement budget enforcement and cost-aware provider selection.
2026-01-11 13:03:53 +00:00
---
title: Vapora Tracking System - Quick Start Guide
date: 2025-11-10
status: READY
type: tracking-quickstart
---
# 🚀 Vapora Tracking System - Quick Start Guide
**⏱️ Time to get running: 5-10 minutes**
This guide walks you through installing and getting started with the Vapora tracking system component in the simplest way possible.
**Note:** This guide is for the tracking system only. For complete Vapora project setup, see [`QUICKSTART.md`](./QUICKSTART.md).
---
## 📋 Prerequisites
You need:
- ✅ Rust 1.70+ (install from https://rustup.rs)
- ✅ Cargo (comes with Rust)
- ✅ Git
- ✅ 500MB free disk space
- ✅ Bash or Zsh shell
**Check if you have everything:**
```bash
rustc --version # Should show Rust 1.70+
cargo --version # Should show Cargo 1.70+
which git # Should show /usr/bin/git or similar
```
---
## 🎯 5-Minute Quick Start
### Step 1: Build the Tracking System
```bash
# Build the tracking crate
cargo build -p vapora-tracking
# Or with backend integration
cargo build -p vapora-backend
```
**Expected output:**
```
Finished `dev` profile [unoptimized + debuginfo] target(s) in X.XXs
```
### Step 3: Run Tests
```bash
# Verify everything works
cargo test -p vapora-tracking --lib
# Should show: test result: ok. 20 passed
```
### Step 4: Start Using It
**Option A: Using Slash Commands (Easiest)**
```bash
# In Claude Code, use the commands:
/log-change "Fixed bug in parser" --impact backend --files 3
/add-todo "Refactor database" --priority H --estimate M
/track-status --project vapora --limit 10
```
**Option B: Using Scripts (Manual Sync)**
```bash
# Start the tracking service
./scripts/start-tracking-service.nu --verbose
# In another terminal, sync projects (replace with your development directory)
./scripts/sync-tracking.nu --projects-dir ~ --verbose
# Check status
/track-status
```
**Option C: Using API (Integration)**
```bash
# Query the API
curl http://localhost:3000/api/v1/tracking/summary
curl http://localhost:3000/api/v1/tracking/entries?limit=10
```
---
## ✅ Verify Installation
After building, verify everything works:
### Test 1: Build Success
```bash
cargo build -p vapora-tracking 2>&1 | tail -3
# Should show: Finished `dev` profile [unoptimized + debuginfo]
```
### Test 2: Tests Pass
```bash
cargo test -p vapora-tracking --lib 2>&1 | grep "test result"
# Should show: test result: ok. 20 passed; 0 failed
```
### Test 3: Clippy Clean
```bash
cargo clippy -p vapora-tracking --lib 2>&1 | grep "warning:" | wc -l
# Should show: 1 (profile warning only, which is expected)
```
### Test 4: Commands Available
```bash
ls ~/.claude/commands/ | grep -E "log-change|add-todo|track-status"
# Should show all 3 commands
```
### Test 5: Skill Available
```bash
ls ~/.claude/skills/tracking.md
# Should show the file exists
```
**If all 5 tests pass: ✅ Installation Complete!**
---
## 🎬 First Time Usage
### Scenario 1: Log Your First Change
**Using Slash Command (Easiest):**
```bash
/log-change "Implemented user authentication" \
--impact backend \
--files 5
```
**What happens:**
1. ✅ Change is logged to database
2. ✅ Timestamp added automatically
3. ✅ Can be queried with `/track-status`
### Scenario 2: Create Your First TODO
**Using Slash Command:**
```bash
/add-todo "Review code changes" \
--priority H \
--estimate M \
--due 2025-11-15
```
**What happens:**
1. ✅ TODO created in database
2. ✅ Can be tracked with `/track-status`
3. ✅ Shows up in exports
### Scenario 3: Check Your Status
**Using Slash Command:**
```bash
/track-status --limit 5
```
**Output:**
```
✅ Summary
Total entries: 3
Changes: 1
TODOs: 2
🔄 Changes
[2025-11-10T14:30:00Z] - Implemented user authentication
Impact: backend | Breaking: no | Files: 5
📋 TODOs
[HIGH] Review code changes (Medium) - Due: 2025-11-15
[HIGH] Write documentation (Small) - Due: 2025-11-12
```
---
## 📚 Next Steps After Installation
### Short Term (Today)
1. ✅ Log 2-3 changes you've made
2. ✅ Create 2-3 TODOs for upcoming work
3. ✅ Run `/track-status` to see results
### Medium Term (This Week)
1. 📝 Set up daily tracking in your workflow
2. 🔄 Sync multiple projects with `sync-tracking.nu`
3. 📊 Export your tracking data with `export-tracking.nu`
### Long Term (Ongoing)
1. 📈 Monitor project progress via `/track-status`
2. 🎯 Use for sprint planning and retrospectives
3. 📉 Generate reports from exported data
4. 🔗 Integrate with other Vapora services
---
2026-01-12 03:17:04 +00:00
## 🆘 Need More Help
feat: Phase 5.3 - Multi-Agent Learning Infrastructure Implement intelligent agent learning from Knowledge Graph execution history with per-task-type expertise tracking, recency bias, and learning curves. ## Phase 5.3 Implementation ### Learning Infrastructure (✅ Complete) - LearningProfileService with per-task-type expertise metrics - TaskTypeExpertise model tracking success_rate, confidence, learning curves - Recency bias weighting: recent 7 days weighted 3x higher (exponential decay) - Confidence scoring prevents overfitting: min(1.0, executions / 20) - Learning curves computed from daily execution windows ### Agent Scoring Service (✅ Complete) - Unified AgentScore combining SwarmCoordinator + learning profiles - Scoring formula: 0.3*base + 0.5*expertise + 0.2*confidence - Rank agents by combined score for intelligent assignment - Support for recency-biased scoring (recent_success_rate) - Methods: rank_agents, select_best, rank_agents_with_recency ### KG Integration (✅ Complete) - KGPersistence::get_executions_for_task_type() - query by agent + task type - KGPersistence::get_agent_executions() - all executions for agent - Coordinator::load_learning_profile_from_kg() - core KG→Learning integration - Coordinator::load_all_learning_profiles() - batch load for multiple agents - Convert PersistedExecution → ExecutionData for learning calculations ### Agent Assignment Integration (✅ Complete) - AgentCoordinator uses learning profiles for task assignment - extract_task_type() infers task type from title/description - assign_task() scores candidates using AgentScoringService - Fallback to load-based selection if no learning data available - Learning profiles stored in coordinator.learning_profiles RwLock ### Profile Adapter Enhancements (✅ Complete) - create_learning_profile() - initialize empty profiles - add_task_type_expertise() - set task-type expertise - update_profile_with_learning() - update swarm profiles from learning ## Files Modified ### vapora-knowledge-graph/src/persistence.rs (+30 lines) - get_executions_for_task_type(agent_id, task_type, limit) - get_agent_executions(agent_id, limit) ### vapora-agents/src/coordinator.rs (+100 lines) - load_learning_profile_from_kg() - core KG integration method - load_all_learning_profiles() - batch loading for agents - assign_task() already uses learning-based scoring via AgentScoringService ### Existing Complete Implementation - vapora-knowledge-graph/src/learning.rs - calculation functions - vapora-agents/src/learning_profile.rs - data structures and expertise - vapora-agents/src/scoring.rs - unified scoring service - vapora-agents/src/profile_adapter.rs - adapter methods ## Tests Passing - learning_profile: 7 tests ✅ - scoring: 5 tests ✅ - profile_adapter: 6 tests ✅ - coordinator: learning-specific tests ✅ ## Data Flow 1. Task arrives → AgentCoordinator::assign_task() 2. Extract task_type from description 3. Query KG for task-type executions (load_learning_profile_from_kg) 4. Calculate expertise with recency bias 5. Score candidates (SwarmCoordinator + learning) 6. Assign to top-scored agent 7. Execution result → KG → Update learning profiles ## Key Design Decisions ✅ Recency bias: 7-day half-life with 3x weight for recent performance ✅ Confidence scoring: min(1.0, total_executions / 20) prevents overfitting ✅ Hierarchical scoring: 30% base load, 50% expertise, 20% confidence ✅ KG query limit: 100 recent executions per task-type for performance ✅ Async loading: load_learning_profile_from_kg supports concurrent loads ## Next: Phase 5.4 - Cost Optimization Ready to implement budget enforcement and cost-aware provider selection.
2026-01-11 13:03:53 +00:00
| Question | Answer Location |
|----------|-----------------|
| How do I use the tracking system? | `TRACKING_SYSTEM_STATUS.md` (How to use section) |
| What are all the features? | `crates/vapora-tracking/README.md` (Features section) |
| How do I deploy it? | `crates/vapora-tracking/INTEGRATION.md` (Deployment section) |
| How do I fix an issue? | `SETUP_TRACKING.md` (Troubleshooting section) |
| What's the architecture? | `TRACKING_DOCUMENTATION_INDEX.md` |
---
## ⚡ Super Quick Reference
```bash
# Build
cargo build -p vapora-tracking
# Test
cargo test -p vapora-tracking --lib
# Use commands
/log-change "Summary" --impact backend
/add-todo "Task" --priority H --estimate M
/track-status --limit 10
# Use scripts
./scripts/sync-tracking.nu --verbose
./scripts/export-tracking.nu json --output report
./scripts/start-tracking-service.nu
# Query API
curl http://localhost:3000/api/v1/tracking/summary
```
---
## ✅ Installation Checklist
- [ ] Rust 1.75+ installed
- [ ] Vapora repo available
- [ ] `cargo build -p vapora-tracking` succeeds
- [ ] `cargo test -p vapora-tracking --lib` shows 20 passed
- [ ] Slash commands copied to `~/.claude/commands/`
- [ ] Skill copied to `~/.claude/skills/`
- [ ] `/log-change` command works
- [ ] `/track-status` shows results
**All checked? ✅ You're ready to go!**
---
**For complete Vapora project setup:** See [`QUICKSTART.md`](./QUICKSTART.md)
**For tracking system deep dive:** See [`SETUP_TRACKING.md`](./SETUP_TRACKING.md)