Vapora/crates/vapora-backend/tests/swarm_api_test.rs

289 lines
8.6 KiB
Rust
Raw Normal View History

feat: Phase 5.3 - Multi-Agent Learning Infrastructure Implement intelligent agent learning from Knowledge Graph execution history with per-task-type expertise tracking, recency bias, and learning curves. ## Phase 5.3 Implementation ### Learning Infrastructure (✅ Complete) - LearningProfileService with per-task-type expertise metrics - TaskTypeExpertise model tracking success_rate, confidence, learning curves - Recency bias weighting: recent 7 days weighted 3x higher (exponential decay) - Confidence scoring prevents overfitting: min(1.0, executions / 20) - Learning curves computed from daily execution windows ### Agent Scoring Service (✅ Complete) - Unified AgentScore combining SwarmCoordinator + learning profiles - Scoring formula: 0.3*base + 0.5*expertise + 0.2*confidence - Rank agents by combined score for intelligent assignment - Support for recency-biased scoring (recent_success_rate) - Methods: rank_agents, select_best, rank_agents_with_recency ### KG Integration (✅ Complete) - KGPersistence::get_executions_for_task_type() - query by agent + task type - KGPersistence::get_agent_executions() - all executions for agent - Coordinator::load_learning_profile_from_kg() - core KG→Learning integration - Coordinator::load_all_learning_profiles() - batch load for multiple agents - Convert PersistedExecution → ExecutionData for learning calculations ### Agent Assignment Integration (✅ Complete) - AgentCoordinator uses learning profiles for task assignment - extract_task_type() infers task type from title/description - assign_task() scores candidates using AgentScoringService - Fallback to load-based selection if no learning data available - Learning profiles stored in coordinator.learning_profiles RwLock ### Profile Adapter Enhancements (✅ Complete) - create_learning_profile() - initialize empty profiles - add_task_type_expertise() - set task-type expertise - update_profile_with_learning() - update swarm profiles from learning ## Files Modified ### vapora-knowledge-graph/src/persistence.rs (+30 lines) - get_executions_for_task_type(agent_id, task_type, limit) - get_agent_executions(agent_id, limit) ### vapora-agents/src/coordinator.rs (+100 lines) - load_learning_profile_from_kg() - core KG integration method - load_all_learning_profiles() - batch loading for agents - assign_task() already uses learning-based scoring via AgentScoringService ### Existing Complete Implementation - vapora-knowledge-graph/src/learning.rs - calculation functions - vapora-agents/src/learning_profile.rs - data structures and expertise - vapora-agents/src/scoring.rs - unified scoring service - vapora-agents/src/profile_adapter.rs - adapter methods ## Tests Passing - learning_profile: 7 tests ✅ - scoring: 5 tests ✅ - profile_adapter: 6 tests ✅ - coordinator: learning-specific tests ✅ ## Data Flow 1. Task arrives → AgentCoordinator::assign_task() 2. Extract task_type from description 3. Query KG for task-type executions (load_learning_profile_from_kg) 4. Calculate expertise with recency bias 5. Score candidates (SwarmCoordinator + learning) 6. Assign to top-scored agent 7. Execution result → KG → Update learning profiles ## Key Design Decisions ✅ Recency bias: 7-day half-life with 3x weight for recent performance ✅ Confidence scoring: min(1.0, total_executions / 20) prevents overfitting ✅ Hierarchical scoring: 30% base load, 50% expertise, 20% confidence ✅ KG query limit: 100 recent executions per task-type for performance ✅ Async loading: load_learning_profile_from_kg supports concurrent loads ## Next: Phase 5.4 - Cost Optimization Ready to implement budget enforcement and cost-aware provider selection.
2026-01-11 13:03:53 +00:00
// Integration tests for Swarm API endpoints
// Tests verify swarm statistics and health monitoring endpoints
use std::sync::Arc;
use vapora_swarm::{SwarmCoordinator, AgentProfile};
/// Helper to create a test agent profile
fn create_test_profile(id: &str, success_rate: f64, load: f64) -> AgentProfile {
AgentProfile {
id: id.to_string(),
roles: vec!["developer".to_string()],
capabilities: vec!["coding".to_string(), "testing".to_string()],
current_load: load,
success_rate,
availability: true,
}
}
#[tokio::test]
async fn test_swarm_coordinator_initialization() {
// Create a SwarmCoordinator
let swarm = Arc::new(SwarmCoordinator::new());
// Register test profiles
let profile1 = create_test_profile("agent-1", 0.95, 0.3);
let profile2 = create_test_profile("agent-2", 0.85, 0.5);
swarm.register_agent(profile1).ok();
swarm.register_agent(profile2).ok();
// Get statistics
let stats = swarm.get_swarm_stats();
// Verify statistics
assert_eq!(stats.total_agents, 2);
assert_eq!(stats.available_agents, 2);
assert!(stats.avg_load > 0.0);
assert!(stats.active_tasks == 0); // No tasks assigned yet
}
#[tokio::test]
async fn test_swarm_health_status_healthy() {
// Create swarm with available agents
let swarm = Arc::new(SwarmCoordinator::new());
let profile1 = create_test_profile("agent-1", 0.95, 0.3);
let profile2 = create_test_profile("agent-2", 0.90, 0.2);
swarm.register_agent(profile1).ok();
swarm.register_agent(profile2).ok();
let stats = swarm.get_swarm_stats();
// Verify health calculation
assert_eq!(stats.total_agents, 2);
assert_eq!(stats.available_agents, 2);
// All agents available = healthy
let is_healthy = stats.total_agents > 0 && stats.available_agents > 0;
assert!(is_healthy);
}
#[tokio::test]
async fn test_swarm_health_status_degraded() {
// Create swarm with some unavailable agents
let swarm = Arc::new(SwarmCoordinator::new());
let available_profile = create_test_profile("agent-1", 0.95, 0.3);
let mut unavailable_profile = create_test_profile("agent-2", 0.85, 0.5);
unavailable_profile.availability = false;
swarm.register_agent(available_profile).ok();
swarm.register_agent(unavailable_profile).ok();
let stats = swarm.get_swarm_stats();
// Verify health calculation
assert_eq!(stats.total_agents, 2);
assert_eq!(stats.available_agents, 1);
// Some unavailable = degraded
let is_degraded = stats.total_agents > 0 && stats.available_agents < stats.total_agents;
assert!(is_degraded);
}
#[tokio::test]
async fn test_swarm_health_status_no_agents() {
// Create empty swarm
let swarm = Arc::new(SwarmCoordinator::new());
let stats = swarm.get_swarm_stats();
// Verify no agents
assert_eq!(stats.total_agents, 0);
assert_eq!(stats.available_agents, 0);
}
#[tokio::test]
async fn test_swarm_statistics_load_calculation() {
// Create swarm with varied load profiles
let swarm = Arc::new(SwarmCoordinator::new());
let light_load = create_test_profile("agent-1", 0.95, 0.1);
let medium_load = create_test_profile("agent-2", 0.85, 0.5);
let high_load = create_test_profile("agent-3", 0.80, 0.9);
swarm.register_agent(light_load).ok();
swarm.register_agent(medium_load).ok();
swarm.register_agent(high_load).ok();
let stats = swarm.get_swarm_stats();
// Verify load calculation (average of 0.1, 0.5, 0.9 = 0.5)
assert_eq!(stats.total_agents, 3);
assert!(stats.avg_load > 0.4 && stats.avg_load < 0.6);
}
#[tokio::test]
async fn test_swarm_statistics_success_rate_variance() {
// Create swarm with different success rates
let swarm = Arc::new(SwarmCoordinator::new());
let high_success = create_test_profile("agent-1", 0.99, 0.2);
let medium_success = create_test_profile("agent-2", 0.50, 0.3);
let low_success = create_test_profile("agent-3", 0.10, 0.1);
swarm.register_agent(high_success).ok();
swarm.register_agent(medium_success).ok();
swarm.register_agent(low_success).ok();
let stats = swarm.get_swarm_stats();
// Verify all agents registered despite variance
assert_eq!(stats.total_agents, 3);
assert_eq!(stats.available_agents, 3);
}
#[tokio::test]
async fn test_swarm_agent_availability_transitions() {
// Create swarm with available agent
let swarm = Arc::new(SwarmCoordinator::new());
let mut profile = create_test_profile("agent-1", 0.95, 0.3);
swarm.register_agent(profile.clone()).ok();
// Verify initial state
let mut stats = swarm.get_swarm_stats();
assert_eq!(stats.available_agents, 1);
// Mark agent unavailable
profile.availability = false;
swarm.register_agent(profile).ok();
// Verify transition
stats = swarm.get_swarm_stats();
assert_eq!(stats.available_agents, 0);
}
#[tokio::test]
async fn test_swarm_unregister_agent() {
// Create swarm with agent
let swarm = Arc::new(SwarmCoordinator::new());
let profile = create_test_profile("agent-1", 0.95, 0.3);
swarm.register_agent(profile).ok();
let mut stats = swarm.get_swarm_stats();
assert_eq!(stats.total_agents, 1);
// Unregister agent
swarm.unregister_agent("agent-1").ok();
// Verify removal
stats = swarm.get_swarm_stats();
assert_eq!(stats.total_agents, 0);
}
#[tokio::test]
async fn test_swarm_task_assignment_selects_best_agent() {
// Create swarm with agents of different quality
let swarm = Arc::new(SwarmCoordinator::new());
let poor_agent = create_test_profile("agent-poor", 0.50, 0.9); // Low success, high load
let good_agent = create_test_profile("agent-good", 0.95, 0.2); // High success, low load
swarm.register_agent(poor_agent).ok();
swarm.register_agent(good_agent).ok();
// Score: success_rate / (1.0 + load)
// agent-poor: 0.50 / (1.0 + 0.9) = 0.50 / 1.9 ≈ 0.26
// agent-good: 0.95 / (1.0 + 0.2) = 0.95 / 1.2 ≈ 0.79
// agent-good should be selected
// Verify agent-good has better score
let poor_score = 0.50 / (1.0 + 0.9);
let good_score = 0.95 / (1.0 + 0.2);
assert!(good_score > poor_score);
}
#[tokio::test]
async fn test_swarm_statistics_consistency() {
// Test that statistics remain consistent with multiple operations
let swarm = Arc::new(SwarmCoordinator::new());
// Initial state
let mut stats = swarm.get_swarm_stats();
assert_eq!(stats.total_agents, 0);
// Add agents
for i in 0..5 {
let profile = create_test_profile(&format!("agent-{}", i), 0.85, 0.3);
swarm.register_agent(profile).ok();
}
stats = swarm.get_swarm_stats();
assert_eq!(stats.total_agents, 5);
assert_eq!(stats.available_agents, 5);
// Update one agent to unavailable
let mut profile = create_test_profile("agent-0", 0.85, 0.3);
profile.availability = false;
swarm.register_agent(profile).ok();
stats = swarm.get_swarm_stats();
assert_eq!(stats.total_agents, 5);
assert_eq!(stats.available_agents, 4);
// Remove one agent
swarm.unregister_agent("agent-1").ok();
stats = swarm.get_swarm_stats();
assert_eq!(stats.total_agents, 4);
assert_eq!(stats.available_agents, 3);
}
#[tokio::test]
async fn test_swarm_large_agent_pool() {
// Test swarm behavior with larger agent pool
let swarm = Arc::new(SwarmCoordinator::new());
// Register 50 agents with varied metrics
for i in 0..50 {
let success_rate = if i % 3 == 0 {
0.95
} else if i % 3 == 1 {
0.75
} else {
0.55
};
let load = (i as f64 % 10.0) / 10.0;
let profile = create_test_profile(&format!("agent-{}", i), success_rate, load);
swarm.register_agent(profile).ok();
}
let stats = swarm.get_swarm_stats();
// Verify all registered
assert_eq!(stats.total_agents, 50);
assert_eq!(stats.available_agents, 50);
// Verify average load is reasonable
assert!(stats.avg_load > 0.0 && stats.avg_load < 1.0);
}
#[tokio::test]
async fn test_swarm_empty_after_unregister_all() {
// Create swarm with agents
let swarm = Arc::new(SwarmCoordinator::new());
for i in 0..3 {
let profile = create_test_profile(&format!("agent-{}", i), 0.85, 0.3);
swarm.register_agent(profile).ok();
}
let mut stats = swarm.get_swarm_stats();
assert_eq!(stats.total_agents, 3);
// Unregister all
for i in 0..3 {
swarm.unregister_agent(&format!("agent-{}", i)).ok();
}
stats = swarm.get_swarm_stats();
assert_eq!(stats.total_agents, 0);
assert_eq!(stats.available_agents, 0);
}