Implement intelligent agent learning from Knowledge Graph execution history with per-task-type expertise tracking, recency bias, and learning curves. ## Phase 5.3 Implementation ### Learning Infrastructure (✅ Complete) - LearningProfileService with per-task-type expertise metrics - TaskTypeExpertise model tracking success_rate, confidence, learning curves - Recency bias weighting: recent 7 days weighted 3x higher (exponential decay) - Confidence scoring prevents overfitting: min(1.0, executions / 20) - Learning curves computed from daily execution windows ### Agent Scoring Service (✅ Complete) - Unified AgentScore combining SwarmCoordinator + learning profiles - Scoring formula: 0.3*base + 0.5*expertise + 0.2*confidence - Rank agents by combined score for intelligent assignment - Support for recency-biased scoring (recent_success_rate) - Methods: rank_agents, select_best, rank_agents_with_recency ### KG Integration (✅ Complete) - KGPersistence::get_executions_for_task_type() - query by agent + task type - KGPersistence::get_agent_executions() - all executions for agent - Coordinator::load_learning_profile_from_kg() - core KG→Learning integration - Coordinator::load_all_learning_profiles() - batch load for multiple agents - Convert PersistedExecution → ExecutionData for learning calculations ### Agent Assignment Integration (✅ Complete) - AgentCoordinator uses learning profiles for task assignment - extract_task_type() infers task type from title/description - assign_task() scores candidates using AgentScoringService - Fallback to load-based selection if no learning data available - Learning profiles stored in coordinator.learning_profiles RwLock ### Profile Adapter Enhancements (✅ Complete) - create_learning_profile() - initialize empty profiles - add_task_type_expertise() - set task-type expertise - update_profile_with_learning() - update swarm profiles from learning ## Files Modified ### vapora-knowledge-graph/src/persistence.rs (+30 lines) - get_executions_for_task_type(agent_id, task_type, limit) - get_agent_executions(agent_id, limit) ### vapora-agents/src/coordinator.rs (+100 lines) - load_learning_profile_from_kg() - core KG integration method - load_all_learning_profiles() - batch loading for agents - assign_task() already uses learning-based scoring via AgentScoringService ### Existing Complete Implementation - vapora-knowledge-graph/src/learning.rs - calculation functions - vapora-agents/src/learning_profile.rs - data structures and expertise - vapora-agents/src/scoring.rs - unified scoring service - vapora-agents/src/profile_adapter.rs - adapter methods ## Tests Passing - learning_profile: 7 tests ✅ - scoring: 5 tests ✅ - profile_adapter: 6 tests ✅ - coordinator: learning-specific tests ✅ ## Data Flow 1. Task arrives → AgentCoordinator::assign_task() 2. Extract task_type from description 3. Query KG for task-type executions (load_learning_profile_from_kg) 4. Calculate expertise with recency bias 5. Score candidates (SwarmCoordinator + learning) 6. Assign to top-scored agent 7. Execution result → KG → Update learning profiles ## Key Design Decisions ✅ Recency bias: 7-day half-life with 3x weight for recent performance ✅ Confidence scoring: min(1.0, total_executions / 20) prevents overfitting ✅ Hierarchical scoring: 30% base load, 50% expertise, 20% confidence ✅ KG query limit: 100 recent executions per task-type for performance ✅ Async loading: load_learning_profile_from_kg supports concurrent loads ## Next: Phase 5.4 - Cost Optimization Ready to implement budget enforcement and cost-aware provider selection.
289 lines
8.6 KiB
Rust
289 lines
8.6 KiB
Rust
// Integration tests for Swarm API endpoints
|
|
// Tests verify swarm statistics and health monitoring endpoints
|
|
|
|
use std::sync::Arc;
|
|
use vapora_swarm::{SwarmCoordinator, AgentProfile};
|
|
|
|
/// Helper to create a test agent profile
|
|
fn create_test_profile(id: &str, success_rate: f64, load: f64) -> AgentProfile {
|
|
AgentProfile {
|
|
id: id.to_string(),
|
|
roles: vec!["developer".to_string()],
|
|
capabilities: vec!["coding".to_string(), "testing".to_string()],
|
|
current_load: load,
|
|
success_rate,
|
|
availability: true,
|
|
}
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_coordinator_initialization() {
|
|
// Create a SwarmCoordinator
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
// Register test profiles
|
|
let profile1 = create_test_profile("agent-1", 0.95, 0.3);
|
|
let profile2 = create_test_profile("agent-2", 0.85, 0.5);
|
|
|
|
swarm.register_agent(profile1).ok();
|
|
swarm.register_agent(profile2).ok();
|
|
|
|
// Get statistics
|
|
let stats = swarm.get_swarm_stats();
|
|
|
|
// Verify statistics
|
|
assert_eq!(stats.total_agents, 2);
|
|
assert_eq!(stats.available_agents, 2);
|
|
assert!(stats.avg_load > 0.0);
|
|
assert!(stats.active_tasks == 0); // No tasks assigned yet
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_health_status_healthy() {
|
|
// Create swarm with available agents
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
let profile1 = create_test_profile("agent-1", 0.95, 0.3);
|
|
let profile2 = create_test_profile("agent-2", 0.90, 0.2);
|
|
|
|
swarm.register_agent(profile1).ok();
|
|
swarm.register_agent(profile2).ok();
|
|
|
|
let stats = swarm.get_swarm_stats();
|
|
|
|
// Verify health calculation
|
|
assert_eq!(stats.total_agents, 2);
|
|
assert_eq!(stats.available_agents, 2);
|
|
|
|
// All agents available = healthy
|
|
let is_healthy = stats.total_agents > 0 && stats.available_agents > 0;
|
|
assert!(is_healthy);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_health_status_degraded() {
|
|
// Create swarm with some unavailable agents
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
let available_profile = create_test_profile("agent-1", 0.95, 0.3);
|
|
let mut unavailable_profile = create_test_profile("agent-2", 0.85, 0.5);
|
|
unavailable_profile.availability = false;
|
|
|
|
swarm.register_agent(available_profile).ok();
|
|
swarm.register_agent(unavailable_profile).ok();
|
|
|
|
let stats = swarm.get_swarm_stats();
|
|
|
|
// Verify health calculation
|
|
assert_eq!(stats.total_agents, 2);
|
|
assert_eq!(stats.available_agents, 1);
|
|
|
|
// Some unavailable = degraded
|
|
let is_degraded = stats.total_agents > 0 && stats.available_agents < stats.total_agents;
|
|
assert!(is_degraded);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_health_status_no_agents() {
|
|
// Create empty swarm
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
let stats = swarm.get_swarm_stats();
|
|
|
|
// Verify no agents
|
|
assert_eq!(stats.total_agents, 0);
|
|
assert_eq!(stats.available_agents, 0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_statistics_load_calculation() {
|
|
// Create swarm with varied load profiles
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
let light_load = create_test_profile("agent-1", 0.95, 0.1);
|
|
let medium_load = create_test_profile("agent-2", 0.85, 0.5);
|
|
let high_load = create_test_profile("agent-3", 0.80, 0.9);
|
|
|
|
swarm.register_agent(light_load).ok();
|
|
swarm.register_agent(medium_load).ok();
|
|
swarm.register_agent(high_load).ok();
|
|
|
|
let stats = swarm.get_swarm_stats();
|
|
|
|
// Verify load calculation (average of 0.1, 0.5, 0.9 = 0.5)
|
|
assert_eq!(stats.total_agents, 3);
|
|
assert!(stats.avg_load > 0.4 && stats.avg_load < 0.6);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_statistics_success_rate_variance() {
|
|
// Create swarm with different success rates
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
let high_success = create_test_profile("agent-1", 0.99, 0.2);
|
|
let medium_success = create_test_profile("agent-2", 0.50, 0.3);
|
|
let low_success = create_test_profile("agent-3", 0.10, 0.1);
|
|
|
|
swarm.register_agent(high_success).ok();
|
|
swarm.register_agent(medium_success).ok();
|
|
swarm.register_agent(low_success).ok();
|
|
|
|
let stats = swarm.get_swarm_stats();
|
|
|
|
// Verify all agents registered despite variance
|
|
assert_eq!(stats.total_agents, 3);
|
|
assert_eq!(stats.available_agents, 3);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_agent_availability_transitions() {
|
|
// Create swarm with available agent
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
let mut profile = create_test_profile("agent-1", 0.95, 0.3);
|
|
swarm.register_agent(profile.clone()).ok();
|
|
|
|
// Verify initial state
|
|
let mut stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.available_agents, 1);
|
|
|
|
// Mark agent unavailable
|
|
profile.availability = false;
|
|
swarm.register_agent(profile).ok();
|
|
|
|
// Verify transition
|
|
stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.available_agents, 0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_unregister_agent() {
|
|
// Create swarm with agent
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
let profile = create_test_profile("agent-1", 0.95, 0.3);
|
|
swarm.register_agent(profile).ok();
|
|
|
|
let mut stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.total_agents, 1);
|
|
|
|
// Unregister agent
|
|
swarm.unregister_agent("agent-1").ok();
|
|
|
|
// Verify removal
|
|
stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.total_agents, 0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_task_assignment_selects_best_agent() {
|
|
// Create swarm with agents of different quality
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
let poor_agent = create_test_profile("agent-poor", 0.50, 0.9); // Low success, high load
|
|
let good_agent = create_test_profile("agent-good", 0.95, 0.2); // High success, low load
|
|
|
|
swarm.register_agent(poor_agent).ok();
|
|
swarm.register_agent(good_agent).ok();
|
|
|
|
// Score: success_rate / (1.0 + load)
|
|
// agent-poor: 0.50 / (1.0 + 0.9) = 0.50 / 1.9 ≈ 0.26
|
|
// agent-good: 0.95 / (1.0 + 0.2) = 0.95 / 1.2 ≈ 0.79
|
|
// agent-good should be selected
|
|
|
|
// Verify agent-good has better score
|
|
let poor_score = 0.50 / (1.0 + 0.9);
|
|
let good_score = 0.95 / (1.0 + 0.2);
|
|
assert!(good_score > poor_score);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_statistics_consistency() {
|
|
// Test that statistics remain consistent with multiple operations
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
// Initial state
|
|
let mut stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.total_agents, 0);
|
|
|
|
// Add agents
|
|
for i in 0..5 {
|
|
let profile = create_test_profile(&format!("agent-{}", i), 0.85, 0.3);
|
|
swarm.register_agent(profile).ok();
|
|
}
|
|
|
|
stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.total_agents, 5);
|
|
assert_eq!(stats.available_agents, 5);
|
|
|
|
// Update one agent to unavailable
|
|
let mut profile = create_test_profile("agent-0", 0.85, 0.3);
|
|
profile.availability = false;
|
|
swarm.register_agent(profile).ok();
|
|
|
|
stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.total_agents, 5);
|
|
assert_eq!(stats.available_agents, 4);
|
|
|
|
// Remove one agent
|
|
swarm.unregister_agent("agent-1").ok();
|
|
|
|
stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.total_agents, 4);
|
|
assert_eq!(stats.available_agents, 3);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_large_agent_pool() {
|
|
// Test swarm behavior with larger agent pool
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
// Register 50 agents with varied metrics
|
|
for i in 0..50 {
|
|
let success_rate = if i % 3 == 0 {
|
|
0.95
|
|
} else if i % 3 == 1 {
|
|
0.75
|
|
} else {
|
|
0.55
|
|
};
|
|
|
|
let load = (i as f64 % 10.0) / 10.0;
|
|
|
|
let profile = create_test_profile(&format!("agent-{}", i), success_rate, load);
|
|
swarm.register_agent(profile).ok();
|
|
}
|
|
|
|
let stats = swarm.get_swarm_stats();
|
|
|
|
// Verify all registered
|
|
assert_eq!(stats.total_agents, 50);
|
|
assert_eq!(stats.available_agents, 50);
|
|
|
|
// Verify average load is reasonable
|
|
assert!(stats.avg_load > 0.0 && stats.avg_load < 1.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_swarm_empty_after_unregister_all() {
|
|
// Create swarm with agents
|
|
let swarm = Arc::new(SwarmCoordinator::new());
|
|
|
|
for i in 0..3 {
|
|
let profile = create_test_profile(&format!("agent-{}", i), 0.85, 0.3);
|
|
swarm.register_agent(profile).ok();
|
|
}
|
|
|
|
let mut stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.total_agents, 3);
|
|
|
|
// Unregister all
|
|
for i in 0..3 {
|
|
swarm.unregister_agent(&format!("agent-{}", i)).ok();
|
|
}
|
|
|
|
stats = swarm.get_swarm_stats();
|
|
assert_eq!(stats.total_agents, 0);
|
|
assert_eq!(stats.available_agents, 0);
|
|
}
|