Vapora/docs/adrs/0023-testing-strategy.md

# ADR-023: Multi-Layer Testing Strategy

**Status**: Accepted | Implemented
**Date**: 2024-11-01
**Deciders**: Quality Assurance Team
**Technical Story**: Building confidence through unit, integration, and real-database tests

---

## Decision

Implementar **multi-layer testing**: unit tests (inline), integration tests (tests/ dir), real DB connections.

---

## Rationale

1. **Unit Tests**: Fast feedback on logic changes
2. **Integration Tests**: Verify components work together
3. **Real DB Tests**: Catch database schema/query issues
4. **218+ Tests**: Comprehensive coverage across 13 crates

---

## Alternatives Considered

### ❌ Unit Tests Only
- **Pros**: Fast
- **Cons**: Miss integration bugs, schema issues

### ❌ Integration Tests Only
- **Pros**: Comprehensive
- **Cons**: Slow, harder to debug

### ✅ Multi-Layer (CHOSEN)
- All three layers catch different issues

---

## Trade-offs

**Pros**:
- ✅ Fast feedback (unit)
- ✅ Integration validation (integration)
- ✅ Real-world confidence (real DB)
- ✅ 218+ tests total coverage

**Cons**:
- ⚠️ Slow full test suite (~5 minutes)
- ⚠️ DB tests require test environment
- ⚠️ More test code to maintain

---

## Implementation

**Unit Tests (Inline)**:
```rust
// crates/vapora-agents/src/learning_profile.rs

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_expertise_score_empty() {
        let profile = TaskTypeLearning {
            agent_id: "test".to_string(),
            task_type: "architecture".to_string(),
            executions_total: 0,
            records: vec![],
            ..Default::default()
        };

        assert_eq!(profile.expertise_score(), 0.0);
    }

    #[test]
    fn test_confidence_weighting() {
        let profile = TaskTypeLearning {
            executions_total: 20,
            ..Default::default()
        };
        assert_eq!(profile.confidence(), 1.0);

        let profile_partial = TaskTypeLearning {
            executions_total: 10,
            ..Default::default()
        };
        assert_eq!(profile_partial.confidence(), 0.5);
    }
}
```

**Integration Tests**:
```rust
// crates/vapora-backend/tests/integration_tests.rs

#[tokio::test]
async fn test_create_project_full_flow() {
    // Setup: create test database
    let db = setup_test_db().await;
    let app_state = create_test_app_state(db.clone()).await;

    // Execute: create project via HTTP
    let response = app_state
        .handle_request(
            "POST",
            "/api/projects",
            json!({
                "title": "Test Project",
                "description": "A test",
            }),
        )
        .await;

    // Verify: response is 201 Created
    assert_eq!(response.status(), 201);

    // Verify: project in database
    let project = db
        .query("SELECT * FROM projects LIMIT 1")
        .await
        .unwrap()
        .take::<Project>(0)
        .unwrap()
        .unwrap();

    assert_eq!(project.title, "Test Project");
}
```

**Real Database Tests**:
```rust
// crates/vapora-backend/tests/database_tests.rs

#[tokio::test]
async fn test_multi_tenant_isolation() {
    let db = setup_real_surrealdb().await;

    // Create projects for two tenants
    let project_1 = db
        .create("projects")
        .content(Project {
            tenant_id: "tenant:1".to_string(),
            title: "Project 1".to_string(),
            ..Default::default()
        })
        .await
        .unwrap();

    let project_2 = db
        .create("projects")
        .content(Project {
            tenant_id: "tenant:2".to_string(),
            title: "Project 2".to_string(),
            ..Default::default()
        })
        .await
        .unwrap();

    // Query: tenant 1 should only see their project
    let results = db
        .query("SELECT * FROM projects WHERE tenant_id = 'tenant:1'")
        .await
        .unwrap()
        .take::<Vec<Project>>(0)
        .unwrap();

    assert_eq!(results.len(), 1);
    assert_eq!(results[0].title, "Project 1");
}
```

**Test Utilities**:
```rust
// crates/vapora-backend/tests/common/mod.rs

pub async fn setup_test_db() -> Surreal<Mem> {
    let db = Surreal::new::<surrealdb::engine::local::Mem>()
        .await
        .unwrap();

    db.use_ns("vapora").use_db("test").await.unwrap();

    // Initialize schema
    init_schema(&db).await.unwrap();

    db
}

pub async fn setup_real_surrealdb() -> Surreal<Ws> {
    // Connect to test SurrealDB instance
    let db = Surreal::new::<Ws>("ws://localhost:8000")
        .await
        .unwrap();

    db.signin(/* test credentials */).await.unwrap();
    db.use_ns("test").use_db("test").await.unwrap();

    db
}
```

**Running Tests**:
```bash
# Run all tests
cargo test --workspace

# Run unit tests only (fast)
cargo test --workspace --lib

# Run integration tests
cargo test --workspace --test "*"

# Run with output
cargo test --workspace -- --nocapture

# Run specific test
cargo test -p vapora-backend test_multi_tenant_isolation

# Coverage report
cargo tarpaulin --workspace --out Html
```

**Key Files**:
- `crates/*/src/` (unit tests inline)
- `crates/*/tests/` (integration tests)
- `crates/*/tests/common/` (test utilities)

---

## Verification

```bash
# Count tests across workspace
cargo test --workspace -- --list | grep "test " | wc -l

# Run all tests with statistics
cargo test --workspace 2>&1 | grep -E "^test |passed|failed"

# Coverage report
cargo tarpaulin --workspace --out Html
# Output: coverage/index.html
```

**Expected Output**:
- 218+ tests total
- All tests passing
- Coverage > 70%
- Unit tests < 5 seconds
- Integration tests < 60 seconds

---

## Consequences

### Testing Cadence
- Pre-commit: run unit tests
- PR: run all tests
- CI/CD: run all tests + coverage

### Test Environment
- Unit tests: in-memory databases
- Integration: SurrealDB in-memory
- Real DB: Docker container (CI/CD only)

### Debugging
- Unit test failure: easy to debug (isolated)
- Integration failure: check component interaction
- DB failure: verify schema and queries

---

## References

- [Rust Testing Documentation](https://doc.rust-lang.org/book/ch11-00-testing.html)
- `crates/*/tests/` (integration tests)
- `crates/vapora-backend/tests/common/` (test utilities)

---

**Related ADRs**: ADR-022 (Error Handling), ADR-004 (SurrealDB)