syntaxis/docs/databases/surrealdb/surrealdb-implementation-plan.md

# SurrealDB Implementation Plan - Step-by-Step Guide

**Status:** Ready for implementation
**Timeline:** 5-6 weeks
**Effort:** 40-50 hours
**Team:** 1 developer + 0.5 QA + 0.5 DevOps

---

## Phase 1: Design & Planning (Week 1)

### Step 1.1: Analyze Current Architecture (2 hours)

**Objective:** Understand existing code before making changes

```bash
# Current structure
syntaxis-core/src/
├── persistence.rs    # ← 1,492 lines, tightly coupled to SQLite
├── lib.rs
├── error.rs
├── config.rs
└── ... other modules

# What needs to change:
# 1. Extract persistence.rs into module with trait
# 2. Create adapters for SQLite and SurrealDB
# 3. Add configuration layer
# 4. Add migration tool
```

**Deliverable:**
- [ ] Document current persistence layer structure
- [ ] Identify all 50+ database methods that need trait
- [ ] List all dependencies on SqlitePool
- [ ] Identify configuration injection points

**Commands:**
```bash
# Find all sqlx usage
grep -r "sqlx::" core/crates/syntaxis-core/src/persistence.rs | wc -l

# Count methods in persistence.rs
grep -E "pub async fn" core/crates/syntaxis-core/src/persistence.rs | wc -l

# Check imports
head -20 core/crates/syntaxis-core/src/persistence.rs
```

### Step 1.2: Design Database Trait Interface (3 hours)

**Objective:** Define the abstraction boundary

**File:** `SURREALDB_EXAMPLE.rs` already has this, but create actual working version

**Create:** `syntaxis-core/src/persistence/mod.rs` (new file structure)

```rust
// syntaxis-core/src/persistence/mod.rs

use async_trait::async_trait;
use crate::error::Result;

#[async_trait]
pub trait Database: Send + Sync + Clone {
    // ====== Project Operations ======
    async fn create_project(&self, name: &str, description: &str, project_type: &str) -> Result<Project>;
    async fn get_project(&self, id: &str) -> Result<Project>;
    async fn list_projects(&self) -> Result<Vec<Project>>;
    async fn update_project(&self, id: &str, name: &str, description: &str) -> Result<Project>;
    async fn delete_project(&self, id: &str) -> Result<()>;

    // ====== Checklist Operations ======
    async fn create_checklist_item(&self, project_id: &str, item: &ChecklistItem) -> Result<ChecklistItem>;
    async fn list_checklist_items(&self, project_id: &str, phase: &str) -> Result<Vec<ChecklistItem>>;
    async fn update_checklist_item(&self, id: &str, completed: bool) -> Result<ChecklistItem>;
    async fn delete_checklist_item(&self, id: &str) -> Result<()>;

    // ====== Phase Operations ======
    async fn record_phase_transition(&self, transition: &PhaseTransition) -> Result<()>;
    async fn get_phase_history(&self, project_id: &str) -> Result<Vec<PhaseTransition>>;
    async fn get_current_phase(&self, project_id: &str) -> Result<String>;

    // ====== More operations... ======
    // (Copy all remaining methods from persistence.rs)
}

pub mod sqlite_impl;
pub mod surrealdb_impl;

pub use sqlite_impl::SqliteDatabase;
pub use surrealdb_impl::SurrealDatabase;
```

**Deliverable:**
- [ ] Database trait with all 50+ method signatures
- [ ] Comprehensive rustdoc for each method
- [ ] Error handling strategy defined
- [ ] Test cases for trait interface

### Step 1.3: Plan Configuration System (2 hours)

**Objective:** Design how to switch backends via config

**File:** Create `syntaxis-core/src/persistence/config.rs`

```rust
use serde::{Deserialize, Serialize};

#[derive(Debug, Deserialize, Serialize)]
pub struct DatabaseConfig {
    pub engine: String,  // "sqlite" or "surrealdb"
    pub sqlite: Option<SqliteConfig>,
    pub surrealdb: Option<SurrealDBConfig>,
}

#[derive(Debug, Deserialize, Serialize)]
pub struct SqliteConfig {
    pub path: String,
    pub max_connections: u32,
    pub timeout_secs: u64,
}

#[derive(Debug, Deserialize, Serialize)]
pub struct SurrealDBConfig {
    pub url: String,
    pub namespace: String,
    pub database: String,
    pub username: String,
    pub password: String,
    pub max_connections: u32,
    pub timeout_secs: u64,
}

impl DatabaseConfig {
    pub async fn create_database(&self) -> Result<Box<dyn Database>> {
        match self.engine.as_str() {
            "sqlite" => {
                let config = self.sqlite.as_ref().ok_or(/* error */)?;
                let db = SqliteDatabase::new(&config.path).await?;
                Ok(Box::new(db))
            }
            "surrealdb" => {
                let config = self.surrealdb.as_ref().ok_or(/* error */)?;
                let db = SurrealDatabase::new(&config.url, &config.namespace, &config.database).await?;
                Ok(Box::new(db))
            }
            _ => Err(LifecycleError::Config(format!("Unknown database engine: {}", self.engine))),
        }
    }
}
```

**Configuration Files:**

```toml
# configs/database-sqlite.toml (default)
[database]
engine = "sqlite"

[database.sqlite]
path = "~/.local/share/core/workspace.db"
max_connections = 5
timeout_secs = 30
```

```toml
# configs/database-surrealdb.toml (cloud-ready)
[database]
engine = "surrealdb"

[database.surrealdb]
url = "ws://surrealdb-server:8000"
namespace = "syntaxis"
database = "projects"
username = "${SURREALDB_USER}"
password = "${SURREALDB_PASSWORD}"
max_connections = 10
timeout_secs = 30
```

**Deliverable:**
- [ ] DatabaseConfig struct defined
- [ ] Configuration loading implemented
- [ ] Environment variable support
- [ ] Configuration validation
- [ ] Example TOML files

### Step 1.4: Testing Strategy (1 hour)

**Objective:** Plan how to test both backends

**Create test utilities:**

```rust
// syntaxis-core/src/persistence/tests/mod.rs

#[cfg(test)]
mod common {
    use crate::persistence::Database;

    /// Run same tests against multiple database implementations
    pub async fn test_create_project_sqlite() { /* ... */ }
    pub async fn test_create_project_surrealdb() { /* ... */ }

    /// Mock database for unit testing business logic
    pub struct MockDatabase { /* ... */ }
    #[async_trait]
    impl Database for MockDatabase { /* ... */ }
}
```

**Deliverable:**
- [ ] Test infrastructure designed
- [ ] Mock Database implementation planned
- [ ] Test cases for each method
- [ ] Integration test strategy

---

## Phase 2: Trait Layer & SQLite Refactoring (Weeks 1-2)

### Step 2.1: Extract Database Trait (6 hours)

**Objective:** Create abstraction layer without changing logic

**Files to create:**
```
syntaxis-core/src/persistence/
├── mod.rs              # Trait definition + module exports
├── config.rs           # Configuration structs
├── error.rs            # Unified error types
├── sqlite_impl.rs      # SQLite adapter (moved from persistence.rs)
├── tests/
│   ├── common.rs       # Shared test utilities
│   └── sqlite_tests.rs # SQLite-specific tests
└── mock.rs             # Mock implementation for tests
```

**Step 2.1a: Create trait module**

```bash
# Backup current persistence.rs
cp core/crates/syntaxis-core/src/persistence.rs \
   core/crates/syntaxis-core/src/persistence.rs.backup

# Create new structure
mkdir -p core/crates/syntaxis-core/src/persistence/tests
```

**Step 2.1b: Move code**

1. Copy `persistence.rs` to `sqlite_impl.rs`
2. Create new `persistence/mod.rs` with trait
3. Update `lib.rs` exports
4. Run tests to ensure no breakage

**Step 2.1c: Update imports everywhere**

```bash
# Find all files using PersistenceLayer
grep -r "use.*persistence::PersistenceLayer" core/crates/

# Update to use trait instead
# Before: use workspace_core::PersistenceLayer;
# After:  use workspace_core::persistence::{Database, SqliteDatabase};
```

**Deliverable:**
- [ ] Database trait defined (50+ methods)
- [ ] SqliteDatabase implements Database
- [ ] No logic changes (refactoring only)
- [ ] All existing tests pass
- [ ] Backward compatibility maintained

### Step 2.2: Configuration Integration (4 hours)

**Objective:** Load database from config, not hardcoded

**Step 2.2a: Update config loading**

In `syntaxis-core/src/config.rs` or wherever config lives:

```rust
pub struct Config {
    pub database: DatabaseConfig,
    // ... other config
}

impl Config {
    pub fn load() -> Result<Self> {
        // Load from TOML
        let contents = std::fs::read_to_string("configs/database.toml")?;
        let config: Self = toml::from_str(&contents)?;
        Ok(config)
    }

    pub async fn create_database(&self) -> Result<Box<dyn Database>> {
        self.database.create_database().await
    }
}
```

**Step 2.2b: Update main.rs in binaries**

In `syntaxis-cli/src/main.rs`:

```rust
// Before
let db = PersistenceLayer::new("workspace.db").await?;

// After
let config = Config::load()?;
let db = config.create_database().await?;
```

**Deliverable:**
- [ ] Configuration loading integrated
- [ ] DatabaseConfig struct implemented
- [ ] All binaries use config-based initialization
- [ ] Environment variables supported
- [ ] Default configs provided

### Step 2.3: Testing SQLite Adapter (4 hours)

**Objective:** Verify SQLite still works after refactoring

**Create comprehensive tests:**

```rust
// syntaxis-core/src/persistence/tests/sqlite_tests.rs

#[tokio::test]
async fn test_sqlite_create_project() {
    let db = SqliteDatabase::new_memory().await.unwrap();
    let project = db.create_project("Test", "Desc", "MultiLang").await.unwrap();
    assert_eq!(project.name, "Test");
}

#[tokio::test]
async fn test_sqlite_list_projects() {
    let db = SqliteDatabase::new_memory().await.unwrap();
    let _ = db.create_project("P1", "D1", "MultiLang").await.unwrap();
    let _ = db.create_project("P2", "D2", "MultiLang").await.unwrap();
    let projects = db.list_projects().await.unwrap();
    assert_eq!(projects.len(), 2);
}

// ... 50+ more tests
```

**Run tests:**

```bash
cargo test -p syntaxis-core --lib persistence

# Expected: 173 tests pass (all SQLite tests)
```

**Deliverable:**
- [ ] 170+ SQLite tests passing
- [ ] Full backward compatibility verified
- [ ] No performance regressions
- [ ] All methods tested

---

## Phase 3: SurrealDB Implementation (Weeks 2-3)

### Step 3.1: Create SurrealDB Adapter (8 hours)

**Objective:** Implement Database trait for SurrealDB

**File:** `syntaxis-core/src/persistence/surrealdb_impl.rs`

```rust
use surrealdb::{Client, Surreal, opt::auth::Root};

#[derive(Clone)]
pub struct SurrealDatabase {
    db: Surreal<Client>,
}

impl SurrealDatabase {
    pub async fn new(url: &str, namespace: &str, database: &str) -> Result<Self> {
        // Connect to SurrealDB
        let db = Surreal::new::<Ws>(url)
            .await
            .map_err(|e| LifecycleError::Database(e.to_string()))?;

        // Authenticate
        db.signin(Root {
            username: "root",
            password: "root",
        })
        .await
        .map_err(|e| LifecycleError::Database(e.to_string()))?;

        // Select namespace and database
        db.use_ns(namespace).use_db(database).await
            .map_err(|e| LifecycleError::Database(e.to_string()))?;

        // Initialize schema
        Self::init_schema(&db).await?;

        Ok(Self { db })
    }

    async fn init_schema(db: &Surreal<Client>) -> Result<()> {
        // Define collections, indices, relationships
        // See SURREALDB_EXAMPLE.rs for details
        Ok(())
    }
}

#[async_trait]
impl Database for SurrealDatabase {
    async fn create_project(&self, name: &str, description: &str, project_type: &str) -> Result<Project> {
        // Implementation from SURREALDB_EXAMPLE.rs
        let id = uuid::Uuid::new_v4().to_string();
        let now = chrono::Utc::now().to_rfc3339();

        let project = Project {
            id: id.clone(),
            name: name.to_string(),
            version: "0.1.0".to_string(),
            description: description.to_string(),
            project_type: project_type.to_string(),
            current_phase: "Creation".to_string(),
            created_at: now.clone(),
            updated_at: now,
        };

        self.db.create::<Project>(("projects", &id))
            .content(&project)
            .await?;

        Ok(project)
    }

    // ... implement all other trait methods
}
```

**Key Implementation Points:**

1. **Connection Management**
   - Use connection pooling
   - Handle authentication
   - Manage namespace/database selection

2. **Schema Definition**
   - Create collections (projects, checklist_items, etc.)
   - Define relationships (RELATE for phase transitions)
   - Create indices for performance

3. **Query Implementation**
   - Use SurrealQL for document queries
   - Use graph syntax for relationships
   - Handle transactions

**Deliverable:**
- [ ] SurrealDatabase struct implemented
- [ ] All 50+ methods implemented
- [ ] Schema initialization working
- [ ] Connection pooling configured

### Step 3.2: Schema Definition (4 hours)

**Objective:** Define SurrealDB schema structure

**File:** `syntaxis-core/src/persistence/surrealdb_impl.rs` (init_schema method)

```rust
async fn init_schema(db: &Surreal<Client>) -> Result<()> {
    // Define collections
    db.query(
        "DEFINE TABLE projects SCHEMAFULL
            COMMENT = 'Project collection';"
    )
    .await?;

    // Define fields
    db.query("DEFINE FIELD projects.id AS string")
        .await?;
    db.query("DEFINE FIELD projects.name AS string")
        .await?;
    db.query("DEFINE FIELD projects.current_phase AS enum<Creation,Development,Testing,Publishing,Archived>")
        .await?;

    // Define relationships
    db.query(
        "DEFINE TABLE transitioned_to SCHEMALESS
            COMMENT = 'Phase transition relationships';"
    )
    .await?;

    // Create indices
    db.query("DEFINE INDEX idx_projects_created ON TABLE projects COLUMNS created_at DESC")
        .await?;

    Ok(())
}
```

**Deliverable:**
- [ ] All collections defined
- [ ] All relationships defined
- [ ] All indices created
- [ ] Schema validation passing

### Step 3.3: Query Optimization (4 hours)

**Objective:** Ensure queries are efficient

**Test patterns:**

```rust
#[tokio::test]
async fn test_surrealdb_graph_query() {
    let db = SurrealDatabase::new("ws://localhost:8000", "test", "test").await.unwrap();

    // Create project
    let project = db.create_project("Test", "Desc", "MultiLang").await.unwrap();

    // Record transition
    db.record_phase_transition(&PhaseTransition {
        project_id: project.id.clone(),
        from_phase: "Creation".to_string(),
        to_phase: "Development".to_string(),
        timestamp: chrono::Utc::now().to_rfc3339(),
        reason: Some("Started dev".to_string()),
    }).await.unwrap();

    // Query graph - should be fast
    let transitions = db.get_phase_history(&project.id).await.unwrap();
    assert_eq!(transitions.len(), 1);
}
```

**Deliverable:**
- [ ] All queries tested and optimized
- [ ] Performance benchmarks < 100ms
- [ ] Graph queries efficient
- [ ] Indices properly used

### Step 3.4: Testing SurrealDB Adapter (4 hours)

**Objective:** Verify SurrealDB implementation matches SQLite behavior

```rust
// syntaxis-core/src/persistence/tests/surrealdb_tests.rs

#[tokio::test]
async fn test_surrealdb_create_project() {
    let db = SurrealDatabase::new("ws://localhost:8000", "test", "test").await.unwrap();
    let project = db.create_project("Test", "Desc", "MultiLang").await.unwrap();
    assert_eq!(project.name, "Test");
}

// ... mirror all SQLite tests for SurrealDB
```

**Setup for tests:**

```bash
# Docker Compose for local testing
# docker-compose.yml
version: '3'
services:
  surrealdb:
    image: surrealdb/surrealdb:latest
    ports:
      - "8000:8000"
    command: start ws://0.0.0.0:8000
```

**Deliverable:**
- [ ] 170+ SurrealDB tests passing
- [ ] Feature parity with SQLite verified
- [ ] Docker setup for local testing
- [ ] Performance benchmarks documented

---

## Phase 4: Migration Tool (Week 3)

### Step 4.1: Design Migration Architecture (2 hours)

**Objective:** Plan how to migrate data safely

**Strategy:**

```
1. Read from SQLite
2. Transform to SurrealDB format
3. Write to SurrealDB
4. Verify integrity
5. Allow rollback
```

**File:** `syntaxis-core/src/persistence/migration.rs`

```rust
pub struct Migrator;

impl Migrator {
    pub async fn migrate_sqlite_to_surrealdb(
        from: &SqliteDatabase,
        to: &SurrealDatabase,
        backup: bool,
    ) -> Result<MigrationReport> {
        // 1. Backup SQLite if requested
        if backup {
            Self::backup_sqlite().await?;
        }

        // 2. Read all data from SQLite
        let projects = from.list_projects().await?;
        let mut report = MigrationReport::default();

        // 3. Write to SurrealDB
        for project in projects {
            to.db.create::<Project>(("projects", &project.id))
                .content(&project)
                .await?;
            report.projects_migrated += 1;
        }

        // 4. Verify integrity
        let surrealdb_projects = to.list_projects().await?;
        if surrealdb_projects.len() != projects.len() {
            return Err(LifecycleError::Migration("Data count mismatch".into()));
        }

        Ok(report)
    }

    pub async fn rollback(backup_path: &str) -> Result<()> {
        // Restore from backup
        Ok(())
    }
}
```

**Deliverable:**
- [ ] Migration strategy documented
- [ ] Data transformation logic designed
- [ ] Rollback strategy designed
- [ ] Verification logic planned

### Step 4.2: CLI Commands (3 hours)

**Objective:** Add migration commands to workspace CLI

**File:** `syntaxis-cli/src/handlers/database.rs` (new)

```rust
use clap::{Subcommand, Args};

#[derive(Args)]
pub struct DatabaseCmd {
    #[command(subcommand)]
    pub command: DatabaseSubcommand,
}

#[derive(Subcommand)]
pub enum DatabaseSubcommand {
    /// Show current database backend
    Status,

    /// List available database backends
    ListBackends,

    /// Migrate data between databases
    Migrate {
        #[arg(long)]
        from: String,  // "sqlite"

        #[arg(long)]
        to: String,    // "surrealdb"

        #[arg(long)]
        target_url: Option<String>,

        #[arg(long)]
        backup: bool,

        #[arg(long)]
        dry_run: bool,
    },

    /// Rollback previous migration
    Rollback {
        #[arg(long)]
        from_backup: String,
    },

    /// Validate database integrity
    Validate,
}

impl DatabaseCmd {
    pub async fn execute(self) -> Result<()> {
        match self.command {
            DatabaseSubcommand::Status => {
                let config = Config::load()?;
                println!("Current backend: {}", config.database.engine);
                Ok(())
            }
            DatabaseSubcommand::Migrate { from, to, backup, dry_run, .. } => {
                if dry_run {
                    println!("DRY RUN: Would migrate from {} to {}", from, to);
                    return Ok(());
                }

                let from_db = /* create from database */;
                let to_db = /* create to database */;

                let report = Migrator::migrate_sqlite_to_surrealdb(&from_db, &to_db, backup).await?;
                println!("Migration complete: {:?}", report);
                Ok(())
            }
            // ... other commands
        }
    }
}
```

**Update main CLI:**

```rust
// syntaxis-cli/src/main.rs
match args.command {
    Commands::Database(cmd) => cmd.execute().await?,
    // ... other commands
}
```

**Deliverable:**
- [ ] `workspace database status` command
- [ ] `workspace database migrate --from sqlite --to surrealdb` command
- [ ] `workspace database migrate --rollback` command
- [ ] `workspace database validate` command
- [ ] Help text complete

### Step 4.3: Migration Testing (2 hours)

**Objective:** Verify migration works correctly

```rust
#[tokio::test]
async fn test_migrate_sqlite_to_surrealdb_small_dataset() {
    // Create test data in SQLite
    let sqlite_db = SqliteDatabase::new_memory().await.unwrap();
    let _ = sqlite_db.create_project("P1", "D1", "MultiLang").await.unwrap();
    let _ = sqlite_db.create_project("P2", "D2", "MultiLang").await.unwrap();

    // Migrate to SurrealDB
    let surrealdb = /* connect to test instance */;
    let report = Migrator::migrate_sqlite_to_surrealdb(&sqlite_db, &surrealdb, true).await.unwrap();

    // Verify
    assert_eq!(report.projects_migrated, 2);
    let projects = surrealdb.list_projects().await.unwrap();
    assert_eq!(projects.len(), 2);
}

#[tokio::test]
async fn test_migrate_with_relationships() {
    // Test migration with complex data (phases, checklists, etc.)
}

#[tokio::test]
async fn test_migration_rollback() {
    // Test rollback functionality
}
```

**Deliverable:**
- [ ] Migration tests passing
- [ ] Small dataset migration verified
- [ ] Large dataset migration tested
- [ ] Rollback functionality verified

---

## Phase 5: Runtime Selection (Weeks 3-4)

### Step 5.1: Backend Enum (2 hours)

**Objective:** Switch backends at runtime

**File:** `syntaxis-core/src/persistence/mod.rs` (add)

```rust
pub enum DatabaseBackend {
    Sqlite(SqliteDatabase),
    SurrealDB(SurrealDatabase),
}

#[async_trait]
impl Database for DatabaseBackend {
    async fn create_project(&self, name: &str, description: &str, project_type: &str) -> Result<Project> {
        match self {
            DatabaseBackend::Sqlite(db) => db.create_project(name, description, project_type).await,
            DatabaseBackend::SurrealDB(db) => db.create_project(name, description, project_type).await,
        }
    }

    // ... delegate all methods
}
```

**Configuration-driven initialization:**

```rust
pub async fn create_database(config: &DatabaseConfig) -> Result<DatabaseBackend> {
    match config.engine.as_str() {
        "sqlite" => {
            let sqlite = config.sqlite.as_ref().ok_or(/* error */)?;
            let db = SqliteDatabase::new(&sqlite.path).await?;
            Ok(DatabaseBackend::Sqlite(db))
        }
        "surrealdb" => {
            let surrealdb = config.surrealdb.as_ref().ok_or(/* error */)?;
            let db = SurrealDatabase::new(&surrealdb.url, &surrealdb.namespace, &surrealdb.database).await?;
            Ok(DatabaseBackend::SurrealDB(db))
        }
        _ => Err(LifecycleError::Config(format!("Unknown engine: {}", config.engine))),
    }
}
```

**Deliverable:**
- [ ] DatabaseBackend enum implemented
- [ ] Runtime selection working
- [ ] Configuration-driven initialization
- [ ] Transparent to application code

### Step 5.2: Binary Integration (4 hours)

**Objective:** Update all binaries to use new system

**syntaxis-cli:**
```rust
// src/main.rs
#[tokio::main]
async fn main() -> Result<()> {
    let config = Config::load()?;
    let db = config.create_database().await?;

    match args.command {
        Commands::Project(cmd) => cmd.execute(&db).await?,
        // ... other commands
    }
    Ok(())
}
```

**syntaxis-tui:**
```rust
// src/main.rs
#[tokio::main]
async fn main() -> Result<()> {
    let config = Config::load()?;
    let db = config.create_database().await?;

    let mut app = App::new(db);
    app.run().await?;
    Ok(())
}
```

**syntaxis-api:**
```rust
// src/main.rs
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = Config::load()?;
    let db = config.create_database().await?;

    let state = AppState::new(db);
    // ... setup and run server
    Ok(())
}
```

**Deliverable:**
- [ ] syntaxis-cli updated
- [ ] syntaxis-tui updated
- [ ] syntaxis-api updated
- [ ] syntaxis-dashboard updated
- [ ] All tests passing

---

## Phase 6: Integration & Testing (Week 4)

### Step 6.1: Unit Tests (8 hours)

**Objective:** Test both backends independently

```bash
# Run SQLite tests
cargo test -p syntaxis-core --lib persistence::tests::sqlite_tests

# Run SurrealDB tests (with docker-compose running)
cargo test -p syntaxis-core --lib persistence::tests::surrealdb_tests

# Run configuration tests
cargo test -p syntaxis-core --lib persistence::tests::config_tests

# Run migration tests
cargo test -p syntaxis-core --lib persistence::tests::migration_tests

# Expected: 340+ tests passing (170 per backend)
```

**Deliverable:**
- [ ] 170+ SQLite tests passing
- [ ] 170+ SurrealDB tests passing
- [ ] Configuration tests passing
- [ ] Migration tests passing
- [ ] >80% code coverage

### Step 6.2: Integration Tests (4 hours)

**Objective:** Test multiple components together

```bash
# Test CLI with SQLite
cargo test -p syntaxis-cli --test integration_sqlite

# Test CLI with SurrealDB
cargo test -p syntaxis-cli --test integration_surrealdb

# Test API with both backends
cargo test -p syntaxis-api --test integration
```

**Deliverable:**
- [ ] CLI integration tests passing
- [ ] TUI integration tests passing
- [ ] API integration tests passing
- [ ] Dashboard integration tests passing

### Step 6.3: E2E Tests (6 hours)

**Objective:** Test real workflows

```bash
# Complete workflow: Create project → Add checklist → Transition phases

# Test with SQLite
./scripts/test-e2e-sqlite.nu

# Test with SurrealDB
./scripts/test-e2e-surrealdb.nu

# Test migration
./scripts/test-migration-e2e.nu
```

**Deliverable:**
- [ ] Create project workflow tested
- [ ] Add tasks workflow tested
- [ ] Phase transition workflow tested
- [ ] Security assessment workflow tested
- [ ] Migration workflow tested

### Step 6.4: Performance Benchmarks (4 hours)

**Objective:** Compare SQLite vs SurrealDB performance

```rust
use criterion::black_box;

#[bench]
fn bench_sqlite_create_project(b: &mut Bencher) {
    let rt = Runtime::new().unwrap();
    let db = rt.block_on(SqliteDatabase::new_memory()).unwrap();

    b.to_async(&rt).iter(|| async {
        db.create_project(black_box("Test"), black_box("Desc"), black_box("MultiLang")).await
    });
}

#[bench]
fn bench_surrealdb_create_project(b: &mut Bencher) {
    // Similar benchmark for SurrealDB
}
```

**Expected Results:**
- Read performance: Similar (< 50ms)
- Write performance: Similar (< 100ms)
- Graph queries: SurrealDB wins (< 50ms vs 200ms+ for SQLite)

**Deliverable:**
- [ ] Performance benchmarks created
- [ ] Benchmark comparison documented
- [ ] Performance targets met
- [ ] Optimization recommendations made

---

## Phase 7: Documentation & Deployment (Week 5)

### Step 7.1: User Documentation (4 hours)

**Create:**

1. **Configuration Guide**
   - How to choose SQLite vs SurrealDB
   - Configuration options explained
   - Environment variables
   - File locations

2. **Migration Guide**
   - Step-by-step migration procedure
   - Backup/restore procedures
   - Rollback procedures
   - Troubleshooting

3. **Performance Tuning**
   - Connection pool sizing
   - Index optimization
   - Query optimization
   - Capacity planning

**Deliverable:**
- [ ] Configuration guide complete
- [ ] Migration guide complete
- [ ] Troubleshooting guide complete
- [ ] Performance tuning guide complete

### Step 7.2: Developer Documentation (3 hours)

**Create:**

1. **Architecture Guide**
   - System design overview
   - Database trait interface
   - Implementation patterns

2. **API Reference**
   - Database trait methods
   - Configuration options
   - Error types

3. **Extension Guide**
   - How to add PostgreSQL support
   - How to add MongoDB support

**Deliverable:**
- [ ] Architecture guide complete
- [ ] API reference complete
- [ ] Extension guide complete
- [ ] Rustdoc updated

### Step 7.3: Deployment Documentation (2 hours)

**Create:**

1. **Docker Setup**
   - docker-compose.yml for SurrealDB
   - Container configuration
   - Networking

2. **Production Deployment**
   - Database initialization
   - Backup strategy
   - Monitoring
   - Disaster recovery

3. **Kubernetes (optional)**
   - Helm charts
   - StatefulSet configuration
   - Service definition

**Deliverable:**
- [ ] Docker setup documented
- [ ] Production deployment guide complete
- [ ] Monitoring setup documented
- [ ] Disaster recovery procedures documented

### Step 7.4: Release Artifacts (3 hours)

**Create:**

1. **CHANGELOG.md entry**
   ```
   ## [0.2.0] - 2025-Q2

   ### Added
   - SurrealDB backend support
   - Database migration tool
   - Configuration-based backend selection
   - Real-time dashboard updates
   - Built-in audit trails

   ### Changed
   - Persistence layer refactored to support multiple backends

   ### Deprecated
   - None

   ### Removed
   - None

   ### Fixed
   - None

   ### Security
   - Configuration validation
   - Credential management
   ```

2. **Migration Guide** (final version)

3. **Compatibility Matrix**
   ```
   | Feature | SQLite | SurrealDB |
   |---------|--------|-----------|
   | Create Project | ✅ | ✅ |
   | Real-time Updates | ❌ | ✅ |
   | Graph Queries | ❌ | ✅ |
   | Versioning | ❌ | ✅ |
   ```

4. **Release Notes**

**Deliverable:**
- [ ] CHANGELOG updated
- [ ] Migration guide finalized
- [ ] Compatibility matrix documented
- [ ] Release notes prepared

---

## Success Criteria

### Code Quality
- [ ] All tests passing (340+ tests)
- [ ] Code coverage > 80%
- [ ] No clippy warnings
- [ ] Properly formatted
- [ ] Full rustdoc documentation

### Functionality
- [ ] SQLite backend working
- [ ] SurrealDB backend working
- [ ] Configuration-based selection
- [ ] Migration tool working
- [ ] Rollback capability

### Performance
- [ ] Read operations < 50ms
- [ ] Write operations < 100ms
- [ ] Graph queries < 50ms (SurrealDB)
- [ ] No memory leaks
- [ ] Connection pooling working

### Documentation
- [ ] User guides complete
- [ ] Developer guides complete
- [ ] API documentation complete
- [ ] Deployment guides complete

---

## Timeline & Resource Allocation

```
Week 1:   Phase 1 (Design)        1 dev   (40 hours)
Week 1-2: Phase 2 (SQLite)         1 dev   (48 hours)
Week 2-3: Phase 3 (SurrealDB)      1 dev   (56 hours)
Week 3:   Phase 4 (Migration)      1 dev   (40 hours)
Week 3-4: Phase 5 (Integration)    1 dev   (40 hours)
Week 4:   Phase 6 (Testing)        1 dev + 0.5 QA (48 hours)
Week 5:   Phase 7 (Docs)           1 dev + 0.5 DevOps (40 hours)

Total:    40-50 hours (240-300 billable hours)
Team:     1 senior dev (full-time) + support
Timeline: 5-6 weeks
```

---

## Getting Started (First Steps)

1. **Read:** All documentation in project root
   - README_SURREALDB.md
   - SURREALDB_QUICK_REFERENCE.md
   - SURREALDB_MIGRATION.md

2. **Create branch:** `feature/surrealdb-support`

3. **Start Phase 1:**
   - [ ] Analyze current persistence.rs
   - [ ] Design Database trait
   - [ ] Plan configuration system
   - [ ] Plan testing strategy

4. **Review:** Have team review design before implementing

5. **Proceed:** Phase by phase, with regular demonstrations

---

## Risks & Mitigation

| Risk | Mitigation |
|------|-----------|
| Breaking changes | Trait pattern prevents this |
| Data loss during migration | Mandatory backup before migration |
| Performance regression | Benchmarking + comparison |
| SurrealDB learning curve | Comprehensive examples provided |
| Operational complexity | Keep SQLite as simple default |

---

**Ready to begin implementation!** 🚀