Rustelo/docs/database_migration_guide.md

# Database Migration Guide

## Overview

Rustelo has been upgraded to use a **database-agnostic architecture** that supports both PostgreSQL and SQLite. This guide will help you migrate existing code to use the new database abstraction layer.

## What Changed

### Before (Old Architecture)
```rust
// Direct PostgreSQL dependency
use sqlx::PgPool;

// Repository tied to PostgreSQL
pub struct AuthRepository {
    pool: PgPool,
}

impl AuthRepository {
    pub fn new(pool: PgPool) -> Self {
        Self { pool }
    }
    
    pub async fn find_user(&self, id: Uuid) -> Result<Option<User>> {
        sqlx::query_as!(User, "SELECT * FROM users WHERE id = $1", id)
            .fetch_optional(&self.pool)
            .await
    }
}
```

### After (New Architecture)
```rust
// Database-agnostic abstractions
use crate::database::{DatabaseConnection, DatabaseType};

// Repository works with any database
pub struct AuthRepository {
    database: DatabaseConnection,
}

impl AuthRepository {
    pub fn new(database: DatabaseConnection) -> Self {
        Self { database }
    }
    
    pub async fn find_user(&self, id: Uuid) -> Result<Option<User>> {
        match self.database.database_type() {
            DatabaseType::PostgreSQL => self.find_user_postgres(id).await,
            DatabaseType::SQLite => self.find_user_sqlite(id).await,
        }
    }
}
```

## Migration Steps

### 1. Update Repository Constructors

**Before:**
```rust
let auth_repository = Arc::new(AuthRepository::new(pool.clone()));
```

**After:**
```rust
// For the new database-agnostic repositories
let auth_repository = Arc::new(database::auth::AuthRepository::new(database.clone()));

// Or create from pool
let auth_repository = Arc::new(database::auth::AuthRepository::from_pool(&database_pool));
```

### 2. Replace Direct SQL Queries

**Before:**
```rust
pub async fn create_user(&self, user: &User) -> Result<()> {
    sqlx::query!(
        "INSERT INTO users (id, email, username) VALUES ($1, $2, $3)",
        user.id,
        user.email,
        user.username
    )
    .execute(&self.pool)
    .await?;
    Ok(())
}
```

**After:**
```rust
pub async fn create_user(&self, user: &User) -> Result<()> {
    match self.database.database_type() {
        DatabaseType::PostgreSQL => self.create_user_postgres(user).await,
        DatabaseType::SQLite => self.create_user_sqlite(user).await,
    }
}

async fn create_user_postgres(&self, user: &User) -> Result<()> {
    self.database
        .execute(
            "INSERT INTO users (id, email, username) VALUES ($1, $2, $3)",
            &[
                user.id.into(),
                user.email.clone().into(),
                user.username.clone().into(),
            ],
        )
        .await?;
    Ok(())
}

async fn create_user_sqlite(&self, user: &User) -> Result<()> {
    self.database
        .execute(
            "INSERT INTO users (id, email, username) VALUES (?, ?, ?)",
            &[
                user.id.to_string().into(),
                user.email.clone().into(),
                user.username.clone().into(),
            ],
        )
        .await?;
    Ok(())
}
```

### 3. Update Row Processing

**Before:**
```rust
let row = sqlx::query_as::<_, UserRow>("SELECT * FROM users WHERE id = $1")
    .bind(id)
    .fetch_optional(&self.pool)
    .await?;
```

**After:**
```rust
let row = self.database
    .fetch_optional(
        "SELECT * FROM users WHERE id = $1", // PostgreSQL
        &[id.into()],
    )
    .await?;

if let Some(row) = row {
    let user = User {
        id: row.get_uuid("id")?,
        email: row.get_string("email")?,
        username: row.get_string("username")?,
        // ... other fields
    };
}
```

### 4. Database Initialization

**Before:**
```rust
let pool = PgPool::connect(&database_url).await?;
let auth_repository = AuthRepository::new(pool);
```

**After:**
```rust
let database_config = DatabaseConfig {
    url: database_url.to_string(),
    max_connections: 10,
    min_connections: 1,
    connect_timeout: Duration::from_secs(30),
    idle_timeout: Duration::from_secs(600),
    max_lifetime: Duration::from_secs(3600),
};

let database_pool = DatabasePool::new(&database_config).await?;
let database = Database::new(database_pool.clone());
let auth_repository = database::auth::AuthRepository::new(database.create_connection());
```

## Database-Specific Considerations

### PostgreSQL vs SQLite Differences

| Feature | PostgreSQL | SQLite |
|---------|------------|--------|
| **Parameter Binding** | `$1, $2, $3` | `?, ?, ?` |
| **UUID Storage** | Native `UUID` type | `TEXT` (string) |
| **Timestamps** | `TIMESTAMP WITH TIME ZONE` | `TEXT` (ISO 8601) |
| **JSON** | `JSONB` | `TEXT` (JSON string) |
| **Arrays** | Native arrays | `TEXT` (JSON array) |
| **Boolean** | `BOOLEAN` | `INTEGER` (0/1) |

### Handling Data Type Differences

```rust
// UUID handling
match self.database.database_type() {
    DatabaseType::PostgreSQL => {
        // PostgreSQL stores UUIDs natively
        params.push(user_id.into());
    }
    DatabaseType::SQLite => {
        // SQLite stores UUIDs as strings
        params.push(user_id.to_string().into());
    }
}

// Timestamp handling
match self.database.database_type() {
    DatabaseType::PostgreSQL => {
        // PostgreSQL stores timestamps natively
        params.push(created_at.into());
    }
    DatabaseType::SQLite => {
        // SQLite stores timestamps as ISO strings
        params.push(created_at.to_rfc3339().into());
    }
}
```

## Migration Checklist

### ✅ Code Migration

- [ ] Replace `PgPool` with `DatabaseConnection`
- [ ] Update repository constructors
- [ ] Split database operations into database-specific methods
- [ ] Handle parameter binding differences (`$1` vs `?`)
- [ ] Handle data type differences (UUID, timestamps, etc.)
- [ ] Update imports to use new database modules

### ✅ Database Migration Files

- [ ] Create separate migration files for PostgreSQL (`*_postgres.sql`)
- [ ] Create separate migration files for SQLite (`*_sqlite.sql`)
- [ ] Update migration runner configuration
- [ ] Test migrations on both database types

### ✅ Configuration

- [ ] Update environment variables
- [ ] Test with different database URLs
- [ ] Update deployment scripts
- [ ] Update documentation

### ✅ Testing

- [ ] Update unit tests to use new abstractions
- [ ] Add integration tests for both database types
- [ ] Test migration path from old to new architecture
- [ ] Performance testing on both databases

## Common Migration Patterns

### Pattern 1: Simple Repository Migration

```rust
// Old
pub struct MyRepository {
    pool: PgPool,
}

// New
pub struct MyRepository {
    database: DatabaseConnection,
}

impl MyRepository {
    // Add convenience constructor
    pub fn from_pool(pool: &DatabasePool) -> Self {
        let connection = DatabaseConnection::from_pool(pool);
        Self::new(connection)
    }
}
```

### Pattern 2: Complex Query Migration

```rust
// Old
pub async fn complex_query(&self) -> Result<Vec<Item>> {
    sqlx::query_as!(
        Item,
        r#"
        SELECT i.*, u.username 
        FROM items i 
        JOIN users u ON i.user_id = u.id 
        WHERE i.created_at > $1
        ORDER BY i.created_at DESC
        "#,
        since
    )
    .fetch_all(&self.pool)
    .await
}

// New
pub async fn complex_query(&self) -> Result<Vec<Item>> {
    match self.database.database_type() {
        DatabaseType::PostgreSQL => self.complex_query_postgres().await,
        DatabaseType::SQLite => self.complex_query_sqlite().await,
    }
}

async fn complex_query_postgres(&self) -> Result<Vec<Item>> {
    let rows = self.database
        .fetch_all(
            r#"
            SELECT i.id, i.name, i.user_id, i.created_at, u.username 
            FROM items i 
            JOIN users u ON i.user_id = u.id 
            WHERE i.created_at > $1
            ORDER BY i.created_at DESC
            "#,
            &[since.into()],
        )
        .await?;

    // Convert rows to Items
    let mut items = Vec::new();
    for row in rows {
        items.push(Item {
            id: row.get_uuid("id")?,
            name: row.get_string("name")?,
            user_id: row.get_uuid("user_id")?,
            username: row.get_string("username")?,
            created_at: row.get_datetime("created_at")?,
        });
    }
    Ok(items)
}
```

## Troubleshooting

### Common Issues and Solutions

#### 1. UUID Conversion Errors
**Problem:** SQLite doesn't support UUID natively
**Solution:** Convert UUIDs to strings for SQLite

```rust
match self.database.database_type() {
    DatabaseType::PostgreSQL => params.push(uuid.into()),
    DatabaseType::SQLite => params.push(uuid.to_string().into()),
}
```

#### 2. Parameter Binding Mismatch
**Problem:** Using PostgreSQL syntax (`$1`) with SQLite
**Solution:** Use database-specific query strings

```rust
let query = match self.database.database_type() {
    DatabaseType::PostgreSQL => "SELECT * FROM users WHERE id = $1",
    DatabaseType::SQLite => "SELECT * FROM users WHERE id = ?",
};
```

#### 3. Migration File Not Found
**Problem:** Missing database-specific migration files
**Solution:** Ensure both `*_postgres.sql` and `*_sqlite.sql` files exist

#### 4. Type Conversion Errors
**Problem:** Data type mismatches between databases
**Solution:** Use the `DatabaseRow` trait methods consistently

```rust
// Always use the trait methods
let user_id = row.get_uuid("user_id")?;  // Handles conversion automatically
let created_at = row.get_datetime("created_at")?;  // Handles conversion automatically
```

## Best Practices

### 1. Database-Agnostic Design
- Write repositories that work with both databases
- Use the database abstraction layer consistently
- Avoid database-specific features when possible

### 2. Migration Strategy
- Migrate one module at a time
- Test thoroughly with both database types
- Keep the old code until migration is complete

### 3. Performance Considerations
- PostgreSQL: Better for complex queries and large datasets
- SQLite: Better for simple queries and small datasets
- Use appropriate database features for your use case

### 4. Testing Strategy
- Test all code paths with both databases
- Use SQLite for fast unit tests
- Use PostgreSQL for integration tests

## Need Help?

If you encounter issues during migration:

1. Check the [example implementations](../server/src/database/) for patterns
2. Review the [test files](../server/tests/) for working examples
3. Consult the [API documentation](../server/src/database/mod.rs)
4. Look at the [migration files](../migrations/) for database schema examples

## Example: Complete Repository Migration

Here's a complete example of migrating a repository:

### Before (Old Implementation)
```rust
use sqlx::PgPool;
use uuid::Uuid;

pub struct PostRepository {
    pool: PgPool,
}

impl PostRepository {
    pub fn new(pool: PgPool) -> Self {
        Self { pool }
    }

    pub async fn create_post(&self, post: &Post) -> Result<()> {
        sqlx::query!(
            "INSERT INTO posts (id, title, content, author_id) VALUES ($1, $2, $3, $4)",
            post.id,
            post.title,
            post.content,
            post.author_id
        )
        .execute(&self.pool)
        .await?;
        Ok(())
    }

    pub async fn find_post(&self, id: Uuid) -> Result<Option<Post>> {
        let row = sqlx::query_as!(
            Post,
            "SELECT id, title, content, author_id, created_at FROM posts WHERE id = $1",
            id
        )
        .fetch_optional(&self.pool)
        .await?;
        Ok(row)
    }
}
```

### After (New Implementation)
```rust
use crate::database::{DatabaseConnection, DatabaseType};
use uuid::Uuid;

pub struct PostRepository {
    database: DatabaseConnection,
}

impl PostRepository {
    pub fn new(database: DatabaseConnection) -> Self {
        Self { database }
    }

    pub fn from_pool(pool: &crate::database::DatabasePool) -> Self {
        let connection = DatabaseConnection::from_pool(pool);
        Self::new(connection)
    }

    pub async fn create_post(&self, post: &Post) -> Result<()> {
        match self.database.database_type() {
            DatabaseType::PostgreSQL => self.create_post_postgres(post).await,
            DatabaseType::SQLite => self.create_post_sqlite(post).await,
        }
    }

    async fn create_post_postgres(&self, post: &Post) -> Result<()> {
        self.database
            .execute(
                "INSERT INTO posts (id, title, content, author_id) VALUES ($1, $2, $3, $4)",
                &[
                    post.id.into(),
                    post.title.clone().into(),
                    post.content.clone().into(),
                    post.author_id.into(),
                ],
            )
            .await?;
        Ok(())
    }

    async fn create_post_sqlite(&self, post: &Post) -> Result<()> {
        self.database
            .execute(
                "INSERT INTO posts (id, title, content, author_id) VALUES (?, ?, ?, ?)",
                &[
                    post.id.to_string().into(),
                    post.title.clone().into(),
                    post.content.clone().into(),
                    post.author_id.to_string().into(),
                ],
            )
            .await?;
        Ok(())
    }

    pub async fn find_post(&self, id: Uuid) -> Result<Option<Post>> {
        match self.database.database_type() {
            DatabaseType::PostgreSQL => self.find_post_postgres(id).await,
            DatabaseType::SQLite => self.find_post_sqlite(id).await,
        }
    }

    async fn find_post_postgres(&self, id: Uuid) -> Result<Option<Post>> {
        let row = self
            .database
            .fetch_optional(
                "SELECT id, title, content, author_id, created_at FROM posts WHERE id = $1",
                &[id.into()],
            )
            .await?;

        if let Some(row) = row {
            Ok(Some(Post {
                id: row.get_uuid("id")?,
                title: row.get_string("title")?,
                content: row.get_string("content")?,
                author_id: row.get_uuid("author_id")?,
                created_at: row.get_datetime("created_at")?,
            }))
        } else {
            Ok(None)
        }
    }

    async fn find_post_sqlite(&self, id: Uuid) -> Result<Option<Post>> {
        let row = self
            .database
            .fetch_optional(
                "SELECT id, title, content, author_id, created_at FROM posts WHERE id = ?",
                &[id.to_string().into()],
            )
            .await?;

        if let Some(row) = row {
            Ok(Some(Post {
                id: row.get_uuid("id")?,
                title: row.get_string("title")?,
                content: row.get_string("content")?,
                author_id: row.get_uuid("author_id")?,
                created_at: row.get_datetime("created_at")?,
            }))
        } else {
            Ok(None)
        }
    }
}
```

This migration guide provides a comprehensive overview of how to transition from the old PostgreSQL-only architecture to the new database-agnostic system.