Rustelo/docs/database_migration_guide.md

Database Migration Guide

Overview

Rustelo has been upgraded to use a database-agnostic architecture that supports both PostgreSQL and SQLite. This guide will help you migrate existing code to use the new database abstraction layer.

What Changed

Before (Old Architecture)

// Direct PostgreSQL dependency
use sqlx::PgPool;

// Repository tied to PostgreSQL
pub struct AuthRepository {
    pool: PgPool,
}

impl AuthRepository {
    pub fn new(pool: PgPool) -> Self {
        Self { pool }
    }
    
    pub async fn find_user(&self, id: Uuid) -> Result<Option<User>> {
        sqlx::query_as!(User, "SELECT * FROM users WHERE id = $1", id)
            .fetch_optional(&self.pool)
            .await
    }
}

After (New Architecture)

// Database-agnostic abstractions
use crate::database::{DatabaseConnection, DatabaseType};

// Repository works with any database
pub struct AuthRepository {
    database: DatabaseConnection,
}

impl AuthRepository {
    pub fn new(database: DatabaseConnection) -> Self {
        Self { database }
    }
    
    pub async fn find_user(&self, id: Uuid) -> Result<Option<User>> {
        match self.database.database_type() {
            DatabaseType::PostgreSQL => self.find_user_postgres(id).await,
            DatabaseType::SQLite => self.find_user_sqlite(id).await,
        }
    }
}

Migration Steps

1. Update Repository Constructors

Before:

let auth_repository = Arc::new(AuthRepository::new(pool.clone()));

After:

// For the new database-agnostic repositories
let auth_repository = Arc::new(database::auth::AuthRepository::new(database.clone()));

// Or create from pool
let auth_repository = Arc::new(database::auth::AuthRepository::from_pool(&database_pool));

2. Replace Direct SQL Queries

Before:

pub async fn create_user(&self, user: &User) -> Result<()> {
    sqlx::query!(
        "INSERT INTO users (id, email, username) VALUES ($1, $2, $3)",
        user.id,
        user.email,
        user.username
    )
    .execute(&self.pool)
    .await?;
    Ok(())
}

After:

pub async fn create_user(&self, user: &User) -> Result<()> {
    match self.database.database_type() {
        DatabaseType::PostgreSQL => self.create_user_postgres(user).await,
        DatabaseType::SQLite => self.create_user_sqlite(user).await,
    }
}

async fn create_user_postgres(&self, user: &User) -> Result<()> {
    self.database
        .execute(
            "INSERT INTO users (id, email, username) VALUES ($1, $2, $3)",
            &[
                user.id.into(),
                user.email.clone().into(),
                user.username.clone().into(),
            ],
        )
        .await?;
    Ok(())
}

async fn create_user_sqlite(&self, user: &User) -> Result<()> {
    self.database
        .execute(
            "INSERT INTO users (id, email, username) VALUES (?, ?, ?)",
            &[
                user.id.to_string().into(),
                user.email.clone().into(),
                user.username.clone().into(),
            ],
        )
        .await?;
    Ok(())
}

3. Update Row Processing

Before:

let row = sqlx::query_as::<_, UserRow>("SELECT * FROM users WHERE id = $1")
    .bind(id)
    .fetch_optional(&self.pool)
    .await?;

After:

let row = self.database
    .fetch_optional(
        "SELECT * FROM users WHERE id = $1", // PostgreSQL
        &[id.into()],
    )
    .await?;

if let Some(row) = row {
    let user = User {
        id: row.get_uuid("id")?,
        email: row.get_string("email")?,
        username: row.get_string("username")?,
        // ... other fields
    };
}

4. Database Initialization

Before:

let pool = PgPool::connect(&database_url).await?;
let auth_repository = AuthRepository::new(pool);

After:

let database_config = DatabaseConfig {
    url: database_url.to_string(),
    max_connections: 10,
    min_connections: 1,
    connect_timeout: Duration::from_secs(30),
    idle_timeout: Duration::from_secs(600),
    max_lifetime: Duration::from_secs(3600),
};

let database_pool = DatabasePool::new(&database_config).await?;
let database = Database::new(database_pool.clone());
let auth_repository = database::auth::AuthRepository::new(database.create_connection());

Database-Specific Considerations

PostgreSQL vs SQLite Differences

Feature	PostgreSQL	SQLite
Parameter Binding	$1, $2, $3	?, ?, ?
UUID Storage	Native UUID type	TEXT (string)
Timestamps	TIMESTAMP WITH TIME ZONE	TEXT (ISO 8601)
JSON	JSONB	TEXT (JSON string)
Arrays	Native arrays	TEXT (JSON array)
Boolean	BOOLEAN	INTEGER (0/1)

Handling Data Type Differences

// UUID handling
match self.database.database_type() {
    DatabaseType::PostgreSQL => {
        // PostgreSQL stores UUIDs natively
        params.push(user_id.into());
    }
    DatabaseType::SQLite => {
        // SQLite stores UUIDs as strings
        params.push(user_id.to_string().into());
    }
}

// Timestamp handling
match self.database.database_type() {
    DatabaseType::PostgreSQL => {
        // PostgreSQL stores timestamps natively
        params.push(created_at.into());
    }
    DatabaseType::SQLite => {
        // SQLite stores timestamps as ISO strings
        params.push(created_at.to_rfc3339().into());
    }
}

Migration Checklist

✅ Code Migration

• Replace PgPool with DatabaseConnection
• Update repository constructors
• Split database operations into database-specific methods
• Handle parameter binding differences ($1 vs ?)
• Handle data type differences (UUID, timestamps, etc.)
• Update imports to use new database modules

✅ Database Migration Files

• Create separate migration files for PostgreSQL (*_postgres.sql)
• Create separate migration files for SQLite (*_sqlite.sql)
• Update migration runner configuration
• Test migrations on both database types

✅ Configuration

• Update environment variables
• Test with different database URLs
• Update deployment scripts
• Update documentation

✅ Testing

• Update unit tests to use new abstractions
• Add integration tests for both database types
• Test migration path from old to new architecture
• Performance testing on both databases

Common Migration Patterns

Pattern 1: Simple Repository Migration

// Old
pub struct MyRepository {
    pool: PgPool,
}

// New
pub struct MyRepository {
    database: DatabaseConnection,
}

impl MyRepository {
    // Add convenience constructor
    pub fn from_pool(pool: &DatabasePool) -> Self {
        let connection = DatabaseConnection::from_pool(pool);
        Self::new(connection)
    }
}

Pattern 2: Complex Query Migration

// Old
pub async fn complex_query(&self) -> Result<Vec<Item>> {
    sqlx::query_as!(
        Item,
        r#"
        SELECT i.*, u.username 
        FROM items i 
        JOIN users u ON i.user_id = u.id 
        WHERE i.created_at > $1
        ORDER BY i.created_at DESC
        "#,
        since
    )
    .fetch_all(&self.pool)
    .await
}

// New
pub async fn complex_query(&self) -> Result<Vec<Item>> {
    match self.database.database_type() {
        DatabaseType::PostgreSQL => self.complex_query_postgres().await,
        DatabaseType::SQLite => self.complex_query_sqlite().await,
    }
}

async fn complex_query_postgres(&self) -> Result<Vec<Item>> {
    let rows = self.database
        .fetch_all(
            r#"
            SELECT i.id, i.name, i.user_id, i.created_at, u.username 
            FROM items i 
            JOIN users u ON i.user_id = u.id 
            WHERE i.created_at > $1
            ORDER BY i.created_at DESC
            "#,
            &[since.into()],
        )
        .await?;

    // Convert rows to Items
    let mut items = Vec::new();
    for row in rows {
        items.push(Item {
            id: row.get_uuid("id")?,
            name: row.get_string("name")?,
            user_id: row.get_uuid("user_id")?,
            username: row.get_string("username")?,
            created_at: row.get_datetime("created_at")?,
        });
    }
    Ok(items)
}

Troubleshooting

Common Issues and Solutions

1. UUID Conversion Errors

Problem: SQLite doesn't support UUID natively Solution: Convert UUIDs to strings for SQLite

match self.database.database_type() {
    DatabaseType::PostgreSQL => params.push(uuid.into()),
    DatabaseType::SQLite => params.push(uuid.to_string().into()),
}

2. Parameter Binding Mismatch

Problem: Using PostgreSQL syntax ($1) with SQLite Solution: Use database-specific query strings

let query = match self.database.database_type() {
    DatabaseType::PostgreSQL => "SELECT * FROM users WHERE id = $1",
    DatabaseType::SQLite => "SELECT * FROM users WHERE id = ?",
};

3. Migration File Not Found

Problem: Missing database-specific migration files Solution: Ensure both *_postgres.sql and *_sqlite.sql files exist

4. Type Conversion Errors

Problem: Data type mismatches between databases Solution: Use the DatabaseRow trait methods consistently

// Always use the trait methods
let user_id = row.get_uuid("user_id")?;  // Handles conversion automatically
let created_at = row.get_datetime("created_at")?;  // Handles conversion automatically

Best Practices

1. Database-Agnostic Design

• Write repositories that work with both databases
• Use the database abstraction layer consistently
• Avoid database-specific features when possible

2. Migration Strategy

• Migrate one module at a time
• Test thoroughly with both database types
• Keep the old code until migration is complete

3. Performance Considerations

• PostgreSQL: Better for complex queries and large datasets
• SQLite: Better for simple queries and small datasets
• Use appropriate database features for your use case

4. Testing Strategy

• Test all code paths with both databases
• Use SQLite for fast unit tests
• Use PostgreSQL for integration tests

Need Help?

If you encounter issues during migration:

1. Check the example implementations for patterns
2. Review the test files for working examples
3. Consult the API documentation
4. Look at the migration files for database schema examples

Example: Complete Repository Migration

Here's a complete example of migrating a repository:

Before (Old Implementation)

use sqlx::PgPool;
use uuid::Uuid;

pub struct PostRepository {
    pool: PgPool,
}

impl PostRepository {
    pub fn new(pool: PgPool) -> Self {
        Self { pool }
    }

    pub async fn create_post(&self, post: &Post) -> Result<()> {
        sqlx::query!(
            "INSERT INTO posts (id, title, content, author_id) VALUES ($1, $2, $3, $4)",
            post.id,
            post.title,
            post.content,
            post.author_id
        )
        .execute(&self.pool)
        .await?;
        Ok(())
    }

    pub async fn find_post(&self, id: Uuid) -> Result<Option<Post>> {
        let row = sqlx::query_as!(
            Post,
            "SELECT id, title, content, author_id, created_at FROM posts WHERE id = $1",
            id
        )
        .fetch_optional(&self.pool)
        .await?;
        Ok(row)
    }
}

After (New Implementation)

use crate::database::{DatabaseConnection, DatabaseType};
use uuid::Uuid;

pub struct PostRepository {
    database: DatabaseConnection,
}

impl PostRepository {
    pub fn new(database: DatabaseConnection) -> Self {
        Self { database }
    }

    pub fn from_pool(pool: &crate::database::DatabasePool) -> Self {
        let connection = DatabaseConnection::from_pool(pool);
        Self::new(connection)
    }

    pub async fn create_post(&self, post: &Post) -> Result<()> {
        match self.database.database_type() {
            DatabaseType::PostgreSQL => self.create_post_postgres(post).await,
            DatabaseType::SQLite => self.create_post_sqlite(post).await,
        }
    }

    async fn create_post_postgres(&self, post: &Post) -> Result<()> {
        self.database
            .execute(
                "INSERT INTO posts (id, title, content, author_id) VALUES ($1, $2, $3, $4)",
                &[
                    post.id.into(),
                    post.title.clone().into(),
                    post.content.clone().into(),
                    post.author_id.into(),
                ],
            )
            .await?;
        Ok(())
    }

    async fn create_post_sqlite(&self, post: &Post) -> Result<()> {
        self.database
            .execute(
                "INSERT INTO posts (id, title, content, author_id) VALUES (?, ?, ?, ?)",
                &[
                    post.id.to_string().into(),
                    post.title.clone().into(),
                    post.content.clone().into(),
                    post.author_id.to_string().into(),
                ],
            )
            .await?;
        Ok(())
    }

    pub async fn find_post(&self, id: Uuid) -> Result<Option<Post>> {
        match self.database.database_type() {
            DatabaseType::PostgreSQL => self.find_post_postgres(id).await,
            DatabaseType::SQLite => self.find_post_sqlite(id).await,
        }
    }

    async fn find_post_postgres(&self, id: Uuid) -> Result<Option<Post>> {
        let row = self
            .database
            .fetch_optional(
                "SELECT id, title, content, author_id, created_at FROM posts WHERE id = $1",
                &[id.into()],
            )
            .await?;

        if let Some(row) = row {
            Ok(Some(Post {
                id: row.get_uuid("id")?,
                title: row.get_string("title")?,
                content: row.get_string("content")?,
                author_id: row.get_uuid("author_id")?,
                created_at: row.get_datetime("created_at")?,
            }))
        } else {
            Ok(None)
        }
    }

    async fn find_post_sqlite(&self, id: Uuid) -> Result<Option<Post>> {
        let row = self
            .database
            .fetch_optional(
                "SELECT id, title, content, author_id, created_at FROM posts WHERE id = ?",
                &[id.to_string().into()],
            )
            .await?;

        if let Some(row) = row {
            Ok(Some(Post {
                id: row.get_uuid("id")?,
                title: row.get_string("title")?,
                content: row.get_string("content")?,
                author_id: row.get_uuid("author_id")?,
                created_at: row.get_datetime("created_at")?,
            }))
        } else {
            Ok(None)
        }
    }
}

This migration guide provides a comprehensive overview of how to transition from the old PostgreSQL-only architecture to the new database-agnostic system.