# AI Backend Integration Guide ## Problem: Why AI Backend Can't Be Just a Library The AI backend (RAG, Knowledge Graph, embeddings) must be **integrated into actual backends** (CLI, TUI, Web) to be useful. Here's how: ## Architecture: AI Backend Integration Layers ```text ┌─────────────────────────────────────────────────────────┐ │ USER INTERFACE │ │ (CLI / TUI / Web - Interactive Forms) │ └──────────────────────┬──────────────────────────────────┘ │ ┌──────────────────────▼──────────────────────────────────┐ │ AI SERVICE LAYER │ │ • RAG search (semantic + keyword) │ │ • Knowledge graph queries │ │ • Batch operations (efficient) │ │ • Persistence (save/load state) │ └──────────────────────┬──────────────────────────────────┘ │ ┌──────────────────────▼──────────────────────────────────┐ │ KNOWLEDGE BASE / STORAGE │ │ • Vector embeddings (HNSW index) │ │ • Full-text index │ │ • Entity relationships (KG) │ │ • Persistent state │ └─────────────────────────────────────────────────────────┘ ```text ## Integration Patterns ### 1. CLI Backend Integration ```rust // Example: Search-enhanced CLI form use typedialog_core::ai::rag::{RagSystem, RagConfig}; pub struct SearchableForm { rag: RagSystem, form: FormDefinition, } impl SearchableForm { pub fn new(form_toml: &str) -> Result { let rag = RagSystem::new(RagConfig::default())?; let form = form_parser::parse_toml(form_toml)?; Ok(SearchableForm { rag, form }) } pub fn execute_with_search(&mut self) -> Result { // Show form let results = execute_form(&self.form)?; // Optionally: Search documents based on form data if let Some(search_query) = results.get("_search") { let docs = self.rag.retrieve(search_query)?; println!("Related documents found: {}", docs.len()); for doc in docs { println!(" - {}: {}", doc.doc_id, doc.content); } } Ok(results) } } ```text **Use Cases:** - Interactive CLI tools with search suggestions - Configuration assistants with knowledge base - Automated documentation lookup during form fill ### 2. TUI Backend Integration ```rust // Example: Split-pane TUI with RAG search use ratatui::prelude::*; pub struct RagTuiPanel { rag: RagSystem, search_results: Vec, selected_result: usize, } impl RagTuiPanel { pub fn draw(&self, f: &mut Frame, area: Rect) { // Left pane: Search input + results // Right pane: Selected document preview // Display semantic score indicators } pub fn on_query(&mut self, query: &str) -> Result<()> { self.search_results = self.rag.retrieve(query)?; self.selected_result = 0; Ok(()) } } impl Widget for RagTuiPanel { fn render(self, area: Rect, buf: &mut Buffer) { // Render search UI with batched results } } ```text **Use Cases:** - Dashboard with AI-powered search - Document browser with semantic search - Knowledge base explorer - Configuration helper with real-time suggestions ### 3. Web Backend Integration ```rust // Example: Web form with RAG API endpoint use axum::{routing::post, Json, Router}; pub struct RagWebService { rag: RagSystem, } impl RagWebService { pub fn router(mut rag: RagSystem) -> Router { Router::new() .route("/search", post(Self::search)) .route("/documents", post(Self::add_documents)) .route("/documents/:id", axum::routing::delete(Self::remove_document)) .with_state(rag) } async fn search( Json(query): Json, ) -> Json> { let results = self.rag.retrieve(&query.text)?; Json(results) } async fn add_documents( Json(docs): Json>, ) -> Json { self.rag.add_documents_batch(docs)?; Json(ApiResponse { success: true }) } } ```text **Use Cases:** - Web SaaS with knowledge base search - Document management system with semantic search - FAQ finder with intelligent matching - Support chatbot with context retrieval ## Service Models ### Model 1: Embedded Service (Simple) ```text Frontend Form → RAG Service (in-memory) → Results ```text - **Pros:** Fast, no network latency, easy to prototype - **Cons:** Can't scale, state lost on restart - **Use when:** Development, small documents, single user - **Impl:** Load documents on startup, search in-memory ### Model 2: Microservice (Production) ```text Frontend Form → HTTP REST API → RAG Service → Vector DB / File Store ```text - **Pros:** Scalable, persistent, can handle large datasets - **Cons:** Network latency, operational complexity - **Use when:** Multiple users, large knowledge base, production SaaS - **Impl:** Axum server with RagSystem, persist to disk/database ### Model 3: Hybrid (Best of Both) ```text Frontend Form → Local Cache (RAG) → Sync ↔ Remote Service ```text - **Pros:** Fast local access, keeps in sync, fallback support - **Cons:** Complexity, consistency challenges - **Use when:** High availability needed, offline support required - **Impl:** Embedded RAG with periodic sync to central service ## Implementation Roadmap ### Phase 1: Embedded Integration ✅ (Current) ```rust // In CLI/TUI/Web backends let mut rag = RagSystem::new(config)?; rag.add_documents_batch(docs)?; let results = rag.retrieve(query)?; ```text **Status:** Ready - batch operations optimized, persistence working ### Phase 2: Microservice Wrapper (Next) ```rust // Create: typedialog-ai-service (separate binary) // Features: // - REST API with Axum // - Persistent storage (SQLite / file-based) // - Health checks, metrics // - Document management API ```text **Scope:** - New binary crate: `crates/typedialog-ai-service/` - Routes: `/search`, `/documents`, `/status` - Persistence layer - Docker support ### Phase 3: Backend Integration **CLI Integration:** - Add `--rag` flag to forms - Search suggestions during field fill - Context-aware help text **TUI Integration:** - Right pane: RAG search results - Keyboard shortcuts for search - Result scoring visualization **Web Integration:** - AJAX search endpoint - Autocomplete with semantic results - Results sidebar in form UI ## Example: End-to-End Integration ```rust // 1. Initialize RAG during app startup let mut rag = RagSystem::new(RagConfig { semantic_weight: 0.6, keyword_weight: 0.4, max_results: 5, min_score: 0.3, })?; // 2. Load knowledge base (batch operation - efficient) let knowledge_base = load_documents_from_file("kb.json")?; rag.add_documents_batch(knowledge_base)?; // 3. Wire into form execution let mut form = MyForm::new()?; form.set_rag_service(rag); // 4. During field fill: suggest relevant docs form.on_field_change(&field_name, &user_input)?; // → RAG search triggered automatically // → Results shown as suggestions // 5. User can explore suggested documents let selected_doc = form.get_suggestion_selection()?; form.populate_field_from_doc(&field_name, &selected_doc)?; ```text ## Performance Considerations ### Batch Operations Critical for Scale | Operation | Count | Sequential | Batch | Speedup | |-----------|-------|-----------|-------|---------| | Add docs | 10 | 10ms | 8ms | 1.25x | | Add docs | 50 | 45ms | 25ms | 1.8x | | Add docs | 100 | 85ms | 35ms | **2.4x** | | Add docs | 500 | 380ms | 95ms | **4.0x** | **Key:** HNSW cache rebuild is expensive. Batch ops avoid N rebuilds. ### Production Recommendations 1. **Use `add_documents_batch()`** for bulk operations 2. **Use `remove_documents_batch()`** for deletions 3. **Enable persistence** with `save_to_file()` / `load_from_file()` 4. **Configure weights** based on use case 5. **Monitor scores** to tune `min_score` threshold ## Next Steps 1. Create `typedialog-ai-service` microservice crate 2. Add REST API wrappers for RAG operations 3. Integrate into CLI/TUI/Web backends 4. Add example integration scenarios 5. Document deployment patterns 6. Create Docker compose for development ## Related Code - **AI Backend:** `crates/typedialog-core/src/ai/` - **RAG System:** `crates/typedialog-core/src/ai/rag.rs` - **Batch Ops:** `add_documents_batch()`, `remove_documents_batch()` - **Persistence:** `save_to_file()`, `load_from_file()`