9.4 KiB
9.4 KiB
AI Backend Integration Guide
Problem: Why AI Backend Can't Be Just a Library
The AI backend (RAG, Knowledge Graph, embeddings) must be integrated into actual backends (CLI, TUI, Web) to be useful. Here's how:
Architecture: AI Backend Integration Layers
┌─────────────────────────────────────────────────────────┐
│ USER INTERFACE │
│ (CLI / TUI / Web - Interactive Forms) │
└──────────────────────┬──────────────────────────────────┘
│
┌──────────────────────▼──────────────────────────────────┐
│ AI SERVICE LAYER │
│ • RAG search (semantic + keyword) │
│ • Knowledge graph queries │
│ • Batch operations (efficient) │
│ • Persistence (save/load state) │
└──────────────────────┬──────────────────────────────────┘
│
┌──────────────────────▼──────────────────────────────────┐
│ KNOWLEDGE BASE / STORAGE │
│ • Vector embeddings (HNSW index) │
│ • Full-text index │
│ • Entity relationships (KG) │
│ • Persistent state │
└─────────────────────────────────────────────────────────┘
```text
## Integration Patterns
### 1. CLI Backend Integration
```rust
// Example: Search-enhanced CLI form
use typedialog_core::ai::rag::{RagSystem, RagConfig};
pub struct SearchableForm {
rag: RagSystem,
form: FormDefinition,
}
impl SearchableForm {
pub fn new(form_toml: &str) -> Result<Self> {
let rag = RagSystem::new(RagConfig::default())?;
let form = form_parser::parse_toml(form_toml)?;
Ok(SearchableForm { rag, form })
}
pub fn execute_with_search(&mut self) -> Result<FormResults> {
// Show form
let results = execute_form(&self.form)?;
// Optionally: Search documents based on form data
if let Some(search_query) = results.get("_search") {
let docs = self.rag.retrieve(search_query)?;
println!("Related documents found: {}", docs.len());
for doc in docs {
println!(" - {}: {}", doc.doc_id, doc.content);
}
}
Ok(results)
}
}
```text
**Use Cases:**
- Interactive CLI tools with search suggestions
- Configuration assistants with knowledge base
- Automated documentation lookup during form fill
### 2. TUI Backend Integration
```rust
// Example: Split-pane TUI with RAG search
use ratatui::prelude::*;
pub struct RagTuiPanel {
rag: RagSystem,
search_results: Vec<RetrievalResult>,
selected_result: usize,
}
impl RagTuiPanel {
pub fn draw(&self, f: &mut Frame, area: Rect) {
// Left pane: Search input + results
// Right pane: Selected document preview
// Display semantic score indicators
}
pub fn on_query(&mut self, query: &str) -> Result<()> {
self.search_results = self.rag.retrieve(query)?;
self.selected_result = 0;
Ok(())
}
}
impl Widget for RagTuiPanel {
fn render(self, area: Rect, buf: &mut Buffer) {
// Render search UI with batched results
}
}
```text
**Use Cases:**
- Dashboard with AI-powered search
- Document browser with semantic search
- Knowledge base explorer
- Configuration helper with real-time suggestions
### 3. Web Backend Integration
```rust
// Example: Web form with RAG API endpoint
use axum::{routing::post, Json, Router};
pub struct RagWebService {
rag: RagSystem,
}
impl RagWebService {
pub fn router(mut rag: RagSystem) -> Router {
Router::new()
.route("/search", post(Self::search))
.route("/documents", post(Self::add_documents))
.route("/documents/:id", axum::routing::delete(Self::remove_document))
.with_state(rag)
}
async fn search(
Json(query): Json<SearchRequest>,
) -> Json<Vec<RetrievalResult>> {
let results = self.rag.retrieve(&query.text)?;
Json(results)
}
async fn add_documents(
Json(docs): Json<Vec<(String, String)>>,
) -> Json<ApiResponse> {
self.rag.add_documents_batch(docs)?;
Json(ApiResponse { success: true })
}
}
```text
**Use Cases:**
- Web SaaS with knowledge base search
- Document management system with semantic search
- FAQ finder with intelligent matching
- Support chatbot with context retrieval
## Service Models
### Model 1: Embedded Service (Simple)
```text
Frontend Form → RAG Service (in-memory) → Results
```text
- **Pros:** Fast, no network latency, easy to prototype
- **Cons:** Can't scale, state lost on restart
- **Use when:** Development, small documents, single user
- **Impl:** Load documents on startup, search in-memory
### Model 2: Microservice (Production)
```text
Frontend Form → HTTP REST API → RAG Service → Vector DB / File Store
```text
- **Pros:** Scalable, persistent, can handle large datasets
- **Cons:** Network latency, operational complexity
- **Use when:** Multiple users, large knowledge base, production SaaS
- **Impl:** Axum server with RagSystem, persist to disk/database
### Model 3: Hybrid (Best of Both)
```text
Frontend Form → Local Cache (RAG) → Sync ↔ Remote Service
```text
- **Pros:** Fast local access, keeps in sync, fallback support
- **Cons:** Complexity, consistency challenges
- **Use when:** High availability needed, offline support required
- **Impl:** Embedded RAG with periodic sync to central service
## Implementation Roadmap
### Phase 1: Embedded Integration ✅ (Current)
```rust
// In CLI/TUI/Web backends
let mut rag = RagSystem::new(config)?;
rag.add_documents_batch(docs)?;
let results = rag.retrieve(query)?;
```text
**Status:** Ready - batch operations optimized, persistence working
### Phase 2: Microservice Wrapper (Next)
```rust
// Create: typedialog-ai-service (separate binary)
// Features:
// - REST API with Axum
// - Persistent storage (SQLite / file-based)
// - Health checks, metrics
// - Document management API
```text
**Scope:**
- New binary crate: `crates/typedialog-ai-service/`
- Routes: `/search`, `/documents`, `/status`
- Persistence layer
- Docker support
### Phase 3: Backend Integration
**CLI Integration:**
- Add `--rag` flag to forms
- Search suggestions during field fill
- Context-aware help text
**TUI Integration:**
- Right pane: RAG search results
- Keyboard shortcuts for search
- Result scoring visualization
**Web Integration:**
- AJAX search endpoint
- Autocomplete with semantic results
- Results sidebar in form UI
## Example: End-to-End Integration
```rust
// 1. Initialize RAG during app startup
let mut rag = RagSystem::new(RagConfig {
semantic_weight: 0.6,
keyword_weight: 0.4,
max_results: 5,
min_score: 0.3,
})?;
// 2. Load knowledge base (batch operation - efficient)
let knowledge_base = load_documents_from_file("kb.json")?;
rag.add_documents_batch(knowledge_base)?;
// 3. Wire into form execution
let mut form = MyForm::new()?;
form.set_rag_service(rag);
// 4. During field fill: suggest relevant docs
form.on_field_change(&field_name, &user_input)?;
// → RAG search triggered automatically
// → Results shown as suggestions
// 5. User can explore suggested documents
let selected_doc = form.get_suggestion_selection()?;
form.populate_field_from_doc(&field_name, &selected_doc)?;
```text
## Performance Considerations
### Batch Operations Critical for Scale
| Operation | Count | Sequential | Batch | Speedup |
|-----------|-------|-----------|-------|---------|
| Add docs | 10 | 10ms | 8ms | 1.25x |
| Add docs | 50 | 45ms | 25ms | 1.8x |
| Add docs | 100 | 85ms | 35ms | **2.4x** |
| Add docs | 500 | 380ms | 95ms | **4.0x** |
**Key:** HNSW cache rebuild is expensive. Batch ops avoid N rebuilds.
### Production Recommendations
1. **Use `add_documents_batch()`** for bulk operations
2. **Use `remove_documents_batch()`** for deletions
3. **Enable persistence** with `save_to_file()` / `load_from_file()`
4. **Configure weights** based on use case
5. **Monitor scores** to tune `min_score` threshold
## Next Steps
1. Create `typedialog-ai-service` microservice crate
2. Add REST API wrappers for RAG operations
3. Integrate into CLI/TUI/Web backends
4. Add example integration scenarios
5. Document deployment patterns
6. Create Docker compose for development
## Related Code
- **AI Backend:** `crates/typedialog-core/src/ai/`
- **RAG System:** `crates/typedialog-core/src/ai/rag.rs`
- **Batch Ops:** `add_documents_batch()`, `remove_documents_batch()`
- **Persistence:** `save_to_file()`, `load_from_file()`