Doc-Lifecycle-Manager Integration Guide

Overview

doc-lifecycle-manager (external project) provides complete documentation lifecycle management for VAPORA, including classification, consolidation, semantic search, real-time updates, and enterprise security features.

Project Location: External project (doc-lifecycle-manager) Status: ✅ Enterprise-Ready Tests: 155/155 passing | Zero unsafe code

What is doc-lifecycle-manager?

A comprehensive Rust-based system that handles documentation throughout its entire lifecycle:

Core Capabilities (Phases 1-3)

Automatic Classification: Categorizes docs (vision, design, specs, ADRs, guides, testing, archive)
Duplicate Detection: Finds similar documents with TF-IDF analysis
Semantic RAG Indexing: Vector embeddings for semantic search
mdBook Generation: Auto-generates documentation websites

Enterprise Features (Phases 4-7)

GraphQL API: Semantic document queries with pagination
Real-Time Events: WebSocket streaming of doc updates
Distributed Tracing: OpenTelemetry with W3C Trace Context
Security: mTLS with automatic certificate rotation
Performance: Comprehensive benchmarking with percentiles
Persistence: SurrealDB backend (feature-gated)

Integration Architecture

Data Flow in VAPORA

Frontend/Agents
    ↓
┌─────────────────────────────────┐
│   VAPORA API Layer (Axum)       │
│   ├─ REST endpoints             │
│   └─ WebSocket gateway          │
└─────────────────────────────────┘
    ↓
┌─────────────────────────────────┐
│  doc-lifecycle-manager Services │
│                                 │
│  ├─ GraphQL Resolver            │
│  ├─ WebSocket Manager           │
│  ├─ Document Classifier         │
│  ├─ RAG Indexer                 │
│  └─ mTLS Auth Manager           │
└─────────────────────────────────┘
    ↓
┌─────────────────────────────────┐
│   Data Layer                    │
│   ├─ SurrealDB (vectors)        │
│   ├─ NATS JetStream (events)    │
│   └─ Redis (cache)              │
└─────────────────────────────────┘

Component Integration Points

1. Documenter Agent ↔ doc-lifecycle-manager

#![allow(unused)]
fn main() {
use vapora_doc_lifecycle::prelude::*;

// On task completion
async fn on_task_completed(task_id: &str) {
    let config = PluginConfig::default();
    let mut docs = DocumenterIntegration::new(config)?;
    docs.on_task_completed(task_id).await?;
}
}

2. Frontend ↔ GraphQL API

{
  documentSearch(query: {
    text_query: "authentication"
    limit: 10
  }) {
    results { id title relevance_score }
  }
}

3. Frontend ↔ WebSocket Events

const ws = new WebSocket("ws://vapora/doc-events");
ws.onmessage = (event) => {
  const { event_type, payload } = JSON.parse(event.data);
  // Update UI on document_indexed, document_updated, etc.
};

4. Agent-to-Agent ↔ NATS JetStream

Task Completed Event
  → Documenter Agent (NATS)
    → Classify + Index
      → Broadcast DocumentIndexed Event
        → All Agents notified

Feature Set by Phase

Phase 1: Foundation & Core Library ✅

Error handling and configuration
Core abstractions and types

Phase 2: Extended Implementation ✅

Document Classifier (7 types)
Consolidator (TF-IDF)
RAG Indexer (markdown-aware)
MDBook Generator

Phase 3: CLI & Automation ✅

4 command handlers
62+ Just recipes
5 NuShell scripts

Phase 4: VAPORA Deep Integration ✅

NATS JetStream events
Vector store trait
Plugin system
Agent coordination

Phase 5: Production Hardening ✅

Real NATS integration
DocServer RBAC (4 roles, 3 visibility levels)
Root Files Keeper (auto-update README, CHANGELOG)
Kubernetes manifests (7 YAML files)

Phase 6: Multi-Agent VAPORA ✅

Agent registry with health checking
CI/CD pipeline (GitHub Actions)
Prometheus monitoring rules
Comprehensive documentation

Phase 7: Advanced Features ✅

SurrealDB Backend: Persistent vector store
OpenTelemetry: W3C Trace Context support
GraphQL API: Query builder with semantic search
WebSocket Events: Real-time subscriptions
mTLS Auth: Certificate rotation
Benchmarking: P95/P99 metrics

How to Use in VAPORA

1. Basic Integration (Documenter Agent)

#![allow(unused)]
fn main() {
// In vapora-backend/documenter_agent.rs

use vapora_doc_lifecycle::prelude::*;

impl DocumenterAgent {
    async fn process_task(&self, task: Task) -> Result<()> {
        let config = PluginConfig::default();
        let mut integration = DocumenterIntegration::new(config)?;

        // Automatically classifies, indexes, and generates docs
        integration.on_task_completed(&task.id).await?;

        Ok(())
    }
}
}

2. GraphQL Queries (Frontend/Agents)

# Search for documentation
query SearchDocs($query: String!) {
  documentSearch(query: {
    text_query: $query
    limit: 10
    visibility: "Public"
  }) {
    results {
      id
      title
      path
      relevance_score
      preview
    }
    total_count
    has_more
  }
}

# Get specific document
query GetDoc($id: ID!) {
  document(id: $id) {
    id
    title
    content
    metadata {
      created_at
      updated_at
      owner_id
    }
  }
}

3. Real-Time Updates (Frontend)

// Connect to doc-lifecycle WebSocket
const docWs = new WebSocket('ws://vapora-api/doc-lifecycle/events');

// Subscribe to document changes
docWs.onopen = () => {
  docWs.send(JSON.stringify({
    type: 'subscribe',
    event_types: ['document_indexed', 'document_updated', 'search_index_rebuilt'],
    min_priority: 5
  }));
};

// Handle updates
docWs.onmessage = (event) => {
  const message = JSON.parse(event.data);

  if (message.event_type === 'document_indexed') {
    console.log('New doc indexed:', message.payload);
    // Refresh documentation view
  }
};

4. Distributed Tracing

All operations are automatically traced:

GET /api/documents?search=auth
  trace_id: 0af7651916cd43dd8448eb211c80319c
  span_id: b7ad6b7169203331

  ├─ graphql_resolver [15ms]
  │  ├─ rbac_check [2ms]
  │  └─ semantic_search [12ms]
  └─ response [1ms]

5. mTLS Security

Service-to-service communication is secured:

# Kubernetes secret for certs
apiVersion: v1
kind: Secret
metadata:
  name: doc-lifecycle-certs
data:
  server.crt: <base64>
  server.key: <base64>
  ca.crt: <base64>

Deployment in VAPORA

Kubernetes Manifests Provided

kubernetes/
├── namespace.yaml                    # Create doc-lifecycle namespace
├── configmap.yaml                    # Configuration
├── deployment.yaml                   # Main service (2 replicas)
├── statefulset-nats.yaml            # NATS JetStream (3 replicas)
├── statefulset-surreal.yaml         # SurrealDB (1 replica)
├── service.yaml                      # Internal services
├── rbac.yaml                         # RBAC configuration
└── prometheus-rules.yaml             # Monitoring rules

Quick Deploy

# Deploy to VAPORA cluster
kubectl apply -f /Tools/doc-lifecycle-manager/kubernetes/

# Verify
kubectl get pods -n doc-lifecycle
kubectl get svc -n doc-lifecycle

Configuration via ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: doc-lifecycle-config
  namespace: doc-lifecycle
data:
  config.json: |
    {
      "mode": "full",
      "classification": {
        "auto_classify": true,
        "confidence_threshold": 0.8
      },
      "rag": {
        "enable_embeddings": true,
        "max_chunk_size": 512
      },
      "nats": {
        "server": "nats://nats:4222",
        "jetstream_enabled": true
      },
      "otel": {
        "enabled": true,
        "jaeger_endpoint": "http://jaeger:14268"
      },
      "mtls": {
        "enabled": true,
        "rotation_days": 30
      }
    }

VAPORA Agent Integration

Documenter Agent

#![allow(unused)]
fn main() {
// Processes documentation tasks
pub struct DocumenterAgent {
    integration: DocumenterIntegration,
    nats: NatsEventHandler,
}

impl DocumenterAgent {
    pub async fn handle_task(&self, task: Task) -> Result<()> {
        // 1. Classify document
        self.integration.on_task_completed(&task.id).await?;

        // 2. Broadcast via NATS
        let event = DocsUpdatedEvent {
            task_id: task.id,
            doc_count: 5,
        };
        self.nats.publish_docs_updated(event).await?;

        Ok(())
    }
}
}

Developer Agent (Uses Search)

#![allow(unused)]
fn main() {
// Searches for relevant documentation
pub struct DeveloperAgent;

impl DeveloperAgent {
    pub async fn find_relevant_docs(&self, task: Task) -> Result<Vec<DocumentResult>> {
        // GraphQL query for semantic search
        let query = DocumentQuery {
            text_query: Some(task.description),
            limit: Some(5),
            visibility: Some("Internal".to_string()),
            ..Default::default()
        };

        // Execute search
        resolver.resolve_document_search(query, user).await
    }
}
}

CodeReviewer Agent (Uses Context)

#![allow(unused)]
fn main() {
// Uses documentation as context for reviews
pub struct CodeReviewerAgent;

impl CodeReviewerAgent {
    pub async fn review_with_context(&self, code: &str) -> Result<Review> {
        // Search for related documentation
        let docs = semantic_search(code_summary).await?;

        // Use docs as context in review
        let review = llm_client
            .review_code(code, &docs.to_context_string())
            .await?;

        Ok(review)
    }
}
}

Performance & Scaling

Expected Performance

Operation	Latency	Throughput
Classify doc	<10ms	1000 docs/sec
GraphQL query	<200ms	50 queries/sec
WebSocket broadcast	<20ms	1000 events/sec
Semantic search	<100ms	50 searches/sec
mTLS validation	<5ms	N/A

Resource Requirements

Deployment Resources:

CPU: 2-4 cores (main service)
Memory: 512MB-2GB
Storage: 50GB (SurrealDB + vectors)

NATS Requirements:

CPU: 1-2 cores
Memory: 256MB-1GB
Persistent volume: 20GB

Monitoring & Observability

Prometheus Metrics

# Error rate
rate(doc_lifecycle_errors_total[5m])

# Latency
histogram_quantile(0.99, doc_lifecycle_request_duration_seconds)

# Service availability
up{job="doc-lifecycle"}

Distributed Tracing

Traces are sent to Jaeger in W3C format:

Trace: 0af7651916cd43dd8448eb211c80319c
├─ Span: graphql_resolver
│  ├─ Span: rbac_check
│  └─ Span: semantic_search
└─ Span: response

Health Checks

# Liveness probe
curl http://doc-lifecycle:8080/health/live

# Readiness probe
curl http://doc-lifecycle:8080/health/ready

Configuration Reference

Environment Variables

# Core
DOC_LIFECYCLE_MODE=full                          # minimal|standard|full
DOC_LIFECYCLE_ENABLED=true

# Classification
CLASSIFIER_AUTO_CLASSIFY=true
CLASSIFIER_CONFIDENCE_THRESHOLD=0.8

# RAG/Search
RAG_ENABLE_EMBEDDINGS=true
RAG_MAX_CHUNK_SIZE=512
RAG_CHUNK_OVERLAP=50

# NATS
NATS_SERVER_URL=nats://nats:4222
NATS_JETSTREAM_ENABLED=true

# SurrealDB (optional)
SURREAL_DB_URL=ws://surrealdb:8000
SURREAL_NAMESPACE=vapora
SURREAL_DATABASE=documents

# OpenTelemetry
OTEL_ENABLED=true
OTEL_JAEGER_ENDPOINT=http://jaeger:14268
OTEL_SERVICE_NAME=vapora-doc-lifecycle

# mTLS
MTLS_ENABLED=true
MTLS_SERVER_CERT=/etc/vapora/certs/server.crt
MTLS_SERVER_KEY=/etc/vapora/certs/server.key
MTLS_CA_CERT=/etc/vapora/certs/ca.crt
MTLS_ROTATION_DAYS=30

Integration Checklist

Immediate (Ready Now)

Core features (Phases 1-3)
VAPORA integration (Phase 4)
Production hardening (Phase 5)
Multi-agent support (Phase 6)
Enterprise features (Phase 7)
Kubernetes deployment
GraphQL API
WebSocket events
Distributed tracing
mTLS security

Planned (Phase 8)

Jaeger exporter
SurrealDB live testing
Load testing
Performance tuning
Production deployment guide

Troubleshooting

Common Issues

1. NATS Connection Failed

# Check NATS service
kubectl get svc -n doc-lifecycle
kubectl logs -n doc-lifecycle deployment/nats

2. GraphQL Query Timeout

# Check semantic search performance
# Query execution should be < 200ms
# Check RAG index size

3. WebSocket Disconnection

# Verify WebSocket port is open
# Check subscription history size
# Monitor event broadcast latency

References

Documentation Files:

/Tools/doc-lifecycle-manager/PHASE_7_COMPLETION.md - Phase 7 details
/Tools/doc-lifecycle-manager/PHASES_COMPLETION.md - All phases overview
/Tools/doc-lifecycle-manager/INTEGRATION_WITH_VAPORA.md - Integration guide
/Tools/doc-lifecycle-manager/kubernetes/README.md - K8s deployment

Source Code:

crates/vapora-doc-lifecycle/src/lib.rs - Main library
crates/vapora-doc-lifecycle/src/graphql_api.rs - GraphQL resolver
crates/vapora-doc-lifecycle/src/websocket_events.rs - WebSocket manager
crates/vapora-doc-lifecycle/src/mtls_auth.rs - Security

Support

For questions or issues:

Check documentation in /Tools/doc-lifecycle-manager/
Review test cases for usage examples
Check Kubernetes logs: kubectl logs -n doc-lifecycle <pod>
Monitor with Prometheus/Grafana

Status: ✅ Ready for Production Deployment Last Updated: 2025-11-10 Maintainer: VAPORA Team

Keyboard shortcuts

VAPORA Platform Documentation