Compare commits
No commits in common. "cc55b97678e18632a9316ed6571caafa4226fcc3" and "d3316d75ba59ced5a007a1145f8b63e7866b4159" have entirely different histories.
cc55b97678
...
d3316d75ba
@ -25,7 +25,7 @@
|
|||||||
// It does NOT catch malformed closing fences with language specifiers (e.g., ```plaintext)
|
// It does NOT catch malformed closing fences with language specifiers (e.g., ```plaintext)
|
||||||
// CommonMark spec requires closing fences to be ``` only (no language)
|
// CommonMark spec requires closing fences to be ``` only (no language)
|
||||||
// Use separate validation script to check closing fences
|
// Use separate validation script to check closing fences
|
||||||
"MD040": true, // fenced-code-language (relaxed - flexible language specifiers)
|
"MD040": false, // fenced-code-language (relaxed - flexible language specifiers)
|
||||||
|
|
||||||
// Formatting - strict whitespace
|
// Formatting - strict whitespace
|
||||||
"MD009": true, // no-hard-tabs
|
"MD009": true, // no-hard-tabs
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
135
CHANGELOG.md
135
CHANGELOG.md
@ -7,90 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
### Added - Workflow Orchestrator (v1.2.0)
|
### Added
|
||||||
|
|
||||||
- **Multi-Stage Workflow Engine**: Complete orchestration system with short-lived agent contexts
|
|
||||||
- `vapora-workflow-engine` crate (26 tests)
|
|
||||||
- 95% cache token cost reduction (from $840/month to $110/month via context management)
|
|
||||||
- Short-lived agent contexts prevent cache token accumulation
|
|
||||||
- Artifact passing between stages (ADR, Code, TestResults, Review, Documentation)
|
|
||||||
- Event-driven coordination via NATS pub/sub for stage progression
|
|
||||||
- Approval gates for governance and quality control
|
|
||||||
- State machine with validated transitions (Draft → Active → WaitingApproval → Completed/Failed)
|
|
||||||
|
|
||||||
- **Workflow Templates**: 4 production-ready templates with stage definitions
|
|
||||||
- **feature_development** (5 stages): architecture_design → implementation (2x parallel) → testing → code_review (approval) → deployment (approval)
|
|
||||||
- **bugfix** (4 stages): investigation → fix_implementation → testing → deployment
|
|
||||||
- **documentation_update** (3 stages): content_creation → review (approval) → publish
|
|
||||||
- **security_audit** (4 stages): code_analysis → penetration_testing → remediation → verification (approval)
|
|
||||||
- Configuration in `config/workflows.toml` with role assignments and agent limits
|
|
||||||
|
|
||||||
- **Kogral Integration**: Filesystem-based knowledge enrichment
|
|
||||||
- Automatic context enrichment from `.kogral/` directory structure
|
|
||||||
- Guidelines: `.kogral/guidelines/{workflow_name}.md`
|
|
||||||
- Patterns: `.kogral/patterns/*.md` (all matching patterns)
|
|
||||||
- ADRs: `.kogral/adrs/*.md` (5 most recent decisions)
|
|
||||||
- Configurable via `KOGRAL_PATH` environment variable
|
|
||||||
- Graceful fallback with warnings if knowledge files missing
|
|
||||||
- Full async I/O with `tokio::fs` operations
|
|
||||||
|
|
||||||
- **CLI Commands**: Complete workflow management from terminal
|
|
||||||
- `vapora-cli` crate with 6 commands
|
|
||||||
- **start**: Launch workflow from template with optional context file
|
|
||||||
- **list**: Display all active workflows in formatted table
|
|
||||||
- **status**: Get detailed workflow status with progress tracking
|
|
||||||
- **approve**: Approve stage waiting for approval (with approver tracking)
|
|
||||||
- **cancel**: Cancel running workflow with reason logging
|
|
||||||
- **templates**: List available workflow templates
|
|
||||||
- Colored terminal output with `colored` crate
|
|
||||||
- UTF8 table formatting with `comfy-table`
|
|
||||||
- HTTP client pattern (communicates with backend REST API)
|
|
||||||
- Environment variable support: `VAPORA_API_URL`
|
|
||||||
|
|
||||||
- **Backend REST API**: 6 workflow orchestration endpoints
|
|
||||||
- `POST /api/workflows/start` - Start workflow from template
|
|
||||||
- `GET /api/workflows` - List all workflows
|
|
||||||
- `GET /api/workflows/{id}` - Get workflow status
|
|
||||||
- `POST /api/workflows/{id}/approve` - Approve stage
|
|
||||||
- `POST /api/workflows/{id}/cancel` - Cancel workflow
|
|
||||||
- `GET /api/workflows/templates` - List templates
|
|
||||||
- Full integration with SwarmCoordinator for agent task assignment
|
|
||||||
- Real-time workflow state updates
|
|
||||||
- WebSocket support for workflow progress streaming
|
|
||||||
|
|
||||||
- **Documentation**: Comprehensive guides and decision records
|
|
||||||
- **ADR-0028**: Workflow Orchestrator architecture decision (275 lines)
|
|
||||||
- Root cause analysis: monolithic session pattern → 3.82B cache tokens
|
|
||||||
- Cost projection: $840/month → $110/month (87% reduction)
|
|
||||||
- Solution: short-lived agent contexts with artifact passing
|
|
||||||
- Trade-offs and alternatives evaluation
|
|
||||||
- **workflow-orchestrator.md**: Complete feature documentation (538 lines)
|
|
||||||
- Architecture overview with component interaction diagrams
|
|
||||||
- 4 workflow templates with stage breakdowns
|
|
||||||
- REST API reference with request/response examples
|
|
||||||
- Kogral integration details
|
|
||||||
- Prometheus metrics reference
|
|
||||||
- Troubleshooting guide
|
|
||||||
- **cli-commands.md**: CLI reference manual (614 lines)
|
|
||||||
- Installation instructions
|
|
||||||
- Complete command reference with examples
|
|
||||||
- Workflow template usage patterns
|
|
||||||
- CI/CD integration examples
|
|
||||||
- Error handling and recovery
|
|
||||||
- **overview.md**: Updated with workflow orchestrator section
|
|
||||||
|
|
||||||
- **Cost Optimization**: Real-world production savings
|
|
||||||
- Before: Monolithic sessions accumulating 3.82B cache tokens/month
|
|
||||||
- After: Short-lived contexts with 190M cache tokens/month
|
|
||||||
- Savings: $730/month (95% reduction)
|
|
||||||
- Per-role breakdown:
|
|
||||||
- Architect: $120 → $6 (95% reduction)
|
|
||||||
- Developer: $360 → $18 (95% reduction)
|
|
||||||
- Reviewer: $240 → $12 (95% reduction)
|
|
||||||
- Tester: $120 → $6 (95% reduction)
|
|
||||||
- ROI: Infrastructure cost paid back in < 1 week
|
|
||||||
|
|
||||||
### Added - Comprehensive Examples System
|
|
||||||
|
|
||||||
- **Comprehensive Examples System**: 26+ executable examples demonstrating all VAPORA capabilities
|
- **Comprehensive Examples System**: 26+ executable examples demonstrating all VAPORA capabilities
|
||||||
- **Basic Examples (6)**: Foundation for each core crate
|
- **Basic Examples (6)**: Foundation for each core crate
|
||||||
@ -163,61 +80,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|
||||||
- **Embeddings Provider Verification**
|
|
||||||
- Confirmed HuggingFace embeddings compile correctly (no errors)
|
|
||||||
- All embedding provider tests passing (Ollama, OpenAI, HuggingFace)
|
|
||||||
- vapora-llm-router: 53 tests passing (30 unit + 11 budget + 12 cost)
|
|
||||||
- Factory function supports 3 providers: Ollama, OpenAI, HuggingFace
|
|
||||||
- Models supported: BGE (small/base/large), MiniLM, MPNet, custom models
|
|
||||||
|
|
||||||
- **Compilation & Testing**
|
- **Compilation & Testing**
|
||||||
- Eliminated all unused import warnings in vapora-backend
|
- Eliminated all unused import warnings in vapora-backend
|
||||||
- Suppressed architectural dead code with appropriate attributes
|
- Suppressed architectural dead code with appropriate attributes
|
||||||
- All 55 tests passing in vapora-backend
|
- All 55 tests passing in vapora-backend
|
||||||
- 0 compilation errors, clean build output
|
- 0 compilation errors, clean build output
|
||||||
|
|
||||||
### Technical Details - Workflow Orchestrator
|
### Technical Details
|
||||||
|
|
||||||
- **New Crates Created (2)**:
|
|
||||||
- `crates/vapora-workflow-engine/` - Core orchestration engine (2,431 lines)
|
|
||||||
- `src/orchestrator.rs` (864 lines) - Workflow lifecycle management + Kogral integration
|
|
||||||
- `src/state.rs` (321 lines) - State machine with validated transitions
|
|
||||||
- `src/template.rs` (298 lines) - Template loading from TOML
|
|
||||||
- `src/artifact.rs` (187 lines) - Inter-stage artifact serialization
|
|
||||||
- `src/events.rs` (156 lines) - NATS event publishing/subscription
|
|
||||||
- `tests/` (26 tests) - Unit + integration tests
|
|
||||||
- `crates/vapora-cli/` - Command-line interface (671 lines)
|
|
||||||
- `src/main.rs` - CLI entry point with clap
|
|
||||||
- `src/client.rs` - HTTP client for backend API
|
|
||||||
- `src/commands.rs` - Command definitions
|
|
||||||
- `src/output.rs` - Terminal UI with colored tables
|
|
||||||
|
|
||||||
- **Modified Files (4)**:
|
|
||||||
- `crates/vapora-backend/src/api/workflow_orchestrator.rs` (NEW) - REST API handlers
|
|
||||||
- `crates/vapora-backend/src/api/mod.rs` - Route registration
|
|
||||||
- `crates/vapora-backend/src/api/state.rs` - Orchestrator state injection
|
|
||||||
- `Cargo.toml` - Workspace members + dependencies
|
|
||||||
|
|
||||||
- **Configuration Files (1)**:
|
|
||||||
- `config/workflows.toml` - Workflow template definitions
|
|
||||||
- 4 templates with stage configurations
|
|
||||||
- Role assignments per stage
|
|
||||||
- Agent limit configurations
|
|
||||||
- Approval requirements
|
|
||||||
|
|
||||||
- **Test Suite**:
|
|
||||||
- Workflow Engine: 26 tests (state transitions, template loading, Kogral integration)
|
|
||||||
- Backend Integration: 5 tests (REST API endpoints)
|
|
||||||
- CLI: Manual testing (no automated tests yet)
|
|
||||||
- Total new tests: 31
|
|
||||||
|
|
||||||
- **Build Status**: Clean compilation
|
|
||||||
- `cargo build --workspace` ✅
|
|
||||||
- `cargo clippy --workspace -- -D warnings` ✅
|
|
||||||
- `cargo test -p vapora-workflow-engine` ✅ (26/26 passing)
|
|
||||||
- `cargo test -p vapora-backend` ✅ (55/55 passing)
|
|
||||||
|
|
||||||
### Technical Details - General
|
|
||||||
|
|
||||||
- **Architecture**: Refactored unused imports from workflow and API modules
|
- **Architecture**: Refactored unused imports from workflow and API modules
|
||||||
- Tests moved to test-only scope for AgentConfig/RegistryConfig types
|
- Tests moved to test-only scope for AgentConfig/RegistryConfig types
|
||||||
|
|||||||
4835
Cargo.lock
generated
4835
Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
13
Cargo.toml
13
Cargo.toml
@ -14,8 +14,6 @@ members = [
|
|||||||
"crates/vapora-analytics",
|
"crates/vapora-analytics",
|
||||||
"crates/vapora-swarm",
|
"crates/vapora-swarm",
|
||||||
"crates/vapora-telemetry",
|
"crates/vapora-telemetry",
|
||||||
"crates/vapora-workflow-engine",
|
|
||||||
"crates/vapora-cli",
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[workspace.package]
|
[workspace.package]
|
||||||
@ -39,7 +37,6 @@ vapora-knowledge-graph = { path = "crates/vapora-knowledge-graph" }
|
|||||||
vapora-analytics = { path = "crates/vapora-analytics" }
|
vapora-analytics = { path = "crates/vapora-analytics" }
|
||||||
vapora-swarm = { path = "crates/vapora-swarm" }
|
vapora-swarm = { path = "crates/vapora-swarm" }
|
||||||
vapora-telemetry = { path = "crates/vapora-telemetry" }
|
vapora-telemetry = { path = "crates/vapora-telemetry" }
|
||||||
vapora-workflow-engine = { path = "crates/vapora-workflow-engine" }
|
|
||||||
|
|
||||||
# SecretumVault - Post-quantum secrets management
|
# SecretumVault - Post-quantum secrets management
|
||||||
secretumvault = { path = "../secretumvault", default-features = false, features = ["server", "surrealdb-storage", "openssl", "cedar"] }
|
secretumvault = { path = "../secretumvault", default-features = false, features = ["server", "surrealdb-storage", "openssl", "cedar"] }
|
||||||
@ -108,15 +105,10 @@ base64 = { version = "0.22" }
|
|||||||
dotenv = "0.15.0"
|
dotenv = "0.15.0"
|
||||||
once_cell = "1.21.3"
|
once_cell = "1.21.3"
|
||||||
|
|
||||||
# CLI
|
# TLS Support
|
||||||
clap = { version = "4.5", features = ["derive", "env"] }
|
axum-server = { version = "0.7", features = ["tls-rustls"] }
|
||||||
colored = "2.1"
|
|
||||||
comfy-table = "7.1"
|
|
||||||
|
|
||||||
# TLS Support (native tokio-rustls, no axum-server)
|
|
||||||
rustls = { version = "0.23" }
|
rustls = { version = "0.23" }
|
||||||
rustls-pemfile = { version = "2.2" }
|
rustls-pemfile = { version = "2.2" }
|
||||||
tokio-rustls = { version = "0.26" }
|
|
||||||
|
|
||||||
# Authentication & Authorization
|
# Authentication & Authorization
|
||||||
jsonwebtoken = { version = "10.2", features = ["rust_crypto"] }
|
jsonwebtoken = { version = "10.2", features = ["rust_crypto"] }
|
||||||
@ -161,6 +153,7 @@ notify = { version = "8.2.0", default-features = false, features = ["macos_fseve
|
|||||||
ignore = "0.4"
|
ignore = "0.4"
|
||||||
|
|
||||||
# CLI support
|
# CLI support
|
||||||
|
clap = { version = "4.5", features = ["derive"] }
|
||||||
dialoguer = "0.12"
|
dialoguer = "0.12"
|
||||||
console = "0.16"
|
console = "0.16"
|
||||||
indicatif = "0.18"
|
indicatif = "0.18"
|
||||||
|
|||||||
44
README.md
44
README.md
@ -12,7 +12,7 @@
|
|||||||
[](https://www.rust-lang.org)
|
[](https://www.rust-lang.org)
|
||||||
[](https://kubernetes.io)
|
[](https://kubernetes.io)
|
||||||
[](https://istio.io)
|
[](https://istio.io)
|
||||||
[](crates/)
|
[](crates/)
|
||||||
|
|
||||||
[Features](#features) • [Quick Start](#quick-start) • [Architecture](#architecture) • [Docs](docs/) • [Contributing](#contributing)
|
[Features](#features) • [Quick Start](#quick-start) • [Architecture](#architecture) • [Docs](docs/) • [Contributing](#contributing)
|
||||||
|
|
||||||
@ -32,7 +32,7 @@
|
|||||||
|
|
||||||
## 🌟 What is Vapora v1.2?
|
## 🌟 What is Vapora v1.2?
|
||||||
|
|
||||||
**VAPORA** is a **15-crate Rust workspace** (244+ tests) delivering an **intelligent development orchestration platform** where teams and AI agents collaborate seamlessly to solve the 4 critical problems in parallel:
|
**VAPORA** is a **13-crate Rust workspace** (218+ tests) delivering an **intelligent development orchestration platform** where teams and AI agents collaborate seamlessly to solve the 4 critical problems in parallel:
|
||||||
|
|
||||||
- ✅ **Context Switching** (Developers unified in one system instead of jumping between tools)
|
- ✅ **Context Switching** (Developers unified in one system instead of jumping between tools)
|
||||||
- ✅ **Knowledge Fragmentation** (Team decisions, code, and docs discoverable with RAG)
|
- ✅ **Knowledge Fragmentation** (Team decisions, code, and docs discoverable with RAG)
|
||||||
@ -65,20 +65,6 @@
|
|||||||
- **Smart Organization**: Feature tags, priority levels, task ordering
|
- **Smart Organization**: Feature tags, priority levels, task ordering
|
||||||
- **Responsive Design**: Works seamlessly from mobile to ultra-wide displays
|
- **Responsive Design**: Works seamlessly from mobile to ultra-wide displays
|
||||||
|
|
||||||
### 🔄 Workflow Orchestrator (v1.2.0)
|
|
||||||
|
|
||||||
- **Multi-Stage Pipelines**: Orchestrate complex workflows with approval gates
|
|
||||||
- **Short-Lived Agent Contexts**: 95% cache token reduction (from $840/month to $110/month)
|
|
||||||
- **Artifact Passing**: ADR, Code, TestResults, Review, Documentation artifacts between stages
|
|
||||||
- **Kogral Integration**: Automatic context enrichment from `.kogral/` filesystem (guidelines, patterns, ADRs)
|
|
||||||
- **CLI Management**: 6 commands (start, list, status, approve, cancel, templates)
|
|
||||||
- **Event-Driven Coordination**: NATS pub/sub for workflow stage progression
|
|
||||||
- **4 Workflow Templates**:
|
|
||||||
- `feature_development` (5 stages: design → implementation → testing → review → deployment)
|
|
||||||
- `bugfix` (4 stages: investigation → fix → testing → deployment)
|
|
||||||
- `documentation_update` (3 stages: creation → review → publish)
|
|
||||||
- `security_audit` (4 stages: analysis → testing → remediation → verification)
|
|
||||||
|
|
||||||
### 🧠 Intelligent Learning & Cost Optimization (Phase 5.3 + 5.4)
|
### 🧠 Intelligent Learning & Cost Optimization (Phase 5.3 + 5.4)
|
||||||
|
|
||||||
- **Per-Task-Type Learning**: Agents build expertise profiles from execution history
|
- **Per-Task-Type Learning**: Agents build expertise profiles from execution history
|
||||||
@ -216,24 +202,13 @@
|
|||||||
cd migrations && surrealdb import --conn http://localhost:8000 *.surql
|
cd migrations && surrealdb import --conn http://localhost:8000 *.surql
|
||||||
|
|
||||||
# Start backend
|
# Start backend
|
||||||
cd crates/vapora-backend
|
cd ../vapora-backend
|
||||||
cargo run
|
cargo run
|
||||||
|
|
||||||
# Start frontend (new terminal)
|
# Start frontend (new terminal)
|
||||||
cd crates/vapora-frontend
|
cd ../vapora-frontend
|
||||||
trunk serve
|
trunk serve
|
||||||
|
|
||||||
# Install CLI (optional - for workflow management)
|
|
||||||
cd crates/vapora-cli
|
|
||||||
cargo build --release
|
|
||||||
cp target/release/vapora ~/.local/bin/
|
|
||||||
|
|
||||||
# CLI Usage
|
|
||||||
vapora workflow start --template feature_development
|
|
||||||
vapora workflow list
|
|
||||||
vapora workflow status <id>
|
|
||||||
vapora workflow approve <id> --approver "Your Name"
|
|
||||||
|
|
||||||
Visit http://localhost:3000 🎉
|
Visit http://localhost:3000 🎉
|
||||||
|
|
||||||
Docker Compose (Full Stack)
|
Docker Compose (Full Stack)
|
||||||
@ -382,8 +357,6 @@ vapora/
|
|||||||
│ ├── vapora-llm-router/ # Multi-provider routing + budget (53 tests)
|
│ ├── vapora-llm-router/ # Multi-provider routing + budget (53 tests)
|
||||||
│ ├── vapora-swarm/ # Swarm coordination + Prometheus (6 tests)
|
│ ├── vapora-swarm/ # Swarm coordination + Prometheus (6 tests)
|
||||||
│ ├── vapora-knowledge-graph/ # Temporal KG + learning curves (13 tests)
|
│ ├── vapora-knowledge-graph/ # Temporal KG + learning curves (13 tests)
|
||||||
│ ├── vapora-workflow-engine/ # Multi-stage workflows + Kogral integration (26 tests)
|
|
||||||
│ ├── vapora-cli/ # CLI commands (start, list, approve, cancel, etc.)
|
|
||||||
│ ├── vapora-frontend/ # Leptos WASM UI (Kanban)
|
│ ├── vapora-frontend/ # Leptos WASM UI (Kanban)
|
||||||
│ ├── vapora-mcp-server/ # MCP protocol gateway
|
│ ├── vapora-mcp-server/ # MCP protocol gateway
|
||||||
│ ├── vapora-tracking/ # Task/project storage layer
|
│ ├── vapora-tracking/ # Task/project storage layer
|
||||||
@ -401,16 +374,9 @@ vapora/
|
|||||||
├── kubernetes/ # K8s manifests (base, overlays, platform)
|
├── kubernetes/ # K8s manifests (base, overlays, platform)
|
||||||
├── migrations/ # SurrealDB migrations
|
├── migrations/ # SurrealDB migrations
|
||||||
├── config/ # Configuration files (TOML)
|
├── config/ # Configuration files (TOML)
|
||||||
│ ├── vapora.toml # Backend configuration
|
|
||||||
│ ├── agents.toml # Agent roles and limits
|
|
||||||
│ ├── workflows.toml # Workflow templates
|
|
||||||
│ └── agent-budgets.toml # Budget enforcement config
|
|
||||||
└── docs/ # Product documentation
|
└── docs/ # Product documentation
|
||||||
├── adrs/ # Architecture Decision Records
|
|
||||||
├── features/ # Feature documentation
|
|
||||||
└── setup/ # Installation and CLI guides
|
|
||||||
|
|
||||||
# Total: 15 crates, 244+ tests
|
# Total: 13 crates, 218+ tests
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
@ -461,7 +461,7 @@
|
|||||||
<p class="tagline">Evaporate complexity</p>
|
<p class="tagline">Evaporate complexity</p>
|
||||||
<h1
|
<h1
|
||||||
data-en="Development Flows<br>When Teams and AI Orchestrate"
|
data-en="Development Flows<br>When Teams and AI Orchestrate"
|
||||||
data-es="El Desarrollo Fluye<br>Cuando los Equipos y la IA Orquestan"
|
data-es="El Desarrollo Fluye<br>Cuando Equipos e IA Orquestan"
|
||||||
>
|
>
|
||||||
Development Flows
|
Development Flows
|
||||||
</h1>
|
</h1>
|
||||||
@ -474,15 +474,15 @@
|
|||||||
>
|
>
|
||||||
<span
|
<span
|
||||||
data-en="orchestrate pipelines for design, implementation, testing, documentation and deployment. Agents learn from history and optimize costs automatically."
|
data-en="orchestrate pipelines for design, implementation, testing, documentation and deployment. Agents learn from history and optimize costs automatically."
|
||||||
data-es=" que orquestan pipelines para diseño, implementación, testing, documentación y deployment. Los agentes aprenden del historial y optimizan costos automáticamente."
|
data-es="orquestan pipelines para diseño, implementación, testing, documentación y deployment. Los agentes aprenden del historial y optimizan costos automáticamente."
|
||||||
>orchestrate pipelines for design, implementation, testing,
|
>orchestrate pipelines for design, implementation, testing,
|
||||||
documentation and deployment. Agents learn from history and optimize
|
documentation and deployment. Agents learn from history and optimize
|
||||||
costs automatically.
|
costs automatically.</span
|
||||||
</span>
|
>
|
||||||
<br><span><strong data-en="100% self-hosted." data-es="100% self-hosted."
|
<strong data-en="100% self-hosted." data-es="100% self-hosted."
|
||||||
>100% self-hosted.</strong
|
>100% self-hosted.</strong
|
||||||
>
|
>
|
||||||
</span>
|
<span data-en="" data-es=""></span>
|
||||||
</p>
|
</p>
|
||||||
</header>
|
</header>
|
||||||
|
|
||||||
|
|||||||
@ -1,36 +1,50 @@
|
|||||||
|
# Workflow Engine Configuration
|
||||||
|
# Phase 0: Workflow templates and execution rules
|
||||||
|
|
||||||
[engine]
|
[engine]
|
||||||
|
# Maximum parallel tasks in a workflow
|
||||||
max_parallel_tasks = 10
|
max_parallel_tasks = 10
|
||||||
|
|
||||||
|
# Workflow timeout (seconds)
|
||||||
workflow_timeout = 3600
|
workflow_timeout = 3600
|
||||||
|
|
||||||
|
# Enable approval gates
|
||||||
approval_gates_enabled = true
|
approval_gates_enabled = true
|
||||||
|
|
||||||
|
# Workflow Templates
|
||||||
|
|
||||||
[[workflows]]
|
[[workflows]]
|
||||||
name = "feature_development"
|
name = "feature_development"
|
||||||
trigger = "manual"
|
description = "Complete feature development workflow"
|
||||||
|
trigger = "task_type:feature"
|
||||||
|
|
||||||
|
# Workflow stages (sequential unless marked parallel)
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "architecture_design"
|
name = "architecture"
|
||||||
agents = ["architect"]
|
agents = ["architect"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
approval_required = true
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "implementation"
|
name = "implementation"
|
||||||
agents = ["developer", "developer"]
|
agents = ["developer"]
|
||||||
|
parallel = true
|
||||||
|
max_parallel = 3
|
||||||
|
|
||||||
|
[[workflows.stages]]
|
||||||
|
name = "review"
|
||||||
|
agents = ["code_reviewer", "security"]
|
||||||
parallel = true
|
parallel = true
|
||||||
max_parallel = 2
|
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "testing"
|
name = "testing"
|
||||||
agents = ["tester"]
|
agents = ["tester"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "code_review"
|
name = "documentation"
|
||||||
agents = ["reviewer"]
|
agents = ["documenter"]
|
||||||
parallel = false
|
parallel = true
|
||||||
approval_required = true
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "deployment"
|
name = "deployment"
|
||||||
@ -40,78 +54,76 @@ approval_required = true
|
|||||||
|
|
||||||
[[workflows]]
|
[[workflows]]
|
||||||
name = "bugfix"
|
name = "bugfix"
|
||||||
trigger = "manual"
|
description = "Bug fix workflow"
|
||||||
|
trigger = "task_type:bugfix"
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "investigation"
|
name = "analysis"
|
||||||
agents = ["developer"]
|
agents = ["developer"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "fix_implementation"
|
name = "fix_implementation"
|
||||||
agents = ["developer"]
|
agents = ["developer"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
|
||||||
|
[[workflows.stages]]
|
||||||
|
name = "review"
|
||||||
|
agents = ["code_reviewer"]
|
||||||
|
parallel = false
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "testing"
|
name = "testing"
|
||||||
agents = ["tester"]
|
agents = ["tester"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "deployment"
|
name = "deployment"
|
||||||
agents = ["devops"]
|
agents = ["devops"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows]]
|
[[workflows]]
|
||||||
name = "documentation_update"
|
name = "documentation_update"
|
||||||
trigger = "manual"
|
description = "Update documentation workflow"
|
||||||
|
trigger = "task_type:documentation"
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "content_creation"
|
name = "content_creation"
|
||||||
agents = ["technical_writer"]
|
agents = ["documenter"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "review"
|
name = "review"
|
||||||
agents = ["reviewer"]
|
agents = ["code_reviewer", "project_manager"]
|
||||||
parallel = false
|
parallel = true
|
||||||
approval_required = true
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "publish"
|
name = "publish"
|
||||||
agents = ["devops"]
|
agents = ["devops"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows]]
|
[[workflows]]
|
||||||
name = "security_audit"
|
name = "security_audit"
|
||||||
trigger = "manual"
|
description = "Security audit workflow"
|
||||||
|
trigger = "task_type:security"
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "code_analysis"
|
name = "audit"
|
||||||
agents = ["security_engineer"]
|
agents = ["security"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "penetration_testing"
|
|
||||||
agents = ["security_engineer"]
|
|
||||||
parallel = false
|
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "remediation"
|
name = "remediation"
|
||||||
agents = ["developer"]
|
agents = ["developer"]
|
||||||
parallel = false
|
parallel = true
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
[[workflows.stages]]
|
||||||
name = "verification"
|
name = "verification"
|
||||||
agents = ["security_engineer"]
|
agents = ["security", "tester"]
|
||||||
|
parallel = true
|
||||||
|
|
||||||
|
[[workflows.stages]]
|
||||||
|
name = "approval"
|
||||||
|
agents = ["decision_maker"]
|
||||||
parallel = false
|
parallel = false
|
||||||
approval_required = true
|
approval_required = true
|
||||||
|
|||||||
@ -2,14 +2,12 @@
|
|||||||
// Phase 2: Complete implementation with NATS integration
|
// Phase 2: Complete implementation with NATS integration
|
||||||
|
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::path::PathBuf;
|
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use chrono::Utc;
|
use chrono::Utc;
|
||||||
use thiserror::Error;
|
use thiserror::Error;
|
||||||
use tracing::{debug, info, warn};
|
use tracing::{debug, info, warn};
|
||||||
use uuid::Uuid;
|
use uuid::Uuid;
|
||||||
use vapora_shared::validation::{SchemaRegistry, ValidationPipeline};
|
|
||||||
|
|
||||||
use crate::learning_profile::{ExecutionData, LearningProfile, TaskTypeExpertise};
|
use crate::learning_profile::{ExecutionData, LearningProfile, TaskTypeExpertise};
|
||||||
use crate::messages::{AgentMessage, TaskAssignment};
|
use crate::messages::{AgentMessage, TaskAssignment};
|
||||||
@ -32,9 +30,6 @@ pub enum CoordinatorError {
|
|||||||
|
|
||||||
#[error("Invalid task state: {0}")]
|
#[error("Invalid task state: {0}")]
|
||||||
InvalidTaskState(String),
|
InvalidTaskState(String),
|
||||||
|
|
||||||
#[error("Validation error: {0}")]
|
|
||||||
ValidationError(String),
|
|
||||||
}
|
}
|
||||||
|
|
||||||
use vapora_llm_router::BudgetManager;
|
use vapora_llm_router::BudgetManager;
|
||||||
@ -51,7 +46,6 @@ pub struct AgentCoordinator {
|
|||||||
swarm_coordinator: Option<Arc<SwarmCoordinator>>,
|
swarm_coordinator: Option<Arc<SwarmCoordinator>>,
|
||||||
learning_profiles: Arc<std::sync::RwLock<HashMap<String, LearningProfile>>>,
|
learning_profiles: Arc<std::sync::RwLock<HashMap<String, LearningProfile>>>,
|
||||||
budget_manager: Option<Arc<BudgetManager>>,
|
budget_manager: Option<Arc<BudgetManager>>,
|
||||||
validation: Arc<ValidationPipeline>,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
impl AgentCoordinator {
|
impl AgentCoordinator {
|
||||||
@ -106,23 +100,12 @@ impl AgentCoordinator {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Initialize validation pipeline
|
|
||||||
let schema_path = resolve_schema_dir();
|
|
||||||
let schema_registry = Arc::new(SchemaRegistry::new(schema_path.clone()));
|
|
||||||
let validation = Arc::new(ValidationPipeline::new(schema_registry));
|
|
||||||
|
|
||||||
info!(
|
|
||||||
"Initialized validation pipeline with schema dir: {}",
|
|
||||||
schema_path.display()
|
|
||||||
);
|
|
||||||
|
|
||||||
Ok(Self {
|
Ok(Self {
|
||||||
registry,
|
registry,
|
||||||
nats_client,
|
nats_client,
|
||||||
swarm_coordinator: Some(swarm_coordinator),
|
swarm_coordinator: Some(swarm_coordinator),
|
||||||
learning_profiles: Arc::new(std::sync::RwLock::new(HashMap::new())),
|
learning_profiles: Arc::new(std::sync::RwLock::new(HashMap::new())),
|
||||||
budget_manager: None,
|
budget_manager: None,
|
||||||
validation,
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -135,18 +118,12 @@ impl AgentCoordinator {
|
|||||||
swarm_coordinator.register_agent(profile.clone()).ok();
|
swarm_coordinator.register_agent(profile.clone()).ok();
|
||||||
}
|
}
|
||||||
|
|
||||||
// Initialize validation pipeline
|
|
||||||
let schema_path = resolve_schema_dir();
|
|
||||||
let schema_registry = Arc::new(SchemaRegistry::new(schema_path));
|
|
||||||
let validation = Arc::new(ValidationPipeline::new(schema_registry));
|
|
||||||
|
|
||||||
Self {
|
Self {
|
||||||
registry,
|
registry,
|
||||||
nats_client: None,
|
nats_client: None,
|
||||||
swarm_coordinator: Some(swarm_coordinator),
|
swarm_coordinator: Some(swarm_coordinator),
|
||||||
learning_profiles: Arc::new(std::sync::RwLock::new(HashMap::new())),
|
learning_profiles: Arc::new(std::sync::RwLock::new(HashMap::new())),
|
||||||
budget_manager: None,
|
budget_manager: None,
|
||||||
validation,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -171,42 +148,6 @@ impl AgentCoordinator {
|
|||||||
context: String,
|
context: String,
|
||||||
priority: u32,
|
priority: u32,
|
||||||
) -> Result<String, CoordinatorError> {
|
) -> Result<String, CoordinatorError> {
|
||||||
// Validate inputs against schema
|
|
||||||
let input = serde_json::json!({
|
|
||||||
"role": role,
|
|
||||||
"title": &title,
|
|
||||||
"description": &description,
|
|
||||||
"context": &context,
|
|
||||||
"priority": priority,
|
|
||||||
});
|
|
||||||
|
|
||||||
let validation_result = self
|
|
||||||
.validation
|
|
||||||
.validate("agents/task_assignment", &input)
|
|
||||||
.await
|
|
||||||
.map_err(|e| CoordinatorError::ValidationError(e.to_string()))?;
|
|
||||||
|
|
||||||
if !validation_result.valid {
|
|
||||||
let error_messages: Vec<String> = validation_result
|
|
||||||
.errors
|
|
||||||
.iter()
|
|
||||||
.map(|e| e.to_string())
|
|
||||||
.collect();
|
|
||||||
|
|
||||||
warn!(
|
|
||||||
"Task assignment validation failed for role {}: {}",
|
|
||||||
role,
|
|
||||||
error_messages.join(", ")
|
|
||||||
);
|
|
||||||
|
|
||||||
return Err(CoordinatorError::ValidationError(format!(
|
|
||||||
"Validation failed: {}",
|
|
||||||
error_messages.join(", ")
|
|
||||||
)));
|
|
||||||
}
|
|
||||||
|
|
||||||
info!("Task assignment validated successfully for role: {}", role);
|
|
||||||
|
|
||||||
// Get all available candidates for role
|
// Get all available candidates for role
|
||||||
let all_agents = self.registry.get_agents_by_role(role);
|
let all_agents = self.registry.get_agents_by_role(role);
|
||||||
let candidates: Vec<_> = all_agents
|
let candidates: Vec<_> = all_agents
|
||||||
@ -603,47 +544,6 @@ fn extract_task_type(title: &str, description: &str, role: &str) -> String {
|
|||||||
role.to_string()
|
role.to_string()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Resolve schema directory path.
|
|
||||||
/// Priority: VAPORA_SCHEMA_DIR env var > workspace root detection > relative
|
|
||||||
/// fallback
|
|
||||||
fn resolve_schema_dir() -> PathBuf {
|
|
||||||
// Check env var first
|
|
||||||
if let Ok(dir) = std::env::var("VAPORA_SCHEMA_DIR") {
|
|
||||||
let path = PathBuf::from(&dir);
|
|
||||||
if path.is_absolute() {
|
|
||||||
return path;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Try to find workspace root by traversing up from CARGO_MANIFEST_DIR
|
|
||||||
if let Ok(manifest_dir) = std::env::var("CARGO_MANIFEST_DIR") {
|
|
||||||
let mut current = PathBuf::from(&manifest_dir);
|
|
||||||
|
|
||||||
// Traverse up looking for workspace root (contains schemas/ directory)
|
|
||||||
for _ in 0..5 {
|
|
||||||
let schemas_path = current.join("schemas");
|
|
||||||
if schemas_path.is_dir() {
|
|
||||||
return schemas_path;
|
|
||||||
}
|
|
||||||
if !current.pop() {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Try current working directory
|
|
||||||
let cwd_schemas = std::env::current_dir()
|
|
||||||
.map(|p| p.join("schemas"))
|
|
||||||
.unwrap_or_else(|_| PathBuf::from("schemas"));
|
|
||||||
|
|
||||||
if cwd_schemas.is_dir() {
|
|
||||||
return cwd_schemas;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fallback to relative path (will fail gracefully if not found)
|
|
||||||
PathBuf::from("schemas")
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Default for AgentCoordinator {
|
impl Default for AgentCoordinator {
|
||||||
fn default() -> Self {
|
fn default() -> Self {
|
||||||
Self::with_registry(Arc::new(AgentRegistry::default()))
|
Self::with_registry(Arc::new(AgentRegistry::default()))
|
||||||
@ -663,9 +563,6 @@ mod tests {
|
|||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn test_coordinator_creation() {
|
async fn test_coordinator_creation() {
|
||||||
// Set schema directory for tests (relative to workspace root)
|
|
||||||
std::env::set_var("VAPORA_SCHEMA_DIR", "../../schemas");
|
|
||||||
|
|
||||||
let registry = Arc::new(AgentRegistry::new(5));
|
let registry = Arc::new(AgentRegistry::new(5));
|
||||||
let coordinator = AgentCoordinator::with_registry(registry);
|
let coordinator = AgentCoordinator::with_registry(registry);
|
||||||
|
|
||||||
@ -674,9 +571,6 @@ mod tests {
|
|||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn test_task_assignment() {
|
async fn test_task_assignment() {
|
||||||
// Set schema directory for tests (relative to workspace root)
|
|
||||||
std::env::set_var("VAPORA_SCHEMA_DIR", "../../schemas");
|
|
||||||
|
|
||||||
let registry = Arc::new(AgentRegistry::new(5));
|
let registry = Arc::new(AgentRegistry::new(5));
|
||||||
|
|
||||||
// Register an agent
|
// Register an agent
|
||||||
@ -701,18 +595,11 @@ mod tests {
|
|||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
|
|
||||||
match &task_id {
|
|
||||||
Ok(_) => {}
|
|
||||||
Err(e) => eprintln!("Task assignment failed: {}", e),
|
|
||||||
}
|
|
||||||
assert!(task_id.is_ok());
|
assert!(task_id.is_ok());
|
||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn test_no_available_agent() {
|
async fn test_no_available_agent() {
|
||||||
// Set schema directory for tests (relative to workspace root)
|
|
||||||
std::env::set_var("VAPORA_SCHEMA_DIR", "../../schemas");
|
|
||||||
|
|
||||||
let registry = Arc::new(AgentRegistry::new(5));
|
let registry = Arc::new(AgentRegistry::new(5));
|
||||||
let coordinator = AgentCoordinator::with_registry(registry);
|
let coordinator = AgentCoordinator::with_registry(registry);
|
||||||
|
|
||||||
|
|||||||
@ -24,7 +24,6 @@ vapora-llm-router = { workspace = true }
|
|||||||
vapora-swarm = { workspace = true }
|
vapora-swarm = { workspace = true }
|
||||||
vapora-tracking = { path = "../vapora-tracking" }
|
vapora-tracking = { path = "../vapora-tracking" }
|
||||||
vapora-knowledge-graph = { path = "../vapora-knowledge-graph" }
|
vapora-knowledge-graph = { path = "../vapora-knowledge-graph" }
|
||||||
vapora-workflow-engine = { workspace = true }
|
|
||||||
|
|
||||||
# Secrets management
|
# Secrets management
|
||||||
secretumvault = { workspace = true }
|
secretumvault = { workspace = true }
|
||||||
@ -83,7 +82,8 @@ clap = { workspace = true }
|
|||||||
prometheus = { workspace = true }
|
prometheus = { workspace = true }
|
||||||
lazy_static = "1.4"
|
lazy_static = "1.4"
|
||||||
|
|
||||||
# TLS (native tokio-rustls)
|
# TLS
|
||||||
|
axum-server = { workspace = true }
|
||||||
rustls = { workspace = true }
|
rustls = { workspace = true }
|
||||||
rustls-pemfile = { workspace = true }
|
rustls-pemfile = { workspace = true }
|
||||||
|
|
||||||
|
|||||||
@ -54,10 +54,6 @@ pub fn error_response(error: VaporaError) -> Response {
|
|||||||
format!("TOML error: {}", msg),
|
format!("TOML error: {}", msg),
|
||||||
),
|
),
|
||||||
VaporaError::InternalError(msg) => (StatusCode::INTERNAL_SERVER_ERROR, msg),
|
VaporaError::InternalError(msg) => (StatusCode::INTERNAL_SERVER_ERROR, msg),
|
||||||
VaporaError::ValidationError(msg) => (
|
|
||||||
StatusCode::BAD_REQUEST,
|
|
||||||
format!("Validation error: {}", msg),
|
|
||||||
),
|
|
||||||
};
|
};
|
||||||
|
|
||||||
let body = Json(json!({
|
let body = Json(json!({
|
||||||
|
|||||||
@ -15,7 +15,6 @@ pub mod swarm;
|
|||||||
pub mod tasks;
|
pub mod tasks;
|
||||||
pub mod tracking;
|
pub mod tracking;
|
||||||
pub mod websocket;
|
pub mod websocket;
|
||||||
pub mod workflow_orchestrator;
|
|
||||||
// pub mod workflows; // TODO: Phase 4 - Re-enable when workflow module imports
|
// pub mod workflows; // TODO: Phase 4 - Re-enable when workflow module imports
|
||||||
// are fixed
|
// are fixed
|
||||||
|
|
||||||
|
|||||||
@ -2,8 +2,6 @@
|
|||||||
|
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use vapora_workflow_engine::WorkflowOrchestrator;
|
|
||||||
|
|
||||||
use crate::services::{AgentService, ProjectService, ProviderAnalyticsService, TaskService};
|
use crate::services::{AgentService, ProjectService, ProviderAnalyticsService, TaskService};
|
||||||
|
|
||||||
/// Application state shared across all API handlers
|
/// Application state shared across all API handlers
|
||||||
@ -13,7 +11,7 @@ pub struct AppState {
|
|||||||
pub task_service: Arc<TaskService>,
|
pub task_service: Arc<TaskService>,
|
||||||
pub agent_service: Arc<AgentService>,
|
pub agent_service: Arc<AgentService>,
|
||||||
pub provider_analytics_service: Arc<ProviderAnalyticsService>,
|
pub provider_analytics_service: Arc<ProviderAnalyticsService>,
|
||||||
pub workflow_orchestrator: Option<Arc<WorkflowOrchestrator>>,
|
// TODO: Phase 4 - Add workflow_service when workflow module is ready
|
||||||
}
|
}
|
||||||
|
|
||||||
impl AppState {
|
impl AppState {
|
||||||
@ -29,13 +27,6 @@ impl AppState {
|
|||||||
task_service: Arc::new(task_service),
|
task_service: Arc::new(task_service),
|
||||||
agent_service: Arc::new(agent_service),
|
agent_service: Arc::new(agent_service),
|
||||||
provider_analytics_service: Arc::new(provider_analytics_service),
|
provider_analytics_service: Arc::new(provider_analytics_service),
|
||||||
workflow_orchestrator: None,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Add workflow orchestrator to state
|
|
||||||
pub fn with_workflow_orchestrator(mut self, orchestrator: Arc<WorkflowOrchestrator>) -> Self {
|
|
||||||
self.workflow_orchestrator = Some(orchestrator);
|
|
||||||
self
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,286 +0,0 @@
|
|||||||
// Workflow Orchestrator API endpoints
|
|
||||||
// Provides REST API for multi-stage workflow orchestration
|
|
||||||
|
|
||||||
use axum::{
|
|
||||||
extract::{Path, State},
|
|
||||||
http::StatusCode,
|
|
||||||
routing::{get, post},
|
|
||||||
Json, Router,
|
|
||||||
};
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use tracing::{error, info};
|
|
||||||
use vapora_shared::VaporaError;
|
|
||||||
use vapora_workflow_engine::{WorkflowInstance, WorkflowStatus};
|
|
||||||
|
|
||||||
use crate::api::error::ApiError;
|
|
||||||
use crate::api::state::AppState;
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct StartWorkflowRequest {
|
|
||||||
pub template: String,
|
|
||||||
#[serde(default)]
|
|
||||||
pub context: serde_json::Value,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct StartWorkflowResponse {
|
|
||||||
pub workflow_id: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct ApproveStageRequest {
|
|
||||||
pub approver: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct CancelWorkflowRequest {
|
|
||||||
pub reason: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct WorkflowInstanceResponse {
|
|
||||||
pub id: String,
|
|
||||||
pub template_name: String,
|
|
||||||
pub status: String,
|
|
||||||
pub current_stage: usize,
|
|
||||||
pub total_stages: usize,
|
|
||||||
pub created_at: String,
|
|
||||||
pub updated_at: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct WorkflowListResponse {
|
|
||||||
pub workflows: Vec<WorkflowInstanceResponse>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct TemplatesResponse {
|
|
||||||
pub templates: Vec<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct MessageResponse {
|
|
||||||
pub success: bool,
|
|
||||||
pub message: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl From<WorkflowInstance> for WorkflowInstanceResponse {
|
|
||||||
fn from(instance: WorkflowInstance) -> Self {
|
|
||||||
Self {
|
|
||||||
id: instance.id,
|
|
||||||
template_name: instance.template_name,
|
|
||||||
status: status_to_string(&instance.status),
|
|
||||||
current_stage: instance.current_stage_idx,
|
|
||||||
total_stages: instance.stages.len(),
|
|
||||||
created_at: instance.created_at.to_rfc3339(),
|
|
||||||
updated_at: instance.updated_at.to_rfc3339(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn status_to_string(status: &WorkflowStatus) -> String {
|
|
||||||
match status {
|
|
||||||
WorkflowStatus::Running => "running".to_string(),
|
|
||||||
WorkflowStatus::WaitingApproval(idx) => format!("waiting_approval:{}", idx),
|
|
||||||
WorkflowStatus::Completed => "completed".to_string(),
|
|
||||||
WorkflowStatus::Failed(err) => format!("failed:{}", err),
|
|
||||||
WorkflowStatus::Cancelled => "cancelled".to_string(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn orchestrator_routes() -> Router<AppState> {
|
|
||||||
Router::new()
|
|
||||||
.route("/", post(start_workflow))
|
|
||||||
.route("/", get(list_workflows))
|
|
||||||
.route("/:id", get(get_workflow))
|
|
||||||
.route("/:id/approve", post(approve_stage))
|
|
||||||
.route("/:id/cancel", post(cancel_workflow))
|
|
||||||
.route("/templates", get(list_templates))
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn start_workflow(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
Json(req): Json<StartWorkflowRequest>,
|
|
||||||
) -> Result<(StatusCode, Json<StartWorkflowResponse>), ApiError> {
|
|
||||||
let orchestrator = state.workflow_orchestrator.as_ref().ok_or_else(|| {
|
|
||||||
ApiError(VaporaError::InternalError(
|
|
||||||
"Workflow orchestrator not available".to_string(),
|
|
||||||
))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let workflow_id = orchestrator
|
|
||||||
.start_workflow(&req.template, req.context)
|
|
||||||
.await
|
|
||||||
.map_err(|e| {
|
|
||||||
error!("Failed to start workflow: {}", e);
|
|
||||||
ApiError(VaporaError::InternalError(e.to_string()))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
template = %req.template,
|
|
||||||
"Workflow started via API"
|
|
||||||
);
|
|
||||||
|
|
||||||
Ok((
|
|
||||||
StatusCode::CREATED,
|
|
||||||
Json(StartWorkflowResponse { workflow_id }),
|
|
||||||
))
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn list_workflows(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
) -> Result<Json<WorkflowListResponse>, ApiError> {
|
|
||||||
let orchestrator = state.workflow_orchestrator.as_ref().ok_or_else(|| {
|
|
||||||
ApiError(VaporaError::InternalError(
|
|
||||||
"Workflow orchestrator not available".to_string(),
|
|
||||||
))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let workflows = orchestrator
|
|
||||||
.list_workflows()
|
|
||||||
.into_iter()
|
|
||||||
.map(WorkflowInstanceResponse::from)
|
|
||||||
.collect();
|
|
||||||
|
|
||||||
Ok(Json(WorkflowListResponse { workflows }))
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn get_workflow(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
Path(id): Path<String>,
|
|
||||||
) -> Result<Json<WorkflowInstanceResponse>, ApiError> {
|
|
||||||
let orchestrator = state.workflow_orchestrator.as_ref().ok_or_else(|| {
|
|
||||||
ApiError(VaporaError::InternalError(
|
|
||||||
"Workflow orchestrator not available".to_string(),
|
|
||||||
))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let workflow = orchestrator.get_workflow(&id).map_err(|e| {
|
|
||||||
error!("Failed to get workflow {}: {}", id, e);
|
|
||||||
ApiError(VaporaError::NotFound(format!("Workflow {} not found", id)))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
Ok(Json(WorkflowInstanceResponse::from(workflow)))
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn approve_stage(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
Path(id): Path<String>,
|
|
||||||
Json(req): Json<ApproveStageRequest>,
|
|
||||||
) -> Result<Json<MessageResponse>, ApiError> {
|
|
||||||
let orchestrator = state.workflow_orchestrator.as_ref().ok_or_else(|| {
|
|
||||||
ApiError(VaporaError::InternalError(
|
|
||||||
"Workflow orchestrator not available".to_string(),
|
|
||||||
))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
orchestrator
|
|
||||||
.approve_stage(&id, &req.approver)
|
|
||||||
.await
|
|
||||||
.map_err(|e| {
|
|
||||||
error!("Failed to approve workflow {}: {}", id, e);
|
|
||||||
ApiError(VaporaError::InternalError(e.to_string()))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_id = %id,
|
|
||||||
approver = %req.approver,
|
|
||||||
"Workflow stage approved via API"
|
|
||||||
);
|
|
||||||
|
|
||||||
Ok(Json(MessageResponse {
|
|
||||||
success: true,
|
|
||||||
message: format!("Workflow {} stage approved", id),
|
|
||||||
}))
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn cancel_workflow(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
Path(id): Path<String>,
|
|
||||||
Json(req): Json<CancelWorkflowRequest>,
|
|
||||||
) -> Result<Json<MessageResponse>, ApiError> {
|
|
||||||
let orchestrator = state.workflow_orchestrator.as_ref().ok_or_else(|| {
|
|
||||||
ApiError(VaporaError::InternalError(
|
|
||||||
"Workflow orchestrator not available".to_string(),
|
|
||||||
))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
orchestrator
|
|
||||||
.cancel_workflow(&id, req.reason.clone())
|
|
||||||
.await
|
|
||||||
.map_err(|e| {
|
|
||||||
error!("Failed to cancel workflow {}: {}", id, e);
|
|
||||||
ApiError(VaporaError::InternalError(e.to_string()))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_id = %id,
|
|
||||||
reason = %req.reason,
|
|
||||||
"Workflow cancelled via API"
|
|
||||||
);
|
|
||||||
|
|
||||||
Ok(Json(MessageResponse {
|
|
||||||
success: true,
|
|
||||||
message: format!("Workflow {} cancelled", id),
|
|
||||||
}))
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn list_templates(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
) -> Result<Json<TemplatesResponse>, ApiError> {
|
|
||||||
let orchestrator = state.workflow_orchestrator.as_ref().ok_or_else(|| {
|
|
||||||
ApiError(VaporaError::InternalError(
|
|
||||||
"Workflow orchestrator not available".to_string(),
|
|
||||||
))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let templates = orchestrator.list_templates();
|
|
||||||
|
|
||||||
Ok(Json(TemplatesResponse { templates }))
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_start_workflow_request_serialization() {
|
|
||||||
let req = StartWorkflowRequest {
|
|
||||||
template: "feature_development".to_string(),
|
|
||||||
context: serde_json::json!({
|
|
||||||
"task": "Add authentication"
|
|
||||||
}),
|
|
||||||
};
|
|
||||||
|
|
||||||
let json = serde_json::to_string(&req).unwrap();
|
|
||||||
let deserialized: StartWorkflowRequest = serde_json::from_str(&json).unwrap();
|
|
||||||
|
|
||||||
assert_eq!(deserialized.template, "feature_development");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_workflow_instance_response_conversion() {
|
|
||||||
use chrono::Utc;
|
|
||||||
use vapora_workflow_engine::config::{StageConfig, WorkflowConfig};
|
|
||||||
|
|
||||||
let config = WorkflowConfig {
|
|
||||||
name: "test".to_string(),
|
|
||||||
trigger: "manual".to_string(),
|
|
||||||
stages: vec![StageConfig {
|
|
||||||
name: "stage1".to_string(),
|
|
||||||
agents: vec!["agent1".to_string()],
|
|
||||||
parallel: false,
|
|
||||||
max_parallel: None,
|
|
||||||
approval_required: false,
|
|
||||||
}],
|
|
||||||
};
|
|
||||||
|
|
||||||
let instance = WorkflowInstance::new(&config, serde_json::json!({}));
|
|
||||||
let response = WorkflowInstanceResponse::from(instance);
|
|
||||||
|
|
||||||
assert_eq!(response.template_name, "test");
|
|
||||||
assert_eq!(response.status, "running");
|
|
||||||
assert_eq!(response.total_stages, 1);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,42 +0,0 @@
|
|||||||
[package]
|
|
||||||
name = "vapora-cli"
|
|
||||||
version.workspace = true
|
|
||||||
edition.workspace = true
|
|
||||||
rust-version.workspace = true
|
|
||||||
authors.workspace = true
|
|
||||||
license.workspace = true
|
|
||||||
repository.workspace = true
|
|
||||||
homepage.workspace = true
|
|
||||||
keywords.workspace = true
|
|
||||||
categories.workspace = true
|
|
||||||
|
|
||||||
[[bin]]
|
|
||||||
name = "vapora"
|
|
||||||
path = "src/main.rs"
|
|
||||||
|
|
||||||
[dependencies]
|
|
||||||
vapora-shared = { workspace = true }
|
|
||||||
|
|
||||||
# CLI framework
|
|
||||||
clap = { workspace = true, features = ["derive", "env"] }
|
|
||||||
|
|
||||||
# HTTP client
|
|
||||||
reqwest = { workspace = true, features = ["json"] }
|
|
||||||
|
|
||||||
# Async runtime
|
|
||||||
tokio = { workspace = true, features = ["full"] }
|
|
||||||
|
|
||||||
# Serialization
|
|
||||||
serde = { workspace = true, features = ["derive"] }
|
|
||||||
serde_json = { workspace = true }
|
|
||||||
|
|
||||||
# Error handling
|
|
||||||
anyhow = { workspace = true }
|
|
||||||
thiserror = { workspace = true }
|
|
||||||
|
|
||||||
# Utilities
|
|
||||||
chrono = { workspace = true }
|
|
||||||
|
|
||||||
# Terminal UI
|
|
||||||
colored = "2.1"
|
|
||||||
comfy-table = "7.1"
|
|
||||||
@ -1,230 +0,0 @@
|
|||||||
use reqwest::Client;
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use serde_json::Value;
|
|
||||||
|
|
||||||
use crate::error::{CliError, Result};
|
|
||||||
|
|
||||||
#[derive(Clone)]
|
|
||||||
pub struct VaporaClient {
|
|
||||||
base_url: String,
|
|
||||||
client: Client,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct StartWorkflowRequest {
|
|
||||||
pub template: String,
|
|
||||||
pub context: Value,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct StartWorkflowResponse {
|
|
||||||
pub workflow_id: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct ApproveStageRequest {
|
|
||||||
pub approver: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct CancelWorkflowRequest {
|
|
||||||
pub reason: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct WorkflowInstanceResponse {
|
|
||||||
pub id: String,
|
|
||||||
pub template_name: String,
|
|
||||||
pub status: String,
|
|
||||||
pub current_stage: usize,
|
|
||||||
pub total_stages: usize,
|
|
||||||
pub created_at: String,
|
|
||||||
pub updated_at: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct WorkflowListResponse {
|
|
||||||
pub workflows: Vec<WorkflowInstanceResponse>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct TemplatesResponse {
|
|
||||||
pub templates: Vec<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct MessageResponse {
|
|
||||||
pub success: bool,
|
|
||||||
pub message: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl VaporaClient {
|
|
||||||
pub fn new(base_url: &str) -> Self {
|
|
||||||
Self {
|
|
||||||
base_url: base_url.trim_end_matches('/').to_string(),
|
|
||||||
client: Client::new(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn start_workflow(&self, template: String, context: Value) -> Result<String> {
|
|
||||||
let url = format!("{}/api/v1/workflow_orchestrator", self.base_url);
|
|
||||||
|
|
||||||
let request = StartWorkflowRequest { template, context };
|
|
||||||
|
|
||||||
let response = self
|
|
||||||
.client
|
|
||||||
.post(&url)
|
|
||||||
.json(&request)
|
|
||||||
.send()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::ApiError(e.to_string()))?;
|
|
||||||
|
|
||||||
if !response.status().is_success() {
|
|
||||||
let status = response.status();
|
|
||||||
let error_text = response
|
|
||||||
.text()
|
|
||||||
.await
|
|
||||||
.unwrap_or_else(|_| "Unknown error".to_string());
|
|
||||||
return Err(CliError::ApiError(format!(
|
|
||||||
"HTTP {}: {}",
|
|
||||||
status, error_text
|
|
||||||
)));
|
|
||||||
}
|
|
||||||
|
|
||||||
let result: StartWorkflowResponse = response
|
|
||||||
.json()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::InvalidResponse(e.to_string()))?;
|
|
||||||
|
|
||||||
Ok(result.workflow_id)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn list_workflows(&self) -> Result<Vec<WorkflowInstanceResponse>> {
|
|
||||||
let url = format!("{}/api/v1/workflow_orchestrator", self.base_url);
|
|
||||||
|
|
||||||
let response = self
|
|
||||||
.client
|
|
||||||
.get(&url)
|
|
||||||
.send()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::ApiError(e.to_string()))?;
|
|
||||||
|
|
||||||
if !response.status().is_success() {
|
|
||||||
return Err(CliError::ApiError(format!("HTTP {}", response.status())));
|
|
||||||
}
|
|
||||||
|
|
||||||
let result: WorkflowListResponse = response
|
|
||||||
.json()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::InvalidResponse(e.to_string()))?;
|
|
||||||
|
|
||||||
Ok(result.workflows)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn get_workflow(&self, workflow_id: &str) -> Result<WorkflowInstanceResponse> {
|
|
||||||
let url = format!(
|
|
||||||
"{}/api/v1/workflow_orchestrator/{}",
|
|
||||||
self.base_url, workflow_id
|
|
||||||
);
|
|
||||||
|
|
||||||
let response = self
|
|
||||||
.client
|
|
||||||
.get(&url)
|
|
||||||
.send()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::ApiError(e.to_string()))?;
|
|
||||||
|
|
||||||
if response.status().as_u16() == 404 {
|
|
||||||
return Err(CliError::NotFound(workflow_id.to_string()));
|
|
||||||
}
|
|
||||||
|
|
||||||
if !response.status().is_success() {
|
|
||||||
return Err(CliError::ApiError(format!("HTTP {}", response.status())));
|
|
||||||
}
|
|
||||||
|
|
||||||
let result: WorkflowInstanceResponse = response
|
|
||||||
.json()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::InvalidResponse(e.to_string()))?;
|
|
||||||
|
|
||||||
Ok(result)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn approve_stage(&self, workflow_id: &str, approver: String) -> Result<String> {
|
|
||||||
let url = format!(
|
|
||||||
"{}/api/v1/workflow_orchestrator/{}/approve",
|
|
||||||
self.base_url, workflow_id
|
|
||||||
);
|
|
||||||
|
|
||||||
let request = ApproveStageRequest { approver };
|
|
||||||
|
|
||||||
let response = self
|
|
||||||
.client
|
|
||||||
.post(&url)
|
|
||||||
.json(&request)
|
|
||||||
.send()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::ApiError(e.to_string()))?;
|
|
||||||
|
|
||||||
if !response.status().is_success() {
|
|
||||||
return Err(CliError::ApiError(format!("HTTP {}", response.status())));
|
|
||||||
}
|
|
||||||
|
|
||||||
let result: MessageResponse = response
|
|
||||||
.json()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::InvalidResponse(e.to_string()))?;
|
|
||||||
|
|
||||||
Ok(result.message)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn cancel_workflow(&self, workflow_id: &str, reason: String) -> Result<String> {
|
|
||||||
let url = format!(
|
|
||||||
"{}/api/v1/workflow_orchestrator/{}/cancel",
|
|
||||||
self.base_url, workflow_id
|
|
||||||
);
|
|
||||||
|
|
||||||
let request = CancelWorkflowRequest { reason };
|
|
||||||
|
|
||||||
let response = self
|
|
||||||
.client
|
|
||||||
.post(&url)
|
|
||||||
.json(&request)
|
|
||||||
.send()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::ApiError(e.to_string()))?;
|
|
||||||
|
|
||||||
if !response.status().is_success() {
|
|
||||||
return Err(CliError::ApiError(format!("HTTP {}", response.status())));
|
|
||||||
}
|
|
||||||
|
|
||||||
let result: MessageResponse = response
|
|
||||||
.json()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::InvalidResponse(e.to_string()))?;
|
|
||||||
|
|
||||||
Ok(result.message)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn list_templates(&self) -> Result<Vec<String>> {
|
|
||||||
let url = format!("{}/api/v1/workflow_orchestrator/templates", self.base_url);
|
|
||||||
|
|
||||||
let response = self
|
|
||||||
.client
|
|
||||||
.get(&url)
|
|
||||||
.send()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::ApiError(e.to_string()))?;
|
|
||||||
|
|
||||||
if !response.status().is_success() {
|
|
||||||
return Err(CliError::ApiError(format!("HTTP {}", response.status())));
|
|
||||||
}
|
|
||||||
|
|
||||||
let result: TemplatesResponse = response
|
|
||||||
.json()
|
|
||||||
.await
|
|
||||||
.map_err(|e| CliError::InvalidResponse(e.to_string()))?;
|
|
||||||
|
|
||||||
Ok(result.templates)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,149 +0,0 @@
|
|||||||
use std::path::PathBuf;
|
|
||||||
|
|
||||||
use anyhow::Result;
|
|
||||||
use clap::Subcommand;
|
|
||||||
|
|
||||||
use crate::client::VaporaClient;
|
|
||||||
use crate::output;
|
|
||||||
|
|
||||||
#[derive(Subcommand)]
|
|
||||||
pub enum Commands {
|
|
||||||
/// Workflow orchestration commands
|
|
||||||
#[command(subcommand)]
|
|
||||||
Workflow(WorkflowCommands),
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Commands {
|
|
||||||
pub async fn execute(self, client: VaporaClient) -> Result<()> {
|
|
||||||
match self {
|
|
||||||
Commands::Workflow(cmd) => cmd.execute(client).await,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Subcommand)]
|
|
||||||
pub enum WorkflowCommands {
|
|
||||||
/// Start a new workflow from template
|
|
||||||
Start {
|
|
||||||
/// Workflow template name
|
|
||||||
#[arg(short, long)]
|
|
||||||
template: String,
|
|
||||||
|
|
||||||
/// Initial context JSON file (optional)
|
|
||||||
#[arg(short, long)]
|
|
||||||
context: Option<PathBuf>,
|
|
||||||
|
|
||||||
/// Enrich with Kogral knowledge (default: true)
|
|
||||||
#[arg(long, default_value_t = true)]
|
|
||||||
kogral: bool,
|
|
||||||
},
|
|
||||||
|
|
||||||
/// List active workflows
|
|
||||||
List,
|
|
||||||
|
|
||||||
/// Get workflow status
|
|
||||||
Status {
|
|
||||||
/// Workflow ID
|
|
||||||
workflow_id: String,
|
|
||||||
},
|
|
||||||
|
|
||||||
/// Approve a stage waiting for approval
|
|
||||||
Approve {
|
|
||||||
/// Workflow ID
|
|
||||||
workflow_id: String,
|
|
||||||
|
|
||||||
/// Approver name
|
|
||||||
#[arg(short, long)]
|
|
||||||
approver: String,
|
|
||||||
},
|
|
||||||
|
|
||||||
/// Cancel a running workflow
|
|
||||||
Cancel {
|
|
||||||
/// Workflow ID
|
|
||||||
workflow_id: String,
|
|
||||||
|
|
||||||
/// Reason for cancellation
|
|
||||||
#[arg(short, long)]
|
|
||||||
reason: String,
|
|
||||||
},
|
|
||||||
|
|
||||||
/// List available workflow templates
|
|
||||||
Templates,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl WorkflowCommands {
|
|
||||||
pub async fn execute(self, client: VaporaClient) -> Result<()> {
|
|
||||||
match self {
|
|
||||||
WorkflowCommands::Start {
|
|
||||||
template,
|
|
||||||
context,
|
|
||||||
kogral,
|
|
||||||
} => {
|
|
||||||
let context_value = if let Some(path) = context {
|
|
||||||
let content = tokio::fs::read_to_string(path).await?;
|
|
||||||
serde_json::from_str(&content)?
|
|
||||||
} else {
|
|
||||||
serde_json::json!({})
|
|
||||||
};
|
|
||||||
|
|
||||||
let mut ctx = context_value;
|
|
||||||
|
|
||||||
if kogral {
|
|
||||||
if let Some(obj) = ctx.as_object_mut() {
|
|
||||||
obj.insert("enable_kogral".to_string(), serde_json::json!(true));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
let workflow_id = client.start_workflow(template.clone(), ctx).await?;
|
|
||||||
|
|
||||||
output::print_success(&format!(
|
|
||||||
"Workflow started: {} (ID: {})",
|
|
||||||
template, workflow_id
|
|
||||||
));
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
WorkflowCommands::List => {
|
|
||||||
let workflows = client.list_workflows().await?;
|
|
||||||
|
|
||||||
output::print_workflows_table(&workflows);
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
WorkflowCommands::Status { workflow_id } => {
|
|
||||||
let workflow = client.get_workflow(&workflow_id).await?;
|
|
||||||
|
|
||||||
output::print_workflow_details(&workflow);
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
WorkflowCommands::Approve {
|
|
||||||
workflow_id,
|
|
||||||
approver,
|
|
||||||
} => {
|
|
||||||
let message = client.approve_stage(&workflow_id, approver).await?;
|
|
||||||
|
|
||||||
output::print_success(&message);
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
WorkflowCommands::Cancel {
|
|
||||||
workflow_id,
|
|
||||||
reason,
|
|
||||||
} => {
|
|
||||||
let message = client.cancel_workflow(&workflow_id, reason).await?;
|
|
||||||
|
|
||||||
output::print_success(&message);
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
WorkflowCommands::Templates => {
|
|
||||||
let templates = client.list_templates().await?;
|
|
||||||
|
|
||||||
output::print_templates(&templates);
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,24 +0,0 @@
|
|||||||
use thiserror::Error;
|
|
||||||
|
|
||||||
#[derive(Error, Debug)]
|
|
||||||
pub enum CliError {
|
|
||||||
#[error("API request failed: {0}")]
|
|
||||||
ApiError(String),
|
|
||||||
|
|
||||||
#[error("Invalid response: {0}")]
|
|
||||||
InvalidResponse(String),
|
|
||||||
|
|
||||||
#[error("Workflow not found: {0}")]
|
|
||||||
NotFound(String),
|
|
||||||
|
|
||||||
#[error("IO error: {0}")]
|
|
||||||
IoError(#[from] std::io::Error),
|
|
||||||
|
|
||||||
#[error("JSON error: {0}")]
|
|
||||||
JsonError(#[from] serde_json::Error),
|
|
||||||
|
|
||||||
#[error("HTTP error: {0}")]
|
|
||||||
HttpError(#[from] reqwest::Error),
|
|
||||||
}
|
|
||||||
|
|
||||||
pub type Result<T> = std::result::Result<T, CliError>;
|
|
||||||
@ -1,31 +0,0 @@
|
|||||||
use anyhow::Result;
|
|
||||||
use clap::Parser;
|
|
||||||
|
|
||||||
mod client;
|
|
||||||
mod commands;
|
|
||||||
mod error;
|
|
||||||
mod output;
|
|
||||||
|
|
||||||
use commands::Commands;
|
|
||||||
|
|
||||||
#[derive(Parser)]
|
|
||||||
#[command(name = "vapora")]
|
|
||||||
#[command(about = "VAPORA CLI - Intelligent Development Orchestration", long_about = None)]
|
|
||||||
#[command(version)]
|
|
||||||
struct Cli {
|
|
||||||
/// Backend API URL
|
|
||||||
#[arg(long, env = "VAPORA_API_URL", default_value = "http://localhost:8001")]
|
|
||||||
api_url: String,
|
|
||||||
|
|
||||||
#[command(subcommand)]
|
|
||||||
command: Commands,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::main]
|
|
||||||
async fn main() -> Result<()> {
|
|
||||||
let cli = Cli::parse();
|
|
||||||
|
|
||||||
let client = client::VaporaClient::new(&cli.api_url);
|
|
||||||
|
|
||||||
cli.command.execute(client).await
|
|
||||||
}
|
|
||||||
@ -1,100 +0,0 @@
|
|||||||
use colored::Colorize;
|
|
||||||
use comfy_table::{presets::UTF8_FULL, Cell, Color, ContentArrangement, Table};
|
|
||||||
|
|
||||||
use crate::client::WorkflowInstanceResponse;
|
|
||||||
|
|
||||||
pub fn print_success(message: &str) {
|
|
||||||
println!("{} {}", "✓".green().bold(), message.green());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[allow(dead_code)]
|
|
||||||
pub fn print_error(message: &str) {
|
|
||||||
eprintln!("{} {}", "✗".red().bold(), message.red());
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn print_workflows_table(workflows: &[WorkflowInstanceResponse]) {
|
|
||||||
if workflows.is_empty() {
|
|
||||||
println!("{}", "No active workflows".yellow());
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut table = Table::new();
|
|
||||||
table
|
|
||||||
.load_preset(UTF8_FULL)
|
|
||||||
.set_content_arrangement(ContentArrangement::Dynamic)
|
|
||||||
.set_header(vec![
|
|
||||||
Cell::new("ID").fg(Color::Cyan),
|
|
||||||
Cell::new("Template").fg(Color::Cyan),
|
|
||||||
Cell::new("Status").fg(Color::Cyan),
|
|
||||||
Cell::new("Progress").fg(Color::Cyan),
|
|
||||||
Cell::new("Created").fg(Color::Cyan),
|
|
||||||
]);
|
|
||||||
|
|
||||||
for workflow in workflows {
|
|
||||||
let status_cell = match workflow.status.as_str() {
|
|
||||||
s if s.starts_with("running") => Cell::new(&workflow.status).fg(Color::Green),
|
|
||||||
s if s.starts_with("waiting") => Cell::new(&workflow.status).fg(Color::Yellow),
|
|
||||||
s if s.starts_with("completed") => Cell::new(&workflow.status).fg(Color::Blue),
|
|
||||||
s if s.starts_with("failed") => Cell::new(&workflow.status).fg(Color::Red),
|
|
||||||
_ => Cell::new(&workflow.status),
|
|
||||||
};
|
|
||||||
|
|
||||||
let progress = format!("{}/{}", workflow.current_stage + 1, workflow.total_stages);
|
|
||||||
|
|
||||||
table.add_row(vec![
|
|
||||||
Cell::new(&workflow.id[..8]),
|
|
||||||
Cell::new(&workflow.template_name),
|
|
||||||
status_cell,
|
|
||||||
Cell::new(progress),
|
|
||||||
Cell::new(&workflow.created_at[..19]),
|
|
||||||
]);
|
|
||||||
}
|
|
||||||
|
|
||||||
println!("{table}");
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn print_workflow_details(workflow: &WorkflowInstanceResponse) {
|
|
||||||
println!("\n{}", "Workflow Details".cyan().bold());
|
|
||||||
println!("{}", "─".repeat(60).cyan());
|
|
||||||
println!("{:<15} {}", "ID:".bold(), workflow.id);
|
|
||||||
println!("{:<15} {}", "Template:".bold(), workflow.template_name);
|
|
||||||
|
|
||||||
let status_colored = match workflow.status.as_str() {
|
|
||||||
s if s.starts_with("running") => workflow.status.green(),
|
|
||||||
s if s.starts_with("waiting") => workflow.status.yellow(),
|
|
||||||
s if s.starts_with("completed") => workflow.status.blue(),
|
|
||||||
s if s.starts_with("failed") => workflow.status.red(),
|
|
||||||
_ => workflow.status.normal(),
|
|
||||||
};
|
|
||||||
println!("{:<15} {}", "Status:".bold(), status_colored);
|
|
||||||
|
|
||||||
println!(
|
|
||||||
"{:<15} {}/{}",
|
|
||||||
"Progress:".bold(),
|
|
||||||
workflow.current_stage + 1,
|
|
||||||
workflow.total_stages
|
|
||||||
);
|
|
||||||
println!("{:<15} {}", "Created:".bold(), workflow.created_at);
|
|
||||||
println!("{:<15} {}", "Updated:".bold(), workflow.updated_at);
|
|
||||||
println!("{}", "─".repeat(60).cyan());
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn print_templates(templates: &[String]) {
|
|
||||||
if templates.is_empty() {
|
|
||||||
println!("{}", "No workflow templates available".yellow());
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
println!("\n{}", "Available Workflow Templates".cyan().bold());
|
|
||||||
println!("{}", "─".repeat(60).cyan());
|
|
||||||
|
|
||||||
for (idx, template) in templates.iter().enumerate() {
|
|
||||||
println!("{:2}. {}", idx + 1, template.green());
|
|
||||||
}
|
|
||||||
|
|
||||||
println!("{}", "─".repeat(60).cyan());
|
|
||||||
println!(
|
|
||||||
"\nUse {} to start a workflow",
|
|
||||||
"vapora workflow start --template <name>".yellow()
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@ -181,16 +181,14 @@ impl ReasoningEngine {
|
|||||||
.iter()
|
.iter()
|
||||||
.filter(|r| !visited.contains(&r.id) && r.solution.is_some())
|
.filter(|r| !visited.contains(&r.id) && r.solution.is_some())
|
||||||
{
|
{
|
||||||
let Some(ref root) = other.root_cause else {
|
if let Some(ref root) = other.root_cause {
|
||||||
continue;
|
|
||||||
};
|
|
||||||
|
|
||||||
if root.contains(chain.last().unwrap()) {
|
if root.contains(chain.last().unwrap()) {
|
||||||
chain.push(other.solution.clone().unwrap());
|
chain.push(other.solution.clone().unwrap());
|
||||||
visited.insert(other.id.clone());
|
visited.insert(other.id.clone());
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if chain.len() > 1 {
|
if chain.len() > 1 {
|
||||||
chains.push(chain);
|
chains.push(chain);
|
||||||
|
|||||||
@ -116,18 +116,17 @@ impl TemporalKG {
|
|||||||
let record = entry.value();
|
let record = entry.value();
|
||||||
|
|
||||||
if record.timestamp > cutoff && record.task_type == task_type {
|
if record.timestamp > cutoff && record.task_type == task_type {
|
||||||
let similarity = match (
|
let similarity = if let Some(ref query_emb) = query_embedding {
|
||||||
&query_embedding,
|
|
||||||
self.get_or_embed(&record.description).await,
|
|
||||||
) {
|
|
||||||
(Some(query_emb), Ok(Some(record_emb))) => {
|
|
||||||
// Phase 5.1: Use vector embedding similarity
|
// Phase 5.1: Use vector embedding similarity
|
||||||
|
if let Ok(Some(record_emb)) = self.get_or_embed(&record.description).await {
|
||||||
Self::compute_vector_similarity(query_emb, &record_emb)
|
Self::compute_vector_similarity(query_emb, &record_emb)
|
||||||
}
|
} else {
|
||||||
_ => {
|
// Fallback to Jaccard if embedding fails
|
||||||
// Fallback to Jaccard if embedding fails or no embedding provider
|
|
||||||
calculate_similarity(description, &record.description)
|
calculate_similarity(description, &record.description)
|
||||||
}
|
}
|
||||||
|
} else {
|
||||||
|
// Fallback to Jaccard if no embedding provider
|
||||||
|
calculate_similarity(description, &record.description)
|
||||||
};
|
};
|
||||||
|
|
||||||
if similarity >= threshold {
|
if similarity >= threshold {
|
||||||
@ -161,12 +160,14 @@ impl TemporalKG {
|
|||||||
|
|
||||||
for task in similar_tasks {
|
for task in similar_tasks {
|
||||||
if task.success {
|
if task.success {
|
||||||
let confidence =
|
let confidence = if let Some(ref query_emb) = query_embedding {
|
||||||
match (&query_embedding, self.get_or_embed(&task.description).await) {
|
if let Ok(Some(task_emb)) = self.get_or_embed(&task.description).await {
|
||||||
(Some(query_emb), Ok(Some(task_emb))) => {
|
|
||||||
Self::compute_vector_similarity(query_emb, &task_emb)
|
Self::compute_vector_similarity(query_emb, &task_emb)
|
||||||
|
} else {
|
||||||
|
calculate_similarity(description, &task.description)
|
||||||
}
|
}
|
||||||
_ => calculate_similarity(description, &task.description),
|
} else {
|
||||||
|
calculate_similarity(description, &task.description)
|
||||||
};
|
};
|
||||||
|
|
||||||
recommendations.push(Recommendation {
|
recommendations.push(Recommendation {
|
||||||
@ -283,15 +284,15 @@ impl TemporalKG {
|
|||||||
for entry in self.records.iter() {
|
for entry in self.records.iter() {
|
||||||
let record = entry.value();
|
let record = entry.value();
|
||||||
if !record.success {
|
if !record.success {
|
||||||
let Some(error) = &record.error else {
|
if let Some(error) = &record.error {
|
||||||
continue;
|
let similarity = if let Some(ref pattern_emb) = pattern_embedding {
|
||||||
};
|
if let Ok(Some(error_emb)) = self.get_or_embed(error).await {
|
||||||
|
|
||||||
let similarity = match (&pattern_embedding, self.get_or_embed(error).await) {
|
|
||||||
(Some(pattern_emb), Ok(Some(error_emb))) => {
|
|
||||||
Self::compute_vector_similarity(pattern_emb, &error_emb)
|
Self::compute_vector_similarity(pattern_emb, &error_emb)
|
||||||
|
} else {
|
||||||
|
calculate_similarity(cause_pattern, error)
|
||||||
}
|
}
|
||||||
_ => calculate_similarity(cause_pattern, error),
|
} else {
|
||||||
|
calculate_similarity(cause_pattern, error)
|
||||||
};
|
};
|
||||||
|
|
||||||
if similarity >= threshold {
|
if similarity >= threshold {
|
||||||
@ -307,6 +308,7 @@ impl TemporalKG {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Deduplicate and count occurrences
|
// Deduplicate and count occurrences
|
||||||
let mut deduped: std::collections::HashMap<String, CausalRelationship> =
|
let mut deduped: std::collections::HashMap<String, CausalRelationship> =
|
||||||
|
|||||||
@ -14,9 +14,6 @@ crate-type = ["rlib"]
|
|||||||
# Internal crates
|
# Internal crates
|
||||||
vapora-shared = { workspace = true }
|
vapora-shared = { workspace = true }
|
||||||
|
|
||||||
# Embeddings
|
|
||||||
stratum-embeddings = { path = "/Users/Akasha/Development/stratumiops/crates/stratum-embeddings", features = ["vapora"] }
|
|
||||||
|
|
||||||
# Secrets management
|
# Secrets management
|
||||||
secretumvault = { workspace = true }
|
secretumvault = { workspace = true }
|
||||||
|
|
||||||
|
|||||||
@ -1,78 +1,129 @@
|
|||||||
//! Embeddings module using stratum-embeddings
|
// Embedding provider implementations for vector similarity in Knowledge Graph
|
||||||
//!
|
// Phase 5.1: Embedding-based KG similarity
|
||||||
//! Provides unified embedding providers (OpenAI, Ollama, HuggingFace,
|
|
||||||
//! FastEmbed) with caching, batch processing, and fallback support.
|
|
||||||
|
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
// Import the trait so methods are available
|
use serde::{Deserialize, Serialize};
|
||||||
use stratum_embeddings::EmbeddingProvider as StratumProvider;
|
use thiserror::Error;
|
||||||
// Re-export stratum-embeddings types
|
|
||||||
pub use stratum_embeddings::{
|
|
||||||
EmbeddingOptions, HuggingFaceModel, HuggingFaceProvider, OllamaModel, OllamaProvider,
|
|
||||||
OpenAiModel, OpenAiProvider,
|
|
||||||
};
|
|
||||||
use tracing::debug;
|
use tracing::debug;
|
||||||
|
|
||||||
pub type Result<T> = std::result::Result<T, EmbeddingError>;
|
#[derive(Debug, Error)]
|
||||||
|
|
||||||
#[derive(Debug, thiserror::Error)]
|
|
||||||
pub enum EmbeddingError {
|
pub enum EmbeddingError {
|
||||||
#[error("Configuration error: {0}")]
|
#[error("Provider error: {0}")]
|
||||||
ConfigError(String),
|
ProviderError(String),
|
||||||
|
|
||||||
#[error("Request failed: {0}")]
|
|
||||||
RequestFailed(String),
|
|
||||||
|
|
||||||
#[error("Invalid input: {0}")]
|
#[error("Invalid input: {0}")]
|
||||||
InvalidInput(String),
|
InvalidInput(String),
|
||||||
|
|
||||||
#[error("Stratum embeddings error: {0}")]
|
#[error("Request failed: {0}")]
|
||||||
StratumError(#[from] stratum_embeddings::EmbeddingError),
|
RequestFailed(String),
|
||||||
|
|
||||||
|
#[error("Configuration error: {0}")]
|
||||||
|
ConfigError(String),
|
||||||
|
|
||||||
|
#[error("HTTP error: {0}")]
|
||||||
|
HttpError(#[from] reqwest::Error),
|
||||||
|
|
||||||
|
#[error("JSON error: {0}")]
|
||||||
|
JsonError(#[from] serde_json::Error),
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Embedding provider trait (vapora compatibility layer)
|
pub type Result<T> = std::result::Result<T, EmbeddingError>;
|
||||||
|
|
||||||
|
/// Trait for embedding providers - converts text to vector embeddings
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
pub trait EmbeddingProvider: Send + Sync {
|
pub trait EmbeddingProvider: Send + Sync {
|
||||||
|
/// Generate embedding for text (returns 1536-dim vector by default)
|
||||||
async fn embed(&self, text: &str) -> Result<Vec<f32>>;
|
async fn embed(&self, text: &str) -> Result<Vec<f32>>;
|
||||||
|
|
||||||
|
/// Batch embed multiple texts (more efficient for providers)
|
||||||
|
async fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
||||||
|
let mut results = Vec::new();
|
||||||
|
for text in texts {
|
||||||
|
results.push(self.embed(text).await?);
|
||||||
|
}
|
||||||
|
Ok(results)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Provider name for metrics/logging
|
||||||
fn provider_name(&self) -> &str;
|
fn provider_name(&self) -> &str;
|
||||||
|
|
||||||
|
/// Model name being used
|
||||||
fn model_name(&self) -> &str;
|
fn model_name(&self) -> &str;
|
||||||
|
|
||||||
|
/// Embedding dimension (usually 1536)
|
||||||
|
fn embedding_dim(&self) -> usize {
|
||||||
|
1536
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ============================================================================
|
// ============================================================================
|
||||||
// Provider Wrappers (delegate to stratum-embeddings)
|
// Ollama Provider (Local, Free)
|
||||||
// ============================================================================
|
// ============================================================================
|
||||||
|
|
||||||
pub struct OllamaEmbedding {
|
pub struct OllamaEmbedding {
|
||||||
provider: OllamaProvider,
|
endpoint: String,
|
||||||
|
model: String,
|
||||||
|
client: reqwest::Client,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl OllamaEmbedding {
|
impl OllamaEmbedding {
|
||||||
pub fn new(_endpoint: String, model: String) -> Result<Self> {
|
pub fn new(endpoint: String, model: String) -> Self {
|
||||||
// Note: stratum-embeddings OllamaProvider uses default endpoint
|
Self {
|
||||||
// The endpoint parameter is kept for API compatibility but not used
|
endpoint,
|
||||||
let model_enum = if model == "nomic-embed-text" {
|
model,
|
||||||
OllamaModel::NomicEmbed
|
client: reqwest::Client::new(),
|
||||||
} else if model == "mxbai-embed-large" {
|
|
||||||
OllamaModel::MxbaiEmbed
|
|
||||||
} else if model == "all-minilm" {
|
|
||||||
OllamaModel::AllMiniLm
|
|
||||||
} else {
|
|
||||||
// Custom model - default dimensions to 768
|
|
||||||
OllamaModel::Custom(model, 768)
|
|
||||||
};
|
|
||||||
|
|
||||||
let provider = OllamaProvider::new(model_enum)?;
|
|
||||||
Ok(Self { provider })
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Serialize)]
|
||||||
|
struct OllamaEmbedRequest {
|
||||||
|
model: String,
|
||||||
|
prompt: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
struct OllamaEmbedResponse {
|
||||||
|
embedding: Vec<f32>,
|
||||||
|
}
|
||||||
|
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
impl EmbeddingProvider for OllamaEmbedding {
|
impl EmbeddingProvider for OllamaEmbedding {
|
||||||
async fn embed(&self, text: &str) -> Result<Vec<f32>> {
|
async fn embed(&self, text: &str) -> Result<Vec<f32>> {
|
||||||
let options = EmbeddingOptions::default_with_cache();
|
if text.is_empty() {
|
||||||
Ok(self.provider.embed(text, &options).await?)
|
return Err(EmbeddingError::InvalidInput("Empty text".to_string()));
|
||||||
|
}
|
||||||
|
|
||||||
|
debug!("Embedding text via Ollama ({})", self.model);
|
||||||
|
|
||||||
|
let request = OllamaEmbedRequest {
|
||||||
|
model: self.model.clone(),
|
||||||
|
prompt: text.to_string(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let response = self
|
||||||
|
.client
|
||||||
|
.post(format!("{}/api/embeddings", self.endpoint))
|
||||||
|
.json(&request)
|
||||||
|
.send()
|
||||||
|
.await
|
||||||
|
.map_err(|e| EmbeddingError::RequestFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
if !response.status().is_success() {
|
||||||
|
return Err(EmbeddingError::RequestFailed(format!(
|
||||||
|
"Status: {}",
|
||||||
|
response.status()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
|
||||||
|
let data: OllamaEmbedResponse = response
|
||||||
|
.json()
|
||||||
|
.await
|
||||||
|
.map_err(|e| EmbeddingError::RequestFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
Ok(data.embedding)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn provider_name(&self) -> &str {
|
fn provider_name(&self) -> &str {
|
||||||
@ -80,39 +131,93 @@ impl EmbeddingProvider for OllamaEmbedding {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn model_name(&self) -> &str {
|
fn model_name(&self) -> &str {
|
||||||
self.provider.model()
|
&self.model
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// OpenAI Provider (Paid, Fast)
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
pub struct OpenAIEmbedding {
|
pub struct OpenAIEmbedding {
|
||||||
provider: OpenAiProvider,
|
api_key: String,
|
||||||
|
model: String,
|
||||||
|
client: reqwest::Client,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl OpenAIEmbedding {
|
impl OpenAIEmbedding {
|
||||||
pub fn new(api_key: String, model: String) -> Result<Self> {
|
pub fn new(api_key: String, model: String) -> Self {
|
||||||
let model_enum = match model.as_str() {
|
Self {
|
||||||
"text-embedding-3-small" => OpenAiModel::TextEmbedding3Small,
|
api_key,
|
||||||
"text-embedding-3-large" => OpenAiModel::TextEmbedding3Large,
|
model,
|
||||||
"text-embedding-ada-002" => OpenAiModel::TextEmbeddingAda002,
|
client: reqwest::Client::new(),
|
||||||
_ => {
|
}
|
||||||
debug!(
|
}
|
||||||
"Unknown OpenAI model '{}', defaulting to text-embedding-3-small",
|
|
||||||
model
|
|
||||||
);
|
|
||||||
OpenAiModel::TextEmbedding3Small
|
|
||||||
}
|
}
|
||||||
};
|
|
||||||
|
|
||||||
let provider = OpenAiProvider::new(api_key, model_enum)?;
|
#[derive(Debug, Serialize)]
|
||||||
Ok(Self { provider })
|
struct OpenAIEmbedRequest {
|
||||||
|
model: String,
|
||||||
|
input: String,
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
encoding_format: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
struct OpenAIEmbedResponse {
|
||||||
|
data: Vec<OpenAIEmbedData>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
struct OpenAIEmbedData {
|
||||||
|
embedding: Vec<f32>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
impl EmbeddingProvider for OpenAIEmbedding {
|
impl EmbeddingProvider for OpenAIEmbedding {
|
||||||
async fn embed(&self, text: &str) -> Result<Vec<f32>> {
|
async fn embed(&self, text: &str) -> Result<Vec<f32>> {
|
||||||
let options = EmbeddingOptions::default_with_cache();
|
if text.is_empty() {
|
||||||
Ok(self.provider.embed(text, &options).await?)
|
return Err(EmbeddingError::InvalidInput("Empty text".to_string()));
|
||||||
|
}
|
||||||
|
|
||||||
|
debug!("Embedding text via OpenAI ({})", self.model);
|
||||||
|
|
||||||
|
let request = OpenAIEmbedRequest {
|
||||||
|
model: self.model.clone(),
|
||||||
|
input: text.to_string(),
|
||||||
|
encoding_format: None,
|
||||||
|
};
|
||||||
|
|
||||||
|
let response = self
|
||||||
|
.client
|
||||||
|
.post("https://api.openai.com/v1/embeddings")
|
||||||
|
.header("Authorization", format!("Bearer {}", self.api_key))
|
||||||
|
.json(&request)
|
||||||
|
.send()
|
||||||
|
.await
|
||||||
|
.map_err(|e| EmbeddingError::RequestFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
if !response.status().is_success() {
|
||||||
|
let status = response.status();
|
||||||
|
let text = response.text().await.unwrap_or_default();
|
||||||
|
return Err(EmbeddingError::RequestFailed(format!(
|
||||||
|
"OpenAI API error {}: {}",
|
||||||
|
status, text
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
|
||||||
|
let data: OpenAIEmbedResponse = response
|
||||||
|
.json()
|
||||||
|
.await
|
||||||
|
.map_err(|e| EmbeddingError::RequestFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
if data.data.is_empty() {
|
||||||
|
return Err(EmbeddingError::RequestFailed(
|
||||||
|
"No embeddings in response".to_string(),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(data.data[0].embedding.clone())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn provider_name(&self) -> &str {
|
fn provider_name(&self) -> &str {
|
||||||
@ -120,38 +225,84 @@ impl EmbeddingProvider for OpenAIEmbedding {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn model_name(&self) -> &str {
|
fn model_name(&self) -> &str {
|
||||||
self.provider.model()
|
&self.model
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// HuggingFace Provider (Free, Flexible)
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
pub struct HuggingFaceEmbedding {
|
pub struct HuggingFaceEmbedding {
|
||||||
provider: HuggingFaceProvider,
|
api_key: String,
|
||||||
|
model: String,
|
||||||
|
client: reqwest::Client,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl HuggingFaceEmbedding {
|
impl HuggingFaceEmbedding {
|
||||||
pub fn new(api_key: String, model: String) -> Result<Self> {
|
pub fn new(api_key: String, model: String) -> Self {
|
||||||
let model_enum = match model.as_str() {
|
Self {
|
||||||
"BAAI/bge-small-en-v1.5" => HuggingFaceModel::BgeSmall,
|
api_key,
|
||||||
"BAAI/bge-base-en-v1.5" => HuggingFaceModel::BgeBase,
|
model,
|
||||||
"BAAI/bge-large-en-v1.5" => HuggingFaceModel::BgeLarge,
|
client: reqwest::Client::new(),
|
||||||
"sentence-transformers/all-MiniLM-L6-v2" => HuggingFaceModel::AllMiniLm,
|
}
|
||||||
"sentence-transformers/all-mpnet-base-v2" => HuggingFaceModel::AllMpnet,
|
}
|
||||||
_ => {
|
|
||||||
debug!("Using custom HuggingFace model: {}", model);
|
|
||||||
HuggingFaceModel::Custom(model, 384) // Default to 384 dims
|
|
||||||
}
|
}
|
||||||
};
|
|
||||||
|
|
||||||
let provider = HuggingFaceProvider::new(api_key, model_enum)?;
|
#[derive(Debug, Deserialize)]
|
||||||
Ok(Self { provider })
|
#[serde(untagged)]
|
||||||
}
|
enum HFEmbedResponse {
|
||||||
|
Single(Vec<f32>),
|
||||||
|
Multiple(Vec<Vec<f32>>),
|
||||||
}
|
}
|
||||||
|
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
impl EmbeddingProvider for HuggingFaceEmbedding {
|
impl EmbeddingProvider for HuggingFaceEmbedding {
|
||||||
async fn embed(&self, text: &str) -> Result<Vec<f32>> {
|
async fn embed(&self, text: &str) -> Result<Vec<f32>> {
|
||||||
let options = EmbeddingOptions::default_with_cache();
|
if text.is_empty() {
|
||||||
Ok(self.provider.embed(text, &options).await?)
|
return Err(EmbeddingError::InvalidInput("Empty text".to_string()));
|
||||||
|
}
|
||||||
|
|
||||||
|
debug!("Embedding text via HuggingFace ({})", self.model);
|
||||||
|
|
||||||
|
let response = self
|
||||||
|
.client
|
||||||
|
.post(format!(
|
||||||
|
"https://api-inference.huggingface.co/pipeline/feature-extraction/{}",
|
||||||
|
self.model
|
||||||
|
))
|
||||||
|
.header("Authorization", format!("Bearer {}", self.api_key))
|
||||||
|
.json(&serde_json::json!({"inputs": text}))
|
||||||
|
.send()
|
||||||
|
.await
|
||||||
|
.map_err(|e| EmbeddingError::RequestFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
if !response.status().is_success() {
|
||||||
|
let status = response.status();
|
||||||
|
let text = response.text().await.unwrap_or_default();
|
||||||
|
return Err(EmbeddingError::RequestFailed(format!(
|
||||||
|
"HuggingFace API error {}: {}",
|
||||||
|
status, text
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
|
||||||
|
let data: HFEmbedResponse = response
|
||||||
|
.json()
|
||||||
|
.await
|
||||||
|
.map_err(|e| EmbeddingError::RequestFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
match data {
|
||||||
|
HFEmbedResponse::Single(embedding) => Ok(embedding),
|
||||||
|
HFEmbedResponse::Multiple(embeddings) => {
|
||||||
|
if embeddings.is_empty() {
|
||||||
|
Err(EmbeddingError::RequestFailed(
|
||||||
|
"No embeddings in response".to_string(),
|
||||||
|
))
|
||||||
|
} else {
|
||||||
|
Ok(embeddings[0].clone())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn provider_name(&self) -> &str {
|
fn provider_name(&self) -> &str {
|
||||||
@ -159,7 +310,7 @@ impl EmbeddingProvider for HuggingFaceEmbedding {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn model_name(&self) -> &str {
|
fn model_name(&self) -> &str {
|
||||||
self.provider.model()
|
&self.model
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -176,7 +327,7 @@ pub async fn create_embedding_provider(provider_name: &str) -> Result<Arc<dyn Em
|
|||||||
.unwrap_or_else(|_| "nomic-embed-text".to_string());
|
.unwrap_or_else(|_| "nomic-embed-text".to_string());
|
||||||
|
|
||||||
debug!("Creating Ollama embedding provider: {}", model);
|
debug!("Creating Ollama embedding provider: {}", model);
|
||||||
Ok(Arc::new(OllamaEmbedding::new(endpoint, model)?))
|
Ok(Arc::new(OllamaEmbedding::new(endpoint, model)))
|
||||||
}
|
}
|
||||||
|
|
||||||
"openai" => {
|
"openai" => {
|
||||||
@ -186,26 +337,22 @@ pub async fn create_embedding_provider(provider_name: &str) -> Result<Arc<dyn Em
|
|||||||
.unwrap_or_else(|_| "text-embedding-3-small".to_string());
|
.unwrap_or_else(|_| "text-embedding-3-small".to_string());
|
||||||
|
|
||||||
debug!("Creating OpenAI embedding provider: {}", model);
|
debug!("Creating OpenAI embedding provider: {}", model);
|
||||||
Ok(Arc::new(OpenAIEmbedding::new(api_key, model)?))
|
Ok(Arc::new(OpenAIEmbedding::new(api_key, model)))
|
||||||
}
|
}
|
||||||
|
|
||||||
"huggingface" => {
|
"huggingface" => {
|
||||||
let api_key = std::env::var("HUGGINGFACE_API_KEY")
|
let api_key = std::env::var("HUGGINGFACE_API_KEY").map_err(|_| {
|
||||||
.or_else(|_| std::env::var("HF_TOKEN"))
|
EmbeddingError::ConfigError("HUGGINGFACE_API_KEY not set".to_string())
|
||||||
.map_err(|_| {
|
|
||||||
EmbeddingError::ConfigError(
|
|
||||||
"HUGGINGFACE_API_KEY or HF_TOKEN not set".to_string(),
|
|
||||||
)
|
|
||||||
})?;
|
})?;
|
||||||
let model = std::env::var("HUGGINGFACE_MODEL")
|
let model = std::env::var("HUGGINGFACE_EMBEDDING_MODEL")
|
||||||
.unwrap_or_else(|_| "BAAI/bge-small-en-v1.5".to_string());
|
.unwrap_or_else(|_| "BAAI/bge-small-en-v1.5".to_string());
|
||||||
|
|
||||||
debug!("Creating HuggingFace embedding provider: {}", model);
|
debug!("Creating HuggingFace embedding provider: {}", model);
|
||||||
Ok(Arc::new(HuggingFaceEmbedding::new(api_key, model)?))
|
Ok(Arc::new(HuggingFaceEmbedding::new(api_key, model)))
|
||||||
}
|
}
|
||||||
|
|
||||||
_ => Err(EmbeddingError::ConfigError(format!(
|
_ => Err(EmbeddingError::ConfigError(format!(
|
||||||
"Unknown embedding provider: {}. Supported: ollama, openai, huggingface",
|
"Unknown embedding provider: {}",
|
||||||
provider_name
|
provider_name
|
||||||
))),
|
))),
|
||||||
}
|
}
|
||||||
@ -221,30 +368,27 @@ mod tests {
|
|||||||
"http://localhost:11434".to_string(),
|
"http://localhost:11434".to_string(),
|
||||||
"nomic-embed-text".to_string(),
|
"nomic-embed-text".to_string(),
|
||||||
);
|
);
|
||||||
assert!(ollama.is_ok());
|
|
||||||
let ollama = ollama.unwrap();
|
|
||||||
assert_eq!(ollama.provider_name(), "ollama");
|
assert_eq!(ollama.provider_name(), "ollama");
|
||||||
assert_eq!(ollama.model_name(), "nomic-embed-text");
|
assert_eq!(ollama.model_name(), "nomic-embed-text");
|
||||||
|
assert_eq!(ollama.embedding_dim(), 1536);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn test_openai_provider_creation() {
|
fn test_openai_provider_creation() {
|
||||||
let openai =
|
let openai =
|
||||||
OpenAIEmbedding::new("test-key".to_string(), "text-embedding-3-small".to_string());
|
OpenAIEmbedding::new("test-key".to_string(), "text-embedding-3-small".to_string());
|
||||||
assert!(openai.is_ok());
|
|
||||||
let openai = openai.unwrap();
|
|
||||||
assert_eq!(openai.provider_name(), "openai");
|
assert_eq!(openai.provider_name(), "openai");
|
||||||
assert_eq!(openai.model_name(), "text-embedding-3-small");
|
assert_eq!(openai.model_name(), "text-embedding-3-small");
|
||||||
|
assert_eq!(openai.embedding_dim(), 1536);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn test_huggingface_provider_creation() {
|
fn test_huggingface_provider_creation() {
|
||||||
let hf =
|
let hf =
|
||||||
HuggingFaceEmbedding::new("test-key".to_string(), "BAAI/bge-small-en-v1.5".to_string());
|
HuggingFaceEmbedding::new("test-key".to_string(), "BAAI/bge-small-en-v1.5".to_string());
|
||||||
assert!(hf.is_ok());
|
|
||||||
let hf = hf.unwrap();
|
|
||||||
assert_eq!(hf.provider_name(), "huggingface");
|
assert_eq!(hf.provider_name(), "huggingface");
|
||||||
assert_eq!(hf.model_name(), "BAAI/bge-small-en-v1.5");
|
assert_eq!(hf.model_name(), "BAAI/bge-small-en-v1.5");
|
||||||
|
assert_eq!(hf.embedding_dim(), 1536);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
@ -158,9 +158,11 @@ impl LLMRouter {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Check budget if provided
|
// Check budget if provided
|
||||||
if let (Some(role), Some(budget_mgr)) = (agent_role, &self.budget_manager) {
|
if let Some(role) = agent_role {
|
||||||
|
if let Some(budget_mgr) = &self.budget_manager {
|
||||||
match budget_mgr.check_budget(role).await {
|
match budget_mgr.check_budget(role).await {
|
||||||
Ok(status) if status.exceeded => {
|
Ok(status) => {
|
||||||
|
if status.exceeded {
|
||||||
// Budget exceeded - use fallback provider
|
// Budget exceeded - use fallback provider
|
||||||
info!(
|
info!(
|
||||||
"Budget exceeded for role {}, using fallback provider: {}",
|
"Budget exceeded for role {}, using fallback provider: {}",
|
||||||
@ -168,22 +170,23 @@ impl LLMRouter {
|
|||||||
);
|
);
|
||||||
return Ok(status.fallback_provider);
|
return Ok(status.fallback_provider);
|
||||||
}
|
}
|
||||||
Ok(status) if status.near_threshold => {
|
|
||||||
|
if status.near_threshold {
|
||||||
// Budget near threshold - prefer cost-efficient providers
|
// Budget near threshold - prefer cost-efficient providers
|
||||||
debug!(
|
debug!(
|
||||||
"Budget near threshold for role {}, selecting cost-efficient provider",
|
"Budget near threshold for role {}, selecting cost-efficient \
|
||||||
|
provider",
|
||||||
role
|
role
|
||||||
);
|
);
|
||||||
return self.select_cost_efficient_provider(task_type).await;
|
return self.select_cost_efficient_provider(task_type).await;
|
||||||
}
|
}
|
||||||
Ok(_) => {
|
|
||||||
// Budget ok, continue with normal routing
|
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
warn!("Budget check failed: {}, continuing with normal routing", e);
|
warn!("Budget check failed: {}, continuing with normal routing", e);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Try to find matching routing rule
|
// Try to find matching routing rule
|
||||||
if let Some(rule) = self.config.find_rule(&context) {
|
if let Some(rule) = self.config.find_rule(&context) {
|
||||||
@ -324,7 +327,13 @@ impl LLMRouter {
|
|||||||
);
|
);
|
||||||
|
|
||||||
// Record spend with budget manager if available
|
// Record spend with budget manager if available
|
||||||
self.record_budget_spend(agent_role, cost as u32).await;
|
if let Some(role) = agent_role {
|
||||||
|
if let Some(budget_mgr) = &self.budget_manager {
|
||||||
|
if let Err(e) = budget_mgr.record_spend(role, cost as u32).await {
|
||||||
|
warn!("Failed to record budget spend: {}", e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
Ok(response)
|
Ok(response)
|
||||||
@ -402,15 +411,6 @@ impl LLMRouter {
|
|||||||
latency_ms: provider.latency_ms(),
|
latency_ms: provider.latency_ms(),
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Record budget spend for agent role
|
|
||||||
async fn record_budget_spend(&self, agent_role: Option<&str>, cost: u32) {
|
|
||||||
if let (Some(role), Some(budget_mgr)) = (agent_role, &self.budget_manager) {
|
|
||||||
if let Err(e) = budget_mgr.record_spend(role, cost).await {
|
|
||||||
warn!("Failed to record budget spend: {}", e);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone)]
|
#[derive(Debug, Clone)]
|
||||||
|
|||||||
@ -1,13 +1,10 @@
|
|||||||
// vapora-mcp-server: Model Context Protocol server for VAPORA v1.0
|
// vapora-mcp-server: Model Context Protocol server for VAPORA v1.0
|
||||||
// Phase 2: Standalone MCP server with HTTP endpoints
|
// Phase 2: Standalone MCP server with HTTP endpoints
|
||||||
// Phase 3: Schema validation pipeline integration
|
|
||||||
|
|
||||||
use std::net::SocketAddr;
|
use std::net::SocketAddr;
|
||||||
use std::path::PathBuf;
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use axum::{
|
use axum::{
|
||||||
extract::{Json, Path, State},
|
extract::{Json, Path},
|
||||||
http::StatusCode,
|
http::StatusCode,
|
||||||
response::IntoResponse,
|
response::IntoResponse,
|
||||||
routing::{get, post},
|
routing::{get, post},
|
||||||
@ -18,7 +15,6 @@ use serde::{Deserialize, Serialize};
|
|||||||
use serde_json::json;
|
use serde_json::json;
|
||||||
use tokio::net::TcpListener;
|
use tokio::net::TcpListener;
|
||||||
use tracing::{info, warn};
|
use tracing::{info, warn};
|
||||||
use vapora_shared::validation::{SchemaRegistry, ValidationPipeline};
|
|
||||||
|
|
||||||
#[derive(Parser)]
|
#[derive(Parser)]
|
||||||
#[command(name = "vapora-mcp-server")]
|
#[command(name = "vapora-mcp-server")]
|
||||||
@ -60,15 +56,6 @@ struct PromptDefinition {
|
|||||||
description: String,
|
description: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
// ============================================================================
|
|
||||||
// App State
|
|
||||||
// ============================================================================
|
|
||||||
|
|
||||||
#[derive(Clone)]
|
|
||||||
struct AppState {
|
|
||||||
validation: Arc<ValidationPipeline>,
|
|
||||||
}
|
|
||||||
|
|
||||||
// ============================================================================
|
// ============================================================================
|
||||||
// Handlers
|
// Handlers
|
||||||
// ============================================================================
|
// ============================================================================
|
||||||
@ -166,59 +153,9 @@ async fn list_tools() -> impl IntoResponse {
|
|||||||
Json(json!({ "tools": tools }))
|
Json(json!({ "tools": tools }))
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn invoke_tool(
|
async fn invoke_tool(Json(request): Json<InvokeToolRequest>) -> impl IntoResponse {
|
||||||
State(state): State<AppState>,
|
|
||||||
Json(request): Json<InvokeToolRequest>,
|
|
||||||
) -> impl IntoResponse {
|
|
||||||
info!("Invoking tool: {}", request.tool);
|
info!("Invoking tool: {}", request.tool);
|
||||||
|
|
||||||
// Validate parameters against schema
|
|
||||||
let schema_name = format!("tools/{}", request.tool);
|
|
||||||
let validation_result = match state
|
|
||||||
.validation
|
|
||||||
.validate(&schema_name, &request.parameters)
|
|
||||||
.await
|
|
||||||
{
|
|
||||||
Ok(result) => result,
|
|
||||||
Err(e) => {
|
|
||||||
warn!("Schema validation error for {}: {}", request.tool, e);
|
|
||||||
return (
|
|
||||||
StatusCode::INTERNAL_SERVER_ERROR,
|
|
||||||
Json(json!({
|
|
||||||
"success": false,
|
|
||||||
"error": format!("Schema validation error: {}", e),
|
|
||||||
})),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Check if validation passed
|
|
||||||
if !validation_result.valid {
|
|
||||||
let error_messages: Vec<String> = validation_result
|
|
||||||
.errors
|
|
||||||
.iter()
|
|
||||||
.map(|e| e.to_string())
|
|
||||||
.collect();
|
|
||||||
|
|
||||||
warn!(
|
|
||||||
"Validation failed for {}: {}",
|
|
||||||
request.tool,
|
|
||||||
error_messages.join(", ")
|
|
||||||
);
|
|
||||||
|
|
||||||
return (
|
|
||||||
StatusCode::BAD_REQUEST,
|
|
||||||
Json(json!({
|
|
||||||
"success": false,
|
|
||||||
"error": "Validation failed",
|
|
||||||
"validation_errors": error_messages,
|
|
||||||
})),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Use validated data (with defaults applied)
|
|
||||||
let validated_params = validation_result.validated_data.unwrap();
|
|
||||||
|
|
||||||
let result = match request.tool.as_str() {
|
let result = match request.tool.as_str() {
|
||||||
"kanban_create_task" => json!({
|
"kanban_create_task" => json!({
|
||||||
"success": true,
|
"success": true,
|
||||||
@ -230,7 +167,7 @@ async fn invoke_tool(
|
|||||||
"message": "Task updated successfully"
|
"message": "Task updated successfully"
|
||||||
}),
|
}),
|
||||||
"get_project_summary" => json!({
|
"get_project_summary" => json!({
|
||||||
"project_id": validated_params.get("project_id").and_then(|v| v.as_str()).unwrap_or("unknown"),
|
"project_id": request.parameters.get("project_id").and_then(|v| v.as_str()).unwrap_or("unknown"),
|
||||||
"total_tasks": 42,
|
"total_tasks": 42,
|
||||||
"completed": 15,
|
"completed": 15,
|
||||||
"in_progress": 12,
|
"in_progress": 12,
|
||||||
@ -251,7 +188,7 @@ async fn invoke_tool(
|
|||||||
"success": true
|
"success": true
|
||||||
}),
|
}),
|
||||||
"get_agent_capabilities" => json!({
|
"get_agent_capabilities" => json!({
|
||||||
"agent_id": validated_params.get("agent_id").and_then(|v| v.as_str()).unwrap_or("unknown"),
|
"agent_id": request.parameters.get("agent_id").and_then(|v| v.as_str()).unwrap_or("unknown"),
|
||||||
"role": "Developer",
|
"role": "Developer",
|
||||||
"capabilities": ["coding", "debugging", "refactoring"],
|
"capabilities": ["coding", "debugging", "refactoring"],
|
||||||
"llm_provider": "claude",
|
"llm_provider": "claude",
|
||||||
@ -364,17 +301,6 @@ async fn main() -> anyhow::Result<()> {
|
|||||||
|
|
||||||
let args = Args::parse();
|
let args = Args::parse();
|
||||||
|
|
||||||
// Initialize validation pipeline
|
|
||||||
let schema_dir = std::env::var("VAPORA_SCHEMA_DIR").unwrap_or_else(|_| "schemas".to_string());
|
|
||||||
let schema_path = PathBuf::from(&schema_dir);
|
|
||||||
|
|
||||||
info!("Loading schemas from: {}", schema_path.display());
|
|
||||||
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(schema_path));
|
|
||||||
let validation = Arc::new(ValidationPipeline::new(registry));
|
|
||||||
|
|
||||||
let state = AppState { validation };
|
|
||||||
|
|
||||||
// Build router
|
// Build router
|
||||||
let app = Router::new()
|
let app = Router::new()
|
||||||
.route("/health", get(health))
|
.route("/health", get(health))
|
||||||
@ -382,8 +308,7 @@ async fn main() -> anyhow::Result<()> {
|
|||||||
.route("/mcp/invoke", post(invoke_tool))
|
.route("/mcp/invoke", post(invoke_tool))
|
||||||
.route("/mcp/resources", get(list_resources))
|
.route("/mcp/resources", get(list_resources))
|
||||||
.route("/mcp/resources/:uri", get(get_resource))
|
.route("/mcp/resources/:uri", get(get_resource))
|
||||||
.route("/mcp/prompts", get(list_prompts))
|
.route("/mcp/prompts", get(list_prompts));
|
||||||
.with_state(state);
|
|
||||||
|
|
||||||
// Bind address
|
// Bind address
|
||||||
let addr = format!("{}:{}", args.host, args.port).parse::<SocketAddr>()?;
|
let addr = format!("{}:{}", args.host, args.port).parse::<SocketAddr>()?;
|
||||||
@ -441,15 +366,7 @@ mod tests {
|
|||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn test_invoke_tool() {
|
async fn test_invoke_tool() {
|
||||||
// Create test state
|
let app = Router::new().route("/mcp/invoke", post(invoke_tool));
|
||||||
let schema_path = PathBuf::from("schemas");
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(schema_path));
|
|
||||||
let validation = Arc::new(ValidationPipeline::new(registry));
|
|
||||||
let state = AppState { validation };
|
|
||||||
|
|
||||||
let app = Router::new()
|
|
||||||
.route("/mcp/invoke", post(invoke_tool))
|
|
||||||
.with_state(state);
|
|
||||||
let server = TestServer::new(app).unwrap();
|
let server = TestServer::new(app).unwrap();
|
||||||
|
|
||||||
let request = json!({
|
let request = json!({
|
||||||
|
|||||||
@ -29,12 +29,6 @@ surrealdb = { workspace = true, optional = true }
|
|||||||
# Logging
|
# Logging
|
||||||
tracing = { workspace = true }
|
tracing = { workspace = true }
|
||||||
|
|
||||||
# Validation
|
|
||||||
regex = { workspace = true }
|
|
||||||
|
|
||||||
# Async runtime (for validation pipeline)
|
|
||||||
tokio = { workspace = true, features = ["process", "io-util"] }
|
|
||||||
|
|
||||||
[features]
|
[features]
|
||||||
default = ["backend"]
|
default = ["backend"]
|
||||||
backend = ["surrealdb"]
|
backend = ["surrealdb"]
|
||||||
|
|||||||
@ -22,10 +22,6 @@ pub enum VaporaError {
|
|||||||
#[error("Invalid input: {0}")]
|
#[error("Invalid input: {0}")]
|
||||||
InvalidInput(String),
|
InvalidInput(String),
|
||||||
|
|
||||||
/// Schema validation error (Nickel contracts)
|
|
||||||
#[error("Validation error: {0}")]
|
|
||||||
ValidationError(String),
|
|
||||||
|
|
||||||
/// Authentication or authorization error
|
/// Authentication or authorization error
|
||||||
#[error("Unauthorized: {0}")]
|
#[error("Unauthorized: {0}")]
|
||||||
Unauthorized(String),
|
Unauthorized(String),
|
||||||
|
|||||||
@ -3,6 +3,5 @@
|
|||||||
|
|
||||||
pub mod error;
|
pub mod error;
|
||||||
pub mod models;
|
pub mod models;
|
||||||
pub mod validation;
|
|
||||||
|
|
||||||
pub use error::{Result, VaporaError};
|
pub use error::{Result, VaporaError};
|
||||||
|
|||||||
@ -1,12 +0,0 @@
|
|||||||
// vapora-shared: Validation module - Schema validation pipeline with Nickel
|
|
||||||
// contracts Phase: Schema Validation Pipeline implementation
|
|
||||||
|
|
||||||
pub mod nickel_bridge;
|
|
||||||
pub mod pipeline;
|
|
||||||
pub mod schema_registry;
|
|
||||||
|
|
||||||
pub use nickel_bridge::NickelCli;
|
|
||||||
pub use pipeline::{ValidationError, ValidationPipeline, ValidationResult};
|
|
||||||
pub use schema_registry::{
|
|
||||||
CompiledSchema, Contract, FieldSchema, FieldType, SchemaRegistry, SchemaSource,
|
|
||||||
};
|
|
||||||
@ -1,254 +0,0 @@
|
|||||||
// vapora-shared: Nickel CLI bridge - Execute Nickel commands for schema
|
|
||||||
// operations Provides interface to nickel query, typecheck, and export commands
|
|
||||||
|
|
||||||
use std::path::{Path, PathBuf};
|
|
||||||
use std::process::Stdio;
|
|
||||||
|
|
||||||
use tokio::process::Command;
|
|
||||||
use tracing::debug;
|
|
||||||
|
|
||||||
use crate::error::{Result, VaporaError};
|
|
||||||
|
|
||||||
/// Nickel CLI bridge for schema operations
|
|
||||||
#[derive(Debug, Clone)]
|
|
||||||
pub struct NickelCli {
|
|
||||||
/// Path to nickel executable
|
|
||||||
nickel_path: PathBuf,
|
|
||||||
|
|
||||||
/// Timeout for nickel commands (milliseconds)
|
|
||||||
timeout_ms: u64,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl NickelCli {
|
|
||||||
/// Create new Nickel CLI bridge with default path
|
|
||||||
pub fn new() -> Self {
|
|
||||||
Self {
|
|
||||||
nickel_path: PathBuf::from("nickel"),
|
|
||||||
timeout_ms: 30_000, // 30 seconds
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Create Nickel CLI bridge with custom path
|
|
||||||
pub fn with_path(nickel_path: PathBuf) -> Self {
|
|
||||||
Self {
|
|
||||||
nickel_path,
|
|
||||||
timeout_ms: 30_000,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Set timeout for Nickel commands
|
|
||||||
pub fn with_timeout(mut self, timeout_ms: u64) -> Self {
|
|
||||||
self.timeout_ms = timeout_ms;
|
|
||||||
self
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Execute `nickel query` to extract schema metadata
|
|
||||||
///
|
|
||||||
/// Returns JSON representation of Nickel value
|
|
||||||
pub async fn query(&self, path: &Path, field: Option<&str>) -> Result<serde_json::Value> {
|
|
||||||
let mut args = vec!["query", "--format", "json"];
|
|
||||||
|
|
||||||
if let Some(f) = field {
|
|
||||||
args.push("--field");
|
|
||||||
args.push(f);
|
|
||||||
}
|
|
||||||
|
|
||||||
let path_str = path.to_str().ok_or_else(|| {
|
|
||||||
VaporaError::ValidationError(format!("Invalid path: {}", path.display()))
|
|
||||||
})?;
|
|
||||||
args.push(path_str);
|
|
||||||
|
|
||||||
debug!("Executing: nickel {}", args.join(" "));
|
|
||||||
|
|
||||||
let output = self.execute_with_timeout(&args).await?;
|
|
||||||
|
|
||||||
if !output.status.success() {
|
|
||||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
|
||||||
return Err(VaporaError::ValidationError(format!(
|
|
||||||
"Nickel query failed: {}",
|
|
||||||
stderr
|
|
||||||
)));
|
|
||||||
}
|
|
||||||
|
|
||||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
|
||||||
|
|
||||||
serde_json::from_str(&stdout).map_err(|e| {
|
|
||||||
VaporaError::ValidationError(format!("Failed to parse Nickel output: {}", e))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Execute `nickel typecheck` to validate schema
|
|
||||||
///
|
|
||||||
/// Returns Ok(()) if typecheck passes, Err with details if it fails
|
|
||||||
pub async fn typecheck(&self, path: &Path) -> Result<()> {
|
|
||||||
let path_str = path.to_str().ok_or_else(|| {
|
|
||||||
VaporaError::ValidationError(format!("Invalid path: {}", path.display()))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let args = vec!["typecheck", path_str];
|
|
||||||
|
|
||||||
debug!("Executing: nickel {}", args.join(" "));
|
|
||||||
|
|
||||||
let output = self.execute_with_timeout(&args).await?;
|
|
||||||
|
|
||||||
if !output.status.success() {
|
|
||||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
|
||||||
return Err(VaporaError::ValidationError(format!(
|
|
||||||
"Nickel typecheck failed: {}",
|
|
||||||
stderr
|
|
||||||
)));
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Execute `nickel export` to generate JSON output
|
|
||||||
///
|
|
||||||
/// Returns exported JSON value
|
|
||||||
pub async fn export(&self, path: &Path) -> Result<serde_json::Value> {
|
|
||||||
let path_str = path.to_str().ok_or_else(|| {
|
|
||||||
VaporaError::ValidationError(format!("Invalid path: {}", path.display()))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let args = vec!["export", "--format", "json", path_str];
|
|
||||||
|
|
||||||
debug!("Executing: nickel {}", args.join(" "));
|
|
||||||
|
|
||||||
let output = self.execute_with_timeout(&args).await?;
|
|
||||||
|
|
||||||
if !output.status.success() {
|
|
||||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
|
||||||
return Err(VaporaError::ValidationError(format!(
|
|
||||||
"Nickel export failed: {}",
|
|
||||||
stderr
|
|
||||||
)));
|
|
||||||
}
|
|
||||||
|
|
||||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
|
||||||
|
|
||||||
serde_json::from_str(&stdout).map_err(|e| {
|
|
||||||
VaporaError::ValidationError(format!("Failed to parse Nickel export: {}", e))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Check if Nickel CLI is available
|
|
||||||
pub async fn is_available(&self) -> bool {
|
|
||||||
let result = Command::new(&self.nickel_path)
|
|
||||||
.arg("--version")
|
|
||||||
.stdout(Stdio::null())
|
|
||||||
.stderr(Stdio::null())
|
|
||||||
.status()
|
|
||||||
.await;
|
|
||||||
|
|
||||||
match result {
|
|
||||||
Ok(status) => status.success(),
|
|
||||||
Err(_) => false,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Get Nickel CLI version
|
|
||||||
pub async fn version(&self) -> Result<String> {
|
|
||||||
let output = Command::new(&self.nickel_path)
|
|
||||||
.arg("--version")
|
|
||||||
.output()
|
|
||||||
.await
|
|
||||||
.map_err(|e| {
|
|
||||||
VaporaError::ValidationError(format!("Failed to get Nickel version: {}", e))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
if !output.status.success() {
|
|
||||||
return Err(VaporaError::ValidationError(
|
|
||||||
"Failed to get Nickel version".to_string(),
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
let version = String::from_utf8_lossy(&output.stdout);
|
|
||||||
Ok(version.trim().to_string())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Execute Nickel command with timeout
|
|
||||||
async fn execute_with_timeout(&self, args: &[&str]) -> Result<std::process::Output> {
|
|
||||||
let child = Command::new(&self.nickel_path)
|
|
||||||
.args(args)
|
|
||||||
.stdout(Stdio::piped())
|
|
||||||
.stderr(Stdio::piped())
|
|
||||||
.spawn()
|
|
||||||
.map_err(|e| {
|
|
||||||
VaporaError::ValidationError(format!("Failed to execute Nickel: {}", e))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
// Wait with timeout
|
|
||||||
let timeout = tokio::time::Duration::from_millis(self.timeout_ms);
|
|
||||||
let result = tokio::time::timeout(timeout, child.wait_with_output()).await;
|
|
||||||
|
|
||||||
match result {
|
|
||||||
Ok(Ok(output)) => Ok(output),
|
|
||||||
Ok(Err(e)) => Err(VaporaError::ValidationError(format!(
|
|
||||||
"Nickel command failed: {}",
|
|
||||||
e
|
|
||||||
))),
|
|
||||||
Err(_) => {
|
|
||||||
// Timeout occurred - process is still running but we give up
|
|
||||||
Err(VaporaError::ValidationError(format!(
|
|
||||||
"Nickel command timed out after {}ms",
|
|
||||||
self.timeout_ms
|
|
||||||
)))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Default for NickelCli {
|
|
||||||
fn default() -> Self {
|
|
||||||
Self::new()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_nickel_cli_creation() {
|
|
||||||
let cli = NickelCli::new();
|
|
||||||
assert_eq!(cli.nickel_path, PathBuf::from("nickel"));
|
|
||||||
assert_eq!(cli.timeout_ms, 30_000);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_nickel_cli_with_path() {
|
|
||||||
let custom_path = PathBuf::from("/usr/local/bin/nickel");
|
|
||||||
let cli = NickelCli::with_path(custom_path.clone());
|
|
||||||
assert_eq!(cli.nickel_path, custom_path);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_nickel_cli_with_timeout() {
|
|
||||||
let cli = NickelCli::new().with_timeout(5_000);
|
|
||||||
assert_eq!(cli.timeout_ms, 5_000);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
#[ignore] // Requires Nickel CLI installed
|
|
||||||
async fn test_nickel_is_available() {
|
|
||||||
let cli = NickelCli::new();
|
|
||||||
let available = cli.is_available().await;
|
|
||||||
// This will pass if nickel is in PATH
|
|
||||||
if available {
|
|
||||||
println!("Nickel CLI is available");
|
|
||||||
} else {
|
|
||||||
println!("Nickel CLI is not available (install it to run this test)");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
#[ignore] // Requires Nickel CLI installed
|
|
||||||
async fn test_nickel_version() {
|
|
||||||
let cli = NickelCli::new();
|
|
||||||
if cli.is_available().await {
|
|
||||||
let version = cli.version().await.unwrap();
|
|
||||||
println!("Nickel version: {}", version);
|
|
||||||
assert!(!version.is_empty());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,636 +0,0 @@
|
|||||||
// vapora-shared: Validation pipeline - Runtime validation against Nickel
|
|
||||||
// contracts Validates JSON inputs against compiled schemas with detailed error
|
|
||||||
// reporting
|
|
||||||
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use regex::Regex;
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use serde_json::Value;
|
|
||||||
use tracing::{debug, warn};
|
|
||||||
|
|
||||||
use super::schema_registry::{CompiledSchema, Contract, FieldType, SchemaRegistry};
|
|
||||||
use crate::error::Result;
|
|
||||||
|
|
||||||
/// Validation error with field context
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
||||||
pub struct ValidationError {
|
|
||||||
/// Field name that failed validation
|
|
||||||
pub field: String,
|
|
||||||
|
|
||||||
/// Human-readable error message
|
|
||||||
pub message: String,
|
|
||||||
|
|
||||||
/// Contract that was violated
|
|
||||||
pub contract: String,
|
|
||||||
|
|
||||||
/// Expected value/type
|
|
||||||
pub expected: Option<String>,
|
|
||||||
|
|
||||||
/// Actual value received
|
|
||||||
pub actual: Option<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ValidationError {
|
|
||||||
/// Create a new validation error
|
|
||||||
pub fn new(
|
|
||||||
field: impl Into<String>,
|
|
||||||
message: impl Into<String>,
|
|
||||||
contract: impl Into<String>,
|
|
||||||
) -> Self {
|
|
||||||
Self {
|
|
||||||
field: field.into(),
|
|
||||||
message: message.into(),
|
|
||||||
contract: contract.into(),
|
|
||||||
expected: None,
|
|
||||||
actual: None,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Add expected value to error
|
|
||||||
pub fn with_expected(mut self, expected: impl Into<String>) -> Self {
|
|
||||||
self.expected = Some(expected.into());
|
|
||||||
self
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Add actual value to error
|
|
||||||
pub fn with_actual(mut self, actual: impl Into<String>) -> Self {
|
|
||||||
self.actual = Some(actual.into());
|
|
||||||
self
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl std::fmt::Display for ValidationError {
|
|
||||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
|
||||||
write!(f, "Field '{}': {}", self.field, self.message)?;
|
|
||||||
|
|
||||||
if let Some(expected) = &self.expected {
|
|
||||||
write!(f, " (expected: {})", expected)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
if let Some(actual) = &self.actual {
|
|
||||||
write!(f, " (got: {})", actual)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
write!(f, " [contract: {}]", self.contract)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Result of validation pipeline
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct ValidationResult {
|
|
||||||
/// Is input valid?
|
|
||||||
pub valid: bool,
|
|
||||||
|
|
||||||
/// Validation errors (if any)
|
|
||||||
pub errors: Vec<ValidationError>,
|
|
||||||
|
|
||||||
/// Validated data with defaults applied
|
|
||||||
pub validated_data: Option<Value>,
|
|
||||||
|
|
||||||
/// Schema name that was used
|
|
||||||
pub schema_name: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ValidationResult {
|
|
||||||
/// Create successful validation result
|
|
||||||
pub fn ok(data: Value, schema_name: impl Into<String>) -> Self {
|
|
||||||
Self {
|
|
||||||
valid: true,
|
|
||||||
errors: vec![],
|
|
||||||
validated_data: Some(data),
|
|
||||||
schema_name: schema_name.into(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Create failed validation result
|
|
||||||
pub fn err(errors: Vec<ValidationError>, schema_name: impl Into<String>) -> Self {
|
|
||||||
Self {
|
|
||||||
valid: false,
|
|
||||||
errors,
|
|
||||||
validated_data: None,
|
|
||||||
schema_name: schema_name.into(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Get all error messages
|
|
||||||
pub fn error_messages(&self) -> Vec<String> {
|
|
||||||
self.errors.iter().map(|e| e.to_string()).collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Check if specific field has error
|
|
||||||
pub fn has_field_error(&self, field_name: &str) -> bool {
|
|
||||||
self.errors.iter().any(|e| e.field == field_name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Main validation pipeline
|
|
||||||
pub struct ValidationPipeline {
|
|
||||||
/// Schema registry
|
|
||||||
registry: Arc<SchemaRegistry>,
|
|
||||||
|
|
||||||
/// Enable strict mode (fail on unknown fields)
|
|
||||||
strict_mode: bool,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ValidationPipeline {
|
|
||||||
/// Create new validation pipeline
|
|
||||||
pub fn new(registry: Arc<SchemaRegistry>) -> Self {
|
|
||||||
Self {
|
|
||||||
registry,
|
|
||||||
strict_mode: false,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Enable strict mode (reject unknown fields)
|
|
||||||
pub fn with_strict_mode(mut self, enabled: bool) -> Self {
|
|
||||||
self.strict_mode = enabled;
|
|
||||||
self
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Validate input against a named schema
|
|
||||||
pub async fn validate(&self, schema_name: &str, input: &Value) -> Result<ValidationResult> {
|
|
||||||
let schema = self.registry.load_schema(schema_name).await?;
|
|
||||||
self.validate_against_schema(&schema, input)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Validate input against compiled schema
|
|
||||||
pub fn validate_against_schema(
|
|
||||||
&self,
|
|
||||||
schema: &CompiledSchema,
|
|
||||||
input: &Value,
|
|
||||||
) -> Result<ValidationResult> {
|
|
||||||
let mut errors = vec![];
|
|
||||||
let mut validated = serde_json::Map::new();
|
|
||||||
|
|
||||||
let input_obj = match input.as_object() {
|
|
||||||
Some(obj) => obj,
|
|
||||||
None => {
|
|
||||||
errors.push(ValidationError::new(
|
|
||||||
"<root>",
|
|
||||||
"Input must be a JSON object",
|
|
||||||
"type",
|
|
||||||
));
|
|
||||||
return Ok(ValidationResult::err(errors, &schema.name));
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Validate all defined fields
|
|
||||||
for field_schema in &schema.fields {
|
|
||||||
let field_value = input_obj.get(&field_schema.name);
|
|
||||||
|
|
||||||
// Check required fields
|
|
||||||
if field_schema.required && field_value.is_none() {
|
|
||||||
errors.push(
|
|
||||||
ValidationError::new(&field_schema.name, "Required field missing", "required")
|
|
||||||
.with_expected(format!("{:?}", field_schema.field_type)),
|
|
||||||
);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Apply default if missing
|
|
||||||
let value = match field_value {
|
|
||||||
Some(v) => v.clone(),
|
|
||||||
None => match &field_schema.default {
|
|
||||||
Some(default) => {
|
|
||||||
debug!(
|
|
||||||
"Applying default for field {}: {:?}",
|
|
||||||
field_schema.name, default
|
|
||||||
);
|
|
||||||
default.clone()
|
|
||||||
}
|
|
||||||
None => continue,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
|
|
||||||
// Type validation
|
|
||||||
if let Err(e) = self.validate_type(&field_schema.name, &value, &field_schema.field_type)
|
|
||||||
{
|
|
||||||
errors.push(e);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Contract validation
|
|
||||||
for contract in &field_schema.contracts {
|
|
||||||
if let Err(e) = self.validate_contract(&field_schema.name, &value, contract) {
|
|
||||||
errors.push(e);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
validated.insert(field_schema.name.clone(), value);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Strict mode: check for unknown fields
|
|
||||||
if self.strict_mode {
|
|
||||||
for (key, _value) in input_obj {
|
|
||||||
if !schema.fields.iter().any(|f| &f.name == key) {
|
|
||||||
errors.push(ValidationError::new(
|
|
||||||
key,
|
|
||||||
"Unknown field (strict mode enabled)",
|
|
||||||
"strict",
|
|
||||||
));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if errors.is_empty() {
|
|
||||||
Ok(ValidationResult::ok(Value::Object(validated), &schema.name))
|
|
||||||
} else {
|
|
||||||
Ok(ValidationResult::err(errors, &schema.name))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Validate value type
|
|
||||||
fn validate_type(
|
|
||||||
&self,
|
|
||||||
field_name: &str,
|
|
||||||
value: &Value,
|
|
||||||
expected: &FieldType,
|
|
||||||
) -> std::result::Result<(), ValidationError> {
|
|
||||||
let valid = match expected {
|
|
||||||
FieldType::String => value.is_string(),
|
|
||||||
FieldType::Number => value.is_number(),
|
|
||||||
FieldType::Bool => value.is_boolean(),
|
|
||||||
FieldType::Array(_) => value.is_array(),
|
|
||||||
FieldType::Object => value.is_object(),
|
|
||||||
FieldType::Enum(variants) => value
|
|
||||||
.as_str()
|
|
||||||
.map(|s| variants.contains(&s.to_string()))
|
|
||||||
.unwrap_or(false),
|
|
||||||
FieldType::Union(types) => {
|
|
||||||
// Valid if matches any type in union
|
|
||||||
types
|
|
||||||
.iter()
|
|
||||||
.any(|t| self.validate_type(field_name, value, t).is_ok())
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
if valid {
|
|
||||||
Ok(())
|
|
||||||
} else {
|
|
||||||
Err(ValidationError::new(field_name, "Type mismatch", "type")
|
|
||||||
.with_expected(format!("{:?}", expected))
|
|
||||||
.with_actual(format!("{:?}", value)))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Validate value against contract
|
|
||||||
fn validate_contract(
|
|
||||||
&self,
|
|
||||||
field_name: &str,
|
|
||||||
value: &Value,
|
|
||||||
contract: &Contract,
|
|
||||||
) -> std::result::Result<(), ValidationError> {
|
|
||||||
match contract {
|
|
||||||
Contract::NonEmpty => {
|
|
||||||
if let Some(s) = value.as_str() {
|
|
||||||
if s.is_empty() {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
"String cannot be empty",
|
|
||||||
"std.string.NonEmpty",
|
|
||||||
));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::MinLength(min) => {
|
|
||||||
if let Some(s) = value.as_str() {
|
|
||||||
if s.len() < *min {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
format!("String too short (minimum {} characters)", min),
|
|
||||||
format!("std.string.length.min {}", min),
|
|
||||||
)
|
|
||||||
.with_expected(format!("≥ {} chars", min))
|
|
||||||
.with_actual(format!("{} chars", s.len())));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::MaxLength(max) => {
|
|
||||||
if let Some(s) = value.as_str() {
|
|
||||||
if s.len() > *max {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
format!("String too long (maximum {} characters)", max),
|
|
||||||
format!("std.string.length.max {}", max),
|
|
||||||
)
|
|
||||||
.with_expected(format!("≤ {} chars", max))
|
|
||||||
.with_actual(format!("{} chars", s.len())));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::Pattern(pattern) => {
|
|
||||||
if let Some(s) = value.as_str() {
|
|
||||||
let re = Regex::new(pattern).map_err(|e| {
|
|
||||||
ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
format!("Invalid regex pattern: {}", e),
|
|
||||||
"std.string.match",
|
|
||||||
)
|
|
||||||
})?;
|
|
||||||
|
|
||||||
if !re.is_match(s) {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
"Value does not match required pattern",
|
|
||||||
format!("std.string.match {}", pattern),
|
|
||||||
)
|
|
||||||
.with_expected(format!("pattern: {}", pattern))
|
|
||||||
.with_actual(s));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::Range { min, max } => {
|
|
||||||
if let Some(n) = value.as_f64() {
|
|
||||||
if n < *min || n > *max {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
format!("Number must be between {} and {}", min, max),
|
|
||||||
format!("std.number.between {} {}", min, max),
|
|
||||||
)
|
|
||||||
.with_expected(format!("[{}, {}]", min, max))
|
|
||||||
.with_actual(n.to_string()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::GreaterThan(min) => {
|
|
||||||
if let Some(n) = value.as_f64() {
|
|
||||||
if n <= *min {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
format!("Number must be greater than {}", min),
|
|
||||||
format!("std.number.greater_than {}", min),
|
|
||||||
)
|
|
||||||
.with_expected(format!("> {}", min))
|
|
||||||
.with_actual(n.to_string()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::LessThan(max) => {
|
|
||||||
if let Some(n) = value.as_f64() {
|
|
||||||
if n >= *max {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
format!("Number must be less than {}", max),
|
|
||||||
format!("std.number.less_than {}", max),
|
|
||||||
)
|
|
||||||
.with_expected(format!("< {}", max))
|
|
||||||
.with_actual(n.to_string()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::Email => {
|
|
||||||
if let Some(s) = value.as_str() {
|
|
||||||
// Simple email validation (not RFC-compliant, but good enough)
|
|
||||||
if !s.contains('@') || !s.contains('.') || s.len() < 5 {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
"Invalid email format",
|
|
||||||
"std.string.Email",
|
|
||||||
)
|
|
||||||
.with_expected("user@example.com")
|
|
||||||
.with_actual(s));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check for multiple @
|
|
||||||
if s.matches('@').count() != 1 {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
"Invalid email format (multiple @ symbols)",
|
|
||||||
"std.string.Email",
|
|
||||||
));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::Url => {
|
|
||||||
if let Some(s) = value.as_str() {
|
|
||||||
if !s.starts_with("http://") && !s.starts_with("https://") {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
"Invalid URL format (must start with http:// or https://)",
|
|
||||||
"std.string.Url",
|
|
||||||
)
|
|
||||||
.with_expected("https://example.com")
|
|
||||||
.with_actual(s));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::Uuid => {
|
|
||||||
if let Some(s) = value.as_str() {
|
|
||||||
let uuid_pattern =
|
|
||||||
r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$";
|
|
||||||
let re = Regex::new(uuid_pattern).unwrap();
|
|
||||||
|
|
||||||
if !re.is_match(s) {
|
|
||||||
return Err(ValidationError::new(
|
|
||||||
field_name,
|
|
||||||
"Invalid UUID format",
|
|
||||||
"std.string.Uuid",
|
|
||||||
)
|
|
||||||
.with_expected("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")
|
|
||||||
.with_actual(s));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Contract::Custom(source) => {
|
|
||||||
// Custom contracts require Nickel evaluation
|
|
||||||
// For now, warn and skip
|
|
||||||
warn!(
|
|
||||||
"Custom contract validation not yet implemented for field {}: {}",
|
|
||||||
field_name, source
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use std::path::PathBuf;
|
|
||||||
|
|
||||||
use serde_json::json;
|
|
||||||
|
|
||||||
use super::*;
|
|
||||||
use crate::validation::schema_registry::{CompiledSchema, Contract, FieldSchema, FieldType};
|
|
||||||
|
|
||||||
fn create_test_schema() -> CompiledSchema {
|
|
||||||
CompiledSchema {
|
|
||||||
name: "test_schema".to_string(),
|
|
||||||
fields: vec![
|
|
||||||
FieldSchema {
|
|
||||||
name: "title".to_string(),
|
|
||||||
field_type: FieldType::String,
|
|
||||||
required: true,
|
|
||||||
contracts: vec![Contract::NonEmpty, Contract::MaxLength(100)],
|
|
||||||
default: None,
|
|
||||||
doc: Some("Task title".to_string()),
|
|
||||||
},
|
|
||||||
FieldSchema {
|
|
||||||
name: "priority".to_string(),
|
|
||||||
field_type: FieldType::Number,
|
|
||||||
required: true,
|
|
||||||
contracts: vec![Contract::Range {
|
|
||||||
min: 0.0,
|
|
||||||
max: 100.0,
|
|
||||||
}],
|
|
||||||
default: None,
|
|
||||||
doc: Some("Priority score".to_string()),
|
|
||||||
},
|
|
||||||
FieldSchema {
|
|
||||||
name: "email".to_string(),
|
|
||||||
field_type: FieldType::String,
|
|
||||||
required: false,
|
|
||||||
contracts: vec![Contract::Email],
|
|
||||||
default: Some(json!("noreply@example.com")),
|
|
||||||
doc: None,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
custom_contracts: vec![],
|
|
||||||
source_path: PathBuf::from("test.ncl"),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_validation_error_display() {
|
|
||||||
let error = ValidationError::new("title", "Field is required", "required")
|
|
||||||
.with_expected("String")
|
|
||||||
.with_actual("null");
|
|
||||||
|
|
||||||
let display = error.to_string();
|
|
||||||
assert!(display.contains("title"));
|
|
||||||
assert!(display.contains("required"));
|
|
||||||
assert!(display.contains("String"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_validation_result_ok() {
|
|
||||||
let data = json!({"title": "Test"});
|
|
||||||
let result = ValidationResult::ok(data.clone(), "test_schema");
|
|
||||||
|
|
||||||
assert!(result.valid);
|
|
||||||
assert!(result.errors.is_empty());
|
|
||||||
assert_eq!(result.validated_data, Some(data));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_validation_result_err() {
|
|
||||||
let errors = vec![ValidationError::new("field1", "Error", "contract1")];
|
|
||||||
let result = ValidationResult::err(errors, "test_schema");
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert_eq!(result.errors.len(), 1);
|
|
||||||
assert!(result.validated_data.is_none());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_validate_valid_input() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("/tmp")));
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
let schema = create_test_schema();
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"title": "Test Task",
|
|
||||||
"priority": 50.0
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
assert!(result.valid, "Validation should pass for valid input");
|
|
||||||
assert!(result.errors.is_empty());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_validate_missing_required_field() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("/tmp")));
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
let schema = create_test_schema();
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"priority": 50.0
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("title"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_validate_contract_violation() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("/tmp")));
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
let schema = create_test_schema();
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"title": "", // Empty string violates NonEmpty
|
|
||||||
"priority": 50.0
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("title"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_validate_with_defaults() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("/tmp")));
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
let schema = create_test_schema();
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"title": "Test",
|
|
||||||
"priority": 50.0
|
|
||||||
// email not provided, should use default
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
assert!(result.valid);
|
|
||||||
|
|
||||||
let validated = result.validated_data.unwrap();
|
|
||||||
assert_eq!(validated["email"], "noreply@example.com");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_validate_number_range() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("/tmp")));
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
let schema = create_test_schema();
|
|
||||||
|
|
||||||
// Out of range
|
|
||||||
let input = json!({
|
|
||||||
"title": "Test",
|
|
||||||
"priority": 150.0 // > 100
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("priority"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_validate_email_format() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("/tmp")));
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
let schema = create_test_schema();
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"title": "Test",
|
|
||||||
"priority": 50.0,
|
|
||||||
"email": "invalid-email" // Missing @
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("email"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,481 +0,0 @@
|
|||||||
// vapora-shared: Schema registry - Loads and caches Nickel schemas for
|
|
||||||
// validation Compiles Nickel schemas to internal representation for runtime
|
|
||||||
// validation
|
|
||||||
|
|
||||||
use std::collections::HashMap;
|
|
||||||
use std::path::{Path, PathBuf};
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use tokio::sync::RwLock;
|
|
||||||
use tracing::{debug, info, warn};
|
|
||||||
|
|
||||||
use super::nickel_bridge::NickelCli;
|
|
||||||
use crate::error::{Result, VaporaError};
|
|
||||||
|
|
||||||
/// Compiled schema representation (from Nickel)
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
||||||
pub struct CompiledSchema {
|
|
||||||
/// Schema name (e.g., "tools/kanban_create_task")
|
|
||||||
pub name: String,
|
|
||||||
|
|
||||||
/// Field definitions with types and contracts
|
|
||||||
pub fields: Vec<FieldSchema>,
|
|
||||||
|
|
||||||
/// Custom contract predicates (Nickel functions)
|
|
||||||
pub custom_contracts: Vec<String>,
|
|
||||||
|
|
||||||
/// Source file path
|
|
||||||
pub source_path: PathBuf,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Field schema with type and validation contracts
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
||||||
pub struct FieldSchema {
|
|
||||||
/// Field name (flattened path for nested fields)
|
|
||||||
pub name: String,
|
|
||||||
|
|
||||||
/// Field type (Nickel type)
|
|
||||||
pub field_type: FieldType,
|
|
||||||
|
|
||||||
/// Is this field required?
|
|
||||||
pub required: bool,
|
|
||||||
|
|
||||||
/// Validation contracts (predicates)
|
|
||||||
pub contracts: Vec<Contract>,
|
|
||||||
|
|
||||||
/// Default value (if field is optional)
|
|
||||||
pub default: Option<serde_json::Value>,
|
|
||||||
|
|
||||||
/// Documentation string
|
|
||||||
pub doc: Option<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Nickel field types
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
|
||||||
pub enum FieldType {
|
|
||||||
/// String type
|
|
||||||
String,
|
|
||||||
|
|
||||||
/// Number type (f64)
|
|
||||||
Number,
|
|
||||||
|
|
||||||
/// Boolean type
|
|
||||||
Bool,
|
|
||||||
|
|
||||||
/// Array type with element type
|
|
||||||
Array(Box<FieldType>),
|
|
||||||
|
|
||||||
/// Object/Record type
|
|
||||||
Object,
|
|
||||||
|
|
||||||
/// Enum with allowed values
|
|
||||||
Enum(Vec<String>),
|
|
||||||
|
|
||||||
/// Union of types (Nickel `|`)
|
|
||||||
Union(Vec<FieldType>),
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Validation contract (Nickel predicate)
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
|
||||||
pub enum Contract {
|
|
||||||
/// Non-empty string (std.string.NonEmpty)
|
|
||||||
NonEmpty,
|
|
||||||
|
|
||||||
/// Minimum string length
|
|
||||||
MinLength(usize),
|
|
||||||
|
|
||||||
/// Maximum string length
|
|
||||||
MaxLength(usize),
|
|
||||||
|
|
||||||
/// Regex pattern match
|
|
||||||
Pattern(String),
|
|
||||||
|
|
||||||
/// Numeric range validation
|
|
||||||
Range { min: f64, max: f64 },
|
|
||||||
|
|
||||||
/// Greater than (exclusive)
|
|
||||||
GreaterThan(f64),
|
|
||||||
|
|
||||||
/// Less than (exclusive)
|
|
||||||
LessThan(f64),
|
|
||||||
|
|
||||||
/// Email validation
|
|
||||||
Email,
|
|
||||||
|
|
||||||
/// URL validation
|
|
||||||
Url,
|
|
||||||
|
|
||||||
/// UUID validation
|
|
||||||
Uuid,
|
|
||||||
|
|
||||||
/// Custom Nickel predicate (function source code)
|
|
||||||
Custom(String),
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Schema source (file or inline definition)
|
|
||||||
#[derive(Debug, Clone)]
|
|
||||||
pub enum SchemaSource {
|
|
||||||
/// Load from Nickel file
|
|
||||||
File(PathBuf),
|
|
||||||
|
|
||||||
/// Inline Nickel code
|
|
||||||
Inline(String),
|
|
||||||
|
|
||||||
/// Pre-compiled schema (for testing)
|
|
||||||
Compiled(CompiledSchema),
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Registry that loads and caches Nickel schemas
|
|
||||||
pub struct SchemaRegistry {
|
|
||||||
/// Cached compiled schemas
|
|
||||||
schemas: Arc<RwLock<HashMap<String, CompiledSchema>>>,
|
|
||||||
|
|
||||||
/// Base directory for schema files
|
|
||||||
schema_dir: PathBuf,
|
|
||||||
|
|
||||||
/// Nickel CLI bridge
|
|
||||||
nickel_cli: NickelCli,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl SchemaRegistry {
|
|
||||||
/// Create a new schema registry
|
|
||||||
pub fn new(schema_dir: PathBuf) -> Self {
|
|
||||||
Self {
|
|
||||||
schemas: Arc::new(RwLock::new(HashMap::new())),
|
|
||||||
schema_dir,
|
|
||||||
nickel_cli: NickelCli::new(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Create registry with custom Nickel CLI path
|
|
||||||
pub fn with_nickel_cli(schema_dir: PathBuf, nickel_path: PathBuf) -> Self {
|
|
||||||
Self {
|
|
||||||
schemas: Arc::new(RwLock::new(HashMap::new())),
|
|
||||||
schema_dir,
|
|
||||||
nickel_cli: NickelCli::with_path(nickel_path),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Load schema from Nickel file, compile and cache
|
|
||||||
pub async fn load_schema(&self, schema_name: &str) -> Result<CompiledSchema> {
|
|
||||||
// Check cache first
|
|
||||||
{
|
|
||||||
let cache = self.schemas.read().await;
|
|
||||||
if let Some(schema) = cache.get(schema_name) {
|
|
||||||
debug!("Schema {} loaded from cache", schema_name);
|
|
||||||
return Ok(schema.clone());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Load from file
|
|
||||||
let schema_path = self.resolve_schema_path(schema_name);
|
|
||||||
let compiled = self.compile_schema(&schema_path, schema_name).await?;
|
|
||||||
|
|
||||||
// Cache it
|
|
||||||
{
|
|
||||||
let mut cache = self.schemas.write().await;
|
|
||||||
cache.insert(schema_name.to_string(), compiled.clone());
|
|
||||||
}
|
|
||||||
|
|
||||||
info!("Schema {} compiled and cached", schema_name);
|
|
||||||
Ok(compiled)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Load schema from source (file, inline, or pre-compiled)
|
|
||||||
pub async fn load_from_source(
|
|
||||||
&self,
|
|
||||||
source: SchemaSource,
|
|
||||||
schema_name: &str,
|
|
||||||
) -> Result<CompiledSchema> {
|
|
||||||
match source {
|
|
||||||
SchemaSource::File(path) => self.compile_schema(&path, schema_name).await,
|
|
||||||
SchemaSource::Inline(code) => self.compile_inline(&code, schema_name).await,
|
|
||||||
SchemaSource::Compiled(schema) => Ok(schema),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Resolve schema name to file path
|
|
||||||
fn resolve_schema_path(&self, schema_name: &str) -> PathBuf {
|
|
||||||
let filename = if schema_name.ends_with(".ncl") {
|
|
||||||
schema_name.to_string()
|
|
||||||
} else {
|
|
||||||
format!("{}.ncl", schema_name)
|
|
||||||
};
|
|
||||||
|
|
||||||
self.schema_dir.join(filename)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Compile Nickel schema from file to internal representation
|
|
||||||
async fn compile_schema(&self, path: &Path, name: &str) -> Result<CompiledSchema> {
|
|
||||||
if !path.exists() {
|
|
||||||
return Err(VaporaError::ValidationError(format!(
|
|
||||||
"Schema file not found: {}",
|
|
||||||
path.display()
|
|
||||||
)));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Use nickel query to extract schema metadata
|
|
||||||
let metadata_json = self.nickel_cli.query(path, None).await?;
|
|
||||||
|
|
||||||
self.parse_nickel_metadata(&metadata_json, name, path)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Compile inline Nickel code
|
|
||||||
async fn compile_inline(&self, code: &str, name: &str) -> Result<CompiledSchema> {
|
|
||||||
// Write to temp file for Nickel CLI processing
|
|
||||||
let temp_dir = std::env::temp_dir();
|
|
||||||
let temp_path = temp_dir.join(format!("vapora_schema_{}.ncl", name.replace('/', "_")));
|
|
||||||
|
|
||||||
tokio::fs::write(&temp_path, code).await.map_err(|e| {
|
|
||||||
VaporaError::ValidationError(format!("Failed to write temp file: {}", e))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let result = self.compile_schema(&temp_path, name).await;
|
|
||||||
|
|
||||||
// Clean up temp file
|
|
||||||
let _ = tokio::fs::remove_file(&temp_path).await;
|
|
||||||
|
|
||||||
result
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Parse Nickel metadata JSON into CompiledSchema
|
|
||||||
fn parse_nickel_metadata(
|
|
||||||
&self,
|
|
||||||
json: &serde_json::Value,
|
|
||||||
name: &str,
|
|
||||||
source_path: &Path,
|
|
||||||
) -> Result<CompiledSchema> {
|
|
||||||
// Extract schema name from metadata or use provided name
|
|
||||||
let schema_name = json["tool_name"]
|
|
||||||
.as_str()
|
|
||||||
.or_else(|| json["schema_name"].as_str())
|
|
||||||
.unwrap_or(name)
|
|
||||||
.to_string();
|
|
||||||
|
|
||||||
// Extract parameters/fields object
|
|
||||||
let params = json["parameters"]
|
|
||||||
.as_object()
|
|
||||||
.or_else(|| json["fields"].as_object());
|
|
||||||
|
|
||||||
let fields = match params {
|
|
||||||
Some(params_obj) => params_obj
|
|
||||||
.iter()
|
|
||||||
.map(|(field_name, field_def)| self.parse_field(field_name, field_def))
|
|
||||||
.collect::<Result<Vec<_>>>()?,
|
|
||||||
None => {
|
|
||||||
warn!("No parameters or fields found in schema {}", name);
|
|
||||||
vec![]
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Extract custom contracts
|
|
||||||
let custom_contracts = json["contracts"]
|
|
||||||
.as_object()
|
|
||||||
.map(|obj| {
|
|
||||||
obj.iter()
|
|
||||||
.map(|(name, _code)| name.clone())
|
|
||||||
.collect::<Vec<_>>()
|
|
||||||
})
|
|
||||||
.unwrap_or_default();
|
|
||||||
|
|
||||||
Ok(CompiledSchema {
|
|
||||||
name: schema_name,
|
|
||||||
fields,
|
|
||||||
custom_contracts,
|
|
||||||
source_path: source_path.to_path_buf(),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Parse a single field definition from Nickel metadata
|
|
||||||
fn parse_field(&self, name: &str, def: &serde_json::Value) -> Result<FieldSchema> {
|
|
||||||
let field_type = self.parse_type(def)?;
|
|
||||||
let contracts = self.extract_contracts(def);
|
|
||||||
let default = def.get("default").cloned();
|
|
||||||
let required = default.is_none();
|
|
||||||
let doc = def.get("doc").and_then(|v| v.as_str()).map(String::from);
|
|
||||||
|
|
||||||
Ok(FieldSchema {
|
|
||||||
name: name.to_string(),
|
|
||||||
field_type,
|
|
||||||
required,
|
|
||||||
contracts,
|
|
||||||
default,
|
|
||||||
doc,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Parse Nickel type from metadata
|
|
||||||
fn parse_type(&self, def: &serde_json::Value) -> Result<FieldType> {
|
|
||||||
// Type information comes from Nickel metadata
|
|
||||||
let type_str = def["type"].as_str().unwrap_or("String");
|
|
||||||
|
|
||||||
let field_type = match type_str {
|
|
||||||
"String" => FieldType::String,
|
|
||||||
"Number" => FieldType::Number,
|
|
||||||
"Bool" => FieldType::Bool,
|
|
||||||
"Object" | "Record" => FieldType::Object,
|
|
||||||
t if t.starts_with("Array") => {
|
|
||||||
// Parse array element type
|
|
||||||
FieldType::Array(Box::new(FieldType::String))
|
|
||||||
}
|
|
||||||
t if t.starts_with("Enum") => {
|
|
||||||
// Extract enum variants
|
|
||||||
if let Some(variants) = def["variants"].as_array() {
|
|
||||||
let values = variants
|
|
||||||
.iter()
|
|
||||||
.filter_map(|v| v.as_str().map(String::from))
|
|
||||||
.collect();
|
|
||||||
FieldType::Enum(values)
|
|
||||||
} else {
|
|
||||||
FieldType::String
|
|
||||||
}
|
|
||||||
}
|
|
||||||
_ => FieldType::String,
|
|
||||||
};
|
|
||||||
|
|
||||||
Ok(field_type)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Extract validation contracts from Nickel metadata
|
|
||||||
fn extract_contracts(&self, def: &serde_json::Value) -> Vec<Contract> {
|
|
||||||
let mut contracts = vec![];
|
|
||||||
|
|
||||||
// Parse contract annotations from Nickel metadata
|
|
||||||
if let Some(annotations) = def["annotations"].as_array() {
|
|
||||||
for ann in annotations {
|
|
||||||
if let Some(contract) = self.parse_contract_annotation(ann) {
|
|
||||||
contracts.push(contract);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
contracts
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Parse a single contract annotation
|
|
||||||
fn parse_contract_annotation(&self, ann: &serde_json::Value) -> Option<Contract> {
|
|
||||||
let name = ann["name"].as_str()?;
|
|
||||||
|
|
||||||
match name {
|
|
||||||
"std.string.NonEmpty" => Some(Contract::NonEmpty),
|
|
||||||
"std.string.Email" => Some(Contract::Email),
|
|
||||||
"std.string.Url" => Some(Contract::Url),
|
|
||||||
"std.string.Uuid" => Some(Contract::Uuid),
|
|
||||||
n if n.starts_with("std.string.length.min") => {
|
|
||||||
let min = ann["args"][0].as_u64()? as usize;
|
|
||||||
Some(Contract::MinLength(min))
|
|
||||||
}
|
|
||||||
n if n.starts_with("std.string.length.max") => {
|
|
||||||
let max = ann["args"][0].as_u64()? as usize;
|
|
||||||
Some(Contract::MaxLength(max))
|
|
||||||
}
|
|
||||||
n if n.starts_with("std.number.between") => {
|
|
||||||
let min = ann["args"][0].as_f64()?;
|
|
||||||
let max = ann["args"][1].as_f64()?;
|
|
||||||
Some(Contract::Range { min, max })
|
|
||||||
}
|
|
||||||
n if n.starts_with("std.number.greater_than") => {
|
|
||||||
let min = ann["args"][0].as_f64()?;
|
|
||||||
Some(Contract::GreaterThan(min))
|
|
||||||
}
|
|
||||||
n if n.starts_with("std.number.less_than") => {
|
|
||||||
let max = ann["args"][0].as_f64()?;
|
|
||||||
Some(Contract::LessThan(max))
|
|
||||||
}
|
|
||||||
n if n.starts_with("std.string.match") => {
|
|
||||||
let pattern = ann["args"][0].as_str()?.to_string();
|
|
||||||
Some(Contract::Pattern(pattern))
|
|
||||||
}
|
|
||||||
_ => {
|
|
||||||
// Custom contract - store source code
|
|
||||||
ann["source"]
|
|
||||||
.as_str()
|
|
||||||
.map(|source| Contract::Custom(source.to_string()))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Invalidate cache for a specific schema (for hot-reload)
|
|
||||||
pub async fn invalidate(&self, schema_name: &str) {
|
|
||||||
let mut cache = self.schemas.write().await;
|
|
||||||
cache.remove(schema_name);
|
|
||||||
debug!("Schema {} invalidated from cache", schema_name);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Invalidate all cached schemas
|
|
||||||
pub async fn invalidate_all(&self) {
|
|
||||||
let mut cache = self.schemas.write().await;
|
|
||||||
let count = cache.len();
|
|
||||||
cache.clear();
|
|
||||||
info!("Invalidated {} schemas from cache", count);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Get all cached schema names
|
|
||||||
pub async fn list_cached(&self) -> Vec<String> {
|
|
||||||
let cache = self.schemas.read().await;
|
|
||||||
cache.keys().cloned().collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Check if schema is cached
|
|
||||||
pub async fn is_cached(&self, schema_name: &str) -> bool {
|
|
||||||
let cache = self.schemas.read().await;
|
|
||||||
cache.contains_key(schema_name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_parse_field_type() {
|
|
||||||
let registry = SchemaRegistry::new(PathBuf::from("/tmp"));
|
|
||||||
|
|
||||||
let string_def = serde_json::json!({"type": "String"});
|
|
||||||
let field_type = registry.parse_type(&string_def).unwrap();
|
|
||||||
assert_eq!(field_type, FieldType::String);
|
|
||||||
|
|
||||||
let number_def = serde_json::json!({"type": "Number"});
|
|
||||||
let field_type = registry.parse_type(&number_def).unwrap();
|
|
||||||
assert_eq!(field_type, FieldType::Number);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_parse_contract_annotation() {
|
|
||||||
let registry = SchemaRegistry::new(PathBuf::from("/tmp"));
|
|
||||||
|
|
||||||
let non_empty = serde_json::json!({"name": "std.string.NonEmpty"});
|
|
||||||
let contract = registry.parse_contract_annotation(&non_empty);
|
|
||||||
assert_eq!(contract, Some(Contract::NonEmpty));
|
|
||||||
|
|
||||||
let min_length = serde_json::json!({
|
|
||||||
"name": "std.string.length.min",
|
|
||||||
"args": [5]
|
|
||||||
});
|
|
||||||
let contract = registry.parse_contract_annotation(&min_length);
|
|
||||||
assert_eq!(contract, Some(Contract::MinLength(5)));
|
|
||||||
|
|
||||||
let range = serde_json::json!({
|
|
||||||
"name": "std.number.between",
|
|
||||||
"args": [0.0, 100.0]
|
|
||||||
});
|
|
||||||
let contract = registry.parse_contract_annotation(&range);
|
|
||||||
assert_eq!(
|
|
||||||
contract,
|
|
||||||
Some(Contract::Range {
|
|
||||||
min: 0.0,
|
|
||||||
max: 100.0
|
|
||||||
})
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_schema_source() {
|
|
||||||
let inline_source = SchemaSource::Inline(r#"{ name = "test" }"#.to_string());
|
|
||||||
assert!(matches!(inline_source, SchemaSource::Inline(_)));
|
|
||||||
|
|
||||||
let file_source = SchemaSource::File(PathBuf::from("test.ncl"));
|
|
||||||
assert!(matches!(file_source, SchemaSource::File(_)));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,352 +0,0 @@
|
|||||||
// Integration tests for ValidationPipeline
|
|
||||||
// Demonstrates end-to-end schema validation workflow
|
|
||||||
|
|
||||||
use std::path::PathBuf;
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use serde_json::json;
|
|
||||||
use vapora_shared::validation::{
|
|
||||||
CompiledSchema, Contract, FieldSchema, FieldType, SchemaRegistry, SchemaSource,
|
|
||||||
ValidationPipeline,
|
|
||||||
};
|
|
||||||
|
|
||||||
/// Create a test schema for MCP tool validation
|
|
||||||
fn create_mcp_tool_schema() -> CompiledSchema {
|
|
||||||
CompiledSchema {
|
|
||||||
name: "tools/kanban_create_task".to_string(),
|
|
||||||
fields: vec![
|
|
||||||
FieldSchema {
|
|
||||||
name: "project_id".to_string(),
|
|
||||||
field_type: FieldType::String,
|
|
||||||
required: true,
|
|
||||||
contracts: vec![Contract::NonEmpty, Contract::Uuid],
|
|
||||||
default: None,
|
|
||||||
doc: Some("Project UUID".to_string()),
|
|
||||||
},
|
|
||||||
FieldSchema {
|
|
||||||
name: "title".to_string(),
|
|
||||||
field_type: FieldType::String,
|
|
||||||
required: true,
|
|
||||||
contracts: vec![
|
|
||||||
Contract::NonEmpty,
|
|
||||||
Contract::MinLength(3),
|
|
||||||
Contract::MaxLength(200),
|
|
||||||
],
|
|
||||||
default: None,
|
|
||||||
doc: Some("Task title (3-200 chars)".to_string()),
|
|
||||||
},
|
|
||||||
FieldSchema {
|
|
||||||
name: "description".to_string(),
|
|
||||||
field_type: FieldType::String,
|
|
||||||
required: false,
|
|
||||||
contracts: vec![],
|
|
||||||
default: Some(json!("")),
|
|
||||||
doc: Some("Task description".to_string()),
|
|
||||||
},
|
|
||||||
FieldSchema {
|
|
||||||
name: "priority".to_string(),
|
|
||||||
field_type: FieldType::String,
|
|
||||||
required: true,
|
|
||||||
contracts: vec![],
|
|
||||||
default: None,
|
|
||||||
doc: Some("Task priority".to_string()),
|
|
||||||
},
|
|
||||||
],
|
|
||||||
custom_contracts: vec![],
|
|
||||||
source_path: PathBuf::from("schemas/tools/kanban_create_task.ncl"),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Create a test schema for agent task assignment
|
|
||||||
fn create_agent_assignment_schema() -> CompiledSchema {
|
|
||||||
CompiledSchema {
|
|
||||||
name: "agents/task_assignment".to_string(),
|
|
||||||
fields: vec![
|
|
||||||
FieldSchema {
|
|
||||||
name: "role".to_string(),
|
|
||||||
field_type: FieldType::Enum(vec![
|
|
||||||
"developer".to_string(),
|
|
||||||
"reviewer".to_string(),
|
|
||||||
"architect".to_string(),
|
|
||||||
"tester".to_string(),
|
|
||||||
]),
|
|
||||||
required: true,
|
|
||||||
contracts: vec![],
|
|
||||||
default: None,
|
|
||||||
doc: Some("Agent role".to_string()),
|
|
||||||
},
|
|
||||||
FieldSchema {
|
|
||||||
name: "title".to_string(),
|
|
||||||
field_type: FieldType::String,
|
|
||||||
required: true,
|
|
||||||
contracts: vec![Contract::NonEmpty, Contract::MaxLength(500)],
|
|
||||||
default: None,
|
|
||||||
doc: Some("Task title".to_string()),
|
|
||||||
},
|
|
||||||
FieldSchema {
|
|
||||||
name: "priority".to_string(),
|
|
||||||
field_type: FieldType::Number,
|
|
||||||
required: false,
|
|
||||||
contracts: vec![Contract::Range {
|
|
||||||
min: 0.0,
|
|
||||||
max: 100.0,
|
|
||||||
}],
|
|
||||||
default: Some(json!(50)),
|
|
||||||
doc: Some("Priority score (0-100)".to_string()),
|
|
||||||
},
|
|
||||||
],
|
|
||||||
custom_contracts: vec![],
|
|
||||||
source_path: PathBuf::from("schemas/agents/task_assignment.ncl"),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_mcp_tool_valid_input() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
|
|
||||||
// Load pre-compiled schema
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let source = SchemaSource::Compiled(schema);
|
|
||||||
let loaded_schema = registry
|
|
||||||
.load_from_source(source, "tools/kanban_create_task")
|
|
||||||
.await
|
|
||||||
.unwrap();
|
|
||||||
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"project_id": "550e8400-e29b-41d4-a716-446655440000",
|
|
||||||
"title": "Implement validation pipeline",
|
|
||||||
"priority": "high"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline
|
|
||||||
.validate_against_schema(&loaded_schema, &input)
|
|
||||||
.unwrap();
|
|
||||||
|
|
||||||
assert!(result.valid, "Valid input should pass validation");
|
|
||||||
assert!(result.errors.is_empty());
|
|
||||||
assert!(result.validated_data.is_some());
|
|
||||||
|
|
||||||
// Check defaults applied
|
|
||||||
let validated = result.validated_data.unwrap();
|
|
||||||
assert_eq!(validated["description"], "");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_mcp_tool_missing_required_field() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"title": "Test Task",
|
|
||||||
// Missing project_id (required)
|
|
||||||
"priority": "high"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("project_id"));
|
|
||||||
assert!(result.error_messages()[0].contains("Required field missing"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_mcp_tool_invalid_uuid() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"project_id": "not-a-valid-uuid",
|
|
||||||
"title": "Test Task",
|
|
||||||
"priority": "high"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("project_id"));
|
|
||||||
|
|
||||||
let error_msg = result.error_messages()[0].to_lowercase();
|
|
||||||
assert!(error_msg.contains("uuid") || error_msg.contains("format"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_mcp_tool_title_too_short() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"project_id": "550e8400-e29b-41d4-a716-446655440000",
|
|
||||||
"title": "AB", // Only 2 chars, min is 3
|
|
||||||
"priority": "high"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("title"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_mcp_tool_title_too_long() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let long_title = "A".repeat(201); // Max is 200
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"project_id": "550e8400-e29b-41d4-a716-446655440000",
|
|
||||||
"title": long_title,
|
|
||||||
"priority": "high"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("title"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_agent_assignment_valid_input() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_agent_assignment_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"role": "developer",
|
|
||||||
"title": "Fix authentication bug"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(result.valid);
|
|
||||||
assert!(result.errors.is_empty());
|
|
||||||
|
|
||||||
// Check default priority applied
|
|
||||||
let validated = result.validated_data.unwrap();
|
|
||||||
assert_eq!(validated["priority"], 50);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_agent_assignment_invalid_role() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_agent_assignment_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"role": "invalid_role", // Not in enum
|
|
||||||
"title": "Test Task"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("role"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_agent_assignment_priority_out_of_range() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_agent_assignment_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"role": "developer",
|
|
||||||
"title": "Test Task",
|
|
||||||
"priority": 150 // > 100
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("priority"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_strict_mode_rejects_unknown_fields() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry).with_strict_mode(true);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"project_id": "550e8400-e29b-41d4-a716-446655440000",
|
|
||||||
"title": "Test Task",
|
|
||||||
"priority": "high",
|
|
||||||
"unknown_field": "value" // Not in schema
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
assert!(result.has_field_error("unknown_field"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_non_strict_mode_allows_unknown_fields() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry).with_strict_mode(false);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"project_id": "550e8400-e29b-41d4-a716-446655440000",
|
|
||||||
"title": "Test Task",
|
|
||||||
"priority": "high",
|
|
||||||
"unknown_field": "value" // Allowed in non-strict mode
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
// Should pass, but unknown_field won't be in validated_data
|
|
||||||
assert!(result.valid);
|
|
||||||
let validated = result.validated_data.unwrap();
|
|
||||||
assert!(!validated.as_object().unwrap().contains_key("unknown_field"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_multiple_validation_errors() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"project_id": "invalid-uuid",
|
|
||||||
"title": "AB", // Too short
|
|
||||||
"priority": "high"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
assert!(!result.valid);
|
|
||||||
// Should have errors for both project_id and title
|
|
||||||
assert!(result.errors.len() >= 2);
|
|
||||||
assert!(result.has_field_error("project_id"));
|
|
||||||
assert!(result.has_field_error("title"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_validation_result_serialization() {
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let schema = create_mcp_tool_schema();
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
let input = json!({
|
|
||||||
"title": "", // Empty violates NonEmpty
|
|
||||||
"priority": "high"
|
|
||||||
});
|
|
||||||
|
|
||||||
let result = pipeline.validate_against_schema(&schema, &input).unwrap();
|
|
||||||
|
|
||||||
// Serialize to JSON
|
|
||||||
let json_result = serde_json::to_string(&result).unwrap();
|
|
||||||
assert!(json_result.contains("valid"));
|
|
||||||
assert!(json_result.contains("errors"));
|
|
||||||
|
|
||||||
// Deserialize back
|
|
||||||
let _deserialized: vapora_shared::validation::ValidationResult =
|
|
||||||
serde_json::from_str(&json_result).unwrap();
|
|
||||||
}
|
|
||||||
@ -1,50 +0,0 @@
|
|||||||
[package]
|
|
||||||
name = "vapora-workflow-engine"
|
|
||||||
version.workspace = true
|
|
||||||
edition.workspace = true
|
|
||||||
rust-version.workspace = true
|
|
||||||
authors.workspace = true
|
|
||||||
license.workspace = true
|
|
||||||
repository.workspace = true
|
|
||||||
homepage.workspace = true
|
|
||||||
keywords.workspace = true
|
|
||||||
categories.workspace = true
|
|
||||||
|
|
||||||
[dependencies]
|
|
||||||
vapora-shared = { workspace = true }
|
|
||||||
vapora-swarm = { workspace = true }
|
|
||||||
vapora-agents = { workspace = true }
|
|
||||||
vapora-knowledge-graph = { workspace = true }
|
|
||||||
|
|
||||||
# Async runtime
|
|
||||||
tokio = { workspace = true, features = ["full"] }
|
|
||||||
futures = { workspace = true }
|
|
||||||
async-trait = { workspace = true }
|
|
||||||
|
|
||||||
# Message queue
|
|
||||||
async-nats = { workspace = true }
|
|
||||||
|
|
||||||
# Serialization
|
|
||||||
serde = { workspace = true, features = ["derive"] }
|
|
||||||
serde_json = { workspace = true }
|
|
||||||
toml = { workspace = true }
|
|
||||||
|
|
||||||
# Utilities
|
|
||||||
uuid = { workspace = true, features = ["v4"] }
|
|
||||||
chrono = { workspace = true, features = ["serde"] }
|
|
||||||
dashmap = { workspace = true }
|
|
||||||
|
|
||||||
# Logging
|
|
||||||
tracing = { workspace = true }
|
|
||||||
|
|
||||||
# Error handling
|
|
||||||
thiserror = { workspace = true }
|
|
||||||
anyhow = { workspace = true }
|
|
||||||
|
|
||||||
# Metrics
|
|
||||||
prometheus = { workspace = true }
|
|
||||||
|
|
||||||
[dev-dependencies]
|
|
||||||
mockall = { workspace = true }
|
|
||||||
wiremock = { workspace = true }
|
|
||||||
tokio = { workspace = true, features = ["test-util"] }
|
|
||||||
@ -1,103 +0,0 @@
|
|||||||
use chrono::{DateTime, Utc};
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
||||||
pub struct Artifact {
|
|
||||||
pub id: String,
|
|
||||||
pub artifact_type: ArtifactType,
|
|
||||||
pub producer_stage: String,
|
|
||||||
pub producer_agent: String,
|
|
||||||
pub content_ref: String,
|
|
||||||
pub metadata: serde_json::Value,
|
|
||||||
pub created_at: DateTime<Utc>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
|
||||||
#[serde(rename_all = "snake_case")]
|
|
||||||
pub enum ArtifactType {
|
|
||||||
Adr,
|
|
||||||
Code,
|
|
||||||
TestResults,
|
|
||||||
Review,
|
|
||||||
Documentation,
|
|
||||||
Custom(String),
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Artifact {
|
|
||||||
pub fn new(
|
|
||||||
artifact_type: ArtifactType,
|
|
||||||
producer_stage: String,
|
|
||||||
producer_agent: String,
|
|
||||||
content_ref: String,
|
|
||||||
) -> Self {
|
|
||||||
Self {
|
|
||||||
id: uuid::Uuid::new_v4().to_string(),
|
|
||||||
artifact_type,
|
|
||||||
producer_stage,
|
|
||||||
producer_agent,
|
|
||||||
content_ref,
|
|
||||||
metadata: serde_json::json!({}),
|
|
||||||
created_at: Utc::now(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn with_metadata(mut self, metadata: serde_json::Value) -> Self {
|
|
||||||
self.metadata = metadata;
|
|
||||||
self
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_artifact_creation() {
|
|
||||||
let artifact = Artifact::new(
|
|
||||||
ArtifactType::Code,
|
|
||||||
"implementation".to_string(),
|
|
||||||
"agent-123".to_string(),
|
|
||||||
"kg://artifact/456".to_string(),
|
|
||||||
);
|
|
||||||
|
|
||||||
assert_eq!(artifact.artifact_type, ArtifactType::Code);
|
|
||||||
assert_eq!(artifact.producer_stage, "implementation");
|
|
||||||
assert_eq!(artifact.producer_agent, "agent-123");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_artifact_with_metadata() {
|
|
||||||
let metadata = serde_json::json!({
|
|
||||||
"language": "rust",
|
|
||||||
"files_changed": 5
|
|
||||||
});
|
|
||||||
|
|
||||||
let artifact = Artifact::new(
|
|
||||||
ArtifactType::Code,
|
|
||||||
"implementation".to_string(),
|
|
||||||
"agent-123".to_string(),
|
|
||||||
"kg://artifact/456".to_string(),
|
|
||||||
)
|
|
||||||
.with_metadata(metadata.clone());
|
|
||||||
|
|
||||||
assert_eq!(artifact.metadata, metadata);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_artifact_type_serialization() {
|
|
||||||
let types = vec![
|
|
||||||
ArtifactType::Adr,
|
|
||||||
ArtifactType::Code,
|
|
||||||
ArtifactType::TestResults,
|
|
||||||
ArtifactType::Review,
|
|
||||||
ArtifactType::Documentation,
|
|
||||||
ArtifactType::Custom("benchmark".to_string()),
|
|
||||||
];
|
|
||||||
|
|
||||||
for artifact_type in types {
|
|
||||||
let json = serde_json::to_string(&artifact_type).unwrap();
|
|
||||||
let deserialized: ArtifactType = serde_json::from_str(&json).unwrap();
|
|
||||||
assert_eq!(artifact_type, deserialized);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,186 +0,0 @@
|
|||||||
use std::path::Path;
|
|
||||||
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
|
|
||||||
use crate::error::{ConfigError, Result};
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Deserialize)]
|
|
||||||
pub struct WorkflowsConfig {
|
|
||||||
pub engine: EngineConfig,
|
|
||||||
pub workflows: Vec<WorkflowConfig>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Deserialize)]
|
|
||||||
pub struct EngineConfig {
|
|
||||||
pub max_parallel_tasks: usize,
|
|
||||||
pub workflow_timeout: u64,
|
|
||||||
pub approval_gates_enabled: bool,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Deserialize)]
|
|
||||||
pub struct WorkflowConfig {
|
|
||||||
pub name: String,
|
|
||||||
pub trigger: String,
|
|
||||||
pub stages: Vec<StageConfig>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
||||||
pub struct StageConfig {
|
|
||||||
pub name: String,
|
|
||||||
pub agents: Vec<String>,
|
|
||||||
#[serde(default)]
|
|
||||||
pub parallel: bool,
|
|
||||||
#[serde(default)]
|
|
||||||
pub max_parallel: Option<usize>,
|
|
||||||
#[serde(default)]
|
|
||||||
pub approval_required: bool,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl WorkflowsConfig {
|
|
||||||
pub fn load<P: AsRef<Path>>(path: P) -> Result<Self> {
|
|
||||||
let content = std::fs::read_to_string(path).map_err(ConfigError::IoError)?;
|
|
||||||
let config: WorkflowsConfig = toml::from_str(&content).map_err(ConfigError::Parse)?;
|
|
||||||
|
|
||||||
config.validate()?;
|
|
||||||
|
|
||||||
Ok(config)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn validate(&self) -> Result<()> {
|
|
||||||
if self.workflows.is_empty() {
|
|
||||||
return Err(ConfigError::Invalid("No workflows defined".to_string()).into());
|
|
||||||
}
|
|
||||||
|
|
||||||
for workflow in &self.workflows {
|
|
||||||
if workflow.stages.is_empty() {
|
|
||||||
return Err(ConfigError::Invalid(format!(
|
|
||||||
"Workflow '{}' has no stages",
|
|
||||||
workflow.name
|
|
||||||
))
|
|
||||||
.into());
|
|
||||||
}
|
|
||||||
|
|
||||||
for stage in &workflow.stages {
|
|
||||||
if stage.agents.is_empty() {
|
|
||||||
return Err(ConfigError::Invalid(format!(
|
|
||||||
"Stage '{}' in workflow '{}' has no agents",
|
|
||||||
stage.name, workflow.name
|
|
||||||
))
|
|
||||||
.into());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn get_workflow(&self, name: &str) -> Option<&WorkflowConfig> {
|
|
||||||
self.workflows.iter().find(|w| w.name == name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_parse_workflow_config() {
|
|
||||||
let toml_str = r#"
|
|
||||||
[engine]
|
|
||||||
max_parallel_tasks = 10
|
|
||||||
workflow_timeout = 3600
|
|
||||||
approval_gates_enabled = true
|
|
||||||
|
|
||||||
[[workflows]]
|
|
||||||
name = "test_workflow"
|
|
||||||
trigger = "manual"
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "design"
|
|
||||||
agents = ["architect"]
|
|
||||||
parallel = false
|
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "implementation"
|
|
||||||
agents = ["developer", "developer"]
|
|
||||||
parallel = true
|
|
||||||
max_parallel = 2
|
|
||||||
approval_required = false
|
|
||||||
"#;
|
|
||||||
|
|
||||||
let config: WorkflowsConfig = toml::from_str(toml_str).unwrap();
|
|
||||||
|
|
||||||
assert_eq!(config.engine.max_parallel_tasks, 10);
|
|
||||||
assert_eq!(config.workflows.len(), 1);
|
|
||||||
assert_eq!(config.workflows[0].name, "test_workflow");
|
|
||||||
assert_eq!(config.workflows[0].stages.len(), 2);
|
|
||||||
assert_eq!(config.workflows[0].stages[0].name, "design");
|
|
||||||
assert!(!config.workflows[0].stages[0].parallel);
|
|
||||||
assert!(config.workflows[0].stages[1].parallel);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_validation_no_workflows() {
|
|
||||||
let config = WorkflowsConfig {
|
|
||||||
engine: EngineConfig {
|
|
||||||
max_parallel_tasks: 10,
|
|
||||||
workflow_timeout: 3600,
|
|
||||||
approval_gates_enabled: true,
|
|
||||||
},
|
|
||||||
workflows: vec![],
|
|
||||||
};
|
|
||||||
|
|
||||||
assert!(config.validate().is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_validation_no_stages() {
|
|
||||||
let toml_str = r#"
|
|
||||||
[engine]
|
|
||||||
max_parallel_tasks = 10
|
|
||||||
workflow_timeout = 3600
|
|
||||||
approval_gates_enabled = true
|
|
||||||
|
|
||||||
[[workflows]]
|
|
||||||
name = "test_workflow"
|
|
||||||
trigger = "manual"
|
|
||||||
stages = []
|
|
||||||
"#;
|
|
||||||
|
|
||||||
let config: WorkflowsConfig = toml::from_str(toml_str).unwrap();
|
|
||||||
assert!(config.validate().is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_get_workflow() {
|
|
||||||
let toml_str = r#"
|
|
||||||
[engine]
|
|
||||||
max_parallel_tasks = 10
|
|
||||||
workflow_timeout = 3600
|
|
||||||
approval_gates_enabled = true
|
|
||||||
|
|
||||||
[[workflows]]
|
|
||||||
name = "workflow_a"
|
|
||||||
trigger = "manual"
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "stage1"
|
|
||||||
agents = ["agent1"]
|
|
||||||
|
|
||||||
[[workflows]]
|
|
||||||
name = "workflow_b"
|
|
||||||
trigger = "manual"
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "stage2"
|
|
||||||
agents = ["agent2"]
|
|
||||||
"#;
|
|
||||||
|
|
||||||
let config: WorkflowsConfig = toml::from_str(toml_str).unwrap();
|
|
||||||
|
|
||||||
assert!(config.get_workflow("workflow_a").is_some());
|
|
||||||
assert!(config.get_workflow("workflow_b").is_some());
|
|
||||||
assert!(config.get_workflow("nonexistent").is_none());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,60 +0,0 @@
|
|||||||
use thiserror::Error;
|
|
||||||
|
|
||||||
#[derive(Error, Debug)]
|
|
||||||
pub enum WorkflowError {
|
|
||||||
#[error("Workflow not found: {0}")]
|
|
||||||
WorkflowNotFound(String),
|
|
||||||
|
|
||||||
#[error("Configuration error: {0}")]
|
|
||||||
ConfigError(#[from] ConfigError),
|
|
||||||
|
|
||||||
#[error("Invalid state transition: {from:?} -> {to:?}")]
|
|
||||||
InvalidTransition { from: String, to: String },
|
|
||||||
|
|
||||||
#[error("No current stage available")]
|
|
||||||
NoCurrentStage,
|
|
||||||
|
|
||||||
#[error("No agents configured for stage")]
|
|
||||||
NoAgentsInStage,
|
|
||||||
|
|
||||||
#[error("Task not found: {0}")]
|
|
||||||
TaskNotFound(String),
|
|
||||||
|
|
||||||
#[error("Stage not waiting for approval")]
|
|
||||||
NotWaitingApproval,
|
|
||||||
|
|
||||||
#[error("Swarm coordination error: {0}")]
|
|
||||||
SwarmError(String),
|
|
||||||
|
|
||||||
#[error("NATS messaging error: {0}")]
|
|
||||||
NatsError(Box<dyn std::error::Error + Send + Sync>),
|
|
||||||
|
|
||||||
#[error("Knowledge graph error: {0}")]
|
|
||||||
KnowledgeGraphError(String),
|
|
||||||
|
|
||||||
#[error("Serialization error: {0}")]
|
|
||||||
SerializationError(#[from] serde_json::Error),
|
|
||||||
|
|
||||||
#[error("IO error: {0}")]
|
|
||||||
IoError(#[from] std::io::Error),
|
|
||||||
|
|
||||||
#[error("Artifact persistence failed: {0}")]
|
|
||||||
ArtifactError(String),
|
|
||||||
|
|
||||||
#[error("Internal error: {0}")]
|
|
||||||
Internal(String),
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Error, Debug)]
|
|
||||||
pub enum ConfigError {
|
|
||||||
#[error("Failed to read config file: {0}")]
|
|
||||||
IoError(#[from] std::io::Error),
|
|
||||||
|
|
||||||
#[error("Failed to parse TOML: {0}")]
|
|
||||||
Parse(#[from] toml::de::Error),
|
|
||||||
|
|
||||||
#[error("Invalid configuration: {0}")]
|
|
||||||
Invalid(String),
|
|
||||||
}
|
|
||||||
|
|
||||||
pub type Result<T> = std::result::Result<T, WorkflowError>;
|
|
||||||
@ -1,321 +0,0 @@
|
|||||||
use std::collections::HashMap;
|
|
||||||
|
|
||||||
use chrono::{DateTime, Utc};
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use serde_json::Value;
|
|
||||||
|
|
||||||
use crate::artifact::Artifact;
|
|
||||||
use crate::config::WorkflowConfig;
|
|
||||||
use crate::error::Result;
|
|
||||||
use crate::stage::{StageState, StageStatus};
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
|
||||||
#[serde(rename_all = "snake_case")]
|
|
||||||
pub enum WorkflowStatus {
|
|
||||||
Running,
|
|
||||||
WaitingApproval(usize),
|
|
||||||
Completed,
|
|
||||||
Failed(String),
|
|
||||||
Cancelled,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
||||||
pub struct WorkflowInstance {
|
|
||||||
pub id: String,
|
|
||||||
pub template_name: String,
|
|
||||||
pub status: WorkflowStatus,
|
|
||||||
pub stages: Vec<StageState>,
|
|
||||||
pub current_stage_idx: usize,
|
|
||||||
pub initial_context: Value,
|
|
||||||
pub accumulated_artifacts: HashMap<String, Artifact>,
|
|
||||||
pub created_at: DateTime<Utc>,
|
|
||||||
pub updated_at: DateTime<Utc>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl WorkflowInstance {
|
|
||||||
pub fn new(template: &WorkflowConfig, initial_context: Value) -> Self {
|
|
||||||
let stages = template
|
|
||||||
.stages
|
|
||||||
.iter()
|
|
||||||
.map(|sc| StageState::new(sc.clone()))
|
|
||||||
.collect();
|
|
||||||
|
|
||||||
Self {
|
|
||||||
id: uuid::Uuid::new_v4().to_string(),
|
|
||||||
template_name: template.name.clone(),
|
|
||||||
status: WorkflowStatus::Running,
|
|
||||||
stages,
|
|
||||||
current_stage_idx: 0,
|
|
||||||
initial_context,
|
|
||||||
accumulated_artifacts: HashMap::new(),
|
|
||||||
created_at: Utc::now(),
|
|
||||||
updated_at: Utc::now(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn current_stage(&self) -> Option<&StageState> {
|
|
||||||
self.stages.get(self.current_stage_idx)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn current_stage_mut(&mut self) -> Option<&mut StageState> {
|
|
||||||
self.stages.get_mut(self.current_stage_idx)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn advance_stage(&mut self) -> Result<bool> {
|
|
||||||
if let Some(stage) = self.current_stage() {
|
|
||||||
for artifact_id in stage.collect_artifacts() {
|
|
||||||
if let Some(artifact) = self.accumulated_artifacts.get(&artifact_id) {
|
|
||||||
tracing::debug!(
|
|
||||||
artifact_id = %artifact_id,
|
|
||||||
artifact_type = ?artifact.artifact_type,
|
|
||||||
"Artifact collected from stage"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
self.current_stage_idx += 1;
|
|
||||||
self.updated_at = Utc::now();
|
|
||||||
|
|
||||||
if self.current_stage_idx >= self.stages.len() {
|
|
||||||
self.status = WorkflowStatus::Completed;
|
|
||||||
Ok(false)
|
|
||||||
} else {
|
|
||||||
Ok(true)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn build_stage_context(&self) -> Value {
|
|
||||||
let mut ctx = self.initial_context.clone();
|
|
||||||
|
|
||||||
if let Some(obj) = ctx.as_object_mut() {
|
|
||||||
obj.insert(
|
|
||||||
"artifacts".to_string(),
|
|
||||||
serde_json::to_value(&self.accumulated_artifacts).unwrap_or_default(),
|
|
||||||
);
|
|
||||||
obj.insert("workflow_id".to_string(), Value::String(self.id.clone()));
|
|
||||||
obj.insert(
|
|
||||||
"stage_idx".to_string(),
|
|
||||||
Value::Number(self.current_stage_idx.into()),
|
|
||||||
);
|
|
||||||
|
|
||||||
if let Some(stage) = self.current_stage() {
|
|
||||||
obj.insert("stage_name".to_string(), Value::String(stage.name.clone()));
|
|
||||||
}
|
|
||||||
|
|
||||||
obj.insert(
|
|
||||||
"previous_stages".to_string(),
|
|
||||||
serde_json::to_value(
|
|
||||||
self.stages[..self.current_stage_idx]
|
|
||||||
.iter()
|
|
||||||
.map(|s| {
|
|
||||||
serde_json::json!({
|
|
||||||
"name": s.name,
|
|
||||||
"status": s.status,
|
|
||||||
"artifacts": s.artifacts_produced,
|
|
||||||
})
|
|
||||||
})
|
|
||||||
.collect::<Vec<_>>(),
|
|
||||||
)
|
|
||||||
.unwrap_or_default(),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
ctx
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn add_artifact(&mut self, artifact: Artifact) {
|
|
||||||
let artifact_id = artifact.id.clone();
|
|
||||||
self.accumulated_artifacts
|
|
||||||
.insert(artifact_id.clone(), artifact);
|
|
||||||
|
|
||||||
if let Some(stage) = self.current_stage_mut() {
|
|
||||||
if !stage.artifacts_produced.contains(&artifact_id) {
|
|
||||||
stage.artifacts_produced.push(artifact_id);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
self.updated_at = Utc::now();
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn is_completed(&self) -> bool {
|
|
||||||
matches!(self.status, WorkflowStatus::Completed)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn is_failed(&self) -> bool {
|
|
||||||
matches!(self.status, WorkflowStatus::Failed(_))
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn is_cancelled(&self) -> bool {
|
|
||||||
matches!(self.status, WorkflowStatus::Cancelled)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn is_waiting_approval(&self) -> bool {
|
|
||||||
matches!(self.status, WorkflowStatus::WaitingApproval(_))
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn cancel(&mut self, reason: String) {
|
|
||||||
self.status = WorkflowStatus::Cancelled;
|
|
||||||
self.updated_at = Utc::now();
|
|
||||||
|
|
||||||
if let Some(stage) = self.current_stage_mut() {
|
|
||||||
let _ = stage.transition(StageStatus::Failed(reason));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn fail(&mut self, error: String) {
|
|
||||||
self.status = WorkflowStatus::Failed(error.clone());
|
|
||||||
self.updated_at = Utc::now();
|
|
||||||
|
|
||||||
if let Some(stage) = self.current_stage_mut() {
|
|
||||||
let _ = stage.transition(StageStatus::Failed(error));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use crate::artifact::ArtifactType;
|
|
||||||
use crate::config::StageConfig;
|
|
||||||
|
|
||||||
fn create_test_workflow_config() -> WorkflowConfig {
|
|
||||||
WorkflowConfig {
|
|
||||||
name: "test_workflow".to_string(),
|
|
||||||
trigger: "manual".to_string(),
|
|
||||||
stages: vec![
|
|
||||||
StageConfig {
|
|
||||||
name: "stage1".to_string(),
|
|
||||||
agents: vec!["agent1".to_string()],
|
|
||||||
parallel: false,
|
|
||||||
max_parallel: None,
|
|
||||||
approval_required: false,
|
|
||||||
},
|
|
||||||
StageConfig {
|
|
||||||
name: "stage2".to_string(),
|
|
||||||
agents: vec!["agent2".to_string()],
|
|
||||||
parallel: false,
|
|
||||||
max_parallel: None,
|
|
||||||
approval_required: false,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_workflow_instance_creation() {
|
|
||||||
let config = create_test_workflow_config();
|
|
||||||
let context = serde_json::json!({"key": "value"});
|
|
||||||
let instance = WorkflowInstance::new(&config, context);
|
|
||||||
|
|
||||||
assert_eq!(instance.template_name, "test_workflow");
|
|
||||||
assert_eq!(instance.status, WorkflowStatus::Running);
|
|
||||||
assert_eq!(instance.stages.len(), 2);
|
|
||||||
assert_eq!(instance.current_stage_idx, 0);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_current_stage() {
|
|
||||||
let config = create_test_workflow_config();
|
|
||||||
let context = serde_json::json!({});
|
|
||||||
let instance = WorkflowInstance::new(&config, context);
|
|
||||||
|
|
||||||
let current = instance.current_stage().unwrap();
|
|
||||||
assert_eq!(current.name, "stage1");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_advance_stage() {
|
|
||||||
let config = create_test_workflow_config();
|
|
||||||
let context = serde_json::json!({});
|
|
||||||
let mut instance = WorkflowInstance::new(&config, context);
|
|
||||||
|
|
||||||
let has_more = instance.advance_stage().unwrap();
|
|
||||||
assert!(has_more);
|
|
||||||
assert_eq!(instance.current_stage_idx, 1);
|
|
||||||
assert_eq!(instance.current_stage().unwrap().name, "stage2");
|
|
||||||
|
|
||||||
let has_more = instance.advance_stage().unwrap();
|
|
||||||
assert!(!has_more);
|
|
||||||
assert_eq!(instance.status, WorkflowStatus::Completed);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_build_stage_context() {
|
|
||||||
let config = create_test_workflow_config();
|
|
||||||
let initial_context = serde_json::json!({"initial": "data"});
|
|
||||||
let instance = WorkflowInstance::new(&config, initial_context);
|
|
||||||
|
|
||||||
let ctx = instance.build_stage_context();
|
|
||||||
assert_eq!(ctx["initial"], "data");
|
|
||||||
assert_eq!(ctx["workflow_id"], instance.id);
|
|
||||||
assert_eq!(ctx["stage_idx"], 0);
|
|
||||||
assert_eq!(ctx["stage_name"], "stage1");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_add_artifact() {
|
|
||||||
let config = create_test_workflow_config();
|
|
||||||
let context = serde_json::json!({});
|
|
||||||
let mut instance = WorkflowInstance::new(&config, context);
|
|
||||||
|
|
||||||
let artifact = Artifact::new(
|
|
||||||
ArtifactType::Code,
|
|
||||||
"stage1".to_string(),
|
|
||||||
"agent1".to_string(),
|
|
||||||
"content_ref".to_string(),
|
|
||||||
);
|
|
||||||
|
|
||||||
let artifact_id = artifact.id.clone();
|
|
||||||
instance.add_artifact(artifact);
|
|
||||||
|
|
||||||
assert!(instance.accumulated_artifacts.contains_key(&artifact_id));
|
|
||||||
assert!(instance
|
|
||||||
.current_stage()
|
|
||||||
.unwrap()
|
|
||||||
.artifacts_produced
|
|
||||||
.contains(&artifact_id));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_workflow_status_checks() {
|
|
||||||
let config = create_test_workflow_config();
|
|
||||||
let context = serde_json::json!({});
|
|
||||||
let mut instance = WorkflowInstance::new(&config, context);
|
|
||||||
|
|
||||||
assert!(!instance.is_completed());
|
|
||||||
assert!(!instance.is_failed());
|
|
||||||
assert!(!instance.is_cancelled());
|
|
||||||
|
|
||||||
instance.status = WorkflowStatus::Completed;
|
|
||||||
assert!(instance.is_completed());
|
|
||||||
|
|
||||||
instance.status = WorkflowStatus::Failed("error".to_string());
|
|
||||||
assert!(instance.is_failed());
|
|
||||||
|
|
||||||
instance.status = WorkflowStatus::Cancelled;
|
|
||||||
assert!(instance.is_cancelled());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_cancel_workflow() {
|
|
||||||
let config = create_test_workflow_config();
|
|
||||||
let context = serde_json::json!({});
|
|
||||||
let mut instance = WorkflowInstance::new(&config, context);
|
|
||||||
|
|
||||||
instance.cancel("User requested".to_string());
|
|
||||||
|
|
||||||
assert!(instance.is_cancelled());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_fail_workflow() {
|
|
||||||
let config = create_test_workflow_config();
|
|
||||||
let context = serde_json::json!({});
|
|
||||||
let mut instance = WorkflowInstance::new(&config, context);
|
|
||||||
|
|
||||||
instance.fail("Task execution failed".to_string());
|
|
||||||
|
|
||||||
assert!(instance.is_failed());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,73 +0,0 @@
|
|||||||
//! Vapora Workflow Engine
|
|
||||||
//!
|
|
||||||
//! Orchestrates multi-stage workflows with learning-based agent selection,
|
|
||||||
//! artifact passing between stages, and cost-aware LLM routing.
|
|
||||||
//!
|
|
||||||
//! # Architecture
|
|
||||||
//!
|
|
||||||
//! The workflow engine uses a state machine approach where each
|
|
||||||
//! WorkflowInstance tracks progress through stages. Each stage can execute
|
|
||||||
//! tasks in parallel or sequentially, with artifacts passed between stages via
|
|
||||||
//! the Knowledge Graph.
|
|
||||||
//!
|
|
||||||
//! # Key Components
|
|
||||||
//!
|
|
||||||
//! - `WorkflowOrchestrator`: Main coordinator managing workflow lifecycle
|
|
||||||
//! - `WorkflowInstance`: State machine tracking individual workflow execution
|
|
||||||
//! - `StageState`: Manages stage execution and task assignment
|
|
||||||
//! - `Artifact`: Data passed between stages
|
|
||||||
//!
|
|
||||||
//! # Example
|
|
||||||
//!
|
|
||||||
//! ```no_run
|
|
||||||
//! use vapora_workflow_engine::{WorkflowOrchestrator, config::WorkflowsConfig};
|
|
||||||
//! use std::sync::Arc;
|
|
||||||
//!
|
|
||||||
//! # async fn example() -> Result<(), Box<dyn std::error::Error>> {
|
|
||||||
//! // Initialize dependencies (SwarmCoordinator, KGPersistence, NATS)
|
|
||||||
//! # let swarm = todo!();
|
|
||||||
//! # let kg = todo!();
|
|
||||||
//! # let nats = todo!();
|
|
||||||
//!
|
|
||||||
//! // Create orchestrator
|
|
||||||
//! let orchestrator = Arc::new(
|
|
||||||
//! WorkflowOrchestrator::new(
|
|
||||||
//! "config/workflows.toml",
|
|
||||||
//! swarm,
|
|
||||||
//! kg,
|
|
||||||
//! nats,
|
|
||||||
//! ).await?
|
|
||||||
//! );
|
|
||||||
//!
|
|
||||||
//! // Start event listener
|
|
||||||
//! orchestrator.clone().start_event_listener().await?;
|
|
||||||
//!
|
|
||||||
//! // Start a workflow
|
|
||||||
//! let workflow_id = orchestrator.start_workflow(
|
|
||||||
//! "feature_development",
|
|
||||||
//! serde_json::json!({
|
|
||||||
//! "task": "Add authentication",
|
|
||||||
//! "requirements": ["OAuth2", "JWT"]
|
|
||||||
//! })
|
|
||||||
//! ).await?;
|
|
||||||
//!
|
|
||||||
//! println!("Workflow started: {}", workflow_id);
|
|
||||||
//! # Ok(())
|
|
||||||
//! # }
|
|
||||||
//! ```
|
|
||||||
|
|
||||||
pub mod artifact;
|
|
||||||
pub mod config;
|
|
||||||
pub mod error;
|
|
||||||
pub mod instance;
|
|
||||||
pub mod metrics;
|
|
||||||
pub mod orchestrator;
|
|
||||||
pub mod stage;
|
|
||||||
|
|
||||||
pub use artifact::{Artifact, ArtifactType};
|
|
||||||
pub use config::{EngineConfig, StageConfig, WorkflowConfig, WorkflowsConfig};
|
|
||||||
pub use error::{ConfigError, Result, WorkflowError};
|
|
||||||
pub use instance::{WorkflowInstance, WorkflowStatus};
|
|
||||||
pub use metrics::WorkflowMetrics;
|
|
||||||
pub use orchestrator::WorkflowOrchestrator;
|
|
||||||
pub use stage::{StageState, StageStatus, TaskState, TaskStatus};
|
|
||||||
@ -1,97 +0,0 @@
|
|||||||
use prometheus::{
|
|
||||||
register_counter, register_int_gauge, Counter, Histogram, HistogramOpts, IntGauge, Registry,
|
|
||||||
};
|
|
||||||
|
|
||||||
pub struct WorkflowMetrics {
|
|
||||||
pub workflows_started: Counter,
|
|
||||||
pub workflows_completed: Counter,
|
|
||||||
pub workflows_failed: Counter,
|
|
||||||
pub stages_completed: Counter,
|
|
||||||
pub active_workflows: IntGauge,
|
|
||||||
pub stage_duration_seconds: Histogram,
|
|
||||||
pub workflow_duration_seconds: Histogram,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl WorkflowMetrics {
|
|
||||||
pub fn new() -> Result<Self, prometheus::Error> {
|
|
||||||
Ok(Self {
|
|
||||||
workflows_started: register_counter!(
|
|
||||||
"vapora_workflows_started_total",
|
|
||||||
"Total workflows started"
|
|
||||||
)?,
|
|
||||||
workflows_completed: register_counter!(
|
|
||||||
"vapora_workflows_completed_total",
|
|
||||||
"Total workflows completed successfully"
|
|
||||||
)?,
|
|
||||||
workflows_failed: register_counter!(
|
|
||||||
"vapora_workflows_failed_total",
|
|
||||||
"Total workflows failed"
|
|
||||||
)?,
|
|
||||||
stages_completed: register_counter!(
|
|
||||||
"vapora_stages_completed_total",
|
|
||||||
"Total stages completed"
|
|
||||||
)?,
|
|
||||||
active_workflows: register_int_gauge!(
|
|
||||||
"vapora_active_workflows",
|
|
||||||
"Currently active workflows"
|
|
||||||
)?,
|
|
||||||
stage_duration_seconds: Histogram::with_opts(
|
|
||||||
HistogramOpts::new("vapora_stage_duration_seconds", "Stage execution duration")
|
|
||||||
.buckets(vec![1.0, 5.0, 15.0, 30.0, 60.0, 120.0, 300.0]),
|
|
||||||
)?,
|
|
||||||
workflow_duration_seconds: Histogram::with_opts(
|
|
||||||
HistogramOpts::new(
|
|
||||||
"vapora_workflow_duration_seconds",
|
|
||||||
"Workflow total duration",
|
|
||||||
)
|
|
||||||
.buckets(vec![60.0, 300.0, 600.0, 1800.0, 3600.0]),
|
|
||||||
)?,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn register(&self, registry: &Registry) -> Result<(), prometheus::Error> {
|
|
||||||
registry.register(Box::new(self.workflows_started.clone()))?;
|
|
||||||
registry.register(Box::new(self.workflows_completed.clone()))?;
|
|
||||||
registry.register(Box::new(self.workflows_failed.clone()))?;
|
|
||||||
registry.register(Box::new(self.stages_completed.clone()))?;
|
|
||||||
registry.register(Box::new(self.active_workflows.clone()))?;
|
|
||||||
registry.register(Box::new(self.stage_duration_seconds.clone()))?;
|
|
||||||
registry.register(Box::new(self.workflow_duration_seconds.clone()))?;
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Default for WorkflowMetrics {
|
|
||||||
fn default() -> Self {
|
|
||||||
Self::new().expect("Failed to create workflow metrics")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use prometheus::{Counter, Histogram, HistogramOpts, IntGauge};
|
|
||||||
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_metrics_operations() {
|
|
||||||
let counter = Counter::new("test_counter", "test").unwrap();
|
|
||||||
counter.inc();
|
|
||||||
assert_eq!(counter.get(), 1.0);
|
|
||||||
|
|
||||||
let gauge = IntGauge::new("test_gauge", "test").unwrap();
|
|
||||||
gauge.set(5);
|
|
||||||
assert_eq!(gauge.get(), 5);
|
|
||||||
|
|
||||||
let histogram = Histogram::with_opts(HistogramOpts::new("test_hist", "test")).unwrap();
|
|
||||||
histogram.observe(10.5);
|
|
||||||
histogram.observe(25.3);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_metrics_registry() {
|
|
||||||
let registry = Registry::new();
|
|
||||||
let counter = Counter::new("workflow_test", "test").unwrap();
|
|
||||||
assert!(registry.register(Box::new(counter)).is_ok());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,705 +0,0 @@
|
|||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use chrono::Utc;
|
|
||||||
use dashmap::DashMap;
|
|
||||||
use futures::StreamExt;
|
|
||||||
use serde_json::Value;
|
|
||||||
use tracing::{debug, error, info, warn};
|
|
||||||
use vapora_agents::messages::{AgentMessage, TaskCompleted, TaskFailed};
|
|
||||||
use vapora_knowledge_graph::persistence::KGPersistence;
|
|
||||||
use vapora_swarm::coordinator::SwarmCoordinator;
|
|
||||||
|
|
||||||
use crate::artifact::{Artifact, ArtifactType};
|
|
||||||
use crate::config::{StageConfig, WorkflowsConfig};
|
|
||||||
use crate::error::{Result, WorkflowError};
|
|
||||||
use crate::instance::{WorkflowInstance, WorkflowStatus};
|
|
||||||
use crate::metrics::WorkflowMetrics;
|
|
||||||
use crate::stage::{StageStatus, TaskState};
|
|
||||||
|
|
||||||
pub struct WorkflowOrchestrator {
|
|
||||||
config: WorkflowsConfig,
|
|
||||||
swarm: Arc<SwarmCoordinator>,
|
|
||||||
#[allow(dead_code)]
|
|
||||||
kg: Arc<KGPersistence>,
|
|
||||||
nats: Arc<async_nats::Client>,
|
|
||||||
active_workflows: DashMap<String, WorkflowInstance>,
|
|
||||||
metrics: Arc<WorkflowMetrics>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl WorkflowOrchestrator {
|
|
||||||
pub async fn new(
|
|
||||||
config_path: &str,
|
|
||||||
swarm: Arc<SwarmCoordinator>,
|
|
||||||
kg: Arc<KGPersistence>,
|
|
||||||
nats: Arc<async_nats::Client>,
|
|
||||||
) -> Result<Self> {
|
|
||||||
let config = WorkflowsConfig::load(config_path)?;
|
|
||||||
let metrics =
|
|
||||||
Arc::new(WorkflowMetrics::new().map_err(|e| {
|
|
||||||
WorkflowError::Internal(format!("Failed to create metrics: {}", e))
|
|
||||||
})?);
|
|
||||||
|
|
||||||
Ok(Self {
|
|
||||||
config,
|
|
||||||
swarm,
|
|
||||||
kg,
|
|
||||||
nats,
|
|
||||||
active_workflows: DashMap::new(),
|
|
||||||
metrics,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn metrics(&self) -> Arc<WorkflowMetrics> {
|
|
||||||
self.metrics.clone()
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn start_workflow(
|
|
||||||
&self,
|
|
||||||
workflow_name: &str,
|
|
||||||
initial_context: Value,
|
|
||||||
) -> Result<String> {
|
|
||||||
let template = self
|
|
||||||
.config
|
|
||||||
.get_workflow(workflow_name)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_name.to_string()))?;
|
|
||||||
|
|
||||||
let instance = WorkflowInstance::new(template, initial_context);
|
|
||||||
let workflow_id = instance.id.clone();
|
|
||||||
|
|
||||||
self.active_workflows.insert(workflow_id.clone(), instance);
|
|
||||||
self.metrics.active_workflows.inc();
|
|
||||||
|
|
||||||
self.execute_current_stage(&workflow_id).await?;
|
|
||||||
|
|
||||||
info!(workflow_id = %workflow_id, "Workflow started: {}", workflow_name);
|
|
||||||
self.metrics.workflows_started.inc();
|
|
||||||
|
|
||||||
Ok(workflow_id)
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn execute_current_stage(&self, workflow_id: &str) -> Result<()> {
|
|
||||||
let (stage_config, approval_required, context) = {
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
|
|
||||||
let stage_config = instance
|
|
||||||
.current_stage()
|
|
||||||
.ok_or(WorkflowError::NoCurrentStage)?
|
|
||||||
.config
|
|
||||||
.clone();
|
|
||||||
|
|
||||||
let stage_name = stage_config.name.clone();
|
|
||||||
|
|
||||||
if stage_config.approval_required && !instance.is_waiting_approval() {
|
|
||||||
instance.status = WorkflowStatus::WaitingApproval(instance.current_stage_idx);
|
|
||||||
drop(instance);
|
|
||||||
return self
|
|
||||||
.publish_approval_request(workflow_id, &stage_name)
|
|
||||||
.await;
|
|
||||||
}
|
|
||||||
|
|
||||||
let context = instance.build_stage_context();
|
|
||||||
|
|
||||||
(
|
|
||||||
stage_config.clone(),
|
|
||||||
stage_config.approval_required,
|
|
||||||
context,
|
|
||||||
)
|
|
||||||
};
|
|
||||||
|
|
||||||
if approval_required {
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
|
|
||||||
if stage_config.parallel {
|
|
||||||
self.assign_parallel_tasks(workflow_id, &stage_config, context)
|
|
||||||
.await?;
|
|
||||||
} else {
|
|
||||||
self.assign_sequential_task(workflow_id, &stage_config, context)
|
|
||||||
.await?;
|
|
||||||
}
|
|
||||||
|
|
||||||
{
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
|
|
||||||
if let Some(stage) = instance.current_stage_mut() {
|
|
||||||
stage.transition(StageStatus::Running)?;
|
|
||||||
stage.mark_started();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn assign_parallel_tasks(
|
|
||||||
&self,
|
|
||||||
workflow_id: &str,
|
|
||||||
stage_config: &StageConfig,
|
|
||||||
_context: Value,
|
|
||||||
) -> Result<()> {
|
|
||||||
let max_parallel = stage_config
|
|
||||||
.max_parallel
|
|
||||||
.unwrap_or(stage_config.agents.len());
|
|
||||||
|
|
||||||
for (i, agent_role) in stage_config.agents.iter().take(max_parallel).enumerate() {
|
|
||||||
let task_description = format!("{} - task {}", stage_config.name, i);
|
|
||||||
|
|
||||||
let assigned_agent = self
|
|
||||||
.swarm
|
|
||||||
.submit_task_for_bidding(
|
|
||||||
format!("wf-{}-stage-{}-{}", workflow_id, stage_config.name, i),
|
|
||||||
task_description.clone(),
|
|
||||||
vec![agent_role.clone()],
|
|
||||||
)
|
|
||||||
.await
|
|
||||||
.map_err(|e| WorkflowError::SwarmError(e.to_string()))?;
|
|
||||||
|
|
||||||
if let Some(agent_id) = assigned_agent {
|
|
||||||
let task_id = format!("wf-{}-stage-{}-{}", workflow_id, stage_config.name, i);
|
|
||||||
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
|
|
||||||
if let Some(stage) = instance.current_stage_mut() {
|
|
||||||
stage.assigned_tasks.insert(
|
|
||||||
task_id.clone(),
|
|
||||||
TaskState::new(task_id.clone(), agent_id.clone(), agent_role.clone()),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
debug!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
stage = %stage_config.name,
|
|
||||||
task_id = %task_id,
|
|
||||||
agent_id = %agent_id,
|
|
||||||
"Task assigned in parallel mode"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn assign_sequential_task(
|
|
||||||
&self,
|
|
||||||
workflow_id: &str,
|
|
||||||
stage_config: &StageConfig,
|
|
||||||
_context: Value,
|
|
||||||
) -> Result<()> {
|
|
||||||
let agent_role = stage_config
|
|
||||||
.agents
|
|
||||||
.first()
|
|
||||||
.ok_or(WorkflowError::NoAgentsInStage)?;
|
|
||||||
|
|
||||||
let task_description = format!("Workflow: {} - Stage: {}", workflow_id, stage_config.name);
|
|
||||||
|
|
||||||
let assigned_agent = self
|
|
||||||
.swarm
|
|
||||||
.submit_task_for_bidding(
|
|
||||||
format!("wf-{}-stage-{}", workflow_id, stage_config.name),
|
|
||||||
task_description,
|
|
||||||
vec![agent_role.clone()],
|
|
||||||
)
|
|
||||||
.await
|
|
||||||
.map_err(|e| WorkflowError::SwarmError(e.to_string()))?;
|
|
||||||
|
|
||||||
if let Some(agent_id) = assigned_agent {
|
|
||||||
let task_id = format!("wf-{}-stage-{}", workflow_id, stage_config.name);
|
|
||||||
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
|
|
||||||
if let Some(stage) = instance.current_stage_mut() {
|
|
||||||
stage.assigned_tasks.insert(
|
|
||||||
task_id.clone(),
|
|
||||||
TaskState::new(task_id.clone(), agent_id.clone(), agent_role.clone()),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
debug!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
stage = %stage_config.name,
|
|
||||||
task_id = %task_id,
|
|
||||||
agent_id = %agent_id,
|
|
||||||
"Task assigned in sequential mode"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn on_task_completed(&self, msg: TaskCompleted) -> Result<()> {
|
|
||||||
let workflow_id = self.find_workflow_for_task(&msg.task_id)?;
|
|
||||||
|
|
||||||
let should_advance = {
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(&workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.clone()))?;
|
|
||||||
|
|
||||||
let Some(stage) = instance.current_stage_mut() else {
|
|
||||||
return Ok(());
|
|
||||||
};
|
|
||||||
|
|
||||||
let stage_name = stage.name.clone();
|
|
||||||
|
|
||||||
if let Some(task) = stage.assigned_tasks.get_mut(&msg.task_id) {
|
|
||||||
task.mark_completed(msg.result.clone(), msg.artifacts.clone());
|
|
||||||
}
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
stage = %stage_name,
|
|
||||||
task_id = %msg.task_id,
|
|
||||||
agent_id = %msg.agent_id,
|
|
||||||
"Task completed"
|
|
||||||
);
|
|
||||||
|
|
||||||
let all_completed = stage.all_tasks_completed();
|
|
||||||
|
|
||||||
for artifact_id in &msg.artifacts {
|
|
||||||
let artifact = Artifact::new(
|
|
||||||
ArtifactType::Custom("agent_output".to_string()),
|
|
||||||
stage_name.clone(),
|
|
||||||
msg.agent_id.clone(),
|
|
||||||
artifact_id.clone(),
|
|
||||||
);
|
|
||||||
instance.add_artifact(artifact);
|
|
||||||
}
|
|
||||||
|
|
||||||
if all_completed {
|
|
||||||
let stage = instance.current_stage_mut().expect("stage exists");
|
|
||||||
|
|
||||||
let duration = stage
|
|
||||||
.started_at
|
|
||||||
.map(|start| (Utc::now() - start).num_seconds() as f64)
|
|
||||||
.unwrap_or(0.0);
|
|
||||||
|
|
||||||
self.metrics.stage_duration_seconds.observe(duration);
|
|
||||||
|
|
||||||
stage.transition(StageStatus::Completed)?;
|
|
||||||
stage.mark_completed();
|
|
||||||
|
|
||||||
self.metrics.stages_completed.inc();
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
stage = %stage_name,
|
|
||||||
duration_sec = duration,
|
|
||||||
"Stage completed"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
all_completed
|
|
||||||
};
|
|
||||||
|
|
||||||
if should_advance {
|
|
||||||
self.advance_to_next_stage(&workflow_id).await?;
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn advance_to_next_stage(&self, workflow_id: &str) -> Result<()> {
|
|
||||||
let should_continue = {
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
instance.advance_stage()?
|
|
||||||
};
|
|
||||||
|
|
||||||
if should_continue {
|
|
||||||
self.execute_current_stage(workflow_id).await?;
|
|
||||||
} else {
|
|
||||||
let duration = {
|
|
||||||
let instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
|
|
||||||
(Utc::now() - instance.created_at).num_seconds() as f64
|
|
||||||
};
|
|
||||||
|
|
||||||
self.metrics.workflow_duration_seconds.observe(duration);
|
|
||||||
self.metrics.workflows_completed.inc();
|
|
||||||
self.metrics.active_workflows.dec();
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
duration_sec = duration,
|
|
||||||
"Workflow completed"
|
|
||||||
);
|
|
||||||
|
|
||||||
self.publish_workflow_completed(workflow_id).await?;
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn approve_stage(&self, workflow_id: &str, approver: &str) -> Result<()> {
|
|
||||||
{
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
|
|
||||||
match instance.status {
|
|
||||||
WorkflowStatus::WaitingApproval(stage_idx)
|
|
||||||
if stage_idx == instance.current_stage_idx =>
|
|
||||||
{
|
|
||||||
instance.status = WorkflowStatus::Running;
|
|
||||||
info!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
stage_idx = stage_idx,
|
|
||||||
approver = %approver,
|
|
||||||
"Stage approved"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
_ => return Err(WorkflowError::NotWaitingApproval),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
self.execute_current_stage(workflow_id).await
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn cancel_workflow(&self, workflow_id: &str, reason: String) -> Result<()> {
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
|
|
||||||
instance.cancel(reason.clone());
|
|
||||||
self.metrics.active_workflows.dec();
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
reason = %reason,
|
|
||||||
"Workflow cancelled"
|
|
||||||
);
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn get_workflow(&self, workflow_id: &str) -> Result<WorkflowInstance> {
|
|
||||||
self.active_workflows
|
|
||||||
.get(workflow_id)
|
|
||||||
.map(|entry| entry.value().clone())
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn list_workflows(&self) -> Vec<WorkflowInstance> {
|
|
||||||
self.active_workflows
|
|
||||||
.iter()
|
|
||||||
.map(|entry| entry.value().clone())
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
fn find_workflow_for_task(&self, task_id: &str) -> Result<String> {
|
|
||||||
for entry in self.active_workflows.iter() {
|
|
||||||
if let Some(stage) = entry.value().current_stage() {
|
|
||||||
if stage.assigned_tasks.contains_key(task_id) {
|
|
||||||
return Ok(entry.key().clone());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Err(WorkflowError::TaskNotFound(task_id.to_string()))
|
|
||||||
}
|
|
||||||
|
|
||||||
#[allow(clippy::excessive_nesting)]
|
|
||||||
pub async fn start_event_listener(self: Arc<Self>) -> Result<()> {
|
|
||||||
let subscriber = self
|
|
||||||
.nats
|
|
||||||
.subscribe("vapora.tasks.completed")
|
|
||||||
.await
|
|
||||||
.map_err(|e| WorkflowError::NatsError(Box::new(e)))?;
|
|
||||||
|
|
||||||
let orchestrator = self.clone();
|
|
||||||
tokio::spawn(async move {
|
|
||||||
let mut subscriber = subscriber;
|
|
||||||
while let Some(msg) = subscriber.next().await {
|
|
||||||
if let Ok(AgentMessage::TaskCompleted(task_completed)) =
|
|
||||||
serde_json::from_slice::<AgentMessage>(&msg.payload)
|
|
||||||
{
|
|
||||||
if let Err(e) = orchestrator.on_task_completed(task_completed).await {
|
|
||||||
error!(error = %e, "Failed to handle task completion");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
let subscriber_failed = self
|
|
||||||
.nats
|
|
||||||
.subscribe("vapora.tasks.failed")
|
|
||||||
.await
|
|
||||||
.map_err(|e| WorkflowError::NatsError(Box::new(e)))?;
|
|
||||||
|
|
||||||
let orchestrator = self.clone();
|
|
||||||
tokio::spawn(async move {
|
|
||||||
let mut subscriber = subscriber_failed;
|
|
||||||
while let Some(msg) = subscriber.next().await {
|
|
||||||
if let Ok(AgentMessage::TaskFailed(task_failed)) =
|
|
||||||
serde_json::from_slice::<AgentMessage>(&msg.payload)
|
|
||||||
{
|
|
||||||
if let Err(e) = orchestrator.on_task_failed(task_failed).await {
|
|
||||||
error!(error = %e, "Failed to handle task failure");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
info!("Workflow orchestrator event listener started");
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn on_task_failed(&self, msg: TaskFailed) -> Result<()> {
|
|
||||||
let workflow_id = self.find_workflow_for_task(&msg.task_id)?;
|
|
||||||
|
|
||||||
{
|
|
||||||
let mut instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get_mut(&workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.clone()))?;
|
|
||||||
|
|
||||||
if let Some(stage) = instance.current_stage_mut() {
|
|
||||||
if let Some(task) = stage.assigned_tasks.get_mut(&msg.task_id) {
|
|
||||||
task.mark_failed(msg.error.clone());
|
|
||||||
}
|
|
||||||
|
|
||||||
if msg.can_retry {
|
|
||||||
warn!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
task_id = %msg.task_id,
|
|
||||||
retry_count = msg.retry_count,
|
|
||||||
"Task failed, will retry"
|
|
||||||
);
|
|
||||||
} else {
|
|
||||||
let stage_name = stage.name.clone();
|
|
||||||
stage.transition(StageStatus::Failed(msg.error.clone()))?;
|
|
||||||
instance.fail(format!("Stage {} failed: {}", stage_name, msg.error));
|
|
||||||
|
|
||||||
self.metrics.workflows_failed.inc();
|
|
||||||
self.metrics.active_workflows.dec();
|
|
||||||
|
|
||||||
error!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
stage = %stage_name,
|
|
||||||
error = %msg.error,
|
|
||||||
"Workflow failed"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn publish_approval_request(&self, workflow_id: &str, stage_name: &str) -> Result<()> {
|
|
||||||
let event = serde_json::json!({
|
|
||||||
"type": "approval_required",
|
|
||||||
"workflow_id": workflow_id,
|
|
||||||
"stage": stage_name,
|
|
||||||
"timestamp": Utc::now().to_rfc3339(),
|
|
||||||
});
|
|
||||||
|
|
||||||
self.nats
|
|
||||||
.publish(
|
|
||||||
"vapora.workflow.approval_required",
|
|
||||||
event.to_string().into(),
|
|
||||||
)
|
|
||||||
.await
|
|
||||||
.map_err(|e| WorkflowError::NatsError(Box::new(e)))?;
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_id = %workflow_id,
|
|
||||||
stage = %stage_name,
|
|
||||||
"Approval request published"
|
|
||||||
);
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn publish_workflow_completed(&self, workflow_id: &str) -> Result<()> {
|
|
||||||
let instance = self
|
|
||||||
.active_workflows
|
|
||||||
.get(workflow_id)
|
|
||||||
.ok_or_else(|| WorkflowError::WorkflowNotFound(workflow_id.to_string()))?;
|
|
||||||
|
|
||||||
let event = serde_json::json!({
|
|
||||||
"type": "workflow_completed",
|
|
||||||
"workflow_id": workflow_id,
|
|
||||||
"template": instance.template_name,
|
|
||||||
"stages_completed": instance.stages.len(),
|
|
||||||
"artifacts": instance.accumulated_artifacts.keys().collect::<Vec<_>>(),
|
|
||||||
"timestamp": Utc::now().to_rfc3339(),
|
|
||||||
});
|
|
||||||
|
|
||||||
self.nats
|
|
||||||
.publish("vapora.workflow.completed", event.to_string().into())
|
|
||||||
.await
|
|
||||||
.map_err(|e| WorkflowError::NatsError(Box::new(e)))?;
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn list_templates(&self) -> Vec<String> {
|
|
||||||
self.config
|
|
||||||
.workflows
|
|
||||||
.iter()
|
|
||||||
.map(|w| w.name.clone())
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Enrich initial context with Kogral knowledge (guidelines, patterns,
|
|
||||||
/// ADRs)
|
|
||||||
pub async fn enrich_context_from_kogral(
|
|
||||||
&self,
|
|
||||||
context: &mut Value,
|
|
||||||
workflow_name: &str,
|
|
||||||
) -> Result<()> {
|
|
||||||
let guidelines = self.query_kogral_guidelines(workflow_name).await?;
|
|
||||||
let patterns = self.query_kogral_patterns(workflow_name).await?;
|
|
||||||
let decisions = self.query_kogral_decisions(workflow_name).await?;
|
|
||||||
|
|
||||||
if let Some(obj) = context.as_object_mut() {
|
|
||||||
obj.insert("kogral_guidelines".to_string(), guidelines);
|
|
||||||
obj.insert("kogral_patterns".to_string(), patterns);
|
|
||||||
obj.insert("kogral_decisions".to_string(), decisions);
|
|
||||||
}
|
|
||||||
|
|
||||||
info!(
|
|
||||||
workflow_name = %workflow_name,
|
|
||||||
"Context enriched with Kogral knowledge"
|
|
||||||
);
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn query_kogral_guidelines(&self, workflow_name: &str) -> Result<Value> {
|
|
||||||
let kogral_path =
|
|
||||||
std::env::var("KOGRAL_PATH").unwrap_or_else(|_| "../kogral/.kogral".to_string());
|
|
||||||
|
|
||||||
let guidelines_path = format!("{}/guidelines/{}.md", kogral_path, workflow_name);
|
|
||||||
|
|
||||||
if let Ok(content) = tokio::fs::read_to_string(&guidelines_path).await {
|
|
||||||
Ok(serde_json::json!({
|
|
||||||
"source": guidelines_path,
|
|
||||||
"content": content,
|
|
||||||
}))
|
|
||||||
} else {
|
|
||||||
warn!(
|
|
||||||
workflow_name = %workflow_name,
|
|
||||||
"No Kogral guidelines found, using empty"
|
|
||||||
);
|
|
||||||
Ok(serde_json::json!({
|
|
||||||
"source": "none",
|
|
||||||
"content": "",
|
|
||||||
}))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[allow(clippy::excessive_nesting)]
|
|
||||||
async fn query_kogral_patterns(&self, workflow_name: &str) -> Result<Value> {
|
|
||||||
let kogral_path =
|
|
||||||
std::env::var("KOGRAL_PATH").unwrap_or_else(|_| "../kogral/.kogral".to_string());
|
|
||||||
|
|
||||||
let patterns_path = format!("{}/patterns", kogral_path);
|
|
||||||
|
|
||||||
let mut patterns = Vec::new();
|
|
||||||
|
|
||||||
if let Ok(mut entries) = tokio::fs::read_dir(&patterns_path).await {
|
|
||||||
while let Ok(Some(entry)) = entries.next_entry().await {
|
|
||||||
if let Some(ext) = entry.path().extension() {
|
|
||||||
if ext == "md" && entry.path().to_string_lossy().contains(workflow_name) {
|
|
||||||
if let Ok(content) = tokio::fs::read_to_string(entry.path()).await {
|
|
||||||
patterns.push(serde_json::json!({
|
|
||||||
"file": entry.file_name().to_string_lossy().to_string(),
|
|
||||||
"content": content,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if patterns.is_empty() {
|
|
||||||
warn!(
|
|
||||||
workflow_name = %workflow_name,
|
|
||||||
"No Kogral patterns found"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(serde_json::json!(patterns))
|
|
||||||
}
|
|
||||||
|
|
||||||
#[allow(clippy::excessive_nesting)]
|
|
||||||
async fn query_kogral_decisions(&self, workflow_name: &str) -> Result<Value> {
|
|
||||||
let kogral_path =
|
|
||||||
std::env::var("KOGRAL_PATH").unwrap_or_else(|_| "../kogral/.kogral".to_string());
|
|
||||||
|
|
||||||
let adrs_path = format!("{}/adrs", kogral_path);
|
|
||||||
|
|
||||||
let mut adrs = Vec::new();
|
|
||||||
|
|
||||||
if let Ok(mut entries) = tokio::fs::read_dir(&adrs_path).await {
|
|
||||||
while let Ok(Some(entry)) = entries.next_entry().await {
|
|
||||||
if let Some(ext) = entry.path().extension() {
|
|
||||||
if ext == "md" {
|
|
||||||
if let Ok(content) = tokio::fs::read_to_string(entry.path()).await {
|
|
||||||
if content.to_lowercase().contains(workflow_name)
|
|
||||||
|| content
|
|
||||||
.to_lowercase()
|
|
||||||
.contains(&workflow_name.replace('_', " "))
|
|
||||||
{
|
|
||||||
adrs.push(serde_json::json!({
|
|
||||||
"file": entry.file_name().to_string_lossy().to_string(),
|
|
||||||
"content": content,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
adrs.sort_by(|a, b| {
|
|
||||||
let a_file = a["file"].as_str().unwrap_or("");
|
|
||||||
let b_file = b["file"].as_str().unwrap_or("");
|
|
||||||
b_file.cmp(a_file)
|
|
||||||
});
|
|
||||||
|
|
||||||
let recent_adrs = adrs.into_iter().take(5).collect::<Vec<_>>();
|
|
||||||
|
|
||||||
if recent_adrs.is_empty() {
|
|
||||||
warn!(
|
|
||||||
workflow_name = %workflow_name,
|
|
||||||
"No relevant Kogral ADRs found"
|
|
||||||
);
|
|
||||||
} else {
|
|
||||||
info!(
|
|
||||||
workflow_name = %workflow_name,
|
|
||||||
count = recent_adrs.len(),
|
|
||||||
"Loaded recent ADRs from Kogral"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok(serde_json::json!(recent_adrs))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
#[test]
|
|
||||||
fn test_orchestrator_module_compiles() {
|
|
||||||
assert!(true);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,307 +0,0 @@
|
|||||||
use std::collections::HashMap;
|
|
||||||
|
|
||||||
use chrono::{DateTime, Utc};
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
|
|
||||||
use crate::config::StageConfig;
|
|
||||||
use crate::error::{Result, WorkflowError};
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
|
||||||
#[serde(rename_all = "snake_case")]
|
|
||||||
pub enum StageStatus {
|
|
||||||
Pending,
|
|
||||||
WaitingApproval,
|
|
||||||
Running,
|
|
||||||
Completed,
|
|
||||||
Failed(String),
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
||||||
pub struct StageState {
|
|
||||||
pub name: String,
|
|
||||||
pub status: StageStatus,
|
|
||||||
pub config: StageConfig,
|
|
||||||
pub assigned_tasks: HashMap<String, TaskState>,
|
|
||||||
pub artifacts_produced: Vec<String>,
|
|
||||||
pub started_at: Option<DateTime<Utc>>,
|
|
||||||
pub completed_at: Option<DateTime<Utc>>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
||||||
pub struct TaskState {
|
|
||||||
pub task_id: String,
|
|
||||||
pub agent_id: String,
|
|
||||||
pub agent_role: String,
|
|
||||||
pub status: TaskStatus,
|
|
||||||
pub result: Option<String>,
|
|
||||||
pub artifacts: Vec<String>,
|
|
||||||
pub assigned_at: DateTime<Utc>,
|
|
||||||
pub completed_at: Option<DateTime<Utc>>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
|
||||||
#[serde(rename_all = "snake_case")]
|
|
||||||
pub enum TaskStatus {
|
|
||||||
Assigned,
|
|
||||||
Running,
|
|
||||||
Completed,
|
|
||||||
Failed(String),
|
|
||||||
}
|
|
||||||
|
|
||||||
impl StageState {
|
|
||||||
pub fn new(config: StageConfig) -> Self {
|
|
||||||
Self {
|
|
||||||
name: config.name.clone(),
|
|
||||||
status: StageStatus::Pending,
|
|
||||||
config,
|
|
||||||
assigned_tasks: HashMap::new(),
|
|
||||||
artifacts_produced: Vec::new(),
|
|
||||||
started_at: None,
|
|
||||||
completed_at: None,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn all_tasks_completed(&self) -> bool {
|
|
||||||
!self.assigned_tasks.is_empty()
|
|
||||||
&& self
|
|
||||||
.assigned_tasks
|
|
||||||
.values()
|
|
||||||
.all(|t| matches!(t.status, TaskStatus::Completed))
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn any_task_failed(&self) -> bool {
|
|
||||||
self.assigned_tasks
|
|
||||||
.values()
|
|
||||||
.any(|t| matches!(t.status, TaskStatus::Failed(_)))
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn collect_artifacts(&self) -> Vec<String> {
|
|
||||||
self.assigned_tasks
|
|
||||||
.values()
|
|
||||||
.flat_map(|t| t.artifacts.clone())
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn transition(&mut self, new_status: StageStatus) -> Result<()> {
|
|
||||||
let valid = matches!(
|
|
||||||
(&self.status, &new_status),
|
|
||||||
(StageStatus::Pending, StageStatus::Running)
|
|
||||||
| (StageStatus::Pending, StageStatus::WaitingApproval)
|
|
||||||
| (StageStatus::WaitingApproval, StageStatus::Running)
|
|
||||||
| (StageStatus::Running, StageStatus::Completed)
|
|
||||||
| (StageStatus::Running, StageStatus::Failed(_))
|
|
||||||
);
|
|
||||||
|
|
||||||
if !valid {
|
|
||||||
return Err(WorkflowError::InvalidTransition {
|
|
||||||
from: format!("{:?}", self.status),
|
|
||||||
to: format!("{:?}", new_status),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
self.status = new_status;
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn mark_started(&mut self) {
|
|
||||||
self.started_at = Some(Utc::now());
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn mark_completed(&mut self) {
|
|
||||||
self.completed_at = Some(Utc::now());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl TaskState {
|
|
||||||
pub fn new(task_id: String, agent_id: String, agent_role: String) -> Self {
|
|
||||||
Self {
|
|
||||||
task_id,
|
|
||||||
agent_id,
|
|
||||||
agent_role,
|
|
||||||
status: TaskStatus::Assigned,
|
|
||||||
result: None,
|
|
||||||
artifacts: Vec::new(),
|
|
||||||
assigned_at: Utc::now(),
|
|
||||||
completed_at: None,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn mark_running(&mut self) {
|
|
||||||
self.status = TaskStatus::Running;
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn mark_completed(&mut self, result: String, artifacts: Vec<String>) {
|
|
||||||
self.status = TaskStatus::Completed;
|
|
||||||
self.result = Some(result);
|
|
||||||
self.artifacts = artifacts;
|
|
||||||
self.completed_at = Some(Utc::now());
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn mark_failed(&mut self, error: String) {
|
|
||||||
self.status = TaskStatus::Failed(error);
|
|
||||||
self.completed_at = Some(Utc::now());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
fn create_test_config() -> StageConfig {
|
|
||||||
StageConfig {
|
|
||||||
name: "test_stage".to_string(),
|
|
||||||
agents: vec!["agent1".to_string()],
|
|
||||||
parallel: false,
|
|
||||||
max_parallel: None,
|
|
||||||
approval_required: false,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_stage_state_creation() {
|
|
||||||
let config = create_test_config();
|
|
||||||
let stage = StageState::new(config.clone());
|
|
||||||
|
|
||||||
assert_eq!(stage.name, "test_stage");
|
|
||||||
assert_eq!(stage.status, StageStatus::Pending);
|
|
||||||
assert!(stage.assigned_tasks.is_empty());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_stage_transitions() {
|
|
||||||
let config = create_test_config();
|
|
||||||
let mut stage = StageState::new(config);
|
|
||||||
|
|
||||||
assert!(stage.transition(StageStatus::Running).is_ok());
|
|
||||||
assert_eq!(stage.status, StageStatus::Running);
|
|
||||||
|
|
||||||
assert!(stage.transition(StageStatus::Completed).is_ok());
|
|
||||||
assert_eq!(stage.status, StageStatus::Completed);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_invalid_stage_transition() {
|
|
||||||
let config = create_test_config();
|
|
||||||
let mut stage = StageState::new(config);
|
|
||||||
|
|
||||||
let result = stage.transition(StageStatus::Completed);
|
|
||||||
assert!(result.is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_all_tasks_completed() {
|
|
||||||
let config = create_test_config();
|
|
||||||
let mut stage = StageState::new(config);
|
|
||||||
|
|
||||||
let mut task1 = TaskState::new(
|
|
||||||
"task1".to_string(),
|
|
||||||
"agent1".to_string(),
|
|
||||||
"developer".to_string(),
|
|
||||||
);
|
|
||||||
task1.mark_completed("done".to_string(), vec![]);
|
|
||||||
|
|
||||||
let mut task2 = TaskState::new(
|
|
||||||
"task2".to_string(),
|
|
||||||
"agent2".to_string(),
|
|
||||||
"developer".to_string(),
|
|
||||||
);
|
|
||||||
task2.mark_completed("done".to_string(), vec![]);
|
|
||||||
|
|
||||||
stage.assigned_tasks.insert("task1".to_string(), task1);
|
|
||||||
stage.assigned_tasks.insert("task2".to_string(), task2);
|
|
||||||
|
|
||||||
assert!(stage.all_tasks_completed());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_any_task_failed() {
|
|
||||||
let config = create_test_config();
|
|
||||||
let mut stage = StageState::new(config);
|
|
||||||
|
|
||||||
let mut task1 = TaskState::new(
|
|
||||||
"task1".to_string(),
|
|
||||||
"agent1".to_string(),
|
|
||||||
"developer".to_string(),
|
|
||||||
);
|
|
||||||
task1.mark_completed("done".to_string(), vec![]);
|
|
||||||
|
|
||||||
let mut task2 = TaskState::new(
|
|
||||||
"task2".to_string(),
|
|
||||||
"agent2".to_string(),
|
|
||||||
"developer".to_string(),
|
|
||||||
);
|
|
||||||
task2.mark_failed("error occurred".to_string());
|
|
||||||
|
|
||||||
stage.assigned_tasks.insert("task1".to_string(), task1);
|
|
||||||
stage.assigned_tasks.insert("task2".to_string(), task2);
|
|
||||||
|
|
||||||
assert!(stage.any_task_failed());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_collect_artifacts() {
|
|
||||||
let config = create_test_config();
|
|
||||||
let mut stage = StageState::new(config);
|
|
||||||
|
|
||||||
let mut task1 = TaskState::new(
|
|
||||||
"task1".to_string(),
|
|
||||||
"agent1".to_string(),
|
|
||||||
"developer".to_string(),
|
|
||||||
);
|
|
||||||
task1.mark_completed("done".to_string(), vec!["artifact1".to_string()]);
|
|
||||||
|
|
||||||
let mut task2 = TaskState::new(
|
|
||||||
"task2".to_string(),
|
|
||||||
"agent2".to_string(),
|
|
||||||
"developer".to_string(),
|
|
||||||
);
|
|
||||||
task2.mark_completed(
|
|
||||||
"done".to_string(),
|
|
||||||
vec!["artifact2".to_string(), "artifact3".to_string()],
|
|
||||||
);
|
|
||||||
|
|
||||||
stage.assigned_tasks.insert("task1".to_string(), task1);
|
|
||||||
stage.assigned_tasks.insert("task2".to_string(), task2);
|
|
||||||
|
|
||||||
let artifacts = stage.collect_artifacts();
|
|
||||||
assert_eq!(artifacts.len(), 3);
|
|
||||||
assert!(artifacts.contains(&"artifact1".to_string()));
|
|
||||||
assert!(artifacts.contains(&"artifact2".to_string()));
|
|
||||||
assert!(artifacts.contains(&"artifact3".to_string()));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_task_state_lifecycle() {
|
|
||||||
let mut task = TaskState::new(
|
|
||||||
"task1".to_string(),
|
|
||||||
"agent1".to_string(),
|
|
||||||
"developer".to_string(),
|
|
||||||
);
|
|
||||||
|
|
||||||
assert_eq!(task.status, TaskStatus::Assigned);
|
|
||||||
|
|
||||||
task.mark_running();
|
|
||||||
assert_eq!(task.status, TaskStatus::Running);
|
|
||||||
|
|
||||||
task.mark_completed("success".to_string(), vec!["artifact1".to_string()]);
|
|
||||||
assert_eq!(task.status, TaskStatus::Completed);
|
|
||||||
assert_eq!(task.result, Some("success".to_string()));
|
|
||||||
assert_eq!(task.artifacts.len(), 1);
|
|
||||||
assert!(task.completed_at.is_some());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_task_failure() {
|
|
||||||
let mut task = TaskState::new(
|
|
||||||
"task1".to_string(),
|
|
||||||
"agent1".to_string(),
|
|
||||||
"developer".to_string(),
|
|
||||||
);
|
|
||||||
|
|
||||||
task.mark_running();
|
|
||||||
task.mark_failed("compilation error".to_string());
|
|
||||||
|
|
||||||
assert!(matches!(task.status, TaskStatus::Failed(_)));
|
|
||||||
assert!(task.completed_at.is_some());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,275 +0,0 @@
|
|||||||
# ADR-0028: Workflow Orchestrator for Cost-Efficient Multi-Agent Pipelines
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
**Accepted** - Implemented in v1.2.0
|
|
||||||
|
|
||||||
## Context
|
|
||||||
|
|
||||||
### The Problem: Excessive LLM Costs from Cache Token Accumulation
|
|
||||||
|
|
||||||
Analysis of real Claude Code usage data (5 weeks, individual developer) reveals a critical cost pattern:
|
|
||||||
|
|
||||||
| Metric | Value |
|
|
||||||
|--------|-------|
|
|
||||||
| Total cost | $1,050.68 |
|
|
||||||
| Weekly average | ~$210 |
|
|
||||||
| Monthly projection | ~$840 |
|
|
||||||
| Cache read tokens | 3.82B (95.7% of total) |
|
|
||||||
| Cache creation tokens | 170M (4.3%) |
|
|
||||||
| Direct input tokens | 2.4M (0.06%) |
|
|
||||||
| Direct output tokens | 366K (0.009%) |
|
|
||||||
|
|
||||||
**The cost is dominated by cache tokens, not generation.**
|
|
||||||
|
|
||||||
### Root Cause: Monolithic Session Pattern
|
|
||||||
|
|
||||||
Current workflow with Claude Code follows a monolithic session pattern:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Session start
|
|
||||||
├─ Message 1: context 50K → cache read 50K
|
|
||||||
├─ Message 2: context 100K → cache read 100K
|
|
||||||
├─ Message 3: context 150K → cache read 150K
|
|
||||||
├─ ...
|
|
||||||
└─ Message 50: context 800K → cache read 800K
|
|
||||||
─────────────────
|
|
||||||
~20M cache reads per session
|
|
||||||
```
|
|
||||||
|
|
||||||
Each message in a long session re-sends the entire conversation history. Over a typical development session (50+ messages), context accumulates to 500K-1M tokens, with each subsequent message re-transmitting all previous context.
|
|
||||||
|
|
||||||
### Why This Matters
|
|
||||||
|
|
||||||
At current pricing (2026 rates):
|
|
||||||
|
|
||||||
- Cache read (Haiku): $0.03/1M tokens
|
|
||||||
- Cache read (Sonnet): $0.30/1M tokens
|
|
||||||
- Cache read (Opus): $1.50/1M tokens
|
|
||||||
|
|
||||||
With 3.82B cache read tokens distributed across Sonnet (51%) and Haiku (38%), the cache cost alone exceeds what direct input/output would cost.
|
|
||||||
|
|
||||||
## Decision
|
|
||||||
|
|
||||||
**Implement a Workflow Orchestrator (vapora-workflow-engine) that executes multi-stage pipelines with short-lived agent contexts.**
|
|
||||||
|
|
||||||
### Architecture: Agents with Short Lifecycles
|
|
||||||
|
|
||||||
Instead of one long session accumulating context, workflows execute as discrete stages:
|
|
||||||
|
|
||||||
```text
|
|
||||||
┌─────────────────────────────────────────────────────────┐
|
|
||||||
│ Task: "Implement feature X" │
|
|
||||||
└─────────────────────────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────────────┼────────────────────┐
|
|
||||||
▼ ▼ ▼
|
|
||||||
┌─────────┐ ┌──────────┐ ┌──────────┐
|
|
||||||
│Architect│ │Developer │ │ Reviewer │
|
|
||||||
│ (Opus) │ │ (Haiku) │ │ (Sonnet) │
|
|
||||||
├─────────┤ ├──────────┤ ├──────────┤
|
|
||||||
│Context: │ │Context: │ │Context: │
|
|
||||||
│ 40K │───────▶│ 25K │───────▶│ 35K │
|
|
||||||
│ 5 msgs │ spec │ 12 msgs │ code │ 4 msgs │
|
|
||||||
│ 200K │ │ 300K │ │ 140K │
|
|
||||||
│ cache │ │ cache │ │ cache │
|
|
||||||
└────┬────┘ └────┬─────┘ └────┬─────┘
|
|
||||||
│ │ │
|
|
||||||
▼ ▼ ▼
|
|
||||||
TERMINATES TERMINATES TERMINATES
|
|
||||||
(context (context (context
|
|
||||||
discarded) discarded) discarded)
|
|
||||||
|
|
||||||
Total cache: ~640K
|
|
||||||
Monolithic equivalent: ~20-40M
|
|
||||||
Reduction: 95-97%
|
|
||||||
```
|
|
||||||
|
|
||||||
### Key Principles
|
|
||||||
|
|
||||||
1. **Context isolation**: Each agent receives only what it needs (spec, relevant files), not full conversation history
|
|
||||||
|
|
||||||
2. **Artifact passing, not conversation passing**: Between agents flows the result (spec, code, review), not the dialogue that produced it
|
|
||||||
|
|
||||||
3. **Short lifecycles**: Agent completes task → context dies → next agent starts fresh
|
|
||||||
|
|
||||||
4. **Persistent memory via Kogral**: Important decisions/patterns stored in knowledge base, not in session context
|
|
||||||
|
|
||||||
## Implementation
|
|
||||||
|
|
||||||
### Components
|
|
||||||
|
|
||||||
1. **vapora-workflow-engine** (new crate):
|
|
||||||
- `WorkflowOrchestrator`: Main coordinator managing workflow lifecycle
|
|
||||||
- `WorkflowInstance`: State machine tracking individual workflow execution
|
|
||||||
- `StageState`: Manages stage execution and task assignment
|
|
||||||
- `Artifact`: Data passed between stages (ADR, Code, TestResults, Review, Documentation)
|
|
||||||
|
|
||||||
2. **Workflow Templates** (`config/workflows.toml`):
|
|
||||||
- `feature_development` (5 stages): architecture → implementation → testing → review → deployment
|
|
||||||
- `bugfix` (4 stages): investigation → fix → testing → deployment
|
|
||||||
- `documentation_update` (3 stages): content → review → publish
|
|
||||||
- `security_audit` (4 stages): analysis → pentesting → remediation → verification
|
|
||||||
|
|
||||||
3. **REST API** (`/api/v1/workflow_orchestrator`):
|
|
||||||
- `POST /` - Start workflow
|
|
||||||
- `GET /` - List active workflows
|
|
||||||
- `GET /:id` - Get workflow status
|
|
||||||
- `POST /:id/approve` - Approve waiting stage
|
|
||||||
- `POST /:id/cancel` - Cancel running workflow
|
|
||||||
- `GET /templates` - List available templates
|
|
||||||
|
|
||||||
4. **CLI** (vapora-cli):
|
|
||||||
- `vapora workflow start --template <name> --context context.json`
|
|
||||||
- `vapora workflow list`
|
|
||||||
- `vapora workflow status <id>`
|
|
||||||
- `vapora workflow approve <id> --approver "Name"`
|
|
||||||
- `vapora workflow cancel <id> --reason "Reason"`
|
|
||||||
- `vapora workflow templates`
|
|
||||||
|
|
||||||
5. **Kogral Integration**:
|
|
||||||
- `enrich_context_from_kogral()` - Loads guidelines, patterns, ADRs
|
|
||||||
- Filesystem-based knowledge retrieval from `.kogral/` directory
|
|
||||||
- Configurable via `KOGRAL_PATH` environment variable
|
|
||||||
|
|
||||||
### Integration with Existing Components
|
|
||||||
|
|
||||||
| Component | Usage |
|
|
||||||
|-----------|-------|
|
|
||||||
| SwarmCoordinator | Task assignment via `submit_task_for_bidding()` |
|
|
||||||
| AgentRegistry | 12 roles with lifecycle management |
|
|
||||||
| LearningProfiles | Expertise-based agent selection |
|
|
||||||
| KGPersistence | Workflow execution history |
|
|
||||||
| NATS JetStream | Inter-stage event coordination |
|
|
||||||
|
|
||||||
## Rationale
|
|
||||||
|
|
||||||
### Why Vapora Already Has the Pieces
|
|
||||||
|
|
||||||
Current Vapora implementation includes:
|
|
||||||
|
|
||||||
| Component | Status | Functionality |
|
|
||||||
|-----------|--------|---------------|
|
|
||||||
| SwarmCoordinator | Complete | Task assignment, load balancing |
|
|
||||||
| AgentRegistry | Complete | 12 roles, lifecycle management |
|
|
||||||
| Learning Profiles | Complete | Expertise scoring with recency bias |
|
|
||||||
| KG Persistence | Complete | SurrealDB, execution history |
|
|
||||||
| NATS Messaging | Complete | Inter-agent communication |
|
|
||||||
| Workflow Templates | Complete | `workflows.toml` with stage definitions |
|
|
||||||
| Artifact Types | Complete | `TaskCompleted.artifacts` field |
|
|
||||||
|
|
||||||
**What was missing**: The orchestration layer that executes workflow templates by loading templates, creating instances, listening for task completions, advancing stages, and passing artifacts.
|
|
||||||
|
|
||||||
### Why Not Alternative Solutions
|
|
||||||
|
|
||||||
| Alternative | Why Not |
|
|
||||||
|-------------|---------|
|
|
||||||
| Manual `/compact` in Claude Code | Requires user discipline, doesn't fundamentally change pattern |
|
|
||||||
| Shorter sessions manually | Loses context continuity, user must track state |
|
|
||||||
| External tools (LiteLLM, CrewAI) | Python-based, doesn't leverage existing Vapora infrastructure |
|
|
||||||
| Just use Haiku everywhere | Quality degradation for complex tasks |
|
|
||||||
|
|
||||||
Vapora already has budget-aware routing, learning profiles, and swarm coordination. The workflow orchestrator completes the picture.
|
|
||||||
|
|
||||||
### Why Kogral Integration
|
|
||||||
|
|
||||||
Kogral provides persistent knowledge that would otherwise bloat session context:
|
|
||||||
|
|
||||||
| Without Kogral | With Kogral |
|
|
||||||
|----------------|-------------|
|
|
||||||
| Guidelines re-explained each session | Query once via MCP, inject 5K tokens |
|
|
||||||
| ADRs repeated in conversation | Reference by ID, inject summary |
|
|
||||||
| Patterns described verbally | Structured retrieval, minimal tokens |
|
|
||||||
|
|
||||||
Kogral transforms "remember our auth pattern" (requires context) into "query pattern:auth" (stateless lookup).
|
|
||||||
|
|
||||||
## Consequences
|
|
||||||
|
|
||||||
### Positive
|
|
||||||
|
|
||||||
1. **~95% reduction in cache token costs**: $840/month → ~$50-100/month for same workload
|
|
||||||
|
|
||||||
2. **Better model allocation**: Opus for architecture (high quality, few tokens), Haiku for implementation (lower quality acceptable, many tokens)
|
|
||||||
|
|
||||||
3. **Leverages existing investment**: Uses SwarmCoordinator, LearningProfiles, KGPersistence already built
|
|
||||||
|
|
||||||
4. **Audit trail**: Each agent execution persisted to KG with tokens, cost, duration
|
|
||||||
|
|
||||||
5. **Parallelization**: Multiple developers can work simultaneously on different parts
|
|
||||||
|
|
||||||
6. **Quality through specialization**: Each agent optimized for its role vs one generalist session
|
|
||||||
|
|
||||||
### Negative
|
|
||||||
|
|
||||||
1. **Orchestration overhead**: Additional component to maintain
|
|
||||||
|
|
||||||
2. **Latency between stages**: Artifact passing adds delay vs continuous conversation
|
|
||||||
|
|
||||||
3. **Context loss between agents**: Agent B doesn't know what Agent A "considered but rejected"
|
|
||||||
|
|
||||||
4. **Debugging complexity**: Issues span multiple agent executions
|
|
||||||
|
|
||||||
### Mitigations
|
|
||||||
|
|
||||||
| Negative | Mitigation |
|
|
||||||
|----------|------------|
|
|
||||||
| Orchestration overhead | Minimal code (~1500 lines), clear separation of concerns |
|
|
||||||
| Latency | Parallel stages where possible, async execution |
|
|
||||||
| Context loss | Kogral captures decisions, not just outcomes |
|
|
||||||
| Debugging | Workflow ID traces all related executions in KG |
|
|
||||||
|
|
||||||
## Metrics for Success
|
|
||||||
|
|
||||||
| Metric | Before | After (Target) |
|
|
||||||
|--------|--------|----------------|
|
|
||||||
| Monthly LLM cost | ~$840 | <$150 |
|
|
||||||
| Cache tokens per task | ~20M | <1M |
|
|
||||||
| Average context size | 500K+ | <50K per agent |
|
|
||||||
| Workflow completion rate | N/A | >95% |
|
|
||||||
|
|
||||||
## Cost Projection
|
|
||||||
|
|
||||||
Based on analyzed usage patterns with optimized workflow:
|
|
||||||
|
|
||||||
| Role | Model | % of Work | Monthly Cost |
|
|
||||||
|------|-------|-----------|--------------|
|
|
||||||
| Architect | Opus | 10% | ~$25 |
|
|
||||||
| Developer | Haiku | 50% | ~$30 |
|
|
||||||
| Reviewer | Sonnet | 25% | ~$40 |
|
|
||||||
| Tester | Haiku | 15% | ~$15 |
|
|
||||||
| **Total** | | | **~$110** |
|
|
||||||
|
|
||||||
**Savings: ~$730/month (87% reduction)**
|
|
||||||
|
|
||||||
## Implementation Status
|
|
||||||
|
|
||||||
- **Status**: Complete (v1.2.0)
|
|
||||||
- **Crates**: vapora-workflow-engine, vapora-cli
|
|
||||||
- **Tests**: 26 unit tests + 1 doc test passing
|
|
||||||
- **Endpoints**: 6 REST API endpoints
|
|
||||||
- **Templates**: 4 pre-configured workflows
|
|
||||||
- **CLI Commands**: 6 workflow management commands
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Usage data: Claude Code usage analysis (5 weeks, 3.82B cache tokens)
|
|
||||||
- Vapora SwarmCoordinator: `crates/vapora-swarm/src/coordinator.rs`
|
|
||||||
- Vapora Workflows Config: `config/workflows.toml`
|
|
||||||
- Kogral MCP: `kogral-mcp` (external project)
|
|
||||||
- Implementation: `crates/vapora-workflow-engine/`
|
|
||||||
- CLI: `crates/vapora-cli/`
|
|
||||||
|
|
||||||
## Related ADRs
|
|
||||||
|
|
||||||
- ADR-0014: Learning-Based Agent Selection
|
|
||||||
- ADR-0015: Budget Enforcement & Cost Optimization
|
|
||||||
- ADR-0013: Knowledge Graph for Temporal Execution History
|
|
||||||
- ADR-0018: Swarm Load Balancing
|
|
||||||
|
|
||||||
## Decision Drivers
|
|
||||||
|
|
||||||
1. **Data-driven**: 95% of cost is cache tokens from long sessions
|
|
||||||
2. **Infrastructure exists**: Vapora has all pieces except orchestrator
|
|
||||||
3. **Kogral synergy**: Persistent knowledge reduces context requirements
|
|
||||||
4. **Measurable outcome**: Clear before/after metrics for validation
|
|
||||||
5. **Production-ready**: Complete implementation with tests and documentation
|
|
||||||
@ -10,16 +10,13 @@ Complete system architecture and design documentation for VAPORA.
|
|||||||
- **[Multi-IA Router](multi-ia-router.md)** — Provider selection, routing rules, and fallback mechanisms
|
- **[Multi-IA Router](multi-ia-router.md)** — Provider selection, routing rules, and fallback mechanisms
|
||||||
- **[Roles, Permissions & Profiles](roles-permissions-profiles.md)** — Cedar policy engine and RBAC implementation
|
- **[Roles, Permissions & Profiles](roles-permissions-profiles.md)** — Cedar policy engine and RBAC implementation
|
||||||
- **[Task, Agent & Doc Manager](task-agent-doc-manager.md)** — Task orchestration and documentation lifecycle
|
- **[Task, Agent & Doc Manager](task-agent-doc-manager.md)** — Task orchestration and documentation lifecycle
|
||||||
- **[Schema Validation Pipeline](schema-validation-pipeline.md)** — Runtime validation with Nickel contracts for MCP tools and agent tasks
|
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
These documents cover:
|
These documents cover:
|
||||||
|
|
||||||
- Complete system architecture and design decisions
|
- Complete system architecture and design decisions
|
||||||
- Multi-agent orchestration and coordination patterns
|
- Multi-agent orchestration and coordination patterns
|
||||||
- Provider routing and selection strategies
|
- Provider routing and selection strategies
|
||||||
- Workflow execution and task management
|
- Workflow execution and task management
|
||||||
- Security, RBAC, and policy enforcement
|
- Security, RBAC, and policy enforcement
|
||||||
- Learning-based agent selection and cost optimization
|
- Learning-based agent selection and cost optimization
|
||||||
- Runtime schema validation with Nickel contracts
|
|
||||||
|
|||||||
@ -1,507 +0,0 @@
|
|||||||
# Schema Validation Pipeline
|
|
||||||
|
|
||||||
Runtime validation system for MCP tools and agent task assignments using Nickel contracts.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The Schema Validation Pipeline prevents downstream errors by validating inputs before execution. It uses Nickel schemas with contracts to enforce type safety, business rules, and data constraints at runtime.
|
|
||||||
|
|
||||||
**Problem Solved:** VAPORA previously assumed valid inputs or failed downstream. This caused:
|
|
||||||
|
|
||||||
- Invalid UUIDs reaching database queries
|
|
||||||
- Empty strings bypassing business logic
|
|
||||||
- Out-of-range priorities corrupting task queues
|
|
||||||
- Malformed contexts breaking agent execution
|
|
||||||
|
|
||||||
**Solution:** Validate all inputs against Nickel schemas before execution.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
```text
|
|
||||||
┌─────────────────────────────────────────────────────────────────┐
|
|
||||||
│ Client Request │
|
|
||||||
│ (MCP Tool Invocation / Agent Task Assignment) │
|
|
||||||
└────────────────────────────┬────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────────────────────────────┐
|
|
||||||
│ ValidationPipeline │
|
|
||||||
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
||||||
│ │ 1. Load schema from SchemaRegistry (cached) │ │
|
|
||||||
│ │ 2. Validate types (String, Number, Array, Object) │ │
|
|
||||||
│ │ 3. Check required fields │ │
|
|
||||||
│ │ 4. Apply contracts (NonEmpty, UUID, Range, etc.) │ │
|
|
||||||
│ │ 5. Apply default values │ │
|
|
||||||
│ │ 6. Return ValidationResult (valid + errors + data) │ │
|
|
||||||
│ └──────────────────────────────────────────────────────────┘ │
|
|
||||||
└────────────────────────────┬────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
┌──────────┴──────────┐
|
|
||||||
│ │
|
|
||||||
▼ ▼
|
|
||||||
┌──────────────┐ ┌─────────────┐
|
|
||||||
│ Valid? │ │ Invalid? │
|
|
||||||
│ Execute │ │ Reject │
|
|
||||||
│ with data │ │ with errors│
|
|
||||||
└──────────────┘ └─────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## Components
|
|
||||||
|
|
||||||
### 1. ValidationPipeline
|
|
||||||
|
|
||||||
Core validation engine in `vapora-shared/src/validation/pipeline.rs`.
|
|
||||||
|
|
||||||
**Key Methods:**
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub async fn validate(
|
|
||||||
&self,
|
|
||||||
schema_name: &str,
|
|
||||||
input: &Value
|
|
||||||
) -> Result<ValidationResult>
|
|
||||||
```
|
|
||||||
|
|
||||||
**Validation Steps:**
|
|
||||||
|
|
||||||
1. Load compiled schema from registry
|
|
||||||
2. Validate field types (String, Number, Bool, Array, Object)
|
|
||||||
3. Check required fields (reject if missing)
|
|
||||||
4. Apply contracts (NonEmpty, UUID, Range, Email, etc.)
|
|
||||||
5. Apply default values for optional fields
|
|
||||||
6. Return ValidationResult with errors (if any)
|
|
||||||
|
|
||||||
**Strict Mode:** Rejects unknown fields not in schema.
|
|
||||||
|
|
||||||
### 2. SchemaRegistry
|
|
||||||
|
|
||||||
Schema loading and caching in `vapora-shared/src/validation/schema_registry.rs`.
|
|
||||||
|
|
||||||
**Features:**
|
|
||||||
|
|
||||||
- **Caching:** Compiled schemas cached in memory (Arc + RwLock)
|
|
||||||
- **Hot Reload:** `invalidate(schema_name)` to reload without restart
|
|
||||||
- **Schema Sources:** File system, embedded string, or URL (future)
|
|
||||||
|
|
||||||
**Schema Structure:**
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct CompiledSchema {
|
|
||||||
pub name: String,
|
|
||||||
pub fields: HashMap<String, FieldSchema>,
|
|
||||||
}
|
|
||||||
|
|
||||||
pub struct FieldSchema {
|
|
||||||
pub field_type: FieldType,
|
|
||||||
pub required: bool,
|
|
||||||
pub contracts: Vec<Contract>,
|
|
||||||
pub default: Option<Value>,
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. NickelBridge
|
|
||||||
|
|
||||||
CLI integration for Nickel operations in `vapora-shared/src/validation/nickel_bridge.rs`.
|
|
||||||
|
|
||||||
**Operations:**
|
|
||||||
|
|
||||||
- `typecheck(path)` — Validate Nickel syntax
|
|
||||||
- `export(path)` — Export schema as JSON
|
|
||||||
- `query(path, field)` — Query specific field
|
|
||||||
- `is_available()` — Check if Nickel CLI is installed
|
|
||||||
|
|
||||||
**Timeout Protection:** 30s default to prevent DoS from malicious Nickel code.
|
|
||||||
|
|
||||||
## Nickel Schemas
|
|
||||||
|
|
||||||
Located in `schemas/` directory (workspace root).
|
|
||||||
|
|
||||||
### Directory Structure
|
|
||||||
|
|
||||||
```text
|
|
||||||
schemas/
|
|
||||||
├── tools/ # MCP tool parameter validation
|
|
||||||
│ ├── kanban_create_task.ncl
|
|
||||||
│ ├── kanban_update_task.ncl
|
|
||||||
│ ├── assign_task_to_agent.ncl
|
|
||||||
│ ├── get_project_summary.ncl
|
|
||||||
│ └── get_agent_capabilities.ncl
|
|
||||||
└── agents/ # Agent task assignment validation
|
|
||||||
└── task_assignment.ncl
|
|
||||||
```
|
|
||||||
|
|
||||||
### Schema Format
|
|
||||||
|
|
||||||
```nickel
|
|
||||||
{
|
|
||||||
tool_name = "example_tool",
|
|
||||||
|
|
||||||
parameters = {
|
|
||||||
# Required field with contracts
|
|
||||||
user_id
|
|
||||||
| String
|
|
||||||
| doc "User UUID"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.match "^[0-9a-f]{8}-[0-9a-f]{4}-...$",
|
|
||||||
|
|
||||||
# Optional field with default
|
|
||||||
priority
|
|
||||||
| Number
|
|
||||||
| doc "Priority score (0-100)"
|
|
||||||
| std.number.between 0 100
|
|
||||||
| default = 50,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Supported Contracts
|
|
||||||
|
|
||||||
| Contract | Description | Example |
|
|
||||||
|----------|-------------|---------|
|
|
||||||
| `std.string.NonEmpty` | String cannot be empty | Required text fields |
|
|
||||||
| `std.string.length.min N` | Minimum length | `min 3` for titles |
|
|
||||||
| `std.string.length.max N` | Maximum length | `max 200` for titles |
|
|
||||||
| `std.string.match PATTERN` | Regex validation | UUID format |
|
|
||||||
| `std.number.between A B` | Numeric range | `between 0 100` |
|
|
||||||
| `std.number.greater_than N` | Minimum value (exclusive) | `> -1` |
|
|
||||||
| `std.number.less_than N` | Maximum value (exclusive) | `< 1000` |
|
|
||||||
| `std.enum.TaggedUnion` | Enum validation | `[| 'low, 'high |]` |
|
|
||||||
|
|
||||||
## Integration Points
|
|
||||||
|
|
||||||
### MCP Server
|
|
||||||
|
|
||||||
Location: `crates/vapora-mcp-server/src/main.rs`
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Initialize validation pipeline
|
|
||||||
let schema_dir = std::env::var("VAPORA_SCHEMA_DIR")
|
|
||||||
.unwrap_or_else(|_| "schemas".to_string());
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from(&schema_dir)));
|
|
||||||
let validation = Arc::new(ValidationPipeline::new(registry));
|
|
||||||
|
|
||||||
// Add to AppState
|
|
||||||
#[derive(Clone)]
|
|
||||||
struct AppState {
|
|
||||||
validation: Arc<ValidationPipeline>,
|
|
||||||
}
|
|
||||||
|
|
||||||
// Validate in handler
|
|
||||||
async fn invoke_tool(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
Json(request): Json<InvokeToolRequest>,
|
|
||||||
) -> impl IntoResponse {
|
|
||||||
let schema_name = format!("tools/{}", request.tool);
|
|
||||||
let validation_result = state
|
|
||||||
.validation
|
|
||||||
.validate(&schema_name, &request.parameters)
|
|
||||||
.await?;
|
|
||||||
|
|
||||||
if !validation_result.valid {
|
|
||||||
return (StatusCode::BAD_REQUEST, Json(validation_errors));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Execute with validated data
|
|
||||||
let validated_params = validation_result.validated_data.unwrap();
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Agent Coordinator
|
|
||||||
|
|
||||||
Location: `crates/vapora-agents/src/coordinator.rs`
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct AgentCoordinator {
|
|
||||||
validation: Arc<ValidationPipeline>,
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
|
|
||||||
impl AgentCoordinator {
|
|
||||||
pub async fn assign_task(
|
|
||||||
&self,
|
|
||||||
role: &str,
|
|
||||||
title: String,
|
|
||||||
description: String,
|
|
||||||
context: String,
|
|
||||||
priority: u32,
|
|
||||||
) -> Result<String, CoordinatorError> {
|
|
||||||
// Validate inputs
|
|
||||||
let input = serde_json::json!({
|
|
||||||
"role": role,
|
|
||||||
"title": &title,
|
|
||||||
"description": &description,
|
|
||||||
"context": &context,
|
|
||||||
"priority": priority,
|
|
||||||
});
|
|
||||||
|
|
||||||
let validation_result = self
|
|
||||||
.validation
|
|
||||||
.validate("agents/task_assignment", &input)
|
|
||||||
.await?;
|
|
||||||
|
|
||||||
if !validation_result.valid {
|
|
||||||
return Err(CoordinatorError::ValidationError(
|
|
||||||
validation_result.errors.join(", ")
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Continue with validated inputs
|
|
||||||
// ...
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Usage Patterns
|
|
||||||
|
|
||||||
### 1. Validating MCP Tool Inputs
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// In MCP server handler
|
|
||||||
let validation_result = state
|
|
||||||
.validation
|
|
||||||
.validate("tools/kanban_create_task", &input)
|
|
||||||
.await?;
|
|
||||||
|
|
||||||
if !validation_result.valid {
|
|
||||||
let errors: Vec<String> = validation_result
|
|
||||||
.errors
|
|
||||||
.iter()
|
|
||||||
.map(|e| e.to_string())
|
|
||||||
.collect();
|
|
||||||
|
|
||||||
return (StatusCode::BAD_REQUEST, Json(json!({
|
|
||||||
"success": false,
|
|
||||||
"validation_errors": errors,
|
|
||||||
})));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Use validated data with defaults applied
|
|
||||||
let validated_data = validation_result.validated_data.unwrap();
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Validating Agent Task Assignments
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// In AgentCoordinator
|
|
||||||
let input = serde_json::json!({
|
|
||||||
"role": role,
|
|
||||||
"title": title,
|
|
||||||
"description": description,
|
|
||||||
"context": context,
|
|
||||||
"priority": priority,
|
|
||||||
});
|
|
||||||
|
|
||||||
let validation_result = self
|
|
||||||
.validation
|
|
||||||
.validate("agents/task_assignment", &input)
|
|
||||||
.await?;
|
|
||||||
|
|
||||||
if !validation_result.valid {
|
|
||||||
warn!("Validation failed: {:?}", validation_result.errors);
|
|
||||||
return Err(CoordinatorError::ValidationError(
|
|
||||||
format!("Invalid input: {}", validation_result.errors.join(", "))
|
|
||||||
));
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Hot Reloading Schemas
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Invalidate single schema
|
|
||||||
registry.invalidate("tools/kanban_create_task").await;
|
|
||||||
|
|
||||||
// Invalidate all schemas (useful for config reload)
|
|
||||||
registry.invalidate_all().await;
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing Schemas
|
|
||||||
|
|
||||||
### Validate Syntax
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nickel typecheck schemas/tools/kanban_create_task.ncl
|
|
||||||
```
|
|
||||||
|
|
||||||
### Export as JSON
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nickel export schemas/tools/kanban_create_task.ncl
|
|
||||||
```
|
|
||||||
|
|
||||||
### Query Specific Field
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nickel query --field parameters.title schemas/tools/kanban_create_task.ncl
|
|
||||||
```
|
|
||||||
|
|
||||||
## Adding New Schemas
|
|
||||||
|
|
||||||
1. Create `.ncl` file in appropriate directory (`tools/` or `agents/`)
|
|
||||||
2. Define `tool_name` or `schema_name`
|
|
||||||
3. Define `parameters` or `fields` with types and contracts
|
|
||||||
4. Add `doc` annotations for documentation
|
|
||||||
5. Test with `nickel typecheck`
|
|
||||||
6. Restart services or use hot-reload
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
|
|
||||||
```nickel
|
|
||||||
# schemas/tools/my_new_tool.ncl
|
|
||||||
{
|
|
||||||
tool_name = "my_new_tool",
|
|
||||||
|
|
||||||
parameters = {
|
|
||||||
name
|
|
||||||
| String
|
|
||||||
| doc "User name (3-50 chars)"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.length.min 3
|
|
||||||
| std.string.length.max 50,
|
|
||||||
|
|
||||||
age
|
|
||||||
| Number
|
|
||||||
| doc "User age (0-120)"
|
|
||||||
| std.number.between 0 120,
|
|
||||||
|
|
||||||
email
|
|
||||||
| String
|
|
||||||
| doc "User email address"
|
|
||||||
| std.string.Email,
|
|
||||||
|
|
||||||
active
|
|
||||||
| Bool
|
|
||||||
| doc "Account status"
|
|
||||||
| default = true,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
- `VAPORA_SCHEMA_DIR` — Schema directory path (default: `"schemas"`)
|
|
||||||
|
|
||||||
**In tests:**
|
|
||||||
|
|
||||||
```rust
|
|
||||||
std::env::set_var("VAPORA_SCHEMA_DIR", "../../schemas");
|
|
||||||
```
|
|
||||||
|
|
||||||
**In production:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export VAPORA_SCHEMA_DIR=/app/schemas
|
|
||||||
```
|
|
||||||
|
|
||||||
## Performance Characteristics
|
|
||||||
|
|
||||||
- **Schema Loading:** ~5-10ms (first load, then cached)
|
|
||||||
- **Validation:** ~0.1-0.5ms per request (in-memory)
|
|
||||||
- **Hot Reload:** ~10-20ms (invalidates cache, reloads from disk)
|
|
||||||
|
|
||||||
**Optimization:** SchemaRegistry uses `Arc<RwLock<HashMap>>` for concurrent reads.
|
|
||||||
|
|
||||||
## Security Considerations
|
|
||||||
|
|
||||||
### Timeout Protection
|
|
||||||
|
|
||||||
NickelBridge enforces 30s timeout on all CLI operations to prevent:
|
|
||||||
|
|
||||||
- Infinite loops in malicious Nickel code
|
|
||||||
- DoS attacks via crafted schemas
|
|
||||||
- Resource exhaustion
|
|
||||||
|
|
||||||
### Input Sanitization
|
|
||||||
|
|
||||||
Contracts prevent:
|
|
||||||
|
|
||||||
- SQL injection (via UUID/Email validation)
|
|
||||||
- XSS attacks (via length limits on text fields)
|
|
||||||
- Buffer overflows (via max length constraints)
|
|
||||||
- Type confusion (via strict type checking)
|
|
||||||
|
|
||||||
### Schema Validation
|
|
||||||
|
|
||||||
All schemas must pass `nickel typecheck` before deployment.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
|
|
||||||
### ValidationResult
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct ValidationResult {
|
|
||||||
pub valid: bool,
|
|
||||||
pub errors: Vec<ValidationError>,
|
|
||||||
pub validated_data: Option<Value>,
|
|
||||||
}
|
|
||||||
|
|
||||||
pub enum ValidationError {
|
|
||||||
MissingField(String),
|
|
||||||
TypeMismatch { field: String, expected: String, got: String },
|
|
||||||
ContractViolation { field: String, contract: String, value: String },
|
|
||||||
InvalidSchema(String),
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Error Response Format
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"success": false,
|
|
||||||
"error": "Validation failed",
|
|
||||||
"validation_errors": [
|
|
||||||
"Field 'project_id' must match UUID pattern",
|
|
||||||
"Field 'title' must be at least 3 characters",
|
|
||||||
"Field 'priority' must be between 0 and 100"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Schema Not Found
|
|
||||||
|
|
||||||
**Error:** `Schema file not found: schemas/tools/my_tool.ncl`
|
|
||||||
|
|
||||||
**Solution:** Check `VAPORA_SCHEMA_DIR` environment variable and ensure schema file exists.
|
|
||||||
|
|
||||||
### Nickel CLI Not Available
|
|
||||||
|
|
||||||
**Error:** `Nickel CLI not found in PATH`
|
|
||||||
|
|
||||||
**Solution:** Install Nickel CLI:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo install nickel-lang-cli
|
|
||||||
```
|
|
||||||
|
|
||||||
### Validation Always Fails
|
|
||||||
|
|
||||||
**Error:** All requests rejected with validation errors
|
|
||||||
|
|
||||||
**Solution:** Check schema syntax with `nickel typecheck`, verify field names match exactly.
|
|
||||||
|
|
||||||
## Future Enhancements
|
|
||||||
|
|
||||||
- [ ] Remote schema loading (HTTP/S3)
|
|
||||||
- [ ] Schema versioning and migration
|
|
||||||
- [ ] Custom contract plugins
|
|
||||||
- [ ] GraphQL schema generation from Nickel
|
|
||||||
- [ ] OpenAPI spec generation
|
|
||||||
|
|
||||||
## Related Documentation
|
|
||||||
|
|
||||||
- [Nickel Language Documentation](https://nickel-lang.org/)
|
|
||||||
- [VAPORA Architecture Overview](vapora-architecture.md)
|
|
||||||
- [Agent Coordination](agent-registry-coordination.md)
|
|
||||||
- [MCP Protocol Integration](../integrations/mcp-server.md)
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- **Implementation:** `crates/vapora-shared/src/validation/`
|
|
||||||
- **Schemas:** `schemas/`
|
|
||||||
- **Tests:** `crates/vapora-shared/tests/validation_integration.rs`
|
|
||||||
- **MCP Integration:** `crates/vapora-mcp-server/src/main.rs`
|
|
||||||
- **Agent Integration:** `crates/vapora-agents/src/coordinator.rs`
|
|
||||||
@ -39,7 +39,6 @@ Unlike fragmented tool ecosystems, Vapora is a single, self-contained system whe
|
|||||||
3. [Multi-Agent Coordination](#multi-agent-coordination)
|
3. [Multi-Agent Coordination](#multi-agent-coordination)
|
||||||
- [Learning-Based Agent Selection (Phase 5.3)](#learning-based-agent-selection-phase-53)
|
- [Learning-Based Agent Selection (Phase 5.3)](#learning-based-agent-selection-phase-53)
|
||||||
- [Budget Enforcement & Cost Optimization (Phase 5.4)](#budget-enforcement--cost-optimization-phase-54)
|
- [Budget Enforcement & Cost Optimization (Phase 5.4)](#budget-enforcement--cost-optimization-phase-54)
|
||||||
- [Workflow Orchestrator (v1.2.0)](#workflow-orchestrator-v120)
|
|
||||||
4. [Knowledge Management](#knowledge-management)
|
4. [Knowledge Management](#knowledge-management)
|
||||||
5. [Cloud-Native & Deployment](#cloud-native--deployment)
|
5. [Cloud-Native & Deployment](#cloud-native--deployment)
|
||||||
6. [Security & Multi-Tenancy](#security--multi-tenancy)
|
6. [Security & Multi-Tenancy](#security--multi-tenancy)
|
||||||
@ -209,22 +208,9 @@ Vapora comes with specialized agents that can be customized, extended, or select
|
|||||||
|
|
||||||
### Agent Orchestration & Workflows
|
### Agent Orchestration & Workflows
|
||||||
|
|
||||||
**Solves**: Dev-Ops Handoff Manual, Task Management Sin Inteligencia, Excessive LLM Costs
|
**Solves**: Dev-Ops Handoff Manual, Task Management Sin Inteligencia
|
||||||
|
|
||||||
Agents work together seamlessly without manual coordination through the **Workflow Orchestrator** (`vapora-workflow-engine`):
|
Agents work together seamlessly without manual coordination:
|
||||||
|
|
||||||
- **Multi-stage workflow execution**:
|
|
||||||
- Pre-configured templates (feature_development, bugfix, documentation_update, security_audit)
|
|
||||||
- Sequential and parallel stage execution
|
|
||||||
- Approval gates for governance and compliance
|
|
||||||
- Artifact passing between stages (ADR, Code, TestResults, Review, Documentation)
|
|
||||||
|
|
||||||
- **Cost-efficient agent coordination**:
|
|
||||||
- Short-lived agent contexts (terminate after task completion)
|
|
||||||
- Context isolation (agents receive only what they need)
|
|
||||||
- Artifact passing instead of conversation accumulation
|
|
||||||
- **~95% reduction in cache token costs** vs monolithic sessions
|
|
||||||
- $840/month → ~$110/month for equivalent workload
|
|
||||||
|
|
||||||
- **Parallel execution**: Multiple agents work on different aspects simultaneously
|
- **Parallel execution**: Multiple agents work on different aspects simultaneously
|
||||||
- Developer writes code while Tester writes tests
|
- Developer writes code while Tester writes tests
|
||||||
@ -244,7 +230,6 @@ Agents work together seamlessly without manual coordination through the **Workfl
|
|||||||
- Security agent approval for sensitive changes
|
- Security agent approval for sensitive changes
|
||||||
- Lead review approval before deployment
|
- Lead review approval before deployment
|
||||||
- Multi-stage review workflows
|
- Multi-stage review workflows
|
||||||
- API/CLI approval commands
|
|
||||||
|
|
||||||
- **Intelligent fallback**:
|
- **Intelligent fallback**:
|
||||||
- If agent fails, escalate or reassign
|
- If agent fails, escalate or reassign
|
||||||
@ -258,11 +243,6 @@ Agents work together seamlessly without manual coordination through the **Workfl
|
|||||||
- Cost-efficient routing with quality/cost ratio optimization
|
- Cost-efficient routing with quality/cost ratio optimization
|
||||||
- Real-time metrics and alerts via Prometheus/Grafana
|
- Real-time metrics and alerts via Prometheus/Grafana
|
||||||
|
|
||||||
- **Kogral integration**:
|
|
||||||
- Context enrichment with guidelines, patterns, and ADRs
|
|
||||||
- Persistent knowledge reduces session context bloat
|
|
||||||
- Filesystem-based retrieval from `.kogral/` directory
|
|
||||||
|
|
||||||
### Learning-Based Agent Selection (Phase 5.3)
|
### Learning-Based Agent Selection (Phase 5.3)
|
||||||
|
|
||||||
**Solves**: Inefficient agent assignment, static task routing
|
**Solves**: Inefficient agent assignment, static task routing
|
||||||
@ -317,61 +297,6 @@ Control costs with intelligent budget management:
|
|||||||
|
|
||||||
- **Real-time monitoring**:
|
- **Real-time monitoring**:
|
||||||
- Prometheus metrics: budget remaining, utilization, fallback triggers
|
- Prometheus metrics: budget remaining, utilization, fallback triggers
|
||||||
|
|
||||||
### Workflow Orchestrator (v1.2.0)
|
|
||||||
|
|
||||||
**Solves**: Excessive LLM cache token costs, monolithic session patterns
|
|
||||||
|
|
||||||
Execute multi-stage pipelines with short-lived agent contexts for cost-efficient workflows:
|
|
||||||
|
|
||||||
- **~95% reduction in cache token costs**:
|
|
||||||
- Monolithic session: ~$840/month (3.82B cache tokens)
|
|
||||||
- Multi-stage workflow: ~$110/month (640K cache tokens)
|
|
||||||
- Agents terminate after task completion, context discarded
|
|
||||||
|
|
||||||
- **Pre-configured workflow templates**:
|
|
||||||
- `feature_development` (5 stages): architecture → implementation (parallel) → testing → review (approval) → deployment (approval)
|
|
||||||
- `bugfix` (4 stages): investigation → fix → testing → deployment
|
|
||||||
- `documentation_update` (3 stages): content → review (approval) → publish
|
|
||||||
- `security_audit` (4 stages): analysis → pentesting → remediation → verification (approval)
|
|
||||||
|
|
||||||
- **Artifact passing between stages**:
|
|
||||||
- ADR (Architecture Decision Record)
|
|
||||||
- Code (source files)
|
|
||||||
- TestResults (execution output)
|
|
||||||
- Review (feedback)
|
|
||||||
- Documentation (generated docs)
|
|
||||||
- Custom (user-defined)
|
|
||||||
|
|
||||||
- **Approval gates for governance**:
|
|
||||||
- Stage pauses until manual approval
|
|
||||||
- API/CLI approval commands
|
|
||||||
- Approver name logged in audit trail
|
|
||||||
- NATS events published (`vapora.workflow.approval_required`)
|
|
||||||
|
|
||||||
- **Kogral integration for context enrichment**:
|
|
||||||
- Guidelines from `.kogral/guidelines/{workflow}.md`
|
|
||||||
- Patterns from `.kogral/patterns/*.md`
|
|
||||||
- Recent ADRs from `.kogral/adrs/*.md` (5 most recent)
|
|
||||||
- Reduces session context by storing knowledge persistently
|
|
||||||
|
|
||||||
- **REST API & CLI**:
|
|
||||||
- Start workflow: `POST /api/v1/workflow_orchestrator` or `vapora workflow start`
|
|
||||||
- List workflows: `GET /api/v1/workflow_orchestrator` or `vapora workflow list`
|
|
||||||
- Get status: `GET /api/v1/workflow_orchestrator/:id` or `vapora workflow status <id>`
|
|
||||||
- Approve stage: `POST /api/v1/workflow_orchestrator/:id/approve` or `vapora workflow approve <id>`
|
|
||||||
- Cancel workflow: `POST /api/v1/workflow_orchestrator/:id/cancel` or `vapora workflow cancel <id>`
|
|
||||||
- List templates: `GET /api/v1/workflow_orchestrator/templates` or `vapora workflow templates`
|
|
||||||
|
|
||||||
- **Prometheus metrics**:
|
|
||||||
- `vapora_workflows_started_total` - Total workflows initiated
|
|
||||||
- `vapora_workflows_completed_total` - Successfully finished
|
|
||||||
- `vapora_workflows_failed_total` - Failed workflows
|
|
||||||
- `vapora_active_workflows` - Currently running (gauge)
|
|
||||||
- `vapora_stage_duration_seconds` - Stage execution time histogram
|
|
||||||
- `vapora_workflow_duration_seconds` - Total workflow time histogram
|
|
||||||
|
|
||||||
**See**: [Workflow Orchestrator Guide](workflow-orchestrator.md) | [CLI Commands](../setup/cli-commands.md) | [ADR-0028](../adrs/0028-workflow-orchestrator.md)
|
|
||||||
- Grafana dashboards: visual budget tracking per role
|
- Grafana dashboards: visual budget tracking per role
|
||||||
- Alerts at 80%, 90%, 100% utilization thresholds
|
- Alerts at 80%, 90%, 100% utilization thresholds
|
||||||
|
|
||||||
|
|||||||
@ -1,538 +0,0 @@
|
|||||||
# Workflow Orchestrator
|
|
||||||
|
|
||||||
Multi-stage workflow execution with cost-efficient agent coordination and artifact passing.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The Workflow Orchestrator (`vapora-workflow-engine`) enables cost-efficient multi-agent pipelines by executing workflows as discrete stages with short-lived agent contexts. Instead of accumulating context in long sessions, agents receive only what they need, produce artifacts, and terminate.
|
|
||||||
|
|
||||||
**Key Benefit**: ~95% reduction in LLM cache token costs compared to monolithic session patterns.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### Core Components
|
|
||||||
|
|
||||||
```text
|
|
||||||
┌─────────────────────────────────────────────────────────┐
|
|
||||||
│ WorkflowOrchestrator │
|
|
||||||
│ ┌─────────────────────────────────────────────────┐ │
|
|
||||||
│ │ WorkflowInstance │ │
|
|
||||||
│ │ ├─ workflow_id: UUID │ │
|
|
||||||
│ │ ├─ template: WorkflowConfig │ │
|
|
||||||
│ │ ├─ current_stage: usize │ │
|
|
||||||
│ │ ├─ stage_states: Vec<StageState> │ │
|
|
||||||
│ │ └─ artifacts: HashMap<String, Artifact> │ │
|
|
||||||
│ └─────────────────────────────────────────────────┘ │
|
|
||||||
└─────────────────────────────────────────────────────────┘
|
|
||||||
│ │ │
|
|
||||||
▼ ▼ ▼
|
|
||||||
┌──────────┐ ┌──────────┐ ┌──────────┐
|
|
||||||
│ NATS │ │ Swarm │ │ KG │
|
|
||||||
│ Listener │ │Coordinator│ │Persistence│
|
|
||||||
└──────────┘ └──────────┘ └──────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
### Workflow Lifecycle
|
|
||||||
|
|
||||||
1. **Template Loading**: Read workflow definition from `config/workflows.toml`
|
|
||||||
2. **Instance Creation**: Create `WorkflowInstance` with initial context
|
|
||||||
3. **Stage Execution**: Orchestrator assigns tasks to agents via SwarmCoordinator
|
|
||||||
4. **Event Listening**: NATS subscribers wait for `TaskCompleted`/`TaskFailed` events
|
|
||||||
5. **Stage Advancement**: When all tasks complete, advance to next stage
|
|
||||||
6. **Artifact Passing**: Accumulated artifacts passed to subsequent stages
|
|
||||||
7. **Completion**: Workflow marked complete, metrics recorded
|
|
||||||
|
|
||||||
## Workflow Templates
|
|
||||||
|
|
||||||
Pre-configured workflows in `config/workflows.toml`:
|
|
||||||
|
|
||||||
### feature_development (5 stages)
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[[workflows]]
|
|
||||||
name = "feature_development"
|
|
||||||
trigger = "manual"
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "architecture_design"
|
|
||||||
agents = ["architect"]
|
|
||||||
parallel = false
|
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "implementation"
|
|
||||||
agents = ["developer", "developer"]
|
|
||||||
parallel = true
|
|
||||||
max_parallel = 2
|
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "testing"
|
|
||||||
agents = ["tester"]
|
|
||||||
parallel = false
|
|
||||||
approval_required = false
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "code_review"
|
|
||||||
agents = ["reviewer"]
|
|
||||||
parallel = false
|
|
||||||
approval_required = true
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "deployment"
|
|
||||||
agents = ["devops"]
|
|
||||||
parallel = false
|
|
||||||
approval_required = true
|
|
||||||
```
|
|
||||||
|
|
||||||
**Stages**: architecture → implementation (parallel) → testing → review (approval) → deployment (approval)
|
|
||||||
|
|
||||||
### bugfix (4 stages)
|
|
||||||
|
|
||||||
**Stages**: investigation → fix → testing → deployment
|
|
||||||
|
|
||||||
### documentation_update (3 stages)
|
|
||||||
|
|
||||||
**Stages**: content creation → review (approval) → publish
|
|
||||||
|
|
||||||
### security_audit (4 stages)
|
|
||||||
|
|
||||||
**Stages**: code analysis → penetration testing → remediation → verification (approval)
|
|
||||||
|
|
||||||
## Stage Types
|
|
||||||
|
|
||||||
### Sequential Stages
|
|
||||||
|
|
||||||
Single agent executes task, advances when complete.
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "architecture_design"
|
|
||||||
agents = ["architect"]
|
|
||||||
parallel = false
|
|
||||||
```
|
|
||||||
|
|
||||||
### Parallel Stages
|
|
||||||
|
|
||||||
Multiple agents execute tasks simultaneously.
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "implementation"
|
|
||||||
agents = ["developer", "developer"]
|
|
||||||
parallel = true
|
|
||||||
max_parallel = 2
|
|
||||||
```
|
|
||||||
|
|
||||||
### Approval Gates
|
|
||||||
|
|
||||||
Stage requires manual approval before advancing.
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "deployment"
|
|
||||||
agents = ["devops"]
|
|
||||||
approval_required = true
|
|
||||||
```
|
|
||||||
|
|
||||||
When `approval_required = true`:
|
|
||||||
|
|
||||||
1. Workflow pauses with status `waiting_approval:<stage_idx>`
|
|
||||||
2. NATS event published to `vapora.workflow.approval_required`
|
|
||||||
3. Admin approves via API or CLI
|
|
||||||
4. Workflow resumes execution
|
|
||||||
|
|
||||||
## Artifacts
|
|
||||||
|
|
||||||
Data passed between stages:
|
|
||||||
|
|
||||||
### Artifact Types
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub enum ArtifactType {
|
|
||||||
Adr, // Architecture Decision Record
|
|
||||||
Code, // Source code files
|
|
||||||
TestResults, // Test execution output
|
|
||||||
Review, // Code review feedback
|
|
||||||
Documentation, // Generated docs
|
|
||||||
Custom(String), // User-defined type
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Artifact Flow
|
|
||||||
|
|
||||||
```text
|
|
||||||
Stage 1: Architecture
|
|
||||||
└─ Produces: Artifact(Adr, "design-spec", ...)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Stage 2: Implementation
|
|
||||||
├─ Consumes: design-spec
|
|
||||||
└─ Produces: Artifact(Code, "feature-impl", ...)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Stage 3: Testing
|
|
||||||
├─ Consumes: feature-impl
|
|
||||||
└─ Produces: Artifact(TestResults, "test-report", ...)
|
|
||||||
```
|
|
||||||
|
|
||||||
Artifacts stored in `WorkflowInstance.accumulated_artifacts` and passed to subsequent stages via context.
|
|
||||||
|
|
||||||
## Kogral Integration
|
|
||||||
|
|
||||||
Enrich workflow context with persistent knowledge from Kogral:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
orchestrator.enrich_context_from_kogral(&mut context, "feature_development").await?;
|
|
||||||
```
|
|
||||||
|
|
||||||
Loads:
|
|
||||||
|
|
||||||
- **Guidelines**: `.kogral/guidelines/{workflow_name}.md`
|
|
||||||
- **Patterns**: `.kogral/patterns/*.md` (matching workflow name)
|
|
||||||
- **ADRs**: `.kogral/adrs/*.md` (5 most recent, containing workflow name)
|
|
||||||
|
|
||||||
Result injected into context:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"task": "Add authentication",
|
|
||||||
"kogral_guidelines": {
|
|
||||||
"source": ".kogral/guidelines/feature_development.md",
|
|
||||||
"content": "..."
|
|
||||||
},
|
|
||||||
"kogral_patterns": [
|
|
||||||
{ "file": "auth-pattern.md", "content": "..." }
|
|
||||||
],
|
|
||||||
"kogral_decisions": [
|
|
||||||
{ "file": "0005-oauth2-implementation.md", "content": "..." }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Configuration**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export KOGRAL_PATH="/path/to/kogral/.kogral"
|
|
||||||
```
|
|
||||||
|
|
||||||
Default: `../kogral/.kogral` (sibling directory)
|
|
||||||
|
|
||||||
## REST API
|
|
||||||
|
|
||||||
All endpoints under `/api/v1/workflow_orchestrator`:
|
|
||||||
|
|
||||||
### Start Workflow
|
|
||||||
|
|
||||||
```http
|
|
||||||
POST /api/v1/workflow_orchestrator
|
|
||||||
Content-Type: application/json
|
|
||||||
|
|
||||||
{
|
|
||||||
"template": "feature_development",
|
|
||||||
"context": {
|
|
||||||
"task": "Implement authentication",
|
|
||||||
"requirements": ["OAuth2", "JWT"]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Response**:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"workflow_id": "3f9a2b1c-5e7f-4a9b-8c2d-1e3f5a7b9c1d"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### List Active Workflows
|
|
||||||
|
|
||||||
```http
|
|
||||||
GET /api/v1/workflow_orchestrator
|
|
||||||
```
|
|
||||||
|
|
||||||
**Response**:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"workflows": [
|
|
||||||
{
|
|
||||||
"id": "3f9a2b1c-5e7f-4a9b-8c2d-1e3f5a7b9c1d",
|
|
||||||
"template_name": "feature_development",
|
|
||||||
"status": "running",
|
|
||||||
"current_stage": 2,
|
|
||||||
"total_stages": 5,
|
|
||||||
"created_at": "2026-01-24T01:23:45.123Z",
|
|
||||||
"updated_at": "2026-01-24T01:45:12.456Z"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Get Workflow Status
|
|
||||||
|
|
||||||
```http
|
|
||||||
GET /api/v1/workflow_orchestrator/:id
|
|
||||||
```
|
|
||||||
|
|
||||||
**Response**: Same as workflow object in list response
|
|
||||||
|
|
||||||
### Approve Stage
|
|
||||||
|
|
||||||
```http
|
|
||||||
POST /api/v1/workflow_orchestrator/:id/approve
|
|
||||||
Content-Type: application/json
|
|
||||||
|
|
||||||
{
|
|
||||||
"approver": "Jane Doe"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Response**:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"success": true,
|
|
||||||
"message": "Workflow 3f9a2b1c stage approved"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Cancel Workflow
|
|
||||||
|
|
||||||
```http
|
|
||||||
POST /api/v1/workflow_orchestrator/:id/cancel
|
|
||||||
Content-Type: application/json
|
|
||||||
|
|
||||||
{
|
|
||||||
"reason": "Requirements changed"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Response**:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"success": true,
|
|
||||||
"message": "Workflow 3f9a2b1c cancelled"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### List Templates
|
|
||||||
|
|
||||||
```http
|
|
||||||
GET /api/v1/workflow_orchestrator/templates
|
|
||||||
```
|
|
||||||
|
|
||||||
**Response**:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"templates": [
|
|
||||||
"feature_development",
|
|
||||||
"bugfix",
|
|
||||||
"documentation_update",
|
|
||||||
"security_audit"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## NATS Events
|
|
||||||
|
|
||||||
Workflow orchestrator publishes/subscribes to NATS JetStream:
|
|
||||||
|
|
||||||
### Subscriptions
|
|
||||||
|
|
||||||
- `vapora.tasks.completed` - Agent task completion events
|
|
||||||
- `vapora.tasks.failed` - Agent task failure events
|
|
||||||
|
|
||||||
### Publications
|
|
||||||
|
|
||||||
- `vapora.workflow.approval_required` - Stage waiting for approval
|
|
||||||
- `vapora.workflow.completed` - Workflow finished successfully
|
|
||||||
|
|
||||||
**Event Format**:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"type": "approval_required",
|
|
||||||
"workflow_id": "3f9a2b1c-5e7f-4a9b-8c2d-1e3f5a7b9c1d",
|
|
||||||
"stage": "code_review",
|
|
||||||
"timestamp": "2026-01-24T01:45:12.456Z"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Metrics
|
|
||||||
|
|
||||||
Prometheus metrics exposed at `/metrics`:
|
|
||||||
|
|
||||||
- `vapora_workflows_started_total` - Total workflows initiated
|
|
||||||
- `vapora_workflows_completed_total` - Successfully finished workflows
|
|
||||||
- `vapora_workflows_failed_total` - Failed workflows
|
|
||||||
- `vapora_stages_completed_total` - Individual stage completions
|
|
||||||
- `vapora_active_workflows` - Currently running workflows (gauge)
|
|
||||||
- `vapora_stage_duration_seconds` - Histogram of stage execution times
|
|
||||||
- `vapora_workflow_duration_seconds` - Histogram of total workflow times
|
|
||||||
|
|
||||||
## Cost Optimization
|
|
||||||
|
|
||||||
### Before: Monolithic Session
|
|
||||||
|
|
||||||
```text
|
|
||||||
Session with 50 messages:
|
|
||||||
├─ Message 1: 50K context → 50K cache reads
|
|
||||||
├─ Message 2: 100K context → 100K cache reads
|
|
||||||
├─ Message 3: 150K context → 150K cache reads
|
|
||||||
└─ Message 50: 800K context → 800K cache reads
|
|
||||||
──────────────────
|
|
||||||
~20M cache reads
|
|
||||||
```
|
|
||||||
|
|
||||||
**Cost**: ~$840/month for typical usage
|
|
||||||
|
|
||||||
### After: Multi-Stage Workflow
|
|
||||||
|
|
||||||
```text
|
|
||||||
Workflow with 3 stages:
|
|
||||||
├─ Architect: 40K context, 5 msgs → 200K cache reads
|
|
||||||
├─ Developer: 25K context, 12 msgs → 300K cache reads
|
|
||||||
└─ Reviewer: 35K context, 4 msgs → 140K cache reads
|
|
||||||
──────────────────
|
|
||||||
~640K cache reads
|
|
||||||
```
|
|
||||||
|
|
||||||
**Cost**: ~$110/month for equivalent work
|
|
||||||
|
|
||||||
**Savings**: ~$730/month (87% reduction)
|
|
||||||
|
|
||||||
## Usage Examples
|
|
||||||
|
|
||||||
See [CLI Commands Guide](../setup/cli-commands.md) for command-line usage.
|
|
||||||
|
|
||||||
### Programmatic Usage
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use vapora_workflow_engine::WorkflowOrchestrator;
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
// Initialize orchestrator
|
|
||||||
let orchestrator = Arc::new(
|
|
||||||
WorkflowOrchestrator::new(
|
|
||||||
"config/workflows.toml",
|
|
||||||
swarm,
|
|
||||||
kg,
|
|
||||||
nats,
|
|
||||||
).await?
|
|
||||||
);
|
|
||||||
|
|
||||||
// Start event listener
|
|
||||||
orchestrator.clone().start_event_listener().await?;
|
|
||||||
|
|
||||||
// Start workflow
|
|
||||||
let workflow_id = orchestrator.start_workflow(
|
|
||||||
"feature_development",
|
|
||||||
serde_json::json!({
|
|
||||||
"task": "Add authentication",
|
|
||||||
"requirements": ["OAuth2", "JWT"]
|
|
||||||
})
|
|
||||||
).await?;
|
|
||||||
|
|
||||||
// Get status
|
|
||||||
let workflow = orchestrator.get_workflow(&workflow_id)?;
|
|
||||||
println!("Status: {:?}", workflow.status);
|
|
||||||
|
|
||||||
// Approve stage (if waiting)
|
|
||||||
orchestrator.approve_stage(&workflow_id, "Jane Doe").await?;
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
### Workflow Templates
|
|
||||||
|
|
||||||
File: `config/workflows.toml`
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[engine]
|
|
||||||
max_parallel_tasks = 10
|
|
||||||
workflow_timeout = 3600
|
|
||||||
approval_gates_enabled = true
|
|
||||||
|
|
||||||
[[workflows]]
|
|
||||||
name = "custom_workflow"
|
|
||||||
trigger = "manual"
|
|
||||||
|
|
||||||
[[workflows.stages]]
|
|
||||||
name = "stage_name"
|
|
||||||
agents = ["agent_role"]
|
|
||||||
parallel = false
|
|
||||||
max_parallel = 1
|
|
||||||
approval_required = false
|
|
||||||
```
|
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Kogral knowledge base path
|
|
||||||
export KOGRAL_PATH="/path/to/kogral/.kogral"
|
|
||||||
|
|
||||||
# NATS connection
|
|
||||||
export NATS_URL="nats://localhost:4222"
|
|
||||||
|
|
||||||
# Backend API (for CLI)
|
|
||||||
export VAPORA_API_URL="http://localhost:8001"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Workflow Stuck in "waiting_approval"
|
|
||||||
|
|
||||||
**Solution**: Use CLI or API to approve:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow approve <workflow_id> --approver "Your Name"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Stage Fails Repeatedly
|
|
||||||
|
|
||||||
**Check**:
|
|
||||||
|
|
||||||
1. Agent availability: `vapora workflow list` (via backend)
|
|
||||||
2. NATS connection: Verify NATS URL and cluster status
|
|
||||||
3. Task requirements: Check if stage agents have required capabilities
|
|
||||||
|
|
||||||
### High Latency Between Stages
|
|
||||||
|
|
||||||
**Causes**:
|
|
||||||
|
|
||||||
- NATS messaging delay (check network)
|
|
||||||
- SwarmCoordinator queue depth (check agent load)
|
|
||||||
- Artifact serialization overhead (reduce artifact size)
|
|
||||||
|
|
||||||
**Mitigation**:
|
|
||||||
|
|
||||||
- Use parallel stages where possible
|
|
||||||
- Increase `max_parallel` in stage config
|
|
||||||
- Optimize artifact content (references instead of full content)
|
|
||||||
|
|
||||||
### Workflow Not Advancing
|
|
||||||
|
|
||||||
**Debug**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check workflow status
|
|
||||||
vapora workflow status <workflow_id>
|
|
||||||
|
|
||||||
# Check backend logs
|
|
||||||
docker logs vapora-backend
|
|
||||||
|
|
||||||
# Check NATS messages
|
|
||||||
nats sub "vapora.tasks.>"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Related Documentation
|
|
||||||
|
|
||||||
- [CLI Commands Guide](../setup/cli-commands.md) - Command-line usage
|
|
||||||
- [Multi-Agent Workflows](../architecture/multi-agent-workflows.md) - Architecture overview
|
|
||||||
- [Agent Registry & Coordination](../architecture/agent-registry-coordination.md) - Agent management
|
|
||||||
- [ADR-0028: Workflow Orchestrator](../adrs/0028-workflow-orchestrator.md) - Decision rationale
|
|
||||||
- [ADR-0014: Learning-Based Agent Selection](../adrs/0014-learning-profiles.md) - Agent selection
|
|
||||||
- [ADR-0015: Budget Enforcement](../adrs/0015-budget-enforcement.md) - Cost control
|
|
||||||
@ -1,614 +0,0 @@
|
|||||||
# CLI Commands Reference
|
|
||||||
|
|
||||||
Command-line interface for VAPORA workflow management.
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
### Build from Source
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd crates/vapora-cli
|
|
||||||
cargo build --release
|
|
||||||
```
|
|
||||||
|
|
||||||
Binary location: `target/release/vapora`
|
|
||||||
|
|
||||||
### Add to PATH
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Copy to local bin
|
|
||||||
cp target/release/vapora ~/.local/bin/
|
|
||||||
|
|
||||||
# Or symlink
|
|
||||||
ln -s $(pwd)/target/release/vapora ~/.local/bin/vapora
|
|
||||||
```
|
|
||||||
|
|
||||||
### Verify Installation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora --version
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Backend API URL (default: http://localhost:8001)
|
|
||||||
export VAPORA_API_URL="http://localhost:8001"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Command-Line Flags
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Override API URL per command
|
|
||||||
vapora --api-url http://production:8001 workflow list
|
|
||||||
```
|
|
||||||
|
|
||||||
## Commands
|
|
||||||
|
|
||||||
### vapora workflow
|
|
||||||
|
|
||||||
Workflow orchestration commands.
|
|
||||||
|
|
||||||
#### start
|
|
||||||
|
|
||||||
Start a new workflow from template.
|
|
||||||
|
|
||||||
**Usage**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow start --template <TEMPLATE> [--context <FILE>] [--kogral <BOOL>]
|
|
||||||
```
|
|
||||||
|
|
||||||
**Arguments**:
|
|
||||||
|
|
||||||
- `-t, --template <TEMPLATE>` - Workflow template name (required)
|
|
||||||
- `-c, --context <FILE>` - Initial context JSON file (optional)
|
|
||||||
- `--kogral <BOOL>` - Enrich with Kogral knowledge (default: true)
|
|
||||||
|
|
||||||
**Examples**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Start feature development workflow
|
|
||||||
vapora workflow start --template feature_development
|
|
||||||
|
|
||||||
# Start with context file
|
|
||||||
vapora workflow start \
|
|
||||||
--template feature_development \
|
|
||||||
--context context.json
|
|
||||||
|
|
||||||
# Start without Kogral enrichment
|
|
||||||
vapora workflow start \
|
|
||||||
--template bugfix \
|
|
||||||
--kogral false
|
|
||||||
```
|
|
||||||
|
|
||||||
**Context File Format** (`context.json`):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"task": "Implement user authentication",
|
|
||||||
"requirements": ["OAuth2", "JWT", "MFA"],
|
|
||||||
"priority": "high"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Output**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
✓ Workflow started: feature_development (ID: 3f9a2b1c)
|
|
||||||
```
|
|
||||||
|
|
||||||
#### list
|
|
||||||
|
|
||||||
List all active workflows.
|
|
||||||
|
|
||||||
**Usage**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow list
|
|
||||||
```
|
|
||||||
|
|
||||||
**Output**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
╔════════════╦════════════════════╦════════════════╦══════════╦═════════════════════╗
|
|
||||||
║ ID ║ Template ║ Status ║ Progress ║ Created ║
|
|
||||||
╠════════════╬════════════════════╬════════════════╬══════════╬═════════════════════╣
|
|
||||||
║ 3f9a2b1c ║ feature_development║ running ║ 2/5 ║ 2026-01-24 01:23:45 ║
|
|
||||||
║ 7d8e3c4a ║ bugfix ║ completed ║ 4/4 ║ 2026-01-24 00:15:32 ║
|
|
||||||
╚════════════╩════════════════════╩════════════════╩══════════╩═════════════════════╝
|
|
||||||
```
|
|
||||||
|
|
||||||
**Status Colors**:
|
|
||||||
|
|
||||||
- **Green**: `running` - Workflow executing
|
|
||||||
- **Yellow**: `waiting_approval` - Stage requires approval
|
|
||||||
- **Blue**: `completed` - Workflow finished successfully
|
|
||||||
- **Red**: `failed` - Workflow encountered error
|
|
||||||
|
|
||||||
#### status
|
|
||||||
|
|
||||||
Get detailed workflow status.
|
|
||||||
|
|
||||||
**Usage**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow status <WORKFLOW_ID>
|
|
||||||
```
|
|
||||||
|
|
||||||
**Arguments**:
|
|
||||||
|
|
||||||
- `<WORKFLOW_ID>` - Workflow identifier (required)
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow status 3f9a2b1c
|
|
||||||
```
|
|
||||||
|
|
||||||
**Output**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Workflow Details
|
|
||||||
────────────────────────────────────────────────────────────
|
|
||||||
ID: 3f9a2b1c-5e7f-4a9b-8c2d-1e3f5a7b9c1d
|
|
||||||
Template: feature_development
|
|
||||||
Status: running
|
|
||||||
Progress: 2/5
|
|
||||||
Created: 2026-01-24T01:23:45.123Z
|
|
||||||
Updated: 2026-01-24T01:45:12.456Z
|
|
||||||
────────────────────────────────────────────────────────────
|
|
||||||
```
|
|
||||||
|
|
||||||
#### approve
|
|
||||||
|
|
||||||
Approve a stage waiting for approval.
|
|
||||||
|
|
||||||
**Usage**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow approve <WORKFLOW_ID> --approver <NAME>
|
|
||||||
```
|
|
||||||
|
|
||||||
**Arguments**:
|
|
||||||
|
|
||||||
- `<WORKFLOW_ID>` - Workflow identifier (required)
|
|
||||||
- `-a, --approver <NAME>` - Approver name (required)
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow approve 3f9a2b1c --approver "Jane Doe"
|
|
||||||
```
|
|
||||||
|
|
||||||
**Output**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
✓ Workflow 3f9a2b1c stage approved
|
|
||||||
```
|
|
||||||
|
|
||||||
**Notes**:
|
|
||||||
|
|
||||||
- Workflow must be in `waiting_approval` status
|
|
||||||
- Approver name logged in audit trail
|
|
||||||
- Workflow resumes execution immediately
|
|
||||||
|
|
||||||
#### cancel
|
|
||||||
|
|
||||||
Cancel a running workflow.
|
|
||||||
|
|
||||||
**Usage**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow cancel <WORKFLOW_ID> --reason <REASON>
|
|
||||||
```
|
|
||||||
|
|
||||||
**Arguments**:
|
|
||||||
|
|
||||||
- `<WORKFLOW_ID>` - Workflow identifier (required)
|
|
||||||
- `-r, --reason <REASON>` - Cancellation reason (required)
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow cancel 3f9a2b1c --reason "Requirements changed"
|
|
||||||
```
|
|
||||||
|
|
||||||
**Output**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
✓ Workflow 3f9a2b1c cancelled
|
|
||||||
```
|
|
||||||
|
|
||||||
**Notes**:
|
|
||||||
|
|
||||||
- Cancels workflow immediately
|
|
||||||
- In-flight tasks may complete
|
|
||||||
- Reason logged in audit trail
|
|
||||||
|
|
||||||
#### templates
|
|
||||||
|
|
||||||
List available workflow templates.
|
|
||||||
|
|
||||||
**Usage**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vapora workflow templates
|
|
||||||
```
|
|
||||||
|
|
||||||
**Output**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Available Workflow Templates
|
|
||||||
────────────────────────────────────────────────────────────
|
|
||||||
1. feature_development
|
|
||||||
2. bugfix
|
|
||||||
3. documentation_update
|
|
||||||
4. security_audit
|
|
||||||
────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
Use vapora workflow start --template <name> to start a workflow
|
|
||||||
```
|
|
||||||
|
|
||||||
## Workflow Templates
|
|
||||||
|
|
||||||
### feature_development
|
|
||||||
|
|
||||||
5-stage workflow for implementing new features.
|
|
||||||
|
|
||||||
**Stages**:
|
|
||||||
|
|
||||||
1. **architecture_design** (architect)
|
|
||||||
2. **implementation** (2x developer, parallel)
|
|
||||||
3. **testing** (tester)
|
|
||||||
4. **code_review** (reviewer, approval required)
|
|
||||||
5. **deployment** (devops, approval required)
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create context
|
|
||||||
cat > feature.json <<EOF
|
|
||||||
{
|
|
||||||
"task": "Add user authentication",
|
|
||||||
"requirements": ["OAuth2", "JWT", "MFA"],
|
|
||||||
"technologies": ["Rust", "axum", "SurrealDB"]
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
# Start workflow
|
|
||||||
vapora workflow start \
|
|
||||||
--template feature_development \
|
|
||||||
--context feature.json
|
|
||||||
|
|
||||||
# Monitor progress
|
|
||||||
vapora workflow list
|
|
||||||
|
|
||||||
# Approve code review stage (when ready)
|
|
||||||
vapora workflow approve <id> --approver "Tech Lead"
|
|
||||||
|
|
||||||
# Approve deployment stage (when ready)
|
|
||||||
vapora workflow approve <id> --approver "Release Manager"
|
|
||||||
```
|
|
||||||
|
|
||||||
### bugfix
|
|
||||||
|
|
||||||
4-stage workflow for fixing bugs.
|
|
||||||
|
|
||||||
**Stages**:
|
|
||||||
|
|
||||||
1. **investigation** (developer)
|
|
||||||
2. **fix_implementation** (developer)
|
|
||||||
3. **testing** (tester)
|
|
||||||
4. **deployment** (devops)
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cat > bugfix.json <<EOF
|
|
||||||
{
|
|
||||||
"bug": "Authentication fails on mobile devices",
|
|
||||||
"severity": "high",
|
|
||||||
"affected_users": 500
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
vapora workflow start --template bugfix --context bugfix.json
|
|
||||||
```
|
|
||||||
|
|
||||||
### documentation_update
|
|
||||||
|
|
||||||
3-stage workflow for documentation changes.
|
|
||||||
|
|
||||||
**Stages**:
|
|
||||||
|
|
||||||
1. **content_creation** (technical_writer)
|
|
||||||
2. **review** (reviewer, approval required)
|
|
||||||
3. **publish** (devops)
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cat > docs.json <<EOF
|
|
||||||
{
|
|
||||||
"topic": "API Authentication Guide",
|
|
||||||
"sections": ["Setup", "OAuth2 Flow", "JWT Tokens"],
|
|
||||||
"format": "markdown"
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
vapora workflow start --template documentation_update --context docs.json
|
|
||||||
```
|
|
||||||
|
|
||||||
### security_audit
|
|
||||||
|
|
||||||
4-stage workflow for security reviews.
|
|
||||||
|
|
||||||
**Stages**:
|
|
||||||
|
|
||||||
1. **code_analysis** (security_engineer)
|
|
||||||
2. **penetration_testing** (security_engineer)
|
|
||||||
3. **remediation** (developer)
|
|
||||||
4. **verification** (security_engineer, approval required)
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cat > security.json <<EOF
|
|
||||||
{
|
|
||||||
"scope": "Authentication module",
|
|
||||||
"compliance": ["OWASP Top 10", "SOC 2"],
|
|
||||||
"priority": "critical"
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
vapora workflow start --template security_audit --context security.json
|
|
||||||
```
|
|
||||||
|
|
||||||
## Common Workflows
|
|
||||||
|
|
||||||
### Check Workflow Status
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# List all workflows
|
|
||||||
vapora workflow list
|
|
||||||
|
|
||||||
# Get specific workflow details
|
|
||||||
vapora workflow status <id>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Approve Multi-Stage Workflow
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Start workflow
|
|
||||||
ID=$(vapora workflow start --template feature_development \
|
|
||||||
--context context.json | grep -oE '[0-9a-f-]{36}')
|
|
||||||
|
|
||||||
# Monitor until waiting for approval
|
|
||||||
watch -n 5 vapora workflow status $ID
|
|
||||||
|
|
||||||
# Approve when ready
|
|
||||||
vapora workflow approve $ID --approver "$(whoami)"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Cancel Stuck Workflow
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Find workflow
|
|
||||||
vapora workflow list
|
|
||||||
|
|
||||||
# Cancel with reason
|
|
||||||
vapora workflow cancel <id> --reason "Timeout exceeded"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Template Discovery
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# List available templates
|
|
||||||
vapora workflow templates
|
|
||||||
|
|
||||||
# Start specific template
|
|
||||||
vapora workflow start --template <name>
|
|
||||||
```
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
|
|
||||||
### Workflow Not Found
|
|
||||||
|
|
||||||
```text
|
|
||||||
✗ Workflow not found: abc123
|
|
||||||
```
|
|
||||||
|
|
||||||
**Cause**: Invalid workflow ID
|
|
||||||
|
|
||||||
**Solution**: Verify ID with `vapora workflow list`
|
|
||||||
|
|
||||||
### API Connection Failed
|
|
||||||
|
|
||||||
```text
|
|
||||||
✗ API request failed: HTTP 500
|
|
||||||
```
|
|
||||||
|
|
||||||
**Cause**: Backend not running or network issue
|
|
||||||
|
|
||||||
**Solution**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check backend status
|
|
||||||
curl http://localhost:8001/health
|
|
||||||
|
|
||||||
# Verify API URL
|
|
||||||
echo $VAPORA_API_URL
|
|
||||||
|
|
||||||
# Check backend logs
|
|
||||||
docker logs vapora-backend
|
|
||||||
```
|
|
||||||
|
|
||||||
### Invalid Template
|
|
||||||
|
|
||||||
```text
|
|
||||||
✗ API request failed: HTTP 404
|
|
||||||
```
|
|
||||||
|
|
||||||
**Cause**: Template name doesn't exist
|
|
||||||
|
|
||||||
**Solution**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# List available templates
|
|
||||||
vapora workflow templates
|
|
||||||
|
|
||||||
# Use exact template name
|
|
||||||
vapora workflow start --template feature_development
|
|
||||||
```
|
|
||||||
|
|
||||||
### Approval Not Allowed
|
|
||||||
|
|
||||||
```text
|
|
||||||
✗ API request failed: Stage not waiting for approval
|
|
||||||
```
|
|
||||||
|
|
||||||
**Cause**: Workflow not in `waiting_approval` status
|
|
||||||
|
|
||||||
**Solution**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check workflow status
|
|
||||||
vapora workflow status <id>
|
|
||||||
|
|
||||||
# Wait for status to change to "waiting_approval"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Advanced Usage
|
|
||||||
|
|
||||||
### Custom API URL
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Production environment
|
|
||||||
vapora --api-url https://vapora.example.com workflow list
|
|
||||||
|
|
||||||
# Local development with custom port
|
|
||||||
vapora --api-url http://localhost:9000 workflow start \
|
|
||||||
--template feature_development
|
|
||||||
```
|
|
||||||
|
|
||||||
### Scripting Workflows
|
|
||||||
|
|
||||||
```bash
|
|
||||||
#!/bin/bash
|
|
||||||
set -e
|
|
||||||
|
|
||||||
# Start workflow and capture ID
|
|
||||||
WORKFLOW_ID=$(vapora workflow start \
|
|
||||||
--template feature_development \
|
|
||||||
--context feature.json \
|
|
||||||
| grep -oE '[0-9a-f-]{36}')
|
|
||||||
|
|
||||||
echo "Started workflow: $WORKFLOW_ID"
|
|
||||||
|
|
||||||
# Poll until completed or failed
|
|
||||||
while true; do
|
|
||||||
STATUS=$(vapora workflow status $WORKFLOW_ID | grep "Status:" | awk '{print $2}')
|
|
||||||
|
|
||||||
if [[ "$STATUS" == "completed" ]]; then
|
|
||||||
echo "Workflow completed successfully"
|
|
||||||
exit 0
|
|
||||||
elif [[ "$STATUS" == "failed"* ]]; then
|
|
||||||
echo "Workflow failed: $STATUS"
|
|
||||||
exit 1
|
|
||||||
elif [[ "$STATUS" == "waiting_approval"* ]]; then
|
|
||||||
echo "Workflow waiting for approval"
|
|
||||||
vapora workflow approve $WORKFLOW_ID --approver "CI/CD Bot"
|
|
||||||
fi
|
|
||||||
|
|
||||||
sleep 10
|
|
||||||
done
|
|
||||||
```
|
|
||||||
|
|
||||||
### JSON Context Generation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Generate context from git commit
|
|
||||||
cat > context.json <<EOF
|
|
||||||
{
|
|
||||||
"task": "Fix bug from commit",
|
|
||||||
"commit": "$(git log -1 --format=%H)",
|
|
||||||
"message": "$(git log -1 --format=%s)",
|
|
||||||
"author": "$(git log -1 --format=%an)",
|
|
||||||
"files": $(git show --name-only --format= | jq -R . | jq -s .)
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
vapora workflow start --template bugfix --context context.json
|
|
||||||
```
|
|
||||||
|
|
||||||
### CI/CD Integration
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# .github/workflows/vapora-deploy.yml
|
|
||||||
name: VAPORA Deployment
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
deploy:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v3
|
|
||||||
|
|
||||||
- name: Install VAPORA CLI
|
|
||||||
run: |
|
|
||||||
curl -L https://github.com/vapora/vapora/releases/latest/download/vapora-cli -o vapora
|
|
||||||
chmod +x vapora
|
|
||||||
sudo mv vapora /usr/local/bin/
|
|
||||||
|
|
||||||
- name: Start deployment workflow
|
|
||||||
env:
|
|
||||||
VAPORA_API_URL: ${{ secrets.VAPORA_API_URL }}
|
|
||||||
run: |
|
|
||||||
vapora workflow start \
|
|
||||||
--template feature_development \
|
|
||||||
--context .github/workflows/context.json
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Command Not Found
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Verify installation
|
|
||||||
which vapora
|
|
||||||
|
|
||||||
# Add to PATH
|
|
||||||
export PATH="$HOME/.local/bin:$PATH"
|
|
||||||
|
|
||||||
# Or use full path
|
|
||||||
/path/to/vapora workflow list
|
|
||||||
```
|
|
||||||
|
|
||||||
### Permission Denied
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Make executable
|
|
||||||
chmod +x /path/to/vapora
|
|
||||||
|
|
||||||
# Or rebuild
|
|
||||||
cargo build --release
|
|
||||||
```
|
|
||||||
|
|
||||||
### SSL Certificate Error
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# For self-signed certificates (development only)
|
|
||||||
export VAPORA_SKIP_TLS_VERIFY=true
|
|
||||||
```
|
|
||||||
|
|
||||||
## Related Documentation
|
|
||||||
|
|
||||||
- [Workflow Orchestrator](../features/workflow-orchestrator.md) - Architecture and API
|
|
||||||
- [Multi-Agent Workflows](../architecture/multi-agent-workflows.md) - Design overview
|
|
||||||
- [ADR-0028: Workflow Orchestrator](../adrs/0028-workflow-orchestrator.md) - Decision rationale
|
|
||||||
- [Deployment Guide](deployment.md) - Production deployment
|
|
||||||
@ -1,128 +0,0 @@
|
|||||||
# VAPORA Validation Schemas
|
|
||||||
|
|
||||||
Nickel schemas for runtime validation of MCP tools and agent tasks.
|
|
||||||
|
|
||||||
## Directory Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
schemas/
|
|
||||||
├── tools/ # MCP tool parameter validation
|
|
||||||
│ ├── kanban_create_task.ncl
|
|
||||||
│ ├── kanban_update_task.ncl
|
|
||||||
│ ├── assign_task_to_agent.ncl
|
|
||||||
│ ├── get_project_summary.ncl
|
|
||||||
│ └── get_agent_capabilities.ncl
|
|
||||||
│
|
|
||||||
└── agents/ # Agent task assignment validation
|
|
||||||
└── task_assignment.ncl
|
|
||||||
```
|
|
||||||
|
|
||||||
## Schema Format
|
|
||||||
|
|
||||||
Nickel schemas define:
|
|
||||||
|
|
||||||
- **Field types** (String, Number, Bool, Array, Object)
|
|
||||||
- **Required/optional fields** (via `default` values)
|
|
||||||
- **Validation contracts** (NonEmpty, Email, UUID, Range, Pattern, etc.)
|
|
||||||
- **Documentation** (via `doc` annotations)
|
|
||||||
|
|
||||||
### Example
|
|
||||||
|
|
||||||
```nickel
|
|
||||||
{
|
|
||||||
tool_name = "example_tool",
|
|
||||||
|
|
||||||
parameters = {
|
|
||||||
# Required field with contracts
|
|
||||||
user_id
|
|
||||||
| String
|
|
||||||
| doc "User UUID"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.match "^[0-9a-f]{8}-[0-9a-f]{4}-...$",
|
|
||||||
|
|
||||||
# Optional field with default
|
|
||||||
priority
|
|
||||||
| Number
|
|
||||||
| doc "Priority score (0-100)"
|
|
||||||
| std.number.between 0 100
|
|
||||||
| default = 50,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Supported Contracts
|
|
||||||
|
|
||||||
| Contract | Description | Example |
|
|
||||||
|----------|-------------|---------|
|
|
||||||
| `std.string.NonEmpty` | String cannot be empty | Required text fields |
|
|
||||||
| `std.string.length.min N` | Minimum length | `min 3` for titles |
|
|
||||||
| `std.string.length.max N` | Maximum length | `max 200` for titles |
|
|
||||||
| `std.string.match PATTERN` | Regex validation | UUID format |
|
|
||||||
| `std.number.between A B` | Numeric range | `between 0 100` |
|
|
||||||
| `std.number.greater_than N` | Minimum value (exclusive) | `> -1` |
|
|
||||||
| `std.number.less_than N` | Maximum value (exclusive) | `< 1000` |
|
|
||||||
| `std.string.Email` | Email format | user@example.com |
|
|
||||||
| `std.string.Url` | URL format | https://... |
|
|
||||||
| `std.string.Uuid` | UUID format | xxxxxxxx-xxxx-... |
|
|
||||||
| `std.enum.TaggedUnion` | Enum validation | `[| 'low, 'high |]` |
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
Schemas are loaded by `ValidationPipeline` at runtime:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use vapora_shared::validation::{SchemaRegistry, ValidationPipeline};
|
|
||||||
|
|
||||||
let registry = Arc::new(SchemaRegistry::new(PathBuf::from("schemas")));
|
|
||||||
let pipeline = ValidationPipeline::new(registry);
|
|
||||||
|
|
||||||
// Validate MCP tool input
|
|
||||||
let result = pipeline.validate("tools/kanban_create_task", &input).await?;
|
|
||||||
|
|
||||||
if !result.valid {
|
|
||||||
return (StatusCode::BAD_REQUEST, Json(result.errors));
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing Schemas
|
|
||||||
|
|
||||||
Validate schema syntax:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nickel typecheck schemas/tools/kanban_create_task.ncl
|
|
||||||
```
|
|
||||||
|
|
||||||
Export schema as JSON:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nickel export schemas/tools/kanban_create_task.ncl
|
|
||||||
```
|
|
||||||
|
|
||||||
Query specific field:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nickel query --field parameters.title schemas/tools/kanban_create_task.ncl
|
|
||||||
```
|
|
||||||
|
|
||||||
## Adding New Schemas
|
|
||||||
|
|
||||||
1. Create `.ncl` file in appropriate directory
|
|
||||||
2. Define `tool_name` or `schema_name`
|
|
||||||
3. Define `parameters` or `fields` with types and contracts
|
|
||||||
4. Add `doc` annotations for documentation
|
|
||||||
5. Test with `nickel typecheck`
|
|
||||||
6. Restart services to reload schemas (or use hot-reload)
|
|
||||||
|
|
||||||
## Hot Reload
|
|
||||||
|
|
||||||
Invalidate cached schema without restart:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
registry.invalidate("tools/kanban_create_task").await;
|
|
||||||
```
|
|
||||||
|
|
||||||
Invalidate all schemas:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
registry.invalidate_all().await;
|
|
||||||
```
|
|
||||||
@ -1,71 +0,0 @@
|
|||||||
# Schema for agent task assignment
|
|
||||||
# Used by AgentCoordinator for internal task assignment validation
|
|
||||||
{
|
|
||||||
schema_name = "task_assignment",
|
|
||||||
|
|
||||||
description = "Validates agent task assignment requests from AgentCoordinator",
|
|
||||||
|
|
||||||
fields = {
|
|
||||||
# Agent role - must match registered roles
|
|
||||||
role
|
|
||||||
| String
|
|
||||||
| doc "Agent role for task execution"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.enum.TaggedUnion
|
|
||||||
| [| 'developer, 'reviewer, 'architect, 'tester, 'documenter, 'devops, 'monitor, 'security |],
|
|
||||||
|
|
||||||
# Task title
|
|
||||||
title
|
|
||||||
| String
|
|
||||||
| doc "Task title (3-500 chars)"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.length.min 3
|
|
||||||
| std.string.length.max 500,
|
|
||||||
|
|
||||||
# Task description
|
|
||||||
description
|
|
||||||
| String
|
|
||||||
| doc "Detailed task description"
|
|
||||||
| default = "",
|
|
||||||
|
|
||||||
# Task context (JSON string)
|
|
||||||
context
|
|
||||||
| String
|
|
||||||
| doc "JSON context for task execution"
|
|
||||||
| default = "{}",
|
|
||||||
|
|
||||||
# Priority score (0-100)
|
|
||||||
priority
|
|
||||||
| Number
|
|
||||||
| doc "Task priority score (0=lowest, 100=highest)"
|
|
||||||
| std.number.between 0 100
|
|
||||||
| default = 50,
|
|
||||||
|
|
||||||
# Optional deadline (ISO 8601 datetime string)
|
|
||||||
deadline
|
|
||||||
| String
|
|
||||||
| doc "Task deadline (ISO 8601 format, optional)"
|
|
||||||
| default = "",
|
|
||||||
},
|
|
||||||
|
|
||||||
# Custom validation contracts
|
|
||||||
contracts = {
|
|
||||||
# Title must not be only whitespace
|
|
||||||
title_not_whitespace = fun fields =>
|
|
||||||
std.string.trim fields.title != "",
|
|
||||||
|
|
||||||
# Context must be valid JSON
|
|
||||||
context_valid_json = fun fields =>
|
|
||||||
if fields.context == "{}" then
|
|
||||||
true
|
|
||||||
else
|
|
||||||
std.string.is_match "^[\\{\\[]" fields.context,
|
|
||||||
|
|
||||||
# If deadline is provided, must match ISO 8601 pattern
|
|
||||||
deadline_iso8601 = fun fields =>
|
|
||||||
if fields.deadline == "" then
|
|
||||||
true
|
|
||||||
else
|
|
||||||
std.string.is_match "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}" fields.deadline,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
@ -1,56 +0,0 @@
|
|||||||
# Schema for assign_task_to_agent MCP tool
|
|
||||||
# Validates agent task assignment requests
|
|
||||||
{
|
|
||||||
tool_name = "assign_task_to_agent",
|
|
||||||
|
|
||||||
description = "Assign a task to an agent by role",
|
|
||||||
|
|
||||||
parameters = {
|
|
||||||
# Agent role - must be valid role
|
|
||||||
role
|
|
||||||
| String
|
|
||||||
| doc "Agent role"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.enum.TaggedUnion
|
|
||||||
| [| 'developer, 'reviewer, 'architect, 'tester, 'documenter, 'devops, 'monitor, 'security |],
|
|
||||||
|
|
||||||
# Task title
|
|
||||||
task_title
|
|
||||||
| String
|
|
||||||
| doc "Task title (3-500 chars)"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.length.min 3
|
|
||||||
| std.string.length.max 500,
|
|
||||||
|
|
||||||
# Task description
|
|
||||||
task_description
|
|
||||||
| String
|
|
||||||
| doc "Task description (detailed requirements)"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.length.min 10,
|
|
||||||
|
|
||||||
# Optional priority (0-100)
|
|
||||||
priority
|
|
||||||
| Number
|
|
||||||
| doc "Task priority (0-100, default: 50)"
|
|
||||||
| std.number.between 0 100
|
|
||||||
| default = 50,
|
|
||||||
|
|
||||||
# Optional context (JSON string)
|
|
||||||
context
|
|
||||||
| String
|
|
||||||
| doc "Additional context as JSON string (optional)"
|
|
||||||
| default = "{}",
|
|
||||||
},
|
|
||||||
|
|
||||||
# Validation contracts
|
|
||||||
contracts = {
|
|
||||||
# Context must be valid JSON if provided
|
|
||||||
context_is_json = fun params =>
|
|
||||||
if params.context == "{}" then
|
|
||||||
true
|
|
||||||
else
|
|
||||||
# Simple JSON validation - starts with { or [
|
|
||||||
std.string.is_match "^[\\{\\[]" params.context,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
@ -1,16 +0,0 @@
|
|||||||
# Schema for get_agent_capabilities MCP tool
|
|
||||||
# Validates agent capability requests
|
|
||||||
{
|
|
||||||
tool_name = "get_agent_capabilities",
|
|
||||||
|
|
||||||
description = "Get detailed agent capabilities and status",
|
|
||||||
|
|
||||||
parameters = {
|
|
||||||
# Agent UUID or ID
|
|
||||||
agent_id
|
|
||||||
| String
|
|
||||||
| doc "Agent UUID or unique identifier"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.length.min 3,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
@ -1,16 +0,0 @@
|
|||||||
# Schema for get_project_summary MCP tool
|
|
||||||
# Validates project summary requests
|
|
||||||
{
|
|
||||||
tool_name = "get_project_summary",
|
|
||||||
|
|
||||||
description = "Get project summary and statistics",
|
|
||||||
|
|
||||||
parameters = {
|
|
||||||
# Project UUID
|
|
||||||
project_id
|
|
||||||
| String
|
|
||||||
| doc "UUID of the project"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.match "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
@ -1,45 +0,0 @@
|
|||||||
# Schema for kanban_create_task MCP tool
|
|
||||||
# Validates task creation requests with contracts
|
|
||||||
{
|
|
||||||
tool_name = "kanban_create_task",
|
|
||||||
|
|
||||||
description = "Create task in Kanban board with validation",
|
|
||||||
|
|
||||||
parameters = {
|
|
||||||
# Project UUID - must be valid UUID format
|
|
||||||
project_id
|
|
||||||
| String
|
|
||||||
| doc "UUID of the target project"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.match "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$",
|
|
||||||
|
|
||||||
# Task title - 3 to 200 characters
|
|
||||||
title
|
|
||||||
| String
|
|
||||||
| doc "Task title (3-200 chars)"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.length.min 3
|
|
||||||
| std.string.length.max 200,
|
|
||||||
|
|
||||||
# Optional description
|
|
||||||
description
|
|
||||||
| String
|
|
||||||
| doc "Task description (optional)"
|
|
||||||
| default = "",
|
|
||||||
|
|
||||||
# Priority level - enum validation
|
|
||||||
priority
|
|
||||||
| String
|
|
||||||
| doc "Task priority level"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.enum.TaggedUnion
|
|
||||||
| [| 'low, 'medium, 'high, 'critical |],
|
|
||||||
},
|
|
||||||
|
|
||||||
# Validation contracts (custom predicates)
|
|
||||||
contracts = {
|
|
||||||
# Title must not be only whitespace
|
|
||||||
title_not_whitespace = fun params =>
|
|
||||||
std.string.trim params.title != "",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
@ -1,31 +0,0 @@
|
|||||||
# Schema for kanban_update_task MCP tool
|
|
||||||
# Validates task status updates and reordering
|
|
||||||
{
|
|
||||||
tool_name = "kanban_update_task",
|
|
||||||
|
|
||||||
description = "Update task status and order in Kanban board",
|
|
||||||
|
|
||||||
parameters = {
|
|
||||||
# Task UUID
|
|
||||||
task_id
|
|
||||||
| String
|
|
||||||
| doc "UUID of the task to update"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.string.match "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$",
|
|
||||||
|
|
||||||
# New status - enum validation
|
|
||||||
status
|
|
||||||
| String
|
|
||||||
| doc "New task status"
|
|
||||||
| std.string.NonEmpty
|
|
||||||
| std.enum.TaggedUnion
|
|
||||||
| [| 'todo, 'doing, 'review, 'done |],
|
|
||||||
|
|
||||||
# Optional order within column
|
|
||||||
order
|
|
||||||
| Number
|
|
||||||
| doc "Order within column (0-based index, optional)"
|
|
||||||
| std.number.greater_than (-1)
|
|
||||||
| default = 0,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
Loading…
x
Reference in New Issue
Block a user