# provctl - Machine Orchestration & Service Control Platform

<div align="center">
  <img src="imgs/provctl_logo_h.svg" alt="provctl Logo" width="600" />
</div>

[![Rust](https://img.shields.io/badge/rust-1.91.0%2B-orange.svg)](https://www.rust-lang.org)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

**provctl** is a production-grade Machine Orchestration & Service Control Platform for managing SSH-based deployments across multiple machines with sophisticated deployment strategies, resilience features, and comprehensive observability.

## Features

✅ **Local Service Management**
- 🔧 **Multi-Backend Support** - systemd (Linux), launchd (macOS), PID files (Unix)
- 🎯 **Unified Interface** - Single CLI for all platforms
- 📊 **Status & Logging** - Comprehensive service monitoring
- ⚙️ **Configuration-Driven** - TOML-based zero-hardcoded-strings approach

✅ **Machine Orchestration**
- 🚀 **SSH Command Execution** - Real async SSH with ssh2-rs
- 📦 **Multiple Deployment Strategies** - Rolling, Blue-Green, Canary
- 🔄 **Connection Pooling** - Per-host reuse with configurable limits
- 🎯 **Retry & Timeout Logic** - Exponential/linear/fibonacci backoff strategies
- 🔒 **Host Key Verification** - SSH known_hosts integration
- 💪 **Circuit Breaker** - Fault tolerance for failing hosts
- 🏥 **Health Checks** - Command/HTTP/TCP monitoring
- 📊 **Metrics & Audit Logging** - Comprehensive observability

✅ **REST API & Web Dashboard**
- 🌐 **Axum REST API** - Production-grade HTTP server
- 📱 **Leptos Dashboard** - Modern Rust web UI
- 📡 **Programmatic Access** - Full API for automation

✅ **Production-Ready**
- Zero unsafe code (`#![forbid(unsafe_code)]`)
- Comprehensive error handling with Result<T> types
- 100% documentation coverage (rustdoc)
- No unwrap() calls in production code

✅ **Developer Friendly**
- Async/await throughout (tokio-based)
- Thread-safe metrics collection (Arc<Mutex<>>)
- Trait-based design for extensibility
- Full rustdoc for all public APIs

## Quick Start

### Installation

```bash
# Option 1: Use the installer
nu scripts/install-provctl.nu

# Option 2: Build manually
cargo build --release
./target/release/provctl --help
```

### Basic Usage

```bash
# Start a service
provctl start my-service

# Stop a service
provctl stop my-service

# Restart a service
provctl restart my-service

# Check status
provctl status my-service

# View logs (last 50 lines)
provctl logs my-service --lines 50
```

## Commands

| Command | Description | Example |
|---------|-------------|---------|
| `start` | Start a service | `provctl start api` |
| `stop` | Stop a service | `provctl stop api` |
| `restart` | Restart a service | `provctl restart api` |
| `status` | Check service status | `provctl status api` |
| `logs` | View service logs | `provctl logs api --lines 100` |

## Configuration

provctl is **100% configuration-driven**. All user-facing strings and defaults come from TOML files:

- **`configs/messages.toml`** - All user-facing messages
- **`configs/defaults.toml`** - Timeouts, paths, and operational parameters

Zero hardcoded strings in code!

## Architecture

provctl uses a **trait-based backend system** for easy extensibility:

```
┌─────────────────────────────────────┐
│        provctl-cli (clap)           │
├─────────────────────────────────────┤
│         Backend Trait               │
├─────────────┬───────────┬───────────┤
│ systemd     │ launchd   │ Pidfile   │
│ (Linux)     │ (macOS)   │ (Unix)    │
└─────────────┴───────────┴───────────┘
```

### Backends

**systemd** (Linux)
- Uses `systemctl` for service management
- Queries `journalctl` for logs
- Best for production Linux systems

**launchd** (macOS)
- Generates plist files automatically
- Uses `launchctl` for management
- Best for macOS development and production

**PID files** (Universal)
- Works on any Unix system
- File-based process tracking
- Fallback when systemd/launchd unavailable

## Project Structure

```
provctl/
├── crates/
│   ├── provctl-core/              # Service management fundamentals
│   ├── provctl-config/            # Configuration loading
│   ├── provctl-backend/           # Backend trait + implementations
│   ├── provctl-cli/               # CLI interface
│   ├── provctl-machines/          # Machine orchestration core
│   │   ├── ssh_async.rs           # Real async SSH
│   │   ├── ssh_pool.rs            # Connection pooling
│   │   ├── ssh_retry.rs           # Retry & timeout logic
│   │   ├── ssh_host_key.rs        # Host key verification
│   │   ├── health_check.rs        # Health monitoring
│   │   ├── metrics.rs             # Metrics & audit logging
│   │   └── ... (other modules)
│   ├── provctl-api/               # REST API server
│   └── provctl-dashboard/         # Web UI (Leptos)
├── configs/
│   ├── messages.toml              # User messages
│   └── defaults.toml              # Configuration defaults
├── .coder/info/                   # Project tracking & docs
├── docs/                          # Documentation
└── scripts/                       # Development & infrastructure scripts
```

## Development

### Build

```bash
cargo build --release
```

### Test

```bash
cargo test --all
```

### Lint & Format

```bash
cargo fmt --all
cargo clippy --all --all-targets
```

### Documentation

```bash
cargo doc --open --no-deps
```

## Integration with syntaxis-api

provctl provides service management for syntaxis-api:

```bash
# In syntaxis-api.nu wrapper:
provctl start syntaxis-api

# Or from provisioning schema:
provctl deploy --config provisioning/syntaxis-api.toml
```

See [provisioning/](provisioning/) for integration examples.

## Code Quality Standards

✅ **No unsafe code** - Enforced via `#![forbid(unsafe_code)]`
✅ **100% documented** - All public APIs have rustdoc comments
✅ **Fully tested** - 332+ unit tests, 100% passing
✅ **Formatted & linted** - `cargo fmt` + `cargo clippy` clean
✅ **Zero hardcoded strings** - All in TOML configuration
✅ **Proper error handling** - Comprehensive Result<T> types


## License

MIT License - See [LICENSE](LICENSE) for details.

## Contributing

Contributions welcome! Please ensure:
- `cargo fmt --all` passes
- `cargo clippy --all --all-targets` passes
- All tests pass: `cargo test --all`
- No unsafe code

## Documentation Overview

| Document | Purpose |
|----------|---------|
| **QUICKSTART.md** | Get started with machine orchestration in 5 minutes |
| **docs/CLI_GUIDE.md** | Complete CLI reference for all commands |
| **docs/ARCHITECTURE.md** | System design and architecture |
| **docs/INSTALLATION_GUIDE.md** | Installation and setup instructions |

## Support

For issues and questions:
- **Quick Start**: See [QUICKSTART.md](QUICKSTART.md)
- **CLI Reference**: See [docs/CLI_GUIDE.md](docs/CLI_GUIDE.md)
- **Documentation**: See [docs/](docs/) directory
- **Architecture**: See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)

---

**Made with ❤️ for the Rust community**