prvng_core/README.md

<p align="center">
  <img src="resources/provisioning_logo.svg" alt="Provisioning Logo" width="300"/>
</p>
<p align="center">
  <img src="resources/logo-text.svg" alt="Provisioning" width="500"/>
</p>

---

# Core Engine

The **Core Engine** is the foundation of the Provisioning platform—a modular, high-performance infrastructure automation system built on **Nushell** and **KCL**. It provides unified CLI tools, core libraries, and extensible architecture for managing cloud infrastructure, Kubernetes clusters, and infrastructure-as-code workflows.

## Overview

The Core Engine serves as the central orchestration layer, providing:

- **Unified CLI Interface** - Single command-line interface for all infrastructure operations
- **Core Libraries** - Reusable Nushell modules for configuration, validation, deployment, and workflow management
- **Provider Abstraction** - Cloud-agnostic interface supporting UpCloud, AWS, and local providers
- **Workflow Orchestration** - Batch operations, dependency resolution, and state management
- **Configuration System** - Hierarchical, config-driven architecture with 476+ configuration accessors

## Project Structure

```
provisioning/core/
├── cli/                    # Command-line interface
│   └── provisioning        # Main CLI entry point (211 lines, 84% reduction)
├── nulib/                  # Core Nushell libraries
│   ├── lib_provisioning/   # Core provisioning libraries
│   │   ├── config/         # Configuration loading and management
│   │   ├── utils/          # Utility functions (SSH, validation, logging)
│   │   ├── providers/      # Provider abstraction layer
│   │   ├── secrets/        # Secrets management (SOPS integration)
│   │   ├── workspace/      # Workspace management
│   │   └── infra_validator/ # Infrastructure validation engine
│   ├── main_provisioning/  # CLI command handlers
│   │   ├── flags.nu        # Centralized flag handling
│   │   ├── dispatcher.nu   # Command routing (80+ shortcuts)
│   │   ├── help_system.nu  # Categorized help system
│   │   └── commands/       # Domain-focused command modules
│   ├── servers/            # Server management modules
│   ├── taskservs/          # Task service modules
│   ├── clusters/           # Cluster management modules
│   └── workflows/          # Workflow orchestration modules
├── scripts/                # Utility scripts
│   └── test/               # Test automation
└── resources/              # Images and logos
```

## Installation

### Prerequisites

- **Nushell 0.107.1+** - Primary shell and scripting environment
- **KCL 0.11.2+** - Configuration language for infrastructure definitions
- **SOPS 3.10.2+** - Secrets management (optional but recommended)
- **Age 1.2.1+** - Encryption tool for secrets (optional)

### Adding to PATH

To use the CLI globally, add it to your PATH:

```bash
# Create symbolic link
ln -sf "$(pwd)/provisioning/core/cli/provisioning" /usr/local/bin/provisioning

# Or add to PATH in your shell config (~/.bashrc, ~/.zshrc, etc.)
export PATH="$PATH:/path/to/project-provisioning/provisioning/core/cli"
```

Verify installation:

```bash
provisioning version
provisioning help
```

## Quick Start

### Basic Commands

```bash
# Show help and available commands
provisioning help

# Show environment and configuration
provisioning env
provisioning allenv

# Validate configuration
provisioning validate config

# List available providers
provisioning providers

# Show system information
provisioning nuinfo
```

### Infrastructure Operations

```bash
# Create new infrastructure workspace
provisioning workspace init my-project

# Create servers
provisioning server create --check

# Install task services (infrastructure components)
provisioning taskserv create kubernetes

# Create complete cluster
provisioning cluster create my-cluster

# SSH into server
provisioning server ssh hostname-01
```

### Quick Reference

For fastest command reference:

```bash
provisioning sc
```

For complete guides:

```bash
provisioning guide from-scratch    # Complete deployment guide
provisioning guide quickstart      # Command shortcuts reference
provisioning guide customize       # Customization patterns
```

## Core Libraries

### Configuration System (`lib_provisioning/config/`)

Hierarchical configuration loading with 476+ config accessors:

- **Defaults** → **User** → **Project** → **Infrastructure** → **Environment** → **Runtime**
- Replaced 200+ ENV variables with config-driven architecture
- Variable interpolation: `{{paths.base}}`, `{{env.HOME}}`, `{{now.date}}`

```nushell
use lib_provisioning/config

# Get configuration value
let value = config get "servers.default_plan"

# Load workspace config
let ws_config = config load-workspace "my-project"
```

### Provider Abstraction (`lib_provisioning/providers/`)

Unified interface for cloud providers:

```nushell
use lib_provisioning/providers

# Get provider interface
let provider = providers get "upcloud"

# Create server using provider
$provider | invoke "create_server" $server_config
```

### Utilities (`lib_provisioning/utils/`)

Common utility functions:

- **SSH Operations** - `utils/ssh.nu`
- **Validation** - `utils/validation.nu`
- **Logging** - `utils/logging.nu`
- **Error Handling** - `utils/error.nu`
- **File Operations** - `utils/files.nu`
- **Version Management** - `utils/version_manager.nu`

### Workflow Orchestration (`workflows/`)

Batch operations with dependency resolution:

```bash
# Submit batch workflow
provisioning batch submit workflows/example.k

# Monitor workflow progress
provisioning batch monitor <workflow-id>

# List all workflows
provisioning workflow list

# Get workflow status
provisioning workflow status <id>
```

## CLI Architecture

### Modular Design

The CLI uses a domain-driven architecture:

- **Infrastructure** - Servers, task services, clusters
- **Orchestration** - Workflows, batch operations, orchestrator management
- **Development** - Modules, layers, versioning, packaging
- **Workspace** - Workspace management, templates
- **Configuration** - Settings, environment, validation
- **Utilities** - SSH, SOPS, cache, plugins

### Command Shortcuts

80+ shortcuts for improved productivity:

| Full Command | Shortcuts | Description |
|--------------|-----------|-------------|
| `server` | `s` | Server operations |
| `taskserv` | `t`, `task` | Task service operations |
| `cluster` | `cl` | Cluster operations |
| `workspace` | `ws` | Workspace management |
| `workflow` | `wf`, `flow` | Workflow operations |
| `batch` | `bat` | Batch operations |
| `module` | `mod` | Module management |
| `generate` | `g`, `gen` | Code generation |
| `validate` | `val` | Validation operations |

See complete reference: `provisioning sc` or `provisioning guide quickstart`

### Bi-directional Help

Help works in both directions:

```bash
provisioning help workspace  # ✅
provisioning workspace help  # ✅ Same result
provisioning ws help         # ✅ Shortcut also works
```

## Configuration

### Configuration Files

With the new structure (2025-09-26):

- **System Defaults**: `provisioning/config/config.defaults.toml`
- **User Config**: `workspace/config/local-overrides.toml`
- **Environment Configs**: `workspace/config/{dev,test,prod}-defaults.toml`
- **Infrastructure Configs**: `workspace/infra/{name}/config.toml`

### Environment Variables

Legacy ENV variables still supported during transition:

```bash
# Show current environment
provisioning env

# Show all configuration and environment
provisioning allenv

# Use specific environment
PROVISIONING_ENV=prod provisioning server list
```

### Debug Flags

```bash
# Enable debug mode
provisioning --debug <command>

# Check mode (dry-run, no changes)
provisioning --check server create

# Auto-confirm operations
provisioning --yes cluster delete

# Specify infrastructure
provisioning --infra my-project server list
```

## Design Principles

### Modularity

Clean separation between libraries, CLI, and extensions:

- **Core Engine** (`core/`) - CLI, libraries, scripts
- **Extensions** (`extensions/`) - Providers, task services, clusters
- **Platform Services** (`platform/`) - Orchestrator, control center, API gateway

### Extensibility

Plugin architecture for extending functionality:

- Provider plugins for cloud integrations
- Task service definitions for infrastructure components
- Cluster templates for complete deployments
- Workflow templates for automation

### Consistency

Unified API patterns across all components:

- Standardized command structure
- Common flag handling
- Consistent error messages
- Predictable output formats

### Performance

Optimized for large-scale infrastructure operations:

- Token-optimized agents (85-90% efficiency)
- Batch operations with parallel execution
- Checkpoint-based recovery
- Incremental state management

## Migration Strategy

The project follows a three-phase migration:

1. **Phase 1** (Complete) - Created symbolic links to existing implementations
2. **Phase 2** (In Progress) - Gradual migration of components to new structure
3. **Phase 3** (Planned) - Consolidate and optimize unified core engine

## Dependencies

### Required

- **Nushell 0.107.1+** - Shell and scripting language
- **KCL 0.11.2+** - Configuration language

### Recommended

- **SOPS 3.10.2+** - Secrets management
- **Age 1.2.1+** - Encryption for secrets
- **K9s 0.50.6+** - Kubernetes management interface

### Optional

- **nu_plugin_tera** - Template rendering
- **nu_plugin_kcl** - KCL integration (CLI `kcl` is required, plugin optional)

## Documentation

### User Guides

- **Quick Start**: `provisioning guide from-scratch`
- **Command Reference**: `provisioning sc`
- **Update Guide**: `provisioning guide update`
- **Customization**: `provisioning guide customize`

### Architecture Documentation

- **CLI Architecture**: `docs/architecture/ADR-006-provisioning-cli-refactoring.md`
- **Configuration System**: See `.claude/features/configuration-system.md`
- **Batch Workflows**: See `.claude/features/batch-workflow-system.md`
- **Orchestrator**: See `.claude/features/orchestrator-architecture.md`

### API Documentation

- **REST API**: See `docs/api/` (when orchestrator is running)
- **Nushell Modules**: See inline documentation in `nulib/` modules

## Testing

### Run Tests

```bash
# Run all tests
nu provisioning/core/scripts/test/test_all.nu

# Run specific test group
nu provisioning/core/scripts/test/test_config.nu
nu provisioning/core/scripts/test/test_cli.nu
```

### Test Coverage

The core engine includes comprehensive test coverage:

- Configuration loading and validation
- Provider interface contracts
- CLI command routing
- Workflow orchestration
- Error handling

## Contributing

When contributing to the Core Engine:

1. **Follow Nushell Patterns** - See `.claude/best_nushell_code.md`
2. **Use Modular Design** - Domain-focused modules in appropriate directories
3. **Add Tests** - Include tests for new functionality
4. **Update Documentation** - Keep README and inline docs current
5. **Follow PAP** - Project Architecture Principles (see `CLAUDE.md`)

## Troubleshooting

### Common Issues

**Missing environment variables:**
```bash
provisioning env          # Check current configuration
provisioning validate config  # Validate configuration files
```

**KCL compilation errors:**
```bash
kcl fmt <file>.k          # Format KCL file
kcl run <file>.k          # Test KCL file
```

**Provider authentication:**
```bash
provisioning providers    # List available providers
provisioning show settings # View provider configuration
```

### Debug Mode

Enable verbose logging:

```bash
provisioning --debug <command>
```

### Getting Help

```bash
provisioning help              # Show main help
provisioning help <category>   # Category-specific help
provisioning <command> help    # Command-specific help
provisioning guide list        # List all guides
```

## Version Information

Check system versions:

```bash
provisioning version           # Show all versions
provisioning nuinfo            # Nushell information
```

## License

See project root LICENSE file.

---

**Maintained By**: Architecture Team
**Last Updated**: 2025-10-07