jesus/prvng_core

Fork 0

Jesús Pérez 99f743f24d

chore fix links to main repo

2025-10-07 10:50:09 +01:00

12 KiB

Raw Blame History

Provisioning Logo

Core Engine

The Core Engine is the foundational component of the Provisioning project, providing the unified CLI interface, core Nushell libraries, and essential utility scripts. Built on Nushell and KCL, it serves as the primary entry point for all infrastructure operations.

Overview

The Core Engine provides:

Unified CLI Interface - Single command-line tool for all infrastructure operations
Core Libraries - Reusable Nushell modules for configuration, validation, and utilities
Provider Abstraction - Cloud-agnostic interface supporting UpCloud, AWS, and local providers
Workflow Integration - Commands for submitting and managing workflows (executed by the orchestrator)
Configuration System - Hierarchical, config-driven architecture with 476+ configuration accessors

Project Structure

provisioning/core/
├── cli/                    # Command-line interface
│   └── provisioning        # Main CLI entry point (211 lines, 84% reduction)
├── nulib/                  # Core Nushell libraries
│   ├── lib_provisioning/   # Core provisioning libraries
│   │   ├── config/         # Configuration loading and management
│   │   ├── utils/          # Utility functions (SSH, validation, logging)
│   │   ├── providers/      # Provider abstraction layer
│   │   ├── secrets/        # Secrets management (SOPS integration)
│   │   ├── workspace/      # Workspace management
│   │   └── infra_validator/ # Infrastructure validation engine
│   ├── main_provisioning/  # CLI command handlers
│   │   ├── flags.nu        # Centralized flag handling
│   │   ├── dispatcher.nu   # Command routing (80+ shortcuts)
│   │   ├── help_system.nu  # Categorized help system
│   │   └── commands/       # Domain-focused command modules
│   ├── servers/            # Server management modules
│   ├── taskservs/          # Task service modules
│   ├── clusters/           # Cluster management modules
│   └── workflows/          # Workflow orchestration modules
├── scripts/                # Utility scripts
│   └── test/               # Test automation
└── resources/              # Images and logos

Installation

Prerequisites

Nushell 0.107.1+ - Primary shell and scripting environment
KCL 0.11.2+ - Configuration language for infrastructure definitions
SOPS 3.10.2+ - Secrets management (optional but recommended)
Age 1.2.1+ - Encryption tool for secrets (optional)

Adding to PATH

To use the CLI globally, add it to your PATH:

# Create symbolic link
ln -sf "$(pwd)/provisioning/core/cli/provisioning" /usr/local/bin/provisioning

# Or add to PATH in your shell config (~/.bashrc, ~/.zshrc, etc.)
export PATH="$PATH:/path/to/project-provisioning/provisioning/core/cli"

Verify installation:

provisioning version
provisioning help

Quick Start

Basic Commands

# Show help and available commands
provisioning help

# Show environment and configuration
provisioning env
provisioning allenv

# Validate configuration
provisioning validate config

# List available providers
provisioning providers

# Show system information
provisioning nuinfo

Infrastructure Operations

# Create new infrastructure workspace
provisioning workspace init my-project

# Create servers
provisioning server create --check

# Install task services (infrastructure components)
provisioning taskserv create kubernetes

# Create complete cluster
provisioning cluster create my-cluster

# SSH into server
provisioning server ssh hostname-01

Quick Reference

For fastest command reference:

provisioning sc

For complete guides:

provisioning guide from-scratch    # Complete deployment guide
provisioning guide quickstart      # Command shortcuts reference
provisioning guide customize       # Customization patterns

Core Libraries

Configuration System (`lib_provisioning/config/`)

Hierarchical configuration loading with 476+ config accessors:

Defaults → User → Project → Infrastructure → Environment → Runtime
Replaced 200+ ENV variables with config-driven architecture
Variable interpolation: {{paths.base}}, {{env.HOME}}, {{now.date}}

use lib_provisioning/config

# Get configuration value
let value = config get "servers.default_plan"

# Load workspace config
let ws_config = config load-workspace "my-project"

Provider Abstraction (`lib_provisioning/providers/`)

Unified interface for cloud providers:

use lib_provisioning/providers

# Get provider interface
let provider = providers get "upcloud"

# Create server using provider
$provider | invoke "create_server" $server_config

Utilities (`lib_provisioning/utils/`)

Common utility functions:

SSH Operations - utils/ssh.nu
Validation - utils/validation.nu
Logging - utils/logging.nu
Error Handling - utils/error.nu
File Operations - utils/files.nu
Version Management - utils/version_manager.nu

Workflow Orchestration (`workflows/`)

Batch operations with dependency resolution:

# Submit batch workflow
provisioning batch submit workflows/example.k

# Monitor workflow progress
provisioning batch monitor <workflow-id>

# List all workflows
provisioning workflow list

# Get workflow status
provisioning workflow status <id>

CLI Architecture

Modular Design

The CLI uses a domain-driven architecture:

Infrastructure - Servers, task services, clusters
Orchestration - Workflows, batch operations, orchestrator management
Development - Modules, layers, versioning, packaging
Workspace - Workspace management, templates
Configuration - Settings, environment, validation
Utilities - SSH, SOPS, cache, plugins

Command Shortcuts

80+ shortcuts for improved productivity:

Full Command	Shortcuts	Description
`server`	`s`	Server operations
`taskserv`	`t`, `task`	Task service operations
`cluster`	`cl`	Cluster operations
`workspace`	`ws`	Workspace management
`workflow`	`wf`, `flow`	Workflow operations
`batch`	`bat`	Batch operations
`module`	`mod`	Module management
`generate`	`g`, `gen`	Code generation
`validate`	`val`	Validation operations

See complete reference: provisioning sc or provisioning guide quickstart

Bi-directional Help

Help works in both directions:

provisioning help workspace  # ✅
provisioning workspace help  # ✅ Same result
provisioning ws help         # ✅ Shortcut also works

Configuration

Configuration Files

With the new structure (2025-09-26):

System Defaults: provisioning/config/config.defaults.toml
User Config: workspace/config/local-overrides.toml
Environment Configs: workspace/config/{dev,test,prod}-defaults.toml
Infrastructure Configs: workspace/infra/{name}/config.toml

Environment Variables

Legacy ENV variables still supported during transition:

# Show current environment
provisioning env

# Show all configuration and environment
provisioning allenv

# Use specific environment
PROVISIONING_ENV=prod provisioning server list

Debug Flags

# Enable debug mode
provisioning --debug <command>

# Check mode (dry-run, no changes)
provisioning --check server create

# Auto-confirm operations
provisioning --yes cluster delete

# Specify infrastructure
provisioning --infra my-project server list

Design Principles

Modularity

Clean separation between libraries, CLI, and extensions:

Core Engine (core/) - CLI, libraries, scripts
Extensions (extensions/) - Providers, task services, clusters
Platform Services (platform/) - Orchestrator, control center, API gateway

Extensibility

Plugin architecture for extending functionality:

Provider plugins for cloud integrations
Task service definitions for infrastructure components
Cluster templates for complete deployments
Workflow templates for automation

Consistency

Unified API patterns across all components:

Standardized command structure
Common flag handling
Consistent error messages
Predictable output formats

Performance

Optimized for large-scale infrastructure operations:

Token-optimized agents (85-90% efficiency)
Batch operations with parallel execution
Checkpoint-based recovery
Incremental state management

Migration Strategy

The project follows a three-phase migration:

Phase 1 (Complete) - Created symbolic links to existing implementations
Phase 2 (In Progress) - Gradual migration of components to new structure
Phase 3 (Planned) - Consolidate and optimize unified core engine

Dependencies

Required

Nushell 0.107.1+ - Shell and scripting language
KCL 0.11.2+ - Configuration language

Optional

nu_plugin_tera - Template rendering
nu_plugin_kcl - KCL integration (CLI kcl is required, plugin optional)

Documentation

User Guides

Quick Start: provisioning guide from-scratch
Command Reference: provisioning sc
Update Guide: provisioning guide update
Customization: provisioning guide customize

Architecture Documentation

CLI Architecture: docs/architecture/ADR-006-provisioning-cli-refactoring.md
Configuration System: See .claude/features/configuration-system.md
Batch Workflows: See .claude/features/batch-workflow-system.md
Orchestrator: See .claude/features/orchestrator-architecture.md

API Documentation

REST API: See docs/api/ (when orchestrator is running)
Nushell Modules: See inline documentation in nulib/ modules

Testing

Run Tests

# Run all tests
nu provisioning/core/scripts/test/test_all.nu

# Run specific test group
nu provisioning/core/scripts/test/test_config.nu
nu provisioning/core/scripts/test/test_cli.nu

Test Coverage

The core engine includes comprehensive test coverage:

Configuration loading and validation
Provider interface contracts
CLI command routing
Workflow orchestration
Error handling

Contributing

When contributing to the Core Engine:

Follow Nushell Patterns - See .claude/best_nushell_code.md
Use Modular Design - Domain-focused modules in appropriate directories
Add Tests - Include tests for new functionality
Update Documentation - Keep README and inline docs current
Follow PAP - Project Architecture Principles (see CLAUDE.md)

Troubleshooting

Common Issues

Missing environment variables:

provisioning env          # Check current configuration
provisioning validate config  # Validate configuration files

KCL compilation errors:

kcl fmt <file>.k          # Format KCL file
kcl run <file>.k          # Test KCL file

Provider authentication:

provisioning providers    # List available providers
provisioning show settings # View provider configuration

Debug Mode

Enable verbose logging:

provisioning --debug <command>

Getting Help

provisioning help              # Show main help
provisioning help <category>   # Category-specific help
provisioning <command> help    # Command-specific help
provisioning guide list        # List all guides

Version Information

Check system versions:

provisioning version           # Show all versions
provisioning nuinfo            # Nushell information

License

See project root LICENSE file.

Maintained By: Architecture Team Last Updated: 2025-10-07

12 KiB Raw Blame History