diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index c15a01c..5a3cf15 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -14,7 +14,7 @@ repos: language: system types: [rust] pass_filenames: false - stages: [pre-commit] + stages: [pre-push] - id: rust-clippy name: Rust linting (cargo clippy) @@ -22,7 +22,7 @@ repos: language: system types: [rust] pass_filenames: false - stages: [pre-commit] + stages: [pre-push] - id: rust-test name: Rust tests @@ -87,16 +87,29 @@ repos: # stages: [commit] # ============================================================================ - # Markdown Hooks (optional - enable if using Markdown) + # Markdown Hooks # ============================================================================ - # - repo: local - # hooks: - # - id: markdownlint - # name: Markdown linting (markdownlint-cli2) - # entry: markdownlint-cli2 - # language: system - # types: [markdown] - # stages: [commit] + - repo: local + hooks: + - id: markdownlint + name: Markdown linting (markdownlint-cli2) + entry: markdownlint-cli2 --config .markdownlint-cli2.jsonc docs/ + language: system + types: [markdown] + pass_filenames: false + stages: [pre-commit] + + # NOTE: Malformed closing fences check disabled + # All 4,043 instances were fixed in 2025-01-09 + # Markdownlint doesn't catch this natively (see config comments) + # Requires Python for proper state tracking, which is not desired + # - id: check-markdown-fences + # name: Check malformed code fences + # entry: .githooks/check-markdown-fences.sh + # language: system + # types: [markdown] + # pass_filenames: true + # stages: [commit] # ============================================================================ # General Pre-commit Hooks diff --git a/README.md b/README.md index 161e7c8..108641e 100644 --- a/README.md +++ b/README.md @@ -25,9 +25,15 @@ ## What is Provisioning? -**Provisioning** is a comprehensive **Infrastructure as Code (IaC)** platform designed to manage complete infrastructure lifecycles: cloud providers, infrastructure services, clusters, and isolated workspaces across multiple cloud/local environments. +**Provisioning** is a comprehensive **Infrastructure as Code (IaC)** platform designed to manage +complete infrastructure lifecycles: cloud providers, infrastructure services, clusters, +and isolated workspaces across multiple cloud/local environments. -Extensible and customizable by design, it delivers type-safe, configuration-driven workflows with enterprise security (encrypted configuration, Cosmian KMS integration, Cedar policy engine, secrets management, authorization and permissions control, compliance checking, anomaly detection) and adaptable deployment modes (interactive UI, CLI automation, unattended CI/CD) suitable for any scale from development to production. +Extensible and customizable by design, it delivers type-safe, configuration-driven workflows +with enterprise security (encrypted configuration, Cosmian KMS integration, Cedar policy engine, +secrets management, authorization and permissions control, compliance checking, anomaly detection) +and adaptable deployment modes (interactive UI, CLI automation, unattended CI/CD) +suitable for any scale from development to production. ### Technical Definition @@ -37,26 +43,26 @@ Declarative Infrastructure as Code (IaC) platform providing: - **Modular, extensible architecture**: cloud providers, task services, clusters, workspaces - **Multi-cloud abstraction layer** with unified API (UpCloud, AWS, local infrastructure) - **High-performance state management**: - - Graph database backend for complex relationships - - Real-time state tracking and queries - - Multi-model data storage (document, graph, relational) + - Graph database backend for complex relationships + - Real-time state tracking and queries + - Multi-model data storage (document, graph, relational) - **Enterprise security stack**: - - Encrypted configuration and secrets management - - Cosmian KMS integration for confidential key management - - Cedar policy engine for fine-grained access control - - Authorization and permissions control via platform services - - Compliance checking and policy enforcement - - Anomaly detection for security monitoring - - Audit logging and compliance tracking + - Encrypted configuration and secrets management + - Cosmian KMS integration for confidential key management + - Cedar policy engine for fine-grained access control + - Authorization and permissions control via platform services + - Compliance checking and policy enforcement + - Anomaly detection for security monitoring + - Audit logging and compliance tracking - **Hybrid orchestration**: Rust-based performance layer + scripting flexibility - **Production-ready features**: - - Batch workflows with dependency resolution - - Checkpoint recovery and automatic rollback - - Parallel execution with state management + - Batch workflows with dependency resolution + - Checkpoint recovery and automatic rollback + - Parallel execution with state management - **Adaptable deployment modes**: - - Interactive TUI for guided setup - - Headless CLI for scripted automation - - Unattended mode for CI/CD pipelines + - Interactive TUI for guided setup + - Headless CLI for scripted automation + - Unattended mode for CI/CD pipelines - **Hierarchical configuration system** with inheritance and overrides ### What It Does @@ -116,7 +122,7 @@ Declarative Infrastructure as Code (IaC) platform providing: ```plaintext Defaults → User → Project → Infrastructure → Environment → Runtime -```plaintext +``` #### 4. **Imperative Scripts** @@ -209,13 +215,13 @@ workspace_librecloud/ # Production workspace workspace_dev/ # Development workspace ├── infra/ └── config/ -```plaintext +``` Switch between workspaces with single command: ```bash provisioning workspace switch librecloud -```plaintext +``` ### 5. **Workflows** @@ -280,7 +286,7 @@ Coordinated sequences of operations with dependency management. │ • Kubernetes Clusters │ │ • Running Services │ └─────────────────────────────────────────────────────────────────┘ -```plaintext +``` ### Directory Structure @@ -342,49 +348,49 @@ project-provisioning/ - **Language**: Rust + Nushell - **Purpose**: Workflow execution, task scheduling, state management - **Features**: - - File-based persistence - - Priority processing - - Retry logic with exponential backoff - - Checkpoint-based recovery - - REST API endpoints + - File-based persistence + - Priority processing + - Retry logic with exponential backoff + - Checkpoint-based recovery + - REST API endpoints #### 2. **Control Center** (`platform/control-center/`) - **Language**: Web UI + Backend API - **Purpose**: Web-based infrastructure management - **Features**: - - Dashboard views - - Real-time monitoring - - Interactive deployments - - Log viewing + - Dashboard views + - Real-time monitoring + - Interactive deployments + - Log viewing #### 3. **MCP Server** (`platform/mcp-server/`) - **Language**: Nushell - **Purpose**: Model Context Protocol integration for AI assistance - **Features**: - - 7 AI-powered settings tools - - Intelligent config completion - - Natural language infrastructure queries + - 7 AI-powered settings tools + - Intelligent config completion + - Natural language infrastructure queries #### 4. **OCI Registry** (`platform/oci-registry/`) - **Purpose**: Extension distribution and versioning - **Features**: - - Task service packages - - Provider packages - - Cluster templates - - Workflow definitions + - Task service packages + - Provider packages + - Cluster templates + - Workflow definitions #### 5. **Installer** (`platform/installer/`) - **Language**: Rust (Ratatui TUI) + Nushell - **Purpose**: Platform installation and setup - **Features**: - - Interactive TUI mode - - Headless CLI mode - - Unattended CI/CD mode - - Configuration generation + - Interactive TUI mode + - Headless CLI mode + - Unattended CI/CD mode + - Configuration generation --- @@ -492,9 +498,9 @@ Three native Rust plugins providing 10-50x performance improvements over HTTP AP - **Three Native Plugins**: auth, KMS, orchestrator - **Performance Gains**: - - KMS operations: ~5ms vs ~50ms (10x faster) - - Orchestrator queries: ~1ms vs ~30ms (30x faster) - - Auth verification: ~10ms vs ~50ms (5x faster) + - KMS operations: ~5ms vs ~50ms (10x faster) + - Orchestrator queries: ~1ms vs ~30ms (30x faster) + - Auth verification: ~10ms vs ~50ms (5x faster) - **OS-Native Keyring**: macOS Keychain, Linux Secret Service, Windows Credential Manager - **KMS Backends**: RustyVault, Age, AWS KMS, Vault, Cosmian - **Graceful Fallback**: Automatic fallback to HTTP if plugins not installed @@ -509,13 +515,13 @@ Enterprise-grade security with 39,699 lines across 12 components. - **API**: 83+ REST endpoints, 111+ CLI commands - **Standards**: GDPR, SOC2, ISO 27001 compliance - **Key Features**: - - RS256 authentication with Argon2id hashing - - Policy-as-code with hot reload - - Multi-factor authentication (TOTP + WebAuthn/FIDO2) - - Dynamic secrets (AWS STS, SSH keys) with TTL - - 5 KMS backends with envelope encryption - - 7-year audit retention with 5 export formats - - Multi-party break-glass approval + - RS256 authentication with Argon2id hashing + - Policy-as-code with hot reload + - Multi-factor authentication (TOTP + WebAuthn/FIDO2) + - Dynamic secrets (AWS STS, SSH keys) with TTL + - 5 KMS backends with envelope encryption + - 7-year audit retention with 5 export formats + - Multi-party break-glass approval --- @@ -524,7 +530,7 @@ Enterprise-grade security with 39,699 lines across 12 components. ### Core Technologies | Technology | Version | Purpose | Why | -|------------|---------|---------|-----| +| ------------ | --------- | --------- | ----- | | **Nickel** | Latest | PRIMARY - Infrastructure-as-code language | Type-safe schemas, lazy evaluation, LSP support, composable records, gradual validation | | **Nushell** | 0.109.0+ | Scripting and task automation | Structured data pipelines, cross-platform, modern built-in parsers (JSON/YAML/TOML) | | **Rust** | Latest | Platform services (orchestrator, control-center, installer) | Performance, memory safety, concurrency, reliability | @@ -533,13 +539,13 @@ Enterprise-grade security with 39,699 lines across 12 components. ### Data & State Management | Technology | Version | Purpose | Features | -|------------|---------|---------|----------| +| ------------ | --------- | --------- | ---------- | | **SurrealDB** | Latest | High-performance graph database backend | Multi-model (document, graph, relational), real-time queries, distributed architecture, complex relationship tracking | ### Platform Services (Rust-based) | Service | Purpose | Security Features | -|---------|---------|-------------------| +| --------- | --------- | ------------------- | | **Orchestrator** | Workflow execution, task scheduling, state management | File-based persistence, retry logic, checkpoint recovery | | **Control Center** | Web-based infrastructure management | **Authorization and permissions control**, RBAC, audit logging | | **Installer** | Platform installation (TUI + CLI modes) | Secure configuration generation, validation | @@ -550,7 +556,7 @@ Enterprise-grade security with 39,699 lines across 12 components. ### Security & Secrets | Technology | Version | Purpose | Enterprise Features | -|------------|---------|---------|---------------------| +| ------------ | --------- | --------- | --------------------- | | **SOPS** | 3.10.2+ | Secrets management | Encrypted configuration files | | **Age** | 1.2.1+ | Encryption | Secure key-based encryption | | **Cosmian KMS** | Latest | Key Management System | Confidential computing, secure key storage, cloud-native KMS | @@ -562,7 +568,7 @@ Enterprise-grade security with 39,699 lines across 12 components. ### Version Management | Component | Purpose | Format | -|-----------|---------|--------| +| ----------- | --------- | -------- | | **versions.ncl** | Core tool versions (Nickel primary) | Nickel schema | | **provider version.ncl** | Provider-specific versions | Nickel schema | | **provisioning setup versions** | Version file generator | Nushell command | @@ -581,7 +587,7 @@ echo $NU_VERSION $PROVIDER_AWS_VERSION ### Optional Tools | Tool | Purpose | -|------|---------| +| ------ | --------- | | **K9s** | Kubernetes management interface | | **nu_plugin_tera** | Nushell plugin for Tera template rendering | | **nu_plugin_kcl** | Nushell plugin for KCL integration (CLI required, plugin optional) | @@ -652,7 +658,7 @@ echo $NU_VERSION $PROVIDER_AWS_VERSION ```bash provisioning server create --infra my-cluster -```plaintext +``` **Step 3**: Provisioning executes workflow @@ -677,13 +683,13 @@ provisioning server create --infra my-cluster 4. Checkpoint after each step 5. Monitor health checks 6. Report completion -```plaintext +``` **Step 4**: Verify deployment ```bash provisioning cluster status my-cluster -```plaintext +``` ### Configuration Hierarchy @@ -701,7 +707,7 @@ Configuration values are resolved through a hierarchy: 5. Environment Config (workspace/config/prod-defaults.toml) ↓ (overridden by) 6. Runtime Flags (--flag value) -```plaintext +``` **Example**: @@ -720,7 +726,7 @@ default_plan = "large" # Overrides user preference # Runtime provisioning server create --plan xlarge # Overrides everything -```plaintext +``` --- @@ -736,7 +742,7 @@ provisioning cluster create k8s-prod --provider upcloud # AWS cluster (same config) provisioning cluster create k8s-prod --provider aws -```plaintext +``` ### 2. **Development → Staging → Production Pipeline** @@ -754,7 +760,7 @@ provisioning cluster create app-stack # Production (HA, larger resources) provisioning workspace switch prod provisioning cluster create app-stack -```plaintext +``` ### 3. **Infrastructure as Code Testing** @@ -770,7 +776,7 @@ provisioning test env run # Cleanup provisioning test env cleanup -```plaintext +``` ### 4. **Batch Multi-Region Deployment** @@ -822,7 +828,7 @@ provisioning workspace switch prod provisioning cluster create --infra backup-restore --wait # All services restored with same configuration -```plaintext +``` ### 6. **CI/CD Integration** @@ -832,21 +838,21 @@ Automated testing and deployment pipelines. # .gitlab-ci.yml test-infrastructure: script: - - provisioning test quick kubernetes - - provisioning test quick postgres + - provisioning test quick kubernetes + - provisioning test quick postgres deploy-staging: script: - - provisioning workspace switch staging - - provisioning cluster create app-stack --check - - provisioning cluster create app-stack --yes + - provisioning workspace switch staging + - provisioning cluster create app-stack --check + - provisioning cluster create app-stack --yes deploy-production: when: manual script: - - provisioning workspace switch prod - - provisioning cluster create app-stack --yes -```plaintext + - provisioning workspace switch prod + - provisioning cluster create app-stack --yes +``` ---