jesus/prvng_core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
prvng_core/CHANGELOG.md
Jesús Pérez 894046ef5a
feat(core): three-layer DAG, unified component arch, commands-registry cache, Nushell 0.112.2 migration 
- DAG architecture: `dag show/validate/export` (nulib/main_provisioning/dag.nu),
    config loader (lib_provisioning/config/loader/dag.nu), taskserv dag-executor.
    Backed by schemas/lib/dag/*.ncl; orchestrator emits NATS events via
    WorkspaceComposition::into_workflow. See ADR-020, ADR-021.
  - Unified Component Architecture: components/mod.nu, main_provisioning/
    {components,workflow,extensions,ontoref-queries}.nu. Full workflow engine with
    topological sort and NATS subject emission. Blocks A-H complete (libre-daoshi).
  - Commands-registry: nulib/commands-registry.ncl (Nickel source, 314 lines) +
    JSON cache at ~/.cache/provisioning/commands-registry.json rebuilt on source
    change. cli/provisioning fast-path alias expansion avoids cold Nu startup.
    ADDING_COMMANDS.md documents new-command workflow.
  - Platform service manager: service-manager.nu (+573), startup.nu (+611),
    service-check.nu (+255); autostart/bootstrap/health/target refactored.
  - Nushell 0.112.2 migration: removed all try/catch and bash redirections;
    external commands prefixed with ^; type signatures enforced. Driven by
    scripts/refactor-try-catch{,-simplified}.nu.
  - TTY stack: removed shlib/*-tty.sh; replaced by cli/tty-dispatch.sh,
    tty-filter.sh, tty-commands.conf.
  - New domain modules: images/ (golden image lifecycle), workspace/{state,sync}.nu,
    main_provisioning/{bootstrap,cluster-deploy,fip,state}.nu, commands/{state,
    build,integrations/auth,utilities/alias}.nu, platform.nu expanded (+874).
  - Config loader overhaul: loader/core.nu slimmed (-759), cache/core.nu
    refactored (-454), removed legacy loaders/file_loader.nu (-330).
  - Thirteen new provisioning-<domain>.nu top-level modules for bash dispatcher.
  - Tests: test_workspace_state.nu (+351); updates to test_oci_registry,
    test_services.
  - README + CHANGELOG updated.
2026-04-17 04:27:33 +01:00

13 KiB
Raw Permalink Blame History

Provisioning Core - Changelog

Date: 2026-04-17 Repository: provisioning/core Status: Nickel IaC (PRIMARY)

📋 Summary

Major refactor: three-layer DAG architecture with workspace composition, Unified Component Architecture (components + workflows + capabilities), Nickel-backed commands-registry with JSON cache for fast CLI startup, consolidated platform service manager, and completed Nushell 0.110/0.112 migration (no try/catch, no bash redirections). TTY stack moved from shlib/ into cli/tty-*. Numerous new domain modules: dag, components, workflow engine, images lifecycle, workspace state/sync, ontoref queries, FIP handler.

🔄 Latest Release (2026-04-17)

Three-Layer DAG Architecture

Scope: Workspace composition as a DAG with formula_id::task_name namespacing, health gates, conditions, and NATS subject emission.

New files:

  • nulib/main_provisioning/dag.nu — dag show/validate/export (DOT/JSON/Mermaid)
  • nulib/lib_provisioning/config/loader/dag.nu — DAG config loader
  • nulib/taskservs/dag-executor.nu — taskserv-level DAG execution helper

Related: ADR-020 (extension capability declarations), ADR-021 (workspace composition DAG). Orchestrator consumes composition via WorkspaceComposition::into_workflow and emits NATS events.

Unified Component Architecture

Scope: Components + workflows + capabilities as first-class citizens (libre-daoshi plan, blocks A-H complete).

New files:

  • nulib/components/mod.nu — component dispatch module
  • nulib/main_provisioning/components.nu — validate capabilities/components, component list/info
  • nulib/main_provisioning/workflow.nu — full workflow engine: run/list/status/ validate, topological sort, NATS event emission (+605 lines)
  • nulib/main_provisioning/extensions.nu — extensions capabilities/graph
  • nulib/main_provisioning/ontoref-queries.nu — on+re-aware CLI queries (describe component/databases/namespace/storage/workflow)

Commands-Registry & Fast-Path Dispatch

Scope: Eliminate Nu startup cost on every prvng invocation.

New files:

  • nulib/commands-registry.ncl — Nickel command catalog (314 lines)
  • nulib/lib_provisioning/utils/command-registry.nu — registry accessor
  • nulib/scripts/validate-command.nu — cache-aware command validator

Behavior: cli/provisioning reads the JSON cache at ~/.cache/provisioning/commands-registry.json, rebuilt automatically via nickel export when the .ncl source is newer. Single-char aliases (s, t, c, e, w, j, b, o, a) are expanded in bash before dispatch. nulib/main_provisioning/ADDING_COMMANDS.md documents the four-step procedure for new commands.

Platform Service Manager

New files:

  • nulib/lib_provisioning/platform/service-manager.nu (+573 lines)
  • nulib/lib_provisioning/platform/startup.nu (+611 lines)
  • nulib/lib_provisioning/utils/service-check.nu (+255 lines)

Refactored: platform/autostart.nu, platform/bootstrap.nu, platform/health.nu, platform/target.nu — unified lifecycle, health probes, and autostart logic.

Nushell 0.112.2 Migration

Scope: Project-wide refactor driven by scripts/refactor-try-catch.nu and scripts/refactor-try-catch-simplified.nu to reach Nushell 0.112.2 compliance.

Enforced:

  • No try/catch — all use do { } | complete
  • No bash redirections (2>&1, 2>/dev/null)
  • External commands prefixed with ^
  • Parenthesized pipelines in if
  • Type signatures: def f [x: string]: nothing -> record { }

TTY Stack Replacement

Removed: shlib/README.md, shlib/auth-login-tty.sh, shlib/mfa-enroll-tty.sh, shlib/setup-wizard-tty.sh.

Replaced by:

  • cli/tty-dispatch.sh (+86 lines) — TTY-safe command dispatcher
  • cli/tty-filter.sh (+137 lines) — command filter
  • cli/tty-commands.conf — TTY command manifest

New Domain Modules

  • nulib/images/ — golden image lifecycle (create, delete, list, state, update, watch)
  • nulib/workspace/state.nu (+641 lines) — workspace state model
  • nulib/workspace/sync.nu (+148 lines) — workspace synchronization
  • nulib/main_provisioning/bootstrap.nu — platform bootstrap
  • nulib/main_provisioning/cluster-deploy.nu — component/taskserv dispatch
  • nulib/main_provisioning/fip.nu (+421 lines) — floating IP handler
  • nulib/main_provisioning/state.nu — state command
  • nulib/main_provisioning/commands/state.nu, commands/build.nu, commands/integrations/auth.nu, commands/utilities/alias.nu
  • nulib/main_provisioning/commands/platform.nu — major expansion (+874 lines)

Config Loader Overhaul

  • lib_provisioning/config/loader/core.nu — slimmed (−759 lines of legacy paths)
  • lib_provisioning/config/cache/core.nu — refactored (−454 lines of dead paths)
  • lib_provisioning/config/cache/nickel.nu — simplified
  • Removed: lib_provisioning/config/loaders/file_loader.nu (−330 lines)
  • Added: config/accessor-minimal.nu, config/accessor/functions.nu helpers

Scripts & Tooling

  • nulib/scripts/ — query-* family (clusters/infra/providers/servers/taskservs/ workspace-info), validate-command, validate-config
  • scripts/auto-refactor-priority.nu, scripts/batch-refactor.sh
  • scripts/build-nixos-image-remote.sh, scripts/deploy-cp-server.sh

CLI Modular Subcommands

New top-level Nu modules referenced by the bash dispatcher: provisioning-batch.nu, provisioning-bootstrap.nu, provisioning-cluster.nu, provisioning-component.nu, provisioning-extension.nu, provisioning-job.nu, provisioning-platform.nu, provisioning-server.nu, provisioning-state.nu, provisioning-status.nu, provisioning-taskserv.nu, provisioning-volume.nu, provisioning-workflow.nu.

Tests

  • nulib/tests/test_workspace_state.nu (+351 lines)
  • Updates to test_oci_registry.nu, test_services.nu

Statistics

Area Files Lines +/−
DAG + Components + Workflows 8 +1800 / −50
Commands-registry + dispatch 6 +900 / −200
Platform service manager 5 +1700 / −300
Config loader/cache refactor 10 +400 / −1500
TTY replacement 4 +250 / −515
New subcommand modules 13 +1700 / −0
Total staged 242 files +21949 / −6012

🔄 Previous Release (2026-01-14)

Terminology Migration: Cluster → Taskserv

Scope: Complete refactoring across nulib/ modules to standardize on taskserv nomenclature

Files Updated:

  • nulib/clusters/handlers.nu - Handler signature updates, ANSI formatting improvements
  • nulib/clusters/run.nu - Function parameter and path updates (+326 lines modified)
  • nulib/clusters/utils.nu - Utility function updates (+144 lines modified)
  • nulib/clusters/discover.nu - Discovery module refactoring
  • nulib/clusters/load.nu - Configuration loader updates
  • nulib/ai/query_processor.nu - AI integration updates
  • nulib/api/routes.nu - API routing adjustments
  • nulib/api/server.nu - Server module updates
  • .pre-commit-config.yaml - Pre-commit hook updates

Changes:

  • Updated function parameters: server_cluster_path → server_taskserv_path
  • Updated record fields: defs.cluster.name → defs.taskserv.name
  • Enhanced output formatting with consistent ANSI styling (yellow_bold, default_dimmed, purple_bold)
  • Improved function documentation and import organization
  • Pre-commit configuration refinements

Rationale: Taskserv better reflects the service-oriented nature of infrastructure components and improves semantic clarity throughout the codebase.

i18n/Localization System

New Feature: Fluent i18n integration for internationalized help system

Implementation:

  • nulib/main_provisioning/help_system_fluent.nu - Fluent-based i18n framework
  • Active locale detection from LANG environment variable
  • Fallback to English (en-US) for missing translations
  • Fluent catalog parsing: locale/{locale}/help.ftl
  • Locale format conversion: es_ES.UTF-8 → es-ES

Features:

  • Automatic locale detection from system LANG
  • Fluent catalog format support for translations
  • Graceful fallback mechanism
  • Category-based color formatting (infrastructure, orchestration, development, etc.)
  • Tab-separated help column formatting

📋 Version History

v1.0.10 (Previous Release)

  • Stable release with Nickel IaC support
  • Base version with core CLI and library system

v1.0.11 (Current - 2026-01-14)

  • Cluster → Taskserv terminology migration
  • Fluent i18n system documentation
  • Enhanced ANSI output formatting

📁 Changes by Directory

cli/ directory

Major Updates (586 lines added to provisioning)

  • Expanded CLI command implementations (+590 lines)
  • Enhanced tools installation system (tools-install: +163 lines)
  • Improved install script for Nushell environment (install_nu.sh: +31 lines)
  • Better CLI routing and command validation
  • Help system enhancements for Nickel-aware commands
  • Support for Nickel schema evaluation and validation

nulib/ directory

Nushell libraries - Nickel-first architecture

Config System

  • config/loader.nu - Nickel schema loading and evaluation
  • config/accessor.nu - Accessor patterns for Nickel fields
  • config/cache/ - Cache system optimized for Nickel evaluation

AI & Documentation

  • ai/README.md - Nickel IaC patterns
  • ai/info_about.md - Nickel-focused documentation
  • ai/lib.nu - AI integration for Nickel schema analysis

Extension System

  • extensions/QUICKSTART.md - Nickel extension quickstart (+50 lines)
  • extensions/README.md - Extension system for Nickel (+63 lines)
  • extensions/loader_oci.nu - OCI registry loader (minor updates)

Infrastructure & Validation

  • infra_validator/rules_engine.nu - Validation rules for Nickel schemas
  • infra_validator/validator.nu - Schema validation support
  • loader-minimal.nu - Minimal loader for lightweight deployments

Clusters & Workflows

  • clusters/discover.nu, clusters/load.nu, clusters/run.nu - Cluster operations updated
  • Plugin definitions updated for Nickel integration (+28-38 lines)

Documentation

  • SERVICE_MANAGEMENT_SUMMARY.md - Expanded service documentation (+90 lines)
  • gitea/IMPLEMENTATION_SUMMARY.md - Gitea integration guide (+89 lines)
  • Extension and validation quickstarts and README updates

plugins/ directory

Nushell plugins for performance optimization

Sub-repositories:

  • nushell-plugins/ - Multiple Nushell plugins
    • _nu_plugin_inquire/ - Interactive form plugin
    • api_nu_plugin_nickel/ - Nickel integration plugin
    • Additional plugin implementations

Plugin Documentation:

  • Build summaries
  • Installation guides
  • Configuration examples
  • Test documentation
  • Fix and limitation reports

scripts/ directory

Utility scripts for system operations

  • Build scripts
  • Installation scripts
  • Testing scripts
  • Development utilities
  • Infrastructure scripts

services/ directory

Service definitions and configurations

  • Service descriptions
  • Service management

forminquire/ directory (ARCHIVED)

Status: DEPRECATED - Archived to .coder/archive/forminquire/

Replacement: TypeDialog forms (.typedialog/provisioning/)

  • Legacy: Jinja2-based form system
  • Archived: 2025-01-09
  • Replaced by: TypeDialog with bash wrappers for TTY-safe input

Additional Files

  • README.md - Core system documentation
  • versions.ncl - Version definitions
  • .gitignore - Git ignore patterns
  • nickel.mod / nickel.mod.lock - Nickel module definitions
  • .githooks/ - Git hooks for development

📊 Change Statistics

Category Files Lines Added Lines Removed Status
CLI 3 780+ 30+ Major update
Config System 15+ 300+ 200+ Refactored
AI/Docs 8+ 350+ 100+ Enhanced
Extensions 5+ 150+ 50+ Updated
Infrastructure 8+ 100+ 70+ Updated
Clusters/Workflows 5+ 80+ 30+ Enhanced
Total 60+ files 1700+ lines 500+ lines Complete

✨ Key Areas

CLI System

  • Command implementations with Nickel support
  • Tools installation system
  • Nushell environment setup
  • Nickel schema evaluation commands
  • Error messages and help text
  • Nickel type checking and validation

Config System

  • Nickel-first loader: Schema evaluation via config/loader.nu
  • Optimized caching: Nickel evaluation cache patterns
  • Field accessors: Nickel record manipulation
  • Schema validation: Type-safe configuration loading

AI & Documentation

  • AI integration for Nickel IaC
  • Extension development guides
  • Service management documentation

Extensions & Infrastructure

  • OCI registry loader optimization
  • Schema-aware extension system
  • Infrastructure validation for Nickel definitions
  • Cluster discovery and operations enhanced

🎯 Current Features

  • Nickel IaC: Type-safe infrastructure definitions
  • CLI System: Unified command interface with 80+ shortcuts
  • Provider Abstraction: Cloud-agnostic operations
  • Config System: Hierarchical configuration with 476+ accessors
  • Workflow Engine: Batch operations with dependency resolution
  • Validation: Schema-aware infrastructure validation
  • AI Integration: Schema-driven configuration generation

Status: Production Date: 2026-01-14 Repository: provisioning/core Version: 1.0.11