894046ef5a
- DAG architecture: `dag show/validate/export` (nulib/main_provisioning/dag.nu),
config loader (lib_provisioning/config/loader/dag.nu), taskserv dag-executor.
Backed by schemas/lib/dag/*.ncl; orchestrator emits NATS events via
WorkspaceComposition::into_workflow. See ADR-020, ADR-021.
- Unified Component Architecture: components/mod.nu, main_provisioning/
{components,workflow,extensions,ontoref-queries}.nu. Full workflow engine with
topological sort and NATS subject emission. Blocks A-H complete (libre-daoshi).
- Commands-registry: nulib/commands-registry.ncl (Nickel source, 314 lines) +
JSON cache at ~/.cache/provisioning/commands-registry.json rebuilt on source
change. cli/provisioning fast-path alias expansion avoids cold Nu startup.
ADDING_COMMANDS.md documents new-command workflow.
- Platform service manager: service-manager.nu (+573), startup.nu (+611),
service-check.nu (+255); autostart/bootstrap/health/target refactored.
- Nushell 0.112.2 migration: removed all try/catch and bash redirections;
external commands prefixed with ^; type signatures enforced. Driven by
scripts/refactor-try-catch{,-simplified}.nu.
- TTY stack: removed shlib/*-tty.sh; replaced by cli/tty-dispatch.sh,
tty-filter.sh, tty-commands.conf.
- New domain modules: images/ (golden image lifecycle), workspace/{state,sync}.nu,
main_provisioning/{bootstrap,cluster-deploy,fip,state}.nu, commands/{state,
build,integrations/auth,utilities/alias}.nu, platform.nu expanded (+874).
- Config loader overhaul: loader/core.nu slimmed (-759), cache/core.nu
refactored (-454), removed legacy loaders/file_loader.nu (-330).
- Thirteen new provisioning-<domain>.nu top-level modules for bash dispatcher.
- Tests: test_workspace_state.nu (+351); updates to test_oci_registry,
test_services.
- README + CHANGELOG updated.
381 lines
13 KiB
Markdown
381 lines
13 KiB
Markdown
# Provisioning Core - Changelog
|
||
|
||
**Date**: 2026-04-17
|
||
**Repository**: provisioning/core
|
||
**Status**: Nickel IaC (PRIMARY)
|
||
|
||
---
|
||
|
||
## 📋 Summary
|
||
|
||
Major refactor: three-layer DAG architecture with workspace composition, Unified
|
||
Component Architecture (components + workflows + capabilities), Nickel-backed
|
||
commands-registry with JSON cache for fast CLI startup, consolidated platform
|
||
service manager, and completed Nushell 0.110/0.112 migration (no try/catch, no
|
||
bash redirections). TTY stack moved from `shlib/` into `cli/tty-*`. Numerous new
|
||
domain modules: `dag`, `components`, `workflow` engine, `images` lifecycle,
|
||
workspace state/sync, ontoref queries, FIP handler.
|
||
|
||
---
|
||
|
||
## 🔄 Latest Release (2026-04-17)
|
||
|
||
### Three-Layer DAG Architecture
|
||
|
||
**Scope**: Workspace composition as a DAG with formula_id::task_name namespacing,
|
||
health gates, conditions, and NATS subject emission.
|
||
|
||
**New files**:
|
||
- `nulib/main_provisioning/dag.nu` — `dag show/validate/export` (DOT/JSON/Mermaid)
|
||
- `nulib/lib_provisioning/config/loader/dag.nu` — DAG config loader
|
||
- `nulib/taskservs/dag-executor.nu` — taskserv-level DAG execution helper
|
||
|
||
**Related**: ADR-020 (extension capability declarations), ADR-021 (workspace
|
||
composition DAG). Orchestrator consumes composition via
|
||
`WorkspaceComposition::into_workflow` and emits NATS events.
|
||
|
||
### Unified Component Architecture
|
||
|
||
**Scope**: Components + workflows + capabilities as first-class citizens
|
||
(libre-daoshi plan, blocks A-H complete).
|
||
|
||
**New files**:
|
||
- `nulib/components/mod.nu` — component dispatch module
|
||
- `nulib/main_provisioning/components.nu` — `validate capabilities/components`,
|
||
`component list/info`
|
||
- `nulib/main_provisioning/workflow.nu` — full workflow engine: run/list/status/
|
||
validate, topological sort, NATS event emission (+605 lines)
|
||
- `nulib/main_provisioning/extensions.nu` — `extensions capabilities/graph`
|
||
- `nulib/main_provisioning/ontoref-queries.nu` — on+re-aware CLI queries
|
||
(describe component/databases/namespace/storage/workflow)
|
||
|
||
### Commands-Registry & Fast-Path Dispatch
|
||
|
||
**Scope**: Eliminate Nu startup cost on every `prvng` invocation.
|
||
|
||
**New files**:
|
||
- `nulib/commands-registry.ncl` — Nickel command catalog (314 lines)
|
||
- `nulib/lib_provisioning/utils/command-registry.nu` — registry accessor
|
||
- `nulib/scripts/validate-command.nu` — cache-aware command validator
|
||
|
||
**Behavior**: `cli/provisioning` reads the JSON cache at
|
||
`~/.cache/provisioning/commands-registry.json`, rebuilt automatically via
|
||
`nickel export` when the `.ncl` source is newer. Single-char aliases
|
||
(`s`, `t`, `c`, `e`, `w`, `j`, `b`, `o`, `a`) are expanded in bash before
|
||
dispatch. `nulib/main_provisioning/ADDING_COMMANDS.md` documents the four-step
|
||
procedure for new commands.
|
||
|
||
### Platform Service Manager
|
||
|
||
**New files**:
|
||
- `nulib/lib_provisioning/platform/service-manager.nu` (+573 lines)
|
||
- `nulib/lib_provisioning/platform/startup.nu` (+611 lines)
|
||
- `nulib/lib_provisioning/utils/service-check.nu` (+255 lines)
|
||
|
||
**Refactored**: `platform/autostart.nu`, `platform/bootstrap.nu`,
|
||
`platform/health.nu`, `platform/target.nu` — unified lifecycle, health probes,
|
||
and autostart logic.
|
||
|
||
### Nushell 0.112.2 Migration
|
||
|
||
**Scope**: Project-wide refactor driven by `scripts/refactor-try-catch.nu` and
|
||
`scripts/refactor-try-catch-simplified.nu` to reach Nushell 0.112.2 compliance.
|
||
|
||
**Enforced**:
|
||
- No `try/catch` — all use `do { } | complete`
|
||
- No bash redirections (`2>&1`, `2>/dev/null`)
|
||
- External commands prefixed with `^`
|
||
- Parenthesized pipelines in `if`
|
||
- Type signatures: `def f [x: string]: nothing -> record { }`
|
||
|
||
### TTY Stack Replacement
|
||
|
||
**Removed**: `shlib/README.md`, `shlib/auth-login-tty.sh`,
|
||
`shlib/mfa-enroll-tty.sh`, `shlib/setup-wizard-tty.sh`.
|
||
|
||
**Replaced by**:
|
||
- `cli/tty-dispatch.sh` (+86 lines) — TTY-safe command dispatcher
|
||
- `cli/tty-filter.sh` (+137 lines) — command filter
|
||
- `cli/tty-commands.conf` — TTY command manifest
|
||
|
||
### New Domain Modules
|
||
|
||
- `nulib/images/` — golden image lifecycle (`create`, `delete`, `list`, `state`,
|
||
`update`, `watch`)
|
||
- `nulib/workspace/state.nu` (+641 lines) — workspace state model
|
||
- `nulib/workspace/sync.nu` (+148 lines) — workspace synchronization
|
||
- `nulib/main_provisioning/bootstrap.nu` — platform bootstrap
|
||
- `nulib/main_provisioning/cluster-deploy.nu` — component/taskserv dispatch
|
||
- `nulib/main_provisioning/fip.nu` (+421 lines) — floating IP handler
|
||
- `nulib/main_provisioning/state.nu` — state command
|
||
- `nulib/main_provisioning/commands/state.nu`, `commands/build.nu`,
|
||
`commands/integrations/auth.nu`, `commands/utilities/alias.nu`
|
||
- `nulib/main_provisioning/commands/platform.nu` — major expansion (+874 lines)
|
||
|
||
### Config Loader Overhaul
|
||
|
||
- `lib_provisioning/config/loader/core.nu` — slimmed (−759 lines of legacy paths)
|
||
- `lib_provisioning/config/cache/core.nu` — refactored (−454 lines of dead paths)
|
||
- `lib_provisioning/config/cache/nickel.nu` — simplified
|
||
- Removed: `lib_provisioning/config/loaders/file_loader.nu` (−330 lines)
|
||
- Added: `config/accessor-minimal.nu`, `config/accessor/functions.nu` helpers
|
||
|
||
### Scripts & Tooling
|
||
|
||
- `nulib/scripts/` — query-* family (clusters/infra/providers/servers/taskservs/
|
||
workspace-info), validate-command, validate-config
|
||
- `scripts/auto-refactor-priority.nu`, `scripts/batch-refactor.sh`
|
||
- `scripts/build-nixos-image-remote.sh`, `scripts/deploy-cp-server.sh`
|
||
|
||
### CLI Modular Subcommands
|
||
|
||
New top-level Nu modules referenced by the bash dispatcher:
|
||
`provisioning-batch.nu`, `provisioning-bootstrap.nu`, `provisioning-cluster.nu`,
|
||
`provisioning-component.nu`, `provisioning-extension.nu`, `provisioning-job.nu`,
|
||
`provisioning-platform.nu`, `provisioning-server.nu`, `provisioning-state.nu`,
|
||
`provisioning-status.nu`, `provisioning-taskserv.nu`, `provisioning-volume.nu`,
|
||
`provisioning-workflow.nu`.
|
||
|
||
### Tests
|
||
|
||
- `nulib/tests/test_workspace_state.nu` (+351 lines)
|
||
- Updates to `test_oci_registry.nu`, `test_services.nu`
|
||
|
||
### Statistics
|
||
|
||
| Area | Files | Lines +/− |
|
||
| ---- | ----- | --------- |
|
||
| DAG + Components + Workflows | 8 | +1800 / −50 |
|
||
| Commands-registry + dispatch | 6 | +900 / −200 |
|
||
| Platform service manager | 5 | +1700 / −300 |
|
||
| Config loader/cache refactor | 10 | +400 / −1500 |
|
||
| TTY replacement | 4 | +250 / −515 |
|
||
| New subcommand modules | 13 | +1700 / −0 |
|
||
| **Total staged** | **242 files** | **+21949 / −6012** |
|
||
|
||
---
|
||
|
||
## 🔄 Previous Release (2026-01-14)
|
||
|
||
### Terminology Migration: Cluster → Taskserv
|
||
|
||
**Scope**: Complete refactoring across nulib/ modules to standardize on taskserv nomenclature
|
||
|
||
**Files Updated**:
|
||
- `nulib/clusters/handlers.nu` - Handler signature updates, ANSI formatting improvements
|
||
- `nulib/clusters/run.nu` - Function parameter and path updates (+326 lines modified)
|
||
- `nulib/clusters/utils.nu` - Utility function updates (+144 lines modified)
|
||
- `nulib/clusters/discover.nu` - Discovery module refactoring
|
||
- `nulib/clusters/load.nu` - Configuration loader updates
|
||
- `nulib/ai/query_processor.nu` - AI integration updates
|
||
- `nulib/api/routes.nu` - API routing adjustments
|
||
- `nulib/api/server.nu` - Server module updates
|
||
- `.pre-commit-config.yaml` - Pre-commit hook updates
|
||
|
||
**Changes**:
|
||
- Updated function parameters: `server_cluster_path` → `server_taskserv_path`
|
||
- Updated record fields: `defs.cluster.name` → `defs.taskserv.name`
|
||
- Enhanced output formatting with consistent ANSI styling (yellow_bold, default_dimmed, purple_bold)
|
||
- Improved function documentation and import organization
|
||
- Pre-commit configuration refinements
|
||
|
||
**Rationale**: Taskserv better reflects the service-oriented nature of infrastructure components and improves semantic clarity throughout the codebase.
|
||
|
||
### i18n/Localization System
|
||
|
||
**New Feature**: Fluent i18n integration for internationalized help system
|
||
|
||
**Implementation**:
|
||
- `nulib/main_provisioning/help_system_fluent.nu` - Fluent-based i18n framework
|
||
- Active locale detection from `LANG` environment variable
|
||
- Fallback to English (en-US) for missing translations
|
||
- Fluent catalog parsing: `locale/{locale}/help.ftl`
|
||
- Locale format conversion: `es_ES.UTF-8` → `es-ES`
|
||
|
||
**Features**:
|
||
- Automatic locale detection from system LANG
|
||
- Fluent catalog format support for translations
|
||
- Graceful fallback mechanism
|
||
- Category-based color formatting (infrastructure, orchestration, development, etc.)
|
||
- Tab-separated help column formatting
|
||
|
||
---
|
||
|
||
## 📋 Version History
|
||
|
||
### v1.0.10 (Previous Release)
|
||
- Stable release with Nickel IaC support
|
||
- Base version with core CLI and library system
|
||
|
||
### v1.0.11 (Current - 2026-01-14)
|
||
- **Cluster → Taskserv** terminology migration
|
||
- **Fluent i18n** system documentation
|
||
- Enhanced ANSI output formatting
|
||
|
||
---
|
||
|
||
## 📁 Changes by Directory
|
||
|
||
### cli/ directory
|
||
|
||
**Major Updates (586 lines added to provisioning)**
|
||
|
||
- Expanded CLI command implementations (+590 lines)
|
||
- Enhanced tools installation system (tools-install: +163 lines)
|
||
- Improved install script for Nushell environment (install_nu.sh: +31 lines)
|
||
- Better CLI routing and command validation
|
||
- Help system enhancements for Nickel-aware commands
|
||
- Support for Nickel schema evaluation and validation
|
||
|
||
### nulib/ directory
|
||
|
||
**Nushell libraries - Nickel-first architecture**
|
||
|
||
**Config System**
|
||
- `config/loader.nu` - Nickel schema loading and evaluation
|
||
- `config/accessor.nu` - Accessor patterns for Nickel fields
|
||
- `config/cache/` - Cache system optimized for Nickel evaluation
|
||
|
||
**AI & Documentation**
|
||
- `ai/README.md` - Nickel IaC patterns
|
||
- `ai/info_about.md` - Nickel-focused documentation
|
||
- `ai/lib.nu` - AI integration for Nickel schema analysis
|
||
|
||
**Extension System**
|
||
- `extensions/QUICKSTART.md` - Nickel extension quickstart (+50 lines)
|
||
- `extensions/README.md` - Extension system for Nickel (+63 lines)
|
||
- `extensions/loader_oci.nu` - OCI registry loader (minor updates)
|
||
|
||
**Infrastructure & Validation**
|
||
- `infra_validator/rules_engine.nu` - Validation rules for Nickel schemas
|
||
- `infra_validator/validator.nu` - Schema validation support
|
||
- `loader-minimal.nu` - Minimal loader for lightweight deployments
|
||
|
||
**Clusters & Workflows**
|
||
- `clusters/discover.nu`, `clusters/load.nu`, `clusters/run.nu` - Cluster operations updated
|
||
- Plugin definitions updated for Nickel integration (+28-38 lines)
|
||
|
||
**Documentation**
|
||
- `SERVICE_MANAGEMENT_SUMMARY.md` - Expanded service documentation (+90 lines)
|
||
- `gitea/IMPLEMENTATION_SUMMARY.md` - Gitea integration guide (+89 lines)
|
||
- Extension and validation quickstarts and README updates
|
||
|
||
### plugins/ directory
|
||
|
||
Nushell plugins for performance optimization
|
||
|
||
**Sub-repositories:**
|
||
|
||
- `nushell-plugins/` - Multiple Nushell plugins
|
||
- `_nu_plugin_inquire/` - Interactive form plugin
|
||
- `api_nu_plugin_nickel/` - Nickel integration plugin
|
||
- Additional plugin implementations
|
||
|
||
**Plugin Documentation:**
|
||
|
||
- Build summaries
|
||
- Installation guides
|
||
- Configuration examples
|
||
- Test documentation
|
||
- Fix and limitation reports
|
||
|
||
### scripts/ directory
|
||
|
||
Utility scripts for system operations
|
||
|
||
- Build scripts
|
||
- Installation scripts
|
||
- Testing scripts
|
||
- Development utilities
|
||
- Infrastructure scripts
|
||
|
||
### services/ directory
|
||
|
||
Service definitions and configurations
|
||
|
||
- Service descriptions
|
||
- Service management
|
||
|
||
### forminquire/ directory (ARCHIVED)
|
||
|
||
**Status**: DEPRECATED - Archived to `.coder/archive/forminquire/`
|
||
|
||
**Replacement**: TypeDialog forms (`.typedialog/provisioning/`)
|
||
|
||
- Legacy: Jinja2-based form system
|
||
- Archived: 2025-01-09
|
||
- Replaced by: TypeDialog with bash wrappers for TTY-safe input
|
||
|
||
### Additional Files
|
||
|
||
- `README.md` - Core system documentation
|
||
- `versions.ncl` - Version definitions
|
||
- `.gitignore` - Git ignore patterns
|
||
- `nickel.mod` / `nickel.mod.lock` - Nickel module definitions
|
||
- `.githooks/` - Git hooks for development
|
||
|
||
---
|
||
|
||
## 📊 Change Statistics
|
||
|
||
| Category | Files | Lines Added | Lines Removed | Status |
|
||
| -------- | ----- | ----------- | ------------- | ------ |
|
||
| CLI | 3 | 780+ | 30+ | Major update |
|
||
| Config System | 15+ | 300+ | 200+ | Refactored |
|
||
| AI/Docs | 8+ | 350+ | 100+ | Enhanced |
|
||
| Extensions | 5+ | 150+ | 50+ | Updated |
|
||
| Infrastructure | 8+ | 100+ | 70+ | Updated |
|
||
| Clusters/Workflows | 5+ | 80+ | 30+ | Enhanced |
|
||
| **Total** | **60+ files** | **1700+ lines** | **500+ lines** | **Complete** |
|
||
|
||
---
|
||
|
||
## ✨ Key Areas
|
||
|
||
### CLI System
|
||
|
||
- Command implementations with Nickel support
|
||
- Tools installation system
|
||
- Nushell environment setup
|
||
- Nickel schema evaluation commands
|
||
- Error messages and help text
|
||
- Nickel type checking and validation
|
||
|
||
### Config System
|
||
|
||
- **Nickel-first loader**: Schema evaluation via config/loader.nu
|
||
- **Optimized caching**: Nickel evaluation cache patterns
|
||
- **Field accessors**: Nickel record manipulation
|
||
- **Schema validation**: Type-safe configuration loading
|
||
|
||
### AI & Documentation
|
||
|
||
- AI integration for Nickel IaC
|
||
- Extension development guides
|
||
- Service management documentation
|
||
|
||
### Extensions & Infrastructure
|
||
|
||
- OCI registry loader optimization
|
||
- Schema-aware extension system
|
||
- Infrastructure validation for Nickel definitions
|
||
- Cluster discovery and operations enhanced
|
||
|
||
---
|
||
|
||
## 🎯 Current Features
|
||
|
||
- **Nickel IaC**: Type-safe infrastructure definitions
|
||
- **CLI System**: Unified command interface with 80+ shortcuts
|
||
- **Provider Abstraction**: Cloud-agnostic operations
|
||
- **Config System**: Hierarchical configuration with 476+ accessors
|
||
- **Workflow Engine**: Batch operations with dependency resolution
|
||
- **Validation**: Schema-aware infrastructure validation
|
||
- **AI Integration**: Schema-driven configuration generation
|
||
|
||
---
|
||
|
||
**Status**: Production
|
||
**Date**: 2026-01-14
|
||
**Repository**: provisioning/core
|
||
**Version**: 1.0.11
|