Project Structure Guide

This document provides a comprehensive overview of the provisioning project’s structure after the major reorganization, explaining both the new development-focused organization and the preserved existing functionality.

Table of Contents

1. Overview
2. New Structure vs Legacy
3. Core Directories
4. Development Workspace
5. File Naming Conventions
6. Navigation Guide
7. Migration Path

Overview

The provisioning project has been restructured to support a dual-organization approach:

• src/: Development-focused structure with build tools, distribution system, and core components
• Legacy directories: Preserved in their original locations for backward compatibility
• workspace/: Development workspace with tools and runtime management

This reorganization enables efficient development workflows while maintaining full backward compatibility with existing deployments.

New Structure vs Legacy

New Development Structure (/src/)

src/
├── config/                      # System configuration
├── control-center/              # Control center application
├── control-center-ui/           # Web UI for control center
├── core/                        # Core system libraries
├── docs/                        # Documentation (new)
├── extensions/                  # Extension framework
├── generators/                  # Code generation tools
├── kcl/                         # KCL configuration language files
├── orchestrator/               # Hybrid Rust/Nushell orchestrator
├── platform/                   # Platform-specific code
├── provisioning/               # Main provisioning
├── templates/                   # Template files
├── tools/                      # Build and development tools
└── utils/                      # Utility scripts

Legacy Structure (Preserved)

repo-cnz/
├── cluster/                     # Cluster configurations (preserved)
├── core/                        # Core system (preserved)
├── generate/                    # Generation scripts (preserved)
├── kcl/                        # KCL files (preserved)
├── klab/                       # Development lab (preserved)
├── nushell-plugins/            # Plugin development (preserved)
├── providers/                  # Cloud providers (preserved)
├── taskservs/                  # Task services (preserved)
└── templates/                  # Template files (preserved)

Development Workspace (/workspace/)

workspace/
├── config/                     # Development configuration
├── extensions/                 # Extension development
├── infra/                      # Development infrastructure
├── lib/                        # Workspace libraries
├── runtime/                    # Runtime data
└── tools/                      # Workspace management tools

Core Directories

/src/core/ - Core Development Libraries

Purpose: Development-focused core libraries and entry points

Key Files:

• nulib/provisioning - Main CLI entry point (symlinks to legacy location)
• nulib/lib_provisioning/ - Core provisioning libraries
• nulib/workflows/ - Workflow management (orchestrator integration)

Relationship to Legacy: Preserves original core/ functionality while adding development enhancements

/src/tools/ - Build and Development Tools

Purpose: Complete build system for the provisioning project

Key Components:

tools/
├── build/                      # Build tools
│   ├── compile-platform.nu     # Platform-specific compilation
│   ├── bundle-core.nu          # Core library bundling
│   ├── validate-kcl.nu         # KCL validation
│   ├── clean-build.nu          # Build cleanup
│   └── test-distribution.nu    # Distribution testing
├── distribution/               # Distribution tools
│   ├── generate-distribution.nu # Main distribution generator
│   ├── prepare-platform-dist.nu # Platform-specific distribution
│   ├── prepare-core-dist.nu    # Core distribution
│   ├── create-installer.nu     # Installer creation
│   └── generate-docs.nu        # Documentation generation
├── package/                    # Packaging tools
│   ├── package-binaries.nu     # Binary packaging
│   ├── build-containers.nu     # Container image building
│   ├── create-tarball.nu       # Archive creation
│   └── validate-package.nu     # Package validation
├── release/                    # Release management
│   ├── create-release.nu       # Release creation
│   ├── upload-artifacts.nu     # Artifact upload
│   ├── rollback-release.nu     # Release rollback
│   ├── notify-users.nu         # Release notifications
│   └── update-registry.nu      # Package registry updates
└── Makefile                    # Main build system (40+ targets)

/src/orchestrator/ - Hybrid Orchestrator

Purpose: Rust/Nushell hybrid orchestrator for solving deep call stack limitations

Key Components:

• src/ - Rust orchestrator implementation
• scripts/ - Orchestrator management scripts
• data/ - File-based task queue and persistence

Integration: Provides REST API and workflow management while preserving all Nushell business logic

/src/provisioning/ - Enhanced Provisioning

Purpose: Enhanced version of the main provisioning with additional features

Key Features:

• Batch workflow system (v3.1.0)
• Provider-agnostic design
• Configuration-driven architecture (v2.0.0)

/workspace/ - Development Workspace

Purpose: Complete development environment with tools and runtime management

Key Components:

• tools/workspace.nu - Unified workspace management interface
• lib/path-resolver.nu - Smart path resolution system
• config/ - Environment-specific development configurations
• extensions/ - Extension development templates and examples
• infra/ - Development infrastructure examples
• runtime/ - Isolated runtime data per user

Development Workspace

Workspace Management

The workspace provides a sophisticated development environment:

Initialization:

cd workspace/tools
nu workspace.nu init --user-name developer --infra-name my-infra

Health Monitoring:

nu workspace.nu health --detailed --fix-issues

Path Resolution:

use lib/path-resolver.nu
let config = (path-resolver resolve_config "user" --workspace-user "john")

Extension Development

The workspace provides templates for developing:

• Providers: Custom cloud provider implementations
• Task Services: Infrastructure service components
• Clusters: Complete deployment solutions

Templates are available in workspace/extensions/{type}/template/

Configuration Hierarchy

The workspace implements a sophisticated configuration cascade:

1. Workspace user configuration (workspace/config/{user}.toml)
2. Environment-specific defaults (workspace/config/{env}-defaults.toml)
3. Workspace defaults (workspace/config/dev-defaults.toml)
4. Core system defaults (config.defaults.toml)

File Naming Conventions

Nushell Files (.nu)

• Commands: kebab-case - create-server.nu, validate-config.nu
• Modules: snake_case - lib_provisioning, path_resolver
• Scripts: kebab-case - workspace-health.nu, runtime-manager.nu

Configuration Files

• TOML: kebab-case.toml - config-defaults.toml, user-settings.toml
• Environment: {env}-defaults.toml - dev-defaults.toml, prod-defaults.toml
• Examples: *.toml.example - local-overrides.toml.example

KCL Files (.k)

• Schemas: PascalCase types - ServerConfig, WorkflowDefinition
• Files: kebab-case.k - server-config.k, workflow-schema.k
• Modules: kcl.mod - Module definition files

Build and Distribution

• Scripts: kebab-case.nu - compile-platform.nu, generate-distribution.nu
• Makefiles: Makefile - Standard naming
• Archives: {project}-{version}-{platform}-{variant}.{ext}

Navigation Guide

Finding Components

Core System Entry Points:

# Main CLI (development version)
/src/core/nulib/provisioning

# Legacy CLI (production version)
/core/nulib/provisioning

# Workspace management
/workspace/tools/workspace.nu

Build System:

# Main build system
cd /src/tools && make help

# Quick development build
make dev-build

# Complete distribution
make all

Configuration Files:

# System defaults
/config.defaults.toml

# User configuration (workspace)
/workspace/config/{user}.toml

# Environment-specific
/workspace/config/{env}-defaults.toml

Extension Development:

# Provider template
/workspace/extensions/providers/template/

# Task service template
/workspace/extensions/taskservs/template/

# Cluster template
/workspace/extensions/clusters/template/

Common Workflows

1. Development Setup:

# Initialize workspace
cd workspace/tools
nu workspace.nu init --user-name $USER

# Check health
nu workspace.nu health --detailed

2. Building Distribution:

# Complete build
cd src/tools
make all

# Platform-specific build
make linux
make macos
make windows

3. Extension Development:

# Create new provider
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider

# Test extension
nu workspace/extensions/providers/my-provider/nulib/provider.nu test

Legacy Compatibility

Existing Commands Still Work:

# All existing commands preserved
./core/nulib/provisioning server create
./core/nulib/provisioning taskserv install kubernetes
./core/nulib/provisioning cluster create buildkit

Configuration Migration:

• ENV variables still supported as fallbacks
• New configuration system provides better defaults
• Migration tools available in src/tools/migration/

Migration Path

For Users

No Changes Required:

• All existing commands continue to work
• Configuration files remain compatible
• Existing infrastructure deployments unaffected

Optional Enhancements:

• Migrate to new configuration system for better defaults
• Use workspace for development environments
• Leverage new build system for custom distributions

For Developers

Development Environment:

1. Initialize development workspace: nu workspace/tools/workspace.nu init
2. Use new build system: cd src/tools && make dev-build
3. Leverage extension templates for custom development

Build System:

1. Use new Makefile for comprehensive build management
2. Leverage distribution tools for packaging
3. Use release management for version control

Orchestrator Integration:

1. Start orchestrator for workflow management: cd src/orchestrator && ./scripts/start-orchestrator.nu
2. Use workflow APIs for complex operations
3. Leverage batch operations for efficiency

Migration Tools

Available Migration Scripts:

• src/tools/migration/config-migration.nu - Configuration migration
• src/tools/migration/workspace-setup.nu - Workspace initialization
• src/tools/migration/path-resolver.nu - Path resolution migration

Validation Tools:

• src/tools/validation/system-health.nu - System health validation
• src/tools/validation/compatibility-check.nu - Compatibility verification
• src/tools/validation/migration-status.nu - Migration status tracking

Architecture Benefits

Development Efficiency

• Build System: Comprehensive 40+ target Makefile system
• Workspace Isolation: Per-user development environments
• Extension Framework: Template-based extension development

Production Reliability

• Backward Compatibility: All existing functionality preserved
• Configuration Migration: Gradual migration from ENV to config-driven
• Orchestrator Architecture: Hybrid Rust/Nushell for performance and flexibility
• Workflow Management: Batch operations with rollback capabilities

Maintenance Benefits

• Clean Separation: Development tools separate from production code
• Organized Structure: Logical grouping of related functionality
• Documentation: Comprehensive documentation and examples
• Testing Framework: Built-in testing and validation tools

This structure represents a significant evolution in the project’s organization while maintaining complete backward compatibility and providing powerful new development capabilities.