# Project Structure - Provisioning Platform Documentation
Complete documentation for the Provisioning Platform - Infrastructure automation with Nushell, KCL, and Rust

## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Project Structure Guide

This document provides a comprehensive overview of the provisioning project’s structure after the major reorganization, explaining both the new development-focused organization and the preserved existing functionality.

## Table of Contents

1. Overview
2. New Structure vs Legacy
3. Core Directories
4. Development Workspace
5. File Naming Conventions
6. Navigation Guide
7. Migration Path

## Overview

The provisioning project has been restructured to support a dual-organization approach:

- `src/`: Development-focused structure with build tools, distribution system, and core components
- Legacy directories: Preserved in their original locations for backward compatibility
- `workspace/`: Development workspace with tools and runtime management

This reorganization enables efficient development workflows while maintaining full backward compatibility with existing deployments.

## New Structure vs Legacy

### New Development Structure (`/src/`)

    src/
    ├── config/                      # System configuration
    ├── control-center/              # Control center application
    ├── control-center-ui/           # Web UI for control center
    ├── core/                        # Core system libraries
    ├── docs/                        # Documentation (new)
    ├── extensions/                  # Extension framework
    ├── generators/                  # Code generation tools
    ├── kcl/                         # KCL configuration language files
    ├── orchestrator/               # Hybrid Rust/Nushell orchestrator
    ├── platform/                   # Platform-specific code
    ├── provisioning/               # Main provisioning
    ├── templates/                   # Template files
    ├── tools/                      # Build and development tools
    └── utils/                      # Utility scripts
    ```plaintext

    ### Legacy Structure (Preserved)

    ```plaintext
    repo-cnz/
    ├── cluster/                     # Cluster configurations (preserved)
    ├── core/                        # Core system (preserved)
    ├── generate/                    # Generation scripts (preserved)
    ├── kcl/                        # KCL files (preserved)
    ├── klab/                       # Development lab (preserved)
    ├── nushell-plugins/            # Plugin development (preserved)
    ├── providers/                  # Cloud providers (preserved)
    ├── taskservs/                  # Task services (preserved)
    └── templates/                  # Template files (preserved)
    ```plaintext

    ### Development Workspace (`/workspace/`)

    ```plaintext
    workspace/
    ├── config/                     # Development configuration
    ├── extensions/                 # Extension development
    ├── infra/                      # Development infrastructure
    ├── lib/                        # Workspace libraries
    ├── runtime/                    # Runtime data
    └── tools/                      # Workspace management tools
    ```plaintext

    ## Core Directories

    ### `/src/core/` - Core Development Libraries

    **Purpose**: Development-focused core libraries and entry points

    **Key Files**:

    - `nulib/provisioning` - Main CLI entry point (symlinks to legacy location)
    - `nulib/lib_provisioning/` - Core provisioning libraries
    - `nulib/workflows/` - Workflow management (orchestrator integration)

    **Relationship to Legacy**: Preserves original `core/` functionality while adding development enhancements

    ### `/src/tools/` - Build and Development Tools

    **Purpose**: Complete build system for the provisioning project

    **Key Components**:

    ```plaintext
    tools/
    ├── build/                      # Build tools
    │   ├── compile-platform.nu     # Platform-specific compilation
    │   ├── bundle-core.nu          # Core library bundling
    │   ├── validate-kcl.nu         # KCL validation
    │   ├── clean-build.nu          # Build cleanup
    │   └── test-distribution.nu    # Distribution testing
    ├── distribution/               # Distribution tools
    │   ├── generate-distribution.nu # Main distribution generator
    │   ├── prepare-platform-dist.nu # Platform-specific distribution
    │   ├── prepare-core-dist.nu    # Core distribution
    │   ├── create-installer.nu     # Installer creation
    │   └── generate-docs.nu        # Documentation generation
    ├── package/                    # Packaging tools
    │   ├── package-binaries.nu     # Binary packaging
    │   ├── build-containers.nu     # Container image building
    │   ├── create-tarball.nu       # Archive creation
    │   └── validate-package.nu     # Package validation
    ├── release/                    # Release management
    │   ├── create-release.nu       # Release creation
    │   ├── upload-artifacts.nu     # Artifact upload
    │   ├── rollback-release.nu     # Release rollback
    │   ├── notify-users.nu         # Release notifications
    │   └── update-registry.nu      # Package registry updates
    └── Makefile                    # Main build system (40+ targets)
    ```plaintext

    ### `/src/orchestrator/` - Hybrid Orchestrator

    **Purpose**: Rust/Nushell hybrid orchestrator for solving deep call stack limitations

    **Key Components**:

    - `src/` - Rust orchestrator implementation
    - `scripts/` - Orchestrator management scripts
    - `data/` - File-based task queue and persistence

    **Integration**: Provides REST API and workflow management while preserving all Nushell business logic

    ### `/src/provisioning/` - Enhanced Provisioning

    **Purpose**: Enhanced version of the main provisioning with additional features

    **Key Features**:

    - Batch workflow system (v3.1.0)
    - Provider-agnostic design
    - Configuration-driven architecture (v2.0.0)

    ### `/workspace/` - Development Workspace

    **Purpose**: Complete development environment with tools and runtime management

    **Key Components**:

    - `tools/workspace.nu` - Unified workspace management interface
    - `lib/path-resolver.nu` - Smart path resolution system
    - `config/` - Environment-specific development configurations
    - `extensions/` - Extension development templates and examples
    - `infra/` - Development infrastructure examples
    - `runtime/` - Isolated runtime data per user

    ## Development Workspace

    ### Workspace Management

    The workspace provides a sophisticated development environment:

    **Initialization**:

    ```bash
    cd workspace/tools
    nu workspace.nu init --user-name developer --infra-name my-infra
    ```plaintext

    **Health Monitoring**:

    ```bash
    nu workspace.nu health --detailed --fix-issues
    ```plaintext

    **Path Resolution**:

    ```nushell
    use lib/path-resolver.nu
    let config = (path-resolver resolve_config "user" --workspace-user "john")
    ```plaintext

    ### Extension Development

    The workspace provides templates for developing:

    - **Providers**: Custom cloud provider implementations
    - **Task Services**: Infrastructure service components
    - **Clusters**: Complete deployment solutions

    Templates are available in `workspace/extensions/{type}/template/`

    ### Configuration Hierarchy

    The workspace implements a sophisticated configuration cascade:

    1. Workspace user configuration (`workspace/config/{user}.toml`)
    2. Environment-specific defaults (`workspace/config/{env}-defaults.toml`)
    3. Workspace defaults (`workspace/config/dev-defaults.toml`)
    4. Core system defaults (`config.defaults.toml`)

    ## File Naming Conventions

    ### Nushell Files (`.nu`)

    - **Commands**: `kebab-case` - `create-server.nu`, `validate-config.nu`
    - **Modules**: `snake_case` - `lib_provisioning`, `path_resolver`
    - **Scripts**: `kebab-case` - `workspace-health.nu`, `runtime-manager.nu`

    ### Configuration Files

    - **TOML**: `kebab-case.toml` - `config-defaults.toml`, `user-settings.toml`
    - **Environment**: `{env}-defaults.toml` - `dev-defaults.toml`, `prod-defaults.toml`
    - **Examples**: `*.toml.example` - `local-overrides.toml.example`

    ### KCL Files (`.k`)

    - **Schemas**: `PascalCase` types - `ServerConfig`, `WorkflowDefinition`
    - **Files**: `kebab-case.k` - `server-config.k`, `workflow-schema.k`
    - **Modules**: `kcl.mod` - Module definition files

    ### Build and Distribution

    - **Scripts**: `kebab-case.nu` - `compile-platform.nu`, `generate-distribution.nu`
    - **Makefiles**: `Makefile` - Standard naming
    - **Archives**: `{project}-{version}-{platform}-{variant}.{ext}`

    ## Navigation Guide

    ### Finding Components

    **Core System Entry Points**:

    ```bash
    # Main CLI (development version)
    /src/core/nulib/provisioning

    # Legacy CLI (production version)
    /core/nulib/provisioning

    # Workspace management
    /workspace/tools/workspace.nu
    ```plaintext

    **Build System**:

    ```bash
    # Main build system
    cd /src/tools && make help

    # Quick development build
    make dev-build

    # Complete distribution
    make all
    ```plaintext

    **Configuration Files**:

    ```bash
    # System defaults
    /config.defaults.toml

    # User configuration (workspace)
    /workspace/config/{user}.toml

    # Environment-specific
    /workspace/config/{env}-defaults.toml
    ```plaintext

    **Extension Development**:

    ```bash
    # Provider template
    /workspace/extensions/providers/template/

    # Task service template
    /workspace/extensions/taskservs/template/

    # Cluster template
    /workspace/extensions/clusters/template/
    ```plaintext

    ### Common Workflows

    **1. Development Setup**:

    ```bash
    # Initialize workspace
    cd workspace/tools
    nu workspace.nu init --user-name $USER

    # Check health
    nu workspace.nu health --detailed
    ```plaintext

    **2. Building Distribution**:

    ```bash
    # Complete build
    cd src/tools
    make all

    # Platform-specific build
    make linux
    make macos
    make windows
    ```plaintext

    **3. Extension Development**:

    ```bash
    # Create new provider
    cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider

    # Test extension
    nu workspace/extensions/providers/my-provider/nulib/provider.nu test
    ```plaintext

    ### Legacy Compatibility

    **Existing Commands Still Work**:

    ```bash
    # All existing commands preserved
    ./core/nulib/provisioning server create
    ./core/nulib/provisioning taskserv install kubernetes
    ./core/nulib/provisioning cluster create buildkit
    ```plaintext

    **Configuration Migration**:

    - ENV variables still supported as fallbacks
    - New configuration system provides better defaults
    - Migration tools available in `src/tools/migration/`

    ## Migration Path

    ### For Users

    **No Changes Required**:

    - All existing commands continue to work
    - Configuration files remain compatible
    - Existing infrastructure deployments unaffected

    **Optional Enhancements**:

    - Migrate to new configuration system for better defaults
    - Use workspace for development environments
    - Leverage new build system for custom distributions

    ### For Developers

    **Development Environment**:

    1. Initialize development workspace: `nu workspace/tools/workspace.nu init`
    2. Use new build system: `cd src/tools && make dev-build`
    3. Leverage extension templates for custom development

    **Build System**:

    1. Use new Makefile for comprehensive build management
    2. Leverage distribution tools for packaging
    3. Use release management for version control

    **Orchestrator Integration**:

    1. Start orchestrator for workflow management: `cd src/orchestrator && ./scripts/start-orchestrator.nu`
    2. Use workflow APIs for complex operations
    3. Leverage batch operations for efficiency

    ### Migration Tools

    **Available Migration Scripts**:

    - `src/tools/migration/config-migration.nu` - Configuration migration
    - `src/tools/migration/workspace-setup.nu` - Workspace initialization
    - `src/tools/migration/path-resolver.nu` - Path resolution migration

    **Validation Tools**:

    - `src/tools/validation/system-health.nu` - System health validation
    - `src/tools/validation/compatibility-check.nu` - Compatibility verification
    - `src/tools/validation/migration-status.nu` - Migration status tracking

    ## Architecture Benefits

    ### Development Efficiency

    - **Build System**: Comprehensive 40+ target Makefile system
    - **Workspace Isolation**: Per-user development environments
    - **Extension Framework**: Template-based extension development

    ### Production Reliability

    - **Backward Compatibility**: All existing functionality preserved
    - **Configuration Migration**: Gradual migration from ENV to config-driven
    - **Orchestrator Architecture**: Hybrid Rust/Nushell for performance and flexibility
    - **Workflow Management**: Batch operations with rollback capabilities

    ### Maintenance Benefits

    - **Clean Separation**: Development tools separate from production code
    - **Organized Structure**: Logical grouping of related functionality
    - **Documentation**: Comprehensive documentation and examples
    - **Testing Framework**: Built-in testing and validation tools

    This structure represents a significant evolution in the project's organization while maintaining complete backward compatibility and providing powerful new development capabilities.