## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Project Structure Guide

This document provides a comprehensive overview of the provisioning project’s structure after the major reorganization, explaining both the new development-focused organization and the preserved existing functionality.

## Table of Contents

1. Overview
2. New Structure vs Legacy
3. Core Directories
4. Development Workspace
5. File Naming Conventions
6. Navigation Guide
7. Migration Path

## Overview

The provisioning project has been restructured to support a dual-organization approach:

- `src/`: Development-focused structure with build tools, distribution system, and core components
- Legacy directories: Preserved in their original locations for backward compatibility
- `workspace/`: Development workspace with tools and runtime management

This reorganization enables efficient development workflows while maintaining full backward compatibility with existing deployments.

## New Structure vs Legacy

### New Development Structure (`/src/`)

    src/
    ├── config/                      # System configuration
    ├── control-center/              # Control center application
    ├── control-center-ui/           # Web UI for control center
    ├── core/                        # Core system libraries
    ├── docs/                        # Documentation (new)
    ├── extensions/                  # Extension framework
    ├── generators/                  # Code generation tools
    ├── schemas/                     # Nickel configuration schemas (migrated from kcl/)
    ├── orchestrator/               # Hybrid Rust/Nushell orchestrator
    ├── platform/                   # Platform-specific code
    ├── provisioning/               # Main provisioning
    ├── templates/                   # Template files
    ├── tools/                      # Build and development tools
    └── utils/                      # Utility scripts

### Legacy Structure (Preserved)

    repo-cnz/
    ├── cluster/                     # Cluster configurations (preserved)
    ├── core/                        # Core system (preserved)
    ├── generate/                    # Generation scripts (preserved)
    ├── schemas/                     # Nickel schemas (migrated from kcl/)
    ├── klab/                       # Development lab (preserved)
    ├── nushell-plugins/            # Plugin development (preserved)
    ├── providers/                  # Cloud providers (preserved)
    ├── taskservs/                  # Task services (preserved)
    └── templates/                  # Template files (preserved)

### Development Workspace (`/workspace/`)

    workspace/
    ├── config/                     # Development configuration
    ├── extensions/                 # Extension development
    ├── infra/                      # Development infrastructure
    ├── lib/                        # Workspace libraries
    ├── runtime/                    # Runtime data
    └── tools/                      # Workspace management tools

## Core Directories

### `/src/core/` - Core Development Libraries

Purpose: Development-focused core libraries and entry points

Key Files:

- `nulib/provisioning` - Main CLI entry point (symlinks to legacy location)
- `nulib/lib_provisioning/` - Core provisioning libraries
- `nulib/workflows/` - Workflow management (orchestrator integration)

Relationship to Legacy: Preserves original `core/` functionality while adding development enhancements

### `/src/tools/` - Build and Development Tools

Purpose: Complete build system for the provisioning project

Key Components:

    tools/
    ├── build/                      # Build tools
    │   ├── compile-platform.nu     # Platform-specific compilation
    │   ├── bundle-core.nu          # Core library bundling
    │   ├── validate-nickel.nu      # Nickel schema validation
    │   ├── clean-build.nu          # Build cleanup
    │   └── test-distribution.nu    # Distribution testing
    ├── distribution/               # Distribution tools
    │   ├── generate-distribution.nu # Main distribution generator
    │   ├── prepare-platform-dist.nu # Platform-specific distribution
    │   ├── prepare-core-dist.nu    # Core distribution
    │   ├── create-installer.nu     # Installer creation
    │   └── generate-docs.nu        # Documentation generation
    ├── package/                    # Packaging tools
    │   ├── package-binaries.nu     # Binary packaging
    │   ├── build-containers.nu     # Container image building
    │   ├── create-tarball.nu       # Archive creation
    │   └── validate-package.nu     # Package validation
    ├── release/                    # Release management
    │   ├── create-release.nu       # Release creation
    │   ├── upload-artifacts.nu     # Artifact upload
    │   ├── rollback-release.nu     # Release rollback
    │   ├── notify-users.nu         # Release notifications
    │   └── update-registry.nu      # Package registry updates
    └── Makefile                    # Main build system (40+ targets)

### `/src/orchestrator/` - Hybrid Orchestrator

Purpose: Rust/Nushell hybrid orchestrator for solving deep call stack limitations

Key Components:

- `src/` - Rust orchestrator implementation
- `scripts/` - Orchestrator management scripts
- `data/` - File-based task queue and persistence

Integration: Provides REST API and workflow management while preserving all Nushell business logic

### `/src/provisioning/` - Enhanced Provisioning

Purpose: Enhanced version of the main provisioning with additional features

Key Features:

- Batch workflow system (v3.1.0)
- Provider-agnostic design
- Configuration-driven architecture (v2.0.0)

### `/workspace/` - Development Workspace

Purpose: Complete development environment with tools and runtime management

Key Components:

- `tools/workspace.nu` - Unified workspace management interface
- `lib/path-resolver.nu` - Smart path resolution system
- `config/` - Environment-specific development configurations
- `extensions/` - Extension development templates and examples
- `infra/` - Development infrastructure examples
- `runtime/` - Isolated runtime data per user

## Development Workspace

### Workspace Management

The workspace provides a sophisticated development environment:

Initialization:

    cd workspace/tools
    nu workspace.nu init --user-name developer --infra-name my-infra

Health Monitoring:

    nu workspace.nu health --detailed --fix-issues

Path Resolution:

    use lib/path-resolver.nu
    let config = (path-resolver resolve_config "user" --workspace-user "john")

### Extension Development

The workspace provides templates for developing:

- Providers: Custom cloud provider implementations
- Task Services: Infrastructure service components
- Clusters: Complete deployment solutions

Templates are available in `workspace/extensions/{type}/template/`

### Configuration Hierarchy

The workspace implements a sophisticated configuration cascade:

1. Workspace user configuration (`workspace/config/{user}.toml`)
2. Environment-specific defaults (`workspace/config/{env}-defaults.toml`)
3. Workspace defaults (`workspace/config/dev-defaults.toml`)
4. Core system defaults (`config.defaults.toml`)

## File Naming Conventions

### Nushell Files (`.nu`)

- Commands: `kebab-case` - `create-server.nu`, `validate-config.nu`
- Modules: `snake_case` - `lib_provisioning`, `path_resolver`
- Scripts: `kebab-case` - `workspace-health.nu`, `runtime-manager.nu`

### Configuration Files

- TOML: `kebab-case.toml` - `config-defaults.toml`, `user-settings.toml`
- Environment: `{env}-defaults.toml` - `dev-defaults.toml`, `prod-defaults.toml`
- Examples: `*.toml.example` - `local-overrides.toml.example`

### Nickel Files (`.ncl`)

- Schemas: `kebab-case.ncl` - `server-config.ncl`, `workflow-schema.ncl`
- Configuration: `manifest.toml` - Package metadata
- Structure: Organized in `schemas/` directories per extension

### Build and Distribution

- Scripts: `kebab-case.nu` - `compile-platform.nu`, `generate-distribution.nu`
- Makefiles: `Makefile` - Standard naming
- Archives: `{project}-{version}-{platform}-{variant}.{ext}`

## Navigation Guide

### Finding Components

Core System Entry Points:

    # Main CLI (development version)
    /src/core/nulib/provisioning

    # Legacy CLI (production version)
    /core/nulib/provisioning

    # Workspace management
    /workspace/tools/workspace.nu

Build System:

    # Main build system
    cd /src/tools && make help

    # Quick development build
    make dev-build

    # Complete distribution
    make all

Configuration Files:

    # System defaults
    /config.defaults.toml

    # User configuration (workspace)
    /workspace/config/{user}.toml

    # Environment-specific
    /workspace/config/{env}-defaults.toml

Extension Development:

    # Provider template
    /workspace/extensions/providers/template/

    # Task service template
    /workspace/extensions/taskservs/template/

    # Cluster template
    /workspace/extensions/clusters/template/

### Common Workflows

1. Development Setup:

    # Initialize workspace
    cd workspace/tools
    nu workspace.nu init --user-name $USER

    # Check health
    nu workspace.nu health --detailed

2. Building Distribution:

    # Complete build
    cd src/tools
    make all

    # Platform-specific build
    make linux
    make macos
    make windows

3. Extension Development:

    # Create new provider
    cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider

    # Test extension
    nu workspace/extensions/providers/my-provider/nulib/provider.nu test

### Legacy Compatibility

Existing Commands Still Work:

    # All existing commands preserved
    ./core/nulib/provisioning server create
    ./core/nulib/provisioning taskserv install kubernetes
    ./core/nulib/provisioning cluster create buildkit

Configuration Migration:

- ENV variables still supported as fallbacks
- New configuration system provides better defaults
- Migration tools available in `src/tools/migration/`

## Migration Path

### For Users

No Changes Required:

- All existing commands continue to work
- Configuration files remain compatible
- Existing infrastructure deployments unaffected

Optional Enhancements:

- Migrate to new configuration system for better defaults
- Use workspace for development environments
- Leverage new build system for custom distributions

### For Developers

Development Environment:

1. Initialize development workspace: `nu workspace/tools/workspace.nu init`
2. Use new build system: `cd src/tools && make dev-build`
3. Leverage extension templates for custom development

Build System:

1. Use new Makefile for comprehensive build management
2. Leverage distribution tools for packaging
3. Use release management for version control

Orchestrator Integration:

1. Start orchestrator for workflow management: `cd src/orchestrator && ./scripts/start-orchestrator.nu`
2. Use workflow APIs for complex operations
3. Leverage batch operations for efficiency

### Migration Tools

Available Migration Scripts:

- `src/tools/migration/config-migration.nu` - Configuration migration
- `src/tools/migration/workspace-setup.nu` - Workspace initialization
- `src/tools/migration/path-resolver.nu` - Path resolution migration

Validation Tools:

- `src/tools/validation/system-health.nu` - System health validation
- `src/tools/validation/compatibility-check.nu` - Compatibility verification
- `src/tools/validation/migration-status.nu` - Migration status tracking

## Architecture Benefits

### Development Efficiency

- Build System: Comprehensive 40+ target Makefile system
- Workspace Isolation: Per-user development environments
- Extension Framework: Template-based extension development

### Production Reliability

- Backward Compatibility: All existing functionality preserved
- Configuration Migration: Gradual migration from ENV to config-driven
- Orchestrator Architecture: Hybrid Rust/Nushell for performance and flexibility
- Workflow Management: Batch operations with rollback capabilities

### Maintenance Benefits

- Clean Separation: Development tools separate from production code
- Organized Structure: Logical grouping of related functionality
- Documentation: Comprehensive documentation and examples
- Testing Framework: Built-in testing and validation tools

This structure represents a significant evolution in the project’s organization while maintaining complete backward compatibility and providing powerful new development capabilities.