# Project Structure Guide\n\nThis document provides a comprehensive overview of the provisioning project's structure after the major reorganization, explaining both the new\ndevelopment-focused organization and the preserved existing functionality.\n\n## Table of Contents\n\n1. [Overview](#overview)\n2. [New Structure vs Legacy](#new-structure-vs-legacy)\n3. [Core Directories](#core-directories)\n4. [Development Workspace](#development-workspace)\n5. [File Naming Conventions](#file-naming-conventions)\n6. [Navigation Guide](#navigation-guide)\n7. [Migration Path](#migration-path)\n\n## Overview\n\nThe provisioning project has been restructured to support a dual-organization approach:\n\n- **`src/`**: Development-focused structure with build tools, distribution system, and core components\n- **Legacy directories**: Preserved in their original locations for backward compatibility\n- **`workspace/`**: Development workspace with tools and runtime management\n\nThis reorganization enables efficient development workflows while maintaining full backward compatibility with existing deployments.\n\n## New Structure vs Legacy\n\n### New Development Structure (`/src/`)\n\n```\nsrc/\n├── config/                      # System configuration\n├── control-center/              # Control center application\n├── control-center-ui/           # Web UI for control center\n├── core/                        # Core system libraries\n├── docs/                        # Documentation (new)\n├── extensions/                  # Extension framework\n├── generators/                  # Code generation tools\n├── schemas/                     # Nickel configuration schemas (migrated from kcl/)\n├── orchestrator/               # Hybrid Rust/Nushell orchestrator\n├── platform/                   # Platform-specific code\n├── provisioning/               # Main provisioning\n├── templates/                   # Template files\n├── tools/                      # Build and development tools\n└── utils/                      # Utility scripts\n```\n\n### Legacy Structure (Preserved)\n\n```\nrepo-cnz/\n├── cluster/                     # Cluster configurations (preserved)\n├── core/                        # Core system (preserved)\n├── generate/                    # Generation scripts (preserved)\n├── schemas/                     # Nickel schemas (migrated from kcl/)\n├── klab/                       # Development lab (preserved)\n├── nushell-plugins/            # Plugin development (preserved)\n├── providers/                  # Cloud providers (preserved)\n├── taskservs/                  # Task services (preserved)\n└── templates/                  # Template files (preserved)\n```\n\n### Development Workspace (`/workspace/`)\n\n```\nworkspace/\n├── config/                     # Development configuration\n├── extensions/                 # Extension development\n├── infra/                      # Development infrastructure\n├── lib/                        # Workspace libraries\n├── runtime/                    # Runtime data\n└── tools/                      # Workspace management tools\n```\n\n## Core Directories\n\n### `/src/core/` - Core Development Libraries\n\n**Purpose**: Development-focused core libraries and entry points\n\n**Key Files**:\n\n- `nulib/provisioning` - Main CLI entry point (symlinks to legacy location)\n- `nulib/lib_provisioning/` - Core provisioning libraries\n- `nulib/workflows/` - Workflow management (orchestrator integration)\n\n**Relationship to Legacy**: Preserves original `core/` functionality while adding development enhancements\n\n### `/src/tools/` - Build and Development Tools\n\n**Purpose**: Complete build system for the provisioning project\n\n**Key Components**:\n\n```\ntools/\n├── build/                      # Build tools\n│   ├── compile-platform.nu     # Platform-specific compilation\n│   ├── bundle-core.nu          # Core library bundling\n│   ├── validate-nickel.nu      # Nickel schema validation\n│   ├── clean-build.nu          # Build cleanup\n│   └── test-distribution.nu    # Distribution testing\n├── distribution/               # Distribution tools\n│   ├── generate-distribution.nu # Main distribution generator\n│   ├── prepare-platform-dist.nu # Platform-specific distribution\n│   ├── prepare-core-dist.nu    # Core distribution\n│   ├── create-installer.nu     # Installer creation\n│   └── generate-docs.nu        # Documentation generation\n├── package/                    # Packaging tools\n│   ├── package-binaries.nu     # Binary packaging\n│   ├── build-containers.nu     # Container image building\n│   ├── create-tarball.nu       # Archive creation\n│   └── validate-package.nu     # Package validation\n├── release/                    # Release management\n│   ├── create-release.nu       # Release creation\n│   ├── upload-artifacts.nu     # Artifact upload\n│   ├── rollback-release.nu     # Release rollback\n│   ├── notify-users.nu         # Release notifications\n│   └── update-registry.nu      # Package registry updates\n└── Makefile                    # Main build system (40+ targets)\n```\n\n### `/src/orchestrator/` - Hybrid Orchestrator\n\n**Purpose**: Rust/Nushell hybrid orchestrator for solving deep call stack limitations\n\n**Key Components**:\n\n- `src/` - Rust orchestrator implementation\n- `scripts/` - Orchestrator management scripts\n- `data/` - File-based task queue and persistence\n\n**Integration**: Provides REST API and workflow management while preserving all Nushell business logic\n\n### `/src/provisioning/` - Enhanced Provisioning\n\n**Purpose**: Enhanced version of the main provisioning with additional features\n\n**Key Features**:\n\n- Batch workflow system (v3.1.0)\n- Provider-agnostic design\n- Configuration-driven architecture (v2.0.0)\n\n### `/workspace/` - Development Workspace\n\n**Purpose**: Complete development environment with tools and runtime management\n\n**Key Components**:\n\n- `tools/workspace.nu` - Unified workspace management interface\n- `lib/path-resolver.nu` - Smart path resolution system\n- `config/` - Environment-specific development configurations\n- `extensions/` - Extension development templates and examples\n- `infra/` - Development infrastructure examples\n- `runtime/` - Isolated runtime data per user\n\n## Development Workspace\n\n### Workspace Management\n\nThe workspace provides a sophisticated development environment:\n\n**Initialization**:\n\n```\ncd workspace/tools\nnu workspace.nu init --user-name developer --infra-name my-infra\n```\n\n**Health Monitoring**:\n\n```\nnu workspace.nu health --detailed --fix-issues\n```\n\n**Path Resolution**:\n\n```\nuse lib/path-resolver.nu\nlet config = (path-resolver resolve_config "user" --workspace-user "john")\n```\n\n### Extension Development\n\nThe workspace provides templates for developing:\n\n- **Providers**: Custom cloud provider implementations\n- **Task Services**: Infrastructure service components\n- **Clusters**: Complete deployment solutions\n\nTemplates are available in `workspace/extensions/{type}/template/`\n\n### Configuration Hierarchy\n\nThe workspace implements a sophisticated configuration cascade:\n\n1. Workspace user configuration (`workspace/config/{user}.toml`)\n2. Environment-specific defaults (`workspace/config/{env}-defaults.toml`)\n3. Workspace defaults (`workspace/config/dev-defaults.toml`)\n4. Core system defaults (`config.defaults.toml`)\n\n## File Naming Conventions\n\n### Nushell Files (`.nu`)\n\n- **Commands**: `kebab-case` - `create-server.nu`, `validate-config.nu`\n- **Modules**: `snake_case` - `lib_provisioning`, `path_resolver`\n- **Scripts**: `kebab-case` - `workspace-health.nu`, `runtime-manager.nu`\n\n### Configuration Files\n\n- **TOML**: `kebab-case.toml` - `config-defaults.toml`, `user-settings.toml`\n- **Environment**: `{env}-defaults.toml` - `dev-defaults.toml`, `prod-defaults.toml`\n- **Examples**: `*.toml.example` - `local-overrides.toml.example`\n\n### Nickel Files (`.ncl`)\n\n- **Schemas**: `kebab-case.ncl` - `server-config.ncl`, `workflow-schema.ncl`\n- **Configuration**: `manifest.toml` - Package metadata\n- **Structure**: Organized in `schemas/` directories per extension\n\n### Build and Distribution\n\n- **Scripts**: `kebab-case.nu` - `compile-platform.nu`, `generate-distribution.nu`\n- **Makefiles**: `Makefile` - Standard naming\n- **Archives**: `{project}-{version}-{platform}-{variant}.{ext}`\n\n## Navigation Guide\n\n### Finding Components\n\n**Core System Entry Points**:\n\n```\n# Main CLI (development version)\n/src/core/nulib/provisioning\n\n# Legacy CLI (production version)\n/core/nulib/provisioning\n\n# Workspace management\n/workspace/tools/workspace.nu\n```\n\n**Build System**:\n\n```\n# Main build system\ncd /src/tools && make help\n\n# Quick development build\nmake dev-build\n\n# Complete distribution\nmake all\n```\n\n**Configuration Files**:\n\n```\n# System defaults\n/config.defaults.toml\n\n# User configuration (workspace)\n/workspace/config/{user}.toml\n\n# Environment-specific\n/workspace/config/{env}-defaults.toml\n```\n\n**Extension Development**:\n\n```\n# Provider template\n/workspace/extensions/providers/template/\n\n# Task service template\n/workspace/extensions/taskservs/template/\n\n# Cluster template\n/workspace/extensions/clusters/template/\n```\n\n### Common Workflows\n\n**1. Development Setup**:\n\n```\n# Initialize workspace\ncd workspace/tools\nnu workspace.nu init --user-name $USER\n\n# Check health\nnu workspace.nu health --detailed\n```\n\n**2. Building Distribution**:\n\n```\n# Complete build\ncd src/tools\nmake all\n\n# Platform-specific build\nmake linux\nmake macos\nmake windows\n```\n\n**3. Extension Development**:\n\n```\n# Create new provider\ncp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider\n\n# Test extension\nnu workspace/extensions/providers/my-provider/nulib/provider.nu test\n```\n\n### Legacy Compatibility\n\n**Existing Commands Still Work**:\n\n```\n# All existing commands preserved\n./core/nulib/provisioning server create\n./core/nulib/provisioning taskserv install kubernetes\n./core/nulib/provisioning cluster create buildkit\n```\n\n**Configuration Migration**:\n\n- ENV variables still supported as fallbacks\n- New configuration system provides better defaults\n- Migration tools available in `src/tools/migration/`\n\n## Migration Path\n\n### For Users\n\n**No Changes Required**:\n\n- All existing commands continue to work\n- Configuration files remain compatible\n- Existing infrastructure deployments unaffected\n\n**Optional Enhancements**:\n\n- Migrate to new configuration system for better defaults\n- Use workspace for development environments\n- Leverage new build system for custom distributions\n\n### For Developers\n\n**Development Environment**:\n\n1. Initialize development workspace: `nu workspace/tools/workspace.nu init`\n2. Use new build system: `cd src/tools && make dev-build`\n3. Leverage extension templates for custom development\n\n**Build System**:\n\n1. Use new Makefile for comprehensive build management\n2. Leverage distribution tools for packaging\n3. Use release management for version control\n\n**Orchestrator Integration**:\n\n1. Start orchestrator for workflow management: `cd src/orchestrator && ./scripts/start-orchestrator.nu`\n2. Use workflow APIs for complex operations\n3. Leverage batch operations for efficiency\n\n### Migration Tools\n\n**Available Migration Scripts**:\n\n- `src/tools/migration/config-migration.nu` - Configuration migration\n- `src/tools/migration/workspace-setup.nu` - Workspace initialization\n- `src/tools/migration/path-resolver.nu` - Path resolution migration\n\n**Validation Tools**:\n\n- `src/tools/validation/system-health.nu` - System health validation\n- `src/tools/validation/compatibility-check.nu` - Compatibility verification\n- `src/tools/validation/migration-status.nu` - Migration status tracking\n\n## Architecture Benefits\n\n### Development Efficiency\n\n- **Build System**: Comprehensive 40+ target Makefile system\n- **Workspace Isolation**: Per-user development environments\n- **Extension Framework**: Template-based extension development\n\n### Production Reliability\n\n- **Backward Compatibility**: All existing functionality preserved\n- **Configuration Migration**: Gradual migration from ENV to config-driven\n- **Orchestrator Architecture**: Hybrid Rust/Nushell for performance and flexibility\n- **Workflow Management**: Batch operations with rollback capabilities\n\n### Maintenance Benefits\n\n- **Clean Separation**: Development tools separate from production code\n- **Organized Structure**: Logical grouping of related functionality\n- **Documentation**: Comprehensive documentation and examples\n- **Testing Framework**: Built-in testing and validation tools\n\nThis structure represents a significant evolution in the project's organization while maintaining complete backward compatibility and providing\npowerful new development capabilities.
