# Project Structure Guide

This document provides a comprehensive overview of the provisioning project's structure after the major reorganization, explaining both the new
development-focused organization and the preserved existing functionality.

## Table of Contents

1. [Overview](#overview)
2. [New Structure vs Legacy](#new-structure-vs-legacy)
3. [Core Directories](#core-directories)
4. [Development Workspace](#development-workspace)
5. [File Naming Conventions](#file-naming-conventions)
6. [Navigation Guide](#navigation-guide)
7. [Migration Path](#migration-path)

## Overview

The provisioning project has been restructured to support a dual-organization approach:

- **`src/`**: Development-focused structure with build tools, distribution system, and core components
- **Legacy directories**: Preserved in their original locations for backward compatibility
- **`workspace/`**: Development workspace with tools and runtime management

This reorganization enables efficient development workflows while maintaining full backward compatibility with existing deployments.

## New Structure vs Legacy

### New Development Structure (`/src/`)

```
src/
├── config/                      # System configuration
├── control-center/              # Control center application
├── control-center-ui/           # Web UI for control center
├── core/                        # Core system libraries
├── docs/                        # Documentation (new)
├── extensions/                  # Extension framework
├── generators/                  # Code generation tools
├── schemas/                     # Nickel configuration schemas (migrated from kcl/)
├── orchestrator/               # Hybrid Rust/Nushell orchestrator
├── platform/                   # Platform-specific code
├── provisioning/               # Main provisioning
├── templates/                   # Template files
├── tools/                      # Build and development tools
└── utils/                      # Utility scripts
```

### Legacy Structure (Preserved)

```
repo-cnz/
├── cluster/                     # Cluster configurations (preserved)
├── core/                        # Core system (preserved)
├── generate/                    # Generation scripts (preserved)
├── schemas/                     # Nickel schemas (migrated from kcl/)
├── klab/                       # Development lab (preserved)
├── nushell-plugins/            # Plugin development (preserved)
├── providers/                  # Cloud providers (preserved)
├── taskservs/                  # Task services (preserved)
└── templates/                  # Template files (preserved)
```

### Development Workspace (`/workspace/`)

```
workspace/
├── config/                     # Development configuration
├── extensions/                 # Extension development
├── infra/                      # Development infrastructure
├── lib/                        # Workspace libraries
├── runtime/                    # Runtime data
└── tools/                      # Workspace management tools
```

## Core Directories

### `/src/core/` - Core Development Libraries

**Purpose**: Development-focused core libraries and entry points

**Key Files**:

- `nulib/provisioning` - Main CLI entry point (symlinks to legacy location)
- `nulib/lib_provisioning/` - Core provisioning libraries
- `nulib/workflows/` - Workflow management (orchestrator integration)

**Relationship to Legacy**: Preserves original `core/` functionality while adding development enhancements

### `/src/tools/` - Build and Development Tools

**Purpose**: Complete build system for the provisioning project

**Key Components**:

```
tools/
├── build/                      # Build tools
│   ├── compile-platform.nu     # Platform-specific compilation
│   ├── bundle-core.nu          # Core library bundling
│   ├── validate-nickel.nu      # Nickel schema validation
│   ├── clean-build.nu          # Build cleanup
│   └── test-distribution.nu    # Distribution testing
├── distribution/               # Distribution tools
│   ├── generate-distribution.nu # Main distribution generator
│   ├── prepare-platform-dist.nu # Platform-specific distribution
│   ├── prepare-core-dist.nu    # Core distribution
│   ├── create-installer.nu     # Installer creation
│   └── generate-docs.nu        # Documentation generation
├── package/                    # Packaging tools
│   ├── package-binaries.nu     # Binary packaging
│   ├── build-containers.nu     # Container image building
│   ├── create-tarball.nu       # Archive creation
│   └── validate-package.nu     # Package validation
├── release/                    # Release management
│   ├── create-release.nu       # Release creation
│   ├── upload-artifacts.nu     # Artifact upload
│   ├── rollback-release.nu     # Release rollback
│   ├── notify-users.nu         # Release notifications
│   └── update-registry.nu      # Package registry updates
└── Makefile                    # Main build system (40+ targets)
```

### `/src/orchestrator/` - Hybrid Orchestrator

**Purpose**: Rust/Nushell hybrid orchestrator for solving deep call stack limitations

**Key Components**:

- `src/` - Rust orchestrator implementation
- `scripts/` - Orchestrator management scripts
- `data/` - File-based task queue and persistence

**Integration**: Provides REST API and workflow management while preserving all Nushell business logic

### `/src/provisioning/` - Enhanced Provisioning

**Purpose**: Enhanced version of the main provisioning with additional features

**Key Features**:

- Batch workflow system (v3.1.0)
- Provider-agnostic design
- Configuration-driven architecture (v2.0.0)

### `/workspace/` - Development Workspace

**Purpose**: Complete development environment with tools and runtime management

**Key Components**:

- `tools/workspace.nu` - Unified workspace management interface
- `lib/path-resolver.nu` - Smart path resolution system
- `config/` - Environment-specific development configurations
- `extensions/` - Extension development templates and examples
- `infra/` - Development infrastructure examples
- `runtime/` - Isolated runtime data per user

## Development Workspace

### Workspace Management

The workspace provides a sophisticated development environment:

**Initialization**:

```
cd workspace/tools
nu workspace.nu init --user-name developer --infra-name my-infra
```

**Health Monitoring**:

```
nu workspace.nu health --detailed --fix-issues
```

**Path Resolution**:

```
use lib/path-resolver.nu
let config = (path-resolver resolve_config "user" --workspace-user "john")
```

### Extension Development

The workspace provides templates for developing:

- **Providers**: Custom cloud provider implementations
- **Task Services**: Infrastructure service components
- **Clusters**: Complete deployment solutions

Templates are available in `workspace/extensions/{type}/template/`

### Configuration Hierarchy

The workspace implements a sophisticated configuration cascade:

1. Workspace user configuration (`workspace/config/{user}.toml`)
2. Environment-specific defaults (`workspace/config/{env}-defaults.toml`)
3. Workspace defaults (`workspace/config/dev-defaults.toml`)
4. Core system defaults (`config.defaults.toml`)

## File Naming Conventions

### Nushell Files (`.nu`)

- **Commands**: `kebab-case` - `create-server.nu`, `validate-config.nu`
- **Modules**: `snake_case` - `lib_provisioning`, `path_resolver`
- **Scripts**: `kebab-case` - `workspace-health.nu`, `runtime-manager.nu`

### Configuration Files

- **TOML**: `kebab-case.toml` - `config-defaults.toml`, `user-settings.toml`
- **Environment**: `{env}-defaults.toml` - `dev-defaults.toml`, `prod-defaults.toml`
- **Examples**: `*.toml.example` - `local-overrides.toml.example`

### Nickel Files (`.ncl`)

- **Schemas**: `kebab-case.ncl` - `server-config.ncl`, `workflow-schema.ncl`
- **Configuration**: `manifest.toml` - Package metadata
- **Structure**: Organized in `schemas/` directories per extension

### Build and Distribution

- **Scripts**: `kebab-case.nu` - `compile-platform.nu`, `generate-distribution.nu`
- **Makefiles**: `Makefile` - Standard naming
- **Archives**: `{project}-{version}-{platform}-{variant}.{ext}`

## Navigation Guide

### Finding Components

**Core System Entry Points**:

```
# Main CLI (development version)
/src/core/nulib/provisioning

# Legacy CLI (production version)
/core/nulib/provisioning

# Workspace management
/workspace/tools/workspace.nu
```

**Build System**:

```
# Main build system
cd /src/tools && make help

# Quick development build
make dev-build

# Complete distribution
make all
```

**Configuration Files**:

```
# System defaults
/config.defaults.toml

# User configuration (workspace)
/workspace/config/{user}.toml

# Environment-specific
/workspace/config/{env}-defaults.toml
```

**Extension Development**:

```
# Provider template
/workspace/extensions/providers/template/

# Task service template
/workspace/extensions/taskservs/template/

# Cluster template
/workspace/extensions/clusters/template/
```

### Common Workflows

**1. Development Setup**:

```
# Initialize workspace
cd workspace/tools
nu workspace.nu init --user-name $USER

# Check health
nu workspace.nu health --detailed
```

**2. Building Distribution**:

```
# Complete build
cd src/tools
make all

# Platform-specific build
make linux
make macos
make windows
```

**3. Extension Development**:

```
# Create new provider
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider

# Test extension
nu workspace/extensions/providers/my-provider/nulib/provider.nu test
```

### Legacy Compatibility

**Existing Commands Still Work**:

```
# All existing commands preserved
./core/nulib/provisioning server create
./core/nulib/provisioning taskserv install kubernetes
./core/nulib/provisioning cluster create buildkit
```

**Configuration Migration**:

- ENV variables still supported as fallbacks
- New configuration system provides better defaults
- Migration tools available in `src/tools/migration/`

## Migration Path

### For Users

**No Changes Required**:

- All existing commands continue to work
- Configuration files remain compatible
- Existing infrastructure deployments unaffected

**Optional Enhancements**:

- Migrate to new configuration system for better defaults
- Use workspace for development environments
- Leverage new build system for custom distributions

### For Developers

**Development Environment**:

1. Initialize development workspace: `nu workspace/tools/workspace.nu init`
2. Use new build system: `cd src/tools && make dev-build`
3. Leverage extension templates for custom development

**Build System**:

1. Use new Makefile for comprehensive build management
2. Leverage distribution tools for packaging
3. Use release management for version control

**Orchestrator Integration**:

1. Start orchestrator for workflow management: `cd src/orchestrator && ./scripts/start-orchestrator.nu`
2. Use workflow APIs for complex operations
3. Leverage batch operations for efficiency

### Migration Tools

**Available Migration Scripts**:

- `src/tools/migration/config-migration.nu` - Configuration migration
- `src/tools/migration/workspace-setup.nu` - Workspace initialization
- `src/tools/migration/path-resolver.nu` - Path resolution migration

**Validation Tools**:

- `src/tools/validation/system-health.nu` - System health validation
- `src/tools/validation/compatibility-check.nu` - Compatibility verification
- `src/tools/validation/migration-status.nu` - Migration status tracking

## Architecture Benefits

### Development Efficiency

- **Build System**: Comprehensive 40+ target Makefile system
- **Workspace Isolation**: Per-user development environments
- **Extension Framework**: Template-based extension development

### Production Reliability

- **Backward Compatibility**: All existing functionality preserved
- **Configuration Migration**: Gradual migration from ENV to config-driven
- **Orchestrator Architecture**: Hybrid Rust/Nushell for performance and flexibility
- **Workflow Management**: Batch operations with rollback capabilities

### Maintenance Benefits

- **Clean Separation**: Development tools separate from production code
- **Organized Structure**: Logical grouping of related functionality
- **Documentation**: Comprehensive documentation and examples
- **Testing Framework**: Built-in testing and validation tools

This structure represents a significant evolution in the project's organization while maintaining complete backward compatibility and providing
powerful new development capabilities.