Project Structure Guide
This document provides a comprehensive overview of the provisioning project’s structure after the major reorganization, explaining both the new development-focused organization and the preserved existing functionality.
Table of Contents
- Overview
- New Structure vs Legacy
- Core Directories
- Development Workspace
- File Naming Conventions
- Navigation Guide
- Migration Path
Overview
The provisioning project has been restructured to support a dual-organization approach:
src/: Development-focused structure with build tools, distribution system, and core components- Legacy directories: Preserved in their original locations for backward compatibility
workspace/: Development workspace with tools and runtime management
This reorganization enables efficient development workflows while maintaining full backward compatibility with existing deployments.
New Structure vs Legacy
New Development Structure (/src/)
src/
├── config/ # System configuration
├── control-center/ # Control center application
├── control-center-ui/ # Web UI for control center
├── core/ # Core system libraries
├── docs/ # Documentation (new)
├── extensions/ # Extension framework
├── generators/ # Code generation tools
├── kcl/ # KCL configuration language files
├── orchestrator/ # Hybrid Rust/Nushell orchestrator
├── platform/ # Platform-specific code
├── provisioning/ # Main provisioning
├── templates/ # Template files
├── tools/ # Build and development tools
└── utils/ # Utility scripts
```plaintext
### Legacy Structure (Preserved)
```plaintext
repo-cnz/
├── cluster/ # Cluster configurations (preserved)
├── core/ # Core system (preserved)
├── generate/ # Generation scripts (preserved)
├── kcl/ # KCL files (preserved)
├── klab/ # Development lab (preserved)
├── nushell-plugins/ # Plugin development (preserved)
├── providers/ # Cloud providers (preserved)
├── taskservs/ # Task services (preserved)
└── templates/ # Template files (preserved)
```plaintext
### Development Workspace (`/workspace/`)
```plaintext
workspace/
├── config/ # Development configuration
├── extensions/ # Extension development
├── infra/ # Development infrastructure
├── lib/ # Workspace libraries
├── runtime/ # Runtime data
└── tools/ # Workspace management tools
```plaintext
## Core Directories
### `/src/core/` - Core Development Libraries
**Purpose**: Development-focused core libraries and entry points
**Key Files**:
- `nulib/provisioning` - Main CLI entry point (symlinks to legacy location)
- `nulib/lib_provisioning/` - Core provisioning libraries
- `nulib/workflows/` - Workflow management (orchestrator integration)
**Relationship to Legacy**: Preserves original `core/` functionality while adding development enhancements
### `/src/tools/` - Build and Development Tools
**Purpose**: Complete build system for the provisioning project
**Key Components**:
```plaintext
tools/
├── build/ # Build tools
│ ├── compile-platform.nu # Platform-specific compilation
│ ├── bundle-core.nu # Core library bundling
│ ├── validate-kcl.nu # KCL validation
│ ├── clean-build.nu # Build cleanup
│ └── test-distribution.nu # Distribution testing
├── distribution/ # Distribution tools
│ ├── generate-distribution.nu # Main distribution generator
│ ├── prepare-platform-dist.nu # Platform-specific distribution
│ ├── prepare-core-dist.nu # Core distribution
│ ├── create-installer.nu # Installer creation
│ └── generate-docs.nu # Documentation generation
├── package/ # Packaging tools
│ ├── package-binaries.nu # Binary packaging
│ ├── build-containers.nu # Container image building
│ ├── create-tarball.nu # Archive creation
│ └── validate-package.nu # Package validation
├── release/ # Release management
│ ├── create-release.nu # Release creation
│ ├── upload-artifacts.nu # Artifact upload
│ ├── rollback-release.nu # Release rollback
│ ├── notify-users.nu # Release notifications
│ └── update-registry.nu # Package registry updates
└── Makefile # Main build system (40+ targets)
```plaintext
### `/src/orchestrator/` - Hybrid Orchestrator
**Purpose**: Rust/Nushell hybrid orchestrator for solving deep call stack limitations
**Key Components**:
- `src/` - Rust orchestrator implementation
- `scripts/` - Orchestrator management scripts
- `data/` - File-based task queue and persistence
**Integration**: Provides REST API and workflow management while preserving all Nushell business logic
### `/src/provisioning/` - Enhanced Provisioning
**Purpose**: Enhanced version of the main provisioning with additional features
**Key Features**:
- Batch workflow system (v3.1.0)
- Provider-agnostic design
- Configuration-driven architecture (v2.0.0)
### `/workspace/` - Development Workspace
**Purpose**: Complete development environment with tools and runtime management
**Key Components**:
- `tools/workspace.nu` - Unified workspace management interface
- `lib/path-resolver.nu` - Smart path resolution system
- `config/` - Environment-specific development configurations
- `extensions/` - Extension development templates and examples
- `infra/` - Development infrastructure examples
- `runtime/` - Isolated runtime data per user
## Development Workspace
### Workspace Management
The workspace provides a sophisticated development environment:
**Initialization**:
```bash
cd workspace/tools
nu workspace.nu init --user-name developer --infra-name my-infra
```plaintext
**Health Monitoring**:
```bash
nu workspace.nu health --detailed --fix-issues
```plaintext
**Path Resolution**:
```nushell
use lib/path-resolver.nu
let config = (path-resolver resolve_config "user" --workspace-user "john")
```plaintext
### Extension Development
The workspace provides templates for developing:
- **Providers**: Custom cloud provider implementations
- **Task Services**: Infrastructure service components
- **Clusters**: Complete deployment solutions
Templates are available in `workspace/extensions/{type}/template/`
### Configuration Hierarchy
The workspace implements a sophisticated configuration cascade:
1. Workspace user configuration (`workspace/config/{user}.toml`)
2. Environment-specific defaults (`workspace/config/{env}-defaults.toml`)
3. Workspace defaults (`workspace/config/dev-defaults.toml`)
4. Core system defaults (`config.defaults.toml`)
## File Naming Conventions
### Nushell Files (`.nu`)
- **Commands**: `kebab-case` - `create-server.nu`, `validate-config.nu`
- **Modules**: `snake_case` - `lib_provisioning`, `path_resolver`
- **Scripts**: `kebab-case` - `workspace-health.nu`, `runtime-manager.nu`
### Configuration Files
- **TOML**: `kebab-case.toml` - `config-defaults.toml`, `user-settings.toml`
- **Environment**: `{env}-defaults.toml` - `dev-defaults.toml`, `prod-defaults.toml`
- **Examples**: `*.toml.example` - `local-overrides.toml.example`
### KCL Files (`.k`)
- **Schemas**: `PascalCase` types - `ServerConfig`, `WorkflowDefinition`
- **Files**: `kebab-case.k` - `server-config.k`, `workflow-schema.k`
- **Modules**: `kcl.mod` - Module definition files
### Build and Distribution
- **Scripts**: `kebab-case.nu` - `compile-platform.nu`, `generate-distribution.nu`
- **Makefiles**: `Makefile` - Standard naming
- **Archives**: `{project}-{version}-{platform}-{variant}.{ext}`
## Navigation Guide
### Finding Components
**Core System Entry Points**:
```bash
# Main CLI (development version)
/src/core/nulib/provisioning
# Legacy CLI (production version)
/core/nulib/provisioning
# Workspace management
/workspace/tools/workspace.nu
```plaintext
**Build System**:
```bash
# Main build system
cd /src/tools && make help
# Quick development build
make dev-build
# Complete distribution
make all
```plaintext
**Configuration Files**:
```bash
# System defaults
/config.defaults.toml
# User configuration (workspace)
/workspace/config/{user}.toml
# Environment-specific
/workspace/config/{env}-defaults.toml
```plaintext
**Extension Development**:
```bash
# Provider template
/workspace/extensions/providers/template/
# Task service template
/workspace/extensions/taskservs/template/
# Cluster template
/workspace/extensions/clusters/template/
```plaintext
### Common Workflows
**1. Development Setup**:
```bash
# Initialize workspace
cd workspace/tools
nu workspace.nu init --user-name $USER
# Check health
nu workspace.nu health --detailed
```plaintext
**2. Building Distribution**:
```bash
# Complete build
cd src/tools
make all
# Platform-specific build
make linux
make macos
make windows
```plaintext
**3. Extension Development**:
```bash
# Create new provider
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider
# Test extension
nu workspace/extensions/providers/my-provider/nulib/provider.nu test
```plaintext
### Legacy Compatibility
**Existing Commands Still Work**:
```bash
# All existing commands preserved
./core/nulib/provisioning server create
./core/nulib/provisioning taskserv install kubernetes
./core/nulib/provisioning cluster create buildkit
```plaintext
**Configuration Migration**:
- ENV variables still supported as fallbacks
- New configuration system provides better defaults
- Migration tools available in `src/tools/migration/`
## Migration Path
### For Users
**No Changes Required**:
- All existing commands continue to work
- Configuration files remain compatible
- Existing infrastructure deployments unaffected
**Optional Enhancements**:
- Migrate to new configuration system for better defaults
- Use workspace for development environments
- Leverage new build system for custom distributions
### For Developers
**Development Environment**:
1. Initialize development workspace: `nu workspace/tools/workspace.nu init`
2. Use new build system: `cd src/tools && make dev-build`
3. Leverage extension templates for custom development
**Build System**:
1. Use new Makefile for comprehensive build management
2. Leverage distribution tools for packaging
3. Use release management for version control
**Orchestrator Integration**:
1. Start orchestrator for workflow management: `cd src/orchestrator && ./scripts/start-orchestrator.nu`
2. Use workflow APIs for complex operations
3. Leverage batch operations for efficiency
### Migration Tools
**Available Migration Scripts**:
- `src/tools/migration/config-migration.nu` - Configuration migration
- `src/tools/migration/workspace-setup.nu` - Workspace initialization
- `src/tools/migration/path-resolver.nu` - Path resolution migration
**Validation Tools**:
- `src/tools/validation/system-health.nu` - System health validation
- `src/tools/validation/compatibility-check.nu` - Compatibility verification
- `src/tools/validation/migration-status.nu` - Migration status tracking
## Architecture Benefits
### Development Efficiency
- **Build System**: Comprehensive 40+ target Makefile system
- **Workspace Isolation**: Per-user development environments
- **Extension Framework**: Template-based extension development
### Production Reliability
- **Backward Compatibility**: All existing functionality preserved
- **Configuration Migration**: Gradual migration from ENV to config-driven
- **Orchestrator Architecture**: Hybrid Rust/Nushell for performance and flexibility
- **Workflow Management**: Batch operations with rollback capabilities
### Maintenance Benefits
- **Clean Separation**: Development tools separate from production code
- **Organized Structure**: Logical grouping of related functionality
- **Documentation**: Comprehensive documentation and examples
- **Testing Framework**: Built-in testing and validation tools
This structure represents a significant evolution in the project's organization while maintaining complete backward compatibility and providing powerful new development capabilities.