## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Distribution Process Documentation

This document provides comprehensive documentation for the provisioning project’s distribution process, covering release workflows, package generation, multi-platform distribution, and rollback procedures.

## Table of Contents

1. Overview
2. Distribution Architecture
3. Release Process
4. Package Generation
5. Multi-Platform Distribution
6. Validation and Testing
7. Release Management
8. Rollback Procedures
9. CI/CD Integration
10. Troubleshooting

## Overview

The distribution system provides a comprehensive solution for creating, packaging, and distributing provisioning across multiple platforms with automated release management.

Key Features:

- Multi-Platform Support: Linux, macOS, Windows with multiple architectures
- Multiple Distribution Variants: Complete and minimal distributions
- Automated Release Pipeline: From development to production deployment
- Package Management: Binary packages, container images, and installers
- Validation Framework: Comprehensive testing and validation
- Rollback Capabilities: Safe rollback and recovery procedures

Location: `/src/tools/` Main Tool: `/src/tools/Makefile` and associated Nushell scripts

## Distribution Architecture

### Distribution Components

    Distribution Ecosystem
    ├── Core Components
    │   ├── Platform Binaries      # Rust-compiled binaries
    │   ├── Core Libraries         # Nushell libraries and CLI
    │   ├── Configuration System   # TOML configuration files
    │   └── Documentation         # User and API documentation
    ├── Platform Packages
    │   ├── Archives              # TAR.GZ and ZIP files
    │   ├── Installers            # Platform-specific installers
    │   └── Container Images      # Docker/OCI images
    ├── Distribution Variants
    │   ├── Complete              # Full-featured distribution
    │   └── Minimal               # Lightweight distribution
    └── Release Artifacts
        ├── Checksums             # SHA256/MD5 verification
        ├── Signatures            # Digital signatures
        └── Metadata              # Release information

### Build Pipeline

    Build Pipeline Flow
    ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
    │   Source Code   │ -> │   Build Stage   │ -> │  Package Stage  │
    │                 │    │                 │    │                 │
    │ - Rust code     │    │ - compile-      │    │ - create-       │
    │ - Nushell libs  │    │   platform      │    │   archives      │
    │ - Nickel schemas│    │ - bundle-core   │    │ - build-        │
    │ - Config files  │    │ - validate-nickel│   │   containers    │
    └─────────────────┘    └─────────────────┘    └─────────────────┘
                                    |
                                    v
    ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
    │ Release Stage   │ <- │ Validate Stage  │ <- │ Distribute Stage│
    │                 │    │                 │    │                 │
    │ - create-       │    │ - test-dist     │    │ - generate-     │
    │   release       │    │ - validate-     │    │   distribution  │
    │ - upload-       │    │   package       │    │ - create-       │
    │   artifacts     │    │ - integration   │    │   installers    │
    └─────────────────┘    └─────────────────┘    └─────────────────┘

### Distribution Variants

Complete Distribution:

- All Rust binaries (orchestrator, control-center, MCP server)
- Full Nushell library suite
- All providers, taskservs, and clusters
- Complete documentation and examples
- Development tools and templates

Minimal Distribution:

- Essential binaries only
- Core Nushell libraries
- Basic provider support
- Essential task services
- Minimal documentation

## Release Process

### Release Types

Release Classifications:

- Major Release (x.0.0): Breaking changes, new major features
- Minor Release (x.y.0): New features, backward compatible
- Patch Release (x.y.z): Bug fixes, security updates
- Pre-Release (x.y.z-alpha/beta/rc): Development/testing releases

### Step-by-Step Release Process

#### 1. Preparation Phase

Pre-Release Checklist:

    # Update dependencies and security
    cargo update
    cargo audit

    # Run comprehensive tests
    make ci-test

    # Update documentation
    make docs

    # Validate all configurations
    make validate-all

Version Planning:

    # Check current version
    git describe --tags --always

    # Plan next version
    make status | grep Version

    # Validate version bump
    nu src/tools/release/create-release.nu --dry-run --version 2.1.0

#### 2. Build Phase

Complete Build:

    # Clean build environment
    make clean

    # Build all platforms and variants
    make all

    # Validate build output
    make test-dist

Build with Specific Parameters:

    # Build for specific platforms
    make all PLATFORMS=linux-amd64,macos-amd64 VARIANTS=complete

    # Build with custom version
    make all VERSION=2.1.0-rc1

    # Parallel build for speed
    make all PARALLEL=true

#### 3. Package Generation

Create Distribution Packages:

    # Generate complete distributions
    make dist-generate

    # Create binary packages
    make package-binaries

    # Build container images
    make package-containers

    # Create installers
    make create-installers

Package Validation:

    # Validate packages
    make test-dist

    # Check package contents
    nu src/tools/package/validate-package.nu packages/

    # Test installation
    make install
    make uninstall

#### 4. Release Creation

Automated Release:

    # Create complete release
    make release VERSION=2.1.0

    # Create draft release for review
    make release-draft VERSION=2.1.0

    # Manual release creation
    nu src/tools/release/create-release.nu \
        --version 2.1.0 \
        --generate-changelog \
        --push-tag \
        --auto-upload

Release Options:

- `--pre-release`: Mark as pre-release
- `--draft`: Create draft release
- `--generate-changelog`: Auto-generate changelog from commits
- `--push-tag`: Push git tag to remote
- `--auto-upload`: Upload assets automatically

#### 5. Distribution and Notification

Upload Artifacts:

    # Upload to GitHub Releases
    make upload-artifacts

    # Update package registries
    make update-registry

    # Send notifications
    make notify-release

Registry Updates:

    # Update Homebrew formula
    nu src/tools/release/update-registry.nu \
        --registries homebrew \
        --version 2.1.0 \
        --auto-commit

    # Custom registry updates
    nu src/tools/release/update-registry.nu \
        --registries custom \
        --registry-url https://packages.company.com \
        --credentials-file ~/.registry-creds

### Release Automation

Complete Automated Release:

    # Full release pipeline
    make cd-deploy VERSION=2.1.0

    # Equivalent manual steps:
    make clean
    make all VERSION=2.1.0
    make create-archives
    make create-installers
    make release VERSION=2.1.0
    make upload-artifacts
    make update-registry
    make notify-release

## Package Generation

### Binary Packages

Package Types:

- Standalone Archives: TAR.GZ and ZIP with all dependencies
- Platform Packages: DEB, RPM, MSI, PKG with system integration
- Portable Packages: Single-directory distributions
- Source Packages: Source code with build instructions

Create Binary Packages:

    # Standard binary packages
    make package-binaries

    # Custom package creation
    nu src/tools/package/package-binaries.nu \
        --source-dir dist/platform \
        --output-dir packages/binaries \
        --platforms linux-amd64,macos-amd64 \
        --format archive \
        --compress \
        --strip \
        --checksum

Package Features:

- Binary Stripping: Removes debug symbols for smaller size
- Compression: GZIP, LZMA, and Brotli compression
- Checksums: SHA256 and MD5 verification
- Signatures: GPG and code signing support

### Container Images

Container Build Process:

    # Build container images
    make package-containers

    # Advanced container build
    nu src/tools/package/build-containers.nu \
        --dist-dir dist \
        --tag-prefix provisioning \
        --version 2.1.0 \
        --platforms "linux/amd64,linux/arm64" \
        --optimize-size \
        --security-scan \
        --multi-stage

Container Features:

- Multi-Stage Builds: Minimal runtime images
- Security Scanning: Vulnerability detection
- Multi-Platform: AMD64, ARM64 support
- Layer Optimization: Efficient layer caching
- Runtime Configuration: Environment-based configuration

Container Registry Support:

- Docker Hub
- GitHub Container Registry
- Amazon ECR
- Google Container Registry
- Azure Container Registry
- Private registries

### Installers

Installer Types:

- Shell Script Installer: Universal Unix/Linux installer
- Package Installers: DEB, RPM, MSI, PKG
- Container Installer: Docker/Podman setup
- Source Installer: Build-from-source installer

Create Installers:

    # Generate all installer types
    make create-installers

    # Custom installer creation
    nu src/tools/distribution/create-installer.nu \
        dist/provisioning-2.1.0-linux-amd64-complete \
        --output-dir packages/installers \
        --installer-types shell,package \
        --platforms linux,macos \
        --include-services \
        --create-uninstaller \
        --validate-installer

Installer Features:

- System Integration: Systemd/Launchd service files
- Path Configuration: Automatic PATH updates
- User/System Install: Support for both user and system-wide installation
- Uninstaller: Clean removal capability
- Dependency Management: Automatic dependency resolution
- Configuration Setup: Initial configuration creation

## Multi-Platform Distribution

### Supported Platforms

Primary Platforms:

- Linux AMD64 (x86_64-unknown-linux-gnu)
- Linux ARM64 (aarch64-unknown-linux-gnu)
- macOS AMD64 (x86_64-apple-darwin)
- macOS ARM64 (aarch64-apple-darwin)
- Windows AMD64 (x86_64-pc-windows-gnu)
- FreeBSD AMD64 (x86_64-unknown-freebsd)

Platform-Specific Features:

- Linux: SystemD integration, package manager support
- macOS: LaunchAgent services, Homebrew packages
- Windows: Windows Service support, MSI installers
- FreeBSD: RC scripts, pkg packages

### Cross-Platform Build

Cross-Compilation Setup:

    # Install cross-compilation targets
    rustup target add aarch64-unknown-linux-gnu
    rustup target add x86_64-apple-darwin
    rustup target add aarch64-apple-darwin
    rustup target add x86_64-pc-windows-gnu

    # Install cross-compilation tools
    cargo install cross

Platform-Specific Builds:

    # Build for specific platform
    make build-platform RUST_TARGET=aarch64-apple-darwin

    # Build for multiple platforms
    make build-cross PLATFORMS=linux-amd64,macos-arm64,windows-amd64

    # Platform-specific distributions
    make linux
    make macos
    make windows

### Distribution Matrix

Generated Distributions:

    Distribution Matrix:
    provisioning-{version}-{platform}-{variant}.{format}

    Examples:
    - provisioning-2.1.0-linux-amd64-complete.tar.gz
    - provisioning-2.1.0-macos-arm64-minimal.tar.gz
    - provisioning-2.1.0-windows-amd64-complete.zip
    - provisioning-2.1.0-freebsd-amd64-minimal.tar.xz

Platform Considerations:

- File Permissions: Executable permissions on Unix systems
- Path Separators: Platform-specific path handling
- Service Integration: Platform-specific service management
- Package Formats: TAR.GZ for Unix, ZIP for Windows
- Line Endings: CRLF for Windows, LF for Unix

## Validation and Testing

### Distribution Validation

Validation Pipeline:

    # Complete validation
    make test-dist

    # Custom validation
    nu src/tools/build/test-distribution.nu \
        --dist-dir dist \
        --test-types basic,integration,complete \
        --platform linux \
        --cleanup \
        --verbose

Validation Types:

- Basic: Installation test, CLI help, version check
- Integration: Server creation, configuration validation
- Complete: Full workflow testing including cluster operations

### Testing Framework

Test Categories:

- Unit Tests: Component-specific testing
- Integration Tests: Cross-component testing
- End-to-End Tests: Complete workflow testing
- Performance Tests: Load and performance validation
- Security Tests: Security scanning and validation

Test Execution:

    # Run all tests
    make ci-test

    # Specific test types
    nu src/tools/build/test-distribution.nu --test-types basic
    nu src/tools/build/test-distribution.nu --test-types integration
    nu src/tools/build/test-distribution.nu --test-types complete

### Package Validation

Package Integrity:

    # Validate package structure
    nu src/tools/package/validate-package.nu dist/

    # Check checksums
    sha256sum -c packages/checksums.sha256

    # Verify signatures
    gpg --verify packages/provisioning-2.1.0.tar.gz.sig

Installation Testing:

    # Test installation process
    ./packages/installers/install-provisioning-2.1.0.sh --dry-run

    # Test uninstallation
    ./packages/installers/uninstall-provisioning.sh --dry-run

    # Container testing
    docker run --rm provisioning:2.1.0 provisioning --version

## Release Management

### Release Workflow

GitHub Release Integration:

    # Create GitHub release
    nu src/tools/release/create-release.nu \
        --version 2.1.0 \
        --asset-dir packages \
        --generate-changelog \
        --push-tag \
        --auto-upload

Release Features:

- Automated Changelog: Generated from git commit history
- Asset Management: Automatic upload of all distribution artifacts
- Tag Management: Semantic version tagging
- Release Notes: Formatted release notes with change summaries

### Versioning Strategy

Semantic Versioning:

- MAJOR.MINOR.PATCH format (for example, 2.1.0)
- Pre-release suffixes (for example, 2.1.0-alpha.1, 2.1.0-rc.2)
- Build metadata (for example, 2.1.0+20250925.abcdef)

Version Detection:

    # Auto-detect next version
    nu src/tools/release/create-release.nu --release-type minor

    # Manual version specification
    nu src/tools/release/create-release.nu --version 2.1.0

    # Pre-release versioning
    nu src/tools/release/create-release.nu --version 2.1.0-rc.1 --pre-release

### Artifact Management

Artifact Types:

- Source Archives: Complete source code distributions
- Binary Archives: Compiled binary distributions
- Container Images: OCI-compliant container images
- Installers: Platform-specific installation packages
- Documentation: Generated documentation packages

Upload and Distribution:

    # Upload to GitHub Releases
    make upload-artifacts

    # Upload to container registries
    docker push provisioning:2.1.0

    # Update package repositories
    make update-registry

## Rollback Procedures

### Rollback Scenarios

Common Rollback Triggers:

- Critical bugs discovered post-release
- Security vulnerabilities identified
- Performance regression
- Compatibility issues
- Infrastructure failures

### Rollback Process

Automated Rollback:

    # Rollback latest release
    nu src/tools/release/rollback-release.nu --version 2.1.0

    # Rollback with specific target
    nu src/tools/release/rollback-release.nu \
        --from-version 2.1.0 \
        --to-version 2.0.5 \
        --update-registries \
        --notify-users

Manual Rollback Steps:

    # 1. Identify target version
    git tag -l | grep -v 2.1.0 | tail -5

    # 2. Create rollback release
    nu src/tools/release/create-release.nu \
        --version 2.0.6 \
        --rollback-from 2.1.0 \
        --urgent

    # 3. Update package managers
    nu src/tools/release/update-registry.nu \
        --version 2.0.6 \
        --rollback-notice "Critical fix for 2.1.0 issues"

    # 4. Notify users
    nu src/tools/release/notify-users.nu \
        --channels slack,discord,email \
        --message-type rollback \
        --urgent

### Rollback Safety

Pre-Rollback Validation:

- Validate target version integrity
- Check compatibility matrix
- Verify rollback procedure testing
- Confirm communication plan

Rollback Testing:

    # Test rollback in staging
    nu src/tools/release/rollback-release.nu \
        --version 2.1.0 \
        --target-version 2.0.5 \
        --dry-run \
        --staging-environment

    # Validate rollback success
    make test-dist DIST_VERSION=2.0.5

### Emergency Procedures

Critical Security Rollback:

    # Emergency rollback (bypasses normal procedures)
    nu src/tools/release/rollback-release.nu \
        --version 2.1.0 \
        --emergency \
        --security-issue \
        --immediate-notify

Infrastructure Failure Recovery:

    # Failover to backup infrastructure
    nu src/tools/release/rollback-release.nu \
        --infrastructure-failover \
        --backup-registry \
        --mirror-sync

## CI/CD Integration

### GitHub Actions Integration

Build Workflow (`.github/workflows/build.yml`):

    name: Build and Distribute
    on:
      push:
        branches: [main]
      pull_request:
        branches: [main]

    jobs:
      build:
        runs-on: ubuntu-latest
        strategy:
          matrix:
            platform: [linux, macos, windows]
        steps:
          - uses: actions/checkout@v4

          - name: Setup Nushell
            uses: hustcer/setup-nu@v3.5

          - name: Setup Rust
            uses: actions-rs/toolchain@v1
            with:
              toolchain: stable

          - name: CI Build
            run: |
              cd src/tools
              make ci-build

          - name: Upload Build Artifacts
            uses: actions/upload-artifact@v4
            with:
              name: build-${{ matrix.platform }}
              path: src/dist/

Release Workflow (`.github/workflows/release.yml`):

    name: Release
    on:
      push:
        tags: ['v*']

    jobs:
      release:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4

          - name: Build Release
            run: |
              cd src/tools
              make ci-release VERSION=${{ github.ref_name }}

          - name: Create Release
            run: |
              cd src/tools
              make release VERSION=${{ github.ref_name }}

          - name: Update Registries
            run: |
              cd src/tools
              make update-registry VERSION=${{ github.ref_name }}

### GitLab CI Integration

GitLab CI Configuration (`.gitlab-ci.yml`):

    stages:
      - build
      - package
      - test
      - release

    build:
      stage: build
      script:
        - cd src/tools
        - make ci-build
      artifacts:
        paths:
          - src/dist/
        expire_in: 1 hour

    package:
      stage: package
      script:
        - cd src/tools
        - make package-all
      artifacts:
        paths:
          - src/packages/
        expire_in: 1 day

    release:
      stage: release
      script:
        - cd src/tools
        - make cd-deploy VERSION=${CI_COMMIT_TAG}
      only:
        - tags

### Jenkins Integration

Jenkinsfile:

    pipeline {
        agent any

        stages {
            stage('Build') {
                steps {
                    dir('src/tools') {
                        sh 'make ci-build'
                    }
                }
            }

            stage('Package') {
                steps {
                    dir('src/tools') {
                        sh 'make package-all'
                    }
                }
            }

            stage('Release') {
                when {
                    tag '*'
                }
                steps {
                    dir('src/tools') {
                        sh "make cd-deploy VERSION=${env.TAG_NAME}"
                    }
                }
            }
        }
    }

## Troubleshooting

### Common Issues

#### Build Failures

Rust Compilation Errors:

    # Solution: Clean and rebuild
    make clean
    cargo clean
    make build-platform

    # Check Rust toolchain
    rustup show
    rustup update

Cross-Compilation Issues:

    # Solution: Install missing targets
    rustup target list --installed
    rustup target add x86_64-apple-darwin

    # Use cross for problematic targets
    cargo install cross
    make build-platform CROSS=true

#### Package Generation Issues

Missing Dependencies:

    # Solution: Install build tools
    sudo apt-get install build-essential
    brew install gnu-tar

    # Check tool availability
    make info

Permission Errors:

    # Solution: Fix permissions
    chmod +x src/tools/build/*.nu
    chmod +x src/tools/distribution/*.nu
    chmod +x src/tools/package/*.nu

#### Distribution Validation Failures

Package Integrity Issues:

    # Solution: Regenerate packages
    make clean-dist
    make package-all

    # Verify manually
    sha256sum packages/*.tar.gz

Installation Test Failures:

    # Solution: Test in clean environment
    docker run --rm -v $(pwd):/work ubuntu:latest /work/packages/installers/install.sh

    # Debug installation
    ./packages/installers/install.sh --dry-run --verbose

### Release Issues

#### Upload Failures

Network Issues:

    # Solution: Retry with backoff
    nu src/tools/release/upload-artifacts.nu \
        --retry-count 5 \
        --backoff-delay 30

    # Manual upload
    gh release upload v2.1.0 packages/*.tar.gz

Authentication Failures:

    # Solution: Refresh tokens
    gh auth refresh
    docker login ghcr.io

    # Check credentials
    gh auth status
    docker system info

#### Registry Update Issues

Homebrew Formula Issues:

    # Solution: Manual PR creation
    git clone https://github.com/Homebrew/homebrew-core
    cd homebrew-core
    # Edit formula
    git add Formula/provisioning.rb
    git commit -m "provisioning 2.1.0"

### Debug and Monitoring

Debug Mode:

    # Enable debug logging
    export PROVISIONING_DEBUG=true
    export RUST_LOG=debug

    # Run with verbose output
    make all VERBOSE=true

    # Debug specific components
    nu src/tools/distribution/generate-distribution.nu \
        --verbose \
        --dry-run

Monitoring Build Progress:

    # Monitor build logs
    tail -f src/tools/build.log

    # Check build status
    make status

    # Resource monitoring
    top
    df -h

This distribution process provides a robust, automated pipeline for creating, validating, and distributing provisioning across multiple platforms while maintaining high quality and reliability standards.