## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Build System Documentation

This document provides comprehensive documentation for the provisioning project’s build system, including the complete Makefile reference with 40+ targets, build tools, compilation instructions, and troubleshooting.

## Table of Contents

1. Overview
2. Quick Start
3. Makefile Reference
4. Build Tools
5. Cross-Platform Compilation
6. Dependency Management
7. Troubleshooting
8. CI/CD Integration

## Overview

The build system is a comprehensive, Makefile-based solution that orchestrates:

- Rust compilation: Platform binaries (orchestrator, control-center, etc.)
- Nushell bundling: Core libraries and CLI tools
- Nickel validation: Configuration schema validation
- Distribution generation: Multi-platform packages
- Release management: Automated release pipelines
- Documentation generation: API and user documentation

Location: `/src/tools/` Main entry point: `/src/tools/Makefile`

## Quick Start

    # Navigate to build system
    cd src/tools

    # View all available targets
    make help

    # Complete build and package
    make all

    # Development build (quick)
    make dev-build

    # Build for specific platform
    make linux
    make macos
    make windows

    # Clean everything
    make clean

    # Check build system status
    make status

## Makefile Reference

### Build Configuration

Variables:

    # Project metadata
    PROJECT_NAME := provisioning
    VERSION := $(git describe --tags --always --dirty)
    BUILD_TIME := $(date -u +"%Y-%m-%dT%H:%M:%SZ")

    # Build configuration
    RUST_TARGET := x86_64-unknown-linux-gnu
    BUILD_MODE := release
    PLATFORMS := linux-amd64,macos-amd64,windows-amd64
    VARIANTS := complete,minimal

    # Flags
    VERBOSE := false
    DRY_RUN := false
    PARALLEL := true

### Build Targets

#### Primary Build Targets

`make all` - Complete build, package, and test

- Runs: `clean build-all package-all test-dist`
- Use for: Production releases, complete validation

`make build-all` - Build all components

- Runs: `build-platform build-core validate-nickel`
- Use for: Complete system compilation

`make build-platform` - Build platform binaries for all targets

    make build-platform
    # Equivalent to:
    nu tools/build/compile-platform.nu \
        --target x86_64-unknown-linux-gnu \
        --release \
        --output-dir dist/platform \
        --verbose=false

`make build-core` - Bundle core Nushell libraries

    make build-core
    # Equivalent to:
    nu tools/build/bundle-core.nu \
        --output-dir dist/core \
        --config-dir dist/config \
        --validate \
        --exclude-dev

`make validate-nickel` - Validate and compile Nickel schemas

    make validate-nickel
    # Equivalent to:
    nu tools/build/validate-nickel.nu \
        --output-dir dist/schemas \
        --format-code \
        --check-dependencies

`make build-cross` - Cross-compile for multiple platforms

- Builds for all platforms in `PLATFORMS` variable
- Parallel execution support
- Failure handling for each platform

#### Package Targets

`make package-all` - Create all distribution packages

- Runs: `dist-generate package-binaries package-containers`

`make dist-generate` - Generate complete distributions

    make dist-generate
    # Advanced usage:
    make dist-generate PLATFORMS=linux-amd64,macos-amd64 VARIANTS=complete

`make package-binaries` - Package binaries for distribution

- Creates platform-specific archives
- Strips debug symbols
- Generates checksums

`make package-containers` - Build container images

- Multi-platform container builds
- Optimized layers and caching
- Version tagging

`make create-archives` - Create distribution archives

- TAR and ZIP formats
- Platform-specific and universal archives
- Compression and checksums

`make create-installers` - Create installation packages

- Shell script installers
- Platform-specific packages (DEB, RPM, MSI)
- Uninstaller creation

#### Release Targets

`make release` - Create a complete release (requires VERSION)

    make release VERSION=2.1.0

Features:

- Automated changelog generation
- Git tag creation and push
- Artifact upload
- Comprehensive validation

`make release-draft` - Create a draft release

- Create without publishing
- Review artifacts before release
- Manual approval workflow

`make upload-artifacts` - Upload release artifacts

- GitHub Releases
- Container registries
- Package repositories
- Verification and validation

`make notify-release` - Send release notifications

- Slack notifications
- Discord announcements
- Email notifications
- Custom webhook support

`make update-registry` - Update package manager registries

- Homebrew formula updates
- APT repository updates
- Custom registry support

#### Development and Testing Targets

`make dev-build` - Quick development build

    make dev-build
    # Fast build with minimal validation

`make test-build` - Test build system

- Validates build process
- Runs with test configuration
- Comprehensive logging

`make test-dist` - Test generated distributions

- Validates distribution integrity
- Tests installation process
- Platform compatibility checks

`make validate-all` - Validate all components

- Nickel schema validation
- Package validation
- Configuration validation

`make benchmark` - Run build benchmarks

- Times build process
- Performance analysis
- Resource usage monitoring

#### Documentation Targets

`make docs` - Generate documentation

    make docs
    # Generates API docs, user guides, and examples

`make docs-serve` - Generate and serve documentation locally

- Starts local HTTP server on port 8000
- Live documentation browsing
- Development documentation workflow

#### Utility Targets

`make clean` - Clean all build artifacts

    make clean
    # Removes all build, distribution, and package directories

`make clean-dist` - Clean only distribution artifacts

- Preserves build cache
- Removes distribution packages
- Faster cleanup option

`make install` - Install the built system locally

- Requires distribution to be built
- Installs to system directories
- Creates uninstaller

`make uninstall` - Uninstall the system

- Removes system installation
- Cleans configuration
- Removes service files

`make status` - Show build system status

    make status
    # Output:
    # Build System Status
    # ===================
    # Project: provisioning
    # Version: v2.1.0-5-g1234567
    # Git Commit: 1234567890abcdef
    # Build Time: 2025-09-25T14:30:22Z
    #
    # Directories:
    #   Source: /Users/user/repo-cnz/src
    #   Tools: /Users/user/repo-cnz/src/tools
    #   Build: /Users/user/repo-cnz/src/target
    #   Distribution: /Users/user/repo-cnz/src/dist
    #   Packages: /Users/user/repo-cnz/src/packages

`make info` - Show detailed system information

- OS and architecture details
- Tool versions (Nushell, Rust, Docker, Git)
- Environment information
- Build prerequisites

#### CI/CD Integration Targets

`make ci-build` - CI build pipeline

- Complete validation build
- Suitable for automated CI systems
- Comprehensive testing

`make ci-test` - CI test pipeline

- Validation and testing only
- Fast feedback for pull requests
- Quality assurance

`make ci-release` - CI release pipeline

- Build and packaging for releases
- Artifact preparation
- Release candidate creation

`make cd-deploy` - CD deployment pipeline

- Complete release and deployment
- Artifact upload and distribution
- User notifications

#### Platform-Specific Targets

`make linux` - Build for Linux only

    make linux
    # Sets PLATFORMS=linux-amd64

`make macos` - Build for macOS only

    make macos
    # Sets PLATFORMS=macos-amd64

`make windows` - Build for Windows only

    make windows
    # Sets PLATFORMS=windows-amd64

#### Debugging Targets

`make debug` - Build with debug information

    make debug
    # Sets BUILD_MODE=debug VERBOSE=true

`make debug-info` - Show debug information

- Make variables and environment
- Build system diagnostics
- Troubleshooting information

## Build Tools

### Core Build Scripts

All build tools are implemented as Nushell scripts with comprehensive parameter validation and error handling.

#### `/src/tools/build/compile-platform.nu`

Purpose: Compiles all Rust components for distribution

Components Compiled:

- `orchestrator` → `provisioning-orchestrator` binary
- `control-center` → `control-center` binary
- `control-center-ui` → Web UI assets
- `mcp-server-rust` → MCP integration binary

Usage:

    nu compile-platform.nu [options]

    Options:
      --target STRING          Target platform (default: x86_64-unknown-linux-gnu)
      --release                Build in release mode
      --features STRING        Comma-separated features to enable
      --output-dir STRING      Output directory (default: dist/platform)
      --verbose                Enable verbose logging
      --clean                  Clean before building

Example:

    nu compile-platform.nu \
        --target x86_64-apple-darwin \
        --release \
        --features "surrealdb,telemetry" \
        --output-dir dist/macos \
        --verbose

#### `/src/tools/build/bundle-core.nu`

Purpose: Bundles Nushell core libraries and CLI for distribution

Components Bundled:

- Nushell provisioning CLI wrapper
- Core Nushell libraries (`lib_provisioning`)
- Configuration system
- Template system
- Extensions and plugins

Usage:

    nu bundle-core.nu [options]

    Options:
      --output-dir STRING      Output directory (default: dist/core)
      --config-dir STRING      Configuration directory (default: dist/config)
      --validate               Validate Nushell syntax
      --compress               Compress bundle with gzip
      --exclude-dev            Exclude development files (default: true)
      --verbose                Enable verbose logging

Validation Features:

- Syntax validation of all Nushell files
- Import dependency checking
- Function signature validation
- Test execution (if tests present)

#### `/src/tools/build/validate-nickel.nu`

Purpose: Validates and compiles Nickel schemas

Validation Process:

1. Syntax validation of all `.ncl` files
2. Schema dependency checking
3. Type constraint validation
4. Example validation against schemas
5. Documentation generation

Usage:

    nu validate-nickel.nu [options]

    Options:
      --output-dir STRING      Output directory (default: dist/schemas)
      --format-code            Format Nickel code during validation
      --check-dependencies     Validate schema dependencies
      --verbose                Enable verbose logging

#### `/src/tools/build/test-distribution.nu`

Purpose: Tests generated distributions for correctness

Test Types:

- Basic: Installation test, CLI help, version check
- Integration: Server creation, configuration validation
- Complete: Full workflow testing including cluster operations

Usage:

    nu test-distribution.nu [options]

    Options:
      --dist-dir STRING        Distribution directory (default: dist)
      --test-types STRING      Test types: basic,integration,complete
      --platform STRING        Target platform for testing
      --cleanup                Remove test files after completion
      --verbose                Enable verbose logging

#### `/src/tools/build/clean-build.nu`

Purpose: Intelligent build artifact cleanup

Cleanup Scopes:

- all: Complete cleanup (build, dist, packages, cache)
- dist: Distribution artifacts only
- cache: Build cache and temporary files
- old: Files older than specified age

Usage:

    nu clean-build.nu [options]

    Options:
      --scope STRING           Cleanup scope: all,dist,cache,old
      --age DURATION          Age threshold for 'old' scope (default: 7d)
      --force                  Force cleanup without confirmation
      --dry-run               Show what would be cleaned without doing it
      --verbose               Enable verbose logging

### Distribution Tools

#### `/src/tools/distribution/generate-distribution.nu`

Purpose: Main distribution generator orchestrating the complete process

Generation Process:

1. Platform binary compilation
2. Core library bundling
3. Nickel schema validation and packaging
4. Configuration system preparation
5. Documentation generation
6. Archive creation and compression
7. Installer generation
8. Validation and testing

Usage:

    nu generate-distribution.nu [command] [options]

    Commands:
      <default>                Generate complete distribution
      quick                    Quick development distribution
      status                   Show generation status

    Options:
      --version STRING         Version to build (default: auto-detect)
      --platforms STRING       Comma-separated platforms
      --variants STRING        Variants: complete,minimal
      --output-dir STRING      Output directory (default: dist)
      --compress               Enable compression
      --generate-docs          Generate documentation
      --parallel-builds        Enable parallel builds
      --validate-output        Validate generated output
      --verbose                Enable verbose logging

Advanced Examples:

    # Complete multi-platform release
    nu generate-distribution.nu \
        --version 2.1.0 \
        --platforms linux-amd64,macos-amd64,windows-amd64 \
        --variants complete,minimal \
        --compress \
        --generate-docs \
        --parallel-builds \
        --validate-output

    # Quick development build
    nu generate-distribution.nu quick \
        --platform linux \
        --variant minimal

    # Status check
    nu generate-distribution.nu status

#### `/src/tools/distribution/create-installer.nu`

Purpose: Creates platform-specific installers

Installer Types:

- shell: Shell script installer (cross-platform)
- package: Platform packages (DEB, RPM, MSI, PKG)
- container: Container image with provisioning
- source: Source distribution with build instructions

Usage:

    nu create-installer.nu DISTRIBUTION_DIR [options]

    Options:
      --output-dir STRING      Installer output directory
      --installer-types STRING Installer types: shell,package,container,source
      --platforms STRING       Target platforms
      --include-services       Include systemd/launchd service files
      --create-uninstaller     Generate uninstaller
      --validate-installer     Test installer functionality
      --verbose                Enable verbose logging

### Package Tools

#### `/src/tools/package/package-binaries.nu`

Purpose: Packages compiled binaries for distribution

Package Formats:

- archive: TAR.GZ and ZIP archives
- standalone: Single binary with embedded resources
- installer: Platform-specific installer packages

Features:

- Binary stripping for size reduction
- Compression optimization
- Checksum generation (SHA256, MD5)
- Digital signing (if configured)

#### `/src/tools/package/build-containers.nu`

Purpose: Builds optimized container images

Container Features:

- Multi-stage builds for minimal image size
- Security scanning integration
- Multi-platform image generation
- Layer caching optimization
- Runtime environment configuration

### Release Tools

#### `/src/tools/release/create-release.nu`

Purpose: Automated release creation and management

Release Process:

1. Version validation and tagging
2. Changelog generation from git history
3. Asset building and validation
4. Release creation (GitHub, GitLab, etc.)
5. Asset upload and verification
6. Release announcement preparation

Usage:

    nu create-release.nu [options]

    Options:
      --version STRING         Release version (required)
      --asset-dir STRING       Directory containing release assets
      --draft                  Create draft release
      --prerelease             Mark as pre-release
      --generate-changelog     Auto-generate changelog
      --push-tag               Push git tag
      --auto-upload            Upload assets automatically
      --verbose                Enable verbose logging

## Cross-Platform Compilation

### Supported Platforms

Primary Platforms:

- `linux-amd64` (x86_64-unknown-linux-gnu)
- `macos-amd64` (x86_64-apple-darwin)
- `windows-amd64` (x86_64-pc-windows-gnu)

Additional Platforms:

- `linux-arm64` (aarch64-unknown-linux-gnu)
- `macos-arm64` (aarch64-apple-darwin)
- `freebsd-amd64` (x86_64-unknown-freebsd)

### Cross-Compilation Setup

Install Rust Targets:

    # Install additional targets
    rustup target add x86_64-apple-darwin
    rustup target add x86_64-pc-windows-gnu
    rustup target add aarch64-unknown-linux-gnu
    rustup target add aarch64-apple-darwin

Platform-Specific Dependencies:

macOS Cross-Compilation:

    # Install osxcross toolchain
    brew install FiloSottile/musl-cross/musl-cross
    brew install mingw-w64

Windows Cross-Compilation:

    # Install Windows dependencies
    brew install mingw-w64
    # or on Linux:
    sudo apt-get install gcc-mingw-w64

### Cross-Compilation Usage

Single Platform:

    # Build for macOS from Linux
    make build-platform RUST_TARGET=x86_64-apple-darwin

    # Build for Windows
    make build-platform RUST_TARGET=x86_64-pc-windows-gnu

Multiple Platforms:

    # Build for all configured platforms
    make build-cross

    # Specify platforms
    make build-cross PLATFORMS=linux-amd64,macos-amd64,windows-amd64

Platform-Specific Targets:

    # Quick platform builds
    make linux      # Linux AMD64
    make macos      # macOS AMD64
    make windows    # Windows AMD64

## Dependency Management

### Build Dependencies

Required Tools:

- Nushell 0.107.1+: Core shell and scripting
- Rust 1.70+: Platform binary compilation
- Cargo: Rust package management
- KCL 0.11.2+: Configuration language
- Git: Version control and tagging

Optional Tools:

- Docker: Container image building
- Cross: Simplified cross-compilation
- SOPS: Secrets management
- Age: Encryption for secrets

### Dependency Validation

Check Dependencies:

    make info
    # Shows versions of all required tools

    # Output example:
    # Tool Versions:
    #   Nushell: 0.107.1
    #   Rust: rustc 1.75.0
    #   Docker: Docker version 24.0.6
    #   Git: git version 2.42.0

Install Missing Dependencies:

    # Install Nushell
    cargo install nu

    # Install Nickel
    cargo install nickel

    # Install Cross (for cross-compilation)
    cargo install cross

### Dependency Caching

Rust Dependencies:

- Cargo cache: `~/.cargo/registry`
- Target cache: `target/` directory
- Cross-compilation cache: `~/.cache/cross`

Build Cache Management:

    # Clean Cargo cache
    cargo clean

    # Clean cross-compilation cache
    cross clean

    # Clean all caches
    make clean SCOPE=cache

## Troubleshooting

### Common Build Issues

#### Rust Compilation Errors

Error: `linker 'cc' not found`

    # Solution: Install build essentials
    sudo apt-get install build-essential  # Linux
    xcode-select --install                 # macOS

Error: `target not found`

    # Solution: Install target
    rustup target add x86_64-unknown-linux-gnu

Error: Cross-compilation linking errors

    # Solution: Use cross instead of cargo
    cargo install cross
    make build-platform CROSS=true

#### Nushell Script Errors

Error: `command not found`

    # Solution: Ensure Nushell is in PATH
    which nu
    export PATH="$HOME/.cargo/bin:$PATH"

Error: Permission denied

    # Solution: Make scripts executable
    chmod +x src/tools/build/*.nu

Error: Module not found

    # Solution: Check working directory
    cd src/tools
    nu build/compile-platform.nu --help

#### Nickel Validation Errors

Error: `nickel command not found`

    # Solution: Install Nickel
    cargo install nickel
    # or
    brew install nickel

Error: Schema validation failed

    # Solution: Check Nickel syntax
    nickel fmt schemas/
    nickel check schemas/

### Build Performance Issues

#### Slow Compilation

Optimizations:

    # Enable parallel builds
    make build-all PARALLEL=true

    # Use faster linker
    export RUSTFLAGS="-C link-arg=-fuse-ld=lld"

    # Increase build jobs
    export CARGO_BUILD_JOBS=8

Cargo Configuration (`~/.cargo/config.toml`):

    [build]
    jobs = 8

    [target.x86_64-unknown-linux-gnu]
    linker = "lld"

#### Memory Issues

Solutions:

    # Reduce parallel jobs
    export CARGO_BUILD_JOBS=2

    # Use debug build for development
    make dev-build BUILD_MODE=debug

    # Clean up between builds
    make clean-dist

### Distribution Issues

#### Missing Assets

Validation:

    # Test distribution
    make test-dist

    # Detailed validation
    nu src/tools/package/validate-package.nu dist/

#### Size Optimization

Optimizations:

    # Strip binaries
    make package-binaries STRIP=true

    # Enable compression
    make dist-generate COMPRESS=true

    # Use minimal variant
    make dist-generate VARIANTS=minimal

### Debug Mode

Enable Debug Logging:

    # Set environment
    export PROVISIONING_DEBUG=true
    export RUST_LOG=debug

    # Run with debug
    make debug

    # Verbose make output
    make build-all VERBOSE=true

Debug Information:

    # Show debug information
    make debug-info

    # Build system status
    make status

    # Tool information
    make info

## CI/CD Integration

### GitHub Actions

Example Workflow (`.github/workflows/build.yml`):

    name: Build and Test
    on: [push, pull_request]

    jobs:
      build:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4

          - name: Setup Nushell
            uses: hustcer/setup-nu@v3.5

          - name: Setup Rust
            uses: actions-rs/toolchain@v1
            with:
              toolchain: stable

          - name: CI Build
            run: |
              cd src/tools
              make ci-build

          - name: Upload Artifacts
            uses: actions/upload-artifact@v4
            with:
              name: build-artifacts
              path: src/dist/

### Release Automation

Release Workflow:

    name: Release
    on:
      push:
        tags: ['v*']

    jobs:
      release:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4

          - name: Build Release
            run: |
              cd src/tools
              make ci-release VERSION=${{ github.ref_name }}

          - name: Create Release
            run: |
              cd src/tools
              make release VERSION=${{ github.ref_name }}

### Local CI Testing

Test CI Pipeline Locally:

    # Run CI build pipeline
    make ci-build

    # Run CI test pipeline
    make ci-test

    # Full CI/CD pipeline
    make ci-release

This build system provides a comprehensive, maintainable foundation for the provisioning project’s development lifecycle, from local development to production releases.