Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Build System Documentation

This document provides comprehensive documentation for the provisioning project’s build system, including the complete Makefile reference with 40+ targets, build tools, compilation instructions, and troubleshooting.

Table of Contents

  1. Overview
  2. Quick Start
  3. Makefile Reference
  4. Build Tools
  5. Cross-Platform Compilation
  6. Dependency Management
  7. Troubleshooting
  8. CI/CD Integration

Overview

The build system is a comprehensive, Makefile-based solution that orchestrates:

  • Rust compilation: Platform binaries (orchestrator, control-center, etc.)
  • Nushell bundling: Core libraries and CLI tools
  • KCL validation: Configuration schema validation
  • Distribution generation: Multi-platform packages
  • Release management: Automated release pipelines
  • Documentation generation: API and user documentation

Location: /src/tools/ Main entry point: /src/tools/Makefile

Quick Start

# Navigate to build system
cd src/tools

# View all available targets
make help

# Complete build and package
make all

# Development build (quick)
make dev-build

# Build for specific platform
make linux
make macos
make windows

# Clean everything
make clean

# Check build system status
make status

Makefile Reference

Build Configuration

Variables:

# Project metadata
PROJECT_NAME := provisioning
VERSION := $(git describe --tags --always --dirty)
BUILD_TIME := $(date -u +"%Y-%m-%dT%H:%M:%SZ")

# Build configuration
RUST_TARGET := x86_64-unknown-linux-gnu
BUILD_MODE := release
PLATFORMS := linux-amd64,macos-amd64,windows-amd64
VARIANTS := complete,minimal

# Flags
VERBOSE := false
DRY_RUN := false
PARALLEL := true

Build Targets

Primary Build Targets

make all - Complete build, package, and test

  • Runs: clean build-all package-all test-dist
  • Use for: Production releases, complete validation

make build-all - Build all components

  • Runs: build-platform build-core validate-kcl
  • Use for: Complete system compilation

make build-platform - Build platform binaries for all targets

make build-platform
# Equivalent to:
nu tools/build/compile-platform.nu \
    --target x86_64-unknown-linux-gnu \
    --release \
    --output-dir dist/platform \
    --verbose=false

make build-core - Bundle core Nushell libraries

make build-core
# Equivalent to:
nu tools/build/bundle-core.nu \
    --output-dir dist/core \
    --config-dir dist/config \
    --validate \
    --exclude-dev

make validate-kcl - Validate and compile KCL schemas

make validate-kcl
# Equivalent to:
nu tools/build/validate-kcl.nu \
    --output-dir dist/kcl \
    --format-code \
    --check-dependencies

make build-cross - Cross-compile for multiple platforms

  • Builds for all platforms in PLATFORMS variable
  • Parallel execution support
  • Failure handling for each platform

Package Targets

make package-all - Create all distribution packages

  • Runs: dist-generate package-binaries package-containers

make dist-generate - Generate complete distributions

make dist-generate
# Advanced usage:
make dist-generate PLATFORMS=linux-amd64,macos-amd64 VARIANTS=complete

make package-binaries - Package binaries for distribution

  • Creates platform-specific archives
  • Strips debug symbols
  • Generates checksums

make package-containers - Build container images

  • Multi-platform container builds
  • Optimized layers and caching
  • Version tagging

make create-archives - Create distribution archives

  • TAR and ZIP formats
  • Platform-specific and universal archives
  • Compression and checksums

make create-installers - Create installation packages

  • Shell script installers
  • Platform-specific packages (DEB, RPM, MSI)
  • Uninstaller creation

Release Targets

make release - Create a complete release (requires VERSION)

make release VERSION=2.1.0

Features:

  • Automated changelog generation
  • Git tag creation and push
  • Artifact upload
  • Comprehensive validation

make release-draft - Create a draft release

  • Create without publishing
  • Review artifacts before release
  • Manual approval workflow

make upload-artifacts - Upload release artifacts

  • GitHub Releases
  • Container registries
  • Package repositories
  • Verification and validation

make notify-release - Send release notifications

  • Slack notifications
  • Discord announcements
  • Email notifications
  • Custom webhook support

make update-registry - Update package manager registries

  • Homebrew formula updates
  • APT repository updates
  • Custom registry support

Development and Testing Targets

make dev-build - Quick development build

make dev-build
# Fast build with minimal validation

make test-build - Test build system

  • Validates build process
  • Runs with test configuration
  • Comprehensive logging

make test-dist - Test generated distributions

  • Validates distribution integrity
  • Tests installation process
  • Platform compatibility checks

make validate-all - Validate all components

  • KCL schema validation
  • Package validation
  • Configuration validation

make benchmark - Run build benchmarks

  • Times build process
  • Performance analysis
  • Resource usage monitoring

Documentation Targets

make docs - Generate documentation

make docs
# Generates API docs, user guides, and examples

make docs-serve - Generate and serve documentation locally

  • Starts local HTTP server on port 8000
  • Live documentation browsing
  • Development documentation workflow

Utility Targets

make clean - Clean all build artifacts

make clean
# Removes all build, distribution, and package directories

make clean-dist - Clean only distribution artifacts

  • Preserves build cache
  • Removes distribution packages
  • Faster cleanup option

make install - Install the built system locally

  • Requires distribution to be built
  • Installs to system directories
  • Creates uninstaller

make uninstall - Uninstall the system

  • Removes system installation
  • Cleans configuration
  • Removes service files

make status - Show build system status

make status
# Output:
# Build System Status
# ===================
# Project: provisioning
# Version: v2.1.0-5-g1234567
# Git Commit: 1234567890abcdef
# Build Time: 2025-09-25T14:30:22Z
#
# Directories:
#   Source: /Users/user/repo-cnz/src
#   Tools: /Users/user/repo-cnz/src/tools
#   Build: /Users/user/repo-cnz/src/target
#   Distribution: /Users/user/repo-cnz/src/dist
#   Packages: /Users/user/repo-cnz/src/packages

make info - Show detailed system information

  • OS and architecture details
  • Tool versions (Nushell, Rust, Docker, Git)
  • Environment information
  • Build prerequisites

CI/CD Integration Targets

make ci-build - CI build pipeline

  • Complete validation build
  • Suitable for automated CI systems
  • Comprehensive testing

make ci-test - CI test pipeline

  • Validation and testing only
  • Fast feedback for pull requests
  • Quality assurance

make ci-release - CI release pipeline

  • Build and packaging for releases
  • Artifact preparation
  • Release candidate creation

make cd-deploy - CD deployment pipeline

  • Complete release and deployment
  • Artifact upload and distribution
  • User notifications

Platform-Specific Targets

make linux - Build for Linux only

make linux
# Sets PLATFORMS=linux-amd64

make macos - Build for macOS only

make macos
# Sets PLATFORMS=macos-amd64

make windows - Build for Windows only

make windows
# Sets PLATFORMS=windows-amd64

Debugging Targets

make debug - Build with debug information

make debug
# Sets BUILD_MODE=debug VERBOSE=true

make debug-info - Show debug information

  • Make variables and environment
  • Build system diagnostics
  • Troubleshooting information

Build Tools

Core Build Scripts

All build tools are implemented as Nushell scripts with comprehensive parameter validation and error handling.

/src/tools/build/compile-platform.nu

Purpose: Compiles all Rust components for distribution

Components Compiled:

  • orchestrator → provisioning-orchestrator binary
  • control-center → control-center binary
  • control-center-ui → Web UI assets
  • mcp-server-rust → MCP integration binary

Usage:

nu compile-platform.nu [options]

Options:
  --target STRING          Target platform (default: x86_64-unknown-linux-gnu)
  --release                Build in release mode
  --features STRING        Comma-separated features to enable
  --output-dir STRING      Output directory (default: dist/platform)
  --verbose                Enable verbose logging
  --clean                  Clean before building

Example:

nu compile-platform.nu \
    --target x86_64-apple-darwin \
    --release \
    --features "surrealdb,telemetry" \
    --output-dir dist/macos \
    --verbose

/src/tools/build/bundle-core.nu

Purpose: Bundles Nushell core libraries and CLI for distribution

Components Bundled:

  • Nushell provisioning CLI wrapper
  • Core Nushell libraries (lib_provisioning)
  • Configuration system
  • Template system
  • Extensions and plugins

Usage:

nu bundle-core.nu [options]

Options:
  --output-dir STRING      Output directory (default: dist/core)
  --config-dir STRING      Configuration directory (default: dist/config)
  --validate               Validate Nushell syntax
  --compress               Compress bundle with gzip
  --exclude-dev            Exclude development files (default: true)
  --verbose                Enable verbose logging

Validation Features:

  • Syntax validation of all Nushell files
  • Import dependency checking
  • Function signature validation
  • Test execution (if tests present)

/src/tools/build/validate-kcl.nu

Purpose: Validates and compiles KCL schemas

Validation Process:

  1. Syntax validation of all .k files
  2. Schema dependency checking
  3. Type constraint validation
  4. Example validation against schemas
  5. Documentation generation

Usage:

nu validate-kcl.nu [options]

Options:
  --output-dir STRING      Output directory (default: dist/kcl)
  --format-code            Format KCL code during validation
  --check-dependencies     Validate schema dependencies
  --verbose                Enable verbose logging

/src/tools/build/test-distribution.nu

Purpose: Tests generated distributions for correctness

Test Types:

  • Basic: Installation test, CLI help, version check
  • Integration: Server creation, configuration validation
  • Complete: Full workflow testing including cluster operations

Usage:

nu test-distribution.nu [options]

Options:
  --dist-dir STRING        Distribution directory (default: dist)
  --test-types STRING      Test types: basic,integration,complete
  --platform STRING        Target platform for testing
  --cleanup                Remove test files after completion
  --verbose                Enable verbose logging

/src/tools/build/clean-build.nu

Purpose: Intelligent build artifact cleanup

Cleanup Scopes:

  • all: Complete cleanup (build, dist, packages, cache)
  • dist: Distribution artifacts only
  • cache: Build cache and temporary files
  • old: Files older than specified age

Usage:

nu clean-build.nu [options]

Options:
  --scope STRING           Cleanup scope: all,dist,cache,old
  --age DURATION          Age threshold for 'old' scope (default: 7d)
  --force                  Force cleanup without confirmation
  --dry-run               Show what would be cleaned without doing it
  --verbose               Enable verbose logging

Distribution Tools

/src/tools/distribution/generate-distribution.nu

Purpose: Main distribution generator orchestrating the complete process

Generation Process:

  1. Platform binary compilation
  2. Core library bundling
  3. KCL schema validation and packaging
  4. Configuration system preparation
  5. Documentation generation
  6. Archive creation and compression
  7. Installer generation
  8. Validation and testing

Usage:

nu generate-distribution.nu [command] [options]

Commands:
  <default>                Generate complete distribution
  quick                    Quick development distribution
  status                   Show generation status

Options:
  --version STRING         Version to build (default: auto-detect)
  --platforms STRING       Comma-separated platforms
  --variants STRING        Variants: complete,minimal
  --output-dir STRING      Output directory (default: dist)
  --compress               Enable compression
  --generate-docs          Generate documentation
  --parallel-builds        Enable parallel builds
  --validate-output        Validate generated output
  --verbose                Enable verbose logging

Advanced Examples:

# Complete multi-platform release
nu generate-distribution.nu \
    --version 2.1.0 \
    --platforms linux-amd64,macos-amd64,windows-amd64 \
    --variants complete,minimal \
    --compress \
    --generate-docs \
    --parallel-builds \
    --validate-output

# Quick development build
nu generate-distribution.nu quick \
    --platform linux \
    --variant minimal

# Status check
nu generate-distribution.nu status

/src/tools/distribution/create-installer.nu

Purpose: Creates platform-specific installers

Installer Types:

  • shell: Shell script installer (cross-platform)
  • package: Platform packages (DEB, RPM, MSI, PKG)
  • container: Container image with provisioning
  • source: Source distribution with build instructions

Usage:

nu create-installer.nu DISTRIBUTION_DIR [options]

Options:
  --output-dir STRING      Installer output directory
  --installer-types STRING Installer types: shell,package,container,source
  --platforms STRING       Target platforms
  --include-services       Include systemd/launchd service files
  --create-uninstaller     Generate uninstaller
  --validate-installer     Test installer functionality
  --verbose                Enable verbose logging

Package Tools

/src/tools/package/package-binaries.nu

Purpose: Packages compiled binaries for distribution

Package Formats:

  • archive: TAR.GZ and ZIP archives
  • standalone: Single binary with embedded resources
  • installer: Platform-specific installer packages

Features:

  • Binary stripping for size reduction
  • Compression optimization
  • Checksum generation (SHA256, MD5)
  • Digital signing (if configured)

/src/tools/package/build-containers.nu

Purpose: Builds optimized container images

Container Features:

  • Multi-stage builds for minimal image size
  • Security scanning integration
  • Multi-platform image generation
  • Layer caching optimization
  • Runtime environment configuration

Release Tools

/src/tools/release/create-release.nu

Purpose: Automated release creation and management

Release Process:

  1. Version validation and tagging
  2. Changelog generation from git history
  3. Asset building and validation
  4. Release creation (GitHub, GitLab, etc.)
  5. Asset upload and verification
  6. Release announcement preparation

Usage:

nu create-release.nu [options]

Options:
  --version STRING         Release version (required)
  --asset-dir STRING       Directory containing release assets
  --draft                  Create draft release
  --prerelease             Mark as pre-release
  --generate-changelog     Auto-generate changelog
  --push-tag               Push git tag
  --auto-upload            Upload assets automatically
  --verbose                Enable verbose logging

Cross-Platform Compilation

Supported Platforms

Primary Platforms:

  • linux-amd64 (x86_64-unknown-linux-gnu)
  • macos-amd64 (x86_64-apple-darwin)
  • windows-amd64 (x86_64-pc-windows-gnu)

Additional Platforms:

  • linux-arm64 (aarch64-unknown-linux-gnu)
  • macos-arm64 (aarch64-apple-darwin)
  • freebsd-amd64 (x86_64-unknown-freebsd)

Cross-Compilation Setup

Install Rust Targets:

# Install additional targets
rustup target add x86_64-apple-darwin
rustup target add x86_64-pc-windows-gnu
rustup target add aarch64-unknown-linux-gnu
rustup target add aarch64-apple-darwin

Platform-Specific Dependencies:

macOS Cross-Compilation:

# Install osxcross toolchain
brew install FiloSottile/musl-cross/musl-cross
brew install mingw-w64

Windows Cross-Compilation:

# Install Windows dependencies
brew install mingw-w64
# or on Linux:
sudo apt-get install gcc-mingw-w64

Cross-Compilation Usage

Single Platform:

# Build for macOS from Linux
make build-platform RUST_TARGET=x86_64-apple-darwin

# Build for Windows
make build-platform RUST_TARGET=x86_64-pc-windows-gnu

Multiple Platforms:

# Build for all configured platforms
make build-cross

# Specify platforms
make build-cross PLATFORMS=linux-amd64,macos-amd64,windows-amd64

Platform-Specific Targets:

# Quick platform builds
make linux      # Linux AMD64
make macos      # macOS AMD64
make windows    # Windows AMD64

Dependency Management

Build Dependencies

Required Tools:

  • Nushell 0.107.1+: Core shell and scripting
  • Rust 1.70+: Platform binary compilation
  • Cargo: Rust package management
  • KCL 0.11.2+: Configuration language
  • Git: Version control and tagging

Optional Tools:

  • Docker: Container image building
  • Cross: Simplified cross-compilation
  • SOPS: Secrets management
  • Age: Encryption for secrets

Dependency Validation

Check Dependencies:

make info
# Shows versions of all required tools

# Output example:
# Tool Versions:
#   Nushell: 0.107.1
#   Rust: rustc 1.75.0
#   Docker: Docker version 24.0.6
#   Git: git version 2.42.0

Install Missing Dependencies:

# Install Nushell
cargo install nu

# Install KCL
cargo install kcl-cli

# Install Cross (for cross-compilation)
cargo install cross

Dependency Caching

Rust Dependencies:

  • Cargo cache: ~/.cargo/registry
  • Target cache: target/ directory
  • Cross-compilation cache: ~/.cache/cross

Build Cache Management:

# Clean Cargo cache
cargo clean

# Clean cross-compilation cache
cross clean

# Clean all caches
make clean SCOPE=cache

Troubleshooting

Common Build Issues

Rust Compilation Errors

Error: linker 'cc' not found

# Solution: Install build essentials
sudo apt-get install build-essential  # Linux
xcode-select --install                 # macOS

Error: target not found

# Solution: Install target
rustup target add x86_64-unknown-linux-gnu

Error: Cross-compilation linking errors

# Solution: Use cross instead of cargo
cargo install cross
make build-platform CROSS=true

Nushell Script Errors

Error: command not found

# Solution: Ensure Nushell is in PATH
which nu
export PATH="$HOME/.cargo/bin:$PATH"

Error: Permission denied

# Solution: Make scripts executable
chmod +x src/tools/build/*.nu

Error: Module not found

# Solution: Check working directory
cd src/tools
nu build/compile-platform.nu --help

KCL Validation Errors

Error: kcl command not found

# Solution: Install KCL
cargo install kcl-cli
# or
brew install kcl

Error: Schema validation failed

# Solution: Check KCL syntax
kcl fmt kcl/
kcl check kcl/

Build Performance Issues

Slow Compilation

Optimizations:

# Enable parallel builds
make build-all PARALLEL=true

# Use faster linker
export RUSTFLAGS="-C link-arg=-fuse-ld=lld"

# Increase build jobs
export CARGO_BUILD_JOBS=8

Cargo Configuration (~/.cargo/config.toml):

[build]
jobs = 8

[target.x86_64-unknown-linux-gnu]
linker = "lld"

Memory Issues

Solutions:

# Reduce parallel jobs
export CARGO_BUILD_JOBS=2

# Use debug build for development
make dev-build BUILD_MODE=debug

# Clean up between builds
make clean-dist

Distribution Issues

Missing Assets

Validation:

# Test distribution
make test-dist

# Detailed validation
nu src/tools/package/validate-package.nu dist/

Size Optimization

Optimizations:

# Strip binaries
make package-binaries STRIP=true

# Enable compression
make dist-generate COMPRESS=true

# Use minimal variant
make dist-generate VARIANTS=minimal

Debug Mode

Enable Debug Logging:

# Set environment
export PROVISIONING_DEBUG=true
export RUST_LOG=debug

# Run with debug
make debug

# Verbose make output
make build-all VERBOSE=true

Debug Information:

# Show debug information
make debug-info

# Build system status
make status

# Tool information
make info

CI/CD Integration

GitHub Actions

Example Workflow (.github/workflows/build.yml):

name: Build and Test
on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Nushell
        uses: hustcer/setup-nu@v3.5

      - name: Setup Rust
        uses: actions-rs/toolchain@v1
        with:
          toolchain: stable

      - name: CI Build
        run: |
          cd src/tools
          make ci-build

      - name: Upload Artifacts
        uses: actions/upload-artifact@v4
        with:
          name: build-artifacts
          path: src/dist/

Release Automation

Release Workflow:

name: Release
on:
  push:
    tags: ['v*']

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build Release
        run: |
          cd src/tools
          make ci-release VERSION=${{ github.ref_name }}

      - name: Create Release
        run: |
          cd src/tools
          make release VERSION=${{ github.ref_name }}

Local CI Testing

Test CI Pipeline Locally:

# Run CI build pipeline
make ci-build

# Run CI test pipeline
make ci-test

# Full CI/CD pipeline
make ci-release

This build system provides a comprehensive, maintainable foundation for the provisioning project’s development lifecycle, from local development to production releases.