Building & Distribution Guide

Complete guide for building, packaging, and releasing TypeDialog.

Overview

Two main workflows:

Building - Compile binaries for your platform
Distribution - Package with configs and installers for release

Critical Dependencies

TypeDialog requires several critical dependencies for building and running. These include both internal dependencies (git submodules or external projects) and external tools.

Internal Dependencies

prov-ecosystem

What: Provisioning ecosystem library for infrastructure generation (used by typedialog-prov-gen)

Location: Expected as sibling directory or git submodule

Installation:

# Option 1: Clone as sibling directory (recommended)
cd /path/to/projects
git clone https://github.com/your-org/prov-ecosystem.git
git clone https://github.com/your-org/typedialog.git

# Option 2: Git submodule
cd typedialog
git submodule add https://github.com/your-org/prov-ecosystem.git deps/prov-ecosystem
git submodule update --init --recursive
```text

**Path Configuration:**

If `prov-ecosystem` is in a non-standard location, update `Cargo.toml`:

```toml
[dependencies]
prov-ecosystem = { path = "../prov-ecosystem" }  # Adjust path as needed
```text

Or use environment variable:

```bash
export PROV_ECOSYSTEM_PATH=/custom/path/to/prov-ecosystem
```text

**Verify:**

```bash
# Check if prov-ecosystem is found
cargo metadata --format-version=1 | grep prov-ecosystem
```text

#### secretumvault

**What:** Unified encryption vault library (used by encryption features)

**Location:** Expected as sibling directory or git submodule

**Installation:**

```bash
# Option 1: Clone as sibling directory (recommended)
cd /path/to/projects
git clone https://github.com/your-org/secretumvault.git
git clone https://github.com/your-org/typedialog.git

# Option 2: Git submodule
cd typedialog
git submodule add https://github.com/your-org/secretumvault.git deps/secretumvault
git submodule update --init --recursive
```text

**Path Configuration:**

If `secretumvault` is in a non-standard location, update `Cargo.toml`:

```toml
[dependencies]
secretumvault = { path = "../secretumvault" }  # Adjust path as needed
```text

Or use environment variable:

```bash
export SECRETUMVAULT_PATH=/custom/path/to/secretumvault
```text

**Verify:**

```bash
# Check if secretumvault is found
cargo metadata --format-version=1 | grep secretumvault
```text

### External Dependencies

#### just (Command Runner)

**What:** Command orchestration tool for running build recipes

**Required:** Yes (for all build workflows)

**Installation:**

```bash
# macOS (Homebrew)
brew install just

# Linux (cargo)
cargo install just

# Linux (from source)
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin

# Arch Linux
pacman -S just

# Verify
just --version
```text

**Configuration:**

No configuration needed. Justfiles are included in the project.

**Verify:**

```bash
# List available recipes
just --list

# Test a recipe
just build::cli
```text

#### Nickel (Configuration Language)

**What:** Programmable configuration language for infrastructure validation

**Required:** Yes (for `typedialog-prov-gen` and Nickel validation features)

**Installation:**

```bash
# macOS (Homebrew)
brew install nickel

# Linux (cargo)
cargo install nickel-lang-cli

# From source
git clone https://github.com/tweag/nickel.git
cd nickel
cargo build --release
sudo cp target/release/nickel /usr/local/bin/

# Verify
nickel --version
```text

**Path Configuration:**

If `nickel` is not in PATH, set the binary location:

```bash
# Add to ~/.bashrc or ~/.zshrc
export NICKEL_BIN=/custom/path/to/nickel

# Or set for current session
export PATH="/custom/path/to/nickel:$PATH"
```text

**Verify:**

```bash
# Test Nickel evaluation
echo '{ x = 1 + 1 }' | nickel eval

# Test with TypeDialog
cargo test -p typedialog-prov-gen --lib
```text

#### shellcheck (Bash Linter)

**What:** Shell script analysis tool for linting bash scripts

**Required:** Yes (for linting 142 bash scripts)

**Installation:**

```bash
# macOS (Homebrew)
brew install shellcheck

# Linux (apt)
sudo apt-get install shellcheck

# Arch Linux
sudo pacman -S shellcheck

# From source (Haskell)
cabal update
cabal install ShellCheck

# Verify
shellcheck --version
```text

**Verify:**

```bash
# Test shellcheck
echo '#!/bin/bash\necho "test"' | shellcheck -

# Lint project scripts
just dev::lint-bash
```text

#### markdownlint-cli2 (Markdown Linter)

**What:** Markdown linting and style checking for documentation

**Required:** Yes (for linting docs/ folder)

**Installation:**

```bash
# Global install (requires Node.js)
npm install -g markdownlint-cli2

# Verify
markdownlint-cli2 --version
```text

**Path Configuration:**

Ensure Node.js is installed and npm global bin is in PATH:

```bash
# Check Node.js
node --version
npm --version

# Add npm global bin to PATH (if needed)
export PATH="$(npm config get prefix)/bin:$PATH"
```text

**Verify:**

```bash
# Test markdownlint
echo '# Test' | markdownlint-cli2 --stdin

# Lint project docs
just dev::lint-markdown
```text

### Dependency Quick Check

Run this script to verify all critical dependencies:

```bash
#!/bin/bash
echo "=== Checking Critical Dependencies ==="

# Check just
if command -v just &> /dev/null; then
    echo "✓ just $(just --version)"
else
    echo "✗ just - NOT FOUND"
    echo "  Install: cargo install just"
fi

# Check nickel
if command -v nickel &> /dev/null; then
    echo "✓ nickel $(nickel --version)"
else
    echo "✗ nickel - NOT FOUND"
    echo "  Install: cargo install nickel-lang-cli"
fi

# Check prov-ecosystem (path dependency)
if cargo metadata --format-version=1 2>/dev/null | grep -q prov-ecosystem; then
    echo "✓ prov-ecosystem - FOUND"
else
    echo "✗ prov-ecosystem - NOT FOUND"
    echo "  Clone to: ../prov-ecosystem"
fi

# Check secretumvault (path dependency)
if cargo metadata --format-version=1 2>/dev/null | grep -q secretumvault; then
    echo "✓ secretumvault - FOUND"
else
    echo "✗ secretumvault - NOT FOUND"
    echo "  Clone to: ../secretumvault"
fi
```text

Save as `scripts/check_deps.sh` and run:

```bash
chmod +x scripts/check_deps.sh
./scripts/check_deps.sh
```text

### Troubleshooting Dependencies

#### prov-ecosystem or secretumvault not found

**Error:**

```text
error: no matching package named `prov-ecosystem` found
location searched: /path/to/typedialog/../prov-ecosystem
```text

**Solutions:**

1. **Clone to expected location:**

   ```bash
   cd /path/to/typedialog/..
   git clone https://github.com/your-org/prov-ecosystem.git
   git clone https://github.com/your-org/secretumvault.git

Update Cargo.toml paths:

# In typedialog/Cargo.toml (workspace level)
prov-ecosystem = { path = "/absolute/path/to/prov-ecosystem" }
secretumvault = { path = "/absolute/path/to/secretumvault" }

Use git submodules:

git submodule add https://github.com/your-org/prov-ecosystem.git deps/prov-ecosystem
# Update Cargo.toml to use deps/prov-ecosystem

Nickel binary not found

Error:

Error: nickel command not found
```text

**Solutions:**

1. **Install via cargo:**

   ```bash
   cargo install nickel-lang-cli

Add to PATH:
```
export PATH="$HOME/.cargo/bin:$PATH"
```

Set explicit path in environment:

export NICKEL_BIN=/usr/local/bin/nickel

just command not found

Error:

just: command not found
```text

**Solutions:**

1. **Install via cargo:**

   ```bash
   cargo install just

Install via package manager:

# macOS
brew install just

# Arch Linux
sudo pacman -S just

Add to PATH:
```
export PATH="$HOME/.cargo/bin:$PATH"
```

Building Binaries

Single Platform Build

Build for your current platform:

# Debug build (default)
just distro::build-all

# Release build (optimized)
just distro::build-release
```text

**Output**: Binaries in `target/release/`

```bash
# Check binaries
ls -la target/release/typedialog*
```text

### Build by Backend

Build specific backend only:

```bash
just build::cli       # CLI backend
just build::tui       # TUI backend
just build::web       # Web backend
just build::all       # All backends
```text

### Development Build

Quick build for development:

```bash
just build::default
```text

## Cross-Compilation

Build for multiple platforms:

### Using cargo-cross

Cross-compile to different targets:

```bash
# All supported targets
just distro::cross

# Specific target
just distro::cross-target x86_64-unknown-linux-gnu
```text

**Supported targets:**

- `x86_64-unknown-linux-gnu` - Linux x86_64
- `aarch64-unknown-linux-gnu` - Linux ARM64
- `x86_64-apple-darwin` - macOS Intel
- `aarch64-apple-darwin` - macOS Apple Silicon
- `x86_64-pc-windows-gnu` - Windows x86_64

**Requires:** `cargo install cross`

### Using Docker

Docker-based cross-compilation:

```bash
# Docker cross-compile
just distro::cross-docker x86_64-unknown-linux-gnu
```text

**Requires:** Docker installed

**Dockerfile:** See `.woodpecker/Dockerfile.cross`

## Creating Distribution Packages

Distribution packages include binaries, configs, and installers.

### Create Package

```bash
# From existing binaries
just distro::create-package

# With specific version
just distro::create-package-version v0.1.0
```text

**Output:** `distribution/typedialog-0.1.0/`

```text
typedialog-0.1.0/
├── bin/
│   ├── typedialog       # CLI binary
│   ├── typedialog-tui   # TUI binary
│   └── typedialog-web   # Web binary
├── config/
│   ├── cli/              # 3 configs: default, dev, production
│   ├── tui/              # 3 configs
│   └── web/              # 3 configs
├── installers/
│   ├── install.sh        # Linux/macOS installer
│   ├── install.ps1       # Windows installer
│   └── README.md         # Installation guide
└── MANIFEST.json         # Package metadata
```text

### Generate Checksums

```bash
just distro::create-checksums
```text

**Output:** `SHA256SUMS` file

Verify integrity:

```bash
sha256sum -c SHA256SUMS
```text

### Full Package Workflow

```bash
# 1. Build release
just distro::build-release

# 2. Create package
just distro::create-package

# 3. Generate checksums
just distro::create-checksums

# Or all together
just distro::package-all
```text

## Distribution Structure

### Package Contents

**bin/** - Compiled binaries

- `typedialog` - CLI backend
- `typedialog-tui` - TUI backend
- `typedialog-web` - Web backend

**config/** - Configuration templates

- CLI configs (default, dev, production)
- TUI configs (default, dev, production)
- Web configs (default, dev, production)

**installers/** - Installation scripts

- `install.sh` - For Linux/macOS
- `install.ps1` - For Windows
- `README.md` - Installation guide

**MANIFEST.json** - Package metadata

```json
{
  "name": "typedialog",
  "version": "0.1.0",
  "created": "...",
  "binaries": [...],
  "configs": {...},
  "installers": {...}
}
```text

### Configuration Management

Pre-configured settings for each backend and environment:

**CLI Backend:**

- `default.toml` - Standard settings
- `dev.toml` - Development (debug enabled)
- `production.toml` - Hardened production

**TUI Backend:**

- `default.toml` - Interactive settings
- `dev.toml` - Development features
- `production.toml` - Optimized production

**Web Backend:**

- `default.toml` - Standard web server
- `dev.toml` - Development (hot reload)
- `production.toml` - Hardened web server

See [configuration.md](configuration.md) for details.

## Build Scripts

All build automation uses bash scripts in `scripts/`:

### build_all.sh

Build all workspace targets:

```bash
./scripts/build_all.sh [debug|release]
```text

### build_cross.sh

Cross-compile for multiple platforms:

```bash
./scripts/build_cross.sh [target]
```text

### create_distro.sh

Create distribution package:

```bash
./scripts/create_distro.sh [version]
```text

### package_release.sh

Prepare release with checksums and notes:

```bash
./scripts/package_release.sh [version]
```text

## Docker Build

### .woodpecker/Dockerfile.cross

Multi-platform Docker build:

```bash
# Build image for specific target
docker build -f .woodpecker/Dockerfile.cross \
    --build-arg TARGET=x86_64-unknown-linux-gnu \
    -t typedialog-builder:latest \
    .

# Run compilation
docker run --rm \
    -v $(pwd)/distribution:/output \
    typedialog-builder:latest
```text

## Complete Release Workflow

Prepare a release for GitHub:

```bash
# 1. Build release binaries
just distro::build-release

# 2. Create distribution package
just distro::create-package

# 3. Generate checksums
just distro::create-checksums

# 4. Prepare release (with notes)
just distro::package-release

# 5. Verify contents
just distro::list-packages

# 6. Create GitHub release (see release.md)
gh release create v0.1.0 \
    release/typedialog-0.1.0.tar.gz \
    release/SHA256SUMS \
    --title "TypeDialog 0.1.0" \
    --notes-file release/RELEASE_NOTES.md
```text

## Utilities

### List packages

```bash
just distro::list-packages
```text

### Generate manifest

```bash
just distro::generate-manifest
```text

### Clean distribution

```bash
just distro::clean-distro
```text

## Troubleshooting

### Build fails

Clear cache and rebuild:

```bash
cargo clean
just distro::build-release
```text

### Cross-compilation fails

Ensure Docker is running:

```bash
docker --version
docker run hello-world
```text

### Missing dependencies

Check all dependencies:

```bash
./scripts/check_deps.sh
```text

Install required tools:

```bash
cargo install cross     # For cross-compilation
cargo install just      # For command orchestration
cargo install nickel-lang-cli  # For Nickel validation
```text

Clone internal dependencies:

```bash
cd /path/to/typedialog/..
git clone https://github.com/your-org/prov-ecosystem.git
git clone https://github.com/your-org/secretumvault.git
```text

### Checksum verification fails

Regenerate checksums:

```bash
just distro::create-checksums
```text

## Platform Support

| Platform | Arch | Status |
 | ---------- | ------ | -------- |
| Linux (glibc) | x86_64 | ✓ Stable |
| Linux (glibc) | ARM64 | ✓ Stable |
| macOS | x86_64 | ✓ Stable |
| macOS | ARM64 | ✓ Stable |
| Windows | x86_64 | ✓ Stable |

## Performance Tips

### Faster builds

```bash
# Use all CPU cores
export CARGO_BUILD_JOBS=$(nproc)
export CARGO_INCREMENTAL=1

# Then build
just distro::build-release
```text

### Parallel testing

```bash
cargo test -- --test-threads=4
```text

## Compliance & SBOM Generation

Generate Software Bill of Materials (SBOM) for supply chain transparency using [`cargo-sbom`](https://crates.io/crates/cargo-sbom).

### Regenerate SBOMs

When dependencies change (new crates, version updates, Cargo.lock changes):

```bash
just distro::generate-sbom
```text

This regenerates:

- **SBOM.spdx.json** - SPDX 2.3 standard format (ISO/IEC 5962:2021)
- **SBOM.cyclonedx.json** - CycloneDX 1.4 format (ECMA standard)

### Audit Dependencies

Check for known security vulnerabilities:

```bash
just ci::audit
```text

### Verify SBOMs in CI

Verify SBOMs are up-to-date during CI pipeline:

```bash
just ci::verify-sbom
```text

Included automatically in full CI run:

```bash
just ci::full
```text

### SBOM Files

**SBOM.spdx.json** (~350-400 KB)

- SPDX 2.3 format (ISO/IEC 5962:2021)
- Complete component inventory with unique identifiers
- Compatible with SPDX validators, GitHub Dependabot, and osv-scanner
- Generated by [cargo-sbom](https://crates.io/crates/cargo-sbom)

**SBOM.cyclonedx.json** (~280-320 KB)

- CycloneDX 1.4 format (ECMA standard)
- Complete component inventory with Package URLs (pURL)
- Compatible with vulnerability scanners and SCA tools (Dependabot, Snyk)
- Generated by [cargo-sbom](https://crates.io/crates/cargo-sbom)

### Supply Chain Security

All dependencies are:

- Permissive or compatible licenses ✓
- Regularly audited for vulnerabilities ✓
- Tracked in version-controlled SBOMs ✓
- Included in releases ✓

---

## Next Steps

- [release.md](release.md) - Release workflow
- [configuration.md](configuration.md) - Configuration options
- [installation.md](installation.md) - User installation
- [development.md](development.md) - Development setup

---

**Build complete!** Ready for distribution.

16 KiB Raw Blame History