TypeDialog/docs/build.md

<div align="center">
  <img src="../assets/typedialog_logo_h_s.svg" alt="TypeDialog Logo" width="600" />
</div>

# Building & Distribution Guide

Complete guide for building, packaging, and releasing TypeDialog.

## Overview

Two main workflows:

1. **Building** - Compile binaries for your platform
2. **Distribution** - Package with configs and installers for release

## Critical Dependencies

TypeDialog requires several critical dependencies for building and running. These include both **internal dependencies** (git submodules or external projects) and **external tools**.

### Internal Dependencies

#### prov-ecosystem

**What:** Provisioning ecosystem library for infrastructure generation (used by `typedialog-prov-gen`)

**Location:** Expected as sibling directory or git submodule

**Installation:**

```bash
# Option 1: Clone as sibling directory (recommended)
cd /path/to/projects
git clone https://github.com/your-org/prov-ecosystem.git
git clone https://github.com/your-org/typedialog.git

# Option 2: Git submodule
cd typedialog
git submodule add https://github.com/your-org/prov-ecosystem.git deps/prov-ecosystem
git submodule update --init --recursive
```

**Path Configuration:**

If `prov-ecosystem` is in a non-standard location, update `Cargo.toml`:

```toml
[dependencies]
prov-ecosystem = { path = "../prov-ecosystem" }  # Adjust path as needed
```

Or use environment variable:

```bash
export PROV_ECOSYSTEM_PATH=/custom/path/to/prov-ecosystem
```

**Verify:**

```bash
# Check if prov-ecosystem is found
cargo metadata --format-version=1 | grep prov-ecosystem
```

#### secretumvault

**What:** Unified encryption vault library (used by encryption features)

**Location:** Expected as sibling directory or git submodule

**Installation:**

```bash
# Option 1: Clone as sibling directory (recommended)
cd /path/to/projects
git clone https://github.com/your-org/secretumvault.git
git clone https://github.com/your-org/typedialog.git

# Option 2: Git submodule
cd typedialog
git submodule add https://github.com/your-org/secretumvault.git deps/secretumvault
git submodule update --init --recursive
```

**Path Configuration:**

If `secretumvault` is in a non-standard location, update `Cargo.toml`:

```toml
[dependencies]
secretumvault = { path = "../secretumvault" }  # Adjust path as needed
```

Or use environment variable:

```bash
export SECRETUMVAULT_PATH=/custom/path/to/secretumvault
```

**Verify:**

```bash
# Check if secretumvault is found
cargo metadata --format-version=1 | grep secretumvault
```

### External Dependencies

#### just (Command Runner)

**What:** Command orchestration tool for running build recipes

**Required:** Yes (for all build workflows)

**Installation:**

```bash
# macOS (Homebrew)
brew install just

# Linux (cargo)
cargo install just

# Linux (from source)
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin

# Arch Linux
pacman -S just

# Verify
just --version
```

**Configuration:**

No configuration needed. Justfiles are included in the project.

**Verify:**

```bash
# List available recipes
just --list

# Test a recipe
just build::cli
```

#### Nickel (Configuration Language)

**What:** Programmable configuration language for infrastructure validation

**Required:** Yes (for `typedialog-prov-gen` and Nickel validation features)

**Installation:**

```bash
# macOS (Homebrew)
brew install nickel

# Linux (cargo)
cargo install nickel-lang-cli

# From source
git clone https://github.com/tweag/nickel.git
cd nickel
cargo build --release
sudo cp target/release/nickel /usr/local/bin/

# Verify
nickel --version
```

**Path Configuration:**

If `nickel` is not in PATH, set the binary location:

```bash
# Add to ~/.bashrc or ~/.zshrc
export NICKEL_BIN=/custom/path/to/nickel

# Or set for current session
export PATH="/custom/path/to/nickel:$PATH"
```

**Verify:**

```bash
# Test Nickel evaluation
echo '{ x = 1 + 1 }' | nickel eval

# Test with TypeDialog
cargo test -p typedialog-prov-gen --lib
```

#### shellcheck (Bash Linter)

**What:** Shell script analysis tool for linting bash scripts

**Required:** Yes (for linting 142 bash scripts)

**Installation:**

```bash
# macOS (Homebrew)
brew install shellcheck

# Linux (apt)
sudo apt-get install shellcheck

# Arch Linux
sudo pacman -S shellcheck

# From source (Haskell)
cabal update
cabal install ShellCheck

# Verify
shellcheck --version
```

**Verify:**

```bash
# Test shellcheck
echo '#!/bin/bash\necho "test"' | shellcheck -

# Lint project scripts
just dev::lint-bash
```

#### markdownlint-cli2 (Markdown Linter)

**What:** Markdown linting and style checking for documentation

**Required:** Yes (for linting docs/ folder)

**Installation:**

```bash
# Global install (requires Node.js)
npm install -g markdownlint-cli2

# Verify
markdownlint-cli2 --version
```

**Path Configuration:**

Ensure Node.js is installed and npm global bin is in PATH:

```bash
# Check Node.js
node --version
npm --version

# Add npm global bin to PATH (if needed)
export PATH="$(npm config get prefix)/bin:$PATH"
```

**Verify:**

```bash
# Test markdownlint
echo '# Test' | markdownlint-cli2 --stdin

# Lint project docs
just dev::lint-markdown
```

### Dependency Quick Check

Run this script to verify all critical dependencies:

```bash
#!/bin/bash
echo "=== Checking Critical Dependencies ==="

# Check just
if command -v just &> /dev/null; then
    echo "✓ just $(just --version)"
else
    echo "✗ just - NOT FOUND"
    echo "  Install: cargo install just"
fi

# Check nickel
if command -v nickel &> /dev/null; then
    echo "✓ nickel $(nickel --version)"
else
    echo "✗ nickel - NOT FOUND"
    echo "  Install: cargo install nickel-lang-cli"
fi

# Check prov-ecosystem (path dependency)
if cargo metadata --format-version=1 2>/dev/null | grep -q prov-ecosystem; then
    echo "✓ prov-ecosystem - FOUND"
else
    echo "✗ prov-ecosystem - NOT FOUND"
    echo "  Clone to: ../prov-ecosystem"
fi

# Check secretumvault (path dependency)
if cargo metadata --format-version=1 2>/dev/null | grep -q secretumvault; then
    echo "✓ secretumvault - FOUND"
else
    echo "✗ secretumvault - NOT FOUND"
    echo "  Clone to: ../secretumvault"
fi
```

Save as `scripts/check_deps.sh` and run:

```bash
chmod +x scripts/check_deps.sh
./scripts/check_deps.sh
```

### Troubleshooting Dependencies

#### prov-ecosystem or secretumvault not found

**Error:**

```text
error: no matching package named `prov-ecosystem` found
location searched: /path/to/typedialog/../prov-ecosystem
```

**Solutions:**

1. **Clone to expected location:**

   ```bash
   cd /path/to/typedialog/..
   git clone https://github.com/your-org/prov-ecosystem.git
   git clone https://github.com/your-org/secretumvault.git
   ```

2. **Update Cargo.toml paths:**

   ```toml
   # In typedialog/Cargo.toml (workspace level)
   prov-ecosystem = { path = "/absolute/path/to/prov-ecosystem" }
   secretumvault = { path = "/absolute/path/to/secretumvault" }
   ```

3. **Use git submodules:**

   ```bash
   git submodule add https://github.com/your-org/prov-ecosystem.git deps/prov-ecosystem
   # Update Cargo.toml to use deps/prov-ecosystem
   ```

#### Nickel binary not found

**Error:**

```text
Error: nickel command not found
```

**Solutions:**

1. **Install via cargo:**

   ```bash
   cargo install nickel-lang-cli
   ```

2. **Add to PATH:**

   ```bash
   export PATH="$HOME/.cargo/bin:$PATH"
   ```

3. **Set explicit path in environment:**

   ```bash
   export NICKEL_BIN=/usr/local/bin/nickel
   ```

#### just command not found

**Error:**

```text
just: command not found
```

**Solutions:**

1. **Install via cargo:**

   ```bash
   cargo install just
   ```

2. **Install via package manager:**

   ```bash
   # macOS
   brew install just

   # Arch Linux
   sudo pacman -S just
   ```

3. **Add to PATH:**

   ```bash
   export PATH="$HOME/.cargo/bin:$PATH"
   ```

## Building Binaries

### Single Platform Build

Build for your current platform:

```bash
# Debug build (default)
just distro::build-all

# Release build (optimized)
just distro::build-release
```

**Output**: Binaries in `target/release/`

```bash
# Check binaries
ls -la target/release/typedialog*
```

### Build by Backend

Build specific backend only:

```bash
just build::cli       # CLI backend
just build::tui       # TUI backend
just build::web       # Web backend
just build::all       # All backends
```

### Development Build

Quick build for development:

```bash
just build::default
```

## Cross-Compilation

Build for multiple platforms:

### Using cargo-cross

Cross-compile to different targets:

```bash
# All supported targets
just distro::cross

# Specific target
just distro::cross-target x86_64-unknown-linux-gnu
```

**Supported targets:**

- `x86_64-unknown-linux-gnu` - Linux x86_64
- `aarch64-unknown-linux-gnu` - Linux ARM64
- `x86_64-apple-darwin` - macOS Intel
- `aarch64-apple-darwin` - macOS Apple Silicon
- `x86_64-pc-windows-gnu` - Windows x86_64

**Requires:** `cargo install cross`

### Using Docker

Docker-based cross-compilation:

```bash
# Docker cross-compile
just distro::cross-docker x86_64-unknown-linux-gnu
```

**Requires:** Docker installed

**Dockerfile:** See `.woodpecker/Dockerfile.cross`

## Creating Distribution Packages

Distribution packages include binaries, configs, and installers.

### Create Package

```bash
# From existing binaries
just distro::create-package

# With specific version
just distro::create-package-version v0.1.0
```

**Output:** `distribution/typedialog-0.1.0/`

```text
typedialog-0.1.0/
├── bin/
│   ├── typedialog       # CLI binary
│   ├── typedialog-tui   # TUI binary
│   └── typedialog-web   # Web binary
├── config/
│   ├── cli/              # 3 configs: default, dev, production
│   ├── tui/              # 3 configs
│   └── web/              # 3 configs
├── installers/
│   ├── install.sh        # Linux/macOS installer
│   ├── install.ps1       # Windows installer
│   └── README.md         # Installation guide
└── MANIFEST.json         # Package metadata
```

### Generate Checksums

```bash
just distro::create-checksums
```

**Output:** `SHA256SUMS` file

Verify integrity:

```bash
sha256sum -c SHA256SUMS
```

### Full Package Workflow

```bash
# 1. Build release
just distro::build-release

# 2. Create package
just distro::create-package

# 3. Generate checksums
just distro::create-checksums

# Or all together:
just distro::package-all
```

## Distribution Structure

### Package Contents

**bin/** - Compiled binaries

- `typedialog` - CLI backend
- `typedialog-tui` - TUI backend
- `typedialog-web` - Web backend

**config/** - Configuration templates

- CLI configs (default, dev, production)
- TUI configs (default, dev, production)
- Web configs (default, dev, production)

**installers/** - Installation scripts

- `install.sh` - For Linux/macOS
- `install.ps1` - For Windows
- `README.md` - Installation guide

**MANIFEST.json** - Package metadata

```json
{
  "name": "typedialog",
  "version": "0.1.0",
  "created": "...",
  "binaries": [...],
  "configs": {...},
  "installers": {...}
}
```

### Configuration Management

Pre-configured settings for each backend and environment:

**CLI Backend:**

- `default.toml` - Standard settings
- `dev.toml` - Development (debug enabled)
- `production.toml` - Hardened production

**TUI Backend:**

- `default.toml` - Interactive settings
- `dev.toml` - Development features
- `production.toml` - Optimized production

**Web Backend:**

- `default.toml` - Standard web server
- `dev.toml` - Development (hot reload)
- `production.toml` - Hardened web server

See [configuration.md](configuration.md) for details.

## Build Scripts

All build automation uses bash scripts in `scripts/`:

### build_all.sh

Build all workspace targets:

```bash
./scripts/build_all.sh [debug|release]
```

### build_cross.sh

Cross-compile for multiple platforms:

```bash
./scripts/build_cross.sh [target]
```

### create_distro.sh

Create distribution package:

```bash
./scripts/create_distro.sh [version]
```

### package_release.sh

Prepare release with checksums and notes:

```bash
./scripts/package_release.sh [version]
```

## Docker Build

### .woodpecker/Dockerfile.cross

Multi-platform Docker build:

```bash
# Build image for specific target
docker build -f .woodpecker/Dockerfile.cross \
    --build-arg TARGET=x86_64-unknown-linux-gnu \
    -t typedialog-builder:latest \
    .

# Run compilation
docker run --rm \
    -v $(pwd)/distribution:/output \
    typedialog-builder:latest
```

## Complete Release Workflow

Prepare a release for GitHub:

```bash
# 1. Build release binaries
just distro::build-release

# 2. Create distribution package
just distro::create-package

# 3. Generate checksums
just distro::create-checksums

# 4. Prepare release (with notes)
just distro::package-release

# 5. Verify contents
just distro::list-packages

# 6. Create GitHub release (see release.md)
gh release create v0.1.0 \
    release/typedialog-0.1.0.tar.gz \
    release/SHA256SUMS \
    --title "TypeDialog 0.1.0" \
    --notes-file release/RELEASE_NOTES.md
```

## Utilities

### List packages

```bash
just distro::list-packages
```

### Generate manifest

```bash
just distro::generate-manifest
```

### Clean distribution

```bash
just distro::clean-distro
```

## Troubleshooting

### Build fails

Clear cache and rebuild:

```bash
cargo clean
just distro::build-release
```

### Cross-compilation fails

Ensure Docker is running:

```bash
docker --version
docker run hello-world
```

### Missing dependencies

Check all dependencies:

```bash
./scripts/check_deps.sh
```

Install required tools:

```bash
cargo install cross     # For cross-compilation
cargo install just      # For command orchestration
cargo install nickel-lang-cli  # For Nickel validation
```

Clone internal dependencies:

```bash
cd /path/to/typedialog/..
git clone https://github.com/your-org/prov-ecosystem.git
git clone https://github.com/your-org/secretumvault.git
```

### Checksum verification fails

Regenerate checksums:

```bash
just distro::create-checksums
```

## Platform Support

| Platform | Arch | Status |
 | ---------- | ------ | -------- |
| Linux (glibc) | x86_64 | ✓ Stable |
| Linux (glibc) | ARM64 | ✓ Stable |
| macOS | x86_64 | ✓ Stable |
| macOS | ARM64 | ✓ Stable |
| Windows | x86_64 | ✓ Stable |

## Performance Tips

### Faster builds

```bash
# Use all CPU cores
export CARGO_BUILD_JOBS=$(nproc)
export CARGO_INCREMENTAL=1

# Then build
just distro::build-release
```

### Parallel testing

```bash
cargo test -- --test-threads=4
```

## Compliance & SBOM Generation

Generate Software Bill of Materials (SBOM) for supply chain transparency using [`cargo-sbom`](https://crates.io/crates/cargo-sbom).

### Regenerate SBOMs

When dependencies change (new crates, version updates, Cargo.lock changes):

```bash
just distro::generate-sbom
```

This regenerates:

- **SBOM.spdx.json** - SPDX 2.3 standard format (ISO/IEC 5962:2021)
- **SBOM.cyclonedx.json** - CycloneDX 1.4 format (ECMA standard)

### Audit Dependencies

Check for known security vulnerabilities:

```bash
just ci::audit
```

### Verify SBOMs in CI

Verify SBOMs are up-to-date during CI pipeline:

```bash
just ci::verify-sbom
```

Included automatically in full CI run:

```bash
just ci::full
```

### SBOM Files

**SBOM.spdx.json** (~350-400 KB)

- SPDX 2.3 format (ISO/IEC 5962:2021)
- Complete component inventory with unique identifiers
- Compatible with SPDX validators, GitHub Dependabot, and osv-scanner
- Generated by [cargo-sbom](https://crates.io/crates/cargo-sbom)

**SBOM.cyclonedx.json** (~280-320 KB)

- CycloneDX 1.4 format (ECMA standard)
- Complete component inventory with Package URLs (pURL)
- Compatible with vulnerability scanners and SCA tools (Dependabot, Snyk)
- Generated by [cargo-sbom](https://crates.io/crates/cargo-sbom)

### Supply Chain Security

All dependencies are:

- Permissive or compatible licenses ✓
- Regularly audited for vulnerabilities ✓
- Tracked in version-controlled SBOMs ✓
- Included in releases ✓

---

## Next Steps

- [release.md](release.md) - Release workflow
- [configuration.md](configuration.md) - Configuration options
- [installation.md](installation.md) - User installation
- [development.md](development.md) - Development setup

---

**Build complete!** Ready for distribution.