nushell-plugins/docs/BUILDING.md

Building and Cross-Compilation Guide

This comprehensive guide covers everything you need to know about building nushell plugins, including cross-compilation for multiple platforms.

Table of Contents

• Quick Start
• Prerequisites
• Build Methods
• Cross-Compilation
• Docker Builds
• Platform Support
• Configuration
• Workflows
• Distribution
• Troubleshooting
• FAQ
• Advanced Topics

Quick Start

Native Build (Host Platform Only)

# Clone repository
git clone https://github.com/YOUR_ORG/nushell-plugins.git
cd nushell-plugins

# Build all plugins for your platform
just build

# Or using the script directly
./scripts/run.sh build_all.nu

Cross-Platform Build

# Build for all supported platforms
just build-cross-all

# Build for specific platform
just build-cross linux-amd64

# Build and create distribution packages
just release-cross

Prerequisites

Required Tools

1. Rust Toolchain (1.70.0+)

   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
   

2. Nushell (0.107.1)

   # Install via cargo
   cargo install nu
   
   # Or download from https://github.com/nushell/nushell/releases
   

3. Just (optional but recommended)

   cargo install just
   

4. Git with submodules

   git clone --recursive https://github.com/YOUR_ORG/nushell-plugins.git
   

Optional Tools

• Docker - For Docker-based cross-compilation
• Cross-compilation toolchains - For native cross-compilation

Build Methods

1. Native Compilation

Build plugins for your current platform using the standard Rust toolchain.

# Basic build
just build

# Verbose output
just build-verbose

# Parallel build (experimental)
just build-parallel

# Build specific plugins
./scripts/run.sh build_all.nu --plugins nu_plugin_clipboard,nu_plugin_image

Pros:

• Fast compilation
• Native debugging support
• Simple setup

Cons:

• Only builds for current platform
• Limited cross-compilation support

2. Cross-Compilation

Build plugins for different platforms using Rust's cross-compilation features.

# List available targets
just build-targets

# Build for specific target
just build-cross linux-amd64
just build-cross darwin-arm64
just build-cross windows-amd64

# Build for all targets
just build-cross-all

# Parallel cross-compilation
just build-cross-parallel

Pros:

• Multiple platforms from single machine
• Fast execution
• No Docker dependency

Cons:

• May require target-specific toolchains
• Some targets may not work on all hosts

3. Docker-Based Cross-Compilation

Use Docker containers with pre-configured cross-compilation environments.

# Build Docker image
just build-docker-image

# Build specific target with Docker
just build-docker linux-arm64

# Force Docker for all builds
./scripts/run.sh build_cross.nu --all-targets --docker

Pros:

• Consistent build environment
• All toolchains pre-installed
• Works on any Docker-capable host

Cons:

• Slower than native compilation
• Requires Docker
• Larger resource usage

Cross-Compilation

Supported Platforms

Platform	Target Triple	Native	Docker	Notes
Linux AMD64	x86_64-unknown-linux-gnu	✅	✅	Most common Linux
Linux ARM64	aarch64-unknown-linux-gnu	⚠️	✅	Requires cross toolchain
macOS Intel	x86_64-apple-darwin	🍎	❌	macOS host only
macOS Apple Silicon	aarch64-apple-darwin	🍎	❌	macOS host only
Windows AMD64	x86_64-pc-windows-msvc	🪟	✅	Windows host or Docker

Legend:

• ✅ Fully supported
• ⚠️ Requires additional setup
• 🍎 macOS host required
• 🪟 Windows host required
• ❌ Not supported

Setting Up Cross-Compilation

Linux → Linux ARM64

# Install cross-compilation toolchain (Ubuntu/Debian)
sudo apt-get install gcc-aarch64-linux-gnu

# Add Rust target
rustup target add aarch64-unknown-linux-gnu

# Configure environment
export CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc

# Build
just build-cross linux-arm64

Any → Windows (Docker)

# Build Docker image with Windows toolchain
just build-docker-image

# Cross-compile to Windows
just build-docker windows-amd64

Configuration Files

Build Targets (etc/build_targets.toml)

[targets.linux-amd64]
rust_target = "x86_64-unknown-linux-gnu"
platform_name = "linux-amd64"
archive_format = "tar.gz"
docker_required = false
native_build = true
description = "Linux x86_64 (64-bit Intel/AMD)"

[targets.linux-arm64]
rust_target = "aarch64-unknown-linux-gnu"
platform_name = "linux-arm64"
archive_format = "tar.gz"
docker_required = true
native_build = false
description = "Linux ARM64 (64-bit ARM)"
linker = "aarch64-linux-gnu-gcc"

Environment Variables

# Cross-compilation linkers
export CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc
export CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++
export AR_aarch64_unknown_linux_gnu=aarch64-linux-gnu-ar

# Cargo configuration
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc

Docker Builds

Docker Image

The cross-compilation Docker image includes:

• Rust toolchain (1.75.0)
• Cross-compilation targets for all supported platforms
• System toolchains (GCC, MinGW, etc.)
• Build tools (cargo, rustfmt, clippy)

Docker Commands

# Build Docker image
just build-docker-image

# Rebuild from scratch
just build-docker-image-fresh

# Show Docker environment info
just docker-info

# Build specific plugin with Docker
just build-docker-plugin nu_plugin_clipboard linux-arm64

# Clean up Docker artifacts
just docker-cleanup

Docker Workflow

# Complete Docker-based workflow
just docker-flow

# This runs:
# 1. just validate-nushell
# 2. just build-docker-image
# 3. just build-cross-all --docker
# 4. just collect-all
# 5. just pack-all

Manual Docker Usage

# Run interactive Docker container
docker run -it --rm \
  -v $(pwd):/workspace \
  -w /workspace \
  nushell-plugins-cross:latest \
  bash

# Build specific target in container
docker run --rm \
  -v $(pwd):/workspace \
  -w /workspace/nu_plugin_clipboard \
  nushell-plugins-cross:latest \
  cargo build --release --target aarch64-unknown-linux-gnu

Platform Support

Linux

Supported Architectures:

• x86_64 (AMD64) - Native and Docker
• ARM64 (AArch64) - Docker recommended
• ARM32 (ARMv7) - Docker only, disabled by default

Dependencies:

# Ubuntu/Debian
sudo apt-get install build-essential libssl-dev pkg-config

# Cross-compilation toolchains
sudo apt-get install gcc-aarch64-linux-gnu gcc-arm-linux-gnueabihf

macOS

Supported Architectures:

• Intel (x86_64) - Native only
• Apple Silicon (ARM64) - Native only

Dependencies:

# Install Xcode command line tools
xcode-select --install

# Add Rust targets
rustup target add x86_64-apple-darwin aarch64-apple-darwin

Cross-compilation:

# On Apple Silicon, build for Intel
just build-cross darwin-amd64

# On Intel, build for Apple Silicon (may not work)
just build-cross darwin-arm64

Windows

Supported Architectures:

• x86_64 (AMD64) - Native and Docker

Native Dependencies:

• Visual Studio Build Tools or Visual Studio Community
• Windows SDK

Docker Alternative:

# Use Docker for Windows builds on any platform
just build-docker windows-amd64

Configuration

Global Configuration

The main configuration file is etc/build_targets.toml:

[metadata]
version = "1.0.0"
description = "Cross-compilation targets for nushell plugins"

[defaults]
docker_image = "nushell-plugins-cross:latest"
strip_binaries = true
optimize_size = true

[targets.custom-target]
rust_target = "x86_64-unknown-linux-musl"
platform_name = "linux-amd64-musl"
archive_format = "tar.gz"
docker_required = false
description = "Linux x86_64 with musl libc"
enabled = true

Environment Configuration

Create an env file for local overrides:

# env file
TARGET_PATH=distribution
INSTALL_BIN_PATH=/usr/local/bin
BIN_ARCHIVES_DIR_PATH=bin_archives
APP_NAME=nushell-plugins

Plugin-Specific Configuration

Some plugins may require special build configurations:

# In plugin's Cargo.toml
[package.metadata.cross]
image = "ghcr.io/cross-rs/cross:aarch64-unknown-linux-gnu"

[target.aarch64-unknown-linux-gnu]
linker = "aarch64-linux-gnu-gcc"

Workflows

Development Workflows

Standard Development

just dev-flow
# 1. Validate nushell version
# 2. Check upstream changes
# 3. Build for host platform
# 4. Run tests
# 5. Show status

Cross-Platform Development

just dev-flow-cross
# 1. Validate nushell version
# 2. Check upstream changes
# 3. Build for all platforms
# 4. Show status

Quality Assurance

just quality-flow
# 1. Validate nushell version
# 2. Format code
# 3. Run clippy linting
# 4. Run tests

Release Workflows

Standard Release

just release-flow
# 1. Validate nushell version
# 2. Build for host platform
# 3. Collect binaries
# 4. Create packages

Cross-Platform Release

just release-flow-cross
# 1. Validate nushell version
# 2. Build for all platforms
# 3. Collect all binaries
# 4. Create packages with checksums

CI Simulation

just ci-flow
# Simulates exactly what GitHub Actions will run

Custom Workflows

You can create custom workflows by combining individual commands:

# Custom workflow for testing
just validate-nushell
just build-cross linux-amd64
just build-cross darwin-arm64
just test
just collect-platform linux-amd64
just pack-platform linux-amd64

Distribution

Collection

Collect built binaries for distribution:

# Collect all platforms
just collect-all

# Collect specific platform
just collect-platform linux-amd64

# List what can be collected
just collect-platforms

Packaging

Create distribution archives:

# Package all platforms
just pack-all

# Package with checksums
just pack-checksums

# Package specific platform
just pack-platform darwin-arm64

# List packaging options
just pack-platforms

Directory Structure

After building and packaging:

distribution/
├── linux-amd64/
│   ├── nu_plugin_clipboard
│   ├── nu_plugin_image
│   ├── install_nu_plugins.nu
│   ├── LICENSE
│   └── README
├── darwin-arm64/
│   └── [same structure]
└── windows-amd64/
    ├── nu_plugin_clipboard.exe
    └── [same structure]

bin_archives/
├── linux-amd64-nushell-plugins.tar.gz
├── darwin-arm64-nushell-plugins.tar.gz
├── windows-amd64-nushell-plugins.zip
└── checksums.txt

Checksums

All distribution packages include SHA256 checksums:

# Generate checksums
just pack-checksums

# Verify downloaded archive
sha256sum -c checksums.txt

# Manual verification
sha256sum linux-amd64-nushell-plugins.tar.gz

Troubleshooting

Common Issues

Version Mismatch

# Error: Nushell version mismatch detected
# Fix: Update versions automatically
just fix-nushell

# Or manually check and fix
./scripts/run.sh check_version.nu --fix

Missing Cross-Compilation Target

# Error: target 'aarch64-unknown-linux-gnu' not found
# Fix: Install the target
rustup target add aarch64-unknown-linux-gnu

Cross-Compilation Linker Error

# Error: linker 'aarch64-linux-gnu-gcc' not found
# Fix: Install cross-compilation toolchain
sudo apt-get install gcc-aarch64-linux-gnu

# Set environment variable
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc

Docker Build Failures

# Error: Docker daemon not running
# Fix: Start Docker service
sudo systemctl start docker

# Error: Docker image not found
# Fix: Build the image
just build-docker-image

# Error: Permission denied in Docker
# Fix: Add user to docker group
sudo usermod -aG docker $USER

Build Failures

Plugin-specific failures:

# Debug specific plugin
cd nu_plugin_clipboard
cargo build --release --verbose

# Check plugin dependencies
cargo tree

Cross-compilation failures:

# Use Docker instead of native cross-compilation
just build-docker linux-arm64

# Check target support
rustup target list | grep installed

Debug Mode

Enable verbose output for troubleshooting:

# Verbose build output
just build-verbose

# Verbose cross-compilation
./scripts/run.sh build_cross.nu --all-targets --verbose

# Debug Docker builds
./scripts/run.sh build_docker_cross.nu --verbose

Log Files

Build logs are preserved for debugging:

# View recent build logs
ls -la target/*/build/*/out/

# Search for specific errors
grep -r "error" target/*/build/*/stderr

FAQ

General Questions

Q: What platforms are supported? A: Linux (AMD64, ARM64), macOS (Intel, Apple Silicon), and Windows (AMD64). See the Platform Support section for details.

Q: Do I need Docker for cross-compilation? A: Not always. Native cross-compilation works for many targets, but Docker provides a more consistent environment and supports more target combinations.

Q: How long does cross-compilation take? A: Building all platforms takes 10-30 minutes depending on your hardware. Parallel builds (just build-cross-parallel) can significantly reduce this time.

Q: Can I add new platforms? A: Yes! Edit etc/build_targets.toml to add new targets. See Configuration for details.

Build Questions

Q: Why do some builds fail on my platform? A: Cross-compilation has limitations. macOS targets can only be built on macOS, and some Linux ARM64 builds require specific toolchains. Use Docker builds as a fallback.

Q: How can I speed up builds? A: Use parallel builds (--parallel), enable incremental compilation, and use Docker layer caching. Consider building only the platforms you need.

Q: Can I build just one plugin? A: Yes! Use ./scripts/run.sh build_all.nu --plugins nu_plugin_clipboard or build manually in the plugin directory.

Q: Why are Windows binaries so large? A: Windows binaries include the MSVC runtime. You can reduce size by using the GNU toolchain or enabling link-time optimization.

Docker Questions

Q: How much disk space does the Docker image use? A: The cross-compilation image is approximately 2-3 GB, including all toolchains and dependencies.

Q: Can I use my own Docker image? A: Yes! Modify the docker.image_tag setting in etc/build_targets.toml to use a custom image.

Q: How do I update the Docker image? A: Run just build-docker-image-fresh to rebuild from scratch with the latest dependencies.

Distribution Questions

Q: How do I install the plugins? A: Use the universal installer: curl -L https://github.com/YOUR_ORG/nushell-plugins/releases/latest/download/install.sh | sh

Q: Can I distribute individual plugins? A: Yes! Each plugin binary is self-contained. Just copy the nu_plugin_* files to the target system.

Q: What's the difference between tar.gz and zip archives? A: Unix-like systems (Linux, macOS) use tar.gz for better compression and permission preservation. Windows uses zip for compatibility.

Troubleshooting Questions

Q: Build fails with "permission denied" errors A: Check that you have write permissions to the target directory and that antivirus software isn't blocking the builds.

Q: Cross-compilation fails with "unsupported target" A: Install the target with rustup target add TARGET_NAME or use Docker builds instead.

Q: Docker builds are very slow A: Make sure Docker has sufficient resources allocated. Consider using Docker BuildKit for faster builds.

Advanced Topics

Custom Build Targets

Add support for new platforms by editing etc/build_targets.toml:

[targets.linux-musl]
rust_target = "x86_64-unknown-linux-musl"
platform_name = "linux-amd64-musl"
archive_format = "tar.gz"
docker_required = false
description = "Linux x86_64 with musl libc"
strip_binaries = true

[environment.linux-musl]
CC = "musl-gcc"

Build Optimization

Link-Time Optimization (LTO)

# In Cargo.toml
[profile.release]
lto = true
codegen-units = 1
panic = "abort"
strip = "symbols"

Size Optimization

[profile.release]
opt-level = "z"  # Optimize for size
lto = true
strip = "symbols"

Build Scripts

Create custom build scripts for complex scenarios:

#!/bin/bash
# scripts/custom_build.sh

# Custom build with specific optimizations
export RUSTFLAGS="-C target-cpu=native"
cargo build --release --target x86_64-unknown-linux-gnu

# Strip debug symbols
strip target/x86_64-unknown-linux-gnu/release/nu_plugin_*

Continuous Integration

The repository includes GitHub Actions workflows for:

• Build & Test (.github/workflows/build.yml)
• Release (.github/workflows/release.yml)
• Nightly Builds (.github/workflows/nightly.yml)

To set up in your own repository:

1. Copy the .github/workflows/ directory
2. Update the repository references
3. Configure GitHub secrets if needed
4. Customize the build matrix for your needs

Plugin Development

When developing new plugins:

1. Use the template:

   just make-plugin nu_plugin_yourname
   

2. Follow the plugin structure:

   nu_plugin_yourname/
   ├── Cargo.toml
   ├── src/
   │   ├── main.rs
   │   └── lib.rs
   └── README.md
   

3. Test cross-compilation early:

   just build-cross-all
   

4. Add to the registry: Update etc/plugin_registry.toml if the plugin has an upstream repository.

Performance Tuning

Parallel Compilation

# Set number of parallel jobs
export CARGO_BUILD_JOBS=8

# Use parallel cross-compilation
just build-cross-parallel

Caching

# Use sccache for distributed compilation caching
cargo install sccache
export RUSTC_WRAPPER=sccache

Memory Usage

# Limit memory usage for large builds
export CARGO_BUILD_RUSTFLAGS="-C link-arg=-Wl,--no-keep-memory"

---

Getting Help

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: Repository Wiki
• Nushell Community: Discord | Reddit

Contributing

See CONTRIBUTING.md for guidelines on contributing to the build system and adding new platforms or features.

---

This documentation is maintained by the nushell-plugins team. Last updated: 2024-09-20