syntaxis/scripts/provisioning

syntaxis Provisioning System

A modular, offline-first provisioning system for building, packaging, and distributing syntaxis binaries and configurations across multiple platforms.

This system focuses on LOCAL and SINGLE-MACHINE provisioning. For remote/multi-host deployment, see integration with project-provisioning.

Overview

The provisioning system is built on modular NuShell scripts that operate independently while supporting orchestrated workflows. Each component can be used standalone or combined through the provctl orchestrator.

Key Features

• ✅ Service Catalog: Central registry of all available services with metadata
• ✅ Service Discovery: Tools to explore available services and presets
• ✅ Modular Design: Each script can be used independently
• ✅ Offline-First: Create self-contained bundles with all artifacts
• ✅ Multi-Platform: Support for Linux, macOS, Windows (x86_64, ARM64)
• ✅ Configuration-Driven: TOML-based configuration (no hardcoded values)
• ✅ Integrity Verification: SHA256 checksums for all artifacts
• ✅ Backup & Safety: Automatic backups before overwriting
• ✅ Manifest Tracking: Installation tracking via TOML manifests
• ✅ Local Service Management: Support for local service startup and monitoring

Architecture

syntaxis/
├── configs/
│   ├── services-catalog.toml      ← Central service registry (NEW)
│   ├── installation.toml           ← Presets and local config
│   └── provisioning/services/      ← Service definitions
│
└── scripts/provisioning/
    ├── service-catalog.nu          ← Service discovery tool (NEW)
    ├── pack.nu                      ← Create distribution bundles
    ├── unpack.nu                    ← Extract bundles
    ├── install.nu                   ← Install binaries locally
    ├── deploy.nu                    ← Deploy configurations
    ├── install-with-presets.nu      ← Local installation with presets
    ├── provctl.nu                   ← Orchestrator for local workflows
    ├── common/
    │   ├── platform.nu              ← OS/arch detection
    │   ├── manifest.nu              ← Manifest I/O
    │   └── validate.nu              ← Integrity checking
    └── README.md                    ← This file

What's New: Service Catalog System

The service catalog is the bridge between syntaxis and project-provisioning.

Service Registry (configs/services-catalog.toml)

Centralized specification of all syntaxis services:

[service.syntaxis-api]
name = "syntaxis-api"
display_name = "syntaxis REST API"
description = "REST API server"
type = "server"

[service.syntaxis-api.server]
default_host = "127.0.0.1"
default_port = 3000
scheme = "http"

[service.syntaxis-api.usage]
health_check_endpoint = "/health"
examples = [
    "syntaxis-api --bind 127.0.0.1:3000",
    "curl http://127.0.0.1:3000/health",
]

Service Discovery Tool (scripts/provisioning/service-catalog.nu)

Explore what's available:

# List all services
nu scripts/provisioning/service-catalog.nu list

# Get details about a service
nu scripts/provisioning/service-catalog.nu info syntaxis-api

# See how to use a service
nu scripts/provisioning/service-catalog.nu usage syntaxis-api

# Show what's in a preset
nu scripts/provisioning/service-catalog.nu preset dev

# View port assignments
nu scripts/provisioning/service-catalog.nu ports

# Export for project-provisioning
nu scripts/provisioning/service-catalog.nu export staging

Why This Matters

The service catalog allows:

1. Users to discover what services are available and how to use them
2. Developers to understand service dependencies and configurations
3. project-provisioning to automatically deploy services with correct:
   • Port assignments
   • Health checks
   • Resource limits
   • Dependencies
   • Startup ordering

Without hardcoding anything in project-provisioning!

Configuration

The system is configured via configs/provisioning.toml. Key sections:

Targets

Define supported platforms and their properties:

[targets.x86_64-unknown-linux-gnu]
enabled = true
os = "linux"
arch = "x86_64"
libc = "glibc"
formats = ["tar.gz", "deb"]
default_prefix = "/usr/local"

Artifacts

Specify what gets included in bundles:

[artifacts]
binaries = [
    { name = "syntaxis-cli", crate = "cli" },
    { name = "syntaxis-tui", crate = "tui" },
    { name = "syntaxis-api", crate = "api" },
]
configs = ["configs/provisioning.toml", "configs/database-default.toml", ...]
docs = ["README.md", "docs/installation.md"]

Packaging Strategies

Configure how bundles are created:

[packaging.tar_gz]
enabled = true
format = "tar.gz"
compression = "gzip"
platforms = ["linux", "macos"]

Commands

1. pack.nu - Create Bundles

Creates distribution bundles from built binaries.

# Create bundle for current platform
nu scripts/provisioning/pack.nu

# Create for specific target
nu scripts/provisioning/pack.nu --target x86_64-unknown-linux-gnu

# Create with specific format
nu scripts/provisioning/pack.nu --target aarch64-apple-darwin --format zip

# List available targets
nu scripts/provisioning/pack.nu --list-targets

# Dry run
nu scripts/provisioning/pack.nu --dry-run

Options:

• --target <TARGET> - Target triple (default: current platform)
• --format <FORMAT> - Package format: tar.gz, zip, deb, rpm (default: tar.gz)
• --output <DIR> - Output directory (default: dist/)
• --verify - Verify bundle after creation
• --list-targets - Show available targets
• --dry-run - Preview without creating

Output:

dist/
├── syntaxis-v0.1.0-x86_64-linux.tar.gz    # The bundle
├── syntaxis-v0.1.0-x86_64-linux.checksums.toml  # SHA256 checksums

2. unpack.nu - Extract Bundles

Extracts and validates bundle contents.

# Extract to current directory
nu scripts/provisioning/unpack.nu bundle.tar.gz

# Extract to specific location with verification
nu scripts/provisioning/unpack.nu bundle.tar.gz --dest /tmp/syntaxis --verify

# Dry run (preview)
nu scripts/provisioning/unpack.nu bundle.tar.gz --dry-run

Options:

• --dest <DIR> - Destination directory (default: current)
• --verify - Verify checksums after extraction
• --dry-run - Preview without extracting

Output:

destination/
└── syntaxis-v0.1.0-x86_64-linux/
    ├── manifest.toml
    ├── bin/
    │   ├── syntaxis-cli
    │   ├── syntaxis-tui
    │   └── syntaxis-api
    ├── configs/
    │   ├── provisioning.toml
    │   └── ...
    └── docs/
        ├── README.md
        └── ...

3. install.nu - Install Binaries

Installs binaries to system from extracted bundles.

# Install from extracted bundle
nu scripts/provisioning/install.nu --source ./syntaxis-v0.1.0

# Install to custom prefix
nu scripts/provisioning/install.nu --prefix /opt/syntaxis

# Install to home directory with symlinks
nu scripts/provisioning/install.nu --prefix ~/.local --create-links

# Install with backup of existing binaries
nu scripts/provisioning/install.nu --backup

# Force overwrite without prompting
nu scripts/provisioning/install.nu --force

Options:

• --source <DIR> - Source directory containing binaries (default: bin/)
• --prefix <PATH> - Installation prefix (default: /usr/local or ~/.local)
• --create-links - Create symlinks in ~/.local/bin
• --backup - Backup existing binaries before overwriting
• --force - Force overwrite without prompting
• --dry-run - Preview installation

Creates:

• Binaries in <prefix>/bin/
• Installation manifest in ~/.syntaxis/manifest.toml

4. deploy.nu - Deploy Configurations

Deploys configuration files to system.

# Deploy to default location
nu scripts/provisioning/deploy.nu --source ./syntaxis-bundle/configs

# Deploy to custom directory
nu scripts/provisioning/deploy.nu --source ./configs --config-dir /etc/syntaxis

# Deploy with backup
nu scripts/provisioning/deploy.nu --backup

# Dry run
nu scripts/provisioning/deploy.nu --dry-run

Options:

• --source <DIR> - Source configs directory (default: configs/)
• --config-dir <PATH> - Target location (default: ~/.config/syntaxis)
• --backup - Backup existing configs before overwriting
• --force - Force overwrite without prompting
• --dry-run - Preview deployment

Creates:

• Config files in <config-dir>/
• Deployment manifest in ~/.syntaxis/deployment.toml

5. provctl.nu - Orchestrator

Coordinates multiple provisioning operations.

# Show help
nu scripts/provisioning/provctl.nu help

# Build a bundle
nu scripts/provisioning/provctl.nu build-bundle --target x86_64-linux

# Extract and install a bundle
nu scripts/provisioning/provctl.nu install-bundle --bundle bundle.tar.gz

# Complete setup: extract + install + deploy
nu scripts/provisioning/provctl.nu full-install --bundle bundle.tar.gz --verify

# Check installation status
nu scripts/provisioning/provctl.nu status

# List available targets
nu scripts/provisioning/provctl.nu list-targets

# Verify bundle integrity
nu scripts/provisioning/provctl.nu verify-bundle --bundle bundle.tar.gz

# Clean staging directories
nu scripts/provisioning/provctl.nu clean

Workflow Examples

Scenario 1: Build and Distribute Bundle (Developer)

# 1. Create bundle for all platforms
nu scripts/provisioning/pack.nu --target x86_64-unknown-linux-gnu --output dist/
nu scripts/provisioning/pack.nu --target aarch64-apple-darwin --output dist/
nu scripts/provisioning/pack.nu --target x86_64-pc-windows-msvc --output dist/

# 2. Bundles are ready in dist/
ls dist/
# → syntaxis-v0.1.0-x86_64-linux.tar.gz
# → syntaxis-v0.1.0-aarch64-darwin.tar.gz
# → syntaxis-v0.1.0-windows.zip

Scenario 2: Install from Bundle (User - Offline)

# 1. User receives bundle: syntaxis-v0.1.0-x86_64-linux.tar.gz

# 2. Option A: Step-by-step (see what's happening)
nu scripts/provisioning/unpack.nu bundle.tar.gz --dest /tmp
nu scripts/provisioning/install.nu --source /tmp/syntaxis-v0.1.0/bin --prefix ~/.local
nu scripts/provisioning/deploy.nu --source /tmp/syntaxis-v0.1.0/configs

# 3. Option B: Orchestrated (one command)
nu scripts/provisioning/provctl.nu full-install --bundle bundle.tar.gz --prefix ~/.local

# 4. Verify installation
nu scripts/provisioning/provctl.nu status
syntaxis-cli --version

Scenario 3: Deploy to Team Infrastructure

# On CI/CD machine (build server)
nu scripts/provisioning/pack.nu --target x86_64-linux --output releases/

# On target machines (multiple VMs/servers)
for machine in server1 server2 server3; do
    scp releases/syntaxis-v0.1.0-x86_64-linux.tar.gz $machine:/tmp/
    ssh $machine "nu provisioning/provctl.nu full-install --bundle /tmp/bundle.tar.gz"
done

Scenario 4: Upgrade Existing Installation

# User already has syntaxis installed

# 1. Get new bundle
curl -O https://releases.example.com/syntaxis-v0.2.0-x86_64-linux.tar.gz

# 2. Install with backup
nu scripts/provisioning/unpack.nu bundle.tar.gz --dest /tmp --verify
nu scripts/provisioning/install.nu --source /tmp/syntaxis-v0.2.0/bin --backup --force
nu scripts/provisioning/deploy.nu --source /tmp/syntaxis-v0.2.0/configs --backup

# 3. Verify new version
syntaxis-cli --version

Common Tasks

List Available Build Targets

nu scripts/provisioning/pack.nu --list-targets

Verify Bundle Integrity

nu scripts/provisioning/provctl.nu verify-bundle --bundle bundle.tar.gz

Check Installation Status

nu scripts/provisioning/provctl.nu status

Create Dry Run

# See what would be packed
nu scripts/provisioning/pack.nu --dry-run

# See what would be installed
nu scripts/provisioning/install.nu --dry-run

# See what would be deployed
nu scripts/provisioning/deploy.nu --dry-run

Find Backup Files

ls -la ~/.local/bin/syntaxis-*.backup.*
ls -la ~/.config/syntaxis/*.backup.*

Clean Staging Directories

nu scripts/provisioning/provctl.nu clean

Manifest Files

Installation tracking is done via TOML manifest files in ~/.syntaxis/:

Installation Manifest

~/.syntaxis/manifest.toml - Created by install.nu:

[installation]
timestamp = "2025-11-17T12:00:00Z"
prefix = "~/.local"
version = "0.1.0"

[[binaries]]
name = "syntaxis-cli"
installed_at = "2025-11-17T12:00:00Z"

[[binaries]]
name = "syntaxis-tui"
installed_at = "2025-11-17T12:00:00Z"

[[binaries]]
name = "syntaxis-api"
installed_at = "2025-11-17T12:00:00Z"

Deployment Manifest

~/.syntaxis/deployment.toml - Created by deploy.nu:

[deployment]
timestamp = "2025-11-17T12:00:05Z"
config_dir = "~/.config/syntaxis"
version = "0.1.0"

[[configs]]
name = "provisioning.toml"
deployed_at = "2025-11-17T12:00:05Z"

Bundle Structure

Bundle Layout

syntaxis-v0.1.0-x86_64-linux/
├── manifest.toml                    # Bundle metadata
├── bin/                             # Compiled binaries
│   ├── syntaxis-cli
│   ├── syntaxis-tui
│   └── syntaxis-api
├── configs/                         # Configuration files
│   ├── provisioning.toml
│   ├── database-default.toml
│   ├── cli/
│   │   └── syntaxis-cli.toml
│   ├── tui/
│   │   └── syntaxis-tui.toml
│   └── api/
│       ├── syntaxis-api.toml
│       └── features/
├── docs/                            # Documentation (offline)
│   ├── README.md
│   ├── docs/installation.md
│   └── CLAUDE.md
└── scripts/                         # Helper scripts (optional)
    └── install.sh                   # Bash installer

Bundle Manifest

manifest.toml inside bundle:

[bundle]
version = "0.1.0"
target = "x86_64-unknown-linux-gnu"
created_at = "2025-11-17T12:00:00Z"
format = "tar.gz"

[artifacts]
binaries = ["syntaxis-cli", "syntaxis-tui", "syntaxis-api"]
configs = ["provisioning.toml", "database-default.toml", ...]
docs = ["README.md", "docs/installation.md", "CLAUDE.md"]

[checksums]
"bin/syntaxis-cli" = "sha256:abcd1234..."
"bin/syntaxis-tui" = "sha256:efgh5678..."
"bin/syntaxis-api" = "sha256:ijkl9012..."

Utility Modules (common/)

platform.nu

Functions for platform detection:

# Detect current OS
detect-os               # → "linux", "macos", "windows"

# Detect current architecture
detect-arch             # → "x86_64", "aarch64"

# Map to cargo target triple
current-target          # → "x86_64-unknown-linux-gnu"

# Check if target is available
is-target-enabled target config_file

# List all available targets
list-targets config_file

# Get supported formats for a target
target-formats target config_file

manifest.nu

Functions for manifest handling:

# Load manifest from file
load manifest_file      # → record

# Create new manifest
create version target format

# Add checksums
add-checksums manifest checksums_map

# Verify checksums
verify-checksums manifest bundle_dir

# Validate manifest structure
validate manifest

# Save manifest to file
save manifest manifest_file

# Show manifest information
info manifest_file

validate.nu

Functions for integrity checking:

# Calculate SHA256 checksum
sha256 file_path

# Generate checksums for directory
generate-checksums source_dir

# Verify single file
verify-file-checksum file_path expected_checksum

# Verify all checksums
verify-checksums checksums_file bundle_dir

# Validate bundle structure
validate-bundle-structure bundle_file

# Validate directory permissions
validate-permissions target_dir --write true

Environment Variables

Scripts respect these environment variables:

• SYNTAXIS_CONFIG_DIR - Configuration directory (default: ~/.config/syntaxis)
• SYNTAXIS_DATA_DIR - Data directory (default: ~/.local/share/syntaxis)
• PROVISIONING_CONFIG - Path to provisioning.toml (default: configs/provisioning.toml)

Error Handling

All scripts provide clear error messages:

# Invalid target
Error: Unsupported combination: os=linux, arch=arm64, libc=custom

# Missing files
Error: Bundle file not found: ./bundle.tar.gz

# Permission denied
Error: Cannot write to /usr/local/bin: insufficient permissions

# Checksum mismatch
Error: Checksum verification failed:
  - bin/syntaxis-cli: mismatch

Future Enhancements

• Convert scripts to Rust binaries for performance
• Homebrew formula support
• Debian/RPM package generation
• Docker image distribution
• Auto-update mechanism
• Telemetry and analytics
• Signed releases (GPG)
• Incremental/delta updates

Troubleshooting

Bundle creation fails

# Check that binaries are built
cargo build --release

# Verify provisioning.toml
nu scripts/provisioning/pack.nu --dry-run

# Check available targets
nu scripts/provisioning/pack.nu --list-targets

Installation fails

# Check permissions
ls -la ~/.local/bin/

# Try with sudo (not recommended)
sudo nu scripts/provisioning/install.nu --prefix /usr/local

# Check manifests
cat ~/.syntaxis/manifest.toml

Configuration deployment fails

# Check source directory
ls -la configs/

# Try dry run
nu scripts/provisioning/deploy.nu --dry-run

# Check target permissions
ls -la ~/.config/

Contributing

To extend the provisioning system:

1. Add new command: Create new-command.nu in scripts/provisioning/
2. Add utilities: Add functions to common/ modules
3. Update config: Add sections to configs/provisioning.toml
4. Update provctl: Add action to provctl.nu
5. Document: Update this README with examples

License

Part of the syntaxis project. See LICENSE for details.

---

Last Updated: 2025-11-17 Status: MVP - Core functionality complete Next: Additional package format support (deb, rpm, brew)