Jesús Pérez c5b510b939 feat: modularize justfile and fix collection/packaging scripts
- Modularize justfile system with dedicated modules:
  * alias.just: Command aliases (h, b, c, s)
  * build.just: Build and cross-compilation commands
  * distro.just: Collection and packaging commands
  * help.just: Comprehensive help system with areas
  * qa.just: Testing and quality assurance
  * tools.just: Development tools and utilities
  * upstream.just: Repository tracking and sync

- Fix platform detection in collect script:
  * Use actual platform names (darwin-arm64) instead of generic "host"
  * Support both "host" argument and auto-detection
  * Filter out .d dependency files from distribution

- Fix packaging script issues:
  * Correct uname command syntax (^uname -m)
  * Fix string interpolation and environment parsing
  * Include plugin binaries in archives (was only packaging metadata)
  * Use proper path join instead of string interpolation
  * Add --force flag to avoid interactive prompts

- Fix justfile absolute paths:
  * Replace relative paths with {{justfile_directory()}} function
  * Enable commands to work from any directory

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-20 19:04:08 +01:00
env
2025-06-27 07:22:35 +01:00
2025-06-27 02:32:07 +01:00

🚀 Nushell Plugins Repository

A comprehensive collection of nushell plugins with automated upstream tracking, dependency management, and development workflows.

📋 Table of Contents

🚀 Quick Start

Prerequisites

# Install required tools
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh  # Rust
cargo install just                                              # Just (optional but recommended)
# Note: Nushell will be auto-installed if missing

Basic Usage

# Clone the repository
git clone <repository-url>
cd nushell-plugins

# Validate nushell version consistency (recommended first step)
just validate-nushell

# Using Just (recommended)
just status              # Show plugin status
just status-mark-locals-ok  # Mark local plugins as OK (recommended)
just upstream-check      # Check for upstream updates
just build              # Build all plugins
just dev-flow           # Complete development workflow

# Using scripts directly (with automatic version checking)
./scripts/run.sh plugin_status.nu
./scripts/run.sh check_upstream_changes.nu
./scripts/run.sh build_all.nu

⚠️ Critical: Version Consistency

This system requires version consistency between your installed nushell and the submodule version. All operations automatically check version consistency and will fail if versions don't match.

  • Auto-fix: just fix-nushell or ./scripts/run.sh --fix --check-only
  • Manual check: just validate-nushell

🎯 Why This Repository?

The Challenge

Developing nushell plugins involves several challenges:

  1. Dependency Management: Nushell evolves rapidly, requiring frequent dependency updates
  2. Upstream Synchronization: Many plugins have upstream repositories that need tracking
  3. Build Consistency: Ensuring all plugins build with the same nushell version
  4. Distribution: Packaging and distributing multiple plugins efficiently

Our Solution

This repository provides:

  • 🔄 Automated Upstream Tracking: Monitors upstream changes and auto-approves nu_* dependency updates
  • 📦 Unified Build System: Builds all plugins with consistent nushell versions
  • 🛡️ Safe Merge Process: Preserves local changes while integrating upstream updates
  • 🚀 Distribution Pipeline: Automated packaging and installation scripts
  • Developer Experience: Just recipes and scripts for common tasks

📦 Plugin Collection

Plugin Status Type Description
nu_plugin_clipboard Tracked Upstream Clipboard operations (copy/paste)
nu_plugin_highlight Tracked Upstream Syntax highlighting for source code
nu_plugin_image 🏠 Local Local Image processing and manipulation
nu_plugin_hashes 🏠 Local Local Hash computation utilities
nu_plugin_desktop_notifications 🏠 Local Local Desktop notification system
nu_plugin_fluent Tracked Upstream Fluent localization framework
nu_plugin_tera Tracked Private Tera templating engine (private repo)
nu_plugin_kcl Tracked Private KCL configuration language (private repo)

Legend

  • Tracked: Has upstream repository with automated tracking
  • 🏠 Local: Developed locally without upstream
  • 🔒 Private: Private repository with tracking (requires authentication)

🏗️ Building and Cross-Compilation

This repository includes a comprehensive cross-platform build system that supports building plugins for multiple platforms from a single machine.

🚀 Quick Build Commands

# Build for your current platform
just build

# Build for all supported platforms
just build-cross-all

# Build for specific platform
just build-cross linux-amd64
just build-cross darwin-arm64
just build-cross windows-amd64

# Complete cross-platform release
just release-cross

🎯 Supported Platforms

Platform Architecture Native Docker Status
Linux AMD64 Full support
Linux ARM64 ⚠️ Docker recommended
macOS Intel 🍎 macOS host only
macOS Apple Silicon 🍎 macOS host only
Windows AMD64 🪟 Windows host or Docker

🐳 Docker Cross-Compilation

For consistent builds across all platforms:

# Build Docker cross-compilation environment
just build-docker-image

# Use Docker for specific targets
just build-docker linux-arm64
just build-docker windows-amd64

# Complete Docker workflow
just docker-flow

📦 Distribution

# Collect binaries for all platforms
just collect-all

# Package with checksums
just pack-checksums

# One-liner: build, collect, and package everything
just release-cross

📖 Comprehensive Guide

For detailed instructions, platform-specific setup, troubleshooting, and advanced topics, see:

➡️ Complete Building and Cross-Compilation Guide

The guide covers:

  • Platform-specific setup and dependencies
  • Native vs Docker cross-compilation
  • Configuration and customization
  • Troubleshooting common issues
  • Performance optimization
  • CI/CD integration
  • Custom target addition

🔄 Development Workflows

Daily Development (Using Just)

# Check what needs attention
just status-attention

# Update from upstream sources
just upstream-check

# Build and test everything
just dev-flow

# Work on specific plugin
just build-plugin nu_plugin_clipboard
just test-plugin nu_plugin_clipboard

Release Workflow

# Complete release pipeline (native)
just release-flow

# Cross-platform release pipeline
just release-flow-cross

# Or step by step:
just build           # Build all plugins (native)
just build-cross-all # Build all plugins (all platforms)
just collect-all     # Collect binaries (all platforms)
just pack-checksums  # Create distribution archives with checksums

Quality Assurance

# Run all quality checks
just quality-flow

# Individual checks
just fmt            # Format code
just lint           # Run clippy
just test           # Run tests
just audit          # Security audit

🎛️ Upstream Tracking System

How It Works

The upstream tracking system automatically:

  1. Fetches latest changes from upstream repositories
  2. Analyzes differences between local and upstream code
  3. Auto-approves changes that only affect nu_* dependencies
  4. Flags other changes for manual review
  5. Preserves local modifications during merges
  6. Excludes specified plugins from tracking operations

Configuration

Plugin Registry (etc/plugin_registry.toml)

[plugins.nu_plugin_example]
upstream_url = "https://github.com/user/nu_plugin_example"
upstream_branch = "main"
auto_ok_on_nu_deps_only = true  # Auto-approve nu_* dependency changes
status = "unknown"              # Current status

Exclusion Configuration (etc/upstream_exclude.toml)

Control which plugins are excluded from upstream operations:

[exclude]
# Plugins to exclude from ALL upstream operations
plugins = [
    "nu_plugin_manual_only",
    "nu_plugin_deprecated"
]

[exclude.check]
# Exclude from checking only (can still be manually merged)
plugins = [
    "nu_plugin_no_auto_check"
]

[exclude.merge]
# Exclude from automatic merging (but can still be checked)
plugins = [
    "nu_plugin_complex_merge"
]

[exclude.patterns]
# Exclude plugins matching patterns
plugins = [
    "nu_plugin_test_*",
    "nu_plugin_experimental_*"
]

Commands

# Check all upstream sources
just upstream-check

# Check specific plugin
just upstream-check-plugin nu_plugin_clipboard

# Preview changes before merging
just upstream-preview nu_plugin_clipboard

# Safely merge upstream changes
just upstream-merge nu_plugin_clipboard

# Merge all pending changes
just upstream-merge-all

# Manage exclusions
just exclude nu_plugin_test add      # Add to exclusion list
just exclude nu_plugin_test remove   # Remove from exclusion list
just exclude-list                    # Show all excluded plugins

Status System

Status Description Action Required
ok Synchronized with upstream None
⚠️ pending Changes detected, needs review Manual review
error Error during sync Investigation
🔥 conflict Merge conflicts Manual resolution
unknown Not yet checked Run upstream check
🏠 local_only No upstream repository None

Status Management

Manual Status Updates

You can manually update plugin status using justfile recipes or scripts:

# Update specific plugin status
just status-update PLUGIN STATUS
just status-update nu_plugin_image ok

# Mark all local development plugins as OK (recommended)
just status-mark-locals-ok

# Using scripts directly
./scripts/run.sh plugin_status.nu update nu_plugin_hashes ok

Common Status Management Tasks

# Check current status
just status                    # Full dashboard
just status-summary           # Quick summary only
just status-attention         # Only plugins needing attention

# Mark local plugins as reviewed/OK
just status-mark-locals-ok     # Marks nu_plugin_image, nu_plugin_hashes, nu_plugin_desktop_notifications as OK

# Individual updates
just status-update nu_plugin_image ok
just status-update nu_plugin_highlight pending

When to Update Status

  • Local plugins (nu_plugin_image, nu_plugin_hashes, nu_plugin_desktop_notifications): Mark as ok after reviewing local changes
  • Upstream plugins: Status is automatically managed by just upstream-check
  • Manual override: Use status-update when you need to override automatic status detection

🛠️ Scripts and Tools

Consolidated Script System

The repository now uses a unified script system with automatic nushell detection and mandatory version consistency checking.

Directory Structure

scripts/
├── run.sh                     # Universal script runner with version checking
├── check_version.nu           # Version consistency validator
├── lib/
│   └── cargo_toml_diff.nu     # Cargo.toml analysis library
├── check_upstream_changes.nu  # Upstream monitoring
├── plugin_status.nu           # Status dashboard
├── safe_merge_upstream.nu     # Safe merge operations
├── build_all.nu              # Build all plugins
├── collect_install.nu        # Collection for distribution
├── pack_dist.nu              # Distribution packaging
├── make_plugin.nu            # Plugin template generator
├── update_nu_versions.nu     # Dependency version management
└── sh/                       # Essential shell scripts only
    ├── update_nushell.sh      # Nushell installer/updater
    ├── update_nu_versions.sh  # Dependency version manager
    ├── test_single.sh         # Testing utilities
    └── test_update.sh         # Testing utilities
etc/
├── plugin_registry.toml       # Central plugin configuration
└── upstream_exclude.toml      # Upstream exclusion configuration

Using the Universal Script Runner

All nushell scripts should be run through the universal wrapper for automatic version checking:

# Universal wrapper with automatic version checking
./scripts/run.sh <script_name> [args...]

# Status and information
./scripts/run.sh plugin_status.nu           # Show dashboard
./scripts/run.sh plugin_status.nu --all     # Include local plugins
./scripts/run.sh plugin_status.nu summary   # Quick summary

# Upstream tracking
./scripts/run.sh check_upstream_changes.nu                    # Check all
./scripts/run.sh check_upstream_changes.nu --plugin highlight # Check one
./scripts/run.sh safe_merge_upstream.nu highlight             # Merge one
./scripts/run.sh safe_merge_upstream.nu --all                 # Merge all pending

# Development
./scripts/run.sh build_all.nu              # Build all plugins
./scripts/run.sh update_nu_versions.nu     # Update nu dependencies

# Version management
./scripts/run.sh --check-only               # Only check version consistency
./scripts/run.sh --fix --check-only        # Auto-fix version mismatches

Direct Nushell Script Usage (Advanced)

⚠️ Warning: Direct usage skips the universal wrapper's version checking - not recommended

# Run from repository root (version checking included in each script)
nu scripts/plugin_status.nu
nu scripts/check_upstream_changes.nu --plugin highlight
nu scripts/build_all.nu --verbose

Version Consistency System

Every script automatically validates that your system nushell version matches the submodule version:

  • Automatic detection: Missing nushell is auto-installed
  • Version validation: System vs submodule version comparison
  • Auto-fix capability: --fix flag resolves version mismatches
  • Clear error messages: Detailed guidance when issues occur

📁 File Structure

nushell-plugins/
├── README.md                   # This file
├── justfile                   # Just task runner recipes
├── env                        # Environment configuration
├── LICENSE                    # License file
├── etc/                       # Configuration files
│   ├── plugin_registry.toml   # Plugin tracking configuration
│   └── upstream_exclude.toml  # Upstream exclusion configuration
├── scripts/                   # All automation scripts
│   ├── run.sh                 # Universal script runner
│   ├── check_version.nu       # Version consistency validator
│   ├── *.nu                   # Nushell script implementations
│   ├── lib/                   # Shared libraries
│   └── sh/                    # Essential shell scripts
├── generate/                  # Plugin templates
│   └── nu_plugin_template/    # Base template for new plugins
├── distribution/              # Build output directory
├── bin_archives/              # Distribution archives
├── nushell/                   # Nushell submodule (for dependency versions)
├── nu_plugin_*/               # Individual plugin directories
└── docs/                      # Documentation

📥 Installation Methods

git clone <repository-url>
cd nushell-plugins
just build                    # Build all plugins
just collect                  # Prepare for installation

# Install specific plugins
cd distribution
nu install_nu_plugins.nu --plugins [nu_plugin_clipboard, nu_plugin_highlight]

Method 2: Pre-built Archive

# Download and extract archive
wget https://releases.../linux-amd64-nushell-plugins.tar.gz
tar -xzf linux-amd64-nushell-plugins.tar.gz
cd nushell-plugins

# Install all plugins
nu install_nu_plugins.nu

# Or install specific plugins
nu install_nu_plugins.nu --plugins [nu_plugin_clipboard]

Method 3: Individual Plugin Installation

# Build and install single plugin
cd nu_plugin_clipboard
cargo build --release
plugin add target/release/nu_plugin_clipboard

Method 4: Using Cargo

# For plugins published to crates.io
cargo install nu_plugin_clipboard
plugin add ~/.cargo/bin/nu_plugin_clipboard

🎮 Using Just Recipes

Just provides convenient shortcuts for all common tasks with automatic version validation:

Version Validation Commands (New!)

just validate-nushell         # Check nushell version consistency
just fix-nushell             # Auto-fix version mismatches

Basic Commands

just                          # Show all available recipes
just status                   # Plugin status dashboard
just build                    # Build all plugins
just test                     # Test all plugins
just clean                    # Clean build artifacts

Upstream Management

just upstream-check           # Check for upstream updates
just upstream-merge highlight # Merge specific plugin
just upstream-merge-all       # Merge all pending

Development Workflows

All workflows now include mandatory version validation as the first step:

just dev-flow                 # Validate → Check → Build → Test → Status
just release-flow             # Validate → Build → Collect → Package
just quality-flow             # Validate → Format → Lint → Test
just update-flow              # Update → Fix Version → Update versions → Check upstream

Plugin-Specific Operations

just build-plugin clipboard   # Build specific plugin
just test-plugin clipboard    # Test specific plugin
just install-plugin clipboard # Install plugin locally

Advanced Features

just interactive              # Interactive plugin selection (requires fzf)
just watch clipboard          # Watch for changes and rebuild (requires entr)
just validate                 # Validate setup and dependencies

FAQ

Q: Why do I get "Version mismatch detected" errors?

A: This system requires your installed nushell version to match the submodule version for consistency. All operations automatically check version consistency and will fail if versions don't match.

Solutions:

  • Auto-fix: just fix-nushell or ./scripts/run.sh --fix --check-only
  • Manual check: just validate-nushell
  • Skip (not recommended): ./scripts/run.sh --no-version-check script.nu

Q: What happened to the bash wrapper scripts?

A: The repository has been consolidated to eliminate script duplication. The old bash wrappers in scripts/sh/ have been removed in favor of:

  • Universal wrapper: ./scripts/run.sh with automatic version checking
  • Direct nu scripts: All scripts moved from scripts/nu/ to scripts/
  • Just recipes: Updated to use the new system

Q: How does the new script system work?

A: The new system provides:

  1. Universal wrapper (scripts/run.sh) with automatic nushell detection and version validation
  2. Consolidated scripts - all nu scripts in scripts/ directory
  3. Mandatory version checking - every operation validates version consistency
  4. Auto-installation - missing nushell is automatically installed

Q: How does automatic upstream tracking work?

A: The system fetches changes from upstream repositories and analyzes them. If only nu_* dependencies changed (like nu-plugin, nu-protocol), it automatically marks the plugin as "OK". Other changes are flagged for manual review. You can exclude specific plugins from tracking using the etc/upstream_exclude.toml file.

Q: What happens to my local changes during upstream merges?

A: Local changes are preserved. The system:

  1. Creates backup branches before any merge
  2. Applies upstream changes in a temporary branch
  3. Restores your local nu_* dependency versions
  4. Tests compilation before applying changes
  5. Rolls back on any failure

Q: Can I disable automatic upstream tracking for a plugin?

A: Yes, set auto_ok_on_nu_deps_only = false in etc/plugin_registry.toml for that plugin.

Q: How do I add a new plugin to the repository?

A: Use the template generator:

just make-plugin nu_plugin_myfeature
# or
nu scripts/nu/make_plugin.nu nu_plugin_myfeature

Q: What if upstream tracking fails?

A: Check the plugin status with just status. Failed plugins show error details. Common issues:

  • Network connectivity to upstream repository
  • Authentication for private repositories
  • Merge conflicts requiring manual resolution

Q: How do I update nushell dependency versions?

A: Use the version updater:

just update-nu-versions       # Update all plugins
just list-nu-versions         # Show current versions

Q: Can I use this with my own private plugins?

A: Yes! Add your private repository URLs to etc/plugin_registry.toml. The system supports SSH URLs for private repositories.

Q: How do I distribute plugins to other systems?

A: Use the distribution pipeline:

just build                    # Build all plugins
just collect                  # Collect binaries
just pack                     # Create archive

The resulting archive contains all binaries and installation scripts.

Q: What's the difference between shell scripts and nushell scripts?

A:

  • Nushell scripts (scripts/nu/): Primary implementation with full features
  • Shell scripts (scripts/sh/): Wrappers for compatibility with non-nushell environments
  • Both provide the same functionality, use whichever fits your environment

Q: How do I contribute a new plugin?

A:

  1. Create the plugin: just make-plugin nu_plugin_yourname
  2. Implement your functionality
  3. Add upstream URL to etc/plugin_registry.toml if it has one
  4. Test: just build-plugin nu_plugin_yourname
  5. Submit a pull request

Q: How do I exclude plugins from upstream tracking?

A: Edit etc/upstream_exclude.toml to exclude plugins from specific operations:

  • Add to [exclude] section to exclude from all operations
  • Add to [exclude.check] to skip automatic checks
  • Add to [exclude.merge] to prevent automatic merging
  • Use patterns like nu_plugin_test_* to exclude multiple plugins

You can also use Just commands:

just exclude nu_plugin_test add      # Add to exclusion
just exclude nu_plugin_test remove   # Remove from exclusion
just exclude-list                    # Show excluded plugins

Q: Can I use this system for other Rust projects?

A: The upstream tracking and dependency management concepts can be adapted, but this system is specifically designed for nushell plugins with nu_* dependencies.

🤝 Contributing

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/amazing-feature
  3. Make your changes
  4. Test thoroughly: just quality-flow
  5. Commit your changes: git commit -m 'Add amazing feature'
  6. Push to the branch: git push origin feature/amazing-feature
  7. Open a pull request

Development Guidelines

  • Follow existing code style and patterns
  • Add tests for new functionality
  • Update documentation for user-facing changes
  • Use just quality-flow before submitting
  • All scripts should work from repository root

Adding New Plugins

  • Use the template generator: just make-plugin nu_plugin_name
  • Follow nushell plugin conventions
  • Add comprehensive documentation
  • Include examples and tests

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

  • Nushell Team for the amazing shell and plugin system
  • Plugin authors for their contributions to the nushell ecosystem
  • Contributors to this repository

Happy Plugin Development! 🎉

Description
Nushell plugins collection for Provisioning project
Readme MIT 16 MiB
Languages
Nushell 37.2%
Rust 29.9%
Shell 10.6%
Just 10.3%
PowerShell 8.2%
Other 3.8%