jesus/nushell-plugins
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
nushell-plugins/scripts
History
Jesús Pérez 2f6089caaf # Summary 
feat: bootstrap installer with robust archive extraction, version mismatch handling, and improved PATH messaging

## Detailed Description

This commit implements a production-ready bootstrap installer with comprehensive error handling, version-agnostic archive extraction, and clear user messaging. All improvements follow DRY principles using symlink-based architecture for single-source-of-truth maintenance.

## Major Changes (Session 2025-10-19)

### 0. Bootstrap Installer & Distribution Improvements (NEW)

#### 0a. Install Script Architecture - DRY Design

**Issue**: Code duplication across installation paths
- `./install.sh`, `./scripts/templates/install.sh`, `installers/bootstrap/install.sh` were separate copies
- Changes required updating multiple files
- Risk of divergence and inconsistency

**Solution**: Implemented symlink-based DRY architecture
- Single source of truth: `installers/bootstrap/install.sh` (1,247 lines)
- Symlinks created:
  - `./install.sh` → `installers/bootstrap/install.sh`
  - `./scripts/templates/install.sh` → `installers/bootstrap/install.sh`
- All changes automatically propagate through symlinks
- No code duplication

**Impact**:
- ✅ Single maintenance point for all install scripts
- ✅ Consistent behavior across all paths
- ✅ Reduced maintenance burden
- ✅ No divergence risk

#### 0b. Archive Extraction - Version-Agnostic Binary Detection

**Issue**: "No Nushell binary found in extracted archive"
- Script failed when extracting from `nushell-full-0.108.0-darwin-arm64.tar.gz`
- Error occurred even though binaries were present at `nushell-full-0.108.0/bin/nu`

**Root Cause**: `find` command returning parent directory itself in results
```bash
# ❌ OLD - find returns parent directory first
local extracted=$(find "$extract_dir" -maxdepth 1 -type d -name "nushell-*" | head -1)
# Returns: /tmp/nushell-install-22919/nushell-extract (parent!)
# Should return: /tmp/nushell-install-22919/nushell-extract/nushell-full-0.108.0 (subdirectory)
```

**Solution**: Added `-not -path` to exclude starting directory
```bash
# ✅ NEW - exclude parent directory
local extracted=$(find "$extract_dir" -maxdepth 1 -type d -name "nushell-*" -not -path "$extract_dir" | head -1)
# Now correctly returns the subdirectory
```

**Additional Improvements**:
- 4-level fallback detection strategy:
  1. Check `$extracted/bin/nu` (subdirectory structure)
  2. Check `$extracted/nu` (flat structure)
  3. Fallback search for any `nushell-*` subdirectory with `bin/nu`
  4. Last resort recursive search for any executable named `nu`
- Validates binaries exist before using them
- Clear error reporting with all search locations

**Version-Agnostic**:
- Uses `nushell-*` pattern (not hardcoded version numbers)
- Works with any Nushell version: 0.107, 0.108, 0.109, etc.
- Supports both `.tar.gz` and `.zip` archive formats

**Impact**:
- ✅ Archive extraction works reliably
- ✅ Works with any Nushell version
- ✅ Clear error messages guide users
- ✅ Multiple archive structure support

#### 0c. Plugin Registration - Version Mismatch Handling

**Issue**: Plugin registration failed with version incompatibility errors
```
Error: nu:🐚:plugin_failed_to_load
× Plugin `polars` is compiled for nushell version 0.107.1, which is not compatible with version 0.108.0
```

**Solution**: Implemented intelligent error classification
- Captures both stdout and stderr from plugin add commands
- Detects version incompatibility: "is not compatible with version" or "is compiled for nushell version"
- Classifies errors into three categories:
  1. **Success**: Plugin registered successfully ✅
  2. **Incompatible**: Version mismatch (skipped gracefully) ⚠️
  3. **Failed**: Other registration failures ❌
- Reports summary with counts of each category
- Installation continues successfully even with version mismatches

**New Error Reporting**:
```
✅ Registered nu_plugin_auth
⚠️  Skipping nu_plugin_polars: Version mismatch (built for different Nushell version)
✅ Successfully registered 13 plugins
⚠️  Skipped 1 incompatible plugins (version mismatch):
  - nu_plugin_polars
```

**Impact**:
- ✅ No installation failures due to version mismatches
- ✅ Users informed of incompatible plugins
- ✅ Clear distinction between error types
- ✅ Installation completes successfully

#### 0d. Shell Configuration PATH Update - Clear Messaging

**Issue**: Confusing PATH update messages
- User sees: "PATH already updated" for all files
- Then sees: "No shell configuration files were updated" warning
- Then sees: "Please manually add to your PATH" error
- **Problem**: Contradictory messages when PATH is already configured everywhere

**Root Cause**: Script conflated two states
- State 1: "Was PATH found in files?" (skips updating if found)
- State 2: "Did we add PATH to any file?" (used for messaging)
- Both states ignored means no update was made, but PATH might already exist

**Solution**: Track two separate states
```bash
local updated=false        # Was PATH ADDED to any file?
local path_found=false     # Was PATH FOUND in any file?

# In loop:
if grep -q "$install_dir" "$config_file"; then
    path_found=true        # Found it! Mark as true
    continue
fi

# After loop:
if [ "$updated" = "true" ]; then
    log_success "Shell configuration updated"
elif [ "$path_found" = "true" ]; then
    log_success "PATH is already configured in your shell configuration files"
else
    log_warn "Could not find or update shell configuration"
fi
```

**New Clear Messages**:
- ✅ "PATH is already configured in your shell configuration files" (when found everywhere)
- ✅ "Shell configuration updated" (when just added)
- ⚠️ "Could not find or update shell configuration" (when file missing)

**Impact**:
- ✅ Non-contradictory messages
- ✅ Users understand what happened
- ✅ No false warnings when PATH already configured
- ✅ Clear guidance when manual action needed

#### 0e. Installation Features

**`--source-path` Option** (Local Installation):
- Install from local archive: `./install.sh --source-path archive.tar.gz`
- Install from local directory: `./install.sh --source-path /path/to/binaries`
- Default behavior: `./install.sh --source-path` uses `./bin_archives`
- Works with custom `--install-dir` paths
- No download needed, offline installation support

**`--uninstall` with Configuration Management**:
- Prompts user: "Remove ~/.config/nushell? [y/N]"
- Removes all installed binaries and plugins
- Preserves user choice for configuration
- Clean uninstall before fresh reinstall

#### 0f. Documentation Updates

**CLAUDE.md**:
- Added "Install Script Architecture (DRY Design)" section
- Documents source of truth and symlink structure
- Explains `--source-path` feature
- Shows version-agnostic archive detection
- Lists DRY architecture benefits

**README.md**:
- Added "Install Script Architecture (DRY Design)" subsection
- Shows symlink structure with arrows
- Provides `--source-path` usage examples
- Explains version-agnostic detection
- "How DRY Works" 3-step explanation

#### 0g. Files Modified

Core Changes:
- `installers/bootstrap/install.sh` (+3 lines for PATH messaging fix)
  - Archive extraction fix with `-not -path`
  - Plugin registration error classification
  - Clear PATH update messaging
  - Total: 1,247 lines (unified)

Auto-Updated via Symlinks:
- `./install.sh` - Auto-updated (1,247 lines)
- `./scripts/templates/install.sh` - Auto-updated (1,247 lines)

Documentation:
- `CLAUDE.md` - Added install architecture section
- `README.md` - Added install architecture subsection
- `CHANGELOG.md` - Added comprehensive entry (+100 lines)

#### 0h. Testing & Verification

All scenarios tested and verified:
- [x] Archive extraction works with version-agnostic detection
- [x] Installation to `~/.local` successful (16 binaries)
- [x] Installation to `~/.local/bin` successful (21 plugins loaded)
- [x] Plugin registration handles version mismatches gracefully
- [x] PATH messaging is clear and non-contradictory
- [x] Clean uninstall followed by fresh reinstall works perfectly

#### 0i. Impact

User-Facing Benefits:
- ✅ Users can install from any version of nushell-full archive
- ✅ Version mismatch plugins skipped without breaking installation
- ✅ Clear, honest error messages
- ✅ Non-confusing PATH update messages
- ✅ Offline installation support via `--source-path`
- ✅ Clean uninstall/reinstall workflow

Developer Benefits:
- ✅ Single source of truth eliminates code duplication
- ✅ Changes propagate automatically through symlinks
- ✅ Reduced maintenance burden
- ✅ Consistent behavior across all paths
- ✅ Production-ready installation process

---

### 1. Help System Integration (New Feature)

**Issue**: Version-update module recipes were not discoverable
- Not shown in `just help modules`
- Not referenced in `just help`
- Not included in help navigation system
- Users had to manually run `just --list` to find update commands

**Solution**:
- Added version-update module to all help outputs
- Updated `justfiles/help.just` to document all 30+ version-update recipes
- Created new `just commands` recipe as discoverable alias for `just --list`
- Integrated version-update into help-all workflow

**Impact**:
- Version-update commands now fully discoverable via help system
- Users can find update commands with: `just help modules`, `just help`, `just commands`
- Improved overall help system navigation

**Files Modified**:
- `justfiles/help.just` (+23 lines)
  - Added version-update module to help sections
  - Added to modules list
  - Added to help-all workflow
  - New `commands` recipe showing all recipes by group

### 2. Build Process Fixes (Phase 3: Bin Archives)

#### 2a. Plugin Archive Collection Bug

**Issue**: "No plugins found to package" warning in Phase 3
- Collected 26 plugin binaries but reported 0
- Archive creation skipped because count was wrong

**Root Cause**: `each` command returns null, so `| length` returned 0
```nushell
# ❌ OLD - each returns null
let plugin_count = (ls nu_plugin_*/target/release/nu_plugin_* | each {|p|
    cp $p.name $"($temp_dir)/"
} | length)  # Returns 0!
```

**Solution**: Separated counting from copying with proper filtering
```nushell
# ✅ NEW - count before operations
let plugins_to_copy = (ls nu_plugin_*/target/release/nu_plugin_* | where type == "file")
let plugin_count = ($plugins_to_copy | length)
```

**Impact**:
- Now correctly collects and reports 26 plugins
- Filters out .d dependency files automatically
- Warning eliminated

#### 2b. Tar Archive Path Handling

**Issue**: Tar command failing silently with relative paths in subshell
- `cd $temp_dir` changes context unpredictably
- Relative path `../$archive_name` fails in subshell
- Archive file not created despite exit code 0

**Root Cause**: Shell context and relative path issues in Nushell `do` block

**Solution**: Used `tar -C` with absolute paths instead of `cd`
```nushell
# ❌ OLD - unreliable context switching
do {
    cd $temp_dir
    tar -czf ../$archive_name .
}

# ✅ NEW - absolute paths, no context switching
tar -C $temp_dir -czf $archive_path .
```

**Additional Improvements**:
- Absolute path construction using `pwd | path join`
- Better error diagnostics with exit code and stderr output
- File verification after creation

**Impact**:
- Tar archives now created successfully
- Robust path handling across platforms
- Clear error messages for debugging

#### 2c. File Size Calculation Type Error

**Issue**: Runtime error when calculating archive size
```
Error: The '/' operator does not work on values of type 'list<filesize>'
```

**Root Cause**: `ls` returns list of records, so `.size` was a list
```nushell
# ❌ OLD - returns list<filesize>
(ls $archive_path).size / 1024 / 1024

# ✅ NEW - returns filesize
(ls $archive_path | get 0.size) / 1024 / 1024
```

**Impact**:
- Proper file size calculation in MB
- No more type errors

**Files Modified**:
- `scripts/create_full_distribution.nu` (+58 lines, refactored plugin collection)
  - Fixed plugin counting logic
  - Improved path handling with absolute paths
  - Enhanced error diagnostics

### 3. Plugin Rebuild Optimization

**Issue**: All plugins marked for rebuild even when dependencies unchanged
- Step 4 (`update_all_plugins.nu`) touched all Cargo.toml files at 01:00:32
- Step 5 saw all files as "newer" than binaries
- Marked ALL plugins for rebuild, though cargo only rebuilt changed ones

**Root Cause**: Script always saved files, even when no changes made
```nushell
# ❌ OLD - always saves, touching file timestamp
$updated_content | to toml | save -f $cargo_toml
```

**Solution**: Only save if content actually changed
```nushell
# ✅ NEW - compare before writing
let original_toml = $content | to toml
let new_toml = $updated_content | to toml

if $original_toml != $new_toml {
    $updated_content | to toml | save -f $cargo_toml
}
```

**Impact**:
- Unchanged files preserve original timestamps
- Only plugins with actual dependency changes are rebuilt
- Efficient rebuild process with accurate file modification detection

**Files Modified**:
- `scripts/update_all_plugins.nu` (+12 lines, added content comparison)
  - Only touches files with real changes
  - Preserves timestamps for efficiency
  - Clearer logic and comments

### 4. Documentation

**Files Modified**:
- `CHANGELOG.md` (+56 lines)
  - Added comprehensive 2025-10-19 entry
  - Documented all fixes with root causes
  - Listed files modified and impact summary

## Technical Details

### Nushell Patterns Used

1. **Proper List Handling**:
   - `ls` returns list of records, access with `| get 0.size`
   - Filter with `where type == "file"` to exclude metadata

2. **Absolute Path Construction**:
   - `pwd | append "path" | path join` for cross-platform paths
   - Safer than string concatenation with `/`

3. **Content Comparison**:
   - Compare TOML string representation before saving
   - Preserves file timestamps for efficiency

4. **Error Diagnostics**:
   - Capture `stderr` from commands
   - Report exit codes and error messages separately

## Testing

- [x] Help system shows version-update module
- [x] `just commands` displays all recipes by group
- [x] Phase 3 bin archive creation works
- [x] Plugin collection reports correct count (26)
- [x] Tar archives created successfully
- [x] File size calculated correctly
- [x] Plugin rebuild only touches changed files
- [x] CHANGELOG updated with all changes

## Files Changed

```
38 files changed, 2721 insertions(+), 2548 deletions(-)

Core Changes:
- justfiles/help.just                  (+23)  Help system integration
- scripts/create_full_distribution.nu  (+58)  Build process fixes
- scripts/update_all_plugins.nu        (+12)  Rebuild optimization
- CHANGELOG.md                         (+56)  Documentation

Dependency Updates:
- All plugin Cargo.toml and Cargo.lock files (version consistency)
```

## Breaking Changes

None. These are bug fixes and optimizations that maintain backward compatibility.

## Migration Notes

No migration needed. Improvements are transparent to users.

## Related Issues

- Help system discoverability
- Build process Phase 3 failures
- Unnecessary plugin rebuilds
- Build process reliability

## Checklist

- [x] Changes follow Rust/Nushell idioms
- [x] Code is well-commented
- [x] Error handling is comprehensive
- [x] Documentation is updated
- [x] All changes tested
- [x] No breaking changes introduced
2025-10-19 02:39:31 +01:00
..
lib
# Summary
2025-10-19 01:17:13 +01:00
templates
# Summary
2025-10-19 02:39:31 +01:00
analyze_nushell_features.nu
feat: Add ARGUMENTS documentation and interactive update mode
2025-10-19 00:05:16 +01:00
audit_crate_dependencies.nu
# Summary
2025-10-19 01:17:13 +01:00
build_all.nu
feat: major repository modernization and tracking cleanup
2025-09-20 19:02:28 +01:00
build_cross.nu
feat: modularize justfile and fix collection/packaging scripts
2025-09-20 19:04:08 +01:00
build_docker_cross.nu
feat: modularize justfile and fix collection/packaging scripts
2025-09-20 19:04:08 +01:00
build_nushell.nu
chore: fix full collect and build
2025-09-24 21:28:50 +01:00
check_upstream_changes.nu
feat: major repository modernization and tracking cleanup
2025-09-20 15:18:58 +01:00
check_version.nu
# Summary
2025-10-19 01:17:13 +01:00
collect_full_binaries.nu
# Summary
2025-10-19 01:17:13 +01:00
collect_install.nu
chore: use script to register plugins in nushell
2025-09-20 20:14:02 +01:00
complete_update.nu
feat: Add ARGUMENTS documentation and interactive update mode
2025-10-19 00:05:16 +01:00
create_distribution_packages.nu
# Summary
2025-10-19 02:39:31 +01:00
create_full_distribution.nu
# Summary
2025-10-19 02:39:31 +01:00
detect_breaking_changes.nu
# Summary
2025-10-19 01:17:13 +01:00
download_nushell.nu
# Summary
2025-10-19 01:17:13 +01:00
install_full_nushell.nu
feat: Add complete Nushell full distribution system
2025-09-24 18:52:07 +01:00
install_plugin.nu
chore: add scripts to register plugins in nushell
2025-09-20 20:13:14 +01:00
make_plugin.nu
feat: major repository modernization and tracking cleanup
2025-09-20 15:18:58 +01:00
pack_dist.nu
feat: major repository modernization and tracking cleanup
2025-09-20 19:02:28 +01:00
plugin_status.nu
feat: major repository modernization and tracking cleanup
2025-09-20 15:18:58 +01:00
README.md
feat: major repository modernization and tracking cleanup
2025-09-20 15:18:58 +01:00
register_plugins.nu
chore: add scripts to register plugins in nushell
2025-09-20 20:13:14 +01:00
run.sh
# Summary
2025-10-19 01:17:13 +01:00
safe_merge_upstream.nu
feat: major repository modernization and tracking cleanup
2025-09-20 15:18:58 +01:00
update_all_plugins.nu
# Summary
2025-10-19 01:17:13 +01:00
update_nu_versions.nu
feat: major repository modernization and tracking cleanup
2025-09-20 15:18:58 +01:00
update_nushell_version.nu
feat: Add ARGUMENTS documentation and interactive update mode
2025-10-19 00:05:16 +01:00
verify_installation.nu
feat: Add complete Nushell full distribution system
2025-09-24 18:52:07 +01:00

README.md

Nushell Plugin Upstream Tracking System

This system provides automated tracking and management of upstream changes for nushell plugins while preserving your local nu_* dependency modifications.

🚀 Quick Start

# Check status of all plugins
./sh/plugin_status.sh

# Check upstream changes for all plugins
./sh/check_upstream.sh

# Check specific plugin only
./sh/check_upstream.sh --plugin nu_plugin_highlight

# Merge all pending plugins
./sh/safe_merge_upstream.sh --all

📋 Supported Operations

✅ For ALL Plugins

# Check upstream changes for ALL plugins
./sh/check_upstream.sh
nu nu/check_upstream_changes.nu

# Show status of ALL plugins
./sh/plugin_status.sh
nu nu/plugin_status.nu

# Merge ALL pending plugins
./sh/safe_merge_upstream.sh --all
nu nu/safe_merge_upstream.nu --all

✅ For ONE Specific Plugin

# Check upstream for ONE plugin
./sh/check_upstream.sh --plugin nu_plugin_highlight
nu nu/check_upstream_changes.nu --plugin nu_plugin_highlight

# Preview merge for ONE plugin
./sh/safe_merge_upstream.sh --preview nu_plugin_highlight
nu nu/safe_merge_upstream.nu --preview nu_plugin_highlight

# Merge ONE plugin
./sh/safe_merge_upstream.sh nu_plugin_highlight
nu nu/safe_merge_upstream.nu nu_plugin_highlight

✅ For SEVERAL Plugins (multiple ways)

Method 1: Run multiple commands

# Check several plugins individually
./sh/check_upstream.sh --plugin nu_plugin_highlight
./sh/check_upstream.sh --plugin nu_plugin_clipboard
./sh/check_upstream.sh --plugin nu_plugin_tera

# Merge several plugins individually
./sh/safe_merge_upstream.sh nu_plugin_highlight
./sh/safe_merge_upstream.sh nu_plugin_clipboard
./sh/safe_merge_upstream.sh nu_plugin_tera

Method 2: Use shell scripting

# Check multiple plugins in a loop
for plugin in nu_plugin_highlight nu_plugin_clipboard nu_plugin_tera; do
    ./sh/check_upstream.sh --plugin $plugin
done

# Merge multiple specific plugins
for plugin in nu_plugin_highlight nu_plugin_clipboard; do
    ./sh/safe_merge_upstream.sh $plugin
done

Method 3: Use the "pending" status to target several

# First, check all to mark them as pending
./sh/check_upstream.sh

# Then merge all pending ones (effectively several plugins)
./sh/safe_merge_upstream.sh --all

🔧 System Components

Core Files

  • plugin_registry.toml - Central configuration and status tracking
  • nu/lib/cargo_toml_diff.nu - Cargo.toml analysis library
  • nu/check_upstream_changes.nu - Main upstream checker with auto-OK logic
  • nu/plugin_status.nu - Status dashboard and management
  • nu/safe_merge_upstream.nu - Safe merge with rollback capability

Shell Wrappers (for compatibility)

  • sh/check_upstream.sh - Shell wrapper for upstream checking
  • sh/plugin_status.sh - Shell wrapper for status dashboard
  • sh/safe_merge_upstream.sh - Shell wrapper for safe merging

🎯 Auto-OK Logic

The system automatically marks plugins as "OK" when:

  • ✅ Only nu_* dependencies changed (nu-plugin, nu-protocol, etc.)
  • ✅ No source code changes (.rs files)
  • ✅ No other dependency changes
  • ✅ Plugin has auto_ok_on_nu_deps_only = true in registry

📊 Plugin Status Types

Status Emoji Description
ok ✅ Synchronized with upstream
pending ⚠️ Changes detected, needs review
error ❌ Error during checking/merging
conflict 🔥 Merge conflicts detected
unknown ❓ Not yet checked
local_only 🏠 No upstream repository

💡 Usage Examples

Daily Workflow

# 1. Check all plugins for updates
./sh/check_upstream.sh

# 2. See what needs attention
./sh/plugin_status.sh attention

# 3. Merge safe updates automatically
./sh/safe_merge_upstream.sh --all

# 4. Review any remaining pending plugins manually
./sh/plugin_status.sh

Specific Plugin Workflow

# 1. Check one plugin
./sh/check_upstream.sh --plugin nu_plugin_highlight

# 2. Preview what would change
./sh/safe_merge_upstream.sh --preview nu_plugin_highlight

# 3. Merge if acceptable
./sh/safe_merge_upstream.sh nu_plugin_highlight

Batch Processing Several Plugins

# Define plugins to process
PLUGINS="nu_plugin_highlight nu_plugin_clipboard nu_plugin_tera"

# Check them all
for plugin in $PLUGINS; do
    echo "Checking $plugin..."
    ./sh/check_upstream.sh --plugin $plugin
done

# Show status
./sh/plugin_status.sh

# Merge pending ones
./sh/safe_merge_upstream.sh --all

🛡️ Safety Features

  • Automatic backups before any merge
  • Temporary branch testing before applying changes
  • Rollback capability if compilation fails
  • Preserves local nu_ versions* during merge
  • Never auto-merges - only auto-marks as OK
  • Test compilation and tests after merge

🔄 Integration with Existing Scripts

The system integrates with your existing update_nu_versions.sh:

# 1. Check upstream first
./sh/check_upstream.sh

# 2. Update local nu versions
./sh/update_nu_versions.sh

# 3. Check status after update
./sh/plugin_status.sh

📁 File Structure

scripts/
├── plugin_registry.toml           # Central registry
├── nu/                           # Nushell scripts
│   ├── lib/
│   │   └── cargo_toml_diff.nu    # Analysis library
│   ├── check_upstream_changes.nu  # Main checker
│   ├── plugin_status.nu          # Status dashboard
│   └── safe_merge_upstream.nu    # Safe merger
└── sh/                           # Shell wrappers
    ├── check_upstream.sh         # Check wrapper
    ├── plugin_status.sh          # Status wrapper
    └── safe_merge_upstream.sh    # Merge wrapper

⚙️ Configuration

Edit plugin_registry.toml to:

  • Add new plugins
  • Configure upstream URLs
  • Enable/disable auto-OK behavior
  • Manage nu_* dependency list

🚨 Troubleshooting

If nushell is not found:

# Install nushell first
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install nu

If plugins fail to merge:

# Check individual plugin status
./sh/plugin_status.sh

# Preview changes first
./sh/safe_merge_upstream.sh --preview PLUGIN_NAME

# Force merge if needed
./sh/safe_merge_upstream.sh --force PLUGIN_NAME

If registry gets corrupted:

# Backup exists at plugin_registry.toml.backup
mv plugin_registry.toml.backup plugin_registry.toml