2b3e574e3d
fix: help system integration, build process optimization, and plugin rebuild efficiency ## Detailed Description This commit addresses critical issues in the help system discoverability, build process robustness, and plugin rebuild efficiency. ### 1. Help System Integration (New Feature) **Issue**: Version-update module recipes were not discoverable - Not shown in `just help modules` - Not referenced in `just help` - Not included in help navigation system - Users had to manually run `just --list` to find update commands **Solution**: - Added version-update module to all help outputs - Updated `justfiles/help.just` to document all 30+ version-update recipes - Created new `just commands` recipe as discoverable alias for `just --list` - Integrated version-update into help-all workflow **Impact**: - Version-update commands now fully discoverable via help system - Users can find update commands with: `just help modules`, `just help`, `just commands` - Improved overall help system navigation **Files Modified**: - `justfiles/help.just` (+23 lines) - Added version-update module to help sections - Added to modules list - Added to help-all workflow - New `commands` recipe showing all recipes by group ### 2. Build Process Fixes (Phase 3: Bin Archives) #### 2a. Plugin Archive Collection Bug **Issue**: "No plugins found to package" warning in Phase 3 - Collected 26 plugin binaries but reported 0 - Archive creation skipped because count was wrong **Root Cause**: `each` command returns null, so `| length` returned 0 ```nushell # ❌ OLD - each returns null let plugin_count = (ls nu_plugin_*/target/release/nu_plugin_* | each {|p| cp $p.name $"($temp_dir)/" } | length) # Returns 0! ``` **Solution**: Separated counting from copying with proper filtering ```nushell # ✅ NEW - count before operations let plugins_to_copy = (ls nu_plugin_*/target/release/nu_plugin_* | where type == "file") let plugin_count = ($plugins_to_copy | length) ``` **Impact**: - Now correctly collects and reports 26 plugins - Filters out .d dependency files automatically - Warning eliminated #### 2b. Tar Archive Path Handling **Issue**: Tar command failing silently with relative paths in subshell - `cd $temp_dir` changes context unpredictably - Relative path `../$archive_name` fails in subshell - Archive file not created despite exit code 0 **Root Cause**: Shell context and relative path issues in Nushell `do` block **Solution**: Used `tar -C` with absolute paths instead of `cd` ```nushell # ❌ OLD - unreliable context switching do { cd $temp_dir tar -czf ../$archive_name . } # ✅ NEW - absolute paths, no context switching tar -C $temp_dir -czf $archive_path . ``` **Additional Improvements**: - Absolute path construction using `pwd | path join` - Better error diagnostics with exit code and stderr output - File verification after creation **Impact**: - Tar archives now created successfully - Robust path handling across platforms - Clear error messages for debugging #### 2c. File Size Calculation Type Error **Issue**: Runtime error when calculating archive size ``` Error: The '/' operator does not work on values of type 'list<filesize>' ``` **Root Cause**: `ls` returns list of records, so `.size` was a list ```nushell # ❌ OLD - returns list<filesize> (ls $archive_path).size / 1024 / 1024 # ✅ NEW - returns filesize (ls $archive_path | get 0.size) / 1024 / 1024 ``` **Impact**: - Proper file size calculation in MB - No more type errors **Files Modified**: - `scripts/create_full_distribution.nu` (+58 lines, refactored plugin collection) - Fixed plugin counting logic - Improved path handling with absolute paths - Enhanced error diagnostics ### 3. Plugin Rebuild Optimization **Issue**: All plugins marked for rebuild even when dependencies unchanged - Step 4 (`update_all_plugins.nu`) touched all Cargo.toml files at 01:00:32 - Step 5 saw all files as "newer" than binaries - Marked ALL plugins for rebuild, though cargo only rebuilt changed ones **Root Cause**: Script always saved files, even when no changes made ```nushell # ❌ OLD - always saves, touching file timestamp $updated_content | to toml | save -f $cargo_toml ``` **Solution**: Only save if content actually changed ```nushell # ✅ NEW - compare before writing let original_toml = $content | to toml let new_toml = $updated_content | to toml if $original_toml != $new_toml { $updated_content | to toml | save -f $cargo_toml } ``` **Impact**: - Unchanged files preserve original timestamps - Only plugins with actual dependency changes are rebuilt - Efficient rebuild process with accurate file modification detection **Files Modified**: - `scripts/update_all_plugins.nu` (+12 lines, added content comparison) - Only touches files with real changes - Preserves timestamps for efficiency - Clearer logic and comments ### 4. Documentation **Files Modified**: - `CHANGELOG.md` (+56 lines) - Added comprehensive 2025-10-19 entry - Documented all fixes with root causes - Listed files modified and impact summary ## Technical Details ### Nushell Patterns Used 1. **Proper List Handling**: - `ls` returns list of records, access with `| get 0.size` - Filter with `where type == "file"` to exclude metadata 2. **Absolute Path Construction**: - `pwd | append "path" | path join` for cross-platform paths - Safer than string concatenation with `/` 3. **Content Comparison**: - Compare TOML string representation before saving - Preserves file timestamps for efficiency 4. **Error Diagnostics**: - Capture `stderr` from commands - Report exit codes and error messages separately ## Testing - [x] Help system shows version-update module - [x] `just commands` displays all recipes by group - [x] Phase 3 bin archive creation works - [x] Plugin collection reports correct count (26) - [x] Tar archives created successfully - [x] File size calculated correctly - [x] Plugin rebuild only touches changed files - [x] CHANGELOG updated with all changes ## Files Changed ``` 38 files changed, 2721 insertions(+), 2548 deletions(-) Core Changes: - justfiles/help.just (+23) Help system integration - scripts/create_full_distribution.nu (+58) Build process fixes - scripts/update_all_plugins.nu (+12) Rebuild optimization - CHANGELOG.md (+56) Documentation Dependency Updates: - All plugin Cargo.toml and Cargo.lock files (version consistency) ``` ## Breaking Changes None. These are bug fixes and optimizations that maintain backward compatibility. ## Migration Notes No migration needed. Improvements are transparent to users. ## Related Issues - Help system discoverability - Build process Phase 3 failures - Unnecessary plugin rebuilds - Build process reliability ## Checklist - [x] Changes follow Rust/Nushell idioms - [x] Code is well-commented - [x] Error handling is comprehensive - [x] Documentation is updated - [x] All changes tested - [x] No breaking changes introduced
Nushell Plugin Upstream Tracking System
This system provides automated tracking and management of upstream changes for nushell plugins while preserving your local nu_* dependency modifications.
🚀 Quick Start
# Check status of all plugins
./sh/plugin_status.sh
# Check upstream changes for all plugins
./sh/check_upstream.sh
# Check specific plugin only
./sh/check_upstream.sh --plugin nu_plugin_highlight
# Merge all pending plugins
./sh/safe_merge_upstream.sh --all
📋 Supported Operations
✅ For ALL Plugins
# Check upstream changes for ALL plugins
./sh/check_upstream.sh
nu nu/check_upstream_changes.nu
# Show status of ALL plugins
./sh/plugin_status.sh
nu nu/plugin_status.nu
# Merge ALL pending plugins
./sh/safe_merge_upstream.sh --all
nu nu/safe_merge_upstream.nu --all
✅ For ONE Specific Plugin
# Check upstream for ONE plugin
./sh/check_upstream.sh --plugin nu_plugin_highlight
nu nu/check_upstream_changes.nu --plugin nu_plugin_highlight
# Preview merge for ONE plugin
./sh/safe_merge_upstream.sh --preview nu_plugin_highlight
nu nu/safe_merge_upstream.nu --preview nu_plugin_highlight
# Merge ONE plugin
./sh/safe_merge_upstream.sh nu_plugin_highlight
nu nu/safe_merge_upstream.nu nu_plugin_highlight
✅ For SEVERAL Plugins (multiple ways)
Method 1: Run multiple commands
# Check several plugins individually
./sh/check_upstream.sh --plugin nu_plugin_highlight
./sh/check_upstream.sh --plugin nu_plugin_clipboard
./sh/check_upstream.sh --plugin nu_plugin_tera
# Merge several plugins individually
./sh/safe_merge_upstream.sh nu_plugin_highlight
./sh/safe_merge_upstream.sh nu_plugin_clipboard
./sh/safe_merge_upstream.sh nu_plugin_tera
Method 2: Use shell scripting
# Check multiple plugins in a loop
for plugin in nu_plugin_highlight nu_plugin_clipboard nu_plugin_tera; do
./sh/check_upstream.sh --plugin $plugin
done
# Merge multiple specific plugins
for plugin in nu_plugin_highlight nu_plugin_clipboard; do
./sh/safe_merge_upstream.sh $plugin
done
Method 3: Use the "pending" status to target several
# First, check all to mark them as pending
./sh/check_upstream.sh
# Then merge all pending ones (effectively several plugins)
./sh/safe_merge_upstream.sh --all
🔧 System Components
Core Files
plugin_registry.toml- Central configuration and status trackingnu/lib/cargo_toml_diff.nu- Cargo.toml analysis librarynu/check_upstream_changes.nu- Main upstream checker with auto-OK logicnu/plugin_status.nu- Status dashboard and managementnu/safe_merge_upstream.nu- Safe merge with rollback capability
Shell Wrappers (for compatibility)
sh/check_upstream.sh- Shell wrapper for upstream checkingsh/plugin_status.sh- Shell wrapper for status dashboardsh/safe_merge_upstream.sh- Shell wrapper for safe merging
🎯 Auto-OK Logic
The system automatically marks plugins as "OK" when:
- ✅ Only nu_* dependencies changed (nu-plugin, nu-protocol, etc.)
- ✅ No source code changes (.rs files)
- ✅ No other dependency changes
- ✅ Plugin has
auto_ok_on_nu_deps_only = truein registry
📊 Plugin Status Types
| Status | Emoji | Description |
|---|---|---|
ok |
✅ | Synchronized with upstream |
pending |
⚠️ | Changes detected, needs review |
error |
❌ | Error during checking/merging |
conflict |
🔥 | Merge conflicts detected |
unknown |
❓ | Not yet checked |
local_only |
🏠 | No upstream repository |
💡 Usage Examples
Daily Workflow
# 1. Check all plugins for updates
./sh/check_upstream.sh
# 2. See what needs attention
./sh/plugin_status.sh attention
# 3. Merge safe updates automatically
./sh/safe_merge_upstream.sh --all
# 4. Review any remaining pending plugins manually
./sh/plugin_status.sh
Specific Plugin Workflow
# 1. Check one plugin
./sh/check_upstream.sh --plugin nu_plugin_highlight
# 2. Preview what would change
./sh/safe_merge_upstream.sh --preview nu_plugin_highlight
# 3. Merge if acceptable
./sh/safe_merge_upstream.sh nu_plugin_highlight
Batch Processing Several Plugins
# Define plugins to process
PLUGINS="nu_plugin_highlight nu_plugin_clipboard nu_plugin_tera"
# Check them all
for plugin in $PLUGINS; do
echo "Checking $plugin..."
./sh/check_upstream.sh --plugin $plugin
done
# Show status
./sh/plugin_status.sh
# Merge pending ones
./sh/safe_merge_upstream.sh --all
🛡️ Safety Features
- Automatic backups before any merge
- Temporary branch testing before applying changes
- Rollback capability if compilation fails
- Preserves local nu_ versions* during merge
- Never auto-merges - only auto-marks as OK
- Test compilation and tests after merge
🔄 Integration with Existing Scripts
The system integrates with your existing update_nu_versions.sh:
# 1. Check upstream first
./sh/check_upstream.sh
# 2. Update local nu versions
./sh/update_nu_versions.sh
# 3. Check status after update
./sh/plugin_status.sh
📁 File Structure
scripts/
├── plugin_registry.toml # Central registry
├── nu/ # Nushell scripts
│ ├── lib/
│ │ └── cargo_toml_diff.nu # Analysis library
│ ├── check_upstream_changes.nu # Main checker
│ ├── plugin_status.nu # Status dashboard
│ └── safe_merge_upstream.nu # Safe merger
└── sh/ # Shell wrappers
├── check_upstream.sh # Check wrapper
├── plugin_status.sh # Status wrapper
└── safe_merge_upstream.sh # Merge wrapper
⚙️ Configuration
Edit plugin_registry.toml to:
- Add new plugins
- Configure upstream URLs
- Enable/disable auto-OK behavior
- Manage nu_* dependency list
🚨 Troubleshooting
If nushell is not found:
# Install nushell first
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install nu
If plugins fail to merge:
# Check individual plugin status
./sh/plugin_status.sh
# Preview changes first
./sh/safe_merge_upstream.sh --preview PLUGIN_NAME
# Force merge if needed
./sh/safe_merge_upstream.sh --force PLUGIN_NAME
If registry gets corrupted:
# Backup exists at plugin_registry.toml.backup
mv plugin_registry.toml.backup plugin_registry.toml