nushell-plugins/updates/108/NUSHELL_UPDATE_AUTOMATION.md

Nushell Update Automation Guide

Version: 1.0 Date: 2025-10-18 Purpose: Automate Nushell version updates for the plugins repository

---

Overview

This guide documents the semi-automated system for updating Nushell versions in the nushell-plugins repository. The system reduces manual work by ~80% while maintaining safety through strategic manual approval checkpoints.

Design Philosophy

• Semi-Automated: Automate repetitive tasks, require human approval for critical decisions
• Safe by Default: Create backups, validate before proceeding, allow rollback
• Comprehensive: Download, analyze, update, build, test, and document
• Idempotent: Can be run multiple times safely
• Observable: Clear logging and status reporting at every step

---

Automation Architecture

System Components

┌─────────────────────────────────────────────────────────────┐
│                    Update Orchestrator                      │
│              (update_nushell_version.nu)                    │
└─────────────────┬───────────────────────────────────────────┘
                  │
    ┌─────────────┼─────────────┬──────────────┬──────────────┐
    │             │             │              │              │
    ▼             ▼             ▼              ▼              ▼
┌─────────┐  ┌─────────┐  ┌─────────┐  ┌──────────┐  ┌─────────┐
│Download │  │Analyze  │  │ Audit   │  │ Detect   │  │ Build   │
│Nushell  │  │Features │  │  Deps   │  │Breaking  │  │Nushell  │
└─────────┘  └─────────┘  └─────────┘  └──────────┘  └─────────┘
     │             │             │            │              │
     └─────────────┴─────────────┴────────────┴──────────────┘
                              │
                              ▼
                    ┌──────────────────┐
                    │ Update Versions  │
                    │  Test Plugins    │
                    │Generate Report   │
                    └──────────────────┘

Script Hierarchy

1. update_nushell_version.nu (Main Orchestrator)

   • Entry point for all updates
   • Coordinates all other scripts
   • Implements manual approval checkpoints
   • Generates comprehensive reports

2. download_nushell.nu (Download Manager)

   • Downloads Nushell tarball from GitHub
   • Verifies extraction and workspace
   • Supports --latest flag

3. analyze_nushell_features.nu (Feature Analyzer)

   • Parses Cargo.toml for available features
   • Validates desired features exist
   • Shows dependency trees
   • Exports JSON analysis

4. audit_crate_dependencies.nu (Dependency Auditor)

   • Scans all plugin dependencies
   • Detects version mismatches
   • Shows dependency matrix
   • Identifies update requirements

5. detect_breaking_changes.nu (Breaking Change Detector)

   • Database of known breaking changes
   • Scans plugin code for affected patterns
   • Generates migration reports
   • Suggests fixes

6. build_nushell.nu (Build Manager)

   • Builds Nushell with selected features
   • Handles workspace builds
   • Manages build artifacts

7. test_plugin_compatibility.nu (Compatibility Tester)

   • Tests plugins against new Nushell version
   • Validates plugin registration
   • Checks command execution
   • Reports compatibility issues

8. validate_code_rules.nu (Code Validator)

   • Validates best_nushell_code.md against actual binary
   • Tests syntax rules
   • Verifies patterns work
   • Documents syntax changes

---

Quick Start

Basic Update

# Update to specific version (recommended)
./scripts/update_nushell_version.nu 0.108.0

# Update to latest release
./scripts/update_nushell_version.nu --latest

# Check current update status
./scripts/update_nushell_version.nu status

Advanced Options

# Skip build step (faster for testing)
./scripts/update_nushell_version.nu 0.108.0 --skip-build

# Auto-approve all steps (use with extreme caution!)
./scripts/update_nushell_version.nu 0.108.0 --auto-approve

# Combination (testing workflow)
./scripts/update_nushell_version.nu 0.108.0 --skip-build --auto-approve

---

Detailed Workflow

Step-by-Step Process

Step 1: Download Nushell Source (Automated)

./scripts/download_nushell.nu 0.108.0 --clean

What It Does:

• Downloads tarball from GitHub tags
• Extracts to ./nushell/ directory
• Verifies workspace structure
• Checks Cargo.toml version

Output:

[INFO] Downloading Nushell 0.108.0 from GitHub...
[INFO] Download URL: https://github.com/nushell/nushell/archive/refs/tags/0.108.0.tar.gz
[SUCCESS] Downloaded: 15.2 MB
[INFO] Extracting tarball...
[SUCCESS] Extracted 43 crates
[INFO] Verifying workspace...
[SUCCESS] Nushell 0.108.0 ready at: ./nushell/

Options:

• --latest: Download latest release automatically
• --clean: Remove existing nushell directory first
• --verify: Verify checksum (future feature)

Step 2: Analyze Features (Automated)

./scripts/analyze_nushell_features.nu --validate --export

What It Does:

• Parses nushell Cargo.toml features section
• Validates desired features exist
• Shows feature dependency tree
• Exports analysis to JSON

Desired Features (configurable in script):

const DESIRED_FEATURES = [
    "mcp",              # Model Context Protocol
    "plugin",           # Plugin system
    "sqlite",           # SQLite support
    "trash-support",    # Trash bin support
    "system-clipboard"  # System clipboard
]

Output:

[INFO] Analyzing Nushell version: 0.108.0
[INFO] Found 37 features

=== Desired Features Analysis ===
[SUCCESS] ✓ mcp
    Dependencies: none
[SUCCESS] ✓ plugin
    Dependencies: nu-plugin, nu-plugin-engine
[SUCCESS] ✓ sqlite
    Dependencies: nu-command/sqlite
...

[SUCCESS] All desired features are valid!
[SUCCESS] Feature analysis exported to: ./tmp/feature_analysis.json

Subcommands:

# Show all available features
./scripts/analyze_nushell_features.nu --show-all

# Show dependency tree for specific feature
./scripts/analyze_nushell_features.nu tree mcp

# Generate build command
./scripts/analyze_nushell_features.nu build-cmd

# Categorize features
./scripts/analyze_nushell_features.nu categorize

Step 3: Audit Dependencies (Automated)

./scripts/audit_crate_dependencies.nu --export

What It Does:

• Scans all plugin Cargo.toml files
• Extracts nu-* dependencies
• Compares versions with nushell core
• Reports mismatches

Output:

[INFO] Nushell Plugin Dependency Audit
[INFO] Found 8 system plugins
[INFO] Found 3 custom plugins

=== Dependency Audit Results ===
Total plugins: 11
Clean: 8
With issues: 3

--- System Plugins ---
  ✅ nu_plugin_query (v0.108.0)
     nu-* dependencies: 3
       • nu-plugin: 0.108.0
       • nu-protocol: 0.108.0 [plugin]
       • nu-engine: 0.108.0
...

--- Custom Plugins ---
  ⚠️ nu_plugin_image (v0.1.0)
     nu-* dependencies: 2
       • nu-plugin: 0.107.1 (path: ../nushell/crates/nu-plugin)
       • nu-protocol: 0.107.1 (path: ../nushell/crates/nu-protocol)

=== Plugins with Version Issues ===
[ERROR] nu_plugin_image
  • nu-plugin: found 0.107.1, expected 0.108.0
  • nu-protocol: found 0.107.1, expected 0.108.0

Subcommands:

# Audit specific plugin
./scripts/audit_crate_dependencies.nu --plugin nu_plugin_image

# Audit only custom plugins
./scripts/audit_crate_dependencies.nu --custom-only

# Show dependency matrix
./scripts/audit_crate_dependencies.nu matrix

Step 4: Detect Breaking Changes (Automated)

./scripts/detect_breaking_changes.nu --scan-plugins --export

What It Does:

• Maintains database of known breaking changes
• Scans plugin source code for affected patterns
• Reports potential issues
• Suggests migration steps

Breaking Change Database:

const BREAKING_CHANGES = {
    "0.108.0": [
        {
            type: "command_rename"
            old_name: "into value"
            new_name: "detect type"
            description: "Command renamed and behavior changed"
            impact: "high"
            migration: "Replace 'into value' with 'detect type'"
        },
        {
            type: "behavior_change"
            component: "stream_error_handling"
            description: "Collecting streams with errors now raises errors"
            impact: "medium"
            migration: "Add explicit error handling when collecting streams"
        }
    ]
}

Output:

[INFO] Detecting breaking changes for 0.108.0...
[INFO] Found 2 breaking changes in database
[INFO] Scanning plugin source code...

=== Breaking Changes Report ===

Breaking Change #1: Command Rename
  Old: into value
  New: detect type
  Impact: high

  Affected Plugins:
    • nu_plugin_image: 2 occurrences in src/main.rs
    • nu_plugin_hashes: 1 occurrence in src/lib.rs

  Migration:
    Replace 'into value' with 'detect type'

Breaking Change #2: Stream Error Handling
  Component: stream_error_handling
  Impact: medium

  Affected Plugins:
    • nu_plugin_image: 5 stream collections need error handling

  Migration:
    Add explicit error handling when collecting streams

[WARN] Breaking changes detected!
[SUCCESS] Report exported to: ./tmp/breaking_changes_report.json

Step 5: Manual Approval Checkpoint #1 ⚠️

The orchestrator pauses and displays:

═══ MANUAL REVIEW REQUIRED ═══
Breaking changes detected. Please review the report above.

Affected plugins: 2
  • nu_plugin_image (3 issues)
  • nu_plugin_hashes (1 issue)

Continue with update? (yes/no):

What to Review:

• Breaking change impact on each plugin
• Number of affected code locations
• Migration complexity
• Risk level

Decision:

• Type yes to continue with updates
• Type no to abort and investigate manually

Step 6: Update Plugin Versions (Semi-Automated)

./scripts/update_nu_versions.nu update

What It Does:

• Updates all plugin Cargo.toml files
• Changes nu-* dependency versions
• Preserves other dependencies
• Creates backup before changes

Output:

[INFO] Current Plugin Versions:

Plugin                  | nu-plugin | nu-protocol | Status
------------------------|-----------|-------------|--------
nu_plugin_image         | 0.107.1   | 0.107.1     | Needs update
nu_plugin_hashes        | 0.107.1   | 0.107.1     | Needs update
nu_plugin_clipboard     | 0.108.0   | 0.108.0     | OK
...

Update all plugin versions? (yes/no): yes

[INFO] Updating plugin versions to 0.108.0...
[SUCCESS] Updated: nu_plugin_image/Cargo.toml
[SUCCESS] Updated: nu_plugin_hashes/Cargo.toml
[SUCCESS] Updated 8 plugins to version 0.108.0

Step 7: Build Nushell (Automated, Optional)

./scripts/build_nushell.nu

What It Does:

• Builds Nushell with selected features
• Builds workspace (includes system plugins)
• Reports build time and artifacts

Build Command Generated:

cd nushell && cargo build --release --workspace \
  --features "mcp,plugin,sqlite,trash-support,system-clipboard,rustls-tls"

Output:

[INFO] Building Nushell 0.108.0 with MCP features...
[WARN] This will take 10-20 minutes...

[INFO] Build started at: 2025-10-18 10:00:00
...
[Cargo build output]
...
[SUCCESS] Build completed at: 2025-10-18 10:15:23
[INFO] Build time: 15m 23s

Artifacts Generated:
  • nushell/target/release/nu (42.3 MB)
  • nushell/target/release/nu_plugin_* (8 plugins)

Step 8: Manual Approval Checkpoint #2 ⚠️

═══ BUILD REVIEW REQUIRED ═══
Please review build results above.

Build status: Success
Build time: 15m 23s
Artifacts: 9 binaries

Proceed with plugin compatibility tests? (yes/no):

What to Review:

• Build completed successfully
• No critical warnings
• Expected artifacts present
• Build time reasonable

Step 9: Test Plugin Compatibility (Automated)

./scripts/test_plugin_compatibility.nu

What It Does:

• Registers each plugin with new nushell
• Tests plugin commands
• Verifies output format
• Reports any failures

Output:

[INFO] Testing plugin compatibility with Nushell 0.108.0...

Testing: nu_plugin_clipboard
  [SUCCESS] Plugin registered
  [SUCCESS] Command 'clipboard' works
  [SUCCESS] Output format correct

Testing: nu_plugin_image
  [SUCCESS] Plugin registered
  [WARN] Command 'to ansi' slow (2.3s)
  [SUCCESS] Output format correct

...

=== Compatibility Summary ===
Total plugins tested: 11
Passed: 10
Failed: 0
Warnings: 1 (performance)

Step 10: Generate Update Report (Automated)

# Generates comprehensive markdown report

Output: ./tmp/update_report_0.108.0.md

Contents:

• Update summary
• Files modified
• Breaking changes encountered
• Test results
• Next steps
• Links to detailed reports

Step 11: Final Manual Approval ⚠️

═══ UPDATE COMPLETE ═══
Review the update report above.

Next steps:
  1. Review changes with: git status
  2. Test the updated plugins
  3. Commit changes when ready

To create a commit:
  git add -A
  git commit -m "chore: update to Nushell 0.108.0"

---

Manual Approval Checkpoints

Checkpoint #1: Breaking Changes Review

When: After detecting breaking changes Purpose: Verify impact is acceptable What to Check:

• Number of breaking changes manageable
• Affected plugins are known
• Migration path is clear
• No unexpected critical changes

Response Options:

• yes: Continue with automatic updates
• no: Stop to manually investigate

Checkpoint #2: Build Review

When: After building Nushell Purpose: Verify build succeeded properly What to Check:

• Build completed without errors
• No suspicious warnings
• All expected binaries created
• Build time is reasonable

Response Options:

• yes: Proceed with plugin tests
• no: Skip plugin tests, review manually

Checkpoint #3: Final Approval

When: After all steps complete Purpose: Review before committing What to Check:

• All tests passed
• Documentation updated
• Changes look correct in git status
• Ready to commit

Response Options:

• Commit changes: git add -A && git commit
• Rollback: git restore .
• Manual fixes: Edit files as needed

---

Configuration

Script Configuration Files

etc/plugin_registry.toml

[plugins.nu_plugin_image]
upstream = "https://github.com/fdncred/nu_plugin_image"
status = "ok"
auto_merge = false

[plugins.nu_plugin_clipboard]
upstream = "https://github.com/FMotalleb/nu_plugin_clipboard"
status = "ok"
auto_merge = true  # Auto-merge nu-* dependency changes

scripts/lib/common_lib.nu

# Shared configuration for all scripts
export const LOG_LEVEL = "INFO"
export const NUSHELL_DIR = "./nushell"
export const TMP_DIR = "./tmp"
export const BACKUP_DIR = "./backups"

# Feature configuration
export const DESIRED_FEATURES = [
    "mcp",
    "plugin",
    "sqlite",
    "trash-support",
    "system-clipboard"
]

Environment Variables

# Override default directories
export NUSHELL_SOURCE_DIR="/custom/path/nushell"
export UPDATE_TMP_DIR="/custom/tmp"

# Control logging
export UPDATE_LOG_LEVEL="DEBUG"  # DEBUG, INFO, WARN, ERROR

# Skip checkpoints (dangerous!)
export UPDATE_AUTO_APPROVE="false"

---

Error Handling

Automatic Recovery

The system implements automatic recovery for common errors:

Download Failures

# Automatically retries with exponential backoff
def download_with_retry [url: string]: nothing -> binary {
    mut attempts = 0
    while $attempts < 3 {
        try {
            return (http get $url)
        } catch {|err|
            $attempts = $attempts + 1
            log_warn $"Download failed, retry ($attempts)/3"
            sleep (2sec * $attempts)
        }
    }
    error make {msg: "Download failed after 3 attempts"}
}

Build Failures

# Cleans and retries once
try {
    cargo build --release
} catch {|err|
    log_warn "Build failed, cleaning and retrying..."
    cargo clean
    cargo build --release
}

Network Errors

# Falls back to git clone if tarball download fails
try {
    download_tarball $version
} catch {
    log_warn "Tarball download failed, trying git clone..."
    git clone --depth 1 --branch $version https://github.com/nushell/nushell
}

Manual Intervention

Some errors require manual intervention:

Merge Conflicts

[ERROR] Merge conflict in nu_plugin_image/Cargo.toml
[INFO] Manual resolution required:
  1. Edit the file to resolve conflicts
  2. Run: git add nu_plugin_image/Cargo.toml
  3. Continue with: ./scripts/update_nushell_version.nu --continue

Missing Dependencies

[ERROR] Missing system dependency: libssl-dev
[INFO] Install with:
  Ubuntu/Debian: sudo apt install libssl-dev
  macOS: brew install openssl
  Then retry the build

---

Logging and Debugging

Log Levels

# Log functions in common_lib.nu
export def log_debug [msg: string]: nothing -> nothing {
    if $env.LOG_LEVEL? == "DEBUG" {
        print $"[(ansi grey)(date now | format date '%H:%M:%S')(ansi reset)] [DEBUG] ($msg)"
    }
}

export def log_info [msg: string]: nothing -> nothing {
    print $"[(ansi blue)(date now | format date '%H:%M:%S')(ansi reset)] [INFO] ($msg)"
}

export def log_success [msg: string]: nothing -> nothing {
    print $"[(ansi green)(date now | format date '%H:%M:%S')(ansi reset)] [SUCCESS] ($msg)"
}

export def log_warn [msg: string]: nothing -> nothing {
    print $"[(ansi yellow)(date now | format date '%H:%M:%S')(ansi reset)] [WARN] ($msg)"
}

export def log_error [msg: string]: nothing -> nothing {
    print $"[(ansi red)(date now | format date '%H:%M:%S')(ansi reset)] [ERROR] ($msg)"
}

Debug Mode

# Enable debug logging
export LOG_LEVEL=DEBUG
./scripts/update_nushell_version.nu 0.108.0

# Enable Nushell tracing
RUST_LOG=debug ./scripts/update_nushell_version.nu 0.108.0

# Save complete log
./scripts/update_nushell_version.nu 0.108.0 2>&1 | tee update.log

Generated Reports

All operations generate detailed reports:

./tmp/
├── feature_analysis.json         # Feature analysis results
├── dependency_audit.json          # Dependency audit results
├── breaking_changes_report.json  # Breaking changes found
├── test_results.json             # Plugin test results
└── update_report_0.108.0.md      # Comprehensive summary

---

Performance Optimization

Parallel Execution

# Analyze and audit in parallel
[
    (do { ./scripts/analyze_nushell_features.nu --export })
    (do { ./scripts/audit_crate_dependencies.nu --export })
] | par-each {|cmd| $cmd }

Caching

# Cache GitHub API responses
def get_latest_version_cached []: nothing -> string {
    let cache_file = "./tmp/latest_version.txt"
    let cache_age = (date now) - (ls $cache_file | get modified | first)

    if $cache_age < 1hr {
        open $cache_file
    } else {
        let version = get_latest_from_github
        $version | save -f $cache_file
        $version
    }
}

Incremental Builds

# Only rebuild changed plugins
def build_changed_plugins []: nothing -> list {
    git diff --name-only HEAD~1
    | lines
    | where {|f| $f | str starts-with "nu_plugin_"}
    | each {|plugin|
        cd $plugin
        cargo build --release
    }
}

---

Testing the Automation

Unit Tests

# Test download script
def test_download []: nothing -> nothing {
    ./scripts/download_nushell.nu 0.108.0 --clean
    assert (("./nushell/Cargo.toml" | path exists) == true)
    assert ((open ./nushell/Cargo.toml | get package.version) == "0.108.0")
}

# Test feature analysis
def test_analyze []: nothing -> nothing {
    let result = (./scripts/analyze_nushell_features.nu --validate | complete)
    assert ($result.exit_code == 0)
    assert (("./tmp/feature_analysis.json" | path exists) == true)
}

Integration Tests

# Test full workflow with skip-build
./scripts/update_nushell_version.nu 0.108.0 --skip-build --auto-approve

# Verify results
test -f ./nushell/Cargo.toml
test -f ./tmp/feature_analysis.json
test -f ./tmp/dependency_audit.json
test -f ./tmp/breaking_changes_report.json

Dry Run Mode

# Future feature: test without making changes
./scripts/update_nushell_version.nu 0.108.0 --dry-run

---

Future Enhancements

Planned Features

1. Automated Rollback

   ./scripts/rollback_version.nu  # Automatic rollback on failure
   

2. CI/CD Integration

   # .github/workflows/update-nushell.yml
   on:
     schedule:
       - cron: '0 0 * * 1'  # Check weekly
   jobs:
     update:
       runs-on: ubuntu-latest
       steps:
         - run: ./scripts/update_nushell_version.nu --latest --auto-approve
   

3. Version Validation

   ./scripts/validate_code_rules.nu  # Test rules against binary
   

4. Interactive Mode

   ./scripts/update_nushell_version.nu --interactive
   # Shows progress, allows step-by-step execution
   

5. Email Notifications

   # Notify on completion or errors
   send_email "Update to 0.108.0 completed" $report
   

Known Limitations

1. Directory Naming: Download script creates nushell-X.Y.Z/ instead of nushell/

   • Workaround: Manual move or symlink
   • Fix planned: Next script version

2. Private Repos: SSH authentication not automated

   • Workaround: Set up SSH keys manually
   • Enhancement: Key management automation

3. Windows Support: Scripts tested on Unix-like systems

   • Workaround: Use WSL or Git Bash
   • Enhancement: Native Windows support

---

Best Practices

Do's ✅

1. Always Review Breaking Changes: Don't auto-approve blindly
2. Test Before Committing: Run quality checks
3. Keep Backups: Git stash before major updates
4. Read Generated Reports: Contains important details
5. Update Documentation: Keep best_nushell_code.md in sync

Don'ts ❌

1. Don't Use --auto-approve in Production: Manual review is critical
2. Don't Skip Testing: Plugin tests catch issues
3. Don't Ignore Warnings: They often indicate real problems
4. Don't Modify Scripts During Update: Can cause inconsistencies
5. Don't Delete tmp/ During Update: Contains important state

---

Troubleshooting

Common Issues

Issue: Download fails with network error

# Solution: Use git clone fallback
cd nushell && git fetch --tags && git checkout 0.108.0

Issue: Build takes too long

# Solution: Use --skip-build for faster iteration
./scripts/update_nushell_version.nu 0.108.0 --skip-build

Issue: Version mismatch after update

# Solution: Run update_nu_versions.nu again
./scripts/update_nu_versions.nu update

Issue: Plugin test failures

# Solution: Test manually with new nushell
./nushell/target/release/nu
> plugin add target/release/nu_plugin_NAME
> plugin list

---

Appendix

Script Reference

Script	Purpose	Time	Required
update_nushell_version.nu	Main orchestrator	3-4h	Yes
download_nushell.nu	Download source	2m	Yes
analyze_nushell_features.nu	Analyze features	1m	Yes
audit_crate_dependencies.nu	Audit deps	1m	Yes
detect_breaking_changes.nu	Find changes	1m	Yes
build_nushell.nu	Build nushell	15m	Optional
test_plugin_compatibility.nu	Test plugins	5m	Optional
validate_code_rules.nu	Validate docs	2m	Optional

File Locations

project-root/
├── scripts/
│   ├── update_nushell_version.nu    # Main orchestrator
│   ├── download_nushell.nu          # Download manager
│   ├── analyze_nushell_features.nu  # Feature analyzer
│   ├── audit_crate_dependencies.nu  # Dependency auditor
│   ├── detect_breaking_changes.nu   # Change detector
│   ├── build_nushell.nu             # Build manager
│   ├── test_plugin_compatibility.nu # Compatibility tester
│   ├── validate_code_rules.nu       # Code validator
│   └── lib/
│       └── common_lib.nu            # Shared utilities
├── etc/
│   └── plugin_registry.toml         # Plugin configuration
├── tmp/                             # Generated reports
│   ├── feature_analysis.json
│   ├── dependency_audit.json
│   ├── breaking_changes_report.json
│   └── update_report_*.md
└── nushell/                         # Downloaded nushell source

---

Automation Guide Version: 1.0 Last Updated: 2025-10-18 Maintained By: Nushell Plugins Team