nushell-plugins/scripts/detect_breaking_changes.nu
Jesús Pérez 2b3e574e3d # Summary
fix: help system integration, build process optimization, and plugin rebuild efficiency

## Detailed Description

This commit addresses critical issues in the help system discoverability, build process robustness, and plugin rebuild efficiency.

### 1. Help System Integration (New Feature)

**Issue**: Version-update module recipes were not discoverable
- Not shown in `just help modules`
- Not referenced in `just help`
- Not included in help navigation system
- Users had to manually run `just --list` to find update commands

**Solution**:
- Added version-update module to all help outputs
- Updated `justfiles/help.just` to document all 30+ version-update recipes
- Created new `just commands` recipe as discoverable alias for `just --list`
- Integrated version-update into help-all workflow

**Impact**:
- Version-update commands now fully discoverable via help system
- Users can find update commands with: `just help modules`, `just help`, `just commands`
- Improved overall help system navigation

**Files Modified**:
- `justfiles/help.just` (+23 lines)
  - Added version-update module to help sections
  - Added to modules list
  - Added to help-all workflow
  - New `commands` recipe showing all recipes by group

### 2. Build Process Fixes (Phase 3: Bin Archives)

#### 2a. Plugin Archive Collection Bug

**Issue**: "No plugins found to package" warning in Phase 3
- Collected 26 plugin binaries but reported 0
- Archive creation skipped because count was wrong

**Root Cause**: `each` command returns null, so `| length` returned 0
```nushell
#  OLD - each returns null
let plugin_count = (ls nu_plugin_*/target/release/nu_plugin_* | each {|p|
    cp $p.name $"($temp_dir)/"
} | length)  # Returns 0!
```

**Solution**: Separated counting from copying with proper filtering
```nushell
#  NEW - count before operations
let plugins_to_copy = (ls nu_plugin_*/target/release/nu_plugin_* | where type == "file")
let plugin_count = ($plugins_to_copy | length)
```

**Impact**:
- Now correctly collects and reports 26 plugins
- Filters out .d dependency files automatically
- Warning eliminated

#### 2b. Tar Archive Path Handling

**Issue**: Tar command failing silently with relative paths in subshell
- `cd $temp_dir` changes context unpredictably
- Relative path `../$archive_name` fails in subshell
- Archive file not created despite exit code 0

**Root Cause**: Shell context and relative path issues in Nushell `do` block

**Solution**: Used `tar -C` with absolute paths instead of `cd`
```nushell
#  OLD - unreliable context switching
do {
    cd $temp_dir
    tar -czf ../$archive_name .
}

#  NEW - absolute paths, no context switching
tar -C $temp_dir -czf $archive_path .
```

**Additional Improvements**:
- Absolute path construction using `pwd | path join`
- Better error diagnostics with exit code and stderr output
- File verification after creation

**Impact**:
- Tar archives now created successfully
- Robust path handling across platforms
- Clear error messages for debugging

#### 2c. File Size Calculation Type Error

**Issue**: Runtime error when calculating archive size
```
Error: The '/' operator does not work on values of type 'list<filesize>'
```

**Root Cause**: `ls` returns list of records, so `.size` was a list
```nushell
#  OLD - returns list<filesize>
(ls $archive_path).size / 1024 / 1024

#  NEW - returns filesize
(ls $archive_path | get 0.size) / 1024 / 1024
```

**Impact**:
- Proper file size calculation in MB
- No more type errors

**Files Modified**:
- `scripts/create_full_distribution.nu` (+58 lines, refactored plugin collection)
  - Fixed plugin counting logic
  - Improved path handling with absolute paths
  - Enhanced error diagnostics

### 3. Plugin Rebuild Optimization

**Issue**: All plugins marked for rebuild even when dependencies unchanged
- Step 4 (`update_all_plugins.nu`) touched all Cargo.toml files at 01:00:32
- Step 5 saw all files as "newer" than binaries
- Marked ALL plugins for rebuild, though cargo only rebuilt changed ones

**Root Cause**: Script always saved files, even when no changes made
```nushell
#  OLD - always saves, touching file timestamp
$updated_content | to toml | save -f $cargo_toml
```

**Solution**: Only save if content actually changed
```nushell
#  NEW - compare before writing
let original_toml = $content | to toml
let new_toml = $updated_content | to toml

if $original_toml != $new_toml {
    $updated_content | to toml | save -f $cargo_toml
}
```

**Impact**:
- Unchanged files preserve original timestamps
- Only plugins with actual dependency changes are rebuilt
- Efficient rebuild process with accurate file modification detection

**Files Modified**:
- `scripts/update_all_plugins.nu` (+12 lines, added content comparison)
  - Only touches files with real changes
  - Preserves timestamps for efficiency
  - Clearer logic and comments

### 4. Documentation

**Files Modified**:
- `CHANGELOG.md` (+56 lines)
  - Added comprehensive 2025-10-19 entry
  - Documented all fixes with root causes
  - Listed files modified and impact summary

## Technical Details

### Nushell Patterns Used

1. **Proper List Handling**:
   - `ls` returns list of records, access with `| get 0.size`
   - Filter with `where type == "file"` to exclude metadata

2. **Absolute Path Construction**:
   - `pwd | append "path" | path join` for cross-platform paths
   - Safer than string concatenation with `/`

3. **Content Comparison**:
   - Compare TOML string representation before saving
   - Preserves file timestamps for efficiency

4. **Error Diagnostics**:
   - Capture `stderr` from commands
   - Report exit codes and error messages separately

## Testing

- [x] Help system shows version-update module
- [x] `just commands` displays all recipes by group
- [x] Phase 3 bin archive creation works
- [x] Plugin collection reports correct count (26)
- [x] Tar archives created successfully
- [x] File size calculated correctly
- [x] Plugin rebuild only touches changed files
- [x] CHANGELOG updated with all changes

## Files Changed

```
38 files changed, 2721 insertions(+), 2548 deletions(-)

Core Changes:
- justfiles/help.just                  (+23)  Help system integration
- scripts/create_full_distribution.nu  (+58)  Build process fixes
- scripts/update_all_plugins.nu        (+12)  Rebuild optimization
- CHANGELOG.md                         (+56)  Documentation

Dependency Updates:
- All plugin Cargo.toml and Cargo.lock files (version consistency)
```

## Breaking Changes

None. These are bug fixes and optimizations that maintain backward compatibility.

## Migration Notes

No migration needed. Improvements are transparent to users.

## Related Issues

- Help system discoverability
- Build process Phase 3 failures
- Unnecessary plugin rebuilds
- Build process reliability

## Checklist

- [x] Changes follow Rust/Nushell idioms
- [x] Code is well-commented
- [x] Error handling is comprehensive
- [x] Documentation is updated
- [x] All changes tested
- [x] No breaking changes introduced
2025-10-19 01:17:13 +01:00

430 lines
13 KiB
Plaintext

#!/usr/bin/env nu
# Detect Breaking Changes Script
# Detects API changes and breaking changes between Nushell versions
#
# Usage:
# detect_breaking_changes.nu 0.107.1 0.108.0 # Compare two versions
# detect_breaking_changes.nu --scan-plugins # Scan plugins for usage of breaking APIs
# detect_breaking_changes.nu --export # Export breaking changes report
use lib/common_lib.nu *
# Known breaking changes database
const BREAKING_CHANGES = {
"0.108.0": [
{
type: "command_rename"
old_name: "into value"
new_name: "detect type"
description: "Command renamed and behavior changed - doesn't operate on cells anymore"
impact: "high"
migration: "Replace 'into value' with 'detect type' and review usage for cell operations"
},
{
type: "behavior_change"
component: "stream_error_handling"
description: "Collecting a stream that contains errors now raises an error itself"
impact: "medium"
migration: "Add explicit error handling when collecting streams that might contain errors"
},
{
type: "feature_addition"
feature: "mcp"
description: "MCP (Model Context Protocol) feature added as optional feature"
impact: "none"
migration: "Enable with: cargo build --features mcp"
}
]
}
# Main entry point
def main [
from_version?: string # Source version (e.g., "0.107.1")
to_version?: string # Target version (e.g., "0.108.0")
--scan-plugins # Scan plugins for breaking API usage
--export # Export breaking changes report
] {
log_info "Nushell Breaking Changes Detector"
# If versions provided, show breaking changes between them
if ($from_version | is-not-empty) and ($to_version | is-not-empty) {
show_breaking_changes $from_version $to_version
} else if $scan_plugins {
scan_plugins_for_breaking_changes
} else {
# Show all known breaking changes
show_all_breaking_changes
}
if $export {
export_breaking_changes_report
}
}
# Show breaking changes between two versions
def show_breaking_changes [
from_version: string
to_version: string
] {
log_info $"Analyzing breaking changes: [$from_version] → ($to_version)"
# Get breaking changes for target version
let changes = get_breaking_changes_for_version $to_version
if ($changes | is-empty) {
log_success $"No known breaking changes between [$from_version] and ($to_version)"
return
}
log_warn $"Found ($changes | length) breaking changes in version ($to_version)"
# Display by impact level
display_breaking_changes_by_impact $changes
}
# Get breaking changes for a specific version
def get_breaking_changes_for_version [
version: string
]: nothing -> list<record> {
$BREAKING_CHANGES | get -i $version | default []
}
# Display breaking changes organized by impact
def display_breaking_changes_by_impact [
changes: list<record>
] {
# Group by impact
let high_impact = $changes | where impact == "high"
let medium_impact = $changes | where impact == "medium"
let low_impact = $changes | where impact == "low" or impact == "none"
if ($high_impact | length) > 0 {
log_error "\n=== HIGH IMPACT CHANGES ==="
display_change_list $high_impact
}
if ($medium_impact | length) > 0 {
log_warn "\n=== MEDIUM IMPACT CHANGES ==="
display_change_list $medium_impact
}
if ($low_impact | length) > 0 {
log_info "\n=== LOW/NO IMPACT CHANGES ==="
display_change_list $low_impact
}
}
# Display list of changes
def display_change_list [
changes: list<record>
] {
$changes | each {|change|
let type_badge = match $change.type {
"command_rename" => "🔄 RENAME",
"behavior_change" => "⚠️ BEHAVIOR",
"api_removal" => "❌ REMOVAL",
"feature_addition" => "✨ FEATURE",
_ => "📝 CHANGE"
}
print $"\n ($type_badge)"
if ($change | get -i old_name | is-not-empty) {
print $" Old: ($change.old_name) → New: ($change.new_name)"
}
if ($change | get -i component | is-not-empty) {
print $" Component: ($change.component)"
}
if ($change | get -i feature | is-not-empty) {
print $" Feature: ($change.feature)"
}
print $" Description: ($change.description)"
print $" Migration: ($change.migration)"
}
}
# Show all known breaking changes
def show_all_breaking_changes [] {
log_info "All Known Breaking Changes"
let all_versions = $BREAKING_CHANGES | columns | sort
$all_versions | each {|version|
let changes = $BREAKING_CHANGES | get $version
log_info $"\n=== Version ($version) - ($changes | length) changes ==="
display_breaking_changes_by_impact $changes
}
}
# Scan plugins for usage of breaking APIs
def scan_plugins_for_breaking_changes [] {
log_info "Scanning plugins for breaking API usage..."
let plugin_dirs = get_plugin_directories
log_info $"Scanning ($plugin_dirs | length) custom plugins"
mut findings = []
for plugin_dir in $plugin_dirs {
let plugin_name = get_plugin_name $plugin_dir
log_info $"Scanning: ($plugin_name)..."
let plugin_findings = scan_plugin_directory $plugin_dir
if ($plugin_findings | length) > 0 {
$findings = ($findings | append {
plugin: $plugin_name
findings: $plugin_findings
})
}
}
# Display results
if ($findings | length) > 0 {
log_warn $"\n=== Breaking API Usage Found ==="
$findings | each {|result|
log_warn $"\n($result.plugin)"
$result.findings | each {|finding|
print $" • ($finding.pattern) in ($finding.file):($finding.line)"
print $" Context: ($finding.context)"
}
}
log_info "\nReview these findings and update code accordingly"
} else {
log_success "No breaking API usage detected!"
}
}
# Scan a plugin directory for breaking API usage
def scan_plugin_directory [
plugin_dir: string
]: nothing -> list<record> {
mut findings = []
# Get all Rust source files
let rust_files = glob $"($plugin_dir)/src/**/*.rs"
for file in $rust_files {
# Check for "into value" usage (breaking in 0.108.0)
let into_value_matches = try {
open $file
| lines
| enumerate
| where {|row| $row.item =~ "into value"}
} catch {
[]
}
if ($into_value_matches | length) > 0 {
for match in $into_value_matches {
$findings = ($findings | append {
pattern: "into value (renamed to 'detect type')"
file: $file
line: ($match.index + 1)
context: ($match.item | str trim)
})
}
}
# Check for stream collection patterns (behavior changed in 0.108.0)
let collect_matches = try {
open $file
| lines
| enumerate
| where {|row| $row.item =~ "collect\(\)" and $row.item =~ "stream"}
} catch {
[]
}
if ($collect_matches | length) > 0 {
for match in $collect_matches {
$findings = ($findings | append {
pattern: "stream.collect() (error behavior changed)"
file: $file
line: ($match.index + 1)
context: ($match.item | str trim)
})
}
}
}
$findings
}
# Export breaking changes report
def export_breaking_changes_report [] {
let output_file = "./tmp/breaking_changes_report.json"
ensure_dir "./tmp"
let report = {
generated_at: (date now | format date "%Y-%m-%d %H:%M:%S")
breaking_changes_database: $BREAKING_CHANGES
versions: ($BREAKING_CHANGES | columns)
}
$report | to json | save -f $output_file
log_success $"Breaking changes report exported to: ($output_file)"
}
# Check if code is affected by breaking changes
def "main check-code" [
code_snippet: string # Code to check
] {
log_info "Checking code for breaking API usage..."
mut issues = []
# Check for "into value"
if ($code_snippet =~ "into value") {
$issues = ($issues | append {
pattern: "into value"
severity: "high"
suggestion: "Replace with 'detect type' (note: behavior changed)"
})
}
# Check for direct stream collection
if ($code_snippet =~ "\.collect\(\)") {
$issues = ($issues | append {
pattern: "stream.collect()"
severity: "medium"
suggestion: "Add explicit error handling for stream errors"
})
}
# Display results
if ($issues | length) > 0 {
log_warn "Potential breaking API usage detected:"
$issues | each {|issue|
let severity_color = if $issue.severity == "high" { "red" } else { "yellow" }
print $" (ansi $severity_color)• ($issue.pattern)(ansi reset)"
print $" → ($issue.suggestion)"
}
} else {
log_success "No breaking API usage detected in code snippet"
}
}
# Generate migration guide for a specific version
def "main migration-guide" [
version: string # Version to generate guide for
] {
log_info $"Generating migration guide for version ($version)"
let changes = get_breaking_changes_for_version $version
if ($changes | is-empty) {
log_info $"No breaking changes documented for version ($version)"
return
}
# Generate markdown migration guide
let guide = generate_migration_markdown $version $changes
let output_file = $"./tmp/MIGRATION_($version).md"
ensure_dir "./tmp"
$guide | save -f $output_file
log_success $"Migration guide generated: ($output_file)"
}
# Generate migration guide in markdown format
def generate_migration_markdown [
version: string
changes: list<record>
]: nothing -> string {
mut content = $"# Migration Guide to Nushell ($version)\n\n"
$content = $"($content)Generated: (date now | format date '%Y-%m-%d %H:%M:%S')\n\n"
$content = $"($content)## Breaking Changes\n\n"
$content = $"($content)This guide covers ($changes | length) breaking changes in Nushell ($version).\n\n"
# Group by impact
let high_impact = $changes | where impact == "high"
let medium_impact = $changes | where impact == "medium"
let low_impact = $changes | where impact == "low" or impact == "none"
if ($high_impact | length) > 0 {
$content = $"($content)### ⚠️ High Impact Changes\n\n"
for change in $high_impact {
$content = ($content + (format_change_markdown $change) + "\n")
}
}
if ($medium_impact | length) > 0 {
$content = $"($content)### 📝 Medium Impact Changes\n\n"
for change in $medium_impact {
$content = ($content + (format_change_markdown $change) + "\n")
}
}
if ($low_impact | length) > 0 {
$content = $"($content)### ✨ Low/No Impact Changes\n\n"
for change in $low_impact {
$content = ($content + (format_change_markdown $change) + "\n")
}
}
$content
}
# Format single change as markdown
def format_change_markdown [
change: record
]: nothing -> string {
mut content = $"#### ($change.type | str title-case)\n\n"
if ($change | get -i old_name | is-not-empty) {
$content = $"($content)**Old**: `($change.old_name)` → **New**: `($change.new_name)`\n\n"
}
$content = $"($content)**Description**: ($change.description)\n\n"
$content = $"($content)**Migration**: ($change.migration)\n\n"
$content
}
# Add a new breaking change to the database
def "main add" [
version: string # Version this applies to
type: string # Type of change
description: string # Description
impact: string # Impact level (high/medium/low/none)
migration: string # Migration instructions
--old-name: string # Old name (for renames)
--new-name: string # New name (for renames)
] {
log_info "This command would add a new breaking change to the database"
log_warn "Note: The database is currently hardcoded in the script"
log_info "For production use, consider storing in external TOML/JSON file"
let new_change = {
type: $type
description: $description
impact: $impact
migration: $migration
}
# Add optional fields
let new_change = if ($old_name | is-not-empty) {
$new_change | merge {old_name: $old_name}
} else {
$new_change
}
let new_change = if ($new_name | is-not-empty) {
$new_change | merge {new_name: $new_name}
} else {
$new_change
}
log_info "New change to add:"
print ($new_change | to json)
}