nushell-plugins/scripts/audit_crate_dependencies.nu
Jesús Pérez 2b3e574e3d # Summary
fix: help system integration, build process optimization, and plugin rebuild efficiency

## Detailed Description

This commit addresses critical issues in the help system discoverability, build process robustness, and plugin rebuild efficiency.

### 1. Help System Integration (New Feature)

**Issue**: Version-update module recipes were not discoverable
- Not shown in `just help modules`
- Not referenced in `just help`
- Not included in help navigation system
- Users had to manually run `just --list` to find update commands

**Solution**:
- Added version-update module to all help outputs
- Updated `justfiles/help.just` to document all 30+ version-update recipes
- Created new `just commands` recipe as discoverable alias for `just --list`
- Integrated version-update into help-all workflow

**Impact**:
- Version-update commands now fully discoverable via help system
- Users can find update commands with: `just help modules`, `just help`, `just commands`
- Improved overall help system navigation

**Files Modified**:
- `justfiles/help.just` (+23 lines)
  - Added version-update module to help sections
  - Added to modules list
  - Added to help-all workflow
  - New `commands` recipe showing all recipes by group

### 2. Build Process Fixes (Phase 3: Bin Archives)

#### 2a. Plugin Archive Collection Bug

**Issue**: "No plugins found to package" warning in Phase 3
- Collected 26 plugin binaries but reported 0
- Archive creation skipped because count was wrong

**Root Cause**: `each` command returns null, so `| length` returned 0
```nushell
#  OLD - each returns null
let plugin_count = (ls nu_plugin_*/target/release/nu_plugin_* | each {|p|
    cp $p.name $"($temp_dir)/"
} | length)  # Returns 0!
```

**Solution**: Separated counting from copying with proper filtering
```nushell
#  NEW - count before operations
let plugins_to_copy = (ls nu_plugin_*/target/release/nu_plugin_* | where type == "file")
let plugin_count = ($plugins_to_copy | length)
```

**Impact**:
- Now correctly collects and reports 26 plugins
- Filters out .d dependency files automatically
- Warning eliminated

#### 2b. Tar Archive Path Handling

**Issue**: Tar command failing silently with relative paths in subshell
- `cd $temp_dir` changes context unpredictably
- Relative path `../$archive_name` fails in subshell
- Archive file not created despite exit code 0

**Root Cause**: Shell context and relative path issues in Nushell `do` block

**Solution**: Used `tar -C` with absolute paths instead of `cd`
```nushell
#  OLD - unreliable context switching
do {
    cd $temp_dir
    tar -czf ../$archive_name .
}

#  NEW - absolute paths, no context switching
tar -C $temp_dir -czf $archive_path .
```

**Additional Improvements**:
- Absolute path construction using `pwd | path join`
- Better error diagnostics with exit code and stderr output
- File verification after creation

**Impact**:
- Tar archives now created successfully
- Robust path handling across platforms
- Clear error messages for debugging

#### 2c. File Size Calculation Type Error

**Issue**: Runtime error when calculating archive size
```
Error: The '/' operator does not work on values of type 'list<filesize>'
```

**Root Cause**: `ls` returns list of records, so `.size` was a list
```nushell
#  OLD - returns list<filesize>
(ls $archive_path).size / 1024 / 1024

#  NEW - returns filesize
(ls $archive_path | get 0.size) / 1024 / 1024
```

**Impact**:
- Proper file size calculation in MB
- No more type errors

**Files Modified**:
- `scripts/create_full_distribution.nu` (+58 lines, refactored plugin collection)
  - Fixed plugin counting logic
  - Improved path handling with absolute paths
  - Enhanced error diagnostics

### 3. Plugin Rebuild Optimization

**Issue**: All plugins marked for rebuild even when dependencies unchanged
- Step 4 (`update_all_plugins.nu`) touched all Cargo.toml files at 01:00:32
- Step 5 saw all files as "newer" than binaries
- Marked ALL plugins for rebuild, though cargo only rebuilt changed ones

**Root Cause**: Script always saved files, even when no changes made
```nushell
#  OLD - always saves, touching file timestamp
$updated_content | to toml | save -f $cargo_toml
```

**Solution**: Only save if content actually changed
```nushell
#  NEW - compare before writing
let original_toml = $content | to toml
let new_toml = $updated_content | to toml

if $original_toml != $new_toml {
    $updated_content | to toml | save -f $cargo_toml
}
```

**Impact**:
- Unchanged files preserve original timestamps
- Only plugins with actual dependency changes are rebuilt
- Efficient rebuild process with accurate file modification detection

**Files Modified**:
- `scripts/update_all_plugins.nu` (+12 lines, added content comparison)
  - Only touches files with real changes
  - Preserves timestamps for efficiency
  - Clearer logic and comments

### 4. Documentation

**Files Modified**:
- `CHANGELOG.md` (+56 lines)
  - Added comprehensive 2025-10-19 entry
  - Documented all fixes with root causes
  - Listed files modified and impact summary

## Technical Details

### Nushell Patterns Used

1. **Proper List Handling**:
   - `ls` returns list of records, access with `| get 0.size`
   - Filter with `where type == "file"` to exclude metadata

2. **Absolute Path Construction**:
   - `pwd | append "path" | path join` for cross-platform paths
   - Safer than string concatenation with `/`

3. **Content Comparison**:
   - Compare TOML string representation before saving
   - Preserves file timestamps for efficiency

4. **Error Diagnostics**:
   - Capture `stderr` from commands
   - Report exit codes and error messages separately

## Testing

- [x] Help system shows version-update module
- [x] `just commands` displays all recipes by group
- [x] Phase 3 bin archive creation works
- [x] Plugin collection reports correct count (26)
- [x] Tar archives created successfully
- [x] File size calculated correctly
- [x] Plugin rebuild only touches changed files
- [x] CHANGELOG updated with all changes

## Files Changed

```
38 files changed, 2721 insertions(+), 2548 deletions(-)

Core Changes:
- justfiles/help.just                  (+23)  Help system integration
- scripts/create_full_distribution.nu  (+58)  Build process fixes
- scripts/update_all_plugins.nu        (+12)  Rebuild optimization
- CHANGELOG.md                         (+56)  Documentation

Dependency Updates:
- All plugin Cargo.toml and Cargo.lock files (version consistency)
```

## Breaking Changes

None. These are bug fixes and optimizations that maintain backward compatibility.

## Migration Notes

No migration needed. Improvements are transparent to users.

## Related Issues

- Help system discoverability
- Build process Phase 3 failures
- Unnecessary plugin rebuilds
- Build process reliability

## Checklist

- [x] Changes follow Rust/Nushell idioms
- [x] Code is well-commented
- [x] Error handling is comprehensive
- [x] Documentation is updated
- [x] All changes tested
- [x] No breaking changes introduced
2025-10-19 01:17:13 +01:00

397 lines
11 KiB
Plaintext
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

#!/usr/bin/env nu
# Audit Crate Dependencies Script
# Scans all plugins and analyzes their nu-* crate dependencies
#
# Usage:
# audit_crate_dependencies.nu # Audit all plugins
# audit_crate_dependencies.nu --plugin NAME # Audit specific plugin
# audit_crate_dependencies.nu --export # Export dependency matrix
use lib/common_lib.nu *
# Main entry point
def main [
--plugin: string # Specific plugin to audit
--export # Export dependency matrix to JSON
--system-only # Only audit system plugins
--custom-only # Only audit custom plugins
] {
log_info "Nushell Plugin Dependency Audit"
# Check nushell source exists
let nushell_dir = "./nushell"
if not ($nushell_dir | path exists) {
log_error "Nushell source not found at [$nushell_dir]"
log_info "Run: download_nushell.nu <version>"
exit 1
}
# Get nushell version
let nushell_version = get_nushell_version $nushell_dir
# Audit plugins
if ($plugin | is-not-empty) {
audit_single_plugin $plugin $nushell_version
} else {
let include_system = not $custom_only
let include_custom = not $system_only
let results = audit_all_plugins $nushell_version $include_system $include_custom
# Display results
display_audit_results $results
# Export if requested
if $export {
export_dependency_matrix $results $nushell_version
}
}
}
# Get nushell version from Cargo.toml
def get_nushell_version [
nushell_dir: string
]: nothing -> string {
let cargo_toml = $"($nushell_dir)/Cargo.toml"
open $cargo_toml | get package.version
}
# Audit all plugins (system + custom)
def audit_all_plugins [
nushell_version: string
include_system: bool
include_custom: bool
]: nothing -> table {
mut all_results = []
# Audit system plugins
if $include_system {
log_info "Auditing system plugins..."
let system_results = audit_system_plugins $nushell_version
$all_results = ($all_results | append $system_results)
}
# Audit custom plugins
if $include_custom {
log_info "Auditing custom plugins..."
let custom_results = audit_custom_plugins $nushell_version
$all_results = ($all_results | append $custom_results)
}
$all_results
}
# Audit nushell workspace plugins
def audit_system_plugins [
nushell_version: string
]: nothing -> table {
let nushell_dir = "./nushell"
let cargo_toml = $"($nushell_dir)/Cargo.toml"
# Get workspace members
let workspace_members = try {
open $cargo_toml | get workspace.members
} catch {
log_error "Could not read workspace members"
return []
}
# Filter to plugin members only
let plugin_members = $workspace_members | where {|it| $it | str starts-with "crates/nu_plugin_"}
log_info $"Found ($plugin_members | length) system plugins"
# Audit each system plugin
$plugin_members | each {|member|
let plugin_name = $member | str replace "crates/" ""
let plugin_cargo_toml = $"($nushell_dir)/($member)/Cargo.toml"
audit_plugin_cargo_toml $plugin_name $plugin_cargo_toml $nushell_version "system"
}
}
# Audit custom plugins (nu_plugin_* in root)
def audit_custom_plugins [
nushell_version: string
]: nothing -> table {
let plugin_dirs = get_plugin_directories
log_info $"Found ($plugin_dirs | length) custom plugins"
$plugin_dirs | each {|plugin_dir|
let plugin_name = get_plugin_name $plugin_dir
let plugin_cargo_toml = $"($plugin_dir)/Cargo.toml"
if ($plugin_cargo_toml | path exists) {
audit_plugin_cargo_toml $plugin_name $plugin_cargo_toml $nushell_version "custom"
} else {
log_warn $"Cargo.toml not found for ($plugin_name)"
null
}
} | compact
}
# Audit single plugin Cargo.toml
def audit_plugin_cargo_toml [
plugin_name: string
cargo_toml_path: string
nushell_version: string
plugin_type: string
]: nothing -> record {
let cargo_data = try {
open $cargo_toml_path
} catch {
log_error $"Failed to read ($cargo_toml_path)"
return {
plugin: $plugin_name
type: $plugin_type
status: "error"
error: "Could not read Cargo.toml"
}
}
# Get plugin version
let plugin_version = try {
$cargo_data | get package.version
} catch {
"unknown"
}
# Extract nu-* dependencies
let dependencies = try {
$cargo_data | get dependencies
} catch {
{}
}
# Filter to nu-* crates only
let nu_deps = $dependencies
| transpose name spec
| where {|row| $row.name | str starts-with "nu-"}
| each {|row|
parse_dependency_spec $row.name $row.spec
}
# Check version consistency
let version_issues = check_version_consistency $nu_deps $nushell_version
{
plugin: $plugin_name
type: $plugin_type
plugin_version: $plugin_version
nu_dependencies: $nu_deps
version_issues: $version_issues
status: (if ($version_issues | is-empty) { "ok" } else { "version_mismatch" })
}
}
# Parse dependency specification
def parse_dependency_spec [
dep_name: string
spec: any
]: nothing -> record {
# Spec can be a string (version) or record (detailed spec)
let spec_type = $spec | describe
if $spec_type == "string" {
{
crate: $dep_name
version: $spec
path: null
features: []
}
} else if $spec_type == "record" {
{
crate: $dep_name
version: ($spec | get -i version | default "none")
path: ($spec | get -i path | default null)
features: ($spec | get -i features | default [])
}
} else {
{
crate: $dep_name
version: "unknown"
path: null
features: []
}
}
}
# Check version consistency with nushell
def check_version_consistency [
nu_deps: table
expected_version: string
]: nothing -> list<record> {
$nu_deps | each {|dep|
if $dep.version != $expected_version and $dep.version != "none" {
{
crate: $dep.crate
found: $dep.version
expected: $expected_version
issue: "version_mismatch"
}
} else {
null
}
} | compact
}
# Display audit results
def display_audit_results [
results: table
] {
log_info "\n=== Dependency Audit Results ==="
# Summary statistics
let total_plugins = $results | length
let ok_plugins = $results | where status == "ok" | length
let issues_plugins = $results | where status != "ok" | length
log_info $"Total plugins: ($total_plugins)"
log_success $"Clean: ($ok_plugins)"
if $issues_plugins > 0 {
log_warn $"With issues: ($issues_plugins)"
}
# Group by type
let system_plugins = $results | where type == "system"
let custom_plugins = $results | where type == "custom"
if ($system_plugins | length) > 0 {
log_info "\n--- System Plugins ---"
display_plugin_group $system_plugins
}
if ($custom_plugins | length) > 0 {
log_info "\n--- Custom Plugins ---"
display_plugin_group $custom_plugins
}
# Show detailed issues
let plugins_with_issues = $results | where status != "ok"
if ($plugins_with_issues | length) > 0 {
log_warn "\n=== Plugins with Version Issues ==="
$plugins_with_issues | each {|plugin|
log_error $"($plugin.plugin)"
$plugin.version_issues | each {|issue|
print $" • ($issue.crate): found [$issue.found], expected ($issue.expected)"
}
}
}
}
# Display plugin group
def display_plugin_group [
plugins: table
] {
$plugins | each {|plugin|
let status_icon = if $plugin.status == "ok" { "✅" } else { "⚠️" }
let dep_count = $plugin.nu_dependencies | length
print $" ($status_icon) ($plugin.plugin) (v($plugin.plugin_version))"
print $" nu-* dependencies: ($dep_count)"
if ($plugin.nu_dependencies | length) > 0 {
$plugin.nu_dependencies | each {|dep|
let path_info = if ($dep.path | is-not-empty) { $" \(path: ($dep.path))" } else { "" }
let features_info = if ($dep.features | is-not-empty) {
$" (($dep.features | str join ', '))"
} else {
""
}
print $" • ($dep.crate): ($dep.version)($path_info)($features_info)"
}
}
}
}
# Export dependency matrix to JSON
def export_dependency_matrix [
results: table
nushell_version: string
] {
let output_file = "./tmp/dependency_audit.json"
ensure_dir "./tmp"
let matrix = {
nushell_version: $nushell_version
audited_at: (date now | format date "%Y-%m-%d %H:%M:%S")
total_plugins: ($results | length)
plugins_ok: ($results | where status == "ok" | length)
plugins_with_issues: ($results | where status != "ok" | length)
plugins: $results
}
$matrix | to json | save -f $output_file
log_success $"Dependency audit exported to: ($output_file)"
}
# Audit single plugin
def audit_single_plugin [
plugin_name: string
nushell_version: string
] {
log_info $"Auditing plugin: ($plugin_name)"
# Check if it's a custom plugin
let custom_cargo_toml = $"./($plugin_name)/Cargo.toml"
if ($custom_cargo_toml | path exists) {
let result = audit_plugin_cargo_toml $plugin_name $custom_cargo_toml $nushell_version "custom"
display_audit_results [$result]
return
}
# Check if it's a system plugin
let system_cargo_toml = $"./nushell/crates/($plugin_name)/Cargo.toml"
if ($system_cargo_toml | path exists) {
let result = audit_plugin_cargo_toml $plugin_name $system_cargo_toml $nushell_version "system"
display_audit_results [$result]
return
}
log_error $"Plugin '($plugin_name)' not found"
log_info "Available custom plugins:"
get_plugin_directories | each {|dir|
print $" • (get_plugin_name $dir)"
}
exit 1
}
# Generate dependency matrix report
def "main matrix" [] {
log_info "Generating dependency matrix..."
let nushell_dir = "./nushell"
if not ($nushell_dir | path exists) {
log_error "Nushell source not found"
exit 1
}
let nushell_version = get_nushell_version $nushell_dir
let results = audit_all_plugins $nushell_version true true
# Create matrix of plugins × crates
let all_crates = $results
| each {|r| $r.nu_dependencies | get crate}
| flatten
| uniq
| sort
log_info "\n=== Dependency Matrix ==="
log_info $"Nushell version: ($nushell_version)"
log_info $"Unique nu-* crates: ($all_crates | length)"
# Show which plugins use which crates
$all_crates | each {|crate|
let users = $results | where {|r|
$crate in ($r.nu_dependencies | get crate)
} | get plugin
print $"\n($crate):"
$users | each {|user|
print $" • ($user)"
}
}
}