Repository Restructuring - Implementation Guide

Status: Ready for Implementation Estimated Time: 12-16 days Priority: High Related: Architecture Analysis

Overview

This guide provides step-by-step instructions for implementing the repository restructuring and distribution system improvements. Each phase includes specific commands, validation steps, and rollback procedures.

Prerequisites

Required Tools

Nushell 0.107.1+
Rust toolchain (for platform builds)
Git
tar/gzip
curl or wget

Recommended Tools

Just (task runner)
ripgrep (for code searches)
fd (for file finding)

Before Starting

Create full backup
Notify team members
Create implementation branch
Set aside dedicated time

Phase 1: Repository Restructuring (Days 1-4)

Day 1: Backup and Analysis

Step 1.1: Create Complete Backup

# Create timestamped backup
BACKUP_DIR="/Users/Akasha/project-provisioning-backup-$(date +%Y%m%d)"
cp -r /Users/Akasha/project-provisioning "$BACKUP_DIR"

# Verify backup
ls -lh "$BACKUP_DIR"
du -sh "$BACKUP_DIR"

# Create backup manifest
find "$BACKUP_DIR" -type f > "$BACKUP_DIR/manifest.txt"
echo "✅ Backup created: $BACKUP_DIR"

Step 1.2: Analyze Current State

cd /Users/Akasha/project-provisioning

# Count workspace directories
echo "=== Workspace Directories ==="
fd workspace -t d

# Analyze workspace contents
echo "=== Active Workspace ==="
du -sh workspace/

echo "=== Backup Workspaces ==="
du -sh _workspace/ backup-workspace/ workspace-librecloud/

# Find obsolete directories
echo "=== Build Artifacts ==="
du -sh target/ wrks/ NO/

# Save analysis
{
    echo "# Current State Analysis - $(date)"
    echo ""
    echo "## Workspace Directories"
    fd workspace -t d
    echo ""
    echo "## Directory Sizes"
    du -sh workspace/ _workspace/ backup-workspace/ workspace-librecloud/ 2>/dev/null
    echo ""
    echo "## Build Artifacts"
    du -sh target/ wrks/ NO/ 2>/dev/null
} > docs/development/current-state-analysis.txt

echo "✅ Analysis complete: docs/development/current-state-analysis.txt"

Step 1.3: Identify Dependencies

# Find all hardcoded paths
echo "=== Hardcoded Paths in Nushell Scripts ==="
rg -t nu "workspace/|_workspace/|backup-workspace/" provisioning/core/nulib/ | tee hardcoded-paths.txt

# Find ENV references (legacy)
echo "=== ENV References ==="
rg "PROVISIONING_" provisioning/core/nulib/ | wc -l

# Find workspace references in configs
echo "=== Config References ==="
rg "workspace" provisioning/config/

echo "✅ Dependencies mapped"

Step 1.4: Create Implementation Branch

# Create and switch to implementation branch
git checkout -b feat/repo-restructure

# Commit analysis
git add docs/development/current-state-analysis.txt
git commit -m "docs: add current state analysis for restructuring"

echo "✅ Implementation branch created: feat/repo-restructure"

Validation:

✅ Backup exists and is complete
✅ Analysis document created
✅ Dependencies mapped
✅ Implementation branch ready

Day 2: Directory Restructuring

Step 2.1: Create New Directory Structure

cd /Users/Akasha/project-provisioning

# Create distribution directory structure
mkdir -p distribution/{packages,installers,registry}
echo "✅ Created distribution/"

# Create workspace structure (keep tracked templates)
mkdir -p workspace/{infra,config,extensions,runtime}/{.gitkeep}
mkdir -p workspace/templates/{minimal,kubernetes,multi-cloud}
echo "✅ Created workspace/"

# Verify
tree -L 2 distribution/ workspace/

Step 2.2: Move Build Artifacts

# Move Rust build artifacts
if [ -d "target" ]; then
    mv target distribution/target
    echo "✅ Moved target/ to distribution/"
fi

# Move KCL packages
if [ -d "provisioning/tools/dist" ]; then
    mv provisioning/tools/dist/* distribution/packages/ 2>/dev/null || true
    echo "✅ Moved packages to distribution/"
fi

# Move any existing packages
find . -name "*.tar.gz" -o -name "*.zip" | grep -v node_modules | while read pkg; do
    mv "$pkg" distribution/packages/
    echo "  Moved: $pkg"
done

Step 2.3: Consolidate Workspaces

# Identify active workspace
echo "=== Current Workspace Status ==="
ls -la workspace/ _workspace/ backup-workspace/ 2>/dev/null

# Interactive workspace consolidation
read -p "Which workspace is currently active? (workspace/_workspace/backup-workspace): " ACTIVE_WS

if [ "$ACTIVE_WS" != "workspace" ]; then
    echo "Consolidating $ACTIVE_WS to workspace/"

    # Merge infra configs
    if [ -d "$ACTIVE_WS/infra" ]; then
        cp -r "$ACTIVE_WS/infra/"* workspace/infra/
    fi

    # Merge configs
    if [ -d "$ACTIVE_WS/config" ]; then
        cp -r "$ACTIVE_WS/config/"* workspace/config/
    fi

    # Merge extensions
    if [ -d "$ACTIVE_WS/extensions" ]; then
        cp -r "$ACTIVE_WS/extensions/"* workspace/extensions/
    fi

    echo "✅ Consolidated workspace"
fi

# Archive old workspace directories
mkdir -p .archived-workspaces
for ws in _workspace backup-workspace workspace-librecloud; do
    if [ -d "$ws" ] && [ "$ws" != "$ACTIVE_WS" ]; then
        mv "$ws" ".archived-workspaces/$(basename $ws)-$(date +%Y%m%d)"
        echo "  Archived: $ws"
    fi
done

echo "✅ Workspaces consolidated"

Step 2.4: Remove Obsolete Directories

# Remove build artifacts (already moved)
rm -rf wrks/
echo "✅ Removed wrks/"

# Remove test/scratch directories
rm -rf NO/
echo "✅ Removed NO/"

# Archive presentations (optional)
if [ -d "presentations" ]; then
    read -p "Archive presentations directory? (y/N): " ARCHIVE_PRES
    if [ "$ARCHIVE_PRES" = "y" ]; then
        tar czf presentations-archive-$(date +%Y%m%d).tar.gz presentations/
        rm -rf presentations/
        echo "✅ Archived and removed presentations/"
    fi
fi

# Remove empty directories
find . -type d -empty -delete 2>/dev/null || true

echo "✅ Cleanup complete"

Step 2.5: Update .gitignore

# Backup existing .gitignore
cp .gitignore .gitignore.backup

# Update .gitignore
cat >> .gitignore << 'EOF'

# ============================================================================
# Repository Restructure (2025-10-01)
# ============================================================================

# Workspace runtime data (user-specific)
/workspace/infra/
/workspace/config/
/workspace/extensions/
/workspace/runtime/

# Distribution artifacts
/distribution/packages/
/distribution/target/

# Build artifacts
/target/
/provisioning/platform/target/
/provisioning/platform/*/target/

# Rust artifacts
**/*.rs.bk
Cargo.lock

# Archived directories
/.archived-workspaces/

# Temporary files
*.tmp
*.temp
/tmp/
/wrks/
/NO/

# Logs
*.log
/workspace/runtime/logs/

# Cache
.cache/
/workspace/runtime/cache/

# IDE
.vscode/
.idea/
*.swp
*.swo
*~

# OS
.DS_Store
Thumbs.db

# Backup files
*.backup
*.bak

EOF

echo "✅ Updated .gitignore"

Step 2.6: Commit Restructuring

# Stage changes
git add -A

# Show what's being committed
git status

# Commit
git commit -m "refactor: restructure repository for clean distribution

- Consolidate workspace directories to single workspace/
- Move build artifacts to distribution/
- Remove obsolete directories (wrks/, NO/)
- Update .gitignore for new structure
- Archive old workspace variants

This is part of Phase 1 of the repository restructuring plan.

Related: docs/architecture/repo-dist-analysis.md"

echo "✅ Restructuring committed"

Validation:

✅ Single workspace/ directory exists
✅ Build artifacts in distribution/
✅ No wrks/, NO/ directories
✅ .gitignore updated
✅ Changes committed

Day 3: Update Path References

Step 3.1: Create Path Update Script

# Create migration script
cat > provisioning/tools/migration/update-paths.nu << 'EOF'
#!/usr/bin/env nu
# Path update script for repository restructuring

# Find and replace path references
export def main [] {
    print "🔧 Updating path references..."

    let replacements = [
        ["_workspace/" "workspace/"]
        ["backup-workspace/" "workspace/"]
        ["workspace-librecloud/" "workspace/"]
        ["wrks/" "distribution/"]
        ["NO/" "distribution/"]
    ]

    let files = (fd -e nu -e toml -e md . provisioning/)

    mut updated_count = 0

    for file in $files {
        mut content = (open $file)
        mut modified = false

        for replacement in $replacements {
            let old = $replacement.0
            let new = $replacement.1

            if ($content | str contains $old) {
                $content = ($content | str replace -a $old $new)
                $modified = true
            }
        }

        if $modified {
            $content | save -f $file
            $updated_count = $updated_count + 1
            print $"  ✓ Updated: ($file)"
        }
    }

    print $"✅ Updated ($updated_count) files"
}
EOF

chmod +x provisioning/tools/migration/update-paths.nu

Step 3.2: Run Path Updates

# Create backup before updates
git stash
git checkout -b feat/path-updates

# Run update script
nu provisioning/tools/migration/update-paths.nu

# Review changes
git diff

# Test a sample file
nu -c "use provisioning/core/nulib/servers/create.nu; print 'OK'"

Step 3.3: Update CLAUDE.md

# Update CLAUDE.md with new paths
cat > CLAUDE.md.new << 'EOF'
# CLAUDE.md

[Keep existing content, update paths section...]

## Updated Path Structure (2025-10-01)

### Core System
- **Main CLI**: `provisioning/core/cli/provisioning`
- **Libraries**: `provisioning/core/nulib/`
- **Extensions**: `provisioning/extensions/`
- **Platform**: `provisioning/platform/`

### User Workspace
- **Active Workspace**: `workspace/` (gitignored runtime data)
- **Templates**: `workspace/templates/` (tracked)
- **Infrastructure**: `workspace/infra/` (user configs, gitignored)

### Build System
- **Distribution**: `distribution/` (gitignored artifacts)
- **Packages**: `distribution/packages/`
- **Installers**: `distribution/installers/`

[Continue with rest of content...]
EOF

# Review changes
diff CLAUDE.md CLAUDE.md.new

# Apply if satisfied
mv CLAUDE.md.new CLAUDE.md

Step 3.4: Update Documentation

# Find all documentation files
fd -e md . docs/

# Update each doc with new paths
# This is semi-automated - review each file

# Create list of docs to update
fd -e md . docs/ > docs-to-update.txt

# Manual review and update
echo "Review and update each documentation file with new paths"
echo "Files listed in: docs-to-update.txt"

Step 3.5: Commit Path Updates

git add -A
git commit -m "refactor: update all path references for new structure

- Update Nushell scripts to use workspace/ instead of variants
- Update CLAUDE.md with new path structure
- Update documentation references
- Add migration script for future path changes

Phase 1.3 of repository restructuring."

echo "✅ Path updates committed"

Validation:

✅ All Nushell scripts reference correct paths
✅ CLAUDE.md updated
✅ Documentation updated
✅ No references to old paths remain

Day 4: Validation and Testing

Step 4.1: Automated Validation

# Create validation script
cat > provisioning/tools/validation/validate-structure.nu << 'EOF'
#!/usr/bin/env nu
# Repository structure validation

export def main [] {
    print "🔍 Validating repository structure..."

    mut passed = 0
    mut failed = 0

    # Check required directories exist
    let required_dirs = [
        "provisioning/core"
        "provisioning/extensions"
        "provisioning/platform"
        "provisioning/kcl"
        "workspace"
        "workspace/templates"
        "distribution"
        "docs"
        "tests"
    ]

    for dir in $required_dirs {
        if ($dir | path exists) {
            print $"  ✓ ($dir)"
            $passed = $passed + 1
        } else {
            print $"  ✗ ($dir) MISSING"
            $failed = $failed + 1
        }
    }

    # Check obsolete directories don't exist
    let obsolete_dirs = [
        "_workspace"
        "backup-workspace"
        "workspace-librecloud"
        "wrks"
        "NO"
    ]

    for dir in $obsolete_dirs {
        if not ($dir | path exists) {
            print $"  ✓ ($dir) removed"
            $passed = $passed + 1
        } else {
            print $"  ✗ ($dir) still exists"
            $failed = $failed + 1
        }
    }

    # Check no old path references
    let old_paths = ["_workspace/" "backup-workspace/" "wrks/"]
    for path in $old_paths {
        let results = (rg -l $path provisioning/ --iglob "!*.md" 2>/dev/null | lines)
        if ($results | is-empty) {
            print $"  ✓ No references to ($path)"
            $passed = $passed + 1
        } else {
            print $"  ✗ Found references to ($path):"
            $results | each { |f| print $"    - ($f)" }
            $failed = $failed + 1
        }
    }

    print ""
    print $"Results: ($passed) passed, ($failed) failed"

    if $failed > 0 {
        error make { msg: "Validation failed" }
    }

    print "✅ Validation passed"
}
EOF

chmod +x provisioning/tools/validation/validate-structure.nu

# Run validation
nu provisioning/tools/validation/validate-structure.nu

Step 4.2: Functional Testing

# Test core commands
echo "=== Testing Core Commands ==="

# Version
provisioning/core/cli/provisioning version
echo "✓ version command"

# Help
provisioning/core/cli/provisioning help
echo "✓ help command"

# List
provisioning/core/cli/provisioning list servers
echo "✓ list command"

# Environment
provisioning/core/cli/provisioning env
echo "✓ env command"

# Validate config
provisioning/core/cli/provisioning validate config
echo "✓ validate command"

echo "✅ Functional tests passed"

Step 4.3: Integration Testing

# Test workflow system
echo "=== Testing Workflow System ==="

# List workflows
nu -c "use provisioning/core/nulib/workflows/management.nu *; workflow list"
echo "✓ workflow list"

# Test workspace commands
echo "=== Testing Workspace Commands ==="

# Workspace info
provisioning/core/cli/provisioning workspace info
echo "✓ workspace info"

echo "✅ Integration tests passed"

Step 4.4: Create Test Report

{
    echo "# Repository Restructuring - Validation Report"
    echo "Date: $(date)"
    echo ""
    echo "## Structure Validation"
    nu provisioning/tools/validation/validate-structure.nu 2>&1
    echo ""
    echo "## Functional Tests"
    echo "✓ version command"
    echo "✓ help command"
    echo "✓ list command"
    echo "✓ env command"
    echo "✓ validate command"
    echo ""
    echo "## Integration Tests"
    echo "✓ workflow list"
    echo "✓ workspace info"
    echo ""
    echo "## Conclusion"
    echo "✅ Phase 1 validation complete"
} > docs/development/phase1-validation-report.md

echo "✅ Test report created: docs/development/phase1-validation-report.md"

Step 4.5: Update README

# Update main README with new structure
# This is manual - review and update README.md

echo "📝 Please review and update README.md with new structure"
echo "   - Update directory structure diagram"
echo "   - Update installation instructions"
echo "   - Update quick start guide"

Step 4.6: Finalize Phase 1

# Commit validation and reports
git add -A
git commit -m "test: add validation for repository restructuring

- Add structure validation script
- Add functional tests
- Add integration tests
- Create validation report
- Document Phase 1 completion

Phase 1 complete: Repository restructuring validated."

# Merge to implementation branch
git checkout feat/repo-restructure
git merge feat/path-updates

echo "✅ Phase 1 complete and merged"

Validation:

✅ All validation tests pass
✅ Functional tests pass
✅ Integration tests pass
✅ Validation report created
✅ README updated
✅ Phase 1 changes merged

Phase 2: Build System Implementation (Days 5-8)

Day 5: Build System Core

Step 5.1: Create Build Tools Directory

mkdir -p provisioning/tools/build
cd provisioning/tools/build

# Create directory structure
mkdir -p {core,platform,extensions,validation,distribution}

echo "✅ Build tools directory created"

Step 5.2: Implement Core Build System

# Create main build orchestrator
# See full implementation in repo-dist-analysis.md
# Copy build-system.nu from the analysis document

# Test build system
nu build-system.nu status

Step 5.3: Implement Core Packaging

# Create package-core.nu
# This packages Nushell libraries, KCL schemas, templates

# Test core packaging
nu build-system.nu build-core --version dev

Step 5.4: Create Justfile

# Create Justfile in project root
# See full Justfile in repo-dist-analysis.md

# Test Justfile
just --list
just status

Validation:

✅ Build system structure exists
✅ Core build orchestrator works
✅ Core packaging works
✅ Justfile functional

Day 6-8: Continue with Platform, Extensions, and Validation

[Follow similar pattern for remaining build system components]

Phase 3: Installation System (Days 9-11)

Day 9: Nushell Installer

Step 9.1: Create install.nu

mkdir -p distribution/installers

# Create install.nu
# See full implementation in repo-dist-analysis.md

Step 9.2: Test Installation

# Test installation to /tmp
nu distribution/installers/install.nu --prefix /tmp/provisioning-test

# Verify
ls -lh /tmp/provisioning-test/

# Test uninstallation
nu distribution/installers/install.nu uninstall --prefix /tmp/provisioning-test

Validation:

✅ Installer works
✅ Files installed to correct locations
✅ Uninstaller works
✅ No files left after uninstall

Rollback Procedures

If Phase 1 Fails

# Restore from backup
rm -rf /Users/Akasha/project-provisioning
cp -r "$BACKUP_DIR" /Users/Akasha/project-provisioning

# Return to main branch
cd /Users/Akasha/project-provisioning
git checkout main
git branch -D feat/repo-restructure

If Build System Fails

# Revert build system commits
git checkout feat/repo-restructure
git revert <commit-hash>

If Installation Fails

# Clean up test installation
rm -rf /tmp/provisioning-test
sudo rm -rf /usr/local/lib/provisioning
sudo rm -rf /usr/local/share/provisioning

Checklist

Phase 1: Repository Restructuring

Day 1: Backup and analysis complete
Day 2: Directory restructuring complete
Day 3: Path references updated
Day 4: Validation passed

Phase 2: Build System

Day 5: Core build system implemented
Day 6: Platform/extensions packaging
Day 7: Package validation
Day 8: Build system tested

Phase 3: Installation

Day 9: Nushell installer created
Day 10: Bash installer and CLI
Day 11: Multi-OS testing

Phase 4: Registry (Optional)

Day 12: Registry system
Day 13: Registry commands
Day 14: Registry hosting

Phase 5: Documentation

Day 15: Documentation updated
Day 16: Release prepared

Notes

Take breaks between phases - Don’t rush
Test thoroughly - Each phase builds on previous
Commit frequently - Small, atomic commits
Document issues - Track any problems encountered
Ask for review - Get feedback at phase boundaries

Support

If you encounter issues:

Check the validation reports
Review the rollback procedures
Consult the architecture analysis
Create an issue in the tracker

Keyboard shortcuts

Provisioning Platform Documentation