Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Workspace Management Guide

This document provides comprehensive guidance on setting up and using development workspaces, including the path resolution system, testing infrastructure, and workspace tools usage.

Table of Contents

  1. Overview
  2. Workspace Architecture
  3. Setup and Initialization
  4. Path Resolution System
  5. Configuration Management
  6. Extension Development
  7. Runtime Management
  8. Health Monitoring
  9. Backup and Restore
  10. Troubleshooting

Overview

The workspace system provides isolated development environments for the provisioning project, enabling:

  • User Isolation: Each developer has their own workspace with isolated runtime data
  • Configuration Cascading: Hierarchical configuration from workspace to core system
  • Extension Development: Template-based extension development with testing
  • Path Resolution: Smart path resolution with workspace-aware fallbacks
  • Health Monitoring: Comprehensive health checks with automatic repairs
  • Backup/Restore: Complete workspace backup and restore capabilities

Location: /workspace/ Main Tool: workspace/tools/workspace.nu

Workspace Architecture

Directory Structure

workspace/
├── config/                          # Development configuration
│   ├── dev-defaults.toml            # Development environment defaults
│   ├── test-defaults.toml           # Testing environment configuration
│   ├── local-overrides.toml.example # User customization template
│   └── {user}.toml                  # User-specific configurations
├── extensions/                      # Extension development
│   ├── providers/                   # Custom provider extensions
│   │   ├── template/                # Provider development template
│   │   └── {user}/                  # User-specific providers
│   ├── taskservs/                   # Custom task service extensions
│   │   ├── template/                # Task service template
│   │   └── {user}/                  # User-specific task services
│   └── clusters/                    # Custom cluster extensions
│       ├── template/                # Cluster template
│       └── {user}/                  # User-specific clusters
├── infra/                          # Development infrastructure
│   ├── examples/                   # Example infrastructures
│   │   ├── minimal/                # Minimal learning setup
│   │   ├── development/            # Full development environment
│   │   └── testing/                # Testing infrastructure
│   ├── local/                      # Local development setups
│   └── {user}/                     # User-specific infrastructures
├── lib/                            # Workspace libraries
│   └── path-resolver.nu            # Path resolution system
├── runtime/                        # Runtime data (per-user isolation)
│   ├── workspaces/{user}/          # User workspace data
│   ├── cache/{user}/               # User-specific cache
│   ├── state/{user}/               # User state management
│   ├── logs/{user}/                # User application logs
│   └── data/{user}/                # User database files
└── tools/                          # Workspace management tools
    ├── workspace.nu                # Main workspace interface
    ├── init-workspace.nu           # Workspace initialization
    ├── workspace-health.nu         # Health monitoring
    ├── backup-workspace.nu         # Backup management
    ├── restore-workspace.nu        # Restore functionality
    ├── reset-workspace.nu          # Workspace reset
    └── runtime-manager.nu          # Runtime data management

Component Integration

Workspace → Core Integration:

  • Workspace paths take priority over core paths
  • Extensions discovered automatically from workspace
  • Configuration cascades from workspace to core defaults
  • Runtime data completely isolated per user

Development Workflow:

  1. Initialize personal workspace
  2. Configure development environment
  3. Develop extensions and infrastructure
  4. Test locally with isolated environment
  5. Deploy to shared infrastructure

Setup and Initialization

Quick Start

# Navigate to workspace
cd workspace/tools

# Initialize workspace with defaults
nu workspace.nu init

# Initialize with specific options
nu workspace.nu init --user-name developer --infra-name my-dev-infra

Complete Initialization

# Full initialization with all options
nu workspace.nu init \
    --user-name developer \
    --infra-name development-env \
    --workspace-type development \
    --template full \
    --overwrite \
    --create-examples

Initialization Parameters:

  • --user-name: User identifier (defaults to $env.USER)
  • --infra-name: Infrastructure name for this workspace
  • --workspace-type: Type (development, testing, production)
  • --template: Template to use (minimal, full, custom)
  • --overwrite: Overwrite existing workspace
  • --create-examples: Create example configurations and infrastructure

Post-Initialization Setup

Verify Installation:

# Check workspace health
nu workspace.nu health --detailed

# Show workspace status
nu workspace.nu status --detailed

# List workspace contents
nu workspace.nu list

Configure Development Environment:

# Create user-specific configuration
cp workspace/config/local-overrides.toml.example workspace/config/$USER.toml

# Edit configuration
$EDITOR workspace/config/$USER.toml

Path Resolution System

The workspace implements a sophisticated path resolution system that prioritizes workspace paths while providing fallbacks to core system paths.

Resolution Hierarchy

Resolution Order:

  1. Workspace User Paths: workspace/{type}/{user}/{name}
  2. Workspace Shared Paths: workspace/{type}/{name}
  3. Workspace Templates: workspace/{type}/template/{name}
  4. Core System Paths: core/{type}/{name} (fallback)

Using Path Resolution

# Import path resolver
use workspace/lib/path-resolver.nu

# Resolve configuration with workspace awareness
let config_path = (path-resolver resolve_path "config" "user" --workspace-user "developer")

# Resolve with automatic fallback to core
let extension_path = (path-resolver resolve_path "extensions" "custom-provider" --fallback-to-core)

# Create missing directories during resolution
let new_path = (path-resolver resolve_path "infra" "my-infra" --create-missing)

Configuration Resolution

Hierarchical Configuration Loading:

# Resolve configuration with full hierarchy
let config = (path-resolver resolve_config "user" --workspace-user "developer")

# Load environment-specific configuration
let dev_config = (path-resolver resolve_config "development" --workspace-user "developer")

# Get merged configuration with all overrides
let merged = (path-resolver resolve_config "merged" --workspace-user "developer" --include-overrides)

Extension Discovery

Automatic Extension Discovery:

# Find custom provider extension
let provider = (path-resolver resolve_extension "providers" "my-aws-provider")

# Discover all available task services
let taskservs = (path-resolver list_extensions "taskservs" --include-core)

# Find cluster definition
let cluster = (path-resolver resolve_extension "clusters" "development-cluster")

Health Checking

Workspace Health Validation:

# Check workspace health with automatic fixes
let health = (path-resolver check_workspace_health --workspace-user "developer" --fix-issues)

# Validate path resolution chain
let validation = (path-resolver validate_paths --workspace-user "developer" --repair-broken)

# Check runtime directories
let runtime_status = (path-resolver check_runtime_health --workspace-user "developer")

Configuration Management

Configuration Hierarchy

Configuration Cascade:

  1. User Configuration: workspace/config/{user}.toml
  2. Environment Defaults: workspace/config/{env}-defaults.toml
  3. Workspace Defaults: workspace/config/dev-defaults.toml
  4. Core System Defaults: config.defaults.toml

Environment-Specific Configuration

Development Environment (workspace/config/dev-defaults.toml):

[core]
name = "provisioning-dev"
version = "dev-${git.branch}"

[development]
auto_reload = true
verbose_logging = true
experimental_features = true
hot_reload_templates = true

[http]
use_curl = false
timeout = 30
retry_count = 3

[cache]
enabled = true
ttl = 300
refresh_interval = 60

[logging]
level = "debug"
file_rotation = true
max_size = "10MB"

Testing Environment (workspace/config/test-defaults.toml):

[core]
name = "provisioning-test"
version = "test-${build.timestamp}"

[testing]
mock_providers = true
ephemeral_resources = true
parallel_tests = true
cleanup_after_test = true

[http]
use_curl = true
timeout = 10
retry_count = 1

[cache]
enabled = false
mock_responses = true

[logging]
level = "info"
test_output = true

User Configuration Example

User-Specific Configuration (workspace/config/{user}.toml):

[core]
name = "provisioning-${workspace.user}"
version = "1.0.0-dev"

[infra]
current = "${workspace.user}-development"
default_provider = "upcloud"

[workspace]
user = "developer"
type = "development"
infra_name = "developer-dev"

[development]
preferred_editor = "code"
auto_backup = true
backup_interval = "1h"

[paths]
# Custom paths for this user
templates = "~/custom-templates"
extensions = "~/my-extensions"

[git]
auto_commit = false
commit_message_template = "[${workspace.user}] ${change.type}: ${change.description}"

[notifications]
slack_webhook = "https://hooks.slack.com/..."
email = "developer@company.com"

Configuration Commands

Workspace Configuration Management:

# Show current configuration
nu workspace.nu config show

# Validate configuration
nu workspace.nu config validate --user-name developer

# Edit user configuration
nu workspace.nu config edit --user-name developer

# Show configuration hierarchy
nu workspace.nu config hierarchy --user-name developer

# Merge configurations for debugging
nu workspace.nu config merge --user-name developer --output merged-config.toml

Extension Development

Extension Types

The workspace provides templates and tools for developing three types of extensions:

  1. Providers: Cloud provider implementations
  2. Task Services: Infrastructure service components
  3. Clusters: Complete deployment solutions

Provider Extension Development

Create New Provider:

# Copy template
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider

# Initialize provider
cd workspace/extensions/providers/my-provider
nu init.nu --provider-name my-provider --author developer

Provider Structure:

workspace/extensions/providers/my-provider/
├── kcl/
│   ├── provider.k          # Provider configuration schema
│   ├── server.k            # Server configuration
│   └── version.k           # Version management
├── nulib/
│   ├── provider.nu         # Main provider implementation
│   ├── servers.nu          # Server management
│   └── auth.nu             # Authentication handling
├── templates/
│   ├── server.j2           # Server configuration template
│   └── network.j2          # Network configuration template
├── tests/
│   ├── unit/               # Unit tests
│   └── integration/        # Integration tests
└── README.md

Test Provider:

# Run provider tests
nu workspace/extensions/providers/my-provider/nulib/provider.nu test

# Test with dry-run
nu workspace/extensions/providers/my-provider/nulib/provider.nu create-server --dry-run

# Integration test
nu workspace/extensions/providers/my-provider/tests/integration/basic-test.nu

Task Service Extension Development

Create New Task Service:

# Copy template
cp -r workspace/extensions/taskservs/template workspace/extensions/taskservs/my-service

# Initialize service
cd workspace/extensions/taskservs/my-service
nu init.nu --service-name my-service --service-type database

Task Service Structure:

workspace/extensions/taskservs/my-service/
├── kcl/
│   ├── taskserv.k          # Service configuration schema
│   ├── version.k           # Version configuration with GitHub integration
│   └── kcl.mod             # KCL module dependencies
├── nushell/
│   ├── taskserv.nu         # Main service implementation
│   ├── install.nu          # Installation logic
│   ├── uninstall.nu        # Removal logic
│   └── check-updates.nu    # Version checking
├── templates/
│   ├── config.j2           # Service configuration template
│   ├── systemd.j2          # Systemd service template
│   └── compose.j2          # Docker Compose template
└── manifests/
    ├── deployment.yaml     # Kubernetes deployment
    └── service.yaml        # Kubernetes service

Cluster Extension Development

Create New Cluster:

# Copy template
cp -r workspace/extensions/clusters/template workspace/extensions/clusters/my-cluster

# Initialize cluster
cd workspace/extensions/clusters/my-cluster
nu init.nu --cluster-name my-cluster --cluster-type web-stack

Testing Extensions:

# Test extension syntax
nu workspace.nu tools validate-extension providers/my-provider

# Run extension tests
nu workspace.nu tools test-extension taskservs/my-service

# Integration test with infrastructure
nu workspace.nu tools deploy-test clusters/my-cluster --infra test-env

Runtime Management

Runtime Data Organization

Per-User Isolation:

runtime/
├── workspaces/
│   ├── developer/          # Developer's workspace data
│   │   ├── current-infra   # Current infrastructure context
│   │   ├── settings.toml   # Runtime settings
│   │   └── extensions/     # Extension runtime data
│   └── tester/             # Tester's workspace data
├── cache/
│   ├── developer/          # Developer's cache
│   │   ├── providers/      # Provider API cache
│   │   ├── images/         # Container image cache
│   │   └── downloads/      # Downloaded artifacts
│   └── tester/             # Tester's cache
├── state/
│   ├── developer/          # Developer's state
│   │   ├── deployments/    # Deployment state
│   │   └── workflows/      # Workflow state
│   └── tester/             # Tester's state
├── logs/
│   ├── developer/          # Developer's logs
│   │   ├── provisioning.log
│   │   ├── orchestrator.log
│   │   └── extensions/
│   └── tester/             # Tester's logs
└── data/
    ├── developer/          # Developer's data
    │   ├── database.db     # Local database
    │   └── backups/        # Local backups
    └── tester/             # Tester's data

Runtime Management Commands

Initialize Runtime Environment:

# Initialize for current user
nu workspace/tools/runtime-manager.nu init

# Initialize for specific user
nu workspace/tools/runtime-manager.nu init --user-name developer

Runtime Cleanup:

# Clean cache older than 30 days
nu workspace/tools/runtime-manager.nu cleanup --type cache --age 30d

# Clean logs with rotation
nu workspace/tools/runtime-manager.nu cleanup --type logs --rotate

# Clean temporary files
nu workspace/tools/runtime-manager.nu cleanup --type temp --force

Log Management:

# View recent logs
nu workspace/tools/runtime-manager.nu logs --action tail --lines 100

# Follow logs in real-time
nu workspace/tools/runtime-manager.nu logs --action tail --follow

# Rotate large log files
nu workspace/tools/runtime-manager.nu logs --action rotate

# Archive old logs
nu workspace/tools/runtime-manager.nu logs --action archive --older-than 7d

Cache Management:

# Show cache statistics
nu workspace/tools/runtime-manager.nu cache --action stats

# Optimize cache
nu workspace/tools/runtime-manager.nu cache --action optimize

# Clear specific cache
nu workspace/tools/runtime-manager.nu cache --action clear --type providers

# Refresh cache
nu workspace/tools/runtime-manager.nu cache --action refresh --selective

Monitoring:

# Monitor runtime usage
nu workspace/tools/runtime-manager.nu monitor --duration 5m --interval 30s

# Check disk usage
nu workspace/tools/runtime-manager.nu monitor --type disk

# Monitor active processes
nu workspace/tools/runtime-manager.nu monitor --type processes --workspace-user developer

Health Monitoring

Health Check System

The workspace provides comprehensive health monitoring with automatic repair capabilities.

Health Check Components:

  • Directory Structure: Validates workspace directory integrity
  • Configuration Files: Checks configuration syntax and completeness
  • Runtime Environment: Validates runtime data and permissions
  • Extension Status: Checks extension functionality
  • Resource Usage: Monitors disk space and memory usage
  • Integration Status: Tests integration with core system

Health Commands

Basic Health Check:

# Quick health check
nu workspace.nu health

# Detailed health check with all components
nu workspace.nu health --detailed

# Health check with automatic fixes
nu workspace.nu health --fix-issues

# Export health report
nu workspace.nu health --report-format json > health-report.json

Component-Specific Health Checks:

# Check directory structure
nu workspace/tools/workspace-health.nu check-directories --workspace-user developer

# Validate configuration files
nu workspace/tools/workspace-health.nu check-config --workspace-user developer

# Check runtime environment
nu workspace/tools/workspace-health.nu check-runtime --workspace-user developer

# Test extension functionality
nu workspace/tools/workspace-health.nu check-extensions --workspace-user developer

Health Monitoring Output

Example Health Report:

{
  "workspace_health": {
    "user": "developer",
    "timestamp": "2025-09-25T14:30:22Z",
    "overall_status": "healthy",
    "checks": {
      "directories": {
        "status": "healthy",
        "issues": [],
        "auto_fixed": []
      },
      "configuration": {
        "status": "warning",
        "issues": [
          "User configuration missing default provider"
        ],
        "auto_fixed": [
          "Created missing user configuration file"
        ]
      },
      "runtime": {
        "status": "healthy",
        "disk_usage": "1.2GB",
        "cache_size": "450MB",
        "log_size": "120MB"
      },
      "extensions": {
        "status": "healthy",
        "providers": 2,
        "taskservs": 5,
        "clusters": 1
      }
    },
    "recommendations": [
      "Consider cleaning cache (>400MB)",
      "Rotate logs (>100MB)"
    ]
  }
}

Automatic Fixes

Auto-Fix Capabilities:

  • Missing Directories: Creates missing workspace directories
  • Broken Symlinks: Repairs or removes broken symbolic links
  • Configuration Issues: Creates missing configuration files with defaults
  • Permission Problems: Fixes file and directory permissions
  • Corrupted Cache: Clears and rebuilds corrupted cache entries
  • Log Rotation: Rotates large log files automatically

Backup and Restore

Backup System

Backup Components:

  • Configuration: All workspace configuration files
  • Extensions: Custom extensions and templates
  • Runtime Data: User-specific runtime data (optional)
  • Logs: Application logs (optional)
  • Cache: Cache data (optional)

Backup Commands

Create Backup:

# Basic backup
nu workspace.nu backup

# Backup with auto-generated name
nu workspace.nu backup --auto-name

# Comprehensive backup including logs and cache
nu workspace.nu backup --auto-name --include-logs --include-cache

# Backup specific components
nu workspace.nu backup --components config,extensions --name my-backup

Backup Options:

  • --auto-name: Generate timestamp-based backup name
  • --include-logs: Include application logs
  • --include-cache: Include cache data
  • --components: Specify components to backup
  • --compress: Create compressed backup archive
  • --encrypt: Encrypt backup with age/sops
  • --remote: Upload to remote storage (S3, etc.)

Restore System

List Available Backups:

# List all backups
nu workspace.nu restore --list-backups

# List backups with details
nu workspace.nu restore --list-backups --detailed

# Show backup contents
nu workspace.nu restore --show-contents --backup-name workspace-developer-20250925_143022

Restore Operations:

# Restore latest backup
nu workspace.nu restore --latest

# Restore specific backup
nu workspace.nu restore --backup-name workspace-developer-20250925_143022

# Selective restore
nu workspace.nu restore --selective --backup-name my-backup

# Restore to different user
nu workspace.nu restore --backup-name my-backup --restore-to different-user

Advanced Restore Options:

  • --selective: Choose components to restore interactively
  • --restore-to: Restore to different user workspace
  • --merge: Merge with existing workspace (don’t overwrite)
  • --dry-run: Show what would be restored without doing it
  • --verify: Verify backup integrity before restore

Reset and Cleanup

Workspace Reset:

# Reset with backup
nu workspace.nu reset --backup-first

# Reset keeping configuration
nu workspace.nu reset --backup-first --keep-config

# Complete reset (dangerous)
nu workspace.nu reset --force --no-backup

Cleanup Operations:

# Clean old data with dry-run
nu workspace.nu cleanup --type old --age 14d --dry-run

# Clean cache forcefully
nu workspace.nu cleanup --type cache --force

# Clean specific user data
nu workspace.nu cleanup --user-name old-user --type all

Troubleshooting

Common Issues

Workspace Not Found

Error: Workspace for user 'developer' not found

# Solution: Initialize workspace
nu workspace.nu init --user-name developer

Path Resolution Errors

Error: Path resolution failed for config/user

# Solution: Fix with health check
nu workspace.nu health --fix-issues

# Manual fix
nu workspace/lib/path-resolver.nu resolve_path "config" "user" --create-missing

Configuration Errors

Error: Invalid configuration syntax in user.toml

# Solution: Validate and fix configuration
nu workspace.nu config validate --user-name developer

# Reset to defaults
cp workspace/config/local-overrides.toml.example workspace/config/developer.toml

Runtime Issues

Error: Runtime directory permissions error

# Solution: Reinitialize runtime
nu workspace/tools/runtime-manager.nu init --user-name developer --force

# Fix permissions manually
chmod -R 755 workspace/runtime/workspaces/developer

Extension Issues

Error: Extension 'my-provider' not found or invalid

# Solution: Validate extension
nu workspace.nu tools validate-extension providers/my-provider

# Reinitialize extension from template
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider

Debug Mode

Enable Debug Logging:

# Set debug environment
export PROVISIONING_DEBUG=true
export PROVISIONING_LOG_LEVEL=debug
export PROVISIONING_WORKSPACE_USER=developer

# Run with debug
nu workspace.nu health --detailed

Performance Issues

Slow Operations:

# Check disk space
df -h workspace/

# Check runtime data size
du -h workspace/runtime/workspaces/developer/

# Optimize workspace
nu workspace.nu cleanup --type cache
nu workspace/tools/runtime-manager.nu cache --action optimize

Recovery Procedures

Corrupted Workspace:

# 1. Backup current state
nu workspace.nu backup --name corrupted-backup --force

# 2. Reset workspace
nu workspace.nu reset --backup-first

# 3. Restore from known good backup
nu workspace.nu restore --latest-known-good

# 4. Validate health
nu workspace.nu health --detailed --fix-issues

Data Loss Prevention:

  • Enable automatic backups: backup_interval = "1h" in user config
  • Use version control for custom extensions
  • Regular health checks: nu workspace.nu health
  • Monitor disk space and set up alerts

This workspace management system provides a robust foundation for development while maintaining isolation and providing comprehensive tools for maintenance and troubleshooting.