provisioning/docs/extension-development-quickstart.md

Extension Development Quick Start Guide

This guide provides a hands-on walkthrough for developing custom extensions using the KCL package and module loader system.

Prerequisites

1. Core provisioning package installed:

   ./provisioning/tools/kcl-packager.nu build --version 1.0.0
   ./provisioning/tools/kcl-packager.nu install dist/provisioning-1.0.0.tar.gz
   

2. Module loader and extension tools available:

   ./provisioning/core/cli/module-loader --help
   ./provisioning/tools/create-extension.nu --help
   

Quick Start: Creating Your First Extension

Step 1: Create Extension from Template

# Interactive creation (recommended for beginners)
./provisioning/tools/create-extension.nu interactive

# Or direct creation
./provisioning/tools/create-extension.nu taskserv my-app \
    --author "Your Name" \
    --description "My custom application service"

Step 2: Navigate and Customize

# Navigate to your new extension
cd extensions/taskservs/my-app/kcl

# View generated files
ls -la
# kcl.mod - Package configuration
# my-app.k - Main taskserv definition
# version.k - Version information
# dependencies.k - Dependencies export
# README.md - Documentation template

Step 3: Customize Configuration

Edit my-app.k to match your service requirements:

# Update the configuration schema
schema MyAppConfig:
    """Configuration for My Custom App"""

    # Your service-specific settings
    database_url: str
    api_key: str
    debug_mode: bool = False

    # Customize resource requirements
    cpu_request: str = "200m"
    memory_request: str = "512Mi"

    # Add your service's port
    port: int = 3000

    check:
        len(database_url) > 0, "Database URL required"
        len(api_key) > 0, "API key required"

Step 4: Test Your Extension

# Test discovery
./provisioning/core/cli/module-loader discover taskservs | grep my-app

# Validate KCL syntax
kcl check my-app.k

# Validate extension structure
./provisioning/tools/create-extension.nu validate ../../../my-app

Step 5: Use in Workspace

# Create test workspace
mkdir -p /tmp/test-my-app
cd /tmp/test-my-app

# Initialize workspace
../provisioning/tools/workspace-init.nu . init

# Load your extension
../provisioning/core/cli/module-loader load taskservs . [my-app]

# Configure in servers.k
cat > servers.k << 'EOF'
import provisioning.settings as settings
import provisioning.server as server
import .taskservs.my-app.my-app as my_app

main_settings: settings.Settings = {
    main_name = "test-my-app"
    runset = {
        wait = True
        output_format = "human"
        output_path = "tmp/deployment"
        inventory_file = "./inventory.yaml"
        use_time = True
    }
}

test_servers: [server.Server] = [
    {
        hostname = "app-01"
        title = "My App Server"
        user = "admin"
        labels = "env: test"

        taskservs = [
            {
                name = "my-app"
                profile = "development"
            }
        ]
    }
]

{
    settings = main_settings
    servers = test_servers
}
EOF

# Test configuration
kcl run servers.k

Common Extension Patterns

Database Service Extension

# Create database service
./provisioning/tools/create-extension.nu taskserv company-db \
    --author "Your Company" \
    --description "Company-specific database service"

# Customize for PostgreSQL with company settings
cd extensions/taskservs/company-db/kcl

Edit the schema:

schema CompanyDbConfig:
    """Company database configuration"""

    # Database settings
    database_name: str = "company_db"
    postgres_version: str = "13"

    # Company-specific settings
    backup_schedule: str = "0 2 * * *"
    compliance_mode: bool = True
    encryption_enabled: bool = True

    # Connection settings
    max_connections: int = 100
    shared_buffers: str = "256MB"

    # Storage settings
    storage_size: str = "100Gi"
    storage_class: str = "fast-ssd"

    check:
        len(database_name) > 0, "Database name required"
        max_connections > 0, "Max connections must be positive"

Monitoring Service Extension

# Create monitoring service
./provisioning/tools/create-extension.nu taskserv company-monitoring \
    --author "Your Company" \
    --description "Company-specific monitoring and alerting"

Customize for Prometheus with company dashboards:

schema CompanyMonitoringConfig:
    """Company monitoring configuration"""

    # Prometheus settings
    retention_days: int = 30
    storage_size: str = "50Gi"

    # Company dashboards
    enable_business_metrics: bool = True
    enable_compliance_dashboard: bool = True

    # Alert routing
    alert_manager_config: AlertManagerConfig

    # Integration settings
    slack_webhook?: str
    email_notifications: [str]

schema AlertManagerConfig:
    """Alert manager configuration"""
    smtp_server: str
    smtp_port: int = 587
    smtp_auth_enabled: bool = True

Legacy System Integration

# Create legacy integration
./provisioning/tools/create-extension.nu taskserv legacy-bridge \
    --author "Your Company" \
    --description "Bridge for legacy system integration"

Customize for mainframe integration:

schema LegacyBridgeConfig:
    """Legacy system bridge configuration"""

    # Legacy system details
    mainframe_host: str
    mainframe_port: int = 23
    connection_type: "tn3270" | "direct" = "tn3270"

    # Data transformation
    data_format: "fixed-width" | "csv" | "xml" = "fixed-width"
    character_encoding: str = "ebcdic"

    # Processing settings
    batch_size: int = 1000
    poll_interval_seconds: int = 60

    # Error handling
    retry_attempts: int = 3
    dead_letter_queue_enabled: bool = True

Advanced Customization

Custom Provider Development

# Create custom cloud provider
./provisioning/tools/create-extension.nu provider company-cloud \
    --author "Your Company" \
    --description "Company private cloud provider"

Complete Infrastructure Stack

# Create complete cluster configuration
./provisioning/tools/create-extension.nu cluster company-stack \
    --author "Your Company" \
    --description "Complete company infrastructure stack"

Testing and Validation

Local Testing Workflow

# 1. Create test workspace
mkdir test-workspace && cd test-workspace
../provisioning/tools/workspace-init.nu . init

# 2. Load your extensions
../provisioning/core/cli/module-loader load taskservs . [my-app, company-db]
../provisioning/core/cli/module-loader load providers . [company-cloud]

# 3. Validate loading
../provisioning/core/cli/module-loader list taskservs .
../provisioning/core/cli/module-loader validate .

# 4. Test KCL compilation
kcl run servers.k

# 5. Dry-run deployment
../provisioning/core/cli/provisioning server create --infra . --check

Continuous Integration Testing

Create .github/workflows/test-extensions.yml:

name: Test Extensions
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install KCL
        run: |
          curl -fsSL https://kcl-lang.io/script/install-cli.sh | bash
          echo "$HOME/.kcl/bin" >> $GITHUB_PATH

      - name: Install Nushell
        run: |
          curl -L https://github.com/nushell/nushell/releases/download/0.107.1/nu-0.107.1-x86_64-unknown-linux-gnu.tar.gz | tar xzf -
          sudo mv nu-0.107.1-x86_64-unknown-linux-gnu/nu /usr/local/bin/

      - name: Build core package
        run: |
          nu provisioning/tools/kcl-packager.nu build --version test

      - name: Test extension discovery
        run: |
          nu provisioning/core/cli/module-loader discover taskservs

      - name: Validate extension syntax
        run: |
          find extensions -name "*.k" -exec kcl check {} \;

      - name: Test workspace creation
        run: |
          mkdir test-workspace
          nu provisioning/tools/workspace-init.nu test-workspace init
          cd test-workspace
          nu ../provisioning/core/cli/module-loader load taskservs . [my-app]
          kcl run servers.k

Best Practices Summary

1. Extension Design

• ✅ Use descriptive names in kebab-case
• ✅ Include comprehensive validation in schemas
• ✅ Provide multiple profiles for different environments
• ✅ Document all configuration options

2. Dependencies

• ✅ Declare all dependencies explicitly
• ✅ Use semantic versioning
• ✅ Test compatibility with different versions

3. Security

• ✅ Never hardcode secrets in schemas
• ✅ Use validation to ensure secure defaults
• ✅ Follow principle of least privilege

4. Documentation

• ✅ Include comprehensive README
• ✅ Provide usage examples
• ✅ Document troubleshooting steps
• ✅ Maintain changelog

5. Testing

• ✅ Test extension discovery and loading
• ✅ Validate KCL syntax
• ✅ Test in multiple environments
• ✅ Include CI/CD validation

Common Issues and Solutions

Extension Not Discovered

Problem: module-loader discover doesn't find your extension

Solutions:

1. Check directory structure: extensions/taskservs/my-service/kcl/
2. Verify kcl.mod exists and is valid
3. Ensure main .k file has correct name
4. Check file permissions

KCL Compilation Errors

Problem: KCL syntax errors in your extension

Solutions:

1. Use kcl check my-service.k to validate syntax
2. Check import statements are correct
3. Verify schema validation rules
4. Ensure all required fields have defaults or are provided

Loading Failures

Problem: Extension loads but doesn't work correctly

Solutions:

1. Check generated import files: cat taskservs.k
2. Verify dependencies are satisfied
3. Test with minimal configuration first
4. Check extension manifest: cat .manifest/taskservs.yaml

Next Steps

1. Explore Examples: Look at existing extensions in extensions/ directory
2. Read Advanced Docs: Study the comprehensive guides:
   • KCL Packaging Guide
   • Infrastructure-Specific Extensions
3. Join Community: Contribute to the provisioning system
4. Share Extensions: Publish useful extensions for others

Support

• Documentation: Package and Loader System Guide
• Templates: Use ./provisioning/tools/create-extension.nu list-templates
• Validation: Use ./provisioning/tools/create-extension.nu validate <path>
• Examples: Check provisioning/examples/ directory

Happy extension development! 🚀