# Multi-Repo Strategy - Provisioning Platform Documentation
Complete documentation for the Provisioning Platform - Infrastructure automation with Nushell, Nickel, and Rust

## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Multi-Repository Strategy Analysis

Date: 2025-10-01 Status: Strategic Analysis Related: Repository Distribution Analysis

## Executive Summary

This document analyzes a multi-repository strategy as an alternative to the monorepo approach. After careful consideration of the provisioning system’s architecture, a hybrid approach with 4 core repositories is recommended, avoiding submodules in favor of a cleaner package-based dependency model.

## Repository Architecture Options

### Option A: Pure Monorepo (Original Recommendation)

Single repository: `provisioning`

Pros:

- Simplest development workflow
- Atomic cross-component changes
- Single version number
- One CI/CD pipeline

Cons:

- Large repository size
- Mixed language tooling (Rust + Nushell)
- All-or-nothing updates
- Unclear ownership boundaries

### Option B: Multi-Repo with Submodules (❌ Not Recommended)

Repositories:

- `provisioning-core` (main, contains submodules)
- `provisioning-platform` (submodule)
- `provisioning-extensions` (submodule)
- `provisioning-workspace` (submodule)

Why Not Recommended:

- Submodule hell: complex, error-prone workflows
- Detached HEAD issues
- Update synchronization nightmares
- Clone complexity for users
- Difficult to maintain version compatibility
- Poor developer experience

### Option C: Multi-Repo with Package Dependencies (✅ RECOMMENDED)

Independent repositories with package-based integration:

- `provisioning-core` - Nushell libraries and Nickel schemas
- `provisioning-platform` - Rust services (orchestrator, control-center, MCP)
- `provisioning-extensions` - Extension marketplace/catalog
- `provisioning-workspace` - Project templates and examples
- `provisioning-distribution` - Release automation and packaging

Why Recommended:

- Clean separation of concerns
- Independent versioning and release cycles
- Language-specific tooling and workflows
- Clear ownership boundaries
- Package-based dependencies (no submodules)
- Easier community contributions

## Recommended Multi-Repo Architecture

### Repository 1: `provisioning-core`

Purpose: Core Nushell infrastructure automation engine

Contents:

    provisioning-core/
    ├── nulib/                   # Nushell libraries
    │   ├── lib_provisioning/    # Core library functions
    │   ├── servers/             # Server management
    │   ├── taskservs/           # Task service management
    │   ├── clusters/            # Cluster management
    │   └── workflows/           # Workflow orchestration
    ├── cli/                     # CLI entry point
    │   └── provisioning         # Pure Nushell CLI
    ├── schemas/                 # Nickel schemas
    │   ├── main.ncl
    │   ├── settings.ncl
    │   ├── server.ncl
    │   ├── cluster.ncl
    │   └── workflows.ncl
    ├── config/                  # Default configurations
    │   └── config.defaults.toml
    ├── templates/               # Core templates
    ├── tools/                   # Build and packaging tools
    ├── tests/                   # Core tests
    ├── docs/                    # Core documentation
    ├── LICENSE
    ├── README.md
    ├── CHANGELOG.md
    └── version.toml             # Core version file

Technology: Nushell, Nickel Primary Language: Nushell Release Frequency: Monthly (stable) Ownership: Core team Dependencies: None (foundation)

Package Output:

- `provisioning-core-{version}.tar.gz` - Installable package
- Published to package registry

Installation Path:

    /usr/local/
    ├── bin/provisioning
    ├── lib/provisioning/
    └── share/provisioning/

### Repository 2: `provisioning-platform`

Purpose: High-performance Rust platform services

Contents:

    provisioning-platform/
    ├── orchestrator/            # Rust orchestrator
    │   ├── src/
    │   ├── tests/
    │   ├── benches/
    │   └── Cargo.toml
    ├── control-center/          # Web control center (Leptos)
    │   ├── src/
    │   ├── tests/
    │   └── Cargo.toml
    ├── mcp-server/              # Model Context Protocol server
    │   ├── src/
    │   ├── tests/
    │   └── Cargo.toml
    ├── api-gateway/             # REST API gateway
    │   ├── src/
    │   ├── tests/
    │   └── Cargo.toml
    ├── shared/                  # Shared Rust libraries
    │   ├── types/
    │   └── utils/
    ├── docs/                    # Platform documentation
    ├── Cargo.toml               # Workspace root
    ├── Cargo.lock
    ├── LICENSE
    ├── README.md
    └── CHANGELOG.md

Technology: Rust, WebAssembly Primary Language: Rust Release Frequency: Bi-weekly (fast iteration) Ownership: Platform team Dependencies:

- `provisioning-core` (runtime integration, loose coupling)

Package Output:

- `provisioning-platform-{version}.tar.gz` - Binaries
- Binaries for: Linux (x86_64, arm64), macOS (x86_64, arm64)

Installation Path:

    /usr/local/
    ├── bin/
    │   ├── provisioning-orchestrator
    │   └── provisioning-control-center
    └── share/provisioning/platform/

Integration with Core:

- Platform services call `provisioning` CLI via subprocess
- No direct code dependencies
- Communication via REST API and file-based queues
- Core and Platform can be deployed independently

### Repository 3: `provisioning-extensions`

Purpose: Extension marketplace and community modules

Contents:

    provisioning-extensions/
    ├── registry/                # Extension registry
    │   ├── index.json          # Searchable index
    │   └── catalog/            # Extension metadata
    ├── providers/               # Additional cloud providers
    │   ├── azure/
    │   ├── gcp/
    │   ├── digitalocean/
    │   └── hetzner/
    ├── taskservs/               # Community task services
    │   ├── databases/
    │   │   ├── mongodb/
    │   │   ├── redis/
    │   │   └── cassandra/
    │   ├── development/
    │   │   ├── gitlab/
    │   │   ├── jenkins/
    │   │   └── sonarqube/
    │   └── observability/
    │       ├── prometheus/
    │       ├── grafana/
    │       └── loki/
    ├── clusters/                # Cluster templates
    │   ├── ml-platform/
    │   ├── data-pipeline/
    │   └── gaming-backend/
    ├── workflows/               # Workflow templates
    ├── tools/                   # Extension development tools
    ├── docs/                    # Extension development guide
    ├── LICENSE
    └── README.md

Technology: Nushell, Nickel Primary Language: Nushell Release Frequency: Continuous (per-extension) Ownership: Community + Core team Dependencies:

- `provisioning-core` (extends core functionality)

Package Output:

- Individual extension packages: `provisioning-ext-{name}-{version}.tar.gz`
- Registry index for discovery

Installation:

    # Install extension via core CLI
    provisioning extension install mongodb
    provisioning extension install azure-provider

Extension Structure: Each extension is self-contained:

    mongodb/
    ├── manifest.toml           # Extension metadata
    ├── taskserv.nu             # Implementation
    ├── templates/              # Templates
    ├── schemas/                # Nickel schemas
    ├── tests/                  # Tests
    └── README.md

### Repository 4: `provisioning-workspace`

Purpose: Project templates and starter kits

Contents:

    provisioning-workspace/
    ├── templates/               # Workspace templates
    │   ├── minimal/            # Minimal starter
    │   ├── kubernetes/         # Full K8s cluster
    │   ├── multi-cloud/        # Multi-cloud setup
    │   ├── microservices/      # Microservices platform
    │   ├── data-platform/      # Data engineering
    │   └── ml-ops/             # MLOps platform
    ├── examples/               # Complete examples
    │   ├── blog-deployment/
    │   ├── e-commerce/
    │   └── saas-platform/
    ├── blueprints/             # Architecture blueprints
    ├── docs/                   # Template documentation
    ├── tools/                  # Template scaffolding
    │   └── create-workspace.nu
    ├── LICENSE
    └── README.md

Technology: Configuration files, Nickel Primary Language: TOML, Nickel, YAML Release Frequency: Quarterly (stable templates) Ownership: Community + Documentation team Dependencies:

- `provisioning-core` (templates use core)
- `provisioning-extensions` (may reference extensions)

Package Output:

- `provisioning-templates-{version}.tar.gz`

Usage:

    # Create workspace from template
    provisioning workspace init my-project --template kubernetes

    # Or use separate tool
    gh repo create my-project --template provisioning-workspace
    cd my-project
    provisioning workspace init

### Repository 5: `provisioning-distribution`

Purpose: Release automation, packaging, and distribution infrastructure

Contents:

    provisioning-distribution/
    ├── release-automation/      # Automated release workflows
    │   ├── build-all.nu        # Build all packages
    │   ├── publish.nu          # Publish to registries
    │   └── validate.nu         # Validation suite
    ├── installers/             # Installation scripts
    │   ├── install.nu          # Nushell installer
    │   ├── install.sh          # Bash installer
    │   └── install.ps1         # PowerShell installer
    ├── packaging/              # Package builders
    │   ├── core/
    │   ├── platform/
    │   └── extensions/
    ├── registry/               # Package registry backend
    │   ├── api/               # Registry REST API
    │   └── storage/           # Package storage
    ├── ci-cd/                  # CI/CD configurations
    │   ├── github/            # GitHub Actions
    │   ├── gitlab/            # GitLab CI
    │   └── jenkins/           # Jenkins pipelines
    ├── version-management/     # Cross-repo version coordination
    │   ├── versions.toml      # Version matrix
    │   └── compatibility.toml  # Compatibility matrix
    ├── docs/                   # Distribution documentation
    │   ├── release-process.md
    │   └── packaging-guide.md
    ├── LICENSE
    └── README.md

Technology: Nushell, Bash, CI/CD Primary Language: Nushell, YAML Release Frequency: As needed Ownership: Release engineering team Dependencies: All repositories (orchestrates releases)

Responsibilities:

- Build packages from all repositories
- Coordinate multi-repo releases
- Publish to package registries
- Manage version compatibility
- Generate release notes
- Host package registry

## Dependency and Integration Model

### Package-Based Dependencies (Not Submodules)

    ┌─────────────────────────────────────────────────────────────┐
    │                  provisioning-distribution                   │
    │              (Release orchestration & registry)              │
    └──────────────────────────┬──────────────────────────────────┘
                               │ publishes packages
                               ↓
                        ┌──────────────┐
                        │   Registry   │
                        └──────┬───────┘
                               │
            ┌──────────────────┼──────────────────┐
            ↓                  ↓                  ↓
    ┌───────────────┐  ┌──────────────┐  ┌──────────────┐
    │  provisioning │  │ provisioning │  │ provisioning │
    │     -core     │  │  -platform   │  │  -extensions │
    └───────┬───────┘  └──────┬───────┘  └──────┬───────┘
            │                 │                  │
            │                 │ depends on       │ extends
            │                 └─────────┐        │
            │                           ↓        │
            └───────────────────────────────────→┘
                        runtime integration

### Integration Mechanisms

#### 1. Core ↔ Platform Integration

Method: Loose coupling via CLI + REST API

    # Platform calls Core CLI (subprocess)
    def create-server [name: string] {
        # Orchestrator executes Core CLI
        ^provisioning server create $name --infra production
    }

    # Core calls Platform API (HTTP)
    def submit-workflow [workflow: record] {
        http post http://localhost:9090/workflows/submit $workflow
    }

Version Compatibility:

    # platform/Cargo.toml
    [package.metadata.provisioning]
    core-version = "^3.0"  # Compatible with core 3.x

#### 2. Core ↔ Extensions Integration

Method: Plugin/module system

    # Extension manifest
    # extensions/mongodb/manifest.toml
    [extension]
    name = "mongodb"
    version = "1.0.0"
    type = "taskserv"
    core-version = "^3.0"

    [dependencies]
    provisioning-core = "^3.0"

    # Extension installation
    # Core downloads and validates extension
    provisioning extension install mongodb
    # → Downloads from registry
    # → Validates compatibility
    # → Installs to ~/.provisioning/extensions/mongodb

#### 3. Workspace Templates

Method: Git templates or package templates

    # Option 1: GitHub template repository
    gh repo create my-infra --template provisioning-workspace
    cd my-infra
    provisioning workspace init

    # Option 2: Template package
    provisioning workspace create my-infra --template kubernetes
    # → Downloads template package
    # → Scaffolds workspace
    # → Initializes configuration

## Version Management Strategy

### Semantic Versioning Per Repository

Each repository maintains independent semantic versioning:

    provisioning-core:       3.2.1
    provisioning-platform:   2.5.3
    provisioning-extensions: (per-extension versioning)
    provisioning-workspace:  1.4.0

### Compatibility Matrix

`provisioning-distribution/version-management/versions.toml`:

    # Version compatibility matrix
    [compatibility]

    # Core versions and compatible platform versions
    [compatibility.core]
    "3.2.1" = { platform = "^2.5", extensions = "^1.0", workspace = "^1.0" }
    "3.2.0" = { platform = "^2.4", extensions = "^1.0", workspace = "^1.0" }
    "3.1.0" = { platform = "^2.3", extensions = "^0.9", workspace = "^1.0" }

    # Platform versions and compatible core versions
    [compatibility.platform]
    "2.5.3" = { core = "^3.2", min-core = "3.2.0" }
    "2.5.0" = { core = "^3.1", min-core = "3.1.0" }

    # Release bundles (tested combinations)
    [bundles]

    [bundles.stable-3.2]
    name = "Stable 3.2 Bundle"
    release-date = "2025-10-15"
    core = "3.2.1"
    platform = "2.5.3"
    extensions = ["mongodb@1.2.0", "redis@1.1.0", "azure@2.0.0"]
    workspace = "1.4.0"

    [bundles.lts-3.1]
    name = "LTS 3.1 Bundle"
    release-date = "2025-09-01"
    lts-until = "2026-09-01"
    core = "3.1.5"
    platform = "2.4.8"
    workspace = "1.3.0"

### Release Coordination

Coordinated releases for major versions:

    # Major release: All repos release together
    provisioning-core:     3.0.0
    provisioning-platform: 2.0.0
    provisioning-workspace: 1.0.0

    # Minor/patch releases: Independent
    provisioning-core:     3.1.0 (adds features, platform stays 2.0.x)
    provisioning-platform: 2.1.0 (improves orchestrator, core stays 3.1.x)

## Development Workflow

### Working on Single Repository

    # Developer working on core only
    git clone https://github.com/yourorg/provisioning-core
    cd provisioning-core

    # Install dependencies
    just install-deps

    # Development
    just dev-check
    just test

    # Build package
    just build

    # Test installation locally
    just install-dev

### Working Across Repositories

    # Scenario: Adding new feature requiring core + platform changes

    # 1. Clone both repositories
    git clone https://github.com/yourorg/provisioning-core
    git clone https://github.com/yourorg/provisioning-platform

    # 2. Create feature branches
    cd provisioning-core
    git checkout -b feat/batch-workflow-v2

    cd ../provisioning-platform
    git checkout -b feat/batch-workflow-v2

    # 3. Develop with local linking
    cd provisioning-core
    just install-dev  # Installs to /usr/local/bin/provisioning

    cd ../provisioning-platform
    # Platform uses system provisioning CLI (local dev version)
    cargo run

    # 4. Test integration
    cd ../provisioning-core
    just test-integration

    cd ../provisioning-platform
    cargo test

    # 5. Create PRs in both repositories
    # PR #123 in provisioning-core
    # PR #456 in provisioning-platform (references core PR)

    # 6. Coordinate merge
    # Merge core PR first, cut release 3.3.0
    # Update platform dependency to core 3.3.0
    # Merge platform PR, cut release 2.6.0

### Testing Cross-Repo Integration

    # Integration tests in provisioning-distribution
    cd provisioning-distribution

    # Test specific version combination
    just test-integration \
        --core 3.3.0 \
        --platform 2.6.0

    # Test bundle
    just test-bundle stable-3.3

## Distribution Strategy

### Individual Repository Releases

Each repository releases independently:

    # Core release
    cd provisioning-core
    git tag v3.2.1
    git push --tags
    # → GitHub Actions builds package
    # → Publishes to package registry

    # Platform release
    cd provisioning-platform
    git tag v2.5.3
    git push --tags
    # → GitHub Actions builds binaries
    # → Publishes to package registry

### Bundle Releases (Coordinated)

Distribution repository creates tested bundles:

    cd provisioning-distribution

    # Create bundle
    just create-bundle stable-3.2 \
        --core 3.2.1 \
        --platform 2.5.3 \
        --workspace 1.4.0

    # Test bundle
    just test-bundle stable-3.2

    # Publish bundle
    just publish-bundle stable-3.2
    # → Creates meta-package with all components
    # → Publishes bundle to registry
    # → Updates documentation

### User Installation Options

#### Option 1: Bundle Installation (Recommended for Users)

    # Install stable bundle (easiest)
    curl -fsSL https://get.provisioning.io | sh

    # Installs:
    # - provisioning-core 3.2.1
    # - provisioning-platform 2.5.3
    # - provisioning-workspace 1.4.0

#### Option 2: Individual Component Installation

    # Install only core (minimal)
    curl -fsSL https://get.provisioning.io/core | sh

    # Add platform later
    provisioning install platform

    # Add extensions
    provisioning extension install mongodb

#### Option 3: Custom Combination

    # Install specific versions
    provisioning install core@3.1.0
    provisioning install platform@2.4.0

## Repository Ownership and Contribution Model

### Core Team Ownership

Repository | Primary Owner | Contribution Model
--- | --- | ---
`provisioning-core` | Core Team | Strict review, stable API
`provisioning-platform` | Platform Team | Fast iteration, performance focus
`provisioning-extensions` | Community + Core | Open contributions, moderated
`provisioning-workspace` | Docs Team | Template contributions welcome
`provisioning-distribution` | Release Engineering | Core team only

### Contribution Workflow

For Core:

1. Create issue in `provisioning-core`
2. Discuss design
3. Submit PR with tests
4. Strict code review
5. Merge to `main`
6. Release when ready

For Extensions:

1. Create extension in `provisioning-extensions`
2. Follow extension guidelines
3. Submit PR
4. Community review
5. Merge and publish to registry
6. Independent versioning

For Platform:

1. Create issue in `provisioning-platform`
2. Implement with benchmarks
3. Submit PR
4. Performance review
5. Merge and release

## CI/CD Strategy

### Per-Repository CI/CD

Core CI (`provisioning-core/.github/workflows/ci.yml`):

    name: Core CI

    on: [push, pull_request]

    jobs:
      test:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v3
          - name: Install Nushell
            run: cargo install nu
          - name: Run tests
            run: just test
          - name: Validate Nickel schemas
            run: just validate-nickel

      package:
        runs-on: ubuntu-latest
        if: startsWith(github.ref, 'refs/tags/v')
        steps:
          - uses: actions/checkout@v3
          - name: Build package
            run: just build
          - name: Publish to registry
            run: just publish
            env:
              REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}

Platform CI (`provisioning-platform/.github/workflows/ci.yml`):

    name: Platform CI

    on: [push, pull_request]

    jobs:
      test:
        strategy:
          matrix:
            os: [ubuntu-latest, macos-latest]
        runs-on: ${{ matrix.os }}
        steps:
          - uses: actions/checkout@v3
          - name: Build
            run: cargo build --release
          - name: Test
            run: cargo test --workspace
          - name: Benchmark
            run: cargo bench

      cross-compile:
        runs-on: ubuntu-latest
        if: startsWith(github.ref, 'refs/tags/v')
        steps:
          - uses: actions/checkout@v3
          - name: Build for Linux x86_64
            run: cargo build --release --target x86_64-unknown-linux-gnu
          - name: Build for Linux arm64
            run: cargo build --release --target aarch64-unknown-linux-gnu
          - name: Publish binaries
            run: just publish-binaries

### Integration Testing (Distribution Repo)

Distribution CI (`provisioning-distribution/.github/workflows/integration.yml`):

    name: Integration Tests

    on:
      schedule:
        - cron: '0 0 * * *'  # Daily
      workflow_dispatch:

    jobs:
      test-bundle:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v3

          - name: Install bundle
            run: |
              nu release-automation/install-bundle.nu stable-3.2

          - name: Run integration tests
            run: |
              nu tests/integration/test-all.nu

          - name: Test upgrade path
            run: |
              nu tests/integration/test-upgrade.nu 3.1.0 3.2.1

## File and Directory Structure Comparison

### Monorepo Structure

    provisioning/                          (One repo, ~500 MB)
    ├── core/                             (Nushell)
    ├── platform/                         (Rust)
    ├── extensions/                       (Community)
    ├── workspace/                        (Templates)
    └── distribution/                     (Build)

### Multi-Repo Structure

    provisioning-core/                     (Repo 1, ~50 MB)
    ├── nulib/
    ├── cli/
    ├── schemas/
    └── tools/

    provisioning-platform/                 (Repo 2, ~150 MB with target/)
    ├── orchestrator/
    ├── control-center/
    ├── mcp-server/
    └── Cargo.toml

    provisioning-extensions/               (Repo 3, ~100 MB)
    ├── registry/
    ├── providers/
    ├── taskservs/
    └── clusters/

    provisioning-workspace/                (Repo 4, ~20 MB)
    ├── templates/
    ├── examples/
    └── blueprints/

    provisioning-distribution/             (Repo 5, ~30 MB)
    ├── release-automation/
    ├── installers/
    ├── packaging/
    └── registry/

## Decision Matrix

Criterion | Monorepo | Multi-Repo
--- | --- | ---
Development Complexity | Simple | Moderate
Clone Size | Large (~500 MB) | Small (50-150 MB each)
Cross-Component Changes | Easy (atomic) | Moderate (coordinated)
Independent Releases | Difficult | Easy
Language-Specific Tooling | Mixed | Clean
Community Contributions | Harder (big repo) | Easier (focused repos)
Version Management | Simple (one version) | Complex (matrix)
CI/CD Complexity | Simple (one pipeline) | Moderate (multiple)
Ownership Clarity | Unclear | Clear
Extension Ecosystem | Monolithic | Modular
Build Time | Long (build all) | Short (build one)
Testing Isolation | Difficult | Easy

## Recommended Approach: Multi-Repo

### Why Multi-Repo Wins for This Project

1. Clear Separation of Concerns

   - Nushell core vs Rust platform are different domains
   - Different teams can own different repos
   - Different release cadences make sense

2. Language-Specific Tooling

   - `provisioning-core`: Nushell-focused, simple testing
   - `provisioning-platform`: Rust workspace, Cargo tooling
   - No mixed tooling confusion

3. Community Contributions

   - Extensions repo is easier to contribute to
   - Don’t need to clone entire monorepo
   - Clearer contribution guidelines per repo

4. Independent Versioning

   - Core can stay stable (3.x for months)
   - Platform can iterate fast (2.x weekly)
   - Extensions have own lifecycles

5. Build Performance

   - Only build what changed
   - Faster CI/CD per repo
   - Parallel builds across repos

6. Extension Ecosystem

   - Extensions repo becomes marketplace
   - Third-party extensions can live separately
   - Registry becomes discovery mechanism

### Implementation Strategy

Phase 1: Split Repositories (Week 1-2)

1. Create 5 new repositories
2. Extract code from monorepo
3. Set up CI/CD for each
4. Create initial packages

Phase 2: Package Integration (Week 3)

1. Implement package registry
2. Create installers
3. Set up version compatibility matrix
4. Test cross-repo integration

Phase 3: Distribution System (Week 4)

1. Implement bundle system
2. Create release automation
3. Set up package hosting
4. Document release process

Phase 4: Migration (Week 5)

1. Migrate existing users
2. Update documentation
3. Archive monorepo
4. Announce new structure

## Conclusion

Recommendation: Multi-Repository Architecture with Package-Based Integration

The multi-repo approach provides:

- ✅ Clear separation between Nushell core and Rust platform
- ✅ Independent release cycles for different components
- ✅ Better community contribution experience
- ✅ Language-specific tooling and workflows
- ✅ Modular extension ecosystem
- ✅ Faster builds and CI/CD
- ✅ Clear ownership boundaries

Avoid: Submodules (complexity nightmare)

Use: Package-based dependencies with version compatibility matrix

This architecture scales better for your project’s growth, supports a community extension ecosystem, and provides professional-grade separation of concerns while maintaining integration through a well-designed package system.

## Next Steps

1. Approve multi-repo strategy
2. Create repository split plan
3. Set up GitHub organizations/teams
4. Implement package registry
5. Begin repository extraction

Would you like me to create a detailed repository split implementation plan next?