Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Multi-Repository Strategy Analysis

Date: 2025-10-01 Status: Strategic Analysis Related: Repository Distribution Analysis

Executive Summary

This document analyzes a multi-repository strategy as an alternative to the monorepo approach. After careful consideration of the provisioning system’s architecture, a hybrid approach with 4 core repositories is recommended, avoiding submodules in favor of a cleaner package-based dependency model.

Repository Architecture Options

Option A: Pure Monorepo (Original Recommendation)

Single repository: provisioning

Pros:

  • Simplest development workflow
  • Atomic cross-component changes
  • Single version number
  • One CI/CD pipeline

Cons:

  • Large repository size
  • Mixed language tooling (Rust + Nushell)
  • All-or-nothing updates
  • Unclear ownership boundaries

Option B: Multi-Repo with Submodules (❌ Not Recommended)

Repositories:

  • provisioning-core (main, contains submodules)
  • provisioning-platform (submodule)
  • provisioning-extensions (submodule)
  • provisioning-workspace (submodule)

Why Not Recommended:

  • Submodule hell: complex, error-prone workflows
  • Detached HEAD issues
  • Update synchronization nightmares
  • Clone complexity for users
  • Difficult to maintain version compatibility
  • Poor developer experience

Option C: Multi-Repo with Package Dependencies (✅ RECOMMENDED)

Independent repositories with package-based integration:

  • provisioning-core - Nushell libraries and KCL schemas
  • provisioning-platform - Rust services (orchestrator, control-center, MCP)
  • provisioning-extensions - Extension marketplace/catalog
  • provisioning-workspace - Project templates and examples
  • provisioning-distribution - Release automation and packaging

Why Recommended:

  • Clean separation of concerns
  • Independent versioning and release cycles
  • Language-specific tooling and workflows
  • Clear ownership boundaries
  • Package-based dependencies (no submodules)
  • Easier community contributions

Repository 1: provisioning-core

Purpose: Core Nushell infrastructure automation engine

Contents:

provisioning-core/
├── nulib/                   # Nushell libraries
│   ├── lib_provisioning/    # Core library functions
│   ├── servers/             # Server management
│   ├── taskservs/           # Task service management
│   ├── clusters/            # Cluster management
│   └── workflows/           # Workflow orchestration
├── cli/                     # CLI entry point
│   └── provisioning         # Pure Nushell CLI
├── kcl/                     # KCL schemas
│   ├── main.k
│   ├── settings.k
│   ├── server.k
│   ├── cluster.k
│   └── workflows.k
├── config/                  # Default configurations
│   └── config.defaults.toml
├── templates/               # Core templates
├── tools/                   # Build and packaging tools
├── tests/                   # Core tests
├── docs/                    # Core documentation
├── LICENSE
├── README.md
├── CHANGELOG.md
└── version.toml             # Core version file

Technology: Nushell, KCL Primary Language: Nushell Release Frequency: Monthly (stable) Ownership: Core team Dependencies: None (foundation)

Package Output:

  • provisioning-core-{version}.tar.gz - Installable package
  • Published to package registry

Installation Path:

/usr/local/
├── bin/provisioning
├── lib/provisioning/
└── share/provisioning/

Repository 2: provisioning-platform

Purpose: High-performance Rust platform services

Contents:

provisioning-platform/
├── orchestrator/            # Rust orchestrator
│   ├── src/
│   ├── tests/
│   ├── benches/
│   └── Cargo.toml
├── control-center/          # Web control center (Leptos)
│   ├── src/
│   ├── tests/
│   └── Cargo.toml
├── mcp-server/              # Model Context Protocol server
│   ├── src/
│   ├── tests/
│   └── Cargo.toml
├── api-gateway/             # REST API gateway
│   ├── src/
│   ├── tests/
│   └── Cargo.toml
├── shared/                  # Shared Rust libraries
│   ├── types/
│   └── utils/
├── docs/                    # Platform documentation
├── Cargo.toml               # Workspace root
├── Cargo.lock
├── LICENSE
├── README.md
└── CHANGELOG.md

Technology: Rust, WebAssembly Primary Language: Rust Release Frequency: Bi-weekly (fast iteration) Ownership: Platform team Dependencies:

  • provisioning-core (runtime integration, loose coupling)

Package Output:

  • provisioning-platform-{version}.tar.gz - Binaries
  • Binaries for: Linux (x86_64, arm64), macOS (x86_64, arm64)

Installation Path:

/usr/local/
├── bin/
│   ├── provisioning-orchestrator
│   └── provisioning-control-center
└── share/provisioning/platform/

Integration with Core:

  • Platform services call provisioning CLI via subprocess
  • No direct code dependencies
  • Communication via REST API and file-based queues
  • Core and Platform can be deployed independently

Repository 3: provisioning-extensions

Purpose: Extension marketplace and community modules

Contents:

provisioning-extensions/
├── registry/                # Extension registry
│   ├── index.json          # Searchable index
│   └── catalog/            # Extension metadata
├── providers/               # Additional cloud providers
│   ├── azure/
│   ├── gcp/
│   ├── digitalocean/
│   └── hetzner/
├── taskservs/               # Community task services
│   ├── databases/
│   │   ├── mongodb/
│   │   ├── redis/
│   │   └── cassandra/
│   ├── development/
│   │   ├── gitlab/
│   │   ├── jenkins/
│   │   └── sonarqube/
│   └── observability/
│       ├── prometheus/
│       ├── grafana/
│       └── loki/
├── clusters/                # Cluster templates
│   ├── ml-platform/
│   ├── data-pipeline/
│   └── gaming-backend/
├── workflows/               # Workflow templates
├── tools/                   # Extension development tools
├── docs/                    # Extension development guide
├── LICENSE
└── README.md

Technology: Nushell, KCL Primary Language: Nushell Release Frequency: Continuous (per-extension) Ownership: Community + Core team Dependencies:

  • provisioning-core (extends core functionality)

Package Output:

  • Individual extension packages: provisioning-ext-{name}-{version}.tar.gz
  • Registry index for discovery

Installation:

# Install extension via core CLI
provisioning extension install mongodb
provisioning extension install azure-provider

Extension Structure: Each extension is self-contained:

mongodb/
├── manifest.toml           # Extension metadata
├── taskserv.nu             # Implementation
├── templates/              # Templates
├── kcl/                    # KCL schemas
├── tests/                  # Tests
└── README.md

Repository 4: provisioning-workspace

Purpose: Project templates and starter kits

Contents:

provisioning-workspace/
├── templates/               # Workspace templates
│   ├── minimal/            # Minimal starter
│   ├── kubernetes/         # Full K8s cluster
│   ├── multi-cloud/        # Multi-cloud setup
│   ├── microservices/      # Microservices platform
│   ├── data-platform/      # Data engineering
│   └── ml-ops/             # MLOps platform
├── examples/               # Complete examples
│   ├── blog-deployment/
│   ├── e-commerce/
│   └── saas-platform/
├── blueprints/             # Architecture blueprints
├── docs/                   # Template documentation
├── tools/                  # Template scaffolding
│   └── create-workspace.nu
├── LICENSE
└── README.md

Technology: Configuration files, KCL Primary Language: TOML, KCL, YAML Release Frequency: Quarterly (stable templates) Ownership: Community + Documentation team Dependencies:

  • provisioning-core (templates use core)
  • provisioning-extensions (may reference extensions)

Package Output:

  • provisioning-templates-{version}.tar.gz

Usage:

# Create workspace from template
provisioning workspace init my-project --template kubernetes

# Or use separate tool
gh repo create my-project --template provisioning-workspace
cd my-project
provisioning workspace init

Repository 5: provisioning-distribution

Purpose: Release automation, packaging, and distribution infrastructure

Contents:

provisioning-distribution/
├── release-automation/      # Automated release workflows
│   ├── build-all.nu        # Build all packages
│   ├── publish.nu          # Publish to registries
│   └── validate.nu         # Validation suite
├── installers/             # Installation scripts
│   ├── install.nu          # Nushell installer
│   ├── install.sh          # Bash installer
│   └── install.ps1         # PowerShell installer
├── packaging/              # Package builders
│   ├── core/
│   ├── platform/
│   └── extensions/
├── registry/               # Package registry backend
│   ├── api/               # Registry REST API
│   └── storage/           # Package storage
├── ci-cd/                  # CI/CD configurations
│   ├── github/            # GitHub Actions
│   ├── gitlab/            # GitLab CI
│   └── jenkins/           # Jenkins pipelines
├── version-management/     # Cross-repo version coordination
│   ├── versions.toml      # Version matrix
│   └── compatibility.toml  # Compatibility matrix
├── docs/                   # Distribution documentation
│   ├── release-process.md
│   └── packaging-guide.md
├── LICENSE
└── README.md

Technology: Nushell, Bash, CI/CD Primary Language: Nushell, YAML Release Frequency: As needed Ownership: Release engineering team Dependencies: All repositories (orchestrates releases)

Responsibilities:

  • Build packages from all repositories
  • Coordinate multi-repo releases
  • Publish to package registries
  • Manage version compatibility
  • Generate release notes
  • Host package registry

Dependency and Integration Model

Package-Based Dependencies (Not Submodules)

┌─────────────────────────────────────────────────────────────┐
│                  provisioning-distribution                   │
│              (Release orchestration & registry)              │
└──────────────────────────┬──────────────────────────────────┘
                           │ publishes packages
                           ↓
                    ┌──────────────┐
                    │   Registry   │
                    └──────┬───────┘
                           │
        ┌──────────────────┼──────────────────┐
        ↓                  ↓                  ↓
┌───────────────┐  ┌──────────────┐  ┌──────────────┐
│  provisioning │  │ provisioning │  │ provisioning │
│     -core     │  │  -platform   │  │  -extensions │
└───────┬───────┘  └──────┬───────┘  └──────┬───────┘
        │                 │                  │
        │                 │ depends on       │ extends
        │                 └─────────┐        │
        │                           ↓        │
        └───────────────────────────────────→┘
                    runtime integration

Integration Mechanisms

1. Core ↔ Platform Integration

Method: Loose coupling via CLI + REST API

# Platform calls Core CLI (subprocess)
def create-server [name: string] {
    # Orchestrator executes Core CLI
    ^provisioning server create $name --infra production
}

# Core calls Platform API (HTTP)
def submit-workflow [workflow: record] {
    http post http://localhost:9090/workflows/submit $workflow
}

Version Compatibility:

# platform/Cargo.toml
[package.metadata.provisioning]
core-version = "^3.0"  # Compatible with core 3.x

2. Core ↔ Extensions Integration

Method: Plugin/module system

# Extension manifest
# extensions/mongodb/manifest.toml
[extension]
name = "mongodb"
version = "1.0.0"
type = "taskserv"
core-version = "^3.0"

[dependencies]
provisioning-core = "^3.0"

# Extension installation
# Core downloads and validates extension
provisioning extension install mongodb
# → Downloads from registry
# → Validates compatibility
# → Installs to ~/.provisioning/extensions/mongodb

3. Workspace Templates

Method: Git templates or package templates

# Option 1: GitHub template repository
gh repo create my-infra --template provisioning-workspace
cd my-infra
provisioning workspace init

# Option 2: Template package
provisioning workspace create my-infra --template kubernetes
# → Downloads template package
# → Scaffolds workspace
# → Initializes configuration

Version Management Strategy

Semantic Versioning Per Repository

Each repository maintains independent semantic versioning:

provisioning-core:       3.2.1
provisioning-platform:   2.5.3
provisioning-extensions: (per-extension versioning)
provisioning-workspace:  1.4.0

Compatibility Matrix

provisioning-distribution/version-management/versions.toml:

# Version compatibility matrix
[compatibility]

# Core versions and compatible platform versions
[compatibility.core]
"3.2.1" = { platform = "^2.5", extensions = "^1.0", workspace = "^1.0" }
"3.2.0" = { platform = "^2.4", extensions = "^1.0", workspace = "^1.0" }
"3.1.0" = { platform = "^2.3", extensions = "^0.9", workspace = "^1.0" }

# Platform versions and compatible core versions
[compatibility.platform]
"2.5.3" = { core = "^3.2", min-core = "3.2.0" }
"2.5.0" = { core = "^3.1", min-core = "3.1.0" }

# Release bundles (tested combinations)
[bundles]

[bundles.stable-3.2]
name = "Stable 3.2 Bundle"
release-date = "2025-10-15"
core = "3.2.1"
platform = "2.5.3"
extensions = ["mongodb@1.2.0", "redis@1.1.0", "azure@2.0.0"]
workspace = "1.4.0"

[bundles.lts-3.1]
name = "LTS 3.1 Bundle"
release-date = "2025-09-01"
lts-until = "2026-09-01"
core = "3.1.5"
platform = "2.4.8"
workspace = "1.3.0"

Release Coordination

Coordinated releases for major versions:

# Major release: All repos release together
provisioning-core:     3.0.0
provisioning-platform: 2.0.0
provisioning-workspace: 1.0.0

# Minor/patch releases: Independent
provisioning-core:     3.1.0 (adds features, platform stays 2.0.x)
provisioning-platform: 2.1.0 (improves orchestrator, core stays 3.1.x)

Development Workflow

Working on Single Repository

# Developer working on core only
git clone https://github.com/yourorg/provisioning-core
cd provisioning-core

# Install dependencies
just install-deps

# Development
just dev-check
just test

# Build package
just build

# Test installation locally
just install-dev

Working Across Repositories

# Scenario: Adding new feature requiring core + platform changes

# 1. Clone both repositories
git clone https://github.com/yourorg/provisioning-core
git clone https://github.com/yourorg/provisioning-platform

# 2. Create feature branches
cd provisioning-core
git checkout -b feat/batch-workflow-v2

cd ../provisioning-platform
git checkout -b feat/batch-workflow-v2

# 3. Develop with local linking
cd provisioning-core
just install-dev  # Installs to /usr/local/bin/provisioning

cd ../provisioning-platform
# Platform uses system provisioning CLI (local dev version)
cargo run

# 4. Test integration
cd ../provisioning-core
just test-integration

cd ../provisioning-platform
cargo test

# 5. Create PRs in both repositories
# PR #123 in provisioning-core
# PR #456 in provisioning-platform (references core PR)

# 6. Coordinate merge
# Merge core PR first, cut release 3.3.0
# Update platform dependency to core 3.3.0
# Merge platform PR, cut release 2.6.0

Testing Cross-Repo Integration

# Integration tests in provisioning-distribution
cd provisioning-distribution

# Test specific version combination
just test-integration \
    --core 3.3.0 \
    --platform 2.6.0

# Test bundle
just test-bundle stable-3.3

Distribution Strategy

Individual Repository Releases

Each repository releases independently:

# Core release
cd provisioning-core
git tag v3.2.1
git push --tags
# → GitHub Actions builds package
# → Publishes to package registry

# Platform release
cd provisioning-platform
git tag v2.5.3
git push --tags
# → GitHub Actions builds binaries
# → Publishes to package registry

Bundle Releases (Coordinated)

Distribution repository creates tested bundles:

cd provisioning-distribution

# Create bundle
just create-bundle stable-3.2 \
    --core 3.2.1 \
    --platform 2.5.3 \
    --workspace 1.4.0

# Test bundle
just test-bundle stable-3.2

# Publish bundle
just publish-bundle stable-3.2
# → Creates meta-package with all components
# → Publishes bundle to registry
# → Updates documentation

User Installation Options

# Install stable bundle (easiest)
curl -fsSL https://get.provisioning.io | sh

# Installs:
# - provisioning-core 3.2.1
# - provisioning-platform 2.5.3
# - provisioning-workspace 1.4.0

Option 2: Individual Component Installation

# Install only core (minimal)
curl -fsSL https://get.provisioning.io/core | sh

# Add platform later
provisioning install platform

# Add extensions
provisioning extension install mongodb

Option 3: Custom Combination

# Install specific versions
provisioning install core@3.1.0
provisioning install platform@2.4.0

Repository Ownership and Contribution Model

Core Team Ownership

RepositoryPrimary OwnerContribution Model
provisioning-coreCore TeamStrict review, stable API
provisioning-platformPlatform TeamFast iteration, performance focus
provisioning-extensionsCommunity + CoreOpen contributions, moderated
provisioning-workspaceDocs TeamTemplate contributions welcome
provisioning-distributionRelease EngineeringCore team only

Contribution Workflow

For Core:

  1. Create issue in provisioning-core
  2. Discuss design
  3. Submit PR with tests
  4. Strict code review
  5. Merge to main
  6. Release when ready

For Extensions:

  1. Create extension in provisioning-extensions
  2. Follow extension guidelines
  3. Submit PR
  4. Community review
  5. Merge and publish to registry
  6. Independent versioning

For Platform:

  1. Create issue in provisioning-platform
  2. Implement with benchmarks
  3. Submit PR
  4. Performance review
  5. Merge and release

CI/CD Strategy

Per-Repository CI/CD

Core CI (provisioning-core/.github/workflows/ci.yml):

name: Core CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install Nushell
        run: cargo install nu
      - name: Run tests
        run: just test
      - name: Validate KCL schemas
        run: just validate-kcl

  package:
    runs-on: ubuntu-latest
    if: startsWith(github.ref, 'refs/tags/v')
    steps:
      - uses: actions/checkout@v3
      - name: Build package
        run: just build
      - name: Publish to registry
        run: just publish
        env:
          REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}

Platform CI (provisioning-platform/.github/workflows/ci.yml):

name: Platform CI

on: [push, pull_request]

jobs:
  test:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v3
      - name: Build
        run: cargo build --release
      - name: Test
        run: cargo test --workspace
      - name: Benchmark
        run: cargo bench

  cross-compile:
    runs-on: ubuntu-latest
    if: startsWith(github.ref, 'refs/tags/v')
    steps:
      - uses: actions/checkout@v3
      - name: Build for Linux x86_64
        run: cargo build --release --target x86_64-unknown-linux-gnu
      - name: Build for Linux arm64
        run: cargo build --release --target aarch64-unknown-linux-gnu
      - name: Publish binaries
        run: just publish-binaries

Integration Testing (Distribution Repo)

Distribution CI (provisioning-distribution/.github/workflows/integration.yml):

name: Integration Tests

on:
  schedule:
    - cron: '0 0 * * *'  # Daily
  workflow_dispatch:

jobs:
  test-bundle:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install bundle
        run: |
          nu release-automation/install-bundle.nu stable-3.2

      - name: Run integration tests
        run: |
          nu tests/integration/test-all.nu

      - name: Test upgrade path
        run: |
          nu tests/integration/test-upgrade.nu 3.1.0 3.2.1

File and Directory Structure Comparison

Monorepo Structure

provisioning/                          (One repo, ~500MB)
├── core/                             (Nushell)
├── platform/                         (Rust)
├── extensions/                       (Community)
├── workspace/                        (Templates)
└── distribution/                     (Build)

Multi-Repo Structure

provisioning-core/                     (Repo 1, ~50MB)
├── nulib/
├── cli/
├── kcl/
└── tools/

provisioning-platform/                 (Repo 2, ~150MB with target/)
├── orchestrator/
├── control-center/
├── mcp-server/
└── Cargo.toml

provisioning-extensions/               (Repo 3, ~100MB)
├── registry/
├── providers/
├── taskservs/
└── clusters/

provisioning-workspace/                (Repo 4, ~20MB)
├── templates/
├── examples/
└── blueprints/

provisioning-distribution/             (Repo 5, ~30MB)
├── release-automation/
├── installers/
├── packaging/
└── registry/

Decision Matrix

CriterionMonorepoMulti-Repo
Development ComplexitySimpleModerate
Clone SizeLarge (~500MB)Small (50-150MB each)
Cross-Component ChangesEasy (atomic)Moderate (coordinated)
Independent ReleasesDifficultEasy
Language-Specific ToolingMixedClean
Community ContributionsHarder (big repo)Easier (focused repos)
Version ManagementSimple (one version)Complex (matrix)
CI/CD ComplexitySimple (one pipeline)Moderate (multiple)
Ownership ClarityUnclearClear
Extension EcosystemMonolithicModular
Build TimeLong (build all)Short (build one)
Testing IsolationDifficultEasy

Why Multi-Repo Wins for This Project

  1. Clear Separation of Concerns

    • Nushell core vs Rust platform are different domains
    • Different teams can own different repos
    • Different release cadences make sense

  2. Language-Specific Tooling

    • provisioning-core: Nushell-focused, simple testing
    • provisioning-platform: Rust workspace, Cargo tooling
    • No mixed tooling confusion

  3. Community Contributions

    • Extensions repo is easier to contribute to
    • Don’t need to clone entire monorepo
    • Clearer contribution guidelines per repo

  4. Independent Versioning

    • Core can stay stable (3.x for months)
    • Platform can iterate fast (2.x weekly)
    • Extensions have own lifecycles

  5. Build Performance

    • Only build what changed
    • Faster CI/CD per repo
    • Parallel builds across repos

  6. Extension Ecosystem

    • Extensions repo becomes marketplace
    • Third-party extensions can live separately
    • Registry becomes discovery mechanism

Implementation Strategy

Phase 1: Split Repositories (Week 1-2)

  1. Create 5 new repositories
  2. Extract code from monorepo
  3. Set up CI/CD for each
  4. Create initial packages

Phase 2: Package Integration (Week 3)

  1. Implement package registry
  2. Create installers
  3. Set up version compatibility matrix
  4. Test cross-repo integration

Phase 3: Distribution System (Week 4)

  1. Implement bundle system
  2. Create release automation
  3. Set up package hosting
  4. Document release process

Phase 4: Migration (Week 5)

  1. Migrate existing users
  2. Update documentation
  3. Archive monorepo
  4. Announce new structure

Conclusion

Recommendation: Multi-Repository Architecture with Package-Based Integration

The multi-repo approach provides:

  • ✅ Clear separation between Nushell core and Rust platform
  • ✅ Independent release cycles for different components
  • ✅ Better community contribution experience
  • ✅ Language-specific tooling and workflows
  • ✅ Modular extension ecosystem
  • ✅ Faster builds and CI/CD
  • ✅ Clear ownership boundaries

Avoid: Submodules (complexity nightmare)

Use: Package-based dependencies with version compatibility matrix

This architecture scales better for your project’s growth, supports a community extension ecosystem, and provides professional-grade separation of concerns while maintaining integration through a well-designed package system.

Next Steps

  1. Approve multi-repo strategy
  2. Create repository split plan
  3. Set up GitHub organizations/teams
  4. Implement package registry
  5. Begin repository extraction

Would you like me to create a detailed repository split implementation plan next?