Multi-Repository Strategy Analysis
Date: 2025-10-01 Status: Strategic Analysis Related: Repository Distribution Analysis
Executive Summary
This document analyzes a multi-repository strategy as an alternative to the monorepo approach. After careful consideration of the provisioning system’s architecture, a hybrid approach with 4 core repositories is recommended, avoiding submodules in favor of a cleaner package-based dependency model.
Repository Architecture Options
Option A: Pure Monorepo (Original Recommendation)
Single repository: provisioning
Pros:
- Simplest development workflow
- Atomic cross-component changes
- Single version number
- One CI/CD pipeline
Cons:
- Large repository size
- Mixed language tooling (Rust + Nushell)
- All-or-nothing updates
- Unclear ownership boundaries
Option B: Multi-Repo with Submodules (❌ Not Recommended)
Repositories:
provisioning-core(main, contains submodules)provisioning-platform(submodule)provisioning-extensions(submodule)provisioning-workspace(submodule)
Why Not Recommended:
- Submodule hell: complex, error-prone workflows
- Detached HEAD issues
- Update synchronization nightmares
- Clone complexity for users
- Difficult to maintain version compatibility
- Poor developer experience
Option C: Multi-Repo with Package Dependencies (✅ RECOMMENDED)
Independent repositories with package-based integration:
provisioning-core- Nushell libraries and KCL schemasprovisioning-platform- Rust services (orchestrator, control-center, MCP)provisioning-extensions- Extension marketplace/catalogprovisioning-workspace- Project templates and examplesprovisioning-distribution- Release automation and packaging
Why Recommended:
- Clean separation of concerns
- Independent versioning and release cycles
- Language-specific tooling and workflows
- Clear ownership boundaries
- Package-based dependencies (no submodules)
- Easier community contributions
Recommended Multi-Repo Architecture
Repository 1: provisioning-core
Purpose: Core Nushell infrastructure automation engine
Contents:
provisioning-core/
├── nulib/ # Nushell libraries
│ ├── lib_provisioning/ # Core library functions
│ ├── servers/ # Server management
│ ├── taskservs/ # Task service management
│ ├── clusters/ # Cluster management
│ └── workflows/ # Workflow orchestration
├── cli/ # CLI entry point
│ └── provisioning # Pure Nushell CLI
├── kcl/ # KCL schemas
│ ├── main.k
│ ├── settings.k
│ ├── server.k
│ ├── cluster.k
│ └── workflows.k
├── config/ # Default configurations
│ └── config.defaults.toml
├── templates/ # Core templates
├── tools/ # Build and packaging tools
├── tests/ # Core tests
├── docs/ # Core documentation
├── LICENSE
├── README.md
├── CHANGELOG.md
└── version.toml # Core version file
```plaintext
**Technology:** Nushell, KCL
**Primary Language:** Nushell
**Release Frequency:** Monthly (stable)
**Ownership:** Core team
**Dependencies:** None (foundation)
**Package Output:**
- `provisioning-core-{version}.tar.gz` - Installable package
- Published to package registry
**Installation Path:**
```plaintext
/usr/local/
├── bin/provisioning
├── lib/provisioning/
└── share/provisioning/
```plaintext
---
### Repository 2: `provisioning-platform`
**Purpose:** High-performance Rust platform services
**Contents:**
```plaintext
provisioning-platform/
├── orchestrator/ # Rust orchestrator
│ ├── src/
│ ├── tests/
│ ├── benches/
│ └── Cargo.toml
├── control-center/ # Web control center (Leptos)
│ ├── src/
│ ├── tests/
│ └── Cargo.toml
├── mcp-server/ # Model Context Protocol server
│ ├── src/
│ ├── tests/
│ └── Cargo.toml
├── api-gateway/ # REST API gateway
│ ├── src/
│ ├── tests/
│ └── Cargo.toml
├── shared/ # Shared Rust libraries
│ ├── types/
│ └── utils/
├── docs/ # Platform documentation
├── Cargo.toml # Workspace root
├── Cargo.lock
├── LICENSE
├── README.md
└── CHANGELOG.md
```plaintext
**Technology:** Rust, WebAssembly
**Primary Language:** Rust
**Release Frequency:** Bi-weekly (fast iteration)
**Ownership:** Platform team
**Dependencies:**
- `provisioning-core` (runtime integration, loose coupling)
**Package Output:**
- `provisioning-platform-{version}.tar.gz` - Binaries
- Binaries for: Linux (x86_64, arm64), macOS (x86_64, arm64)
**Installation Path:**
```plaintext
/usr/local/
├── bin/
│ ├── provisioning-orchestrator
│ └── provisioning-control-center
└── share/provisioning/platform/
```plaintext
**Integration with Core:**
- Platform services call `provisioning` CLI via subprocess
- No direct code dependencies
- Communication via REST API and file-based queues
- Core and Platform can be deployed independently
---
### Repository 3: `provisioning-extensions`
**Purpose:** Extension marketplace and community modules
**Contents:**
```plaintext
provisioning-extensions/
├── registry/ # Extension registry
│ ├── index.json # Searchable index
│ └── catalog/ # Extension metadata
├── providers/ # Additional cloud providers
│ ├── azure/
│ ├── gcp/
│ ├── digitalocean/
│ └── hetzner/
├── taskservs/ # Community task services
│ ├── databases/
│ │ ├── mongodb/
│ │ ├── redis/
│ │ └── cassandra/
│ ├── development/
│ │ ├── gitlab/
│ │ ├── jenkins/
│ │ └── sonarqube/
│ └── observability/
│ ├── prometheus/
│ ├── grafana/
│ └── loki/
├── clusters/ # Cluster templates
│ ├── ml-platform/
│ ├── data-pipeline/
│ └── gaming-backend/
├── workflows/ # Workflow templates
├── tools/ # Extension development tools
├── docs/ # Extension development guide
├── LICENSE
└── README.md
```plaintext
**Technology:** Nushell, KCL
**Primary Language:** Nushell
**Release Frequency:** Continuous (per-extension)
**Ownership:** Community + Core team
**Dependencies:**
- `provisioning-core` (extends core functionality)
**Package Output:**
- Individual extension packages: `provisioning-ext-{name}-{version}.tar.gz`
- Registry index for discovery
**Installation:**
```bash
# Install extension via core CLI
provisioning extension install mongodb
provisioning extension install azure-provider
```plaintext
**Extension Structure:**
Each extension is self-contained:
```plaintext
mongodb/
├── manifest.toml # Extension metadata
├── taskserv.nu # Implementation
├── templates/ # Templates
├── kcl/ # KCL schemas
├── tests/ # Tests
└── README.md
```plaintext
---
### Repository 4: `provisioning-workspace`
**Purpose:** Project templates and starter kits
**Contents:**
```plaintext
provisioning-workspace/
├── templates/ # Workspace templates
│ ├── minimal/ # Minimal starter
│ ├── kubernetes/ # Full K8s cluster
│ ├── multi-cloud/ # Multi-cloud setup
│ ├── microservices/ # Microservices platform
│ ├── data-platform/ # Data engineering
│ └── ml-ops/ # MLOps platform
├── examples/ # Complete examples
│ ├── blog-deployment/
│ ├── e-commerce/
│ └── saas-platform/
├── blueprints/ # Architecture blueprints
├── docs/ # Template documentation
├── tools/ # Template scaffolding
│ └── create-workspace.nu
├── LICENSE
└── README.md
```plaintext
**Technology:** Configuration files, KCL
**Primary Language:** TOML, KCL, YAML
**Release Frequency:** Quarterly (stable templates)
**Ownership:** Community + Documentation team
**Dependencies:**
- `provisioning-core` (templates use core)
- `provisioning-extensions` (may reference extensions)
**Package Output:**
- `provisioning-templates-{version}.tar.gz`
**Usage:**
```bash
# Create workspace from template
provisioning workspace init my-project --template kubernetes
# Or use separate tool
gh repo create my-project --template provisioning-workspace
cd my-project
provisioning workspace init
```plaintext
---
### Repository 5: `provisioning-distribution`
**Purpose:** Release automation, packaging, and distribution infrastructure
**Contents:**
```plaintext
provisioning-distribution/
├── release-automation/ # Automated release workflows
│ ├── build-all.nu # Build all packages
│ ├── publish.nu # Publish to registries
│ └── validate.nu # Validation suite
├── installers/ # Installation scripts
│ ├── install.nu # Nushell installer
│ ├── install.sh # Bash installer
│ └── install.ps1 # PowerShell installer
├── packaging/ # Package builders
│ ├── core/
│ ├── platform/
│ └── extensions/
├── registry/ # Package registry backend
│ ├── api/ # Registry REST API
│ └── storage/ # Package storage
├── ci-cd/ # CI/CD configurations
│ ├── github/ # GitHub Actions
│ ├── gitlab/ # GitLab CI
│ └── jenkins/ # Jenkins pipelines
├── version-management/ # Cross-repo version coordination
│ ├── versions.toml # Version matrix
│ └── compatibility.toml # Compatibility matrix
├── docs/ # Distribution documentation
│ ├── release-process.md
│ └── packaging-guide.md
├── LICENSE
└── README.md
```plaintext
**Technology:** Nushell, Bash, CI/CD
**Primary Language:** Nushell, YAML
**Release Frequency:** As needed
**Ownership:** Release engineering team
**Dependencies:** All repositories (orchestrates releases)
**Responsibilities:**
- Build packages from all repositories
- Coordinate multi-repo releases
- Publish to package registries
- Manage version compatibility
- Generate release notes
- Host package registry
---
## Dependency and Integration Model
### Package-Based Dependencies (Not Submodules)
```plaintext
┌─────────────────────────────────────────────────────────────┐
│ provisioning-distribution │
│ (Release orchestration & registry) │
└──────────────────────────┬──────────────────────────────────┘
│ publishes packages
↓
┌──────────────┐
│ Registry │
└──────┬───────┘
│
┌──────────────────┼──────────────────┐
↓ ↓ ↓
┌───────────────┐ ┌──────────────┐ ┌──────────────┐
│ provisioning │ │ provisioning │ │ provisioning │
│ -core │ │ -platform │ │ -extensions │
└───────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
│ │ depends on │ extends
│ └─────────┐ │
│ ↓ │
└───────────────────────────────────→┘
runtime integration
```plaintext
### Integration Mechanisms
#### 1. **Core ↔ Platform Integration**
**Method:** Loose coupling via CLI + REST API
```nushell
# Platform calls Core CLI (subprocess)
def create-server [name: string] {
# Orchestrator executes Core CLI
^provisioning server create $name --infra production
}
# Core calls Platform API (HTTP)
def submit-workflow [workflow: record] {
http post http://localhost:9090/workflows/submit $workflow
}
```plaintext
**Version Compatibility:**
```toml
# platform/Cargo.toml
[package.metadata.provisioning]
core-version = "^3.0" # Compatible with core 3.x
```plaintext
#### 2. **Core ↔ Extensions Integration**
**Method:** Plugin/module system
```nushell
# Extension manifest
# extensions/mongodb/manifest.toml
[extension]
name = "mongodb"
version = "1.0.0"
type = "taskserv"
core-version = "^3.0"
[dependencies]
provisioning-core = "^3.0"
# Extension installation
# Core downloads and validates extension
provisioning extension install mongodb
# → Downloads from registry
# → Validates compatibility
# → Installs to ~/.provisioning/extensions/mongodb
```plaintext
#### 3. **Workspace Templates**
**Method:** Git templates or package templates
```bash
# Option 1: GitHub template repository
gh repo create my-infra --template provisioning-workspace
cd my-infra
provisioning workspace init
# Option 2: Template package
provisioning workspace create my-infra --template kubernetes
# → Downloads template package
# → Scaffolds workspace
# → Initializes configuration
```plaintext
---
## Version Management Strategy
### Semantic Versioning Per Repository
Each repository maintains independent semantic versioning:
```plaintext
provisioning-core: 3.2.1
provisioning-platform: 2.5.3
provisioning-extensions: (per-extension versioning)
provisioning-workspace: 1.4.0
```plaintext
### Compatibility Matrix
**`provisioning-distribution/version-management/versions.toml`:**
```toml
# Version compatibility matrix
[compatibility]
# Core versions and compatible platform versions
[compatibility.core]
"3.2.1" = { platform = "^2.5", extensions = "^1.0", workspace = "^1.0" }
"3.2.0" = { platform = "^2.4", extensions = "^1.0", workspace = "^1.0" }
"3.1.0" = { platform = "^2.3", extensions = "^0.9", workspace = "^1.0" }
# Platform versions and compatible core versions
[compatibility.platform]
"2.5.3" = { core = "^3.2", min-core = "3.2.0" }
"2.5.0" = { core = "^3.1", min-core = "3.1.0" }
# Release bundles (tested combinations)
[bundles]
[bundles.stable-3.2]
name = "Stable 3.2 Bundle"
release-date = "2025-10-15"
core = "3.2.1"
platform = "2.5.3"
extensions = ["mongodb@1.2.0", "redis@1.1.0", "azure@2.0.0"]
workspace = "1.4.0"
[bundles.lts-3.1]
name = "LTS 3.1 Bundle"
release-date = "2025-09-01"
lts-until = "2026-09-01"
core = "3.1.5"
platform = "2.4.8"
workspace = "1.3.0"
```plaintext
### Release Coordination
**Coordinated releases** for major versions:
```bash
# Major release: All repos release together
provisioning-core: 3.0.0
provisioning-platform: 2.0.0
provisioning-workspace: 1.0.0
# Minor/patch releases: Independent
provisioning-core: 3.1.0 (adds features, platform stays 2.0.x)
provisioning-platform: 2.1.0 (improves orchestrator, core stays 3.1.x)
```plaintext
---
## Development Workflow
### Working on Single Repository
```bash
# Developer working on core only
git clone https://github.com/yourorg/provisioning-core
cd provisioning-core
# Install dependencies
just install-deps
# Development
just dev-check
just test
# Build package
just build
# Test installation locally
just install-dev
```plaintext
### Working Across Repositories
```bash
# Scenario: Adding new feature requiring core + platform changes
# 1. Clone both repositories
git clone https://github.com/yourorg/provisioning-core
git clone https://github.com/yourorg/provisioning-platform
# 2. Create feature branches
cd provisioning-core
git checkout -b feat/batch-workflow-v2
cd ../provisioning-platform
git checkout -b feat/batch-workflow-v2
# 3. Develop with local linking
cd provisioning-core
just install-dev # Installs to /usr/local/bin/provisioning
cd ../provisioning-platform
# Platform uses system provisioning CLI (local dev version)
cargo run
# 4. Test integration
cd ../provisioning-core
just test-integration
cd ../provisioning-platform
cargo test
# 5. Create PRs in both repositories
# PR #123 in provisioning-core
# PR #456 in provisioning-platform (references core PR)
# 6. Coordinate merge
# Merge core PR first, cut release 3.3.0
# Update platform dependency to core 3.3.0
# Merge platform PR, cut release 2.6.0
```plaintext
### Testing Cross-Repo Integration
```bash
# Integration tests in provisioning-distribution
cd provisioning-distribution
# Test specific version combination
just test-integration \
--core 3.3.0 \
--platform 2.6.0
# Test bundle
just test-bundle stable-3.3
```plaintext
---
## Distribution Strategy
### Individual Repository Releases
Each repository releases independently:
```bash
# Core release
cd provisioning-core
git tag v3.2.1
git push --tags
# → GitHub Actions builds package
# → Publishes to package registry
# Platform release
cd provisioning-platform
git tag v2.5.3
git push --tags
# → GitHub Actions builds binaries
# → Publishes to package registry
```plaintext
### Bundle Releases (Coordinated)
Distribution repository creates tested bundles:
```bash
cd provisioning-distribution
# Create bundle
just create-bundle stable-3.2 \
--core 3.2.1 \
--platform 2.5.3 \
--workspace 1.4.0
# Test bundle
just test-bundle stable-3.2
# Publish bundle
just publish-bundle stable-3.2
# → Creates meta-package with all components
# → Publishes bundle to registry
# → Updates documentation
```plaintext
### User Installation Options
#### Option 1: Bundle Installation (Recommended for Users)
```bash
# Install stable bundle (easiest)
curl -fsSL https://get.provisioning.io | sh
# Installs:
# - provisioning-core 3.2.1
# - provisioning-platform 2.5.3
# - provisioning-workspace 1.4.0
```plaintext
#### Option 2: Individual Component Installation
```bash
# Install only core (minimal)
curl -fsSL https://get.provisioning.io/core | sh
# Add platform later
provisioning install platform
# Add extensions
provisioning extension install mongodb
```plaintext
#### Option 3: Custom Combination
```bash
# Install specific versions
provisioning install core@3.1.0
provisioning install platform@2.4.0
```plaintext
---
## Repository Ownership and Contribution Model
### Core Team Ownership
| Repository | Primary Owner | Contribution Model |
|------------|---------------|-------------------|
| `provisioning-core` | Core Team | Strict review, stable API |
| `provisioning-platform` | Platform Team | Fast iteration, performance focus |
| `provisioning-extensions` | Community + Core | Open contributions, moderated |
| `provisioning-workspace` | Docs Team | Template contributions welcome |
| `provisioning-distribution` | Release Engineering | Core team only |
### Contribution Workflow
**For Core:**
1. Create issue in `provisioning-core`
2. Discuss design
3. Submit PR with tests
4. Strict code review
5. Merge to `main`
6. Release when ready
**For Extensions:**
1. Create extension in `provisioning-extensions`
2. Follow extension guidelines
3. Submit PR
4. Community review
5. Merge and publish to registry
6. Independent versioning
**For Platform:**
1. Create issue in `provisioning-platform`
2. Implement with benchmarks
3. Submit PR
4. Performance review
5. Merge and release
---
## CI/CD Strategy
### Per-Repository CI/CD
**Core CI (`provisioning-core/.github/workflows/ci.yml`):**
```yaml
name: Core CI
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Nushell
run: cargo install nu
- name: Run tests
run: just test
- name: Validate KCL schemas
run: just validate-kcl
package:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v3
- name: Build package
run: just build
- name: Publish to registry
run: just publish
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
```plaintext
**Platform CI (`provisioning-platform/.github/workflows/ci.yml`):**
```yaml
name: Platform CI
on: [push, pull_request]
jobs:
test:
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- name: Build
run: cargo build --release
- name: Test
run: cargo test --workspace
- name: Benchmark
run: cargo bench
cross-compile:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v3
- name: Build for Linux x86_64
run: cargo build --release --target x86_64-unknown-linux-gnu
- name: Build for Linux arm64
run: cargo build --release --target aarch64-unknown-linux-gnu
- name: Publish binaries
run: just publish-binaries
```plaintext
### Integration Testing (Distribution Repo)
**Distribution CI (`provisioning-distribution/.github/workflows/integration.yml`):**
```yaml
name: Integration Tests
on:
schedule:
- cron: '0 0 * * *' # Daily
workflow_dispatch:
jobs:
test-bundle:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install bundle
run: |
nu release-automation/install-bundle.nu stable-3.2
- name: Run integration tests
run: |
nu tests/integration/test-all.nu
- name: Test upgrade path
run: |
nu tests/integration/test-upgrade.nu 3.1.0 3.2.1
```plaintext
---
## File and Directory Structure Comparison
### Monorepo Structure
```plaintext
provisioning/ (One repo, ~500MB)
├── core/ (Nushell)
├── platform/ (Rust)
├── extensions/ (Community)
├── workspace/ (Templates)
└── distribution/ (Build)
```plaintext
### Multi-Repo Structure
```plaintext
provisioning-core/ (Repo 1, ~50MB)
├── nulib/
├── cli/
├── kcl/
└── tools/
provisioning-platform/ (Repo 2, ~150MB with target/)
├── orchestrator/
├── control-center/
├── mcp-server/
└── Cargo.toml
provisioning-extensions/ (Repo 3, ~100MB)
├── registry/
├── providers/
├── taskservs/
└── clusters/
provisioning-workspace/ (Repo 4, ~20MB)
├── templates/
├── examples/
└── blueprints/
provisioning-distribution/ (Repo 5, ~30MB)
├── release-automation/
├── installers/
├── packaging/
└── registry/
```plaintext
---
## Decision Matrix
| Criterion | Monorepo | Multi-Repo |
|-----------|----------|------------|
| **Development Complexity** | Simple | Moderate |
| **Clone Size** | Large (~500MB) | Small (50-150MB each) |
| **Cross-Component Changes** | Easy (atomic) | Moderate (coordinated) |
| **Independent Releases** | Difficult | Easy |
| **Language-Specific Tooling** | Mixed | Clean |
| **Community Contributions** | Harder (big repo) | Easier (focused repos) |
| **Version Management** | Simple (one version) | Complex (matrix) |
| **CI/CD Complexity** | Simple (one pipeline) | Moderate (multiple) |
| **Ownership Clarity** | Unclear | Clear |
| **Extension Ecosystem** | Monolithic | Modular |
| **Build Time** | Long (build all) | Short (build one) |
| **Testing Isolation** | Difficult | Easy |
---
## Recommended Approach: Multi-Repo
### Why Multi-Repo Wins for This Project
1. **Clear Separation of Concerns**
- Nushell core vs Rust platform are different domains
- Different teams can own different repos
- Different release cadences make sense
2. **Language-Specific Tooling**
- `provisioning-core`: Nushell-focused, simple testing
- `provisioning-platform`: Rust workspace, Cargo tooling
- No mixed tooling confusion
3. **Community Contributions**
- Extensions repo is easier to contribute to
- Don't need to clone entire monorepo
- Clearer contribution guidelines per repo
4. **Independent Versioning**
- Core can stay stable (3.x for months)
- Platform can iterate fast (2.x weekly)
- Extensions have own lifecycles
5. **Build Performance**
- Only build what changed
- Faster CI/CD per repo
- Parallel builds across repos
6. **Extension Ecosystem**
- Extensions repo becomes marketplace
- Third-party extensions can live separately
- Registry becomes discovery mechanism
### Implementation Strategy
**Phase 1: Split Repositories (Week 1-2)**
1. Create 5 new repositories
2. Extract code from monorepo
3. Set up CI/CD for each
4. Create initial packages
**Phase 2: Package Integration (Week 3)**
1. Implement package registry
2. Create installers
3. Set up version compatibility matrix
4. Test cross-repo integration
**Phase 3: Distribution System (Week 4)**
1. Implement bundle system
2. Create release automation
3. Set up package hosting
4. Document release process
**Phase 4: Migration (Week 5)**
1. Migrate existing users
2. Update documentation
3. Archive monorepo
4. Announce new structure
---
## Conclusion
**Recommendation: Multi-Repository Architecture with Package-Based Integration**
The multi-repo approach provides:
- ✅ Clear separation between Nushell core and Rust platform
- ✅ Independent release cycles for different components
- ✅ Better community contribution experience
- ✅ Language-specific tooling and workflows
- ✅ Modular extension ecosystem
- ✅ Faster builds and CI/CD
- ✅ Clear ownership boundaries
**Avoid:** Submodules (complexity nightmare)
**Use:** Package-based dependencies with version compatibility matrix
This architecture scales better for your project's growth, supports a community extension ecosystem, and provides professional-grade separation of concerns while maintaining integration through a well-designed package system.
---
## Next Steps
1. **Approve multi-repo strategy**
2. **Create repository split plan**
3. **Set up GitHub organizations/teams**
4. **Implement package registry**
5. **Begin repository extraction**
Would you like me to create a detailed **repository split implementation plan** next?