provisioning/docs/src/development/build-system.md

452 lines
8.8 KiB
Markdown
Raw Normal View History

2026-01-17 03:58:28 +00:00
# Build System
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
Building, testing, and packaging the Provisioning platform and extensions with Cargo, Just, and Nickel.
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
## Build Tools
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
| Tool | Purpose | Version Required |
| ---- | ------- | ---------------- |
| **Cargo** | Rust compilation and testing | Latest stable |
| **Just** | Task runner for common operations | Latest |
| **Nickel** | Schema validation and type checking | 1.15.1+ |
| **Nushell** | Script execution and testing | 0.109.0+ |
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
## Building Platform Services
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Build All Services
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Build all Rust services in release mode
cd provisioning/platform
cargo build --release --workspace
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Or using just task runner
just build-platform
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
Binary outputs in `target/release/`:
- `provisioning-orchestrator`
- `provisioning-control-center`
- `provisioning-vault-service`
- `provisioning-installer`
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Build Individual Service
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Orchestrator service
cd provisioning/platform/crates/orchestrator
cargo build --release
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Control Center service
cd provisioning/platform/crates/control-center
cargo build --release
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Development build (faster compilation)
cargo build
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Testing
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Run All Tests
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Rust unit and integration tests
cargo test --workspace
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Nushell script tests
just test-nushell
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Complete test suite
just test-all
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Test Specific Component
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Test orchestrator crate
cargo test -p provisioning-orchestrator
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Test with output visible
cargo test -p provisioning-orchestrator -- --nocapture
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Test specific function
cargo test -p provisioning-orchestrator test_workflow_creation
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Run tests matching pattern
cargo test workflow
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Security Tests
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Run 350+ security test cases
cargo test -p security --test '*'
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Specific security component
cargo test -p security authentication
cargo test -p security authorization
cargo test -p security kms
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Code Quality
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Formatting
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Format all Rust code
cargo fmt --all
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Check formatting without modifying
cargo fmt --all -- --check
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Format Nickel schemas
nickel fmt provisioning/schemas/**/*.ncl
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Linting
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Run Clippy linter
cargo clippy --all -- -D warnings
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Auto-fix Clippy warnings
cargo clippy --all --fix
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Clippy with all features enabled
cargo clippy --all --all-features -- -D warnings
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Nickel Validation
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Type check Nickel schemas
nickel typecheck provisioning/schemas/main.ncl
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Evaluate schema
nickel eval provisioning/schemas/main.ncl
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Format Nickel files
nickel fmt provisioning/schemas/**/*.ncl
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Continuous Integration
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
The platform uses automated CI workflows for quality assurance.
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### GitHub Actions Pipeline
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
Key CI jobs:
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
```text
1. Rust Build and Test
- cargo build --release --workspace
- cargo test --workspace
- cargo clippy --all -- -D warnings
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
2. Nushell Validation
- nu --check core/cli/provisioning
- Run Nushell test suite
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
3. Nickel Schema Validation
- nickel typecheck schemas/main.ncl
- Validate all schema files
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
4. Security Tests
- Run 350+ security test cases
- Vulnerability scanning
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
5. Documentation Build
- mdbook build docs
- Markdown linting
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Packaging and Distribution
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Create Release Package
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Build optimized binaries
cargo build --release --workspace
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Strip debug symbols (reduce binary size)
strip target/release/provisioning-orchestrator
strip target/release/provisioning-control-center
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Create distribution archive
just package
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Package Structure
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
```text
provisioning-5.0.0-linux-x86_64.tar.gz
├── bin/
│ ├── provisioning # Main CLI
│ ├── provisioning-orchestrator # Orchestrator service
│ ├── provisioning-control-center # Control Center
│ ├── provisioning-vault-service # Vault service
│ └── provisioning-installer # Platform installer
├── lib/
│ └── nulib/ # Nushell libraries
├── schemas/ # Nickel schemas
├── config/
│ └── config.defaults.toml # Default configuration
├── systemd/
│ └── *.service # Systemd unit files
└── README.md
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Cross-Platform Builds
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Supported Targets
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Linux x86_64 (primary platform)
cargo build --release --target x86_64-unknown-linux-gnu
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Linux ARM64 (Raspberry Pi, cloud ARM instances)
cargo build --release --target aarch64-unknown-linux-gnu
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# macOS x86_64
cargo build --release --target x86_64-apple-darwin
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# macOS ARM64 (Apple Silicon)
cargo build --release --target aarch64-apple-darwin
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Cross-Compilation Setup
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Add target architectures
rustup target add x86_64-unknown-linux-gnu
rustup target add aarch64-unknown-linux-gnu
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Install cross-compilation tool
2026-01-14 04:53:21 +00:00
cargo install cross
2026-01-17 03:58:28 +00:00
# Cross-compile with Docker
cross build --release --target aarch64-unknown-linux-gnu
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Just Task Runner
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
Common build tasks in `justfile`:
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
```just
# Build all components
build-all: build-platform build-plugins
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Build platform services
build-platform:
cd platform && cargo build --release --workspace
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Run all tests
test: test-rust test-nushell test-integration
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Test Rust code
test-rust:
cargo test --workspace
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Test Nushell scripts
test-nushell:
nu scripts/test/test_all.nu
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Format all code
fmt:
cargo fmt --all
nickel fmt schemas/**/*.ncl
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Lint all code
lint:
cargo clippy --all -- -D warnings
nickel typecheck schemas/main.ncl
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Create release package
package:
./scripts/package.nu
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Clean build artifacts
clean:
cargo clean
rm -rf target/
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
Usage examples:
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
just build-all # Build everything
just test # Run all tests
just fmt # Format code
just lint # Run linters
just package # Create distribution
just clean # Remove artifacts
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Performance Optimization
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Release Builds
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
```toml
# Cargo.toml
[profile.release]
opt-level = 3 # Maximum optimization
lto = "fat" # Link-time optimization
codegen-units = 1 # Better optimization, slower compile
strip = true # Strip debug symbols
panic = "abort" # Smaller binary size
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Build Time Optimization
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
```toml
# Cargo.toml
[profile.dev]
opt-level = 1 # Basic optimization
incremental = true # Faster recompilation
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
Speed up compilation:
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Use faster linker (Linux)
sudo apt install lld
2026-01-14 04:53:21 +00:00
export RUSTFLAGS="-C link-arg=-fuse-ld=lld"
2026-01-17 03:58:28 +00:00
# Parallel compilation
cargo build -j 8
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Use cargo-watch for auto-rebuild
cargo install cargo-watch
cargo watch -x build
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Development Workflow
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Recommended Workflow
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# 1. Start development
just clean
just build-all
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# 2. Make changes to code
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# 3. Test changes quickly
cargo check # Fast syntax check
cargo test <specific-test> # Test specific functionality
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# 4. Full validation before commit
just fmt
just lint
just test
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# 5. Create package for testing
just package
```
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Hot Reload Development
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Auto-rebuild on file changes
cargo watch -x build
# Auto-test on changes
cargo watch -x test
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Run service with auto-reload
cargo watch -x 'run --bin provisioning-orchestrator'
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Debugging Builds
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Debug Information
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Build with full debug info
cargo build
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Build with debug info in release mode
cargo build --release --profile release-with-debug
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Run with backtraces
RUST_BACKTRACE=1 cargo run
RUST_BACKTRACE=full cargo run
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Build Verbosity
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Verbose build output
cargo build -vv
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Show build commands
cargo build -vvv
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Show timing information
cargo build --timings
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
### Dependency Tree
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# View dependency tree
cargo tree
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Duplicate dependencies
cargo tree --duplicates
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Build graph visualization
cargo depgraph | dot -Tpng > deps.png
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Best Practices
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
- Always run `just test` before committing
- Use `cargo fmt` and `cargo clippy` for code quality
- Test on multiple platforms before release
- Strip binaries for production distributions
- Version binaries with semantic versioning
- Cache dependencies in CI/CD
- Use release profile for production builds
- Document build requirements in README
- Automate common tasks with Just
- Keep build times reasonable (<5 min)
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
## Troubleshooting
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
### Common Build Issues
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
**Compilation fails with linker error:**
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
```bash
# Install build dependencies
sudo apt install build-essential pkg-config libssl-dev
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
**Out of memory during build:**
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Reduce parallel jobs
cargo build -j 2
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Use more swap space
sudo fallocate -l 8G /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
```
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
**Clippy warnings:**
2026-01-14 04:53:21 +00:00
2026-01-14 04:53:58 +00:00
```bash
2026-01-17 03:58:28 +00:00
# Fix automatically where possible
cargo clippy --all --fix
2026-01-14 04:53:21 +00:00
2026-01-17 03:58:28 +00:00
# Allow specific lints temporarily
#[allow(clippy::too_many_arguments)]
2026-01-14 04:53:21 +00:00
```
2026-01-17 03:58:28 +00:00
## Related Documentation
- [Testing](testing.md) - Testing strategies and procedures
- [Contributing](contributing.md) - Contribution guidelines including build requirements