chore: fix docs
This commit is contained in:
Jesús Pérez
parent
f9443f1ea5
commit
2e374043a2
GPG Key ID:
9F243E355E0BC939
File diff suppressed because it is too large
Load Diff
|
Before Width: | Height: | Size: 434 KiB After Width: | Height: | Size: 433 KiB |
@ -143,7 +143,7 @@
|
|||||||
<a href="https://github.com/provisioning/provisioning-platform" title="Git repository" aria-label="Git repository">
|
<a href="https://github.com/provisioning/provisioning-platform" title="Git repository" aria-label="Git repository">
|
||||||
<i id="git-repository-button" class="fa fa-github"></i>
|
<i id="git-repository-button" class="fa fa-github"></i>
|
||||||
</a>
|
</a>
|
||||||
<a href="https://github.com/provisioning/provisioning-platform/edit/main/provisioning/docs/src/architecture/adr/ADR-009-security-system-complete.md" title="Suggest an edit" aria-label="Suggest an edit">
|
<a href="https://github.com/provisioning/provisioning-platform/edit/main/provisioning/docs/src/architecture/adr/adr-009-security-system-complete.md" title="Suggest an edit" aria-label="Suggest an edit">
|
||||||
<i id="git-edit-button" class="fa fa-edit"></i>
|
<i id="git-edit-button" class="fa fa-edit"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
@ -200,7 +200,7 @@
|
|||||||
<p><strong>Features</strong>:</p>
|
<p><strong>Features</strong>:</p>
|
||||||
<ul>
|
<ul>
|
||||||
<li>RS256 asymmetric signing</li>
|
<li>RS256 asymmetric signing</li>
|
||||||
<li>Access tokens (15min) + refresh tokens (7d)</li>
|
<li>Access tokens (15 min) + refresh tokens (7 d)</li>
|
||||||
<li>Token rotation and revocation</li>
|
<li>Token rotation and revocation</li>
|
||||||
<li>Argon2id password hashing</li>
|
<li>Argon2id password hashing</li>
|
||||||
<li>5 user roles (Admin, Developer, Operator, Viewer, Auditor)</li>
|
<li>5 user roles (Admin, Developer, Operator, Viewer, Auditor)</li>
|
||||||
@ -266,7 +266,7 @@
|
|||||||
<p><strong>Location</strong>: <code>provisioning/platform/orchestrator/src/secrets/</code></p>
|
<p><strong>Location</strong>: <code>provisioning/platform/orchestrator/src/secrets/</code></p>
|
||||||
<p><strong>Features</strong>:</p>
|
<p><strong>Features</strong>:</p>
|
||||||
<ul>
|
<ul>
|
||||||
<li>AWS STS temporary credentials (15min-12h)</li>
|
<li>AWS STS temporary credentials (15 min-12 h)</li>
|
||||||
<li>SSH key pair generation (Ed25519)</li>
|
<li>SSH key pair generation (Ed25519)</li>
|
||||||
<li>UpCloud API subaccounts</li>
|
<li>UpCloud API subaccounts</li>
|
||||||
<li>TTL manager with auto-cleanup</li>
|
<li>TTL manager with auto-cleanup</li>
|
||||||
@ -283,7 +283,7 @@
|
|||||||
<li>Vault OTP (one-time passwords)</li>
|
<li>Vault OTP (one-time passwords)</li>
|
||||||
<li>Vault CA (certificate authority signing)</li>
|
<li>Vault CA (certificate authority signing)</li>
|
||||||
<li>Auto-deployment to authorized_keys</li>
|
<li>Auto-deployment to authorized_keys</li>
|
||||||
<li>Background cleanup every 5min</li>
|
<li>Background cleanup every 5 min</li>
|
||||||
</ul>
|
</ul>
|
||||||
<p><strong>API</strong>: 7 endpoints
|
<p><strong>API</strong>: 7 endpoints
|
||||||
<strong>CLI</strong>: 10 commands
|
<strong>CLI</strong>: 10 commands
|
||||||
@ -294,12 +294,12 @@
|
|||||||
<p><strong>Location</strong>: <code>provisioning/platform/control-center/src/mfa/</code></p>
|
<p><strong>Location</strong>: <code>provisioning/platform/control-center/src/mfa/</code></p>
|
||||||
<p><strong>Features</strong>:</p>
|
<p><strong>Features</strong>:</p>
|
||||||
<ul>
|
<ul>
|
||||||
<li>TOTP (RFC 6238, 6-digit codes, 30s window)</li>
|
<li>TOTP (RFC 6238, 6-digit codes, 30 s window)</li>
|
||||||
<li>WebAuthn/FIDO2 (YubiKey, Touch ID, Windows Hello)</li>
|
<li>WebAuthn/FIDO2 (YubiKey, Touch ID, Windows Hello)</li>
|
||||||
<li>QR code generation</li>
|
<li>QR code generation</li>
|
||||||
<li>10 backup codes per user</li>
|
<li>10 backup codes per user</li>
|
||||||
<li>Multiple devices per user</li>
|
<li>Multiple devices per user</li>
|
||||||
<li>Rate limiting (5 attempts/5min)</li>
|
<li>Rate limiting (5 attempts/5 min)</li>
|
||||||
</ul>
|
</ul>
|
||||||
<p><strong>API</strong>: 13 endpoints
|
<p><strong>API</strong>: 13 endpoints
|
||||||
<strong>CLI</strong>: 15 commands
|
<strong>CLI</strong>: 15 commands
|
||||||
@ -338,7 +338,7 @@
|
|||||||
<p><strong>Features</strong>:</p>
|
<p><strong>Features</strong>:</p>
|
||||||
<ul>
|
<ul>
|
||||||
<li>Multi-party approval (2+ approvers, different teams)</li>
|
<li>Multi-party approval (2+ approvers, different teams)</li>
|
||||||
<li>Emergency JWT tokens (4h max, special claims)</li>
|
<li>Emergency JWT tokens (4 h max, special claims)</li>
|
||||||
<li>Auto-revocation (expiration + inactivity)</li>
|
<li>Auto-revocation (expiration + inactivity)</li>
|
||||||
<li>Enhanced audit (7-year retention)</li>
|
<li>Enhanced audit (7-year retention)</li>
|
||||||
<li>Real-time alerts</li>
|
<li>Real-time alerts</li>
|
||||||
@ -368,7 +368,7 @@
|
|||||||
↓
|
↓
|
||||||
2. Rate Limiting (100 req/min per IP)
|
2. Rate Limiting (100 req/min per IP)
|
||||||
↓
|
↓
|
||||||
3. JWT Authentication (RS256, 15min tokens)
|
3. JWT Authentication (RS256, 15 min tokens)
|
||||||
↓
|
↓
|
||||||
4. MFA Verification (TOTP/WebAuthn for sensitive ops)
|
4. MFA Verification (TOTP/WebAuthn for sensitive ops)
|
||||||
↓
|
↓
|
||||||
@ -381,12 +381,9 @@
|
|||||||
8. Audit Logging (structured JSON, GDPR-compliant)
|
8. Audit Logging (structured JSON, GDPR-compliant)
|
||||||
↓
|
↓
|
||||||
9. Response
|
9. Response
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="emergency-access-flow"><a class="header" href="#emergency-access-flow">Emergency Access Flow</a></h3>
|
||||||
### Emergency Access Flow
|
<pre><code class="language-plaintext">1. Emergency Request (reason + justification)
|
||||||
|
|
||||||
```plaintext
|
|
||||||
1. Emergency Request (reason + justification)
|
|
||||||
↓
|
↓
|
||||||
2. Multi-Party Approval (2+ approvers, different teams)
|
2. Multi-Party Approval (2+ approvers, different teams)
|
||||||
↓
|
↓
|
||||||
@ -395,118 +392,93 @@
|
|||||||
4. Enhanced Audit (7-year retention, immutable)
|
4. Enhanced Audit (7-year retention, immutable)
|
||||||
↓
|
↓
|
||||||
5. Auto-Revocation (expiration/inactivity)
|
5. Auto-Revocation (expiration/inactivity)
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<hr />
|
||||||
---
|
<h2 id="technology-stack"><a class="header" href="#technology-stack">Technology Stack</a></h2>
|
||||||
|
<h3 id="backend-rust"><a class="header" href="#backend-rust">Backend (Rust)</a></h3>
|
||||||
## Technology Stack
|
<ul>
|
||||||
|
<li><strong>axum</strong>: HTTP framework</li>
|
||||||
### Backend (Rust)
|
<li><strong>jsonwebtoken</strong>: JWT handling (RS256)</li>
|
||||||
|
<li><strong>cedar-policy</strong>: Authorization engine</li>
|
||||||
- **axum**: HTTP framework
|
<li><strong>totp-rs</strong>: TOTP implementation</li>
|
||||||
- **jsonwebtoken**: JWT handling (RS256)
|
<li><strong>webauthn-rs</strong>: WebAuthn/FIDO2</li>
|
||||||
- **cedar-policy**: Authorization engine
|
<li><strong>aws-sdk-kms</strong>: AWS KMS integration</li>
|
||||||
- **totp-rs**: TOTP implementation
|
<li><strong>argon2</strong>: Password hashing</li>
|
||||||
- **webauthn-rs**: WebAuthn/FIDO2
|
<li><strong>tracing</strong>: Structured logging</li>
|
||||||
- **aws-sdk-kms**: AWS KMS integration
|
</ul>
|
||||||
- **argon2**: Password hashing
|
<h3 id="frontend-typescriptreact"><a class="header" href="#frontend-typescriptreact">Frontend (TypeScript/React)</a></h3>
|
||||||
- **tracing**: Structured logging
|
<ul>
|
||||||
|
<li><strong>React 18</strong>: UI framework</li>
|
||||||
### Frontend (TypeScript/React)
|
<li><strong>Leptos</strong>: Rust WASM framework</li>
|
||||||
|
<li><strong>@simplewebauthn/browser</strong>: WebAuthn client</li>
|
||||||
- **React 18**: UI framework
|
<li><strong>qrcode.react</strong>: QR code generation</li>
|
||||||
- **Leptos**: Rust WASM framework
|
</ul>
|
||||||
- **@simplewebauthn/browser**: WebAuthn client
|
<h3 id="cli-nushell"><a class="header" href="#cli-nushell">CLI (Nushell)</a></h3>
|
||||||
- **qrcode.react**: QR code generation
|
<ul>
|
||||||
|
<li><strong>Nushell 0.107</strong>: Shell and scripting</li>
|
||||||
### CLI (Nushell)
|
<li><strong>nu_plugin_kcl</strong>: KCL integration</li>
|
||||||
|
</ul>
|
||||||
- **Nushell 0.107**: Shell and scripting
|
<h3 id="infrastructure"><a class="header" href="#infrastructure">Infrastructure</a></h3>
|
||||||
- **nu_plugin_kcl**: KCL integration
|
<ul>
|
||||||
|
<li><strong>HashiCorp Vault</strong>: Secrets management, KMS, SSH CA</li>
|
||||||
### Infrastructure
|
<li><strong>AWS KMS</strong>: Key management service</li>
|
||||||
|
<li><strong>PostgreSQL/SurrealDB</strong>: Data storage</li>
|
||||||
- **HashiCorp Vault**: Secrets management, KMS, SSH CA
|
<li><strong>SOPS</strong>: Config encryption</li>
|
||||||
- **AWS KMS**: Key management service
|
</ul>
|
||||||
- **PostgreSQL/SurrealDB**: Data storage
|
<hr />
|
||||||
- **SOPS**: Config encryption
|
<h2 id="security-guarantees"><a class="header" href="#security-guarantees">Security Guarantees</a></h2>
|
||||||
|
<h3 id="authentication"><a class="header" href="#authentication">Authentication</a></h3>
|
||||||
---
|
<p>✅ RS256 asymmetric signing (no shared secrets)
|
||||||
|
✅ Short-lived access tokens (15 min)
|
||||||
## Security Guarantees
|
|
||||||
|
|
||||||
### Authentication
|
|
||||||
|
|
||||||
✅ RS256 asymmetric signing (no shared secrets)
|
|
||||||
✅ Short-lived access tokens (15min)
|
|
||||||
✅ Token revocation support
|
✅ Token revocation support
|
||||||
✅ Argon2id password hashing (memory-hard)
|
✅ Argon2id password hashing (memory-hard)
|
||||||
✅ MFA enforced for production operations
|
✅ MFA enforced for production operations</p>
|
||||||
|
<h3 id="authorization"><a class="header" href="#authorization">Authorization</a></h3>
|
||||||
### Authorization
|
<p>✅ Fine-grained permissions (Cedar policies)
|
||||||
|
|
||||||
✅ Fine-grained permissions (Cedar policies)
|
|
||||||
✅ Context-aware (MFA, IP, time windows)
|
✅ Context-aware (MFA, IP, time windows)
|
||||||
✅ Hot reload policies (no downtime)
|
✅ Hot reload policies (no downtime)
|
||||||
✅ Deny by default
|
✅ Deny by default</p>
|
||||||
|
<h3 id="secrets-management"><a class="header" href="#secrets-management">Secrets Management</a></h3>
|
||||||
### Secrets Management
|
<p>✅ No static credentials stored
|
||||||
|
|
||||||
✅ No static credentials stored
|
|
||||||
✅ Time-limited secrets (1h default)
|
✅ Time-limited secrets (1h default)
|
||||||
✅ Auto-revocation on expiry
|
✅ Auto-revocation on expiry
|
||||||
✅ Encryption at rest (KMS)
|
✅ Encryption at rest (KMS)
|
||||||
✅ Memory-only decryption
|
✅ Memory-only decryption</p>
|
||||||
|
<h3 id="audit--compliance"><a class="header" href="#audit--compliance">Audit & Compliance</a></h3>
|
||||||
### Audit & Compliance
|
<p>✅ Immutable audit logs
|
||||||
|
|
||||||
✅ Immutable audit logs
|
|
||||||
✅ GDPR-compliant (PII anonymization)
|
✅ GDPR-compliant (PII anonymization)
|
||||||
✅ SOC2 controls implemented
|
✅ SOC2 controls implemented
|
||||||
✅ ISO 27001 controls verified
|
✅ ISO 27001 controls verified
|
||||||
✅ 7-year retention for break-glass
|
✅ 7-year retention for break-glass</p>
|
||||||
|
<h3 id="emergency-access"><a class="header" href="#emergency-access">Emergency Access</a></h3>
|
||||||
### Emergency Access
|
<p>✅ Multi-party approval required
|
||||||
|
|
||||||
✅ Multi-party approval required
|
|
||||||
✅ Time-limited sessions (4h max)
|
✅ Time-limited sessions (4h max)
|
||||||
✅ Enhanced audit logging
|
✅ Enhanced audit logging
|
||||||
✅ Auto-revocation
|
✅ Auto-revocation
|
||||||
✅ Cannot be disabled
|
✅ Cannot be disabled</p>
|
||||||
|
<hr />
|
||||||
---
|
<h2 id="performance-characteristics"><a class="header" href="#performance-characteristics">Performance Characteristics</a></h2>
|
||||||
|
<div class="table-wrapper"><table><thead><tr><th>Component</th><th>Latency</th><th>Throughput</th><th>Memory</th></tr></thead><tbody>
|
||||||
## Performance Characteristics
|
<tr><td>JWT Auth</td><td><5 ms</td><td>10,000/s</td><td>~10 MB</td></tr>
|
||||||
|
<tr><td>Cedar Authz</td><td><10 ms</td><td>5,000/s</td><td>~50 MB</td></tr>
|
||||||
| Component | Latency | Throughput | Memory |
|
<tr><td>Audit Log</td><td><5 ms</td><td>20,000/s</td><td>~100 MB</td></tr>
|
||||||
|-----------|---------|------------|--------|
|
<tr><td>KMS Encrypt</td><td><50 ms</td><td>1,000/s</td><td>~20 MB</td></tr>
|
||||||
| JWT Auth | <5ms | 10,000/s | ~10MB |
|
<tr><td>Dynamic Secrets</td><td><100 ms</td><td>500/s</td><td>~50 MB</td></tr>
|
||||||
| Cedar Authz | <10ms | 5,000/s | ~50MB |
|
<tr><td>MFA Verify</td><td><50 ms</td><td>2,000/s</td><td>~30 MB</td></tr>
|
||||||
| Audit Log | <5ms | 20,000/s | ~100MB |
|
</tbody></table>
|
||||||
| KMS Encrypt | <50ms | 1,000/s | ~20MB |
|
</div>
|
||||||
| Dynamic Secrets | <100ms | 500/s | ~50MB |
|
<p><strong>Total Overhead</strong>: ~10-20 ms per request
|
||||||
| MFA Verify | <50ms | 2,000/s | ~30MB |
|
<strong>Memory Usage</strong>: ~260 MB total for all security components</p>
|
||||||
|
<hr />
|
||||||
**Total Overhead**: ~10-20ms per request
|
<h2 id="deployment-options"><a class="header" href="#deployment-options">Deployment Options</a></h2>
|
||||||
**Memory Usage**: ~260MB total for all security components
|
<h3 id="development"><a class="header" href="#development">Development</a></h3>
|
||||||
|
<pre><code class="language-bash"># Start all services
|
||||||
---
|
|
||||||
|
|
||||||
## Deployment Options
|
|
||||||
|
|
||||||
### Development
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Start all services
|
|
||||||
cd provisioning/platform/kms-service && cargo run &
|
cd provisioning/platform/kms-service && cargo run &
|
||||||
cd provisioning/platform/orchestrator && cargo run &
|
cd provisioning/platform/orchestrator && cargo run &
|
||||||
cd provisioning/platform/control-center && cargo run &
|
cd provisioning/platform/control-center && cargo run &
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="production"><a class="header" href="#production">Production</a></h3>
|
||||||
### Production
|
<pre><code class="language-bash"># Kubernetes deployment
|
||||||
|
|
||||||
```bash
|
|
||||||
# Kubernetes deployment
|
|
||||||
kubectl apply -f k8s/security-stack.yaml
|
kubectl apply -f k8s/security-stack.yaml
|
||||||
|
|
||||||
# Docker Compose
|
# Docker Compose
|
||||||
@ -516,16 +488,11 @@ docker-compose up -d kms orchestrator control-center
|
|||||||
systemctl start provisioning-kms
|
systemctl start provisioning-kms
|
||||||
systemctl start provisioning-orchestrator
|
systemctl start provisioning-orchestrator
|
||||||
systemctl start provisioning-control-center
|
systemctl start provisioning-control-center
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<hr />
|
||||||
---
|
<h2 id="configuration"><a class="header" href="#configuration">Configuration</a></h2>
|
||||||
|
<h3 id="environment-variables"><a class="header" href="#environment-variables">Environment Variables</a></h3>
|
||||||
## Configuration
|
<pre><code class="language-bash"># JWT
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# JWT
|
|
||||||
export JWT_ISSUER="control-center"
|
export JWT_ISSUER="control-center"
|
||||||
export JWT_AUDIENCE="orchestrator,cli"
|
export JWT_AUDIENCE="orchestrator,cli"
|
||||||
export JWT_PRIVATE_KEY_PATH="/keys/private.pem"
|
export JWT_PRIVATE_KEY_PATH="/keys/private.pem"
|
||||||
@ -543,12 +510,9 @@ export VAULT_TOKEN="..."
|
|||||||
# MFA
|
# MFA
|
||||||
export MFA_TOTP_ISSUER="Provisioning"
|
export MFA_TOTP_ISSUER="Provisioning"
|
||||||
export MFA_WEBAUTHN_RP_ID="provisioning.example.com"
|
export MFA_WEBAUTHN_RP_ID="provisioning.example.com"
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="config-files"><a class="header" href="#config-files">Config Files</a></h3>
|
||||||
### Config Files
|
<pre><code class="language-toml"># provisioning/config/security.toml
|
||||||
|
|
||||||
```toml
|
|
||||||
# provisioning/config/security.toml
|
|
||||||
[jwt]
|
[jwt]
|
||||||
issuer = "control-center"
|
issuer = "control-center"
|
||||||
audience = ["orchestrator", "cli"]
|
audience = ["orchestrator", "cli"]
|
||||||
@ -576,16 +540,11 @@ retention_days = 365
|
|||||||
retention_break_glass_days = 2555 # 7 years
|
retention_break_glass_days = 2555 # 7 years
|
||||||
export_format = "json"
|
export_format = "json"
|
||||||
pii_anonymization = true
|
pii_anonymization = true
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<hr />
|
||||||
---
|
<h2 id="testing"><a class="header" href="#testing">Testing</a></h2>
|
||||||
|
<h3 id="run-all-tests"><a class="header" href="#run-all-tests">Run All Tests</a></h3>
|
||||||
## Testing
|
<pre><code class="language-bash"># Control Center (JWT, MFA)
|
||||||
|
|
||||||
### Run All Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Control Center (JWT, MFA)
|
|
||||||
cd provisioning/platform/control-center
|
cd provisioning/platform/control-center
|
||||||
cargo test
|
cargo test
|
||||||
|
|
||||||
@ -599,187 +558,179 @@ cargo test
|
|||||||
|
|
||||||
# Config Encryption (Nushell)
|
# Config Encryption (Nushell)
|
||||||
nu provisioning/core/nulib/lib_provisioning/config/encryption_tests.nu
|
nu provisioning/core/nulib/lib_provisioning/config/encryption_tests.nu
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="integration-tests"><a class="header" href="#integration-tests">Integration Tests</a></h3>
|
||||||
### Integration Tests
|
<pre><code class="language-bash"># Full security flow
|
||||||
|
|
||||||
```bash
|
|
||||||
# Full security flow
|
|
||||||
cd provisioning/platform/orchestrator
|
cd provisioning/platform/orchestrator
|
||||||
cargo test --test security_integration_tests
|
cargo test --test security_integration_tests
|
||||||
cargo test --test break_glass_integration_tests
|
cargo test --test break_glass_integration_tests
|
||||||
```plaintext
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Monitoring & Alerts
|
|
||||||
|
|
||||||
### Metrics to Monitor
|
|
||||||
|
|
||||||
- Authentication failures (rate, sources)
|
|
||||||
- Authorization denials (policies, resources)
|
|
||||||
- MFA failures (attempts, users)
|
|
||||||
- Token revocations (rate, reasons)
|
|
||||||
- Break-glass activations (frequency, duration)
|
|
||||||
- Secrets generation (rate, types)
|
|
||||||
- Audit log volume (events/sec)
|
|
||||||
|
|
||||||
### Alerts to Configure
|
|
||||||
|
|
||||||
- Multiple failed auth attempts (5+ in 5min)
|
|
||||||
- Break-glass session created
|
|
||||||
- Compliance report non-compliant
|
|
||||||
- Incident severity critical/high
|
|
||||||
- Token revocation spike
|
|
||||||
- KMS errors
|
|
||||||
- Audit log export failures
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Maintenance
|
|
||||||
|
|
||||||
### Daily
|
|
||||||
|
|
||||||
- Monitor audit logs for anomalies
|
|
||||||
- Review failed authentication attempts
|
|
||||||
- Check break-glass sessions (should be zero)
|
|
||||||
|
|
||||||
### Weekly
|
|
||||||
|
|
||||||
- Review compliance reports
|
|
||||||
- Check incident response status
|
|
||||||
- Verify backup code usage
|
|
||||||
- Review MFA device additions/removals
|
|
||||||
|
|
||||||
### Monthly
|
|
||||||
|
|
||||||
- Rotate KMS keys
|
|
||||||
- Review and update Cedar policies
|
|
||||||
- Generate compliance reports (GDPR, SOC2, ISO)
|
|
||||||
- Audit access control matrix
|
|
||||||
|
|
||||||
### Quarterly
|
|
||||||
|
|
||||||
- Full security audit
|
|
||||||
- Penetration testing
|
|
||||||
- Compliance certification review
|
|
||||||
- Update security documentation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Migration Path
|
|
||||||
|
|
||||||
### From Existing System
|
|
||||||
|
|
||||||
1. **Phase 1**: Deploy security infrastructure
|
|
||||||
- KMS service
|
|
||||||
- Orchestrator with auth middleware
|
|
||||||
- Control Center
|
|
||||||
|
|
||||||
2. **Phase 2**: Migrate authentication
|
|
||||||
- Enable JWT authentication
|
|
||||||
- Migrate existing users
|
|
||||||
- Disable old auth system
|
|
||||||
|
|
||||||
3. **Phase 3**: Enable MFA
|
|
||||||
- Require MFA enrollment for admins
|
|
||||||
- Gradual rollout to all users
|
|
||||||
|
|
||||||
4. **Phase 4**: Enable Cedar authorization
|
|
||||||
- Deploy initial policies (permissive)
|
|
||||||
- Monitor authorization decisions
|
|
||||||
- Tighten policies incrementally
|
|
||||||
|
|
||||||
5. **Phase 5**: Enable advanced features
|
|
||||||
- Break-glass procedures
|
|
||||||
- Compliance reporting
|
|
||||||
- Incident response
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Future Enhancements
|
|
||||||
|
|
||||||
### Planned (Not Implemented)
|
|
||||||
|
|
||||||
- **Hardware Security Module (HSM)** integration
|
|
||||||
- **OAuth2/OIDC** federation
|
|
||||||
- **SAML SSO** for enterprise
|
|
||||||
- **Risk-based authentication** (IP reputation, device fingerprinting)
|
|
||||||
- **Behavioral analytics** (anomaly detection)
|
|
||||||
- **Zero-Trust Network** (service mesh integration)
|
|
||||||
|
|
||||||
### Under Consideration
|
|
||||||
|
|
||||||
- **Blockchain audit log** (immutable append-only log)
|
|
||||||
- **Quantum-resistant cryptography** (post-quantum algorithms)
|
|
||||||
- **Confidential computing** (SGX/SEV enclaves)
|
|
||||||
- **Distributed break-glass** (multi-region approval)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Consequences
|
|
||||||
|
|
||||||
### Positive
|
|
||||||
|
|
||||||
✅ **Enterprise-grade security** meeting GDPR, SOC2, ISO 27001
|
|
||||||
✅ **Zero static credentials** (all dynamic, time-limited)
|
|
||||||
✅ **Complete audit trail** (immutable, GDPR-compliant)
|
|
||||||
✅ **MFA-enforced** for sensitive operations
|
|
||||||
✅ **Emergency access** with enhanced controls
|
|
||||||
✅ **Fine-grained authorization** (Cedar policies)
|
|
||||||
✅ **Automated compliance** (reports, incident response)
|
|
||||||
|
|
||||||
### Negative
|
|
||||||
|
|
||||||
⚠️ **Increased complexity** (12 components to manage)
|
|
||||||
⚠️ **Performance overhead** (~10-20ms per request)
|
|
||||||
⚠️ **Memory footprint** (~260MB additional)
|
|
||||||
⚠️ **Learning curve** (Cedar policy language, MFA setup)
|
|
||||||
⚠️ **Operational overhead** (key rotation, policy updates)
|
|
||||||
|
|
||||||
### Mitigations
|
|
||||||
|
|
||||||
- Comprehensive documentation (ADRs, guides, API docs)
|
|
||||||
- CLI commands for all operations
|
|
||||||
- Automated monitoring and alerting
|
|
||||||
- Gradual rollout with feature flags
|
|
||||||
- Training materials for operators
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Related Documentation
|
|
||||||
|
|
||||||
- **JWT Auth**: `docs/architecture/JWT_AUTH_IMPLEMENTATION.md`
|
|
||||||
- **Cedar Authz**: `docs/architecture/CEDAR_AUTHORIZATION_IMPLEMENTATION.md`
|
|
||||||
- **Audit Logging**: `docs/architecture/AUDIT_LOGGING_IMPLEMENTATION.md`
|
|
||||||
- **MFA**: `docs/architecture/MFA_IMPLEMENTATION_SUMMARY.md`
|
|
||||||
- **Break-Glass**: `docs/architecture/BREAK_GLASS_IMPLEMENTATION_SUMMARY.md`
|
|
||||||
- **Compliance**: `docs/architecture/COMPLIANCE_IMPLEMENTATION_SUMMARY.md`
|
|
||||||
- **Config Encryption**: `docs/user/CONFIG_ENCRYPTION_GUIDE.md`
|
|
||||||
- **Dynamic Secrets**: `docs/user/DYNAMIC_SECRETS_QUICK_REFERENCE.md`
|
|
||||||
- **SSH Keys**: `docs/user/SSH_TEMPORAL_KEYS_USER_GUIDE.md`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Approval
|
|
||||||
|
|
||||||
**Architecture Team**: Approved
|
|
||||||
**Security Team**: Approved (pending penetration test)
|
|
||||||
**Compliance Team**: Approved (pending audit)
|
|
||||||
**Engineering Team**: Approved
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Date**: 2025-10-08
|
|
||||||
**Version**: 1.0.0
|
|
||||||
**Status**: Implemented and Production-Ready
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
|
<hr />
|
||||||
|
<h2 id="monitoring--alerts"><a class="header" href="#monitoring--alerts">Monitoring & Alerts</a></h2>
|
||||||
|
<h3 id="metrics-to-monitor"><a class="header" href="#metrics-to-monitor">Metrics to Monitor</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Authentication failures (rate, sources)</li>
|
||||||
|
<li>Authorization denials (policies, resources)</li>
|
||||||
|
<li>MFA failures (attempts, users)</li>
|
||||||
|
<li>Token revocations (rate, reasons)</li>
|
||||||
|
<li>Break-glass activations (frequency, duration)</li>
|
||||||
|
<li>Secrets generation (rate, types)</li>
|
||||||
|
<li>Audit log volume (events/sec)</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="alerts-to-configure"><a class="header" href="#alerts-to-configure">Alerts to Configure</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Multiple failed auth attempts (5+ in 5 min)</li>
|
||||||
|
<li>Break-glass session created</li>
|
||||||
|
<li>Compliance report non-compliant</li>
|
||||||
|
<li>Incident severity critical/high</li>
|
||||||
|
<li>Token revocation spike</li>
|
||||||
|
<li>KMS errors</li>
|
||||||
|
<li>Audit log export failures</li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="maintenance"><a class="header" href="#maintenance">Maintenance</a></h2>
|
||||||
|
<h3 id="daily"><a class="header" href="#daily">Daily</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Monitor audit logs for anomalies</li>
|
||||||
|
<li>Review failed authentication attempts</li>
|
||||||
|
<li>Check break-glass sessions (should be zero)</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="weekly"><a class="header" href="#weekly">Weekly</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Review compliance reports</li>
|
||||||
|
<li>Check incident response status</li>
|
||||||
|
<li>Verify backup code usage</li>
|
||||||
|
<li>Review MFA device additions/removals</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="monthly"><a class="header" href="#monthly">Monthly</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Rotate KMS keys</li>
|
||||||
|
<li>Review and update Cedar policies</li>
|
||||||
|
<li>Generate compliance reports (GDPR, SOC2, ISO)</li>
|
||||||
|
<li>Audit access control matrix</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="quarterly"><a class="header" href="#quarterly">Quarterly</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Full security audit</li>
|
||||||
|
<li>Penetration testing</li>
|
||||||
|
<li>Compliance certification review</li>
|
||||||
|
<li>Update security documentation</li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="migration-path"><a class="header" href="#migration-path">Migration Path</a></h2>
|
||||||
|
<h3 id="from-existing-system"><a class="header" href="#from-existing-system">From Existing System</a></h3>
|
||||||
|
<ol>
|
||||||
|
<li>
|
||||||
|
<p><strong>Phase 1</strong>: Deploy security infrastructure</p>
|
||||||
|
<ul>
|
||||||
|
<li>KMS service</li>
|
||||||
|
<li>Orchestrator with auth middleware</li>
|
||||||
|
<li>Control Center</li>
|
||||||
|
</ul>
|
||||||
|
</li>
|
||||||
|
<li>
|
||||||
|
<p><strong>Phase 2</strong>: Migrate authentication</p>
|
||||||
|
<ul>
|
||||||
|
<li>Enable JWT authentication</li>
|
||||||
|
<li>Migrate existing users</li>
|
||||||
|
<li>Disable old auth system</li>
|
||||||
|
</ul>
|
||||||
|
</li>
|
||||||
|
<li>
|
||||||
|
<p><strong>Phase 3</strong>: Enable MFA</p>
|
||||||
|
<ul>
|
||||||
|
<li>Require MFA enrollment for admins</li>
|
||||||
|
<li>Gradual rollout to all users</li>
|
||||||
|
</ul>
|
||||||
|
</li>
|
||||||
|
<li>
|
||||||
|
<p><strong>Phase 4</strong>: Enable Cedar authorization</p>
|
||||||
|
<ul>
|
||||||
|
<li>Deploy initial policies (permissive)</li>
|
||||||
|
<li>Monitor authorization decisions</li>
|
||||||
|
<li>Tighten policies incrementally</li>
|
||||||
|
</ul>
|
||||||
|
</li>
|
||||||
|
<li>
|
||||||
|
<p><strong>Phase 5</strong>: Enable advanced features</p>
|
||||||
|
<ul>
|
||||||
|
<li>Break-glass procedures</li>
|
||||||
|
<li>Compliance reporting</li>
|
||||||
|
<li>Incident response</li>
|
||||||
|
</ul>
|
||||||
|
</li>
|
||||||
|
</ol>
|
||||||
|
<hr />
|
||||||
|
<h2 id="future-enhancements"><a class="header" href="#future-enhancements">Future Enhancements</a></h2>
|
||||||
|
<h3 id="planned-not-implemented"><a class="header" href="#planned-not-implemented">Planned (Not Implemented)</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Hardware Security Module (HSM)</strong> integration</li>
|
||||||
|
<li><strong>OAuth2/OIDC</strong> federation</li>
|
||||||
|
<li><strong>SAML SSO</strong> for enterprise</li>
|
||||||
|
<li><strong>Risk-based authentication</strong> (IP reputation, device fingerprinting)</li>
|
||||||
|
<li><strong>Behavioral analytics</strong> (anomaly detection)</li>
|
||||||
|
<li><strong>Zero-Trust Network</strong> (service mesh integration)</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="under-consideration"><a class="header" href="#under-consideration">Under Consideration</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Blockchain audit log</strong> (immutable append-only log)</li>
|
||||||
|
<li><strong>Quantum-resistant cryptography</strong> (post-quantum algorithms)</li>
|
||||||
|
<li><strong>Confidential computing</strong> (SGX/SEV enclaves)</li>
|
||||||
|
<li><strong>Distributed break-glass</strong> (multi-region approval)</li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="consequences"><a class="header" href="#consequences">Consequences</a></h2>
|
||||||
|
<h3 id="positive"><a class="header" href="#positive">Positive</a></h3>
|
||||||
|
<p>✅ <strong>Enterprise-grade security</strong> meeting GDPR, SOC2, ISO 27001
|
||||||
|
✅ <strong>Zero static credentials</strong> (all dynamic, time-limited)
|
||||||
|
✅ <strong>Complete audit trail</strong> (immutable, GDPR-compliant)
|
||||||
|
✅ <strong>MFA-enforced</strong> for sensitive operations
|
||||||
|
✅ <strong>Emergency access</strong> with enhanced controls
|
||||||
|
✅ <strong>Fine-grained authorization</strong> (Cedar policies)
|
||||||
|
✅ <strong>Automated compliance</strong> (reports, incident response)</p>
|
||||||
|
<h3 id="negative"><a class="header" href="#negative">Negative</a></h3>
|
||||||
|
<p>⚠️ <strong>Increased complexity</strong> (12 components to manage)
|
||||||
|
⚠️ <strong>Performance overhead</strong> (~10-20 ms per request)
|
||||||
|
⚠️ <strong>Memory footprint</strong> (~260 MB additional)
|
||||||
|
⚠️ <strong>Learning curve</strong> (Cedar policy language, MFA setup)
|
||||||
|
⚠️ <strong>Operational overhead</strong> (key rotation, policy updates)</p>
|
||||||
|
<h3 id="mitigations"><a class="header" href="#mitigations">Mitigations</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Comprehensive documentation (ADRs, guides, API docs)</li>
|
||||||
|
<li>CLI commands for all operations</li>
|
||||||
|
<li>Automated monitoring and alerting</li>
|
||||||
|
<li>Gradual rollout with feature flags</li>
|
||||||
|
<li>Training materials for operators</li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="related-documentation"><a class="header" href="#related-documentation">Related Documentation</a></h2>
|
||||||
|
<ul>
|
||||||
|
<li><strong>JWT Auth</strong>: <code>docs/architecture/JWT_AUTH_IMPLEMENTATION.md</code></li>
|
||||||
|
<li><strong>Cedar Authz</strong>: <code>docs/architecture/CEDAR_AUTHORIZATION_IMPLEMENTATION.md</code></li>
|
||||||
|
<li><strong>Audit Logging</strong>: <code>docs/architecture/AUDIT_LOGGING_IMPLEMENTATION.md</code></li>
|
||||||
|
<li><strong>MFA</strong>: <code>docs/architecture/MFA_IMPLEMENTATION_SUMMARY.md</code></li>
|
||||||
|
<li><strong>Break-Glass</strong>: <code>docs/architecture/BREAK_GLASS_IMPLEMENTATION_SUMMARY.md</code></li>
|
||||||
|
<li><strong>Compliance</strong>: <code>docs/architecture/COMPLIANCE_IMPLEMENTATION_SUMMARY.md</code></li>
|
||||||
|
<li><strong>Config Encryption</strong>: <code>docs/user/CONFIG_ENCRYPTION_GUIDE.md</code></li>
|
||||||
|
<li><strong>Dynamic Secrets</strong>: <code>docs/user/DYNAMIC_SECRETS_QUICK_REFERENCE.md</code></li>
|
||||||
|
<li><strong>SSH Keys</strong>: <code>docs/user/SSH_TEMPORAL_KEYS_USER_GUIDE.md</code></li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="approval"><a class="header" href="#approval">Approval</a></h2>
|
||||||
|
<p><strong>Architecture Team</strong>: Approved
|
||||||
|
<strong>Security Team</strong>: Approved (pending penetration test)
|
||||||
|
<strong>Compliance Team</strong>: Approved (pending audit)
|
||||||
|
<strong>Engineering Team</strong>: Approved</p>
|
||||||
|
<hr />
|
||||||
|
<p><strong>Date</strong>: 2025-10-08
|
||||||
|
<strong>Version</strong>: 1.0.0
|
||||||
|
<strong>Status</strong>: Implemented and Production-Ready</p>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
<nav class="nav-wrapper" aria-label="Page navigation">
|
<nav class="nav-wrapper" aria-label="Page navigation">
|
||||||
<!-- Mobile navigation buttons -->
|
<!-- Mobile navigation buttons -->
|
||||||
<a rel="prev" href="../../architecture/adr/ADR-008-cedar-authorization.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
<a rel="prev" href="../../architecture/adr/adr-008-cedar-authorization.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
<i class="fa fa-angle-left"></i>
|
<i class="fa fa-angle-left"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
@ -793,7 +744,7 @@ cargo test --test break_glass_integration_tests
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<nav class="nav-wide-wrapper" aria-label="Page navigation">
|
<nav class="nav-wide-wrapper" aria-label="Page navigation">
|
||||||
<a rel="prev" href="../../architecture/adr/ADR-008-cedar-authorization.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
<a rel="prev" href="../../architecture/adr/adr-008-cedar-authorization.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
<i class="fa fa-angle-left"></i>
|
<i class="fa fa-angle-left"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@ -235,28 +235,21 @@
|
|||||||
│ - Access security context │
|
│ - Access security context │
|
||||||
│ - Execute business logic │
|
│ - Execute business logic │
|
||||||
└────────────────────────────────┘
|
└────────────────────────────────┘
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="implementation-details"><a class="header" href="#implementation-details">Implementation Details</a></h2>
|
||||||
## Implementation Details
|
<h3 id="1-security-context-builder-middlewaresecurity_contextrs"><a class="header" href="#1-security-context-builder-middlewaresecurity_contextrs">1. Security Context Builder (<code>middleware/security_context.rs</code>)</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: Build complete security context from authenticated requests.</p>
|
||||||
### 1. Security Context Builder (`middleware/security_context.rs`)
|
<p><strong>Key Features</strong>:</p>
|
||||||
|
<ul>
|
||||||
**Purpose**: Build complete security context from authenticated requests.
|
<li>Extracts JWT token claims</li>
|
||||||
|
<li>Determines MFA verification status</li>
|
||||||
**Key Features**:
|
<li>Extracts IP address (X-Forwarded-For, X-Real-IP)</li>
|
||||||
|
<li>Extracts user agent and session info</li>
|
||||||
- Extracts JWT token claims
|
<li>Provides permission checking methods</li>
|
||||||
- Determines MFA verification status
|
</ul>
|
||||||
- Extracts IP address (X-Forwarded-For, X-Real-IP)
|
<p><strong>Lines of Code</strong>: 275</p>
|
||||||
- Extracts user agent and session info
|
<p><strong>Example</strong>:</p>
|
||||||
- Provides permission checking methods
|
<pre><code class="language-rust">pub struct SecurityContext {
|
||||||
|
|
||||||
**Lines of Code**: 275
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct SecurityContext {
|
|
||||||
pub user_id: String,
|
pub user_id: String,
|
||||||
pub token: ValidatedToken,
|
pub token: ValidatedToken,
|
||||||
pub mfa_verified: bool,
|
pub mfa_verified: bool,
|
||||||
@ -272,162 +265,123 @@ impl SecurityContext {
|
|||||||
pub fn has_permission(&self, permission: &str) -> bool { ... }
|
pub fn has_permission(&self, permission: &str) -> bool { ... }
|
||||||
pub fn has_any_permission(&self, permissions: &[&str]) -> bool { ... }
|
pub fn has_any_permission(&self, permissions: &[&str]) -> bool { ... }
|
||||||
pub fn has_all_permissions(&self, permissions: &[&str]) -> bool { ... }
|
pub fn has_all_permissions(&self, permissions: &[&str]) -> bool { ... }
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<h3 id="2-enhanced-authentication-middleware-middlewareauthrs"><a class="header" href="#2-enhanced-authentication-middleware-middlewareauthrs">2. Enhanced Authentication Middleware (<code>middleware/auth.rs</code>)</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: JWT token validation with revocation checking.</p>
|
||||||
### 2. Enhanced Authentication Middleware (`middleware/auth.rs`)
|
<p><strong>Key Features</strong>:</p>
|
||||||
|
<ul>
|
||||||
**Purpose**: JWT token validation with revocation checking.
|
<li>Bearer token extraction</li>
|
||||||
|
<li>JWT signature validation (RS256)</li>
|
||||||
**Key Features**:
|
<li>Expiry, issuer, audience checks</li>
|
||||||
|
<li>Token revocation status</li>
|
||||||
- Bearer token extraction
|
<li>Security context injection</li>
|
||||||
- JWT signature validation (RS256)
|
</ul>
|
||||||
- Expiry, issuer, audience checks
|
<p><strong>Lines of Code</strong>: 245</p>
|
||||||
- Token revocation status
|
<p><strong>Flow</strong>:</p>
|
||||||
- Security context injection
|
<ol>
|
||||||
|
<li>Extract <code>Authorization: Bearer <token></code> header</li>
|
||||||
**Lines of Code**: 245
|
<li>Validate JWT with TokenValidator</li>
|
||||||
|
<li>Build SecurityContext</li>
|
||||||
**Flow**:
|
<li>Inject into request extensions</li>
|
||||||
|
<li>Continue to next middleware or return 401</li>
|
||||||
1. Extract `Authorization: Bearer <token>` header
|
</ol>
|
||||||
2. Validate JWT with TokenValidator
|
<p><strong>Error Responses</strong>:</p>
|
||||||
3. Build SecurityContext
|
<ul>
|
||||||
4. Inject into request extensions
|
<li><code>401 Unauthorized</code>: Missing/invalid token, expired, revoked</li>
|
||||||
5. Continue to next middleware or return 401
|
<li><code>403 Forbidden</code>: Insufficient permissions</li>
|
||||||
|
</ul>
|
||||||
**Error Responses**:
|
<h3 id="3-mfa-verification-middleware-middlewaremfars"><a class="header" href="#3-mfa-verification-middleware-middlewaremfars">3. MFA Verification Middleware (<code>middleware/mfa.rs</code>)</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: Enforce MFA for sensitive operations.</p>
|
||||||
- `401 Unauthorized`: Missing/invalid token, expired, revoked
|
<p><strong>Key Features</strong>:</p>
|
||||||
- `403 Forbidden`: Insufficient permissions
|
<ul>
|
||||||
|
<li>Path-based MFA requirements</li>
|
||||||
### 3. MFA Verification Middleware (`middleware/mfa.rs`)
|
<li>Method-based enforcement (all DELETEs)</li>
|
||||||
|
<li>Production environment protection</li>
|
||||||
**Purpose**: Enforce MFA for sensitive operations.
|
<li>Clear error messages</li>
|
||||||
|
</ul>
|
||||||
**Key Features**:
|
<p><strong>Lines of Code</strong>: 290</p>
|
||||||
|
<p><strong>MFA Required For</strong>:</p>
|
||||||
- Path-based MFA requirements
|
<ul>
|
||||||
- Method-based enforcement (all DELETEs)
|
<li>Production deployments (<code>/production/</code>, <code>/prod/</code>)</li>
|
||||||
- Production environment protection
|
<li>All DELETE operations</li>
|
||||||
- Clear error messages
|
<li>Server operations (POST, PUT, DELETE)</li>
|
||||||
|
<li>Cluster operations (POST, PUT, DELETE)</li>
|
||||||
**Lines of Code**: 290
|
<li>Batch submissions</li>
|
||||||
|
<li>Rollback operations</li>
|
||||||
**MFA Required For**:
|
<li>Configuration changes (POST, PUT, DELETE)</li>
|
||||||
|
<li>Secret management</li>
|
||||||
- Production deployments (`/production/`, `/prod/`)
|
<li>User/role management</li>
|
||||||
- All DELETE operations
|
</ul>
|
||||||
- Server operations (POST, PUT, DELETE)
|
<p><strong>Example</strong>:</p>
|
||||||
- Cluster operations (POST, PUT, DELETE)
|
<pre><code class="language-rust">fn requires_mfa(method: &str, path: &str) -> bool {
|
||||||
- Batch submissions
|
|
||||||
- Rollback operations
|
|
||||||
- Configuration changes (POST, PUT, DELETE)
|
|
||||||
- Secret management
|
|
||||||
- User/role management
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
fn requires_mfa(method: &str, path: &str) -> bool {
|
|
||||||
if path.contains("/production/") { return true; }
|
if path.contains("/production/") { return true; }
|
||||||
if method == "DELETE" { return true; }
|
if method == "DELETE" { return true; }
|
||||||
if path.contains("/deploy") { return true; }
|
if path.contains("/deploy") { return true; }
|
||||||
// ...
|
// ...
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<h3 id="4-enhanced-authorization-middleware-middlewareauthzrs"><a class="header" href="#4-enhanced-authorization-middleware-middlewareauthzrs">4. Enhanced Authorization Middleware (<code>middleware/authz.rs</code>)</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: Cedar policy evaluation with audit logging.</p>
|
||||||
### 4. Enhanced Authorization Middleware (`middleware/authz.rs`)
|
<p><strong>Key Features</strong>:</p>
|
||||||
|
<ul>
|
||||||
**Purpose**: Cedar policy evaluation with audit logging.
|
<li>Builds Cedar authorization request from HTTP request</li>
|
||||||
|
<li>Maps HTTP methods to Cedar actions (GET→Read, POST→Create, etc.)</li>
|
||||||
**Key Features**:
|
<li>Extracts resource types from paths</li>
|
||||||
|
<li>Evaluates Cedar policies with context (MFA, IP, time, workspace)</li>
|
||||||
- Builds Cedar authorization request from HTTP request
|
<li>Logs all authorization decisions to audit log</li>
|
||||||
- Maps HTTP methods to Cedar actions (GET→Read, POST→Create, etc.)
|
<li>Non-blocking audit logging (tokio::spawn)</li>
|
||||||
- Extracts resource types from paths
|
</ul>
|
||||||
- Evaluates Cedar policies with context (MFA, IP, time, workspace)
|
<p><strong>Lines of Code</strong>: 380</p>
|
||||||
- Logs all authorization decisions to audit log
|
<p><strong>Resource Mapping</strong>:</p>
|
||||||
- Non-blocking audit logging (tokio::spawn)
|
<pre><code class="language-rust">/api/v1/servers/srv-123 → Resource::Server("srv-123")
|
||||||
|
|
||||||
**Lines of Code**: 380
|
|
||||||
|
|
||||||
**Resource Mapping**:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/api/v1/servers/srv-123 → Resource::Server("srv-123")
|
|
||||||
/api/v1/taskserv/kubernetes → Resource::TaskService("kubernetes")
|
/api/v1/taskserv/kubernetes → Resource::TaskService("kubernetes")
|
||||||
/api/v1/cluster/prod → Resource::Cluster("prod")
|
/api/v1/cluster/prod → Resource::Cluster("prod")
|
||||||
/api/v1/config/settings → Resource::Config("settings")
|
/api/v1/config/settings → Resource::Config("settings")</code></pre>
|
||||||
```plaintext
|
<p><strong>Action Mapping</strong>:</p>
|
||||||
|
<pre><code class="language-rust">GET → Action::Read
|
||||||
**Action Mapping**:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
GET → Action::Read
|
|
||||||
POST → Action::Create
|
POST → Action::Create
|
||||||
PUT → Action::Update
|
PUT → Action::Update
|
||||||
DELETE → Action::Delete
|
DELETE → Action::Delete</code></pre>
|
||||||
```plaintext
|
<h3 id="5-rate-limiting-middleware-middlewarerate_limitrs"><a class="header" href="#5-rate-limiting-middleware-middlewarerate_limitrs">5. Rate Limiting Middleware (<code>middleware/rate_limit.rs</code>)</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: Prevent API abuse with per-IP rate limiting.</p>
|
||||||
### 5. Rate Limiting Middleware (`middleware/rate_limit.rs`)
|
<p><strong>Key Features</strong>:</p>
|
||||||
|
<ul>
|
||||||
**Purpose**: Prevent API abuse with per-IP rate limiting.
|
<li>Sliding window rate limiting</li>
|
||||||
|
<li>Per-IP request tracking</li>
|
||||||
**Key Features**:
|
<li>Configurable limits and windows</li>
|
||||||
|
<li>Exempt IP support</li>
|
||||||
- Sliding window rate limiting
|
<li>Automatic cleanup of old entries</li>
|
||||||
- Per-IP request tracking
|
<li>Statistics tracking</li>
|
||||||
- Configurable limits and windows
|
</ul>
|
||||||
- Exempt IP support
|
<p><strong>Lines of Code</strong>: 420</p>
|
||||||
- Automatic cleanup of old entries
|
<p><strong>Configuration</strong>:</p>
|
||||||
- Statistics tracking
|
<pre><code class="language-rust">pub struct RateLimitConfig {
|
||||||
|
pub max_requests: u32, // for example, 100
|
||||||
**Lines of Code**: 420
|
pub window_duration: Duration, // for example, 60 seconds
|
||||||
|
pub exempt_ips: Vec<IpAddr>, // for example, internal services
|
||||||
**Configuration**:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct RateLimitConfig {
|
|
||||||
pub max_requests: u32, // e.g., 100
|
|
||||||
pub window_duration: Duration, // e.g., 60 seconds
|
|
||||||
pub exempt_ips: Vec<IpAddr>, // e.g., internal services
|
|
||||||
pub enabled: bool,
|
pub enabled: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
// Default: 100 requests per minute
|
// Default: 100 requests per minute</code></pre>
|
||||||
```plaintext
|
<p><strong>Statistics</strong>:</p>
|
||||||
|
<pre><code class="language-rust">pub struct RateLimitStats {
|
||||||
**Statistics**:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct RateLimitStats {
|
|
||||||
pub total_ips: usize, // Number of tracked IPs
|
pub total_ips: usize, // Number of tracked IPs
|
||||||
pub total_requests: u32, // Total requests made
|
pub total_requests: u32, // Total requests made
|
||||||
pub limited_ips: usize, // IPs that hit the limit
|
pub limited_ips: usize, // IPs that hit the limit
|
||||||
pub config: RateLimitConfig,
|
pub config: RateLimitConfig,
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<h3 id="6-security-integration-module-security_integrationrs"><a class="header" href="#6-security-integration-module-security_integrationrs">6. Security Integration Module (<code>security_integration.rs</code>)</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: Helper module to integrate all security components.</p>
|
||||||
### 6. Security Integration Module (`security_integration.rs`)
|
<p><strong>Key Features</strong>:</p>
|
||||||
|
<ul>
|
||||||
**Purpose**: Helper module to integrate all security components.
|
<li><code>SecurityComponents</code> struct grouping all middleware</li>
|
||||||
|
<li><code>SecurityConfig</code> for configuration</li>
|
||||||
**Key Features**:
|
<li><code>initialize()</code> method to set up all components</li>
|
||||||
|
<li><code>disabled()</code> method for development mode</li>
|
||||||
- `SecurityComponents` struct grouping all middleware
|
<li><code>apply_security_middleware()</code> helper for router setup</li>
|
||||||
- `SecurityConfig` for configuration
|
</ul>
|
||||||
- `initialize()` method to set up all components
|
<p><strong>Lines of Code</strong>: 265</p>
|
||||||
- `disabled()` method for development mode
|
<p><strong>Usage Example</strong>:</p>
|
||||||
- `apply_security_middleware()` helper for router setup
|
<pre><code class="language-rust">use provisioning_orchestrator::security_integration::{
|
||||||
|
|
||||||
**Lines of Code**: 265
|
|
||||||
|
|
||||||
**Usage Example**:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use provisioning_orchestrator::security_integration::{
|
|
||||||
SecurityComponents, SecurityConfig
|
SecurityComponents, SecurityConfig
|
||||||
};
|
};
|
||||||
|
|
||||||
@ -450,15 +404,10 @@ let app = Router::new()
|
|||||||
.route("/api/v1/servers", post(create_server))
|
.route("/api/v1/servers", post(create_server))
|
||||||
.route("/api/v1/servers/:id", delete(delete_server));
|
.route("/api/v1/servers/:id", delete(delete_server));
|
||||||
|
|
||||||
let secured_app = apply_security_middleware(app, &security);
|
let secured_app = apply_security_middleware(app, &security);</code></pre>
|
||||||
```plaintext
|
<h2 id="integration-with-appstate"><a class="header" href="#integration-with-appstate">Integration with AppState</a></h2>
|
||||||
|
<h3 id="updated-appstate-structure"><a class="header" href="#updated-appstate-structure">Updated AppState Structure</a></h3>
|
||||||
## Integration with AppState
|
<pre><code class="language-rust">pub struct AppState {
|
||||||
|
|
||||||
### Updated AppState Structure
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct AppState {
|
|
||||||
// Existing fields
|
// Existing fields
|
||||||
pub task_storage: Arc<dyn TaskStorage>,
|
pub task_storage: Arc<dyn TaskStorage>,
|
||||||
pub batch_coordinator: BatchCoordinator,
|
pub batch_coordinator: BatchCoordinator,
|
||||||
@ -477,13 +426,9 @@ pub struct AppState {
|
|||||||
|
|
||||||
// NEW: Security components
|
// NEW: Security components
|
||||||
pub security: SecurityComponents,
|
pub security: SecurityComponents,
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<h3 id="initialization-in-mainrs"><a class="header" href="#initialization-in-mainrs">Initialization in main.rs</a></h3>
|
||||||
|
<pre><code class="language-rust">#[tokio::main]
|
||||||
### Initialization in main.rs
|
|
||||||
|
|
||||||
```rust
|
|
||||||
#[tokio::main]
|
|
||||||
async fn main() -> Result<()> {
|
async fn main() -> Result<()> {
|
||||||
let args = Args::parse();
|
let args = Args::parse();
|
||||||
|
|
||||||
@ -538,33 +483,26 @@ async fn main() -> Result<()> {
|
|||||||
axum::serve(listener, app).await?;
|
axum::serve(listener, app).await?;
|
||||||
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<h2 id="protected-endpoints"><a class="header" href="#protected-endpoints">Protected Endpoints</a></h2>
|
||||||
|
<h3 id="endpoint-categories"><a class="header" href="#endpoint-categories">Endpoint Categories</a></h3>
|
||||||
## Protected Endpoints
|
<div class="table-wrapper"><table><thead><tr><th>Category</th><th>Example Endpoints</th><th>Auth Required</th><th>MFA Required</th><th>Cedar Policy</th></tr></thead><tbody>
|
||||||
|
<tr><td><strong>Health</strong></td><td><code>/health</code></td><td>❌</td><td>❌</td><td>❌</td></tr>
|
||||||
### Endpoint Categories
|
<tr><td><strong>Read-Only</strong></td><td><code>GET /api/v1/servers</code></td><td>✅</td><td>❌</td><td>✅</td></tr>
|
||||||
|
<tr><td><strong>Server Mgmt</strong></td><td><code>POST /api/v1/servers</code></td><td>✅</td><td>❌</td><td>✅</td></tr>
|
||||||
| Category | Example Endpoints | Auth Required | MFA Required | Cedar Policy |
|
<tr><td><strong>Server Delete</strong></td><td><code>DELETE /api/v1/servers/:id</code></td><td>✅</td><td>✅</td><td>✅</td></tr>
|
||||||
|----------|-------------------|---------------|--------------|--------------|
|
<tr><td><strong>Taskserv Mgmt</strong></td><td><code>POST /api/v1/taskserv</code></td><td>✅</td><td>❌</td><td>✅</td></tr>
|
||||||
| **Health** | `/health` | ❌ | ❌ | ❌ |
|
<tr><td><strong>Cluster Mgmt</strong></td><td><code>POST /api/v1/cluster</code></td><td>✅</td><td>✅</td><td>✅</td></tr>
|
||||||
| **Read-Only** | `GET /api/v1/servers` | ✅ | ❌ | ✅ |
|
<tr><td><strong>Production</strong></td><td><code>POST /api/v1/production/*</code></td><td>✅</td><td>✅</td><td>✅</td></tr>
|
||||||
| **Server Mgmt** | `POST /api/v1/servers` | ✅ | ❌ | ✅ |
|
<tr><td><strong>Batch Ops</strong></td><td><code>POST /api/v1/batch/submit</code></td><td>✅</td><td>✅</td><td>✅</td></tr>
|
||||||
| **Server Delete** | `DELETE /api/v1/servers/:id` | ✅ | ✅ | ✅ |
|
<tr><td><strong>Rollback</strong></td><td><code>POST /api/v1/rollback</code></td><td>✅</td><td>✅</td><td>✅</td></tr>
|
||||||
| **Taskserv Mgmt** | `POST /api/v1/taskserv` | ✅ | ❌ | ✅ |
|
<tr><td><strong>Config Write</strong></td><td><code>POST /api/v1/config</code></td><td>✅</td><td>✅</td><td>✅</td></tr>
|
||||||
| **Cluster Mgmt** | `POST /api/v1/cluster` | ✅ | ✅ | ✅ |
|
<tr><td><strong>Secrets</strong></td><td><code>GET /api/v1/secret/*</code></td><td>✅</td><td>✅</td><td>✅</td></tr>
|
||||||
| **Production** | `POST /api/v1/production/*` | ✅ | ✅ | ✅ |
|
</tbody></table>
|
||||||
| **Batch Ops** | `POST /api/v1/batch/submit` | ✅ | ✅ | ✅ |
|
</div>
|
||||||
| **Rollback** | `POST /api/v1/rollback` | ✅ | ✅ | ✅ |
|
<h2 id="complete-authentication-flow"><a class="header" href="#complete-authentication-flow">Complete Authentication Flow</a></h2>
|
||||||
| **Config Write** | `POST /api/v1/config` | ✅ | ✅ | ✅ |
|
<h3 id="step-by-step-flow"><a class="header" href="#step-by-step-flow">Step-by-Step Flow</a></h3>
|
||||||
| **Secrets** | `GET /api/v1/secret/*` | ✅ | ✅ | ✅ |
|
<pre><code class="language-plaintext">1. CLIENT REQUEST
|
||||||
|
|
||||||
## Complete Authentication Flow
|
|
||||||
|
|
||||||
### Step-by-Step Flow
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
1. CLIENT REQUEST
|
|
||||||
├─ Headers:
|
├─ Headers:
|
||||||
│ ├─ Authorization: Bearer <jwt_token>
|
│ ├─ Authorization: Bearer <jwt_token>
|
||||||
│ ├─ X-Forwarded-For: 192.168.1.100
|
│ ├─ X-Forwarded-For: 192.168.1.100
|
||||||
@ -644,14 +582,10 @@ async fn main() -> Result<()> {
|
|||||||
|
|
||||||
9. CLIENT RESPONSE
|
9. CLIENT RESPONSE
|
||||||
└─ 200 OK: Server deleted successfully
|
└─ 200 OK: Server deleted successfully
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="configuration"><a class="header" href="#configuration">Configuration</a></h2>
|
||||||
## Configuration
|
<h3 id="environment-variables"><a class="header" href="#environment-variables">Environment Variables</a></h3>
|
||||||
|
<pre><code class="language-bash"># JWT Configuration
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# JWT Configuration
|
|
||||||
JWT_ISSUER=control-center
|
JWT_ISSUER=control-center
|
||||||
JWT_AUDIENCE=orchestrator
|
JWT_AUDIENCE=orchestrator
|
||||||
PUBLIC_KEY_PATH=/path/to/keys/public.pem
|
PUBLIC_KEY_PATH=/path/to/keys/public.pem
|
||||||
@ -672,125 +606,107 @@ RATE_LIMIT_EXEMPT_IPS=10.0.0.1,10.0.0.2
|
|||||||
# Audit Logging
|
# Audit Logging
|
||||||
AUDIT_ENABLED=true
|
AUDIT_ENABLED=true
|
||||||
AUDIT_RETENTION_DAYS=365
|
AUDIT_RETENTION_DAYS=365
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="development-mode"><a class="header" href="#development-mode">Development Mode</a></h3>
|
||||||
### Development Mode
|
<p>For development/testing, all security can be disabled:</p>
|
||||||
|
<pre><code class="language-rust">// In main.rs
|
||||||
For development/testing, all security can be disabled:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// In main.rs
|
|
||||||
let security = if env::var("DEVELOPMENT_MODE").unwrap_or("false".to_string()) == "true" {
|
let security = if env::var("DEVELOPMENT_MODE").unwrap_or("false".to_string()) == "true" {
|
||||||
SecurityComponents::disabled(audit_logger.clone())
|
SecurityComponents::disabled(audit_logger.clone())
|
||||||
} else {
|
} else {
|
||||||
SecurityComponents::initialize(security_config, audit_logger.clone()).await?
|
SecurityComponents::initialize(security_config, audit_logger.clone()).await?
|
||||||
};
|
};</code></pre>
|
||||||
```plaintext
|
<h2 id="testing"><a class="header" href="#testing">Testing</a></h2>
|
||||||
|
<h3 id="integration-tests"><a class="header" href="#integration-tests">Integration Tests</a></h3>
|
||||||
## Testing
|
<p>Location: <code>provisioning/platform/orchestrator/tests/security_integration_tests.rs</code></p>
|
||||||
|
<p><strong>Test Coverage</strong>:</p>
|
||||||
### Integration Tests
|
<ul>
|
||||||
|
<li>✅ Rate limiting enforcement</li>
|
||||||
Location: `provisioning/platform/orchestrator/tests/security_integration_tests.rs`
|
<li>✅ Rate limit statistics</li>
|
||||||
|
<li>✅ Exempt IP handling</li>
|
||||||
**Test Coverage**:
|
<li>✅ Authentication missing token</li>
|
||||||
|
<li>✅ MFA verification for sensitive operations</li>
|
||||||
- ✅ Rate limiting enforcement
|
<li>✅ Cedar policy evaluation</li>
|
||||||
- ✅ Rate limit statistics
|
<li>✅ Complete security flow</li>
|
||||||
- ✅ Exempt IP handling
|
<li>✅ Security components initialization</li>
|
||||||
- ✅ Authentication missing token
|
<li>✅ Configuration defaults</li>
|
||||||
- ✅ MFA verification for sensitive operations
|
</ul>
|
||||||
- ✅ Cedar policy evaluation
|
<p><strong>Lines of Code</strong>: 340</p>
|
||||||
- ✅ Complete security flow
|
<p><strong>Run Tests</strong>:</p>
|
||||||
- ✅ Security components initialization
|
<pre><code class="language-bash">cd provisioning/platform/orchestrator
|
||||||
- ✅ Configuration defaults
|
|
||||||
|
|
||||||
**Lines of Code**: 340
|
|
||||||
|
|
||||||
**Run Tests**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd provisioning/platform/orchestrator
|
|
||||||
cargo test security_integration_tests
|
cargo test security_integration_tests
|
||||||
```plaintext
|
|
||||||
|
|
||||||
## File Summary
|
|
||||||
|
|
||||||
| File | Purpose | Lines | Tests |
|
|
||||||
|------|---------|-------|-------|
|
|
||||||
| `middleware/security_context.rs` | Security context builder | 275 | 8 |
|
|
||||||
| `middleware/auth.rs` | JWT authentication | 245 | 5 |
|
|
||||||
| `middleware/mfa.rs` | MFA verification | 290 | 15 |
|
|
||||||
| `middleware/authz.rs` | Cedar authorization | 380 | 4 |
|
|
||||||
| `middleware/rate_limit.rs` | Rate limiting | 420 | 8 |
|
|
||||||
| `middleware/mod.rs` | Module exports | 25 | 0 |
|
|
||||||
| `security_integration.rs` | Integration helpers | 265 | 2 |
|
|
||||||
| `tests/security_integration_tests.rs` | Integration tests | 340 | 11 |
|
|
||||||
| **Total** | | **2,240** | **53** |
|
|
||||||
|
|
||||||
## Benefits
|
|
||||||
|
|
||||||
### Security
|
|
||||||
|
|
||||||
- ✅ Complete authentication flow with JWT validation
|
|
||||||
- ✅ MFA enforcement for sensitive operations
|
|
||||||
- ✅ Fine-grained authorization with Cedar policies
|
|
||||||
- ✅ Rate limiting prevents API abuse
|
|
||||||
- ✅ Complete audit trail for compliance
|
|
||||||
|
|
||||||
### Architecture
|
|
||||||
|
|
||||||
- ✅ Modular middleware design
|
|
||||||
- ✅ Clear separation of concerns
|
|
||||||
- ✅ Reusable security components
|
|
||||||
- ✅ Easy to test and maintain
|
|
||||||
- ✅ Configuration-driven behavior
|
|
||||||
|
|
||||||
### Operations
|
|
||||||
|
|
||||||
- ✅ Can enable/disable features independently
|
|
||||||
- ✅ Development mode for testing
|
|
||||||
- ✅ Comprehensive error messages
|
|
||||||
- ✅ Real-time statistics and monitoring
|
|
||||||
- ✅ Non-blocking audit logging
|
|
||||||
|
|
||||||
## Future Enhancements
|
|
||||||
|
|
||||||
1. **Token Refresh**: Automatic token refresh before expiry
|
|
||||||
2. **IP Whitelisting**: Additional IP-based access control
|
|
||||||
3. **Geolocation**: Block requests from specific countries
|
|
||||||
4. **Advanced Rate Limiting**: Per-user, per-endpoint limits
|
|
||||||
5. **Session Management**: Track active sessions, force logout
|
|
||||||
6. **2FA Integration**: Direct integration with TOTP/SMS providers
|
|
||||||
7. **Policy Hot Reload**: Update Cedar policies without restart
|
|
||||||
8. **Metrics Dashboard**: Real-time security metrics visualization
|
|
||||||
|
|
||||||
## Related Documentation
|
|
||||||
|
|
||||||
- Cedar Policy Language
|
|
||||||
- JWT Token Management
|
|
||||||
- MFA Setup Guide
|
|
||||||
- Audit Log Format
|
|
||||||
- Rate Limiting Best Practices
|
|
||||||
|
|
||||||
## Version History
|
|
||||||
|
|
||||||
| Version | Date | Changes |
|
|
||||||
|---------|------|---------|
|
|
||||||
| 1.0.0 | 2025-10-08 | Initial implementation |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Maintained By**: Security Team
|
|
||||||
**Review Cycle**: Quarterly
|
|
||||||
**Last Reviewed**: 2025-10-08
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
|
<h2 id="file-summary"><a class="header" href="#file-summary">File Summary</a></h2>
|
||||||
|
<div class="table-wrapper"><table><thead><tr><th>File</th><th>Purpose</th><th>Lines</th><th>Tests</th></tr></thead><tbody>
|
||||||
|
<tr><td><code>middleware/security_context.rs</code></td><td>Security context builder</td><td>275</td><td>8</td></tr>
|
||||||
|
<tr><td><code>middleware/auth.rs</code></td><td>JWT authentication</td><td>245</td><td>5</td></tr>
|
||||||
|
<tr><td><code>middleware/mfa.rs</code></td><td>MFA verification</td><td>290</td><td>15</td></tr>
|
||||||
|
<tr><td><code>middleware/authz.rs</code></td><td>Cedar authorization</td><td>380</td><td>4</td></tr>
|
||||||
|
<tr><td><code>middleware/rate_limit.rs</code></td><td>Rate limiting</td><td>420</td><td>8</td></tr>
|
||||||
|
<tr><td><code>middleware/mod.rs</code></td><td>Module exports</td><td>25</td><td>0</td></tr>
|
||||||
|
<tr><td><code>security_integration.rs</code></td><td>Integration helpers</td><td>265</td><td>2</td></tr>
|
||||||
|
<tr><td><code>tests/security_integration_tests.rs</code></td><td>Integration tests</td><td>340</td><td>11</td></tr>
|
||||||
|
<tr><td><strong>Total</strong></td><td></td><td><strong>2,240</strong></td><td><strong>53</strong></td></tr>
|
||||||
|
</tbody></table>
|
||||||
|
</div>
|
||||||
|
<h2 id="benefits"><a class="header" href="#benefits">Benefits</a></h2>
|
||||||
|
<h3 id="security"><a class="header" href="#security">Security</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>✅ Complete authentication flow with JWT validation</li>
|
||||||
|
<li>✅ MFA enforcement for sensitive operations</li>
|
||||||
|
<li>✅ Fine-grained authorization with Cedar policies</li>
|
||||||
|
<li>✅ Rate limiting prevents API abuse</li>
|
||||||
|
<li>✅ Complete audit trail for compliance</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="architecture-1"><a class="header" href="#architecture-1">Architecture</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>✅ Modular middleware design</li>
|
||||||
|
<li>✅ Clear separation of concerns</li>
|
||||||
|
<li>✅ Reusable security components</li>
|
||||||
|
<li>✅ Easy to test and maintain</li>
|
||||||
|
<li>✅ Configuration-driven behavior</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="operations"><a class="header" href="#operations">Operations</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>✅ Can enable/disable features independently</li>
|
||||||
|
<li>✅ Development mode for testing</li>
|
||||||
|
<li>✅ Comprehensive error messages</li>
|
||||||
|
<li>✅ Real-time statistics and monitoring</li>
|
||||||
|
<li>✅ Non-blocking audit logging</li>
|
||||||
|
</ul>
|
||||||
|
<h2 id="future-enhancements"><a class="header" href="#future-enhancements">Future Enhancements</a></h2>
|
||||||
|
<ol>
|
||||||
|
<li><strong>Token Refresh</strong>: Automatic token refresh before expiry</li>
|
||||||
|
<li><strong>IP Whitelisting</strong>: Additional IP-based access control</li>
|
||||||
|
<li><strong>Geolocation</strong>: Block requests from specific countries</li>
|
||||||
|
<li><strong>Advanced Rate Limiting</strong>: Per-user, per-endpoint limits</li>
|
||||||
|
<li><strong>Session Management</strong>: Track active sessions, force logout</li>
|
||||||
|
<li><strong>2FA Integration</strong>: Direct integration with TOTP/SMS providers</li>
|
||||||
|
<li><strong>Policy Hot Reload</strong>: Update Cedar policies without restart</li>
|
||||||
|
<li><strong>Metrics Dashboard</strong>: Real-time security metrics visualization</li>
|
||||||
|
</ol>
|
||||||
|
<h2 id="related-documentation"><a class="header" href="#related-documentation">Related Documentation</a></h2>
|
||||||
|
<ul>
|
||||||
|
<li>Cedar Policy Language</li>
|
||||||
|
<li>JWT Token Management</li>
|
||||||
|
<li>MFA Setup Guide</li>
|
||||||
|
<li>Audit Log Format</li>
|
||||||
|
<li>Rate Limiting Best Practices</li>
|
||||||
|
</ul>
|
||||||
|
<h2 id="version-history"><a class="header" href="#version-history">Version History</a></h2>
|
||||||
|
<div class="table-wrapper"><table><thead><tr><th>Version</th><th>Date</th><th>Changes</th></tr></thead><tbody>
|
||||||
|
<tr><td>1.0.0</td><td>2025-10-08</td><td>Initial implementation</td></tr>
|
||||||
|
</tbody></table>
|
||||||
|
</div>
|
||||||
|
<hr />
|
||||||
|
<p><strong>Maintained By</strong>: Security Team
|
||||||
|
<strong>Review Cycle</strong>: Quarterly
|
||||||
|
<strong>Last Reviewed</strong>: 2025-10-08</p>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
<nav class="nav-wrapper" aria-label="Page navigation">
|
<nav class="nav-wrapper" aria-label="Page navigation">
|
||||||
<!-- Mobile navigation buttons -->
|
<!-- Mobile navigation buttons -->
|
||||||
<a rel="prev" href="../architecture/orchestrator_info.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
<a rel="prev" href="../architecture/orchestrator-info.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
<i class="fa fa-angle-left"></i>
|
<i class="fa fa-angle-left"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
@ -804,7 +720,7 @@ cargo test security_integration_tests
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<nav class="nav-wide-wrapper" aria-label="Page navigation">
|
<nav class="nav-wide-wrapper" aria-label="Page navigation">
|
||||||
<a rel="prev" href="../architecture/orchestrator_info.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
<a rel="prev" href="../architecture/orchestrator-info.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
<i class="fa fa-angle-left"></i>
|
<i class="fa fa-angle-left"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
|
|||||||
@ -186,20 +186,17 @@
|
|||||||
→ "Type not supported" errors
|
→ "Type not supported" errors
|
||||||
→ Cannot handle complex nested workflows
|
→ Cannot handle complex nested workflows
|
||||||
→ Performance bottlenecks with recursive calls
|
→ Performance bottlenecks with recursive calls
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Solution:</strong> Rust orchestrator provides:</p>
|
||||||
**Solution:** Rust orchestrator provides:
|
<ol>
|
||||||
|
<li><strong>Task queue management</strong> (file-based, reliable)</li>
|
||||||
1. **Task queue management** (file-based, reliable)
|
<li><strong>Priority scheduling</strong> (intelligent task ordering)</li>
|
||||||
2. **Priority scheduling** (intelligent task ordering)
|
<li><strong>Deep call stack elimination</strong> (Rust handles recursion)</li>
|
||||||
3. **Deep call stack elimination** (Rust handles recursion)
|
<li><strong>Performance optimization</strong> (async/await, parallel execution)</li>
|
||||||
4. **Performance optimization** (async/await, parallel execution)
|
<li><strong>State management</strong> (workflow checkpointing)</li>
|
||||||
5. **State management** (workflow checkpointing)
|
</ol>
|
||||||
|
<h3 id="how-it-works-today-monorepo"><a class="header" href="#how-it-works-today-monorepo">How It Works Today (Monorepo)</a></h3>
|
||||||
### How It Works Today (Monorepo)
|
<pre><code class="language-plaintext">┌─────────────────────────────────────────────────────────────┐
|
||||||
|
|
||||||
```plaintext
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ User │
|
│ User │
|
||||||
└───────────────────────────┬─────────────────────────────────┘
|
└───────────────────────────┬─────────────────────────────────┘
|
||||||
│ calls
|
│ calls
|
||||||
@ -237,39 +234,29 @@
|
|||||||
│ • taskservs.nu │
|
│ • taskservs.nu │
|
||||||
│ • clusters.nu │
|
│ • clusters.nu │
|
||||||
└────────────────┘
|
└────────────────┘
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="three-execution-modes"><a class="header" href="#three-execution-modes">Three Execution Modes</a></h3>
|
||||||
### Three Execution Modes
|
<h4 id="mode-1-direct-mode-simple-operations"><a class="header" href="#mode-1-direct-mode-simple-operations">Mode 1: Direct Mode (Simple Operations)</a></h4>
|
||||||
|
<pre><code class="language-bash"># No orchestrator needed
|
||||||
#### Mode 1: Direct Mode (Simple Operations)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# No orchestrator needed
|
|
||||||
provisioning server list
|
provisioning server list
|
||||||
provisioning env
|
provisioning env
|
||||||
provisioning help
|
provisioning help
|
||||||
|
|
||||||
# Direct Nushell execution
|
# Direct Nushell execution
|
||||||
provisioning (CLI) → Nushell scripts → Result
|
provisioning (CLI) → Nushell scripts → Result
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="mode-2-orchestrated-mode-complex-operations"><a class="header" href="#mode-2-orchestrated-mode-complex-operations">Mode 2: Orchestrated Mode (Complex Operations)</a></h4>
|
||||||
#### Mode 2: Orchestrated Mode (Complex Operations)
|
<pre><code class="language-bash"># Uses orchestrator for coordination
|
||||||
|
|
||||||
```bash
|
|
||||||
# Uses orchestrator for coordination
|
|
||||||
provisioning server create --orchestrated
|
provisioning server create --orchestrated
|
||||||
|
|
||||||
# Flow:
|
# Flow:
|
||||||
provisioning CLI → Orchestrator API → Task Queue → Nushell executor
|
provisioning CLI → Orchestrator API → Task Queue → Nushell executor
|
||||||
↓
|
↓
|
||||||
Result back to user
|
Result back to user
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="mode-3-workflow-mode-batch-operations"><a class="header" href="#mode-3-workflow-mode-batch-operations">Mode 3: Workflow Mode (Batch Operations)</a></h4>
|
||||||
#### Mode 3: Workflow Mode (Batch Operations)
|
<pre><code class="language-bash"># Complex workflows with dependencies
|
||||||
|
provisioning workflow submit server-cluster.ncl
|
||||||
```bash
|
|
||||||
# Complex workflows with dependencies
|
|
||||||
provisioning workflow submit server-cluster.k
|
|
||||||
|
|
||||||
# Flow:
|
# Flow:
|
||||||
provisioning CLI → Orchestrator Workflow Engine → Dependency Graph
|
provisioning CLI → Orchestrator Workflow Engine → Dependency Graph
|
||||||
@ -279,20 +266,13 @@ provisioning CLI → Orchestrator Workflow Engine → Dependency Graph
|
|||||||
Nushell scripts for each task
|
Nushell scripts for each task
|
||||||
↓
|
↓
|
||||||
Checkpoint state
|
Checkpoint state
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<hr />
|
||||||
---
|
<h2 id="integration-patterns"><a class="header" href="#integration-patterns">Integration Patterns</a></h2>
|
||||||
|
<h3 id="pattern-1-cli-submits-tasks-to-orchestrator"><a class="header" href="#pattern-1-cli-submits-tasks-to-orchestrator">Pattern 1: CLI Submits Tasks to Orchestrator</a></h3>
|
||||||
## Integration Patterns
|
<p><strong>Current Implementation:</strong></p>
|
||||||
|
<p><strong>Nushell CLI (<code>core/nulib/workflows/server_create.nu</code>):</strong></p>
|
||||||
### Pattern 1: CLI Submits Tasks to Orchestrator
|
<pre><code class="language-nushell"># Submit server creation workflow to orchestrator
|
||||||
|
|
||||||
**Current Implementation:**
|
|
||||||
|
|
||||||
**Nushell CLI (`core/nulib/workflows/server_create.nu`):**
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Submit server creation workflow to orchestrator
|
|
||||||
export def server_create_workflow [
|
export def server_create_workflow [
|
||||||
infra_name: string
|
infra_name: string
|
||||||
--orchestrated
|
--orchestrated
|
||||||
@ -312,12 +292,9 @@ export def server_create_workflow [
|
|||||||
do-server-create $infra_name
|
do-server-create $infra_name
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Rust Orchestrator (<code>platform/orchestrator/src/api/workflows.rs</code>):</strong></p>
|
||||||
**Rust Orchestrator (`platform/orchestrator/src/api/workflows.rs`):**
|
<pre><code class="language-rust">// Receive workflow submission from Nushell CLI
|
||||||
|
|
||||||
```rust
|
|
||||||
// Receive workflow submission from Nushell CLI
|
|
||||||
#[axum::debug_handler]
|
#[axum::debug_handler]
|
||||||
async fn create_server_workflow(
|
async fn create_server_workflow(
|
||||||
State(state): State<Arc<AppState>>,
|
State(state): State<Arc<AppState>>,
|
||||||
@ -341,13 +318,9 @@ async fn create_server_workflow(
|
|||||||
workflow_id: task.id,
|
workflow_id: task.id,
|
||||||
status: "queued",
|
status: "queued",
|
||||||
}))
|
}))
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<p><strong>Flow:</strong></p>
|
||||||
|
<pre><code class="language-plaintext">User → provisioning server create --orchestrated
|
||||||
**Flow:**
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
User → provisioning server create --orchestrated
|
|
||||||
↓
|
↓
|
||||||
Nushell CLI prepares task
|
Nushell CLI prepares task
|
||||||
↓
|
↓
|
||||||
@ -358,14 +331,10 @@ Orchestrator queues task
|
|||||||
Returns workflow ID immediately
|
Returns workflow ID immediately
|
||||||
↓
|
↓
|
||||||
User can monitor: provisioning workflow monitor <id>
|
User can monitor: provisioning workflow monitor <id>
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="pattern-2-orchestrator-executes-nushell-scripts"><a class="header" href="#pattern-2-orchestrator-executes-nushell-scripts">Pattern 2: Orchestrator Executes Nushell Scripts</a></h3>
|
||||||
### Pattern 2: Orchestrator Executes Nushell Scripts
|
<p><strong>Orchestrator Task Executor (<code>platform/orchestrator/src/executor.rs</code>):</strong></p>
|
||||||
|
<pre><code class="language-rust">// Orchestrator spawns Nushell to execute business logic
|
||||||
**Orchestrator Task Executor (`platform/orchestrator/src/executor.rs`):**
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Orchestrator spawns Nushell to execute business logic
|
|
||||||
pub async fn execute_task(task: Task) -> Result<TaskResult> {
|
pub async fn execute_task(task: Task) -> Result<TaskResult> {
|
||||||
match task.task_type {
|
match task.task_type {
|
||||||
TaskType::ServerCreate => {
|
TaskType::ServerCreate => {
|
||||||
@ -391,13 +360,9 @@ pub async fn execute_task(task: Task) -> Result<TaskResult> {
|
|||||||
}
|
}
|
||||||
// Other task types...
|
// Other task types...
|
||||||
}
|
}
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<p><strong>Flow:</strong></p>
|
||||||
|
<pre><code class="language-plaintext">Orchestrator task queue has pending task
|
||||||
**Flow:**
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
Orchestrator task queue has pending task
|
|
||||||
↓
|
↓
|
||||||
Executor picks up task
|
Executor picks up task
|
||||||
↓
|
↓
|
||||||
@ -410,14 +375,10 @@ Returns result to orchestrator
|
|||||||
Orchestrator updates task status
|
Orchestrator updates task status
|
||||||
↓
|
↓
|
||||||
User monitors via: provisioning workflow status <id>
|
User monitors via: provisioning workflow status <id>
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="pattern-3-bidirectional-communication"><a class="header" href="#pattern-3-bidirectional-communication">Pattern 3: Bidirectional Communication</a></h3>
|
||||||
### Pattern 3: Bidirectional Communication
|
<p><strong>Nushell Calls Orchestrator API:</strong></p>
|
||||||
|
<pre><code class="language-nushell"># Nushell script checks orchestrator status during execution
|
||||||
**Nushell Calls Orchestrator API:**
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Nushell script checks orchestrator status during execution
|
|
||||||
export def check-orchestrator-health [] {
|
export def check-orchestrator-health [] {
|
||||||
let response = (http get http://localhost:9090/health)
|
let response = (http get http://localhost:9090/health)
|
||||||
|
|
||||||
@ -435,12 +396,9 @@ export def report-progress [task_id: string, progress: int] {
|
|||||||
status: "in_progress"
|
status: "in_progress"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Orchestrator Monitors Nushell Execution:</strong></p>
|
||||||
**Orchestrator Monitors Nushell Execution:**
|
<pre><code class="language-rust">// Orchestrator tracks Nushell subprocess
|
||||||
|
|
||||||
```rust
|
|
||||||
// Orchestrator tracks Nushell subprocess
|
|
||||||
pub async fn execute_with_monitoring(task: Task) -> Result<TaskResult> {
|
pub async fn execute_with_monitoring(task: Task) -> Result<TaskResult> {
|
||||||
let mut child = Command::new("nu")
|
let mut child = Command::new("nu")
|
||||||
.arg("-c")
|
.arg("-c")
|
||||||
@ -470,33 +428,25 @@ pub async fn execute_with_monitoring(task: Task) -> Result<TaskResult>
|
|||||||
).await??;
|
).await??;
|
||||||
|
|
||||||
Ok(TaskResult::from_exit_status(result))
|
Ok(TaskResult::from_exit_status(result))
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<hr />
|
||||||
|
<h2 id="multi-repo-architecture-impact"><a class="header" href="#multi-repo-architecture-impact">Multi-Repo Architecture Impact</a></h2>
|
||||||
---
|
<h3 id="repository-split-doesnt-change-integration-model"><a class="header" href="#repository-split-doesnt-change-integration-model">Repository Split Doesn’t Change Integration Model</a></h3>
|
||||||
|
<p><strong>In Multi-Repo Setup:</strong></p>
|
||||||
## Multi-Repo Architecture Impact
|
<p><strong>Repository: <code>provisioning-core</code></strong></p>
|
||||||
|
<ul>
|
||||||
### Repository Split Doesn't Change Integration Model
|
<li>Contains: Nushell business logic</li>
|
||||||
|
<li>Installs to: <code>/usr/local/lib/provisioning/</code></li>
|
||||||
**In Multi-Repo Setup:**
|
<li>Package: <code>provisioning-core-3.2.1.tar.gz</code></li>
|
||||||
|
</ul>
|
||||||
**Repository: `provisioning-core`**
|
<p><strong>Repository: <code>provisioning-platform</code></strong></p>
|
||||||
|
<ul>
|
||||||
- Contains: Nushell business logic
|
<li>Contains: Rust orchestrator</li>
|
||||||
- Installs to: `/usr/local/lib/provisioning/`
|
<li>Installs to: <code>/usr/local/bin/provisioning-orchestrator</code></li>
|
||||||
- Package: `provisioning-core-3.2.1.tar.gz`
|
<li>Package: <code>provisioning-platform-2.5.3.tar.gz</code></li>
|
||||||
|
</ul>
|
||||||
**Repository: `provisioning-platform`**
|
<p><strong>Runtime Integration (Same as Monorepo):</strong></p>
|
||||||
|
<pre><code class="language-plaintext">User installs both packages:
|
||||||
- Contains: Rust orchestrator
|
|
||||||
- Installs to: `/usr/local/bin/provisioning-orchestrator`
|
|
||||||
- Package: `provisioning-platform-2.5.3.tar.gz`
|
|
||||||
|
|
||||||
**Runtime Integration (Same as Monorepo):**
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
User installs both packages:
|
|
||||||
provisioning-core-3.2.1 → /usr/local/lib/provisioning/
|
provisioning-core-3.2.1 → /usr/local/lib/provisioning/
|
||||||
provisioning-platform-2.5.3 → /usr/local/bin/provisioning-orchestrator
|
provisioning-platform-2.5.3 → /usr/local/bin/provisioning-orchestrator
|
||||||
|
|
||||||
@ -504,14 +454,10 @@ Orchestrator expects core at: /usr/local/lib/provisioning/
|
|||||||
Core expects orchestrator at: http://localhost:9090/
|
Core expects orchestrator at: http://localhost:9090/
|
||||||
|
|
||||||
No code dependencies, just runtime coordination!
|
No code dependencies, just runtime coordination!
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="configuration-based-integration"><a class="header" href="#configuration-based-integration">Configuration-Based Integration</a></h3>
|
||||||
### Configuration-Based Integration
|
<p><strong>Core Package (<code>provisioning-core</code>) config:</strong></p>
|
||||||
|
<pre><code class="language-toml"># /usr/local/share/provisioning/config/config.defaults.toml
|
||||||
**Core Package (`provisioning-core`) config:**
|
|
||||||
|
|
||||||
```toml
|
|
||||||
# /usr/local/share/provisioning/config/config.defaults.toml
|
|
||||||
|
|
||||||
[orchestrator]
|
[orchestrator]
|
||||||
enabled = true
|
enabled = true
|
||||||
@ -522,12 +468,9 @@ auto_start = true # Start orchestrator if not running
|
|||||||
[execution]
|
[execution]
|
||||||
default_mode = "orchestrated" # Use orchestrator by default
|
default_mode = "orchestrated" # Use orchestrator by default
|
||||||
fallback_to_direct = true # Fall back if orchestrator down
|
fallback_to_direct = true # Fall back if orchestrator down
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Platform Package (<code>provisioning-platform</code>) config:</strong></p>
|
||||||
**Platform Package (`provisioning-platform`) config:**
|
<pre><code class="language-toml"># /usr/local/share/provisioning/platform/config.toml
|
||||||
|
|
||||||
```toml
|
|
||||||
# /usr/local/share/provisioning/platform/config.toml
|
|
||||||
|
|
||||||
[orchestrator]
|
[orchestrator]
|
||||||
host = "127.0.0.1"
|
host = "127.0.0.1"
|
||||||
@ -539,14 +482,10 @@ nushell_binary = "nu" # Expects nu in PATH
|
|||||||
provisioning_lib = "/usr/local/lib/provisioning"
|
provisioning_lib = "/usr/local/lib/provisioning"
|
||||||
max_concurrent_tasks = 10
|
max_concurrent_tasks = 10
|
||||||
task_timeout_seconds = 3600
|
task_timeout_seconds = 3600
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="version-compatibility"><a class="header" href="#version-compatibility">Version Compatibility</a></h3>
|
||||||
### Version Compatibility
|
<p><strong>Compatibility Matrix (<code>provisioning-distribution/versions.toml</code>):</strong></p>
|
||||||
|
<pre><code class="language-toml">[compatibility.platform."2.5.3"]
|
||||||
**Compatibility Matrix (`provisioning-distribution/versions.toml`):**
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[compatibility.platform."2.5.3"]
|
|
||||||
core = "^3.2" # Platform 2.5.3 compatible with core 3.2.x
|
core = "^3.2" # Platform 2.5.3 compatible with core 3.2.x
|
||||||
min-core = "3.2.0"
|
min-core = "3.2.0"
|
||||||
api-version = "v1"
|
api-version = "v1"
|
||||||
@ -555,30 +494,20 @@ api-version = "v1"
|
|||||||
platform = "^2.5" # Core 3.2.1 compatible with platform 2.5.x
|
platform = "^2.5" # Core 3.2.1 compatible with platform 2.5.x
|
||||||
min-platform = "2.5.0"
|
min-platform = "2.5.0"
|
||||||
orchestrator-api = "v1"
|
orchestrator-api = "v1"
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<hr />
|
||||||
---
|
<h2 id="execution-flow-examples"><a class="header" href="#execution-flow-examples">Execution Flow Examples</a></h2>
|
||||||
|
<h3 id="example-1-simple-server-creation-direct-mode"><a class="header" href="#example-1-simple-server-creation-direct-mode">Example 1: Simple Server Creation (Direct Mode)</a></h3>
|
||||||
## Execution Flow Examples
|
<p><strong>No Orchestrator Needed:</strong></p>
|
||||||
|
<pre><code class="language-bash">provisioning server list
|
||||||
### Example 1: Simple Server Creation (Direct Mode)
|
|
||||||
|
|
||||||
**No Orchestrator Needed:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
provisioning server list
|
|
||||||
|
|
||||||
# Flow:
|
# Flow:
|
||||||
CLI → servers/list.nu → Query state → Return results
|
CLI → servers/list.nu → Query state → Return results
|
||||||
(Orchestrator not involved)
|
(Orchestrator not involved)
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="example-2-server-creation-with-orchestrator"><a class="header" href="#example-2-server-creation-with-orchestrator">Example 2: Server Creation with Orchestrator</a></h3>
|
||||||
### Example 2: Server Creation with Orchestrator
|
<p><strong>Using Orchestrator:</strong></p>
|
||||||
|
<pre><code class="language-bash">provisioning server create --orchestrated --infra wuji
|
||||||
**Using Orchestrator:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
provisioning server create --orchestrated --infra wuji
|
|
||||||
|
|
||||||
# Detailed Flow:
|
# Detailed Flow:
|
||||||
1. User executes command
|
1. User executes command
|
||||||
@ -612,7 +541,7 @@ provisioning server create --orchestrated --infra wuji
|
|||||||
nu -c "use /usr/local/lib/provisioning/servers/create.nu; create-server 'wuji'"
|
nu -c "use /usr/local/lib/provisioning/servers/create.nu; create-server 'wuji'"
|
||||||
↓
|
↓
|
||||||
13. Nushell executes business logic:
|
13. Nushell executes business logic:
|
||||||
- Reads KCL config
|
- Reads Nickel config
|
||||||
- Calls provider API (UpCloud/AWS)
|
- Calls provider API (UpCloud/AWS)
|
||||||
- Creates server
|
- Creates server
|
||||||
- Returns result
|
- Returns result
|
||||||
@ -623,14 +552,10 @@ provisioning server create --orchestrated --infra wuji
|
|||||||
↓
|
↓
|
||||||
16. User monitors: provisioning workflow status abc-123
|
16. User monitors: provisioning workflow status abc-123
|
||||||
→ Shows: "Server wuji created successfully"
|
→ Shows: "Server wuji created successfully"
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="example-3-batch-workflow-with-dependencies"><a class="header" href="#example-3-batch-workflow-with-dependencies">Example 3: Batch Workflow with Dependencies</a></h3>
|
||||||
### Example 3: Batch Workflow with Dependencies
|
<p><strong>Complex Workflow:</strong></p>
|
||||||
|
<pre><code class="language-bash">provisioning batch submit multi-cloud-deployment.ncl
|
||||||
**Complex Workflow:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
provisioning batch submit multi-cloud-deployment.k
|
|
||||||
|
|
||||||
# Workflow contains:
|
# Workflow contains:
|
||||||
- Create 5 servers (parallel)
|
- Create 5 servers (parallel)
|
||||||
@ -638,7 +563,7 @@ provisioning batch submit multi-cloud-deployment.k
|
|||||||
- Deploy applications (depends on Kubernetes)
|
- Deploy applications (depends on Kubernetes)
|
||||||
|
|
||||||
# Detailed Flow:
|
# Detailed Flow:
|
||||||
1. CLI submits KCL workflow to orchestrator
|
1. CLI submits Nickel workflow to orchestrator
|
||||||
↓
|
↓
|
||||||
2. Orchestrator parses workflow
|
2. Orchestrator parses workflow
|
||||||
↓
|
↓
|
||||||
@ -674,24 +599,26 @@ provisioning batch submit multi-cloud-deployment.k
|
|||||||
8. If failure occurs, can retry from checkpoint
|
8. If failure occurs, can retry from checkpoint
|
||||||
↓
|
↓
|
||||||
9. User monitors real-time: provisioning batch monitor <id>
|
9. User monitors real-time: provisioning batch monitor <id>
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<hr />
|
||||||
|
<h2 id="why-this-architecture"><a class="header" href="#why-this-architecture">Why This Architecture</a></h2>
|
||||||
|
<h3 id="orchestrator-benefits"><a class="header" href="#orchestrator-benefits">Orchestrator Benefits</a></h3>
|
||||||
|
<ol>
|
||||||
|
<li>
|
||||||
|
<p><strong>Eliminates Deep Call Stack Issues</strong></p>
|
||||||
|
<pre><code>
|
||||||
|
Without Orchestrator:
|
||||||
|
template.nu → calls → cluster.nu → calls → taskserv.nu → calls → provider.nu
|
||||||
|
(Deep nesting causes "Type not supported" errors)
|
||||||
|
|
||||||
---
|
With Orchestrator:
|
||||||
|
Orchestrator → spawns → Nushell subprocess (flat execution)
|
||||||
## Why This Architecture?
|
(No deep nesting, fresh Nushell context for each task)
|
||||||
|
|
||||||
### Orchestrator Benefits
|
|
||||||
|
|
||||||
1. **Eliminates Deep Call Stack Issues**
|
|
||||||
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
<p>Without Orchestrator:
|
</li>
|
||||||
template.nu → calls → cluster.nu → calls → taskserv.nu → calls → provider.nu
|
</ol>
|
||||||
(Deep nesting causes “Type not supported” errors)</p>
|
<pre><code>
|
||||||
<p>With Orchestrator:
|
|
||||||
Orchestrator → spawns → Nushell subprocess (flat execution)
|
|
||||||
(No deep nesting, fresh Nushell context for each task)</p>
|
|
||||||
<pre><code class="language-plaintext">
|
|
||||||
2. **Performance Optimization**
|
2. **Performance Optimization**
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
@ -721,7 +648,7 @@ Orchestrator → spawns → Nushell subprocess (flat execution)
|
|||||||
|
|
||||||
Each does what it's best at!
|
Each does what it's best at!
|
||||||
</code></pre>
|
</code></pre>
|
||||||
<h3 id="why-not-pure-rust"><a class="header" href="#why-not-pure-rust">Why NOT Pure Rust?</a></h3>
|
<h3 id="why-not-pure-rust"><a class="header" href="#why-not-pure-rust">Why NOT Pure Rust</a></h3>
|
||||||
<p><strong>Question:</strong> Why not implement everything in Rust?</p>
|
<p><strong>Question:</strong> Why not implement everything in Rust?</p>
|
||||||
<p><strong>Answer:</strong></p>
|
<p><strong>Answer:</strong></p>
|
||||||
<ol>
|
<ol>
|
||||||
@ -767,14 +694,10 @@ Orchestrator → spawns → Nushell subprocess (flat execution)
|
|||||||
→ /usr/local/share/provisioning/platform/ (platform configs)
|
→ /usr/local/share/provisioning/platform/ (platform configs)
|
||||||
|
|
||||||
3. Sets up systemd/launchd service for orchestrator
|
3. Sets up systemd/launchd service for orchestrator
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="runtime-coordination"><a class="header" href="#runtime-coordination">Runtime Coordination</a></h3>
|
||||||
### Runtime Coordination
|
<p><strong>Core package expects orchestrator:</strong></p>
|
||||||
|
<pre><code class="language-nushell"># core/nulib/lib_provisioning/orchestrator/client.nu
|
||||||
**Core package expects orchestrator:**
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# core/nulib/lib_provisioning/orchestrator/client.nu
|
|
||||||
|
|
||||||
# Check if orchestrator is running
|
# Check if orchestrator is running
|
||||||
export def orchestrator-available [] {
|
export def orchestrator-available [] {
|
||||||
@ -799,12 +722,9 @@ export def ensure-orchestrator [] {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Platform package executes core scripts:</strong></p>
|
||||||
**Platform package executes core scripts:**
|
<pre><code class="language-rust">// platform/orchestrator/src/executor/nushell.rs
|
||||||
|
|
||||||
```rust
|
|
||||||
// platform/orchestrator/src/executor/nushell.rs
|
|
||||||
|
|
||||||
pub struct NushellExecutor {
|
pub struct NushellExecutor {
|
||||||
provisioning_lib: PathBuf, // /usr/local/lib/provisioning
|
provisioning_lib: PathBuf, // /usr/local/lib/provisioning
|
||||||
@ -837,19 +757,12 @@ impl NushellExecutor {
|
|||||||
|
|
||||||
self.execute_script(&script).await
|
self.execute_script(&script).await
|
||||||
}
|
}
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<hr />
|
||||||
|
<h2 id="configuration-examples"><a class="header" href="#configuration-examples">Configuration Examples</a></h2>
|
||||||
---
|
<h3 id="core-package-config"><a class="header" href="#core-package-config">Core Package Config</a></h3>
|
||||||
|
<p><strong><code>/usr/local/share/provisioning/config/config.defaults.toml</code>:</strong></p>
|
||||||
## Configuration Examples
|
<pre><code class="language-toml">[orchestrator]
|
||||||
|
|
||||||
### Core Package Config
|
|
||||||
|
|
||||||
**`/usr/local/share/provisioning/config/config.defaults.toml`:**
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[orchestrator]
|
|
||||||
enabled = true
|
enabled = true
|
||||||
endpoint = "http://localhost:9090"
|
endpoint = "http://localhost:9090"
|
||||||
timeout_seconds = 60
|
timeout_seconds = 60
|
||||||
@ -875,14 +788,10 @@ force_direct = [
|
|||||||
"help",
|
"help",
|
||||||
"version"
|
"version"
|
||||||
]
|
]
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="platform-package-config"><a class="header" href="#platform-package-config">Platform Package Config</a></h3>
|
||||||
### Platform Package Config
|
<p><strong><code>/usr/local/share/provisioning/platform/config.toml</code>:</strong></p>
|
||||||
|
<pre><code class="language-toml">[server]
|
||||||
**`/usr/local/share/provisioning/platform/config.toml`:**
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[server]
|
|
||||||
host = "127.0.0.1"
|
host = "127.0.0.1"
|
||||||
port = 8080
|
port = 8080
|
||||||
|
|
||||||
@ -899,70 +808,61 @@ checkpoint_interval_seconds = 30
|
|||||||
binary = "nu" # Expects nu in PATH
|
binary = "nu" # Expects nu in PATH
|
||||||
provisioning_lib = "/usr/local/lib/provisioning"
|
provisioning_lib = "/usr/local/lib/provisioning"
|
||||||
env_vars = { NU_LIB_DIRS = "/usr/local/lib/provisioning" }
|
env_vars = { NU_LIB_DIRS = "/usr/local/lib/provisioning" }
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<hr />
|
||||||
---
|
<h2 id="key-takeaways"><a class="header" href="#key-takeaways">Key Takeaways</a></h2>
|
||||||
|
<h3 id="1-orchestrator-is-essential"><a class="header" href="#1-orchestrator-is-essential">1. <strong>Orchestrator is Essential</strong></a></h3>
|
||||||
## Key Takeaways
|
<ul>
|
||||||
|
<li>Solves deep call stack problems</li>
|
||||||
### 1. **Orchestrator is Essential**
|
<li>Provides performance optimization</li>
|
||||||
|
<li>Enables complex workflows</li>
|
||||||
- Solves deep call stack problems
|
<li>NOT optional for production use</li>
|
||||||
- Provides performance optimization
|
</ul>
|
||||||
- Enables complex workflows
|
<h3 id="2-integration-is-loose-but-coordinated"><a class="header" href="#2-integration-is-loose-but-coordinated">2. <strong>Integration is Loose but Coordinated</strong></a></h3>
|
||||||
- NOT optional for production use
|
<ul>
|
||||||
|
<li>No code dependencies between repos</li>
|
||||||
### 2. **Integration is Loose but Coordinated**
|
<li>Runtime integration via CLI + REST API</li>
|
||||||
|
<li>Configuration-driven coordination</li>
|
||||||
- No code dependencies between repos
|
<li>Works in both monorepo and multi-repo</li>
|
||||||
- Runtime integration via CLI + REST API
|
</ul>
|
||||||
- Configuration-driven coordination
|
<h3 id="3-best-of-both-worlds"><a class="header" href="#3-best-of-both-worlds">3. <strong>Best of Both Worlds</strong></a></h3>
|
||||||
- Works in both monorepo and multi-repo
|
<ul>
|
||||||
|
<li>Rust: High-performance coordination</li>
|
||||||
### 3. **Best of Both Worlds**
|
<li>Nushell: Flexible business logic</li>
|
||||||
|
<li>Clean separation of concerns</li>
|
||||||
- Rust: High-performance coordination
|
<li>Each technology does what it’s best at</li>
|
||||||
- Nushell: Flexible business logic
|
</ul>
|
||||||
- Clean separation of concerns
|
<h3 id="4-multi-repo-doesnt-change-integration"><a class="header" href="#4-multi-repo-doesnt-change-integration">4. <strong>Multi-Repo Doesn’t Change Integration</strong></a></h3>
|
||||||
- Each technology does what it's best at
|
<ul>
|
||||||
|
<li>Same runtime model as monorepo</li>
|
||||||
### 4. **Multi-Repo Doesn't Change Integration**
|
<li>Package installation sets up paths</li>
|
||||||
|
<li>Configuration enables discovery</li>
|
||||||
- Same runtime model as monorepo
|
<li>Versioning ensures compatibility</li>
|
||||||
- Package installation sets up paths
|
</ul>
|
||||||
- Configuration enables discovery
|
<hr />
|
||||||
- Versioning ensures compatibility
|
<h2 id="conclusion"><a class="header" href="#conclusion">Conclusion</a></h2>
|
||||||
|
<p>The confusing example in the multi-repo doc was <strong>oversimplified</strong>. The real architecture is:</p>
|
||||||
---
|
<pre><code class="language-plaintext">✅ Orchestrator IS USED and IS ESSENTIAL
|
||||||
|
|
||||||
## Conclusion
|
|
||||||
|
|
||||||
The confusing example in the multi-repo doc was **oversimplified**. The real architecture is:
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
✅ Orchestrator IS USED and IS ESSENTIAL
|
|
||||||
✅ Platform (Rust) coordinates Core (Nushell) execution
|
✅ Platform (Rust) coordinates Core (Nushell) execution
|
||||||
✅ Loose coupling via CLI + REST API (not code dependencies)
|
✅ Loose coupling via CLI + REST API (not code dependencies)
|
||||||
✅ Works identically in monorepo and multi-repo
|
✅ Works identically in monorepo and multi-repo
|
||||||
✅ Configuration-based integration (no hardcoded paths)
|
✅ Configuration-based integration (no hardcoded paths)
|
||||||
```plaintext
|
|
||||||
|
|
||||||
The orchestrator provides:
|
|
||||||
|
|
||||||
- Performance layer (async, parallel execution)
|
|
||||||
- Workflow engine (complex dependencies)
|
|
||||||
- State management (checkpoints, recovery)
|
|
||||||
- Task queue (reliable execution)
|
|
||||||
|
|
||||||
While Nushell provides:
|
|
||||||
|
|
||||||
- Business logic (providers, taskservs, clusters)
|
|
||||||
- Template rendering (Jinja2 via nu_plugin_tera)
|
|
||||||
- Configuration management (KCL integration)
|
|
||||||
- User-facing scripting
|
|
||||||
|
|
||||||
**Multi-repo just splits WHERE the code lives, not HOW it works together.**
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
|
<p>The orchestrator provides:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Performance layer (async, parallel execution)</li>
|
||||||
|
<li>Workflow engine (complex dependencies)</li>
|
||||||
|
<li>State management (checkpoints, recovery)</li>
|
||||||
|
<li>Task queue (reliable execution)</li>
|
||||||
|
</ul>
|
||||||
|
<p>While Nushell provides:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Business logic (providers, taskservs, clusters)</li>
|
||||||
|
<li>Template rendering (Jinja2 via nu_plugin_tera)</li>
|
||||||
|
<li>Configuration management (KCL integration)</li>
|
||||||
|
<li>User-facing scripting</li>
|
||||||
|
</ul>
|
||||||
|
<p><strong>Multi-repo just splits WHERE the code lives, not HOW it works together.</strong></p>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
|
|||||||
@ -190,7 +190,7 @@
|
|||||||
<ul>
|
<ul>
|
||||||
<li><strong>Rust compilation</strong>: Platform binaries (orchestrator, control-center, etc.)</li>
|
<li><strong>Rust compilation</strong>: Platform binaries (orchestrator, control-center, etc.)</li>
|
||||||
<li><strong>Nushell bundling</strong>: Core libraries and CLI tools</li>
|
<li><strong>Nushell bundling</strong>: Core libraries and CLI tools</li>
|
||||||
<li><strong>KCL validation</strong>: Configuration schema validation</li>
|
<li><strong>Nickel validation</strong>: Configuration schema validation</li>
|
||||||
<li><strong>Distribution generation</strong>: Multi-platform packages</li>
|
<li><strong>Distribution generation</strong>: Multi-platform packages</li>
|
||||||
<li><strong>Release management</strong>: Automated release pipelines</li>
|
<li><strong>Release management</strong>: Automated release pipelines</li>
|
||||||
<li><strong>Documentation generation</strong>: API and user documentation</li>
|
<li><strong>Documentation generation</strong>: API and user documentation</li>
|
||||||
@ -249,7 +249,7 @@ PARALLEL := true
|
|||||||
</ul>
|
</ul>
|
||||||
<p><strong><code>make build-all</code></strong> - Build all components</p>
|
<p><strong><code>make build-all</code></strong> - Build all components</p>
|
||||||
<ul>
|
<ul>
|
||||||
<li>Runs: <code>build-platform build-core validate-kcl</code></li>
|
<li>Runs: <code>build-platform build-core validate-nickel</code></li>
|
||||||
<li>Use for: Complete system compilation</li>
|
<li>Use for: Complete system compilation</li>
|
||||||
</ul>
|
</ul>
|
||||||
<p><strong><code>make build-platform</code></strong> - Build platform binaries for all targets</p>
|
<p><strong><code>make build-platform</code></strong> - Build platform binaries for all targets</p>
|
||||||
@ -270,11 +270,11 @@ nu tools/build/bundle-core.nu \
|
|||||||
--validate \
|
--validate \
|
||||||
--exclude-dev
|
--exclude-dev
|
||||||
</code></pre>
|
</code></pre>
|
||||||
<p><strong><code>make validate-kcl</code></strong> - Validate and compile KCL schemas</p>
|
<p><strong><code>make validate-nickel</code></strong> - Validate and compile Nickel schemas</p>
|
||||||
<pre><code class="language-bash">make validate-kcl
|
<pre><code class="language-bash">make validate-nickel
|
||||||
# Equivalent to:
|
# Equivalent to:
|
||||||
nu tools/build/validate-kcl.nu \
|
nu tools/build/validate-nickel.nu \
|
||||||
--output-dir dist/kcl \
|
--output-dir dist/schemas \
|
||||||
--format-code \
|
--format-code \
|
||||||
--check-dependencies
|
--check-dependencies
|
||||||
</code></pre>
|
</code></pre>
|
||||||
@ -374,7 +374,7 @@ make dist-generate PLATFORMS=linux-amd64,macos-amd64 VARIANTS=complete
|
|||||||
</ul>
|
</ul>
|
||||||
<p><strong><code>make validate-all</code></strong> - Validate all components</p>
|
<p><strong><code>make validate-all</code></strong> - Validate all components</p>
|
||||||
<ul>
|
<ul>
|
||||||
<li>KCL schema validation</li>
|
<li>Nickel schema validation</li>
|
||||||
<li>Package validation</li>
|
<li>Package validation</li>
|
||||||
<li>Configuration validation</li>
|
<li>Configuration validation</li>
|
||||||
</ul>
|
</ul>
|
||||||
@ -550,22 +550,22 @@ Options:
|
|||||||
<li>Function signature validation</li>
|
<li>Function signature validation</li>
|
||||||
<li>Test execution (if tests present)</li>
|
<li>Test execution (if tests present)</li>
|
||||||
</ul>
|
</ul>
|
||||||
<h4 id="srctoolsbuildvalidate-kclnu"><a class="header" href="#srctoolsbuildvalidate-kclnu"><code>/src/tools/build/validate-kcl.nu</code></a></h4>
|
<h4 id="srctoolsbuildvalidate-nickelnu"><a class="header" href="#srctoolsbuildvalidate-nickelnu"><code>/src/tools/build/validate-nickel.nu</code></a></h4>
|
||||||
<p><strong>Purpose</strong>: Validates and compiles KCL schemas</p>
|
<p><strong>Purpose</strong>: Validates and compiles Nickel schemas</p>
|
||||||
<p><strong>Validation Process</strong>:</p>
|
<p><strong>Validation Process</strong>:</p>
|
||||||
<ol>
|
<ol>
|
||||||
<li>Syntax validation of all <code>.k</code> files</li>
|
<li>Syntax validation of all <code>.ncl</code> files</li>
|
||||||
<li>Schema dependency checking</li>
|
<li>Schema dependency checking</li>
|
||||||
<li>Type constraint validation</li>
|
<li>Type constraint validation</li>
|
||||||
<li>Example validation against schemas</li>
|
<li>Example validation against schemas</li>
|
||||||
<li>Documentation generation</li>
|
<li>Documentation generation</li>
|
||||||
</ol>
|
</ol>
|
||||||
<p><strong>Usage</strong>:</p>
|
<p><strong>Usage</strong>:</p>
|
||||||
<pre><code class="language-bash">nu validate-kcl.nu [options]
|
<pre><code class="language-bash">nu validate-nickel.nu [options]
|
||||||
|
|
||||||
Options:
|
Options:
|
||||||
--output-dir STRING Output directory (default: dist/kcl)
|
--output-dir STRING Output directory (default: dist/schemas)
|
||||||
--format-code Format KCL code during validation
|
--format-code Format Nickel code during validation
|
||||||
--check-dependencies Validate schema dependencies
|
--check-dependencies Validate schema dependencies
|
||||||
--verbose Enable verbose logging
|
--verbose Enable verbose logging
|
||||||
</code></pre>
|
</code></pre>
|
||||||
@ -613,7 +613,7 @@ Options:
|
|||||||
<ol>
|
<ol>
|
||||||
<li>Platform binary compilation</li>
|
<li>Platform binary compilation</li>
|
||||||
<li>Core library bundling</li>
|
<li>Core library bundling</li>
|
||||||
<li>KCL schema validation and packaging</li>
|
<li>Nickel schema validation and packaging</li>
|
||||||
<li>Configuration system preparation</li>
|
<li>Configuration system preparation</li>
|
||||||
<li>Documentation generation</li>
|
<li>Documentation generation</li>
|
||||||
<li>Archive creation and compression</li>
|
<li>Archive creation and compression</li>
|
||||||
@ -818,8 +818,8 @@ make windows # Windows AMD64
|
|||||||
<pre><code class="language-bash"># Install Nushell
|
<pre><code class="language-bash"># Install Nushell
|
||||||
cargo install nu
|
cargo install nu
|
||||||
|
|
||||||
# Install KCL
|
# Install Nickel
|
||||||
cargo install kcl-cli
|
cargo install nickel
|
||||||
|
|
||||||
# Install Cross (for cross-compilation)
|
# Install Cross (for cross-compilation)
|
||||||
cargo install cross
|
cargo install cross
|
||||||
@ -873,17 +873,17 @@ chmod +x src/tools/build/*.nu
|
|||||||
cd src/tools
|
cd src/tools
|
||||||
nu build/compile-platform.nu --help
|
nu build/compile-platform.nu --help
|
||||||
</code></pre>
|
</code></pre>
|
||||||
<h4 id="kcl-validation-errors"><a class="header" href="#kcl-validation-errors">KCL Validation Errors</a></h4>
|
<h4 id="nickel-validation-errors"><a class="header" href="#nickel-validation-errors">Nickel Validation Errors</a></h4>
|
||||||
<p><strong>Error</strong>: <code>kcl command not found</code></p>
|
<p><strong>Error</strong>: <code>nickel command not found</code></p>
|
||||||
<pre><code class="language-bash"># Solution: Install KCL
|
<pre><code class="language-bash"># Solution: Install Nickel
|
||||||
cargo install kcl-cli
|
cargo install nickel
|
||||||
# or
|
# or
|
||||||
brew install kcl
|
brew install nickel
|
||||||
</code></pre>
|
</code></pre>
|
||||||
<p><strong>Error</strong>: Schema validation failed</p>
|
<p><strong>Error</strong>: Schema validation failed</p>
|
||||||
<pre><code class="language-bash"># Solution: Check KCL syntax
|
<pre><code class="language-bash"># Solution: Check Nickel syntax
|
||||||
kcl fmt kcl/
|
nickel fmt schemas/
|
||||||
kcl check kcl/
|
nickel check schemas/
|
||||||
</code></pre>
|
</code></pre>
|
||||||
<h3 id="build-performance-issues"><a class="header" href="#build-performance-issues">Build Performance Issues</a></h3>
|
<h3 id="build-performance-issues"><a class="header" href="#build-performance-issues">Build Performance Issues</a></h3>
|
||||||
<h4 id="slow-compilation"><a class="header" href="#slow-compilation">Slow Compilation</a></h4>
|
<h4 id="slow-compilation"><a class="header" href="#slow-compilation">Slow Compilation</a></h4>
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@ -222,21 +222,17 @@
|
|||||||
├── CI/CD Pipeline # Continuous integration/deployment
|
├── CI/CD Pipeline # Continuous integration/deployment
|
||||||
├── Data Platform # Data processing and analytics
|
├── Data Platform # Data processing and analytics
|
||||||
└── Custom Clusters # User-defined clusters
|
└── Custom Clusters # User-defined clusters
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="extension-discovery"><a class="header" href="#extension-discovery">Extension Discovery</a></h3>
|
||||||
### Extension Discovery
|
<p><strong>Discovery Order</strong>:</p>
|
||||||
|
<ol>
|
||||||
**Discovery Order**:
|
<li><code>workspace/extensions/{type}/{user}/{name}</code> - User-specific extensions</li>
|
||||||
|
<li><code>workspace/extensions/{type}/{name}</code> - Workspace shared extensions</li>
|
||||||
1. `workspace/extensions/{type}/{user}/{name}` - User-specific extensions
|
<li><code>workspace/extensions/{type}/template</code> - Templates</li>
|
||||||
2. `workspace/extensions/{type}/{name}` - Workspace shared extensions
|
<li>Core system paths (fallback)</li>
|
||||||
3. `workspace/extensions/{type}/template` - Templates
|
</ol>
|
||||||
4. Core system paths (fallback)
|
<p><strong>Path Resolution</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Automatic extension discovery
|
||||||
**Path Resolution**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Automatic extension discovery
|
|
||||||
use workspace/lib/path-resolver.nu
|
use workspace/lib/path-resolver.nu
|
||||||
|
|
||||||
# Find provider extension
|
# Find provider extension
|
||||||
@ -247,55 +243,42 @@ let taskservs = (path-resolver list_extensions "taskservs" --include-core)
|
|||||||
|
|
||||||
# Resolve cluster definition
|
# Resolve cluster definition
|
||||||
let cluster_path = (path-resolver resolve_extension "clusters" "web-stack")
|
let cluster_path = (path-resolver resolve_extension "clusters" "web-stack")
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="provider-development"><a class="header" href="#provider-development">Provider Development</a></h2>
|
||||||
## Provider Development
|
<h3 id="provider-architecture"><a class="header" href="#provider-architecture">Provider Architecture</a></h3>
|
||||||
|
<p>Providers implement cloud resource management through a standardized interface that supports multiple cloud platforms while maintaining consistent APIs.</p>
|
||||||
### Provider Architecture
|
<p><strong>Core Responsibilities</strong>:</p>
|
||||||
|
<ul>
|
||||||
Providers implement cloud resource management through a standardized interface that supports multiple cloud platforms while maintaining consistent APIs.
|
<li><strong>Authentication</strong>: Secure API authentication and credential management</li>
|
||||||
|
<li><strong>Resource Management</strong>: Server creation, deletion, and lifecycle management</li>
|
||||||
**Core Responsibilities**:
|
<li><strong>Configuration</strong>: Provider-specific settings and validation</li>
|
||||||
|
<li><strong>Error Handling</strong>: Comprehensive error handling and recovery</li>
|
||||||
- **Authentication**: Secure API authentication and credential management
|
<li><strong>Rate Limiting</strong>: API rate limiting and retry logic</li>
|
||||||
- **Resource Management**: Server creation, deletion, and lifecycle management
|
</ul>
|
||||||
- **Configuration**: Provider-specific settings and validation
|
<h3 id="creating-a-new-provider"><a class="header" href="#creating-a-new-provider">Creating a New Provider</a></h3>
|
||||||
- **Error Handling**: Comprehensive error handling and recovery
|
<p><strong>1. Initialize from Template</strong>:</p>
|
||||||
- **Rate Limiting**: API rate limiting and retry logic
|
<pre><code class="language-bash"># Copy provider template
|
||||||
|
|
||||||
### Creating a New Provider
|
|
||||||
|
|
||||||
**1. Initialize from Template**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Copy provider template
|
|
||||||
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-cloud
|
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-cloud
|
||||||
|
|
||||||
# Navigate to new provider
|
# Navigate to new provider
|
||||||
cd workspace/extensions/providers/my-cloud
|
cd workspace/extensions/providers/my-cloud
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>2. Update Configuration</strong>:</p>
|
||||||
**2. Update Configuration**:
|
<pre><code class="language-bash"># Initialize provider metadata
|
||||||
|
|
||||||
```bash
|
|
||||||
# Initialize provider metadata
|
|
||||||
nu init-provider.nu \
|
nu init-provider.nu \
|
||||||
--name "my-cloud" \
|
--name "my-cloud" \
|
||||||
--display-name "MyCloud Provider" \
|
--display-name "MyCloud Provider" \
|
||||||
--author "$USER" \
|
--author "$USER" \
|
||||||
--description "MyCloud platform integration"
|
--description "MyCloud platform integration"
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="provider-structure"><a class="header" href="#provider-structure">Provider Structure</a></h3>
|
||||||
### Provider Structure
|
<pre><code class="language-plaintext">my-cloud/
|
||||||
|
|
||||||
```plaintext
|
|
||||||
my-cloud/
|
|
||||||
├── README.md # Provider documentation
|
├── README.md # Provider documentation
|
||||||
├── kcl/ # KCL configuration schemas
|
├── schemas/ # Nickel configuration schemas
|
||||||
│ ├── settings.k # Provider settings schema
|
│ ├── settings.ncl # Provider settings schema
|
||||||
│ ├── servers.k # Server configuration schema
|
│ ├── servers.ncl # Server configuration schema
|
||||||
│ ├── networks.k # Network configuration schema
|
│ ├── networks.ncl # Network configuration schema
|
||||||
│ └── kcl.mod # KCL module dependencies
|
│ └── manifest.toml # Nickel module dependencies
|
||||||
├── nulib/ # Nushell implementation
|
├── nulib/ # Nushell implementation
|
||||||
│ ├── provider.nu # Main provider interface
|
│ ├── provider.nu # Main provider interface
|
||||||
│ ├── servers/ # Server management
|
│ ├── servers/ # Server management
|
||||||
@ -330,14 +313,10 @@ my-cloud/
|
|||||||
└── mock/ # Mock data and services
|
└── mock/ # Mock data and services
|
||||||
├── api-responses.json # Mock API responses
|
├── api-responses.json # Mock API responses
|
||||||
└── test-configs.toml # Test configurations
|
└── test-configs.toml # Test configurations
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="provider-implementation"><a class="header" href="#provider-implementation">Provider Implementation</a></h3>
|
||||||
### Provider Implementation
|
<p><strong>Main Provider Interface</strong> (<code>nulib/provider.nu</code>):</p>
|
||||||
|
<pre><code class="language-nushell">#!/usr/bin/env nu
|
||||||
**Main Provider Interface** (`nulib/provider.nu`):
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
#!/usr/bin/env nu
|
|
||||||
# MyCloud Provider Implementation
|
# MyCloud Provider Implementation
|
||||||
|
|
||||||
# Provider metadata
|
# Provider metadata
|
||||||
@ -494,12 +473,9 @@ export def "provider test" [
|
|||||||
_ => (error make {msg: $"Unknown test type: ($test_type)"})
|
_ => (error make {msg: $"Unknown test type: ($test_type)"})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Authentication Module</strong> (<code>nulib/auth/client.nu</code>):</p>
|
||||||
**Authentication Module** (`nulib/auth/client.nu`):
|
<pre><code class="language-nushell"># API client setup and authentication
|
||||||
|
|
||||||
```nushell
|
|
||||||
# API client setup and authentication
|
|
||||||
|
|
||||||
export def setup_api_client [config: record] -> record {
|
export def setup_api_client [config: record] -> record {
|
||||||
# Validate credentials
|
# Validate credentials
|
||||||
@ -541,83 +517,64 @@ def test_auth_api [client: record] -> bool {
|
|||||||
|
|
||||||
$response.status == "success"
|
$response.status == "success"
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Nickel Configuration Schema</strong> (<code>schemas/settings.ncl</code>):</p>
|
||||||
|
<pre><code class="language-nickel"># MyCloud Provider Configuration Schema
|
||||||
|
|
||||||
**KCL Configuration Schema** (`kcl/settings.k`):
|
let MyCloudConfig = {
|
||||||
|
# MyCloud provider configuration
|
||||||
```kcl
|
api_url | string | default = "https://api.my-cloud.com",
|
||||||
# MyCloud Provider Configuration Schema
|
api_key | string,
|
||||||
|
api_secret | string,
|
||||||
schema MyCloudConfig:
|
timeout | number | default = 30,
|
||||||
"""MyCloud provider configuration"""
|
retries | number | default = 3,
|
||||||
|
|
||||||
api_url?: str = "https://api.my-cloud.com"
|
|
||||||
api_key: str
|
|
||||||
api_secret: str
|
|
||||||
timeout?: int = 30
|
|
||||||
retries?: int = 3
|
|
||||||
|
|
||||||
# Rate limiting
|
# Rate limiting
|
||||||
rate_limit?: {
|
rate_limit | {
|
||||||
requests_per_minute?: int = 60
|
requests_per_minute | number | default = 60,
|
||||||
burst_size?: int = 10
|
burst_size | number | default = 10,
|
||||||
} = {}
|
} | default = {},
|
||||||
|
|
||||||
# Default settings
|
# Default settings
|
||||||
defaults?: {
|
defaults | {
|
||||||
zone?: str = "us-east-1"
|
zone | string | default = "us-east-1",
|
||||||
template?: str = "ubuntu-22.04"
|
template | string | default = "ubuntu-22.04",
|
||||||
network?: str = "default"
|
network | string | default = "default",
|
||||||
} = {}
|
} | default = {},
|
||||||
|
} in
|
||||||
|
MyCloudConfig
|
||||||
|
|
||||||
check:
|
let MyCloudServerConfig = {
|
||||||
len(api_key) > 0, "API key cannot be empty"
|
# MyCloud server configuration
|
||||||
len(api_secret) > 0, "API secret cannot be empty"
|
name | string,
|
||||||
timeout > 0, "Timeout must be positive"
|
plan | string,
|
||||||
retries >= 0, "Retries must be non-negative"
|
zone | string | optional,
|
||||||
|
template | string | default = "ubuntu-22.04",
|
||||||
schema MyCloudServerConfig:
|
storage | number | default = 25,
|
||||||
"""MyCloud server configuration"""
|
tags | { } | default = {},
|
||||||
|
|
||||||
name: str
|
|
||||||
plan: str
|
|
||||||
zone?: str
|
|
||||||
template?: str = "ubuntu-22.04"
|
|
||||||
storage?: int = 25
|
|
||||||
tags?: {str: str} = {}
|
|
||||||
|
|
||||||
# Network configuration
|
# Network configuration
|
||||||
network?: {
|
network | {
|
||||||
vpc_id?: str
|
vpc_id | string | optional,
|
||||||
subnet_id?: str
|
subnet_id | string | optional,
|
||||||
public_ip?: bool = true
|
public_ip | bool | default = true,
|
||||||
firewall_rules?: [FirewallRule] = []
|
firewall_rules | array | default = [],
|
||||||
}
|
} | optional,
|
||||||
|
} in
|
||||||
|
MyCloudServerConfig
|
||||||
|
|
||||||
check:
|
let FirewallRule = {
|
||||||
len(name) > 0, "Server name cannot be empty"
|
# Firewall rule configuration
|
||||||
plan in ["small", "medium", "large", "xlarge"], "Invalid plan"
|
port | (number | string),
|
||||||
storage >= 10, "Minimum storage is 10GB"
|
protocol | string | default = "tcp",
|
||||||
storage <= 2048, "Maximum storage is 2TB"
|
source | string | default = "0.0.0.0/0",
|
||||||
|
description | string | optional,
|
||||||
schema FirewallRule:
|
} in
|
||||||
"""Firewall rule configuration"""
|
FirewallRule
|
||||||
|
</code></pre>
|
||||||
port: int | str
|
<h3 id="provider-testing"><a class="header" href="#provider-testing">Provider Testing</a></h3>
|
||||||
protocol: str = "tcp"
|
<p><strong>Unit Testing</strong> (<code>tests/unit/test-servers.nu</code>):</p>
|
||||||
source: str = "0.0.0.0/0"
|
<pre><code class="language-nushell"># Unit tests for server management
|
||||||
description?: str
|
|
||||||
|
|
||||||
check:
|
|
||||||
protocol in ["tcp", "udp", "icmp"], "Invalid protocol"
|
|
||||||
```plaintext
|
|
||||||
|
|
||||||
### Provider Testing
|
|
||||||
|
|
||||||
**Unit Testing** (`tests/unit/test-servers.nu`):
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Unit tests for server management
|
|
||||||
|
|
||||||
use ../../../nulib/provider.nu
|
use ../../../nulib/provider.nu
|
||||||
|
|
||||||
@ -664,12 +621,9 @@ def main [] {
|
|||||||
test_invalid_plan
|
test_invalid_plan
|
||||||
print "✅ All server management tests passed"
|
print "✅ All server management tests passed"
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Integration Testing</strong> (<code>tests/integration/test-lifecycle.nu</code>):</p>
|
||||||
**Integration Testing** (`tests/integration/test-lifecycle.nu`):
|
<pre><code class="language-nushell"># Integration tests for complete server lifecycle
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Integration tests for complete server lifecycle
|
|
||||||
|
|
||||||
use ../../../nulib/provider.nu
|
use ../../../nulib/provider.nu
|
||||||
|
|
||||||
@ -702,54 +656,41 @@ def main [] {
|
|||||||
test_complete_lifecycle
|
test_complete_lifecycle
|
||||||
print "✅ All integration tests passed"
|
print "✅ All integration tests passed"
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="task-service-development"><a class="header" href="#task-service-development">Task Service Development</a></h2>
|
||||||
## Task Service Development
|
<h3 id="task-service-architecture"><a class="header" href="#task-service-architecture">Task Service Architecture</a></h3>
|
||||||
|
<p>Task services are infrastructure components that can be deployed and managed across different environments. They provide standardized interfaces for installation, configuration, and lifecycle management.</p>
|
||||||
### Task Service Architecture
|
<p><strong>Core Responsibilities</strong>:</p>
|
||||||
|
<ul>
|
||||||
Task services are infrastructure components that can be deployed and managed across different environments. They provide standardized interfaces for installation, configuration, and lifecycle management.
|
<li><strong>Installation</strong>: Service deployment and setup</li>
|
||||||
|
<li><strong>Configuration</strong>: Dynamic configuration management</li>
|
||||||
**Core Responsibilities**:
|
<li><strong>Health Checking</strong>: Service status monitoring</li>
|
||||||
|
<li><strong>Version Management</strong>: Automatic version updates from GitHub</li>
|
||||||
- **Installation**: Service deployment and setup
|
<li><strong>Integration</strong>: Integration with other services and clusters</li>
|
||||||
- **Configuration**: Dynamic configuration management
|
</ul>
|
||||||
- **Health Checking**: Service status monitoring
|
<h3 id="creating-a-new-task-service"><a class="header" href="#creating-a-new-task-service">Creating a New Task Service</a></h3>
|
||||||
- **Version Management**: Automatic version updates from GitHub
|
<p><strong>1. Initialize from Template</strong>:</p>
|
||||||
- **Integration**: Integration with other services and clusters
|
<pre><code class="language-bash"># Copy task service template
|
||||||
|
|
||||||
### Creating a New Task Service
|
|
||||||
|
|
||||||
**1. Initialize from Template**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Copy task service template
|
|
||||||
cp -r workspace/extensions/taskservs/template workspace/extensions/taskservs/my-service
|
cp -r workspace/extensions/taskservs/template workspace/extensions/taskservs/my-service
|
||||||
|
|
||||||
# Navigate to new service
|
# Navigate to new service
|
||||||
cd workspace/extensions/taskservs/my-service
|
cd workspace/extensions/taskservs/my-service
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>2. Initialize Service</strong>:</p>
|
||||||
**2. Initialize Service**:
|
<pre><code class="language-bash"># Initialize service metadata
|
||||||
|
|
||||||
```bash
|
|
||||||
# Initialize service metadata
|
|
||||||
nu init-service.nu \
|
nu init-service.nu \
|
||||||
--name "my-service" \
|
--name "my-service" \
|
||||||
--display-name "My Custom Service" \
|
--display-name "My Custom Service" \
|
||||||
--type "database" \
|
--type "database" \
|
||||||
--github-repo "myorg/my-service"
|
--github-repo "myorg/my-service"
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="task-service-structure"><a class="header" href="#task-service-structure">Task Service Structure</a></h3>
|
||||||
### Task Service Structure
|
<pre><code class="language-plaintext">my-service/
|
||||||
|
|
||||||
```plaintext
|
|
||||||
my-service/
|
|
||||||
├── README.md # Service documentation
|
├── README.md # Service documentation
|
||||||
├── kcl/ # KCL schemas
|
├── schemas/ # Nickel schemas
|
||||||
│ ├── version.k # Version and GitHub integration
|
│ ├── version.ncl # Version and GitHub integration
|
||||||
│ ├── config.k # Service configuration schema
|
│ ├── config.ncl # Service configuration schema
|
||||||
│ └── kcl.mod # Module dependencies
|
│ └── manifest.toml # Module dependencies
|
||||||
├── nushell/ # Nushell implementation
|
├── nushell/ # Nushell implementation
|
||||||
│ ├── taskserv.nu # Main service interface
|
│ ├── taskserv.nu # Main service interface
|
||||||
│ ├── install.nu # Installation logic
|
│ ├── install.nu # Installation logic
|
||||||
@ -776,14 +717,10 @@ my-service/
|
|||||||
├── unit/ # Unit tests
|
├── unit/ # Unit tests
|
||||||
├── integration/ # Integration tests
|
├── integration/ # Integration tests
|
||||||
└── fixtures/ # Test fixtures and data
|
└── fixtures/ # Test fixtures and data
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="task-service-implementation"><a class="header" href="#task-service-implementation">Task Service Implementation</a></h3>
|
||||||
### Task Service Implementation
|
<p><strong>Main Service Interface</strong> (<code>nushell/taskserv.nu</code>):</p>
|
||||||
|
<pre><code class="language-nushell">#!/usr/bin/env nu
|
||||||
**Main Service Interface** (`nushell/taskserv.nu`):
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
#!/usr/bin/env nu
|
|
||||||
# My Custom Service Task Service Implementation
|
# My Custom Service Task Service Implementation
|
||||||
|
|
||||||
export const SERVICE_NAME = "my-service"
|
export const SERVICE_NAME = "my-service"
|
||||||
@ -986,136 +923,120 @@ export def "taskserv test" [
|
|||||||
_ => (error make {msg: $"Unknown test type: ($test_type)"})
|
_ => (error make {msg: $"Unknown test type: ($test_type)"})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Version Configuration</strong> (<code>schemas/version.ncl</code>):</p>
|
||||||
|
<pre><code class="language-nickel"># Version management with GitHub integration
|
||||||
|
|
||||||
**Version Configuration** (`kcl/version.k`):
|
let version_config = {
|
||||||
|
service_name = "my-service",
|
||||||
```kcl
|
|
||||||
# Version management with GitHub integration
|
|
||||||
|
|
||||||
version_config: VersionConfig = {
|
|
||||||
service_name = "my-service"
|
|
||||||
|
|
||||||
# GitHub repository for version checking
|
# GitHub repository for version checking
|
||||||
github = {
|
github = {
|
||||||
owner = "myorg"
|
owner = "myorg",
|
||||||
repo = "my-service"
|
repo = "my-service",
|
||||||
|
|
||||||
# Release configuration
|
# Release configuration
|
||||||
release = {
|
release = {
|
||||||
tag_prefix = "v"
|
tag_prefix = "v",
|
||||||
prerelease = false
|
prerelease = false,
|
||||||
draft = false
|
draft = false,
|
||||||
}
|
},
|
||||||
|
|
||||||
# Asset patterns for different platforms
|
# Asset patterns for different platforms
|
||||||
assets = {
|
assets = {
|
||||||
linux_amd64 = "my-service-{version}-linux-amd64.tar.gz"
|
linux_amd64 = "my-service-{version}-linux-amd64.tar.gz",
|
||||||
darwin_amd64 = "my-service-{version}-darwin-amd64.tar.gz"
|
darwin_amd64 = "my-service-{version}-darwin-amd64.tar.gz",
|
||||||
windows_amd64 = "my-service-{version}-windows-amd64.zip"
|
windows_amd64 = "my-service-{version}-windows-amd64.zip",
|
||||||
}
|
},
|
||||||
}
|
},
|
||||||
|
|
||||||
# Version constraints and compatibility
|
# Version constraints and compatibility
|
||||||
compatibility = {
|
compatibility = {
|
||||||
min_kubernetes_version = "1.20.0"
|
min_kubernetes_version = "1.20.0",
|
||||||
max_kubernetes_version = "1.28.*"
|
max_kubernetes_version = "1.28.*",
|
||||||
|
|
||||||
# Dependencies
|
# Dependencies
|
||||||
requires = {
|
requires = {
|
||||||
"cert-manager": ">=1.8.0"
|
"cert-manager" = ">=1.8.0",
|
||||||
"ingress-nginx": ">=1.0.0"
|
"ingress-nginx" = ">=1.0.0",
|
||||||
}
|
},
|
||||||
|
|
||||||
# Conflicts
|
# Conflicts
|
||||||
conflicts = {
|
conflicts = {
|
||||||
"old-my-service": "*"
|
"old-my-service" = "*",
|
||||||
}
|
},
|
||||||
}
|
},
|
||||||
|
|
||||||
# Installation configuration
|
# Installation configuration
|
||||||
installation = {
|
installation = {
|
||||||
default_namespace = "my-service"
|
default_namespace = "my-service",
|
||||||
create_namespace = true
|
create_namespace = true,
|
||||||
|
|
||||||
# Resource requirements
|
# Resource requirements
|
||||||
resources = {
|
resources = {
|
||||||
requests = {
|
requests = {
|
||||||
cpu = "100m"
|
cpu = "100m",
|
||||||
memory = "128Mi"
|
memory = "128Mi",
|
||||||
}
|
},
|
||||||
limits = {
|
limits = {
|
||||||
cpu = "500m"
|
cpu = "500m",
|
||||||
memory = "512Mi"
|
memory = "512Mi",
|
||||||
}
|
},
|
||||||
}
|
},
|
||||||
|
|
||||||
# Persistence
|
# Persistence
|
||||||
persistence = {
|
persistence = {
|
||||||
enabled = true
|
enabled = true,
|
||||||
storage_class = "default"
|
storage_class = "default",
|
||||||
size = "10Gi"
|
size = "10Gi",
|
||||||
}
|
},
|
||||||
}
|
},
|
||||||
|
|
||||||
# Health check configuration
|
# Health check configuration
|
||||||
health_check = {
|
health_check = {
|
||||||
initial_delay_seconds = 30
|
initial_delay_seconds = 30,
|
||||||
period_seconds = 10
|
period_seconds = 10,
|
||||||
timeout_seconds = 5
|
timeout_seconds = 5,
|
||||||
failure_threshold = 3
|
failure_threshold = 3,
|
||||||
|
|
||||||
# Health endpoints
|
# Health endpoints
|
||||||
endpoints = {
|
endpoints = {
|
||||||
liveness = "/health/live"
|
liveness = "/health/live",
|
||||||
readiness = "/health/ready"
|
readiness = "/health/ready",
|
||||||
}
|
},
|
||||||
}
|
},
|
||||||
}
|
} in
|
||||||
```plaintext
|
version_config
|
||||||
|
</code></pre>
|
||||||
## Cluster Development
|
<h2 id="cluster-development"><a class="header" href="#cluster-development">Cluster Development</a></h2>
|
||||||
|
<h3 id="cluster-architecture"><a class="header" href="#cluster-architecture">Cluster Architecture</a></h3>
|
||||||
### Cluster Architecture
|
<p>Clusters represent complete deployment solutions that combine multiple task services, providers, and configurations to create functional environments.</p>
|
||||||
|
<p><strong>Core Responsibilities</strong>:</p>
|
||||||
Clusters represent complete deployment solutions that combine multiple task services, providers, and configurations to create functional environments.
|
<ul>
|
||||||
|
<li><strong>Service Orchestration</strong>: Coordinate multiple task service deployments</li>
|
||||||
**Core Responsibilities**:
|
<li><strong>Dependency Management</strong>: Handle service dependencies and startup order</li>
|
||||||
|
<li><strong>Configuration Management</strong>: Manage cross-service configuration</li>
|
||||||
- **Service Orchestration**: Coordinate multiple task service deployments
|
<li><strong>Health Monitoring</strong>: Monitor overall cluster health</li>
|
||||||
- **Dependency Management**: Handle service dependencies and startup order
|
<li><strong>Scaling</strong>: Handle cluster scaling operations</li>
|
||||||
- **Configuration Management**: Manage cross-service configuration
|
</ul>
|
||||||
- **Health Monitoring**: Monitor overall cluster health
|
<h3 id="creating-a-new-cluster"><a class="header" href="#creating-a-new-cluster">Creating a New Cluster</a></h3>
|
||||||
- **Scaling**: Handle cluster scaling operations
|
<p><strong>1. Initialize from Template</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># Copy cluster template
|
||||||
### Creating a New Cluster
|
|
||||||
|
|
||||||
**1. Initialize from Template**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Copy cluster template
|
|
||||||
cp -r workspace/extensions/clusters/template workspace/extensions/clusters/my-stack
|
cp -r workspace/extensions/clusters/template workspace/extensions/clusters/my-stack
|
||||||
|
|
||||||
# Navigate to new cluster
|
# Navigate to new cluster
|
||||||
cd workspace/extensions/clusters/my-stack
|
cd workspace/extensions/clusters/my-stack
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>2. Initialize Cluster</strong>:</p>
|
||||||
**2. Initialize Cluster**:
|
<pre><code class="language-bash"># Initialize cluster metadata
|
||||||
|
|
||||||
```bash
|
|
||||||
# Initialize cluster metadata
|
|
||||||
nu init-cluster.nu \
|
nu init-cluster.nu \
|
||||||
--name "my-stack" \
|
--name "my-stack" \
|
||||||
--display-name "My Application Stack" \
|
--display-name "My Application Stack" \
|
||||||
--type "web-application"
|
--type "web-application"
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="cluster-implementation"><a class="header" href="#cluster-implementation">Cluster Implementation</a></h3>
|
||||||
### Cluster Implementation
|
<p><strong>Main Cluster Interface</strong> (<code>nushell/cluster.nu</code>):</p>
|
||||||
|
<pre><code class="language-nushell">#!/usr/bin/env nu
|
||||||
**Main Cluster Interface** (`nushell/cluster.nu`):
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
#!/usr/bin/env nu
|
|
||||||
# My Application Stack Cluster Implementation
|
# My Application Stack Cluster Implementation
|
||||||
|
|
||||||
export const CLUSTER_NAME = "my-stack"
|
export const CLUSTER_NAME = "my-stack"
|
||||||
@ -1223,26 +1144,20 @@ export def "cluster delete" [
|
|||||||
deleted_at: (date now)
|
deleted_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="testing-and-validation"><a class="header" href="#testing-and-validation">Testing and Validation</a></h2>
|
||||||
## Testing and Validation
|
<h3 id="testing-framework"><a class="header" href="#testing-framework">Testing Framework</a></h3>
|
||||||
|
<p><strong>Test Types</strong>:</p>
|
||||||
### Testing Framework
|
<ul>
|
||||||
|
<li><strong>Unit Tests</strong>: Individual function and module testing</li>
|
||||||
**Test Types**:
|
<li><strong>Integration Tests</strong>: Cross-component interaction testing</li>
|
||||||
|
<li><strong>End-to-End Tests</strong>: Complete workflow testing</li>
|
||||||
- **Unit Tests**: Individual function and module testing
|
<li><strong>Performance Tests</strong>: Load and performance validation</li>
|
||||||
- **Integration Tests**: Cross-component interaction testing
|
<li><strong>Security Tests</strong>: Security and vulnerability testing</li>
|
||||||
- **End-to-End Tests**: Complete workflow testing
|
</ul>
|
||||||
- **Performance Tests**: Load and performance validation
|
<h3 id="extension-testing-commands"><a class="header" href="#extension-testing-commands">Extension Testing Commands</a></h3>
|
||||||
- **Security Tests**: Security and vulnerability testing
|
<p><strong>Workspace Testing Tools</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># Validate extension syntax and structure
|
||||||
### Extension Testing Commands
|
|
||||||
|
|
||||||
**Workspace Testing Tools**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Validate extension syntax and structure
|
|
||||||
nu workspace.nu tools validate-extension providers/my-cloud
|
nu workspace.nu tools validate-extension providers/my-cloud
|
||||||
|
|
||||||
# Run extension unit tests
|
# Run extension unit tests
|
||||||
@ -1253,14 +1168,10 @@ nu workspace.nu tools test-extension clusters/my-stack --test-type integration -
|
|||||||
|
|
||||||
# Performance testing
|
# Performance testing
|
||||||
nu workspace.nu tools test-extension providers/my-cloud --test-type performance --duration 5m
|
nu workspace.nu tools test-extension providers/my-cloud --test-type performance --duration 5m
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="automated-testing"><a class="header" href="#automated-testing">Automated Testing</a></h3>
|
||||||
### Automated Testing
|
<p><strong>Test Runner</strong> (<code>tests/run-tests.nu</code>):</p>
|
||||||
|
<pre><code class="language-nushell">#!/usr/bin/env nu
|
||||||
**Test Runner** (`tests/run-tests.nu`):
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
#!/usr/bin/env nu
|
|
||||||
# Automated test runner for extensions
|
# Automated test runner for extensions
|
||||||
|
|
||||||
def main [
|
def main [
|
||||||
@ -1320,24 +1231,19 @@ def main [
|
|||||||
completed_at: (date now)
|
completed_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="publishing-and-distribution"><a class="header" href="#publishing-and-distribution">Publishing and Distribution</a></h2>
|
||||||
## Publishing and Distribution
|
<h3 id="extension-publishing"><a class="header" href="#extension-publishing">Extension Publishing</a></h3>
|
||||||
|
<p><strong>Publishing Process</strong>:</p>
|
||||||
### Extension Publishing
|
<ol>
|
||||||
|
<li><strong>Validation</strong>: Comprehensive testing and validation</li>
|
||||||
**Publishing Process**:
|
<li><strong>Documentation</strong>: Complete documentation and examples</li>
|
||||||
|
<li><strong>Packaging</strong>: Create distribution packages</li>
|
||||||
1. **Validation**: Comprehensive testing and validation
|
<li><strong>Registry</strong>: Publish to extension registry</li>
|
||||||
2. **Documentation**: Complete documentation and examples
|
<li><strong>Versioning</strong>: Semantic version tagging</li>
|
||||||
3. **Packaging**: Create distribution packages
|
</ol>
|
||||||
4. **Registry**: Publish to extension registry
|
<h3 id="publishing-commands"><a class="header" href="#publishing-commands">Publishing Commands</a></h3>
|
||||||
5. **Versioning**: Semantic version tagging
|
<pre><code class="language-bash"># Validate extension for publishing
|
||||||
|
|
||||||
### Publishing Commands
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Validate extension for publishing
|
|
||||||
nu workspace.nu tools validate-for-publish providers/my-cloud
|
nu workspace.nu tools validate-for-publish providers/my-cloud
|
||||||
|
|
||||||
# Create distribution package
|
# Create distribution package
|
||||||
@ -1348,14 +1254,10 @@ nu workspace.nu tools publish-extension providers/my-cloud --registry official
|
|||||||
|
|
||||||
# Tag version
|
# Tag version
|
||||||
nu workspace.nu tools tag-extension providers/my-cloud --version 1.0.0 --push
|
nu workspace.nu tools tag-extension providers/my-cloud --version 1.0.0 --push
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="extension-registry"><a class="header" href="#extension-registry">Extension Registry</a></h3>
|
||||||
### Extension Registry
|
<p><strong>Registry Structure</strong>:</p>
|
||||||
|
<pre><code class="language-plaintext">Extension Registry
|
||||||
**Registry Structure**:
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
Extension Registry
|
|
||||||
├── providers/
|
├── providers/
|
||||||
│ ├── aws/ # Official AWS provider
|
│ ├── aws/ # Official AWS provider
|
||||||
│ ├── upcloud/ # Official UpCloud provider
|
│ ├── upcloud/ # Official UpCloud provider
|
||||||
@ -1368,16 +1270,11 @@ Extension Registry
|
|||||||
├── web-stacks/ # Web application stacks
|
├── web-stacks/ # Web application stacks
|
||||||
├── data-platforms/ # Data processing platforms
|
├── data-platforms/ # Data processing platforms
|
||||||
└── ci-cd/ # CI/CD pipelines
|
└── ci-cd/ # CI/CD pipelines
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="best-practices"><a class="header" href="#best-practices">Best Practices</a></h2>
|
||||||
## Best Practices
|
<h3 id="code-quality"><a class="header" href="#code-quality">Code Quality</a></h3>
|
||||||
|
<p><strong>Function Design</strong>:</p>
|
||||||
### Code Quality
|
<pre><code class="language-nushell"># Good: Single responsibility, clear parameters, comprehensive error handling
|
||||||
|
|
||||||
**Function Design**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Good: Single responsibility, clear parameters, comprehensive error handling
|
|
||||||
export def "provider create-server" [
|
export def "provider create-server" [
|
||||||
name: string # Server name (must be unique in region)
|
name: string # Server name (must be unique in region)
|
||||||
plan: string # Server plan (see list-plans for options)
|
plan: string # Server plan (see list-plans for options)
|
||||||
@ -1401,12 +1298,9 @@ def create [n, p] {
|
|||||||
# Missing validation and error handling
|
# Missing validation and error handling
|
||||||
api_call $n $p
|
api_call $n $p
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Configuration Management</strong>:</p>
|
||||||
**Configuration Management**:
|
<pre><code class="language-nushell"># Good: Configuration-driven with validation
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Good: Configuration-driven with validation
|
|
||||||
def get_api_endpoint [provider: string] -> string {
|
def get_api_endpoint [provider: string] -> string {
|
||||||
let config = get-config-value $"providers.($provider).api_url"
|
let config = get-config-value $"providers.($provider).api_url"
|
||||||
|
|
||||||
@ -1424,14 +1318,10 @@ def get_api_endpoint [provider: string] -> string {
|
|||||||
def get_api_endpoint [] {
|
def get_api_endpoint [] {
|
||||||
"https://api.provider.com" # Never hardcode!
|
"https://api.provider.com" # Never hardcode!
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="error-handling"><a class="header" href="#error-handling">Error Handling</a></h3>
|
||||||
### Error Handling
|
<p><strong>Comprehensive Error Context</strong>:</p>
|
||||||
|
<pre><code class="language-nushell">def create_server_with_context [name: string, config: record] -> record {
|
||||||
**Comprehensive Error Context**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
def create_server_with_context [name: string, config: record] -> record {
|
|
||||||
try {
|
try {
|
||||||
# Validate configuration
|
# Validate configuration
|
||||||
validate_server_config $config
|
validate_server_config $config
|
||||||
@ -1470,14 +1360,10 @@ def create_server_with_context [name: string, config: record] -> record {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="testing-practices"><a class="header" href="#testing-practices">Testing Practices</a></h3>
|
||||||
### Testing Practices
|
<p><strong>Test Organization</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Organize tests by functionality
|
||||||
**Test Organization**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Organize tests by functionality
|
|
||||||
# tests/unit/server-creation-test.nu
|
# tests/unit/server-creation-test.nu
|
||||||
|
|
||||||
def test_valid_server_creation [] {
|
def test_valid_server_creation [] {
|
||||||
@ -1513,14 +1399,10 @@ def test_invalid_inputs [] {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="documentation-standards"><a class="header" href="#documentation-standards">Documentation Standards</a></h3>
|
||||||
### Documentation Standards
|
<p><strong>Function Documentation</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Comprehensive function documentation
|
||||||
**Function Documentation**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Comprehensive function documentation
|
|
||||||
def "provider create-server" [
|
def "provider create-server" [
|
||||||
name: string # Server name - must be unique within the provider
|
name: string # Server name - must be unique within the provider
|
||||||
plan: string # Server size plan (run 'provider list-plans' for options)
|
plan: string # Server size plan (run 'provider list-plans' for options)
|
||||||
@ -1557,74 +1439,52 @@ def "provider create-server" [
|
|||||||
|
|
||||||
# Implementation...
|
# Implementation...
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="troubleshooting"><a class="header" href="#troubleshooting">Troubleshooting</a></h2>
|
||||||
## Troubleshooting
|
<h3 id="common-development-issues"><a class="header" href="#common-development-issues">Common Development Issues</a></h3>
|
||||||
|
<h4 id="extension-not-found"><a class="header" href="#extension-not-found">Extension Not Found</a></h4>
|
||||||
### Common Development Issues
|
<p><strong>Error</strong>: <code>Extension 'my-provider' not found</code></p>
|
||||||
|
<pre><code class="language-bash"># Solution: Check extension location and structure
|
||||||
#### Extension Not Found
|
|
||||||
|
|
||||||
**Error**: `Extension 'my-provider' not found`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Solution: Check extension location and structure
|
|
||||||
ls -la workspace/extensions/providers/my-provider
|
ls -la workspace/extensions/providers/my-provider
|
||||||
nu workspace/lib/path-resolver.nu resolve_extension "providers" "my-provider"
|
nu workspace/lib/path-resolver.nu resolve_extension "providers" "my-provider"
|
||||||
|
|
||||||
# Validate extension structure
|
# Validate extension structure
|
||||||
nu workspace.nu tools validate-extension providers/my-provider
|
nu workspace.nu tools validate-extension providers/my-provider
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="configuration-errors"><a class="header" href="#configuration-errors">Configuration Errors</a></h4>
|
||||||
|
<p><strong>Error</strong>: <code>Invalid Nickel configuration</code></p>
|
||||||
|
<pre><code class="language-bash"># Solution: Validate Nickel syntax
|
||||||
|
nickel check workspace/extensions/providers/my-provider/schemas/
|
||||||
|
|
||||||
#### Configuration Errors
|
# Format Nickel files
|
||||||
|
nickel fmt workspace/extensions/providers/my-provider/schemas/
|
||||||
**Error**: `Invalid KCL configuration`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Solution: Validate KCL syntax
|
|
||||||
kcl check workspace/extensions/providers/my-provider/kcl/
|
|
||||||
|
|
||||||
# Format KCL files
|
|
||||||
kcl fmt workspace/extensions/providers/my-provider/kcl/
|
|
||||||
|
|
||||||
# Test with example data
|
# Test with example data
|
||||||
kcl run workspace/extensions/providers/my-provider/kcl/settings.k -D api_key="test"
|
nickel eval workspace/extensions/providers/my-provider/schemas/settings.ncl
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="api-integration-issues"><a class="header" href="#api-integration-issues">API Integration Issues</a></h4>
|
||||||
#### API Integration Issues
|
<p><strong>Error</strong>: <code>Authentication failed</code></p>
|
||||||
|
<pre><code class="language-bash"># Solution: Test credentials and connectivity
|
||||||
**Error**: `Authentication failed`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Solution: Test credentials and connectivity
|
|
||||||
curl -H "Authorization: Bearer $API_KEY" https://api.provider.com/auth/test
|
curl -H "Authorization: Bearer $API_KEY" https://api.provider.com/auth/test
|
||||||
|
|
||||||
# Debug API calls
|
# Debug API calls
|
||||||
export PROVISIONING_DEBUG=true
|
export PROVISIONING_DEBUG=true
|
||||||
export PROVISIONING_LOG_LEVEL=debug
|
export PROVISIONING_LOG_LEVEL=debug
|
||||||
nu workspace/extensions/providers/my-provider/nulib/provider.nu test --test-type basic
|
nu workspace/extensions/providers/my-provider/nulib/provider.nu test --test-type basic
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="debug-mode"><a class="header" href="#debug-mode">Debug Mode</a></h3>
|
||||||
### Debug Mode
|
<p><strong>Enable Extension Debugging</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># Set debug environment
|
||||||
**Enable Extension Debugging**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Set debug environment
|
|
||||||
export PROVISIONING_DEBUG=true
|
export PROVISIONING_DEBUG=true
|
||||||
export PROVISIONING_LOG_LEVEL=debug
|
export PROVISIONING_LOG_LEVEL=debug
|
||||||
export PROVISIONING_WORKSPACE_USER=$USER
|
export PROVISIONING_WORKSPACE_USER=$USER
|
||||||
|
|
||||||
# Run extension with debug
|
# Run extension with debug
|
||||||
nu workspace/extensions/providers/my-provider/nulib/provider.nu create-server test-server small --dry-run
|
nu workspace/extensions/providers/my-provider/nulib/provider.nu create-server test-server small --dry-run
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="performance-optimization"><a class="header" href="#performance-optimization">Performance Optimization</a></h3>
|
||||||
### Performance Optimization
|
<p><strong>Extension Performance</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># Profile extension performance
|
||||||
**Extension Performance**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Profile extension performance
|
|
||||||
time nu workspace/extensions/providers/my-provider/nulib/provider.nu list-servers
|
time nu workspace/extensions/providers/my-provider/nulib/provider.nu list-servers
|
||||||
|
|
||||||
# Monitor resource usage
|
# Monitor resource usage
|
||||||
@ -1633,10 +1493,8 @@ nu workspace/tools/runtime-manager.nu monitor --duration 1m --interval 5s
|
|||||||
# Optimize API calls (use caching)
|
# Optimize API calls (use caching)
|
||||||
export PROVISIONING_CACHE_ENABLED=true
|
export PROVISIONING_CACHE_ENABLED=true
|
||||||
export PROVISIONING_CACHE_TTL=300 # 5 minutes
|
export PROVISIONING_CACHE_TTL=300 # 5 minutes
|
||||||
```plaintext
|
|
||||||
|
|
||||||
This extension development guide provides a comprehensive framework for creating high-quality, maintainable extensions that integrate seamlessly with provisioning's architecture and workflows.
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
|
<p>This extension development guide provides a comprehensive framework for creating high-quality, maintainable extensions that integrate seamlessly with provisioning’s architecture and workflows.</p>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
|
|||||||
@ -633,7 +633,7 @@ export def main [] {
|
|||||||
"provisioning/core"
|
"provisioning/core"
|
||||||
"provisioning/extensions"
|
"provisioning/extensions"
|
||||||
"provisioning/platform"
|
"provisioning/platform"
|
||||||
"provisioning/kcl"
|
"provisioning/schemas"
|
||||||
"workspace"
|
"workspace"
|
||||||
"workspace/templates"
|
"workspace/templates"
|
||||||
"distribution"
|
"distribution"
|
||||||
|
|||||||
@ -206,29 +206,21 @@
|
|||||||
│ - File-based │ │ - Monitoring │ │ - Workflows │
|
│ - File-based │ │ - Monitoring │ │ - Workflows │
|
||||||
│ - Simple logging│ │ - Validation │ │ - REST APIs │
|
│ - Simple logging│ │ - Validation │ │ - REST APIs │
|
||||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="existing-system-integration"><a class="header" href="#existing-system-integration">Existing System Integration</a></h2>
|
||||||
## Existing System Integration
|
<h3 id="command-line-interface-integration"><a class="header" href="#command-line-interface-integration">Command-Line Interface Integration</a></h3>
|
||||||
|
<p><strong>Seamless CLI Compatibility</strong>:</p>
|
||||||
### Command-Line Interface Integration
|
<pre><code class="language-bash"># All existing commands continue to work unchanged
|
||||||
|
./core/nulib/provisioning server create web-01 2xCPU-4 GB
|
||||||
**Seamless CLI Compatibility**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# All existing commands continue to work unchanged
|
|
||||||
./core/nulib/provisioning server create web-01 2xCPU-4GB
|
|
||||||
./core/nulib/provisioning taskserv install kubernetes
|
./core/nulib/provisioning taskserv install kubernetes
|
||||||
./core/nulib/provisioning cluster create buildkit
|
./core/nulib/provisioning cluster create buildkit
|
||||||
|
|
||||||
# New commands available alongside existing ones
|
# New commands available alongside existing ones
|
||||||
./src/core/nulib/provisioning server create web-01 2xCPU-4GB --orchestrated
|
./src/core/nulib/provisioning server create web-01 2xCPU-4 GB --orchestrated
|
||||||
nu workspace/tools/workspace.nu health --detailed
|
nu workspace/tools/workspace.nu health --detailed
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Path Resolution Integration</strong>:</p>
|
||||||
**Path Resolution Integration**:
|
<pre><code class="language-nushell"># Automatic path resolution between systems
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Automatic path resolution between systems
|
|
||||||
use workspace/lib/path-resolver.nu
|
use workspace/lib/path-resolver.nu
|
||||||
|
|
||||||
# Resolves to workspace path if available, falls back to core
|
# Resolves to workspace path if available, falls back to core
|
||||||
@ -236,14 +228,10 @@ let config_path = (path-resolver resolve_path "config" "user" --fallback-to-core
|
|||||||
|
|
||||||
# Seamless extension discovery
|
# Seamless extension discovery
|
||||||
let provider_path = (path-resolver resolve_extension "providers" "upcloud")
|
let provider_path = (path-resolver resolve_extension "providers" "upcloud")
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="configuration-system-bridge"><a class="header" href="#configuration-system-bridge">Configuration System Bridge</a></h3>
|
||||||
### Configuration System Bridge
|
<p><strong>Dual Configuration Support</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Configuration bridge supports both ENV and TOML
|
||||||
**Dual Configuration Support**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Configuration bridge supports both ENV and TOML
|
|
||||||
def get-config-value-bridge [key: string, default: string = ""] -> string {
|
def get-config-value-bridge [key: string, default: string = ""] -> string {
|
||||||
# Try new TOML configuration first
|
# Try new TOML configuration first
|
||||||
let toml_value = try {
|
let toml_value = try {
|
||||||
@ -273,14 +261,10 @@ def get-config-value-bridge [key: string, default: string = ""] -> string {
|
|||||||
help: $"Migrate from ($env_key) environment variable to ($key) in config file"
|
help: $"Migrate from ($env_key) environment variable to ($key) in config file"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="data-integration"><a class="header" href="#data-integration">Data Integration</a></h3>
|
||||||
### Data Integration
|
<p><strong>Shared Data Access</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Unified data access across old and new systems
|
||||||
**Shared Data Access**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Unified data access across old and new systems
|
|
||||||
def get-server-info [server_name: string] -> record {
|
def get-server-info [server_name: string] -> record {
|
||||||
# Try new orchestrator data store first
|
# Try new orchestrator data store first
|
||||||
let orchestrator_data = try {
|
let orchestrator_data = try {
|
||||||
@ -302,14 +286,10 @@ def get-server-info [server_name: string] -> record {
|
|||||||
|
|
||||||
error make {msg: $"Server not found: ($server_name)"}
|
error make {msg: $"Server not found: ($server_name)"}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="process-integration"><a class="header" href="#process-integration">Process Integration</a></h3>
|
||||||
### Process Integration
|
<p><strong>Hybrid Process Management</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Orchestrator-aware process management
|
||||||
**Hybrid Process Management**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Orchestrator-aware process management
|
|
||||||
def create-server-integrated [
|
def create-server-integrated [
|
||||||
name: string,
|
name: string,
|
||||||
plan: string,
|
plan: string,
|
||||||
@ -331,33 +311,24 @@ def check-orchestrator-available [] -> bool {
|
|||||||
false
|
false
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="api-compatibility-and-versioning"><a class="header" href="#api-compatibility-and-versioning">API Compatibility and Versioning</a></h2>
|
||||||
## API Compatibility and Versioning
|
<h3 id="rest-api-versioning"><a class="header" href="#rest-api-versioning">REST API Versioning</a></h3>
|
||||||
|
<p><strong>API Version Strategy</strong>:</p>
|
||||||
### REST API Versioning
|
<ul>
|
||||||
|
<li><strong>v1</strong>: Legacy compatibility API (existing functionality)</li>
|
||||||
**API Version Strategy**:
|
<li><strong>v2</strong>: Enhanced API with orchestrator features</li>
|
||||||
|
<li><strong>v3</strong>: Full workflow and batch operation support</li>
|
||||||
- **v1**: Legacy compatibility API (existing functionality)
|
</ul>
|
||||||
- **v2**: Enhanced API with orchestrator features
|
<p><strong>Version Header Support</strong>:</p>
|
||||||
- **v3**: Full workflow and batch operation support
|
<pre><code class="language-bash"># API calls with version specification
|
||||||
|
|
||||||
**Version Header Support**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# API calls with version specification
|
|
||||||
curl -H "API-Version: v1" http://localhost:9090/servers
|
curl -H "API-Version: v1" http://localhost:9090/servers
|
||||||
curl -H "API-Version: v2" http://localhost:9090/workflows/servers/create
|
curl -H "API-Version: v2" http://localhost:9090/workflows/servers/create
|
||||||
curl -H "API-Version: v3" http://localhost:9090/workflows/batch/submit
|
curl -H "API-Version: v3" http://localhost:9090/workflows/batch/submit
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="api-compatibility-layer"><a class="header" href="#api-compatibility-layer">API Compatibility Layer</a></h3>
|
||||||
### API Compatibility Layer
|
<p><strong>Backward Compatible Endpoints</strong>:</p>
|
||||||
|
<pre><code class="language-rust">// Rust API compatibility layer
|
||||||
**Backward Compatible Endpoints**:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Rust API compatibility layer
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
#[derive(Debug, Serialize, Deserialize)]
|
||||||
struct ApiRequest {
|
struct ApiRequest {
|
||||||
version: Option<String>,
|
version: Option<String>,
|
||||||
@ -392,54 +363,40 @@ async fn handle_v1_request(payload: serde_json::Value) -> Result<ApiRespon
|
|||||||
|
|
||||||
// Transform response to v1 format
|
// Transform response to v1 format
|
||||||
Ok(transform_to_v1_response(result))
|
Ok(transform_to_v1_response(result))
|
||||||
}
|
}</code></pre>
|
||||||
```plaintext
|
<h3 id="schema-evolution"><a class="header" href="#schema-evolution">Schema Evolution</a></h3>
|
||||||
|
<p><strong>Backward Compatible Schema Changes</strong>:</p>
|
||||||
### Schema Evolution
|
<pre><code class="language-nickel"># API schema with version support
|
||||||
|
let ServerCreateRequest = {
|
||||||
**Backward Compatible Schema Changes**:
|
|
||||||
|
|
||||||
```kcl
|
|
||||||
# API schema with version support
|
|
||||||
schema ServerCreateRequest {
|
|
||||||
# V1 fields (always supported)
|
# V1 fields (always supported)
|
||||||
name: str
|
name | string,
|
||||||
plan: str
|
plan | string,
|
||||||
zone?: str = "auto"
|
zone | string | default = "auto",
|
||||||
|
|
||||||
# V2 additions (optional for backward compatibility)
|
# V2 additions (optional for backward compatibility)
|
||||||
orchestrated?: bool = false
|
orchestrated | bool | default = false,
|
||||||
workflow_options?: WorkflowOptions
|
workflow_options | { } | optional,
|
||||||
|
|
||||||
# V3 additions
|
# V3 additions
|
||||||
batch_options?: BatchOptions
|
batch_options | { } | optional,
|
||||||
dependencies?: [str] = []
|
dependencies | array | default = [],
|
||||||
|
|
||||||
# Version constraints
|
# Version constraints
|
||||||
api_version?: str = "v1"
|
api_version | string | default = "v1",
|
||||||
|
} in
|
||||||
check:
|
ServerCreateRequest
|
||||||
len(name) > 0, "Name cannot be empty"
|
|
||||||
plan in ["1xCPU-2GB", "2xCPU-4GB", "4xCPU-8GB", "8xCPU-16GB"], "Invalid plan"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Conditional validation based on API version
|
# Conditional validation based on API version
|
||||||
schema WorkflowOptions:
|
let WorkflowOptions = {
|
||||||
wait_for_completion?: bool = true
|
wait_for_completion | bool | default = true,
|
||||||
timeout_seconds?: int = 300
|
timeout_seconds | number | default = 300,
|
||||||
retry_count?: int = 3
|
retry_count | number | default = 3,
|
||||||
|
} in
|
||||||
check:
|
WorkflowOptions
|
||||||
timeout_seconds > 0, "Timeout must be positive"
|
</code></pre>
|
||||||
retry_count >= 0, "Retry count must be non-negative"
|
<h3 id="client-sdk-compatibility"><a class="header" href="#client-sdk-compatibility">Client SDK Compatibility</a></h3>
|
||||||
```plaintext
|
<p><strong>Multi-Version Client Support</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Nushell client with version support
|
||||||
### Client SDK Compatibility
|
|
||||||
|
|
||||||
**Multi-Version Client Support**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Nushell client with version support
|
|
||||||
def "client create-server" [
|
def "client create-server" [
|
||||||
name: string,
|
name: string,
|
||||||
plan: string,
|
plan: string,
|
||||||
@ -472,16 +429,11 @@ def "client create-server" [
|
|||||||
"API-Version": $api_version
|
"API-Version": $api_version
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="database-migration-strategies"><a class="header" href="#database-migration-strategies">Database Migration Strategies</a></h2>
|
||||||
## Database Migration Strategies
|
<h3 id="database-architecture-evolution"><a class="header" href="#database-architecture-evolution">Database Architecture Evolution</a></h3>
|
||||||
|
<p><strong>Migration Strategy</strong>:</p>
|
||||||
### Database Architecture Evolution
|
<pre><code class="language-plaintext">Database Evolution Path
|
||||||
|
|
||||||
**Migration Strategy**:
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
Database Evolution Path
|
|
||||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||||
│ File-based │ → │ SQLite │ → │ SurrealDB │
|
│ File-based │ → │ SQLite │ → │ SurrealDB │
|
||||||
│ Storage │ │ Migration │ │ Full Schema │
|
│ Storage │ │ Migration │ │ Full Schema │
|
||||||
@ -490,14 +442,10 @@ Database Evolution Path
|
|||||||
│ - Text logs │ │ - Transactions │ │ - Real-time │
|
│ - Text logs │ │ - Transactions │ │ - Real-time │
|
||||||
│ - Simple state │ │ - Backup/restore│ │ - Clustering │
|
│ - Simple state │ │ - Backup/restore│ │ - Clustering │
|
||||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="migration-scripts"><a class="header" href="#migration-scripts">Migration Scripts</a></h3>
|
||||||
### Migration Scripts
|
<p><strong>Automated Database Migration</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Database migration orchestration
|
||||||
**Automated Database Migration**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Database migration orchestration
|
|
||||||
def migrate-database [
|
def migrate-database [
|
||||||
--from: string = "filesystem",
|
--from: string = "filesystem",
|
||||||
--to: string = "surrealdb",
|
--to: string = "surrealdb",
|
||||||
@ -533,12 +481,9 @@ def migrate-database [
|
|||||||
print $"Migration from ($from) to ($to) completed successfully"
|
print $"Migration from ($from) to ($to) completed successfully"
|
||||||
{from: $from, to: $to, status: "completed", migrated_at: (date now)}
|
{from: $from, to: $to, status: "completed", migrated_at: (date now)}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>File System to SurrealDB Migration</strong>:</p>
|
||||||
**File System to SurrealDB Migration**:
|
<pre><code class="language-nushell">def migrate_filesystem_to_surrealdb [] -> record {
|
||||||
|
|
||||||
```nushell
|
|
||||||
def migrate_filesystem_to_surrealdb [] -> record {
|
|
||||||
# Initialize SurrealDB connection
|
# Initialize SurrealDB connection
|
||||||
let db = (connect-surrealdb)
|
let db = (connect-surrealdb)
|
||||||
|
|
||||||
@ -585,14 +530,10 @@ def migrate_filesystem_to_surrealdb [] -> record {
|
|||||||
status: "completed"
|
status: "completed"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="data-integrity-verification"><a class="header" href="#data-integrity-verification">Data Integrity Verification</a></h3>
|
||||||
### Data Integrity Verification
|
<p><strong>Migration Verification</strong>:</p>
|
||||||
|
<pre><code class="language-nushell">def verify-migration [from: string, to: string] -> record {
|
||||||
**Migration Verification**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
def verify-migration [from: string, to: string] -> record {
|
|
||||||
print "Verifying data integrity..."
|
print "Verifying data integrity..."
|
||||||
|
|
||||||
let source_data = (read-source-data $from)
|
let source_data = (read-source-data $from)
|
||||||
@ -629,16 +570,11 @@ def verify-migration [from: string, to: string] -> record {
|
|||||||
verified_at: (date now)
|
verified_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="deployment-considerations"><a class="header" href="#deployment-considerations">Deployment Considerations</a></h2>
|
||||||
## Deployment Considerations
|
<h3 id="deployment-architecture"><a class="header" href="#deployment-architecture">Deployment Architecture</a></h3>
|
||||||
|
<p><strong>Hybrid Deployment Model</strong>:</p>
|
||||||
### Deployment Architecture
|
<pre><code class="language-plaintext">Deployment Architecture
|
||||||
|
|
||||||
**Hybrid Deployment Model**:
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
Deployment Architecture
|
|
||||||
┌─────────────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
│ Load Balancer / Reverse Proxy │
|
│ Load Balancer / Reverse Proxy │
|
||||||
└─────────────────────┬───────────────────────────────────────────┘
|
└─────────────────────┬───────────────────────────────────────────┘
|
||||||
@ -653,14 +589,10 @@ Deployment Architecture
|
|||||||
│- Files │ │- Compat │ │- DB │
|
│- Files │ │- Compat │ │- DB │
|
||||||
│- Logs │ │- Monitor │ │- Queue │
|
│- Logs │ │- Monitor │ │- Queue │
|
||||||
└────────┘ └────────────┘ └────────┘
|
└────────┘ └────────────┘ └────────┘
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="deployment-strategies"><a class="header" href="#deployment-strategies">Deployment Strategies</a></h3>
|
||||||
### Deployment Strategies
|
<p><strong>Blue-Green Deployment</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># Blue-Green deployment with integration bridge
|
||||||
**Blue-Green Deployment**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Blue-Green deployment with integration bridge
|
|
||||||
# Phase 1: Deploy new system alongside existing (Green environment)
|
# Phase 1: Deploy new system alongside existing (Green environment)
|
||||||
cd src/tools
|
cd src/tools
|
||||||
make all
|
make all
|
||||||
@ -686,12 +618,9 @@ nginx-traffic-split --new-backend 90%
|
|||||||
# Phase 4: Complete cutover
|
# Phase 4: Complete cutover
|
||||||
nginx-traffic-split --new-backend 100%
|
nginx-traffic-split --new-backend 100%
|
||||||
/opt/provisioning-v1/bin/orchestrator stop
|
/opt/provisioning-v1/bin/orchestrator stop
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Rolling Update</strong>:</p>
|
||||||
**Rolling Update**:
|
<pre><code class="language-nushell">def rolling-deployment [
|
||||||
|
|
||||||
```nushell
|
|
||||||
def rolling-deployment [
|
|
||||||
--target-version: string,
|
--target-version: string,
|
||||||
--batch-size: int = 3,
|
--batch-size: int = 3,
|
||||||
--health-check-interval: duration = 30sec
|
--health-check-interval: duration = 30sec
|
||||||
@ -741,14 +670,10 @@ def rolling-deployment [
|
|||||||
completed_at: (date now)
|
completed_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="configuration-deployment"><a class="header" href="#configuration-deployment">Configuration Deployment</a></h3>
|
||||||
### Configuration Deployment
|
<p><strong>Environment-Specific Deployment</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># Development deployment
|
||||||
**Environment-Specific Deployment**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Development deployment
|
|
||||||
PROVISIONING_ENV=dev ./deploy.sh \
|
PROVISIONING_ENV=dev ./deploy.sh \
|
||||||
--config-source config.dev.toml \
|
--config-source config.dev.toml \
|
||||||
--enable-debug \
|
--enable-debug \
|
||||||
@ -767,14 +692,10 @@ PROVISIONING_ENV=prod ./deploy.sh \
|
|||||||
--enable-all-monitoring \
|
--enable-all-monitoring \
|
||||||
--backup-before-deploy \
|
--backup-before-deploy \
|
||||||
--health-check-timeout 5m
|
--health-check-timeout 5m
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="container-integration"><a class="header" href="#container-integration">Container Integration</a></h3>
|
||||||
### Container Integration
|
<p><strong>Docker Deployment with Bridge</strong>:</p>
|
||||||
|
<pre><code class="language-dockerfile"># Multi-stage Docker build supporting both systems
|
||||||
**Docker Deployment with Bridge**:
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
# Multi-stage Docker build supporting both systems
|
|
||||||
FROM rust:1.70 as builder
|
FROM rust:1.70 as builder
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY . .
|
COPY . .
|
||||||
@ -797,12 +718,9 @@ ENV PROVISIONING_NEW_PATH=/app/bin
|
|||||||
|
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
CMD ["/app/bin/bridge-start.sh"]
|
CMD ["/app/bin/bridge-start.sh"]
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Kubernetes Integration</strong>:</p>
|
||||||
**Kubernetes Integration**:
|
<pre><code class="language-yaml"># Kubernetes deployment with bridge sidecar
|
||||||
|
|
||||||
```yaml
|
|
||||||
# Kubernetes deployment with bridge sidecar
|
|
||||||
apiVersion: apps/v1
|
apiVersion: apps/v1
|
||||||
kind: Deployment
|
kind: Deployment
|
||||||
metadata:
|
metadata:
|
||||||
@ -841,16 +759,11 @@ spec:
|
|||||||
- name: legacy-data
|
- name: legacy-data
|
||||||
persistentVolumeClaim:
|
persistentVolumeClaim:
|
||||||
claimName: provisioning-data
|
claimName: provisioning-data
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="monitoring-and-observability"><a class="header" href="#monitoring-and-observability">Monitoring and Observability</a></h2>
|
||||||
## Monitoring and Observability
|
<h3 id="integrated-monitoring-architecture"><a class="header" href="#integrated-monitoring-architecture">Integrated Monitoring Architecture</a></h3>
|
||||||
|
<p><strong>Monitoring Stack Integration</strong>:</p>
|
||||||
### Integrated Monitoring Architecture
|
<pre><code class="language-plaintext">Observability Architecture
|
||||||
|
|
||||||
**Monitoring Stack Integration**:
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
Observability Architecture
|
|
||||||
┌─────────────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
│ Monitoring Dashboard │
|
│ Monitoring Dashboard │
|
||||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||||
@ -879,14 +792,10 @@ Observability Architecture
|
|||||||
│ - Compatibility │
|
│ - Compatibility │
|
||||||
│ - Migration │
|
│ - Migration │
|
||||||
└───────────────────┘
|
└───────────────────┘
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="metrics-integration"><a class="header" href="#metrics-integration">Metrics Integration</a></h3>
|
||||||
### Metrics Integration
|
<p><strong>Unified Metrics Collection</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Metrics bridge for legacy and new systems
|
||||||
**Unified Metrics Collection**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Metrics bridge for legacy and new systems
|
|
||||||
def collect-system-metrics [] -> record {
|
def collect-system-metrics [] -> record {
|
||||||
let legacy_metrics = collect-legacy-metrics
|
let legacy_metrics = collect-legacy-metrics
|
||||||
let new_metrics = collect-new-metrics
|
let new_metrics = collect-new-metrics
|
||||||
@ -935,14 +844,10 @@ def collect-new-metrics [] -> record {
|
|||||||
database_stats: (get-database-metrics)
|
database_stats: (get-database-metrics)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="logging-integration"><a class="header" href="#logging-integration">Logging Integration</a></h3>
|
||||||
### Logging Integration
|
<p><strong>Unified Logging Strategy</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Structured logging bridge
|
||||||
**Unified Logging Strategy**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Structured logging bridge
|
|
||||||
def log-integrated [
|
def log-integrated [
|
||||||
level: string,
|
level: string,
|
||||||
message: string,
|
message: string,
|
||||||
@ -970,14 +875,10 @@ def log-integrated [
|
|||||||
# Send to monitoring system
|
# Send to monitoring system
|
||||||
send-to-monitoring $log_entry
|
send-to-monitoring $log_entry
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="health-check-integration"><a class="header" href="#health-check-integration">Health Check Integration</a></h3>
|
||||||
### Health Check Integration
|
<p><strong>Comprehensive Health Monitoring</strong>:</p>
|
||||||
|
<pre><code class="language-nushell">def health-check-integrated [] -> record {
|
||||||
**Comprehensive Health Monitoring**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
def health-check-integrated [] -> record {
|
|
||||||
let health_checks = [
|
let health_checks = [
|
||||||
{name: "legacy-system", check: (check-legacy-health)},
|
{name: "legacy-system", check: (check-legacy-health)},
|
||||||
{name: "orchestrator", check: (check-orchestrator-health)},
|
{name: "orchestrator", check: (check-orchestrator-health)},
|
||||||
@ -1007,16 +908,11 @@ def health-check-integrated [] -> record {
|
|||||||
checked_at: (date now)
|
checked_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="legacy-system-bridge"><a class="header" href="#legacy-system-bridge">Legacy System Bridge</a></h2>
|
||||||
## Legacy System Bridge
|
<h3 id="bridge-architecture"><a class="header" href="#bridge-architecture">Bridge Architecture</a></h3>
|
||||||
|
<p><strong>Bridge Component Design</strong>:</p>
|
||||||
### Bridge Architecture
|
<pre><code class="language-nushell"># Legacy system bridge module
|
||||||
|
|
||||||
**Bridge Component Design**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Legacy system bridge module
|
|
||||||
export module bridge {
|
export module bridge {
|
||||||
# Bridge state management
|
# Bridge state management
|
||||||
export def init-bridge [] -> record {
|
export def init-bridge [] -> record {
|
||||||
@ -1070,14 +966,10 @@ export module bridge {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="bridge-operation-modes"><a class="header" href="#bridge-operation-modes">Bridge Operation Modes</a></h3>
|
||||||
### Bridge Operation Modes
|
<p><strong>Compatibility Mode</strong>:</p>
|
||||||
|
<pre><code class="language-nushell"># Full compatibility with legacy system
|
||||||
**Compatibility Mode**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Full compatibility with legacy system
|
|
||||||
def run-compatibility-mode [] {
|
def run-compatibility-mode [] {
|
||||||
print "Starting bridge in compatibility mode..."
|
print "Starting bridge in compatibility mode..."
|
||||||
|
|
||||||
@ -1098,12 +990,9 @@ def run-compatibility-mode [] {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Migration Mode</strong>:</p>
|
||||||
**Migration Mode**:
|
<pre><code class="language-nushell"># Gradual migration with traffic splitting
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Gradual migration with traffic splitting
|
|
||||||
def run-migration-mode [
|
def run-migration-mode [
|
||||||
--new-system-percentage: int = 50
|
--new-system-percentage: int = 50
|
||||||
] {
|
] {
|
||||||
@ -1126,39 +1015,33 @@ def run-migration-mode [
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="migration-pathways"><a class="header" href="#migration-pathways">Migration Pathways</a></h2>
|
||||||
## Migration Pathways
|
<h3 id="migration-phases"><a class="header" href="#migration-phases">Migration Phases</a></h3>
|
||||||
|
<p><strong>Phase 1: Parallel Deployment</strong></p>
|
||||||
### Migration Phases
|
<ul>
|
||||||
|
<li>Deploy new system alongside existing</li>
|
||||||
**Phase 1: Parallel Deployment**
|
<li>Enable bridge for compatibility</li>
|
||||||
|
<li>Begin data synchronization</li>
|
||||||
- Deploy new system alongside existing
|
<li>Monitor integration health</li>
|
||||||
- Enable bridge for compatibility
|
</ul>
|
||||||
- Begin data synchronization
|
<p><strong>Phase 2: Gradual Migration</strong></p>
|
||||||
- Monitor integration health
|
<ul>
|
||||||
|
<li>Route increasing traffic to new system</li>
|
||||||
**Phase 2: Gradual Migration**
|
<li>Migrate data in background</li>
|
||||||
|
<li>Validate consistency</li>
|
||||||
- Route increasing traffic to new system
|
<li>Address integration issues</li>
|
||||||
- Migrate data in background
|
</ul>
|
||||||
- Validate consistency
|
<p><strong>Phase 3: Full Migration</strong></p>
|
||||||
- Address integration issues
|
<ul>
|
||||||
|
<li>Complete traffic cutover</li>
|
||||||
**Phase 3: Full Migration**
|
<li>Decommission legacy system</li>
|
||||||
|
<li>Clean up bridge components</li>
|
||||||
- Complete traffic cutover
|
<li>Finalize data migration</li>
|
||||||
- Decommission legacy system
|
</ul>
|
||||||
- Clean up bridge components
|
<h3 id="migration-automation"><a class="header" href="#migration-automation">Migration Automation</a></h3>
|
||||||
- Finalize data migration
|
<p><strong>Automated Migration Orchestration</strong>:</p>
|
||||||
|
<pre><code class="language-nushell">def execute-migration-plan [
|
||||||
### Migration Automation
|
|
||||||
|
|
||||||
**Automated Migration Orchestration**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
def execute-migration-plan [
|
|
||||||
migration_plan: string,
|
migration_plan: string,
|
||||||
--dry-run: bool = false,
|
--dry-run: bool = false,
|
||||||
--skip-backup: bool = false
|
--skip-backup: bool = false
|
||||||
@ -1208,12 +1091,9 @@ def execute-migration-plan [
|
|||||||
results: $migration_results
|
results: $migration_results
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Migration Validation</strong>:</p>
|
||||||
**Migration Validation**:
|
<pre><code class="language-nushell">def validate-migration-readiness [] -> record {
|
||||||
|
|
||||||
```nushell
|
|
||||||
def validate-migration-readiness [] -> record {
|
|
||||||
let checks = [
|
let checks = [
|
||||||
{name: "backup-available", check: (check-backup-exists)},
|
{name: "backup-available", check: (check-backup-exists)},
|
||||||
{name: "new-system-healthy", check: (check-new-system-health)},
|
{name: "new-system-healthy", check: (check-new-system-health)},
|
||||||
@ -1240,18 +1120,12 @@ def validate-migration-readiness [] -> record {
|
|||||||
validated_at: (date now)
|
validated_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="troubleshooting-integration-issues"><a class="header" href="#troubleshooting-integration-issues">Troubleshooting Integration Issues</a></h2>
|
||||||
## Troubleshooting Integration Issues
|
<h3 id="common-integration-problems"><a class="header" href="#common-integration-problems">Common Integration Problems</a></h3>
|
||||||
|
<h4 id="api-compatibility-issues"><a class="header" href="#api-compatibility-issues">API Compatibility Issues</a></h4>
|
||||||
### Common Integration Problems
|
<p><strong>Problem</strong>: Version mismatch between client and server</p>
|
||||||
|
<pre><code class="language-bash"># Diagnosis
|
||||||
#### API Compatibility Issues
|
|
||||||
|
|
||||||
**Problem**: Version mismatch between client and server
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Diagnosis
|
|
||||||
curl -H "API-Version: v1" http://localhost:9090/health
|
curl -H "API-Version: v1" http://localhost:9090/health
|
||||||
curl -H "API-Version: v2" http://localhost:9090/health
|
curl -H "API-Version: v2" http://localhost:9090/health
|
||||||
|
|
||||||
@ -1260,14 +1134,10 @@ curl http://localhost:9090/api/versions
|
|||||||
|
|
||||||
# Update client API version
|
# Update client API version
|
||||||
export PROVISIONING_API_VERSION=v2
|
export PROVISIONING_API_VERSION=v2
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="configuration-bridge-issues"><a class="header" href="#configuration-bridge-issues">Configuration Bridge Issues</a></h4>
|
||||||
#### Configuration Bridge Issues
|
<p><strong>Problem</strong>: Configuration not found in either system</p>
|
||||||
|
<pre><code class="language-nushell"># Diagnosis
|
||||||
**Problem**: Configuration not found in either system
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Diagnosis
|
|
||||||
def diagnose-config-issue [key: string] -> record {
|
def diagnose-config-issue [key: string] -> record {
|
||||||
let toml_result = try {
|
let toml_result = try {
|
||||||
get-config-value $key
|
get-config-value $key
|
||||||
@ -1296,14 +1166,10 @@ def migrate-single-config [key: string] {
|
|||||||
print $"Migrated ($key) from environment variable"
|
print $"Migrated ($key) from environment variable"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="database-integration-issues"><a class="header" href="#database-integration-issues">Database Integration Issues</a></h4>
|
||||||
#### Database Integration Issues
|
<p><strong>Problem</strong>: Data inconsistency between systems</p>
|
||||||
|
<pre><code class="language-nushell"># Diagnosis and repair
|
||||||
**Problem**: Data inconsistency between systems
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
# Diagnosis and repair
|
|
||||||
def repair-data-consistency [] -> record {
|
def repair-data-consistency [] -> record {
|
||||||
let legacy_data = (read-legacy-data)
|
let legacy_data = (read-legacy-data)
|
||||||
let new_data = (read-new-data)
|
let new_data = (read-new-data)
|
||||||
@ -1331,27 +1197,20 @@ def repair-data-consistency [] -> record {
|
|||||||
repaired_at: (date now)
|
repaired_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="debug-tools"><a class="header" href="#debug-tools">Debug Tools</a></h3>
|
||||||
### Debug Tools
|
<p><strong>Integration Debug Mode</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># Enable comprehensive debugging
|
||||||
**Integration Debug Mode**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Enable comprehensive debugging
|
|
||||||
export PROVISIONING_DEBUG=true
|
export PROVISIONING_DEBUG=true
|
||||||
export PROVISIONING_LOG_LEVEL=debug
|
export PROVISIONING_LOG_LEVEL=debug
|
||||||
export PROVISIONING_BRIDGE_DEBUG=true
|
export PROVISIONING_BRIDGE_DEBUG=true
|
||||||
export PROVISIONING_INTEGRATION_TRACE=true
|
export PROVISIONING_INTEGRATION_TRACE=true
|
||||||
|
|
||||||
# Run with integration debugging
|
# Run with integration debugging
|
||||||
provisioning server create test-server 2xCPU-4GB --debug-integration
|
provisioning server create test-server 2xCPU-4 GB --debug-integration
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Health Check Debugging</strong>:</p>
|
||||||
**Health Check Debugging**:
|
<pre><code class="language-nushell">def debug-integration-health [] -> record {
|
||||||
|
|
||||||
```nushell
|
|
||||||
def debug-integration-health [] -> record {
|
|
||||||
print "=== Integration Health Debug ==="
|
print "=== Integration Health Debug ==="
|
||||||
|
|
||||||
# Check all integration points
|
# Check all integration points
|
||||||
@ -1384,10 +1243,8 @@ def debug-integration-health [] -> record {
|
|||||||
debug_timestamp: (date now)
|
debug_timestamp: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
|
||||||
|
|
||||||
This integration guide provides a comprehensive framework for seamlessly integrating new development components with existing production systems while maintaining reliability, compatibility, and clear migration pathways.
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
|
<p>This integration guide provides a comprehensive framework for seamlessly integrating new development components with existing production systems while maintaining reliability, compatibility, and clear migration pathways.</p>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
|
|||||||
@ -202,68 +202,53 @@
|
|||||||
├── docs/ # Documentation (new)
|
├── docs/ # Documentation (new)
|
||||||
├── extensions/ # Extension framework
|
├── extensions/ # Extension framework
|
||||||
├── generators/ # Code generation tools
|
├── generators/ # Code generation tools
|
||||||
├── kcl/ # KCL configuration language files
|
├── schemas/ # Nickel configuration schemas (migrated from kcl/)
|
||||||
├── orchestrator/ # Hybrid Rust/Nushell orchestrator
|
├── orchestrator/ # Hybrid Rust/Nushell orchestrator
|
||||||
├── platform/ # Platform-specific code
|
├── platform/ # Platform-specific code
|
||||||
├── provisioning/ # Main provisioning
|
├── provisioning/ # Main provisioning
|
||||||
├── templates/ # Template files
|
├── templates/ # Template files
|
||||||
├── tools/ # Build and development tools
|
├── tools/ # Build and development tools
|
||||||
└── utils/ # Utility scripts
|
└── utils/ # Utility scripts
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="legacy-structure-preserved"><a class="header" href="#legacy-structure-preserved">Legacy Structure (Preserved)</a></h3>
|
||||||
### Legacy Structure (Preserved)
|
<pre><code class="language-plaintext">repo-cnz/
|
||||||
|
|
||||||
```plaintext
|
|
||||||
repo-cnz/
|
|
||||||
├── cluster/ # Cluster configurations (preserved)
|
├── cluster/ # Cluster configurations (preserved)
|
||||||
├── core/ # Core system (preserved)
|
├── core/ # Core system (preserved)
|
||||||
├── generate/ # Generation scripts (preserved)
|
├── generate/ # Generation scripts (preserved)
|
||||||
├── kcl/ # KCL files (preserved)
|
├── schemas/ # Nickel schemas (migrated from kcl/)
|
||||||
├── klab/ # Development lab (preserved)
|
├── klab/ # Development lab (preserved)
|
||||||
├── nushell-plugins/ # Plugin development (preserved)
|
├── nushell-plugins/ # Plugin development (preserved)
|
||||||
├── providers/ # Cloud providers (preserved)
|
├── providers/ # Cloud providers (preserved)
|
||||||
├── taskservs/ # Task services (preserved)
|
├── taskservs/ # Task services (preserved)
|
||||||
└── templates/ # Template files (preserved)
|
└── templates/ # Template files (preserved)
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="development-workspace-workspace"><a class="header" href="#development-workspace-workspace">Development Workspace (<code>/workspace/</code>)</a></h3>
|
||||||
### Development Workspace (`/workspace/`)
|
<pre><code class="language-plaintext">workspace/
|
||||||
|
|
||||||
```plaintext
|
|
||||||
workspace/
|
|
||||||
├── config/ # Development configuration
|
├── config/ # Development configuration
|
||||||
├── extensions/ # Extension development
|
├── extensions/ # Extension development
|
||||||
├── infra/ # Development infrastructure
|
├── infra/ # Development infrastructure
|
||||||
├── lib/ # Workspace libraries
|
├── lib/ # Workspace libraries
|
||||||
├── runtime/ # Runtime data
|
├── runtime/ # Runtime data
|
||||||
└── tools/ # Workspace management tools
|
└── tools/ # Workspace management tools
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="core-directories"><a class="header" href="#core-directories">Core Directories</a></h2>
|
||||||
## Core Directories
|
<h3 id="srccore---core-development-libraries"><a class="header" href="#srccore---core-development-libraries"><code>/src/core/</code> - Core Development Libraries</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: Development-focused core libraries and entry points</p>
|
||||||
### `/src/core/` - Core Development Libraries
|
<p><strong>Key Files</strong>:</p>
|
||||||
|
<ul>
|
||||||
**Purpose**: Development-focused core libraries and entry points
|
<li><code>nulib/provisioning</code> - Main CLI entry point (symlinks to legacy location)</li>
|
||||||
|
<li><code>nulib/lib_provisioning/</code> - Core provisioning libraries</li>
|
||||||
**Key Files**:
|
<li><code>nulib/workflows/</code> - Workflow management (orchestrator integration)</li>
|
||||||
|
</ul>
|
||||||
- `nulib/provisioning` - Main CLI entry point (symlinks to legacy location)
|
<p><strong>Relationship to Legacy</strong>: Preserves original <code>core/</code> functionality while adding development enhancements</p>
|
||||||
- `nulib/lib_provisioning/` - Core provisioning libraries
|
<h3 id="srctools---build-and-development-tools"><a class="header" href="#srctools---build-and-development-tools"><code>/src/tools/</code> - Build and Development Tools</a></h3>
|
||||||
- `nulib/workflows/` - Workflow management (orchestrator integration)
|
<p><strong>Purpose</strong>: Complete build system for the provisioning project</p>
|
||||||
|
<p><strong>Key Components</strong>:</p>
|
||||||
**Relationship to Legacy**: Preserves original `core/` functionality while adding development enhancements
|
<pre><code class="language-plaintext">tools/
|
||||||
|
|
||||||
### `/src/tools/` - Build and Development Tools
|
|
||||||
|
|
||||||
**Purpose**: Complete build system for the provisioning project
|
|
||||||
|
|
||||||
**Key Components**:
|
|
||||||
|
|
||||||
```plaintext
|
|
||||||
tools/
|
|
||||||
├── build/ # Build tools
|
├── build/ # Build tools
|
||||||
│ ├── compile-platform.nu # Platform-specific compilation
|
│ ├── compile-platform.nu # Platform-specific compilation
|
||||||
│ ├── bundle-core.nu # Core library bundling
|
│ ├── bundle-core.nu # Core library bundling
|
||||||
│ ├── validate-kcl.nu # KCL validation
|
│ ├── validate-nickel.nu # Nickel schema validation
|
||||||
│ ├── clean-build.nu # Build cleanup
|
│ ├── clean-build.nu # Build cleanup
|
||||||
│ └── test-distribution.nu # Distribution testing
|
│ └── test-distribution.nu # Distribution testing
|
||||||
├── distribution/ # Distribution tools
|
├── distribution/ # Distribution tools
|
||||||
@ -284,122 +269,94 @@ tools/
|
|||||||
│ ├── notify-users.nu # Release notifications
|
│ ├── notify-users.nu # Release notifications
|
||||||
│ └── update-registry.nu # Package registry updates
|
│ └── update-registry.nu # Package registry updates
|
||||||
└── Makefile # Main build system (40+ targets)
|
└── Makefile # Main build system (40+ targets)
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="srcorchestrator---hybrid-orchestrator"><a class="header" href="#srcorchestrator---hybrid-orchestrator"><code>/src/orchestrator/</code> - Hybrid Orchestrator</a></h3>
|
||||||
### `/src/orchestrator/` - Hybrid Orchestrator
|
<p><strong>Purpose</strong>: Rust/Nushell hybrid orchestrator for solving deep call stack limitations</p>
|
||||||
|
<p><strong>Key Components</strong>:</p>
|
||||||
**Purpose**: Rust/Nushell hybrid orchestrator for solving deep call stack limitations
|
<ul>
|
||||||
|
<li><code>src/</code> - Rust orchestrator implementation</li>
|
||||||
**Key Components**:
|
<li><code>scripts/</code> - Orchestrator management scripts</li>
|
||||||
|
<li><code>data/</code> - File-based task queue and persistence</li>
|
||||||
- `src/` - Rust orchestrator implementation
|
</ul>
|
||||||
- `scripts/` - Orchestrator management scripts
|
<p><strong>Integration</strong>: Provides REST API and workflow management while preserving all Nushell business logic</p>
|
||||||
- `data/` - File-based task queue and persistence
|
<h3 id="srcprovisioning---enhanced-provisioning"><a class="header" href="#srcprovisioning---enhanced-provisioning"><code>/src/provisioning/</code> - Enhanced Provisioning</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: Enhanced version of the main provisioning with additional features</p>
|
||||||
**Integration**: Provides REST API and workflow management while preserving all Nushell business logic
|
<p><strong>Key Features</strong>:</p>
|
||||||
|
<ul>
|
||||||
### `/src/provisioning/` - Enhanced Provisioning
|
<li>Batch workflow system (v3.1.0)</li>
|
||||||
|
<li>Provider-agnostic design</li>
|
||||||
**Purpose**: Enhanced version of the main provisioning with additional features
|
<li>Configuration-driven architecture (v2.0.0)</li>
|
||||||
|
</ul>
|
||||||
**Key Features**:
|
<h3 id="workspace---development-workspace"><a class="header" href="#workspace---development-workspace"><code>/workspace/</code> - Development Workspace</a></h3>
|
||||||
|
<p><strong>Purpose</strong>: Complete development environment with tools and runtime management</p>
|
||||||
- Batch workflow system (v3.1.0)
|
<p><strong>Key Components</strong>:</p>
|
||||||
- Provider-agnostic design
|
<ul>
|
||||||
- Configuration-driven architecture (v2.0.0)
|
<li><code>tools/workspace.nu</code> - Unified workspace management interface</li>
|
||||||
|
<li><code>lib/path-resolver.nu</code> - Smart path resolution system</li>
|
||||||
### `/workspace/` - Development Workspace
|
<li><code>config/</code> - Environment-specific development configurations</li>
|
||||||
|
<li><code>extensions/</code> - Extension development templates and examples</li>
|
||||||
**Purpose**: Complete development environment with tools and runtime management
|
<li><code>infra/</code> - Development infrastructure examples</li>
|
||||||
|
<li><code>runtime/</code> - Isolated runtime data per user</li>
|
||||||
**Key Components**:
|
</ul>
|
||||||
|
<h2 id="development-workspace"><a class="header" href="#development-workspace">Development Workspace</a></h2>
|
||||||
- `tools/workspace.nu` - Unified workspace management interface
|
<h3 id="workspace-management"><a class="header" href="#workspace-management">Workspace Management</a></h3>
|
||||||
- `lib/path-resolver.nu` - Smart path resolution system
|
<p>The workspace provides a sophisticated development environment:</p>
|
||||||
- `config/` - Environment-specific development configurations
|
<p><strong>Initialization</strong>:</p>
|
||||||
- `extensions/` - Extension development templates and examples
|
<pre><code class="language-bash">cd workspace/tools
|
||||||
- `infra/` - Development infrastructure examples
|
|
||||||
- `runtime/` - Isolated runtime data per user
|
|
||||||
|
|
||||||
## Development Workspace
|
|
||||||
|
|
||||||
### Workspace Management
|
|
||||||
|
|
||||||
The workspace provides a sophisticated development environment:
|
|
||||||
|
|
||||||
**Initialization**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd workspace/tools
|
|
||||||
nu workspace.nu init --user-name developer --infra-name my-infra
|
nu workspace.nu init --user-name developer --infra-name my-infra
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Health Monitoring</strong>:</p>
|
||||||
**Health Monitoring**:
|
<pre><code class="language-bash">nu workspace.nu health --detailed --fix-issues
|
||||||
|
</code></pre>
|
||||||
```bash
|
<p><strong>Path Resolution</strong>:</p>
|
||||||
nu workspace.nu health --detailed --fix-issues
|
<pre><code class="language-nushell">use lib/path-resolver.nu
|
||||||
```plaintext
|
|
||||||
|
|
||||||
**Path Resolution**:
|
|
||||||
|
|
||||||
```nushell
|
|
||||||
use lib/path-resolver.nu
|
|
||||||
let config = (path-resolver resolve_config "user" --workspace-user "john")
|
let config = (path-resolver resolve_config "user" --workspace-user "john")
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="extension-development"><a class="header" href="#extension-development">Extension Development</a></h3>
|
||||||
### Extension Development
|
<p>The workspace provides templates for developing:</p>
|
||||||
|
<ul>
|
||||||
The workspace provides templates for developing:
|
<li><strong>Providers</strong>: Custom cloud provider implementations</li>
|
||||||
|
<li><strong>Task Services</strong>: Infrastructure service components</li>
|
||||||
- **Providers**: Custom cloud provider implementations
|
<li><strong>Clusters</strong>: Complete deployment solutions</li>
|
||||||
- **Task Services**: Infrastructure service components
|
</ul>
|
||||||
- **Clusters**: Complete deployment solutions
|
<p>Templates are available in <code>workspace/extensions/{type}/template/</code></p>
|
||||||
|
<h3 id="configuration-hierarchy"><a class="header" href="#configuration-hierarchy">Configuration Hierarchy</a></h3>
|
||||||
Templates are available in `workspace/extensions/{type}/template/`
|
<p>The workspace implements a sophisticated configuration cascade:</p>
|
||||||
|
<ol>
|
||||||
### Configuration Hierarchy
|
<li>Workspace user configuration (<code>workspace/config/{user}.toml</code>)</li>
|
||||||
|
<li>Environment-specific defaults (<code>workspace/config/{env}-defaults.toml</code>)</li>
|
||||||
The workspace implements a sophisticated configuration cascade:
|
<li>Workspace defaults (<code>workspace/config/dev-defaults.toml</code>)</li>
|
||||||
|
<li>Core system defaults (<code>config.defaults.toml</code>)</li>
|
||||||
1. Workspace user configuration (`workspace/config/{user}.toml`)
|
</ol>
|
||||||
2. Environment-specific defaults (`workspace/config/{env}-defaults.toml`)
|
<h2 id="file-naming-conventions"><a class="header" href="#file-naming-conventions">File Naming Conventions</a></h2>
|
||||||
3. Workspace defaults (`workspace/config/dev-defaults.toml`)
|
<h3 id="nushell-files-nu"><a class="header" href="#nushell-files-nu">Nushell Files (<code>.nu</code>)</a></h3>
|
||||||
4. Core system defaults (`config.defaults.toml`)
|
<ul>
|
||||||
|
<li><strong>Commands</strong>: <code>kebab-case</code> - <code>create-server.nu</code>, <code>validate-config.nu</code></li>
|
||||||
## File Naming Conventions
|
<li><strong>Modules</strong>: <code>snake_case</code> - <code>lib_provisioning</code>, <code>path_resolver</code></li>
|
||||||
|
<li><strong>Scripts</strong>: <code>kebab-case</code> - <code>workspace-health.nu</code>, <code>runtime-manager.nu</code></li>
|
||||||
### Nushell Files (`.nu`)
|
</ul>
|
||||||
|
<h3 id="configuration-files"><a class="header" href="#configuration-files">Configuration Files</a></h3>
|
||||||
- **Commands**: `kebab-case` - `create-server.nu`, `validate-config.nu`
|
<ul>
|
||||||
- **Modules**: `snake_case` - `lib_provisioning`, `path_resolver`
|
<li><strong>TOML</strong>: <code>kebab-case.toml</code> - <code>config-defaults.toml</code>, <code>user-settings.toml</code></li>
|
||||||
- **Scripts**: `kebab-case` - `workspace-health.nu`, `runtime-manager.nu`
|
<li><strong>Environment</strong>: <code>{env}-defaults.toml</code> - <code>dev-defaults.toml</code>, <code>prod-defaults.toml</code></li>
|
||||||
|
<li><strong>Examples</strong>: <code>*.toml.example</code> - <code>local-overrides.toml.example</code></li>
|
||||||
### Configuration Files
|
</ul>
|
||||||
|
<h3 id="nickel-files-ncl"><a class="header" href="#nickel-files-ncl">Nickel Files (<code>.ncl</code>)</a></h3>
|
||||||
- **TOML**: `kebab-case.toml` - `config-defaults.toml`, `user-settings.toml`
|
<ul>
|
||||||
- **Environment**: `{env}-defaults.toml` - `dev-defaults.toml`, `prod-defaults.toml`
|
<li><strong>Schemas</strong>: <code>kebab-case.ncl</code> - <code>server-config.ncl</code>, <code>workflow-schema.ncl</code></li>
|
||||||
- **Examples**: `*.toml.example` - `local-overrides.toml.example`
|
<li><strong>Configuration</strong>: <code>manifest.toml</code> - Package metadata</li>
|
||||||
|
<li><strong>Structure</strong>: Organized in <code>schemas/</code> directories per extension</li>
|
||||||
### KCL Files (`.k`)
|
</ul>
|
||||||
|
<h3 id="build-and-distribution"><a class="header" href="#build-and-distribution">Build and Distribution</a></h3>
|
||||||
- **Schemas**: `PascalCase` types - `ServerConfig`, `WorkflowDefinition`
|
<ul>
|
||||||
- **Files**: `kebab-case.k` - `server-config.k`, `workflow-schema.k`
|
<li><strong>Scripts</strong>: <code>kebab-case.nu</code> - <code>compile-platform.nu</code>, <code>generate-distribution.nu</code></li>
|
||||||
- **Modules**: `kcl.mod` - Module definition files
|
<li><strong>Makefiles</strong>: <code>Makefile</code> - Standard naming</li>
|
||||||
|
<li><strong>Archives</strong>: <code>{project}-{version}-{platform}-{variant}.{ext}</code></li>
|
||||||
### Build and Distribution
|
</ul>
|
||||||
|
<h2 id="navigation-guide"><a class="header" href="#navigation-guide">Navigation Guide</a></h2>
|
||||||
- **Scripts**: `kebab-case.nu` - `compile-platform.nu`, `generate-distribution.nu`
|
<h3 id="finding-components"><a class="header" href="#finding-components">Finding Components</a></h3>
|
||||||
- **Makefiles**: `Makefile` - Standard naming
|
<p><strong>Core System Entry Points</strong>:</p>
|
||||||
- **Archives**: `{project}-{version}-{platform}-{variant}.{ext}`
|
<pre><code class="language-bash"># Main CLI (development version)
|
||||||
|
|
||||||
## Navigation Guide
|
|
||||||
|
|
||||||
### Finding Components
|
|
||||||
|
|
||||||
**Core System Entry Points**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Main CLI (development version)
|
|
||||||
/src/core/nulib/provisioning
|
/src/core/nulib/provisioning
|
||||||
|
|
||||||
# Legacy CLI (production version)
|
# Legacy CLI (production version)
|
||||||
@ -407,12 +364,9 @@ The workspace implements a sophisticated configuration cascade:
|
|||||||
|
|
||||||
# Workspace management
|
# Workspace management
|
||||||
/workspace/tools/workspace.nu
|
/workspace/tools/workspace.nu
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Build System</strong>:</p>
|
||||||
**Build System**:
|
<pre><code class="language-bash"># Main build system
|
||||||
|
|
||||||
```bash
|
|
||||||
# Main build system
|
|
||||||
cd /src/tools && make help
|
cd /src/tools && make help
|
||||||
|
|
||||||
# Quick development build
|
# Quick development build
|
||||||
@ -420,12 +374,9 @@ make dev-build
|
|||||||
|
|
||||||
# Complete distribution
|
# Complete distribution
|
||||||
make all
|
make all
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Configuration Files</strong>:</p>
|
||||||
**Configuration Files**:
|
<pre><code class="language-bash"># System defaults
|
||||||
|
|
||||||
```bash
|
|
||||||
# System defaults
|
|
||||||
/config.defaults.toml
|
/config.defaults.toml
|
||||||
|
|
||||||
# User configuration (workspace)
|
# User configuration (workspace)
|
||||||
@ -433,12 +384,9 @@ make all
|
|||||||
|
|
||||||
# Environment-specific
|
# Environment-specific
|
||||||
/workspace/config/{env}-defaults.toml
|
/workspace/config/{env}-defaults.toml
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Extension Development</strong>:</p>
|
||||||
**Extension Development**:
|
<pre><code class="language-bash"># Provider template
|
||||||
|
|
||||||
```bash
|
|
||||||
# Provider template
|
|
||||||
/workspace/extensions/providers/template/
|
/workspace/extensions/providers/template/
|
||||||
|
|
||||||
# Task service template
|
# Task service template
|
||||||
@ -446,25 +394,18 @@ make all
|
|||||||
|
|
||||||
# Cluster template
|
# Cluster template
|
||||||
/workspace/extensions/clusters/template/
|
/workspace/extensions/clusters/template/
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="common-workflows"><a class="header" href="#common-workflows">Common Workflows</a></h3>
|
||||||
### Common Workflows
|
<p><strong>1. Development Setup</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># Initialize workspace
|
||||||
**1. Development Setup**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Initialize workspace
|
|
||||||
cd workspace/tools
|
cd workspace/tools
|
||||||
nu workspace.nu init --user-name $USER
|
nu workspace.nu init --user-name $USER
|
||||||
|
|
||||||
# Check health
|
# Check health
|
||||||
nu workspace.nu health --detailed
|
nu workspace.nu health --detailed
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>2. Building Distribution</strong>:</p>
|
||||||
**2. Building Distribution**:
|
<pre><code class="language-bash"># Complete build
|
||||||
|
|
||||||
```bash
|
|
||||||
# Complete build
|
|
||||||
cd src/tools
|
cd src/tools
|
||||||
make all
|
make all
|
||||||
|
|
||||||
@ -472,109 +413,95 @@ make all
|
|||||||
make linux
|
make linux
|
||||||
make macos
|
make macos
|
||||||
make windows
|
make windows
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>3. Extension Development</strong>:</p>
|
||||||
**3. Extension Development**:
|
<pre><code class="language-bash"># Create new provider
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create new provider
|
|
||||||
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider
|
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider
|
||||||
|
|
||||||
# Test extension
|
# Test extension
|
||||||
nu workspace/extensions/providers/my-provider/nulib/provider.nu test
|
nu workspace/extensions/providers/my-provider/nulib/provider.nu test
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="legacy-compatibility"><a class="header" href="#legacy-compatibility">Legacy Compatibility</a></h3>
|
||||||
### Legacy Compatibility
|
<p><strong>Existing Commands Still Work</strong>:</p>
|
||||||
|
<pre><code class="language-bash"># All existing commands preserved
|
||||||
**Existing Commands Still Work**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# All existing commands preserved
|
|
||||||
./core/nulib/provisioning server create
|
./core/nulib/provisioning server create
|
||||||
./core/nulib/provisioning taskserv install kubernetes
|
./core/nulib/provisioning taskserv install kubernetes
|
||||||
./core/nulib/provisioning cluster create buildkit
|
./core/nulib/provisioning cluster create buildkit
|
||||||
```plaintext
|
|
||||||
|
|
||||||
**Configuration Migration**:
|
|
||||||
|
|
||||||
- ENV variables still supported as fallbacks
|
|
||||||
- New configuration system provides better defaults
|
|
||||||
- Migration tools available in `src/tools/migration/`
|
|
||||||
|
|
||||||
## Migration Path
|
|
||||||
|
|
||||||
### For Users
|
|
||||||
|
|
||||||
**No Changes Required**:
|
|
||||||
|
|
||||||
- All existing commands continue to work
|
|
||||||
- Configuration files remain compatible
|
|
||||||
- Existing infrastructure deployments unaffected
|
|
||||||
|
|
||||||
**Optional Enhancements**:
|
|
||||||
|
|
||||||
- Migrate to new configuration system for better defaults
|
|
||||||
- Use workspace for development environments
|
|
||||||
- Leverage new build system for custom distributions
|
|
||||||
|
|
||||||
### For Developers
|
|
||||||
|
|
||||||
**Development Environment**:
|
|
||||||
|
|
||||||
1. Initialize development workspace: `nu workspace/tools/workspace.nu init`
|
|
||||||
2. Use new build system: `cd src/tools && make dev-build`
|
|
||||||
3. Leverage extension templates for custom development
|
|
||||||
|
|
||||||
**Build System**:
|
|
||||||
|
|
||||||
1. Use new Makefile for comprehensive build management
|
|
||||||
2. Leverage distribution tools for packaging
|
|
||||||
3. Use release management for version control
|
|
||||||
|
|
||||||
**Orchestrator Integration**:
|
|
||||||
|
|
||||||
1. Start orchestrator for workflow management: `cd src/orchestrator && ./scripts/start-orchestrator.nu`
|
|
||||||
2. Use workflow APIs for complex operations
|
|
||||||
3. Leverage batch operations for efficiency
|
|
||||||
|
|
||||||
### Migration Tools
|
|
||||||
|
|
||||||
**Available Migration Scripts**:
|
|
||||||
|
|
||||||
- `src/tools/migration/config-migration.nu` - Configuration migration
|
|
||||||
- `src/tools/migration/workspace-setup.nu` - Workspace initialization
|
|
||||||
- `src/tools/migration/path-resolver.nu` - Path resolution migration
|
|
||||||
|
|
||||||
**Validation Tools**:
|
|
||||||
|
|
||||||
- `src/tools/validation/system-health.nu` - System health validation
|
|
||||||
- `src/tools/validation/compatibility-check.nu` - Compatibility verification
|
|
||||||
- `src/tools/validation/migration-status.nu` - Migration status tracking
|
|
||||||
|
|
||||||
## Architecture Benefits
|
|
||||||
|
|
||||||
### Development Efficiency
|
|
||||||
|
|
||||||
- **Build System**: Comprehensive 40+ target Makefile system
|
|
||||||
- **Workspace Isolation**: Per-user development environments
|
|
||||||
- **Extension Framework**: Template-based extension development
|
|
||||||
|
|
||||||
### Production Reliability
|
|
||||||
|
|
||||||
- **Backward Compatibility**: All existing functionality preserved
|
|
||||||
- **Configuration Migration**: Gradual migration from ENV to config-driven
|
|
||||||
- **Orchestrator Architecture**: Hybrid Rust/Nushell for performance and flexibility
|
|
||||||
- **Workflow Management**: Batch operations with rollback capabilities
|
|
||||||
|
|
||||||
### Maintenance Benefits
|
|
||||||
|
|
||||||
- **Clean Separation**: Development tools separate from production code
|
|
||||||
- **Organized Structure**: Logical grouping of related functionality
|
|
||||||
- **Documentation**: Comprehensive documentation and examples
|
|
||||||
- **Testing Framework**: Built-in testing and validation tools
|
|
||||||
|
|
||||||
This structure represents a significant evolution in the project's organization while maintaining complete backward compatibility and providing powerful new development capabilities.
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
|
<p><strong>Configuration Migration</strong>:</p>
|
||||||
|
<ul>
|
||||||
|
<li>ENV variables still supported as fallbacks</li>
|
||||||
|
<li>New configuration system provides better defaults</li>
|
||||||
|
<li>Migration tools available in <code>src/tools/migration/</code></li>
|
||||||
|
</ul>
|
||||||
|
<h2 id="migration-path"><a class="header" href="#migration-path">Migration Path</a></h2>
|
||||||
|
<h3 id="for-users"><a class="header" href="#for-users">For Users</a></h3>
|
||||||
|
<p><strong>No Changes Required</strong>:</p>
|
||||||
|
<ul>
|
||||||
|
<li>All existing commands continue to work</li>
|
||||||
|
<li>Configuration files remain compatible</li>
|
||||||
|
<li>Existing infrastructure deployments unaffected</li>
|
||||||
|
</ul>
|
||||||
|
<p><strong>Optional Enhancements</strong>:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Migrate to new configuration system for better defaults</li>
|
||||||
|
<li>Use workspace for development environments</li>
|
||||||
|
<li>Leverage new build system for custom distributions</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="for-developers"><a class="header" href="#for-developers">For Developers</a></h3>
|
||||||
|
<p><strong>Development Environment</strong>:</p>
|
||||||
|
<ol>
|
||||||
|
<li>Initialize development workspace: <code>nu workspace/tools/workspace.nu init</code></li>
|
||||||
|
<li>Use new build system: <code>cd src/tools && make dev-build</code></li>
|
||||||
|
<li>Leverage extension templates for custom development</li>
|
||||||
|
</ol>
|
||||||
|
<p><strong>Build System</strong>:</p>
|
||||||
|
<ol>
|
||||||
|
<li>Use new Makefile for comprehensive build management</li>
|
||||||
|
<li>Leverage distribution tools for packaging</li>
|
||||||
|
<li>Use release management for version control</li>
|
||||||
|
</ol>
|
||||||
|
<p><strong>Orchestrator Integration</strong>:</p>
|
||||||
|
<ol>
|
||||||
|
<li>Start orchestrator for workflow management: <code>cd src/orchestrator && ./scripts/start-orchestrator.nu</code></li>
|
||||||
|
<li>Use workflow APIs for complex operations</li>
|
||||||
|
<li>Leverage batch operations for efficiency</li>
|
||||||
|
</ol>
|
||||||
|
<h3 id="migration-tools"><a class="header" href="#migration-tools">Migration Tools</a></h3>
|
||||||
|
<p><strong>Available Migration Scripts</strong>:</p>
|
||||||
|
<ul>
|
||||||
|
<li><code>src/tools/migration/config-migration.nu</code> - Configuration migration</li>
|
||||||
|
<li><code>src/tools/migration/workspace-setup.nu</code> - Workspace initialization</li>
|
||||||
|
<li><code>src/tools/migration/path-resolver.nu</code> - Path resolution migration</li>
|
||||||
|
</ul>
|
||||||
|
<p><strong>Validation Tools</strong>:</p>
|
||||||
|
<ul>
|
||||||
|
<li><code>src/tools/validation/system-health.nu</code> - System health validation</li>
|
||||||
|
<li><code>src/tools/validation/compatibility-check.nu</code> - Compatibility verification</li>
|
||||||
|
<li><code>src/tools/validation/migration-status.nu</code> - Migration status tracking</li>
|
||||||
|
</ul>
|
||||||
|
<h2 id="architecture-benefits"><a class="header" href="#architecture-benefits">Architecture Benefits</a></h2>
|
||||||
|
<h3 id="development-efficiency"><a class="header" href="#development-efficiency">Development Efficiency</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Build System</strong>: Comprehensive 40+ target Makefile system</li>
|
||||||
|
<li><strong>Workspace Isolation</strong>: Per-user development environments</li>
|
||||||
|
<li><strong>Extension Framework</strong>: Template-based extension development</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="production-reliability"><a class="header" href="#production-reliability">Production Reliability</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Backward Compatibility</strong>: All existing functionality preserved</li>
|
||||||
|
<li><strong>Configuration Migration</strong>: Gradual migration from ENV to config-driven</li>
|
||||||
|
<li><strong>Orchestrator Architecture</strong>: Hybrid Rust/Nushell for performance and flexibility</li>
|
||||||
|
<li><strong>Workflow Management</strong>: Batch operations with rollback capabilities</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="maintenance-benefits"><a class="header" href="#maintenance-benefits">Maintenance Benefits</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Clean Separation</strong>: Development tools separate from production code</li>
|
||||||
|
<li><strong>Organized Structure</strong>: Logical grouping of related functionality</li>
|
||||||
|
<li><strong>Documentation</strong>: Comprehensive documentation and examples</li>
|
||||||
|
<li><strong>Testing Framework</strong>: Built-in testing and validation tools</li>
|
||||||
|
</ul>
|
||||||
|
<p>This structure represents a significant evolution in the project’s organization while maintaining complete backward compatibility and providing powerful new development capabilities.</p>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@ -18,7 +18,7 @@ with others.
|
|||||||
|
|
||||||
The OFL allows the licensed fonts to be used, studied, modified and
|
The OFL allows the licensed fonts to be used, studied, modified and
|
||||||
redistributed freely as long as they are not sold by themselves. The
|
redistributed freely as long as they are not sold by themselves. The
|
||||||
fonts, including any derivative works, can be bundled, embedded,
|
fonts, including any derivative works, can be bundled, embedded,
|
||||||
redistributed and/or sold with any software provided that any reserved
|
redistributed and/or sold with any software provided that any reserved
|
||||||
names are not used by derivative works. The fonts and derivatives,
|
names are not used by derivative works. The fonts and derivatives,
|
||||||
however, cannot be released under any other type of license. The
|
however, cannot be released under any other type of license. The
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -191,42 +191,27 @@
|
|||||||
<p><strong>Best for</strong>: Non-critical environments, development, staging</p>
|
<p><strong>Best for</strong>: Non-critical environments, development, staging</p>
|
||||||
<pre><code class="language-bash"># Direct update without downtime consideration
|
<pre><code class="language-bash"># Direct update without downtime consideration
|
||||||
provisioning t create <taskserv> --infra <project>
|
provisioning t create <taskserv> --infra <project>
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="strategy-2-rolling-updates-recommended"><a class="header" href="#strategy-2-rolling-updates-recommended">Strategy 2: Rolling Updates (Recommended)</a></h3>
|
||||||
### Strategy 2: Rolling Updates (Recommended)
|
<p><strong>Best for</strong>: Production environments, high availability</p>
|
||||||
|
<pre><code class="language-bash"># Update servers one by one
|
||||||
**Best for**: Production environments, high availability
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Update servers one by one
|
|
||||||
provisioning s update --infra <project> --rolling
|
provisioning s update --infra <project> --rolling
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="strategy-3-blue-green-deployment-safest"><a class="header" href="#strategy-3-blue-green-deployment-safest">Strategy 3: Blue-Green Deployment (Safest)</a></h3>
|
||||||
### Strategy 3: Blue-Green Deployment (Safest)
|
<p><strong>Best for</strong>: Critical production, zero-downtime requirements</p>
|
||||||
|
<pre><code class="language-bash"># Create new infrastructure, switch traffic, remove old
|
||||||
**Best for**: Critical production, zero-downtime requirements
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create new infrastructure, switch traffic, remove old
|
|
||||||
provisioning ws init <project>-green
|
provisioning ws init <project>-green
|
||||||
# ... configure and deploy
|
# ... configure and deploy
|
||||||
# ... switch traffic
|
# ... switch traffic
|
||||||
provisioning ws delete <project>-blue
|
provisioning ws delete <project>-blue
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="step-1-check-for-updates"><a class="header" href="#step-1-check-for-updates">Step 1: Check for Updates</a></h2>
|
||||||
## Step 1: Check for Updates
|
<h3 id="11-check-all-task-services"><a class="header" href="#11-check-all-task-services">1.1 Check All Task Services</a></h3>
|
||||||
|
<pre><code class="language-bash"># Check all taskservs for updates
|
||||||
### 1.1 Check All Task Services
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check all taskservs for updates
|
|
||||||
provisioning t check-updates
|
provisioning t check-updates
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">📦 Task Service Update Check:
|
||||||
|
|
||||||
```plaintext
|
|
||||||
📦 Task Service Update Check:
|
|
||||||
|
|
||||||
NAME CURRENT LATEST STATUS
|
NAME CURRENT LATEST STATUS
|
||||||
kubernetes 1.29.0 1.30.0 ⬆️ update available
|
kubernetes 1.29.0 1.30.0 ⬆️ update available
|
||||||
@ -236,19 +221,13 @@ postgres 15.5 16.1 ⬆️ update available
|
|||||||
redis 7.2.3 7.2.3 ✅ up-to-date
|
redis 7.2.3 7.2.3 ✅ up-to-date
|
||||||
|
|
||||||
Updates available: 3
|
Updates available: 3
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="12-check-specific-task-service"><a class="header" href="#12-check-specific-task-service">1.2 Check Specific Task Service</a></h3>
|
||||||
### 1.2 Check Specific Task Service
|
<pre><code class="language-bash"># Check specific taskserv
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check specific taskserv
|
|
||||||
provisioning t check-updates kubernetes
|
provisioning t check-updates kubernetes
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">📦 Kubernetes Update Check:
|
||||||
|
|
||||||
```plaintext
|
|
||||||
📦 Kubernetes Update Check:
|
|
||||||
|
|
||||||
Current: 1.29.0
|
Current: 1.29.0
|
||||||
Latest: 1.30.0
|
Latest: 1.30.0
|
||||||
@ -264,19 +243,13 @@ Breaking Changes:
|
|||||||
• None
|
• None
|
||||||
|
|
||||||
Recommended: ✅ Safe to update
|
Recommended: ✅ Safe to update
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="13-check-version-status"><a class="header" href="#13-check-version-status">1.3 Check Version Status</a></h3>
|
||||||
### 1.3 Check Version Status
|
<pre><code class="language-bash"># Show detailed version information
|
||||||
|
|
||||||
```bash
|
|
||||||
# Show detailed version information
|
|
||||||
provisioning version show
|
provisioning version show
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">📋 Component Versions:
|
||||||
|
|
||||||
```plaintext
|
|
||||||
📋 Component Versions:
|
|
||||||
|
|
||||||
COMPONENT CURRENT LATEST DAYS OLD STATUS
|
COMPONENT CURRENT LATEST DAYS OLD STATUS
|
||||||
kubernetes 1.29.0 1.30.0 45 ⬆️ update
|
kubernetes 1.29.0 1.30.0 45 ⬆️ update
|
||||||
@ -284,51 +257,32 @@ containerd 1.7.13 1.7.13 0 ✅ current
|
|||||||
cilium 1.14.5 1.15.0 30 ⬆️ update
|
cilium 1.14.5 1.15.0 30 ⬆️ update
|
||||||
postgres 15.5 16.1 60 ⬆️ update (major)
|
postgres 15.5 16.1 60 ⬆️ update (major)
|
||||||
redis 7.2.3 7.2.3 0 ✅ current
|
redis 7.2.3 7.2.3 0 ✅ current
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="14-check-for-security-updates"><a class="header" href="#14-check-for-security-updates">1.4 Check for Security Updates</a></h3>
|
||||||
### 1.4 Check for Security Updates
|
<pre><code class="language-bash"># Check for security-related updates
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check for security-related updates
|
|
||||||
provisioning version updates --security-only
|
provisioning version updates --security-only
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="step-2-plan-your-update"><a class="header" href="#step-2-plan-your-update">Step 2: Plan Your Update</a></h2>
|
||||||
## Step 2: Plan Your Update
|
<h3 id="21-review-current-configuration"><a class="header" href="#21-review-current-configuration">2.1 Review Current Configuration</a></h3>
|
||||||
|
<pre><code class="language-bash"># Show current infrastructure
|
||||||
### 2.1 Review Current Configuration
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Show current infrastructure
|
|
||||||
provisioning show settings --infra my-production
|
provisioning show settings --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="22-backup-configuration"><a class="header" href="#22-backup-configuration">2.2 Backup Configuration</a></h3>
|
||||||
### 2.2 Backup Configuration
|
<pre><code class="language-bash"># Create configuration backup
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create configuration backup
|
|
||||||
cp -r workspace/infra/my-production workspace/infra/my-production.backup-$(date +%Y%m%d)
|
cp -r workspace/infra/my-production workspace/infra/my-production.backup-$(date +%Y%m%d)
|
||||||
|
|
||||||
# Or use built-in backup
|
# Or use built-in backup
|
||||||
provisioning ws backup my-production
|
provisioning ws backup my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">✅ Backup created: workspace/backups/my-production-20250930.tar.gz
|
||||||
|
</code></pre>
|
||||||
```plaintext
|
<h3 id="23-create-update-plan"><a class="header" href="#23-create-update-plan">2.3 Create Update Plan</a></h3>
|
||||||
✅ Backup created: workspace/backups/my-production-20250930.tar.gz
|
<pre><code class="language-bash"># Generate update plan
|
||||||
```plaintext
|
|
||||||
|
|
||||||
### 2.3 Create Update Plan
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Generate update plan
|
|
||||||
provisioning plan update --infra my-production
|
provisioning plan update --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">📝 Update Plan for my-production:
|
||||||
|
|
||||||
```plaintext
|
|
||||||
📝 Update Plan for my-production:
|
|
||||||
|
|
||||||
Phase 1: Minor Updates (Low Risk)
|
Phase 1: Minor Updates (Low Risk)
|
||||||
• containerd: No update needed
|
• containerd: No update needed
|
||||||
@ -348,23 +302,15 @@ Recommended Order:
|
|||||||
|
|
||||||
Total Estimated Time: 30 minutes
|
Total Estimated Time: 30 minutes
|
||||||
Recommended: Test in staging environment first
|
Recommended: Test in staging environment first
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="step-3-update-task-services"><a class="header" href="#step-3-update-task-services">Step 3: Update Task Services</a></h2>
|
||||||
## Step 3: Update Task Services
|
<h3 id="31-update-non-critical-service-cilium-example"><a class="header" href="#31-update-non-critical-service-cilium-example">3.1 Update Non-Critical Service (Cilium Example)</a></h3>
|
||||||
|
<h4 id="dry-run-update"><a class="header" href="#dry-run-update">Dry-Run Update</a></h4>
|
||||||
### 3.1 Update Non-Critical Service (Cilium Example)
|
<pre><code class="language-bash"># Test update without applying
|
||||||
|
|
||||||
#### Dry-Run Update
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Test update without applying
|
|
||||||
provisioning t create cilium --infra my-production --check
|
provisioning t create cilium --infra my-production --check
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🔍 CHECK MODE: Simulating Cilium update
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🔍 CHECK MODE: Simulating Cilium update
|
|
||||||
|
|
||||||
Current: 1.14.5
|
Current: 1.14.5
|
||||||
Target: 1.15.0
|
Target: 1.15.0
|
||||||
@ -377,33 +323,21 @@ Would perform:
|
|||||||
|
|
||||||
Estimated downtime: <1 minute per node
|
Estimated downtime: <1 minute per node
|
||||||
No errors detected. Ready to update.
|
No errors detected. Ready to update.
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="generate-updated-configuration"><a class="header" href="#generate-updated-configuration">Generate Updated Configuration</a></h4>
|
||||||
#### Generate Updated Configuration
|
<pre><code class="language-bash"># Generate new configuration
|
||||||
|
|
||||||
```bash
|
|
||||||
# Generate new configuration
|
|
||||||
provisioning t generate cilium --infra my-production
|
provisioning t generate cilium --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">✅ Generated Cilium configuration (version 1.15.0)
|
||||||
|
Saved to: workspace/infra/my-production/taskservs/cilium.ncl
|
||||||
```plaintext
|
</code></pre>
|
||||||
✅ Generated Cilium configuration (version 1.15.0)
|
<h4 id="apply-update"><a class="header" href="#apply-update">Apply Update</a></h4>
|
||||||
Saved to: workspace/infra/my-production/taskservs/cilium.k
|
<pre><code class="language-bash"># Apply update
|
||||||
```plaintext
|
|
||||||
|
|
||||||
#### Apply Update
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Apply update
|
|
||||||
provisioning t create cilium --infra my-production
|
provisioning t create cilium --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🚀 Updating Cilium on my-production...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🚀 Updating Cilium on my-production...
|
|
||||||
|
|
||||||
Downloading Cilium 1.15.0... ⏳
|
Downloading Cilium 1.15.0... ⏳
|
||||||
✅ Downloaded
|
✅ Downloaded
|
||||||
@ -423,19 +357,13 @@ Verifying connectivity... ⏳
|
|||||||
🎉 Cilium update complete!
|
🎉 Cilium update complete!
|
||||||
Version: 1.14.5 → 1.15.0
|
Version: 1.14.5 → 1.15.0
|
||||||
Downtime: 0 minutes
|
Downtime: 0 minutes
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="verify-update"><a class="header" href="#verify-update">Verify Update</a></h4>
|
||||||
#### Verify Update
|
<pre><code class="language-bash"># Verify updated version
|
||||||
|
|
||||||
```bash
|
|
||||||
# Verify updated version
|
|
||||||
provisioning version taskserv cilium
|
provisioning version taskserv cilium
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">📦 Cilium Version Info:
|
||||||
|
|
||||||
```plaintext
|
|
||||||
📦 Cilium Version Info:
|
|
||||||
|
|
||||||
Installed: 1.15.0
|
Installed: 1.15.0
|
||||||
Latest: 1.15.0
|
Latest: 1.15.0
|
||||||
@ -444,49 +372,33 @@ Status: ✅ Up-to-date
|
|||||||
Nodes:
|
Nodes:
|
||||||
✅ web-01: 1.15.0 (running)
|
✅ web-01: 1.15.0 (running)
|
||||||
✅ web-02: 1.15.0 (running)
|
✅ web-02: 1.15.0 (running)
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="32-update-critical-service-kubernetes-example"><a class="header" href="#32-update-critical-service-kubernetes-example">3.2 Update Critical Service (Kubernetes Example)</a></h3>
|
||||||
### 3.2 Update Critical Service (Kubernetes Example)
|
<h4 id="test-in-staging-first"><a class="header" href="#test-in-staging-first">Test in Staging First</a></h4>
|
||||||
|
<pre><code class="language-bash"># If you have staging environment
|
||||||
#### Test in Staging First
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# If you have staging environment
|
|
||||||
provisioning t create kubernetes --infra my-staging --check
|
provisioning t create kubernetes --infra my-staging --check
|
||||||
provisioning t create kubernetes --infra my-staging
|
provisioning t create kubernetes --infra my-staging
|
||||||
|
|
||||||
# Run integration tests
|
# Run integration tests
|
||||||
provisioning test kubernetes --infra my-staging
|
provisioning test kubernetes --infra my-staging
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="backup-current-state"><a class="header" href="#backup-current-state">Backup Current State</a></h4>
|
||||||
#### Backup Current State
|
<pre><code class="language-bash"># Backup Kubernetes state
|
||||||
|
|
||||||
```bash
|
|
||||||
# Backup Kubernetes state
|
|
||||||
kubectl get all -A -o yaml > k8s-backup-$(date +%Y%m%d).yaml
|
kubectl get all -A -o yaml > k8s-backup-$(date +%Y%m%d).yaml
|
||||||
|
|
||||||
# Backup etcd (if using external etcd)
|
# Backup etcd (if using external etcd)
|
||||||
provisioning t backup kubernetes --infra my-production
|
provisioning t backup kubernetes --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="schedule-maintenance-window"><a class="header" href="#schedule-maintenance-window">Schedule Maintenance Window</a></h4>
|
||||||
#### Schedule Maintenance Window
|
<pre><code class="language-bash"># Set maintenance mode (optional, if supported)
|
||||||
|
|
||||||
```bash
|
|
||||||
# Set maintenance mode (optional, if supported)
|
|
||||||
provisioning maintenance enable --infra my-production --duration 30m
|
provisioning maintenance enable --infra my-production --duration 30m
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="update-kubernetes"><a class="header" href="#update-kubernetes">Update Kubernetes</a></h4>
|
||||||
#### Update Kubernetes
|
<pre><code class="language-bash"># Update control plane first
|
||||||
|
|
||||||
```bash
|
|
||||||
# Update control plane first
|
|
||||||
provisioning t create kubernetes --infra my-production --control-plane-only
|
provisioning t create kubernetes --infra my-production --control-plane-only
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🚀 Updating Kubernetes control plane on my-production...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🚀 Updating Kubernetes control plane on my-production...
|
|
||||||
|
|
||||||
Draining control plane: web-01... ⏳
|
Draining control plane: web-01... ⏳
|
||||||
✅ web-01 drained
|
✅ web-01 drained
|
||||||
@ -501,17 +413,12 @@ Verifying control plane... ⏳
|
|||||||
✅ Control plane healthy
|
✅ Control plane healthy
|
||||||
|
|
||||||
🎉 Control plane update complete!
|
🎉 Control plane update complete!
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<pre><code class="language-bash"># Update worker nodes one by one
|
||||||
```bash
|
|
||||||
# Update worker nodes one by one
|
|
||||||
provisioning t create kubernetes --infra my-production --workers-only --rolling
|
provisioning t create kubernetes --infra my-production --workers-only --rolling
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🚀 Updating Kubernetes workers on my-production...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🚀 Updating Kubernetes workers on my-production...
|
|
||||||
|
|
||||||
Rolling update: web-02...
|
Rolling update: web-02...
|
||||||
Draining... ⏳
|
Draining... ⏳
|
||||||
@ -529,44 +436,28 @@ Rolling update: web-02...
|
|||||||
🎉 Worker update complete!
|
🎉 Worker update complete!
|
||||||
Updated: web-02
|
Updated: web-02
|
||||||
Version: 1.30.0
|
Version: 1.30.0
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="verify-update-1"><a class="header" href="#verify-update-1">Verify Update</a></h4>
|
||||||
#### Verify Update
|
<pre><code class="language-bash"># Verify Kubernetes cluster
|
||||||
|
|
||||||
```bash
|
|
||||||
# Verify Kubernetes cluster
|
|
||||||
kubectl get nodes
|
kubectl get nodes
|
||||||
provisioning version taskserv kubernetes
|
provisioning version taskserv kubernetes
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">NAME STATUS ROLES AGE VERSION
|
||||||
|
|
||||||
```plaintext
|
|
||||||
NAME STATUS ROLES AGE VERSION
|
|
||||||
web-01 Ready control-plane 30d v1.30.0
|
web-01 Ready control-plane 30d v1.30.0
|
||||||
web-02 Ready <none> 30d v1.30.0
|
web-02 Ready <none> 30d v1.30.0
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<pre><code class="language-bash"># Run smoke tests
|
||||||
```bash
|
|
||||||
# Run smoke tests
|
|
||||||
provisioning test kubernetes --infra my-production
|
provisioning test kubernetes --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="33-update-database-postgresql-example"><a class="header" href="#33-update-database-postgresql-example">3.3 Update Database (PostgreSQL Example)</a></h3>
|
||||||
### 3.3 Update Database (PostgreSQL Example)
|
<p>⚠️ <strong>WARNING</strong>: Database updates may require data migration. Always backup first!</p>
|
||||||
|
<h4 id="backup-database"><a class="header" href="#backup-database">Backup Database</a></h4>
|
||||||
⚠️ **WARNING**: Database updates may require data migration. Always backup first!
|
<pre><code class="language-bash"># Backup PostgreSQL database
|
||||||
|
|
||||||
#### Backup Database
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Backup PostgreSQL database
|
|
||||||
provisioning t backup postgres --infra my-production
|
provisioning t backup postgres --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🗄️ Backing up PostgreSQL...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🗄️ Backing up PostgreSQL...
|
|
||||||
|
|
||||||
Creating dump: my-production-postgres-20250930.sql... ⏳
|
Creating dump: my-production-postgres-20250930.sql... ⏳
|
||||||
✅ Dump created (2.3 GB)
|
✅ Dump created (2.3 GB)
|
||||||
@ -575,19 +466,13 @@ Compressing... ⏳
|
|||||||
✅ Compressed (450 MB)
|
✅ Compressed (450 MB)
|
||||||
|
|
||||||
Saved to: workspace/backups/postgres/my-production-20250930.sql.gz
|
Saved to: workspace/backups/postgres/my-production-20250930.sql.gz
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="check-compatibility"><a class="header" href="#check-compatibility">Check Compatibility</a></h4>
|
||||||
#### Check Compatibility
|
<pre><code class="language-bash"># Check if data migration is needed
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check if data migration is needed
|
|
||||||
provisioning t check-migration postgres --from 15.5 --to 16.1
|
provisioning t check-migration postgres --from 15.5 --to 16.1
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🔍 PostgreSQL Migration Check:
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🔍 PostgreSQL Migration Check:
|
|
||||||
|
|
||||||
From: 15.5
|
From: 15.5
|
||||||
To: 16.1
|
To: 16.1
|
||||||
@ -605,19 +490,13 @@ Estimated Time: 15-30 minutes (depending on data size)
|
|||||||
Estimated Downtime: 15-30 minutes
|
Estimated Downtime: 15-30 minutes
|
||||||
|
|
||||||
Recommended: Use streaming replication for zero-downtime upgrade
|
Recommended: Use streaming replication for zero-downtime upgrade
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="perform-update"><a class="header" href="#perform-update">Perform Update</a></h4>
|
||||||
#### Perform Update
|
<pre><code class="language-bash"># Update PostgreSQL (with automatic migration)
|
||||||
|
|
||||||
```bash
|
|
||||||
# Update PostgreSQL (with automatic migration)
|
|
||||||
provisioning t create postgres --infra my-production --migrate
|
provisioning t create postgres --infra my-production --migrate
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🚀 Updating PostgreSQL on my-production...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🚀 Updating PostgreSQL on my-production...
|
|
||||||
|
|
||||||
⚠️ Major version upgrade detected (15.5 → 16.1)
|
⚠️ Major version upgrade detected (15.5 → 16.1)
|
||||||
Automatic migration will be performed
|
Automatic migration will be performed
|
||||||
@ -646,29 +525,19 @@ Verifying data integrity... ⏳
|
|||||||
🎉 PostgreSQL update complete!
|
🎉 PostgreSQL update complete!
|
||||||
Version: 15.5 → 16.1
|
Version: 15.5 → 16.1
|
||||||
Downtime: 18 minutes
|
Downtime: 18 minutes
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h4 id="verify-update-2"><a class="header" href="#verify-update-2">Verify Update</a></h4>
|
||||||
#### Verify Update
|
<pre><code class="language-bash"># Verify PostgreSQL
|
||||||
|
|
||||||
```bash
|
|
||||||
# Verify PostgreSQL
|
|
||||||
provisioning version taskserv postgres
|
provisioning version taskserv postgres
|
||||||
ssh db-01 "psql --version"
|
ssh db-01 "psql --version"
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="step-4-update-multiple-services"><a class="header" href="#step-4-update-multiple-services">Step 4: Update Multiple Services</a></h2>
|
||||||
## Step 4: Update Multiple Services
|
<h3 id="41-batch-update-sequentially"><a class="header" href="#41-batch-update-sequentially">4.1 Batch Update (Sequentially)</a></h3>
|
||||||
|
<pre><code class="language-bash"># Update multiple taskservs one by one
|
||||||
### 4.1 Batch Update (Sequentially)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Update multiple taskservs one by one
|
|
||||||
provisioning t update --infra my-production --taskservs cilium,containerd,redis
|
provisioning t update --infra my-production --taskservs cilium,containerd,redis
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🚀 Updating 3 taskservs on my-production...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🚀 Updating 3 taskservs on my-production...
|
|
||||||
|
|
||||||
[1/3] Updating cilium... ⏳
|
[1/3] Updating cilium... ⏳
|
||||||
✅ cilium updated (1.15.0)
|
✅ cilium updated (1.15.0)
|
||||||
@ -682,19 +551,13 @@ provisioning t update --infra my-production --taskservs cilium,containerd,redis
|
|||||||
🎉 All updates complete!
|
🎉 All updates complete!
|
||||||
Updated: 3 taskservs
|
Updated: 3 taskservs
|
||||||
Total time: 8 minutes
|
Total time: 8 minutes
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="42-parallel-update-non-dependent-services"><a class="header" href="#42-parallel-update-non-dependent-services">4.2 Parallel Update (Non-Dependent Services)</a></h3>
|
||||||
### 4.2 Parallel Update (Non-Dependent Services)
|
<pre><code class="language-bash"># Update taskservs in parallel (if they don't depend on each other)
|
||||||
|
|
||||||
```bash
|
|
||||||
# Update taskservs in parallel (if they don't depend on each other)
|
|
||||||
provisioning t update --infra my-production --taskservs redis,postgres --parallel
|
provisioning t update --infra my-production --taskservs redis,postgres --parallel
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🚀 Updating 2 taskservs in parallel on my-production...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🚀 Updating 2 taskservs in parallel on my-production...
|
|
||||||
|
|
||||||
redis: Updating... ⏳
|
redis: Updating... ⏳
|
||||||
postgres: Updating... ⏳
|
postgres: Updating... ⏳
|
||||||
@ -705,50 +568,35 @@ postgres: ✅ Updated (16.1)
|
|||||||
🎉 All updates complete!
|
🎉 All updates complete!
|
||||||
Updated: 2 taskservs
|
Updated: 2 taskservs
|
||||||
Total time: 3 minutes (parallel)
|
Total time: 3 minutes (parallel)
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="step-5-update-server-configuration"><a class="header" href="#step-5-update-server-configuration">Step 5: Update Server Configuration</a></h2>
|
||||||
## Step 5: Update Server Configuration
|
<h3 id="51-update-server-resources"><a class="header" href="#51-update-server-resources">5.1 Update Server Resources</a></h3>
|
||||||
|
<pre><code class="language-bash"># Edit server configuration
|
||||||
### 5.1 Update Server Resources
|
provisioning sops workspace/infra/my-production/servers.ncl
|
||||||
|
</code></pre>
|
||||||
```bash
|
<p><strong>Example: Upgrade server plan</strong></p>
|
||||||
# Edit server configuration
|
<pre><code class="language-kcl"># Before
|
||||||
provisioning sops workspace/infra/my-production/servers.k
|
|
||||||
```plaintext
|
|
||||||
|
|
||||||
**Example: Upgrade server plan**
|
|
||||||
|
|
||||||
```kcl
|
|
||||||
# Before
|
|
||||||
{
|
{
|
||||||
name = "web-01"
|
name = "web-01"
|
||||||
plan = "1xCPU-2GB" # Old plan
|
plan = "1xCPU-2 GB" # Old plan
|
||||||
}
|
}
|
||||||
|
|
||||||
# After
|
# After
|
||||||
{
|
{
|
||||||
name = "web-01"
|
name = "web-01"
|
||||||
plan = "2xCPU-4GB" # New plan
|
plan = "2xCPU-4 GB" # New plan
|
||||||
}
|
}
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<pre><code class="language-bash"># Apply server update
|
||||||
```bash
|
|
||||||
# Apply server update
|
|
||||||
provisioning s update --infra my-production --check
|
provisioning s update --infra my-production --check
|
||||||
provisioning s update --infra my-production
|
provisioning s update --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="52-update-server-os"><a class="header" href="#52-update-server-os">5.2 Update Server OS</a></h3>
|
||||||
### 5.2 Update Server OS
|
<pre><code class="language-bash"># Update operating system packages
|
||||||
|
|
||||||
```bash
|
|
||||||
# Update operating system packages
|
|
||||||
provisioning s update --infra my-production --os-update
|
provisioning s update --infra my-production --os-update
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🚀 Updating OS packages on my-production servers...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🚀 Updating OS packages on my-production servers...
|
|
||||||
|
|
||||||
web-01: Updating packages... ⏳
|
web-01: Updating packages... ⏳
|
||||||
✅ web-01: 24 packages updated
|
✅ web-01: 24 packages updated
|
||||||
@ -760,23 +608,15 @@ db-01: Updating packages... ⏳
|
|||||||
✅ db-01: 24 packages updated
|
✅ db-01: 24 packages updated
|
||||||
|
|
||||||
🎉 OS updates complete!
|
🎉 OS updates complete!
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="step-6-rollback-procedures"><a class="header" href="#step-6-rollback-procedures">Step 6: Rollback Procedures</a></h2>
|
||||||
## Step 6: Rollback Procedures
|
<h3 id="61-rollback-task-service"><a class="header" href="#61-rollback-task-service">6.1 Rollback Task Service</a></h3>
|
||||||
|
<p>If update fails or causes issues:</p>
|
||||||
### 6.1 Rollback Task Service
|
<pre><code class="language-bash"># Rollback to previous version
|
||||||
|
|
||||||
If update fails or causes issues:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Rollback to previous version
|
|
||||||
provisioning t rollback cilium --infra my-production
|
provisioning t rollback cilium --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🔄 Rolling back Cilium on my-production...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🔄 Rolling back Cilium on my-production...
|
|
||||||
|
|
||||||
Current: 1.15.0
|
Current: 1.15.0
|
||||||
Target: 1.14.5 (previous version)
|
Target: 1.14.5 (previous version)
|
||||||
@ -792,35 +632,22 @@ Verifying connectivity... ⏳
|
|||||||
|
|
||||||
🎉 Rollback complete!
|
🎉 Rollback complete!
|
||||||
Version: 1.15.0 → 1.14.5
|
Version: 1.15.0 → 1.14.5
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="62-rollback-from-backup"><a class="header" href="#62-rollback-from-backup">6.2 Rollback from Backup</a></h3>
|
||||||
### 6.2 Rollback from Backup
|
<pre><code class="language-bash"># Restore configuration from backup
|
||||||
|
|
||||||
```bash
|
|
||||||
# Restore configuration from backup
|
|
||||||
provisioning ws restore my-production --from workspace/backups/my-production-20250930.tar.gz
|
provisioning ws restore my-production --from workspace/backups/my-production-20250930.tar.gz
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="63-emergency-rollback"><a class="header" href="#63-emergency-rollback">6.3 Emergency Rollback</a></h3>
|
||||||
### 6.3 Emergency Rollback
|
<pre><code class="language-bash"># Complete infrastructure rollback
|
||||||
|
|
||||||
```bash
|
|
||||||
# Complete infrastructure rollback
|
|
||||||
provisioning rollback --infra my-production --to-snapshot <snapshot-id>
|
provisioning rollback --infra my-production --to-snapshot <snapshot-id>
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="step-7-post-update-verification"><a class="header" href="#step-7-post-update-verification">Step 7: Post-Update Verification</a></h2>
|
||||||
## Step 7: Post-Update Verification
|
<h3 id="71-verify-all-components"><a class="header" href="#71-verify-all-components">7.1 Verify All Components</a></h3>
|
||||||
|
<pre><code class="language-bash"># Check overall health
|
||||||
### 7.1 Verify All Components
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check overall health
|
|
||||||
provisioning health --infra my-production
|
provisioning health --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🏥 Health Check: my-production
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🏥 Health Check: my-production
|
|
||||||
|
|
||||||
Servers:
|
Servers:
|
||||||
✅ web-01: Healthy
|
✅ web-01: Healthy
|
||||||
@ -837,26 +664,17 @@ Clusters:
|
|||||||
✅ buildkit: 2/2 replicas (healthy)
|
✅ buildkit: 2/2 replicas (healthy)
|
||||||
|
|
||||||
Overall Status: ✅ All systems healthy
|
Overall Status: ✅ All systems healthy
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="72-verify-version-updates"><a class="header" href="#72-verify-version-updates">7.2 Verify Version Updates</a></h3>
|
||||||
### 7.2 Verify Version Updates
|
<pre><code class="language-bash"># Verify all versions are updated
|
||||||
|
|
||||||
```bash
|
|
||||||
# Verify all versions are updated
|
|
||||||
provisioning version show
|
provisioning version show
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="73-run-integration-tests"><a class="header" href="#73-run-integration-tests">7.3 Run Integration Tests</a></h3>
|
||||||
### 7.3 Run Integration Tests
|
<pre><code class="language-bash"># Run comprehensive tests
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run comprehensive tests
|
|
||||||
provisioning test all --infra my-production
|
provisioning test all --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<p><strong>Expected Output:</strong></p>
|
||||||
**Expected Output:**
|
<pre><code class="language-plaintext">🧪 Running Integration Tests...
|
||||||
|
|
||||||
```plaintext
|
|
||||||
🧪 Running Integration Tests...
|
|
||||||
|
|
||||||
[1/5] Server connectivity... ⏳
|
[1/5] Server connectivity... ⏳
|
||||||
✅ All servers reachable
|
✅ All servers reachable
|
||||||
@ -874,70 +692,66 @@ provisioning test all --infra my-production
|
|||||||
✅ All applications healthy
|
✅ All applications healthy
|
||||||
|
|
||||||
🎉 All tests passed!
|
🎉 All tests passed!
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="74-monitor-for-issues"><a class="header" href="#74-monitor-for-issues">7.4 Monitor for Issues</a></h3>
|
||||||
### 7.4 Monitor for Issues
|
<pre><code class="language-bash"># Monitor logs for errors
|
||||||
|
|
||||||
```bash
|
|
||||||
# Monitor logs for errors
|
|
||||||
provisioning logs --infra my-production --follow --level error
|
provisioning logs --infra my-production --follow --level error
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="update-checklist"><a class="header" href="#update-checklist">Update Checklist</a></h2>
|
||||||
## Update Checklist
|
<p>Use this checklist for production updates:</p>
|
||||||
|
<ul>
|
||||||
Use this checklist for production updates:
|
<li><input disabled="" type="checkbox"/>
|
||||||
|
Check for available updates</li>
|
||||||
- [ ] Check for available updates
|
<li><input disabled="" type="checkbox"/>
|
||||||
- [ ] Review changelog and breaking changes
|
Review changelog and breaking changes</li>
|
||||||
- [ ] Create configuration backup
|
<li><input disabled="" type="checkbox"/>
|
||||||
- [ ] Test update in staging environment
|
Create configuration backup</li>
|
||||||
- [ ] Schedule maintenance window
|
<li><input disabled="" type="checkbox"/>
|
||||||
- [ ] Notify team/users of maintenance
|
Test update in staging environment</li>
|
||||||
- [ ] Update non-critical services first
|
<li><input disabled="" type="checkbox"/>
|
||||||
- [ ] Verify each update before proceeding
|
Schedule maintenance window</li>
|
||||||
- [ ] Update critical services with rolling updates
|
<li><input disabled="" type="checkbox"/>
|
||||||
- [ ] Backup database before major updates
|
Notify team/users of maintenance</li>
|
||||||
- [ ] Verify all components after update
|
<li><input disabled="" type="checkbox"/>
|
||||||
- [ ] Run integration tests
|
Update non-critical services first</li>
|
||||||
- [ ] Monitor for issues (30 minutes minimum)
|
<li><input disabled="" type="checkbox"/>
|
||||||
- [ ] Document any issues encountered
|
Verify each update before proceeding</li>
|
||||||
- [ ] Close maintenance window
|
<li><input disabled="" type="checkbox"/>
|
||||||
|
Update critical services with rolling updates</li>
|
||||||
## Common Update Scenarios
|
<li><input disabled="" type="checkbox"/>
|
||||||
|
Backup database before major updates</li>
|
||||||
### Scenario 1: Minor Security Patch
|
<li><input disabled="" type="checkbox"/>
|
||||||
|
Verify all components after update</li>
|
||||||
```bash
|
<li><input disabled="" type="checkbox"/>
|
||||||
# Quick security update
|
Run integration tests</li>
|
||||||
|
<li><input disabled="" type="checkbox"/>
|
||||||
|
Monitor for issues (30 minutes minimum)</li>
|
||||||
|
<li><input disabled="" type="checkbox"/>
|
||||||
|
Document any issues encountered</li>
|
||||||
|
<li><input disabled="" type="checkbox"/>
|
||||||
|
Close maintenance window</li>
|
||||||
|
</ul>
|
||||||
|
<h2 id="common-update-scenarios"><a class="header" href="#common-update-scenarios">Common Update Scenarios</a></h2>
|
||||||
|
<h3 id="scenario-1-minor-security-patch"><a class="header" href="#scenario-1-minor-security-patch">Scenario 1: Minor Security Patch</a></h3>
|
||||||
|
<pre><code class="language-bash"># Quick security update
|
||||||
provisioning t check-updates --security-only
|
provisioning t check-updates --security-only
|
||||||
provisioning t update --infra my-production --security-patches --yes
|
provisioning t update --infra my-production --security-patches --yes
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="scenario-2-major-version-upgrade"><a class="header" href="#scenario-2-major-version-upgrade">Scenario 2: Major Version Upgrade</a></h3>
|
||||||
### Scenario 2: Major Version Upgrade
|
<pre><code class="language-bash"># Careful major version update
|
||||||
|
|
||||||
```bash
|
|
||||||
# Careful major version update
|
|
||||||
provisioning ws backup my-production
|
provisioning ws backup my-production
|
||||||
provisioning t check-migration <service> --from X.Y --to X+1.Y
|
provisioning t check-migration <service> --from X.Y --to X+1.Y
|
||||||
provisioning t create <service> --infra my-production --migrate
|
provisioning t create <service> --infra my-production --migrate
|
||||||
provisioning test all --infra my-production
|
provisioning test all --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="scenario-3-emergency-hotfix"><a class="header" href="#scenario-3-emergency-hotfix">Scenario 3: Emergency Hotfix</a></h3>
|
||||||
### Scenario 3: Emergency Hotfix
|
<pre><code class="language-bash"># Apply critical hotfix immediately
|
||||||
|
|
||||||
```bash
|
|
||||||
# Apply critical hotfix immediately
|
|
||||||
provisioning t create <service> --infra my-production --hotfix --yes
|
provisioning t create <service> --infra my-production --hotfix --yes
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="troubleshooting-updates"><a class="header" href="#troubleshooting-updates">Troubleshooting Updates</a></h2>
|
||||||
## Troubleshooting Updates
|
<h3 id="issue-update-fails-mid-process"><a class="header" href="#issue-update-fails-mid-process">Issue: Update fails mid-process</a></h3>
|
||||||
|
<p><strong>Solution:</strong></p>
|
||||||
### Issue: Update fails mid-process
|
<pre><code class="language-bash"># Check update status
|
||||||
|
|
||||||
**Solution:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check update status
|
|
||||||
provisioning t status <taskserv> --infra my-production
|
provisioning t status <taskserv> --infra my-production
|
||||||
|
|
||||||
# Resume failed update
|
# Resume failed update
|
||||||
@ -945,14 +759,10 @@ provisioning t update <taskserv> --infra my-production --resume
|
|||||||
|
|
||||||
# Or rollback
|
# Or rollback
|
||||||
provisioning t rollback <taskserv> --infra my-production
|
provisioning t rollback <taskserv> --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="issue-service-not-starting-after-update"><a class="header" href="#issue-service-not-starting-after-update">Issue: Service not starting after update</a></h3>
|
||||||
### Issue: Service not starting after update
|
<p><strong>Solution:</strong></p>
|
||||||
|
<pre><code class="language-bash"># Check logs
|
||||||
**Solution:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check logs
|
|
||||||
provisioning logs <taskserv> --infra my-production
|
provisioning logs <taskserv> --infra my-production
|
||||||
|
|
||||||
# Verify configuration
|
# Verify configuration
|
||||||
@ -960,41 +770,34 @@ provisioning t validate <taskserv> --infra my-production
|
|||||||
|
|
||||||
# Rollback if necessary
|
# Rollback if necessary
|
||||||
provisioning t rollback <taskserv> --infra my-production
|
provisioning t rollback <taskserv> --infra my-production
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h3 id="issue-data-migration-fails"><a class="header" href="#issue-data-migration-fails">Issue: Data migration fails</a></h3>
|
||||||
### Issue: Data migration fails
|
<p><strong>Solution:</strong></p>
|
||||||
|
<pre><code class="language-bash"># Check migration logs
|
||||||
**Solution:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check migration logs
|
|
||||||
provisioning t migration-logs <taskserv> --infra my-production
|
provisioning t migration-logs <taskserv> --infra my-production
|
||||||
|
|
||||||
# Restore from backup
|
# Restore from backup
|
||||||
provisioning t restore <taskserv> --infra my-production --from <backup-file>
|
provisioning t restore <taskserv> --infra my-production --from <backup-file>
|
||||||
```plaintext
|
</code></pre>
|
||||||
|
<h2 id="best-practices"><a class="header" href="#best-practices">Best Practices</a></h2>
|
||||||
## Best Practices
|
<ol>
|
||||||
|
<li><strong>Always Test First</strong>: Test updates in staging before production</li>
|
||||||
1. **Always Test First**: Test updates in staging before production
|
<li><strong>Backup Everything</strong>: Create backups before any update</li>
|
||||||
2. **Backup Everything**: Create backups before any update
|
<li><strong>Update Gradually</strong>: Update one service at a time</li>
|
||||||
3. **Update Gradually**: Update one service at a time
|
<li><strong>Monitor Closely</strong>: Watch for errors after each update</li>
|
||||||
4. **Monitor Closely**: Watch for errors after each update
|
<li><strong>Have Rollback Plan</strong>: Always have a rollback strategy</li>
|
||||||
5. **Have Rollback Plan**: Always have a rollback strategy
|
<li><strong>Document Changes</strong>: Keep update logs for reference</li>
|
||||||
6. **Document Changes**: Keep update logs for reference
|
<li><strong>Schedule Wisely</strong>: Update during low-traffic periods</li>
|
||||||
7. **Schedule Wisely**: Update during low-traffic periods
|
<li><strong>Verify Thoroughly</strong>: Run tests after each update</li>
|
||||||
8. **Verify Thoroughly**: Run tests after each update
|
</ol>
|
||||||
|
<h2 id="next-steps"><a class="header" href="#next-steps">Next Steps</a></h2>
|
||||||
## Next Steps
|
<ul>
|
||||||
|
<li><strong><a href="customize-infrastructure.html">Customize Guide</a></strong> - Customize your infrastructure</li>
|
||||||
- **[Customize Guide](customize-infrastructure.md)** - Customize your infrastructure
|
<li><strong><a href="from-scratch.html">From Scratch Guide</a></strong> - Deploy new infrastructure</li>
|
||||||
- **[From Scratch Guide](from-scratch.md)** - Deploy new infrastructure
|
<li><strong><a href="../development/workflow.html">Workflow Guide</a></strong> - Automate with workflows</li>
|
||||||
- **[Workflow Guide](../development/workflow.md)** - Automate with workflows
|
</ul>
|
||||||
|
<h2 id="quick-reference"><a class="header" href="#quick-reference">Quick Reference</a></h2>
|
||||||
## Quick Reference
|
<pre><code class="language-bash"># Update workflow
|
||||||
|
|
||||||
```bash
|
|
||||||
# Update workflow
|
|
||||||
provisioning t check-updates
|
provisioning t check-updates
|
||||||
provisioning ws backup my-production
|
provisioning ws backup my-production
|
||||||
provisioning t create <taskserv> --infra my-production --check
|
provisioning t create <taskserv> --infra my-production --check
|
||||||
@ -1002,12 +805,9 @@ provisioning t create <taskserv> --infra my-production
|
|||||||
provisioning version taskserv <taskserv>
|
provisioning version taskserv <taskserv>
|
||||||
provisioning health --infra my-production
|
provisioning health --infra my-production
|
||||||
provisioning test all --infra my-production
|
provisioning test all --infra my-production
|
||||||
```plaintext
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
*This guide is part of the provisioning project documentation. Last updated: 2025-09-30*
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
|
<hr />
|
||||||
|
<p><em>This guide is part of the provisioning project documentation. Last updated: 2025-09-30</em></p>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
|
|||||||
@ -314,229 +314,202 @@
|
|||||||
├── configuration/ # Configuration docs
|
├── configuration/ # Configuration docs
|
||||||
├── troubleshooting/ # Troubleshooting guides
|
├── troubleshooting/ # Troubleshooting guides
|
||||||
└── quick-reference/ # Quick references
|
└── quick-reference/ # Quick references
|
||||||
```plaintext
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Key Concepts
|
|
||||||
|
|
||||||
### Infrastructure as Code (IaC)
|
|
||||||
|
|
||||||
The provisioning platform uses **declarative configuration** to manage infrastructure. Instead of manually creating resources, you define what you want in Nickel configuration files, and the system makes it happen.
|
|
||||||
|
|
||||||
### Mode-Based Architecture
|
|
||||||
|
|
||||||
The system supports four operational modes:
|
|
||||||
|
|
||||||
- **Solo**: Single developer local development
|
|
||||||
- **Multi-user**: Team collaboration with shared services
|
|
||||||
- **CI/CD**: Automated pipeline execution
|
|
||||||
- **Enterprise**: Production deployment with strict compliance
|
|
||||||
|
|
||||||
### Extension System
|
|
||||||
|
|
||||||
Extensibility through:
|
|
||||||
|
|
||||||
- **Providers**: Cloud platform integrations (AWS, UpCloud, Local)
|
|
||||||
- **Task Services**: Infrastructure components (Kubernetes, databases, etc.)
|
|
||||||
- **Clusters**: Complete deployment configurations
|
|
||||||
|
|
||||||
### OCI-Native Distribution
|
|
||||||
|
|
||||||
Extensions and packages distributed as OCI artifacts, enabling:
|
|
||||||
|
|
||||||
- Industry-standard packaging
|
|
||||||
- Efficient caching and bandwidth
|
|
||||||
- Version pinning and rollback
|
|
||||||
- Air-gapped deployments
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Documentation by Role
|
|
||||||
|
|
||||||
### For New Users
|
|
||||||
|
|
||||||
1. Start with **[Installation Guide](getting-started/installation-guide.md)**
|
|
||||||
2. Read **[Getting Started](getting-started/getting-started.md)**
|
|
||||||
3. Follow **[From Scratch Guide](guides/from-scratch.md)**
|
|
||||||
4. Reference **[Quickstart Cheatsheet](guides/quickstart-cheatsheet.md)**
|
|
||||||
|
|
||||||
### For Developers
|
|
||||||
|
|
||||||
1. Review **[System Overview](architecture/system-overview.md)**
|
|
||||||
2. Study **[Design Principles](architecture/design-principles.md)**
|
|
||||||
3. Read relevant **[ADRs](architecture/)**
|
|
||||||
4. Follow **[Development Guide](development/README.md)**
|
|
||||||
5. Reference **KCL Quick Reference**
|
|
||||||
|
|
||||||
### For Operators
|
|
||||||
|
|
||||||
1. Understand **[Mode System](infrastructure/mode-system)**
|
|
||||||
2. Learn **[Service Management](operations/service-management-guide.md)**
|
|
||||||
3. Review **[Infrastructure Management](infrastructure/infrastructure-management.md)**
|
|
||||||
4. Study **[OCI Registry](integration/oci-registry-guide.md)**
|
|
||||||
|
|
||||||
### For Architects
|
|
||||||
|
|
||||||
1. Read **[System Overview](architecture/system-overview.md)**
|
|
||||||
2. Study all **[ADRs](architecture/)**
|
|
||||||
3. Review **[Integration Patterns](architecture/integration-patterns.md)**
|
|
||||||
4. Understand **[Multi-Repo Architecture](architecture/multi-repo-architecture.md)**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## System Capabilities
|
|
||||||
|
|
||||||
### ✅ Infrastructure Automation
|
|
||||||
|
|
||||||
- Multi-cloud support (AWS, UpCloud, Local)
|
|
||||||
- Declarative configuration with KCL
|
|
||||||
- Automated dependency resolution
|
|
||||||
- Batch operations with rollback
|
|
||||||
|
|
||||||
### ✅ Workflow Orchestration
|
|
||||||
|
|
||||||
- Hybrid Rust/Nushell orchestration
|
|
||||||
- Checkpoint-based recovery
|
|
||||||
- Parallel execution with limits
|
|
||||||
- Real-time monitoring
|
|
||||||
|
|
||||||
### ✅ Test Environments
|
|
||||||
|
|
||||||
- Containerized testing
|
|
||||||
- Multi-node cluster simulation
|
|
||||||
- Topology templates
|
|
||||||
- Automated cleanup
|
|
||||||
|
|
||||||
### ✅ Mode-Based Operation
|
|
||||||
|
|
||||||
- Solo: Local development
|
|
||||||
- Multi-user: Team collaboration
|
|
||||||
- CI/CD: Automated pipelines
|
|
||||||
- Enterprise: Production deployment
|
|
||||||
|
|
||||||
### ✅ Extension Management
|
|
||||||
|
|
||||||
- OCI-native distribution
|
|
||||||
- Automatic dependency resolution
|
|
||||||
- Version management
|
|
||||||
- Local and remote sources
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Key Achievements
|
|
||||||
|
|
||||||
### 🚀 Batch Workflow System (v3.1.0)
|
|
||||||
|
|
||||||
- Provider-agnostic batch operations
|
|
||||||
- Mixed provider support (UpCloud + AWS + local)
|
|
||||||
- Dependency resolution with soft/hard dependencies
|
|
||||||
- Real-time monitoring and rollback
|
|
||||||
|
|
||||||
### 🏗️ Hybrid Orchestrator (v3.0.0)
|
|
||||||
|
|
||||||
- Solves Nushell deep call stack limitations
|
|
||||||
- Preserves all business logic
|
|
||||||
- REST API for external integration
|
|
||||||
- Checkpoint-based state management
|
|
||||||
|
|
||||||
### ⚙️ Configuration System (v2.0.0)
|
|
||||||
|
|
||||||
- Migrated from ENV to config-driven
|
|
||||||
- Hierarchical configuration loading
|
|
||||||
- Variable interpolation
|
|
||||||
- True IaC without hardcoded fallbacks
|
|
||||||
|
|
||||||
### 🎯 Modular CLI (v3.2.0)
|
|
||||||
|
|
||||||
- 84% reduction in main file size
|
|
||||||
- Domain-driven handlers
|
|
||||||
- 80+ shortcuts
|
|
||||||
- Bi-directional help system
|
|
||||||
|
|
||||||
### 🧪 Test Environment Service (v3.4.0)
|
|
||||||
|
|
||||||
- Automated containerized testing
|
|
||||||
- Multi-node cluster topologies
|
|
||||||
- CI/CD integration ready
|
|
||||||
- Template-based configurations
|
|
||||||
|
|
||||||
### 🔄 Workspace Switching (v2.0.5)
|
|
||||||
|
|
||||||
- Centralized workspace management
|
|
||||||
- Single-command workspace switching
|
|
||||||
- Active workspace tracking
|
|
||||||
- User preference system
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Technology Stack
|
|
||||||
|
|
||||||
| Component | Technology | Purpose |
|
|
||||||
|-----------|------------|---------|
|
|
||||||
| **Core CLI** | Nushell 0.107.1 | Shell and scripting |
|
|
||||||
| **Configuration** | KCL 0.11.2 | Type-safe IaC |
|
|
||||||
| **Orchestrator** | Rust | High-performance coordination |
|
|
||||||
| **Templates** | Jinja2 (nu_plugin_tera) | Code generation |
|
|
||||||
| **Secrets** | SOPS 3.10.2 + Age 1.2.1 | Encryption |
|
|
||||||
| **Distribution** | OCI (skopeo/crane/oras) | Artifact management |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Support
|
|
||||||
|
|
||||||
### Getting Help
|
|
||||||
|
|
||||||
- **Documentation**: You're reading it!
|
|
||||||
- **Quick Reference**: Run `provisioning sc` or `provisioning guide quickstart`
|
|
||||||
- **Help System**: Run `provisioning help` or `provisioning <command> help`
|
|
||||||
- **Interactive Shell**: Run `provisioning nu` for Nushell REPL
|
|
||||||
|
|
||||||
### Reporting Issues
|
|
||||||
|
|
||||||
- Check **[Troubleshooting Guide](infrastructure/troubleshooting-guide.md)**
|
|
||||||
- Review **[FAQ](troubleshooting/troubleshooting-guide.md)**
|
|
||||||
- Enable debug mode: `provisioning --debug <command>`
|
|
||||||
- Check logs: `provisioning platform logs <service>`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Contributing
|
|
||||||
|
|
||||||
This project welcomes contributions! See **[Development Guide](development/README.md)** for:
|
|
||||||
|
|
||||||
- Development setup
|
|
||||||
- Code style guidelines
|
|
||||||
- Testing requirements
|
|
||||||
- Pull request process
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
[Add license information]
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Version History
|
|
||||||
|
|
||||||
| Version | Date | Major Changes |
|
|
||||||
|---------|------|---------------|
|
|
||||||
| **3.5.0** | 2025-10-06 | Mode system, OCI registry, comprehensive documentation |
|
|
||||||
| **3.4.0** | 2025-10-06 | Test environment service |
|
|
||||||
| **3.3.0** | 2025-09-30 | Interactive guides system |
|
|
||||||
| **3.2.0** | 2025-09-30 | Modular CLI refactoring |
|
|
||||||
| **3.1.0** | 2025-09-25 | Batch workflow system |
|
|
||||||
| **3.0.0** | 2025-09-25 | Hybrid orchestrator architecture |
|
|
||||||
| **2.0.5** | 2025-10-02 | Workspace switching system |
|
|
||||||
| **2.0.0** | 2025-09-23 | Configuration system migration |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Maintained By**: Provisioning Team
|
|
||||||
**Last Review**: 2025-10-06
|
|
||||||
**Next Review**: 2026-01-06
|
|
||||||
</code></pre>
|
</code></pre>
|
||||||
|
<hr />
|
||||||
|
<h2 id="key-concepts"><a class="header" href="#key-concepts">Key Concepts</a></h2>
|
||||||
|
<h3 id="infrastructure-as-code-iac"><a class="header" href="#infrastructure-as-code-iac">Infrastructure as Code (IaC)</a></h3>
|
||||||
|
<p>The provisioning platform uses <strong>declarative configuration</strong> to manage infrastructure. Instead of manually creating resources, you define what you want in Nickel configuration files, and the system makes it happen.</p>
|
||||||
|
<h3 id="mode-based-architecture"><a class="header" href="#mode-based-architecture">Mode-Based Architecture</a></h3>
|
||||||
|
<p>The system supports four operational modes:</p>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Solo</strong>: Single developer local development</li>
|
||||||
|
<li><strong>Multi-user</strong>: Team collaboration with shared services</li>
|
||||||
|
<li><strong>CI/CD</strong>: Automated pipeline execution</li>
|
||||||
|
<li><strong>Enterprise</strong>: Production deployment with strict compliance</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="extension-system"><a class="header" href="#extension-system">Extension System</a></h3>
|
||||||
|
<p>Extensibility through:</p>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Providers</strong>: Cloud platform integrations (AWS, UpCloud, Local)</li>
|
||||||
|
<li><strong>Task Services</strong>: Infrastructure components (Kubernetes, databases, etc.)</li>
|
||||||
|
<li><strong>Clusters</strong>: Complete deployment configurations</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="oci-native-distribution"><a class="header" href="#oci-native-distribution">OCI-Native Distribution</a></h3>
|
||||||
|
<p>Extensions and packages distributed as OCI artifacts, enabling:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Industry-standard packaging</li>
|
||||||
|
<li>Efficient caching and bandwidth</li>
|
||||||
|
<li>Version pinning and rollback</li>
|
||||||
|
<li>Air-gapped deployments</li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="documentation-by-role"><a class="header" href="#documentation-by-role">Documentation by Role</a></h2>
|
||||||
|
<h3 id="for-new-users"><a class="header" href="#for-new-users">For New Users</a></h3>
|
||||||
|
<ol>
|
||||||
|
<li>Start with <strong><a href="getting-started/installation-guide.html">Installation Guide</a></strong></li>
|
||||||
|
<li>Read <strong><a href="getting-started/getting-started.html">Getting Started</a></strong></li>
|
||||||
|
<li>Follow <strong><a href="guides/from-scratch.html">From Scratch Guide</a></strong></li>
|
||||||
|
<li>Reference <strong><a href="guides/quickstart-cheatsheet.html">Quickstart Cheatsheet</a></strong></li>
|
||||||
|
</ol>
|
||||||
|
<h3 id="for-developers"><a class="header" href="#for-developers">For Developers</a></h3>
|
||||||
|
<ol>
|
||||||
|
<li>Review <strong><a href="architecture/system-overview.html">System Overview</a></strong></li>
|
||||||
|
<li>Study <strong><a href="architecture/design-principles.html">Design Principles</a></strong></li>
|
||||||
|
<li>Read relevant <strong><a href="architecture/">ADRs</a></strong></li>
|
||||||
|
<li>Follow <strong><a href="development/README.html">Development Guide</a></strong></li>
|
||||||
|
<li>Reference <strong>KCL Quick Reference</strong></li>
|
||||||
|
</ol>
|
||||||
|
<h3 id="for-operators"><a class="header" href="#for-operators">For Operators</a></h3>
|
||||||
|
<ol>
|
||||||
|
<li>Understand <strong><a href="infrastructure/mode-system">Mode System</a></strong></li>
|
||||||
|
<li>Learn <strong><a href="operations/service-management-guide.html">Service Management</a></strong></li>
|
||||||
|
<li>Review <strong><a href="infrastructure/infrastructure-management.html">Infrastructure Management</a></strong></li>
|
||||||
|
<li>Study <strong><a href="integration/oci-registry-guide.html">OCI Registry</a></strong></li>
|
||||||
|
</ol>
|
||||||
|
<h3 id="for-architects"><a class="header" href="#for-architects">For Architects</a></h3>
|
||||||
|
<ol>
|
||||||
|
<li>Read <strong><a href="architecture/system-overview.html">System Overview</a></strong></li>
|
||||||
|
<li>Study all <strong><a href="architecture/">ADRs</a></strong></li>
|
||||||
|
<li>Review <strong><a href="architecture/integration-patterns.html">Integration Patterns</a></strong></li>
|
||||||
|
<li>Understand <strong><a href="architecture/multi-repo-architecture.html">Multi-Repo Architecture</a></strong></li>
|
||||||
|
</ol>
|
||||||
|
<hr />
|
||||||
|
<h2 id="system-capabilities"><a class="header" href="#system-capabilities">System Capabilities</a></h2>
|
||||||
|
<h3 id="-infrastructure-automation"><a class="header" href="#-infrastructure-automation">✅ Infrastructure Automation</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Multi-cloud support (AWS, UpCloud, Local)</li>
|
||||||
|
<li>Declarative configuration with KCL</li>
|
||||||
|
<li>Automated dependency resolution</li>
|
||||||
|
<li>Batch operations with rollback</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-workflow-orchestration"><a class="header" href="#-workflow-orchestration">✅ Workflow Orchestration</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Hybrid Rust/Nushell orchestration</li>
|
||||||
|
<li>Checkpoint-based recovery</li>
|
||||||
|
<li>Parallel execution with limits</li>
|
||||||
|
<li>Real-time monitoring</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-test-environments"><a class="header" href="#-test-environments">✅ Test Environments</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Containerized testing</li>
|
||||||
|
<li>Multi-node cluster simulation</li>
|
||||||
|
<li>Topology templates</li>
|
||||||
|
<li>Automated cleanup</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-mode-based-operation"><a class="header" href="#-mode-based-operation">✅ Mode-Based Operation</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Solo: Local development</li>
|
||||||
|
<li>Multi-user: Team collaboration</li>
|
||||||
|
<li>CI/CD: Automated pipelines</li>
|
||||||
|
<li>Enterprise: Production deployment</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-extension-management"><a class="header" href="#-extension-management">✅ Extension Management</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>OCI-native distribution</li>
|
||||||
|
<li>Automatic dependency resolution</li>
|
||||||
|
<li>Version management</li>
|
||||||
|
<li>Local and remote sources</li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="key-achievements"><a class="header" href="#key-achievements">Key Achievements</a></h2>
|
||||||
|
<h3 id="-batch-workflow-system-v310"><a class="header" href="#-batch-workflow-system-v310">🚀 Batch Workflow System (v3.1.0)</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Provider-agnostic batch operations</li>
|
||||||
|
<li>Mixed provider support (UpCloud + AWS + local)</li>
|
||||||
|
<li>Dependency resolution with soft/hard dependencies</li>
|
||||||
|
<li>Real-time monitoring and rollback</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-hybrid-orchestrator-v300"><a class="header" href="#-hybrid-orchestrator-v300">🏗️ Hybrid Orchestrator (v3.0.0)</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Solves Nushell deep call stack limitations</li>
|
||||||
|
<li>Preserves all business logic</li>
|
||||||
|
<li>REST API for external integration</li>
|
||||||
|
<li>Checkpoint-based state management</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-configuration-system-v200"><a class="header" href="#-configuration-system-v200">⚙️ Configuration System (v2.0.0)</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Migrated from ENV to config-driven</li>
|
||||||
|
<li>Hierarchical configuration loading</li>
|
||||||
|
<li>Variable interpolation</li>
|
||||||
|
<li>True IaC without hardcoded fallbacks</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-modular-cli-v320"><a class="header" href="#-modular-cli-v320">🎯 Modular CLI (v3.2.0)</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>84% reduction in main file size</li>
|
||||||
|
<li>Domain-driven handlers</li>
|
||||||
|
<li>80+ shortcuts</li>
|
||||||
|
<li>Bi-directional help system</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-test-environment-service-v340"><a class="header" href="#-test-environment-service-v340">🧪 Test Environment Service (v3.4.0)</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Automated containerized testing</li>
|
||||||
|
<li>Multi-node cluster topologies</li>
|
||||||
|
<li>CI/CD integration ready</li>
|
||||||
|
<li>Template-based configurations</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="-workspace-switching-v205"><a class="header" href="#-workspace-switching-v205">🔄 Workspace Switching (v2.0.5)</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Centralized workspace management</li>
|
||||||
|
<li>Single-command workspace switching</li>
|
||||||
|
<li>Active workspace tracking</li>
|
||||||
|
<li>User preference system</li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="technology-stack"><a class="header" href="#technology-stack">Technology Stack</a></h2>
|
||||||
|
<div class="table-wrapper"><table><thead><tr><th>Component</th><th>Technology</th><th>Purpose</th></tr></thead><tbody>
|
||||||
|
<tr><td><strong>Core CLI</strong></td><td>Nushell 0.107.1</td><td>Shell and scripting</td></tr>
|
||||||
|
<tr><td><strong>Configuration</strong></td><td>KCL 0.11.2</td><td>Type-safe IaC</td></tr>
|
||||||
|
<tr><td><strong>Orchestrator</strong></td><td>Rust</td><td>High-performance coordination</td></tr>
|
||||||
|
<tr><td><strong>Templates</strong></td><td>Jinja2 (nu_plugin_tera)</td><td>Code generation</td></tr>
|
||||||
|
<tr><td><strong>Secrets</strong></td><td>SOPS 3.10.2 + Age 1.2.1</td><td>Encryption</td></tr>
|
||||||
|
<tr><td><strong>Distribution</strong></td><td>OCI (skopeo/crane/oras)</td><td>Artifact management</td></tr>
|
||||||
|
</tbody></table>
|
||||||
|
</div>
|
||||||
|
<hr />
|
||||||
|
<h2 id="support"><a class="header" href="#support">Support</a></h2>
|
||||||
|
<h3 id="getting-help"><a class="header" href="#getting-help">Getting Help</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li><strong>Documentation</strong>: You’re reading it!</li>
|
||||||
|
<li><strong>Quick Reference</strong>: Run <code>provisioning sc</code> or <code>provisioning guide quickstart</code></li>
|
||||||
|
<li><strong>Help System</strong>: Run <code>provisioning help</code> or <code>provisioning <command> help</code></li>
|
||||||
|
<li><strong>Interactive Shell</strong>: Run <code>provisioning nu</code> for Nushell REPL</li>
|
||||||
|
</ul>
|
||||||
|
<h3 id="reporting-issues"><a class="header" href="#reporting-issues">Reporting Issues</a></h3>
|
||||||
|
<ul>
|
||||||
|
<li>Check <strong><a href="infrastructure/troubleshooting-guide.html">Troubleshooting Guide</a></strong></li>
|
||||||
|
<li>Review <strong><a href="troubleshooting/troubleshooting-guide.html">FAQ</a></strong></li>
|
||||||
|
<li>Enable debug mode: <code>provisioning --debug <command></code></li>
|
||||||
|
<li>Check logs: <code>provisioning platform logs <service></code></li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="contributing"><a class="header" href="#contributing">Contributing</a></h2>
|
||||||
|
<p>This project welcomes contributions! See <strong><a href="development/README.html">Development Guide</a></strong> for:</p>
|
||||||
|
<ul>
|
||||||
|
<li>Development setup</li>
|
||||||
|
<li>Code style guidelines</li>
|
||||||
|
<li>Testing requirements</li>
|
||||||
|
<li>Pull request process</li>
|
||||||
|
</ul>
|
||||||
|
<hr />
|
||||||
|
<h2 id="license"><a class="header" href="#license">License</a></h2>
|
||||||
|
<p>[Add license information]</p>
|
||||||
|
<hr />
|
||||||
|
<h2 id="version-history"><a class="header" href="#version-history">Version History</a></h2>
|
||||||
|
<div class="table-wrapper"><table><thead><tr><th>Version</th><th>Date</th><th>Major Changes</th></tr></thead><tbody>
|
||||||
|
<tr><td><strong>3.5.0</strong></td><td>2025-10-06</td><td>Mode system, OCI registry, comprehensive documentation</td></tr>
|
||||||
|
<tr><td><strong>3.4.0</strong></td><td>2025-10-06</td><td>Test environment service</td></tr>
|
||||||
|
<tr><td><strong>3.3.0</strong></td><td>2025-09-30</td><td>Interactive guides system</td></tr>
|
||||||
|
<tr><td><strong>3.2.0</strong></td><td>2025-09-30</td><td>Modular CLI refactoring</td></tr>
|
||||||
|
<tr><td><strong>3.1.0</strong></td><td>2025-09-25</td><td>Batch workflow system</td></tr>
|
||||||
|
<tr><td><strong>3.0.0</strong></td><td>2025-09-25</td><td>Hybrid orchestrator architecture</td></tr>
|
||||||
|
<tr><td><strong>2.0.5</strong></td><td>2025-10-02</td><td>Workspace switching system</td></tr>
|
||||||
|
<tr><td><strong>2.0.0</strong></td><td>2025-09-23</td><td>Configuration system migration</td></tr>
|
||||||
|
</tbody></table>
|
||||||
|
</div>
|
||||||
|
<hr />
|
||||||
|
<p><strong>Maintained By</strong>: Provisioning Team
|
||||||
|
<strong>Last Review</strong>: 2025-10-06
|
||||||
|
<strong>Next Review</strong>: 2026-01-06</p>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
|
|||||||
61149
docs/book/print.html
61149
docs/book/print.html
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
1
docs/examples/workspaces/cost-optimized/README.md
Normal file
1
docs/examples/workspaces/cost-optimized/README.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Cost-Optimized Multi-Provider Workspace
|
||||||
@ -3,7 +3,7 @@
|
|||||||
<head>
|
<head>
|
||||||
<!-- Book generated using mdBook -->
|
<!-- Book generated using mdBook -->
|
||||||
<meta charset="UTF-8">
|
<meta charset="UTF-8">
|
||||||
<title>Orchestrator Info - Provisioning Platform Documentation</title>
|
<title>Cost-Optimized Multi-Provider Workspace - Provisioning Platform Documentation</title>
|
||||||
|
|
||||||
|
|
||||||
<!-- Custom HTML head -->
|
<!-- Custom HTML head -->
|
||||||
@ -12,33 +12,33 @@
|
|||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
<meta name="theme-color" content="#ffffff">
|
<meta name="theme-color" content="#ffffff">
|
||||||
|
|
||||||
<link rel="icon" href="../favicon.svg">
|
<link rel="icon" href="../../../favicon.svg">
|
||||||
<link rel="shortcut icon" href="../favicon.png">
|
<link rel="shortcut icon" href="../../../favicon.png">
|
||||||
<link rel="stylesheet" href="../css/variables.css">
|
<link rel="stylesheet" href="../../../css/variables.css">
|
||||||
<link rel="stylesheet" href="../css/general.css">
|
<link rel="stylesheet" href="../../../css/general.css">
|
||||||
<link rel="stylesheet" href="../css/chrome.css">
|
<link rel="stylesheet" href="../../../css/chrome.css">
|
||||||
<link rel="stylesheet" href="../css/print.css" media="print">
|
<link rel="stylesheet" href="../../../css/print.css" media="print">
|
||||||
|
|
||||||
<!-- Fonts -->
|
<!-- Fonts -->
|
||||||
<link rel="stylesheet" href="../FontAwesome/css/font-awesome.css">
|
<link rel="stylesheet" href="../../../FontAwesome/css/font-awesome.css">
|
||||||
<link rel="stylesheet" href="../fonts/fonts.css">
|
<link rel="stylesheet" href="../../../fonts/fonts.css">
|
||||||
|
|
||||||
<!-- Highlight.js Stylesheets -->
|
<!-- Highlight.js Stylesheets -->
|
||||||
<link rel="stylesheet" id="highlight-css" href="../highlight.css">
|
<link rel="stylesheet" id="highlight-css" href="../../../highlight.css">
|
||||||
<link rel="stylesheet" id="tomorrow-night-css" href="../tomorrow-night.css">
|
<link rel="stylesheet" id="tomorrow-night-css" href="../../../tomorrow-night.css">
|
||||||
<link rel="stylesheet" id="ayu-highlight-css" href="../ayu-highlight.css">
|
<link rel="stylesheet" id="ayu-highlight-css" href="../../../ayu-highlight.css">
|
||||||
|
|
||||||
<!-- Custom theme stylesheets -->
|
<!-- Custom theme stylesheets -->
|
||||||
|
|
||||||
|
|
||||||
<!-- Provide site root and default themes to javascript -->
|
<!-- Provide site root and default themes to javascript -->
|
||||||
<script>
|
<script>
|
||||||
const path_to_root = "../";
|
const path_to_root = "../../../";
|
||||||
const default_light_theme = "ayu";
|
const default_light_theme = "ayu";
|
||||||
const default_dark_theme = "navy";
|
const default_dark_theme = "navy";
|
||||||
</script>
|
</script>
|
||||||
<!-- Start loading toc.js asap -->
|
<!-- Start loading toc.js asap -->
|
||||||
<script src="../toc.js"></script>
|
<script src="../../../toc.js"></script>
|
||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
<div id="mdbook-help-container">
|
<div id="mdbook-help-container">
|
||||||
@ -102,7 +102,7 @@
|
|||||||
<!-- populated by js -->
|
<!-- populated by js -->
|
||||||
<mdbook-sidebar-scrollbox class="sidebar-scrollbox"></mdbook-sidebar-scrollbox>
|
<mdbook-sidebar-scrollbox class="sidebar-scrollbox"></mdbook-sidebar-scrollbox>
|
||||||
<noscript>
|
<noscript>
|
||||||
<iframe class="sidebar-iframe-outer" src="../toc.html"></iframe>
|
<iframe class="sidebar-iframe-outer" src="../../../toc.html"></iframe>
|
||||||
</noscript>
|
</noscript>
|
||||||
<div id="sidebar-resize-handle" class="sidebar-resize-handle">
|
<div id="sidebar-resize-handle" class="sidebar-resize-handle">
|
||||||
<div class="sidebar-resize-indicator"></div>
|
<div class="sidebar-resize-indicator"></div>
|
||||||
@ -137,13 +137,13 @@
|
|||||||
<h1 class="menu-title">Provisioning Platform Documentation</h1>
|
<h1 class="menu-title">Provisioning Platform Documentation</h1>
|
||||||
|
|
||||||
<div class="right-buttons">
|
<div class="right-buttons">
|
||||||
<a href="../print.html" title="Print this book" aria-label="Print this book">
|
<a href="../../../print.html" title="Print this book" aria-label="Print this book">
|
||||||
<i id="print-button" class="fa fa-print"></i>
|
<i id="print-button" class="fa fa-print"></i>
|
||||||
</a>
|
</a>
|
||||||
<a href="https://github.com/provisioning/provisioning-platform" title="Git repository" aria-label="Git repository">
|
<a href="https://github.com/provisioning/provisioning-platform" title="Git repository" aria-label="Git repository">
|
||||||
<i id="git-repository-button" class="fa fa-github"></i>
|
<i id="git-repository-button" class="fa fa-github"></i>
|
||||||
</a>
|
</a>
|
||||||
<a href="https://github.com/provisioning/provisioning-platform/edit/main/provisioning/docs/src/architecture/orchestrator_info.md" title="Suggest an edit" aria-label="Suggest an edit">
|
<a href="https://github.com/provisioning/provisioning-platform/edit/main/provisioning/docs/src/../examples/workspaces/cost-optimized/README.md" title="Suggest an edit" aria-label="Suggest an edit">
|
||||||
<i id="git-edit-button" class="fa fa-edit"></i>
|
<i id="git-edit-button" class="fa fa-edit"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
@ -172,130 +172,17 @@
|
|||||||
|
|
||||||
<div id="content" class="content">
|
<div id="content" class="content">
|
||||||
<main>
|
<main>
|
||||||
<p>Execution Complete</p>
|
<h1 id="cost-optimized-multi-provider-workspace"><a class="header" href="#cost-optimized-multi-provider-workspace">Cost-Optimized Multi-Provider Workspace</a></h1>
|
||||||
<p>Perfect question! Let me explain clearly:</p>
|
|
||||||
<p>The Orchestrator IS USED and IS CRITICAL</p>
|
|
||||||
<p>That code example was misleading. Here’s the real architecture:</p>
|
|
||||||
<p>How It Actually Works</p>
|
|
||||||
<p>┌──────────────────────────────────────────────────┐
|
|
||||||
│ User runs: provisioning server create –orchestrated
|
|
||||||
└───────────────────┬──────────────────────────────┘
|
|
||||||
↓
|
|
||||||
┌───────────────────────┐
|
|
||||||
│ Nushell CLI │
|
|
||||||
│ (provisioning) │
|
|
||||||
└───────────┬───────────┘
|
|
||||||
↓ HTTP POST
|
|
||||||
┌───────────────────────────────┐
|
|
||||||
│ Rust Orchestrator Daemon │
|
|
||||||
│ (provisioning-orchestrator) │
|
|
||||||
│ │
|
|
||||||
│ • Task Queue │
|
|
||||||
│ • Workflow Engine │
|
|
||||||
│ • Dependency Resolution │
|
|
||||||
│ • Parallel Execution │
|
|
||||||
└───────────┬───────────────────┘
|
|
||||||
↓ spawns subprocess
|
|
||||||
┌───────────────────────────────┐
|
|
||||||
│ Nushell Business Logic │
|
|
||||||
│ nu -c “use servers/create.nu”│
|
|
||||||
│ │
|
|
||||||
│ Executes actual provider │
|
|
||||||
│ API calls, configuration │
|
|
||||||
└───────────────────────────────┘
|
|
||||||
The Flow in Detail</p>
|
|
||||||
<ol>
|
|
||||||
<li>User Command:</li>
|
|
||||||
</ol>
|
|
||||||
<p>provisioning server create wuji –orchestrated
|
|
||||||
2. Nushell CLI submits to orchestrator:</p>
|
|
||||||
<h1 id="cli-code"><a class="header" href="#cli-code">CLI code</a></h1>
|
|
||||||
<p>http post <a href="http://localhost:9090/workflows/servers/create">http://localhost:9090/workflows/servers/create</a> {
|
|
||||||
infra: “wuji”
|
|
||||||
params: {…}
|
|
||||||
}</p>
|
|
||||||
<h1 id="returns-workflow_id--abc-123"><a class="header" href="#returns-workflow_id--abc-123">Returns: workflow_id = “abc-123”</a></h1>
|
|
||||||
<ol>
|
|
||||||
<li>Orchestrator receives and queues:</li>
|
|
||||||
</ol>
|
|
||||||
<p>// Orchestrator receives HTTP request
|
|
||||||
async fn create_server_workflow(request) {
|
|
||||||
let task = Task::new(TaskType::ServerCreate, request);
|
|
||||||
task_queue.enqueue(task).await; // Queue for execution
|
|
||||||
return workflow_id; // Return immediately
|
|
||||||
}
|
|
||||||
4. Orchestrator executes via Nushell subprocess:</p>
|
|
||||||
<p>// Orchestrator spawns Nushell to run business logic
|
|
||||||
async fn execute_task(task: Task) {
|
|
||||||
let output = Command::new(“nu”)
|
|
||||||
.arg(“-c”)
|
|
||||||
.arg(“use /usr/local/lib/provisioning/servers/create.nu; create-server ‘wuji’”)
|
|
||||||
.output()
|
|
||||||
.await?;</p>
|
|
||||||
<pre><code>// Orchestrator manages: retry, checkpointing, monitoring
|
|
||||||
</code></pre>
|
|
||||||
<p>}
|
|
||||||
5. Nushell executes the actual work:</p>
|
|
||||||
<h1 id="serverscreatenu"><a class="header" href="#serverscreatenu">servers/create.nu</a></h1>
|
|
||||||
<p>export def create-server [name: string] {
|
|
||||||
# This is the business logic
|
|
||||||
# Calls UpCloud API, creates server, etc.
|
|
||||||
let provider = (load-provider)
|
|
||||||
$provider | create-vm $name
|
|
||||||
}
|
|
||||||
Why This Architecture?</p>
|
|
||||||
<p>Problem It Solves</p>
|
|
||||||
<p>Without Orchestrator (Old Way):</p>
|
|
||||||
<p>provisioning → template.nu → cluster.nu → taskserv.nu → provider.nu
|
|
||||||
(Deep call stack = crashes!)
|
|
||||||
With Orchestrator (Current):</p>
|
|
||||||
<p>provisioning → Orchestrator → spawns fresh Nushell subprocess for each task
|
|
||||||
(No deep nesting, parallel execution, recovery)
|
|
||||||
What Orchestrator Provides</p>
|
|
||||||
<p>Task Queue - Reliable execution even if system crashes
|
|
||||||
Parallel Execution - Run 10 tasks at once (Rust async)
|
|
||||||
Workflow Engine - Handle complex dependencies
|
|
||||||
Checkpointing - Resume from failure
|
|
||||||
Monitoring - Real-time progress tracking
|
|
||||||
What Nushell Provides</p>
|
|
||||||
<p>Business Logic - Provider integrations, config generation
|
|
||||||
Flexibility - Easy to modify without recompiling
|
|
||||||
Readability - Shell-like syntax for infrastructure ops
|
|
||||||
Multi-Repo Impact: NONE on Integration</p>
|
|
||||||
<p>In Monorepo:</p>
|
|
||||||
<p>provisioning/
|
|
||||||
├── core/nulib/ # Nushell code
|
|
||||||
└── platform/orchestrator/ # Rust code
|
|
||||||
In Multi-Repo:</p>
|
|
||||||
<p>provisioning-core/ # Separate repo, installs to /usr/local/lib/provisioning
|
|
||||||
provisioning-platform/ # Separate repo, installs to /usr/local/bin/provisioning-orchestrator
|
|
||||||
Integration is the same:</p>
|
|
||||||
<p>Orchestrator calls: nu -c “use /usr/local/lib/provisioning/servers/create.nu”
|
|
||||||
Nushell calls: http post <a href="http://localhost:9090/workflows/">http://localhost:9090/workflows/</a>…
|
|
||||||
No code dependency, just runtime coordination!</p>
|
|
||||||
<p>The Orchestrator IS Essential</p>
|
|
||||||
<p>The orchestrator:</p>
|
|
||||||
<p>✅ IS USED for all complex operations
|
|
||||||
✅ IS CRITICAL for workflow system (v3.0)
|
|
||||||
✅ IS REQUIRED for batch operations (v3.1)
|
|
||||||
✅ SOLVES deep call stack issues
|
|
||||||
✅ PROVIDES performance and reliability
|
|
||||||
That misleading code example showed how Platform doesn’t link to Core code, but it absolutely uses the orchestrator for coordination.</p>
|
|
||||||
<p>Does this clear it up? The orchestrator is the performance and reliability layer that makes the whole system work!</p>
|
|
||||||
<p>Cost: $0.1565 USD
|
|
||||||
Duration: 137.69s
|
|
||||||
Turns: 40
|
|
||||||
Total tokens: 7466(7 in, 7459 out)</p>
|
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
<nav class="nav-wrapper" aria-label="Page navigation">
|
<nav class="nav-wrapper" aria-label="Page navigation">
|
||||||
<!-- Mobile navigation buttons -->
|
<!-- Mobile navigation buttons -->
|
||||||
<a rel="prev" href="../architecture/nickel-executable-examples.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
<a rel="prev" href="../../../../examples/workspaces/multi-region-ha/index.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
<i class="fa fa-angle-left"></i>
|
<i class="fa fa-angle-left"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
<a rel="next prefetch" href="../architecture/orchestrator-auth-integration.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
|
<a rel="next prefetch" href="../../../quick-reference/master.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
|
||||||
<i class="fa fa-angle-right"></i>
|
<i class="fa fa-angle-right"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
@ -305,11 +192,11 @@ Total tokens: 7466(7 in, 7459 out)</p>
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<nav class="nav-wide-wrapper" aria-label="Page navigation">
|
<nav class="nav-wide-wrapper" aria-label="Page navigation">
|
||||||
<a rel="prev" href="../architecture/nickel-executable-examples.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
<a rel="prev" href="../../../../examples/workspaces/multi-region-ha/index.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
<i class="fa fa-angle-left"></i>
|
<i class="fa fa-angle-left"></i>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
<a rel="next prefetch" href="../architecture/orchestrator-auth-integration.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
|
<a rel="next prefetch" href="../../../quick-reference/master.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
|
||||||
<i class="fa fa-angle-right"></i>
|
<i class="fa fa-angle-right"></i>
|
||||||
</a>
|
</a>
|
||||||
</nav>
|
</nav>
|
||||||
@ -324,13 +211,13 @@ Total tokens: 7466(7 in, 7459 out)</p>
|
|||||||
</script>
|
</script>
|
||||||
|
|
||||||
|
|
||||||
<script src="../elasticlunr.min.js"></script>
|
<script src="../../../elasticlunr.min.js"></script>
|
||||||
<script src="../mark.min.js"></script>
|
<script src="../../../mark.min.js"></script>
|
||||||
<script src="../searcher.js"></script>
|
<script src="../../../searcher.js"></script>
|
||||||
|
|
||||||
<script src="../clipboard.min.js"></script>
|
<script src="../../../clipboard.min.js"></script>
|
||||||
<script src="../highlight.js"></script>
|
<script src="../../../highlight.js"></script>
|
||||||
<script src="../book.js"></script>
|
<script src="../../../book.js"></script>
|
||||||
|
|
||||||
<!-- Custom JS scripts -->
|
<!-- Custom JS scripts -->
|
||||||
|
|
||||||
@ -0,0 +1 @@
|
|||||||
|
# Multi-Provider Web App Workspace
|
||||||
227
docs/examples/workspaces/multi-provider-web-app/index.html
Normal file
227
docs/examples/workspaces/multi-provider-web-app/index.html
Normal file
@ -0,0 +1,227 @@
|
|||||||
|
<!DOCTYPE HTML>
|
||||||
|
<html lang="en" class="ayu sidebar-visible" dir="ltr">
|
||||||
|
<head>
|
||||||
|
<!-- Book generated using mdBook -->
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<title>Multi-Provider Web App Workspace - Provisioning Platform Documentation</title>
|
||||||
|
|
||||||
|
|
||||||
|
<!-- Custom HTML head -->
|
||||||
|
|
||||||
|
<meta name="description" content="Complete documentation for the Provisioning Platform - Infrastructure automation with Nushell, KCL, and Rust">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<meta name="theme-color" content="#ffffff">
|
||||||
|
|
||||||
|
<link rel="icon" href="../../../favicon.svg">
|
||||||
|
<link rel="shortcut icon" href="../../../favicon.png">
|
||||||
|
<link rel="stylesheet" href="../../../css/variables.css">
|
||||||
|
<link rel="stylesheet" href="../../../css/general.css">
|
||||||
|
<link rel="stylesheet" href="../../../css/chrome.css">
|
||||||
|
<link rel="stylesheet" href="../../../css/print.css" media="print">
|
||||||
|
|
||||||
|
<!-- Fonts -->
|
||||||
|
<link rel="stylesheet" href="../../../FontAwesome/css/font-awesome.css">
|
||||||
|
<link rel="stylesheet" href="../../../fonts/fonts.css">
|
||||||
|
|
||||||
|
<!-- Highlight.js Stylesheets -->
|
||||||
|
<link rel="stylesheet" id="highlight-css" href="../../../highlight.css">
|
||||||
|
<link rel="stylesheet" id="tomorrow-night-css" href="../../../tomorrow-night.css">
|
||||||
|
<link rel="stylesheet" id="ayu-highlight-css" href="../../../ayu-highlight.css">
|
||||||
|
|
||||||
|
<!-- Custom theme stylesheets -->
|
||||||
|
|
||||||
|
|
||||||
|
<!-- Provide site root and default themes to javascript -->
|
||||||
|
<script>
|
||||||
|
const path_to_root = "../../../";
|
||||||
|
const default_light_theme = "ayu";
|
||||||
|
const default_dark_theme = "navy";
|
||||||
|
</script>
|
||||||
|
<!-- Start loading toc.js asap -->
|
||||||
|
<script src="../../../toc.js"></script>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="mdbook-help-container">
|
||||||
|
<div id="mdbook-help-popup">
|
||||||
|
<h2 class="mdbook-help-title">Keyboard shortcuts</h2>
|
||||||
|
<div>
|
||||||
|
<p>Press <kbd>←</kbd> or <kbd>→</kbd> to navigate between chapters</p>
|
||||||
|
<p>Press <kbd>S</kbd> or <kbd>/</kbd> to search in the book</p>
|
||||||
|
<p>Press <kbd>?</kbd> to show this help</p>
|
||||||
|
<p>Press <kbd>Esc</kbd> to hide this help</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div id="body-container">
|
||||||
|
<!-- Work around some values being stored in localStorage wrapped in quotes -->
|
||||||
|
<script>
|
||||||
|
try {
|
||||||
|
let theme = localStorage.getItem('mdbook-theme');
|
||||||
|
let sidebar = localStorage.getItem('mdbook-sidebar');
|
||||||
|
|
||||||
|
if (theme.startsWith('"') && theme.endsWith('"')) {
|
||||||
|
localStorage.setItem('mdbook-theme', theme.slice(1, theme.length - 1));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (sidebar.startsWith('"') && sidebar.endsWith('"')) {
|
||||||
|
localStorage.setItem('mdbook-sidebar', sidebar.slice(1, sidebar.length - 1));
|
||||||
|
}
|
||||||
|
} catch (e) { }
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<!-- Set the theme before any content is loaded, prevents flash -->
|
||||||
|
<script>
|
||||||
|
const default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? default_dark_theme : default_light_theme;
|
||||||
|
let theme;
|
||||||
|
try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
|
||||||
|
if (theme === null || theme === undefined) { theme = default_theme; }
|
||||||
|
const html = document.documentElement;
|
||||||
|
html.classList.remove('ayu')
|
||||||
|
html.classList.add(theme);
|
||||||
|
html.classList.add("js");
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<input type="checkbox" id="sidebar-toggle-anchor" class="hidden">
|
||||||
|
|
||||||
|
<!-- Hide / unhide sidebar before it is displayed -->
|
||||||
|
<script>
|
||||||
|
let sidebar = null;
|
||||||
|
const sidebar_toggle = document.getElementById("sidebar-toggle-anchor");
|
||||||
|
if (document.body.clientWidth >= 1080) {
|
||||||
|
try { sidebar = localStorage.getItem('mdbook-sidebar'); } catch(e) { }
|
||||||
|
sidebar = sidebar || 'visible';
|
||||||
|
} else {
|
||||||
|
sidebar = 'hidden';
|
||||||
|
}
|
||||||
|
sidebar_toggle.checked = sidebar === 'visible';
|
||||||
|
html.classList.remove('sidebar-visible');
|
||||||
|
html.classList.add("sidebar-" + sidebar);
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<nav id="sidebar" class="sidebar" aria-label="Table of contents">
|
||||||
|
<!-- populated by js -->
|
||||||
|
<mdbook-sidebar-scrollbox class="sidebar-scrollbox"></mdbook-sidebar-scrollbox>
|
||||||
|
<noscript>
|
||||||
|
<iframe class="sidebar-iframe-outer" src="../../../toc.html"></iframe>
|
||||||
|
</noscript>
|
||||||
|
<div id="sidebar-resize-handle" class="sidebar-resize-handle">
|
||||||
|
<div class="sidebar-resize-indicator"></div>
|
||||||
|
</div>
|
||||||
|
</nav>
|
||||||
|
|
||||||
|
<div id="page-wrapper" class="page-wrapper">
|
||||||
|
|
||||||
|
<div class="page">
|
||||||
|
<div id="menu-bar-hover-placeholder"></div>
|
||||||
|
<div id="menu-bar" class="menu-bar sticky">
|
||||||
|
<div class="left-buttons">
|
||||||
|
<label id="sidebar-toggle" class="icon-button" for="sidebar-toggle-anchor" title="Toggle Table of Contents" aria-label="Toggle Table of Contents" aria-controls="sidebar">
|
||||||
|
<i class="fa fa-bars"></i>
|
||||||
|
</label>
|
||||||
|
<button id="theme-toggle" class="icon-button" type="button" title="Change theme" aria-label="Change theme" aria-haspopup="true" aria-expanded="false" aria-controls="theme-list">
|
||||||
|
<i class="fa fa-paint-brush"></i>
|
||||||
|
</button>
|
||||||
|
<ul id="theme-list" class="theme-popup" aria-label="Themes" role="menu">
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="default_theme">Auto</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="light">Light</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="rust">Rust</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="coal">Coal</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="navy">Navy</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="ayu">Ayu</button></li>
|
||||||
|
</ul>
|
||||||
|
<button id="search-toggle" class="icon-button" type="button" title="Search (`/`)" aria-label="Toggle Searchbar" aria-expanded="false" aria-keyshortcuts="/ s" aria-controls="searchbar">
|
||||||
|
<i class="fa fa-search"></i>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h1 class="menu-title">Provisioning Platform Documentation</h1>
|
||||||
|
|
||||||
|
<div class="right-buttons">
|
||||||
|
<a href="../../../print.html" title="Print this book" aria-label="Print this book">
|
||||||
|
<i id="print-button" class="fa fa-print"></i>
|
||||||
|
</a>
|
||||||
|
<a href="https://github.com/provisioning/provisioning-platform" title="Git repository" aria-label="Git repository">
|
||||||
|
<i id="git-repository-button" class="fa fa-github"></i>
|
||||||
|
</a>
|
||||||
|
<a href="https://github.com/provisioning/provisioning-platform/edit/main/provisioning/docs/src/../examples/workspaces/multi-provider-web-app/README.md" title="Suggest an edit" aria-label="Suggest an edit">
|
||||||
|
<i id="git-edit-button" class="fa fa-edit"></i>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="search-wrapper" class="hidden">
|
||||||
|
<form id="searchbar-outer" class="searchbar-outer">
|
||||||
|
<input type="search" id="searchbar" name="searchbar" placeholder="Search this book ..." aria-controls="searchresults-outer" aria-describedby="searchresults-header">
|
||||||
|
</form>
|
||||||
|
<div id="searchresults-outer" class="searchresults-outer hidden">
|
||||||
|
<div id="searchresults-header" class="searchresults-header"></div>
|
||||||
|
<ul id="searchresults">
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Apply ARIA attributes after the sidebar and the sidebar toggle button are added to the DOM -->
|
||||||
|
<script>
|
||||||
|
document.getElementById('sidebar-toggle').setAttribute('aria-expanded', sidebar === 'visible');
|
||||||
|
document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
|
||||||
|
Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
|
||||||
|
link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<div id="content" class="content">
|
||||||
|
<main>
|
||||||
|
<h1 id="multi-provider-web-app-workspace"><a class="header" href="#multi-provider-web-app-workspace">Multi-Provider Web App Workspace</a></h1>
|
||||||
|
|
||||||
|
</main>
|
||||||
|
|
||||||
|
<nav class="nav-wrapper" aria-label="Page navigation">
|
||||||
|
<!-- Mobile navigation buttons -->
|
||||||
|
<a rel="prev" href="../../../guides/provider-hetzner.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
|
<i class="fa fa-angle-left"></i>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
<a rel="next prefetch" href="../../../../examples/workspaces/multi-region-ha/index.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
|
||||||
|
<i class="fa fa-angle-right"></i>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
<div style="clear: both"></div>
|
||||||
|
</nav>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<nav class="nav-wide-wrapper" aria-label="Page navigation">
|
||||||
|
<a rel="prev" href="../../../guides/provider-hetzner.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
|
<i class="fa fa-angle-left"></i>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
<a rel="next prefetch" href="../../../../examples/workspaces/multi-region-ha/index.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
|
||||||
|
<i class="fa fa-angle-right"></i>
|
||||||
|
</a>
|
||||||
|
</nav>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<script>
|
||||||
|
window.playground_copyable = true;
|
||||||
|
</script>
|
||||||
|
|
||||||
|
|
||||||
|
<script src="../../../elasticlunr.min.js"></script>
|
||||||
|
<script src="../../../mark.min.js"></script>
|
||||||
|
<script src="../../../searcher.js"></script>
|
||||||
|
|
||||||
|
<script src="../../../clipboard.min.js"></script>
|
||||||
|
<script src="../../../highlight.js"></script>
|
||||||
|
<script src="../../../book.js"></script>
|
||||||
|
|
||||||
|
<!-- Custom JS scripts -->
|
||||||
|
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
1
docs/examples/workspaces/multi-region-ha/README.md
Normal file
1
docs/examples/workspaces/multi-region-ha/README.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Multi-Region High Availability Workspace
|
||||||
227
docs/examples/workspaces/multi-region-ha/index.html
Normal file
227
docs/examples/workspaces/multi-region-ha/index.html
Normal file
@ -0,0 +1,227 @@
|
|||||||
|
<!DOCTYPE HTML>
|
||||||
|
<html lang="en" class="ayu sidebar-visible" dir="ltr">
|
||||||
|
<head>
|
||||||
|
<!-- Book generated using mdBook -->
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<title>Multi-Region High Availability Workspace - Provisioning Platform Documentation</title>
|
||||||
|
|
||||||
|
|
||||||
|
<!-- Custom HTML head -->
|
||||||
|
|
||||||
|
<meta name="description" content="Complete documentation for the Provisioning Platform - Infrastructure automation with Nushell, KCL, and Rust">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<meta name="theme-color" content="#ffffff">
|
||||||
|
|
||||||
|
<link rel="icon" href="../../../favicon.svg">
|
||||||
|
<link rel="shortcut icon" href="../../../favicon.png">
|
||||||
|
<link rel="stylesheet" href="../../../css/variables.css">
|
||||||
|
<link rel="stylesheet" href="../../../css/general.css">
|
||||||
|
<link rel="stylesheet" href="../../../css/chrome.css">
|
||||||
|
<link rel="stylesheet" href="../../../css/print.css" media="print">
|
||||||
|
|
||||||
|
<!-- Fonts -->
|
||||||
|
<link rel="stylesheet" href="../../../FontAwesome/css/font-awesome.css">
|
||||||
|
<link rel="stylesheet" href="../../../fonts/fonts.css">
|
||||||
|
|
||||||
|
<!-- Highlight.js Stylesheets -->
|
||||||
|
<link rel="stylesheet" id="highlight-css" href="../../../highlight.css">
|
||||||
|
<link rel="stylesheet" id="tomorrow-night-css" href="../../../tomorrow-night.css">
|
||||||
|
<link rel="stylesheet" id="ayu-highlight-css" href="../../../ayu-highlight.css">
|
||||||
|
|
||||||
|
<!-- Custom theme stylesheets -->
|
||||||
|
|
||||||
|
|
||||||
|
<!-- Provide site root and default themes to javascript -->
|
||||||
|
<script>
|
||||||
|
const path_to_root = "../../../";
|
||||||
|
const default_light_theme = "ayu";
|
||||||
|
const default_dark_theme = "navy";
|
||||||
|
</script>
|
||||||
|
<!-- Start loading toc.js asap -->
|
||||||
|
<script src="../../../toc.js"></script>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="mdbook-help-container">
|
||||||
|
<div id="mdbook-help-popup">
|
||||||
|
<h2 class="mdbook-help-title">Keyboard shortcuts</h2>
|
||||||
|
<div>
|
||||||
|
<p>Press <kbd>←</kbd> or <kbd>→</kbd> to navigate between chapters</p>
|
||||||
|
<p>Press <kbd>S</kbd> or <kbd>/</kbd> to search in the book</p>
|
||||||
|
<p>Press <kbd>?</kbd> to show this help</p>
|
||||||
|
<p>Press <kbd>Esc</kbd> to hide this help</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div id="body-container">
|
||||||
|
<!-- Work around some values being stored in localStorage wrapped in quotes -->
|
||||||
|
<script>
|
||||||
|
try {
|
||||||
|
let theme = localStorage.getItem('mdbook-theme');
|
||||||
|
let sidebar = localStorage.getItem('mdbook-sidebar');
|
||||||
|
|
||||||
|
if (theme.startsWith('"') && theme.endsWith('"')) {
|
||||||
|
localStorage.setItem('mdbook-theme', theme.slice(1, theme.length - 1));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (sidebar.startsWith('"') && sidebar.endsWith('"')) {
|
||||||
|
localStorage.setItem('mdbook-sidebar', sidebar.slice(1, sidebar.length - 1));
|
||||||
|
}
|
||||||
|
} catch (e) { }
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<!-- Set the theme before any content is loaded, prevents flash -->
|
||||||
|
<script>
|
||||||
|
const default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? default_dark_theme : default_light_theme;
|
||||||
|
let theme;
|
||||||
|
try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
|
||||||
|
if (theme === null || theme === undefined) { theme = default_theme; }
|
||||||
|
const html = document.documentElement;
|
||||||
|
html.classList.remove('ayu')
|
||||||
|
html.classList.add(theme);
|
||||||
|
html.classList.add("js");
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<input type="checkbox" id="sidebar-toggle-anchor" class="hidden">
|
||||||
|
|
||||||
|
<!-- Hide / unhide sidebar before it is displayed -->
|
||||||
|
<script>
|
||||||
|
let sidebar = null;
|
||||||
|
const sidebar_toggle = document.getElementById("sidebar-toggle-anchor");
|
||||||
|
if (document.body.clientWidth >= 1080) {
|
||||||
|
try { sidebar = localStorage.getItem('mdbook-sidebar'); } catch(e) { }
|
||||||
|
sidebar = sidebar || 'visible';
|
||||||
|
} else {
|
||||||
|
sidebar = 'hidden';
|
||||||
|
}
|
||||||
|
sidebar_toggle.checked = sidebar === 'visible';
|
||||||
|
html.classList.remove('sidebar-visible');
|
||||||
|
html.classList.add("sidebar-" + sidebar);
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<nav id="sidebar" class="sidebar" aria-label="Table of contents">
|
||||||
|
<!-- populated by js -->
|
||||||
|
<mdbook-sidebar-scrollbox class="sidebar-scrollbox"></mdbook-sidebar-scrollbox>
|
||||||
|
<noscript>
|
||||||
|
<iframe class="sidebar-iframe-outer" src="../../../toc.html"></iframe>
|
||||||
|
</noscript>
|
||||||
|
<div id="sidebar-resize-handle" class="sidebar-resize-handle">
|
||||||
|
<div class="sidebar-resize-indicator"></div>
|
||||||
|
</div>
|
||||||
|
</nav>
|
||||||
|
|
||||||
|
<div id="page-wrapper" class="page-wrapper">
|
||||||
|
|
||||||
|
<div class="page">
|
||||||
|
<div id="menu-bar-hover-placeholder"></div>
|
||||||
|
<div id="menu-bar" class="menu-bar sticky">
|
||||||
|
<div class="left-buttons">
|
||||||
|
<label id="sidebar-toggle" class="icon-button" for="sidebar-toggle-anchor" title="Toggle Table of Contents" aria-label="Toggle Table of Contents" aria-controls="sidebar">
|
||||||
|
<i class="fa fa-bars"></i>
|
||||||
|
</label>
|
||||||
|
<button id="theme-toggle" class="icon-button" type="button" title="Change theme" aria-label="Change theme" aria-haspopup="true" aria-expanded="false" aria-controls="theme-list">
|
||||||
|
<i class="fa fa-paint-brush"></i>
|
||||||
|
</button>
|
||||||
|
<ul id="theme-list" class="theme-popup" aria-label="Themes" role="menu">
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="default_theme">Auto</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="light">Light</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="rust">Rust</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="coal">Coal</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="navy">Navy</button></li>
|
||||||
|
<li role="none"><button role="menuitem" class="theme" id="ayu">Ayu</button></li>
|
||||||
|
</ul>
|
||||||
|
<button id="search-toggle" class="icon-button" type="button" title="Search (`/`)" aria-label="Toggle Searchbar" aria-expanded="false" aria-keyshortcuts="/ s" aria-controls="searchbar">
|
||||||
|
<i class="fa fa-search"></i>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h1 class="menu-title">Provisioning Platform Documentation</h1>
|
||||||
|
|
||||||
|
<div class="right-buttons">
|
||||||
|
<a href="../../../print.html" title="Print this book" aria-label="Print this book">
|
||||||
|
<i id="print-button" class="fa fa-print"></i>
|
||||||
|
</a>
|
||||||
|
<a href="https://github.com/provisioning/provisioning-platform" title="Git repository" aria-label="Git repository">
|
||||||
|
<i id="git-repository-button" class="fa fa-github"></i>
|
||||||
|
</a>
|
||||||
|
<a href="https://github.com/provisioning/provisioning-platform/edit/main/provisioning/docs/src/../examples/workspaces/multi-region-ha/README.md" title="Suggest an edit" aria-label="Suggest an edit">
|
||||||
|
<i id="git-edit-button" class="fa fa-edit"></i>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="search-wrapper" class="hidden">
|
||||||
|
<form id="searchbar-outer" class="searchbar-outer">
|
||||||
|
<input type="search" id="searchbar" name="searchbar" placeholder="Search this book ..." aria-controls="searchresults-outer" aria-describedby="searchresults-header">
|
||||||
|
</form>
|
||||||
|
<div id="searchresults-outer" class="searchresults-outer hidden">
|
||||||
|
<div id="searchresults-header" class="searchresults-header"></div>
|
||||||
|
<ul id="searchresults">
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Apply ARIA attributes after the sidebar and the sidebar toggle button are added to the DOM -->
|
||||||
|
<script>
|
||||||
|
document.getElementById('sidebar-toggle').setAttribute('aria-expanded', sidebar === 'visible');
|
||||||
|
document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
|
||||||
|
Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
|
||||||
|
link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<div id="content" class="content">
|
||||||
|
<main>
|
||||||
|
<h1 id="multi-region-high-availability-workspace"><a class="header" href="#multi-region-high-availability-workspace">Multi-Region High Availability Workspace</a></h1>
|
||||||
|
|
||||||
|
</main>
|
||||||
|
|
||||||
|
<nav class="nav-wrapper" aria-label="Page navigation">
|
||||||
|
<!-- Mobile navigation buttons -->
|
||||||
|
<a rel="prev" href="../../../../examples/workspaces/multi-provider-web-app/index.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
|
<i class="fa fa-angle-left"></i>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
<a rel="next prefetch" href="../../../../examples/workspaces/cost-optimized/index.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
|
||||||
|
<i class="fa fa-angle-right"></i>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
<div style="clear: both"></div>
|
||||||
|
</nav>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<nav class="nav-wide-wrapper" aria-label="Page navigation">
|
||||||
|
<a rel="prev" href="../../../../examples/workspaces/multi-provider-web-app/index.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
|
||||||
|
<i class="fa fa-angle-left"></i>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
<a rel="next prefetch" href="../../../../examples/workspaces/cost-optimized/index.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
|
||||||
|
<i class="fa fa-angle-right"></i>
|
||||||
|
</a>
|
||||||
|
</nav>
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<script>
|
||||||
|
window.playground_copyable = true;
|
||||||
|
</script>
|
||||||
|
|
||||||
|
|
||||||
|
<script src="../../../elasticlunr.min.js"></script>
|
||||||
|
<script src="../../../mark.min.js"></script>
|
||||||
|
<script src="../../../searcher.js"></script>
|
||||||
|
|
||||||
|
<script src="../../../clipboard.min.js"></script>
|
||||||
|
<script src="../../../highlight.js"></script>
|
||||||
|
<script src="../../../book.js"></script>
|
||||||
|
|
||||||
|
<!-- Custom JS scripts -->
|
||||||
|
|
||||||
|
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
@ -25,9 +25,15 @@
|
|||||||
|
|
||||||
## What is Provisioning
|
## What is Provisioning
|
||||||
|
|
||||||
**Provisioning** is a comprehensive **Infrastructure as Code (IaC)** platform designed to manage complete infrastructure lifecycles: cloud providers, infrastructure services, clusters, and isolated workspaces across multiple cloud/local environments.
|
**Provisioning** is a comprehensive **Infrastructure as Code (IaC)** platform designed to manage
|
||||||
|
complete infrastructure lifecycles: cloud providers, infrastructure services, clusters,
|
||||||
|
and isolated workspaces across multiple cloud/local environments.
|
||||||
|
|
||||||
Extensible and customizable by design, it delivers type-safe, configuration-driven workflows with enterprise security (encrypted configuration, Cosmian KMS integration, Cedar policy engine, secrets management, authorization and permissions control, compliance checking, anomaly detection) and adaptable deployment modes (interactive UI, CLI automation, unattended CI/CD) suitable for any scale from development to production.
|
Extensible and customizable by design, it delivers type-safe, configuration-driven workflows
|
||||||
|
with enterprise security (encrypted configuration, Cosmian KMS integration, Cedar policy engine,
|
||||||
|
secrets management, authorization and permissions control, compliance checking, anomaly detection)
|
||||||
|
and adaptable deployment modes (interactive UI, CLI automation, unattended CI/CD)
|
||||||
|
suitable for any scale from development to production.
|
||||||
|
|
||||||
### Technical Definition
|
### Technical Definition
|
||||||
|
|
||||||
@ -87,7 +93,7 @@ server: Server {
|
|||||||
plan = "medium" # Abstract size, provider-specific translation
|
plan = "medium" # Abstract size, provider-specific translation
|
||||||
provider = "upcloud" # Switch to "aws" or "local" as needed
|
provider = "upcloud" # Switch to "aws" or "local" as needed
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 2. **Dependency Hell**
|
#### 2. **Dependency Hell**
|
||||||
|
|
||||||
@ -98,7 +104,7 @@ server: Server {
|
|||||||
```kcl
|
```kcl
|
||||||
# Provisioning resolves: containerd → etcd → kubernetes → cilium
|
# Provisioning resolves: containerd → etcd → kubernetes → cilium
|
||||||
taskservs = ["cilium"] # Automatically installs all dependencies
|
taskservs = ["cilium"] # Automatically installs all dependencies
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 3. **Configuration Sprawl**
|
#### 3. **Configuration Sprawl**
|
||||||
|
|
||||||
@ -108,7 +114,7 @@ taskservs = ["cilium"] # Automatically installs all dependencies
|
|||||||
|
|
||||||
```plaintext
|
```plaintext
|
||||||
Defaults → User → Project → Infrastructure → Environment → Runtime
|
Defaults → User → Project → Infrastructure → Environment → Runtime
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 4. **Imperative Scripts**
|
#### 4. **Imperative Scripts**
|
||||||
|
|
||||||
@ -201,13 +207,13 @@ workspace_librecloud/ # Production workspace
|
|||||||
workspace_dev/ # Development workspace
|
workspace_dev/ # Development workspace
|
||||||
├── infra/
|
├── infra/
|
||||||
└── config/
|
└── config/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Switch between workspaces with single command:
|
Switch between workspaces with single command:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
provisioning workspace switch librecloud
|
provisioning workspace switch librecloud
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 5. **Workflows**
|
### 5. **Workflows**
|
||||||
|
|
||||||
@ -272,7 +278,7 @@ Coordinated sequences of operations with dependency management.
|
|||||||
│ • Kubernetes Clusters │
|
│ • Kubernetes Clusters │
|
||||||
│ • Running Services │
|
│ • Running Services │
|
||||||
└─────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Directory Structure
|
### Directory Structure
|
||||||
|
|
||||||
@ -315,7 +321,7 @@ project-provisioning/
|
|||||||
├── api/ # API documentation
|
├── api/ # API documentation
|
||||||
├── architecture/ # Architecture docs
|
├── architecture/ # Architecture docs
|
||||||
└── development/ # Development guides
|
└── development/ # Development guides
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Platform Services
|
### Platform Services
|
||||||
|
|
||||||
@ -463,8 +469,8 @@ Comprehensive version tracking and updates.
|
|||||||
### Core Technologies
|
### Core Technologies
|
||||||
|
|
||||||
| Technology | Version | Purpose | Why |
|
| Technology | Version | Purpose | Why |
|
||||||
|------------|---------|---------|-----|
|
| ------------ | --------- | --------- | ----- |
|
||||||
| **Nushell** | 0.107.1+ | Primary shell and scripting language | Structured data pipelines, cross-platform, modern built-in parsers (JSON/YAML/TOML) |
|
| **Nushell** | 0.107.1+ | Primary shell and scripting language | Data pipelines, cross-platform, modern parsers |
|
||||||
| **KCL** | 0.11.3+ | Configuration language | Type safety, schema validation, immutability, constraint checking |
|
| **KCL** | 0.11.3+ | Configuration language | Type safety, schema validation, immutability, constraint checking |
|
||||||
| **Rust** | Latest | Platform services (orchestrator, control-center, installer) | Performance, memory safety, concurrency, reliability |
|
| **Rust** | Latest | Platform services (orchestrator, control-center, installer) | Performance, memory safety, concurrency, reliability |
|
||||||
| **Tera** | Latest | Template engine | Jinja2-like syntax, configuration file rendering, variable interpolation, filters and functions |
|
| **Tera** | Latest | Template engine | Jinja2-like syntax, configuration file rendering, variable interpolation, filters and functions |
|
||||||
@ -472,13 +478,13 @@ Comprehensive version tracking and updates.
|
|||||||
### Data & State Management
|
### Data & State Management
|
||||||
|
|
||||||
| Technology | Version | Purpose | Features |
|
| Technology | Version | Purpose | Features |
|
||||||
|------------|---------|---------|----------|
|
| ------------ | --------- | --------- | ---------- |
|
||||||
| **SurrealDB** | Latest | High-performance graph database backend | Multi-model (document, graph, relational), real-time queries, distributed architecture, complex relationship tracking |
|
| **SurrealDB** | Latest | Graph database backend | Multi-model, real-time queries, distributed, relationships |
|
||||||
|
|
||||||
### Platform Services (Rust-based)
|
### Platform Services (Rust-based)
|
||||||
|
|
||||||
| Service | Purpose | Security Features |
|
| Service | Purpose | Security Features |
|
||||||
|---------|---------|-------------------|
|
| --------- | --------- | ------------------- |
|
||||||
| **Orchestrator** | Workflow execution, task scheduling, state management | File-based persistence, retry logic, checkpoint recovery |
|
| **Orchestrator** | Workflow execution, task scheduling, state management | File-based persistence, retry logic, checkpoint recovery |
|
||||||
| **Control Center** | Web-based infrastructure management | **Authorization and permissions control**, RBAC, audit logging |
|
| **Control Center** | Web-based infrastructure management | **Authorization and permissions control**, RBAC, audit logging |
|
||||||
| **Installer** | Platform installation (TUI + CLI modes) | Secure configuration generation, validation |
|
| **Installer** | Platform installation (TUI + CLI modes) | Secure configuration generation, validation |
|
||||||
@ -487,7 +493,7 @@ Comprehensive version tracking and updates.
|
|||||||
### Security & Secrets
|
### Security & Secrets
|
||||||
|
|
||||||
| Technology | Version | Purpose | Enterprise Features |
|
| Technology | Version | Purpose | Enterprise Features |
|
||||||
|------------|---------|---------|---------------------|
|
| ------------ | --------- | --------- | --------------------- |
|
||||||
| **SOPS** | 3.10.2+ | Secrets management | Encrypted configuration files |
|
| **SOPS** | 3.10.2+ | Secrets management | Encrypted configuration files |
|
||||||
| **Age** | 1.2.1+ | Encryption | Secure key-based encryption |
|
| **Age** | 1.2.1+ | Encryption | Secure key-based encryption |
|
||||||
| **Cosmian KMS** | Latest | Key Management System | Confidential computing, secure key storage, cloud-native KMS |
|
| **Cosmian KMS** | Latest | Key Management System | Confidential computing, secure key storage, cloud-native KMS |
|
||||||
@ -496,7 +502,7 @@ Comprehensive version tracking and updates.
|
|||||||
### Optional Tools
|
### Optional Tools
|
||||||
|
|
||||||
| Tool | Purpose |
|
| Tool | Purpose |
|
||||||
|------|---------|
|
| ------ | --------- |
|
||||||
| **K9s** | Kubernetes management interface |
|
| **K9s** | Kubernetes management interface |
|
||||||
| **nu_plugin_tera** | Nushell plugin for Tera template rendering |
|
| **nu_plugin_tera** | Nushell plugin for Tera template rendering |
|
||||||
| **nu_plugin_kcl** | Nushell plugin for KCL integration (CLI required, plugin optional) |
|
| **nu_plugin_kcl** | Nushell plugin for KCL integration (CLI required, plugin optional) |
|
||||||
@ -529,7 +535,7 @@ Comprehensive version tracking and updates.
|
|||||||
9. Task services installed on servers
|
9. Task services installed on servers
|
||||||
↓
|
↓
|
||||||
10. State persisted and monitored
|
10. State persisted and monitored
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Example Workflow: Deploy Kubernetes Cluster
|
### Example Workflow: Deploy Kubernetes Cluster
|
||||||
|
|
||||||
@ -552,13 +558,13 @@ let config = {
|
|||||||
taskservs = ["kubernetes", "cilium", "rook-ceph"],
|
taskservs = ["kubernetes", "cilium", "rook-ceph"],
|
||||||
} in
|
} in
|
||||||
config
|
config
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Step 2**: Submit to Provisioning
|
**Step 2**: Submit to Provisioning
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
provisioning server create --infra my-cluster
|
provisioning server create --infra my-cluster
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Step 3**: Provisioning executes workflow
|
**Step 3**: Provisioning executes workflow
|
||||||
|
|
||||||
@ -583,13 +589,13 @@ provisioning server create --infra my-cluster
|
|||||||
4. Checkpoint after each step
|
4. Checkpoint after each step
|
||||||
5. Monitor health checks
|
5. Monitor health checks
|
||||||
6. Report completion
|
6. Report completion
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Step 4**: Verify deployment
|
**Step 4**: Verify deployment
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
provisioning cluster status my-cluster
|
provisioning cluster status my-cluster
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Hierarchy
|
### Configuration Hierarchy
|
||||||
|
|
||||||
@ -607,7 +613,7 @@ Configuration values are resolved through a hierarchy:
|
|||||||
5. Environment Config (workspace/config/prod-defaults.toml)
|
5. Environment Config (workspace/config/prod-defaults.toml)
|
||||||
↓ (overridden by)
|
↓ (overridden by)
|
||||||
6. Runtime Flags (--flag value)
|
6. Runtime Flags (--flag value)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Example**:
|
**Example**:
|
||||||
|
|
||||||
@ -626,7 +632,7 @@ default_plan = "large" # Overrides user preference
|
|||||||
|
|
||||||
# Runtime
|
# Runtime
|
||||||
provisioning server create --plan xlarge # Overrides everything
|
provisioning server create --plan xlarge # Overrides everything
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -642,7 +648,7 @@ provisioning cluster create k8s-prod --provider upcloud
|
|||||||
|
|
||||||
# AWS cluster (same config)
|
# AWS cluster (same config)
|
||||||
provisioning cluster create k8s-prod --provider aws
|
provisioning cluster create k8s-prod --provider aws
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. **Development → Staging → Production Pipeline**
|
### 2. **Development → Staging → Production Pipeline**
|
||||||
|
|
||||||
@ -660,7 +666,7 @@ provisioning cluster create app-stack
|
|||||||
# Production (HA, larger resources)
|
# Production (HA, larger resources)
|
||||||
provisioning workspace switch prod
|
provisioning workspace switch prod
|
||||||
provisioning cluster create app-stack
|
provisioning cluster create app-stack
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. **Infrastructure as Code Testing**
|
### 3. **Infrastructure as Code Testing**
|
||||||
|
|
||||||
@ -676,7 +682,7 @@ provisioning test env run <env-id>
|
|||||||
|
|
||||||
# Cleanup
|
# Cleanup
|
||||||
provisioning test env cleanup <env-id>
|
provisioning test env cleanup <env-id>
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. **Batch Multi-Region Deployment**
|
### 4. **Batch Multi-Region Deployment**
|
||||||
|
|
||||||
@ -708,12 +714,12 @@ let batch_workflow = {
|
|||||||
parallel_limit = 3, # All at once
|
parallel_limit = 3, # All at once
|
||||||
} in
|
} in
|
||||||
batch_workflow
|
batch_workflow
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
provisioning batch submit workflows/multi-region.ncl
|
provisioning batch submit workflows/multi-region.ncl
|
||||||
provisioning batch monitor <workflow-id>
|
provisioning batch monitor <workflow-id>
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 5. **Automated Disaster Recovery**
|
### 5. **Automated Disaster Recovery**
|
||||||
|
|
||||||
@ -727,7 +733,7 @@ provisioning workspace switch prod
|
|||||||
provisioning cluster create --infra backup-restore --wait
|
provisioning cluster create --infra backup-restore --wait
|
||||||
|
|
||||||
# All services restored with same configuration
|
# All services restored with same configuration
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 6. **CI/CD Integration**
|
### 6. **CI/CD Integration**
|
||||||
|
|
||||||
@ -751,7 +757,7 @@ deploy-production:
|
|||||||
script:
|
script:
|
||||||
- provisioning workspace switch prod
|
- provisioning workspace switch prod
|
||||||
- provisioning cluster create app-stack --yes
|
- provisioning cluster create app-stack --yes
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -10,9 +10,11 @@
|
|||||||
**Last Updated**: 2025-01-02 (Phase 3.A Cleanup Complete)
|
**Last Updated**: 2025-01-02 (Phase 3.A Cleanup Complete)
|
||||||
**Status**: ✅ Primary documentation source (145 files consolidated)
|
**Status**: ✅ Primary documentation source (145 files consolidated)
|
||||||
|
|
||||||
Welcome to the comprehensive documentation for the Provisioning Platform - a modern, cloud-native infrastructure automation system built with Nushell, KCL, and Rust.
|
Welcome to the comprehensive documentation for the Provisioning Platform - a modern, cloud-native infrastructure automation system built with Nushell,
|
||||||
|
KCL, and Rust.
|
||||||
|
|
||||||
> **Note**: Architecture Decision Records (ADRs) and high-level design documentation are in `docs/` directory. This location contains all user-facing, operational, and product documentation.
|
> **Note**: Architecture Decision Records (ADRs) and design documentation are in `docs/`
|
||||||
|
> directory. This location contains user-facing, operational, and product documentation.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -21,7 +23,7 @@ Welcome to the comprehensive documentation for the Provisioning Platform - a mod
|
|||||||
### 🚀 Getting Started
|
### 🚀 Getting Started
|
||||||
|
|
||||||
| Document | Description | Audience |
|
| Document | Description | Audience |
|
||||||
|----------|-------------|----------|
|
| ---------- | ------------- | ---------- |
|
||||||
| **[Installation Guide](getting-started/installation-guide.md)** | Install and configure the system | New Users |
|
| **[Installation Guide](getting-started/installation-guide.md)** | Install and configure the system | New Users |
|
||||||
| **[Getting Started](getting-started/getting-started.md)** | First steps and basic concepts | New Users |
|
| **[Getting Started](getting-started/getting-started.md)** | First steps and basic concepts | New Users |
|
||||||
| **[Quick Reference](getting-started/quickstart-cheatsheet.md)** | Command cheat sheet | All Users |
|
| **[Quick Reference](getting-started/quickstart-cheatsheet.md)** | Command cheat sheet | All Users |
|
||||||
@ -30,7 +32,7 @@ Welcome to the comprehensive documentation for the Provisioning Platform - a mod
|
|||||||
### 📚 User Guides
|
### 📚 User Guides
|
||||||
|
|
||||||
| Document | Description |
|
| Document | Description |
|
||||||
|----------|-------------|
|
| ---------- | ------------- |
|
||||||
| **[CLI Reference](infrastructure/cli-reference.md)** | Complete command reference |
|
| **[CLI Reference](infrastructure/cli-reference.md)** | Complete command reference |
|
||||||
| **[Workspace Management](infrastructure/workspace-setup.md)** | Workspace creation and management |
|
| **[Workspace Management](infrastructure/workspace-setup.md)** | Workspace creation and management |
|
||||||
| **[Workspace Switching](infrastructure/workspace-switching-guide.md)** | Switch between workspaces |
|
| **[Workspace Switching](infrastructure/workspace-switching-guide.md)** | Switch between workspaces |
|
||||||
@ -45,7 +47,7 @@ Welcome to the comprehensive documentation for the Provisioning Platform - a mod
|
|||||||
### 🏗️ Architecture
|
### 🏗️ Architecture
|
||||||
|
|
||||||
| Document | Description |
|
| Document | Description |
|
||||||
|----------|-------------|
|
| ---------- | ------------- |
|
||||||
| **[System Overview](architecture/system-overview.md)** | High-level architecture |
|
| **[System Overview](architecture/system-overview.md)** | High-level architecture |
|
||||||
| **[Multi-Repo Architecture](architecture/multi-repo-architecture.md)** | Repository structure and OCI distribution |
|
| **[Multi-Repo Architecture](architecture/multi-repo-architecture.md)** | Repository structure and OCI distribution |
|
||||||
| **[Design Principles](architecture/design-principles.md)** | Architectural philosophy |
|
| **[Design Principles](architecture/design-principles.md)** | Architectural philosophy |
|
||||||
@ -55,7 +57,7 @@ Welcome to the comprehensive documentation for the Provisioning Platform - a mod
|
|||||||
### 📋 Architecture Decision Records (ADRs)
|
### 📋 Architecture Decision Records (ADRs)
|
||||||
|
|
||||||
| ADR | Title | Status |
|
| ADR | Title | Status |
|
||||||
|-----|-------|--------|
|
| ----- | ------- | -------- |
|
||||||
| **[ADR-001](architecture/adr/adr-001-project-structure.md)** | Project Structure Decision | Accepted |
|
| **[ADR-001](architecture/adr/adr-001-project-structure.md)** | Project Structure Decision | Accepted |
|
||||||
| **[ADR-002](architecture/adr/adr-002-distribution-strategy.md)** | Distribution Strategy | Accepted |
|
| **[ADR-002](architecture/adr/adr-002-distribution-strategy.md)** | Distribution Strategy | Accepted |
|
||||||
| **[ADR-003](architecture/adr/adr-003-workspace-isolation.md)** | Workspace Isolation | Accepted |
|
| **[ADR-003](architecture/adr/adr-003-workspace-isolation.md)** | Workspace Isolation | Accepted |
|
||||||
@ -66,7 +68,7 @@ Welcome to the comprehensive documentation for the Provisioning Platform - a mod
|
|||||||
### 🔌 API Documentation
|
### 🔌 API Documentation
|
||||||
|
|
||||||
| Document | Description |
|
| Document | Description |
|
||||||
|----------|-------------|
|
| ---------- | ------------- |
|
||||||
| **[REST API](api-reference/rest-api.md)** | HTTP API endpoints |
|
| **[REST API](api-reference/rest-api.md)** | HTTP API endpoints |
|
||||||
| **[WebSocket API](api-reference/websocket.md)** | Real-time event streams |
|
| **[WebSocket API](api-reference/websocket.md)** | Real-time event streams |
|
||||||
| **[Extensions API](development/extensions.md)** | Extension integration APIs |
|
| **[Extensions API](development/extensions.md)** | Extension integration APIs |
|
||||||
@ -76,7 +78,7 @@ Welcome to the comprehensive documentation for the Provisioning Platform - a mod
|
|||||||
### 🛠️ Development
|
### 🛠️ Development
|
||||||
|
|
||||||
| Document | Description |
|
| Document | Description |
|
||||||
|----------|-------------|
|
| ---------- | ------------- |
|
||||||
| **[Development README](development/README.md)** | Developer overview |
|
| **[Development README](development/README.md)** | Developer overview |
|
||||||
| **[Implementation Guide](development/implementation-guide.md)** | Implementation details |
|
| **[Implementation Guide](development/implementation-guide.md)** | Implementation details |
|
||||||
| **[Provider Development](development/quick-provider-guide.md)** | Create cloud providers |
|
| **[Provider Development](development/quick-provider-guide.md)** | Create cloud providers |
|
||||||
@ -87,13 +89,13 @@ Welcome to the comprehensive documentation for the Provisioning Platform - a mod
|
|||||||
### 🐛 Troubleshooting
|
### 🐛 Troubleshooting
|
||||||
|
|
||||||
| Document | Description |
|
| Document | Description |
|
||||||
|----------|-------------|
|
| ---------- | ------------- |
|
||||||
| **[Troubleshooting Guide](troubleshooting/troubleshooting-guide.md)** | Common issues and solutions |
|
| **[Troubleshooting Guide](troubleshooting/troubleshooting-guide.md)** | Common issues and solutions |
|
||||||
|
|
||||||
### 📖 How-To Guides
|
### 📖 How-To Guides
|
||||||
|
|
||||||
| Document | Description |
|
| Document | Description |
|
||||||
|----------|-------------|
|
| ---------- | ------------- |
|
||||||
| **[From Scratch](guides/from-scratch.md)** | Complete deployment from zero |
|
| **[From Scratch](guides/from-scratch.md)** | Complete deployment from zero |
|
||||||
| **[Update Infrastructure](guides/update-infrastructure.md)** | Safe update procedures |
|
| **[Update Infrastructure](guides/update-infrastructure.md)** | Safe update procedures |
|
||||||
| **[Customize Infrastructure](guides/customize-infrastructure.md)** | Layer and template customization |
|
| **[Customize Infrastructure](guides/customize-infrastructure.md)** | Layer and template customization |
|
||||||
@ -101,13 +103,13 @@ Welcome to the comprehensive documentation for the Provisioning Platform - a mod
|
|||||||
### 🔐 Configuration
|
### 🔐 Configuration
|
||||||
|
|
||||||
| Document | Description |
|
| Document | Description |
|
||||||
|----------|-------------|
|
| ---------- | ------------- |
|
||||||
| **[Workspace Config Architecture](configuration/workspace-config-architecture.md)** | Configuration architecture |
|
| **[Workspace Config Architecture](configuration/workspace-config-architecture.md)** | Configuration architecture |
|
||||||
|
|
||||||
### 📦 Quick References
|
### 📦 Quick References
|
||||||
|
|
||||||
| Document | Description |
|
| Document | Description |
|
||||||
|----------|-------------|
|
| ---------- | ------------- |
|
||||||
| **[Quickstart Cheatsheet](getting-started/quickstart-cheatsheet.md)** | Command shortcuts |
|
| **[Quickstart Cheatsheet](getting-started/quickstart-cheatsheet.md)** | Command shortcuts |
|
||||||
| **[OCI Quick Reference](quick-reference/oci.md)** | OCI operations |
|
| **[OCI Quick Reference](quick-reference/oci.md)** | OCI operations |
|
||||||
|
|
||||||
@ -158,7 +160,7 @@ provisioning/docs/src/
|
|||||||
├── configuration/ # Configuration docs
|
├── configuration/ # Configuration docs
|
||||||
├── troubleshooting/ # Troubleshooting guides
|
├── troubleshooting/ # Troubleshooting guides
|
||||||
└── quick-reference/ # Quick references
|
└── quick-reference/ # Quick references
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -166,7 +168,8 @@ provisioning/docs/src/
|
|||||||
|
|
||||||
### Infrastructure as Code (IaC)
|
### Infrastructure as Code (IaC)
|
||||||
|
|
||||||
The provisioning platform uses **declarative configuration** to manage infrastructure. Instead of manually creating resources, you define what you want in Nickel configuration files, and the system makes it happen.
|
The provisioning platform uses **declarative configuration** to manage infrastructure. Instead of manually creating resources, you define what you
|
||||||
|
want in Nickel configuration files, and the system makes it happen.
|
||||||
|
|
||||||
### Mode-Based Architecture
|
### Mode-Based Architecture
|
||||||
|
|
||||||
@ -317,7 +320,7 @@ Extensions and packages distributed as OCI artifacts, enabling:
|
|||||||
## Technology Stack
|
## Technology Stack
|
||||||
|
|
||||||
| Component | Technology | Purpose |
|
| Component | Technology | Purpose |
|
||||||
|-----------|------------|---------|
|
| ----------- | ------------ | --------- |
|
||||||
| **Core CLI** | Nushell 0.107.1 | Shell and scripting |
|
| **Core CLI** | Nushell 0.107.1 | Shell and scripting |
|
||||||
| **Configuration** | KCL 0.11.2 | Type-safe IaC |
|
| **Configuration** | KCL 0.11.2 | Type-safe IaC |
|
||||||
| **Orchestrator** | Rust | High-performance coordination |
|
| **Orchestrator** | Rust | High-performance coordination |
|
||||||
@ -365,7 +368,7 @@ This project welcomes contributions! See **[Development Guide](development/READM
|
|||||||
## Version History
|
## Version History
|
||||||
|
|
||||||
| Version | Date | Major Changes |
|
| Version | Date | Major Changes |
|
||||||
|---------|------|---------------|
|
| --------- | ------ | --------------- |
|
||||||
| **3.5.0** | 2025-10-06 | Mode system, OCI registry, comprehensive documentation |
|
| **3.5.0** | 2025-10-06 | Mode system, OCI registry, comprehensive documentation |
|
||||||
| **3.4.0** | 2025-10-06 | Test environment service |
|
| **3.4.0** | 2025-10-06 | Test environment service |
|
||||||
| **3.3.0** | 2025-09-30 | Interactive guides system |
|
| **3.3.0** | 2025-09-30 | Interactive guides system |
|
||||||
|
|||||||
@ -35,22 +35,22 @@
|
|||||||
- [Package and Loader System](architecture/package-and-loader-system.md)
|
- [Package and Loader System](architecture/package-and-loader-system.md)
|
||||||
- [Config Loading Architecture](architecture/config-loading-architecture.md)
|
- [Config Loading Architecture](architecture/config-loading-architecture.md)
|
||||||
- [Nickel Executable Examples](architecture/nickel-executable-examples.md)
|
- [Nickel Executable Examples](architecture/nickel-executable-examples.md)
|
||||||
- [Orchestrator Info](architecture/orchestrator_info.md)
|
- [Orchestrator Info](architecture/orchestrator-info.md)
|
||||||
- [Orchestrator Auth Integration](architecture/orchestrator-auth-integration.md)
|
- [Orchestrator Auth Integration](architecture/orchestrator-auth-integration.md)
|
||||||
- [Repo Dist Analysis](architecture/repo-dist-analysis.md)
|
- [Repo Dist Analysis](architecture/repo-dist-analysis.md)
|
||||||
- [TypeDialog Nickel Integration](architecture/typedialog-nickel-integration.md)
|
- [TypeDialog Nickel Integration](architecture/typedialog-nickel-integration.md)
|
||||||
|
|
||||||
### Architecture Decision Records
|
### Architecture Decision Records
|
||||||
|
|
||||||
- [ADR-001: Project Structure](architecture/adr/ADR-001-project-structure.md)
|
- [ADR-001: Project Structure](architecture/adr/adr-001-project-structure.md)
|
||||||
- [ADR-002: Distribution Strategy](architecture/adr/ADR-002-distribution-strategy.md)
|
- [ADR-002: Distribution Strategy](architecture/adr/adr-002-distribution-strategy.md)
|
||||||
- [ADR-003: Workspace Isolation](architecture/adr/ADR-003-workspace-isolation.md)
|
- [ADR-003: Workspace Isolation](architecture/adr/adr-003-workspace-isolation.md)
|
||||||
- [ADR-004: Hybrid Architecture](architecture/adr/ADR-004-hybrid-architecture.md)
|
- [ADR-004: Hybrid Architecture](architecture/adr/adr-004-hybrid-architecture.md)
|
||||||
- [ADR-005: Extension Framework](architecture/adr/ADR-005-extension-framework.md)
|
- [ADR-005: Extension Framework](architecture/adr/adr-005-extension-framework.md)
|
||||||
- [ADR-006: Provisioning CLI Refactoring](architecture/adr/ADR-006-provisioning-cli-refactoring.md)
|
- [ADR-006: Provisioning CLI Refactoring](architecture/adr/adr-006-provisioning-cli-refactoring.md)
|
||||||
- [ADR-007: KMS Simplification](architecture/adr/ADR-007-kms-simplification.md)
|
- [ADR-007: KMS Simplification](architecture/adr/adr-007-kms-simplification.md)
|
||||||
- [ADR-008: Cedar Authorization](architecture/adr/ADR-008-cedar-authorization.md)
|
- [ADR-008: Cedar Authorization](architecture/adr/adr-008-cedar-authorization.md)
|
||||||
- [ADR-009: Security System Complete](architecture/adr/ADR-009-security-system-complete.md)
|
- [ADR-009: Security System Complete](architecture/adr/adr-009-security-system-complete.md)
|
||||||
- [ADR-010: Configuration Format Strategy](architecture/adr/ADR-010-configuration-format-strategy.md)
|
- [ADR-010: Configuration Format Strategy](architecture/adr/ADR-010-configuration-format-strategy.md)
|
||||||
- [ADR-011: Nickel Migration](architecture/adr/ADR-011-nickel-migration.md)
|
- [ADR-011: Nickel Migration](architecture/adr/ADR-011-nickel-migration.md)
|
||||||
- [ADR-012: Nushell Nickel Plugin CLI Wrapper](architecture/adr/adr-012-nushell-nickel-plugin-cli-wrapper.md)
|
- [ADR-012: Nushell Nickel Plugin CLI Wrapper](architecture/adr/adr-012-nushell-nickel-plugin-cli-wrapper.md)
|
||||||
@ -236,18 +236,8 @@
|
|||||||
### Multi-Provider Workspace Examples
|
### Multi-Provider Workspace Examples
|
||||||
|
|
||||||
- [Multi-Provider Web App Workspace](../examples/workspaces/multi-provider-web-app/README.md)
|
- [Multi-Provider Web App Workspace](../examples/workspaces/multi-provider-web-app/README.md)
|
||||||
- DigitalOcean (compute) + AWS (database) + Hetzner (storage)
|
|
||||||
- Cost: ~$165/month
|
|
||||||
|
|
||||||
- [Multi-Region High Availability Workspace](../examples/workspaces/multi-region-ha/README.md)
|
- [Multi-Region High Availability Workspace](../examples/workspaces/multi-region-ha/README.md)
|
||||||
- Global deployment across 3 providers (US, EU, APAC)
|
|
||||||
- 99.99% uptime with automatic failover
|
|
||||||
- Cost: ~$311/month
|
|
||||||
|
|
||||||
- [Cost-Optimized Multi-Provider Workspace](../examples/workspaces/cost-optimized/README.md)
|
- [Cost-Optimized Multi-Provider Workspace](../examples/workspaces/cost-optimized/README.md)
|
||||||
- Hetzner (compute) + AWS (managed services) + DigitalOcean (CDN)
|
|
||||||
- 53% cost savings vs all-AWS
|
|
||||||
- Cost: ~$280/month
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -134,7 +134,7 @@ See [Security Policies](security-policies.md) for complete details.
|
|||||||
## Supported LLM Providers
|
## Supported LLM Providers
|
||||||
|
|
||||||
| Provider | Models | Best For |
|
| Provider | Models | Best For |
|
||||||
|----------|--------|----------|
|
| ---------- | -------- | ---------- |
|
||||||
| **Anthropic** | Claude Sonnet 4, Claude Opus 4 | Complex configs, long context |
|
| **Anthropic** | Claude Sonnet 4, Claude Opus 4 | Complex configs, long context |
|
||||||
| **OpenAI** | GPT-4 Turbo, GPT-4 | Fast suggestions, tool calling |
|
| **OpenAI** | GPT-4 Turbo, GPT-4 | Fast suggestions, tool calling |
|
||||||
| **Local** | Llama 3, Mistral | Air-gapped, privacy-critical |
|
| **Local** | Llama 3, Mistral | Air-gapped, privacy-critical |
|
||||||
|
|||||||
1
docs/src/ai/ai-agents.md
Normal file
1
docs/src/ai/ai-agents.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# AI Agents
|
||||||
1
docs/src/ai/ai-assisted-forms.md
Normal file
1
docs/src/ai/ai-assisted-forms.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# AI-Assisted Forms
|
||||||
1
docs/src/ai/api-reference.md
Normal file
1
docs/src/ai/api-reference.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# API Reference
|
||||||
1
docs/src/ai/architecture.md
Normal file
1
docs/src/ai/architecture.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Architecture
|
||||||
1
docs/src/ai/config-generation.md
Normal file
1
docs/src/ai/config-generation.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Configuration Generation
|
||||||
1
docs/src/ai/configuration.md
Normal file
1
docs/src/ai/configuration.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Configuration
|
||||||
1
docs/src/ai/cost-management.md
Normal file
1
docs/src/ai/cost-management.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Cost Management
|
||||||
1
docs/src/ai/mcp-integration.md
Normal file
1
docs/src/ai/mcp-integration.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# MCP Integration
|
||||||
1
docs/src/ai/natural-language-config.md
Normal file
1
docs/src/ai/natural-language-config.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Natural Language Configuration
|
||||||
1
docs/src/ai/rag-system.md
Normal file
1
docs/src/ai/rag-system.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# RAG System
|
||||||
1
docs/src/ai/security-policies.md
Normal file
1
docs/src/ai/security-policies.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Security Policies
|
||||||
1
docs/src/ai/troubleshooting-with-ai.md
Normal file
1
docs/src/ai/troubleshooting-with-ai.md
Normal file
@ -0,0 +1 @@
|
|||||||
|
# Troubleshooting with AI
|
||||||
@ -36,7 +36,7 @@ extension-name/
|
|||||||
│ └── generate.nu # Generation commands
|
│ └── generate.nu # Generation commands
|
||||||
├── README.md # Extension documentation
|
├── README.md # Extension documentation
|
||||||
└── metadata.toml # Extension metadata
|
└── metadata.toml # Extension metadata
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Provider Extension API
|
## Provider Extension API
|
||||||
|
|
||||||
@ -140,7 +140,7 @@ schema ServerConfig {
|
|||||||
bandwidth?: int
|
bandwidth?: int
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Nushell Implementation
|
#### Nushell Implementation
|
||||||
|
|
||||||
@ -227,7 +227,7 @@ export def "test-connection" [config: record] -> record {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Create `nulib/create.nu`:
|
Create `nulib/create.nu`:
|
||||||
|
|
||||||
@ -362,7 +362,7 @@ def wait-for-server-ready [server_id: string] -> string {
|
|||||||
|
|
||||||
error make { msg: "Server creation timeout" }
|
error make { msg: "Server creation timeout" }
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Provider Registration
|
### Provider Registration
|
||||||
|
|
||||||
@ -400,7 +400,7 @@ available = ["us-east-1", "us-west-2", "eu-west-1"]
|
|||||||
[support]
|
[support]
|
||||||
documentation = "https://docs.example.com/provider"
|
documentation = "https://docs.example.com/provider"
|
||||||
issues = "https://github.com/example/provider/issues"
|
issues = "https://github.com/example/provider/issues"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Task Service Extension API
|
## Task Service Extension API
|
||||||
|
|
||||||
@ -477,7 +477,7 @@ Create `schemas/version.ncl`:
|
|||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Nushell Implementation
|
#### Nushell Implementation
|
||||||
|
|
||||||
@ -669,7 +669,7 @@ def check-health [] -> record {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Cluster Extension API
|
## Cluster Extension API
|
||||||
|
|
||||||
@ -806,7 +806,7 @@ Create `schemas/cluster.ncl`:
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Nushell Implementation
|
#### Nushell Implementation
|
||||||
|
|
||||||
@ -1010,7 +1010,7 @@ def resolve-component-dependencies [components: list<record>] -> list<record> {
|
|||||||
|
|
||||||
$sorted
|
$sorted
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Extension Registration and Discovery
|
## Extension Registration and Discovery
|
||||||
|
|
||||||
@ -1090,7 +1090,7 @@ export def test_server_creation_check_mode [] {
|
|||||||
assert ($result.check_mode == true)
|
assert ($result.check_mode == true)
|
||||||
assert ($result.would_create == true)
|
assert ($result.would_create == true)
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Integration Tests
|
#### Integration Tests
|
||||||
|
|
||||||
@ -1123,7 +1123,7 @@ export def test_full_server_lifecycle [] {
|
|||||||
let final_info = try { get-server-info $server_id } catch { null }
|
let final_info = try { get-server-info $server_id } catch { null }
|
||||||
assert ($final_info == null)
|
assert ($final_info == null)
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Running Tests
|
### Running Tests
|
||||||
|
|
||||||
@ -1136,7 +1136,7 @@ nu tests/integration_tests.nu
|
|||||||
|
|
||||||
# Run all tests
|
# Run all tests
|
||||||
nu tests/run_all_tests.nu
|
nu tests/run_all_tests.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Documentation Requirements
|
## Documentation Requirements
|
||||||
|
|
||||||
@ -1171,7 +1171,7 @@ Common usage patterns and examples.
|
|||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
Common issues and solutions.
|
Common issues and solutions.
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Best Practices
|
## Best Practices
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Integration Examples
|
# Integration Examples
|
||||||
|
|
||||||
This document provides comprehensive examples and patterns for integrating with provisioning APIs, including client libraries, SDKs, error handling strategies, and performance optimization.
|
This document provides comprehensive examples and patterns for integrating with provisioning APIs, including client libraries, SDKs, error handling
|
||||||
|
strategies, and performance optimization.
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
@ -1587,4 +1588,5 @@ class EventDrivenWorkflowManager {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
This comprehensive integration documentation provides developers with everything needed to successfully integrate with provisioning, including complete client implementations, error handling strategies, performance optimizations, and common integration patterns.
|
This comprehensive integration documentation provides developers with everything needed to successfully integrate with provisioning, including
|
||||||
|
complete client implementations, error handling strategies, performance optimizations, and common integration patterns.
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Path Resolution API
|
# Path Resolution API
|
||||||
|
|
||||||
This document describes the path resolution system used throughout the provisioning infrastructure for discovering configurations, extensions, and resolving workspace paths.
|
This document describes the path resolution system used throughout the provisioning infrastructure for discovering configurations, extensions, and
|
||||||
|
resolving workspace paths.
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
@ -23,7 +24,7 @@ The system follows a specific hierarchy for loading configuration files:
|
|||||||
4. Infrastructure config (infra/config.toml)
|
4. Infrastructure config (infra/config.toml)
|
||||||
5. Environment config (config.{env}.toml)
|
5. Environment config (config.{env}.toml)
|
||||||
6. Runtime overrides (CLI arguments, ENV vars)
|
6. Runtime overrides (CLI arguments, ENV vars)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Search Paths
|
### Configuration Search Paths
|
||||||
|
|
||||||
@ -36,7 +37,7 @@ $HOME/.config/provisioning/config.user.toml
|
|||||||
$PWD/config.project.toml
|
$PWD/config.project.toml
|
||||||
$PROVISIONING_KLOUD_PATH/config.infra.toml
|
$PROVISIONING_KLOUD_PATH/config.infra.toml
|
||||||
$PWD/config.{PROVISIONING_ENV}.toml
|
$PWD/config.{PROVISIONING_ENV}.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Path Resolution API
|
## Path Resolution API
|
||||||
|
|
||||||
@ -62,7 +63,7 @@ Resolves configuration file paths using the search hierarchy.
|
|||||||
use path-resolution.nu *
|
use path-resolution.nu *
|
||||||
let config_path = (resolve-config-path "config.user.toml" [])
|
let config_path = (resolve-config-path "config.user.toml" [])
|
||||||
# Returns: "/home/user/.config/provisioning/config.user.toml"
|
# Returns: "/home/user/.config/provisioning/config.user.toml"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### `resolve-extension-path(type: string, name: string) -> record`
|
#### `resolve-extension-path(type: string, name: string) -> record`
|
||||||
|
|
||||||
@ -83,7 +84,7 @@ Discovers extension paths (providers, taskservs, clusters).
|
|||||||
templates_path: "/usr/local/provisioning/providers/upcloud/templates",
|
templates_path: "/usr/local/provisioning/providers/upcloud/templates",
|
||||||
exists: true
|
exists: true
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### `resolve-workspace-paths() -> record`
|
#### `resolve-workspace-paths() -> record`
|
||||||
|
|
||||||
@ -101,7 +102,7 @@ Gets current workspace path configuration.
|
|||||||
clusters: "/usr/local/provisioning/cluster",
|
clusters: "/usr/local/provisioning/cluster",
|
||||||
extensions: "/workspace/extensions"
|
extensions: "/workspace/extensions"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Path Interpolation
|
### Path Interpolation
|
||||||
|
|
||||||
@ -137,7 +138,7 @@ let result = (interpolate-path $template {
|
|||||||
git: { branch: "main" }
|
git: { branch: "main" }
|
||||||
})
|
})
|
||||||
# Returns: "/usr/local/provisioning/infra/admin/main"
|
# Returns: "/usr/local/provisioning/infra/admin/main"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Extension Discovery API
|
## Extension Discovery API
|
||||||
|
|
||||||
@ -172,7 +173,7 @@ Discovers all available providers.
|
|||||||
has_templates: true
|
has_templates: true
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### `get-provider-config(name: string) -> record`
|
#### `get-provider-config(name: string) -> record`
|
||||||
|
|
||||||
@ -203,7 +204,7 @@ Gets provider-specific configuration and paths.
|
|||||||
description: "UpCloud provider for server provisioning"
|
description: "UpCloud provider for server provisioning"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Task Service Discovery
|
### Task Service Discovery
|
||||||
|
|
||||||
@ -232,7 +233,7 @@ Discovers all available task services.
|
|||||||
enabled: true
|
enabled: true
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### `get-taskserv-config(name: string) -> record`
|
#### `get-taskserv-config(name: string) -> record`
|
||||||
|
|
||||||
@ -261,7 +262,7 @@ Gets task service configuration and version information.
|
|||||||
supports_versions: ["1.26.x", "1.27.x", "1.28.x"]
|
supports_versions: ["1.26.x", "1.27.x", "1.28.x"]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Cluster Discovery
|
### Cluster Discovery
|
||||||
|
|
||||||
@ -282,7 +283,7 @@ Discovers all available cluster configurations.
|
|||||||
enabled: true
|
enabled: true
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Environment Management API
|
## Environment Management API
|
||||||
|
|
||||||
@ -329,7 +330,7 @@ Gets environment-specific configuration.
|
|||||||
rollback: true
|
rollback: true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Environment Switching
|
### Environment Switching
|
||||||
|
|
||||||
@ -377,7 +378,7 @@ Discovers available workspaces and infrastructure directories.
|
|||||||
valid: true
|
valid: true
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### `set-current-workspace(path: string) -> null`
|
#### `set-current-workspace(path: string) -> null`
|
||||||
|
|
||||||
@ -431,7 +432,7 @@ Analyzes project structure and identifies components.
|
|||||||
"config.prod.toml"
|
"config.prod.toml"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Caching and Performance
|
## Caching and Performance
|
||||||
|
|
||||||
@ -464,7 +465,7 @@ Gets path resolution cache statistics.
|
|||||||
hit_rate: 0.85,
|
hit_rate: 0.85,
|
||||||
last_invalidated: "2025-09-26T10:00:00Z"
|
last_invalidated: "2025-09-26T10:00:00Z"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Cross-Platform Compatibility
|
## Cross-Platform Compatibility
|
||||||
|
|
||||||
@ -490,7 +491,7 @@ normalize-path "path/to/file" # Returns: "path\to\file"
|
|||||||
|
|
||||||
# On Unix
|
# On Unix
|
||||||
normalize-path "path\to\file" # Returns: "path/to/file"
|
normalize-path "path\to\file" # Returns: "path/to/file"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### `join-paths(segments: list<string>) -> string`
|
#### `join-paths(segments: list<string>) -> string`
|
||||||
|
|
||||||
@ -527,7 +528,7 @@ Validates all paths in configuration.
|
|||||||
],
|
],
|
||||||
checks_performed: 15
|
checks_performed: 15
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### `validate-extension-structure(type: string, path: string) -> record`
|
#### `validate-extension-structure(type: string, path: string) -> record`
|
||||||
|
|
||||||
@ -552,7 +553,7 @@ Validates extension directory structure.
|
|||||||
{ file: "templates/server.j2", exists: false }
|
{ file: "templates/server.j2", exists: false }
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Command-Line Interface
|
## Command-Line Interface
|
||||||
|
|
||||||
@ -577,7 +578,7 @@ provisioning env switch prod
|
|||||||
|
|
||||||
# Set workspace
|
# Set workspace
|
||||||
provisioning workspace set /path/to/infra
|
provisioning workspace set /path/to/infra
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Integration Examples
|
## Integration Examples
|
||||||
|
|
||||||
@ -607,7 +608,7 @@ class PathResolver:
|
|||||||
resolver = PathResolver()
|
resolver = PathResolver()
|
||||||
paths = resolver.get_paths()
|
paths = resolver.get_paths()
|
||||||
providers = resolver.discover_providers()
|
providers = resolver.discover_providers()
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### JavaScript/Node.js Integration
|
### JavaScript/Node.js Integration
|
||||||
|
|
||||||
@ -640,7 +641,7 @@ class PathResolver {
|
|||||||
const resolver = new PathResolver();
|
const resolver = new PathResolver();
|
||||||
const paths = await resolver.getPaths();
|
const paths = await resolver.getPaths();
|
||||||
const providers = await resolver.discoverExtensions('providers');
|
const providers = await resolver.discoverExtensions('providers');
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Error Handling
|
## Error Handling
|
||||||
|
|
||||||
@ -705,7 +706,7 @@ provisioning debug cache-stats
|
|||||||
|
|
||||||
# Profile path resolution
|
# Profile path resolution
|
||||||
provisioning debug profile-paths
|
provisioning debug profile-paths
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Security Considerations
|
## Security Considerations
|
||||||
|
|
||||||
@ -725,4 +726,5 @@ Path resolution respects file system permissions:
|
|||||||
- Extension directories require read/execute access
|
- Extension directories require read/execute access
|
||||||
- Workspace directories may require write access for operations
|
- Workspace directories may require write access for operations
|
||||||
|
|
||||||
This path resolution API provides a comprehensive and flexible system for managing the complex path requirements of multi-provider, multi-environment infrastructure provisioning.
|
This path resolution API provides a comprehensive and flexible system for managing the complex path requirements of multi-provider, multi-environment
|
||||||
|
infrastructure provisioning.
|
||||||
|
|||||||
@ -31,7 +31,7 @@ export def list-servers [] -> table { ... }
|
|||||||
export def get-server-plans [] -> table { ... }
|
export def get-server-plans [] -> table { ... }
|
||||||
export def get-regions [] -> list { ... }
|
export def get-regions [] -> list { ... }
|
||||||
export def get-pricing [plan: string] -> record { ... }
|
export def get-pricing [plan: string] -> record { ... }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Provider Configuration
|
### Provider Configuration
|
||||||
|
|
||||||
@ -51,7 +51,7 @@ Each provider requires configuration in Nickel format:
|
|||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Creating a Custom Provider
|
## Creating a Custom Provider
|
||||||
|
|
||||||
@ -65,7 +65,7 @@ provisioning/extensions/providers/my-provider/
|
|||||||
│ ├── main.ncl # Nickel schema
|
│ ├── main.ncl # Nickel schema
|
||||||
│ └── defaults.ncl # Default configuration
|
│ └── defaults.ncl # Default configuration
|
||||||
└── README.md # Provider documentation
|
└── README.md # Provider documentation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Implementation Template
|
### 2. Implementation Template
|
||||||
|
|
||||||
@ -90,7 +90,7 @@ export def list-servers [] {
|
|||||||
}
|
}
|
||||||
|
|
||||||
# ... other required functions
|
# ... other required functions
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Nickel Schema
|
### 3. Nickel Schema
|
||||||
|
|
||||||
@ -109,7 +109,7 @@ export def list-servers [] {
|
|||||||
region | String = "us-east-1",
|
region | String = "us-east-1",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Provider Discovery
|
## Provider Discovery
|
||||||
|
|
||||||
@ -124,7 +124,7 @@ provisioning module discover providers
|
|||||||
|
|
||||||
# Load provider
|
# Load provider
|
||||||
provisioning module load providers workspace my-provider
|
provisioning module load providers workspace my-provider
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Provider API Examples
|
## Provider API Examples
|
||||||
|
|
||||||
@ -140,19 +140,19 @@ let plan = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
create-servers $plan
|
create-servers $plan
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### List Servers
|
### List Servers
|
||||||
|
|
||||||
```nushell
|
```nushell
|
||||||
list-servers | where status == "running" | select hostname ip_address
|
list-servers | where status == "running" | select hostname ip_address
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Get Pricing
|
### Get Pricing
|
||||||
|
|
||||||
```nushell
|
```nushell
|
||||||
get-pricing "small" | to yaml
|
get-pricing "small" | to yaml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Testing Providers
|
## Testing Providers
|
||||||
|
|
||||||
@ -161,7 +161,7 @@ Use the test environment system to test providers:
|
|||||||
```bash
|
```bash
|
||||||
# Test provider without real resources
|
# Test provider without real resources
|
||||||
provisioning test env single my-provider --check
|
provisioning test env single my-provider --check
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Provider Development Guide
|
## Provider Development Guide
|
||||||
|
|
||||||
|
|||||||
@ -22,7 +22,7 @@ All API endpoints (except health checks) require JWT authentication via the Auth
|
|||||||
|
|
||||||
```http
|
```http
|
||||||
Authorization: Bearer <jwt_token>
|
Authorization: Bearer <jwt_token>
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Getting Access Token
|
### Getting Access Token
|
||||||
|
|
||||||
@ -35,7 +35,7 @@ Content-Type: application/json
|
|||||||
"password": "password",
|
"password": "password",
|
||||||
"mfa_code": "123456"
|
"mfa_code": "123456"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Orchestrator API Endpoints
|
## Orchestrator API Endpoints
|
||||||
|
|
||||||
@ -52,7 +52,7 @@ Check orchestrator health status.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "Orchestrator is healthy"
|
"data": "Orchestrator is healthy"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Task Management
|
### Task Management
|
||||||
|
|
||||||
@ -87,7 +87,7 @@ List all workflow tasks.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /tasks/{id}
|
#### GET /tasks/{id}
|
||||||
|
|
||||||
@ -116,7 +116,7 @@ Get specific task status and details.
|
|||||||
"error": null
|
"error": null
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Workflow Submission
|
### Workflow Submission
|
||||||
|
|
||||||
@ -133,7 +133,7 @@ Submit server creation workflow.
|
|||||||
"check_mode": false,
|
"check_mode": false,
|
||||||
"wait": true
|
"wait": true
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -142,7 +142,7 @@ Submit server creation workflow.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "uuid-task-id"
|
"data": "uuid-task-id"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /workflows/taskserv/create
|
#### POST /workflows/taskserv/create
|
||||||
|
|
||||||
@ -159,7 +159,7 @@ Submit task service workflow.
|
|||||||
"check_mode": false,
|
"check_mode": false,
|
||||||
"wait": true
|
"wait": true
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -168,7 +168,7 @@ Submit task service workflow.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "uuid-task-id"
|
"data": "uuid-task-id"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /workflows/cluster/create
|
#### POST /workflows/cluster/create
|
||||||
|
|
||||||
@ -185,7 +185,7 @@ Submit cluster workflow.
|
|||||||
"check_mode": false,
|
"check_mode": false,
|
||||||
"wait": true
|
"wait": true
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -194,7 +194,7 @@ Submit cluster workflow.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "uuid-task-id"
|
"data": "uuid-task-id"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Batch Operations
|
### Batch Operations
|
||||||
|
|
||||||
@ -231,7 +231,7 @@ Execute batch workflow operation.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -255,7 +255,7 @@ Execute batch workflow operation.
|
|||||||
]
|
]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /batch/operations
|
#### GET /batch/operations
|
||||||
|
|
||||||
@ -276,7 +276,7 @@ List all batch operations.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /batch/operations/{id}
|
#### GET /batch/operations/{id}
|
||||||
|
|
||||||
@ -305,7 +305,7 @@ Get batch operation status.
|
|||||||
]
|
]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /batch/operations/{id}/cancel
|
#### POST /batch/operations/{id}/cancel
|
||||||
|
|
||||||
@ -322,7 +322,7 @@ Cancel running batch operation.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "Operation cancelled"
|
"data": "Operation cancelled"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### State Management
|
### State Management
|
||||||
|
|
||||||
@ -348,7 +348,7 @@ Get real-time workflow progress.
|
|||||||
"estimated_time_remaining": 180
|
"estimated_time_remaining": 180
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /state/workflows/{id}/snapshots
|
#### GET /state/workflows/{id}/snapshots
|
||||||
|
|
||||||
@ -372,7 +372,7 @@ Get workflow state snapshots.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /state/system/metrics
|
#### GET /state/system/metrics
|
||||||
|
|
||||||
@ -395,7 +395,7 @@ Get system-wide metrics.
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /state/system/health
|
#### GET /state/system/health
|
||||||
|
|
||||||
@ -416,7 +416,7 @@ Get system health status.
|
|||||||
"last_check": "2025-09-26T10:00:00Z"
|
"last_check": "2025-09-26T10:00:00Z"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /state/statistics
|
#### GET /state/statistics
|
||||||
|
|
||||||
@ -434,7 +434,7 @@ Get state manager statistics.
|
|||||||
"average_workflow_duration": 300
|
"average_workflow_duration": 300
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Rollback and Recovery
|
### Rollback and Recovery
|
||||||
|
|
||||||
@ -449,7 +449,7 @@ Create new checkpoint.
|
|||||||
"name": "before_major_update",
|
"name": "before_major_update",
|
||||||
"description": "Checkpoint before deploying v2.0.0"
|
"description": "Checkpoint before deploying v2.0.0"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -458,7 +458,7 @@ Create new checkpoint.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "checkpoint-uuid"
|
"data": "checkpoint-uuid"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /rollback/checkpoints
|
#### GET /rollback/checkpoints
|
||||||
|
|
||||||
@ -479,7 +479,7 @@ List all checkpoints.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /rollback/checkpoints/{id}
|
#### GET /rollback/checkpoints/{id}
|
||||||
|
|
||||||
@ -503,7 +503,7 @@ Get specific checkpoint details.
|
|||||||
"operations_count": 25
|
"operations_count": 25
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /rollback/execute
|
#### POST /rollback/execute
|
||||||
|
|
||||||
@ -515,7 +515,7 @@ Execute rollback operation.
|
|||||||
{
|
{
|
||||||
"checkpoint_id": "checkpoint-uuid"
|
"checkpoint_id": "checkpoint-uuid"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Or for partial rollback:
|
Or for partial rollback:
|
||||||
|
|
||||||
@ -523,7 +523,7 @@ Or for partial rollback:
|
|||||||
{
|
{
|
||||||
"operation_ids": ["op-1", "op-2", "op-3"]
|
"operation_ids": ["op-1", "op-2", "op-3"]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -538,7 +538,7 @@ Or for partial rollback:
|
|||||||
"duration": 45.5
|
"duration": 45.5
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /rollback/restore/{id}
|
#### POST /rollback/restore/{id}
|
||||||
|
|
||||||
@ -555,7 +555,7 @@ Restore system state from checkpoint.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "State restored from checkpoint checkpoint-uuid"
|
"data": "State restored from checkpoint checkpoint-uuid"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### GET /rollback/statistics
|
#### GET /rollback/statistics
|
||||||
|
|
||||||
@ -573,7 +573,7 @@ Get rollback system statistics.
|
|||||||
"average_rollback_time": 30.5
|
"average_rollback_time": 30.5
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Control Center API Endpoints
|
## Control Center API Endpoints
|
||||||
|
|
||||||
@ -591,7 +591,7 @@ Authenticate user and get JWT token.
|
|||||||
"password": "secure_password",
|
"password": "secure_password",
|
||||||
"mfa_code": "123456"
|
"mfa_code": "123456"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -609,7 +609,7 @@ Authenticate user and get JWT token.
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /auth/refresh
|
#### POST /auth/refresh
|
||||||
|
|
||||||
@ -621,7 +621,7 @@ Refresh JWT token.
|
|||||||
{
|
{
|
||||||
"token": "current-jwt-token"
|
"token": "current-jwt-token"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -633,7 +633,7 @@ Refresh JWT token.
|
|||||||
"expires_at": "2025-09-26T18:00:00Z"
|
"expires_at": "2025-09-26T18:00:00Z"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /auth/logout
|
#### POST /auth/logout
|
||||||
|
|
||||||
@ -646,7 +646,7 @@ Logout and invalidate token.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "Successfully logged out"
|
"data": "Successfully logged out"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### User Management
|
### User Management
|
||||||
|
|
||||||
@ -676,7 +676,7 @@ List all users.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /users
|
#### POST /users
|
||||||
|
|
||||||
@ -692,7 +692,7 @@ Create new user.
|
|||||||
"roles": ["operator"],
|
"roles": ["operator"],
|
||||||
"enabled": true
|
"enabled": true
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -707,7 +707,7 @@ Create new user.
|
|||||||
"enabled": true
|
"enabled": true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### PUT /users/{id}
|
#### PUT /users/{id}
|
||||||
|
|
||||||
@ -725,7 +725,7 @@ Update existing user.
|
|||||||
"roles": ["admin", "operator"],
|
"roles": ["admin", "operator"],
|
||||||
"enabled": false
|
"enabled": false
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -734,7 +734,7 @@ Update existing user.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "User updated successfully"
|
"data": "User updated successfully"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### DELETE /users/{id}
|
#### DELETE /users/{id}
|
||||||
|
|
||||||
@ -751,7 +751,7 @@ Delete user.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "User deleted successfully"
|
"data": "User deleted successfully"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Policy Management
|
### Policy Management
|
||||||
|
|
||||||
@ -775,7 +775,7 @@ List all policies.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### POST /policies
|
#### POST /policies
|
||||||
|
|
||||||
@ -796,7 +796,7 @@ Create new policy.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -809,7 +809,7 @@ Create new policy.
|
|||||||
"version": "1.0.0"
|
"version": "1.0.0"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### PUT /policies/{id}
|
#### PUT /policies/{id}
|
||||||
|
|
||||||
@ -826,7 +826,7 @@ Update policy.
|
|||||||
"name": "updated_policy",
|
"name": "updated_policy",
|
||||||
"rules": [...]
|
"rules": [...]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
|
|
||||||
@ -835,7 +835,7 @@ Update policy.
|
|||||||
"success": true,
|
"success": true,
|
||||||
"data": "Policy updated successfully"
|
"data": "Policy updated successfully"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Audit Logging
|
### Audit Logging
|
||||||
|
|
||||||
@ -870,7 +870,7 @@ Get audit logs.
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Error Responses
|
## Error Responses
|
||||||
|
|
||||||
@ -881,7 +881,7 @@ All endpoints may return error responses in this format:
|
|||||||
"success": false,
|
"success": false,
|
||||||
"error": "Detailed error message"
|
"error": "Detailed error message"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### HTTP Status Codes
|
### HTTP Status Codes
|
||||||
|
|
||||||
@ -908,7 +908,7 @@ Rate limit headers are included in responses:
|
|||||||
X-RateLimit-Limit: 100
|
X-RateLimit-Limit: 100
|
||||||
X-RateLimit-Remaining: 95
|
X-RateLimit-Remaining: 95
|
||||||
X-RateLimit-Reset: 1632150000
|
X-RateLimit-Reset: 1632150000
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Monitoring Endpoints
|
## Monitoring Endpoints
|
||||||
|
|
||||||
@ -929,7 +929,7 @@ orchestrator_tasks_total{status="failed"} 5
|
|||||||
orchestrator_task_duration_seconds_bucket{le="10"} 50
|
orchestrator_task_duration_seconds_bucket{le="10"} 50
|
||||||
orchestrator_task_duration_seconds_bucket{le="30"} 120
|
orchestrator_task_duration_seconds_bucket{le="30"} 120
|
||||||
orchestrator_task_duration_seconds_bucket{le="+Inf"} 155
|
orchestrator_task_duration_seconds_bucket{le="+Inf"} 155
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### WebSocket /ws
|
### WebSocket /ws
|
||||||
|
|
||||||
@ -944,7 +944,7 @@ ws.onmessage = function(event) {
|
|||||||
const data = JSON.parse(event.data);
|
const data = JSON.parse(event.data);
|
||||||
console.log('Event:', data);
|
console.log('Event:', data);
|
||||||
};
|
};
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Event Format:**
|
**Event Format:**
|
||||||
|
|
||||||
@ -961,7 +961,7 @@ ws.onmessage = function(event) {
|
|||||||
"status": "completed"
|
"status": "completed"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## SDK Examples
|
## SDK Examples
|
||||||
|
|
||||||
@ -1003,7 +1003,7 @@ class ProvisioningClient:
|
|||||||
client = ProvisioningClient('http://localhost:9090', 'your-jwt-token')
|
client = ProvisioningClient('http://localhost:9090', 'your-jwt-token')
|
||||||
result = client.create_server_workflow('production', 'config.ncl')
|
result = client.create_server_workflow('production', 'config.ncl')
|
||||||
print(f"Task ID: {result['data']}")
|
print(f"Task ID: {result['data']}")
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### JavaScript/Node.js SDK Example
|
### JavaScript/Node.js SDK Example
|
||||||
|
|
||||||
@ -1041,7 +1041,7 @@ class ProvisioningClient {
|
|||||||
const client = new ProvisioningClient('http://localhost:9090', 'your-jwt-token');
|
const client = new ProvisioningClient('http://localhost:9090', 'your-jwt-token');
|
||||||
const result = await client.createServerWorkflow('production', 'config.ncl');
|
const result = await client.createServerWorkflow('production', 'config.ncl');
|
||||||
console.log(`Task ID: ${result.data}`);
|
console.log(`Task ID: ${result.data}`);
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Webhook Integration
|
## Webhook Integration
|
||||||
|
|
||||||
@ -1061,7 +1061,7 @@ endpoints = [
|
|||||||
secret = "webhook-secret"
|
secret = "webhook-secret"
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Webhook Payload
|
### Webhook Payload
|
||||||
|
|
||||||
@ -1076,7 +1076,7 @@ endpoints = [
|
|||||||
},
|
},
|
||||||
"signature": "sha256=calculated-signature"
|
"signature": "sha256=calculated-signature"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Pagination
|
## Pagination
|
||||||
|
|
||||||
@ -1092,7 +1092,7 @@ X-Total-Count: 1500
|
|||||||
X-Limit: 50
|
X-Limit: 50
|
||||||
X-Offset: 100
|
X-Offset: 100
|
||||||
Link: </api/endpoint?offset=150&limit=50>; rel="next"
|
Link: </api/endpoint?offset=150&limit=50>; rel="next"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## API Versioning
|
## API Versioning
|
||||||
|
|
||||||
@ -1100,7 +1100,7 @@ The API uses header-based versioning:
|
|||||||
|
|
||||||
```http
|
```http
|
||||||
Accept: application/vnd.provisioning.v1+json
|
Accept: application/vnd.provisioning.v1+json
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Current version: v1
|
Current version: v1
|
||||||
|
|
||||||
@ -1115,4 +1115,4 @@ cargo test --test api_tests
|
|||||||
|
|
||||||
# Run load tests
|
# Run load tests
|
||||||
cargo test --test load_tests --release
|
cargo test --test load_tests --release
|
||||||
```plaintext
|
```
|
||||||
|
|||||||
@ -1084,4 +1084,5 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|||||||
3. **Error Scenarios**: Test error handling paths
|
3. **Error Scenarios**: Test error handling paths
|
||||||
4. **Load Testing**: Validate performance under load
|
4. **Load Testing**: Validate performance under load
|
||||||
|
|
||||||
This comprehensive SDK documentation provides developers with everything needed to integrate with provisioning using their preferred programming language, complete with examples, best practices, and detailed API references.
|
This comprehensive SDK documentation provides developers with everything needed to integrate with provisioning using their preferred programming
|
||||||
|
language, complete with examples, best practices, and detailed API references.
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# WebSocket API Reference
|
# WebSocket API Reference
|
||||||
|
|
||||||
This document provides comprehensive documentation for the WebSocket API used for real-time monitoring, event streaming, and live updates in provisioning.
|
This document provides comprehensive documentation for the WebSocket API used for real-time monitoring, event streaming, and live updates in
|
||||||
|
provisioning.
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
@ -887,4 +888,5 @@ The server implements rate limiting to prevent abuse:
|
|||||||
- Sensitive information is filtered based on user permissions
|
- Sensitive information is filtered based on user permissions
|
||||||
- PII and secrets are never transmitted
|
- PII and secrets are never transmitted
|
||||||
|
|
||||||
This WebSocket API provides a robust, real-time communication channel for monitoring and managing provisioning with comprehensive security and performance features.
|
This WebSocket API provides a robust, real-time communication channel for monitoring and managing provisioning with comprehensive security and
|
||||||
|
performance features.
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Architecture Documentation
|
# Architecture Documentation
|
||||||
|
|
||||||
This directory contains comprehensive architecture documentation for provisioning, including Architecture Decision Records (ADRs) and system design documentation.
|
This directory contains comprehensive architecture documentation for provisioning, including Architecture Decision Records (ADRs) and system design
|
||||||
|
documentation.
|
||||||
|
|
||||||
## Architecture Decision Records (ADRs)
|
## Architecture Decision Records (ADRs)
|
||||||
|
|
||||||
@ -125,4 +126,5 @@ All significant architectural changes follow a review process:
|
|||||||
4. **Documentation Phase**: Update related documentation and integration patterns
|
4. **Documentation Phase**: Update related documentation and integration patterns
|
||||||
5. **Implementation Phase**: Guide implementation according to architectural decisions
|
5. **Implementation Phase**: Guide implementation according to architectural decisions
|
||||||
|
|
||||||
This architecture documentation represents the collective wisdom and experience of building a sophisticated, production-ready infrastructure automation platform.
|
This architecture documentation represents the collective wisdom and experience of building a sophisticated, production-ready infrastructure
|
||||||
|
automation platform.
|
||||||
|
|||||||
@ -37,7 +37,7 @@ src/
|
|||||||
├── control-center/ # Web UI management interface
|
├── control-center/ # Web UI management interface
|
||||||
├── tools/ # Development and utility tools
|
├── tools/ # Development and utility tools
|
||||||
└── extensions/ # Plugin and extension framework
|
└── extensions/ # Plugin and extension framework
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Key Structural Principles
|
### Key Structural Principles
|
||||||
|
|
||||||
|
|||||||
@ -75,7 +75,7 @@ Implement a **layered distribution strategy** with clear separation between deve
|
|||||||
├── scripts/ # Development tools
|
├── scripts/ # Development tools
|
||||||
├── tests/ # Test suites
|
├── tests/ # Test suites
|
||||||
└── tools/ # Build and development utilities
|
└── tools/ # Build and development utilities
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Key Distribution Principles
|
### Key Distribution Principles
|
||||||
|
|
||||||
@ -160,7 +160,7 @@ System Defaults (lowest precedence)
|
|||||||
└── Infrastructure Configuration
|
└── Infrastructure Configuration
|
||||||
└── Environment Configuration
|
└── Environment Configuration
|
||||||
└── Runtime Configuration (highest precedence)
|
└── Runtime Configuration (highest precedence)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Workspace Management
|
### Workspace Management
|
||||||
|
|
||||||
|
|||||||
@ -6,7 +6,8 @@ Accepted
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
Provisioning required a clear strategy for managing user-specific data, configurations, and customizations separate from system-wide installations. Key challenges included:
|
Provisioning required a clear strategy for managing user-specific data, configurations,
|
||||||
|
and customizations separate from system-wide installations. Key challenges included:
|
||||||
|
|
||||||
1. **Configuration Conflicts**: User settings mixed with system defaults, causing unclear precedence
|
1. **Configuration Conflicts**: User settings mixed with system defaults, causing unclear precedence
|
||||||
2. **State Management**: User state (cache, logs, temporary files) scattered across filesystem
|
2. **State Management**: User state (cache, logs, temporary files) scattered across filesystem
|
||||||
@ -57,7 +58,7 @@ Implement **isolated user workspaces** with clear boundaries and hierarchical co
|
|||||||
├── logs/ # User-specific logs
|
├── logs/ # User-specific logs
|
||||||
├── state/ # Local state files
|
├── state/ # Local state files
|
||||||
└── backups/ # Automatic workspace backups
|
└── backups/ # Automatic workspace backups
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Hierarchy (Precedence Order)
|
### Configuration Hierarchy (Precedence Order)
|
||||||
|
|
||||||
@ -150,7 +151,7 @@ provisioning workspace init --template=developer
|
|||||||
# Workspace status and validation
|
# Workspace status and validation
|
||||||
provisioning workspace status
|
provisioning workspace status
|
||||||
provisioning workspace validate
|
provisioning workspace validate
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Resolution Process
|
### Configuration Resolution Process
|
||||||
|
|
||||||
@ -171,7 +172,7 @@ provisioning workspace restore --input ~/backup/provisioning-workspace.tar.gz
|
|||||||
|
|
||||||
# Migrate workspace to new version
|
# Migrate workspace to new version
|
||||||
provisioning workspace migrate --from-version 2.0.0 --to-version 3.0.0
|
provisioning workspace migrate --from-version 2.0.0 --to-version 3.0.0
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Security Considerations
|
### Security Considerations
|
||||||
|
|
||||||
|
|||||||
@ -8,7 +8,8 @@ Accepted
|
|||||||
|
|
||||||
Provisioning encountered fundamental limitations with a pure Nushell implementation that required architectural solutions:
|
Provisioning encountered fundamental limitations with a pure Nushell implementation that required architectural solutions:
|
||||||
|
|
||||||
1. **Deep Call Stack Limitations**: Nushell's `open` command fails in deep call contexts (`enumerate | each`), causing "Type not supported" errors in template.nu:71
|
1. **Deep Call Stack Limitations**: Nushell's `open` command fails in deep call contexts
|
||||||
|
(`enumerate | each`), causing "Type not supported" errors in template.nu:71
|
||||||
2. **Performance Bottlenecks**: Complex workflow orchestration hitting Nushell's performance limits
|
2. **Performance Bottlenecks**: Complex workflow orchestration hitting Nushell's performance limits
|
||||||
3. **Concurrency Constraints**: Limited parallel processing capabilities in Nushell for batch operations
|
3. **Concurrency Constraints**: Limited parallel processing capabilities in Nushell for batch operations
|
||||||
4. **Integration Complexity**: Need for REST API endpoints and external system integration
|
4. **Integration Complexity**: Need for REST API endpoints and external system integration
|
||||||
@ -123,7 +124,8 @@ http post "http://localhost:9090/workflows/servers/create" {
|
|||||||
### Alternative 1: Pure Nushell Implementation
|
### Alternative 1: Pure Nushell Implementation
|
||||||
|
|
||||||
Continue with Nushell-only approach and work around limitations.
|
Continue with Nushell-only approach and work around limitations.
|
||||||
**Rejected**: Technical limitations are fundamental and cannot be worked around without compromising functionality. Deep call stack issues are architectural.
|
**Rejected**: Technical limitations are fundamental and cannot be worked around without compromising functionality. Deep call stack issues are
|
||||||
|
architectural.
|
||||||
|
|
||||||
### Alternative 2: Complete Rust Rewrite
|
### Alternative 2: Complete Rust Rewrite
|
||||||
|
|
||||||
|
|||||||
@ -71,7 +71,7 @@ extensions/
|
|||||||
└── external-tool/
|
└── external-tool/
|
||||||
├── extension.toml
|
├── extension.toml
|
||||||
└── nulib/
|
└── nulib/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Extension Manifest (extension.toml)
|
### Extension Manifest (extension.toml)
|
||||||
|
|
||||||
@ -103,7 +103,7 @@ config_schema = "schemas/schema.ncl"
|
|||||||
config_prefix = "custom_provider"
|
config_prefix = "custom_provider"
|
||||||
required_env_vars = ["CUSTOM_PROVIDER_API_KEY"]
|
required_env_vars = ["CUSTOM_PROVIDER_API_KEY"]
|
||||||
optional_config = ["custom_provider.region", "custom_provider.timeout"]
|
optional_config = ["custom_provider.region", "custom_provider.timeout"]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Key Framework Principles
|
### Key Framework Principles
|
||||||
|
|
||||||
@ -202,7 +202,7 @@ provisioning server create --provider custom-provider
|
|||||||
# Extension management
|
# Extension management
|
||||||
provisioning extension disable custom-provider
|
provisioning extension disable custom-provider
|
||||||
provisioning extension update custom-provider
|
provisioning extension update custom-provider
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Integration
|
### Configuration Integration
|
||||||
|
|
||||||
@ -217,7 +217,7 @@ timeout = 30
|
|||||||
|
|
||||||
# Extension configuration follows same hierarchy rules
|
# Extension configuration follows same hierarchy rules
|
||||||
# System defaults → User config → Environment config → Runtime
|
# System defaults → User config → Environment config → Runtime
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Security and Isolation
|
### Security and Isolation
|
||||||
|
|
||||||
@ -256,7 +256,7 @@ export def create-server [name: string, config: record] -> record {
|
|||||||
http post $"($config.custom_provider.api_endpoint)/servers" $payload
|
http post $"($config.custom_provider.api_endpoint)/servers" $payload
|
||||||
| from json
|
| from json
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Task Service Extension Pattern
|
### Task Service Extension Pattern
|
||||||
|
|
||||||
@ -272,7 +272,7 @@ export def install [server: string] -> nothing {
|
|||||||
export def uninstall [server: string] -> nothing {
|
export def uninstall [server: string] -> nothing {
|
||||||
kubectl delete deployment custom-service --server $server
|
kubectl delete deployment custom-service --server $server
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## References
|
## References
|
||||||
|
|
||||||
|
|||||||
@ -7,7 +7,9 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The main provisioning CLI script (`provisioning/core/nulib/provisioning`) had grown to **1,329 lines** with a massive 1,100+ line match statement handling all commands. This monolithic structure created multiple critical problems:
|
The main provisioning CLI script (`provisioning/core/nulib/provisioning`) had grown to
|
||||||
|
**1,329 lines** with a massive 1,100+ line match statement handling all commands. This
|
||||||
|
monolithic structure created multiple critical problems:
|
||||||
|
|
||||||
### Problems Identified
|
### Problems Identified
|
||||||
|
|
||||||
@ -53,7 +55,7 @@ provisioning/core/nulib/
|
|||||||
│ ├── orchestration.nu (64 lines)
|
│ ├── orchestration.nu (64 lines)
|
||||||
│ ├── utilities.nu (157 lines)
|
│ ├── utilities.nu (157 lines)
|
||||||
│ └── workspace.nu (56 lines)
|
│ └── workspace.nu (56 lines)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Key Components
|
### Key Components
|
||||||
|
|
||||||
@ -66,7 +68,7 @@ export def parse_common_flags [flags: record]: nothing -> record
|
|||||||
export def build_module_args [flags: record, extra: string = ""]: nothing -> string
|
export def build_module_args [flags: record, extra: string = ""]: nothing -> string
|
||||||
export def set_debug_env [flags: record]
|
export def set_debug_env [flags: record]
|
||||||
export def get_debug_flag [flags: record]: nothing -> string
|
export def get_debug_flag [flags: record]: nothing -> string
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Benefits:**
|
**Benefits:**
|
||||||
|
|
||||||
@ -82,7 +84,7 @@ Central routing with 80+ command mappings:
|
|||||||
```nushell
|
```nushell
|
||||||
export def get_command_registry []: nothing -> record # 80+ shortcuts
|
export def get_command_registry []: nothing -> record # 80+ shortcuts
|
||||||
export def dispatch_command [args: list, flags: record] # Main router
|
export def dispatch_command [args: list, flags: record] # Main router
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Features:**
|
**Features:**
|
||||||
|
|
||||||
@ -96,7 +98,7 @@ export def dispatch_command [args: list, flags: record] # Main router
|
|||||||
Seven focused modules organized by domain:
|
Seven focused modules organized by domain:
|
||||||
|
|
||||||
| Module | Lines | Responsibility |
|
| Module | Lines | Responsibility |
|
||||||
|--------|-------|----------------|
|
| -------- | ------- | ---------------- |
|
||||||
| `infrastructure.nu` | 117 | Server, taskserv, cluster, infra |
|
| `infrastructure.nu` | 117 | Server, taskserv, cluster, infra |
|
||||||
| `orchestration.nu` | 64 | Workflow, batch, orchestrator |
|
| `orchestration.nu` | 64 | Workflow, batch, orchestrator |
|
||||||
| `development.nu` | 72 | Module, layer, version, pack |
|
| `development.nu` | 72 | Module, layer, version, pack |
|
||||||
@ -153,7 +155,7 @@ export def handle_infrastructure_command [
|
|||||||
ops: string
|
ops: string
|
||||||
flags: record # ⬅️ Abstraction, not concrete flags
|
flags: record # ⬅️ Abstraction, not concrete flags
|
||||||
]
|
]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Implementation Details
|
## Implementation Details
|
||||||
|
|
||||||
@ -186,7 +188,7 @@ provisioning help workspace
|
|||||||
provisioning workspace help # ⬅️ NEW: Bi-directional
|
provisioning workspace help # ⬅️ NEW: Bi-directional
|
||||||
provisioning ws help # ⬅️ NEW: With shortcuts
|
provisioning ws help # ⬅️ NEW: With shortcuts
|
||||||
provisioning help ws # ⬅️ NEW: Shortcut in help
|
provisioning help ws # ⬅️ NEW: Shortcut in help
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Implementation:**
|
**Implementation:**
|
||||||
|
|
||||||
@ -196,7 +198,7 @@ let first_op = if ($ops_list | length) > 0 { ($ops_list | get 0) } else { "" }
|
|||||||
if $first_op in ["help" "h"] {
|
if $first_op in ["help" "h"] {
|
||||||
exec $"($env.PROVISIONING_NAME)" help $task --notitles
|
exec $"($env.PROVISIONING_NAME)" help $task --notitles
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Command Shortcuts
|
### Command Shortcuts
|
||||||
|
|
||||||
@ -249,14 +251,14 @@ Comprehensive test suite created (`tests/test_provisioning_refactor.nu`):
|
|||||||
🎯 Testing command routing... ✅
|
🎯 Testing command routing... ✅
|
||||||
|
|
||||||
📊 TEST RESULTS: 6 passed, 0 failed
|
📊 TEST RESULTS: 6 passed, 0 failed
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Results
|
## Results
|
||||||
|
|
||||||
### Quantitative Improvements
|
### Quantitative Improvements
|
||||||
|
|
||||||
| Metric | Before | After | Improvement |
|
| Metric | Before | After | Improvement |
|
||||||
|--------|--------|-------|-------------|
|
| -------- | -------- | ------- | ------------- |
|
||||||
| **Main file size** | 1,329 lines | 211 lines | **84% reduction** |
|
| **Main file size** | 1,329 lines | 211 lines | **84% reduction** |
|
||||||
| **Command handler** | 1 massive match (1,100+ lines) | 7 focused modules | **Domain separation** |
|
| **Command handler** | 1 massive match (1,100+ lines) | 7 focused modules | **Domain separation** |
|
||||||
| **Flag handling** | Repeated 50+ times | 1 function | **98% duplication removal** |
|
| **Flag handling** | Repeated 50+ times | 1 function | **98% duplication removal** |
|
||||||
@ -329,7 +331,7 @@ Comprehensive test suite created (`tests/test_provisioning_refactor.nu`):
|
|||||||
let arg_include_notuse = if $include_notuse { $"--include_notuse "} else { "" }
|
let arg_include_notuse = if $include_notuse { $"--include_notuse "} else { "" }
|
||||||
run_module $"($str_ops) ($str_infra) ($use_check)..." "server" --exec
|
run_module $"($str_ops) ($str_infra) ($use_check)..." "server" --exec
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### After: Clean, Reusable
|
### After: Clean, Reusable
|
||||||
|
|
||||||
@ -338,7 +340,7 @@ def handle_server [ops: string, flags: record] {
|
|||||||
let args = build_module_args $flags $ops
|
let args = build_module_args $flags $ops
|
||||||
run_module $args "server" --exec
|
run_module $args "server" --exec
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Reduction: 10 lines → 3 lines (70% reduction)**
|
**Reduction: 10 lines → 3 lines (70% reduction)**
|
||||||
|
|
||||||
@ -370,7 +372,9 @@ See `docs/development/COMMAND_HANDLER_GUIDE.md` for:
|
|||||||
|
|
||||||
## Conclusion
|
## Conclusion
|
||||||
|
|
||||||
This refactoring transforms the provisioning CLI from a monolithic, hard-to-maintain script into a modular, well-organized system following software engineering best practices. The 84% reduction in main file size, elimination of code duplication, and comprehensive test coverage position the project for sustainable long-term growth.
|
This refactoring transforms the provisioning CLI from a monolithic, hard-to-maintain script into a modular, well-organized system following software
|
||||||
|
engineering best practices. The 84% reduction in main file size, elimination of code duplication, and comprehensive test coverage position the project
|
||||||
|
for sustainable long-term growth.
|
||||||
|
|
||||||
The new architecture enables:
|
The new architecture enables:
|
||||||
|
|
||||||
|
|||||||
@ -7,7 +7,8 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The KMS service initially supported 4 backends: HashiCorp Vault, AWS KMS, Age, and Cosmian KMS. This created unnecessary complexity and unclear guidance about which backend to use for different environments.
|
The KMS service initially supported 4 backends: HashiCorp Vault, AWS KMS, Age, and Cosmian KMS. This created unnecessary complexity and unclear
|
||||||
|
guidance about which backend to use for different environments.
|
||||||
|
|
||||||
### Problems with 4-Backend Approach
|
### Problems with 4-Backend Approach
|
||||||
|
|
||||||
@ -254,7 +255,7 @@ See `docs/migration/KMS_SIMPLIFICATION.md` for detailed steps.
|
|||||||
|
|
||||||
- [Age Encryption](https://github.com/FiloSottile/age) - Modern encryption tool
|
- [Age Encryption](https://github.com/FiloSottile/age) - Modern encryption tool
|
||||||
- [Cosmian KMS](https://cosmian.com/kms/) - Enterprise KMS with confidential computing
|
- [Cosmian KMS](https://cosmian.com/kms/) - Enterprise KMS with confidential computing
|
||||||
- [ADR-006](ADR-006-provisioning-cli-refactoring.md) - Previous KMS integration
|
- [ADR-006](adr-006-provisioning-cli-refactoring.md) - Previous KMS integration
|
||||||
- [Migration Guide](../migration/KMS_SIMPLIFICATION.md) - Detailed migration steps
|
- [Migration Guide](../migration/KMS_SIMPLIFICATION.md) - Detailed migration steps
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|||||||
@ -7,7 +7,8 @@
|
|||||||
|
|
||||||
## Context and Problem Statement
|
## Context and Problem Statement
|
||||||
|
|
||||||
The Provisioning platform requires fine-grained authorization controls to manage access to infrastructure resources across multiple environments (development, staging, production). The authorization system must:
|
The Provisioning platform requires fine-grained authorization controls to manage access to infrastructure resources across multiple environments
|
||||||
|
(development, staging, production). The authorization system must:
|
||||||
|
|
||||||
1. Support complex authorization rules (MFA, IP restrictions, time windows, approvals)
|
1. Support complex authorization rules (MFA, IP restrictions, time windows, approvals)
|
||||||
2. Be auditable and version-controlled
|
2. Be auditable and version-controlled
|
||||||
@ -138,7 +139,7 @@ Use Casbin authorization library.
|
|||||||
│ Allow / Deny │
|
│ Allow / Deny │
|
||||||
│ │
|
│ │
|
||||||
└─────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Policy Organization
|
#### Policy Organization
|
||||||
|
|
||||||
@ -149,7 +150,7 @@ provisioning/config/cedar-policies/
|
|||||||
├── development.cedar # Development environment policies
|
├── development.cedar # Development environment policies
|
||||||
├── admin.cedar # Administrative policies
|
├── admin.cedar # Administrative policies
|
||||||
└── README.md # Documentation
|
└── README.md # Documentation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Rust Implementation
|
#### Rust Implementation
|
||||||
|
|
||||||
@ -160,7 +161,7 @@ provisioning/platform/orchestrator/src/security/
|
|||||||
├── authorization.rs # Middleware integration (380 lines)
|
├── authorization.rs # Middleware integration (380 lines)
|
||||||
├── mod.rs # Module exports
|
├── mod.rs # Module exports
|
||||||
└── tests.rs # Comprehensive tests (450 lines)
|
└── tests.rs # Comprehensive tests (450 lines)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Key Components
|
#### Key Components
|
||||||
|
|
||||||
@ -199,7 +200,7 @@ AuthorizationContext {
|
|||||||
force: bool, // Force flag
|
force: bool, // Force flag
|
||||||
additional: HashMap, // Additional context
|
additional: HashMap, // Additional context
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Example Policy
|
#### Example Policy
|
||||||
|
|
||||||
@ -214,7 +215,7 @@ permit (
|
|||||||
) when {
|
) when {
|
||||||
context.mfa_verified == true
|
context.mfa_verified == true
|
||||||
};
|
};
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Integration Points
|
### Integration Points
|
||||||
|
|
||||||
@ -341,7 +342,8 @@ Use Cedar for high-level policies, code for fine-grained checks.
|
|||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
Cedar policy language is inspired by decades of authorization research (XACML, AWS IAM) and production experience at AWS. It balances expressiveness with safety.
|
Cedar policy language is inspired by decades of authorization research (XACML, AWS IAM) and production experience at AWS. It balances expressiveness
|
||||||
|
with safety.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -8,7 +8,8 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The Provisioning platform required a comprehensive, enterprise-grade security system covering authentication, authorization, secrets management, MFA, compliance, and emergency access. The system needed to be production-ready, scalable, and compliant with GDPR, SOC2, and ISO 27001.
|
The Provisioning platform required a comprehensive, enterprise-grade security system covering authentication, authorization, secrets management, MFA,
|
||||||
|
compliance, and emergency access. The system needed to be production-ready, scalable, and compliant with GDPR, SOC2, and ISO 27001.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -266,7 +267,7 @@ Implement a complete security architecture using 12 specialized components organ
|
|||||||
8. Audit Logging (structured JSON, GDPR-compliant)
|
8. Audit Logging (structured JSON, GDPR-compliant)
|
||||||
↓
|
↓
|
||||||
9. Response
|
9. Response
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Emergency Access Flow
|
### Emergency Access Flow
|
||||||
|
|
||||||
@ -280,7 +281,7 @@ Implement a complete security architecture using 12 specialized components organ
|
|||||||
4. Enhanced Audit (7-year retention, immutable)
|
4. Enhanced Audit (7-year retention, immutable)
|
||||||
↓
|
↓
|
||||||
5. Auto-Revocation (expiration/inactivity)
|
5. Auto-Revocation (expiration/inactivity)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -364,7 +365,7 @@ Implement a complete security architecture using 12 specialized components organ
|
|||||||
## Performance Characteristics
|
## Performance Characteristics
|
||||||
|
|
||||||
| Component | Latency | Throughput | Memory |
|
| Component | Latency | Throughput | Memory |
|
||||||
|-----------|---------|------------|--------|
|
| ----------- | --------- | ------------ | -------- |
|
||||||
| JWT Auth | <5 ms | 10,000/s | ~10 MB |
|
| JWT Auth | <5 ms | 10,000/s | ~10 MB |
|
||||||
| Cedar Authz | <10 ms | 5,000/s | ~50 MB |
|
| Cedar Authz | <10 ms | 5,000/s | ~50 MB |
|
||||||
| Audit Log | <5 ms | 20,000/s | ~100 MB |
|
| Audit Log | <5 ms | 20,000/s | ~100 MB |
|
||||||
@ -386,7 +387,7 @@ Implement a complete security architecture using 12 specialized components organ
|
|||||||
cd provisioning/platform/kms-service && cargo run &
|
cd provisioning/platform/kms-service && cargo run &
|
||||||
cd provisioning/platform/orchestrator && cargo run &
|
cd provisioning/platform/orchestrator && cargo run &
|
||||||
cd provisioning/platform/control-center && cargo run &
|
cd provisioning/platform/control-center && cargo run &
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Production
|
### Production
|
||||||
|
|
||||||
@ -401,7 +402,7 @@ docker-compose up -d kms orchestrator control-center
|
|||||||
systemctl start provisioning-kms
|
systemctl start provisioning-kms
|
||||||
systemctl start provisioning-orchestrator
|
systemctl start provisioning-orchestrator
|
||||||
systemctl start provisioning-control-center
|
systemctl start provisioning-control-center
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -428,7 +429,7 @@ export VAULT_TOKEN="..."
|
|||||||
# MFA
|
# MFA
|
||||||
export MFA_TOTP_ISSUER="Provisioning"
|
export MFA_TOTP_ISSUER="Provisioning"
|
||||||
export MFA_WEBAUTHN_RP_ID="provisioning.example.com"
|
export MFA_WEBAUTHN_RP_ID="provisioning.example.com"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Config Files
|
### Config Files
|
||||||
|
|
||||||
@ -461,7 +462,7 @@ retention_days = 365
|
|||||||
retention_break_glass_days = 2555 # 7 years
|
retention_break_glass_days = 2555 # 7 years
|
||||||
export_format = "json"
|
export_format = "json"
|
||||||
pii_anonymization = true
|
pii_anonymization = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -484,7 +485,7 @@ cargo test
|
|||||||
|
|
||||||
# Config Encryption (Nushell)
|
# Config Encryption (Nushell)
|
||||||
nu provisioning/core/nulib/lib_provisioning/config/encryption_tests.nu
|
nu provisioning/core/nulib/lib_provisioning/config/encryption_tests.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Integration Tests
|
### Integration Tests
|
||||||
|
|
||||||
@ -493,7 +494,7 @@ nu provisioning/core/nulib/lib_provisioning/config/encryption_tests.nu
|
|||||||
cd provisioning/platform/orchestrator
|
cd provisioning/platform/orchestrator
|
||||||
cargo test --test security_integration_tests
|
cargo test --test security_integration_tests
|
||||||
cargo test --test break_glass_integration_tests
|
cargo test --test break_glass_integration_tests
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -9,13 +9,16 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The provisioning project historically used a single configuration format (YAML/TOML environment variables) for all purposes. As the system evolved, different parts naturally adopted different formats:
|
The provisioning project historically used a single configuration format (YAML/TOML environment variables) for all purposes. As the system evolved,
|
||||||
|
different parts naturally adopted different formats:
|
||||||
|
|
||||||
- **TOML** for modular provider and platform configurations (`providers/*.toml`, `platform/*.toml`)
|
- **TOML** for modular provider and platform configurations (`providers/*.toml`, `platform/*.toml`)
|
||||||
- **KCL** for infrastructure-as-code definitions with type safety
|
- **KCL** for infrastructure-as-code definitions with type safety
|
||||||
- **YAML** for workspace metadata
|
- **YAML** for workspace metadata
|
||||||
|
|
||||||
However, the workspace configuration remained in **YAML** (`provisioning.yaml`), creating inconsistency and leaving type-unsafe configuration handling. Meanwhile, complete KCL schemas for workspace configuration were designed but unused.
|
However, the workspace configuration remained in **YAML** (`provisioning.yaml`),
|
||||||
|
creating inconsistency and leaving type-unsafe configuration handling. Meanwhile,
|
||||||
|
complete KCL schemas for workspace configuration were designed but unused.
|
||||||
|
|
||||||
**Problem**: Three different formats in the same system without documented rationale or consistent patterns.
|
**Problem**: Three different formats in the same system without documented rationale or consistent patterns.
|
||||||
|
|
||||||
@ -26,7 +29,7 @@ However, the workspace configuration remained in **YAML** (`provisioning.yaml`),
|
|||||||
Adopt a **three-format strategy** with clear separation of concerns:
|
Adopt a **three-format strategy** with clear separation of concerns:
|
||||||
|
|
||||||
| Format | Purpose | Use Cases |
|
| Format | Purpose | Use Cases |
|
||||||
|--------|---------|-----------|
|
| -------- | --------- | ----------- |
|
||||||
| **KCL** | Infrastructure as Code & Schemas | Workspace config, infrastructure definitions, type-safe validation |
|
| **KCL** | Infrastructure as Code & Schemas | Workspace config, infrastructure definitions, type-safe validation |
|
||||||
| **TOML** | Application Configuration & Settings | System defaults, provider settings, user preferences, interpolation |
|
| **TOML** | Application Configuration & Settings | System defaults, provider settings, user preferences, interpolation |
|
||||||
| **YAML** | Metadata & Kubernetes Resources | K8s manifests, tool metadata, version tracking, CI/CD resources |
|
| **YAML** | Metadata & Kubernetes Resources | K8s manifests, tool metadata, version tracking, CI/CD resources |
|
||||||
@ -72,7 +75,7 @@ Current (Nickel):
|
|||||||
├── config/*.toml.j2
|
├── config/*.toml.j2
|
||||||
├── nickel/*.ncl.j2
|
├── nickel/*.ncl.j2
|
||||||
└── README.md
|
└── README.md
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Expected Outcome**:
|
**Expected Outcome**:
|
||||||
|
|
||||||
@ -328,7 +331,7 @@ provisioning/kcl/templates/
|
|||||||
├── server.ncl # Actually Nushell/Jinja2 template
|
├── server.ncl # Actually Nushell/Jinja2 template
|
||||||
├── taskserv.ncl # Actually Nushell/Jinja2 template
|
├── taskserv.ncl # Actually Nushell/Jinja2 template
|
||||||
└── ... # 15 more template files
|
└── ... # 15 more template files
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
This causes:
|
This causes:
|
||||||
|
|
||||||
@ -353,7 +356,7 @@ provisioning/templates/
|
|||||||
│ ├── workspace.ncl.j2
|
│ ├── workspace.ncl.j2
|
||||||
│ └── ...
|
│ └── ...
|
||||||
└── README.md
|
└── README.md
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Outcome
|
### Outcome
|
||||||
|
|
||||||
|
|||||||
@ -9,7 +9,8 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The provisioning platform historically used KCL (KLang) as the primary infrastructure-as-code language for all configuration schemas. As the system evolved through four migration phases (Foundation, Core, Complex, Highly Complex), KCL's limitations became increasingly apparent:
|
The provisioning platform historically used KCL (KLang) as the primary infrastructure-as-code language for all configuration schemas. As the system
|
||||||
|
evolved through four migration phases (Foundation, Core, Complex, Highly Complex), KCL's limitations became increasingly apparent:
|
||||||
|
|
||||||
### Problems with KCL
|
### Problems with KCL
|
||||||
|
|
||||||
@ -87,7 +88,7 @@ The provisioning system required:
|
|||||||
### Migration Complete
|
### Migration Complete
|
||||||
|
|
||||||
| Metric | Value |
|
| Metric | Value |
|
||||||
|--------|-------|
|
| -------- | ------- |
|
||||||
| KCL files migrated | 40 |
|
| KCL files migrated | 40 |
|
||||||
| Nickel files created | 72 |
|
| Nickel files created | 72 |
|
||||||
| Modules converted | 24 core modules |
|
| Modules converted | 24 core modules |
|
||||||
@ -127,14 +128,14 @@ let defaults = import "./defaults.ncl" in
|
|||||||
DefaultServerDefaults_upcloud = defaults.server_defaults_upcloud,
|
DefaultServerDefaults_upcloud = defaults.server_defaults_upcloud,
|
||||||
DefaultServerUpcloud = defaults.server_upcloud,
|
DefaultServerUpcloud = defaults.server_upcloud,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Active Workspaces (`workspace_librecloud/nickel/`)
|
### Active Workspaces (`workspace_librecloud/nickel/`)
|
||||||
|
|
||||||
- **47 Nickel files** in productive use
|
- **47 Nickel files** in productive use
|
||||||
- **2 infrastructures**:
|
- **2 infrastructures**:
|
||||||
- `wuji` - Kubernetes cluster with 20 taskservs
|
- `wuji` - Kubernetes cluster with 20 taskservs
|
||||||
- `sgoyol` - Support servers group
|
- `sgoyol` - Support servers group
|
||||||
- **Two deployment modes** fully implemented and tested
|
- **Two deployment modes** fully implemented and tested
|
||||||
- **Daily production usage** validated ✅
|
- **Daily production usage** validated ✅
|
||||||
|
|
||||||
@ -150,7 +151,7 @@ let defaults = import "./defaults.ncl" in
|
|||||||
## Comparison: KCL vs Nickel
|
## Comparison: KCL vs Nickel
|
||||||
|
|
||||||
| Aspect | KCL | Nickel | Winner |
|
| Aspect | KCL | Nickel | Winner |
|
||||||
|--------|-----|--------|--------|
|
| -------- | ----- | -------- | -------- |
|
||||||
| **Mental Model** | Python-like with schemas | JSON with functions | Nickel |
|
| **Mental Model** | Python-like with schemas | JSON with functions | Nickel |
|
||||||
| **Performance** | Baseline | 60% faster evaluation | Nickel |
|
| **Performance** | Baseline | 60% faster evaluation | Nickel |
|
||||||
| **Type System** | Rigid schemas | Gradual typing + contracts | Nickel |
|
| **Type System** | Rigid schemas | Gradual typing + contracts | Nickel |
|
||||||
@ -179,7 +180,7 @@ let defaults = import "./defaults.ncl" in
|
|||||||
enable_preemption | Bool,
|
enable_preemption | Bool,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**File 2: Defaults** (`batch_defaults.ncl`):
|
**File 2: Defaults** (`batch_defaults.ncl`):
|
||||||
|
|
||||||
@ -192,7 +193,7 @@ let defaults = import "./defaults.ncl" in
|
|||||||
enable_preemption = false,
|
enable_preemption = false,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**File 3: Main** (`batch.ncl`):
|
**File 3: Main** (`batch.ncl`):
|
||||||
|
|
||||||
@ -206,7 +207,7 @@ let defaults = import "./batch_defaults.ncl" in
|
|||||||
defaults.scheduler & o, # Level 2: Makers
|
defaults.scheduler & o, # Level 2: Makers
|
||||||
DefaultScheduler = defaults.scheduler, # Level 3: Instances
|
DefaultScheduler = defaults.scheduler, # Level 3: Instances
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Hybrid Pattern Benefits
|
### Hybrid Pattern Benefits
|
||||||
|
|
||||||
@ -228,7 +229,7 @@ provisioning/schemas/
|
|||||||
├── generator/ # Declarations, gap analysis, changes
|
├── generator/ # Declarations, gap analysis, changes
|
||||||
├── integrations/ # Runtime, GitOps, main
|
├── integrations/ # Runtime, GitOps, main
|
||||||
└── main.ncl # Entry point with namespace organization
|
└── main.ncl # Entry point with namespace organization
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Import pattern**:
|
**Import pattern**:
|
||||||
|
|
||||||
@ -238,7 +239,7 @@ provisioning.lib # For Storage, TaskServDef
|
|||||||
provisioning.config.settings # For Settings, Defaults
|
provisioning.config.settings # For Settings, Defaults
|
||||||
provisioning.infrastructure.compute.server
|
provisioning.infrastructure.compute.server
|
||||||
provisioning.operations.workflows
|
provisioning.operations.workflows
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -257,7 +258,7 @@ provisioning.operations.workflows
|
|||||||
# workspace_librecloud/nickel/main.ncl
|
# workspace_librecloud/nickel/main.ncl
|
||||||
import "../../provisioning/schemas/main.ncl"
|
import "../../provisioning/schemas/main.ncl"
|
||||||
import "../../provisioning/extensions/taskservs/kubernetes/nickel/main.ncl"
|
import "../../provisioning/extensions/taskservs/kubernetes/nickel/main.ncl"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 2. Production Mode (Hermetic Deployment)
|
#### 2. Production Mode (Hermetic Deployment)
|
||||||
|
|
||||||
@ -265,7 +266,7 @@ Create immutable snapshots for reproducible deployments:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
provisioning workspace freeze --version "2025-12-15-prod-v1" --env production
|
provisioning workspace freeze --version "2025-12-15-prod-v1" --env production
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Frozen structure** (`.frozen/{version}/`):
|
**Frozen structure** (`.frozen/{version}/`):
|
||||||
|
|
||||||
@ -273,7 +274,7 @@ provisioning workspace freeze --version "2025-12-15-prod-v1" --env production
|
|||||||
├── provisioning/schemas/ # Snapshot of central schemas
|
├── provisioning/schemas/ # Snapshot of central schemas
|
||||||
├── extensions/ # Snapshot of all extensions
|
├── extensions/ # Snapshot of all extensions
|
||||||
└── workspace/ # Snapshot of workspace configs
|
└── workspace/ # Snapshot of workspace configs
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**All imports rewritten to local paths**:
|
**All imports rewritten to local paths**:
|
||||||
|
|
||||||
@ -286,7 +287,7 @@ provisioning workspace freeze --version "2025-12-15-prod-v1" --env production
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
provisioning deploy --frozen "2025-12-15-prod-v1" --infra wuji
|
provisioning deploy --frozen "2025-12-15-prod-v1" --infra wuji
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Benefits**:
|
**Benefits**:
|
||||||
|
|
||||||
@ -313,7 +314,7 @@ typedialog form --schema server.ncl --output json
|
|||||||
|
|
||||||
# Interactive form → Nickel output
|
# Interactive form → Nickel output
|
||||||
typedialog form --input form.toml --output nickel
|
typedialog form --input form.toml --output nickel
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Value**: Amplifies Nickel ecosystem beyond IaC:
|
**Value**: Amplifies Nickel ecosystem beyond IaC:
|
||||||
|
|
||||||
@ -329,31 +330,31 @@ typedialog form --input form.toml --output nickel
|
|||||||
### Expression-Based Structure
|
### Expression-Based Structure
|
||||||
|
|
||||||
| KCL | Nickel |
|
| KCL | Nickel |
|
||||||
|-----|--------|
|
| ----- | -------- |
|
||||||
| Multiple top-level let bindings | Single root expression with `let...in` chaining |
|
| Multiple top-level let bindings | Single root expression with `let...in` chaining |
|
||||||
|
|
||||||
### Schema Inheritance → Record Merging
|
### Schema Inheritance → Record Merging
|
||||||
|
|
||||||
| KCL | Nickel |
|
| KCL | Nickel |
|
||||||
|-----|--------|
|
| ----- | -------- |
|
||||||
| `schema Server(defaults.ServerDefaults)` | `defaults.ServerDefaults & { overrides }` |
|
| `schema Server(defaults.ServerDefaults)` | `defaults.ServerDefaults & { overrides }` |
|
||||||
|
|
||||||
### Optional Fields
|
### Optional Fields
|
||||||
|
|
||||||
| KCL | Nickel |
|
| KCL | Nickel |
|
||||||
|-----|--------|
|
| ----- | -------- |
|
||||||
| `field?: type` | `field = null` or `field = ""` |
|
| `field?: type` | `field = null` or `field = ""` |
|
||||||
|
|
||||||
### Union Types
|
### Union Types
|
||||||
|
|
||||||
| KCL | Nickel |
|
| KCL | Nickel |
|
||||||
|-----|--------|
|
| ----- | -------- |
|
||||||
| `"ubuntu" \| "debian" \| "centos"` | `[\\| 'ubuntu, 'debian, 'centos \\|]` |
|
| `"ubuntu" | "debian" | "centos"` | `[\\| 'ubuntu, 'debian, 'centos \\|]` |
|
||||||
|
|
||||||
### Boolean/Null Conversion
|
### Boolean/Null Conversion
|
||||||
|
|
||||||
| KCL | Nickel |
|
| KCL | Nickel |
|
||||||
|-----|--------|
|
| ----- | -------- |
|
||||||
| `True` / `False` / `None` | `true` / `false` / `null` |
|
| `True` / `False` / `None` | `true` / `false` / `null` |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@ -6,7 +6,9 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The provisioning system integrates with Nickel for configuration management in advanced scenarios. Users need to evaluate Nickel files and work with their output in Nushell scripts. The `nu_plugin_nickel` plugin provides this integration.
|
The provisioning system integrates with Nickel for configuration management in advanced
|
||||||
|
scenarios. Users need to evaluate Nickel files and work with their output in Nushell
|
||||||
|
scripts. The `nu_plugin_nickel` plugin provides this integration.
|
||||||
|
|
||||||
The architectural decision was whether the plugin should:
|
The architectural decision was whether the plugin should:
|
||||||
|
|
||||||
@ -30,7 +32,7 @@ import "lib/validation" as valid
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Module system includes:
|
Module system includes:
|
||||||
|
|
||||||
@ -90,7 +92,7 @@ Implement the `nu_plugin_nickel` plugin as a **CLI wrapper** that invokes the ex
|
|||||||
│ ✅ Cell path access works │
|
│ ✅ Cell path access works │
|
||||||
│ ✅ Piping works │
|
│ ✅ Piping works │
|
||||||
└─────────────────────────────┘
|
└─────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Implementation Characteristics
|
### Implementation Characteristics
|
||||||
|
|
||||||
@ -115,7 +117,7 @@ Implement the `nu_plugin_nickel` plugin as a **CLI wrapper** that invokes the ex
|
|||||||
### Why CLI Wrapper Is The Correct Choice
|
### Why CLI Wrapper Is The Correct Choice
|
||||||
|
|
||||||
| Aspect | Pure Rust (nickel-lang-core) | CLI Wrapper (chosen) |
|
| Aspect | Pure Rust (nickel-lang-core) | CLI Wrapper (chosen) |
|
||||||
|--------|-------------------------------|----------------------|
|
| -------- | ------------------------------- | ---------------------- |
|
||||||
| **Module resolution** | ❓ Undocumented API | ✅ Official, proven |
|
| **Module resolution** | ❓ Undocumented API | ✅ Official, proven |
|
||||||
| **Search paths** | ❓ How to configure? | ✅ CLI handles it |
|
| **Search paths** | ❓ How to configure? | ✅ CLI handles it |
|
||||||
| **Standard library** | ❓ How to access? | ✅ Automatic discovery |
|
| **Standard library** | ❓ How to access? | ✅ Automatic discovery |
|
||||||
@ -295,7 +297,7 @@ cmd.arg("export").arg(file).arg("--format").arg(format);
|
|||||||
cmd.arg("export").arg(format).arg(file);
|
cmd.arg("export").arg(format).arg(file);
|
||||||
// Results in: "nickel export json /file"
|
// Results in: "nickel export json /file"
|
||||||
// ↑ This triggers auto-import of nonexistent JSON module
|
// ↑ This triggers auto-import of nonexistent JSON module
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Caching Strategy
|
### Caching Strategy
|
||||||
|
|
||||||
@ -323,7 +325,7 @@ This enables Nushell cell path access:
|
|||||||
|
|
||||||
```nushell
|
```nushell
|
||||||
nickel-export json /config.ncl | .database.host # ✅ Works
|
nickel-export json /config.ncl | .database.host # ✅ Works
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Testing Strategy
|
## Testing Strategy
|
||||||
|
|
||||||
@ -351,7 +353,7 @@ nickel-export json /workspace/config.ncl | .database
|
|||||||
# Verify output types
|
# Verify output types
|
||||||
nickel-export json /workspace/config.ncl | type
|
nickel-export json /workspace/config.ncl | type
|
||||||
# Should show: record, not string
|
# Should show: record, not string
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Configuration Integration
|
## Configuration Integration
|
||||||
|
|
||||||
|
|||||||
@ -6,7 +6,9 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The provisioning system requires interactive user input for configuration workflows, workspace initialization, credential setup, and guided deployment scenarios. The system architecture combines Rust (performance-critical), Nushell (scripting), and Nickel (declarative configuration), creating challenges for interactive form-based input and multi-user collaboration.
|
The provisioning system requires interactive user input for configuration workflows, workspace initialization, credential setup, and guided deployment
|
||||||
|
scenarios. The system architecture combines Rust (performance-critical), Nushell (scripting), and Nickel (declarative configuration), creating
|
||||||
|
challenges for interactive form-based input and multi-user collaboration.
|
||||||
|
|
||||||
### The Interactive Configuration Problem
|
### The Interactive Configuration Problem
|
||||||
|
|
||||||
@ -70,11 +72,13 @@ The provisioning system requires interactive user input for configuration workfl
|
|||||||
|
|
||||||
## Decision
|
## Decision
|
||||||
|
|
||||||
Integrate **typdialog** with its **Web UI backend** as the standard interactive configuration interface for the provisioning platform. The major achievement of typdialog is not the TUI - it is the Web UI backend that enables browser-based forms, multi-user collaboration, and seamless integration with the provisioning orchestrator.
|
Integrate **typdialog** with its **Web UI backend** as the standard interactive configuration interface for the provisioning platform. The major
|
||||||
|
achievement of typdialog is not the TUI - it is the Web UI backend that enables browser-based forms, multi-user collaboration, and seamless
|
||||||
|
integration with the provisioning orchestrator.
|
||||||
|
|
||||||
### Architecture Diagram
|
### Architecture Diagram
|
||||||
|
|
||||||
```
|
```text
|
||||||
┌─────────────────────────────────────────┐
|
┌─────────────────────────────────────────┐
|
||||||
│ Nushell Script │
|
│ Nushell Script │
|
||||||
│ │
|
│ │
|
||||||
@ -149,7 +153,7 @@ Integrate **typdialog** with its **Web UI backend** as the standard interactive
|
|||||||
### Why TUI Dialog Integration Is Required
|
### Why TUI Dialog Integration Is Required
|
||||||
|
|
||||||
| Aspect | Shell Prompts (current) | Web Forms | TUI Dialog (chosen) |
|
| Aspect | Shell Prompts (current) | Web Forms | TUI Dialog (chosen) |
|
||||||
|--------|-------------------------|-----------|---------------------|
|
| -------- | ------------------------- | ----------- | --------------------- |
|
||||||
| **User Experience** | ❌ Basic text only | ✅ Rich UI | ✅ Rich TUI |
|
| **User Experience** | ❌ Basic text only | ✅ Rich UI | ✅ Rich TUI |
|
||||||
| **Validation** | ❌ Manual, error-prone | ✅ Built-in | ✅ Built-in |
|
| **Validation** | ❌ Manual, error-prone | ✅ Built-in | ✅ Built-in |
|
||||||
| **Security** | ❌ Plain text, history | ⚠️ Network risk | ✅ Secure terminal |
|
| **Security** | ❌ Plain text, history | ⚠️ Network risk | ✅ Secure terminal |
|
||||||
|
|||||||
@ -6,7 +6,9 @@
|
|||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
The provisioning system manages sensitive data across multiple infrastructure layers: cloud provider credentials, database passwords, API keys, SSH keys, encryption keys, and service tokens. The current security architecture (ADR-009) includes SOPS for encrypted config files and Age for key management, but lacks a centralized secrets management solution with dynamic secrets, access control, and audit logging.
|
The provisioning system manages sensitive data across multiple infrastructure layers: cloud provider credentials, database passwords, API keys, SSH
|
||||||
|
keys, encryption keys, and service tokens. The current security architecture (ADR-009) includes SOPS for encrypted config files and Age for key
|
||||||
|
management, but lacks a centralized secrets management solution with dynamic secrets, access control, and audit logging.
|
||||||
|
|
||||||
### Current Secrets Management Challenges
|
### Current Secrets Management Challenges
|
||||||
|
|
||||||
@ -91,7 +93,7 @@ Integrate **SecretumVault** as the centralized secrets management system for the
|
|||||||
|
|
||||||
### Architecture Diagram
|
### Architecture Diagram
|
||||||
|
|
||||||
```
|
```text
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
│ Provisioning CLI / Orchestrator / Services │
|
│ Provisioning CLI / Orchestrator / Services │
|
||||||
│ │
|
│ │
|
||||||
@ -184,7 +186,7 @@ Integrate **SecretumVault** as the centralized secrets management system for the
|
|||||||
### Why SecretumVault Is Required
|
### Why SecretumVault Is Required
|
||||||
|
|
||||||
| Aspect | SOPS + Age (current) | HashiCorp Vault | SecretumVault (chosen) |
|
| Aspect | SOPS + Age (current) | HashiCorp Vault | SecretumVault (chosen) |
|
||||||
|--------|----------------------|-----------------|------------------------|
|
| -------- | ---------------------- | ----------------- | ------------------------ |
|
||||||
| **Dynamic Secrets** | ❌ Static only | ✅ Full support | ✅ Full support |
|
| **Dynamic Secrets** | ❌ Static only | ✅ Full support | ✅ Full support |
|
||||||
| **Rust Native** | ⚠️ External CLI | ❌ Go binary | ✅ Pure Rust |
|
| **Rust Native** | ⚠️ External CLI | ❌ Go binary | ✅ Pure Rust |
|
||||||
| **Cedar Integration** | ❌ None | ❌ Custom policies | ✅ Native Cedar |
|
| **Cedar Integration** | ❌ None | ❌ Custom policies | ✅ Native Cedar |
|
||||||
@ -197,7 +199,7 @@ Integrate **SecretumVault** as the centralized secrets management system for the
|
|||||||
| **High Availability** | ❌ Single file | ✅ Raft cluster | ✅ Raft cluster |
|
| **High Availability** | ❌ Single file | ✅ Raft cluster | ✅ Raft cluster |
|
||||||
| **Performance** | ✅ Fast (local) | ⚠️ Network latency | ✅ Rust performance |
|
| **Performance** | ✅ Fast (local) | ⚠️ Network latency | ✅ Rust performance |
|
||||||
|
|
||||||
### Why Not Continue with SOPS Alone?
|
### Why Not Continue with SOPS Alone
|
||||||
|
|
||||||
SOPS is excellent for **static secrets in git**, but inadequate for:
|
SOPS is excellent for **static secrets in git**, but inadequate for:
|
||||||
|
|
||||||
@ -212,7 +214,7 @@ SOPS is excellent for **static secrets in git**, but inadequate for:
|
|||||||
- SOPS: Configuration files with long-lived secrets (gitops workflow)
|
- SOPS: Configuration files with long-lived secrets (gitops workflow)
|
||||||
- SecretumVault: Runtime dynamic secrets, short-lived credentials, audit trail
|
- SecretumVault: Runtime dynamic secrets, short-lived credentials, audit trail
|
||||||
|
|
||||||
### Why SecretumVault Over HashiCorp Vault?
|
### Why SecretumVault Over HashiCorp Vault
|
||||||
|
|
||||||
**HashiCorp Vault Limitations**:
|
**HashiCorp Vault Limitations**:
|
||||||
|
|
||||||
@ -333,7 +335,7 @@ secretum_vault_raft_leader_changes
|
|||||||
**Cons**: Vendor lock-in, multi-cloud complexity, cost at scale
|
**Cons**: Vendor lock-in, multi-cloud complexity, cost at scale
|
||||||
**Decision**: REJECTED - Against open-source and multi-cloud principles
|
**Decision**: REJECTED - Against open-source and multi-cloud principles
|
||||||
|
|
||||||
### Alternative 4: CyberArk, 1Password, etc.
|
### Alternative 4: CyberArk, 1Password, and Others
|
||||||
|
|
||||||
**Pros**: Enterprise features
|
**Pros**: Enterprise features
|
||||||
**Cons**: Proprietary, expensive, poor API integration
|
**Cons**: Proprietary, expensive, poor API integration
|
||||||
|
|||||||
@ -254,7 +254,7 @@ All AI components are **schema-aware**, **security-enforced**, and **human-super
|
|||||||
### Why AI Integration Is Essential
|
### Why AI Integration Is Essential
|
||||||
|
|
||||||
| Aspect | Manual Config | AI-Assisted (chosen) |
|
| Aspect | Manual Config | AI-Assisted (chosen) |
|
||||||
|--------|---------------|----------------------|
|
| -------- | --------------- | ---------------------- |
|
||||||
| **Learning Curve** | 🔴 Steep | 🟢 Gentle |
|
| **Learning Curve** | 🔴 Steep | 🟢 Gentle |
|
||||||
| **Time to Deploy** | 🔴 Hours | 🟢 Minutes |
|
| **Time to Deploy** | 🔴 Hours | 🟢 Minutes |
|
||||||
| **Error Rate** | 🔴 High | 🟢 Low (validated) |
|
| **Error Rate** | 🔴 High | 🟢 Low (validated) |
|
||||||
@ -376,7 +376,7 @@ pub async fn ai_generate_config(request: GenerateRequest) -> Result<Config> {
|
|||||||
No single LLM provider is best for all tasks:
|
No single LLM provider is best for all tasks:
|
||||||
|
|
||||||
| Provider | Best For | Considerations |
|
| Provider | Best For | Considerations |
|
||||||
|----------|----------|----------------|
|
| ---------- | ---------- | ---------------- |
|
||||||
| **Anthropic (Claude)** | Long context, accuracy | ✅ Best for complex configs |
|
| **Anthropic (Claude)** | Long context, accuracy | ✅ Best for complex configs |
|
||||||
| **OpenAI (GPT-4)** | Tool calling, speed | ✅ Best for quick suggestions |
|
| **OpenAI (GPT-4)** | Tool calling, speed | ✅ Best for quick suggestions |
|
||||||
| **Local (Llama, Mistral)** | Privacy, cost | ✅ Best for air-gapped envs |
|
| **Local (Llama, Mistral)** | Privacy, cost | ✅ Best for air-gapped envs |
|
||||||
|
|||||||
@ -73,12 +73,12 @@ The Provisioning Platform is a modern, cloud-native infrastructure automation sy
|
|||||||
│ └───────────────────────────────────────────────────────────┘ │
|
│ └───────────────────────────────────────────────────────────┘ │
|
||||||
│ │
|
│ │
|
||||||
└─────────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Key Metrics
|
### Key Metrics
|
||||||
|
|
||||||
| Metric | Value | Description |
|
| Metric | Value | Description |
|
||||||
|--------|-------|-------------|
|
| -------- | ------- | ------------- |
|
||||||
| **Codebase Size** | ~50,000 LOC | Nushell (60%), Rust (30%), Nickel (10%) |
|
| **Codebase Size** | ~50,000 LOC | Nushell (60%), Rust (30%), Nickel (10%) |
|
||||||
| **Extensions** | 100+ | Providers, taskservs, clusters |
|
| **Extensions** | 100+ | Providers, taskservs, clusters |
|
||||||
| **Supported Providers** | 3 | AWS, UpCloud, Local |
|
| **Supported Providers** | 3 | AWS, UpCloud, Local |
|
||||||
@ -183,7 +183,7 @@ The Provisioning Platform is a modern, cloud-native infrastructure automation sy
|
|||||||
│ └────────────────┘ └──────────────────┘ └───────────────────┘ │
|
│ └────────────────┘ └──────────────────┘ └───────────────────┘ │
|
||||||
│ │
|
│ │
|
||||||
└────────────────────────────────────────────────────────────────────────────┘
|
└────────────────────────────────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Multi-Repository Architecture
|
### Multi-Repository Architecture
|
||||||
|
|
||||||
@ -199,7 +199,7 @@ Core system functionality
|
|||||||
├── Configuration system
|
├── Configuration system
|
||||||
├── Workflow engine
|
├── Workflow engine
|
||||||
└── Build/distribution tools
|
└── Build/distribution tools
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Distribution**: `oci://registry/provisioning-core:v3.5.0`
|
**Distribution**: `oci://registry/provisioning-core:v3.5.0`
|
||||||
|
|
||||||
@ -220,7 +220,7 @@ All provider, taskserv, cluster extensions
|
|||||||
├── buildkit/
|
├── buildkit/
|
||||||
├── web/
|
├── web/
|
||||||
└── (10+ more)
|
└── (10+ more)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Distribution**: Each extension as separate OCI artifact
|
**Distribution**: Each extension as separate OCI artifact
|
||||||
|
|
||||||
@ -235,7 +235,7 @@ Platform services
|
|||||||
├── control-center/ (Rust/Yew)
|
├── control-center/ (Rust/Yew)
|
||||||
├── mcp-server/ (Rust)
|
├── mcp-server/ (Rust)
|
||||||
└── api-gateway/ (Rust)
|
└── api-gateway/ (Rust)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Distribution**: Docker images in OCI registry
|
**Distribution**: Docker images in OCI registry
|
||||||
|
|
||||||
@ -268,7 +268,7 @@ Domain Handlers (7 modules)
|
|||||||
├── generation.nu (78 lines)
|
├── generation.nu (78 lines)
|
||||||
├── utilities.nu (157 lines)
|
├── utilities.nu (157 lines)
|
||||||
└── configuration.nu (316 lines)
|
└── configuration.nu (316 lines)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Key Features**:
|
**Key Features**:
|
||||||
|
|
||||||
@ -288,7 +288,7 @@ Domain Handlers (7 modules)
|
|||||||
4. Environment config (workspace/config/{env}-defaults.toml)
|
4. Environment config (workspace/config/{env}-defaults.toml)
|
||||||
5. Infrastructure config (workspace/infra/{name}/config.toml)
|
5. Infrastructure config (workspace/infra/{name}/config.toml)
|
||||||
6. Runtime overrides (CLI flags, ENV variables)
|
6. Runtime overrides (CLI flags, ENV variables)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Variable Interpolation**:
|
**Variable Interpolation**:
|
||||||
|
|
||||||
@ -326,7 +326,7 @@ src/
|
|||||||
├── container_manager.rs
|
├── container_manager.rs
|
||||||
├── test_orchestrator.rs
|
├── test_orchestrator.rs
|
||||||
└── topologies.rs
|
└── topologies.rs
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Key Features**:
|
**Key Features**:
|
||||||
|
|
||||||
@ -349,7 +349,7 @@ workflows/
|
|||||||
├── cluster.nu // Cluster deployment
|
├── cluster.nu // Cluster deployment
|
||||||
├── batch.nu // Batch operations
|
├── batch.nu // Batch operations
|
||||||
└── management.nu // Workflow monitoring
|
└── management.nu // Workflow monitoring
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Batch Workflow Features**:
|
**Batch Workflow Features**:
|
||||||
|
|
||||||
@ -364,7 +364,7 @@ workflows/
|
|||||||
**Extension Types**:
|
**Extension Types**:
|
||||||
|
|
||||||
| Type | Count | Purpose | Example |
|
| Type | Count | Purpose | Example |
|
||||||
|------|-------|---------|---------|
|
| ------ | ------- | --------- | --------- |
|
||||||
| **Providers** | 3 | Cloud platform integration | AWS, UpCloud, Local |
|
| **Providers** | 3 | Cloud platform integration | AWS, UpCloud, Local |
|
||||||
| **Task Services** | 50+ | Infrastructure components | Kubernetes, Postgres |
|
| **Task Services** | 50+ | Infrastructure components | Kubernetes, Postgres |
|
||||||
| **Clusters** | 10+ | Complete configurations | Buildkit, Web cluster |
|
| **Clusters** | 10+ | Complete configurations | Buildkit, Web cluster |
|
||||||
@ -386,7 +386,7 @@ extension-name/
|
|||||||
├── docs/ // Documentation
|
├── docs/ // Documentation
|
||||||
├── tests/ // Extension tests
|
├── tests/ // Extension tests
|
||||||
└── manifest.yaml // Extension metadata
|
└── manifest.yaml // Extension metadata
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**OCI Distribution**:
|
**OCI Distribution**:
|
||||||
Each extension packaged as OCI artifact:
|
Each extension packaged as OCI artifact:
|
||||||
@ -410,7 +410,7 @@ provisioning module load taskserv my-workspace kubernetes containerd
|
|||||||
|
|
||||||
# List loaded modules
|
# List loaded modules
|
||||||
provisioning module list taskserv my-workspace
|
provisioning module list taskserv my-workspace
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Layer System** (Configuration Inheritance):
|
**Layer System** (Configuration Inheritance):
|
||||||
|
|
||||||
@ -420,7 +420,7 @@ Layer 1: Core (provisioning/extensions/{type}/{name})
|
|||||||
Layer 2: Workspace (workspace/extensions/{type}/{name})
|
Layer 2: Workspace (workspace/extensions/{type}/{name})
|
||||||
↓
|
↓
|
||||||
Layer 3: Infrastructure (workspace/infra/{infra}/extensions/{type}/{name})
|
Layer 3: Infrastructure (workspace/infra/{infra}/extensions/{type}/{name})
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Resolution Priority**: Infrastructure → Workspace → Core
|
**Resolution Priority**: Infrastructure → Workspace → Core
|
||||||
|
|
||||||
@ -449,14 +449,14 @@ let { TaskservDependencies } = import "provisioning/dependencies.ncl" in
|
|||||||
conflicts = ["docker", "podman"],
|
conflicts = ["docker", "podman"],
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 8. **Service Management**
|
#### 8. **Service Management**
|
||||||
|
|
||||||
**Supported Services**:
|
**Supported Services**:
|
||||||
|
|
||||||
| Service | Type | Category | Purpose |
|
| Service | Type | Category | Purpose |
|
||||||
|---------|------|----------|---------|
|
| --------- | ------ | ---------- | --------- |
|
||||||
| orchestrator | Platform | Orchestration | Workflow coordination |
|
| orchestrator | Platform | Orchestration | Workflow coordination |
|
||||||
| control-center | Platform | UI | Web management interface |
|
| control-center | Platform | UI | Web management interface |
|
||||||
| coredns | Infrastructure | DNS | Local DNS resolution |
|
| coredns | Infrastructure | DNS | Local DNS resolution |
|
||||||
@ -479,7 +479,7 @@ provisioning platform health
|
|||||||
|
|
||||||
# View logs
|
# View logs
|
||||||
provisioning platform logs orchestrator --follow
|
provisioning platform logs orchestrator --follow
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 9. **Test Environment Service**
|
#### 9. **Test Environment Service**
|
||||||
|
|
||||||
@ -495,7 +495,7 @@ Container Manager (bollard)
|
|||||||
Docker API
|
Docker API
|
||||||
↓
|
↓
|
||||||
Isolated Test Containers
|
Isolated Test Containers
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Test Types**:
|
**Test Types**:
|
||||||
|
|
||||||
@ -552,7 +552,7 @@ The platform supports four operational modes that adapt the system from individu
|
|||||||
│ │ cores, 128 GB │ cores, 64 GB │ 256 GB per user │
|
│ │ cores, 128 GB │ cores, 64 GB │ 256 GB per user │
|
||||||
│ │ │ │ │
|
│ │ │ │ │
|
||||||
└───────────────┴───────────────┴───────────────┴───────────────────────┘
|
└───────────────┴───────────────┴───────────────┴───────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Mode Configuration
|
### Mode Configuration
|
||||||
|
|
||||||
@ -571,7 +571,7 @@ provisioning mode switch multi-user
|
|||||||
|
|
||||||
# Validate mode requirements
|
# Validate mode requirements
|
||||||
provisioning mode validate enterprise
|
provisioning mode validate enterprise
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Mode-Specific Workflows
|
### Mode-Specific Workflows
|
||||||
|
|
||||||
@ -586,7 +586,7 @@ provisioning platform start orchestrator
|
|||||||
|
|
||||||
# 3. Create infrastructure
|
# 3. Create infrastructure
|
||||||
provisioning server create
|
provisioning server create
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Multi-User Mode
|
#### Multi-User Mode
|
||||||
|
|
||||||
@ -605,7 +605,7 @@ provisioning extension pull upcloud kubernetes
|
|||||||
|
|
||||||
# 5. Unlock workspace
|
# 5. Unlock workspace
|
||||||
provisioning workspace unlock my-infra
|
provisioning workspace unlock my-infra
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### CI/CD Mode
|
#### CI/CD Mode
|
||||||
|
|
||||||
@ -622,7 +622,7 @@ deploy:
|
|||||||
- provisioning server create
|
- provisioning server create
|
||||||
after_script:
|
after_script:
|
||||||
- provisioning workspace cleanup
|
- provisioning workspace cleanup
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Enterprise Mode
|
#### Enterprise Mode
|
||||||
|
|
||||||
@ -646,7 +646,7 @@ provisioning infra create
|
|||||||
|
|
||||||
# 6. Release
|
# 6. Release
|
||||||
provisioning workspace unlock prod-deployment
|
provisioning workspace unlock prod-deployment
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -684,12 +684,12 @@ provisioning workspace unlock prod-deployment
|
|||||||
│ └────────────────────────────────────────────────────────────┘ │
|
│ └────────────────────────────────────────────────────────────┘ │
|
||||||
│ │
|
│ │
|
||||||
└──────────────────────────────────────────────────────────────────────┘
|
└──────────────────────────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Port Allocation
|
### Port Allocation
|
||||||
|
|
||||||
| Service | Port | Protocol | Purpose |
|
| Service | Port | Protocol | Purpose |
|
||||||
|---------|------|----------|---------|
|
| --------- | ------ | ---------- | --------- |
|
||||||
| Orchestrator | 8080 | HTTP/WS | REST API, WebSocket |
|
| Orchestrator | 8080 | HTTP/WS | REST API, WebSocket |
|
||||||
| Control Center | 3000 | HTTP | Web UI |
|
| Control Center | 3000 | HTTP | Web UI |
|
||||||
| CoreDNS | 5353 | UDP/TCP | DNS resolution |
|
| CoreDNS | 5353 | UDP/TCP | DNS resolution |
|
||||||
@ -807,7 +807,7 @@ provisioning workspace unlock prod-deployment
|
|||||||
│ └─────────────────────────────────────────────────────────┘ │
|
│ └─────────────────────────────────────────────────────────┘ │
|
||||||
│ │
|
│ │
|
||||||
└────────────────────────────────────────────────────────────────┘
|
└────────────────────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Data Flow
|
### Data Flow
|
||||||
|
|
||||||
@ -820,7 +820,7 @@ provisioning workspace unlock prod-deployment
|
|||||||
4. Load environment config (workspace/config/{env}-defaults.toml)
|
4. Load environment config (workspace/config/{env}-defaults.toml)
|
||||||
5. Load infrastructure config (workspace/infra/{name}/config.toml)
|
5. Load infrastructure config (workspace/infra/{name}/config.toml)
|
||||||
6. Apply runtime overrides (ENV variables, CLI flags)
|
6. Apply runtime overrides (ENV variables, CLI flags)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**State Persistence**:
|
**State Persistence**:
|
||||||
|
|
||||||
@ -832,7 +832,7 @@ Create checkpoint (JSON)
|
|||||||
Save to ~/.provisioning/orchestrator/data/checkpoints/
|
Save to ~/.provisioning/orchestrator/data/checkpoints/
|
||||||
↓
|
↓
|
||||||
On failure, load checkpoint and resume
|
On failure, load checkpoint and resume
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**OCI Artifact Flow**:
|
**OCI Artifact Flow**:
|
||||||
|
|
||||||
@ -842,7 +842,7 @@ On failure, load checkpoint and resume
|
|||||||
3. Extension stored as OCI artifact
|
3. Extension stored as OCI artifact
|
||||||
4. Pull when needed (provisioning oci pull)
|
4. Pull when needed (provisioning oci pull)
|
||||||
5. Cache locally (~/.provisioning/cache/oci/)
|
5. Cache locally (~/.provisioning/cache/oci/)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -915,7 +915,7 @@ On failure, load checkpoint and resume
|
|||||||
│ └────────────────────────────────────────────────────────┘ │
|
│ └────────────────────────────────────────────────────────┘ │
|
||||||
│ │
|
│ │
|
||||||
└─────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Secret Management
|
### Secret Management
|
||||||
|
|
||||||
@ -927,7 +927,7 @@ provisioning sops workspace/secrets/keys.yaml.enc
|
|||||||
|
|
||||||
# Encryption happens automatically on save
|
# Encryption happens automatically on save
|
||||||
# Decryption happens automatically on load
|
# Decryption happens automatically on load
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**KMS Integration** (Enterprise):
|
**KMS Integration** (Enterprise):
|
||||||
|
|
||||||
@ -939,7 +939,7 @@ secrets:
|
|||||||
type: "aws" # or "vault"
|
type: "aws" # or "vault"
|
||||||
region: "us-east-1"
|
region: "us-east-1"
|
||||||
key_id: "arn:aws:kms:..."
|
key_id: "arn:aws:kms:..."
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Image Signing and Verification
|
### Image Signing and Verification
|
||||||
|
|
||||||
@ -951,7 +951,7 @@ cosign sign oci://registry/kubernetes:1.28.0
|
|||||||
|
|
||||||
# Verify signature
|
# Verify signature
|
||||||
cosign verify oci://registry/kubernetes:1.28.0
|
cosign verify oci://registry/kubernetes:1.28.0
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Enterprise Mode** (Mandatory):
|
**Enterprise Mode** (Mandatory):
|
||||||
|
|
||||||
@ -960,7 +960,7 @@ cosign verify oci://registry/kubernetes:1.28.0
|
|||||||
provisioning extension pull kubernetes --verify-signature
|
provisioning extension pull kubernetes --verify-signature
|
||||||
|
|
||||||
# System blocks unsigned artifacts
|
# System blocks unsigned artifacts
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -979,7 +979,7 @@ User Machine
|
|||||||
├── ~/.provisioning/orchestrator/data/
|
├── ~/.provisioning/orchestrator/data/
|
||||||
├── ~/.provisioning/services/
|
├── ~/.provisioning/services/
|
||||||
└── Process Management (PID files, logs)
|
└── Process Management (PID files, logs)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: Simple, fast startup, no Docker dependency
|
**Pros**: Simple, fast startup, no Docker dependency
|
||||||
**Cons**: Platform-specific binaries, manual updates
|
**Cons**: Platform-specific binaries, manual updates
|
||||||
@ -994,7 +994,7 @@ Docker Daemon
|
|||||||
├── Container: provisioning-gitea
|
├── Container: provisioning-gitea
|
||||||
├── Container: provisioning-oci-registry
|
├── Container: provisioning-oci-registry
|
||||||
└── Volumes: ~/.provisioning/data/
|
└── Volumes: ~/.provisioning/data/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: Consistent environment, easy updates
|
**Pros**: Consistent environment, easy updates
|
||||||
**Cons**: Requires Docker, resource overhead
|
**Cons**: Requires Docker, resource overhead
|
||||||
@ -1032,7 +1032,7 @@ services:
|
|||||||
image: ghcr.io/project-zot/zot:latest
|
image: ghcr.io/project-zot/zot:latest
|
||||||
ports:
|
ports:
|
||||||
- "5000:5000"
|
- "5000:5000"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: Easy multi-service orchestration, declarative
|
**Pros**: Easy multi-service orchestration, declarative
|
||||||
**Cons**: Local only, no HA
|
**Cons**: Local only, no HA
|
||||||
@ -1078,7 +1078,7 @@ spec:
|
|||||||
- name: data
|
- name: data
|
||||||
persistentVolumeClaim:
|
persistentVolumeClaim:
|
||||||
claimName: orchestrator-data
|
claimName: orchestrator-data
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: HA, scalability, production-ready
|
**Pros**: HA, scalability, production-ready
|
||||||
**Cons**: Complex setup, Kubernetes required
|
**Cons**: Complex setup, Kubernetes required
|
||||||
@ -1095,7 +1095,7 @@ services:
|
|||||||
endpoint: "https://orchestrator.company.com"
|
endpoint: "https://orchestrator.company.com"
|
||||||
tls_enabled: true
|
tls_enabled: true
|
||||||
auth_token_path: "~/.provisioning/tokens/orchestrator.token"
|
auth_token_path: "~/.provisioning/tokens/orchestrator.token"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: No local resources, centralized
|
**Pros**: No local resources, centralized
|
||||||
**Cons**: Network dependency, latency
|
**Cons**: Network dependency, latency
|
||||||
@ -1118,7 +1118,7 @@ Nushell Business Logic
|
|||||||
Rust Orchestrator
|
Rust Orchestrator
|
||||||
↓ (updates state)
|
↓ (updates state)
|
||||||
File-based Task Queue
|
File-based Task Queue
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Communication**: HTTP API + stdin/stdout JSON
|
**Communication**: HTTP API + stdin/stdout JSON
|
||||||
|
|
||||||
@ -1135,7 +1135,7 @@ Provider Implementations:
|
|||||||
├── AWS Provider (aws-sdk-rust, aws cli)
|
├── AWS Provider (aws-sdk-rust, aws cli)
|
||||||
├── UpCloud Provider (upcloud API)
|
├── UpCloud Provider (upcloud API)
|
||||||
└── Local Provider (Docker, libvirt)
|
└── Local Provider (Docker, libvirt)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 3. **OCI Registry Integration**
|
#### 3. **OCI Registry Integration**
|
||||||
|
|
||||||
@ -1153,7 +1153,7 @@ Pull (provisioning oci pull)
|
|||||||
Cache (~/.provisioning/cache/oci/)
|
Cache (~/.provisioning/cache/oci/)
|
||||||
↓
|
↓
|
||||||
Load into Workspace
|
Load into Workspace
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 4. **Gitea Integration** (Multi-user, Enterprise)
|
#### 4. **Gitea Integration** (Multi-user, Enterprise)
|
||||||
|
|
||||||
@ -1169,7 +1169,7 @@ Perform Changes
|
|||||||
Commit + Push
|
Commit + Push
|
||||||
↓
|
↓
|
||||||
Release Lock (Delete lock file)
|
Release Lock (Delete lock file)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Benefits**:
|
**Benefits**:
|
||||||
|
|
||||||
@ -1192,7 +1192,7 @@ Zones:
|
|||||||
├── *.prov.local (Internal services)
|
├── *.prov.local (Internal services)
|
||||||
├── *.infra.local (Infrastructure nodes)
|
├── *.infra.local (Infrastructure nodes)
|
||||||
└── *.test.local (Test environments)
|
└── *.test.local (Test environments)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -1201,7 +1201,7 @@ Zones:
|
|||||||
### Performance Characteristics
|
### Performance Characteristics
|
||||||
|
|
||||||
| Metric | Value | Notes |
|
| Metric | Value | Notes |
|
||||||
|--------|-------|-------|
|
| -------- | ------- | ------- |
|
||||||
| **CLI Startup Time** | < 100 ms | Nushell cold start |
|
| **CLI Startup Time** | < 100 ms | Nushell cold start |
|
||||||
| **CLI Response Time** | < 50 ms | Most commands |
|
| **CLI Response Time** | < 50 ms | Most commands |
|
||||||
| **Workflow Submission** | < 200 ms | To orchestrator |
|
| **Workflow Submission** | < 200 ms | To orchestrator |
|
||||||
@ -1264,7 +1264,7 @@ Zones:
|
|||||||
### Version History
|
### Version History
|
||||||
|
|
||||||
| Version | Date | Major Features |
|
| Version | Date | Major Features |
|
||||||
|---------|------|----------------|
|
| --------- | ------ | ---------------- |
|
||||||
| **v3.5.0** | 2025-10-06 | Mode system, OCI distribution, comprehensive docs |
|
| **v3.5.0** | 2025-10-06 | Mode system, OCI distribution, comprehensive docs |
|
||||||
| **v3.4.0** | 2025-10-06 | Test environment service |
|
| **v3.4.0** | 2025-10-06 | Test environment service |
|
||||||
| **v3.3.0** | 2025-09-30 | Interactive guides |
|
| **v3.3.0** | 2025-09-30 | Interactive guides |
|
||||||
@ -1316,12 +1316,12 @@ Zones:
|
|||||||
|
|
||||||
### ADRs
|
### ADRs
|
||||||
|
|
||||||
- **[ADR-001](ADR-001-project-structure.md)** - Project structure
|
- **[ADR-001](adr-001-project-structure.md)** - Project structure
|
||||||
- **[ADR-002](ADR-002-distribution-strategy.md)** - Distribution strategy
|
- **[ADR-002](adr-002-distribution-strategy.md)** - Distribution strategy
|
||||||
- **[ADR-003](ADR-003-workspace-isolation.md)** - Workspace isolation
|
- **[ADR-003](adr-003-workspace-isolation.md)** - Workspace isolation
|
||||||
- **[ADR-004](ADR-004-hybrid-architecture.md)** - Hybrid architecture
|
- **[ADR-004](adr-004-hybrid-architecture.md)** - Hybrid architecture
|
||||||
- **[ADR-005](ADR-005-extension-framework.md)** - Extension framework
|
- **[ADR-005](adr-005-extension-framework.md)** - Extension framework
|
||||||
- **[ADR-006](ADR-006-provisioning-cli-refactoring.md)** - CLI refactoring
|
- **[ADR-006](adr-006-provisioning-cli-refactoring.md)** - CLI refactoring
|
||||||
|
|
||||||
### User Guides
|
### User Guides
|
||||||
|
|
||||||
|
|||||||
@ -104,7 +104,7 @@ loader.nu (full configuration)
|
|||||||
├── Interpolation functions
|
├── Interpolation functions
|
||||||
├── Validation functions
|
├── Validation functions
|
||||||
└── Config merging logic
|
└── Config merging logic
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Usage Examples
|
## Usage Examples
|
||||||
|
|
||||||
@ -115,7 +115,7 @@ loader.nu (full configuration)
|
|||||||
./provisioning help infrastructure
|
./provisioning help infrastructure
|
||||||
./provisioning workspace list
|
./provisioning workspace list
|
||||||
./provisioning version
|
./provisioning version
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Medium Path (Status Operations)
|
### Medium Path (Status Operations)
|
||||||
|
|
||||||
@ -124,7 +124,7 @@ loader.nu (full configuration)
|
|||||||
./provisioning status
|
./provisioning status
|
||||||
./provisioning workspace active
|
./provisioning workspace active
|
||||||
./provisioning config validate
|
./provisioning config validate
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Full Path (Infrastructure Operations)
|
### Full Path (Infrastructure Operations)
|
||||||
|
|
||||||
@ -133,7 +133,7 @@ loader.nu (full configuration)
|
|||||||
./provisioning server create --infra myinfra
|
./provisioning server create --infra myinfra
|
||||||
./provisioning taskserv create kubernetes
|
./provisioning taskserv create kubernetes
|
||||||
./provisioning workflow submit batch.yaml
|
./provisioning workflow submit batch.yaml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Implementation Details
|
## Implementation Details
|
||||||
|
|
||||||
@ -154,7 +154,7 @@ if $is_fast_command {
|
|||||||
# Load full configuration (0.091s)
|
# Load full configuration (0.091s)
|
||||||
load-provisioning-config
|
load-provisioning-config
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Minimal Config Structure
|
### Minimal Config Structure
|
||||||
|
|
||||||
@ -172,7 +172,7 @@ The minimal loader returns a lightweight config record:
|
|||||||
base: "/path/to/workspace_librecloud"
|
base: "/path/to/workspace_librecloud"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
This is sufficient for:
|
This is sufficient for:
|
||||||
|
|
||||||
@ -256,7 +256,7 @@ time nu -c "use config/accessor.nu *; get-config"
|
|||||||
|
|
||||||
# Benchmark help command
|
# Benchmark help command
|
||||||
time ./provisioning help infrastructure
|
time ./provisioning help infrastructure
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## See Also
|
## See Also
|
||||||
|
|
||||||
|
|||||||
@ -18,7 +18,7 @@ Control-Center uses **SurrealDB with kv-mem backend**, an embedded in-memory dat
|
|||||||
url = "memory" # In-memory backend
|
url = "memory" # In-memory backend
|
||||||
namespace = "control_center"
|
namespace = "control_center"
|
||||||
database = "main"
|
database = "main"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Storage**: In-memory (data persists during process lifetime)
|
**Storage**: In-memory (data persists during process lifetime)
|
||||||
|
|
||||||
@ -31,12 +31,12 @@ namespace = "control_center"
|
|||||||
database = "main"
|
database = "main"
|
||||||
username = "root"
|
username = "root"
|
||||||
password = "secret"
|
password = "secret"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Why SurrealDB kv-mem
|
### Why SurrealDB kv-mem
|
||||||
|
|
||||||
| Feature | SurrealDB kv-mem | RocksDB | PostgreSQL |
|
| Feature | SurrealDB kv-mem | RocksDB | PostgreSQL |
|
||||||
|---------|------------------|---------|------------|
|
| --------- | ------------------ | --------- | ------------ |
|
||||||
| **Deployment** | Embedded (no server) | Embedded | Server only |
|
| **Deployment** | Embedded (no server) | Embedded | Server only |
|
||||||
| **Build Deps** | None | libclang, bzip2 | Many |
|
| **Build Deps** | None | libclang, bzip2 | Many |
|
||||||
| **Docker** | Simple | Complex | External service |
|
| **Docker** | Simple | Complex | External service |
|
||||||
@ -83,13 +83,13 @@ Orchestrator uses simple file-based storage by default:
|
|||||||
[orchestrator.storage]
|
[orchestrator.storage]
|
||||||
type = "filesystem" # Default
|
type = "filesystem" # Default
|
||||||
backend_path = "{{orchestrator.paths.data_dir}}/queue.rkvs"
|
backend_path = "{{orchestrator.paths.data_dir}}/queue.rkvs"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Resolved Path**:
|
**Resolved Path**:
|
||||||
|
|
||||||
```plaintext
|
```plaintext
|
||||||
{{workspace.path}}/.orchestrator/data/queue.rkvs
|
{{workspace.path}}/.orchestrator/data/queue.rkvs
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Optional: SurrealDB Backend
|
### Optional: SurrealDB Backend
|
||||||
|
|
||||||
@ -105,7 +105,7 @@ namespace = "orchestrator"
|
|||||||
database = "tasks"
|
database = "tasks"
|
||||||
username = "root"
|
username = "root"
|
||||||
password = "secret"
|
password = "secret"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -122,7 +122,7 @@ All services load configuration in this order (priority: low → high):
|
|||||||
4. User Config ~/Library/Application Support/provisioning/user_config.yaml
|
4. User Config ~/Library/Application Support/provisioning/user_config.yaml
|
||||||
5. Environment Variables PROVISIONING_*, CONTROL_CENTER_*, ORCHESTRATOR_*
|
5. Environment Variables PROVISIONING_*, CONTROL_CENTER_*, ORCHESTRATOR_*
|
||||||
6. Runtime Overrides --config flag or API updates
|
6. Runtime Overrides --config flag or API updates
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Variable Interpolation
|
### Variable Interpolation
|
||||||
|
|
||||||
@ -136,7 +136,7 @@ data_dir = "{{paths.base}}/data" # Resolves to: /Users/.../data
|
|||||||
[database]
|
[database]
|
||||||
url = "rocksdb://{{paths.data_dir}}/control-center.db"
|
url = "rocksdb://{{paths.data_dir}}/control-center.db"
|
||||||
# Resolves to: rocksdb:///Users/.../data/control-center.db
|
# Resolves to: rocksdb:///Users/.../data/control-center.db
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Supported Variables**:
|
**Supported Variables**:
|
||||||
|
|
||||||
@ -151,7 +151,7 @@ url = "rocksdb://{{paths.data_dir}}/control-center.db"
|
|||||||
Each platform service has its own `config.defaults.toml`:
|
Each platform service has its own `config.defaults.toml`:
|
||||||
|
|
||||||
| Service | Config File | Purpose |
|
| Service | Config File | Purpose |
|
||||||
|---------|-------------|---------|
|
| --------- | ------------- | --------- |
|
||||||
| **Orchestrator** | `provisioning/platform/orchestrator/config.defaults.toml` | Workflow management, queue settings |
|
| **Orchestrator** | `provisioning/platform/orchestrator/config.defaults.toml` | Workflow management, queue settings |
|
||||||
| **Control-Center** | `provisioning/platform/control-center/config.defaults.toml` | Web UI, auth, database |
|
| **Control-Center** | `provisioning/platform/control-center/config.defaults.toml` | Web UI, auth, database |
|
||||||
| **MCP Server** | `provisioning/platform/mcp-server/config.defaults.toml` | AI integration settings |
|
| **MCP Server** | `provisioning/platform/mcp-server/config.defaults.toml` | AI integration settings |
|
||||||
@ -181,7 +181,7 @@ base = "{{workspace.path}}/.orchestrator"
|
|||||||
data_dir = "{{orchestrator.paths.base}}/data"
|
data_dir = "{{orchestrator.paths.base}}/data"
|
||||||
logs_dir = "{{orchestrator.paths.base}}/logs"
|
logs_dir = "{{orchestrator.paths.base}}/logs"
|
||||||
queue_dir = "{{orchestrator.paths.data_dir}}/queue"
|
queue_dir = "{{orchestrator.paths.data_dir}}/queue"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Control-Center**:
|
**Control-Center**:
|
||||||
|
|
||||||
@ -190,7 +190,7 @@ queue_dir = "{{orchestrator.paths.data_dir}}/queue"
|
|||||||
base = "{{workspace.path}}/.control-center"
|
base = "{{workspace.path}}/.control-center"
|
||||||
data_dir = "{{paths.base}}/data"
|
data_dir = "{{paths.base}}/data"
|
||||||
logs_dir = "{{paths.base}}/logs"
|
logs_dir = "{{paths.base}}/logs"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Result** (workspace: `workspace-librecloud`):
|
**Result** (workspace: `workspace-librecloud`):
|
||||||
|
|
||||||
@ -204,7 +204,7 @@ workspace-librecloud/
|
|||||||
├── data/
|
├── data/
|
||||||
│ └── control-center.db
|
│ └── control-center.db
|
||||||
└── logs/
|
└── logs/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -223,7 +223,7 @@ export CONTROL_CENTER_DATABASE_URL="rocksdb:///custom/path/db"
|
|||||||
|
|
||||||
# Override JWT secret
|
# Override JWT secret
|
||||||
export CONTROL_CENTER_JWT_ISSUER="my-issuer"
|
export CONTROL_CENTER_JWT_ISSUER="my-issuer"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Orchestrator
|
### Orchestrator
|
||||||
|
|
||||||
@ -237,13 +237,13 @@ export ORCHESTRATOR_STORAGE_SURREALDB_URL="ws://localhost:8000"
|
|||||||
|
|
||||||
# Override concurrency
|
# Override concurrency
|
||||||
export ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS=10
|
export ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS=10
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Naming Convention
|
### Naming Convention
|
||||||
|
|
||||||
```plaintext
|
```plaintext
|
||||||
{SERVICE}_{SECTION}_{KEY} = value
|
{SERVICE}_{SECTION}_{KEY} = value
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Examples**:
|
**Examples**:
|
||||||
|
|
||||||
@ -264,7 +264,7 @@ export ORCHESTRATOR_QUEUE_MAX_CONCURRENT_TASKS=10
|
|||||||
base = "/app/provisioning"
|
base = "/app/provisioning"
|
||||||
data_dir = "/data" # Mounted volume
|
data_dir = "/data" # Mounted volume
|
||||||
logs_dir = "/var/log/orchestrator" # Mounted volume
|
logs_dir = "/var/log/orchestrator" # Mounted volume
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Docker Compose volumes**:
|
**Docker Compose volumes**:
|
||||||
|
|
||||||
@ -283,7 +283,7 @@ volumes:
|
|||||||
orchestrator-data:
|
orchestrator-data:
|
||||||
orchestrator-logs:
|
orchestrator-logs:
|
||||||
control-center-data:
|
control-center-data:
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Native Deployment
|
### Native Deployment
|
||||||
|
|
||||||
@ -294,7 +294,7 @@ volumes:
|
|||||||
base = "/Users/Akasha/project-provisioning/provisioning"
|
base = "/Users/Akasha/project-provisioning/provisioning"
|
||||||
data_dir = "{{workspace.path}}/.orchestrator/data"
|
data_dir = "{{workspace.path}}/.orchestrator/data"
|
||||||
logs_dir = "{{workspace.path}}/.orchestrator/logs"
|
logs_dir = "{{workspace.path}}/.orchestrator/logs"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -314,7 +314,7 @@ provisioning validate config
|
|||||||
|
|
||||||
# Show service-specific config
|
# Show service-specific config
|
||||||
PROVISIONING_DEBUG=true ./orchestrator --show-config
|
PROVISIONING_DEBUG=true ./orchestrator --show-config
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -328,7 +328,7 @@ PROVISIONING_DEBUG=true ./orchestrator --show-config
|
|||||||
|
|
||||||
# KMS database location (Native)
|
# KMS database location (Native)
|
||||||
{{workspace.path}}/.kms/data/kms.db
|
{{workspace.path}}/.kms/data/kms.db
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
KMS also integrates with Control-Center's KMS hybrid backend (local + remote):
|
KMS also integrates with Control-Center's KMS hybrid backend (local + remote):
|
||||||
|
|
||||||
@ -341,7 +341,7 @@ database_path = "{{paths.data_dir}}/kms.db"
|
|||||||
|
|
||||||
[kms.remote]
|
[kms.remote]
|
||||||
server_url = "http://localhost:9998" # Cosmian KMS server
|
server_url = "http://localhost:9998" # Cosmian KMS server
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -2,7 +2,10 @@
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Provisioning is built on a foundation of architectural principles that guide design decisions, ensure system quality, and maintain consistency across the codebase. These principles have evolved from real-world experience and represent lessons learned from complex infrastructure automation challenges.
|
Provisioning is built on a foundation of architectural principles that guide design decisions,
|
||||||
|
ensure system quality, and maintain consistency across the codebase.
|
||||||
|
These principles have evolved from real-world experience
|
||||||
|
and represent lessons learned from complex infrastructure automation challenges.
|
||||||
|
|
||||||
## Core Architectural Principles
|
## Core Architectural Principles
|
||||||
|
|
||||||
@ -10,7 +13,8 @@ Provisioning is built on a foundation of architectural principles that guide des
|
|||||||
|
|
||||||
**Principle**: Fully agnostic and configuration-driven, not hardcoded. Use abstraction layers dynamically loaded from configurations.
|
**Principle**: Fully agnostic and configuration-driven, not hardcoded. Use abstraction layers dynamically loaded from configurations.
|
||||||
|
|
||||||
**Rationale**: Infrastructure as Code (IaC) systems must be flexible enough to adapt to any environment without code changes. Hardcoded values defeat the purpose of IaC and create maintenance burdens.
|
**Rationale**: Infrastructure as Code (IaC) systems must be flexible enough to adapt to any environment
|
||||||
|
without code changes. Hardcoded values defeat the purpose of IaC and create maintenance burdens.
|
||||||
|
|
||||||
**Implementation Guidelines**:
|
**Implementation Guidelines**:
|
||||||
|
|
||||||
@ -39,13 +43,14 @@ api_endpoint = "https://ec2.amazonaws.com"
|
|||||||
if config.providers.aws.regions.is_empty() {
|
if config.providers.aws.regions.is_empty() {
|
||||||
regions = vec!["us-west-2"]; // Hardcoded fallback
|
regions = vec!["us-west-2"]; // Hardcoded fallback
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Hybrid Architecture Optimization
|
### 2. Hybrid Architecture Optimization
|
||||||
|
|
||||||
**Principle**: Use each language for what it does best - Rust for coordination, Nushell for business logic.
|
**Principle**: Use each language for what it does best - Rust for coordination, Nushell for business logic.
|
||||||
|
|
||||||
**Rationale**: Different languages have different strengths. Rust excels at performance-critical coordination tasks, while Nushell excels at configuration management and domain-specific operations.
|
**Rationale**: Different languages have different strengths. Rust excels at performance-critical coordination tasks, while Nushell excels at
|
||||||
|
configuration management and domain-specific operations.
|
||||||
|
|
||||||
**Implementation Guidelines**:
|
**Implementation Guidelines**:
|
||||||
|
|
||||||
@ -73,13 +78,14 @@ Nushell Layer:
|
|||||||
├── Template generation and Infrastructure as Code
|
├── Template generation and Infrastructure as Code
|
||||||
├── CLI user interfaces and interactive tools
|
├── CLI user interfaces and interactive tools
|
||||||
└── Domain-specific business logic
|
└── Domain-specific business logic
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Configuration-First Architecture
|
### 3. Configuration-First Architecture
|
||||||
|
|
||||||
**Principle**: All system behavior is determined by configuration, with clear hierarchical precedence and validation.
|
**Principle**: All system behavior is determined by configuration, with clear hierarchical precedence and validation.
|
||||||
|
|
||||||
**Rationale**: True Infrastructure as Code requires that all behavior be configurable without code changes. Configuration hierarchy provides flexibility while maintaining predictability.
|
**Rationale**: True Infrastructure as Code requires that all behavior be configurable without code changes. Configuration hierarchy provides
|
||||||
|
flexibility while maintaining predictability.
|
||||||
|
|
||||||
**Configuration Hierarchy** (precedence order):
|
**Configuration Hierarchy** (precedence order):
|
||||||
|
|
||||||
@ -112,7 +118,7 @@ Nushell Layer:
|
|||||||
├── control-center/ # Web-based management interface
|
├── control-center/ # Web-based management interface
|
||||||
├── tools/ # Development and utility tools
|
├── tools/ # Development and utility tools
|
||||||
└── extensions/ # Plugin and extension framework
|
└── extensions/ # Plugin and extension framework
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Domain Responsibilities**:
|
**Domain Responsibilities**:
|
||||||
|
|
||||||
@ -125,7 +131,8 @@ Nushell Layer:
|
|||||||
|
|
||||||
**Principle**: Components are isolated, modular, and independently deployable with clear interface contracts.
|
**Principle**: Components are isolated, modular, and independently deployable with clear interface contracts.
|
||||||
|
|
||||||
**Rationale**: Isolation enables independent development, testing, and deployment. Clear interfaces prevent tight coupling and enable system evolution.
|
**Rationale**: Isolation enables independent development, testing, and deployment. Clear interfaces prevent tight coupling and enable system
|
||||||
|
evolution.
|
||||||
|
|
||||||
**Implementation Guidelines**:
|
**Implementation Guidelines**:
|
||||||
|
|
||||||
@ -171,13 +178,14 @@ System Level:
|
|||||||
├── Automatic recovery procedures
|
├── Automatic recovery procedures
|
||||||
├── Data backup and restoration
|
├── Data backup and restoration
|
||||||
└── Disaster recovery capabilities
|
└── Disaster recovery capabilities
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 7. Performance Through Parallelism
|
### 7. Performance Through Parallelism
|
||||||
|
|
||||||
**Principle**: Design for parallel execution and efficient resource utilization while maintaining correctness.
|
**Principle**: Design for parallel execution and efficient resource utilization while maintaining correctness.
|
||||||
|
|
||||||
**Rationale**: Infrastructure operations often involve multiple independent resources that can be processed in parallel for significant performance gains.
|
**Rationale**: Infrastructure operations often involve multiple independent resources that can be processed in parallel for significant performance
|
||||||
|
gains.
|
||||||
|
|
||||||
**Implementation Guidelines**:
|
**Implementation Guidelines**:
|
||||||
|
|
||||||
@ -213,7 +221,7 @@ Isolation Boundaries:
|
|||||||
├── Extension sandboxing
|
├── Extension sandboxing
|
||||||
├── Provider credential isolation
|
├── Provider credential isolation
|
||||||
└── Process and network isolation
|
└── Process and network isolation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Development Methodology Principles
|
## Development Methodology Principles
|
||||||
|
|
||||||
@ -221,7 +229,8 @@ Isolation Boundaries:
|
|||||||
|
|
||||||
**Principle**: Tests should be configuration-driven and validate both happy path and error conditions.
|
**Principle**: Tests should be configuration-driven and validate both happy path and error conditions.
|
||||||
|
|
||||||
**Rationale**: Infrastructure systems must work across diverse environments and configurations. Tests must validate the configuration-driven nature of the system.
|
**Rationale**: Infrastructure systems must work across diverse environments and configurations. Tests must validate the configuration-driven nature of
|
||||||
|
the system.
|
||||||
|
|
||||||
**Testing Strategy**:
|
**Testing Strategy**:
|
||||||
|
|
||||||
@ -243,7 +252,7 @@ System Testing:
|
|||||||
├── Upgrade and migration tests
|
├── Upgrade and migration tests
|
||||||
├── Performance and scalability tests
|
├── Performance and scalability tests
|
||||||
└── Security and isolation tests
|
└── Security and isolation tests
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Error Handling Principles
|
## Error Handling Principles
|
||||||
|
|
||||||
@ -281,7 +290,7 @@ System Errors:
|
|||||||
├── Memory and resource exhaustion
|
├── Memory and resource exhaustion
|
||||||
├── Process communication failures
|
├── Process communication failures
|
||||||
└── External dependency failures
|
└── External dependency failures
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 12. Observable Operations
|
### 12. Observable Operations
|
||||||
|
|
||||||
@ -309,7 +318,7 @@ Monitoring:
|
|||||||
├── Real-time status reporting
|
├── Real-time status reporting
|
||||||
├── Workflow progress tracking
|
├── Workflow progress tracking
|
||||||
└── Alert integration capabilities
|
└── Alert integration capabilities
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Evolution and Maintenance Principles
|
## Evolution and Maintenance Principles
|
||||||
|
|
||||||
@ -361,7 +370,7 @@ Improvement:
|
|||||||
├── Performance optimization based on metrics
|
├── Performance optimization based on metrics
|
||||||
├── Security enhancement and hardening
|
├── Security enhancement and hardening
|
||||||
└── Test coverage improvement and validation
|
└── Test coverage improvement and validation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Trade-off Management
|
## Trade-off Management
|
||||||
|
|
||||||
@ -391,13 +400,16 @@ Security vs. Usability:
|
|||||||
├── Extension sandboxing vs. functionality
|
├── Extension sandboxing vs. functionality
|
||||||
├── Authentication requirements vs. ease of use
|
├── Authentication requirements vs. ease of use
|
||||||
└── Audit logging vs. performance overhead
|
└── Audit logging vs. performance overhead
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Conclusion
|
## Conclusion
|
||||||
|
|
||||||
These design principles form the foundation of provisioning's architecture. They guide decision making, ensure quality, and provide a framework for system evolution. Adherence to these principles has enabled the development of a sophisticated, reliable, and maintainable infrastructure automation platform.
|
These design principles form the foundation of provisioning's architecture. They guide decision making, ensure quality, and provide a framework for
|
||||||
|
system evolution. Adherence to these principles has enabled the development of a sophisticated, reliable, and maintainable infrastructure automation
|
||||||
|
platform.
|
||||||
|
|
||||||
The principles are living guidelines that evolve with the system while maintaining core architectural integrity. They serve as both implementation guidance and evaluation criteria for new features and modifications.
|
The principles are living guidelines that evolve with the system while maintaining core architectural integrity. They serve as both implementation
|
||||||
|
guidance and evaluation criteria for new features and modifications.
|
||||||
|
|
||||||
Success in applying these principles is measured by:
|
Success in applying these principles is measured by:
|
||||||
|
|
||||||
|
|||||||
@ -54,7 +54,7 @@ This document describes the **hybrid selective integration** of prov-ecosystem a
|
|||||||
│ ✅ gitops: Event-driven automation │
|
│ ✅ gitops: Event-driven automation │
|
||||||
│ ✅ provctl-machines: SSH advanced │
|
│ ✅ provctl-machines: SSH advanced │
|
||||||
└─────────────────────────────────────────────┘
|
└─────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -81,7 +81,7 @@ pub enum ContainerRuntime {
|
|||||||
|
|
||||||
pub struct RuntimeDetector { ... }
|
pub struct RuntimeDetector { ... }
|
||||||
pub struct ComposeAdapter { ... }
|
pub struct ComposeAdapter { ... }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Nushell Functions**:
|
**Nushell Functions**:
|
||||||
|
|
||||||
@ -91,7 +91,7 @@ runtime-exec # Execute command in detected runtime
|
|||||||
runtime-compose # Adapt docker-compose for runtime
|
runtime-compose # Adapt docker-compose for runtime
|
||||||
runtime-info # Get runtime details
|
runtime-info # Get runtime details
|
||||||
runtime-list # List all available runtimes
|
runtime-list # List all available runtimes
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Benefits**:
|
**Benefits**:
|
||||||
|
|
||||||
@ -120,7 +120,7 @@ pub enum DeploymentStrategy {
|
|||||||
BlueGreen,
|
BlueGreen,
|
||||||
Canary,
|
Canary,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Nushell Functions**:
|
**Nushell Functions**:
|
||||||
|
|
||||||
@ -131,7 +131,7 @@ ssh-pool-status # Check pool status
|
|||||||
ssh-deployment-strategies # List strategies
|
ssh-deployment-strategies # List strategies
|
||||||
ssh-retry-config # Configure retry strategy
|
ssh-retry-config # Configure retry strategy
|
||||||
ssh-circuit-breaker-status # Check circuit breaker
|
ssh-circuit-breaker-status # Check circuit breaker
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Features**:
|
**Features**:
|
||||||
|
|
||||||
@ -165,7 +165,7 @@ pub enum BackupBackend {
|
|||||||
pub struct BackupJob { ... }
|
pub struct BackupJob { ... }
|
||||||
pub struct RetentionPolicy { ... }
|
pub struct RetentionPolicy { ... }
|
||||||
pub struct BackupManager { ... }
|
pub struct BackupManager { ... }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Nushell Functions**:
|
**Nushell Functions**:
|
||||||
|
|
||||||
@ -176,7 +176,7 @@ backup-list # List snapshots
|
|||||||
backup-schedule # Schedule regular backups
|
backup-schedule # Schedule regular backups
|
||||||
backup-retention # Configure retention policy
|
backup-retention # Configure retention policy
|
||||||
backup-status # Check backup status
|
backup-status # Check backup status
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Features**:
|
**Features**:
|
||||||
|
|
||||||
@ -208,7 +208,7 @@ pub enum GitProvider {
|
|||||||
|
|
||||||
pub struct GitOpsRule { ... }
|
pub struct GitOpsRule { ... }
|
||||||
pub struct GitOpsOrchestrator { ... }
|
pub struct GitOpsOrchestrator { ... }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Nushell Functions**:
|
**Nushell Functions**:
|
||||||
|
|
||||||
@ -220,7 +220,7 @@ gitops-event-types # List supported events
|
|||||||
gitops-rule-config # Configure GitOps rule
|
gitops-rule-config # Configure GitOps rule
|
||||||
gitops-deployments # List active deployments
|
gitops-deployments # List active deployments
|
||||||
gitops-status # Get GitOps status
|
gitops-status # Get GitOps status
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Features**:
|
**Features**:
|
||||||
|
|
||||||
@ -252,7 +252,7 @@ service-status # Get service status
|
|||||||
service-list # List all services
|
service-list # List all services
|
||||||
service-restart-policy # Configure restart policy
|
service-restart-policy # Configure restart policy
|
||||||
service-detect-init # Detect init system
|
service-detect-init # Detect init system
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Features**:
|
**Features**:
|
||||||
|
|
||||||
@ -330,7 +330,7 @@ provisioning/
|
|||||||
├── backup.ncl # Backup schema
|
├── backup.ncl # Backup schema
|
||||||
├── gitops.ncl # GitOps schema
|
├── gitops.ncl # GitOps schema
|
||||||
└── service.ncl # Service schema
|
└── service.ncl # Service schema
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -347,7 +347,7 @@ runtime-exec "docker ps" --check
|
|||||||
|
|
||||||
# Adapt compose file
|
# Adapt compose file
|
||||||
let compose_cmd = (runtime-compose "./docker-compose.yml")
|
let compose_cmd = (runtime-compose "./docker-compose.yml")
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### SSH Advanced
|
### SSH Advanced
|
||||||
|
|
||||||
@ -360,7 +360,7 @@ let results = (ssh-pool-exec $hosts "systemctl status provisioning" --strategy p
|
|||||||
|
|
||||||
# Check circuit breaker
|
# Check circuit breaker
|
||||||
ssh-circuit-breaker-status
|
ssh-circuit-breaker-status
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Backup System
|
### Backup System
|
||||||
|
|
||||||
@ -377,7 +377,7 @@ backup-create "full-backup" ["/home" "/opt"] \
|
|||||||
|
|
||||||
# Restore from snapshot
|
# Restore from snapshot
|
||||||
backup-restore "snapshot-001" --restore_path "."
|
backup-restore "snapshot-001" --restore_path "."
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### GitOps Events
|
### GitOps Events
|
||||||
|
|
||||||
@ -390,7 +390,7 @@ gitops-watch --provider "github" --webhook-port 8080
|
|||||||
|
|
||||||
# Manually trigger deployment
|
# Manually trigger deployment
|
||||||
gitops-trigger "deploy-app" --environment "prod"
|
gitops-trigger "deploy-app" --environment "prod"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Service Management
|
### Service Management
|
||||||
|
|
||||||
@ -408,7 +408,7 @@ service-status "my-app"
|
|||||||
|
|
||||||
# Set restart policy
|
# Set restart policy
|
||||||
service-restart-policy "my-app" --policy "on-failure" --delay-secs 5
|
service-restart-policy "my-app" --policy "on-failure" --delay-secs 5
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -424,7 +424,7 @@ provisioning ssh pool connect|exec|status|strategies
|
|||||||
provisioning backup create|restore|list|schedule|retention|status
|
provisioning backup create|restore|list|schedule|retention|status
|
||||||
provisioning gitops rules|watch|trigger|events|config|deployments|status
|
provisioning gitops rules|watch|trigger|events|config|deployments|status
|
||||||
provisioning service install|start|stop|restart|status|list|policy|detect-init
|
provisioning service install|start|stop|restart|status|list|policy|detect-init
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration
|
### Configuration
|
||||||
|
|
||||||
@ -439,7 +439,7 @@ let { IntegrationConfig } = import "provisioning/integrations.ncl" in
|
|||||||
gitops = { ... },
|
gitops = { ... },
|
||||||
service = { ... },
|
service = { ... },
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Plugins
|
### Plugins
|
||||||
|
|
||||||
@ -452,7 +452,7 @@ provisioning plugin list
|
|||||||
# nu_plugin_ssh_advanced
|
# nu_plugin_ssh_advanced
|
||||||
# nu_plugin_backup
|
# nu_plugin_backup
|
||||||
# nu_plugin_gitops
|
# nu_plugin_gitops
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -465,21 +465,21 @@ cd provisioning/platform/integrations/provisioning-bridge
|
|||||||
cargo test --all
|
cargo test --all
|
||||||
cargo test -p provisioning-bridge --lib
|
cargo test -p provisioning-bridge --lib
|
||||||
cargo test -p provisioning-bridge --doc
|
cargo test -p provisioning-bridge --doc
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Nushell Tests
|
### Nushell Tests
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
nu provisioning/core/nulib/integrations/runtime.nu
|
nu provisioning/core/nulib/integrations/runtime.nu
|
||||||
nu provisioning/core/nulib/integrations/ssh_advanced.nu
|
nu provisioning/core/nulib/integrations/ssh_advanced.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Performance
|
## Performance
|
||||||
|
|
||||||
| Operation | Performance |
|
| Operation | Performance |
|
||||||
|-----------|-------------|
|
| ----------- | ------------- |
|
||||||
| Runtime detection | ~50 ms (cached: ~1 ms) |
|
| Runtime detection | ~50 ms (cached: ~1 ms) |
|
||||||
| SSH pool init | ~100 ms per connection |
|
| SSH pool init | ~100 ms per connection |
|
||||||
| SSH command exec | 90% faster with pooling |
|
| SSH command exec | 90% faster with pooling |
|
||||||
|
|||||||
@ -2,7 +2,8 @@
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Provisioning implements sophisticated integration patterns to coordinate between its hybrid Rust/Nushell architecture, manage multi-provider workflows, and enable extensible functionality. This document outlines the key integration patterns, their implementations, and best practices.
|
Provisioning implements sophisticated integration patterns to coordinate between its hybrid Rust/Nushell architecture, manage multi-provider
|
||||||
|
workflows, and enable extensible functionality. This document outlines the key integration patterns, their implementations, and best practices.
|
||||||
|
|
||||||
## Core Integration Patterns
|
## Core Integration Patterns
|
||||||
|
|
||||||
@ -618,4 +619,5 @@ mod integration_tests {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
These integration patterns provide the foundation for the system's sophisticated multi-component architecture, enabling reliable, scalable, and maintainable infrastructure automation.
|
These integration patterns provide the foundation for the system's sophisticated multi-component architecture, enabling reliable, scalable, and
|
||||||
|
maintainable infrastructure automation.
|
||||||
|
|||||||
@ -6,7 +6,8 @@
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
This document describes the multi-repository architecture for the provisioning system, enabling modular development, independent versioning, and distributed extension management through OCI registry integration.
|
This document describes the multi-repository architecture for the provisioning system, enabling modular development, independent versioning, and
|
||||||
|
distributed extension management through OCI registry integration.
|
||||||
|
|
||||||
## Architecture Goals
|
## Architecture Goals
|
||||||
|
|
||||||
@ -60,7 +61,7 @@ provisioning-core/
|
|||||||
├── architecture/ # Architecture docs
|
├── architecture/ # Architecture docs
|
||||||
└── development/ # Development guides
|
└── development/ # Development guides
|
||||||
|
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Distribution**:
|
**Distribution**:
|
||||||
|
|
||||||
@ -131,7 +132,7 @@ provisioning-extensions/
|
|||||||
├── extension-guide.md # Extension development guide
|
├── extension-guide.md # Extension development guide
|
||||||
└── publishing.md # Publishing guide
|
└── publishing.md # Publishing guide
|
||||||
|
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Distribution**:
|
**Distribution**:
|
||||||
Each extension published separately as OCI artifact:
|
Each extension published separately as OCI artifact:
|
||||||
@ -166,13 +167,13 @@ platforms:
|
|||||||
- linux/arm64
|
- linux/arm64
|
||||||
|
|
||||||
min_provisioning_version: "3.0.0"
|
min_provisioning_version: "3.0.0"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**CI/CD**:
|
**CI/CD**:
|
||||||
|
|
||||||
- Build and publish each extension independently
|
- Build and publish each extension independently
|
||||||
- Git tag format: `{extension-type}/{extension-name}/v{version}`
|
- Git tag format: `{extension-type}/{extension-name}/v{version}`
|
||||||
- Example: `taskservs/kubernetes/v1.28.0`
|
- Example: `taskservs/kubernetes/v1.28.0`
|
||||||
- Automated publishing to OCI registry on tag
|
- Automated publishing to OCI registry on tag
|
||||||
- Run extension-specific tests before publishing
|
- Run extension-specific tests before publishing
|
||||||
|
|
||||||
@ -214,7 +215,7 @@ provisioning-platform/
|
|||||||
├── deployment.md
|
├── deployment.md
|
||||||
└── api-reference.md
|
└── api-reference.md
|
||||||
|
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Distribution**:
|
**Distribution**:
|
||||||
Standard Docker images in OCI registry:
|
Standard Docker images in OCI registry:
|
||||||
@ -256,7 +257,7 @@ OCI Registry (localhost:5000 or harbor.company.com)
|
|||||||
├── mcp-server:v1.0.0
|
├── mcp-server:v1.0.0
|
||||||
└── api-gateway:v1.0.0
|
└── api-gateway:v1.0.0
|
||||||
|
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### OCI Artifact Structure
|
### OCI Artifact Structure
|
||||||
|
|
||||||
@ -280,7 +281,7 @@ kubernetes-1.28.0.tar.gz
|
|||||||
├── manifest.yaml # Extension manifest
|
├── manifest.yaml # Extension manifest
|
||||||
└── oci-manifest.json # OCI manifest metadata
|
└── oci-manifest.json # OCI manifest metadata
|
||||||
|
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -345,7 +346,7 @@ dependencies:
|
|||||||
nickel: "provisioning-nickel"
|
nickel: "provisioning-nickel"
|
||||||
platform: "provisioning-platform"
|
platform: "provisioning-platform"
|
||||||
test: "provisioning-test"
|
test: "provisioning-test"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Dependency Resolution
|
### Dependency Resolution
|
||||||
|
|
||||||
@ -377,7 +378,7 @@ provisioning dep validate
|
|||||||
|
|
||||||
# Show dependency tree
|
# Show dependency tree
|
||||||
provisioning dep tree kubernetes
|
provisioning dep tree kubernetes
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -414,7 +415,7 @@ provisioning oci delete kubernetes:1.28.0
|
|||||||
provisioning oci copy \
|
provisioning oci copy \
|
||||||
localhost:5000/provisioning-extensions/kubernetes:1.28.0 \
|
localhost:5000/provisioning-extensions/kubernetes:1.28.0 \
|
||||||
harbor.company.com/provisioning-extensions/kubernetes:1.28.0
|
harbor.company.com/provisioning-extensions/kubernetes:1.28.0
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### OCI Configuration
|
### OCI Configuration
|
||||||
|
|
||||||
@ -433,7 +434,7 @@ provisioning oci config
|
|||||||
cache_dir: "~/.provisioning/oci-cache"
|
cache_dir: "~/.provisioning/oci-cache"
|
||||||
tls_enabled: false
|
tls_enabled: false
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -461,7 +462,7 @@ provisioning generate extension taskserv redis
|
|||||||
# │ └── README.md
|
# │ └── README.md
|
||||||
# ├── tests/
|
# ├── tests/
|
||||||
# └── manifest.yaml
|
# └── manifest.yaml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Test Extension Locally
|
### 2. Test Extension Locally
|
||||||
|
|
||||||
@ -474,7 +475,7 @@ provisioning taskserv create redis --infra test-env --check
|
|||||||
|
|
||||||
# Run extension tests
|
# Run extension tests
|
||||||
provisioning test extension redis
|
provisioning test extension redis
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Package Extension
|
### 3. Package Extension
|
||||||
|
|
||||||
@ -486,7 +487,7 @@ provisioning oci package validate ./extensions/taskservs/redis
|
|||||||
provisioning oci package ./extensions/taskservs/redis
|
provisioning oci package ./extensions/taskservs/redis
|
||||||
|
|
||||||
# Output: redis-1.0.0.tar.gz
|
# Output: redis-1.0.0.tar.gz
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Publish Extension
|
### 4. Publish Extension
|
||||||
|
|
||||||
@ -506,7 +507,7 @@ provisioning oci tags redis
|
|||||||
# ├───────────┼─────────┼───────────────────────────────────────────────────┤
|
# ├───────────┼─────────┼───────────────────────────────────────────────────┤
|
||||||
# │ redis │ 1.0.0 │ localhost:5000/provisioning-extensions/redis:1.0.0│
|
# │ redis │ 1.0.0 │ localhost:5000/provisioning-extensions/redis:1.0.0│
|
||||||
# └───────────┴─────────┴───────────────────────────────────────────────────┘
|
# └───────────┴─────────┴───────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 5. Use Published Extension
|
### 5. Use Published Extension
|
||||||
|
|
||||||
@ -523,7 +524,7 @@ provisioning oci tags redis
|
|||||||
provisioning dep resolve
|
provisioning dep resolve
|
||||||
|
|
||||||
# Extension automatically downloaded and installed
|
# Extension automatically downloaded and installed
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -548,7 +549,7 @@ provisioning oci-registry stop
|
|||||||
|
|
||||||
# Check status
|
# Check status
|
||||||
provisioning oci-registry status
|
provisioning oci-registry status
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Remote Registry (Multi-User/Enterprise)
|
### Remote Registry (Multi-User/Enterprise)
|
||||||
|
|
||||||
@ -566,7 +567,7 @@ dependencies:
|
|||||||
platform: "provisioning/platform"
|
platform: "provisioning/platform"
|
||||||
tls_enabled: true
|
tls_enabled: true
|
||||||
auth_token_path: "~/.provisioning/tokens/harbor"
|
auth_token_path: "~/.provisioning/tokens/harbor"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Features**:
|
**Features**:
|
||||||
|
|
||||||
@ -598,7 +599,7 @@ done
|
|||||||
|
|
||||||
# Update workspace configurations to use OCI
|
# Update workspace configurations to use OCI
|
||||||
provisioning workspace migrate-to-oci workspace_prod
|
provisioning workspace migrate-to-oci workspace_prod
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Phase 3: Repository Split
|
### Phase 3: Repository Split
|
||||||
|
|
||||||
@ -680,7 +681,7 @@ provisioning workspace migrate-to-oci workspace_prod
|
|||||||
## Implementation Status
|
## Implementation Status
|
||||||
|
|
||||||
| Component | Status | Notes |
|
| Component | Status | Notes |
|
||||||
|-----------|--------|-------|
|
| ----------- | -------- | ------- |
|
||||||
| **Nickel Schemas** | ✅ Complete | OCI schemas in `dependencies.ncl` |
|
| **Nickel Schemas** | ✅ Complete | OCI schemas in `dependencies.ncl` |
|
||||||
| **OCI Client** | ✅ Complete | `oci/client.nu` with skopeo/crane/oras |
|
| **OCI Client** | ✅ Complete | `oci/client.nu` with skopeo/crane/oras |
|
||||||
| **OCI Commands** | ✅ Complete | `oci/commands.nu` CLI interface |
|
| **OCI Commands** | ✅ Complete | `oci/commands.nu` CLI interface |
|
||||||
|
|||||||
@ -6,7 +6,9 @@
|
|||||||
|
|
||||||
## Executive Summary
|
## Executive Summary
|
||||||
|
|
||||||
This document analyzes a **multi-repository strategy** as an alternative to the monorepo approach. After careful consideration of the provisioning system's architecture, a **hybrid approach with 4 core repositories** is recommended, avoiding submodules in favor of a cleaner package-based dependency model.
|
This document analyzes a **multi-repository strategy** as an alternative to the monorepo approach. After careful consideration of the provisioning
|
||||||
|
system's architecture, a **hybrid approach with 4 core repositories** is recommended, avoiding submodules in favor of a cleaner package-based
|
||||||
|
dependency model.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -103,7 +105,7 @@ provisioning-core/
|
|||||||
├── README.md
|
├── README.md
|
||||||
├── CHANGELOG.md
|
├── CHANGELOG.md
|
||||||
└── version.toml # Core version file
|
└── version.toml # Core version file
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Technology:** Nushell, Nickel
|
**Technology:** Nushell, Nickel
|
||||||
**Primary Language:** Nushell
|
**Primary Language:** Nushell
|
||||||
@ -123,7 +125,7 @@ provisioning-core/
|
|||||||
├── bin/provisioning
|
├── bin/provisioning
|
||||||
├── lib/provisioning/
|
├── lib/provisioning/
|
||||||
└── share/provisioning/
|
└── share/provisioning/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -161,7 +163,7 @@ provisioning-platform/
|
|||||||
├── LICENSE
|
├── LICENSE
|
||||||
├── README.md
|
├── README.md
|
||||||
└── CHANGELOG.md
|
└── CHANGELOG.md
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Technology:** Rust, WebAssembly
|
**Technology:** Rust, WebAssembly
|
||||||
**Primary Language:** Rust
|
**Primary Language:** Rust
|
||||||
@ -184,7 +186,7 @@ provisioning-platform/
|
|||||||
│ ├── provisioning-orchestrator
|
│ ├── provisioning-orchestrator
|
||||||
│ └── provisioning-control-center
|
│ └── provisioning-control-center
|
||||||
└── share/provisioning/platform/
|
└── share/provisioning/platform/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Integration with Core:**
|
**Integration with Core:**
|
||||||
|
|
||||||
@ -233,7 +235,7 @@ provisioning-extensions/
|
|||||||
├── docs/ # Extension development guide
|
├── docs/ # Extension development guide
|
||||||
├── LICENSE
|
├── LICENSE
|
||||||
└── README.md
|
└── README.md
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Technology:** Nushell, Nickel
|
**Technology:** Nushell, Nickel
|
||||||
**Primary Language:** Nushell
|
**Primary Language:** Nushell
|
||||||
@ -254,7 +256,7 @@ provisioning-extensions/
|
|||||||
# Install extension via core CLI
|
# Install extension via core CLI
|
||||||
provisioning extension install mongodb
|
provisioning extension install mongodb
|
||||||
provisioning extension install azure-provider
|
provisioning extension install azure-provider
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Extension Structure:**
|
**Extension Structure:**
|
||||||
Each extension is self-contained:
|
Each extension is self-contained:
|
||||||
@ -267,7 +269,7 @@ mongodb/
|
|||||||
├── schemas/ # Nickel schemas
|
├── schemas/ # Nickel schemas
|
||||||
├── tests/ # Tests
|
├── tests/ # Tests
|
||||||
└── README.md
|
└── README.md
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -296,7 +298,7 @@ provisioning-workspace/
|
|||||||
│ └── create-workspace.nu
|
│ └── create-workspace.nu
|
||||||
├── LICENSE
|
├── LICENSE
|
||||||
└── README.md
|
└── README.md
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Technology:** Configuration files, Nickel
|
**Technology:** Configuration files, Nickel
|
||||||
**Primary Language:** TOML, Nickel, YAML
|
**Primary Language:** TOML, Nickel, YAML
|
||||||
@ -321,7 +323,7 @@ provisioning workspace init my-project --template kubernetes
|
|||||||
gh repo create my-project --template provisioning-workspace
|
gh repo create my-project --template provisioning-workspace
|
||||||
cd my-project
|
cd my-project
|
||||||
provisioning workspace init
|
provisioning workspace init
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -360,7 +362,7 @@ provisioning-distribution/
|
|||||||
│ └── packaging-guide.md
|
│ └── packaging-guide.md
|
||||||
├── LICENSE
|
├── LICENSE
|
||||||
└── README.md
|
└── README.md
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Technology:** Nushell, Bash, CI/CD
|
**Technology:** Nushell, Bash, CI/CD
|
||||||
**Primary Language:** Nushell, YAML
|
**Primary Language:** Nushell, YAML
|
||||||
@ -406,7 +408,7 @@ provisioning-distribution/
|
|||||||
│ ↓ │
|
│ ↓ │
|
||||||
└───────────────────────────────────→┘
|
└───────────────────────────────────→┘
|
||||||
runtime integration
|
runtime integration
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Integration Mechanisms
|
### Integration Mechanisms
|
||||||
|
|
||||||
@ -425,7 +427,7 @@ def create-server [name: string] {
|
|||||||
def submit-workflow [workflow: record] {
|
def submit-workflow [workflow: record] {
|
||||||
http post http://localhost:9090/workflows/submit $workflow
|
http post http://localhost:9090/workflows/submit $workflow
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Version Compatibility:**
|
**Version Compatibility:**
|
||||||
|
|
||||||
@ -433,7 +435,7 @@ def submit-workflow [workflow: record] {
|
|||||||
# platform/Cargo.toml
|
# platform/Cargo.toml
|
||||||
[package.metadata.provisioning]
|
[package.metadata.provisioning]
|
||||||
core-version = "^3.0" # Compatible with core 3.x
|
core-version = "^3.0" # Compatible with core 3.x
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 2. **Core ↔ Extensions Integration**
|
#### 2. **Core ↔ Extensions Integration**
|
||||||
|
|
||||||
@ -457,7 +459,7 @@ provisioning extension install mongodb
|
|||||||
# → Downloads from registry
|
# → Downloads from registry
|
||||||
# → Validates compatibility
|
# → Validates compatibility
|
||||||
# → Installs to ~/.provisioning/extensions/mongodb
|
# → Installs to ~/.provisioning/extensions/mongodb
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 3. **Workspace Templates**
|
#### 3. **Workspace Templates**
|
||||||
|
|
||||||
@ -474,7 +476,7 @@ provisioning workspace create my-infra --template kubernetes
|
|||||||
# → Downloads template package
|
# → Downloads template package
|
||||||
# → Scaffolds workspace
|
# → Scaffolds workspace
|
||||||
# → Initializes configuration
|
# → Initializes configuration
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -489,7 +491,7 @@ provisioning-core: 3.2.1
|
|||||||
provisioning-platform: 2.5.3
|
provisioning-platform: 2.5.3
|
||||||
provisioning-extensions: (per-extension versioning)
|
provisioning-extensions: (per-extension versioning)
|
||||||
provisioning-workspace: 1.4.0
|
provisioning-workspace: 1.4.0
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Compatibility Matrix
|
### Compatibility Matrix
|
||||||
|
|
||||||
@ -528,7 +530,7 @@ lts-until = "2026-09-01"
|
|||||||
core = "3.1.5"
|
core = "3.1.5"
|
||||||
platform = "2.4.8"
|
platform = "2.4.8"
|
||||||
workspace = "1.3.0"
|
workspace = "1.3.0"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Release Coordination
|
### Release Coordination
|
||||||
|
|
||||||
@ -543,7 +545,7 @@ provisioning-workspace: 1.0.0
|
|||||||
# Minor/patch releases: Independent
|
# Minor/patch releases: Independent
|
||||||
provisioning-core: 3.1.0 (adds features, platform stays 2.0.x)
|
provisioning-core: 3.1.0 (adds features, platform stays 2.0.x)
|
||||||
provisioning-platform: 2.1.0 (improves orchestrator, core stays 3.1.x)
|
provisioning-platform: 2.1.0 (improves orchestrator, core stays 3.1.x)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -568,7 +570,7 @@ just build
|
|||||||
|
|
||||||
# Test installation locally
|
# Test installation locally
|
||||||
just install-dev
|
just install-dev
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Working Across Repositories
|
### Working Across Repositories
|
||||||
|
|
||||||
@ -609,7 +611,7 @@ cargo test
|
|||||||
# Merge core PR first, cut release 3.3.0
|
# Merge core PR first, cut release 3.3.0
|
||||||
# Update platform dependency to core 3.3.0
|
# Update platform dependency to core 3.3.0
|
||||||
# Merge platform PR, cut release 2.6.0
|
# Merge platform PR, cut release 2.6.0
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Testing Cross-Repo Integration
|
### Testing Cross-Repo Integration
|
||||||
|
|
||||||
@ -624,7 +626,7 @@ just test-integration \
|
|||||||
|
|
||||||
# Test bundle
|
# Test bundle
|
||||||
just test-bundle stable-3.3
|
just test-bundle stable-3.3
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -648,7 +650,7 @@ git tag v2.5.3
|
|||||||
git push --tags
|
git push --tags
|
||||||
# → GitHub Actions builds binaries
|
# → GitHub Actions builds binaries
|
||||||
# → Publishes to package registry
|
# → Publishes to package registry
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Bundle Releases (Coordinated)
|
### Bundle Releases (Coordinated)
|
||||||
|
|
||||||
@ -671,7 +673,7 @@ just publish-bundle stable-3.2
|
|||||||
# → Creates meta-package with all components
|
# → Creates meta-package with all components
|
||||||
# → Publishes bundle to registry
|
# → Publishes bundle to registry
|
||||||
# → Updates documentation
|
# → Updates documentation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### User Installation Options
|
### User Installation Options
|
||||||
|
|
||||||
@ -685,7 +687,7 @@ curl -fsSL https://get.provisioning.io | sh
|
|||||||
# - provisioning-core 3.2.1
|
# - provisioning-core 3.2.1
|
||||||
# - provisioning-platform 2.5.3
|
# - provisioning-platform 2.5.3
|
||||||
# - provisioning-workspace 1.4.0
|
# - provisioning-workspace 1.4.0
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Option 2: Individual Component Installation
|
#### Option 2: Individual Component Installation
|
||||||
|
|
||||||
@ -698,7 +700,7 @@ provisioning install platform
|
|||||||
|
|
||||||
# Add extensions
|
# Add extensions
|
||||||
provisioning extension install mongodb
|
provisioning extension install mongodb
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Option 3: Custom Combination
|
#### Option 3: Custom Combination
|
||||||
|
|
||||||
@ -706,7 +708,7 @@ provisioning extension install mongodb
|
|||||||
# Install specific versions
|
# Install specific versions
|
||||||
provisioning install core@3.1.0
|
provisioning install core@3.1.0
|
||||||
provisioning install platform@2.4.0
|
provisioning install platform@2.4.0
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -715,7 +717,7 @@ provisioning install platform@2.4.0
|
|||||||
### Core Team Ownership
|
### Core Team Ownership
|
||||||
|
|
||||||
| Repository | Primary Owner | Contribution Model |
|
| Repository | Primary Owner | Contribution Model |
|
||||||
|------------|---------------|-------------------|
|
| ------------ | --------------- | ------------------- |
|
||||||
| `provisioning-core` | Core Team | Strict review, stable API |
|
| `provisioning-core` | Core Team | Strict review, stable API |
|
||||||
| `provisioning-platform` | Platform Team | Fast iteration, performance focus |
|
| `provisioning-platform` | Platform Team | Fast iteration, performance focus |
|
||||||
| `provisioning-extensions` | Community + Core | Open contributions, moderated |
|
| `provisioning-extensions` | Community + Core | Open contributions, moderated |
|
||||||
@ -786,7 +788,7 @@ jobs:
|
|||||||
run: just publish
|
run: just publish
|
||||||
env:
|
env:
|
||||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Platform CI (`provisioning-platform/.github/workflows/ci.yml`):**
|
**Platform CI (`provisioning-platform/.github/workflows/ci.yml`):**
|
||||||
|
|
||||||
@ -821,7 +823,7 @@ jobs:
|
|||||||
run: cargo build --release --target aarch64-unknown-linux-gnu
|
run: cargo build --release --target aarch64-unknown-linux-gnu
|
||||||
- name: Publish binaries
|
- name: Publish binaries
|
||||||
run: just publish-binaries
|
run: just publish-binaries
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Integration Testing (Distribution Repo)
|
### Integration Testing (Distribution Repo)
|
||||||
|
|
||||||
@ -852,7 +854,7 @@ jobs:
|
|||||||
- name: Test upgrade path
|
- name: Test upgrade path
|
||||||
run: |
|
run: |
|
||||||
nu tests/integration/test-upgrade.nu 3.1.0 3.2.1
|
nu tests/integration/test-upgrade.nu 3.1.0 3.2.1
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -867,7 +869,7 @@ provisioning/ (One repo, ~500 MB)
|
|||||||
├── extensions/ (Community)
|
├── extensions/ (Community)
|
||||||
├── workspace/ (Templates)
|
├── workspace/ (Templates)
|
||||||
└── distribution/ (Build)
|
└── distribution/ (Build)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Multi-Repo Structure
|
### Multi-Repo Structure
|
||||||
|
|
||||||
@ -900,14 +902,14 @@ provisioning-distribution/ (Repo 5, ~30 MB)
|
|||||||
├── installers/
|
├── installers/
|
||||||
├── packaging/
|
├── packaging/
|
||||||
└── registry/
|
└── registry/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Decision Matrix
|
## Decision Matrix
|
||||||
|
|
||||||
| Criterion | Monorepo | Multi-Repo |
|
| Criterion | Monorepo | Multi-Repo |
|
||||||
|-----------|----------|------------|
|
| ----------- | ---------- | ------------ |
|
||||||
| **Development Complexity** | Simple | Moderate |
|
| **Development Complexity** | Simple | Moderate |
|
||||||
| **Clone Size** | Large (~500 MB) | Small (50-150 MB each) |
|
| **Clone Size** | Large (~500 MB) | Small (50-150 MB each) |
|
||||||
| **Cross-Component Changes** | Easy (atomic) | Moderate (coordinated) |
|
| **Cross-Component Changes** | Easy (atomic) | Moderate (coordinated) |
|
||||||
@ -1007,7 +1009,8 @@ The multi-repo approach provides:
|
|||||||
|
|
||||||
**Use:** Package-based dependencies with version compatibility matrix
|
**Use:** Package-based dependencies with version compatibility matrix
|
||||||
|
|
||||||
This architecture scales better for your project's growth, supports a community extension ecosystem, and provides professional-grade separation of concerns while maintaining integration through a well-designed package system.
|
This architecture scales better for your project's growth, supports a community extension ecosystem, and provides professional-grade separation of
|
||||||
|
concerns while maintaining integration through a well-designed package system.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -16,7 +16,7 @@ Need to define infrastructure/schemas?
|
|||||||
├─ Need type-safe UIs? → Nickel + TypeDialog ✅
|
├─ Need type-safe UIs? → Nickel + TypeDialog ✅
|
||||||
├─ Application settings? → Use TOML (not KCL/Nickel)
|
├─ Application settings? → Use TOML (not KCL/Nickel)
|
||||||
└─ K8s/CI-CD config? → Use YAML (not KCL/Nickel)
|
└─ K8s/CI-CD config? → Use YAML (not KCL/Nickel)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -43,7 +43,7 @@ server_defaults: ServerDefaults = {
|
|||||||
memory_gb = 8,
|
memory_gb = 8,
|
||||||
os = "ubuntu",
|
os = "ubuntu",
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Note**: KCL is deprecated. Use Nickel for new projects.
|
**Note**: KCL is deprecated. Use Nickel for new projects.
|
||||||
|
|
||||||
@ -60,7 +60,7 @@ server_defaults: ServerDefaults = {
|
|||||||
os | String,
|
os | String,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**server_defaults.ncl**:
|
**server_defaults.ncl**:
|
||||||
|
|
||||||
@ -73,7 +73,7 @@ server_defaults: ServerDefaults = {
|
|||||||
os = "ubuntu",
|
os = "ubuntu",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**server.ncl**:
|
**server.ncl**:
|
||||||
|
|
||||||
@ -89,7 +89,7 @@ let defaults = import "./server_defaults.ncl" in
|
|||||||
|
|
||||||
DefaultServer = defaults.server,
|
DefaultServer = defaults.server,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Usage**:
|
**Usage**:
|
||||||
|
|
||||||
@ -104,7 +104,7 @@ my_custom = server.defaults.server & {
|
|||||||
cpu_cores = 16,
|
cpu_cores = 16,
|
||||||
custom_monitoring_level = "verbose" # ✅ Works!
|
custom_monitoring_level = "verbose" # ✅ Works!
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Key Differences**:
|
**Key Differences**:
|
||||||
|
|
||||||
@ -139,7 +139,7 @@ provision_upcloud: ProvisionUpcloud = {
|
|||||||
api_password = ""
|
api_password = ""
|
||||||
servers = []
|
servers = []
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Nickel (from `provisioning/extensions/providers/upcloud/nickel/`)
|
#### Nickel (from `provisioning/extensions/providers/upcloud/nickel/`)
|
||||||
|
|
||||||
@ -166,7 +166,7 @@ provision_upcloud: ProvisionUpcloud = {
|
|||||||
servers | Array,
|
servers | Array,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**upcloud_defaults.ncl**:
|
**upcloud_defaults.ncl**:
|
||||||
|
|
||||||
@ -191,7 +191,7 @@ provision_upcloud: ProvisionUpcloud = {
|
|||||||
servers = [],
|
servers = [],
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**upcloud_main.ncl** (from actual codebase):
|
**upcloud_main.ncl** (from actual codebase):
|
||||||
|
|
||||||
@ -215,7 +215,7 @@ let defaults = import "./upcloud_defaults.ncl" in
|
|||||||
DefaultServerUpcloud = defaults.server_upcloud,
|
DefaultServerUpcloud = defaults.server_upcloud,
|
||||||
DefaultProvisionUpcloud = defaults.provision_upcloud,
|
DefaultProvisionUpcloud = defaults.provision_upcloud,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Usage Comparison**:
|
**Usage Comparison**:
|
||||||
|
|
||||||
@ -244,7 +244,7 @@ production_stack = upcloud.make_provision_upcloud {
|
|||||||
monitoring_enabled = true, # ✅ Custom field allowed!
|
monitoring_enabled = true, # ✅ Custom field allowed!
|
||||||
backup_schedule = "24h", # ✅ Custom field allowed!
|
backup_schedule = "24h", # ✅ Custom field allowed!
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -253,7 +253,7 @@ production_stack = upcloud.make_provision_upcloud {
|
|||||||
### Evaluation Speed
|
### Evaluation Speed
|
||||||
|
|
||||||
| File Type | KCL | Nickel | Improvement |
|
| File Type | KCL | Nickel | Improvement |
|
||||||
|-----------|-----|--------|------------|
|
| ----------- | ----- | -------- | ------------ |
|
||||||
| Simple schema (100 lines) | 45 ms | 18 ms | 60% faster |
|
| Simple schema (100 lines) | 45 ms | 18 ms | 60% faster |
|
||||||
| Complex config (500 lines) | 180 ms | 72 ms | 60% faster |
|
| Complex config (500 lines) | 180 ms | 72 ms | 60% faster |
|
||||||
| Large nested (2000 lines) | 420 ms | 160 ms | 62% faster |
|
| Large nested (2000 lines) | 420 ms | 160 ms | 62% faster |
|
||||||
@ -269,7 +269,7 @@ production_stack = upcloud.make_provision_upcloud {
|
|||||||
### Memory Usage
|
### Memory Usage
|
||||||
|
|
||||||
| Configuration | KCL | Nickel | Improvement |
|
| Configuration | KCL | Nickel | Improvement |
|
||||||
|---------------|-----|--------|------------|
|
| --------------- | ----- | -------- | ------------ |
|
||||||
| Platform schemas (422 files) | ~180 MB | ~85 MB | 53% less |
|
| Platform schemas (422 files) | ~180 MB | ~85 MB | 53% less |
|
||||||
| Full workspace (47 files) | ~45 MB | ~22 MB | 51% less |
|
| Full workspace (47 files) | ~45 MB | ~22 MB | 51% less |
|
||||||
| Single provider ext | ~8 MB | ~4 MB | 50% less |
|
| Single provider ext | ~8 MB | ~4 MB | 50% less |
|
||||||
@ -296,14 +296,14 @@ schema ServerConfig:
|
|||||||
web_server: ServerConfig = {
|
web_server: ServerConfig = {
|
||||||
name = "web-01",
|
name = "web-01",
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Nickel (Recommended)**:
|
**Nickel (Recommended)**:
|
||||||
|
|
||||||
```nickel
|
```nickel
|
||||||
let defaults = import "./server_defaults.ncl" in
|
let defaults = import "./server_defaults.ncl" in
|
||||||
web_server = defaults.make_server { name = "web-01" }
|
web_server = defaults.make_server { name = "web-01" }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Winner**: Nickel (simpler, cleaner)
|
**Winner**: Nickel (simpler, cleaner)
|
||||||
|
|
||||||
@ -339,7 +339,7 @@ taskserv_cilium: TaskServ = {
|
|||||||
{name = "kubernetes", wait_for_health = true}
|
{name = "kubernetes", wait_for_health = true}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Nickel** (from wuji/main.ncl):
|
**Nickel** (from wuji/main.ncl):
|
||||||
|
|
||||||
@ -355,7 +355,7 @@ let ts_containerd = import "./taskservs/containerd.ncl" in
|
|||||||
containerd = ts_containerd.containerd,
|
containerd = ts_containerd.containerd,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Winner**: Nickel (modular, scalable to 20 taskservs)
|
**Winner**: Nickel (modular, scalable to 20 taskservs)
|
||||||
|
|
||||||
@ -375,7 +375,7 @@ schema ServerConfig:
|
|||||||
monitoring_level: str = "basic"
|
monitoring_level: str = "basic"
|
||||||
|
|
||||||
# All existing configs need updating...
|
# All existing configs need updating...
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Nickel**:
|
**Nickel**:
|
||||||
|
|
||||||
@ -390,7 +390,7 @@ my_server = server.defaults.server & {
|
|||||||
custom_tags = ["production", "critical"],
|
custom_tags = ["production", "critical"],
|
||||||
grafana_dashboard = "web-servers",
|
grafana_dashboard = "web-servers",
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Winner**: Nickel (no schema modifications needed)
|
**Winner**: Nickel (no schema modifications needed)
|
||||||
|
|
||||||
@ -415,7 +415,7 @@ server: Server = {
|
|||||||
cpu = 4,
|
cpu = 4,
|
||||||
memory = 8,
|
memory = 8,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Problem**: Inheritance creates rigid hierarchies, breaking changes propagate
|
**Problem**: Inheritance creates rigid hierarchies, breaking changes propagate
|
||||||
|
|
||||||
@ -439,7 +439,7 @@ server = make_server {
|
|||||||
cpu = 4,
|
cpu = 4,
|
||||||
memory = 8,
|
memory = 8,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Advantage**: Flexible composition via record merging, no inheritance rigidity
|
**Advantage**: Flexible composition via record merging, no inheritance rigidity
|
||||||
|
|
||||||
@ -456,7 +456,7 @@ schema Config:
|
|||||||
check:
|
check:
|
||||||
timeout > 0, "Timeout must be positive"
|
timeout > 0, "Timeout must be positive"
|
||||||
timeout < 300, "Timeout must be < 5 min"
|
timeout < 300, "Timeout must be < 5 min"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: Validation at schema definition
|
**Pros**: Validation at schema definition
|
||||||
**Cons**: Overhead during compilation, rigid
|
**Cons**: Overhead during compilation, rigid
|
||||||
@ -482,7 +482,7 @@ let validate_config = fun config =>
|
|||||||
|
|
||||||
# Apply only when needed
|
# Apply only when needed
|
||||||
my_config = validate_config { timeout = 10 }
|
my_config = validate_config { timeout = 10 }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: Lazy evaluation, optional, fine-grained control
|
**Pros**: Lazy evaluation, optional, fine-grained control
|
||||||
**Cons**: Must invoke validation explicitly
|
**Cons**: Must invoke validation explicitly
|
||||||
@ -507,7 +507,7 @@ scheduler_config: Scheduler = {
|
|||||||
strategy = "priority",
|
strategy = "priority",
|
||||||
workers = 8,
|
workers = 8,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**After (Nickel - Current)**:
|
**After (Nickel - Current)**:
|
||||||
|
|
||||||
@ -520,7 +520,7 @@ scheduler_config: Scheduler = {
|
|||||||
workers | Number,
|
workers | Number,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
`scheduler_defaults.ncl`:
|
`scheduler_defaults.ncl`:
|
||||||
|
|
||||||
@ -531,7 +531,7 @@ scheduler_config: Scheduler = {
|
|||||||
workers = 4,
|
workers = 4,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
`scheduler.ncl`:
|
`scheduler.ncl`:
|
||||||
|
|
||||||
@ -549,7 +549,7 @@ let defaults = import "./scheduler_defaults.ncl" in
|
|||||||
workers = 8,
|
workers = 8,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -564,7 +564,7 @@ schema Mode:
|
|||||||
check:
|
check:
|
||||||
deployment_type in ["solo", "multiuser", "cicd", "enterprise"],
|
deployment_type in ["solo", "multiuser", "cicd", "enterprise"],
|
||||||
"Invalid deployment type"
|
"Invalid deployment type"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**After (Nickel - Current)**:
|
**After (Nickel - Current)**:
|
||||||
|
|
||||||
@ -582,7 +582,7 @@ schema Mode:
|
|||||||
deployment_type = 'solo,
|
deployment_type = 'solo,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Benefits**: Type-safe, no string validation needed
|
**Benefits**: Type-safe, no string validation needed
|
||||||
|
|
||||||
@ -605,7 +605,7 @@ web_server: Server = {
|
|||||||
cpu = 8,
|
cpu = 8,
|
||||||
memory = 16,
|
memory = 16,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**After (Nickel - Current)**:
|
**After (Nickel - Current)**:
|
||||||
|
|
||||||
@ -629,7 +629,7 @@ let make_server = fun config =>
|
|||||||
defaults.server_defaults & config & {
|
defaults.server_defaults & config & {
|
||||||
name = config.name,
|
name = config.name,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Advantage**: Explicit, flexible, composable
|
**Advantage**: Explicit, flexible, composable
|
||||||
|
|
||||||
@ -654,14 +654,14 @@ nickel export wuji/main.ncl --format json
|
|||||||
# Changes to central provisioning reflected immediately
|
# Changes to central provisioning reflected immediately
|
||||||
vim ../../provisioning/schemas/lib/main.ncl
|
vim ../../provisioning/schemas/lib/main.ncl
|
||||||
nickel export wuji/main.ncl # Uses updated schemas
|
nickel export wuji/main.ncl # Uses updated schemas
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Imports** (relative, central):
|
**Imports** (relative, central):
|
||||||
|
|
||||||
```nickel
|
```nickel
|
||||||
import "../../provisioning/schemas/main.ncl"
|
import "../../provisioning/schemas/main.ncl"
|
||||||
import "../../provisioning/extensions/taskservs/kubernetes/nickel/main.ncl"
|
import "../../provisioning/extensions/taskservs/kubernetes/nickel/main.ncl"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -692,7 +692,7 @@ provisioning deploy \
|
|||||||
provisioning deploy \
|
provisioning deploy \
|
||||||
--frozen "2025-12-10-prod-v0" \
|
--frozen "2025-12-10-prod-v0" \
|
||||||
--infra wuji
|
--infra wuji
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Frozen Imports** (rewritten to local):
|
**Frozen Imports** (rewritten to local):
|
||||||
|
|
||||||
@ -702,7 +702,7 @@ import "../../provisioning/schemas/main.ncl"
|
|||||||
|
|
||||||
# Rewritten in frozen snapshot
|
# Rewritten in frozen snapshot
|
||||||
import "./provisioning/schemas/main.ncl"
|
import "./provisioning/schemas/main.ncl"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Benefits**:
|
**Benefits**:
|
||||||
|
|
||||||
@ -725,7 +725,7 @@ import "./provisioning/schemas/main.ncl"
|
|||||||
let A = { x = 1 }
|
let A = { x = 1 }
|
||||||
let B = { y = 2 }
|
let B = { y = 2 }
|
||||||
{ A = A, B = B }
|
{ A = A, B = B }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Error: `unexpected token`
|
Error: `unexpected token`
|
||||||
|
|
||||||
@ -736,7 +736,7 @@ Error: `unexpected token`
|
|||||||
let A = { x = 1 } in
|
let A = { x = 1 } in
|
||||||
let B = { y = 2 } in
|
let B = { y = 2 } in
|
||||||
{ A = A, B = B }
|
{ A = A, B = B }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -749,7 +749,7 @@ let B = { y = 2 } in
|
|||||||
let StorageVol = {
|
let StorageVol = {
|
||||||
mount_path : String | null = null,
|
mount_path : String | null = null,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Error: `this can't be used as a contract`
|
Error: `this can't be used as a contract`
|
||||||
|
|
||||||
@ -762,7 +762,7 @@ Error: `this can't be used as a contract`
|
|||||||
let StorageVol = {
|
let StorageVol = {
|
||||||
mount_path = null,
|
mount_path = null,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -776,7 +776,7 @@ let StorageVol = {
|
|||||||
get_value = fun x => x + 1,
|
get_value = fun x => x + 1,
|
||||||
result = get_value 5,
|
result = get_value 5,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Error: Functions can't be serialized
|
Error: Functions can't be serialized
|
||||||
|
|
||||||
@ -788,7 +788,7 @@ Error: Functions can't be serialized
|
|||||||
get_value | not_exported = fun x => x + 1,
|
get_value | not_exported = fun x => x + 1,
|
||||||
result = get_value 5,
|
result = get_value 5,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -799,7 +799,7 @@ Error: Functions can't be serialized
|
|||||||
```nickel
|
```nickel
|
||||||
let defaults = import "./defaults.ncl" in
|
let defaults = import "./defaults.ncl" in
|
||||||
defaults.scheduler_config # But file has "scheduler"
|
defaults.scheduler_config # But file has "scheduler"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
Error: `field not found`
|
Error: `field not found`
|
||||||
|
|
||||||
@ -808,7 +808,7 @@ Error: `field not found`
|
|||||||
```nickel
|
```nickel
|
||||||
let defaults = import "./defaults.ncl" in
|
let defaults = import "./defaults.ncl" in
|
||||||
defaults.scheduler # Correct name from defaults.ncl
|
defaults.scheduler # Correct name from defaults.ncl
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -830,7 +830,7 @@ defaults.scheduler # Correct name from defaults.ncl
|
|||||||
validate_config | not_exported = fun x => x,
|
validate_config | not_exported = fun x => x,
|
||||||
data = { foo = "bar" },
|
data = { foo = "bar" },
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -840,13 +840,13 @@ defaults.scheduler # Correct name from defaults.ncl
|
|||||||
|
|
||||||
1. **Follow Three-File Pattern**
|
1. **Follow Three-File Pattern**
|
||||||
|
|
||||||
```
|
```nickel
|
||||||
|
|
||||||
module_contracts.ncl # Types only
|
module_contracts.ncl # Types only
|
||||||
module_defaults.ncl # Values only
|
module_defaults.ncl # Values only
|
||||||
module.ncl # Instances + interface
|
module.ncl # Instances + interface
|
||||||
|
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
2. **Use Hybrid Interface** (4 levels)
|
2. **Use Hybrid Interface** (4 levels)
|
||||||
- Level 1: Direct defaults (inspection)
|
- Level 1: Direct defaults (inspection)
|
||||||
@ -940,7 +940,7 @@ typedialog form --schema server.ncl --output json
|
|||||||
|
|
||||||
# 4. Output back to Nickel
|
# 4. Output back to Nickel
|
||||||
typedialog form --input form.toml --output nickel
|
typedialog form --input form.toml --output nickel
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Benefits
|
### Benefits
|
||||||
|
|
||||||
@ -967,7 +967,7 @@ provisioning/schemas/config/workspace_config/main.ncl
|
|||||||
- custom_settings (advanced, optional)
|
- custom_settings (advanced, optional)
|
||||||
|
|
||||||
# Output: workspace_config.ncl (valid Nickel!)
|
# Output: workspace_config.ncl (valid Nickel!)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -1050,7 +1050,7 @@ provisioning/schemas/config/workspace_config/main.ncl
|
|||||||
modes = import "./deployment/modes/main.ncl",
|
modes = import "./deployment/modes/main.ncl",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Usage**:
|
**Usage**:
|
||||||
|
|
||||||
@ -1061,7 +1061,7 @@ provisioning.lib.Storage
|
|||||||
provisioning.config.settings
|
provisioning.config.settings
|
||||||
provisioning.infrastructure.compute.server
|
provisioning.infrastructure.compute.server
|
||||||
provisioning.operations.workflows
|
provisioning.operations.workflows
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -1101,7 +1101,7 @@ let defaults_lib = import "./defaults.ncl" in
|
|||||||
DefaultServerDefaults_upcloud = defaults_lib.server_defaults_upcloud,
|
DefaultServerDefaults_upcloud = defaults_lib.server_defaults_upcloud,
|
||||||
DefaultServerUpcloud = defaults_lib.server_upcloud,
|
DefaultServerUpcloud = defaults_lib.server_upcloud,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -1159,14 +1159,14 @@ let ts_youki = import "./taskservs/youki.ncl" in
|
|||||||
youki = ts_youki.youki,
|
youki = ts_youki.youki,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Summary Table
|
## Summary Table
|
||||||
|
|
||||||
| Aspect | KCL | Nickel | Recommendation |
|
| Aspect | KCL | Nickel | Recommendation |
|
||||||
|--------|-----|--------|---|
|
| -------- | ----- | -------- | --- |
|
||||||
| **Learning Curve** | 10 hours | 3 hours | Nickel |
|
| **Learning Curve** | 10 hours | 3 hours | Nickel |
|
||||||
| **Performance** | Baseline | 60% faster | Nickel |
|
| **Performance** | Baseline | 60% faster | Nickel |
|
||||||
| **Flexibility** | Limited | Excellent | Nickel |
|
| **Flexibility** | Limited | Excellent | Nickel |
|
||||||
|
|||||||
@ -6,7 +6,8 @@
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Complete authentication and authorization flow integration for the Provisioning Orchestrator, connecting all security components (JWT validation, MFA verification, Cedar authorization, rate limiting, and audit logging) into a cohesive security middleware chain.
|
Complete authentication and authorization flow integration for the Provisioning Orchestrator, connecting all security components (JWT validation, MFA
|
||||||
|
verification, Cedar authorization, rate limiting, and audit logging) into a cohesive security middleware chain.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
@ -69,7 +70,7 @@ The middleware chain is applied in this specific order to ensure proper security
|
|||||||
│ - Access security context │
|
│ - Access security context │
|
||||||
│ - Execute business logic │
|
│ - Execute business logic │
|
||||||
└────────────────────────────────┘
|
└────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Implementation Details
|
## Implementation Details
|
||||||
|
|
||||||
@ -107,7 +108,7 @@ impl SecurityContext {
|
|||||||
pub fn has_any_permission(&self, permissions: &[&str]) -> bool { ... }
|
pub fn has_any_permission(&self, permissions: &[&str]) -> bool { ... }
|
||||||
pub fn has_all_permissions(&self, permissions: &[&str]) -> bool { ... }
|
pub fn has_all_permissions(&self, permissions: &[&str]) -> bool { ... }
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Enhanced Authentication Middleware (`middleware/auth.rs`)
|
### 2. Enhanced Authentication Middleware (`middleware/auth.rs`)
|
||||||
|
|
||||||
@ -170,7 +171,7 @@ fn requires_mfa(method: &str, path: &str) -> bool {
|
|||||||
if path.contains("/deploy") { return true; }
|
if path.contains("/deploy") { return true; }
|
||||||
// ...
|
// ...
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Enhanced Authorization Middleware (`middleware/authz.rs`)
|
### 4. Enhanced Authorization Middleware (`middleware/authz.rs`)
|
||||||
|
|
||||||
@ -194,7 +195,7 @@ fn requires_mfa(method: &str, path: &str) -> bool {
|
|||||||
/api/v1/taskserv/kubernetes → Resource::TaskService("kubernetes")
|
/api/v1/taskserv/kubernetes → Resource::TaskService("kubernetes")
|
||||||
/api/v1/cluster/prod → Resource::Cluster("prod")
|
/api/v1/cluster/prod → Resource::Cluster("prod")
|
||||||
/api/v1/config/settings → Resource::Config("settings")
|
/api/v1/config/settings → Resource::Config("settings")
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Action Mapping**:
|
**Action Mapping**:
|
||||||
|
|
||||||
@ -203,7 +204,7 @@ GET → Action::Read
|
|||||||
POST → Action::Create
|
POST → Action::Create
|
||||||
PUT → Action::Update
|
PUT → Action::Update
|
||||||
DELETE → Action::Delete
|
DELETE → Action::Delete
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 5. Rate Limiting Middleware (`middleware/rate_limit.rs`)
|
### 5. Rate Limiting Middleware (`middleware/rate_limit.rs`)
|
||||||
|
|
||||||
@ -231,7 +232,7 @@ pub struct RateLimitConfig {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Default: 100 requests per minute
|
// Default: 100 requests per minute
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Statistics**:
|
**Statistics**:
|
||||||
|
|
||||||
@ -242,7 +243,7 @@ pub struct RateLimitStats {
|
|||||||
pub limited_ips: usize, // IPs that hit the limit
|
pub limited_ips: usize, // IPs that hit the limit
|
||||||
pub config: RateLimitConfig,
|
pub config: RateLimitConfig,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 6. Security Integration Module (`security_integration.rs`)
|
### 6. Security Integration Module (`security_integration.rs`)
|
||||||
|
|
||||||
@ -285,7 +286,7 @@ let app = Router::new()
|
|||||||
.route("/api/v1/servers/:id", delete(delete_server));
|
.route("/api/v1/servers/:id", delete(delete_server));
|
||||||
|
|
||||||
let secured_app = apply_security_middleware(app, &security);
|
let secured_app = apply_security_middleware(app, &security);
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Integration with AppState
|
## Integration with AppState
|
||||||
|
|
||||||
@ -312,7 +313,7 @@ pub struct AppState {
|
|||||||
// NEW: Security components
|
// NEW: Security components
|
||||||
pub security: SecurityComponents,
|
pub security: SecurityComponents,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Initialization in main.rs
|
### Initialization in main.rs
|
||||||
|
|
||||||
@ -373,14 +374,14 @@ async fn main() -> Result<()> {
|
|||||||
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Protected Endpoints
|
## Protected Endpoints
|
||||||
|
|
||||||
### Endpoint Categories
|
### Endpoint Categories
|
||||||
|
|
||||||
| Category | Example Endpoints | Auth Required | MFA Required | Cedar Policy |
|
| Category | Example Endpoints | Auth Required | MFA Required | Cedar Policy |
|
||||||
|----------|-------------------|---------------|--------------|--------------|
|
| ---------- | ------------------- | --------------- | -------------- | -------------- |
|
||||||
| **Health** | `/health` | ❌ | ❌ | ❌ |
|
| **Health** | `/health` | ❌ | ❌ | ❌ |
|
||||||
| **Read-Only** | `GET /api/v1/servers` | ✅ | ❌ | ✅ |
|
| **Read-Only** | `GET /api/v1/servers` | ✅ | ❌ | ✅ |
|
||||||
| **Server Mgmt** | `POST /api/v1/servers` | ✅ | ❌ | ✅ |
|
| **Server Mgmt** | `POST /api/v1/servers` | ✅ | ❌ | ✅ |
|
||||||
@ -478,7 +479,7 @@ async fn main() -> Result<()> {
|
|||||||
|
|
||||||
9. CLIENT RESPONSE
|
9. CLIENT RESPONSE
|
||||||
└─ 200 OK: Server deleted successfully
|
└─ 200 OK: Server deleted successfully
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@ -506,7 +507,7 @@ RATE_LIMIT_EXEMPT_IPS=10.0.0.1,10.0.0.2
|
|||||||
# Audit Logging
|
# Audit Logging
|
||||||
AUDIT_ENABLED=true
|
AUDIT_ENABLED=true
|
||||||
AUDIT_RETENTION_DAYS=365
|
AUDIT_RETENTION_DAYS=365
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Development Mode
|
### Development Mode
|
||||||
|
|
||||||
@ -519,7 +520,7 @@ let security = if env::var("DEVELOPMENT_MODE").unwrap_or("false".to_string()) ==
|
|||||||
} else {
|
} else {
|
||||||
SecurityComponents::initialize(security_config, audit_logger.clone()).await?
|
SecurityComponents::initialize(security_config, audit_logger.clone()).await?
|
||||||
};
|
};
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
@ -546,12 +547,12 @@ Location: `provisioning/platform/orchestrator/tests/security_integration_tests.r
|
|||||||
```bash
|
```bash
|
||||||
cd provisioning/platform/orchestrator
|
cd provisioning/platform/orchestrator
|
||||||
cargo test security_integration_tests
|
cargo test security_integration_tests
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## File Summary
|
## File Summary
|
||||||
|
|
||||||
| File | Purpose | Lines | Tests |
|
| File | Purpose | Lines | Tests |
|
||||||
|------|---------|-------|-------|
|
| ------ | --------- | ------- | ------- |
|
||||||
| `middleware/security_context.rs` | Security context builder | 275 | 8 |
|
| `middleware/security_context.rs` | Security context builder | 275 | 8 |
|
||||||
| `middleware/auth.rs` | JWT authentication | 245 | 5 |
|
| `middleware/auth.rs` | JWT authentication | 245 | 5 |
|
||||||
| `middleware/mfa.rs` | MFA verification | 290 | 15 |
|
| `middleware/mfa.rs` | MFA verification | 290 | 15 |
|
||||||
@ -610,7 +611,7 @@ cargo test security_integration_tests
|
|||||||
## Version History
|
## Version History
|
||||||
|
|
||||||
| Version | Date | Changes |
|
| Version | Date | Changes |
|
||||||
|---------|------|---------|
|
| --------- | ------ | --------- |
|
||||||
| 1.0.0 | 2025-10-08 | Initial implementation |
|
| 1.0.0 | 2025-10-08 | Initial implementation |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@ -8,9 +8,11 @@ That code example was misleading. Here's the real architecture:
|
|||||||
|
|
||||||
How It Actually Works
|
How It Actually Works
|
||||||
|
|
||||||
┌──────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────
|
||||||
|
─┐
|
||||||
│ User runs: provisioning server create --orchestrated
|
│ User runs: provisioning server create --orchestrated
|
||||||
└───────────────────┬──────────────────────────────┘
|
└───────────────────┬─────────────────────────────
|
||||||
|
─┘
|
||||||
↓
|
↓
|
||||||
┌───────────────────────┐
|
┌───────────────────────┐
|
||||||
│ Nushell CLI │
|
│ Nushell CLI │
|
||||||
@ -59,7 +61,7 @@ async fn create_server_workflow(request) {
|
|||||||
task_queue.enqueue(task).await; // Queue for execution
|
task_queue.enqueue(task).await; // Queue for execution
|
||||||
return workflow_id; // Return immediately
|
return workflow_id; // Return immediately
|
||||||
}
|
}
|
||||||
```
|
```text
|
||||||
|
|
||||||
2. Orchestrator executes via Nushell subprocess:
|
2. Orchestrator executes via Nushell subprocess:
|
||||||
|
|
||||||
@ -74,7 +76,7 @@ async fn execute_task(task: Task) {
|
|||||||
|
|
||||||
// Orchestrator manages: retry, checkpointing, monitoring
|
// Orchestrator manages: retry, checkpointing, monitoring
|
||||||
}
|
}
|
||||||
```
|
```text
|
||||||
|
|
||||||
3. Nushell executes the actual work:
|
3. Nushell executes the actual work:
|
||||||
|
|
||||||
@ -6,7 +6,9 @@
|
|||||||
|
|
||||||
## Executive Summary
|
## Executive Summary
|
||||||
|
|
||||||
This document clarifies **how the Rust orchestrator integrates with Nushell core** in both monorepo and multi-repo architectures. The orchestrator is a **critical performance layer** that coordinates Nushell business logic execution, solving deep call stack limitations while preserving all existing functionality.
|
This document clarifies **how the Rust orchestrator integrates with Nushell core** in both monorepo and multi-repo architectures. The orchestrator is
|
||||||
|
a **critical performance layer** that coordinates Nushell business logic execution, solving deep call stack limitations while preserving all existing
|
||||||
|
functionality.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -21,7 +23,7 @@ Deep call stack in Nushell (template.nu:71)
|
|||||||
→ "Type not supported" errors
|
→ "Type not supported" errors
|
||||||
→ Cannot handle complex nested workflows
|
→ Cannot handle complex nested workflows
|
||||||
→ Performance bottlenecks with recursive calls
|
→ Performance bottlenecks with recursive calls
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Solution:** Rust orchestrator provides:
|
**Solution:** Rust orchestrator provides:
|
||||||
|
|
||||||
@ -72,7 +74,7 @@ Deep call stack in Nushell (template.nu:71)
|
|||||||
│ • taskservs.nu │
|
│ • taskservs.nu │
|
||||||
│ • clusters.nu │
|
│ • clusters.nu │
|
||||||
└────────────────┘
|
└────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Three Execution Modes
|
### Three Execution Modes
|
||||||
|
|
||||||
@ -86,7 +88,7 @@ provisioning help
|
|||||||
|
|
||||||
# Direct Nushell execution
|
# Direct Nushell execution
|
||||||
provisioning (CLI) → Nushell scripts → Result
|
provisioning (CLI) → Nushell scripts → Result
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Mode 2: Orchestrated Mode (Complex Operations)
|
#### Mode 2: Orchestrated Mode (Complex Operations)
|
||||||
|
|
||||||
@ -98,7 +100,7 @@ provisioning server create --orchestrated
|
|||||||
provisioning CLI → Orchestrator API → Task Queue → Nushell executor
|
provisioning CLI → Orchestrator API → Task Queue → Nushell executor
|
||||||
↓
|
↓
|
||||||
Result back to user
|
Result back to user
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Mode 3: Workflow Mode (Batch Operations)
|
#### Mode 3: Workflow Mode (Batch Operations)
|
||||||
|
|
||||||
@ -114,7 +116,7 @@ provisioning CLI → Orchestrator Workflow Engine → Dependency Graph
|
|||||||
Nushell scripts for each task
|
Nushell scripts for each task
|
||||||
↓
|
↓
|
||||||
Checkpoint state
|
Checkpoint state
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -147,7 +149,7 @@ export def server_create_workflow [
|
|||||||
do-server-create $infra_name
|
do-server-create $infra_name
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Rust Orchestrator (`platform/orchestrator/src/api/workflows.rs`):**
|
**Rust Orchestrator (`platform/orchestrator/src/api/workflows.rs`):**
|
||||||
|
|
||||||
@ -177,7 +179,7 @@ async fn create_server_workflow(
|
|||||||
status: "queued",
|
status: "queued",
|
||||||
}))
|
}))
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Flow:**
|
**Flow:**
|
||||||
|
|
||||||
@ -193,7 +195,7 @@ Orchestrator queues task
|
|||||||
Returns workflow ID immediately
|
Returns workflow ID immediately
|
||||||
↓
|
↓
|
||||||
User can monitor: provisioning workflow monitor <id>
|
User can monitor: provisioning workflow monitor <id>
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 2: Orchestrator Executes Nushell Scripts
|
### Pattern 2: Orchestrator Executes Nushell Scripts
|
||||||
|
|
||||||
@ -227,7 +229,7 @@ pub async fn execute_task(task: Task) -> Result<TaskResult> {
|
|||||||
// Other task types...
|
// Other task types...
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Flow:**
|
**Flow:**
|
||||||
|
|
||||||
@ -245,7 +247,7 @@ Returns result to orchestrator
|
|||||||
Orchestrator updates task status
|
Orchestrator updates task status
|
||||||
↓
|
↓
|
||||||
User monitors via: provisioning workflow status <id>
|
User monitors via: provisioning workflow status <id>
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 3: Bidirectional Communication
|
### Pattern 3: Bidirectional Communication
|
||||||
|
|
||||||
@ -270,7 +272,7 @@ export def report-progress [task_id: string, progress: int] {
|
|||||||
status: "in_progress"
|
status: "in_progress"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Orchestrator Monitors Nushell Execution:**
|
**Orchestrator Monitors Nushell Execution:**
|
||||||
|
|
||||||
@ -306,7 +308,7 @@ pub async fn execute_with_monitoring(task: Task) -> Result<TaskResult> {
|
|||||||
|
|
||||||
Ok(TaskResult::from_exit_status(result))
|
Ok(TaskResult::from_exit_status(result))
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -339,7 +341,7 @@ Orchestrator expects core at: /usr/local/lib/provisioning/
|
|||||||
Core expects orchestrator at: http://localhost:9090/
|
Core expects orchestrator at: http://localhost:9090/
|
||||||
|
|
||||||
No code dependencies, just runtime coordination!
|
No code dependencies, just runtime coordination!
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration-Based Integration
|
### Configuration-Based Integration
|
||||||
|
|
||||||
@ -357,7 +359,7 @@ auto_start = true # Start orchestrator if not running
|
|||||||
[execution]
|
[execution]
|
||||||
default_mode = "orchestrated" # Use orchestrator by default
|
default_mode = "orchestrated" # Use orchestrator by default
|
||||||
fallback_to_direct = true # Fall back if orchestrator down
|
fallback_to_direct = true # Fall back if orchestrator down
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Platform Package (`provisioning-platform`) config:**
|
**Platform Package (`provisioning-platform`) config:**
|
||||||
|
|
||||||
@ -374,7 +376,7 @@ nushell_binary = "nu" # Expects nu in PATH
|
|||||||
provisioning_lib = "/usr/local/lib/provisioning"
|
provisioning_lib = "/usr/local/lib/provisioning"
|
||||||
max_concurrent_tasks = 10
|
max_concurrent_tasks = 10
|
||||||
task_timeout_seconds = 3600
|
task_timeout_seconds = 3600
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Version Compatibility
|
### Version Compatibility
|
||||||
|
|
||||||
@ -390,7 +392,7 @@ api-version = "v1"
|
|||||||
platform = "^2.5" # Core 3.2.1 compatible with platform 2.5.x
|
platform = "^2.5" # Core 3.2.1 compatible with platform 2.5.x
|
||||||
min-platform = "2.5.0"
|
min-platform = "2.5.0"
|
||||||
orchestrator-api = "v1"
|
orchestrator-api = "v1"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -406,7 +408,7 @@ provisioning server list
|
|||||||
# Flow:
|
# Flow:
|
||||||
CLI → servers/list.nu → Query state → Return results
|
CLI → servers/list.nu → Query state → Return results
|
||||||
(Orchestrator not involved)
|
(Orchestrator not involved)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Example 2: Server Creation with Orchestrator
|
### Example 2: Server Creation with Orchestrator
|
||||||
|
|
||||||
@ -458,7 +460,7 @@ provisioning server create --orchestrated --infra wuji
|
|||||||
↓
|
↓
|
||||||
16. User monitors: provisioning workflow status abc-123
|
16. User monitors: provisioning workflow status abc-123
|
||||||
→ Shows: "Server wuji created successfully"
|
→ Shows: "Server wuji created successfully"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Example 3: Batch Workflow with Dependencies
|
### Example 3: Batch Workflow with Dependencies
|
||||||
|
|
||||||
@ -509,7 +511,7 @@ provisioning batch submit multi-cloud-deployment.ncl
|
|||||||
8. If failure occurs, can retry from checkpoint
|
8. If failure occurs, can retry from checkpoint
|
||||||
↓
|
↓
|
||||||
9. User monitors real-time: provisioning batch monitor <id>
|
9. User monitors real-time: provisioning batch monitor <id>
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -519,7 +521,7 @@ provisioning batch submit multi-cloud-deployment.ncl
|
|||||||
|
|
||||||
1. **Eliminates Deep Call Stack Issues**
|
1. **Eliminates Deep Call Stack Issues**
|
||||||
|
|
||||||
```
|
```text
|
||||||
|
|
||||||
Without Orchestrator:
|
Without Orchestrator:
|
||||||
template.nu → calls → cluster.nu → calls → taskserv.nu → calls → provider.nu
|
template.nu → calls → cluster.nu → calls → taskserv.nu → calls → provider.nu
|
||||||
@ -529,7 +531,7 @@ provisioning batch submit multi-cloud-deployment.ncl
|
|||||||
Orchestrator → spawns → Nushell subprocess (flat execution)
|
Orchestrator → spawns → Nushell subprocess (flat execution)
|
||||||
(No deep nesting, fresh Nushell context for each task)
|
(No deep nesting, fresh Nushell context for each task)
|
||||||
|
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
2. **Performance Optimization**
|
2. **Performance Optimization**
|
||||||
|
|
||||||
@ -552,7 +554,7 @@ provisioning batch submit multi-cloud-deployment.ncl
|
|||||||
- Workflow checkpoints (resume on failure)
|
- Workflow checkpoints (resume on failure)
|
||||||
- Progress tracking (real-time monitoring)
|
- Progress tracking (real-time monitoring)
|
||||||
- Retry logic (automatic recovery)
|
- Retry logic (automatic recovery)
|
||||||
```
|
```
|
||||||
|
|
||||||
1. **Clean Separation**
|
1. **Clean Separation**
|
||||||
|
|
||||||
@ -561,7 +563,7 @@ provisioning batch submit multi-cloud-deployment.ncl
|
|||||||
Business Logic (Nushell): Providers, taskservs, workflows
|
Business Logic (Nushell): Providers, taskservs, workflows
|
||||||
|
|
||||||
Each does what it's best at!
|
Each does what it's best at!
|
||||||
```
|
```
|
||||||
|
|
||||||
### Why NOT Pure Rust
|
### Why NOT Pure Rust
|
||||||
|
|
||||||
@ -606,7 +608,7 @@ curl -fsSL https://get.provisioning.io | sh
|
|||||||
→ /usr/local/share/provisioning/platform/ (platform configs)
|
→ /usr/local/share/provisioning/platform/ (platform configs)
|
||||||
|
|
||||||
3. Sets up systemd/launchd service for orchestrator
|
3. Sets up systemd/launchd service for orchestrator
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Runtime Coordination
|
### Runtime Coordination
|
||||||
|
|
||||||
@ -638,7 +640,7 @@ export def ensure-orchestrator [] {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Platform package executes core scripts:**
|
**Platform package executes core scripts:**
|
||||||
|
|
||||||
@ -677,7 +679,7 @@ impl NushellExecutor {
|
|||||||
self.execute_script(&script).await
|
self.execute_script(&script).await
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -714,7 +716,7 @@ force_direct = [
|
|||||||
"help",
|
"help",
|
||||||
"version"
|
"version"
|
||||||
]
|
]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Platform Package Config
|
### Platform Package Config
|
||||||
|
|
||||||
@ -738,7 +740,7 @@ checkpoint_interval_seconds = 30
|
|||||||
binary = "nu" # Expects nu in PATH
|
binary = "nu" # Expects nu in PATH
|
||||||
provisioning_lib = "/usr/local/lib/provisioning"
|
provisioning_lib = "/usr/local/lib/provisioning"
|
||||||
env_vars = { NU_LIB_DIRS = "/usr/local/lib/provisioning" }
|
env_vars = { NU_LIB_DIRS = "/usr/local/lib/provisioning" }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -784,7 +786,7 @@ The confusing example in the multi-repo doc was **oversimplified**. The real arc
|
|||||||
✅ Loose coupling via CLI + REST API (not code dependencies)
|
✅ Loose coupling via CLI + REST API (not code dependencies)
|
||||||
✅ Works identically in monorepo and multi-repo
|
✅ Works identically in monorepo and multi-repo
|
||||||
✅ Configuration-based integration (no hardcoded paths)
|
✅ Configuration-based integration (no hardcoded paths)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
The orchestrator provides:
|
The orchestrator provides:
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# KCL Package and Module Loader System
|
# KCL Package and Module Loader System
|
||||||
|
|
||||||
This document describes the new package-based architecture implemented for the provisioning system, replacing hardcoded extension paths with a flexible module discovery and loading system.
|
This document describes the new package-based architecture implemented for the provisioning system, replacing hardcoded extension paths with a
|
||||||
|
flexible module discovery and loading system.
|
||||||
|
|
||||||
## Architecture Overview
|
## Architecture Overview
|
||||||
|
|
||||||
@ -43,7 +44,7 @@ Contains fundamental schemas for provisioning:
|
|||||||
module-loader discover taskservs # List all taskservs
|
module-loader discover taskservs # List all taskservs
|
||||||
module-loader discover providers --format yaml # List providers as YAML
|
module-loader discover providers --format yaml # List providers as YAML
|
||||||
module-loader discover clusters redis # Search for redis clusters
|
module-loader discover clusters redis # Search for redis clusters
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Supported Module Types
|
#### Supported Module Types
|
||||||
|
|
||||||
@ -65,7 +66,7 @@ module-loader load clusters . [buildkit]
|
|||||||
module-loader init workspace/infra/production \
|
module-loader init workspace/infra/production \
|
||||||
--taskservs [kubernetes, cilium] \
|
--taskservs [kubernetes, cilium] \
|
||||||
--providers [upcloud]
|
--providers [upcloud]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Generated Files
|
#### Generated Files
|
||||||
|
|
||||||
@ -101,7 +102,7 @@ workspace/infra/my-project/
|
|||||||
├── tmp/ # Temporary files
|
├── tmp/ # Temporary files
|
||||||
├── resources/ # Resource definitions
|
├── resources/ # Resource definitions
|
||||||
└── clusters/ # Cluster configurations
|
└── clusters/ # Cluster configurations
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Import Patterns
|
### Import Patterns
|
||||||
|
|
||||||
@ -111,7 +112,7 @@ workspace/infra/my-project/
|
|||||||
# Hardcoded relative paths
|
# Hardcoded relative paths
|
||||||
import ../../../kcl/server as server
|
import ../../../kcl/server as server
|
||||||
import ../../../extensions/taskservs/kubernetes/kcl/kubernetes as k8s
|
import ../../../extensions/taskservs/kubernetes/kcl/kubernetes as k8s
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### After (New System)
|
#### After (New System)
|
||||||
|
|
||||||
@ -121,7 +122,7 @@ import provisioning.server as server
|
|||||||
|
|
||||||
# Auto-generated module imports (after loading)
|
# Auto-generated module imports (after loading)
|
||||||
import .taskservs.nclubernetes.kubernetes as k8s
|
import .taskservs.nclubernetes.kubernetes as k8s
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Package Distribution
|
## Package Distribution
|
||||||
|
|
||||||
@ -136,7 +137,7 @@ import .taskservs.nclubernetes.kubernetes as k8s
|
|||||||
|
|
||||||
# Create release
|
# Create release
|
||||||
./provisioning/tools/kcl-packager.nu build --format tar.gz --include-docs
|
./provisioning/tools/kcl-packager.nu build --format tar.gz --include-docs
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Package Installation Methods
|
### Package Installation Methods
|
||||||
|
|
||||||
@ -145,21 +146,21 @@ import .taskservs.nclubernetes.kubernetes as k8s
|
|||||||
```toml
|
```toml
|
||||||
[dependencies]
|
[dependencies]
|
||||||
provisioning = { path = "~/.kcl/packages/provisioning", version = "0.0.1" }
|
provisioning = { path = "~/.kcl/packages/provisioning", version = "0.0.1" }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Method 2: Git Repository (For distributed teams)
|
#### Method 2: Git Repository (For distributed teams)
|
||||||
|
|
||||||
```toml
|
```toml
|
||||||
[dependencies]
|
[dependencies]
|
||||||
provisioning = { git = "https://github.com/your-org/provisioning-kcl", version = "v0.0.1" }
|
provisioning = { git = "https://github.com/your-org/provisioning-kcl", version = "v0.0.1" }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Method 3: KCL Registry (When available)
|
#### Method 3: KCL Registry (When available)
|
||||||
|
|
||||||
```toml
|
```toml
|
||||||
[dependencies]
|
[dependencies]
|
||||||
provisioning = { version = "0.0.1" }
|
provisioning = { version = "0.0.1" }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Developer Workflows
|
## Developer Workflows
|
||||||
|
|
||||||
@ -180,7 +181,7 @@ module-loader load providers . [upcloud]
|
|||||||
# Validate and deploy
|
# Validate and deploy
|
||||||
kcl run servers.ncl
|
kcl run servers.ncl
|
||||||
provisioning server create --infra . --check
|
provisioning server create --infra . --check
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Extension Development
|
### 2. Extension Development
|
||||||
|
|
||||||
@ -195,7 +196,7 @@ echo 'provisioning = { path = "~/.kcl/packages/provisioning", version = "0.0.1"
|
|||||||
|
|
||||||
# Develop and test
|
# Develop and test
|
||||||
module-loader discover taskservs # Should find your service
|
module-loader discover taskservs # Should find your service
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Workspace Migration
|
### 3. Workspace Migration
|
||||||
|
|
||||||
@ -208,7 +209,7 @@ workspace-migrate.nu workspace/infra/old-project
|
|||||||
|
|
||||||
# Verify migration
|
# Verify migration
|
||||||
module-loader validate workspace/infra/old-project
|
module-loader validate workspace/infra/old-project
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Multi-Environment Management
|
### 4. Multi-Environment Management
|
||||||
|
|
||||||
@ -222,7 +223,7 @@ module-loader load providers . [local]
|
|||||||
cd workspace/infra/prod
|
cd workspace/infra/prod
|
||||||
module-loader load taskservs . [redis, postgres, kubernetes, monitoring]
|
module-loader load taskservs . [redis, postgres, kubernetes, monitoring]
|
||||||
module-loader load providers . [upcloud, aws] # Multi-cloud
|
module-loader load providers . [upcloud, aws] # Multi-cloud
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Module Management
|
## Module Management
|
||||||
|
|
||||||
@ -239,7 +240,7 @@ module-loader validate .
|
|||||||
|
|
||||||
# Show workspace info
|
# Show workspace info
|
||||||
workspace-init.nu . info
|
workspace-init.nu . info
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Unloading Modules
|
### Unloading Modules
|
||||||
|
|
||||||
@ -249,7 +250,7 @@ module-loader unload taskservs . redis
|
|||||||
module-loader unload providers . aws
|
module-loader unload providers . aws
|
||||||
|
|
||||||
# This regenerates import files automatically
|
# This regenerates import files automatically
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Module Information
|
### Module Information
|
||||||
|
|
||||||
@ -258,7 +259,7 @@ module-loader unload providers . aws
|
|||||||
module-loader info taskservs kubernetes
|
module-loader info taskservs kubernetes
|
||||||
module-loader info providers upcloud
|
module-loader info providers upcloud
|
||||||
module-loader info clusters buildkit
|
module-loader info clusters buildkit
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## CI/CD Integration
|
## CI/CD Integration
|
||||||
|
|
||||||
@ -281,7 +282,7 @@ module-loader validate $env.WORKSPACE_PATH
|
|||||||
|
|
||||||
# Deploy infrastructure
|
# Deploy infrastructure
|
||||||
provisioning server create --infra $env.WORKSPACE_PATH
|
provisioning server create --infra $env.WORKSPACE_PATH
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
@ -291,14 +292,14 @@ provisioning server create --infra $env.WORKSPACE_PATH
|
|||||||
|
|
||||||
```plaintext
|
```plaintext
|
||||||
Error: module not found
|
Error: module not found
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Solution**: Verify modules are loaded and regenerate imports
|
**Solution**: Verify modules are loaded and regenerate imports
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
module-loader list taskservs .
|
module-loader list taskservs .
|
||||||
module-loader load taskservs . [kubernetes, cilium, containerd]
|
module-loader load taskservs . [kubernetes, cilium, containerd]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Provider Configuration Issues
|
#### Provider Configuration Issues
|
||||||
|
|
||||||
@ -311,7 +312,7 @@ module-loader load taskservs . [kubernetes, cilium, containerd]
|
|||||||
```bash
|
```bash
|
||||||
kcl-packager.nu install --version latest
|
kcl-packager.nu install --version latest
|
||||||
kcl run --dry-run servers.ncl
|
kcl run --dry-run servers.ncl
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Debug Commands
|
### Debug Commands
|
||||||
|
|
||||||
@ -327,7 +328,7 @@ nickel typecheck workspace/infra/my-project/*.ncl
|
|||||||
|
|
||||||
# Show module manifests
|
# Show module manifests
|
||||||
cat workspace/infra/my-project/.manifest/taskservs.yaml
|
cat workspace/infra/my-project/.manifest/taskservs.yaml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Best Practices
|
## Best Practices
|
||||||
|
|
||||||
@ -363,19 +364,19 @@ For existing workspaces, follow these steps:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp -r workspace/infra/existing workspace/infra/existing-backup
|
cp -r workspace/infra/existing workspace/infra/existing-backup
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Analyze Migration Requirements
|
### 2. Analyze Migration Requirements
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
workspace-migrate.nu workspace/infra/existing dry-run
|
workspace-migrate.nu workspace/infra/existing dry-run
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Perform Migration
|
### 3. Perform Migration
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
workspace-migrate.nu workspace/infra/existing
|
workspace-migrate.nu workspace/infra/existing
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Load Required Modules
|
### 4. Load Required Modules
|
||||||
|
|
||||||
@ -383,20 +384,20 @@ workspace-migrate.nu workspace/infra/existing
|
|||||||
cd workspace/infra/existing
|
cd workspace/infra/existing
|
||||||
module-loader load taskservs . [kubernetes, cilium]
|
module-loader load taskservs . [kubernetes, cilium]
|
||||||
module-loader load providers . [upcloud]
|
module-loader load providers . [upcloud]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 5. Test and Validate
|
### 5. Test and Validate
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
kcl run servers.ncl
|
kcl run servers.ncl
|
||||||
module-loader validate .
|
module-loader validate .
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 6. Deploy
|
### 6. Deploy
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
provisioning server create --infra . --check
|
provisioning server create --infra . --check
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Future Enhancements
|
## Future Enhancements
|
||||||
|
|
||||||
|
|||||||
@ -6,7 +6,9 @@
|
|||||||
|
|
||||||
## Executive Summary
|
## Executive Summary
|
||||||
|
|
||||||
This document analyzes the current project structure and provides a comprehensive plan for optimizing the repository organization and distribution strategy. The goal is to create a professional-grade infrastructure automation system with clear separation of concerns, efficient development workflow, and user-friendly distribution.
|
This document analyzes the current project structure and provides a comprehensive plan for optimizing the repository organization and distribution
|
||||||
|
strategy. The goal is to create a professional-grade infrastructure automation system with clear separation of concerns, efficient development
|
||||||
|
workflow, and user-friendly distribution.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -214,7 +216,7 @@ project-provisioning/
|
|||||||
├── README.md # Project README
|
├── README.md # Project README
|
||||||
├── CHANGELOG.md # Changelog
|
├── CHANGELOG.md # Changelog
|
||||||
└── CLAUDE.md # AI assistant instructions
|
└── CLAUDE.md # AI assistant instructions
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Key Principles
|
### Key Principles
|
||||||
|
|
||||||
@ -258,7 +260,7 @@ project-provisioning/
|
|||||||
├── templates/
|
├── templates/
|
||||||
├── config/
|
├── config/
|
||||||
└── docs/
|
└── docs/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 2. **provisioning-platform** (Optional)
|
#### 2. **provisioning-platform** (Optional)
|
||||||
|
|
||||||
@ -281,7 +283,7 @@ project-provisioning/
|
|||||||
└── share/
|
└── share/
|
||||||
└── provisioning/
|
└── provisioning/
|
||||||
└── platform/
|
└── platform/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 3. **provisioning-extensions** (Optional)
|
#### 3. **provisioning-extensions** (Optional)
|
||||||
|
|
||||||
@ -300,7 +302,7 @@ project-provisioning/
|
|||||||
├── taskservs/
|
├── taskservs/
|
||||||
├── clusters/
|
├── clusters/
|
||||||
└── workflows/
|
└── workflows/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 4. **provisioning-plugins** (Optional)
|
#### 4. **provisioning-plugins** (Optional)
|
||||||
|
|
||||||
@ -317,7 +319,7 @@ project-provisioning/
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
~/.config/nushell/plugins/
|
~/.config/nushell/plugins/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Installation Paths
|
### Installation Paths
|
||||||
|
|
||||||
@ -345,7 +347,7 @@ project-provisioning/
|
|||||||
├── config/ # Default configs
|
├── config/ # Default configs
|
||||||
│ └── config.defaults.toml
|
│ └── config.defaults.toml
|
||||||
└── docs/ # Documentation
|
└── docs/ # Documentation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### User Configuration
|
#### User Configuration
|
||||||
|
|
||||||
@ -359,7 +361,7 @@ project-provisioning/
|
|||||||
│ └── clusters/
|
│ └── clusters/
|
||||||
├── cache/ # Cache directory
|
├── cache/ # Cache directory
|
||||||
└── plugins/ # User plugins
|
└── plugins/ # User plugins
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Project Workspace
|
#### Project Workspace
|
||||||
|
|
||||||
@ -378,7 +380,7 @@ project-provisioning/
|
|||||||
│ ├── state/
|
│ ├── state/
|
||||||
│ └── cache/
|
│ └── cache/
|
||||||
└── extensions/ # Project-specific extensions
|
└── extensions/ # Project-specific extensions
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Hierarchy
|
### Configuration Hierarchy
|
||||||
|
|
||||||
@ -389,7 +391,7 @@ Priority (highest to lowest):
|
|||||||
3. Project config ./workspace/config/config.toml
|
3. Project config ./workspace/config/config.toml
|
||||||
4. User config ~/.provisioning/config/config.user.toml
|
4. User config ~/.provisioning/config/config.user.toml
|
||||||
5. System config /usr/local/share/provisioning/config/config.defaults.toml
|
5. System config /usr/local/share/provisioning/config/config.defaults.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -409,7 +411,7 @@ build/
|
|||||||
├── create-installers.nu # Installer generation
|
├── create-installers.nu # Installer generation
|
||||||
├── validate-package.nu # Package validation
|
├── validate-package.nu # Package validation
|
||||||
└── publish-registry.nu # Registry publishing
|
└── publish-registry.nu # Registry publishing
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Build System Implementation
|
### Build System Implementation
|
||||||
|
|
||||||
@ -587,7 +589,7 @@ export def "main status" [] {
|
|||||||
$packages | select name size
|
$packages | select name size
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Justfile Integration
|
### Justfile Integration
|
||||||
|
|
||||||
@ -715,7 +717,7 @@ check-licenses:
|
|||||||
# Security audit
|
# Security audit
|
||||||
audit:
|
audit:
|
||||||
cargo audit --file provisioning/platform/Cargo.lock
|
cargo audit --file provisioning/platform/Cargo.lock
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -976,7 +978,7 @@ export def "main upgrade" [
|
|||||||
error make { msg: "Upgrade failed" }
|
error make { msg: "Upgrade failed" }
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Bash Installer (For Systems Without Nushell)
|
### Bash Installer (For Systems Without Nushell)
|
||||||
|
|
||||||
@ -1090,7 +1092,7 @@ main() {
|
|||||||
|
|
||||||
# Run main
|
# Run main
|
||||||
main "$@"
|
main "$@"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -1115,7 +1117,7 @@ cp -r /Users/Akasha/project-provisioning /Users/Akasha/project-provisioning.back
|
|||||||
|
|
||||||
# Analyze workspaces
|
# Analyze workspaces
|
||||||
fd workspace -t d > workspace-dirs.txt
|
fd workspace -t d > workspace-dirs.txt
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Deliverables:**
|
**Deliverables:**
|
||||||
|
|
||||||
@ -1144,7 +1146,7 @@ mv provisioning/tools/dist distribution/packages/
|
|||||||
|
|
||||||
# Remove obsolete
|
# Remove obsolete
|
||||||
rm -rf NO/ wrks/ presentations/
|
rm -rf NO/ wrks/ presentations/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Deliverables:**
|
**Deliverables:**
|
||||||
|
|
||||||
@ -1417,7 +1419,7 @@ provisioning upgrade --version 3.2.0
|
|||||||
|
|
||||||
# Migrate workspace
|
# Migrate workspace
|
||||||
provisioning workspace migrate --from workspace.backup --to workspace/
|
provisioning workspace migrate --from workspace.backup --to workspace/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Option 2: In-Place Migration
|
#### Option 2: In-Place Migration
|
||||||
|
|
||||||
@ -1425,7 +1427,7 @@ provisioning workspace migrate --from workspace.backup --to workspace/
|
|||||||
# Run migration script
|
# Run migration script
|
||||||
provisioning migrate --check # Dry run
|
provisioning migrate --check # Dry run
|
||||||
provisioning migrate # Execute migration
|
provisioning migrate # Execute migration
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### For Developers
|
### For Developers
|
||||||
|
|
||||||
@ -1442,7 +1444,7 @@ just install-dev
|
|||||||
|
|
||||||
# Verify
|
# Verify
|
||||||
provisioning --version
|
provisioning --version
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -1551,7 +1553,7 @@ provisioning --version
|
|||||||
## Timeline Summary
|
## Timeline Summary
|
||||||
|
|
||||||
| Phase | Duration | Key Deliverables |
|
| Phase | Duration | Key Deliverables |
|
||||||
|-------|----------|------------------|
|
| ------- | ---------- | ------------------ |
|
||||||
| Phase 1: Restructuring | 3-4 days | Clean directory structure, updated paths |
|
| Phase 1: Restructuring | 3-4 days | Clean directory structure, updated paths |
|
||||||
| Phase 2: Build System | 3-4 days | Working build system, all package types |
|
| Phase 2: Build System | 3-4 days | Working build system, all package types |
|
||||||
| Phase 3: Installation | 2-3 days | Installers, pure Nushell CLI |
|
| Phase 3: Installation | 2-3 days | Installers, pure Nushell CLI |
|
||||||
@ -1592,7 +1594,8 @@ This comprehensive plan transforms the provisioning system into a professional-g
|
|||||||
- **Extensible**: Package registry for community extensions
|
- **Extensible**: Package registry for community extensions
|
||||||
- **Well Documented**: Complete guides for users and developers
|
- **Well Documented**: Complete guides for users and developers
|
||||||
|
|
||||||
The implementation will take approximately **2-3 weeks** and will result in a production-ready system suitable for both individual developers and enterprise deployments.
|
The implementation will take approximately **2-3 weeks** and will result in a production-ready system suitable for both individual developers and
|
||||||
|
enterprise deployments.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -2,7 +2,8 @@
|
|||||||
|
|
||||||
## Executive Summary
|
## Executive Summary
|
||||||
|
|
||||||
Provisioning is an **Infrastructure Automation Platform** built with a hybrid Rust/Nushell architecture. It enables Infrastructure as Code (IaC) with multi-provider support (AWS, UpCloud, local), sophisticated workflow orchestration, and configuration-driven operations.
|
Provisioning is an **Infrastructure Automation Platform** built with a hybrid Rust/Nushell architecture. It enables Infrastructure as Code (IaC) with
|
||||||
|
multi-provider support (AWS, UpCloud, local), sophisticated workflow orchestration, and configuration-driven operations.
|
||||||
|
|
||||||
The system solves fundamental technical challenges through architectural innovation and hybrid language design.
|
The system solves fundamental technical challenges through architectural innovation and hybrid language design.
|
||||||
|
|
||||||
@ -54,7 +55,7 @@ The system solves fundamental technical challenges through architectural innovat
|
|||||||
│ • UpCloud │ • Services │ • Containers │
|
│ • UpCloud │ • Services │ • Containers │
|
||||||
│ • Others │ • Storage │ • Host Services │
|
│ • Others │ • Storage │ • Host Services │
|
||||||
└─────────────────┴─────────────────┴─────────────────────────────┘
|
└─────────────────┴─────────────────┴─────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Core Components
|
## Core Components
|
||||||
|
|
||||||
@ -172,7 +173,7 @@ The system solves fundamental technical challenges through architectural innovat
|
|||||||
]
|
]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Provider Ecosystem
|
### 4. Provider Ecosystem
|
||||||
|
|
||||||
@ -249,7 +250,7 @@ The system solves fundamental technical challenges through architectural innovat
|
|||||||
```plaintext
|
```plaintext
|
||||||
1. Workspace Discovery → 2. Configuration Loading → 3. Hierarchy Merge →
|
1. Workspace Discovery → 2. Configuration Loading → 3. Hierarchy Merge →
|
||||||
4. Variable Interpolation → 5. Schema Validation → 6. Runtime Application
|
4. Variable Interpolation → 5. Schema Validation → 6. Runtime Application
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Workflow Execution Flow
|
### Workflow Execution Flow
|
||||||
|
|
||||||
@ -257,7 +258,7 @@ The system solves fundamental technical challenges through architectural innovat
|
|||||||
1. Workflow Submission → 2. Dependency Analysis → 3. Task Scheduling →
|
1. Workflow Submission → 2. Dependency Analysis → 3. Task Scheduling →
|
||||||
4. Parallel Execution → 5. State Tracking → 6. Result Aggregation →
|
4. Parallel Execution → 5. State Tracking → 6. Result Aggregation →
|
||||||
7. Error Handling → 8. Cleanup/Rollback
|
7. Error Handling → 8. Cleanup/Rollback
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Provider Integration Flow
|
### Provider Integration Flow
|
||||||
|
|
||||||
@ -265,7 +266,7 @@ The system solves fundamental technical challenges through architectural innovat
|
|||||||
1. Provider Discovery → 2. Configuration Validation → 3. Authentication →
|
1. Provider Discovery → 2. Configuration Validation → 3. Authentication →
|
||||||
4. Resource Planning → 5. Operation Execution → 6. State Persistence →
|
4. Resource Planning → 5. Operation Execution → 6. State Persistence →
|
||||||
7. Result Reporting
|
7. Result Reporting
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Technology Stack
|
## Technology Stack
|
||||||
|
|
||||||
@ -350,4 +351,5 @@ The system solves fundamental technical challenges through architectural innovat
|
|||||||
- **Configuration Schema**: Extensible configuration with validation
|
- **Configuration Schema**: Extensible configuration with validation
|
||||||
- **Workflow Engine**: Custom workflow definitions and execution
|
- **Workflow Engine**: Custom workflow definitions and execution
|
||||||
|
|
||||||
This system architecture represents a mature, production-ready platform for Infrastructure as Code with unique architectural innovations and proven scalability.
|
This system architecture represents a mature, production-ready platform for Infrastructure as Code with unique architectural innovations and proven
|
||||||
|
scalability.
|
||||||
|
|||||||
@ -19,7 +19,7 @@ TypeDialog Form (Auto-generated)
|
|||||||
User fills form interactively
|
User fills form interactively
|
||||||
↓
|
↓
|
||||||
Nickel output config (Type-safe)
|
Nickel output config (Type-safe)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -35,7 +35,7 @@ TypeDialog Form Engine
|
|||||||
Nickel Integration
|
Nickel Integration
|
||||||
↓
|
↓
|
||||||
Schema Contracts
|
Schema Contracts
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Data Flow
|
### Data Flow
|
||||||
|
|
||||||
@ -51,7 +51,7 @@ User Input
|
|||||||
Validation (against Nickel contracts)
|
Validation (against Nickel contracts)
|
||||||
↓
|
↓
|
||||||
Output (JSON/YAML/TOML/Nickel)
|
Output (JSON/YAML/TOML/Nickel)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -69,14 +69,14 @@ cargo build --release
|
|||||||
|
|
||||||
# Install (optional)
|
# Install (optional)
|
||||||
cargo install --path ./crates/typedialog
|
cargo install --path ./crates/typedialog
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Verify Installation
|
### Verify Installation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
typedialog --version
|
typedialog --version
|
||||||
typedialog --help
|
typedialog --help
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -97,7 +97,7 @@ let defaults = import "./defaults.ncl" in
|
|||||||
|
|
||||||
DefaultServer = defaults.server,
|
DefaultServer = defaults.server,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 2: Define TypeDialog Form (TOML)
|
### Step 2: Define TypeDialog Form (TOML)
|
||||||
|
|
||||||
@ -151,13 +151,13 @@ label = "Tags"
|
|||||||
type = "multiselect"
|
type = "multiselect"
|
||||||
options = ["production", "staging", "testing", "development"]
|
options = ["production", "staging", "testing", "development"]
|
||||||
help = "Select applicable tags"
|
help = "Select applicable tags"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 3: Render Form (CLI)
|
### Step 3: Render Form (CLI)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
typedialog form --config server_form.toml --backend cli
|
typedialog form --config server_form.toml --backend cli
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Output**:
|
**Output**:
|
||||||
|
|
||||||
@ -175,14 +175,14 @@ Create a new server configuration
|
|||||||
◯ staging
|
◯ staging
|
||||||
◯ testing
|
◯ testing
|
||||||
◯ development
|
◯ development
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 4: Validate Against Nickel Schema
|
### Step 4: Validate Against Nickel Schema
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Validation happens automatically
|
# Validation happens automatically
|
||||||
# If input matches Nickel contract, proceeds to output
|
# If input matches Nickel contract, proceeds to output
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 5: Output to Nickel
|
### Step 5: Output to Nickel
|
||||||
|
|
||||||
@ -191,7 +191,7 @@ typedialog form \
|
|||||||
--config server_form.toml \
|
--config server_form.toml \
|
||||||
--output nickel \
|
--output nickel \
|
||||||
--backend cli
|
--backend cli
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Output file** (`server_config_output.ncl`):
|
**Output file** (`server_config_output.ncl`):
|
||||||
|
|
||||||
@ -204,7 +204,7 @@ typedialog form \
|
|||||||
monitoring = true,
|
monitoring = true,
|
||||||
tags = ["production"],
|
tags = ["production"],
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -241,7 +241,7 @@ You want an interactive CLI wizard for infrastructure provisioning.
|
|||||||
|
|
||||||
DefaultInfra = defaults,
|
DefaultInfra = defaults,
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 2: Create Comprehensive Form
|
### Step 2: Create Comprehensive Form
|
||||||
|
|
||||||
@ -330,7 +330,7 @@ required = true
|
|||||||
validation_pattern = "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
|
validation_pattern = "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
|
||||||
help = "For alerts and notifications"
|
help = "For alerts and notifications"
|
||||||
placeholder = "admin@company.com"
|
placeholder = "admin@company.com"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 3: Run Interactive Wizard
|
### Step 3: Run Interactive Wizard
|
||||||
|
|
||||||
@ -339,7 +339,7 @@ typedialog form \
|
|||||||
--config infrastructure_wizard.toml \
|
--config infrastructure_wizard.toml \
|
||||||
--backend tui \
|
--backend tui \
|
||||||
--output nickel
|
--output nickel
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Output** (`infrastructure_config.ncl`):
|
**Output** (`infrastructure_config.ncl`):
|
||||||
|
|
||||||
@ -354,7 +354,7 @@ typedialog form \
|
|||||||
backup_retention_days = 30,
|
backup_retention_days = 30,
|
||||||
email = "ops@company.com",
|
email = "ops@company.com",
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 4: Use Output in Infrastructure
|
### Step 4: Use Output in Infrastructure
|
||||||
|
|
||||||
@ -390,7 +390,7 @@ let schemas = import "../../provisioning/schemas/main.ncl" in
|
|||||||
# default fallback
|
# default fallback
|
||||||
{},
|
{},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -528,7 +528,7 @@ section = "advanced"
|
|||||||
label = "Tags"
|
label = "Tags"
|
||||||
type = "multiselect"
|
type = "multiselect"
|
||||||
options = ["production", "staging", "testing", "development"]
|
options = ["production", "staging", "testing", "development"]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Output Structure
|
### Output Structure
|
||||||
|
|
||||||
@ -554,7 +554,7 @@ options = ["production", "staging", "testing", "development"]
|
|||||||
monitoring_interval = 30,
|
monitoring_interval = 30,
|
||||||
tags = ["production"],
|
tags = ["production"],
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -570,7 +570,7 @@ typedialog server --port 8080
|
|||||||
curl -X POST http://localhost:8080/forms \
|
curl -X POST http://localhost:8080/forms \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d @server_form.toml
|
-d @server_form.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Response Format
|
### Response Format
|
||||||
|
|
||||||
@ -588,7 +588,7 @@ curl -X POST http://localhost:8080/forms \
|
|||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Submit Form
|
### Submit Form
|
||||||
|
|
||||||
@ -603,7 +603,7 @@ curl -X POST http://localhost:8080/forms/srv_abc123/submit \
|
|||||||
"monitoring": true,
|
"monitoring": true,
|
||||||
"tags": ["production"]
|
"tags": ["production"]
|
||||||
}'
|
}'
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Response
|
### Response
|
||||||
|
|
||||||
@ -621,7 +621,7 @@ curl -X POST http://localhost:8080/forms/srv_abc123/submit \
|
|||||||
"tags": ["production"]
|
"tags": ["production"]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -641,7 +641,7 @@ ServerConfig = {
|
|||||||
|
|
||||||
# If user enters invalid value
|
# If user enters invalid value
|
||||||
# TypeDialog rejects before serializing
|
# TypeDialog rejects before serializing
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Validation Rules in Form
|
### Validation Rules in Form
|
||||||
|
|
||||||
@ -653,7 +653,7 @@ min = 1
|
|||||||
max = 32
|
max = 32
|
||||||
help = "Must be 1-32 cores"
|
help = "Must be 1-32 cores"
|
||||||
# TypeDialog enforces before user can submit
|
# TypeDialog enforces before user can submit
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -675,7 +675,7 @@ provisioning init --wizard
|
|||||||
|
|
||||||
# 4. Provisioning uses output
|
# 4. Provisioning uses output
|
||||||
# provisioning server create --from-config infrastructure_config.ncl
|
# provisioning server create --from-config infrastructure_config.ncl
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Implementation in Nushell
|
### Implementation in Nushell
|
||||||
|
|
||||||
@ -704,7 +704,7 @@ def provisioning_init_wizard [] {
|
|||||||
print "Infrastructure configuration created!"
|
print "Infrastructure configuration created!"
|
||||||
print "Use: provisioning deploy --from-config"
|
print "Use: provisioning deploy --from-config"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -720,7 +720,7 @@ name = "backup_retention"
|
|||||||
label = "Backup Retention (days)"
|
label = "Backup Retention (days)"
|
||||||
type = "number"
|
type = "number"
|
||||||
visible_if = "enable_backup == true" # Only shown if backup enabled
|
visible_if = "enable_backup == true" # Only shown if backup enabled
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Dynamic Defaults
|
### Dynamic Defaults
|
||||||
|
|
||||||
@ -737,7 +737,7 @@ name = "cpu_cores"
|
|||||||
type = "number"
|
type = "number"
|
||||||
default_from = "deployment_mode" # Can reference other fields
|
default_from = "deployment_mode" # Can reference other fields
|
||||||
# solo → default 2, enterprise → default 16
|
# solo → default 2, enterprise → default 16
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Custom Validation
|
### Custom Validation
|
||||||
|
|
||||||
@ -747,7 +747,7 @@ name = "memory_gb"
|
|||||||
type = "number"
|
type = "number"
|
||||||
validation_rule = "memory_gb >= cpu_cores * 2"
|
validation_rule = "memory_gb >= cpu_cores * 2"
|
||||||
help = "Memory must be at least 2 GB per CPU core"
|
help = "Memory must be at least 2 GB per CPU core"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -767,7 +767,7 @@ typedialog form --config form.toml --output yaml
|
|||||||
|
|
||||||
# Output to TOML (for application config)
|
# Output to TOML (for application config)
|
||||||
typedialog form --config form.toml --output toml
|
typedialog form --config form.toml --output toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -779,7 +779,7 @@ TypeDialog supports three rendering backends:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
typedialog form --config form.toml --backend cli
|
typedialog form --config form.toml --backend cli
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: Lightweight, SSH-friendly, no dependencies
|
**Pros**: Lightweight, SSH-friendly, no dependencies
|
||||||
**Cons**: Basic UI
|
**Cons**: Basic UI
|
||||||
@ -788,7 +788,7 @@ typedialog form --config form.toml --backend cli
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
typedialog form --config form.toml --backend tui
|
typedialog form --config form.toml --backend tui
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: Rich UI, keyboard navigation, sections
|
**Pros**: Rich UI, keyboard navigation, sections
|
||||||
**Cons**: Requires terminal support
|
**Cons**: Requires terminal support
|
||||||
@ -798,7 +798,7 @@ typedialog form --config form.toml --backend tui
|
|||||||
```bash
|
```bash
|
||||||
typedialog form --config form.toml --backend web --port 3000
|
typedialog form --config form.toml --backend web --port 3000
|
||||||
# Opens http://localhost:3000
|
# Opens http://localhost:3000
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pros**: Beautiful UI, remote access, multi-user
|
**Pros**: Beautiful UI, remote access, multi-user
|
||||||
**Cons**: Requires browser, network
|
**Cons**: Requires browser, network
|
||||||
@ -818,7 +818,7 @@ typedialog form --config form.toml --backend web --port 3000
|
|||||||
[[fields]]
|
[[fields]]
|
||||||
name = "cpu_cores" # Must match Nickel field name
|
name = "cpu_cores" # Must match Nickel field name
|
||||||
type = "number" # Must match Nickel type
|
type = "number" # Must match Nickel type
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Problem: Validation fails
|
### Problem: Validation fails
|
||||||
|
|
||||||
@ -831,7 +831,7 @@ type = "number" # Must match Nickel type
|
|||||||
name = "cpu_cores"
|
name = "cpu_cores"
|
||||||
validation_pattern = "^[1-9][0-9]*$"
|
validation_pattern = "^[1-9][0-9]*$"
|
||||||
help = "Must be positive integer"
|
help = "Must be positive integer"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Problem: Output not valid Nickel
|
### Problem: Output not valid Nickel
|
||||||
|
|
||||||
@ -843,7 +843,7 @@ help = "Must be positive integer"
|
|||||||
[[fields]]
|
[[fields]]
|
||||||
name = "required_field"
|
name = "required_field"
|
||||||
required = true # User must provide value
|
required = true # User must provide value
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -862,7 +862,7 @@ required = true # User must provide value
|
|||||||
email = "",
|
email = "",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 2: Define Form
|
### Step 2: Define Form
|
||||||
|
|
||||||
@ -891,14 +891,14 @@ type = "confirm"
|
|||||||
name = "email"
|
name = "email"
|
||||||
type = "text"
|
type = "text"
|
||||||
required = true
|
required = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 3: User Interaction
|
### Step 3: User Interaction
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
$ typedialog form --config workspace_form.toml --backend tui
|
$ typedialog form --config workspace_form.toml --backend tui
|
||||||
# User fills form interactively
|
# User fills form interactively
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 4: Output
|
### Step 4: Output
|
||||||
|
|
||||||
@ -912,7 +912,7 @@ $ typedialog form --config workspace_form.toml --backend tui
|
|||||||
email = "ops@company.com",
|
email = "ops@company.com",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 5: Use in Provisioning
|
### Step 5: Use in Provisioning
|
||||||
|
|
||||||
@ -928,7 +928,7 @@ let schemas = import "provisioning/schemas/main.ncl" in
|
|||||||
provider = config.workspace.provider,
|
provider = config.workspace.provider,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -24,7 +24,7 @@ enabled = true
|
|||||||
name = "my-service"
|
name = "my-service"
|
||||||
version = "1.0.0"
|
version = "1.0.0"
|
||||||
# Error: Required field missing: enabled
|
# Error: Required field missing: enabled
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Type Validation
|
### 2. Type Validation
|
||||||
|
|
||||||
@ -48,7 +48,7 @@ enabled = true
|
|||||||
|
|
||||||
# Invalid - wrong type
|
# Invalid - wrong type
|
||||||
port = "8080" # Error: Expected int, got string
|
port = "8080" # Error: Expected int, got string
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Enum Validation
|
### 3. Enum Validation
|
||||||
|
|
||||||
@ -65,7 +65,7 @@ environment = "prod"
|
|||||||
|
|
||||||
# Invalid
|
# Invalid
|
||||||
environment = "production" # Error: Must be one of: dev, staging, prod
|
environment = "production" # Error: Must be one of: dev, staging, prod
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Range Validation
|
### 4. Range Validation
|
||||||
|
|
||||||
@ -86,7 +86,7 @@ port = 80 # Error: Must be >= 1024
|
|||||||
|
|
||||||
# Invalid - above maximum
|
# Invalid - above maximum
|
||||||
port = 70000 # Error: Must be <= 65535
|
port = 70000 # Error: Must be <= 65535
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 5. Pattern Validation
|
### 5. Pattern Validation
|
||||||
|
|
||||||
@ -103,7 +103,7 @@ email = "admin@example.com"
|
|||||||
|
|
||||||
# Invalid
|
# Invalid
|
||||||
email = "not-an-email" # Error: Does not match pattern
|
email = "not-an-email" # Error: Does not match pattern
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 6. Deprecated Fields
|
### 6. Deprecated Fields
|
||||||
|
|
||||||
@ -119,7 +119,7 @@ old_field = "new_field"
|
|||||||
|
|
||||||
# Config using deprecated field
|
# Config using deprecated field
|
||||||
old_field = "value" # Warning: old_field is deprecated. Use new_field instead.
|
old_field = "value" # Warning: old_field is deprecated. Use new_field instead.
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Using Schema Validator
|
## Using Schema Validator
|
||||||
|
|
||||||
@ -137,7 +137,7 @@ provisioning platform validate orchestrator
|
|||||||
|
|
||||||
# Validate with detailed output
|
# Validate with detailed output
|
||||||
provisioning workspace config validate --verbose
|
provisioning workspace config validate --verbose
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Programmatic Usage
|
### Programmatic Usage
|
||||||
|
|
||||||
@ -167,7 +167,7 @@ if ($result.warnings | length) > 0 {
|
|||||||
print $" • ($warning.message)"
|
print $" • ($warning.message)"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pretty Print Results
|
### Pretty Print Results
|
||||||
|
|
||||||
@ -175,7 +175,7 @@ if ($result.warnings | length) > 0 {
|
|||||||
# Validate and print formatted results
|
# Validate and print formatted results
|
||||||
let result = (validate-workspace-config $config)
|
let result = (validate-workspace-config $config)
|
||||||
print-validation-results $result
|
print-validation-results $result
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Schema Examples
|
## Schema Examples
|
||||||
|
|
||||||
@ -216,7 +216,7 @@ type = "bool"
|
|||||||
[fields.debug.log_level]
|
[fields.debug.log_level]
|
||||||
type = "string"
|
type = "string"
|
||||||
enum = ["debug", "info", "warn", "error"]
|
enum = ["debug", "info", "warn", "error"]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Provider Schema (AWS)
|
### Provider Schema (AWS)
|
||||||
|
|
||||||
@ -273,7 +273,7 @@ fields = ["old_region_field"]
|
|||||||
|
|
||||||
[deprecated_replacements]
|
[deprecated_replacements]
|
||||||
old_region_field = "provider.region"
|
old_region_field = "provider.region"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Platform Service Schema (Orchestrator)
|
### Platform Service Schema (Orchestrator)
|
||||||
|
|
||||||
@ -319,7 +319,7 @@ max = 10000
|
|||||||
|
|
||||||
[fields.queue.storage_path]
|
[fields.queue.storage_path]
|
||||||
type = "string"
|
type = "string"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### KMS Service Schema
|
### KMS Service Schema
|
||||||
|
|
||||||
@ -366,7 +366,7 @@ fields = ["old_kms_type"]
|
|||||||
|
|
||||||
[deprecated_replacements]
|
[deprecated_replacements]
|
||||||
old_kms_type = "kms.provider"
|
old_kms_type = "kms.provider"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Validation Workflow
|
## Validation Workflow
|
||||||
|
|
||||||
@ -382,7 +382,7 @@ provisioning workspace config validate
|
|||||||
# Fix errors and revalidate
|
# Fix errors and revalidate
|
||||||
vim ~/workspaces/dev/config/provisioning.yaml
|
vim ~/workspaces/dev/config/provisioning.yaml
|
||||||
provisioning workspace config validate
|
provisioning workspace config validate
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. CI/CD Pipeline
|
### 2. CI/CD Pipeline
|
||||||
|
|
||||||
@ -398,7 +398,7 @@ validate-config:
|
|||||||
only:
|
only:
|
||||||
changes:
|
changes:
|
||||||
- "*/config/**/*"
|
- "*/config/**/*"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Pre-Deployment
|
### 3. Pre-Deployment
|
||||||
|
|
||||||
@ -412,7 +412,7 @@ provisioning platform validate --all
|
|||||||
if [[ $? -eq 0 ]]; then
|
if [[ $? -eq 0 ]]; then
|
||||||
provisioning deploy --workspace production
|
provisioning deploy --workspace production
|
||||||
fi
|
fi
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Error Messages
|
## Error Messages
|
||||||
|
|
||||||
@ -430,7 +430,7 @@ Errors:
|
|||||||
|
|
||||||
⚠️ Warnings:
|
⚠️ Warnings:
|
||||||
• Field old_field is deprecated. Use new_field instead.
|
• Field old_field is deprecated. Use new_field instead.
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Error Details
|
### Error Details
|
||||||
|
|
||||||
@ -449,7 +449,7 @@ Each error includes:
|
|||||||
[fields.hostname]
|
[fields.hostname]
|
||||||
type = "string"
|
type = "string"
|
||||||
pattern = "^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
|
pattern = "^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 2: Email Validation
|
### Pattern 2: Email Validation
|
||||||
|
|
||||||
@ -457,7 +457,7 @@ pattern = "^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
|
|||||||
[fields.email]
|
[fields.email]
|
||||||
type = "string"
|
type = "string"
|
||||||
pattern = "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
|
pattern = "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 3: Semantic Version
|
### Pattern 3: Semantic Version
|
||||||
|
|
||||||
@ -465,7 +465,7 @@ pattern = "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
|
|||||||
[fields.version]
|
[fields.version]
|
||||||
type = "string"
|
type = "string"
|
||||||
pattern = "^\\d+\\.\\d+\\.\\d+(-[a-zA-Z0-9]+)?$"
|
pattern = "^\\d+\\.\\d+\\.\\d+(-[a-zA-Z0-9]+)?$"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 4: URL Validation
|
### Pattern 4: URL Validation
|
||||||
|
|
||||||
@ -473,7 +473,7 @@ pattern = "^\\d+\\.\\d+\\.\\d+(-[a-zA-Z0-9]+)?$"
|
|||||||
[fields.url]
|
[fields.url]
|
||||||
type = "string"
|
type = "string"
|
||||||
pattern = "^https?://[a-zA-Z0-9.-]+(:[0-9]+)?(/.*)?$"
|
pattern = "^https?://[a-zA-Z0-9.-]+(:[0-9]+)?(/.*)?$"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 5: IPv4 Address
|
### Pattern 5: IPv4 Address
|
||||||
|
|
||||||
@ -481,7 +481,7 @@ pattern = "^https?://[a-zA-Z0-9.-]+(:[0-9]+)?(/.*)?$"
|
|||||||
[fields.ip_address]
|
[fields.ip_address]
|
||||||
type = "string"
|
type = "string"
|
||||||
pattern = "^(?:[0-9]{1,3}\\.){3}[0-9]{1,3}$"
|
pattern = "^(?:[0-9]{1,3}\\.){3}[0-9]{1,3}$"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 6: AWS Resource ID
|
### Pattern 6: AWS Resource ID
|
||||||
|
|
||||||
@ -497,7 +497,7 @@ pattern = "^ami-[a-f0-9]{8,17}$"
|
|||||||
[fields.vpc_id]
|
[fields.vpc_id]
|
||||||
type = "string"
|
type = "string"
|
||||||
pattern = "^vpc-[a-f0-9]{8,17}$"
|
pattern = "^vpc-[a-f0-9]{8,17}$"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Testing Validation
|
## Testing Validation
|
||||||
|
|
||||||
@ -506,7 +506,7 @@ pattern = "^vpc-[a-f0-9]{8,17}$"
|
|||||||
```nushell
|
```nushell
|
||||||
# Run validation test suite
|
# Run validation test suite
|
||||||
nu provisioning/tests/config_validation_tests.nu
|
nu provisioning/tests/config_validation_tests.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Integration Tests
|
### Integration Tests
|
||||||
|
|
||||||
@ -515,7 +515,7 @@ nu provisioning/tests/config_validation_tests.nu
|
|||||||
provisioning test validate --workspace dev
|
provisioning test validate --workspace dev
|
||||||
provisioning test validate --workspace staging
|
provisioning test validate --workspace staging
|
||||||
provisioning test validate --workspace prod
|
provisioning test validate --workspace prod
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Custom Validation
|
### Custom Validation
|
||||||
|
|
||||||
@ -537,7 +537,7 @@ def validate-custom-config [config: record] {
|
|||||||
|
|
||||||
$result
|
$result
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Best Practices
|
## Best Practices
|
||||||
|
|
||||||
@ -548,7 +548,7 @@ def validate-custom-config [config: record] {
|
|||||||
provisioning workspace config validate
|
provisioning workspace config validate
|
||||||
|
|
||||||
# Don't wait for deployment
|
# Don't wait for deployment
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Use Strict Schemas
|
### 2. Use Strict Schemas
|
||||||
|
|
||||||
@ -560,7 +560,7 @@ min = 1024
|
|||||||
max = 65535
|
max = 65535
|
||||||
|
|
||||||
# Don't leave fields unvalidated
|
# Don't leave fields unvalidated
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Document Patterns
|
### 3. Document Patterns
|
||||||
|
|
||||||
@ -570,7 +570,7 @@ max = 65535
|
|||||||
type = "string"
|
type = "string"
|
||||||
pattern = "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
|
pattern = "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
|
||||||
# Example: user@example.com
|
# Example: user@example.com
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Handle Deprecation
|
### 4. Handle Deprecation
|
||||||
|
|
||||||
@ -578,7 +578,7 @@ pattern = "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$"
|
|||||||
# Always provide replacement guidance
|
# Always provide replacement guidance
|
||||||
[deprecated_replacements]
|
[deprecated_replacements]
|
||||||
old_field = "new_field" # Clear migration path
|
old_field = "new_field" # Clear migration path
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 5. Test Schemas
|
### 5. Test Schemas
|
||||||
|
|
||||||
@ -586,7 +586,7 @@ old_field = "new_field" # Clear migration path
|
|||||||
# Include test cases in comments
|
# Include test cases in comments
|
||||||
# Valid: "admin@example.com"
|
# Valid: "admin@example.com"
|
||||||
# Invalid: "not-an-email"
|
# Invalid: "not-an-email"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
@ -597,7 +597,7 @@ old_field = "new_field" # Clear migration path
|
|||||||
|
|
||||||
# Solution: Ensure schema exists
|
# Solution: Ensure schema exists
|
||||||
ls -la /Users/Akasha/project-provisioning/provisioning/config/*.schema.toml
|
ls -la /Users/Akasha/project-provisioning/provisioning/config/*.schema.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern Not Matching
|
### Pattern Not Matching
|
||||||
|
|
||||||
@ -606,7 +606,7 @@ ls -la /Users/Akasha/project-provisioning/provisioning/config/*.schema.toml
|
|||||||
|
|
||||||
# Debug: Test pattern separately
|
# Debug: Test pattern separately
|
||||||
echo "my-hostname" | grep -E "^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
|
echo "my-hostname" | grep -E "^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Type Mismatch
|
### Type Mismatch
|
||||||
|
|
||||||
@ -621,7 +621,7 @@ cat ~/workspaces/dev/config/provisioning.yaml | yq '.server.port'
|
|||||||
vim ~/workspaces/dev/config/provisioning.yaml
|
vim ~/workspaces/dev/config/provisioning.yaml
|
||||||
# Change: port: "8080"
|
# Change: port: "8080"
|
||||||
# To: port: 8080
|
# To: port: 8080
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Additional Resources
|
## Additional Resources
|
||||||
|
|
||||||
|
|||||||
@ -60,7 +60,7 @@ This guide describes the metadata-driven authentication system implemented over
|
|||||||
┌────────────▼─────────────┐
|
┌────────────▼─────────────┐
|
||||||
│ Result/Response │
|
│ Result/Response │
|
||||||
└─────────────────────────┘
|
└─────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Data Flow
|
### Data Flow
|
||||||
|
|
||||||
@ -107,7 +107,7 @@ cd project-provisioning
|
|||||||
nu tests/test-fase5-e2e.nu
|
nu tests/test-fase5-e2e.nu
|
||||||
nu tests/test-security-audit-day20.nu
|
nu tests/test-security-audit-day20.nu
|
||||||
nu tests/test-metadata-cache-benchmark.nu
|
nu tests/test-metadata-cache-benchmark.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Usage Guide
|
## Usage Guide
|
||||||
|
|
||||||
@ -131,7 +131,7 @@ provisioning batch submit workflows/batch-deploy.ncl
|
|||||||
|
|
||||||
# Check without executing
|
# Check without executing
|
||||||
provisioning server create --name test --check
|
provisioning server create --name test --check
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Authentication Flow
|
### Authentication Flow
|
||||||
|
|
||||||
@ -156,7 +156,7 @@ Are you sure? [yes/no] yes
|
|||||||
$ provisioning taskserv delete postgres web-01
|
$ provisioning taskserv delete postgres web-01
|
||||||
Auth check: Check auth for destructive operation
|
Auth check: Check auth for destructive operation
|
||||||
✓ Taskserv deleted
|
✓ Taskserv deleted
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Check Mode (Bypass Auth for Testing)
|
### Check Mode (Bypass Auth for Testing)
|
||||||
|
|
||||||
@ -168,7 +168,7 @@ provisioning server create --name test --check
|
|||||||
Dry-run mode - no changes will be made
|
Dry-run mode - no changes will be made
|
||||||
✓ Would create server: test
|
✓ Would create server: test
|
||||||
✓ Would deploy taskservs: []
|
✓ Would deploy taskservs: []
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Non-Interactive CI/CD Mode
|
### Non-Interactive CI/CD Mode
|
||||||
|
|
||||||
@ -181,7 +181,7 @@ provisioning batch submit workflows/batch.ncl --yes --check
|
|||||||
|
|
||||||
# With environment variable
|
# With environment variable
|
||||||
PROVISIONING_NON_INTERACTIVE=1 provisioning server create --name web-02 --yes
|
PROVISIONING_NON_INTERACTIVE=1 provisioning server create --name web-02 --yes
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Migration Path
|
## Migration Path
|
||||||
|
|
||||||
@ -199,7 +199,7 @@ export def delete-server [name: string, --yes] {
|
|||||||
if not $yes { ... manual confirmation ... }
|
if not $yes { ... manual confirmation ... }
|
||||||
# ... deletion logic ...
|
# ... deletion logic ...
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**New Pattern** (After Fase 5):
|
**New Pattern** (After Fase 5):
|
||||||
|
|
||||||
@ -218,7 +218,7 @@ export def delete-server [name: string, --yes] {
|
|||||||
# Operation type: "delete" automatically detected
|
# Operation type: "delete" automatically detected
|
||||||
# ... deletion logic ...
|
# ... deletion logic ...
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Phase 2: Adding Metadata Headers
|
### Phase 2: Adding Metadata Headers
|
||||||
|
|
||||||
@ -237,7 +237,7 @@ export def delete-server [name: string, --yes] {
|
|||||||
export def create-server [name: string] {
|
export def create-server [name: string] {
|
||||||
# Logic here
|
# Logic here
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
1. Register in `provisioning/schemas/main.ncl`:
|
1. Register in `provisioning/schemas/main.ncl`:
|
||||||
|
|
||||||
@ -255,7 +255,7 @@ let server_create = {
|
|||||||
},
|
},
|
||||||
} in
|
} in
|
||||||
server_create
|
server_create
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
1. Handler integration (happens in dispatcher):
|
1. Handler integration (happens in dispatcher):
|
||||||
|
|
||||||
@ -265,7 +265,7 @@ server_create
|
|||||||
# 2. Validates auth based on requirements
|
# 2. Validates auth based on requirements
|
||||||
# 3. Checks permission levels
|
# 3. Checks permission levels
|
||||||
# 4. Calls handler if validation passes
|
# 4. Calls handler if validation passes
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Phase 3: Validating Migration
|
### Phase 3: Validating Migration
|
||||||
|
|
||||||
@ -284,7 +284,7 @@ nu utils/search-scripts.nu by-tags server delete
|
|||||||
|
|
||||||
# List all migrated scripts
|
# List all migrated scripts
|
||||||
nu utils/search-scripts.nu list
|
nu utils/search-scripts.nu list
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Developer Guide
|
## Developer Guide
|
||||||
|
|
||||||
@ -306,7 +306,7 @@ let new_feature_command = {
|
|||||||
},
|
},
|
||||||
} in
|
} in
|
||||||
new_feature_command
|
new_feature_command
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Step 2: Add metadata header to script**
|
**Step 2: Add metadata header to script**
|
||||||
|
|
||||||
@ -321,7 +321,7 @@ new_feature_command
|
|||||||
export def feature-command [param: string] {
|
export def feature-command [param: string] {
|
||||||
# Implementation
|
# Implementation
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Step 3: Implement handler function**
|
**Step 3: Implement handler function**
|
||||||
|
|
||||||
@ -338,7 +338,7 @@ export def handle-feature-command [
|
|||||||
|
|
||||||
# Your logic here
|
# Your logic here
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Step 4: Test with check mode**
|
**Step 4: Test with check mode**
|
||||||
|
|
||||||
@ -348,12 +348,12 @@ provisioning feature command --check
|
|||||||
|
|
||||||
# Full execution
|
# Full execution
|
||||||
provisioning feature command --yes
|
provisioning feature command --yes
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Metadata Field Reference
|
### Metadata Field Reference
|
||||||
|
|
||||||
| Field | Type | Required | Description |
|
| Field | Type | Required | Description |
|
||||||
|-------|------|----------|-------------|
|
| ------- | ------ | ---------- | ------------- |
|
||||||
| name | string | Yes | Command canonical name |
|
| name | string | Yes | Command canonical name |
|
||||||
| domain | string | Yes | Command category (infrastructure, orchestration, etc.) |
|
| domain | string | Yes | Command category (infrastructure, orchestration, etc.) |
|
||||||
| description | string | Yes | Human-readable description |
|
| description | string | Yes | Human-readable description |
|
||||||
@ -395,7 +395,7 @@ if (get-operation-duration "my-operation") > 2000 {
|
|||||||
submit-to-orchestrator $operation
|
submit-to-orchestrator $operation
|
||||||
return "Operation submitted in background"
|
return "Operation submitted in background"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pattern 2: For Batch Operations**
|
**Pattern 2: For Batch Operations**
|
||||||
|
|
||||||
@ -405,7 +405,7 @@ nu -c "
|
|||||||
use core/nulib/workflows/batch.nu *
|
use core/nulib/workflows/batch.nu *
|
||||||
batch submit workflows/batch-deploy.ncl --parallel-limit 5
|
batch submit workflows/batch-deploy.ncl --parallel-limit 5
|
||||||
"
|
"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Pattern 3: For Metadata Overhead**
|
**Pattern 3: For Metadata Overhead**
|
||||||
|
|
||||||
@ -414,7 +414,7 @@ batch submit workflows/batch-deploy.ncl --parallel-limit 5
|
|||||||
# Current: 40-100x faster with warm cache
|
# Current: 40-100x faster with warm cache
|
||||||
# Target: >95% cache hit rate
|
# Target: >95% cache hit rate
|
||||||
# Achieved: Metadata stays in cache for 1 hour (TTL)
|
# Achieved: Metadata stays in cache for 1 hour (TTL)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
@ -432,12 +432,12 @@ nu tests/test-metadata-cache-benchmark.nu
|
|||||||
|
|
||||||
# Run all tests
|
# Run all tests
|
||||||
for test in tests/test-*.nu { nu $test }
|
for test in tests/test-*.nu { nu $test }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Test Coverage
|
### Test Coverage
|
||||||
|
|
||||||
| Test Suite | Category | Coverage |
|
| Test Suite | Category | Coverage |
|
||||||
|-----------|----------|----------|
|
| ----------- | ---------- | ---------- |
|
||||||
| E2E Tests | Integration | 7 test groups, 40+ checks |
|
| E2E Tests | Integration | 7 test groups, 40+ checks |
|
||||||
| Security Audit | Auth | 5 audit categories, 100% pass |
|
| Security Audit | Auth | 5 audit categories, 100% pass |
|
||||||
| Benchmarks | Performance | 6 benchmark categories |
|
| Benchmarks | Performance | 6 benchmark categories |
|
||||||
@ -459,7 +459,7 @@ for test in tests/test-*.nu { nu $test }
|
|||||||
```bash
|
```bash
|
||||||
# Check if command is in metadata
|
# Check if command is in metadata
|
||||||
grep "command_name" provisioning/schemas/main.ncl
|
grep "command_name" provisioning/schemas/main.ncl
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Issue: Auth check failing
|
### Issue: Auth check failing
|
||||||
|
|
||||||
@ -474,7 +474,7 @@ nu -c "
|
|||||||
use core/nulib/lib_provisioning/commands/traits.nu *
|
use core/nulib/lib_provisioning/commands/traits.nu *
|
||||||
get-command-metadata 'server create'
|
get-command-metadata 'server create'
|
||||||
"
|
"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Issue: Slow command execution
|
### Issue: Slow command execution
|
||||||
|
|
||||||
@ -486,7 +486,7 @@ rm ~/.cache/provisioning/command_metadata.json
|
|||||||
|
|
||||||
# Check cache hit rate
|
# Check cache hit rate
|
||||||
nu tests/test-metadata-cache-benchmark.nu
|
nu tests/test-metadata-cache-benchmark.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Issue: Nushell syntax error
|
### Issue: Nushell syntax error
|
||||||
|
|
||||||
@ -499,14 +499,14 @@ nu --ide-check 100 <file.nu>
|
|||||||
# Check for common issues
|
# Check for common issues
|
||||||
grep "try {" <file.nu> # Should be empty
|
grep "try {" <file.nu> # Should be empty
|
||||||
grep "let mut" <file.nu> # Should be empty
|
grep "let mut" <file.nu> # Should be empty
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Performance Characteristics
|
## Performance Characteristics
|
||||||
|
|
||||||
### Baseline Metrics
|
### Baseline Metrics
|
||||||
|
|
||||||
| Operation | Cold | Warm | Improvement |
|
| Operation | Cold | Warm | Improvement |
|
||||||
|-----------|------|------|-------------|
|
| ----------- | ------ | ------ | ------------- |
|
||||||
| Metadata Load | 200 ms | 2-5 ms | 40-100x |
|
| Metadata Load | 200 ms | 2-5 ms | 40-100x |
|
||||||
| Auth Check | <5 ms | <5 ms | Same |
|
| Auth Check | <5 ms | <5 ms | Same |
|
||||||
| Command Dispatch | <10 ms | <10 ms | Same |
|
| Command Dispatch | <10 ms | <10 ms | Same |
|
||||||
@ -519,7 +519,7 @@ Scenario: 20 sequential commands
|
|||||||
Without cache: 20 × 200 ms = 4 seconds
|
Without cache: 20 × 200 ms = 4 seconds
|
||||||
With cache: 1 × 200 ms + 19 × 5 ms = 295 ms
|
With cache: 1 × 200 ms + 19 × 5 ms = 295 ms
|
||||||
Speedup: ~13.5x faster
|
Speedup: ~13.5x faster
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Next Steps
|
## Next Steps
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Build System Documentation
|
# Build System Documentation
|
||||||
|
|
||||||
This document provides comprehensive documentation for the provisioning project's build system, including the complete Makefile reference with 40+ targets, build tools, compilation instructions, and troubleshooting.
|
This document provides comprehensive documentation for the provisioning project's build system, including the complete Makefile reference with 40+
|
||||||
|
targets, build tools, compilation instructions, and troubleshooting.
|
||||||
|
|
||||||
## Table of Contents
|
## Table of Contents
|
||||||
|
|
||||||
@ -1071,4 +1072,5 @@ make ci-test
|
|||||||
make ci-release
|
make ci-release
|
||||||
```
|
```
|
||||||
|
|
||||||
This build system provides a comprehensive, maintainable foundation for the provisioning project's development lifecycle, from local development to production releases.
|
This build system provides a comprehensive, maintainable foundation for the provisioning project's development lifecycle, from local development to
|
||||||
|
production releases.
|
||||||
|
|||||||
@ -2,11 +2,12 @@
|
|||||||
|
|
||||||
**Target Audience**: Developers working on the provisioning CLI
|
**Target Audience**: Developers working on the provisioning CLI
|
||||||
**Last Updated**: 2025-09-30
|
**Last Updated**: 2025-09-30
|
||||||
**Related**: [ADR-006 CLI Refactoring](../architecture/adr/ADR-006-provisioning-cli-refactoring.md)
|
**Related**: [ADR-006 CLI Refactoring](../architecture/adr/adr-006-provisioning-cli-refactoring.md)
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
The provisioning CLI uses a **modular, domain-driven architecture** that separates concerns into focused command handlers. This guide shows you how to work with this architecture.
|
The provisioning CLI uses a **modular, domain-driven architecture** that separates concerns into focused command handlers. This guide shows you how to
|
||||||
|
work with this architecture.
|
||||||
|
|
||||||
### Key Architecture Principles
|
### Key Architecture Principles
|
||||||
|
|
||||||
@ -33,7 +34,7 @@ provisioning/core/nulib/
|
|||||||
│ ├── generation.nu (78 lines) - Generate commands
|
│ ├── generation.nu (78 lines) - Generate commands
|
||||||
│ ├── utilities.nu (157 lines) - SSH, SOPS, cache, providers
|
│ ├── utilities.nu (157 lines) - SSH, SOPS, cache, providers
|
||||||
│ └── configuration.nu (316 lines) - Env, show, init, validate
|
│ └── configuration.nu (316 lines) - Env, show, init, validate
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Adding New Commands
|
## Adding New Commands
|
||||||
|
|
||||||
@ -42,14 +43,14 @@ provisioning/core/nulib/
|
|||||||
Commands are organized by domain. Choose the appropriate handler:
|
Commands are organized by domain. Choose the appropriate handler:
|
||||||
|
|
||||||
| Domain | Handler | Responsibility |
|
| Domain | Handler | Responsibility |
|
||||||
|--------|---------|----------------|
|
| -------- | --------- | ---------------- |
|
||||||
| `infrastructure.nu` | Server/taskserv/cluster/infra lifecycle |
|
| infrastructure | `infrastructure.nu` | Server/taskserv/cluster/infra lifecycle |
|
||||||
| `orchestration.nu` | Workflow/batch operations, orchestrator control |
|
| orchestration | `orchestration.nu` | Workflow/batch operations, orchestrator control |
|
||||||
| `development.nu` | Module discovery, layers, versions, packaging |
|
| development | `development.nu` | Module discovery, layers, versions, packaging |
|
||||||
| `workspace.nu` | Workspace and template management |
|
| workspace | `workspace.nu` | Workspace and template management |
|
||||||
| `configuration.nu` | Environment, settings, initialization |
|
| configuration | `configuration.nu` | Environment, settings, initialization |
|
||||||
| `utilities.nu` | SSH, SOPS, cache, providers, utilities |
|
| utilities | `utilities.nu` | SSH, SOPS, cache, providers, utilities |
|
||||||
| `generation.nu` | Generate commands (server, taskserv, etc.) |
|
| generation | `generation.nu` | Generate commands (server, taskserv, etc.) |
|
||||||
|
|
||||||
### Step 2: Add Command to Handler
|
### Step 2: Add Command to Handler
|
||||||
|
|
||||||
@ -91,7 +92,7 @@ def handle_server [ops: string, flags: record] {
|
|||||||
let args = build_module_args $flags $ops
|
let args = build_module_args $flags $ops
|
||||||
run_module $args "server" --exec
|
run_module $args "server" --exec
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**That's it!** The command is now available as `provisioning server status`.
|
**That's it!** The command is now available as `provisioning server status`.
|
||||||
|
|
||||||
@ -114,7 +115,7 @@ export def get_command_registry []: nothing -> record {
|
|||||||
# ... rest of registry
|
# ... rest of registry
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Note**: Most shortcuts are already configured. You only need to add new shortcuts if you're creating completely new command categories.
|
**Note**: Most shortcuts are already configured. You only need to add new shortcuts if you're creating completely new command categories.
|
||||||
|
|
||||||
@ -131,7 +132,7 @@ def handle_taskserv [ops: string, flags: record] {
|
|||||||
let args = build_module_args $flags $ops
|
let args = build_module_args $flags $ops
|
||||||
run_module $args "taskserv" --exec
|
run_module $args "taskserv" --exec
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**After:**
|
**After:**
|
||||||
|
|
||||||
@ -154,7 +155,7 @@ def handle_taskserv [ops: string, flags: record] {
|
|||||||
let args = build_module_args $flags $ops
|
let args = build_module_args $flags $ops
|
||||||
run_module $args "taskserv" --exec
|
run_module $args "taskserv" --exec
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Working with Flags
|
## Working with Flags
|
||||||
|
|
||||||
@ -175,14 +176,14 @@ let args = build_module_args $parsed_flags $ops
|
|||||||
|
|
||||||
# Set environment variables based on flags
|
# Set environment variables based on flags
|
||||||
set_debug_env $parsed_flags
|
set_debug_env $parsed_flags
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Available Flag Parsing
|
### Available Flag Parsing
|
||||||
|
|
||||||
The `parse_common_flags` function normalizes these flags:
|
The `parse_common_flags` function normalizes these flags:
|
||||||
|
|
||||||
| Flag Record Field | Description |
|
| Flag Record Field | Description |
|
||||||
|-------------------|-------------|
|
| ------------------- | ------------- |
|
||||||
| `show_version` | Version display (`--version`, `-v`) |
|
| `show_version` | Version display (`--version`, `-v`) |
|
||||||
| `show_info` | Info display (`--info`, `-i`) |
|
| `show_info` | Info display (`--info`, `-i`) |
|
||||||
| `show_about` | About display (`--about`, `-a`) |
|
| `show_about` | About display (`--about`, `-a`) |
|
||||||
@ -238,7 +239,7 @@ export def build_module_args [flags: record, extra: string = ""]: nothing -> str
|
|||||||
# ... rest of function
|
# ... rest of function
|
||||||
$"($extra) ($use_check)($use_yes)($use_wait)($str_timeout)..."
|
$"($extra) ($use_check)($use_yes)($use_wait)($str_timeout)..."
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Adding New Shortcuts
|
## Adding New Shortcuts
|
||||||
|
|
||||||
@ -264,7 +265,7 @@ export def get_command_registry []: nothing -> record {
|
|||||||
# ... rest of registry
|
# ... rest of registry
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Important**: After adding a shortcut, update the help system in `help_system.nu` to document it.
|
**Important**: After adding a shortcut, update the help system in `help_system.nu` to document it.
|
||||||
|
|
||||||
@ -275,7 +276,7 @@ export def get_command_registry []: nothing -> record {
|
|||||||
```bash
|
```bash
|
||||||
# Run comprehensive test suite
|
# Run comprehensive test suite
|
||||||
nu tests/test_provisioning_refactor.nu
|
nu tests/test_provisioning_refactor.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Test Coverage
|
### Test Coverage
|
||||||
|
|
||||||
@ -312,7 +313,7 @@ export def main [] {
|
|||||||
|
|
||||||
# ... rest of main
|
# ... rest of main
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Manual Testing
|
### Manual Testing
|
||||||
|
|
||||||
@ -326,7 +327,7 @@ provisioning/core/cli/provisioning --debug my-command test
|
|||||||
# Test help
|
# Test help
|
||||||
provisioning/core/cli/provisioning my-command help
|
provisioning/core/cli/provisioning my-command help
|
||||||
provisioning/core/cli/provisioning help my-command # Bi-directional
|
provisioning/core/cli/provisioning help my-command # Bi-directional
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Common Patterns
|
## Common Patterns
|
||||||
|
|
||||||
@ -339,7 +340,7 @@ def handle_simple_command [ops: string, flags: record] {
|
|||||||
let args = build_module_args $flags $ops
|
let args = build_module_args $flags $ops
|
||||||
run_module $args "module_name" --exec
|
run_module $args "module_name" --exec
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 2: Command with Validation
|
### Pattern 2: Command with Validation
|
||||||
|
|
||||||
@ -359,7 +360,7 @@ def handle_validated_command [ops: string, flags: record] {
|
|||||||
let args = build_module_args $flags $ops
|
let args = build_module_args $flags $ops
|
||||||
run_module $args "module_name" --exec
|
run_module $args "module_name" --exec
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 3: Command with Subcommands
|
### Pattern 3: Command with Subcommands
|
||||||
|
|
||||||
@ -381,7 +382,7 @@ def handle_complex_command [ops: string, flags: record] {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Pattern 4: Command with Flag-Based Routing
|
### Pattern 4: Command with Flag-Based Routing
|
||||||
|
|
||||||
@ -400,7 +401,7 @@ def handle_flag_routed_command [ops: string, flags: record] {
|
|||||||
run_module $args "module_name" --exec
|
run_module $args "module_name" --exec
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Best Practices
|
## Best Practices
|
||||||
|
|
||||||
@ -426,7 +427,7 @@ print " • containerd"
|
|||||||
print " • cilium"
|
print " • cilium"
|
||||||
print ""
|
print ""
|
||||||
print "Use 'provisioning taskserv list' to see all available taskservs"
|
print "Use 'provisioning taskserv list' to see all available taskservs"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Leverage Centralized Functions
|
### 3. Leverage Centralized Functions
|
||||||
|
|
||||||
@ -447,7 +448,7 @@ def handle_good [ops: string, flags: record] {
|
|||||||
let args = build_module_args $flags $ops
|
let args = build_module_args $flags $ops
|
||||||
run_module $args "module" --exec
|
run_module $args "module" --exec
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Document Your Changes
|
### 4. Document Your Changes
|
||||||
|
|
||||||
@ -485,7 +486,7 @@ use ../../lib_provisioning *
|
|||||||
# ❌ Wrong
|
# ❌ Wrong
|
||||||
use ../main_provisioning/flags *
|
use ../main_provisioning/flags *
|
||||||
use lib_provisioning *
|
use lib_provisioning *
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Issue: "Parse mismatch: expected colon"
|
### Issue: "Parse mismatch: expected colon"
|
||||||
|
|
||||||
@ -503,7 +504,7 @@ export def my_function [param: string]: nothing -> string {
|
|||||||
export def my_function [param: string] -> string {
|
export def my_function [param: string] -> string {
|
||||||
"result"
|
"result"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Issue: "Command not routing correctly"
|
### Issue: "Command not routing correctly"
|
||||||
|
|
||||||
@ -513,7 +514,7 @@ export def my_function [param: string] -> string {
|
|||||||
|
|
||||||
```nushell
|
```nushell
|
||||||
"myshortcut" => "domain command"
|
"myshortcut" => "domain command"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Issue: "Flags not being passed"
|
### Issue: "Flags not being passed"
|
||||||
|
|
||||||
@ -524,7 +525,7 @@ export def my_function [param: string] -> string {
|
|||||||
```nushell
|
```nushell
|
||||||
let args = build_module_args $flags $ops
|
let args = build_module_args $flags $ops
|
||||||
run_module $args "module" --exec
|
run_module $args "module" --exec
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Quick Reference
|
## Quick Reference
|
||||||
|
|
||||||
@ -542,10 +543,10 @@ tests/
|
|||||||
└── test_provisioning_refactor.nu - Test suite
|
└── test_provisioning_refactor.nu - Test suite
|
||||||
docs/
|
docs/
|
||||||
├── architecture/
|
├── architecture/
|
||||||
│ └── ADR-006-provisioning-cli-refactoring.md - Architecture docs
|
│ └── adr-006-provisioning-cli-refactoring.md - Architecture docs
|
||||||
└── development/
|
└── development/
|
||||||
└── COMMAND_HANDLER_GUIDE.md - This guide
|
└── COMMAND_HANDLER_GUIDE.md - This guide
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Key Functions
|
### Key Functions
|
||||||
|
|
||||||
@ -569,7 +570,7 @@ help-orchestration []: nothing -> string
|
|||||||
# In commands/*.nu
|
# In commands/*.nu
|
||||||
handle_*_command [command: string, ops: string, flags: record]
|
handle_*_command [command: string, ops: string, flags: record]
|
||||||
# Example: handle_infrastructure_command, handle_workspace_command
|
# Example: handle_infrastructure_command, handle_workspace_command
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Testing Commands
|
### Testing Commands
|
||||||
|
|
||||||
@ -586,11 +587,11 @@ provisioning/core/cli/provisioning --debug my-command test
|
|||||||
# Test help
|
# Test help
|
||||||
provisioning/core/cli/provisioning help my-command
|
provisioning/core/cli/provisioning help my-command
|
||||||
provisioning/core/cli/provisioning my-command help # Bi-directional
|
provisioning/core/cli/provisioning my-command help # Bi-directional
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Further Reading
|
## Further Reading
|
||||||
|
|
||||||
- **[ADR-006: CLI Refactoring](../architecture/adr/ADR-006-provisioning-cli-refactoring.md)** - Complete architectural decision record
|
- **[ADR-006: CLI Refactoring](../architecture/adr/adr-006-provisioning-cli-refactoring.md)** - Complete architectural decision record
|
||||||
- **[Project Structure](project-structure.md)** - Overall project organization
|
- **[Project Structure](project-structure.md)** - Overall project organization
|
||||||
- **[Workflow Development](workflow.md)** - Workflow system architecture
|
- **[Workflow Development](workflow.md)** - Workflow system architecture
|
||||||
- **[Development Integration](integration.md)** - Integration patterns
|
- **[Development Integration](integration.md)** - Integration patterns
|
||||||
|
|||||||
@ -6,20 +6,24 @@ Implemented graceful CTRL-C handling for sudo password prompts during server cre
|
|||||||
|
|
||||||
## Problem Statement
|
## Problem Statement
|
||||||
|
|
||||||
When `fix_local_hosts: true` is set, the provisioning tool requires sudo access to modify `/etc/hosts` and SSH config. When a user cancels the sudo password prompt (no password, wrong password, timeout), the system would:
|
When `fix_local_hosts: true` is set, the provisioning tool requires sudo access to
|
||||||
|
modify `/etc/hosts` and SSH config. When a user cancels the sudo password prompt (no
|
||||||
|
password, wrong password, timeout), the system would:
|
||||||
|
|
||||||
1. Exit with code 1 (sudo failed)
|
1. Exit with code 1 (sudo failed)
|
||||||
2. Propagate null values up the call stack
|
2. Propagate null values up the call stack
|
||||||
3. Show cryptic Nushell errors about pipeline failures
|
3. Show cryptic Nushell errors about pipeline failures
|
||||||
4. Leave the operation in an inconsistent state
|
4. Leave the operation in an inconsistent state
|
||||||
|
|
||||||
**Important Unix Limitation**: Pressing CTRL-C at the sudo password prompt sends SIGINT to the entire process group, interrupting Nushell before exit code handling can occur. This **cannot be caught** and is expected Unix behavior.
|
**Important Unix Limitation**: Pressing CTRL-C at the sudo password prompt sends SIGINT to the entire process group, interrupting Nushell before exit
|
||||||
|
code handling can occur. This **cannot be caught** and is expected Unix behavior.
|
||||||
|
|
||||||
## Solution Architecture
|
## Solution Architecture
|
||||||
|
|
||||||
### Key Principle: Return Values, Not Exit Codes
|
### Key Principle: Return Values, Not Exit Codes
|
||||||
|
|
||||||
Instead of using `exit 130` which kills the entire process, we use **return values** to signal cancellation and let each layer of the call stack handle it gracefully.
|
Instead of using `exit 130` which kills the entire process, we use **return values**
|
||||||
|
to signal cancellation and let each layer of the call stack handle it gracefully.
|
||||||
|
|
||||||
### Three-Layer Approach
|
### Three-Layer Approach
|
||||||
|
|
||||||
@ -60,7 +64,7 @@ def run_sudo_with_interrupt_check [
|
|||||||
}
|
}
|
||||||
true
|
true
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Design Decision**: Return `bool` instead of throwing error or calling `exit`. This allows the caller to decide how to handle cancellation.
|
**Design Decision**: Return `bool` instead of throwing error or calling `exit`. This allows the caller to decide how to handle cancellation.
|
||||||
|
|
||||||
@ -72,7 +76,7 @@ if $server.fix_local_hosts and not (check_sudo_cached) {
|
|||||||
print "ℹ You will be prompted for your password, or press CTRL-C to cancel"
|
print "ℹ You will be prompted for your password, or press CTRL-C to cancel"
|
||||||
print " Tip: Run 'sudo -v' beforehand to cache credentials\n"
|
print " Tip: Run 'sudo -v' beforehand to cache credentials\n"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Design Decision**: Warn users upfront so they're not surprised by the password prompt.
|
**Design Decision**: Warn users upfront so they're not surprised by the password prompt.
|
||||||
|
|
||||||
@ -86,7 +90,7 @@ if $result.exit_code == 1 and ($result.stderr | str contains "password is requir
|
|||||||
print "\n⚠ Operation cancelled"
|
print "\n⚠ Operation cancelled"
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Design Decision**: Use `do --ignore-errors` + `complete` to capture both exit code and stderr without throwing exceptions.
|
**Design Decision**: Use `do --ignore-errors` + `complete` to capture both exit code and stderr without throwing exceptions.
|
||||||
|
|
||||||
@ -103,7 +107,7 @@ let all_succeeded = ($settings.data.servers | reduce -f true { |server, acc|
|
|||||||
$acc
|
$acc
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Design Decision**: Nushell doesn't allow mutable variable capture in closures. Use `reduce` for accumulating boolean state across iterations.
|
**Design Decision**: Nushell doesn't allow mutable variable capture in closures. Use `reduce` for accumulating boolean state across iterations.
|
||||||
|
|
||||||
@ -115,7 +119,7 @@ if not $ssh_result {
|
|||||||
_print "\n✗ Server creation cancelled"
|
_print "\n✗ Server creation cancelled"
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Design Decision**: Check return value and provide context-specific message before returning.
|
**Design Decision**: Check return value and provide context-specific message before returning.
|
||||||
|
|
||||||
@ -145,7 +149,7 @@ Return false to settings.nu
|
|||||||
settings.nu handles false gracefully (no append)
|
settings.nu handles false gracefully (no append)
|
||||||
↓
|
↓
|
||||||
Clean exit, no cryptic errors
|
Clean exit, no cryptic errors
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Nushell Idioms Used
|
## Nushell Idioms Used
|
||||||
|
|
||||||
@ -156,7 +160,7 @@ Captures both stdout, stderr, and exit code without throwing:
|
|||||||
```nushell
|
```nushell
|
||||||
let result = (do --ignore-errors { ^sudo command } | complete)
|
let result = (do --ignore-errors { ^sudo command } | complete)
|
||||||
# result = { stdout: "...", stderr: "...", exit_code: 1 }
|
# result = { stdout: "...", stderr: "...", exit_code: 1 }
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. `reduce` for Accumulation
|
### 2. `reduce` for Accumulation
|
||||||
|
|
||||||
@ -173,7 +177,7 @@ $servers | each { |s|
|
|||||||
let all_succeeded = ($servers | reduce -f true { |s, acc|
|
let all_succeeded = ($servers | reduce -f true { |s, acc|
|
||||||
$acc and (check_server $s)
|
$acc and (check_server $s)
|
||||||
})
|
})
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Early Returns for Error Handling
|
### 3. Early Returns for Error Handling
|
||||||
|
|
||||||
@ -183,7 +187,7 @@ if not $condition {
|
|||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
# Continue with happy path
|
# Continue with happy path
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Testing Scenarios
|
## Testing Scenarios
|
||||||
|
|
||||||
@ -197,7 +201,7 @@ provisioning -c server create
|
|||||||
# ⚠ Operation cancelled - sudo password required but not provided
|
# ⚠ Operation cancelled - sudo password required but not provided
|
||||||
# ℹ Run 'sudo -v' first to cache credentials
|
# ℹ Run 'sudo -v' first to cache credentials
|
||||||
# ✗ Server creation cancelled
|
# ✗ Server creation cancelled
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Scenario 2: Pre-cached Credentials
|
### Scenario 2: Pre-cached Credentials
|
||||||
|
|
||||||
@ -206,7 +210,7 @@ sudo -v
|
|||||||
provisioning -c server create
|
provisioning -c server create
|
||||||
|
|
||||||
# Expected: No password prompt, smooth operation
|
# Expected: No password prompt, smooth operation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Scenario 3: Wrong Password 3 Times
|
### Scenario 3: Wrong Password 3 Times
|
||||||
|
|
||||||
@ -217,7 +221,7 @@ provisioning -c server create
|
|||||||
# Password: [wrong]
|
# Password: [wrong]
|
||||||
|
|
||||||
# Expected: Same as CTRL-C (treated as cancellation)
|
# Expected: Same as CTRL-C (treated as cancellation)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Scenario 4: Multiple Servers, Cancel on Second
|
### Scenario 4: Multiple Servers, Cancel on Second
|
||||||
|
|
||||||
@ -226,7 +230,7 @@ provisioning -c server create
|
|||||||
# - First server completes successfully
|
# - First server completes successfully
|
||||||
# - Second server shows cancellation message
|
# - Second server shows cancellation message
|
||||||
# - Operation stops, doesn't proceed to third
|
# - Operation stops, doesn't proceed to third
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Maintenance Notes
|
## Maintenance Notes
|
||||||
|
|
||||||
@ -247,7 +251,7 @@ if $result.exit_code == 1 and ($result.stderr | str contains "password is requir
|
|||||||
print "\n⚠ Operation cancelled - sudo password required"
|
print "\n⚠ Operation cancelled - sudo password required"
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Common Pitfalls
|
### Common Pitfalls
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Configuration Management
|
# Configuration Management
|
||||||
|
|
||||||
This document provides comprehensive guidance on provisioning's configuration architecture, environment-specific configurations, validation, error handling, and migration strategies.
|
This document provides comprehensive guidance on provisioning's configuration architecture, environment-specific configurations, validation, error
|
||||||
|
handling, and migration strategies.
|
||||||
|
|
||||||
## Table of Contents
|
## Table of Contents
|
||||||
|
|
||||||
@ -16,7 +17,8 @@ This document provides comprehensive guidance on provisioning's configuration ar
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Provisioning implements a sophisticated configuration management system that has migrated from environment variable-based configuration to a hierarchical TOML configuration system with comprehensive validation and interpolation support.
|
Provisioning implements a sophisticated configuration management system that has migrated from environment variable-based configuration to a
|
||||||
|
hierarchical TOML configuration system with comprehensive validation and interpolation support.
|
||||||
|
|
||||||
**Key Features**:
|
**Key Features**:
|
||||||
|
|
||||||
@ -61,7 +63,7 @@ Configuration Hierarchy (Low → High Precedence)
|
|||||||
│ 6. Runtime environment variables │ ← Runtime overrides
|
│ 6. Runtime environment variables │ ← Runtime overrides
|
||||||
│ (PROVISIONING_* variables) │
|
│ (PROVISIONING_* variables) │
|
||||||
└─────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Access Patterns
|
### Configuration Access Patterns
|
||||||
|
|
||||||
@ -85,7 +87,7 @@ let log_level = (get-config-env "logging.level" "info")
|
|||||||
|
|
||||||
# Interpolated configuration
|
# Interpolated configuration
|
||||||
let data_path = (get-config-interpolated "paths.data") # Resolves {{paths.base}}/data
|
let data_path = (get-config-interpolated "paths.data") # Resolves {{paths.base}}/data
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Migration from ENV Variables
|
### Migration from ENV Variables
|
||||||
|
|
||||||
@ -96,7 +98,7 @@ export PROVISIONING_UPCLOUD_API_KEY="your-key"
|
|||||||
export PROVISIONING_UPCLOUD_API_URL="https://api.upcloud.com"
|
export PROVISIONING_UPCLOUD_API_URL="https://api.upcloud.com"
|
||||||
export PROVISIONING_LOG_LEVEL="debug"
|
export PROVISIONING_LOG_LEVEL="debug"
|
||||||
export PROVISIONING_BASE_PATH="/usr/local/provisioning"
|
export PROVISIONING_BASE_PATH="/usr/local/provisioning"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**After (Config-based)**:
|
**After (Config-based)**:
|
||||||
|
|
||||||
@ -111,7 +113,7 @@ level = "debug"
|
|||||||
|
|
||||||
[paths]
|
[paths]
|
||||||
base = "/usr/local/provisioning"
|
base = "/usr/local/provisioning"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Configuration Files
|
## Configuration Files
|
||||||
|
|
||||||
@ -193,7 +195,7 @@ rollback_enabled = true
|
|||||||
enabled = false
|
enabled = false
|
||||||
endpoint = ""
|
endpoint = ""
|
||||||
sample_rate = 0.1
|
sample_rate = 0.1
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### User Configuration (`~/.config/provisioning/config.toml`)
|
### User Configuration (`~/.config/provisioning/config.toml`)
|
||||||
|
|
||||||
@ -239,7 +241,7 @@ email = "your-email@domain.com"
|
|||||||
[git]
|
[git]
|
||||||
auto_commit = true
|
auto_commit = true
|
||||||
commit_prefix = "[{{env.USER}}]"
|
commit_prefix = "[{{env.USER}}]"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Project Configuration (`./provisioning.toml`)
|
### Project Configuration (`./provisioning.toml`)
|
||||||
|
|
||||||
@ -286,7 +288,7 @@ audit_logging = true
|
|||||||
[team]
|
[team]
|
||||||
admins = ["alice@company.com", "bob@company.com"]
|
admins = ["alice@company.com", "bob@company.com"]
|
||||||
developers = ["dev-team@company.com"]
|
developers = ["dev-team@company.com"]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Infrastructure Configuration (`./.provisioning.toml`)
|
### Infrastructure Configuration (`./.provisioning.toml`)
|
||||||
|
|
||||||
@ -334,7 +336,7 @@ alertmanager_enabled = true
|
|||||||
enabled = true
|
enabled = true
|
||||||
schedule = "0 2 * * *" # Daily at 2 AM
|
schedule = "0 2 * * *" # Daily at 2 AM
|
||||||
retention_days = 30
|
retention_days = 30
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Environment-Specific Configuration
|
## Environment-Specific Configuration
|
||||||
|
|
||||||
@ -395,7 +397,7 @@ file_watcher = true
|
|||||||
parallel_tests = true
|
parallel_tests = true
|
||||||
cleanup_after_tests = true
|
cleanup_after_tests = true
|
||||||
mock_external_apis = true
|
mock_external_apis = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Testing Environment (`config.test.toml`)
|
### Testing Environment (`config.test.toml`)
|
||||||
|
|
||||||
@ -444,7 +446,7 @@ enabled = false
|
|||||||
[validation]
|
[validation]
|
||||||
strict_mode = true
|
strict_mode = true
|
||||||
fail_fast = true
|
fail_fast = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Production Environment (`config.prod.toml`)
|
### Production Environment (`config.prod.toml`)
|
||||||
|
|
||||||
@ -503,7 +505,7 @@ max_connections = 100
|
|||||||
parallel_operations = true
|
parallel_operations = true
|
||||||
batch_operations = true
|
batch_operations = true
|
||||||
connection_pooling = true
|
connection_pooling = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## User Overrides and Customization
|
## User Overrides and Customization
|
||||||
|
|
||||||
@ -520,7 +522,7 @@ cp src/provisioning/config-examples/config.user.toml ~/.config/provisioning/conf
|
|||||||
|
|
||||||
# Customize for your environment
|
# Customize for your environment
|
||||||
$EDITOR ~/.config/provisioning/config.toml
|
$EDITOR ~/.config/provisioning/config.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Common User Customizations**:
|
**Common User Customizations**:
|
||||||
|
|
||||||
@ -553,7 +555,7 @@ dev_cluster = "cluster create development --infra {{env.USER}}-dev"
|
|||||||
desktop_notifications = true
|
desktop_notifications = true
|
||||||
sound_notifications = false
|
sound_notifications = false
|
||||||
slack_webhook = "{{env.SLACK_WEBHOOK_URL}}"
|
slack_webhook = "{{env.SLACK_WEBHOOK_URL}}"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Workspace-Specific Configuration
|
### Workspace-Specific Configuration
|
||||||
|
|
||||||
@ -580,7 +582,7 @@ shared_extensions = false
|
|||||||
[infra]
|
[infra]
|
||||||
current = "{{workspace.user}}-development"
|
current = "{{workspace.user}}-development"
|
||||||
auto_create = true
|
auto_create = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Validation and Error Handling
|
## Validation and Error Handling
|
||||||
|
|
||||||
@ -600,7 +602,7 @@ provisioning config show --validate
|
|||||||
|
|
||||||
# Debug configuration loading
|
# Debug configuration loading
|
||||||
provisioning config debug
|
provisioning config debug
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Validation Rules**:
|
**Validation Rules**:
|
||||||
|
|
||||||
@ -637,7 +639,7 @@ def validate_configuration [config: record] -> record {
|
|||||||
errors: $errors
|
errors: $errors
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Error Handling
|
### Error Handling
|
||||||
|
|
||||||
@ -669,7 +671,7 @@ def get_api_endpoint_bad [provider: string] -> string {
|
|||||||
"https://default-api.com"
|
"https://default-api.com"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Comprehensive Error Context**:
|
**Comprehensive Error Context**:
|
||||||
|
|
||||||
@ -694,7 +696,7 @@ def load_provider_config [provider: string] -> record {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Interpolation and Dynamic Values
|
## Interpolation and Dynamic Values
|
||||||
|
|
||||||
@ -724,7 +726,7 @@ version_with_git = "{{core.version}}-{{git.commit}}"
|
|||||||
hostname = "{{system.hostname}}"
|
hostname = "{{system.hostname}}"
|
||||||
platform = "{{system.platform}}"
|
platform = "{{system.platform}}"
|
||||||
architecture = "{{system.arch}}"
|
architecture = "{{system.arch}}"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Complex Interpolation Examples
|
### Complex Interpolation Examples
|
||||||
|
|
||||||
@ -741,7 +743,7 @@ runtime = "{{paths.base}}/runtime/{{git.branch}}"
|
|||||||
[providers.upcloud]
|
[providers.upcloud]
|
||||||
cache_path = "{{paths.cache}}/providers/upcloud/{{env.USER}}"
|
cache_path = "{{paths.cache}}/providers/upcloud/{{env.USER}}"
|
||||||
log_file = "{{paths.logs}}/upcloud-{{now.date}}.log"
|
log_file = "{{paths.logs}}/upcloud-{{now.date}}.log"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Environment-Aware Configuration**:
|
**Environment-Aware Configuration**:
|
||||||
|
|
||||||
@ -762,7 +764,7 @@ tags = {
|
|||||||
version = "{{core.version}}",
|
version = "{{core.version}}",
|
||||||
deployment_time = "{{now.iso8601}}"
|
deployment_time = "{{now.iso8601}}"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Interpolation Functions
|
### Interpolation Functions
|
||||||
|
|
||||||
@ -795,7 +797,7 @@ def resolve_interpolation_key [key_path: string, context: record] -> string {
|
|||||||
$path => (get_nested_config_value $path $context)
|
$path => (get_nested_config_value $path $context)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Migration Strategies
|
## Migration Strategies
|
||||||
|
|
||||||
@ -847,7 +849,7 @@ def get-config-with-env-fallback [
|
|||||||
help: $"Set ($config_key) in configuration or ($env_var) environment variable"
|
help: $"Set ($config_key) in configuration or ($env_var) environment variable"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Migration Tools
|
### Migration Tools
|
||||||
|
|
||||||
@ -862,7 +864,7 @@ nu src/tools/migration/validate-migration.nu --check-env-usage
|
|||||||
|
|
||||||
# Generate configuration from current environment
|
# Generate configuration from current environment
|
||||||
nu src/tools/migration/generate-config.nu --output-file config.migrated.toml
|
nu src/tools/migration/generate-config.nu --output-file config.migrated.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
@ -881,7 +883,7 @@ provisioning config init --template user
|
|||||||
|
|
||||||
# Verify configuration loading order
|
# Verify configuration loading order
|
||||||
provisioning config debug
|
provisioning config debug
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Invalid Configuration Syntax
|
#### Invalid Configuration Syntax
|
||||||
|
|
||||||
@ -896,7 +898,7 @@ provisioning validate config --file config.user.toml
|
|||||||
|
|
||||||
# Show parsing errors
|
# Show parsing errors
|
||||||
provisioning config check --verbose
|
provisioning config check --verbose
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Interpolation Errors
|
#### Interpolation Errors
|
||||||
|
|
||||||
@ -911,7 +913,7 @@ provisioning config interpolation --test "{{env.USER}}"
|
|||||||
|
|
||||||
# Show interpolation context
|
# Show interpolation context
|
||||||
provisioning config debug --show-interpolation
|
provisioning config debug --show-interpolation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Provider Configuration Issues
|
#### Provider Configuration Issues
|
||||||
|
|
||||||
@ -926,7 +928,7 @@ provisioning providers upcloud config --show-schema
|
|||||||
|
|
||||||
# Test provider configuration
|
# Test provider configuration
|
||||||
provisioning providers upcloud test --dry-run
|
provisioning providers upcloud test --dry-run
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Debug Commands
|
### Debug Commands
|
||||||
|
|
||||||
@ -947,7 +949,7 @@ provisioning config get paths.base --trace
|
|||||||
|
|
||||||
# Show interpolation resolution
|
# Show interpolation resolution
|
||||||
provisioning config interpolation --debug "{{paths.data}}/{{env.USER}}"
|
provisioning config interpolation --debug "{{paths.data}}/{{env.USER}}"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Performance Optimization
|
### Performance Optimization
|
||||||
|
|
||||||
@ -962,7 +964,7 @@ provisioning config cache --clear
|
|||||||
|
|
||||||
# Show cache statistics
|
# Show cache statistics
|
||||||
provisioning config cache --stats
|
provisioning config cache --stats
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Startup Optimization**:
|
**Startup Optimization**:
|
||||||
|
|
||||||
@ -976,6 +978,7 @@ skip_unused_sections = true
|
|||||||
[cache]
|
[cache]
|
||||||
config_cache_ttl = 3600
|
config_cache_ttl = 3600
|
||||||
interpolation_cache = true
|
interpolation_cache = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
This configuration management system provides a robust, flexible foundation that supports development workflows while maintaining production reliability and security requirements.
|
This configuration management system provides a robust, flexible foundation that supports development workflows while maintaining production
|
||||||
|
reliability and security requirements.
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Workspace Management Guide
|
# Workspace Management Guide
|
||||||
|
|
||||||
This document provides comprehensive guidance on setting up and using development workspaces, including the path resolution system, testing infrastructure, and workspace tools usage.
|
This document provides comprehensive guidance on setting up and using development workspaces, including the path resolution system, testing
|
||||||
|
infrastructure, and workspace tools usage.
|
||||||
|
|
||||||
## Table of Contents
|
## Table of Contents
|
||||||
|
|
||||||
@ -73,7 +74,7 @@ workspace/
|
|||||||
├── restore-workspace.nu # Restore functionality
|
├── restore-workspace.nu # Restore functionality
|
||||||
├── reset-workspace.nu # Workspace reset
|
├── reset-workspace.nu # Workspace reset
|
||||||
└── runtime-manager.nu # Runtime data management
|
└── runtime-manager.nu # Runtime data management
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Component Integration
|
### Component Integration
|
||||||
|
|
||||||
@ -105,7 +106,7 @@ nu workspace.nu init
|
|||||||
|
|
||||||
# Initialize with specific options
|
# Initialize with specific options
|
||||||
nu workspace.nu init --user-name developer --infra-name my-dev-infra
|
nu workspace.nu init --user-name developer --infra-name my-dev-infra
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Complete Initialization
|
### Complete Initialization
|
||||||
|
|
||||||
@ -118,7 +119,7 @@ nu workspace.nu init \
|
|||||||
--template full \
|
--template full \
|
||||||
--overwrite \
|
--overwrite \
|
||||||
--create-examples
|
--create-examples
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Initialization Parameters**:
|
**Initialization Parameters**:
|
||||||
|
|
||||||
@ -142,7 +143,7 @@ nu workspace.nu status --detailed
|
|||||||
|
|
||||||
# List workspace contents
|
# List workspace contents
|
||||||
nu workspace.nu list
|
nu workspace.nu list
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Configure Development Environment**:
|
**Configure Development Environment**:
|
||||||
|
|
||||||
@ -152,7 +153,7 @@ cp workspace/config/local-overrides.toml.example workspace/config/$USER.toml
|
|||||||
|
|
||||||
# Edit configuration
|
# Edit configuration
|
||||||
$EDITOR workspace/config/$USER.toml
|
$EDITOR workspace/config/$USER.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Path Resolution System
|
## Path Resolution System
|
||||||
|
|
||||||
@ -181,7 +182,7 @@ let extension_path = (path-resolver resolve_path "extensions" "custom-provider"
|
|||||||
|
|
||||||
# Create missing directories during resolution
|
# Create missing directories during resolution
|
||||||
let new_path = (path-resolver resolve_path "infra" "my-infra" --create-missing)
|
let new_path = (path-resolver resolve_path "infra" "my-infra" --create-missing)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Resolution
|
### Configuration Resolution
|
||||||
|
|
||||||
@ -196,7 +197,7 @@ let dev_config = (path-resolver resolve_config "development" --workspace-user "d
|
|||||||
|
|
||||||
# Get merged configuration with all overrides
|
# Get merged configuration with all overrides
|
||||||
let merged = (path-resolver resolve_config "merged" --workspace-user "developer" --include-overrides)
|
let merged = (path-resolver resolve_config "merged" --workspace-user "developer" --include-overrides)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Extension Discovery
|
### Extension Discovery
|
||||||
|
|
||||||
@ -211,7 +212,7 @@ let taskservs = (path-resolver list_extensions "taskservs" --include-core)
|
|||||||
|
|
||||||
# Find cluster definition
|
# Find cluster definition
|
||||||
let cluster = (path-resolver resolve_extension "clusters" "development-cluster")
|
let cluster = (path-resolver resolve_extension "clusters" "development-cluster")
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Health Checking
|
### Health Checking
|
||||||
|
|
||||||
@ -226,7 +227,7 @@ let validation = (path-resolver validate_paths --workspace-user "developer" --re
|
|||||||
|
|
||||||
# Check runtime directories
|
# Check runtime directories
|
||||||
let runtime_status = (path-resolver check_runtime_health --workspace-user "developer")
|
let runtime_status = (path-resolver check_runtime_health --workspace-user "developer")
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Configuration Management
|
## Configuration Management
|
||||||
|
|
||||||
@ -268,7 +269,7 @@ refresh_interval = 60
|
|||||||
level = "debug"
|
level = "debug"
|
||||||
file_rotation = true
|
file_rotation = true
|
||||||
max_size = "10 MB"
|
max_size = "10 MB"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Testing Environment** (`workspace/config/test-defaults.toml`):
|
**Testing Environment** (`workspace/config/test-defaults.toml`):
|
||||||
|
|
||||||
@ -295,7 +296,7 @@ mock_responses = true
|
|||||||
[logging]
|
[logging]
|
||||||
level = "info"
|
level = "info"
|
||||||
test_output = true
|
test_output = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### User Configuration Example
|
### User Configuration Example
|
||||||
|
|
||||||
@ -332,7 +333,7 @@ commit_message_template = "[${workspace.user}] ${change.type}: ${change.descript
|
|||||||
[notifications]
|
[notifications]
|
||||||
slack_webhook = "https://hooks.slack.com/..."
|
slack_webhook = "https://hooks.slack.com/..."
|
||||||
email = "developer@company.com"
|
email = "developer@company.com"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Commands
|
### Configuration Commands
|
||||||
|
|
||||||
@ -353,7 +354,7 @@ nu workspace.nu config hierarchy --user-name developer
|
|||||||
|
|
||||||
# Merge configurations for debugging
|
# Merge configurations for debugging
|
||||||
nu workspace.nu config merge --user-name developer --output merged-config.toml
|
nu workspace.nu config merge --user-name developer --output merged-config.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Extension Development
|
## Extension Development
|
||||||
|
|
||||||
@ -376,7 +377,7 @@ cp -r workspace/extensions/providers/template workspace/extensions/providers/my-
|
|||||||
# Initialize provider
|
# Initialize provider
|
||||||
cd workspace/extensions/providers/my-provider
|
cd workspace/extensions/providers/my-provider
|
||||||
nu init.nu --provider-name my-provider --author developer
|
nu init.nu --provider-name my-provider --author developer
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Provider Structure**:
|
**Provider Structure**:
|
||||||
|
|
||||||
@ -397,7 +398,7 @@ workspace/extensions/providers/my-provider/
|
|||||||
│ ├── unit/ # Unit tests
|
│ ├── unit/ # Unit tests
|
||||||
│ └── integration/ # Integration tests
|
│ └── integration/ # Integration tests
|
||||||
└── README.md
|
└── README.md
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Test Provider**:
|
**Test Provider**:
|
||||||
|
|
||||||
@ -410,7 +411,7 @@ nu workspace/extensions/providers/my-provider/nulib/provider.nu create-server --
|
|||||||
|
|
||||||
# Integration test
|
# Integration test
|
||||||
nu workspace/extensions/providers/my-provider/tests/integration/basic-test.nu
|
nu workspace/extensions/providers/my-provider/tests/integration/basic-test.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Task Service Extension Development
|
### Task Service Extension Development
|
||||||
|
|
||||||
@ -423,7 +424,7 @@ cp -r workspace/extensions/taskservs/template workspace/extensions/taskservs/my-
|
|||||||
# Initialize service
|
# Initialize service
|
||||||
cd workspace/extensions/taskservs/my-service
|
cd workspace/extensions/taskservs/my-service
|
||||||
nu init.nu --service-name my-service --service-type database
|
nu init.nu --service-name my-service --service-type database
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Task Service Structure**:
|
**Task Service Structure**:
|
||||||
|
|
||||||
@ -445,7 +446,7 @@ workspace/extensions/taskservs/my-service/
|
|||||||
└── manifests/
|
└── manifests/
|
||||||
├── deployment.yaml # Kubernetes deployment
|
├── deployment.yaml # Kubernetes deployment
|
||||||
└── service.yaml # Kubernetes service
|
└── service.yaml # Kubernetes service
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Cluster Extension Development
|
### Cluster Extension Development
|
||||||
|
|
||||||
@ -458,7 +459,7 @@ cp -r workspace/extensions/clusters/template workspace/extensions/clusters/my-cl
|
|||||||
# Initialize cluster
|
# Initialize cluster
|
||||||
cd workspace/extensions/clusters/my-cluster
|
cd workspace/extensions/clusters/my-cluster
|
||||||
nu init.nu --cluster-name my-cluster --cluster-type web-stack
|
nu init.nu --cluster-name my-cluster --cluster-type web-stack
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Testing Extensions**:
|
**Testing Extensions**:
|
||||||
|
|
||||||
@ -471,7 +472,7 @@ nu workspace.nu tools test-extension taskservs/my-service
|
|||||||
|
|
||||||
# Integration test with infrastructure
|
# Integration test with infrastructure
|
||||||
nu workspace.nu tools deploy-test clusters/my-cluster --infra test-env
|
nu workspace.nu tools deploy-test clusters/my-cluster --infra test-env
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Runtime Management
|
## Runtime Management
|
||||||
|
|
||||||
@ -509,7 +510,7 @@ runtime/
|
|||||||
│ ├── database.db # Local database
|
│ ├── database.db # Local database
|
||||||
│ └── backups/ # Local backups
|
│ └── backups/ # Local backups
|
||||||
└── tester/ # Tester's data
|
└── tester/ # Tester's data
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Runtime Management Commands
|
### Runtime Management Commands
|
||||||
|
|
||||||
@ -521,7 +522,7 @@ nu workspace/tools/runtime-manager.nu init
|
|||||||
|
|
||||||
# Initialize for specific user
|
# Initialize for specific user
|
||||||
nu workspace/tools/runtime-manager.nu init --user-name developer
|
nu workspace/tools/runtime-manager.nu init --user-name developer
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Runtime Cleanup**:
|
**Runtime Cleanup**:
|
||||||
|
|
||||||
@ -534,7 +535,7 @@ nu workspace/tools/runtime-manager.nu cleanup --type logs --rotate
|
|||||||
|
|
||||||
# Clean temporary files
|
# Clean temporary files
|
||||||
nu workspace/tools/runtime-manager.nu cleanup --type temp --force
|
nu workspace/tools/runtime-manager.nu cleanup --type temp --force
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Log Management**:
|
**Log Management**:
|
||||||
|
|
||||||
@ -550,7 +551,7 @@ nu workspace/tools/runtime-manager.nu logs --action rotate
|
|||||||
|
|
||||||
# Archive old logs
|
# Archive old logs
|
||||||
nu workspace/tools/runtime-manager.nu logs --action archive --older-than 7d
|
nu workspace/tools/runtime-manager.nu logs --action archive --older-than 7d
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Cache Management**:
|
**Cache Management**:
|
||||||
|
|
||||||
@ -566,7 +567,7 @@ nu workspace/tools/runtime-manager.nu cache --action clear --type providers
|
|||||||
|
|
||||||
# Refresh cache
|
# Refresh cache
|
||||||
nu workspace/tools/runtime-manager.nu cache --action refresh --selective
|
nu workspace/tools/runtime-manager.nu cache --action refresh --selective
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Monitoring**:
|
**Monitoring**:
|
||||||
|
|
||||||
@ -579,7 +580,7 @@ nu workspace/tools/runtime-manager.nu monitor --type disk
|
|||||||
|
|
||||||
# Monitor active processes
|
# Monitor active processes
|
||||||
nu workspace/tools/runtime-manager.nu monitor --type processes --workspace-user developer
|
nu workspace/tools/runtime-manager.nu monitor --type processes --workspace-user developer
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Health Monitoring
|
## Health Monitoring
|
||||||
|
|
||||||
@ -612,7 +613,7 @@ nu workspace.nu health --fix-issues
|
|||||||
|
|
||||||
# Export health report
|
# Export health report
|
||||||
nu workspace.nu health --report-format json > health-report.json
|
nu workspace.nu health --report-format json > health-report.json
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Component-Specific Health Checks**:
|
**Component-Specific Health Checks**:
|
||||||
|
|
||||||
@ -628,7 +629,7 @@ nu workspace/tools/workspace-health.nu check-runtime --workspace-user developer
|
|||||||
|
|
||||||
# Test extension functionality
|
# Test extension functionality
|
||||||
nu workspace/tools/workspace-health.nu check-extensions --workspace-user developer
|
nu workspace/tools/workspace-health.nu check-extensions --workspace-user developer
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Health Monitoring Output
|
### Health Monitoring Output
|
||||||
|
|
||||||
@ -674,7 +675,7 @@ nu workspace/tools/workspace-health.nu check-extensions --workspace-user develop
|
|||||||
]
|
]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Automatic Fixes
|
### Automatic Fixes
|
||||||
|
|
||||||
@ -715,7 +716,7 @@ nu workspace.nu backup --auto-name --include-logs --include-cache
|
|||||||
|
|
||||||
# Backup specific components
|
# Backup specific components
|
||||||
nu workspace.nu backup --components config,extensions --name my-backup
|
nu workspace.nu backup --components config,extensions --name my-backup
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Backup Options**:
|
**Backup Options**:
|
||||||
|
|
||||||
@ -740,7 +741,7 @@ nu workspace.nu restore --list-backups --detailed
|
|||||||
|
|
||||||
# Show backup contents
|
# Show backup contents
|
||||||
nu workspace.nu restore --show-contents --backup-name workspace-developer-20250925_143022
|
nu workspace.nu restore --show-contents --backup-name workspace-developer-20250925_143022
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Restore Operations**:
|
**Restore Operations**:
|
||||||
|
|
||||||
@ -756,7 +757,7 @@ nu workspace.nu restore --selective --backup-name my-backup
|
|||||||
|
|
||||||
# Restore to different user
|
# Restore to different user
|
||||||
nu workspace.nu restore --backup-name my-backup --restore-to different-user
|
nu workspace.nu restore --backup-name my-backup --restore-to different-user
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Advanced Restore Options**:
|
**Advanced Restore Options**:
|
||||||
|
|
||||||
@ -779,7 +780,7 @@ nu workspace.nu reset --backup-first --keep-config
|
|||||||
|
|
||||||
# Complete reset (dangerous)
|
# Complete reset (dangerous)
|
||||||
nu workspace.nu reset --force --no-backup
|
nu workspace.nu reset --force --no-backup
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Cleanup Operations**:
|
**Cleanup Operations**:
|
||||||
|
|
||||||
@ -792,7 +793,7 @@ nu workspace.nu cleanup --type cache --force
|
|||||||
|
|
||||||
# Clean specific user data
|
# Clean specific user data
|
||||||
nu workspace.nu cleanup --user-name old-user --type all
|
nu workspace.nu cleanup --user-name old-user --type all
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
@ -805,7 +806,7 @@ nu workspace.nu cleanup --user-name old-user --type all
|
|||||||
```bash
|
```bash
|
||||||
# Solution: Initialize workspace
|
# Solution: Initialize workspace
|
||||||
nu workspace.nu init --user-name developer
|
nu workspace.nu init --user-name developer
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Path Resolution Errors
|
#### Path Resolution Errors
|
||||||
|
|
||||||
@ -817,7 +818,7 @@ nu workspace.nu health --fix-issues
|
|||||||
|
|
||||||
# Manual fix
|
# Manual fix
|
||||||
nu workspace/lib/path-resolver.nu resolve_path "config" "user" --create-missing
|
nu workspace/lib/path-resolver.nu resolve_path "config" "user" --create-missing
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Configuration Errors
|
#### Configuration Errors
|
||||||
|
|
||||||
@ -829,7 +830,7 @@ nu workspace.nu config validate --user-name developer
|
|||||||
|
|
||||||
# Reset to defaults
|
# Reset to defaults
|
||||||
cp workspace/config/local-overrides.toml.example workspace/config/developer.toml
|
cp workspace/config/local-overrides.toml.example workspace/config/developer.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Runtime Issues
|
#### Runtime Issues
|
||||||
|
|
||||||
@ -841,7 +842,7 @@ nu workspace/tools/runtime-manager.nu init --user-name developer --force
|
|||||||
|
|
||||||
# Fix permissions manually
|
# Fix permissions manually
|
||||||
chmod -R 755 workspace/runtime/workspaces/developer
|
chmod -R 755 workspace/runtime/workspaces/developer
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Extension Issues
|
#### Extension Issues
|
||||||
|
|
||||||
@ -853,7 +854,7 @@ nu workspace.nu tools validate-extension providers/my-provider
|
|||||||
|
|
||||||
# Reinitialize extension from template
|
# Reinitialize extension from template
|
||||||
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider
|
cp -r workspace/extensions/providers/template workspace/extensions/providers/my-provider
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Debug Mode
|
### Debug Mode
|
||||||
|
|
||||||
@ -867,7 +868,7 @@ export PROVISIONING_WORKSPACE_USER=developer
|
|||||||
|
|
||||||
# Run with debug
|
# Run with debug
|
||||||
nu workspace.nu health --detailed
|
nu workspace.nu health --detailed
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Performance Issues
|
### Performance Issues
|
||||||
|
|
||||||
@ -883,7 +884,7 @@ du -h workspace/runtime/workspaces/developer/
|
|||||||
# Optimize workspace
|
# Optimize workspace
|
||||||
nu workspace.nu cleanup --type cache
|
nu workspace.nu cleanup --type cache
|
||||||
nu workspace/tools/runtime-manager.nu cache --action optimize
|
nu workspace/tools/runtime-manager.nu cache --action optimize
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Recovery Procedures
|
### Recovery Procedures
|
||||||
|
|
||||||
@ -901,7 +902,7 @@ nu workspace.nu restore --latest-known-good
|
|||||||
|
|
||||||
# 4. Validate health
|
# 4. Validate health
|
||||||
nu workspace.nu health --detailed --fix-issues
|
nu workspace.nu health --detailed --fix-issues
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Data Loss Prevention**:
|
**Data Loss Prevention**:
|
||||||
|
|
||||||
@ -910,4 +911,5 @@ nu workspace.nu health --detailed --fix-issues
|
|||||||
- Regular health checks: `nu workspace.nu health`
|
- Regular health checks: `nu workspace.nu health`
|
||||||
- Monitor disk space and set up alerts
|
- Monitor disk space and set up alerts
|
||||||
|
|
||||||
This workspace management system provides a robust foundation for development while maintaining isolation and providing comprehensive tools for maintenance and troubleshooting.
|
This workspace management system provides a robust foundation for development while maintaining isolation and providing comprehensive tools for
|
||||||
|
maintenance and troubleshooting.
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Distribution Process Documentation
|
# Distribution Process Documentation
|
||||||
|
|
||||||
This document provides comprehensive documentation for the provisioning project's distribution process, covering release workflows, package generation, multi-platform distribution, and rollback procedures.
|
This document provides comprehensive documentation for the provisioning project's distribution process, covering release workflows, package
|
||||||
|
generation, multi-platform distribution, and rollback procedures.
|
||||||
|
|
||||||
## Table of Contents
|
## Table of Contents
|
||||||
|
|
||||||
@ -17,7 +18,8 @@ This document provides comprehensive documentation for the provisioning project'
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
The distribution system provides a comprehensive solution for creating, packaging, and distributing provisioning across multiple platforms with automated release management.
|
The distribution system provides a comprehensive solution for creating, packaging, and distributing provisioning across multiple platforms with
|
||||||
|
automated release management.
|
||||||
|
|
||||||
**Key Features**:
|
**Key Features**:
|
||||||
|
|
||||||
@ -53,7 +55,7 @@ Distribution Ecosystem
|
|||||||
├── Checksums # SHA256/MD5 verification
|
├── Checksums # SHA256/MD5 verification
|
||||||
├── Signatures # Digital signatures
|
├── Signatures # Digital signatures
|
||||||
└── Metadata # Release information
|
└── Metadata # Release information
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Build Pipeline
|
### Build Pipeline
|
||||||
|
|
||||||
@ -77,7 +79,7 @@ Build Pipeline Flow
|
|||||||
│ - upload- │ │ package │ │ - create- │
|
│ - upload- │ │ package │ │ - create- │
|
||||||
│ artifacts │ │ - integration │ │ installers │
|
│ artifacts │ │ - integration │ │ installers │
|
||||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Distribution Variants
|
### Distribution Variants
|
||||||
|
|
||||||
@ -127,7 +129,7 @@ make docs
|
|||||||
|
|
||||||
# Validate all configurations
|
# Validate all configurations
|
||||||
make validate-all
|
make validate-all
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Version Planning**:
|
**Version Planning**:
|
||||||
|
|
||||||
@ -140,7 +142,7 @@ make status | grep Version
|
|||||||
|
|
||||||
# Validate version bump
|
# Validate version bump
|
||||||
nu src/tools/release/create-release.nu --dry-run --version 2.1.0
|
nu src/tools/release/create-release.nu --dry-run --version 2.1.0
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 2. Build Phase
|
#### 2. Build Phase
|
||||||
|
|
||||||
@ -155,7 +157,7 @@ make all
|
|||||||
|
|
||||||
# Validate build output
|
# Validate build output
|
||||||
make test-dist
|
make test-dist
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Build with Specific Parameters**:
|
**Build with Specific Parameters**:
|
||||||
|
|
||||||
@ -168,7 +170,7 @@ make all VERSION=2.1.0-rc1
|
|||||||
|
|
||||||
# Parallel build for speed
|
# Parallel build for speed
|
||||||
make all PARALLEL=true
|
make all PARALLEL=true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 3. Package Generation
|
#### 3. Package Generation
|
||||||
|
|
||||||
@ -186,7 +188,7 @@ make package-containers
|
|||||||
|
|
||||||
# Create installers
|
# Create installers
|
||||||
make create-installers
|
make create-installers
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Package Validation**:
|
**Package Validation**:
|
||||||
|
|
||||||
@ -200,7 +202,7 @@ nu src/tools/package/validate-package.nu packages/
|
|||||||
# Test installation
|
# Test installation
|
||||||
make install
|
make install
|
||||||
make uninstall
|
make uninstall
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### 4. Release Creation
|
#### 4. Release Creation
|
||||||
|
|
||||||
@ -219,7 +221,7 @@ nu src/tools/release/create-release.nu \
|
|||||||
--generate-changelog \
|
--generate-changelog \
|
||||||
--push-tag \
|
--push-tag \
|
||||||
--auto-upload
|
--auto-upload
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Release Options**:
|
**Release Options**:
|
||||||
|
|
||||||
@ -242,7 +244,7 @@ make update-registry
|
|||||||
|
|
||||||
# Send notifications
|
# Send notifications
|
||||||
make notify-release
|
make notify-release
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Registry Updates**:
|
**Registry Updates**:
|
||||||
|
|
||||||
@ -258,7 +260,7 @@ nu src/tools/release/update-registry.nu \
|
|||||||
--registries custom \
|
--registries custom \
|
||||||
--registry-url https://packages.company.com \
|
--registry-url https://packages.company.com \
|
||||||
--credentials-file ~/.registry-creds
|
--credentials-file ~/.registry-creds
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Release Automation
|
### Release Automation
|
||||||
|
|
||||||
@ -277,7 +279,7 @@ make release VERSION=2.1.0
|
|||||||
make upload-artifacts
|
make upload-artifacts
|
||||||
make update-registry
|
make update-registry
|
||||||
make notify-release
|
make notify-release
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Package Generation
|
## Package Generation
|
||||||
|
|
||||||
@ -305,7 +307,7 @@ nu src/tools/package/package-binaries.nu \
|
|||||||
--compress \
|
--compress \
|
||||||
--strip \
|
--strip \
|
||||||
--checksum
|
--checksum
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Package Features**:
|
**Package Features**:
|
||||||
|
|
||||||
@ -331,7 +333,7 @@ nu src/tools/package/build-containers.nu \
|
|||||||
--optimize-size \
|
--optimize-size \
|
||||||
--security-scan \
|
--security-scan \
|
||||||
--multi-stage
|
--multi-stage
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Container Features**:
|
**Container Features**:
|
||||||
|
|
||||||
@ -374,7 +376,7 @@ nu src/tools/distribution/create-installer.nu \
|
|||||||
--include-services \
|
--include-services \
|
||||||
--create-uninstaller \
|
--create-uninstaller \
|
||||||
--validate-installer
|
--validate-installer
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Installer Features**:
|
**Installer Features**:
|
||||||
|
|
||||||
@ -418,7 +420,7 @@ rustup target add x86_64-pc-windows-gnu
|
|||||||
|
|
||||||
# Install cross-compilation tools
|
# Install cross-compilation tools
|
||||||
cargo install cross
|
cargo install cross
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Platform-Specific Builds**:
|
**Platform-Specific Builds**:
|
||||||
|
|
||||||
@ -433,7 +435,7 @@ make build-cross PLATFORMS=linux-amd64,macos-arm64,windows-amd64
|
|||||||
make linux
|
make linux
|
||||||
make macos
|
make macos
|
||||||
make windows
|
make windows
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Distribution Matrix
|
### Distribution Matrix
|
||||||
|
|
||||||
@ -448,7 +450,7 @@ Examples:
|
|||||||
- provisioning-2.1.0-macos-arm64-minimal.tar.gz
|
- provisioning-2.1.0-macos-arm64-minimal.tar.gz
|
||||||
- provisioning-2.1.0-windows-amd64-complete.zip
|
- provisioning-2.1.0-windows-amd64-complete.zip
|
||||||
- provisioning-2.1.0-freebsd-amd64-minimal.tar.xz
|
- provisioning-2.1.0-freebsd-amd64-minimal.tar.xz
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Platform Considerations**:
|
**Platform Considerations**:
|
||||||
|
|
||||||
@ -475,7 +477,7 @@ nu src/tools/build/test-distribution.nu \
|
|||||||
--platform linux \
|
--platform linux \
|
||||||
--cleanup \
|
--cleanup \
|
||||||
--verbose
|
--verbose
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Validation Types**:
|
**Validation Types**:
|
||||||
|
|
||||||
@ -503,7 +505,7 @@ make ci-test
|
|||||||
nu src/tools/build/test-distribution.nu --test-types basic
|
nu src/tools/build/test-distribution.nu --test-types basic
|
||||||
nu src/tools/build/test-distribution.nu --test-types integration
|
nu src/tools/build/test-distribution.nu --test-types integration
|
||||||
nu src/tools/build/test-distribution.nu --test-types complete
|
nu src/tools/build/test-distribution.nu --test-types complete
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Package Validation
|
### Package Validation
|
||||||
|
|
||||||
@ -518,7 +520,7 @@ sha256sum -c packages/checksums.sha256
|
|||||||
|
|
||||||
# Verify signatures
|
# Verify signatures
|
||||||
gpg --verify packages/provisioning-2.1.0.tar.gz.sig
|
gpg --verify packages/provisioning-2.1.0.tar.gz.sig
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Installation Testing**:
|
**Installation Testing**:
|
||||||
|
|
||||||
@ -531,7 +533,7 @@ gpg --verify packages/provisioning-2.1.0.tar.gz.sig
|
|||||||
|
|
||||||
# Container testing
|
# Container testing
|
||||||
docker run --rm provisioning:2.1.0 provisioning --version
|
docker run --rm provisioning:2.1.0 provisioning --version
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Release Management
|
## Release Management
|
||||||
|
|
||||||
@ -547,7 +549,7 @@ nu src/tools/release/create-release.nu \
|
|||||||
--generate-changelog \
|
--generate-changelog \
|
||||||
--push-tag \
|
--push-tag \
|
||||||
--auto-upload
|
--auto-upload
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Release Features**:
|
**Release Features**:
|
||||||
|
|
||||||
@ -575,7 +577,7 @@ nu src/tools/release/create-release.nu --version 2.1.0
|
|||||||
|
|
||||||
# Pre-release versioning
|
# Pre-release versioning
|
||||||
nu src/tools/release/create-release.nu --version 2.1.0-rc.1 --pre-release
|
nu src/tools/release/create-release.nu --version 2.1.0-rc.1 --pre-release
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Artifact Management
|
### Artifact Management
|
||||||
|
|
||||||
@ -598,7 +600,7 @@ docker push provisioning:2.1.0
|
|||||||
|
|
||||||
# Update package repositories
|
# Update package repositories
|
||||||
make update-registry
|
make update-registry
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Rollback Procedures
|
## Rollback Procedures
|
||||||
|
|
||||||
@ -626,7 +628,7 @@ nu src/tools/release/rollback-release.nu \
|
|||||||
--to-version 2.0.5 \
|
--to-version 2.0.5 \
|
||||||
--update-registries \
|
--update-registries \
|
||||||
--notify-users
|
--notify-users
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Manual Rollback Steps**:
|
**Manual Rollback Steps**:
|
||||||
|
|
||||||
@ -650,7 +652,7 @@ nu src/tools/release/notify-users.nu \
|
|||||||
--channels slack,discord,email \
|
--channels slack,discord,email \
|
||||||
--message-type rollback \
|
--message-type rollback \
|
||||||
--urgent
|
--urgent
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Rollback Safety
|
### Rollback Safety
|
||||||
|
|
||||||
@ -673,7 +675,7 @@ nu src/tools/release/rollback-release.nu \
|
|||||||
|
|
||||||
# Validate rollback success
|
# Validate rollback success
|
||||||
make test-dist DIST_VERSION=2.0.5
|
make test-dist DIST_VERSION=2.0.5
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Emergency Procedures
|
### Emergency Procedures
|
||||||
|
|
||||||
@ -686,7 +688,7 @@ nu src/tools/release/rollback-release.nu \
|
|||||||
--emergency \
|
--emergency \
|
||||||
--security-issue \
|
--security-issue \
|
||||||
--immediate-notify
|
--immediate-notify
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Infrastructure Failure Recovery**:
|
**Infrastructure Failure Recovery**:
|
||||||
|
|
||||||
@ -696,7 +698,7 @@ nu src/tools/release/rollback-release.nu \
|
|||||||
--infrastructure-failover \
|
--infrastructure-failover \
|
||||||
--backup-registry \
|
--backup-registry \
|
||||||
--mirror-sync
|
--mirror-sync
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## CI/CD Integration
|
## CI/CD Integration
|
||||||
|
|
||||||
@ -739,7 +741,7 @@ jobs:
|
|||||||
with:
|
with:
|
||||||
name: build-${{ matrix.platform }}
|
name: build-${{ matrix.platform }}
|
||||||
path: src/dist/
|
path: src/dist/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Release Workflow** (`.github/workflows/release.yml`):
|
**Release Workflow** (`.github/workflows/release.yml`):
|
||||||
|
|
||||||
@ -769,7 +771,7 @@ jobs:
|
|||||||
run: |
|
run: |
|
||||||
cd src/tools
|
cd src/tools
|
||||||
make update-registry VERSION=${{ github.ref_name }}
|
make update-registry VERSION=${{ github.ref_name }}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### GitLab CI Integration
|
### GitLab CI Integration
|
||||||
|
|
||||||
@ -809,7 +811,7 @@ release:
|
|||||||
- make cd-deploy VERSION=${CI_COMMIT_TAG}
|
- make cd-deploy VERSION=${CI_COMMIT_TAG}
|
||||||
only:
|
only:
|
||||||
- tags
|
- tags
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Jenkins Integration
|
### Jenkins Integration
|
||||||
|
|
||||||
@ -848,7 +850,7 @@ pipeline {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
@ -867,7 +869,7 @@ make build-platform
|
|||||||
# Check Rust toolchain
|
# Check Rust toolchain
|
||||||
rustup show
|
rustup show
|
||||||
rustup update
|
rustup update
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Cross-Compilation Issues**:
|
**Cross-Compilation Issues**:
|
||||||
|
|
||||||
@ -879,7 +881,7 @@ rustup target add x86_64-apple-darwin
|
|||||||
# Use cross for problematic targets
|
# Use cross for problematic targets
|
||||||
cargo install cross
|
cargo install cross
|
||||||
make build-platform CROSS=true
|
make build-platform CROSS=true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Package Generation Issues
|
#### Package Generation Issues
|
||||||
|
|
||||||
@ -892,7 +894,7 @@ brew install gnu-tar
|
|||||||
|
|
||||||
# Check tool availability
|
# Check tool availability
|
||||||
make info
|
make info
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Permission Errors**:
|
**Permission Errors**:
|
||||||
|
|
||||||
@ -901,7 +903,7 @@ make info
|
|||||||
chmod +x src/tools/build/*.nu
|
chmod +x src/tools/build/*.nu
|
||||||
chmod +x src/tools/distribution/*.nu
|
chmod +x src/tools/distribution/*.nu
|
||||||
chmod +x src/tools/package/*.nu
|
chmod +x src/tools/package/*.nu
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Distribution Validation Failures
|
#### Distribution Validation Failures
|
||||||
|
|
||||||
@ -914,7 +916,7 @@ make package-all
|
|||||||
|
|
||||||
# Verify manually
|
# Verify manually
|
||||||
sha256sum packages/*.tar.gz
|
sha256sum packages/*.tar.gz
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Installation Test Failures**:
|
**Installation Test Failures**:
|
||||||
|
|
||||||
@ -924,7 +926,7 @@ docker run --rm -v $(pwd):/work ubuntu:latest /work/packages/installers/install.
|
|||||||
|
|
||||||
# Debug installation
|
# Debug installation
|
||||||
./packages/installers/install.sh --dry-run --verbose
|
./packages/installers/install.sh --dry-run --verbose
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Release Issues
|
### Release Issues
|
||||||
|
|
||||||
@ -940,7 +942,7 @@ nu src/tools/release/upload-artifacts.nu \
|
|||||||
|
|
||||||
# Manual upload
|
# Manual upload
|
||||||
gh release upload v2.1.0 packages/*.tar.gz
|
gh release upload v2.1.0 packages/*.tar.gz
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Authentication Failures**:
|
**Authentication Failures**:
|
||||||
|
|
||||||
@ -952,7 +954,7 @@ docker login ghcr.io
|
|||||||
# Check credentials
|
# Check credentials
|
||||||
gh auth status
|
gh auth status
|
||||||
docker system info
|
docker system info
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Registry Update Issues
|
#### Registry Update Issues
|
||||||
|
|
||||||
@ -965,7 +967,7 @@ cd homebrew-core
|
|||||||
# Edit formula
|
# Edit formula
|
||||||
git add Formula/provisioning.rb
|
git add Formula/provisioning.rb
|
||||||
git commit -m "provisioning 2.1.0"
|
git commit -m "provisioning 2.1.0"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Debug and Monitoring
|
### Debug and Monitoring
|
||||||
|
|
||||||
@ -983,7 +985,7 @@ make all VERBOSE=true
|
|||||||
nu src/tools/distribution/generate-distribution.nu \
|
nu src/tools/distribution/generate-distribution.nu \
|
||||||
--verbose \
|
--verbose \
|
||||||
--dry-run
|
--dry-run
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Monitoring Build Progress**:
|
**Monitoring Build Progress**:
|
||||||
|
|
||||||
@ -997,6 +999,7 @@ make status
|
|||||||
# Resource monitoring
|
# Resource monitoring
|
||||||
top
|
top
|
||||||
df -h
|
df -h
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
This distribution process provides a robust, automated pipeline for creating, validating, and distributing provisioning across multiple platforms while maintaining high quality and reliability standards.
|
This distribution process provides a robust, automated pipeline for creating, validating, and distributing provisioning across multiple platforms
|
||||||
|
while maintaining high quality and reliability standards.
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Examples and Tutorials
|
# Examples and Tutorials
|
||||||
|
|
||||||
This directory contains comprehensive examples and step-by-step tutorials for using Infrastructure Automation. Each example includes complete configurations, scripts, and explanations.
|
This directory contains comprehensive examples and step-by-step tutorials for using Infrastructure Automation. Each example includes complete
|
||||||
|
configurations, scripts, and explanations.
|
||||||
|
|
||||||
## Available Examples
|
## Available Examples
|
||||||
|
|
||||||
@ -81,7 +82,7 @@ cat README.md
|
|||||||
|
|
||||||
# Check requirements
|
# Check requirements
|
||||||
./check-requirements.sh
|
./check-requirements.sh
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Customize Configuration
|
### 3. Customize Configuration
|
||||||
|
|
||||||
@ -93,7 +94,7 @@ nano my-settings.ncl
|
|||||||
# Update configuration for your environment
|
# Update configuration for your environment
|
||||||
cp config.toml my-config.toml
|
cp config.toml my-config.toml
|
||||||
nano my-config.toml
|
nano my-config.toml
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Deploy
|
### 4. Deploy
|
||||||
|
|
||||||
@ -106,7 +107,7 @@ provisioning validate config --settings my-settings.ncl
|
|||||||
|
|
||||||
# Deploy for real
|
# Deploy for real
|
||||||
./deploy.sh
|
./deploy.sh
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 5. Verify and Test
|
### 5. Verify and Test
|
||||||
|
|
||||||
@ -116,14 +117,14 @@ provisioning validate config --settings my-settings.ncl
|
|||||||
|
|
||||||
# Manual testing
|
# Manual testing
|
||||||
provisioning show servers --infra my-example
|
provisioning show servers --infra my-example
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 6. Clean Up
|
### 6. Clean Up
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# When done, clean up resources
|
# When done, clean up resources
|
||||||
./cleanup.sh
|
./cleanup.sh
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Example Categories
|
## Example Categories
|
||||||
|
|
||||||
@ -210,7 +211,7 @@ provisioning validate config
|
|||||||
|
|
||||||
# Test provider connectivity
|
# Test provider connectivity
|
||||||
provisioning provider test local
|
provisioning provider test local
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Contributing Examples
|
## Contributing Examples
|
||||||
|
|
||||||
@ -230,7 +231,7 @@ example-name/
|
|||||||
└── docs/ # Additional documentation
|
└── docs/ # Additional documentation
|
||||||
├── architecture.md # Architecture explanation
|
├── architecture.md # Architecture explanation
|
||||||
└── troubleshooting.md # Common issues
|
└── troubleshooting.md # Common issues
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Example Template
|
### Example Template
|
||||||
|
|
||||||
@ -266,7 +267,7 @@ Brief description of what this example demonstrates.
|
|||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
[Common issues and solutions]
|
[Common issues and solutions]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Submission Process
|
### Submission Process
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Quick Start: Local Development Setup
|
# Quick Start: Local Development Setup
|
||||||
|
|
||||||
This example demonstrates how to set up a basic local development environment using provisioning. Perfect for learning the fundamentals without requiring cloud resources.
|
This example demonstrates how to set up a basic local development environment using provisioning. Perfect for learning the fundamentals without
|
||||||
|
requiring cloud resources.
|
||||||
|
|
||||||
## What You'll Learn
|
## What You'll Learn
|
||||||
|
|
||||||
@ -40,7 +41,7 @@ This example creates:
|
|||||||
│ └──────────────┘ │
|
│ └──────────────┘ │
|
||||||
│ Local Network │
|
│ Local Network │
|
||||||
└─────────────────────────────────────┘
|
└─────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Files in This Example
|
## Files in This Example
|
||||||
|
|
||||||
@ -64,7 +65,7 @@ This example creates:
|
|||||||
# ✅ Local provider configured
|
# ✅ Local provider configured
|
||||||
# ✅ Sufficient disk space
|
# ✅ Sufficient disk space
|
||||||
# ✅ Docker available (if using containers)
|
# ✅ Docker available (if using containers)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 2: Review Configuration
|
### Step 2: Review Configuration
|
||||||
|
|
||||||
@ -77,7 +78,7 @@ cat config.toml
|
|||||||
|
|
||||||
# Understand what will be created
|
# Understand what will be created
|
||||||
provisioning show settings --settings settings.ncl
|
provisioning show settings --settings settings.ncl
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 3: Validate Configuration
|
### Step 3: Validate Configuration
|
||||||
|
|
||||||
@ -87,7 +88,7 @@ provisioning validate config --settings settings.ncl
|
|||||||
|
|
||||||
# Should show:
|
# Should show:
|
||||||
# ✅ Configuration validation passed!
|
# ✅ Configuration validation passed!
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 4: Deploy (Dry Run)
|
### Step 4: Deploy (Dry Run)
|
||||||
|
|
||||||
@ -99,7 +100,7 @@ provisioning validate config --settings settings.ncl
|
|||||||
provisioning server create --settings settings.ncl --check
|
provisioning server create --settings settings.ncl --check
|
||||||
|
|
||||||
# Review the planned changes
|
# Review the planned changes
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 5: Deploy for Real
|
### Step 5: Deploy for Real
|
||||||
|
|
||||||
@ -111,7 +112,7 @@ provisioning server create --settings settings.ncl --check
|
|||||||
provisioning server list --settings settings.ncl
|
provisioning server list --settings settings.ncl
|
||||||
|
|
||||||
# Wait for servers to be ready (should be quick for local)
|
# Wait for servers to be ready (should be quick for local)
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 6: Install Services
|
### Step 6: Install Services
|
||||||
|
|
||||||
@ -123,7 +124,7 @@ provisioning taskserv create postgresql --servers db-dev-01 --settings settings.
|
|||||||
|
|
||||||
# Verify installations
|
# Verify installations
|
||||||
provisioning taskserv list --settings settings.ncl --installed
|
provisioning taskserv list --settings settings.ncl --installed
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 7: Verify Deployment
|
### Step 7: Verify Deployment
|
||||||
|
|
||||||
@ -134,7 +135,7 @@ provisioning taskserv list --settings settings.ncl --installed
|
|||||||
# Manual verification
|
# Manual verification
|
||||||
provisioning server ssh web-dev-01 --settings settings.ncl --command "curl -I localhost"
|
provisioning server ssh web-dev-01 --settings settings.ncl --command "curl -I localhost"
|
||||||
provisioning server ssh db-dev-01 --settings settings.ncl --command "pg_isready"
|
provisioning server ssh db-dev-01 --settings settings.ncl --command "pg_isready"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Working with the Development Environment
|
## Working with the Development Environment
|
||||||
|
|
||||||
@ -149,7 +150,7 @@ provisioning server ssh db-dev-01 --settings settings.ncl
|
|||||||
|
|
||||||
# Execute commands remotely
|
# Execute commands remotely
|
||||||
provisioning server ssh web-dev-01 --settings settings.ncl --command "systemctl status nginx"
|
provisioning server ssh web-dev-01 --settings settings.ncl --command "systemctl status nginx"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Deploying Code
|
### Deploying Code
|
||||||
|
|
||||||
@ -159,7 +160,7 @@ provisioning server copy ./my-app/ web-dev-01:/var/www/html/ --settings settings
|
|||||||
|
|
||||||
# Restart services
|
# Restart services
|
||||||
provisioning server ssh web-dev-01 --settings settings.ncl --command "sudo systemctl restart nginx"
|
provisioning server ssh web-dev-01 --settings settings.ncl --command "sudo systemctl restart nginx"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Database Operations
|
### Database Operations
|
||||||
|
|
||||||
@ -169,7 +170,7 @@ provisioning server ssh db-dev-01 --settings settings.ncl --command "psql -U pos
|
|||||||
|
|
||||||
# Create development database
|
# Create development database
|
||||||
provisioning server ssh db-dev-01 --settings settings.ncl --command "createdb myapp_development"
|
provisioning server ssh db-dev-01 --settings settings.ncl --command "createdb myapp_development"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Monitoring and Logs
|
### Monitoring and Logs
|
||||||
|
|
||||||
@ -183,7 +184,7 @@ provisioning taskserv logs postgresql --settings settings.ncl
|
|||||||
|
|
||||||
# Check resource usage
|
# Check resource usage
|
||||||
provisioning show costs --settings settings.ncl
|
provisioning show costs --settings settings.ncl
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Configuration Details
|
## Configuration Details
|
||||||
|
|
||||||
@ -293,7 +294,7 @@ taskservs: {
|
|||||||
maxmemory = "128 MB"
|
maxmemory = "128 MB"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Environment Configuration (config.toml)
|
### Environment Configuration (config.toml)
|
||||||
|
|
||||||
@ -342,7 +343,7 @@ max_disk = "50 GB"
|
|||||||
[notifications]
|
[notifications]
|
||||||
# Disable notifications for local dev
|
# Disable notifications for local dev
|
||||||
enabled = false
|
enabled = false
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
@ -357,7 +358,7 @@ systemctl status docker
|
|||||||
|
|
||||||
# Configure local provider
|
# Configure local provider
|
||||||
provisioning provider test local
|
provisioning provider test local
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Issue: Port Conflicts
|
#### Issue: Port Conflicts
|
||||||
|
|
||||||
@ -367,7 +368,7 @@ sudo netstat -tulpn | grep :80
|
|||||||
sudo ss -tulpn | grep :80
|
sudo ss -tulpn | grep :80
|
||||||
|
|
||||||
# Change ports in settings.ncl if needed
|
# Change ports in settings.ncl if needed
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Issue: Insufficient Resources
|
#### Issue: Insufficient Resources
|
||||||
|
|
||||||
@ -378,7 +379,7 @@ df -h
|
|||||||
|
|
||||||
# Reduce server resources in settings.ncl
|
# Reduce server resources in settings.ncl
|
||||||
plan = "minimal" # Instead of "development"
|
plan = "minimal" # Instead of "development"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Issue: Permission Errors
|
#### Issue: Permission Errors
|
||||||
|
|
||||||
@ -388,7 +389,7 @@ sudo usermod -aG docker $USER
|
|||||||
newgrp docker
|
newgrp docker
|
||||||
|
|
||||||
# Or use podman instead
|
# Or use podman instead
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Getting Help
|
### Getting Help
|
||||||
|
|
||||||
@ -404,7 +405,7 @@ provisioning show logs --settings settings.ncl
|
|||||||
|
|
||||||
# Validate configuration
|
# Validate configuration
|
||||||
provisioning validate config --settings settings.ncl --detailed
|
provisioning validate config --settings settings.ncl --detailed
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Cleanup
|
## Cleanup
|
||||||
|
|
||||||
@ -420,7 +421,7 @@ provisioning taskserv delete --all --settings settings.ncl --yes
|
|||||||
|
|
||||||
# Verify cleanup
|
# Verify cleanup
|
||||||
provisioning server list --settings settings.ncl
|
provisioning server list --settings settings.ncl
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Next Steps
|
## Next Steps
|
||||||
|
|
||||||
|
|||||||
@ -17,7 +17,7 @@ This guide will help you create custom providers, task services, and cluster con
|
|||||||
### Extension Types
|
### Extension Types
|
||||||
|
|
||||||
| Extension Type | Purpose | Examples |
|
| Extension Type | Purpose | Examples |
|
||||||
|----------------|---------|----------|
|
| ---------------- | --------- | ---------- |
|
||||||
| **Providers** | Cloud platform integrations | Custom cloud, on-premises |
|
| **Providers** | Cloud platform integrations | Custom cloud, on-premises |
|
||||||
| **Task Services** | Software components | Custom databases, monitoring |
|
| **Task Services** | Software components | Custom databases, monitoring |
|
||||||
| **Clusters** | Service orchestration | Application stacks, platforms |
|
| **Clusters** | Service orchestration | Application stacks, platforms |
|
||||||
@ -41,7 +41,7 @@ my-extension/
|
|||||||
├── docs/ # Documentation
|
├── docs/ # Documentation
|
||||||
├── extension.toml # Extension metadata
|
├── extension.toml # Extension metadata
|
||||||
└── README.md # Extension documentation
|
└── README.md # Extension documentation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Extension Metadata
|
### Extension Metadata
|
||||||
|
|
||||||
@ -71,7 +71,7 @@ system_packages = ["curl", "jq"]
|
|||||||
[configuration]
|
[configuration]
|
||||||
required_env = ["CUSTOM_CLOUD_API_KEY"]
|
required_env = ["CUSTOM_CLOUD_API_KEY"]
|
||||||
optional_env = ["CUSTOM_CLOUD_REGION"]
|
optional_env = ["CUSTOM_CLOUD_REGION"]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Creating Custom Providers
|
## Creating Custom Providers
|
||||||
|
|
||||||
@ -143,7 +143,7 @@ A provider handles:
|
|||||||
],
|
],
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 2: Implement Provider Logic
|
### Step 2: Implement Provider Logic
|
||||||
|
|
||||||
@ -343,7 +343,7 @@ def custom_cloud_wait_for_server [
|
|||||||
print $"Waiting for server status: ($current_status) -> ($target_status)"
|
print $"Waiting for server status: ($current_status) -> ($target_status)"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 3: Provider Registration
|
### Step 3: Provider Registration
|
||||||
|
|
||||||
@ -368,7 +368,7 @@ export def get_provider_info [] -> record {
|
|||||||
auth_methods: ["api_key", "oauth"]
|
auth_methods: ["api_key", "oauth"]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Creating Custom Task Services
|
## Creating Custom Task Services
|
||||||
|
|
||||||
@ -442,7 +442,7 @@ Task services handle:
|
|||||||
data_directories = ["/var/lib/customdb"],
|
data_directories = ["/var/lib/customdb"],
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 2: Implement Service Logic
|
### Step 2: Implement Service Logic
|
||||||
|
|
||||||
@ -793,7 +793,7 @@ def check_port [port: int] -> bool {
|
|||||||
let result = (^nc -z localhost $port | complete)
|
let result = (^nc -z localhost $port | complete)
|
||||||
return ($result.exit_code == 0)
|
return ($result.exit_code == 0)
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Creating Custom Clusters
|
## Creating Custom Clusters
|
||||||
|
|
||||||
@ -901,7 +901,7 @@ Clusters orchestrate multiple services to work together as a cohesive applicatio
|
|||||||
],
|
],
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Step 2: Implement Cluster Logic
|
### Step 2: Implement Cluster Logic
|
||||||
|
|
||||||
@ -1171,7 +1171,7 @@ def calculate_cluster_cost [config: record] -> float {
|
|||||||
|
|
||||||
return ($web_cost + $app_cost + $db_cost + $lb_cost)
|
return ($web_cost + $app_cost + $db_cost + $lb_cost)
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Extension Testing
|
## Extension Testing
|
||||||
|
|
||||||
@ -1192,7 +1192,7 @@ tests/
|
|||||||
└── fixtures/ # Test data
|
└── fixtures/ # Test data
|
||||||
├── configs/
|
├── configs/
|
||||||
└── mocks/
|
└── mocks/
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Example Unit Test
|
### Example Unit Test
|
||||||
|
|
||||||
@ -1248,7 +1248,7 @@ export def test_api_call_formatting [] {
|
|||||||
assert equal $api_payload.machine_type "small"
|
assert equal $api_payload.machine_type "small"
|
||||||
assert equal $api_payload.zone "us-west-1a"
|
assert equal $api_payload.zone "us-west-1a"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Integration Test
|
### Integration Test
|
||||||
|
|
||||||
@ -1286,7 +1286,7 @@ export def test_server_listing [] {
|
|||||||
assert ($servers | is-not-empty)
|
assert ($servers | is-not-empty)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Publishing Extensions
|
## Publishing Extensions
|
||||||
|
|
||||||
@ -1304,7 +1304,7 @@ my-extension-package/
|
|||||||
│ ├── nulib/
|
│ ├── nulib/
|
||||||
│ └── templates/
|
│ └── templates/
|
||||||
└── tests/ # Test files
|
└── tests/ # Test files
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Publishing Configuration
|
### Publishing Configuration
|
||||||
|
|
||||||
@ -1338,7 +1338,7 @@ extensions = []
|
|||||||
[build]
|
[build]
|
||||||
include = ["src/**", "examples/**", "README.md", "LICENSE"]
|
include = ["src/**", "examples/**", "README.md", "LICENSE"]
|
||||||
exclude = ["tests/**", ".git/**", "*.tmp"]
|
exclude = ["tests/**", ".git/**", "*.tmp"]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Publishing Process
|
### Publishing Process
|
||||||
|
|
||||||
@ -1354,7 +1354,7 @@ provisioning extension build .
|
|||||||
|
|
||||||
# 4. Publish to registry
|
# 4. Publish to registry
|
||||||
provisioning extension publish ./dist/my-custom-provider-1.0.0.tar.gz
|
provisioning extension publish ./dist/my-custom-provider-1.0.0.tar.gz
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Best Practices
|
## Best Practices
|
||||||
|
|
||||||
@ -1368,7 +1368,7 @@ extension/
|
|||||||
├── templates/ # Configuration templates
|
├── templates/ # Configuration templates
|
||||||
├── tests/ # Comprehensive tests
|
├── tests/ # Comprehensive tests
|
||||||
└── docs/ # Documentation
|
└── docs/ # Documentation
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 2. Error Handling
|
### 2. Error Handling
|
||||||
|
|
||||||
@ -1384,7 +1384,7 @@ if ($api_response | get -o status | default "" | str contains "error") {
|
|||||||
help: "Check your API key and network connectivity"
|
help: "Check your API key and network connectivity"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 3. Configuration Validation
|
### 3. Configuration Validation
|
||||||
|
|
||||||
@ -1406,7 +1406,7 @@ if ($api_response | get -o status | default "" | str contains "error") {
|
|||||||
else
|
else
|
||||||
(std.fail "Configuration validation failed"),
|
(std.fail "Configuration validation failed"),
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### 4. Testing
|
### 4. Testing
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Extension Registry Service
|
# Extension Registry Service
|
||||||
|
|
||||||
A high-performance Rust microservice that provides a unified REST API for extension discovery, versioning, and download from multiple Git-based sources and OCI registries.
|
A high-performance Rust microservice that provides a unified REST API for extension discovery, versioning, and download from multiple Git-based
|
||||||
|
sources and OCI registries.
|
||||||
|
|
||||||
> **Source**: `provisioning/platform/crates/extension-registry/`
|
> **Source**: `provisioning/platform/crates/extension-registry/`
|
||||||
|
|
||||||
@ -47,7 +48,7 @@ The extension registry uses a trait-based architecture separating source and dis
|
|||||||
│ └───────────────────────────────────────────────────────────────┘ │
|
│ └───────────────────────────────────────────────────────────────┘ │
|
||||||
│ │
|
│ │
|
||||||
└────────────────────────────────────────────────────────────────────┘
|
└────────────────────────────────────────────────────────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Request Strategies
|
### Request Strategies
|
||||||
|
|
||||||
@ -71,7 +72,7 @@ The extension registry uses a trait-based architecture separating source and dis
|
|||||||
```bash
|
```bash
|
||||||
cd provisioning/platform/extension-registry
|
cd provisioning/platform/extension-registry
|
||||||
cargo build --release
|
cargo build --release
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@ -99,7 +100,7 @@ auth_token_path = "/path/to/oci-token.txt"
|
|||||||
[cache]
|
[cache]
|
||||||
capacity = 1000
|
capacity = 1000
|
||||||
ttl_seconds = 300
|
ttl_seconds = 300
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Multi-Instance Configuration (Recommended)
|
### Multi-Instance Configuration (Recommended)
|
||||||
|
|
||||||
@ -185,7 +186,7 @@ capacity = 1000
|
|||||||
ttl_seconds = 300
|
ttl_seconds = 300
|
||||||
enable_metadata_cache = true
|
enable_metadata_cache = true
|
||||||
enable_list_cache = true
|
enable_list_cache = true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Notes
|
### Configuration Notes
|
||||||
|
|
||||||
@ -211,7 +212,7 @@ REGISTRY_OCI_REGISTRY=registry.example.com
|
|||||||
REGISTRY_OCI_NAMESPACE=extensions
|
REGISTRY_OCI_NAMESPACE=extensions
|
||||||
REGISTRY_CACHE_CAPACITY=2000
|
REGISTRY_CACHE_CAPACITY=2000
|
||||||
REGISTRY_CACHE_TTL=600
|
REGISTRY_CACHE_TTL=600
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## API Endpoints
|
## API Endpoints
|
||||||
|
|
||||||
@ -221,31 +222,31 @@ REGISTRY_CACHE_TTL=600
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
GET /api/v1/extensions?type=provider&limit=10
|
GET /api/v1/extensions?type=provider&limit=10
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Get Extension
|
#### Get Extension
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
GET /api/v1/extensions/{type}/{name}
|
GET /api/v1/extensions/{type}/{name}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### List Versions
|
#### List Versions
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
GET /api/v1/extensions/{type}/{name}/versions
|
GET /api/v1/extensions/{type}/{name}/versions
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Download Extension
|
#### Download Extension
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
GET /api/v1/extensions/{type}/{name}/{version}
|
GET /api/v1/extensions/{type}/{name}/{version}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Search Extensions
|
#### Search Extensions
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
GET /api/v1/extensions/search?q=kubernetes&type=taskserv
|
GET /api/v1/extensions/search?q=kubernetes&type=taskserv
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### System Endpoints
|
### System Endpoints
|
||||||
|
|
||||||
@ -253,7 +254,7 @@ GET /api/v1/extensions/search?q=kubernetes&type=taskserv
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
GET /api/v1/health
|
GET /api/v1/health
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response** (with multi-backend aggregation):
|
**Response** (with multi-backend aggregation):
|
||||||
|
|
||||||
@ -275,7 +276,7 @@ GET /api/v1/health
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Status Values**:
|
**Status Values**:
|
||||||
- `healthy`: All configured backends are healthy
|
- `healthy`: All configured backends are healthy
|
||||||
@ -286,13 +287,13 @@ GET /api/v1/health
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
GET /api/v1/metrics
|
GET /api/v1/metrics
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Cache Statistics
|
#### Cache Statistics
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
GET /api/v1/cache/stats
|
GET /api/v1/cache/stats
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Response**:
|
**Response**:
|
||||||
|
|
||||||
@ -306,7 +307,7 @@ GET /api/v1/cache/stats
|
|||||||
"version_misses": 512,
|
"version_misses": 512,
|
||||||
"size": 4096
|
"size": 4096
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Extension Naming Conventions
|
## Extension Naming Conventions
|
||||||
|
|
||||||
@ -329,7 +330,7 @@ GET /api/v1/cache/stats
|
|||||||
```bash
|
```bash
|
||||||
docker build -t extension-registry:latest .
|
docker build -t extension-registry:latest .
|
||||||
docker run -d -p 8082:8082 -v $(pwd)/config.toml:/app/config.toml:ro extension-registry:latest
|
docker run -d -p 8082:8082 -v $(pwd)/config.toml:/app/config.toml:ro extension-registry:latest
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Kubernetes
|
### Kubernetes
|
||||||
|
|
||||||
@ -347,7 +348,7 @@ spec:
|
|||||||
image: extension-registry:latest
|
image: extension-registry:latest
|
||||||
ports:
|
ports:
|
||||||
- containerPort: 8082
|
- containerPort: 8082
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Migration Guide: Single to Multi-Instance
|
## Migration Guide: Single to Multi-Instance
|
||||||
|
|
||||||
@ -371,7 +372,7 @@ token_path = "/path/to/token"
|
|||||||
[oci]
|
[oci]
|
||||||
registry = "registry.example.com"
|
registry = "registry.example.com"
|
||||||
namespace = "extensions"
|
namespace = "extensions"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### After Migration (Automatic)
|
### After Migration (Automatic)
|
||||||
|
|
||||||
@ -386,7 +387,7 @@ token_path = "/path/to/token"
|
|||||||
[[distributions.oci]]
|
[[distributions.oci]]
|
||||||
registry = "registry.example.com"
|
registry = "registry.example.com"
|
||||||
namespace = "extensions"
|
namespace = "extensions"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Gradual Upgrade Path
|
### Gradual Upgrade Path
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Extension Development Guide
|
# Extension Development Guide
|
||||||
|
|
||||||
This document provides comprehensive guidance on creating providers, task services, and clusters for provisioning, including templates, testing frameworks, publishing, and best practices.
|
This document provides comprehensive guidance on creating providers, task services, and clusters for provisioning, including templates, testing
|
||||||
|
frameworks, publishing, and best practices.
|
||||||
|
|
||||||
## Table of Contents
|
## Table of Contents
|
||||||
|
|
||||||
@ -55,7 +56,7 @@ Extension Ecosystem
|
|||||||
├── CI/CD Pipeline # Continuous integration/deployment
|
├── CI/CD Pipeline # Continuous integration/deployment
|
||||||
├── Data Platform # Data processing and analytics
|
├── Data Platform # Data processing and analytics
|
||||||
└── Custom Clusters # User-defined clusters
|
└── Custom Clusters # User-defined clusters
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Extension Discovery
|
### Extension Discovery
|
||||||
|
|
||||||
@ -80,13 +81,14 @@ let taskservs = (path-resolver list_extensions "taskservs" --include-core)
|
|||||||
|
|
||||||
# Resolve cluster definition
|
# Resolve cluster definition
|
||||||
let cluster_path = (path-resolver resolve_extension "clusters" "web-stack")
|
let cluster_path = (path-resolver resolve_extension "clusters" "web-stack")
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Provider Development
|
## Provider Development
|
||||||
|
|
||||||
### Provider Architecture
|
### Provider Architecture
|
||||||
|
|
||||||
Providers implement cloud resource management through a standardized interface that supports multiple cloud platforms while maintaining consistent APIs.
|
Providers implement cloud resource management through a standardized interface that supports multiple cloud platforms while maintaining consistent
|
||||||
|
APIs.
|
||||||
|
|
||||||
**Core Responsibilities**:
|
**Core Responsibilities**:
|
||||||
|
|
||||||
@ -106,7 +108,7 @@ cp -r workspace/extensions/providers/template workspace/extensions/providers/my-
|
|||||||
|
|
||||||
# Navigate to new provider
|
# Navigate to new provider
|
||||||
cd workspace/extensions/providers/my-cloud
|
cd workspace/extensions/providers/my-cloud
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**2. Update Configuration**:
|
**2. Update Configuration**:
|
||||||
|
|
||||||
@ -117,7 +119,7 @@ nu init-provider.nu \
|
|||||||
--display-name "MyCloud Provider" \
|
--display-name "MyCloud Provider" \
|
||||||
--author "$USER" \
|
--author "$USER" \
|
||||||
--description "MyCloud platform integration"
|
--description "MyCloud platform integration"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Provider Structure
|
### Provider Structure
|
||||||
|
|
||||||
@ -163,7 +165,7 @@ my-cloud/
|
|||||||
└── mock/ # Mock data and services
|
└── mock/ # Mock data and services
|
||||||
├── api-responses.json # Mock API responses
|
├── api-responses.json # Mock API responses
|
||||||
└── test-configs.toml # Test configurations
|
└── test-configs.toml # Test configurations
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Provider Implementation
|
### Provider Implementation
|
||||||
|
|
||||||
@ -327,7 +329,7 @@ export def "provider test" [
|
|||||||
_ => (error make {msg: $"Unknown test type: ($test_type)"})
|
_ => (error make {msg: $"Unknown test type: ($test_type)"})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Authentication Module** (`nulib/auth/client.nu`):
|
**Authentication Module** (`nulib/auth/client.nu`):
|
||||||
|
|
||||||
@ -374,7 +376,7 @@ def test_auth_api [client: record] -> bool {
|
|||||||
|
|
||||||
$response.status == "success"
|
$response.status == "success"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Nickel Configuration Schema** (`schemas/settings.ncl`):
|
**Nickel Configuration Schema** (`schemas/settings.ncl`):
|
||||||
|
|
||||||
@ -431,7 +433,7 @@ let FirewallRule = {
|
|||||||
description | string | optional,
|
description | string | optional,
|
||||||
} in
|
} in
|
||||||
FirewallRule
|
FirewallRule
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Provider Testing
|
### Provider Testing
|
||||||
|
|
||||||
@ -485,7 +487,7 @@ def main [] {
|
|||||||
test_invalid_plan
|
test_invalid_plan
|
||||||
print "✅ All server management tests passed"
|
print "✅ All server management tests passed"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Integration Testing** (`tests/integration/test-lifecycle.nu`):
|
**Integration Testing** (`tests/integration/test-lifecycle.nu`):
|
||||||
|
|
||||||
@ -523,13 +525,14 @@ def main [] {
|
|||||||
test_complete_lifecycle
|
test_complete_lifecycle
|
||||||
print "✅ All integration tests passed"
|
print "✅ All integration tests passed"
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Task Service Development
|
## Task Service Development
|
||||||
|
|
||||||
### Task Service Architecture
|
### Task Service Architecture
|
||||||
|
|
||||||
Task services are infrastructure components that can be deployed and managed across different environments. They provide standardized interfaces for installation, configuration, and lifecycle management.
|
Task services are infrastructure components that can be deployed and managed across different environments. They provide standardized interfaces for
|
||||||
|
installation, configuration, and lifecycle management.
|
||||||
|
|
||||||
**Core Responsibilities**:
|
**Core Responsibilities**:
|
||||||
|
|
||||||
@ -549,7 +552,7 @@ cp -r workspace/extensions/taskservs/template workspace/extensions/taskservs/my-
|
|||||||
|
|
||||||
# Navigate to new service
|
# Navigate to new service
|
||||||
cd workspace/extensions/taskservs/my-service
|
cd workspace/extensions/taskservs/my-service
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**2. Initialize Service**:
|
**2. Initialize Service**:
|
||||||
|
|
||||||
@ -560,7 +563,7 @@ nu init-service.nu \
|
|||||||
--display-name "My Custom Service" \
|
--display-name "My Custom Service" \
|
||||||
--type "database" \
|
--type "database" \
|
||||||
--github-repo "myorg/my-service"
|
--github-repo "myorg/my-service"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Task Service Structure
|
### Task Service Structure
|
||||||
|
|
||||||
@ -597,7 +600,7 @@ my-service/
|
|||||||
├── unit/ # Unit tests
|
├── unit/ # Unit tests
|
||||||
├── integration/ # Integration tests
|
├── integration/ # Integration tests
|
||||||
└── fixtures/ # Test fixtures and data
|
└── fixtures/ # Test fixtures and data
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Task Service Implementation
|
### Task Service Implementation
|
||||||
|
|
||||||
@ -807,7 +810,7 @@ export def "taskserv test" [
|
|||||||
_ => (error make {msg: $"Unknown test type: ($test_type)"})
|
_ => (error make {msg: $"Unknown test type: ($test_type)"})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Version Configuration** (`schemas/version.ncl`):
|
**Version Configuration** (`schemas/version.ncl`):
|
||||||
|
|
||||||
@ -894,7 +897,7 @@ let version_config = {
|
|||||||
},
|
},
|
||||||
} in
|
} in
|
||||||
version_config
|
version_config
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Cluster Development
|
## Cluster Development
|
||||||
|
|
||||||
@ -920,7 +923,7 @@ cp -r workspace/extensions/clusters/template workspace/extensions/clusters/my-st
|
|||||||
|
|
||||||
# Navigate to new cluster
|
# Navigate to new cluster
|
||||||
cd workspace/extensions/clusters/my-stack
|
cd workspace/extensions/clusters/my-stack
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**2. Initialize Cluster**:
|
**2. Initialize Cluster**:
|
||||||
|
|
||||||
@ -930,7 +933,7 @@ nu init-cluster.nu \
|
|||||||
--name "my-stack" \
|
--name "my-stack" \
|
||||||
--display-name "My Application Stack" \
|
--display-name "My Application Stack" \
|
||||||
--type "web-application"
|
--type "web-application"
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Cluster Implementation
|
### Cluster Implementation
|
||||||
|
|
||||||
@ -1045,7 +1048,7 @@ export def "cluster delete" [
|
|||||||
deleted_at: (date now)
|
deleted_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Testing and Validation
|
## Testing and Validation
|
||||||
|
|
||||||
@ -1075,7 +1078,7 @@ nu workspace.nu tools test-extension clusters/my-stack --test-type integration -
|
|||||||
|
|
||||||
# Performance testing
|
# Performance testing
|
||||||
nu workspace.nu tools test-extension providers/my-cloud --test-type performance --duration 5m
|
nu workspace.nu tools test-extension providers/my-cloud --test-type performance --duration 5m
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Automated Testing
|
### Automated Testing
|
||||||
|
|
||||||
@ -1142,7 +1145,7 @@ def main [
|
|||||||
completed_at: (date now)
|
completed_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Publishing and Distribution
|
## Publishing and Distribution
|
||||||
|
|
||||||
@ -1170,7 +1173,7 @@ nu workspace.nu tools publish-extension providers/my-cloud --registry official
|
|||||||
|
|
||||||
# Tag version
|
# Tag version
|
||||||
nu workspace.nu tools tag-extension providers/my-cloud --version 1.0.0 --push
|
nu workspace.nu tools tag-extension providers/my-cloud --version 1.0.0 --push
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Extension Registry
|
### Extension Registry
|
||||||
|
|
||||||
@ -1190,7 +1193,7 @@ Extension Registry
|
|||||||
├── web-stacks/ # Web application stacks
|
├── web-stacks/ # Web application stacks
|
||||||
├── data-platforms/ # Data processing platforms
|
├── data-platforms/ # Data processing platforms
|
||||||
└── ci-cd/ # CI/CD pipelines
|
└── ci-cd/ # CI/CD pipelines
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Best Practices
|
## Best Practices
|
||||||
|
|
||||||
@ -1223,7 +1226,7 @@ def create [n, p] {
|
|||||||
# Missing validation and error handling
|
# Missing validation and error handling
|
||||||
api_call $n $p
|
api_call $n $p
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Configuration Management**:
|
**Configuration Management**:
|
||||||
|
|
||||||
@ -1246,7 +1249,7 @@ def get_api_endpoint [provider: string] -> string {
|
|||||||
def get_api_endpoint [] {
|
def get_api_endpoint [] {
|
||||||
"https://api.provider.com" # Never hardcode!
|
"https://api.provider.com" # Never hardcode!
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Error Handling
|
### Error Handling
|
||||||
|
|
||||||
@ -1292,7 +1295,7 @@ def create_server_with_context [name: string, config: record] -> record {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Testing Practices
|
### Testing Practices
|
||||||
|
|
||||||
@ -1335,7 +1338,7 @@ def test_invalid_inputs [] {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Documentation Standards
|
### Documentation Standards
|
||||||
|
|
||||||
@ -1379,7 +1382,7 @@ def "provider create-server" [
|
|||||||
|
|
||||||
# Implementation...
|
# Implementation...
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
@ -1396,7 +1399,7 @@ nu workspace/lib/path-resolver.nu resolve_extension "providers" "my-provider"
|
|||||||
|
|
||||||
# Validate extension structure
|
# Validate extension structure
|
||||||
nu workspace.nu tools validate-extension providers/my-provider
|
nu workspace.nu tools validate-extension providers/my-provider
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Configuration Errors
|
#### Configuration Errors
|
||||||
|
|
||||||
@ -1411,7 +1414,7 @@ nickel fmt workspace/extensions/providers/my-provider/schemas/
|
|||||||
|
|
||||||
# Test with example data
|
# Test with example data
|
||||||
nickel eval workspace/extensions/providers/my-provider/schemas/settings.ncl
|
nickel eval workspace/extensions/providers/my-provider/schemas/settings.ncl
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### API Integration Issues
|
#### API Integration Issues
|
||||||
|
|
||||||
@ -1425,7 +1428,7 @@ curl -H "Authorization: Bearer $API_KEY" https://api.provider.com/auth/test
|
|||||||
export PROVISIONING_DEBUG=true
|
export PROVISIONING_DEBUG=true
|
||||||
export PROVISIONING_LOG_LEVEL=debug
|
export PROVISIONING_LOG_LEVEL=debug
|
||||||
nu workspace/extensions/providers/my-provider/nulib/provider.nu test --test-type basic
|
nu workspace/extensions/providers/my-provider/nulib/provider.nu test --test-type basic
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Debug Mode
|
### Debug Mode
|
||||||
|
|
||||||
@ -1439,7 +1442,7 @@ export PROVISIONING_WORKSPACE_USER=$USER
|
|||||||
|
|
||||||
# Run extension with debug
|
# Run extension with debug
|
||||||
nu workspace/extensions/providers/my-provider/nulib/provider.nu create-server test-server small --dry-run
|
nu workspace/extensions/providers/my-provider/nulib/provider.nu create-server test-server small --dry-run
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Performance Optimization
|
### Performance Optimization
|
||||||
|
|
||||||
@ -1455,6 +1458,7 @@ nu workspace/tools/runtime-manager.nu monitor --duration 1m --interval 5s
|
|||||||
# Optimize API calls (use caching)
|
# Optimize API calls (use caching)
|
||||||
export PROVISIONING_CACHE_ENABLED=true
|
export PROVISIONING_CACHE_ENABLED=true
|
||||||
export PROVISIONING_CACHE_TTL=300 # 5 minutes
|
export PROVISIONING_CACHE_TTL=300 # 5 minutes
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
This extension development guide provides a comprehensive framework for creating high-quality, maintainable extensions that integrate seamlessly with provisioning's architecture and workflows.
|
This extension development guide provides a comprehensive framework for creating high-quality, maintainable extensions that integrate seamlessly with
|
||||||
|
provisioning's architecture and workflows.
|
||||||
|
|||||||
@ -3,7 +3,8 @@
|
|||||||
**Last Updated**: 2025-10-10
|
**Last Updated**: 2025-10-10
|
||||||
**Version**: 1.0.0
|
**Version**: 1.0.0
|
||||||
|
|
||||||
This glossary defines key terminology used throughout the Provisioning Platform documentation. Terms are listed alphabetically with definitions, usage context, and cross-references to related documentation.
|
This glossary defines key terminology used throughout the Provisioning Platform documentation. Terms are listed alphabetically with definitions, usage
|
||||||
|
context, and cross-references to related documentation.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -33,7 +34,8 @@ This glossary defines key terminology used throughout the Provisioning Platform
|
|||||||
|
|
||||||
### Agent
|
### Agent
|
||||||
|
|
||||||
**Definition**: A specialized component that performs a specific task in the system orchestration (for example, autonomous execution units in the orchestrator).
|
**Definition**: A specialized component that performs a specific task in the system orchestration (for example, autonomous execution units in the
|
||||||
|
orchestrator).
|
||||||
|
|
||||||
**Where Used**:
|
**Where Used**:
|
||||||
|
|
||||||
@ -1595,7 +1597,7 @@ provisioning workspace create <name>
|
|||||||
## Symbol and Acronym Index
|
## Symbol and Acronym Index
|
||||||
|
|
||||||
| Symbol/Acronym | Full Term | Category |
|
| Symbol/Acronym | Full Term | Category |
|
||||||
|----------------|-----------|----------|
|
| ---------------- | ----------- | ---------- |
|
||||||
| ADR | Architecture Decision Record | Architecture |
|
| ADR | Architecture Decision Record | Architecture |
|
||||||
| API | Application Programming Interface | Integration |
|
| API | Application Programming Interface | Integration |
|
||||||
| CLI | Command-Line Interface | User Interface |
|
| CLI | Command-Line Interface | User Interface |
|
||||||
@ -1711,7 +1713,7 @@ provisioning workspace create <name>
|
|||||||
### Avoiding Confusion
|
### Avoiding Confusion
|
||||||
|
|
||||||
| Don't Say | Say Instead | Reason |
|
| Don't Say | Say Instead | Reason |
|
||||||
|-----------|-------------|--------|
|
| ----------- | ------------- | -------- |
|
||||||
| "Task service" | "Taskserv" | Standard platform term |
|
| "Task service" | "Taskserv" | Standard platform term |
|
||||||
| "Configuration file" | "Config" or "Settings" | Context-dependent |
|
| "Configuration file" | "Config" or "Settings" | Context-dependent |
|
||||||
| "Worker" | "Agent" or "Task" | Clarify context |
|
| "Worker" | "Agent" or "Task" | Clarify context |
|
||||||
@ -1748,7 +1750,7 @@ provisioning workspace create <name>
|
|||||||
## Version History
|
## Version History
|
||||||
|
|
||||||
| Version | Date | Changes |
|
| Version | Date | Changes |
|
||||||
|---------|------|---------|
|
| --------- | ------ | --------- |
|
||||||
| 1.0.0 | 2025-10-10 | Initial comprehensive glossary |
|
| 1.0.0 | 2025-10-10 | Initial comprehensive glossary |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@ -7,7 +7,8 @@
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
This guide provides step-by-step instructions for implementing the repository restructuring and distribution system improvements. Each phase includes specific commands, validation steps, and rollback procedures.
|
This guide provides step-by-step instructions for implementing the repository restructuring and distribution system improvements. Each phase includes
|
||||||
|
specific commands, validation steps, and rollback procedures.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,7 @@
|
|||||||
# Integration Guide
|
# Integration Guide
|
||||||
|
|
||||||
This document explains how the new project structure integrates with existing systems, API compatibility and versioning, database migration strategies, deployment considerations, and monitoring and observability.
|
This document explains how the new project structure integrates with existing systems, API compatibility and versioning, database migration
|
||||||
|
strategies, deployment considerations, and monitoring and observability.
|
||||||
|
|
||||||
## Table of Contents
|
## Table of Contents
|
||||||
|
|
||||||
@ -16,7 +17,8 @@ This document explains how the new project structure integrates with existing sy
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Provisioning has been designed with integration as a core principle, ensuring seamless compatibility between new development-focused components and existing production systems while providing clear migration pathways.
|
Provisioning has been designed with integration as a core principle, ensuring seamless compatibility between new development-focused components and
|
||||||
|
existing production systems while providing clear migration pathways.
|
||||||
|
|
||||||
**Integration Principles**:
|
**Integration Principles**:
|
||||||
|
|
||||||
@ -38,7 +40,7 @@ Integration Ecosystem
|
|||||||
│ - File-based │ │ - Monitoring │ │ - Workflows │
|
│ - File-based │ │ - Monitoring │ │ - Workflows │
|
||||||
│ - Simple logging│ │ - Validation │ │ - REST APIs │
|
│ - Simple logging│ │ - Validation │ │ - REST APIs │
|
||||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Existing System Integration
|
## Existing System Integration
|
||||||
|
|
||||||
@ -55,7 +57,7 @@ Integration Ecosystem
|
|||||||
# New commands available alongside existing ones
|
# New commands available alongside existing ones
|
||||||
./src/core/nulib/provisioning server create web-01 2xCPU-4 GB --orchestrated
|
./src/core/nulib/provisioning server create web-01 2xCPU-4 GB --orchestrated
|
||||||
nu workspace/tools/workspace.nu health --detailed
|
nu workspace/tools/workspace.nu health --detailed
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Path Resolution Integration**:
|
**Path Resolution Integration**:
|
||||||
|
|
||||||
@ -68,7 +70,7 @@ let config_path = (path-resolver resolve_path "config" "user" --fallback-to-core
|
|||||||
|
|
||||||
# Seamless extension discovery
|
# Seamless extension discovery
|
||||||
let provider_path = (path-resolver resolve_extension "providers" "upcloud")
|
let provider_path = (path-resolver resolve_extension "providers" "upcloud")
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration System Bridge
|
### Configuration System Bridge
|
||||||
|
|
||||||
@ -105,7 +107,7 @@ def get-config-value-bridge [key: string, default: string = ""] -> string {
|
|||||||
help: $"Migrate from ($env_key) environment variable to ($key) in config file"
|
help: $"Migrate from ($env_key) environment variable to ($key) in config file"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Data Integration
|
### Data Integration
|
||||||
|
|
||||||
@ -134,7 +136,7 @@ def get-server-info [server_name: string] -> record {
|
|||||||
|
|
||||||
error make {msg: $"Server not found: ($server_name)"}
|
error make {msg: $"Server not found: ($server_name)"}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Process Integration
|
### Process Integration
|
||||||
|
|
||||||
@ -163,7 +165,7 @@ def check-orchestrator-available [] -> bool {
|
|||||||
false
|
false
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## API Compatibility and Versioning
|
## API Compatibility and Versioning
|
||||||
|
|
||||||
@ -182,7 +184,7 @@ def check-orchestrator-available [] -> bool {
|
|||||||
curl -H "API-Version: v1" http://localhost:9090/servers
|
curl -H "API-Version: v1" http://localhost:9090/servers
|
||||||
curl -H "API-Version: v2" http://localhost:9090/workflows/servers/create
|
curl -H "API-Version: v2" http://localhost:9090/workflows/servers/create
|
||||||
curl -H "API-Version: v3" http://localhost:9090/workflows/batch/submit
|
curl -H "API-Version: v3" http://localhost:9090/workflows/batch/submit
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### API Compatibility Layer
|
### API Compatibility Layer
|
||||||
|
|
||||||
@ -225,7 +227,7 @@ async fn handle_v1_request(payload: serde_json::Value) -> Result<ApiResponse, Ap
|
|||||||
// Transform response to v1 format
|
// Transform response to v1 format
|
||||||
Ok(transform_to_v1_response(result))
|
Ok(transform_to_v1_response(result))
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Schema Evolution
|
### Schema Evolution
|
||||||
|
|
||||||
@ -259,7 +261,7 @@ let WorkflowOptions = {
|
|||||||
retry_count | number | default = 3,
|
retry_count | number | default = 3,
|
||||||
} in
|
} in
|
||||||
WorkflowOptions
|
WorkflowOptions
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Client SDK Compatibility
|
### Client SDK Compatibility
|
||||||
|
|
||||||
@ -299,7 +301,7 @@ def "client create-server" [
|
|||||||
"API-Version": $api_version
|
"API-Version": $api_version
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Database Migration Strategies
|
## Database Migration Strategies
|
||||||
|
|
||||||
@ -317,7 +319,7 @@ Database Evolution Path
|
|||||||
│ - Text logs │ │ - Transactions │ │ - Real-time │
|
│ - Text logs │ │ - Transactions │ │ - Real-time │
|
||||||
│ - Simple state │ │ - Backup/restore│ │ - Clustering │
|
│ - Simple state │ │ - Backup/restore│ │ - Clustering │
|
||||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Migration Scripts
|
### Migration Scripts
|
||||||
|
|
||||||
@ -360,7 +362,7 @@ def migrate-database [
|
|||||||
print $"Migration from ($from) to ($to) completed successfully"
|
print $"Migration from ($from) to ($to) completed successfully"
|
||||||
{from: $from, to: $to, status: "completed", migrated_at: (date now)}
|
{from: $from, to: $to, status: "completed", migrated_at: (date now)}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**File System to SurrealDB Migration**:
|
**File System to SurrealDB Migration**:
|
||||||
|
|
||||||
@ -412,7 +414,7 @@ def migrate_filesystem_to_surrealdb [] -> record {
|
|||||||
status: "completed"
|
status: "completed"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Data Integrity Verification
|
### Data Integrity Verification
|
||||||
|
|
||||||
@ -456,7 +458,7 @@ def verify-migration [from: string, to: string] -> record {
|
|||||||
verified_at: (date now)
|
verified_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Deployment Considerations
|
## Deployment Considerations
|
||||||
|
|
||||||
@ -480,7 +482,7 @@ Deployment Architecture
|
|||||||
│- Files │ │- Compat │ │- DB │
|
│- Files │ │- Compat │ │- DB │
|
||||||
│- Logs │ │- Monitor │ │- Queue │
|
│- Logs │ │- Monitor │ │- Queue │
|
||||||
└────────┘ └────────────┘ └────────┘
|
└────────┘ └────────────┘ └────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Deployment Strategies
|
### Deployment Strategies
|
||||||
|
|
||||||
@ -513,7 +515,7 @@ nginx-traffic-split --new-backend 90%
|
|||||||
# Phase 4: Complete cutover
|
# Phase 4: Complete cutover
|
||||||
nginx-traffic-split --new-backend 100%
|
nginx-traffic-split --new-backend 100%
|
||||||
/opt/provisioning-v1/bin/orchestrator stop
|
/opt/provisioning-v1/bin/orchestrator stop
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Rolling Update**:
|
**Rolling Update**:
|
||||||
|
|
||||||
@ -568,7 +570,7 @@ def rolling-deployment [
|
|||||||
completed_at: (date now)
|
completed_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Configuration Deployment
|
### Configuration Deployment
|
||||||
|
|
||||||
@ -594,7 +596,7 @@ PROVISIONING_ENV=prod ./deploy.sh \
|
|||||||
--enable-all-monitoring \
|
--enable-all-monitoring \
|
||||||
--backup-before-deploy \
|
--backup-before-deploy \
|
||||||
--health-check-timeout 5m
|
--health-check-timeout 5m
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Container Integration
|
### Container Integration
|
||||||
|
|
||||||
@ -624,7 +626,7 @@ ENV PROVISIONING_NEW_PATH=/app/bin
|
|||||||
|
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
CMD ["/app/bin/bridge-start.sh"]
|
CMD ["/app/bin/bridge-start.sh"]
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Kubernetes Integration**:
|
**Kubernetes Integration**:
|
||||||
|
|
||||||
@ -668,7 +670,7 @@ spec:
|
|||||||
- name: legacy-data
|
- name: legacy-data
|
||||||
persistentVolumeClaim:
|
persistentVolumeClaim:
|
||||||
claimName: provisioning-data
|
claimName: provisioning-data
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Monitoring and Observability
|
## Monitoring and Observability
|
||||||
|
|
||||||
@ -706,7 +708,7 @@ Observability Architecture
|
|||||||
│ - Compatibility │
|
│ - Compatibility │
|
||||||
│ - Migration │
|
│ - Migration │
|
||||||
└───────────────────┘
|
└───────────────────┘
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Metrics Integration
|
### Metrics Integration
|
||||||
|
|
||||||
@ -762,7 +764,7 @@ def collect-new-metrics [] -> record {
|
|||||||
database_stats: (get-database-metrics)
|
database_stats: (get-database-metrics)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Logging Integration
|
### Logging Integration
|
||||||
|
|
||||||
@ -797,7 +799,7 @@ def log-integrated [
|
|||||||
# Send to monitoring system
|
# Send to monitoring system
|
||||||
send-to-monitoring $log_entry
|
send-to-monitoring $log_entry
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Health Check Integration
|
### Health Check Integration
|
||||||
|
|
||||||
@ -834,7 +836,7 @@ def health-check-integrated [] -> record {
|
|||||||
checked_at: (date now)
|
checked_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Legacy System Bridge
|
## Legacy System Bridge
|
||||||
|
|
||||||
@ -897,7 +899,7 @@ export module bridge {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Bridge Operation Modes
|
### Bridge Operation Modes
|
||||||
|
|
||||||
@ -925,7 +927,7 @@ def run-compatibility-mode [] {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Migration Mode**:
|
**Migration Mode**:
|
||||||
|
|
||||||
@ -953,7 +955,7 @@ def run-migration-mode [
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Migration Pathways
|
## Migration Pathways
|
||||||
|
|
||||||
@ -1035,7 +1037,7 @@ def execute-migration-plan [
|
|||||||
results: $migration_results
|
results: $migration_results
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Migration Validation**:
|
**Migration Validation**:
|
||||||
|
|
||||||
@ -1067,7 +1069,7 @@ def validate-migration-readiness [] -> record {
|
|||||||
validated_at: (date now)
|
validated_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Troubleshooting Integration Issues
|
## Troubleshooting Integration Issues
|
||||||
|
|
||||||
@ -1087,7 +1089,7 @@ curl http://localhost:9090/api/versions
|
|||||||
|
|
||||||
# Update client API version
|
# Update client API version
|
||||||
export PROVISIONING_API_VERSION=v2
|
export PROVISIONING_API_VERSION=v2
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Configuration Bridge Issues
|
#### Configuration Bridge Issues
|
||||||
|
|
||||||
@ -1123,7 +1125,7 @@ def migrate-single-config [key: string] {
|
|||||||
print $"Migrated ($key) from environment variable"
|
print $"Migrated ($key) from environment variable"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
#### Database Integration Issues
|
#### Database Integration Issues
|
||||||
|
|
||||||
@ -1158,7 +1160,7 @@ def repair-data-consistency [] -> record {
|
|||||||
repaired_at: (date now)
|
repaired_at: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
### Debug Tools
|
### Debug Tools
|
||||||
|
|
||||||
@ -1173,7 +1175,7 @@ export PROVISIONING_INTEGRATION_TRACE=true
|
|||||||
|
|
||||||
# Run with integration debugging
|
# Run with integration debugging
|
||||||
provisioning server create test-server 2xCPU-4 GB --debug-integration
|
provisioning server create test-server 2xCPU-4 GB --debug-integration
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
**Health Check Debugging**:
|
**Health Check Debugging**:
|
||||||
|
|
||||||
@ -1211,6 +1213,7 @@ def debug-integration-health [] -> record {
|
|||||||
debug_timestamp: (date now)
|
debug_timestamp: (date now)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
This integration guide provides a comprehensive framework for seamlessly integrating new development components with existing production systems while maintaining reliability, compatibility, and clear migration pathways.
|
This integration guide provides a comprehensive framework for seamlessly integrating new development components with existing production systems while
|
||||||
|
maintaining reliability, compatibility, and clear migration pathways.
|
||||||
|
|||||||
@ -11,7 +11,8 @@ The KMS service has been simplified from supporting 4 backends (Vault, AWS KMS,
|
|||||||
- **Age**: Development and local testing
|
- **Age**: Development and local testing
|
||||||
- **Cosmian KMS**: Production deployments
|
- **Cosmian KMS**: Production deployments
|
||||||
|
|
||||||
This simplification reduces complexity, removes unnecessary cloud provider dependencies, and provides a clearer separation between development and production use cases.
|
This simplification reduces complexity, removes unnecessary cloud provider dependencies, and provides a clearer separation between development and
|
||||||
|
production use cases.
|
||||||
|
|
||||||
## What Changed
|
## What Changed
|
||||||
|
|
||||||
@ -563,6 +564,7 @@ Use this checklist to track your migration:
|
|||||||
|
|
||||||
## Conclusion
|
## Conclusion
|
||||||
|
|
||||||
The KMS simplification reduces complexity while providing better separation between development and production use cases. Age offers a fast, offline solution for development, while Cosmian KMS provides enterprise-grade security for production deployments.
|
The KMS simplification reduces complexity while providing better separation between development and production use cases. Age offers a fast, offline
|
||||||
|
solution for development, while Cosmian KMS provides enterprise-grade security for production deployments.
|
||||||
|
|
||||||
For questions or issues, please refer to the documentation or open an issue.
|
For questions or issues, please refer to the documentation or open an issue.
|
||||||
|
|||||||
@ -31,7 +31,7 @@ Replaces the Python implementation with significant performance improvements whi
|
|||||||
• Configuration access: Microsecond latency
|
• Configuration access: Microsecond latency
|
||||||
• Memory efficient: Small struct footprint
|
• Memory efficient: Small struct footprint
|
||||||
• Zero-copy string operations where possible
|
• Zero-copy string operations where possible
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
@ -45,7 +45,7 @@ src/
|
|||||||
├── tools.rs # AI-powered parsing tools
|
├── tools.rs # AI-powered parsing tools
|
||||||
├── errors.rs # Error handling
|
├── errors.rs # Error handling
|
||||||
└── performance_test.rs # Performance benchmarking
|
└── performance_test.rs # Performance benchmarking
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Key Features
|
## Key Features
|
||||||
|
|
||||||
@ -58,7 +58,7 @@ src/
|
|||||||
## Rust vs Python Comparison
|
## Rust vs Python Comparison
|
||||||
|
|
||||||
| Metric | Python MCP Server | Rust MCP Server | Improvement |
|
| Metric | Python MCP Server | Rust MCP Server | Improvement |
|
||||||
|--------|------------------|-----------------|-------------|
|
| -------- | ------------------ | ----------------- | ------------- |
|
||||||
| **Startup Time** | ~500 ms | ~50 ms | **10x faster** |
|
| **Startup Time** | ~500 ms | ~50 ms | **10x faster** |
|
||||||
| **Memory Usage** | ~50 MB | ~5 MB | **10x less** |
|
| **Memory Usage** | ~50 MB | ~5 MB | **10x less** |
|
||||||
| **Parsing Latency** | ~1 ms | ~0.001 ms | **1000x faster** |
|
| **Parsing Latency** | ~1 ms | ~0.001 ms | **1000x faster** |
|
||||||
@ -79,7 +79,7 @@ cargo test
|
|||||||
|
|
||||||
# Run benchmarks
|
# Run benchmarks
|
||||||
cargo run --bin provisioning-mcp-server --release
|
cargo run --bin provisioning-mcp-server --release
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@ -90,7 +90,7 @@ export PROVISIONING_PATH=/path/to/provisioning
|
|||||||
export PROVISIONING_AI_PROVIDER=openai
|
export PROVISIONING_AI_PROVIDER=openai
|
||||||
export OPENAI_API_KEY=your-key
|
export OPENAI_API_KEY=your-key
|
||||||
export PROVISIONING_DEBUG=true
|
export PROVISIONING_DEBUG=true
|
||||||
```plaintext
|
```
|
||||||
|
|
||||||
## Integration Benefits
|
## Integration Benefits
|
||||||
|
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user