prvng_platform/infrastructure/oci-registry/README.md

OCI Registry Service\n\nComprehensive OCI (Open Container Initiative) registry deployment and management for the provisioning system.\nSupports multiple registry implementations: Zot (lightweight), Harbor (full-featured),\nand Distribution (OCI reference implementation).\n\n## Table of Contents\n\n- Overview\n- Registry Types\n- Quick Start\n- Installation\n- Configuration\n- Management\n- Namespaces\n- Access Control\n- Monitoring\n- Troubleshooting\n- Advanced Usage\n\n## Overview\n\nThe OCI registry service provides artifact storage and distribution for:\n\n- Extension Packages: Providers, taskservs, clusters\n- KCL Schemas: Configuration schemas and modules\n- Platform Images: Orchestrator, control-center, services\n- Test Artifacts: Development and testing images\n\n### Features\n\n- Multi-Registry Support: Zot, Harbor, Distribution\n- Namespace Organization: Logical separation of artifacts\n- Access Control: RBAC, policies, authentication\n- Monitoring: Prometheus metrics, health checks\n- Garbage Collection: Automatic cleanup of unused artifacts\n- High Availability: Optional HA configurations\n- TLS/SSL: Secure communication\n- UI Interface: Web-based management (Zot, Harbor)\n\n## Registry Types\n\n### Zot (Recommended for Development)\n\nLightweight, fast, OCI-native registry with search and UI.\n\nPros:\n\n- Fast startup and low resource usage\n- Built-in UI and search\n- Prometheus metrics\n- Automatic garbage collection\n- Good for development and small deployments\n\nCons:\n\n- Less mature than Distribution\n- Fewer enterprise features than Harbor\n\nUse Cases:\n\n- Development environments\n- CI/CD pipelines\n- Small to medium deployments\n- Quick prototyping\n\n### Harbor (Recommended for Production)\n\nFull-featured enterprise registry with replication, scanning, and RBAC.\n\nPros:\n\n- Enterprise-grade features\n- Vulnerability scanning (Trivy)\n- Replication and mirroring\n- Advanced RBAC\n- Webhooks and notifications\n- Mature and battle-tested\n\nCons:\n\n- Higher resource requirements\n- More complex setup\n- Heavier than Zot/Distribution\n\nUse Cases:\n\n- Production deployments\n- Multi-tenant environments\n- Security-critical applications\n- Large-scale deployments\n\n### Distribution (OCI Reference)\n\nOfficial OCI registry reference implementation.\n\nPros:\n\n- OCI standard compliance\n- Lightweight and simple\n- Well-documented\n- Industry standard\n\nCons:\n\n- No built-in UI\n- No search functionality\n- Manual garbage collection\n- Basic feature set\n\nUse Cases:\n\n- OCI standard compliance required\n- Minimal registry needs\n- Custom integrations\n- Educational purposes\n\n## Quick Start\n\n### Start Zot Registry (Default)\n\n\n# Start Zot in background\ncd provisioning/platform/oci-registry/zot\ndocker-compose up -d\n\n# Initialize with namespaces and policies\nnu ../scripts/init-registry.nu --registry-type zot\n\n# Check health\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry health"\n\n# Access UI\nopen http://localhost:5000\n\n\n### Start Harbor Registry\n\n\n# Start Harbor\ncd provisioning/platform/oci-registry/harbor\ndocker-compose up -d\n\n# Wait for services to be ready (takes ~2 minutes)\nsleep 120\n\n# Initialize\nnu ../scripts/init-registry.nu --registry-type harbor --admin-password Harbor12345\n\n# Access UI\nopen http://localhost\n# Login: admin / Harbor12345\n\n\n### Start Distribution Registry\n\n\n# Start Distribution with UI\ncd provisioning/platform/oci-registry/distribution\ndocker-compose up -d\n\n# Initialize\nnu ../scripts/init-registry.nu --registry-type distribution\n\n# Access UI (if included)\nopen http://localhost:8080\n\n\n## Installation\n\n### Prerequisites\n\n- Docker (20.10+)\n- Docker Compose (2.0+)\n- Nushell (0.107+)\n\n### Setup\n\n\n# Clone configurations (already included)\ncd provisioning/platform/oci-registry\n\n# Choose registry type\nREGISTRY_TYPE="zot" # or "harbor" or "distribution"\n\n# Generate TLS certificates (optional, for HTTPS)\n./scripts/generate-certs.nu\n\n# Start registry\ncd $REGISTRY_TYPE\ndocker-compose up -d\n\n# Initialize\nnu ../scripts/init-registry.nu --registry-type $REGISTRY_TYPE\n\n# Verify\ndocker-compose ps\n\n\n## Configuration\n\n### Zot Configuration\n\nFile: zot/config.json\n\nKey settings:\n\n\n{\n "storage": {\n "rootDirectory": "/var/lib/registry",\n "dedupe": true,\n "gc": true,\n "gcInterval": "24h"\n },\n "http": {\n "address": "0.0.0.0",\n "port": "5000"\n },\n "extensions": {\n "search": {"enable": true},\n "metrics": {"enable": true},\n "ui": {"enable": true}\n },\n "accessControl": {\n "repositories": {\n "provisioning-extensions/**": {\n "policies": [\n {\n "users": ["provisioning"],\n "actions": ["read", "create", "update", "delete"]\n }\n ]\n }\n }\n }\n}\n\n\n### Harbor Configuration\n\nFile: harbor/harbor.yml\n\nKey settings:\n\n\nhostname: harbor.provisioning.local\nharbor_admin_password: Harbor12345\n\ndatabase:\n password: root123\n\ntrivy:\n ignore_unfixed: false\n skip_update: false\n\nlog:\n level: info\n\n\n### Distribution Configuration\n\nFile: distribution/config.yml\n\nKey settings:\n\n\nstorage:\n filesystem:\n rootdirectory: /var/lib/registry\n delete:\n enabled: true\n\nhttp:\n addr: :5000\n tls:\n certificate: /etc/docker/registry/certs/cert.pem\n key: /etc/docker/registry/certs/key.pem\n\nauth:\n htpasswd:\n realm: Registry\n path: /etc/docker/registry/htpasswd\n\n\n## Management\n\n### Using Nushell Commands\n\n\n# Start registry\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry start --type zot"\n\n# Stop registry\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry stop --type zot"\n\n# Check status\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry status --type zot"\n\n# View logs\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry logs --type zot --follow"\n\n# Health check\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry health --type zot"\n\n# Initialize\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry init --type zot"\n\n# List namespaces\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry namespaces"\n\n\n### Using Docker Compose\n\n\n# Start\ncd provisioning/platform/oci-registry/zot\ndocker-compose up -d\n\n# Stop\ndocker-compose down\n\n# View logs\ndocker-compose logs -f\n\n# Restart\ndocker-compose restart\n\n# Remove (including volumes)\ndocker-compose down -v\n\n\n## Namespaces\n\n### Default Namespaces\n\n| Namespace | Description | Public | Retention |\n| ----------- | ------------- | -------- | ----------- |\n| provisioning-extensions | Extension packages | No | 10 tags, 90 days |\n| provisioning-kcl | KCL schemas | No | 20 tags, 180 days |\n| provisioning-platform | Platform images | No | 5 tags, 30 days |\n| provisioning-test | Test artifacts | Yes | 3 tags, 7 days |\n\n### Manage Namespaces\n\n\n# Setup all namespaces\nnu scripts/setup-namespaces.nu --registry-type zot\n\n# List namespaces\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; oci-registry namespaces"\n\n# Create namespace\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; \\n oci-registry namespace create my-namespace --type zot"\n\n# Get namespace info\nnu scripts/setup-namespaces.nu namespace info provisioning-extensions\n\n\n## Access Control\n\n### Policies\n\nDefault access policies:\n\nprovisioning-extensions:\n\n- Authenticated: Read, Write, Delete\n- Anonymous: None\n\nprovisioning-kcl:\n\n- Authenticated: Read, Write\n- Anonymous: None\n\nprovisioning-platform:\n\n- Authenticated: Read only (except admin)\n- Anonymous: None\n\nprovisioning-test:\n\n- Authenticated: Read, Write, Delete\n- Anonymous: Read only\n\n### Configure Policies\n\n\n# Apply all policies\nnu scripts/configure-policies.nu --registry-type zot\n\n# Show policy for namespace\nnu scripts/configure-policies.nu policy show provisioning-extensions\n\n# List all policies\nnu scripts/configure-policies.nu policy list\n\n\n### Authentication\n\nZot/Distribution (htpasswd):\n\n\n# Create user\nhtpasswd -Bc htpasswd provisioning\n\n# Login\ndocker login localhost:5000\n\n\nHarbor (Database):\n\n\n# Login via UI or CLI\ndocker login localhost\n# Username: admin\n# Password: Harbor12345\n\n# Create users via Harbor UI\n# Admin → Users → New User\n\n\n## Monitoring\n\n### Health Checks\n\n\n# Full health check\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; \\n oci-registry health --type zot"\n\n# API check\ncurl http://localhost:5000/v2/\n\n# Catalog check\ncurl http://localhost:5000/v2/_catalog\n\n\n### Metrics\n\nZot:\n\n\n# Prometheus metrics\ncurl http://localhost:5000/metrics\n\n# Visualize with Prometheus\n# Add to prometheus.yml:\n# - targets: ['localhost:5000']\n\n\nDistribution:\n\n\n# Metrics on debug port\ncurl http://localhost:5001/metrics\n\n\nHarbor:\n\n\n# Metrics endpoint\ncurl http://localhost:9090/metrics\n\n# View in Harbor UI\n# Admin → System Settings → Metrics\n\n\n### Logs\n\n\n# Zot logs\ndocker-compose logs -f zot\n\n# Harbor logs\ndocker-compose logs -f core registry nginx\n\n# Distribution logs\ndocker-compose logs -f registry\n\n# Nushell command\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry; \\n oci-registry logs --type zot --follow --tail 100"\n\n\n## Troubleshooting\n\n### Registry Not Starting\n\n\n# Check Docker daemon\ndocker ps\n\n# Check ports\nlsof -i :5000\n\n# View logs\ndocker-compose logs\n\n# Rebuild\ndocker-compose down -v\ndocker-compose up -d --build\n\n\n### Cannot Push Images\n\n\n# Check authentication\ndocker login localhost:5000\n\n# Check permissions\n# Ensure user has write access to namespace\n\n# Check storage\ndf -h # Ensure disk space available\n\n# Check registry health\ncurl http://localhost:5000/v2/\n\n\n### Slow Performance\n\n\n# Enable deduplication (Zot)\n# In config.json: "dedupe": true\n\n# Increase resources (Docker)\n# Docker → Preferences → Resources\n\n# Run garbage collection\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry/service; \\n run-oci-registry-gc --type zot"\n\n\n### TLS/Certificate Issues\n\n\n# Regenerate certificates\n./scripts/generate-certs.nu\n\n# Trust certificate\n# macOS: Add to Keychain Access\n# Linux: Copy to /usr/local/share/ca-certificates/\n\n# Skip TLS verification (testing only)\ndocker login --insecure localhost:5000\n\n\n## Advanced Usage\n\n### High Availability (Harbor)\n\n\n# harbor/docker-compose.yml\n# Add multiple registry instances\nregistry-1:\n image: goharbor/registry-photon:v2.9.0\n ...\n\nregistry-2:\n image: goharbor/registry-photon:v2.9.0\n ...\n\n# Add load balancer\nnginx:\n ...\n depends_on:\n - registry-1\n - registry-2\n\n\n### S3 Backend (Distribution)\n\n\n# distribution/config.yml\nstorage:\n s3:\n accesskey: AKIAIOSFODNN7EXAMPLE\n secretkey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\n region: us-west-1\n bucket: my-registry-bucket\n rootdirectory: /registry\n\n\n### Replication (Harbor)\n\n\n# Harbor UI → Replications → New Replication Rule\n# Source: Local registry\n# Destination: Remote registry\n# Trigger: Manual/Scheduled/Event-based\n\n\n### Webhooks\n\nZot (via config.json):\n\n\n{\n "http": {\n "notifications": {\n "endpoints": [\n {\n "name": "orchestrator",\n "url": "http://orchestrator:8080/registry/events",\n "headers": {\n "Authorization": ["Bearer token"]\n }\n }\n ]\n }\n }\n}\n\n\nHarbor (via scripts):\n\n\nnu scripts/configure-policies.nu --registry-type harbor\n# Webhooks configured automatically\n\n\n### Garbage Collection\n\nZot (automatic):\n\n\n{\n "storage": {\n "gc": true,\n "gcInterval": "24h"\n }\n}\n\n\nDistribution (manual):\n\n\n# Run GC\ndocker-compose exec registry \\n registry garbage-collect /etc/docker/registry/config.yml\n\n# Or via Nushell\nnu -c "use provisioning/core/nulib/lib_provisioning/oci_registry/service; \\n run-oci-registry-gc --type distribution"\n\n\nHarbor (UI):\n\n\nAdmin → System Settings → Garbage Collection → Run GC\n\n\n## API Reference\n\n### OCI API (All Registries)\n\n\n# List repositories\ncurl http://localhost:5000/v2/_catalog\n\n# List tags\ncurl http://localhost:5000/v2/{repository}/tags/list\n\n# Get manifest\ncurl http://localhost:5000/v2/{repository}/manifests/{tag}\n\n# Delete image (requires delete enabled)\ncurl -X DELETE http://localhost:5000/v2/{repository}/manifests/{digest}\n\n\n### Harbor API\n\n\n# List projects\ncurl -u admin:Harbor12345 \\n http://localhost/api/v2.0/projects\n\n# Create project\ncurl -X POST -u admin:Harbor12345 \\n -H "Content-Type: application/json" \\n -d '{"project_name":"test","metadata":{"public":"false"}}' \\n http://localhost/api/v2.0/projects\n\n# Scan image\ncurl -X POST -u admin:Harbor12345 \\n http://localhost/api/v2.0/projects/{project}/repositories/{repo}/artifacts/{tag}/scan\n\n\n## Performance Tuning\n\n### Zot\n\n\n{\n "storage": {\n "dedupe": true, // Enable deduplication\n "gc": true, // Enable GC\n "gcInterval": "12h" // More frequent GC\n },\n "http": {\n "http2": true // Enable HTTP/2\n }\n}\n\n\n### Distribution\n\n\nstorage:\n cache:\n blobdescriptor: redis # Use Redis for caching\n\nredis:\n addr: redis:6379\n pool:\n maxidle: 16\n maxactive: 64\n\n\n### Harbor\n\n\njobservice:\n max_job_workers: 20 # Increase concurrent jobs\n\ndatabase:\n max_idle_conns: 100\n max_open_conns: 900 # Increase DB connections\n\n\n## Security Best Practices\n\n1. Use TLS/SSL for all connections\n2. Strong passwords for admin accounts\n3. Regular updates of registry software\n4. Scan images for vulnerabilities (Harbor/Trivy)\n5. Least privilege access control\n6. Network isolation (Docker networks)\n7. Regular backups of registry data\n8. Audit logging enabled\n9. Rate limiting for API access\n10. Secrets management (not in configs)\n\n## Backup and Restore\n\n### Backup\n\n\n# Backup Zot\ndocker-compose stop zot\ntar czf zot-backup-$(date +%Y%m%d).tar.gz \\n -C /var/lib/docker/volumes zot-data\n\n# Backup Harbor\ndocker-compose stop\ntar czf harbor-backup-$(date +%Y%m%d).tar.gz \\n -C /var/lib/docker/volumes \\n harbor-registry harbor-database\n\n# Backup Distribution\ndocker-compose stop registry\ntar czf dist-backup-$(date +%Y%m%d).tar.gz \\n -C /var/lib/docker/volumes registry-data\n\n\n### Restore\n\n\n# Restore (example for Zot)\ndocker-compose down -v\ntar xzf zot-backup-20250106.tar.gz -C /var/lib/docker/volumes\ndocker-compose up -d\n\n\n## Migration Between Registries\n\n\n# Example: Zot → Harbor\n\n# 1. Export from Zot\nfor repo in $(curl http://localhost:5000/v2/_catalog | jq -r '.repositories[]'); do\n for tag in $(curl http://localhost:5000/v2/$repo/tags/list | jq -r '.tags[]'); do\n docker pull localhost:5000/$repo:$tag\n docker tag localhost:5000/$repo:$tag harbor.local/$repo:$tag\n docker push harbor.local/$repo:$tag\n done\ndone\n\n# 2. Or use skopeo\nskopeo sync --src docker --dest docker \\n localhost:5000/provisioning-extensions \\n harbor.local/provisioning-extensions\n\n\n## References\n\n- Zot: https://github.com/project-zot/zot\n- Harbor: https://goharbor.io/docs/\n- Distribution: https://github.com/distribution/distribution\n- OCI Spec: https://github.com/opencontainers/distribution-spec\n\n## Support\n\nFor issues or questions:\n\n1. Check logs: docker-compose logs\n2. Review this documentation\n3. Check GitHub issues for respective registry\n4. Contact provisioning team\n\n---\n\nVersion: 1.0.0\nLast Updated: 2025-01-06\nMaintainer: Provisioning Platform Team