prvng_platform/extension-registry

Extension Registry Service

A high-performance Rust microservice that provides a unified REST API for extension discovery, versioning, and download from multiple sources (Gitea releases and OCI registries).

Features

• Multi-Backend Support: Fetch extensions from Gitea releases and OCI registries
• Unified REST API: Single API for all extension operations
• Smart Caching: LRU cache with TTL to reduce backend API calls
• Prometheus Metrics: Built-in metrics for monitoring
• Health Monitoring: Health checks for all backends
• Type-Safe: Strong typing for extension metadata
• Async/Await: High-performance async operations with Tokio
• Docker Support: Production-ready containerization

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Extension Registry API                    │
│                         (axum)                               │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌────────────────┐  ┌────────────────┐  ┌──────────────┐ │
│  │  Gitea Client  │  │   OCI Client   │  │  LRU Cache   │ │
│  │  (reqwest)     │  │   (reqwest)    │  │  (parking)   │ │
│  └────────────────┘  └────────────────┘  └──────────────┘ │
│         │                    │                    │         │
└─────────┼────────────────────┼────────────────────┼─────────┘
          │                    │                    │
          ▼                    ▼                    ▼
    ┌──────────┐         ┌──────────┐        ┌──────────┐
    │  Gitea   │         │   OCI    │        │  Memory  │
    │ Releases │         │ Registry │        │          │
    └──────────┘         └──────────┘        └──────────┘

Installation

Building from Source

cd provisioning/platform/extension-registry
cargo build --release

Docker Build

docker build -t extension-registry:latest .

Running with Cargo

cargo run -- --config config.toml --port 8082

Running with Docker

docker run -d \
  -p 8082:8082 \
  -v $(pwd)/config.toml:/app/config.toml:ro \
  -v $(pwd)/tokens:/app/tokens:ro \
  extension-registry:latest

Configuration

Create a config.toml file (see config.example.toml):

[server]
host = "0.0.0.0"
port = 8082
workers = 4
enable_cors = true
enable_compression = true

# Gitea backend (optional)
[gitea]
url = "https://gitea.example.com"
organization = "provisioning-extensions"
token_path = "/path/to/gitea-token.txt"
timeout_seconds = 30
verify_ssl = true

# OCI registry backend (optional)
[oci]
registry = "registry.example.com"
namespace = "provisioning"
auth_token_path = "/path/to/oci-token.txt"
timeout_seconds = 30
verify_ssl = true

# Cache configuration
[cache]
capacity = 1000
ttl_seconds = 300
enable_metadata_cache = true
enable_list_cache = true

Note: At least one backend (Gitea or OCI) must be configured.

API Endpoints

Extension Operations

List Extensions

GET /api/v1/extensions

Query parameters:

• type (optional): Filter by extension type (provider, taskserv, cluster)
• source (optional): Filter by source (gitea, oci)
• limit (optional): Maximum results (default: 100)
• offset (optional): Pagination offset (default: 0)

Example:

curl http://localhost:8082/api/v1/extensions?type=provider&limit=10

Response:

[
  {
    "name": "aws",
    "type": "provider",
    "version": "1.2.0",
    "description": "AWS provider for provisioning",
    "author": "provisioning-team",
    "repository": "https://gitea.example.com/org/aws_prov",
    "source": "gitea",
    "published_at": "2025-10-06T12:00:00Z",
    "download_url": "https://gitea.example.com/org/aws_prov/releases/download/1.2.0/aws_prov.tar.gz",
    "size": 1024000
  }
]

Get Extension

GET /api/v1/extensions/{type}/{name}

Example:

curl http://localhost:8082/api/v1/extensions/provider/aws

List Versions

GET /api/v1/extensions/{type}/{name}/versions

Example:

curl http://localhost:8082/api/v1/extensions/provider/aws/versions

Response:

[
  {
    "version": "1.2.0",
    "published_at": "2025-10-06T12:00:00Z",
    "download_url": "https://gitea.example.com/org/aws_prov/releases/download/1.2.0/aws_prov.tar.gz",
    "size": 1024000
  },
  {
    "version": "1.1.0",
    "published_at": "2025-09-15T10:30:00Z",
    "download_url": "https://gitea.example.com/org/aws_prov/releases/download/1.1.0/aws_prov.tar.gz",
    "size": 980000
  }
]

Download Extension

GET /api/v1/extensions/{type}/{name}/{version}

Example:

curl -O http://localhost:8082/api/v1/extensions/provider/aws/1.2.0

Returns binary data with Content-Type: application/octet-stream.

Search Extensions

GET /api/v1/extensions/search?q={query}

Query parameters:

• q (required): Search query
• type (optional): Filter by extension type
• limit (optional): Maximum results (default: 50)

Example:

curl http://localhost:8082/api/v1/extensions/search?q=kubernetes&type=taskserv

System Endpoints

Health Check

GET /api/v1/health

Example:

curl http://localhost:8082/api/v1/health

Response:

{
  "status": "healthy",
  "version": "0.1.0",
  "uptime": 3600,
  "backends": {
    "gitea": {
      "enabled": true,
      "healthy": true
    },
    "oci": {
      "enabled": true,
      "healthy": true
    }
  }
}

Metrics

GET /api/v1/metrics

Returns Prometheus-formatted metrics:

# HELP http_requests_total Total HTTP requests
# TYPE http_requests_total counter
http_requests_total 1234

# HELP cache_hits_total Total cache hits
# TYPE cache_hits_total counter
cache_hits_total 567

# HELP cache_misses_total Total cache misses
# TYPE cache_misses_total counter
cache_misses_total 123

Cache Statistics

GET /api/v1/cache/stats

Response:

{
  "list_entries": 45,
  "metadata_entries": 120,
  "version_entries": 80,
  "total_entries": 245
}

Extension Naming Conventions

Gitea Repositories

Extensions in Gitea follow specific naming patterns:

• Providers: {name}_prov (e.g., aws_prov, upcloud_prov)
• Task Services: {name}_taskserv (e.g., kubernetes_taskserv, postgres_taskserv)
• Clusters: {name}_cluster (e.g., buildkit_cluster, ci_cluster)

OCI Artifacts

Extensions in OCI registries follow these patterns:

• Providers: {namespace}/{name}-provider (e.g., provisioning/aws-provider)
• Task Services: {namespace}/{name}-taskserv (e.g., provisioning/kubernetes-taskserv)
• Clusters: {namespace}/{name}-cluster (e.g., provisioning/buildkit-cluster)

Caching Strategy

The service implements a multi-level LRU cache with TTL:

1. List Cache: Caches extension lists (filtered by type/source)
2. Metadata Cache: Caches individual extension metadata
3. Version Cache: Caches version lists per extension

Cache behavior:

• Capacity: Configurable (default: 1000 entries)
• TTL: Configurable (default: 5 minutes)
• Eviction: LRU (Least Recently Used)
• Invalidation: Automatic on TTL expiration

Cache keys:

• List: list:{type}:{source}
• Metadata: {type}/{name}
• Versions: {type}/{name}/versions

Error Handling

The API uses standard HTTP status codes:

• 200 OK: Successful operation
• 400 Bad Request: Invalid input (e.g., invalid extension type)
• 401 Unauthorized: Authentication failed
• 404 Not Found: Extension not found
• 429 Too Many Requests: Rate limit exceeded
• 500 Internal Server Error: Server error

Error response format:

{
  "error": "not_found",
  "message": "Extension provider/nonexistent not found"
}

Metrics and Monitoring

Prometheus Metrics

Available metrics:

• http_requests_total: Total HTTP requests
• http_request_duration_seconds: Request duration histogram
• cache_hits_total: Total cache hits
• cache_misses_total: Total cache misses
• extensions_total: Total extensions served

Health Checks

The health endpoint checks:

• Service uptime
• Gitea backend connectivity
• OCI backend connectivity
• Overall service status

Development

Project Structure

extension-registry/
├── Cargo.toml              # Rust dependencies
├── src/
│   ├── main.rs             # Entry point
│   ├── lib.rs              # Library exports
│   ├── config.rs           # Configuration management
│   ├── error.rs            # Error types
│   ├── api/
│   │   ├── handlers.rs     # HTTP handlers
│   │   └── routes.rs       # Route definitions
│   ├── gitea/
│   │   ├── client.rs       # Gitea API client
│   │   └── models.rs       # Gitea data models
│   ├── oci/
│   │   ├── client.rs       # OCI registry client
│   │   └── models.rs       # OCI data models
│   ├── cache/
│   │   └── lru_cache.rs    # LRU caching
│   └── models/
│       └── extension.rs    # Extension models
├── tests/
│   └── integration_test.rs # Integration tests
├── Dockerfile              # Docker build
└── README.md               # This file

Running Tests

# Run all tests
cargo test

# Run with output
cargo test -- --nocapture

# Run specific test
cargo test test_health_check

Code Quality

# Format code
cargo fmt

# Run clippy
cargo clippy

# Check for security vulnerabilities
cargo audit

Deployment

Systemd Service

Create /etc/systemd/system/extension-registry.service:

[Unit]
Description=Extension Registry Service
After=network.target

[Service]
Type=simple
User=registry
WorkingDirectory=/opt/extension-registry
ExecStart=/usr/local/bin/extension-registry --config /etc/extension-registry/config.toml
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl enable extension-registry
sudo systemctl start extension-registry
sudo systemctl status extension-registry

Docker Compose

version: '3.8'

services:
  extension-registry:
    image: extension-registry:latest
    ports:
      - "8082:8082"
    volumes:
      - ./config.toml:/app/config.toml:ro
      - ./tokens:/app/tokens:ro
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8082/api/v1/health"]
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 5s

Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: extension-registry
spec:
  replicas: 3
  selector:
    matchLabels:
      app: extension-registry
  template:
    metadata:
      labels:
        app: extension-registry
    spec:
      containers:
      - name: extension-registry
        image: extension-registry:latest
        ports:
        - containerPort: 8082
        volumeMounts:
        - name: config
          mountPath: /app/config.toml
          subPath: config.toml
        - name: tokens
          mountPath: /app/tokens
        livenessProbe:
          httpGet:
            path: /api/v1/health
            port: 8082
          initialDelaySeconds: 5
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /api/v1/health
            port: 8082
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: config
        configMap:
          name: extension-registry-config
      - name: tokens
        secret:
          secretName: extension-registry-tokens
---
apiVersion: v1
kind: Service
metadata:
  name: extension-registry
spec:
  selector:
    app: extension-registry
  ports:
  - port: 8082
    targetPort: 8082
  type: ClusterIP

Security

Authentication

• Gitea: Token-based authentication via token_path
• OCI: Optional token authentication via auth_token_path

Best Practices

1. Store tokens securely: Use file permissions (600) for token files
2. Enable SSL verification: Set verify_ssl = true in production
3. Use HTTPS: Always use HTTPS for Gitea and OCI registries
4. Limit CORS: Configure CORS appropriately for production
5. Rate limiting: Consider adding rate limiting for public APIs
6. Network isolation: Run service in isolated network segments

Performance

Benchmarks

Typical performance characteristics:

• Cached requests: <5ms response time
• Uncached requests: 50-200ms (depends on backend latency)
• Cache hit ratio: ~85-95% in typical workloads
• Throughput: 1000+ req/s on modern hardware

Optimization Tips

1. Increase cache capacity: For large extension catalogs
2. Tune TTL: Balance freshness vs. performance
3. Use multiple workers: Scale with CPU cores
4. Enable compression: Reduce bandwidth usage
5. Connection pooling: Reuse HTTP connections to backends

Troubleshooting

Common Issues

Service won't start

• Check configuration file syntax
• Verify token files exist and are readable
• Check network connectivity to backends

Extensions not found

• Verify backend configuration (URL, organization, namespace)
• Check backend connectivity with health endpoint
• Review logs for authentication errors

Slow responses

• Check backend latency
• Increase cache capacity or TTL
• Review Prometheus metrics for bottlenecks

Logging

Enable debug logging:

extension-registry --log-level debug

Enable JSON logging for structured logs:

extension-registry --json-log

License

Part of the Provisioning Project.

Contributing

See main project documentation for contribution guidelines.

Support

For issues and questions, please refer to the main provisioning project documentation.