jesus/prvng_platform

Jesús Pérez 79faf3de6c

chore: fix docs

2026-01-14 03:25:20 +00:00

16 KiB

Raw Blame History

Extension Registry Service\n\nA high-performance Rust microservice that provides a unified REST API for extension discovery, versioning,\nand download from multiple sources (Gitea releases and OCI registries).\n\n## Features\n\n- Multi-Backend Support: Fetch extensions from Gitea releases and OCI registries\n- Unified REST API: Single API for all extension operations\n- Smart Caching: LRU cache with TTL to reduce backend API calls\n- Prometheus Metrics: Built-in metrics for monitoring\n- Health Monitoring: Health checks for all backends\n- Type-Safe: Strong typing for extension metadata\n- Async/Await: High-performance async operations with Tokio\n- Docker Support: Production-ready containerization\n\n## Architecture\n\n{$detected_lang}\n┌─────────────────────────────────────────────────\n────────────┐\n│ Extension Registry API │\n│ (axum) │\n├─────────────────────────────────────────────────\n────────────┤\n│ │\n│ ┌────────────────┐ ┌────────────────┐ \n┌──────────────┐ │\n│ │ Gitea Client │ │ OCI Client │ │ LRU Cache │ │\n│ │ (reqwest) │ │ (reqwest) │ │ (parking) │ │\n│ └────────────────┘ └────────────────┘ \n└──────────────┘ │\n│ │ │ │ │\n└─────────┼────────────────────┼──────────────────\n──┼─────────┘\n │ │ │\n ▼ ▼ ▼\n ┌──────────┐ ┌──────────┐ ┌──────────┐\n │ Gitea │ │ OCI │ │ Memory │\n │ Releases │ │ Registry │ │ │\n └──────────┘ └──────────┘ └──────────┘\n\n\n## Installation\n\n### Building from Source\n\n`{$detected_lang}\ncd provisioning/platform/extension-registry\ncargo build --release\n`\n\n### Docker Build\n\n`{$detected_lang}\ndocker build -t extension-registry:latest .\n`\n\n### Running with Cargo\n\n`{$detected_lang}\ncargo run -- --config config.toml --port 8082\n`\n\n### Running with Docker\n\n`{$detected_lang}\ndocker run -d \\n -p 8082:8082 \\n -v $(pwd)/config.toml:/app/config.toml:ro \\n -v $(pwd)/tokens:/app/tokens:ro \\n extension-registry:latest\n`\n\n## Configuration\n\nCreate a `config.toml` file (see `config.example.toml`):\n\n{$detected_lang}\n[server]\nhost = "0.0.0.0"\nport = 8082\nworkers = 4\nenable_cors = true\nenable_compression = true\n\n# Gitea backend (optional)\n[gitea]\nurl = "https://gitea.example.com"\norganization = "provisioning-extensions"\ntoken_path = "/path/to/gitea-token.txt"\ntimeout_seconds = 30\nverify_ssl = true\n\n# OCI registry backend (optional)\n[oci]\nregistry = "registry.example.com"\nnamespace = "provisioning"\nauth_token_path = "/path/to/oci-token.txt"\ntimeout_seconds = 30\nverify_ssl = true\n\n# Cache configuration\n[cache]\ncapacity = 1000\nttl_seconds = 300\nenable_metadata_cache = true\nenable_list_cache = true\n\n\nNote: At least one backend (Gitea or OCI) must be configured.\n\n## API Endpoints\n\n### Extension Operations\n\n#### List Extensions\n\n`{$detected_lang}\nGET /api/v1/extensions\n`\n\nQuery parameters:\n\n- `type` (optional): Filter by extension type (`provider`, `taskserv`, `cluster`)\n- `source` (optional): Filter by source (`gitea`, `oci`)\n- `limit` (optional): Maximum results (default: 100)\n- `offset` (optional): Pagination offset (default: 0)\n\nExample:\n\n`{$detected_lang}\ncurl http://localhost:8082/api/v1/extensions?type=provider&limit=10\n`\n\nResponse:\n\n`{$detected_lang}\n[\n {\n "name": "aws",\n "type": "provider",\n "version": "1.2.0",\n "description": "AWS provider for provisioning",\n "author": "provisioning-team",\n "repository": "https://gitea.example.com/org/aws_prov",\n "source": "gitea",\n "published_at": "2025-10-06T12:00:00Z",\n "download_url": "https://gitea.example.com/org/aws_prov/releases/download/1.2.0/aws_prov.tar.gz",\n "size": 1024000\n }\n]\n`\n\n#### Get Extension\n\n`{$detected_lang}\nGET /api/v1/extensions/{type}/{name}\n`\n\nExample:\n\n`{$detected_lang}\ncurl http://localhost:8082/api/v1/extensions/provider/aws\n`\n\n#### List Versions\n\n`{$detected_lang}\nGET /api/v1/extensions/{type}/{name}/versions\n`\n\nExample:\n\n`{$detected_lang}\ncurl http://localhost:8082/api/v1/extensions/provider/aws/versions\n`\n\nResponse:\n\n`{$detected_lang}\n[\n {\n "version": "1.2.0",\n "published_at": "2025-10-06T12:00:00Z",\n "download_url": "https://gitea.example.com/org/aws_prov/releases/download/1.2.0/aws_prov.tar.gz",\n "size": 1024000\n },\n {\n "version": "1.1.0",\n "published_at": "2025-09-15T10:30:00Z",\n "download_url": "https://gitea.example.com/org/aws_prov/releases/download/1.1.0/aws_prov.tar.gz",\n "size": 980000\n }\n]\n`\n\n#### Download Extension\n\n`{$detected_lang}\nGET /api/v1/extensions/{type}/{name}/{version}\n`\n\nExample:\n\n`{$detected_lang}\ncurl -O http://localhost:8082/api/v1/extensions/provider/aws/1.2.0\n`\n\nReturns binary data with `Content-Type: application/octet-stream`.\n\n#### Search Extensions\n\n`{$detected_lang}\nGET /api/v1/extensions/search?q={query}\n`\n\nQuery parameters:\n\n- `q` (required): Search query\n- `type` (optional): Filter by extension type\n- `limit` (optional): Maximum results (default: 50)\n\nExample:\n\n`{$detected_lang}\ncurl http://localhost:8082/api/v1/extensions/search?q=kubernetes&type=taskserv\n`\n\n### System Endpoints\n\n#### Health Check\n\n`{$detected_lang}\nGET /api/v1/health\n`\n\nExample:\n\n`{$detected_lang}\ncurl http://localhost:8082/api/v1/health\n`\n\nResponse:\n\n`{$detected_lang}\n{\n "status": "healthy",\n "version": "0.1.0",\n "uptime": 3600,\n "backends": {\n "gitea": {\n "enabled": true,\n "healthy": true\n },\n "oci": {\n "enabled": true,\n "healthy": true\n }\n }\n}\n`\n\n#### Metrics\n\n`{$detected_lang}\nGET /api/v1/metrics\n`\n\nReturns Prometheus-formatted metrics:\n\n`{$detected_lang}\n# HELP http_requests_total Total HTTP requests\n# TYPE http_requests_total counter\nhttp_requests_total 1234\n\n# HELP cache_hits_total Total cache hits\n# TYPE cache_hits_total counter\ncache_hits_total 567\n\n# HELP cache_misses_total Total cache misses\n# TYPE cache_misses_total counter\ncache_misses_total 123\n`\n\n#### Cache Statistics\n\n`{$detected_lang}\nGET /api/v1/cache/stats\n`\n\nResponse:\n\n`{$detected_lang}\n{\n "list_entries": 45,\n "metadata_entries": 120,\n "version_entries": 80,\n "total_entries": 245\n}\n`\n\n## Extension Naming Conventions\n\n### Gitea Repositories\n\nExtensions in Gitea follow specific naming patterns:\n\n- Providers: `{name}_prov` (e.g., `aws_prov`, `upcloud_prov`)\n- Task Services: `{name}_taskserv` (e.g., `kubernetes_taskserv`, `postgres_taskserv`)\n- Clusters: `{name}_cluster` (e.g., `buildkit_cluster`, `ci_cluster`)\n\n### OCI Artifacts\n\nExtensions in OCI registries follow these patterns:\n\n- Providers: `{namespace}/{name}-provider` (e.g., `provisioning/aws-provider`)\n- Task Services: `{namespace}/{name}-taskserv` (e.g., `provisioning/kubernetes-taskserv`)\n- Clusters: `{namespace}/{name}-cluster` (e.g., `provisioning/buildkit-cluster`)\n\n## Caching Strategy\n\nThe service implements a multi-level LRU cache with TTL:\n\n1. List Cache: Caches extension lists (filtered by type/source)\n2. Metadata Cache: Caches individual extension metadata\n3. Version Cache: Caches version lists per extension\n\nCache behavior:\n\n- Capacity: Configurable (default: 1000 entries)\n- TTL: Configurable (default: 5 minutes)\n- Eviction: LRU (Least Recently Used)\n- Invalidation: Automatic on TTL expiration\n\nCache keys:\n\n- List: `list:{type}:{source}`\n- Metadata: `{type}/{name}`\n- Versions: `{type}/{name}/versions`\n\n## Error Handling\n\nThe API uses standard HTTP status codes:\n\n- `200 OK`: Successful operation\n- `400 Bad Request`: Invalid input (e.g., invalid extension type)\n- `401 Unauthorized`: Authentication failed\n- `404 Not Found`: Extension not found\n- `429 Too Many Requests`: Rate limit exceeded\n- `500 Internal Server Error`: Server error\n\nError response format:\n\n`{$detected_lang}\n{\n "error": "not_found",\n "message": "Extension provider/nonexistent not found"\n}\n`\n\n## Metrics and Monitoring\n\n### Prometheus Metrics\n\nAvailable metrics:\n\n- `http_requests_total`: Total HTTP requests\n- `http_request_duration_seconds`: Request duration histogram\n- `cache_hits_total`: Total cache hits\n- `cache_misses_total`: Total cache misses\n- `extensions_total`: Total extensions served\n\n### Health Checks\n\nThe health endpoint checks:\n\n- Service uptime\n- Gitea backend connectivity\n- OCI backend connectivity\n- Overall service status\n\n## Development\n\n### Project Structure\n\n{$detected_lang}\nextension-registry/\n├── Cargo.toml # Rust dependencies\n├── src/\n│ ├── main.rs # Entry point\n│ ├── lib.rs # Library exports\n│ ├── config.rs # Configuration management\n│ ├── error.rs # Error types\n│ ├── api/\n│ │ ├── handlers.rs # HTTP handlers\n│ │ └── routes.rs # Route definitions\n│ ├── gitea/\n│ │ ├── client.rs # Gitea API client\n│ │ └── models.rs # Gitea data models\n│ ├── oci/\n│ │ ├── client.rs # OCI registry client\n│ │ └── models.rs # OCI data models\n│ ├── cache/\n│ │ └── lru_cache.rs # LRU caching\n│ └── models/\n│ └── extension.rs # Extension models\n├── tests/\n│ └── integration_test.rs # Integration tests\n├── Dockerfile # Docker build\n└── README.md # This file\n\n\n### Running Tests\n\n`{$detected_lang}\n# Run all tests\ncargo test\n\n# Run with output\ncargo test -- --nocapture\n\n# Run specific test\ncargo test test_health_check\n`\n\n### Code Quality\n\n`{$detected_lang}\n# Format code\ncargo fmt\n\n# Run clippy\ncargo clippy\n\n# Check for security vulnerabilities\ncargo audit\n`\n\n## Deployment\n\n### Systemd Service\n\nCreate `/etc/systemd/system/extension-registry.service`:\n\n`{$detected_lang}\n[Unit]\nDescription=Extension Registry Service\nAfter=network.target\n\n[Service]\nType=simple\nUser=registry\nWorkingDirectory=/opt/extension-registry\nExecStart=/usr/local/bin/extension-registry --config /etc/extension-registry/config.toml\nRestart=on-failure\nRestartSec=5s\n\n[Install]\nWantedBy=multi-user.target\n`\n\nEnable and start:\n\n`{$detected_lang}\nsudo systemctl enable extension-registry\nsudo systemctl start extension-registry\nsudo systemctl status extension-registry\n`\n\n### Docker Compose\n\n`{$detected_lang}\nversion: '3.8'\n\nservices:\n extension-registry:\n image: extension-registry:latest\n ports:\n - "8082:8082"\n volumes:\n - ./config.toml:/app/config.toml:ro\n - ./tokens:/app/tokens:ro\n restart: unless-stopped\n healthcheck:\n test: ["CMD", "curl", "-f", "http://localhost:8082/api/v1/health"]\n interval: 30s\n timeout: 3s\n retries: 3\n start_period: 5s\n`\n\n### Kubernetes Deployment\n\n{$detected_lang}\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: extension-registry\nspec:\n replicas: 3\n selector:\n matchLabels:\n app: extension-registry\n template:\n metadata:\n labels:\n app: extension-registry\n spec:\n containers:\n - name: extension-registry\n image: extension-registry:latest\n ports:\n - containerPort: 8082\n volumeMounts:\n - name: config\n mountPath: /app/config.toml\n subPath: config.toml\n - name: tokens\n mountPath: /app/tokens\n livenessProbe:\n httpGet:\n path: /api/v1/health\n port: 8082\n initialDelaySeconds: 5\n periodSeconds: 10\n readinessProbe:\n httpGet:\n path: /api/v1/health\n port: 8082\n initialDelaySeconds: 5\n periodSeconds: 5\n volumes:\n - name: config\n configMap:\n name: extension-registry-config\n - name: tokens\n secret:\n secretName: extension-registry-tokens\n---\napiVersion: v1\nkind: Service\nmetadata:\n name: extension-registry\nspec:\n selector:\n app: extension-registry\n ports:\n - port: 8082\n targetPort: 8082\n type: ClusterIP\n\n\n## Security\n\n### Authentication\n\n- Gitea: Token-based authentication via `token_path`\n- OCI: Optional token authentication via `auth_token_path`\n\n### Best Practices\n\n1. Store tokens securely: Use file permissions (600) for token files\n2. Enable SSL verification: Set `verify_ssl = true` in production\n3. Use HTTPS: Always use HTTPS for Gitea and OCI registries\n4. Limit CORS: Configure CORS appropriately for production\n5. Rate limiting: Consider adding rate limiting for public APIs\n6. Network isolation: Run service in isolated network segments\n\n## Performance\n\n### Benchmarks\n\nTypical performance characteristics:\n\n- Cached requests: <5ms response time\n- Uncached requests: 50-200ms (depends on backend latency)\n- Cache hit ratio: ~85-95% in typical workloads\n- Throughput: 1000+ req/s on modern hardware\n\n### Optimization Tips\n\n1. Increase cache capacity: For large extension catalogs\n2. Tune TTL: Balance freshness vs. performance\n3. Use multiple workers: Scale with CPU cores\n4. Enable compression: Reduce bandwidth usage\n5. Connection pooling: Reuse HTTP connections to backends\n\n## Troubleshooting\n\n### Common Issues\n\n#### Service won't start\n\n- Check configuration file syntax\n- Verify token files exist and are readable\n- Check network connectivity to backends\n\n#### Extensions not found\n\n- Verify backend configuration (URL, organization, namespace)\n- Check backend connectivity with health endpoint\n- Review logs for authentication errors\n\n#### Slow responses\n\n- Check backend latency\n- Increase cache capacity or TTL\n- Review Prometheus metrics for bottlenecks\n\n### Logging\n\nEnable debug logging:\n\n`{$detected_lang}\nextension-registry --log-level debug\n`\n\nEnable JSON logging for structured logs:\n\n`{$detected_lang}\nextension-registry --json-log\n`\n\n## License\n\nPart of the Provisioning Project.\n\n## Contributing\n\nSee main project documentation for contribution guidelines.\n\n## Support\n\nFor issues and questions, please refer to the main provisioning project documentation.