prvng_platform/provisioning-server/README.md

# Provisioning API Server

A comprehensive REST API server for remote provisioning operations, enabling thin clients and CI/CD pipeline integration.

## Features

- **Comprehensive REST API**: Complete provisioning operations via HTTP
- **JWT Authentication**: Secure token-based authentication
- **RBAC System**: Role-based access control (Admin, Operator, Developer, Viewer)
- **Async Operations**: Long-running tasks with status tracking
- **Nushell Integration**: Direct execution of provisioning CLI commands
- **Audit Logging**: Complete operation tracking for compliance
- **Metrics**: Prometheus-compatible metrics endpoint
- **CORS Support**: Configurable cross-origin resource sharing
- **Health Checks**: Built-in health and readiness endpoints

## Architecture

```
┌─────────────────┐
│  REST Client    │
│  (curl, CI/CD)  │
└────────┬────────┘
         │ HTTPS/JWT
         ▼
┌─────────────────┐
│  API Gateway    │
│  - Routes       │
│  - Auth         │
│  - RBAC         │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Async Task Mgr  │
│ - Queue         │
│ - Status        │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Nushell Exec    │
│ - CLI wrapper   │
│ - Timeout       │
└─────────────────┘
```

## Installation

### Building from Source

```bash
cd provisioning/platform/provisioning-server
cargo build --release
```

The binary will be at `target/release/provisioning-server`.

### Using Docker

```bash
# Build image
docker build -t provisioning-server .

# Run container
docker run -d \
  -p 8083:8083 \
  -e JWT_SECRET="your-secret-here" \
  -e PROVISIONING_CLI_PATH="/usr/local/bin/provisioning" \
  provisioning-server
```

### Using Docker Compose

```bash
# Start server
docker-compose up -d

# View logs
docker-compose logs -f

# Stop server
docker-compose down
```

## Configuration

### Configuration File

Create `config.toml`:

```toml
[server]
host = "0.0.0.0"
port = 8083
cors_enabled = true

[auth]
jwt_secret = "your-secret-key-here"
token_expiry_hours = 24
refresh_token_expiry_hours = 168

[provisioning]
cli_path = "/usr/local/bin/provisioning"
timeout_seconds = 300
max_concurrent_operations = 10

[logging]
level = "info"
json_format = false
```

### Environment Variables

Alternatively, use environment variables:

```bash
export SERVER_HOST=0.0.0.0
export SERVER_PORT=8083
export JWT_SECRET="your-secret-key"
export PROVISIONING_CLI_PATH="/usr/local/bin/provisioning"
export LOG_LEVEL=info
```

See `.env.example` for all available variables.

## Usage

### Starting the Server

```bash
# Using config file
provisioning-server --config config.toml

# Using environment variables
provisioning-server

# Custom settings
provisioning-server \
  --host 0.0.0.0 \
  --port 8083 \
  --jwt-secret "my-secret" \
  --cli-path "/usr/local/bin/provisioning" \
  --log-level debug
```

### Authentication

#### Login

```bash
curl -X POST http://localhost:8083/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{
    "username": "admin",
    "password": "admin123"
  }'
```

Response:
```json
{
  "token": "eyJhbGc...",
  "refresh_token": "eyJhbGc...",
  "expires_in": 86400
}
```

#### Using Token

Include the token in subsequent requests:

```bash
export TOKEN="eyJhbGc..."

curl -X GET http://localhost:8083/v1/servers \
  -H "Authorization: Bearer $TOKEN"
```

#### Refresh Token

```bash
curl -X POST http://localhost:8083/v1/auth/refresh \
  -H "Content-Type: application/json" \
  -d '{
    "refresh_token": "eyJhbGc..."
  }'
```

### Default Users

| Username   | Password     | Role      |
|------------|--------------|-----------|
| admin      | admin123     | admin     |
| operator   | operator123  | operator  |
| developer  | dev123       | developer |

**⚠️ IMPORTANT**: Change these credentials in production!

## API Endpoints

### Authentication

- `POST /v1/auth/login` - User login
- `POST /v1/auth/refresh` - Refresh access token

### Servers

- `GET /v1/servers` - List all servers
- `POST /v1/servers/create` - Create new server
- `DELETE /v1/servers/{id}` - Delete server
- `GET /v1/servers/{id}/status` - Get server status

#### Create Server Example

```bash
curl -X POST http://localhost:8083/v1/servers/create \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "workspace": "production",
    "provider": "upcloud",
    "plan": "2xCPU-4GB",
    "hostname": "web-01",
    "zone": "de-fra1",
    "check_mode": false,
    "tags": ["web", "production"]
  }'
```

Response:
```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "pending",
  "created_at": "2025-10-06T12:00:00Z",
  "updated_at": "2025-10-06T12:00:00Z",
  "result": null,
  "error": null
}
```

### Taskservs

- `GET /v1/taskservs` - List all taskservs
- `POST /v1/taskservs/create` - Create taskserv
- `DELETE /v1/taskservs/{id}` - Delete taskserv
- `GET /v1/taskservs/{id}/status` - Get taskserv status

#### Create Taskserv Example

```bash
curl -X POST http://localhost:8083/v1/taskservs/create \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "workspace": "production",
    "name": "kubernetes",
    "servers": ["web-01", "web-02"],
    "check_mode": false
  }'
```

### Workflows

- `POST /v1/workflows/submit` - Submit workflow
- `GET /v1/workflows/{id}` - Get workflow details
- `GET /v1/workflows/{id}/status` - Get workflow status
- `POST /v1/workflows/{id}/cancel` - Cancel workflow

#### Submit Workflow Example

```bash
curl -X POST http://localhost:8083/v1/workflows/submit \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "workspace": "production",
    "workflow_type": "cluster_create",
    "parameters": {
      "cluster_name": "k8s-prod",
      "node_count": 3
    }
  }'
```

### Operations

- `GET /v1/operations` - List all operations
- `GET /v1/operations/{id}` - Get operation status
- `POST /v1/operations/{id}/cancel` - Cancel operation

#### Check Operation Status

```bash
curl -X GET http://localhost:8083/v1/operations/550e8400-e29b-41d4-a716-446655440000 \
  -H "Authorization: Bearer $TOKEN"
```

Response:
```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "completed",
  "created_at": "2025-10-06T12:00:00Z",
  "updated_at": "2025-10-06T12:05:00Z",
  "result": {
    "server_id": "srv-abc123",
    "ip_address": "10.0.0.1"
  },
  "error": null
}
```

### Workspaces

- `GET /v1/workspaces` - List workspaces
- `POST /v1/workspaces/create` - Create workspace
- `GET /v1/workspaces/{name}` - Get workspace details

### System

- `GET /health` - Health check (no auth required)
- `GET /v1/version` - Version information
- `GET /v1/metrics` - Prometheus metrics (requires auth)

## RBAC Permissions

### Admin Role

Full system access including:
- All server operations
- All taskserv operations
- All cluster operations
- Workspace management
- Workflow management
- System administration
- Audit log access

### Operator Role

Infrastructure operations:
- Create/delete/read servers
- Create/delete/read taskservs
- Create/delete/read clusters
- Submit/cancel workflows
- View operations

### Developer Role

Read access plus SSH:
- Read servers/taskservs/clusters
- SSH to servers
- View workflows
- View operations

### Viewer Role

Read-only access:
- List all resources
- View status
- System health checks

## Security Best Practices

### Production Deployment

1. **Change Default Credentials**: Update all default usernames/passwords
2. **Use Strong JWT Secret**: Generate a secure random string (32+ characters)
3. **Enable TLS**: Use HTTPS in production
4. **Restrict CORS**: Configure specific allowed origins
5. **Enable mTLS**: For client certificate authentication
6. **Regular Token Rotation**: Implement token refresh strategy
7. **Audit Logging**: Enable audit logs for compliance

### JWT Secret Generation

```bash
# Generate secure random secret
openssl rand -base64 32
```

### TLS Configuration

```toml
[server]
tls_cert = "/path/to/cert.pem"
tls_key = "/path/to/key.pem"
```

### mTLS Configuration

```toml
[auth]
mtls_enabled = true
mtls_ca_cert = "/path/to/ca.pem"
```

## Monitoring

### Health Check

```bash
curl http://localhost:8083/health
```

Response:
```json
{
  "status": "healthy",
  "version": "0.1.0",
  "uptime_seconds": 3600
}
```

### Metrics

```bash
curl http://localhost:8083/v1/metrics \
  -H "Authorization: Bearer $TOKEN"
```

Response:
```json
{
  "uptime_seconds": 3600,
  "tasks": {
    "total": 42,
    "running": 3
  }
}
```

## Development

### Running Tests

```bash
cargo test
```

### Running with Debug Logging

```bash
RUST_LOG=debug provisioning-server
```

### Code Structure

```
src/
├── main.rs              # Application entry point
├── lib.rs               # Library exports
├── config.rs            # Configuration management
├── error.rs             # Error types
├── models/              # Request/response models
├── auth/                # Authentication & authorization
│   ├── jwt.rs           # JWT token handling
│   ├── rbac.rs          # Role-based access control
│   └── mod.rs           # Auth module exports
├── executor/            # Command execution
│   ├── nushell.rs       # Nushell CLI wrapper
│   ├── async_task.rs    # Async task manager
│   └── mod.rs           # Executor exports
└── api/                 # API endpoints
    ├── routes.rs        # Route definitions
    ├── servers.rs       # Server endpoints
    ├── taskservs.rs     # Taskserv endpoints
    ├── workflows.rs     # Workflow endpoints
    ├── operations.rs    # Operation endpoints
    ├── workspaces.rs    # Workspace endpoints
    ├── auth.rs          # Auth endpoints
    ├── system.rs        # System endpoints
    └── mod.rs           # API module exports
```

## CI/CD Integration

### GitHub Actions Example

```yaml
name: Deploy Infrastructure

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Login
        run: |
          TOKEN=$(curl -X POST https://api.example.com/v1/auth/login \
            -H "Content-Type: application/json" \
            -d '{"username":"${{ secrets.API_USER }}","password":"${{ secrets.API_PASS }}"}' \
            | jq -r '.token')
          echo "TOKEN=$TOKEN" >> $GITHUB_ENV

      - name: Create Server
        run: |
          curl -X POST https://api.example.com/v1/servers/create \
            -H "Authorization: Bearer $TOKEN" \
            -H "Content-Type: application/json" \
            -d '{
              "workspace": "production",
              "provider": "upcloud",
              "plan": "2xCPU-4GB",
              "hostname": "web-${{ github.run_number }}"
            }'
```

### GitLab CI Example

```yaml
deploy:
  stage: deploy
  script:
    - |
      TOKEN=$(curl -X POST $API_URL/v1/auth/login \
        -H "Content-Type: application/json" \
        -d "{\"username\":\"$API_USER\",\"password\":\"$API_PASS\"}" \
        | jq -r '.token')
    - |
      curl -X POST $API_URL/v1/workflows/submit \
        -H "Authorization: Bearer $TOKEN" \
        -H "Content-Type: application/json" \
        -d @workflow.json
  only:
    - main
```

## Troubleshooting

### Common Issues

**Authentication Failed**
- Check JWT secret configuration
- Verify username/password
- Check token expiration

**Operation Timeout**
- Increase `timeout_seconds` in config
- Check provisioning CLI accessibility
- Verify network connectivity

**Permission Denied**
- Check user role assignments
- Verify RBAC permissions
- Review audit logs

**Server Not Starting**
- Check port availability
- Verify configuration file
- Check log output

### Debug Mode

Enable detailed logging:

```bash
RUST_LOG=debug provisioning-server --log-level debug
```

## License

See project root LICENSE file.

## Support

For issues and questions:
- Check documentation: `/docs`
- Review API reference: `/v1/health`
- Contact: infrastructure team