Orchestrator Authentication & Authorization Integration

Version: 1.0.0 Date: 2025-10-08 Status: Implemented

Overview

Complete authentication and authorization flow integration for the Provisioning Orchestrator, connecting all security components (JWT validation, MFA verification, Cedar authorization, rate limiting, and audit logging) into a cohesive security middleware chain.

Architecture

Security Middleware Chain

The middleware chain is applied in this specific order to ensure proper security:

┌─────────────────────────────────────────────────────────────────┐
│                    Incoming HTTP Request                        │
└────────────────────────┬────────────────────────────────────────┘
                         │
                         ▼
        ┌────────────────────────────────┐
        │  1. Rate Limiting Middleware   │
        │  - Per-IP request limits       │
        │  - Sliding window              │
        │  - Exempt IPs                  │
        └────────────┬───────────────────┘
                     │ (429 if exceeded)
                     ▼
        ┌────────────────────────────────┐
        │  2. Authentication Middleware  │
        │  - Extract Bearer token        │
        │  - Validate JWT signature      │
        │  - Check expiry, issuer, aud   │
        │  - Check revocation            │
        └────────────┬───────────────────┘
                     │ (401 if invalid)
                     ▼
        ┌────────────────────────────────┐
        │  3. MFA Verification           │
        │  - Check MFA status in token   │
        │  - Enforce for sensitive ops   │
        │  - Production deployments      │
        │  - All DELETE operations       │
        └────────────┬───────────────────┘
                     │ (403 if required but missing)
                     ▼
        ┌────────────────────────────────┐
        │  4. Authorization Middleware   │
        │  - Build Cedar request         │
        │  - Evaluate policies           │
        │  - Check permissions           │
        │  - Log decision                │
        └────────────┬───────────────────┘
                     │ (403 if denied)
                     ▼
        ┌────────────────────────────────┐
        │  5. Audit Logging Middleware   │
        │  - Log complete request        │
        │  - User, action, resource      │
        │  - Authorization decision      │
        │  - Response status             │
        └────────────┬───────────────────┘
                     │
                     ▼
        ┌────────────────────────────────┐
        │      Protected Handler         │
        │  - Access security context     │
        │  - Execute business logic      │
        └────────────────────────────────┘
```plaintext

## Implementation Details

### 1. Security Context Builder (`middleware/security_context.rs`)

**Purpose**: Build complete security context from authenticated requests.

**Key Features**:

- Extracts JWT token claims
- Determines MFA verification status
- Extracts IP address (X-Forwarded-For, X-Real-IP)
- Extracts user agent and session info
- Provides permission checking methods

**Lines of Code**: 275

**Example**:

```rust
pub struct SecurityContext {
    pub user_id: String,
    pub token: ValidatedToken,
    pub mfa_verified: bool,
    pub ip_address: IpAddr,
    pub user_agent: Option<String>,
    pub permissions: Vec<String>,
    pub workspace: String,
    pub request_id: String,
    pub session_id: Option<String>,
}

impl SecurityContext {
    pub fn has_permission(&self, permission: &str) -> bool { ... }
    pub fn has_any_permission(&self, permissions: &[&str]) -> bool { ... }
    pub fn has_all_permissions(&self, permissions: &[&str]) -> bool { ... }
}
```plaintext

### 2. Enhanced Authentication Middleware (`middleware/auth.rs`)

**Purpose**: JWT token validation with revocation checking.

**Key Features**:

- Bearer token extraction
- JWT signature validation (RS256)
- Expiry, issuer, audience checks
- Token revocation status
- Security context injection

**Lines of Code**: 245

**Flow**:

1. Extract `Authorization: Bearer <token>` header
2. Validate JWT with TokenValidator
3. Build SecurityContext
4. Inject into request extensions
5. Continue to next middleware or return 401

**Error Responses**:

- `401 Unauthorized`: Missing/invalid token, expired, revoked
- `403 Forbidden`: Insufficient permissions

### 3. MFA Verification Middleware (`middleware/mfa.rs`)

**Purpose**: Enforce MFA for sensitive operations.

**Key Features**:

- Path-based MFA requirements
- Method-based enforcement (all DELETEs)
- Production environment protection
- Clear error messages

**Lines of Code**: 290

**MFA Required For**:

- Production deployments (`/production/`, `/prod/`)
- All DELETE operations
- Server operations (POST, PUT, DELETE)
- Cluster operations (POST, PUT, DELETE)
- Batch submissions
- Rollback operations
- Configuration changes (POST, PUT, DELETE)
- Secret management
- User/role management

**Example**:

```rust
fn requires_mfa(method: &str, path: &str) -> bool {
    if path.contains("/production/") { return true; }
    if method == "DELETE" { return true; }
    if path.contains("/deploy") { return true; }
    // ...
}
```plaintext

### 4. Enhanced Authorization Middleware (`middleware/authz.rs`)

**Purpose**: Cedar policy evaluation with audit logging.

**Key Features**:

- Builds Cedar authorization request from HTTP request
- Maps HTTP methods to Cedar actions (GET→Read, POST→Create, etc.)
- Extracts resource types from paths
- Evaluates Cedar policies with context (MFA, IP, time, workspace)
- Logs all authorization decisions to audit log
- Non-blocking audit logging (tokio::spawn)

**Lines of Code**: 380

**Resource Mapping**:

```rust
/api/v1/servers/srv-123    → Resource::Server("srv-123")
/api/v1/taskserv/kubernetes → Resource::TaskService("kubernetes")
/api/v1/cluster/prod        → Resource::Cluster("prod")
/api/v1/config/settings     → Resource::Config("settings")
```plaintext

**Action Mapping**:

```rust
GET    → Action::Read
POST   → Action::Create
PUT    → Action::Update
DELETE → Action::Delete
```plaintext

### 5. Rate Limiting Middleware (`middleware/rate_limit.rs`)

**Purpose**: Prevent API abuse with per-IP rate limiting.

**Key Features**:

- Sliding window rate limiting
- Per-IP request tracking
- Configurable limits and windows
- Exempt IP support
- Automatic cleanup of old entries
- Statistics tracking

**Lines of Code**: 420

**Configuration**:

```rust
pub struct RateLimitConfig {
    pub max_requests: u32,          // e.g., 100
    pub window_duration: Duration,  // e.g., 60 seconds
    pub exempt_ips: Vec<IpAddr>,    // e.g., internal services
    pub enabled: bool,
}

// Default: 100 requests per minute
```plaintext

**Statistics**:

```rust
pub struct RateLimitStats {
    pub total_ips: usize,      // Number of tracked IPs
    pub total_requests: u32,   // Total requests made
    pub limited_ips: usize,    // IPs that hit the limit
    pub config: RateLimitConfig,
}
```plaintext

### 6. Security Integration Module (`security_integration.rs`)

**Purpose**: Helper module to integrate all security components.

**Key Features**:

- `SecurityComponents` struct grouping all middleware
- `SecurityConfig` for configuration
- `initialize()` method to set up all components
- `disabled()` method for development mode
- `apply_security_middleware()` helper for router setup

**Lines of Code**: 265

**Usage Example**:

```rust
use provisioning_orchestrator::security_integration::{
    SecurityComponents, SecurityConfig
};

// Initialize security
let config = SecurityConfig {
    public_key_path: PathBuf::from("keys/public.pem"),
    jwt_issuer: "control-center".to_string(),
    jwt_audience: "orchestrator".to_string(),
    cedar_policies_path: PathBuf::from("policies"),
    auth_enabled: true,
    authz_enabled: true,
    mfa_enabled: true,
    rate_limit_config: RateLimitConfig::new(100, 60),
};

let security = SecurityComponents::initialize(config, audit_logger).await?;

// Apply to router
let app = Router::new()
    .route("/api/v1/servers", post(create_server))
    .route("/api/v1/servers/:id", delete(delete_server));

let secured_app = apply_security_middleware(app, &security);
```plaintext

## Integration with AppState

### Updated AppState Structure

```rust
pub struct AppState {
    // Existing fields
    pub task_storage: Arc<dyn TaskStorage>,
    pub batch_coordinator: BatchCoordinator,
    pub dependency_resolver: DependencyResolver,
    pub state_manager: Arc<WorkflowStateManager>,
    pub monitoring_system: Arc<MonitoringSystem>,
    pub progress_tracker: Arc<ProgressTracker>,
    pub rollback_system: Arc<RollbackSystem>,
    pub test_orchestrator: Arc<TestOrchestrator>,
    pub dns_manager: Arc<DnsManager>,
    pub extension_manager: Arc<ExtensionManager>,
    pub oci_manager: Arc<OciManager>,
    pub service_orchestrator: Arc<ServiceOrchestrator>,
    pub audit_logger: Arc<AuditLogger>,
    pub args: Args,

    // NEW: Security components
    pub security: SecurityComponents,
}
```plaintext

### Initialization in main.rs

```rust
#[tokio::main]
async fn main() -> Result<()> {
    let args = Args::parse();

    // Initialize AppState (creates audit_logger)
    let state = Arc::new(AppState::new(args).await?);

    // Initialize security components
    let security_config = SecurityConfig {
        public_key_path: PathBuf::from("keys/public.pem"),
        jwt_issuer: env::var("JWT_ISSUER").unwrap_or("control-center".to_string()),
        jwt_audience: "orchestrator".to_string(),
        cedar_policies_path: PathBuf::from("policies"),
        auth_enabled: env::var("AUTH_ENABLED").unwrap_or("true".to_string()) == "true",
        authz_enabled: env::var("AUTHZ_ENABLED").unwrap_or("true".to_string()) == "true",
        mfa_enabled: env::var("MFA_ENABLED").unwrap_or("true".to_string()) == "true",
        rate_limit_config: RateLimitConfig::new(
            env::var("RATE_LIMIT_MAX").unwrap_or("100".to_string()).parse().unwrap(),
            env::var("RATE_LIMIT_WINDOW").unwrap_or("60".to_string()).parse().unwrap(),
        ),
    };

    let security = SecurityComponents::initialize(
        security_config,
        state.audit_logger.clone()
    ).await?;

    // Public routes (no auth)
    let public_routes = Router::new()
        .route("/health", get(health_check));

    // Protected routes (full security chain)
    let protected_routes = Router::new()
        .route("/api/v1/servers", post(create_server))
        .route("/api/v1/servers/:id", delete(delete_server))
        .route("/api/v1/taskserv", post(create_taskserv))
        .route("/api/v1/cluster", post(create_cluster))
        // ... more routes
        ;

    // Apply security middleware to protected routes
    let secured_routes = apply_security_middleware(protected_routes, &security)
        .with_state(state.clone());

    // Combine routes
    let app = Router::new()
        .merge(public_routes)
        .merge(secured_routes)
        .layer(CorsLayer::permissive());

    // Start server
    let listener = tokio::net::TcpListener::bind("0.0.0.0:9090").await?;
    axum::serve(listener, app).await?;

    Ok(())
}
```plaintext

## Protected Endpoints

### Endpoint Categories

| Category | Example Endpoints | Auth Required | MFA Required | Cedar Policy |
|----------|-------------------|---------------|--------------|--------------|
| **Health** | `/health` | ❌ | ❌ | ❌ |
| **Read-Only** | `GET /api/v1/servers` | ✅ | ❌ | ✅ |
| **Server Mgmt** | `POST /api/v1/servers` | ✅ | ❌ | ✅ |
| **Server Delete** | `DELETE /api/v1/servers/:id` | ✅ | ✅ | ✅ |
| **Taskserv Mgmt** | `POST /api/v1/taskserv` | ✅ | ❌ | ✅ |
| **Cluster Mgmt** | `POST /api/v1/cluster` | ✅ | ✅ | ✅ |
| **Production** | `POST /api/v1/production/*` | ✅ | ✅ | ✅ |
| **Batch Ops** | `POST /api/v1/batch/submit` | ✅ | ✅ | ✅ |
| **Rollback** | `POST /api/v1/rollback` | ✅ | ✅ | ✅ |
| **Config Write** | `POST /api/v1/config` | ✅ | ✅ | ✅ |
| **Secrets** | `GET /api/v1/secret/*` | ✅ | ✅ | ✅ |

## Complete Authentication Flow

### Step-by-Step Flow

```plaintext
1. CLIENT REQUEST
   ├─ Headers:
   │  ├─ Authorization: Bearer <jwt_token>
   │  ├─ X-Forwarded-For: 192.168.1.100
   │  ├─ User-Agent: MyClient/1.0
   │  └─ X-MFA-Verified: true
   └─ Path: DELETE /api/v1/servers/prod-srv-01

2. RATE LIMITING MIDDLEWARE
   ├─ Extract IP: 192.168.1.100
   ├─ Check limit: 45/100 requests in window
   ├─ Decision: ALLOW (under limit)
   └─ Continue →

3. AUTHENTICATION MIDDLEWARE
   ├─ Extract Bearer token
   ├─ Validate JWT:
   │  ├─ Signature: ✅ Valid (RS256)
   │  ├─ Expiry: ✅ Valid until 2025-10-09 10:00:00
   │  ├─ Issuer: ✅ control-center
   │  ├─ Audience: ✅ orchestrator
   │  └─ Revoked: ✅ Not revoked
   ├─ Build SecurityContext:
   │  ├─ user_id: "user-456"
   │  ├─ workspace: "production"
   │  ├─ permissions: ["read", "write", "delete"]
   │  ├─ mfa_verified: true
   │  └─ ip_address: 192.168.1.100
   ├─ Decision: ALLOW (valid token)
   └─ Continue →

4. MFA VERIFICATION MIDDLEWARE
   ├─ Check endpoint: DELETE /api/v1/servers/prod-srv-01
   ├─ Requires MFA: ✅ YES (DELETE operation)
   ├─ MFA status: ✅ Verified
   ├─ Decision: ALLOW (MFA verified)
   └─ Continue →

5. AUTHORIZATION MIDDLEWARE
   ├─ Build Cedar request:
   │  ├─ Principal: User("user-456")
   │  ├─ Action: Delete
   │  ├─ Resource: Server("prod-srv-01")
   │  └─ Context:
   │     ├─ mfa_verified: true
   │     ├─ ip_address: "192.168.1.100"
   │     ├─ time: 2025-10-08T14:30:00Z
   │     └─ workspace: "production"
   ├─ Evaluate Cedar policies:
   │  ├─ Policy 1: Allow if user.role == "admin" ✅
   │  ├─ Policy 2: Allow if mfa_verified == true ✅
   │  └─ Policy 3: Deny if not business_hours ❌
   ├─ Decision: ALLOW (2 allow, 1 deny = allow)
   ├─ Log to audit: Authorization GRANTED
   └─ Continue →

6. AUDIT LOGGING MIDDLEWARE
   ├─ Record:
   │  ├─ User: user-456 (IP: 192.168.1.100)
   │  ├─ Action: ServerDelete
   │  ├─ Resource: prod-srv-01
   │  ├─ Authorization: GRANTED
   │  ├─ MFA: Verified
   │  └─ Timestamp: 2025-10-08T14:30:00Z
   └─ Continue →

7. PROTECTED HANDLER
   ├─ Execute business logic
   ├─ Delete server prod-srv-01
   └─ Return: 200 OK

8. AUDIT LOGGING (Response)
   ├─ Update event:
   │  ├─ Status: 200 OK
   │  ├─ Duration: 1.234s
   │  └─ Result: SUCCESS
   └─ Write to audit log

9. CLIENT RESPONSE
   └─ 200 OK: Server deleted successfully
```plaintext

## Configuration

### Environment Variables

```bash
# JWT Configuration
JWT_ISSUER=control-center
JWT_AUDIENCE=orchestrator
PUBLIC_KEY_PATH=/path/to/keys/public.pem

# Cedar Policies
CEDAR_POLICIES_PATH=/path/to/policies

# Security Toggles
AUTH_ENABLED=true
AUTHZ_ENABLED=true
MFA_ENABLED=true

# Rate Limiting
RATE_LIMIT_MAX=100
RATE_LIMIT_WINDOW=60
RATE_LIMIT_EXEMPT_IPS=10.0.0.1,10.0.0.2

# Audit Logging
AUDIT_ENABLED=true
AUDIT_RETENTION_DAYS=365
```plaintext

### Development Mode

For development/testing, all security can be disabled:

```rust
// In main.rs
let security = if env::var("DEVELOPMENT_MODE").unwrap_or("false".to_string()) == "true" {
    SecurityComponents::disabled(audit_logger.clone())
} else {
    SecurityComponents::initialize(security_config, audit_logger.clone()).await?
};
```plaintext

## Testing

### Integration Tests

Location: `provisioning/platform/orchestrator/tests/security_integration_tests.rs`

**Test Coverage**:

- ✅ Rate limiting enforcement
- ✅ Rate limit statistics
- ✅ Exempt IP handling
- ✅ Authentication missing token
- ✅ MFA verification for sensitive operations
- ✅ Cedar policy evaluation
- ✅ Complete security flow
- ✅ Security components initialization
- ✅ Configuration defaults

**Lines of Code**: 340

**Run Tests**:

```bash
cd provisioning/platform/orchestrator
cargo test security_integration_tests
```plaintext

## File Summary

| File | Purpose | Lines | Tests |
|------|---------|-------|-------|
| `middleware/security_context.rs` | Security context builder | 275 | 8 |
| `middleware/auth.rs` | JWT authentication | 245 | 5 |
| `middleware/mfa.rs` | MFA verification | 290 | 15 |
| `middleware/authz.rs` | Cedar authorization | 380 | 4 |
| `middleware/rate_limit.rs` | Rate limiting | 420 | 8 |
| `middleware/mod.rs` | Module exports | 25 | 0 |
| `security_integration.rs` | Integration helpers | 265 | 2 |
| `tests/security_integration_tests.rs` | Integration tests | 340 | 11 |
| **Total** | | **2,240** | **53** |

## Benefits

### Security

- ✅ Complete authentication flow with JWT validation
- ✅ MFA enforcement for sensitive operations
- ✅ Fine-grained authorization with Cedar policies
- ✅ Rate limiting prevents API abuse
- ✅ Complete audit trail for compliance

### Architecture

- ✅ Modular middleware design
- ✅ Clear separation of concerns
- ✅ Reusable security components
- ✅ Easy to test and maintain
- ✅ Configuration-driven behavior

### Operations

- ✅ Can enable/disable features independently
- ✅ Development mode for testing
- ✅ Comprehensive error messages
- ✅ Real-time statistics and monitoring
- ✅ Non-blocking audit logging

## Future Enhancements

1. **Token Refresh**: Automatic token refresh before expiry
2. **IP Whitelisting**: Additional IP-based access control
3. **Geolocation**: Block requests from specific countries
4. **Advanced Rate Limiting**: Per-user, per-endpoint limits
5. **Session Management**: Track active sessions, force logout
6. **2FA Integration**: Direct integration with TOTP/SMS providers
7. **Policy Hot Reload**: Update Cedar policies without restart
8. **Metrics Dashboard**: Real-time security metrics visualization

## Related Documentation

- Cedar Policy Language
- JWT Token Management
- MFA Setup Guide
- Audit Log Format
- Rate Limiting Best Practices

## Version History

| Version | Date | Changes |
|---------|------|---------|
| 1.0.0 | 2025-10-08 | Initial implementation |

---

**Maintained By**: Security Team
**Review Cycle**: Quarterly
**Last Reviewed**: 2025-10-08