## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Orchestrator Authentication & Authorization Integration

Version: 1.0.0 Date: 2025-10-08 Status: Implemented

## Overview

Complete authentication and authorization flow integration for the Provisioning Orchestrator, connecting all security components (JWT validation, MFA verification, Cedar authorization, rate limiting, and audit logging) into a cohesive security middleware chain.

## Architecture

### Security Middleware Chain

The middleware chain is applied in this specific order to ensure proper security:

    ┌─────────────────────────────────────────────────────────────────┐
    │                    Incoming HTTP Request                        │
    └────────────────────────┬────────────────────────────────────────┘
                             │
                             ▼
            ┌────────────────────────────────┐
            │  1. Rate Limiting Middleware   │
            │  - Per-IP request limits       │
            │  - Sliding window              │
            │  - Exempt IPs                  │
            └────────────┬───────────────────┘
                         │ (429 if exceeded)
                         ▼
            ┌────────────────────────────────┐
            │  2. Authentication Middleware  │
            │  - Extract Bearer token        │
            │  - Validate JWT signature      │
            │  - Check expiry, issuer, aud   │
            │  - Check revocation            │
            └────────────┬───────────────────┘
                         │ (401 if invalid)
                         ▼
            ┌────────────────────────────────┐
            │  3. MFA Verification           │
            │  - Check MFA status in token   │
            │  - Enforce for sensitive ops   │
            │  - Production deployments      │
            │  - All DELETE operations       │
            └────────────┬───────────────────┘
                         │ (403 if required but missing)
                         ▼
            ┌────────────────────────────────┐
            │  4. Authorization Middleware   │
            │  - Build Cedar request         │
            │  - Evaluate policies           │
            │  - Check permissions           │
            │  - Log decision                │
            └────────────┬───────────────────┘
                         │ (403 if denied)
                         ▼
            ┌────────────────────────────────┐
            │  5. Audit Logging Middleware   │
            │  - Log complete request        │
            │  - User, action, resource      │
            │  - Authorization decision      │
            │  - Response status             │
            └────────────┬───────────────────┘
                         │
                         ▼
            ┌────────────────────────────────┐
            │      Protected Handler         │
            │  - Access security context     │
            │  - Execute business logic      │
            └────────────────────────────────┘

## Implementation Details

### 1. Security Context Builder (`middleware/security_context.rs`)

Purpose: Build complete security context from authenticated requests.

Key Features:

- Extracts JWT token claims
- Determines MFA verification status
- Extracts IP address (X-Forwarded-For, X-Real-IP)
- Extracts user agent and session info
- Provides permission checking methods

Lines of Code: 275

Example:

    pub struct SecurityContext {
        pub user_id: String,
        pub token: ValidatedToken,
        pub mfa_verified: bool,
        pub ip_address: IpAddr,
        pub user_agent: Option<String>,
        pub permissions: Vec<String>,
        pub workspace: String,
        pub request_id: String,
        pub session_id: Option<String>,
    }

    impl SecurityContext {
        pub fn has_permission(&self, permission: &str) -> bool { ... }
        pub fn has_any_permission(&self, permissions: &[&str]) -> bool { ... }
        pub fn has_all_permissions(&self, permissions: &[&str]) -> bool { ... }
    }

### 2. Enhanced Authentication Middleware (`middleware/auth.rs`)

Purpose: JWT token validation with revocation checking.

Key Features:

- Bearer token extraction
- JWT signature validation (RS256)
- Expiry, issuer, audience checks
- Token revocation status
- Security context injection

Lines of Code: 245

Flow:

1. Extract `Authorization: Bearer <token>` header
2. Validate JWT with TokenValidator
3. Build SecurityContext
4. Inject into request extensions
5. Continue to next middleware or return 401

Error Responses:

- `401 Unauthorized`: Missing/invalid token, expired, revoked
- `403 Forbidden`: Insufficient permissions

### 3. MFA Verification Middleware (`middleware/mfa.rs`)

Purpose: Enforce MFA for sensitive operations.

Key Features:

- Path-based MFA requirements
- Method-based enforcement (all DELETEs)
- Production environment protection
- Clear error messages

Lines of Code: 290

MFA Required For:

- Production deployments (`/production/`, `/prod/`)
- All DELETE operations
- Server operations (POST, PUT, DELETE)
- Cluster operations (POST, PUT, DELETE)
- Batch submissions
- Rollback operations
- Configuration changes (POST, PUT, DELETE)
- Secret management
- User/role management

Example:

    fn requires_mfa(method: &str, path: &str) -> bool {
        if path.contains("/production/") { return true; }
        if method == "DELETE" { return true; }
        if path.contains("/deploy") { return true; }
        // ...
    }

### 4. Enhanced Authorization Middleware (`middleware/authz.rs`)

Purpose: Cedar policy evaluation with audit logging.

Key Features:

- Builds Cedar authorization request from HTTP request
- Maps HTTP methods to Cedar actions (GET→Read, POST→Create, etc.)
- Extracts resource types from paths
- Evaluates Cedar policies with context (MFA, IP, time, workspace)
- Logs all authorization decisions to audit log
- Non-blocking audit logging (tokio::spawn)

Lines of Code: 380

Resource Mapping:

    /api/v1/servers/srv-123    → Resource::Server("srv-123")
    /api/v1/taskserv/kubernetes → Resource::TaskService("kubernetes")
    /api/v1/cluster/prod        → Resource::Cluster("prod")
    /api/v1/config/settings     → Resource::Config("settings")

Action Mapping:

    GET    → Action::Read
    POST   → Action::Create
    PUT    → Action::Update
    DELETE → Action::Delete

### 5. Rate Limiting Middleware (`middleware/rate_limit.rs`)

Purpose: Prevent API abuse with per-IP rate limiting.

Key Features:

- Sliding window rate limiting
- Per-IP request tracking
- Configurable limits and windows
- Exempt IP support
- Automatic cleanup of old entries
- Statistics tracking

Lines of Code: 420

Configuration:

    pub struct RateLimitConfig {
        pub max_requests: u32,          // for example, 100
        pub window_duration: Duration,  // for example, 60 seconds
        pub exempt_ips: Vec<IpAddr>,    // for example, internal services
        pub enabled: bool,
    }

    // Default: 100 requests per minute

Statistics:

    pub struct RateLimitStats {
        pub total_ips: usize,      // Number of tracked IPs
        pub total_requests: u32,   // Total requests made
        pub limited_ips: usize,    // IPs that hit the limit
        pub config: RateLimitConfig,
    }

### 6. Security Integration Module (`security_integration.rs`)

Purpose: Helper module to integrate all security components.

Key Features:

- `SecurityComponents` struct grouping all middleware
- `SecurityConfig` for configuration
- `initialize()` method to set up all components
- `disabled()` method for development mode
- `apply_security_middleware()` helper for router setup

Lines of Code: 265

Usage Example:

    use provisioning_orchestrator::security_integration::{
        SecurityComponents, SecurityConfig
    };

    // Initialize security
    let config = SecurityConfig {
        public_key_path: PathBuf::from("keys/public.pem"),
        jwt_issuer: "control-center".to_string(),
        jwt_audience: "orchestrator".to_string(),
        cedar_policies_path: PathBuf::from("policies"),
        auth_enabled: true,
        authz_enabled: true,
        mfa_enabled: true,
        rate_limit_config: RateLimitConfig::new(100, 60),
    };

    let security = SecurityComponents::initialize(config, audit_logger).await?;

    // Apply to router
    let app = Router::new()
        .route("/api/v1/servers", post(create_server))
        .route("/api/v1/servers/:id", delete(delete_server));

    let secured_app = apply_security_middleware(app, &security);

## Integration with AppState

### Updated AppState Structure

    pub struct AppState {
        // Existing fields
        pub task_storage: Arc<dyn TaskStorage>,
        pub batch_coordinator: BatchCoordinator,
        pub dependency_resolver: DependencyResolver,
        pub state_manager: Arc<WorkflowStateManager>,
        pub monitoring_system: Arc<MonitoringSystem>,
        pub progress_tracker: Arc<ProgressTracker>,
        pub rollback_system: Arc<RollbackSystem>,
        pub test_orchestrator: Arc<TestOrchestrator>,
        pub dns_manager: Arc<DnsManager>,
        pub extension_manager: Arc<ExtensionManager>,
        pub oci_manager: Arc<OciManager>,
        pub service_orchestrator: Arc<ServiceOrchestrator>,
        pub audit_logger: Arc<AuditLogger>,
        pub args: Args,

        // NEW: Security components
        pub security: SecurityComponents,
    }

### Initialization in main.rs

    #[tokio::main]
    async fn main() -> Result<()> {
        let args = Args::parse();

        // Initialize AppState (creates audit_logger)
        let state = Arc::new(AppState::new(args).await?);

        // Initialize security components
        let security_config = SecurityConfig {
            public_key_path: PathBuf::from("keys/public.pem"),
            jwt_issuer: env::var("JWT_ISSUER").unwrap_or("control-center".to_string()),
            jwt_audience: "orchestrator".to_string(),
            cedar_policies_path: PathBuf::from("policies"),
            auth_enabled: env::var("AUTH_ENABLED").unwrap_or("true".to_string()) == "true",
            authz_enabled: env::var("AUTHZ_ENABLED").unwrap_or("true".to_string()) == "true",
            mfa_enabled: env::var("MFA_ENABLED").unwrap_or("true".to_string()) == "true",
            rate_limit_config: RateLimitConfig::new(
                env::var("RATE_LIMIT_MAX").unwrap_or("100".to_string()).parse().unwrap(),
                env::var("RATE_LIMIT_WINDOW").unwrap_or("60".to_string()).parse().unwrap(),
            ),
        };

        let security = SecurityComponents::initialize(
            security_config,
            state.audit_logger.clone()
        ).await?;

        // Public routes (no auth)
        let public_routes = Router::new()
            .route("/health", get(health_check));

        // Protected routes (full security chain)
        let protected_routes = Router::new()
            .route("/api/v1/servers", post(create_server))
            .route("/api/v1/servers/:id", delete(delete_server))
            .route("/api/v1/taskserv", post(create_taskserv))
            .route("/api/v1/cluster", post(create_cluster))
            // ... more routes
            ;

        // Apply security middleware to protected routes
        let secured_routes = apply_security_middleware(protected_routes, &security)
            .with_state(state.clone());

        // Combine routes
        let app = Router::new()
            .merge(public_routes)
            .merge(secured_routes)
            .layer(CorsLayer::permissive());

        // Start server
        let listener = tokio::net::TcpListener::bind("0.0.0.0:9090").await?;
        axum::serve(listener, app).await?;

        Ok(())
    }

## Protected Endpoints

### Endpoint Categories

Category | Example Endpoints | Auth Required | MFA Required | Cedar Policy
--- | --- | --- | --- | ---
Health | `/health` | ❌ | ❌ | ❌
Read-Only | `GET /api/v1/servers` | ✅ | ❌ | ✅
Server Mgmt | `POST /api/v1/servers` | ✅ | ❌ | ✅
Server Delete | `DELETE /api/v1/servers/:id` | ✅ | ✅ | ✅
Taskserv Mgmt | `POST /api/v1/taskserv` | ✅ | ❌ | ✅
Cluster Mgmt | `POST /api/v1/cluster` | ✅ | ✅ | ✅
Production | `POST /api/v1/production/*` | ✅ | ✅ | ✅
Batch Ops | `POST /api/v1/batch/submit` | ✅ | ✅ | ✅
Rollback | `POST /api/v1/rollback` | ✅ | ✅ | ✅
Config Write | `POST /api/v1/config` | ✅ | ✅ | ✅
Secrets | `GET /api/v1/secret/*` | ✅ | ✅ | ✅

## Complete Authentication Flow

### Step-by-Step Flow

    1. CLIENT REQUEST
       ├─ Headers:
       │  ├─ Authorization: Bearer <jwt_token>
       │  ├─ X-Forwarded-For: 192.168.1.100
       │  ├─ User-Agent: MyClient/1.0
       │  └─ X-MFA-Verified: true
       └─ Path: DELETE /api/v1/servers/prod-srv-01

    2. RATE LIMITING MIDDLEWARE
       ├─ Extract IP: 192.168.1.100
       ├─ Check limit: 45/100 requests in window
       ├─ Decision: ALLOW (under limit)
       └─ Continue →

    3. AUTHENTICATION MIDDLEWARE
       ├─ Extract Bearer token
       ├─ Validate JWT:
       │  ├─ Signature: ✅ Valid (RS256)
       │  ├─ Expiry: ✅ Valid until 2025-10-09 10:00:00
       │  ├─ Issuer: ✅ control-center
       │  ├─ Audience: ✅ orchestrator
       │  └─ Revoked: ✅ Not revoked
       ├─ Build SecurityContext:
       │  ├─ user_id: "user-456"
       │  ├─ workspace: "production"
       │  ├─ permissions: ["read", "write", "delete"]
       │  ├─ mfa_verified: true
       │  └─ ip_address: 192.168.1.100
       ├─ Decision: ALLOW (valid token)
       └─ Continue →

    4. MFA VERIFICATION MIDDLEWARE
       ├─ Check endpoint: DELETE /api/v1/servers/prod-srv-01
       ├─ Requires MFA: ✅ YES (DELETE operation)
       ├─ MFA status: ✅ Verified
       ├─ Decision: ALLOW (MFA verified)
       └─ Continue →

    5. AUTHORIZATION MIDDLEWARE
       ├─ Build Cedar request:
       │  ├─ Principal: User("user-456")
       │  ├─ Action: Delete
       │  ├─ Resource: Server("prod-srv-01")
       │  └─ Context:
       │     ├─ mfa_verified: true
       │     ├─ ip_address: "192.168.1.100"
       │     ├─ time: 2025-10-08T14:30:00Z
       │     └─ workspace: "production"
       ├─ Evaluate Cedar policies:
       │  ├─ Policy 1: Allow if user.role == "admin" ✅
       │  ├─ Policy 2: Allow if mfa_verified == true ✅
       │  └─ Policy 3: Deny if not business_hours ❌
       ├─ Decision: ALLOW (2 allow, 1 deny = allow)
       ├─ Log to audit: Authorization GRANTED
       └─ Continue →

    6. AUDIT LOGGING MIDDLEWARE
       ├─ Record:
       │  ├─ User: user-456 (IP: 192.168.1.100)
       │  ├─ Action: ServerDelete
       │  ├─ Resource: prod-srv-01
       │  ├─ Authorization: GRANTED
       │  ├─ MFA: Verified
       │  └─ Timestamp: 2025-10-08T14:30:00Z
       └─ Continue →

    7. PROTECTED HANDLER
       ├─ Execute business logic
       ├─ Delete server prod-srv-01
       └─ Return: 200 OK

    8. AUDIT LOGGING (Response)
       ├─ Update event:
       │  ├─ Status: 200 OK
       │  ├─ Duration: 1.234s
       │  └─ Result: SUCCESS
       └─ Write to audit log

    9. CLIENT RESPONSE
       └─ 200 OK: Server deleted successfully

## Configuration

### Environment Variables

    # JWT Configuration
    JWT_ISSUER=control-center
    JWT_AUDIENCE=orchestrator
    PUBLIC_KEY_PATH=/path/to/keys/public.pem

    # Cedar Policies
    CEDAR_POLICIES_PATH=/path/to/policies

    # Security Toggles
    AUTH_ENABLED=true
    AUTHZ_ENABLED=true
    MFA_ENABLED=true

    # Rate Limiting
    RATE_LIMIT_MAX=100
    RATE_LIMIT_WINDOW=60
    RATE_LIMIT_EXEMPT_IPS=10.0.0.1,10.0.0.2

    # Audit Logging
    AUDIT_ENABLED=true
    AUDIT_RETENTION_DAYS=365

### Development Mode

For development/testing, all security can be disabled:

    // In main.rs
    let security = if env::var("DEVELOPMENT_MODE").unwrap_or("false".to_string()) == "true" {
        SecurityComponents::disabled(audit_logger.clone())
    } else {
        SecurityComponents::initialize(security_config, audit_logger.clone()).await?
    };

## Testing

### Integration Tests

Location: `provisioning/platform/orchestrator/tests/security_integration_tests.rs`

Test Coverage:

- ✅ Rate limiting enforcement
- ✅ Rate limit statistics
- ✅ Exempt IP handling
- ✅ Authentication missing token
- ✅ MFA verification for sensitive operations
- ✅ Cedar policy evaluation
- ✅ Complete security flow
- ✅ Security components initialization
- ✅ Configuration defaults

Lines of Code: 340

Run Tests:

    cd provisioning/platform/orchestrator
    cargo test security_integration_tests

## File Summary

File | Purpose | Lines | Tests
--- | --- | --- | ---
`middleware/security_context.rs` | Security context builder | 275 | 8
`middleware/auth.rs` | JWT authentication | 245 | 5
`middleware/mfa.rs` | MFA verification | 290 | 15
`middleware/authz.rs` | Cedar authorization | 380 | 4
`middleware/rate_limit.rs` | Rate limiting | 420 | 8
`middleware/mod.rs` | Module exports | 25 | 0
`security_integration.rs` | Integration helpers | 265 | 2
`tests/security_integration_tests.rs` | Integration tests | 340 | 11
Total |  | 2,240 | 53

## Benefits

### Security

- ✅ Complete authentication flow with JWT validation
- ✅ MFA enforcement for sensitive operations
- ✅ Fine-grained authorization with Cedar policies
- ✅ Rate limiting prevents API abuse
- ✅ Complete audit trail for compliance

### Architecture

- ✅ Modular middleware design
- ✅ Clear separation of concerns
- ✅ Reusable security components
- ✅ Easy to test and maintain
- ✅ Configuration-driven behavior

### Operations

- ✅ Can enable/disable features independently
- ✅ Development mode for testing
- ✅ Comprehensive error messages
- ✅ Real-time statistics and monitoring
- ✅ Non-blocking audit logging

## Future Enhancements

1. Token Refresh: Automatic token refresh before expiry
2. IP Whitelisting: Additional IP-based access control
3. Geolocation: Block requests from specific countries
4. Advanced Rate Limiting: Per-user, per-endpoint limits
5. Session Management: Track active sessions, force logout
6. 2FA Integration: Direct integration with TOTP/SMS providers
7. Policy Hot Reload: Update Cedar policies without restart
8. Metrics Dashboard: Real-time security metrics visualization

## Related Documentation

- Cedar Policy Language
- JWT Token Management
- MFA Setup Guide
- Audit Log Format
- Rate Limiting Best Practices

## Version History

Version | Date | Changes
--- | --- | ---
1.0.0 | 2025-10-08 | Initial implementation

Maintained By: Security Team Review Cycle: Quarterly Last Reviewed: 2025-10-08