# MFA Admin Setup Guide - Production Operations Manual

**Document Version**: 1.0.0
**Last Updated**: 2025-10-08
**Target Audience**: Platform Administrators, Security Team
**Prerequisites**: Control Center deployed, admin user created

---

## 📋 Table of Contents

1. [Overview](#overview)
2. [MFA Requirements](#mfa-requirements)
3. [Admin Enrollment Process](#admin-enrollment-process)
4. [TOTP Setup (Authenticator Apps)](#totp-setup-authenticator-apps)
5. [WebAuthn Setup (Hardware Keys)](#webauthn-setup-hardware-keys)
6. [Enforcing MFA via Cedar Policies](#enforcing-mfa-via-cedar-policies)
7. [Backup Codes Management](#backup-codes-management)
8. [Recovery Procedures](#recovery-procedures)
9. [Troubleshooting](#troubleshooting)
10. [Best Practices](#best-practices)
11. [Audit and Compliance](#audit-and-compliance)

---

## Overview

### What is MFA

Multi-Factor Authentication (MFA) adds a second layer of security beyond passwords. Admins must provide:

1. **Something they know**: Password
2. **Something they have**: TOTP code (authenticator app) or WebAuthn device (YubiKey, Touch ID)

### Why MFA for Admins

Administrators have elevated privileges including:

- Server creation/deletion
- Production deployments
- Secret management
- User management
- Break-glass approval

**MFA protects against**:

- Password compromise (phishing, leaks, brute force)
- Unauthorized access to critical systems
- Compliance violations (SOC2, ISO 27001)

### MFA Methods Supported

| Method | Type | Examples | Recommended For |
|--------|------|----------|-----------------|
| **TOTP** | Software | Google Authenticator, Authy, 1Password | All admins (primary) |
| **WebAuthn/FIDO2** | Hardware | YubiKey, Touch ID, Windows Hello | High-security admins |
| **Backup Codes** | One-time | 10 single-use codes | Emergency recovery |

---

## MFA Requirements

### Mandatory MFA Enforcement

**All administrators MUST enable MFA** for:

- Production environment access
- Server creation/deletion operations
- Deployment to production clusters
- Secret access (KMS, dynamic secrets)
- Break-glass approval
- User management operations

### Grace Period

- **Development**: MFA optional (not recommended)
- **Staging**: MFA recommended, not enforced
- **Production**: MFA **mandatory** (enforced by Cedar policies)

### Timeline for Rollout

```plaintext
Week 1-2: Pilot Program
  ├─ Platform admins enable MFA
  ├─ Document issues and refine process
  └─ Create training materials

Week 3-4: Full Deployment
  ├─ All admins enable MFA
  ├─ Cedar policies enforce MFA for production
  └─ Monitor compliance

Week 5+: Maintenance
  ├─ Regular MFA device audits
  ├─ Backup code rotation
  └─ User support for MFA issues
```plaintext

---

## Admin Enrollment Process

### Step 1: Initial Login (Password Only)

```bash
# Login with username/password
provisioning login --user admin@example.com --workspace production

# Response (partial token, MFA not yet verified):
{
  "status": "mfa_required",
  "partial_token": "eyJhbGci...",  # Limited access token
  "message": "MFA enrollment required for production access"
}
```plaintext

**Partial token limitations**:

- Cannot access production resources
- Can only access MFA enrollment endpoints
- Expires in 15 minutes

### Step 2: Choose MFA Method

```bash
# Check available MFA methods
provisioning mfa methods

# Output:
Available MFA Methods:
  • TOTP (Authenticator apps) - Recommended for all users
  • WebAuthn (Hardware keys) - Recommended for high-security roles
  • Backup Codes - Emergency recovery only

# Check current MFA status
provisioning mfa status

# Output:
MFA Status:
  TOTP: Not enrolled
  WebAuthn: Not enrolled
  Backup Codes: Not generated
  MFA Required: Yes (production workspace)
```plaintext

### Step 3: Enroll MFA Device

Choose one or both methods (TOTP + WebAuthn recommended):

- [TOTP Setup](#totp-setup-authenticator-apps)
- [WebAuthn Setup](#webauthn-setup-hardware-keys)

### Step 4: Verify and Activate

After enrollment, login again with MFA:

```bash
# Login (returns partial token)
provisioning login --user admin@example.com --workspace production

# Verify MFA code (returns full access token)
provisioning mfa verify 123456

# Response:
{
  "status": "authenticated",
  "access_token": "eyJhbGci...",      # Full access token (15 min)
  "refresh_token": "eyJhbGci...",     # Refresh token (7 days)
  "mfa_verified": true,
  "expires_in": 900
}
```plaintext

---

## TOTP Setup (Authenticator Apps)

### Supported Authenticator Apps

| App | Platform | Notes |
|-----|----------|-------|
| **Google Authenticator** | iOS, Android | Simple, widely used |
| **Authy** | iOS, Android, Desktop | Cloud backup, multi-device |
| **1Password** | All platforms | Integrated with password manager |
| **Microsoft Authenticator** | iOS, Android | Enterprise integration |
| **Bitwarden** | All platforms | Open source |

### Step-by-Step TOTP Enrollment

#### 1. Initiate TOTP Enrollment

```bash
provisioning mfa totp enroll
```plaintext

**Output**:

```plaintext
╔════════════════════════════════════════════════════════════╗
║                   TOTP ENROLLMENT                          ║
╚════════════════════════════════════════════════════════════╝

Scan this QR code with your authenticator app:

█████████████████████████████████
█████████████████████████████████
████ ▄▄▄▄▄ █▀ █▀▀██ ▄▄▄▄▄ ████
████ █   █ █▀▄ ▀ ▄█ █   █ ████
████ █▄▄▄█ █ ▀▀ ▀▀█ █▄▄▄█ ████
████▄▄▄▄▄▄▄█ █▀█ ▀ █▄▄▄▄▄▄████
█████████████████████████████████
█████████████████████████████████

Manual entry (if QR code doesn't work):
  Secret: JBSWY3DPEHPK3PXP
  Account: admin@example.com
  Issuer: Provisioning Platform

TOTP Configuration:
  Algorithm: SHA1
  Digits: 6
  Period: 30 seconds
```plaintext

#### 2. Add to Authenticator App

**Option A: Scan QR Code (Recommended)**

1. Open authenticator app (Google Authenticator, Authy, etc.)
2. Tap "+" or "Add Account"
3. Select "Scan QR Code"
4. Point camera at QR code displayed in terminal
5. Account added automatically

**Option B: Manual Entry**

1. Open authenticator app
2. Tap "+" or "Add Account"
3. Select "Enter a setup key" or "Manual entry"
4. Enter:
   - **Account name**: <admin@example.com>
   - **Key**: `JBSWY3DPEHPK3PXP` (secret shown above)
   - **Type of key**: Time-based
5. Save account

#### 3. Verify TOTP Code

```bash
# Get current code from authenticator app (6 digits, changes every 30s)
# Example code: 123456

provisioning mfa totp verify 123456
```plaintext

**Success Response**:

```plaintext
✓ TOTP verified successfully!

Backup Codes (SAVE THESE SECURELY):
  1. A3B9-C2D7-E1F4
  2. G8H5-J6K3-L9M2
  3. N4P7-Q1R8-S5T2
  4. U6V3-W9X1-Y7Z4
  5. A2B8-C5D1-E9F3
  6. G7H4-J2K6-L8M1
  7. N3P9-Q5R2-S7T4
  8. U1V6-W3X8-Y2Z5
  9. A9B4-C7D2-E5F1
 10. G3H8-J1K5-L6M9

⚠ Store backup codes in a secure location (password manager, encrypted file)
⚠ Each code can only be used once
⚠ These codes allow access if you lose your authenticator device

TOTP enrollment complete. MFA is now active for your account.
```plaintext

#### 4. Save Backup Codes

**Critical**: Store backup codes in a secure location:

```bash
# Copy backup codes to password manager or encrypted file
# NEVER store in plaintext, email, or cloud storage

# Example: Store in encrypted file
provisioning mfa backup-codes --save-encrypted ~/secure/mfa-backup-codes.enc

# Or display again (requires existing MFA verification)
provisioning mfa backup-codes --show
```plaintext

#### 5. Test TOTP Login

```bash
# Logout to test full login flow
provisioning logout

# Login with password (returns partial token)
provisioning login --user admin@example.com --workspace production

# Get current TOTP code from authenticator app
# Verify with TOTP code (returns full access token)
provisioning mfa verify 654321

# ✓ Full access granted
```plaintext

---

## WebAuthn Setup (Hardware Keys)

### Supported WebAuthn Devices

| Device Type | Examples | Security Level |
|-------------|----------|----------------|
| **USB Security Keys** | YubiKey 5, SoloKey, Titan Key | Highest |
| **NFC Keys** | YubiKey 5 NFC, Google Titan | High (mobile compatible) |
| **Biometric** | Touch ID (macOS), Windows Hello, Face ID | High (convenience) |
| **Platform Authenticators** | Built-in laptop/phone biometrics | Medium-High |

### Step-by-Step WebAuthn Enrollment

#### 1. Check WebAuthn Support

```bash
# Verify WebAuthn support on your system
provisioning mfa webauthn check

# Output:
WebAuthn Support:
  ✓ Browser: Chrome 120.0 (WebAuthn supported)
  ✓ Platform: macOS 14.0 (Touch ID available)
  ✓ USB: YubiKey 5 NFC detected
```plaintext

#### 2. Initiate WebAuthn Registration

```bash
provisioning mfa webauthn register --device-name "YubiKey-Admin-Primary"
```plaintext

**Output**:

```plaintext
╔════════════════════════════════════════════════════════════╗
║               WEBAUTHN DEVICE REGISTRATION                 ║
╚════════════════════════════════════════════════════════════╝

Device Name: YubiKey-Admin-Primary
Relying Party: provisioning.example.com

⚠ Please insert your security key and touch it when it blinks

Waiting for device interaction...
```plaintext

#### 3. Complete Device Registration

**For USB Security Keys (YubiKey, SoloKey)**:

1. Insert USB key into computer
2. Terminal shows "Touch your security key"
3. Touch the gold/silver contact on the key (it will blink)
4. Registration completes

**For Touch ID (macOS)**:

1. Terminal shows "Touch ID prompt will appear"
2. Touch ID dialog appears on screen
3. Place finger on Touch ID sensor
4. Registration completes

**For Windows Hello**:

1. Terminal shows "Windows Hello prompt"
2. Windows Hello biometric prompt appears
3. Complete biometric scan (fingerprint/face)
4. Registration completes

**Success Response**:

```plaintext
✓ WebAuthn device registered successfully!

Device Details:
  Name: YubiKey-Admin-Primary
  Type: USB Security Key
  AAGUID: 2fc0579f-8113-47ea-b116-bb5a8 d9202a
  Credential ID: kZj8C3bx...
  Registered: 2025-10-08T14:32:10Z

You can now use this device for authentication.
```plaintext

#### 4. Register Additional Devices (Optional)

**Best Practice**: Register 2+ WebAuthn devices (primary + backup)

```bash
# Register backup YubiKey
provisioning mfa webauthn register --device-name "YubiKey-Admin-Backup"

# Register Touch ID (for convenience on personal laptop)
provisioning mfa webauthn register --device-name "MacBook-TouchID"
```plaintext

#### 5. List Registered Devices

```bash
provisioning mfa webauthn list

# Output:
Registered WebAuthn Devices:

  1. YubiKey-Admin-Primary (USB Security Key)
     Registered: 2025-10-08T14:32:10Z
     Last Used: 2025-10-08T14:32:10Z

  2. YubiKey-Admin-Backup (USB Security Key)
     Registered: 2025-10-08T14:35:22Z
     Last Used: Never

  3. MacBook-TouchID (Platform Authenticator)
     Registered: 2025-10-08T14:40:15Z
     Last Used: 2025-10-08T15:20:05Z

Total: 3 devices
```plaintext

#### 6. Test WebAuthn Login

```bash
# Logout to test
provisioning logout

# Login with password (partial token)
provisioning login --user admin@example.com --workspace production

# Authenticate with WebAuthn
provisioning mfa webauthn verify

# Output:
⚠ Insert and touch your security key
[Touch YubiKey when it blinks]

✓ WebAuthn verification successful
✓ Full access granted
```plaintext

---

## Enforcing MFA via Cedar Policies

### Production MFA Enforcement Policy

**Location**: `provisioning/config/cedar-policies/production.cedar`

```cedar
// Production operations require MFA verification
permit (
  principal,
  action in [
    Action::"server:create",
    Action::"server:delete",
    Action::"cluster:deploy",
    Action::"secret:read",
    Action::"user:manage"
  ],
  resource in Environment::"production"
) when {
  // MFA MUST be verified
  context.mfa_verified == true
};

// Admin role requires MFA for ALL production actions
permit (
  principal in Role::"Admin",
  action,
  resource in Environment::"production"
) when {
  context.mfa_verified == true
};

// Break-glass approval requires MFA
permit (
  principal,
  action == Action::"break_glass:approve",
  resource
) when {
  context.mfa_verified == true &&
  principal.role in [Role::"Admin", Role::"SecurityLead"]
};
```plaintext

### Development/Staging Policies (MFA Recommended, Not Required)

**Location**: `provisioning/config/cedar-policies/development.cedar`

```cedar
// Development: MFA recommended but not enforced
permit (
  principal,
  action,
  resource in Environment::"dev"
) when {
  // MFA not required for dev, but logged if missing
  true
};

// Staging: MFA recommended for destructive operations
permit (
  principal,
  action in [Action::"server:delete", Action::"cluster:delete"],
  resource in Environment::"staging"
) when {
  // Allow without MFA but log warning
  context.mfa_verified == true || context has mfa_warning_acknowledged
};
```plaintext

### Policy Deployment

```bash
# Validate Cedar policies
provisioning cedar validate --policies config/cedar-policies/

# Test policies with sample requests
provisioning cedar test --policies config/cedar-policies/ \
  --test-file tests/cedar-test-cases.yaml

# Deploy to production (requires MFA + approval)
provisioning cedar deploy production --policies config/cedar-policies/production.cedar

# Verify policy is active
provisioning cedar status production
```plaintext

### Testing MFA Enforcement

```bash
# Test 1: Production access WITHOUT MFA (should fail)
provisioning login --user admin@example.com --workspace production
provisioning server create web-01 --plan medium --check

# Expected: Authorization denied (MFA not verified)

# Test 2: Production access WITH MFA (should succeed)
provisioning login --user admin@example.com --workspace production
provisioning mfa verify 123456
provisioning server create web-01 --plan medium --check

# Expected: Server creation initiated
```plaintext

---

## Backup Codes Management

### Generating Backup Codes

Backup codes are automatically generated during first MFA enrollment:

```bash
# View existing backup codes (requires MFA verification)
provisioning mfa backup-codes --show

# Regenerate backup codes (invalidates old ones)
provisioning mfa backup-codes --regenerate

# Output:
⚠ WARNING: Regenerating backup codes will invalidate all existing codes.
Continue? (yes/no): yes

New Backup Codes:
  1. X7Y2-Z9A4-B6C1
  2. D3E8-F5G2-H9J4
  3. K6L1-M7N3-P8Q2
  4. R4S9-T6U1-V3W7
  5. X2Y5-Z8A3-B9C4
  6. D7E1-F4G6-H2J8
  7. K5L9-M3N6-P1Q4
  8. R8S2-T5U7-V9W3
  9. X4Y6-Z1A8-B3C5
 10. D9E2-F7G4-H6J1

✓ Backup codes regenerated successfully
⚠ Save these codes in a secure location
```plaintext

### Using Backup Codes

**When to use backup codes**:

- Lost authenticator device (phone stolen, broken)
- WebAuthn key not available (traveling, left at office)
- Authenticator app not working (time sync issue)

**Login with backup code**:

```bash
# Login (partial token)
provisioning login --user admin@example.com --workspace production

# Use backup code instead of TOTP/WebAuthn
provisioning mfa verify-backup X7Y2-Z9A4-B6C1

# Output:
✓ Backup code verified
⚠ Backup code consumed (9 remaining)
⚠ Enroll a new MFA device as soon as possible
✓ Full access granted (temporary)
```plaintext

### Backup Code Storage Best Practices

**✅ DO**:

- Store in password manager (1Password, Bitwarden, LastPass)
- Print and store in physical safe
- Encrypt and store in secure cloud storage (with encryption key stored separately)
- Share with trusted IT team member (encrypted)

**❌ DON'T**:

- Email to yourself
- Store in plaintext file on laptop
- Save in browser notes/bookmarks
- Share via Slack/Teams/unencrypted chat
- Screenshot and save to Photos

**Example: Encrypted Storage**:

```bash
# Encrypt backup codes with Age
provisioning mfa backup-codes --export | \
  age -p -o ~/secure/mfa-backup-codes.age

# Decrypt when needed
age -d ~/secure/mfa-backup-codes.age
```plaintext

---

## Recovery Procedures

### Scenario 1: Lost Authenticator Device (TOTP)

**Situation**: Phone stolen/broken, authenticator app not accessible

**Recovery Steps**:

```bash
# Step 1: Use backup code to login
provisioning login --user admin@example.com --workspace production
provisioning mfa verify-backup X7Y2-Z9A4-B6C1

# Step 2: Remove old TOTP enrollment
provisioning mfa totp unenroll

# Step 3: Enroll new TOTP device
provisioning mfa totp enroll
# [Scan QR code with new phone/authenticator app]
provisioning mfa totp verify 654321

# Step 4: Generate new backup codes
provisioning mfa backup-codes --regenerate
```plaintext

### Scenario 2: Lost WebAuthn Key (YubiKey)

**Situation**: YubiKey lost, stolen, or damaged

**Recovery Steps**:

```bash
# Step 1: Login with alternative method (TOTP or backup code)
provisioning login --user admin@example.com --workspace production
provisioning mfa verify 123456  # TOTP from authenticator app

# Step 2: List registered WebAuthn devices
provisioning mfa webauthn list

# Step 3: Remove lost device
provisioning mfa webauthn remove "YubiKey-Admin-Primary"

# Output:
⚠ Remove WebAuthn device "YubiKey-Admin-Primary"?
This cannot be undone. (yes/no): yes

✓ Device removed

# Step 4: Register new WebAuthn device
provisioning mfa webauthn register --device-name "YubiKey-Admin-Replacement"
```plaintext

### Scenario 3: All MFA Methods Lost

**Situation**: Lost phone (TOTP), lost YubiKey, no backup codes

**Recovery Steps** (Requires Admin Assistance):

```bash
# User contacts Security Team / Platform Admin

# Admin performs MFA reset (requires 2+ admin approval)
provisioning admin mfa-reset admin@example.com \
  --reason "Employee lost all MFA devices (phone + YubiKey)" \
  --ticket SUPPORT-12345

# Output:
⚠ MFA Reset Request Created

Reset Request ID: MFA-RESET-20251008-001
User: admin@example.com
Reason: Employee lost all MFA devices (phone + YubiKey)
Ticket: SUPPORT-12345

Required Approvals: 2
Approvers: 0/2

# Two other admins approve (with their own MFA)
provisioning admin mfa-reset approve MFA-RESET-20251008-001 \
  --reason "Verified via video call + employee badge"

# After 2 approvals, MFA is reset
✓ MFA reset approved (2/2 approvals)
✓ User admin@example.com can now re-enroll MFA devices

# User re-enrolls TOTP and WebAuthn
provisioning mfa totp enroll
provisioning mfa webauthn register --device-name "YubiKey-New"
```plaintext

### Scenario 4: Backup Codes Depleted

**Situation**: Used 9 out of 10 backup codes

**Recovery Steps**:

```bash
# Login with last backup code
provisioning login --user admin@example.com --workspace production
provisioning mfa verify-backup D9E2-F7G4-H6J1

# Output:
⚠ WARNING: This is your LAST backup code!
✓ Backup code verified
⚠ Regenerate backup codes immediately!

# Immediately regenerate backup codes
provisioning mfa backup-codes --regenerate

# Save new codes securely
```plaintext

---

## Troubleshooting

### Issue 1: "Invalid TOTP code" Error

**Symptoms**:

```plaintext
provisioning mfa verify 123456
✗ Error: Invalid TOTP code
```plaintext

**Possible Causes**:

1. **Time sync issue** (most common)
2. Wrong secret key entered during enrollment
3. Code expired (30-second window)

**Solutions**:

```bash
# Check time sync (device clock must be accurate)
# macOS:
sudo sntp -sS time.apple.com

# Linux:
sudo ntpdate pool.ntp.org

# Verify TOTP configuration
provisioning mfa totp status

# Output:
TOTP Configuration:
  Algorithm: SHA1
  Digits: 6
  Period: 30 seconds
  Time Window: ±1 period (90 seconds total)

# Check system time vs NTP
date && curl -s http://worldtimeapi.org/api/ip | grep datetime

# If time is off by >30 seconds, sync time and retry
```plaintext

### Issue 2: WebAuthn Not Detected

**Symptoms**:

```plaintext
provisioning mfa webauthn register
✗ Error: No WebAuthn authenticator detected
```plaintext

**Solutions**:

```bash
# Check USB connection (for hardware keys)
# macOS:
system_profiler SPUSBDataType | grep -i yubikey

# Linux:
lsusb | grep -i yubico

# Check browser WebAuthn support
provisioning mfa webauthn check

# Try different USB port (USB-A vs USB-C)

# For Touch ID: Ensure finger is enrolled in System Preferences
# For Windows Hello: Ensure biometrics are configured in Settings
```plaintext

### Issue 3: "MFA Required" Despite Verification

**Symptoms**:

```plaintext
provisioning server create web-01
✗ Error: Authorization denied (MFA verification required)
```plaintext

**Cause**: Access token expired (15 min) or MFA verification not in token claims

**Solution**:

```bash
# Check token expiration
provisioning auth status

# Output:
Authentication Status:
  Logged in: Yes
  User: admin@example.com
  Access Token: Expired (issued 16 minutes ago)
  MFA Verified: Yes (but token expired)

# Re-authenticate (will prompt for MFA again)
provisioning login --user admin@example.com --workspace production
provisioning mfa verify 654321

# Verify MFA claim in token
provisioning auth decode-token

# Output (JWT claims):
{
  "sub": "admin@example.com",
  "role": "Admin",
  "mfa_verified": true,  # ← Must be true
  "mfa_method": "totp",
  "iat": 1696766400,
  "exp": 1696767300
}
```plaintext

### Issue 4: QR Code Not Displaying

**Symptoms**: QR code appears garbled or doesn't display in terminal

**Solutions**:

```bash
# Use manual entry instead
provisioning mfa totp enroll --manual

# Output (no QR code):
Manual TOTP Setup:
  Secret: JBSWY3DPEHPK3PXP
  Account: admin@example.com
  Issuer: Provisioning Platform

Enter this secret manually in your authenticator app.

# Or export QR code to image file
provisioning mfa totp enroll --qr-image ~/mfa-qr.png
open ~/mfa-qr.png  # View in image viewer
```plaintext

### Issue 5: Backup Code Not Working

**Symptoms**:

```plaintext
provisioning mfa verify-backup X7Y2-Z9A4-B6C1
✗ Error: Invalid or already used backup code
```plaintext

**Possible Causes**:

1. Code already used (single-use only)
2. Backup codes regenerated (old codes invalidated)
3. Typo in code entry

**Solutions**:

```bash
# Check backup code status (requires alternative login method)
provisioning mfa backup-codes --status

# Output:
Backup Codes Status:
  Total Generated: 10
  Used: 3
  Remaining: 7
  Last Used: 2025-10-05T10:15:30Z

# Contact admin for MFA reset if all codes used
# Or use alternative MFA method (TOTP, WebAuthn)
```plaintext

---

## Best Practices

### For Individual Admins

#### 1. Use Multiple MFA Methods

**✅ Recommended Setup**:

- **Primary**: TOTP (Google Authenticator, Authy)
- **Backup**: WebAuthn (YubiKey or Touch ID)
- **Emergency**: Backup codes (stored securely)

```bash
# Enroll all three
provisioning mfa totp enroll
provisioning mfa webauthn register --device-name "YubiKey-Primary"
provisioning mfa backup-codes --save-encrypted ~/secure/codes.enc
```plaintext

#### 2. Secure Backup Code Storage

```bash
# Store in password manager (1Password example)
provisioning mfa backup-codes --show | \
  op item create --category "Secure Note" \
    --title "Provisioning MFA Backup Codes" \
    --vault "Work"

# Or encrypted file
provisioning mfa backup-codes --export | \
  age -p -o ~/secure/mfa-backup-codes.age
```plaintext

#### 3. Regular Device Audits

```bash
# Monthly: Review registered devices
provisioning mfa devices --all

# Remove unused/old devices
provisioning mfa webauthn remove "Old-YubiKey"
provisioning mfa totp remove "Old-Phone"
```plaintext

#### 4. Test Recovery Procedures

```bash
# Quarterly: Test backup code login
provisioning logout
provisioning login --user admin@example.com --workspace dev
provisioning mfa verify-backup [test-code]

# Verify backup codes are accessible
cat ~/secure/mfa-backup-codes.enc | age -d
```plaintext

### For Security Teams

#### 1. MFA Enrollment Verification

```bash
# Generate MFA enrollment report
provisioning admin mfa-report --format csv > mfa-enrollment.csv

# Output (CSV):
# User,MFA_Enabled,TOTP,WebAuthn,Backup_Codes,Last_MFA_Login,Role
# admin@example.com,Yes,Yes,Yes,10,2025-10-08T14:00:00Z,Admin
# dev@example.com,No,No,No,0,Never,Developer
```plaintext

#### 2. Enforce MFA Deadlines

```bash
# Set MFA enrollment deadline
provisioning admin mfa-deadline set 2025-11-01 \
  --roles Admin,Developer \
  --environment production

# Send reminder emails
provisioning admin mfa-remind \
  --users-without-mfa \
  --template "MFA enrollment required by Nov 1"
```plaintext

#### 3. Monitor MFA Usage

```bash
# Audit: Find production logins without MFA
provisioning audit query \
  --action "auth:login" \
  --filter 'mfa_verified == false && environment == "production"' \
  --since 7d

# Alert on repeated MFA failures
provisioning monitoring alert create \
  --name "MFA Brute Force" \
  --condition "mfa_failures > 5 in 5 min" \
  --action "notify security-team"
```plaintext

#### 4. MFA Reset Policy

**MFA Reset Requirements**:

- User verification (video call + ID check)
- Support ticket created (incident tracking)
- 2+ admin approvals (different teams)
- Time-limited reset window (24 hours)
- Mandatory re-enrollment before production access

```bash
# MFA reset workflow
provisioning admin mfa-reset create user@example.com \
  --reason "Lost all devices" \
  --ticket SUPPORT-12345 \
  --expires-in 24h

# Requires 2 approvals
provisioning admin mfa-reset approve MFA-RESET-001
```plaintext

### For Platform Admins

#### 1. Cedar Policy Best Practices

```cedar
// Require MFA for high-risk actions
permit (
  principal,
  action in [
    Action::"server:delete",
    Action::"cluster:delete",
    Action::"secret:delete",
    Action::"user:delete"
  ],
  resource
) when {
  context.mfa_verified == true &&
  context.mfa_age_seconds < 300  // MFA verified within last 5 minutes
};
```plaintext

#### 2. MFA Grace Periods (For Rollout)

```bash
# Development: No MFA required
export PROVISIONING_MFA_REQUIRED=false

# Staging: MFA recommended (warnings only)
export PROVISIONING_MFA_REQUIRED=warn

# Production: MFA mandatory (strict enforcement)
export PROVISIONING_MFA_REQUIRED=true
```plaintext

#### 3. Backup Admin Account

**Emergency Admin** (break-glass scenario):

- Separate admin account with MFA enrollment
- Credentials stored in physical safe
- Only used when primary admins locked out
- Requires incident report after use

```bash
# Create emergency admin
provisioning admin create emergency-admin@example.com \
  --role EmergencyAdmin \
  --mfa-required true \
  --max-concurrent-sessions 1

# Print backup codes and store in safe
provisioning mfa backup-codes --show --user emergency-admin@example.com > emergency-codes.txt
# [Print and store in physical safe]
```plaintext

---

## Audit and Compliance

### MFA Audit Logging

All MFA events are logged to the audit system:

```bash
# View MFA enrollment events
provisioning audit query \
  --action-type "mfa:*" \
  --since 30d

# Output (JSON):
[
  {
    "timestamp": "2025-10-08T14:32:10Z",
    "action": "mfa:totp:enroll",
    "user": "admin@example.com",
    "result": "success",
    "device_type": "totp",
    "ip_address": "203.0.113.42"
  },
  {
    "timestamp": "2025-10-08T14:35:22Z",
    "action": "mfa:webauthn:register",
    "user": "admin@example.com",
    "result": "success",
    "device_name": "YubiKey-Admin-Primary",
    "ip_address": "203.0.113.42"
  }
]
```plaintext

### Compliance Reports

#### SOC2 Compliance (Access Control)

```bash
# Generate SOC2 access control report
provisioning compliance report soc2 \
  --control "CC6.1" \
  --period "2025-Q3"

# Output:
SOC2 Trust Service Criteria - CC6.1 (Logical Access)

MFA Enforcement:
  ✓ MFA enabled for 100% of production admins (15/15)
  ✓ MFA verified for 98.7% of production logins (2,453/2,485)
  ✓ MFA policies enforced via Cedar authorization
  ✓ Failed MFA attempts logged and monitored

Evidence:
  - Cedar policy: production.cedar (lines 15-25)
  - Audit logs: mfa-verification-logs-2025-q3.json
  - Enrollment report: mfa-enrollment-status.csv
```plaintext

#### ISO 27001 Compliance (A.9.4.2 - Secure Log-on)

```bash
# ISO 27001 A.9.4.2 compliance report
provisioning compliance report iso27001 \
  --control "A.9.4.2" \
  --format pdf \
  --output iso27001-a942-mfa-report.pdf

# Report Sections:
# 1. MFA Implementation Details
# 2. Enrollment Procedures
# 3. Audit Trail
# 4. Policy Enforcement
# 5. Recovery Procedures
```plaintext

#### GDPR Compliance (MFA Data Handling)

```bash
# GDPR data subject request (MFA data export)
provisioning compliance gdpr export admin@example.com \
  --include mfa

# Output (JSON):
{
  "user": "admin@example.com",
  "mfa_data": {
    "totp_enrolled": true,
    "totp_enrollment_date": "2025-10-08T14:32:10Z",
    "webauthn_devices": [
      {
        "name": "YubiKey-Admin-Primary",
        "registered": "2025-10-08T14:35:22Z",
        "last_used": "2025-10-08T16:20:05Z"
      }
    ],
    "backup_codes_remaining": 7,
    "mfa_login_history": [...]  # Last 90 days
  }
}

# GDPR deletion (MFA data removal after account deletion)
provisioning compliance gdpr delete admin@example.com --include-mfa
```plaintext

### MFA Metrics Dashboard

```bash
# Generate MFA metrics
provisioning admin mfa-metrics --period 30d

# Output:
MFA Metrics (Last 30 Days)

Enrollment:
  Total Users: 42
  MFA Enabled: 38 (90.5%)
  TOTP Only: 22 (57.9%)
  WebAuthn Only: 3 (7.9%)
  Both TOTP + WebAuthn: 13 (34.2%)
  No MFA: 4 (9.5%) ⚠

Authentication:
  Total Logins: 3,847
  MFA Verified: 3,802 (98.8%)
  MFA Failed: 45 (1.2%)
  Backup Code Used: 7 (0.2%)

Devices:
  TOTP Devices: 35
  WebAuthn Devices: 47
  Backup Codes Remaining (avg): 8.3

Incidents:
  MFA Resets: 2
  Lost Devices: 3
  Lockouts: 1
```plaintext

---

## Quick Reference Card

### Daily Admin Operations

```bash
# Login with MFA
provisioning login --user admin@example.com --workspace production
provisioning mfa verify 123456

# Check MFA status
provisioning mfa status

# View registered devices
provisioning mfa devices
```plaintext

### MFA Management

```bash
# TOTP
provisioning mfa totp enroll              # Enroll TOTP
provisioning mfa totp verify 123456       # Verify TOTP code
provisioning mfa totp unenroll            # Remove TOTP

# WebAuthn
provisioning mfa webauthn register --device-name "YubiKey"  # Register key
provisioning mfa webauthn list            # List devices
provisioning mfa webauthn remove "YubiKey"  # Remove device

# Backup Codes
provisioning mfa backup-codes --show      # View codes
provisioning mfa backup-codes --regenerate  # Generate new codes
provisioning mfa verify-backup X7Y2-Z9A4-B6C1  # Use backup code
```plaintext

### Emergency Procedures

```bash
# Lost device recovery (use backup code)
provisioning login --user admin@example.com
provisioning mfa verify-backup [code]
provisioning mfa totp enroll  # Re-enroll new device

# MFA reset (admin only)
provisioning admin mfa-reset user@example.com --reason "Lost all devices"

# Check MFA compliance
provisioning admin mfa-report
```plaintext

---

## Summary Checklist

### For New Admins

- [ ] Complete initial login with password
- [ ] Enroll TOTP (Google Authenticator, Authy)
- [ ] Verify TOTP code successfully
- [ ] Save backup codes in password manager
- [ ] Register WebAuthn device (YubiKey or Touch ID)
- [ ] Test full login flow with MFA
- [ ] Store backup codes in secure location
- [ ] Verify production access works with MFA

### For Security Team

- [ ] Deploy Cedar MFA enforcement policies
- [ ] Verify 100% admin MFA enrollment
- [ ] Configure MFA audit logging
- [ ] Setup MFA compliance reports (SOC2, ISO 27001)
- [ ] Document MFA reset procedures
- [ ] Train admins on MFA usage
- [ ] Create emergency admin account (break-glass)
- [ ] Schedule quarterly MFA audits

### For Platform Team

- [ ] Configure MFA settings in `config/mfa.toml`
- [ ] Deploy Cedar policies with MFA requirements
- [ ] Setup monitoring for MFA failures
- [ ] Configure alerts for MFA bypass attempts
- [ ] Document MFA architecture in ADR
- [ ] Test MFA enforcement in all environments
- [ ] Verify audit logs capture MFA events
- [ ] Create runbooks for MFA incidents

---

## Support and Resources

### Documentation

- **MFA Implementation**: `/docs/architecture/MFA_IMPLEMENTATION_SUMMARY.md`
- **Cedar Policies**: `/docs/operations/CEDAR_POLICIES_PRODUCTION_GUIDE.md`
- **Break-Glass**: `/docs/operations/BREAK_GLASS_TRAINING_GUIDE.md`
- **Audit Logging**: `/docs/architecture/AUDIT_LOGGING_IMPLEMENTATION.md`

### Configuration Files

- **MFA Config**: `provisioning/config/mfa.toml`
- **Cedar Policies**: `provisioning/config/cedar-policies/production.cedar`
- **Control Center**: `provisioning/platform/control-center/config.toml`

### CLI Help

```bash
provisioning mfa help          # MFA command help
provisioning mfa totp --help   # TOTP-specific help
provisioning mfa webauthn --help  # WebAuthn-specific help
```plaintext

### Contact

- **Security Team**: <security@example.com>
- **Platform Team**: <platform@example.com>
- **Support Ticket**: <https://support.example.com>

---

**Document Status**: ✅ Complete
**Review Date**: 2025-11-08
**Maintained By**: Security Team, Platform Team
