# SSH Temporal Key Management System\n\n## Overview\n\nThe SSH Temporal Key Management System provides automated generation, deployment, and cleanup of short-lived SSH keys\nfor secure server access. It eliminates the need for static SSH keys by generating keys on-demand with automatic expiration.\n\n## Features\n\n### Core Features\n\n- **Short-Lived Keys**: Keys expire automatically after a configurable TTL (default: 1 hour)\n- **Multiple Key Types**:\n  - Dynamic Key Pairs (Ed25519)\n  - Vault OTP (One-Time Password)\n  - Vault CA-Signed Certificates\n- **Automatic Cleanup**: Background task removes expired keys from servers\n- **Audit Trail**: All key operations are logged\n- **REST API**: HTTP endpoints for integration\n- **Nushell CLI**: User-friendly command-line interface\n\n### Security Features\n\n- ✅ Ed25519 keys (modern, secure algorithm)\n- ✅ Automatic expiration and cleanup\n- ✅ Private keys never stored on disk (only in memory)\n- ✅ Vault integration for enterprise scenarios\n- ✅ SSH fingerprint tracking\n- ✅ Per-key audit logging\n\n## Architecture\n\n```\n┌─────────────────────────────────────────────────\n────────────┐\n│                   SSH Key Manager                           │\n├─────────────────────────────────────────────────\n────────────┤\n│                                                             │\n│  ┌──────────────┐  ┌──────────────┐  \n┌──────────────┐       │\n│  │ Key Generator│  │ Key Deployer │  │   Temporal   │       │\n│  │  (Ed25519)   │  │ (SSH Deploy) │  │   Manager    │       │\n│  └──────────────┘  └──────────────┘  \n└──────────────┘       │\n│                                                             │\n│  ┌──────────────┐  ┌──────────────┐                         │\n│  │    Vault     │  │ Authorized   │                         │\n│  │ SSH Engine   │  │ Keys Manager │                         │\n│  └──────────────┘  └──────────────┘                         │\n│                                                             │\n└─────────────────────────────────────────────────\n────────────┘\n         │                    │                    │\n         ▼                    ▼                    ▼\n    REST API            Nushell CLI          Background Tasks\n```\n\n## Key Types\n\n### 1. Dynamic Key Pairs (Default)\n\nGenerated on-demand Ed25519 keys that are automatically deployed and cleaned up.\n\n**Use Case**: Quick SSH access without Vault infrastructure\n\n**Example**:\n\n```\nssh generate-key server.example.com --user root --ttl 30min\n```\n\n### 2. Vault OTP (One-Time Password)\n\nVault generates a one-time password for SSH authentication.\n\n**Use Case**: Single-use SSH access with centralized authentication\n\n**Requirements**: Vault with SSH secrets engine in OTP mode\n\n**Example**:\n\n```\nssh generate-key server.example.com --type otp --ip 192.168.1.100\n```\n\n### 3. Vault CA-Signed Certificates\n\nVault acts as SSH CA, signing user public keys with short TTL.\n\n**Use Case**: Enterprise scenarios with SSH CA infrastructure\n\n**Requirements**: Vault with SSH secrets engine in CA mode\n\n**Example**:\n\n```\nssh generate-key server.example.com --type ca --principal admin --ttl 1hr\n```\n\n## REST API Endpoints\n\nBase URL: `http://localhost:9090`\n\n### Generate SSH Key\n\n```\nPOST /api/v1/ssh/generate\n\n{\n  "key_type": "dynamickeypair",  // or "otp", "certificate"\n  "user": "root",\n  "target_server": "server.example.com",\n  "ttl_seconds": 3600,\n  "allowed_ip": "192.168.1.100",  // optional, for OTP\n  "principal": "admin"             // optional, for CA\n}\n\nResponse:\n{\n  "success": true,\n  "data": {\n    "id": "uuid",\n    "key_type": "dynamickeypair",\n    "public_key": "ssh-ed25519 AAAA...",\n    "private_key": "-----BEGIN OPENSSH PRIVATE KEY-----...",\n    "fingerprint": "SHA256:...",\n    "user": "root",\n    "target_server": "server.example.com",\n    "created_at": "2024-01-01T00:00:00Z",\n    "expires_at": "2024-01-01T01:00:00Z",\n    "deployed": false\n  }\n}\n```\n\n### Deploy SSH Key\n\n```\nPOST /api/v1/ssh/{key_id}/deploy\n\nResponse:\n{\n  "success": true,\n  "data": {\n    "key_id": "uuid",\n    "server": "server.example.com",\n    "success": true,\n    "deployed_at": "2024-01-01T00:00:00Z"\n  }\n}\n```\n\n### List SSH Keys\n\n```\nGET /api/v1/ssh/keys\n\nResponse:\n{\n  "success": true,\n  "data": [\n    {\n      "id": "uuid",\n      "key_type": "dynamickeypair",\n      "user": "root",\n      "target_server": "server.example.com",\n      "expires_at": "2024-01-01T01:00:00Z",\n      "deployed": true\n    }\n  ]\n}\n```\n\n### Revoke SSH Key\n\n```\nPOST /api/v1/ssh/{key_id}/revoke\n\nResponse:\n{\n  "success": true,\n  "data": "Key uuid revoked successfully"\n}\n```\n\n### Get SSH Key\n\n```\nGET /api/v1/ssh/{key_id}\n\nResponse:\n{\n  "success": true,\n  "data": {\n    "id": "uuid",\n    "key_type": "dynamickeypair",\n    ...\n  }\n}\n```\n\n### Cleanup Expired Keys\n\n```\nPOST /api/v1/ssh/cleanup\n\nResponse:\n{\n  "success": true,\n  "data": {\n    "cleaned_count": 5,\n    "cleaned_key_ids": ["uuid1", "uuid2", ...]\n  }\n}\n```\n\n### Get Statistics\n\n```\nGET /api/v1/ssh/stats\n\nResponse:\n{\n  "success": true,\n  "data": {\n    "total_generated": 42,\n    "active_keys": 10,\n    "expired_keys": 32,\n    "keys_by_type": {\n      "dynamic": 35,\n      "otp": 5,\n      "certificate": 2\n    },\n    "last_cleanup_count": 5,\n    "last_cleanup_at": "2024-01-01T00:00:00Z"\n  }\n}\n```\n\n## Nushell CLI Commands\n\n### Generate Key\n\n```\nssh generate-key <server> [options]\n\nOptions:\n  --user <name>                 SSH user (default: root)\n  --ttl <duration>              Key lifetime (default: 1hr)\n  --type <ca|otp|dynamic>       Key type (default: dynamic)\n  --ip <address>                Allowed IP (OTP mode)\n  --principal <name>            Principal (CA mode)\n\nExamples:\n  ssh generate-key server.example.com\n  ssh generate-key server.example.com --user deploy --ttl 30min\n  ssh generate-key server.example.com --type ca --principal admin\n```\n\n### Deploy Key\n\n```\nssh deploy-key <key_id>\n\nExample:\n  ssh deploy-key abc-123-def-456\n```\n\n### List Keys\n\n```\nssh list-keys [--expired]\n\nExample:\n  ssh list-keys\n  ssh list-keys | where deployed == true\n```\n\n### Revoke Key\n\n```\nssh revoke-key <key_id>\n\nExample:\n  ssh revoke-key abc-123-def-456\n```\n\n### Connect with Auto-Generated Key\n\n```\nssh connect <server> [options]\n\nOptions:\n  --user <name>                 SSH user (default: root)\n  --ttl <duration>              Key lifetime (default: 1hr)\n  --type <ca|otp|dynamic>       Key type (default: dynamic)\n  --keep                        Keep key after disconnect\n\nExample:\n  ssh connect server.example.com --user deploy\n```\n\nThis command:\n\n1. Generates a temporal SSH key\n2. Deploys it to the server\n3. Opens SSH connection\n4. Revokes the key after disconnect (unless --keep is used)\n\n### Show Statistics\n\n```\nssh stats\n\nExample output:\n  SSH Key Statistics:\n    Total generated: 42\n    Active keys: 10\n    Expired keys: 32\n\n  Keys by type:\n    dynamic: 35\n    otp: 5\n    certificate: 2\n\n  Last cleanup: 2024-01-01T00:00:00Z\n    Cleaned keys: 5\n```\n\n### Manual Cleanup\n\n```\nssh cleanup\n\nExample output:\n  ✓ Cleaned up 5 expired keys\n  Cleaned key IDs:\n    - abc-123\n    - def-456\n    ...\n```\n\n## Configuration\n\n### Orchestrator Configuration\n\nAdd to orchestrator startup:\n\n```\nuse provisioning_orchestrator::{SshKeyManager, SshConfig};\n\n// Create SSH configuration\nlet ssh_config = SshConfig {\n    vault_enabled: false,           // Enable Vault integration\n    vault_addr: None,               // Vault address\n    vault_token: None,              // Vault token\n    vault_mount_point: "ssh".to_string(),\n    vault_mode: "ca".to_string(),   // "ca" or "otp"\n    default_ttl: Duration::hours(1),\n    cleanup_interval: Duration::minutes(5),\n    provisioning_key_path: Some("/path/to/provisioning/key".to_string()),\n};\n\n// Create SSH key manager\nlet ssh_manager = Arc::new(SshKeyManager::new(ssh_config).await?);\n\n// Start background cleanup task\nArc::clone(&ssh_manager).start_cleanup_task().await;\n```\n\n### Vault SSH Configuration\n\n#### OTP Mode\n\n```\n# Enable SSH secrets engine\nvault secrets enable ssh\n\n# Configure OTP role\nvault write ssh/roles/otp_key_role \\n    key_type=otp \\n    default_user=root \\n    cidr_list=0.0.0.0/0\n```\n\n#### CA Mode\n\n```\n# Enable SSH secrets engine\nvault secrets enable ssh\n\n# Generate SSH CA\nvault write ssh/config/ca generate_signing_key=true\n\n# Configure CA role\nvault write ssh/roles/default \\n    key_type=ca \\n    ttl=1h \\n    max_ttl=24h \\n    allow_user_certificates=true \\n    allowed_users="*" \\n    default_extensions="permit-pty,permit-port-forwarding"\n\n# Get CA public key (add to servers' /etc/ssh/trusted-user-ca-keys.pem)\nvault read -field=public_key ssh/config/ca\n```\n\nServer configuration (`/etc/ssh/sshd_config`):\n\n```\nTrustedUserCAKeys /etc/ssh/trusted-user-ca-keys.pem\n```\n\n## Deployment\n\n### Prerequisites\n\n1. **Orchestrator**: Running on port 8080\n2. **SSH Access**: Provisioning key for deploying to servers\n3. **Vault** (optional): For OTP or CA modes\n\n### Environment Variables\n\n```\n# Vault integration (optional)\nexport VAULT_ADDR=https://vault.example.com:8200\nexport VAULT_TOKEN=your-vault-token\n\n# Provisioning SSH key path\nexport PROVISIONING_SSH_KEY=/path/to/provisioning/key\n```\n\n### Integration with Workflows\n\nThe SSH key manager integrates with existing workflows:\n\n```\n# In server creation workflow\nlet ssh_key = (ssh generate-key $server --ttl 30min)\nssh deploy-key $ssh_key.id\n\n# Execute remote commands\nssh root@$server "install-kubernetes.sh"\n\n# Auto-revoke after workflow\nssh revoke-key $ssh_key.id\n```\n\n## Security Considerations\n\n1. **Private Key Exposure**: Private keys are only shown once during generation\n2. **Key Storage**: Keys stored in memory only, not on disk\n3. **Cleanup**: Automatic cleanup removes expired keys from servers\n4. **Audit Logging**: All operations logged for security audit\n5. **Vault Integration**: Optional Vault integration for enterprise security\n6. **TTL Limits**: Enforce maximum TTL to prevent long-lived keys\n\n## Troubleshooting\n\n### Key Deployment Fails\n\nCheck SSH connectivity:\n\n```\nssh -i /path/to/provisioning/key root@server.example.com\n```\n\nVerify SSH daemon is running:\n\n```\nsystemctl status sshd\n```\n\n### Cleanup Not Working\n\nCheck orchestrator logs:\n\n```\ntail -f ./data/orchestrator.log | grep SSH\n```\n\nManual cleanup:\n\n```\nssh cleanup\n```\n\n### Vault Integration Issues\n\nTest Vault connectivity:\n\n```\nvault status\nvault token lookup\n```\n\nCheck SSH secrets engine:\n\n```\nvault secrets list\nvault read ssh/config/ca\n```\n\n## Performance\n\n- **Key Generation**: <100ms (Ed25519)\n- **Key Deployment**: ~1s (depends on SSH latency)\n- **Cleanup Task**: Every 5 minutes (configurable)\n- **Concurrent Keys**: Unlimited (memory bound)\n\n## Future Enhancements\n\n- [ ] SSH certificate rotation\n- [ ] Integration with KMS for key encryption\n- [ ] WebSocket notifications for key expiration\n- [ ] Prometheus metrics export\n- [ ] SSH session recording\n- [ ] Role-based key generation policies\n\n## References\n\n- RFC 8709: Ed25519 and Ed448 Public Key Algorithms for SSH\n- Vault SSH Secrets Engine: <https://www.vaultproject.io/docs/secrets/ssh>\n- OpenSSH Certificate Authentication: <https://man.openbsd.org/ssh-keygen>