prvng_platform/crates/orchestrator/docs/ssh-key-management.md
2026-01-14 03:25:20 +00:00

13 KiB

SSH Temporal Key Management System\n\n## Overview\n\nThe SSH Temporal Key Management System provides automated generation, deployment, and cleanup of short-lived SSH keys\nfor secure server access. It eliminates the need for static SSH keys by generating keys on-demand with automatic expiration.\n\n## Features\n\n### Core Features\n\n- Short-Lived Keys: Keys expire automatically after a configurable TTL (default: 1 hour)\n- Multiple Key Types:\n - Dynamic Key Pairs (Ed25519)\n - Vault OTP (One-Time Password)\n - Vault CA-Signed Certificates\n- Automatic Cleanup: Background task removes expired keys from servers\n- Audit Trail: All key operations are logged\n- REST API: HTTP endpoints for integration\n- Nushell CLI: User-friendly command-line interface\n\n### Security Features\n\n- Ed25519 keys (modern, secure algorithm)\n- Automatic expiration and cleanup\n- Private keys never stored on disk (only in memory)\n- Vault integration for enterprise scenarios\n- SSH fingerprint tracking\n- Per-key audit logging\n\n## Architecture\n\n{$detected_lang}\n┌─────────────────────────────────────────────────\n────────────┐\n│ SSH Key Manager │\n├─────────────────────────────────────────────────\n────────────┤\n│ │\n│ ┌──────────────┐ ┌──────────────┐ \n┌──────────────┐ │\n│ │ Key Generator│ │ Key Deployer │ │ Temporal │ │\n│ │ (Ed25519) │ │ (SSH Deploy) │ │ Manager │ │\n│ └──────────────┘ └──────────────┘ \n└──────────────┘ │\n│ │\n│ ┌──────────────┐ ┌──────────────┐ │\n│ │ Vault │ │ Authorized │ │\n│ │ SSH Engine │ │ Keys Manager │ │\n│ └──────────────┘ └──────────────┘ │\n│ │\n└─────────────────────────────────────────────────\n────────────┘\n │ │ │\n ▼ ▼ ▼\n REST API Nushell CLI Background Tasks\n\n\n## Key Types\n\n### 1. Dynamic Key Pairs (Default)\n\nGenerated on-demand Ed25519 keys that are automatically deployed and cleaned up.\n\nUse Case: Quick SSH access without Vault infrastructure\n\nExample:\n\n{$detected_lang}\nssh generate-key server.example.com --user root --ttl 30min\n\n\n### 2. Vault OTP (One-Time Password)\n\nVault generates a one-time password for SSH authentication.\n\nUse Case: Single-use SSH access with centralized authentication\n\nRequirements: Vault with SSH secrets engine in OTP mode\n\nExample:\n\n{$detected_lang}\nssh generate-key server.example.com --type otp --ip 192.168.1.100\n\n\n### 3. Vault CA-Signed Certificates\n\nVault acts as SSH CA, signing user public keys with short TTL.\n\nUse Case: Enterprise scenarios with SSH CA infrastructure\n\nRequirements: Vault with SSH secrets engine in CA mode\n\nExample:\n\n{$detected_lang}\nssh generate-key server.example.com --type ca --principal admin --ttl 1hr\n\n\n## REST API Endpoints\n\nBase URL: http://localhost:9090\n\n### Generate SSH Key\n\n{$detected_lang}\nPOST /api/v1/ssh/generate\n\n{\n "key_type": "dynamickeypair", // or "otp", "certificate"\n "user": "root",\n "target_server": "server.example.com",\n "ttl_seconds": 3600,\n "allowed_ip": "192.168.1.100", // optional, for OTP\n "principal": "admin" // optional, for CA\n}\n\nResponse:\n{\n "success": true,\n "data": {\n "id": "uuid",\n "key_type": "dynamickeypair",\n "public_key": "ssh-ed25519 AAAA...",\n "private_key": "-----BEGIN OPENSSH PRIVATE KEY-----...",\n "fingerprint": "SHA256:...",\n "user": "root",\n "target_server": "server.example.com",\n "created_at": "2024-01-01T00:00:00Z",\n "expires_at": "2024-01-01T01:00:00Z",\n "deployed": false\n }\n}\n\n\n### Deploy SSH Key\n\n{$detected_lang}\nPOST /api/v1/ssh/{key_id}/deploy\n\nResponse:\n{\n "success": true,\n "data": {\n "key_id": "uuid",\n "server": "server.example.com",\n "success": true,\n "deployed_at": "2024-01-01T00:00:00Z"\n }\n}\n\n\n### List SSH Keys\n\n{$detected_lang}\nGET /api/v1/ssh/keys\n\nResponse:\n{\n "success": true,\n "data": [\n {\n "id": "uuid",\n "key_type": "dynamickeypair",\n "user": "root",\n "target_server": "server.example.com",\n "expires_at": "2024-01-01T01:00:00Z",\n "deployed": true\n }\n ]\n}\n\n\n### Revoke SSH Key\n\n{$detected_lang}\nPOST /api/v1/ssh/{key_id}/revoke\n\nResponse:\n{\n "success": true,\n "data": "Key uuid revoked successfully"\n}\n\n\n### Get SSH Key\n\n{$detected_lang}\nGET /api/v1/ssh/{key_id}\n\nResponse:\n{\n "success": true,\n "data": {\n "id": "uuid",\n "key_type": "dynamickeypair",\n ...\n }\n}\n\n\n### Cleanup Expired Keys\n\n{$detected_lang}\nPOST /api/v1/ssh/cleanup\n\nResponse:\n{\n "success": true,\n "data": {\n "cleaned_count": 5,\n "cleaned_key_ids": ["uuid1", "uuid2", ...]\n }\n}\n\n\n### Get Statistics\n\n{$detected_lang}\nGET /api/v1/ssh/stats\n\nResponse:\n{\n "success": true,\n "data": {\n "total_generated": 42,\n "active_keys": 10,\n "expired_keys": 32,\n "keys_by_type": {\n "dynamic": 35,\n "otp": 5,\n "certificate": 2\n },\n "last_cleanup_count": 5,\n "last_cleanup_at": "2024-01-01T00:00:00Z"\n }\n}\n\n\n## Nushell CLI Commands\n\n### Generate Key\n\n{$detected_lang}\nssh generate-key <server> [options]\n\nOptions:\n --user <name> SSH user (default: root)\n --ttl <duration> Key lifetime (default: 1hr)\n --type <ca|otp|dynamic> Key type (default: dynamic)\n --ip <address> Allowed IP (OTP mode)\n --principal <name> Principal (CA mode)\n\nExamples:\n ssh generate-key server.example.com\n ssh generate-key server.example.com --user deploy --ttl 30min\n ssh generate-key server.example.com --type ca --principal admin\n\n\n### Deploy Key\n\n{$detected_lang}\nssh deploy-key <key_id>\n\nExample:\n ssh deploy-key abc-123-def-456\n\n\n### List Keys\n\n{$detected_lang}\nssh list-keys [--expired]\n\nExample:\n ssh list-keys\n ssh list-keys | where deployed == true\n\n\n### Revoke Key\n\n{$detected_lang}\nssh revoke-key <key_id>\n\nExample:\n ssh revoke-key abc-123-def-456\n\n\n### Connect with Auto-Generated Key\n\n{$detected_lang}\nssh connect <server> [options]\n\nOptions:\n --user <name> SSH user (default: root)\n --ttl <duration> Key lifetime (default: 1hr)\n --type <ca|otp|dynamic> Key type (default: dynamic)\n --keep Keep key after disconnect\n\nExample:\n ssh connect server.example.com --user deploy\n\n\nThis command:\n\n1. Generates a temporal SSH key\n2. Deploys it to the server\n3. Opens SSH connection\n4. Revokes the key after disconnect (unless --keep is used)\n\n### Show Statistics\n\n{$detected_lang}\nssh stats\n\nExample output:\n SSH Key Statistics:\n Total generated: 42\n Active keys: 10\n Expired keys: 32\n\n Keys by type:\n dynamic: 35\n otp: 5\n certificate: 2\n\n Last cleanup: 2024-01-01T00:00:00Z\n Cleaned keys: 5\n\n\n### Manual Cleanup\n\n{$detected_lang}\nssh cleanup\n\nExample output:\n ✓ Cleaned up 5 expired keys\n Cleaned key IDs:\n - abc-123\n - def-456\n ...\n\n\n## Configuration\n\n### Orchestrator Configuration\n\nAdd to orchestrator startup:\n\n{$detected_lang}\nuse provisioning_orchestrator::{SshKeyManager, SshConfig};\n\n// Create SSH configuration\nlet ssh_config = SshConfig {\n vault_enabled: false, // Enable Vault integration\n vault_addr: None, // Vault address\n vault_token: None, // Vault token\n vault_mount_point: "ssh".to_string(),\n vault_mode: "ca".to_string(), // "ca" or "otp"\n default_ttl: Duration::hours(1),\n cleanup_interval: Duration::minutes(5),\n provisioning_key_path: Some("/path/to/provisioning/key".to_string()),\n};\n\n// Create SSH key manager\nlet ssh_manager = Arc::new(SshKeyManager::new(ssh_config).await?);\n\n// Start background cleanup task\nArc::clone(&ssh_manager).start_cleanup_task().await;\n\n\n### Vault SSH Configuration\n\n#### OTP Mode\n\n{$detected_lang}\n# Enable SSH secrets engine\nvault secrets enable ssh\n\n# Configure OTP role\nvault write ssh/roles/otp_key_role \\n key_type=otp \\n default_user=root \\n cidr_list=0.0.0.0/0\n\n\n#### CA Mode\n\n{$detected_lang}\n# Enable SSH secrets engine\nvault secrets enable ssh\n\n# Generate SSH CA\nvault write ssh/config/ca generate_signing_key=true\n\n# Configure CA role\nvault write ssh/roles/default \\n key_type=ca \\n ttl=1h \\n max_ttl=24h \\n allow_user_certificates=true \\n allowed_users="*" \\n default_extensions="permit-pty,permit-port-forwarding"\n\n# Get CA public key (add to servers' /etc/ssh/trusted-user-ca-keys.pem)\nvault read -field=public_key ssh/config/ca\n\n\nServer configuration (/etc/ssh/sshd_config):\n\n{$detected_lang}\nTrustedUserCAKeys /etc/ssh/trusted-user-ca-keys.pem\n\n\n## Deployment\n\n### Prerequisites\n\n1. Orchestrator: Running on port 8080\n2. SSH Access: Provisioning key for deploying to servers\n3. Vault (optional): For OTP or CA modes\n\n### Environment Variables\n\n{$detected_lang}\n# Vault integration (optional)\nexport VAULT_ADDR=https://vault.example.com:8200\nexport VAULT_TOKEN=your-vault-token\n\n# Provisioning SSH key path\nexport PROVISIONING_SSH_KEY=/path/to/provisioning/key\n\n\n### Integration with Workflows\n\nThe SSH key manager integrates with existing workflows:\n\n{$detected_lang}\n# In server creation workflow\nlet ssh_key = (ssh generate-key $server --ttl 30min)\nssh deploy-key $ssh_key.id\n\n# Execute remote commands\nssh root@$server "install-kubernetes.sh"\n\n# Auto-revoke after workflow\nssh revoke-key $ssh_key.id\n\n\n## Security Considerations\n\n1. Private Key Exposure: Private keys are only shown once during generation\n2. Key Storage: Keys stored in memory only, not on disk\n3. Cleanup: Automatic cleanup removes expired keys from servers\n4. Audit Logging: All operations logged for security audit\n5. Vault Integration: Optional Vault integration for enterprise security\n6. TTL Limits: Enforce maximum TTL to prevent long-lived keys\n\n## Troubleshooting\n\n### Key Deployment Fails\n\nCheck SSH connectivity:\n\n{$detected_lang}\nssh -i /path/to/provisioning/key root@server.example.com\n\n\nVerify SSH daemon is running:\n\n{$detected_lang}\nsystemctl status sshd\n\n\n### Cleanup Not Working\n\nCheck orchestrator logs:\n\n{$detected_lang}\ntail -f ./data/orchestrator.log | grep SSH\n\n\nManual cleanup:\n\n{$detected_lang}\nssh cleanup\n\n\n### Vault Integration Issues\n\nTest Vault connectivity:\n\n{$detected_lang}\nvault status\nvault token lookup\n\n\nCheck SSH secrets engine:\n\n{$detected_lang}\nvault secrets list\nvault read ssh/config/ca\n\n\n## Performance\n\n- Key Generation: <100ms (Ed25519)\n- Key Deployment: ~1s (depends on SSH latency)\n- Cleanup Task: Every 5 minutes (configurable)\n- Concurrent Keys: Unlimited (memory bound)\n\n## Future Enhancements\n\n- [ ] SSH certificate rotation\n- [ ] Integration with KMS for key encryption\n- [ ] WebSocket notifications for key expiration\n- [ ] Prometheus metrics export\n- [ ] SSH session recording\n- [ ] Role-based key generation policies\n\n## References\n\n- RFC 8709: Ed25519 and Ed448 Public Key Algorithms for SSH\n- Vault SSH Secrets Engine: https://www.vaultproject.io/docs/secrets/ssh\n- OpenSSH Certificate Authentication: https://man.openbsd.org/ssh-keygen