# SecretumVault How-To Guide Step-by-step instructions for common tasks with SecretumVault. ## Table of Contents 1. [Quick Start (CLI + Filesystem)](#quick-start-cli--filesystem) 2. [Getting Started](#getting-started) 3. [Initialize Vault](#initialize-vault) 4. [Unseal Vault](#unseal-vault) 5. [Manage Secrets](#manage-secrets) 6. [Configure Engines](#configure-engines) 7. [Setup Authorization](#setup-authorization) 8. [Configure TLS](#configure-tls) 9. [Integrate with Kubernetes](#integrate-with-kubernetes) 10. [Backup & Restore](#backup--restore) 11. [Monitor & Troubleshoot](#monitor--troubleshoot) --- ## Quick Start (CLI + Filesystem) **Fastest way to get SecretumVault running locally with CLI and filesystem storage.** ### Prerequisites ```bash # Rust toolchain installed rustc --version # Should be 1.75+ # Build with server and CLI features cd secretumvault cargo build --features server,cli ``` ### Step 1: Create Configuration ```bash # Create config file cat > config/svault.toml <<'EOF' [vault] crypto_backend = "openssl" [server] address = "0.0.0.0:8200" [storage] backend = "filesystem" [storage.filesystem] path = "data" [seal] seal_type = "shamir" [seal.shamir] shares = 5 threshold = 3 [engines.kv] path = "/secret" versioned = true [engines.transit] path = "/transit" [logging] level = "info" format = "json" EOF ``` ### Step 2: Start Server (Terminal 1) ```bash cargo run --features server,cli -- server -c config/svault.toml ``` **Expected output**: ```json {"level":"INFO","message":"Loading configuration from \"config/svault.toml\""} {"level":"INFO","message":"Vault initialized successfully"} {"level":"WARN","message":"Starting HTTP server on http://0.0.0.0:8200"} {"level":"WARN","message":"TLS not configured. For production, configure tls_cert and tls_key"} ``` **Leave this terminal running.** ### Step 3: Initialize Vault (Terminal 2) ```bash # Open new terminal cargo run --features server,cli -- operator init --shares 5 --threshold 3 ``` **Expected output**: ```text Vault Initialization ==================== Unseal Key 1: YjVmN2E4ZDktMzQ1Ni03ODkwLWFiY2QtZWYxMjM0NTY3ODkw Unseal Key 2: MmQ3ZjRhOGMtOTAxMi0zNDU2LTc4OTAtYWJjZGVmMTIzNDU2 Unseal Key 3: OGNhYjEyMzQtNTY3OC05MDEyLTM0NTYtNzg5MGFiY2RlZjEy Unseal Key 4: ZjEyMzQ1NjctODkwMS0yMzQ1LTY3ODktMDEyMzQ1Njc4OTAx Unseal Key 5: YWJjZGVmMTIzNC01Njc4LTkwMTItMzQ1Ni03ODkwYWJjZGVm Initial Root Token: hvs.CAESIJ4k8n2jW8h3mK... IMPORTANT: Store these keys securely! - You need 3 keys to unseal the vault - If lost, the vault cannot be unsealed - Root token grants full access ``` **⚠️ CRITICAL**: Copy and save all keys immediately to a password manager! ### Step 4: Verify Vault Status ```bash # Check if sealed curl -s http://localhost:8200/v1/sys/status | jq . ``` **Expected output**: ```json { "status": "success", "data": { "sealed": true, "initialized": true, "engines": ["/secret", "/transit"] } } ``` **Note**: `"sealed": true` means vault is locked and **cannot store secrets yet**. ### Step 5: Unseal Vault ```bash # Use 3 of the 5 unseal keys from Step 3 cargo run --features server,cli -- operator unseal \ --shares "YjVmN2E4ZDktMzQ1Ni03ODkwLWFiY2QtZWYxMjM0NTY3ODkw" \ --shares "MmQ3ZjRhOGMtOTAxMi0zNDU2LTc4OTAtYWJjZGVmMTIzNDU2" \ --shares "OGNhYjEyMzQtNTY3OC05MDEyLTM0NTYtNzg5MGFiY2RlZjEy" ``` **Expected output**: ```text ✓ Vault unsealed successfully! ``` ### Step 6: Verify Unsealed Status ```bash curl -s http://localhost:8200/v1/sys/status | jq .data.sealed ``` **Expected output**: `false` **Now the vault is ready to store secrets!** ### Step 7: Store Your First Secret ```bash curl -X POST http://localhost:8200/v1/secret/data/myapp/database \ -H "Content-Type: application/json" \ -d '{ "username": "admin", "password": "supersecret123", "host": "db.example.com" }' ``` **Expected output**: ```json { "status": "success", "data": {"path": "data/myapp/database"}, "error": null } ``` ### Step 8: Verify File Was Created ```bash # List files in storage find data/secrets -type f ``` **Expected output**: ```text data/secrets/secret/data/myapp/database ``` ```bash # View encrypted content (JSON format) cat data/secrets/secret/data/myapp/database ``` **Expected output** (encrypted): ```json { "ciphertext": [12,45,78,90,...], "nonce": [34,56,78,90,...], "algorithm": "AES-256-GCM" } ``` **Note**: The data is encrypted at rest using the master key. ### Step 9: Read Secret Back ```bash curl -s http://localhost:8200/v1/secret/data/myapp/database | jq . ``` **Expected output** (decrypted): ```json { "status": "success", "data": { "username": "admin", "password": "supersecret123", "host": "db.example.com" } } ``` ### Step 10: List All Secrets ```bash curl -s http://localhost:8200/v1/secret/data/ | jq . ``` ### Common Issues **Q: Why are `data/secrets/`, `data/keys/` folders empty?** A: The vault is **sealed**. Folders are created automatically but files only appear after: 1. Vault is initialized (`operator init`) 2. Vault is unsealed (`operator unseal` with 3+ keys) 3. Secrets are stored (`POST /v1/secret/data/...`) **Q: Getting `"sealed": true` but I unsealed it?** A: Vault seals automatically on restart. Run `operator unseal` again after each server restart. **Q: Can't store secrets, getting errors?** A: Verify vault is unsealed: ```bash curl -s http://localhost:8200/v1/sys/status | jq .data.sealed # Must return: false ``` **Q: Where are my encryption keys stored?** A: Keys are in memory only when unsealed. The master key is sealed using Shamir Secret Sharing and requires threshold unseal keys to reconstruct. ### Next Steps - **Enable TLS**: See [Configure TLS](#configure-tls) section - **Create policies**: See [Setup Authorization](#setup-authorization) section - **Use Transit Engine**: See [Configure Engines](#configure-engines) section - **Production deployment**: See [Deployment Guide](../operations/deployment.md) --- ## Getting Started ### 1. Start Vault Locally **Using Docker Compose** (recommended for development): ```bash # Navigate to project cd secretumvault # Build image docker build -t secretumvault:latest . # Start all services docker-compose up -d # Verify vault is running curl http://localhost:8200/v1/sys/health ``` **Using Cargo**: ```bash # Create configuration cat > svault.toml <<'EOF' [vault] crypto_backend = "openssl" [server] address = "0.0.0.0" port = 8200 [storage] backend = "etcd" [storage.etcd] endpoints = ["http://localhost:2379"] [seal] seal_type = "shamir" threshold = 2 shares = 3 [engines.kv] path = "secret/" versioned = true [logging] level = "info" format = "json" EOF # Start vault (requires etcd running) cargo run --release -- server --config svault.toml ``` ### 2. Verify Health ```bash curl http://localhost:8200/v1/sys/health ``` Response: ```json { "initialized": false, "sealed": true, "standby": false, "performance_standby": false, "replication_performance_mode": "disabled", "replication_dr_mode": "disabled", "server_time_utc": 1703142600, "version": "0.1.0" } ``` Key fields: - `initialized: false` - Vault not initialized yet - `sealed: true` - Master key is sealed (expected before initialization) --- ## Initialize Vault ### 1. Generate Unseal Keys Create a request to initialize vault with Shamir Secret Sharing: ```bash curl -X POST http://localhost:8200/v1/sys/init \ -H "Content-Type: application/json" \ -d '{ "shares": 5, "threshold": 3 }' ``` Parameters: - `shares: 5` - Total unseal keys generated (5 people get 1 key each) - `threshold: 3` - Need 3 keys to unseal (quorum) Response: ```json { "keys": [ "key_1_base64_encoded", "key_2_base64_encoded", "key_3_base64_encoded", "key_4_base64_encoded", "key_5_base64_encoded" ], "root_token": "root_token_abc123def456" } ``` ### 2. Store Keys Securely **CRITICAL: Store unseal keys immediately in a secure location!** Save in password manager (Bitwarden, 1Password, LastPass): - Each unseal key separately (don't store all together) - Distribute keys to different people/locations - Test that stored keys are retrievable Save root token separately: - Store in same password manager - Label clearly: "Root Token - SecretumVault" - Keep temporary access only ### 3. Verify Initialization ```bash curl http://localhost:8200/v1/sys/health ``` Response should now show `initialized: true` and `sealed: true` --- ## Unseal Vault Vault must be unsealed before it can serve requests. ### 1. Unseal with Keys You need `threshold` keys (e.g., 3 of 5) to unseal. **Unseal with first key:** ```bash curl -X POST http://localhost:8200/v1/sys/unseal \ -H "Content-Type: application/json" \ -d '{ "key": "first_unseal_key_from_storage" }' ``` Response: ```json { "sealed": true, "t": 3, "n": 5, "progress": 1 } ``` Progress shows 1/3 keys provided. **Unseal with second key:** ```bash curl -X POST http://localhost:8200/v1/sys/unseal \ -H "Content-Type: application/json" \ -d '{ "key": "second_unseal_key_from_storage" }' ``` Response shows `progress: 2/3` **Unseal with third key (final):** ```bash curl -X POST http://localhost:8200/v1/sys/unseal \ -H "Content-Type: application/json" \ -d '{ "key": "third_unseal_key_from_storage" }' ``` Response: ```json { "sealed": false, "t": 3, "n": 5, "progress": 0 } ``` `sealed: false` means vault is now unsealed! ### 2. Verify Unsealed State ```bash curl http://localhost:8200/v1/sys/health ``` Should show `sealed: false` ### 3. Auto-Unseal (Future) For production, configure auto-unseal via AWS KMS or GCP Cloud KMS (planned): ```toml [seal] seal_type = "aws-kms" [seal.aws-kms] key_id = "arn:aws:kms:us-east-1:account:key/id" region = "us-east-1" ``` --- ## Manage Secrets ### 1. Store a Secret **HTTP Request:** ```bash curl -X POST http://localhost:8200/v1/secret/data/myapp \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "data": { "username": "admin", "password": "supersecret123", "api_key": "sk_live_abc123" } }' ``` Environment variable setup: ```bash # From initialization response export VAULT_TOKEN="root_token_abc123" ``` Response: ```json { "request_id": "req_123", "lease_id": "", "renewable": false, "lease_duration": 0, "data": null, "wrap_info": null, "warnings": null, "auth": null } ``` Status `201 Created` indicates success. ### 2. Read a Secret **HTTP Request:** ```bash curl -X GET http://localhost:8200/v1/secret/data/myapp \ -H "X-Vault-Token: $VAULT_TOKEN" ``` Response: ```json { "request_id": "req_124", "lease_id": "", "renewable": false, "lease_duration": 0, "data": { "data": { "username": "admin", "password": "supersecret123", "api_key": "sk_live_abc123" }, "metadata": { "created_time": "2025-12-21T10:30:00Z", "deletion_time": "", "destroyed": false, "version": 1 } } } ``` Extract secret data: ```bash # Get password field curl -s http://localhost:8200/v1/secret/data/myapp \ -H "X-Vault-Token: $VAULT_TOKEN" | jq '.data.data.password' ``` Output: `"supersecret123"` ### 3. Update a Secret ```bash curl -X POST http://localhost:8200/v1/secret/data/myapp \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "data": { "username": "admin", "password": "newsecret456", "api_key": "sk_live_abc123" } }' ``` New version created (version 2). Previous versions retained. ### 4. Delete a Secret ```bash curl -X DELETE http://localhost:8200/v1/secret/data/myapp \ -H "X-Vault-Token: $VAULT_TOKEN" ``` Soft delete: metadata retained, data destroyed. ### 5. List Secrets ```bash curl -X LIST http://localhost:8200/v1/secret/metadata \ -H "X-Vault-Token: $VAULT_TOKEN" ``` Response: ```json { "data": { "keys": [ "myapp", "database-prod", "aws-credentials" ] } } ``` ### 6. Restore from Version View available versions: ```bash curl -X GET http://localhost:8200/v1/secret/metadata/myapp \ -H "X-Vault-Token: $VAULT_TOKEN" ``` Response shows all versions with timestamps. Get specific version: ```bash curl -X GET http://localhost:8200/v1/secret/data/myapp?version=1 \ -H "X-Vault-Token: $VAULT_TOKEN" ``` --- ## Configure Engines ### 1. Enable Additional Engines Edit `svault.toml`: ```toml [engines.kv] path = "secret/" versioned = true [engines.transit] path = "transit/" versioned = true [engines.pki] path = "pki/" versioned = false [engines.database] path = "database/" versioned = false ``` Restart vault: ```bash # Docker Compose docker-compose restart vault # Or Cargo # Kill running process (Ctrl+C) and restart cargo run --release -- server --config svault.toml ``` ### 2. Use Transit Engine (Encryption) Create encryption key: ```bash curl -X POST http://localhost:8200/v1/transit/keys/my-key \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "exportable": false, "key_size": 256, "type": "aes-gcm" }' ``` Encrypt data: ```bash # Plaintext must be base64 encoded PLAINTEXT=$(echo -n "sensitive data" | base64) curl -X POST http://localhost:8200/v1/transit/encrypt/my-key \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d "{\"plaintext\": \"$PLAINTEXT\"}" ``` Response: ```json { "data": { "ciphertext": "vault:v1:abc123def456..." } } ``` Decrypt data: ```bash CIPHERTEXT="vault:v1:abc123def456..." curl -X POST http://localhost:8200/v1/transit/decrypt/my-key \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d "{\"ciphertext\": \"$CIPHERTEXT\"}" ``` Response: ```json { "data": { "plaintext": "c2Vuc2l0aXZlIGRhdGE=" } } ``` Decode plaintext: ```bash echo "c2Vuc2l0aXZlIGRhdGE=" | base64 -d # Output: sensitive data ``` ### 3. Mount at Custom Path Change mount path in config: ```toml [engines.kv] path = "app-secrets/" # Instead of "secret/" versioned = true ``` Then access at: ```bash curl http://localhost:8200/v1/app-secrets/data/myapp \ -H "X-Vault-Token: $VAULT_TOKEN" ``` --- ## Setup Authorization ### 1. Create Cedar Policies Create policy directory: ```bash mkdir -p /etc/secretumvault/policies ``` Create policy file: ```bash cat > /etc/secretumvault/policies/default.cedar <<'EOF' permit ( principal, action, resource ) when { principal has policies && principal.policies.contains("admin") }; deny ( principal, action == Action::"write", resource ) unless { context.time_of_day < 20:00 }; EOF ``` Update config: ```toml [auth] cedar_policies_dir = "/etc/secretumvault/policies" ``` ### 2. Create Auth Token ```bash curl -X POST http://localhost:8200/v1/auth/token/create \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "policies": ["default", "app-reader"], "ttl": "24h", "renewable": true }' ``` Response: ```json { "auth": { "client_token": "s.abc123def456", "policies": ["default", "app-reader"], "metadata": { "created_time": "2025-12-21T10:30:00Z", "ttl": "24h" } } } ``` Use token: ```bash export APP_TOKEN="s.abc123def456" curl http://localhost:8200/v1/secret/data/myapp \ -H "X-Vault-Token: $APP_TOKEN" ``` ### 3. Renew Token ```bash curl -X POST http://localhost:8200/v1/auth/token/renew \ -H "X-Vault-Token: $APP_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "increment": "24h" }' ``` ### 4. Revoke Token ```bash curl -X POST http://localhost:8200/v1/auth/token/revoke \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "token": "s.abc123def456" }' ``` --- ## Configure TLS ### 1. Generate Self-Signed Certificate For development: ```bash openssl req -x509 -newkey rsa:4096 -keyout tls.key -out tls.crt \ -days 365 -nodes \ -subj "/CN=localhost/O=SecretumVault/C=US" ``` ### 2. Configure Vault Update `svault.toml`: ```toml [server] address = "0.0.0.0" port = 8200 tls_cert = "/path/to/tls.crt" tls_key = "/path/to/tls.key" ``` ### 3. Access via HTTPS ```bash # Allow self-signed certificate curl --insecure https://localhost:8200/v1/sys/health \ -H "X-Vault-Token: $VAULT_TOKEN" # Or with CA certificate curl --cacert tls.crt https://localhost:8200/v1/sys/health \ -H "X-Vault-Token: $VAULT_TOKEN" ``` ### 4. Production Certificate (Let's Encrypt) For Kubernetes with cert-manager, use the Helm installation which handles automatic certificate renewal. --- ## Integrate with Kubernetes ### 1. Deploy Vault ```bash # Apply manifests kubectl apply -f k8s/01-namespace.yaml kubectl apply -f k8s/02-configmap.yaml kubectl apply -f k8s/03-deployment.yaml kubectl apply -f k8s/04-service.yaml kubectl apply -f k8s/05-etcd.yaml # Wait for pods kubectl -n secretumvault wait --for=condition=ready pod -l app=vault --timeout=300s ``` ### 2. Initialize and Unseal Port-forward vault: ```bash kubectl -n secretumvault port-forward svc/vault 8200:8200 & ``` Initialize (from earlier steps): ```bash curl -X POST http://localhost:8200/v1/sys/init \ -H "Content-Type: application/json" \ -d '{"shares": 3, "threshold": 2}' ``` Save keys, then unseal (from earlier steps). ### 3. Create Kubernetes ServiceAccount ```bash cat > /tmp/app-sa.yaml <<'EOF' apiVersion: v1 kind: ServiceAccount metadata: name: myapp namespace: default EOF kubectl apply -f /tmp/app-sa.yaml ``` ### 4. Pod Secret Injection Create ClusterRoleBinding to allow reading vault-config: ```bash cat > /tmp/vault-reader.yaml <<'EOF' apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: vault-reader rules: - apiGroups: [""] resources: ["secrets"] verbs: ["get"] - apiGroups: [""] resources: ["configmaps"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: vault-reader roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: vault-reader subjects: - kind: ServiceAccount name: myapp namespace: default EOF kubectl apply -f /tmp/vault-reader.yaml ``` ### 5. Deploy Application Pod ```bash cat > /tmp/myapp-pod.yaml <<'EOF' apiVersion: v1 kind: Pod metadata: name: myapp namespace: default spec: serviceAccountName: myapp containers: - name: app image: myapp:latest env: - name: VAULT_ADDR value: "http://vault.secretumvault.svc.cluster.local:8200" - name: VAULT_TOKEN valueFrom: secretKeyRef: name: vault-token key: token volumeMounts: - name: vault-config mountPath: /etc/vault readOnly: true volumes: - name: vault-config configMap: name: vault-config namespace: secretumvault EOF kubectl apply -f /tmp/myapp-pod.yaml ``` --- ## Backup & Restore ### 1. Backup Secrets Export all secrets: ```bash # List all secrets SECRETS=$(curl -s http://localhost:8200/v1/secret/metadata \ -H "X-Vault-Token: $VAULT_TOKEN" | jq -r '.data.keys[]') # Backup each secret for secret in $SECRETS; do curl -s http://localhost:8200/v1/secret/data/$secret \ -H "X-Vault-Token: $VAULT_TOKEN" > $secret-backup.json done ``` ### 2. Export with Encryption Encrypt backup before storing: ```bash # Create transit key for backups curl -X POST http://localhost:8200/v1/transit/keys/backup-key \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d '{"type": "aes-gcm"}' # Backup and encrypt for secret in $SECRETS; do CONTENT=$(curl -s http://localhost:8200/v1/secret/data/$secret \ -H "X-Vault-Token: $VAULT_TOKEN" | base64) ENCRYPTED=$(curl -s -X POST http://localhost:8200/v1/transit/encrypt/backup-key \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d "{\"plaintext\": \"$CONTENT\"}" | jq -r '.data.ciphertext') echo "$ENCRYPTED" > $secret-backup.enc done ``` ### 3. Restore Secrets ```bash # List backup files for backup in *-backup.json; do secret=${backup%-backup.json} # Read backup curl -X POST http://localhost:8200/v1/secret/data/$secret \ -H "X-Vault-Token: $VAULT_TOKEN" \ -H "Content-Type: application/json" \ -d @$backup done ``` --- ## Monitor & Troubleshoot ### 1. Check Vault Health ```bash curl http://localhost:8200/v1/sys/health | jq . ``` Key fields to check: - `sealed`: Should be `false` - `initialized`: Should be `true` - `standby`: Should be `false` (or expected leader state) ### 2. View Metrics Prometheus metrics endpoint: ```bash curl http://localhost:9090/metrics | grep vault ``` Common metrics: - `vault_secrets_stored_total` - Total secrets stored - `vault_secrets_read_total` - Total secrets read - `vault_operations_encrypt` - Encryption operations - `vault_tokens_created` - Tokens created ### 3. Check Logs Docker Compose: ```bash docker-compose logs -f vault ``` Kubernetes: ```bash kubectl -n secretumvault logs -f deployment/vault ``` Look for: - `ERROR` entries with details - `WARN` for unexpected but recoverable conditions - `INFO` for normal operations ### 4. Verify Storage Connectivity Check etcd from vault pod: ```bash kubectl -n secretumvault exec deployment/vault -- \ curl http://vault-etcd-client:2379/health ``` ### 5. Test Token Access Validate token is working: ```bash curl -X GET http://localhost:8200/v1/auth/token/self \ -H "X-Vault-Token: $VAULT_TOKEN" | jq '.auth' ``` Response shows token metadata and policies. ### 6. Common Issues **Issue: "sealed: true" after restart** - Solution: Run unseal procedure with stored keys **Issue: "permission denied" on secret read** - Solution: Check Cedar policies, verify token has correct policies **Issue: Storage connection error** - Solution: Verify backend endpoint in config (etcd DNS/IP) **Issue: High memory usage** - Solution: Check number of active leases, revoke old tokens **Issue: Slow operations** - Solution: Check storage backend performance, review metrics --- **For more details**, see: - [Architecture Guide](ARCHITECTURE.md) - [Configuration Reference](CONFIGURATION.md) - [Deployment Guide](../DEPLOYMENT.md)