440 lines
8.5 KiB
Markdown
440 lines
8.5 KiB
Markdown
|
|
# SOPS + typedialog Integration Demo
|
||
|
|
|
||
|
|
Complete walkthrough of using SOPS backend with typedialog encryption.
|
||
|
|
|
||
|
|
## Quick Start (5 minutes)
|
||
|
|
|
||
|
|
### 1. Install Prerequisites
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Install SOPS
|
||
|
|
brew install sops
|
||
|
|
|
||
|
|
# Verify installation
|
||
|
|
sops --version
|
||
|
|
which age-keygen
|
||
|
|
```
|
||
|
|
|
||
|
|
### 2. Setup SOPS Configuration
|
||
|
|
|
||
|
|
SOPS requires a `.sops.yaml` configuration file. For testing, we'll use Age (no external KMS).
|
||
|
|
|
||
|
|
Create `.sops.yaml` in your project root:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Generate an Age key for SOPS
|
||
|
|
age-keygen -o ~/.age/sops-key.txt
|
||
|
|
|
||
|
|
# Extract public key
|
||
|
|
grep "^# public key:" ~/.age/sops-key.txt
|
||
|
|
# Output: # public key: age1xxxxxxxxxxxxxxxxxxxxxx
|
||
|
|
|
||
|
|
# Create .sops.yaml
|
||
|
|
cat > .sops.yaml << 'EOF'
|
||
|
|
creation_rules:
|
||
|
|
- path_regex: .*
|
||
|
|
age: age1xxxxxxxxxxxxxxxxxxxxxx # Replace with your public key
|
||
|
|
EOF
|
||
|
|
|
||
|
|
# Verify config
|
||
|
|
cat .sops.yaml
|
||
|
|
```
|
||
|
|
|
||
|
|
### 3. Test SOPS Encryption Manually
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Create a test file
|
||
|
|
echo 'secret: my-password-123' > test-secret.yaml
|
||
|
|
|
||
|
|
# Encrypt with SOPS
|
||
|
|
sops -e -i test-secret.yaml
|
||
|
|
|
||
|
|
# View encrypted content (should look like YAML but encrypted)
|
||
|
|
cat test-secret.yaml
|
||
|
|
|
||
|
|
# Decrypt to verify
|
||
|
|
export SOPS_AGE_KEY_FILE=~/.age/sops-key.txt
|
||
|
|
sops -d test-secret.yaml
|
||
|
|
|
||
|
|
# Cleanup
|
||
|
|
rm test-secret.yaml
|
||
|
|
```
|
||
|
|
|
||
|
|
**Expected Output:**
|
||
|
|
```yaml
|
||
|
|
secret: ENC[AES256_GCM,data:xxxxx,iv:xxxxx,tag:xxxxx,type:str]
|
||
|
|
sops:
|
||
|
|
version: 3.9.0
|
||
|
|
...
|
||
|
|
```
|
||
|
|
|
||
|
|
When you decrypt, you should see:
|
||
|
|
```yaml
|
||
|
|
secret: my-password-123
|
||
|
|
```
|
||
|
|
|
||
|
|
### 4. Test typedialog with SOPS
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Option A: Interactive (uses stdin)
|
||
|
|
typedialog form examples/08-encryption/simple-login.toml \
|
||
|
|
--encrypt --backend sops \
|
||
|
|
--format json
|
||
|
|
|
||
|
|
# You'll be prompted for:
|
||
|
|
# Username: alice
|
||
|
|
# Password: secretpass123
|
||
|
|
|
||
|
|
# Option B: Verify encryption works (redaction test)
|
||
|
|
typedialog form examples/08-encryption/simple-login.toml \
|
||
|
|
--redact \
|
||
|
|
--format json
|
||
|
|
|
||
|
|
# Output should show:
|
||
|
|
# {
|
||
|
|
# "username": "alice",
|
||
|
|
# "password": "[REDACTED]"
|
||
|
|
# }
|
||
|
|
```
|
||
|
|
|
||
|
|
### 5. Verify SOPS Ciphertext Format
|
||
|
|
|
||
|
|
After encryption, check the output format:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# The encrypted password should look like:
|
||
|
|
# "sops:v1:4f5a6b7c8d9e0f1a2b3c4d5e6f..."
|
||
|
|
|
||
|
|
# This format is:
|
||
|
|
# - sops:v1: version prefix
|
||
|
|
# - hex-encoded encrypted YAML content
|
||
|
|
|
||
|
|
# You can verify it's hex:
|
||
|
|
echo "4f5a6b7c8d9e0f1a" | xxd -r -p
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Advanced: Multi-Backend Configuration
|
||
|
|
|
||
|
|
### Using SOPS for Database Credentials
|
||
|
|
|
||
|
|
Edit `examples/08-encryption/credentials.toml`:
|
||
|
|
|
||
|
|
```toml
|
||
|
|
[[fields]]
|
||
|
|
name = "db_password"
|
||
|
|
type = "password"
|
||
|
|
prompt = "Database password"
|
||
|
|
required = true
|
||
|
|
sensitive = true
|
||
|
|
encryption_backend = "sops"
|
||
|
|
# SOPS reads configuration from .sops.yaml
|
||
|
|
# No additional config needed
|
||
|
|
```
|
||
|
|
|
||
|
|
Run with SOPS:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
typedialog form examples/08-encryption/credentials.toml \
|
||
|
|
--encrypt --backend sops \
|
||
|
|
--format json
|
||
|
|
```
|
||
|
|
|
||
|
|
### Field-Level Encryption
|
||
|
|
|
||
|
|
Different fields can use different backends:
|
||
|
|
|
||
|
|
```toml
|
||
|
|
# Field 1: Age encryption (local)
|
||
|
|
[[fields]]
|
||
|
|
name = "api_key"
|
||
|
|
type = "text"
|
||
|
|
sensitive = true
|
||
|
|
encryption_backend = "age"
|
||
|
|
|
||
|
|
# Field 2: SOPS encryption (team-friendly)
|
||
|
|
[[fields]]
|
||
|
|
name = "db_password"
|
||
|
|
type = "password"
|
||
|
|
sensitive = true
|
||
|
|
encryption_backend = "sops"
|
||
|
|
|
||
|
|
# Field 3: AWS KMS (enterprise)
|
||
|
|
[[fields]]
|
||
|
|
name = "master_key"
|
||
|
|
type = "password"
|
||
|
|
sensitive = true
|
||
|
|
encryption_backend = "awskms"
|
||
|
|
|
||
|
|
[fields.encryption_config]
|
||
|
|
region = "us-east-1"
|
||
|
|
key_id = "arn:aws:kms:us-east-1:..."
|
||
|
|
```
|
||
|
|
|
||
|
|
Same form, different backends per field! 🎉
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## SOPS with AWS KMS (Production Setup)
|
||
|
|
|
||
|
|
To use SOPS with AWS KMS instead of Age:
|
||
|
|
|
||
|
|
### 1. Configure .sops.yaml for AWS KMS
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cat > .sops.yaml << 'EOF'
|
||
|
|
creation_rules:
|
||
|
|
- path_regex: .*
|
||
|
|
kms: arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012
|
||
|
|
aws_region: us-east-1
|
||
|
|
EOF
|
||
|
|
```
|
||
|
|
|
||
|
|
### 2. Configure AWS Credentials
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Option A: AWS CLI
|
||
|
|
aws configure
|
||
|
|
export AWS_REGION=us-east-1
|
||
|
|
|
||
|
|
# Option B: Environment variables
|
||
|
|
export AWS_ACCESS_KEY_ID=xxxxx
|
||
|
|
export AWS_SECRET_ACCESS_KEY=xxxxx
|
||
|
|
export AWS_REGION=us-east-1
|
||
|
|
|
||
|
|
# Option C: IAM Role (on EC2/ECS)
|
||
|
|
# Credentials automatically detected
|
||
|
|
```
|
||
|
|
|
||
|
|
### 3. Test with SOPS
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Create test file
|
||
|
|
echo 'secret: my-secret' > test.yaml
|
||
|
|
|
||
|
|
# Encrypt with AWS KMS
|
||
|
|
sops -e -i test.yaml
|
||
|
|
|
||
|
|
# Decrypt (requires AWS credentials and KMS permissions)
|
||
|
|
sops -d test.yaml
|
||
|
|
|
||
|
|
# Use with typedialog
|
||
|
|
typedialog form examples/08-encryption/simple-login.toml \
|
||
|
|
--encrypt --backend sops \
|
||
|
|
--format json
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## SOPS with GCP KMS
|
||
|
|
|
||
|
|
### 1. Configure .sops.yaml for GCP KMS
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cat > .sops.yaml << 'EOF'
|
||
|
|
creation_rules:
|
||
|
|
- path_regex: .*
|
||
|
|
gcp_kms:
|
||
|
|
- resource_id: projects/MY_PROJECT/locations/global/keyRings/MY_KEYRING/cryptoKeys/MY_KEY
|
||
|
|
EOF
|
||
|
|
```
|
||
|
|
|
||
|
|
### 2. Configure GCP Credentials
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Option A: Service account key
|
||
|
|
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json
|
||
|
|
|
||
|
|
# Option B: gcloud authentication
|
||
|
|
gcloud auth application-default login
|
||
|
|
```
|
||
|
|
|
||
|
|
### 3. Test
|
||
|
|
|
||
|
|
```bash
|
||
|
|
sops -e -i test.yaml
|
||
|
|
sops -d test.yaml
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Troubleshooting
|
||
|
|
|
||
|
|
### Problem: "no identity matched key"
|
||
|
|
|
||
|
|
**Cause**: SOPS can't find the Age key
|
||
|
|
|
||
|
|
**Solution**:
|
||
|
|
```bash
|
||
|
|
# Set the Age key file
|
||
|
|
export SOPS_AGE_KEY_FILE=~/.age/sops-key.txt
|
||
|
|
|
||
|
|
# Or verify the key exists
|
||
|
|
ls -la ~/.age/sops-key.txt
|
||
|
|
|
||
|
|
# Or check .sops.yaml has correct public key
|
||
|
|
cat .sops.yaml
|
||
|
|
grep "^# public key:" ~/.age/sops-key.txt
|
||
|
|
```
|
||
|
|
|
||
|
|
### Problem: "config file not found"
|
||
|
|
|
||
|
|
**Cause**: `.sops.yaml` not found in current directory or parent
|
||
|
|
|
||
|
|
**Solution**:
|
||
|
|
```bash
|
||
|
|
# Verify .sops.yaml exists
|
||
|
|
cat .sops.yaml
|
||
|
|
|
||
|
|
# Or create it
|
||
|
|
cat > .sops.yaml << 'EOF'
|
||
|
|
creation_rules:
|
||
|
|
- path_regex: .*
|
||
|
|
age: age1xxxxxxxxxxxxxxxxxxxxxx
|
||
|
|
EOF
|
||
|
|
```
|
||
|
|
|
||
|
|
### Problem: typedialog says "SOPS encryption error: config file not found"
|
||
|
|
|
||
|
|
**Cause**: Same as above - SOPS config not found
|
||
|
|
|
||
|
|
**Solution**:
|
||
|
|
```bash
|
||
|
|
# Create .sops.yaml in the directory where you run typedialog
|
||
|
|
cat > .sops.yaml << 'EOF'
|
||
|
|
creation_rules:
|
||
|
|
- path_regex: .*
|
||
|
|
age: age1xxxxxxxxxxxxxxxxxxxxxx
|
||
|
|
EOF
|
||
|
|
|
||
|
|
# Then run typedialog in the same directory
|
||
|
|
typedialog form examples/08-encryption/simple-login.toml \
|
||
|
|
--encrypt --backend sops --format json
|
||
|
|
```
|
||
|
|
|
||
|
|
### Problem: "Backend 'sops' not available"
|
||
|
|
|
||
|
|
**Cause**: SOPS feature not enabled or sops binary not installed
|
||
|
|
|
||
|
|
**Solution**:
|
||
|
|
```bash
|
||
|
|
# Install sops
|
||
|
|
brew install sops
|
||
|
|
|
||
|
|
# Or rebuild typedialog with SOPS feature
|
||
|
|
cargo build --features encryption,sops
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Testing Commands
|
||
|
|
|
||
|
|
### Quick Verification (No Encryption Service Required)
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Redaction only - no SOPS needed
|
||
|
|
typedialog form examples/08-encryption/simple-login.toml --redact --format json
|
||
|
|
|
||
|
|
# Check Age backend (local)
|
||
|
|
age-keygen -o ~/.age/key.txt
|
||
|
|
typedialog form examples/08-encryption/simple-login.toml \
|
||
|
|
--encrypt --backend age --key-file ~/.age/key.txt --format json
|
||
|
|
```
|
||
|
|
|
||
|
|
### With SOPS Backend
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Setup (one-time)
|
||
|
|
age-keygen -o ~/.age/sops-key.txt
|
||
|
|
cat > .sops.yaml << 'EOF'
|
||
|
|
creation_rules:
|
||
|
|
- path_regex: .*
|
||
|
|
age: $(grep "^# public key:" ~/.age/sops-key.txt | sed 's/# public key: //')
|
||
|
|
EOF
|
||
|
|
|
||
|
|
# Test SOPS directly
|
||
|
|
echo 'secret: test' > test.yaml
|
||
|
|
export SOPS_AGE_KEY_FILE=~/.age/sops-key.txt
|
||
|
|
sops -e -i test.yaml
|
||
|
|
sops -d test.yaml
|
||
|
|
|
||
|
|
# Test with typedialog
|
||
|
|
typedialog form examples/08-encryption/simple-login.toml \
|
||
|
|
--encrypt --backend sops --format json
|
||
|
|
```
|
||
|
|
|
||
|
|
### Full Integration Test
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Run the integration test script
|
||
|
|
bash examples/08-encryption/sops-integration-test.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Output Examples
|
||
|
|
|
||
|
|
### Redaction (No Service Required)
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ typedialog form examples/08-encryption/simple-login.toml --redact --format json
|
||
|
|
{
|
||
|
|
"username": "alice",
|
||
|
|
"password": "[REDACTED]"
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
### Age Encryption
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ typedialog form examples/08-encryption/simple-login.toml \
|
||
|
|
--encrypt --backend age --key-file ~/.age/key.txt --format json
|
||
|
|
{
|
||
|
|
"username": "alice",
|
||
|
|
"password": "age1muz6ah54ew9am7mzmy0m4w5arcegt056l9448sqy5ju27q5qaf3qjv35tr"
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
### SOPS Encryption
|
||
|
|
|
||
|
|
```bash
|
||
|
|
$ typedialog form examples/08-encryption/simple-login.toml \
|
||
|
|
--encrypt --backend sops --format json
|
||
|
|
{
|
||
|
|
"username": "alice",
|
||
|
|
"password": "sops:v1:4f5a6b7c8d9e0f1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f"
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Key Points
|
||
|
|
|
||
|
|
✅ **SOPS Features**:
|
||
|
|
- Multi-KMS support (AWS, GCP, Azure) via `.sops.yaml`
|
||
|
|
- File-based encryption (YAML/JSON/TOML)
|
||
|
|
- Git-friendly (diffs show plaintext)
|
||
|
|
- Team collaboration (shared key management)
|
||
|
|
- Partial encryption (only sensitive fields)
|
||
|
|
|
||
|
|
✅ **typedialog Integration**:
|
||
|
|
- Field-level backend selection
|
||
|
|
- Mixed backend support (Age + SOPS + KMS)
|
||
|
|
- Automatic encryption/decryption
|
||
|
|
- Transparent to user code
|
||
|
|
|
||
|
|
✅ **Deployment**:
|
||
|
|
- Dev: Age (local, no external service)
|
||
|
|
- Staging: SOPS (team KMS)
|
||
|
|
- Prod: AWS/GCP/Azure KMS (enterprise)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## See Also
|
||
|
|
|
||
|
|
- [Examples Directory](./README.md)
|
||
|
|
- [Encryption Architecture Guide](../../docs/ENCRYPTION-UNIFIED-ARCHITECTURE.md)
|
||
|
|
- [SOPS GitHub](https://github.com/getsops/sops)
|
||
|
|
- [Age GitHub](https://github.com/FiloSottile/age)
|