TypeDialog/docs/ENCRYPTION-SERVICES-SETUP.md

# HOW-TO: Configure and Run Encryption Services for typedialog

## Overview

This guide walks through setting up **Age** (local file-based encryption) and **RustyVault** (HTTP-based encryption service) to test the typedialog encryption pipeline end-to-end.

**Service Matrix:**

| Backend | Type | Setup Complexity | Network | Requires |
|---------|------|------------------|---------|----------|
| **Age** | Local file-based | Trivial | None | age CLI tool |
| **RustyVault** | HTTP vault server | Moderate | localhost:8200 | Docker or manual build |
| **SOPS** | External tool | Complex | Varies | sops CLI + backends |

This guide covers Age (trivial) and RustyVault (moderate). SOPS is skipped for now.

---

## Part 1: Age Backend (Local File Encryption)

### What is Age?

Age is a simple, modern encryption tool using X25519 keys. Perfect for development because:
- No daemon/service required
- Keys stored as plaintext files
- Single binary

### Installation

**macOS (via Homebrew):**
```bash
brew install age
```

**Linux (Ubuntu/Debian):**
```bash
sudo apt-get install age
```

**Manual (any OS):**
```bash
# Download from https://github.com/FiloSottile/age/releases
# Extract and add to PATH
tar xzf age-v1.1.1-linux-amd64.tar.gz
sudo mv age/age /usr/local/bin/
sudo mv age/age-keygen /usr/local/bin/
```

**Verify installation:**
```bash
age --version
# age v1.1.1
```

### Generate Age Key Pair

Age uses a single private key file that contains both public and private components. The public key is derived from the private key.

**Generate keys for testing:**
```bash
# Create a test directory
mkdir -p ~/.age

# Generate private key
age-keygen -o ~/.age/key.txt

# Output will show:
# Public key: age1...xxx (save this, shown in file)
# Written to /home/user/.age/key.txt
```

**Verify key generation:**
```bash
# Check private key exists
cat ~/.age/key.txt
# Output: AGE-SECRET-KEY-1XXXX...

# Extract public key (age CLI does this automatically)
grep "^public key:" ~/.age/key.txt | cut -d' ' -f3
```

### Test Age Encryption Locally

**Create a test plaintext file:**
```bash
echo "This is a secret message" > test_message.txt
```

**Encrypt with age:**
```bash
# Get public key from private key
PUBLIC_KEY=$(grep "^public key:" ~/.age/key.txt | cut -d' ' -f3)

# Encrypt
age -r "$PUBLIC_KEY" test_message.txt > test_message.age

# Verify ciphertext is unreadable
cat test_message.age
# Output: AGE-ENCRYPTION-V1...binary...
```

**Decrypt with age:**
```bash
# Decrypt (will prompt for passphrase if key is encrypted)
age -d -i ~/.age/key.txt test_message.age

# Output: This is a secret message
```

### Configure typedialog to Use Age

**Environment variables:**
```bash
export AGE_KEY_FILE="$HOME/.age/key.txt"
```

**CLI flags:**
```bash
# Redact mode (no encryption needed)
typedialog form examples/08-encryption/simple-login.toml --redact --format json

# Encrypt mode (requires Age backend)
typedialog form examples/08-encryption/simple-login.toml --encrypt --backend age --key-file ~/.age/key.txt --format json
```

See `examples/08-encryption/README.md` for more example forms and test cases.

**TOML form configuration:**
```toml
[[fields]]
name = "password"
type = "password"
prompt = "Enter password"
sensitive = true
encryption_backend = "age"

[fields.encryption_config]
key = "~/.age/key.txt"
```

---

## Part 2: RustyVault Backend (HTTP Service)

### What is RustyVault?

RustyVault is a Rust implementation of HashiCorp Vault's Transit API:
- HTTP-based encryption/decryption service
- Suitable for production environments
- API-compatible with Vault Transit secrets engine

### Installation & Setup

**Option A: Docker (Recommended for testing)**

RustyVault provides official Docker images. Check availability:
```bash
# Search Docker Hub
docker search rustyvault

# Or build from source
git clone https://github.com/Tongsuo-Project/RustyVault.git
cd RustyVault
docker build -t rustyvault:latest .
```

**Option B: Manual Build (if Docker not available)**

```bash
# Prerequisites: Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Clone and build
git clone https://github.com/Tongsuo-Project/RustyVault.git
cd RustyVault
cargo build --release

# Binary at: target/release/rustyvault
```

### Run RustyVault Service

**Using Docker (single command):**
```bash
docker run -d \
  --name rustyvault \
  -p 8200:8200 \
  -e RUSTYVAULT_LOG_LEVEL=info \
  rustyvault:latest

# Verify it started
docker logs rustyvault | head -20
```

**Using local binary:**
```bash
# Create config directory
mkdir -p ~/.rustyvault
cd ~/.rustyvault

# Create minimal config (rustyvault.toml)
cat > config.toml <<'EOF'
[server]
address = "127.0.0.1:8200"
tls_disable = true

[backend]
type = "inmem"  # In-memory storage (ephemeral)
EOF

# Run service
~/RustyVault/target/release/rustyvault server -c config.toml
```

**Verify service is running:**
```bash
# In another terminal
curl -s http://localhost:8200/v1/sys/health | jq .
# Should return health status JSON
```

### Configure RustyVault for Encryption

**Initialize RustyVault (first time only):**
```bash
# Generate initial token
VAULT_INIT=$(curl -s -X POST http://localhost:8200/v1/sys/init \
  -d '{"secret_shares": 1, "secret_threshold": 1}' | jq -r .keys[0])

# Unseal vault
curl -s -X PUT http://localhost:8200/v1/sys/unseal \
  -d "{\"key\": \"$VAULT_INIT\"}" > /dev/null

# Save root token
ROOT_TOKEN=$(curl -s -X POST http://localhost:8200/v1/sys/unseal \
  -d "{\"key\": \"$VAULT_INIT\"}" | jq -r .auth.client_token)

export VAULT_TOKEN="$ROOT_TOKEN"
```

**Enable Transit secrets engine:**
```bash
curl -s -X POST http://localhost:8200/v1/sys/mounts/transit \
  -H "X-Vault-Token: $VAULT_TOKEN" \
  -d '{"type": "transit"}' | jq .
```

**Create encryption key:**
```bash
curl -s -X POST http://localhost:8200/v1/transit/keys/typedialog-key \
  -H "X-Vault-Token: $VAULT_TOKEN" \
  -d '{}' | jq .

# Verify key created
curl -s http://localhost:8200/v1/transit/keys/typedialog-key \
  -H "X-Vault-Token: $VAULT_TOKEN" | jq .
```

### Test RustyVault Encryption

**Encrypt data via HTTP:**
```bash
# Plaintext (base64 encoded)
PLAINTEXT=$(echo -n "my-secret-password" | base64)

curl -s -X POST http://localhost:8200/v1/transit/encrypt/typedialog-key \
  -H "X-Vault-Token: $VAULT_TOKEN" \
  -d "{\"plaintext\": \"$PLAINTEXT\"}" | jq .data.ciphertext
```

**Decrypt data via HTTP:**
```bash
# From encryption output above
CIPHERTEXT="vault:v1:..."

curl -s -X POST http://localhost:8200/v1/transit/decrypt/typedialog-key \
  -H "X-Vault-Token: $VAULT_TOKEN" \
  -d "{\"ciphertext\": \"$CIPHERTEXT\"}" | jq -r .data.plaintext | base64 -d
```

### Configure typedialog to Use RustyVault

**Environment variables:**
```bash
export VAULT_ADDR="http://localhost:8200"
export VAULT_TOKEN="s.xxxx..." # Token from above
```

**CLI flags:**
```bash
typedialog form examples/08-encryption/credentials.toml \
  --encrypt \
  --backend rustyvault \
  --vault-addr http://localhost:8200 \
  --vault-token "s.xxxx..." \
  --vault-key-path "transit/keys/typedialog-key" \
  --format json
```

This form includes field-level RustyVault configuration in the `vault_token` field.

**TOML form configuration:**
```toml
[[fields]]
name = "password"
type = "password"
prompt = "Enter password"
sensitive = true
encryption_backend = "rustyvault"

[fields.encryption_config]
vault_addr = "http://localhost:8200"
vault_token = "s.xxxx..."
key_path = "transit/keys/typedialog-key"
```

---

## Part 3: Complete Integration Test Workflow

### Script: Setup Everything

Create `scripts/encryption-test-setup.sh`:

```bash
#!/usr/bin/env bash
set -e

echo "=== typedialog Encryption Services Setup ==="

# Age Setup
echo "1. Setting up Age..."
if ! command -v age &> /dev/null; then
  echo "   ✗ age not installed. Run: brew install age"
  exit 1
fi

mkdir -p ~/.age
if [ ! -f ~/.age/key.txt ]; then
  echo "   → Generating Age keys..."
  age-keygen -o ~/.age/key.txt
fi
export AGE_KEY_FILE="$HOME/.age/key.txt"
echo "   ✓ Age configured at: $AGE_KEY_FILE"

# RustyVault Setup (Docker)
echo ""
echo "2. Setting up RustyVault (Docker)..."
if ! command -v docker &> /dev/null; then
  echo "   ⚠ Docker not installed, skipping RustyVault"
  echo "   → Install Docker or skip RustyVault tests"
else
  if ! docker ps | grep -q rustyvault; then
    echo "   → Starting RustyVault container..."
    docker run -d \
      --name rustyvault \
      -p 8200:8200 \
      -e RUSTYVAULT_LOG_LEVEL=info \
      rustyvault:latest
    sleep 2
  fi

  # Initialize vault
  echo "   → Initializing RustyVault..."
  VAULT_INIT=$(curl -s -X POST http://localhost:8200/v1/sys/init \
    -d '{"secret_shares": 1, "secret_threshold": 1}' | jq -r .keys[0])

  curl -s -X PUT http://localhost:8200/v1/sys/unseal \
    -d "{\"key\": \"$VAULT_INIT\"}" > /dev/null

  # Get root token
  RESPONSE=$(curl -s -X GET http://localhost:8200/v1/sys/unseal \
    -H "X-Vault-Token: $VAULT_INIT")
  export VAULT_TOKEN=$(echo "$RESPONSE" | jq -r .auth.client_token // "root")
  export VAULT_ADDR="http://localhost:8200"

  # Enable transit
  curl -s -X POST http://localhost:8200/v1/sys/mounts/transit \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -d '{"type": "transit"}' > /dev/null 2>&1 || true

  # Create key
  curl -s -X POST http://localhost:8200/v1/transit/keys/typedialog-key \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -d '{}' > /dev/null 2>&1 || true

  echo "   ✓ RustyVault running at: http://localhost:8200"
  echo "   ✓ Token: $VAULT_TOKEN"
fi

echo ""
echo "=== Setup Complete ==="
echo ""
echo "Test Age encryption:"
echo "  typedialog form test.toml --encrypt --backend age --key-file ~/.age/key.txt"
echo ""
echo "Test RustyVault encryption:"
echo "  export VAULT_ADDR='http://localhost:8200'"
echo "  export VAULT_TOKEN='$VAULT_TOKEN'"
echo "  typedialog form test.toml --encrypt --backend rustyvault --vault-key-path 'transit/keys/typedialog-key'"
```

**Make executable and run:**
```bash
chmod +x scripts/encryption-test-setup.sh
./scripts/encryption-test-setup.sh
```

### Test Case 1: Age Redaction (No Service Required)

**Option A: Use pre-built example (Recommended)**
```bash
typedialog form examples/08-encryption/simple-login.toml --redact --format json

# Expected output:
# {"username": "alice", "password": "[REDACTED]"}
```

**Option B: Create form manually**
```bash
# Create test form
cat > test_redaction.toml <<'EOF'
name = "test_form"
display_mode = "complete"

[[fields]]
name = "username"
type = "text"
prompt = "Username"

[[fields]]
name = "password"
type = "password"
prompt = "Password"
sensitive = true
EOF

# Test redaction (requires no service)
typedialog form test_redaction.toml --redact --format json
```

### Test Case 2: Age Encryption (Service Not Required, Key File Required)

**Option A: Use pre-built example (Recommended)**
```bash
# Prerequisites: Age key generated (from setup script)
./scripts/encryption-test-setup.sh

# Test with simple form
typedialog form examples/08-encryption/simple-login.toml \
  --encrypt --backend age --key-file ~/.age/key.txt --format json

# Expected output: password field contains age ciphertext
# {"username": "alice", "password": "age1muz6ah54ew9am7mzmy0m4w5..."}

# Or test with full credentials form
typedialog form examples/08-encryption/credentials.toml \
  --encrypt --backend age --key-file ~/.age/key.txt --format json
```

**Option B: Create form manually**
```bash
# Generate Age key if not exists
mkdir -p ~/.age
if [ ! -f ~/.age/key.txt ]; then
  age-keygen -o ~/.age/key.txt
fi

# Create test form
cat > test_age_encrypt.toml <<'EOF'
name = "test_form"
display_mode = "complete"

[[fields]]
name = "username"
type = "text"
prompt = "Username"

[[fields]]
name = "password"
type = "password"
prompt = "Password"
sensitive = true
encryption_backend = "age"

[fields.encryption_config]
key = "~/.age/key.txt"
EOF

# Test encryption (requires Age key file)
typedialog form test_age_encrypt.toml --encrypt --backend age --key-file ~/.age/key.txt --format json
```

### Test Case 3: RustyVault Encryption (Service Required)

**Prerequisites: RustyVault running**
```bash
# Start RustyVault and setup (requires Docker)
./scripts/encryption-test-setup.sh

# Verify service is healthy
curl http://localhost:8200/v1/sys/health | jq .
```

**Option A: Use pre-built example (Recommended)**
```bash
# Export Vault credentials
export VAULT_ADDR="http://localhost:8200"
export VAULT_TOKEN="root"

# Test with simple form
typedialog form examples/08-encryption/simple-login.toml \
  --encrypt --backend rustyvault \
  --vault-key-path "transit/keys/typedialog-key" \
  --format json

# Expected output: password field contains vault ciphertext
# {"username": "alice", "password": "vault:v1:K8..."}

# Or test with full credentials form (demonstrates field-level config)
typedialog form examples/08-encryption/credentials.toml \
  --encrypt --backend rustyvault \
  --vault-key-path "transit/keys/typedialog-key" \
  --format json
```

**Option B: Create form manually**
```bash
cat > test_vault_encrypt.toml <<'EOF'
name = "test_form"
display_mode = "complete"

[[fields]]
name = "username"
type = "text"
prompt = "Username"

[[fields]]
name = "password"
type = "password"
prompt = "Password"
sensitive = true
encryption_backend = "rustyvault"

[fields.encryption_config]
vault_addr = "http://localhost:8200"
key_path = "transit/keys/typedialog-key"
EOF

# Test encryption with RustyVault
export VAULT_TOKEN="s.xxxx" # From setup output
export VAULT_ADDR="http://localhost:8200"

typedialog form test_vault_encrypt.toml \
  --encrypt \
  --backend rustyvault \
  --vault-addr http://localhost:8200 \
  --vault-token "$VAULT_TOKEN" \
  --vault-key-path "transit/keys/typedialog-key" \
  --format json

# Expected output: password field contains vault ciphertext
# {"username": "alice", "password": "vault:v1:..."}
```

---

## Part 4: Run Actual Integration Tests

### Test Case: Age Roundtrip (Encrypt → Decrypt)

Once Age is set up, these test scenarios validate the pipeline:

**Scenario 1: Redaction works (no encryption service)**
```bash
cargo test --test nickel_integration test_encryption_roundtrip_with_redaction -- --nocapture

# Expected: PASS - redacts sensitive fields
```

**Scenario 2: Metadata mapping works**
```bash
cargo test --test nickel_integration test_encryption_metadata_to_field_definition -- --nocapture

# Expected: PASS - EncryptionMetadata maps to FieldDefinition
```

**Scenario 3: Auto-detection of password fields**
```bash
cargo test --test nickel_integration test_encryption_auto_detection_from_field_type -- --nocapture

# Expected: PASS - Password fields auto-marked as sensitive
```

### Run All Encryption Tests

```bash
cargo test --test nickel_integration test_encryption -- --nocapture
```

**Current status:**
- ✅ 5 tests passing (redaction, metadata mapping)
- ⏳ 0 tests for actual Age encryption roundtrip (not yet implemented)
- ⏳ 0 tests for RustyVault integration (backend not implemented)

---

## Part 5: Troubleshooting

### Age Issues

**Problem: `age: command not found`**
```bash
# Install age
brew install age  # macOS
sudo apt install age  # Linux
```

**Problem: Permission denied on ~/.age/key.txt**
```bash
chmod 600 ~/.age/key.txt
```

**Problem: Invalid key format**
```bash
# Regenerate keys
rm ~/.age/key.txt
age-keygen -o ~/.age/key.txt
```

### RustyVault Issues

**Problem: Docker container won't start**
```bash
# Check logs
docker logs rustyvault

# Remove and restart
docker rm -f rustyvault
docker run -d --name rustyvault -p 8200:8200 rustyvault:latest
```

**Problem: Vault initialization fails**
```bash
# Check if vault is responding
curl -s http://localhost:8200/v1/sys/health

# If not, restart container
docker restart rustyvault
```

**Problem: Transit API not working**
```bash
# Verify token
echo $VAULT_TOKEN

# Check auth
curl -s http://localhost:8200/v1/sys/mounts \
  -H "X-Vault-Token: $VAULT_TOKEN"
```

**Problem: Can't connect from typedialog**
```bash
# Verify network
curl -s http://localhost:8200/v1/sys/health | jq .

# Check environment variables
echo $VAULT_ADDR
echo $VAULT_TOKEN

# Test encryption endpoint
curl -s -X POST http://localhost:8200/v1/transit/encrypt/typedialog-key \
  -H "X-Vault-Token: $VAULT_TOKEN" \
  -d '{"plaintext": "dGVzdA=="}' | jq .
```

---

## Part 6: Next Steps

Once services are running, implement:

1. **test_age_encrypt_roundtrip** - Encrypt with Age, decrypt, verify plaintext
2. **test_rustyvault_encrypt_roundtrip** - Encrypt with RustyVault, decrypt, verify
3. **test_cli_encrypt_age** - Run `typedialog form --encrypt --backend age`, verify output is ciphertext
4. **test_cli_encrypt_rustyvault** - Run `typedialog form --encrypt --backend rustyvault`, verify output is ciphertext
5. **Integration test script** - Single script that tests all pipelines end-to-end

---

## References

- **Age**: https://github.com/FiloSottile/age
- **RustyVault**: https://github.com/Tongsuo-Project/RustyVault
- **HashiCorp Vault Transit**: https://www.vaultproject.io/api-docs/secret/transit