TypeDialog/docs/ENCRYPTION-UNIFIED-ARCHITECTURE.md

# Unified Encryption Architecture

This document explains the updated encryption architecture for typedialog and how it integrates with the unified encryption system from prov-ecosystem.

## Overview

typedialog now uses a **unified encryption API** from the `encrypt` crate, eliminating backend-specific code and enabling support for multiple encryption backends (Age, SOPS, SecretumVault, AWS/GCP/Azure KMS) through a single API.

**Key Benefits:**
- Single code path supports all backends
- Configuration-driven backend selection
- Multi-backend support in TOML and Nickel schemas
- Post-quantum cryptography ready (via SecretumVault)
- Cleaner, more maintainable code

## Architecture Changes

### Before: Direct Backend Instantiation

```rust
// OLD: Direct Age backend instantiation
use encrypt::backend::age::AgeBackend;

let backend = AgeBackend::with_defaults()?;
let ciphertext = backend.encrypt(&plaintext)?;

// To support other backends, need separate code paths
#[match]
"sops" => { /* SOPS code */ }
"rustyvault" => { /* RustyVault code */ }
```

### After: Unified API with BackendSpec

```rust
// NEW: Configuration-driven, backend-agnostic
use encrypt::{encrypt, BackendSpec};

// Same code for all backends
let spec = BackendSpec::age_default();
let ciphertext = encrypt(&plaintext, &spec)?;

// Or SOPS
let spec = BackendSpec::sops();
let ciphertext = encrypt(&plaintext, &spec)?;

// Or KMS
let spec = BackendSpec::aws_kms(region, key_id);
let ciphertext = encrypt(&plaintext, &spec)?;
```

## Integration Points

### 1. TOML Form Configuration

No changes required for existing TOML configurations. The system transparently uses the new API:

```toml
[[fields]]
name = "password"
type = "password"
sensitive = true
encryption_backend = "age"
encryption_config = { key_file = "~/.age/key.txt" }
```

The `encryption_bridge` module automatically converts this to `BackendSpec::age(...)` and uses the unified API.

### 2. Nickel Schema Integration

Nickel support remains unchanged but now uses the unified backend system:

```nickel
{
  # Age backend for development
  dev_password | Sensitive Backend="age" Key="~/.age/key.txt" = "",

  # SOPS for staging
  staging_secret | Sensitive Backend="sops" = "",

  # SecretumVault Transit Engine for production (post-quantum)
  prod_token | Sensitive Backend="secretumvault"
             Vault="https://vault.internal:8200"
             Key="app-key" = "",

  # AWS KMS for cloud-native deployments
  aws_secret | Sensitive Backend="awskms"
             Region="us-east-1"
             KeyId="arn:aws:kms:..." = "",
}
```

### 3. Internal Encryption Function

The `transform_sensitive_value()` function now uses the unified API:

```rust
// File: crates/typedialog-core/src/helpers.rs

fn transform_sensitive_value(
    value: &Value,
    field: &FieldDefinition,
    context: &EncryptionContext,
    _global_config: Option<&EncryptionDefaults>,
) -> Result<Value> {
    // Convert field definition to BackendSpec
    let spec = crate::encryption_bridge::field_to_backend_spec(field, None)?;

    // Use unified API
    let plaintext = serde_json::to_string(value)?;
    let ciphertext = encrypt::encrypt(&plaintext, &spec)?;

    Ok(Value::String(ciphertext))
}
```

## New Bridge Module

New `encryption_bridge.rs` module provides seamless conversion:

```rust
// File: crates/typedialog-core/src/encryption_bridge.rs

pub fn field_to_backend_spec(
    field: &FieldDefinition,
    default_backend: Option<&str>,
) -> Result<encrypt::BackendSpec>
```

**Conversion Logic:**
1. Reads `field.encryption_backend` (or uses default)
2. Extracts `field.encryption_config` (backend-specific settings)
3. Validates required configuration for the backend
4. Returns `BackendSpec` ready for use with `encrypt::encrypt()`

**Supported Backends:**
- ✓ Age (with custom key paths)
- ✓ SOPS (minimal config)
- ✓ SecretumVault (vault_addr, vault_token, key_name)
- ✓ AWS KMS (region, key_id)
- ✓ GCP KMS (project_id, key_ring, crypto_key, location)
- ✓ Azure KMS (vault_name, tenant_id)

## Configuration Changes

### Cargo.toml

No changes required - encryption support includes all commonly used backends by default.

To customize backends:

```toml
[dependencies]
# Default: age + other major backends
typedialog-core = { path = "...", features = ["encryption"] }

# Only Age
typedialog-core = { path = "...", features = ["encryption"] }
# (Age is default in encrypt crate)

# Custom selection
typedialog-core = { path = "...", features = ["encryption"] }
# Depends on encrypt crate configuration
```

### Environment Variables

Backend-specific configuration via environment:

**Age:**
```bash
# Uses ~/.age/key.txt by default
# Or specify via field config: encryption_config = { key_file = "/custom/path" }
```

**SOPS:**
```bash
# Uses .sops.yaml in current/parent directories
```

**SecretumVault:**
```bash
export VAULT_ADDR="https://vault.internal:8200"
export VAULT_TOKEN="hvs.CAAA..."
```

**AWS KMS:**
```bash
export AWS_REGION="us-east-1"
# AWS credentials from standard chain (env vars, ~/.aws/credentials, IAM roles)
```

**GCP KMS:**
```bash
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
```

**Azure KMS:**
```bash
# Azure CLI authentication or environment variables
```

## Error Handling

Enhanced error messages guide users through troubleshooting:

```
Error: Encryption failed: Backend 'age' not available.
  Enable feature 'age' in Cargo.toml

Error: Encryption failed: SecretumVault backend requires
  vault_addr in encryption_config

Error: Encryption failed: AWS KMS backend requires region
  in encryption_config
```

## Testing

### Running Encryption Tests

```bash
# All encryption tests
cargo test --features encryption

# Only encryption integration tests
cargo test --features encryption --test encryption_integration

# Specific test
cargo test --features encryption test_age_roundtrip_encrypt_decrypt
```

### Test Results

All 15 encryption integration tests pass:
- ✓ 7 encryption behavior tests
- ✓ 8 Age backend roundtrip tests

```
running 15 tests
test encryption_tests::test_explicit_non_sensitive_overrides_password_type ... ok
test encryption_tests::test_auto_detect_password_field_as_sensitive ... ok
test encryption_tests::test_redaction_preserves_non_sensitive ... ok
test encryption_tests::test_multiple_sensitive_fields ... ok
test encryption_tests::test_redaction_in_json_output ... ok
test encryption_tests::test_unknown_backend_error ... ok
test encryption_tests::test_redaction_in_yaml_output ... ok
test age_roundtrip_tests::test_age_backend_availability ... ok
test age_roundtrip_tests::test_age_invalid_ciphertext_fails ... ok
test age_roundtrip_tests::test_age_encryption_produces_ciphertext ... ok
test age_roundtrip_tests::test_age_roundtrip_encrypt_decrypt ... ok
test age_roundtrip_tests::test_age_handles_empty_string ... ok
test age_roundtrip_tests::test_age_handles_unicode ... ok
test age_roundtrip_tests::test_age_encryption_different_ciphertexts ... ok
test age_roundtrip_tests::test_age_handles_large_values ... ok

test result: ok. 15 passed; 0 failed
```

## Migration Path

### For Existing Users

**No action required.** The system is backward compatible:

1. Existing TOML forms work unchanged
2. Existing Nickel schemas work unchanged
3. Internal implementation now uses unified API
4. No visible changes to users

### For New Deployments

Can now use additional backends:

```toml
# Now supports more backends in single codebase
[[fields]]
name = "secret"
type = "text"
sensitive = true
encryption_backend = "secretumvault"  # Or awskms, gcpkms, azurekms
encryption_config = { vault_addr = "...", vault_token = "..." }
```

### For Code Extending typedialog

If extending typedialog with custom encryption logic:

```rust
// OLD: Manual backend instantiation (still works)
use encrypt::backend::age::AgeBackend;
let backend = AgeBackend::new(pub_key, priv_key)?;

// NEW: Use bridge module + unified API (recommended)
use encrypt::encrypt;
use typedialog_core::encryption_bridge;

let spec = encryption_bridge::field_to_backend_spec(&field, None)?;
let ciphertext = encrypt(&plaintext, &spec)?;
```

## Multi-Backend Support

### Same Code, Different Configs

```toml
# Development (Age)
[[fields]]
name = "db_password"
sensitive = true
encryption_backend = "age"

# Production (SecretumVault - post-quantum)
[[fields]]
name = "db_password"
sensitive = true
encryption_backend = "secretumvault"
encryption_config = { vault_addr = "https://vault.prod:8200", vault_token = "..." }
```

Same Rust code handles both without changes.

### CLI Overrides

```bash
# Override backend from command line
typedialog form config.toml --encrypt --backend secretumvault

# Use with environment variables
export VAULT_ADDR="https://vault.internal:8200"
export VAULT_TOKEN="hvs.CAAA..."
typedialog form config.toml --encrypt --backend secretumvault
```

## Feature Flags

Backends are feature-gated in the encrypt crate:

```rust
// With feature enabled
#[cfg(feature = "age")]
{
    let spec = BackendSpec::age_default();
    encrypt(&plaintext, &spec)?;  // Works
}

// Without feature
#[cfg(not(feature = "age"))]
{
    let spec = BackendSpec::age_default();
    encrypt(&plaintext, &spec)?;  // Returns: Backend 'age' not available
}
```

## Post-Quantum Cryptography

### SecretumVault Transit Engine

For post-quantum cryptography support, use SecretumVault with ML-KEM/ML-DSA:

```toml
[[fields]]
name = "pqc_secret"
sensitive = true
encryption_backend = "secretumvault"
encryption_config = {
    vault_addr = "https://pq-vault.internal:8200",
    vault_token = "hvs.CAAA...",
    key_name = "pqc-key"  # Uses ML-KEM encapsulation, ML-DSA signatures
}
```

**Requirements:**
- SecretumVault server configured with Transit Engine
- Post-quantum crypto backend enabled (aws-lc-rs or Tongsuo)

## Troubleshooting

### "Backend not available" Error

**Problem:** Encryption fails with "Backend 'age' not available"

**Solution:** Feature may not be enabled in encrypt crate. Check:

```bash
# Check available backends
cargo build --features encryption --verbose

# Look for feature compilation
# It should show: "Compiling encrypt ... with features: age,..."
```

### "Invalid ciphertext" After Update

**Problem:** Old ciphertexts fail to decrypt

**Solution:** Age format hasn't changed. Verify:
1. Same Age key is used
2. Ciphertext format is valid (hex-encoded)
3. Key file permissions: `chmod 600 ~/.age/key.txt`

### Form TOML Backward Compatibility

**Problem:** Existing TOML forms stop working after update

**Solution:** No breaking changes. Forms should work as-is. If not:

1. Verify encryption_backend name is valid
2. Check encryption_config required fields
3. Test with: `cargo test --features encryption`

## Testing with Mock Backend

For faster tests without real encryption keys:

```bash
# In development/testing
cargo test --features test-util
```

MockBackend provides deterministic encryption for CI/CD:

```rust
#[cfg(test)]
use encrypt::test_util::MockBackend;

let backend = MockBackend::new();
let ct = backend.encrypt("secret")?;
// Fast, reproducible, no real keys needed
```

## See Also

- [ENCRYPTION-QUICK-START.md](ENCRYPTION-QUICK-START.md) - Getting started with encryption
- [ENCRYPTION-SERVICES-SETUP.md](ENCRYPTION-SERVICES-SETUP.md) - Setting up encryption services
- [../../prov-ecosystem/docs/guides/ENCRYPTION.md](../../prov-ecosystem/docs/guides/ENCRYPTION.md) - Comprehensive encryption guide
- [encryption_bridge.rs](crates/typedialog-core/src/encryption_bridge.rs) - Bridge module source
- [../../prov-ecosystem/crates/encrypt](../../prov-ecosystem/crates/encrypt) - encrypt crate source