Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

KMS Simplification Migration Guide

Version: 0.2.0 Date: 2025-10-08 Status: Active

Overview

The KMS service has been simplified from supporting 4 backends (Vault, AWS KMS, Age, Cosmian) to supporting only 2 backends:

  • Age: Development and local testing
  • Cosmian KMS: Production deployments

This simplification reduces complexity, removes unnecessary cloud provider dependencies, and provides a clearer separation between development and production use cases.

What Changed

Removed

  • ❌ HashiCorp Vault backend (src/vault/)
  • ❌ AWS KMS backend (src/aws/)
  • ❌ AWS SDK dependencies (aws-sdk-kms, aws-config, aws-credential-types)
  • ❌ Envelope encryption helpers (AWS-specific)
  • ❌ Complex multi-backend configuration

Added

  • ✅ Age backend for development (src/age/)
  • ✅ Cosmian KMS backend for production (src/cosmian/)
  • ✅ Simplified configuration (provisioning/config/kms.toml)
  • ✅ Clear dev/prod separation
  • ✅ Better error messages

Modified

  • 🔄 KmsBackendConfig enum (now only Age and Cosmian)
  • 🔄 KmsError enum (removed Vault/AWS-specific errors)
  • 🔄 Service initialization logic
  • 🔄 README and documentation
  • 🔄 Cargo.toml dependencies

Why This Change?

Problems with Previous Approach

  1. Unnecessary Complexity: 4 backends for simple use cases
  2. Cloud Lock-in: AWS KMS dependency limited flexibility
  3. Operational Overhead: Vault requires server setup even for dev
  4. Dependency Bloat: AWS SDK adds significant compile time
  5. Unclear Use Cases: When to use which backend?

Benefits of Simplified Approach

  1. Clear Separation: Age = dev, Cosmian = prod
  2. Faster Compilation: Removed AWS SDK (saves ~30s)
  3. Offline Development: Age works without network
  4. Enterprise Security: Cosmian provides confidential computing
  5. Easier Maintenance: 2 backends instead of 4

Migration Steps

For Development Environments

If you were using Vault or AWS KMS for development:

Step 1: Install Age

# macOS
brew install age

# Ubuntu/Debian
apt install age

# From source
go install filippo.io/age/cmd/...@latest

Step 2: Generate Age Keys

mkdir -p ~/.config/provisioning/age
age-keygen -o ~/.config/provisioning/age/private_key.txt
age-keygen -y ~/.config/provisioning/age/private_key.txt > ~/.config/provisioning/age/public_key.txt

Step 3: Update Configuration

Replace your old Vault/AWS config:

Old (Vault):

[kms]
type = "vault"
address = "http://localhost:8200"
token = "${VAULT_TOKEN}"
mount_point = "transit"

New (Age):

[kms]
environment = "dev"

[kms.age]
public_key_path = "~/.config/provisioning/age/public_key.txt"
private_key_path = "~/.config/provisioning/age/private_key.txt"

Step 4: Re-encrypt Development Secrets

# Export old secrets (if using Vault)
vault kv get -format=json secret/dev > dev-secrets.json

# Encrypt with Age
cat dev-secrets.json | age -r $(cat ~/.config/provisioning/age/public_key.txt) > dev-secrets.age

# Test decryption
age -d -i ~/.config/provisioning/age/private_key.txt dev-secrets.age

For Production Environments

If you were using Vault or AWS KMS for production:

Step 1: Set Up Cosmian KMS

Choose one of these options:

Option A: Cosmian Cloud (Managed)

# Sign up at https://cosmian.com
# Get API credentials
export COSMIAN_KMS_URL=https://kms.cosmian.cloud
export COSMIAN_API_KEY=your-api-key

Option B: Self-Hosted Cosmian KMS

# Deploy Cosmian KMS server
# See: https://docs.cosmian.com/kms/deployment/

# Configure endpoint
export COSMIAN_KMS_URL=https://kms.example.com
export COSMIAN_API_KEY=your-api-key

Step 2: Create Master Key in Cosmian

# Using Cosmian CLI
cosmian-kms create-key \
  --algorithm AES \
  --key-length 256 \
  --key-id provisioning-master-key

# Or via API
curl -X POST $COSMIAN_KMS_URL/api/v1/keys \
  -H "X-API-Key: $COSMIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "algorithm": "AES",
    "keyLength": 256,
    "keyId": "provisioning-master-key"
  }'

Step 3: Migrate Production Secrets

From Vault to Cosmian:

# Export secrets from Vault
vault kv get -format=json secret/prod > prod-secrets.json

# Import to Cosmian
# (Use temporary Age encryption for transfer)
cat prod-secrets.json | \
  age -r $(cat ~/.config/provisioning/age/public_key.txt) | \
  base64 > prod-secrets.enc

# On production server with Cosmian
cat prod-secrets.enc | \
  base64 -d | \
  age -d -i ~/.config/provisioning/age/private_key.txt | \
  # Re-encrypt with Cosmian
  curl -X POST $COSMIAN_KMS_URL/api/v1/encrypt \
    -H "X-API-Key: $COSMIAN_API_KEY" \
    -d @-

From AWS KMS to Cosmian:

# Decrypt with AWS KMS
aws kms decrypt \
  --ciphertext-blob fileb://encrypted-data \
  --output text \
  --query Plaintext | \
  base64 -d > plaintext-data

# Encrypt with Cosmian
curl -X POST $COSMIAN_KMS_URL/api/v1/encrypt \
  -H "X-API-Key: $COSMIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"keyId\":\"provisioning-master-key\",\"data\":\"$(base64 plaintext-data)\"}"

Step 4: Update Production Configuration

Old (AWS KMS):

[kms]
type = "aws-kms"
region = "us-east-1"
key_id = "arn:aws:kms:us-east-1:123456789012:key/..."

New (Cosmian):

[kms]
environment = "prod"

[kms.cosmian]
server_url = "${COSMIAN_KMS_URL}"
api_key = "${COSMIAN_API_KEY}"
default_key_id = "provisioning-master-key"
tls_verify = true
use_confidential_computing = false  # Enable if using SGX/SEV

Step 5: Test Production Setup

# Set environment
export PROVISIONING_ENV=prod
export COSMIAN_KMS_URL=https://kms.example.com
export COSMIAN_API_KEY=your-api-key

# Start KMS service
cargo run --bin kms-service

# Test encryption
curl -X POST http://localhost:8082/api/v1/kms/encrypt \
  -H "Content-Type: application/json" \
  -d '{"plaintext":"SGVsbG8=","context":"env=prod"}'

# Test decryption
curl -X POST http://localhost:8082/api/v1/kms/decrypt \
  -H "Content-Type: application/json" \
  -d '{"ciphertext":"...","context":"env=prod"}'

Configuration Comparison

Before (4 Backends)

# Development could use any backend
[kms]
type = "vault"  # or "aws-kms"
address = "http://localhost:8200"
token = "${VAULT_TOKEN}"

# Production used Vault or AWS
[kms]
type = "aws-kms"
region = "us-east-1"
key_id = "arn:aws:kms:..."

After (2 Backends)

# Clear environment-based selection
[kms]
dev_backend = "age"
prod_backend = "cosmian"
environment = "${PROVISIONING_ENV:-dev}"

# Age for development
[kms.age]
public_key_path = "~/.config/provisioning/age/public_key.txt"
private_key_path = "~/.config/provisioning/age/private_key.txt"

# Cosmian for production
[kms.cosmian]
server_url = "${COSMIAN_KMS_URL}"
api_key = "${COSMIAN_API_KEY}"
default_key_id = "provisioning-master-key"
tls_verify = true

Breaking Changes

API Changes

Removed Functions

  • generate_data_key() - Now only available with Cosmian backend
  • envelope_encrypt() - AWS-specific, removed
  • envelope_decrypt() - AWS-specific, removed
  • rotate_key() - Now handled server-side by Cosmian

Changed Error Types

Before:

KmsError::VaultError(String)
KmsError::AwsKmsError(String)

After:

KmsError::AgeError(String)
KmsError::CosmianError(String)

Updated Configuration Enum

Before:

enum KmsBackendConfig {
    Vault { address, token, mount_point, ... },
    AwsKms { region, key_id, assume_role },
}

After:

enum KmsBackendConfig {
    Age { public_key_path, private_key_path },
    Cosmian { server_url, api_key, default_key_id, tls_verify },
}

Code Migration

Rust Code

Before (AWS KMS):

use kms_service::{KmsService, KmsBackendConfig};

let config = KmsBackendConfig::AwsKms {
    region: "us-east-1".to_string(),
    key_id: "arn:aws:kms:...".to_string(),
    assume_role: None,
};

let kms = KmsService::new(config).await?;

After (Cosmian):

use kms_service::{KmsService, KmsBackendConfig};

let config = KmsBackendConfig::Cosmian {
    server_url: env::var("COSMIAN_KMS_URL")?,
    api_key: env::var("COSMIAN_API_KEY")?,
    default_key_id: "provisioning-master-key".to_string(),
    tls_verify: true,
};

let kms = KmsService::new(config).await?;

Nushell Code

Before (Vault):

# Set Vault environment
$env.VAULT_ADDR = "http://localhost:8200"
$env.VAULT_TOKEN = "root"

# Use KMS
kms encrypt "secret-data"

After (Age for dev):

# Set environment
$env.PROVISIONING_ENV = "dev"

# Age keys automatically loaded from config
kms encrypt "secret-data"

Rollback Plan

If you need to rollback to Vault/AWS KMS:

# Checkout previous version
git checkout tags/v0.1.0

# Rebuild with old dependencies
cd provisioning/platform/kms-service
cargo clean
cargo build --release

# Restore old configuration
cp provisioning/config/kms.toml.backup provisioning/config/kms.toml

Testing the Migration

Development Testing

# 1. Generate Age keys
age-keygen -o /tmp/test_private.txt
age-keygen -y /tmp/test_private.txt > /tmp/test_public.txt

# 2. Test encryption
echo "test-data" | age -r $(cat /tmp/test_public.txt) > /tmp/encrypted

# 3. Test decryption
age -d -i /tmp/test_private.txt /tmp/encrypted

# 4. Start KMS service with test keys
export PROVISIONING_ENV=dev
# Update config to point to /tmp keys
cargo run --bin kms-service

Production Testing

# 1. Set up test Cosmian instance
export COSMIAN_KMS_URL=https://kms-staging.example.com
export COSMIAN_API_KEY=test-api-key

# 2. Create test key
cosmian-kms create-key --key-id test-key --algorithm AES --key-length 256

# 3. Test encryption
curl -X POST $COSMIAN_KMS_URL/api/v1/encrypt \
  -H "X-API-Key: $COSMIAN_API_KEY" \
  -d '{"keyId":"test-key","data":"dGVzdA=="}'

# 4. Start KMS service
export PROVISIONING_ENV=prod
cargo run --bin kms-service

Troubleshooting

Age Keys Not Found

# Check keys exist
ls -la ~/.config/provisioning/age/

# Regenerate if missing
age-keygen -o ~/.config/provisioning/age/private_key.txt
age-keygen -y ~/.config/provisioning/age/private_key.txt > ~/.config/provisioning/age/public_key.txt

Cosmian Connection Failed

# Check network connectivity
curl -v $COSMIAN_KMS_URL/api/v1/health

# Verify API key
curl $COSMIAN_KMS_URL/api/v1/version \
  -H "X-API-Key: $COSMIAN_API_KEY"

# Check TLS certificate
openssl s_client -connect kms.example.com:443

Compilation Errors

# Clean and rebuild
cd provisioning/platform/kms-service
cargo clean
cargo update
cargo build --release

Support

  • Documentation: See README.md
  • Issues: Report on project issue tracker
  • Cosmian Support: https://docs.cosmian.com/support/

Timeline

  • 2025-10-08: Migration guide published
  • 2025-10-15: Deprecation notices for Vault/AWS
  • 2025-11-01: Old backends removed from codebase
  • 2025-11-15: Migration complete, old configs unsupported

FAQs

Q: Can I still use Vault if I really need to? A: No, Vault support has been removed. Use Age for dev or Cosmian for prod.

Q: What about AWS KMS for existing deployments? A: Migrate to Cosmian KMS. The API is similar, and migration tools are provided.

Q: Is Age secure enough for production? A: No. Age is designed for development only. Use Cosmian KMS for production.

Q: Does Cosmian support confidential computing? A: Yes, Cosmian KMS supports SGX and SEV for confidential computing workloads.

Q: How much does Cosmian cost? A: Cosmian offers both cloud and self-hosted options. Contact Cosmian for pricing.

Q: Can I use my own KMS backend? A: Not currently supported. Only Age and Cosmian are available.

Checklist

Use this checklist to track your migration:

Development Migration

  • Install Age (brew install age or equivalent)
  • Generate Age keys (age-keygen)
  • Update provisioning/config/kms.toml to use Age backend
  • Export secrets from Vault/AWS (if applicable)
  • Re-encrypt secrets with Age
  • Test KMS service startup
  • Test encrypt/decrypt operations
  • Update CI/CD pipelines (if applicable)
  • Update documentation

Production Migration

  • Set up Cosmian KMS server (cloud or self-hosted)
  • Create master key in Cosmian
  • Export production secrets from Vault/AWS
  • Re-encrypt secrets with Cosmian
  • Update provisioning/config/kms.toml to use Cosmian backend
  • Set environment variables (COSMIAN_KMS_URL, COSMIAN_API_KEY)
  • Test KMS service startup in staging
  • Test encrypt/decrypt operations in staging
  • Load test Cosmian integration
  • Update production deployment configs
  • Deploy to production
  • Verify all secrets accessible
  • Decommission old KMS infrastructure

Conclusion

The KMS simplification reduces complexity while providing better separation between development and production use cases. Age offers a fast, offline solution for development, while Cosmian KMS provides enterprise-grade security for production deployments.

For questions or issues, please refer to the documentation or open an issue.