HeliosDB Lite - Transparent Data Encryption (TDE)

Overview

HeliosDB Lite implements transparent data encryption (TDE) using AES-256-GCM, a NIST-approved authenticated encryption algorithm. All data stored on disk, including table data and catalog metadata, is automatically encrypted and decrypted without any changes to your application code.

Features

• AES-256-GCM Encryption: Industry-standard authenticated encryption with associated data (AEAD)
• Transparent Operation: Zero application changes required - encryption happens automatically
• Multiple Key Sources: Support for environment variables, files, and cloud KMS (planned)
• Secure Key Management: Keys are stored securely in memory with automatic zeroing on drop
• Password-based Keys: Derive encryption keys from passwords using Argon2
• Metadata Encryption: Both data and catalog metadata are encrypted
• Minimal Performance Overhead: Typically less than 3% performance impact

Configuration

Basic Configuration

Create a config.toml file with encryption settings:

[encryption]
enabled = true
algorithm = "Aes256Gcm"
key_source = { Environment = "HELIOSDB_ENCRYPTION_KEY" }
rotation_interval_days = 90

Key Source Options

1. Environment Variable (Recommended for Development)

[encryption]
enabled = true
key_source = { Environment = "HELIOSDB_ENCRYPTION_KEY" }

Set the environment variable with a 64-character hex string (32 bytes):

export HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"

2. File-based Key (Recommended for Production)

[encryption]
enabled = true
key_source = { File = "/secure/path/to/encryption.key" }

The key file should contain either: - 32 raw bytes (binary file) - 64 hexadecimal characters (text file)

3. Cloud KMS (Planned for Future Release)

[encryption]
enabled = true
key_source = { Kms = { provider = "aws", key_id = "alias/heliosdb-key" } }

Generating Encryption Keys

Using OpenSSL

Generate a random 256-bit key:

openssl rand -hex 32 > encryption.key

Using Rust Code

use heliosdb_lite::crypto::KeyManager;

let km = KeyManager::generate_random();
let hex_key = km.export_as_hex();
println!("Generated key: {}", hex_key);

// Save to file
std::fs::write("encryption.key", hex_key)?;

From Password (Argon2 Key Derivation)

use heliosdb_lite::crypto::KeyManager;

let password = "your_strong_password_here";
let salt = b"unique_salt_16bytes!"; // Should be random and stored

let km = KeyManager::from_password(password, salt)?;

Usage Examples

Embedded Mode with Encryption

use heliosdb_lite::{Config, EmbeddedDatabase};

// Set encryption key
std::env::set_var("HELIOSDB_ENCRYPTION_KEY",
    "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef");

// Load configuration
let mut config = Config::default();
config.encryption.enabled = true;
config.encryption.key_source =
    heliosdb_lite::config::KeySource::Environment("HELIOSDB_ENCRYPTION_KEY".to_string());

// Create database - all data will be encrypted
let db = EmbeddedDatabase::new_with_config("./secure-db", &config)?;

// Use normally - encryption is transparent
db.execute("CREATE TABLE secrets (id INT, data TEXT)")?;
db.execute("INSERT INTO secrets VALUES (1, 'classified')")?;

let results = db.query("SELECT * FROM secrets", &[])?;

In-Memory Database with Encryption

use heliosdb_lite::{Config, EmbeddedDatabase};

let mut config = Config::in_memory();
config.encryption.enabled = true;
config.encryption.key_source =
    heliosdb_lite::config::KeySource::Environment("HELIOSDB_ENCRYPTION_KEY".to_string());

let db = EmbeddedDatabase::new_in_memory_with_config(&config)?;

Checking Encryption Status

use heliosdb_lite::storage::StorageEngine;

let storage = StorageEngine::open("./db", &config)?;

if storage.is_encrypted() {
    println!("Database is encrypted");
    if let Some(info) = storage.encryption_info() {
        println!("Encryption: {}", info);
    }
}

Security Best Practices

1. Key Management

• Never commit keys to version control
• Use environment variables for development
• Use dedicated key management services (KMS) for production
• Rotate keys regularly (every 90 days recommended)
• Store keys separately from data

2. Key Storage

• Use file permissions to restrict key file access:

  chmod 600 /path/to/encryption.key
  chown database-user:database-group /path/to/encryption.key
  

• Consider using encrypted filesystems for key storage

• Use hardware security modules (HSM) for critical environments

3. Password-based Keys

If using password-based key derivation: - Use strong passwords (minimum 16 characters) - Use random salts (minimum 16 bytes) - Store salts securely but separately from passwords - Never hardcode passwords in application code

4. Backup Considerations

• Critical: Backup encryption keys separately from database files
• Without the key, encrypted data is irrecoverable
• Test key recovery procedures regularly
• Consider key escrow for disaster recovery

Performance Impact

Transparent encryption adds minimal overhead:

Operation	Overhead	Notes
Point Queries	< 2%	Negligible for most workloads
Sequential Scans	< 3%	Slightly higher for large scans
Inserts	< 2.5%	Includes encryption time
Updates	< 2.5%	Encrypt + decrypt overhead

Performance tested on: - CPU: AMD EPYC / Intel Xeon - Data size: 1GB - 100GB - AES-NI hardware acceleration enabled

Technical Details

Encryption Algorithm

• Algorithm: AES-256-GCM (Galois/Counter Mode)
• Key Size: 256 bits (32 bytes)
• Nonce: 96 bits (12 bytes), randomly generated per encryption
• Authentication Tag: 128 bits (16 bytes), provides integrity verification

Data Format

Encrypted data structure:

[Nonce (12 bytes)][Encrypted Data][Authentication Tag (16 bytes)]

What Gets Encrypted

1. Table Data: All tuples stored in tables
2. Catalog Metadata: Table schemas, column definitions
3. Row Counters: Internal row ID counters
4. All RocksDB Values: Everything passed through StorageEngine.put()

What Doesn't Get Encrypted

1. Keys: RocksDB keys remain unencrypted for indexing
2. WAL Markers: Write-ahead log metadata
3. System Metadata: Database version, configuration

Key Rotation (Planned for Phase 2)

Future support for seamless key rotation:

use heliosdb_lite::crypto::KeyManager;

let old_km = KeyManager::from_source(&old_key_source)?;
let new_km = KeyManager::generate_random();

// Rotate all data from old key to new key
storage.rotate_encryption_key(&old_km, &new_km)?;

Troubleshooting

Error: "Environment variable 'HELIOSDB_ENCRYPTION_KEY' not found"

Solution: Set the environment variable before starting the database:

export HELIOSDB_ENCRYPTION_KEY="your_64_char_hex_key"

Error: "Decryption failed"

Possible causes: 1. Wrong encryption key 2. Corrupted data 3. Key mismatch between encryption and decryption

Solution: Verify you're using the correct key. If data is corrupted, restore from backup.

Error: "Hex key must be 64 characters"

Solution: Ensure your hex key is exactly 64 hexadecimal characters (0-9, a-f):

# Generate correct key
openssl rand -hex 32

Error: "Failed to read key file"

Possible causes: 1. File doesn't exist 2. Insufficient permissions 3. Invalid file format

Solution:

# Check file exists
ls -l /path/to/encryption.key

# Check permissions
chmod 600 /path/to/encryption.key

# Verify file format (should be 64 hex chars or 32 bytes)
cat /path/to/encryption.key | wc -c

Migration Guide

Enabling Encryption on Existing Database

Warning: This will require re-encrypting all data. Plan for downtime.

1. Backup your data:

   pg_dump -h localhost -U heliosdb -d mydb > backup.sql
   

2. Generate encryption key:

   openssl rand -hex 32 > encryption.key
   chmod 600 encryption.key
   

3. Update configuration:

   [encryption]
   enabled = true
   key_source = { File = "/path/to/encryption.key" }
   

4. Migrate data (future utility):

   heliosdb-migrate --enable-encryption --key-file encryption.key --database ./db
   

Disabling Encryption

To disable encryption and decrypt existing data:

1. Update configuration:

   [encryption]
   enabled = false
   

2. Run migration (future utility):

   heliosdb-migrate --disable-encryption --database ./db
   

Compliance and Certifications

HeliosDB Lite's encryption implementation uses:

• NIST-approved algorithms: AES-256-GCM (FIPS 140-2)
• OWASP recommended practices: Authenticated encryption
• Industry-standard libraries: aes-gcm crate (audited)

Suitable for compliance with: - GDPR (data protection) - HIPAA (healthcare data) - PCI DSS (payment card data) - SOC 2 (security controls)

FAQ

Q: Can I use encryption in in-memory mode? A: Yes, encryption works in both persistent and in-memory modes.

Q: Is there a performance difference between file and environment keys? A: No, key source doesn't affect runtime performance. The key is loaded once at startup.

Q: Can I encrypt only specific tables? A: Not currently. Encryption is database-wide (all-or-nothing).

Q: What happens if I lose my encryption key? A: Data is permanently irrecoverable. Always backup your keys securely.

Q: Can I change the encryption algorithm? A: Currently only AES-256-GCM is supported. This is the industry-standard AEAD algorithm.

Q: Does encryption protect against SQL injection? A: No. Encryption protects data at rest. Use parameterized queries to prevent SQL injection.

Q: Is encryption enabled by default? A: No. You must explicitly enable encryption in the configuration.

Related Documentation

• Configuration Guide
• Security Best Practices
• Key Management
• Performance Tuning

Support

For encryption-related issues: - GitHub Issues: https://github.com/heliosdb/heliosdb/issues - Security issues: security@heliosdb.io - Documentation: https://docs.heliosdb.io