Skip to content

Transparent Data Encryption Tutorial

Version: 2.5.0-dev Last Updated: 2025-12-01 Estimated Time: 45 minutes Difficulty: Intermediate


Table of Contents

  1. Introduction
  2. Encryption Architecture
  3. Setting Up Encryption
  4. Key Management
  5. Working with Encrypted Data
  6. Security Best Practices
  7. Compliance Scenarios
  8. Recovery Scenarios
  9. Performance Considerations
  10. Troubleshooting

Introduction

What is Transparent Data Encryption (TDE)?

Transparent Data Encryption (TDE) is a security feature that encrypts data at rest without requiring changes to application code. With TDE enabled, HeliosDB-Lite automatically:

  • Encrypts all data before writing to disk
  • Decrypts data when reading from disk
  • Protects against unauthorized access to database files
  • Maintains full query functionality with zero code changes

Why Use Encryption?

Security Benefits: - Protects sensitive data from unauthorized file access - Prevents data theft if storage media is stolen - Ensures data privacy during backups - Adds defense-in-depth security layer

Compliance Requirements: - HIPAA: Required for healthcare PHI (Protected Health Information) - GDPR: Recommended for EU personal data protection - PCI-DSS: Mandatory for payment card data - SOC 2: Required for Type II certification - State Laws: California CCPA, New York SHIELD Act, etc.

How HeliosDB-Lite Implements Encryption

HeliosDB-Lite uses industry-standard AES-256-GCM (Advanced Encryption Standard with Galois/Counter Mode):

  • Algorithm: AES-256 (NIST FIPS 140-2 approved)
  • Mode: GCM (provides both encryption and authentication)
  • Key Size: 256 bits (32 bytes)
  • Authentication: Built-in integrity verification
  • Performance: Hardware-accelerated on modern CPUs (AES-NI)

What Gets Encrypted: - ✅ All table data (tuples) - ✅ Catalog metadata (schemas, column definitions) - ✅ Index data - ✅ Row counters and internal state

What Doesn't Get Encrypted: - ❌ Database keys (for indexing efficiency) - ❌ Configuration files - ❌ WAL metadata structure


Encryption Architecture

AES-256-GCM Algorithm

Advanced Encryption Standard (AES-256): - Block Cipher: Operates on 128-bit blocks - Key Size: 256 bits (highest security level) - Rounds: 14 encryption rounds - NIST Approved: FIPS 140-2 compliant

Galois/Counter Mode (GCM): - Authenticated Encryption: Combines encryption + authentication - Authentication Tag: 128-bit tag prevents tampering - Counter Mode: Turns block cipher into stream cipher - Parallel Processing: Can utilize multiple CPU cores

Key Derivation with Argon2

When using password-based encryption, HeliosDB-Lite uses Argon2:

  • Algorithm: Argon2id (winner of Password Hashing Competition)
  • Purpose: Derive 256-bit keys from passwords
  • Protection: Resistant to GPU/ASIC attacks
  • Configurable: Memory-hard and CPU-intensive parameters

Argon2 Parameters:

// Default parameters (balanced security/performance)
let argon2 = Argon2::default();  // ~19ms on modern CPU
// - Memory cost: 19 MiB
// - Time cost: 2 iterations
// - Parallelism: 1 thread

Encryption Flow Diagram

┌─────────────────────────────────────────────────────────┐
│                   Application Layer                      │
│  put(key: "users:1", value: "John Doe")                 │
└────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│                  Storage Engine                          │
│  if encryption_enabled:                                  │
│    encrypted = encrypt(value, encryption_key)            │
└────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│               AES-256-GCM Encryption                     │
│  1. Generate random 12-byte nonce                        │
│  2. Encrypt plaintext → ciphertext                       │
│  3. Generate 16-byte authentication tag                  │
│  4. Return: [nonce|ciphertext|tag]                       │
└────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│                    RocksDB                               │
│  Store encrypted blob to disk                            │
└─────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│                  Disk Storage                            │
│  Binary data (unintelligible without key)                │
└─────────────────────────────────────────────────────────┘

Data Format on Disk

Each encrypted value follows this format:

┌──────────────┬──────────────────┬─────────────────┐
│   Nonce      │   Ciphertext     │   Auth Tag      │
│  (12 bytes)  │   (variable)     │   (16 bytes)    │
└──────────────┴──────────────────┴─────────────────┘

Example:

Plaintext: "Hello, World!" (13 bytes)
Encrypted: [12-byte nonce][13-byte ciphertext][16-byte tag] = 41 bytes total
Overhead: 28 bytes (12 + 16)


Setting Up Encryption

Prerequisites

Before enabling encryption, ensure you have:

  1. ✅ HeliosDB-Lite 2.5.0 or later
  2. ✅ Secure method to store encryption keys
  3. ✅ Backup strategy for keys
  4. ✅ Understanding of key management policies

Method 1: Environment Variable Key (Development)

Step 1: Generate an Encryption Key

# Using HeliosDB-Lite CLI
heliosdb-lite keygen

# Or using OpenSSL
openssl rand -hex 32
# Example output: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

Step 2: Set Environment Variable

# Linux/macOS
export HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"

# Windows PowerShell
$env:HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"

# Windows CMD
set HELIOSDB_ENCRYPTION_KEY=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

Step 3: Configure Encrypted Database (Rust)

use heliosdb_lite::{Config, storage::StorageEngine};

fn main() -> heliosdb_lite::Result<()> {
    // Create configuration with encryption enabled
    let mut config = Config::default();
    config.storage.path = Some("./encrypted_db".into());

    // Enable encryption
    config.encryption.enabled = true;
    config.encryption.key_source =
        heliosdb_lite::config::KeySource::Environment(
            "HELIOSDB_ENCRYPTION_KEY".to_string()
        );

    // Open encrypted database
    let storage = StorageEngine::open("./encrypted_db", &config)?;

    println!("✓ Encrypted database opened successfully");
    println!("✓ Encryption: {}",
        storage.encryption_info().unwrap_or("Disabled".to_string()));

    Ok(())
}

Step 4: Verify Encryption

// Insert some data
storage.put(b"test_key", b"sensitive_data")?;

// Verify it's encrypted on disk
let raw_data = storage.db.get(b"test_key")?.unwrap();
assert_ne!(raw_data, b"sensitive_data"); // Should be encrypted

// But transparent decryption works
let decrypted = storage.get(b"test_key")?.unwrap();
assert_eq!(decrypted, b"sensitive_data"); // ✓ Correct plaintext

Method 2: File-Based Key (Production)

Step 1: Generate and Store Key File

# Generate key and save to file
heliosdb-lite keygen > /secure/path/encryption.key

# Verify key file (should be 64 hex characters)
cat /secure/path/encryption.key
# Output: 0123456789abcdef...

# Set strict file permissions
chmod 600 /secure/path/encryption.key
chown dbuser:dbgroup /secure/path/encryption.key

Step 2: Configure with File-Based Key

use heliosdb_lite::{Config, storage::StorageEngine};

fn main() -> heliosdb_lite::Result<()> {
    let mut config = Config::default();
    config.storage.path = Some("./production_db".into());

    // Use file-based key
    config.encryption.enabled = true;
    config.encryption.key_source =
        heliosdb_lite::config::KeySource::File(
            "/secure/path/encryption.key".into()
        );

    let storage = StorageEngine::open("./production_db", &config)?;
    println!("✓ Production database encrypted with file-based key");

    Ok(())
}

Step 3: Configuration File (TOML)

# config.toml
[storage]
path = "/var/lib/heliosdb/data"

[encryption]
enabled = true
algorithm = "Aes256Gcm"

[encryption.key_source]
File = "/etc/heliosdb/encryption.key"

# Optional: Key rotation settings (future feature)
rotation_interval_days = 90

Load configuration:

let config = Config::from_file("config.toml")?;
let storage = StorageEngine::open("/var/lib/heliosdb/data", &config)?;

Method 3: Password-Based Key (Interactive)

Use Case: User-specific databases, encrypted backups

use heliosdb_lite::crypto::KeyManager;

fn setup_password_based_encryption() -> heliosdb_lite::Result<()> {
    // Get password from user (use secure input in production)
    let password = rpassword::prompt_password("Enter encryption password: ")?;

    // Generate or load salt (must be stored for key derivation)
    let salt = load_or_generate_salt()?; // 16+ bytes

    // Derive key from password
    let key_manager = KeyManager::from_password(&password, &salt)?;

    println!("✓ Key derived from password using Argon2");
    println!("✓ Salt: {} bytes", salt.len());

    // Use the derived key
    // Note: You'll need to pass this to StorageEngine manually
    // or store it temporarily in an environment variable

    Ok(())
}

fn load_or_generate_salt() -> heliosdb_lite::Result<Vec<u8>> {
    let salt_path = "./.db_salt";

    if std::path::Path::new(salt_path).exists() {
        // Load existing salt
        Ok(std::fs::read(salt_path)?)
    } else {
        // Generate new salt (16 bytes minimum)
        let salt: [u8; 16] = rand::random();
        std::fs::write(salt_path, &salt)?;
        println!("✓ Generated new salt: {}", salt_path);
        Ok(salt.to_vec())
    }
}

Method 4: KMS Integration (Enterprise - Planned)

Future Support (Phase 2):

// AWS KMS example (planned)
config.encryption.key_source = heliosdb_lite::config::KeySource::Kms {
    provider: "aws".to_string(),
    key_id: "alias/heliosdb-production-key".to_string(),
};

// Azure Key Vault example (planned)
config.encryption.key_source = heliosdb_lite::config::KeySource::Kms {
    provider: "azure".to_string(),
    key_id: "https://myvault.vault.azure.net/keys/heliosdb-key".to_string(),
};

// Google Cloud KMS example (planned)
config.encryption.key_source = heliosdb_lite::config::KeySource::Kms {
    provider: "gcp".to_string(),
    key_id: "projects/my-project/locations/us/keyRings/my-ring/cryptoKeys/heliosdb".to_string(),
};

Key Management

Key Storage Options Comparison

Method Security Ease of Use Production Ready Use Case
Environment Variable ⚠️ Medium ✅ Easy ⚠️ Dev/Test Only Local development
File-Based ✅ Good ✅ Easy ✅ Yes (with permissions) Single-server deployments
Password-Derived ⚠️ Medium ⚡ Interactive ⚠️ Limited User-specific databases
KMS/HSM ✅ Excellent ⚡ Complex ✅ Yes Enterprise production

Key Generation Best Practices

Generate Cryptographically Strong Keys:

use heliosdb_lite::crypto::KeyManager;

// Method 1: Using KeyManager
let km = KeyManager::generate_random();
let hex_key = km.export_as_hex();
println!("Generated key: {}", hex_key);

// Method 2: Using OpenSSL (CLI)
// openssl rand -hex 32

// Method 3: Using /dev/urandom (Linux)
// dd if=/dev/urandom bs=32 count=1 2>/dev/null | xxd -p -c 32

Key Properties: - ✅ 256 bits (32 bytes) of random data - ✅ Generated from cryptographically secure source (rand::random()) - ✅ Encoded as 64 hexadecimal characters - ❌ Never use predictable values (timestamps, counters, etc.) - ❌ Never reuse keys across databases

Key Backup and Storage

Backup Strategy:

# 1. Backup key to secure location
cp /etc/heliosdb/encryption.key /secure/backup/encryption.key.backup

# 2. Store in password manager (recommended for small teams)
# Example: 1Password, LastPass, Bitwarden

# 3. Use key escrow service (enterprise)
# Store encrypted key with recovery mechanism

# 4. Hardware Security Module (HSM)
# For highest security requirements

Storage Best Practices:

# Set minimal permissions (owner read-only)
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key

# Verify permissions
ls -la /etc/heliosdb/encryption.key
# Expected: -r-------- 1 heliosdb heliosdb 64 Dec 01 10:00 encryption.key

Environment Variable Security:

# For systemd services
[Service]
Environment="HELIOSDB_ENCRYPTION_KEY=<key>"
EnvironmentFile=/etc/heliosdb/secrets.env  # Preferred

# For Docker
docker run -e HELIOSDB_ENCRYPTION_KEY=<key> heliosdb-lite
# Or use Docker secrets (recommended)
docker secret create heliosdb_key encryption.key

# For Kubernetes
kubectl create secret generic heliosdb-encryption \
  --from-file=key=/path/to/encryption.key

Key Rotation Procedure

Current Limitation: Automatic key rotation is planned for Phase 2. Manual rotation requires:

Manual Rotation Steps:

# 1. Backup database
heliosdb-lite backup --input /var/lib/heliosdb --output /backup/db.tar.gz

# 2. Generate new key
NEW_KEY=$(heliosdb-lite keygen)
echo $NEW_KEY > /tmp/new_encryption.key

# 3. Export data (decrypted)
heliosdb-lite export --input /var/lib/heliosdb --output /tmp/export.sql

# 4. Create new database with new key
export HELIOSDB_ENCRYPTION_KEY=$NEW_KEY
heliosdb-lite init --path /var/lib/heliosdb-new

# 5. Import data (encrypted with new key)
heliosdb-lite import --input /tmp/export.sql --output /var/lib/heliosdb-new

# 6. Verify and switch
mv /var/lib/heliosdb /var/lib/heliosdb-old
mv /var/lib/heliosdb-new /var/lib/heliosdb

# 7. Update key in secure storage
cp /tmp/new_encryption.key /etc/heliosdb/encryption.key
chmod 400 /etc/heliosdb/encryption.key

Planned Automatic Rotation (Phase 2):

// Future API
storage.rotate_encryption_key(&new_key_manager)?;
// - Online rotation (no downtime)
// - Gradual re-encryption in background
// - Dual-key support during transition

Key Recovery

Lost Key Scenarios:

Scenario Recovery Possible? Solution
Key file deleted ✅ Yes Restore from backup
Backup exists ✅ Yes Use backup key
No backup NO Data is permanently lost
Wrong key used ❌ No Must use correct original key
Corrupted key ❌ No Need exact original key

Recovery Procedure:

# 1. Locate backup key
ls -la /backup/keys/

# 2. Verify key is correct length (64 hex chars)
wc -c /backup/keys/encryption.key.backup
# Expected: 64

# 3. Test key with small database copy
cp -r /var/lib/heliosdb /tmp/test_db
export HELIOSDB_ENCRYPTION_KEY=$(cat /backup/keys/encryption.key.backup)
heliosdb-lite verify --path /tmp/test_db

# 4. If successful, restore to production
cp /backup/keys/encryption.key.backup /etc/heliosdb/encryption.key
systemctl restart heliosdb

Working with Encrypted Data

Transparent Operations (No Code Changes)

Good News: Encryption is completely transparent! Your application code doesn't change.

Without Encryption:

// Regular database operations
let storage = StorageEngine::open("./db", &config)?;
storage.put(b"key1", b"value1")?;
let value = storage.get(b"key1")?.unwrap();

With Encryption:

// Exact same code! Encryption is automatic
let storage = StorageEngine::open("./db", &config)?;  // Opens encrypted DB
storage.put(b"key1", b"value1")?;  // Automatically encrypted
let value = storage.get(b"key1")?.unwrap();  // Automatically decrypted

Table Operations

Create Encrypted Tables:

use heliosdb_lite::{Schema, Column, DataType, Tuple, Value};

// Schema definition (same as without encryption)
let schema = Schema::new(vec![
    Column::new("id", DataType::Int4).primary_key(),
    Column::new("ssn", DataType::Text),
    Column::new("credit_card", DataType::Text),
    Column::new("salary", DataType::Float8),
]);

// Create table (metadata is encrypted)
let catalog = storage.catalog();
catalog.create_table("employees", schema)?;

// Insert data (automatically encrypted)
storage.insert_tuple("employees", Tuple::new(vec![
    Value::Int4(1),
    Value::String("123-45-6789".to_string()),
    Value::String("4111-1111-1111-1111".to_string()),
    Value::Float8(125000.0),
]))?;

// Query data (automatically decrypted)
let employees = storage.scan_table("employees")?;
for emp in employees {
    println!("Employee: {:?}", emp);
    // All values are automatically decrypted
}

SQL Queries (Server Mode)

PostgreSQL Protocol (encryption is transparent):

-- Connect to encrypted database
psql -h localhost -p 5432 -d encrypted_db

-- All queries work normally
CREATE TABLE patients (
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    diagnosis TEXT,
    prescription TEXT
);

-- Insert sensitive medical data (encrypted automatically)
INSERT INTO patients (name, diagnosis, prescription)
VALUES ('John Doe', 'Type 2 Diabetes', 'Metformin 500mg');

-- Query returns decrypted data
SELECT * FROM patients WHERE id = 1;

-- Joins, aggregations, etc. all work
SELECT diagnosis, COUNT(*)
FROM patients
GROUP BY diagnosis;

Performance Considerations

Encryption Overhead:

Based on benchmarks:

Operation Unencrypted Encrypted Overhead
Small writes (64B) 1.2 µs 1.23 µs +2.5%
Medium writes (1KB) 1.5 µs 1.54 µs +2.7%
Large writes (16KB) 3.8 µs 3.91 µs +2.9%
Point reads 0.8 µs 0.82 µs +2.5%
Table scans (1000 rows) 850 µs 875 µs +2.9%
Indexed lookups 1.1 µs 1.13 µs +2.7%

Performance Tips:

  1. Use Hardware Acceleration:

    # Verify CPU supports AES-NI
    lscpu | grep aes
    # Or on macOS:
    sysctl -a | grep aes
    
    # If available, encryption overhead is minimal (<3%)
    

  2. Batch Operations:

    // Less efficient: Many small writes
    for i in 0..1000 {
        storage.put(&format!("key_{}", i).as_bytes(), b"value")?;
    }
    
    // More efficient: Batch inserts
    let tuples: Vec<Tuple> = (0..1000)
        .map(|i| Tuple::new(vec![Value::Int4(i), Value::String("data".into())]))
        .collect();
    storage.insert_batch("table", tuples)?;
    

  3. Memory Considerations:

    // Each encrypted value has 28-byte overhead (12-byte nonce + 16-byte tag)
    // For small values, this can be significant
    
    // Example:
    // 1-byte value → 29 bytes encrypted (29x overhead)
    // 100-byte value → 128 bytes encrypted (1.28x overhead)
    // 1KB value → 1052 bytes encrypted (1.05x overhead)
    
    // Recommendation: Use larger values when possible
    

Debugging Encrypted Databases

Check Encryption Status:

let storage = StorageEngine::open("./db", &config)?;

// Method 1: Check encryption flag
if storage.is_encrypted() {
    println!("✓ Database is encrypted");
} else {
    println!("⚠ Database is NOT encrypted");
}

// Method 2: Get encryption info
if let Some(info) = storage.encryption_info() {
    println!("Encryption: {}", info);
    // Output: "Enabled: AES-256-GCM"
} else {
    println!("Encryption: Disabled");
}

Verify Data is Encrypted on Disk:

// Insert test data
storage.put(b"test_key", b"plaintext_value")?;

// Check raw data in RocksDB
let raw_data = storage.db.get(b"test_key")?.unwrap();

// Raw data should NOT match plaintext
assert_ne!(raw_data, b"plaintext_value", "Data is not encrypted!");

// But storage.get() should decrypt it
let decrypted = storage.get(b"test_key")?.unwrap();
assert_eq!(decrypted, b"plaintext_value", "Decryption failed!");

println!("✓ Data is encrypted on disk");
println!("✓ Transparent decryption works");

Logging and Monitoring:

// Enable encryption audit logging (future feature)
config.audit.log_encryption_events = true;

// Logged events:
// - Key loaded from source
// - Encryption enabled/disabled
// - Encryption errors
// - Key rotation (future)

Security Best Practices

Key Management Security

DO:

Use strong random keys:

# Good: Cryptographically secure random
openssl rand -hex 32

# Bad: Predictable or weak
echo "my_password_123" | md5sum  # ❌ DON'T DO THIS

Separate keys from data:

# Good: Key on different volume/server
/etc/heliosdb/encryption.key  # Different volume from /var/lib/heliosdb

# Bad: Key in same directory as database
/var/lib/heliosdb/encryption.key  # ❌ Same volume

Use proper file permissions:

# Good: Minimal permissions
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key

# Bad: World-readable
chmod 644 /etc/heliosdb/encryption.key  # ❌ Others can read

Implement key rotation:

# Rotate keys every 90 days (recommended)
# Rotate immediately after:
# - Security incident
# - Employee departure
# - Suspected compromise

Backup keys securely:

# Good: Encrypted backup in separate location
gpg --encrypt --recipient admin@company.com encryption.key
aws s3 cp encryption.key.gpg s3://secure-backups/keys/

# Good: Hardware security module (HSM)
# Store master key in HSM, use derived keys for databases

DON'T:

Never commit keys to version control:

# Add to .gitignore
echo "*.key" >> .gitignore
echo ".env" >> .gitignore
echo "config/secrets.toml" >> .gitignore

Never log or print keys:

// Bad: Logs the key
println!("Key: {}", key_manager.export_as_hex());  // ❌ DON'T

// Good: Log only metadata
println!("Key loaded from: {}", key_manager.source());  // ✅ OK

Never use hardcoded keys:

// Bad: Hardcoded key in source code
const ENCRYPTION_KEY: &str = "0123...";  // ❌ NEVER DO THIS

// Good: Load from secure source
let key = KeyManager::from_source(&config.encryption.key_source)?;  // ✅

Never share keys between environments:

# Bad: Same key for dev, staging, prod
# HELIOSDB_ENCRYPTION_KEY=same_key_everywhere  # ❌

# Good: Different key per environment
# DEV:     Different key for development
# STAGING: Different key for staging
# PROD:    Different key for production

Access Control

Principle of Least Privilege:

# Database process user
useradd -r -s /bin/false heliosdb

# Key file permissions
chown root:heliosdb /etc/heliosdb/encryption.key
chmod 440 /etc/heliosdb/encryption.key

# Database files
chown heliosdb:heliosdb /var/lib/heliosdb
chmod 700 /var/lib/heliosdb

# Only heliosdb user can access database and keys

Network Security (Server Mode):

# config.toml
[server]
host = "127.0.0.1"  # Localhost only (use reverse proxy for external)
port = 5432
ssl_enabled = true   # Always use SSL/TLS for network traffic

[server.ssl]
cert_file = "/etc/heliosdb/server.crt"
key_file = "/etc/heliosdb/server.key"
ca_file = "/etc/heliosdb/ca.crt"

Integration with Audit Logging

Enable Comprehensive Auditing:

use heliosdb_lite::Config;

let mut config = Config::default();

// Enable encryption
config.encryption.enabled = true;
config.encryption.key_source = /* ... */;

// Enable audit logging
config.audit.enabled = true;
config.audit.log_file = Some("/var/log/heliosdb/audit.log".into());
config.audit.log_format = heliosdb_lite::audit::LogFormat::Json;

// Audit all data access
config.audit.log_reads = true;
config.audit.log_writes = true;
config.audit.log_deletes = true;

// Combined: Encrypted data + full audit trail
let storage = StorageEngine::open("./db", &config)?;

Audit Log Example (JSON format):

{
  "timestamp": "2025-12-01T10:30:45Z",
  "event_type": "read",
  "table": "patients",
  "user": "doctor_smith",
  "query": "SELECT * FROM patients WHERE id = 123",
  "rows_affected": 1,
  "encryption_enabled": true,
  "source_ip": "192.168.1.100"
}

Compliance Reporting:

# Generate encryption compliance report
heliosdb-lite audit-report \
  --input /var/log/heliosdb/audit.log \
  --type encryption-compliance \
  --output /reports/encryption_compliance_2025_12.pdf

Compliance Scenarios

HIPAA Compliance Setup

Health Insurance Portability and Accountability Act requirements:

Required Controls: 1. ✅ Encryption at rest (covered by TDE) 2. ✅ Encryption in transit (use SSL/TLS) 3. ✅ Access controls (OS-level permissions) 4. ✅ Audit logging (enable audit logs) 5. ✅ Key management (secure key storage)

Configuration:

# hipaa_config.toml

[encryption]
enabled = true
algorithm = "Aes256Gcm"

[encryption.key_source]
File = "/secure/hipaa/encryption.key"

[server]
ssl_enabled = true

[server.ssl]
cert_file = "/etc/heliosdb/hipaa_server.crt"
key_file = "/etc/heliosdb/hipaa_server.key"
min_tls_version = "1.2"  # HIPAA requires TLS 1.2+

[audit]
enabled = true
log_file = "/var/log/heliosdb/hipaa_audit.log"
log_format = "Json"
log_reads = true
log_writes = true
log_deletes = true
log_schema_changes = true

[audit.retention]
days = 2555  # 7 years (HIPAA requirement)

HIPAA-Compliant Rust Code:

use heliosdb_lite::{Config, Schema, Column, DataType, Tuple, Value};

fn setup_hipaa_database() -> heliosdb_lite::Result<()> {
    // Load HIPAA configuration
    let config = Config::from_file("hipaa_config.toml")?;

    // Verify encryption is enabled
    assert!(config.encryption.enabled, "HIPAA requires encryption!");
    assert!(config.audit.enabled, "HIPAA requires audit logging!");

    // Open database
    let storage = StorageEngine::open("/var/lib/heliosdb/hipaa", &config)?;

    // Create PHI table (Protected Health Information)
    let catalog = storage.catalog();
    let schema = Schema::new(vec![
        Column::new("patient_id", DataType::Int4).primary_key(),
        Column::new("name", DataType::Text),
        Column::new("ssn", DataType::Text),
        Column::new("diagnosis", DataType::Text),
        Column::new("medications", DataType::Text),
        Column::new("created_at", DataType::Timestamp),
    ]);

    catalog.create_table("patient_records", schema)?;

    println!("✓ HIPAA-compliant database configured");
    println!("  - Encryption: AES-256-GCM");
    println!("  - SSL/TLS: Enabled");
    println!("  - Audit logging: 7-year retention");

    Ok(())
}

Verification Checklist:

# 1. Verify encryption is enabled
heliosdb-lite verify --config hipaa_config.toml --check encryption

# 2. Verify SSL/TLS certificates
openssl s_client -connect localhost:5432 -showcerts

# 3. Verify audit logging
tail -f /var/log/heliosdb/hipaa_audit.log

# 4. Verify file permissions
ls -la /var/lib/heliosdb/hipaa
# Expected: drwx------ (700)

# 5. Verify key security
ls -la /secure/hipaa/encryption.key
# Expected: -r-------- (400)

GDPR Data Protection

General Data Protection Regulation requirements:

GDPR Principles: 1. Data Protection by Design: Encryption enabled from day one 2. Right to Erasure: Secure deletion of personal data 3. Data Portability: Export encrypted backups 4. Breach Notification: Audit logs for incident response 5. Privacy by Default: Minimal data collection

GDPR Configuration:

# gdpr_config.toml

[encryption]
enabled = true
algorithm = "Aes256Gcm"
key_source = { File = "/etc/heliosdb/gdpr_key" }

[audit]
enabled = true
log_file = "/var/log/heliosdb/gdpr_audit.log"
log_format = "Json"

# Log personal data access (GDPR Article 15)
log_reads = true
log_writes = true
log_deletes = true

# Retention policy (GDPR Article 17)
[audit.retention]
days = 730  # 2 years minimum

[storage]
# Enable secure deletion (GDPR "Right to Erasure")
secure_delete = true
overwrite_deleted_data = true

GDPR-Compliant Data Operations:

// Right to Access (GDPR Article 15)
fn export_user_data(storage: &StorageEngine, user_id: i32)
    -> heliosdb_lite::Result<String>
{
    // Query all user data (automatically decrypted)
    let query = format!("SELECT * FROM users WHERE id = {}", user_id);
    let results = storage.execute_sql(&query)?;

    // Export as JSON (for data portability)
    let json = serde_json::to_string_pretty(&results)?;

    // Log access (for audit trail)
    log::info!("User data exported for user_id={}", user_id);

    Ok(json)
}

// Right to Erasure (GDPR Article 17)
fn delete_user_data(storage: &StorageEngine, user_id: i32)
    -> heliosdb_lite::Result<()>
{
    // Delete user from all tables
    storage.execute_sql(&format!("DELETE FROM users WHERE id = {}", user_id))?;
    storage.execute_sql(&format!("DELETE FROM orders WHERE user_id = {}", user_id))?;
    storage.execute_sql(&format!("DELETE FROM preferences WHERE user_id = {}", user_id))?;

    // Secure deletion (overwrite on disk)
    // Enabled via config.storage.secure_delete = true

    // Log deletion (compliance requirement)
    log::warn!("User data deleted for user_id={} (GDPR erasure)", user_id);

    Ok(())
}

// Data Breach Notification (GDPR Article 33)
fn check_encryption_breach(storage: &StorageEngine) -> heliosdb_lite::Result<()> {
    if !storage.is_encrypted() {
        log::error!("CRITICAL: Database is not encrypted! GDPR violation!");
        // Trigger incident response
        notify_data_protection_officer("Unencrypted database detected")?;
    }
    Ok(())
}

PCI-DSS Requirements

Payment Card Industry Data Security Standard:

PCI-DSS Requirements for Encryption: - Requirement 3: Protect stored cardholder data - Requirement 4: Encrypt transmission of cardholder data - Requirement 8: Identify and authenticate access

PCI-DSS Configuration:

# pci_config.toml

[encryption]
enabled = true  # Requirement 3.4: Render PAN unreadable
algorithm = "Aes256Gcm"  # Strong cryptography (Requirement 3.5)

[encryption.key_source]
File = "/secure/pci/encryption.key"

[encryption.key_rotation]
enabled = true
interval_days = 90  # Key rotation (Requirement 3.6.4)

[audit]
enabled = true  # Requirement 10: Track and monitor access
log_file = "/var/log/heliosdb/pci_audit.log"

# Log all access to cardholder data
log_reads = true
log_writes = true
log_deletes = true
log_schema_changes = true

[audit.retention]
days = 365  # Minimum 1 year (Requirement 10.7)

[server.ssl]
enabled = true  # Requirement 4: Encrypt in transit
min_tls_version = "1.2"

PCI-DSS Compliant Schema:

use heliosdb_lite::{Schema, Column, DataType};

fn create_pci_compliant_schema() -> Schema {
    Schema::new(vec![
        Column::new("transaction_id", DataType::Int8).primary_key(),

        // PAN (Primary Account Number) - MUST be encrypted
        Column::new("card_number", DataType::Text),

        // Cardholder name
        Column::new("cardholder_name", DataType::Text),

        // Expiration date - Can be stored with encryption
        Column::new("expiry_month", DataType::Int2),
        Column::new("expiry_year", DataType::Int4),

        // NEVER store these (PCI-DSS Requirement 3.2):
        // - CVV/CVV2 (Card Verification Value)
        // - Full magnetic stripe data
        // - PIN or PIN block

        Column::new("amount", DataType::Numeric),
        Column::new("timestamp", DataType::Timestamp),
    ])
}

// Example: Storing payment card data (encrypted)
fn process_payment(storage: &StorageEngine, card_data: CardData)
    -> heliosdb_lite::Result<()>
{
    // Data is automatically encrypted by TDE
    storage.insert_tuple("transactions", Tuple::new(vec![
        Value::Int8(generate_transaction_id()),
        Value::String(card_data.pan),  // Encrypted at rest
        Value::String(card_data.name), // Encrypted at rest
        Value::Int2(card_data.exp_month),
        Value::Int4(card_data.exp_year),
        Value::Numeric(card_data.amount),
        Value::Timestamp(Utc::now()),
    ]))?;

    // Audit log automatically records access

    Ok(())
}

PCI-DSS Compliance Verification:

# 1. Verify encryption (Requirement 3.4)
heliosdb-lite verify --config pci_config.toml --check encryption
# ✓ AES-256-GCM enabled

# 2. Verify key management (Requirement 3.5, 3.6)
ls -la /secure/pci/encryption.key
# ✓ Permissions: 400 (read-only by owner)

# 3. Verify audit logging (Requirement 10)
tail /var/log/heliosdb/pci_audit.log
# ✓ All access logged with timestamps

# 4. Verify SSL/TLS (Requirement 4)
openssl s_client -connect localhost:5432 | grep Protocol
# ✓ TLSv1.2 or higher

# 5. Check for prohibited data (Requirement 3.2)
grep -i "cvv\|cvv2\|cvc" /var/lib/heliosdb/pci/
# ✓ No results (good - not storing CVV)

Recovery Scenarios

Lost Key Recovery

Scenario: Encryption key file was accidentally deleted.

If you have a backup:

# 1. Stop database
systemctl stop heliosdb

# 2. Restore key from backup
cp /backup/keys/encryption.key.20251201 /etc/heliosdb/encryption.key

# 3. Set correct permissions
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key

# 4. Verify key works
heliosdb-lite verify --config /etc/heliosdb/config.toml --check encryption

# 5. Restart database
systemctl start heliosdb
systemctl status heliosdb

If you DON'T have a backup:

⚠️ CRITICAL: Without the encryption key, your data is PERMANENTLY LOST.

There is NO way to recover encrypted data without the original key.

Prevention:
- Always maintain secure backups of encryption keys
- Store backups in multiple locations
- Use key management services (KMS/HSM)
- Document key recovery procedures
- Test recovery procedures regularly

Database Migration with Encryption

Scenario: Migrate encrypted database to new server.

Step-by-Step Migration:

# On old server:

# 1. Backup encrypted database
tar -czf database_backup.tar.gz /var/lib/heliosdb/

# 2. Backup encryption key SEPARATELY
tar -czf encryption_key.tar.gz /etc/heliosdb/encryption.key

# 3. Transfer to new server (use secure channel)
scp database_backup.tar.gz user@newserver:/tmp/
scp encryption_key.tar.gz user@newserver:/tmp/

# On new server:

# 4. Extract database
mkdir -p /var/lib/heliosdb
tar -xzf /tmp/database_backup.tar.gz -C /

# 5. Extract encryption key
mkdir -p /etc/heliosdb
tar -xzf /tmp/encryption_key.tar.gz -C /

# 6. Set permissions
chmod 700 /var/lib/heliosdb
chown heliosdb:heliosdb /var/lib/heliosdb
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key

# 7. Verify database opens correctly
heliosdb-lite verify --path /var/lib/heliosdb

# 8. Start database
systemctl start heliosdb

# 9. Test connectivity
psql -h localhost -p 5432 -c "SELECT 1"

Alternative: Export/Import Method:

# On old server:
# 1. Export data in decrypted format
heliosdb-lite export --input /var/lib/heliosdb --output /tmp/export.sql

# 2. Transfer export file
scp /tmp/export.sql user@newserver:/tmp/

# On new server:
# 3. Generate NEW encryption key for new server
heliosdb-lite keygen > /etc/heliosdb/encryption.key
chmod 400 /etc/heliosdb/encryption.key

# 4. Initialize encrypted database with NEW key
export HELIOSDB_ENCRYPTION_KEY=$(cat /etc/heliosdb/encryption.key)
heliosdb-lite init --path /var/lib/heliosdb --encrypted

# 5. Import data (will be encrypted with NEW key)
heliosdb-lite import --input /tmp/export.sql --output /var/lib/heliosdb

# Result: Data re-encrypted with new key on new server

Backup and Restore

Full Encrypted Backup:

# Backup strategy: Backup encrypted database files directly
# Advantage: Fast, no decryption needed
# Disadvantage: Requires same encryption key for restore

# 1. Consistent snapshot (stop writes)
heliosdb-lite checkpoint --path /var/lib/heliosdb

# 2. Backup database files (encrypted)
tar -czf /backup/heliosdb_encrypted_20251201.tar.gz /var/lib/heliosdb/

# 3. Backup encryption key SEPARATELY and SECURELY
gpg --encrypt --recipient backup@company.com /etc/heliosdb/encryption.key
cp encryption.key.gpg /secure/backup/keys/

# 4. Verify backup
tar -tzf /backup/heliosdb_encrypted_20251201.tar.gz | head

# 5. Test restore to temporary location
mkdir /tmp/restore_test
tar -xzf /backup/heliosdb_encrypted_20251201.tar.gz -C /tmp/restore_test
heliosdb-lite verify --path /tmp/restore_test/var/lib/heliosdb

Decrypted Logical Backup:

# Backup strategy: Export to SQL (decrypted)
# Advantage: Key-independent, can restore with different key
# Disadvantage: Slower, decrypted data must be secured

# 1. Export to SQL (automatically decrypts during export)
heliosdb-lite export \
  --input /var/lib/heliosdb \
  --output /backup/heliosdb_export_20251201.sql

# 2. Encrypt the SQL export for security
gpg --encrypt --recipient backup@company.com \
  /backup/heliosdb_export_20251201.sql

# 3. Store encrypted SQL file
mv heliosdb_export_20251201.sql.gpg /secure/backup/exports/

# 4. Restore procedure
gpg --decrypt /secure/backup/exports/heliosdb_export_20251201.sql.gpg \
  > /tmp/restore.sql

heliosdb-lite import \
  --input /tmp/restore.sql \
  --output /var/lib/heliosdb_restored

# 5. Secure cleanup
shred -u /tmp/restore.sql  # Securely delete decrypted SQL

Automated Backup Script:

#!/bin/bash
# backup_encrypted_db.sh

set -e

DB_PATH="/var/lib/heliosdb"
BACKUP_DIR="/backup/heliosdb"
KEY_FILE="/etc/heliosdb/encryption.key"
DATE=$(date +%Y%m%d_%H%M%S)

# 1. Create consistent snapshot
heliosdb-lite checkpoint --path "$DB_PATH"

# 2. Backup database (encrypted)
tar -czf "$BACKUP_DIR/db_$DATE.tar.gz" "$DB_PATH"

# 3. Backup key (encrypted with GPG)
gpg --encrypt --recipient backup@company.com "$KEY_FILE" \
  -o "$BACKUP_DIR/key_$DATE.gpg"

# 4. Verify backup integrity
tar -tzf "$BACKUP_DIR/db_$DATE.tar.gz" > /dev/null

# 5. Retention: Keep last 30 days
find "$BACKUP_DIR" -name "db_*.tar.gz" -mtime +30 -delete
find "$BACKUP_DIR" -name "key_*.gpg" -mtime +30 -delete

# 6. Upload to cloud (optional)
# aws s3 cp "$BACKUP_DIR/db_$DATE.tar.gz" s3://backups/heliosdb/
# aws s3 cp "$BACKUP_DIR/key_$DATE.gpg" s3://backups/heliosdb/keys/

echo "✓ Backup completed: $DATE"

Schedule with cron:

# Run daily at 2 AM
0 2 * * * /usr/local/bin/backup_encrypted_db.sh >> /var/log/heliosdb_backup.log 2>&1

Performance Considerations

Benchmarking Encrypted vs Unencrypted

Run Built-in Benchmarks:

# Run encryption benchmarks
cargo bench --bench encryption_benchmark

# Sample output:
# Encryption/put_get/64_bytes    time: [1.23 µs 1.25 µs 1.27 µs]
# Unencrypted/put_get/64_bytes   time: [1.20 µs 1.22 µs 1.24 µs]
# Overhead: ~2.5%

Custom Performance Testing:

use std::time::Instant;
use heliosdb_lite::{Config, storage::StorageEngine};

fn benchmark_encryption_overhead() -> heliosdb_lite::Result<()> {
    let iterations = 10_000;

    // Test 1: Encrypted database
    let mut config = Config::in_memory();
    config.encryption.enabled = true;
    config.encryption.key_source = /* ... */;

    let encrypted_storage = StorageEngine::open_in_memory(&config)?;

    let start = Instant::now();
    for i in 0..iterations {
        encrypted_storage.put(
            &format!("key_{}", i).as_bytes(),
            b"test_data_payload_1234567890"
        )?;
    }
    let encrypted_duration = start.elapsed();

    // Test 2: Unencrypted database
    let mut config_unenc = Config::in_memory();
    config_unenc.encryption.enabled = false;

    let unencrypted_storage = StorageEngine::open_in_memory(&config_unenc)?;

    let start = Instant::now();
    for i in 0..iterations {
        unencrypted_storage.put(
            &format!("key_{}", i).as_bytes(),
            b"test_data_payload_1234567890"
        )?;
    }
    let unencrypted_duration = start.elapsed();

    // Calculate overhead
    let overhead = (encrypted_duration.as_micros() as f64
                   / unencrypted_duration.as_micros() as f64 - 1.0) * 100.0;

    println!("Results ({} iterations):", iterations);
    println!("  Unencrypted: {:?}", unencrypted_duration);
    println!("  Encrypted:   {:?}", encrypted_duration);
    println!("  Overhead:    {:.2}%", overhead);

    Ok(())
}

Optimization Tips

1. Use Appropriate Value Sizes:

// Inefficient: Many tiny encrypted values
// Each has 28-byte overhead (nonce + tag)
for i in 0..1000 {
    storage.put(&format!("counter_{}", i).as_bytes(), &[i as u8])?;
    // 1-byte value → 29 bytes stored (29x overhead)
}

// Efficient: Batch into larger values
let counters: Vec<u8> = (0..1000).map(|i| i as u8).collect();
storage.put(b"counters", &counters)?;
// 1000-byte value → 1028 bytes stored (1.028x overhead)

2. Minimize Metadata Operations:

// Less efficient: Frequent catalog queries (metadata is encrypted)
for table_name in table_names {
    let schema = catalog.get_table_schema(table_name)?; // Decrypts metadata
    process_schema(schema);
}

// More efficient: Cache metadata
let schemas: HashMap<String, Schema> = table_names.iter()
    .map(|name| (name.clone(), catalog.get_table_schema(name)))
    .collect()?;

for (name, schema) in schemas {
    process_schema(schema); // No decryption needed
}

3. Batch Writes:

// Use transactions or batch inserts to amortize encryption overhead
storage.begin_transaction()?;
for tuple in large_dataset {
    storage.insert_tuple("table", tuple)?;
}
storage.commit()?; // Single fsync

Hardware Acceleration

Verify AES-NI Support:

# Linux
lscpu | grep -i aes
# Look for: Flags: ... aes ...

cat /proc/cpuinfo | grep aes
# Should show "aes" flag

# macOS
sysctl -a | grep -i aes
# macOS kern.hv_support: 1 (includes AES support)

# Check with Rust
cargo build --release
# Compiler automatically uses AES-NI if available

Performance Impact of AES-NI:

CPU Without AES-NI With AES-NI Speedup
Intel Core i7 ~15 µs ~1.2 µs 12x faster
AMD Ryzen 7 ~14 µs ~1.1 µs 13x faster
ARM (modern) ~10 µs ~0.9 µs 11x faster

Note: HeliosDB-Lite's <3% overhead assumes AES-NI is available.


Troubleshooting

Common Issues

Issue: "Environment variable 'HELIOSDB_ENCRYPTION_KEY' not found"

Cause: Encryption key not set in environment.

Solution:

# Check if variable is set
echo $HELIOSDB_ENCRYPTION_KEY

# Set the variable
export HELIOSDB_ENCRYPTION_KEY="<64-character-hex-key>"

# For systemd services
sudo systemctl edit heliosdb
# Add:
# [Service]
# Environment="HELIOSDB_ENCRYPTION_KEY=<key>"

# For Docker
docker run -e HELIOSDB_ENCRYPTION_KEY="<key>" heliosdb-lite

Issue: "Decryption failed: Authentication tag verification failed"

Causes: 1. Wrong encryption key 2. Corrupted data 3. Key changed after data was encrypted

Solutions:

# 1. Verify you're using the correct key
cat /etc/heliosdb/encryption.key
# Should be 64 hex characters

# 2. Check for data corruption
heliosdb-lite verify --path /var/lib/heliosdb

# 3. Try restore from backup
cp -r /var/lib/heliosdb /var/lib/heliosdb.corrupted
tar -xzf /backup/heliosdb_latest.tar.gz -C /var/lib/

# 4. Restore correct key
cp /backup/keys/encryption.key /etc/heliosdb/encryption.key

Issue: "Hex key must be 64 characters (32 bytes), got X"

Cause: Invalid key format.

Solution:

# Generate proper key (64 hex chars)
heliosdb-lite keygen
# Or:
openssl rand -hex 32

# Verify length
echo -n "<your-key>" | wc -c
# Should output: 64

Issue: "Failed to read key file: Permission denied"

Cause: Insufficient file permissions.

Solution:

# Check permissions
ls -la /etc/heliosdb/encryption.key

# Fix ownership
sudo chown heliosdb:heliosdb /etc/heliosdb/encryption.key

# Fix permissions
sudo chmod 400 /etc/heliosdb/encryption.key

# Verify database process user
ps aux | grep heliosdb
# Ensure it's running as 'heliosdb' user

Performance Issues

Issue: "Database operations are slow after enabling encryption"

Diagnosis:

# 1. Check if AES-NI is available
lscpu | grep aes

# 2. Run benchmarks
cargo bench --bench encryption_benchmark

# 3. Profile encryption operations
cargo build --release --features profiling
perf record ./target/release/heliosdb-lite
perf report

Solutions:

// 1. Batch operations
storage.begin_transaction()?;
for item in items {
    storage.insert_tuple("table", item)?;
}
storage.commit()?;

// 2. Increase cache size
config.storage.cache_size = 1024 * 1024 * 1024; // 1 GB

// 3. Use in-memory mode for temporary data
let temp_config = Config::in_memory();
// Encryption overhead is minimal in-memory

Security Issues

Issue: "Security audit flagged encryption key in logs"

Immediate Action:

# 1. Rotate key immediately
NEW_KEY=$(heliosdb-lite keygen)
# Follow key rotation procedure

# 2. Scrub logs
sudo find /var/log -name "*.log" -exec grep -l "HELIOSDB_ENCRYPTION_KEY" {} \; \
  -exec shred -u {} \;

# 3. Update code to prevent logging
# Remove any println!() or log statements that include keys

Prevention:

// Bad: Logs key
log::debug!("Using key: {}", key_manager.export_as_hex());  // ❌

// Good: Logs only metadata
log::debug!("Key loaded from: {:?}", key_manager.source());  // ✅

Issue: "Key file has insecure permissions"

Fix:

# Set minimal permissions
sudo chmod 400 /etc/heliosdb/encryption.key
sudo chown heliosdb:heliosdb /etc/heliosdb/encryption.key

# Verify
ls -la /etc/heliosdb/encryption.key
# Expected: -r-------- 1 heliosdb heliosdb 64 Dec 01 10:00 encryption.key

# Audit permissions recursively
find /etc/heliosdb -type f -not -perm 400 -ls
# Should be empty

Getting Help

Diagnostic Information to Collect:

# System info
uname -a
lscpu | grep -i aes

# Database info
heliosdb-lite --version
heliosdb-lite verify --path /var/lib/heliosdb

# Configuration
cat /etc/heliosdb/config.toml | grep -A 10 encryption

# Logs
tail -100 /var/log/heliosdb/heliosdb.log

# Permissions
ls -laR /etc/heliosdb
ls -laR /var/lib/heliosdb

Report Issues: - GitHub Issues: https://github.com/dimensigon/HeliosDB-Lite/issues - Security Issues: security@heliosdb.io (for sensitive issues) - Community Forum: https://discuss.heliosdb.io


Summary

You've learned how to:

Enable transparent data encryption with AES-256-GCM ✅ Manage encryption keys securely using multiple methods ✅ Configure encryption for various deployment scenarios ✅ Achieve compliance with HIPAA, GDPR, and PCI-DSS ✅ Maintain performance with <3% overhead ✅ Handle recovery scenarios and migrations ✅ Troubleshoot common encryption issues

Key Takeaways:

  1. Encryption is transparent: No application code changes needed
  2. Security is paramount: Protect keys with same rigor as data
  3. Backup keys separately: Data is unrecoverable without keys
  4. Monitor performance: Encryption overhead is minimal with AES-NI
  5. Plan for compliance: Enable audit logging with encryption
  6. Test recovery procedures: Verify backups and key recovery regularly

Next Steps:


Version History: - 2.5.0-dev (2025-12-01): Initial comprehensive tutorial - 2.1.0 (2024-11-13): Encryption feature released

Related Documentation: - Encryption Integration Summary - Security Review Report - API Reference: Crypto Module