Transparent Data Encryption Tutorial¶
Version: 2.5.0-dev Last Updated: 2025-12-01 Estimated Time: 45 minutes Difficulty: Intermediate
Table of Contents¶
- Introduction
- Encryption Architecture
- Setting Up Encryption
- Key Management
- Working with Encrypted Data
- Security Best Practices
- Compliance Scenarios
- Recovery Scenarios
- Performance Considerations
- Troubleshooting
Introduction¶
What is Transparent Data Encryption (TDE)?¶
Transparent Data Encryption (TDE) is a security feature that encrypts data at rest without requiring changes to application code. With TDE enabled, HeliosDB-Lite automatically:
- Encrypts all data before writing to disk
- Decrypts data when reading from disk
- Protects against unauthorized access to database files
- Maintains full query functionality with zero code changes
Why Use Encryption?¶
Security Benefits: - Protects sensitive data from unauthorized file access - Prevents data theft if storage media is stolen - Ensures data privacy during backups - Adds defense-in-depth security layer
Compliance Requirements: - HIPAA: Required for healthcare PHI (Protected Health Information) - GDPR: Recommended for EU personal data protection - PCI-DSS: Mandatory for payment card data - SOC 2: Required for Type II certification - State Laws: California CCPA, New York SHIELD Act, etc.
How HeliosDB-Lite Implements Encryption¶
HeliosDB-Lite uses industry-standard AES-256-GCM (Advanced Encryption Standard with Galois/Counter Mode):
- Algorithm: AES-256 (NIST FIPS 140-2 approved)
- Mode: GCM (provides both encryption and authentication)
- Key Size: 256 bits (32 bytes)
- Authentication: Built-in integrity verification
- Performance: Hardware-accelerated on modern CPUs (AES-NI)
What Gets Encrypted: - ✅ All table data (tuples) - ✅ Catalog metadata (schemas, column definitions) - ✅ Index data - ✅ Row counters and internal state
What Doesn't Get Encrypted: - ❌ Database keys (for indexing efficiency) - ❌ Configuration files - ❌ WAL metadata structure
Encryption Architecture¶
AES-256-GCM Algorithm¶
Advanced Encryption Standard (AES-256): - Block Cipher: Operates on 128-bit blocks - Key Size: 256 bits (highest security level) - Rounds: 14 encryption rounds - NIST Approved: FIPS 140-2 compliant
Galois/Counter Mode (GCM): - Authenticated Encryption: Combines encryption + authentication - Authentication Tag: 128-bit tag prevents tampering - Counter Mode: Turns block cipher into stream cipher - Parallel Processing: Can utilize multiple CPU cores
Key Derivation with Argon2¶
When using password-based encryption, HeliosDB-Lite uses Argon2:
- Algorithm: Argon2id (winner of Password Hashing Competition)
- Purpose: Derive 256-bit keys from passwords
- Protection: Resistant to GPU/ASIC attacks
- Configurable: Memory-hard and CPU-intensive parameters
Argon2 Parameters:
// Default parameters (balanced security/performance)
let argon2 = Argon2::default(); // ~19ms on modern CPU
// - Memory cost: 19 MiB
// - Time cost: 2 iterations
// - Parallelism: 1 thread
Encryption Flow Diagram¶
┌─────────────────────────────────────────────────────────┐
│ Application Layer │
│ put(key: "users:1", value: "John Doe") │
└────────────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ Storage Engine │
│ if encryption_enabled: │
│ encrypted = encrypt(value, encryption_key) │
└────────────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ AES-256-GCM Encryption │
│ 1. Generate random 12-byte nonce │
│ 2. Encrypt plaintext → ciphertext │
│ 3. Generate 16-byte authentication tag │
│ 4. Return: [nonce|ciphertext|tag] │
└────────────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ RocksDB │
│ Store encrypted blob to disk │
└─────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ Disk Storage │
│ Binary data (unintelligible without key) │
└─────────────────────────────────────────────────────────┘
Data Format on Disk¶
Each encrypted value follows this format:
┌──────────────┬──────────────────┬─────────────────┐
│ Nonce │ Ciphertext │ Auth Tag │
│ (12 bytes) │ (variable) │ (16 bytes) │
└──────────────┴──────────────────┴─────────────────┘
Example:
Plaintext: "Hello, World!" (13 bytes)
Encrypted: [12-byte nonce][13-byte ciphertext][16-byte tag] = 41 bytes total
Overhead: 28 bytes (12 + 16)
Setting Up Encryption¶
Prerequisites¶
Before enabling encryption, ensure you have:
- ✅ HeliosDB-Lite 2.5.0 or later
- ✅ Secure method to store encryption keys
- ✅ Backup strategy for keys
- ✅ Understanding of key management policies
Method 1: Environment Variable Key (Development)¶
Step 1: Generate an Encryption Key
# Using HeliosDB-Lite CLI
heliosdb-lite keygen
# Or using OpenSSL
openssl rand -hex 32
# Example output: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
Step 2: Set Environment Variable
# Linux/macOS
export HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
# Windows PowerShell
$env:HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
# Windows CMD
set HELIOSDB_ENCRYPTION_KEY=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
Step 3: Configure Encrypted Database (Rust)
use heliosdb_lite::{Config, storage::StorageEngine};
fn main() -> heliosdb_lite::Result<()> {
// Create configuration with encryption enabled
let mut config = Config::default();
config.storage.path = Some("./encrypted_db".into());
// Enable encryption
config.encryption.enabled = true;
config.encryption.key_source =
heliosdb_lite::config::KeySource::Environment(
"HELIOSDB_ENCRYPTION_KEY".to_string()
);
// Open encrypted database
let storage = StorageEngine::open("./encrypted_db", &config)?;
println!("✓ Encrypted database opened successfully");
println!("✓ Encryption: {}",
storage.encryption_info().unwrap_or("Disabled".to_string()));
Ok(())
}
Step 4: Verify Encryption
// Insert some data
storage.put(b"test_key", b"sensitive_data")?;
// Verify it's encrypted on disk
let raw_data = storage.db.get(b"test_key")?.unwrap();
assert_ne!(raw_data, b"sensitive_data"); // Should be encrypted
// But transparent decryption works
let decrypted = storage.get(b"test_key")?.unwrap();
assert_eq!(decrypted, b"sensitive_data"); // ✓ Correct plaintext
Method 2: File-Based Key (Production)¶
Step 1: Generate and Store Key File
# Generate key and save to file
heliosdb-lite keygen > /secure/path/encryption.key
# Verify key file (should be 64 hex characters)
cat /secure/path/encryption.key
# Output: 0123456789abcdef...
# Set strict file permissions
chmod 600 /secure/path/encryption.key
chown dbuser:dbgroup /secure/path/encryption.key
Step 2: Configure with File-Based Key
use heliosdb_lite::{Config, storage::StorageEngine};
fn main() -> heliosdb_lite::Result<()> {
let mut config = Config::default();
config.storage.path = Some("./production_db".into());
// Use file-based key
config.encryption.enabled = true;
config.encryption.key_source =
heliosdb_lite::config::KeySource::File(
"/secure/path/encryption.key".into()
);
let storage = StorageEngine::open("./production_db", &config)?;
println!("✓ Production database encrypted with file-based key");
Ok(())
}
Step 3: Configuration File (TOML)
# config.toml
[storage]
path = "/var/lib/heliosdb/data"
[encryption]
enabled = true
algorithm = "Aes256Gcm"
[encryption.key_source]
File = "/etc/heliosdb/encryption.key"
# Optional: Key rotation settings (future feature)
rotation_interval_days = 90
Load configuration:
let config = Config::from_file("config.toml")?;
let storage = StorageEngine::open("/var/lib/heliosdb/data", &config)?;
Method 3: Password-Based Key (Interactive)¶
Use Case: User-specific databases, encrypted backups
use heliosdb_lite::crypto::KeyManager;
fn setup_password_based_encryption() -> heliosdb_lite::Result<()> {
// Get password from user (use secure input in production)
let password = rpassword::prompt_password("Enter encryption password: ")?;
// Generate or load salt (must be stored for key derivation)
let salt = load_or_generate_salt()?; // 16+ bytes
// Derive key from password
let key_manager = KeyManager::from_password(&password, &salt)?;
println!("✓ Key derived from password using Argon2");
println!("✓ Salt: {} bytes", salt.len());
// Use the derived key
// Note: You'll need to pass this to StorageEngine manually
// or store it temporarily in an environment variable
Ok(())
}
fn load_or_generate_salt() -> heliosdb_lite::Result<Vec<u8>> {
let salt_path = "./.db_salt";
if std::path::Path::new(salt_path).exists() {
// Load existing salt
Ok(std::fs::read(salt_path)?)
} else {
// Generate new salt (16 bytes minimum)
let salt: [u8; 16] = rand::random();
std::fs::write(salt_path, &salt)?;
println!("✓ Generated new salt: {}", salt_path);
Ok(salt.to_vec())
}
}
Method 4: KMS Integration (Enterprise - Planned)¶
Future Support (Phase 2):
// AWS KMS example (planned)
config.encryption.key_source = heliosdb_lite::config::KeySource::Kms {
provider: "aws".to_string(),
key_id: "alias/heliosdb-production-key".to_string(),
};
// Azure Key Vault example (planned)
config.encryption.key_source = heliosdb_lite::config::KeySource::Kms {
provider: "azure".to_string(),
key_id: "https://myvault.vault.azure.net/keys/heliosdb-key".to_string(),
};
// Google Cloud KMS example (planned)
config.encryption.key_source = heliosdb_lite::config::KeySource::Kms {
provider: "gcp".to_string(),
key_id: "projects/my-project/locations/us/keyRings/my-ring/cryptoKeys/heliosdb".to_string(),
};
Key Management¶
Key Storage Options Comparison¶
| Method | Security | Ease of Use | Production Ready | Use Case |
|---|---|---|---|---|
| Environment Variable | ⚠️ Medium | ✅ Easy | ⚠️ Dev/Test Only | Local development |
| File-Based | ✅ Good | ✅ Easy | ✅ Yes (with permissions) | Single-server deployments |
| Password-Derived | ⚠️ Medium | ⚡ Interactive | ⚠️ Limited | User-specific databases |
| KMS/HSM | ✅ Excellent | ⚡ Complex | ✅ Yes | Enterprise production |
Key Generation Best Practices¶
Generate Cryptographically Strong Keys:
use heliosdb_lite::crypto::KeyManager;
// Method 1: Using KeyManager
let km = KeyManager::generate_random();
let hex_key = km.export_as_hex();
println!("Generated key: {}", hex_key);
// Method 2: Using OpenSSL (CLI)
// openssl rand -hex 32
// Method 3: Using /dev/urandom (Linux)
// dd if=/dev/urandom bs=32 count=1 2>/dev/null | xxd -p -c 32
Key Properties:
- ✅ 256 bits (32 bytes) of random data
- ✅ Generated from cryptographically secure source (rand::random())
- ✅ Encoded as 64 hexadecimal characters
- ❌ Never use predictable values (timestamps, counters, etc.)
- ❌ Never reuse keys across databases
Key Backup and Storage¶
Backup Strategy:
# 1. Backup key to secure location
cp /etc/heliosdb/encryption.key /secure/backup/encryption.key.backup
# 2. Store in password manager (recommended for small teams)
# Example: 1Password, LastPass, Bitwarden
# 3. Use key escrow service (enterprise)
# Store encrypted key with recovery mechanism
# 4. Hardware Security Module (HSM)
# For highest security requirements
Storage Best Practices:
# Set minimal permissions (owner read-only)
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Verify permissions
ls -la /etc/heliosdb/encryption.key
# Expected: -r-------- 1 heliosdb heliosdb 64 Dec 01 10:00 encryption.key
Environment Variable Security:
# For systemd services
[Service]
Environment="HELIOSDB_ENCRYPTION_KEY=<key>"
EnvironmentFile=/etc/heliosdb/secrets.env # Preferred
# For Docker
docker run -e HELIOSDB_ENCRYPTION_KEY=<key> heliosdb-lite
# Or use Docker secrets (recommended)
docker secret create heliosdb_key encryption.key
# For Kubernetes
kubectl create secret generic heliosdb-encryption \
--from-file=key=/path/to/encryption.key
Key Rotation Procedure¶
Current Limitation: Automatic key rotation is planned for Phase 2. Manual rotation requires:
Manual Rotation Steps:
# 1. Backup database
heliosdb-lite backup --input /var/lib/heliosdb --output /backup/db.tar.gz
# 2. Generate new key
NEW_KEY=$(heliosdb-lite keygen)
echo $NEW_KEY > /tmp/new_encryption.key
# 3. Export data (decrypted)
heliosdb-lite export --input /var/lib/heliosdb --output /tmp/export.sql
# 4. Create new database with new key
export HELIOSDB_ENCRYPTION_KEY=$NEW_KEY
heliosdb-lite init --path /var/lib/heliosdb-new
# 5. Import data (encrypted with new key)
heliosdb-lite import --input /tmp/export.sql --output /var/lib/heliosdb-new
# 6. Verify and switch
mv /var/lib/heliosdb /var/lib/heliosdb-old
mv /var/lib/heliosdb-new /var/lib/heliosdb
# 7. Update key in secure storage
cp /tmp/new_encryption.key /etc/heliosdb/encryption.key
chmod 400 /etc/heliosdb/encryption.key
Planned Automatic Rotation (Phase 2):
// Future API
storage.rotate_encryption_key(&new_key_manager)?;
// - Online rotation (no downtime)
// - Gradual re-encryption in background
// - Dual-key support during transition
Key Recovery¶
Lost Key Scenarios:
| Scenario | Recovery Possible? | Solution |
|---|---|---|
| Key file deleted | ✅ Yes | Restore from backup |
| Backup exists | ✅ Yes | Use backup key |
| No backup | ❌ NO | Data is permanently lost |
| Wrong key used | ❌ No | Must use correct original key |
| Corrupted key | ❌ No | Need exact original key |
Recovery Procedure:
# 1. Locate backup key
ls -la /backup/keys/
# 2. Verify key is correct length (64 hex chars)
wc -c /backup/keys/encryption.key.backup
# Expected: 64
# 3. Test key with small database copy
cp -r /var/lib/heliosdb /tmp/test_db
export HELIOSDB_ENCRYPTION_KEY=$(cat /backup/keys/encryption.key.backup)
heliosdb-lite verify --path /tmp/test_db
# 4. If successful, restore to production
cp /backup/keys/encryption.key.backup /etc/heliosdb/encryption.key
systemctl restart heliosdb
Working with Encrypted Data¶
Transparent Operations (No Code Changes)¶
Good News: Encryption is completely transparent! Your application code doesn't change.
Without Encryption:
// Regular database operations
let storage = StorageEngine::open("./db", &config)?;
storage.put(b"key1", b"value1")?;
let value = storage.get(b"key1")?.unwrap();
With Encryption:
// Exact same code! Encryption is automatic
let storage = StorageEngine::open("./db", &config)?; // Opens encrypted DB
storage.put(b"key1", b"value1")?; // Automatically encrypted
let value = storage.get(b"key1")?.unwrap(); // Automatically decrypted
Table Operations¶
Create Encrypted Tables:
use heliosdb_lite::{Schema, Column, DataType, Tuple, Value};
// Schema definition (same as without encryption)
let schema = Schema::new(vec![
Column::new("id", DataType::Int4).primary_key(),
Column::new("ssn", DataType::Text),
Column::new("credit_card", DataType::Text),
Column::new("salary", DataType::Float8),
]);
// Create table (metadata is encrypted)
let catalog = storage.catalog();
catalog.create_table("employees", schema)?;
// Insert data (automatically encrypted)
storage.insert_tuple("employees", Tuple::new(vec![
Value::Int4(1),
Value::String("123-45-6789".to_string()),
Value::String("4111-1111-1111-1111".to_string()),
Value::Float8(125000.0),
]))?;
// Query data (automatically decrypted)
let employees = storage.scan_table("employees")?;
for emp in employees {
println!("Employee: {:?}", emp);
// All values are automatically decrypted
}
SQL Queries (Server Mode)¶
PostgreSQL Protocol (encryption is transparent):
-- Connect to encrypted database
psql -h localhost -p 5432 -d encrypted_db
-- All queries work normally
CREATE TABLE patients (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL,
diagnosis TEXT,
prescription TEXT
);
-- Insert sensitive medical data (encrypted automatically)
INSERT INTO patients (name, diagnosis, prescription)
VALUES ('John Doe', 'Type 2 Diabetes', 'Metformin 500mg');
-- Query returns decrypted data
SELECT * FROM patients WHERE id = 1;
-- Joins, aggregations, etc. all work
SELECT diagnosis, COUNT(*)
FROM patients
GROUP BY diagnosis;
Performance Considerations¶
Encryption Overhead:
Based on benchmarks:
| Operation | Unencrypted | Encrypted | Overhead |
|---|---|---|---|
| Small writes (64B) | 1.2 µs | 1.23 µs | +2.5% |
| Medium writes (1KB) | 1.5 µs | 1.54 µs | +2.7% |
| Large writes (16KB) | 3.8 µs | 3.91 µs | +2.9% |
| Point reads | 0.8 µs | 0.82 µs | +2.5% |
| Table scans (1000 rows) | 850 µs | 875 µs | +2.9% |
| Indexed lookups | 1.1 µs | 1.13 µs | +2.7% |
Performance Tips:
-
Use Hardware Acceleration:
-
Batch Operations:
// Less efficient: Many small writes for i in 0..1000 { storage.put(&format!("key_{}", i).as_bytes(), b"value")?; } // More efficient: Batch inserts let tuples: Vec<Tuple> = (0..1000) .map(|i| Tuple::new(vec![Value::Int4(i), Value::String("data".into())])) .collect(); storage.insert_batch("table", tuples)?; -
Memory Considerations:
// Each encrypted value has 28-byte overhead (12-byte nonce + 16-byte tag) // For small values, this can be significant // Example: // 1-byte value → 29 bytes encrypted (29x overhead) // 100-byte value → 128 bytes encrypted (1.28x overhead) // 1KB value → 1052 bytes encrypted (1.05x overhead) // Recommendation: Use larger values when possible
Debugging Encrypted Databases¶
Check Encryption Status:
let storage = StorageEngine::open("./db", &config)?;
// Method 1: Check encryption flag
if storage.is_encrypted() {
println!("✓ Database is encrypted");
} else {
println!("⚠ Database is NOT encrypted");
}
// Method 2: Get encryption info
if let Some(info) = storage.encryption_info() {
println!("Encryption: {}", info);
// Output: "Enabled: AES-256-GCM"
} else {
println!("Encryption: Disabled");
}
Verify Data is Encrypted on Disk:
// Insert test data
storage.put(b"test_key", b"plaintext_value")?;
// Check raw data in RocksDB
let raw_data = storage.db.get(b"test_key")?.unwrap();
// Raw data should NOT match plaintext
assert_ne!(raw_data, b"plaintext_value", "Data is not encrypted!");
// But storage.get() should decrypt it
let decrypted = storage.get(b"test_key")?.unwrap();
assert_eq!(decrypted, b"plaintext_value", "Decryption failed!");
println!("✓ Data is encrypted on disk");
println!("✓ Transparent decryption works");
Logging and Monitoring:
// Enable encryption audit logging (future feature)
config.audit.log_encryption_events = true;
// Logged events:
// - Key loaded from source
// - Encryption enabled/disabled
// - Encryption errors
// - Key rotation (future)
Security Best Practices¶
Key Management Security¶
DO:
✅ Use strong random keys:
# Good: Cryptographically secure random
openssl rand -hex 32
# Bad: Predictable or weak
echo "my_password_123" | md5sum # ❌ DON'T DO THIS
✅ Separate keys from data:
# Good: Key on different volume/server
/etc/heliosdb/encryption.key # Different volume from /var/lib/heliosdb
# Bad: Key in same directory as database
/var/lib/heliosdb/encryption.key # ❌ Same volume
✅ Use proper file permissions:
# Good: Minimal permissions
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Bad: World-readable
chmod 644 /etc/heliosdb/encryption.key # ❌ Others can read
✅ Implement key rotation:
# Rotate keys every 90 days (recommended)
# Rotate immediately after:
# - Security incident
# - Employee departure
# - Suspected compromise
✅ Backup keys securely:
# Good: Encrypted backup in separate location
gpg --encrypt --recipient admin@company.com encryption.key
aws s3 cp encryption.key.gpg s3://secure-backups/keys/
# Good: Hardware security module (HSM)
# Store master key in HSM, use derived keys for databases
DON'T:
❌ Never commit keys to version control:
# Add to .gitignore
echo "*.key" >> .gitignore
echo ".env" >> .gitignore
echo "config/secrets.toml" >> .gitignore
❌ Never log or print keys:
// Bad: Logs the key
println!("Key: {}", key_manager.export_as_hex()); // ❌ DON'T
// Good: Log only metadata
println!("Key loaded from: {}", key_manager.source()); // ✅ OK
❌ Never use hardcoded keys:
// Bad: Hardcoded key in source code
const ENCRYPTION_KEY: &str = "0123..."; // ❌ NEVER DO THIS
// Good: Load from secure source
let key = KeyManager::from_source(&config.encryption.key_source)?; // ✅
❌ Never share keys between environments:
# Bad: Same key for dev, staging, prod
# HELIOSDB_ENCRYPTION_KEY=same_key_everywhere # ❌
# Good: Different key per environment
# DEV: Different key for development
# STAGING: Different key for staging
# PROD: Different key for production
Access Control¶
Principle of Least Privilege:
# Database process user
useradd -r -s /bin/false heliosdb
# Key file permissions
chown root:heliosdb /etc/heliosdb/encryption.key
chmod 440 /etc/heliosdb/encryption.key
# Database files
chown heliosdb:heliosdb /var/lib/heliosdb
chmod 700 /var/lib/heliosdb
# Only heliosdb user can access database and keys
Network Security (Server Mode):
# config.toml
[server]
host = "127.0.0.1" # Localhost only (use reverse proxy for external)
port = 5432
ssl_enabled = true # Always use SSL/TLS for network traffic
[server.ssl]
cert_file = "/etc/heliosdb/server.crt"
key_file = "/etc/heliosdb/server.key"
ca_file = "/etc/heliosdb/ca.crt"
Integration with Audit Logging¶
Enable Comprehensive Auditing:
use heliosdb_lite::Config;
let mut config = Config::default();
// Enable encryption
config.encryption.enabled = true;
config.encryption.key_source = /* ... */;
// Enable audit logging
config.audit.enabled = true;
config.audit.log_file = Some("/var/log/heliosdb/audit.log".into());
config.audit.log_format = heliosdb_lite::audit::LogFormat::Json;
// Audit all data access
config.audit.log_reads = true;
config.audit.log_writes = true;
config.audit.log_deletes = true;
// Combined: Encrypted data + full audit trail
let storage = StorageEngine::open("./db", &config)?;
Audit Log Example (JSON format):
{
"timestamp": "2025-12-01T10:30:45Z",
"event_type": "read",
"table": "patients",
"user": "doctor_smith",
"query": "SELECT * FROM patients WHERE id = 123",
"rows_affected": 1,
"encryption_enabled": true,
"source_ip": "192.168.1.100"
}
Compliance Reporting:
# Generate encryption compliance report
heliosdb-lite audit-report \
--input /var/log/heliosdb/audit.log \
--type encryption-compliance \
--output /reports/encryption_compliance_2025_12.pdf
Compliance Scenarios¶
HIPAA Compliance Setup¶
Health Insurance Portability and Accountability Act requirements:
Required Controls: 1. ✅ Encryption at rest (covered by TDE) 2. ✅ Encryption in transit (use SSL/TLS) 3. ✅ Access controls (OS-level permissions) 4. ✅ Audit logging (enable audit logs) 5. ✅ Key management (secure key storage)
Configuration:
# hipaa_config.toml
[encryption]
enabled = true
algorithm = "Aes256Gcm"
[encryption.key_source]
File = "/secure/hipaa/encryption.key"
[server]
ssl_enabled = true
[server.ssl]
cert_file = "/etc/heliosdb/hipaa_server.crt"
key_file = "/etc/heliosdb/hipaa_server.key"
min_tls_version = "1.2" # HIPAA requires TLS 1.2+
[audit]
enabled = true
log_file = "/var/log/heliosdb/hipaa_audit.log"
log_format = "Json"
log_reads = true
log_writes = true
log_deletes = true
log_schema_changes = true
[audit.retention]
days = 2555 # 7 years (HIPAA requirement)
HIPAA-Compliant Rust Code:
use heliosdb_lite::{Config, Schema, Column, DataType, Tuple, Value};
fn setup_hipaa_database() -> heliosdb_lite::Result<()> {
// Load HIPAA configuration
let config = Config::from_file("hipaa_config.toml")?;
// Verify encryption is enabled
assert!(config.encryption.enabled, "HIPAA requires encryption!");
assert!(config.audit.enabled, "HIPAA requires audit logging!");
// Open database
let storage = StorageEngine::open("/var/lib/heliosdb/hipaa", &config)?;
// Create PHI table (Protected Health Information)
let catalog = storage.catalog();
let schema = Schema::new(vec![
Column::new("patient_id", DataType::Int4).primary_key(),
Column::new("name", DataType::Text),
Column::new("ssn", DataType::Text),
Column::new("diagnosis", DataType::Text),
Column::new("medications", DataType::Text),
Column::new("created_at", DataType::Timestamp),
]);
catalog.create_table("patient_records", schema)?;
println!("✓ HIPAA-compliant database configured");
println!(" - Encryption: AES-256-GCM");
println!(" - SSL/TLS: Enabled");
println!(" - Audit logging: 7-year retention");
Ok(())
}
Verification Checklist:
# 1. Verify encryption is enabled
heliosdb-lite verify --config hipaa_config.toml --check encryption
# 2. Verify SSL/TLS certificates
openssl s_client -connect localhost:5432 -showcerts
# 3. Verify audit logging
tail -f /var/log/heliosdb/hipaa_audit.log
# 4. Verify file permissions
ls -la /var/lib/heliosdb/hipaa
# Expected: drwx------ (700)
# 5. Verify key security
ls -la /secure/hipaa/encryption.key
# Expected: -r-------- (400)
GDPR Data Protection¶
General Data Protection Regulation requirements:
GDPR Principles: 1. Data Protection by Design: Encryption enabled from day one 2. Right to Erasure: Secure deletion of personal data 3. Data Portability: Export encrypted backups 4. Breach Notification: Audit logs for incident response 5. Privacy by Default: Minimal data collection
GDPR Configuration:
# gdpr_config.toml
[encryption]
enabled = true
algorithm = "Aes256Gcm"
key_source = { File = "/etc/heliosdb/gdpr_key" }
[audit]
enabled = true
log_file = "/var/log/heliosdb/gdpr_audit.log"
log_format = "Json"
# Log personal data access (GDPR Article 15)
log_reads = true
log_writes = true
log_deletes = true
# Retention policy (GDPR Article 17)
[audit.retention]
days = 730 # 2 years minimum
[storage]
# Enable secure deletion (GDPR "Right to Erasure")
secure_delete = true
overwrite_deleted_data = true
GDPR-Compliant Data Operations:
// Right to Access (GDPR Article 15)
fn export_user_data(storage: &StorageEngine, user_id: i32)
-> heliosdb_lite::Result<String>
{
// Query all user data (automatically decrypted)
let query = format!("SELECT * FROM users WHERE id = {}", user_id);
let results = storage.execute_sql(&query)?;
// Export as JSON (for data portability)
let json = serde_json::to_string_pretty(&results)?;
// Log access (for audit trail)
log::info!("User data exported for user_id={}", user_id);
Ok(json)
}
// Right to Erasure (GDPR Article 17)
fn delete_user_data(storage: &StorageEngine, user_id: i32)
-> heliosdb_lite::Result<()>
{
// Delete user from all tables
storage.execute_sql(&format!("DELETE FROM users WHERE id = {}", user_id))?;
storage.execute_sql(&format!("DELETE FROM orders WHERE user_id = {}", user_id))?;
storage.execute_sql(&format!("DELETE FROM preferences WHERE user_id = {}", user_id))?;
// Secure deletion (overwrite on disk)
// Enabled via config.storage.secure_delete = true
// Log deletion (compliance requirement)
log::warn!("User data deleted for user_id={} (GDPR erasure)", user_id);
Ok(())
}
// Data Breach Notification (GDPR Article 33)
fn check_encryption_breach(storage: &StorageEngine) -> heliosdb_lite::Result<()> {
if !storage.is_encrypted() {
log::error!("CRITICAL: Database is not encrypted! GDPR violation!");
// Trigger incident response
notify_data_protection_officer("Unencrypted database detected")?;
}
Ok(())
}
PCI-DSS Requirements¶
Payment Card Industry Data Security Standard:
PCI-DSS Requirements for Encryption: - Requirement 3: Protect stored cardholder data - Requirement 4: Encrypt transmission of cardholder data - Requirement 8: Identify and authenticate access
PCI-DSS Configuration:
# pci_config.toml
[encryption]
enabled = true # Requirement 3.4: Render PAN unreadable
algorithm = "Aes256Gcm" # Strong cryptography (Requirement 3.5)
[encryption.key_source]
File = "/secure/pci/encryption.key"
[encryption.key_rotation]
enabled = true
interval_days = 90 # Key rotation (Requirement 3.6.4)
[audit]
enabled = true # Requirement 10: Track and monitor access
log_file = "/var/log/heliosdb/pci_audit.log"
# Log all access to cardholder data
log_reads = true
log_writes = true
log_deletes = true
log_schema_changes = true
[audit.retention]
days = 365 # Minimum 1 year (Requirement 10.7)
[server.ssl]
enabled = true # Requirement 4: Encrypt in transit
min_tls_version = "1.2"
PCI-DSS Compliant Schema:
use heliosdb_lite::{Schema, Column, DataType};
fn create_pci_compliant_schema() -> Schema {
Schema::new(vec![
Column::new("transaction_id", DataType::Int8).primary_key(),
// PAN (Primary Account Number) - MUST be encrypted
Column::new("card_number", DataType::Text),
// Cardholder name
Column::new("cardholder_name", DataType::Text),
// Expiration date - Can be stored with encryption
Column::new("expiry_month", DataType::Int2),
Column::new("expiry_year", DataType::Int4),
// NEVER store these (PCI-DSS Requirement 3.2):
// - CVV/CVV2 (Card Verification Value)
// - Full magnetic stripe data
// - PIN or PIN block
Column::new("amount", DataType::Numeric),
Column::new("timestamp", DataType::Timestamp),
])
}
// Example: Storing payment card data (encrypted)
fn process_payment(storage: &StorageEngine, card_data: CardData)
-> heliosdb_lite::Result<()>
{
// Data is automatically encrypted by TDE
storage.insert_tuple("transactions", Tuple::new(vec![
Value::Int8(generate_transaction_id()),
Value::String(card_data.pan), // Encrypted at rest
Value::String(card_data.name), // Encrypted at rest
Value::Int2(card_data.exp_month),
Value::Int4(card_data.exp_year),
Value::Numeric(card_data.amount),
Value::Timestamp(Utc::now()),
]))?;
// Audit log automatically records access
Ok(())
}
PCI-DSS Compliance Verification:
# 1. Verify encryption (Requirement 3.4)
heliosdb-lite verify --config pci_config.toml --check encryption
# ✓ AES-256-GCM enabled
# 2. Verify key management (Requirement 3.5, 3.6)
ls -la /secure/pci/encryption.key
# ✓ Permissions: 400 (read-only by owner)
# 3. Verify audit logging (Requirement 10)
tail /var/log/heliosdb/pci_audit.log
# ✓ All access logged with timestamps
# 4. Verify SSL/TLS (Requirement 4)
openssl s_client -connect localhost:5432 | grep Protocol
# ✓ TLSv1.2 or higher
# 5. Check for prohibited data (Requirement 3.2)
grep -i "cvv\|cvv2\|cvc" /var/lib/heliosdb/pci/
# ✓ No results (good - not storing CVV)
Recovery Scenarios¶
Lost Key Recovery¶
Scenario: Encryption key file was accidentally deleted.
If you have a backup:
# 1. Stop database
systemctl stop heliosdb
# 2. Restore key from backup
cp /backup/keys/encryption.key.20251201 /etc/heliosdb/encryption.key
# 3. Set correct permissions
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# 4. Verify key works
heliosdb-lite verify --config /etc/heliosdb/config.toml --check encryption
# 5. Restart database
systemctl start heliosdb
systemctl status heliosdb
If you DON'T have a backup:
⚠️ CRITICAL: Without the encryption key, your data is PERMANENTLY LOST.
There is NO way to recover encrypted data without the original key.
Prevention:
- Always maintain secure backups of encryption keys
- Store backups in multiple locations
- Use key management services (KMS/HSM)
- Document key recovery procedures
- Test recovery procedures regularly
Database Migration with Encryption¶
Scenario: Migrate encrypted database to new server.
Step-by-Step Migration:
# On old server:
# 1. Backup encrypted database
tar -czf database_backup.tar.gz /var/lib/heliosdb/
# 2. Backup encryption key SEPARATELY
tar -czf encryption_key.tar.gz /etc/heliosdb/encryption.key
# 3. Transfer to new server (use secure channel)
scp database_backup.tar.gz user@newserver:/tmp/
scp encryption_key.tar.gz user@newserver:/tmp/
# On new server:
# 4. Extract database
mkdir -p /var/lib/heliosdb
tar -xzf /tmp/database_backup.tar.gz -C /
# 5. Extract encryption key
mkdir -p /etc/heliosdb
tar -xzf /tmp/encryption_key.tar.gz -C /
# 6. Set permissions
chmod 700 /var/lib/heliosdb
chown heliosdb:heliosdb /var/lib/heliosdb
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# 7. Verify database opens correctly
heliosdb-lite verify --path /var/lib/heliosdb
# 8. Start database
systemctl start heliosdb
# 9. Test connectivity
psql -h localhost -p 5432 -c "SELECT 1"
Alternative: Export/Import Method:
# On old server:
# 1. Export data in decrypted format
heliosdb-lite export --input /var/lib/heliosdb --output /tmp/export.sql
# 2. Transfer export file
scp /tmp/export.sql user@newserver:/tmp/
# On new server:
# 3. Generate NEW encryption key for new server
heliosdb-lite keygen > /etc/heliosdb/encryption.key
chmod 400 /etc/heliosdb/encryption.key
# 4. Initialize encrypted database with NEW key
export HELIOSDB_ENCRYPTION_KEY=$(cat /etc/heliosdb/encryption.key)
heliosdb-lite init --path /var/lib/heliosdb --encrypted
# 5. Import data (will be encrypted with NEW key)
heliosdb-lite import --input /tmp/export.sql --output /var/lib/heliosdb
# Result: Data re-encrypted with new key on new server
Backup and Restore¶
Full Encrypted Backup:
# Backup strategy: Backup encrypted database files directly
# Advantage: Fast, no decryption needed
# Disadvantage: Requires same encryption key for restore
# 1. Consistent snapshot (stop writes)
heliosdb-lite checkpoint --path /var/lib/heliosdb
# 2. Backup database files (encrypted)
tar -czf /backup/heliosdb_encrypted_20251201.tar.gz /var/lib/heliosdb/
# 3. Backup encryption key SEPARATELY and SECURELY
gpg --encrypt --recipient backup@company.com /etc/heliosdb/encryption.key
cp encryption.key.gpg /secure/backup/keys/
# 4. Verify backup
tar -tzf /backup/heliosdb_encrypted_20251201.tar.gz | head
# 5. Test restore to temporary location
mkdir /tmp/restore_test
tar -xzf /backup/heliosdb_encrypted_20251201.tar.gz -C /tmp/restore_test
heliosdb-lite verify --path /tmp/restore_test/var/lib/heliosdb
Decrypted Logical Backup:
# Backup strategy: Export to SQL (decrypted)
# Advantage: Key-independent, can restore with different key
# Disadvantage: Slower, decrypted data must be secured
# 1. Export to SQL (automatically decrypts during export)
heliosdb-lite export \
--input /var/lib/heliosdb \
--output /backup/heliosdb_export_20251201.sql
# 2. Encrypt the SQL export for security
gpg --encrypt --recipient backup@company.com \
/backup/heliosdb_export_20251201.sql
# 3. Store encrypted SQL file
mv heliosdb_export_20251201.sql.gpg /secure/backup/exports/
# 4. Restore procedure
gpg --decrypt /secure/backup/exports/heliosdb_export_20251201.sql.gpg \
> /tmp/restore.sql
heliosdb-lite import \
--input /tmp/restore.sql \
--output /var/lib/heliosdb_restored
# 5. Secure cleanup
shred -u /tmp/restore.sql # Securely delete decrypted SQL
Automated Backup Script:
#!/bin/bash
# backup_encrypted_db.sh
set -e
DB_PATH="/var/lib/heliosdb"
BACKUP_DIR="/backup/heliosdb"
KEY_FILE="/etc/heliosdb/encryption.key"
DATE=$(date +%Y%m%d_%H%M%S)
# 1. Create consistent snapshot
heliosdb-lite checkpoint --path "$DB_PATH"
# 2. Backup database (encrypted)
tar -czf "$BACKUP_DIR/db_$DATE.tar.gz" "$DB_PATH"
# 3. Backup key (encrypted with GPG)
gpg --encrypt --recipient backup@company.com "$KEY_FILE" \
-o "$BACKUP_DIR/key_$DATE.gpg"
# 4. Verify backup integrity
tar -tzf "$BACKUP_DIR/db_$DATE.tar.gz" > /dev/null
# 5. Retention: Keep last 30 days
find "$BACKUP_DIR" -name "db_*.tar.gz" -mtime +30 -delete
find "$BACKUP_DIR" -name "key_*.gpg" -mtime +30 -delete
# 6. Upload to cloud (optional)
# aws s3 cp "$BACKUP_DIR/db_$DATE.tar.gz" s3://backups/heliosdb/
# aws s3 cp "$BACKUP_DIR/key_$DATE.gpg" s3://backups/heliosdb/keys/
echo "✓ Backup completed: $DATE"
Schedule with cron:
# Run daily at 2 AM
0 2 * * * /usr/local/bin/backup_encrypted_db.sh >> /var/log/heliosdb_backup.log 2>&1
Performance Considerations¶
Benchmarking Encrypted vs Unencrypted¶
Run Built-in Benchmarks:
# Run encryption benchmarks
cargo bench --bench encryption_benchmark
# Sample output:
# Encryption/put_get/64_bytes time: [1.23 µs 1.25 µs 1.27 µs]
# Unencrypted/put_get/64_bytes time: [1.20 µs 1.22 µs 1.24 µs]
# Overhead: ~2.5%
Custom Performance Testing:
use std::time::Instant;
use heliosdb_lite::{Config, storage::StorageEngine};
fn benchmark_encryption_overhead() -> heliosdb_lite::Result<()> {
let iterations = 10_000;
// Test 1: Encrypted database
let mut config = Config::in_memory();
config.encryption.enabled = true;
config.encryption.key_source = /* ... */;
let encrypted_storage = StorageEngine::open_in_memory(&config)?;
let start = Instant::now();
for i in 0..iterations {
encrypted_storage.put(
&format!("key_{}", i).as_bytes(),
b"test_data_payload_1234567890"
)?;
}
let encrypted_duration = start.elapsed();
// Test 2: Unencrypted database
let mut config_unenc = Config::in_memory();
config_unenc.encryption.enabled = false;
let unencrypted_storage = StorageEngine::open_in_memory(&config_unenc)?;
let start = Instant::now();
for i in 0..iterations {
unencrypted_storage.put(
&format!("key_{}", i).as_bytes(),
b"test_data_payload_1234567890"
)?;
}
let unencrypted_duration = start.elapsed();
// Calculate overhead
let overhead = (encrypted_duration.as_micros() as f64
/ unencrypted_duration.as_micros() as f64 - 1.0) * 100.0;
println!("Results ({} iterations):", iterations);
println!(" Unencrypted: {:?}", unencrypted_duration);
println!(" Encrypted: {:?}", encrypted_duration);
println!(" Overhead: {:.2}%", overhead);
Ok(())
}
Optimization Tips¶
1. Use Appropriate Value Sizes:
// Inefficient: Many tiny encrypted values
// Each has 28-byte overhead (nonce + tag)
for i in 0..1000 {
storage.put(&format!("counter_{}", i).as_bytes(), &[i as u8])?;
// 1-byte value → 29 bytes stored (29x overhead)
}
// Efficient: Batch into larger values
let counters: Vec<u8> = (0..1000).map(|i| i as u8).collect();
storage.put(b"counters", &counters)?;
// 1000-byte value → 1028 bytes stored (1.028x overhead)
2. Minimize Metadata Operations:
// Less efficient: Frequent catalog queries (metadata is encrypted)
for table_name in table_names {
let schema = catalog.get_table_schema(table_name)?; // Decrypts metadata
process_schema(schema);
}
// More efficient: Cache metadata
let schemas: HashMap<String, Schema> = table_names.iter()
.map(|name| (name.clone(), catalog.get_table_schema(name)))
.collect()?;
for (name, schema) in schemas {
process_schema(schema); // No decryption needed
}
3. Batch Writes:
// Use transactions or batch inserts to amortize encryption overhead
storage.begin_transaction()?;
for tuple in large_dataset {
storage.insert_tuple("table", tuple)?;
}
storage.commit()?; // Single fsync
Hardware Acceleration¶
Verify AES-NI Support:
# Linux
lscpu | grep -i aes
# Look for: Flags: ... aes ...
cat /proc/cpuinfo | grep aes
# Should show "aes" flag
# macOS
sysctl -a | grep -i aes
# macOS kern.hv_support: 1 (includes AES support)
# Check with Rust
cargo build --release
# Compiler automatically uses AES-NI if available
Performance Impact of AES-NI:
| CPU | Without AES-NI | With AES-NI | Speedup |
|---|---|---|---|
| Intel Core i7 | ~15 µs | ~1.2 µs | 12x faster |
| AMD Ryzen 7 | ~14 µs | ~1.1 µs | 13x faster |
| ARM (modern) | ~10 µs | ~0.9 µs | 11x faster |
Note: HeliosDB-Lite's <3% overhead assumes AES-NI is available.
Troubleshooting¶
Common Issues¶
Issue: "Environment variable 'HELIOSDB_ENCRYPTION_KEY' not found"¶
Cause: Encryption key not set in environment.
Solution:
# Check if variable is set
echo $HELIOSDB_ENCRYPTION_KEY
# Set the variable
export HELIOSDB_ENCRYPTION_KEY="<64-character-hex-key>"
# For systemd services
sudo systemctl edit heliosdb
# Add:
# [Service]
# Environment="HELIOSDB_ENCRYPTION_KEY=<key>"
# For Docker
docker run -e HELIOSDB_ENCRYPTION_KEY="<key>" heliosdb-lite
Issue: "Decryption failed: Authentication tag verification failed"¶
Causes: 1. Wrong encryption key 2. Corrupted data 3. Key changed after data was encrypted
Solutions:
# 1. Verify you're using the correct key
cat /etc/heliosdb/encryption.key
# Should be 64 hex characters
# 2. Check for data corruption
heliosdb-lite verify --path /var/lib/heliosdb
# 3. Try restore from backup
cp -r /var/lib/heliosdb /var/lib/heliosdb.corrupted
tar -xzf /backup/heliosdb_latest.tar.gz -C /var/lib/
# 4. Restore correct key
cp /backup/keys/encryption.key /etc/heliosdb/encryption.key
Issue: "Hex key must be 64 characters (32 bytes), got X"¶
Cause: Invalid key format.
Solution:
# Generate proper key (64 hex chars)
heliosdb-lite keygen
# Or:
openssl rand -hex 32
# Verify length
echo -n "<your-key>" | wc -c
# Should output: 64
Issue: "Failed to read key file: Permission denied"¶
Cause: Insufficient file permissions.
Solution:
# Check permissions
ls -la /etc/heliosdb/encryption.key
# Fix ownership
sudo chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Fix permissions
sudo chmod 400 /etc/heliosdb/encryption.key
# Verify database process user
ps aux | grep heliosdb
# Ensure it's running as 'heliosdb' user
Performance Issues¶
Issue: "Database operations are slow after enabling encryption"¶
Diagnosis:
# 1. Check if AES-NI is available
lscpu | grep aes
# 2. Run benchmarks
cargo bench --bench encryption_benchmark
# 3. Profile encryption operations
cargo build --release --features profiling
perf record ./target/release/heliosdb-lite
perf report
Solutions:
// 1. Batch operations
storage.begin_transaction()?;
for item in items {
storage.insert_tuple("table", item)?;
}
storage.commit()?;
// 2. Increase cache size
config.storage.cache_size = 1024 * 1024 * 1024; // 1 GB
// 3. Use in-memory mode for temporary data
let temp_config = Config::in_memory();
// Encryption overhead is minimal in-memory
Security Issues¶
Issue: "Security audit flagged encryption key in logs"¶
Immediate Action:
# 1. Rotate key immediately
NEW_KEY=$(heliosdb-lite keygen)
# Follow key rotation procedure
# 2. Scrub logs
sudo find /var/log -name "*.log" -exec grep -l "HELIOSDB_ENCRYPTION_KEY" {} \; \
-exec shred -u {} \;
# 3. Update code to prevent logging
# Remove any println!() or log statements that include keys
Prevention:
// Bad: Logs key
log::debug!("Using key: {}", key_manager.export_as_hex()); // ❌
// Good: Logs only metadata
log::debug!("Key loaded from: {:?}", key_manager.source()); // ✅
Issue: "Key file has insecure permissions"¶
Fix:
# Set minimal permissions
sudo chmod 400 /etc/heliosdb/encryption.key
sudo chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Verify
ls -la /etc/heliosdb/encryption.key
# Expected: -r-------- 1 heliosdb heliosdb 64 Dec 01 10:00 encryption.key
# Audit permissions recursively
find /etc/heliosdb -type f -not -perm 400 -ls
# Should be empty
Getting Help¶
Diagnostic Information to Collect:
# System info
uname -a
lscpu | grep -i aes
# Database info
heliosdb-lite --version
heliosdb-lite verify --path /var/lib/heliosdb
# Configuration
cat /etc/heliosdb/config.toml | grep -A 10 encryption
# Logs
tail -100 /var/log/heliosdb/heliosdb.log
# Permissions
ls -laR /etc/heliosdb
ls -laR /var/lib/heliosdb
Report Issues: - GitHub Issues: https://github.com/dimensigon/HeliosDB-Lite/issues - Security Issues: security@heliosdb.io (for sensitive issues) - Community Forum: https://discuss.heliosdb.io
Summary¶
You've learned how to:
✅ Enable transparent data encryption with AES-256-GCM ✅ Manage encryption keys securely using multiple methods ✅ Configure encryption for various deployment scenarios ✅ Achieve compliance with HIPAA, GDPR, and PCI-DSS ✅ Maintain performance with <3% overhead ✅ Handle recovery scenarios and migrations ✅ Troubleshoot common encryption issues
Key Takeaways:
- Encryption is transparent: No application code changes needed
- Security is paramount: Protect keys with same rigor as data
- Backup keys separately: Data is unrecoverable without keys
- Monitor performance: Encryption overhead is minimal with AES-NI
- Plan for compliance: Enable audit logging with encryption
- Test recovery procedures: Verify backups and key recovery regularly
Next Steps:
- Explore Audit Logging Tutorial
- Review Security Best Practices
- Set up Automated Backups
- Configure High Availability
Version History: - 2.5.0-dev (2025-12-01): Initial comprehensive tutorial - 2.1.0 (2024-11-13): Encryption feature released
Related Documentation: - Encryption Integration Summary - Security Review Report - API Reference: Crypto Module