Migration Guide: v3.0.0 to v3.1.0¶

Published: 2025-12-08 Target Audience: Developers, System Administrators, DevOps Engineers

Table of Contents¶

Overview
What's New in v3.1.0
Breaking Changes
New Features Overview
CLI Flag Changes
Code Migration Examples
Single-User to Multi-User Transition
Backward Compatibility Notes
Troubleshooting Migration Issues
FAQ

Overview¶

HeliosDB-Lite v3.1.0 introduces comprehensive multi-user ACID transaction support for in-memory mode with user-triggered persistence. This release maintains 100% backward compatibility with v3.0.0 while adding powerful new capabilities for multi-user applications.

Migration Complexity: Low Estimated Migration Time: 30 minutes - 2 hours Breaking Changes: None Recommended Upgrade Path: Direct upgrade from v3.0.0

What's New in v3.1.0¶

Major Features¶

Multi-User Session Management
Concurrent user sessions with independent transaction contexts
Per-user resource quotas and connection limits
Session timeout and cleanup mechanisms
Full PostgreSQL-compatible session management
Enhanced Locking and Concurrency
Advanced lock manager with deadlock detection
Three lock modes: Shared, Exclusive, Update
Configurable lock timeouts
Wait-for graph based deadlock resolution
Configurable Isolation Levels
READ COMMITTED (default)
REPEATABLE READ
SERIALIZABLE with conflict detection
User-Triggered Persistence
Manual dump/restore commands
Incremental dump support (append mode)
Automated backup scheduling
Dirty state tracking
Resource Management
Per-user memory quotas
Transaction and query timeouts
System-wide resource limits
Connection pooling support

Enhancements¶

Improved CLI with new dump and restore commands
Enhanced configuration with 4 new sections
Dirty state warnings on shutdown
Comprehensive API documentation
Performance optimizations for high-concurrency workloads

Breaking Changes¶

None. HeliosDB-Lite v3.1.0 is 100% backward compatible with v3.0.0.

Existing applications will continue to work without any code changes.

New Features Overview¶

1. Session Management¶

Before (v3.0.0):

// Single embedded instance, no session concept
let db = EmbeddedDatabase::new_in_memory()?;
db.execute("INSERT INTO users VALUES (1, 'Alice')")?;

After (v3.1.0):

// Optional: Explicit session management for multi-user apps
use heliosdb_lite::session::{SessionManager, SessionConfig, IsolationLevel, User};

let db = EmbeddedDatabase::new_in_memory()?;
let manager = SessionManager::new(SessionConfig::default());

let user = User::new("alice", "password");
let session_id = manager.create_session(&user, IsolationLevel::ReadCommitted)?;

manager.begin_transaction(session_id)?;
db.execute("INSERT INTO users VALUES (1, 'Alice')")?;
manager.commit_transaction(session_id)?;

Migration Note: Session management is optional. Existing code continues to work without sessions.

2. Dump/Restore Commands¶

New CLI Commands:

# Dump in-memory database to file
heliosdb-lite dump --output backup.heliodump

# Restore from dump file
heliosdb-lite restore --input backup.heliodump

# Incremental backup (append mode)
heliosdb-lite dump --output backup.heliodump --append

Programmatic API:

use heliosdb_lite::storage::dump::{DumpManager, DumpOptions, DumpMode, CompressionType};

let dump_manager = DumpManager::new(db.storage_engine());

let options = DumpOptions {
    output_path: "/backups/db.heliodump".into(),
    mode: DumpMode::Full,
    compress: true,
    compression_type: CompressionType::Zstd,
    tables: None,
    append: false,
};

dump_manager.dump(options)?;

3. Isolation Levels¶

Before (v3.0.0): - Default MVCC snapshot isolation (similar to REPEATABLE READ)

After (v3.1.0):

-- Set isolation level per transaction
BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
-- or
BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED;
-- or
BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;

// Via API
let mut txn = Transaction::new(db.storage(), snapshot, lock_manager);
txn.set_isolation_level(IsolationLevel::Serializable);

4. Resource Quotas¶

New Configuration:

[resource_quotas]
memory_limit_per_user = 1073741824  # 1GB
connection_limit_per_user = 10
transaction_timeout_secs = 300
query_timeout_secs = 60

CLI Flag Changes¶

Start Command¶

New Flags (v3.1.0):

# Before (v3.0.0): --memory flag implied embedded mode
heliosdb-lite start --memory --port 5432

# After (v3.1.0): Same command, now supports multi-user sessions
heliosdb-lite start --memory --port 5432

# New option: Explicitly require --data-dir for persistent mode
heliosdb-lite start --data-dir /var/lib/heliosdb --port 5432

Migration: No changes required. Existing commands work identically.

REPL Command¶

Before (v3.0.0):

heliosdb-lite repl --memory

After (v3.1.0):

# Same command, now with session support
heliosdb-lite repl --memory

# New: Execute single query and exit
heliosdb-lite repl --memory --execute "SELECT * FROM users"

Migration: No changes required.

New Commands (v3.1.0)¶

Dump Command¶

heliosdb-lite dump [OPTIONS]

Options:
  --output <PATH>           Output file path (required)
  --compress <TYPE>         Compression: none, lz4, zstd (default: zstd)
  --compression-level <N>   Compression level (default: 3)
  --tables <TABLES>         Comma-separated table list
  --append                  Incremental dump mode
  --verbose                 Verbose output

Restore Command¶

heliosdb-lite restore [OPTIONS]

Options:
  --input <PATH>            Input file path (required)
  --mode <MODE>             Restore mode: clean, append (default: clean)
  --tables <TABLES>         Comma-separated table list
  --on-conflict <ACTION>    Conflict resolution: error, skip, update
  --verbose                 Verbose output

Status Command¶

heliosdb-lite status [OPTIONS]

Options:
  --format <FORMAT>         Output format: text, json (default: text)

Code Migration Examples¶

Example 1: Basic Application (No Changes Required)¶

v3.0.0 Code:

use heliosdb_lite::EmbeddedDatabase;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = EmbeddedDatabase::new_in_memory()?;

    db.execute("CREATE TABLE users (id INT PRIMARY KEY, name TEXT)")?;
    db.execute("INSERT INTO users VALUES (1, 'Alice')")?;

    let results = db.query("SELECT * FROM users", &[])?;
    println!("Users: {:?}", results);

    Ok(())
}

v3.1.0 Code:

// SAME CODE - NO CHANGES REQUIRED
use heliosdb_lite::EmbeddedDatabase;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = EmbeddedDatabase::new_in_memory()?;

    db.execute("CREATE TABLE users (id INT PRIMARY KEY, name TEXT)")?;
    db.execute("INSERT INTO users VALUES (1, 'Alice')")?;

    let results = db.query("SELECT * FROM users", &[])?;
    println!("Users: {:?}", results);

    Ok(())
}

Example 2: Adding Multi-User Support (Optional)¶

v3.1.0 with Sessions:

use heliosdb_lite::{EmbeddedDatabase, session::*};
use std::sync::Arc;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = Arc::new(EmbeddedDatabase::new_in_memory()?);
    let manager = Arc::new(SessionManager::new(SessionConfig::default()));

    db.execute("CREATE TABLE accounts (id INT PRIMARY KEY, balance INT)")?;
    db.execute("INSERT INTO accounts VALUES (1, 1000)")?;

    // Create user session
    let user = User::new("alice", "password");
    let session_id = manager.create_session(&user, IsolationLevel::Serializable)?;

    // Execute transaction
    manager.begin_transaction(session_id)?;

    let balance: i32 = db.query_one("SELECT balance FROM accounts WHERE id = 1", &[])?;
    db.execute(&format!("UPDATE accounts SET balance = {} WHERE id = 1", balance - 100))?;

    manager.commit_transaction(session_id)?;
    manager.destroy_session(session_id)?;

    Ok(())
}

Example 3: Adding Dump/Restore¶

Before (v3.0.0):

// No built-in dump/restore
// Had to use RocksDB backup or custom serialization

After (v3.1.0):

use heliosdb_lite::storage::dump::{DumpManager, DumpOptions, DumpMode, CompressionType};

fn backup_database(db: &EmbeddedDatabase) -> Result<(), Box<dyn std::error::Error>> {
    let dump_manager = DumpManager::new(db.storage_engine());

    let options = DumpOptions {
        output_path: "/backups/db.heliodump".into(),
        mode: DumpMode::Full,
        compress: true,
        compression_type: CompressionType::Zstd,
        tables: None,
        append: false,
    };

    let report = dump_manager.dump(options)?;
    println!("Dumped {} tables, {} rows", report.tables_dumped, report.rows_dumped);

    Ok(())
}

Example 4: Isolation Level Configuration¶

New in v3.1.0:

use heliosdb_lite::session::IsolationLevel;

// Per-session isolation level
let session_id = manager.create_session(&user, IsolationLevel::Serializable)?;

// Or set via SQL
db.execute("SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL SERIALIZABLE")?;

// Or per-transaction
db.execute("BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED")?;

Single-User to Multi-User Transition¶

Step 1: Assess Current Usage¶

Questions to ask: 1. Do you have concurrent database access? 2. Do you need per-user resource limits? 3. Do you require different isolation levels for different operations? 4. Do you need session-based auditing?

If NO to all: Continue using v3.1.0 without sessions (same as v3.0.0) If YES to any: Adopt session management APIs

Step 2: Add Session Management (If Needed)¶

Minimal changes:

// Before
let db = EmbeddedDatabase::new_in_memory()?;

// After (add these lines)
use heliosdb_lite::session::{SessionManager, SessionConfig, User, IsolationLevel};
let manager = SessionManager::new(SessionConfig::default());
let user = User::new("default_user", "password");
let session_id = manager.create_session(&user, IsolationLevel::ReadCommitted)?;

Step 3: Update Transaction Handling¶

Before:

db.execute("BEGIN")?;
db.execute("INSERT INTO users VALUES (1, 'Alice')")?;
db.execute("COMMIT")?;

After (with sessions):

manager.begin_transaction(session_id)?;
db.execute("INSERT INTO users VALUES (1, 'Alice')")?;
manager.commit_transaction(session_id)?;

Or keep using SQL (no API changes):

db.execute("BEGIN")?;
db.execute("INSERT INTO users VALUES (1, 'Alice')")?;
db.execute("COMMIT")?;
// This still works in v3.1.0!

Step 4: Add Backup Strategy¶

# Setup cron job for daily backups
0 2 * * * heliosdb-lite dump --output /backups/daily-$(date +\%Y\%m\%d).heliodump

Backward Compatibility Notes¶

API Compatibility¶

✅ 100% Compatible: - EmbeddedDatabase::new_in_memory() - EmbeddedDatabase::execute() - EmbeddedDatabase::query() - EmbeddedDatabase::query_one() - All existing SQL commands

New (Non-Breaking): - SessionManager (optional) - LockManager (optional) - DumpManager (optional) - Transaction::set_isolation_level()

Configuration Compatibility¶

v3.0.0 config.toml works unchanged in v3.1.0

Optional new sections:

[session]  # Optional
[locks]    # Optional
[dump]     # Optional
[resource_quotas]  # Optional

Database File Compatibility¶

In-memory mode: No files, always compatible
Persistent mode: RocksDB format unchanged
Dump format: New .heliodump format (v3.1.0+)

Wire Protocol Compatibility¶

PostgreSQL wire protocol: Unchanged
HTTP REST API: Unchanged
All existing clients continue to work

Troubleshooting Migration Issues¶

Issue 1: "Session not found" Errors¶

Symptom:

Error: SessionNotFound(123)

Cause: Using session APIs without creating session

Solution:

// Create session before use
let session_id = manager.create_session(&user, IsolationLevel::ReadCommitted)?;
manager.begin_transaction(session_id)?;

Issue 2: Lock Timeout Errors¶

Symptom:

Error: Lock timeout exceeded

Cause: High lock contention with default 30-second timeout

Solution:

[locks]
timeout_ms = 60000  # Increase to 60 seconds

Issue 3: Serialization Failures¶

Symptom:

Error: SerializationFailure("Read-write conflict")

Cause: Using SERIALIZABLE isolation with concurrent updates

Solution:

// Implement retry logic
for attempt in 0..3 {
    match manager.commit_transaction(session_id) {
        Ok(()) => break,
        Err(Error::SerializationFailure(_)) if attempt < 2 => {
            // Retry transaction
            continue;
        }
        Err(e) => return Err(e),
    }
}

Issue 4: Out of Memory¶

Symptom:

Error: memory limit exceeded for user

Cause: Per-user memory quota exceeded

Solution:

[resource_quotas]
memory_limit_per_user = 2147483648  # Increase to 2GB

Issue 5: Dump File Not Found¶

Symptom:

Error: No such file or directory: /backups/db.heliodump

Cause: Dump directory doesn't exist

Solution:

mkdir -p /backups
heliosdb-lite dump --output /backups/db.heliodump

FAQ¶

Q1: Do I need to change my code to upgrade to v3.1.0?¶

A: No. v3.1.0 is 100% backward compatible. Existing code works without changes.

Q2: Can I use v3.1.0 without session management?¶

A: Yes. Session management is optional. Use it only if you need multi-user features.

Q3: How do I migrate from v3.0.0 to v3.1.0?¶

A: 1. Download v3.1.0 binary 2. Replace v3.0.0 binary 3. No code changes required 4. Optionally adopt new features (sessions, dump/restore)

Q4: Are dump files from v3.1.0 compatible with future versions?¶

A: Yes. Dump format is version-stamped and designed for forward compatibility.

Q5: Can I downgrade from v3.1.0 to v3.0.0?¶

A: Yes, but: - Dump files (.heliodump) won't work in v3.0.0 - Session management code won't compile - Configuration sections [session], [locks], [dump], [resource_quotas] will be ignored

Q6: Does v3.1.0 require more memory than v3.0.0?¶

A: - Single-user mode: Same memory usage - Multi-user mode: +2-4 KB per session, +32 bytes per lock - Overall: Minimal increase (<5%) for typical workloads

Q7: How do I enable automated backups?¶

A:

[dump]
schedule = "0 2 * * *"  # Daily at 2 AM
dump_dir = "/var/backups/heliosdb"

Or via cron:

0 2 * * * heliosdb-lite dump --output /backups/daily.heliodump

Q8: Can I use v3.1.0 features with persistent mode?¶

A: Partial. Session management works in both modes. Dump/restore is primarily for in-memory mode (persistent mode already has WAL/RocksDB backups).

Q9: What isolation level should I use?¶

A: - READ COMMITTED: Default, best for most web applications - REPEATABLE READ: For reports requiring consistency - SERIALIZABLE: For financial transactions requiring strict consistency

Q10: How do I test my migration?¶

A: 1. Deploy v3.1.0 to staging environment 2. Run existing test suite (should pass 100%) 3. Test new features (dump/restore, sessions) if adopting 4. Monitor for errors in logs 5. Roll out to production gradually

Migration Checklist¶

[ ] Review what's new in v3.1.0
[ ] Verify no breaking changes affect your application
[ ] Download v3.1.0 binary or update dependency
[ ] Test in development environment
[ ] Update configuration (optional new sections)
[ ] Adopt new features (sessions, dump/restore) if needed
[ ] Update documentation and runbooks
[ ] Setup automated backups (if using in-memory mode)
[ ] Deploy to staging and verify
[ ] Deploy to production
[ ] Monitor for issues

Support¶

Issues: https://github.com/heliosdb/heliosdb/issues Discussions: https://github.com/heliosdb/heliosdb/discussions Documentation: https://docs.heliosdb.com/

Version: 3.1.0 Last Updated: 2025-12-08 Maintained by: HeliosDB Team