Getting Started with HeliosDB-Lite¶

Version: 2.5.0-dev Last Updated: 2025-12-01 Estimated Time: 30 minutes

Table of Contents¶

Introduction
Installation
Your First Database
Embedded vs Server Mode
Quick Feature Tour
Next Steps

Introduction¶

What is HeliosDB-Lite?¶

HeliosDB-Lite is a PostgreSQL-compatible embedded database written in Rust that combines the simplicity of SQLite with the power of PostgreSQL. It's designed for developers who need a production-grade database without the operational overhead of managing a separate database server.

Key Characteristics: - Embedded: Runs directly in your application process (like SQLite) - PostgreSQL Compatible: 95%+ PostgreSQL 17 syntax compatibility - Dual Mode: Use as embedded library OR standalone server - Feature-Rich: Vector search, time-travel, branching, encryption - Zero Dependencies: Single binary, no external database server needed

Key Features Overview¶

Feature	Description	Use Case
Embedded Mode	In-process database, no network overhead	Desktop apps, mobile, IoT
Server Mode	PostgreSQL wire protocol server	Replace PostgreSQL for small-medium apps
Vector Search	Built-in semantic search with HNSW	AI/ML applications, RAG pipelines
Product Quantization	8-16x memory compression for vectors	Memory-efficient vector storage
Time-Travel	Query historical data at any point in time	Auditing, debugging, analytics
Branching	Git-like database branches	Safe schema changes, testing
Encryption (TDE)	AES-256-GCM transparent encryption	HIPAA, GDPR, SOC2 compliance
Materialized Views	Auto-refreshing pre-computed queries	Real-time dashboards, analytics

When to Use HeliosDB-Lite¶

Perfect for: - Desktop and mobile applications - Edge computing and IoT devices - Microservices with embedded databases - AI/ML applications with vector search - Applications requiring audit trails - Development and testing environments - Small to medium web applications

Not ideal for: - Multi-node clustering (use full HeliosDB) - Extremely high write throughput (>100K writes/sec) - Distributed transactions across nodes

Installation¶

Prerequisites¶

Rust: 1.75 or later
Operating System: Linux, macOS, or Windows
Disk Space: ~500 MB for build, ~50 MB for binary

Option 1: Build from Source (Recommended)¶

# Clone the repository
git clone https://github.com/dimensigon/HeliosDB-Lite
cd HeliosDB-Lite

# Build release binary
cargo build --release

# The binary will be at: ./target/release/heliosdb-lite
./target/release/heliosdb-lite --version

Expected Output:

heliosdb-lite 2.5.0-dev
PostgreSQL-compatible embedded database with vector search

Option 2: Install via Cargo¶

# Install from crates.io (when published)
cargo install heliosdb-lite

# Verify installation
heliosdb-lite --version

Option 3: Docker¶

# Pull the Docker image
docker pull heliosdb/lite:latest

# Run interactive REPL
docker run -it --rm heliosdb/lite:latest repl --memory

# Run server mode
docker run -p 5432:5432 heliosdb/lite:latest start --port 5432

Verify Installation¶

# Check version
heliosdb-lite --version

# Show help
heliosdb-lite --help

# Test REPL (in-memory database)
heliosdb-lite repl --memory

In the REPL, type:

CREATE TABLE test (id INT, name TEXT);
INSERT INTO test VALUES (1, 'Hello HeliosDB');
SELECT * FROM test;
\q

If you see the inserted data, your installation is working correctly!

Your First Database¶

Using the Interactive REPL¶

The REPL (Read-Eval-Print Loop) is the fastest way to explore HeliosDB-Lite.

Start an In-Memory Database¶

heliosdb-lite repl --memory

Output:

HeliosDB-Lite v2.5.0-dev
Type \h for help, \q to quit

heliosdb>

Create Your First Table¶

-- Create a users table
CREATE TABLE users (
    id INT PRIMARY KEY,
    name TEXT NOT NULL,
    email TEXT NOT NULL,
    age INT
);

Output: Query OK (0.002s)

Insert Data¶

-- Insert individual records
INSERT INTO users VALUES (1, 'Alice Johnson', 'alice@example.com', 30);
INSERT INTO users VALUES (2, 'Bob Smith', 'bob@example.com', 25);
INSERT INTO users VALUES (3, 'Charlie Brown', 'charlie@example.com', 35);

Output: Inserted 1 row (0.001s) (for each INSERT)

Query Data¶

-- Select all users
SELECT * FROM users;

-- Select with WHERE clause
SELECT name, email FROM users WHERE age > 28;

-- Aggregates
SELECT COUNT(*), AVG(age) FROM users;

-- Order and limit
SELECT * FROM users ORDER BY age DESC LIMIT 2;

Example Output:

 id | name          | email                | age
----|---------------|----------------------|-----
  1 | Alice Johnson | alice@example.com    |  30
  2 | Bob Smith     | bob@example.com      |  25
  3 | Charlie Brown | charlie@example.com  |  35

(3 rows, 0.003s)

Update and Delete¶

-- Update data
UPDATE users SET age = 31 WHERE name = 'Alice Johnson';

-- Delete data
DELETE FROM users WHERE age < 28;

-- Verify changes
SELECT * FROM users;

REPL Meta-Commands¶

HeliosDB-Lite supports PostgreSQL-style meta-commands:

\h                  -- Show help
\d                  -- List tables
\d users            -- Describe table 'users'
\dS                 -- List system views
\l                  -- List databases/branches
\q                  -- Quit REPL

Using a Persistent Database¶

To save data between sessions, use a persistent file:

# Create database in current directory
heliosdb-lite repl -d ./mydb.helio

# Or specify full path
heliosdb-lite repl -d /path/to/my/database.helio

All commands work the same, but data persists after quitting!

Basic SQL Operations Summary¶

Operation	SQL Syntax	Example
Create Table	`CREATE TABLE name (columns)`	`CREATE TABLE users (id INT, name TEXT)`
Insert	`INSERT INTO table VALUES (...)`	`INSERT INTO users VALUES (1, 'Alice')`
Select	`SELECT columns FROM table`	`SELECT * FROM users WHERE age > 25`
Update	`UPDATE table SET col = val`	`UPDATE users SET age = 31 WHERE id = 1`
Delete	`DELETE FROM table WHERE ...`	`DELETE FROM users WHERE age < 18`
Aggregates	`SELECT COUNT(*), SUM(col), AVG(col)`	`SELECT AVG(age) FROM users`
Join	`SELECT ... FROM t1 JOIN t2 ON ...`	`SELECT * FROM users JOIN orders ON users.id = orders.user_id`
Group By	`SELECT col, COUNT(*) GROUP BY col`	`SELECT region, SUM(sales) FROM orders GROUP BY region`

Embedded vs Server Mode¶

HeliosDB-Lite offers two deployment modes with different use cases.

Embedded Mode (Rust API)¶

Use When: You want to embed the database directly in your Rust application.

Benefits: - No network overhead (in-process) - Single binary deployment - Simplest setup - Maximum performance

Setup¶

Add to your Cargo.toml:

[dependencies]
heliosdb-lite = "2.5.0-dev"

Basic Example¶

use heliosdb_lite::{EmbeddedDatabase, Result};

fn main() -> Result<()> {
    // Create or open database
    let db = EmbeddedDatabase::new("./myapp.helio")?;

    // Create table
    db.execute("CREATE TABLE IF NOT EXISTS products (
        id INT PRIMARY KEY,
        name TEXT NOT NULL,
        price DECIMAL(10, 2)
    )")?;

    // Insert data
    db.execute("INSERT INTO products VALUES (1, 'Laptop', 999.99)")?;
    db.execute("INSERT INTO products VALUES (2, 'Mouse', 29.99)")?;

    // Query data
    let results = db.query("SELECT * FROM products WHERE price < 100", &[])?;

    for row in results {
        println!("{:?}", row);
    }

    Ok(())
}

In-Memory Database¶

For testing or temporary data:

use heliosdb_lite::{EmbeddedDatabase, Result};

fn main() -> Result<()> {
    // Create in-memory database
    let db = EmbeddedDatabase::new_in_memory()?;

    // Use normally - data lives only in RAM
    db.execute("CREATE TABLE temp (id INT, data TEXT)")?;
    db.execute("INSERT INTO temp VALUES (1, 'Temporary data')")?;

    Ok(())
    // Database is destroyed when 'db' goes out of scope
}

Transactions¶

use heliosdb_lite::{EmbeddedDatabase, Result};

fn transfer_funds(db: &EmbeddedDatabase, from: i32, to: i32, amount: i32) -> Result<()> {
    // Begin transaction
    let tx = db.begin_transaction()?;

    // Execute multiple operations
    tx.execute(&format!("UPDATE accounts SET balance = balance - {} WHERE id = {}", amount, from))?;
    tx.execute(&format!("UPDATE accounts SET balance = balance + {} WHERE id = {}", amount, to))?;

    // Commit (or rollback on error)
    tx.commit()?;

    Ok(())
}

Server Mode (PostgreSQL Protocol)¶

Use When: You want to connect from multiple applications or use PostgreSQL clients.

Benefits: - Language-agnostic (use any PostgreSQL client) - Multiple concurrent connections - Drop-in PostgreSQL replacement - Use with ORMs (SQLAlchemy, Diesel, etc.)

Start the Server¶

# Initialize database directory
heliosdb-lite init ./mydb

# Start server on default port (5432)
heliosdb-lite start --data ./mydb

# Start on custom port
heliosdb-lite start --port 5433 --data ./mydb

# Start with authentication
heliosdb-lite start --data ./mydb --auth-required

Output:

HeliosDB-Lite Server v2.5.0-dev
Listening on 127.0.0.1:5432
Database: ./mydb
Ready to accept connections

Connect with psql¶

# Connect with psql
psql postgresql://localhost:5432/mydb

# Or with authentication
psql postgresql://username:password@localhost:5432/mydb

Once connected, use standard PostgreSQL commands:

-- Create table
CREATE TABLE orders (
    id SERIAL PRIMARY KEY,
    customer_id INT NOT NULL,
    total DECIMAL(10, 2),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Insert data
INSERT INTO orders (customer_id, total) VALUES (1, 299.99);

-- Query
SELECT * FROM orders WHERE customer_id = 1;

Connect from Python¶

import psycopg2

# Connect to HeliosDB-Lite server
conn = psycopg2.connect(
    host="localhost",
    port=5432,
    database="mydb",
    user="postgres",  # default user
    password=""
)

# Create cursor
cur = conn.cursor()

# Execute query
cur.execute("SELECT * FROM users WHERE age > %s", (25,))
rows = cur.fetchall()

for row in rows:
    print(row)

# Cleanup
cur.close()
conn.close()

Connect from Node.js¶

const { Client } = require('pg');

// Connect to HeliosDB-Lite server
const client = new Client({
  host: 'localhost',
  port: 5432,
  database: 'mydb',
  user: 'postgres',
  password: ''
});

await client.connect();

// Execute query
const result = await client.query('SELECT * FROM users WHERE age > $1', [25]);
console.log(result.rows);

await client.end();

Comparison Table¶

Feature	Embedded Mode	Server Mode
Setup	Add Rust dependency	Start server process
Performance	Fastest (no network)	Network overhead
Concurrency	Single application	Multiple clients
Language Support	Rust only	Any PostgreSQL client
Deployment	Single binary	Server + clients
Use Case	Apps, mobile, IoT	Multi-app, web services

Quick Feature Tour¶

Now that you understand the basics, let's explore HeliosDB-Lite's unique features.

Vector Search¶

Store and search embeddings for semantic search, recommendations, and RAG pipelines.

Example: Semantic Document Search¶

-- Create table with vector column
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    title TEXT,
    content TEXT,
    embedding VECTOR(384)  -- 384-dimensional vector
);

-- Create HNSW index for fast similarity search
CREATE INDEX idx_docs_embedding ON documents
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 200);

-- Insert document with embedding (from your embedding model)
INSERT INTO documents (title, content, embedding) VALUES (
    'Introduction to Databases',
    'A database is an organized collection of data...',
    '[0.023, -0.145, 0.892, ...]'::VECTOR(384)
);

-- Semantic search: find similar documents
SELECT
    id,
    title,
    embedding <=> '[0.015, -0.120, 0.850, ...]'::VECTOR(384) AS distance
FROM documents
ORDER BY distance ASC
LIMIT 10;

With Product Quantization (8-16x compression):

-- Create index with PQ compression
CREATE INDEX idx_docs_pq ON documents
USING hnsw (embedding vector_cosine_ops)
WITH (
    m = 16,
    ef_construction = 200,
    quantization = 'pq',
    pq_subquantizers = 8,
    pq_centroids = 256
);

-- Same queries, but 8-16x less memory!
-- 768-dim vector: 3KB -> 8 bytes (384x compression)

Learn More: See Vector Search Tutorial (coming soon)

Time-Travel Queries¶

Access historical data at any point in time without manual snapshots.

Example: Audit Historical Changes¶

-- Create table
CREATE TABLE prices (
    product_id INT,
    price DECIMAL(10, 2),
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Insert initial price
INSERT INTO prices VALUES (1, 99.99, NOW());

-- Update price multiple times
UPDATE prices SET price = 89.99, updated_at = NOW() WHERE product_id = 1;
UPDATE prices SET price = 79.99, updated_at = NOW() WHERE product_id = 1;

-- Query current price
SELECT * FROM prices WHERE product_id = 1;
-- Returns: (1, 79.99, 2025-12-01 15:30:00)

-- Query price as of yesterday
SELECT * FROM prices
AS OF TIMESTAMP '2025-11-30 12:00:00'
WHERE product_id = 1;
-- Returns: (1, 99.99, 2025-11-30 10:00:00)

-- Compare current vs historical
SELECT
    current.price AS current_price,
    historical.price AS historical_price,
    current.price - historical.price AS price_change
FROM prices current
JOIN prices AS OF TIMESTAMP '2025-11-30 00:00:00' historical
  ON current.product_id = historical.product_id
WHERE current.product_id = 1;

Rust API:

use heliosdb_lite::{EmbeddedDatabase, Result};

fn main() -> Result<()> {
    let db = EmbeddedDatabase::new("./mydb.helio")?;

    // Query as of specific timestamp
    let historical = db.query_as_of_timestamp(
        "SELECT * FROM prices WHERE product_id = $1",
        &[1],
        "2025-11-30 12:00:00"
    )?;

    println!("Historical price: {:?}", historical);

    Ok(())
}

Learn More: See Time-Travel Guide

Database Branching¶

Git-like branches for safe schema changes and testing.

Example: Test Schema Changes¶

-- Create development branch
CREATE BRANCH dev FROM main;

-- Switch to dev branch
USE BRANCH dev;

-- Safely test schema changes
ALTER TABLE users ADD COLUMN preferences JSONB;
INSERT INTO users VALUES (999, 'Test User', 'test@example.com', 25, '{"theme": "dark"}');

-- Test your changes
SELECT * FROM users WHERE id = 999;

-- If satisfied, merge back to main
USE BRANCH main;
MERGE BRANCH dev INTO main;

-- Clean up
DROP BRANCH dev;

Rust API:

use heliosdb_lite::{EmbeddedDatabase, Result};

fn main() -> Result<()> {
    let db = EmbeddedDatabase::new("./mydb.helio")?;

    // Create branch
    db.create_branch("dev", Some("main"), None)?;

    // Switch to branch
    db.switch_branch("dev")?;

    // Make changes
    db.execute("ALTER TABLE products ADD COLUMN rating FLOAT")?;

    // Test changes...

    // Merge back
    db.switch_branch("main")?;
    db.merge_branch("dev")?;

    // Drop branch
    db.drop_branch("dev")?;

    Ok(())
}

Learn More: See Branching Guide

Materialized Views with Auto-Refresh¶

Pre-compute expensive queries for instant dashboard performance.

Example: Real-Time Dashboard¶

-- Create materialized view
CREATE MATERIALIZED VIEW sales_dashboard AS
SELECT
    DATE_TRUNC('day', order_date) AS day,
    COUNT(*) AS total_orders,
    SUM(amount) AS total_revenue,
    AVG(amount) AS avg_order_value
FROM orders
GROUP BY DATE_TRUNC('day', order_date);

-- Query is instant (no aggregation needed)
SELECT * FROM sales_dashboard
WHERE day = CURRENT_DATE;

-- Configure auto-refresh
ALTER MATERIALIZED VIEW sales_dashboard SET (
    auto_refresh = true,
    staleness_threshold_sec = 300,  -- Refresh if data is 5 min old
    cpu_threshold = 0.5              -- Only when CPU < 50%
);

-- Check staleness
SELECT * FROM pg_mv_staleness()
WHERE view_name = 'sales_dashboard';

Learn More: See Materialized Views Guide

Transparent Data Encryption¶

Protect sensitive data with AES-256-GCM encryption.

Example: Encrypted Database¶

use heliosdb_lite::{EmbeddedDatabase, Config, encryption::KeyDerivation};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create encrypted database
    let config = Config {
        encryption_key: Some("your-secure-32-byte-key-here!!!!".to_string()),
        key_derivation: KeyDerivation::Argon2 {
            memory_cost: 65536,
            time_cost: 3,
            parallelism: 4,
        },
        ..Default::default()
    };

    let db = EmbeddedDatabase::open_with_config("./encrypted.helio", config)?;

    // All operations are transparently encrypted
    db.execute("CREATE TABLE secrets (id INT, data TEXT)")?;
    db.execute("INSERT INTO secrets VALUES (1, 'Top secret information')")?;

    // Data is encrypted on disk, decrypted in memory
    let results = db.query("SELECT * FROM secrets", &[])?;
    println!("{:?}", results);

    Ok(())
}

Learn More: See Encryption Tutorial (coming soon)

Next Steps¶

Congratulations! You now know the basics of HeliosDB-Lite.

Recommended Learning Path¶

Core Skills (Start here)
Parameterized Queries - SQL injection protection
Transactions - ACID guarantees
Indexing - Performance optimization
Advanced Features (Explore next)
Vector Search Tutorial - Semantic search
Encryption Tutorial - Data protection
Materialized Views Tutorial - Analytics
Time-Travel Tutorial - Historical queries
Branching Tutorial - Git-like workflows
Production Deployment
Performance Tuning
Security Best Practices
Monitoring and Observability
Backup and Recovery

Example Projects¶

Explore complete example applications:

# Basic CRUD operations
cargo run --example quickstart

# Vector search with OpenAI embeddings
cargo run --example vector_search_demo

# Encryption demo
cargo run --example encryption_demo

# PostgreSQL server mode
cargo run --example pg_server

# Materialized views
cargo run --example mv_scheduler_demo

All examples are in the examples/ directory.

Documentation¶

User Guides: docs/guides/
Architecture: docs/architecture/
API Reference: docs.rs/heliosdb-lite
SQL Reference: docs/SQL_REFERENCE.md

Community and Support¶

GitHub Repository: dimensigon/HeliosDB-Lite
Issue Tracker: Report bugs
Discussions: Ask questions
Discord: Coming soon

Quick Reference Card¶

Common Commands:

-- Database operations
CREATE TABLE name (columns);
INSERT INTO table VALUES (...);
SELECT * FROM table WHERE condition;
UPDATE table SET col = val WHERE condition;
DELETE FROM table WHERE condition;

-- Advanced features
CREATE INDEX idx ON table USING hnsw (vector_col);
CREATE BRANCH branch FROM main;
CREATE MATERIALIZED VIEW mv AS SELECT ...;
SELECT * FROM table AS OF TIMESTAMP '...';

-- System introspection
\d                             -- List tables
\d table                       -- Describe table
SELECT * FROM pg_database_branches();
SELECT * FROM pg_mv_staleness();
SELECT * FROM pg_vector_index_stats();

Common Rust Patterns:

// Open database
let db = EmbeddedDatabase::new("./db.helio")?;

// Execute query
db.execute("INSERT INTO users VALUES (1, 'Alice')")?;

// Query with results
let results = db.query("SELECT * FROM users", &[])?;

// Transactions
let tx = db.begin_transaction()?;
tx.execute("UPDATE ...")?;
tx.commit()?;

// Time-travel
db.query_as_of_timestamp("SELECT ...", &[], "2025-12-01")?;

// Branching
db.create_branch("dev", Some("main"), None)?;
db.switch_branch("dev")?;

Troubleshooting¶

Common Issues¶

Issue: cargo build fails with "linker not found" Solution: Install build tools:

# Linux
sudo apt-get install build-essential

# macOS
xcode-select --install

Issue: REPL shows "Permission denied" when creating database Solution: Check directory permissions:

chmod 755 /path/to/database/directory

Issue: Server mode shows "Address already in use" Solution: Port 5432 is already taken. Use a different port:

heliosdb-lite start --port 5433 --data ./mydb

Issue: Vector queries are slow Solution: Create an HNSW index:

CREATE INDEX idx ON table USING hnsw (vector_col);

Getting Help¶

If you encounter issues:

Check the Known Issues document
Search existing GitHub Issues
Ask in GitHub Discussions
Open a new issue with:
HeliosDB-Lite version (heliosdb-lite --version)
Rust version (rustc --version)
Operating system
Minimal reproducible example

Appendix: Full Example Application¶

Here's a complete example application that demonstrates multiple features:

use heliosdb_lite::{EmbeddedDatabase, Result};

fn main() -> Result<()> {
    // Create database
    let db = EmbeddedDatabase::new("./app.helio")?;

    // Create schema
    db.execute("
        CREATE TABLE IF NOT EXISTS users (
            id SERIAL PRIMARY KEY,
            name TEXT NOT NULL,
            email TEXT NOT NULL UNIQUE,
            created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
        )
    ")?;

    db.execute("
        CREATE TABLE IF NOT EXISTS posts (
            id SERIAL PRIMARY KEY,
            user_id INT NOT NULL,
            title TEXT NOT NULL,
            content TEXT,
            embedding VECTOR(384),
            created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
        )
    ")?;

    // Create indexes
    db.execute("CREATE INDEX IF NOT EXISTS idx_posts_user ON posts(user_id)")?;
    db.execute("
        CREATE INDEX IF NOT EXISTS idx_posts_embedding ON posts
        USING hnsw (embedding vector_cosine_ops)
        WITH (m = 16, ef_construction = 200)
    ")?;

    // Insert data
    let tx = db.begin_transaction()?;
    tx.execute("INSERT INTO users (name, email) VALUES ('Alice', 'alice@example.com')")?;
    tx.execute("INSERT INTO users (name, email) VALUES ('Bob', 'bob@example.com')")?;
    tx.commit()?;

    // Query with join
    let results = db.query("
        SELECT u.name, COUNT(p.id) as post_count
        FROM users u
        LEFT JOIN posts p ON u.id = p.user_id
        GROUP BY u.name
    ", &[])?;

    for row in results {
        println!("{:?}", row);
    }

    // Create materialized view for analytics
    db.execute("
        CREATE MATERIALIZED VIEW IF NOT EXISTS user_stats AS
        SELECT
            u.id,
            u.name,
            COUNT(p.id) as total_posts,
            MAX(p.created_at) as last_post_at
        FROM users u
        LEFT JOIN posts p ON u.id = p.user_id
        GROUP BY u.id, u.name
    ")?;

    // Query materialized view
    let stats = db.query("SELECT * FROM user_stats", &[])?;
    println!("User stats: {:?}", stats);

    Ok(())
}

Run this example:

cargo run --example full_app

Congratulations! You're now ready to build production applications with HeliosDB-Lite.

For more advanced topics, explore the documentation or try the tutorials.

Document Version: 1.0 Feedback: Report issues or suggest improvements