Skip to content
HeliosDB HeliosDB Full

Time-Series Features in HeliosDB

Time-Series Features in HeliosDB

Version: 6.0

Overview

HeliosDB provides enterprise-grade time-series data management capabilities designed for IoT, observability, financial analytics, and log processing workloads. The time-series engine delivers sub-millisecond query latency, 10x+ compression ratios, and throughput exceeding 1M+ points per second.

Key Capabilities

FeatureDescriptionPerformance
Native Time-Series StorageColumnar storage optimized for time-ordered dataZero-copy batch operations
Gorilla CompressionFacebook’s industry-standard compression algorithm10-15x compression ratio
Time-Based PartitioningHourly/daily/weekly/monthly/yearly partitionsPartition pruning for queries
Retention PoliciesAutomatic data expiration with TTL and size limitsBackground cleanup
DownsamplingMulti-tier aggregation with configurable intervalsPreserves statistical properties
Continuous AggregatesPre-computed rollups for fast analyticsReal-time materialization
Window FunctionsTumbling, sliding, and session windowsTime-based analysis
Gap FillingInterpolation strategies for missing dataForward/backward/linear fill

Architecture

                    HeliosDB Time-Series Architecture


    ┌──────────────────────────────────────────────────────────────────────┐
    │                        Time-Series API Layer                         │
    │   write_point() | query_range() | downsample() | set_retention()     │
    ├──────────────────────────────────────────────────────────────────────┤
    │                     High-Performance Ingestion                        │
    │  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────────┐  │
    │  │  Batching  │  │Out-of-Order│  │  Backfill  │  │  Compression   │  │
    │  │  Buffer    │  │  Handler   │  │  Support   │  │   Pipeline     │  │
    │  └────────────┘  └────────────┘  └────────────┘  └────────────────┘  │
    ├──────────────────────────────────────────────────────────────────────┤
    │                      Query Engine Layer                               │
    │  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────────┐  │
    │  │Time Range  │  │  Window    │  │ Time-Based │  │   Result       │  │
    │  │  Queries   │  │ Functions  │  │   Joins    │  │   Caching      │  │
    │  └────────────┘  └────────────┘  └────────────┘  └────────────────┘  │
    ├──────────────────────────────────────────────────────────────────────┤
    │                      Data Management Layer                            │
    │  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────────┐  │
    │  │ Retention  │  │Downsampling│  │ Partition  │  │   Tiered       │  │
    │  │  Engine    │  │  Engine    │  │  Manager   │  │   Storage      │  │
    │  └────────────┘  └────────────┘  └────────────┘  └────────────────┘  │
    ├──────────────────────────────────────────────────────────────────────┤
    │                      Compression Layer                                │
    │  ┌─────────────────────────────────────────────────────────────────┐ │
    │  │                    Gorilla Compressor                            │ │
    │  │  Delta-of-Delta (Timestamps) | XOR Bitpacking (Values)          │ │
    │  │  Dictionary Compression (Metrics/Tags)                           │ │
    │  └─────────────────────────────────────────────────────────────────┘ │
    ├──────────────────────────────────────────────────────────────────────┤
    │                      LSM Storage Engine                               │
    └──────────────────────────────────────────────────────────────────────┘

Native Time-Series Storage Engine

HeliosDB’s time-series storage is built on a columnar architecture that separates timestamps, values, and metadata for optimal compression and query performance.

Time-Bucketed Aggregations

HeliosDB supports automatic time-bucketing for analytical queries with multiple aggregation functions.

Aggregation Functions

FunctionDescription
AverageMean of all values in bucket
MinMinimum value
MaxMaximum value
SumSum of all values
CountNumber of data points
FirstFirst value in bucket
LastLast value in bucket
StdDevStandard deviation
Percentile(n)Nth percentile

Example: Time Bucket Query

-- Aggregate sensor readings by 5-minute buckets
SELECT
    time_bucket('5 minutes', timestamp) AS bucket,
    sensor_id,
    AVG(temperature) AS avg_temp,
    MAX(temperature) AS max_temp,
    MIN(temperature) AS min_temp
FROM sensor_readings
WHERE timestamp BETWEEN '2025-01-01' AND '2025-01-02'
GROUP BY bucket, sensor_id
ORDER BY bucket;

Downsampling and Retention Policies

Multi-Tier Downsampling

Configure cascading downsampling tiers to reduce storage while preserving analytical value.

Continuous Aggregates

Pre-compute rollups for commonly accessed time ranges.

Benefits

  • Query Performance: Pre-computed results eliminate runtime aggregation
  • Storage Efficiency: Aggregated data is smaller than raw data
  • Real-Time Updates: Continuous background processing
  • Late Data Handling: Configurable lag for out-of-order points

Time-Based Partitioning

Partition Strategies

StrategyPartition ID FormatUse Case
HourlyYYYYMMDDHHHigh-frequency data
DailyYYYYMMDDStandard metrics
WeeklyYYYYWWLow-frequency data
MonthlyYYYYMMLong-term storage
YearlyYYYYHistorical archives
Custom(secs)timestamp/intervalFlexible partitioning

Compression Performance

Gorilla Algorithm Results

Data TypeCompression RatioThroughput
IoT Temperature8-12x500K+ pts/sec
Network Metrics5-8x500K+ pts/sec
CPU/System Metrics5-10x500K+ pts/sec
High-Frequency Trading10-15x500K+ pts/sec

Compression Pipeline

  1. Delta-of-Delta Encoding (Timestamps)

    • Regular intervals compress to 1-4 bits per timestamp
    • 16-64x compression for uniformly sampled data

  2. XOR + Bit-packing (Values)

    • Exploits temporal correlation in values
    • 4-20x compression for slowly changing values

  3. Dictionary Compression (Metrics/Tags)

    • String to u32 ID mapping
    • 10-20x reduction for metric names

Window Functions

Window Types

  • Tumbling: Fixed-size, non-overlapping windows
  • Sliding: Fixed-size, overlapping windows
  • Session: Dynamic windows based on inactivity gap

Gap Filling and Interpolation

Fill Strategies

StrategyDescription
NullLeave gaps as null/None
ZeroFill with zero values
ForwardUse previous known value
BackwardUse next known value
LinearLinear interpolation between points

Time-Zone Handling

HeliosDB stores all timestamps in UTC and provides time-zone conversion at query time:

-- Query with timezone conversion
SELECT
    timestamp AT TIME ZONE 'America/New_York' AS local_time,
    value
FROM metrics
WHERE timestamp > NOW() - INTERVAL '24 hours';

Integration with MVCC

Time-series data integrates with HeliosDB’s Multi-Version Concurrency Control:

  • Point-in-Time Queries: Query data as it existed at any past moment
  • Consistent Snapshots: Transactional reads across time ranges
  • Conflict-Free Writes: Append-only model eliminates write conflicts
DocumentDescription
Quick Start GuideGet started in 10 minutes
ExamplesCode examples for common use cases

Performance Targets

Indicative figures, measured on representative fixtures; reproduce on your own hardware.

MetricTargetAchieved
Ingestion throughput1M pts/sec500K+ pts/sec
Compression ratio8-10x10-15x
Compression latency<5ms/1K pts<3ms/1K pts
Decompression latency<3ms/1K pts<2ms/1K pts
Query latency (time range)<10ms<5ms
Partition pruning95% reduction95%+

Use Cases

IoT and Sensor Networks

  • High-volume sensor data ingestion
  • Edge device telemetry
  • Industrial monitoring

Observability and Monitoring

  • Infrastructure metrics (CPU, memory, disk)
  • Application performance monitoring
  • Log aggregation and analysis

Financial Data

  • High-frequency trading ticks
  • Market data feeds
  • Risk analytics

Operational Analytics

  • Real-time dashboards
  • Anomaly detection
  • Trend analysis

See Also: HeliosDB Feature Index