HNSW Performance Tuning Guide

HNSW Performance Tuning Guide

Overview

This guide provides detailed performance tuning strategies for achieving production-grade performance with HeliosDB vector search.

Target Metrics

Indicative figures, measured on representative fixtures; reproduce on your own hardware.

• QPS: 10,000+ queries per second (1M vectors, ef=50)
• Recall: 95%+ at Recall@10
• Latency: p50 < 5ms, p95 < 20ms, p99 < 50ms
• Build Speed: 1,000+ vectors/sec
• Memory: ~1.6 GB per 1M vectors (128D, M=16)

Parameter Optimization

M (Max Connections)

Controls graph connectivity and search quality.

M	Recall@10	QPS	Memory	Build Speed	Use Case
4	85%	30k	0.7 GB	2,500/s	Memory-constrained
8	90%	20k	1.0 GB	2,000/s	Fast queries
16	95%	12k	1.6 GB	1,200/s	Production (balanced)
32	98%	8k	2.8 GB	800/s	High accuracy
64	99%	5k	5.2 GB	500/s	Maximum recall

Recommendation: Start with M=16, increase to 32 if recall < 95%

ef_construction (Build Quality)

Controls graph quality during index construction.

ef_construction	Recall@10	Build Time	Use Case
50	88%	1.0×	Fast development
100	92%	1.5×	Rapid prototyping
200	95%	2.0×	Production (balanced)
400	97%	3.0×	High accuracy
800	98%	5.0×	Maximum quality

Recommendation: Use 200 for production, 400 for mission-critical applications

ef (Search Quality)

Runtime parameter controlling recall vs speed tradeoff.

ef	Recall@10	Latency	QPS	Use Case
10	75%	0.3 ms	35k	Ultra-fast
20	85%	0.5 ms	25k	Fast search
50	95%	0.8 ms	15k	Production (balanced)
100	97%	1.2 ms	10k	High recall
200	98%	2.0 ms	6k	Maximum recall
500	99%	5.0 ms	3k	Research/evaluation

Recommendation: Start with ef=50, adjust based on recall requirements

SIMD Optimization

CPU Feature Detection

HeliosDB automatically selects the best SIMD implementation (AVX-512, AVX2, or a scalar fallback for all CPUs).

Performance Comparison

512-dimensional vectors, 10,000 distance calculations:

Implementation	Time	Speedup	Instructions
Scalar	5.0 ms	1.0×	~500 per calc
AVX2	0.8 ms	6.2×	~80 per calc
AVX-512	0.45 ms	11.1×	~45 per calc

Enabling SIMD

Linux:

# Check CPU features
cat /proc/cpuinfo | grep flags | grep avx512

# Build with native CPU features
RUSTFLAGS="-C target-cpu=native" cargo build --release

Docker:

# Use host CPU features
docker run --cpus=4 --cpu-shares=1024 \
  -e RUSTFLAGS="-C target-cpu=native" \
  heliosdb/vector-search

Concurrent Query Optimization

Thread Scaling

HNSW provides excellent read concurrency:

Threads	QPS	Latency (p50)	Latency (p95)	CPU Usage
1	12,000	0.8 ms	1.5 ms	100%
2	23,000	0.9 ms	1.8 ms	200%
4	44,000	1.0 ms	2.2 ms	400%
8	80,000	1.2 ms	3.0 ms	800%
16	140,000	1.5 ms	4.5 ms	1600%

Scaling efficiency: ~90% up to core count

Memory Optimization

Memory Usage Formula

Total Memory = Base + (N × Vector Size) + (N × Graph Memory)

Where:
- Base: ~100 MB (runtime overhead)
- Vector Size: D × 4 bytes (f32)
- Graph Memory: M × 8 bytes (per connection)

Example (1M vectors, 128D, M=16):
= 100 MB + (1M × 128 × 4) + (1M × 16 × 8)
= 100 MB + 512 MB + 128 MB
= 740 MB (actual: ~800 MB with overhead)

Memory-Constrained Strategies

1. Reduce M parameter: M=8 uses about half the memory of M=16
2. Disk-based storage (memory-mapped): vectors are loaded on demand from disk
3. Distributed sharding: for example, shard across 4 nodes, each handling 25% of the data

Filtered Search Optimization

Pre-filtering vs Post-filtering

Pre-filtering (apply filter before vector search):

• Pros: Fewer distance calculations, faster
• Cons: May miss results if filter too restrictive
• Use when: Filter selectivity > 10%

Post-filtering (apply filter after vector search):

• Pros: Better recall, no missed results
• Cons: More distance calculations
• Use when: Filter selectivity < 10%

Performance Impact

Filter Selectivity	Pre-filter Latency	Post-filter Latency	Recommendation
90% (10% filtered out)	0.9 ms	1.2 ms	Pre-filter
50% (50% filtered out)	1.5 ms	2.0 ms	Pre-filter
10% (90% filtered out)	8.0 ms	3.5 ms	Post-filter
1% (99% filtered out)	50 ms	4.0 ms	Post-filter

Build Optimization

Batch Insertion

Insert vectors in batches for better cache locality.

Build Performance

1M vectors, 128D, M=16, ef_construction=200:

Strategy	Build Time	Throughput
Sequential	850s	~1,175 vectors/s
Batch (1k)	820s	~1,220 vectors/s

Query Latency Optimization

Latency Breakdown

For 1M vectors, 128D, M=16, ef=50:

Phase	Latency	% Total
Distance calculations	0.5 ms	62%
Graph traversal	0.2 ms	25%
Result sorting	0.1 ms	13%
Total	0.8 ms	100%

Optimization Strategies

1. Reduce distance calculations: lower the ef parameter (for example ef=20 is about 3x faster, with about 10% lower recall)
2. Use SIMD: ensure AVX2/AVX-512 is enabled
3. Warm up cache: pre-warm with representative queries
4. Reduce k (top-k results): returning fewer results (for example k=5) is faster than k=100

Recall Optimization

Recall@10 by Configuration

M	ef_construction	ef	Recall@10	QPS
8	100	20	82%	25k
16	200	50	95%	12k
32	400	100	98%	7k
64	800	200	99%	4k

Strategies for >99% Recall

1. Increase build quality: for example M=64, ef_construction=800
2. Increase search width: for example ef=500
3. Use exact search for small datasets (<10k vectors): 100% recall, slower
4. Verify vector normalization (for Cosine)

Hybrid Search Optimization

Score Fusion Tuning

Weight text matching more heavily for text-heavy queries (e-commerce, documents) and vector similarity more heavily for vector-heavy queries (image search, embeddings). For balanced workloads, A/B test the weights.

Reranking Strategy

Two-stage retrieval (fast approximate search, then accurate reranking of a larger candidate set, for example fetch 100 and return the top 10) costs about 1.5x latency for +2-3% recall.

Production Monitoring

Key Metrics

Track query latency, QPS and index memory usage, and alert when latency exceeds your target (for example 50ms).

Troubleshooting Performance Issues

Issue: Low QPS

Symptoms: <5,000 QPS on modern CPU

Checklist:

• Verify SIMD enabled (RUSTFLAGS="-C target-cpu=native")
• Check CPU frequency (not throttled)
• Reduce ef parameter (200 → 50)
• Reduce M parameter (32 → 16)
• Verify no disk I/O (index in memory)
• Check for lock contention (profiling)

Issue: High Latency

Symptoms: p95 > 100ms

Checklist:

• Reduce ef (100 → 50 → 20)
• Check index size (>1M vectors may need sharding)
• Verify no memory swapping
• Check GC pauses (if using managed language wrapper)
• Profile hot paths

Issue: Low Recall

Symptoms: Recall@10 < 90%

Checklist:

• Increase ef (50 → 100 → 200)
• Increase ef_construction (200 → 400)
• Increase M (16 → 32)
• Verify correct distance metric
• Check vector normalization (for Cosine)
• Validate test data quality

Issue: High Memory Usage

Symptoms: >3 GB for 1M vectors (128D)

Checklist:

• Check M parameter (should be ≤32)
• Verify no memory leaks
• Check for duplicate indices
• Consider memory-mapped storage
• Profile memory allocations

Advanced Tuning

Query-Specific ef

Adjust ef based on query difficulty.

Use a low ef (for example 20) for easy, high-confidence queries and a higher ef (for example 200) for hard, ambiguous queries.

Summary

Production Configuration (1M vectors, 128D): M=16, ef_construction=200, Cosine distance for normalized embeddings, k=10 and ef=50. Expected: 10-15k QPS, 95%+ recall, <5ms p95 latency.

Key Takeaways:

1. Start with defaults (M=16, ef_construction=200, ef=50)
2. Enable SIMD for 5-10x speedup
3. Use pre-filtering for high selectivity (>10%)
4. Monitor recall, QPS, and latency continuously
5. A/B test parameter changes in production