+
Skip to content

ankur-anand/unisondb

Repository files navigation

UnisonDB 🚀

UnisonDB

A Hybrid KV Store for Fast Writes, Efficient Reads, and Seamless Replication — with Explicit Transactions, LOB and Wide Column Support, Powered by Logs and Trees.

ci-tests Coverage Status

Overview

UnisonDB is a high-performance, replicated key-value store that blends the best of WALs, Memtables, and B-Trees to achieve:

  • Blazing-fast writes without LSM compaction overhead.

  • Optimized range queries with minimal disk I/O using B-Trees

  • Efficient replication via gRPC WAL streaming & B-Tree snapshots

  • Seamless multi-region scaling with rapid fail over.

  • LOB support via chunked, transactional writes for large object handling.

  • Flexible wide-column data modeling for dynamic, nested records

Why is Traditional KV Replication Insufficient?

Most traditional key-value stores were designed for simple, point-in-time key-value operations — and their replication models reflect that. While this works for basic use cases, it quickly breaks down under real-world demands like multi-key transactions, large object handling, and fine-grained updates.

Key-Level Replication Only

Replication is often limited to raw key-value pairs. There’s no understanding of higher-level constructs like rows, columns, or chunks — making it impossible to efficiently replicate partial updates or large structured objects.

No Transactional Consistency

Replication happens on a per-operation basis, not as part of an atomic unit. Without multi-key transactional guarantees, systems can fall into inconsistent states across replicas, especially during batch operations, network partitions, or mid-transaction failures.

Chunked LOB Writes Become Risky

When large values are chunked and streamed to the store, traditional replication models expose chunks as they arrive. If a transfer fails mid-way, replicas may store incomplete or corrupted objects, with no rollback or recovery mechanism.

No Awareness of Column-Level Changes

Wide-column data is treated as flat keys or opaque blobs. If only a single column is modified, traditional systems replicate the entire row, wasting bandwidth, increasing storage overhead, and making efficient synchronization impossible.

Operational Complexity Falls on the User

Without built-in transactional semantics, developers must implement their own logic for deduplication, rollback, consistency checks, and coordination — which adds fragility and complexity to the system.

Storage Engine Tradeoffs

• LSM-Trees (e.g., RocksDB) excel at fast writes but suffer from high read amplification and costly background compactions, which hurt latency and predictability.

• B+Trees (e.g., BoltDB,LMDB) offer efficient point lookups and range scans, but struggle with high-speed inserts and lack native replication support.

How UnisonDB Solves This. ✅

UnisonDB combines append-only logs for high-throughput ingest with B-Trees for fast and efficient range reads — while offering:

  • Transactional, multi-key replication with commit visibility guarantees.
  • Chunked LOB writes that are fully atomic.
  • Column-aware replication for efficient syncing of wide-column updates.
  • Isolation by default — once a network-aware transaction is started, all intermediate writes are fully isolated and not visible to readers until a successful txn.Commit().
  • Built-in replication via gRPC WAL streaming + B-Tree snapshots.
  • Zero-compaction overhead, high write throughput, and optimized reads.

Architecture Overview

storage architecture

Development

make lint
make test

certificate for Local host

brew install mkcert

## install local CA
mkcert -install

## Generate gRPC TLS Certificates
## these certificate are valid for hostnames/IPs localhost 127.0.0.1 ::1

mkcert -key-file grpc.key -cert-file grpc.crt localhost 127.0.0.1 ::1

License

Apache License, Version 2.0

About

Edge-First, Globally Replicated Multi-Column Key-Value Database. (WIP)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published
点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载