Redis vs Aerospike

Redis is a single instance, single-threaded in-memory data structure store that can be used as a database, cache, message broker, and streaming engine. This approach delivers good performance on a single in-memory instance, but clustering is only supported with an add-on process called the Redis cluster proxy (from Redis Enterprise).

Redis 6.0 added multi-threading of network I/O, providing some parallel processing of data ingest. By default, multi-threading is turned off. This limited support of multi-threading excludes many of the advantages of multi-core processors.

Aerospike is a distributed, multi-threaded database. It is engineered to get the most out of compute, network, and I/O resources. 

Aerospike focuses on the minute details of CPU, shared memory, processor cache, and NVMe. 

Its Hybrid Memory Architecture™ (HMA) enables the use of flash storage (SSD, PCIe, NVMe) in parallel to perform reads at sub-millisecond latencies at very high throughput (100K to 1M+ TPS), even under  heavy write loads. This enables enormous vertical scaleup at a 5x lower total cost of ownership (TCO) than pure RAM.

Aerospike bypasses the operating system’s file system and directly utilizes a flash device as a block device using a custom data layout.

Aerospike uses multi-threading extensively to achieve maximum parallelism for all major functions and exploits the power of modern multi-core processors.

Redis is a key-value store where data is stored as key-value pairs. There is no notion of records, logical records, or bins. That means that any data operation must be performed on individual key-value pairs. Redis Enterprise supports additional data models, such as document, spatial, search, time series, and vector.

However, the limitations of Redis’ key-value implementation of different data models pose ongoing scalability and latency challenges.

Aerospike distributes and stores  sets of records contained in namespaces (akin to “databases”). Each record has a key and named fields (“bins”). A bin can contain different types of data from the simple (e.g., integer, string) to the complex (e.g., nested data in a list of maps of sets).

This provides considerable schema flexibility.

Aerospike supports fast processing of Collection Data Types (CDTs) which contain any number of scalar data type elements and nesting elements such as lists and maps. Nested data types can be treated exactly as a document.

Aerospike’s structure enables users to model, store, and manage key-value data, JSON documents, and graph data with high performance at scale.

Redis works well as a cache because of in-memory performance. It is offered solely as a caching product on Azure (Azure Cache for Redis). AWS offers Amazon ElastiCache for Redis as a fully managed caching service.

LRU eviction relies on sampling based methods, which are less precise.

Flexible configuration options enable Aerospike to act as 

(1) a high-speed cache to an existing relational or non-relational data store to promote real-time data access and offload work from the back end 

or 

(2) an ultra-fast real-time data management platform with persistence. 

Aerospike can store all data and indexes in DRAM, all data and indexes on SSDs (Flash), or a combination of the two (data on SSDs and indexes in DRAM).

7.1 adds precise LRU eviction, providing an efficient persistence of record read activity using a default read percent that will extend the record TTL accordingly.

Redis database RDB (persistence) is a single instance in-memory database. Redis Enterprise provides the Redis cluster proxy to make a group of RDB instances into a shared-nothing cluster.

When a client request comes in, it is up to the proxy to decide which server has the desired data, and it forwards the request to that server.

This adds significant latency to each transaction and is therefore a significant factor in scaling workloads.

Aerospike was designed from the outset as a distributed database. All nodes are aware of each other. 

Aerospike features a Smart Client™ that automatically distributes both data and traffic to all the nodes in a cluster. 

Automatic client load balancing improves both performance and correctness. This ensures a single hop to data for the lowest possible latencies.

Redis is an in-memory data store, meaning that data is not automatically stored in a non-volatile medium like SSDs. This means that should a Redis server fail, there is potential for data loss.

Redis offers optional persistence methods through database snapshotting or append-only files (AOF) in a file system on disk. 

Snapshots (RDB) are problematic when data loss can’t be tolerated. If you are snapshotting your database every hour, a node failure could cost you an hour's worth of transactions. You can configure the AOF with a setting to minimize data loss, which negatively impacts performance and scalability.

Designed as an operational distributed database, Aerospike employs a specialized log-structured file system that optimizes using flash drives (SSDs) as primary data storage without heavy dependence on RAM for performance.

Aerospike uses SSDs as raw devices, employing a proprietary log-structured file system rather than relying on file system, block, and page cache layers for I/O.  This provides distinct performance and reliability advantages. Aerospike uses raw-device block writes and lightweight defragmentation.

Firms can choose from hybrid memory (indexes-only in DRAM and data on Flash), all DRAM (i.e. in-memory), all-flash, and as of Aerospike 7.1, support for NVMe-compatible, low-cost cloud block storage, and common enterprise networked attached storage (NAS).

As described above, Redis was not designed as a distributed database.  The Redis cluster proxy is an add-on process to the running RDB instance. Each RDB instance is separate and stand-alone. 

Data sharded across a Redis “cluster” must first be routed to the correct node, and then the operation is executed on the RDB instance.

Aerospike clients (Smart Clients) are aware of the data distribution across the cluster; therefore, they can send their requests directly to the node responsible for storing the data. This reduces the network hops required, improving the database's performance.

Aerospike’s Smart Client layer maintains a dynamic partition map that identifies the primary node for each partition. This enables the client layer to route read or write requests directly to the correct nodes without any additional network hops.

Since Aerospike writes synchronously to all copies of the data, there is no delay for a quorum read across the cluster to get a consistent version of the data.

Redis’ largely single-threaded operation negates the benefits of multi-core processors, so scaling up is limited.

Scaling out with Redis (i.e., adding cluster nodes) is a labor-intensive process, where data must be resharded when nodes are added. Redis provides some automation of the resharding of data.

Aerospike handles massive customer growth without having to add many nodes based on its SSD-friendly Hybrid Memory Architecture and flexible configuration options.

Data distribution

Aerospike distributes data across cluster nodes automatically. When a node joins or leaves the cluster, the data is automatically redistributed.

Aerospike automatically shards data into 4,096 logical partitions evenly distributed across cluster nodes. When cluster nodes are added, partitions from other cluster nodes are automatically migrated to the new node,  resulting in very little data movement.

Vertical scaling

Aerospike exploits SSDs, multi-core CPUs, and other hardware and networking technologies to scale vertically, making efficient use of these resources. You can scale by adding SSDs. (There is no theoretical upper limit for the amount of resources that can be added to a single node.)

Horizontal scaling

The data distribution is random; therefore, scaling an Aerospike cluster in and out results in less data movement. Also, Aerospike follows a peer-to-peer architecture, meaning that no node has a special role. In this architecture, the load gets equally distributed across all the cluster nodes.

Redis has supported a number of ad hoc replication mechanisms, but none guaranteed anything stronger than causal consistency. RedisRaft aims to bring strict serializability to Redis through the Raft consensus algorithm.

Redis supports eventual consistency and has invented consistency terms called “near strong consistency” and “strong eventual consistency.” Eventual consistency is weak consistency.

Redis documentation states: “Redis Cluster does not guarantee strong consistency. In practical terms, this means that under certain conditions, it is possible that Redis Cluster will lose writes that were acknowledged by the system to the client.”

RedisRaft is a Redis module that implements the Raft Consensus Algorithm, making it possible to create strongly consistent clusters of Redis servers. The Raft algorithm is provided by a standalone Raft library.

Aerospike provides distinct high availability (AP) and (strong consistency) (CP) modes to support varying customer use cases.

The independent Jepsen testing in 2018 validated Aerospike’s claim of strong consistency. Strong consistency mode prevents stale reads, dirty reads, and data loss.

With strong consistency, each write can be configured for linearizability (provides a single linear view among all clients) or session consistency (an individual process sees the sequential set of updates).  

Each read can be configured for linearizability, session consistency, allow replica reads (read from master or any replica of data), and allow unavailable responses (read from the master, any replica, or an unavailable partition).

Aerospike’s roster-based consistency algorithm requires only N+1 copies to handle N failures. Aerospike automatically detects and responds to many network and node failures to ensure high availability of data without requiring operator intervention.

High Availability (AP)/partition tolerant mode emphasizes data availability over consistency in failure scenarios.

Modes and consistency levels can be defined at the namespace level (database level).

Redis Sentinel enables you to monitor the state of your Redis cluster. It will alert you if a master node fails and helps you manage automated failover. 

Sentinel is not scalable as  it is not a clustering solution, so all writes go to the master. Also, it does not support sharding.

Redis Cluster is a clustering solution but does not have robust  HA features or strong consistency.

Aerospike users typically maintain replication factor two (RF2) (one primary, one replica copy) for high availability. 

Aerospike automatically detects and responds to many network and node failures (“self-healing”) to ensure high availability of data, prevent data loss or performance degradations without requiring operator intervention.

Redis has  a primary-replica topology, which supports multiple replicas from a primary instance. Redis replicas are read-only, so all writes must be done on the primary, negatively impacting  performance and scalability. Only asynchronous replication is supported.

Redis supports Redis Active-Active Geo-Distribution, which implements conflict-free replicated data types (CRDTs) that do not provide strong consistency. CRDTs are relatively unproven in enterprise production environments.

Supports multi-site deployments for varied business purposes: Continuous operations, fast localized data access, disaster recovery, global transaction processing, edge-to-core computing, and more. 

Asynchronous active-active replication (via Cross Datacenter Replication, XDR): is achieved in sub-millisecond or single-digit milliseconds. All or part of the data in two or more independent data centers gets replicated asynchronously. The replication can be one way or two ways.

The clients can read and write from the data center close to them.

The expected lag between data centers is in the order of a few milliseconds. Optimizations to minimize transfer of frequently updated data.

XDR also supports selective replication (i.e., data filtering) and performance optimizations to minimize transfer of frequently updated data.

Synchronous active-active replication (via multi-site clustering): A single cluster is formed across multiple data centers. Achieved in part via rack awareness, pegging primary and replica partitions to distinct data centers. Automatically enforces strong data consistency.

The clients can read data from the node close to them; the expected latency will be less than a millisecond. However, the write requests may need to be written in a different data center, which may increase the latency to a few hundred milliseconds.

ODBC/JDBC and SQL (CData) Spark-Redis (open source) Tanzu (VMware), Cloud Foundry, Nagios, and Trino (via Trino website).

Performance-optimized connectors for Aerospike are available for many popular open source and third-party offerings,  including Kafka, Spark, Presto-Trino, JMS, Pulsar, Event Stream Processing (ESP), and Elasticsearch. These connectors, in turn, provide broader access to Aerospike from popular enterprise tools for business analytics, AI, event processing, and more.

Redis offers optional persistence methods through database snapshotting or append-only files (AOF) in a file system on disk. 

Snapshots are problematic when data loss can’t be tolerated. If you are snapshotting your database every hour, a node failure could cost you an hour's worth of transactions. You can configure the AOF with a setting to minimize data loss, but this negatively impacts performance and scalability.

Persisting to another database such as rocksDB or Speedb adds overhead to Redis nodes or extra server nodes, adding to operating costs and latency.

Flexible configuration options enable Aerospike to act as 

(1) an ultra-fast real-time data management platform with persistence. 

(2) a high-speed cache to an existing relational or non-relational data store to promote real-time data access and offload work from the back end.

Aerospike can store all data and indexes in DRAM, all data and indexes on SSDs (Flash), or a combination of the two (data on SSDs and indexes in DRAM).

Redis Data Integration (RDI) creates a data streaming pipeline that mirrors data from an existing database to Redis Enterprise. It is a separate product/process from Redis Enterprise.

The Redis integrated Change Data Capture (CDC) capability tracks the updates in the command database transaction log, transforms row-level change data into a Redis data structure, and replicates the updates to the query database.

Change Data Capture (CDC) is an Aerospike cluster feature. 

Granular options for capturing (and replicating) changed data, ranging from full namespaces (databases) to subsets of select records. 

Aerospike logs minimal information about each change  (not the full record), batching  changed data and shipping only the latest version of a record. 

Thus, multiple local writes for one record generate only one remote write – an important feature for “hot” data. 

Aerospike also provides Change Data notifications to external systems, like Kafka or other databases.

Multi-tenancy relies on sharding as a method to isolate data in Redis. You can assign multiple shards to a database to meet any data set size or throughput requirements. You may enable persistence, replication, eviction policy, and flash as a RAM extension at the database level.

A shard is an open source Redis instance. Redis, being a single-threaded process, runs on one CPU core.

Aerospike’s key features for multi-tenancy are separate namespaces (databases), role-based access control in conjunction with sets (akin to RDBMS tables), operational rate quotas, and user-specified storage limits to cap data set size.

Redis makes no claims of optimizations for hardware platforms. It relies on in-memory performance to deliver low-latency response.

Redis is predominantly single-threaded by design, though muti-threaded network I/O was added in Redis 6 (turned off by default).

Aerospike is designed and implemented explicitly to exploit advances in modern hardware to maximize runtime performance and cost efficiency.  

Aerospike is massively multi-threaded to get the most from today’s multi-core processors.

NVMe, Flash, and SSDs are treated as raw block devices to reduce I/O for the lowest latency, avoiding overhead from standard storage drivers and file systems. Aerospike data structures are partitioned with fine-grained locks to avoid memory contention for more efficient use of multi-core CPUs. Application Device Queues (ADQs) are used with certain networking devices to reduce context switching and keep data in local processor caches.