DynamoDB vs. Aerospike

DynamoDB distributes and stores tables of items (rows). Each item can have a primary key, a sort key, and one or more attributes (named fields).

Each attribute can contain different types of data, from scalar (integer, string) to complex (lists and maps with up to 32 levels of nesting).

Each item has a size limit of 400KB. Modeling data with vertical partitioning is recommended as the “solution to the large item quandary” for items that exceed 400KB.

With DynamoDB, users can model, store, and manage key-value data and JSON documents with high performance at scale. AWS recommends Neptune as a graph database, although some users have developed modeling techniques and technologies to use DynamoDB as a backend for graph data storage. AWS recommends OpenSearch Service for vector storage and search.

Primary access pattern is key-value, with value types including string, integer, double, blob/byte, boolean, lists (ordered or unordered), and maps (unordered, key-ordered, or value-ordered), HyperLogLog, and geospatial.

Aerospike allows multiple different data types to be stored under a single key using separate bins (analogous to columns in relational databases), providing additional flexibility.

Aerospike supports declarative secondary indexing on individual bins or on entries in a list or map.

DynamoDB is a fully automated database-as-a-service (DBaaS) for AWS only.

Aerospike can be deployed 

• On-premises

• As a cloud-managed service on AWS, Azure, or GCP

• In hybrid configurations

• With the type: entry-hyperlink id: 37YzjBztIfEYstZhhnf1He (AKO), which supports Amazon Elastic Kubernetes Service (EKS), Microsoft Azure Kubernetes Service (AKS), Google Kubernetes Service (GKS), and local deployments.

DynamoDB is an AWS database service that distributes and stores data partitions in one or more cloud regions. 

Data is stored in a B-tree structure, and SSDs are used to persist these data structures.

A single engine covers all-RAM, all-NVMe, and hybrid configurations. In NVMe-based options, including hybrid, durability is a native property of the storage engine, not an add-on. Records are written to NVMe as part of the normal write path, and a write can be configured to commit to the device before it is acknowledged, so an acknowledged write survives a node restart or power loss, with no snapshot interval during which recent writes could be lost. On restart, a node comes back from its persisted data quickly rather than replaying a large log to rebuild an in-memory dataset.

DynamoDB maintains multiple request router instances that perform user authentication and authorization and consult a partition metadata component to identify all partition replicas. Depending on the nature of the request (read or write) and the desired data consistency level, the router will forward the request to the replica, serving as the partition leader or follower. 

Beyond basic CRUD capabilities, the Aerospike Smart Clients handle a great deal on the application's behalf: they track cluster state and refresh their partition map as nodes join, leave, or fail, transparently retry and fail over to a replica on timeouts or node loss, manage connection pooling and load balancing across the cluster, and honor policies such as rack-aware reads.

Coverage extends well beyond the officially supported set. PHP and Ruby have Aerospike-maintained, community-supported clients; Spring Data Aerospike provides an idiomatic Spring layer over the Java client; and community drivers exist for languages such as Elixir, Erlang, and Perl.

Designed for distributed computing on AWS, DynamoDB automatically hashes data into partitions to promote even data distribution throughout the cluster. Items are hashed on unique key values (e.g., a single primary key or composite of the primary key and a sort key). DynamoDB implements a consistent hashing technique to minimize data movement when partitions are added or removed. No user involvement is required.

Horizontal scaling is achieved with a separate AWS Application Auto Scaling service, which allows users to set policies on DynamoDB tables and global secondary indexes to automatically adjust provisioning based on consumption. No user involvement is needed after policies are set.

Vertical scaling is not within user control.

Aerospike distributes data across logical partitions evenly spread over nodes; adding or removing a node migrates only the partitions that must move, minimizing data movement. It also scales vertically by exploiting NVMe, multi-core CPUs, and fast networking, with a peer-to-peer architecture in which no node has a special role.

DynamoDB charges for reading, writing, and storing data. Optional services (such as backup/restore, data import/export, and others) incur additional fees.

The platform can be configured in on-demand capacity mode or provisioned capacity mode. With on-demand, charges are based on read and write request units (RRUs, WRUs) consumed by the user workload. AWS automatically ramps up or down in response. Users do not specify throughput. With provisioned capacity, users specify the number of RRUs and WRUs per second they expect to need and are charged accordingly. Auto scaling can be used in provisioned mode to address cost and performance concerns.

With either mode, a single application read or write command may require multiple RRUs or WRUs, depending on item size, data consistency level, transactional aspects, and other factors. 

Aerospike charges primarily by unique production data volume and for additional features. On top of that, there is no charge for operations; there are no charges based on the number of servers or cores.

Additional features typically involve a fixed percentage uplift based on your unique production data volume. Features are individual and not bundled unless they are part of a service and are stated explicitly.

Low TCO at scale is one of the most common reasons that customers choose Aerospike.

Each partition in DynamoDB is replicated across three servers (one leader, one synchronous follower, and one asynchronous follower). DynamoDB automatically detects and responds to many network and node failures to ensure high data availability without user intervention.

Aerospike automatically detects and responds to many network and node failures to ensure high availability of data without requiring operator intervention.

Aerospike's recommended number of replicas for achieving high availability is 2. Higher replication factors are possible (3,4,5, ...).

Because Aerospike uses a peer-to-peer architecture, failures don't cause intermittent outages. If a node fails, the cluster can still immediately respond to all of the requests related to the failed node.

Aerospike auto-heals itself after a failure, reducing the need for operator intervention during critical events.

Supports eventual consistency by default for reads to maximize throughput. Strongly consistent reads are also supported on tables and local secondary indexes but consume added read capacity units, which results in higher operational costs.

Writes are strongly consistent.

No public record of Jepsen testing available.

Aerospike offers distinct AP and CP modes per namespace, and neither mode makes the usual hard trade-off implied by the CAP theorem (Consistency, Availability, Partition tolerance). Its AP mode still replicates quickly enough that the eventual-consistency window stays tight, well short of what teams typically brace for with an AP system. Its CP mode doesn't sacrifice availability to the degree many CP databases do, since consensus happens per partition rather than across the whole cluster, and read and write consistency levels are configurable rather than requiring every replica to acknowledge. Its strong-consistency mode was validated by Jepsen testing and prevents stale reads, dirty reads, and data loss.

Aerospike offers true distributed multi-record Atomicity, Consistency, Isolation, Durability (ACID) transactions that provide strict serializability, the strongest correctness guarantee available in a distributed database. Transactions spanning multiple records and their replicas execute as if in a single real-time order, preventing not only stale and dirty reads but subtler anomalies like causal reverse, where a later transaction's effects become visible without an earlier, causally related one also being visible. Few distributed databases, relational or NoSQL, offer this guarantee at all.

It supports multi-site deployments for varied business purposes, including continuous operations, fast localized data access, and more. Global tables enable replicas to be stored in different regions, with read/write access occurring in any region. Changes are automatically replicated across regions asynchronously, typically within a second.

Reads spanning multiple regions of global tables operate under eventual consistency. Write conflicts across regions are resolved with a last-writer-wins strategy. Transactions are not supported across regions in global tables. Note: As of Q4’24, global tables will soon have strong consistency as well.

Users who want to maintain copies of tables (or subsets of tables) across different DynamoDB database instances can implement their own solutions using other AWS services (e.g., Streams and Lambda) if desired.

Aerospike offers two multi-site models with very different consistency characteristics.

The first is asynchronous active-active replication via Cross-Datacenter Replication (XDR), which supports star, one-way, and mesh topologies with filtered replication. Because each cluster accepts local writes and ships them asynchronously, the same record can be updated in more than one cluster before replication catches up, so conflicts are resolved by last-write-wins, either whole-record or, with bin convergence enabled, at the individual field level, so writes to different fields both survive.

Last-write-wins has an important limitation: it converges every copy to the same final state but does not preserve intermediate updates, and it depends on reasonably synchronized clocks. That makes XDR active-active a good fit where the latest value is what matters (session state, profiles, last-known location, and caches) and a poor fit where every update must be counted (wallets, ledgers, and inventory). Often the right fix is a data-modeling one: if you model changes additively in a Collection Data Type (appending each update rather than overwriting a value), concurrent updates land as distinct entries and the merged record accumulates every contribution, letting many "every update counts" workloads use async active-active safely.

The second model, synchronous active-active via multi-site clustering, forms one cluster spanning data centers and uses strong-consistency mode to prevent conflicts entirely, so no committed write is lost. The trade-off is the physics of synchronous replication: each write pays the inter-site round trip, so it needs low-latency (typically metro or regional) links, or a tolerance of higher latency for database writes.

In addition to access by partition key (and an optional sort key), DynamoDB supports secondary indexes. These are stored as DynamoDB tables (and, therefore, B-tree structures).

A local secondary index (LSI) shares the same partition key as the user table it indexes but can have a different sort key and can copy a subset of the table’s attributes. Users may define up to 5 LSIs per table.

A global secondary index (GSI) can have a different partition key and sort key from the user table and copy a subset of the table’s attributes. GSIs support only eventually consistent reads because the index and corresponding user table partitions may not be co-located. Users may define up to 20 GSIs per table by default; increases can be requested via AWS support.

Aerospike uses a proprietary partitioned data structure for its indexes, employing fine-grained individual locks to reduce memory contention across partitions. These structures ensure that frequently accessed data has locality and falls within a single cache line, reducing cache misses and data stalls. For example, the index entry in Aerospike is exactly 64 bytes, the same size as an X86 64-bit cache line.

By default, secondary indexes are kept in DRAM for fast access and are co-located with the primary index, but they can also be stored on SSD to save on memory.

Each secondary index entry references only primary and replicated records local to the node. When a query involving a secondary index executes, records are read in parallel from all nodes; results are aggregated on each node and then returned to the Aerospike client layer for return to the application.

Firms can also opt to store all user and index data (both primary and secondary) on SSDs to improve cost efficiency while still maintaining single-digit millisecond response times.

However, Aerospike can only use a single index in each query.

DynamoDB supports PartiQL, a SQL-92 compatible language maintained by Amazon and offered as open source with Apache 2.0 licensing. Limitations apply to PartiQL operations supported by DynamoDB. For example, reads (SELECTs) support only certain aggregate and conditional functions. Writes (UPDATE, INSERT, DELETE) are also supported with certain limitations  (e.g., updates and deletes can only apply to 1 item at a time). Use of secondary indexes is supported through explicit inclusion in query syntax (“... FROM table.index”).

Similar queries can result in significant cost differences (RCU consumption), leading some users to recommend exporting DynamoDB data to a system that supports a full SQL dialect in an efficient manner.

Dynamo also offers a native (noSQL) query API.

Aerospike Quick Look (AQL) offers SQL-like querying capabilities combined with advanced features like secondary indexing, expressions, and user-defined functions (UDFs). It integrates with popular SQL-based tools while providing a native API for developers to leverage Aerospike's full performance potential.

Aerospike recommends using its native API for optimal performance and full control over data access.

• The Aerospike API allows the implementation of specific SQL operations, such as SELECT, UPDATE, CREATE, and DELETE, with fine-grained performance control.

• Aerospike's data modeling approach aims to minimize the need for SQL constructs like JOINs to maximize performance and scalability benefits.

SQL access is available via Aerospike-built connectors, optimized for high performance with Spark and Presto/Trino. Application developers can use simple SQL with JDBC and the community-contributed JDBC Connector.

The Aerospike Spark connector can read data from Aerospike namespaces using up to 32K Spark partitions. Predicate filtering and scan-by-partition also promote strong performance.

JOINs, unions, intersections, aggregations, and other sophisticated SQL operations are fully supported. Query federation is supported, and the use of secondary indexes is supported.

AWS offers various separate services for caching, auto-scaling, security, and other integrations with DynamoDB. Third-party offerings for integrating with Kafka, Spark, and other technologies are also available.

Aerospike provides optimized connectors for Kafka, Spark, Presto/Trino, JMS, Pulsar, Event Stream Processing (ESP), Elasticsearch, LangChain, LangGraph, and others, giving analytics, AI, and event-processing tools broad access to Aerospike data.

DynamoDB is designed as a persistent, distributed cloud-based store. A separate service (DynamoDB Accelerator or DAX) is recommended for caching and is designed to be used with a DynamoDB backend to improve performance for target scenarios. DAX does not support other back-end stores. 

Flexible configuration options enable Aerospike to act as: 

(1) a high-speed cache to an existing relational or non-relational data store to promote real-time data access and offload work from the back end 

or 

(2) an ultra-fast real-time data management platform with persistence. 

Aerospike can store all data and indexes in DRAM, all data and indexes on SSDs (Flash), or a combination of the two (data on SSDs and indexes in DRAM). As of Aerospike 7.1, adds support for NVMe-compatible, low-cost cloud block storage, and common enterprise networked attached storage (NAS).