Aerospike Database 7.2: Enhanced clustering and XDR controls

We are excited to announce the general availability of Aerospike Database 7.2. This is an operations-focused release with enhancements to multi-zone deployments through a new active-rack feature and a new version of shipping control for Cross Datacenter Replication (XDR). Version shipping controls provide flexible choices for backups and connectors and will subsequently be leveraged by the Aerospike Backup Service (ABS).

Let’s look at these new features one at a time.

Active-passive cross-AZ deployments 

Multi-zone deployments take advantage of Aerospike’s rack awareness, with each availability zone (AZ) being designated a unique rack ID. Prior to Database 7.2, the master and replica partitions could only be evenly distributed across AZs, with each zone receiving an equal share of both. This meant that all AZs in a cross-AZ cluster deployment were active.

While Aerospike clients can be configured (by policy) to prefer reading from a local zone shared by the application and a cluster rack, the even distribution of partitions means that write operations are distributed evenly to all zones while paying the latency and egress cost associated with cross-AZ traffic (for half the writes in a two-AZ deployment, two-thirds of the writes in a three-AZ deployment, etc.).

With the active-rack configuration parameter introduced by Database 7.2, the operator can designate an AZ as the active zone in the Aerospike cluster, with the other AZs passively receiving synchronous replica writes. In this active-passive cluster deployment, all write operations go to one AZ, avoiding slower cross-AZ acks and reducing egress cost to the minimum.

As a result, this cluster deployment enables AZ fault-tolerant disaster recovery (DR) with the lowest latency profile and lowest cloud provider costs. 

In Aerospike clusters configured for strong consistency (SC), a multi-zone deployment can be insulated from losing availability during a network split between the AZs (or if an AZ goes down). In the case of a three-AZ cluster deployment, the two AZs that can still talk to each other will have full copies of the data and full availability without risking lost writes or data inconsistency due to the fact that 2/3rds of the nodes retain a majority. A popular variant of this is having two AZs populated, each with half the active cluster nodes and a third AZ with a single quiesced node acting as the “tie-breaker” in the event of a network split. The remaining AZ, together with the tie-breaker, form a majority of the nodes.

The new active-rack configuration gives a two-AZ SC deployment the same benefit as a tie-breaker deployment, without needing a third zone, and avoiding operational complications such as a very different config for the active nodes vs. the quiesced one, avoiding potential networking issues in the tie-breaker node, and reducing the likelihood of an AZ failure (by having only two). If the passive AZ goes down, the active AZ (where active-rack is true) continues serving the applications in its AZ with full availability. If the active AZ goes down, a passive AZ can be promoted with a single dynamic configuration change.

Multi-site cluster deployments across regions can also benefit in the same way from an active-passive deployment using active-rack.

XDR version shipping controls

XDR is used for the asynchronous delivery of data from an Aerospike source cluster to a variety of destinations. XDR keeps track of the unique identifier (digest) of records that are modified (inserted, updated, deleted), expire, or are evicted. When it's time to ship the record, the XDR reads it from storage and ships it to the destination. Therefore, if there are multiple updates to the same record in a short period of time, XDR only ships the latest version of the record. This optimization helps XDR achieve maximum throughput while keeping two remote Aerospike Database clusters in sync. However, this prevents XDR from shipping all versions of a record, which may be desired when Aerospike is shipping data to a non-Aerospike destination through a connector, such as Kafka, Pulsar, or JMS.

Database 7.2 introduces a dynamically configurable mechanism to control how XDR ships versions through two new configuration parameters - ship-versions-policy and ship-versions-interval.

Ship the latest version

By configuring XDR ship-versions-policy latest, you explicitly state that you want the same behavior XDR had prior to Database 7.2. Lag between the source cluster and its destination can result in intermediate versions being skipped.

Ensure shipping at least one version in a defined interval

The new ship-versions-policy interval ensures that at least one record version will ship per ship-versions-interval in seconds. If the lag between source cluster and destination is such that this cannot be achieved, backpressure will be applied by refusing further writes to the record until its current version has shipped. This interval policy can be used to guarantee a desired update granularity for connectors and backups.

Ship all versions

ship-versions-policy all is the policy that instructs XDR to confirm that every single version of the record is written to the destinations or else applies backpressure in the form of an XDR hot key error code, refusing the next write until the current version has shipped.

Similarly to the interval policy, in low-throughput use cases, this policy can be used to ensure that the destination (such as another database via Kafka) receives every update to every record. The operator does need to consider the backpressure implications when lag between source and destination increases.

Get started with Aerospike Database 7.2

For more details, read the Database 7.2 release notes. You can download Aerospike Enterprise Edition (EE) and run it in a single-node evaluation or get started with a 60-day multi-node trial.