Geo-distribution deployment models: Achieving business outcomes while minimizing latency

The Aerospike Database offers multiple deployment strategies for building geographically distributed applications. Some deployments are intra-regional; other clustering deployment options are between regions within the same continent and even between regions located on different continents.

Before we delve into the options themselves, there are terms that we want to clarify – “region,” “availability zone,” and “site” / “multi-site.”

Region: A certain geographical area, roughly within a radius of 50 miles (80 km). Regions are typically located near major metropolitan and business areas where customers (companies) are located. Each region usually consists of three to six availability zones.

Availability zone: A distinct data center that is located within a region. Each availability zone has independent power, cooling, and physical security completely separate from other data centers (availability zones) in that region. Moreover, all availability zones are connected with a redundant, low-latency private fiber-optic network. Availability zones are scattered throughout the region, with a minimal distance between them, so a failure in one availability zone will not affect other availability zones within the same region.

The above description is general and true for all public clouds (AWS, Google Cloud, Azure, etc.). Hence, when we discuss the distance between availability zones, that distance is relatively small, typically no more than a few tens of miles. The distance between regions can be significantly larger – hundreds and even thousands of miles apart.

Site / Multi-site: A site referring to the Aerospike Database is a data center and equivalent to a data center. “Multi-site” refers to multiple data centers/availability zones, but they do not need to be within the same region.

Let’s review the clustering deployment models that Aerospike offers. The first two deployment models are for a Jepsen-certified, strongly consistent cluster that can be local (within the same region) or stretched between continents. Note that the Jepsen test is an independent industry test to ascertain whether a distributed software’s consistency claims are valid. The remaining five deployment models are for high-availability clusters, which may also be local or spread worldwide.

Strong consistency models

The Aerospike Database supports both linearizable and session consistency for transactions and more precise application development. For deployment and site configuration, this translates to active-active configurations for synchronous clustering. This can be configured for either single-region or multi-region.

Active-active, single-region multi-site, synchronous clustering

This option deploys an Aerospike Database cluster across two or more availability zones in the same region. It is used for applications and use cases that must have strong consistency between multiple sites.

Resilience: This option provides strong consistency between availability zones, meaning the exact same data is always available in all zones. Hence, if one of the sites goes down for whatever reason, the remaining site(s) continue to function uninterrupted and will support the application.

Consistency: Jepsen-certified strong consistency mode.

Latency: Due to the close proximity between availability zones in the cluster, both read and write activities will have extremely low latency, typically under 1 ms for each.

Use this deployment option for:

• Apps that must have strong consistency, such as:

  • A regional bank (for money-related applications)

  • Regional businesses that sell online and must maintain constantly updated inventory between sites

• High-availability, active-active setup between two or more data centers within the same region

Figure 1: Active-active, single-region multi-site, synchronous clustering

Active-active, multi-regional, multi-site, synchronous clustering

While the prior option provides strong consistency between availability zones located in the same region, this option does the same between remote regions.

In essence, active-active, multi-regional, multi-site, synchronous clustering provides a Jepsen-certified, strongly consistent stretched cluster between sites that can be located thousands of miles apart. Such sites can be on different sides of the same continent, like U.S. East and U.S. West, or different continents, say the U.S. and Europe. No matter what, the data in each site will be strongly consistent. Latency will depend on the distance between sites. For example, it can be approximately 100ms for strongly consistent writes across the United States or approximately 200ms from the U.S. West Coast to Western Europe.

Resilience: This is a strongly consistent stretched cluster between remote regions. Even if a severe catastrophe hits a certain region and drops multiple availability zones within that region (say, because of a powerful hurricane) – other remote regions are unaffected and continue to work and support the application(s) served by the stretched cluster. Consistency: Jepsen-certified strong consistency. Writes are never lost, and there are never any conflicts.

Latency: Depends on setup for the stretched cluster and the distance between regions:

• Read latency:

  • Extremely low, typically < 1ms if Reads are from the local site (yet data may be stale)

  • Higher if need linearized Reads (i.e., not stale) from a remote site. In this case, Read latency depends on distance.

• Write latency: Depends on the distance between sites (regions). For example:

  • U.S. East to U.S. West: < 100ms

  • U.S. West to Western Europe < 200ms

Use this deployment option for:

• Apps requiring multi-region, Jepsen-certified strong consistency, including between continents. A few examples:

  • Global banking

  • Global financial applications

  • Global inventory management

Figure 2: Active-active, multi-regional, multi-site, synchronous clustering

High availability models

The Aerospike Database, by default, is set for availability under partition tolerance, or AP mode, when considering the CAP theorem. Users have many data center/site/zone/region configurations at their fingertips for applications that don’t necessarily require strong consistency.

Disaster recovery (DR)

Used for disaster recovery and specifically for a hot failover scenario, this option is for two sites that are relatively close to one another, typically two different availability zones within the same region.

Resilience: This is a high-availability setup in which the replica site is always available should anything happen to the main site.

Consistency: Strong consistency mode in the main site, high availability mode in DR site (typically < 2ms).

Latency:

• Reads:

  • Main site: Extremely low latency, typically < 1ms

  • DR site: No read activity unless the main site goes down and hot failover occurs

• Writes:

  • Main site: Extremely low latency, typically < 1ms

  • DR site: Stream of writes sent from main site to DR site; latency is a few milliseconds

Use this deployment option for:

• Main site and a DR site, with a hot failover scenario

Figure 3: Disaster Recovery (DR) configuration

Single-region, multi-zone, highly available single cluster

This deployment option provides high availability in case a certain zone goes down, yet the cluster continues to operate. Used for highly available apps that require availability zone failure tolerance.

Resilience: Regional cluster with multiple zones; the cluster will continue to operate even if a certain zone goes down.

Consistency: High availability mode, synchronous updates

Latency:

• Reads: Extremely low, typically < 1ms

• Writes: Extremely low, typically ~ single-digit ms or less for strong consistency (dependent on inter-zone latencies)

Use this deployment option for:

• Highly available apps that must have availability-zone failure tolerance:

  • AdTech

  • Gaming

  • Fraud detection

  • Application caching

Figure 4: Single-region, multi-zone, highly available single cluster

Active-active cross-datacenter replication

The active-active cross-datacenter replication (XDR) deployment model serves a cluster comprised of independent data centers that can be dispersed around the world. Each data center handles local reads and writes. Then, each data center sends streams of writes to all other data centers.

Resilience: Highly resilient since it is comprised of independent data centers. Even if one or more data centers go down, the rest of the data centers continue to function independently and serve the app(s).

Consistency: High availability mode

Latency:

• Reads: Extremely low latency from the local site, typically < 1ms

• Writes:

  • Local data center: Low latency, typically 1-4ms

  • Other data centers: Updates to all data centers (based on high availability) depend on the distance between sites

    • For example: U.S. Coast to Coast < 100ms; U.S. West Coast to Europe < 200ms

Use this deployment option for:

• Applications that need instantaneous reads and fast local writes. Examples of such apps:

  • AdTech

  • Fraud detection

  • Recommendation engines

Figure 5: Active-active cross-datacenter replication

Active-passive cross-datacenter replication – Cascading updates

This deployment model is used for a large cluster that provides high availability and consists of multiple independent data centers. Datacenters can be spread around the world. One data center is used for all Writes, and then it ships streams of Writes in a cascading model to replicas in all other data centers. For example, writes are made on U.S. West Coast, then shipped to U.S. East Coast, then to Europe, and then to Asia.

Resilience: Highly resilient since comprised of independent data centers. Even if one or more data centers go down, the rest of the data centers continue to function independently and serve the app(s).

Consistency: High availability mode

Latency:

• Reads: Extremely low, typically < 1ms

• Writes:

  • Source data center: Low, typically 1-4ms

  • Remote data centers: Remote updates depend on the distance from the client to the source data center. The updates then cascade down through the regional cluster

    • Eventual consistency depends on the distance to the source. Ex: U.S. Coast to Coast < 100ms; U.S. East Coast to Europe < 200ms

Use this deployment option for:

• Applications (or companies) that want to maintain a single location for Writes and then copy these Writes to all other locations in a cascading manner. This is mostly used for experiential apps, such as:

  • Suggestion engines (purchasing, movies, etc.)

  • AdTech

Figure 6: Active-passive cross-datacenter replication – Cascading updates

Active-passive cross-datacenter replication – Hub and spoke updates

This seventh and last deployment model is very similar to the previous model. It is used for a large cluster that provides high availability and consists of multiple independent data centers. Datacenters can be spread around the world. One data center is used for all writes, only this time, it ships streams of writes in a “hub and spoke” model to replicas in all other data centers. For example, the hub for Writes is on U.S. East Coast. It then sends streams of writes to U.S. West, Europe, and Asia.

Resilience: Highly resilient since it is comprised of independent data centers. Even if one or more data centers go down, the rest of the data centers continue to function independently and serve the app(s).

Consistency: High availability mode

Latency:

• Reads: Extremely low, typically < 1ms

• Writes:

  • Source data center: Low, typically 1-4ms

  • Remote data centers: Remote writes depend on the distance from the client to the source data center. The updates then cascade down

    • Eventual consistency depends on the distance to the source. Ex: U.S. Coast to Coast < 100ms; U.S. East Coast to Europe < 200ms

Use this deployment option for:

• Applications (or companies) that want to maintain a single location for Writes and then copy these Writes to all other locations in a hub and spoke manner. This is mostly used for experiential apps, such as:

  • Suggestion engines (purchasing, movies, etc.)

  • AdTech

  • Social feeds

Figure 7: Active-passive cross-datacenter replication – Hub and spoke updates

Summary

The Aerospike Database offers multiple deployment options for building geographically distributed applications with the ability to configure Aerospike clusters across sites, data centers, availability zones, regions – and even cloud providers simultaneously. This enables applications to select the resiliency, availability, and accuracy that their application requires.