Launch an Aerospike Stack on Amazon EC2

In a previous blog, we guided you through the process of deploying a single-node Aerospike cluster using the automated AWS CloudFormation template from the Aerospike Quick Start guide. Here is a link to that blog, How to deploy Aerospike Database 5 using AWS Quick Start.

In this second blog about the Aerospike Quick Start guide, we’ll be using some of the same steps, but instead, we will deploy a three-node cluster in two AWS Availability Zones. Note that you won’t be able to use the default license included in the Aerospike AMI. In the case of a multi-node deployment, you can BYOL (bring your own license) or request a free three-node evaluation license using this link.

As explained in the previous blog, setting up and configuring all the tools necessary to deploy Aerospike on Amazon EC2 is time-consuming. AWS CloudFormation gives you an easy way to model a collection of related AWS and third-party resources, provisioning them quickly and consistently by treating infrastructure as code.

In partnership with the AWS Quick Start team, we created an automated deployment guide to help launch Aerospike Database clusters from your AWS environment. The Aerospike Quick Start allows you to take advantage of the automated process using an AWS CloudFormation template.

Launch a three-node Aerospike Database Stack using AWS CloudFormation

The Aerospike Quick Start is an automated reference deployment guide intended to help you quickly build a new AWS environment consisting of the VPC, subnets, NAT gateways, security groups, bastion hosts, and other infrastructure components required to deploy a fully functional stack. Then it deploys the Aerospike Database 5 cluster.

This guide was designed for developers to set up an environment to begin building on AWS with the Aerospike Enterprise database or IT teams to test enterprise features like high availability. In a matter of five to ten minutes, you can start working with an Aerospike, in-memory namespace with a 50GB EBS volume attached.

The following steps will guide you on adjusting the template to deploy a three-node cluster into a new VPC and receive a free evaluation license for the Aerospike data platform. If you already have a multi-node license, you don’t need to request a new one.

The evaluation license includes access to the Aerospike connect product line to connect the database with popular data frameworks like Kafka and Spark. For more information, visit the Aerospike Connect page.

Follow this link to request your free three-node evaluation license. You’ll receive an email with instructions on how use the Aerospike Feature Key File during the deployment phase.

The Aerospike Quick Start deploys a general-purpose cluster and is not intended for a high-performance workload. However, it is important to note that Aerospike supports HMA™ (Hybrid Memory Architecture), where by default, indexes are stored in DRAM but can be configured to store indexes on an NVMe flash device. Data may also be stored in DRAM or on more cost-effective SSDs. An instance type with local NVMe SSDs, like the m5d.xlarge, needs to be selected to utilize this configuration.  If you are using SSDs, you will define the size of the EBS with a 0 value. For more information on HMA or a customized deployment, contact us.

High level steps to get started

Follow the instructions in the deployment guide, under the “How to deploy” section, that include these steps:

Quick Start Reference Architecture

This blog will walk through how the automated reference deployment allows users to quickly and easily launch a basic Aerospike Enterprise instance, or cluster from the AWS CLI or console.

AWS Cloud architecture diagram
Figure 1. Architecture
Step 1: Setup your AWS account:

If you don’t have an AWS account, create one and follow these steps.

  • Navigate to the AWS Management Console and sign into your account
  • Create an EC2 key pair to provide a secure connection to the instance

To do this you can begin by typing EC2 into the search bar at the top of the AWS page.

  • Select the EC2 option. In the “Resources” section of the EC2 console, select “Key pairs,” then click “Create key pair.”
  • Assign a name, leave the “RSA” and “.pem” file options selected, and click “Create key pair.” Remember to create the key in the same region where you want to create your Aerospike cluster.

This creates the key pair as well as initiates a download of the pem file.

You will want to take note of where the file was saved, as it will be used later to connect to the database.

Step 2: Initiate the Aerospike Database 5 Enterprise Edition Quick Start

Once the key pair is created and saved, navigate back to the search bar at the top of the screen and search for “Aerospike.” Select the “Aerospike Database Enterprise Edition” option from the results.

The main page contains a product overview for Aerospike and pricing, usage, and support information. When finished reviewing this information, head back to the top of the page and click “Continue to Subscribe.”  Read through the terms and conditions, then click “Accept Terms.”  The request may take a minute to process.

  • Once complete, navigate to the Aerospike Quick Start page
  • From the “How to deploy” section, select
    • Deploy Aerospike into a new VPC
  • Selecting a new VPC will build an Aerospike Database environment that spans two availability zones and utilizes a Linux bastion host for secure access to the Aerospike server, which resides within a private subnet. You can check the architecture under the “What you’ll deploy” section.
Step 3: Enter your Configuration 

For this overview, we will guide you through deploying a three-node cluster.

  • Specify the Stack Name at the top of the page, and review the setup parameters.
  • It is important to choose at least two Availability Zones, then select the number of zones used in the previous step; these must match.
  • Working with the default values, continue until the “EC2 key pair” option. Then select the key pair created at the beginning of this tutorial.
  • Provide the number of instances to provision for the Aerospike cluster and modify this parameter to “3”. This will configure all the networking and additional resources for multi-node cluster deployment. Note that the Aerospike Feature Key File will need to be provided further down the page.
AWS CloudFormation basic stack parameters
Figure 2. Basic stack parameters
  • Select default values, until reaching the instance type to deploy.
    • For general-purpose, select a cost-effective instance type. The m5.large works well for development and pre-production environments.
    • For testing an HMA configuration, you will want to use instance types with NVMe SSDs. The r5d.xlarge works well for high performance and availability configurations.
AWS stack configuration and instance selection
Figure 3. Instance selection

To learn more about best practices for performance and high availability configurations, check out this Aerospike for AWS EC2 Deployment Guide

  • Define SSD size in GB, then click “Next”.
    • For a single-node in-memory namespace development environment, an EBS 50GB works well.
    • For a multi-node cluster with NVMe SSDs, you will enter a 0 value. Post-Quick Start you will want to follow the instructions to configure your NVMe SSDs.

    Note that the Aerospike database is highly optimized to run on Flash and SSD devices and is capable of providing unique throughput and low latency on Flash through Hybrid Memory index use and direct device access. Depending on whether you want an in-memory database, a hybrid SSD/memory database, an in-memory cache with no persistence, or a database deployed to the cloud (such as on Amazon EC2), you can provision your SSDs and RAM appropriately. Refer to the Aerospike Capacity Planning Guide to learn more about configuring Aerospike for high performance and availability.

  • Under “Feature key file,” paste the base64 encoded file of your Aerospike Feature Key File to be able to run the three-node cluster.Here is an example of how to encode the key file using the base64 command with the corresponding output which is a long string of characters that looks like this:

    base64 TRIAL-Aerospike_FeatureKey-3-Node-30-days.conf

    Feature key file output sample

  • On this page, configure tags, permissions, and advanced options using defaults for this exercise.
  • Review your configuration settings and check the boxes at the bottom of the page to acknowledge the required capabilities and click “Create Stack.”
Step 4: Connect to the Aerospike Server

After the stack creation is complete, a connection to the Aerospike server can be made by first connecting to the bastion host, then from the host to other servers. The IP addresses for the bastion host and the other instances can be found in the EC2 dashboard from your AWS console.

To begin, open a terminal window and configure the SSH agent to add the .pem file identity, remember to change permissions first.

chmod 0400 <key-pair filename>.pem

ssh-add -K <key-pair filename>.pem

Next, connect to the bastion host.

ssh –A [email protected]<bastion-IP-address>

Finally, connect to the Aerospike server using the Private IP address.

ssh [email protected]<instance-IP-address>

Step 5: Configure your NVMe SSDs (for multi-node cluster)

If you selected instances with NVMe SSDs you can configure the namespace

  • Get device name using `fdisk -l`, the example below assumes that instance type is r5d.xlarge and device is located at `/dev/nvme1n1`. Different instances have different names for the devices, if you have trouble identifying them, please contact us at https://aerospike.com/forms/contact-us/
  • Append namespace with following content in aerospike.conf :
    • In the following example namespace name is ‘device’, replication-factor is `2`, memory-size is ‘4G’, device is `/dev/nvme1n1’ and write-block-size is `1024K’. It was found that for r5d.xlarge instance the device is located at ` /dev/nvme1n1`
    namespace device {
      replication-factor 2
      memory-size 4G
      storage-engine device {
           	   device /dev/nvme1n1
              write-block-size 1024K
      }
    }
    
  • Restart aerospike
    $ systemctl restart aerospike
Step 6: Write to the Aerospike Database

With the connection established, open the AQL prompt and write a record to the database, then read it back.

aql
INSERT INTO test.foo (PK, bar) VALUES (“Key1”, “We did it!”)
SELECT * FROM test.foo

Now that you have verified a working Aerospike Cluster, you are ready to start working with Aerospike!

Build fast, scale up, save more in AWS.