Launch an Aerospike Stack on Amazon EC2

The reality of setting up and configuring all the tools necessary to deploy Aerospike on Amazon EC2 is time-consuming. AWS CloudFormation gives you an easy way to model a collection of related AWS and third-party resources, provisioning them quickly and consistently by treating infrastructure as code.

Partnering with the AWS Quick Start team, we created an automated reference deployment to help deploy and manage Aerospike Database clusters from your AWS environment. The Aerospike Quick Start Reference Deployment on AWS allows you to take advantage of AWS CloudFormation’s seamless automation of the Aerospike data platform.

Launch an Aerospike Database Stack on AWS CloudFormation

The Aerospike Quick Start is an automated reference deployment intended to help you to quickly build a new AWS environment consisting of the VPC, subnets, NAT gateways, security groups, bastion hosts, and other infrastructure components. It then deploys Aerospike Database Enterprise Edition into this new VPC.

There are  two options to get started:

  1. A single-node free tier development environment with an enterprise trial licenseThis option is designed for developers to set up an environment to begin building on AWS with the Aerospike Enterprise database. In a matter of five to ten minutes you can start working with an Aeropsike, in-memory namespace with a 50GB EBS volume attached.
  2. A multi-node Aerospike cluster, built to allow you to bring-your-own-license (BYOL), which requires an Aerospike feature key file.This option is designed for existing Aerospike customers to automatically deploy a large multi node cluster with their Aerospike feature key file.

It is important to note that Aerospike supports a Hybrid Memory Architecture where by default, indexes are stored in DRAM, but can be configured to store indexes on an NVMe flash device. Data may also be stored in DRAM or on more cost-effective SSDs.

To utilize this configuration, an instance type with local NVMe SSDs, like the m5d.xlarge, needs to be selected. If you are using SSDs, you will define the size of the EBS with a 0 value.

High level steps to get started

To deploy either option, you will follow the instructions in the deployment guide and includes these steps:

  • Sign in to your AWS account. If you don’t have an account, sign up at https://aws.amazon.com.
  • Subscribe to the Amazon Machine Image (AMI) for Aerospike Database Enterprise Edition in AWS Marketplace and choose the AWS Region.
  • Launch the Quick Start. You can choose from two options:
    • Deploy Aerospike into a new VPC
    • Deploy Aerospike into an existing VPC
  • Test your deployment.

Quick Start Reference Architecture

This blog will walk through how the automated reference deployment allows users to quickly and easily launch a basic Aerospike Enterprise instance, or cluster from the AWS CLI or console.

AWS Cloud architecture diagram
Figure 1. Architecture
Step 1: Setup your AWS account for Aerospike Enterprise CloudFormation Support:

The first step is to sign up for Aerospike Enterprise or use the single node configuration, free tier license.  Once you create your account, follow these steps.

  • Navigate to the AWS Management Console and sign into your account
  • Create an EC2 key pair to provide a secure connection to the instance

To do this you can begin by typing EC2 into the search bar at the top of the page.

  • Select the EC2 option. In the “Resources” section of the EC2 console select “Key pairs” then click “Create key pair”.
  • Assign a name and leave the “pem” file option selected, then click “Create key pair”

This creates the key pair as well as initiates a download of the pem file.

You will want to take note of where the file was saved as it will be used later to connect to the database.

Step 2: Initiate the Aerospike Database Enterprise Edition Quick Start

Once the key pair has been created, navigate back to the search bar at the top of the screen and search for “Aerospike”. Select the “Aerospike Database Enterprise Edition” option from the results.  The Aerospike Quick Start can also be initiated from the AWS Marketplace.

The main page contains a product overview for Aerospike as well as pricing, usage, and support information. When finished reviewing this information, head back to the top of the page and click “Continue to Subscribe”.  Read through the terms and conditions, then click, “Accept Terms”.  The request may take a minute to process.

  • Once complete, click “Continue to Configuration”.
  • From the “Delivery Method” dropdown menu, select the “CloudFormation Template”. This template provides a complete solution configuration making the reference deployment a snap.
  • Select whether to use a new or existing Virtual Private Cloud Cluster. For this tutorial a new VPC Cluster will be used.
  • Selecting a new VPC with the default setup will build an Aerospike Database environment that spans two to three availability zones and utilizes a Linux bastion host for secure access to the Aerospike server, which resides within a private subnet.
  • Aerospike Enterprise on AWS Quick Start reference architecture:
  • Select the latest Aerospike Enterprise version and select the “Region” and click “Continue to Launch”.
  • After reviewing the configuration from the “Choose Action” dropdown, select “Launch CloudFormation” and click “Launch”.
Step 3: Enter your Configuration 

For this overview, the options on this page will be left at default values.

  • Specify the Stack Name at the top of the page, and review the setup parameters.
  • It is important to choose at least two Availability Zones, then select the number of zones used in the previous step; these must match.
  • Working with the default values, continue until the “EC2 key pair” option. Then select the key pair created at the beginning of this tutorial.
  • Provide the number of instances to provision for the Aerospike cluster. If this is left at one, it will automatically set up the enterprise trial license. Important!  For any other value, an Aerospike feature key file will need to be provided further down the page.
AWS CloudFormation basic stack parameters
Figure 2. Basic stack parameters
  • Select default values, until reaching the instance type to deploy.
    • For a single-node instance of Aerospike, select a cost effective instance type. The m5.large works well for development and pre-production environments.
    • For a high performance applications you will want to use instance types with NVMe SSDs. The r5d.xlarge works well for high performance and availability configurations.
AWS stack configuration and instance selection
Figure 3. Instance selection

To learn more about best practices for performance and high availability configurations checkout this Aerospike for AWS EC2 Deployment Guide

  • Define SSD size in GB, then click “Next”.
    • For a single-node in-memory namespace development environment an EBS 50GB works well.
    • For a multi-node cluster with NVMe SSDs, you will enter a 0 value. Post Quick Start you will want to follow the instructions to configure your NVMe SSDs.

Note that the Aerospike database is highly optimized to run on Flash and SSD devices, and is capable – through Hybrid Memory index use, and direct device access – of providing unique throughput and low latency on Flash.  Depending on whether you want an in-memory database, a hybrid SSD/memory database, an in-memory cache with no persistence, or a database deployed to the cloud (such as on Amazon EC2), you can provision your SSDs and RAM appropriately. To learn more about how to configure Aerospike for high performance and availability, refer to the Aerospike Capacity Planning Guide.

  • On this page configure tags, permissions, and advanced options. These will also be left at the default.
  • Review your configuration settings and check the boxes at the bottom of the page to acknowledge the required capabilities and click “Create Stack”.
Step 4: Connect to the Aerospike Server

After the stack creation is complete, a connection to the Aerospike server can be made by first connecting to the bastion host, then from the host to the server.

To begin, open a terminal window and configure the SSH agent to add the .pem file identity.

ssh-add -K <key-pair filename>.pem

Next, connect to the bastion host.

ssh –A [email protected]<bastion-IP-address>

Finally, connect to the Aerospike server.

ssh [email protected]<instance-IP-address>

Step 5: Configure your NVMe SSDs (for multi-node cluster)
  • Get device name using `fdisk -l`, the example below assumes that instance type is r5d.xlarge and device is located at `/dev/nvme1n1`
  • Append namespace with following content in aerospike.conf :
    • In the following example namespace name is ‘device’, replication-factor is `2`, memory-size is ‘4G’, device is `/dev/nvme1n1’ and write-block-size is `1024K’. It was found that for r5d.xlarge instance the device is located at ` /dev/nvme1n1`
namespace device {
  replication-factor 2
  memory-size 4G
  storage-engine device {
       	   device /dev/nvme1n1
          write-block-size 1024K
  }
}
  • Restart aerospike

$ systemctl restart aerospike

Step 6: Write to the Aerospike Database

With the connection established, open the AQL prompt and write a record to the database, then read it back.

aql
INSERT INTO test.foo (PK, bar) VALUES (‘Key1’, ‘We did it!’)
SELECT * FROM test.foo

Now that you have verified a working Aerospike Cluster, you are ready to start working with Aerospike!