Manage solid state storage devices

This page describes how to manage solid state storage devices (SSDs) in your Aerospike nodes.

The Aerospike database recovers quickly after adding, removing, or replacing storage devices. You may use any type of storage device in your Aerospike nodes, including SSDs and hard disk drives (HDDs). Aerospike is optimized for SSDs.

SSD changes require a restart. You can use a Fast Start or a Cold Start if your devices are hot-swappable, and your storage-engine type as summarized in the following table:

Storage type	Hot-swappable	Not hot-swappable
pmem	fast start	cold start
memory	cold start	cold start
device	fast start	cold start
Index stored on flash or pmem	fast start	fast start

Initialize SSDs

You must initialize SSDs before using them in an Aerospike deployment. Initializing SSDs ensures that old data, partitioning, and master boot records (MBRs) do not interfere with Aerospike operations. Older devices containing Aerospike data must be entirely zeroized before reuse, but to save time, newly-provisioned devices can have their headers zeroized and the rest of the write blocks trimmed.

To learn more about initializing multiple SSDs in a single operation, see How to zeroize multiple storage devices simultaneously.

New SSDs

Newly-provisioned devices from cloud service providers are new partitions of existing drives that have not been zeroized, meaning they may still contain some data or random noise instead of zeros. Because of the way Aerospike works, accessing raw blocks on the device without a storage system, any non-zero blocks on the drive may get reinterpreted by Aerospike and affect your operations.

For new SSDs, zeroize the 8 MiB device header using blkdiscard -z, included on most Linux distributions in the util-linux package.

blkdiscard -z --length 8MiB /dev/DEVICE_ID

Used SSDs

If the drive was previously part of an Aerospike deployment, you need to fully zeroize it before use.

First, check whether your drive supports RZAT (Read Zero After Trim). See this Wikipedia article for a non-exhaustive list.

If the drive supports RZAT, perform a trim on the entire device.
Terminal window
```
blkdiscard /dev/DEVICE_ID
```
Then zeroize the 8 MiB header.
Terminal window
```
blkdiscard -z --length 8MiB /dev/DEVICE_ID
```
If the drive does not support RZAT, you must zeroize the entire device, not just the header.
Terminal window
```
blkdiscard -z /dev/DEVICE_ID
```

Adding a hot-swap SSD

After you install and initialize a hot-swap SSD, update /etc/aerospike/aerospike.conf with the name of the SSD. We recommend using the World Wide Identifier (WWID), rather than the /dev name, such as /dev/sda or /dev/nvme0n1p1. WWIDs are unique and do not change, while /dev names can change after a restart. Use the ls command to find WWIDs:
Terminal window
```
ls -l /dev/disk/by-id/
```
The following is an example of the output from a system with a Non-Volatile Memory Express (NVMe) drive controller. The long output shows that the WWIDs are symlinked to /dev/nvme0n1XX:
Terminal window
```
lrwxrwxrwx 1 root root 13 Mar  6 07:34 nvme-eui.8ce38e100077d4b6 -> ../../nvme0n1
lrwxrwxrwx 1 root root 15 Mar  6 07:34 nvme-eui.8ce38e100077d4b6-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 15 Mar  6 07:34 nvme-eui.8ce38e100077d4b6-part2 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 15 Mar  6 07:34 nvme-eui.8ce38e100077d4b6-part3 -> ../../nvme0n1p3
```

Enter your SSD partition names in the storage-engine device section of the namespace configuration:

namespace NAMESPACE-NAME {

    storage-engine device {
        device /dev/disk/by-id/nvme-eui.8ce38e100077d4b6-part1
        device /dev/disk/by-id/nvme-eui.8ce38e100077d4b6-part2
        device /dev/disk/by-id/nvme-eui.8ce38e100077d4b6-part3
        flush-size 128K
        max-record-size 1M
    }
}

To optimize SSD performance, investigate flush-size. For most devices 128KiB seems to be the optimal flush-size.

Restart Aerospike

Stop and then Fast Start the server if your data is stored on disk (pmem or flash).

Cold Start the server if your data is in memory (shmem).

See index-type for more information about storage types.

Non-hot-swappable devices

If your SSDs are not hot-swappable, you must stop the server to replace them, then launch a cold start if your storage type is shmem.

Launch a fast start when your storage types are pmem or flash.

After the server starts, initialize your new SSDs, then enter the new SSD names in the appropriate namespaces in aerospike.conf.

Managing SSD replacements on multiple nodes

If multiple Aerospike nodes require SSD replacements, perform the same procedure on each node one at a time. Wait for data migrations to complete before continuing to the next node.