Use All-Flash for Aerospike on Kubernetes

Overview

This page explains how to create a cluster where Aerospike data and index both use flash storage. For more information on all-flash and other storage configurations, see the Aerospike documentation for namespace storage configuration.

In Aerospike Database 6.3 and later, all-flash storage can run in unprivileged mode if all the required flash kernel parameters are set on all the Kubernetes nodes. See Linux best practices for All-Flash deployment.

In Aerospike Database 6.2 and earlier, all-flash storage requires the Aerospike pod to be run in elevated privileged mode. In privileged mode when using index on flash, Aerospike tweaks the Kubernetes worker node's sysctls for virtual memory.

See privileged: true in the podSpec section of the example CR file.

caution

There is an issue with Kubernetes versions 1.21 and 1.22 where volumeDevices mapping is ignored when containers run in privileged mode.

This bug does not affect Kubernetes versions 1.23 and later or containerd versions 1.15.10 or later in our testing. We recommend the same or higher versions of Kubernetes and containerd for running All-Flash Aerospike clusters.

Procedure

To set this up, modify the storage section in your Aerospike CR as shown in this example.

apiVersion: asdb.aerospike.com/v1
kind: AerospikeCluster
metadata:
  name: aerocluster
  namespace: aerospike

spec:
  size: 2
  image: aerospike/aerospike-server-enterprise:8.0.0.2

  storage:
    filesystemVolumePolicy:
      initMethod: deleteFiles
      cascadeDelete: true
    blockVolumePolicy:
      cascadeDelete: true
    volumes:
      - name: workdir
        aerospike:
          path: /opt/aerospike
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Filesystem
            size: 1Gi
      - name: ns
        aerospike:
          path: /test/dev/xvdf
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Block
            size: 5Gi
      - name: ns-index
        aerospike:
          path: /test/dev/xvdf-index
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Filesystem
            size: 5Gi
      - name: aerospike-config-secret
        source:
          secret:
            secretName: aerospike-secret
        aerospike:
          path: /etc/aerospike/secret

  podSpec:
    multiPodPerHost: true
    aerospikeContainer:
      securityContext:
        privileged: true

  aerospikeAccessControl:
    users:
      - name: admin
        secretName: auth-secret
        roles:
          - sys-admin
          - user-admin
          - data-admin
          - read
          - write

  aerospikeConfig:
    service:
      feature-key-file: /etc/aerospike/secret/features.conf
    security: {}
    network:
      service:
        port: 3000
      fabric:
        port: 3001
      heartbeat:
        port: 3002
    namespaces:
      - name: test
        replication-factor: 2
        index-type:
          type: flash
          mounts:
            - /test/dev/xvdf-index
          mounts-budget: 4294967296
        storage-engine:
          type: device
          devices:
            - /test/dev/xvdf

For the full CR file, see the example all-flash cluster CR.

This and other example CRs are available in the main Aerospike Kubernetes Operator repository.

Save and exit the CR file, then use kubectl to apply the change.

kubectl apply -f aerospike-cluster.yaml