All-flash storage configuration for Aerospike on Kubernetes

Overview

This page explains how to create a cluster where Aerospike stores data and index flash storage. For more information on all-flash and other storage configurations, see Configure namespace storage.

In privileged mode when using index on flash, Aerospike tweaks the Kubernetes worker node’s sysctls for virtual memory. See privileged: true in the podSpec section of the example CR file.

Note

• In Aerospike Database 6.3 and later, all-flash storage can run in unprivileged mode if all the required flash kernel parameters are set on all the Kubernetes nodes. See Linux best practices for All Flash deployment.

• Prior to Aerospike Database 6.3, all-flash storage requires the Aerospike pod to be run in elevated privileged mode.

Caution

The [volumeDevices mapping bug]https://github.com/kubernetes/kubernetes/issues/85624) in Kubernetes 1.21 and 1.22 does not affect Kubernetes 1.23 and later, or containerd versions 1.15.10 or later in our testing. We recommend the same or higher versions of Kubernetes and containerd for running All Flash Aerospike clusters.

Configure all-flash storage

To set this up, modify the storage section in your Aerospike CR file as shown in the following example.

apiVersion: asdb.aerospike.com/v1
kind: AerospikeCluster
metadata:
  name: aerocluster
  namespace: aerospike

spec:
  size: 2
  image: aerospike/aerospike-server-enterprise:8.0.0.2

  storage:
    filesystemVolumePolicy:
      initMethod: deleteFiles
      cascadeDelete: true
    blockVolumePolicy:
      cascadeDelete: true
    volumes:
      - name: workdir
        aerospike:
          path: /opt/aerospike
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Filesystem
            size: 1Gi
      - name: ns
        aerospike:
          path: /test/dev/xvdf
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Block
            size: 5Gi
      - name: ns-index
        aerospike:
          path: /test/dev/xvdf-index
        source:
          persistentVolume:
            storageClass: ssd
            volumeMode: Filesystem
            size: 5Gi
      - name: aerospike-config-secret
        source:
          secret:
            secretName: aerospike-secret
        aerospike:
          path: /etc/aerospike/secret

  podSpec:
    multiPodPerHost: true
    aerospikeContainer:
      securityContext:
        privileged: true

  aerospikeAccessControl:
    users:
      - name: admin
        secretName: auth-secret
        roles:
          - sys-admin
          - user-admin
          - data-admin
          - read
          - write

  aerospikeConfig:
    service:
      feature-key-file: /etc/aerospike/secret/features.conf
    security: {}
    network:
      service:
        port: 3000
      fabric:
        port: 3001
      heartbeat:
        port: 3002
    namespaces:
      - name: test
        replication-factor: 2
        index-type:
          type: flash
          mounts:
            - /test/dev/xvdf-index
          mounts-budget: 4294967296
        storage-engine:
          type: device
          devices:
            - /test/dev/xvdf

For the full CR file, see the example all-flash cluster CR.

This and other example CRs are available in the main Aerospike Kubernetes Operator repository.

Save and exit the CR file, then use kubectl to apply the change.

kubectl apply -f aerospike-cluster.yaml