--- title: "Provisioning storage for Aerospike on Kubernetes" description: "Provision persistent storage for Aerospike on Kubernetes using StorageClasses, GCE, or local volumes." --- # Provisioning storage for Aerospike on Kubernetes > For the complete documentation index see: [llms.txt](https://aerospike.com/docs/llms.txt) > > All documentation pages available in markdown. To use persistent storage external to the pods, set up [StorageClasses](https://kubernetes.io/docs/concepts/storage/storage-classes/). StorageClasses are configuration files with instructions for dynamically provisioning the persistent storage required by the Aerospike cluster configuration in your CR file. The specific storage configuration depends on the environment in which you deploy your Kubernetes cluster. Each cloud provider has its own way to set up storage provisioners that dynamically create and attach storage devices to the containers. ## Setup 1. To set up an Aerospike cluster with persistent storage, define your desired storage class in a `storage-class.yaml` file. 2. Specify the storage class name in your CR file. AKO reads the storage class information from the CR file and configures the [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) so Kubernetes can provision the required storage. In these examples, the file is in the directory where you run `kubectl`. 3. Add the storage class configurations to this file, then use `kubectl` to apply these changes to the cluster. ## Google Cloud The following `storage-class.yaml` file uses the GCE provisioner (`kubernetes.io/gce-pd`) to create a StorageClass called `ssd`. ```yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ssd provisioner: kubernetes.io/gce-pd parameters: type: pd-ssd ``` ## Local volume This example uses a local SSD identified as `/dev/nvme0n1`. Attach this SSD to each Kubernetes worker node that will provide the primary storage device for the Aerospike cluster deployment. ### Create a discovery directory Before you deploy the local volume provisioner, create a discovery directory on each worker node and link the block devices you want to use. The provisioner discovers local block volumes from this directory. The directory is defined in `aerospike_local_volume_provisioner.yaml` under `storageClassMap`: provisioner/templates/provisioner.yaml ```yaml apiVersion: v1 kind: ConfigMap metadata: name: local-provisioner-config namespace: aerospike data: useNodeNameOnly: "true" storageClassMap: | local-ssd: hostDir: /mnt/disks mountDir: /mnt/disks ... ``` Terminal window ```bash lsblk ``` Output ```text NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT nvme0n1 8:16 0 375G 0 disk nvme0n2 8:32 0 375G 0 disk ``` Create the directory: Terminal window ```shell mkdir /mnt/disks ``` As a privileged user, link the first block device: Terminal window ```shell ln -s /dev/nvme0n1 /mnt/disks/ ``` Link the second block device: Terminal window ```shell ln -s /dev/nvme0n2 /mnt/disks/ ``` ::: note You can also use your own discovery directory, but make sure that the provisioner is configured to point to the same directory. ::: ### Deploy the local volume provisioner To automate the local volume provisioning, we create and run a provisioner based on [kubernetes-sigs/sig-storage-local-static-provisioner](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner). The provisioner runs as a DaemonSet that manages the local SSDs on each node based on a discovery directory, creates and deletes the PersistentVolumes, and cleans up the storage when it is released. The local volume static provisioner for this example is defined in [aerospike\_local\_volume\_provisioner.yaml](https://github.com/aerospike/aerospike-kubernetes-operator/tree/v4.4.1/config/samples/storage/aerospike_local_volume_provisioner.yaml). The storage class is defined in [local\_storage\_class.yaml](https://github.com/aerospike/aerospike-kubernetes-operator/blob/v4.4.1/config/samples/storage/local_storage_class.yaml). This and other example CRs are available in [the main Aerospike Kubernetes Operator repository](https://github.com/aerospike/aerospike-kubernetes-operator/tree/v4.4.1/config/samples). ::: note 1. If the provisioner configuration uses `shred` for block cleanup, PersistentVolume release and reclaim time is proportional to the size of the block device. 2. You can change the `local-provisioner-config` ConfigMap in [aerospike\_local\_volume\_provisioner.yaml](https://github.com/aerospike/aerospike-kubernetes-operator/tree/v4.4.1/config/samples/storage/aerospike_local_volume_provisioner.yaml) to use `blkdiscard` as the cleanup method if the disk supports it. For more information, see the [FAQs](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/master/docs/faqs). Refer to the `blockCleanerCommand` section in the following configuration: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: local-provisioner-config namespace: aerospike data: useNodeNameOnly: "true" storageClassMap: | local-ssd: hostDir: /mnt/disks mountDir: /mnt/disks blockCleanerCommand: - "/scripts/blkdiscard.sh" volumeMode: Block ``` When you use `blkdiscard`, make sure the drive deterministically returns zeros after TRIM (RZAT). Check with your hardware provider or cloud provider. If you are unsure, it is safer to use `dd`. ::: 1. Create a local storage class: Terminal window ```shell kubectl create -f local_storage_class.yaml ``` 2. Then deploy the provisioner: Terminal window ```shell kubectl create -f aerospike_local_volume_provisioner.yaml ``` 3. Verify the persistent volumes were created. Terminal window ```shell kubectl get pv ``` kubectl output ```text NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE local-pv-342b45ed 375Gi RWO Delete Available "local-ssd" 3s local-pv-3587dbec 375Gi RWO Delete Available "local-ssd" 3s ``` ::: note The storage class configured here is `local-ssd`. Use this value in the Aerospike cluster CR so Kubernetes can request PV resources from the provisioner. :::