Skip to main content
Loading

ABS Configuration Parameters

Overview

Aerospike Backup Service (ABS) reads the configuration file aerospike-backup-service.yml to create connections to Aerospike Database namespaces and storage destinations, as well as backup policies, routines, and schedules.

This page explains two sample configuration files included in the ABS GitHub repository and details about the most common parameters that you may want to adjust. These parameters cover a majority of use cases for testing ABS in an application workflow.

For a full list of parameters, see the Schemas section in the REST API specification.

Example configuration files

Docker Compose configuration

The following sample backup service configuration supplied with the Docker Compose stack for ABS consists of four sections:

  • aerospike-clusters defines the location and access credentials for ABS to communicate to the Aerospike database, calling that cluster absCluster1. Since this is a Docker Compose stack, it uses the Aerospike Database Docker container name "aerospike-cluster" as the hostname instead of an IP address.
  • storage defines the location of the storage for database backups. Here, it creates a storage type called minioStorage that uses the s3-endpoint-override parameter to send backed up data to MinIO instead of Amazon S3. You can define multiple storage types that can later be used in multiple backup policies.
  • In backup-policies, a new policy called keepFilesPolicy is defined with simple instructions to run in a single thread and keep all previous backups. A policy is a set of instructions defining how to do a specific type of backup. You can define multiple policies that can be used in various backup routines.
    note

    Backup policies are defined in the configuration file. In contrast, restore policies are not defined beforehand; they are sent in the body of each restore request.

  • backup-routines specifies a routine called minioKeepFilesRoutine that runs the keepFilesPolicy policy daily for full backups and hourly for incremental backups. Routines specify the source cluster to back up data from, a storage type as defined under the storage section, and a namespace from the source cluster to back up. You can define multiple routines that can be run according to different schedules or on demand.
Default Docker Compose ABS Configuration:
aerospike-clusters:
absCluster1:
seed-nodes:
- host-name: "aerospike-cluster"
port: 3000
credentials:
user: admin
password: admin

storage:
minioStorage:
# Use "aws-s3" for S3 or compatible and "local" for local storage.
type: "aws-s3"
# as-backup-bucket is expected to exist in MinIO.
path: s3://as-backup-bucket/minioStorage
s3-region: eu-central-1
s3-profile: minio
s3-endpoint-override: http://minio:9000

backup-policies:
keepFilesPolicy:
# Run backup operations in a single thread.
parallel: 1
# Previous full backups are not deleted when a new one is created.
remove-files: KeepAll

backup-routines:
minioKeepFilesRoutine:
# 24 hours interval for full backups.
interval-cron: "@daily"
# 1 hour interval for incremental backups.
incr-interval-cron: "@hourly"
source-cluster: absCluster1
storage: minioStorage
namespace: test
backup-policy: keepFilesPolicy

Linux configuration

The default configuration file supplied with the Linux distributions is smaller and simpler than the configuration in the Docker Compose setup. By default, it sets up a connection to a namespace called "test" in an Aerospike database accessible at 127.0.0.1:3000. It stores backup files locally at /var/lib/aerospike-backup-service.

Default Linux ABS Configuration:
aerospike-clusters:
cluster1:
use-services-alternate: false
seed-nodes:
- host-name: "127.0.0.1"
port: 3000
credentials:
user: "admin"
password: "admin"

storage:
local1:
type: "local"
path: "/var/lib/aerospike-backup-service"

backup-policies:
policy1:
type: 1
parallel: 1
remove-files: KeepAll

backup-routines:
routine1:
interval-cron: "1/30 * * * * *" # every 30 seconds
incr-interval-cron: "1/5 * * * * *" # every 5 seconds
backup-policy: "policy1"
source-cluster: "cluster1"
storage: "local1"
namespace: "test"

aerospike-clusters

ParameterTypeDescriptionExample
conn-timeoutintegerThe connection timeout in milliseconds.5000
credentialsobjectThe authentication details to the Aerospike cluster.
labelstringThe cluster name.testCluster
seed-nodesarrayThe seed nodes details.
tlsobjectThe cluster TLS configuration.
use-services-alternatebooleanUse "services-alternate" instead of "services" in info request during cluster tending.false

credentials

aerospike-clusters -> credentials

ParameterTypeDescriptionExample
auth-modestringThe authentication mode string (INTERNAL, EXTERNAL, EXTERNAL_INSECURE, PKI).
passwordstringThe password for the cluster authentication.testPswd
password-pathstringThe file path with the password string, will take precedence over the password field./path/to/pass.txt
userstringThe username for the cluster authentication.testUser

tls

aerospike-clusters -> tls

ParameterTypeDescriptionExample
cafilestringPath to a trusted CA certificate file./path/to/cafile.pem
capathstringPath to a directory of trusted CA certificates./path/to/ca
certfilestringPath to the chain file for mutual authentication if Aerospike Cluster supports it./path/to/certfile.pem
cipher-suitestringTLS cipher selection criteria. The format is the same as OpenSSL's Cipher List Format.ECDHE-ECDSA-AES256-GCM-SHA384
keyfilestringPath to the key for mutual authentication (if Aerospike cluster supports it)./path/to/keyfile.pem
keyfile-passwordstringPassword to load protected TLS-keyfile (env:VAR, file:PATH, PASSWORD).file:/path/to/password
namestringThe default TLS name used to authenticate each TLS socket connection.tls-name
protocolsstringTLS protocol selection criteria. This format is the same as Apache's SSL Protocol.TLSv1.2

storage

ParameterTypeDescriptionExample
pathstringThe root path for the backup repository.backups
s3-endpoint-overridestringAn alternative endpoint for the S3 SDK to communicate (AWS S3 optional).http://host.docker.internal:9000
s3-log-levelstringThe log level of the AWS S3 SDK (AWS S3 optional).FATAL
s3-profilestringThe S3 profile name (AWS S3 optional).default
s3-regionstringThe S3 region string (AWS S3 optional).eu-central-1
typestringThe type of the storage provider, "local" or "aws-s3".local

backup-policies

ParameterTypeDescriptionExample
bandwidthintegerThrottles backup write operations to the backup file(s) to not exceed the given bandwidth in MiB/s.10000
compressionobjectCompression details.
encryptionobjectEncryption details.
file-limitintegerFile size limit (in MB) for the backup directory. If an .asb backup file crosses this size threshold, a new backup file is created.1024
max-recordsintegerAn approximate limit for the number of records to process. Available in server 4.9 and above.10000
max-retriesintegerMaximum number of retries before aborting the current transaction.3
no-binsbooleanOnly backup record metadata; digest, TTL, generation count, key.
no-indexesbooleanDo not back up any secondary index definitions.
no-recordsbooleanDo not back up any record data; metadata or bin data.
no-udfsbooleanDo not back up any UDF modules.
parallelintegerMaximum number of scan calls to run in parallel.1
records-per-secondintegerLimit total returned records per second (RPS). If RPS is zero (the default), the records-per-second limit is not applied.1000
remove-artifactsbooleanClear directory or remove output file.
remove-filesobjectWhether to clear the output directory (default: KeepAll).
retry-delayintegerRetryDelay defines the delay in milliseconds before retrying a failed operation.500
sealedbooleanSealed determines whether backup should include keys updated during the backup process. When true, the backup contains only records that last modified before backup started. When false (default), records updated during backup might be included in the backup, but it's not guaranteed.
socket-timeoutintegerSocket timeout in milliseconds. If this value is 0, it is set to total-timeout. If both are 0, there is no socket idle time limit.1000
total-timeoutintegerTotal socket timeout in milliseconds. Default is 0, that is, no timeout.2000

compression

backup-policies -> compression

ParameterTypeDescriptionExample
levelintegerThe compression level to use, or -1 if unspecified.
modestringThe compression mode to use, NONE (default) or ZSTD.NONE

encryption

backup-policies -> encryption

ParameterTypeDescriptionExample
key-envstringThe name of the environment variable containing the encryption key.
key-filestringThe path to the file containing the encryption key.
key-secretstringThe secret keyword in Aerospike Secret Agent containing the encryption key.
modestringThe encryption mode to use; NONE, AES128, or AES256.NONE

backup-routines

ParameterTypeDescriptionExample
backup-policystringThe name of the corresponding backup policy.daily
bin-listarrayThe list of backup bin names. Optional: An empty list implies backing up all bins.["dataBin"]
incr-interval-cronstringThe interval for incremental backup as a cron expression string (optional).*/10 * * * * *
interval-cronstringThe interval for full backup as a cron expression string.0 0 * * * *
namespacesarrayThe list of the namespaces to back up. Optional: Empty list implies backup of whole cluster.[`"source-ns1"]`
partition-liststringBack up list of partition filters. Partition filters can be ranges, individual partitions, or records after a specific digest within a single partition. Default number of partitions to back up: 0 to 4095: all partitions.0-1000
prefer-racksarrayA list of Aerospike Server rack IDs to prefer when reading records for a backup.[0]
secret-agentstringThe Secret Agent configuration for the routine (optional).sa
set-listarrayThe list of backup set names. Optional: An empty list implies backing up all sets.["set1"]
source-clusterstringThe name of the corresponding source cluster.testCluster
storagestringThe name of the corresponding storage provider configuration.aws