ABS Configuration Reference
Aerospike Backup Service (ABS) reads the configuration file aerospike-backup-service.yml to create connections to Aerospike Database namespaces and storage destinations, as well as backup policies, routines, and schedules.
For the parameter list in OpenAPI format, see the Schemas section in the REST API specification.
Configuration options
Search here for individual configuration options. Expand the cards to see more details and/or usage examples.
Options
Connection timeout in milliseconds.
conn-timeout
5000The authentication mode used by the cluster.
INTERNAL, EXTERNAL, PKI
File path with the password string. Mutually exclusive with password field.
aerospike-clusters: cluster1: credentials: user: "admin" password-path: "/path/to/pass.txt"Plaintext password for cluster authentication. If it starts with secrets, path to password stored in Aerospike Secret Agent. Only use this plaintext authentication method for testing, never in production.
aerospike-clusters: cluster1: credentials: user: "admin" password: "admin"Name of secret agent to use to fetch password. Use a previously configured secret agent in the secret-agents configuration stanza.
aerospike-clusters: cluster1: credentials: secret-agent-name: "demoSa"Configuration for one or more secret agents to use to fetch the password.
aerospike-clusters: cluster1: credentials: secret-agent: ... <secret agent config here> ...Plaintext username for cluster authentication.
aerospike-clusters: cluster1: credentials: user: "admin" password: "admin"Authentication details in plain text for the Aerospike cluster.
aerospike-clusters: cluster1: credentials: user: "admin" password: "admin"A user-selected name for the cluster. It is used only in logs and error messages.
Maximum number of simultaneous allowed data reads, or scans, from the cluster during backup. This is a cluster-scope limit for all backup policies to prevent cluster overload.
aerospike-clusters: absDefaultCluster: seed-nodes: - host-name: "localhost" port: 3000 credentials: user: "tester" password: "psw" max-parallel-scans: 8Any non-negative integer
Host name to connect to the seed node for communication with the Aerospike Database cluster.
aerospike-clusters: absCluster1: seed-nodes: - host-name: "aerospike-cluster" port: 3000 credentials: user: admin password: adminPort to connect to the seed node for communication with the Aerospike Database cluster.
aerospike-clusters: absCluster1: seed-nodes: - host-name: "aerospike-cluster" port: 3000 credentials: user: admin password: adminOptional TLS certificate name used for secure connections.
A stanza with connection details of the seed nodes, or the nodes that Aerospike Backup Service uses to communicate with the cluster. On a multi-node cluster, you can supply connection information for a single node and the rest of the nodes will automatically be discovered.
aerospike-clusters: absCluster1: seed-nodes: - host-name: "aerospike-cluster" port: 3000Path to a trusted CA certificate file.
Path to a directory of trusted CA certificates.
Path to the chain file for mutual authentication if the Aerospike cluster supports it.
TLS cipher selection criteria. The format is the same as OpenSSL’s Cipher List Format.
Password to load protected TLS-keyfile (env:VAR, file:PATH, PASSWORD).
Path to the key for mutual authentication if the Aerospike cluster supports it.
Default TLS name used to authenticate each TLS socket connection.
TLS protocol selection criteria. This format is the same as Apache’s SSL Protocol.
A sub-stanza of a particular cluster with details about the cluster TLS configuration.
Whether to use “services-alternate” instead of “services” in info request during cluster tending.
Throttles backup write operations to the backup file(s) to not exceed the given bandwidth in MiB/s.
bandwidth: 10000Compression level to use, or -1 if unspecified.
Compression mode to use. Options are NONE (default) or ZSTD.
`NONE`NONE, ZSTD
Compression details.
Enables incremental backups to start while a full or incremental backup is ongoing.
Name of the environment variable containing the encryption key.
Path to the file containing the encryption key. The encryption key must be a valid PEM file.
encryption: mode: "AES256" key-file: "/data/aerospike-backup-service/etc/aerospike-backup-service/keyfile1.dat"Secret keyword in Aerospike Secret Agent containing the encryption key.
Encryption mode to use. Options are NONE, AES128, or AES256.
`NONE`NONE, AES128, AES256
Controls backup encryption information like the location of an encryption key or the keyword for Aerospike Secret Agent.
File size limit in MB for the backup file. If an .asb backup file crosses this size threshold, a new backup file is created.
file-limit: 1024Approximate limit for the number of records to process. Available in Database 4.9 and later.
max-records: 10000Maximum number of retries before aborting the current transaction. Removed in 3.0 and replaced with the retry-policy substanza.
"parallel":1, "max-retries": 3Only back up record metadata; digest, TTL, generation count, key.
Whether to back up secondary index definitions.
Whether to back up record data, metadata or bin data.
Whether to back up UDF modules.
Maximum number of threads to use for writing backup files. This defaults to the same value as parallel.
1Maximum number of scan calls to run in parallel. Each scan call processes a subset of the total data partitions. This value should always be equal to or less than aerospike-clusters.CLUSTER_NAME.max-parallel-scans.
1Limit total returned records per second (RPS). If RPS is zero (the default), the records-per-second limit is not applied.
1000Clear directory or remove output file.
The total number of full backups to retain. The minimum is 1, meaning each new full backup deletes the previous one. If not specified, all full backups are kept.
full: 5Any integer equal to or greater than 1
The number of most recent full backups for which incremental backups are retained. Cannot exceed the value of full. If omitted, all incremental backups are kept. A value of 0 means that all previous incremental backups will be deleted after each full backup is made.
incremental: 3Integer values equal to or less than 0 and equal to or less than full.
A stanza that optionally specifies retention rules for a given backup policy.
removeFilesPolicy: retention: full: 5 incremental: 3Delay in milliseconds to wait before retrying a failed operation.
500Initial delay between retry attempts, in milliseconds.
Maximum number of retry attempts that will be made. If set to 0, no retries will be performed.
Increases the delay between subsequent retry attempts. The actual delay is calculated as: BaseTimeout * (Multiplier ^ attemptNumber)
Define a policy in this section for the initial waiting time before a retry, number of retries, and a multiplier that extends the wait interval. A retry policy can also be sent as part of a JSON restore request.
backup-policies: example-backup-policy: ... # additional stanzas retry-policy: base-timeout: 1000 # 1000 milliseconds max-retries: 4 multiplier: 2 ... # additional stanzasWhether backup should include keys updated during the backup process. When true, the backup contains only records that were last modified before backup started. When false, records updated during backup may be included in the backup.
Socket timeout in milliseconds. If this value is 0, it is set to total-timeout. If both socket-timeout and total-timeout are 0, there is no socket idle time limit.
1000Total socket timeout in milliseconds.
2000Name of the corresponding backup policy.
Example backup routine containing a policy called keepFilesPolicy
backup-routines: minioRoutine: interval-cron: "@daily" incr-interval-cron: "@hourly" source-cluster: absCluster1 storage: minioStorage namespaces: ["test"] backup-policy: keepFilesPolicyList of bin names to back up. An empty list backs up all bins.
["dataBin"]Interval for incremental backup as a cron expression string. Intervals are extended cron (7 symbols: year, month, week, day, hour, minute, second). All times are in UTC time zone. Supported reserved words: @yearly, @monthly, @weekly, @daily, @hourly
*/10 * * * * *Interval for full backup as a cron expression string. Intervals are extended cron (7 symbols: year, month, week, day, hour, minute, second). All times are in UTC time zone. Supported reserved words: @yearly, @monthly, @weekly, @daily, @hourly
0 0 * * * *List of namespaces to back up. Empty list implies backup of whole cluster.
["source-ns1"]List of nodes to back up. Formatted as a list of IP addresses and/or host names followed by port numbers. Empty list implies backup of whole cluster.
IP_ADDRESS:PORT(,IP_ADDRESS:PORT,...)Filter that specifies individual partitions or a range of partitions to back up. Partition filters can be ranges, individual partitions, or records after a specific digest within a single partition.
- A range is specified as ”
- ”: “100-50” backs up 50 partitions starting from 100. - A single partition is specified as a number: “0” backs up the first partition at position 0.
- Multiple entries can be comma-separated: “0,100,200,300,400,500” backs up only the partitions in the list.
By default, all partitions are backed up. This field is mutually exclusive with node-list.
Back up partitions 100, 200, and a range of 400 partitions starting at 300.
100,200,300-400List of Aerospike Database rack IDs to prefer when reading records for a backup.
[0]The Secret Agent configuration for the routine (optional).
secret-agent-configurationList of set names to back up. An empty list backs up all sets.
["set1"]Name of the corresponding source cluster.
testClusterName of the corresponding storage provider configuration.
Example using minIO storage
storage: minioStorageA top-level stanza specifying the details of one or more backup routines.
Example routine called routine1
backup-routines: routine1: interval-cron: "1/30 * * * * *" incr-interval-cron: "1/5 * * * * *" backup-policy: "policy1" source-cluster: "cluster1" storage: "local1" namespaces: ["test"]A top-level stanza containing cluster objects for each of the Aerospike clusters that ABS is connected to.
aerospike-clusters: cluster1: use-services-alternate: false seed-nodes: - host-name: "127.0.0.1" port: 3000 credentials: user: "admin" password: "admin"The HTTP listen address.
Example service stanza with http and logger substanzas
service: http: address: 10.0.0.1 port: 8080Specifies a custom path for the ABS API endpoints.
Example service stanza with http and logger substanzas
service: http: address: 10.0.0.1 port: 8080The HTTP listen port.
Example service stanza with http stanza and port of 8080
service: http: address: 10.0.0.1 port: 8080Rate limiter token bucket size, also known as the burst threshold.
Example service stanza with all service.http.rate substanzas listed
service: http: address: 10.0.0.1 port: 8080 rate: size: tps: white-list:Rate limiter tokens per second threshold.
Example service stanza with all service.http.rate substanzas listed
service: http: address: 10.0.0.1 port: 8080 rate: size: tps: white-list:List of IP addresses allowed during rate limiting. All addresses are allowed by default.
Example service stanza with all service.http.rate substanzas listed
service: http: address: 10.0.0.1 port: 8080 rate: size: tps: white-list:HTTP rate limiter configuration.
Example service stanza with all service.http.rate substanzas listed
service: http: address: 10.0.0.1 port: 8080 rate: size: tps: white-list:Timeout for HTTP server operations in milliseconds.
Example service stanza with a 10-second timeout
service: http: address: 10.0.0.1 port: 8080 timeout: 10000Configures the HTTP server for ABS.
Example service stanza with http and logger substanzas
service: http: port: 9000 logger: level: INFO file-writer: filename: /var/log/aerospike-backup-service.logDetermines if the rotated log files should be compressed using gzip. The default is to not compress.
Example file-writer substanza enabling file compression with gzip
service: logger: level: INFO file-writer: filename: /var/log/aerospike-backup-service.log compress: trueLog destination.
Example service stanza with logger substanza
service: logger: level: INFO file-writer: filename: /var/log/aerospike-backup-service.logMaximum number of days to retain log files based on the timestamp in the filename. Default is to not remove log files based on age.
Example logger stanza set to remove backups after 30 days
service: logger: maxage: 30 file-writer: filename: /var/log/aerospike-backup-service.logMaximum number of log files to retain. The default is to retain all log files.
Example logger config keeping 10 old log files
service: logger: maxbackups: 10 file-writer: filename: /var/log/aerospike-backup-service.logMaximum size in megabytes of the log file before it gets rotated.
Example service stanza with logger substanza
service: logger: level: INFO file-writer: filename: /var/log/aerospike-backup-service.logSubstanza that configures the file writing protocol, such as the destination directory.
Example logger stanza with the filename specified
service: logger: level: INFO file-writer: filename: /var/log/aerospike-backup-service.logFormat is the logger format (PLAIN, JSON).
Example stanza setting the format to JSON
service: logger: level: INFO format: JSON file-writer: filename: /var/log/aerospike-backup-service.logPLAIN, JSON
Log level.
Example logger stanza setting the level to INFO
service: logger: level: INFO format: PLAIN file-writer: filename: /var/log/aerospike-backup-service.logTRACE, DEBUG, INFO, WARN, WARNING, ERROR
Whether to enable logging to the standard output.
Example service stanza with logger substanza
service: logger: level: INFO stdout-writer: false file-writer: filename: /var/log/aerospike-backup-service.logConfigures the ABS logger, such as enabling the logs, setting the log level, and setting the log format.
Example service stanza with logger substanza
service: logger: level: INFO file-writer: filename: /var/log/aerospike-backup-service.logTop-level stanza that configures the operation of the Aerospike Backup Service, including logging and HTTP endpoint.
Example service stanza with http and logger substanzas
service: http: port: 9000 logger: level: INFO file-writer: filename: /var/log/aerospike-backup-service.logAzure storage account key for Shared Key authentication. This is sensitive information. Can be a path in secret agent or an actual value.
string
Azure storage account name for Shared Key authentication.
string
Azure Active Directory client ID for AAD authentication.
string
Azure Active Directory client secret for AAD authentication. This is sensitive information. Can be a path in secret agent or an actual value.
string
Name of the Azure Blob container.
string
Azure Blob service endpoint URL.
string
The minimum size in bytes of individual chunks uploaded during a multipart upload.
integer
Root path for the backup repository within the container. If not specified, backups will be saved in the container’s root.
path: backupsstring
Secret Agent configuration (optional). Link to a preconfigured agent. Mutually exclusive with secret-agent.
string
Secret Agent object definition. Mutually exclusive with secret-agent-name.
Azure Active Directory tenant ID for AAD authentication.
string
Stanza defining Azure storage options. Mutually exclusive with other storage objects. Can also take a secret-agent substanza.
storage: azure-blob-storage-example: azure-storage: endpoint: http://127.0.0.1:6000/devstoreaccount1 container-name: testcontainer path: backups account-name: devstoreaccount1GCP storage bucket name.
string
Alternative URL. It is not recommended to use an alternate URL in a production environment.
string
Path to the file containing the service account key in JSON format.
string
Service account key in JSON format. This is sensitive information. Can be a path in secret agent or an actual value.
string
The minimum size in bytes of individual chunks uploaded during a multipart upload.
integer
Root path for the backup repository. If not specified, backups will be saved in the bucket’s root.
path: backupsstring
Secret Agent configuration (optional). Link to one of preconfigured agents. Mutually exclusive with secret-agent.
string
Secret Agent object definition. Mutually exclusive with secret-agent-name.
object
Stanza defining GCP storage options. Mutually exclusive with other storage objects. Can also take a secret-agent substanza.
storage: gcp-storage-example: gcp-storage: key-file-path: key-file.json bucket-name: gcp-backup-bucket path: backups endpoint: http://127.0.0.1:9020Root path for the backup directory when backing up locally.
storage: example-local-storage-object: local-storage: path: backupsStanza defining local storage options. Mutually exclusive with other storage objects.
storage: example-local-storage-object: local-storage: path: backupsRoot path for the backup directory.
path: backupsAlternative endpoint for the S3 SDK to communicate (AWS S3 optional).
s3-endpoint-override: http://host.docker.internal:9000Log level of the AWS S3 SDK (AWS S3 optional).
s3-log-level: FATALThe S3 profile name (AWS S3 optional).
s3-profile: defaultS3 region string (AWS S3 optional).
s3-region: eu-central-1Access Key ID for authentication with S3 StaticCredentialsProvider. This is sensitive information. Can be a path in secret agent or an actual value.
string
The S3 bucket name.
string
The maximum number of simultaneous requests allowed from S3. This parameter was named max_async_connections until 3.1
integer
The minimum size in bytes of individual chunks uploaded during a multipart upload. This parameter was named min_part_size until version 3.1.
integer
The root path for the backup repository within the bucket. If not specified, backups will be saved in the bucket’s root.
path: backupsstring
An alternative endpoint for the S3 SDK to communicate (AWS S3 optional).
s3-endpoint-override: http://host.docker.internal:9000string
The log level of the AWS S3 SDK (AWS S3 optional).
string
The S3 profile name (AWS S3 optional).
string
The S3 region string.
region: eu-central-1string
Secret Access Key for authentication with S3 StaticCredentialsProvider. This is sensitive information. Can be a path in Aerospike Secret Agent or an actual value.
string
Secret Agent configuration (optional). Link to one of preconfigured agents. Mutually exclusive with secret-agent.
string
Secret Agent object definition. Mutually exclusive with secret-agent-name.
Stanza defining AWS S3 storage options. Mutually exclusive with other storage objects. Can also take a secret-agent substanza.
storage: aws-s3-example-object: s3-storage: bucket: as-backup-bucket path: backups s3-region: eu-central-1Type of the storage provider.
type: localA top-level stanza with options for configuring local or remote storage. Define a custom name for each storage object you create, then use further parameters to define each object’s details.
storage: # Example 1: Local Storage storage1: local-storage: path: /local/backups # Example 2: S3 Storage storage2: s3-storage: bucket: my-backup-bucket path: backups s3-profile: default s3-region: eu-central-1