--- title: "ABS configuration file examples" description: "Examples and parameter guides for configuring the Aerospike Backup Service (ABS) using YAML." --- # ABS configuration file examples > For the complete documentation index see: [llms.txt](https://aerospike.com/docs/llms.txt) > > All documentation pages available in markdown. This page explains the sections of an example Aerospike Backup Service (ABS) configuration file, illustrating the basic requirements for a backup configuration. Aerospike publishes a [JSON schema for this configuration file](https://raw.githubusercontent.com/aerospike/aerospike-backup-service/refs/tags/v3.4.0/docs/config.schema.json) that enables autocomplete and validation in the VSCode and IntelliJ editors. View full configuration file ```yaml # yaml-language-server: $schema=https://raw.githubusercontent.com/aerospike/aerospike-backup-service/refs/tags/v3.4.0/docs/config.schema.json aerospike-clusters: abs-cluster: # <--- Custom user-defined cluster name seed-nodes: - host-name: localhost port: 3000 credentials: user: tester password: secrets:cluster-creds:password # Secret Agent path secret-agent-name: secret-agent # <--- Refers to the secret agent name under secret-agents secret-agents: secret-agent: # <--- Custom user-defined secret agent name address: localhost port: 5000 connection-type: tcp storage: s3: # <--- Custom user-defined storage name s3-storage: # Storage type; can be one of "local-storage", "s3-storage", "azure-storage", "gcp-storage" path: backups bucket: as-backup-bucket s3-region: eu-central-1 min-part-size: 5_242_880 # Minimum upload chunk size in bytes backup-policies: dailyBackupPolicy: # <--- Custom user-defined policy name parallel: 8 # Parallelism level (May affect performance) parallel-write: 8 # Write parallelism level (May affect performance) file-limit: 1000 # Max backup file size in MB (May affect performance) compression: # Backup files will be compressed before uploading (May affect performance) mode: ZSTD level: 3 concurrent-incremental: false with-cluster-configuration: false retention: full: 10 # Retain 10 full backups incremental: 5 # Retain incremental backups for the 5 latest full backups backup-routines: dailyLocalBackupRoutine: # <--- Custom routine name interval-cron: "@daily" # Full backup will be triggered daily at midnight incr-interval-cron: "0 */2 * * * *" # Incremental backups every 2 hours source-cluster: abs-cluster # <--- Refers to the cluster name under aerospike-clusters storage: s3 # <--- Refers to the storage name under storage backup-policy: dailyBackupPolicy # <--- Refers to the policy name under backup-policies namespaces: ["test"] # <--- Required field; use [] to back up the whole cluster ``` ## Configuration file components ### `aerospike-clusters` This example configuration connects to the Aerospike Database cluster named `abs-cluster`. The `seed-nodes` parameter defines the host and port for the Aerospike cluster on the network. The `credentials` parameter passes a plaintext username and a reference to a password stored in the Secret Agent. You can also use a plaintext username and password without the Secret Agent reference for testing purposes. If a credential value starts with `secrets:`, ABS treats it as a Secret Agent reference in the format `secrets:RESOURCE_LABEL:KEY`. For path syntax examples, see [Secret usage with backup and restore](https://aerospike.com/docs/database/tools/backup-and-restore/absctl/secrets). Configure Secret Agent connections for ABS in the [`secret-agents`](#secret-agents) section. The middle segment is a Secret Agent resource label you define when storing secrets in the agent. ::: note ABS does not support directly passing a Kubernetes Secret. ::: #### `max-parallel-scans` `max-parallel-scans` sets a per-cluster limit on the number of concurrent scans across all routines. ```yaml aerospike-clusters: abs-cluster: max-parallel-scans: 4 ``` #### `prefer-racks` Set `prefer-racks` on a cluster for routines that do not set `rack-list`, `node-list`, or `partition-list`. `prefer-racks` only affects which replica is read for each partition. ABS prefers nodes in `prefer-racks[0]` first, then `prefer-racks[1]`, and so on, but reads from other racks when a replica is not available on a preferred rack. Each backup is complete and independently restorable. This setting is mutually exclusive with `rack-list`, `node-list`, and `partition-list` on any routine that uses this cluster. Use `prefer-racks` on a single ABS deployment when you want a full backup that prefers local rack reads and reduces cross-datacenter traffic. For parallel backup sliced by rack, use [`rack-list`](#rack-list) on separate routines. See [Rack-aware backup: prefer-racks vs rack-list](#rack-aware-backup-prefer-racks-vs-rack-list). ```yaml aerospike-clusters: abs-cluster: # <--- Custom user-defined cluster name seed-nodes: - host-name: localhost port: 3000 credentials: user: tester password: secrets:cluster-creds:password # Secret Agent path secret-agent-name: secret-agent # <--- Refers to the Secret Agent name under secret-agents prefer-racks: [100, 101] ``` ### `secret-agents` The `secret-agents` section defines one or more connections to the [Aerospike Secret Agent](https://aerospike.com/docs/database/manage/security/secrets/#connecting-to-aerospike-secret-agent). ```yaml secret-agents: secret-agent: # <--- Custom user-defined secret agent name address: localhost port: 5000 connection-type: tcp ``` To configure TLS, use `ca-file` alone for server verification, or add `cert-file`, `key-file`, and `name` together for mutual TLS authentication: ```yaml secret-agents: secure-agent: address: secret-agent.example.com port: 3005 connection-type: tcp ca-file: /etc/ssl/certs/ca.pem # CA certificate to verify the server cert-file: /etc/ssl/certs/client-cert.pem # Client certificate for mutual TLS key-file: /etc/ssl/private/client-key.pem # Client private key for mutual TLS name: secret-agent.example.com # Required for mutual TLS (SNI) ``` ### `storage` The `storage` section sets up four different storage destinations: local, S3, Azure, and GCP. Each one has a separate label, here `storage1`, `storage2`, and so on. You can set up as many or as few destinations as you want in your own configuration. Note the specific authentication requirements for the different cloud storage providers. You can set an object storage class on S3, Azure, and GCP destinations. Use `storage-class.data` to choose the durability and availability tier for backup data, and `storage-class.metadata` (S3 and Azure only) to choose the tier for metadata files. Supported values depend on the provider, and `metadata` accepts a narrower list than `data` so that metadata stays quickly retrievable: - S3: `storage-class.data` accepts `STANDARD`, `GLACIER`, `STANDARD_IA`, `ONEZONE_IA`, `INTELLIGENT_TIERING`, `DEEP_ARCHIVE`, `OUTPOSTS`, `GLACIER_IR`, `SNOW`, and `EXPRESS_ONEZONE`. `storage-class.metadata` only accepts fast-retrieval classes: `STANDARD`, `STANDARD_IA`, `INTELLIGENT_TIERING`, `EXPRESS_ONEZONE`, `ONEZONE_IA`, and `OUTPOSTS`. - Azure Blob Storage: `storage-class.data` accepts `Hot`, `Cool`, `Cold`, and `Archive`. `storage-class.metadata` only accepts `Hot`, `Cool`, and `Cold`. - Google Cloud Storage: `storage-class.data` accepts `STANDARD`, `NEARLINE`, `COLDLINE`, and `ARCHIVE`. GCP has no configurable metadata tier, so ABS always stores GCP metadata in the `STANDARD` class. ```yaml storage: s3: # <--- Custom user-defined storage name s3-storage: # Storage type; can be one of "local-storage", "s3-storage", "azure-storage", "gcp-storage" path: backups bucket: as-backup-bucket s3-region: eu-central-1 min-part-size: 5_242_880 # Minimum upload chunk size in bytes ``` Additional storage examples: ```yaml storage: # Example 1: Local Storage storage1: local-storage: path: /local/backups # Example 2: S3 Storage storage2: s3-storage: bucket: my-backup-bucket path: backups s3-profile: default s3-region: eu-central-1 storage-class: data: STANDARD metadata: STANDARD # Example 3: Azure Storage storage3: azure-storage: account-name: my-storage-account account-key: my-secret-key container-name: my-container endpoint: 'https://my-storage-account.blob.core.windows.net' path: backups storage-class: data: Hot metadata: Hot # Example 4: GCP Storage storage4: gcp-storage: bucket-name: my-gcp-bucket key-file-path: /path/to/service-account-key.json endpoint: 'https://storage.googleapis.com' path: backups storage-class: data: STANDARD ``` ### `backup-policies` A backup policy is a set of parameters that define how to perform a backup. You can create many different policies, then schedule each to run at different times or under different circumstances. The following example defines a policy called `dailyBackupPolicy`, which is later scheduled to run daily in the `backup-routines` section. With `parallel`, it specifies the number of parallel reader threads that read Aerospike partitions. You can also specify the size of each backup file with `file-limit` in MB, as well as a compression algorithm and compression level with `compression`. Set `concurrent-incremental: true` to allow incremental backups to run alongside a full backup of the same routine. The default is `false`, which skips incremental backups while another backup for that routine is in progress. Set `with-cluster-configuration: true` to include cluster configuration in the backup. The default is `false`. In the `retention` section, this example specifies that the previous 10 full backups are retained in storage, in addition to any incremental backups made during the last 5 full backups. ABS 3.4 introduces two additional optional scan controls: `max-concurrent-nodes` limits how many nodes ABS scans at once, and `use-scan-compression` enables server-side scan compression (Aerospike Database Enterprise Edition only). Omit these fields to preserve the existing behavior of scanning every node in parallel without compression. ```yaml backup-policies: dailyBackupPolicy: # <--- Custom user-defined policy name parallel: 8 # Parallelism level (May affect performance) parallel-write: 8 # Write parallelism level (May affect performance) file-limit: 1000 # Max backup file size in MB (May affect performance) compression: # Backup files will be compressed before uploading (May affect performance) mode: ZSTD level: 3 concurrent-incremental: false with-cluster-configuration: false retention: full: 10 # Retain 10 full backups incremental: 5 # Retain incremental backups for the 5 latest full backups ``` ### `timestamp-format` Set [`timestamp-format`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#service.backup.timestamp-format) on `service.backup` to append a human-readable suffix to backup path timestamps. Allowed values are `ISO` (for example `2006-01-02T15-04-05`), `EU` (for example `02-Jan-2006-15-04-05`), and `US` (for example `Jan-02-2006-15-04-05`). If you omit this field, ABS uses the Unix epoch timestamp only. ### `backup-routines` The `backup-routines` section schedules one or more routines to run from specified sources at specified intervals. In this example, the routine `dailyLocalBackupRoutine` backs up the `test` namespace from source cluster `abs-cluster` to the storage destination `s3`, following the rules defined in `dailyBackupPolicy`. This routine performs full and incremental backups based on cron intervals. It performs an incremental backup every two hours and a full backup each day at midnight (UTC+0). ::: note All cron times are in the UTC+0 time zone. ABS does not support converting between time zones. ::: ```yaml backup-routines: dailyLocalBackupRoutine: # <--- Custom routine name interval-cron: "@daily" # Full backup will be triggered daily at midnight incr-interval-cron: "0 */2 * * * *" # Incremental backups every 2 hours source-cluster: abs-cluster # <--- Refers to the cluster name under aerospike-clusters storage: s3 # <--- Refers to the storage name under storage backup-policy: dailyBackupPolicy # <--- Refers to the policy name under backup-policies namespaces: ["test"] # <--- Required field; use [] to back up the whole cluster ``` ::: note ABS 3.4.0 and later rejects duplicate values in list-type configuration fields, including `seed-nodes` and `prefer-racks` on `aerospike-clusters`, and `namespaces`, `set-list`, `bin-list`, `rack-list`, and `node-list` on backup routines. ABS returns a validation error if any of these fields repeats the same value. ::: #### `node-list` Set `node-list` on a routine to back up partitions whose master is on one of the listed nodes when the job starts. Each node can be an IP address, hostname, or node ID with port. See [`node-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.node-list) in the configuration reference. Each routine with `node-list` is potentially partial: assign non-overlapping node slices across parallel ABS instances so together they cover every partition you intend to back up. See [Parallel backups](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/3.4.0/parallel-backup). `node-list` is mutually exclusive with `partition-list`, `rack-list`, and the cluster’s [`prefer-racks`](#prefer-racks) setting. If partition ownership changes during a long backup, scans can read from the current master outside the listed node, and partitions whose masters migrate onto a listed node after the job starts are not included in that routine’s backup. See the topology note under [`rack-list`](#rack-list). #### `rack-list` Set `rack-list` on a routine to back up partitions whose master is on a node in one of the listed racks when the job starts. Rack IDs are integers, such as `rack-list: [1]` or `rack-list: [1, 2]`. A single routine can list multiple rack IDs. ABS backs up the union of primary partitions whose masters are on nodes in any listed rack. For parallel deployment, assign one rack ID per routine so non-overlapping slices together cover the namespace. Each routine with `rack-list` is potentially partial. In a cluster with N racks and evenly distributed masters, one routine per rack often backs up about 1/N of the namespace. In namespaces with [active rack](https://aerospike.com/docs/database/manage/namespace/rack-aware#designate-an-active-rack) enabled, all masters sit on the active rack; routines that target other racks may back up few or no partitions. Parallel backup by rack assumes masters are spread across racks. With active rack, use a single full backup with `prefer-racks`, or slice by [`node-list`](#node-list) or [`partition-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.partition-list) instead. Run one routine per rack (on separate ABS instances or schedules) and restore all slices to recover the full data set. See [Parallel backups](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/3.4.0/parallel-backup). `rack-list` is mutually exclusive with `node-list`, `partition-list`, and the cluster’s [`prefer-racks`](#prefer-racks) setting. :::caution Topology changes during backup When the job starts, ABS resolves primary (master) partitions for nodes in the listed racks and scans that fixed partition set. If partition ownership changes during a long backup, scans can read from the current master for a selected partition, which may be on a different rack than when the job started. Plan capacity and cross-zone traffic accordingly. If a partition’s master migrates onto a listed rack after the job starts, that partition was not in the initial slice and is not included in that routine’s backup. Verify behavior for your cluster layout with ABS engineering if migrations run during backups. ::: ```yaml backup-routines: rack1Backup: interval-cron: "@daily" source-cluster: abs-cluster storage: s3 backup-policy: dailyBackupPolicy namespaces: ["test"] rack-list: [1] rack2Backup: interval-cron: "@daily" source-cluster: abs-cluster storage: s3 backup-policy: dailyBackupPolicy namespaces: ["test"] rack-list: [2] ``` ## Rack-aware backup: prefer-racks vs rack-list In a rack-aware namespace, each partition has one master copy on a single node. Other nodes may hold replica copies of the same partition. ABS uses your settings to decide which copy to read: - `prefer-racks`: Back up every partition in the namespace. For each partition, read from a replica on a preferred rack when one exists; otherwise read from another rack. - `rack-list` or `node-list`: Back up partitions whose master is on the listed racks or nodes when the job starts. Each routine covers a slice of the namespace; restore all slices for a full data set. See [Rack awareness](https://aerospike.com/docs/database/manage/namespace/rack-aware) for more information on masters, replicas, and racks. | Goal | Use | Do not use | | --- | --- | --- | | Full backup with preferred local rack reads | `prefer-racks` on the cluster (single routine) | `prefer-racks` on N parallel ABS instances | | Parallel backup sliced by rack | `rack-list` per routine (N routines) | `prefer-racks` per ABS instance | | Parallel backup sliced by host | [`node-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.node-list) | — | ::: caution Running one ABS instance per rack, each with `prefer-racks: [N]`, creates N independent full backups of the entire namespace. Restoring every slice together runs overlapping full restores and can make progress metrics misleading. Use [`rack-list`](#rack-list) per routine when you need non-overlapping shards. ::: For CLI equivalents, see [`absctl backup`](https://aerospike.com/docs/database/tools/backup-and-restore/absctl/backup/use#back-up-by-rack) (`--prefer-racks` vs `--rack-list`). ## GitHub repository example files This section explains the two sample configuration files included in the [ABS GitHub repository](https://github.com/aerospike/aerospike-backup-service/). Default Docker Compose ABS Configuration: The following sample backup service configuration supplied with the Docker Compose stack for ABS consists of four sections: - `aerospike-clusters` defines the location and access credentials for ABS to communicate with Aerospike Database, calling that cluster `absCluster1`. Since this is a Docker Compose stack, it uses the Aerospike Database Docker container name `"aerospike-cluster"` as the hostname instead of an IP address. - `storage` defines the location of the storage for database backups. Here, it creates a storage type called `minioStorage` that uses the `s3-endpoint-override` parameter to send backed up data to MinIO instead of Amazon S3. You can define multiple storage types that can later be used in multiple backup policies. - In `backup-policies`, a new policy called `keepFilesPolicy` is defined with simple instructions to run in a single thread and keep all previous backups. A policy is a set of instructions defining how to do a specific type of backup. You can define multiple policies that can be used in various backup routines. ::: note Backup policies are defined in the configuration file. In contrast, restore policies are not defined beforehand; they are sent in the body of each restore request. ::: - `backup-routines` specifies a routine called `minioKeepFilesRoutine` that runs the `keepFilesPolicy` policy daily for full backups and hourly for incremental backups. Routines specify the source cluster to back up data from, a storage type as defined under the `storage` section, and a namespace from the source cluster to back up. You can define multiple routines that can be run according to different schedules or on demand. ```yaml aerospike-clusters: absCluster1: seed-nodes: - host-name: "aerospike-cluster" port: 3000 credentials: user: admin password: admin storage: minioStorage: s3-storage: bucket: my-backup-bucket path: backups s3-profile: default s3-region: eu-central-1 s3-endpoint-override: http://minio:9000 backup-policies: keepFilesPolicy: # Run backup operations in a single thread. parallel: 1 backup-routines: minioKeepFilesRoutine: # 24 hours interval for full backups. interval-cron: "@daily" # 1 hour interval for incremental backups. incr-interval-cron: "@hourly" source-cluster: absCluster1 storage: minioStorage namespaces: ["test"] backup-policy: keepFilesPolicy ``` Default Linux ABS Configuration: The default configuration file supplied with Linux distributions is smaller and simpler than the configuration in the Docker Compose setup. By default, it sets up a connection to a namespace called `"test"` in an Aerospike Database cluster accessible at `127.0.0.1:3000`. It stores backup files locally at `/var/lib/aerospike-backup-service`. ```yaml aerospike-clusters: cluster1: use-services-alternate: false seed-nodes: - host-name: "127.0.0.1" port: 3000 credentials: user: "admin" password: "admin" storage: local: local-storage: path: /var/lib/aerospike-backup-service backup-policies: policy1: parallel: 1 backup-routines: routine1: interval-cron: "@weekly" incr-interval-cron: "@daily" backup-policy: "policy1" source-cluster: "cluster1" storage: "local" namespaces: ["test"] ```