--- title: "Parallel backups" description: "Run parallel Aerospike backups using multiple ABS instances and cluster slicing by nodes, racks, or partitions." --- # Parallel backups > For the complete documentation index see: [llms.txt](https://aerospike.com/docs/llms.txt) > > All documentation pages available in markdown. You can run two or more identical Aerospike Backup Service (ABS) instances, each pointed at a different slice of the cluster, to back up and restore the full data set in parallel. You can declare the cluster slice by racks with [`rack-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.rack-list), by nodes with [`node-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.node-list), or by partitions with [`partition-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.partition-list). ABS treats every backup routine as an independent job definition. If you configure two backup routines on separate ABS instances that share the same source cluster and storage destination, and differ only in the slice each routine reads, you can start both routines at the same time, double the aggregate scan bandwidth, and still end up with a complete and non-overlapping backup set. `rack-list`, `node-list`, and `partition-list` are mutually exclusive fields. For parallel backup by rack, use routine-level `rack-list` on separate ABS instances. Omit cluster-level [`prefer-racks`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#aerospike-clusters.CLUSTER_NAME.prefer-racks) when a routine sets `rack-list`, `node-list`, or `partition-list`. See [Rack-aware backup: prefer-racks vs rack-list](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config-examples#rack-aware-backup-prefer-racks-vs-rack-list). :::caution Topology changes during backup When the job starts, ABS resolves primary (master) partitions for the listed racks or nodes and scans that fixed partition set. If partition ownership changes during a long backup, scans can read from the current master for a selected partition, which may be on a different rack or node than when the job started. If a partition’s master migrates onto a listed rack or node after the job starts, that partition was not in the initial slice and is not included in that routine’s backup. See the topology note in [ABS configuration file examples](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config-examples#rack-list). ::: ## Prerequisites - A running Aerospike Database deployment - Two or more ABS instances - `asbench` (for generating test data) - `asadm` (for verifying test data restoration) ## Configuration file sections The following example shows the minimum content necessary for the first part of the configuration file, `aerospike-backup-service.yml`. The `aerospike-clusters` and `storage` sections are identical between ABS instances in a parallel deployment. Only the cluster slice changes between the two backup routines in the `backup-routines` section. ```yaml aerospike-clusters: absDefaultCluster: seed-nodes: - host-name: 10.0.48.7 port: 4333 tls-name: asd.aerospike.com credentials: user: tester password: psw tls: name: asd.aerospike.com ca-file: /etc/ssl/certs/ca.aerospike.com.pem storage: gcp: gcp-storage: bucket-name: abs-testing-bucket path: parallel-backup-test ``` See [ABS configuration file examples](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config-examples) and [ABS configuration file reference](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config) for more example configuration files and information about additional parameters. ## Backup routines Each tab below shows two backup routines that differ only in their cluster slice. Assign each routine a slice that does not overlap the others: together they must cover every partition you intend to back up, with no partition scanned by more than one routine. - [Slice by nodes with \`node-list\`](#tab-panel-3004) - [Slice by partitions with \`partition-list\`](#tab-panel-3005) - [Slice by racks with \`rack-list\`](#tab-panel-3006) First ABS instance YAML file ```yaml backup-routines: fullBackupA: source-cluster: absDefaultCluster storage: gcp interval-cron: '@daily' namespaces: [source-ns1] node-list: - 10.0.48.7:asd.aerospike.com:4333 - 10.0.48.8:asd.aerospike.com:4333 ``` Second ABS instance YAML file ```yaml backup-routines: fullBackupB: source-cluster: absDefaultCluster storage: gcp interval-cron: '@daily' namespaces: [source-ns1] node-list: - 10.0.48.9:asd.aerospike.com:4333 - 10.0.48.10:asd.aerospike.com:4333 - 10.0.48.11:asd.aerospike.com:4333 ``` [`node-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.node-list) limits each routine to partitions whose master is on one of the listed nodes when the job starts. Use `node-list` when you run multiple ABS instances in parallel and want each instance to back up a fixed set of cluster nodes, such as nodes in one availability zone or datacenter. Each routine is potentially partial: together, non-overlapping `node-list` slices must cover every partition you intend to back up. If partition ownership changes during a long backup, see the topology note in [ABS configuration file examples](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config-examples#rack-list). First ABS instance YAML file ```yaml backup-routines: fullBackupA: source-cluster: absDefaultCluster storage: gcp interval-cron: '@daily' namespaces: [source-ns1] partition-list: "0-2048" ``` Second ABS instance YAML file ```yaml backup-routines: fullBackupB: source-cluster: absDefaultCluster storage: gcp interval-cron: '@daily' namespaces: [source-ns1] partition-list: "2048-2048" ``` [`partition-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.partition-list) limits each routine to the partitions in the listed ranges. Use `partition-list` when you want fixed partition ranges per ABS instance—for example, an even 50/50 split—without tying the slice to current node or rack ownership. When you specify partition ranges, use the syntax `start-count`, where the first number is the partition to start at, and the second number is the number of partitions to include. There are 4096 partitions, so splitting at 0-2048 and 2048-2048 covers all partitions across `fullBackupA` and `fullBackupB`. First ABS instance YAML file ```yaml backup-routines: fullBackupA: source-cluster: absDefaultCluster storage: gcp interval-cron: '@daily' namespaces: [source-ns1] rack-list: - 1 ``` Second ABS instance YAML file ```yaml backup-routines: fullBackupB: source-cluster: absDefaultCluster storage: gcp interval-cron: '@daily' namespaces: [source-ns1] rack-list: - 2 ``` [`rack-list`](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config#backup-routines.ROUTINE_NAME.rack-list) limits each routine to partitions whose master is on a node in one of the listed racks when the job starts. This field requires ABS 3.4.0 or later. Use `rack-list` when your cluster assigns nodes to racks for availability zones or data centers and you want each ABS instance to back up one rack slice in parallel. Each routine is potentially partial. In a cluster with N racks and evenly distributed masters, one routine per rack often backs up about 1/N of the namespace. With [active rack](https://aerospike.com/docs/database/manage/namespace/rack-aware#designate-an-active-rack) enabled, all masters sit on the active rack; routines for other racks may back up few or no partitions. Rack IDs are non-negative integers configured on the Aerospike cluster. The example uses `1` and `2` as placeholders on separate ABS instances; replace them with the rack IDs from your deployment. Restore all slices for a full data set. See [ABS configuration file examples](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config-examples#rack-list) and [Rack-aware backup: prefer-racks vs rack-list](https://aerospike.com/docs/database/tools/backup-and-restore/backup-service/config-examples#rack-aware-backup-prefer-racks-vs-rack-list). ## Example parallel backup and restore workflow This section outlines a series of steps to write test records to a namespace, set up parallel backups with ABS, run the backups, and restore the database. Follow this end-to-end workflow to populate a namespace with test data, configure parallel backups using ABS, and verify the process through a full database restore. ### Execute a parallel backup test 1. Generate test data using `asbench`. The benchmark tool writes one million 1 KiB records into the namespace: Terminal window ```shell asbench -U tester -P psw -h 10.0.48.7:asd.aerospike.com:4333 -n source-ns1 -w I -k 1000000 -o B960 --tls-enable --tls-cafile=/etc/ssl/certs/ca.aerospike.com.pem ``` See the [`asbench` documentation](https://aerospike.com/docs/database/tools/asbench) for more information. 2. Start each backup routine separately through the REST API: Terminal window ```shell curl -X POST http://service1:8080/v1/backups/full/fullBackupA curl -X POST http://service2:8080/v1/backups/full/fullBackupB ``` A POST request to the `/v1/backups/full/{routine}` endpoint triggers an immediate full backup, regardless of how the backup routine is scheduled in its `interval-cron` parameter. The legacy `/v1/backups/schedule/{routine}` endpoint is deprecated, but still supported. Each routine writes to `parallel-backup-test/fullBackupA/` or `parallel-backup-test/fullBackupB/`. 3. Inspect the `metadata.yaml` file in each namespace’s subdirectory inside the backup output directory. Their `record-count` values should sum to the expected total of `1000000`. ABS writes one metadata file per namespace, per routine. ### Restore test data 1. Truncate the target cluster so the restore starts without any data present. Run the three following commands in sequence. Terminal window ```shell asadm -h 10.0.48.7:asd.aerospike.com:4333 -U tester -P psw --tls-enable --tls-cafile=/etc/ssl/certs/ca.aerospike.com.pem Admin> enable Admin+> manage truncate ns source-ns1 ``` The `manage truncate` command removes all records immediately. 2. Create separate request bodies for your restore requests. restoreA.json ```json { "routine": "fullBackupA", "time": '"$(($(date +%s)*1000))"', "destination": { "seed-nodes": [ { "host-name": "10.0.48.7", "port": 4333, "tls-name": "asd.aerospike.com" } ], "credentials": { "user": "tester", "password": "psw" }, "tls": { "name": "asd.aerospike.com", "ca-file": "/etc/ssl/certs/ca.aerospike.com.pem" } }, "policy": { "parallel": 8, "batch-size": 100 } } ``` restoreB.json ```json { "routine": "fullBackupB", "time": '"$(($(date +%s)*1000))"', "destination": { "seed-nodes": [ { "host-name": "10.0.48.7", "port": 4333, "tls-name": "asd.aerospike.com" } ], "credentials": { "user": "tester", "password": "psw" }, "tls": { "name": "asd.aerospike.com", "ca-file": "/etc/ssl/certs/ca.aerospike.com.pem" } }, "policy": { "parallel": 8, "batch-size": 100 } } ``` 3. Restore each routine’s backup with the JSON requests you defined in the previous step. Terminal window ```shell curl -X POST http://service1:8080/v1/restore/2025-04-10T12:34:02Z -H "Content-Type: application/json" -d @restoreA.json curl -X POST http://service2:8080/v1/restore/2025-04-10T12:34:28Z -H "Content-Type: application/json" -d @restoreB.json ``` The JSON body follows the schema in the API examples and includes `parallel` and `batch-size` under `policy` to control writer concurrency. ::: note You can also pass the raw JSON content after the `-d` flag rather than saving it as a file. ::: 4. Verify the object counts: Terminal window ```shell asadm -h 10.0.48.7:4333 -U tester -P psw --tls-enable --tls-cafile=/etc/ssl/certs/ca.aerospike.com.pem Admin> info ``` Each of the five nodes should report about 200,000 objects and zero migrations. ## Scaling guidelines ABS exposes a cluster-wide throttle with the `max-parallel-scans` parameter. Keep this value below the total vCPU count of the Aerospike nodes. Adjust per-reader threads with `parallel` in a backup policy and per-writer threads with `parallel-write`. Adjust writer batch size with `batch-size` in the restore request. You can monitor throughput via the built-in `/metrics` endpoint, and basic health with `/health` and `/ready`.