Parallel backups

You can run two or more identical Aerospike Backup Service (ABS) instances, each pointed at a different slice of the cluster, to back up and restore the full data set in parallel. You can declare the cluster slice by racks with rack-list, by nodes with node-list, or by partitions with partition-list.

ABS treats every backup routine as an independent job definition.

If you configure two backup routines on separate ABS instances that share the same source cluster and storage destination, and differ only in the slice each routine reads, you can start both routines at the same time, double the aggregate scan bandwidth, and still end up with a complete and non-overlapping backup set.

rack-list, node-list, and partition-list are mutually exclusive fields.

Prerequisites

• A running Aerospike Database deployment
• Two or more ABS instances
• asbench (for generating test data)
• asadm (for verifying test data restoration)

Configuration file sections

The following example shows the minimum content necessary for the first part of the configuration file, aerospike-backup-service.yml.

The aerospike-clusters and storage sections are identical between ABS instances in a parallel deployment. Only the cluster slice changes between the two backup routines in the backup-routines section.

aerospike-clusters:
  absDefaultCluster:
    seed-nodes:
      - host-name: 10.0.48.7
        port: 4333
        tls-name: asd.aerospike.com
    credentials:
      user: tester
      password: psw
    tls:
      name: asd.aerospike.com
      ca-file: /etc/ssl/certs/ca.aerospike.com.pem

storage:
  gcp:
    gcp-storage:
      bucket-name: abs-testing-bucket
      path: parallel-backup-test

See ABS configuration file examples and ABS configuration file reference for more example configuration files and information about additional parameters.

Backup routines

Each tab below shows two backup routines that differ only in their cluster slice.

Assign each routine a slice that does not overlap the others: together they must cover every partition you intend to back up, with no partition scanned by more than one routine.

Slice by nodes with `node-list`

First ABS instance YAML file

backup-routines:
  fullBackupA:
    source-cluster: absDefaultCluster
    storage: gcp
    interval-cron: '@daily'
    namespaces: [source-ns1]
    node-list:
      - 10.0.48.7:asd.aerospike.com:4333
      - 10.0.48.8:asd.aerospike.com:4333

Second ABS instance YAML file

backup-routines:
  fullBackupB:
    source-cluster: absDefaultCluster
    storage: gcp
    interval-cron: '@daily'
    namespaces: [source-ns1]
    node-list:
      - 10.0.48.9:asd.aerospike.com:4333
      - 10.0.48.10:asd.aerospike.com:4333
      - 10.0.48.11:asd.aerospike.com:4333

node-list limits each routine to the partitions owned by the listed nodes at the moment the job starts. Use node-list when you run multiple ABS instances in parallel and want each instance to back up a fixed set of cluster nodes, such as nodes in one availability zone or in one data center.

Slice by partitions with `partition-list`

First ABS instance YAML file

backup-routines:
  fullBackupA:
    source-cluster: absDefaultCluster
    storage: gcp
    interval-cron: '@daily'
    namespaces: [source-ns1]
    partition-list: "0-2048"

Second ABS instance YAML file

backup-routines:
  fullBackupB:
    source-cluster: absDefaultCluster
    storage: gcp
    interval-cron: '@daily'
    namespaces: [source-ns1]
    partition-list: "2048-2048"

partition-list limits each routine to the partitions in the listed ranges. Use partition-list when you want fixed partition ranges per ABS instance—for example, an even 50/50 split—without tying the slice to current node or rack ownership.

When you specify partition ranges, use the syntax start-count, where the first number is the partition to start at, and the second number is the number of partitions to include. There are 4096 partitions, so splitting at 0-2048 and 2048-2048 covers all partitions across fullBackupA and fullBackupB.

Slice by racks with `rack-list`

First ABS instance YAML file

backup-routines:
  fullBackupA:
    source-cluster: absDefaultCluster
    storage: gcp
    interval-cron: '@daily'
    namespaces: [source-ns1]
    rack-list:
      - 1

Second ABS instance YAML file

backup-routines:
  fullBackupB:
    source-cluster: absDefaultCluster
    storage: gcp
    interval-cron: '@daily'
    namespaces: [source-ns1]
    rack-list:
      - 2

rack-list limits each routine to the partitions owned by nodes in the listed rack IDs. This field requires ABS 3.4.0 or later. Use rack-list when your cluster already assigns nodes to racks for availability zones or data centers and you want each ABS instance to align with that layout. Rack IDs are non-negative integers configured on the Aerospike cluster. The example uses 1 and 2 as placeholders; replace them with the rack IDs from your deployment.

Note

rack-list differs from prefer-racks on the cluster configuration.

• With rack-list, only partitions owned by the specified racks are backed up.

• With prefer-racks, all partitions are backed up, but reads are directed to the specified racks when possible to reduce cross-datacenter traffic.

rack-list is mutually exclusive with prefer-racks, node-list, and partition-list.

Example parallel backup and restore workflow

This section outlines a series of steps to write test records to a namespace, set up parallel backups with ABS, run the backups, and restore the database.

Follow this end-to-end workflow to populate a namespace with test data, configure parallel backups using ABS, and verify the process through a full database restore.

Execute a parallel backup test

1. Generate test data using asbench.

   The benchmark tool writes one million 1 KiB records into the namespace:

   asbench -U tester -P psw -h 10.0.48.7:asd.aerospike.com:4333 -n source-ns1 -w I -k 1000000 -o B960 --tls-enable --tls-cafile=/etc/ssl/certs/ca.aerospike.com.pem

   See the asbench documentation for more information.

2. Start each backup routine separately through the REST API:

   curl -X POST http://service1:8080/v1/backups/full/fullBackupA
   curl -X POST http://service2:8080/v1/backups/full/fullBackupB

   A POST request to the /v1/backups/full/{routine} endpoint triggers an immediate full backup, regardless of how the backup routine is scheduled in its interval-cron parameter. The legacy /v1/backups/schedule/{routine} endpoint is deprecated, but still supported. Each routine writes to parallel-backup-test/fullBackupA/<timestamp> or parallel-backup-test/fullBackupB/<timestamp>.

3. Inspect the metadata.yaml file in each namespace’s subdirectory inside the backup output directory. Their record-count values should sum to the expected total of 1000000. ABS writes one metadata file per namespace, per routine.

Restore test data

1. Truncate the target cluster so the restore starts without any data present. Run the three following commands in sequence.

   asadm -h 10.0.48.7:asd.aerospike.com:4333 -U tester -P psw --tls-enable --tls-cafile=/etc/ssl/certs/ca.aerospike.com.pem
   
   Admin> enable
   
   Admin+> manage truncate ns source-ns1

   The manage truncate command removes all records immediately.

2. Create separate request bodies for your restore requests.

   restoreA.json

   {
       "routine": "fullBackupA",
       "time": '"$(($(date +%s)*1000))"',
       "destination": {
         "seed-nodes": [
           {
             "host-name": "10.0.48.7",
             "port": 4333,
             "tls-name": "asd.aerospike.com"
           }
         ],
         "credentials": {
           "user": "tester",
           "password": "psw"
         },
         "tls": {
           "name": "asd.aerospike.com",
           "ca-file": "/etc/ssl/certs/ca.aerospike.com.pem"
         }
       },
       "policy": {
         "parallel": 8,
         "batch-size": 100
       }
     }

   restoreB.json

   {
       "routine": "fullBackupB",
       "time": '"$(($(date +%s)*1000))"',
       "destination": {
         "seed-nodes": [
           {
             "host-name": "10.0.48.7",
             "port": 4333,
             "tls-name": "asd.aerospike.com"
           }
         ],
         "credentials": {
           "user": "tester",
           "password": "psw"
         },
         "tls": {
           "name": "asd.aerospike.com",
           "ca-file": "/etc/ssl/certs/ca.aerospike.com.pem"
         }
       },
       "policy": {
         "parallel": 8,
         "batch-size": 100
       }
     }

3. Restore each routine’s backup with the JSON requests you defined in the previous step.

   curl -X POST http://service1:8080/v1/restore/2025-04-10T12:34:02Z -H "Content-Type: application/json" -d @restoreA.json
   
   curl -X POST http://service2:8080/v1/restore/2025-04-10T12:34:28Z -H "Content-Type: application/json" -d @restoreB.json

   The JSON body follows the schema in the API examples and includes parallel and batch-size under policy to control writer concurrency.

   Note

   You can also pass the raw JSON content after the -d flag rather than saving it as a file.

4. Verify the object counts:

   asadm -h 10.0.48.7:4333 -U tester -P psw --tls-enable --tls-cafile=/etc/ssl/certs/ca.aerospike.com.pem
   Admin> info

   Each of the five nodes should report about 200,000 objects and zero migrations.

Scaling guidelines

ABS exposes a cluster-wide throttle with the max-parallel-scans parameter. Keep this value below the total vCPU count of the Aerospike nodes. Adjust per-reader threads with parallel in a backup policy and per-writer threads with parallel-write. Adjust writer batch size with batch-size in the restore request.

You can monitor throughput via the built-in /metrics endpoint, and basic health with /health and /ready.