Back up data with Aerospike Backup Service

For the complete documentation index see: llms.txt

All documentation pages available in markdown.

This page is the operator guide for running backups with Aerospike Backup Service (ABS). For the product model (routines, incremental chains, skip rules), see Concepts. For YAML field detail, see Configure ABS.

Applies to

Product: Aerospike Backup Service (ABS) 3.7.x.
Audience: database administrators and operators who schedule, trigger, and verify backups.
Outcome: the backup job finishes, and GET /v1/backups/full lists a new backup whose record-count matches the records you expected.

Set BASE_URL to your ABS listen address:

export BASE_URL=http://localhost:8080

Before you start

ABS is installed and can reach the Aerospike Database cluster. See Install.
The Database user has read privilege or higher on the namespaces you back up. See Privileges.
Storage credentials can create objects in the backup destination.

Create a routine

A routine names the cluster, storage, policy, namespaces, and schedule. This example backs up the app namespace daily, with incremental backups every two hours.

aerospike-clusters:
  prod:
    seed-nodes:
      - host-name: 10.0.1.10
        port: 3000
    credentials:
      user: backup-user
      password: secrets:cluster-creds:password
      secret-agent-name: secret-agent

secret-agents:
  secret-agent:
    address: secret-agent.example.com
    port: 3005
    connection-type: TCP

storage:
  s3-backups:
    s3-storage:
      bucket: company-as-backups
      path: prod
      s3-region: us-east-1
      min-part-size: 52428800

backup-policies:
  daily:
    parallel: 8
    parallel-write: 8
    file-limit: 1000
    compression:
      mode: ZSTD
      level: 3
    incr-mode: differential
    retention:
      full: 14
      incremental: 7

backup-routines:
  app-daily:
    interval-cron: "@daily"
    incr-interval-cron: "0 0 0/2 * * ? *"
    source-cluster: prod
    storage: s3-backups
    backup-policy: daily
    namespaces: ["app"]

Cron expressions are Quartz (Seconds Minutes Hours Day-of-Month Month Day-of-Week, optional Year). @hourly, @daily, @weekly, @monthly, and @yearly are also valid. Cron fields follow schedule-timezone (UTC if you omit it). On-demand delay and list from/to are milliseconds and are not converted. See Quartz schedules.

namespaces: [] backs up every namespace in the cluster. Optional set-list and bin-list restrict the backup to named sets or bins. Omit incr-mode or set it to differential unless you want cumulative incrementals.

The Database user needs read or higher. See Concepts.

Choose a retention policy

The retention stanza on a backup policy limits how many backups ABS keeps in storage. If you omit retention, ABS keeps every backup and storage use grows without bound.

Use these guidelines to size it:

retention.full is the number of full backups to keep. The minimum is 1, which means each new full backup deletes the previous one and leaves a single restore point.
retention.incremental is the number of most recent full backups for which ABS keeps incremental backups. It can’t exceed full. A value of 0 deletes all earlier incremental backups after each full backup. If you set only full, ABS applies the same count to incrementals.
Set full to cover your recovery point objective. For example, with a daily full backup, full: 14 lets you restore to any full backup from the last 14 days. Incremental backups let you restore to points between those full backups, and you usually need them only for the most recent full backups, so a smaller incremental value saves storage.
ABS applies retention in the background after each successful full backup. It doesn’t delete backups when an incremental backup completes, and it never deletes preemptively. Provision storage for at least one more full backup than full, because the new backup exists before ABS removes the oldest one.
An on-demand full backup also triggers retention, so it can remove older backups sooner than the schedule would.

backup-policies:
  daily:
    retention:
      full: 14 # Keep 14 full backups
      incremental: 3 # Keep incrementals for the 3 latest full backups

See retention in the configuration reference.

Filter records with an expression

The filter-exp field on a routine applies an Aerospike filter expression during the backup query. Only matching records are written to the backup.

Filter expressions are not plain text. You cannot write age > 25 in YAML. Build the expression with an Aerospike client library, encode it as Base64, and paste that string into the routine. filter-exp is mutually exclusive with a multi-set set-list (use one set, or omit set-list to include all sets).

import as "github.com/aerospike/aerospike-client-go/v8"

exp, err := as.ExpGreater(as.ExpIntBin("age"), as.ExpIntVal(25)).Base64()
// exp == "kwOTUQKjYWdlGQ=="

import com.aerospike.client.exp.Exp;
import com.aerospike.client.exp.Expression;

Expression filter = Exp.build(Exp.gt(Exp.intBin("age"), Exp.val(25)));
System.out.println(filter.getBase64());
// prints "kwOTUQKjYWdlGQ=="

from aerospike_helpers.expressions import GT, IntBin

# Assuming client is an active aerospike.Client connection
filter_exp = GT(IntBin("age"), 25).compile()
b64 = client.get_expression_base64(filter_exp)
print(b64)
# prints "kwOTUQKjYWdlGQ=="

If you already have a Base64-encoded filter expression, paste it into filter-exp.

backup-routines:
  adultsBackup:
    interval-cron: "@daily"
    source-cluster: prod
    storage: s3-backups
    backup-policy: daily
    namespaces: ["test"]
    set-list: ["users"]
    filter-exp: "kwOTUQKjYWdlGQ=="  # age > 25

Filter	Base64 value
`age > 25`	`kwOTUQKjYWdlGQ==`
`country = "US"`	`kwGTUQOnY291bnRyeaMDVVM=`

After the job finishes, compare record-count on GET /v1/backups/full with the number of records you expected the expression to select.

Apply configuration

Write the file ABS already loads (package default: /etc/aerospike-backup-service/aerospike-backup-service.yml), then apply it.

Reload without a process restart:

curl -X POST "$BASE_URL/v1/config/apply"

Restart the process when you change anything in the service section, such as the HTTP or HTTPS listener, logging, or service.backup settings:

systemctl restart aerospike-backup-service

Jobs that are already running keep the configuration they started with. New jobs use the applied configuration.

You can also change clusters, storage, policies, and routines through the configuration API (PUT/POST under /v1/config/...). GET responses redact literal secrets as "[secret]". Leave that sentinel in place when you PUT an unchanged secret. See API usage examples.

Confirm the routine is loaded

curl -s "$BASE_URL/v1/config/routines"
curl -s "$BASE_URL/v1/config/clusters"

The routine name in those documents is the name you pass to backup and restore endpoints.

Trigger an on-demand backup

Full backup:

curl -X POST "$BASE_URL/v1/backups/full/app-daily"

Incremental backup (requires a completed full backup):

curl -X POST "$BASE_URL/v1/backups/incremental/app-daily"

A 202 Accepted response means ABS queued the job. An optional delay query parameter waits that many milliseconds before the job starts.

The legacy POST /v1/backups/schedule/ROUTINE_NAME endpoint still triggers a full backup. Prefer /v1/backups/full/.

Watch a running backup

curl -s "$BASE_URL/v1/backups/currentBackup/app-daily"

While a full backup runs, the response includes a full object with total-records, done-records, percentage-done, and metrics.pipeline. After the job finishes, full is omitted and last-full is updated.

metrics.pipeline is a REST field on this status object, not a Prometheus metric. Use it when tuning throughput. See Performance tuning.

List backups

curl -s "$BASE_URL/v1/backups/full"
curl -s "$BASE_URL/v1/backups/incremental"

Optional from and to query parameters filter by Unix time in milliseconds.

The list contains only backups in the storage configured for the routine. After a storage change, historical objects in the old destination do not appear here.

A successful list entry includes key (the storage path of the backup), record-count, namespace, created, and finished. Copy key when you restore from a path. See Restore data.

Verify a backup

Confirm HTTP 202 on the trigger request.
Wait until GET /v1/backups/currentBackup/ROUTINE_NAME no longer includes full or incremental.
Confirm GET /v1/backups/full shows a new entry whose record-count matches the namespace.
Optionally confirm aerospike_backup_service_backup_events_total{outcome="success"} increased. See Monitor ABS.

If the job never appears, check skip outcomes: no prior full backup, overlap with a running job, or a disabled routine. See Monitor ABS.

Pause and resume a routine

Disable scheduling without deleting the routine:

curl -X PUT "$BASE_URL/v1/config/routines/app-daily/disable"

Enable it again:

curl -X PUT "$BASE_URL/v1/config/routines/app-daily/enable"

You can also set disabled: true on the routine in the YAML file and apply the configuration. interval-cron itself is required, so do not comment it out.

Cancel a running backup

curl -X POST "$BASE_URL/v1/backups/cancel/app-daily"

ABS stops running full and incremental jobs for that routine and deletes partial backup artifacts. Prometheus records outcome="canceled" when the scrape sees the event.