Skip to content

Aerospike Backup Service concepts

For the complete documentation index see: llms.txt

All documentation pages available in markdown.

Aerospike Backup Service (ABS) is a standalone service that backs up and restores Aerospike Database clusters. This page describes the model behind it: what the moving parts are and how a backup behaves once it runs.

For the YAML that expresses this model, see Configure ABS. For every field and its default, see the configuration reference.

Applies to

  • Product: Aerospike Backup Service (ABS) 3.7.x.
  • Audience: operators, SREs, and developers who are new to ABS or who design backup routines.
  • Prerequisites: none.
  • Outcome: you can name the objects a routine ties together, explain how full and incremental backups form a restore chain, and list the database privileges that backup and restore require.

How ABS fits your deployment

ABS runs as its own process alongside the database. Nothing is installed on the Aerospike Database nodes, and no part of ABS runs inside the database.

PartRole
Aerospike DatabaseHolds your records. ABS reads from it during a backup and writes to it during a restore.
ABSSchedules and runs backup and restore jobs, tracks their progress, and serves the REST API and metrics.
Backup storageHolds the backup files: local disk, Amazon S3, Google Cloud Storage, or Azure Blob Storage.

During a backup, ABS reads records as an ordinary database client and never writes to the source cluster. A restore writes only to the cluster the request names as its destination.

ABS keeps no separate catalog of backups. It discovers what exists by scanning the storage configured for each routine, so a replacement instance pointed at the same storage with the same routine names finds the backups that are already there.

The objects a backup routine ties together

You give names to a few kinds of object, and a backup routine is the one ABS schedules.

ObjectWhat it is
Cluster (aerospike-clusters)How ABS connects to an Aerospike Database cluster (seed nodes, credentials, optional TLS).
Storage (storage)Where backup files are written (local disk, Amazon S3, Google Cloud Storage, or Azure Blob Storage).
Backup policy (backup-policies)How a backup runs (parallelism, compression, encryption, retention, compact, incremental mode).
Backup routine (backup-routines)What to back up, from which cluster, to which storage, on which schedule.
Secret Agent (secret-agents)Optional connections to Aerospike Secret Agent, which supplies passwords and encryption keys instead of storing them in plain text. Clusters, storage destinations, and routines can each reference an agent by name.

A routine names one cluster, one storage destination, and usually one policy. Those objects must exist before a routine can reference them, and one cluster, storage destination, or policy can be reused by many routines.

Process-level settings such as the listen address, HTTP and HTTPS timeouts, schedule timezone, and logging live in a separate service section. They change how ABS runs, not what it backs up. See service.

Restore is not one of these objects. There is no restore routine: each restore request carries its own policy and names the storage to read from and the cluster to write to. See Restore data.

Full and incremental backups

A full backup copies the selected namespaces, or every namespace in the cluster when the routine’s namespace list is empty (namespaces: []). ABS reads records as a database client and never writes to the source cluster.

An incremental backup copies records that changed after a lower-bound timestamp. Incremental backups do not run until at least one full backup has completed successfully. The lower bound depends on incr-mode on the backup policy:

incr-modeWhat the next incremental copies
differential (default)Records changed since the last successful backup, full or incremental.
cumulative (ABS 3.7.0 and later)Records changed since the last successful full backup.

The schedule, the incremental/ storage layout, and the backup APIs are the same for both modes. Cumulative incrementals grow until the next full backup; restore-by-timestamp then needs fewer files. See Restore needs from the chain.

A restore by timestamp loads the latest full backup before that time, then applies incremental backups according to incr-mode. The backup timestamp is the time the job started, not the time it finished. The restore time field is Unix time in milliseconds (an absolute instant). schedule-timezone does not change how that instant is interpreted.

Backup files use the Aerospike backup format (.asb). Record data, secondary indexes, set indexes, and user-defined functions (UDFs) are stored in that format unless the policy sets no-records, no-indexes, or no-udfs. When ABS writes a directory backup, metadata such as UDFs and indexes can appear in separate metadata_*.asb files.

metadata.yaml and the incremental chain

On startup or after POST /v1/config/apply, ABS scans each routine’s storage for ROUTINE_NAME/backup/ and ROUTINE_NAME/incremental/ and reads the newest metadata.yaml files.

  • If a full backup metadata.yaml exists, ABS uses it as the baseline.
  • For incr-mode: differential, if a later incremental metadata.yaml exists, ABS uses that later timestamp as the next incremental’s lower bound.
  • For incr-mode: cumulative, the next incremental’s lower bound is the last successful full backup, even when later incrementals exist.
  • If ABS finds no metadata.yaml, it skips incremental jobs until a full backup completes. Logs include incremental backup skipped with a reason such as backup skipped: initial full backup not yet completed.

ABS does not copy backup files when you change a routine’s storage destination. After the change, ABS sees only what already exists in the new destination. See Change backup storage destination.

Complete backups versus cluster slices

By default a routine reads every partition in the namespaces it selects, and the resulting backup is complete and independently restorable.

A routine can instead be restricted to a slice of the cluster, chosen by rack, by node, or by partition range. Each slice is potentially partial: no single slice restores the whole namespace, and you must restore every slice to recover the full data set. Slices let several ABS instances back up one cluster in parallel. See Parallel backups.

Preferring a rack is a different idea. The backup still covers every partition; only the choice of which replica ABS reads changes. See Rack-aware backup: prefer-racks vs rack-list.

When to use a routine versus on-demand

Routines always own the namespace (and optional set, bin, and filter-expression) filters. The schedule (interval-cron, incr-interval-cron) is how ABS starts jobs unattended. Cron fields are evaluated in UTC unless you set schedule-timezone. See Quartz schedules.

POST /v1/backups/full/ROUTINE_NAME and POST /v1/backups/incremental/ROUTINE_NAME start the same job definition immediately. Use on-demand backups for tests, after a storage change, or when you need a baseline before a maintenance window.

An incremental backup is valid only after a successful full backup for that routine, and only while metadata.yaml in the configured storage still describes that chain.

Schedule overlap

ABS does not queue a skipped run. The next scheduled time tries again.

  • Two full backups for the same routine do not overlap. If a full backup is still running when the next full backup is due, ABS skips the new full backup.
  • A running incremental backup does not block a full backup. The full backup starts and the incremental job continues.
  • By default, ABS skips a new incremental backup if another backup for the same routine is running, or if a full backup is scheduled at the same time. Set concurrent-incremental to true on the policy to allow those overlaps.
  • A successful incremental backup that finds no changed records is not a skip. ABS records outcome="success" and does not write a new metadata.yaml for namespaces with no changes.

See Monitor ABS for skip, cancel, retry, and empty-incremental signals.

Restore needs from the chain

To restore to time T with incr-mode: differential, storage must still contain:

  1. The full backup that started most recently before T.
  2. Every incremental backup between that full backup and T.

With incr-mode: cumulative, restore-by-timestamp uses that full backup plus the latest incremental that covers T. Intermediate incrementals in the same cycle are not required. The trade-off is more data copied on each incremental run.

Retention (retention.full and retention.incremental on the policy) deletes older backups after a successful full backup. Size retention so a restore still has a complete chain for your recovery point objective.

Timestamp restore and path restore are described in Restore data.

Privileges

The Aerospike Database user that ABS uses must be able to read (backup) or write (restore) the target namespaces. These credentials belong to the cluster the routine or restore request names. See aerospike-clusters.

  • Backup: read or higher.
  • Restore of records: read-write.
  • Restore of UDFs: udf-admin (Database 6.0.0 and later) or data-admin.
  • Restore of secondary indexes and set indexes: sindex-admin (Database 6.0.0 and later) or data-admin.

See Configuring access control.