Loading
API Usage Examples
Use this section as a guide to get familiar with the most common operations performed with the Aerospike Backup Service (ABS) REST API.
Read configurations
This section details how to fetch configurations for clusters, policies, and storage options. This is useful for setting up or verifying the configuration of your system.
Get cluster configuration
This endpoint returns the configurations of existing clusters, including the default cluster setup with seed nodes and credentials.
Request:
GET {{baseUrl}}/v1/config/clusters
Response:
{
"absDefaultCluster": {
"seed-nodes": [
{
"host-name": "host.docker.internal",
"port": 3000
}
],
"credentials": {
"user": "tester",
"password": "psw"
}
}
}
Full schema:
{
"Config": {
"description": "Config represents the service configuration file.",
"aerospike-clusters": {
"<*>": {
"description": "AerospikeCluster represents the configuration for an Aerospike cluster for backup.",
"conn-timeout": 5000, // The connection timeout in milliseconds.
"credentials": {
"description": "Authentication details to the Aerospike cluster.",
"auth-mode": "INTERNAL", // Authentication mode string (INTERNAL, EXTERNAL, EXTERNAL_INSECURE, PKI).
"password": "testPswd", // Password for the cluster authentication.
"password-path": "/path/to/pass.txt", // The file path with the password string, will take precedence over the password field.
"user": "testUser" // The username for the cluster authentication.
},
"label": "testCluster", // The cluster name.
"seed-nodes": [
{
"description": "SeedNode represents details of a node in the Aerospike cluster.",
"host-name": "localhost", // The host name of the node. (required)
"port": 3000, // Port of the node. (required)
"tls-name": "certName" // TLS certificate name used for secure connections (if enabled).
}
],
"tls": {
"description": "The cluster TLS configuration.",
"cafile": "/path/to/cafile.pem", // Path to a trusted CA certificate file.
"capath": "/path/to/ca", // Path to a directory of trusted CA certificates.
"certfile": "/path/to/certfile.pem", // Path to the chain file for mutual authentication (if Aerospike Cluster supports it).
"cipher-suite": "ECDHE-ECDSA-AES256-GCM-SHA384", // TLS cipher selection criteria. The format is the same as OpenSSL's Cipher List Format.
"keyfile": "/path/to/keyfile.pem", // Path to the key for mutual authentication (if Aerospike cluster supports it).
"keyfile-password": "file:/path/to/password", // Password to load protected TLS-keyfile (env:VAR, file:PATH, PASSWORD).
"name": "tls-name", // Default TLS name used to authenticate each TLS socket connection.
"protocols": "TLSv1.2" // TLS protocol selection criteria. This format is the same as Apache's SSL Protocol.
},
"use-services-alternate": true // Whether should use "services-alternate" instead of "services" in info request during cluster tending.
}
},
"backup-policies": {
"<*>": {
"description": "BackupPolicy represents a scheduled backup policy.",
"bandwidth": 10000, // Throttles backup write operations to the backup file(s) to not exceed the given bandwidth in MiB/s.
"compression": {
"description": "Compression details.",
"level": -1, // The compression level to use (or -1 if unspecified).
"mode": "NONE" // The compression mode to be used (default is NONE).
},
"encryption": {
"description": "Encryption details.",
"key-env": "ENCRYPTION_KEY_ENV", // Name of the environment variable containing the encryption key.
"key-file": "/path/to/keyfile", // Path to the file containing the encryption key.
"key-secret": "SECRET_KEY", // Secret keyword in Aerospike Secret Agent containing the encryption key.
"mode": "NONE" // Encryption mode to be used (NONE, AES128, AES256).
},
"file-limit": 1024, // File size limit (in MB) for the backup directory. If an .asb backup file crosses this size threshold, a new backup file will be created.
"max-records": 10000, // Approximate limit for the number of records to process. Available in Database 4.9 and above.
"max-retries": 3, // Maximum number of retries before aborting the current transaction.
"no-bins": false, // Only backup record metadata (digest, TTL, generation count, key).
"no-indexes": false, // Do not back up any secondary index definitions.
"no-records": false, // Do not back up any record data (metadata or bin data).
"no-udfs": false, // Do not back up any UDF modules.
"parallel": 1, // Maximum number of scan calls to run in parallel.
"records-per-second": 1000, // Limit total returned records per second RPS. If RPS is zero (the default), the records-per-second limit is not applied.
"remove-artifacts": false, // Clear directory or remove output file.
"remove-files": {
"description": "Whether to clear the output directory (default: KeepAll)."
},
"retry-delay": 500, // RetryDelay defines the delay in milliseconds before retrying a failed operation.
"sealed": false, // Sealed determines whether backup should include keys updated during the backup process. When true, the backup contains only records that last modified before backup started. When false (default), records updated during backup might be included in the backup, but it's not guaranteed.
"socket-timeout": 1000, // Socket timeout in milliseconds. If this value is 0, it is set to total-timeout. If both are 0, there is no socket idle time limit.
"total-timeout": 2000 // Total socket timeout in milliseconds. Default is 0, that is, no timeout.
}
},
"backup-routines": {
"<*>": {
"description": "BackupRoutine represents a scheduled backup operation routine.",
"backup-policy": "daily", // Name of the corresponding backup policy. (required)
"bin-list": [ "dataBin" ], // List of backup bin names (optional, an empty list implies backing up all bins).
"incr-interval-cron": "*/10 * * * * *", // Interval for incremental backup as a cron expression string (optional).
"interval-cron": "0 0 * * * *", // Interval for full backup as a cron expression string. (required)
"namespaces": [ "source-ns1" ], // List of the namespaces to back up (optional, empty list implies backup up whole cluster).
"partition-list": "0-1000", // Back up list of partition filters. Partition filters can be ranges, individual partitions, or records after a specific digest within a single partition. Default number of partitions to back up: 0 to 4095: all partitions.
"prefer-racks": [ 0 ], // List of Aerospike Database rack IDs to prefer when reading records for a backup.
"secret-agent": "sa", // Secret Agent configuration for the routine (optional).
"set-list": [ "set1" ], // List of backup set names (optional, an empty list implies backing up all sets).
"source-cluster": "testCluster", // Name of the corresponding source cluster. (required)
"storage": "aws" // Name of the corresponding storage provider configuration. (required)
}
},
"secret-agent": {
"<*>": {
"description": "SecretAgent represents the configuration of an Aerospike Secret Agent for a backup/restore operation.",
"address": "localhost", // Address of the Secret Agent.
"port": "8080", // Port the Secret Agent is running on.
"timeout": 5000, // Timeout in milliseconds.
"tls-ca": "/path/to/ca.pem", // Path to a trusted CA certificate file in PEM format.
"tls-enabled": true // Indicates whether TLS is enabled.
}
},
"service": {
"description": "BackupServiceConfig represents the backup service configuration properties.",
"http": {
"description": "HTTPServer is the backup service HTTP server configuration.",
"address": "0.0.0.0", // Address to listen on.
"context-path": "/", // ContextPath customizes path for the API endpoints.
"port": 8080, // Port to listen on.
"rate": {
"description": "HTTP rate limiter configuration.",
"size": 1024, // Rate limiter token bucket size (bursts threshold).
"tps": 1024, // Rate limiter tokens per second threshold.
"white-list": [ "127.0.0.1" ] // List of ips to whitelist in rate limiting.
}
},
"logger": {
"description": "Logger is the backup service logger configuration.",
"capture-shared": false, // Whether to capture logs from the shared libraries.
"file-writer": {
"description": "File writer logging configuration.",
"compress": false, // Compress determines if the rotated log files should be compressed using gzip. Default is not to perform compression.
"filename": "log.txt", // Filename is the file to write logs to.
"maxage": 30, // MaxAge is the maximum number of days to retain old log files based on the timestamp encoded in their filename. Default is not to remove old log files based on age.
"maxbackups": 5, // MaxBackups is the maximum number of old log files to retain. Default is to retain all old log files.
"maxsize": 100 // MaxSize is the maximum size in megabytes of the log file before it gets rotated.
},
"format": "PLAIN", // Format is the logger format (PLAIN, JSON).
"level": "DEBUG", // Level is the logger level.
"stdout-writer": true // Whether to enable logging to the standard output.
}
},
"storage": {
"<*>": {
"description": "Storage represents the configuration for a backup storage details.",
"path": "backups", // The root path for the backup repository. (required)
"s3-endpoint-override": "http://host.docker.internal:9000", // An alternative endpoint for the S3 SDK to communicate (AWS S3 optional).
"s3-log-level": "FATAL", // Log level of the AWS S3 SDK (AWS S3 optional).
"s3-profile": "default", // S3 profile name (AWS S3 optional).
"s3-region": "eu-central-1", // S3 region string (AWS S3 optional).
"type": 0 // Type of the storage provider (0 - Local, 1 - AWS S3). (required)
}
}
}
}
Get routine configuration
Retrieves all configured backup routines.
Request:
GET {{baseUrl}}/v1/config/routines
Response:
{
"routine1": {
"backup-policy": "keepFilesPolicy",
"source-cluster": "absDefaultCluster",
"storage": "local",
"interval-cron": "@yearly",
"namespaces": ["source-ns7"]
},
"routine2": {
"backup-policy": "removeFilesPolicy",
"source-cluster": "absDefaultCluster",
"storage": "local",
"interval-cron": "@yearly",
"namespaces": ["source-ns8"],
"set-list": ["backupSet"],
"bin-list": ["backupBin"]
}
}
Get storage configuration
Returns all the configured storage endpoints, including, if applicable, cloud storage endpoint information such as region and path.
Request:
GET {{baseUrl}}/v1/config/storage
Response:
{
"local": {
"type": 0,
"path": "./localStorage"
},
"minio": {
"type": 1,
"path": "s3://as-backup-bucket/storage1",
"s3-region": "eu-central-1",
"s3-profile": "minio",
"s3-endpoint-override": "http://host.docker.internal:9000"
}
}
Retrieve full backup list
Provides a list of backups for each configured routine, including details such as creation time, namespace, and storage location.
Request:
GET {{baseUrl}}/v1/backups/full
Response:
{
"routine1": [
{
"created": "2024-03-14T13:13:28.96962301Z",
"from": "0001-01-01T00:00:00Z",
"namespace": "source-ns7",
"byte-count": 48,
"file-count": 1,
"Key": "s3://as-backup-bucket/storage1/minio/backup/1710422008983/source-ns4"
}
],
"routine2": [
{
"created": "2024-03-14T13:13:29.105744927Z",
"from": "0001-01-01T00:00:00Z",
"namespace": "source-ns8",
"byte-count": 48,
"file-count": 1,
"key": "localStorage/filterBySetAndBin/backup/source-ns8"
}
]
}
Get information about a running backup
Shows information about currently running backups for a specific backup routine, including number of records completed, start time, progress toward completion, and estimated end time.
Request:
Provide the name of the backup routine you want to examine in your request.
GET {{baseUrl}}/v1/backups/currentBackup/ROUTINE_NAME
Response:
This example response shows two backups in progress, one full and one incremental.
{
"full": {
"done-records": 50,
"estimated-end-time": "2024-01-02T15:10:05Z07:00",
"percentage-done": 50,
"start-time": "2024-01-02T15:04:05Z07:00",
"total-records": 100
},
"incremental": {
"done-records": 50,
"estimated-end-time": "2024-01-02T15:10:05Z07:00",
"percentage-done": 50,
"start-time": "2024-01-02T15:04:05Z07:00",
"total-records": 100
}
}
Restore backup (direct restoration)
Direct restore using a specific backup
The destination field says where to restore to. It can be one of the clusters from the Get Cluster Configuration section or any other Aerospike cluster.
This request restores a backup from a specified path to a designated destination.
The no-generation parameter allows overwriting of existing keys if set to true.
In the source section, path is the key value returned as a response in the Full Backup List example. The type parameter under source denotes S3 storage if set to 1 and local storage if set to 0.
Request:
POST {{baseUrl}}/v1/restore/full
Request body:
{
"destination": {
"seed-nodes": [
{
"host-name": "localhost",
"port": 3000
}
],
"credentials": {
"user": "tester",
"password": "psw"
}
},
"policy": {
"no-generation": "true"
},
"source": {
"path": "s3://as-backup-bucket/storage1/minio/backup/1710422008983/source-ns4",
"type": 1,
"s3-region": "eu-central-1"
}
}
The response is a job ID. You can get the status of this job with the endpoint GET {{baseUrl}}/v1/restore/status/:JOB_ID.
Response:
123456789
By default, a restore policy includes a retry policy that configures ABS to retry a failed restore operation after 2000 milliseconds. You can configure how many times ABS should retry a failed restore before returning an error.
Full schema:
{
"RestoreRequest": {
"description": "RestoreRequest represents a restore operation request.",
"destination": {
"description": "AerospikeCluster represents the configuration for an Aerospike cluster for backup.",
"conn-timeout": 5000,
"credentials": {
"description": "Authentication details to the Aerospike cluster.",
"auth-mode": "INTERNAL", // Authentication mode string (INTERNAL, EXTERNAL, EXTERNAL_INSECURE, PKI).
"password": "testPswd", // Password for the cluster authentication.
"password-path": "/path/to/pass.txt", // The file path with the password string, takes precedence over the password field.
"user": "testUser" // The username for the cluster authentication.
},
"label": "testCluster", // The cluster name.
"seed-nodes": [
{
"description": "SeedNode represents details of a node in the Aerospike cluster.",
"host-name": "localhost", // The host name of the node. (required)
"port": 3000, // Port of the node. (required)
"tls-name": "certName" // TLS certificate name used for secure connections (if enabled).
}
],
"tls": {
"description": "The cluster TLS configuration.",
"cafile": "/path/to/cafile.pem", // Path to a trusted CA certificate file.
"capath": "/path/to/ca", // Path to a directory of trusted CA certificates.
"certfile": "/path/to/certfile.pem", // Path to the chain file for mutual authentication (if Aerospike Cluster supports it).
"cipher-suite": "ECDHE-ECDSA-AES256-GCM-SHA384", // TLS cipher selection criteria. The format is the same as OpenSSL's Cipher List Format.
"keyfile": "/path/to/keyfile.pem", // Path to the key for mutual authentication (if Aerospike cluster supports it).
"keyfile-password": "file:/path/to/password", // Password to load protected TLS-keyfile (env:VAR, file:PATH, PASSWORD).
"name": "tls-name", // Default TLS name used to authenticate each TLS socket connection.
"protocols": "TLSv1.2" // TLS protocol selection criteria. This format is the same as Apache's SSL Protocol.
},
"use-services-alternate": true // Whether should use "services-alternate" instead of "services" in info request during cluster tending.
},
"policy": {
"description": "RestorePolicy represents a policy for the restore operation.",
"bandwidth": 50000, // Throttles read operations from the backup file(s) to not exceed the given I/O bandwidth in MiB/s and its database write operations to not exceed the given number of transactions per second.
"batch-size": 128, // The max allowed number of records per an async batch write call. Default is 128 with batch writes enabled, or 16 without batch writes.
"bin-list": [ "bin1", "bin2" ], // The bins to restore (optional, an empty list implies restoring all bins).
"disable-batch-writes": false, // Disables the use of batch writes when restoring records to the Aerospike cluster. By default, the cluster is checked for batch write support.
"max-async-batches": 32, // The max number of outstanding async record batch write calls at a time.
"namespace": {
"description": "details", // Namespace details for the restore operation. By default, the data is restored to the namespace from which it was taken.
"destination": "destination-ns", // Destination namespace name. (required)
"source": "source-ns" // Original namespace name. (required)
},
"no-generation": false, // Records from backups take precedence. This option disables the generation check. With this option, records from the backup always overwrite records that already exist in the namespace, regardless of generation numbers.
"no-indexes": false, // Do not restore any secondary index definitions.
"no-records": false, // Do not restore any record data (metadata or bin data). By default, record data, secondary index definitions, and UDF modules will be restored.
"no-udfs": false, // Do not restore any UDF modules.
"parallel": 8, // The number of client threads to spawn for writing to the cluster.
"replace": false, // Replace records. This controls how records from the backup overwrite existing records in the namespace. By default, restoring a record from a backup only replaces the bins contained in the backup; all other bins of an existing record remain untouched.
"retry-policy": {
"baseTimeout", 5000 // Milliseconds before retrying the first time
"maxRetries", 5 // Number of times to retry
"multiplier", 2 // The baseTimeout is multiplied by this number between retries to prolong the retry length. Therefore, keep the multiplier and maximum retry values low to avoid excessively long retry periods.
} // Configuration of retries for each restore write operation. If nil, no retries will be performed.
"set-list": [ "set1", "set2" ], // The sets to restore (optional, an empty list implies restoring all sets).
"timeout": 1000, // Timeout (ms) for Aerospike commands to write records, create indexes and create UDFs.
"tps": 4000, // Throttles read operations from the backup file(s) to not exceed the given I/O bandwidth in MiB/s and its database write operations to not exceed the given number of transactions per second.
"unique": false // Existing records take precedence. With this option, only records that do not exist in the namespace are restored, regardless of generation numbers. If a record exists in the namespace, the record from the backup is ignored.
},
"secret-agent": {
"description": "SecretAgent represents the configuration of an Aerospike Secret Agent for a backup/restore operation.",
"address": "localhost", // Address of the Secret Agent.
"port": "8080", // Port the Secret Agent is running on.
"timeout": 5000, // Timeout in milliseconds.
"tls-ca": "/path/to/ca.pem", // Path to a trusted CA certificate file in PEM format.
"tls-enabled": true // Indicates whether TLS is enabled.
},
"source": {
"description": "Storage represents the configuration for a backup storage details.",
"path": "backups", // The root path for the backup repository. (required)
"s3-endpoint-override": "http://host.docker.internal:9000", // An alternative endpoint for the S3 SDK to communicate (AWS S3 optional).
"s3-log-level": "FATAL", // Log level of the AWS S3 SDK (AWS S3 optional).
"s3-profile": "default", // S3 profile name (AWS S3 optional).
"s3-region": "eu-central-1", // S3 region string (AWS S3 optional).
"type": 0 // Type of the storage provider (0 - Local, 1 - AWS S3). (required)
}
}
}
Restore using routine name and timestamp
This option restores the most recent full backup for the given timestamp and then applies all subsequent incremental backups up to that timestamp.
In this example, the destination and policy fields are the same as in the previous example.
Request:
POST {{baseUrl}}/v1/restore/timestamp
Request body:
{
"destination": {
"seed-nodes": [
{
"host-name": "localhost",
"port": 3000
}
],
"credentials": {
"user": "tester",
"password": "psw"
}
},
"policy": {
"no-generation": "true"
},
"routine": "routine1",
"time": "1710671632452"
}
The response is a job ID. You can get job status with the endpoint GET {{baseUrl}}/v1/restore/status/:JOB_ID.
Response:
123456789