Install and Test ABS

Overview

This page describes how to install Aerospike Backup Service (ABS) on Docker Compose or directly on a Linux installation, either on bare metal or as a virtual machine.

Install ABS with Docker Compose

Docker Compose is a way to package a set of Docker containers that are set up to communicate with each other automatically. The Docker Compose stack that Aerospike provides at the ABS GitHub repository is an example, bare-bones configuration that you can later modify according to your own needs. It includes an Aerospike database, ABS, and a storage container.

By starting with ABS on Docker Compose using the provided sample stack, you can quickly test out backup policies and different configuration parameters in a matter of minutes before going further and eventually integrating ABS into a more complex environment on your own.

Download and set up ABS on Docker Compose

Run Docker Compose. If you already have Docker Desktop, then your installation includes Compose and you may directly launch Docker Desktop. If you installed the other Docker services in a different way, see the official Docker documentation for instructions on installing Docker Compose.
Clone the ABS GitHub repository.
Navigate to the docker-compose directory in your local copy of the ABS repository and run docker compose up -d from your terminal. This command uses the existing docker-compose.yaml file to set up three containers:

An Aerospike Database container with data to be backed up.
A MinIO container for storing backup data.
An ABS container configured with a sample aerospike-backup-service.yml file to back up data from the Aerospike container and store it in the MinIO container.

The output from docker compose up -d should be similar to the following:

[+] Running 26/12
✔ aerospike-backup-service Pulled                                                                     25.8s 
✔ minio-client Pulled                                                                                 10.1s 
✔ aerospike-cluster Pulled                                                                            11.3s 
✔ minio Pulled                                                                                         5.5s 
                                                                                        
[+] Running 5/5
✔ Network docker-compose_default      Created                                                          0.0s 
✔ Container minio                     Healthy                                                          0.2s 
✔ Container aerospike-cluster         Healthy                                                          0.2s 
✔ Container minio-client              Exited                                                           0.1s 
✔ Container aerospike-backup-service  Started                                                          0.0s 

Verify installation by running docker ps. The minio-client container is necessary for installation, but not for running the service, so while four containers are used during setup, only three need to appear as running for testing.

The output should be similar to the following:

CONTAINER ID   IMAGE                                                                              COMMAND                  CREATED              STATUS                             PORTS                              NAMES
05e7e6247776   aerospike.jfrog.io/ecosystem-container-prod-local/aerospike-backup-service:latest   "./backup -c config.…"   About a minute ago   Up 53 seconds (health: starting)   0.0.0.0:8080->8080/tcp             aerospike-backup-service
7a359245eb3a   minio/minio:latest                                                                 "/usr/bin/docker-ent…"   About a minute ago   Up About a minute (healthy)        0.0.0.0:9000-9001->9000-9001/tcp   minio
bf93abfb953c   aerospike/aerospike-server-enterprise:6.4.0.10                                      "/usr/bin/as-tini-st…"   About a minute ago   Up About a minute (healthy)        0.0.0.0:3000-3003->3000-3003/tcp   aerospike-cluster

Test ABS in Docker Compose stack

You can test the different parts of the ABS configuration in any order with the REST endpoints that the service exposes. This section illustrates a short series of steps to examine and make a change to a particular backup policy.

1. Check the health endpoint

Run curl -X GET http://localhost:8080/health from your terminal to send a request to the /health endpoint. You should see Ok as a response.

2. View all backup routines

The Docker Compose package includes one sample backup routine. Send a request to the /v1/backups/full endpoint to view all routines.

curl -X GET http://localhost:8080/v1/backups/full

{"minioKeepFilesRoutine":
  [
    {
        "created":"2024-06-12T16:28:22.82156221Z",
        "from":"0001-01-01T00:00:00Z",
        "namespace":"test",
        "byte-count":42,
        "file-count":1,
        "key":"s3://as-backup-bucket/minioStorage/minioKeepFilesRoutine/backup/1718209702821/data/test"
    }
  ]
}                           

3. Check details of a specific routine

View the parameters of one specific routine by providing its name after /v1/config/routines:

curl -X GET http://localhost:8080/v1/config/routines/minioKeepFilesRoutine

{
    "backup-policy":"keepFilesPolicy",
    "source-cluster":"absCluster1",
    "storage":"minioStorage",
    "interval-cron":"@daily",
    "incr-interval-cron":"@hourly"
}

The routine schedules the keepFilesPolicy backup policy on the absCluster1 source cluster to minioStorage, with full backups daily and incremental backups hourly.

4. Check details of a specific policy

View the parameters of one specific policy by providing its name after /v1/config/policies:

curl -X GET http://localhost:8080/v1/config/policies/keepFilesPolicy

{
    "parallel":1,
    "remove-files":"KeepAll"
}

The keepFilesPolicy backup policy runs single-threaded and does not clear the output directory before saving the files.

5. Modify a policy

To modify an existing policy, send a PUT request. Use -H "Content-Type: application/json" to communicate that the data you are sending is of JSON type, and use -d to send the full definition of the new policy. In this example, the updated policy adds a new parameter to set a maximum of three retries. You must send the full JSON schema for the policy, even if you are only adding or changing a few parameters.

curl -X PUT -H "Content-Type: application/json" -d '{"parallel":1,"remove-files":"KeepAll","max-retries": 3}' http://localhost:8080/v1/config/policies/keepFilesPolicy

6. Check that the policy modifications took place

There is no response to a PUT request, so send another GET request to check that the policy has been updated.

curl -X GET http://localhost:8080/v1/config/policies/keepFilesPolicy

{
    "parallel":1,
    "remove-files":"KeepAll",
    "max-retries": 3
}

7. Trigger an immediate backup and view all backups again

To trigger a new backup routine to run immediately, send a POST request to the /v1/backups/schedule/ endpoint with the routine name and, optionally, a delay in milliseconds.

curl -X POST https://localhost:8080/v1/backups/schedule/minioKeepFilesRoutine?delay=1000

There is no feedback response in the terminal, so run another check of all backups to make sure it worked.

curl -X GET http://localhost:8080/v1/backups/full
    
{
  "minioKeepFilesRoutine":
    [
      {
        "created":"2024-06-12T16:28:22.82156221Z",
        "from":"0001-01-01T00:00:00Z",
        "namespace":"test",
        "byte-count":42,
        "file-count":1,
        "key":"s3://as-backup-bucket/minioStorage/minioKeepFilesRoutine/backup/1718209702821/data/test"
      },
      {
        "created":"2024-06-12T16:55:23.791971461Z",
        "from":"0001-01-01T00:00:00Z",
        "namespace":"test",
        "byte-count":42,
        "file-count":1,
        "key":"s3://as-backup-bucket/minioStorage/minioKeepFilesRoutine/backup/1718211323791/data/test"
      }
    ]
} 

In this example, there is a new backup present in the list. The timestamp is less than one hour from the original backup, which was triggered upon starting ABS, showing that the second backup was the one manually triggered by the POST request.

For more examples and the full JSON schemas for the request types, see Examples.

Install ABS on Linux

Linux installation packages are available from the GitHub repository under releases.

Check the release page for the supported Linux distributions.

Install and verify

Download the Linux package for your distribution from the GitHub Releases page. You can also build from the source code following the instructions in the GitHub README.
Navigate to the directory where you saved the installation package and run the following command to install the backup service using Linux.

Debian

sudo dpkg -i aerospike-backup-service_2.0.0_amd64.deb

RPM

sudo rpm -i aerospike-backup-service-2.0.0-1.x86_64.rpm

Verify the installation.

sudo systemctl status aerospike-backup-service

You should see output similar to the following. The exact file paths may differ depending on your machine and Linux distribution.

● aerospike-backup-service.service - Aerospike Backup Service
     Loaded: loaded (/lib/systemd/system/aerospike-backup-service.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-12-20 11:08:58 UTC; 14min ago
   Main PID: 229439 (aerospike-backu)
      Tasks: 26 (limit: 19160)
     Memory: 32.3M
        CPU: 6.562s
     CGroup: /system.slice/aerospike-backup-service.service
             └─229439 /usr/bin/aerospike-backup-service --config /etc/aerospike-backup-service/aerospike-backup-service.yml

Modify default configuration

Modify aerospike-backup-service.yml to update these parameters. By default, this configuration file is stored at /etc/aerospike-backup-service/.

See ABS Configuration Parameters for details and examples of the different parameters. Restart the service after modifying the parameters.

Service management

Service management commands such as restarting, viewing logs, and stopping the service vary depending on your Linux distribution.

Debian service management

Action	Command
Restart service	`sudo systemctl restart aerospike-backup-service`
Check service logs	`sudo journalctl -u aerospike-backup-service -n 100 --no-page -f`
Stop service	`sudo systemctl stop aerospike-backup-service`
Remove service (keep config)	`sudo dpkg -r aerospike-backup-service`
Remove service (remove completely)	`sudo dpkg -P aerospike-backup-service`
Remove backup files	`sudo rm -rf /var/lib/aerospike-backup-service`

RPM service management

Action	Command
Restart service	`sudo systemctl restart aerospike-backup-service`
Check service logs	`sudo journalctl -u aerospike-backup-service -n 100 --no-page -f`
Stop service	`sudo systemctl stop aerospike-backup-service`
Remove service	`sudo rpm -e aerospike-backup-service`
Remove backup files	`sudo rm -rf /var/lib/aerospike-backup-service`

Overview​

Install ABS with Docker Compose​

Download and set up ABS on Docker Compose​

Test ABS in Docker Compose stack​

1. Check the health endpoint​

2. View all backup routines​

3. Check details of a specific routine​

4. Check details of a specific policy​

5. Modify a policy​

6. Check that the policy modifications took place​

7. Trigger an immediate backup and view all backups again​

Install ABS on Linux​

Install and verify​

Modify default configuration​

Service management​

Debian service management​

RPM service management​