Estimate resources for Aerospike restore

Estimate memory and file descriptor usage for asrestore.

Overview

asrestore can use significant resources, depending on the options you set. We recommend running it on a separate host or container from Aerospike Database.

The command-line flags used in the following formulas are defined in Run Aerospike restore. These formulas describe how resource usage scales; they do not provide exact answers.

Allocate slightly more resources than the formulas suggest.

Estimate memory usage for a restore

The following have the greatest impact on memory used by asrestore:

• The system where you run asrestore
• The asrestore version
• The --parallel, --batch-size, and --max-async-batches flags

Formula to approximate memory usage for restore

To approximate memory usage:

1. Multiply the values you set for --parallel and --batch-size.
2. Multiply the result by avg_record_size.
3. Multiply --batch-size by --max-async-batches.
4. Add the results from steps 2 and 3.
5. Add --max-async-batches.

The final result is the approximate memory usage for a restore of your data.

This formula still holds even if --disable-batch-writes is set. In that case, the only change is the default --batch-size value.

Calculate number of file descriptors

asrestore may need to open many backup files and network sockets. If it cannot open the required number of file descriptors, it can fail with “too many open files” errors.

By default, asrestore opens a backup file for each of its --parallel threads.

If asrestore is running against Database 6.0.0 or later, it uploads records in batches. In this case, it opens up to --max-async-batches sockets to the server for record upload.

If the server version is older than 6.0.0 or the --disable-batch-writes flag is used, each record is uploaded individually, so --batch-size × --max-async-batches is the number of sockets that may be opened.

Formula to calculate approximate file descriptors for restore

To calculate the approximate number of file descriptors for a data restore with batch writes:

• Sum the values you set for --parallel and --max-async-batches.

To calculate the approximate file descriptors for a data restore without batch writes:

1. Multiply --batch-size by --max-async-batches.
2. Add the result to --parallel.

The final sum is the approximate number of file descriptors needed for a data restore.