---
title: "Estimate resource usage for absctl backup"
description: "Estimate resource usage for Aerospike absctl backup, including memory, disk space, query threads, and file descriptors."
---

# Estimate resource usage for absctl backup

> For the complete documentation index see: [llms.txt](https://aerospike.com/docs/llms.txt)
> 
> All documentation pages available in markdown.

`absctl backup` can use significant resources depending on the options that you set when running the tool. We recommend running the tool on a separate host or container from Aerospike Database.

The command-line flags used in the following formulas are defined in [Run Aerospike backup](https://aerospike.com/docs/database/tools/backup-and-restore/absctl/backup/use).

::: note
These formulas describe how resource usage scales. They do not provide exact numbers for resource usage. You should allocate slightly more resources to `absctl backup` than the formulas suggest.
:::

## Estimate memory usage for a backup

`absctl backup`’s memory usage is largely affected by the `--parallel` flag. By default, `absctl backup` distributes work across threads by assigning each thread a unique file to write to. These files each have a constant size buffer associated with them. The file buffer’s default value is 4 MiB for local backups. Cloud provider chunk sizes default to 50 MiB. The internal pipeline buffer that processes records is 256 records. Multiply this value by the average record size to get the internal pipeline buffer size.

In `absctl` 1.0.0, the local buffer and cloud chunk or block sizes default to 5 MiB. In `absctl` 1.1.0, the local default is 4 MiB and the cloud defaults are 50 MiB.

::: note
The Go runtime itself also uses a small amount of memory.
:::

### Formula to approximate memory usage for backup

Use the following formulas to estimate memory used by file buffers:

-   Local file: `--parallel × --local-buffer-size`
-   Standard output: `--std-buffer`
-   Amazon S3: `--parallel × --s3-chunk-size × max(1, --s3-upload-concurrency)`
-   Google Cloud Storage: `--parallel × --gcp-chunk-size`
-   Azure Blob Storage: `--parallel × --azure-block-size`

`absctl` 1.1.0 uses one concurrent Azure upload per backup file. Add `--parallel × 256 × average record size` for the internal record pipeline. Cloud SDK clients, the Go runtime, compression, encryption, checksums, and in-flight requests require additional memory, so allocate capacity above the calculated estimate.

## Estimate disk space for a backup

To estimate how much disk space is needed to back up your data, use the `--estimate` flag when running `absctl backup`.

When you use the `--estimate` flag, you may not use `--parallel`. The two flags are mutually exclusive.

Terminal window

```shell
absctl backup --namespace NAME --estimate
```

The `--estimate` flag reads 10,000 records by default from the specified namespace and prints the average size of the sampled records. The number of records can be changed with `--estimate-samples`.

To estimate the amount of disk space needed for a backup:

1.  Multiply the estimated record size, returned by the `--estimate` flag, by the number of records in the namespace.
    
2.  Add 10% to account for backup file overhead.
    

The result is the approximate disk space needed for a backup.

::: note
Per-record filters (`filter-exp`, `modified-after`, `modified-before`, `no-ttl-only`, `after-digest`, and `partition-list`) and `node-list` are not accounted for in the estimate, and using these options will have no effect on the estimate.
:::

## Server-side query thread usage

When `absctl backup` runs queries against an Aerospike cluster, it uses server-side query threads to process the requests. Understanding how these threads are allocated helps you tune both the backup tool and the database for optimal performance.

### How query threads work

Each backup query initiated by `--parallel` runs on the server. The server allocates threads from a shared pool to process these queries.

| Server configuration | Default | Description |
| --- | --- | --- |
| [`query-threads-limit`](https://aerospike.com/docs/database/reference/config#service__query-threads-limit) | 128 | Maximum total threads for all queries across the server |
| [`single-query-threads`](https://aerospike.com/docs/database/reference/config#namespace__single-query-threads) | 4 | Maximum threads per individual query |

With the defaults, each query uses up to 4 threads, allowing approximately 32 concurrent queries (128 ÷ 4) before queries start waiting for threads.

### Relationship with `--parallel`

The `--parallel` flag controls how many concurrent queries `absctl backup` runs. Each query consumes server-side query threads:

-   If `--parallel` exceeds `query-threads-limit ÷ single-query-threads`, some queries will wait for threads.
-   For a cluster with multiple nodes, the thread limits apply per node.

**Example**: With defaults (`query-threads-limit=128`, `single-query-threads=4`) and `--parallel 8`:

-   Each of the 8 parallel queries uses up to 4 threads
-   Total thread usage: up to 32 threads per node (8 × 4), well within the 128-thread limit

### Tuning recommendations

For backup-heavy workloads, consider adjusting server-side settings:

| Goal | Adjustment |
| --- | --- |
| Support more concurrent queries | Increase `query-threads-limit` |
| Make individual queries faster | Increase `single-query-threads` |
| Limit backup impact on other queries | Decrease `query-threads-limit` or use `--records-per-second` to throttle |

::: note
Increasing `single-query-threads` makes individual queries faster but reduces the number of concurrent queries the server can handle. Balance this based on your workload: use higher values for fewer, faster queries, or lower values to support more concurrent operations.
:::

To view current query thread settings, use:

Terminal window

```shell
asinfo -v "get-config:context=service" | tr ';' '\n' | grep query-threads

asinfo -v "get-config:context=namespace;id=NAMESPACE" | tr ';' '\n' | grep query-threads
```

To adjust dynamically:

Terminal window

```shell
asinfo -v "set-config:context=service;query-threads-limit=256"

asinfo -v "set-config:context=namespace;id=NAMESPACE;single-query-threads=8"
```

## Calculate number of file descriptors

`absctl backup` may need to open many backup files and network sockets. If `absctl backup` cannot open the required number of file descriptors, it can fail with “too many open files” errors.

By default, `absctl backup` opens a new backup file for each of its `--parallel` threads. Each thread may have to open a network socket to each node in the cluster.

To approximate the maximum number of file descriptors needed for a backup:

1.  Set `N` to the value of `--parallel`.
    
2.  Set `C` to the number of nodes in the cluster.
    
3.  Estimate file descriptors as (N) (output files) + (N \\times C) (network sockets) + a small constant for overhead (logs, DNS, etc).
    

This estimate is intentionally conservative. If you are backing up to cloud storage, increase the estimate to account for additional connections used for upload concurrency.