---
title: "XDR stack overflow risk (AER-6898)"
description: "Safe upgrade instructions for Aerospike Database Enterprise and Federal Edition deployments affected by the XDR recovery stack overflow crash."
---

# XDR stack overflow risk (AER-6898)

> For the complete documentation index see: [llms.txt](https://aerospike.com/docs/llms.txt)
> 
> All documentation pages available in markdown.

This page describes the XDR recovery drain defect (AER-6898) and provides safe upgrade instructions for affected Aerospike Database versions. This defect can cause stack overflow crashes or silent data corruption in Enterprise and Federal Edition clusters that use XDR. A standard rolling upgrade on a cluster running an affected version may trigger the crash. Follow the [safe upgrade procedure](#safe-upgrade-procedure) below.

::: tip
You do not need to follow the safe upgrade procedure if any of the following are true:

-   You are running Aerospike Community Edition.
-   You are not using XDR.
-   You are currently running a version of Aerospike Database prior to the _first affected_ version or equal to or later than the _fixed in_ version specified in [Affected versions](#affected-versions) below.
:::

## Background

When a node transitions from master to prole while XDR recovery is active, the XDR retry queue is not drained. Up to 1,000 in-flight shipment requests can remain outstanding and continue to count against the `n_in_progress` limit. If that node later becomes master again in regular (non-recovery) mode, the calculation `50 - n_in_progress` underflows because the result is unsigned — sizing a stack-allocated variable-length array far beyond what the thread stack can accommodate.

The crash typically manifests as a segfault in `process_trans_q`, or as silent corruption of the primary index if the overflowed stack lands in mapped memory. Both outcomes can destabilize the cluster.

## Affected versions

This defect applies to Enterprise and Federal Edition only, affecting any version that included AER-6795 or AER-6804:

| Release line | First affected | Fixed in |
| --- | --- | --- |
| 6.1.x | 6.1.0.47 | No patch available |
| 6.2.x | 6.2.0.37 | No patch available |
| 6.3.x | 6.3.0.35 | 6.3.0.36 |
| 6.4.x | 6.4.0.30 | 6.4.0.34 |
| 7.0.x | 7.0.0.22 | No patch available |
| 7.1.x | 7.1.0.14 | 7.1.0.23 |
| 7.2.x | 7.2.0.8 | 7.2.0.17 |
| 8.0.x | 8.0.0.3 | 8.0.0.15 |
| 8.1.x | 8.1.0.0 | 8.1.2.0 |

::: note
The 6.1.x, 6.2.x, and 7.0.x release lines reached end of life before this defect was discovered. Clusters still running these versions should upgrade directly to a patched release in a supported line.
:::

## Detecting a vulnerable state

A node is in a vulnerable state if its XDR retry queue exceeds 50 entries and its partition role is prole. Run the following `asinfo` command for each configured datacenter and namespace:

```plaintext
asinfo -l -v "xdr-dc-state:dc=DC-NAME;namespace=NAMESPACE"
```

In the output, look for partitions where `role` is `P` (prole) and `retry_q` exceeds 50:

```plaintext
namespace:partition:role:state:safe_lst:first_trans_lut:trans_q:retry_q:windows:in_progress

NAMESPACE:0:P:R:0:0:0:1000:0:1000
```

If any partition shows `retry_q > 50` on a prole node, the cluster is in the vulnerable state.

::: caution
Because partition role assignments change during a rolling upgrade, all deployments running an affected version should use the safe upgrade procedure regardless of whether a vulnerable state is currently detected.
:::

## Safe upgrade procedure

1.  Record the current UTC timestamp. You need to know the time prior to beginning step 2 so you can rewind XDR shipping after the upgrade to avoid missing records written during the upgrade window.
    
2.  Disable XDR shipping across all nodes by removing all namespaces from all datacenter configurations. This drains the retry queue on every node and prevents the poisoned `n_in_progress` state from persisting across rolling restarts.
    
    For each namespace in each datacenter, use the `set-config` info command:
    
    ```plaintext
    asinfo -v "set-config:context=xdr;dc=DC-NAME;namespace=NAMESPACE;action=remove"
    ```
    
    Repeat for every namespace and every datacenter.
    
3.  Wait until XDR activity is quiescent on all nodes before continuing. You can verify using:
    
    ```plaintext
    asinfo -l -v "xdr-dc-state:dc=DC-NAME;namespace=NAMESPACE"
    ```
    
4.  Perform a rolling upgrade to the fixed release across all nodes. See [Upgrade or repair an Aerospike server](https://aerospike.com/docs/database/install/upgrade).
    
5.  After all nodes are upgraded, recreate the XDR datacenter namespace configuration. Add each namespace back to its datacenter, rewinding XDR prior to the timestamp recorded in step 1. For example, if the upgrade took 30 minutes, rewind at least 1800 seconds:
    
    ```plaintext
    asinfo -v "set-config:context=xdr;dc=DC-NAME;namespace=NAMESPACE;action=add;rewind=1800"
    ```