Configure change notification

The change-notification system informs external systems (such as outbound connectors, and the XDR Proxy) when records are inserted, modified, or deleted in an Aerospike database. This system uses Aerospike’s cross-datacenter replication (XDR) to publish changes as they occur.

Each message that the system sends contains either an updated/inserted record or a record deletion notification. Each update/insert/touch message contains the full database record including all, or a subset, of the record’s bins. Commands on CDT List or Map types, such as adding a list element or removing a map entry, are currently unsupported.

In case of partial bin updates such as these, the full record bin is retransmitted. It also ships the userKey and the set name if they are available. The bins are never shipped with the delete notification.

Only records which have been successfully delivered to an external system are acknowledged by Aerospike.

There might be instances where records are re-ordered on the network or across external systems, such as instances of outbound connectors and instances of the XDR Proxy. In such cases, a message for an older version of a record might be delivered after a message for a newer version of the same record.

Caution

In rare cases, a change might be lost. For instance, changes may be lost if a node is removed from an Aerospike cluster after data migration between nodes is finished, but before all notifications are sent.

Prerequisites

• Ensure that you are running Aerospike Database Enterprise Edition, version 4.4 or later.
• Ensure that your digitally signed feature-key-file sets asdb-change-notification to true.

Procedure

To configure change notification for use with the external systems mentioned above, follow these steps on each node of your Aerospike cluster:

Open the /etc/aerospike/aerospike.conf configuration file for editing.

If there is no xdr section in the file, create one.

xdr {

    }

Create an XDR block in the xdr section, placing the connector property at the top of the block and setting its value to true.

Tip

The property ‘connector’ configures both change notification for sending messages to outbound connectors, and the XDR Proxy.

Here is an example. The name kafkaDC is arbitrary. You can name your blocks as you see fit. In this example, kafkaDC means only that the target “datacenter” is Kafka.

xdr {
     # Change notification XDR block that round-robins between two connector nodes
     dc kafkaDC {
             connector true
             node-address-port 10.0.0.1 8080
             node-address-port 10.0.0.2 8080
             namespace test {
             }
     }
}

For information about the properties in this example, see Configuration Parameter Reference.

Save and exit the file.

Restart the Aerospike cluster node.

What to do next

Install, configure, and start the external systems that you plan to use in your system for replicating or streaming data from Aerospike.

• If you plan to use the XDR Proxy, see Overview for the Aerospike XDR Proxy.
• If you plan to use the ESP Outbound connector, see Load Balancing XDR Proxy Instances with ESP Outbound Connectors.

Related information

• To learn how changes are logged, hot keys are optimized, and retransmits are handled, refer to the documentation for cross-datacenter replication:
  • Cross-Datacenter Replication (XDR)
  • Cross-Datacenter Replication (XDR) Architecture