Network configuration

This page describes how to configure critical network ports on an Aerospike Database, and how to compress fabric traffic between nodes.

Applies to: Aerospike Database 8.2.0 and later for fabric wire compression. Earlier releases cover network ports only.

Aerospike Database’s network configuration section sets up critical network ports to be used by other nodes, applications, and tools. The following table describes the ports used by Aerospike Database and cross-datacenter replication (XDR).

Name	Default port	Description
service	3000	Application, tools, and remote XDR use the service port for database operations and cluster state.
fabric	3001	Intra-cluster communication port. Replica writes, migrations, and other node-to-node communications use the fabric port. You can compress those payloads. See Compress fabric traffic.
mesh heartbeat	3002	Heartbeat protocol ports form and maintain the cluster. Only one heartbeat port may be configured. Mesh heartbeat and fabric should run on the same NIC.
multicast heartbeat	9918	Heartbeat protocol ports are used to form and maintain the cluster. Only one heartbeat port may be configured.
admin	3003	A dedicated admin port for continuous access by the monitoring stack exporter, as well as emergency access to unresponsive nodes using asadm and asinfo.

Verify that all application and XDR nodes can communicate to the service port on all Aerospike nodes, and that each node can communicate over the configured heartbeat and fabric ports.

Note

The telnet-compatible info port 3003 was removed in Database 8.1.0. Use the admin port instead.

Configure network sections

The network section of the Aerospike configuration file requires the following sections:

• service
• fabric
• heartbeat
• admin

To isolate fabric (inter-node replication, migration) and heartbeat from service traffic or XDR traffic, add an address distinct from the service address to the heartbeat and fabric sections.

Note

For information on how to configure the heartbeat section, see heartbeat configuration which defines the interface for intra-cluster communications.

The service section

The service sub-context of network binds client-facing interfaces. It is not the top-level service context, where process and cluster settings such as SMD compression live. See Compress fabric traffic.

The following table describes each configuration item in the service section.

Configuration item	Description
address	Interfaces or IP addresses to bind and listen to. Multiple IP addresses are allowed.
access-address	Interfaces or IP addresses to publish for clients, typically clients within the same subnet or datacenter.
alternate-access-address	Interfaces or IP addresses to publish for clients that can’t connect to access-address interfaces or IP addresses. If the items specified here are actual interfaces and not mapped over NAT, then the corresponding address configuration must be specified unless address any is set. Clients requiring the alternate-access-address to be returned must request it by specifying useServicesAlternate in their client policy.
access-port	When configured, this port is published to the clients. Requires port forwarding to be set up when the value is different than the port.
alternate-access-port	When configured and the client specifies useServicesAlternate in the client policy, this port is published to the clients. Requires port forwarding to be set up when the value is different than the port.

Example 1: Service section configuration

Host with 2 network interfaces, x.x.x.x and y.y.y.y, with x.x.x.x for clients within the same subnet or datacenter (private IP) and y.y.y.y for clients in a different subnet or datacenter (public IP). The IP address y.y.y.y is not mapped over NAT:

network {
  service {
    address x.x.x.x
    address y.y.y.y
    access-address x.x.x.x
    alternate-access-address y.y.y.y
  }
}

The access-address x.x.x.x prevents the y.y.y.y IP from also being published. If access-address is not specified, all IPs specified as address are published.

Example 2: Service section configuration

If the y.y.y.y IP is mapped over NAT:

network {
  service {
    address x.x.x.x
    access-address x.x.x.x
    alternate-access-address y.y.y.y
  }
}

Or, as address is published by default when not overwritten through access-address:

network {
  service {
    address x.x.x.x
    alternate-access-address y.y.y.y
  }
}

Example 3: Service section configuration

This alternate configuration works in most cases: setting address to any binds to all available interfaces, then publishes the specific access-address and alternate-access-address.

network {
  service {
    address any
    access-address x.x.x.x
    alternate-access-address y.y.y.y
  }
}

Fabric section configuration examples

To isolate intra-cluster fabric traffic from regular client traffic, specify an address different from the service section. By default, the fabric address is set to any. For heartbeat-specific examples, see Network heartbeat configuration.

aerospike.conf

network {
  fabric {
    address any
    port 3001   # Intra-cluster communication port (migrates, replication, etc).
  }
}

Fabric compression is not configured in network.fabric. Replica-write and migration compression are namespace settings. SMD compression is a top-level service setting. See Compress fabric traffic.

Admin section

The following table describes each configuration item in the optional admin section.

Configuration item	Description
port	Port that is not secured (non-TLS) at which the server listens for admin client connections.
address	IP address at which the server listens (binds) for non-secure (non-TLS) admin connections.
tls-authenticate-client	false: Only the client authenticating the server. any: Two-way (mutual) authentication, both client and server need to be authenticated. user-defined: Two-way (mutual) authentication along with subject validation.
tls-port	TLS-enabled port where the server listens for admin client connections.
tls-name	Specifies which TLS parameters to use for the given context’s TLS connections.
tls-address	IP address where the server listens (binds) for secured (TLS) admin connections.
disable-localhost	When set to true, the service will not listen on localhost.

Example configuration:

network {
  admin {
    port 3003
    address any
    tls-port 3004
    tls-name asd_node
    tls-address any
    tls-authenticate-client any
    disable-localhost false
  }
}

You can use the admin port to remove an unresponsive node. See Ejecting an unresponsive node using asadm and the admin port.

Compress fabric traffic

Replica writes, partition migrations, and system metadata (SMD) full-sync messages all travel on the fabric port. From Database 8.2.0, Enterprise Edition can compress each of them with Zstd.

Compression is opt-in per transport. Each transport has its own mode and level, and enabling one does not enable the others:

Traffic	Context	Mode setting	Configure it on
Replica writes	namespace	replication-compression-mode	Configure wire compression
Partition migrations	namespace	migrate-compression-mode	Configure wire compression
SMD full-sync	service	smd-compression-mode	Configure SMD wire compression

SMD compression belongs to the top-level service context, not to the network.service sub-context described earlier on this page, and not to network.fabric. Nothing in the network context configures compression.

Wire compression changes only what crosses the fabric. It does not change what is stored. To compress records on storage, see Configure storage compression. All compression is server-to-server: clients are unaffected and no client upgrade is required.

In a multi-AZ deployment, fabric is the traffic that crosses availability zone boundaries, and cloud providers typically bill that transfer in both directions. Compressing fabric payloads reduces that cost, at the price of CPU on both the sending and receiving node.

Note

Prior to Database 8.2.0, fabric traffic was uncompressed.

Fabric byte counters

Each transport rides a separate fabric channel, and asinfo -v 'statistics' reports bytes for each one. These are the denominators for a compression savings ratio:

Statistic	Fabric channel	Traffic
fabric_rw_bytes_sent / fabric_rw_bytes_received	rw	Replica writes
fabric_bulk_bytes_sent / fabric_bulk_bytes_received	bulk	Partition migrations
fabric_meta_bytes_sent / fabric_meta_bytes_received	meta	SMD
fabric_ctrl_bytes_sent / fabric_ctrl_bytes_received	ctrl	Migration and clustering control

These counters are node-wide and include uncompressed traffic, message headers, and retransmits. For how to pair them with the per-transport bytes-saved counters, see Estimate the savings ratio.

When compression starts

Caution

Replica-write and SMD compression stay off until every node in the cluster runs Database 8.2.0 or later. If you enable either setting during a rolling upgrade, it reports as on and compresses nothing until the last node upgrades. There is no error.

Migration compression is the exception. It is negotiated per link, so it starts between any two nodes that already run a build that supports it.

Older nodes continue to receive uncompressed traffic throughout the upgrade. There is no unsafe mixed-version state, and a compression failure on a replica write is retransmitted uncompressed without the client seeing an error.

More information

• Configure the heartbeat section.
• Configure SMD wire compression to reduce intra-cluster fabric traffic from service metadata full-sync events.
• Manage namespaces.
• Configure Rack awareness to enable Aerospike to support top-of-rack switch failure.
• Configure Aerospike Database.
• Change settings at runtime with Dynamic runtime config.
• Configuration reference.
• Metrics reference.