Install the Aerospike Kafka Source Connector

Prerequisites

Aerospike Database change notification configuration

Installation prerequisites on Linux

Supported operating systems

The connector is supported on the following operating systems:

RHEL/CentOS (8 and 9)
Ubuntu (20.04, 22.04 and 24.04 LTS)
Debian (11, 12 and 13)
Amazon Linux 2023

Java Runtime Environment

This connector is a Java web-application that requires Java 11 or later. Both Oracle and OpenJDK Java Runtime Environments are supported.

Java 17 is supported starting with Aerospike Kafka source connector (outbound) version 4.1.0.

Installation

The installation steps below apply to both the Debian (*.deb) and RHEL (*.rpm) packages.

Install Java 11 or later version

The Aerospike Kafka source connector package does not include a Java runtime environment. Most supported platforms provide official JDK 11 packages. For platforms that do not, you can get Oracle’s JDK 11 builds from jdk.java.net/11/.

RHEL and CentOS

sudo yum install java-11-openjdk

Debian and Ubuntu

sudo apt-get install openjdk-11-jre

Amazon Linux

sudo dnf install java-11-openjdk

Install connector package

Go to the Aerospike Enterprise downloads site to download the Aerospike Kafka source (outbound) connector package for your platform and install it.

RHEL and CentOS

sudo rpm -i aerospike-kafka-outbound-5.3.13-1.noarch.rpm

Debian and Ubuntu

sudo dpkg -i aerospike-kafka-outbound-5.3.13.all.deb

Amazon Linux

sudo rpm -i aerospike-kafka-outbound-5.3.13-1.noarch.rpm

Verify or update source connector configuration

The source connector’s configuration can be found in the /etc/aerospike-kafka-outbound/ directory. For details on how to configure the Kafka provider and enable TLS, see the configuration guide.

Start the source connector

The source connector package includes a Systemd service definition. The installation procedure creates an aerospike-kafka-outbound service.

Enable the source connector to start on system startup or reboot.

$ sudo systemctl enable aerospike-kafka-outbound

To start the source connector service run.

$ sudo systemctl start aerospike-kafka-outbound

Install Kafka outbound connector on Kubernetes with helm-charts

If your deployment situation allows the use of Helm charts, you can install the Kafka outbound connector on a Kubernetes deployment using Helm.

Add the Aerospike Helm repository.

helm repo add aerospike https://aerospike.github.io/helm-charts --force-update
helm repo update

Create the file values.yaml with your Kafka outbound connector configuration. The following example includes a basic Kafka outbound connector configuration as part of the Helm installation.

replicaCount: 3

image:
  repository: aerospike/aerospike-kafka-outbound
  tag: "latest"

connectorConfig:
  service:
    port: 8080
    manage:
      port: 8081

  # Format of the Kafka destination message.
  format:
    mode: flat-json
    metadata-key: metadata

  # Aerospike record routing to a Kafka destination.
  routing:
    mode: static
    destination: aerospike

  # Kafka producer initialization properties.
  producer-props:
    bootstrap.servers:
      - kafka-broker-1:9092
      - kafka-broker-2:9092
      - kafka-broker-3:9092

  # The logging properties.
  logging:
    enable-console-logging: true

connectorSecrets: []

serviceAccount:
  create: true
  name: aerospike-kafka-outbound

Install the Kafka outbound connector by running the following Helm command.

helm install aerospike-kafka-outbound aerospike/aerospike-kafka-outbound --values values.yaml --atomic --wait --debug --create-namespace --namespace aerospike

Deploying on Docker

Aerospike provides Docker images for the Aerospike Kafka source connector on Docker Hub.

sudo docker run \
 -v <path to local aerospike-kafka-outbound.yml>:/etc/aerospike-kafka-outbound/aerospike-kafka-outbound.yml \
 container.aerospike.com/aerospike/aerospike-kafka-outbound:latest

<path to local aerospike-kafka-outbound.yml>: The path (including the name) of the configuration file in your local filesystem.