---
title: "Configure streaming from Aerospike to Kafka"
description: "Guide on configuring Aerospike to Kafka streaming via change notifications and the aerospike-kafka-outbound.yml file."
---

# Configure streaming from Aerospike to Kafka

> For the complete documentation index see: [llms.txt](https://aerospike.com/docs/llms.txt)
> 
> All documentation pages available in markdown.

Configuring streaming from Aerospike to Kafka involves setting up your Aerospike database to send change notifications and modifying the configuration file `/etc/aerospike-kafka-outbound/aerospike-kafka-outbound.yml`.

## Set Up Aerospike Database

In your Aerospike database, you must configure Cross-Datacenter Replication (XDR) and enable change notification. You must also configure your Kafka outbound cluster to be a “connector” XDR datacenter and the namespace must point to this as xdr remote datacenter.

See the [change notification configuration parameters and example config](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/change-notification).

## Modify the Config File

Configure streaming from Aerospike to Kafka by modifying the `/etc/aerospike-kafka-outbound/aerospike-kafka-outbound.yml` YAML configuration file.

The configuration file has the following sections:

-   [`service`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/service) Configures the source connector’s listening ports, TLS, and network interface.
-   [`producer-props`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/producer-props) Configures connection properties for the target Kafka message broker.
-   [`bin-transforms`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/bin-transforms) Specifies the bin transformations to apply on the Aerospike record.
-   [`format`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/format) Specifies the message format to use for the outbound messages sent to the message broker.
-   [`batching`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/batching) Specifies how to collect a group of Change Notification Records into a single batch of outbound message sent to the message broker.
-   [`record-ordering`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/record-ordering) Configures ordering of outbound records.
-   [`routing`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/routing) - configures how incoming record updates/deletes from Aerospike are routed to the message broker.
-   [`namespaces`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/namespaces) Configures bin transforms, format, routing at namespace and set level.
-   [`logging`](https://aerospike.com/docs/connectors/streaming/kafka/outbound/configure/logging) Configures the destination and level for the source connector’s logs.

::: caution
If the route or format is not specified for an Aerospike record, then the record is skipped and logged at the _warn_ level.
:::

## Cascading Configuration Values

You can configure the `bin-transforms`, `format`, and `routing` sections to include `default` values, values for each particular namespace, and values for each particular set within a namespace. For each Aerospike record, the appropriate values are applied.

Here is an annotated example:

```yaml
routing:

  mode: static

  destination: default <1>

‎‎

namespaces:

  users:

    routing:

      mode: static

      destination: users <2>

    format:

      mode: flat-json

      metadata-key: metadata

    sets:

      premium:

        routing:

          mode: static

          destination: premium <3>
```

1.  Specifies that records in all namespaces are to be shipped to the `default` topic.
2.  Specifies that records in the `users` namespace are to be shipped to the `users` topic.
3.  Specifies that records in the `premium` set in the `users` namespace are to be shipped to the `premium` topic.

## Example

A sample configuration is shown below:

```yaml
# port that the connector runs on

service:

  port: 8080

# kafka producer props https://kafka.apache.org/23/javadoc/org/apache/kafka/clients/producer/ProducerConfig.html

producer-props:

  bootstrap.servers:

    - 192.168.5.20:9092

    - 192.168.5.30:9092

# log location if not stdout

logging:

  file: /var/log/aerospike-kafka-outbound/aerospike-kafka-outbound.log

# one of json, flat-json, message-pack, avro, or kafka

format:

  mode: json

# a list of transformations and mappings.

bin-transforms:

  map:

   yellow: red

   transforms: //will be done in order

    - uppercase

routing:

  mode: static

  destination: default

namespaces:

  users:

    routing:

      mode: static

      destination: users

    format:

      mode: flat-json

      metadata-key: metadata

    sets:

      premium:

        routing:

          mode: static

          destination: premium

        bin-transforms:

          map:

            gold: platinum
```