In this blog post, we detail how to create an Amazon Managed Streaming for Apache Kafka (Amazon MSK) resource using AWS Identity and Access Management (AWS IAM) in roles and policies to authenticate user access. In the initial step, we establish an Aerospike Database cluster and insert sample messages into the database. Subsequently, we observe in real time how these messages are streamed to Amazon MSK using Aerospike's Kafka Source Connector.
A message transformer for the Aerospike Kafka Inbound Connector
Aerospike database along with the Aerospike Connect for Kafka enables you to build low latency and high throughput Kafka based streaming pipelines. Streaming from Apache Kafka to Aerospike Database Enterprise Edition relies on Aerospike’s Kafka inbound connector, which is implemented as a “sink” connector in the open-source Kafka Connect framework. This inbound connector allows you to stream messages from Kafka, translates each message to an Aerospike record, and performs the corresponding insert, update, or delete for that record in Aerospike. Visit Streaming to Aerospike from Apache Kafka for more details.