AEROSPIKE CONNECTOR FOR EVENT STREAM PROCESSING

Aerospike Connect for Event Stream Processing

Aerospike Connect for Event Stream Processing (ESP) is a streaming connector that integrates Aerospike Database 7 with HTTP-based systems in the cloud or on premises to build efficient streaming pipelines.

Stream Change Notification with Aerospike Connect for Event Stream Processing (ESP)

Event stream processing (ESP) systems commonly rely on Change Data Capture (CDC) data patterns which inform external systems when records are inserted, modified, or deleted in a database. These CDC patterns (in Aerospike CDC is implemented as “Change Notification” but unlike CDC, may include several changes) are published as they occur via the Aerospike Cross-Datacenter Replication (XDR) feature.

diagram: Aerospike Connect for Event Stream Processing data pipeline

Benefits

Why Aerospike Connect for ESP

Connect for ESP integrates Aerospike Database 7 with HTTP-based systems whether in the cloud or on-premises via CDC change notification.

Support for serverless design
Supports serverless processing design patterns, where CDCs are streamed to a Lambda function in AWS or Google Cloud Functions .
Easy integration
Easily integrate the Aerospike Real-time Data Platform with HTTP-based systems either in the cloud or on-premises.
Supports efficient data exchange between systems
Supports JSON and binary formats such as Avro and MessagePack.
Simplifies operations
Removes the overhead associated with operationalizing Kafka or Pulsar production clusters for use cases that do not warrant it.

FEATURES

Key features of Aerospike Connect for ESP

Connect for ESP converts CDC change notifications over XDR into HTTP requests and streams them to downstream consumers.

Supports HTTP-based systems

Converts CDC Change Notification over XDR into HTTP/1 or HTTP/2 to enable connection with broader HTTP-based systems.

Efficient data exchange between systems

Serializes the change notification payload into a text format such as JSON, or a binary format, such as Avro or MessagePack.

Change notifications for big data ecosystems

Triggers AWS Lambda via API Gateway or Application Load Balancer, or Google Functions to process CDC change notifications and make it available to the broader big data ecosystem in AWS or GCP.

Enable message ordering

Ships LUT (last-update-time) of the record to enable downstream applications to build their own custom logic to order messages.

Delivery guarantee

Extends XDR’s at-least-once delivery guarantee to ensure zero message loss.

Ease of deployment

Deploys seamlessly either in the cloud or on-premises.

Message filtering

Can be used in conjunction with XDR filter expressions to filter out records before transmission.

Learn more about Connect for ESP

Aerospike Connect for Event Stream Processing (ESP) integrates with other systems in the cloud or on-prem that can ingest CDC change notification as HTTP requests.

Read solution brief

Use Cases

Aerospike Connect for Event Stream Processing powers use cases across architectures

Aerospike Connect for ESP makes it easy for enterprises to enable use cases requiring streaming data changes (Change Notification) from the Aerospike database to other HTTP-based systems.

Search

Export shortened Aerospike Change Notification (CDC) messages to Elasticsearch using its Document REST API. This pattern can be extended to make Aerospike data available for analysis in Splunk using their HTTP Event Collector.