Aerospike Connect for Event Stream Processing (ESP) is a streaming connector that integrates Aerospike Database 6 with HTTP-based systems in the cloud or on premises to build efficient streaming pipelines.
AEROSPIKE CONNECTOR FOR EVENT STREAM PROCESSING
Aerospike Connect for Event Stream Processing
Stream Change Notification with Aerospike Connect for Event Stream Processing (ESP)
Event stream processing (ESP) systems commonly rely on Change Data Capture (CDC) data patterns which inform external systems when records are inserted, modified, or deleted in a database. These CDC patterns (in Aerospike CDC is implemented as “Change Notification” but unlike CDC, may include several changes) are published as they occur via the Aerospike Cross-Datacenter Replication (XDR) feature.
Benefits
Why Aerospike Connect for ESP
Connect for ESP integrates Aerospike Database 6 with HTTP-based systems whether in the cloud or on-premises via CDC change notification.
-
Support for serverless design
Supports serverless processing design patterns, where CDCs are streamed to a Lambda function in AWS or Google Cloud Functions . -
Easy integration
Easily integrate the Aerospike Real-time Data Platform with HTTP-based systems either in the cloud or on-premises. -
Supports efficient data exchange between systems
Supports JSON and binary formats such as Avro and MessagePack. -
Simplifies operations
Removes the overhead associated with operationalizing Kafka or Pulsar production clusters for use cases that do not warrant it.
FEATURES
Key features of Aerospike Connect for ESP
Connect for ESP converts CDC change notifications over XDR into HTTP requests and streams them to downstream consumers.
Supports HTTP-based systems
Converts CDC Change Notification over XDR into HTTP/1 or HTTP/2 to enable connection with broader HTTP-based systems.
Efficient data exchange between systems
Serializes the change notification payload into a text format such as JSON, or a binary format, such as Avro or MessagePack.
Change notifications for big data ecosystems
Triggers AWS Lambda via API Gateway or Application Load Balancer, or Google Functions to process CDC change notifications and make it available to the broader big data ecosystem in AWS or GCP.
Enable message ordering
Ships LUT (last-update-time) of the record to enable downstream applications to build their own custom logic to order messages.
Delivery guarantee
Extends XDR’s at-least-once delivery guarantee to ensure zero message loss.
Ease of deployment
Deploys seamlessly either in the cloud or on-premises.
Message filtering
Can be used in conjunction with XDR filter expressions to filter out records before transmission.
Learn more about Connect for ESP
Use Cases
Aerospike Connect for Event Stream Processing powers use cases across architectures
Aerospike Connect for ESP makes it easy for enterprises to enable use cases requiring streaming data changes (Change Notification) from the Aerospike database to other HTTP-based systems.
Search
Export shortened Aerospike Change Notification (CDC) messages to Elasticsearch using its Document REST API. This pattern can be extended to make Aerospike data available for analysis in Splunk using their HTTP Event Collector.
Backup
Back up Aerospike data to S3 (via AWS Lambda) or Google Cloud Storage (via Google Cloud Functions) for archival purposes.
AI/ML data processing
Stream Aerospike data to AWS SageMaker (via AWS Lambda) or Google AI (via Google Cloud Functions) to accelerate your cloud native AI/ML pipelines.
Compliance
Encrypt Aerospike data for compliance using AWS KMS via Lambda, before ingesting it into your data lake.
Web application development
Develop a web application in the language of your choice to ingest and process Aerospike Change Notification (CDC).
Autoscale streaming connectors
Autoscale Aerospike streaming connectors for Kafka, JMS, or Pulsar to enable you to build a cloud-native streaming pipeline.
