Announcing Aerospike Connect: Add-on Integrations for Spark and Kafka

Srini Srinivasan
Founder and Chief Technology Officer
March 12, 2019|3 min read

Aerospike is pleased to announce the availability of Aerospike Connect for Kafka, Aerospike Connect for Spark and the Aerospike REST Client. The Aerospike Connect product and the REST client eliminate the need for custom coding to integrate the Aerospike database with other enterprise systems and dramatically reduce support, maintenance and server costs.

Aerospike is widely used for processing real-time decisions on the edge and is now increasingly used as a real-time system of record (see Figure 1). Such a system of record typically stores petabytes of data, is used at the core of an enterprise and distributes subsets of data to terabyte-scale database systems running at the edge as systems of engagement.

With Aerospike Connect, it now becomes possible for enterprises to leverage real-time data stored in Aerospike using big data frameworks that are already heavily used in the enterprise. Using the up-to-date real-time data from Aerospike, analysis and ETL applications using popular frameworks like Apache Spark and Apache Kafka can now vastly improve the quality and timeliness of their results in machine learning based applications.


Figure 1

Aerospike Connect for Spark combines transactional and historical data stored in the Aerospike database with streaming event data for consumption by machine learning and artificial intelligence engines using Apache Spark. Spark streaming and ML algorithms will now be able to use the real-time data in Aerospike to frequently update and improve models so that edge systems can improve real-time decisioning processes. Thus, Aerospike Connect for Spark significantly reduces iteration time for model improvements while improving the performance of real-time decisioning engines.

Aerospike Connect for Kafka makes it easy for enterprises to exchange data bi-directionally between the Aerospike database and enterprise transactional systems both at the edge and the core. Aerospike Connect for Kafka can be deployed in two modes: inbound only, for ingesting data from Apache Kafka into Aerospike; bi-directional, for ingesting data into and out of Aerospike. Furthermore, the infrastructure used to implement the Kafka support can be used to extract data from Aerospike into other custom end points.

Aerospike REST API is an easy and standard interface for the Aerospike database that enables developers to work with the Aerospike data layer without the need for custom coding. Developer can now interface in a familiar programming language of their choice for new applications and microservices.

As more and more digital transformation use cases emerge for real-time operations on massive amounts of data, the Aerospike Connect portfolio of integrations deliver a seamless and cost-effective way to power petabyte scale applications and analytics. Aerospike has the unique ability to fuel real-time data applications at a tremendous scale – and at a fraction of the cost. Now customers who require those benefits can connect data stored in Aerospike with a wide variety of applications throughout the enterprise.

Useful links:

Press release

Product pages: Aerospike Connect for Spark Aerospike Connect for Kafka

Solution Briefs: Aerospike Connect for Kafka Aerospike Connect for Spark