Gartner recently published a market guide for hybrid transaction/analytical processing (HTAP). It is a must read paper for CIO, data professional and anyone who is involved with analytics driven digital transformation initiative. Gartner’s market guide is a good overview of the current state of the in-memory computing technologies and how they enable HTAP. Since Aerospike is one of the vendors recognized in the guide, I thought I would use this post to share our point of view on HTAP and how database technologies have evolved to support it. You will find a link below this post for complimentary access to the report.
Transactions and analysis– closing gap between data and action
Earlier management information systems enabled reporting-oriented, “descriptive” analytics on operational data. These were produced after the fact, so there was always a time gap between the reporting, decisions, and action taken on the analysis because someone had to review the report and create the insight.
As technology evolved, diagnostic and then predictive capabilities became available. The gap between data and action got smaller, but never closed completely. With the advent of business intelligence, decision makers could derive prediction scenarios of the future by looking at historic data, but there was still a time gap from insight to prediction and then to action.
Transactional analytics is a further evolutionary step and one that closes the gap between analysis and action. With transactional analytics, insight and decision take place instantaneously with a transaction at the moment of engagement with the customer, partner or device.
For instance, let’s say a consumer selects, pays for, and downloads a movie on his tablet. During this transaction, his credit card is charged and checked for fraud while personalized recommendations are served to him in real time. The entire transaction occurs in well under a minute and database transactions happen in milliseconds. Transactional analytics is in play at the moment of interaction to drive customer satisfaction, accept payment, deter fraud, and bring additional revenue sources via a real-time offer. Each of these interactions is a business moment –the most valuable instance when a business critical event happens, and it is driven by systems of engagement that are using transactional data in analytics applications.
Engagement systems focus on mission-critical interactions in the business moment, and span activity that occurs across mobile, cloud, and social. These systems are situationally aware and contextually based on behavior at the real-time instance of the transaction. They process enormous quantities of data as interactions occur via click, swipe, micropayment, device reading, cyber packet transmission, etc.
Traditional architectures are lacking
Traditional data architectures maintain the transactional application and analytics applications in different silos and have data integration pipes to bring data between the two. This architecture cannot support the data load and processing speed required by mission critical engagement systems using transactional data for real-time analytics. Here’s why:
Complexity. With traditional architectures, data is extracted from the operational database, transformed and loaded into the analytical database. This requires the database replication, extraction, transformation, and loading (ETL) tools, along with enterprise service buses (ESBs). Message-oriented middleware (MOM) and other integration tools.
Latency. Traditionally, it can take from hours up to weeks from the moment data is generated by a transaction to when it can be used for analytics. Although this can be adequate for other types of analytics or processes, it doesn’t work for others that need more immediate access to data.
Synchronization. If analytical and transactional data storage are separated, when business users want to drill down into the operational database for data details, the data source is often out of sync because of analytic latency.
Data duplication. In a traditional architecture, multiple copies of the same data must be administered, monitored, managed and kept consistent, which can lead to inaccuracies, timing differences, and inconsistencies.
Lean and simple architecture
To deliver transactional analytics requires a new architecture that is very lean and simple, one that brings transactional data to analytics applications together in real time, at the interaction point, to enable split-second insights and decisions.
Gartner refers to two approaches to transactional analytics: point-of-decision HTAP, and in-process HTAP:
Point-of-decision HTAP, whereby the transaction processing and analytic aspects are segregated into distinct, independently designed applications. This allows advanced analytics to be performed on “live” transactional data, something that is very hard to achieve in traditional architectures.
In-process HTAP, whereby real-time analytics and transaction processing techniques are woven together in the same application guide and optimize the execution of transactional processes.
All the vendor offerings included in the Gartner Guide, with the exception of Aerospike, fall into one of three categories:
Traditional relational databases
that are suited either for operational or analytics
New relational or NoSQL databases
that are optimized primarily for scale.
Caching
, which are better suited to front relational databases to improve speed.
When a database is born analytics, operational or caching, is it architected to do that first. The result is that it cannot be optimized for transactional analytics and isn’t lean and simple. It will run with higher latencies, and without the reliability and predictability required at higher data volumes and workloads.
Database Purpose-built to power real-time transactional analytics applications
Aerospike is built from the ground up for high speed, high volume data processing required by transactional applications with decision engines. Hybrid memory architecture differentiates Aerospike from other databases. Unlike pure in-memory databases, Aerospike runs in volatile memory (DRAM), but also uses non-volatile memory (flash storage, SSD) which gives us the advantage of processing more data with low latencies with fewer server nodes than any equivalent database, with as low as a 1:4 ratio.
Aerospike maintains single digit millisecond latencies in processing millions of transactions per second and can do so predictably. It has built-in reliability and functionalities to support mission critical transactional analysis applications. Aerospike Smart Client™ is built into Aerospike client drivers and development libraries, and enable applications to maintain stability with changing conditions.
With rich, optimized functionality, development time and hassles are decreased with Aerospike, which lowers TCO.
Not a “me too”
Processing data for transactional applications that have advanced decision algorithms is all Aerospike does. 100% of our customers – who are in financial services, e-commerce, telecom, marketing, and other industries – are deploying Aerospike in transactional analytics for mission critical applications. They have come to us for predictability, reliability and speed at high workloads. They have a comfort level with Aerospike, and that is why we see increasing growth in our customer base.
Read the Gartner report
The Gartner Market Guide for HTAP-Enabling In-Memory Computing Technologies is available here.
For more information visit:
Aerospike blog post “Aerospike Database Powers Systems of Engagement” here
Aerospike blog post “What’s an Aerospike Anyway?” here
Aerospike Architecture here.
Sources:
Gartner Market Guide for HTAP-Enabling In-Memory Computing Technologies (link)
