Retail/eCommerce Data Architectures Driven by the Amazon Effect

default photo
Bharath Yadla
Vice President Product Strategy, Ecosystems for Aerospike
November 12, 2018|4 min read

Amazon has been dominating with its scale of operations into every industry sector that it has been getting into. We call that “Amazonian effect” – these Amazonian effects have been brutal not just on brick-and-mortar retailers but also on online businesses as well.

Amazon’s commerce dominance is due to the agility and scale at which Amazon does its business with the help of their intelligent IT. Central and core to their intelligent IT is their Data Architecture that powers both agility and intelligence for smart decisioning in real time.

Amazon’s intelligent data architecture powers critical decisions for ecommerce that we at Aerospike have also found our clients using our database for.

Providing both product and user recommendations in real-time

The ability to provide product recommendations that drive both cross-sell and up-sell based on product characteristics and user purchase behavior is crucial in real-time. This would need real-time analysis of user, clickstream behavior, purchase history, basket analysis, social media profiles – much like AdTech targeting. For ecommerce providers, this ability translates to real dollars.

35% of’s revenue is generated by its recommendation engine

For product recommendations, the ability to provide recommendations in near real-time for either clustered products or substitutes based on availability, out of or in-stocks, price points, etc. can determine whether the ecommerce site gets a sale or not.

Fraud prevention of CNP transactions in real-time

This is a big change and transformation for traditional retailers that usually process Card Present transactions at the cash registers. If CNP, you have less time to determine fraud. If CP, then you can push the decision to the card provider. One circumstance ripe for fraud is for multi-channel payments (a combination of e.g. gift card with mobile) need to be processed as CNP transactions. This added complexity gives fraudsters more surface area and opportunity to attack. This would drive the requirements of real-time decisioning and processing of transaction into very compressed cycles – in the millisecond range.

The accuracy of identifying fraud highly depends on the amount of data that can be processed by any decisioning engine within a very small span of time. Aerospike has multiple customers that rely on our database to support their fraud prevention solutions.

Dynamic pricing to compete effectively with other marketplaces

This is another major tremor that is felt by the traditional retailers due to Amazon. Typically, product pricing and promotion is based on lot of data like the product propensity, seasonality, value estimated, consumer willingness to pay, demand, competitive prices, etc. All of these data points in an ecommerce-first business is driven by external environmental factors and the terrain changes rapidly. This would mean that data needs to be brought in from multiple sources and intelligence is to be gleaned, processed and decisions to be made in real-time. The speed with which a retailer can react to pricing or proactively change pricing to defend or offend the competition is a must and certainly hinges on the effectiveness of their data architecture.

Additional real-time solutions for Ecommerce

In addition to product recommendations, fraud prevention and dynamic pricing, other real-time solutions our customers depend on Aerospike technology for that Amazon no-doubt streamlines include real-time messaging, effective stock information, merchandising movements, and weak consumer demand indicators.

What real-time Ecommerce solutions require

The above mentioned decisions are heavily data dependent – both in ability to handle the amount of data (scale) for accuracy and the ability to process for decisioning (speed). This in turn depends and is driven by how the “Enterprise Data Architecture” is built.

Typically, traditional retail data architectures have been following the patterns of traditional enterprise IT frameworks: “batch data syncing” is a hallmark for price sync, product sync ,etc. They are rooted in transactional data stores and data warehouses, but attempt to cover real-time needs with tiny caching architectures covering very specific items e.g. page loads or pricing sync. However with omnichannel experiences becoming new norm, and ecommerce business hours being always-on, unless the data architectures fundamentally transform to fit the agility and scale, retailers with a traditional data architecture won’t stand a chance versus Amazon.

Advice from-the-trenches

All of the above critical decisions in ecommerce / retail drive both defensive and offensive strategies against Amazon for retailers. In order to build “real-time data architecture”, any enterprise would need to devise their data architectures around databases that have the following characteristics:

  • Ease of scale

  • Super low-latency for both reads & writes

  • Predictably performing

  • Ease of operations

  • Yet lower cost of ownership

This is exactly what we have been learning from our digitally native ecommerce customers and traditional retailers who are raising defenses and offenses to fight Amazonian effects with a better “Data Architecture”.

To learn more, feel free to register for our online webinar, “Ecommerce, ML and Recommendation Engines: Why The Database Matters.”