Is your data layer ready for operational AI? 6 infrastructure requirements

Enterprises that put artificial intelligence into live operations tend to hit the same wall. Once AI moves from analysis into action, a model that worked in the demo has to make decisions in millisecond windows against continuously changing reality, with millions of operations running at once, state changing with every transaction, and infrastructure costs climbing with every new workload.

Operational AI runs as a continuous operating loop. Each decision starts from authoritative state: the current record of what's true for a customer, an account, a computing job, or an order. The system assembles the relevant context around that state, makes a decision, and acts on it. The action has a consequence, and the consequence becomes the updated state, which is where the next decision starts.

The Aerospike operational AI technical assessment measures how well your data infrastructure keeps that loop running, across six dimensions. Each dimension is an infrastructure requirement, and falling short on any one of them can slow or break the continuous operating loop under real enterprise conditions at production scale.

Enterprises running operational decisioning at scale, such as PayPal, Barclays, Flipkart, and Wayfair, have been tested on all six. For each dimension below, we explain what it measures and show what it looks like when an organization has architected its operational AI to meet the requirement.

1. Freshness

How current is the data your decisions depend on?

Freshness measures how quickly a consequence becomes the updated state, from the moment something changes in the business to the moment the next decision can use it.

Take a declined card payment. Seconds later, the same card shows up at a different merchant, and the fraud model reads the account record to score the new attempt. If the decline hasn’t made it into the authoritative state yet, the model sees a clean first attempt instead of a card that was declined seconds ago, which is a strong fraud signal. The charge goes through, and when the real cardholder disputes it, the merchant usually covers the loss.

A sub-millisecond read doesn't help if the record is a few seconds old because ingestion or persistence fell behind under load. In the card example, a dashboard tracking read latency would look healthy the whole time the fraud model was missing the retries. That's why the assessment measures end-to-end freshness, from the event through ingestion, persistence, retrieval, and the application's response.

The Trade Desk delivers real-time bids across 10 billion user profiles

The Trade Desk runs one of the world's largest demand-side platforms, processing 12 million queries per second across 10 billion user profiles. Every bid decision depends on combining real-time situational data with anonymized data about the user audience and their interests. If the profile isn't available in time, the platform either bids without it or skips the opportunity entirely.

On their previous Cassandra infrastructure, user profiles took too long to become available for bidding. The Trade Desk's term for it was "thawing" the delay between a profile sitting in cold storage and being ready for a live bid decision. That delay cost them opportunities they couldn't measure because the bids never happened.

After moving to Aerospike, cold-store profile access runs at 8 milliseconds, down from what the team described only as too slow to bid on. The server footprint went from 500 machines to 60. The compute capacity that used to go toward data retrieval now goes toward bidding, which means more of the available inventory gets evaluated before the auction closes.

2. Latency

Do your slowest responses still fit inside the decision's time budget?

Latency is how long the data layer takes to answer a read or a write. In the loop, that's the time to serve authoritative state and relevant context to a decision and to record what happened after it. In production, watch tail latency: response time at the 99th (p99) and 99.99th (p99.99) percentiles. If a data layer's p99 is 200 milliseconds, one request in 100 takes at least that long, even if the average looks like 3 milliseconds, because averages and medians hide the slow requests in the tail. For a fraud model with a 100-millisecond budget, each of those slow lookups is a transaction it can't score in time. Depending on how the system is set up, it either declines the payment and turns away a legitimate customer or approves it without a score and takes on the fraud risk.

Barclays keeps fraud checks under 100 ms at p99.99

Barclays consolidated its fraud systems onto one machine learning platform running on Aerospike. Each fraud check has a fixed budget of 100 milliseconds, and that window has to cover both pulling the data and running the model that scores it. Every millisecond spent on the state and context retrieval is a millisecond the model doesn’t get. Dheeraj Mudgil, Vice President, Enterprise Fraud Architect at Barclays, puts the tradeoff in numbers. “When you only have about 100 milliseconds to detect fraud, even 40 milliseconds of database calls consume 40% of your detection budget.”

Barclays measures against that budget at p99.99 and now consistently gets response times under 100 milliseconds, with lower latency than the previous system.

3. Update behavior

What happens when multiple actors change the same record at the same time?

In operational AI, reads and writes have to coexist against the same evolving reality, often on the same records at the same moment. A fraud system evaluates overlapping transactions on the same account. In personalization, the engine updates session state while the recommendation model reads it; in agentic workflows, two agents can finish tasks that change the same inventory record.

The question is whether the second writer (agent or application) sees the first writer's update before making its own change. Without that guarantee, two writers can read the same account, each see a different version, and one of them writes back a result based on data that has already changed. Strong consistency guarantees that a write, once acknowledged, is visible to every subsequent read. That means the second writer reads the updated record, not the version before the first writer acted on it. Each decision treats the record as the authoritative state because each decision sees every change that came before it.

PayPal keeps risk decisions reliable across regions

PayPal's risk platform evaluates logins, account updates, wallet changes, purchases, transfers, and withdrawals. Each one has a consequence that becomes the updated state for whatever PayPal evaluates next. Multiple models may run in sequence or in parallel before PayPal returns a decision.

Strong consistency keeps each account update in order and visible to every subsequent read. Cross Datacenter Replication (XDR) copies those updates between PayPal's data centers, so risk models in each region work from the replicated account data. Together, they keep PayPal's risk decisions reliable across regions at more than 8 million database transactions per second. PayPal raised the share of fraud detection calculations finished within the required 200ms window from 98.5% to 99.95%, cutting misses by 97%.

4. Scale

Does your infrastructure sustain performance as data and traffic grow?

Scale means keeping latency and throughput steady as data grows, traffic spikes, and teams add new workloads, while also storing large volumes of data. As models and agents multiply inside one environment, the number of decisions running through the loop goes up fast. A system that performs well at 10 million operations per second but degrades at 50 million isn't ready for that kind of growth.

The test is linear scalability, meaning throughput grows in proportion to the capacity you add. If doubling the nodes roughly doubles operations per second at the same latency, you can plan for growth by adding hardware instead of re-architecting, re-engineering a cache, or changing the data model.

Flipkart hit 95 million transactions per second at peak

Flipkart serves more than 200 million users and hit 95 million transactions per second during its Big Billion Days sale, with Aerospike serving search, product listings, and recommendations through the peak.

Flipkart runs more than 170 Aerospike clusters year-round on bare metal under Kubernetes, managed through the Aerospike Kubernetes Operator, and keeps sub-millisecond latency for search and recommendations. It reports 100% availability with the operator and no production incidents caused by it.

At Barclays, scale meant data growth, with fraud data going from 3TB to more than 30TB in three years. Throughput reached 4x the prior system consisting of a relational database and cache, and tail latency stayed under 100 ms at P99.99.

5. Resilience

When a node, a network link, or a whole data center fails, does the system keep running?

Resilience is whether the system keeps operating through hardware failures, network partitions, data center outages, and traffic spikes without losing data or breaking SLAs. In operational AI, a failing data layer can also break the operating loop through stale decisions, lost transactions, or silent data corruption, which are harder to spot than a slowdown.

Typical requirements include no data loss when a node fails, automatic rebalancing, and replication across sites for disaster recovery.

AppLovin manages more than 70 billion ad requests a day

AppLovin processes more than 70 billion ad requests a day across 35 production clusters, 350+ servers, and about 250 billion stored records. For each request, the system has roughly 50 milliseconds to decide whether to bid and which ad to serve, so it can't wait on a slow profile lookup or a node that's down. John Krystynak, AppLovin's Founder and CTO, says the goal was “to be able to put a billion users into place and access any one of their records in a millisecond or two.”

AppLovin replicates data across sites with XDR and handles more than 1.4 million writes per second, with device profile lookups in about 1 to 2 milliseconds. It has scaled to serve more ads while reducing its server count, across more than 700 TB of disk and 70 TB of RAM.

6. Economics

Can you afford to run this at production scale as workloads grow?

Economics decides whether a production AI system can keep growing. An architecture that delivers real-time state, context, and decisioning at the right latency and scale, but needs all data in DRAM, will see infrastructure costs grow linearly, or faster, with every new workload, agent, and data source.

Some costs don't show up on the database bill. Teams whose primary database can't serve reads fast enough often put a cache like Redis in front of it, then pay for and operate two systems. The cache needs its own monitoring, failover, and consistency management.

Forter and PayPal cut their server counts

Forter replaced a two-layer architecture, with Redis as the cache and Couchbase as the persistent datastore, with a single Aerospike platform for real-time browsing-session fraud analytics. Consolidating cut servers from 178 to 50 and hardware costs by 40%, and p99 latency dropped from 13.3 ms to 1.3 ms. 

PayPal replaced its previous in-memory database because it was getting too expensive to scale. On Aerospike, PayPal runs on 120 servers instead of the 1,024 it needed before, about 8x fewer. It projected $9 million in hardware savings, with estimated hardware cost falling from $12.5 million to $3.5 million and total cost of ownership (TCO) down more than 80% over five years. Aerospike's patented Hybrid Memory Architecture (HMA) makes the smaller footprint possible by keeping indexes in memory and data on SSD, so PayPal doesn't need to load the full dataset into DRAM.

Where does your architecture stand?

Each enterprise in this post started by identifying the decision, the authoritative state and relevant context behind it, and the time budget it had to meet, then measured its infrastructure against that. Aerospike has spent years solving this class of problem in payments, AdTech, telecom, and e-commerce. AI is making it more universal across enterprises and more intense within each environment, so the same six dimensions now apply well beyond those industries.

The Aerospike operational AI technical assessment scores your data platform on these six dimensions in a few minutes. You get a readiness rating for each dimension, plus recommendations to improve your readiness for operational AI at scale.