How Dataminr and AppsFlyer deliver real-time AI decisions across billions of events per day

Every day, millions of microdecisions unfold in the background of modern systems: fraud engines approve or deny payments instantly, crisis detection models parse thousands of live signals before a headline breaks, and recommendation systems update in real time as users scroll. 

In this new frontier of AI decisioning, data platforms must sustain both intelligence and reliability under extreme load. Achieving this balance requires not only powerful models but also predictable, low-latency systems that can react continuously without compromising consistency or budget.

At AWS re:Invent 2024, this topic took center stage. Evan Cummack, Chief Product Officer of Aerospike, was joined by Tony Du, Senior Director of Engineering at Dataminr, and Victor Gershkovich, R&D Group Leader at AppsFlyer, to discuss how their teams turn data into action.

How Dataminr turns billions of signals into instant alerts

Dataminr is a real-time information discovery company that provides instant alerts on global events and threats across public, private, and government sectors. From natural disasters and transportation incidents to geopolitical conflicts and cybersecurity breaches, the platform continuously analyzes billions of text, imagery, audio, and sensor data each day to identify emerging events.

To achieve that coordination, Dataminr’s AI stack, built on Aerospike, concurrently combines more than 50 models across natural language processing, computer vision, and audio analysis:

• Predictive AI: Dozens of lightweight models classify and cluster raw signals, performing deduplication and relevance scoring at ingestion. Each output is written to Aerospike for immediate retrieval downstream.

• Generative AI: This builds on those enriched records to produce dynamic, continuously updating event briefs that evolve as new context arrives.

With 50-plus models operating in parallel, even microseconds of I/O latency can delay downstream inference. Aerospike acts as the shared memory layer between models, synchronizing features, metadata, and cross-model references.

"Many of our AI models are trained for a specific use case on a specific domain type of content, and oftentimes they rely on each other to function properly," Du explained. "Our orchestration system manages these connections seamlessly using technologies such as Aerospike. The database serves as a sync for all of our AI models' output. It also serves as a source where our orchestration system would pull the data from and feed back to other AI models in real time."

The result is a system that filters out 95% of noise at ingestion (e.g., discarding duplicates, low-confidence signals, and irrelevant data), ensuring only verified, high-confidence events surface to end users. For example, during the 2024 Francis Scott Key Bridge collapse in Baltimore, Dataminr delivered an alert over an hour before mainstream news coverage. Similarly, when a U.S. airport network received a Dataminr alert about an imminent DDoS (Distributed Denial of Service) attack, it was able to immediately fortify its systems. Eight minutes later, the attack hit with over 3 million requests but failed to breach the secured infrastructure.

How AppsFlyer scales ad attribution to 7 million queries per second

AppsFlyer is a mobile attribution and marketing analytics platform that helps brands understand which ads, campaigns, and channels drive app installs and conversions. When billions of impressions and clicks need to be matched to downstream actions in real time, the question isn’t how fast a model can run, but whether the underlying data platform can keep pace with the market itself. 

Modern ad attribution depends on the ability to connect distributed, ephemeral events into a coherent sequence that defines user behavior. Each interaction must be evaluated against thousands of concurrent campaigns, budgets, and device identifiers, with a decision window that lasts less than a blink. Speed matters because attribution data directly informs budget allocation and real-time bidding. Delayed attribution means campaigns can't optimize in-flight, and ad spend is wasted on underperforming channels.

AppsFlyer’s real-time attribution engine is engineered for that speed. The company processes hundreds of billions of events each and every day, handling roughly 2.5 million writes per second and 4.5 million reads per second while maintaining p99 latency under one millisecond. Its architecture fuses horizontal scale with predictable performance:

• Event ingestion and normalization: Incoming data from SDKs, APIs, and partner feeds is validated and deduplicated in real time.

• Low-latency key-value storage: Aerospike serves as the foundation for high-speed lookups, allowing the system to instantly link a click or impression to a later install or transaction.

• Predictive adjustment: Lightweight models monitor incoming signal quality, detect anomalies, and automatically rebalance workload distribution.

• Analytical refinement: Downstream, larger models use aggregated outcomes to train and optimize attribution logic without interrupting the live stream.

This design allows AppsFlyer to scale dynamically based on campaign traffic and seasonal load. “Our traffic trends are very dynamic and unpredictable. For example, large sports or shopping campaigns can increase traffic dramatically,” Gershkovich said.

Achieving that elasticity is only half the challenge. In attribution, consistency latency matters as much as peak throughput. The system must guarantee that a write or read operation takes the same amount of time, whether traffic volume is 1 million or 100 million events per second. Aerospike's Hybrid Memory Architecture maintains uniform latency across varying loads by keeping hot data in memory while persisting to storage, ensuring performance doesn't degrade as data volume grows.

That stability transforms latency from a liability into an advantage. When every lookup happens in microseconds, attribution logic can adapt in real time and incorporate context like device type, campaign ID, or session metadata on the fly. 

On the infrastructure side, Kubernetes and the Aerospike Kubernetes Operator automate provisioning, version upgrades, and cluster scaling. AWS Graviton-based instances delivered over 20% better price-performance compared to Intel-based compute, while the overall shift to Aerospike reduced infrastructure costs by roughly 60% compared to their previous database solution.

What's next for real-time AI decisioning

As we've seen with Dataminr and AppsFlyer, real-time AI systems can take very different shapes. One detects global crises while the other optimizes ad spend, but both are built around fast decision paths that operate inside strict latency windows, with slower analysis happening outside the critical path to improve future decisions.

Together, these components form a feedback loop where real-time decisions generate outcomes that are continuously analyzed, refined, and fed back into the system. This tightens models and thresholds without interrupting live operations.

This shift toward continuous, self-improving systems marks a fundamental change in how AI systems operate. And behind the scenes, platforms like Aerospike will continue ensuring that as AI systems grow more sophisticated, they never outpace the infrastructure supporting them.