Introducing Aerospike Vector Search

Artificial intelligence (AI) is omnipresent across use cases and industries. Virtually every business process is fair game as organizations look to maximize AI’s value. Generative AI (GenAI) in particular, is all the buzz, especially large language models (LLMs) like ChatGPT. At the same time, classical, or “predictive” AI, is also undergoing changes as vector solutions emerge with advanced techniques that improve the performance and efficiency of existing AI and machine learning (ML) use cases and support emerging ones. Vectors are also being used to encode sets of feature data in more compact forms to feed into ML models.

Challenges in modern AI: Taming data surges and dangerous outcomes

Companies are getting serious about modernizing their AI and ML applications and adopting GenAI solutions, but they’re also hitting roadblocks. Traditional databases struggle to effectively handle and process the vast influx of real-time data from a diverse range of sources, which is crucial for powering AI models and generating timely outputs. The natural language capabilities of LLMs are impressive but demand loads of compute power (and deep pockets). LLMs also tend to “hallucinate” sometimes dangerous outcomes, posing serious problems for companies who want to support generative AI applications in production. The AI applications of the future will need to deal with this issue, as well as the onslaught of heterogeneous data, and they’ll need a solution that can corral and serve it up quickly at scale. 

That solution is Aerospike. 

I joined Aerospike about two years ago because it was clear to me that the company offered something unique: a powerful, large-scale, high-performance system that is made for this new AI age. The company already had, and continues to add to, a long roster of enterprise customers - PayPal, Adobe, Myntra, Flipkart, and Riskified, to name a few - who use Aerospike for classical AI use cases like e-commerce, fraud detection, and customer 360.

Our recently announced funding will accelerate our vision for production-level AI, which requires functionality that has not been generally available on the market and which enterprises need in order to expand their AI innovations and tap into AI’s enormous revenue opportunities. 

From challenges to opportunities: Aerospike Vector Search

Vectors are emerging as the new data representational entity that is fundamental to many aspects of modern AI/ML. The generality of vectors in their ability to encode diverse content types like text, images, and audio/video in a homogenous way makes them a versatile tool. Furthermore, the ability of vectors to capture the semantic meaning of content, as opposed to just the words or pixels, makes them a powerful mathematical tool to do the kind of fuzzy reasoning that is underpinning progress toward artificial general intelligence (AGI).

Vectors are part of the internal lingua-franca of most modern ML models and are fundamental to fueling both classical AI/ML and GenAI applications. To be most effective, vector databases need the ability to ingest, process, and query vast amounts of data efficiently, quickly, and cost-effectively. In addition, semantic approximate search using vectors introduces a new dimension of performance - that of accuracy, or quality, of retrieval. Aerospike is stepping in to fill this need with the only enterprise-grade vector solution that delivers consistent accuracy at scale. 

Our highly performant vector solution inherits the search and retrieval capabilities of our core database and extends its performance, scale, and cost benefits for vector storage and retrieval. As customers of our core database already know, the full-fledged, multi-model Aerospike database doesn’t hit the performance or scale limits of competing databases with features such as fast retrieval across a large key space to rapidly handle billions of records. Additionally, Aerospike provides a range of storage configurations to suit our customers’ performance and cost needs, including our patented Hybrid Memory Architecture, combining in-memory and SSD-based storage for high performance at low cost. 

Aerospike’s vector database extends this flexibility by adding an independently scalable compute and cache layer above the core storage layer that handles the complex vector index management and traversal operations. This tiered architecture enables our vector solution to offer customers different operating points of choice depending on their application needs and cost sensitivities. These choices span high ingestion/indexing rates, high query throughput, or low latency of vector queries (or even all of them at once!). On the quality front, we combat the retrieval accuracy reduction that often accompanies fast parallel ingestion using patented techniques that balance index freshness and quality, as well as a self-healing mechanism that will continue to improve the index behind the scenes.

Combining graph and vector for deeper connections

Aerospike’s ability to offer key-value, document, graph, and now a vector database on the same platform gives us the unique ability to act as a flexible and extensible AI data platform. Graphs and vectors are both modern extensions to the classical feature stores used in AI/ML. Graphs dynamically capture, update, and retrieve relationship linkages between entities more naturally than traditional key-value databases. Meanwhile, vectors provide the compact and dense mathematical encodings that capture the semantic meaning of the entities and matching based on semantic similarities. We anticipate that the two will work together for a variety of AI use cases: semantic search enabling discovery of new connections in knowledge graphs, for example, or conversely, using semantic clustering on graphs to allow for better cross-entity reasoning.

Aerospike was built for AI

Aerospike’s traditional core strengths in scale and performance, when leveraged for our vector offering, will further improve the performance of classical predictive AI/ML and fill a critical need in the evolving generative AI space. Our investors recognize how Aeropsike was built from the ground up to be AI-ready, and with their backing, we’re ready to bring AI to full fruition for mission-critical enterprise applications. 

Learn more about Aerospike Vector Search.