Lexical search vs. semantic search in data architectures

The transition from traditional keyword matching to intent-based semantic search changes how organizations manage information retrieval. Lexical retrieval systems have served as the industry standard for decades because they’re precise for structured data and exact keyword matches by using inverted indexes. 

Conversely, semantic search uses natural language processing and vector search to interpret the contextual meaning of user queries, helping a search engine return relevant information even if the query didn’t use the exact words. 

But beyond relevance, choosing between these methodologies also determines the underlying architecture, computational cost, and how consistently and reliably the system works.

What is lexical retrieval?

Lexical search matches a query against an index of stored text. This could be the literal characters, or the basic units of text, usually words, that a search engine uses to index and match content, which are called tokens. 

This approach is rule-based and predictable because it uses exact words or simple variations of them within a text to decide what results to return. Enterprises usually use lexical retrieval to search structured data such as product codes, identifiers, or technical documentation where the user is looking for an exact keyword. 

Text analysis, tokenization, and morphological processing

First, data needs to be analyzed and undergo tokenization, where raw text is broken down into individual basic terms, followed by normalization techniques such as stemming or lemmatization. 

• Stemming reduces a word to its root form, such as converting "searching" and "searched" to the root "search,” which helps the search system match variants of a term.

• Lemmatization reduces a word to its basest “dictionary” form, or “lemma,” analyzing vocabulary and part-of-speech, such as converting "better" to "good" or "ran" to "run." Unlike stemming, it delivers a valid word rather than a truncated root.

This processing also includes synonym expansion, where predefined lists of related terms broaden the search query. However, this is considered an extension of token matching rather than the system understanding the context. 

Engineering the inverted index

Next, most lexical search implementations use the inverted index, a data structure that maps each unique token to a list of documents or records containing that token. When the search system runs a search query, it finds the corresponding lists, often called postings lists, for each word in the query and then performs intersections or unions to find matching documents. This finds results quickly even with big datasets because the system uses the precomputed index rather than scanning the whole table. 

Scoring and relevance with the BM25 algorithm

To provide an ordered list of results, lexical systems use ranking functions to decide how well each document matches the query. The current industry standard for this task is the Best Matching 25 (BM25) algorithm, which improves upon earlier TF-IDF (Term Frequency-Inverse Document Frequency) models by giving less extra credit for repeated words and adjusting for document length so longer texts don’t dominate the results.

M25 uses tunable parameters, typically k (controlling saturation) and b (controlling length normalization), to refine its behavior for specific datasets. 

What is semantic search? 

Semantic search changes the search experience by interpreting the intent and semantic meaning of a query rather than merely matching an exact keyword. With natural language processing (NLP) and machine learning models, semantic search engines convert unstructured data into numeric representations called vector embeddings. These embeddings are placed into a continuous vector space with many dimensions, where items that are closer together represent more similar meanings. 

Generating vector embeddings

So how do you turn text to vectors? Enterprise data, such as product descriptions or technical manuals, is broken into smaller units called chunks and passed through neural models, such as Bidirectional Encoder Representations from Transformers (BERT) or Sentence Transformers. Each chunk is converted into a dense vector, typically containing 768 or 1024 dimensions, where each dimension represents one aspect of the semantic content. 

Mathematical similarity and distance metrics

Unlike lexical search’s token intersections, semantic search finds relevant information by calculating the distance between the query vector and document vectors, using one of several methods:

The choice of metric must align with the specific embedding model used so the search query retrieves relevant results accurately. 

Semantic understanding and intent capture

Semantic search is good at handling complex queries where what the user is looking for isn’t easily described with specific keywords. For instance, a lexical system search for "budget-friendly smartphones with high-quality cameras" might not find a document that uses "affordable" instead of "budget-friendly.”

A semantic search engine, however, recognizes the semantic similarity between these concepts through its high-dimensional representations, returning relevant information that lexical keyword matching would omit. 

Infrastructure tradeoffs and the computational cost of search

While semantic search understands the user question better, it’s a bigger computational and memory load on the system than lexical retrieval. Storing dense vectors requires more RAM than the sparse inverted indexes of full-text search, which adds up when datasets grow. 

How much memory do vector databases use?

A standard 1024-dimensional embedding, stored in float32 precision, uses approximately 4 KB per vector. In a production environment with a replication factor of 3 (or 2 in the case of Aerospike) for high availability, that means 12 KB per indexed record. For 100 million vectors, that means 1.2 TB of RAM. 

That’s a lot. To reduce these costs, enterprise teams use techniques to make the vector data less precise, and so take up less space. 

Scalar quantization reduces the precision of each dimension from a 32-bit float to an 8-bit integer, reducing storage requirements to 25% while still providing almost the same retrieval quality.

Techniques such as Matryoshka Representation Learning reduce vectors to lower dimensions, such as 128 instead of 1024, for an additional 56.6% storage reduction.

Comparing lexical and semantic infrastructure needs

Unlike lexical retrieval, where the index is compact and runs efficiently on standard hardware, semantic search requires high-performance multi-core processors and specialized vector stores for the similarity calculations.

Unlike lexical retrieval, where the index is compact and runs efficiently on standard hardware, semantic search requires high-performance multi-core processors and specialized vector stores for the similarity calculations.

Performance benchmarks and real-world evaluation

Okay, but how well do they work? This gets tested by the Benchmarking-IR (BEIR) benchmark, which evaluates models on how well they answer medical questions, retrieve legal documents, and search code they haven’t seen before. 

Lexical vs. dense retrieval in zero-shot settings

But while dense semantic models are becoming more powerful, lexical BM25 still works, particularly when a semantic model has not been specifically fine-tuned. 

Latency and throughput considerations

For high-performance systems, latency is as important as accuracy. Lexical retrieval is generally as much as ten times faster than semantic search for exact matches in small to medium datasets. However, as datasets grow, semantic search using approximate nearest neighbor (ANN) algorithms finds similar items much faster than checking every option individually, even as the dataset gets large, by narrowing the search space to a subset of candidate vectors. 

Hybrid search as the production standard

Because neither lexical nor semantic search is perfect by itself, the standard hybrid search uses both. Hybrid retrieval uses both methods, running BM25 keyword matching and vector-based semantic search in parallel to merge their results into one search result list. 

Result fusion through reciprocal rank fusion

So how do you combine the two methods? Lexical scores are based on term frequency and rarity, while semantic measures are based on vector distance. 

Hybrid systems combine multiple rankings using reciprocal rank fusion (RRF). Instead of relying on raw scores, RRF focuses on a document’s position in each ranking. Higher-ranked documents get more weight, and a constant factor determines how strongly the top positions influence the final combined ranking.

Documents appearing at the top of both lexical and semantic lists receive the highest final score, so the system captures both precise facts and what the user meant. 

Multi-stage reranking pipelines

For even more accurate results, organizations often use a two-stage retrieval pipeline. 

In the first stage, fast retrievers such as BM25 and ANN vector search find a candidate set of several hundred documents. 

In the second stage, a cross-encoder reranker, which uses a lot more computational resources, applies neural logic only to those top candidates to produce the final ranking. This architecture gets both sub-millisecond initial retrieval and precise results from language models.

BEIR 2.0 Benchmarks (nDCG@10)

These results show that dense retrieval now consistently outperforms lexical search by 10-12% in average ranking accuracy (nDCG@10). However, the gap closes in specialized domains such as legal or technical documentation where exact keyword matches are important. 

Furthermore, dense models often struggle with "adversarial" queries, such as those containing negations (e.g., "smartphones without OLED"). Lexical systems are sometimes more reliable because they match the "without" or "not" tokens.

Addressing the challenges of request fan-out and tail latency

In big environments, one user interaction rarely results in one database call. Instead, a complex search query often triggers a "fan-out" of operations, where a parent node coordinates dozens or hundreds of internal requests, such as fetching embeddings, querying multiple vector shards, and retrieving metadata.

The mathematics of tail latency amplification

This becomes a problem because the overall response time is not determined by the average latency but by the "weakest link,” or the slowest operation in the chain. What that means is, if a database call has a 99th percentile (P99) latency of 1ms, a user interaction requiring 100 parallel calls has only a 36.6% chance of completing without experiencing a tail-latency delay.

Impact of fan-out on user experience

As systems grow and fan-out increases, the parent request response time converges toward the worst-case child response time. This means tail latencies need to be strictly bounded to prevent this from happening. 

How to bound tail latency 

Enterprise systems manage this through techniques such as hedged requests, which send a duplicate request to a secondary replica if the first takes longer than the 95th percentile, and by using architectures that reduce unpredictable stalls associated with garbage collection or non-budgeted background maintenance. Databases written in C or C++ are often preferred for these high-fan-out search workloads over managed runtimes because they offer more predictable memory management. 

Aerospike and search

Getting the right balance between the precision of lexical retrieval and the context-awareness of semantic search is important today. While lexical search provides the necessary speed and exactness for structured identifiers, semantic search provides the intuitive, intent-based interactions that users now expect. Combining them requires a high-performance, low-latency data layer that remains predictable under the volatile conditions of real-world production traffic.

Aerospike does this by providing the predictable foundation to manage large-scale request fan-outs and volatile usage patterns. Its patented Hybrid Memory Architecture delivers near-in-memory performance at the cost of NVMe storage, so hybrid search systems produce consistent latencies without cache refreshes or gradual performance changes. 

Organizations looking to grow their search infrastructure while keeping systems running without overprovisioning should explore the predictable behavior of the Aerospike Database.