Skip to main content
Loading

Developing with Vector Search

AVS provides a set of API abstractions for working with vectors. Before learning about your specific client, it is important to review these concepts at a high level.

Vector Records

You can interact with AVS by putting, getting, and searching vector records. A record is any object that contains both your vector embeddings and any data you want to associate with that vector. For example, an image search application has a vector embedding that contains a statistical representation based on the output of a model on a JPEG image, and a path to where that JPEG is stored. This is how the semantic image search example application works.

You can add as many vectors to a record as you like, and you can create and search across records separately. This can be helpful for testing various models or performing different types of searches (e.g., a text search versus an image search). It also makes it simple to work with existing datasets and pipelines. Simply add a vector embedding to your record and create an index.

You interact with records via a key, which is used for put and get calls.

Indexes

To search across a set of vectors, you need to build an index. An index is how AVS traverses the HNSW neighborhoods when performing queries. Indexes are independent from your records, and you can create an index using the Admin Client. Based on your model, configure each index for a distance calculation and dimensionality. If either of the distance calculation or dimensionality changes, you must rebuild your index. You need to specify which field in your record has the corresponding vector to be indexed. After creating an index, AVS will start scanning for records with that vector bin in Aerospike. For more information about how AVS uses Aerospike, see Aerospike Vector Search Architecture.

AVS supports cosine, dot-product, manhattan, squared-euclidean, and hamming distance calculations currently only using HNSW. More index types are planned for the future.

Searches

After building an index across a set of entries, you can search those entries quickly and effectively. You need to specify an index to search across and provide a search vector. Your application needs to generate that search vector from a machine learning model. For example, if you used the CLIP model to generate image embeddings, you need to utilize the CLIP model to generate an embedding based on the search term provide by the user.