Set index

You can index the membership of records to their set by creating a set index to improve set-level operations.

Set indexes are compatible with warm restarts.

Memory consumption

Set indexes consume approximately 16 bytes per record in the set. See the capacity planning guide for more information.

Effectiveness

Without a set index, small sets require a full namespace primary index traversal to locate its records. Adding a set index improves performance by avoiding this overhead, though the benefit diminishes as set size increases relative to the namespace.

The set index can be dynamically configured, even on a single node. Then, queries with and without a set index can be compared to each other. If the set index does not provide a significant advantage, it can be discarded. Set indexes may be enabled and disabled even while queries are in progress.

A query uses a set index if it exists, otherwise it uses the primary index.

Key considerations

• The 1% rule: Generally, if a set exceeds 1% of the namespace, a set index offers diminishing returns and may not justify the memory cost.
• I/O impact: For large, disk-resident records, the cutoff is often lower because device I/O becomes the primary bottleneck rather than index traversal.
• Optimization: The ideal threshold depends on your specific hardware and use case.

Set indexes make the secondary index hack obsolete

Historically, users tracked small sets by creating a secondary index on a dedicated set name bin. However, set indexes are a more efficient alternative.

Why switch to set indexes?

• No schema changes: You don’t need to add extra bins to your records.
• Faster startup: Significantly lower impact on node warm-up times compared to secondary indexes.
• Simple management: Same info command/asadm interface used for secondary indexes.
• No filter required: You can query all records in the indexed set without adding a filter predicate.
• Correct and paginated: Queries on a set index are always correct and support pagination.

Low-risk implementation

Set indexes can be enabled or disabled dynamically, even on a single node and while queries are running. This allows you to perform real-time A/B testing: if the performance gain doesn’t justify the memory usage, you can discard the index instantly without downtime.

Managing set indexes

Starting with Database 8.1.2, create and remove set indexes using sindex-create and sindex-delete. This uses the sindex-admin privilege (same as secondary indexes) and works on Aerospike Cloud. The legacy method using enable-index or the set-config info command is deprecated.

For details and examples, see Managing sets in a namespace.