Tutorial for Expression Indexes

This tutorial introduces expression indexes in Aerospike Database. This tutorial assumes you are familiar with secondary indexes and expressions in Aerospike Database.

Overview

Expression indexes build sparse secondary indexes over computed values defined by an Aerospike expression. Instead of indexing every record or persisting derived bins, the index evaluates your expression and includes only the records that match.

In this tutorial, you will:

• Create your first expression index
• Run your first query against it
• Compare expression indexes against traditional secondary indexes

Note

Prior to Database 8.1, secondary indexes were limited to raw data, or fields other than the primary key.

Why it matters

• Simpler application code — push filter logic into the index; no extra bins or duplicated client filters.
• Reduced memory footprint — indexes include only relevant records.
• Total cost of ownership — reduced RAM and compute requirements lower infrastructure spend.
• Faster queries — less to scan means lower latency.

When to use expression indexes

Use them when you would otherwise:

• Persist a derived value solely to make it indexable.
• Index every record even though only a subset are needed for business logic.
• Re-implement the same filter logic across multiple clients or languages.

Get started

Prerequisites

• A running instance of Aerospike Database 8.1.0 or later
• Updated Client (minimum versions):
  • Java: v9.1.0+
  • Python: v17.1.0+
  • C: v7.1.0+
  • C#: v8.1.0+
  • Go: v8.4.0+
  • Node.js: v6.3.0+
  • Rust: not available
  • PHP: not available

Verify your server version using asinfo:

    # Verify you’re running Aerospike 8.1.0+
    asinfo -v build

Create expression, index, and query

1. Create the expression.

   The following example creates the “adult users in target countries” expression. For this example, let’s assume an adult is someone 18 or older, and our target countries are Australia, Canada and Botswana.

   For this tutorial, we’ll generate the Base64 representation of the expression in the client and then use the asadm command-line tool to create the index. This approach separates application logic (defining the filter) from database administration (creating the index).

   Java

   // Build an expression that indexes age only for adults in AU/CA/BW that are 18 or older
   Expression filterExp = Exp.build(
           Exp.cond(
               Exp.and(
                   Exp.ge(// Is the age 18 or older?
                           Exp.intBin("age"),
                           Exp.val(18)
                   ),
                   ListExp.getByValue( // Do they live in a target country?
                           ListReturnType.EXISTS,
                           Exp.stringBin("country"),
                           Exp.val(List.of("Australia", "Canada", "Botswana"))
                   )
               ),
               Exp.intBin("age"), // If true, return the age of the customer to be indexed
               Exp.unknown() // returns "unknown" to exclude the record from the index
           )
       );
   System.out.println(filterExp.getBase64());
   // Prints: lHuTEJMEk1ECo2FnZRKVfwEAkxYNk1EDp2NvdW50cnmSfpOqA0F1c3RyYWxpYacDQ2FuYWRhqQNCb3Rzd2FuYZNRAqNhZ2WRAA==

   Python

   # pip install aerospike
   import aerospike
   from aerospike_helpers import expressions as exp
   
   target_countries = ["Australia", "Canada", "Botswana"]
   
   # Build the "OR" condition for the country check
   country_is_target = exp.Or(
       *[exp.Eq(exp.StrBin("country"), c) for c in target_countries]
   )
   
   # Compile the final expression
   filter_exp = exp.Cond(
       # IF block: Both conditions must be true
       exp.And(
           exp.GE(exp.IntBin("age"), 18),
           country_is_target
       ),
       # Return the value of the 'age' bin for indexing
       exp.IntBin("age"),
       # ELSE Return "unknown" to exclude the record from the index
       exp.Unknown()
   ).compile()
   
   # Get Base64 for use with asadm `exp_base64`
   client = aerospike.client({"hosts": [("127.0.0.1", 3000)]}).connect()
   expr_b64 = client.get_expression_base64(filter_exp)
   print(expr_b64)
   # prints: lHuTEJMEk1ECo2FnZRKVfwEAkxYNk1EDp2NvdW50cnmSfpOqA0F1c3RyYWxpYacDQ2FuYWRhqQNCb3Rzd2FuYZNRAqNhZ2WRAA==
   client.close()

   C#

   // Build an expression that indexes age only for adults in AU/CA/BW
   Expression filterExp = Exp.Build(
       // The `Cond` expression is the top-level IF/THEN/ELSE
       Exp.Cond(
           // IF block: Defines the conditions for the index
           Exp.And(
               // Condition 1: Is the age 18 or older?
               Exp.GE(Exp.IntBin("age"), Exp.Val(18)),
               // Condition 2: Do they live in a target country?
               ListExp.GetByValue(
                   ListReturnType.EXISTS, // Check for existence
                   Exp.StringBin("country"),
                   Exp.Val(new[] { "Australia", "Canada", "Botswana" })
               )
           ),
           // If true, return the integer value from the 'age' bin.
           // This value is what gets indexed.
           Exp.IntBin("age"),
           // ELSE If false, return "unknown".
           // This excludes the record from the index.
           Exp.Unknown()
       )
   );
   
   // Print the Base64-encoded expression for use with asadm
   string exprBase64 = filterExp.GetBase64();
   Console.WriteLine("Expression Base64:");
   Console.WriteLine(exprBase64);
   // Prints: lHuTEJMEk1ECo2FnZRKVfwEAkxYNk1EDp2NvdW50cnmSfpOqA0F1c3RyYWxpYacDQ2FuYWRhqQNCb3Rzd2FuYZNRAqNhZ2WRAA==

   All of the client examples above will print the same Base64-encoded expression string to the console.

2. Create the expression index.

   Use the asadm command-line tool to create a numeric secondary index on the expression. Copy the full Base64 string from the previous step’s output and use it in the command below.

   asadm -e "enable; manage sindex create numeric cust_index ns test set cust_data exp_base64 lHuTEJMEk1ECo2FnZRKVfwEAkxYNk1EDp2NvdW50cnmSfpOqA0F1c3RyYWxpYacDQ2FuYWRhqQNCb3Rzd2FuYZNRAqNhZ2WRAA=="

3. Insert some test data.

   Java

   // Define the namespace and set for the records
   final String NAMESPACE = "test";
   final String SET = "cust_data";
   
   // A helper method to insert a customer record
   public void insert(int key, String name, int age, String country) {
       client.put(null,
               new Key(NAMESPACE, SET, key),
               new Bin("name", name),
               new Bin("age", age),
               new Bin("country", country));
   }
   
   // Insert sample customers to demonstrate which records get indexed
   public void insertData() {
       insert(1, "Tim", 312, "Australia");
       insert(2, "Bob", 47, "Canada");
       insert(3, "Jo", 15, "USA"); // not indexed
       insert(4, "Steven", 23, "Botswana");
       insert(5, "Susan", 32, "Canada");
       insert(6, "Jess", 17, "Botswana"); // not indexed
       insert(7, "Sam", 18, "USA"); // not indexed
       insert(8, "Alex", 47, "Canada");
       insert(9, "Pam", 56, "Australia");
       insert(10, "Vivek", 12, "India"); // not indexed
       insert(11, "Kiril", 22, "Sweden"); // not indexed
       insert(12, "Bill", 23, "UK"); // not indexed
   }

   Python

   import aerospike
   
   # Sample customer data
   customers = [
       (1, "Tim", 312, "Australia"),
       (2, "Bob", 47, "Canada"),
       (3, "Jo", 15, "USA"),           # not indexed
       (4, "Steven", 23, "Botswana"),
       (5, "Susan", 32, "Canada"),
       (6, "Jess", 17, "Botswana"),    # not indexed
       (7, "Sam", 18, "USA"),          # not indexed
       (8, "Alex", 47, "Canada"),
       (9, "Pam", 56, "Australia"),
       (10, "Vivek", 12, "India"),     # not indexed
       (11, "Kiril", 22, "Sweden"),    # not indexed
       (12, "Bill", 23, "UK")          # not indexed
   ]
   
   # Client configuration (replace with your actual configuration)
   config = {"hosts": [("127.0.0.1", 3000)]}
   client = aerospike.client(config).connect()
   
   # Inserts data
   for key, name, age, country in customers:
       client.put(
           ("test", "cust_data", key),
           {
               "name": name,
               "age": age,
               "country": country
           }
       )
   
   client.close()

   C#

   // Define the namespace and set for the records
   const string NAMESPACE = "test";
   const string SET = "cust_data";
   
   // A helper method to insert a customer record
   public void Insert(int key, string name, int age, string country)
   {
       client.Put(null,
           new Key(NAMESPACE, SET, key),
           new Bin("name", name),
           new Bin("age", age),
           new Bin("country", country));
   }
   
   // Insert sample customers to demonstrate which records get indexed
   public void InsertData()
   {
       Insert(1, "Tim", 312, "Australia");
       Insert(2, "Bob", 47, "Canada");
       Insert(3, "Jo", 15, "USA"); // not indexed
       Insert(4, "Steven", 23, "Botswana");
       Insert(5, "Susan", 32, "Canada");
       Insert(6, "Jess", 17, "Botswana"); // not indexed
       Insert(7, "Sam", 18, "USA"); // not indexed
       Insert(8, "Alex", 47, "Canada");
       Insert(9, "Pam", 56, "Australia");
       Insert(10, "Vivek", 12, "India"); // not indexed
       Insert(11, "Kiril", 22, "Sweden"); // not indexed
       Insert(12, "Bill", 23, "UK"); // not indexed
   }

4. Validate index entries.

   You should see 6 entries in the expression index (records 1, 2, 4, 5, 8, 9).

   # Confirm only matching records (6) were indexed
   asinfo -v 'sindex-stat:namespace=test;indexname=cust_index' -l
   
   entries=6
   ...

5. Query your Expression Index.

   Run a query to filter for ages 5 and above. Notice the results still only shows ages 18 and above.

   Java

   // Query via the expression index to fetch records for customers that are 5 and older
   Statement stmt = new Statement();
   stmt.setNamespace("test");
   stmt.setSetName("cust_data");
   // Use `rangeByIndex` to query the expression index
   stmt.setFilter(Filter.rangeByIndex("cust_index", 5, Integer.MAX_VALUE));
   
   // Execute the query and print the results
   try (RecordSet recordSet = client.query(null, stmt)) {
       while (recordSet.next()) {
           System.out.println(recordSet.getRecord());
       }
   }

   Python

   import sys
   import aerospike
   from aerospike import predicates as p
   
   # Connect to the database
   config = {"hosts": [("127.0.0.1", 3000)]}
   client = aerospike.client(config).connect()
   
   # Create a query object
   q = client.query("test", "cust_data")
   # Set the query to use the expression index with a range filter
   q.where_with_index_name("cust_index",
               p.range(None, aerospike.INDEX_TYPE_DEFAULT, 5, sys.maxsize))
   
   # Define a function to handle each record returned by the query
   def handle(record):
       key, meta, bins = record
       print(bins)
   
   # Execute the query and apply the handler to each record
   q.foreach(handle)
   client.close()

   C#

   // Query via the expression index to fetch records for customers that are 5 and older
   Statement stmt = new();
   stmt.SetNamespace("test");
   stmt.SetSetName("cust_data");
   // Use `RangeByIndex` to query the expression index
   stmt.SetFilter(Filter.RangeByIndex("cust_index", 5, int.MaxValue));
   
   // Execute the query and print the results
   try
   {
       using RecordSet recordSet = client.Query(null, stmt);
       while (recordSet.Next())
       {
           Console.WriteLine(recordSet.Record);
       }
   }

   Since we stored age, you can filter for records above 25 by simply modifying the query:

   Java

   // Query via the expression index to fetch records for customers that are 25 and older
   Statement stmt = new Statement();
   stmt.setNamespace("test");
   stmt.setSetName("cust_data");
   // Update the range to start at 25
   stmt.setFilter(Filter.rangeByIndex("cust_index", 25, Integer.MAX_VALUE));
   
   try (RecordSet recordSet = client.query(null, stmt)) {
       while (recordSet.next()) {
           System.out.println(recordSet.getRecord());
       }
   }

   Python

   import sys
   import aerospike
   from aerospike import predicates as p
   
   config = {"hosts": [("127.0.0.1", 3000)]}
   client = aerospike.client(config).connect()
   
   # Create a query and apply the new range
   q = client.query("test", "cust_data")
   q.where_with_index_name("cust_index",
               p.range(None, aerospike.INDEX_TYPE_DEFAULT, 25, sys.maxsize))
   
   def handle(record):
       key, meta, bins = record
       print(bins)
   
   q.foreach(handle)
   client.close()

   C#

   // Query via the expression index to fetch records for customers that are 25 and older
   Statement stmt = new();
   stmt.SetNamespace("test");
   stmt.SetSetName("cust_data");
   // Update the range to start at 25
   stmt.SetFilter(Filter.RangeByIndex("cust_index", 25, int.MaxValue));
   
   // Execute the query and print the results
   try
   {
       using RecordSet recordSet = client.Query(null, stmt);
       while (recordSet.Next())
       {
           Console.WriteLine(recordSet.Record);
       }
   }

6. That’s it. You’re done!

   Have questions or feedback? You can submit them or schedule a call with the product team using this form.