Secondary index

Jump to the Code block for a combined complete example.

Basic SI queries can employ the following index filters:

Equality comparison against string or numeric indexes.
Range comparison against numeric indexes. Range result sets are inclusive of lower and upper limits.
Point-In-Region or Region-Contain-Point comparisons against geo indexes.

Filter Expressions can also be used with secondary index queries.

Policies

See Basic Queries for query policy information.

Setup

The following examples will use the setup and record structure below to illustrate secondary index queries in an Aerospike database.

import aerospike
from aerospike_helpers import expressions as exp
from aerospike_helpers.operations import operations, map_operations
from aerospike import predicates as p
from aerospike import GeoJSON

# Define host configuration
config = {
    'hosts': [ ('127.0.0.1', 3000) ]
}
# Establishes a connection to the server
client = aerospike.client(config).connect()

The record structure:

Occurred: Integer
Reported: Integer
Posted: Integer
Report: Map
{
    shape: List,
    summary: String,
    city: String,
    state: String,
    duration: String
}
Location: GeoJSON

Create an index

The following command uses the Aerospike Admin (asadm) to create an integer index on the sandbox namespace, ufodata set, and occurred bin. This is the recommended way to create a secondary index.

asadm -e 'enable; manage sindex create numeric occurred_idx ns sandbox set ufodata bin occurred'

The Aersopike Client API can be used to create a secondary index as well. The following example creates the same index as the example above.

# Create index
client.index_integer_create(
    'sandbox', # namespace
    'ufodata', # set name
    'occurred', # bin name
    'occurred_idx', # index name
    # optional policy
)

In this example, the IndexType is NUMERIC. Aerospike supports index types NUMERIC,STRING, GEO2DSPHERE, and BLOB as of server 7.0.

Remove an index

The following command uses the Aerospike Admin (asadm) to remove the index created above.

asadm -e 'enable; manage sindex delete occurred_idx ns sandbox set ufodata'

The Aersopike Client API can be used to remove a secondary index as well. The following example removes the same index as the example above.

# Remove index
client.index_remove(
    'sandbox', # namespace
    'occurred_idx', # index name
    # optional policy
)

Query an index

The following example queries the sandbox namespace and ufodata set name, with an inclusive range filter on the occurred bin, returning records with a bin value between 20210101 and 20211231.

# Create the query
query = client.query('sandbox', 'ufodata')

# Set index filter
query.where(p.between('occurred', 20210101, 20211231))

# Create callback function
def record_set(record):
    (key, meta, bins) = record
    # Do something
    print('Key: {0} | Record: {1}'.format(key[2], bins))

# Execute the query
query.foreach(record_set)

# Close the connection to the server
client.close()

Query an index with a Filter Expression

The following example will use the geo string defined in the expandable section below.

View the language specific data creation

# Create geo region
region = GeoJSON({
    'type': 'Polygon',
    'coordinates': [
        [
            [-109.061279296875, 36.97622678464096],
            [-102.01904296874999, 36.97622678464096],
            [-102.01904296874999, 41.0130657870063],
            [-109.061279296875, 41.0130657870063],
            [-109.061279296875, 36.97622678464096]
        ]
    ]
})

This example queries the same namespace and set name, while using the same index filter as the example above, but adds a Filter Expression to the query policy to only return records with a location bin value within the geo region specified above.

# Build the expression
expr = exp.CmpGeo(exp.GeoBin('location'), region).compile()

# Create the policy
query_policy = {'expressions': expr}

# Create the query
query = client.query('sandbox', 'ufodata')

# Set index filter
query.where(p.between('occurred', 20210101, 20211231))

# Create callback function
def record_set(record):
    (key, meta, bins) = record
    # Do something
    print('Key: {0} | Record: {1}'.format(key[2], bins))

# Execute the query
query.foreach(record_set, policy=query_policy)

# Close the connection to the server
client.close()

Query an index then create a batch operation on the returned keys

Currently, Transaction and CDT operations are not available for basic queries. The following example shows how a basic query and a batch operation can be combined to use transaction operations on the results.

This example queries the same namespace and set name, while using the same index filter, geo region, and Filter Expression as above, but only returns record metadata, which is then used to create a batch operation to return only the city and state from the report map.

# Build the expression
expr = exp.CmpGeo(exp.GeoBin('location'), region).compile()

# Create the policy
query_policy = {'expressions': expr}

# Create the options
query_options = {'nobins': True}

# Create the query
query = client.query('sandbox', 'ufodata')

# Set index filter
query.where(p.between('occurred', 20210101, 20211231))

# Create callback function
keys = []
def record_set(record):
    (key, meta, bins) = record
    keys.append(key)

# Execute the query
query.foreach(record_set, policy=query_policy, options=query_options)

# Create map key list
mapKeys = ['city', 'state']

# Create operations
ops = [
    map_operations.map_get_by_key_list('report', mapKeys, aerospike.MAP_RETURN_VALUE),
    # This is the only way a read will work right now
    operations.touch()
]

# Get 'city' and 'state' from report map for each record
batchRecords = client.batch_operate(keys, ops)

# Access the records
for batchRecord in batchRecords.batch_records:
    record = batchRecord.record
    # Do something
    print('Record: ', record[2])

# Close the connection to the server
client.close()

Pagination

See Pagination for more information.

Code block

Expand this section for a single code block to execute a basic SI query

import aerospike
from aerospike_helpers import expressions as exp
from aerospike_helpers.operations import operations, map_operations
from aerospike import predicates as p
from aerospike import GeoJSON

# Define host configuration
config = {
    'hosts': [ ('127.0.0.1', 3000) ]
}
# Establishes a connection to the server
client = aerospike.client(config).connect()

# Create geo region
region = GeoJSON({
    'type': 'Polygon',
    'coordinates': [
        [
            [-109.061279296875, 36.97622678464096],
            [-102.01904296874999, 36.97622678464096],
            [-102.01904296874999, 41.0130657870063],
            [-109.061279296875, 41.0130657870063],
            [-109.061279296875, 36.97622678464096]
        ]
    ]
})

# Build the expression
expr = exp.CmpGeo(exp.GeoBin('location'), region).compile()

# Create the policy
query_policy = {'expressions': expr}

# Create the query
query = client.query('sandbox', 'ufodata')

# Set index filter
query.where(p.between('occurred', 20210101, 20211231))

# Create callback function
def record_set(record):
    (key, meta, bins) = record
    # Do something
    print('Key: {0} | Record: {1}'.format(key[2], bins))

# Execute the query
query.foreach(record_set, policy=query_policy)

# Close the connection to the server
client.close()