Background queries

Jump to the Code block for a combined complete example.

A client application can also issue an asynchronous background query to the database and apply a series of write transaction operations to each record. This is more efficient than a query to retrieve records followed by updates to them for cases where data needs to be manipulated on the client side. Transactional operations are typically more efficient than using Lua UDFs because the server doesn’t need to translate internal objects to another language. Many client libraries also provide an API to poll for the completion of a background query.

Refer to Background Queries for more information.

Setup

The following examples will use the setup and record structure below to illustrate background queries in an Aerospike database.

import com.aerospike.client.AerospikeClient;
import com.aerospike.client.Value;
import com.aerospike.client.cdt.MapOperation;
import com.aerospike.client.cdt.MapPolicy;
import com.aerospike.client.cdt.MapReturnType;
import com.aerospike.client.exp.Exp;
import com.aerospike.client.exp.Expression;
import com.aerospike.client.exp.ExpWriteFlags;
import com.aerospike.client.exp.ExpOperation;
import com.aerospike.client.exp.ListExp;
import com.aerospike.client.exp.MapExp;
import com.aerospike.client.policy.WritePolicy;
import com.aerospike.client.query.Filter;
import com.aerospike.client.query.Statement;
import com.aerospike.client.task.ExecuteTask;

// Establishes a connection to the server
AerospikeClient client = new AerospikeClient("127.0.0.1", 3000);

The record structure:

Occurred: Integer
Reported: Integer
Posted: Integer
Report: Map
{
    shape: List,
    summary: String,
    city: String,
    state: String,
    duration: String
}
Location: GeoJSON

Policies

Background queries can define policies to pass to the executed task.

The following example creates a policy that defines a filter expression looking for records that do not already have a numShapes bin.

// Create new write policy
WritePolicy writePolicy = new WritePolicy();
writePolicy.filterExp = Exp.build(
    Exp.not(
        Exp.binExists("numShapes")
    )
);

Query

Just like basic queries background queries can be run on the primary index or, using a filter, on a secondary index.

Primary index

The following example creates a primary index background query using the Filter Expression defined in the policies example to find all records without a numShapes bin, then get the length of the shape key in the report map and write that value to a new bin called numShapes.

// Create statement
Statement stmt = new Statement();

// Set namspace and set name
stmt.setNamespace("sandbox");
stmt.setSetName("ufodata");

Expression exp = Exp.build(
    ListExp.size(
        MapExp.getByKey(MapReturnType.VALUE, Exp.Type.LIST, Exp.val("shape"), Exp.mapBin("report"))
    )
);

ExecuteTask task = client.execute(writePolicy, stmt,
    ExpOperation.write("numShapes", exp, ExpWriteFlags.DEFAULT)
);

Secondary index

The following example uses a secondary index created on the occurred bin.

asadm -e 'enable; manage sindex create numeric occurred_idx ns sandbox set ufodata bin occurred'

Then creates a secondary index background query using a Filter Expression checking for the existence of a posted bin on records that have an occurred bin value inclusively between 20210101 and 20211231, that updates the report map by adding a recent key with a value of true.

// Create new write policy
WritePolicy writePolicy = new WritePolicy();
writePolicy.filterExp = Exp.build(
    Exp.binExists("posted")
);

// Create statement
Statement stmt = new Statement();

// Set namspace and set name
stmt.setNamespace("sandbox");
stmt.setSetName("ufodata");

// Set index filter
stmt.setFilter(Filter.range("occurred", 20210101, 20211231));

ExecuteTask task = client.execute(writePolicy, stmt,
    MapOperation.put(MapPolicy.Default, "report", Value.get("recent"), Value.get(true))
);

Tracking

Once a background query has been executed, a query status can be obtained to query nodes for task completion.

// Return the query status
int status = task.queryStatus();

String queryStatus;
switch (status){
    case 0: queryStatus = "not found";
            break;
    case 1: queryStatus = "in progress";
            break;
    case 2: queryStatus = "complete";
            break;
}
System.out.format("Query %s.", queryStatus);

// Close the connection to the server
client.close();

Code block

Expand this section for a single code block to execute a background query

import com.aerospike.client.AerospikeClient;
import com.aerospike.client.Value;
import com.aerospike.client.cdt.MapOperation;
import com.aerospike.client.cdt.MapPolicy;
import com.aerospike.client.cdt.MapReturnType;
import com.aerospike.client.exp.Exp;
import com.aerospike.client.exp.Expression;
import com.aerospike.client.exp.ExpWriteFlags;
import com.aerospike.client.exp.ExpOperation;
import com.aerospike.client.exp.ListExp;
import com.aerospike.client.exp.MapExp;
import com.aerospike.client.policy.WritePolicy;
import com.aerospike.client.query.Filter;
import com.aerospike.client.query.Statement;
import com.aerospike.client.task.ExecuteTask;

// Establishes a connection to the server
AerospikeClient client = new AerospikeClient("127.0.0.1", 3000);

// Create new write policy
WritePolicy writePolicy = new WritePolicy();
writePolicy.filterExp = Exp.build(
    Exp.not(
        Exp.binExists("numShapes")
    )
);

// Create statement
Statement stmt = new Statement();

// Set namspace and set name
stmt.setNamespace("sandbox");
stmt.setSetName("ufodata");

Expression exp = Exp.build(
    ListExp.size(
        MapExp.getByKey(MapReturnType.VALUE, Exp.Type.LIST, Exp.val("shape"), Exp.mapBin("report"))
    )
);

ExecuteTask task = client.execute(writePolicy, stmt,
    ExpOperation.write("numShapes", exp, ExpWriteFlags.DEFAULT)
);

// Return the query status
int status = task.queryStatus();

String queryStatus;
switch (status){
    case 0: queryStatus = "not found";
            break;
    case 1: queryStatus = "in progress";
            break;
    case 2: queryStatus = "complete";
            break;
}
System.out.format("Query %s.", queryStatus);

// Close the connection to the server
client.close();