Background queries

Jump to the Code block for a combined complete example.

A client application can also issue an asynchronous background query to the database and apply a series of write transaction operations to each record. This is more efficient than a query to retrieve records followed by updates to them for cases where data needs to be manipulated on the client side. Transactional operations are typically more efficient than using Lua UDFs because the server doesn’t need to translate internal objects to another language. Many client libraries also provide an API to poll for the completion of a background query.

Refer to Background Queries for more information.

Setup

The following examples will use the setup and record structure below to illustrate bakground queries in an Aerospike database.

const Aerospike = require('aerospike');
const map = Aerospike.maps;
const exp = Aerospike.exp;

// Define host configuration
let config = {hosts: '127.0.0.1:3000'};

The record structure:

Occurred: Integer
Reported: Integer
Posted: Integer
Report: Map
{
    shape: List,
    summary: String,
    city: String,
    state: String,
    duration: String
}
Location: GeoJSON

Policies

Background queries can define policies to pass to the executed task.

The following example creates a policy that defines a filter expression looking for records that do not already have a numShapes bin.

// Create new query policy
let queryPolicy = new Aerospike.QueryPolicy({
    filterExpression: exp.not(exp.binExists('numShapes'))
});

Query

Just like basic queries background queries can be run on the primary index or, using a filter, on a secondary index.

Primary index

The following example creates a primary index background query using the Filter Expression defined in the policies example to find all records without a numShapes bin, then get the length of the shape key in the report map and write that value to a new bin called numShapes.

// Create the expression
let expr = exp.lists.size(
    exp.maps.getByKey(exp.binMap('report'), exp.str('shape'), exp.type.LIST, map.returnType.VALUE)
);

// Create the ops
let ops = [
    exp.operations.write('numShapes', expr, 0)
]

;(async () => {
    // Establishes a connection to the server
    let client = await Aerospike.connect(config);

    // Create query
    let query = client.query('sandbox', 'ufodata');

    // Execute the query
    let job = await query.operate(ops, queryPolicy);

    // Close the connection to the server
    client.close();
})();

Secondary index

The following example uses a secondary index created on the occurred bin.

asadm -e 'enable; manage sindex create numeric occurred_idx ns sandbox set ufodata bin occurred'

Then creates a secondary index background query using a Filter Expression checking for the existence of a posted bin on records that have an occurred bin value inclusively between 20210101 and 20211231, that updates the report map by adding a recent key with a value of true.

// Create new query policy
let queryPolicy = new Aerospike.QueryPolicy({
    filterExpression: (exp.binExists('posted'))
});

// Create the ops
let ops = [
    map.put('report', 'recent', true)
];

;(async () => {
    // Establishes a connection to the server
    let client = await Aerospike.connect(config);

    // Create query
    let query = client.query('sandbox', 'ufodata');

    // Create index filter
    query.where(Aerospike.filter.range('occurred', 20210101, 20211231));

    // Execute the query
    let job = await query.operate(ops, queryPolicy);

    // Close the connection to the server
    client.close();
})();

Tracking

Once a background query has been executed, a query status can be obtained to query nodes for task completion.

// Return the query status
let jobInfo = await job.info();

switch (jobInfo.status){
    case 0: console.info('Query not found');
            break;
    case 1: console.info('Query in progress,' + jobInfo.progressPct + '% complete');
            break;
    case 2: console.info('Query complete,' + jobInfo.recordsRead + ' records read');
            break;
}

Code block

Expand this section for a single code block to execute a background query

const Aerospike = require('aerospike');
const map = Aerospike.maps;
const exp = Aerospike.exp;

// Define host configuration
let config = {hosts: '127.0.0.1:3000'};

// Create new query policy
let queryPolicy = new Aerospike.QueryPolicy({
    filterExpression: exp.not(exp.binExists('numShapes'))
});

// Create the expression
let expr = exp.lists.size(
    exp.maps.getByKey(exp.binMap('report'), exp.str('shape'), exp.type.LIST, map.returnType.VALUE)
);

// Create the ops
let ops = [
    exp.operations.write('numShapes', expr, 0)
]

;(async () => {
    // Establishes a connection to the server
    let client = await Aerospike.connect(config);

    // Create query
    let query = client.query('sandbox', 'ufodata');

    // Execute the query
    let job = await query.operate(ops, queryPolicy);

    // Return the query status
    let jobInfo = await job.info();

    switch (jobInfo.status){
        case 0: console.info('Query not found');
                break;
        case 1: console.info('Query in progress,' + jobInfo.progressPct + '% complete');
                break;
        case 2: console.info('Query complete,' + jobInfo.recordsRead + ' records read');
                break;
    }

    // Close the connection to the server
    client.close();
})();