Background queries

Jump to the Code block for a combined complete example.

A client application can also issue an asynchronous background query to the database and apply a series of write transaction operations to each record. This is more efficient than a query to retrieve records followed by updates to them for cases where data needs to be manipulated on the client side. Transactional operations are typically more efficient than using Lua UDFs because the server doesn’t need to translate internal objects to another language. Many client libraries also provide an API to poll for the completion of a background query.

Refer to Background Queries for more information.

Setup

The following examples will use the setup and record structure below to illustrate background queries in an Aerospike database.

import (
    "fmt"
    "github.com/aerospike/aerospike-client-go/v6"
)

// Establishes a connection to the server
client, err := aerospike.NewClient("127.0.0.1", 3000)
if err != nil {
  log.Fatal(err)
}
defer client.Close()

The record structure:

Occurred: Integer
Reported: Integer
Posted: Integer
Report: Map
{
    shape: List,
    summary: String,
    city: String,
    state: String,
    duration: String
}
Location: GeoJSON

Policies

Background queries can define policies to pass to the executed task.

The following example creates a policy that defines a filter expression looking for records that do not already have a numShapes bin.

// Create new write policy
writePolicy := aerospike.NewWritePolicy(0,0)
writePolicy.FilterExpression = aerospike.ExpNot(
    aerospike.ExpBinExists("numShapes"))

Query

Just like basic queries background queries can be run on the primary index or, using a filter, on a secondary index.

Primary index

The following example creates a primary index background query using the Filter Expression defined in the policies example to find all records without a numShapes bin, then get the length of the shape key in the report map and write that value to a new bin called numShapes.

// Create statement
stmt := aerospike.NewStatement("sandbox", "ufodata")

exp := aerospike.ExpListSize(
    aerospike.ExpMapGetByKey(aerospike.MapReturnType.VALUE, aerospike.ExpTypeLIST, aerospike.ExpStringVal("shape"), aerospike.ExpMapBin("report")))

task, err := client.QueryExecute(nil, writePolicy, stmt, aerospike.ExpWriteOp("numShapes", exp, aerospike.ExpWriteFlagDefault))
if err != nil {
  log.Fatal(err)
}

Secondary index

The following example uses a secondary index created on the occurred bin.

asadm -e 'enable; manage sindex create numeric occurred_idx ns sandbox set ufodata bin occurred'

Then creates a secondary index background query using a Filter Expression checking for the existence of a posted bin on records that have an occurred bin value inclusively between 20210101 and 20211231, that updates the report map by adding a recent key with a value of true.

// Create new write policy
writePolicy := aerospike.NewWritePolicy(0,0)
writePolicy.FilterExpression = aerospike.ExpBinExists("posted")

// Create statement
stmt := aerospike.NewStatement("sandbox", "ufodata")

// Set index filter
stmt.SetFilter(aerospike.NewRangeFilter("occurred", 20210101, 20211231))

task, err := client.QueryExecute(nil, writePolicy, stmt,
    aerospike.MapPutOp(aerospike.DefaultMapPolicy(), "report", aerospike.NewValue("recent"), aerospike.NewValue(true)))
if err != nil {
  log.Fatal(err)
}

Tracking

Once a background query has been executed, a query status can be obtained to query nodes for task completion.

// Return the query status
status, err := task.IsDone();
if err != nil {
    // Handle error
} else {
    if status {
        fmt.Printf("Query complete")
    } else {
        fmt.Printf("Query in progress")
    }
}

Code block

Expand this section for a single code block to execute a background query

import (
    "fmt"
    "github.com/aerospike/aerospike-client-go/v6"
)

func main() {
    // Establishes a connection to the server
    client, err := aerospike.NewClient("127.0.0.1", 3000)
    if err != nil {
        log.Fatal(err)
    }
    defer client.Close()

    writePolicy := aerospike.NewWritePolicy(0,0)
    writePolicy.FilterExpression = aerospike.ExpNot(
        aerospike.ExpBinExists("numShapes"))

    // Create statement
    stmt := aerospike.NewStatement("sandbox", "ufodata")

    exp := aerospike.ExpListSize(
        aerospike.ExpMapGetByKey(aerospike.MapReturnType.VALUE, aerospike.ExpTypeLIST, aerospike.ExpStringVal("shape"), aerospike.ExpMapBin("report")))

    task, err := client.QueryExecute(nil, writePolicy, stmt, aerospike.ExpWriteOp("numShapes", exp, aerospike.ExpWriteFlagDefault))
    if err != nil {
        log.Fatal(err)
    }

    status, err := task.IsDone();
    if err != nil {
        // Handle error
    } else {
        if status {
            fmt.Printf("Query complete")
        } else {
            fmt.Printf("Query in progress")
        }
    }
}