Primary index

Prior to Server 6.0, primary index (PI) queries were called scans and had policies defined through the scan policy. See Queries for more information.

Jump to the Code block for a combined complete example.

Basic PI queries have the following features:

• Filter records by set name.
• Filter records by filter expressions.
• Limit the number of records returned, useful for pagination.
• Return only record digests and metadata (generation and TTL).
• Return specified bins.

Setup

The following examples will use the setup and record structure below to illustrate primary index queries in an Aerospike database.

import (
    "fmt"
    "log"

    "github.com/aerospike/aerospike-client-go/v8"
)

// Establishes a connection to the server
client, err := aerospike.NewClient("127.0.0.1", 3000)
if err != nil {
  log.Fatal(err)
}
defer client.Close()

The record structure:

Occurred: Integer
Reported: Integer
Posted: Integer
Report: Map
{
    shape: List,
    summary: String,
    city: String,
    state: String,
    duration: String
}
Location: GeoJSON

Policies

See Basic Queries for query policy information.

Query a set

The following example queries the sandbox namespace and ufodata set name, while limiting the record set to only 20 records.

Note

If you are querying a smaller set, creating a set index may provide better performance.

Recordset.Results() yields a *Result on every send. That pointer is never nil, even when a node or partition fails mid-query. On a partial failure the client sends &Result{Err: err} with Record left nil. Check Err before reading Record.Bins; when Err is nil, Record is always set.

Caution

An error mid-query means the result set is incomplete. The records received before the error do not represent the full set of matching records, and the loop still ends normally, so code after it cannot tell a complete result from a truncated one. The examples below log the error and continue, which suits reporting and best-effort queries. Where the result must be complete, treat the error as fatal or propagate it to the caller instead of continuing.

// Create statement
stmt := aerospike.NewStatement("sandbox", "ufodata")

// Set max records to return
queryPolicy := aerospike.NewQueryPolicy()
queryPolicy.MaxRecords = 20

// Execute the query
recordSet, err := client.Query(queryPolicy, stmt)
if err != nil {
  log.Fatal(err)
}
defer recordSet.Close()

// Get the results
for res := range recordSet.Results() {
    if res.Err != nil {
        log.Printf("query error: %v", res.Err)
        continue
    }
    fmt.Printf("Record: %v\n", res.Record.Bins)
}

Query with a metadata filter

The following example queries the same namespace and set as the example above, but also adds a metadata Filter Expression that will only return records that are greater than 16 KiB.

A filter expression reduces what the query returns, not what it reads. The server still reads every record in the set on every node and evaluates the expression against each one, so the cluster-wide cost is the same as an unfiltered scan. This example and the data filter example below omit MaxRecords to keep the filter in focus; set it, or use a secondary index, to bound that cost in production.

Note

ExpRecordSize requires Database 7.0 or later. On earlier servers, use ExpDeviceSize or ExpMemorySize instead. Both of those expressions are superseded by ExpRecordSize on Database 7.0 and later, and each returns zero when the namespace is not backed by the matching storage type.

// Create query policy
queryPolicy := aerospike.NewQueryPolicy()
queryPolicy.FilterExpression = aerospike.ExpGreater(
    aerospike.ExpRecordSize(),
    aerospike.ExpIntVal(1024 * 16))

// Create statement
stmt := aerospike.NewStatement("sandbox", "ufodata")

// Execute the query
recordSet, err := client.Query(queryPolicy, stmt)
if err != nil {
  log.Fatal(err)
}
defer recordSet.Close()

// Get the results
for res := range recordSet.Results() {
    if res.Err != nil {
        log.Printf("query error: %v", res.Err)
        continue
    }
    fmt.Printf("Record: %v\n", res.Record.Bins)
}

Query with a data filter

The following example queries the same namespace and set as the example above, but also adds a data Filter Expression that returns only records where the occurred bin value is in the inclusive range 20210101 to 20211231.

Take a look at secondary index queries to see how this same query can be run more efficiently with an index.

// Create query policy
queryPolicy := aerospike.NewQueryPolicy()
queryPolicy.FilterExpression = aerospike.ExpLet(
    aerospike.ExpDef("bin", aerospike.ExpIntBin("occurred")),
    aerospike.ExpAnd(
        aerospike.ExpGreaterEq(aerospike.ExpVar("bin"), aerospike.ExpIntVal(20210101)),
        aerospike.ExpLessEq(aerospike.ExpVar("bin"), aerospike.ExpIntVal(20211231))))

// Create statement
stmt := aerospike.NewStatement("sandbox", "ufodata")

// Execute the query
recordSet, err := client.Query(queryPolicy, stmt)
if err != nil {
  log.Fatal(err)
}
defer recordSet.Close()

// Get the results
for res := range recordSet.Results() {
    if res.Err != nil {
        log.Printf("query error: %v", res.Err)
        continue
    }
    fmt.Printf("Record: %v\n", res.Record.Bins)
}

Pagination

Pagination uses a combination of a partition filter and a defined maximum records to return query results. The partition filter maintains a cursor identifying the end of the current page and the beginning of the next. Moving to the next page of results is as simple as executing the query again, with the previously defined partition filter.

Defining a maximum number of records per page to return guarantees that no page will contain more than the maximum number, but some pages may contain fewer than the maximum number. Also, if you run the same paginated query multiple times, the number of results per page may differ, depending on the order in which they are delivered by the nodes in the cluster.

The following example executes a query with an Expression Filter identifying records with more than 3 shape items in the report map, returning 10 records per page. The partition filter is set to query all 4096 partitions in the database.

Call QueryPartitions inside the IsDone loop so the last drained page does not trigger an extra round trip. Close each page’s Recordset before requesting the next one.

// Create query policy
queryPolicy := aerospike.NewQueryPolicy()
queryPolicy.FilterExpression = aerospike.ExpGreater(
    aerospike.ExpListSize(
        aerospike.ExpMapGetByKey(aerospike.MapReturnType.VALUE, aerospike.ExpTypeLIST, aerospike.ExpStringVal("shape"), aerospike.ExpMapBin("report"))),
    aerospike.ExpIntVal(3))

// Set max records to return
queryPolicy.MaxRecords = 10

// Create statement
stmt := aerospike.NewStatement("sandbox", "ufodata")

// Create the partition filter
partitionFilter := aerospike.NewPartitionFilterAll()

page := 0
for !partitionFilter.IsDone() {
    recordSet, err := client.QueryPartitions(queryPolicy, stmt, partitionFilter)
    if err != nil {
        log.Fatal(err)
    }

    count := 0
    for res := range recordSet.Results() {
        if res.Err != nil {
            log.Printf("query error: %v", res.Err)
            continue
        }
        fmt.Printf("Record: %v\n", res.Record.Bins)
        count++
    }
    recordSet.Close()

    page++
    fmt.Printf("\nPage %d | %d records\n\n", page, count)
}

Code block

Expand this section for a single code block to execute a basic PI query

import (
    "fmt"
    "log"

    "github.com/aerospike/aerospike-client-go/v8"
)

func main() {
    // Establishes a connection to the server
    client, err := aerospike.NewClient("127.0.0.1", 3000)
    if err != nil {
        log.Fatal(err)
    }
    defer client.Close()

    // Create query policy
    queryPolicy := aerospike.NewQueryPolicy()
    queryPolicy.FilterExpression = aerospike.ExpLet(
        aerospike.ExpDef("bin", aerospike.ExpIntBin("occurred")),
        aerospike.ExpAnd(
            aerospike.ExpGreaterEq(aerospike.ExpVar("bin"), aerospike.ExpIntVal(20210101)),
            aerospike.ExpLessEq(aerospike.ExpVar("bin"), aerospike.ExpIntVal(20211231))))

    // Create statement
    stmt := aerospike.NewStatement("sandbox", "ufodata")

    // Execute the query
    recordSet, err := client.Query(queryPolicy, stmt)
    if err != nil {
        log.Fatal(err)
    }
    defer recordSet.Close()

    // Get the results
    for res := range recordSet.Results() {
        if res.Err != nil {
            log.Printf("query error: %v", res.Err)
            continue
        }
        fmt.Printf("Record: %v\n", res.Record.Bins)
    }
}