Query Operations

Description

The Aerospike Query Operations provide the ability to query data in the Aerospike database. The queries can optionally be performed on secondary indexes, which have been created in the database.

Usage

Before you can execute a query, you first need to build a query using as_query. See as_query for details on building queries.

Once you have a query defined, then you can execute the query. When aerospike_query_foreach() is executed, it will process the results and create records on the stack. Because the records are on the stack, they will only be available within the context of the callback function.

Walk-through

First, we define a query using as_query. The query will be for the "test" namespace and "demo" set. We will add a where predicate on "bin2", on which we have already created a secondary index.

as_query query;
as_query_init(&query, "test", "demo");
 
as_query_where_init(&query, 1);
as_query_where(&query, "bin2", as_integer_equals(100));

Now that we have a query defined, we want to execute it using aerospike_query_foreach().

if (aerospike_query_foreach(&as, &err, NULL, &query, callback, NULL) != AEROSPIKE_OK) {
    fprintf(stderr, "error(%d) %s at [%s:%d]", err.code, err.message, err.file, err.line);
}

The callback provided to the function above is implemented as:

bool callback(const as_val* val, void* udata)
{
    if (!val) {
        return false; // Query complete.
    }
 
    as_record* rec = as_record_fromval(val);
    // Process record
    // Do not call as_record_destroy() because the calling function will do that for you.
    return true;
}

When you are finished with the query, you should destroy the resources allocated to it.

as_query_destroy(&query);

Collaboration diagram for Query Operations:

Data Structures
struct	as_query

Typedefs
typedef bool(*	aerospike_query_foreach_callback) (const as_val *val, void *udata)
typedef bool(*	as_async_query_record_listener) (as_error *err, as_record *record, void *udata, as_event_loop *event_loop)

Functions
AS_EXTERN as_status	aerospike_query_async (aerospike *as, as_error *err, const as_policy_query *policy, as_query *query, as_async_query_record_listener listener, void *udata, as_event_loop *event_loop)
AS_EXTERN as_status	aerospike_query_background (aerospike *as, as_error *err, const as_policy_write *policy, const as_query *query, uint64_t *query_id)
AS_EXTERN as_status	aerospike_query_foreach (aerospike *as, as_error *err, const as_policy_query *policy, as_query *query, aerospike_query_foreach_callback callback, void *udata)
static as_status	aerospike_query_info (aerospike *as, as_error *err, const as_policy_info *policy, const as_query *query, uint64_t query_id, as_job_info *info)
AS_EXTERN as_status	aerospike_query_partitions (aerospike *as, as_error *err, const as_policy_query *policy, as_query *query, as_partition_filter *pf, aerospike_query_foreach_callback callback, void *udata)
AS_EXTERN as_status	aerospike_query_partitions_async (aerospike *as, as_error *err, const as_policy_query *policy, as_query *query, as_partition_filter *pf, as_async_query_record_listener listener, void *udata, as_event_loop *event_loop)
static as_status	aerospike_query_wait (aerospike *as, as_error *err, const as_policy_info *policy, const as_query *query, uint64_t query_id, uint32_t interval_ms)
AS_EXTERN bool	as_query_apply (as_query *query, const char *module, const char *function, const as_list *arglist)
AS_EXTERN bool	as_query_compare (as_query *q1, as_query *q2)
AS_EXTERN void	as_query_destroy (as_query *query)
AS_EXTERN bool	as_query_from_bytes (as_query *query, const uint8_t *bytes, uint32_t bytes_size)
AS_EXTERN as_query *	as_query_from_bytes_new (const uint8_t *bytes, uint32_t bytes_size)
AS_EXTERN as_query *	as_query_init (as_query *query, const char *ns, const char *set)
static bool	as_query_is_done (as_query *query)
AS_EXTERN as_query *	as_query_new (const char *ns, const char *set)
AS_EXTERN bool	as_query_select (as_query *query, const char *bin)
AS_EXTERN bool	as_query_select_init (as_query *query, uint16_t n)
static void	as_query_set_paginate (as_query *query, bool paginate)
static void	as_query_set_partitions (as_query *query, as_partitions_status *parts_all)
AS_EXTERN bool	as_query_to_bytes (const as_query *query, uint8_t **bytes, uint32_t *bytes_size)
AS_EXTERN bool	as_query_where (as_query *query, const char *bin, as_predicate_type type, as_index_type itype, as_index_datatype dtype,...)
AS_EXTERN bool	as_query_where_init (as_query *query, uint16_t n)
AS_EXTERN bool	as_query_where_with_ctx (as_query *query, const char *bin, struct as_cdt_ctx *ctx, as_predicate_type type, as_index_type itype, as_index_datatype dtype,...)

Typedef Documentation

aerospike_query_foreach_callback

typedef bool(* aerospike_query_foreach_callback) (const as_val *val, void *udata)

This callback will be called for each value or record returned from a query. Multiple threads will likely be calling this callback in parallel. Therefore, your callback implementation should be thread safe.

A regular foreground query always returns as_record instances:

bool callback(const as_val* val, void* udata)
{
    if (!val) {
        return false; // Query complete.
    }
 
    as_record* rec = as_record_fromval(val);
    // Process record
    // Do not call as_record_destroy() because the calling function will do that for you.
    return true;
}

An aggregation query using a UDF returns as_val instances. The as_val type depends on what the UDF returns:

bool callback(const as_val* val, void* udata)
{
    if (!val) {
        return false; // Query complete.
    }
 
    // Ensure UDF returned val is the expected type:
    as_integer* i = as_integer_fromval(val);
    
    if (!i) {
        return false;
    }
 
    // Process integer
    return true;
}

Parameters

val	The value received from the query.
udata	User-data provided to the calling function.

Returns

true to continue to the next value. Otherwise, iteration will end.

Definition at line 150 of file aerospike_query.h.

as_async_query_record_listener

typedef bool(* as_async_query_record_listener) (as_error *err, as_record *record, void *udata, as_event_loop *event_loop)

Asynchronous query user callback. This function is called for each record returned. This function is also called once when the query completes or an error has occurred.

Parameters

err	This error structure is only populated on command failure. NULL on success.
record	Returned record. The record will be NULL on final query completion or query error.
udata	User data that is forwarded from asynchronous command function.
event_loop	Event loop that this command was executed on. Use this event loop when running nested asynchronous commands when single threaded behavior is desired for the group of commands.

Returns

true to continue to the next value. Otherwise, the query will end.

Definition at line 167 of file aerospike_query.h.

Function Documentation

aerospike_query_async()

AS_EXTERN as_status aerospike_query_async	(	aerospike *	as,
		as_error *	err,
		const as_policy_query *	policy,
		as_query *	query,
		as_async_query_record_listener	listener,
		void *	udata,
		as_event_loop *	event_loop )

Asynchronously execute a query and call the listener function for each result item. Standard queries are supported, but aggregation queries are not supported in async mode.

bool my_listener(as_error* err, as_record* record, void* udata, as_event_loop* event_loop)
{
    if (err) {
        printf("Query failed: %d %s\n", err->code, err->message);
        return false;
    }
 
    if (! record) {
        printf("Query ended\n");
        return false;
    }
 
    // Process record
    // Do not call as_record_destroy() because the calling function will do that for you.
    return true;
}
 
as_query query;
as_query_init(&query, "test", "demo");
as_query_select(&query, "bin1");
as_query_where(&query, "bin2", as_integer_equals(100));
 
if (aerospike_query_async(&as, &err, NULL, &query, my_listener, NULL, NULL) != AEROSPIKE_OK) {
    fprintf(stderr, "error(%d) %s at [%s:%d]", err.code, err.message, err.file, err.line);
}
as_query_destroy(&query);

Parameters

as	Aerospike cluster instance.
err	Error detail structure that is populated if an error occurs.
policy	Query policy configuration parameters, pass in NULL for default.
query	Query definition.
listener	The function to be called for each returned value.
udata	User-data to be passed to the callback.
event_loop	Event loop assigned to run this command. If NULL, an event loop will be chosen by round-robin.

Returns

AEROSPIKE_OK if async query succesfully queued. Otherwise an error.

aerospike_query_background()

AS_EXTERN as_status aerospike_query_background	(	aerospike *	as,
		as_error *	err,
		const as_policy_write *	policy,
		const as_query *	query,
		uint64_t *	query_id )

Apply user defined function on records that match the query filter. Records are not returned to the client. This asynchronous server call will return before the command is complete. The user can optionally wait for command completion.

as_query query;
as_query_init(&query, "test", "demo");
as_query_select(&query, "bin1");
as_query_where(&query, "bin2", as_integer_equals(100));
as_query_apply(&query, "my_lua.lua", "my_lua_function", NULL);
uint64_t query_id = 0;
 
if (aerospike_query_background(&as, &err, NULL, &query, &query_id) == AEROSPIKE_OK) {
    aerospike_query_wait(as, &err, NULL, &query, query_id, 0);
}
else {
    fprintf(stderr, "error(%d) %s at [%s:%d]", err.code, err.message, err.file, err.line);
}
as_query_destroy(&query);

Parameters

as	Aerospike cluster instance.
err	Error detail structure that is populated if an error occurs.
policy	Write configuration parameters, pass in NULL for default.
query	The query to execute against the cluster.
query_id	The id for the query job, which can be used for obtaining query status.

Returns

AEROSPIKE_OK on success, otherwise an error.

aerospike_query_foreach()

AS_EXTERN as_status aerospike_query_foreach	(	aerospike *	as,
		as_error *	err,
		const as_policy_query *	policy,
		as_query *	query,
		aerospike_query_foreach_callback	callback,
		void *	udata )

Execute a query and call the callback function for each result item. Multiple threads will likely be calling the callback in parallel. Therefore, your callback implementation should be thread safe.

bool callback(const as_val* val, void* udata)
{
    if (!val) {
        return false; // Query complete.
    }
 
    as_record* rec = as_record_fromval(val);
    // Process record
    // Do not call as_record_destroy() because the calling function will do that for you.
    return true;
}
 
as_query query;
as_query_init(&query, "test", "demo");
as_query_select(&query, "bin1");
as_query_where(&query, "bin2", as_integer_equals(100));
 
if (aerospike_query_foreach(&as, &err, NULL, &query, callback, NULL) != AEROSPIKE_OK) {
    fprintf(stderr, "error(%d) %s at [%s:%d]", err.code, err.message, err.file, err.line);
}
as_query_destroy(&query);

Parameters

as	Aerospike cluster instance.
err	Error detail structure that is populated if an error occurs.
policy	Query policy configuration parameters, pass in NULL for default.
query	Query definition.
callback	Query callback function called for each result value.
udata	User-data to be passed to the callback.

Returns

AEROSPIKE_OK on success, otherwise an error.

aerospike_query_info()

static as_status aerospike_query_info ( aerospike * as, as_error * err, const as_policy_info * policy, const as_query * query, uint64_t query_id, as_job_info * info )

inlinestatic

Check the progress of a background query running on the database.

Parameters

as	Aerospike cluster instance.
err	Error detail structure that is populated if an error occurs.
policy	Info configuration parameters, pass in NULL for default.
query	The query that was executed against the cluster.
query_id	The id for the query job, which can be used for obtaining query status.
info	Information about this background query, to be populated by this operation.

Returns

AEROSPIKE_OK on success, otherwise an error.

Definition at line 452 of file aerospike_query.h.

References aerospike_job_info().

aerospike_query_partitions()

AS_EXTERN as_status aerospike_query_partitions	(	aerospike *	as,
		as_error *	err,
		const as_policy_query *	policy,
		as_query *	query,
		as_partition_filter *	pf,
		aerospike_query_foreach_callback	callback,
		void *	udata )

Query records with a partition filter. Multiple threads will likely be calling the callback in parallel. Therefore, your callback implementation should be thread safe. Requires server version 6.0+.

bool callback(const as_val* val, void* udata)
{
    if (!val) {
        return false; // Query complete.
    }
 
    as_record* rec = as_record_fromval(val);
    // Process record
    // Do not call as_record_destroy() because the calling function will do that for you.
    return true;
}
 
as_query query;
as_query_init(&query, "test", "demo");
as_query_select(&query, "bin1");
as_query_where(&query, "bin2", as_integer_equals(100));
 
as_partition_filter pf;
as_partition_filter_set_range(&pf, 0, 1024);
 
if (aerospike_query_partitions(&as, &err, NULL, &query, &pf, callback, NULL) != AEROSPIKE_OK) {
    fprintf(stderr, "error(%d) %s at [%s:%d]", err.code, err.message, err.file, err.line);
}
as_query_destroy(&query);

Parameters

as	Aerospike cluster instance.
err	Error detail structure that is populated if an error occurs.
policy	Query policy configuration parameters, pass in NULL for default.
query	Query definition.
pf	Partition filter.
callback	Query callback function called for each result value.
udata	User-data to be passed to the callback.

Returns

AEROSPIKE_OK on success. Otherwise an error occurred.

aerospike_query_partitions_async()

AS_EXTERN as_status aerospike_query_partitions_async	(	aerospike *	as,
		as_error *	err,
		const as_policy_query *	policy,
		as_query *	query,
		as_partition_filter *	pf,
		as_async_query_record_listener	listener,
		void *	udata,
		as_event_loop *	event_loop )

Asynchronously query records with a partition filter. Standard queries are supported, but aggregation queries are not supported in async mode. Requires server version 6.0+.

bool my_listener(as_error* err, as_record* record, void* udata, as_event_loop* event_loop)
{
    if (err) {
        printf("Query failed: %d %s\n", err->code, err->message);
        return false;
    }
 
    if (! record) {
        printf("Query ended\n");
        return false;
    }
 
    // Process record
    // Do not call as_record_destroy() because the calling function will do that for you.
    return true;
}
 
as_query query;
as_query_init(&query, "test", "demo");
as_query_select(&query, "bin1");
as_query_where(&query, "bin2", as_integer_equals(100));
 
as_partition_filter pf;
as_partition_filter_set_range(&pf, 0, 1024);
 
if (aerospike_query_partitions_async(&as, &err, NULL, &query, &pf, my_listener, NULL, NULL) 
    != AEROSPIKE_OK) {
    fprintf(stderr, "error(%d) %s at [%s:%d]", err.code, err.message, err.file, err.line);
}
as_query_destroy(&query);

Parameters

as	Aerospike cluster instance.
err	Error detail structure that is populated if an error occurs.
policy	Query policy configuration parameters, pass in NULL for default.
query	Query definition.
pf	Partition filter.
listener	The function to be called for each returned value.
udata	User-data to be passed to the callback.
event_loop	Event loop assigned to run this command. If NULL, an event loop will be chosen by round-robin.

Returns

AEROSPIKE_OK if async query succesfully queued. Otherwise an error.

aerospike_query_wait()

static as_status aerospike_query_wait ( aerospike * as, as_error * err, const as_policy_info * policy, const as_query * query, uint64_t query_id, uint32_t interval_ms )

inlinestatic

Wait for a background query to be completed by servers.

Parameters

as	Aerospike cluster instance.
err	Error detail structure that is populated if an error occurs.
policy	Info configuration parameters, pass in NULL for default.
query	The query that was executed against the cluster.
query_id	The id for the query job, which can be used for obtaining query status.
interval_ms	Polling interval in milliseconds. If zero, 1000 ms is used.

Returns

AEROSPIKE_OK on success, otherwise an error.

Definition at line 429 of file aerospike_query.h.

References aerospike_job_wait().

as_query_apply()

AS_EXTERN bool as_query_apply ( as_query * query, const char * module, const char * function, const as_list * arglist )

related

Apply a function to the results of the query.

as_query_apply(&query, "my_module", "my_function", NULL);

Parameters

query	The query to apply the function to.
module	The module containing the function to invoke.
function	The function in the module to invoke.
arglist	The arguments to use when calling the function.

Returns

On success, true. Otherwise an error occurred.

as_query_compare()

AS_EXTERN bool as_query_compare ( as_query * q1, as_query * q2 )

related

Compare query objects.

as_query_destroy()

AS_EXTERN void as_query_destroy ( as_query * query )

related

Destroy the query and associated resources.

as_query_from_bytes()

AS_EXTERN bool as_query_from_bytes ( as_query * query, const uint8_t * bytes, uint32_t bytes_size )

related

Deserialize bytes to query definition. Query definition is assumed to be on the stack. as_query_destroy() should be called when done with the query definition.

Returns

true on success and false on failure.

as_query_from_bytes_new()

AS_EXTERN as_query * as_query_from_bytes_new ( const uint8_t * bytes, uint32_t bytes_size )

related

Create query definition on the heap and deserialize bytes to that query definition. as_query_destroy() should be called when done with the query definition.

Returns

query definition on success and NULL on failure.

as_query_init()

AS_EXTERN as_query * as_query_init ( as_query * query, const char * ns, const char * set )

related

Initialize a stack allocated as_query.

as_query query;
as_query_init(&query, "test", "demo");

Parameters

query	The query to initialize.
ns	The namespace to query.
set	The set to query.

Returns

On success, the initialized query. Otherwise NULL.

as_query_is_done()

static bool as_query_is_done ( as_query * query )

related

If using query pagination, did the previous paginated query with this query instance return all records?

Definition at line 899 of file as_query.h.

References as_partitions_status::done, and as_query::parts_all.

as_query_new()

AS_EXTERN as_query * as_query_new ( const char * ns, const char * set )

related

Create and initialize a new heap allocated as_query.

as_query* query = as_query_new("test", "demo");

Parameters

ns	The namespace to query.
set	The set to query.

Returns

On success, the new query. Otherwise NULL.

as_query_select()

AS_EXTERN bool as_query_select ( as_query * query, const char * bin )

related

Select bins to be projected from matching records.

You have to ensure as_query.select has sufficient capacity, prior to adding a bin. If capacity is sufficient then false is returned.

as_query_select_init(&query, 2);
as_query_select(&query, "bin1");
as_query_select(&query, "bin2");

Parameters

query	The query to modify.
bin	The name of the bin to select.

Returns

On success, true. Otherwise an error occurred.

as_query_select_init()

AS_EXTERN bool as_query_select_init ( as_query * query, uint16_t n )

related

Initializes as_query.select with a capacity of n using malloc().

For stack allocation, use as_query_select_inita().

as_query_select_init(&query, 2);
as_query_select(&query, "bin1");
as_query_select(&query, "bin2");

Parameters

query	The query to initialize.
n	The number of bins to allocate.

Returns

On success, the initialized. Otherwise an error occurred.

as_query_set_paginate()

static void as_query_set_paginate ( as_query * query, bool paginate )

related

Set if records should be read in pages in conjunction with max_records policy. If true, the client will save the status of all partitions after the query completes. The partition status can be used to resume the query if terminated early by error, user callback, or max_records being reached. Use as_query_set_partitions() or as_partition_filter_set_partitions() to resume a query.

The partition status will be destroyed when as_query_destroy() is called.

Definition at line 873 of file as_query.h.

References as_query::paginate.

as_query_set_partitions()

static void as_query_set_partitions ( as_query * query, as_partitions_status * parts_all )

related

Set completion status of all partitions from a previous query that ended early. The query will resume from this point.

Definition at line 886 of file as_query.h.

References as_partitions_status_reserve(), and as_query::parts_all.

as_query_to_bytes()

AS_EXTERN bool as_query_to_bytes ( const as_query * query, uint8_t ** bytes, uint32_t * bytes_size )

related

Serialize query definition to bytes.

as_query_where()

AS_EXTERN bool as_query_where ( as_query * query, const char * bin, as_predicate_type type, as_index_type itype, as_index_datatype dtype, ... )

related

Add a predicate to the query.

You have to ensure as_query.where has sufficient capacity, prior to adding a predicate. If capacity is insufficient then false is returned.

String predicates are not owned by as_query. If the string is allocated on the heap, the caller is responsible for freeing the string after the query has been executed. as_query_destroy() will not free this string predicate.

as_query_where_init(&query, 3);
as_query_where(&query, "bin1", as_string_equals("abc"));
as_query_where(&query, "bin1", as_integer_equals(123));
as_query_where(&query, "bin1", as_integer_range(0,123));

Parameters

query	The query add the predicate to.
bin	The name of the bin the predicate will apply to.
type	The type of predicate.
itype	The type of index.
dtype	The underlying data type that the index is based on.
...	The values for the predicate.

Returns

On success, true. Otherwise an error occurred.

as_query_where_init()

AS_EXTERN bool as_query_where_init ( as_query * query, uint16_t n )

related

Initializes as_query.where with a capacity of n using malloc().

For stack allocation, use as_query_where_inita().

as_query_where_init(&query, 1);
as_query_where(&query, "bin1", as_integer_equals(123));

Parameters

query	The query to initialize.
n	The number of as_predicate to allocate.

Returns

On success, true. Otherwise an error occurred.

as_query_where_with_ctx()

AS_EXTERN bool as_query_where_with_ctx ( as_query * query, const char * bin, struct as_cdt_ctx * ctx, as_predicate_type type, as_index_type itype, as_index_datatype dtype, ... )

related

Add a predicate and context to the query.

You have to ensure as_query.where has sufficient capacity, prior to adding a predicate. If capacity is insufficient then false is returned.

String predicates are not owned by as_query. If the string is allocated on the heap, the caller is responsible for freeing the string after the query has been executed. as_query_destroy() will not free this string predicate.

as_cdt_ctx ctx;
as_cdt_ctx_init(&ctx, 1);
as_cdt_ctx_add_list_rank(&ctx, -1);
as_query_where_init(&query, 3);
as_query_where_with_ctx(&query, "bin1", &ctx, as_string_equals("abc"));
as_query_where_with_ctx(&query, "bin1", &ctx, as_integer_equals(123));
as_query_where_with_ctx(&query, "bin1", &ctx, as_integer_range(0,123));

Parameters

query	The query add the predicate to.
bin	The name of the bin the predicate will apply to.
ctx	The CDT context describing the path to locate the data to be indexed.
type	The type of predicate.
itype	The type of index.
dtype	The underlying data type that the index is based on.
...	The values for the predicate.

Returns

On success, true. Otherwise an error occurred.