Skip to content

HyperLogLog expressions

For the complete documentation index see: llms.txt

All documentation pages available in markdown.

HyperLogLog (HLL) expressions run the same read and modify operations as the HLL data type — estimated cardinality, union, intersection, similarity, describe, may-contain — inside an expression tree rather than against a named bin.

Read expressions (hll_get_count, hll_get_union, …) evaluate to an integer, float, list, or HLL value. That makes them projection: they shape what a command returns for each record, in operate, in a batch, and in a query. Wrap one in comparison or logic to get the Boolean a record filter needs.

Modify expressions (hll_add, hll_update, …) evaluate to a new HLL value and leave the record untouched. Storing that value takes a write operation expression; see Operations and expressions for how the two APIs differ.

An HLL operand is a bin, read with bin_hll, or any expression that evaluates to an HLL. Reading stored data can evaluate to unknown; see Unknown results.

The Developer SDK’s AEL text syntax exposes the same HLL read functions as path methods, for example $.h.hllCount() > 1000000.

Composing expressions

An expression evaluates to a value, and that value is what the next expression operates on — see Expressions compose.

hll_add is the clearest case: it evaluates to an HLL value, and hll_get_count counts that value, neither of them touching the record. Most HLL operations take a nested expression as their bin operand the same way, including the union and intersection reads.

hll_describe composes in the other direction. It evaluates to a two-element list, so pair it with a list read such as list_get_by_index to pull out the index bit count or the MinHash bit count on its own.

Each operation’s Example shows the code in nine tabs: Aerospike Expression Language (AEL) text on the Java SDK and Python SDK tabs, and the Exp builder on the other seven. See the AEL reference for AEL grammar.

Modify

hll_add

hll_add(policy, items, index_bit_count, bin)
Description

Returns the HLL that bin evaluates to, with items added. It cannot bring a sketch into existence: if bin names a record bin that does not exist, the expression returns AS_ERR_OP_NOT_APPLICABLE.

Arguments
NameTypeDescription
policyHLL policy

Write policy for the transform. Supplied by your client as an HLL policy object.

itemslist

Elements to add to the sketch.

index_bit_countinteger

Number of index bits, between 4 and 16. It has no effect here: it sizes a sketch at creation, and this expression cannot create one. Use hll_init to build a sketch of a given size.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
hll
Introduced
5.2.0.4
Example

Add string ids from list bin new_user_ids into the existing visitors sketch, then require a non-zero estimated count.

String exp = "$.visitors.hllAdd($.new_user_ids, indexBits: 10).hllCount() > 0";

hll_add_mh

hll_add_mh(policy, items, index_bit_count, minhash_bit_count, bin)
Description

Returns the HLL that bin evaluates to, with items added. Like hll_add, it requires the operand to already be an HLL. MinHash bits make hll_get_intersect_count and hll_get_similarity reliable, but they belong to the sketch being operated on — set them where it is created, with the Operate API add_mh or init.

Arguments
NameTypeDescription
policyHLL policy

Write policy for the transform. Supplied by your client as an HLL policy object.

itemslist

Elements to add to the sketch.

index_bit_countinteger

Number of index bits, between 4 and 16. It has no effect here: it sizes a sketch at creation, and this expression cannot create one. Use hll_init to build a sketch of a given size.

minhash_bit_countinteger

Number of MinHash bits, between 4 and 51, summing with the index bits to 64 or fewer. Like index_bit_count it has no effect here; see hll_init.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
hll
Introduced
5.2.0.4
Example

Same as hll_add, taking both bit counts. Neither has an effect on an existing sketch.

String exp = "$.visitors.hllAdd($.new_user_ids, indexBits: 10, minHashBits: 20)"
+ ".hllCount() > 0";

hll_init

hll_init(policy, index_bit_count[, minhash_bit_count], bin)
Description

Returns a new, empty HLL sized by index_bit_count and, where given, minhash_bit_count. Unlike hll_add, the bit counts take effect here, because this expression is the one that builds the sketch. The record is not touched: storing the result takes a write operation expression.

Arguments
NameTypeDescription
policyHLL policy

Write policy for the transform. Supplied by your client as an HLL policy object.

index_bit_countinteger

Number of index bits, between 4 and 16. More bits give a more accurate estimate and a larger sketch.

minhash_bit_countinteger

Number of MinHash bits, between 4 and 51, summing with the index bits to 64 or fewer. MinHash bits improve intersection and similarity estimates. Optional; omit them for a plain HyperLogLog.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
hll
Introduced
5.2.0.4
Example

Size a fresh sketch at index_bit_count 12 and confirm it is empty. Because hll_init discards whatever the operand held, the count is 0 however many values visitors contains.

String exp = "$.visitors.hllInit(indexBits: 12).hllCount() == 0";

hll_update

hll_update(policy, items, bin)
Description

Returns the HLL that bin evaluates to, with items added. It differs from hll_add only in taking no bit counts.

Arguments
NameTypeDescription
policyHLL policy

Write policy for the transform. Supplied by your client as an HLL policy object.

itemslist

Elements to add to the sketch.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
hll
Introduced
5.2.0.4
Example

Add values from list bin new_user_ids to an existing HLL visitors (no index/minhash creation path).

String exp = "$.visitors.hllAdd($.new_user_ids).hllCount() > 0";

Read

hll_describe

hll_describe(bin)
Description

Returns a two-element list holding the HLL’s index bit count and MinHash bit count. Pair it with a list read such as list_get_by_index to pull out one of the two.

Arguments
NameTypeDescription
binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
list
Introduced
5.2.0.4
Example

The HLL’s index bit count (describe result index 0) is fewer than 16; uses a list read on the describe list.

String exp = "($.visitors.hllDescribe()).[0] < 16";

hll_get_count

hll_get_count(bin)
Description

Returns the estimated number of distinct elements in the HLL. The error depends on the index bit count the sketch was built with; see Error bounds.

Arguments
NameTypeDescription
binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
integer
Introduced
5.2.0.4
Example

Filter to records whose HLL visitors sketch estimates more than 100 unique ids.

String exp = "$.visitors.hllCount() > 100";

hll_get_intersect_count

hll_get_intersect_count(hll_list, bin)
Description

Returns the estimated number of distinct elements common to hll_list and the HLL that bin evaluates to. Accuracy depends on MinHash bits: the estimate is stable when the sketches carry them and unstable when they do not. See Error bounds.

Arguments
NameTypeDescription
hll_listlist

HLLs to intersect with the operand.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
integer
Introduced
5.2.0.4
Example

Estimated intersection size between cohort_a and visitors is at least 50.

String exp = "$.visitors.hllIntersectCount($.cohort_a) >= 50";

hll_get_similarity

hll_get_similarity(hll_list, bin)
Description

Returns the estimated Jaccard index of hll_list and the HLL that bin evaluates to — the size of their intersection over the size of their union, from 0.0 for disjoint sets to 1.0 for identical ones. See Error bounds.

Arguments
NameTypeDescription
hll_listlist

HLLs to compare against the operand.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
float
Introduced
5.2.0.4
Example

Jaccard-style similarity between cohort_a and visitors is at least 0.5.

String exp = "$.visitors.hllSimilarity($.cohort_a) >= 0.5";

hll_get_union

hll_get_union(hll_list, bin)
Description

Returns an HLL representing the union of hll_list and the HLL that bin evaluates to.

Arguments
NameTypeDescription
hll_listlist

HLLs to union with the operand.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
hll
Introduced
5.2.0.4
Example

Union of HLL cohort_a with visitors, then require a non-empty estimate (via hll_get_count on the union result).

String exp = "$.visitors.hllUnion($.cohort_a).hllCount() > 0";

hll_get_union_count

hll_get_union_count(hll_list, bin)
Description

Returns the estimated number of distinct elements in the union of hll_list and the HLL that bin evaluates to.

Arguments
NameTypeDescription
hll_listlist

HLLs to union with the operand.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
integer
Introduced
5.2.0.4
Example

Estimated cardinality of the union of cohort_a and visitors exceeds 500.

String exp = "$.visitors.hllUnionCount($.cohort_a) > 500";

hll_may_contain

hll_may_contain(values, bin)
Description

Returns 1 if the HLL that bin evaluates to may contain every element of values, otherwise 0. A positive answer is probabilistic; a negative answer is certain.

Arguments
NameTypeDescription
valueslist

Elements to look for; the result is 1 only when the HLL may contain every one.

binhll

HLL bin to operate on, or any expression that evaluates to an HLL.

Returns
integer
Introduced
5.2.0.4
Example

The HLL may contain every candidate key listed in list bin candidate_keys (returns 1 when the check is positive).

String exp = "$.visitors.hllMayContain($.candidate_keys) == 1";