HyperLogLog operations

Jump to the Code block for a combined complete example.

A HyperLogLog (HLL) bin estimates how many distinct values have been added to it, in a fixed amount of space, without storing the values. Use one to count unique visitors, unique search terms, or any other set too large to store exactly.

The Node.js client builds HLL operations with Aerospike.hll, and runs them with client.operate(). For every HLL operation and its arguments, see HLL operations.

Setup

The examples on this page use the following connection and key:

const Aerospike = await import("aerospike");
const hll = Aerospike.hll;

// Set hosts to your server's address and port
const config = { hosts: "YOUR_HOST_ADDRESS:YOUR_PORT" };

// Creates a key with the namespace "sandbox", set "pages", and user key "home"
const key = new Aerospike.Key("sandbox", "pages", "home");

// Establishes a connection to the server
const client = await Aerospike.connect(config);

Add values and read the count

hll.add(bin, values, indexBits) adds the values in an array to the HLL bin, creating the bin if it doesn’t exist. The index bit count, from 4 to 16, sets the bin’s size and accuracy when add() creates it: more bits give a closer estimate in a larger bin. hll.getCount(bin) returns the estimated number of distinct values in the bin.

The following example adds five visitor IDs to a visitors bin with 10 index bits, including one ID twice. It reads the estimated count in the same operate() call.

const ops = [
  // Add the values, creating the bin with 10 index bits if it doesn't exist
  hll.add("visitors", ["user-1", "user-2", "user-3", "user-1", "user-4"], 10),
  // Read the estimated distinct count
  hll.getCount("visitors"),
];

const record = await client.operate(key, ops);
console.info("Distinct visitors: %d", record.bins.visitors); // 4

On its own, hll.add() returns the number of values that updated the HLL bin, that is, the values that changed one of its registers. Because both operations in the example return a result for the visitors bin, record.bins.visitors holds only the last one, the count. See Returning from operate().

Create an empty bin

hll.init(bin, indexBits) creates an empty HLL bin, or resets an existing one. hll.describe(bin) returns the bin’s index bit count and its MinHash bit count as a two-element array.

const record = await client.operate(key, [
  hll.init("visitors", 12),
  hll.describe("visitors"),
]);
console.info(record.bins.visitors); // [ 12, 0 ]

Compare with other records

Reading an HLL bin with client.get() returns its value as a Buffer. Pass an array of those values to the operations that compare a bin with other HLL values:

Operation	Returns
hll.getUnionCount(bin, list)	The estimated distinct count of the bin and the values in list combined.
hll.getIntersectCount(bin, list)	The estimated count of values in the bin and in every value in list. When the HLLs have 0 MinHash bits, list can hold at most 2 values.
hll.getSimilarity(bin, list)	The estimated similarity of the bin and the values in list, from 0 to 1. When the HLLs have 0 MinHash bits, list can hold at most 2 values.
hll.setUnion(bin, list)	Merges the values in list into the bin. Doesn’t return a count.

The following example adds visitors to the pricing page, then estimates how many distinct visitors the home and pricing pages have between them.

// Add visitors to the pricing page
const pricingKey = new Aerospike.Key("sandbox", "pages", "pricing");
await client.operate(pricingKey, [hll.add("visitors", ["user-3", "user-5"], 10)]);

// Read the pricing page's HLL value
const pricing = await client.get(pricingKey);

// Estimate the distinct visitors across both pages
const record = await client.operate(key, [
  hll.getUnionCount("visitors", [pricing.bins.visitors]),
]);
console.info("Visitors to either page: %d", record.bins.visitors);

Run each comparison in its own operate() call, or the bin holds only the last result.

Code block

Expand this section for a single code block to count distinct values

const Aerospike = await import("aerospike");
const hll = Aerospike.hll;

// Set hosts to your server's address and port
const config = { hosts: "YOUR_HOST_ADDRESS:YOUR_PORT" };

// Creates a key with the namespace "sandbox", set "pages", and user key "home"
const key = new Aerospike.Key("sandbox", "pages", "home");

// Establishes a connection to the server
const client = await Aerospike.connect(config);

const ops = [
  // Add the values, creating the bin with 10 index bits if it doesn't exist
  hll.add("visitors", ["user-1", "user-2", "user-3", "user-1", "user-4"], 10),
  // Read the estimated distinct count
  hll.getCount("visitors"),
];

const record = await client.operate(key, ops);
console.info("Distinct visitors: %d", record.bins.visitors);

// Close the connection to the server
await client.close();