Skip to main content

Stream UDFs

Use Aerospike Stream UDFs to filter, transform, and aggregate query results in a distributed fashion. Stream UDFs can process a stream of data by defining a sequence of operations to perform. Stream UDF aggregations run in a distributed fashion to exploit the power of multiple machines.

Stream UDFs perform read-only operations on a collection of records. The Stream UDF system allows a MapReduce style of programming, which is used for common, highly parallel jobs such as word count—each row is accessed and a list of words and their counts are emitted. The top results are calculated in a reduce phase. For simple aggregations where counters are simply incremented in a context instead of continually creating and destroying objects, Aerospike provides optimal implementation.

Operators

Aerospike Stream UDFs support these operators:

  • filter—Filter data in the stream that satisfies a predicate.
  • map—Transform a piece of data.
  • aggregate—Reduce partitions of data to a single value.
  • reduce—Allow parallel processing of each group of output data.

These operators are considered primitives and can build complex operations (see Developing Stream UDFs).

filter

Filter values in the stream using the predicate p. p tests each value in the stream and returns true if the value should be passed through or false to drop the value.

Example

filter(p: (a: Value) -> Boolean) -> Stream

Parameters

  • p—The predicate to apply to each value in the stream.

Returns

A stream of filtered values.


map

Transforms a value to a different value using the function f.

map(f: (a: Value) -> Value) -> Stream

Parameters

  • f—The function to apply to each value in the stream.

Returns

A stream of the transformed values.


reduce

Reduce the values in the stream to a single value and apply the associative binary operator op to each value. For subsequent calls to op, the last returned value is used as the a parameter.

reduce(op: (a: Value, b: Value) -> Value) -> Stream

Parameters

  • op—The associative binary operation to apply to each value in the stream.

Returns

A stream containing a single value.


aggregate

Takes the current value (parameter x) and the next value in the input stream, and returns the aggregate value (parameter a).

aggregate(x: Value, op: (a: Value, b: Value) -> Value) -> Stream

Parameters

  • x—The initial (neutral) value passed to the operator op.
  • op—The function to apply to each value in the stream.

Returns

A stream containing a single value.


Examples

References