Configure Bin Transforms for Aerospike Connect for Elasticsearch

The bin-transforms configuration specifies the transformations to apply on the Aerospike record bin names to have a desired corresponding attribute name and filtering bins using include/exclude in the published message. The transforms can be specified as a mapping or a series of transforms to apply on the bin names. In case a bin name is specified in the mapping section, then the transforms are not applied on the matched bin.

Include and exclude

The include and exclude options specify bins to include into or exclude from further processing.

info

exclude always gets higher precedence over include.

Both options have a mandatory property called type which specifies one of the three supported types:

Type	Description
all	include or exclude all the bins. This is the default value for include.
none	include or exclude no bins. This is the default value for exclude.
specified	include or exclude specified bins. This type requires additional property called bins which specifies list of bins to include or exclude.

Examples

Include specified bins

An Aerospike record has a customer's details spread across bins. You want to include subset of those bins into a published message. The following configuration ships only those bins which are part of the include list.

...
bin-transforms:
  ...
  include:
    type: specified
    bins: 
      - name
      - dob
      - mobile_no
  ...
...

Input	Output
name, dob, age, mobile_no, email_id, fax_no	name, dob, mobile_no

Exclude specified bins

An Aerospike record has a customer's details spread across bins. You want to exclude subset of those bins from a published message. The following configuration ships only those bins which are not part of the exclude list.

...
bin-transforms:
  ...
  exclude:
    type: specified
    bins: 
      - age
      - fax_no
  ...
...

Input	Output
name, dob, age, mobile_no, email_id, fax_no	name, dob, mobile_no, email_id

Exclude all bins

Below configuration will only ship a metadata of the record, no bins.

...
bin-transforms:
  ...
  exclude:
    type: all
  ...
...

Map

The map option specifies the name of one or more bins and the names of their corresponding attributes in the published messages. For example, suppose that your Aerospike records contained these bins and messages should have these attributes:

Bins	attributes
lastname	last_name
firstname	first_name
mobile	mobile_phone

The entries for the map option in the bin-transforms section of your configuration file would look like this:

Example

...
bin-transforms:
  ...
  map:
    lastname: last_name
    firstname: first_name
    mobile: mobile_phone
  ...
...

caution

If a bin name is specified in map then the transforms section is not applied on this bin name.

Transforms

If your Aerospike records include bins whose names do not match attribute names and that are not listed as entries for the map option, then you must use the transforms option to specify how to map bin names to attribute names.

The following transforms are supported:

Transform	Description
lowercase	Converts to lowercase.
uppercase	Converts to uppercase.
trim	Trims leading and trailing whitespace.
regex	Matches against a regex pattern and replaces all occurrences with a replacement. The regex and replacement use Java regex syntax.

Example

This example shows a mapping of the bin name red-color to the attribute name red. It also shows transforms to apply to all other bin names.

...
bin-transforms:
  ...
  map:
    red-color: red
  transforms:
    - regex:
        pattern: '-'
        replacement: '_'
    - uppercase
  ...
...

If an Aerospike record includes the bins red-color and blue-color, the corresponding attributes in the published message will have the names red and BLUE_COLOR, as shown in the table below:

Aerospike Bin Name	Transformed name
red-color	red
blue-color	BLUE_COLOR