Configure bin transforms for Aerospike Connect for Kafka outbound

The bin-transforms configuration specifies the transformations to apply on the Aerospike record bin names to have a desired corresponding attribute name and filtering bins using include/exclude in the published message. The transforms can be specified as a mapping or a series of transforms to apply on the bin names. In case a bin name is specified in the mapping section, then the transforms are not applied on the matched bin.

Include and exclude

The include and exclude options specify bins to include into or exclude from further processing.

Both options have a mandatory property called type which specifies one of the three supported types:

Type	Description
`all`	`include` or `exclude` all the bins. This is the default value for `include`.
`none`	`include` or `exclude` no bins. This is the default value for `exclude`.
`specified`	`include` or `exclude` specified bins. This `type` requires additional property called `bins` which specifies list of bins to `include` or `exclude`.

Examples

Include specified bins

An Aerospike record has a customer’s details spread across bins. You want to include subset of those bins into a published message. The following configuration ships only those bins which are part of the include list.

...
bin-transforms:
  ...
  include:
    type: specified
    bins:
      - name
      - dob
      - mobile_no
  ...
...

Input	Output
name, dob, age, mobile_no, email_id, fax_no	name, dob, mobile_no

Exclude specified bins

An Aerospike record has a customer’s details spread across bins. You want to exclude subset of those bins from a published message. The following configuration ships only those bins which are not part of the exclude list.

...
bin-transforms:
  ...
  exclude:
    type: specified
    bins:
      - age
      - fax_no
  ...
...

Input	Output
name, dob, age, mobile_no, email_id, fax_no	name, dob, mobile_no, email_id

Exclude all bins

Below configuration will only ship a metadata of the record, no bins.

...
bin-transforms:
  ...
  exclude:
    type: all
  ...
...

Map

The map option specifies the name of one or more bins and the names of their corresponding attributes in the published messages. For example, suppose that your Aerospike records contained these bins and messages should have these attributes:

Bins	attributes
`lastname`	`last_name`
`firstname`	`first_name`
`mobile`	`mobile_phone`

The entries for the map option in the bin-transforms section of your configuration file would look like this:

Example

...
bin-transforms:
  ...
  map:
    lastname: last_name
    firstname: first_name
    mobile: mobile_phone
  ...
...

Transforms

If your Aerospike records include bins whose names do not match attribute names and that are not listed as entries for the map option, then you must use the transforms option to specify how to map bin names to attribute names.

The following transforms are supported:

Transform	Description
`lowercase`	Converts to lowercase.
`uppercase`	Converts to uppercase.
`trim`	Trims leading and trailing whitespace.
`regex`	Matches against a regex pattern and replaces all occurrences with a replacement. The regex and replacement use Java regex syntax.

Example

This example shows a mapping of the bin name red-color to the attribute name red. It also shows transforms to apply to all other bin names.

...
bin-transforms:
  ...
  map:
    red-color: red
  transforms:
    - regex:
        pattern: '-'
        replacement: '_'
    - uppercase
  ...
...

If an Aerospike record includes the bins red-color and blue-color, the corresponding attributes in the published message will have the names red and BLUE_COLOR, as shown in the table below:

Aerospike Bin Name	Transformed name
red-color	red
blue-color	BLUE_COLOR