Skip to main content
Loading

Configure Formats for Aerospike Connect for Elasticsearch

Overview

This page descibes format section of the aerospike-elasticsearch-outbound.yaml file which specifies the serialization format of the data output.

Aerospike record metadata

Every Aerospike record has a metadata associated with it. Following is the list of metadata fields that XDR sends to connector with every record:

MetadataTypeDescriptionPresent in
msgstringWrite/Delete operation.Delete and Write
namespacestringNamespace of the Aerospike record.Delete and Write
setstringSet of the Aerospike record.Write
userKeylong, double, bytes or stringUser key of the Aerospike record.Write (only if user key is stored on server)
digestbytesDigest of the Aerospike record.Delete and Write
genintGeneration of the Aerospike record.Write
lutintLast update time of the Aerospike record.Write
expintExpiry of the Aerospike record.Write
durablebooleanWhether the delete is durable.Delete
caution
  • HyperLogLog bins are forwarded as is, as bytes or (for string-based formats) as Base64-encoded strings.
danger

Elasticsearch has strict checks for data-types of all the documents of an index. All Aerospike record bins should be converted to Elasticsearch document fields as per the index's mapping/schema. The connector will report Elasticsearch's mapper_parsing_exception in case of mapping violation.

The only supported format is custom. The payload-format of this custom format can either be JSON or FlatJSON.

Example

...
format:
mode: custom
payload-format:
mode: flat-json
batch-formatter-class: com.aerospike.connect.outbound.transformer.examples.elasticsearch.ElasticsearchCustomJsonBatchFormatter
...