Loading
Configure Formats for Aerospike Connect for Elasticsearch
Overview
This page descibes format
section of the aerospike-elasticsearch-outbound.yaml
file which specifies the serialization format of the data output.
Aerospike record metadata
Every Aerospike record has a metadata associated with it. Following is the list of metadata fields that XDR sends to connector with every record:
Metadata | Type | Description | Present in |
---|---|---|---|
msg | string | Write/Delete operation. | Delete and Write |
namespace | string | Namespace of the Aerospike record. | Delete and Write |
set | string | Set of the Aerospike record. | Write |
userKey | long, double, bytes or string | User key of the Aerospike record. | Write (only if user key is stored on server) |
digest | bytes | Digest of the Aerospike record. | Delete and Write |
gen | int | Generation of the Aerospike record. | Write |
lut | int | Last update time of the Aerospike record. | Write |
exp | int | Expiry of the Aerospike record. | Write |
durable | boolean | Whether the delete is durable. | Delete |
caution
- HyperLogLog bins are forwarded as is, as bytes or (for string-based formats) as Base64-encoded strings.
danger
Elasticsearch has strict checks for data-types of all the documents of an index. All Aerospike record bins should be converted to Elasticsearch document fields as per the index's mapping/schema. The connector will report Elasticsearch's mapper_parsing_exception
in case of mapping violation.
The only supported format is custom. The payload-format
of this custom format can either be JSON or FlatJSON.
Example
...
format:
mode: custom
payload-format:
mode: flat-json
batch-formatter-class: com.aerospike.connect.outbound.transformer.examples.elasticsearch.ElasticsearchCustomJsonBatchFormatter
...