Specify serialization format of the data output

The format section of the /etc/aerospike-esp-outbound/aerospike-esp-outbound.yml specifies the serialization format of the data output. The supported formats are:

XDR: Internal Aerospike XDR wire format required to ship to XDR Proxy
Flat JSON
JSON
Avro
MessagePack
Custom

The default format for ESP outbound connector is XDR.

Aerospike record metadata

The Aerospike record metadata has the following fields:

Metadata	Type	Description	Present in
`msg`	string	Write/Delete operation.	Delete and Write
`namespace`	string	Namespace of the Aerospike record.	Delete and Write
`set`	string	Set of the Aerospike record.	Delete and Write
`userKey`	long, double, bytes or string	User key of the Aerospike record.	Write (only if user key is stored on server)
`digest`	bytes	Digest of the Aerospike record.	Delete and Write
`gen`	int	Generation of the Aerospike record.	Write
`lut`	int	Last update time of the Aerospike record.	Write
`exp`	int	Expiry of the Aerospike record.	Write
`durable`	boolean	Whether the delete is durable.	Delete

Avro

Specifies that the data be serialized as Avro format. The Avro schema can be a map or a record. In the case of a map, specifying the type of the map values is sufficient. In the case of a record, the exact field names and types are required.

Option	Required	Default	Expected value	Description
`mode`	yes		avro	Selects avro format.
`schema`	no			Schema of the data.
`schema-file`	no			File containing the schema of the data.
`stringify-map-keys`	no	true		Whether the numeric keys in CDT maps should be converted to strings. See stringify map keys in avro

Either schema or schema-file has to be specified.

For more information about serializing data in the Avro format, see Avro Format.

Flat JSON

Specifies that the data be serialized to Flat JSON format. The type information of bins is not part of the output.

Option	Required	Default	Expected value	Description
`mode`	yes		flat-json	Selects Flat JSON format.
`metadata-key`	yes			Aerospike record metadata will be inserted into this top level key.
`stringify-map-keys`	no	true		Whether the numeric keys in CDT maps should be converted to strings. See stringify map keys

Example

format:
  mode: flat-json
  metadata-key: metadata

For more information about serializing data in the Flat JSON format, see Flat JSON Format.

JSON

Specifies that the data be serialized to JSON format with the type information of bins.

Option	Required	Default	Expected value	Description
`mode`	yes		json	Selects JSON format.
`stringify-map-keys`	no	true		Whether the numeric keys in CDT maps should be converted to strings. See stringify map keys

Stringify Map Keys

Aerospike allows many data types as keys in collection data types (CDTs) Maps, but JSON only allows strings as keys in objects. Currently, the ESP connector only permits conversion of numeric types to string keys when converting Aerospike CDT maps.

An Aerospike CDT map with any other data type as the key will fail to be parsed in the ESP connector. Also, an Aerospike CDT map with a numeric key, and stringify-map-keys set to false will fail to be parsed in the ESP connector.

Example

format:
  mode: json

For more information about serializing data in the JSON format, see JSON Format.

MessagePack

Specifies that the data be serialized to MessagePack format with the type information about bins.

Option	Required	Expected value	Description
`mode`	yes	message-pack	Selects message pack format.

Example

format:
  mode: message-pack

For more information about serializing data in the MessagePack format, see MessagePack Format.

Custom

Specifies that the data be serialized with user provided custom code. See Format Transform.

Example

format:
  mode: custom
  class: com.aerospike.connect.outbound.example.Formatter