Configure ordering of outbound records

For the complete documentation index see: llms.txt

All documentation pages available in markdown.

The record-ordering section configures the ordering of records dispatched to the outbound destination. When record-ordering is enabled it guarantees all writes for a record (uniquely identified by namespace and digest) from this connector instance to the outbound destination will be in non-decreasing LUT (last update time) order.

This feature is implemented by maintaining a cache of the records along with its LUT (last update time). A new record, received from Aerospike XDR, is dispatched to the outbound destination only when the LUT of the new record is greater than or equal to the LUT of the cached instance. Records with higher or same LUT are retried until successfully dispatched. Records with lower LUT than the cached instance are acknowledged as successful without being dispatched. They are stale and can be skipped, which improves handling for hot keys (records with many rapid updates).

To keep the RAM memory consumed by the LUT (last update time) cache within reasonable bounds an entry is deleted after a configured duration (lut-cache-ttl-seconds) elapses from the last update to the record’s LUT. Each LUT cache entry consumes about 256 bytes.

Cache entry size	Bytes per million cache entries	Number of entries per GiB
256 bytes	244 MiB	Approximately 4.19 million cache entries.

The options for the record-ordering are as follows

By default, the record-ordering block is not present in the configuration, and record ordering is therefore disabled. Record ordering must be explicitly enabled by configuring this block.

Option	Default	Description
`lut-cache-ttl-seconds`	30	Time to live in seconds for a last update time (LUT) cache entry. An entry is deleted from the cache after `lut-cache-ttl-seconds` elapses from the last update to the record’s LUT. This must be configured to a value greater than the maximum delay a record can encounter between dispatch by Aerospike XDR to being received by the outbound connector. The delay might be caused by slow networks, VM pauses, VM migrations, or other factors.
`enable`	true	Enables record ordering when the `record-ordering` block is present.

Examples

# lut-cache-ttl-seconds defaults to 30
record-ordering:
  enable: true

# Explicitly disable record ordering.
record-ordering:
  enable: false

# enable defaults to true
record-ordering:
  lut-cache-ttl-seconds: 60

record-ordering:
  enable: true
  lut-cache-ttl-seconds: 60