Aerospike 4.2: Storage Efficiency and Speed Improvements

default photo
Brian Bulkowski
Aerospike Founder and CTO
June 5, 2018|3 min read

Aerospike is pleased to announce the availability of our 4.2 release. Close on the heels of our 4.0 release providing consistency, and our 4.1 release with a number of enterprise security and performance features, we have released our 4.2 release with further enhancements.

The theme of these changes is improvements of efficiency and speed.

We have changed our internal storage format. This change requires an extra step during upgrade: please see the following operational guide.

The big change: our new 4.2 Storage Format is far more efficient.

Lower overhead per Record

The overhead per record is far lower. For modest size records, such as 200 byte records, this can result in a 2x benefit on-disk, which results in higher performance, lower wear for writes, and – critically – a massive decrease in the amount of Flash storage to support a given deployment. Deleted record tombstones, for example, become 2.7x smaller. For large objects the benefit is not as great, but still can be substantial, as most use cases contain a blend of large and small objects.

Larger Records supported

8 MB records are now supported. A frequently requested user feature raises the maximum size of a database record to 8 megabytes, from the previous 1 megabyte. Also, the number of logical devices per namespace is now 128.

Faster Restarts

A side effect of this work also dramatically improves time for “fast restart”. With fast restart, Aerospike uses its patented shared memory use for indexes, allowing process restarts to use the in-memory index. Still, on restart, the shared memory must be validated. This performance improvement can be dramatic, taking multi-minute restarts to a few seconds.

Default Number of “Sprigs” Increased

We have increased the default number of “sprigs”. The sprig system is a per-partition hash table which sits above the red-black tree in our core primary index. By using larger hash tables, we decrease the number of memory accesses per lookup. We have improved our core internal data structures in the “sprig” area, which allows us to increase the default size of the sprig layer. This decreases the load on the memory subsystem for each lookup, increasing straight-line performance.

Intra-cluster Communication Code Improvements

We have further improved our intra-cluster communication code. By code inspection, we found our use of JEMalloc allowed us to remove a lock, resulting in higher parallelism and a better than 2x intracluster communication performance improvement.

Additional Improvements

In other changes, the Aerospike management tools have been updated to use the open source PEX tools, which allow offline installation and remove dependencies on internet packages. Several performance histograms have been improved, as well as the histograms relating to object sizes which are read and written. Please see our upgrade guide for details.

Release notes

We look forward to continued improvements of Aerospike.