Skip to content

Read-write transactions

For the complete documentation index see: llms.txt

All documentation pages available in markdown.

In this tutorial, you run read-modify-write (R-M-W) updates from several concurrent workers. You see how concurrent updates lose writes, then use the record’s generation to prevent that loss. Choose your client in any tab group. The page remembers your choice.

The Java SDK and Python SDK tabs use the Aerospike Developer SDKs. The Java and Python tabs use the legacy clients.

Prerequisites

  • An Aerospike Database running on your computer and listening on port 3000, with a namespace named test. The Java SDK and Python SDK tabs require Aerospike Database 8.2.0 or later. For the other clients, see the Client Compatibility Matrix. To set up a database, see the Quickstart.
  • Your client installed. To install it, follow the Install the client step of the Hello, World! tutorial.
  • Familiarity with the Hello, World! and Basic operations tutorials.

Introduction

By default, Aerospike runs a single command on a single record atomically and in isolation. A command can have several operations on that record. Some updates fit in one command, such as incrementing a number, or using an expression to compute a bin from other bins in the record.

An update that needs general logic, such as concatenating alternate letters, or data from outside the record can’t run as one command of operations or expressions. A User-Defined Function (UDF) can run complex logic in a transactional update of a single record, but UDFs aren’t suitable for every situation, for reasons such as performance and ease of use. Most such updates therefore use the R-M-W pattern: read the record, modify the bins on the client, and write the updated record back to the server.

Aerospike Database 8.0 and later also supports multi-record transactions. This tutorial covers only updates to a single record.

This tutorial first shows how read-write commands can lose writes when several clients run at the same time. It then shows how to make a write conditional on the record’s version. An interleaved read and write is rejected, so no write is lost.

The problem of lost writes

In a concurrent setting, several clients can run a read-modify-write on the same record and get in each other’s way. The transactions can interleave, so a read-modify-write attempt is lost if another client updates the record without having read the first client’s update.

To show this, the tutorial uses the record’s generation, which is the record’s version. The generation is record metadata, and the database increments it on each successful update of the record.

The record has an integer bin named gen_times_2. It holds 2 times the generation that the record has after an update. A client first reads the record’s generation, and then writes a new value that is 2 times the generation the record has after the write.

With a single client, the bin satisfies this rule after every read-modify-write. With several clients, a client can write a value that it computed from an older generation, so the bin no longer holds 2 times the current generation. That’s a lost write.

The tutorial detects lost writes by checking the relationship between the new generation and the value in the bin. Then it shows how a conditional write with a generation check solves the problem.

Import the client modules

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.LockSupport;
import java.util.function.Supplier;
import com.aerospike.client.sdk.AerospikeException;
import com.aerospike.client.sdk.Cluster;
import com.aerospike.client.sdk.ClusterDefinition;
import com.aerospike.client.sdk.DataSet;
import com.aerospike.client.sdk.Key;
import com.aerospike.client.sdk.Record;
import com.aerospike.client.sdk.Session;
import com.aerospike.client.sdk.policy.Behavior;

Connect to the database

Connect to the database on port 3000 of your computer. This is the default port. If your database runs on another host or port, change the address.

Cluster cluster = new ClusterDefinition("127.0.0.1", 3000).connect();
Session session = cluster.createSession(Behavior.DEFAULT);

Write the test record

Write one record with an integer bin named gen_times_2, set to 1. The record is new, so its generation is 1.

Key key = DataSet.of("test", "rmw").id("id-1");
session.upsert(key).bin("gen_times_2").setTo(1).execute();

Run workers concurrently

To simulate concurrent clients, the test starts several workers. Each worker runs the read-modify-write function 10 times in a row. A function returns true if its write was lost, and false if it wasn’t. The test reports whether any worker lost a write.

The functions in this tutorial return a value, instead of printing, so they run on any thread.

static final int TXNS_PER_WORKER = 10;
static void sleepMs(long ms) {
LockSupport.parkNanos(TimeUnit.MILLISECONDS.toNanos(ms));
}
static void runTest(int workers, Supplier<Boolean> rmw) {
ExecutorService pool = Executors.newFixedThreadPool(workers);
List<CompletableFuture<Boolean>> futures = new ArrayList<>();
for (int w = 0; w < workers; w++) {
futures.add(CompletableFuture.supplyAsync(() -> {
boolean lost = false;
for (int i = 0; i < TXNS_PER_WORKER; i++) {
lost |= rmw.get();
}
return lost;
}, pool));
}
boolean lostWrites = false;
for (CompletableFuture<Boolean> future : futures) {
lostWrites |= future.join();
}
pool.shutdown();
System.out.println("Workers: " + workers + ", lost writes: " + (lostWrites ? "yes" : "no"));
}

Read and write without a check

The first function is a simple read-modify-write. It does the following:

  • Reads the record, which returns the record’s generation.
  • Waits up to 100 milliseconds at random to simulate time an application spends computing a new value.
  • Computes the new value of gen_times_2, which is 2 times the generation the write creates: 2 times the sum of the generation the function read and 1.
  • Writes the new value, and gets the record’s new generation in the same command.
  • Returns true if the new value divided by 2 isn’t the new generation. That means another client wrote the record between the read and the write, and the first write was lost. Otherwise, it returns false.
static boolean rmwSimple(Session session, Key key) {
Record record = session.query(key).execute().getFirstRecord();
int readGen = record.generation;
sleepMs(ThreadLocalRandom.current().nextInt(0, 100));
int newValue = 2 * (readGen + 1);
Record written = session.upsert(key).bin("gen_times_2").setTo(newValue).execute().getFirstRecord();
return newValue / 2 != written.generation;
}

Run the test without a check

With greater concurrency, a larger share of the read-write transactions are lost. With one worker, none are lost. With two or more workers, the transactions interleave, and the value in the gen_times_2 bin doesn’t stay at 2 times the generation. The test prints yes when any worker lost a write.

for (int workers = 1; workers <= 4; workers++) {
runTest(workers, () -> rmwSimple(session, key));
}
Workers: 1, lost writes: no
Workers: 2, lost writes: yes
Workers: 3, lost writes: yes
Workers: 4, lost writes: yes
Output

Lost writes depend on timing. With two or more workers, you almost always see yes. If a row shows no, run the program again.

Read and write with a generation check

To prevent lost writes, make the write conditional on the record not having changed since the read. This is a check-and-set operation. The write succeeds only if the record’s generation is still the generation the function read. Otherwise, the write fails with a generation error, and the client must retry the whole read-modify-write.

The function retries a failed write up to three times. Before each retry, it uses exponential backoff: 40, 80, and then 160 milliseconds.

If the write still fails after the retries, the function gives up. If the function gives up on a write, that write is not applied, so it is not a lost write. The function returns false in that case. No write is overwritten or lost, and gen_times_2 always stays at 2 times the generation.

static boolean rmwWithGenCheck(Session session, Key key) {
for (int retry = 0; retry <= 3; retry++) {
Record record = session.query(key).execute().getFirstRecord();
int readGen = record.generation;
sleepMs(ThreadLocalRandom.current().nextInt(0, 100));
int newValue = 2 * (readGen + 1);
try {
Record written = session.upsert(key).bin("gen_times_2").setTo(newValue)
.ensureGenerationIs(readGen).execute().getFirstRecord();
return newValue / 2 != written.generation;
} catch (AerospikeException.GenerationException e) {
if (retry < 3) sleepMs((1L << (retry + 1)) * 20);
}
}
return false;
}

The ensureGenerationIs method makes the write conditional on the generation. A mismatch throws AerospikeException.GenerationException.

Run the test with the generation check

Run the same test with the generation check. No worker loses a write. Under heavy concurrency, a worker can still run out of retries and give up on a write, but giving up never corrupts the record.

for (int workers = 2; workers <= 4; workers++) {
runTest(workers, () -> rmwWithGenCheck(session, key));
}
Workers: 2, lost writes: no
Workers: 3, lost writes: no
Workers: 4, lost writes: no
Output

Delete the record and close the connection

Delete the test record, then close the client.

session.delete(key).execute();
cluster.close();

Complete program

This program combines the steps. It writes the test record, runs four workers without a check and then with the generation check, and deletes the record.

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.LockSupport;
import java.util.function.Supplier;
import com.aerospike.client.sdk.AerospikeException;
import com.aerospike.client.sdk.Cluster;
import com.aerospike.client.sdk.ClusterDefinition;
import com.aerospike.client.sdk.DataSet;
import com.aerospike.client.sdk.Key;
import com.aerospike.client.sdk.Record;
import com.aerospike.client.sdk.Session;
import com.aerospike.client.sdk.policy.Behavior;
public class ReadModifyWrite {
static final int TXNS_PER_WORKER = 10;
static void sleepMs(long ms) {
LockSupport.parkNanos(TimeUnit.MILLISECONDS.toNanos(ms));
}
static void runTest(int workers, Supplier<Boolean> rmw) {
ExecutorService pool = Executors.newFixedThreadPool(workers);
List<CompletableFuture<Boolean>> futures = new ArrayList<>();
for (int w = 0; w < workers; w++) {
futures.add(CompletableFuture.supplyAsync(() -> {
boolean lost = false;
for (int i = 0; i < TXNS_PER_WORKER; i++) {
lost |= rmw.get();
}
return lost;
}, pool));
}
boolean lostWrites = false;
for (CompletableFuture<Boolean> future : futures) {
lostWrites |= future.join();
}
pool.shutdown();
System.out.println("Workers: " + workers + ", lost writes: " + (lostWrites ? "yes" : "no"));
}
static boolean rmwSimple(Session session, Key key) {
Record record = session.query(key).execute().getFirstRecord();
int readGen = record.generation;
sleepMs(ThreadLocalRandom.current().nextInt(0, 100));
int newValue = 2 * (readGen + 1);
Record written = session.upsert(key).bin("gen_times_2").setTo(newValue).execute().getFirstRecord();
return newValue / 2 != written.generation;
}
static boolean rmwWithGenCheck(Session session, Key key) {
for (int retry = 0; retry <= 3; retry++) {
Record record = session.query(key).execute().getFirstRecord();
int readGen = record.generation;
sleepMs(ThreadLocalRandom.current().nextInt(0, 100));
int newValue = 2 * (readGen + 1);
try {
Record written = session.upsert(key).bin("gen_times_2").setTo(newValue)
.ensureGenerationIs(readGen).execute().getFirstRecord();
return newValue / 2 != written.generation;
} catch (AerospikeException.GenerationException e) {
if (retry < 3) sleepMs((1L << (retry + 1)) * 20);
}
}
return false;
}
public static void main(String[] args) {
try (Cluster cluster = new ClusterDefinition("127.0.0.1", 3000).connect()) {
Session session = cluster.createSession(Behavior.DEFAULT);
Key key = DataSet.of("test", "rmw").id("id-1");
session.upsert(key).bin("gen_times_2").setTo(1).execute();
runTest(4, () -> rmwSimple(session, key));
runTest(4, () -> rmwWithGenCheck(session, key));
session.delete(key).execute();
}
}
}
Workers: 4, lost writes: yes
Workers: 4, lost writes: no
Output

What you learned

You ran read-modify-write transactions from several concurrent workers and saw writes get lost. You made the write conditional on the record’s generation and added retries with exponential backoff when the check fails, so no write was lost.

Next steps