Skip to content

Batched commands

For the complete documentation index see: llms.txt

All documentation pages available in markdown.

A batch is a series of commands that are sent together to the database server. A batch groups multiple commands into one unit and passes it in a single network trip to each database node.

Batch advantages

  • Batches combine multiple record commands, including updates, deletes, reads and UDFs.
  • Batching allows any key-value operation, such as mixing record-level gets and deletes, as well as bin-level transaction commands including increment, prepend/append, Map and List operations, and bitwise operations.
  • Each batch type applies the same command(s) to a list of record keys: batch write, batch exists, batch operate, batch read, batch delete, and batch UDF.
  • When batching multiple updates, the number of connections between client and the server is reduced.
  • Batches optimize the use of network resources, such as packets and network sockets. By increasing throughput, batch can more efficiently use network resources in some cases.
  • Batched commands are used extensively in Aerospike data modeling patterns, where denormalized designs rely on batch reads and writes to access related records efficiently.

Batch workflow

The client groups the primary keys in the batch by cluster node and creates a sub-batch request to each node. If a sub-batch contains only a single key, the client optimizes by sending the equivalent single-record command instead of a batch protocol message — for example, get() for a batch read, operate() for a batch operate, or delete() for a batch delete. Batch requests occur in series or in parallel depending on the batch policy. Parallel requests in synchronous mode require extra threads, which are created or taken from a thread pool.

Batched commands use a single network socket to each cluster node, which helps with parallelizing requests. Multiple keys use one network request, which is beneficial for a large number of small records, but not as beneficial when the number of records per node is small, or the data returned per record is very large. Batch requests sometimes increase the latency of some requests, but only because clients normally wait until all the keys are retrieved from the cluster nodes before it returns control to the caller.

Some clients, such as the C client, deliver each record as soon as it arrives, allowing client applications to process data from the fastest server first. In the Java client, sync batch and async batch with RecordArrayListener wait until all responses arrive from the cluster nodes. Async batch calls the RecordSequenceListener to send back records one at a time as they are received.

Batch considerations

  • Batches are not transactions - they are neither atomic nor isolated. A batch is a group of commands that execute in parallel, saving round-trip time (RTT) and potentially making more efficient use of network resources, for example when batching many commands with small payloads. A batch can be part of a transaction by setting a transaction ID (Txn) in the BatchPolicy.txn.
  • There is no guarantee for the order of execution across nodes. The commands in a sub-batch can be processed in order, a behavior controlled by a batch policy flag. Otherwise, the commands in a sub-batch execute in parallel on the node.
  • There is no rollback processing for failed commands in a batch.
  • You can configure the batch policy to stop or continue batch processing if an operation fails.
  • Use the operate command to combine multiple changes with the same key into a single batch entry.
  • Starting with Database 6.0.0, you can combine any type of read and write commands against one or more keys.
  • Batches use more resources than end-to-end (client and server) read/write commands unless batches are really large. We recommend that you use read/write commands with smaller batch sizes.

Batch commands

Inlining batches

If inline is set to true through flags in the batch policy, the node executes every operation in its sub-batch, one after the other.

  • Batch policy allowInline controls whether to inline sub-batch commands where the keys are in an in-memory namespace. The default value is true.
  • Batch policy allowInlineSSD controls whether to inline sub-batch commands where the keys are in an SSD based namespace. The default value is false.
  • For batch commands with smaller records, for example, 1KiB per record, using inline processing is faster for in-memory namespaces.
  • If it is not inline, the sub-batch commands are split and executed by different threads.
  • Inlining sub-batches does not tend to improve latency when the keys are in an SSD-based namespace. You should benchmark to compare performance.
  • When a sub-batch is inlined, one thread executes the commands. The thread is not released until the sub-batch processing is complete. Large inlined batches may divert server resources toward batch commands over single-record commands.

Filtering batch commands

You can attach a record filter expression to any batched commands. The server applies the filter to each record in the batch to determine whether the operation should proceed.

Multiple batch commands to the same key in a batch

  • Unless you inline the batch request to be serviced by the same service thread, multiple batch ops to same key, such as [K1, K3…K1, K5…K1, K2], will get distributed to different service threads.
  • Client library or the server cannot decipher your intent of commands on K1 and consolidate them into one batch command on K1.
  • Each operation on K1 can proceed, potentially, out of order when distributed to different service threads.

If you don’t want this situation, either consolidate commands for the same key into one key entry in the batch, or inline the batch request.

Code examples of batch commands

The following examples demonstrate batch write, exists, operate, read, and delete commands. They use a set of records with a string bin and an integer bin:

BinTypeValue
namestring"Alice", "Bob", "Carol", "Dave", "Erin"
visitsinteger1

Each section below shows an excerpt. See the Code block at the bottom of the page for the full, continuous program in each language.

Batch write

Batch writes send multiple write commands in a single network round-trip. Requires Database 6.0+. Each entry in the batch is a per-key write record that carries its own operations.

Same write to multiple keys

When every key receives the same bins, build one write entry per key with the same operations and that key’s values. This is the common pattern for seeding or initializing uniform data across many keys.

DataSet demo = DataSet.of("test", "demo");
List<Key> keys = demo.ids("user1", "user2", "user3", "user4", "user5");


session.upsert(demo)
    .bins("name", "visits")
    .id("user1").values("Alice", 1)
    .id("user2").values("Bob", 1)
    .id("user3").values("Carol", 1)
    .id("user4").values("Dave", 1)
    .id("user5").values("Erin", 1)
    .execute();

Different writes per key

When each key needs different bins or values, each write entry carries its own operations. This example writes two completely different records in one batch call.

session
    .upsert(demo.id("sensor1"))
        .bin("type").setTo("temperature")
        .bin("reading").setTo(22.5)
    .upsert(demo.id("alert1"))
        .bin("level").setTo("warning")
        .bin("message").setTo("High temp")
    .execute();

Batch exists

Check whether keys exist. Returns metadata only, without bin data. In Java, Go, and C#, getHeader is a variant that returns record metadata (generation and TTL) without reading bins.

try (RecordStream rs = session.exists(keys).includeMissingKeys().execute()) {
    rs.forEach(result -> System.out.printf("Key %s: exists=%s%n",
        result.getKey().userKey, result.asBoolean()));
}

Batch operate

Execute read and write operations against multiple keys using the operate command. Each key’s operations apply atomically, as in a single-record operate; the batch as a whole does not. Read results use the same projection model as single-record operate (bin projection and operation projection). Foreground queries support operation projection from Database 8.1.2 and later.

The following example increments visits and reads back the updated values.

try (RecordStream rs = session.upsert(keys)
        .bin("visits").add(1)
        .bin("name").get()
        .bin("visits").get()
        .execute()) {
    rs.forEach(result -> {
        Record rec = result.recordOrThrow();
        // visits holds a result per operation: none for add, then get's value
        System.out.printf("%s: visits=%s%n",
            rec.getString("name"), rec.getList("visits").get(1));
    });
}

Batch read

Read records, all bins or a projection, for a list of keys.

try (RecordStream rs = session.query(keys).readingOnlyBins("name", "visits").execute()) {
    rs.forEach(result -> {
        Record rec = result.recordOrThrow();
        System.out.printf("%s: visits=%d%n",
            rec.getString("name"), rec.getInt("visits"));
    });
}

Batch delete

Delete multiple records by key.

RecordStream failures = session.delete(keys).execute().failures();


if (!failures.hasNext()) {
    System.out.println("All records deleted.");
}

Code block

Expand for the full batched commands example 
import com.aerospike.client.sdk.Cluster;
import com.aerospike.client.sdk.ClusterDefinition;
import com.aerospike.client.sdk.DataSet;
import com.aerospike.client.sdk.Key;
import com.aerospike.client.sdk.Record;
import com.aerospike.client.sdk.RecordStream;
import com.aerospike.client.sdk.Session;
import com.aerospike.client.sdk.policy.Behavior;
import java.util.List;


public class BatchExample {
    public static void main(String[] args) {
        // Connect
        try (Cluster cluster = new ClusterDefinition("127.0.0.1", 3000).connect()) {
            Session session = cluster.createSession(Behavior.DEFAULT);


            // Batch write
            DataSet demo = DataSet.of("test", "demo");
            List<Key> keys = demo.ids("user1", "user2", "user3", "user4", "user5");


            session.upsert(demo)
                .bins("name", "visits")
                .id("user1").values("Alice", 1)
                .id("user2").values("Bob", 1)
                .id("user3").values("Carol", 1)
                .id("user4").values("Dave", 1)
                .id("user5").values("Erin", 1)
                .execute();


            // Batch exists
            try (RecordStream rs = session.exists(keys).includeMissingKeys().execute()) {
                rs.forEach(result -> System.out.printf("Key %s: exists=%s%n",
                    result.getKey().userKey, result.asBoolean()));
            }


            // Batch operate
            try (RecordStream rs = session.upsert(keys)
                    .bin("visits").add(1)
                    .bin("name").get()
                    .bin("visits").get()
                    .execute()) {
                rs.forEach(result -> {
                    Record rec = result.recordOrThrow();
                    // visits holds a result per operation: none for add, then get's value
                    System.out.printf("%s: visits=%s%n",
                        rec.getString("name"), rec.getList("visits").get(1));
                });
            }


            // Batch read
            try (RecordStream rs = session.query(keys).readingOnlyBins("name", "visits").execute()) {
                rs.forEach(result -> {
                    Record rec = result.recordOrThrow();
                    System.out.printf("%s: visits=%d%n",
                        rec.getString("name"), rec.getInt("visits"));
                });
            }


            // Batch delete
            RecordStream failures = session.delete(keys).execute().failures();


            if (!failures.hasNext()) {
                System.out.println("All records deleted.");
            }
        }
    }
}

Log examples

Batch information coming from batch-sub. Stats on commands including batches for the namespace called test.

Ticker log line changes:

{test} batch-sub: tsvc (0,0) proxy (0,0,0) read (959,0,0,51,1) write (0,0,0,0) delete (0,0,0,0,0) udf (0,0,0,0) lang (0,0,0,0)

When the cluster size changes you might also see proxy events included in a batch.

{test} from-proxy-batch-sub: tsvc (0,0) read (959,0,0,51,1) write (0,0,0,0) delete (0,0,0,0,0) udf (0,0,0,0) lang (0,0,0,0)

Batch specific errors

ValueErrorDescription
150AS_ERR_BATCH_DISABLEDBatch functionality has been disabled by configuring batch-index-threads to 0
152AS_ERR_BATCH_QUEUES_FULLAll batch queues are full. Controlled by the batch-max-buffers-per-queue configuration parameter

Refer to Error Codes.

Known limitations

Batch writes were not supported prior to the Database 6.0.0, see the Client Matrix.

Client references

Refer to these topics for language-specific code examples: