Incompatible API changes

Version 16.0.2

[CLIENT-3324] Transactions: remove redundant commit statuses that are already returned as AerospikeError exceptions

These constants have been removed, because status responses are returned only if commit() is successful.

aerospike.COMMIT_VERIFY_FAILED
aerospike.COMMIT_MARK_ROLL_FORWARD_ABANDONED

Version 16.0.1

[CLIENT-3272] Rename MRT_STATE* constants to TXN_STATE*

STATE constants have been renamed as indicated below:

Original name	Updated name
MRT_STATE_OPEN	TXN_STATE_OPEN
MRT_STATE_VERIFIED	TXN_STATE_VERIFIED
MRT_STATE_COMMITTED	TXN_STATE_COMMITTED
MRT_STATE_ABORTED	TXN_STATE_ABORTED

[CLIENT-3272] Remove MRT prefix from MRT_{ABORT,COMMIT}* constants to make consistent with C client

ABORT and COMMIT constants have been renamed as indicated below:

Original name	Updated name
MRT_COMMIT_OK	COMMIT_OK
MRT_COMMIT_ALREADY_COMMITTED	COMMIT_ALREADY_COMMITTED
MRT_COMMIT_VERIFY_FAILED	COMMIT_VERIFY_FAILED
MRT_COMMIT_MARK_ROLL_FORWARD_ABANDONED	COMMIT_MARK_ROLL_FORWARD_ABANDONED
MRT_COMMIT_ROLL_FORWARD_ABANDONED	COMMIT_ROLL_FORWARD_ABANDONED
MRT_COMMIT_CLOSE_ABANDONED	COMMIT_CLOSE_ABANDONED
MRT_ABORT_OK	ABORT_OK
MRT_ABORT_ALREADY_ABORTED	ABORT_ALREADY_ABORTED
MRT_ABORT_ROLL_BACK_ABANDONED	ABORT_ROLL_BACK_ABANDONED
MRT_ABORT_CLOSE_ABANDONED	ABORT_CLOSE_ABANDONED

Version 16.0.0

[CLIENT-3121] Support Multi-Record Commands

aerospike_helpers.metrics.Cluster.tran_count has been renamed to command_count.

Version 15.1.0

[CLIENT-1938] Remove aerospike.version to single source the client’s version

When importing the aerospike module, you can get the installed client version. Before 15.1.0, you can use aerospike.__version__ to get the version string. In 15.1.0, the new way to get the version string is importlib.metadata.version:

from importlib.metadata import version
version("aerospike")

Version 15.0.0

[CLIENT-1824] Add HyperLogLog class to represent HLL values

When HyperLogLog server values are returned, they are returned as aerospike_helpers.HyperLogLog instances instead of Python bytes objects. Note that aerospike_helpers.HyperLogLog is a subclass of bytes.

[CLIENT-2444] Only string, integer, and bytes are supported map key types

If any invalid type is used as a map key (i.e any type other than string, integer, or bytes) in an operation, the operation fails regardless of the server version used. This restriction on map key types is enforced by the client.

Deprecations

• In query policies, the "short_query" option has been deprecated in favor of the "expected_duration" option.

Version 14.0.0

[CLIENT-2405] Remove Python 3.7 support

Update your Python to any version between 3.8 and 3.11 inclusive to use this Python client release.

[CLIENT-2654] Remove Debian 10 support

Please update your distribution to a newer version of Debian that is supported by the Python client. Debian 11 and 12 is supported by Python client 14.0.0.

[CLIENT-2645] client.batch_operate() now takes in an optional ttl parameter instead of taking a ttl value through a batch write policy

Old Behavior

In the previous release, you are able to pass in a command-level batch write policy with a ttl option to set each record’s ttl in client.batch_operate().

Course of action

In this Python client release, this behavior no longer works. You must pass in the TTL value to the optional ttl parameter in client.batch_operate().

Background

This was changed because for CLIENT-2645, the C client added the batch write policy’s ttl option to be set in a client-level batch write policy (see the 14.0.0 release notes). To support both client-level and command-level batch write policy TTLs, the ttl parameter in client.batch_operate() was added for setting a command-level TTL.

[CLIENT-2604] Remove “map_write_mode” option in map policy in favor of “map_write_flags”

Old Behavior

The map write mode option is only usable in server versions < 4.3.0, and has been replaced by the map write flags option.

Course of action

Upgrade to server version 4.3.0 or greater. Replace map write mode with map write flags. Refer to the linked documentation above for those two options.

Versions 13.0.0, 11.2.0, 7.2.0, and 6.2.0

[CLIENT-2258] Remove auto-serialization and auto-deserialization

This client has removed support for these features because of the CWE-502 vulnerability.:

1. Storing / serializing AS_BYTES_PYTHON type values in the server
2. Deserializing AS_BYTES_PYTHON type values from the server

Old Behavior

By default, when the Python client receives an unsupported Python type not listed in the table here, it tries to serialize the unsupported type using the pickle library that comes with the standard library. The value is encoded on the server as a bytearray of type AS_BYTES_PYTHON. Then, when the Python client fetches data from the server that has the type AS_BYTES_PYTHON, it uses the pickle library to deserialize the data back into a Python object.

Issue

There is a possibility that malicious Python code can be encoded into an AS_BYTES_PYTHON value in the server, and an application can fetch that data which would cause the pickle library to deserialize the data and run arbitrary code. (see the documentation for the pickle library)

Course of action

If your application stores AS_BYTES_PYTHON values in an Aerospike server, there are two steps you must take to address this vulnerability:

Step 1: Replace all AS_BYTES_PYTHON values in the server using an older client version

The application must replace any unsupported Python types serialized with AS_BYTES_PYTHON with supported types instead. For instance, if the application used an older client version (< 13.0.0) to store a class in the server:

class Foo:
    a: int
    b: str

    def __init__(self, a: int, b: str):
        self.a = a
        self.b = b

client.put(key, {"foo": Foo(42, "welcome")})

That class instance would be serialized in the server as an AS_BYTES_PYTHON type. Use a client version that still supports deserializing AS_BYTES_PYTHON types to read this record so that it’s deserialized back into a class instance in Python. Replace the class instance with a supported Python type, and replace the record in the server:

_, _, record = client.get(key)
foo_instance = record["foo_instance"]
foo_replacement = {
    "a": foo_instance.a,
    "b": foo_instance.b
}
client.put(key, {"foo": foo_replacement})

It is also possible that booleans are stored in the server as AS_BYTES_PYTHON types. To ensure the client does not restore the booleans in the server as AS_BYTES_PYTHON again, verify that the send_bool_as option in the client configuration is set to a constant other than PY_BYTES.

You can use the blob finder tool to find all the language specific blobs in your server. Call ./blobfinder -u to print the command usage of the tool:

jnguyen@as-PF3WN8Y6:~$ ./blobfinder -o blobs.txt
Scan test
recs=2 jblobs=0 cblobs=0 pblobs=2
jnguyen@as-PF3WN8Y6:~$ cat blobs.txt
test,demo,ae6a515f37de4adacc4496e2da1fea848c3a9270,a,pblob
test,demo,b7f4b83889e2da67de683e1df6919a1eacc446c8,a,pblob

Once all AS_BYTES_PYTHON values in the server have been replaced, move on to the next step to update the client version and any existing code that may rely on auto-serialization or auto-deserialization.

Step 2: Update application code to use new client version

This section describes changes to the API that removes support for serializing and deserializing AS_BYTES_PYTHON values, as well as how to update your application code accordingly.

PY_BYTES option for client config send_bool_as has been removed

It is no longer possible to send Python booleans to the server as AS_BYTES_PYTHON types.

Previously, the send_bool_as constants here had these values:

aerospike.PY_BYTES = 0
aerospike.INTEGER = 1
aerospike.AS_BOOL = 2

After removing aerospike.PY_BYTES, the new values of these constants are:

aerospike.INTEGER = 0
aerospike.AS_BOOL = 1

Course of action: If your application sends Python booleans to the server with send_bool_as set to PY_BYTES, you must change it so the application sends booleans as the same server type you chose in step 1 to replace the old booleans.

Backports only (11.2.0, 7.2.0, and 7.2.0): aerospike.Client(): client config option send_bool_as’s default value has been changed to aerospike.AS_BOOL.

client.put(): aerospike.SERIALIZER_PYTHON option has been removed

Scenario: client.put()’s serializer parameter is set to aerospike.SERIALIZE_PYTHON.

Old behavior: client.put() would use the pickle library to serialize unsupported types.

New behavior: A runtime error would occur at the line calling client.put(). Note that the other serialization constants’ integer values did not change.

Course of action:

You must serialize the unsupported type yourself using either:

1. An instance-level serializer in the client config.
2. A module-level serializer with client.put()’s serializer parameter set to aerospike.SERIALIZE_USER.

client.put(): serializer parameter’s default value has been changed to aerospike.SERIALIZER_NONE

Scenario:

1. A module-level serializer was set, and a client instance-level serializer was not set.
2. client.put()’s serializer parameter is not set to anything.

Old behavior: client.put() uses the pickle library to serialize unsupported types.

New behavior: client.put() fails to serialize unsupported types and throws a ClientError (-1).

Course of action:

Provide your own serializer by specifying:

1. A instance-level serializer.
2. A module-level serializer with client.put()’s serializer parameter set to aerospike.SERIALIZE_USER.

Operations using client.operate() and expressions

Old behavior: When passing unsupported Python types into operations, the client uses the pickle library by default to serialize or deserialize them.

New behavior:

Instead of serializing unsupported types using the pickle library, the client by default throws a ClientError (-1) saying that unsupported Python types cannot be serialized.

Course of action: The application can override the default behavior by providing its own instance-level serializer in the client configuration.

Reading AS_BYTES_PYTHON values from the server

Old behavior: The client deserializes these values to their original Python objects by default.

New behavior: The client returns the raw bytes of the serialized object as a bytearray. This prevents exceptions from being thrown when a client comes across an AS_BYTES_PYTHON value from the server, which may potentially break an application if the exception isn’t handled. A future server release will remove support for AS_BYTES_PYTHON.

Course of action: In case there are any leftover AS_BYTES_PYTHON values in the server after upgrading your application to use client version 13.0.0, you must use cPickle in your application code to deserialize the bytearray in order to get the original Python object.

# Retrieve a bin containing an AS_BYTES_PYTHON value
>>> _, _, bins = client.get(("test", "demo", 2))
>>> bins["a"]
bytearray(b'\x80\x04\x95\t\x00\x00\x00\x00\x00\x00\x00K\x01K\x02K\x03\x87\x94.')
# Deserialize with cPickle
>>> import pickle
>>> pickle.loads(bins["a"])
(1, 2, 3)

NOTE: Using cPickle to deserialize data isn’t recommended in general because of the CWE-502 vulnerability, so it is best to convert this data into supported types for the Python client or to use your own serializer and deserializer.

Version 12.0.0

Remove deprecated methods

The following methods have been replaced / removed either to clean up the API or replace with newer methods:

Old	New	Reason for removing
aerospike.Client.info()	aerospike.Client.info_all()	Use info_all() to send a command to all cluster nodes, and info_single_node() to target a specific cluster node. Sending info commands to specific nodes is a rare use case; you normally only need to send info commands to all cluster nodes or any non-specific node.
aerospike.Client.get_key_digest()	aerospike.calc_digest()	get_key_digest() should be usable without a client connection.
aerospike.Client.scan_info()	aerospike.Client.job_info()	scan_info() uses the deprecated info command “scan-show”.
aerospike.Client.info_node()	aerospike.Client.info_single_node()	info_node() calls a deprecated C client method “aerospike_info_host()” under the hood.
aerospike.Client.list_*()	aerospike_helpers.operations.list_operations	The old list operation functions cannot be used in client.operate(), so they cannot be used to perform multiple operations atomically on a single record, whereas you can do this with client.operate() and the list operations in aerospike_helpers. It’s also easier to maintain one set of list operation functions instead of two, so keeping the latter is more convenient. Lastly, the old list operation functions didn’t support accessing nested elements in CDTs using contexts.
aerospike.Client.map_*()	aerospike_helpers.operations.map_operations	The old map operations follow the same reasoning for removal of the old list operations. Refer to the reason for removing for the old list operations above.

Replacing aerospike.Client.info()

If sending an info command to all cluster nodes, the outputs of both functions should be the same. Assuming the outputs of both function examples are for a 3 node cluster, change this:

client.info(command="build")
# {'BB9040011AC4202': (None, '6.3.0.6\n'), 'BB9020011AC4202': (None, '6.3.0.6\n'), 'BB9030011AC4202': (None, '6.3.0.6\n')}

To this instead:

client.info_all(command="build")
# {'BB9040011AC4202': (None, '6.3.0.6\n'), 'BB9020011AC4202': (None, '6.3.0.6\n'), 'BB9030011AC4202': (None, '6.3.0.6\n')}

In the following example client.info() was called with an explicit list of hosts:

# Send info command to two nodes in a three node cluster
node_hosts = [
    ("172.17.0.2", 3000),
    ("172.17.0.3", 3000)
]
response = client.info(command="build", hosts=node_hosts)
print(response)
# For each node, the first item in the tuple is an error tuple containing exception info that was raised for that node
# And the second item in the tuple is the response from the server
# {'BB9020011AC4202': (None, '6.3.0.5\n'), 'BB9030011AC4202': (None, '6.3.0.5\n')}

This alternative code snippet makes use of Client.info_single_node() instead:

node_hosts = [
    ("172.17.0.2", 3000),
    ("172.17.0.3", 3000)
]
# Get node names of the above hosts
all_nodes_info = client.get_node_names()
print(all_nodes_info)
# [{'address': '172.17.0.2', 'port': 3000, 'node_name': 'BB9020011AC4202'},
# {'address': '172.17.0.4', 'port': 3000, 'node_name': 'BB9040011AC4202'},
# {'address': '172.17.0.3', 'port': 3000, 'node_name': 'BB9030011AC4202'}]
node_names = []
for node_info in all_nodes_info:
    if (node_info["address"], node_info["port"]) in node_hosts:
        node_names.append(node_info["node_name"])
print(node_names)
# ['BB9020011AC4202', 'BB9030011AC4202']

response = {}
for node_name in node_names:
    try:
        node_response = client.info_single_node(command="build", host=node_name)
        response[node_name] = node_response
    except Exception as e:
            #  Handle the error gracefully here
            response[node_name] = e
print(response)
# Instead of mapping each node name to a tuple, it is mapped to either an Exception object if an error was raised in a specific node
# or a string containing the command and response
# {'BB9020011AC4202': 'build\t6.3.0.5\n', 'BB9030011AC4202': 'build\t6.3.0.5\n'}

Replacing aerospike.Client.get_key_digest()

>>> client.get_key_digest(ns="test", set="demo", key=1)
bytearray(b'\xb7\xf4\xb88\x89\xe2\xdag\xdeh>\x1d\xf6\x91\x9a\x1e\xac\xc4F\xc8')
>>> aerospike.calc_digest(ns="test", set="demo", key=1)
bytearray(b'\xb7\xf4\xb88\x89\xe2\xdag\xdeh>\x1d\xf6\x91\x9a\x1e\xac\xc4F\xc8')

For aerospike.calc_digest(), you can also pass in a bytearray as an argument for the key parameter, in addition to an int or str value. This was not originally available in aerospike.Client.get_key_digest().

Replacing aerospike.Client.scan_info()

The following example uses Client.scan_info():

client.udf_put("./my_udf.lua")

job_id = client.scan_apply("test", "demo", "my_udf", "my_udf", ["number", 10])

scan_status = client.scan_info(job_id)
print(scan_status)
# {'progress_pct': 100, 'records_scanned': 0, 'status': 3}

The alternative code sample uses Client.job_info() instead:

client.udf_put("./my_udf.lua")

scan = client.scan("test", "demo")
scan.apply("my_udf", "my_udf", ["number", 10])
job_id = scan.execute_background()

scan_status = client.job_info(job_id, aerospike.JOB_SCAN)
print(scan_status)
# {'progress_pct': 100, 'records_read': 0, 'status': 2}

Note that for records read, client.job_info() returns this value with the key "records_read" instead of "records_scanned".

The constants returned by the two methods are also different.

The scan constants aerospike.SCAN_* (docs) have been replaced with job constants aerospike.JOB_* (docs). In our example above, aerospike.SCAN_STATUS_COMPLETED (constant value of 3) should be replaced with aerospike.JOB_STATUS_COMPLETED (constant value of 2).

Replacing aerospike.Client.info_node()

Assume we want to send an info command to a specific node in a three node cluster.

The following example uses Client.info_node():

response = client.info_node(command="build", host=("172.17.0.2", 3000))
print(response)
# build  6.3.0.5
#

The alternative code sample uses Client.info_single_node() instead:

cluster_info = client.get_node_names()
print(cluster_info)
# [{'address': '172.17.0.2', 'port': 3000, 'node_name': 'BB9020011AC4202'},
# {'address': '172.17.0.3', 'port': 3000, 'node_name': 'BB9030011AC4202'},
# {'address': '172.17.0.4', 'port': 3000, 'node_name': 'BB9040011AC4202'}]

# Get node name for 172.17.0.2
node_name = None
for node_info in cluster_info:
    if node_info["address"] == "172.17.0.2" and node_info["port"] == 3000:
        node_name = node_info["node_name"]
print(node_name)
# ['BB9020011AC4202']

if node_name == None:
    # TODO: handle case where socket address doesn't point to a node in the cluster
    pass
response = client.info_single_node(command="build", host=node_name)
print(response)
# build  6.3.0.5
#

Replacing aerospike.Client.list_*() and aerospike.Client.map_*() operations

When performing operations, we need to pass the operations into client.operate().

The following example uses Client.list_get():

list_value = client.list_get(key, "bin_name", 0)
print(list_value)
# "asdf"

The alternative code sample uses the list_operations helper:

from aerospike_helpers.operations import list_operations as list_ops

ops = [
    list_ops.list_get("bin_name", 0)
]
recs = client.operate(key, ops)
print(recs[2])
# {"bin_name": "asdf"}

See the documentation for client.operate() for more details.

Remove deprecated attributes or constant values

Location	Old	New	Reason for removing
Batch policy	"send_set_name"	None	The client ignores this policy and always sends set name to the server.
Scan policy	"fail_on_cluster_change"	None	No longer has any effect. Abort the scan if the cluster is not in a stable state. Only used for server versions < 4.9.
Client config	"timeout"	"total_timeout"	"timeout" is a duplicate of the "total_timeout" policy
aerospike module	aerospike.SCAN_*	aerospike.JOB_*	The scan constants were previously used by client.scan_info() to report the status of a scan, and those constants were removed in this release. client.job_info() should be used instead, and it uses the aerospike.JOB_* constants to report the job status.

timeout

There are two changes to note:

1. Previously, if config["timeout"] was set in the client config dictionary like this:

config = {
    "timeout": 1000
}
client = aerospike.client(config)

It would set "total_timeout" to config["timeout"] for these global policies (policies which would be applied to every command):

• read
• write
• apply
• operate
• query
• scan
• remove
• batch

In this client version, "timeout" can no longer be set in the client config, but you can maintain the same behavior by replacing that dictionary key with "total_timeout".

2. "timeout" can no longer be set in dictionaries for the above policies either. For instance if you have a read policy:

read_policy = {
    "timeout": 1000
}

It must be replaced with the proper policy "total_timeout":

read_policy = {
    "total_timeout": 1000
}

NOTE: you must use "timeout" for individual admin and info policies. Admin and info policy dictionaries do not use "total_timeout".

Version 11.0.1

[CLIENT-2267] Revert adding base64 methods to aerospike module

These methods have been removed from the aerospike module due to memory errors when they are called:

1. aerospike.get_expression_base64()
2. aerospike.get_cdtctx_base64()

Course of action:

Call these methods from a client instance instead of the aerospike module. Assuming client is an instance of aerospike.Client, the above calls should be replaced by:

1. client.get_expression_base64()
2. client.get_cdtctx_base64()

Version 11.0.0

[CLIENT-701] Batch methods: stop accepting a tuple of keys and bins

For the following functions:

• client.get_many()
• client.exists_many()

The keys parameter no longer takes in a tuple of keys. It only takes in a list of keys.

In addition, client.select_many() no longer takes in a tuple for the keys and bins parameters. Those parameters only take in a list of keys and bins, respectively.

Course of action:

Change code such as this:

keys = (("test", "demo", 1), ("test", "demo", 2))
bins = ("bin1", "bin2")
client.select_many(keys, bins)

…to this instead:

keys = [("test", "demo", 1), ("test", "demo", 2)]
bins = ["bin1", "bin2"]
client.select_many(keys, bins)

[CLIENT-2144] Expressions: add support for comparing KeyOrderedDicts (new server feature)

Before server 6.3, it is possible to compare maps in expressions if the maps were nested inside a list. For example, this code would work before server 6.3:

from aerospike_helpers.expressions import base as expr

client.put(key, {"bin": [{"a": 1}]})
exp = expr.Eq(expr.ListBin("bin"), [{"a": 1}]).compile()
record = client.get(key, {"expressions": exp})
print(record[2])
# {'bin': [{'a': 1}]}

This is now unsupported in server 6.3 because comparing unordered maps can potentially lead to inconsistent results. However, it is possible in server 6.3 to compare key-ordered maps in expressions.

Course of action:

For those using a server version < 6.3, no action is necessary. But it is recommended not to compare unordered maps in expressions.

For those upgrading to server 6.3, maps stored in the server must be key-ordered in order to be compared against in expressions. If the maps in the server are already key-ordered, and you would like to compare them in expressions, you must wrap any dictionaries in expressions with the KeyOrderedDict class.

For example, the code above must store the map as a key ordered map before comparing it in an expression:

from aerospike_helpers.expressions import base as expr
from aerospike import KeyOrderedDict

client.put(key, {"bin": [KeyOrderedDict({"a": 1})]})
exp = expr.Eq(expr.ListBin("bin"), [KeyOrderedDict({"a": 1})]).compile()
record = client.get(key, {"expressions": exp})
print(record[2])
# {'bin': [{'a': 1}]}

[CLIENT-2197] Return AEROSPIKE_ERR_NAMESPACE_NOT_FOUND instead of AEROSPIKE_ERR_CLIENT when a namespace cannot be found

Course of action: Change code such as this:

from aerospike import exception as exc
key = ("nonexistent_namespace", "demo", 1)
try:
    client.get(key)
except exc.ClientError:
    print("Incorrect namespace")

…to this instead:

from aerospike import exception as exc
key = ("nonexistent_namespace", "demo", 1)
try:
    client.get(key)
except exc.NamespaceNotFound:
    print("Incorrect namespace")

[CLIENT-2143] Return last error code received when scan/query maxRetries is exceeded

When running a query or scan, if max_retries is exceeded, the command will return the last suberror that was received instead of a MaxRetriesExceeded error. For example, if you try to run a query on a non-indexed bin, the client will return an IndexNotFound error from the last attempt to query the bin.

This code will no longer work:

query = client.query("test", "demo")
query.select("bin_without_index")
query.where(p.equals("bin_without_index", 1))
def callback(input_tuple):
    pass

try:
    query.foreach(callback)
except exc.MaxRetriesExceeded:
    print("Query failed")

Course of action:

When handling a MaxRetriesExceeded exception, change it to the exact error that is expected to get thrown during the last query attempt. In this case, it is an IndexNotFound error:

try:
    query.foreach(callback)
except exc.IndexNotFound:
    print("Query failed")

Version 10.0.0

[CLIENT-1989] get_cdtctx_base64(): take in context directly instead of a dictionary.

When passing in a context array to get_cdtctx_base64(), you must to pass it in directly as a parameter instead of embedding it in a dictionary.

This change is to make the API more easy to use and consistent with the other API methods.

Course of action:

This code:

get_cdtctx_base64({'ctx': ctx})

should be changed to:

get_cdtctx_base64(ctx)

Version 9.0.0

[CLIENT-2074] Change default send_bool_as constant to AS_BOOL

By default, the client now sends the boolean as an Aerospike server boolean type (aerospike.AS_BOOL) instead of a Python boolean type (aerospike.PY_BYTES).

For developers using server 5.6 and later

The server encodes Python booleans as the “blob” type and server booleans as the “booleans” type. Because of the ordering of types in a Collection Data Type (CDT), this may break rank operations for CDTs that store a mix of booleans and other types of values.

Course of action: Check that rank operations on CDTs containing both boolean and integers are handled properly.

For developers using server version earlier than 5.6

The server will fail to perform the write operation if the client uses the default send_bool_as value, aerospike.AS_BOOL, to write a boolean value to the server.

Course of actions:

1. Change the default send_bool_as to a value other than aerospike.AS_BOOL.
2. Upgrade the server to version 5.6 to support server-native boolean types.

[CLIENT-2008] batch_get_ops(): Remove meta field

batch_get_ops() previously had this function signature:

batch_get_ops(keys, ops[, meta, policy: dict]) → [key, meta, bins]

Now the meta parameter has been removed:

batch_get_ops(keys, ops[, policy: dict]) → [key, meta, bins]

[CLIENT-2012] scan_apply(): Report correct error value and message if scan wait fails

When scan_apply() is called and it internally calls scan_wait() and fails, it previously returned this error:

"error": "(-2, 'Unable to perform scan_wait on the scan', 'src/main/client/scan.c', 215, False)"

Now it returns an error from scan_wait() directly in order to identify the cause of the error.

Version 8.0.0

Connect when calling client constructor and conditionally no-op connect()

• Calling the aerospike.Client constructor establishes the connection.
  • If user authentication is required, pass username and password in the client configuration dictionary with the keys 'user' and 'password'.
• aerospike.Client.connect() does nothing unless the connection was closed beforehand by calling close().

Before version 8.0.0:

config = {
    'hosts': hosts,
    'policies': {'auth_mode': aerospike.AUTH_INTERNAL},
}
client = aerospike.client(config)
client.connect(user, password)

At version 8.0.0:

config = {
    'hosts': hosts,
    'policies': {'auth_mode': aerospike.AUTH_INTERNAL},
    'user': user,
    'password': password
}
client = aerospike.client(config)
# following is no-op
client.connect(user, password)

Having the client connect to the server when the constructor is called makes the Python client’s behavior more consistent with our other clients, and it also removes the possibility of an application trying to perform server operations with the client without a connection.

If you are using a try/except block and have the constructor call outside of the try block, move the constructor call into the try block. If an exception happens, it would be coming from the constructor and not the connect() method.

This line of code:

client = aerospike.client(config).connect()

should not have an issue.

But code such as this:

client = aerospike.client(config)
try:
    client.connect()
except Exception:
    # eat exception and do something

Should be changed to this instead:

try:
    client = aerospike.client(config)
except Exception:
    # eat exception and do something

Version 7.0.0

Remove old predexp

Old predexp functionality has been removed. Code using old predexp must be converted to the new expression syntax. See Version 7.0.0 - Replace predicate expressions with Aerospike expressions

Removed support for Debian 8

Debian 8 has reached its end of life. Python client installs on debian 8 may fail.

IndexNotFound and IndexNotReadable errors can now trigger retries

Previously these errors would cause commands to fail immediately. Now they trigger retries if that command’s policy allows for them. If your application uses commands with retries and checks for the IndexNotFound or IndexNotReadable errors, switch to checking for the MaxRetriesExceeded error instead.

Bytes blobs read from the database will now be read as bytes instead of bytearray

Bytes and Bytearray objects written by the Python client to Aerospike are stored on the server as bytes blobs. Previously, these bytes blobs were always read as bytearrays. When bytes were written as dictionary keys, they would be read as bytearrays causing an unhashable key error. If your application reads bytes blobs and type checks the result expecting bytearray, the expected type should change to bytes. If your application relies on bytearray results being mutable and encounters errors like “TypeError: ‘bytes’ object does not support item assignment” the result has probably switched to bytes. bytes objects are not mutable so should not be directly modified.”

Query policies max_records and records_per_second are now fields of the Query class

Query policies max_records and records_per_second have been moved from the query policy to be fields of the Query class. max_records and records_per_second policies will be ignored, so if your code uses them, change to Query.max_records and Query.records_per_second instead.

Version 7.0.0 - Replace predicate expressions with Aerospike expressions

The old predexp functionality has been replaced by Expressions. Expressions include all previous predicate expression functionality (with a different, improved syntax) plus support for record metadata expressions and list/map/bit/hyperloglog expression methods analogous to those used in operations.

See the Python client expressions documentation for more details.

Use of the new expressions API require server version 5.2.0.4 or later. Convert code using old predexp to the new syntax. Here is an example:

import aerospike
from aerospike_helpers import expressions as exp
from aerospike import predexp

# (c >= 11 and c <= 20) or (d >= 3 and d <= 5)
# Using old removed predexp.
predexps = [
    predexp.integer_bin("c"),
    predexp.integer_value(11),
    predexp.integer_greatereq(),
    predexp.integer_bin("c"),
    predexp.integer_value(20),
    predexp.integer_lesseq(),
    predexp.predexp_and(2)
    predexp.integer_bin("d"),
    predexp.integer_value(3),
    predexp.integer_greatereq(),
    predexp.integer_bin("d"),
    predexp.integer_value(5),
    predexp.integer_lesseq(),
    predexp.predexp_and(2),
    predexp.predexp_or(2)
]

# Converted to new expressions.
expression = exp.Or(
    exp.And(
        exp.GE(exp.IntBin("c"), 11),
        exp.LE(exp.IntBin("c"), 20)
    ),
    exp.And(
        exp.GE(exp.IntBin("d"), 3),
        exp.LE(exp.IntBin("d"), 5)
    )
).compile()

Version 6.0.0

Info_node() no longer works when security is enabled

• Because of changes in client authentication internals in C client 5.2.0, info_node() will fail if used when authentication is enabled. Checkout info_single_node() for a replacement.

Dropped support for scan/query options percent, priority, fail_on_cluster_change

• These options have been dropped and are no longer sent to the server. They will need to be removed from application code.

Dropped support for CentOS 6 and Ubuntu 16.04

• CentOS 6 and Ubuntu 16.04 have reached their support end of life. Aerospike Python client installs on CentOS6 and Ubuntu 16.04 will fail.

Python 3.5 Support Discontinued

• Python 3.5 has reached end of life. The Aerospike Python client will no longer be distributed for Python 3.5.

Version 5.0.0

Aerospike Server 4.9 and up Required

• Aerospike Python client 5.0.0 requires Aerospike server 4.9 or later. Attemping to connect to a server older than 4.9 will yield the error “-10, Failed to connect”.

Python 2.7 and 3.4 Support Discontinued

• Python 2.7 and 3.4 have reached end of life. The Aerospike Python client will no longer be distributed for them.

Version 4.0.0

Scan policy changes

• Policy key fail_on_cluster_change has no effect for Aerospike server versions >= 4.9.

Scan option changes

• Option key percent no longer accepts 0 as a valid percent.

Version 3.7.0

Read Consistency Level Changes

Read policy changes for commands involving AP (availability) namespaces

• Policy key consistency_level was replaced by read_mode_ap for the read, batch and, operate policies.
  • POLICY_CONSISTENCY_ALL was replaced by POLICY_READ_MODE_AP_ALL
  • POLICY_CONSISTENCY_ONE was replaced by POLICY_READ_MODE_AP_ONE

Read policy changes for commands involving SC (strong consistency) namespaces

• Policy key linearize_read was replaced by read_mode_sc for the read, batch, and operate policies. Added aerospike.POLICY_READ_MODE_SC_SESSION, aerospike.POLICY_READ_MODE_SC_LINEARIZE, aerospike.POLICY_READ_MODE_SC_ALLOW_REPLICA, and aerospike.POLICY_READ_MODE_SC_ALLOW_UNAVAILABLE as options for this key.

Version 3.6.0

Backwards Incompatible API changes

Shared memory layout change

Shared memory layout has changed, and accordingly the default SHM key has changed from 0xA7000000 to 0xA8000000. If manually specifiying an SHM key, it is crucial to ensure that a separate key is used in order to prevent this version’s client from sharing memory with a previous version.

Removed Unused Exceptions

The BinExistsError, BinNotFound exceptions have been removed.

Version 3.2.0

Additional Features

• Updated to C client 4.3.11
• Added client.map_get_by_value_list and client.map_get_by_key_list These methods require Aerospike Server version >= 3.16.0.1 .

Version 3.1.0

Additional Features

• Updated to C client 4.3.6
• Added exists policy field to operate policies

Backwards Incompatible API changes

• Updated the args passed to AerospikeError constructor internally to contain 5 arguments. The arguments previously were error code, error message, error file, error line. A fifth argument in_doubt has been added to the internal calls. so the arguments passed to the constructor are now : error_code, error_message, error_file, error_line, in_doubt

  This means that code such as the following will now raise a ValueError

try:
    client.get(key)
except AerospikeError as e:
    code, msg, file, line = e.args
    print(code, msg, file, line)

This can be fixed by unpacking the fifth value from the Error’s args tuple

try:
    client.get(key)
except AerospikeError as e:
    code, msg, file, line, in_doubt = e.args
    print(code, msg, file, line)

Version 3.0.0

Additional Features

• Updated to C client 4.3.1
• Added a new list increment operation OP_LIST_INCREMENT . It can be used to increase an element of a list by a provided amount.
• Added the option to specify a specific node to run on a scan on, via a new optional nodename parameter.
• Added the option to specify that Query.results and Query.foreach should not return bins, via a new optional parameter options to both methods.
• Added aerospike.info_all() to allow sending of an info command to all nodes in the current cluster.
• Added linearize_read option to read policies. Requires Enterprise server >= 4.0.0

Deprecations

• client#info has been deprecated. In order to send requests to the entire cluster, the new method client#info_all should be used. In order to send requests to hosts specified in a list, we recommend a loop invoking multiple calls to aerospike.info_node. See below for an example implementation:

def info_to_host_list(client, request, hosts, policy=None):
  output = {}
  for host in hosts:
      try:
          response = client.info_node(request, host, policy)
          output[host] = response
      except Exception as e:
          #  Handle the error gracefully here
          output[host] = e
  return output

• Setting of global policy defaults via top level entries in the ‘policies’ dictionary in the constructor config dict has been deprecated. See the constructor documentation for the new recommended method of specifying defaults.

Backwards Incompatible API changes

LDT Removal

Removed LDT (client#llist) methods and support. Server version 3.14 is the last version of the Aerospike server to support the functionality.

Index Methods

• Methods which create indexes, ( index_string_create, index_integer_create, index_list_create, index_map_keys_create, index_map_values_create and, index_geo2dsphere_create ), will now raise an IndexFoundError if the specified bin has already been indexed, or if an index with the same name already exists.

• Methods which drop indexes (index_remove), will now raise an IndexNotFound if the named index does not exist.

Shared memory layout change

Shared memory layout has changed, and accordingly the default SHM key has changed from 0xA6000000 to 0xA7000000 . If manually specifiying an SHM key, it is crucial to ensure that a separate key is used in order to prevent this version’s client from sharing memory with a previous version.

Changed and removed policy field names:

In all policies, (except for info and admin), timeout has been split into total_timeout and socket_timeout total_timeout is an int representing total command timeout in milliseconds. The total_timeout is tracked on the client and sent to the server along with the command in the wire protocol. The client will most likely timeout first, but the server also has the capability to timeout the command. If total_timeout is not zero and total_timeout is reached before the command completes, the command will return error TimeoutError. If total_timeout is zero, there will be no total time limit. See the documentation for individual policies for the default values.

socket_timeout is an int representing socket idle timeout in milliseconds when processing a database command. If socket_timeout is not zero and the socket has been idle for at least socket_timeout, both max_retries and total_timeout are checked. If max_retries and total_timeout are not exceeded, the command is retried. If both socket_timeout and total_timeout are non-zero and socket_timeout > total_timeout, then socket_timeout will be set to total_timeout. If socket_timeout is zero, there will be no socket idle limit. See the documentation for individual policies for the default values.

retry has beeen removed.

max_retries has been added, it is an integer specifying the number of times a command should be retried before aborting. If max_retries is exceeded, TimeoutError will be raised.

WARNING: Database writes that are not idempotent (such as client#increment) should not be retried because the write operation may be performed multiple times if the client timed out previous command attempts. It’s important to use a distinct write policy for non-idempotent writes which sets max_retries = 0;

The default value for max_retries is 2.

Changes in policy defaults for aerospike.client constructor

In this version, individual config dictionaries (read, write, apply, operate, scan, query, batch, remove) for method types should be used inside of the top level policies dictionary for setting policies. In previous versions, individual policies were set at the top level policies dictionary rather than in per method type dictionaries. The type of policy which affects each method can be found in the documenation. See the main documentation for the keys and values available for these new configuration dictionaries. See below for an example of the change in constructor usage:

#  Pre Python client 3.0.0

hosts = [('localhost', 3000)]

timeout = 2345
key_policy = aerospike.POLICY_KEY_SEND

# This changes defaults for all methods, requiring a config dictionary to be passed in to all methods which
# should use different values.
policies = {timeout': timeout, 'key': key_policy, 'retry': aerospike.POLICY_RETRY_ONCE}
config = {'hosts': hosts, 'policies': policies}

client = aerospike.client(config)

#  Post Python client 3.0.0

hosts = [('localhost', 3000)]

write_total_timeout = 4321
read_total_timeout = 1234
write_key_policy = aerospike.POLICY_KEY_SEND

read_policy = {'total_timeout': read_total_timeout, 'max_retries': 1}
write_policy = {'total_timeout': write_total_timeout, 'key': write_key_policy, 'max_retries': 1}

# operate policies affect methods such as client#increment, so these should not be retried since they are not idempotent.
operate_policy = {'max_retries': 0}

# Change the defaults for read, write, and operate methods, all other methods will use builtin defaults.
policies = {'read': read_policy, 'write': write_policy, 'operate': operate_policy}

config = {'hosts': hosts, 'policies': policies}

client = aerospike.client(config)