Aerospike Basic Operations

For an interactive Jupyter notebook experience:

Basic CRUD (Create, Read, Update, and Delete) operations in Aerospike, and how multiple operations on a record are performed in a single request.

This notebook requires Aerospike database running on localhost and that python and the Aerospike python client have been installed (pip install aerospike). Visit Aerospike notebooks repo for additional details and the docker container.

Ensure the database is running

This notebook requires that Aerospike database is running.

!asd >& /dev/null
!pgrep -x asd >/dev/null && echo "Aerospike database is running!" || echo "**Aerospike database is not running!**"

Output:

Aerospike database is running!

Initialize the client

Initialize the client and connect it to the database.

import aerospike
import sys
config = {
  'hosts': [ ('127.0.0.1', 3000) ]
}
try:
  client = aerospike.client(config).connect()
except:
  import sys
  print("failed to connect to the cluster with", config['hosts'])
  sys.exit(1)
print('Client initialized and connected to database')

Output:

Client initialized and connected to database

Understanding Records in Aerospike

Data in Aerospike consists of records. A record belongs to a namespace (equivalent to a database) and optionally to a set (equivalent to a table). A record has multiple bins (or fields), which are named, strongly-typed containers that hold both atomic (string, integer, bytes) and complex (map, list) data types.

A record has two metadata values:

• generation: the number of times it has been modified
• ttl: seconds remaining until record expiration (default = 0; never expire)

Expired records are garbage-collected by the database. On a write or touch operations, a record's ttl is updated based on the specified policy.

Record Structure

The following code cell illustrates the record structure.

Note:

• key: (namespace, set, user_key, digest), digest is computed from the first three elements.
• metadata: {'gen': generation, 'ttl': ttl}
• bins: key-value pairs of data bins dictionary

And also:

• digest (the bytearray in output) is the actual record unique identifier.
• user_key input to produce digest may or may not be stored with the record, and is governed by the key policy.

#  Record structure
#  key: (namespace, set, user_key, digest), digest is computed from first three values 
#  metadata: {'gen': generation, 'ttl': ttl}
#  bins: key-value pairs of data bins dictionary

namespace = 'test'
demoset = 'demo'
user_key = 'foo'
meta = {'ttl': 0}
bins = {'name': 'John Doe', 'age': 15, 'gpa': 4.3 }
policy = {'key': aerospike.POLICY_KEY_SEND}   # policy to store the user_key along with the record

# insert/update the record
try:
    client.put((namespace, demoset, user_key), bins, meta, policy)
except:
  print('failed to put record')
  sys.exit(1)
print('Successfully wrote the record.')

# read back the record
try:
    (key, metadata, bins)= client.get((namespace, demoset, user_key), policy)
except:
  print('failed to get record')
  sys.exit(1)
print('Successfully read the record.')
print ('Key: ', key)
print ('Metadata: ', metadata)
print ('Bins: ', bins)

Output:

Successfully wrote the record.
Successfully read the record.
Key:  ('test', 'demo', 'foo', bytearray(b'\xf5~\xc1\x835\xf7\x10\x0c\x04X\xf8\xa6D\xbc\xbcvm\x93G\x1e'))
Metadata:  {'ttl': 2592000, 'gen': 1}
Bins:  {'name': 'John Doe', 'age': 15, 'gpa': 4.3}

Writing Records

The Python client's "put" operations is used to write data to the Aerospike cluster.

Defining the Key

As described above, the key tuple serves as the record's unique identifier.

Below we define a key tuple in the set "characters" with the user key "bender" in namespace "test".

# create the key tuple identifying the record
key = ('test', 'characters', 'bender')

Specifying Record Data

Specify record data in a dict, where the top-level object fields represent the bin names, and field values correspond to bin values.

This example writes six bins: name, serialnum, lastsentence, composition, apartment, and quote_cnt.

# The record data to write to the cluster
bins = {
  'name': 'Bender',
  'serialnum': 2716057,
  'lastsentence': {
    'BBS': "Well, we're boned",
    'TBwaBB': 'I love you, meatbags!',
    'BG': 'Whip harder, Professor!',
    'ltWGY': 'Into the breach, meatbags. Or not, whatever'},
  'composition': [ "40% zinc", "40% titanium", "30% iron", "40% dolomite" ],
  'apartment': bytearray(b'\x24'),
  'quote_cnt': 47
}

Storing the Record

Store the record in the database with put.

# Put the record to the database.
try:
    client.put(key, bins)
except:
  print('failed to put record')
  sys.exit(1)
print('Successfully stored the record.')

try:
    (key, metadata, bins) = client.get(key, policy)
except:
  print('failed to get record')
  sys.exit(1)
print ('Bins: ', bins)

Output:

Successfully stored the record.
Bins:  {'name': 'Bender', 'serialnum': 2716057, 'lastsentence': {'ltWGY': 'Into the breach, meatbags. Or not, whatever', 'BG': 'Whip harder, Professor!', 'BBS': "Well, we're boned", 'TBwaBB': 'I love you, meatbags!'}, 'composition': ['40% zinc', '40% titanium', '30% iron', '40% dolomite'], 'apartment': bytearray(b'$'), 'quote_cnt': 47}

Appending, Prepending and Incrementing a Bin

The following APIs are used to prepend and append string bins, and increment integer bins:

try:
    client.prepend(key, 'name', 'Dr. ')
    client.append(key, 'name', ' Bending Rodriguez')
    client.increment(key, 'quote_cnt', 3)
except:
  print('failed to get record')
  sys.exit(1)

try:
    (key, metadata, bins)= client.get(key, policy)
except:
  print('failed to get record')
  sys.exit(1)
print ('Bins: ', bins)

Output:

Bins:  {'name': 'Dr. Bender Bending Rodriguez', 'serialnum': 2716057, 'lastsentence': {'ltWGY': 'Into the breach, meatbags. Or not, whatever', 'BG': 'Whip harder, Professor!', 'BBS': "Well, we're boned", 'TBwaBB': 'I love you, meatbags!'}, 'composition': ['40% zinc', '40% titanium', '30% iron', '40% dolomite'], 'apartment': bytearray(b'$'), 'quote_cnt': 50}

Reading Records

There are multiple ways to read records from Aerospike database, "get" being the simplest one. You will need the key as the record unique identifier as discussed above. Note the record was written above.

It returns:

• key — The key tuple of the record that was read.
• meta — The dict containing the record metadata gen and ttl fields.
• bins — The dict containing the bins of the record.

Meta and bins are None if the record is not found.

# key of the record
key = ('test', 'demo', 'foo')
# Retrieve the record using the key.
try:
    (key, meta, bins) = client.get(key, policy)
except:
  print('failed to get record')
  sys.exit(1)
print('Successfully read the record.')
print ('Key: ', key)
print ('Metadata: ', metadata)
print ('Bins: ', bins)

Output:

Successfully read the record.
Key:  ('test', 'demo', 'foo', bytearray(b'\xf5~\xc1\x835\xf7\x10\x0c\x04X\xf8\xa6D\xbc\xbcvm\x93G\x1e'))
Metadata:  {'ttl': 2592000, 'gen': 4}
Bins:  {'name': 'John Doe', 'age': 15, 'gpa': 4.3}

Projecting the Bins of a Record

It is possible to project or read specific bins of a record. The following example illustrates this. Note that the second argument is a tuple of bin names to project.

# retrieve only specified bins in the record
try:
    (key, meta, bins) = client.select(('test','demo','foo'), ('age', 'gpa'))
except:
  print('failed to get record')
  sys.exit(1)
print ('Bins: ', bins)

Output:

Bins:  {'age': 15, 'gpa': 4.3}

Checking if a Record Exists

Use "exists" to check existence of a record in the database. Note, meta is None if the record is not found.

# Retrieve the record using a non-existent key.
(key, metadata) = client.exists(('test','demo','foo'), policy)
print('User-key, Metadata: ', key[2], metadata)
(key, metadata) = client.exists(('test','demo','nonexistent'), policy)
print('User-key, Metadata: ', key[2], metadata)

Output:

User-key, Metadata:  foo {'ttl': 2592000, 'gen': 1}
User-key, Metadata:  nonexistent None

Batch Read Operations

The get_many and exists_many batch operations allow the application to access multiple records.

import pprint
pp = pprint.PrettyPrinter(depth=4)
keys = []
for i in range(1,3):
  key = ('test', 'demo', 'key' + str(i))
  client.put(key, {'batch': i}, policy)
for i in range(1,4):
  key = ('test', 'demo', 'key' + str(i))
  keys.append(key)
records = client.get_many(keys)
pp.pprint (records)

Output:

[(('test',
   'demo',
   'key1',
   bytearray(b'\xec\x91\x19-K\x7f\x8c\xe3]]x\xd3K\xcae\xcb\xaa\xaa\xc9`')),
  {'gen': 1, 'ttl': 2592000},
  {'batch': 1}),
 (('test',
   'demo',
   'key2',
   bytearray(b'&o\xc5\xc9\x94\xab\x9d\n\xa1\xc3g\xd9\x9a\xbc\xe6\xd8'
             b'\x9e\x97p\x83')),
  {'gen': 1, 'ttl': 2592000},
  {'batch': 2}),
 (('test',
   'demo',
   'key3',
   bytearray(b'cN\x96\xf7\xff\\\xd9hS8\x18\xdc"v\x18\xe2\xe4D\'\xa5')),
  None,
  None)]

Deleting Records

To delete records from the database, there is "remove" operation.

An exception is thrown if the record does not exist.

# Key of the record to be deleted
key1 = ('test', 'demo', 'key1')

# Delete the record
try:
    client.remove(key1)
except:
  print('failed to delete record: ', key1)
  sys.exit(1)
(key, metadata) = client.exists(key1, policy)
print('Key, metadata: ', key1, metadata)
print('Successfully deleted ', key1)
    
# will throw an exception
nokey = ('test', 'demo', 'non-existent')
try:
    client.remove(key)
except:
  print('failed to delete record: ', nokey)

Output:

Key, metadata:  ('test', 'demo', 'key1') None
Successfully deleted  ('test', 'demo', 'key1')
failed to delete record:  ('test', 'demo', 'non-existent')

Multiple Operations on a Single Record

Multiple operations on a single a record can be conveniently performed in a single transaction. Multiple updates as well as reads of a record may be performed atomically in a single transaction. Operations are performed in the order they are specified. Below is a full example of a multi-op request.

from __future__ import print_function
import aerospike
from aerospike_helpers.operations import operations as op_helpers
from aerospike import exception as ex
import sys

config = { 'hosts': [('127.0.0.1', 3000)] }
client = aerospike.client(config).connect()

try:
    key = ('test', 'demo', 1)
    client.put(key, {'age': 25, 'career': 'delivery boy'})
    ops = [
        op_helpers.increment("age", 1000),
        op_helpers.write("name", "J."),
        op_helpers.prepend("name", "Phillip "),
        op_helpers.append("name", " Fry"),
        op_helpers.read("name"),
        op_helpers.read("career"),
        op_helpers.read("age")
    ]
    (key, metadata, bins) = client.operate(key, ops, {'ttl':0}, {'total_timeout':500})
except ex.AerospikeError as e:
    print("Error: {0} [{1}]".format(e.msg, e.code))
    sys.exit(1)

print('Key: ', key)
print('--------------------------')
print('Metadata: ', metadata)
print('--------------------------')
print('Bins: ', bins) # shows only bins specified in read operations

Output:

Key:  ('test', 'demo', 1, bytearray(b'\xb7\xf4\xb88\x89\xe2\xdag\xdeh>\x1d\xf6\x91\x9a\x1e\xac\xc4F\xc8'))
--------------------------
Metadata:  {'ttl': 2592000, 'gen': 2}
--------------------------
Bins:  {'name': 'Phillip J. Fry', 'career': 'delivery boy', 'age': 1025}

Multi-Op Example: Read and Delete in One Request

The example shows reading and deleting a record in a single operation. The record was create above.

from aerospike_helpers.operations import operations
key = ('test', 'demo', 1)
ops = [ operations.read('name'), 
        operations.delete() ]
_, _, bins = client.operate(key, ops)
print('Bins: ', bins)
# will throw an exception
(key_, metadata) = client.exists(key, policy)
print('Key, metadata: ', key, metadata)
print('Successfully deleted ', key)

Output:

Bins:  {'name': 'Phillip J. Fry'}
Key, metadata:  ('test', 'demo', 1) None
Successfully deleted  ('test', 'demo', 1)

Next steps

Visit Aerospike notebooks repo to run additional Aerospike notebooks. To run a different notebook, download the notebook from the repo to your local machine, and then click on File > Open, and select Upload.