The Aerospike Python client

Two clients, two jobs

In Parts 1 and 2, you used the Aerospike Spark connector for everything: writing feature data, reading training sets, and querying metadata. That made sense because those were batch operations where Spark’s distributed processing was the point.

Model serving is a different problem. When a ride request comes in, the platform needs to look up features for one specific driver and get a prediction back in milliseconds. Spark, however, is designed for throughput over large datasets, not latency on individual records. A Spark read to fetch a single row is too slow for this application’s needs.

The Aerospike Python client is designed for single-record reads. They often complete in well under a millisecond. This difference becomes critical as the feature store grows. Even with a fast model, per-request query scans over very large datasets can dominate latency. For serving, you want direct primary-key reads and only the bins the model needs.

Connect to Aerospike

1. Run Cell 16 to connect the Aerospike Python client.

Cell 16: Connect the Aerospike Python client

import aerospike

as_config = {
    'hosts': [('127.0.0.1', 3000)]
}

as_client = aerospike.client(as_config).connect()
print("Connected to Aerospike")

The client connects directly to your single Aerospike node. In production, you’d have a cluster of Aerospike nodes and define the connection slightly differently.

1. Run Cell 17 to read a single driver record by primary key.

Cell 17: Read a single driver record by key

record_key = ('test', 'driver-features', 'driver_042')

(returned_key, metadata, bins) = as_client.get(record_key)
driver_id = bins.get("driver_id", record_key[2])
print(f"Driver: {driver_id}")
for bin_name, value in bins.items():
    print(f"  {bin_name}: {value}")

Expected output

Driver: driver_042
  driver_id: driver_042
  ds_decl_rate: 0.061
  ds_avg_rating: 4.72
  da_trips_today: 11
  label: 0

The get() call takes a tuple of (namespace, set, primary_key) and returns the full record. For records written through the Spark connector, the returned key tuple may omit the user key (returned_key[2] can be None), so printing from bins["driver_id"] or record_key[2] is more reliable for display. Your feature values may differ due to the random seed in Part 2’s data generation.

Latency comparison

To see why the Python client matters for serving, compare loading multiple records with both clients in one cell.

Benchmark environment

Latency results depend on your hardware resources, current system load, and local setup. The example measurements in this tutorial were collected on a base model MacBook Pro with an M2 Pro processor.

1. Run Cell 18 to compare Aerospike and Spark latency over multiple records.

Cell 18: Compare Aerospike and Spark latency over multiple records

import time
from pyspark.sql.types import StructType, StructField, StringType, DoubleType, LongType, IntegerType

records_to_load = 5
driver_ids = [f"driver_{i:03d}" for i in range(1, records_to_load + 1)]

# --- Aerospike Python client ---
python_timings = []
print(f"Loading {records_to_load} records with Aerospike...")
for i, driver_id in enumerate(driver_ids, start=1):
    key = ('test', 'driver-features', driver_id)
    start = time.perf_counter()
    (_, _, _) = as_client.get(key)
    elapsed_ms = (time.perf_counter() - start) * 1000
    python_timings.append(elapsed_ms)
    print(f"Loaded {i}/{records_to_load}. Elapsed time: {elapsed_ms:.2f} ms")

python_avg = sum(python_timings) / len(python_timings)
print(f"All {records_to_load} records loaded. Average time: {python_avg:.2f} ms per record.")
print()

# --- Spark connector ---
schema = StructType([
    StructField('driver_id', StringType(), False),
    StructField('ds_decl_rate', DoubleType(), True),
    StructField('ds_avg_rating', DoubleType(), True),
    StructField('da_trips_today', LongType(), True),
    StructField('label', IntegerType(), True)
])

spark_timings = []
print(f"Loading {records_to_load} records with Spark...")
for i, driver_id in enumerate(driver_ids, start=1):
    start = time.perf_counter()
    _ = spark.read \
        .format("aerospike") \
        .schema(schema) \
        .option("aerospike.read-set", "driver-features") \
        .option("aerospike.infered-key-type", "string") \
        .option("aerospike.sindex-enable", "false") \
        .load().where(f'driver_id = "{driver_id}"').collect()[0]
    elapsed_ms = (time.perf_counter() - start) * 1000
    spark_timings.append(elapsed_ms)
    print(f"Loaded {i}/{records_to_load}. Elapsed time: {elapsed_ms:.2f} ms")

spark_avg = sum(spark_timings) / len(spark_timings)
print(f"All {records_to_load} records loaded. Average time: {spark_avg:.2f} ms per record.")

Expected output

Loading 5 records with Aerospike...
Loaded 1/5. Elapsed time: 0.33 ms
Loaded 2/5. Elapsed time: 0.25 ms
Loaded 3/5. Elapsed time: 0.22 ms
Loaded 4/5. Elapsed time: 0.24 ms
Loaded 5/5. Elapsed time: 0.27 ms
All 5 records loaded. Average time: 0.26 ms per record.

Loading 5 records with Spark...
Loaded 1/5. Elapsed time: 1823.41 ms
Loaded 2/5. Elapsed time: 1601.82 ms
Loaded 3/5. Elapsed time: 1495.67 ms
Loaded 4/5. Elapsed time: 1579.30 ms
Loaded 5/5. Elapsed time: 1522.11 ms
All 5 records loaded. Average time: 1604.46 ms per record.

You can change records_to_load at the top of the cell to retry with different sample sizes. The progress messages and averages will automatically reflect that value.

Both return the same data, but the read time shows that switching to the Aerospike Python client is the right choice for this use case. The Python client stays fast across repeated key lookups, while Spark remains much slower for this serving pattern.

From here on, all feature retrieval for serving uses the Python client.

Checklist

• I understand why model serving uses the Python client instead of the Spark connector.
• I can read a single record from Aerospike with sub-millisecond latency.