Aerospike Notebooks Readme/Tips
For an interactive Jupyter notebook experience: 
Here are some tips and tricks for ease of use and productive experience with Aerospike notebooks.
This notebook requires Aerospike
database running on localhost and that python and the Aerospike python
client have been installed (pip install aerospike). Visit Aerospike
notebooks
repo for
additional details and the docker container.
Learn about Jupyter Notebook
The Jupyter Notebook provides "a web-based application suitable for capturing the whole computation process: developing, documenting, and executing code, as well as communicating the results". New to notebooks? Here is one source to learn more about the Jupyter Notebook.
Find and run Aerospike notebook.
Visit Aerospike notebooks repo to find additional Aerospike notebooks. To run another notebook, download the notebook from the repo to your local machine, and then click on File->Open, and select Upload.
Access shell commands
Use the ! line magic and %%bash cell magic to access shell commands.
That is, you can access a shell command on any line by prefixing it with
a "!", and an entire cell can have bash shell commands if it starts with
%%bash. Here are some examples:
# Accessing shell commands
!ps
!whoami
# Start the Aerospike database.
!asd >& /dev/null
Output:
PID TTY TIME CMD
508 pts/0 00:00:00 ps
jovyan
%%bash
# bash cell
# Check if the Aerospike database is running.
pgrep -x asd >/dev/null && echo "Aerospike database is running" || echo "**Aerospike database is not running!**"
ps -axu | grep asd
Output:
Aerospike database is running
jovyan 42 1.1 0.1 3581124 81956 ? Ssl 01:18 0:05 asd
jovyan 520 0.0 0.0 6308 724 ? S 01:27 0:00 grep asd
Note: Shell commands are accessible in Java kernel through %sh line magic. However it has limitations. We suggest changing the kernel to Python to use shell commands.
Examine server log
It is useful to examine the server log. Assuming it is located at /var/log/aerospike/aerospike.log, and you have the permissions, you can run the following to view the last 10 lines of the log. (Adjust the log path to your setting.)
# View the last 10 lines of the log:
!echo "End of server log:"; tail -10 /var/log/aerospike/aerospike.log
Output:
End of server log:
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 2: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7ff4a56d13c0]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 3: /lib/x86_64-linux-gnu/libpthread.so.0(raise+0xcb) [0x7ff4a56d124b]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 4: asd(cf_log_write_no_return+0x97) [0x5601cc0f7ab6]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 5: asd(xmem_delete_namespace_blocks+0x1ed) [0x5601cbe9f33a]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 6: asd(as_namespaces_setup+0x37f) [0x5601cbe9fd7e]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 7: asd(as_namespaces_init+0x18) [0x5601cbee6203]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 8: asd(as_run+0x326) [0x5601cbeb7bf5]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 9: asd(main+0xd) [0x5601cbe9a803]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 10: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7ff4a50f10b3]
Jan 22 2021 01:27:00 GMT: WARNING (as): (log.c:628) stacktrace: frame 11: asd(_start+0x2e) [0x5601cbe9b10e]
View database state.
The command line tool "aql" can be very handy to examine the data and metadata in the database. For a more complete description of the capabilities, see the
doc. Assuming the database has namespace "test", the following commands can be executed.# Insert a record in set "demo" in namsepace "test" with Primary Key (PK) 1 and a bin or field "testbin"
# with value "hello world!".
!aql -c "INSERT INTO test.demo (PK, 'testbin') VALUES (1, 'hello world!')"
# View all records in the set.
!aql -c "SELECT * FROM test.demo"
# Delete the record
!aql -c "DELETE FROM test.demo WHERE PK = 1"
!aql -c "SELECT * FROM test.demo"
Output:
INSERT INTO test.demo (PK, 'testbin') VALUES (1, 'hello world!')
OK, 1 record affected.
SELECT * FROM test.demo
+----------------+
| testbin |
+----------------+
| "hello world!" |
+----------------+
1 row in set (0.168 secs)
OK
DELETE FROM test.demo WHERE PK = 1
OK, 1 record affected.
SELECT * FROM test.demo
0 rows in set (0.153 secs)
OK
View cluster state.
Another useful utility is asadm which can be used to view various aspects of the database cluster. For a more complete description of its capabilities, see the
doc.# Show the features enabled in this database.
!asadm -e "features"
# Display summary info for the cluster
!asadm -e "summary"
# View the config
!asadm -e "show config"
Output:
Seed: [('127.0.0.1', 3000, None)]
Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.conf
~~~~~~~~~~~~~~~~~~~~~~~Features (2021-01-22 01:27:05 UTC)~~~~~~~~~~~~~~~~~~~~~~~
NODE : jupyter-aerospike-2daeros-2dotebooks-2edocker-2djxk71mnd:3000
AGGREGATION : NO
BATCH : NO
INDEX-ON-DEVICE: NO
INDEX-ON-PMEM : NO
KVS : YES
LDT : NO
QUERY : NO
RACK-AWARE : NO
SC : NO
SCAN : YES
SECURITY : NO
SINDEX : NO
TLS (FABRIC) : NO
TLS (HEARTBEAT): NO
TLS (SERVICE) : NO
UDF : NO
XDR DESTINATION: NO
XDR SOURCE : NO
Seed: [('127.0.0.1', 3000, None)]
Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.conf
Cluster
=======
1. Server Version : E-5.4.0.1
2. OS Version : Ubuntu 20.04.1 LTS (4.19.150+)
3. Cluster Size : 1
4. Devices : Total 1, per-node 1
5. Memory : Total 1.000 GB, 0.00% used (0.000 B), 100.00% available (1.000 GB)
6. Disk : Total 4.000 GB, 0.00% used (0.000 B), 99.00% available contiguous space (3.960 GB)
7. Usage (Unique Data): None
8. Active Namespaces : 0 of 1
9. Features : KVS, Scan
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Namespaces~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace Devices Memory Disk Replication Rack Master
. (Total,Per-Node) (Total,Used%,Avail%) (Total,Used%,Avail%) Factor Aware Objects
test (1, 1) (1.000 GB, 0.00, 100.00) (4.000 GB, 0.00, 99.00) 2 False 0.000
Number of rows: 1
Seed: [('127.0.0.1', 3000, None)]
Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.conf
~~~~~~~~~~~~~~~~~~~~~~~~~Service Configuration (2021-01-22 01:27:09 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~
NODE : jupyter-aerospike-2daeros-2dotebooks-2edocker-2djxk71mnd:3000
advertise-ipv6 : false
auto-pin : none
batch-index-threads : 8
batch-max-buffers-per-queue : 255
batch-max-requests : 5000
batch-max-unused-buffers : 256
batch-without-digests : false
cluster-name : null
debug-allocations : none
disable-udf-execution : false
enable-benchmarks-fabric : false
enable-health-check : false
enable-hist-info : false
enable-ldap : false
enable-security : false
fabric.address : any
fabric.channel-bulk-fds : 2
fabric.channel-bulk-recv-threads : 4
fabric.channel-ctrl-fds : 1
fabric.channel-ctrl-recv-threads : 4
fabric.channel-meta-fds : 1
fabric.channel-meta-recv-threads : 4
fabric.channel-rw-fds : 8
fabric.channel-rw-recv-pools : 1
fabric.channel-rw-recv-threads : 16
fabric.keepalive-enabled : true
fabric.keepalive-intvl : 1
fabric.keepalive-probes : 10
fabric.keepalive-time : 1
fabric.latency-max-ms : 5
fabric.port : 3001
fabric.recv-rearm-threshold : 1024
fabric.send-threads : 8
fabric.tls-name : null
fabric.tls-port : 0
feature-key-file : /etc/aerospike/features.conf
heartbeat.address : any
heartbeat.connect-timeout-ms : 500
heartbeat.interval : 150
heartbeat.mode : mesh
heartbeat.mtu : 1460
heartbeat.port : 3002
heartbeat.protocol : v3
heartbeat.timeout : 10
heartbeat.tls-name : null
heartbeat.tls-port : 0
indent-allocations : false
info-threads : 16
info.address : any
info.port : 3003
keep-caps-ssd-health : false
ldap-login-threads : 8
ldap.disable-tls : false
ldap.polling-period : 300
ldap.query-base-dn : null
ldap.query-user-dn : null
ldap.query-user-password-file : null
ldap.role-query-base-dn : null
ldap.role-query-search-ou : false
ldap.server : null
ldap.session-ttl : 86400
ldap.tls-ca-file : null
ldap.token-hash-method : sha-256
ldap.user-dn-pattern : null
ldap.user-query-pattern : null
log-local-time : false
log-millis : false
microsecond-histograms : false
migrate-fill-delay : 0
migrate-max-num-incoming : 4
migrate-threads : 1
min-cluster-size : 1
node-id : BB906214131EE62
node-id-interface : null
paxos-single-replica-limit : 1
pidfile : /var/run/aerospike/asd.pid
privilege-refresh-period : 300
proto-fd-idle-ms : 0
proto-fd-max : 15000
proto-slow-netio-sleep-ms : 1
query-batch-size : 100
query-buf-size : 2097152
query-bufpool-size : 256
query-in-transaction-thread : false
query-long-q-max-size : 500
query-microbenchmark : false
query-pre-reserve-partitions : false
query-priority : 10
query-priority-sleep-us : 1
query-rec-count-bound : 18446744073709551615
query-req-in-query-thread : false
query-req-max-inflight : 100
query-short-q-max-size : 500
query-threads : 6
query-threshold : 10
query-untracked-time-ms : 1000
query-worker-threads : 15
report-authentication-sinks : 0
report-data-op-sinks : 0
report-sys-admin-sinks : 0
report-user-admin-sinks : 0
report-violation-sinks : 0
run-as-daemon : true
scan-max-done : 100
scan-threads-limit : 128
service-threads : 8
service.access-port : 0
service.address : any
service.alternate-access-port : 0
service.port : 3000
service.tls-access-port : 0
service.tls-alternate-access-port: 0
service.tls-name : null
service.tls-port : 0
sindex-builder-threads : 4
sindex-gc-max-rate : 50000
sindex-gc-period : 10
stay-quiesced : false
syslog-local : -1
ticker-interval : 10
transaction-max-ms : 1000
transaction-retry-ms : 1002
vault-ca : null
vault-path : null
vault-token-file : null
vault-url : null
work-directory : /opt/aerospike
~~~~~~~~~~~~~~~~~~~~~~~~~Network Configuration (2021-01-22 01:27:09 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~
NODE : jupyter-aerospike-2daeros-2dotebooks-2edocker-2djxk71mnd:3000
fabric.address : any
fabric.channel-bulk-fds : 2
fabric.channel-bulk-recv-threads : 4
fabric.channel-ctrl-fds : 1
fabric.channel-ctrl-recv-threads : 4
fabric.channel-meta-fds : 1
fabric.channel-meta-recv-threads : 4
fabric.channel-rw-fds : 8
fabric.channel-rw-recv-pools : 1
fabric.channel-rw-recv-threads : 16
fabric.keepalive-enabled : true
fabric.keepalive-intvl : 1
fabric.keepalive-probes : 10
fabric.keepalive-time : 1
fabric.latency-max-ms : 5
fabric.port : 3001
fabric.recv-rearm-threshold : 1024
fabric.send-threads : 8
fabric.tls-name : null
fabric.tls-port : 0
heartbeat.address : any
heartbeat.connect-timeout-ms : 500
heartbeat.interval : 150
heartbeat.mode : mesh
heartbeat.mtu : 1460
heartbeat.port : 3002
heartbeat.protocol : v3
heartbeat.timeout : 10
heartbeat.tls-name : null
heartbeat.tls-port : 0
info.address : any
info.port : 3003
service.access-port : 0
service.address : any
service.alternate-access-port : 0
service.port : 3000
service.tls-access-port : 0
service.tls-alternate-access-port: 0
service.tls-name : null
service.tls-port : 0
~~~~~~~~~~~~~~~~~~~~~~~~~~test Namespace Configuration (2021-01-22 01:27:09 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~
NODE : jupyter-aerospike-2daeros-2dotebooks-2edocker-2djxk71mnd:3000
allow-ttl-without-nsup : false
background-scan-max-rps : 10000
conflict-resolution-policy : generation
conflict-resolve-writes : false
data-in-index : false
default-ttl : 2592000
disable-cold-start-eviction : false
disable-write-dup-res : false
disallow-null-setname : false
enable-benchmarks-batch-sub : false
enable-benchmarks-ops-sub : false
enable-benchmarks-read : false
enable-benchmarks-udf : false
enable-benchmarks-udf-sub : false
enable-benchmarks-write : false
enable-hist-proxy : false
evict-hist-buckets : 10000
evict-tenths-pct : 5
geo2dsphere-within.earth-radius-meters : 6371000
geo2dsphere-within.level-mod : 1
geo2dsphere-within.max-cells : 12
geo2dsphere-within.max-level : 30
geo2dsphere-within.min-level : 1
geo2dsphere-within.strict : true
high-water-disk-pct : 0
high-water-memory-pct : 0
ignore-migrate-fill-delay : false
index-stage-size : 1073741824
index-type : shmem
memory-size : 1073741824
migrate-order : 5
migrate-retransmit-ms : 5000
migrate-sleep : 1
nsid : 0
nsup-hist-period : 3600
nsup-period : 120
nsup-threads : 1
partition-tree-sprigs : 256
prefer-uniform-balance : true
rack-id : 0
read-consistency-level-override : off
reject-non-xdr-writes : false
reject-xdr-writes : false
replication-factor : 2
sindex.num-partitions : 32
single-bin : false
single-scan-threads : 4
stop-writes-pct : 90
storage-engine : device
storage-engine.cache-replica-writes : false
storage-engine.cold-start-empty : false
storage-engine.commit-min-size : 0
storage-engine.commit-to-device : false
storage-engine.compression : none
storage-engine.compression-level : 0
storage-engine.data-in-memory : true
storage-engine.defrag-lwm-pct : 50
storage-engine.defrag-queue-min : 0
storage-engine.defrag-sleep : 1000
storage-engine.defrag-startup-minimum : 10
storage-engine.direct-files : false
storage-engine.disable-odsync : false
storage-engine.enable-benchmarks-storage : false
storage-engine.encryption-key-file : null
storage-engine.file[0] : /opt/aerospike/data/test.dat
storage-engine.filesize : 4294967296
storage-engine.flush-max-ms : 1000
storage-engine.max-write-cache : 67108864
storage-engine.min-avail-pct : 5
storage-engine.post-write-queue : 0
storage-engine.read-page-cache : false
storage-engine.scheduler-mode : null
storage-engine.serialize-tomb-raider : false
storage-engine.sindex-startup-device-scan: false
storage-engine.tomb-raider-sleep : 1000
storage-engine.write-block-size : 1048576
strong-consistency : false
strong-consistency-allow-expunge : false
tomb-raider-eligible-age : 86400
tomb-raider-period : 86400
transaction-pending-limit : 20
truncate-threads : 4
write-commit-level-override : off
xdr-bin-tombstone-ttl : 86400
xdr-tomb-raider-period : 120
xdr-tomb-raider-threads : 1
Next steps
Visit Aerospike notebooks repo to run additional Aerospike notebooks. To run a different notebook, download the notebook from the repo to your local machine, and then click on File > Open, and select Upload.