Skip to content

Write and read data

For the complete documentation index see: llms.txt

All documentation pages available in markdown.

With the stack running, load dispute records into Aerospike to simulate authorization-service writes at runtime. Then confirm they are searchable in Elasticsearch before a support agent picks up the case.

The sample dataset includes 14 disputes across 10 fictional merchants. Several merchants appear more than once, including Cozmic Pizza. Its two disputes, one open and one resolved, are the targets of the search example in the next part.

  1. Write 14 dispute records to the test.disputes set and verify Cross-Datacenter Replication (XDR) delivery. Each record has a transaction ID, merchant name, amount in cents, dispute status, and a memo agents might search on. The script waits for pending XDR work to finish, captures the cumulative success counter, writes the records using aql in the tools container, and waits for the counter to increase by 14. Rerun this complete step to verify another batch without tearing down the stack.

    Terminal
    Terminal
    write_and_wait_for_xdr() {
      attempt=1
      while [ "$attempt" -le 30 ]; do
        stats="$(docker compose exec -T aerospike asinfo -v 'get-stats:context=xdr;dc=esDC')" || return 1
        baseline="$(printf '%s\n' "$stats" | tr ';' '\n' | awk -F= '$1 == "success" {print $2}')"
        in_queue="$(printf '%s\n' "$stats" | tr ';' '\n' | awk -F= '$1 == "in_queue" {print $2}')"
        in_progress="$(printf '%s\n' "$stats" | tr ';' '\n' | awk -F= '$1 == "in_progress" {print $2}')"
        case "$baseline" in ''|*[!0-9]*) echo 'Could not read the XDR success counter' >&2; return 1;; esac
        case "$in_queue" in ''|*[!0-9]*) echo 'Could not read the XDR queue counter' >&2; return 1;; esac
        case "$in_progress" in ''|*[!0-9]*) echo 'Could not read the XDR in-progress counter' >&2; return 1;; esac
        [ "$in_queue" -eq 0 ] && [ "$in_progress" -eq 0 ] && break
        attempt=$((attempt + 1))
        sleep 1
      done
      if [ "$in_queue" -ne 0 ] || [ "$in_progress" -ne 0 ]; then
        echo 'Timed out waiting for pending XDR deliveries' >&2
        return 1
      fi
    

      target=$((baseline + 14))
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1001', 'Cozmic Pizza', 650, 'open', 'Weekly pizza night standing order')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1002', 'Orbit Stream Media', 1599, 'open', 'Streaming plan renewal')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1003', 'Cozmic Pizza', 475, 'resolved', 'Duplicate charge corrected')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1004', 'Metro Transit Pass', 220, 'open', 'Commute pass reload')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1005', 'Hercules Books', 4299, 'under_review', 'Textbook purchase')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1006', 'Sunshine Kitchen', 8932, 'open', 'Group dinner charge')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1007', 'Orion Airlines', 28750, 'under_review', 'Round-trip fare')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1008', 'Moonrise Market', 3412, 'open', 'Weekly grocery order')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1009', 'Gaia Pharmacy', 1250, 'resolved', 'Prescription copay')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1010', 'Starlight Outfitters', 15699, 'open', 'Camping equipment rental')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1011', 'Nova Home Furnishings', 45000, 'under_review', 'Office chair delivery')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1012', 'Orbit Stream Media', 1599, 'open', 'Second household subscription')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1013', 'Hercules Books', 1899, 'open', 'Audiobook bundle')" || return 1
      docker compose exec -T aerospike-tools aql -h aerospike -c "INSERT INTO test.disputes (PK, merchant, amount_cents, status, memo) VALUES ('txn-1014', 'Metro Transit Pass', 220, 'resolved', 'Pass reload refund issued')" || return 1
    

      attempt=1
      while [ "$attempt" -le 30 ]; do
        stats="$(docker compose exec -T aerospike asinfo -v 'get-stats:context=xdr;dc=esDC')" || return 1
        success="$(printf '%s\n' "$stats" | tr ';' '\n' | awk -F= '$1 == "success" {print $2}')"
        case "$success" in ''|*[!0-9]*) echo 'Could not read the XDR success counter' >&2; return 1;; esac
        [ "$success" -ge "$target" ] && {
          printf 'baseline=%s target=%s success=%s\n' "$baseline" "$target" "$success"
          return 0
        }
        attempt=$((attempt + 1))
        sleep 1
      done
      echo 'Timed out waiting for 14 XDR deliveries' >&2
      printf '%s\n' "$stats" >&2
      return 1
    }
    write_and_wait_for_xdr
    batch_status=$?
    unset -f write_and_wait_for_xdr
    [ "$batch_status" -eq 0 ]
    Insert and XDR delivery result
    Expected output
    OK, 1 record affected.
    ...
    OK, 1 record affected.
    baseline=... target=... success=...

    Each insert command prints OK, 1 record affected. once. The final line confirms that the cumulative XDR success counter reached the target for this batch.

    If the script reports a timeout, see Troubleshoot.

  2. Verify all 14 records reached Elasticsearch. The Elasticsearch connector also creates the aerospike Elasticsearch index on the first write. Refresh the index after XDR acknowledges the batch, then count the records.

    Terminal
    Terminal
    curl -fsS -X POST 'http://localhost:9200/aerospike/_refresh' > /dev/null
    curl -fsS 'http://localhost:9200/aerospike/_count?pretty'
    Index record count
    Expected output
    {
        "count": 14,
        "_shards": {
            "total": 1,
            "successful": 1,
            "skipped": 0,
            "failed": 0
        }
    }

The matching counts confirm the local Aerospike-to-Elasticsearch pipeline delivers data end to end. Aerospike only acknowledges a change once the Elasticsearch connector confirms delivery, so a matching count here means every record actually arrived, not just that Aerospike attempted to send it.

All 14 disputes are indexed. You can now run the same search as a support agent.