# Specify JMS queues and topics

## Overview

This page describes how to configure the queues and topics sections of the `/etc/aerospike-jms-inbound/aerospike-jms-inbound.yml` file. These sections specify the properties of the JMS queues and topics from which you want the JMS inbound connector to consume messages.

## Example

Following is an example of a `topics` section that configures one topic. The name of the topic is `employees`. A `queues` section includes the same properties.

-   [Version 2.0.0 and later](#tab-panel-1161)
-   [Version 1.1.0 and earlier](#tab-panel-1162)

```yaml
topics:

  employees:

    session-ratio: 1

    jms-transaction-size: 16

    jms-message-receive-timeout: 100

    aerospike-operation:

      type: write

      max-retries: 2

      total-timeout: 0

      record-exists-action: update

      send-key: false

    mapping:

      bins:

        all-value-fields: false

        type: multi-bins

        map:

          binOne:

            source: value-field

            field-name: field-one

      key-field:

        source: value-field

        field-name: employeeID

      namespace:

        mode: static

        value: east

      set:

        mode: dynamic

        source: value-field

        field-name: subLocation

      ttl:

        mode: dynamic

        source: value-field

        field-name: ttl

    parsing: json
```

```yaml
topics:

  employees:

    session-ratio: 1

    jms-transaction-size: 16

    jms-message-receive-timeout: 100

    aerospike-operation:

      type: write

      max-retries: 2

      total-timeout: 0

      record-exists-action: update

      send-key: false

    mapping:

      all-fields: false

      bins:

        field-one: binOne

      key-field: employeeID

      namespace:

        mode: static

        value: east

      set:

        mode: dynamic

        field-name: subLocation

      ttl:

        mode: dynamic

        field-name: ttl

    parsing: json
```

This table shows how, using this example configuration, a JMS inbound connector maps JSON messages from the JMS message broker to Aerospike records:

| JSON | Aerospike Namespace | Aerospike Set | Aerospike Key | Aerospike Bin | TTL |
| --- | --- | --- | --- | --- | --- |
| `{"field-one":"one","subLocation":"BNG", "employeeID": E123, ttl: "100H"}` | east | BNG | E123 | Bin(“binOne”, one) | 100H |
| `{"field-one":["one"],"subLocation":"MYS", "employeeID": E456, ttl: "10M"}` | east | MYS | E456 | Bin(“binOne”, List(“one”)) | 10M |
| `{"field-one":{"one": "one"},"subLocation":"DLH", "employeeID": E789, ttl: "1D"}` | east | DLH | E789 | Bin(“binOne”, Map(“one” to “one”)) | 1D |
| `{"field-one":"one","subLocation":"MUM", "employeeID": E987, ttl: "100"}` | east | MUM | E987 | Bin(“binOne”, one) | 100 |

## Procedure

### Listing queues and topics to configure

The configuration file must include at least one of these sections:

| Property | Required | Description |
| --- | --- | --- |
| queues | no | A list of the queues to consume messages from. |
| topics | no | A list of the topics to consume messages from. |

The first level in a `topics` or `queues` section is the name of a topic or queue. For example, suppose you want to configure three topics: `employees`, `offices`, and `buildings`. Your `topics` section would, in outline, look like this:

```yaml
topics:

  employees:

    ...

  offices:

    ...

  buildings:

    ...
```

### Configuring queues and topics

For each topic or a queue that you name in their corresponding section, you can set the following properties:

-   [Version 2.0.0 and later](#tab-panel-1163)
-   [Version 1.1.0 and earlier](#tab-panel-1164)

| Option | Required | Default | Description |
| --- | --- | --- | --- |
| `session-ratio` | no | 1 | The ratio of total JMS sessions to use for this source. See [Distributing JMS sessions across queues and topics](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/topics/#distributing-jms-sessions-across-queues-and-topics) for details. |
| `jms-transaction-size` | no | 16 | The number of JMS messages to consume in a single JMS transaction. |
| `jms-message-receive-timeout` | no | 100 | The number of milliseconds to wait for a JMS message before timing out. A value of zero signifies that there is no timeout. |
| `aerospike-operation` | no | write | Specifies how to apply writes and deletes to your Aerospike database. See [Configuring Aerospike operations to perform on records](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/topics/#configure-aerospike-operations) for details. |
| `mapping` | in some cases |  | The mapping of the JMS message to an Aerospike record. It is required if you do not implement `message-transformer` for the topic/queue or implement a `message-transformer` with [InboundMessage<K, M>](https://github.com/aerospike/aerospike-connect-inbound-sdk/blob/1.3.2/src/main/java/com/aerospike/connect/inbound/model/InboundMessage.java). See [Map JMS messages to Aerospike records](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/topics/#map-jms-messages-to-aerospike-records) for details. |
| `parsing` | yes |  | The JMS message parsing config. See [Specifying the format of inbound messages](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/topics/#specifying-the-format-of-inbound-messages) for details. |
| `message-transformer` | no |  | The configuration parameters for a message transformer. See [Aerospike JMS Inbound Message Transformer](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/message-transformer) for details. |

| Option | Required | Default | Description |
| --- | --- | --- | --- |
| `session-ratio` | no | 1 | The ratio of total JMS sessions to use for this destination. See [Distributing JMS sessions across queues and topics](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/topics/#distributing-jms-sessions-across-queues-and-topics) for details. |
| `jms-transaction-size` | no | 16 | The number of JMS messages to consume in a single JMS transaction. |
| `jms-message-receive-timeout` | no | 100 | The number of milliseconds to wait for a JMS message before timing out. A value of zero signifies that there is no timeout. |
| `aerospike-operation` | no | write | The Aerospike command. See [Configuring Aerospike commands to perform on records](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/topics/#configure-aerospike-operations) for details. |
| `mapping` | yes |  | The JMS message to Aerospike record mapping. See [Configuring how to map JMS messages to Aerospike records](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/topics/#map-jms-messages-to-aerospike-records) for details. |
| `parsing` | yes |  | The JMS message parsing config. See [Specifying the format of inbound messages](https://aerospike.com/docs/connectors/streaming/jms/inbound/configure/topics/#specifying-the-format-of-inbound-messages) for details. |

### Distributing JMS sessions across queues and topics

You can set this property for each topic or queue that you specify in the `queues` and `topics` sections. However, to set it for an individual topic or queue, you must first know the values for all of the queues and topics that you plan to configure in `/etc/aerospike-jms-inbound/aerospike-jms-inbound.yml`.

To calculate the values of this property for all the queues and topics that you want to use:

1.  Taking the total number of JMS sessions as a whole (1), represent the whole as the fraction _n_/_n_, where _n_ represents the number of equal divisions of the whole that you plan to distribute among your queues and topics.
    
    The value of _n_ must be greater than or equal to the number of your queues and topics, and less than or equal to the number of JMS sessions. The number of JMS sessions is the product of two values that you set in your JMS message broker: `max-connections` and `max-sessions-per-connection`. For example, if you set `max-connections` to 4 and `max-sessions-per-connection` to 2, then the number of JMS sessions is 8. Therefore, if the number of your topics or queues is 4, _n_ can be greater than or equal to 4, and less than or equal to 8.
    
    For example, suppose that you plan to distribute sessions across two topics and one queue. You smallest _n_ you could use would be 3 because you have three sources. The fractional representation of the whole would be 3/3. (You don’t have to use a value for _n_ that equals the number of your queues and topics, however.)
    
2.  Split this fraction across your queues and topics.
    
    For example, you have two topics (T1 and T2) and one queue (Q1) and the fraction is 3/3. Each source must be allocated some portion of the sessions. So, you give each source 1/3.
    
3.  Take the numerator of each fraction.
    
4.  Specify the numerators as the values of `session-ratio` for your queues and topics.
    
    To continue with the example from step 2, this table shows the value of `session-ratio` for each source:
    
    | Source | Type | `session-ratio` |
    | --- | --- | --- |
    | T1 | Topic | 1 |
    | T2 | Topic | 1 |
    | Q1 | Queue | 1 |
    
    You might choose to distribute the sessions unequally across your queues and topics. For example, if you have topics T1 and T2, and queue Q1, you could use a value of 5 for _n_, making 5/5, and then set these values for `session-ratio`:
    
    | Source | Type | `session-ratio` |
    | --- | --- | --- |
    | T1 | Topic | 2 |
    | T2 | Topic | 2 |
    | Q1 | Queue | 1 |
    
    In this case, topics T1 and T2 would each be allocated two-fifths of the sessions, while topic T3 would be allocated one-fifth.
    
    If the number of JMS sessions is not evenly divisible by _n_, then the remaining sessions are distributed randomly to your queues and topics at startup.
    
    ::: note
    If you set `session-ratio` for one of the queues and topics that you plan to use, then you must set it for all of those queues and topics.
    :::
    

## Configure Aerospike operations

-   [Version 2.0.0 and later](#tab-panel-1165)
-   [Version 1.1.0 and earlier](#tab-panel-1166)

Use the `aerospike-operation` option to specify how to apply writes and deletes to your Aerospike database.

| Option | Required | Default | Description |
| --- | --- | --- | --- |
| `type` | no | `write` | Specifies the type of command to perform in Aerospike. Valid values are: `write`, `delete`. |
| `max-retries` | no | 2 | Specifies the maximum number of times to retry a transaction. |
| `total-timeout` | no | null | Specifies the number of milliseconds to wait before a transaction times out. If null, the `total-timeout` configured in the `aerospike` section is used. |
| `record-exists-action` | no | `update` | Specifies the action to take if a record already exists and `type` is set to `write`. Valid values are `update`, `update-only`, `replace`, `replace-only`, `create-only`. |
| `send-key` | no | `false` | Specifies to send a user-defined key, in addition to a hash digest. Applies only when the value for the property `type` is `write`. |
| `durable-delete` | no | `false` | Specifies to leave a tombstone for the record. Applies only when the value for the property `type` is `delete`. |
| `ignore-error-codes` | no | empty set | Consider a record as successfully processed if it throws [AerospikeException](https://github.com/aerospike/aerospike-client-java/blob/f46baecd7ace882244897c50d88b688fd545f5e2/client/src/com/aerospike/client/AerospikeException.java) with the given [ResultCode](https://github.com/aerospike/aerospike-client-java/blob/8f3d217237b280c1bc0420c58efa63428f8625bd/client/src/com/aerospike/client/ResultCode.java). |

### Examples

```yaml
# An aerospike write operation with all relevant options.

aerospike-operation:

  type: write

  max-retries: 3

  total-timeout: 0

  record-exists-action: update-only

  send-key: true

  ignore-error-codes:

    - 22

    - 13
```

```yaml
# An aerospike delete operation with all relevant options.

aerospike-operation:

  type: delete

  max-retries: 3

  total-timeout: 0

  durable-delete: true

  ignore-error-codes:

    - 22

    - 13
```

Use the `aerospike-operation` option to specify how to apply writes and deletes to your Aerospike database.

| Option | Required | Default | Description |
| --- | --- | --- | --- |
| `type` | no | `write` | Specifies the type of command to perform in Aerospike. Valid values are: `write`, `delete`. |
| `max-retries` | no | 2 | Specifies the maximum number of times to retry a transaction. |
| `total-timeout` | no | 0 | Specifies the number of milliseconds to wait before a transaction times. A value of 0 indicates that there is no time limit. |
| `record-exists-action` | no | `update` | Specifies the action to take if a record already exists and `type` is set to `write`. Valid values are `update`, `update-only`, `replace`, `replace-only`, `create-only`. |
| `send-key` | no | `false` | Specifies to send a user-defined key, in addition to a hash digest. Applies only when the value for the property `type` is `write`. |
| `durable-delete` | no | `false` | Specifies to leave a tombstone for the record. Applies only when the value for the property `type` is `delete`. |

## Map JMS messages to Aerospike records

Use the `mapping` option to specify how to map JMS messages in a topic or queue to Aerospike records.

-   [Version 2.0.0 and later](#tab-panel-1167)
-   [Version 1.1.0 and earlier](#tab-panel-1168)

| Option | Required | Description |
| --- | --- | --- |
| `namespace` | yes | Specifies whether the name of the namespace is set in the configuration file or in messages. See [Specify `namespace`, `set`, and `ttl`](#specify-namespace-set-and-ttl) for details. |
| `set` | no | Specifies whether the name of the set is set in the configuration file or in messages. See [Specify `namespace`, `set`, and `ttl`](#specify-namespace-set-and-ttl) for details. |
| `ttl` | no | Specifies whether the time to live is set in the configuration file or in messages. See [Specify `namespace`, `set`, and `ttl`](#specify-namespace-set-and-ttl) for details. |
| `bins` | yes | Mapping of incoming message to Aerospike bins. See [Mapping fields in messages to bins in Aerospike records](#mapping-fields-in-messages-to-bins-in-aerospike-records) for details. |
| `key-field` | no\* | The Field selection configuration value for the Aerospike record key. See [Key-field and Digest config](#key-field-and-digest-config) for details. |
| `digest` | no\* | The Field selection configuration value for the Aerospike record digest. The field has to be a valid [Aerospike RIPEMD-160](https://support.aerospike.com/s/article/FAQ-How-primary-keys-and-digests-are-used-in-Aerospike) digest. The digest field should be plain bytes or a Base64 encoded string. See [Key-field and Digest config](#key-field-and-digest-config) for details. |

Either `key-field` or `digest` has to be specified in a `mapping`.

### Key-field and Digest config

Use these two options to generate `key-field` or `digest`.

| Option | Required | Description |
| --- | --- | --- |
| `source` | yes | The source of the value. |
| `field-name` | no | Name of the field to extract the value from. Only applies when source is `value-field`. |

#### Possible values for the `source` option

| Value | Description |
| --- | --- |
| `key` | Use the whole of the source record key as the value. |
| `value` | Use the whole of the source record value as the value. |
| `value-field` | Extract a value from the source record value. |

| Option | Required | Default | Description |
| --- | --- | --- | --- |
| `all-fields` | yes | false | Indicates whether all fields from the JMS message should be converted to Aerospike record bins. |
| `bins` | no |  | Mapping of JMS message field names to Aerospike bin names. |
| `key-field` | yes |  | The JMS message field to use as the Aerospike record key. |
| `namespace` | yes |  | Specifies whether the name of the namespace is set in the configuration file or in messages. See [Specifying where to find values for `namespace`, `set`, and `ttl`](#specify-namespace-set-and-ttl) for details. |
| `set` | no |  | Specifies whether the name of the set is set in the configuration file or in messages. See [Specifying where to find values for `namespace`, `set`, and `ttl`](#specify-namespace-set-and-ttl) for details. |
| `ttl` | no |  | Specifies whether the time to live is set in the configuration file or in messages. See [Specifying where to find values for `namespace`, `set`, and `ttl`](#specify-namespace-set-and-ttl) for details. |

## Specify `namespace`, `set`, and `ttl`

Configuration to obtain values for `namespace`, `set`, and `ttl` requires definitions for the following properties:

| Option | Required | Description |
| --- | --- | --- |
| `mode` | yes | How the value of a given field should be obtained. The possible options are `static` and `dynamic`. |

### Static: Specify the values in the configuration file

To set a static or unchanging name for the namespace, name for the set, or value for the time to live for an Aerospike record, you can use this option:

| Option | Required | Description |
| --- | --- | --- |
| value | yes | The static name or value. |

This example sets the name of a namespace to `east` for all records that an inbound connector creates from the messages that it receives:

```yaml
mapping:

  ...

  namespace:

    mode: static

    value: east

  ...
```

This example sets the time to live for all records that an inbound connector creates from the messages that it receives:

```yaml
mapping:

  ...

  ttl:

    mode: static

    value: -1

  ...
```

Choose from among the following values when setting the time to live for Aerospike records:

| Value | Description |
| --- | --- |
| `-2` | Specifies not to reset the time to live when a record is updated. |
| `-1` | Specifies that the time to live never expires. |
| `0` | Specifies to set the time to live to the default that is configured for the namespace in the Aerospike database. |
| Integer greater than 0 | Specifies by default the time to live in seconds. However, you can use one of the following suffixes to specify a different unit of time: `M` for minutes, `H` for hours, `D` for days. |

### Dynamic: Specify message fields from which to obtain values

Select a value dynamically from the JMS message.

-   [Version 2.0.0 and later](#tab-panel-1169)
-   [Version 1.1.0 and earlier](#tab-panel-1170)

| Option | Required | Description |
| --- | --- | --- |
| `source` | yes | The source of the value. |
| `field-name` | no | Name of the field to extract the value from. Only applies when source is `value-field`. |

#### Possible values for the `source` option

| Value | Description |
| --- | --- |
| `key` | Use the whole of the source record key as the value. |
| `value` | Use the whole of the source record value as the value. |
| `value-field` | Extract a value from the source record value. |

#### Example

```yaml
mapping:

  ...

  namespace:

    mode: dynamic

    source: value-field

    field-name: category

  set:

    mode: dynamic

    source: value-field

    field-name: brand

  ttl:

    mode: dynamic

    source: value-field

    field-name: validForSec

  ...
```

## Mapping fields in messages to bins in Aerospike records

For topics and queues that you list in their corresponding section, you must map fields from messages to bins in Aerospike records.

| Option | Required | Description |
| --- | --- | --- |
| type | yes | Number of bins that exist in Aerospike records that correspond to messages in the topic/queue. Valid values are `single-bin`, `multi-bins`. |

### Single bin

Use these two options to specify the source of the value to write to single-bin Aerospike records.

| Option | Required | Description |
| --- | --- | --- |
| `source` | yes | The source of the value. |
| `field-name` | no | Name of the field to extract the value from. Only applies when source is `value-field`. |

#### Possible values for the `source` option

| Value | Description |
| --- | --- |
| `key` | Use the whole of the source record key as the value. |
| `value` | Use the whole of the source record value as the value. |
| `value-field` | Extract a value from the source record value. |

#### Examples

```yaml
# A single bin Aerospike record with the bin value extracted from the JMS

# record field "name".

bins:

  type: single-bin

  source: value-field

  field-name: name
```

```yaml
# A single bin Aerospike record with the bin value equal to the whole of the

# JMS message value.

bins:

  type: single-bin

  source: value
```

### Multiple bins

Use these options to specify the source of the values to write to Aerospike records that are comprised of multiple bins.

| Option | Required | Default | Description |
| --- | --- | --- | --- |
| `all-value-fields` | no | false | Indicates if all fields from the JMS message value should be converted to Aerospike record bins. |
| `map` | no |  | Map of the Aerospike record bin name to the source of the bin value. Use this option only when `all-value-fields` is false. |
| `source` | yes |  | Source of the value. See the table below for the possible values. Used with the `map` option |
| `field-name` | no |  | Name of the field to extract the value from. Only applies when source is `value-field`. Used with the `map` option. |

#### Possible values for the `source` option

| Value | Description |
| --- | --- |
| `key` | Use the whole of the source record key as the value. |
| `value` | Use the whole of the source record value as the value. |
| `value-field` | Extract a value from the source record value. |

#### Examples

```yaml
# Write all the JMS message value fields as Aerospike record bins.

bins:

  type: multi-bins

  all-value-fields: true
```

```yaml
# Extract the values of Aerospike bins "name" and "city" from the JMS fields `firstName` and `residence`.

bins:

  type: multi-bins

  map:

    name:

      source: value-field

      field-name: firstName

    city:

      source: value-field

      field-name: residence
```

```yaml
# Use the whole of the JMS message value as the Aerospike bin "metadata".

bins:

  type: multi-bins

  map:

    metadata:

      source: value
```

| Option | Required | Expected Value | Description |
| --- | --- | --- | --- |
| mode | yes | dynamic | Specifies that the name or value comes from a field. |
| field-name | yes |  | Name of field in the message to extract the value from. |

## Specifying the format of inbound messages

The message parsing configuration.

| Value | Description |
| --- | --- |
| `json` | Specifies that the message is a JMS TextMessage in JSON format. |
| `msgpack` | Specifies that the message is a JMS BytesMessage in MsgPack format. |
| `map` | Specifies that the message is a JMS MapMessage. |
| `java-serialized` | Specifies that the message is a JMS ObjectMessage in Java-serialized object format. The serialized object should be in the class path. The JAR containing the class file and its dependencies should be put in `/opt/aerospike-jms-inbound/usr-lib`. |