Configure the Elasticsearch client for Aerospike Connect for Elasticsearch
The es-client section of the aerospike-elasticsearch-outbound.yaml file specifies the configuration parameters for creating Elasticsearch Java Client. This client is used for all communication from Aerospike Connect for Elasticsearch to the Elasticsearch cluster.
Cluster configuration
The cluster-config section defines the Elasticsearch cluster type.
| Option | Required | Description | Details |
|---|---|---|---|
| type | Yes | The type of the Elasticsearch cluster. | The supported options are on-prem and cloud. |
Each type has a specific required configuration.
| Type | Option | Description | Details |
|---|---|---|---|
| on-prem | nodes | A list of Elasticsearch nodes to connect to. All valid properties of Node can be configured. | Choose this option if you manage the Elasticsearch cluster, and you have access to the cluster nodes. |
| cloud | cloud-id | A valid elastic cloud cloudId that will route to a cluster. The cloudId is located in your Elastic Cloud user console | Choose this option if you are using the Elastic Cloud hosted service. |
Default Headers
An optional default-headers section is a set of key-value pairs that sets the default request headers which are sent with each request.
Example
...es-client: ... default-headers: environment: prod ......Auth configuration
The auth-config section sets options for authenticating with the Elasticsearch cluster.
| Option | Required | Default | Description | Details |
|---|---|---|---|---|
| type | Yes | none | The type of authentication method. | The supported options are basic, token, api-key, and none. |
Some of these types have extra configuration options.
| Config Type | Option | Required | Description |
|---|---|---|---|
| basic | username | Yes | Username for basic auth. |
| basic | password-file | Yes | File path of the file containing the password for basic authentication. |
| token | token-file | Yes | File path of the file containing the authentication token. |
| api-key | api-key-id-file | Yes | File path of the file containing the API key ID. |
| api-key | api-key-secret-file | Yes | File path of the file containing the API key Secret. |
Transport options
The transport-options section is optional. It sets configurations for all application-specific or request-specific options, including headers and query parameters.
| Option | Default | Description |
|---|---|---|
| headers | empty map | All request specific headers. |
| parameters | empty map | All request specific headers query parameters. |
Request Config
The elasticsearch-request-config section sets options for customizing the underlying HTTP client at request level. RequestConfig has more details on some of these options:
| Option | Default | Description |
|---|---|---|
| expect-continue-enabled | false | Determines whether the ‘Expect: 100-Continue’ handshake is enabled for entity enclosing methods. Refer to RequestConfig for more details. |
| proxy | null | HTTP proxy for request execution. All valid properties of HttpHost can be configured. |
| local-address | null | Local address for request execution. |
| stale-connection-check-enabled | false | Whether or not to use stale connection check. |
| cookie-spec | null | Name of the cookie specification for HTTP state management. |
| redirects-enabled | true | Whether or not redirects are handled automatically. |
| relative-redirects-allowed | true | Whether or not relative redirects are rejected. HTTP specification requires the location value be an absolute URI. |
| circular-redirects-allowed | false | Whether or not circular redirects (redirects to the same location) are allowed. |
| max-redirects | 50 | Maximum number of redirects to be followed. The limit on number of redirects is intended to prevent infinite loops. |
| authentication-enabled | true | Whether or not authentication is handled automatically. |
| target-preferred-auth-schemes | null | Order of preference for supported authentication schemes in the form of a list when authenticating with the target host. |
| proxy-preferred-auth-schemes | null | Order of preference for supported authentication schemes in the form of a list when authenticating with the proxy host. |
| connection-request-timeout | -1 | Timeout in milliseconds used when requesting a connection from the connection manager. |
| connect-timeout | -1 | Timeout in milliseconds until a connection is established. |
| socket-timeout | -1 | Socket timeout SO_TIMEOUT in milliseconds. The timeout for waiting for data, specifically, the maximum period of inactivity between two consecutive data packets. |
| content-compression-enabled | true | Whether or not the target server is asked to compress content. |
| normalize-uri | true | Whether or not the client should normalize URIs in requests. |
Other configurations
You can also configure a few optional parameters:
| Option | Default | Description |
|---|---|---|
| enable-compression | false | Whether the REST client should compress requests using gzip content encoding and add the “Accept-Encoding: gzip” header to receive compressed responses. |
| path-prefix | Empty String | Sets the path’s prefix for every request used by the HTTP client. |
| meta-header-enabled | false | Whether to send a X-Elastic-Client-Meta header that describes the runtime environment. |
| strict-deprecation-mode-enabled | false | Whether the REST client should return any response containing at least one warning header as a failure. |
| tls-config | null | An optional TLS configuration for connecting to the Elasticsearch cluster. The configuration options here are the same as Service section’s TLS config. |
| max-conn-total | 0 | Maximum total connections to the Elasticsearch cluster. |
| max-conn-per-route | 0 | Maximum connection per route to the Elasticsearch cluster. |
| conn-time-to-live-millis | -1 | Maximum time to live in milliseconds for persistent connections. |
| io-thread-count | Number of available processors | The number of I/O dispatch threads to be used by the I/O reactor. |
| user-agent | Defaults to aerospike-elasticsearch-outbound:{connector-version} where connector-version is the version of the deployed aerospike-elasticsearch-outbound connector. | The User-agent to be set with every request. |
| cookie-management-disabled | false | Disable state (cookie) management. |
| auth-caching-disabled | false | Disable authentication scheme caching. |
| connection-state-disabled | false | Disable connection state tracking. |
Example
...es-client: cluster-config: type: cloud cloud-id: abc123 default-headers: env: dev enable-compression: true path-prefix: /my/prefix meta-header-enabled: true strict-deprecation-mode-enabled: true auth-config: type: basic username: elastic password-file: /etc/aerospike-elasticsearch-outbound/secret/basic-auth-password.txt tls-config: trust-store: store-file: /etc/aerospike-elasticsearch-outbound/secret/tls/ca.my-domain.com.truststore.jks store-password-file: /etc/aerospike-elasticsearch-outbound/secret/tls/storepass transport-options: headers: over-tls: true parameters: my-param1: hello-world elasticsearch-request-config: expect-continue-enabled: true proxy: hostname: localhost port: 1234 scheme: http local-address: 192.168.123.234 stale-connection-check-enabled: true cookie-spec: test-cookie-spec redirects-enabled: false relative-redirects-allowed: false circular-redirects-allowed: false max-redirects: 5 authentication-enabled: true target-preferred-auth-schemes: - Basic - NTLM proxy-preferred-auth-schemes: - CredSSP connection-request-timeout: 2500 connect-timeout: 1000 socket-timeout: 10000 content-compression-enabled: false normalize-uri: false max-conn-total: 10 max-conn-per-route: 20 conn-time-to-live-millis: 30000 io-thread-count: 30 user-agent: my-test-webapp cookie-management-disabled: false auth-caching-disabled: true connection-state-disabled: true...