Skip to main content
Aerospike Vector is now live with LangChain integrationTell me more
Loading

Modifying an Existing Configuration of the Kafka Sink Connector

You can modify the configuration of the Kafka inbound (sink) connector even after the connector is deployed. In both standalone and distributed modes, the first step is to edit the configuration file.

Edit the configuration file​

Edit the file KAFKA_DIR/etc/aerospike-kafka-inbound.yml. See configuration for details.

Standalone Mode​

Kill the running connector​

  1. Use ps aux to list the processes running on the system and to locate the JVM process that is running the connector.
  2. Note the ID of the process.
  3. Send a kill signal to the process by running this command:
kill -9 PROCESS_ID

Restart the connector​

Run the following command:

KAFKA_DIR/bin/connect-standalone.sh KAFKA_CONNECT_CONFIG_PATH AEROSPIKE_SINK_PROPERTIES_PATH
  • KAFKA_DIR: The directory where the Kafka package is located.
  • KAFKA_CONNECT_CONFIG_PATH: The path to the file (including the filename and extension) that you are using to configure the worker in Kafka Connect.
  • AEROSPIKE_SINK_PROPERTIES_PATH: The path to the file (including the filename and extension) that you created when you deployed the connector. See "Standalone mode" in step 4 of "Deploying the Kafka Inbound Connector".

Distributed Mode​

View existing config​

To view the current configuration that is being used by the connector, issue this request to Kafka Connect's REST interface:

GET /connectors/aerospike-sink/tasks HTTP/1.1
Host: HOSTNAME_OR_IP

Use the hostname or IP address of any of the Kafka Connect nodes. As stated in the Connect REST Interface page of the Kafka Connect documentation, you can make requests to any cluster member; the REST API automatically forwards requests if required.

Copy the edited file​

Copy the edited configuration file to each of the other Kafka Connect nodes, replacing the previous version of the file.

Update the connector​

  1. Set the aerosink variable, filling in the values as necessary.
    aerosink =
    {
      "connector.class": "com.aerospike.connect.kafka.inbound.AerospikeSinkConnector",
      "config-file": "/etc/aerospike-kafka-inbound/inbound.yml",
      "tasks.max": "",
      "topics": ""
    }
  • config-file: Path to a YAML or JSON file containing the inbound connector configuration.
    • Instead of config-file, you can also use config as a key and pass the raw JSON to it directly instead of using an external file.
  • tasks.max: The maximum number of tasks that can be created for the connector. A task runs as a process in Kafka Connect.
  • topics: A list of comma-separated names of the topics for the connector to subscribe to.

  1. Set this variable:

    kafkaEndpoint="URI"

    kafkaEndpoint: This is the REST endpoint for the Kafka Connect service. You can make requests to any cluster member; the REST API automatically forwards requests, if required.

  2. Issue a request to Kafka Connect's REST interface. The request updates all of the connector tasks together.

    curl -X PUT --header "Content-Type:application/json" --data ${aerosink} ${endpoint}/connectors/aerospike-sink/config

Verify the changes​

To verify, issue the same request that you issued in step 1 to view the configuration.

Last updated on