Deploy in Starburst Enterprise Platform on AWS

These instructions describe how to deploy the Aerospike Trino connector in Starburst Enterprise on AWS. Starburst Enterprise is an enterprise-ready distribution of open source Trino (formerly PrestoSQL).

The Aerospike connector does not support the following Starburst Enterprise features:

• Materialized Views
• Caching Service
• Atlas Integration
• Data Catalog - AWS Glue and Hive Metastore

Prerequisites

• A valid Starburst Enterprise license
• Python 3.8
• Java SE 11
• Verify the Trino version in the Starburst Enterprise release that you plan to use. For information on the corresponding connector release, see Deploying the Trino Connector and a Trino Cluster on Bare Metal or in VMs.

Provision EC2 instance

1. Log into the AWS Management Console, then select Services > Compute > EC2.

   

2. Select Launch Instances.

3. Select AMI > Ubuntu Image 20.04 XXX > 64 Bit(X86) > Select.

   

4. Select Choose Instance Type > t3.2xlarge.

   

5. Select Next: Configure Instance Details.

6. Go to Add Storage > Add New Volume and check the specified settings. When you have confirmed the settings, click Next: Add Tags.

   

7. Choose Add Tags > Add Tag then enter the appropriate value for Key and Value. For example, Development and Starburst_Node_Dev_X or QA and Starburst_Node_QA_X.

   

8. At the bottom of the screen, click Next: Configure Security Group.

9. If you need ports other than SSH, like port 8080 or 3000-3003, select Configure Security Group > Add Rule. Add rules to allow the necessary ports.

   

10. Click Review and Launch.

11. Review the settings, then click Launch.

12. Select one of the options:

    • Choose an existing key pair
    • Create a new key pair
    • Proceed without a key pair

    If existing keys are available, select Choose an existing key pair, select the Public/Private key, tick the box to confirm you have access, then click Launch Instances.

    

Deploy Starburst and Aerospike

Pick the public IP address of the recently-created instance before proceeding to the next steps from the AWS console. For the following examples, we assume the IP address is 192.0.2.0.

Note

The following steps have been tested in Terminal on Mac.

1. Use PuTTY (Windows) or Terminal (Mac) to log into the configured instance.

   ssh -i ~/Downloads/Private_Key_Latest.pem ubuntu@192.0.2.0

2. Install Java and Python, if they are not pre-configured on your instance.

   sudo apt update
   sudo apt-get install openjdk-11-jdk
   sudo apt install python3.8

3. Download Starburst Enterprise LTS version.

4. Upload it to the /home/ubuntu directory on your instance.

5. Go to the /home/ubuntu directory on your instance and install UUID.

   cd /home/ubuntu
   sudo apt-get install uuid

6. Run the UUID command and capture the node ID for node.properties.

   uuid

7. Upload all relevant files to the home/ubuntu directory, including the following:

   • A valid Starburst Enterprise license.
   • The Starburst Enterprise download in .tar.gz format.

8. Make the /etc/sep directory and copy over your license key.

   sudo mkdir -p /etc/sep
   cd /etc/sep
   sudo cp /home/ubuntu/*.license .

9. Move to the /home/ubuntu directory, then uncompress the Starburst installation package.

   cd /home/ubuntu
   tar xvfz starburst-enterprise-370-e.1.tar.gz

10. Move the uncompressed Starburst file to /opt/starburst, then go to that directory and create a symlink.

    sudo mv starburst-enterprise-370-e.1 /opt/starburst
    cd /opt/starburst
    sudo ln -s /etc/sep etc

11. Go to the /opt/starburst/plugin directory and make a directory for Aerospike.

    cd /opt/starburst/plugin/
    sudo mkdir aerospike

12. Copy all the JAR files from the Aerospike Trino Connector download into the new /opt/starburst/plugin/aerospike directory.

13. Create a /var/trino/data directory and give it read/write/execute permissions for all users.

    mkdir -p /var/trino/data
    sudo chmod 777 /var/trino/data

14. Move into the /etc/sep directory and edit the node.properties file.

    /etc/sep
    sudo vi node.properties

15. Set the following configurations. For node.id use the UUID you captured previously with the uuid command.

    node.environment=production
    node.id=XXXXXXXXXXXX ## UUID captured previously
    node.data-dir=/var/trino/data

16. Save and exit the file.

17. Edit jvm.config.

    sudo vi jvm.config

18. Configure Trino in Starburst Enterprise. For more information, see Configuring Trino.

    Here’s a sample configuration:

    -server
    -Xmx16G
    -XX:-UseBiasedLocking
    -XX:+UseG1GC
    -XX:G1HeapRegionSize=32M
    -XX:+ExplicitGCInvokesConcurrent
    -XX:+ExitOnOutOfMemoryError
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:-OmitStackTraceInFastThrow
    -XX:ReservedCodeCacheSize=512M
    -XX:PerMethodRecompilationCutoff=10000
    -XX:PerBytecodeRecompilationCutoff=10000
    -Djdk.attach.allowAttachSelf=true
    -Djdk.nio.maxCachedBufferSize=2000000

19. Save the file and exit.

20. Edit config.properties.

    vi config.properties

21. Set the cluster configurations, using one of the two following options.

    Option 1: Single-Node Deployment

    coordinator=true
    node-scheduler.include-coordinator=true
    http-server.http.port=8080
    query.max-memory=50GB
    query.max-memory-per-node=1GB
    discovery.uri=http://[coordinator node IP address]:8080

    Option 2: Multi-Node Deployment

    If you plan to use one or more worker nodes, one set of configurations applies to the coordinator node, and a second set applies to each worker node.

    On the coordinator node:

    coordinator=true
    node-scheduler.include-coordinator=true
    http-server.http.port=8080
    query.max-memory=50GB
    query.max-memory-per-node=1GB
    discovery.uri=http://[coordinator node IP address]:8080

    On each worker node:

    coordinator=false
    query.max-memory=50GB
    query.max-memory-per-node=1GB
    discovery.uri=http://[coordinator node IP address]:8080

    Tip

    You can use any DevOps automation tool to automate applying configurations to worker nodes.

22. Save and exit the config.properties file.

23. Edit log.properties.

    sudo vi log.properties

24. Set the following configuration:

    io.trino=INFO

25. Save the file and exit.

26. Create a catalog directory and move into it.

    sudo mkdir catalog
    cd catalog

27. Create a file named aerospike.properties.

    sudo vi aerospike.properties

28. Set your configurations. For available settings, see Configuration Properties for Aerospike Connect for Trino.

    Example configuration file:

    connector.name=aerospike
    aerospike.hostlist=xx.xx.xx.xx:3000,yy.yy.yy.yy:3000,zz.zz.zz.zz:3000
    ## aerospike node list
    aerospike.split-number=8
    aerospike.strict-schemas=false
    aerospike.record-key-hidden=false
    aerospike.enable-statistics=true
    aerospike.insert-require-key=true
    aerospike.table-desc-dir=/etc/trino/aerospike
    aerospike.clientpolicy.tls.enabled=false

29. Save the file and exit.

    By default, the Trino connector uses heuristics to rapidly infer schemas without the need for you to specify them. However, you can choose to specify the schema by creating a table definition. For more information, see Specifying Trino Schemas.

30. Go to /usr/bin then run the Starburst launcher.

    cd /usr/bin
    python3 /opt/starburst/bin/launcher.py run

    

31. Watch the Server Started messages and troubleshoot any errors.

    Although we have tested connecting to Starburst Enterprise with DBeaver, we encourage you to try out other Starburst clients based on your needs.

For additional information on how to develop SQL queries to generate insights from Aerospike data, see the following:

• SQL Support for the Trino Connector
• Examples of Querying Aerospike Databases