Databricks Connection Steps

GenRocket can connect to a Databricks cluster or SQL Warehouse via JDBC as long as the Databricks JDBC driver is available.

Step 1 - Create a JDBC Config File (config.properties)

For GenRocket to connect to Databricks via JDBC, you must create a JDBC Config File (config.properties). Any application can connect to Databricks by reading the information included in this file.


A template and example are shown below.

 

Template:

driver=<JDBCDriverClassName>
user=<user_or_token> (Optional)
password=<personal_access_token> (Optional)
url=jdbc:databricks://<server-hostname>:443/<schema>;transportMode=http;ssl=1;httpPath=<http-path>[;additional-properties]
batchCount=1000


Example 1: Using URL-based Authentication

GenRocket will read this config file and connect to a Databricks SQL Warehouse using a Personal Access Token.


driver=com.databricks.client.jdbc.Driver
url=jdbc:databricks://someid.cloud.databricks.com:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/warehouses/abcd1234efgh5678;PWD=<personal-access-token>;UID=token;ConnCatalog=grcat
batchCount=1000

 

Example 2 - Using user/password Properties (Supported)

GenRocket can also use user and password properties for authentication.


driver=com.databricks.client.jdbc.Driver
user=token
password=<personal-access-token>
url=jdbc:databricks://some-id.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath=/sql/1.0/warehouses/abcd1234efgh5678
batchCount=1000

 

Important Notes on Authentication

  • URL-based authentication is recommended and works consistently across environments.
  • user/password authentication is supported, but may depend on:
    • JDBC driver version
    • Runtime environment behavior
  • If connection issues occur when using user/password authentication, switch to URL-based authentication.

 

Notes:

  • server-hostname: Found in your Databricks workspace URL
  • httpPath: Found in your SQL Warehouse or cluster connection details
  • schema: Typically default, unless using a custom schema

Authentication Parameters (URL-based)

  • AuthMech=3 - Enables token-based authentication
  • UID=token - Indicates token usage
  • PWD - Databricks Personal Access Token

Additional Parameters

  • ConnCatalog - (Optional) Catalog name
  • transportMode=http + ssl=1 - Required


Step 2 - (Optional) Encrypt the JDBC Config file

You can encrypt this file for additional security.

genrocket -ef  <pathToConfigFile>


Step 3 - Update Your JDBC Organization Resource Value

The resource.jdbc.directory Organization Resource specifies the location of the JDBC Config file on the logged-in user's local machine.

This resource value should contain the full directory path.

/home/user/dev


Step 4 - Update your Profile in the .genrocket Folder

Each user has a profile stored in the .genrocket folder on their local machine. You must download and replace your profile (in ~/.genrocket folder). GenRocket will read this resource value from your profile to find the path to the JDBC Config file on your system establish a connection to Databricks.


This can be done in one of two ways: 

Step 5 - Download and place the JDBC connector Jar file in the $GENROCKET_HOME/lib/ folder

  • Download and place the Databricks JDBC Driver in your genrocket/lib folder.
  • Ensure the driver version is compatible with your JDK.


Step 6 - Test if GenRocket Runtime can connect to Databricks

You can test the JDBC connection before referencing the resource in GenRocket using:

genrocket -tjdbc <pathToConfigFile>


Step 7 - Reference the JDBC Resource in your Generators or Receivers

When you create a Scenario with a Generator or Receiver that references your resource.jdbc.directory resource, it will connect directly to your database. Any GenRocket feature that uses a database connection must also reference this file.

 

Troubleshooting

Connection Issues

  • Verify:
    • Personal Access Token is valid
    • SQL Warehouse or cluster is running
    • httpPath is correct
    • Port 443 is open

 

Driver Not Found

If you see errors like:

java.lang.ClassNotFoundException: com.databricks.client.jdbc.Driver

  • Ensure the JDBC driver .jar file is placed in:

$GENROCKET_HOME/lib/