Issue
Error occurs on querying data from Azure Databricks database through Pega application.
Symptoms
Error
Could not initialize class
com.databricks.client.jdbc42.internal.apache.arrow.memory.RootAllocator
Impact
User is unable to display data from Azure Databricks database on Pega UI.
Steps to reproduce
Query data from Azure Databricks through Pega application.
Root Cause
EnableArrow is a JDBC connection parameter used by the Databricks JDBC driver. When EnableArrow is enabled (default in some driver versions), the driver transfers query results from Databricks to the user using the Apache Arrow format. While Apache Arrow improves data transfer speed, it also introduces additional native dependencies and increases memory requirements.
Note
When Arrow is enabled:
- The driver loads Arrow-related libraries internally.
- The Arrow initializes Arrow encoders and decoders within the com.databricks.client class.
If any Arrow component fails to load or initialize, the static initializer of com.databricks.client throws an exception. This results in the following error:
Could not initialize class com.databricks.client
Solution
Add the following parameter to the JDBC URL:
EnableArrow=0
Setting EnableArrow=0 completely disables Arrow. As a result, the JDBC driver reverts to the standard (non-Arrow) data transfer mechanism, which eliminates the dependency-related issue.
References
Requirements for creating external relational database instances