I am trying to connect my spark thrift server using play framework via anorm.
- Play framework version - 2.8.18
- Connecting to jdbc as explained here
- Anorm - 2.7.0
- Spark thrift server version - spark 3.2.1, Hadoop 3.2
- Scala - 2.13.5
Jdbc driver & dependencies I have
Exception I am getting as response when I hit url
"java.sql.SQLFeatureNotSupportedException: Method not supported",
"at org.apache.hive.jdbc.HiveResultSetMetaData.getTableName(HiveResultSetMetaData.java:102)",
"at anorm.MetaData$.tableName$1(MetaData.scala:148)",
"at anorm.MetaData$.$anonfun$parse$1(MetaData.scala:153)",
"at anorm.MetaData$.$anonfun$parse$1$adapted(MetaData.scala:139)",
"at scala.collection.immutable.List.map(List.scala:246)",
"at anorm.MetaData$.parse(MetaData.scala:139)",
"at anorm.Cursor$.apply(Cursor.scala:27)",
"at anorm.Sql$.unsafeCursor(Anorm.scala:250)",
"at anorm.Sql$.$anonfun$withResult$1(Anorm.scala:256)",
"at scala.Function1.$anonfun$andThen$1(Function1.scala:85)",
"at resource.AbstractManagedResource.$anonfun$acquireFor$1(AbstractManagedResource.scala:85)",
"at scala.util.control.Exception$Catch.$anonfun$either$1(Exception.scala:251)",
"at scala.util.control.Exception$Catch.apply(Exception.scala:227)",
"at scala.util.control.Exception$Catch.either(Exception.scala:251)",
"at resource.AbstractManagedResource.acquireFor(AbstractManagedResource.scala:85)",
"at resource.ManagedResourceOperations$$anon$1.$anonfun$acquireFor$1(ManagedResourceOperations.scala:47)",
"at resource.AbstractManagedResource.$anonfun$acquireFor$1(AbstractManagedResource.scala:85)",
"at scala.util.control.Exception$Catch.$anonfun$either$1(Exception.scala:251)",
"at scala.util.control.Exception$Catch.apply(Exception.scala:227)",
"at scala.util.control.Exception$Catch.either(Exception.scala:251)",
"at resource.AbstractManagedResource.acquireFor(AbstractManagedResource.scala:85)",
"at resource.ManagedResourceOperations$$anon$1.acquireFor(ManagedResourceOperations.scala:47)",
"at resource.DeferredExtractableManagedResource.acquireFor(AbstractManagedResource.scala:25)",
"at resource.ManagedResourceOperations.apply(ManagedResourceOperations.scala:31)",
"at resource.ManagedResourceOperations.apply$(ManagedResourceOperations.scala:31)",
"at resource.DeferredExtractableManagedResource.apply(AbstractManagedResource.scala:22)",
"at resource.ManagedResourceOperations.acquireAndGet(ManagedResourceOperations.scala:29)",
"at resource.ManagedResourceOperations.acquireAndGet$(ManagedResourceOperations.scala:29)",
"at resource.DeferredExtractableManagedResource.acquireAndGet(AbstractManagedResource.scala:22)",
"at anorm.Sql$.$anonfun$asTry$1(Anorm.scala:263)",
"at scala.util.Try$.apply(Try.scala:210)",
"at anorm.Sql$.asTry(Anorm.scala:263)",
"at anorm.WithResult.asTry(SqlResult.scala:215)",
"at anorm.WithResult.asTry$(SqlResult.scala:213)",
"at anorm.SimpleSql.asTry(SimpleSql.scala:6)",
"at anorm.WithResult.as(SqlResult.scala:205)",
"at anorm.WithResult.as$(SqlResult.scala:204)",
"at anorm.SimpleSql.as(SimpleSql.scala:6)",
"at v2.thrift.ThriftQueryServiceImpl.$anonfun$getDataV2$5(ThriftQueryServiceImpl.scala:33)",
"at scala.collection.immutable.List.map(List.scala:246)",
"at scala.collection.immutable.List.map(List.scala:79)",
"at v2.thrift.ThriftQueryServiceImpl.$anonfun$getDataV2$4(ThriftQueryServiceImpl.scala:32)",
"at play.api.db.DefaultDatabase.withConnection(Databases.scala:180)",
"at play.api.db.DefaultDatabase.withConnection(Databases.scala:174)",
"at v2.thrift.ThriftQueryServiceImpl.$anonfun$getDataV2$3(ThriftQueryServiceImpl.scala:31)",
"at play.api.cache.redis.RecoverWithDefault.recoverFrom(RecoveryPolicy.scala:111)",
"at play.api.cache.redis.RecoverWithDefault.recoverFrom$(RecoveryPolicy.scala:109)",
"at play.api.cache.redis.LogAndDefaultPolicy.play$api$cache$redis$Reports$$super$recoverFrom(RecoveryPolicy.scala:125)",
"at play.api.cache.redis.Reports.recoverFrom(RecoveryPolicy.scala:70)",
"at play.api.cache.redis.Reports.recoverFrom$(RecoveryPolicy.scala:66)",
"at play.api.cache.redis.LogAndDefaultPolicy.recoverFrom(RecoveryPolicy.scala:125)",
"at play.api.cache.redis.impl.dsl$RecoveryFuture$$anonfun$recoverWithFuture$1.applyOrElse(dsl.scala:33)",
"at play.api.cache.redis.impl.dsl$RecoveryFuture$$anonfun$recoverWithFuture$1.applyOrElse(dsl.scala:31)",
"at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:454)",
"at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63)",
"at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100)",
"at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)",
"at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)",
"at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100)",
"at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49)",
"at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)",
"at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)",
"at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)",
"at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)",
"at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)",
"at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)"
Seems like anorm is calling function HiveResultSetMetaData.getTableName. When I checked this in hive jdbc driver source code it is not implemented and throws an exception.
I am able to connect to thrift server properly from a plain Java class below with above dependency.
package org.example;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
public class HiveJDBCConnect {
public static void main(String[] args) {
Connection con = null;
try {
String conStr = "jdbc:hive2://spark-thrift:10000/test_schema;auth=noSasl";
Class.forName("org.apache.hive.jdbc.HiveDriver");
con = DriverManager.getConnection(conStr, "root", "root");
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery("select key2 from ir_test");
while(rs.next()) {
System.out.print(rs.getString(1));
System.out.println();
}
System.out.println("show database successfully.");
} catch (Exception ex) {
ex.printStackTrace();
} finally {
try {
if (con != null)
con.close();
} catch (Exception ex) {
}
}
}}
I tried with the following driver as well with no luck.
- spark-project-hive-jdbc - This one even connection is not getting due to the issue mentioned here from connection pool I am using.
How can I resolve this ? Is there any driver out there which can connect to spark thrift server which has these method defined ?
