Apache Spark with Hive on Eclipse IDE throw the privilege error - A read-only database issue

Question

I try to test Apache Spark with Hive integration on Eclipse IDE. These are the versions of each projects - Hadoop 2.7.4, Spark 2.2 and hive-2.3.2 with MySQL 5.7 on Eclipse Mars IDE. The contents of hive-site.xml is like below,

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value> 
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>password</value>
</property>
<property>
  <name>hive.metastore.port</name>
  <value>33306</value>
</property>
<property>
    <name>hive.server2.enable.doAs</name>
    <value>false</value>
</property>

And the spark-hive Java API code on Eclipse IDE are

public static void main(String[] args) {
        Logger.getLogger("org").setLevel(Level.OFF);
        Logger.getLogger("akka").setLevel(Level.OFF);

        // TODO Auto-generated method stub
        SparkSession spark = SparkSession.builder()
                .appName("Spark Hive Test")
                .master("local[*]")
                .config("spark.sql.warehouse.dir","/user/hive/warehouse")
                .enableHiveSupport()
                .getOrCreate();

        System.out.println(spark.version());

        String tableName = "testHiveDriverTable";
      spark.sql("drop table if exists " + tableName);
      String createTable = "create table " + tableName + " (key int, value string) "
                + "ROW FORMAT DELIMITED "
                + "FIELDS TERMINATED BY ',' "
                + "LINES TERMINATED BY '\n' "
                + "STORED AS TEXTFILE";

        spark.sql(createTable);
      spark.sql("LOAD DATA LOCAL INPATH '/home/m_usr/temp/hiveTest.txt' INTO TABLE " + tableName);

      // Queries are expressed in HiveQL
      spark.sql("SELECT * FROM " + tableName).show();

      // Aggregation queries are also supported.
      spark.sql("SELECT COUNT(*) FROM " + tableName).show();


        spark.stop();
        spark.close();
    }

This code throw the exception:

java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
    at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown Source)
    at com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324)
    at com.jolbox.bonecp.ConnectionHandle.<init>(ConnectionHandle.java:262)
    at com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:115)
    at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Caused by: ERROR 25505: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
    at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
    at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
    at org.apache.derby.impl.sql.conn.GenericAuthorizer.setReadOnlyConnection(Unknown Source)
    at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.setReadOnly(Unknown Source)
    ... 8 more

But when Eclipse IDE be executed on root mode, no exceptions are thrown. I think this issue is related with privileges problem. However I have no idea which process bring this issue. And one more issue is the exception is throws from Apache Derby, not MySQL. My MySQL configuration on hive-site.xml seems to be wrong.

Hi Joseph. I believe I have edited a number of your posts in the past to (1) correct case errors, (2) remove chatty/pleading material like _Your any advice will be deeply appreciated. Waiting for any reply. Thanks in advanced_. In the past, I may even have asked you to make your questions a bit more succinct. However, I noticed this one in the recent question feed, and still the same sorts of edits are necessary. Would you take note of the edits made to your posts, and post new questions accordingly please? It seems you're creating work for volunteers at present. — halfer, Jan 08 '18 at 11:27
Your English seems fine, just asking for the chat to be reduced. Thanks! — halfer, Jan 10 '18 at 14:36

Apache Spark with Hive on Eclipse IDE throw the privilege error - A read-only database issue

0 Answers0