0

I am getting below error while trying to create external table in Hive from Spark and saving the contents .

 The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rw-rw-rw-

I even tried setting permission to /tmp/hive folder to chmod 777 but no success.

Amit_Hora
  • 716
  • 1
  • 8
  • 27
  • Check this similar question. I had the same issue when running on Windows and the root cause was the incorrect version of winutils. http://stackoverflow.com/questions/33234311/spark-1-5-1-spark-shell-throws-runtimeexception – J Maurer Apr 17 '16 at 14:58

3 Answers3

0

You might be wrong about giving permission because I think you gave permission to local /tmp/hive directory but you should give permission to /tmp/hive in HDFS

hadoop fs -chmod 777 /tmp/hive
Tarık Yılmaz
  • 369
  • 6
  • 10
  • I already did that ,and also gave full control for /tmp/hive folder that is getting creating where i have eclipse installed as i am running Spark in local mode from IDE – Amit_Hora Apr 15 '16 at 17:43
0

Check in hive-site.xml if you are using hdfs, cfs or local filesystem first of all.

For cfs, you will have to use sudo dse hadoop fs -chmod 777 /tmp/hive

For hdfs, you will have to use sudo hdfs dfs -chmod 777 /tmp/hive or sudo hadoop fs -chmod 777 /tmp/hive

And local i guess, you already took care of.

Abhishek Anand
  • 1,940
  • 14
  • 27
  • Location of Hive warehouse is /apps/hive/warehouse But /tmp/hive folder is also getting created in Drive where i Have eclipse installed – Amit_Hora Apr 15 '16 at 17:43
  • Set this parameter in your scala code then "spark.eventLog.dir" while creating spark context, and point it to some proper directory. Refer to http://spark.apache.org/docs/latest/configuration.html – Abhishek Anand Apr 15 '16 at 18:01
0

The better way to set scratch directory is /user/${username}/tmp/hive in hive-site.xml. This way all the query details will be written to submitted user scratch dir. In that you case you do not need to change permissions tot he scratch dir. Otherwise global scratch dir should be writable by all users.

Raju Bairishetti
  • 344
  • 2
  • 7