0

I am trying to load json file using json serde. I have added the serde jar file successfully.

1) My json jar file placed on path /apps/hive/warehouse/lib/ I have run this command successfully

add jar hdfs:///apps/hive/warehouse/lib/json-serde-1.3-jar-with-dependencies.jar; converting to local hdfs:///apps/hive/warehouse/lib/json-serde-1.3-jar-with-dependencies.jar Added [/tmp/6f1a54b9-65c4-4e32-8e87-25d60ef775c6_resources/json-serde-1.3-jar-with-dependencies.jar] to class path Added resources: [hdfs:///apps/hive/warehouse/lib/json-serde-1.3-jar-with-dependencies.jar]

2) Now when i am trying to upload json file on this path /apps/hive/warehouse/lib/ or tmp/ using ambari GUI. I am unable to upload it gives error 500. see attached imageerror while uploading json file 3) I have also tried this command but beacuse i am unable to upload json file. so, it doesnot work

hadoop fs -put tmp/test.json /apps/hive/warehouse/lib/test.json

Kindly help me solving this issue

user4440416
  • 71
  • 1
  • 11

1 Answers1

0

To load the file into hive, first copy the file to an HDFS Location. Like below:

hadoop fs -put /complete_path_to_my_json/json_to_upload.json /app/hive/a_temp_location

Then once you have the table created with JSON SerDe like below:

create table if not exists my_json_table (id int, name string, designation string) row format serde 'org.apache.hive.hcatalog.d.JsonSerDe';

You can load the data into the table with the following statement:

load data inpath '/app/hive/a_temp_location/json_to_upload.json' into table my_json_table;
James Jithin
  • 10,183
  • 5
  • 36
  • 51
  • My json file name test.json placed on desktop in my pc . now i am trying to upload it on hdfs directory using ambari gui it gives error 500 and not uploading on it. kindly see attached image i have edited question – user4440416 Apr 11 '19 at 09:51
  • Mostly it could be with the permission to upload to the directory. Please refer - https://community.hortonworks.com/questions/115942/unable-to-load-json-file-via-ambari.html – James Jithin Apr 11 '19 at 11:57
  • unable to understand.. please help how to check if maria_dev has write permission on that directory or not? – user4440416 Apr 11 '19 at 15:03
  • Try to put a file to the location from console as maria_dev and see what happens! – James Jithin Apr 11 '19 at 15:05
  • From the console where you tried hadoop fs commands, execute the following: touch /tmp/testfile && hadoop fs -put /tmp/testfile /apps/hive – James Jithin Apr 11 '19 at 15:24
  • touch /tmp/test.json && hadoop fs -put /tmp/test.json /apps/hive/warehouse/lib/test.json .. this command executed without any error on shell – – user4440416 Apr 11 '19 at 15:59
  • Alright. Then try moving the file using the hadoop fs -put command to an HDFS location and then load it into the hive table from there. – James Jithin Apr 11 '19 at 16:00
  • Please try again to a different destination. The response doesn't seem valid. – James Jithin Apr 11 '19 at 16:12
  • You can provide the permission if you know the location where ambari tries to upload. – James Jithin Apr 11 '19 at 17:14