I want to copy data from one DB table to another DB table using hive on EMR. Below is the HQL using which I'm copying data along with the date partition.
insert into Target.exttbl_user_identification_details PARTITION(load_date="2018-04-23") select * from Source.exttbl_user_identification_details;
Error Log: (States, Previous writer likely failed to write... so I am unlikely to write too)
INFO : Compiling command(queryId=hive_20180424083045_4bf96c97-dbab-4578-8e1b-a1c2b02e1559): insert into Target.exttbl_user_identification_details PARTITION(load_date="2018-04-23") select * from Source.exttbl_user_identification_details
INFO : Semantic Analysis Completed
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:exttbl_user_identification_details.user_id, type:varchar(70), comment:null), FieldSchema(name:exttbl_user_identification_details.signed_up_flag, type:varchar(20), comment:null), FieldSchema(name:exttbl_user_identification_details.identifier, type:varchar(70), comment:null)], properties:null)
INFO : Completed compiling command(queryId=hive_20180424083045_4bf96c97-dbab-4578-8e1b-a1c2b02e1559); Time taken: 0.838 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryId=hive_20180424083045_4bf96c97-dbab-4578-8e1b-a1c2b02e1559): insert into Target.exttbl_user_identification_details PARTITION(load_date="2018-04-23") select * from Source.exttbl_user_identification_details
INFO : Query ID = hive_20180424083045_4bf96c97-dbab-4578-8e1b-a1c2b02e1559
INFO : Total jobs = 1
INFO : Launching Job 1 out of 1
INFO : Starting task [Stage-1:MAPRED] in serial mode
INFO : Tez session hasn't been created yet. Opening session
ERROR : Failed to execute tez graph.
java.io.IOException: Previous writer likely failed to write hdfs://ip-*-*-*-*.ec2.internal:8020/tmp/hive/ganesh/_tez_session_dir/ea5b020d-deb8-4846-947f-db52c5b9b884/hive-hcatalog-core-2.1.1-amzn-0.jar. Failing because I am unlikely to write too.
at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:1022) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:902) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:845) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:466) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:252) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager$TezSessionPoolSession.openInternal(TezSessionPoolManager.java:622) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:206) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:283) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:155) [hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) [hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073) [hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744) [hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453) [hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171) [hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1166) [hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242) [hive-service-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91) [hive-service-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:334) [hive-service-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_141]
at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_141]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) [hadoop-common-2.7.3-amzn-2.jar:?]
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:347) [hive-service-2.1.1-amzn-0.jar:2.1.1-amzn-0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_141]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
INFO : Completed executing command(queryId=hive_20180424083045_4bf96c97-dbab-4578-8e1b-a1c2b02e1559); Time taken: 25.045 seconds+
Please suggest if I'm missing any configuration or anything here.
