4

Hive query output that is using UDFs consists of these 2 warnings at the end. How do I suppress these 2 warnings. Please note that the 2 warnings come right after the output as part of output.

WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
WARN: Please see http://www.slf4j.org/codes.html#release for an explanation.

hadoop version

Hadoop 2.6.0-cdh5.4.0

hive --version

Hive 1.1.0-cdh5.4.0
visakh
  • 2,503
  • 8
  • 29
  • 55
user3441798
  • 41
  • 1
  • 4

3 Answers3

1

If you use beeline instead of Hive the error goes away. Not the best solution, but I'm planning to post to the CDH user group asking the same question to see if it's a bug that can be fixed.

secastro
  • 11
  • 1
  • Did you get any update from CDH on this warning issue? I use Hadoop 2.6.0-cdh5.12.2 version and issue still persists. – Ash Mar 20 '18 at 23:07
0

This error occurs due to adding of assembly jar which which contains classes from icl-over-slf4j.jar (which is causing the stdout messages) and slf4j-log4j12.jar.

You can try couple of things to begin with:

  1. Try removing the assembly jar, in case if using.

  2. Look at the following link: https://issues.apache.org/jira/browse/HIVE-12179 This suggest that we can trigger a flag in Hive where spark-assembly is loaded only if HIVE_ADD_SPARK_ASSEMBLY = "true".

https://community.hortonworks.com/questions/34311/warning-message-in-hive-output-after-upgrading-to.html : Although there is a workaround if to avoid any end time changes and that is to manually remove the 2 lines from the end of the files using shell script.

Deepesh Rehi
  • 883
  • 8
  • 31
0

Have tried to set HIVE_ADD_SPARK_ASSEMBLY=false, but it didn't work.

Finally, I found a post question at Cloudera community. See: https://community.cloudera.com/t5/Support-Questions/Warning-message-in-Hive-output-after-upgrading-to-hive/td-p/157141

You could try the follow command, it works for me!

hive -S -d ns=$hiveDB -d tab=$t -d dunsCol=$c1 -d phase="$ph1" -d error=$c2 -d ts=$eColumnArray -d reporting_window=$rDate -f $dir'select_count.hsql' | grep -v "^WARN" > $gOutPut 2> /dev/null
Wang Zhong
  • 125
  • 2
  • 9