0

I have a Hive Udf that works well in hive terminal, What I want i want to execute it via shell script. On hive terminal i am able to execute following commands :

use mashery_db;
add jar hdfs://nameservice1/tmp/nextdata_aggregations/custom_jar/readerCheck.jar;
add file hdfs://nameservice1/tmp/GeoLite2-City.mmdb;
CREATE TEMPORARY FUNCTION geoip AS 'com.mashery.nextdata.hive.udf.GeoIPGenericUDF';

But when I am adding the above code in shell script

hive -e "use mashery_db;"
hive -e "add jar hdfs://nameservice1/tmp/nextdata_aggregations/custom_jar/readerCheck.jar;"
hive -e "add file hdfs://nameservice1/tmp/GeoLite2-City.mmdb;"
hive -e "CREATE TEMPORARY FUNCTION geoip AS 'com.mashery.nextdata.hive.udf.GeoIPGenericUDF';"

The 1st 'hive -e' works well and adds the jar but the last one create temporary function doesn't work. I am getting below error:

FAILED: ParseException line 1:35 mismatched input 'com' expecting StringLiteral near 'AS' in create function statement

I have also tried with single quotes

hive -e "CREATE TEMPORARY FUNCTION geoip AS 'com.mashery.nextdata.hive.udf.GeoIPGenericUDF';"

then I am getting FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask

FAILED: Class com.mashery.nextdata.hive.udf.GeoIPGenericUDF not found

Does hive Udf supports shell script ,if it does whats wrong I am doing. Thanks in advance

Mohit Rane
  • 279
  • 7
  • 23
  • 3
    Try importing the jar and creating the function all in one call to hive. i.e. `hive -e "add jar path_to_jar/foo.jar; create temporary function foo as 'com.package.UDF';"` – o-90 May 13 '16 at 18:28
  • @GoBrewers14 thanks it worked for me :-) – Mohit Rane May 16 '16 at 07:11

2 Answers2

1

Each invocation of hive -e spawns a new process with a new hive shell that has no memory of what the previous one did, so hive 'forgets' where the UDF is... One solution is to chain them in just one command, but it's better form to put all your hive commands in a file (for instance "commands.hql") and use hive -f commands.hql instead of -e.

File would look like this:

use mashery_db;
add jar hdfs://nameservice1/tmp/nextdata_aggregations/custom_jar/readerCheck.jar;
add file hdfs://nameservice1/tmp/GeoLite2-City.mmdb;
CREATE TEMPORARY FUNCTION geoip AS 'com.mashery.nextdata.hive.udf.GeoIPGenericUDF';"
Roberto Congiu
  • 5,123
  • 1
  • 27
  • 37
0

You can get this to work with both hive -e and hive -f:

hive -e "use mashery_db;
add jar hdfs://nameservice1/tmp/nextdata_aggregations/custom_jar/readerCheck.jar;
add file hdfs://nameservice1/tmp/GeoLite2-City.mmdb;
CREATE TEMPORARY FUNCTION geoip AS 'com.mashery.nextdata.hive.udf.GeoIPGenericUDF';"

Creating them as a file and using hive -f hive_file.hql would work as well.

Aaron Faltesek
  • 319
  • 2
  • 11