1

I wrote a UDF which uses Joda Time. I included it as a dependency in pom.xml. When I run my pig script I get the error

ERROR 2998: Unhandled internal error. org.joda.time.LocalDate.parse(Ljava/lang/String;)Lorg/joda/time/LocalDate;

I am pretty new to java, maven and hadoop. How can I bundle joda into my jar (Like statical linked in c/c++) which I run on the cluster?

This is NOT a duplicate of how to include external jar file using PIG since I cant use -D option and registering Joda Time in the pig script has no effect.

Community
  • 1
  • 1
ManuelSchneid3r
  • 15,850
  • 12
  • 65
  • 103
  • possible duplicate of [how to include external jar file using PIG](http://stackoverflow.com/questions/10423990/how-to-include-external-jar-file-using-pig) – NotGaeL Jun 27 '15 at 18:57
  • No my UDF is included and runs. – ManuelSchneid3r Jun 27 '15 at 18:58
  • your code is throwing a java.lang.NoClassDefFoundError exception. Find the jar that is missing (joda-time) then include it as a runtime dependency on your pig script. – NotGaeL Jun 27 '15 at 19:13
  • @elcodedocle Parameter -D is not accepted on my macine (0.13). – ManuelSchneid3r Jun 27 '15 at 19:27
  • A clear case of Stackoverflowitis. Down and closevoting, while ignoring the content of the question in an arrogant way, referring to an other, apparently duplicate questing containing some similiar words. Thank you Fred for actually reading the question. – ManuelSchneid3r Jun 28 '15 at 13:00
  • A close vote is not arrogant, it's a "hey look I found a very similar question here!" Neither was my suggestion, which was wrong but was more than reasonable. I try to help you, you complain. That is rude, man. Also "No my UDF is included and runs." is an incoherent answer to my duplicate suggestion. "registering Joda Time in the pig script has no effect" makes more sense and of course I would have retired my duplicate vote after reading it, but I am not on a 24h watch to monitor and answer your question neither know everything about your problem (that's why you need 5 votes to close BTW). – NotGaeL Jun 28 '15 at 22:52
  • I have to say I am sorry. I did not want to be offensive, especially not to you. I am influenced by the last few questions I asked where people silently downvoted. I just think that people trigger the close vote based on premature decisions – ManuelSchneid3r Jun 28 '15 at 23:00
  • Ok, cool. I didn't downvote you or anything I just thought your question might have been answered before and wanted to help. I am glad you were able to solve your problem and hope I can be of more help next time :-) – NotGaeL Jun 28 '15 at 23:05

1 Answers1

1

The problem is that pig comes with Joda 1.6 where LocalDate does not have a parse() method. The easiest and most safe fix would be to use 1.6 in maven and rewrite your code with 1.6 Joda Time.

Frederic
  • 3,274
  • 1
  • 21
  • 37
  • Which pig version are you referrring to? I will try it later – ManuelSchneid3r Jun 28 '15 at 10:18
  • Tons of thanks. I spent about 20 hours to get this to work. All the tips in the internet are just wrong. *NO* need to pass additional jar, *NO* need to register a jar. Just use the correct api. You could elaborate on how the UDF uses the joda-time from Pig, which is still a mystery for me. (Coming from c/c++) – ManuelSchneid3r Jun 28 '15 at 12:57