-1

I am having an input file with date as one of the columns. while loading data i mentioned it is chararray. I want to change this into date format using ToDate(userstring, format) function . My date is in dd/mm/yyyy format. Script is as below, 

mydata = load '/testinput' using PigStorage(';') as (pdate:chararray, time:chararray, gpower:double, sm1: double, sm2:double, sm3:double);

getdate = foreach mydata generate ToDate(pdate, 'dd/mm/yyyy'), time, gpower, sm1, sm2, sm3;

This is giving me error with error code 1066.

Can anybody please help me to get the issue resolved.

Taryn
  • 242,637
  • 56
  • 362
  • 405
user3836231
  • 3
  • 1
  • 4

1 Answers1

0

You should get rid of the header in the first line of your input.

Also, you should use dd/MM/yyyy as a date format (see http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html).

Frederic
  • 3,274
  • 1
  • 21
  • 37
  • Fred, while processing I am removing header. I kept it for reference. I tried with the option mentioned by you and it is working fine now. Thank you very much :). I have one more doubt if you can help me. I wrote header into other file while processing. But I am not able to append the header while writing to output file. I used ‘union’ function to merge header and data value but the header is going into bottom. Is there any function we can add the header? – user3836231 Jul 14 '14 at 13:12
  • please accept my answer if it did help you. for your other question, please see this answer: http://stackoverflow.com/questions/13696036/export-from-pig-to-csv – Frederic Jul 14 '14 at 13:21
  • I had tried the option PigStorage('\t','-schema') earlier but it doesnt help me as it stores header part seperately. I wanted header as first record :(. – user3836231 Jul 14 '14 at 13:45
  • as explained in the answer: what you'll want to do is to issue a `hadoop fs -getmerge /user/hadoop/csvoutput ./output.csv` from your pigscript. – Frederic Jul 14 '14 at 13:50