0

I'm currently doing a bulk load from Greenplum to SAS. Initially there was one field with a backslash "\" at the end of the column causing to throw an error during loading. To resolve it I changed the format from TEXT to CSV and worked fine. But loading more data I encountered this error:

gpfdist error - line too long in file

I've been doing some search but couldn't assess if the cause is due to that the max_length to set when starting the gpfdist service. I also saw that there is a limit for Windows which is 1MB? Greatly appreciate your help.

By the way here are some additional info which might help:

-Greenplum version: 4.2.1.0 build 3 -Gpfdist installed in Windows along with SAS Applications -Script submitted to Greenplum based on SAS Logs: CREATE EXTERNAL TABLE ( ) LOCATION ('gpfdist://:8081/fileout.dat') FORMAT 'CSV' ( DELIMITER '|' NULL '\N') ENCODING 'LATIN1'

Thanks!

mtryingtocode
  • 939
  • 3
  • 13
  • 26

2 Answers2

0

"Line too long" sorts of errors usually indicate that you've got extra delimiters buried in VARCHAR/TEXT columns that throw the parsing of the file off.

Another possibility is that you've got hidden control characters, extra linebreaks or other nasty stuff hidden in your file that again is throwing your formatting off. Gpfdist can handle a lot of different data errors and keep going, but extra delimeters throws it for a loop.

Scan your load file looking for extra pipe characters in a line.

Another option would be to re-export your data, picking a different delimiter.

Bart K
  • 684
  • 5
  • 10
0

Please try an alternate solution, by selecting the input format as Text and client encoding as ISO_8859_5 in the session and see if that will help you. In my case it worked.

sai kishore patro
  • 1