1

i am using kettle 3.2. i have a requirement like i need to write CSV file based on the data from the another table. As the number of rows increases the CSV files also increases. i mean if i have 2 rows then the file will created like 1st row data in one.csv and second data in two.csv

The design which i made works fine up to 650 rows but after 700 it is showing the error 

 2012/12/31 14:21:27 - CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA,
 build 10572 from 2009-05-12 08.45.26 by buildguy) : Unexpected error :
 2012/12/31 14:21:27 - CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA,
 build 10572 from 2009-05-12 08.45.26 by buildguy) :
 org.pentaho.di.core.exception.KettleException:  2012/12/31 14:21:27 -
 CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA, build 10572 from
 2009-05-12 08.45.26 by buildguy) : Error opening new file :
 org.apache.commons.vfs.FileSystemException: Could not create file
 "file:///home/admin/Desktop/data-integration3.2/CT_1056800_20121231_142127.CSV".
 2012/12/31 14:21:27 - CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA,
 build 10572 from 2009-05-12 08.45.26 by buildguy) :  2012/12/31
 14:21:27 - CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA, build 10572
 from 2009-05-12 08.45.26 by buildguy) :    at
 org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.openNewFile(TextFileOutput.java:654)
 2012/12/31 14:21:27 - CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA,
 build 10572 from 2009-05-12 08.45.26 by buildguy) :    at
 org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.setDataWriterForFilename(TextFileOutput.java:187)
 2012/12/31 14:21:27 - CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA,
 build 10572 from 2009-05-12 08.45.26 by buildguy) :    at
 org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.processRow(TextFileOutput.java:163)
 2012/12/31 14:21:27 - CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA,
 build 10572 from 2009-05-12 08.45.26 by buildguy) :    at
 org.pentaho.di.trans.step.BaseStep.runStepThread(BaseStep.java:2889)
 2012/12/31 14:21:27 - CSV FILE OUTPUT.0 - ERROR (version 3.2.0-GA,
 build 10572 from 2009-05-12 08.45.26 by buildguy) :    at
 org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.run(TextFileOutput.java:849)
 2012/12/31 14:21:27 - EXTRACTION_FOR_CUSTOMER - ERROR (version
 3.2.0-GA, build 10572 from 2009-05-12 08.45.26 by buildguy) : Errors detected!

please help me to solve this. thanks in advance

thar45
  • 3,518
  • 4
  • 31
  • 48
Avinash
  • 93
  • 2
  • 10
  • 1
    Does this reliably happen between 650 and 700? Maybe it has something to do with exceeding the number of open file handles a process can have on your system. The solution would then be to change your code to ensure that Kettle closes them. – Gordon Seidoh Worley Jan 01 '13 at 17:59
  • Why are you using Kettle 3.2? It's very very old.. I'd try the latest and greatest as well, shouldnt take 5 minutes to give it a try. maybe 3.2 was leaking handles as Gordon suggests.. – Codek Jan 02 '13 at 11:11
  • Sorry i cannot able to change the version. Am working for a customer who is already using 3.2 version(In which they have all their jobs running) and they are not willing to change the version. so i need to fix it in the 3.2 version. – Avinash Jan 02 '13 at 12:54
  • I checked in kettle 4.3 also but the same error occurs – Avinash Jan 04 '13 at 10:36

1 Answers1

0

Finally i found the solution. I solved this using Modify java script object for creating large number of CSV files (>8000). I used inbuilt function

appendToFile(filepath,content)

Thank you for all your support

Avinash
  • 93
  • 2
  • 10