1

I'm trying to export a a recarray to csv file with rec2csv so I can retrieve it later with csv2rec. The problem is that the rec2csv is exporting with a blank line between each rows, so csv2rec cannot read it later. How can I fix this problem with the function rec2csv?

Basically, what I'm trying to do is this:

ticker = 'GOOG'
startdate = datetime.date(2011,1,1)
enddate = datetime.date.today()
fh = finance.fetch_historical_yahoo(ticker, startdate, enddate)
r = mlab.csv2rec(fh); fh.close()
r.sort()

After some calculations,

fl = open(r'J:\export.csv', 'w')
mlab.rec2csv(r,fl); fl.close()

Then I want to be able to import this file again with:

ff = mlab.csv2rec('J:\\export.csv')

This give an error message (IndexError: list index out of range) because there are blank lines between rows.

catun
  • 11
  • 2
  • 1
    It shouldn't be doing that... What does your data look like? Do you have a string array (or field) that has newlines in it? By the way, the functions you're referring to aren't numpy functions. They're matplotlib functions. – Joe Kington Apr 02 '12 at 23:28
  • I'm trying to fetch stock data from yahoo finance using csv2rec and save it as csv file with rec2csv. When I do it, the file looks like this: \n date,open,high,low,close,volume,adj_close \n 2011-01-03,49.399999999999999,50.090000000000003,49.100000000000001,49.899999999999999,12745700,49.43 \n 2011-01-04,50.140000000000001,50.829999999999998,50.100000000000001,50.829999999999998,20270600,50.350000000000001 \n 2011-01-05,50.369999999999997,51.649999999999999,50.369999999999997,51.560000000000002,17732400,51.07 \n I don't know how to eliminate this empty line between each data. – catun Apr 03 '12 at 09:04
  • Can you show us a reproducible example? (Or a bit more of what you're doing, at least?) – Joe Kington Apr 03 '12 at 13:54
  • Things work perfectly here... It could be a line ending problem. Windows does things I don't understand when opening files it thinks are text. What happens if you either a) Open the file using the `'wb'` mode, or b) just pass in the file name instead of manually opening the file. (The result should be the same). – Joe Kington Apr 04 '12 at 23:27

1 Answers1

1

I'm guessing it's because you're not opening the file in binary mode. rec2csv uses the builtin csv module, which expects the file to be opened in 'wb' mode on Windows.

An easy solution is to just pass in the filename, instead of opening and closing the file manually.

So either do:

mlab.rec2csv(r, 'J:\\export.csv')

or do:

with open('J:\\export.csv', 'wb') as outfile:
    mlab.rec2csv(r, outfile)
Joe Kington
  • 275,208
  • 71
  • 604
  • 463
  • I got a message error: File "C:\Python32\lib\site-packages\matplotlib\mlab.py", line 2656, in rec2csv writer.writerow(header) TypeError: 'str' does not support the buffer interface. Is this related to python version, because I am using Python 3 and matplotlib for this version. – catun Apr 06 '12 at 16:01