python strange date format when reading from a csv file

Question

I try to read some data from many .csv files and store them into arrays to plot them. To be specific there are two arrays. In the first one, I throw in the date and time information. In the original data, this is in the format of

Date=2009/07/29 15:28:46

the second array is in the form

SpectrumMax=1.440973

when plotting these values, the program is terminated with message:

....
ValueError: invalid literal for float(): 2009/07/29 15:28:46

When printing the array, where the datetime is stored, it shows:

...
20:32:53', '2013
.09.05 06:46:07', '2013.09.06 08:56:47', '2013.09.09 06:44:08', '2013.09.10 21:2
0:59', '2013.09.12 18:32:34', '2013.09.16 16:13:37', '2013.09.18 06:35:41', '201
3.09.20 06:34:03', '2013.09.23 07:44:46', '2013.09.26 11:08:33', '2013.09.26 12:
36:39', '2013.09.26 13:36:59']

Compared to the initial data, the dates changed their separators from slash to dot. Why is this and how can I suppress this obviously automatic conversion?

The original code is:

#!/usr/bin/env python
# -*- coding: utf-8 -*- 

import os
from pylab import *
import csv
import datetime as DT


import numpy as np



mydir = ('C:\\...\\RickPython\\Background')

f = os.listdir(mydir)

data = [[0],[1]]

os.chdir(mydir)

files = os.listdir(mydir)

for filename in files:
    ifile  = open(filename, "rb")
    #print filename
    #ifile  = open('Bkg_20090729152846.csv', "rb")
    reader = csv.reader(ifile, delimiter='=')
    #is_in_csv = False

    #search_string = 'Date=2009/07/29 15:28:46'

    for num, row in enumerate(reader):
        if 'Date' in row:
            #is_in_csv = True
            #print('Is Date in csv ?'), is_in_csv
            data[0].append(row[1])
            #print row

        if 'SpectrumMax' in row:
            #is_in_csv = True
            #print('Is SpectrumMax in csv ?'), is_in_csv
            data[1].append(float(row[1]))
            #print num, row



#print('Line number %s and Row %s')% (num, row)
#print data
#print num

ifile.close()

x = data[0]
y = data[1]


plot_date(x, y, 'r+')


show()

The big question is, how to plot the data with date/time on x-axis?

Your analysis is incorrect; nothing in your code would change the `date` column format; your input format must be producing that format instead. — Martijn Pieters, Jan 03 '14 at 13:56
Your error, on the other hand, points to there being dates in the second column, `row[1]`, where you expected there to be a `SpectrumMax` floating point value. I think your understanding of the input file is wrong. — Martijn Pieters, Jan 03 '14 at 13:57
Why are you using `csv` to parse a `key=value` format file? I would not use that module for such a file; can you show us a sample of the files lines? — Martijn Pieters, Jan 03 '14 at 13:59
@Martin Pieters When printing x, I get a list of strings with the date and time. When printing y, this gives me the SpectrumMax values. Pasting the whole data in here would be to much... Sorry. But this seems to be ok. — Finestructure, Jan 03 '14 at 14:23
Ok. So it seems to me that some csv files are corrupt. The format differs from the expected format. Sorry guys. I have to ensure consistent conditions. Thanks so far. I will investigate and give feedback.... — Finestructure, Jan 03 '14 at 14:28
@Martijn Pieters: Here are some lines:SpectrumStartLine=25 Date=2013.04.11 15:56:03 ReferenceFile=Bkg_20130411155603.csv Result=1 SpectrumMax=1.390054 IgrResult=1 IgrPP=4.793056 IgrMin=-12.98821 IgrMax=-8.195156 IgrMinLimit=2 IgrMaxLimit=9 RangeCount=2 Range1_Name=H20 Range1_Result=1 Range1_Value=-1.872486 Which module would you use instead? — Finestructure, Jan 03 '14 at 14:50

python strange date format when reading from a csv file

0 Answers0