1

Objective

I'm trying to load a .csv file called persondata into DB alldata TABLE persondata and then run a query on it for firstname, lastname and dateofbirth (dob). The .csv has only 4 records in it.
Date format in csv is MM/DD/YYYY, the output should be YYYY/MM/DD, YYYY-MM-DD or YYYYMMDD.

SQL

LOAD DATA LOCAL INFILE 'C:/Users/john.smith/Desktop/persondata.csv'
INTO TABLE alldata.persondata
FIELDS TERMINATED BY ','
(firstname, lastname, dob, apptdate, icd9, cpt)
SET dob = str_to_date(@dob, '%c/%e/%Y')
;

SELECT firstname, lastname, dob
FROM alldata.persondata  

Problem and Error Message I'm Getting

firstname and lastname return proper values but dob returns null for all 4 records. In the csv file, the first three colums (A, B, C) are firstname, lastname, dob. So same order as in the table persondata.

Error:

4 row(s) affected, 8 warning(s): 1265 Data truncated for column 'dob' at row 1 1411 Incorrect datetime value: '19850708' for function str_to_date 1265

Help pages I consulted:

(using str_to_date in general)
How to convert csv date format to into mysql db
(using 'SET column = str_to_date...')
MySql load data infile STR_TO_DATE returning blank?
(other)
How to change string date to MySQL date format at time of import of CSV using MySQL's LOAD DATA LOCAL INFILE
Cannot transform mm/dd/yyyy in excel to csv yyyymmdd date format using SSIS
MySQL str_to_date produces NULL despite valid formatting

Additional Information:

I experimented with this query and a bunch of variations of it but no luck:

SET dob = date_format(str_to_date(@dob, '%c/%e/%Y'), '%Y/%c/%e')

I'm not seeing a huge amount of consensus on how to write this. Some people specify the output format of %Y/%d/%m and some don't. But isn't that the only date format that mysql supports? This makes me think I shouldn't have to write it. Not sure if this is even related. I've seen a few syntaxes of the entire thing. I've read through all the support pages and I think that I understand the 'SET' command.

*this is my first post on stackoverflow so please let me know if should present anything differently

Community
  • 1
  • 1

1 Answers1

0

You're using "dob" where you should be using "@dob" in your column list... line 4 should be:

(firstname, lastname, @dob, apptdate, icd9, cpt)
                      ^^^^

This is because mysql reads the date into a VARIABLE (indicated by the @), and then the later SET command manipulates the variable to match the actual column (which is identified correctly dob without the @).

Also, I think you're using slashes in your string format when the date doesn't appear to have slashes in it. That is, the error says the date is "19850708" (July 8th, 1985). I believe you want:

%Y%m%d

That is, change line 5 to:

SET dob = str_to_date(@dob, '%Y%m%d')

%m and %d are the 2-digit month and day, which you need since you have '07' instead of just '7' for July, for example. After your comments I'm not sure which date format is correct, but certainly the @dob vs dob variable issue is real. You may want to replace every instance of @dob with something like @original_dob just to alleviate future confusion.

See this page: http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_str-to-date

Chipmonkey
  • 863
  • 7
  • 18
  • sorry, my date format is M/D/YYYY because the xls is MM/DD/YYYY but when I save it to .csv it automatically becomes M/D/YYY, and so I'm using %c/%e%Y – William Capozzoli Nov 11 '14 at 14:54
  • Are you sure %Y%m%d is correct here? Per the dev.mysql support page you sent me it says, "Literal characters in format must match literally in str. Format specifiers in format must match a date or time part in str. " so aren't I supposed to write %c%e%Y because that's the format of the 'input', @dob? column dob, which is M/D/YYYY?. I think I've seen it written both ways on StackOverflow pages. Ah! I tried your idea, its still returning null – William Capozzoli Nov 11 '14 at 15:05
  • I guess I'm not completely certain, but the date in the error message was my only clue, and it said '19850708', which is YYYYMMDD. Can you actually post some lines from the .csv file, including lines with one-digit months and days? If you do in fact have slashes then you'll need slashes, but then I don't know where the error message is getting the 1985 date. – Chipmonkey Nov 13 '14 at 19:53
  • Ok, I saw another issue, which was with the variable names. The @dob variable name gets confused with the "dob" column name. I've updated my answer, let me know if this version works. Sorry I missed that before. – Chipmonkey Nov 13 '14 at 20:02
  • I added @dob to my column list on line 4 but that didn't help. I added @dob to my SELECT statement so it reads `SELECT firstname, lastname, @dob` and now the dob for each record no longer returns null or error, but 4/7/1992. In my .csv, the dob of the 4th record is 4/7/1992. Right now the dates in the csv are 10/12/1999 3/11/1984 10/4/1987 and 4/7/1992 – William Capozzoli Nov 15 '14 at 20:56
  • Very strange - I can uncomment the entire `SET` statement (line 5) and I still get this same exact result of 4/7/1992. The warnings I get say `1265 data truncated for column 'dob' at row 1` and `1265 data truncated for column 'dob' at row 2` and so forth. I can also edit the 4/7/1992 dob in the csv to 5/5/1992, save it, re-execute the sql and still get 4/7/1992. If I reboot MySQL then it will show the 5/5/1992. I have DROP DATABASE and DROP TABLE in there. Idk how to Reply with linebreaks, can I just supply a new 'Answer' and paste my whole code there? – William Capozzoli Nov 15 '14 at 21:04
  • I re-started MySQL, uncommented the SET statement, ran the query and got all null values. But when I included your idea of putting @dob in line 4, it started returning 4/7/1992. I'm getting so confused, I feel like I should close this Question and re-open a new one but include my entire query as well as the csv file data... – William Capozzoli Nov 15 '14 at 21:10