rows missing in mysql

Question

I am trying to import data from CSV with rows 1.7 million approx in MYSQL using bulk insert after importing I get only 1.04 million rows for all the tables.

I used

select count (*) from table_name;

command to check the number of rows.

for crosschecking the rows I used the pandas' module in python.

I used this command LOAD DATA INFILE " " INTO TABLE CSVImport — A R MOHAMMED, Dec 16 '19 at 04:11

score 0 · Answer 1 · answered Dec 14 '19 at 13:16

For an import job of that size, you'll be wise to break up your CSV into chunks of about 50,000 rows each, then import them one by one. Watch for errors on each import job, and COUNT(*) the rows after each one.

You will probably find some rows in your data did not load correctly. Some reasons:

Letters where there should be numbers
Inconsistently formatted dates
Text fields longer than allowed by the column in the table
Too many, or not enough, commas
Bad field escaping: In CSV when you want to write Stephen "The Rifleman" Flemmi you must write Stephen ""The Rifleman"" Flemmi doubling the quote characters.

rows missing in mysql

1 Answers1