Stack Overflow
Stack Overflow

Questions tagged [csv-import]

CSV Import refers to execution of a script that parses a CSV file and imports structured data in a database.

CSV Import refers to execution of a script that parses a CSV file and imports structured data in the database.

CSV Importing is most commonly related to building custom import scripts in various programming languages. Usually, the CSV file is built with a certain structure and order of the columns, and the script is built in a way to expect the same columns in that same order. Once the data is extracted from the CSV file, the script goes through all CSV lines and inserts the structured data into the database.

211 questions
476
votes
7 answers

Create Pandas DataFrame from a string

In order to test some functionality I would like to create a DataFrame from a string. Let's say my test data looks like: TESTDATA="""col1;col2;col3 1;4.4;99 2;4.5;200 3;4.7;65 4;3.2;140 """ What is the simplest way to read that data into a Pandas…
Emil L
  • 20,219
  • 3
  • 44
  • 65
127
votes
4 answers

pandas read_csv and filter columns with usecols

I have a csv file which isn't coming in correctly with pandas.read_csv when I filter the columns with usecols and use multiple indexes. import pandas as pd csv = r"""dummy,date,loc,x bar,20090101,a,1 bar,20090102,a,3 bar,20090103,a,5 …
chip
  • 2,262
  • 2
  • 20
  • 24
27
votes
8 answers

Commas within CSV Data

I have a CSV file which I am directly importing to a SQL server table. In the CSV file each column is separated by a comma. But my problem is that I have a column "address", and the data in this column contains commas. So what is happening is that…
Sidharth
  • 1,251
  • 1
  • 25
  • 40
10
votes
3 answers

LOAD DATA LOCAL INFILE gives the error The used command is not allowed with this MySQL version

I have a PHP script that calls MySQL's LOAD DATA INFILE to load data from CSV files. However, on production server, I ended up with the following error: Access denied for user ... (using password: yes) As a quick workaround, I changed the command…
Salman A
  • 262,204
  • 82
  • 430
  • 521
8
votes
2 answers

How to include a Library into my Swift Project?

I want to read a .csv data in Swift, so I have informed me how to make this. Finally I got into this: https://github.com/Flinesoft/CSVImporter The CSVImporter. But it says: "You can of course also just include this framework manually into your…
Blub
  • 121
  • 1
  • 1
  • 6
8
votes
3 answers

SQLite import csv file with comma in text fields

I want to import csv file into SQLite db using sqlite> .separator , sqlite> .mode csv data sqlite> .import test.csv data where data is the table name with three columns, just like the file. The file has some string value that are encapsulated…
Tautvydas
  • 2,027
  • 3
  • 25
  • 38
7
votes
2 answers

Bulk insert from csv file - Ignore rows with errors - SQL Server

I am trying to import data from a csv file to SQL Server. There are thousands of entries in the csv file and we have a lot of rows with incorrect data in it. Some of the rows in the CSV File…
P_Ar
  • 377
  • 2
  • 9
  • 25
6
votes
1 answer

Arabic text CSV file import?

I can't seem to get to the bottom of this, I want to read a csv file that contains Arabic characters but it's not reading it properly. this is my sessionInfo R version 3.2.4 Revised (2016-03-16 r70336) Platform: x86_64-w64-mingw32/x64…
Rayan Sp
  • 1,002
  • 7
  • 17
  • 29
5
votes
1 answer

Rails: delete_all raises PG::NotNullViolation

I am importing a CSV with a few hundred rows into my rails database. Occasionally the user wants to force overwrite the data so I figured that it would be best to destroy all the data and start fresh. something…
Blair Anderson
  • 19,463
  • 8
  • 77
  • 114
4
votes
1 answer

LOAD DATA LOCAL INFILE not importing any records

I am working on a project that requires importing csv data into a mysql table using Workbench. I tried running the command below. I am not getting any errors, however no records are being imported in to the table. Note, I tried changing the line…
AnchovyLegend
  • 12,139
  • 38
  • 147
  • 231
4
votes
1 answer

How can I process a large CSV file line by line?

I've successfully written a script that downloads a CSV file with the use of cURL and then parses the CSV into a array as follows: $rows = array_map(function($a) { return str_getcsv($a, $delimiter); }, explode("\n", $result)); Then I iterate…
Max
  • 803
  • 3
  • 10
  • 24
4
votes
1 answer

It is possible to run mongoimport from Heroku?

I have a very large dataset of cvs file that need to be regularly imported MongoDB. I'd like to have the importer run off Heroku and write to a MongoDB Heroku add-on. mongoimport is a separate binary tool that the MongoDB guys provide. So, it is…
andy
  • 8,775
  • 13
  • 77
  • 122
4
votes
2 answers

SQL Server 2012 Import CSV to mapped Columns

I'm currently trying to import about 10000 rows (from a CSV file) into an existing table. I only have 1 column which I'm trying to import, but in my table I have a another column called TypeId which I need to set to a static value i.e.…
D-W
  • 5,201
  • 14
  • 47
  • 74
3
votes
1 answer

MYSQL Crashes while importing CSV using import wizard

I have set up MySQL 8.0.23 365764 CE. Have created the Schema and Table in it. Have 3 Tables. Importing Tab1 and 2 from CSV works perfectly fine. Tab1 has 66 and Tab2 54714 records. I am using Importing Wizard here. For some reason, MYSQL Workbench…
Ashwin Kumar
  • 109
  • 13
3
votes
1 answer

Import more than 255 Characters from excel to sql server (previous question - how to load text qualifed CSV dynamically to sql server using ssis)

I have a CSV file which has salesforce data and is generated using python API call and i am trying to load the file in MS SQL DB using SSIS. I need a C# script to rectify the issue because the data is not consistent (no of columns and order of…
Amanpreet Kaur
  • 31
  • 5
1
2 3
…
14 15