Pandas DataFrame method that writes the object's records to a SQL database. Be sure to also include the [pandas] tag.
Pandas DataFrame method to_sql can be used to write its records to a SQL database.
The documentation:
Questions tagged [pandas-to-sql]
144 questions
0
votes
1 answer
Interface Error when importing Pandas data frame into SQL Server
I have a pandas data frame called : data
I am trying to read this pandas dataframe into a table in sql server.
I am able to read data into python from sql but I am expiering problems loading the dataframe into a table.
I have tried a few examples…
Helena Viljoen
- 19
- 4
0
votes
0 answers
pd.to_sql returning a KeyError: '\x02' on Windows Server 2008 R2 (Python 3.6.4) (Pandas 0.25)
I have some code that I can run locally on my windows 10 machine that writes to a table on a DB2 sql database and runs perfectly. For reasons I can't seem to figure out, it doesn't work on the windows server 2008 R2 machine.
Works fine locally, but…
Chester Poon
- 13
- 4
0
votes
0 answers
Is there a way to print a message every time a "chunk" gets written to a SQL database?
So I have a huge data frame that I am writing to a table in a SQL database by splitting the data frame into chunks of size 10,000 (I was told that speeds up the process). I use the following code:
from sqlalchemy import create_engine
…
Vincent_Adultman
- 78
- 11
0
votes
1 answer
Same code structure - different results when calling .to_sql()
Background
I have 3 separate python scripts that share the same structure and effectively do the same thing - call a request to an API, retrieve XML data, convert XML to ElementTree object then to pandas DataFrame object then use .to_sql() to import…
Jamm6
- 3
- 5
0
votes
0 answers
How to update all the values of column in a db2 table using python
I have a table in db2 as below:
index--------name---------pass
0----------------A-------------No
1----------------B-------------No
2----------------C-------------No
I need to change it…
ss301
- 514
- 9
- 22
0
votes
1 answer
Execution failed on sql : insert dataframe to postgresql server
I'm trying to connect postgresql server and insert some dataframe to a table and this is my code
import datetime
import gzip
import logging
import os
import sys
import traceback
import numpy as np
import pandas as pd
import psycopg2
from…
newbieeeeee
- 3
- 3
0
votes
1 answer
Pandas parsing errors due to extra commas in columns when trying to append rows to SQL table using read_csv w/to_sql
I'm trying to update a table in my database with any new rows from a .csv file that contains just 2 columns with a comma as the delimiter.
However, each of these columns contains many extra commas (and double-quotes, single-quotes, spaces etc.) Both…
gulfy
- 47
- 2
- 6
0
votes
0 answers
date columns to_sql with oracle server
I am using to_sql to append a table into Oracle. However, I am getting the following error
(cx_Oracle.DatabaseError) ORA-01843: not a valid month [SQL: 'INSERT ... INTO
I tried to define the type of column for the date columns in the table
…
Santiago
- 57
- 1
- 6
0
votes
2 answers
How to convert python datetime to timestamp and insert in oracle database using to_sql
I get DatabaseError: ORA-00904: "DAT_ULT_ALT": invalid identifier when I try to insert a datetime to a timestamp in oracle using to_sql from pandas with SQL Alchemy engine. My code:
import sqlalchemy as sa
import datetime
import…
Patricia Rocha Faria
- 1
- 1
- 3
0
votes
1 answer
how to write the connection_object for pandas.DataFrame to create a table in sql
I am trying to convert a csv file (consist of 40 headers and 4000 rows) into a table in sql database server. I read somewhere that I can do it in 2 lines as simple as:
df = pandas.read_csv("file.csv")
df.to_sql(tableName, conn)
I tried to make conn…
Amy2020
- 1
- 4
0
votes
1 answer
Pandas to_sql 'append' to an existing table causes Python crash
My problem is essentially this: When I try to use to_sql with if_exists = 'append' and name is set to a table on my SQL Server that already exists python crashes.
This is my code:
@event.listens_for(engine, 'before_cursor_execute') def…
Missyinvisible
- 33
- 2
- 7
0
votes
1 answer
Pandas to_sql sending additional blank rows to db table
I am reading a csv file into a dataframe and writing it to a sqlite db table and getting this warning:
UserWarning: The spaces in these column names will not be changed. In pandas versions < 0.14, spaces were converted to underscores.
dtype=dtype)…
Mark D
- 157
- 1
- 4
- 13
0
votes
2 answers
MySQLdb: Pandas data-frame to SQL database error: not all arguments converted during string formatting
So, I wanted to move a dataframe to a database into a table but I am encountering a few problems. I made the dataframe using the pandas library and wanted to upload the data into a table in my database.
Relevant snippets of my Code and the error…
Hrithik Diwakar
- 119
- 2
- 11
0
votes
1 answer
Error while using the to_sql function
I am using the following code to extract data from SQL into a Data Frame and it works fine.
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server=DESKTOP-5422GFU;"
…
Shawn
- 1
- 3
0
votes
1 answer
to_sql error: Unknown object type numpy.ndarray during describe
I am trying to insert a dataframe into a sql table and I get the following error
ProgrammingError: (pyodbc.ProgrammingError) ('Unknown object type numpy.ndarray during describe', 'HY000')
for the following code
merged.to_sql('pmg.cwc.EmSignals1',…
user3896886
- 1
- 1
- 1