Stack Overflow
Stack Overflow

Questions tagged [pandas-to-sql]

Pandas DataFrame method that writes the object's records to a SQL database. Be sure to also include the [pandas] tag.

Pandas DataFrame method to_sql can be used to write its records to a SQL database.

The documentation:

144 questions
0
votes
1 answer

Problem loading a function in Python to_sql

I have the following code in two scripts called Playing_DB_Around.py with the following code: import data_base.Calc_Func as calc import pandas as pd df = pd.DataFrame({'Name': ['John', 'Jane', 'Jim'], 'Age': [25, 30, 35]}) db =…
MCM
  • 1,479
  • 2
  • 17
  • 22
0
votes
0 answers

Convert a string to dictionary - without quoting the values

I have a dataframe with some column names and data types as following: column_name data_type modifiedDate DateTime city VARCHAR(50) state …
noClueAboutWhatIMDoing
  • 17
  • 5
0
votes
0 answers

pandas to_sql error: “table already exists” when I’m using param if_exists=‘append'

I am trying to run a script once per day at a specific time. The script is looking to get stock market data from an API and then append the data into the sqlite3 database I have already created. Within my sqlite3 database I have one table per asset…
jbaker3004
  • 11
  • 2
0
votes
1 answer

pandas df.to_sql if column value exits replace or update row

I'm using pandas df.to_sql to inserting rows to postgresql database. df.to_sql('example_table', engine, if_exists='replace',index=False) example_table has 3 columns :'id' ,'name' ,'datetime' I want to add a checking logic before inserting ,that if…
William
  • 3,724
  • 9
  • 43
  • 76
0
votes
1 answer

DataError: (psycopg2.errors.StringDataRightTruncation) value too long for type character varying(256)

I am trying to write a Python data frame to redshift. I wrote this code - df.to_sql(sheetname, con=conn,if_exists='replace',chunksize=250, index=False) conn.autocommit = True I am getting below error: DataError:…
Thentu matcha
  • 33
  • 4
0
votes
2 answers

Searching special data by 'WHERE' command for CLOB type

Look at below table. I tried to print 'CORP_CODE' out with tuple which has same 'CORP_NAME' in it. SO, I wrote this code. SELECT CORP_CODE FROM COMPANY_INFO WHERE CORP_NAME = '다코' However, There is important error. The above code show ORA-00932:…
Null물을 흘린다
  • 31
  • 4
0
votes
0 answers

Data lost after insert to PostgreSQL Data base

I build pretty easy etl procces that get data from one DB and put it to another one. eng1=sqlalchemy.create_engine(url1) eng2=sqlalchemy.create_engine(url2) for df in pd.read_sql(query,eng,chuncksize=1000): df.to_sql(table_name,eng2,…
Daniel
  • 1
0
votes
1 answer

Getting ValueError: (NVARCHAR) not a string in pandas.to_sql

I am trying to insert a pandas dataframe in SingleStore. dtype of columns are either string/object. I am getting the below error for this piece of code. Error : ValueError: country (NVARCHAR(255)) not a string Code : # Insert whole DataFrame into…
user2258035
  • 55
  • 4
0
votes
0 answers

export pandas df to sql server if data not exists

I have a df named "review" like this : usuarios fecha calificacion comentario branch_id Cinthya De Sousa 2022-09-20 1 estrella Empleados que te atienden con poca naturalidad. 53 Luana Antonella Gomez Lucero 2022-08-23 1 estrella Desastrosa…
Maximiliano Vazquez
  • 196
  • 1
  • 2
  • 12
0
votes
2 answers

Data Frame .to_sql() causes error when INSERTING into SQL Server database

I am attempting to generate an INSERT statement into an SQL Server database from a Data Frame: data_frame.to_sql(table, engine, if_exists='append', index=False) The Data Frame is created from a JSON Response. I am having issues inserting String…
cpa390
  • 13
  • 7
0
votes
2 answers

Uploading table to SQL Server from python using to_sql

I have connected to my SQL Server database to extract a table and then I use python to manipulate this table. After the table is edited, I would like insert it back into my works SQL Server database as a new table. I have tried numerous methods of…
user17582908
0
votes
0 answers

pandas.sql for postgres what chunksize and what method

I have a 1M and 10M rows tables (10 columns) I am uploading to a postgres DB using pandas.to_sql. I noticed that using method="multi" severely reduces performance, 5 to 6x with chuncksizes 5k, 10k, 20k. even reducing chunksize to 100 didnt help. Is…
Courvoisier
  • 904
  • 12
  • 26
0
votes
0 answers

Python df.to_sql( ) update MS SQL table

I have a data frame which looks as follows: index staffmember title basedin yearsexperience skillsdomain product competency abbreviation 0 Arnold Schwartzenegger Terminator2 USA 23.0 Platform Spectrum Required …
Jeff
  • 1
  • 2
0
votes
1 answer

Error with delimiters on dataframe when trying to upload it to MSSQL

So I've been trying to upload a dataframe to an specific table that is under MSSQL, I've trying to use the BCPANDAS library to upload the data to it. However there's an issue with the data that has a lot of strings on it that contains multiple…
silentninja89
  • 171
  • 1
  • 10
0
votes
1 answer

How to tweak Pandas method to_sql, so that my data are formatted with right no. of decimals?

There's an Oracle table which I need to fill with weather data. This is what the table looks like: "IDGRID" NUMBER(12,0) NOT NULL, "DAY" DATE CONSTRAINT NOT NULL, "TEMPERATURE_MAX" NUMBER(3,1) NOT NULL, "TEMPERATURE_MIN" NUMBER(3,1) NOT…
Dobedani
  • 508
  • 5
  • 20
1 2 3
…
9 10