pandas to_sql for MS SQL

Question

I'm trying to save a dataframe to MS SQL that uses Windows authentication. I've tried using engine, engine.connect(), engine.raw_connection() and they all throw up errors: 'Engine' object has no attribute 'cursor', 'Connection' object has no attribute 'cursor', and Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': ... respectively.

params = urllib.parse.quote('DRIVER={ODBC Driver 13 for SQL Server};'
                           'SERVER=server;'
                           'DATABASE=db;'
                           'TRUSTED_CONNECTION=Yes;')

engine = create_engine('mssql+pyodbc:///?odbc_connect=%s' % params)

df.to_sql(table_name,engine, index=False)

score 2 · Answer 1 · answered Mar 08 '19 at 18:18

This will do exactly what you want.

# Insert from dataframe to table in SQL Server
import time
import pandas as pd
import pyodbc

# create timer
start_time = time.time()
from sqlalchemy import create_engine


df = pd.read_csv("C:\\your_path\\CSV1.csv")

conn_str = (
    r'DRIVER={SQL Server Native Client 11.0};'
    r'SERVER=name_of_your_server;'
    r'DATABASE=name_of_your_database;'
    r'Trusted_Connection=yes;'
)
cnxn = pyodbc.connect(conn_str)

cursor = cnxn.cursor()

for index,row in df.iterrows():
    cursor.execute('INSERT INTO dbo.Table_1([Name],[Address],[Age],[Work]) values (?,?,?,?)', 
                    row['Name'], 
                    row['Address'], 
                    row['Age'],
                    row['Work'])
    cnxn.commit()
cursor.close()
cnxn.close()

# see total time to do insert
print("%s seconds ---" % (time.time() - start_time))

You're importing sql alchemy and creating an engine but not using it. This is pure pyodbc. — holdenSaysNoPhonies, May 13 '21 at 19:34

score 1 · Answer 2 · answered Apr 14 '21 at 01:18

Here is an update to my original answer. Basically, this is the old-school way of doing things (INSERT INTO). I recently stumbled upon a super-easy, scalable, and controllable, way of pushing data from Python to SQL Server. Try the sample code and post back if you have additional questions.

import pyodbc
import pandas as pd

engine = "mssql+pyodbc://your_server_name/your_database_name?driver=SQL Server Native Client 11.0?trusted_connection=yes"

... dataframe here...

dataframe.to_sql(x, engine, if_exists='append', index=True)

dataframe is pretty self explanatory.

x = the name yo uwant your table to be in SQL Server.

pandas to_sql for MS SQL

2 Answers2