Stack Overflow
Stack Overflow

Questions tagged [read-sql]

45 questions
1
vote
1 answer

How to unit test the columns names are valid for a function that reads data with `pandas.read_sql()`?

I have a function to read data from a MySQL database: def example(mysql_engine) -> DataFrame: query = """SELECT col_1 FROM xxx.xxx""" df = pandas.read_sql(query, mysql_engine) return df mysql_engine is returned by another function, now…
wawawa
  • 2,835
  • 6
  • 44
  • 105
1
vote
2 answers

pandas sqlite read_sql dynamic in clause

I am trying to use pandas read_sql function to query some data from sqlite DB. I need to use parameterized SQL which contains in clause (List) and some static parameters. Below is my query battingDataQuery = ('SELECT ID, MATCH_DATE, ROLE, DOWN_NUM,…
Neel
  • 613
  • 4
  • 14
  • 32
1
vote
2 answers

Couldn't translate host name PostgreSQL

I'm trying to work with db. So I connect to the PostgreSQL database with the following connection info. I'm using Jupyter Notebook. from sqlalchemy import create_engine POSTGRES_DIALECT = 'postgresql' POSTGRES_SERVER = 'server' POSTGRES_DBNAME =…
Max AweTery
  • 103
  • 6
1
vote
1 answer

Pandas read_sql very slow reading from Jaydebeapi Netezza

I'm using Pandas read sql to read netezza table through jdbc/jaydebeapi. start_time = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] print(" DB Start Date Time is " + str(start_time)) int_df = pd.read_sql(query,conn) end_time =…
matterwaves
  • 11
  • 3
1
vote
0 answers

PyODBC+Pandas+Read_SQL: Error: The cursor's connection has been closed

I am reading tables as SELECT * FROM TABLE (sql); from an ODBC data source via PyODBC and fetching/loading all the rows using Pandas read_sql(). However, there are 200+ tables and some have 100,000 rows so for that have been using chunksize to read…
ManiK
  • 377
  • 1
  • 21
1
vote
1 answer

How do I set current schema by using ibm_db_dbi to connect database?

I've been trying to use ibm_db_dbi to connect the database and read table by pandas.read_sql. After creating the connection, I use set_current_schema('mySchema') to set the current schema for myConnection, so that I don't need to specify the schema…
Terry
  • 15
  • 5
0
votes
0 answers

pd.read_sql(sql) If sql has two columns of the same field, the result is only one column of fields

pd.read_sql(sql) If sql has two columns of the same field, the result is only one column of fields pd.read_sql() dont work ? import pandas as pd sql = "select name, name, age from table;" df = pd.read_sql(sql, con) print(df.columns) # name,…
程全海
  • 1
0
votes
0 answers

read_sql_query connection parameter con on multiple dataframes

I have three pandas dataframes and i would like to use SQL to join them together. i have a question on how i assign 'con' in pd.read_sql_query function. import pandas as pd df1 = pd.read_csv("C:/Users/name/sheet1.csv") df2 =…
kaispace30098
  • 120
  • 8
0
votes
0 answers

Unable to get a pandas dataframe from hiveql query using Python

I am running this script on Jupyter Notebook (engine python 3.6): pd.read_sql('''select * from giadb.staging_eban limit 1''',engine) The engine is created with SQLAlchemy and it is a Hive connection. The table has 283 columns. The statement remains…
gianluca
  • 1
  • 3
0
votes
2 answers

How to execute SQL query with parameters in Pandas

I am using pyscopg2 and have a method that returns a list based on a SQL query. The below works fine... def checkAnalysisStartDate(self): session = self.connection() cursor = session.cursor() ids = self.getAnalysisIds() # this is a list…
JD2775
  • 3,658
  • 7
  • 30
  • 52
0
votes
0 answers

Change Datatype of Values in a Pandas SQL Insert Statement

Using Pandas read_sql() function to INSERT data into a SQL table called table_1. Pulling data from a primary database and writing to table_1. # Creates the table read_sql( f""" CREATE TABLE IF NOT EXISTS {table_1} (id varchar,…
Larry
  • 45
  • 1
  • 8
0
votes
0 answers

Pandas read_sql_query with WHERE condition on VARCHAR

I am trying to pull data from a certain database, the database is huge so I would like to query it before getting the data into python. I want to query on a specific column [Type] that I know is type varchar(50). As a test tried to query on an…
AVad
  • 5
  • 2
0
votes
1 answer

snowflake convert_timezone does not work in pandas read sql

convert_timezone works for me in the snowflake console: select convert_timezone('America/Los_Angeles', '2022-05-25T14:27:30Z'::TIMESTAMP_TZ) gives: 2022-05-25T07:27:30-07:00 but it does not work in pandas read sql: pd.read_sql("select…
Ezer K
  • 3,637
  • 3
  • 18
  • 34
0
votes
2 answers

sql to dataframe in function

new to functions so obviously there are lack of knowledge around the basics of this. I want to create a function which fetch data from sql server but I don't know how to pass it to a dataframe. This dataframe will later be used by other…
Martinswe
  • 43
  • 6
0
votes
1 answer

getting Nan values in pandas read_sql

I read data from MariaDB using pandas read_sql. What I am trying to do is give each data frame an index with same continuation in each chunk. Index of first chunk should 1 to 2000 and for second chunk to should 2001 to 4000. For that I have added a…
BetaTester
  • 25
  • 8
1
2
3