0

I have tried multiple solutions and way around to solve this issue, probably something is still I am missing.

I want to insert a list of values to my database. Here is what I am doing -

import psycopg2
import pandas as pd


Region = [   
    "Region1",
    "Region2",

]

qa  = "endpoint1"
def insert_many(data_list):
    """Add data to the table."""
    insert_query = """INSERT INTO pbi_forecast(a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u) 
            VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)"""

    conn = None
    try:
        conn = psycopg2.connect(
            database='db',
            user='user',
            host='host1',
            port=5432,
            password=pw
)
        cur = conn.cursor()
        cur.executemany(insert_query, data_list)
        conn.commit()
        cur.close()
    except(Exception, psycopg2.DatabaseError) as error:
        print(error)
    finally:
        if conn is not None:
            conn.close()


# connect prod2
conn_prod2 = psycopg2.connect(
    database='db',
    user='user',
    host='host2',
    port=5432,
    password=pw
)
cur_prod2 = conn_prod2.cursor()

for re in region:

    sql_prod2_select = f"select * from pbi_forecast  where \
                        run_date >= '2022-04-20 00:00:00'\
                        and run_date <= '2022-04-22 00:00:00'\
                        and region = '{re}' ;"

    cur_prod2.execute(sql_prod2_select)
    forecast = pd.DataFrame(cur_prod2.fetchall())
    data_list = [list(row) for row in forecast.itertuples(index=False)]
    insert_many(data_list)

I am getting integer out of range error when running it. If I restrict the insert record to somewhere 50 records it works but when running it without any limit it throws this error.

enter image description here

Thanks in advance.

simi
  • 53
  • 2
  • 9
  • Have you checked [this question](https://stackoverflow.com/questions/50626058/psycopg2-cant-adapt-type-numpy-int64) ? It looks very similar to your problem. – ye olde noobe May 12 '22 at 13:51
  • Hi, yes I had tried this and infact it works as well for a while but after few inserts I get Error: integer out of range , Can't understand the reason for this behavior :/ – simi May 12 '22 at 22:20
  • Edited my post to indicate the recent development – simi May 13 '22 at 08:53

0 Answers0