I have several CSV files that I'm trying to upload to a PostgreSQL database.
My current file/function setup works perfectly fine for files without NULLs, but it's when I have null values that I hit an issue.
In the CSV, those nulls are currently empty cells. The function I'm using to upload them to the database looks like this:
insert_query_string = ("""
INSERT INTO sample_table (
primaryKey,
color,
date,
place,
numeric1,
numeric2,
numeric3)
VALUES (%s, %s, %s, %s, %s, %s, %s)
""")
def loadData(cur, conn, query):
"""
Loads data blahblahblah
"""
try:
data_csv = csv.reader(open(data_path + 'data.csv'))
header = next(data_csv)
for row in data_csv:
print(row)
cur.execute(query, row)
print(' - data loaded.')
conn.commit()
except FileNotFoundError:
print("CSV file not found.")
def main():
conn = None
try:
# connect
print("Connecting ... ")
conn = pg.connect(**nfl_params)
cur = conn.cursor()
# load
print("Loading data ... ")
loadData(cur, conn, insert_query_string)
except (Exception, pg.DatabaseError) as error:
print(error)
finally:
if conn is not None:
conn.close()
print("Database Connection closed.")
if __name__ == "__main__":
main()
Some of the numeric columns are null, and yet I can't seem to figure out how to get them into the database. As is, I'm getting invalid input syntax for type numeric: ""
so I thought it was trying to read it as a string or something.
This stackoverflow question seems to be saying to replace the empty cells to 'null' or 'NULL' but that doesn't seem to work either.
My problem, as far as I can tell, is because csv's don't have anyway to store null values other than empty, which isn't working for me. I'm thinking for now that I'll replace the null values with -1 or something, but this doesn't seem to be the cleanest way to do it.
Thanks in advance!