0

I have a makefile in which I am trying to copy the output of a Python program into a table hosted on my PostgreSQL server.

My query looks something like this:

Python3 filter.py | psql -X -U $(DBUSER) -d $(DBNAME) -h $(DBHOST) -p $(DBPORT) -1 -e \
   -c "COPY table1(col1, col2) FROM STDIN with (format csv, header true, delimiter '|')"

However, when I execute it I get the following error message:

ERROR:  missing data for column "col1"
CONTEXT:  COPY table1, line 168061: ""

To try and understand the issue, I exported the output of the Python program to a CSV file and I realized that the issue comes from the fact that there are no more entries after line 168060. So line 168061 is indeed empty.

I have tried using the NULL option for COPY but it did not work, I get the same error.

I also managed to import into table1 the content of the physical CSV file using a very similar COPY statement. This works, but my objective would be to achieve this without having to create a file.

Any ideas?

Laurenz Albe
  • 209,280
  • 17
  • 206
  • 263
Christian Helleboid
  • 1
  • Hi Christian, Try to write a more clear title, something more specific so the people can have a clue before reading the content. By the way, have you tried not sending the empty line at the end? – mnesarco Dec 02 '19 at 14:10
  • Thanks, indeed it was my first post so I will try to be more precise next time :) – Christian Helleboid Dec 04 '19 at 13:00

2 Answers2

0

The solution is to change filter.py so that it doesn't output the offending empty line.

Laurenz Albe
  • 209,280
  • 17
  • 206
  • 263
0

thank you all for your answers,

Indeed I simply did an easy modification in the filter.py and it works now. I was just too focused on doing it in during the import with PSQL.

Cheers :)

Christian Helleboid
  • 1