Questions tagged [postgresql-copy]

COPY is an SQL command in PostgreSQL to move data between files and tables. There is also the meta-command \copy in the psql interface.

Besides the shell utilities pg_dump and pg_restore for backup and restore, there is also the SQL command COPY in PostgreSQL to move data between files and database tables quickly. Several file formats are supported: text, csv and binary.

The meta-command \copy in the psql interface is a wrapper for the SQL command that reads and writes files local to the client (while COPY is for files on the server).

Examples

Write all rows from a table employees to a CSV file on the DB server with SQL COPY:

COPY employees TO '/path/to/employees.csv' (FORMAT csv);

There are more examples for COPY in the manual .

Read data from a CSV file on the client machine into a table with matching structure with \copy in psql (appending to existing data):

\copy employees FROM '/path/to/employees.csv' (FORMAT csv);
147 questions
0
votes
2 answers

PostgreSQL benchmarking over a RAMdisk?

I have been considering the idea of moving to a RAMdisk for a while. I know its risks, but just wanted to do a little benchmark. I just had two questions: (a) when reading the query plan, will it still differentiate between disk and buffers hits? If…
Zeruno
  • 1,391
  • 2
  • 20
  • 39
0
votes
0 answers

How to COPY CSV file into table resolving foreign key values into ids

I'm expert at mssql but a beginner with PostgreSQL so all my assumptions are likely wrong. Boss is asking me to get a 300 MB CSV file into PostgreSQL 12 (1/4 million rows and 100+ columns). The file has usernames in 20 foreign key columns that…
0
votes
1 answer

is there an way to upload 212 columns csv files in PostgreSQL

I have a csv file with 122 columns I am trying this in Postgres. I am trying this create tble appl_train (); \copy appl_train FROM '/path/ to /file' DELIMITER ',' CSV HEADER; I get this error ERROR: extra data after last expected column CONTEXT:…
0
votes
2 answers

How do I solve "ERROR: missing data for column" with "psql"?

I have a makefile in which I am trying to copy the output of a Python program into a table hosted on my PostgreSQL server. My query looks something like this: Python3 filter.py | psql -X -U $(DBUSER) -d $(DBNAME) -h $(DBHOST) -p $(DBPORT) -1 -e \ …
0
votes
2 answers

How to import table from PostgreSQL to SAP Hana?

I'm trying to export data from PostgreSQL and import it into SAP Hana. The problem which is '\n' i.e. line breaks are getting automatically removed from the TEXT data. Example: A 1,750 word essay is 11 to 12 paragraphs. A 2,000 word essay is 13 to…
0
votes
2 answers

Quoted NULL values makes PostgreSQL COPY command fail

I have a big CSV file with all columns quoted with ". Null values are represented as "", and column separator is |. I would like to use the COPYcommand from postgresql (version 10.7) to ingest those files. I have tried many combination, but the more…
jlandercy
  • 7,183
  • 1
  • 39
  • 57
0
votes
1 answer

How to upload script on pgadmn?

I cannot upload the script on PGADMN. Please help me in resolving error. SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET check_function_bodies = false; SET client_min_messages = warning; SET…
0
votes
0 answers

Method to Copy data into a View in Postgresql

The PostgreSQL documentation states that the COPY command can only be done on Tables, but I need to import a massive CSV file into a view. The reason it needs to be a view, is because the CSV I've been given has a large number of [unnormalized]…
Guy Park
  • 959
  • 12
  • 25
0
votes
0 answers

Is there a COPY command equivalent for unselecting columns in pgAdmin whose values are meant to be default?

Several of my tables have fields with default values, including this one: CREATE TABLE IF NOT EXISTS checks ( shipmentid INT NOT NULL, status VARCHAR(10) DEFAULT 'COUNTED' NOT NULL, date DATE NOT…
0
votes
1 answer

PostgreSQL COPY, Is that possible to write data into multiple tables at once

I have a bulk dataset which has to be loaded into multiple PostgreSQL tables. Is that possible to accomplish this by using COPY command inside a single transaction? So I can rollback everything if something goes wrong.
srimaln91
  • 1,176
  • 10
  • 21
0
votes
1 answer

ERROR: extra data after last expected column on postgres

When I tried to copy a very large txt file into my postgres database, I got a following error below. Note that I created a table with a single column and am not using any delimiter when importing the txt file. db1=# create table travis_2018_data (v…
Jov
  • 83
  • 1
  • 2
  • 12
0
votes
1 answer

Export PostgreSQL table to CSV with headers only if number of rows in table is more than 0

I am exporting the table to CSV with header using COPY command in postgreSQL. Even if the table result is 0 it is creating CSV file only with header. I need CSV file with header only if number of rows is greater than 0.
Karthik M L
  • 21
  • 1
  • 4
0
votes
0 answers

Export to CSV file with windows path

I'm using Putty to connect to Postgres 9.6. I'm trying to export to csv five but I keep getting this error: ERROR: relative path not allowed for COPY to file This is my code: COPY (select * from res_users) TO E'G:\\Test.csv' DELIMITER ',' CSV…
Long
  • 13
  • 1
  • 4
0
votes
1 answer

How copy from csv to postgresql table and ignore columns without header

I have an script in python that basically have this workflow Receive a csv with header. Create table on database on PostgreSQL where the fields are the headers of csv. Copy the data of csv to table created in step2 here a fragment of code of…
Cyberguille
  • 1,552
  • 3
  • 28
  • 58
0
votes
1 answer

Postgresql, using Copy function to export results to XML file

I have written some SQL functions which output XML results. Since the output size is big, I need to write them directly to a xml file. To this I'm using COPY function as follows: COPY(SELECT generate_xml_file()) TO '/Users/TEMP/test.xml' But the…
KTB
  • 1,499
  • 6
  • 27
  • 43