0

i'm trying to import the data from a cURL with the next command in psql:

COPY testtable FROM PROGRAM 'curl https://.....'

This is the data in the URL:

[{"date":"20201006T120000Z","uri":"secret","val":"1765.756"},{"date":"20201006T120500Z","uri":"secret","val":"2015.09258"},{"date":"20201006T121000Z","uri":"secret","val":"2283.0885"}]

But psql returns

ERROR: missing data for column "uri"

I've tried copying it on tables with the columns as text and json format. Also tried adding (DELIMITER ',') but that returns

ERROR: extra data after last expected column

i feel like the problem could be caused from the "[]" in the start and the end of the data, but i im not sure.

These are the definitions of the tables that i used.

Table "public.test_table"

Column Type Modifiers
date text not null
uri text
val text

Indexes: "test_table_pkey" PRIMARY KEY, btree (date)

Table "public.test_table2"

Column Type Modifiers
date json
uri json
val json
Lautaro Aguilera
  • 53
  • 5

1 Answers1

1

COPY only supports csv, text, and binary formats. It does not support JSON. It will import or export data into or out of json fields as a whole, but will not assemble or parse them.

You could use a staging table with one row and one column.

create temp table stage(x jsonb);

COPY stage FROM PROGRAM 'curl https://.....';

insert into test_table select f.* from stage,
   jsonb_populate_recordset(null::test_table, x) f;

If PostgreSQL offered a pg_read_program() function, you could use that directly rather than creating a stage table. But it doesn't (but you could make one in C, or plpythonu or plperlu)

jjanes
  • 37,812
  • 5
  • 27
  • 34