R fails to write table to PostgreSQL via rPostgreSQL

Question

I am new to PostgreSQL and I'm trying to use R to write table to PostgreSQL. See the R code below; make some reproducible data frame first:

> Time<-c('201512', '201511', '201510')
> Department<-c('ABC', 'BCA', 'NBA')
> Pro_Type<-c('standard', 'maintain', 'sustaining')
> Man_month<-c('111.4', '124.1', '232.1')
> ID<-c('1','2', '3')
> melt_short<-data.frame(Time, Department, Pro_Type, Man_month, ID)
> melt_short
   Time  Department   Pro_Type Man_month ID
1 201512        ABC   standard     111.4  1
2 201511        BCA   maintain     124.1  2
3 201510        NBA sustaining     232.1  3

then creates the connection:

 require("RPostgreSQL")

# loads the PostgreSQL driver
  drv <- dbDriver("PostgreSQL")
# creates a connection to the postgres database
  con <- dbConnect(drv, dbname = "XXX",
             host = "XXX.XX.XX.XXX", port = 5432,
             user = "postgres", password = XXXXXX)

then I create a table in PostgreSQL DB:

 CREATE TABLE dms_melt
(
  "Time" numeric,
  "Department" character(1),
  "Pro_type" character(1),
  "Man_month" numeric,
  "ID" numeric NOT NULL,
  CONSTRAINT dms_melt_pkey PRIMARY KEY ("ID")
)
WITH (
  OIDS=FALSE
);
 ALTER TABLE dms_melt
 OWNER TO postgres;

Finally execute the write table code and got the error:

dbWriteTable(con, "dms_melt", value = melt_short, append = T,row.names=F)

Error in postgresqlgetResult(new.con) : 
RS-DBI driver: (could not Retrieve the result : ERROR:  value too long for   
type character(1) CONTEXT:  COPY dms_melt, line 1, column Department: "ABC"

It should be not so hard but I've been searching for solution for a while but in vain. And actually I've used the "mtcars" data set to write table and with no problem.

data(mtcars)
df <- data.frame(carname = rownames(mtcars), 
                 mtcars, 
                 row.names = NULL)
df$carname <- as.character(df$carname)

dbWriteTable(con, "cartable", 
             value = df, append = TRUE, row.names = FALSE)
[1] TRUE

inserted the PostgreSQL which I created as:

CREATE TABLE cartable
(
  carname character varying,
  mpg numeric(3,1),
  cyl numeric(1,0),
  disp numeric(4,1),
  hp numeric(3,0),
  drat numeric(3,2),
  wt numeric(4,3),
  qsec numeric(4,2),
  vs numeric(1,0),
  am numeric(1,0),
  gear numeric(1,0),
  carb numeric(1,0)
 )
 WITH (
  OIDS=FALSE
 );
 ALTER TABLE cartable
 OWNER TO postgres;

Thanks in advance for any suggestions and ideas.

@Parfait: Hi, it's factor in the data frame. Thanks. – Samoth Apr 16 '16 at 03:35 — Samoth, Apr 16 '16 at 03:35

score 3 · Accepted Answer · answered Apr 16 '16 at 05:09

Pay attention to the data type of the following two table columns:

...
"Department" character(1),
"Pro_type" character(1),
...

Now let's look at the data you're trying to insert:

   Time  Department   Pro_Type Man_month ID
1 201512        ABC   standard     111.4  1
2 201511        BCA   maintain     124.1  2
3 201510        NBA sustaining     232.1  3

Now the error message:

Error in postgresqlgetResult(new.con) :
RS-DBI driver: (could not Retrieve the result : ERROR:  value too long for
type character(1) CONTEXT:  COPY dms_melt, line 1, column Department: "ABC"

The character(N) data type in PostgreSQL is a fixed-length string type. Every value in that column must be a string of characters with length N. For character(1), this means every value must comprise a single character.

The value 'ABC' is a string with length 3. It is too long for the character(1) data type.

You may be confusing the R character vector type with the SQL character type. In R, each element of a character vector is a variable length string. Ironically, you can construct a character vector in R with the syntax character(N) where N is the number of variable-length string elements in the resulting vector. In SQL, you can view a column as being a vector, but data types always refer to the scalar type of a single cell value. The N in the SQL syntax character(N) refers to number of characters in each string.

The reason your mtcars test worked is because the only string column in that table is carname which has data type character varying with no length specification. From the PostgreSQL documentation:

If character varying is used without length specifier, the type accepts strings of any size. The latter is a PostgreSQL extension.

So, you can solve the problem by using character varying (or its more concise name varchar) for these two columns. You can omit the length specification just as in the mtcars schema, or if you are aware of a contractual length limit on the values of those columns you can specify varchar(N) where N is the length limit. Also, if Department values are always 3 characters long, you have the option of going with char(3) for a fixed-length type.

Summary:

SQL
==========

character(N)            fixed-length strings of length N
char(N)                 alias for character(N)
character               alias for character(1)
char                    alias for character(1)
character varying(N)    variable-length strings of maximum length N
varchar(N)              alias for character varying(N)
character varying       variable-length strings of unlimited length
varchar                 alias for character varying

R
==========

character(N)            vector of length N of variable-length strings

R fails to write table to PostgreSQL via rPostgreSQL

1 Answers1