4

I am trying to load integer64 as character in fread ?fread indicates that the integer64 argument is not implemented but the options(datatable.integer64) is. Though fread keeps loading as int64.

How can I tell fread to load as character. EDIT [If colClasses is the answer, I think it does not allow to specify a single column name or index and the table I load has tens of columns so unpracticable... => This was WRONG]

Here is a sample

#for int 64
library(bit64)
#for fast everything
library(data.table)

#here is a sample
df <- structure(list(IDFD = structure(c(5.13878419797985e-299, 5.13878419797985e-299, 
+ 5.13878419797985e-299, 5.13878419797987e-299, 5.13878419797987e-299, 
+ 5.13878419797987e-299, 5.13878419797987e-299, 5.13878419797987e-299, 
+ 5.13878419797988e-299, 5.13878419797988e-299), class = "integer64")), .Names = "IDFD", row.names = c(NA, 
+ -10L), class = c("data.table", "data.frame"))
#write the sample to file
write.csv(df,"test.csv",quote=F,row.names=F)

#I can't load it as characters
options(datatable.integer64='character')
str(fread("test.csv",integer64='character'))
Classes ‘data.table’ and 'data.frame':  10 obs. of  1 variable:
 $ IDFD:Class 'integer64'  num [1:10] 5.14e-299 5.14e-299 5.14e-299 5.14e-299 5.14e-299 ...
smci
  • 32,567
  • 20
  • 113
  • 146
statquant
  • 13,672
  • 21
  • 91
  • 162

1 Answers1

11

This is implemented in v1.8.11, on R-Forge but not yet on CRAN. From NEWS :

o fread's integer64 argument implemented. Allows reading of integer64 data as 'double' or 'character' instead of bit64::integer64 (which remains the default as before). Thanks to Chris Neff for the suggestion. The default can be changed globally; e.g, options(datatable.integer64="character")

Regarding :

If colClasses is the answer, I think it does not allow to specify a single column name or index and the table I load has tens of columns so unpracticable...

colClasses in fread does let you override type for one or a few columns (by name or by number), and the rest will be automatically detected. For exactly the reason you state. If it doesn't, please report as a bug. An alternative to colClasses is the datatable.integer64 global option which lets you tell fread that whenever it detects integer64 it should load it as character or double instead (in v1.8.11 as well).

Matt Dowle
  • 58,872
  • 22
  • 166
  • 224
  • 2
    Understood, just tried `colClasses=c(MyCol1='character',MyCol2='character')` and it magically worked, so simple that I never tried, you should stop making things so easy for us !!! – statquant Sep 10 '13 at 12:15
  • 3
    @statquant Ha. But there's a new even simpler way. Try `colClasses=list(character=c('MyCol1','Mycol2'))`. Or `colClasses=list(character=3:8)`. That way you can save repeating "character" over and over, and pass ranges of columns, etc. – Matt Dowle Sep 10 '13 at 12:37
  • 1
    Can't +1 more than once sorry :) – statquant Sep 10 '13 at 12:41