R feather preserve large numbers

Question

I have a CSV (that I read in using fread from data.table) that has some rather large numbers in it which I convert to feather format and then save as a feather file, but when I read in this new file the numbers are totally different. This occurred on 777 different CSV files that I converted so it is not something specific to the CSV. I tried making a reproducible example but found that there was no issue if I generated the numbers in R.

Code with data from CSV:

options(digits = 22)
t<-tempfile()
i<-as.data.frame(data$total)
head(i)

            x
1 10661832289
2 10836572665
3 11011492485
4 11180245212
5 11331994931
6 11486446777

feather::write_feather(i,t)
feather::read_feather(t)

# A tibble: 36 × 1
                         x
                     <dbl>
1  5.2676450557158410e-314
2  5.3539782724388232e-314
3  5.4404001462775578e-314
4  5.5237750713302844e-314
5  5.5987493942542471e-314

Reproducible code: Running this code does not result in the same problem. Thus it must be a retained class from reading in the CSV.

options(digits = 22)
t<-tempfile()
i<-as.data.frame(runif(10,10000000000,13000000000))
head(i)
feather::write_feather(i,t)
feather::read_feather(t)

My session info:

sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] bit64_0.9-5 bit_1.1-12 

loaded via a namespace (and not attached):
 [1] lazyeval_0.2.0    plyr_1.8.4        assertthat_0.1    hms_0.3          
 [5] tools_3.3.3       haven_0.2.1       tibble_1.2        Rcpp_0.12.9      
 [9] feather_0.3.1     data.table_1.10.4 xlsxjars_0.6.1    rJava_0.9-8      
[13] xlsx_0.5.7

@HongOoi it shows as str(i) : 'data.frame': 36 obs. of 1 variable: $ x:Class 'integer64' num [1:36] 5.27e-314 5.35e-314 5.44e-314 5.52e-314 5.60e-314 ... I see it already has the incorrect numbers - do you think it might be something to do with the bit64::integer64 ? — Geyer Bisschoff, Mar 14 '17 at 10:44
Yep. You should file an issue with feather; as a workaround, specify the class of your variable as `numeric` when reading it in. — Hong Ooi, Mar 14 '17 at 10:46

score 0 · Answer 1 · answered Jul 25 '22 at 22:22

0

I had a similar problem with saving data from a CSV file to a feather file. I was able to correct the problem by changing the number format from integer to numeric.

answered Jul 25 '22 at 22:22

Wesley Burnett

1

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jul 27 '22 at 00:44

R feather preserve large numbers

1 Answers1

Linked