How to use RevoScaleR to read SAS data chunk-by-chunk and output into a format other than xdf?

Question

I am trying to develop some code using RevoScaleR and noticed that it has options like rowsPerRead and can output xdf files in chunks using the rxImport method. However my use cases I want to apply some custom transformation (to the entire chunk; not just columns) and output the results in fst format instead of xdf (as I find it to be faster).

Is there a way to apply customer transforms of each chunk being read it using RevoScaleR?

Answer 1

Following the example it's like the below except it doesn't seem to work and just eats up lots of RAM (at least on Windows)

library(RevoScaleR)
a = RxSasData("file.sas7bdat")
rxOpen(a)
rxNext(a)
rxClose(a)