1

I am trying to read a file with 35.000.000 rows and 105 columns in R and decided to use Revolution R Enterprise 7.4, with this code:

 input <- RxTextData(data, isFixedFormat = F,delimiter = "\t")

 s <- rxImport(inData = input,outFile = output,overwrite = T)

but the code throws the following error:

Error in doTryCatch(return(expr), name, parentenv, handler) : Write error: expected 8 got

Does anybody know how can I solve this error?

Thanks

Matt Parker
  • 26,709
  • 7
  • 54
  • 72
narteaga
  • 147
  • 2
  • 12
  • It looks like you left some of the error message out - could you post the rest? Also, does rxImport give you any progress report at all? Something like "10000 rows processed..." That might help you identify a row with an error in its formatting. Finally: you might try setting `numRows = 100` or so, just to see if you can read in any of the rows at all with your current code. – Matt Parker Jun 23 '15 at 01:34
  • thanks for your answer...In the console doesn't appear other error message out neither a progress report !, besides i tried with numRows=100 and the function doesn't read the file, in the console appears the same error. – narteaga Jun 23 '15 at 03:23
  • end of file **end of file Error in doTryCatch(return(expr), name, parentenv, handler) : no message recieved from pipe Error in doTryCatch(return(expr), name, parentenv, handler) : Write error: expected 8 got ** – narteaga Jun 23 '15 at 13:57
  • Hmm... that's a strange error. Can you successfully read the data using `read.table` with `nrows = 100`? Seems like there's a bug somewhere, either in the TSV file or in rxImport. – Matt Parker Jun 23 '15 at 19:19
  • 1
    i include the parameter type="text" 's <- rxImport(inData = input,outFile = output,overwrite = T,type="text)', and the fucntion reads the file correctly – narteaga Jun 23 '15 at 19:55
  • This helped me to get the error less: removing a statement where i set some parallelization options; i removed this line from my code: `rxOptions(numCoresToUse = detectCores() - 1)` and seem to be getting the error less frequently. Can't confirm that this is a complete fix, but seems to help. – RyanStochastic Sep 08 '16 at 15:45

2 Answers2

2

I'm an engineer on Microsoft R Server, and this error is an identified bug that is currently being addressed, This error occurs for data frames over 4GB in size.

It will be fixed in the next release of MRO/MRS.

Kirill Glushko - Microsoft
  • 306
  • 1
  • 10
  • Thanks @Kirill, any estimate of when this will be fixed, or can you suggest a workaround? – RyanStochastic Aug 23 '16 at 20:11
  • @RyanStochastic, We are testing a patch currently. It will be available for versions 7.4.1 through the most current release. Please contact Microsoft Customer Support Service to open a support ticket so we ensure you get the correct patch for your platform. – Kirill Glushko - Microsoft Aug 30 '16 at 00:47
  • @RyanStochastic, We have finished certification on both Windows and Linux patches, and they are available for public download here: https://support.microsoft.com/en-us/kb/3171055 – Kirill Glushko - Microsoft Nov 28 '16 at 20:20
  • Hi Kirill, I'm still seeing this issue on enabling 'RevoScaleR' package in RStudio. > library("RevoScaleR", lib.loc="/opt/cloudera/parcels/MRS-9.0.1/lib64/R/library") Error : .onLoad failed in loadNamespace() for 'RevoScaleR', details: call: doTryCatch(return(expr), name, parentenv, handler) error: message header: expected 8 got Error: package or namespace load failed for ‘RevoScaleR’ – lallantop Jan 03 '17 at 18:59
  • @pikoooz, This occurring right when you attempt to load the library is a slightly different issue, and it is indicative of an issue particular to your installation. We have seen issues with RStudio + RevoScaleR + Cloudera. please ensure that you run through the following guide: https://support.rstudio.com/hc/en-us/articles/235841328-Using-RStudio-Server-with-Microsoft-R-Server-for-Cloudera If these do not work, please contact Microsoft Support so we can start an investigation into your specific issue. – Kirill Glushko - Microsoft Jan 19 '17 at 22:15
1

The patch for Windows and Linux has been made public Please follow this link to download the patch: https://support.microsoft.com/en-us/kb/3171055

This patch is for current release versions (Linux version 8.0.5 and Windows version 8.0.3)

If you require this patch for a different version, please contact Microsoft support or feel free to address your concerns on this question on StackOverflow.

-Kirill

Kirill Glushko - Microsoft
  • 306
  • 1
  • 10