5

I have a sas7bdat file of size around 80 GB. Since my pc has a memory of 4 GB the only way I can see is reading some of its rows. I tried using the sas7bdat package in R which gives the error "big endian files are not supported" The read_sas() function in haven seems to work but the function supports reading specific columns only while I need to read any subset of rows with all columns. For example, it will be fine if I can read 1% of the data to understand it. Is there any way to do this? Any package which can work? Later on I plan to read parts of the file and divide it into 100 or so sections

Madhur Modi
  • 93
  • 6
  • Is this a commercial, research, educational or not for profit type work? – Reeza Aug 17 '17 at 19:36
  • I am trying it out for educational purposes. I don't have a sas licence – Madhur Modi Aug 18 '17 at 00:19
  • SAS is free for educational purposes. See SAS Analytics U and you can download the software. 80GB is still big, so I do recommend switching your work folder to a shared location rather than the default but you'll be able to process it there. Or at least read it in and export it back out to a txt or whatever form you want. – Reeza Aug 18 '17 at 00:36
  • Is there an alternative which does not require using sas? Basically I want to handle this problem using R – Madhur Modi Aug 18 '17 at 08:55
  • The parsing logic is written in R syntax here: https://github.com/BioStatMatt/sas7bdat/blob/master/R/sas7bdat.R I think you could get frew k rows of raw data and run that without the 'magic number' check. Maybe? – pinegulf Sep 27 '17 at 07:50

1 Answers1

1

If you have Windows you can use the SAS Universal Viewer, which is free, and export the dataset to CSV. Then you can import the CSV into R in more readable chunks using this method.

Craig Srsen
  • 465
  • 4
  • 8