2

I have a dataset of 7 million observations that I need to split it into 5000 datasets and export into csv file using where condition. I am running loop from 1 to 5000 and creating dataset as:

Loop start(i=0 to i=5000);    
    data x;
    set data y;
    where column = i;
    run;

    proc export data=x outfile ="path" dbms=csv
    replace;
    quit;

    loop end;

But it is taking too much time.

Can anyone suggest how to do it faster?

James Z
  • 12,209
  • 10
  • 24
  • 44
Shashi Bhushan
  • 21
  • 1
  • Does this answer your question? [Split large SAS dataset into smaller datasets](https://stackoverflow.com/questions/21439993/split-large-sas-dataset-into-smaller-datasets) – Llex Dec 30 '19 at 13:41
  • Here is a hint. There is no need to use any procedure to write a CSV file. You can do it with a data step. And you can write many files with a single data step. – Tom Dec 30 '19 at 15:34
  • 7E6 rows is not a huge data set. What are the naming conventions for the created csv files ? How many columns does the data set contain, and what kind of variables ? Can you add a proc contents report to the question ? As for "too much time"... What hardware is the data set residing on -- is it local disk or connection to remote data base? As for target path is it local disk or network share ? For local disk resources are you using spinning disks or ssd ? – Richard Jan 01 '20 at 13:00

0 Answers0