appending files in a macro using sas

Question

Actually I have just started learning the SAS and I am not that good in macro. I am working on large data which is divided into different parts for each day. But at the end I have to append them as a single day.The code I have used is-

  options nosource fullstimer cpucount=2;

  options nonumber nodate nocenter source mprint FMTSEARCH=(iu) ls=180 ;

  options errors=2;
  dm 'clear list'; dm 'clear log';
  libname taq 'G:\cq2011';
  libname mytaq 'G:\taqsas11';
  libname iu 'G:\program\iustructure';
  libname ct 'G:\ct2011';
  libname ctq2 'G:\ctq2011';
  libname no 'G:\nbbo2011';

  option msglevel=i mprint source nodate nonumber replace fullstimer notes;
  %let iupath=F:\program\iustructure;

  data _null_; time=time(); put time=; format time time11.2; run;

  %let me1=A; %let me2=B; %let me3=C; %let me4=D; %let me5=J; %let me6=K; 
  %let me7=M; %let me8=N; 
  %let me9=P; %let me10=T; %let me11=W; %let me12=X; %let me13=Y; %let 
   me14=Z; 
  %let me15=I; %let me16=S; %let me17=Q; %let me18=V;


  proc printto log="G:\saslog\sas2011.txt";
  run; 


  DATA dno;
  INFILE 'G:\cq2011\cqdate2011.txt';
  INPUT dno $ 1-12;
  run;
  proc contents; 
  PROC PRINT; RUN; 
  DATA tno;
  INFILE 'G:\ct2011\ctdate2011.txt';
  INPUT tno $ 1-8;
  RUN;
  proc contents; 
  PROC PRINT; RUN; 

  %MACRO read(n);

  %DO s=1 %TO &n;

  DATA _null_;
      SET dno;
        IF _n_ =&s THEN DO; 
              CALL SYMPUT('dfile', dno);

      END;
  RUN;
  DATA _null_1;
      SET tno;
        IF _n_ =&s THEN DO; 
              CALL SYMPUT('tfile', tno);
      END;
  RUN;


    %let extract=%substr(&dfile,1,8);
    %let extra=%substr(&tfile,1,8);

   %let mon=%substr(&dfile,6,1);
   data nbbotmp(keep=symbol TIME bestbid bestofr totbdp totodp);
   set taq.cq_&dfile;
   by symbol time;
   array exBid(18); array exOfr(18); array exBidsiz(18); array exOfrsiz(18);
   retain exBid1-exBid18 exOfr1-exOfr18 exBidsiz1-exBidsiz18 exOfrsiz1- 
   exOfrsiz18 bestbid bestofr ;
   if mode in (4, 7, 9, 11, 13, 14, 15, 19, 20, 27, 28) then do; bid=0; 
   ofr=9999999; end;
    else if bid<=0 then do; bid=0; ofr=9999999; end;
    else if ofr<=0 then do; bid=0; ofr=9999999; end;
    else if bidsiz<=0 then do; bid=0; ofr=9999999; end;
    else if ofrsiz<=0 then do; bid=0; ofr=9999999; end;
    else if 0>ofr-bid then do; bid=0; ofr=9999999; end;
    else if ofr-bid>5 then do; bid=0; ofr=9999999; end;
   if first.symbol then do;
    do i=1 to 18; exBid(i)=.; exOfr(i)=.; exBidsiz(i)=0; exOfrsiz(i)=0; end;
    end;
   %do l=1 %to 18; %if &l^=1 %then else; if ex="&&me&l" then kEx=&l; %end;
   if kEx>0 then do;
    exBid(kEx) = bid; exOfr(kEx) = ofr; exBidsiz(kEx) = bidsiz; exOfrsiz(kEx) 
    = ofrsiz;
    bestbid = max(of exBid1-exBid18); bestofr = min(of exOfr1-exOfr18);
    end;
    if bestbid=0 then do; bestbid=.; bestofr=.; end; *set strange quotes to 
    be missing;
    else if bestofr=9999999 then do; bestbid=.; bestofr=.; end;
    else do;
        totbdp=0; totodp=0;
        %do l=1 %to 18;
            if exBid(&l)=bestbid then totbdp=totbdp+exbidsiz(&l);
            if exofr(&l)=bestofr then totodp=totodp+exofrsiz(&l);
        %end;
    end;
    if last.time then do; time=time+1; output; end;
   length totbdp totodp 4;
   run;


    options replace;
    data no.nbbo_&dfile (drop=lbo lbb lbod lbbd ls  compress=yes);set 
     nbbotmp; by symbol time;
     lbo=lag(bestofr); lbb=lag(bestbid); lbod=lag(totodp); lbbd=lag(totBdp); 
       ls=lag(symbol);
      if bestofr ne lbo then output;
     else if bestbid ne lbb then output;
        else if totodp ne lbod then output;
            else if totBdp ne lbbd then output;
                else if symbol ne ls then output;
        run;

    data qsymbol; length dfile $ 11;
    set no.nbbo_&dfile (keep=symbol); by symbol; 
    if first.symbol then do; dfile="&dfile"; output; end; 
    run;


      %if &s=1 %then %do; data mytaq.qsymbol2; set qsymbol; run; %end;
      %else %do; proc append base=mytaq.qsymbol2 data=qsymbol force; run; 
      %end;
     %END;

    %MEND; 
    %read(2); 

Our data consists of files of type 20110101_1 to 20110101_n, 20110102_1 to 20110102_m and so on for the whole year. We need to merge the files for one date into a single file.That is,we want to append 20110101_1 to 20110101_n into one single file named 20110101_1. Note that the number of subfiles for each date varies.

Answer 1

Your question isn't 100% clear to me, but I'm interpreting it as:

How do I combine files together with a similar naming scheme

You can do this a number of ways, but there are two simple ways.

If you have a dataset of filenames, which it looks like you do (?), then you can write an append macro like so:

%macro append_file(base=, data=);
  proc append base=&base. data=&data. force;
  run;
%mend append_file;

And then call it from the dataset that has the filenames.

data _null_; *initially change this to something so you can debug it;
  merge dno tno; *it looks like a side-to-side is okay here?;
  call_str = cats('%append_file(base=',tno,',data=',dno,')');
  call execute(call_str);
run;

Second, if your filename is derived from just, _1 _2 _3 _4, you can do it fairly simply by relatively simplistic macro loops.

%macro dnames(n);
%do i = 1 to &n;
&prefix._&n.
%end;
%mend dnames;

data want;
  set 
    %dnames(4)
  ;
run;

The loop you have above is a more complex version that seems to sort of combine both of these. It's not sufficiently explained from your question what the other requirements are, and it may be that they're needed, but odds are you can fit it closer to one or the other of these methods.