Run SAS batch jobs via SH files: SAS script contains export to csv - where is / how to get the output?

Question

As university staff, I am allowed to run batch jobs on WRDS. I followed the instructions, also created the wrapper and send the job.

#!/bin/bash
#$ -cwd
#$ -m abe
#$ -M myemail@email.com
echo "Starting Job at `date`"
sas my_sas_script.sas
echo "Ending Job at `date`"

I called the shell script via

qsub my_program.sh

The job finished very quickly(!). In my home directory, I got my logfiles (the "o" and "e" files).

In my SAS script I have an export to CSV line, however, I cannot it anywhere. Is it because the SAS script didn't run successfully? Do I have to adjust the code (In fact it's the code from this question, however not as a not-working macro but as a simple code SAS data step view and data wrap in a macro for loop; I first start with my libraries and then continue with the code).

edit: I run my_program.sh again and got the following log (shortened);

NOTE: Unable to open SASUSER.PROFILE. WORK.PROFILE will be opened instead.
NOTE: All profile changes will be lost at the end of the session.
.
.
.

35         data  _v_&tables / view=_v_&tables;
36              set &taq_ds;
37              where symbol = &stock and   
                           _
                           22
                           76
37       !
WARNING: Apparent symbolic reference STOCK not resolved.
ERROR 22-322: Syntax error, expecting one of the following: a name, 
          a quoted string, a numeric constant, a datetime constant, 
          a missing value, INPUT, PUT.  

ERROR 76-322: Syntax error, statement will be ignored.

38              (time between '9:30:00't and '16:00:00't) and   
39              mode = 12 and                                   
40              EX = 'N';                                       
ERROR: Syntax error while parsing WHERE clause.
41         run;

NOTE: The SAS System stopped processing this step because of errors.
NOTE: DATA statement used (Total process time):
  real time           3.39 seconds
  cpu time            0.73 seconds

46         data xtemp2;
47              set _v_&tables;
47              set _v_&tables;
                ___
                180
ERROR 180-322: Statement is not valid or it is used out of proper order.


48              by symbol date time;
                __
                180

ERROR 180-322: Statement is not valid or it is used out of proper order.

49              format itime rtime time12.;
                ______
                180

ERROR 180-322: Statement is not valid or it is used out of proper order.

50              if first.symbol = 1 or first.date = 1 then do;
                __
                180

ERROR 180-322: Statement is not valid or it is used out of proper order.

51         
54              rtime = time;
              _____
             180

ERROR 180-322: Statement is not valid or it is used out of proper order.

55                 iprice = bid;
                   ______
                   180

ERROR 180-322: Statement is not valid or it is used out of proper order.

56                 oprice = ofr;
               ______
               180

ERROR 180-322: Statement is not valid or it is used out of proper order.

57                 itime = &start_time;
                   _____
                   180

ERROR 180-322: Statement is not valid or it is used out of proper order.

58              end;
                ___
                180

ERROR 180-322: Statement is not valid or it is used out of proper order.

59          
60              if time >= itime then do;
                __
                180

ERROR 180-322: Statement is not valid or it is used out of proper order.

61                    output;
                      ______
                       180

ERROR 180-322: Statement is not valid or it is used out of proper order.

62                    itime = itime + &interval_seconds;
                      _____
                      180

ERROR 180-322: Statement is not valid or it is used out of proper order.

63                    do while(time >= itime);
                      __
                      180

ERROR 180-322: Statement is not valid or it is used out of proper order.

64                        output;
                          ______
                          180

ERROR 180-322: Statement is not valid or it is used out of proper order.

65                        itime = itime + &interval_seconds;
                          _____
                          180

ERROR 180-322: Statement is not valid or it is used out of proper order.

66                    end;
                      ___
                      180

ERROR 180-322: Statement is not valid or it is used out of proper order.

67             end;
               ___
               180

ERROR 180-322: Statement is not valid or it is used out of proper order.

68             rtime = time;
               _____
               180

ERROR 180-322: Statement is not valid or it is used out of proper order.

69             iprice = bid;
               ______
               180

ERROR 180-322: Statement is not valid or it is used out of proper order.

70             oprice = ofr;
               ______
               180

ERROR 180-322: Statement is not valid or it is used out of proper order.

71             retain itime iprice oprice;
               ______
               180

ERROR 180-322: Statement is not valid or it is used out of proper order.

73         run;
74         
75         Title "Final output -- 1min interval"; 
76         proc print data=work.xtemp2 (obs=100);
ERROR: File WORK.XTEMP2.DATA does not exist.
77              var symbol date itime iprice oprice;
ERROR: No data set open to look up variables.
ERROR: No data set open to look up variables.
ERROR: No data set open to look up variables.
ERROR: No data set open to look up variables.
ERROR: No data set open to look up variables.
78         run;

NOTE: The SAS System stopped processing this step because of errors.
NOTE: SAS set option OBS=0 and will continue to check statements. 
  This might cause NOTE: No observations in data set.
NOTE: PROCEDURE PRINT used (Total process time):
  real time           0.00 seconds
  cpu time            0.00 seconds

NOTE: PROCEDURE EXPORT used (Total process time):
  real time           0.00 seconds
  cpu time            0.00 seconds

WARNING: Apparent symbolic reference STOCK not resolved.
NOTE: The SAS System stopped processing this step because of errors.
81         proc export data=work.xtemp2 outfile="/home/Output/
81       ! &filename" dbms=csv;
82         run;
83         
84         DM 'log; file "/home/Logs/ &filename.log" replace';
WARNING: Apparent symbolic reference STOCKLOG not resolved.
84       !                                                                  
85         DM "log; clear; ";                                               

ERROR: Errors printed on page 2.

Try simple program that exports a single table such as sashelp.class -- show the code and the log. If it does not work it might be because of host system security policies. If sashelp.class export does work and you can't still locate the output that would be a puzzling question. If the export path is not specifically set output should go to the current working folder (which is not the WORK folder) http://support.sas.com/documentation/cdl/en/hostwin/63285/HTML/default/viewer.htm#startsas.htm#workdir — Richard, Mar 03 '18 at 15:07
Dear both, thank you very much; I'll run Richard suggestion and then post the log. — eternity1, Mar 03 '18 at 16:21
It looks like your macro var `stock` is not defined correctly. The data step would always take the value of the variable before executing the step. So, either the variable doesn't have a quoted string (in that case, use `"&stock"`), or the value is inconsistent (feed issue). — samkart, Mar 03 '18 at 18:39
Hi samkart, thank you for pointing out. I have defined the variable as a macro variable: %let stock = 'COP'; using ' instead of quotation marks. In fact I do not have multiple set statements as such, but rather have a data step view and then the actual data step. — eternity1, Mar 03 '18 at 18:44

Allan Bowe · Answer 1 · 2018-03-03T18:08:16.467

It would appear that your SAS program is executing just fine, it just fails to finish due to syntax errors, namely (in the first instance) that the macro variable "STOCK" is undefined.

I suggest running the program first in a regular (clean) instance of SAS and make sure it completes without errors.

If it runs fine there, but not in shell batch, then there must be an issue with one of the following:

autoexec (make sure you are running the same autoexec in your batched instance)
config (check the sasv9.cfg file in use)
user permissions (is the batch account the same account / have the same permissions as the 'successful' account?)

Run SAS batch jobs via SH files: SAS script contains export to csv - where is / how to get the output?

1 Answers1

Linked