Running Macros Overnight (SAS)

One of the nuances of data science pertaining to large data sets, is the need to run queries/macros that can sometimes take hours to reach completion. Being away from the computer during the compilation and processing of data, limits the user from being able to check the work product for errors during the compilation process.

As this pertains to SAS Programming, there are few tips that I recommend before running overnight macros on large data sets.

Use OBS =

If it is the size of the data set that creates the processing slowdown, as it often is, then it is probably best to create a test data set for processing prior to running the macro on the original set.

I typically would recommend creating a new set utilizing the first 1,000 – 10,000 observations from the original dataset, and then subsequently running the macro on the test set before proceeding.

To create a test set using the OBS function, utilize the sample code below:

Data SETB;
Set SETA(obs = 10000); /* Reads the first 10000 observations from SETA and creates SETB */
Run;

Now run your macro, and make sure that the code performs the task for which it was created.

Prevent Log from Printing
By default, SAS will halt processing when the output log reaches a set limit. At this point, the user will be prompted with the following dialogue box.

If the user is not present to make a selection, the program halts until prompted otherwise. I neglected to consider this SAS feature when I was a relatively novice programmer, and as a result, I came in one morning to find that a perfectly functional macro that I had ran overnight, never completed processing.

To avoid a similar fate, I would STRONGLY recommend enabling this option prior to running your macro code overnight.

This option prevents messages from being written to the SAS log. Therefore, by enabling this option, such as in the example below, you can sleep easily knowing that your code will run seamlessly, and without SAS system interruptions.

/* Add the following OPTIONS statement to the top of your SAS program to suppress printing to the log: */

options nonotes nosource nosource2 errors=0;

/* If you want to change the options back to the default and start printing to the log again, submit the following OPTIONS statement: */

options notes source source2 errors=20;

It should be noted that green text generated within the SAS log will not be prevented from displaying, even with the above listed option toggled. (Blue text will not be displayed). Therefore, to be absolutely sure that your macro runs without ceasing due to a full log prompt, I would recommend potentially placing one of these code lines within your macro:

DM "Log; Clear; ";  /* Clears the SAS log only */
/* OR */
DM Log "Out; Clear; Log ; Clear;";  /* Clears both the SAS log and the SAS output window */

(Green Text)

(Blue Text)

It should also be noted that text generated from options utilized and enabled for Macro debugging will still run, and still generate text to the SAS log, even if the option above is enabled.

Therefore, I recommend the following complete set of steps for running macro programs overnight:

1. Compile the macro in your current SAS Session with macro debugging options enabled, be sure that there is (DM "log; clear; ";) somewhere within your macro code.

2. If the macro compiled correctly, test your macro on a smaller variation of your current dataset by utilizing the OBS = option. If successful, save your work and exit completely out of SAS.

3. Re-open SAS and the program that you were previously working on. Re-establish the libraries and the options utilized in the prior session. DO NOT re-enable macro debugging options.

4. Re-compile the macro.

5. Clear the SAS Log.

7. Be sure that (options nonotes nosource nosource2 errors=0;) is enabled.

8. Run the macro code.

If you follow these steps, hopefully your overnight macro running will process with ease and without error.