Cool V9 SAS Compress() Function Tricks

In SAS Version 9 there is a new option available for the compress() function. This new third option allows you to use "modifiers" to modify what compress() is doing. There are too many modifiers to list here, but they are worth looking up in the SAS V9 documentation.

Here is an example of a snippet of code I recently created to get rid of "non-printable" hex chars. This is a pretty standard data cleaning routine and is quite useful when some bad hex chars can creep into your text data. Instead of hunting and pecking for the funky hex chars you can just tell compress() to keep only the
"printable characters".

data _null_;
  x = 'A '  '16'x  'bad'  '18'x  ' sequence, with puncuation?';

  put x=;
  x = compress(x,,"kw");  * k is for keep, w is for "write-able";
  put x=;
run;

Notice in the compress() function there is no second parameter, and there is a new third parameter specified: "kw".
K is for keep, and W is for write-able. So this reads as keep only
a-zA-Zwhitespace0-9punctuation.

Pretty nice, eh?

As I said, there is a bunch of other modifiers available so take a look at the documentation. And happy coding!

Also, there are more examples of using the compress function with the optional third argument at my i-Doc site: http://idoc.pelicanprogramming.com/functions/COMPRESS.html

Creating Numeric Buckets

The other day I was writing some code that was needed for a report. Part of the report was to take a number (integer) and fit it into a set of "buckets" at intervals of 100, rounded up. Confused? Here's some examples of what I needed:
3 --> 100
101 --> 200
1536 --> 1600
64 --> 100


Here's the line of code I used to accomplish it:
newNumber = ( ceil( myNumber/100 ) ) * 100;

Certainly not the most cerebral code ever written, but (hopefully) worth sharing.

This type of problem (creating numeric buckets) is fairly common and I was wondering if anyone else had a different way of solving it?

Hex It

Sometimes you need to specify an ASCII text character you can't see or print. You can specify any ASCII character using it's hex value and a hex literal in SAS. A hex literal in SAS is any of the 16 hex characters(0-9 and A-F) in quotes followed by an x. Such as '3A'x. You can see all the hex values for ASCII characters here: www.lookuptables.com

A classic example of this is creating a tab delimited file using a data _null_ step.

data _null_;
  set myData;
  put @01 var1 '09'x
      @10 var2 '09'x
  ;
run;

Another useful time to specify hex characters is to get rid of them. Suppose you
have some "dirty data" that somehow has some weird non-printable characters in it. You look at the text using the hex32 format and discover that some form feed characters somehow snuck into there (0x0C). The easiest way to get rid of them is to compress() the variable. Such as

myVar = compress(myVar,'0C'x); 

This will remove all occurrences of the character specified from the text variable.

Input Into Numeric

Many times you have a variable in SAS that is character and you want to convert it to numeric. This tends to come up a lot when importing from Excel. Excel shows a number, but SAS reads the column in as a character variable.

It is a bit difficult to *replace* the original character variable with a numeric one, but it is trivial to create a new numeric variable. Just use the input() function.

The syntax is:
numericVar = input(charVar, informat.);

NumericVar is the numeric variable you are hoping to create.
CharVar is the character variable that holds the 'number'.
Informat is a numeric informat that tells SAS how to translate the numeric 'characters' into a useful number. Dates are often used to illustrate this concept:

data _null_;
  charDate = '01mar06';
  numDate = input(charDate, date7.);
run;

In the above case, date7. tells SAS how to interpret the character string '01mar06'
into a number (in this case, the number of days since Jan 01, 1960).

Of course your character variable can be something as simple as '1234'. In that case this would work:

data _null_;
  charVar = '1234';
  numvAR = input(charVar, 4.);
run;

People often confuse the input() function with the put() function. I always
remembered by emphasizing the n sound with this little refrain:

Input Into Numeric.

Macro Debugging

Here's another SAS options related post. Have you ever had the frustration of debugging a big macro you didn't write? (Who hasn't!)

If you answered "yes" to the above then you probably know all about options MACROGEN and MPRINT. And you've probably spent a considerable amount of time staring at the log and all the messy MPRINT statements. Often the "bug" you are trying to find isn't necessarily in the macro code itself, but in the code it generates. Here's an easy way to get to that generated code so you can work directly with the logic it contains.

filename mprint "/tmp/code_to_debug.sas";
options mfile;

This will take the code generated by any subsequent macros and write it to the external file referenced by the filename statement.

Options?

Here's something a little different. I have a colleague who asked if there was a way to save the current SAS options and then restore them somewhere in the middle of a job stream. Instead of keeping track of what linesize, pagesize, etc had been set to, he wanted to just reset everything back to some "base". I did not know a way to do that off the top of my head, but after a little poking around in the onLineDoc I thought I had found the answer:

proc optsave saves your current options to a data set or to a registry key.

proc optload loads and sets the options from a data set or registry key.

So at the beginning of your sas session you could have code like this:

proc optSave data = work.myOptions;
run;

And then anywhere in the code that you wanted to set the options back to the way they were when sas started you could run code like this:

proc optLoad data = work.myOptions;
run;

You would think that would do the trick, right?

Unfortunately it doesn't seem to work as advertised. My colleague wrote back with the following:

Try this code. I got inconsistant result:

options ls=100;
proc optsave out=work.ycOpt1;run;
%put ls is %sysfunc(getoption(LS));

options ls=120;
proc optload data=work.ycOpt1;run;
%put ls is %sysfunc(getoption(LS));

When it gets restored linesize (ls) is. . . 96!?

This is using SAS 9.13 on Windows XP. However, the exact same code works as expected under Unix and the linesize option gets correctly restored to 100.

What gives?

How To Be Nice

Now that my daughter is mobile, she is constantly interacting with other kids at the park. Being the little social butterfly that she is, she has no problem walking up to other kids and trying to take their toys. Of course, at 15 months old she doesn't really know any better, but I still find myself trailing behind her saying "Be nice. Play nice." Some day soon she might actually listen.

But if you're using batch SAS on Unix, you can be nice today!

The "nice" command is used to raise/lower the priority of your background sas jobs. Generally we all run jobs at the same nice priority, but by lowering your priority you can let your big background jobs run without interfering with other people's jobs. This can be useful during the heavy use times and you are not too concerned about whether your job runs in 30 mins or 60 mins. It essentially lets you get out of the way of other users without putting your jobs on hold. Nice!

Here's how it works. The higher the nice number the lower your priority. The syntax of nice is:
nice -N /your/command/
where N is the number to move your priority. Positive lowers your priority by that amount, negative raises your priority by that amount.

So instead of me running my sas command like this:
$ sas myBigSAS.sas -autoexec "/my/autoexec.sas" &

I can be nice and run it like this:
$ nice -15 sas myBigSAS.sas -autoexec "/my/autoexec.sas" &

That will run my command with a higher nice value (low priority) freeing up resources for other users.

MOD For Append

Well, it's been a while since the last post. And I don't really have a good excuse other than life sometimes gets busy. But anyways, if anyone is still reading or happens to stumble by, here is another little SAS tip for you:

An easy (and fast!) way to APPEND to a text file is to use the MOD option on the file statement.

Such as:

data _null_;
file outFile MOD;
put 'new stuff being added to the end of the file';
run;

Macro %str() Tip

Today's post is just a quick little SAS MACRO tip. Suppose you are working in a macro and you need to compare a macro variable to an empty string. You can simply say:

%if &myVar = %then ...;

But that is not the most intuitive, especially when you have more complex logic such as:

%if &myVar = and &nextVar = something %then ...;

Looks a little confusing. Like I forgot to type something after the equals sign!

I like to handle this by using %STR( ). That way you can see that I am definetly testing for a blank. Such as:

%if &myVar = %str( ) %then ...;

Happy coding!

Stop Stop Stop Stop Executing!

Yikes! Here's a little something that seems quite obvious, but had never really occured to me. The other day I was talking to another programmer from businessresearchers.com about getting SAS to stop if there is an ERROR when running in batch mode. We don't want the SAS session to abort, just stop processing the code.

If you run something like the following in batch mode you might be a bit surprised at the results:

data junk;
  do i = 1 to 10;
  output;
  end;
run;

* purposeful error in this libname and data step;
libname s "sdfasdfjsadf/";

data junk2;
  set s.something;
run;

* SAS will now set obs=0 and go into "syntax check mode"
  whatever that's supposed to mean...;


* but look what happens here;
proc sql;
  insert into junk
  set i = 11;
quit;

So SAS sets obs=0 and enters syntax check mode,
but the SQL insert still executes. I am pretty sure SQL insert, update and delete will all be executed. It makes sense if you consider that obs=0 is an option that affects input not output; ie, it limits the observations being read into a step not being output. But still, it's a little counter-intuitive that "syntax check mode" would even allow a step to execute at all...

As far as I know, the only real way around this is to wrap your code in a macro and check the value of &SYSERR before any steps you DEFINETLY do not want executed if there is an error. Could this be another opportunity to use %GOTO?

SUGI 31 Paper Deadline

Get those proofreaders proofing! The deadline is today!

Documentation Tools

What tool do you use to document systems built with SAS? By "system" I am thinking of a solution built out of multiple programs. I have traditionally used excel and dropped boxes and lines everywhere a-la visio to diagram program flow. I'll be the first to admit; it's not perfect, but it works. Kinda.

Are there any good open-source tools that you like to use to do this type of system documentation?