Category Archives: SAS Replacement

WPS v3.2.2 Released

 

Earlier in the Week, World Programming released an update to WPS. Version 3.2.2 is mainly a maintenance release with a number of fixes.  There are some improvements and the two that caught my eye are:

25591: WPS can now handle record lengths up to, or even greater than 32K when writing to SAS7BDAT files.

25596: WPS on Linux now supports Sybase IQ 16.0 client drivers.

There are number of other fixes that are probably more important than the two I chose above (especially if you are on MVS).

You can upgrade your installation by going to the WPS Website and logging into the download servers (User ID and Password required.) You can also read a list of all the changes by clicking on the change log file on the right hand side of screen.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is an authorized reseller of WPS in North America.

Introducing WPS Express

Today, World Programming LTD announced the availability of WPS Express. WPS Express is a product for those interested in learning the Language of SAS. WPS Express comes with all the database drivers and other modules of the Standard desktop version of WPS.

What separates WPS Express from the Standard Edition desktop experience is the number of records that can be processed. Currently, WPS Express processes 100 records.

What WPS Express is meant be is a free product that allows you to learn the Language of SAS. As such, 100 records are probably sufficient to learn to program in the language, connect to many different databases, and run R.

One other caveat is that WPS Express is licensed to an individual and not to any organization. Again, it’s worth noting that this is a product to learn how to write code in the Language of SAS. Also, WPS Express is licensed on an annual basis so you will have to renew your license every year.

You can find WPS Express by going to the World Programming website and taking a look at: https://www.worldprogramming.com/try-or-buy/wps/editions/express

If you are interested in a more formal WPS training, especially on how to use the WPS Workbench, I recommend that you reach out to Art Tabachneck. Art has a placement company called Analyst Finder that helps companies and recruiters find analytical talent. Art also has a one-day training seminar and he can do the training online. I’ve seen the syllabus and slide deck and think it’s quite complete with regards to getting a thorough understanding of the power of WPS. Interested parties can reach out to Art at: art297@rogers.com

WPS Express, due to its 100 record limitations is not a practical product to use for evaluating whether to swap out SAS for WPS. Every organization would need the standard edition to process an unlimited number of records so that they could compare output of the products.

MineQuest Business Analytics is able and willing to help you and your organization with your evaluation of WPS. We can arrange for a free 30-day evaluation of the workstation products, both OS X and Windows as well as on all supported server platforms.

Interested in a quote or a free 30-day evaluation of the standard edition of WPS? If your organization is located in North America, simply fill out the Evaluation Request from our website.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is an authorized reseller of WPS in North America.

PROC REG WPS v3.2–New Graphics and PMML

So, those of you who have downloaded WPS v3.2, there are a number of new features. I want to show two new features using PROC REG. WPS now has the ability to create plots for PROC REG. Quite handy indeed!

Also, in Proc REG for v3.2, we see experimental support for PMML (Predictive Model Markup Language).

Here is some sample code that demonstrates the plots.

*–> Data is census population data from 1790 to 2010;
data census;
   input year pop @@;
   pop2 = Round(Pop/1000000,.1);
   popsq=pop2*pop2;
   lpop=lag(pop2);
cards;
1790 3929214 1800 5308483 1810 7239881 1820 9638453 1830 12860702 1840 17063353
1850 23191876 1860 31443321 1870 38558371 1880 50189209 1890 62979766 1900 76212168
1910 92228496 1920 106021537 1930 123202624 1940 142164569 1950 161325798
1960 189323175 1970 213302031 1980 236542199 1990 258709873 2000 291421906 2010 308745538
;;;;
run;

*–> PROC REG with the PMML attribute to output the model in PMML form.;

filename outfile ‘c:\temp\regpmml.txt’;
Proc Reg data=census outpmml=outfile pmmlver=”4_2″ plots;
model pop2 = year lpop;
Title “US Census Population – PROC REG”;
run;

 

US Census Population – PROC REG
The REG Procedure
Model: MODEL1
Dependent variable: pop2

Number of Observations Read 23
Number of Observations Used 22
Number of Observations with Missing Values 1

Analysis of Variance
Source DF Sum of Squares Mean Square F Value Pr > F
Model 2 206768 103384 9307.59 <.0001
Error 19 211.04266 11.10751    
Corrected Total 21 206979      

Root MSE 3.332793 R-Square 0.998980
Dependent Mean 111.704545 Adj R-Sq 0.998873
Coeff Var 2.983579    

Parameter Estimates
Variable DF Parameter Estimate Standard Error t Value Pr > |t|
Intercept 1 -299.75395 71.30929 -4.20 0.0005
year 1 0.16607 0.03878 4.28 0.0004
lpop 1 0.97176 0.02754 35.28 <.0001

ResidualPlot2

DiagnosticsPanel3

 

The PMML output generated is:

<?xml version=”1.0″ encoding=”utf-8″ ?>
<PMML version=”4.2″ xmlns=”
http://www.dmg.org/PMML-4_2″>
    <Header copyright=”World Programming Limited 2002-2015″>
        <Application name=”World Programming System (WPS)” version=”3.2.0″/>
    </Header>
    <DataDictionary numbeOfFields=”5″>
        <DataField name=”year” optype=”continuous” dataType=”double”/>
        <DataField name=”pop” optype=”continuous” dataType=”double”/>
        <DataField name=”pop2″ optype=”continuous” dataType=”double”/>
        <DataField name=”popsq” optype=”continuous” dataType=”double”/>
        <DataField name=”lpop” optype=”continuous” dataType=”double”/>
    </DataDictionary>
    <RegressionModel functionName=”regression” targetFieldName=”pop2″>
        <MiningSchema>
            <MiningField name=”year”/>
            <MiningField name=”lpop”/>
            <MiningField name=”pop2″ usageType=”target”/>
        </MiningSchema>
        <RegressionTable intercept=”-299.753951850233″>
            <NumericPredictor name=”year” coefficient=”0.166074316077245″/>
            <NumericPredictor name=”lpop” coefficient=”0.971762137737628″/>
        </RegressionTable>
    </RegressionModel>
</PMML>

Interested in a free 30 day evaluation of WPS? If your organization is located in North America, simply fill out the Evaluation Request from our website.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

Why WPS needs to be part of your Corporate BI Stack

Recently, I’ve been talking to a few customers about why they decided to bring WPS into the company. After all, these firms have lots of money and talent. They can pretty much license any software they feel they need as long as it gets the job done. Of course, there are constraints due to pricing and training, but for the most part, these companies have free reign.

Below are the four major topics that everyone has touched on. Remember, these are large firms that are stalwarts in the analytics field offering products and services that are dependent on their IT and business staffs to generate revenue.

Innovation

WPS is rapidly growing and introducing additional procedures to the product. The customers that I have spoken with have all stated that WPS contains all the PROCS that they need to access, analyze and report on the data. Remember, we are talking about Fortune 500 companies here so that says a lot about how fleshed out the product is at this point.

Efficiency in Licensing

If you are a large corporation, it is likely that you have offices overseas. Licensing WPS is a dream compared to our competitors. There’s no multiple sales teams to have to work with and no differentiated licensing.

Also mentioned was that ALL the library modules are included in the price. There is no longer any confusion on what is part of the product.

Cost Reductions

It’s well known that WPS is a high value low cost alternative to the SAS System. Whether considering expanding the footprint with workstations or servers, WPS is an extremely competitive proposition. This is especially true on the server side. Since WPS is priced so competitively, even small workgroups can easily afford a server for their department.

Sole Source provider

One of the most interesting responses I received, and one that caught my attention (especially from a risk mitigation perspective) was that they didn’t want to find themselves beholden to a single source supplier of the language. I asked why they were concerned about that issue specifically. The three major points brought up are:

  1. They lacked flexibility in how they could use the product to deliver data, analytics and reports to their customers.
  2. They could take advantage of new concepts and features as they are introduced across two platforms.
  3. Fear that they would be held hostage in pricing negotiations. With a multiple providers, they felt they had leverage if they chose to not expand their footprint with the sole source provider.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

Post Installation Steps for WPS Workstations

We recently wrote a short technical document on a set of post installation steps that MineQuest Business Analytics recommends after you install WPS on your workstation. We are often asked what needs to be done after WPS is installed to get the greatest performance out of WPS without too much hassle.

The document walks you through modifying your WPS configuration file, moving your work folder to another drive, why you want to install R (for using PROC R of course!), creating an autoexec.sas file, turning out write caching and a few other pointers. You don’t need to to all of the suggestions, after all they are just suggestions, but they are useful modifications that will enable you to get more out of WPS on your workstation.

You can find the document “Post Installation Steps for WPS Workstations” in the Papers Section of the MineQuest website.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

A Summer Project

One of my summer projects is building and performance tuning a relatively inexpensive analytics server. Many of the parts that are being used have been scavenged from another server or two that have been retired. One thing I want to do this summer is report on what I have discovered in performance tuning a modest server.

The server consist of a six core AMD processor and 16GB of RAM to start out with. I would like to experiment with different combinations of RAM, hard drives, hard disk controller cards and perhaps an SSD or two. The OS will be Linux, Ubuntu 14.04 specifically.

My baseline build has just two work drives in RAID-0 and use the SATA 3 ports on the motherboard. I will use the Workstation Performance Assessment Program that I wrote about back in 2012. I’ve slightly modified that program so that it doesn’t spew output in the listing with the exception of the actual performance benchmark.

One thing I have already learned is that you need to make sure that you have the Write Cache enabled. In Ubuntu, you would do this by going to Disks and clicking on the options button at the top right of the dialog box and then selecting Drive Settings. Simply select the Write Cache and click on Enable Write Cache. You will need to do this for each disk in the raid array.

ubuntu_disk_cache

When I enable the write cache, my timings for the PROCs and data steps that took place on data sets that existed on the work array dropped 35%. That’s a big improvement!

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

Who says only big companies can afford to utilize Business Intelligence?

One of the reasons I got into reselling WPS was the fact (and it’s still a fact) that it’s very expensive for a new firm or startup to utilize SAS products. Actually, it’s prohibitively expensive. A commercial startup business is looking at $8700 for a desktop license that provides access to BASE, GRAPH and STAT. That $8700 is for the first year and it doesn’t include access to a database, Open Source R or reading and writing to desktop files like Excel and Access. Add those necessities in the price and you are looking at more than $15,000 for the first year and more than $4200 for renewal.

With WPS our pricing is different. We kind of joke that whatever SAS does pricing wise, we do just the opposite. We don’t have a high barrier in terms of cost to start using our products. Actually, we encourage you to use our products! Currently, we charge $1,311 for a single desktop license. That’s the cost for the first year and it includes all the database engines that you would want.

We don’t have a high barrier to using the language. If you are already familiar with the language of SAS, then you are ready to go with WPS.

We don’t have a high barrier when it comes to accessing your SAS data sets. We can read and write SAS data sets just fine.

But enough about barriers, let’s talk about servers.

The pricing differential is even greater when you start looking at servers. You can license a small WPS server for less than $5,700. That’s a two LCPU server and it includes all the bells and whistles that our desktop licenses include as well. Meaning it includes all the database access engines. The nice thing about our licensing is that we don’t have client license fees. Client license fees are fees that you pay to be able to access the server you just bought! It’s a stupid fee and we try not to do stupid things!

Another way we differ from our competitor is that we don’t have Data Service Provider fees. Let’s face it, many small companies (and large companies too) provide data and reports to their customers and vendors for further analysis and research. As a DSP, you will pay significantly more for your SAS license than what is listed. Expect to pay at least 30% more and often times, a lot more.

If you’re a startup, the message is clear. You probably don’t have a lot of money to toss around and cash flow is an issue. MineQuest has partnered with Balboa Capital to help company’s manage their licensing costs. By working with Balboa Capital, you can manage your license costs by paying a monthly amount of money towards your license. You will have to take out a two year WPS license to qualify for the program, but it’s an easy and efficient way to manage your resources.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

On ErrorAbend

One issue I always had with the SAS system as a developer was when I had a job that ran in batch that had an error. The SAS System would set the number of observations to zero and go into syntax checking mode for the remainder of the program.

This had some virtues but more often than not, the error was thrown because I had misspelled a variable name in a MEANS statement or FREQ statement that was used for checking my output. This would cause SAS to go into the syntax checking mode and all the rest of my program would not execute even though it was proper.

WPS, when running in batch doesn’t do this but if you want the same effect for your batch jobs, it’s easy enough to implement. Consider the following macro – called %ErrorAbend. %ErrorAbend simply checks that the program is not running in the FOREground and checks the value of the &syserr variable after every PROC or data step and if it returns a value of 3, then issues a note and sets the number of observations to zero.

%macro onerrorabend;
  %if %eval(&syserr eq 3) and &sysenv NE FORE %then %do;
     options obs=0;
     %put NOTE: WPS has been set with OPTION OBS=0 and will continue to check statements.
  %end;
%mend;

Below is a sample program that when run in batch, puts the system into syntax checking mode and basically stops the execution of any downstream statements.

data a b;
do ii=1 to 2000;
  x=ranuni(0)* 10;
  y=Round(ranuni(0),.01)* 100;
  z=round(ranuni(0),.01)* 10000;
  

  a=ranuni(0)* 10;
  b=Round(ranuni(0),.01)* 100;
  c=round(ranuni(0),.01)* 10000;
  
  e=ranuni(0)* 10;
  f=Round(ranuni(0),.01)* 100;
  g=round(ranuni(0),.01)* 10000;
  
  i=ranuni(0)* 10;
  j=Round(ranuni(0),.01)* 100;
  k=round(ranuni(0),.01)* 10000;

  output;
end;
run;

proc freq data=a;
tables ik;
run;

%onerrorabend;


proc means data=b;
run;

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

Thoughts on Mapping and Geocoding in WPS

I’ve been working on a framework for a set of new macros to be included in the Bridge to R that I think will be very useful for many WPS users. Coming from a background in Demography, I’ve always been partial to maps and charts. There’s a plethora of open source products out there as well as API’s from Google, Bing, etc… that allow a user to create some pretty darn nice maps.

I recently became aware of a new R library by Hadley Wickham and David Kahle called ggmap. Professor Wickham has created some phenomenal software for the open source R system. Hadley created ggplot2 that is truly the standard for graphics in the R world. He has also written a book on ggplot2 that is well worth purchasing and can be found on Amazon by clicking here.

For most of us in the WPS world, we are rather limited to the native graphics available in the product. That was mostly overcome with the Bridge to R that we created a few years ago. So there are two things that I see as important at this juncture that needs to be addressed and that is geocoding and mapping.

First I want to discuss geocoding. Geocoding has always been this strange process that is 10x more convoluted than it really needs to be. For most of us, we want to provide an address from out data set and get back latitude and longitude for that address. For a much smaller group of users, they want to enhance their data with latitude and longitude as well as zip code, etc… Either way, using external services from commercial companies to do such a thing is often expensive. This is especially true or smaller data sets where there is a standard fee plus so much per name.

The second aspect worth discussing is the availability of mapping software and the associated cost. Some of these programs are expensive to say the least. Unless you intend to make a career out of creating a map (and I am not) then other alternatives need to be looked at to keep costs down.

So, long story short, it’s worth investigating interfacing into open source and cost free solutions for map making using WPS. I have briefly looked at Google, Bing and OpenStreetMaps. There are pros and cons to each one of them but I want easy and nice looking and that’s the driver behind my development.

Professor’s Wickham and Kahle have done a lot in this area trying address the short comings of R for mapping. I could not approach their creative genius and determination in creating ggmap, but I can create an interface that makes using ggmap easier for WPS users. So, my summer adventure is to create a clean interface using WPS and the Bridge to R so that WPS users will have some extraordinary maps that they can create using ggmap.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

Building a BI Consulting Company

Over the last few weeks, I’ve been engaged in a series of conversations regarding consulting and necessary hardware and software to run a successful consulting house. In the last year we’ve seen so many references to “big data” and many of us in the consulting field just shrug our shoulders and smirk because we’ve understood that “big data” is a lot of hype for most of us. If you want to be precise about it, the term (and what we should be concerned with) is actually “big analytics.”

As a BI consultant or consulting house, you don’t have to replicate your client’s systems or data warehouse to consult on “big analytics.” As a matter of fact, some of the most successful BI consulting going on today are with companies that have outsourced a portion of their analytics to a third party. For example, loyalty cards are a driving force in retail and many organizations have outsourced this to third party analytics firms. We also see a growing opportunity in health care for fraud detection and pricing of procedures and prescriptions.

So the question comes down to what is your consulting focus? Is it providing knowledge and programming expertise to a company and perform the consulting remotely (or even onsite) or is it more encompassing and moving in the direction where you have the client’s data on your systems and perform a daily/weekly/monthly service?

I’m inclined to argue that the more financially successful firms that are offering consulting are the ones that are taking client’s data and providing the analytics services away from the client. The rates and fees are higher than when you are on site and there is limited travel time and expense to deal with.

I often see quotes for servers that they have been solicited from Dell, IBM or HP when they are sizing hardware to run WPS. I am amazed at how reasonably an organization can purchase or lease hardware that is immensely powerful for processing data sets when running WPS. I’ve seen 16 and 32 core servers that can run dozens of WPS jobs simultaneously priced between $40K and $60K.

I’m convinced that if you have a good services offering (and a decent sales staff who can find you clients) that this is the golden age in analytics for smaller firms and firms considering jumping into this space. My observations with advertising agencies and others who offer such services bears out that the supply of talent is low and the demand is high.

Of course, hardware cost is just one factor in this line of business so in a future column we will talk about how software cost and licensing can constrain you to the point where you can’t provide any services to third parties or it can set you free and allow you to make significantly more money. Software licensing is a major component to running a profitable BI/Analytics service.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.