Category Archives: WPS V3

test, Test and TEST

I’m always amazed and somewhat peeved about how much error checking one has to have in their SAS language program. Even for the simplest things. So today’s mantra is test, Test, and TEST!

I was writing some code the other day to copy a WPS data set from my PC to a server share. The basic code is really quite simple, only three lines using PROC COPY. But what if the user has misidentified the source libname or the destination libname? Do you just let it blow up and hope the user looks at the logs? And then you have the data sets to be copied if you are using the SELECT statement. Do you check if the data set already exist and if so, just overwrite the file?

Although it was trivial, albeit time consuming to write the code to check for these conditions, it is well worth it. I purposely decided not to automatically overwrite an existing data set on the server. And that is good for two reasons. First, I want the user to be forced to make the decision to overwrite the data set by use of a PROC DATASETS or PROC DELETE before the copy takes place. That makes it their responsibility to delete the data set.

Secondly, I found out that writing the data set with the same name can sometimes create problems under Windows when the server folder is shared. I have had some experiences where Windows locks the file on the server and the copy never takes place. The copy procedure just hangs with a .lck extension on the file. So something is going on where it’s just not reliable.

One interesting thing to note, I don’t seem to have the problem with a lock on Linux. The copy takes place without issue every single time.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

 

WPS 3.01 Now GA

Just a note to let you know that World Programming has released version 3.01. If you are running any older versions of WPS, I would suggest that you install the new release because of the bug fixes and a few new procedures.

This current version contains PROC REPORT, PROC DBLOAD and PROC RELEASE (for mainframe users). There are some significant performance improvements for PROC LOGISTIC and FASTCLUS.

Also, there are three new WPS data access engines. The new databases supported include Kognitio, PostgreSQL and Vertica. There’s also support for Bulk Loading for DB2 and Greenplum.

The number of database access engines that are included in WPS is pretty amazing. SAS Institute charges dearly for database access engines and I believe that’s a problem for shops that are looking to roll out Business Intelligence across the enterprise.

With the introduction of PROC REPORT, most Pharma organizations and CRO’s can move their SAS Language code base easily over to WPS. If I was running a CRO or doing the analytics work that required FDA approval, I would be taking a serious look at WPS in lieu of of the SAS System. It’s so cost effective, it could mean the difference in profitability for a small or mid-size CRO.

Finally, there’s still time to get into the action to win a Google Nexus 7 Tablet. If you register to take out a WPS evaluation before September 30th, 2012, you will automatically be registered in the drawing for the tablet. Certain conditions apply so read the previous blog post for all the details.

About the author: Phil Rack is President of MineQuest, LLC and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

Bridge to R Demo Reel

Finally making some headway on a short demo reel for the Bridge to R. We decided against narrating anything and just included some pop-ups and pointers to show what is happening in the video.

The video shows a few things that I believe are important. First, that the Bridge to R handles R statements and gets the text listing and log back into the Eclipse Workbench using WPS. Secondly, in addition to handling the text side of things, the Bridge can handle graphs, plots and maps. Third, and quite important actually is that the Bridge to R does a lot of work in the background to catch errors coming back from R and reporting them in the log.

The video can be viewed at:

http://www.minequest.com/downloads/Bridge2Rvids/bridgev3hd/Bridge2rV3HD.mp4

or tinyurl:

http://tinyurl.com/89rsrcr

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.

Adventures in Porting

We’ve been busy porting the new version of the Bridge 2 R over to both the Mac and Linux platforms from Windows. The Windows release of the Bridge always allowed for the use of the Bridge from within the Eclipse Workbench. WPS didn’t have the Workbench as the GUI on the Max or Linux until version 3 which is the latest release. So, here’s what I found porting a large program that has to talk to different operating platforms (i.e. calls to the OS) for such things as delete files, move files, copy files, read directories, etc… and still interface with R.

The mundane part of porting was converting a lot of “\” to “/” throughout the code. In retrospect, we could have done a better job writing the Bridge in the first place to accommodate these conventions, but we didn’t have the intention of porting code back then either.

Here’s a couple of the gotcha’s that we experienced. When you read a directory on Linux or OS X, the structure is slightly different between the two and you have to accommodate that issue. The other BIG issue is that the pathnames are much longer on Linux and OS X when reading and writing to the WPS work folders. We ended up resizing our string variables to handle that specific difference.

The above might sound trivial but one think we discovered is that when you restart your server on OS X and Linux, the new work folder is contained inside the previous folder. For example, your original folder, let’s call it work1 is now hosting work2, your new folder. Now the path name is /work1/work2. But in reality, the names of the work folders are not work1 or work2 but long strings that can be hundreds of characters long. If you have a user who likes to restart their WPS Server, you can eat up a lot of string space quickly.

Since we store a lot of metadata for the Bridge 2 R inside the work folder, R has to be able to cope with very long filenames and I’m not convinced that it really copes all that well. Speaking of file names, here’s another anomaly between Windows and Linux/OS X systems. if you have a filename such as “myfile.txt ” (note the blank space at the end of .txt) Windows handles that just fine. Windows will interpret that as meaning you wanted “myfile.txt” However, if you write such a file or try to read a file with that name under Linux or OS X, then those two names are distinctly different. On Ubuntu or Fedora, that name shows up as “myfile.txt\” when you list the files from the terminal.

It took us about three days to port the Linux version of the Bridge over from Windows. Much of that time was spent dealing with the issues in the previous paragraph. We then took the ported Linux code and tested it on OS X. It took about 20 minutes to modify the section dealing with the difference in reading directories between the two platforms, and we then had a new version of the Bridge to R running on OS X.

In retrospect, porting the code over to Unix/Linux systems was worth the effort. It took a few days for us to do the porting and much of that was due to being naive about the new ported destinations. I will talk soon about the new enhancements (and a programming change users will have to make) in the Bridge to R in my next post.

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.

Software We Use and Love on a Daily Basis

WPS – World Programming System from World Programming LTD. WPS is a SAS Language compatible software system that implements many components of the SAS language on many platforms. WPS starts at just over $1,206 on a desktop. Check out the MineQuest website for more information.

R – R Project for Statistical Computing. The 64-bit port is definitely the way to fly if you are using R. R has amazing graphics and Hadley Wickham’s ggplot2 is worth the effort of learning R.

Microsoft Office 2007 and 2010 – Microsoft Office is the standard for writing documentation, use of spreadsheets and email on both Windows and OS X platforms. I probably use Excel, Word and Outlook more than any other office productivity too.

Skype – used a lot for both domestic and foreign phone calls and text messaging. Skype is easy to use and can provide your company with the ability to do business overseas at reasonable costs. Also, for a mere $5 a month, you can have group video calling as well as calls to any phone in the US and Canada.

MeetingBurner – MeetingBurner is relatively new and since we have such few users on a web conference (typically five or six) this makes perfect economic sense. We’ve not used it much but it is fast and is free for organizations that will have fewer than 15 attendees in a meeting. One great plus is that it integrates Skype for audio.

Oracles Virtual Box – We use this to reduce our exposure to running multiple physical servers. Oracle’s VirtualBox saves a lot of money for testing software because it can dramatically reduce your power consumption by not having individual servers.

Nuance Paper Port – It’s amazing how much paper we scan here. Bills, invoices, checks and all kinds of business related materials. With both a Canon MFC printer and a Brother MFC printer, we just throw documents into the hopper and scan away. You can learn about Paper Port by going to their website here.

UltraEdit – UltraEdit from IDM is the standard for programming editors. We use it on the Linux and Windows desktops. Soon we will be using it on the Mac desktop to port the Bridge to R over to OS X.

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.

Printing in WPS V3

The new version of WPS (version 3) is out and available and everyone has been busy sending out new license keys for this release. In our previous blog post, I presented some of the highlights of this release but want to talk about enhanced printing ability in V3.

The rendering of each page is a bit different than what you are previously used to in V2.5.x. You can see the difference in how the page is rendered in the screen shot below.

Notice the page breaks between the pages. One delightful aspect of this enhancement is that when you change line size settings in your programs, the page that is rendered reflects this. This is really great! The ability to print the reports from the Eclipse Workbench are also much improved.

One thing to be mindful of is that the options Orientation=landscape || Portrait is not supported. This isn’t that much of a big deal as you might at first expect. I simply set my LS=132 and PS = 45 and on my  printer properties for the print driver (a Canon MF-4150), I have a form called WPSDuplex_LS for duplex landscape mode. I can set the font for this print job automatically when I use this form. The font I use is Courier 9pt. This works like a charm for me.

Evaluations

MineQuest is offering free 30 day evaluations of WPS V3. You can contact us at (614) 457-3714 to request an evaluation or by email. If you prefer to request your evaluation by email, send a request to info@minequest.com.

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.

Introducing WPS Version 3

It’s a new year and version 3 of WPS is out and available. There’s a lot of new items in this release and I think it has been well worth the wait. I personally feel that this is the block buster release that will turn heads in the corporate environment due to so many added features. With version 3, World Programming has made the software more friendly and easier to use, added new procedure support both statistically and in the core language, and added support for more language elements.

I’ve been beta testing WPS 3 for a few months now and so I’ve had a bit of time to try the software out and evaluate it extensively. Below is a list of items and features that really stood out to me in WPS V3. It’s far from being exhaustive but these are probably the most pertinent for many WPS users and SAS customers who are considering WPS as an alternative to SAS.

Workbench Support on Additional Platforms

The Eclipse workbench is available on Unix, Linux, Windows, AIX, Solaris and OS X platforms. The GUI that you’ve been using on Windows is now available on all the platforms except for z/OS. Amazingly, the Eclipse Workbench is identical (and I mean identical) across the platforms. Now when companies and organizations add servers to their data center and the users will have the same identical experience (read no learning curve) on all the platforms.

Apple OS X Support

WPS has been available on Apple’s OS X for a while, but now it has the Eclipse Workbench as the GUI. The WPS implementation on OS X is native, meaning you don’t have to run Parallels or Boot Camp to run WPS on your Mac. WPS is a true 64-bit implementation on OS X. The technical specifications for the Mac OS X implementation is that it needs to be an x86 chip and you will need version 10.5 (Leopard) or higher. WPS V3 works fine on Lion as you can see in the screen shot below.

Click here to view larger image.

About a month or two ago I bought a Mac Mini just to test WPS and to port the Bridge to R over to it. The Mac implementation is a bit prettier than on Windows or Linux in my opinion. It may be the font, but it does look nice as you can see in the image above. In testing WPS on OS X, I was amazed how easy it was to transfer my knowledge of the WPS implementation on Windows over to OS X.

There’s a lot of folks who love their Macs and WPS running natively gives the Mac faithful a SAS language compatible system that takes full advantage of their hardware. Mac users give up nothing over the Windows or Linux versions of WPS and desktop pricing for the Mac is the same as for the other desktop platforms. I’m excited for the university students where there is a preponderance of Macs and I can see the take up of WPS because of the hardware and operating system support.

Linux Support

As mentioned earlier, the workbench is also available on Linux. I have it running on both Ubuntu 10.4 LTS and Fedora 13. As you can see in the screen shots below, WPS looks the same on each of the platforms. One thing to note about the Linux version, it is only available as a server license. But for shops that have a Linux platform, WPS executes exactly like the Windows version in that it provides the end users the same experience as it pertains to the GUI.

Click here to view larger image.

Many shops run Red Hat as their preferred version of Linux and WPS does run just dandy on that platform. I find that many of the larger customers we have do use Red Hat, but I suspect that the smaller businesses use Ubuntu or Fedora instead.

AIX and Solaris

The workbench is also available on these platforms. I don’t have a license or even a machine to run these two implementations on, but I would suspect that the experience is identical to the Apple and Linux versions discussed above.

WPS Link

This is an exciting feature that is being introduced in WPS 3.0. WPS Link is a Client/Server architecture that allows you to remotely execute code on Linux or Unix servers. So for example, you can be running WPS on your Windows or Mac OS X desktop and submit your WPS code over to a WPS Linux server. The log and listing files come back from the remote machine to your desktop. WPS Link is different from SAS/Connect in that this is just a client to execute your programs on the remote box. You can’t upload and download files from the desktop to the server. But, if a lot of your processing takes place on the server and that’s where your data sets reside, this is a nifty solution for processing data on a remote server.

WPS Link is easy to setup from you WPS Desktop. Simply go to the Server Connection tab and select Local. Right mouse click on Local and you get a drop down menu. Select New || Server Connection || Remote SSH.

From that point you get a dialog box where you simply fill out the hostname (or IP Address) the Connection Name (how about “WPS Remote Server”) your user name and the Launch Command. The Launch command was the most difficult part for me. It’s simply the location of the folder where WPSLinks module is located. On my server it is:

/home/minequest/wps-3.0.0.0.19782-private/bin/wpslinks

Make sure you have SSH running on your Linux server and you will be able to remotely start WPS on the server from your desktop!

Submitting code from the desktop to the WPS Server is relatively easy. Simply go to the Submit Icon and you will find a downward facing arrow next to it. You can select whether you want to run the code Locally or on the Server. One of the cool innovations with WPS Link is that you can have multiple servers registered and choose which WPS server you would like to run your code on. Note the options in the drop down menu shown below.

WPS Link comes standard as part of your WPS purchase. So if you license WPS on the desktop, WPS Link will be available to you as part of your desktop license. Also be aware that in this first release, WPS Link requires WPS running on an AIX, Solaris, Linux Servers and Linux on Mainframe System/z. I suspect that you will see Windows Servers being supported in a later release. But for our customers running large volumes of data on Linux or Unix, this feature will be very welcome.

One other note of importance on WPS Link. You can connect from a desktop to a server and server-to-server. You cannot link from desktop-to- desktop.

Procedure Support

In version 3, World Programming has added support for the following statistical procedures.

  • T-Test
  • GLM
  • GLMMOD
  • STDIZE
  • PRINCOMP
  • DISTANCE
  • FACTOR

Database Engine Support

One of the things that SAS does that I really don’t like is the up-charge for database engines. With WPS, you get the database engines as part of the WPS license. You don’t buy these separately. With version 3 of WPS, there is now support for an XML Libname engine and Sybase databases.

If you’re a shop that has a few different databases from different vendors, it would be financially wise to take a look at WPS just for the cost reduction with the included access engines.

Other Enhancements

WPS has now implemented multi-threaded support for PROC MEANS and PROC SUMMARY. It’s always fun to look at the execution time for CPU and Real Time when dealing with multi-threaded procedures. From some of the WPS documentation you can see that the multi-threaded support that is used in MEANS and SUMMARY has made it into procedures that make use of this code such as PROC TTEST and I assume PROC CORR.

Also, for those of you who were unaware, PROC SORT is also multi-threaded and is very fast. If you do a fair amount of sorting in your environment, it would be beneficial to be running on a 64-bit platform and load up with memory. The more memory you have available to WPS, the more data it can store in RAM when sorting and this results in faster sort times.

There is now an import and export wizard for importing files into WPS and writing out WPS data sets to text files. This is pretty big in my opinion. I could never remember the PROC IMPORT statements even though I use them often. Probably everyone uses a code template for importing and exporting but this just makes life easier.

The data set viewer has been enhanced over the previous release. You can view the variables labels instead of just the variable names. You can also show and hide variable columns.

Also supported for the first time is the SYSTASK and WAITFOR commands. I’ve been waiting for these two language statements to be supported for a while. As part of the Bridge to R on Windows there is a module called MPExec. MPExec allows you to run multiple programs in parallel. I’ve had to resort to using other methods in lieu of SYSTASK and WAITFOR to get this to work. Now that these two commands are available, we will update MPExec to use these features instead, thus MPExec will become portable to other platforms.

Performance

Performance has improved in each release of WPS and WPS v3 is no exception. With multi-threaded support for more procedures and better data handling, WPS is a very viable candidate to replace SAS at many organizations. MineQuest has many customers who run large data sets (millions, ten millions and hundred millions) of records through the WPS System and performance is more than adequate. These companies are able to save loads of money over the competing software system.

Expandability

With affordable pricing, customers can use WPS to perform reporting and analytics in areas and for departments that just wasn’t justifiable before. Using WPS in lieu of SAS allows many organizations to expand the analytics platform and run more data through the system. This is because such advantageous pricing allows them to purchase an additional server at great savings over our competitors.

Data Service Providers

We love Data Service Providers! Seriously, we find that DSP’s are some of our best customers. With WPS V3, licensing terms stay pretty much the same. World Programming LTD does not have DSP fees and you are free to use WPS to service your customers by providing them with data sets, reports and analytics. For any organization that is paying DSP fees, we can dramatically reduce your license fees. Your customer is your customer.

Caveats

Before you download the latest release of WPS, there is one caveat to keep in mind for upgrading. If you are using batch command files to run your jobs, you need to modify the name of the WPS executable in your .bat or .cmd file. The program name you want to use is WPS.EXE (as opposed to the former name which was WPSI.EXE).

Also in WPS V3 the WPD dataset has changed and is faster and more robust. You can still read WPD V2 datasets but V3 datasets are the default in this release.

Finally, you will need a new license key to be able to install V3. Don’t expect to simply download it and run it using the V2 key as you have been able to do previously. If you are a MineQuest customer, contact us if you want to upgrade to V3 and we will facilitate getting you a license key.

Evaluations

MineQuest is offering free 30 day evaluations of WPS V3. You can contact us at (614) 457-3714 to request an evaluation or by email. If you prefer to request your evaluation by email, send a request to info@minequest.com.

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.