All posts by Phil Rack

Phil Rack, Owner of MineQuest Business Analytics, LLC. Grand Rapids, MI USA For more than twenty five years, I've worked as a SAS Consultant, specializing in the financial industry. My motives for this blog are to inform and educate other consultants as well as clients who use SAS or WPS and how they can more effectively use technology to further their business objectives.

Why Microsoft? Why?

As a business user and knowledge worker, I am extremely dependent upon my workstation and laptop. Like many of my readers, we spend a great deal of time developing a work-flow for analytics as well as business processes for running our business. When I am forced to accept updates and my systems go south, I must spend time fixing this. That’s why I get so frustrated with Microsoft.

The other thing that has really gotten my goat is that ads are now starting to appear in my OS. There are ads in the file explorer window! I want a clean, non-cluttered interface and OS. I don’t want to be hampered by an OS that makes me lose concentration of crap popping up to distract me.

So, I have decided to embark on an experiment. I’m going to setup a used Dell laptop to use as my primary machine running Linux. If after 60 days or so, I will decide if I make the permanent move away from MS for my full-time machine and only use the Windows workstation for testing code.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful TucsArizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Technical Document – Post Installation Steps for WPS Workstations

We just finished updating a document for WPS users, specifically those on the Windows Platforms entitled, “Configuring Your WPS Workstation after Installing WPS v3.3.” This document helps those who are evaluating the WPS product on Windows learn about and install some features that is specific to WPS.

The document is short, only 14 pages but touches on modifying the WPS.CFG file as well as installing R and Python to get the greatest amount of utility out of WPS. If you have installed WPS on a Windows Workstation and are looking to get additional utility out of your WPS software, this document is for you.

To download the file, click here or download at:

http://minequest.com/downloads/Post-Installation-Steps-for-WPS-Workstations.pdf (1008KB)

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Why WPS v3.3 is important

A few weeks ago, I posted a blog about the new release of WPS v3.3. Quite a few people and companies have been waiting for this release and I want to talk about why this is such an historic release. I also want to write about what this release means to data scientists.

First, this release is truly a blockbuster. With the Interop for R and Python modules, this is the first time as far as I am aware, where a software product allows a developer to easily use the Language of SAS, R and Python to implement a program, project or product – all in one development environment. As data science has evolved, R and Python have become more popular but BOTH lack the data management capabilities of WPS. With WPS, you not only get the language of SAS to process your data, but you also get a multitude of database engines to read\write and access data in the most popular databases.

The beauty of such integration is ease of development. If you have been a developer for as long as I have, you know the time demands of learning a new development environment. Now, with WPS v3.3 a developer can stay within a single development environment for all of their analytical development needs. The developer doesn’t have to learn an R IDE to write, test and execute R code. The same can be said for Python. The Python interface with WPS means having development control as well as execution control of python programs or similarly R programs.

Organizations want to take advantage of R and Python integration because it allows them to create and expand programs and projects. It will quickly become apparent for third-party developers, i.e. those who want to create vertical market applications that these two additional languages radically increase their tool sets going forward. For many, it also means running production jobs that are self-contained, meaning control and execution is controlled by WPS and not a bunch of separate tasks or processes having to be handled individually.

I have been fortunate to have had access to Alpha and Beta builds of WPS for v3.3 and was totally blown away by the improvements and additions that I saw. For example:

  • The data step is faster.
  • Inclusion of the Python programming language.
  • Implementation of Proc IML.
  • Faster data engine access for many databases including multi-threaded loading.
  • More complete graphics output in statistical procedures.
  • PDF support

But what makes this one of the most compelling releases is the integration of R and Python. With Python specifically, one now has access to executing Machine Learning code from WPS. I’m sure many of you who read this blog are involved in credit scoring, fraud detection, anti-money laundering, market basket analysis, loyalty programs and other real-time analytics. With Python and OpenGL and the CUDA libraries, one can now perform incredibly high speed processing of data on your desktop/server GPU.

When looking at the breadth of the WPS offering, it’s amazing how much is included for the data scientist who needs to work in multiple languages supporting data analytics. With all the database engines (including Hadoop), WPS Graphics, IML, R, Python, WPS Statistics, and WPS Time Series, I almost faint thinking what the cost would be for something similar from our competitor SAS Institute.

The other aspect of WPS v3.3 that is so enticing is the licensing. There are two components to this that deserve discussion. The first, is that Data Service Providers (DSP’s) can make use of this software to develop and provide statistical and mathematical models for third parties. Let’s face it, many if not most organizations lack the expertise to develop these models for companies.

The second component is the creation of Vertical Market Applications using the WPS software. The absolute reasonable cost of the software is a driving factor in being able to create and resell your VMA at a price that small, mid-size and large companies can all afford. Using WPS as the basis for your VMA is advantageous because you’re not beholden to some other organization requiring you to pay partnership fees for access to the software and marketing. I think you would be shocked to learn about how competitive and perhaps ruthless a software vendor can be when it comes to introducing and pricing a competing VMA.

If you want to learn more about the latest release of WPS v3.3, especially as it pertains to workstations, read the previous blog post and download the latest brochure for v3.3. You can request an evaluation of WPS v3.3 by contacting info@minequest.com or filling out our evaluation or quote request form here.

I hope everyone had a Merry Christmas and wishing everyone a Happy New Year.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Product Overview WPS for Workstations v3.3

We just updated our Product Overview document entitled WPS for Workstations v3.3. This document explains the features and breadth of the WPS Workstation product for version 3.3.

The document contains a list of all the database engines and procedures that are included in the workstation product. For organizations considering WPS, this is a good place to start to understand the WPS offerings on OS X and Windows.

You can access the document by clicking here (1.4mb pdf)

or at: http://minequest.com/downloads/WPS-for-Workstations-Marketing-Brochure.pdf

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

WPS v3.3 Now Available

On Thursday December 15th WPL introduced WPS v3.3. This new version is available for immediate download. With v3.3, WPS includes a slew of new Procedures that will be of great value to those who hold WPS licenses and those who are looking to convert over to WPS from SAS.

New Language Procedures

Matrix Language Support is now available with PROC IML. PROC IML is included as a standard procedure in WPS and is not an additional cost module. There is a 430 page programming guide in PDF format that is included in the installation folder detailing how to use the Matrix Programming Language.

Python Support is now included in WPS with PROC PYTHON. PROC Python allows a WPS user to create, edit and invoke python programs from within WPS. The implementation of PROC PYTHON is very similar to PROC R. PROC PYTHON is included in WPS and is not an additional cost module.

ODS Support

WPS now includes output to PDF as well as HTML and Text output destinations. Note that PDF support is available on all platforms except z/OS at this time.

New Statistical Procedures

PROC ACECLUSProvides two methods for approximating the within-cluster covariance structure for a clustering model under the assumption of equal multivariate Gaussian distributed clusters.

PROC CANCORR – Identifies and measures the associations among two sets of variables.

PROC GENMOD – its generalized linear models.

PROC LIFEREG – Fits parametric, accelerated failure time models in the presence of left-, right- and interval censored data.

PROC LIFETEST – Estimates non-parametric survival functions in the presence of censored data using Kaplan-Maier or actuarial methods.

PROC LOESS – Fits non-parametric regression surfaces to multi-dimensional input data. The smoothness of the non-parametric model can be controlled. Outliers in the input data are detected.

PROC MI – Imputes the values of missing values in an input dataset.

PROC MIXED – Fits a mixed linear model to input data.

PROC MODECLUS – Produces various cluster output statistics.

PROC PHREG – Fits the Cox proportional hazards model to survival data.

PROC PROBIT – Fits binary or ordinal response regression models, useful for dose-response type analysis. Various types of model are supported by the procedure. Parameter estimates are generated through the use of maximum likelihood estimation. Model fit statistics enable the quality of the generated model to be assessed.

PROC VARCOMP – Fits generalized linear models with random effects, where the associated covariance matrix is assumed to be diagonal.

Note that WPS Statistics is included in the cost of a WPS license and is not a module that needs to be licensed separately at an additional cost.

New Graphics Procedure

PROC GBARLINE – The GBARLINE Procedure has been added to WPS. This procedure allows you generate bar charts on which plot data has been overlaid on to the bar chart.

New Data Engine

XLSX Engine – This is a cross platform engine that provides read and write access to file in Microsoft Excel format. You can process Excel data on any platform you choose and are no longer limited to Windows platforms. The XLSX engine is included in WPS and is not an additional cost module.

Data Engine Enhancements

NETEZZAM -Is a replacement engine for the NETEZZA Engine. NETEZZAM provides for multi-threaded operation using a new architecture enabling significant performance increases. The NETEZZAM engine is included in WPS and is not an additional cost module.

ORACLEM – Is also a replacement for the ORACLE Engine of prior releases. ORACLEM is also multi-threaded bringing performance increases. The ORACLEM engine is included in WPS and is not an additional cost module.

Both the above engines provide for the ability to Bulk-Load data.

There are a number of additional language features and workbench features that are worth investigating as well. WPS v3.3 is a major release where the functionality and language and procedures have been augmented.

For a list of all the WPS Procedures and Database Engines that are currently supported in v3.3, you can download a two-page brochure from MineQuest. This brochure lists the database engines that are supported on the Linux, OS X and Windows platforms as well as language support and PROC Support.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

The Application Economy

I pretty much finished up my Christmas shopping two weeks early this year. Even the wrapping and delivery completed thanks to Amazon this year. I’ve never had my shopping done so early in December and I’m darn happy about that!

That gave me time to watch some TV this weekend and since much of College Football is over for the season I ended up cruising over to Bloomberg TV. I watched a program called “Hello World” on the Russian Tech scene and it was fascinating to learn about what was being created in Russia.

The sponsor of the show was CA (aka Computer Associates) and they had an interesting and entertaining commercial titled “The Front Porch” which is about the Application Economy. We as analytical developers rarely think about software as an application the same way as consumers do. Our customers are often different departments or divisions in the corporation we work at. We don’t work at creating an application product that meets the needs of tens-of-thousands of users, or even millions. We mostly develop products used for tens of people or if we are lucky, hundreds.

A lot of the reason for that is that many of us don’t see what we do as developing an application that is consumed by users outside of our organization. The cost of commercial software is often so high that it makes it cost prohibitive to invest the hundreds of hours needed to create the application. The other issue many run into is the availability of data that can meet the needs of the consumer and is not protected by agreements.

The market has responded with software such as Python and R. However, the problem with both is the amount of data that can be processed. We live in a Big Data world and expecting data to fit into available memory is often not practical. Many of us are also dependent on using the Language of SAS for processing and displaying of data.

Obviously, WPS is a better choice than SAS when it compares to pricing, especially on the desktop. If you create an application that requires, say, WPS on a workstation, it is much easier to make a sale (your application and a WPS license) when the first-year cost is one-tenth the cost of the SAS system.

In future articles, I want to touch on creating applications for resale using WPS. I want to talk about “applications” for such things as Smart Cities, Marketing, Credit Scoring and Fraud Analytics.

We truly live in an era where we as analysts and statistical developers can contribute our skills starting a business, providing a product and doing it all with minimal cash outlay. The internet is a money pipe into the home and business. Don’t let the opportunity pass you by.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Disruptive Analytics

I picked up Disruptive Analytics available on Amazon which is Thomas Dinsmore’s recent book a few days ago and thought I would leave my impressions. Note this is not a review! First, I really enjoyed the history of the analytics platforms. The second and third chapters were very informative (History and Open Source respectively) and I learned a few things!

Regarding Open Source, I agree that we will see Python supplant R as the “go to language” for analytics in the Open Source arena. It might take a few years but if my customers interests are indicative of this trend, it will happen.

Dinsmore does an admirable job in Chapter 4 on Hadoop. This chapter is fairly dense reading for me mainly because there are a lot of terms and definitions in this chapter. If you were ever looking for an overview of the Hadoop ecosystem, this is probably a good start.

The other chapter I really liked was Chapter 6. This chapter deals with streaming analytics and I believe we are just in the infancy of this revolution. Smart Cities will be a very visible platform for many people to see and benefit from streaming analytics.

I would like to see in a future edition a presentation of the role of the analytics workstation and flash memory in the analytics framework. Data Scientist who are developing algorithms and processing data are often using workstations in lieu of servers. Perhaps even a few pages on how nVidia is revolutionizing the analytics world with CUDA processing on high power workstations. I think I would enjoy that.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is an authorized reseller of WPS in North America.

Ubuntu 16.04 Released and Quick Test Drive

In the last week, Canonical has brought forth a new release of Ubuntu and it is pretty nice! Version 16.04 has a number of great features that should be of value to those who use Linux. One thing that Ubuntu has at this point is a vertical line of products. I can’t think of any other vendor who has an OS that runs on Phones, tablets, notebooks/workstations, servers and mainframes.

I decided to give it a try on one of my workstations running it in an Oracle Virtual Machine (Virtualbox to be specific) to see how WPS runs on this new release. Just to cut to the chase, it runs quite well. As a matter of fact, once I got the VM to use all of its allotted storage, WPS ran like a charm.

clip_image002

A couple of things that might be of interest to potential Ubuntu upgraders. First, Ubuntu 16.04 supports ZFS. That might be important to a few sites. The second is the support for LXD 2.0. From the Ubuntu website –

LXD 2.0

Ubuntu 16.04 LTS includes LXD, a new, lightweight, network-aware, container manager offering a VM-like experience built on top of Linux containers.

LXD comes pre-installed with all Ubuntu 16.04 server installations, including cloud images and can easily be installed on the Desktop version too. It can be used standalone through its simple command line client, through Juju to deploy your charms inside containers or with OpenStack for large scale deployments.

All the LXC components – LXC, LXCFS and LXD – are at version 2.0 in Ubuntu 16.04 LTS.

In addition to trying Ubuntu 16.04 in a VM, I have also tested it on a small server (6 LCPU with 32GB of RAM) running WPS. Although I have not benchmark tested this exhaustively, it does appear that using v16.04 with WPS 3.3.2 (which is the latest release) provides a modest performance increase. This is easily observed with multi-threaded Procedures such as Means and Summary.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is an authorized reseller of WPS in North America.

WPS v3.2.2 Released

 

Earlier in the Week, World Programming released an update to WPS. Version 3.2.2 is mainly a maintenance release with a number of fixes.  There are some improvements and the two that caught my eye are:

25591: WPS can now handle record lengths up to, or even greater than 32K when writing to SAS7BDAT files.

25596: WPS on Linux now supports Sybase IQ 16.0 client drivers.

There are number of other fixes that are probably more important than the two I chose above (especially if you are on MVS).

You can upgrade your installation by going to the WPS Website and logging into the download servers (User ID and Password required.) You can also read a list of all the changes by clicking on the change log file on the right hand side of screen.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is an authorized reseller of WPS in North America.

Introducing WPS Express

Today, World Programming LTD announced the availability of WPS Express. WPS Express is a product for those interested in learning the Language of SAS. WPS Express comes with all the database drivers and other modules of the Standard desktop version of WPS.

What separates WPS Express from the Standard Edition desktop experience is the number of records that can be processed. Currently, WPS Express processes 100 records.

What WPS Express is meant be is a free product that allows you to learn the Language of SAS. As such, 100 records are probably sufficient to learn to program in the language, connect to many different databases, and run R.

One other caveat is that WPS Express is licensed to an individual and not to any organization. Again, it’s worth noting that this is a product to learn how to write code in the Language of SAS. Also, WPS Express is licensed on an annual basis so you will have to renew your license every year.

You can find WPS Express by going to the World Programming website and taking a look at: https://www.worldprogramming.com/try-or-buy/wps/editions/express

If you are interested in a more formal WPS training, especially on how to use the WPS Workbench, I recommend that you reach out to Art Tabachneck. Art has a placement company called Analyst Finder that helps companies and recruiters find analytical talent. Art also has a one-day training seminar and he can do the training online. I’ve seen the syllabus and slide deck and think it’s quite complete with regards to getting a thorough understanding of the power of WPS. Interested parties can reach out to Art at: art297@rogers.com

WPS Express, due to its 100 record limitations is not a practical product to use for evaluating whether to swap out SAS for WPS. Every organization would need the standard edition to process an unlimited number of records so that they could compare output of the products.

MineQuest Business Analytics is able and willing to help you and your organization with your evaluation of WPS. We can arrange for a free 30-day evaluation of the workstation products, both OS X and Windows as well as on all supported server platforms.

Interested in a quote or a free 30-day evaluation of the standard edition of WPS? If your organization is located in North America, simply fill out the Evaluation Request from our website.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is an authorized reseller of WPS in North America.