All posts by Phil Rack

Phil Rack, Owner of MineQuest Business Analytics, LLC. Grand Rapids, MI USA For more than twenty five years, I've worked as a SAS Consultant, specializing in the financial industry. My motives for this blog are to inform and educate other consultants as well as clients who use SAS or WPS and how they can more effectively use technology to further their business objectives.

An Update on my Ubuntu Experiment

Just an update on my Ubuntu experiment that I wrote about in the last blog post.  I do have most everything installed that I want for the experiment. I’m going to use Ubuntu 16.04 as much as possible and only revert back to Windows when absolutely necessary.

So far, I’ve went back to my Windows workstation for QuickBooks to update some accounting information. I guess I could probably put QuickBooks into the cloud but at this point I’m going to hold off on that application until I get other things tested.

I have to admit this laptop is on the heavy side but it has a really decent battery. On a full-charge I get around 7 hours. My 17” I7 Toshiba laptop gets maybe three hours and that’s if I put it into Eco Mode. The Dell e6420 also has an NFC hotspot and a finger print reader. I’m not using either one of those and don’t think there’s a driver for those hardware elements either.

I do have WPS, R, Python and R Studio all working well. I’m using Thunderbird for my email application and LibreOffice (using it to write this post) for my writing and documentation chores. I’m looking for a OneDrive connector that works seamlessly but so far, I’m still searching.

Just for kicks, I did try using Microsoft Office Online. This is the web version of office. It seems that I tend to skip letters in words or Office online would randomly adds spaces between words in my text. I’m not sure why but it seems to be constantly saving whatever I’ve entered. Meaning not saving in say 5 minute intervals or something similar. Could be something with Firefox too.

One thing I need to mention is that the performance is good. I have run dozens of WPS programs and they all have executed in a reasonable amount of time. I live in a world where the processing I do is measured in data sets of 10 million or less for my development. Using WPS on this laptop is a good experience and processes these datasets quickly.

I did get two emails over the week asking about WPS on Ubuntu. WPS does run fine on Ubuntu and I have no complaints on that. One emailer asked about the price of WPS on Linux and whether it was really cost effective as a workstation product.

WPL licenses WPS on Linux as a server product. You don’t get workstation pricing on Linux. Workstation pricing is only for Windows and Macintosh products. If you are expecting a Linux Workstation to be as cost effective (i.e. priced the same) as a Windows or Macintosh Workstation, then you will be disappointed.

I suppose that one could always license a small WPS Linux Server (2LCPU and say 8GB of RAM) and run it in a virtual machine using Xen or Virtual Box and still be less expensive than our competitor’s Windows workstation pricing. But if you’re a developer, you probably need to think about development and test platforms quite a bit.

Here’s a suggestion. Say you are a developer and are looking to develop a vertical market application. You could easily license both Macintosh and Windows workstation products and still be under $2500 a year. Of course, you have the cost of a MacBook or Mac Workstation to add on to that, but it is still very doable and capable as a development machine. Actually, a Mac Mini could work nicely for an Apple development machine. Just make sure you don’t buy the newer model that is not upgradeable since they soldered the RAM in place enforcing planned obsolescence.

The Macintosh product license could suffice for developing and testing code that is to be executed on Linux/Unix computing platforms. Of course, the Windows product license would do the same for Workstation and Server products.

I have taken a set of macro’s that I have in a library and compiled them under OS X. I’ve then copied them over to the Linux machine and it executes flawlessly. This is not an exhaustive test by any means but it does demonstrate what can be accomplished using a Macintosh Workstation. Perhaps a more complete Proof-of-Concept should be attempted and we could certainly arrange an evaluation for those products if you desire.

And finally, one more thought. If you are a developer creating a vertical market application and your target platforms are Windows, Linux and OS X and you need to have WPS Link and WPS Communicate for connecting to a remote server, then you will need a Linux Server or a Windows Server for test and development. If your product does not require pushing code that’s to be executed on the remote server, then just the Mac and Windows Workstations will likely suffice.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

I’ve always been a fool for a bargain…

My last blog post, I was pretty incensed with Microsoft and Windows 10. I just don’t understand some of their decisions as it pertains to professional developers who use their tools and have to struggle with how they force upgrades and perform reboots. I’m still upset so I’ve decided to try a little experiment.

I recently came across a five or six-year-old Dell laptop that was priced (I thought anyway) on eBay at an incredible price. This laptop is a Dell E6420 with an Intel I7 CPU and 4GB of RAM laptop for sale at $150 or best offer. Doing a quick perusal online, this laptop has a screen resolution of 1600×900, an NVidia graphics card and with a 9-cell battery which can get (supposedly) up to 10 hours of battery life. Hoping I was not making a rash decision, I decided to give it a try and make it part of my collection.

This little machine was missing a power adapter and a hard drive. It did power up on battery power so I know it works. I found a power adapter on eBay for $20 and I have an older 500GB HDD sitting in a file drawer. I think I might have a couple of 4GB SODIMM’s somewhere that I tore out of a Mac Mini. If I can’t find those SODIMM’s, then I will buy some RAM online. I may also discard the DVD drive and replace it with a small SSD down the road. I can’t remember the last time I used a DVD in a computer so it’s easy to let that go.

If you are like me, you probably have hundreds of cables, plugs and other accessories laying around and at least 20% of it you have no idea what it belongs to. Yet, you’re afraid to get rid of it because it might be something you need down the road. I guess that is one of the few advantages of being a hoarder. i.e. being able to talk yourself into almost any purchase!

So, being the Froogle individual consciousness consumer I am, I ended up winning this machine at best price of $80. It sounds so counter-intuitive to spend money on five-year-old piece of technology but it does give me the opportunity to try an experiment that I’ve wanted to do for the last year. Now motivated by Microsoft, I decided to just do it.

I’m sure many of you have seen the Dell XPS 13 laptop running Linux. It’s fairly pricey bit of kit but it does get rave reviews. It’s obvious that the XPS 13 is also more powerful. I’m also intrigued by the thought of trying to setup an environment where I’m not using OS X or a Windows environment at all. So right now, I’m not concerned with getting a machine with the ultimate in performance. I’m also a bit concerned with working on such a small screen for long periods.

I’m not at all confident that all the programs exist that I want (or need) to use are available on Linux. But, this is an interesting experiment non-the-less.

The idea behind this concept is to see if I can set up an everyday working environment for writing, programming and performing desktop analytics. I want to be able to plug this into a large monitor and use it as a typical workstation when I’m not lugging it around.

Also, I am not adverse paying money for software. I’m not a believer that everything I put on a PC or laptop with Ubuntu must be free or open source. I do expect to purchase a fair amount of software including UltraEdit.

Initial Software

I’m jumping ahead here because I’m going to have to solve some issues with the Ubuntu Install. I want to make sure the trackpad and power options work. Probably be wrestling with the webcam as well. These seem to be issues I found when Googling the internet. Update: the only thing I can’t get to work properly is the sound from the laptops speakers. I can get sound via Bluetooth, HDMI and the headphone jack. I am not sure how much time I want to spend on the sound issue since I typically use a Logitech USB headset or Bluetooth headset from SoundPEATS. Specifically, the model QY7 if you want some quality sound out of a set of ear buds.

I will post back on how this works out. In the meantime, if you have not yet downloaded WPS v3.3, I strongly suggest you grab that release and step up your game!

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Why Microsoft? Why?

As a business user and knowledge worker, I am extremely dependent upon my workstation and laptop. Like many of my readers, we spend a great deal of time developing a work-flow for analytics as well as business processes for running our business. When I am forced to accept updates and my systems go south, I must spend time fixing this. That’s why I get so frustrated with Microsoft.

The other thing that has really gotten my goat is that ads are now starting to appear in my OS. There are ads in the file explorer window! I want a clean, non-cluttered interface and OS. I don’t want to be hampered by an OS that makes me lose concentration of crap popping up to distract me.

So, I have decided to embark on an experiment. I’m going to setup a used Dell laptop to use as my primary machine running Linux. If after 60 days or so, I will decide if I make the permanent move away from MS for my full-time machine and only use the Windows workstation for testing code.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Technical Document – Post Installation Steps for WPS Workstations

We just finished updating a document for WPS users, specifically those on the Windows Platforms entitled, “Configuring Your WPS Workstation after Installing WPS v3.3.” This document helps those who are evaluating the WPS product on Windows learn about and install some features that is specific to WPS.

The document is short, only 14 pages but touches on modifying the WPS.CFG file as well as installing R and Python to get the greatest amount of utility out of WPS. If you have installed WPS on a Windows Workstation and are looking to get additional utility out of your WPS software, this document is for you.

To download the file, click here or download at:

http://minequest.com/downloads/Post-Installation-Steps-for-WPS-Workstations.pdf (1008KB)

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Why WPS v3.3 is important

A few weeks ago, I posted a blog about the new release of WPS v3.3. Quite a few people and companies have been waiting for this release and I want to talk about why this is such an historic release. I also want to write about what this release means to data scientists.

First, this release is truly a blockbuster. With the Interop for R and Python modules, this is the first time as far as I am aware, where a software product allows a developer to easily use the Language of SAS, R and Python to implement a program, project or product – all in one development environment. As data science has evolved, R and Python have become more popular but BOTH lack the data management capabilities of WPS. With WPS, you not only get the language of SAS to process your data, but you also get a multitude of database engines to read\write and access data in the most popular databases.

The beauty of such integration is ease of development. If you have been a developer for as long as I have, you know the time demands of learning a new development environment. Now, with WPS v3.3 a developer can stay within a single development environment for all of their analytical development needs. The developer doesn’t have to learn an R IDE to write, test and execute R code. The same can be said for Python. The Python interface with WPS means having development control as well as execution control of python programs or similarly R programs.

Organizations want to take advantage of R and Python integration because it allows them to create and expand programs and projects. It will quickly become apparent for third-party developers, i.e. those who want to create vertical market applications that these two additional languages radically increase their tool sets going forward. For many, it also means running production jobs that are self-contained, meaning control and execution is controlled by WPS and not a bunch of separate tasks or processes having to be handled individually.

I have been fortunate to have had access to Alpha and Beta builds of WPS for v3.3 and was totally blown away by the improvements and additions that I saw. For example:

  • The data step is faster.
  • Inclusion of the Python programming language.
  • Implementation of Proc IML.
  • Faster data engine access for many databases including multi-threaded loading.
  • More complete graphics output in statistical procedures.
  • PDF support

But what makes this one of the most compelling releases is the integration of R and Python. With Python specifically, one now has access to executing Machine Learning code from WPS. I’m sure many of you who read this blog are involved in credit scoring, fraud detection, anti-money laundering, market basket analysis, loyalty programs and other real-time analytics. With Python and OpenGL and the CUDA libraries, one can now perform incredibly high speed processing of data on your desktop/server GPU.

When looking at the breadth of the WPS offering, it’s amazing how much is included for the data scientist who needs to work in multiple languages supporting data analytics. With all the database engines (including Hadoop), WPS Graphics, IML, R, Python, WPS Statistics, and WPS Time Series, I almost faint thinking what the cost would be for something similar from our competitor SAS Institute.

The other aspect of WPS v3.3 that is so enticing is the licensing. There are two components to this that deserve discussion. The first, is that Data Service Providers (DSP’s) can make use of this software to develop and provide statistical and mathematical models for third parties. Let’s face it, many if not most organizations lack the expertise to develop these models for companies.

The second component is the creation of Vertical Market Applications using the WPS software. The absolute reasonable cost of the software is a driving factor in being able to create and resell your VMA at a price that small, mid-size and large companies can all afford. Using WPS as the basis for your VMA is advantageous because you’re not beholden to some other organization requiring you to pay partnership fees for access to the software and marketing. I think you would be shocked to learn about how competitive and perhaps ruthless a software vendor can be when it comes to introducing and pricing a competing VMA.

If you want to learn more about the latest release of WPS v3.3, especially as it pertains to workstations, read the previous blog post and download the latest brochure for v3.3. You can request an evaluation of WPS v3.3 by contacting info@minequest.com or filling out our evaluation or quote request form here.

I hope everyone had a Merry Christmas and wishing everyone a Happy New Year.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Product Overview WPS for Workstations v3.3

We just updated our Product Overview document entitled WPS for Workstations v3.3. This document explains the features and breadth of the WPS Workstation product for version 3.3.

The document contains a list of all the database engines and procedures that are included in the workstation product. For organizations considering WPS, this is a good place to start to understand the WPS offerings on OS X and Windows.

You can access the document by clicking here (1.4mb pdf)

or at: http://minequest.com/downloads/WPS-for-Workstations-Marketing-Brochure.pdf

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

WPS v3.3 Now Available

On Thursday December 15th WPL introduced WPS v3.3. This new version is available for immediate download. With v3.3, WPS includes a slew of new Procedures that will be of great value to those who hold WPS licenses and those who are looking to convert over to WPS from SAS.

New Language Procedures

Matrix Language Support is now available with PROC IML. PROC IML is included as a standard procedure in WPS and is not an additional cost module. There is a 430 page programming guide in PDF format that is included in the installation folder detailing how to use the Matrix Programming Language.

Python Support is now included in WPS with PROC PYTHON. PROC Python allows a WPS user to create, edit and invoke python programs from within WPS. The implementation of PROC PYTHON is very similar to PROC R. PROC PYTHON is included in WPS and is not an additional cost module.

ODS Support

WPS now includes output to PDF as well as HTML and Text output destinations. Note that PDF support is available on all platforms except z/OS at this time.

New Statistical Procedures

PROC ACECLUSProvides two methods for approximating the within-cluster covariance structure for a clustering model under the assumption of equal multivariate Gaussian distributed clusters.

PROC CANCORR – Identifies and measures the associations among two sets of variables.

PROC GENMOD – its generalized linear models.

PROC LIFEREG – Fits parametric, accelerated failure time models in the presence of left-, right- and interval censored data.

PROC LIFETEST – Estimates non-parametric survival functions in the presence of censored data using Kaplan-Maier or actuarial methods.

PROC LOESS – Fits non-parametric regression surfaces to multi-dimensional input data. The smoothness of the non-parametric model can be controlled. Outliers in the input data are detected.

PROC MI – Imputes the values of missing values in an input dataset.

PROC MIXED – Fits a mixed linear model to input data.

PROC MODECLUS – Produces various cluster output statistics.

PROC PHREG – Fits the Cox proportional hazards model to survival data.

PROC PROBIT – Fits binary or ordinal response regression models, useful for dose-response type analysis. Various types of model are supported by the procedure. Parameter estimates are generated through the use of maximum likelihood estimation. Model fit statistics enable the quality of the generated model to be assessed.

PROC VARCOMP – Fits generalized linear models with random effects, where the associated covariance matrix is assumed to be diagonal.

Note that WPS Statistics is included in the cost of a WPS license and is not a module that needs to be licensed separately at an additional cost.

New Graphics Procedure

PROC GBARLINE – The GBARLINE Procedure has been added to WPS. This procedure allows you generate bar charts on which plot data has been overlaid on to the bar chart.

New Data Engine

XLSX Engine – This is a cross platform engine that provides read and write access to file in Microsoft Excel format. You can process Excel data on any platform you choose and are no longer limited to Windows platforms. The XLSX engine is included in WPS and is not an additional cost module.

Data Engine Enhancements

NETEZZAM -Is a replacement engine for the NETEZZA Engine. NETEZZAM provides for multi-threaded operation using a new architecture enabling significant performance increases. The NETEZZAM engine is included in WPS and is not an additional cost module.

ORACLEM – Is also a replacement for the ORACLE Engine of prior releases. ORACLEM is also multi-threaded bringing performance increases. The ORACLEM engine is included in WPS and is not an additional cost module.

Both the above engines provide for the ability to Bulk-Load data.

There are a number of additional language features and workbench features that are worth investigating as well. WPS v3.3 is a major release where the functionality and language and procedures have been augmented.

For a list of all the WPS Procedures and Database Engines that are currently supported in v3.3, you can download a two-page brochure from MineQuest. This brochure lists the database engines that are supported on the Linux, OS X and Windows platforms as well as language support and PROC Support.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

The Application Economy

I pretty much finished up my Christmas shopping two weeks early this year. Even the wrapping and delivery completed thanks to Amazon this year. I’ve never had my shopping done so early in December and I’m darn happy about that!

That gave me time to watch some TV this weekend and since much of College Football is over for the season I ended up cruising over to Bloomberg TV. I watched a program called “Hello World” on the Russian Tech scene and it was fascinating to learn about what was being created in Russia.

The sponsor of the show was CA (aka Computer Associates) and they had an interesting and entertaining commercial titled “The Front Porch” which is about the Application Economy. We as analytical developers rarely think about software as an application the same way as consumers do. Our customers are often different departments or divisions in the corporation we work at. We don’t work at creating an application product that meets the needs of tens-of-thousands of users, or even millions. We mostly develop products used for tens of people or if we are lucky, hundreds.

A lot of the reason for that is that many of us don’t see what we do as developing an application that is consumed by users outside of our organization. The cost of commercial software is often so high that it makes it cost prohibitive to invest the hundreds of hours needed to create the application. The other issue many run into is the availability of data that can meet the needs of the consumer and is not protected by agreements.

The market has responded with software such as Python and R. However, the problem with both is the amount of data that can be processed. We live in a Big Data world and expecting data to fit into available memory is often not practical. Many of us are also dependent on using the Language of SAS for processing and displaying of data.

Obviously, WPS is a better choice than SAS when it compares to pricing, especially on the desktop. If you create an application that requires, say, WPS on a workstation, it is much easier to make a sale (your application and a WPS license) when the first-year cost is one-tenth the cost of the SAS system.

In future articles, I want to touch on creating applications for resale using WPS. I want to talk about “applications” for such things as Smart Cities, Marketing, Credit Scoring and Fraud Analytics.

We truly live in an era where we as analysts and statistical developers can contribute our skills starting a business, providing a product and doing it all with minimal cash outlay. The internet is a money pipe into the home and business. Don’t let the opportunity pass you by.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Disruptive Analytics

I picked up Disruptive Analytics available on Amazon which is Thomas Dinsmore’s recent book a few days ago and thought I would leave my impressions. Note this is not a review! First, I really enjoyed the history of the analytics platforms. The second and third chapters were very informative (History and Open Source respectively) and I learned a few things!

Regarding Open Source, I agree that we will see Python supplant R as the “go to language” for analytics in the Open Source arena. It might take a few years but if my customers interests are indicative of this trend, it will happen.

Dinsmore does an admirable job in Chapter 4 on Hadoop. This chapter is fairly dense reading for me mainly because there are a lot of terms and definitions in this chapter. If you were ever looking for an overview of the Hadoop ecosystem, this is probably a good start.

The other chapter I really liked was Chapter 6. This chapter deals with streaming analytics and I believe we are just in the infancy of this revolution. Smart Cities will be a very visible platform for many people to see and benefit from streaming analytics.

I would like to see in a future edition a presentation of the role of the analytics workstation and flash memory in the analytics framework. Data Scientist who are developing algorithms and processing data are often using workstations in lieu of servers. Perhaps even a few pages on how nVidia is revolutionizing the analytics world with CUDA processing on high power workstations. I think I would enjoy that.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is an authorized reseller of WPS in North America.

Ubuntu 16.04 Released and Quick Test Drive

In the last week, Canonical has brought forth a new release of Ubuntu and it is pretty nice! Version 16.04 has a number of great features that should be of value to those who use Linux. One thing that Ubuntu has at this point is a vertical line of products. I can’t think of any other vendor who has an OS that runs on Phones, tablets, notebooks/workstations, servers and mainframes.

I decided to give it a try on one of my workstations running it in an Oracle Virtual Machine (Virtualbox to be specific) to see how WPS runs on this new release. Just to cut to the chase, it runs quite well. As a matter of fact, once I got the VM to use all of its allotted storage, WPS ran like a charm.

clip_image002

A couple of things that might be of interest to potential Ubuntu upgraders. First, Ubuntu 16.04 supports ZFS. That might be important to a few sites. The second is the support for LXD 2.0. From the Ubuntu website –

LXD 2.0

Ubuntu 16.04 LTS includes LXD, a new, lightweight, network-aware, container manager offering a VM-like experience built on top of Linux containers.

LXD comes pre-installed with all Ubuntu 16.04 server installations, including cloud images and can easily be installed on the Desktop version too. It can be used standalone through its simple command line client, through Juju to deploy your charms inside containers or with OpenStack for large scale deployments.

All the LXC components – LXC, LXCFS and LXD – are at version 2.0 in Ubuntu 16.04 LTS.

In addition to trying Ubuntu 16.04 in a VM, I have also tested it on a small server (6 LCPU with 32GB of RAM) running WPS. Although I have not benchmark tested this exhaustively, it does appear that using v16.04 with WPS 3.3.2 (which is the latest release) provides a modest performance increase. This is easily observed with multi-threaded Procedures such as Means and Summary.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is an authorized reseller of WPS in North America.