WPS Analytics – data encryption

 

One of the new features of WPS Analytics is the ability to encrypt a WPS dataset. This has been a long desired (by me!) feature that makes its appearance in the newest release.

data cd;
input @1 CD $2. @4 Party $10.;
Label Party = ‘Party Affiliation’;
cards;
01 Democrat
02 Republican
03 Democrat
04 Republican
05 Republican
06 Republican
07 Democrat
08 Republican
09 Democrat
run;

data cdtestencrypt(encryptkey=”testpwd” encrypt=AES);
set cd;
run;

So we have the temporary dataset cdtestencrypt encrypted. If we try to run a PROC CONTENTS on it, the WPS log returns.

39 proc contents data=cdtestencrypt;
40 run;

ERROR: Data set WORK.cdtestencrypt is encrypted but the ENCRYPTKEY option is not set
NOTE: Procedure contents step took :
real time : 0.003
cpu time : 0.000

We get the same message if we try to read it back into another dataset or use it in any procedure. To be able to use the encrypted dataset, you will need to unencrypt it. Here’s how.

data a;
set cdtestencrypt(encryptkey=”testpwd”);
run;

and the log returns

51 data a;
52 set cdtestencrypt(encryptkey=”testpwd”);
53 run;

NOTE: 9 observations were read from “WORK.cdtestencrypt”
NOTE: Data set “WORK.a” has 9 observation(s) and 2 variable(s)
NOTE: The data step took :
real time : 0.008
cpu time : 0.000

Using the encrypted data set in a proc is as easy as:

63 proc print data =cdtestencrypt(encryptkey=”testpwd”);
64 run;

NOTE: 9 observations were read from “WORK.cdtestencrypt”
NOTE: Procedure print step took :

real time : 0.011
cpu time : 0.000

So, there you have it. You can now encrypt and decrypt WPD datasets to more fully protect your data from prying eyes and unauthorized use.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

GMAP with WPS Analytics v4

In the last blog post, we wrote about the new features in WPS Analytics v4. I wanted to spend some time on one procedure that is new to this release and that is GMAP. GMAP allows you to create thematic maps (also known as chloropleth maps) to visually display your data among boundaries such as state, county and other political areas.

I became quite excited when I found out that mapping was going to be made available in this release. To accompany GMAP, WPS Analytics also includes a few complimentary procedures such as GREDUCE and GREMOVE. PROC GREDUCE allows you to create a lower resolution map and PROC GREMOVE provides a way for you to combine areas into larger regions.

One procedure that is missing at the moment is PROC GPROJECT. In the map below of U.S. Census Regions, you can see the output using unprojected coordinates. It may be possible to use PROC Python to project a map coordinate file to something like an Albers projection. Python has a lot of mapping routines and I wouldn’t be surprised if some developer(s) have created software to change projections.

 

clip_image002

Here is the relatively simple code for creating the map above.

data states48;
set wpsmaps.usstates50;

     if (state > ’56’) or
        (state in ( ’02’, ’15’,’07’, ’14’, ’43’, ’52’)) then delete ;
  run;

 
data regions;
set wpsmaps.censusregions;

if state = ’15’ or state=’02’ then delete;
label regiontext = ‘Region’;
if region=1 then regiontext=‘East   ‘;
if region=2 then regiontext=‘South  ‘;
if region=3 then regiontext=‘Midwest’;
if region=4 then regiontext=‘West   ‘;
run;

 
pattern1 v=ms c=lightBlue; pattern2 v=ms c=Red;
pattern3 v=ms c=Yellow; pattern4 v=ms c=orange;

  
proc gmap data = regions map=states48;
  id state ;
  choro regiontext / discrete;
Title HEIGHT= .25in ‘U.S. Census Regions’;
run;

You can also create maps at the county level. Below is another simple map that highlights the counties that are in the State of Ohio and make up the Cleveland, Columbus or Cincinnati metropolitan areas. The counties that are not part of these three MSA’s is shaded light gray.

clip_image004

The code that I used to create the Ohio Counties map is below.

  data oh;

  set wpsmaps.uscounties;

  where state=’39’;

 

  MSA=‘Other     ;

  if geoid in(‘39041’,‘39045’,‘39049’,‘39089’,‘39097’,‘39117’,‘39129’,‘39159’)

then MSA=‘Columbus  ;

if geoid in(‘39015’,‘39017’,‘39025’,‘39061’,‘39165’)

then MSA=‘Cincinnati’;

  if geoid in(‘39035’,‘39055’,‘39085’,‘39093’,‘39103’)

then MSA=‘Cleveland ‘;

  keep state county msa geoid segment;

  run;

 

  data ohcnty;

    set wpsmaps.uscounties;

    where state=’39’;

  run;

 

  *–> Create the map for the state of Ohio and define the three major MSAs;

pattern4 v=ms c=lightgray;

pattern2 v=ms c=Red;

pattern3 v=ms c=lightblue;

pattern1 v=ms c=yellow;

 

 

proc gmap data = oh(where=(segment < 5)) map=ohcnty;

 where state = ’39’;

  id state county ;

  choro msa / discrete;

Title1 HEIGHT= .25in “Ohio Counties Part of the State’s Largest MSA’s”;

 Title2 “Cleveland, Columbus, Cincinnati MSA”;

 Footnote1 HEIGHT=16pt BOLD J=Center font=Arial ” “;

 footnote2 ” “; Footnote3 ” “;

 Footnote4 HEIGHT=10pt J=left ‘Created by MineQuest Business Analytics’ J=Right font=Arial ‘PROC GMAP using WPS Analytics v4.0’;

 Footnote5 HEIGHT=10pt J=left ‘Authorized Reseller of WPS’ J=Right font=Arial “March 28, 2018 “;

run;

 

With PROC GMAP and congressional district mapping data sets, you can even plot the congressional districts in a state depicting which districts are controlled by the Republicans and which are controlled by the Democrats.

Below is another WPS Analytics generated map that shows which of the nine congressional districts are held by each party.

image

As shipped, WPS Analytics v4.0 only contains map files for France. MineQuest has created a set of map files for the U.S. State and County outlines. These data sets were used in the creation of the two maps in this blog post.

If you licensed your WPS software from MineQuest, World Programming or other resellers, send us an email at info@minequest.com and request the map coordinate files used above. We ask that you provide contact information (Name, Company, Title, Address and Email) so we can send you brochures and updates about our services and products.

Finally, it’s worth pointing out what a great value WPS Analytics is compared to our competition. WPS continues to be added on to, fleshed out, enhanced, or whatever description you want to throw out. The product is an amazing offering with so many procedures and database engines included in the annual license fee.

Speaking of license fees, WPS Analytics is about 10% the cost of our competitors first year fees and about one-third the cost of their renewal price.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

WPS v4–A Hot New Release

Today, WPL has announced the availability of WPS v4. WPS v4 is a rebranding of WPS and is now called WPS Analytics. This is a major release and much more than a mere rebranding of the product. As a a matter of fact, this may be one of the biggest releases by WPL in its history. There’s a lot here so let’s dig in.

This release introduces a new product library to the mix with new procedures that encompass Machine Learning and other Data Science features in both the workstation and server platforms. For most of us, this is what we’ve been waiting for from WPL. The Data Science Components are integrated with a drag-and-drop canvas so that model developers can create models by writing very little code.

One can see from this introduction where WPL is headed with this technology. I suspect we will see the Data Science components become quickly filled out with additional releases.

Note: Customers with v3.x license keys will need to acquire v4 license keys before installing WPS v4 software. V3.x keys will not work on this v4 release. Either contact World Programming directly or your reseller acquire a new set of keys.

 

New Machine Learning Procedures in v4

DECISIONTREE
Creates a regression or classification tree that can be used to predict the value of a target variable from one or more input variables.

GMM
Fits a Gaussian mixture model to a multivariate distribution where the number of components in the mixture may be unknown.

MLP
Creates a Multilayer Perceptron (MLP) neural network – a non-linear machine learning algorithm used for classification and regression.

OPTIMALBIN
Determines the best way to group observations according to the value of an input variable, while ensuring that the grouped data can still be used to accurately predict the value of a target variable.

SVM
Creates a Support Vector Machine that can be used to analyze classification or regression data to predict the value of a target variable.

WPS Analytics version 4 introduces visual predictive data modeling tools using the Workbench Workflow perspective for data mining, predictive modeling tasks, and machine learning. Essentially, this is a drag-and-drop canvass for building the models above.

New Graphical Procedures in v4

GMAP
Enables you to create maps that contain geographic information. Only France is currently supported in this version.

GREMOVE
Enables you to create a map that combines areas into larger regions.

GREDUCE
Enables you to create a lower-resolution version of a map using a density variable appended to the map dataset.

MAPIMPORT
Enables the import of ESRI shapefile data.

SGSCATTER
Creates a grid of multiple scatter plots, where the number of plots are specified using one of the COMPARE, MATRIX, or PLOT statements.

New Statistical Procedures in v4

The following statistical procedures have been added:

BIN
Bins data.

GLMSELECT
Enables effect selection within general linear models.

ICLIFETEST
Analyses interval-censored data.

MDS
Fits multidimensional scaling models to data.

MIANALYZE
Analyzes the datasets generated by procedures that have used imputed missing values created by PROC MI.

PLS
Fits a partial least squares model to input data and provides principal component regression and reduced rank regression.

POWER
Determines the power function of a distribution.

TRANSREG
Fits a specified model to a set of transforms.

 

New General Procedure added in v4

TABLEAU
Exports a dataset to a Tableau Data Extract file.

 

New Data engines added in v4

MARIADB
Enables access to MariaDB databases.

DUMMY
This engine discards data written to it. The DUMMY engine is assumed for null locations:

On Windows any pathname with null as the leaf element, on z/OS any DD name declared with type DUMMY, and on other platforms any location rooted in /dev/null.

In WPS v4 the following database engines have been upgraded and now support multi-threading.

NETEZZA
ODBC
ORACLE
SQLSERVER

 

New Time Series Procedure

PROC LOAN
Enables you to calculate various aspects of a loan, such a principle, repayment amounts, balloon payments, and so on.

 

New Operations Research Procedure

SHEWHART
Enables you to specify Shewhart’s statistical process control.

 

New Platform Support

With Version 4.0, WPS is now also supported on Linux on PowerPC Little Endian architecture.

As a developer, I’m pretty excited about this particular release. The ability to do some thematic mapping is important to lots of industries so that is a welcome inclusion. There’s also a number of performance improvements in this release.

We will write a bit more about this release in the coming weeks. There’s so much material to cover and so much work that has gone into the product. It’s amazing to think that you can purchase a WPS Workstation license from MineQuest for less than 10% of the first year license fee charged by our competitor.

If you are interested in a free 30 day evaluation or a quote for WPS Analytics on workstations or servers, send a note to sales@minequest.com.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Tools vs a Tool Chest of Tools

Back in the day when I owned free-standing homes, I used to go to Lowes or Home Depot probably twice a week. Buying a condo where most of the maintenance is provided was the smartest thing my wife and I did. It cut my Home Depot visits down to maybe once a month and freed up my time to pursue and work on other tasks.

What made me think about this was a conversation thread where two “quants” were arguing over which software products they use. The were extremely specific about the problem they were trying to solve, and one referred to his search as looking for the best tool. That statement made me think about Home Depot, which got me to wondering… are analytical software products best viewed as independent tools or a tool chest loaded with “tools”?

It depends on the software provider and how they look at the market. I think most analytical software providers view the customer as someone who is only interested in a basic set of tools and beyond that will purchase additional tools at an additional price. Of course, there are exceptions to the rule here. Some customers are only looking to solve a problem such as a forecasting problem. They are only interested in a package that forecasts.

Other customers look at the breadth of the toolkit and are interested in applying the tools in the tool chest to other analytic problems. For example, WPS comes quite complete with a well-rounded set of tools for handling data collection, data acquisition, ETL, reporting and many of the more advanced statistical tasks. WPS tends to include all the tools in one package and is priced with heavy discounts when compared to other providers.

Our competitor on the other hand views the customer as one who should buy each tool set individually. If you want graphics, you buy the graphics package. If you want more statistically oriented procedures, you then must buy the statistics tool set. If you want to access data in a database, then you need to buy a specific tool for accessing that data.

This is one aspect that distinguishes WPS from its competitor, SAS. SAS makes sure that you pay for each tool set individually and strives to optimize the tool sets so that you must purchase multiple tool sets to do a task. Of course, this costs you a lot of money.

This practice also has another negative and that is it retards your ability to apply analytical tools to your specific problem. Many of us in the BI space are problem solvers. We are assigned a task to solve and set out to solve the problem. How often have you found the situation where the tool or library that you want, or need is not available? It’s one of these issues that profoundly impacts our ability to completes assignments in a timely fashion and within budget.

When you license WPS on a Windows or OS X Workstation or on a Windows or Linux Server the library modules and database engines that are included with the current release (v3.3) are:

clip_image002

If you are a small business or a startup, purchasing analytical software piecemeal is a costly headache. It is not start up or small business friendly and is designed to gouge the customer as much as possible when cash flow is a precious asset.

clip_image003

Which brings us to the practice of charging extremely high initial fees to acquire the software and then offering discounts for “maintenance” after the first year. I’ll go into more specifics about pricing along with real world examples in another blog post, but suffice to say, this is another excuse for the practice of gouging the customer. The argument that is often used for sustaining this practice is that the cost of bringing on the customer for the first year must include the substantial cost of support. I’m going to lay it on the line and call this a myth. It is not reality and is simply an argument set forth to price the software at the very highest levels.

Seriously, what is the added cost? The software provider still has to provide documentation to existing customers and that is in electronic format. It is the exact same documentation for new customers. So, the cost is the same.

Shipping the software is almost always done by digital download. It is the same downloaded software for existing customers as for new customers. Hence, the cost is the same.

It is well known that our competitor is very difficult to do business with. Even they know it but I’m willing to bet that the sales and marketing culture at that place is so ingrained that it cannot change.

clip_image004

When you and your organization are looking to add new licenses, reduce annual cost and be able to provide your employees with a tool chest that can solve a whole host of problems without having to go back to purchasing for a new acquisition, then WPS is easily the product for you.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Introducing MBA Maps

Street Level Mapping and Location Analytics for the rest of us

 

Over the years we have seen the importance of visual presentation of data using charts and graphs. In our minds, it’s equally important to be able to present data in the form of maps. Whether you are interested in presenting your data at the street level or at a higher geographic level, maps are a great way for viewing spatial relationships.

Here at MineQuest we’ve been working on creating a hybrid mapping service for WPS users. Our first release of the software called MBA Maps (MineQuest Business Analytics Maps) is a first cut at being able to use street level mapping to display data. This release (v0.1) is heavily weighted towards the utilities needed to manipulate data and interface into the cloud services that create the map tiles and perform the geocoding.

Initially, MBA Maps will be available on Windows workstation and followed by Mac workstations a bit later. Linux compatibility will also be made available as well as server platforms for Windows and Linux. We use the Interop for R (PROC R) procedure quite heavily to geocode and create maps. Specifically, we use David Kahle and Hadley Wickham’s ggmap tools for creating street level maps. We use WPS for the data processing and background program creation that manages the mapping calls. WPS is also used as the standard interface for all data management. Meaning, we store all our data in WPS data sets.

We will be releasing the first version of MBA Maps in January. Almost all the programming has been completed and testing continues. The documentation is still a work in progress and is about 50+ pages in length. There are numerous examples (see below) that are used in the documents, but we want to add some additional sample programs to the release. We’re strong believers in learning from code examples around here. We are also trying to include some data sets for learning how to use MBA Maps.

We anticipate creating a free version of MBA Maps that will be limited to using Google services. There are some restrictions by Google and they are a maximum of 2,500 records that can be geocoded a day and 2,500 map requests per day.

Eventually, and sooner rather than later (goal is 2nd quarter 2018) we will create a set of licenses so that those customers who have purchased their WPS licenses through MineQuest will receive a highly enhanced version of MBA Maps as part of their WPS purchase. This enhanced version will allow for using other services for geocoding and mapping. Services such as Bing and OSM will be provided. Access to a remote server hosted by MineQuest may also be included.

Those who purchase their licenses from other resellers or directly from World Programming will have the opportunity to acquire MBA Maps as a separate purchase from MineQuest. Pricing is undecided at this time but there will be an annual license and you will have to renew your MBA Maps licenses through MineQuest to access these additional features and services.

All versions will require a registration and license agreement.

What exactly does MBA Maps do? It allows you to create street level maps and geocode your data. If you have a small number of records to geocode (less than 2500 a day) and a small number of maps to display (also less than 2500), MBA Maps is probably all you need to create and display maps of your data.

If you have larger needs than the 2500 limitation for maps and geocoding, then you will need the non-free version of MBA Maps to get around those limitations. That will be available at a later date.

Here are a few examples of what you can do with MBA Maps.

MBA Maps can geocode a City name, a place in a city such as a park or monument or an address. In the next few pages are examples of creating a “street map” using Google map tiles and placing position markers and labels on the map.

 

This first example demonstrates placing a city on a map and labeling it. The data used was geocode at the City level (i.e. places) and using that to create and display the map.

 

Code used to create Midwest State Capitals.

libname mapdata ‘c:\mbamaps\mapdata’;

%gstreetmap(data=mapdata.midwestatecapitals,

maptype=roadmap,

label=locality,

title=”Midwest State Capitals \nUnited States”,

footnote=”Google Map Created by MBA Maps \nMineQuest Business Analytics LLC \nCopyright (c) 2017″,

mapout=”c:\\temp\\midweststatecapitals.jpg”,

mapwidth=11,

mapheight=8);

 

clip_image002

 

 

The second example is plotting attractions on the map. Here we geocoded some London attractions using the name of the attractions. We then use the name of the attraction (variable name site) as the label on the map.

 

Code for creating London Attractions Map

%gstreetmap(data=mapdata.londonattractions,

maptype=roadmap,

label=site,

title=”London Attractions, 2017″,

footnote=”Google Map Created by MBA Maps \nMineQuest Business Analytics LLC, \nCopyright (c) 2017″,

mapout=”c:\\temp\\londonattractions.jpg”,

mapwidth=8, mapheight=11);

 

clip_image004

 

 

 

The third example is a map of gas stations in Powell, Ohio. We first geocoded the addresses of the gas stations to get their proper latitude and longitude. We then took the geocoded addresses and used them to plot the points on a street map. This demonstrates plotting of addresses on a map.

 

Code for Creating Powell Ohio Gas Stations Map.

 

%gstreetmap(data=mapdata.geocodedpowellgasstations,

 maptype=roadmap,

 label=company,

 title=”Powell Ohio Gas Stations \nUnited States”,

 footnote=”Google Map Created by MBA Maps \nMineQuest Business Analytics LLC \nCopyright (c) 2017″,

 mapout=”c:\temp\powellgas.jpg”,

 mapwidth=11,

 mapheight=8);    

 

 

clip_image006

 

 

 

 

Finally, our fourth example using MBA Maps demonstrates how we can even create a hybrid map. A hybrid map is a satellite view with street names embedded in the map. The map below uses the homes estimated value as a label for the marker. We geocoded a number of addresses at Skyline Country Club in Tucson, AZ. The records we used also contained a value of the homes current price. We used a hybrid satellite map to plot the addresses with the estimated home values on a Google map.

 

 Code for creating hybrid street level map.

%gstreetmap(data=mapdata.homeprices,

  maptype=hybrid, label=homevalue,

  title=”Skyline 5th Fairway Home Values \nTucson AZ”,

  footnote=”Google Map created by MBA Maps \nMineQuest Business Analytics LLC \nCopyright (c) 2017″,

  mapout=”c:\temp\homeprices.jpg”,

  mapwidth=11, mapheight=8);

 

 

clip_image008

 

 


 

Example code for geocoding street addresses which is used for the hybrid google map above.

libname mapdata ‘c:\mbamaps\mapdata’;

 

data a;

length homevalue $6;

input homeaddress $1-44 price 45-48;

homevalue = put(price,comma5.)||’K’;

cards;

4795 E Quail Creek Dr. Tucson AZ 85718       464

4779 E Quail Creek Dr. Tucson, AZ 85718      544

4757 E Cherry Hills Dr. Tucson, AZ 85718     470

4749 E Cherry Hills Dr. Tucson, AZ 85718     449

4880 E Placita Tres Vidas, Tucson, AZ 85718  799

4860 E Placita Tres Vidas, Tucson, AZ 85718 1795

4840 E Placita Tres Vidas, Tucson, AZ 85718  728

;;;;

run;

 

Proc print data=a;

Title “Data to be geocoded”;

run;

 

 

%geocode(api=google, data=a, outdata=b, geocodevariable=homeaddress);

 

proc print data=b;

Title “Geocoded Home Addresses”;

run;

 

 

*–> merge in the geocoded data set with the existing data set;

 

data homeprices;

merge a b;

run;

 

This gives you an idea of what we will be introducing in January of 2018. We have lots of ideas behind our mapping and want to create a whole area of tools and applications for location analytics. Watch this space for an announcement of availability.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Alexa my love, help me understand

I have this love and dislike affair with Amazon’s Alexa products. I’m a huge Alexa user and admirer. I like her voice. I get a smile when she says to me “Have a lovely day.” I’m overjoyed when I’m laying in bed and I can ask her to turn off the Christmas Tree lights or the outdoor patio lights without having to get out of the comfort and warmth of my bed to manually flip a switch.

With Alexa I check my weather forecasts at least three times a day by asking, “Alexa, open Big Sky.” She reads my news brief in the morning. She gives me a run down on what is on my office calendar for the day. She even tells me jokes when I ask. Heck, Alexa even runs our HVAC system at home. It’s convenient to ask her what the indoor temperature is or to ask her to increase the indoor temperature two degrees. In short, Alexa is the third member of our family. Out of sight, but not out of mind!

But Alexa has a few short-comings that irritates me like many men’s first wife. Sometimes she ignores me outright. Sometimes she pretends not to understand me. Even though I’ve said the phrase, “Turn on home office lights” hundreds of times, she responds, “I don’t understand that.” Huh?

Alexa resents that I have another woman in my life and that woman is not virtual but very real. That is my wife. Alexa really doesn’t play well with her. There’s a certain tension between the two of them. I’m not sure about the reasons for this but my wife often must ask her twice to do the same command. That causes some irritation for my wife and eventually for me. Humorously, my wife’s solution is to raise her voice and repeat the command. I sometimes feel sorry for Alexa when she gets told the second time to turn off the fan. It’s almost a scolding.

However, my biggest issue is with shared access. When you live in a multi-member household, members are going to have different email accounts, lists, calendars, etc. So far, the folks at Amazon think that everyone should be sharing the same calendar or email account. I don’t want to know what is on my wife’s calendar or hear how many new Gmail emails are sitting in her inbox. I want to know about just mine. I would love for the solution to be that Alexa, using voice profiles, flips to the email and calendar accounts for the person speaking.

I’m an ardent user of Amazon based or aware Alexa products. My household has two Dots, two Echo’s, a Fire HD 10 Tablet, an Ecobee thermostat, a Netatmo Weather Station and two Fire TV devices. That’s not counting all the wall switches and outlet switches we have to control overhead fan’s and lights.

One of the major issues I have with Alexa is how certain skills are available for a Dot or Echo, but not for the “Fire HD 10 Tablet with Alexa.” I can’t understand why I can’t use “drop in” calling or make phone calls using the tablet like I can with the Echo or Dot. Why am I restricted in such a way from taking my tablet down the hall to a conference room or out on to the patio to make a call or drop-in on another device? If Amazon wants to make a genuine push into the business world (especially for those who work from home) this is something they should address soon. It’s unrealistic to think that many people would have both a tablet and a Dot/Echo sitting on their desks to address calling features.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Edge Analytical Processing

There’s been a lot of work going on in the BI space for the last few years. Some of it interesting, some not so much. But one thing I find very intriguing is what I call Edge Analytics. Think of Edge Computing applied to business and governmental analytical processing.

EA Processing has many facets and I want to cover just a couple of them in this blog post. So exactly what is EA Processing and what advantages does it bring to my organization? EA Processing, in a broad sense allows a company to process most of their data locally. When the demand is high, meaning lots of jobs consuming lots of CPU cycles and I/O bandwidth, the company has the option of firing up a server on the cloud to extend processing capacity. This might only last a few hours or be something that occurs at month end or year-end processing.

Using EA Processing negates having a secondary server (or sets of servers) on premises that doesn’t take up data center space, electricity and maintenance. EA Processing is useful for allowing consultants and contractors to have access to compute power in a highly controlled environment and that can theoretically be more secure than on premise analytic servers.

When viewed broadly, EA Processing can also serve as a center for disaster recovery. With the hurricanes that hit Houston, Texas and ravaged most of the State of Florida, it has become apparent to many that having EA Processing capability is an important feature to consider when building out your BI stack.

Finally, consider this example of EA Processing that is not truly cloud based but does use remote connectivity to do basically the same thing. Say your company has offices in Munich Germany as well as Los Angeles, California. Your organization has data scientist and data analyst at both sites. The time difference between Munich and Los Angeles is eight (8) hours.

The BI staff in Munich can have access to all the computing power on the analytics servers in Los Angeles since most of the staff isn’t even up or at work when the Munich staff are at the office. The Los Angeles BI Staff (for the most part) also have access to the Munich servers since the Munich staff are already out the door heading home while they are work.

Both locations have access to multiple servers to fill in the need when high process demand from one site or another is required. Both locations have control of their local data, work under an identical security model and each have low latency when running local jobs.

With WPS, you can easily configure remote processing with the workbench. You can use WPS Link to run your jobs on any WPS Server. You can also use WPS Communicate to run jobs or parts of jobs on any WPS Server.

When you have multiple WPS Servers, it’s invaluable to keep them busy. Most companies in my experience have servers sitting idle for a good part of the day. Analysts (and I’m speaking from experience here) get really frustrated and testy having to wait for jobs to execute simply because resources are not properly set up to server the users in the most expedient and cost-conscious way.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Improving Screen Time – Workplace Ergonomics

I spend a lot of time in front of a monitor looking at reports, source code, web pages and videos. Over the years I’ve developed a real interest in ways to make my work environment more comfortable and whether you consider it to be an element of age I’ve noticed that my vision is not as sharp as I remember.

I’ve gone the gamut by buying larger screens monitors to trying glasses made specifically for the length I sit from the monitor I seem that I get eye fatigue after just a few hours. At one point a few years ago my eyes would start to go cross-eyed at the end of the day. I’d have to focus at distant objects for quite a while so my eyes wouldn’t go crossed when I looked back at the monitor.

I’ve recently started trying out some computer glasses that are clip-ons and are tinted yellow and are blue light blocking. When I first started out using them I was unimpressed. It took me a few hours to adjust to the tint but after that, the contrast is much higher. Text is clearer and easier for me to read.

The specific glasses I bought are clip-on’s and are called the GameKing Classic. They really have made a difference for me. I just keep them clipped on a pair of reading glasses that are corrected for the distance that I sit between the monitor and my chair.

clip_image002

The glasses come with a cleaning cloth and nice carrying case. I do find that I smudge them up a lot during the day. I never realized how often I adjust my glasses or scratch my nose before I bought these.

Anyway, for $20, it’s a worthy investment.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Thoughts on Open Source

tucson_sunset

Unbelievable sunsets in Tucson, AZ

I recently read an article on Linux.com by Esther Shein titled “The Case for Open Source Software at Work” where she discusses the results of a survey on the use of Open Source software in companies. Pretty interesting read and it makes the argument that IT workers feel about the importance for having source code.

The elephant in the room that is never presented is how the value is measured by accounting or say purchasing. For example, how much perceived value is derived by other parts of the company because they look at the software as being free… i.e. cost free?

Individuals are different in their purchasing and use habits. Most individuals I know are driven by price as the first factor and popularity and completeness as other factors in their consideration.

I can’t recall ever seeing a survey of corporate types that measure the desire for specific software where free vs open source code is derived. I imagine that it may be measured internally by some companies but I would love to see a public survey that addresses that issue.

My own opinion, derived from looking at MS Office vs Libre Office is that quality and support is the most important driver for desktop office software. Every large company that I have consulted with use MS Office. They may use an older version but they use MS Office.

When I switch my thoughts to analytical software, I see the same thing. Corporations purchase or license software like WPS or SAS because of support and completeness. Documentation is also a big factor here too. Individuals who don’t have the financial resources to license analytical software like the aforementioned products gravitate towards free software.

I do grudgingly use R when needed but I prefer WPS over any other analytical software. It’s based on a language that I have used for 30 years and feel very comfortable with. I find it much easier to debug my code and like that if I chose to build a product, I know it will run on Windows, OS X, Linux and the mainframe.

When I factor in that I can license WPS for a bit over $3 a day on a Windows or Mac workstation (and our competitor charges just north of $41 a day for your first year) I find it compelling to have WPS in my BI stack. I can still use R and Python but the language of SAS is just too rich and broad to ignore.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Cost of Porting Language of SAS to Python?

I recently had a discussion with a self-proclaimed data scientist who made a statement that was so broad, I had to challenge it. The discussion taking place is where a technical recruiter who was having difficulty finding SAS/WPS language developers for their marketing group was expressing some frustration. The young data scientist (still in college) jumped into the conversation with a statement “When you could execute the same applications on a open source tool like python it’s not surprising SAS is fading away.” [SIC]

Well first, I’m not real sure he understood that Python or R could not execute SAS language code. The other aspect, at least to me was the shocking naiveté of the statement simply because this data scientist never addressed the economics of the matter. So let’s do it for him.

Performing some back of the envelope calculations, say a programmer that is knowledgeable in both SAS and Python was given a contract to port 30 Language of SAS programs that average 3,000 lines of code. Let’s assume that on average the programmer can convert, test and document each program in two weeks. I’m going to estimate (probably on the low side) that the programmer is paid $85 an hour to do this conversion.

The programmers cost to convert all the programs would be 60 weeks x 40 hours a week x $85 = $204,000. One can procure a license for an 8 vcpu WPS server for ~$22,000 a year. Comparing the cost of a WPS license to spending $204,000 to convert it to Python, it would take more than nine years before you started to see a pay back on the conversion. Most ROI calculations I see in the tech industry are predicated on three years and not nine.

I just don’t see the ROI of converting existing Language of SAS code to Python unless you want to pay more money and be RAM constrained. However, I do see the ROI in converting your SAS Institute licenses to WPL licenses and execute the same code for much less. The pay back is almost immediate!

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.