Tag Archives: Windows

CleanWork for Windows

Recently, we decided to go back through some of our older programs and take a look at them and see if they could be updated and/or made open source. We wrote Cleanwork years ago and we often provided it to organizations that used our consulting services as a freebie and a way to say “Thank You.”

CleanWork does pretty much what the name says. It is a WPS program that when run, will clean out the work folders of old and orphaned directories that are no longer used. WPS comes with a cleanwork program for Linux and Mac but not for Windows. The version written by MineQuest will run on Windows Workstations running Vista, 7, 8, 8.1 and 10. It will also run on Windows Servers such as Windows Server 2008, 2008 R2, 2012 and 2012 R2. Basically, it will run on all Windows Servers except 2003 and before. It also runs on all Windows Workstations except XP and before.

Cleanwork is packaged in a zip file that contains the source code, the Usage Document, License and a sample program. Cleanwork has been tested to execute only on the WPS platform.

If you are running WPS on a Windows Server you may want to set cleanwork to run on a schedule. This is a perfect utility to automate and run on a regular schedule. For busy server installations, I could see setting a scheduler to run cleanwork every few hours.

The zip file contains five files. These are:

clean.sas – a sample program for running the cleanwork utility.

cleanwork_source.sas – the actual source code that implements the utility.

CleanWorkUsage.docx – a Microsoft Word document that explain how to use cleanwork.

SASMACR.wpccat – a compiled version of the macro that  is ready to run.

license.txt – The license agreement for use of the source code and user document.

You can find the download by going to the bottom of the page here.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

 

 

 

Richmond, CA Hackathon – Meeting of the Minds

On the weekend of October 17-18th the Meeting of the Minds Civic Hackathon will take place in Richmond, California.  Amongst the various tools and facilities that will be available for the Hackathon, World Programming will be providing WPS software (www.teamwpc.co.uk/products/wps ) and support for any SAS programmers taking part in the event who would like to create and run programs in the language of SAS. The WPS software will be available on a server provided by Cisco and also for installation onto your own workstations running Linux, OS X or Windows.  Teams who use WPS software at this event will be given a license at no cost, and can use the product and all of its features for an additional 6 months after the event.

There will be data sets that can be used to create civic oriented applications and the data is categorized into Economic Development, Public Spaces, Health and Environment, Sustainability, Digital Divide and Education. So there is plenty of data available for a myriad of subject matter experts to use.

There is a $5,000 cash prize from Qualcomm awarded to the winner.

More information on the Hackathon can be found at: http://tinyurl.com/p6ymuot

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

Some thoughts on a rainy Monday

The more I use Linux, the more I come around to understand just how much I can do with it. As a matter of fact, I could easily do without Widows and switch 100% of the way over to Linux if I wanted. The desktop(s) and business applications have really gotten that good.

Windows 8 just soured me on the whole MS ecosystem. When they bolted on the Metro interface on a server OS — that was the last straw for me. Who ever made that decision to strap on a touch interface to a server should be let go. Shown the door. Asked to leave…

I have Apple hardware here in the office, and it runs well, but I just have not been able to embrace it like so many others have. Apple makes some fine hardware and there’s a load of support for Office productivity applications as well as analytical apps. WPS runs quite well on OS X as well as R. As a matter of fact, I see a lot of R users who work on OS X as there preferred platform.

But Linux, and specifically Ubuntu 12.04 and 14.04 have been especially good. I don’t have memory issues when I run large simulations in R that require a lot of RAM. With Windows, that is often a problem, trying to allocate a large block of memory and there’s not sufficient contiguous memory to hold a large array, vector or data frame. The memory management is significantly different under Linux than under Windows.

Use of NVidia’s CUDA framework seems to be predominantly used on Linux and not Windows. I’m not sure why that is to be honest.

I’ve been reading a lot of articles stating that MS is working feverishly trying to get Windows 9 out the door. No doubt (at least in my mind) it has to do with the terrible Metro interface and people staying away in hoards. Of course, you can slap Start8 by Stardock on Windows 8 and it makes it useable by implementing the start button, and kudos to Stardock for doing such a thing, but I still can’t find a way to embrace MS on the desktop any longer.

An interesting phenomena that I have been witnessing is how much analytical and scientific development has been happening over the years on the Linux platforms. There are a lot of tools out there that are helpful if you are a data scientist or working with “BIG DATA” as it pertains to Linux. My experiences in reselling WPS is that there is an equal amount of interest (perhaps more) in using Linux on servers than in running Windows servers. Cost is one factor but performance is also a factor. Linux often out performs Windows Servers dollar for dollar and CPU second to CPU second.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

Integrating WPS and Skype

One of the things I like to do in programming is integrating other tools with WPS. I don’t know why I find that so fun to do, but I just find it rewarding for some reason. Every once in a while, I stumble across a problem to be solved and it just seems like it should be something that is easy, simply because it’s so useful. That is rarely the case as we all know.

I always wanted to have some way for an interactive WPS job or a WPS batch job to notify me of an error when it reaches completion or to notify me that it has reached a certain point in the job process. One of the main requirements is that the notification be sent by text message so that I could be notified if I wasn’t sitting directly in front of the monitor of the machine that the code was submitted on.

We have been able to send text messages for years (see “Sending SMS Text via WPS”) but that was kind of clumsy because as far as I could tell, it would only send the message to my cell phone. I want more! Give me more!

Skype is the logical choice for me because I sit in front of a PC most of the day that has Skype running. I also have Skype installed on Skype is the logical choice for me because I sit in front of a PC most of the day that has Skype running. I also have Skype installed on mobile Apple and Android devices so I can receive messages through those devices pretty easily.

All I needed to do is install Skype on a Windows Server. The Server I installed it on is running Windows Essentials 2012 R2. Getting Skype on that platform can be a chore but it can be done. I ended up installing the Windows 8 version of Skype on the server and after getting it to run properly (i.e. sending and receiving messages) I installed the classic Skype for Desktop on the machine. If you are running a previous version of Windows Server… well good luck on getting it to install.

Usage

%WPS2Skype(SkypeID=minequest_llc,

                         category=-U,

                         SkypeMsgTxt = “Your job reached the halfway point you awesome guy!”);

 

Where

SkypeID is the ID of the recipient of the message

Category = -U or –CC

SkypeMsgTxt =”Your message text here.”

 

If you are a WPS programmer or are familiar with the macro language, then the above should be self-explanatory with the exception of Category. Category can take one of two options, either –U or –CC. If we want to send the message to an individual user than we use –U. If we want to send the message to a list of contacts then we use –CC. A contact list is somewhat similar to a group and you should read up on what a contact list is and how it can be created in the Skype documentation.

Running the following code:

%WPS2Skype(SkypeID=xxxxxxxxxx,

                         category=-U,

                         SkypeMsgTxt = “Your job reached the halfway point you awesome guy!”);

 

We see in the WPS log the output when the message was sent:

 

2         %WPS2Skype(SkypeID=xxxxxxxxxx,

3         category=-U,

4         SkypeMsgTxt=”Your job reached the halfway point you awesome guy!”);

 

NOTE: WPS2Skype returned execution to WPS.

 

And what I see in the Skype message window on my workstation is:

 

 

 

clip_image001

 

 

Of course, this will show up on all the devices that I have logged in to Skype using my Skype ID.

 

I also extended the Skype interface into the RunWPS.CMD file. The RunWPS.CMD is a Windows command file that allows you to run WPS as a batch job. With the integration of Skype with RunWPS.CMD, I catch all return codes from WPS and send out a Skype message if the return code is greater than zero. This is easily modified but I’m not personally interested in getting messages when jobs complete properly. I’m only interested when jobs terminate abnormally.

 

The WPS2SKYPE utility will be available in our next release. There are a few more utilities that I want to add to the package and I still have documentation that needs to either be written or cleaned up.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

I’ve Grown Weary of Windows

This blog post is going to be a rant.

I’m so frustrated with Windows Server 2012 R2 that I can spit nails. Who in their right mind at Microsoft thought changing the interface on a SERVER to what is used in Windows 8 was a good idea? If you want to do any real administrative work on the server it is just a nightmare.

I’ve played with Windows 8 on a desktop and didn’t care for it and decided that Windows 7 was so much better for productivity. The mixture of a tablet OS and a desktop OS is just a disaster. In my opinion, MS not only missed the boat, but continues to ignore the market place as it centers on business users.

Going forward, I’m going to start recommending that clients use Linux on their servers and just forget about using Windows Server products. It just isn’t worth the hassle and with the number of talented Linux users and administrators growing every day, there isn’t any upside anymore to using Windows on the Server. There are incredible cost savings in both dollars and time using Linux instead of Windows.

My own thoughts on the server is that Linux is faster than Windows. You don’t have all that eye candy eating up resources. Linux is faster, more robust and has virtually the same number of databases available that you have under Windows. The exception being SQL Server. If you have to run SQL Server than put it on the smallest box possible and minimize your exposure to Windows. There are many databases that you can use on Linux that will fill the void of SQL Server. For example, DB/2, Oracle, MySQL, MariaDB, PostgreSQL, Teradata, Vertica, Sybase, SAND, Netezza, Kognitio, Informix, and Greenplum all run on Linux x86. And the kicker is that all of the above DB’s are supported and accessible from WPS.

I’m also starting to review and reconsider my position of Windows on the desktop. If Windows 9 is the abortion that Windows 8 (and 8.1) continues to be, then you can bet that I will start using Linux on my desktop or (God forbid that I’m saying this…) OS X. I talk to a lot of analytics users and this is something that we all agree on. I need to be productive at work and I’m more productive with Linux and OS X than with Windows 8.

That’s the bottom line.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

test, Test and TEST

I’m always amazed and somewhat peeved about how much error checking one has to have in their SAS language program. Even for the simplest things. So today’s mantra is test, Test, and TEST!

I was writing some code the other day to copy a WPS data set from my PC to a server share. The basic code is really quite simple, only three lines using PROC COPY. But what if the user has misidentified the source libname or the destination libname? Do you just let it blow up and hope the user looks at the logs? And then you have the data sets to be copied if you are using the SELECT statement. Do you check if the data set already exist and if so, just overwrite the file?

Although it was trivial, albeit time consuming to write the code to check for these conditions, it is well worth it. I purposely decided not to automatically overwrite an existing data set on the server. And that is good for two reasons. First, I want the user to be forced to make the decision to overwrite the data set by use of a PROC DATASETS or PROC DELETE before the copy takes place. That makes it their responsibility to delete the data set.

Secondly, I found out that writing the data set with the same name can sometimes create problems under Windows when the server folder is shared. I have had some experiences where Windows locks the file on the server and the copy never takes place. The copy procedure just hangs with a .lck extension on the file. So something is going on where it’s just not reliable.

One interesting thing to note, I don’t seem to have the problem with a lock on Linux. The copy takes place without issue every single time.

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

 

Try WPS and Enter to Win a Google Nexus 7 Tablet

Have you ever found yourself interested in exploring and evaluating the capabilities of WPS (pronounced whips), a SAS language compatible software application? Whether you are looking at ways to save money on reduced license fees or expand your BI infrastructure, WPS is a high value, low cost alternative for many organizations who are currently using the SAS System.

WPS implements the BASE language including old and new style macro’s and the data step with functions, formats and informats. WPS also includes the most popular BASE PROCS and some graphics including GCHART, GPLOT and GREPLAY.

WPS Core supports 41 Base Procedures.
WPS Engine Modules supports 24 database and access engines.
WPS Statistics contains 15 Statistical Procedures.
WPS Graphics supports GCHART, GPLOT and GREPLAY.

Remember, WPS is priced inclusively, meaning you don’t pay separately for each module. You can learn more about WPS by visiting  our web site at: http://www.minequest.com/WPS.html. Also, available for download is a brochure that describes WPS in more detail including a list of supported PROCS and database access engines at: http://www.minequest.com/downloads/WPS_General_Overview.pdf.

As they often say, the best way to learn about something is to try it. Between August 10, 2012 and September 30, 2012, you can become eligible to win a Google Nexus 7 tablet by taking out a free 30 day evaluation copy of WPS on Windows (desktop and/or servers), Linux on x86 or Mac OS X. You must request your evaluation from MineQuest Business Analytics, LLC, an authorized WPS reseller. See Rules for drawing for more information.

Rules for drawing: Only one evaluation request per person. Must be a resident of the United States or Canada. Must not be a current WPS license holder or have taken out an evaluation in the last nine (9) months. Requests for evaluations must be made through MineQuest at: http://www.minequest.com/ReqEval.html . Drawing to be held on October 3, 2012

 

About the author: Phil Rack is President of MineQuest, LLC and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

PDF Support for the Bridge to R

We’re still developing and adding on to the Bridge to R and want to announce a major enhancement to the product. We will be supporting output to PDF under the Windows version. A number of customers have asked for PDF support and we believe adding this is an easy and quick way for our customers to start creating reports using a PDF destination.

Here is a code snippet that shows how you would create a PDF file. We introduce two new macros that do the work. The first one, %PDF takes the parameters for creating the PDF file. The second macro, %PDFClose terminates the code for what will appear in the PDF file.

   1: options ls=133 ps=60;

   2:

   3: %pdf(autoview=TRUE,

   4:      pdffile='c:\temp\pdfnewandbig.pdf',

   5:      pdfoptions= -pfn200 -pps4 -pfs8 -pot);

   6:

   7:    proc print data=a;

   8:       title 'Sample PDF Output using the Bridge to R';

   9:       footnote 'Generated the Bridge to R by MineQuest';

  10:    RUN;

  11:

  12:    Proc Means data=a n nmiss min max range std maxdec=4;

  13:    run;

  14:

  15: %pdfclose;

We allow you to add a number of parameters to modify the PDF output. The parameters above tell the macro which font to use (-pfn200), the page size (-pps4), the point size (-pfs8) and to use a landscape orientation (-pot). There are many other parameters that you can use including encryption.

The PDF functionality will be available in the next release of the Bridge to R which should be available by the end of October. There will be some updated documentation that will include PDF support and the new documents will remove any references to the Bridge running under the SAS System since we don’t support that any longer.

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.

Background on the Bridge to R for WPS Users

I want to spend some time discussing a few issues surrounding the Bridge to R, it’s background and the philosophy of the product. I do get a number of inquiries on the Bridge so I think it would be prudent to talk about it as well as how we arrived at the pricing.

I created the Bridge to R originally as an exercise to see if I could run R programs from within WPS which is a SAS Language alternative software system. I wanted to be able to feed R data that was created and manipulated by the WPS System and get all the R output back into WPS. I worked on this over a period of a month and eureka! it worked. Since one programs WPS in the language of SAS, I was able to instantly port the Bridge to R to work directly with SAS without any changes.

Once I wrote some documentation, I put the Bridge to R out on the internets for both SAS and WPS users alike to use. The idea originally was to have the Bridge be able to work in hybrid environments — i.e. organizations using both SAS and WPS so that developers could use R in a very portable fashion. I had the Bridge to R out on the MineQuest website for four to six months and it had a reasonable number of downloads. There were many more SAS users who downloaded the Bridge than WPS users, but that is understandable considering the size of the user base.

I eventually came to the realization that SAS users were not really interested in using the Bridge to R as a method to write portable statistical routines between WPS and SAS, but instead were using the Bridge to R as a way to get around licensing SAS’s IML Studio product. The Bridge to R was a free download and had no pricing attached to it at the beginning. Since I was starting to resell WPS in North America, I was not thrilled by that because I want users to license WPS!

At that point, I felt it was necessary to pull back support on the Bridge to R for SAS Users and only offer it to WPS users. Being able to access R from within the WPS environment directly was a huge money saver in contrast to the SAS product offering and I felt that there should be a reward for SAS Language Developers and statistical analysts who want to use R to get the product at low cost or no cost.

At that point a decision was made to only support WPS and dropped support for SAS. There are numerous reasons for this decision and a few of the most pertinent ones are support costs, licensing a SAS development environment was prohibitive, and I wanted to offer an incentive to companies to purchase WPS licenses from my company, MineQuest.

After a fair amount of discussion, it was decided that MineQuest would provide a copy of the Bridge to R to any organization that licensed WPS from us on either Windows or Linux platforms. This copy of the Bridge is provided as a product from MineQuest that is totally separate from the WPS product created by World Programming LTD. Since MineQuest only resells WPS in North America, we only supply the Bridge to R with WPS to North American customers. So that’s the incentive, if you buy a WPS license from MineQuest and in North America, you get the Bridge to R as part of the purchase.

So what about WPS customers who are not from North America or who purchased their WPS licenses directly from World Programming? We still want to provide the Bridge to R to your company, but the economics of the situation dictate that we charge a small amount for the software. Currently, we charge $199 for the desktop version, and $599 for a server version of the Bridge.

I’ve been asked on the phone as well as by email, why I don’t distribute the Bridge to R to all WPS users. The answer is because there’s costs involved in the development, maintenance and support of the product. Every time a new version of WPS roles out or a new release of R hits CRAN, we make sure that the Bridge to R continues to operate on all the Windows platforms as well as on Fedora Linux and Ubuntu. We run a multitude of test programs on XP, Vista and Windows Server for both 32-bit and 64-bit installations as well as for the Linux platforms. All of this takes time and effort and quite honestly, we can’t do this for free. Hence, we charge WPS licensee holders a fee if they didn’t purchase their license from MineQuest.

So there’s the history and background on the Bridge to R and why we price it the way we do. I hope you understand that there’s significant cost in the development and support of the Bridge to R. The bottom line is that if having access to R is important to you for advanced analytical techniques and graphics, then you should strongly consider licensing your WPS software directly from MineQuest.

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.

Google Refine 2.0 – Cleaning Messy Data

I’ve been playing around with Google Refine off and on for a few days and thought I would share my impressions of the software. First, Google Refine is a “power tool for working with messy data, cleaning it up, transforming it from one format to another…” and that pretty well describes what it does.

With Google Refine on your desktop, you can collapse data that is similar and find anomalies in your data, fix the data and continue on with your analytics project. Google Refine is a web based system and all your interaction is performed via your web browser. This can be an advantage in that you can work with cleaning your data on either Windows, Mac or Linux platforms. My testing so far has only been on the Windows Vista platform.

Installation is pretty easy although there is a mistake in the documentation. The instructions say to “Download, unzip, and double-click on refine.exe.” The problems is that there’s no refine.exe in the zip file. The correct file you want to double click on is google-refine.exe.

After you install the software, you will probably be faced with the tasks of importing and exporting data to Google Refine. Using WPS 2.5, you can export your data to Refine in either CSV format or if you have the Bridge to R you can use XML format. The only database connection that I can find that Refine supports is Freebase. It does not seem to support DB’s like MySql, DB2, etc… So essentially, you are going to have to get your data into a format that Refine can handle in regards to data transfer.

Refine has lots of features and they do have a set of videos that demonstrate how to use the software. I would suggest that you view them before getting your hands dirty with the software. The videos can be found at:

http://code.google.com/p/google-refine/wiki/Screencasts

I’ll write some more about the software in a later blog posting, but I do believe that Google Refine 2.0 is a great addition to a data analysts arsenal of tools.

Links: http://code.google.com/p/google-refine/wiki/Downloads?tm=2

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.