Tag Archives: Geocoding

Introducing MBA Maps

Street Level Mapping and Location Analytics for the rest of us

 

Over the years we have seen the importance of visual presentation of data using charts and graphs. In our minds, it’s equally important to be able to present data in the form of maps. Whether you are interested in presenting your data at the street level or at a higher geographic level, maps are a great way for viewing spatial relationships.

Here at MineQuest we’ve been working on creating a hybrid mapping service for WPS users. Our first release of the software called MBA Maps (MineQuest Business Analytics Maps) is a first cut at being able to use street level mapping to display data. This release (v0.1) is heavily weighted towards the utilities needed to manipulate data and interface into the cloud services that create the map tiles and perform the geocoding.

Initially, MBA Maps will be available on Windows workstation and followed by Mac workstations a bit later. Linux compatibility will also be made available as well as server platforms for Windows and Linux. We use the Interop for R (PROC R) procedure quite heavily to geocode and create maps. Specifically, we use David Kahle and Hadley Wickham’s ggmap tools for creating street level maps. We use WPS for the data processing and background program creation that manages the mapping calls. WPS is also used as the standard interface for all data management. Meaning, we store all our data in WPS data sets.

We will be releasing the first version of MBA Maps in January. Almost all the programming has been completed and testing continues. The documentation is still a work in progress and is about 50+ pages in length. There are numerous examples (see below) that are used in the documents, but we want to add some additional sample programs to the release. We’re strong believers in learning from code examples around here. We are also trying to include some data sets for learning how to use MBA Maps.

We anticipate creating a free version of MBA Maps that will be limited to using Google services. There are some restrictions by Google and they are a maximum of 2,500 records that can be geocoded a day and 2,500 map requests per day.

Eventually, and sooner rather than later (goal is 2nd quarter 2018) we will create a set of licenses so that those customers who have purchased their WPS licenses through MineQuest will receive a highly enhanced version of MBA Maps as part of their WPS purchase. This enhanced version will allow for using other services for geocoding and mapping. Services such as Bing and OSM will be provided. Access to a remote server hosted by MineQuest may also be included.

Those who purchase their licenses from other resellers or directly from World Programming will have the opportunity to acquire MBA Maps as a separate purchase from MineQuest. Pricing is undecided at this time but there will be an annual license and you will have to renew your MBA Maps licenses through MineQuest to access these additional features and services.

All versions will require a registration and license agreement.

What exactly does MBA Maps do? It allows you to create street level maps and geocode your data. If you have a small number of records to geocode (less than 2500 a day) and a small number of maps to display (also less than 2500), MBA Maps is probably all you need to create and display maps of your data.

If you have larger needs than the 2500 limitation for maps and geocoding, then you will need the non-free version of MBA Maps to get around those limitations. That will be available at a later date.

Here are a few examples of what you can do with MBA Maps.

MBA Maps can geocode a City name, a place in a city such as a park or monument or an address. In the next few pages are examples of creating a “street map” using Google map tiles and placing position markers and labels on the map.

 

This first example demonstrates placing a city on a map and labeling it. The data used was geocode at the City level (i.e. places) and using that to create and display the map.

 

Code used to create Midwest State Capitals.

libname mapdata ‘c:\mbamaps\mapdata’;

%gstreetmap(data=mapdata.midwestatecapitals,

maptype=roadmap,

label=locality,

title=”Midwest State Capitals \nUnited States”,

footnote=”Google Map Created by MBA Maps \nMineQuest Business Analytics LLC \nCopyright (c) 2017″,

mapout=”c:\\temp\\midweststatecapitals.jpg”,

mapwidth=11,

mapheight=8);

 

clip_image002

 

 

The second example is plotting attractions on the map. Here we geocoded some London attractions using the name of the attractions. We then use the name of the attraction (variable name site) as the label on the map.

 

Code for creating London Attractions Map

%gstreetmap(data=mapdata.londonattractions,

maptype=roadmap,

label=site,

title=”London Attractions, 2017″,

footnote=”Google Map Created by MBA Maps \nMineQuest Business Analytics LLC, \nCopyright (c) 2017″,

mapout=”c:\\temp\\londonattractions.jpg”,

mapwidth=8, mapheight=11);

 

clip_image004

 

 

 

The third example is a map of gas stations in Powell, Ohio. We first geocoded the addresses of the gas stations to get their proper latitude and longitude. We then took the geocoded addresses and used them to plot the points on a street map. This demonstrates plotting of addresses on a map.

 

Code for Creating Powell Ohio Gas Stations Map.

 

%gstreetmap(data=mapdata.geocodedpowellgasstations,

 maptype=roadmap,

 label=company,

 title=”Powell Ohio Gas Stations \nUnited States”,

 footnote=”Google Map Created by MBA Maps \nMineQuest Business Analytics LLC \nCopyright (c) 2017″,

 mapout=”c:\temp\powellgas.jpg”,

 mapwidth=11,

 mapheight=8);    

 

 

clip_image006

 

 

 

 

Finally, our fourth example using MBA Maps demonstrates how we can even create a hybrid map. A hybrid map is a satellite view with street names embedded in the map. The map below uses the homes estimated value as a label for the marker. We geocoded a number of addresses at Skyline Country Club in Tucson, AZ. The records we used also contained a value of the homes current price. We used a hybrid satellite map to plot the addresses with the estimated home values on a Google map.

 

 Code for creating hybrid street level map.

%gstreetmap(data=mapdata.homeprices,

  maptype=hybrid, label=homevalue,

  title=”Skyline 5th Fairway Home Values \nTucson AZ”,

  footnote=”Google Map created by MBA Maps \nMineQuest Business Analytics LLC \nCopyright (c) 2017″,

  mapout=”c:\temp\homeprices.jpg”,

  mapwidth=11, mapheight=8);

 

 

clip_image008

 

 


 

Example code for geocoding street addresses which is used for the hybrid google map above.

libname mapdata ‘c:\mbamaps\mapdata’;

 

data a;

length homevalue $6;

input homeaddress $1-44 price 45-48;

homevalue = put(price,comma5.)||’K’;

cards;

4795 E Quail Creek Dr. Tucson AZ 85718       464

4779 E Quail Creek Dr. Tucson, AZ 85718      544

4757 E Cherry Hills Dr. Tucson, AZ 85718     470

4749 E Cherry Hills Dr. Tucson, AZ 85718     449

4880 E Placita Tres Vidas, Tucson, AZ 85718  799

4860 E Placita Tres Vidas, Tucson, AZ 85718 1795

4840 E Placita Tres Vidas, Tucson, AZ 85718  728

;;;;

run;

 

Proc print data=a;

Title “Data to be geocoded”;

run;

 

 

%geocode(api=google, data=a, outdata=b, geocodevariable=homeaddress);

 

proc print data=b;

Title “Geocoded Home Addresses”;

run;

 

 

*–> merge in the geocoded data set with the existing data set;

 

data homeprices;

merge a b;

run;

 

This gives you an idea of what we will be introducing in January of 2018. We have lots of ideas behind our mapping and want to create a whole area of tools and applications for location analytics. Watch this space for an announcement of availability.

 

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in beautiful Tucson, Arizona. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS consulting and contract programming services and is an authorized reseller of WPS in North America.

Geocoding with WPS and the Bridge to R

 

One of the things that I truly enjoy is the flexibility of the language of SAS and how well WPS allows you to integrate the product with other services. One aspect of research, whether you are in the social sciences, marketing or some other area of study is the use and application of location data. 

I’m not talking necessarily about getting the data from your cell phone on where you are or have been, but in taking address data and using it to create a business advantage. Visualizing data on a map is important for many people but it’s often a laborious task to get all the data enhanced so that it can be mapped or plotted. Specifically, I’m talking about taking an address and finding additional information such as political districts and latitude and longitude for each address.

Using the Bridge to R and WPS, we can use R to geocode our data. In our example, we will use ggmap which was developed and written by Hadley Wickham and David Kahle. It is a truly amazing application and every time I use it, I learn something new.

In this example, I have 20 records that have the name and address of gas stations in Cascade Township which is a part of Grand Rapids, Michigan. What we want to do is geocode these 20 records to find their latitude and longitude. Below is the entire code snippet to do just that.

data gasstations;
input company $1-29 address $30-52 city $53-64 state $66-67;
addr2geocode=trim(address)||', '||trim(city)||', '||trim(state);
cards;
Citgo Gas Station            5189 28th St Se        Grand Rapids MI
28th Street BP               5155 28th St Se        Grand Rapids MI
Twenty-Eighth Street C Store 5556 28th St Se        Grand Rapids MI
Speedway                     4045 28th St Se        Grand Rapids MI
Speedway                     2305 E Paris Ave Se    Grand Rapids MI
Superamerica                 2305 E Paris Ave Se    Grand Rapids MI
Shell Food Mart              3960 28th St Se        Grand Rapids MI
Admiral Petroleum            3927 28th St Se        Grand Rapids MI
Cascade C Store              4591 Cascade Rd Se     Grand Rapids MI
Friendly Food Shops          6799 Cascade Rd Se     Grand Rapids MI
Family Fare Quick Stop       6799 Cascade Rd Se     Grand Rapids MI
Cascade Citgo                6820 Cascade Rd Se     Grand Rapids MI
Dutton Fuel Mart LLC         2560 E Beltline Ave Se Grand Rapids MI
Centerpointe Marathon        2560 E Beltline Ave Se Grand Rapids MI
Shell Food Mart              2600 E Beltline Ave Se Grand Rapids MI
Speedway                     4018 Cascade Rd Se     Grand Rapids MI
Grand Rapids Gas Incorporated3214 28th St Se        Grand Rapids MI
Cascade Shell                4033 Cascade Rd Se     Grand Rapids MI
Speedway                     4665 44th St Se        Kentwood     MI
Super Petroleum Incorporated 2411 28th St Se        Grand Rapids MI
;;;;
run;

proc print data=gasstations;
var addr2geocode;
run;


%rstart(dataformat=csv,data=gasstations,rGraphicsFile=);
datalines4;

   attach(gasstations)

   library(ggmap)

   gaddress <- as.character(gasstations$addr2geocode)
   locs <- geocode(gaddress,output="more")

;;;;
%rstop(import=locs);

proc print data=locs(drop=var2);
run;

The output that is returned from the PROC Print looks like:

                                                 The WPS System                     19:07 Thursday, November 14, 2013    1
                                                                                                                                    
 Obs          lon          lat type             loctype                                                                             
                                                                                                                                    
   1  -85.5396645   42.9127946 street_address   rooftop                                                                             
   2   -85.540616    42.913151 street_address   rooftop                                                                             
   3  -85.5308863   42.9129211 street_address   range_interpolated                                                                  
   4  -85.5678243   42.9128289 street_address   rooftop                                                                             
   5  -85.5692342    42.921498 street_address   range_interpolated                                                                  
   6  -85.5692342    42.921498 street_address   range_interpolated                                                                  
   7  -85.5685794   42.9125298 street_address   range_interpolated                                                                  
   8  -85.5700129   42.9125533 street_address   range_interpolated                                                                  
                                                                                                                                    
 Obs address                                                                                               north        south       
                                                                                                                                    
   1 5189 28th street southeast, grand rapids, mi 49508, usa                                         42.91414358  42.91144562       
   2 5155 28th street southeast, grand rapids, mi 49512, usa                                         42.91449998  42.91180202       
   3 5556 28th street southeast, grand rapids, mi 49512, usa                                         42.91427683  42.91157887       
   4 4045 28th street southeast, grand rapids, mi 49512, usa                                         42.91417788  42.91147992       
   5 2305 east paris avenue southeast, grand rapids, mi 49546, usa                                   42.92284723  42.92014927       
   6 2305 east paris avenue southeast, grand rapids, mi 49546, usa                                   42.92284723  42.92014927       
   7 3960 28th street southeast, grand rapids, mi 49512, usa                                         42.91388553  42.91118757       
   8 3927 28th street southeast, grand rapids, mi 49512, usa                                         42.91389553  42.91119757       
                                                                                                                                    
 Obs         east         west  postal_code country         administrative_area_level_2 administrative_area_level_1 locality        
                                                                                                                                    
   1 -85.53831552 -85.54101348        49508 united states             kent                      michigan            grand rapids    
   2 -85.53926702 -85.54196498        49512 united states             kent                      michigan            grand rapids    
   3 -85.52953782 -85.53223578        49512 united states             kent                      michigan            grand rapids    
   4 -85.56647532 -85.56917328        49512 united states             kent                      michigan            grand rapids    
   5 -85.56787602 -85.57057398        49546 united states             kent                      michigan            grand rapids    
   6 -85.56787602 -85.57057398        49546 united states             kent                      michigan            grand rapids    
   7 -85.56723042 -85.56992838        49512 united states             kent                      michigan            grand rapids    
   8 -85.56866387 -85.57136183        49512 united states             kent                      michigan            grand rapids    
                                                                                                                                    
 Obs street                                  streetNo point_of_interest query                                                       
                                                                                                                                    
   1 28th street southeast                       5189        NA         5189 28th St Se, Grand Rapids, MI                           
   2 28th street southeast                       5155        NA         5155 28th St Se, Grand Rapids, MI                           
   3 28th street southeast                       5556        NA         5556 28th St Se, Grand Rapids, MI                           
   4 28th street southeast                       4045        NA         4045 28th St Se, Grand Rapids, MI                           
   5 east paris avenue southeast                 2305        NA         2305 E Paris Ave Se, Grand Rapids, MI                       
   6 east paris avenue southeast                 2305        NA         2305 E Paris Ave Se, Grand Rapids, MI                       
   7 28th street southeast                       3960        NA         3960 28th St Se, Grand Rapids, MI                           
   8 28th street southeast                       3927        NA         3927 28th St Se, Grand Rapids, MI                                                                                      The WPS System                     19:07 Thursday, November 14, 2013    2
                                                                                                                                    
 Obs          lon          lat type             loctype                                                                             
                                                                                                                                    
   9   -85.556224    42.946438 street_address   rooftop                                                                             
  10    -85.50019    42.915388 street_address   rooftop                                                                             
  11    -85.50019    42.915388 street_address   rooftop                                                                             
  12   -85.499809    42.913584 street_address   rooftop                                                                             
  13   -85.583252    42.916731 street_address   rooftop                                                                             
  14   -85.583252    42.916731 street_address   rooftop                                                                             
  15   -85.583243    42.916087 street_address   rooftop                                                                             
  16   -85.570236    42.947743 street_address   rooftop                                                                             
                                                                                                                                    
 Obs address                                                                                               north        south       
                                                                                                                                    
   9 4591 cascade road southeast, grand rapids, mi 49546, usa                                        42.94778698  42.94508902       
  10 6799 cascade road southeast, grand rapids, mi 49546, usa                                        42.91673698  42.91403902       
  11 6799 cascade road southeast, grand rapids, mi 49546, usa                                        42.91673698  42.91403902       
  12 6820 cascade road southeast, grand rapids, mi 49546, usa                                        42.91493298  42.91223502       
  13 2560 east beltline avenue southeast, centerpointe mall, grand rapids, mi 49546, usa             42.91807998  42.91538202       
  14 2560 east beltline avenue southeast, centerpointe mall, grand rapids, mi 49546, usa             42.91807998  42.91538202       
  15 2600 east beltline avenue southeast, centerpointe mall, grand rapids, mi 49546, usa             42.91743598  42.91473802       
  16 4018 cascade road southeast, grand rapids, mi 49546, usa                                        42.94909198  42.94639402       
                                                                                                                                    
 Obs         east         west  postal_code country         administrative_area_level_2 administrative_area_level_1 locality        
                                                                                                                                    
   9 -85.55487502 -85.55757298        49546 united states             kent                      michigan            grand rapids    
  10 -85.49884102 -85.50153898        49546 united states             kent                      michigan            grand rapids    
  11 -85.49884102 -85.50153898        49546 united states             kent                      michigan            grand rapids    
  12 -85.49846002 -85.50115798        49546 united states             kent                      michigan            grand rapids    
  13 -85.58190302 -85.58460098        49546 united states             kent                      michigan            grand rapids    
  14 -85.58190302 -85.58460098        49546 united states             kent                      michigan            grand rapids    
  15 -85.58189402 -85.58459198        49546 united states             kent                      michigan            grand rapids    
  16 -85.56888702 -85.57158498        49546 united states             kent                      michigan            grand rapids    
                                                                                                                                    
 Obs street                                  streetNo point_of_interest query                                                       
                                                                                                                                    
   9 cascade road southeast                      4591        NA         4591 Cascade Rd Se, Grand Rapids, MI                        
  10 cascade road southeast                      6799        NA         6799 Cascade Rd Se, Grand Rapids, MI                        
  11 cascade road southeast                      6799        NA         6799 Cascade Rd Se, Grand Rapids, MI                        
  12 cascade road southeast                      6820        NA         6820 Cascade Rd Se, Grand Rapids, MI                        
  13 east beltline avenue southeast              2560        NA         2560 E Beltline Ave Se, Grand Rapids, MI                    
  14 east beltline avenue southeast              2560        NA         2560 E Beltline Ave Se, Grand Rapids, MI                    
  15 east beltline avenue southeast              2600        NA         2600 E Beltline Ave Se, Grand Rapids, MI                    
  16 cascade road southeast                      4018        NA         4018 Cascade Rd Se, Grand Rapids, MI                                                                                   The WPS System                     19:07 Thursday, November 14, 2013    3
                                                                                                                                    
 Obs          lon          lat type             loctype                                                                             
                                                                                                                                    
  17  -85.5879549    42.912112 street_address   rooftop                                                                             
  18   -85.569238    42.948355 street_address   rooftop                                                                             
  19  -85.5499342   42.8836312 street_address   range_interpolated                                                                  
  20   -85.607639    42.912997 street_address   rooftop                                                                             
                                                                                                                                    
 Obs address                                                                                               north        south       
                                                                                                                                    
  17 3214 28th street southeast, grand rapids, mi 49512, usa                                         42.91346098  42.91076302       
  18 4033 cascade road southeast, grand rapids, mi 49546, usa                                        42.94970398  42.94700602       
  19 4665 44th street southeast, kentwood, mi 49512, usa                                             42.88497343  42.88227547       
  20 2411 28th street southeast, grand rapids, mi 49512, usa                                         42.91434598  42.91164802       
                                                                                                                                    
 Obs         east         west  postal_code country         administrative_area_level_2 administrative_area_level_1 locality        
                                                                                                                                    
  17 -85.58660592 -85.58930388        49512 united states             kent                      michigan            grand rapids    
  18 -85.56788902 -85.57058698        49546 united states             kent                      michigan            grand rapids    
  19 -85.54858527 -85.55128323        49512 united states             kent                      michigan            kentwood        
  20 -85.60629002 -85.60898798        49512 united states             kent                      michigan            grand rapids    
                                                                                                                                    
 Obs street                                  streetNo point_of_interest query                                                       
                                                                                                                                    
  17 28th street southeast                       3214        NA         3214 28th St Se, Grand Rapids, MI                           
  18 cascade road southeast                      4033        NA         4033 Cascade Rd Se, Grand Rapids, MI                        
  19 44th street southeast                       4665        NA         4665 44th St Se, Kentwood, MI                               
  20 28th street southeast                       2411        NA         2411 28th St Se, Grand Rapids, MI                           
                                                                                                                

About the author: Phil Rack is President of MineQuest Business Analytics, LLC located in Grand Rapids, Michigan. Phil has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and is a authorized reseller of WPS in North America.

How Important are Gecoding Services?

I’ve been thinking about writing a Geocoder for WPS for some time now and have elements of it done. One thing that strikes me as being problematic ,and I should have thought about this aspect earlier, is the issue of keeping address and map tables up-to-date. I’m not sure how often these tables would have to be up-dated but the amount of data is tremendous that would have to be downloaded on say a quarterly basis.

So, I’m now reconsidering the direction I took and think a better way is to simply allow the user (i.e. a WPS licensee holder who purchased their WPS license from MineQuest) to move their address data to a remote service for geocoding. There are a number of free and pay services offering geocoding and I don’t like the idea of paying for such a thing if it is at all avoidable.

Looking at the pay services, Bulk Geocoder has pricing on their website and they charge $500 for 100,000 records. I personally think that’s a substantial amount of money to pay. The free services limit the amount of data (i.e. the number of addresses) they will geocode for free but some of them are quite lenient. For example, Microsoft will allow a maximum of 200,000 addresses to be geocoded at a time. That’s pretty decent and I suspect that would handle the bulk requirements of many WPS license holders who need to geocode addresses.

Since this would be a bulk geocoding system, i.e. not processing a record interactively, a web service written for and using WPS to communicate with the geocoding organization would be very useful and quite economic. SAS likes to add a tax to companies who are Data Service Providers and many of these so called DSP’s are in the advertising and marketing industries. If they can realize a savings over using the SAS System, then this is something that we need to strongly consider offering.

There’s some time needed for further research and I have questions on turnaround time, but if the turnaround time is low, then the cost savings over using an outside geocoding service or if you are a SAS user and have to use SAS/GIS (think $$$) then this would be a very reasonable solution.

About the author: Phil Rack is President of MineQuest, LLC. and has been a SAS language developer for more than 25 years. MineQuest provides WPS and SAS consulting and contract programming services and a reseller of WPS in North America.