Tuesday, September 24

Google Map Javascript API and Fusion Table Layer


Introduction: Google Maps API is a collection of APIs that enable you to overlay your own data on a customized Google Map. Google Fusion Tables allow you to upload, store, share, search, and manage large structured data sets in the cloud. It can store up to 100 MB of data  per table, and each row in a table can have an associated location, line, or polygon feature.
The Google Maps API provides a FusionTablesLayer object that connects to these Fusion Tables and can automatically render the location data in a Fusion Table as well as display additional information about each feature through clickable overlay.
Fusion Tables also supports an SQL like query language, which you can use to filter the features shown on a map. The combination of Fusion Tables with the Maps API makes it easy to host large sets of data in the cloud, and visualize them in your Maps API application.


Steps to load data in Fusion Table and display data in Google Map:


(1) Get Sample Data: There are several formats of data can be used. I am using comma separated value (.csv) file. Any kind of data with address information can be used. My file contains addresses of Hospitals in Illinois.

(2) Log on to Google Fusion Table: Open a browser and go to
http://www.google.com/fusiontables. If you have Google Account,
use that to log on. If you don't have a Google account then you'll have to create a new account.



(3) Import Data: Once you logged in, you will see the “import New Table” screen. Select “From this Computer” and browse to select a data file from your machine. Click “next” button.


.



(4) Preview uploaded data: Your data will start to be uploaded. Click “Next” button.



(5) Set up Description: You can set up table description from final import screen. Click “Finish”.


(6)  Check Translated data : When the import has been finished, the rows which have been translated into map location will be highlighted with yellow.


(7)  Share data: In order to share this data with people or view this from webpage, the data needs to be shared. Click “Share” button and then set settings for sharing. Click “save”. 

The next screen will give you a link to share and html code to embed the map in webpage.
Here is the code:

<iframe width="500" height="300" scrolling="yes" frameborder="no" src="https://www.google.com/fusiontables/embedviz?viz=GVIZ&amp;t=TABLE&amp;q=select+col0%2C+col1%2C+col2%2C+col3%2C+col4%2C+col5%2C+col6+from+1RWovO01nPAv7UFqUjUzSVqhokdTHdMhZAMLupDk&amp;containerId=gviz_canvas"></iframe>


(8) Visualize data in map: Click the tab –‘Map of Hospital name’. The next screen will show the ‘Geocoding’ on progress status and when it is finished you will see Illinois map with the hospitals marked as red circles

(9) Visualize data using Google Map API and Fusion Table Layer:
You can also add data from fusion table to your application using Google Map API. You need the unique identifier for the fusion table.
Here is the sample code in javascript:


<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
<meta http-equiv="content-type" content="text/html; charset=UTF-8"/>
<title>Google Maps JavaScript API v3 Example: Fusion Tables Layer</title>
<link href="http://code.google.com/apis/maps/documentation/javascript/examples/default.css" rel="stylesheet" type="text/css" />
<script type="text/javascript" src="http://maps.googleapis.com/maps/api/js?sensor=false"></script>
<script type="text/javascript">

function initialize() {
var chicago = new google.maps.LatLng(41.850033, -87.6500523);
map = new google.maps.Map(document.getElementById('map_canvas'), {
center: chicago,
zoom: 11,
mapTypeId: 'roadmap'
});



layer = new google.maps.FusionTablesLayer({
      query: {
        select: 'Location',
        from: '1RWovO01nPAv7UFqUjUzSVqhokdTHdMhZAMLupDk'
      }

    });
    layer.setMap(map);
}

</script>
</head>
<body onload="initialize()">
<div id="map_canvas"></div>
</body>
</html>

The ‘query’ property of fusionTableLayerOptions object is ‘SQL’ query. The  select statement queries the ‘location’ field from ‘1RWovO01nPAv7UFqUjUzSVqhokdTHdMhZAMLupDk' which is the identifier of my fusion table.

Save this code as .html and open it in a browser to see the hospitals.It opens ‘Infoview window’ and displays stored information about the hospital when you click any hospital marker.


Monday, September 16

Thematic/Choropleth Map with ArcGIS


Introduction:   With Geographic Information System (GIS) we can visualize and analyze spatial data. Any data that includes information about location - be it a street address, zip code, longitude, latitude or X/Y coordinates - can be considered as spatial data. Here I’ll create a very simple thematic/Choropleth map with ArcGIS ArcMap 10.2. Trial version of ArcGIS  for desktop  can be downloaded from this location- http://www.esri.com/software/arcgis/arcgis-for-desktop/free-trial.

Steps:
Start   Program-> ArcGIS-ArcMap 10.2 – then open an new empty map.
Click the “plus”  sign on the standard toolbar to add Data. Alternatively we can add data from File menu and then selecting Add dat. When the “Add Dialog box” appears, navigate to the folder where data file/shape file is stored. We can also click on “Connect To Folder” () and navigate to data file. I have downloaded 2010 USA Population data  (State_2010Census_DP1.zip) from http://www.census.gov website and unzipped the files at c:\zip\CensusData\State2010Census.
After adding “State-2010Census_DB1.shp” file, it’ll appear as USA map with state boundaries as follows:

We can use Pan,Zoom-in, zoom-out  tools to move around the map. Data layer (state_2010census_DB1) is visible on table content pane. Right click on data layer and then select “open Attribute Table” to see the fields in this dataset. Here is how the table looks like:
We are interested in two fields -  DP0010001 (total Population ) and DP0160001 (Avg. Household Size).
 Let’s change the field name - Right click on data layer->properties->fields  and then select DP0010001 and change alias as ‘Total Population’. We’ll do the same for DP0160001 to name it ‘Avg. Household Size’.




Now we’ll use the field- ‘Total Population’ and make it a Choropleth map.  Let’s open Layer property page by  Right clicking on data layer->Properties->Symbology. On Symbology tab select ‘Total Population’ field from value dropdown and select 7 from classes drop down. Select Color Ramp of your choice. It should look like as screen below:

 To format the labels, click on label > format Labels -> Number Format-> highlight Numeric and check “Show Thousands Separator” Hit OK.
To display USA State Name, Right click on data later->properties->Labels and then select “STUSPS10”  from Label Field drop down and press apply. Again right click on data layer and then choose “Label Features”.

The map should look like this :
Now let’s add another layer for Household Size. Copy data layer (state_2010Census_DP1) and paste it on Layers. You’ll see two data layers (state_2010Census_DP1) under layers. To display “Avg. Household Size” data, right click on new data layer- and navigate to properties->symbology. On symbology tab select “Dot density” under quantities and select “Avg. Household Size”  under field selection.


It shows 1 dot is equivalent to 0.1. To add this layer on top of population layer that we built earlier, we need to make this layer transparent. To do this, right click on new data layer and then properties and then display – change “transparent” to 70%.

The map will show both layers when check both data layers.

To export the map, click view and select layout view.  From “insert” menu select “Title” and “legend”.
Here is the screenshot of final map.


.

Thursday, September 12

Dynamic Calendar Using Javascript And HTML

Introduction : In this post I'll submit Java script code to create a dynamic calendar. Users have the ability of selecting year or month.

/* code*/

<HTML>
<HEAD>
<TITLE>Calendar</TITLE>
<STYLE TYPE="text/css">
TD, TH {text-align:center}
</STYLE>
<SCRIPT LANGUAGE="JavaScript">

// day of week of month's first day
function getFirstDay(theYear, theMonth){
    var firstDate = new Date(theYear,theMonth,1)
    return firstDate.getDay()
}

// # of days in the month
function getMonthLen(theYear, theMonth) {
    var oneDay = 1000 * 60 * 60 * 24
    var thisMonth = new Date(theYear, theMonth, 1)
    var nextMonth = new Date(theYear, theMonth + 1, 1)
    var numdays = Math.ceil((nextMonth.getTime() -
        thisMonth.getTime())/oneDay)
    return numdays
}


// Months Array
var theMonths = ["January","February","March","April","May","June","July","August",
"September","October","November","December"]

// return IE4+ or W3C DOM reference for an ID
function getObject(obj) {
    var theObj
    if (document.all) {
        if (typeof obj == "string") {
            return document.all(obj)
        } else {
            return obj.style
        }
    }
    if (document.getElementById) {
        if (typeof obj == "string") {
            return document.getElementById(obj)
        } else {
            return obj.style
        }
    }
    return null
}

/************************
  Calendar
*************************/

// clear and re-populate table based on form's selections
function populateTable(form) {
    var theMonth = form.chooseMonth.selectedIndex
    var theYear = parseInt(form.chooseYear.options[form.chooseYear.selectedIndex].text)

    // initialize date-dependent variables
    var firstDay = getFirstDay(theYear, theMonth)
    var howMany = getMonthLen(theYear, theMonth)
   
    // fill in month/year in table header
    getObject("tableHeader").innerHTML = theMonths[theMonth] +
    " " + theYear
   
    // initialize vars for table creation
    var dayCounter = 1
    var TBody = getObject("tableBody")
    // clear any existing rows
    while (TBody.rows.length > 0) {
        TBody.deleteRow(0)
    }
    var newR, newC
    var done=false
    while (!done) {
        // create new row at end
        newR = TBody.insertRow(TBody.rows.length)
        for (var i = 0; i < 7; i++) {
            // create new cell at end of row
            newC = newR.insertCell(newR.cells.length)
            if (TBody.rows.length == 1 && i < firstDay) {
                // no content for boxes before first day
                newC.innerHTML = ""  
                continue
            }
            if (dayCounter == howMany) {
                // no more rows after this one
                done = true
            }
            // plug in date (or empty for boxes after last day)
            newC.innerHTML = (dayCounter <= howMany) ?
                dayCounter++ : ""
        }
       
    }
}

/*******************
  year 10 yrs from current year
********************/
// select year
function fillYears() {
    var today = new Date()
    var thisYear = today.getFullYear()
    var yearChooser = document.dateChooser.chooseYear
    for (i = thisYear; i < thisYear + 10; i++) {
        yearChooser.options[yearChooser.options.length] = new Option(i, i)
    }
    setCurrMonth(today)
}
// select month

function setCurrMonth(today) {
    document.dateChooser.chooseMonth.selectedIndex = today.getMonth()
}
</SCRIPT>
</HEAD>

<BODY align=center onLoad="fillYears(); populateTable(document.dateChooser)">
<H1>Calendar</H1>
<HR>
<TABLE ID="calendarTable" BORDER=1 ALIGN="center">
<TR>
    <TH ID="tableHeader" COLSPAN=7 bgcolor=green></TH>
</TR>
<TR bgcolor=lightblue><TH>Sun</TH><TH>Mon</TH><TH>Tue</TH><TH>Wed</TH>
<TH>Thu</TH><TH>Fri</TH><TH>Sat</TH></TR>
<TBODY ID="tableBody"></TBODY>
<TR>
    <TD COLSPAN=7>
    <P>
    <FORM NAME="dateChooser">
        <SELECT NAME="chooseMonth"
        onChange="populateTable(this.form)">
            <OPTION SELECTED>January<OPTION>February
            <OPTION>March<OPTION>April<OPTION>May
            <OPTION>June<OPTION>July<OPTION>August
            <OPTION>September<OPTION>October
            <OPTION>November<OPTION>December
    </SELECT>
    <SELECT NAME="chooseYear" onChange="populateTable(this.form)">
    </SELECT>
    </FORM>
    </P></TD>
</TR>
</TABLE>
</BODY>
</HTML>
         
Output:

       
 

Wednesday, September 11

Import and Process XML Data into SQL Server Database



Introduction:  How to import an xml file to SQL Database and how to parse the data ? In my last post I
demonstrated how to convert XML file to CSV file. Once we have a CSV file we can import data either by Sql Server management studio import task or by SSIS.  Here I will demonstrate a different approach in following two steps.

(1)Import XML data to SQL Database using OPENROWSET function
(2)Parse data using OPENXML function.

 Import XML Data to SQL Server using OPENROWSET:

Steps:

(A) Get a XML File and save it.  Here is a dummy xml file (blSurvey.xml) that I am going to import to SQL Database.

<?xml version="1.0" encoding="iso-8859-1" ?>
<HEADER>
<SURVEYDATA>
       <SURVEY_ID>1234567</SURVEY_ID>
       <CLIENT_ID>6666</CLIENT_ID>
       <SERVICE>MD</SERVICE>
       <RECDATE>2013-02-18</RECDATE>
       <DISDATE>2012-02-13</DISDATE>
       <ANALYSIS>
              <RESPONSE>
                     <VARNAME>A1</VARNAME>
                     <VALUE>4</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>A14</VARNAME>
                     <VALUE>5</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>CP1</VARNAME>
                     <VALUE>5</VALUE>
              </RESPONSE>
       </ANALYSIS>
       <DEMOGRAPHICS>
              <RESPONSE>
                     <VARNAME>AGE</VARNAME>
                     <VALUE>56</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>BARCODE</VARNAME>
                     <VALUE>1</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>COMPLETE</VARNAME>
                     <VALUE>YES</VALUE>
              </RESPONSE>
       </DEMOGRAPHICS>
       <HCAHPS>
              <RESPONSE>
                     <VARNAME>CG_1</VARNAME>
                     <VALUE>Yes</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>CG_11</VARNAME>
                     <VALUE>No</VALUE>
              </RESPONSE>
       </HCAHPS>
</SURVEYDATA>
<SURVEYDATA>
       <SURVEY_ID>678910</SURVEY_ID>
       <CLIENT_ID>5555</CLIENT_ID>
       <SERVICE>MD</SERVICE>
       <RECDATE>2013-01-18</RECDATE>
       <DISDATE>2012-01-12</DISDATE>
       <ANALYSIS>
              <RESPONSE>
                     <VARNAME>A1</VARNAME>
                     <VALUE>5</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>A14</VARNAME>
                     <VALUE>5</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>CP1</VARNAME>
                     <VALUE>5</VALUE>
              </RESPONSE>
       </ANALYSIS>
       <DEMOGRAPHICS>
              <RESPONSE>
                     <VARNAME>AGE</VARNAME>
                     <VALUE>44</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>BARCODE</VARNAME>
                     <VALUE>1</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>DISTRIB</VARNAME>
                     <VALUE>Paper</VALUE>
              </RESPONSE>
       </DEMOGRAPHICS>
       <HCAHPS>
              <RESPONSE>
                     <VARNAME>CG_1</VARNAME>
                     <VALUE>Yes</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>CG_11</VARNAME>
                     <VALUE>No</VALUE>
              </RESPONSE>
              <RESPONSE>
                     <VARNAME>CG_13</VARNAME>
                     <VALUE>Always</VALUE>
              </RESPONSE>
       </HCAHPS>
</SURVEYDATA>
</HEADER>

 (B) Create a table and use OPENROWSET method to load the data. Here is the script.
I saved the above file as blSurvey.xml in my c drive. Change your file name and location.
I created a table called tmp_surveydata and imported blSurvey.xml as XML data type.

--create table--
CREATE TABLE tmp_surveydata
(
Id INT IDENTITY PRIMARY KEY,
XMLData XML, --load data as XML data type
LoadedDateTime DATETIME
)
--Load data
INSERT INTO tmp_surveyData(XMLData, LoadedDateTime)
SELECT CONVERT(XML, BulkColumn) AS BulkColumn, GETDATE()
FROM OPENROWSET(BULK 'C:\blSurvey.xml', SINGLE_BLOB) AS x;

--check data
SELECT * FROM tmp_surveydata

Querying the table will show the output as  hyperlink. Data is loaded as XMLdata type.





When I click the hyperlink, the data is displayed as following:

 HEADER>
  <SURVEYDATA>
    <SURVEY_ID>1234567</SURVEY_ID>
    <CLIENT_ID>6666</CLIENT_ID>
    <SERVICE>MD</SERVICE>
    <RECDATE>2013-02-18</RECDATE>
    <DISDATE>2012-02-13</DISDATE>
    <ANALYSIS>
      <RESPONSE>
        <VARNAME>A1</VARNAME>
        <VALUE>4</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>A14</VARNAME>
        <VALUE>5</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>CP1</VARNAME>
        <VALUE>5</VALUE>
      </RESPONSE>
    </ANALYSIS>
    <DEMOGRAPHICS>
      <RESPONSE>
        <VARNAME>AGE</VARNAME>
        <VALUE>56</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>BARCODE</VARNAME>
        <VALUE>1</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>COMPLETE</VARNAME>
        <VALUE>YES</VALUE>
      </RESPONSE>
    </DEMOGRAPHICS>
    <HCAHPS>
      <RESPONSE>
        <VARNAME>CG_1</VARNAME>
        <VALUE>Yes</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>CG_11</VARNAME>
        <VALUE>No</VALUE>
      </RESPONSE>
    </HCAHPS>
  </SURVEYDATA>
  <SURVEYDATA>
    <SURVEY_ID>678910</SURVEY_ID>
    <CLIENT_ID>5555</CLIENT_ID>
    <SERVICE>MD</SERVICE>
    <RECDATE>2013-01-18</RECDATE>
    <DISDATE>2012-01-12</DISDATE>
    <ANALYSIS>
      <RESPONSE>
        <VARNAME>A1</VARNAME>
        <VALUE>5</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>A14</VARNAME>
        <VALUE>5</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>CP1</VARNAME>
        <VALUE>5</VALUE>
      </RESPONSE>
    </ANALYSIS>
    <DEMOGRAPHICS>
      <RESPONSE>
        <VARNAME>AGE</VARNAME>
        <VALUE>44</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>BARCODE</VARNAME>
        <VALUE>1</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>DISTRIB</VARNAME>
        <VALUE>Paper</VALUE>
      </RESPONSE>
    </DEMOGRAPHICS>
    <HCAHPS>
      <RESPONSE>
        <VARNAME>CG_1</VARNAME>
        <VALUE>Yes</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>CG_11</VARNAME>
        <VALUE>No</VALUE>
      </RESPONSE>
      <RESPONSE>
        <VARNAME>CG_13</VARNAME>
        <VALUE>Always</VALUE>
      </RESPONSE>
    </HCAHPS>
  </SURVEYDATA>
</HEADER>

Now, data has been loaded into SQL table as XML Data type. It’s time to process the data.
(2) Parse XML data using OPENXML and sp_xml_preparedocument: The following script is used to parse the XML document.

DECLARE @XML AS XML, @Doc AS INT, @SQL NVARCHAR (MAX)

SELECT @XML = XMLData FROM tmp_Surveydata

EXEC sp_xml_preparedocument @Doc OUTPUT, @XML

select SURVEY_ID,CLIENT_ID,SERVICE,RECDATE,DISDATE,ANALYSIS_VARNAME,ANALYSIS_VALUE,DEMOGRAPHICS_VARNAME,DEMOGRAPHICS_VALUE,HCAHPS_VARNAME,HCAHPS_VALUE
from OPENXML(@Doc,'HEADER/SURVEYDATA/ANALYSIS/RESPONSE')
WITH (
SURVEY_ID [varchar] (50) '../../SURVEY_ID',
CLIENT_ID [varchar] (50) '../../CLIENT_ID',
SERVICE [varchar](50) '../../SERVICE',
RECDATE [varchar](50) '../../RECDATE',
DISDATE [varchar](50) '../../DISDATE',
ANALYSIS_VARNAME [varchar](50) 'VARNAME',
ANALYSIS_VALUE [varchar](50) 'VALUE',
DEMOGRAPHICS_VARNAME [varchar](50) '../../DEMOGRAPHICS/RESPONSE/VARNAME',
DEMOGRAPHICS_VALUE [varchar](50) '../../DEMOGRAPHICS/RESPONSE/VALUE',
HCAHPS_VARNAME [varchar](50) '../../HCAHPS/RESPONSE/VARNAME',
HCAHPS_VALUE [varchar](50) '../../HCAHPS/RESPONSE/VALUE')

exec sp_xml_removedocument @Doc



The stored procedure sp_xml_predpareddocument prepare the xml document and retuns an integer value. It is stored in internal cache.The OPENXML function takes parameters – (a) Internal representation of an XML document created by calling sp_xml_preparedocument (b) Xpath pattern to identify the nodes in XML document.
To retrieve all survey data,I started from RESPONSE node under Analysis( HEADER/SURVEYDATA/ANALYSIS/RESPONSE) and getting VARNAME and VALUE directly from this node. To get  SURVEY_ID or CLIENT_ID, I need to go two levels up to reach SURVEYDATA node and I am using ../../ to reach SURVEYDATA node.
../ is used to reach parent node and ../../ is used to reach grand parent node.
To get Demographics information,I am going to SurveyData node from Analysis/Response node by using ../../and then navigating to DEMOGRAPHICS/RESPONSE node.


The output:




Finally, I called sp_xml_removedocument stored procedure to free the internal cache.

If we like to insert this data into a SQL table in this format we can simply change the query as follows:

SELECT @XML = XMLData FROM tmp_Surveydata

EXEC sp_xml_preparedocument @Doc OUTPUT, @XML

select SURVEY_ID,CLIENT_ID,SERVICE,RECDATE,DISDATE,ANALYSIS_VARNAME,ANALYSIS_VALUE,DEMOGRAPHICS_VARNAME,DEMOGRAPHICS_VALUE,HCAHPS_VARNAME,HCAHPS_VALUE into tableName
from OPENXML(@Doc,'HEADER/SURVEYDATA/ANALYSIS/RESPONSE')
WITH (
SURVEY_ID [varchar] (50) '../../SURVEY_ID',
CLIENT_ID [varchar] (50) '../../CLIENT_ID',
SERVICE [varchar](50) '../../SERVICE',
RECDATE [varchar](50) '../../RECDATE',
DISDATE [varchar](50) '../../DISDATE',
ANALYSIS_VARNAME [varchar](50) 'VARNAME',
ANALYSIS_VALUE [varchar](50) 'VALUE',
DEMOGRAPHICS_VARNAME [varchar](50) '../../DEMOGRAPHICS/RESPONSE/VARNAME',
DEMOGRAPHICS_VALUE [varchar](50) '../../DEMOGRAPHICS/RESPONSE/VALUE',
HCAHPS_VARNAME [varchar](50) '../../HCAHPS/RESPONSE/VARNAME',
HCAHPS_VALUE [varchar](50) '../../HCAHPS/RESPONSE/VALUE')