Introduction
The article describes "IoT on Azure" [1] real-time bus tracking web application "enRoute" (hereinafter - web app) intended primarily for New York City commuters and respective MTA Agencies [2,3]. It allows real-time tracking of any of 316 MTA bus routes on any mobile or stationary Internet-enabled computer hardware platform (OS independent).
Technical merits
Web app allows continious real-time MTA bus monitoring via dynamically updated UI comprised of two data visualization controls: bus stop list showing vehicles status and interactive bus map showing the bus route with accurate vehicles locations. It can operate in either auto-update or manual modes. Aesthetically-rich and intuitive UI implements a set of innovative styling solutions (HTML5/CSS3) adapted for high-end mobile devices with big screen size/high resolution, and for compact form-factor devices with small-screen size, like smartphones.
This IoT solution utilizes novell capabilities of real-time mass transit data feeds (GTFS, SIRI, OBA [5-7]) and technical-enabling vectors of Microsoft Azure cloud services. It can be categorized as a real-time "SCADA over Internet" distributed system with primary focus on monitoring (supervisory) and data acquisition functionality.
Broader impact
Daily ridership pertinent to Metropolitan Transportation Authority (MTA) in New York City exceeds 10 million [2,3] (almost 2x the entire population of Norway). Proposed solution is addressing rather important use-case of high socio-economic significance, providing valuable data service to mass-transit comutters in the NY City. This particular web app is customized for a specific purpose of real-time MTA NY Bus monitoring, but its core solutions can be applied to other mass-transit domains, like: NY MTA Subway, LIRR and MNR.
Audiences
Individual commuters are expected to form a major and rather substantial User group. NY City governmental agencies and some infrastructure services may utilize the capabilities provided by this solution. Also, this web app could be of interest to the variety of businesses headquartered in the New York City.
Browser/platform compatibility
Web application is capable of running on any computer hardware platform within major web browser, i.e. FireFox, Chrome, Safari or Opera (note: certain compatibility issues exist while running on IE).
Complementary Windows app "Perpetuum-M"
This web app is accompanied by "Perpetuum-M" Windows counterpart application capable of running on any PC hardware platform equipped with Win 7/8. Win app will potentially provide much wider coverage in terms of its static/dynamic data feeds and mass-transportation vehicle types, like: NY MTA Subway, LIRR and MNR (refer to another contest submission article [9, 20] containing a demo app).
Useful tips for New York City commuters using this app
"Wait for the Second Bus" DrABELL (the Author) rule: If you see on the screen next two (or more) buses "coming in bunches", i.e. one closely following the other(s), then skip the first one (which most likely will come well-packed) and wait for the secdond one, which may arrive almost empty with plenty of seats available. Thus, just couple extra minutes of wait time may transform into the huge advantage of comfortably seating for the entire bus trip.
Demo
Fully-operational prototype of this web app has been developed/deployed to Azure cloud [16] and is currently available for testing/evaluation. It contains two different web pages specifically adapted for the following form-factors:
- Big-screen devices w/resolution >1024x600 (Tablet/Notebook/Desktop PC): link
- Compact form-factor devices w/resolution <1024x600 (e.g. Smartphones): link
Sample screenshot images are shown in the following Fig 1 and Fig 2, corresponding to the popular M1 bus route in Manhattan, NY City. Click on either the links above, or the following demo screenshot images to open the web page and start real-time tracking any of 316 MTA Bus Routes (made of 15,358 Bus Stops) existing in the New York City.
Fig 1a. NY City real-time bus map web app tracking popular M1 bus route (big screen demo snapshot)
Thie bus map to the right of the screen implements "Find Point of Interest" feature shown in following demo screenshot (Fig 1a). Pushpins are dynamically added to the map pointing to selected item "caffe" locations nearby. User can select many popular POI item from drop-down list or just type in the search text box and click on search button.
Fig 1b. NY City real-time bus map: tracking route M1 and displaying POI nearby (big screen demosnapshot)
Second "mobile" web page is adapted for compact mobile devices with screen resolution less than 1024x600 . It does not contain virtual map: instead, the dynamically updated route stop/bus list will adjust itself to fit the width of the screen (see sample screenshot below):
Fig. 2. NY City real-time bus tracking web page for small-screen mobile devices (e.g. iPhone), Demo screenshot
Key Operations
Web app has rather intuitive and highly-responsive UI. User can select any MTA Bus Route from the DropDown list and within couple seconds the screen will be updated showing the Buses and Stops in a real-time mode. The Button with UpDownArrow icon ( ↓↑ )allows to toggle the observable Bus Route directions ("To" or "From"). The Auto-update timer can be turned ON/OFF upon User's discretion - refer to the checkbox labeled "AUTO". The auto-update timer is set to 20 sec in this contest demo version (will be increased to 30 sec in a production), but User can force the screen update at any time by clicking on "Refresh" Button labeled w/incomplete circle icon (↻).
Browsers compatibility
This web app is fully-compatible with all major web browsers, namely: Mozilla FireFox (v.36+ recommended), Google Chrome, Apple Safari and Opera latest versions. It's partially compatible with Internet Explorer (styling in IE running this web app may be not consistent with CSS3 spec).
Azure IoT app latency
Exact timing is really hard to measure as too many Internet protocol uncertaintices are factored in. Some preliminary testing on high-speed Internet connection gave performance (latency) estimates of 1...3 sec, which is within acceptable range for this type of IoT apps. In the current demo web app the auto-update interval is set to 20 sec (will be 30 sec in production), so plenty of time margin exists.
Web-query customization
Web app allows selection of particular bus route and direction on page load using web query 'bus' and 'dir' parameters like shown below:
- ?bus=M1 (web query param corresponds to Manhattan Bus Route M1 to display on page load)
- ?bus=q60 (web query param corresponds to Bus Route Q60 to display on page load)
- ?bus=q60&dir=0 (corresponds to Bus Route Q60 headed to Queens)
- ?bus=Q60&dir=1 (corresponds to Bus Route Q60 headed to Manhattan)
Background
Internet of Things (IoT) [1] is considered a relatively new technical trend, though many of its core concepts have been known for decades. Apparent similarity can be noticed with existing distributed SCADA systems [4], in particular, telemetry systems, wireless sensors networks, distributed control and automation systems. The core differences are rooted in packet-switching communication protocols (TCP/IP and respectively, HTTP) used by IoT instead of circuit/channel switching techniques typical for traditional SCADA systems. IoT is a relatively new semantic arrival, which did not become a "household name" yet: thus, the following subchapter "IoT in a Nutshell" is added in order to ensure a definitive clarity and also providing some basic body of knowledge pertinent to this subject domain.
IoT in a Nutshell
What is IoT?
IoT is a distributed SCADA (i.e. Supervisory Control and Data Acquisition) built on Internet-related packet-switching protocols (most notably TCP/IP and HTTP).
How IoT is different from traditional Internet?
As stated above, IoT is adding more SCADA functionality/use-cases to the traditional Internet, which can be categorized as mostly "Internet of People". IoT should be considered an extension to the traditional Internet, even though many use-cases pertinent to distributed SCADA systems existed long before the term IoT (and even Interner per se) was coined. Important difference lies in the fact that in traditional Internet the receiving part of data/signals is primarily a human being, while IoT is adding more use-cases pertinent to direct machine-to-machine interactions and machine-learning.
How IoT is different from traditional SCADA?
Data/Signal transport mechanism is an important classification criteria. The difference between packet-switching networks and circuit-switching ones is goind much deeper than just semantics: by analogy, look at the difference between wired and wireless networks, which is quite substantial to be considered the two different sub-classes in networking taxonomy. From design/analytical perspectives, it's seems reasonable to place IoT in a separate category, different from traditional SCADA.
What type of IoT projects could we expect to be successful in the near future?
Mostly real-time monitoring and supervision apps in many non-critical areas, e.g. variety of hobby/educational projects like monitoring water level in a fish tank, or vending machine inventory, or finding free parking spots in big Cities, and, of course, in a kind of critical area of home security surveillance systems. Regarding non-realtime apps dealing with "historical" data - it could be variety of meteorological/environmental statistical apps, crowdsourcing data/signal aggegation apps (e.g. monitoring City noise levels and other factors affecting quality of life)
What body of knowledge/set of skills required for IoT development?
Serious IoT projects of significant practical/commercial value, going beyond just some metaphorically-trivial "Arduino driving LED" or "Internet keeps my coffee hot" sorta of (pretty much like that notoriously-omnipresent and widely-overhyped Timer-555 hobby stuff of the 70s) require the combination of IT, Software/Firmware development, Electrical Engineering and Electronic Design knowledge/skills, plus some working knowledge of Metrology (i.e. Instrumentation and Measurement) and, possibly, Control Engineering. IoT app developer should be thinking like Electrical and Control Engineers in addition to their general programming mind set. In particular, any IoT project developer must give a serious consideration to a relatively high cost of failure/error in the real life "material" world of sensors and actuators (i.e. control devices), and be exremely cautious about any IoT projects that may cause poterntially-serious safety/health hazard.
What type of open source data feeds pertinent to mass transportation currently exists?
Pertinent to mass-transit domain: vehicles and stops real-time monitoring, for e.g. bus routes and stops monitoring as described in this project article. More than 5000 MTA buses in the NYC [18] serving 316 routes are equipped with GPS system providing a real-time vehicle monitoring capability. Bus stops location can be obtained from a "static" GTFS data feed, or in a realtime using SIRI/OBA data feeds [2-7]. Another set of search queries allow finding a nearest stops/bus routes from the GPS reading on the User's mobile computing device (in other words, based on geo-location service on client's computer if enabled).
Are IoT apps pertain just to Cloud technology domain?
No, IoT can be implemented practically on any web hosting servers, but cloud technology in general (an Microsoft Azure in particular) provides a convenient project integration tools, for example, easy publishing from development platform (like Visual Studio 2012/2013) to Azure web hosting servers.
What IoT modeling tools exist?
Regarding the software development part, IoT is a good fit for existing Unified Modeling Language (UML) with a focus on Entity-relationship, State Machine and Activity diagram. Pertinent to a subject domain (i.e. sensors, actuators, and electronic circuitry in general) other computer modeling/simulation tools could be handy: for example digital filter design software (in the world of signals and noise it's not all about moving averages, but also much more sophisticated FIR/IIR filtering technique), circuit simulation software like SPICE, variety of circuit schematic capture/PCB layout software, etc.
What about Azure domain name customization, e.g. busny.info instead of busny.azurewebsites.net?
Entire procedure of domain customization on Azure is simple, straightforward and well documented [19]. Make appropriate modification of domain DNS A Record and CNAME Record to point at Azure web site IP address and then use "Azure Manage custom domains" feature to add domain names. Note: make sure that all 3 essential records appear (pertinent to this web app, it would be: busny.net, www.busny.net and busny.azurewebsites.net).
What could be the most significant socio-economic implications of the IoT proliferation?
Cost reduction is expected to be a major driving factor behind IoT proliferation. Going beyond just economic metrics, IoT could serve both practical and didactic aspects of our social life. It can invigorate the interest to various engineering disciplines and activities. For example, even such trivial sample hobby project as fish tank water level monitoring could extend the knowledge base of software developers to another subjects domain (Metrology, i.e. Instrumentation and Measurement), so they may learn for a first time about physical principles of measuring the hydrostatic pressure in order to calculate the water level. And so on... The educational aspect of IoT proliferation is quite important.
What are the most significant Pros and Cons of IoT apps?
The most critical factors pertinent to the IoT proliferation are listed below:
| IoT Pros | IoT Cons |
- Cost-efficiency
- Scalability
- Accessibility
- Flexibility
- Easy/Low Maintenace
|
- Security and Safety concerns
- Privacy concerns
- Operational reliability risk factors
- Low quality of general app software
- Http Signal Propagation/Timing uncertainty
|
Following sub-chapter provides elaborated risk/reward analysis.
IoT competitive advantages
Almost universal accessibility of the Internet and relatively low cost of associated communication equipment/services are major forces behind rapidly increasing interest to IoT. Cost-effectiveness of IoT solutions is a major competitive advantage in comparison with traditional distributed SCADA systems utlizing circuit/channel switching. Beyond pure economic advantages, IoT solutions provides the almsot unparalleled scalability and flexibility accompanied by relatively short developemnt cycle due to existence of many RAD software tools. Democratic nature and "inclusiveness" of the underlying web technologies provide tremendous opportunity for creating a variety of SCADA applications with unparallel level of "crowdsourcing" data/signal aggregation. Pertinent to this particular solution, it can be a real-time feedback sent by mass transit commutters corresponding to vechicle occupancy, stops conditions, availability of free wi-fi hot spots at bus (or any other mass-transit vehicles) stops, etc.
IoT Success Stories
The operative word in the majority of currently existing IoT success stories so far seems to be a "real-time" and "monitoring". Quite notably many commercially successful IoT apps pertain to the real-time remote home monitoring/Internet security surveillance systems. For the sake of objectivity, it's relevant to notice that this type of apps made its debut long before the term "IoT" was coined, so it's a sort of new semantic re-arrangement within existing high-tech taxonomy. But going beyond just terminological aspects, the aforementioned class of IoT apps represents rather promising technical trend. Others apps of high technical and societal significance are: real-time weather data and road traffic monitoring systems, and, essentially, any mass-transit supervisory/data acquisition apps (like this one) providing valuable input to the commuters in a real-time mode.
Note: for the time being, some technology enthusiasts start field testing of the software (on the Bus Route Q28 and highly-popular in Queens Q60) and sent me some positive feedback regarding the accuracy and good overall UI of the web app (I'm also gonna start my own road tests as soon as I finish this article and all residual software updates :)
IoT risk factors
Safety and Security concerns
So far, security and safety concerns outweigh all other IoT operational risks. Internet in its current state is rather vulnerable to malicious cyber-attacks with many security/privacy breaches, info-leaks and financial fraud cases frequently decorating the news headlines worlwide. In addition to the “virtual digital damage”, IoT may cause serious direct physical, thus, much more dangerous damages to the connected devices and, potentially, to the Users. In the light of such existing cyber-security concerns, for the time being IoT probably shall not be used in any life-support or mission-critical applications/systems.
IoT operational reliability
Beyond safety and security factors, another major set of concerns relates to IoT operational reliability. As a general rule, any IoT apps with core operations depending on accurate timing (i.e. “data/signal transport delay”), in particular, any closed-loop automatic control systems where stability concerns are utmost important, should be considered a potentially dangerous solutions and should not be recommended for public use.
IoT quality control
IoT quality control is extremely important. For the IoT to be a successful/significant technology trend rather than "just another piece of techno-jargon" its software apps must be a subject to the same strict quality standards as, for example, applied to the hardware/firmware in embedded applications. IoT software developer should adhere to the way of thinking typical for most engineering disciplines, e.g. Electrical/Control Engineering: the latter is very focused on the quality and reliability of their products, mainly because the cost of the engineering errors in the real life world is typically much higher than in artificial "digital realities" of traditional Internet.
System Architecture and Development process
This web app and complementary win app [20] both adhere to the fundamental concept of separation of programming concepts, i.e. Data Layer, Business Intelligence (BI) and User Interface (GUI) as shown in the following generalized Architectural diagram (see Fig. 3a).
Fig. 3a. Generalized Architectural diagram (covers both Web/Win App)
Second diagram (see Fig, 3b) shows components that pertain to this particular web app. Presentation layer utilizes advanced HTML5/CSS3 and Azure AJAX partial page rendering technique for better app responsiveness and lower data traffic.
Fig. 3b. Scaled-up Web app solution (includes potential enhancements)
Technology surface area
This web app has been developed using primarily a Microsoft technology set: .NET4.5, C#, HTML5/CSS3 and AJAX. HTTP protocol is used for data communication. For the future development, Azure WebSockets and SignalR are considered as a viable alternative. Bing Map search functionality utilized in the current version could be potentially replaced with Azure search service.
Development tools
This web app utilizes Microsoft Azure cloud services and real-time data feeds provided by NYC MTA API for the developers (the latter requires a valid API access key [6,7]). The core development was completed using Visual Studio 2013 Express edition, later upgraded to Studio 2013 Community edition with Update 4 (free for small development teams).
.NET target framework
Target framework for this web app is set to the latest .NET 4.5 (a default for web project compiled by Visual Studio 2013 community edition).
Data Feeds
This app requires several web requests to be sent to correponding SIRI/OBA end points as detailed in the spec [6,7]. Response data stream could be formatted either as JSON or XML document. The solution implements a proprietary parser, which deserializes the downward data stream into corresponding software object using .NET/C#.
Static Data Feeds
GTFS
General Transit Feed Specification (GTFS, originally: Google Transit Feed Specification) defines a common format for public transportation schedules and associated geographic information [5]. A GTFS feed is a collection of CSV files encapsulated into a single ZIP file (main list follows):
- agency.txt
- routes.txt
- trips.txt
- stop_times.txt
- stops.txt
- calendar.txt
Dedicated data import procedure parses and inserts data from GTFS static feed into application Databases (SQL Servers): the most important records in a context of this web app and complementary win app [20] resides in agency, routes and stops tables.
Realtime Data Feeds
OBA
The open source OneBusAway (OBA) API allows "discovering" the static/baseline information about bus services, in particular, covered under NYC MTA [7].
SIRI
Service Interface for Real Time Information (SIRI) is an XML protocol allowing distributed computers to exchange real-time information about public transport services and vehicles [6].
Parser
The parser module de-serialize the OBA/SIRI "stops-routes-agency" Xml encoded data streams and transforms them into corresponding software objects to be consumed by this Web app and complementary Win app [20]. It's relevant to mention that Agencies and Stops data could be also obtained from the "static" GFTS data feeds and stored in application database, thus eliminating this particular data request from real-time update cycles. The final data set contains Stops static data with superimposed real-time Buses position data.
Microsoft AJAX
High responsivenes of the web app is achieved via AJAX partial web page rendering. For the future development, Azure WebSockets and SignalR are considered as a viable alternative to AJAX technology.
User Settings Data Persistence
The app implements session variables and permanent cookies for better page customization; data on selected Bus Route and update mode persists between the session. Note: web app preserves its core functionality without cookies (in case they may be disabled on the client's computer).
Azure Geospatial Search Engine
Current web application utilizes intrinsic geospatial search capabilities of Microsoft Bing Map technology, allowing the User to find points of interest nearby the bus routes (or in any User-selected area on the map). Future development may include Azure search services to enhance this functionality.
Web API
In addition to this web application, the entire solution will include two sets of Web API related to:
- Real-time data feed for complementary WinApp on PC "Perpetuum-M" [20]
- Feedback channel for gathering the "crowsdourcing" data
Feedback channel
In addition to the downstreams provided by OBA/SIRI, the upstream functionality, i.e a crowdsourcing data channel may enhance the data capability of the solution. Currently under consideration are several upstream data feeds providing the following metrics: bus occupancy and bus awaiting population at stops. In addition to this, a set of unique proprietary metrics reflecting energy efficiency, commuting safery and comfort [10] could be added to the upstream data channel in a complementary WinApp "Perpetuum-M" [20] utilizing specific technical enabling vectors of modern "Wintel" PC platforms.
IoT technical-enabling vectors of modern "Wintel" PC
The aforementioned "upstream" (feedback) part of the solution may utilize various technical-enabling vectors of modern "Intel Inside" PC platforms, in particular, integrated sensors: GPS, Axelerometers, Position and Orientation Sensors (optionally - electronic compass). Compementary Win app [20] will implement bi-directional functionality: consuming vehicle/stops location downstreams and sending a "crowdsourcing" upstream data set back to the server. Such upstream data feedback may include multiple scalar metrics based on the embedded sensors' reading (PC Tablets/Ultrabooks) and proprietary algorithms: Ergometric Driving Efficiency Ratio (EDER), Acceleration Conformance Ratio (ACR), Dynametric Driving Comfort Ratio (DDCR ), Dynametric Driving Risk Ratio (DDRR) developed by Author [10].
Using the code
Typical Bus Route and Bus Stop Xml data structure is shown below (see Listing 1, 2):
Listing 1. NY MTA Bus Route element, XML-encoded
<route>
<id>MTA NYCT_M1</id>
<shortName>M1</shortName>
<longName>Harlem - East Village</longName>
<description>via 5th Av / Madison Av</description>
<type>3</type>
<url>http://web.mta.info/nyct/bus/schedule/manh/m001cur.pdf</url>
<color>EE352E</color>
<textColor>FFFFFF</textColor>
<agencyId>MTA NYCT</agencyId>
</route>
Listing 2. NY MTA Bus Stop element, XML-encoded
<stop>
<id>MTA_400001</id>
<lat>40.73064</lat>
<lon>-73.99044</lon>
<direction>N</direction>
<name>4 AV/E 9 ST</name>
<code>400001</code>
<locationType>0</locationType>
<wheelchairBoarding>UNKNOWN</wheelchairBoarding>
<routeIds>
<string>MTA NYCT_M1</string>
<string>MTA NYCT_M2</string>
<string>MTA NYCT_M3</string>
</routeIds>
</stop>
Note: the Author recently opened an issue ticket on OBA GitHub [8] requesting the reduced-size response document
In a context of core app functionality, the most important data items of the Xml node in Listing 2 are: stop id, coordinates (Latitude/Longitude), name and direction. The collection of stops can be displayed on the map (in particular, using Bing technology as it was done in application "HeMoSiBi" [9]) and also shown in pure text format as data list with super-imposed bus location retrieved in real-time (actually, "pseudo" real-time as the intrinsic Internet protocol uncertainties and delays do not allow precise timing: in general, response/process time within 1...2 sec is feasible and reasonable for this type of application). System timer implemented via AJAX keeps updating the selected vehicles (buses) location relative to stops at predefined interval (set for 10 sec in this functional proto). Button control allows forcing immediate data update.
GIS Functions
Calculation of the distance between two geo-point on the surface is one of the core GIS functions. Multiple algorithms (see the Listing 3) exist, which differ by performance/accuracy [11-13].
Listing 3. Basic GIS functions to calculate distance between two geo-point on the surface
using System;
namespace BusNY
{
internal enum UnitSystem { SI = 0, US = 1 }
internal static class GIS
{
#region internal: properties (read-only)
internal static double EarthRadiusKm { get {return _radiusEarthKM;} }
internal static double EarthRadiusMiles { get { return _radiusEarthMiles; } }
internal static double m2km { get { return _m2km; } }
internal static double Deg2rad { get { return _toRad; } }
#endregion
#region private: const
private const double _radiusEarthMiles = 3959;
private const double _radiusEarthKM = 6371;
private const double _m2km = 1.60934;
private const double _toRad = Math.PI / 180;
#endregion
#region Method 1: Haversine algo
internal static double DistanceHaversine(double Lat1,
double Lon1,
double Lat2,
double Lon2,
UnitSystem UnitSys ){
try {
double _radLat1 = Lat1 * _toRad;
double _radLat2 = Lat2 * _toRad;
double _dLatHalf = (_radLat2 - _radLat1) / 2;
double _dLonHalf = Math.PI * (Lon2 - Lon1) / 360;
double _a = Math.Sin(_dLatHalf);
_a *= _a;
double _b = Math.Sin(_dLonHalf);
_b *= _b * Math.Cos(_radLat1) * Math.Cos(_radLat2);
double _centralAngle = 2 * Math.Atan2(Math.Sqrt(_a + _b), Math.Sqrt(1 - _a - _b));
if (UnitSys == UnitSystem.SI) { return _radiusEarthKM * _centralAngle; }
else { return _radiusEarthMiles * _centralAngle; }
}
catch { throw; }
}
#endregion
#region Method 2: Spherical Law of Cosines
internal static double DistanceSLC(double Lat1,
double Lon1,
double Lat2,
double Lon2,
UnitSystem UnitSys ){
try {
double _radLat1 = Lat1 * _toRad;
double _radLat2 = Lat2 * _toRad;
double _radLon1 = Lon1 * _toRad;
double _radLon2 = Lon2 * _toRad;
double _centralAngle = Math.Acos(Math.Sin(_radLat1) * Math.Sin(_radLat2) +
Math.Cos(_radLat1) * Math.Cos(_radLat2) * Math.Cos(_radLon2 - _radLon1));
if (UnitSys == UnitSystem.SI) { return _radiusEarthKM * _centralAngle; }
else { return _radiusEarthMiles * _centralAngle; }
}
catch { throw; }
}
#endregion
#region Method 3: Spherical Earth projection
public static double DistanceSEP(double Lat1,
double Lon1,
double Lat2,
double Lon2,
UnitSystem UnitSys ){
try
{
double _radLat1 = Lat1 * _toRad;
double _radLat2 = Lat2 * _toRad;
double _dLat = (_radLat2 - _radLat1);
double _dLon = (Lon2 - Lon1) * _toRad;
double _a = (_dLon) * Math.Cos((_radLat1 + _radLat2) / 2);
double _centralAngle = Math.Sqrt(_a * _a + _dLat * _dLat);
if (UnitSys == UnitSystem.SI) { return _radiusEarthKM * _centralAngle; }
else { return _radiusEarthMiles * _centralAngle; }
}
catch { throw; }
}
#endregion
}
}
Mathematically speaking, all three algos result in computation of a great-circle (orthodromic) distance on Earth between 2 points, though the accuracy and performance are different. In a context of current application they all provide reasonably good approximation with error margin typically not exceeding couple meters within NY City boundaries. All three aforementioned algorithms are based on spherical model of the Earth. More accurate ellipsoidal model (e.g. Vincenty’s solution) exists reducing the error margin to the fraction of mm (such accuracy is obvious "oiverkill" for current application), but also substantially increasing the computational complexity. Therefore, Method 3 (Spherical Earth projection ) has been selected providing the highest computational performance with reasonable accuracy.
Bus icon: pure HTML5/CSS solution
Bus icon is not included in standard Unicode character set, thus to avoid the use of any custom image files in data transport layer (such image files can increase the size of the communication packet by order of magnitude, thus adding heavy load to the real-time operation) an alternative, rather compact pure HTML5/CSS3 solution has been developed using just div as shown in HTML5/CSS3 code snippet below (see Listing 4 and corresponding screenshot shown below):
Listing 4. Bus icon composed of <div> HTML elements
<!DOCTYPE html">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>BUS ICON</title>
<style type="text/css">
.b16 {
width:12px;
height:14px;
border-top: 2px solid #000099;
border-left: 2px solid #000099;
border-right: 2px solid #000099;
border-top-left-radius: 4px;
border-top-right-radius: 4px;
background:#f0f0f0;
}
.b16 .b {
width:100%;
height:4px;
margin-top:4px;
border-top: 2px solid #000099;
border-bottom: 2px solid #000099;
background:#ffff00;
}
.b16 .b .bb {
background:#000099;
width:6px;
margin:0 auto;
height:4px;
}
</style>
</head>
<body>
<!--
<div class="b16"><div class="b"><div class="bb"></div></div></div>
</body>
</html>
Corresponding to the Listing 4, Bus Icons (16px and 24px) "made of div, no graphics" are shown below in Fig. 2:
Fig.4. Bus icons "made of div"
Flexible solution using pseudo-elements
More flexible and compact solution to produce bus pseudo-icons is shown in the following Listing. Better scalability is achieved via using em unit; bus color can be changed with just couple CSS lines extending the flexibility of the solution (in real use-case there is a need for multiple pseudo-icons w/different color/size):
Listing 4a. Advanced CSS3 styling of HTML5 <div> and pseudo-elements (:before and :after)
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>ADVANCED HTML5/CSS3: BUS PSEUDO-ICONS MADE OF DIV</title>
<style type="text/css">
.b1
{
display : inline-block;
vertical-align : middle;
width : 0.8em;
height : 0.8em;
border : 0.2em solid;
border-bottom : none;
border-radius : 0.3em 0.3em 0em 0em;
}
.b1:before
{
content :'';
display : block;
height : 0.2em;
padding : 0;
margin-top : 0.25em;
border-top : 0.15em solid;
border-bottom : 0.15em solid;
}
.b1:after
{
content :'';
display : block;
width : 0.4em;
height : 0.4em;
margin :-0.4em auto;
}
.b1, .b1:before { border-color: #000099; }
.b1:after {background : #000099; }
.b2
{
display : inline-block;
vertical-align : middle;
width : 1.7em;
height : 1.7em;
border : 0.3em solid;
border-bottom : none;
border-radius : 0.4em 0.4em 0em 0em;
}
.b2:before
{
content :'';
display : block;
height : 0.4em;
margin-top : 0.6em;
border-top : 0.3em solid;
border-bottom : 0.2em solid;
}
.b2:after
{
content :'';
display : block;
width : 0.9em;
height : 0.8em;
margin :-0.8em auto;
}
.b2, .b2:before { border-color: darkgreen; }
.b2:after {background : darkgreen; }
.b2:before { background : Yellow;}
</style>
</head>
<body>
<table>
<tr>
<td>Bus pseudo-icon, 1em, blue w/white headlights</td>
<td><div class="b1"></div></td>
</tr>
<tr>
<td>Bus pseudo-icon, 2em, green w/yellow headlights</td>
<td><div class="b2"></div></td>
</tr>
</table>
</body>
</html>
Corresponding pseudo-icon sample images are shown below:
Fig.4a. Bus pseudo-icons "made of div" and advanced CSS3 technique
Other advanced HTML5/CSS3 formatting solutions published by Author on Codeproject [14, 15, 21, 22].
SortedDictionary for Bus Routes
Bus Routes are obtained from MTA data feed and stored in Dictionary<string, string> (the Key represents Bus Route code, the value corresponds to the MTA Agency). The list is sorted alphabetically (as string values), thus looking like: M1, M100, M2, etc. It would be much more convenient for Users if the list was sorted basem on Bus Route prefix first, and then of its numerical part, so it will look like M1, M2, M100, etc. In order to achieve this two-level sorting the following algorithm was developed, utilizing SortedDictionary object included in.Net (see the following Listing ):
Listing 5. Bus Routes sorting algorithms
#region method: Sort Routes Dictionary
enum enumRoutePrefix { B, BM, Bx, BxM, M, Q, QM, S, X }
public static SortedDictionary<string, string> SortRoutesDictionary(Dictionary<string, string> UnsortedDictionary)
{
SortedDictionary<string, string> _sorted = new SortedDictionary<string, string>();
try
{
foreach (KeyValuePair<string, string> _kv in UnsortedDictionary)
{
foreach (enumRoutePrefix _ePfx in Enum.GetValues(typeof(enumRoutePrefix)))
{
string _pfx = _ePfx.ToString();
if (_kv.Key.Contains(_pfx))
{
string _lsb = Regex.Replace(_kv.Key, "[^0-9]", String.Empty);
string _mod = _kv.Key.Replace(_lsb, int.Parse(String.Concat(_lsb)).ToString("D4"));
_sorted.Add(_mod, _kv.Key);
break;
}
}
}
return _sorted;
}
catch { return null; }
}
#endregion
Cookies and Session Variables
Session variables are typically used to preserve data fields between web Page re-load, pretty much like session cookies. Persistant Cookies are stored on the User's machine typically reflecting preference/settings pertinent to a particular app; the remains in effect until deleted or expired. Use of persistant cookies is a sort of controversial issue (the comprehensive discussion of this issue may go far beyond the boundaries of this article), thus as a general rule it's not recommended to use such technique in any app critical functionality. It still may be usefull as an auxiliary convenience feature; pertinent to current app, such persistent cookies are used to store the User preference regarding Bus Route and update mode (manual/automatic) as shown in the following code snippet:
Listing 6a. Write Persistent Cookies procedure to store User preferences: Bus Route and update mode
#region method: WriteCookies()
protected void WriteCookies()
{
try
{
Response.Cookies[_cookiesRoot][_cookiesAuto] = chkAuto.Checked.ToString();
Response.Cookies[_cookiesRoot][_cookiesRoute] = cmbRoutes.Text;
Response.Cookies[_cookiesRoot][_cookiesLastRecorded] = GetTimeNYC().ToString();
Response.Cookies[_cookiesRoot].Expires = GetTimeNYC().AddYears(_cookiesExpiresAddYear);
}
catch { }
}
#endregion
Listing 6b. Read and apply Cookies on the first Page Load
if (Request.Cookies[_cookiesRoot] == null)
{
chkAuto.Checked = true;
cmbRoutes.SelectedIndex = 0;
}
else
{
if (Request.Cookies[_cookiesRoot][_cookiesRoute]==null) { cmbRoutes.SelectedIndex = 0; }
else
{
string _route = Server.HtmlEncode(Request.Cookies[_cookiesRoot][_cookiesRoute]);
cmbRoutes.Text = _route;
}
if (Request.Cookies[_cookiesRoot][_cookiesAuto] == null){chkAuto.Checked = true;}
else
{
bool _auto = true;
if (bool.TryParse(Server.HtmlEncode(Request.Cookies[_cookiesRoot][_cookiesAuto]), out _auto))
{ chkAuto.Checked = _auto; }
else { chkAuto.Checked = true; }
}
}
In the code snippets shown above, _cookiesRoot var refers to the root key while other vars refer to subkeys. Custom Function GetTimeNYC() described below (see Listing 8) is used to set cookie expiration pertinent to the actual New York Time invariant to the web hosting server location.
Listing 6c. Use of Session variable to toggle Bus direction code
protected void btnDir_Click(object sender, EventArgs e)
{
int _dirCode=0;
try
{
if (int.TryParse(Session[_svarDirection].ToString(), out _dirCode))
{
_dirCode = (_dirCode == 0) ? 1 : 0;
Session[_svarDirection] = _dirCode;
}
}
catch { }
}
The code snippet above demonstrates the use of Session variables with life span limited to the current web session (unlike the cookies described above, it's non-persistent on User's hard drive).
Toggle Button made of ASP.NET CheckBox and CSS
The latest web app release implements aesthetic ToggleButton [ 17] built on the underlying ASP.NET CheckBox control with CSS3 styling as shwon in the following listings:
Listing 7. Toggle Button solution utilizing ASP.NET CheckBox control and HTML5/CSS3 styling
<style type="text/css">
div.divToggleButton input[type=checkbox]
{
display: none;
white-space: nowrap;
}
div.divToggleButton label
{
display: block;
float: left;
cursor: pointer;
}
div.divToggleButton input[type=checkbox]:checked + label::before,
div.divToggleButton input[type=checkbox]:not(:checked) + label::before,
div.divToggleButton input[type=checkbox] + label
{
width: 40pt;
height: 40pt;
line-height: 40pt;
}
div.divToggleButton input[type=checkbox] + label
{
vertical-align: middle;
text-align:center;
font-size: 16pt;
font-family:Arial, Calibri;
border: 1px solid #bdbdbd;
border-radius: 5px;
background: #f0f0f0;
background-image: -moz-linear-gradient(top, #fdfdfd, #f9f9f9 50%, #e5e5e5 50%, #fdfdfd);
background-image: -webkit-gradient(linear, center top, center bottom,
from(#fdfdfd), color-stop(0.5, #f9f9f9), color-stop(0.5, #e5e5e5 ), to(#fdfdfd));
background-image: linear-gradient(to bottom, #fdfdfd, #f9f9f9 50%, #e5e5e5 50%, #fdfdfd);
}
div.divToggleButton input[type=checkbox]:not(:checked) + label::before
{
content: "M";
color: #303030;
opacity: 0.6;
}
div.divToggleButton input[type=checkbox]:checked + label::before
{
content : "A\2714";
color : #000090;
font-weight : bold;
}
</style>
</head>
<body>
<form id="form1" runat="server">
<div>
<div class="divToggleButton">
<asp:CheckBox ID="chkToggleButton" runat="server" AutoPostBack="true" />
<asp:Label ID="lblToggleButton" AssociatedControlID="chkToggleButton" runat="server" ToolTip="Toggle between Auto and Manual mode"/>
</div>
</div>
</form>
</body>
This ToggleButton solution preserves all the functionality of the underlying CheckBox control, thus it works "just out of the box" - no changes to C# code behind are required.
Demo screenshots corresponding to checked/unchecked ToggleButton states are shown below:
| Fig 5a. ToggleButton visual state corresponding to Auto-update mode |
| Fig 5b. ToggleButton visual state corresponding to Manual mode |
Fig. 5(a,b) ToggleButton visual states
Web app utilizing this ToggleButton solution looks more aesthetically-pleasing as shown in the following demo screenshot pertinent to Bus Route Q60 in the NY City:
Fig 6. Sample screenshot shows ToggleButton "AUTO" (CSS3 style) replacing standard ASP.NET CheckBox.
Blinking Notice
In order to send some sort of visual notification of web page status update that happened on AJAX-timer event or manual screen refresh, the following "blinking div" notification CSSs pseudo-class was added to the web page (see the following CSS3/HTML5 code Listings and sample screenshot). The 2-sec blinking effect is achieved via pure CSS3 (no javascripting whatsoever). The solution is compatible with all major web browsers.
Listing 8a. Blinking screen update notice: CSS3 style
.divUpdatedNotice
{
width : 60pt;
height : 18pt;
line-height : 18pt;
display : inline-block;
vertical-align : middle;
text-align : center;
font-size : 11pt;
font-family : Arial, Calibri;
font-weight : 600;
background-color : #000099;
color : #fafafa;
opacity : 0;
-webkit-animation-name : blinkUpdateNotice;
-webkit-animation-duration : 2s;
animation-name : blinkUpdateNotice;
animation-duration : 2s;
}
@-webkit-keyframes blinkUpdateNotice{ from {opacity: 0;} to {opacity: 1;} }
@keyframes blinkUpdateNotice{ from {opacity: 0;} to {opacity: 1;}}
Listing 8a. Blinking screen update notice: sample HTML5
<div class="divUpdatedNotice">UPDATED</div>
Fig. 7. Blinking [UPDATED] notice implemented in standard/mobile web pages using CSS3 as shown above
HTML5 Select element style (CSS3)
Styling of rather important HTML Select element (also known as DropDown List) is still a non-trivial task. After a great deal of Googling and "what if" attemps, the pure CSS3 solution was developed (no javascripting), providing a desired level of simplicity and aesthetic pleasing, as shown in the Listing below:
Listing 9. Styling of HTML5 SELECT (aka DropDown List) element w/pure CSS3
select#selPOI
{
float : right;
margin-top : 0.6em;
width : 9em;
height : 2.5em;
padding : 0.2em 0.4em 0.2em 0.4em;
vertical-align : middle;
border : 1px solid #e9e9e9;
-moz-border-radius : 0.5em;
-webkit-border-radius : 0.5em;
border-radius : 0.5em;
box-shadow : inset 0 0 4px #a0a0a0;
-webkit-appearance : none;
-moz-appearance : none;
appearance : none;
background : url(/Img/favicon.ico) 94% / 10% no-repeat #fdfdfd;
font-family : Arial, Calibri, Tahoma, Verdana;
font-size : 1em;
color : #303030;
cursor : pointer;
}
select#selPOI option
{
font-size : 1em;
padding : 0.2em 0.4em 0.2em 0.4em;
}
select#selPOI option[selected]{ font-weight:bold}
select#selPOI option:nth-child(even) { background-color:#f5f5f5; }
A sample screenshot image shown below demonstrates the "final good": aesthetically-enhanced HTML5 Select element styled via pure CSS3 with all functionality preserved (populated w/sample search items pertinent to this web app).
Fig. 8. Sample screenshot of HTML5 SELECT element style (pure CSS3 solution, no javascript)
Points of Interest
Proper Time-out in Asynchronous operations
In order to achieve high-responsivenes of this web app UI, the core real-time bus tracking operations are performed in asynchronous manner via AJAX technology. To avoid the potential uncertanties of program execution dealing with data stream aggregation coming asynchronously from multiple threads, it's quite important to properly set a time-out period to be substantially lesser than AJAX timer interval. This demo web app has the update interval set to 20 sec, therefore, implementing a 5 sec time-out (see the following code snippet) for the delay-prone online data reading procedure will improve the overall operational stability.
Listing 9. Reading online data stream in realtime (notice 5 sec time out)
public static string GetHTML(string url)
{
const int _timeOut = 5000;
HttpWebRequest _request;
try
{
_request = (HttpWebRequest)WebRequest.Create(url);
_request.Timeout = _timeOut;
using (WebResponse _response = _request.GetResponse())
{
using (Stream _stream = _response.GetResponseStream())
{
using (StreamReader _streamReader = new StreamReader(_stream))
{
return _streamReader.ReadToEnd();
}
}
}
}
catch { return null; }
finally { _request = null; }
}
Realtime Stamp issue rooted in server geolocation
Server-side TimeStamping of the real-time data downstream was found to be not a trivial task. The problem is rooted in fact that the Azure Server hosting this particular NYC-dedicated web app can be located far way in some different time zone (even though it seems a bit irrational to make a virtual round trip to a server overseas just to get a chunk of data rendered by NY MTA service Company that I can possibly see through my window: beyond just little programming inconvenience, the server location could be an important design consideration in a context of real-time app like this). The programmatic workaround to this issue was to use Universal Time with fixed offset pertinent to NY City time zone, though the "fixed offset" may actually vary due to daylight saving time issue (frankly, a better option for this type of real-time IoT where accurate timing/low latency is rather critical would be using a server local to NYC).
Listing 10. GetTimeNYC() function to get server time-zone invariant accurate NY Time
protected DateTime GetTimeNYC()
{
try { return DateTime.Now.ToUniversalTime().AddHours(_offsetHour); }
catch { throw; }
}
Azure Web Hosting Plans/Scale levels
Azure websites implements rather convenient and cost-effective scalability management tools: the scales are ranging from the Free to Standard levels as follows:
- Free
- Shared
- Basic
- Standard
Original web app was created using Free Azure web sites option, which was found insufficient for this type of real-time app with heavy data traffic. The site quickly went down due to traffic quotas limitation, so it was later upscaled to Shared, then to Basic, and even to the Standard level. The final Scale level selection would be made upon completion of the comprehensive load/stress testing of the app.
Azure Autoscale feature
Azure Autoscale is particular useful for this type of application. Intensity of web requests, tightly correlated with daily commuting traffic may vary dramatically: from very high level at rush hours to very low at night time. Autoscaling capability of Azure web applications (in terms of operational Instances count) is a highly-desirable, convenient technical-enabling vector, providing significant money-saving opportunity.
Bus mapping upgrade: from Bing Maps V6.3 to V7.0
The Author has been a pioneer and enthusiast of the Microsoft Bing Maps (aka Virtual Earth) technology from the day of its inception, and since developed plenty of interactive mapping solution. For the last several years the major focus was made on Virtual Earth V6.3 [23]. Technology became obsolete and the issue with polyline has been noticed: it disappers at higher zoom level (see the following images):
Fig 9: Polyline issue in Bing Maps V6.3
| |
| Fig. 9a: M1 Bus route polyline (fragment) is clearly visible | Fig. 9b: Bus route polyline disappears at higher zoom |
The polyline issue was resolved by upgrading the mapping solution to Microsoft Bing Maps AJAX Control, Version 7.0 [24] - see the sample screenshot below:
Fig.10: M1 Bus route polyline (fragment shown) is clearly visible at any zoom level (Bing Maps V 7.0)
Complementary Windows app connected to Azure Web API
Windows 7/8 complementary application will be adapted for any PC platform ranging from smart phones to powerful desktop computers with focus made on high-end Tablets/Notebooks equipped with Intel i3/i5/i7 CPU and Windows 7/8 [20]. In addition to NYC MTA Bus services, that Win app will also cover NYC MTA Subway, LIRR and MNR (see the demo screenshot below). The app will utilize plenty of the previously developed HeMoSiBi™ [9] project code base and consume Azure-hosted services (WebAPI) with code base similar to the real-time SIRI/OBA data parser implemented in this web app. For more information refer to the article [20].
Fig 11. Extended HeMoSiBi™ app (NY-2015 edition) demo screenshot shows popular Subway 7-Train
Disclaimer of Warranty and Limitation of Liability
Proposed solution does not implement any open/closed-loop control system of a physical devices on a recipient side. It outputs to the human-readable devices (i.e. computer monitor/display) and can be categorized as distributed real-time Data Acquisition sytem with some supervisory functions, and optional user feedback channels (crowdsourcing data aggregator).
THE SOFTWARE SOLUTION IS PROVIDED "AS-IS," "WITH ALL FAULTS," AND "AS AVAILABLE" AND END USER BEAR ALL RISK OF USING IT. THE AUTHOR/COMPANY GIVES NO EXPRESS WARRANTIES, GUARANTEES, OR CONDITIONS IN RELATION TO THIS SOFTWARE. IN NO EVENT SHALL AUTHOR/COMPANY BE HELD LIABLE FOR ANY ACTUAL OR IMPLIED, DIRECT OR INDIRECT, INCIDENTAL OR CONSEQUENTIAL, OR ANY OTHER DAMAGES ARISING OUT OF THE USE OF, OR INABILITY TO USE THIS SOFTWARE.
History
- Dec 2014. Application-specific Database and GTFS-compliant MTA Bus static data import procedure has been developed
- Jan 2015. Application Real-time SIRI data import API (response parser) for Bus Routes-Stops has been prototyped.
- Jan 2015. Application Real-time OBA data import API (response parser) for Routes-Agencies has been prototyped.
- Feb 2015. Data visualization module for Win / Web app has been prototyped.
- Mar 11, 2015: First draft of the article has been published.
- Mar 11, 2015: System integration on Azure platform started.
- Mar 12, 2015: Working DEMO uploaded to BusNY.azurewebsites.net
- Mar 14, 2015: Managed Azure cloud websites, created a backup, upscaled to "shared"
- Mar 16, 2015: Added code samples, description and demo screenshots
- Mar 17, 2015: ToggleButton was added, replacing standard ASP.NET CheckBox control
- Mar 19, 2015: Interactive "bus map" page added to the solution
- Mar 20, 2015: Blinking update notice feature added to the solution
- Mar 20, 2015: Web query customization feature added to the solution
- Mar 20, 2015: Major article update: text, images, code samples
- May 6, 2015: Major solution updates, code improvement.
List of Acronyms
| ACR | Acceleration Conformance Ratio |
| AJAX | Asynchronous JavaScript and XML |
| API | Application Programming Interface |
| CEN | European Committee for Standardization |
| CSV | Comma-Separated Values (data format) |
| GIS | Geographic Information System |
| DDCR | Dynametric Driving Comfort Ratio |
| DDRR | Dynametric Driving Risk Ratio |
| EDER | Ergometric Driving Efficiency Ratio |
| FIR | Finite Impulse Response |
| GTFS | General Transit Feed Specification |
| IIR | Infinite Impulse Response |
| IoT | Internet of Things |
| JSON | Javascript Object Notation |
| LIRR | Long Island Rail Road |
| MNR | Metro-North Railroad |
| MTA | Metropolitan Transportation Authority (NY) |
| NSF | National Science Foundation (US) |
| NYCTA | New York City Transit Authority |
| OBA | One Bus Away (“Discovery" API) |
| PCB | Printed Circuit Board |
| POI | Point Of Interest |
| REST | Representational State Transfer |
| SBIR | Small Business Innovation Research |
| SCADA | Supervisory Control and Data Acquisition |
| SIRI | Service Interface for Real Time Information |
| SOAP | Simple Object Access Protocol |
| SPICE | Simulation Program with Integrated Circuit Emphasis |
| TCP/IP | Transmission Control Protocol/Internet Protocol |
| WPF | Windows Presentation Foundation |
| UML | Unified Modeling Language |
| XML | Extensible Markup Language |
| XAML | Extensible Application Markup Language |
References
- IoT
- Transportation in New York City
- MTA NY
- Supervisory Control and Data Acquisition (SCADA)
- General Transit Feed Specification (GTFS)
- Service Interface for Real Time Information (SIRI)
- OneBusAway (OBA)
- Reduce references output of stops-for-route API (GitHub OBA #122)
- HeMoSiBi™ - Her Most Significant Bit (AIC-2013 FINALIST app)
- Ergometric Efficiency Profiler of Vehicles (proposal to NSF SBIR Phase I, 2014)
- Haversine Formula (wiki)
- Spherical law of cosines (wiki)
- Geographical distance (wiki)
- HTML5 Tables formatting: alternate rows, color gradients, shadows (Codeproject)
- HTML5/CSS3 graphic enhancement: buttons, inputs (Codeproject)
- Azure web app: New York City real-time bus tracking
- Toggle Button using ASP.NET Checkbox and CSS
- MTA Regional Bus Operation (wiki)
- Configuring a custom domain name for an Azure Website (GoDaddy)
- Perpetuum-M: Realtime bus tracking app for Windows
- Advanced CSS3: pseudo-icons made of HTML5 div Elements
- Advanced CSS3 Styling of HTML5 SELECT Element
- Bing Maps AJAX Control v6.3 Documentation (CHM)
- Bing Maps AJAX Control, Version 7.0
- Azure web app: Engineering Calculator VOLTMATTER
