Introduction
This article shows how to build an Azure IoT Solution for water utilities
This solution has been recognized by Microsoft because of its ability to analyze, store, and report data for 500,000 meters and 250 million transactions/year using various Azure Services. Developing a Microsoft Azure IoT solution helped to greatly reduce the cost of such solution for utilities industry. Also it has the capacity to scale to trillions of meter readings. We have been interviewed by Microsoft to publish a case study on this solution.
This Solution has also been covered by a news Channel in Califonia. You can see the news here: http://www.nbclosangeles.com/video/?anchor_tag=%21%2Fon-air%2Fas-seen-on%2FSmart-Meters-Help-Long-Beach-Crack-Down-on-Water-Wasters%2F298093051&anchor_tag=%21%2Fon-air%2Fas-seen-on%2FSmart-Meters-Help-Long-Beach-Crack-Down-on-Water-Wasters%2F298093051#!/on-air/as-seen-on/Drought-Fighting-/298093051
<o:p>
Because Azure provides an integrated development environment with ready access to servers, networks, software, and development tools, it reduces development time by about 50 percent. That’s a phenomenal acceleration. <o:p>
Table of Content
<o:p>
The IoT component used in this solution are:
- Utility Meter: The meters are CDMA based wireless device which communicates over the internet to connect to the Azure services. It works on embedded code installed on memory of the device.
- Azure Cloud Service: a PaaS offering from Microsoft Azure to host your web application. It is used to create highly-available & infinitely-scalable applications. Once your application is deployed, Azure handles the rest. From provisioning, to load-balancing, to health monitoring. Your application is backed by an industry-leading 99.95% monthly SLA.
- Azure Machine Learning: The Azure Machine Learning Service is a powerful cloud-based predictive analytics service that makes it possible to quickly create analytics solutions. It is a fully managed service.
- Service Bus Queue: Service Bus Queues support a brokered messaging communication model. A message producer (sender) hands off a message to the queue and then continues its processing. Asynchronously, a message consumer (receiver) pulls the message from the queue and processes it. Queues offer First In, First out (FIFO) message delivery.
- Azure Table Storage: Azure Tables are a non-relational, key-value-pair, storage system suitable for storing massive amounts of unstructured data. The non-relational stores like Table Storage are optimized for simple retrieval and fast inserts.
- Azure Virtual Network with ExpressRoute: Azure ExpressRoute enables you to create private connections between Azure datacenters and infrastructure that’s on your premises or in a colocation environment. ExpressRoute connections do not go over the public Internet, and offer more reliability, faster speeds, lower latencies and higher security than typical connections over the Internet.
Today Organizations are seeing measurable benefits from Internet of Things (IoT). Utility companies are eliminating costly and inconvenient home visits to read meters by introducing smart meters that report more granular usage data without human intervention. This is just an examples of the myriad ways in which IoT is making new products, business models, and processes possible.
Organizations can see benefits from adopting IoT. But the primary benefits that our customers are seeing typically fall into three categories: improved customer and citizen experiences, accelerating growth and business performance, and improving safety and reducing risk. That’s why there are already more than a billion connected devices and machines in use today. That said, the opportunity for IoT-enabled transformation has grown massively in the last couple of years, and continues to do so.
With severe water shortage conditions threatening many parts of the world, water utilities see the need to adopt a smart-meter system. This will help in terms of reduce cost, managing meter readings etc. However, the cost of such systems can be cost-prohibitive The water industry has lagged far behind the electric power industry in the use of smart meters.
Most utilities can’t afford to build private wireless networks or datacenters filled with servers and sophisticated analytical software. We designed a cost-effective smart meter solution that uses the existing Wireless network for meter data transmission and Microsoft Azure for data management.
Microsoft Azure has played a vital role in this solution right from using different service on Cloud and managing those services, but the most useful was in terms of Performance, go-to-market time & cost efficiency. The other Services used in this solution were a Web Role which host the application code on PaaS. An Infrastructure service like Virtual Network with express route for private and dedicated Communication is established. Providing a Secure environment for the utility Meter to communicate and avoiding data security Concerns.
Now Azure has improved over the last couple of years to provide different service that can even enhance more for the solution like this. The services like Azure Machine Learning, Streaming Analytics Providing a Real time analytics, Events Hubs to process large amount of data faster between two Azure Services. As we move ahead we will see how these Azure components are used in this solution.
Let’s get started with the implementation of Water Utility Meter Data Management System. A utility Meter collects the daily consumption data and sends it to the Azure Cloud Service daily using a wireless network in a 2 way communication. This communication works with a WCF service hosted on Azure PaaS as a Cloud Service and a Meter in the field with some embedded code and wireless chip installed for communication.
This solution is divided into below steps:
- Utility Meter collecting the daily Consumption usage
- Transfer of Usage data from Meter to Cloud Service
- Instructions sent from Cloud Service to Meter to update firmware, configurations and data set that needs to be collected.
- Using web applications to display the Consumption Data
The hardware Device which works on embedded code. It has 2G wireless capabilities with CDMA protocol. We implemented Virtual Network with Azure ExpressRoute. It enables you to create private connections between Azure datacenters and infrastructure that’s on your premises or in a colocation environment. ExpressRoute connections do not go over the public Internet, and offer more reliability, faster speeds, lower latencies and higher security than typical connections over the Internet..
When a meter connects to a cloud service the data is parsed and added to Service Bus Queue component. This message is pooled in a worker role and pushed to Azure Table storage - a powerful No-SQL storage provided by Azure.
Using Service Bus Queue has helped to provide faster processing of data. The up time for the Meter-to-Cloud service communication reduces the battery life of the Meter. Service Bus Queue also helped us to optimize the usage of meter and hence increase the battery life to an 8-10 Years life span for the Meter. Once the battery is dry out we need to replace the hardware device itself.
This was just one single communication instance, with this architecture we are able to process more than 500,000 transaction per day and that too in 5 hours window only.
This data stored in Azure Table storage is used by different solutions build around it like Web applications and Machine learning to provide predictive analytics.
The smart utility meter provides various benefits to us like eliminating infrastructure barriers. It also provides a deep insight into water usage to the water utilities and their customers.
It reduces development time by 50 percent by using the Cloud and scales to accommodate trillions of transactions.
The meters are CDMA based where communication happens over the internet to connect the Azure services creating a perfect solution of an ‘Internet of Things’.
While starting the project we required a hardware meter which was supported with the Embedded Code to collect consumption logs and then pass this data to the Cloud service where the data gets processed. This communication was in the form of hexadecimal data passed in the form of HTTP packet from wireless signal.
The meter is responsible for imitating the communication. The meter which stores configuration in the small memory capacity it has. These configuration can be like the URL to send the data. Below is an example of how a Cloud service is called by a Meter.
When an input communication is established from a Meter to a Cloud Service it receives the HTTP packet as a stream object as shown in code snippet below:
The next step is to pass this data into the Service Bus Queue and respond to utility Meter with acknowledgment of data received. This way we also reduced the battery operated Utility meter wait- time and hence achieved an increased life for the meter.
A worker role, which runs in background and works on processing, the Azure Worker role is added which is a continuous running service in Azure which poles the data.
Once this input is retrieved from the Service Bus Queue, it is processed for different validations and then moved to a next step that is to store the data into an Azure Table storage. This No-SQL component works effectively with a combination of a Primary Key and Row Key based Point Queries.
Below are the details shown to consumer about the usage in detailed manner, these chart shows a monthly data consumed by the User,
The Core challenge with the solution was to improve performance so that less battery is consumed thus increasing life of the devices. A Machine Learning Service is used to create different predictions that will help us to get more advantages of Microsoft Azure. Likewise a utility would be interested in predicting the Consumption of meter based on its historical data maintained. Similarly a prediction was made on battery Usage wherein if battery dropdown below a certain threshold limit it would send an alert after implementing this experiment in Azure ML.
You can contact at prakash@snp.com for details if you are interested in any discussion or if you have any questions.
References
Azure IoT: http://www.microsoft.com/en-in/server-cloud/internet-of-things.aspx
Azure Cloud Service : http://azure.microsoft.com/en-us/services/cloud-services/
Azure Machine Learnining : http://azure.microsoft.com/en-in/services/machine-learning/
Azure Express Route: http://azure.microsoft.com/en-us/services/expressroute/
Azure Table Storage: http://azure.microsoft.com/en-in/documentation/articles/storage-dotnet-how-to-use-tables/
Water Utility Review: http://www.nbclosangeles.com/video/?anchor_tag=!%2Fon-air%2Fas-seen-on%2FSmart-Meters-Help-Long-Beach-Crack-Down-on-Water-Wasters%2F298093051&anchor_tag=!%2Fon-air%2Fas-seen-on%2FSmart-Meters-Help-Long-Beach-Crack-Down-on-Water-Wasters%2F298093051#!/on-air/as-seen-on/Neighbors-Describe-Fire--Dog-Rescue/298153881
History
2015-02-22: First completed draft.
