This article describes the design and implementation of the small tool, tester for exploring Azure IoT Hub with virtual MQTT Device.
Contents
- Virtual MQTT Device for interfacing with Azure IoT Hub using the MQTT direct protocol
- Sending simulated telemetry data (periodically or manually)
- Each MQTT Device is hosted in own AppDomain
- Exploring Azure IoT Hub via the REST Endpoints
- MQTT v3.1.1 protocol on port 8883
- Using the MQTT protocol directly based on the MSDN Doc
- No Device SDK
- Using M2Mqtt - MQTT Client library for .Net
- Multi-tenants usage (opening multiple instances with multiple namespaces with multiple virtual devices)
- REST API Requests (templates)
New Features in the Version 1.1
- Adding a Modules node in the simulated device
- Simulation of the Device Streams (still in the preview)
- Using a device connection string for connectivity to the Azure IoT Hub
- Simulation of the devices connected to the Azure IoT Central
- Simulation of the Plug and Play devices connected to the Azure IoT Central (still in the preview)
- Highlighting styles in the json formatted text
In general, the Internet of Things (IoT) is a rapidly growing network of connected things to collect and exchange data. The Microsoft Azure platform built-in from the End-To-End infrastructure for very fast ingesting your things to the data stream pipe, analyzing, managing and visualizing your things in the real-time process. All things are connected to the Azure IoT infrastructure via the Hub knows as the Azure IoT Hub, see the following picture:
Things such as devices, applications, etc. are using a specific protocol (MQTT, AMQP, HTTPs) to send a telemetry data and receiving a non-telemetry data from the cloud backend process, if it's required. The collect and exchange data services are provided in real-time in the secure and reliable manner.
The Azure IoT Hub represents an entry component for things connectivity, every in/out data goes through this cloud service with multiple endpoints divided into two groups such as device-facing endpoints and service-facing endpoints. More details about this hierarchy can be found here.
It is very important to know and understand this paradigm of the IoT Hub Endpoints, their protocols, data exchange, behavior, functionalities, etc.
For example, the following screen snippet shows a connectivity of the two things such as smart phone (Device1) and Raspberry Pi3 (Device2). The Device1 and Device2 want to communicate with each other via the Azure IoT Hub:
As you can see, each thing (device) needs to send and receive data to/from Azure IoT Hub, where this example is required to handle some kind of bridge using an Azure Function. Basically, there are at least six endpoints of the Azure IoT Hub involved during this process. It would be nice to have a tool available for troubleshooting, simulation and testing.
From the service-facing endpoints, Microsoft SDK includes very useful tiny tool knows as Device Explorer Twin. This tool can simulate consuming and invoking a data behind the Azure IoT Hub, for instance, capturing a telemetry data (D2C messages), sending a non-telemetry data to the device (C2D messages), invoking a direct device method, etc.
That's great. What about the other side of the Azure IoT Hub, such as device-facing side? Well, it can be achieved by Azure IoT Hub Tester that is included in this article. The Azure IoT Hub Tester can be used in the model first approach, where the devices do not exist and it can be created virtually for End-To-End process.
The following screen snippet shows the position of the Azure IoT Hub Tester, which helps to simulate our example with Device1 and Device2:
Recently, Microsoft introduced new features in the Azure IoT Hub known as Device Twin. From the device-facing point of the view, the new features are currently supported only by devices with MQTT protocol, therefore the Azure IoT Hub Tester has built-in virtual devices for MQTT protocol.
Note that the Azure IoT Hub is not a generic MQTT broker for pub/sub communications, it is a subset of the MQTT broker for Azure Internet of Things. In other words, the Azure IoT Hub has pre-built all topics for publishers and subscribers, it is not possible to create own topic, etc. More details about this MQTT support by Azure IoT Hub can be found here.
The Azure IoT Hub Tester has been designed and built for Virtual MQTT Devices created and isolated in their own app domain and ready to integrate with the Azure IoT Hub. In addition, the tester has the capability to provide the REST API calls based on the MSDN Documents for both sides such as device and service (cloud) endpoints.
As I have mentioned earlier, the service-facing endpoints can be explored and simulated by Device Explorer Twin. The following screen snippet shows an example using a Device Explorer Twin for exploring a telemetry data generated by Raspberry:
So, having the Azure IoT Hub a capability to test on each side, simulate, exploring and managing individual service endpoint, it will enable us to better understand a main component of the Azure IoT infrastructure. The following pictures shows the "magic" tiny tools.
Device Explorer Twin:
Azure IoT Hub Tester:
For exploring features of the Microsoft Azure IoT Hub, we need at least a free Azure account and create a free Azure IoT Hub. Once we have it, we can connect our testers to the Azure IoT Hub like it is shown in the following picture:
That's great. The above scenario will allow us to create virtual devices, sending telemetry and/or non-telemetry data to the Azure IoT Hub, see which data are ingested into the Default Event Hub, exploring device twin properties, invoking a device direct method, etc. and of course any changes on the service-facing side are propagated on the virtual device such as receiving messages on the specific topic. Also, based on the needs, the virtual device can send the telemetry samples periodically with a random and/or explicitly values including a real time.
The following screen snippet shows more connected testers around the Azure IoT Combo such as IoT Hub and Stream Analytics:
As you can see, using a tester in the IoT pipeline, we can simulate and explore flowing a telemetry and non-telemetry data between integrated components. The above example shows an output of the stream job to the Azure Service Bus and Azure Storage. Both of them can be explored by testers. More examples of the Azure IoT Hub Tester can be found in this article latter.
OK, let's describe its concept and design. I am assuming you have some knowledge of the Azure IoT Infrastructure.
The Azure IoT Hub Tester concept is based on creating a virtual MQTT Device integrated with the Azure IoT Hub using the MQTT protocol described here. Each virtual device is hosted in own app domain and internally communicated via WCF pipe with a default domain, where the Windows Form is located. Based on the registered device in the Azure IoT Hub, the tester will create (in the first step) its private app domain, where the MQTT Client (proxy) can be hosted. Once the device client has been successfully created, it can be connected to the Azure IoT Hub and subscribed on all specific topics such as twin, direct method, C2D messages, etc.
The following picture shows briefly what kind of endpoints are handled by device-facing and service-facing sides:
Basically, there are two groups of the device messaging. The first one is for sending a D2C messages, receiving C2D messages and file upload. The second one is a group of new features knows as Device Twin. Note, that presently the new features are supported only by MQTT protocol. The tester can handle multiple tenants with multiple devices hosted in own isolated app domain, so if for any reason the virtual device will "dead" or disconnected, it will not have any impact to other ones.
The concept and design of the Azure IoT Hub Tester is similar to my Azure Service Bus Tester described in details in my article.
Each virtual device created by this tester will have its own node in the Devices root. The Device node (it represents a virtual MQTT Device) has the capability to send a telemetry data manually or periodically every 3 seconds. The message payload must be in the json formatted text with explicitly values and/or using an alias values for their random substitution during the publishing process. The name of the Device node is a deviceId. The device has 3 child nodes which each of them represents a virtual device subscriber for specific topic such as twin, methods and C2D messages. For example, if the service (cloud) change the device twin desired property, then its virtual device will receive a notification message in the twin node.
The following picture shows an example of the 3 devices connected to the Azure IoT Hub (ba2017-iot):
As you can see the above picture, the Azure IoT Hub namespace (ba2017-iot) has a node REST-API. This node is for REST calls. It is divided into more groups categorized based on REST APIs described in the MSDN Document. The REST-API node is created from the template file related to the specific Azure IoT Hub namespace.
OK, it's show time. Let's describe what this tiny tool can do for you. I am assuming you have some knowledge of the Azure IoT Hub.
Before you start using the Azure IoT Hub Tester, I am assuming you already read Azure IoT Hub developer guide articles and you have the Azure IoT Hub account. Note, that the tool will work only with an Azure IoT Hub, so prepare your connection string to the Azure IoT Hub.
Connect to the Azure IoT Hub
Right click on the root node (Azure IoT Hubs) and select Connect menu, like it is shown in the following screen snippet:
The Connect click will show up the NamespaceDialog, where we have to type two columns, such as the Namespace name represents an Azure IoT Hub and its Connection String, see the following screen snippet:
The tester can be connected to more than one Azure IoT Hub, just it is required to populate the above NamespaceDialog. Note, that these grid values are persisted, so the next time we need to select a grid row for the connectivity to the specific Azure IoT Hub.
Pressing the OK button, the tester will look at the template file with your namespace (for instance: ba2017-iot.json) and it will create a tree-node for this Azure IoT Hub namespace. The following step is showing this example.
Note
For test purposes of the Azure IoT Hub Tester, I have created an Azure IoT Hub account. This account is valid through April 28, 2017.
Namespace: ba2017-iotdemo
ConnectionString: HostName=ba2017-iotdemo.azure-devices.net;
SharedAccessKeyName=iothubowner;SharedAccessKey=uQsW/ekXOzeMrqvq9dEqLsR4Uf1zxZSjSTjGGzwmWEU=
Please, register your device in the Azure IoT Hub and use it for your test. Note that the MQTT protocol didn't allow a multiple connection for the same deviceId.
Connect Device
In this step, I will demonstrate how the Virtual MQTT Device can be created in the tester app domain.
Right click on the Devices node. On the right panel, you should see your all registered devices. If this grid is empty, you don't have any device registered in the selected Azure IoT Hub, so please register one using a Device Explorer Twin or azure portal.
As you can see in the following screen snippet, my ba2017-iot hub contains 3 devices such as DeviceB11, myDevice and myFirstDevice. Note, that every time you select this Devices node (or click for Refresh), the tester will go to the Azure IoT Hub to obtain all registered devices.
Let's select the first device (like is shown in the above picture). Right click on the Devices node and select the Connect action. Behind this action, the tester needs to create all magic work such as creating a connected Virtual MQTT Device hosted in the own app domain, marshaling interface and communication to the default domain where is its UI interface representations, see the following screen snippet:
Now, the tester has one virtual MQTT Device (deviceId = DeviceB11) connected to the Azure IoT Hub (ba2017-iot) and it is ready to work for all device-facing features built in the Azure IoT Hub.
One more thing, the tester is able to register a device using a REST API. The following picture shows an example of this call:
OK, that's great. Let's explore this virtual MQTT device such as Device Twin, Direct Method, D2C and C2D messaging. For this kind of use case, we will need to have the Device Explorer Twin tool opened for accessing a service-facing endpoints. The Device Explorer Twin tool will represent an Azure IoT Hub backend services. I am assuming you have an experience with this Microsoft tool.
Device Twin
The Device Twin can be demonstrated for both sides, when the service populated its desired properties and from the device-facing side, when the device populated its reported properties.
The following screen snippet shows an example, where the myDevice desired properties has been populated by Device Explorer Twin tool:
As you can see the above picture, the desired property with the name abc has been populated by value 123456. Once the Send button has been clicked, we can see in the Azure IoT Hub tester its virtual device (in this example, myDevice) received a message in the twin node. This is a notification message, that the device twin desired properties were changed:
That's great! The device got the notification, so it's up to device responsibility to update its reported properties based on the evaluation of the device twin version, etc.
So, type the payload for Publisher, for instance, like it is shown in the following screen snippet:
Click on the Publish button to send this message to the Azure IoT Hub.
To verify this step, we can click for Refresh button on the Device Explorer Twin tool:
You can create more properties, populate or delete them, etc., to see how the Azure IoT Hub handling a sync between the desired and reported properties.
In the case of see all Device Twin properties, right click on the Twin node and select Get. The Azure IoT Hub will send you back a message on this topic.
Direct Method
The Direct Method allows to invoke a device method by service-facing endpoint in the sync manner. It is similar to the RPC call. The following picture shows a Device Explorer Twin tool how to invoke method on myDevice:
Clicking on Call Method button, the Azure IoT Hub will send the message to the myDevice.
The following screen snippet shows this message in the methods node:
Now, the virtual MQTT Device (myDevice) has ~60 seconds to send a response back to the Azure IoT Hub, otherwise the call will be terminated with error message on the invoker side.
So, type some payload, for instance, like is shown in the following screen snippet and click the button Publish:
After that, you can see a response in the Device Explorer Twin tool included the return status code:
Send D2C Messages
This is a test to demonstrate sending telemetry events known as Device-To-Cloud (D2C) messaging. Our virtual device (myDevice) will publish event to the Azure IoT Hub and on the service-facing side, we will have a Device Explorer Twin tool to see these messages. The test started on the myDevice node like is shown in the following screen snippet:
As you can see, the tester comes with some sample in the Publisher payload. I have highlighted the built-in values which can be replaced by random values during the runtime publishing (except the $DateTime.UtcNow). This tester feature allows to generate random telemetry event for each click on the button Publish.
Using the Device Explorer Twin, we can see a D2C messages like is shown in the following picture:
That's great! Just simply sending one telemetry event. For periodically sending more telemetry samples by virtual device, we need to put this json object as one item of array, see the following screen snippet with highlighted brackets. Note that the tester will send maximum 100 random samples with a period of the 3 seconds.
To stop this "auto-sampling" process, simple click on the Stop Sampling menu like is shown in the following picture:
Note 1: As you can see, the above context menu has the Load Samples item. Use this feature for loading the samples from the file system. The content of the file must be a json array of the telemetry samples.
Note 2: The tester allows a simultaneously sampling from multiple virtual devices.
Note 3: You can send a non-telemetry event (message) to the Azure IoT Hub Routes simple adding a property bag (name/value pair) in the query string of the topic, for example:
devices/Device2/messages/events/location=BA&$.to=Device1
Receive C2D Messages
This is a demonstration of the sending a Cloud-to-Device (C2D) message generated by the Device Explorer Twin tool and receiving by virtual MQTT Device hosted by the Azure IoT Hub Tester.
Let's type some text, for instance, Hello Device, like it is shown in the following picture:
Clicking on the Send button, the C2D message is sent to the Azure IoT Hub and stored in the device-facing queue.
The virtual device has already subscriber on this topic, so it will pull-up and ACK (this is a MQTT Device) receiving a message. This ACK feedback is sent to the Device Explorer Twin monitor (see the above picture).
Back to the virtual device. In our example, the device myDevice received this C2D message in the messages node, see the following picture:
As you can see, the above C2D Subscriber payload shows our text message such as Hello Device.
That's all for the Virtual MQTT Devices, next I am going to describe an additional feature of the Azure IoT Hub Tester such as REST API requests.
REST-API
The REST-API node allows to send a http(s) request to the url address. The request headers can be simple inserted as a name/value pair with a colon delimiter. For instance, content-type:application/json. Each line represents one header, only.
There is an one special header such as Authorization header in the request. If this header doesn't exist it, the runtime client proxy will generate one based on the connectivity to the Azure IoT Hub.
Using this REST-API node is very straightforward like another REST tool such as setting the Url, headers, method and clicking on the Send button. The request will return a response status and payload.
As I mentioned earlier, the Azure IoT Hub has many endpoints, many of them can be handled also by REST API, so the Azure IoT Hub Tester allows to provide these REST API calls with minimum settings.
Based on the MSDN Document IoT Hub REST, each request is described by a simple json object, which will be used for creating its tree node in the tester. Each of this request definition represents one item in the json array. Note that the name of the file where array is stored must be a name of the Azure IoT Hub namespace, for instance ba2017-iot with an extension json, otherwise the tester will not find it.
The structure of this request template is shown in the following code snippet:
[
{
"category": "DeviceApi",
"name": "QueryDevices",
"method": "POST",
"url": "/devices/query?api-version=2016-11-14",
"headers": "content-type:application/json | x-ms-max-item-count:100",
"payload": {
"query": "SELECT * FROM devices WHERE deviceId='myFirstDevice' "
},
"description": null
},
{
...
}
]
As you can see in the above code, the headers splitter is character pipe (|) and the url address can have only path and query, the protocol with the domain will be added by tester during the runtime. If the url address contains a full address (protocol, domain, etc.), the tester will accept it without any modification.
The article download contains sample of the REST-API templates under the name ba2017-iot.json, so please rename it based on your namespace name.
OK, let's continue with our tool description. Once we made a connection to the Azure IoT Hub, the tester will create a tree node with two child entities such as REST-API and Devices. If the tester will find the template file (in our example file ba2017-iot.json) in the binaries folder, it will take each template item of the array and create its tree node representing.
The following screen snippet shows an example of the above request template for QueryDevices in the category DeviceApi:
As you can see, the above Rest Client panel, the POST request is ready to send it. Based on the needs, the request payload for query can be modified.
Clicking on the Send button, the following screen snippet will show its Response:
Let's explore another request, for instance, request for invoking a direct method on the device.
Select the SendDirectMethod in the TwinApi category and I am assuming you have created some virtual device (in my example, it is a myFirstDevice).
Once you select a SendDirectMethod node, the request will show up with template values. In the url textbox, replace the myFirstDevice for your deviceId if it is necessary, otherwise the request will terminate with error.
Sending the above request to the Azure IoT Hub, the virtual device will receive a method request message and after its response publishing, the following response example will show up in the Response panel of the invoker call:
That's great. The above example demonstrated, that the Azure IoT Hub tool can be used on the both sides such as device-facing (MQTT) side and service-facing (REST) side.
Let's now demonstrate something what we can NOT do it in the Virtual MQTT Device, there is no topic for that, but we can handle it by REST API calls.
Device File Upload
This demonstration requires to configure your blob storage for File Upload feature. From the device point of the view, the file upload is divided into three steps and they are handling by Http protocol.
Step 1
The device will send a request to obtain a reference for creating a full url address (included SAS token) for blob upload call. The following screen snippet shows this example:
After clicking on the Send button, the Azure IoT Hub will send the response:
As you can see, in the above response payload, there is info for generating a url address for blob upload, but also a very important property such as correlationId. This value represents a state machine Id created in the Azure IoT Hub. Note, that maximum number of requests for file upload is 10 and there is a TTL time for each state machine in the case if the completion (Step 3) will not be sent. The TTL time is in the range 1 - 48 hrs.
The following screen snippet shows a tester Log panel, where correlationId and generated url address for blob uploading will be shown:
Step 2
This step is dedicated to blob uploading. Select the next node such as UploadBlobFile.
The device has enough time to upload a blob file to the storage container referenced by Azure IoT Hub. The following screen snippet shows this request call. Note that the specific header for blob type must be added and the url address can be copy/paste to the url textbox.
When the request finishes, the status Created will show up its successful process.
Step 3
This is the last and very important step in the file upload state machine. This step is signaling that the upload process has been completed. Note, when this step is missing, the device has to start the process file upload again from Step 1, etc.
So, select the next node such as NotifyUploadBlobFile and copy/paste a correlationId into the request payload like is shown in the following picture:
Based on this last request, the Azure IoT Hub can generate (if this option is enabled), a file upload notification using the AMQP protocol.
That's all for the Device File Upload.
Send C2D Message using REST
This is an advanced example to show how easy can be used an Azure Function when the Azure IoT Hub endpoint required AMQP protocol. Basically, there are a few endpoints on the service-facing side where we need to handle with AMQP protocol, such as the endpoint for Cloud-To-Device (C2D) messaging, C2D Feedback notifications and File upload notifications.
The following screen snippet shows this concept. We can create an Azure Function (AFN), let's say HtppC2D for http trigger, adding a Nuget package for Azure IoT Hub SDK (Microsoft.Azure.Devices) and transforming http request to the AMQP message.
The http request template should look like it is shown in the following code:
request template:
{
"category": "Misc",
"name": "HttpC2D",
"method": "POST",
"url": "https://xxxxxxxxxx.azurewebsites.net/api/HttpC2D?code=XXXXXXXXXXXXX&name=myDevice",
"headers": "accept: application/json | content-type: application/json |
iothub-messageId:00000000000000000000000000000000 |
iothub-correlationId: 1234567890 | iothub-app-location: BA",
"payload": {
"name": "Hello Device",
"ts": "$DateTime.UtcNow"
},
"description": null
}
Note that the url query string contains parameter name such as a deviceId where the request is going to be forwarded.
The following code snippet shows an example of the HttpC2D function implementation. Note, there is one app settings for configuration string of your Azure IoT Hub.
run.csx
using System;
using System.Text;
using System.Net;
using System.Runtime.Serialization;
using System.Net.Http.Headers;
using Microsoft.Azure.Devices;
using System.Configuration;
public static async Task<httpresponsemessage> Run(HttpRequestMessage req, TraceWriter log)
{
log.Info("C# HTTP trigger function processed a request.");
string name = req.GetQueryNameValuePairs()
.FirstOrDefault(q => string.Compare(q.Key, "name", true) == 0)
.Value;
if (string.IsNullOrEmpty(name))
return req.CreateResponse(HttpStatusCode.BadRequest, "Missing name of the device");
var payload = await req.Content.ReadAsByteArrayAsync();
Message c2dmsg = new Message(payload);
c2dmsg.Ack = DeliveryAcknowledgement.Full;
foreach (KeyValuePair<string, ienumerable="">> header in req.Headers)
{
log.Info($"H: {header.Key} = {header.Value.First()}");
if (header.Key.StartsWith("iothub-app-"))
c2dmsg.Properties.Add(header.Key.Substring(11), header.Value.First());
else if (header.Key.StartsWith("iothub-correlationid"))
c2dmsg.CorrelationId = header.Value.First();
else if (header.Key.StartsWith("iothub-messageid"))
c2dmsg.MessageId = header.Value.First();
else if (header.Key.StartsWith("iothub-userid"))
c2dmsg.UserId = header.Value.First();
else if (header.Key.StartsWith("iothub-to"))
c2dmsg.To = header.Value.First();
}
c2dmsg.Properties.Add("afn-timestamp", DateTime.UtcNow.ToString("o"));
string connectionString = ConfigurationManager.AppSettings["myIoTHub2"];
var client = ServiceClient.CreateFromConnectionString(connectionString);
await client.SendAsync(name, c2dmsg);
return req.CreateResponse(HttpStatusCode.NoContent);
}
function.json
{
"bindings": [
{
"authLevel": "function",
"name": "req",
"type": "httpTrigger",
"direction": "in",
"methods": [
"post"
]
},
{
"name": "$return",
"type": "http",
"direction": "out"
}
],
"disabled": false
}
project.json
{
"frameworks": {
"net46":{
"dependencies": {
"Microsoft.Azure.Devices": "1.2.3"
}
}
}
}
Once we have deployed HttpC2D function and HttpC2D request template in the file (don't forget to use your function url address with your deviceId), we can call it like is shown in the following screen snippet:
and the following picture shows a received C2D message by virtual device (Device1):
In the above advanced example, it has been demonstrated how easily this tool can be extended by using a predefined Http template to call it. We can use the same functionality to create AFN for C2D Feedback notifications, etc.
First of all, the following are prerequisites:
-
Visual Studio 2017
-
M2Mqtt - MQTT Client Library for .NET version 4.3.0
-
Microsoft Azure IoT Hub account
-
Connectivity to the Internet
-
Downloading packages (source and/or exe) for this article (option for creating assemblies)
The implementation concept and design of the Azure IoT Hub Tester is similar to my Azure Service Bus Tester article. It is based on the model, where each device is hosted in own app domain to have a full isolation between each other. This decentralized model required an across the domain communications between the default domain, where the Windows Form is hosted and the device domain. Each device is represented by the remoting object MqttClientActivator, which its instance is marshaling for the specific app domain. '
The following picture shows a MqttClientActivator class:
I am going to describe few methods how this class has been implemented.
The following code snippet shows a method for creating a MqttClientActivator:
public static MqttClientActivator Create(AppDomain appDomain, ConfigData config)
{
string _assemblyName = Assembly.GetAssembly(typeof(MqttClientActivator)).FullName;
MqttClientActivator activator = appDomain.CreateInstanceAndUnwrap
(_assemblyName, typeof(MqttClientActivator).ToString()) as MqttClientActivator;
activator.SetClient(config);
return activator;
}
As you can see, this is a straightforward coding for marshaling object in the specific domain. Once we have an instance of the activator, we can setup it based on the config needs.
That's shown in the following private method:
private void SetClient(ConfigData config)
{
try
{
if (_client == null)
{
_client = new MqttClient(config.BrokerAddress.Trim(),
config.BrokerPort, true, null, null, MqttSslProtocols.TLSv1_2);
_client.ProtocolVersion = MqttProtocolVersion.Version_3_1_1;
_client.ConnectionClosed += Client_ConnectionClosed;
_client.MqttMsgPublishReceived += client_MqttMsgPublishReceived;
_client.MqttMsgPublished += Client_MqttMsgPublished;
_client.MqttMsgSubscribed += Client_MqttMsgSubscribed;
_client.MqttMsgUnsubscribed += client_MqttMsgUnsubscribed;
_name = config.BrokerAddress + "/" + config.Name;
this._configData = config;
this.AddToStorage(this);
LogMessage($"[{this.Name}] Client has been created in the appDomain
{AppDomain.CurrentDomain.FriendlyName}");
}
else
{
throw new InvalidOperationException("The MqttClient has been already setup");
}
}
finally
{
CallContext.FreeNamedDataSlot("_config");
}
}
As you can see the above implementation, the activator is a wrapper box around the MqttClient object allows to us to communicate it from the other app domains. Once we setup all callbacks for the MqttClient proxy, we can store its reference into the process data slot.
The following code snippet shows an implementation of the public method for Connect device to the Azure Iot Hub. Note that this method can be called by any appdomain:
public void Connect(string password = null)
{
if (_client != null)
{
try
{
if (_client.IsConnected == false)
{
if (string.IsNullOrEmpty(password) == false)
this._configData.Password = password;
byte connCode = _client.Connect(this._configData.Name, this._configData.Username,
this._configData.Password, false, MqttMsgBase.QOS_LEVEL_AT_LEAST_ONCE,
false, "$iothub/twin/GET/?$rid=911", "Disconnected", false, 60);
if(connCode != 0)
throw new Exception($"Connect failed, code = {connCode}");
_topics = new string[] { "$iothub/methods/POST/#",
$"devices/{_configData.Name}/messages/devicebound/#",
"$iothub/twin/PATCH/properties/desired/#", "$iothub/twin/res/#" };
ushort subCode = _client.Subscribe(_topics, new byte[]
{ MqttMsgBase.QOS_LEVEL_AT_LEAST_ONCE,
MqttMsgBase.QOS_LEVEL_AT_LEAST_ONCE,
MqttMsgBase.QOS_LEVEL_AT_LEAST_ONCE,
MqttMsgBase.QOS_LEVEL_AT_LEAST_ONCE });
LogMessage($"[{this.Name}] Connected", "HighlightInfo");
}
}
catch (Exception ex)
{
RemoveFromStorage(this);
LogMessage($"[{this.Name}] Connecting device failed: {ex.Message}", "Error");
throw ex;
}
}
}
As you can see, the above implementation follows all requested topics by Azure IoT Hub document for connection and subscribers. So, once the device (proxy) finished the above method, the message can be received and handled by the following callbacks:
private void Client_ConnectionClosed(object sender, EventArgs e)
{
LogMessage($"[{this.Name}] Connection closed", "Warning");
var payload = new MqttMsgEventArgs("$iothub/clientproxy/",
Encoding.UTF8.GetBytes("Disconnected"), false, 0, false);
this.ForwardMessageAsync(payload).Wait();
}
void client_MqttMsgPublishReceived(object sender, MqttMsgPublishEventArgs e)
{
LogMessage($"[{this.Name}] Subscriber received at {e.Topic}", "HighlightInfo");
this.ForwardMessageAsync(new MqttMsgEventArgs(e)).Wait();
}
private void Client_MqttMsgPublished(object sender, MqttMsgPublishedEventArgs e)
{
LogMessage($"[{this.Name}] Response from publish {e.MessageId.ToString()}");
}
private void Client_MqttMsgSubscribed(object sender, MqttMsgSubscribedEventArgs e)
{
LogMessage($"[{this.Name}] Response from subscribe {e.MessageId.ToString()}");
}
private void client_MqttMsgUnsubscribed(object sender, MqttMsgUnsubscribedEventArgs e)
{
LogMessage($"[{this.Name}] Response from unsibscribe {e.MessageId.ToString()}");
}
}
As you can see, the above implementation is using a private method ForwardMessageAsync to forward a message to the default (UI) domain for its representation in the treeview model. The communication channel is used WCF model via name pipe.
The next code snippet shows a public remoting method for publishing message on the topic:
public ushort Publish(string topic, string payload, byte qos, bool retain)
{
if (_client.IsConnected)
return _client.Publish(topic, Encoding.UTF8.GetBytes(payload), qos, retain);
else
return 0;
}
As you can see, the above implementation is actual a marshaling wrapper around the original method from the M2Mqtt library.
So, the remoting object MqttClientActivator implemented all public methods necessary for communication with a MqttClient proxy across the app domain boundary. The following example shows how easy is this invoking from the default (UI) domain, for instance, when we click for publishing message:
MqttClientActivator client = HostServices.Current.GetClient(name);
var code = client.Publish(e.Topic, jsontext, e.QoS, e.Retain);
The first step is to obtain a reference to the MqttClientActivator based on the name. Once we have it, we can call any of its public remoting method, for instance the Publish method.
That's all for implementation.
This article gives to you a tiny tester for Azure IoT Hub. It can be your helper while evaluating and exploring the Azure IoT Hub endpoints, for developing MQTT Devices, troubleshooting a IoT Data, or simulation of the device C2D and/or D2C messaging. I hope you will find it useful.
There are many new features in the Azure IoT Hub, so this is a first update of the Tester in order to support the upcoming Plug and Play IoT Devices functionalities.
Let's describe each new feature of the Azure IoT Hub individually and how it is handled by Azure IoT Hub Tester:
A1. Device Modules
Based on the new Azure IoT Hub features such as modules and streams, the device treeNode has been extended. The following screen snippet shows an example for device1:
Note that the Modules node is handled the same as with Devices, so by selecting this node, we can see on the right side all registered modules below this device. The following screen snippet shows an example of the modules within the device1. Selecting the module and double clicking on the row, the module will be added to the device simulation:
A2. Device Streams
Presently (October 2019), the Device Streams feature is still in the public preview, so it can change once it reaches GA. The Device Streams feature can be simulated/tested by this tester. The concept of the device streams is similar to the direct method, where this sync communication is used for handshaking and switching to the web socket communication between the device and cloud backend app via the Azure IoT Hub coordinator.
The following screen snippet shows the REST Client (invoker and consumer) of the device stream:
When the device accepted, this client inquire will send back response (like a device method) to acknowledge and switching to the web sockets receiver:
For test purposes, you can send by readwrite ServiceStream Buffer to the readonly DeviceStream Buffer some content.
Note, that the device similar to the Upload File feature, doesn't need to know any details about the endpoint and authorization token. All communication metadata are transparent and dynamic to the device. They are obtained during the handshaking step from the client invoker via the Azure IoT Hub.
To have some implementation picture about the device streams, the following code snippet shows a piece of the receiver located in the Form1.cs file - method DeviceStreamWorker:
using (var wsClient = new ClientWebSocket())
{
wsClient.Options.SetRequestHeader("Authorization", $"Bearer {nstate.Tag}");
wsClient.ConnectAsync(new Uri(node.Text), ct.Token).Wait();
receiveResult = wsClient.ReceiveAsync(new ArraySegment<byte>(buffer, 0, buffer.Length),
ct.Token).Result;
nstate.Payload = Encoding.UTF8.GetString(buffer, 0, receiveResult.Count);
wsClient.CloseAsync(WebSocketCloseStatus.NormalClosure, String.Empty,
ct.Token).ConfigureAwait(false);
}
Note, that the Bearer token and Uri address are obtained from the message received by device subscriber on the topic: $iothub/streams/POST/#.
A3. Using the Device Connection String for Device Connectivity
The Azure IoT Hub Tester allows to connect devices based on the Shared access policy to the Azure IoT Hub. Once we have this policy, such as iothubowner policy, the tester can do anything from the device and service facing endpoints sides of the Azure IoT Hub.
In the case, if there is no available this connection string to the Azure IoT Hub (but we have only a device connection string), or in other words, we have access only to the device-facing endpoint of the Azure IoT Hub, this new tester updates will help you.
In this case, just entry the device connection string in the Namespace dialog grid and type the namespace of the Azure IoT Hub from this connection string.
The following screen snippet shows this instruction. As you can see, the first entry is the connection string for Azure IoT Hub with a policy iothubowner.
Note, when you added more devices into the NamespaceDialog below the same namespace, all of them will show up in the Devices grid for their connection availability.
A4. Connecting Virtual Devices to the Azure IoT Central
Azure IoT Central (Azure IoTC) is a SaaS application built on the top of the Azure IoT Hub to simplify IoT development. The Azure IoTC abstracted all complexity around its internal Azure IoT Hub, which enabled full control over the internal IoT Hub by IoTC. The Azure IoTC is giving to you a specific access to its infrastructure. One of the access is via metadata (scopeId, deviceId and primary key) to create a device connection string for device connectivity to the Azure IoTC. Let's look at how we can use it for our Azure IoT Hub Tester with a virtual MQTT devices.
I am assuming that you have some knowledge and experience with an Azure IoT Central. I will use my IoT Central with two devices such as device1 and device2.
Let's connect a device1 simulated by Azure IoT Hub Tester to the Azure IoT Central. The following screen snippet shows my device1:
To obtain the metadata for creating a device connection string, click on Connect. You should get the detail dialog for device connectivity with an Azure IoT Central, see my dialog:
I have highlighted the following properties in the above Device Connection dialog for our needs such as ScopeID, DeviceID and PrimaryKey.
Now, back to our Azure IoT Hub Tester and opening the NamespaceDialog to enter the device connection string.
Use Copy/Paste those properties from the IoTC to the following textbox in the order to see how they show up:
Note, there is 'space' delimiter between each property. Once all properties have been inserted, the button Get is enabled and waiting for pressing to generate a device connection string. Pressing on the button Get, the device connection is created in the yellow textbox (or error message). If this connection string doesn't exist in the namespace grid, the button Add is enabled:
Pressing on the button Add:
As you can see, the above Namespace grid contains a device connection string to the IoT Central, so when this row is selected and clicking on the button OK, we will enable to select device like from the Azure IoT Hub, see the following screen snippet:
That's great! Now we can simulate device to the Azure IoT Central, exploring the device twins, etc.
As I have mentioned, the IoTC abstracted all entities around the internal IoT Hub, device twins, etc., so based on the above picture, where the twin has been obtained from the internal IoT Hub, the following screen snippet shows how desired properties are mapped to the device Settings:
As you can see, the desired properties are object:
"desired":{
"num":{ "value":12 },
"text":{ "value":"ABCD" },
"mytoggle":{ "value":true }
}
In the case of the reported properties, they are a primitives, see the following:
"reported":{
"battery":90,
"fan":true,
"echo":"ABCD1234",
"echo2":"1111",
"maxtemp":25
}
I have demonstrated some differences between the Azure IoT Hub and Azure IoT Central abstraction for device twins. Note that the big difference is for responding and behavior of the direct method.
A4.1 Using a master (group) key for creating a device connection string for Azure IoT Central
The IoTC offers a master (group) key for generating a device connection string for any registered device in the IoT Central. The following screen snippet show a way how to get it:
The above master key can be inserted to the textbox with a prefix character '@'. See, the following screen snippet. Note that the deviceId is typed manually:
A5. Using a Plug and Play IoT Device Connected to the Azure IoT Central
The Plug and Play IoT Feature is still in the public preview. Its concept is based on the IoT Plug and Play schema created by Microsoft upcoming Digital Twin Definition Language (DTDL) where the core schema describes the device capabilities. The Azure IoT Central has opened a public preview for Plug and Play IoT Devices. Let's go through this great upcoming feature. Note, that the device CapabilityModel schema is common schema for device and IoT application sides.
The above screen snippet shows adding a new IoT PnP device to the templates collection. In my example, the AzureKit ESP32 device was selected.
Based on this template, we can create a New Device, see the following dialog:
After this step, this IoTC application (such as iotc-preview) has one registered device azurekit-1. Note, that this application has 7 days free trail period with maximum 5 devices.
From now, we can see how the CapabilityModel of the AzureKit device has been plugged into the IoTC app. The following screen snippet shows About the device:
As you can see, the About properties are empty, that's fine because we don't use it yet.
The same situation with missing data is also on the next page such as Overview:
The next click on the Commands shows all invoking methods on the device with their inputs. Note, that clicking on the Run button, the response will immediately end with an error because the device is not connected and all commands are in the sync call pattern.
To add the IoTC device azurekit-1 into the Azure IoT Hub Tester, we can use the same steps as described in the paragraph A4 in this article.
The following screen snippet shows this entry to the Tester:
And after that, we can connect the device azurekit-1 to the IoT Central application. Note that this device is still in our tester and is NOT the IoT Plug and Play device, it is a regular device. In other words, there is no Capability Model inserted with this device.
Ok, let's add the Capability Model to the device connected by our Tester.
First of all, the IoT Central needs to export this Capability Model, see the following screen snippet:
The following picture shows an overview of the logical integration between the IoT Central app and our Tester, where the Capability Model represents a Plug and Play mechanism:
After what the Load Samples loaded into the CapabilityModel, the connected device is reconfigured based on this schema. See the following screen snippet:
As you can see, the Model node has been added to the device node, the methods node shows all possible methods for this device handled by model schema and of course, the telemetry sample has been created. Note, that the telemetry data can be generated randomly or explicitly inserted.
The full CapabilityModel of the IoT Plug and Play device can be explored (and edited) in the Model node.
As you can see, the Model node has two sub-nodes. These nodes show reported properties, so by using copy/paste, we can populate the device twins, see the following result:
Note, that the device twin reported properties cannot by removed in one simple null setup, like we can do it for desired properties, the tester has a capability to setup null for each property in the reported collection.
OK, now back to the iotc-preview application to see how About result was changed:
To see changes on the Overview dashboard, press the button on Publish multiple times. Note, that wrapping the json telemetry object by array, the samples will be automatically generated based on the appsettings configuration in the config file.
and the dashboard shows a telemetry data published by the Azure IoT Hub Tester from the Plug and Play azurkit-1 device:
That's all for this version. I am expecting soon new Microsoft updates for Plug and Play IoT Devices. It will be nice to have capability to store the Device Capability Model into the device twins. It will allow us to get the Capability model by device based on the ad-hoc needs. OK, we will see how this feature will turn out. I am sure, the next version for this Tester will follow as well.
- 31st March, 2017: Initial version
