1. Overview
The area which got IoT an edge over the conventional embedded system is the Cloud. Bunch of services running in the core gives immense computational power to miniature physical devices. IoT is all about data, more data, it's processing, analytics and then decision making.
Leveraging this capability of cloud and vast sets of services available for data collection, processing, analysis and machine learning, a new domain is consistantly emerging - which is health care.
A wide range of fitness tracking devices are available today that collects activity data from the user, sends it to their cloud, analyze and provides a statistical overview of their health condition.
But with increasing number of devices and their user, data volume exceeds imagination. Data traffic suppresses all records, security of the data and protection of personal details of the user becomes a concern and computation keeps becoming expenssive.
A new architecture concept has emerged out of these challenges in recent time which is called Fog Computing. This is essentially a reference architecture that offers data storage, analytics and decision making at the edge of the network.
This white paper by CISCO titled "Fog Computing and the Internet of Things: Extend the Cloud to Where the Things Are" can give you a very good idea about how some of the major IT giants have started thinking about the services that can run at the edge of the network.
This architecture phillosophy is rather simple: process the data coming from sensor devices at the edge ( in local network), filter and process the data locally and take decisions that can be taken locally. The interpreted and aggregated data is then sent to the cloud(if necessary). This reduces data traffic, decision latency and saves the user from usual privacy concerns of his data being lost.
CISCO's paper made me start thinking about this area off late and I really got interested in the entire concept. If you can run set of utility services in your local network then you can customize them as per user needs or geography or other factors. Healthcare and activity tracking being one of the areas of IoT that has drawn significant attention was automatically an area of attention for me.
Therefore I wanted to explore the area of Fog Computing for IoT in the context of a health care/ fitness tracking solution.
During my research with current developments in this direction I realized that most of the fitness apps available today are all cloud based app. be it Basis Peak, or Fit Bit, or any other fitness tracker, the devices collect data of user movement, stores it locally, then send the data to backbone cloud, process it and tell the user about their activity summery.
When you buy a fitness tracker gadget or use a fitness tracker app, the app shows you, your current activity like walking, jogging, swimming etc along with the time for which you are into that activity and probable calorie loss.
Once the data is sent to cloud and is analyzed, the aggregated analysis becomes available which more or less gives the "pointers", like how healthy is the person, how much sleep does he needs, how much walk does one neds to have and so one.
One of the common strategy that the architecture adopts is they bundle the fitness tracking with a mobile companion app. The app collects the data from the device, may or may not perform some raw operation on the data and finally pushes the data to the cloud.
So, it occours that if there is already an edge layer working which is reponssible for gathering data from wearable fitness devices, why not develop an architecture at the edge itself?
As most of current fitness devices are linked to their respective clouds, I had to pick up the design from the basic right at the activity tracking level. As the objective of mine was mainly to create a "fog based activity tracking" architecture, I wanted to go with Android phones and wanted to use it as activity tracking device. Unfortunately the Google fitness API that allows Android phone to be used as a fitness device is also linked to Google cloud which eliminated the use of it. So the overall problem got extended into some distinct problem sets
- Creating an Android based activity classification right at the Android phone without using any cloud based APIs
- Send the activity log to a local service
- Aggregate the activity data at a local service, and finally
- take decisions at the local service
Now, the next question becomes what kind of decisions can be linked with local activity tracking architecture. If you are creating an activity tracking, you want the algorithms to be roboust, well tested and certified before you can offer it to the users. So, critical medical decisions like disorder detection like sleep disorder was little difficult to develop at the edge. It is at this time during my research I came across this fantastic research paper titled "Estimation of Metabolic Rate for Location-based Human Adaptive Air-conditioner in Smart Home"[IEEE].
This paper adopts proximity sensor based tracking of human movement in a room and controls air conditioning using the accumulated data. According to the paper, the comfort level of the human beings depends upon their metabolic rate which in turn is dependent of their gender, age and amount of movement or activity.
I thought "wouldn't it be nice if my air conditioner knows what temperature to set depending upon how much work I am carrying out?". For example if I am induldged in many physical activities, I will be burning more calories, increasing the body temperature. On the other hand, I would require less colling if I am resting and my body temperature is.
As such a system would require an activity tracking app to tell it about the nature of activity and it's effect on the desired body temperature. This leads towards the work we are going to present here: fHealth.
This article will not only present the idea about the fHealth but we are also going to see how to approch the design of such a distributed system and how to prototype the idea.
Having lots of iHealth-care, eHealth-Care,mHealth-Care, cHeath-Care systems it's now time for more personalized framework, fHealth- The fog based health care system which we will demonstrate by integrating an activity tracking app with smart climate control system. Read On!
2. Architecture Design
Figure 2.1 Architecture fHealth System
The overall design contains local service mesh that consume data coming from an Android activity tracking app. This data is than pushed to a decision making service which in this case will be a C# UI client that processes that data and takes decision about the desired temperature ( perceived as desired fan speed from hence forth). The decision is sent to Intel Edison connected to fog. The device is connected with a temperature sensor which based on tempereature value and desired fan speed being sent from the decision unit controls a fan symbolic to the climate control unit.
As you can see the symbolic climate control system is an add on which is controlled by the decision taken on the local decision system. The local decision ( which is essentially aggregated activity data) can be used in wide range of applications including "Lazyness Alert" and such systems.
Figure 2.2 Detailed Components of the framework
Figure 2.2 represents the detailed components part of the project. User carries his mobile where accelerometer data is tracked. Classifier is implemented at the mobile that classifies the user activity based on sensor data into following categories:
- Walking
- Running
- Leg Shaking
- Resting
This data is sent to a local Mqtt server ( as the reference needs a local service to handle the connection). The data is then consumed by an analysis system which is implemented in C#. This may generate alert for the user or as in 2.1 can send it to a physical device that controls an enviromental parameter like temperature.
In terms of programming, we need to create an activity tracker in Android, a local Mqtt broker, a C# client as decision system, a Node.js Intel Edison application for climate control. let us take up the topics one by one in the proceeding sections.
3. Activity Tracker Android App
Figure 3.1 Android Studio activity_main layout
The proposed Android prototype app is a single "form", single layout app. The objective of this app is to periodically collect sensor data, classify them and then send it over the local cloud to a local service. We are using Mqtt as a the gateway protocol. The layout has a single EditText called edServer and a Button btnConnect to connect to the server.
The layout xml is as given bellow
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent"
android:layout_height="match_parent" android:paddingLeft="@dimen/activity_horizontal_margin"
android:paddingRight="@dimen/activity_horizontal_margin"
android:paddingTop="@dimen/activity_vertical_margin"
android:paddingBottom="@dimen/activity_vertical_margin" tools:context=".MainActivity">
<TextView android:text="Activity Classifier Topic: rupam/activity]" android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/tvHello" />
<TextView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:textAppearance="?android:attr/textAppearanceSmall"
android:text="MqTT Server"
android:id="@+id/tvServer"
android:layout_marginTop="56dp"
android:layout_below="@+id/tvHello"
android:layout_alignParentLeft="true"
android:layout_alignParentStart="true" />
<EditText
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/edServer"
android:width="200dp"
android:layout_alignTop="@+id/tvServer"
android:layout_toRightOf="@+id/tvServer"
android:layout_toEndOf="@+id/tvServer"
android:text="192.168.1.3" />
<Button
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="Connect"
android:id="@+id/btnConnect"
android:layout_below="@+id/edServer"
android:layout_centerHorizontal="true" />
<TextView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:textAppearance="?android:attr/textAppearanceSmall"
android:text="Status"
android:id="@+id/tvStatus"
android:layout_centerVertical="true"
android:layout_centerHorizontal="true" />
</RelativeLayout>
You need to first add Mqtt library to the project. We use paho client for java by Eclipse as Mqtt library. Select Project as the tab from top left corner as shown in figure 3,2 and drag the downloaded jar file inside the lib folder inside app. If you don't see the lib folder, then go to your project directory from explorer, go inside the app directory and create a folder called lib. Now copy the jar file there.
3.2 Adding Mqtt Paho jar file to project
Now right click on the jar file in project tab and add as library.
As we will need to access network features in the app, you need user's permission for accessing network services. Add "android.permission.INTERNET" in uses-permission section before <application> tag. Add android:screenOrientation="portrait" to fix the layout as potrait. All the sensor classification is based on the consideration that user has kept the phone vertical. It also prevents the app reset on a screen rotation.
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="in.co.integratedideas.activityclassifier" >
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<application
android:allowBackup="true"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:theme="@style/AppTheme" >
<activity
android:name=".MainActivity"
android:label="@string/app_name"
android:screenOrientation="portrait"
>
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
</manifest>
Now, in the MainActivity.java implement SensorEventListener. Which requires yiou to implement onSensorChanged method.
Declare Sensormanager class object in the Activity class.
private SensorManager sensorManager;
in onCreate method, register accelerometer sensor event for sensorManager. This automatically triggers onSensorChanged event listener for every change in accelerometer data.
In onCreate, alos initialize the edServer and btnConnect members. Add event listener to btnConnect so that whenever it is clicked, the app is connected to the Mqtt broker specified in edServer.
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
sensorManager = (SensorManager) getSystemService(SENSOR_SERVICE);
sensorManager.registerListener(this, sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER), sensorManager.SENSOR_DELAY_UI);
edServer=(EditText)findViewById(R.id.edServer);
btnConnect=(Button)findViewById(R.id.btnConnect);
tvStatus=(TextView)findViewById(R.id.tvStatus);
btnConnect.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
broker="tcp://"+edServer.getText().toString().trim()+":1883";
ConnectionClass con=new ConnectionClass();
con.execute();
}
});
}
As the broker address requires a tcp:// precedence by Paho java Mqtt client, concat this with the edSever value before initiating the connection.
The connection is established by executing an inner class by name ConnectionClass which extends AsyncTask. The connection is established as a background process inside doInBackground overrriden method of the class. This is because the Android application doesn't allow any network related operation to be carried out in the main thread.
public class ConnectionClass extends AsyncTask<ActivityGroup, String, String>
{
Exception exc = null;
@Override
protected void onPreExecute() {
super.onPreExecute();
}
@Override protected String doInBackground(ActivityGroup... params) {
try
{
if (Looper.myLooper()==null)
Looper.prepare();
sampleClient = new MqttClient(broker, clientId, persistence);
MqttConnectOptions connOpts = new MqttConnectOptions();
connOpts.setCleanSession(true);
IMqttToken imt=sampleClient.connectWithResult(connOpts);
Log.d("MQTT MODULE.....","....DONE..."+sampleClient.getServerURI()+"--"+imt.getResponse().getPayload());
if(sampleClient.isConnected()) {
return "CONNECTED";
}
else
{
return "Connection Failed.....";
}
}
catch(Exception ex )
{
Log.d("MQTT MODULE", "CONNECTION FAILED " + ex.getMessage() + " broker: " + broker + " clientId " + clientId);
return "FAILED";
}
}
@Override protected void onPostExecute(String result) {
super.onPostExecute(result);
if(result!= null)
{
isConnected=true;
tvStatus.setText(result);
}
}
}
boolean isConnected;
in the postExecute method, which is called after the connection attempt is completed ( either success or fail) we put the connection status in tvStatus which is a TextView object for logging any status messages including printing current detected activity, result of Mqtt operations, connections and so on.
sampleClient is the Mqtt client that is used to publish messages to Mqtt broker synchronously and receive published data for any subscribed channel.
Once the app is triggered and connection is established, sensor data keeps changing, triggering onSensorchanged event handler. Let us go inside the event handler to see how user activity is classified based on the accelerometer data.
float prevX=-1,prevY=-1,prevZ=-1;
float DX=0;
long lastUpdate = System.currentTimeMillis();
int WAIT=1000;
boolean detected=false;
@Override
public void onSensorChanged(SensorEvent event) {
String command="";
if(!isConnected)
{
}
float[] values = event.values;
float x = Round(values[0], 1);
float y = Round(values[1], 1);
float z = Round(values[2], 1);
command="";
long actualTime = System.currentTimeMillis();
if ((actualTime - lastUpdate) > WAIT)
{
long diffTime = (actualTime - lastUpdate);
lastUpdate = actualTime;
boolean sit_stand_sleep=false;
float diffX=x-prevX;
float diffY=y-prevY;
float diffz=z-prevZ;
double xz=Math.abs(diffX)* Math.abs(diffz);
double xyz=Math.abs(diffX)+Math.abs(diffz)+Math.abs(diffY);
if(xz>1.1 )
{
if(xz>40)
{
Log.d("RECOGNIZED", "RUNNING "+xz);
command="RUNNING_"+xz;
}
else {
Log.d("RECOGNIZED", "WALKING " + xz);
command="WALKING_"+xz;
}
sit_stand_sleep=true;
}
else
{
if(xz>.1)
{
Log.d("RECOGNIZED", "SHAKING LEGS");
command="SHAKING LEGS";
}
else
{
if(xyz<.3)
{
Log.d("RECOGNIZED", "MOBILE NOT WITH USER");
command="MOBILE NOT WITH USER";
}
else {
Log.d("RECOGNIZED", "RESTING");
command="RESTING";
}
}
}
if(prevX!=-1)
{
DX=DX+diffX;
prevX = x;
prevZ = y;
prevZ = z;
}
else
{
prevX = x;
prevY = y;
prevZ = z;
}
if(command.length()>1)
{
Send(command);
command = "";
}
}
}
Our hypothesis is based on the observation that whenever user moves ( even for ever so slight, there will be a variation in sensor data. If the mobile is not with the user( placed on a table), there will be no movement of xyz at all. This result in near zero value for the xyz product. If an extremely low value for this is observed then the system generates a message "Mobile not with User".
Figure 3.3 : X-Y-Z directions when mobile is vertical ( source: stackoverflow)
When the mobile is kept vertical ( see figure 3.3) , any movement of the user causes a substantial change in x and z. if user is at the resting position, there will still be changes in sensor data which will be more than when the phone is idle but extremely lesser than when the user is actually moving. This is how RESTING and 'Phone not with user' activity scenerios are detected.
if(xyz<.3)
{
Log.d("RECOGNIZED", "MOBILE NOT WITH USER");
command="MOBILE NOT WITH USER";
}
else {
Log.d("RECOGNIZED", "RESTING");
command="RESTING";
}
When user is moving very fast ( because within indoor context, you don't technically have a case called running) the xz variation is significantly high. We tap into this variation and classify the activity as running in case xz is significantly high.
In case xz is more than the leg shaking case and less than the running, then it is case of walking. Leg shaking was introduced to detect user doing some work with moderate mobility, for instance reading some books or painting.
if(xz>40)
{
Log.d("RECOGNIZED", "RUNNING "+xz);
command="RUNNING_"+xz;
}
else {
Log.d("RECOGNIZED", "WALKING " + xz);
command="WALKING_"+xz;
}
Therefore by using FUZZY rule set we classify the activity into WALKING, RUNNING, RESTING, LEG SHAKING and MOBILE NOT WITH USER.
Interestingly you may have noticed that when you walk very slow, you sweat lesser, when you walk faster you sweat more. The logic is that not only the metabolic rate( which ultimately result in burn in the calorie and thus the body temperature) not only depends upon the activity but also the intensity of it.
Therefore with both WALKING and RUNNING we also embed the intensity of the activity. As resting always results in near zero excess calorie loss( other than what body burns due to normal biological operations) we do not incorporate any intensity value with this activity and also with leg shaking activity.
The parameters xz as well as xyz are product of difference of the their respective accelerometer data difference from current value and the previous value.
double xz=Math.abs(diffX)* Math.abs(diffz);
double xyz=Math.abs(diffX)+Math.abs(diffz)+Math.abs(diffY);
The difference is calculated as bellow.
float diffX=x-prevX;
float diffY=y-prevY;
float diffz=z-prevZ;
As you might have noticed, the resultant classified activity is stored in a String variable called command.
After the activity is classified, the app checks for the connectivity with the broker and if the app is connected with the broker, it publishes the activity data to the broker.
if(command.length()>1)
{
Send(command);
command = "";
}
Figure 3.4: Screenshot of running App
The above figure shows the screenshot of the running app. you can also add tvStatus.setText(command) before calling the Send() method in order to display the current activity. The above screen shows my screen while testing. The server ( technically broker) address is the address of our local Mqtt broker.
Send method checks if the sampleClient is connected or not. if connected, publishes the command.
void Send(String content)
{
final String data=content;
AsyncTask.execute(new Runnable()
{
@Override
public void run()
{
try
{
if(sampleClient.isConnected())
{
MqttMessage message = new MqttMessage(data.getBytes());
message.setQos(qos);
sampleClient.publish(topic, message);
}
else
{
}
Log.d("MQTT MODULE",data+" SENT");
}
catch(Exception ex)
{
}
}
});
}
4. Mqtt Broker
We have published the process of installing and running Mqtt broker and instance in windows in our foot gesture based text entry system tutorial. The section is reproduced here for the sake of continuity of this article.
For testing the app, you can use iot.eclipse.org. But As the broker is remote, there is a significant latency in data communication. The foot gesture based text entry system is in itself little time consuming. So, you wouldn't want to add a network latency with remote server. Right?
So, we will go with a local Mqtt broker for Windows. Windows allows only 1063 connections (I do not know why) or clients. But, as we would connect only one client with our broker, Windows Mqtt server running in local machine will just do fine.
You need to install Win32 Open SSL first.
Now download and install Mosquitto Broker for Windows
Now go to OpenSSL installed directory which must be in C:\OpenSSL-Win32 and copy pthreadVC2.dll file and replace the same in Mosquitto installation directory. That's it.
From command prompt, go to Mosquitto directory and run mosquitto.exe in order to see the broker running in your system. To close the broker, use ctrl+c. IP address of the broker will be your PC's local IP address which you can obtain with ifconfig command.
5. C# Client for Analyzing Activity and Taking Decision on Desired temperature
Firstly we define a class for Activity. An activity will have a start time, end time, name and intensity. RESTING and LEG SHAKING will have intensity 0 ( which means that their intensity will not effect the temperature).
Start time is when the current activity was started. End time is when it ends. When user is in a particular state, say for example RESTING, the activity will have a start time, the end time will be null. When user changes the activity, say from resting to walking, the end time for resting will be logged and a new activity by name walking will be started with it's start time.
As WALKING and RUNNING both have intensity values, as there is any change in the current intensity, the end time will be marked and another instance of WALKING will be created with new intensity.
public class Activity
{
public String Action { get; set; }
public string Start { get; set; }
public string Stop { get; set; }
public string TotalPeriod { get; set; }
public double Intensity { get; set; }
public Activity()
{
}
public Activity(string Action, string Start, string Stop, string TotalPeriod, double Intensity)
{
this.Action = Action;
this.Start = Start;
this.Stop = Stop;
this.TotalPeriod = TotalPeriod;
this.Intensity = Intensity;
}
}
Because start and end time will not always be defined, we decalre Start and Stop as String rather than DateTime.
We define a List of the Activity class to log consecutive activity instances.
List<Activity> lv = new List<Activity>();
A StopWatch instance is created to keep track of the TotalPeriod of the ongoing activity.
Stopwatch st = new Stopwatch();
The App is a single form app which more or less acts like a service. The app consumes activity data published by the activity tracker mobile app, aggregates the data, takes the decision and as a demonstration of global gateway release publishes the data to iot.eclipse.org ( an outside broker). Our Edison service is connected to iot.eclipse.org, making the entire fog a combination of services running locally and in cloud, collaborating with each other in order to offer a complete solution.
When the app starts the IP address of the PC is fetched by calling LocalIPAddress() method which loops through all the DNS entries and look for the IP address entry in the address table. The Mqtt broker running in the PC will have the same IP address as the PC address. therefore fetching and displaying IP address makes it easy for the user to connect to the broker from his Android activity tracker app.
Figure 5.1: C# App UI
public string LocalIPAddress()
{
IPHostEntry host;
string localIP = "";
host = Dns.GetHostEntry(Dns.GetHostName());
foreach (IPAddress ip in host.AddressList)
{
if (ip.AddressFamily == AddressFamily.InterNetwork)
{
localIP = ip.ToString();
break;
}
}
return localIP;
}
private void Form1_Load(object sender, EventArgs e)
{
labIP.Text = LocalIPAddress();
}
Note that, because Android app and C# app are collaborating within a fog framework, both must be part of the same WiFi network.
Note that there are mainly two buttons with lables Start broker and Connect to broker. Essentially you can start your Mosquitto broker from the command prompt. But as we are perceiving our C# app more like a fog service, i wanted to start the server from our app itself and attach it with the window.
With .Net's Diagonestic.Process() class starting an external service is pretty simple.
SetParent() method is used to attach the window handle of a forked process with the UI component. We are also killing any Mosquitto services already running in the PC before starting a new one as we have assumed 1883 as the default port for the mqtt broker. A new broker instance can not start if a broker is already running because it can not get the desired free port.
[DllImport("user32.dll", SetLastError = true)]
static extern IntPtr SetParent(IntPtr hWndChild, IntPtr hWndNewParent);
private void button1_Click(object sender, EventArgs e)
{
string s=Environment.GetFolderPath(Environment.SpecialFolder.ProgramFilesX86);
s = s + "\\mosquitto\\mosquitto.exe";
procMqtt = new System.Diagnostics.Process();
procMqtt.StartInfo = new System.Diagnostics.ProcessStartInfo(s," -v");
procMqtt.StartInfo.UseShellExecute = true;
Process[] pname = Process.GetProcessesByName("mosquitto");
for (int i = 0; i < pname.Length; i++)
{
pname[i].Kill();
}
procMqtt.Start();
System.Threading.Thread.Sleep(2000);
SetParent(procMqtt.MainWindowHandle, groupBox2.Handle);
}
Once the broker instance is started the form looks similar to figure 5.2
Figure 5.2 Starting broker instance from the app and attaching it to the group box
Recall that we are using two Mqtt brokers in our case: One the local broker which you have started from the app. This broker is the fog interface between local activity tracker service and C# client.
The other broker is iot.eclipse.org which is used as the global broker. C# client sends desired temperature level to Edison device using Eclipse gateway. This is no way necessory and you can connect your entire architecture in the local context. But because by design a fog is expected to mitigate the analyzed data to global cloud we have used two different gateways.
Therefore our PC C# app has two Mqtt clients: mc and mqSpeedSender. mc is connected to local broker you started from the app whose IP address will be local IP address of the PC and mqSpeedSender's address will be iot.eclipse.org.
You can observe in figure 5.1 and 5.2 that the IP address being shown of my system is 10.0.x.y. However, the local IP address will be of the format 192.168.x.y. This is because I have installed Gulp in order to test some of my Python apps globally before pushing them to AWS with Cloud Foundry. However, by defination the fog service must be run within the local context. So in order to get the exact local address, you may change the LocalIPAddress() method and check for an entry with 192 as the first byte of the address as bellow.
public string LocalIPAddress()
{
IPHostEntry host;
string localIP = "";
host = Dns.GetHostEntry(Dns.GetHostName());
foreach (IPAddress ip in host.AddressList)
{
if (ip.AddressFamily == AddressFamily.InterNetwork)
{
if (ip.ToString().Contains("192"))
{
localIP = ip.ToString();
break;
}
}
}
return localIP;
}
Once done you will see your form showing the local IP address which is the prerequisite of a fog framework. if you see an IP address something like below, then you are all set.
Figure 5.3: Correct Local IP Address after fixing LocalIPAddress() Method
Now, when Connect to Broker button is clicked, the app must get connected with forked broker with a Mqtt client instance called mc and with global Eclipse broker with another instance called mqttSpeedSend
MqttClient mc = null;
MqttClient mqSpeedSender = null;
System.Diagnostics.Process procMqtt;
private void button2_Click(object sender, EventArgs e)
{
try
{
var ip = IPAddress.Parse(labIP.Text);
mc = new MqttClient(ip);
mc.Connect("RUPAM");
mc.Subscribe(new string[]{topic},new byte[]{(byte)0});
mc.MqttMsgPublishReceived += mc_MqttMsgPublishReceived;
mc.MqttMsgSubscribed += mc_MqttMsgSubscribed;
MessageBox.Show("Connected");
mc.Publish(topic, GetBytes("VM Broker Started"));
for (int i = 0; i < 4; i++)
{
chart1.Series[i].Points.AddY(0);
}
mqSpeedSender = new MqttClient("iot.eclipse.org");
mqSpeedSender.Connect("RUPAM");
}
catch
{
}
}
After connection to broker, you will see the message exchanges right in your broker window because we started the broker with -v ( verbose) option.
Figure 5.4: Connection to Broker Established
Notice that we add an event handler for message receieved in local broker for mc
mc.MqttMsgPublishReceived += mc_MqttMsgPublishReceived;
Whenever new message arrives ( the activity data from Android activity tracking app), this event handler will be called.
This is our main analysis method. We perform several tasks here. let us have a look at the entire method first and then we shall break down the logical parts of the event handler and explain them one by one.
void mc_MqttMsgPublishReceived(object sender, uPLibrary.Networking.M2Mqtt.Messages.MqttMsgPublishEventArgs e)
{
this.Invoke((MethodInvoker)delegate
{
if (e.Message[1] == (byte)0)
{
}
else
{
try
{
string command = "";
for (int i = 0; i < e.Message.Length; i++)
{
command = command + ((char)('A' + ((int)e.Message[i] - 65))).ToString();
}
double intensity = 0;
String name = "";
if (command.Contains("_"))
{
string[] parts = command.Split(new char[] { '_' });
intensity = double.Parse(parts[1]);
name = parts[0];
}
else
{
name = command;
}
if (!command.Contains("MOBILE"))
{
if (lv.Count > 0)
{
if (lv[lv.Count - 1].Action.Equals(name))
{
if (intensity == 0)
{
TimeSpan ts = DateTime.Now.Subtract(DateTime.Parse(lv[lv.Count - 1].Start));
lv[lv.Count - 1].TotalPeriod = ts.ToString();
}
else
{
lv[lv.Count - 1].Stop = DateTime.Now.ToString();
TimeSpan ts = DateTime.Now.Subtract(DateTime.Parse(lv[lv.Count - 1].Start));
lv[lv.Count - 1].TotalPeriod = ts.ToString();
var activity = new Activity(name, ""+DateTime.Now, null, null, intensity);
lv.Add(activity);
}
}
else
{
lv[lv.Count - 1].Stop = DateTime.Now.ToString();
TimeSpan ts = DateTime.Now.Subtract(DateTime.Parse(lv[lv.Count - 1].Start));
lv[lv.Count - 1].TotalPeriod = ts.ToString();
var activity = new Activity(name, "" + DateTime.Now, null, null, intensity);
lv.Add(activity);
}
}
else
{
var activity = new Activity(name, "" + DateTime.Now, null, null, intensity);
lv.Add(activity);
}
#region updating chart
try
{
var walk = lv.Where(x => x.Action.Equals("WALKING")).ToList();
var wt = walk.Select(x => TimeSpan.Parse(x.TotalPeriod).TotalSeconds + x.Intensity / 30).Sum();
var rest = lv.Where(x => x.Action.Equals("RESTING")).ToList();
var rt = rest.Select(x => TimeSpan.Parse(x.TotalPeriod).TotalSeconds + x.Intensity / 30).Sum();
var run = lv.Where(x => x.Action.Equals("RUNNING")).ToList();
var runt = run.Select(x => TimeSpan.Parse(x.TotalPeriod).TotalSeconds + x.Intensity / 30).Sum();
var shake = lv.Where(x => x.Action.Contains("SHAKING")).ToList();
var st = shake.Select(x => TimeSpan.Parse(x.TotalPeriod).TotalSeconds + x.Intensity / 30).Sum();
chart1.Series[0].Points.AddY(rt);
chart1.Series[1].Points.AddY(st);
chart1.Series[2].Points.AddY(wt);
chart1.Series[3].Points.AddY(runt);
var speed = (wt + runt + st / 3) / rt;
speed = (double)(Math.Floor(speed * 100)) / 100.0;
speed = speed / 3.0;
var ss = speed;
if (speed > 1.0)
{
speed = 1;
}
if (speed < .45)
{
speed = .45;
}
labSpeed.Text = "Speed=" + speed;
try
{
mqSpeedSender.Publish("rupam/Fan", GetBytesForEdison(speed.ToString()));
}
catch
{
}
}
catch
{
}
#endregion
chart1.Invalidate();
chart1.Update();
dataGridView1.DataSource = null;
dataGridView1.DataSource = lv;
dataGridView1.FirstDisplayedScrollingRowIndex = dataGridView1.RowCount - 1;
}
}
catch
{
}
}
});
}
1. Converting Byte data to string: Note that Mqtt data is always published in byte pattern. So we need to convert the byte to String
string command = "";
for (int i = 0; i < e.Message.Length; i++)
{
command = command + ((char)('A' + ((int)e.Message[i] - 65))).ToString();
}
One of the questions that may arrive in your mind is why can't we simply use Encoding.UTF8.GetString() directly? in fact to tell you the truth, I started with the same method. then I realized many devices like Intel Edison generates two bytes for a character, one character byte followed by '\0'. The GetString() method failed in those contest. After lots of trials and errors I could realize that texts are sent with ASCII encoded in many times. I was even unable to obtain the string message from ASCIIEncoding.GetString() method. So, I finally setteled for extracting equivalent integer, subtracting the ascii value, extracting index of the character and adding it with character 'A'. This is certainly not clean but works for all Mqtt exchanges ( tested with data being published from Node.js, Python, Android and Another C# client).
2. Stroring Activity in lv list
Recall that when the mobile is not with the user, "MOBILE not with user command is sent". We do not want to track this activity. Therefore we go inside the analysis loop only if the command doesn't contain MOBILE keyword.
Now, logically the Activity that is being sent from Android mobile may be same as the previous one ( when user is continuesly resting, after every periodic instance RESTING command will arrive) we do not want to create a new log entry. But when an Activity comes which is different from the previous one, we want to pack the previous one by appending the ending time End and TotalPeriod. This is done by checking the last element of lv which is the list of Activities with the current activity.
if (lv.Count > 0)
{
if (lv[lv.Count - 1].Action.Equals(name))
{
if (intensity == 0)
{
TimeSpan ts = DateTime.Now.Subtract(DateTime.Parse(lv[lv.Count - 1].Start));
lv[lv.Count - 1].TotalPeriod = ts.ToString();
}
else
{
lv[lv.Count - 1].Stop = DateTime.Now.ToString();
TimeSpan ts = DateTime.Now.Subtract(DateTime.Parse(lv[lv.Count - 1].Start));
lv[lv.Count - 1].TotalPeriod = ts.ToString();
var activity = new Activity(name, ""+DateTime.Now, null, null, intensity);
lv.Add(activity);
}
}
In case a new activity arrives, pack up the last one, create a blank activity which is going to be updated with the new one. The packing process will include updating the End time which is nothing but the current time and calculating total Period which is the time stamp difference between the start and the end time.
else
{
lv[lv.Count - 1].Stop = DateTime.Now.ToString();
TimeSpan ts = DateTime.Now.Subtract(DateTime.Parse(lv[lv.Count - 1].Start));
lv[lv.Count - 1].TotalPeriod = ts.ToString();
var activity = new Activity(name, "" + DateTime.Now, null, null, intensity);
lv.Add(activity);
}
3. Data Aggregation data aggregation is basically a summation process where we find out total time for which the user is walking, running and shaking his legs as climate control depends upon these parameters. Now note that the Walk and Run also has an intensity value attached to it which we want to incorporate in our decision making. So, we use a scale factor of 30 for intensity value ( from observation we have found out that when xz value in Android app is about 37 ( which is appended as the intensity value for walk and run) user is moving with about 1.2m/s speed. Therefore by scaling xz value with a factor of 30 gives use user speed in m/s.
var walk = lv.Where(x => x.Action.Equals("WALKING")).ToList();
var wt = walk.Select(x => TimeSpan.Parse(x.TotalPeriod).TotalSeconds + x.Intensity / 30).Sum();
var rest = lv.Where(x => x.Action.Equals("RESTING")).ToList();
var rt = rest.Select(x => TimeSpan.Parse(x.TotalPeriod).TotalSeconds + x.Intensity / 30).Sum();
var run = lv.Where(x => x.Action.Equals("RUNNING")).ToList();
var runt = run.Select(x => TimeSpan.Parse(x.TotalPeriod).TotalSeconds + x.Intensity / 30).Sum();
var shake = lv.Where(x => x.Action.Contains("SHAKING")).ToList();
var st = shake.Select(x => TimeSpan.Parse(x.TotalPeriod).TotalSeconds + x.Intensity / 30).Sum();
So we are adding the intensity value along with the total period for which an activity is performed.
4. Calculating Fan Speed ( Defuzzification) Note that the fan speed is directly propertional to user movement which is a sum of total walk, run and shaking legs. It is also inversely propertional to the total resting period. Therefore when the user rests for long , the speed should automatically come down. We also limit our speed value between .45 and 1. In common term
desired temperature scale=total movement/total rest
var speed = (wt + runt + st / 3) / rt;
speed = (double)(Math.Floor(speed * 100)) / 100.0;
speed = speed / 3.0;
var ss = speed;
if (speed > 1.0)
{
speed = 1;
}
if (speed < .45)
{
speed = .45;
}
5. Updating the chart: What's a fitness/activity tracking app witout visualization? We we add the aggregated result as a line chart in our MS Chart control.
chart1.Series[0].Points.AddY(rt);
chart1.Series[1].Points.AddY(st);
chart1.Series[2].Points.AddY(wt);
chart1.Series[3].Points.AddY(runt);
6. Send the desired Speed to Edison: Once the analysis method calculates the comfort level fan speed value ( which you can use as a desired Air conditioner value, and use it to control your AC by adding an IR remote control unit with Edison and implementing the protocol of your AC makers remote), it is sent to global gateway to demonstrate the entire workflow of a fog system.
try
{
mqSpeedSender.Publish("rupam/Fan", GetBytesForEdison(speed.ToString()));
}
catch
{
}
Where getBytesForEdison() is a method where we remove any extra '\0' ( second byte added with the actual character byte)
static byte[] GetBytesForEdison(string str)
{
byte[] bytes = new byte[str.Length * sizeof(char)];
System.Buffer.BlockCopy(str.ToCharArray(), 0, bytes, 0, bytes.Length);
List<byte> newBytes = new List<byte>();
for (int i = 0; i < bytes.Length; i++)
{
if (bytes[i] > 0)
{
newBytes.Add(bytes[i]);
}
}
return newBytes.ToArray();
}
Figure 5.5 Result Interpretation
Figure 5.5 Shows how the activity tracker responds to user activity and how speed is varying depending upon the user mobility. All you have to do is start your Android app, connect to broker and keep it in your poket ( vertically- Like trouser pocket) and you will see the lines varying as you start varying your activity.
6. Intel Edison Based Fan Control
Figure 6.1 Schematic
We use PWM for Speed control of the fan. We we take a PC USB fan which operates at 5v and hook up the circuit as shown above. BC548 is controlled through the PWM port D5 of Edison. Assuming that you are using Grove shield with Edison, all you have to do is take a grove connector cable and cut from middle. The red wire is the Vcc 5v which needs to be connected to red wire of the fan. The Black wire of cut connector goes to third pin of BC548 ( ground).
black wire of the fan goes to 1st pin of BC548.
We hook a LCD to display current temperature and Speed with I2C port and a temperature sensor with A0.
Now as you change the duty cycle of PWM from your program, the fan speed will vary. This change of duty cycle is sent from our C# app after analyzing the data coming from Android Activity tracking app.
var mraa = require ('mraa');
var tempPin = new mraa.Aio(2);
var LCD = require('jsupm_i2clcd');
var myLCD = new LCD.Jhd1313m1(6, 0x3E, 0x62);
var B=3975;
loop();
var cnt=0;
var r=100,g=0,b=0;
var fanPin=new mraa.Pwm(6);
fanPin.enable(true);
var fanSpeed=0.65;
fanPin.write(fanSpeed);
var mqtt = require('mqtt');
var client = mqtt.connect('mqtt://iot.eclipse.org');
client.subscribe('rupam/Fan/#')
client.handleMessage=function(packet,cb)
{
var payload = packet.payload.toString()
console.log(payload.length);
payload = payload.replace(/(\r\n|\n|\r)/gm,"");
console.log(payload)
try{
fanSpeed=Number(payload.trim());
fanPin.write(fanSpeed);
}catch(ex)
{
console.log(ex.message);
}
cb()
}
function loop()
{
myLCD.clear();
myLCD.setCursor(0,0);
myLCD.write("Temp: ");
myLCD.setCursor(0,6);
var tempValue=tempPin.read();
var resistance=(1023.0-tempValue)*10000.0/tempValue;
var temperature=1/(Math.log(resistance/10000.0)/B+1/298.15)-273.15;
myLCD.write(""+temperature);
myLCD.setCursor(1,0);
myLCD.write("Fan Speed:");
myLCD.setCursor(1,10);
myLCD.write(""+fanSpeed);
if(fanSpeed<=.65 && fanSpeed>.5)
{
myLCD.setColor(255,0,0);
}
if(fanSpeed<.85 && fanSpeed>.65)
{
myLCD.setColor(255,255,0);
}
if(fanSpeed>=.85)
{
myLCD.setColor(0,255,0);
}
setTimeout(loop,1000);
}
Whenever Edison receives data over channel rupam/fan (you are advised to change this channel for your experiements as many reader may be trying the same channel) it writes that value through pwm's write() method.
You can start walking and can observe that the temperature will drop as the fan speed increases :)
6. Conclusion
In this project, we have created a Fog computing reference architecture for a fitness app. We have designed our own fitness tracker from the scratch in Android without using any libraries. Even though the tracker is extremely basic, we have found it to be very efficient in detecting activities correctly. We have then created a C# client to harness the power of the language to analyze the activity data and convert to a comfortable temperature value. This is then sent to Intel Edison which runs a Node.js app to consume the speed from global Mqtt gateway and change the fan speed accordingly. So, in this project we have shown how to combine Android companion app, a client pc app and an IoT app in a fog context. You can deploy many such services in your local network and create a service chain of utility services. This architecture allows you to create your own intelligent world without depending on the cloud based APIs and without risking your personal data to be leaked in cloud.
Stay Fit, live comfortably and code franatically.
