Raspberry Pi - Part 3 IoT Hello World

In order to utilize the NodeMCU firmware it must be flashed onto the device and for this we need to download the ESP8266Flasher executible. The flasher application is fairly straight forward to use and there is plenty of documentation on site on it's usage. We get the firmware to flash onto the device from nodemcu-build.com, where you can pick what functionality you require for the device then have them build it for you. When they are done you will be emailed a link where the image can be downloaded. When flashing the device pin gpio0/D3 must be held low. On the ESP8266-1 this was an issue but on the ESP8266-12e this was not an issue as I believe they have the hardware on board to sense when it is being flashed. At any rate a heads up, if the flash isn't working this may be the reason.

Once we have the device flashed with the latest NodeMCU firmware we need some way of programming it. I highly recommend the development IDE ESPlorer by Victor Brutskiy. It is a free application, written in Java that supports both the AT command and LUA based languages. It also has the option to program in MicroPython but I didn't try it so I have no idea if it works or if the language is available yet.

ESPlorer supported platforms;
	Windows(x86, x86-64)
	Linux(x86, x86-64, ARM Soft & hard float)
	Solaris(x86, x86-64)
	MacOS X(x86, x86-64, PPC, PPC64)

The ESPlorer IDE is a powerful tool but I only use a subset of the functionality that it provides. Instead of describing all the details of the IDE we will go through the process of creating, saving to the device as a file and running.

The device keeps an area in non-volatile memory set up as a flat file system where files to be run may be stored. To format the file system on the device click the 'Format' button, caution this will clear the file area erasing all files that currently reside there. To get statistics on the file system click the 'FS Info' button, this will tell you how many files you have saved and there names as well as the total file area size and how much is in use and free. The last button grouped with the others is the 'Reload' button that displays/refreshes a list of buttons with file names below the reload button itself that when clicked will execute that file.

I won't describe how to use the application or all the functionality available but instead will describe the important pieces throughout the tutorial to get done what we need to do and if you would like more information about the application or how to use it I will leave it to you to google, there has been a lot written about this application so you shouldn't have any problems.

To showcase some of the functionality of the EPS8266 device and how to use the IDE we'll create a simple application to blink an LED on gpio0/D3 on/off a set number of times. Additionally each time through the loop the current cnt will be printed to the terminal.

Before we can start coding we need to assemble the very simple circuit shown in the schematic below consisting of a current limiting resistor and an LED from port D3 to ground. The value of the resistor required will depend on the LED that you select and can be calculated from values that are obtained from the LED's datasheet. You will need to know the forward voltage drop (V_f) and the forward current (i) of the ODED and from this the resistance can be calculated by R = (V_s - V_f) / i where V_s is the supply voltage. As an example if we have an LED with V_f = 1.8V and i = 30mA respectively with a supply voltage of 3.3V then; R = (3.3V - 1.8V) / .030 = 50ohm. I used a 330ohm resistor in my example which worked fine and lights the LED but it is just a tad dimmer than what it would have been had I used a resistor of the value calculated.

Now that we have things wired up we can concentrate on the code. As can be seen in the code snippet that follows the NodeMCU eLua language is very Python-ish syntactically and even if you don't know Python should be fairly easy to understand.

pin = 3        -- gpio0/D3
target = 5     -- Our target count
 
local function blinky_func(target_cnt)
  local cnt = 0
   while cnt < target_cnt do
        -- gpio.write(pin, state)
        --    where state = gpio.FLOAT, gpio.HIGH or gpio.LOW
        gpio.write(pin, gpio.HIGH)
        tmr.delay(1000000)   -- wait 1,000,000 us = 1 second CAUTION Blocking
        gpio.write(pin, gpio.LOW)
        tmr.delay(1000000)   -- wait 1,000,000 us = 1 second CAUTION Blocking
        cnt = cnt + 1
        print(cnt)
    end
end
 
-- gpio.mode(pin, mode[, gpio.PULLUP])
--      where mode = gpio.INPUT, gpio.OUTPUT
gpio.mode(pin, gpio.OUTPUT)
 
-- Call the blinky_func function
blinky_func(target)  

There is a lot more functionality available in the eLUA language (See NodeMCU firmware documentation ) and we will go into greater detail when we get to the actual IoT Hello World application. Once the code has been entered into the editor window there are a number of things we can do from here;

1. control+s to save the file locally and to the devices file system and run it.
2. Save to ESP - saves the file to the device and runs it.
3. Send to ESP - downloads it to the device and runs it.
4. Once it has been saved to the device you may click on the Reload button and a list of files that have been saved on the device will appear, you can then click on the file you wish to run.
5. Lastly if the file has been saved to the device you can send the 'dofile("blinky_func.lua")' command to the device by entering the command in the bottom right textbox and clicking on the Send button just to the right of the textbox.

We'll start the installation by getting the core applications Node.js and the Node Package Manager (NOM). I always do the update/upgrades to get machine up to date before doing any installations, it's a good practice to get into.

sudo apt-get update
sudo apt-get upgrade
 
sudo apt-get install npm
sudo apt-get install node

Because we'll be using Express we need to install some middleware using the package manager. There are two ways to configure the development environment for developing Node.js applications; the first method involves putting the dependency files in an area on disk and when a project is created, links are used to access the required modules in the global area. Secondly the dependency files are downloaded into the local directory and a global area may still be used but not necessary. I have my environment set up using the first method of linking to a global area and I'll show the steps necessary to do it both ways.

Installing the application specific modules is next and we have the ability to control where the modules reside by using command line arguments when invoking the install commandls;

sudo npm [--save] [--global | -g] install module-name
	
	where the optional auguments;
		-save = Installs a copy locally as well as global area
		-global = Only create a copy in the global area.

If no arguments are supplied the files are copied only into the working directory. If you previously downloaded the middleware files into the global area or are starting a new project and want to reference them use the link option.

Below is a list of the middleware modules we will be using in this project.

• ejs - Simple templating language that let's you generate HTML markup with plain JavaScript.
• body-parser - Parses incoming request bodies.
• mqtt - Publish/Subscribe messaging protocol.
• morgan - HTTP request logger.
• pm2 - Runs the application in the background, provides load balanving and keeps the process alive.

This series of commands establishes links to the modules we want to be able to access in the global area this action has the side effect of creating the node_modules subdirectory.

sudo npm link express
sudo npm link ejs
sudo npm link body-parser
sudo npm link mqtt
sudo npm link morgan

When we want to install the modules locally we use the install command line argument and if we want to also save a copy globally add the --save arguemnt.

sudo npm install express
sudo npm install ejs
sudo npm install body-parser
sudo npm install mqtt
sudo npm install morgan

We may have gotten a little ahead of ourselves, before we can download module(s) to our working copy don't we need a working copy?

Creating the application environment or working copy is something that will become second nature if you are to do a lot of Node.js development. It's fairly easy to do just takes a few steps and the rest is automated. Basically you create a directory, change it's security permissions and invoke npm init to allow Node to do the setup. The example shown below creates the project 'home-automation' subdirectory and initialize it.

sudo mkdir home-automation
sudo chmode 777 home-automation
cd home-autmation
sudo npm init

When npm init is invoked the application prompts for a series of questions and for the most part you can just accept the defaults, but remember the name you used for the entry point during the init, you will need to create this file in the root directory. A package.json file is created by the init process and the entry point name you specified will be used by Visual Studio Code when running or degugging the application or when starting the application using the sudo npm start command. I don't use the npm start method as it takes extra time to set things up. If I'm developing on the Pi I won't be using Visual Studio Code so I invoke directly using sudo node home-automation.js.

Notice that I shortened the name to 'home' and the entry point to app.js. I chaged the name just because I wanted to and to show that they don't need to be the same and I changed the entry point because I don't want this to be confused with file name index that has other extensions such as; html, asp, etc..

There is one other way in which to get the dependency modules you need and this is what makes the package manager cool. You can work backwards by supplying a dependency group, such as the example given below in the package.json file;

"dependencies": {
	"body-parser": "1.15.2",
	"express": "4.13.4",
	"ejs": "2.4.2",
	"jade": "1.11.0",
	"mqtt": "1.12.0"
}

then issue the command:

npm will go to the web and pull down the modules it needs and put them in the local node_modules direcory. I really like this system because not only does it allows the user that wants to share a project to include the package.json that contains the dependency group but the end user can also get the correct version of the module that is available.

During the development phase I started the application in the foreground so that I could stop or restart it quickly using ctrl-c.

Once the project is complete and it's time for it to go into production you would then use the pm2 utility to start the application in the background as illustrated in the image that follows.

In the image above you will notice that I started mosquitto before the node application, it must be running to make an mqtt connection. Refer to my previous article concerning setting up the message broker, if you are unfamiliar with mosquitto or configuring it as a message broker.

We've already gone over the mosquitto stuff in my previous article so we won't beat a dead horse. In the /etc/mosquitto/mosquitto.conf file you will need to modify the contents and at least be sure these fields are set correctly. Refer to section 2.3 when you are ready to run mosquitto.

# Change this value to your Raspberry Pi's port and IP address
listener 1883 192.168.254.20
 
# When this is set to true no security is used 
allow_anonymous true

The app.js javascript file is the entry point for our Node.js web server application. In the snippet, from app.js that follows we configure our callback methods for our mqtt broker messaging. The first callback is triggered when a user connects and they are immediately subscribed to the 'node/1/state/' topic. The second callback is triggered when a 'state' message is recieved from the mqtt client and passed on to the socket.io middleware's emit method that pushes the 'state' message out to connected clients.

// ==================> mqtt Handlers <====================
// When a client connects we subscribe to mqtt state msg.
client.on('connect', function(){
    console.log('Connecting...');
    client.subscribe('node/1/state/');
})
 
// Handler to receive the state msgs that we subscribed to above, from the ESP device.
client.on('message', function(topic, msg){
    console.log(String('{ ' + topic + ': ' + String(msg) + ' }'));
	// Incomming msg will either me a 0 or a 1 and we push the current state
	//	to the client to update their state status.
    if (msg == 1) {
        io.emit('state', 'ON');
    }
    else
    {
        io.emit('state', 'OFF');
    }
});

The portion of the client code in the snippet below is the JavaScript code that is used in our bi-direction or push/pull protocol using socket.io. The first call back defines the action taken when the client connects to the server, it creates another callback that receives the 'state' message sent by the server and updates the '#testy' field. Yeah I'm bad at naming stuff. The lower part in the jquery ready area are the Button event handlers that just pass the 'cmd' message on to the server.

<script type='text/javascript'>
    var socket = io.connect('192.168.254.20:3000');
    <!-- On connection we set a handler for the 'state' msg -->
    socket.on('connect', function() {
        socket.on('state', function(msg) {
            console.log(String('on.state: ' + String(msg)));
            $('#testy').text(String(msg));
        });
    });
    $(document).ready(function() {
        <!-- This keeps the form from submitting! -->
        $('form').submit(function(){ return false; });
        <!--
			Click handlers for the two buttons.  Send the appropriate
			'cmd' message to the server.
		-->
        $('#btnOnId').click(
            function($e) {
                socket.emit('cmd', 'ON');
                }
        ),
        $('#btnOffId').click(
            function($e) {
                socket.emit('cmd', 'OFF');
            }
        )
    });
</script>

When I started looking for example code for this project I had a stroke of good luck and found pretty much a piece of drop in code from a gentleman at foobarflier.io...thanks man if you're reading this. Simple connected object with NodeMCU and MQTT . I modified the code to suite my needs and commented the crap out of it but the firmware was well written so the mods were minimal and went very well.

Before powering up the ESP8266 we need to wire the LED and the feedback loop that we'll use for our project. Remember that the pin numbers are what the firmware uses to reference the ports not the gpio numbers.

Once you have downloaded and unzipped the files fire up ESPlorer and load the files in the Raspi-Part3\code\lua directory into the editor;

• config.lua - Contains the config settings for the project. You will need to edit this file and replace;
  • module.SSID - Replace with your SSID informationion
  • module.HOST - Host IP address
  • module.PORT - Port number you'll be using
  • module.CMD_PIN - ESP8266 gpio pin to use to control the LED.
  • module.STATE_PIN - ESP8266 gpio pin to use for the LED sate.
• setup.lua - Firmware that establishes network connection to
• application.lua - Establishes mqtt connection and contains callbacks.
• test.lua - Entry point for the application.

Now download the files into the device and run the test.lua module, you should see the connection messages if everything is working as it should. Remember to leave the test.lua file name as it is until you are sure that the code is as you want it, then change the name to init.lua to have it auto start.

Since we've covered most of this material in previous sections I have run through the IoT Hello project fairly quick. The only thing new was the socket.io stuff and it didn't prove to be a significant change to waht we had before. But for the benefit of completeness I will run through the steps that need to be performed to get this project up and running. If you have any problems please read the next section on Trouble Shooting.

1. On the Pi install Mosquitto, NPM and middleware, if you haven't done so already.
2. Unzip the download file into a working directory.
3. Navigate to the code->node->IoT-Hello directory, transfer it and it's contents to a working directory on the Pi.
4. On the Pi edit /etc/mosquitto/mosquitto.conf and replace the network information.
5. Start mosquitto
6. From a terminal on the Pi navigate to the IoT-Hello directory and invoke the sudo npm install command to install the required middleware.
7. Edit app.js you will need to modify the g_broker variable with your network information.
8. Start app.js
9. Set up the LED and associated hardware on you breadboard.
10. On the PC fire up the ESPlorer application and connect to the ESP8266-12e.
11. Load the files in the code->lua directory.
12. In the config.lua modify the SSID, HOST information and other as needed.
13. Download the files to the ESP device and start the test.lua file.
14. Fire up your favorite browser, navigate to the address and port you specified in the app.js file and if everything worked you should see the page displayed at the beginning of this tutorial and be able to click on the buttons and control the LED and the state should update.

I realize there are a lot of steps involved in getting this whole thing working and is pretty involved for just a simple application but I believe the whole process is fairly straight forward. On the plus side most, if not all of these steps would be necessary for a more sophisticated application.

So you followed the instructions, to the letter and it doesn't seem to work. During the course of developing this project I formulated some debugging techniques that I'll pass on. One piece of advice, the mqtt_spy utility is your best friend!

First let's be sure that Mosquitto is running correctly. I found the best way to do this is to run it interactively,

if it runs without giving an error message fire up mqtt_spy and publish/subscribe to any topic, "test" for instance and publish a test message and see if it shows up in the subscription pane. If it does then the mqtt stuff is working. If you got errors on startup or the test message didn't make it through check your settings in /etc/mosquitto/mosquitto.conf. Once Mosquitto is running properly you can move on but don't move on unless it is.

The next step is to get the ESP8266 LED and loopback wired up and the firmware running. If it is running correctly you should see the connection information displayed in the ESPlorer output pane. If the firmware gives errors or it seems to hang check the settings in config.lua file. Once you do get a connection established you can fire up mqtt_spy and publish an 'ON' or 'OFF' message under the 'node/1/cmd/' topic to control the LED. Likewise you can subscribe to the 'node/1/state/' topic to recieve state updates.

If you got all that working and you can't access the web page or it doesn't do anything when you click either of the buttons check the settings in the app.js file, that's really the only thing that can functionally go wrong. If you didn't set the project up correctly that's another story and in that case you need to go back and re-read that section of the tutorial.

And last but certainly not least if everything works correctly my name is Mike Hankey and if not my name is Elmer Fudd. Happy coding!