Embedded IoT Databases with Actian Zen and C/C++

The growing number of smart, connected IoT devices generates exponentially increasing data for your organization: sensors, actuators, control systems and smart cameras capturing real time data. So an obvious question is, how do you manage this data? These edge devices are often resource constrained — relatively slow CPU, limited memory and often little storage.

From a developer standpoint, the constrained resources on edge devices demand both a small codebase size and highly efficient use of storage space. Embedded application development has traditionally been done in C/C++ for compact, efficient programs, and using fairly rudimentary direct access means of manipulating data.

New embedded databases are proving themselves a worthy alternative responding to the requirements of the Internet of Things with local data management capabilities delivering the performance and availability. They operate on different hardware and software platforms to capture real time data, efficiently process and summarize it for other systems, and then distribute it for analysis and making meaningful decisions.

Actian Zen

Actian Zen is the first enterprise-ready embedded database that offers hybrid data management and analytics with the ability to scale-down and scale-out the data-centric IoT solutions.

Working with multiple gateways and edge devices means working with multiple databases and data management solutions on each application node. Usually this requires custom data transformation and ETL integration code for moving data between embedded and central databases, which makes the deployment and maintenance of a hybrid data system a headache. This not only makes the deployment cycle slower, but also results in unnecessary complexity and higher management cost.

Actian Zen provides a common data type and file format across a wide range of platforms. It works across edge devices and gateways ensuring a simpler, more powerful integration experience for developers and enables them to create value-added application features without having to worry about customized ETL operations.

To demonstrate how easy it is to set up and use Actian Zen in an embedded environment, and how well programming against Zen translates between server, desktop, and embedded environments, we'll walk through a simple example using a Raspberry Pi device as the embedded system. We chose Raspberry Pi because it is inexpensive, flexible and resembles many off-the-shelf and prototyping IoT solutions. We’ll write a simple C++ program to capture some time-series data on the device, write it to the Zen database and retrieve data.

Setup

Let’s first configure the Raspberry Pi basic setup. We need to install Raspbian on the Raspberry Pi. Raspbian is a Debian-based GNU/Linux distribution and is officially supported and recommended operating system for Raspberry Pi devices. Here’s what you need to flash Raspbian.

• Raspberry Pi
• A 4GB microSD card

You can download the Raspbian OS here. Once you have the zip file, you can choose any software to flash the image. For example, you can use Etcher which is available to download and use for free on Windows, Mac and Linux.

Once the flashing process is complete, insert the microSD card on your Raspberry Pi. Connect the Raspberry Pi to a monitor, keyboard and USB mouse. Power on the Raspberry Pi and let it boot. Configure the basic desired settings on it before moving ahead.

The rest of this article assumes the following prerequisites:

• You should be familiar with C/C++ programming.
• You understand basic SQL commands like SELECT and INSERT.

Installing Zen

To download Actian Zen for Raspbian, access the downloads page, and select the following in the drop down menus:

• Product: Actian Zen (PSQL)
• Release: v13 R2 Evaluation
• Platform: Raspbian ARM 32-bit

Click the Apply Filter button to retrieve the pertinent downloads.

Once retrieved, expand the Zen Edge v13 R2 Trial node, and download the Actian Zen Edge database v13 R2 Trial download for Raspbian ARM.

Once the download is complete, apply the following steps for installation. Open the terminal window of your Raspbian system (either locally or remote using SSH), log in as the default Raspberry Pi user and enter the following commands.

$ cd Downloads  
$ sudo cp Zen-IoT-linux-13.30-035.000.armhf.tar.gz /usr/local

Install it using following commands.

$ cd /usr/local  
$ sudo tar -zxf Zen-IoT-linux-13.30-035.000.armhf.tar.gz 
$ cd psql/etc  
$ sudo ./preinstall.sh  
$ sudo ./postinstall.sh 

Development Setup

We’ll be working with only C++ in this article so you don’t strictly need any IDE. A simple text editor can work. On the other hand, if you’re obsessed with IDEs for development, then you can choose from Geany, Visual Studio Code or your preferred editor. You need just one simple command to install the IDE.

$ sudo apt-get install geany

Or:

$ curl -s https://packagecloud.io/install/repositories/headmelted/codebuilds/script.deb.sh | sudo bash 
$ sudo apt-get install code-oss 

Choice of IDE is completely yours. You can refer to this link for more details on IDEs and compilers for C++ development.

We'll use the Btrieve 2 SDK to provide code access APIs for Actian Zen. Go to the downloads page and select the following in the drop-down menus:

• Product: Actian Zen (PSQL)
• Release: SDKs
• Platform: Btrieve 2

Click the Apply Filter button to retrieve the pertinent downloads.

Choose Btrieve 2 Linux SDK for PSQL 13 for your platform.

Execute the following commands once the downloading is complete.

$ cd Downloads  
$ tar xf PSQL-SDK-Btrieve2API-13.30.034.000-Linux-noarch.tar.gz

The Btrieve 2 API components will be expanded after execution of this command. Just to keep our directory structure clean, let’s create another folder for our development projects.

$ cd Desktop 
$ mkdir Programs

Last but not the least, we need to copy two required files from the “include” folder to our project directory.

$ cp include/btrieveC.h ~/Programs  
$ cp include/btrieveCpp.h ~/Programs

Time Series Demo

Now that we have everything set up, we'll create a simple C++ example that captures time series temperature data and writes it to the Actian Zen database. To simulate a real-world example without getting too complicated, we'll simply read the built-in SOC temperature register (/sys/class/thermal/thermal_zone0/temp), write it to the database. Later we can also query it to retrieve data. Let’s get coding.

First, create the functions to capture CPU state, file creation and loading.

#include <stdio.h>
#include <string>
#include <iostream>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#include <regex>
#include <vector>
#include "btrieveCpp.h"
using namespace std;
static char *btrieveFileName = (char *)"coretemp.btr";
typedef struct CpuStatus {
   double temperature;
   double frequency;
   double* cpuLoad;
} record_t;
double toDouble(std::string s) {
   std::replace(s.begin(), s.end(), ',', '.');
   return std::atof(s.c_str());
   }
string do_console_command_get_result(char* command) {
   FILE* pipe = popen(command, "r");
   if (!pipe)
      return "ERROR";
   char buffer[128];
   string result = "";
   while (!feof(pipe))
   {
      if (fgets(buffer, 128, pipe) != NULL)
         result += buffer;
   }
   pclose(pipe);
   return(result);
}
double getTemperature(int zone) {
   char buffer[100];
   sprintf(buffer, "cat /sys/class/thermal/thermal_zone%d/temp", zone);
   string result = do_console_command_get_result(buffer);
   if (result.size() > 0)  result.resize(result.size() - 1);
   return toDouble(result);
}
double getCpuFrequency(int cpu) {
   char buffer[100];
   sprintf(buffer, "cat /sys/devices/system/cpu/cpu%d/cpufreq/cpuinfo_cur_freq", cpu);
   string result = do_console_command_get_result(buffer);
   if (result.size() > 0)  result.resize(result.size() - 1);
   return toDouble(result);
}
vector<string> tokenize(const string& str, char delim) {
   vector<std::string> tokens;
   std::stringstream mySstream(str);
   string temp;
   while (getline(mySstream, temp, delim))
      tokens.push_back(temp);
      return tokens;
}
double *getMpStat() {
   double* result = new double[4];
   std::regex regex(R"(\d{2}:\d{2}:\d{2}\s+(\d)\s+\d+[.,]\d{2}\s+\d+[.,]\d{2}\s+\d+[.,]\d{2}\s+\d+[.,]\d{2}\s+\d+[.,]\d{2}\s+\d+[.,]\d{2}\s+\d+[.,]\d{2}\s+\d+[.,]\d{2}\s+\d+[.,]\d{2}\s+(\d+[.,]\d{2}))");
   string s = do_console_command_get_result((char*)"mpstat -P ALL 1 1");
   vector<string> lines = tokenize(s, '\n');
   for (size_t i = 0; i < lines.size(); ++i)
   {
      smatch matches;
      if (regex_search(lines[i], matches, regex))
      {
         int cpunum = stoi(matches[1].str());
         if (cpunum >= 0 && cpunum < 4) {
            result[cpunum] = 100 - toDouble(matches[2].str());
         }
      }
   }
   return result;
}
void printHelp() {
   printf("\n\nUsage:\n coretempmon [options]\n\n");
   printf("-h            This help\n");
   printf("-s            Silent output, just data");
   printf("-d <milliseconds>  Monitor delay in seconds (default=5)\n");
}
void monitoring(unsigned int delay, int loops, bool silent) {
   if (!silent) {
   printf("Core Temperature Monitor 1.0\n");
   printf("Scan delay is %d seconds\n", delay);
   printf("Stop monitoring using [ctrl]-[c]\n");
   printf("Time Temperature \u2103 Freq_CPU1 CPULoad1  %%CPULoad2 \%%CPULoad3 \%%CPULoad3\n");
   }
   
   bool infinite = (loops == -1);
   int counter = loops;
   record_t data;
      // Getting the CPU stats
   while (infinite || counter-- >0)
   {
      usleep(delay * 1000);
      data.cpuLoad = getMpStat();
      data.frequency = getCpuFrequency(0);
      data.temperature = getTemperature(0);
   
      printf("%0.2f \u2103 %0.0f MHz %0.2f %0.2f %0.2f %0.2f\n", data.temperature, data.frequency, data.cpuLoad[0], data.cpuLoad[1], data.cpuLoad[2], data.cpuLoad[3]);
      data.cpuLoad;
   }
}
static Btrieve::StatusCode
createFile(BtrieveClient *btrieveClient) {
   Btrieve::StatusCode status;
   BtrieveFileAttributes btrieveFileAttributes;
   // If Seting Fixed Record Length fails
   if ((status = btrieveFileAttributes.SetFixedRecordLength(sizeof(record_t))) != Btrieve::STATUS_CODE_NO_ERROR)
   {
      printf("Error: BtrieveFileAttributes::SetFixedRecordLength():%d:%s.\n", status, Btrieve::StatusCodeToString(status));
      goto leave;
   }
   // If Creating file fails
   if ((status = btrieveClient->FileCreate(&btrieveFileAttributes, btrieveFileName, Btrieve::CREATE_MODE_OVERWRITE)) != Btrieve::STATUS_CODE_NO_ERROR)
   {
      printf("Error: BtrieveClient::FileCreate():%d:%s.\n", status, Btrieve::StatusCodeToString(status));
      goto leave;
   }
leave:
   return status;
}
static Btrieve::StatusCode
openFile(BtrieveClient *btrieveClient, BtrieveFile *btrieveFile) {
   Btrieve::StatusCode status;
   // If opening file fails
   if ((status = btrieveClient->FileOpen(btrieveFile, btrieveFileName, NULL, Btrieve::OPEN_MODE_NORMAL)) != Btrieve::STATUS_CODE_NO_ERROR)
   {
      printf("Error: BtrieveClient::FileOpen():%d:%s.\n", status, Btrieve::StatusCodeToString(status));
      goto leave;
   }
leave:
   return status;
}
static Btrieve::StatusCode
loadFile(BtrieveFile *btrieveFile) {
   Btrieve::StatusCode status = Btrieve::STATUS_CODE_NO_ERROR;
   return status;
}
static Btrieve::StatusCode
closeFile(BtrieveClient *btrieveClient, BtrieveFile *btrieveFile) {
   Btrieve::StatusCode status;
   // Closing the File fails.
   if ((status = btrieveClient->FileClose(btrieveFile)) != Btrieve::STATUS_CODE_NO_ERROR)
   {
      printf("Error: BtrieveClient::FileClose():%d:%s.\n", status, Btrieve::StatusCodeToString(status));
      goto leave;
   }
leave:
   return status;
}

Once you have defined all the necessary functions, you can call them to get your data to query and can apply different conditions on it.

int main(int argc, char *argv[])
{
   BtrieveClient btrieveClient(0x4232, 0);
   Btrieve::StatusCode status = Btrieve::STATUS_CODE_UNKNOWN;
   BtrieveFile btrieveFile;
   
   // If creating the File fails.
   if ((status = createFile(&btrieveClient)) != Btrieve::STATUS_CODE_NO_ERROR)
   {
      goto leave;
   }
   // If opening the File fails.
   if ((status = openFile(&btrieveClient, &btrieveFile)) != Btrieve::STATUS_CODE_NO_ERROR)
   {
      goto leave;
   }
   // If loading the File fails.
   if ((status = loadFile(&btrieveFile)) != Btrieve::STATUS_CODE_NO_ERROR)
   {
      goto leave;
   }
   else {
      int c;
      unsigned int delay = 5000;
      int loops = -1;
      bool silent = false;
      while ((c = getopt(argc, argv, "shd:n:")) != -1) {
         switch (c) {
         case 'd': {
            delay = atoi(optarg);
            if (delay < 1) { delay = 5000; }
            break;
         }
         case 'n':
            loops = atoi(optarg);
            break;
         case 's':
            silent = true;
            break;
         default:
            abort();
         }
      }
      if (delay > 0) {
         monitoring(delay, loops, silent);
         }
      }
   // If closeing the File fails.
   if ((status = closeFile(&btrieveClient, &btrieveFile)) != Btrieve::STATUS_CODE_NO_ERROR)
   {
      goto leave;
   }
leave:
   // If there wasn't a failure.
   if (status == Btrieve::STATUS_CODE_NO_ERROR)
      return 0;
   return 1;
}

The above loops through the records and output the CPU temperature along with its frequency. Now build and run from your editor.

Feel free to experiment with other examples and applying different queries.

The above example is a simple dummy example to explain the use case. Zen Edge can do much more than that. Feel free to experiment with the above dummy example using multiple edge devices, add some sensors and implement more analysis on the time series data.