Node-RED and handling Sensor Data

In 2 former Blog posts I explained how to install Node-RED and how I used TinkerForge components to receive environment data from a WeatherStation and store that in simple txt-fliles. In this post I will show how to read data from text-files and store the data in the correct format in a time series Database InfluxDB.

From the node-red website “Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways”. It also is low-code to some extend and the web-based editor makes is simple to setup the data-flows and deploy it with a single click.

I have a Ubuntu server running a Java program to fetch data from a TinkerForge Weatherstation. The Java program will write sensor data every time the sensor data changes. The data is appended to 4 different text-files, one for each measurement, Temperature, Humidity, Ambient Light and Air-pressure. Only the value it stored, no time stamp. This is important to understand, as this means that the data in the text-files cannot be used to check a value in the past. It can only be used real-time. There are many other solutions to this problem, I kept it very simple, my option is one which works for me and more important forces me to use many different tools and options. A perfect learning experience.

In this example you do not need the TinkerForge Weatherstation as I will be using an inject node from node-RED to simulate raw data input. But I will still show the setup I have created for the WeatherhStation.

the basics

Make sure node-RED is running, if needed check install Node-RED. Start your favorite browser and connect to your node-RED interface. http://<Server-IP-address&gt;:1880. My example print-screen shows a few extra Tabs, you should see 1 tab called “Flow 1”.

Creating a flow means dragging in nodes from the left of the screen in the canvas and connecting the nodes to make node-RED perform actions on the data.

Lets start with a node to read data from a text-file. As explained we need an option to read data when data is added to the text-file. In Linux/UNIX terms, we need a option to tail a file. Scroll down the nodes on the left of the screen until you get to the storage section. You might need to open it by clicking the “>”. Drag the tail node on to the canvas and left double click it.

In the Filename field put in the the absolute path to the file you want to monitor. In this example we set File type to “Text – returns String” and we can set Split on “true”. And you can add an Name to the node, which might help in understanding the flow setup, I used “Temperature”. Click Done when you are ready.

I like to add a debug node to every flow node so I can very simple check what is going on in the flow and how the data looks which is being handled in the dataflow. Search for the debug node and drag it to the canvas, it is at the top of the node menu in the common section. Now simply left-click and hold the right connector of the tail node (now called – Temperature if you followed my example) and while holding the mouse button, go to the debug node left connector and release the left mouse button while the connector is over the left connector of the debug node.

We can now deploy this new flow to see if data is indeed coming in if you have setup your TinkerForge Weatherstation and also the Java program should be running. In case you did not do that you could also use the inject node to so you can push data into your dataflow without having the sensors setup. Search for the inject node, drag it on the canvas and configure the inject node with the following configuration. Double click the inject node. Select string as payload type and type some random temperature. Give it some name and click Done. Connect the inject node with the debug node. Select the Red Deploy button on the right top of the node-RED editor, this will deploy the Flow to the node-RED server. Now click drop-down icon at the top left of the node-RED editor.

This will open the debug screen on the right of the screen and will show any data that is received by the debug node. If you have setup the TinkerForge Weatherstation and Java program you might see data coming in, remember the Java program only gets data when triggered. You can also inject data yourself by pushing the inject data button at the left side of the inject node. As the inject node is set to a fix string every inject will be the same string, if you want can change the string, just open the inject node change the value and Deploy the flow.

Calling a function

In this example I want the data to be presented in a InfluxDB database as some floating point value. And at the moment we have a string, we will use the function node to transform the string to a floating point value. Search for the function node in the function section and drag it onto the canvas and connect the inject and/or tail node to this function node. Also add an extra debug node connected to the function node.

Double click it. Give it a logical name. In the Function field there is just 1 command; return msg; This will do noting at the moment, as it is not performing any function on the message payload. Replace the Function with the following code;

var newMsg = 
{
    payload: parseFloat(msg.payload)
}

return newMsg;

This short code is very simple, it defines a variable newMsg. The payload received is parsed from String to a Floating point number and the code returns the newMsg variable as the payload / output of the Function node.

Click done to save the function node. For now I disabled the tail node (called Temperature in the example). To disable/enable a node double click the node and you will find a Enabled/Disabled button at the bottom of the Edit tail node screen. And as always Deploy the changes by clicking the Deploy button at the top right of the Editor screen.

Open the debug screen at the right of the editor screen.

Now click the inject button of the inject node. In the debug screen you should now see 2 debug messages. The first debug message should show say; msg.payload : string[X] X being the size of the string you set in the inject node. The second debug message should give msg.payload : number.

Store data in AN influxdb Database USING Node-RED

The final step is to store the data with a timestamp. As explained the data feed from the sensors (Or in this example, from the inject node) is just a string which we parse to a floating point format and this value we want to store. But there is a small problem, if you check the overview of your available nodes, you will not find any InfluxDB nodes. In my next post I will explain how to get this done and also show you how to setup a new Database you can use, if needed check out my post about installing InfluxDB on Ubuntu server 20.04.