Using the lwIP stack with RF24Ethernet and RF24 radios - Another experiment in progress

 Using the lwIP stack with RF24Ethernet and RF24 radios

Another experiment in networking 

 So I finally got some time to play around with the lwIP stack over the holidays, and decided to try my hand at getting it working with the RF24Ethernet/RF24Gateway layers, so users can more conveniently handle TCP/IP traffic over the nRF24 radios.

 

 Overview:

 The RF24Gateway and RF24Ethernet libraries work together to handle TCP/IP and UDP traffic over nRF24 or nRF52 radio links. Users can communicate using standard protocols and techniques, without additional programming or advanced knowledge of the radio software or hardware.

The RF24Gateway library runs on more powerful systems with their own IP stack like a Raspberry Pi or other Linux devices with GPIO and SPI capabilities. The RF24Ethernet library is designed for Micro-controllers like those found using the Arduino platform. Now it supports the faster devices with a more advanced IP stack, lwIP.

 In studying and playing around with lwIP and the Arduino platform, most if not all of the client examples for Arduino utilize a single connection at a time, whereas lwIP supports multiple connections. In server mode, the default behaviour is to listen on a given port, with a backlog. This means that in the current configuration, lwIP will handle the first connection normally, then accept a second connection, but won't complete the second connection until the first connection has finished.

I've also introduced default timeouts, 30 seconds server side, so that if no data is sent or received for a given timeout period the server will disconnect the client. For the server side, call server.setTimeout(30000); . For the client, users need to implement their own timeout conditions. The server timeout can be set to 0 to disable it.

Usage: 

 Usage remains exactly the same as before, with users utilizing the Arduino Ethernet API and client/server functions to connect, disconnect and transfer data between systems. This means that RF24Ethernet users don't have to learn a new API to use the library, as long as they are familiar with the standard Arduino Ethernet API.

As of this writing UDP support is not included but will be added at a later date, TCP/IP support is functional and tested via MQTT and HTTP. The Arduino lwIP library can be installed using the Arduino Library Manager.

 Conclusion:

 I've been wanting to configure lwIP to go along with the RF24Ethernet library for a long time, but just didn't have the time or expertise to sit down and make it happen. With the recent holidays, I was able to put things together, and it came out working very nicely. It is still in prototyping stages, likely with some bugs to work out, but it is functional and available for testing at https://github.com/TMRh20/Sketches/tree/master/RF24Ethernet-lwIP

Update: Jan 18 2026:

As of today, with the latest commits, the server and client tests I am running have been stable for 24 hours. The main bugs have been worked out, and the system is operating normally.  A pull request for the new version of RF24Ethernet will probably be made in the next week or two, pushing the code to a side branch at https://github.com/nRF24/RF24Ethernet and concluding the initial prototyping. Any users making use of the code in the Sketches repository will be encouraged to move to the new branch in the main repo.

 NEW: 

The RF24Ethernet library no longer requires RF24Gateway running! Nodes can simply run RF24Ethernet and establish TCP/IP connections between themselves! See the new Headless examples found in the examples/Headless directory. 

Instructions:

 1. Download the RF24Ethernet library from the link above and place into your Documents/Arduino/libaries/ folder

2. Download and install the RF24Network library from ZIP from https://github.com/nRF24/RF24Network

 3. Edit the RF24Network_config.h file to set MAX_PAYLOAD_SIZE to 1514

4. The library is configured to detect CPU speeds over 50Mhz and use lwIP automatically, but users can edit RF24Ethernet.h and #define USE_LWIP as required

5. Enjoy!!! 

 

 

 

 

 

 

 

 

 

Automation etc with Node Red - How to make your Dashboard2 chart data persistent using a SQLite database

 How to make your chart data persistent using a SQLite database

Reliable charts and graphs with Node Red

 So I've been using Node Red for quite some time now as part of my general IoT network, involving wireless radio modules and numerous sensors etc. One main drawback of using Node Red, is that by default, it doesn't store data long term, and your charts etc will go blank after a restart or shutdown.

 Using a SQLite database combined with Node Red, I've been able to store information for the long term that persists even after restarts and shutdowns and this is how I did it.

 

Above is an image of a simple Node Red flow, logging and retrieving information from a database.

  So first things first, I will go through the process of the above flow:

1. Arduino and other sensors send in data over MQTT (Purple "houseTemp" and "houseHum") 

2. We need to combine the information into a single record and insert it into a database. (Light orange "Filter Dups & Insert")

 3. The data is then logged to a SQLite database

.4 At the same time, we select data from the database, so we can insert the newest data into some charts

5.  The results of that selection are fed directly into some charts:

 

 

 Detailed Process & Code: 

1. An Arduino sends in a message via MQTT with the msg.topic "houseTemp" and a float that represents the temperature (msg.payload), and another message with the msg.topic "houseHum" and a float that represents the humidity as the msg.payload.

2.  Then we combine the 2 results into 1, and insert the data using a function node:

 let temp = context.get('temp3') || null;

let hum = context.get('hum3') || null;

let ctr = context.get('ctr3') || 0;

if (msg.topic == "houseTemp") {

   context.set('temp3', msg.payload);

} else {

   context.set('hum3', msg.payload);

}

let sendMsg = false;

ctr = ctr + 1;

if (ctr >= 2) {

   ctr = 0;

   sendMsg = true;

}

context.set('ctr3', ctr);

msg.topic = "INSERT INTO HOUSETEMP (TIMESTAMP, TEMPERATURE, HUMIDITY) VALUES (" + Date.now() + ", " + temp + ", " + hum + ")";

if (sendMsg == true) {   

if (hum != null && temp != null) {

      return msg;

   }

} else {

   return;

} 

3. The data is logged to a SQL database, per the "INSERT" command in the previous function. 

4. Lets "select" or retrieve all the data including the data that was just logged:

msg.topic = "SELECT * FROM HOUSETEMP ORDER BY TIMESTAMP DESC LIMIT 1000";

return msg;

 

4. The database node responds immediately to the previous function, returning data

 

5. Display data in the charts:

 The main things to consider are having all of your variable names correct, and

 using the correct "properties" configuration in your Chart Node.

 

General DB Maintenance: 

In order to create a table, you need to add an "inject" node and set the msg.topic to something like:

CREATE TABLE HOUSETEMP ( TIMESTAMP INT PRIMARY KEY NOT NULL, TEMPERATURE float NOT NULL, HUMIDITY float NOT NULL)

 Since we are using a Linux timestamp, we can use the following formula to clean up old data via another "inject" node:

DELETE FROM HOUSETEMP WHERE TIMESTAMP < strftime('%s', 'now', '-14 days'); 

  

 Overview:

This generally works pretty well and keeps the graphs & charts up to date with good data, storing it for up to 2 weeks. Of course, this is a heavy-handed approach, replacing all the chart data each time. A better approach is identified below, using an external MySQL server to offload activity.

A bit better solution would be to make the SELECT function node only return 1 record at a time. Then, add a blank inject node that triggers only once on startup, that both resets the charts ( send JSON msg.payload of []  ), and loads all existing data into the charts. The arrays coming out of the database at the end would need to be reversed as well, so the newest data is at the end of the array.

This is a much less intensive way to go about it, but a little more complicated to set up. 

 

RF24: An advanced example for the Arduino Uno Q

 RF24: An advanced example for the Arduino Uno Q

 A new take on the RF24 GettingStarted example

In testing out the RF24 library and related features, I managed to get a few different things working together on the Arduino Uno Q to provide a more advanced example of RF24 interaction. 

 

 The main thing the sketch does is transfer data over RF24 radio link along with a standard Arduino device running the RF24 GettingStarted sketch included with the RF24 library.

Then, any data received is displayed in the Serial Monitor, Python Console and an associated webpage to demonstrate usage of incoming data etc.

 A button is included on the webpage that allows users to toggle the LED matrix on/off, which demonstrates how to pass data from Webpage to Python to Microcontroller. 

All in all its a simple concept, but it did take some figuring to work out how the functions and stuff all relate through JavaScript, HTML, Python, C++ etc.

My Arduino Q sketches can all be found here:

https://github.com/TMRh20/Sketches/tree/master/ArduinoQ 

For a limited time, a live real-time demo of a similar sketch will be available online here:

TMRh20s Example Webpage

The main functionality is tied into 4 files:

sketch/sketch.ino - The Arduino sketch handles radio input/output & passes its data to Python

python/main.py - The middleman between the webserver and the Arduino sketch

assets/app.js - JavaScript handling, communicates between index.html and Python via websockets

assets/index.html - The main webpage, provides variables that can be modified via JavaScript

Trying out the Arduino Uno Q - A quick and dirty overview

 Trying out the Arduino Uno Q - A quick and dirty overview

 Love it or hate it, it... exists 

 With the release of the new Arduino Uno Q board, I decided to get one and try it out. After a few days of playing around and experimentation I have to say I'm less than impressed.

 The concept seems to be a solid one, combining a standard Micro-Processor Unit (MPU) with a Microcontroller Unit (MCU), but the execution has been a little rough.

 The first and main thing to probably address is the Bridge concept, which is a way for users to make the MCU 'talk' to the MPU. It is driven by a serial connection running at 115200 baud, using RPC, which altogether is far too slow to pass copious amounts of data, or do it really quickly. On the other hand, it can be used to pass information between programs, be it Python to Python or C++ (Arduino) to Python.

 One of the first things I tried, of course, is running RF24 radio based applications. It appears to work OK initially with the core RF24 driver and RF24Network, but once I got into the RF24Mesh layers, I started to notice problems. I thought it was basic speed issues, but it turned out to be a radio power supply related issue it seems. After changing out the radio and calling 'radio.setPALevel(RF24_PA_MIN,0);', it worked.

Next, I attempted to create a basic TUN network interface in Linux using Python, to see about creating a RF24Gateway like interaction, but realized quickly that this too was outside the general capabilities of the device. It runs a Python virtual environment within a Docker container, with no apparent way to configure things, so I can't seem to get it working. That means an RF24Gateway style library is out of the question for now.

It seems that so much of what I've tried do to so far just doesn't work. There was also a forced, breaking update that happened recently, to version 3.0, where users have to manually download a new version of App Lab in order to program their devices.

 One essential thing to note, is that when the App Lab stops responding, or the Serial/Monitor output gets corrupted, I need to close App Lab, and run `arduino-app-cli system cleanup` from the command line to get it working again.

 Some of the features still seem promising, the ability to easily incorporate "Bricks" and features into your programs, and the underlying software for the MPU is Debian/Linux along with Arduino, so you can still do a lot with it. 

I don't think I'll get another one, but I will definitely be playing around with the one I've got for a while. 

In the meantime, I've included a new example in the RF24 core library that users can try with the Arduino Uno Q that allows users to configure the radio and print debug information.

I've also added an example sketch to go along with the stock RF24 GettingStarted example here:

https://github.com/TMRh20/Sketches/tree/master/ArduinoQ