RF24Gateway - What Does It Do And How Do I Use It?

 RF24Gateway - What Does It Do And How Do I Use It?

 A simplified explanation...

 So maybe you've taken a look at the RF24 communication stack and have seen RF24Gateway, along with the RF24Ethernet libraries, and are wondering what they do? I've noticed some confusion among users who aren't sure what the libraries do exactly.

 The simplest explanation, is that it allows users to use nRF24L01 radios as standard Network Interface Cards (NICs). Once RF24Gateway is installed, its like having a standard NIC linked into the wireless RF24 radio network.

This means users don't need to understand programming, RF24 APIs, or anything much about how the RF24 comm. stack operates in order to use RF24Gateway &/or RF24Ethernet. It means users can use standard networking tools like the 'ping' command to test network connectivity, or more advanced tools & protocols like MQTT, HTTP, etc to interact with their Arduino devices or other Raspberry Pi/Linux devices wireless, over RF24Gateway & the radio network.

Linux users, for example, can connect to other Linux devices over the RF24 based network by using SSH, SSL or other secure protocols.

A good general understanding of standard networking and protocols is probably going to be beneficial to anyone trying to make use of RF24Gateway/RF24Ethernet. The system uses the proprietary ESB protocol by Nordic for communication, and although it operates on similar frequencies to WiFi, it is not a WiFi based system. However, it does allow your RF24Ethernet & RF24Gateway devices to interact with WiFi devices on your network, because RF24Gateway can link the WiFi & RF24 networks together using standard networking techniques like forwarding and routing.

 This means advanced users can also do things like link RF24 based sensor networks together by using VPNs, SSH tunneling, etc and create a centralized environment to collect and display data, control devices, or otherwise interact with the networks. I've experimented with automation systems like Home Assistant and Node Red to create these types of networks myself, and there are a number of alternatives that can be easily integrated into the system. Networks can work over very wide geographical areas, covering many KM locally, via radio link, and spanning the globe using VPNs or SSH tunneling.

 RF24Ethernet is companion software to RF24Gateway, which is meant to run on smaller devices like Arduino Uno, Nano, etc. while RF24Gateway is designed solely for Linux devices. RF24Gateway typically requires little to no programming, while RF24Ethernet lets users build web servers, sensor nodes, interactive nodes with on/off switches, lighting controllers, etc. and customize things to the nth degree, or simply build off the included examples.

 ESP32 based devices have become extremely popular, but in a large scale sensor network, WiFi is a very power hungry alternative to nRF24 based devices, which can offer similar features, wireless protocols & connectivity. 

To sum it up, RF24Gateway & RF24Ethernet provide connectivity & functionality similar to a WiFi network, but using much less power, and are able to support the smallest devices like ATTiny all the way to larger, much more powerful computers like Raspberry Pi and other Linux based devices that support GPIO & SPI capabilities. By default, the system uses the RF24Mesh and RF24Network libraries, so all of the internal networking and routing is handled automatically.

 

Creating IoT Sensor Networks & Systems using nRF24 & nRF52840 - An Analysis

 Creating IoT Sensor Networks & Systems using nRF24 & nRF52840

A Brief Analysis

In playing around with the nRF24 and nRF52x devices, I've been running and monitoring a couple large IoT networks of them over the past few years. In that time, I've developed a system that works very well, with nodes staying connected over long periods of time, despite the mesh protocols and need to re-establish connectivity periodically.

The system is based on TCP/IP & Node-Red, and makes heavy use of HTTP & MQTT protocols. Nodes either interact with an MQTT server, or are polled periodically via HTTP requests. Devices range from Raspberry Pi running RF24Gateway & Node-Red, to a wide range of Arduino devices utilizing RF24 and nrf_to_nrf libraries.

A graph showing connection times for various network nodes

 The graph above shows connection times for nodes connecting via MQTT. They report in periodically, showing how long they have been connected to the MQTT server. By default, in the MQTT library I use, the keep-alive interval is 10 seconds. This means each node must communicate with the MQTT server successfully every 10 seconds or be disconnected. Given the mesh nature of the system, this works out very well, with nodes staying connected for hours and days at a time.

With the default logic in place, it is very interesting to see how nodes assemble themselves around the master node over time. At first, it can be very chaotic, with many nodes on a network, each grabbing a unique identifier and place in the network. Over time, the nodes with the best connectivity find their way to a direct connection with the master node, or at least a secondary connection, with only one routing node between them and the master node.
One would almost think there was a more intelligent algorithm handling placement of nodes in the networks, but it is a relatively simply methodology, with nodes contacting the master node first, then secondary (children) nodes, then children of those nodes, etc. in order to establish a connection to the mesh. Nodes will also contact their parent node periodically to verify connectivity to the mesh.

 

 Ping Response Times for a Connected Node

 The above picture shows ping response times for a connected Arduino node. At the left of the top graph one can see that the node was able to re-establish its connection with better connectivity to the master node, not having its traffic routed through another node. Response times vary, with the node typically responding in about 8mS, up to around 275ms in some cases when there is a direct connection to the master node.

"External Network Activity"

 

The above picture shows the results while "external activity" interferes with the network. During this time, due to the nature of radio and jamming techniques, most nodes on the network lose connectivity to the mesh, but the self-healing nature of the network ensures that they only go offline while the jamming is taking place. Nodes quickly re-converge around the master node as the mesh re-establishes itself, and every sensor or device resumes normal operations. 

HTTP based nodes reporting in during the jamming

 

I recently updated many of my nodes, due to bug fixes and testing, per the above graph. In it, as can be seen, some of my HTTP nodes lost connectivity during the jamming. Others stayed online with only small blips in connectivity. Depending on proximity to other nodes and the master node, as well as the location of the jamming device, different nodes are affected differently, but again, they all resume operation after the interference stops.

Temperature & Humidity Readings from a Connected Node

This system works extremely well, with a distant node reporting in its temperature and humidity over time. You can see in the chart above, it has been reporting in consistently.

 

Ping Statistics over a Number of Days

 In the above graph, you can see the success rate of pinging one of my sensor nodes over the course of a few days. In this picture, the success rate is down a bit from the norm, due to ongoing "external network activity" beyond my control. This has been the case for years and years, with one of my neighbors continuously interacting with my network on an almost daily basis. I can't understand the obsession, but I've witnessed it on many occasions.

Conclusion:

The end results kind of speak for themselves. In a given networking scenario where reliability and consistency is key, making use of the advanced features of TCP/IP networking in combination with HTTP & MQTT allows users to construct networks to suit many different scenarios. This system is highly reliable and can be customized to the nth degree. 

The combination of nRF24L01 & nRF52x radios works quite well, and allows even more advanced functionality in a given network. With the 52x devices supporting things like encryption, Bluetooth & I2S, users can make great use of these features, with everything being done wireless.

Since we are using a mesh based system, and TCP/IP was designed to operate even on partially damaged networks, the combination works very well. Add that to recent fixes and updates in the RF24 communication stack, and you have a system that works extremely well over long periods of time. I measure uptime on the network, and nodes tend to only go offline when there is a power outage, hardware failure or they are manually taken offline.

Considering that there are great quality nRF24 based radios available with range anywhere from 2.5-5 KM (E-01 ML01DP5 or 2G4M27D), networks can be constructed that cover wide distances and manage themselves automatically. The network will support the smallest of devices, ranging from ATTiny all the way up to Raspberry Pi and other Linux based devices that have GPIO & SPI capabilities.

Libraries & Software Used:

1.  RF24, RF24Network, RF24Mesh, RF24Ethernet (Arduino) & RF24Gateway (Linux/Raspberry Pi)

2. nrf_to_nrf radio library (nRF52x devices)

3. Node-Red (Linux/Raspberry Pi)

4. Mosquitto (MQTT broker)

 

All of the software and systems used are open source, so anybody can create similar systems. Any software that is MQTT compatible like Home Assistant or Node Red will work.

Recreating nRF24 greater than 32 byte payload issue with nRF52840

 Recreating nRF24 greater than 32-byte payload issue with nRF52840

A D.O.S. attack against nRF24L01P that do not validate Dynamic Payload Sizes

 I made a video demonstrating how to send 33 byte payloads to the nRF24L01P and either cause the device to flush its RX buffer, or create a denial of service attack for any devices that do not validate Dynamic Payload sizes.

 

I find it interesting that the nRF24L01P devices accept these payloads at all, as well as sending an ACK to the transmitting device.

 

The nrf_to_nrf Radio Library for nRF52x Devices and nRF24L01 Compatible Communication.

 The nrf_to_nrf Radio Library for nRF52x Devices and nRF24L01 Compatible Communication.

A brief tutorial on payload sizes & handling 

Max Payload Sizes: The nrf_to_nrf Core Library

   The nrf_to_nrf library allows many configuration options, but here I want to talk about payload sizes and communication using the core nrf_to_nrf library, as well as that in combination with RF24Network and higher libraries.

 The max payload size of the nRF52840s is 127 bytes. This is with CRC disabled and static payload sizes. The radios support 0, 8, 16 or 24-bit CRC, so that extra data takes up 0, 1, 2 or 3 bytes depending on your chosen options. nRF24 devices only support 0, 8 or 16-bit CRC, so users are limited in their selection when communicating with nRF24 devices.

 Similarly, the nRF24 will only handle 32-bytes maximum payload size, with any CRC value and either static or dynamic payloads. The nRF52x devices are slightly different, so one needs to be aware how changing these options affect the max payload size.

Max Payload Sizes:

1. Static Payloads, CRC Disabled: 127 bytes

2. Static Payloads, 16-bit CRC: 125 bytes

3. Dynamic Payloads, w/16-bit CRC: 123 bytes

4. Static Payloads, Encryption enabled, 16-bit CRC: 113 bytes

5. Dynamic Payloads, Encryption enabled, 16-bit CRC: 111 bytes

 

Max Payload Sizes: The RF24Network Library 

 The above is with the core nrf_to_nrf driver. Once we get into libraries like the RF24Network library, another 8-bytes of data are required for the network header, which contains information like the destination address, sender address, and a counter.

This means that when communicating solely between nRF52 devices, we almost always want to configure the max payload size to 123, since RF24Network will figure out the max payload size based on this information, and fragment payloads accordingly. By default RF24Network is nRF24 compatible, so only is configured to handle 32-byte payloads.

To configure RF24Network, before calling network.begin(); we want to call radio.begin(); followed by radio.enableDynamicPayloads(123); This will allow maximum payload sizes prior to fragmentation.

The same is true when using the RF24Mesh library, call radio.begin(); followed by radio.enableDynamicPayloads(123); prior to calling mesh.begin();

If using encryption, the RF24Network layer will take this into account, so the max payload sizes with RF24Network are as follows:

 Max Payload Sizes in RF24Network:

1. RF24Network, default: 24 bytes before fragmentation, 144 bytes with fragmentation

2. RF24Network, w/enableDynamicPayloads(123): 115 bytes before fragmentation, 144 bytes with fragmentation

3. RF24Network, w/enableDynamicPayloads(123) + enableEncryption = true: 103 bytes before fragmentation, 144 bytes with fragmentation