NRF52840 and Arduino: Encrypted and Authenticated Radio Mesh Networks

 NRF52840 and Arduino: 

Encrypted and Authenticated Radio Mesh Networks

In a recent post, I mentioned using the on-board encryption module of the NRF52840, and I've made some decent headway so far! I've finally gotten my encrypted driver working with RF24Network and RF24Mesh and am pretty happy with the results. 

One of the nicest parts of this is it doesn't add any overhead to RF24Network or RF24Mesh itself, since the encryption is incorporated at OSI layer 2. This means that when we transmit a 32-byte payload with RF24Network or RF24Mesh, all those layers see is the unencrypted data. The radio at layer 2 will add some data onto each payload and encrypt it, then decrypt it and present the unencrypted data to the higher layers.

For example, the current overhead of this implementation for encryption is 12-bytes. A 5-byte Initialization Vector (IV), a 4-byte Message Authentication Code (MAC) and a 3-byte packet counter. The radio driver adds this data on in the background, so if you have a 24-byte payload to send, RF24Network adds on its 8-byte header and the 32-bytes of data is sent to the radio driver. The radio driver will then encrypt the data, send an actual 44 bytes of data over-the-air, then decrypt it and present 32-bytes of data back to the RF24Network layer.

Tip: If using this library with the RF24Network or RF24Mesh layers, open RF24Network.h and set the MAX_FRAME_SIZE to 111. Then, per the included examples, the higher layers will handle the larger packet sizes that the NRF52840 is capable of. Make sure to set it back to 32 for communication with nrf24 devices.

There are still a number of questions I have regarding this implementation, but in any case, the library is functional, but don't quite expect full security just yet. There are a number of things to be investigated and worked out regarding how exactly this CCM module is supposed to be driven.

Another thing that is intriguing here is the break from the OSI model, since encryption is typically defined to be implemented at the presentation layer, way high up on the stack. I don't know why, but it feels like a better opportunity to encrypt as much of the data as possible, publicly transmitting only what absolutely needs to be public information. The design of the RF24 communication stack generally follows the OSI model, so the hope is that this will make up for a lack of presentation layer moving forward. It looks like some tasks can still be left up to the presentation layer, like rekeying and managing timestamps to prevent things like replay attacks.

The library with encryption enabled has now been released, get it via Platform IO, Arduino Library Manager or directly from GitHub for the latest changes. Note that Platform IO will install the proper RF24Network and RF24Mesh dependencies, while Arduino Users would need to install the separate RF24Network and RF24Mesh branches manually for now.

nRF24l01+ Library Roundup

Overview and Status of RF24 Arduino/Linux Comm. Stack

Six, seven years? Has it really been that long? According to GitHub that is in fact the case, as my first commits to my own fork of the RF24 library took place in early 2014. It all started by identifying existing limitations in the available libraries and working to achieve the highest level of performance and reliability possible. The main fork for RF24 at the time was written and maintained by ManiacBug, but he dropped off the scene shortly after publishing this very nicely designed library. Other users attempted to add support for various devices and address certain bugs, but nobody had really taken a thorough look at the capabilities of the hardware vs what was achievable at the time. Only through a lot of reading, testing and learning to program was I able to make those initial changes to begin work improving the RF24 core library. Looking back now, I am very grateful that I picked ManiacBugs code base as a place to get back into programming in C/C++, as it was well thought out and I discovered ways of doing things that I previously did not understand or know about.

And with that began a long and arduous journey into the internal workings of nrf24l01+, Arduino (AVR) and RPi (Linux) devices with the stated goal of optimizing the library to whatever extent possible. Once the RF24 library began to take shape, I began looking at the RF24Network library, another very nicely designed bit of code by ManiacBug, but it had its limitations and problems. Many issues were addressed, features and functionality added in order to push the nrf24 devices to the test in a multi-device scenario. It performed much better than expected using this OSI Layer 3 (network) library.

At that point, I began to really understand the OSI model, how the different layers actually work together and what they do down the last bit. Based on a number of conversations and user input, I came up with the idea for RF24Mesh, which is basically another layer on top of RF24Network -> RF24 that handles addressing, similar to DHCP, but able to verify connectivity and reconnect nodes at any point in the mesh. This allows nodes to move around and reconnect quickly as required while maintaining the mesh structure. The Network & Mesh layers use a 'master' node that acts like the gateway in a standard IP network, and it provides addressing and resolution for RF24Mesh.

Based on input and discussions with users and the performance of the radio modules when used in a multi-device mesh/network scenario along with the emergence of IoT and sensor networks, it seemed like the thing to do would be to work my way up the OSI model, so that is exactly what I did. Upon discovering the uIP stack for Arduino, I realized it would be possible to add ip4 support to some of the smallest devices like ATmega328 AVRs, and larger devices like RPi could just use their own IP stack and encryption etc. The uIP stack and related software was quite a challenge to understand and implement, as uIP is written in C and is designed to be as small and lightweight as possible as opposed to being human readable and easy to understand. The UIPEthernet library was also essential in helping me to understand and implement RF24Ethernet.

With that, an OSI RF24 comm. stack was established, with RF24, RF24Network, RF24Mesh, RF24Ethernet and RF24Gateway libraries all working together and inter-operational at any of the layers. This means that on a RF24Gateway/RF24Ethernet mesh network incorporating RPi and Arduino nodes, devices can choose to operate using OSI layer 2,3,4,5 and/or 7 (Data Link, Network, Transport, Session, Application). When using the higher layer libraries (RF24Gateway & RF24Ethernet) standard IP connectivity is established and devices can utilize standard encryption & authentication protocols (SSH, HTTPS etc.) to secure their traffic as per the OSI model.

The RF24 Communication Stack vs The OSI Model

With the associated libraries and enhanced functionality, the RF24 comm. stack is able to provide IoT and sensor network connectivity and capabilities to suit many different scenarios at a very low price point with much lower power consumption than WiFi networks. With the production of many devices like RPi, ESP8266 and ESP32 which support WiFi, users can construct and deploy IoT networks very easily, extending them as required.

Even now, it is really kind of cool to witness the system in action, using these little radios on tiny little computers we refer to as 'Arduinos'. The methodology and processes in place to manage a network of devices, fragmentation & reassembly, mitigate wireless data collisions and ensure delivery of data are quite effective, from the radio hardware and auto-ack functionality, all the way up to the TCP/IP and Mesh levels. With TCP/IP for example, using Raspberry Pi devices, the MTU is 1500-bytes, so each packet can require up to 63 sequential, successful data transmissions at the core RF24 layer, but this still works relatively well in real-world testing/usage.

It has been a lot of fun and a great learning experience so far. The creation of RF24Gateway really allowed the limits of the radios to be tested, and provided an excuse to play around with different network/IP configurations, routing scenarios and traffic handling at any level of the OSI model. A full and complete understanding of how devices operate on the internet or similar networks helps dramatically in troubleshooting, testing and development of systems and software that operate using these protocols.

Going forward, the development of the RF24 libraries has slowed significantly along with bug reports and issues. My focus has recently been on cleaning things up, finding and fixing bugs, mostly in the higher layers, and improving the user interface via examples and documentation. I had previously almost given up on the RF24Gateway and Ethernet layers as being too difficult to fix, but with the rise of more powerful MCUs and the potential for expanding their capabilities, it seems worthwhile to take another crack at working out bugs and issues.

 It would be nice to find another similar radio device with lower power consumption and/or more advanced capabilities and throughput to create more robust networks and mesh capabilities. The current design allows for speeds up to about 20KB/s over IP (RF24Gateway) using two Linux devices, but it would be nice to have a higher throughput to allow for more nodes, longer range and more advanced communication scenarios.

In conclusion I want to extend a big thank you to everybody who contributed ideas, analysis, information and programming skills along the way. An extra thank you to Avamander (GitHub) who has played a big part in ongoing maintenance, support and development!

Update: Aug 2020

Wow, I mentioned that I would be working on finding and fixing bugs due to the slowdown involving support, dev and maintenance etc, but did not expect to spend so much time, or that I would be able to identify so many issues, their root cause and a solution. Some pretty significant issues have been fixed throughout the RF24 stack, including 1 major bug in RF24, affecting all libraries, a number of bugs/issues affecting functionality and reliability in RF24Network and RF24Mesh, and a memory issue in RF24Ethernet affecting stability.
RF24Gateway has been updated with better handling of interrupts, a few corrections and the first release made, v1.0 due to all the issues that have been addressed. Again, a portion of these improvements are due to assistance, testing and input from a select few users. Users should notice a marked improvement in stability, functionality and the ability to recover from significant errors/hardware issues.

RF24 Communication Stack:
https://github.com/nRF24
https://nrf24.github.io/RF24Gateway
https://nrf24.github.io/RF24Ethernet

RF24 Addressing: A review of NRF24L01+ radio addressing

Pipes, Addresses and Acknowledegments

There still seems to be a fair bit of questions, misconceptions and misunderstandings regarding the nature of RF24 radio addressing and how to set up communication scenarios. This is a review of RF24 pipes, addressing and acknowledgements that should provide any user with enough information to set up any supported configuration.



Acknowledgements (ACKs):

The radios support an automatic acknowledgement feature, enabled by default, in which the receiving radio switches into transmit mode after reception, and acknowledges reception of data.

1. A sends to B
2. B responds to A
3. A is now aware that B received the payload successfully

The acknowledgment process is very important to understand, because correct addressing is required for this feature to function properly.




Simplest Explanation/Rules of Addressing (Using ESB/Auto-Ack):

1. When using Auto-Ack, every radio can utilize a maximum of 6 unique addresses at once, which other radios can send data to.

2. Each address is assigned to a logical "pipe", and each radio has only 6 pipes that can be assigned addresses.

3. Since auto-ack implies a bi-directional communication channel, each address/pipe can only support communication to a single radio using acknowledgements. If two or more devices send data to a single pipe/address, auto-ack should be disabled to prevent ACK spamming and erroneous acknowledgements.
   Exceptions:
   a: Disabling Auto-Ack - This prevents excessive spamming of acknowledgements and errant acknowledgement.
   b: Timing of Transmissions - Using a token-ring style network or other method of preventing simultaneous transmissions would allow multiple radios to send data to the same pipe/address while maintaining the reliability of acknowledgements.

Note: At 1MBPS, a transmission can take up to 85ms or so, and overlapping transmissions are likely with any significant traffic

4. Per manufacturer design, the addresses assigned to pipes 1-5 must be the same, except for a single byte.



Basic Scenario - One to One Communication with AutoAck:

For purposes of standardization and to allow the system to be scaled, it is generally recommended to use two radio addresses. With auto-ack disabled, a single address can be used for all radios, or groups of radios.


Code:
Radio #1:
byte address_1[] = {0xCC,0xCC,0xCC,0xCC,0xCC};
radio.openReadingPipe(1,address_1);
Radio #2:
byte address_2[] = {0xC3,0xCC,0xCC,0xCC,0xCC};
radio.openReadingPipe(1,address_2);


Process - Radio #1 sending to Radio #2:

1. Radio #1 opens pipe 0 for writing (and internally, reading) using address of Radio #2 (CCCCCCCCC3)
2. Radio #2 receives the data on the first attempt, and (internally) opens a writing pipe using its own address (CCCCCCCCC3) and sends an acknowledgement back to Radio #1
3. Radio #1 receives the ACK, so does not engage in a series of timed retries





Basic Scenario - One to One or Many to Many Communication with Multicast:

With AutoAck disabled, radios can share the same address, and the system can still be scaled to support any number of devices.

Code:
byte address_1[] = {0xCC,0xCC,0xCC,0xCC,0xCC};
radio.setAutoAck(0);
radio.openReadingPipe(1,address_1);

In this situation, any radio can send a message to all other radios sharing the same address:

Code:

radio.stopListening();
radio.openWritingPipe(address_1);
radio.write(&data,sizeof(data));
radio.startListening();





Simple Mesh Scenario - Five-to-Five Communication with AutoAck:




In the above scenario any radio can send or receive data from any other radio in the mesh, thus creating a 'true mesh' of 5 nodes.

Radio #1 will always open a writing pipe using the address assigned to pipe#1 of the recipient.
Radio #2 will always open a writing pipe using the address assigned to pipe#2 of the recipient.
Radio #3 will always open a writing pipe using the address assigned to pipe#3 of the recipient.
... and so on

Process Confusion:
In this example of addressing, if Radio#2 and Radio#3 were to both open a writing pipe to Radio#1 using the same pipe/address, there would be confusion with acknowledgements. Radio#1 will respond to both devices with acknowledgements, since they have both opened a writing and reading pipe using the same address.




Complex Network/Mesh Scenario - Many-to-Many Communication with AutoAck:


As can be seen, this format of addressing is very similar to creating sub-nets with IPV4 addressing, except here we are using HEX instead of decimals, and there are 5 bytes per address instead of 4. Each radio is linked as shown in the image below, and each radio has a unique communication channel (pipe/address combination) to use when communicating with any of its parent or child nodes.

Due to the correlation of addresses and associated topology, each radio only needs to be aware of its parent and child nodes. Any traffic sent to a given node will either be a: Accepted b: Routed to parent c: Routed to a child

Address configuration, routing etc, can be managed very simply by masking and bit-shifting.





In the above example, pipe0 is NOT assigned a reading address, leaving it available to be used for broadcast or multicast. With AutoAck disabled on a single pipe, or by using selective acknowledgements, all radios or groups of radios can share an address, thus all radios will receive all messages sent to that address.

See the RF24Network Documentation and Source Code for more information regarding this type of configuration.



Summary:

In many low-throughput situations, the radios will appear to function correctly even with improper addressing, and may not cause any obvious problems, however, performance and functionality will most certainly be affected.

In any situation where reliability and acknowledged payloads are important, proper addressing and an understanding of the underlying processes behind acknowledged payloads is very important.

RF24Gateway - Creating Hybrid IoT Networks using TCP/IP and RF24Network

 RF24Gateway User Interface Example

RF24Gateway is a new complimentary library to RF24Ethernet, designed for RPi and Linux based devices with SPI and GPIO capabilities (BBB, Intel Edison, Galileo, etc).

The RF24Gateway library acts as a network/internet gateway, and allows Arduino/AVR based sensor nodes to interact with each other, the environment, and the world using TCP/IP and/or lower level RF24Network messages.

In short, it allows hybrid IoT networks to be created using Arduino and nrf24l01 modules. Some nodes are capable of internet or local network communication using standard protocols (TCP/IP) and/or RF24Network messages, while other nodes can communicate only at the network level, using only the lower layer RF24Network messaging.

Once RF24Gateway is configured, nodes can be controlled completely using standardized tools and protocols. Additional client examples have been added to demonstrate how to control and/or receive information from sensor nodes using standard tools and scripting methods via Bash, NodeJS and Python scripts. Any method of establishing a TCP connection can be used, with the examples generally using HTTP to some degree as well.

Libraries and code that utilizes RF24Network can utilize RF24Gateway in much the same way, since external data types (TCP/IP) are handled out of sight from the user. RF24Network is used exactly the same way, except that the gateway.update() function is called instead of network.update() or mesh.update(). The included examples demonstrate usage.

Note: RF24Gateway defaults to using a TUN interface with RF24Mesh enabled. RF24Ethernet examples have been updated to accommodate this.

Current Status:

April 2015: Near completion. Integrated fully with RF24Ethernet. Graphical (NCurses) interface example finalized.

Source Code:
https://github.com/TMRh20/RF24Gateway

Documentation:
http://nrf24.github.io/RF24Gateway

Download (See documentation for installation info):
https://github.com/TMRh20/RF24Gateway/archive/master.zip

ENC28J60 Network Modules, Arduino and the Internet of Things (IoT):

An experiment, overview and analysis of TCP/IP networking with Arduino Ethernet

The module in question was sent to me by the friendly people at ICStation.com , in exchange for a review of the module itself.

Video Review:

ENC28J60 Mini Network Module:

• 3.3V operation
• Create a mini Arduino Web Server or Client
• Plugs into your home router or modem just like a computer

The uIP TCP/IP stack is software driven, while the ENC28J60 module handles the data mainly at the network (Ethernet) level. One main difference between UIPEthernet and the official Arduino Ethernet library is the need to keep the TCP/IP stack updated via software with the ENC28J60.

Hardware: 

The ENC28J60 was very easy to wire up, using the default SPI pins shown at http://arduino.cc/en/Reference/SPI along with the VCC and Ground pins.

The device seems to work ok connected to the 3.3v output of the Arduino, but it seems best to use an external power supply or battery.

Software:

Two main libraries appear to be available: Ethercard and the UIPEthernet libraries. This post will focus on usage via UIPEthernet, since it shares a common API with the official Arduino Ethernet library.

I am using the UIPEthernet library and examples found at: https://github.com/TMRh20/arduino_uip/tree/fix_errata12

In attempting to utilize the modules and UIPEthernet for creating a simple webserver, I ran into a number of issues with data being sent or received incorrectly. Over a number of days, and benefiting from the information gained in building the RF24Ethernet library, I have been working to improve UIPEthernet, and also created some examples to demonstrate the issues and current usage.

(A pull request has been made regarding these suggested changes to the main library)

Overview of Library Changes (At the time of this post):
Changes:

• Introduce client.waitAvailable(); function  - Ensures the IP stack is processing incoming data while waiting for incoming data for a maximum defined duration in milliseconds: client.waitAvailable(3000);
• Introduce Ethernet.update(); function - Allows user applications to ensure the IP stack is processing data while delaying or performing other tasks: Ethernet.update();
• Perform additional processing of data while calling client.available();
• Per testing, reduce max segment size (MSS) and the receive window to 511 bytes from 512 due to unidentified issue.
• Reduce number of packets per socket to prevent data loss
• Re-open the TCP window at timed intervals when data has not been sent or received on an open connection to prevent stalling during large downloads
• Fix for outgoing data corruption
• Force application to wait while writing data 
• Better TCP window handling
• Other minor changes, add examples

Changes - Technical Summary:

The first changes introduced added some new functionality, geared towards a software driven IP stack.  With the addition of additional processing and the Ethernet.update() function, the IP stack is more responsive to ICMP requests etc.

The new examples included with my branch of the library clearly demonstrate some of the issues I encountered when attempting to create a simple web client and/or web server. The first issue I noticed involved the device crashing or stalling, and then failing to reconnect or respond to pings. Testing indicated that the hardware itself was still responding and receving packets, so the issue seemed to be software based. These issues mostly seem to be addressed in the fix_errata12 branch, but testing via large dowloads from a web server indicated there were still some remaining issues, mostly surrounding data corruption and flow control.

Most of the remaining changes attempt to address issues with corrupted or out of sequence data, and to manage the flow of data better. In some situations, for example, web servers will send TCP payloads equal in size to 1/2 of the TCP Max Segment Size (MSS) which causes problems with flow control. The IP stack will now only close the TCP window if data is being received fast enought to do so, which appears to reduce issues with these transfers. The main issues come when a sending device sends only 1/2 MSS, and the recipient closes the window. The library will now count individual bytes in memory vs packets with data in them, and only close the TCP window if the assigned data buffers are completely full.

Testing has been very successful, with the device demonstrating full functionality for extended periods of time, downloading 70+MB of data without any errors detected in the data or hardware. The new web server examples and HttpClientTest.ino example seem to be good displays of the current capabilities and error free operation, and can be used to demonstrate the issues, when used with the original library code, or when changes are reverted.

Summary:
 
The mini ENC28J60 modules are an inexpensive and effective way to add internet connectivity to any Arduino project, and their size make them very suitable for use with Arduino.

I found the hardware itself to be very reliable and easy to use, although there seemed to be a few minor issues with the related Arduino library (UIPEthernet) that was tested.
As noted, I have submitted some workarounds/fixes for review and possible integration into the main library at the time of this post, so users are encouraged to use the library linked above.

ICStation.com:

A big thank you to www.ICStation.com for sending me this product, as well as being very patient.

It has been a number of months since I actually received it, but their small contribution has lead to the creation of the RF24Ethernet TCP/IP Radio Network library, and some potential improvements to the UIPEthernet library. A wealth of Arduino and Raspberry Pi related products, gizmos and devices are available on their website, and their staff seem very friendly.