A simple guide to managing memory and variables in C++

Transferring Data between Systems Using RF24

A simple guide to managing memory and variables in C++

One of the most common issues that seems to be coming up more and more recently relates to transferring data between devices and how different devices store data in memory. With more devices using 32 and 64 bit processors, integrating things between the newer devices and 8-bit MCUs can require a little bit of forethought and planning.

Memory:

What it all comes down to is memory and how different devices store data in memory. This is very important to understand when transferring data between systems and ensuring that code is portable between various devices.

When using an 8-bit device, such as an ATMega 328 based MCU (Arduino Uno, Nano, etc) the smallest unit for storage in memory is 8bits or 1-byte. This means that even a boolean (true/false) variable will take up 8-bits of memory space. On a 32 bit system, (ESP32, ESP8266) the same boolean value will use 32-bits of memory space. Why does this matter? When transferring data between systems or creating code, certain variables will take up different amounts of space in memory depending on the architecture (8-bit, 32-bit etc) thus introducing inconsistencies and misinterpretation of data.

Examples:

char is equivalent to int8_t
byte is equivalent to uint8_t
int is equivalent to int16_t on an Arduino Uno (8-bit device) and int32_t on ESP32 (32-bit device)
unsigned int is equivalent to uint16_t on Arduino Uno (8-bit device) and uint32_t on ESP32 (32-bit device)

As another example, see https://www.arduino.cc/reference/en/language/variables/data-types/int/

Using variables like int, long etc can cause confusion because they are interpreted differently on different devices. Sending a 2-byte integer from an 8-bit system to a 32-bit system works fine, as long as the size of the variable is specified on both systems such as int16_t, rather than trying to use 'int' on both systems.

Take for example the following data structure:

struct myStruct{

  bool variable1;

  uint32_t var2;

  uint8_t var3;

  int16_t var4; 

}

On an 8-bit system, this data structure will use 8 bytes of memory space, but on a 32-bit system it will take up 12-bytes. This is because the minimum unit for storage on a 32-bit system is 32-bits, so 32-bit devices will need to add padding wherever the data does not align in chunks of 4-bytes (32-bits). When using data structures, devices will typically attempt to pack the data as efficiently as possible, so re-arranging the data structure properly will prevent the 32-bit system from adding padding and making the data structures incompatible.

struct myStruct{  

  uint32_t var2;

  int16_t var4;

  uint8_t var3;  

  bool variable1; 

}

In the above example, the data is arranged in chunks of 4-bytes, starting with a 32-bit unsigned integer (4-bytes), followed by a 16-bit signed integer (2-bytes), an unsigned 8-bit integer (1-byte) and a boolean variable (1-byte), so the 32-bit system will pack the data into memory exactly the same as the 8-bit system, using only 8-bytes of memory space.

This is important, because when sending data between devices, users can just send the data structure as-is and if packed properly, the data will align regardless of whether it is a 32-bit, 64-bit or 8-bit system.

When developing code, specifying the size of all your variables and aligning data properly in your data structures will ensure that the code and communication is portable between devices.

https://www.gnu.org/software/libc/manual/html_node/Integers.html

Testing New RF24 Arduino Library Fork for NRF24L01 Radio Modules: Comparative Testing - Transfer Rates

Overview:

Per my previous blog posts, I have been working on some improvements to the RF24 radio library, and the main benefits of the changes have been realized. The following video compares the difference against the fork found at https://github.com/gcopeland/rf24. I did not compare against the main library since the gcopeland fork outperforms it quite a bit already.

The new library, source code and documentation can be found at https://github.com/TMRh20/RF24. Info in previous blog posts.

More to come...

 Arduino: Using the full potential of NRF24L01 radio modules
A New, Optimized Fork of the RF24 Radio Library: High speed data transfers and more!
Includes updates and new features for the RF24Network Library

Updated Jan 2015
  NRF24L01+ radio modules are very inexpensive, and provide a robust interface for transferring data wirelessly between devices with minimal resource and power consumption. I've been working with them more and more as time goes on, but have always struggled with some of the inner workings and limitations of the current libraries available.

After initially studying the operation of the radio modules and reviewing the details in the data sheet, I was convinced that the modules could perform much faster. Further research into the additional library forks, blog posts, and countless hours of testing revealed that the modules can be very sensitive to the timing of things. I also discovered a number of bugs and/or issues that would hinder performance and/or reliability. 

Initial testing proved very fruitful, with speeds maxing out the configured data-rate of the chip, and reliability was improved over previous iterations of the library. Over the course of the following year, the library has been further optimized and extended, with many new features, bug-fixes, and improved reliability and performance.

(See bottom of page for download links and documentation)

Taking Advantage of the Improvements:

From a user perspective, very little is changed from previous forks/libraries beyond the usage of the available() function. This is the main compatibility difference between this and the previous libraries. The available() function will always return 1 if data is available, to align the library with standard Arduino functions. From a technical perspective, the improvements are dramatic.

 Users will benefit from the improvements just by using the new library with old code, and some things not previously possible, can now be achieved. The included .ino examples have been configured to demonstrate 'standard' usage, but advanced users can still drive the chip outside the 'manufacturer recommended operation'

Additional functions have been added to aid in streaming or rapid-transmission situations, where 2 or more payloads are sent in rapid succession, or streamed at a high transfer rate. These include writeFast, writeBlocking, and txStandby. Use of these functions allow users to maximize throughput without overrunning the FIFO buffers. See the documentation for more info on usage. 


The addressing format has been extended to allow the use of 24,32 or 40-bit addresses, as well as defining and handling of addresses via byte arrays or integers.

For example, the following addresses are the same, and either format can be used:

uint64_t myAddress = 0x68524d5431LL;   ( Old Format   )
uint64_t myAddresses[] = { 0x68524d5431LL, 0x68524d5432LL};

byte myAddress[] = "1TMRh";                    ( New Format )
byte myAddress[] = {'1','T','M','R','h'};
byte myAddresses[][6] = {"1TMRh","2TMRh"};

Technical Info:
One of the primary factors in increasing efficiency was eliminating power ups and power downs from the general operation of the radio, which many of the existing forks already had identified. A power up takes 1.5ms or 1500us, where a transition from standby-I or standby-ii takes only 130us. In addition, leaving the CE pin high while data is written allows the possibility to have 0 delay if the TX FIFO buffer is kept busy, which this library makes use of via the writeFast() and writeBlocking() features.

This changes the operation of the radio a little bit, in that the radio needs to be powered up or powered down manually, instead of being powered down after every write.

The radio modules have shown to be very sensitive to the timing and order of operations, and many hours of testing and review have determined the optimal settings and order of operations to achieve the highest speed and reliability. Delays have been removed where possible, added where required, and minimized in every case.

As mentioned, the the available() and isAckPayloadAvailable() functions now simply check the FIFO buffer to see if a payload is available. This allowed simplification of the write() function, and should help to ensure that no packets are missed. Previous iterations used the interrupt flag, which can result in a number of issues.

The overall change in response is apparent when running example sketches like the GettingStarted_CallResponse sketch included with the library or the Transfer examples.

At this point, there are so many changes, bugfixes and details, it is best to see GitHub for all of the changes and technical info.

RF24Network: The standard RF24Network library has been updated to support the new changes, and a DEV version has implemented many new features like fragmentation/reassembly and multicast. This library is recommended if connecting any number of nodes, and provides addressing, routing, etc to help manage data in a network configuration. RF24Mesh is an overlay for RF24Network that provides automatic addrressing, and a dynamic topology for nodes running RF24Network and/or RF24Ethernet.

RF24Mesh: Automates addressing etc. for RF24Network and provides a seamless, self-healing network on top of all the layer 3 network features. 

RF24Ethernet adds a surprising level of reliability, consistency, and ease of use to nodes running RF24Network, by using standard TCP/IP networking. It uses the nrf24l01 radios as a standard Network Interface Card (NIC). It uses a Rasberry Pi or other Linux platforms as a network gateway, to allow RF24Network nodes to connect directly to web services or act as a web-server etc. Allows users to control nrf24l01 sensors or systems easily, reliably, and simply using any device with a web-browser, MQTT, etc. The API is very similar to the standard Arduino Ethernet library.

Class documentation now available here

Testing & Results:
Testing is now fully automated, with the introduction of TCP/IP support, with standard networking tools being used to test transfer speeds and reliability. Users can easily make use of the full data-rate in no-ack mode, full-duplex communication at 1MBPS with Ack-Payloads, or half-rate if using Enhanced ShockBurst (ESB). This has allowed many improvements to related libraries like RF24Network as well as the low level RF24 radio driver.

Since the initial release, many users including previous RF24 contributors have adopted the new library and provided feedback towards further improvements .See here for a demo of the initial transfer rate testing.
 
The wireless audio sketch/library that inspired these improvements is now linked below. The audio library itself is limited to around 16-20khz sample rate, which produces very reasonable sound quality for voice transmission. The limitation is due to the use of interrupts for virtually every part of the library, however, this makes it very simple to configure and use. See here for the development sketch, which was used to design the library, and is capable of higher quality audio. The wireless audio portion of the TMRpcm library has been updated to allow audio streaming and multicasting directly from SD card over RF24 modules as well.

New:

See the new store for reliable RF24 hardware and kits.

Reference Material/Libraries:

Raspberry Pi Simple Library Installer:
http://tmrh20.github.io/RF24Installer/RPi/install.sh

RF24 - Low Level Radio Driver (Generally used for device-to-device communication)
TMRh20 RF24 Fork on GitHub  - Download
RF24 Documentation 

RF24Network (Provides addressing, routing and many other features for use with multiple devices)
TMRh20 RF24Network Fork on GitHub - Download 

RF24Mesh (Dynamic 'mesh' layer for RF24Network)
RF24Mesh on GitHub - Download
Documentation

RF24Ethernet (Modelled after Arduino Ethernet API - TCP/IP over RF24Network)
RF24Ethernet on Github - Download
Documentation

RF24Audio (Digital Audio over RF24 radio modules)
RF24Audio Library

MySensors.org - Create user friendly sensor networks using RF24 radio modules

Alternative: RadioHead NRF24 Library - (Supports multiple radio devices)

NRF24L01 Data sheet
Original/Old RF24 Library by ManiacBug