Monday

The RF24 Communication Stack: What are the different layers and which one do I use?

 The RF24 Communication Stack:

What are the different layers and which one do I use?

Over the past number of years, I've been developing a communication stack for nrf24l01+ radios, generally to be used with Arduino and Raspberry Pi devices. The comm. stack is organized into separate layers, per the OSI model, which allows users to establish communication between any number of devices depending on their needs.


1. The RF24 layer: This layer is generally used for simple device-to-device communication, where speed and simplicity are key. One example is for radio control of a single device, where a stream of small data packets are communicated from one device directly to another in rapid succession. Streaming of audio in real-time is another example where the RF24 layer would be used directly. Using the RF24 layer can be a bit complex, and some knowledge of radio communication is beneficial.

2. The RF24Network layer: This layer expands on the RF24 layer by providing a number of features, both to enhance radio communication and provide users with the features of an OSI layer 3 (network) layer. RF24Network nodes are arranged in a static tree topology, and this layer provides all the features to manage them. RF24Network handles routing, fragmentation/reassembly of large packets, and communication to any other device is straight-forward. Users don't require knowledge of the RF24 layer to operate a network of nodes at the RF24Network layer, and it can be much simpler than using the RF24 layer directly. Recommended when not able to use the RF24Mesh library, when there is a need for static nodes, or to simplify one-to-one communication.

3. The RF24Mesh layer: This layer expands on all underlying layers by providing an automated, self-healing network that allows nodes to move around and or re-establish connectivity as required. This layer is generally recommended when creating a network of devices, as it automates addressing for RF24Network, and provides a more seamless interaction. Users generally communicate using the RF24Network API, with RF24Mesh providing address lookups and automation for the network. 

4. The RF24Ethernet/RF24Gateway layer: This layer allows communication using standard networking protocols, such as TCP, UDP, ICMP. It is generally recommended to run RF24Gateway on a Raspberry Pi as the master node, with smaller devices running RF24Ethernet. Allows very simple or very complex communication scenarios, with users requiring little to no knowledge of radio or RF24 programming APIs. 

The RF24Ethernet API is very similar to the Arduino Ethernet API, and RF24Gateway allows users to use standard networking tools to communicate with devices running RF24Ethernet. Recommended for communication scenarios where speed is not essential, but reliability and consistency is, such as a home automation system using MQTT and Node-Red and/or a sensor network reporting data. Users can run a wireless, Arduino based webserver, or interact with nodes using their mobile device over MQTT, HTTP, etc. 


As can be demonstrated, each layer has its place, providing communication capabilities for a very wide range of scenarios, and allowing users to benefit from the built-in features or to dig right in and customize things to the nth degree. Generally, the higher you go up in the stack, the simpler the interaction, with complex underlying code providing the simplest and most advanced user interaction.


Related Code and Documentation:

https://github.com/nRF24/


Sunday

Sensors & IoT Stuff: Home Automation with NodeRed, RF24Ethernet/RF24Gateway & MQTT

Sensors & IoT Stuff: Home Automation with NodeRed, RF24Ethernet/RF24Gateway & MQTT 

Control a homemade, Arduino-based RGB light, other devices or display data using NodeRed


I made a video demonstrating how to configure NodeRed with RF24Ethernet/RF24Gateway to control a homemade, Arduino-based RGB light. The process is fairly straightforward, with the Arduino node subscribing to the MQTT server, and the NodeRed device publishing some RGB values. The same system can be used to send or receive data from a wide range of devices, thus enabling an nrf24l01 based home automation/IoT and/or monitoring system.

The video assumes users are already somewhat familiar with the RF24 communication stack, having the radios installed and ready to go. The sketch used in the video is very closely based on the existing RF24Ethernet MQTT examples, with some extra code to control some NeoPixel based lighting.

Note: As of mosquitto 2.0 you need to add  listener 1883 and  allow_anonymous true in /etc/mosquitto/mosquitto.conf.

With this setup, users can add simple input/output toggles, sliders and charts to manage devices and display incoming sensor data. It is just a matter of configuring the Arduino devices to handle the input/output messages to and from MQTT. NodeRed can also log incoming MQTT data to a database, to store and display historical data.

This system has the benefit of requiring no apps, so everything can be controlled from a mobile device or PC etc. with some simple button clicks. Everything used is also completely open-source and free, so setup costs are minimal, and there is a host of information available for Arduino and Raspberry Pi.


Monday

AutoAnalogAudio Arduino library updated with ESP32 Support

 AutoAnalogAudio Arduino library updated with ESP32 Support

ESP32 DAC/ADC Output/Input via the I2S peripheral

Previously this year, I received some ESP32 based MCUs with OLED displays from DigitSpace, and used these devices to add ESP32 support to the AutoAnalogAudio library. It is a bit different from previous iterations, since instead of AVR interrupts and PWM, the ESP32 uses much more advanced peripherals. The I2S capabilities of the ESP32 provide a fairly seamless interaction when in/outputting audio signals, since it is just a matter of configuring the I2S, DAC and/or ADC and feeding and/or drawing data from the device. The main challenge in this case was combining the I2S functionality with RTOS tasks to provide a simple, asynchronous audio interface as is the case with the AutoAnalogAudio library.

The capabilities are much the same, but the ESP32 portion is still a bit in development, although working nicely at this point. The ESP32 example is specifically for the ESP32 device, since some minor API changes were required to manage RTOS tasks in combination with the I2S peripheral. This allows asynchronous handling of audio, so users can start playback and manage other tasks however desired while audio playback occurs.

Hardware:

1 x ESP32 OLED Wifi Kit
1 x MAX9815 Microphone Preamp
1 x TDA7297 PA Module

The MAX9815 will provide some audio to sample, and the TDA7297 was used to amplify the output of the onboard DAC. The above hardware as mentioned is provided by DigitSpace.

Connecting the Hardware:

With the ESP32, the AAAudio library samples the ADC on analog channel 4 by default, and sound output is on DAC1 and/or DAC2 (See previous post for pinout) The ADC channel cannot currently be changed, but that is on the to-do list.

Limitations:

The ESP32 onboard ADC will provide relatively good quality audio sampling, at 12-bits. The onboard DAC however, is only 8-bits, so the output will not be quite as high quality as with the Arduino Due. Using a higher sample rate can make up a for the low bit-rate of the DAC, as higher sample-rate * bit-rate = quality.

Overview:

Updating the library for ESP32 support has been fairly interesting, as it required a fair bit of learning and development time to understand I2S and how to control the related peripherals, while providing the same or similar behavior as with currently supported devices like the Due or Uno.

All in all it seems to perform pretty well. Audio input or output from virtually any source can be managed via the AAAudio library.

The updated library has been released and should be available via the Arduino Library Manager.






Friday

ESP32 - Playing around with the I2S peripheral and DAC audio output

 ESP32 Arduino - Playing around with the I2S peripheral, ADC and DAC audio output

 In some of my last posts, I mentioned the ESP32 based boards I recently received from DigitSpace, and I finally found some time to play around with some of the audio based peripherals. In this case I'm starting small, attempting to create a simple sine wave output using the internal DAC.

The documentation for the I2S peripheral is a bit limited it seems, but there is enough information in the Arduino Github repo to make sense of things and get some basic audio output going.

The general theory seems to be pretty simple compared to working with previous libraries, TMRpcm and AutoAnalogAudio, which utilize timer driven interrupts and nested interrupts (AVR devices) to handle asynchronous audio processing. The ESP32 provides a number of methods to handle audio processing, with peripherals designed to offload this work from the central processor, making the user interface mostly about buffering and handling audio data.


The above code will produce a simple sine wave output on the DAC pins (see previous ESP32 related posts for pinout) and can be toggled off/on by entering a '1' on the Arduino Serial monitor. An audio file can also be defined and played by entering a '2' in the Serial monitor. In this case the sketch is setup for a 32Khz, 16-bit, Mono audio file.



In this case the hardest part is following the documentation found above and figuring out just how simple it actually is to handle the I2S peripheral. Really, you just configure the appropriate sample rate and output pins, mode etc, and then feed data into the peripheral. If doing any digital signal processing, its really about providing enough CPU time to handle the data and feed it into I2S. If handling audio files, the hardest part is decoding.

This should prove to be a good basis for the AutoAnalogAudio library since the only thing left to do is add ADC support and either use timers or RTOS tasks to handle asynchronous processing of the audio.

Monday

A simple guide to managing memory and variables in C++

Transferring Data between Systems Using RF24
A simple guide to managing memory and variables in C++

One of the most common issues that seems to be coming up more and more recently relates to transferring data between devices and how different devices store data in memory. With more devices using 32 and 64 bit processors, integrating things between the newer devices and 8-bit MCUs can require a little bit of forethought and planning.

Memory:

What it all comes down to is memory and how different devices store data in memory. This is very important to understand when transferring data between systems and ensuring that code is portable between various devices.

When using an 8-bit device, such as an ATMega 328 based MCU (Arduino Uno, Nano, etc) the smallest unit for storage in memory is 8bits or 1-byte. This means that even a boolean (true/false) variable will take up 8-bits of memory space. On a 32 bit system, (ESP32, ESP8266) the same boolean value will use 32-bits of memory space. Why does this matter? When transferring data between systems or creating code, certain variables will take up different amounts of space in memory depending on the architecture (8-bit, 32-bit etc) thus introducing inconsistencies and misinterpretation of data.

As an example, see https://www.arduino.cc/reference/en/language/variables/data-types/int/

Using variables like int, long etc can cause confusion because they are interpreted differently on different devices. Sending a 2-byte integer from an 8-bit system to a 32-bit system works fine, as long as the size of the variable is specified on both systems such as int16_t, rather than trying to use 'int' on both systems.

Take for example the following data structure:

struct myStruct{
  bool variable1;
  uint32_t var2;
  uint8_t var3;
  int16_t var4; 
}

On an 8-bit system, this data structure will use 8 bytes of memory space, but on a 32-bit system it will take up 12-bytes. This is because the minimum unit for storage on a 32-bit system is 32-bits, so 32-bit devices will need to add padding wherever the data does not align in chunks of 4-bytes (32-bits). When using data structures, devices will typically attempt to pack the data as efficiently as possible, so re-arranging the data structure properly will prevent the 32-bit system from adding padding and making the data structures incompatible.

struct myStruct{  
  uint32_t var2;
  int16_t var4;
  uint8_t var3;  
  bool variable1; 
}

In the above example, the data is arranged in chunks of 4-bytes, starting with a 32-bit unsigned integer (4-bytes), followed by a 16-bit signed integer (2-bytes), an unsigned 8-bit integer (1-byte) and a boolean variable (1-byte), so the 32-bit system will pack the data into memory exactly the same as the 8-bit system, using only 8-bytes of memory space.

This is important, because when sending data between devices, users can just send the data structure as-is and if packed properly, the data will align regardless of whether it is a 32-bit, 64-bit or 8-bit system.

When developing code, specifying the size of all your variables and aligning data properly in your data structures will ensure that the code and communication is portable between devices.

https://www.gnu.org/software/libc/manual/html_node/Integers.html



Friday

First Little Project with ESP32 - WiFi Kit 32 OLED/BLE

Taking the First Steps - ESP32 - WiFi Kit 32 OLED/BLE
Attempting to build something useful

As per my previous post I have an opportunity to utilize these ESP32 modules in projects, so I thought I'd start with something pretty straightforward. In this case, I'll be using both ESP32 modules and a DHT22 temperature and humidity sensor, all provided by DigitSpace, to manage a few tasks:

1. Get and display the current date/time
2. ESP #1 get and display the current household temperature and humidity
3. Pass the Temp/Humidity info to ESP#2
4. ESP#1 Download current weather information and display it using the OLED
5. ESP#1 Download local daily Covid19 stats and display it using the OLED
6. ESP#2 receive the info from ESP#1 and alert via LED and OLED if temp/humidity out of range

The overall goal here is to create something useful while running the devices through some general tasks that test out the operation and programming for the things I am most interested in here, the WiFi and the OLED display.

Hardware:

2xESP32 WiFi Kit 32 w/OLED
1xDHT22 Temperature/Humidity sensor

Note: The ESP modules come with the pins separate, you will need a soldering iron to connect them.

Setting Up The IDE & Software:

1. Install the ESP32 Arduino Core per the installation instructions for Arduino Boards Manager
2. Select Heltec WiFi Kit 32 for the board
3. I chose the U8g2lib for the OLED display. It is available via Arduino Library Manager

Getting it All Up and Running

Graphics Library:

I started out with just the U8g2lib for the OLED, choosing the font group, font size etc, and figuring out basically how to use it. In this case, with the font I chose, I can fit five lines of 20 characters onto the screen, so I created 5 character arrays, and have them drawn on screen every loop. Then it is just a matter of loading the buffers with whatever information I want displayed.

For that, I created a simple function to take in a char array along with its length and the intended line number to display it on. Then all I have to do is put data in the buffers at any given time, and it gets displayed on screen. This may be over-complicating things a bit, but there is no messing about with what to refresh, what lines to draw etc, just put data in the buffers and it gets drawn.

Current Date/Time:

This part was surprisingly easy. It is a matter of specifying an NTP server or pool of servers, the GMT and DST offsets, then calling a standard time function to retrieve the information as desired.

WiFi Connectivity, HTTPS/SSL:

There are a number of examples and tutorials regarding getting connected to WiFi itself, so I'll leave that part out.

The client and SSL/HTTPS connectivity is nice to have, but seems to be a bit painful, since you apparently need to grab the certificate for the root authority of the site in question and put it in your code in order to establish a connection.

In order to grab the certificate, I found the easiest way is with a simple Linux command:
openssl s_client -showcerts -connect <hostname>:443

Beyond that, I pretty much followed the tutorial found here for setting up SSL/HTTPS connections.

It seems that once connected to a host via SSL, the ESP32 will continue to use the certificate as specified, since in order to connect to a second website for information,  I had to use SSL and specify the root_ca cert for the second site, even though I could have connected without SSL. I'm still looking into how to clear that from memory and allow normal connections etc.

The DHT22 Sensor:

This step was pretty simple also. The sensor itself connects to 3.3v, GND and any suitable GPIO pin. There is a DHT library specifically for the ESP32 available in the Arduino IDE with included examples.

The Modules in Operation:

The main module will grab information from the internet on startup, time/date, weather and covid19 stats for the local area.
The date/time and household temperature & humidity are constantly displayed on screen and updated regularly.
The bottom portion of the screen alternates between weather and covid19 information every 10 seconds.
The weather is updated every hour, and daily covid19 data refreshed every 6 hours.

The secondary module will receive the humidity and temperature sensor data from the main module as it is refreshed via http get request. If out of range a message will display on screen, and the onboard LED will flash. Users can disable the flashing LED by querying http://<IP-OF-ESP32>/s
An 's' character will stop the alerting, any other character will resume.

A video of the modules in operation is below:



The onboard LED does not show up well in the video because of the light on the camera. The OLED text was too bright and blurry on screen without the light, so it was a bit of  a compromise.

Overview/Result:

This was a nice project to work on with some opportunities to try a few new things, learn some things and mess around with some new hardware. The code itself could be cleaned up a bit and fine tuned, since I didn't pay much attention to conserving memory or program space, but there is more than enough to handle the tasks at hand.

I think I'll keep the main device in operation, and maybe look at updating the code to handle additional tasks & processing. I think the secondary device could easily be an AVR based device with an audible alerting system, using RF24 for data transfer. It is nice to have the ESP32, but it seems a bit overkill for the situation.

This was a good way to test and get used to having the OLED display and WiFi capabilities on hand to do with whatever is needed. All in all I'm really liking these devices. More to come with ESP32 projects and development!

Code/Sketches:

https://github.com/TMRh20/Sketches/tree/master/ESP32_WiFiKit32



Saturday

Upcoming Projects: ESP32 - WiFi Kit 32 OLED/BLE

Playing around with ESP32 - WiFi Kit 32 OLED/BLE
New Toys to Play With!

I was recently provided some nice new toys to play with by the people at DigitSpace and will be documenting some upcoming projects involving these devices. The main player will be an ESP32 with built in OLED, which is very nice to have, not just because of the ESP32 capabilities and processing power, but the built-in OLED provides a nice way to get output and information without having to watch a serial monitor or send data to another device etc.

ESP32 WiFi Kit32 OLED from www.digitspace.com

I initially had some problems connecting the devices directly to computers via USB cable. I used different cables that work fine with my ESP8266 modules, but these would not work with any Windows, Linux or Mac devices I tried. Response, communication and professionalism was great, but neither I nor the supplier could seem to find a solution, even after contacting the manufacturer.

By chance, I ordered another module of the same model from a different supplier, and it came with a CH340G based USB->TTL module! It is starting to look like this is a known issue. So I tried it with these devices and BAM! Working nicely. Upload speed needed to be 256000 or lower.

The 5v pin on my TTL modules is connected directly to the USB 5v, so it can power the ESP32 via the 5v pin, even when configured to operate at 3.3v. If the RTS and DTR pins are not exposed, you only need to connect TX & RX, then press RST then PGM, then release RST then PGM buttons to put it into upload mode. RST again after complete. If exposed, connect DTR to GPIO0 and RTS to RST per the pinout below, and uploading will be automatic.

ESP32 WiFi Kit32 OLED Pinout

Now with the capabilities here, including a 240Mhz processor, BLE, WiFi and built in OLED, you might not think this module would be cheaper than an AVR based Arduino, but in some cases it is! Per the DigitSpace store these modules offer a great platform at a great price. Beyond the USB->TTL requirement, installation is easy. The Arduino IDE includes this device in the default ESP32 boards, and there are many examples etc. to test with.

I now have some working ESP32 devices and some other hardware that was given to use in upcoming projects! Some will relate to existing libraries and projects, but as usual I'm not sure exactly how or if I am going to be able to pull it off. Its always a challenge to understand the internal workings of newer MCUs and related code to produce the results I am expecting. It will be a challenge, but it seemed like a good idea at the time, so why not?

In any case, stay tuned for more to come with development & projects on ESP32 modules!


The RF24 Communication Stack: What are the different layers and which one do I use?

 The RF24 Communication Stack: What are the different layers and which one do I use? Over the past number of years, I've been developing...