AutoAnalogAudio Arduino library updated with ESP32 Support
ESP32 DAC/ADC Output/Input via the I2S peripheral
Previously this year, I received some ESP32 based MCUs with OLED displays from DigitSpace, and used these devices to add ESP32 support to the AutoAnalogAudio library. It is a bit different from previous iterations, since instead of AVR interrupts and PWM, the ESP32 uses much more advanced peripherals. The I2S capabilities of the ESP32 provide a fairly seamless interaction when in/outputting audio signals, since it is just a matter of configuring the I2S, DAC and/or ADC and feeding and/or drawing data from the device. The main challenge in this case was combining the I2S functionality with RTOS tasks to provide a simple, asynchronous audio interface as is the case with the AutoAnalogAudio library.
The capabilities are much the same, but the ESP32 portion is still a bit in development, although working nicely at this point. The ESP32 example is specifically for the ESP32 device, since some minor API changes were required to manage RTOS tasks in combination with the I2S peripheral. This allows asynchronous handling of audio, so users can start playback and manage other tasks however desired while audio playback occurs.
Hardware:
1 x ESP32 OLED Wifi Kit 1 x MAX9815 Microphone Preamp 1 x TDA7297 PA Module
The MAX9815 will provide some audio to sample, and the TDA7297 was used to amplify the output of the onboard DAC. The above hardware as mentioned is provided by DigitSpace.
Connecting the Hardware:
With the ESP32, the AAAudio library samples the ADC on analog channel 4 by default, and sound output is on DAC1 and/or DAC2 (See previous post for pinout) The ADC channel cannot currently be changed, but that is on the to-do list.
Limitations:
The ESP32 onboard ADC will provide relatively good quality audio sampling, at 12-bits. The onboard DAC however, is only 8-bits, so the output will not be quite as high quality as with the Arduino Due. Using a higher sample rate can make up a for the low bit-rate of the DAC, as higher sample-rate * bit-rate = quality.
Overview:
Updating the library for ESP32 support has been fairly interesting, as it required a fair bit of learning and development time to understand I2S and how to control the related peripherals, while providing the same or similar behavior as with currently supported devices like the Due or Uno.
All in all it seems to perform pretty well. Audio input or output from virtually any source can be managed via the AAAudio library.
The updated library has been released and should be available via the Arduino Library Manager.
ESP32 Arduino - Playing around with the I2S peripheral, ADC and DAC audio output
In some of my last posts, I mentioned the ESP32 based boards I recently received from DigitSpace, and I finally found some time to play around with some of the audio based peripherals. In this case I'm starting small, attempting to create a simple sine wave output using the internal DAC.
The documentation for the I2S peripheral is a bit limited it seems, but there is enough information in the Arduino Github repo to make sense of things and get some basic audio output going.
The general theory seems to be pretty simple compared to working with previous libraries, TMRpcm and AutoAnalogAudio, which utilize timer driven interrupts and nested interrupts (AVR devices) to handle asynchronous audio processing. The ESP32 provides a number of methods to handle audio processing, with peripherals designed to offload this work from the central processor, making the user interface mostly about buffering and handling audio data.
The above code will produce a simple sine wave output on the DAC pins (see previous ESP32 related posts for pinout) and can be toggled off/on by entering a '1' on the Arduino Serial monitor. An audio file can also be defined and played by entering a '2' in the Serial monitor. In this case the sketch is setup for a 32Khz, 16-bit, Mono audio file.
In this case the hardest part is following the documentation found above and figuring out just how simple it actually is to handle the I2S peripheral. Really, you just configure the appropriate sample rate and output pins, mode etc, and then feed data into the peripheral. If doing any digital signal processing, its really about providing enough CPU time to handle the data and feed it into I2S. If handling audio files, the hardest part is decoding.
This should prove to be a good basis for the AutoAnalogAudio library since the only thing left to do is add ADC support and either use timers or RTOS tasks to handle asynchronous processing of the audio.
A simple guide to managing memory and variables in C++
One of the most common issues that seems to be coming up more and more recently relates to transferring data between devices and how different devices store data in memory. With more devices using 32 and 64 bit processors, integrating things between the newer devices and 8-bit MCUs can require a little bit of forethought and planning.
Memory:
What it all comes down to is memory and how different devices store data in memory. This is very important to understand when transferring data between systems and ensuring that code is portable between various devices.
When using an 8-bit device, such as an ATMega 328 based MCU (Arduino Uno, Nano, etc) the smallest unit for storage in memory is 8bits or 1-byte. This means that even a boolean (true/false) variable will take up 8-bits of memory space. On a 32 bit system, (ESP32, ESP8266) the same boolean value will use 32-bits of memory space. Why does this matter? When transferring data between systems or creating code, certain variables will take up different amounts of space in memory depending on the architecture (8-bit, 32-bit etc) thus introducing inconsistencies and misinterpretation of data.
Examples:
char is equivalent to int8_t byte is equivalent to uint8_t int is equivalent to int16_t on an Arduino Uno (8-bit device) and int32_t on ESP32 (32-bit device) unsigned int is equivalent to uint16_t on Arduino Uno (8-bit device) and uint32_t on ESP32 (32-bit device)
Using variables like int, long etc can cause confusion because they are interpreted differently on different devices. Sending a 2-byte integer from an 8-bit system to a 32-bit system works fine, as long as the size of the variable is specified on both systems such as int16_t, rather than trying to use 'int' on both systems.
Take for example the following data structure:
struct myStruct{
bool variable1;
uint32_t var2;
uint8_t var3;
int16_t var4;
}
On an 8-bit system, this data structure will use 8 bytes of memory space, but on a 32-bit system it will take up 12-bytes. This is because the minimum unit for storage on a 32-bit system is 32-bits, so 32-bit devices will need to add padding wherever the data does not align in chunks of 4-bytes (32-bits). When using data structures, devices will typically attempt to pack the data as efficiently as possible, so re-arranging the data structure properly will prevent the 32-bit system from adding padding and making the data structures incompatible.
struct myStruct{
uint32_t var2;
int16_t var4;
uint8_t var3;
bool variable1;
}
In the above example, the data is arranged in chunks of 4-bytes, starting with a 32-bit unsigned integer (4-bytes), followed by a 16-bit signed integer (2-bytes), an unsigned 8-bit integer (1-byte) and a boolean variable (1-byte), so the 32-bit system will pack the data into memory exactly the same as the 8-bit system, using only 8-bytes of memory space.
This is important, because when sending data between devices, users can just send the data structure as-is and if packed properly, the data will align regardless of whether it is a 32-bit, 64-bit or 8-bit system.
When developing code, specifying the size of all your variables and aligning data properly in your data structures will ensure that the code and communication is portable between devices.
Taking the First Steps - ESP32 - WiFi Kit 32 OLED/BLE
Attempting to build something useful
As per my previous post I have an opportunity to utilize these ESP32 modules in projects, so I thought I'd start with something pretty straightforward. In this case, I'll be using both ESP32 modules and a DHT22 temperature and humidity sensor, all provided by DigitSpace, to manage a few tasks:
1. Get and display the current date/time
2. ESP #1 get and display the current household temperature and humidity
3. Pass the Temp/Humidity info to ESP#2
4. ESP#1 Download current weather information and display it using the OLED
5. ESP#1 Download local daily Covid19 stats and display it using the OLED
6. ESP#2 receive the info from ESP#1 and alert via LED and OLED if temp/humidity out of range
The overall goal here is to create something useful while running the devices through some general tasks that test out the operation and programming for the things I am most interested in here, the WiFi and the OLED display.
Note: The ESP modules come with the pins separate, you will need a soldering iron to connect them.
Setting Up The IDE & Software:
1. Install the ESP32 Arduino Core per the installation instructions for Arduino Boards Manager
2. Select Heltec WiFi Kit 32 for the board
3. I chose the U8g2lib for the OLED display. It is available via Arduino Library Manager
Getting it All Up and Running
Graphics Library:
I started out with just the U8g2lib for the OLED, choosing the font group, font size etc, and figuring out basically how to use it. In this case, with the font I chose, I can fit five lines of 20 characters onto the screen, so I created 5 character arrays, and have them drawn on screen every loop. Then it is just a matter of loading the buffers with whatever information I want displayed.
For that, I created a simple function to take in a char array along with its length and the intended line number to display it on. Then all I have to do is put data in the buffers at any given time, and it gets displayed on screen. This may be over-complicating things a bit, but there is no messing about with what to refresh, what lines to draw etc, just put data in the buffers and it gets drawn.
Current Date/Time:
This part was surprisingly easy. It is a matter of specifying an NTP server or pool of servers, the GMT and DST offsets, then calling a standard time function to retrieve the information as desired.
WiFi Connectivity, HTTPS/SSL:
There are a number of examples and tutorials regarding getting connected to WiFi itself, so I'll leave that part out.
The client and SSL/HTTPS connectivity is nice to have, but seems to be a bit painful, since you apparently need to grab the certificate for the root authority of the site in question and put it in your code in order to establish a connection.
In order to grab the certificate, I found the easiest way is with a simple Linux command: openssl s_client -showcerts -connect <hostname>:443
Beyond that, I pretty much followed the tutorial found here for setting up SSL/HTTPS connections.
It seems that once connected to a host via SSL, the ESP32 will continue to use the certificate as specified, since in order to connect to a second website for information, I had to use SSL and specify the root_ca cert for the second site, even though I could have connected without SSL. I'm still looking into how to clear that from memory and allow normal connections etc.
The DHT22 Sensor:
This step was pretty simple also. The sensor itself connects to 3.3v, GND and any suitable GPIO pin. There is a DHT library specifically for the ESP32 available in the Arduino IDE with included examples.
The Modules in Operation:
The main module will grab information from the internet on startup, time/date, weather and covid19 stats for the local area.
The date/time and household temperature & humidity are constantly displayed on screen and updated regularly.
The bottom portion of the screen alternates between weather and covid19 information every 10 seconds.
The weather is updated every hour, and daily covid19 data refreshed every 6 hours.
The secondary module will receive the humidity and temperature sensor data from the main module as it is refreshed via http get request. If out of range a message will display on screen, and the onboard LED will flash. Users can disable the flashing LED by querying http://<IP-OF-ESP32>/s
An 's' character will stop the alerting, any other character will resume.
A video of the modules in operation is below:
The onboard LED does not show up well in the video because of the light on the camera. The OLED text was too bright and blurry on screen without the light, so it was a bit of a compromise.
Overview/Result:
This was a nice project to work on with some opportunities to try a few new things, learn some things and mess around with some new hardware. The code itself could be cleaned up a bit and fine tuned, since I didn't pay much attention to conserving memory or program space, but there is more than enough to handle the tasks at hand.
I think I'll keep the main device in operation, and maybe look at updating the code to handle additional tasks & processing. I think the secondary device could easily be an AVR based device with an audible alerting system, using RF24 for data transfer. It is nice to have the ESP32, but it seems a bit overkill for the situation.
This was a good way to test and get used to having the OLED display and WiFi capabilities on hand to do with whatever is needed. All in all I'm really liking these devices. More to come with ESP32 projects and development!
I was recently provided some nice new toys to play with by the people at DigitSpace and will be documenting some upcoming projects involving these devices. The main player will be an ESP32 with built in OLED, which is very nice to have, not just because of the ESP32 capabilities and processing power, but the built-in OLED provides a nice way to get output and information without having to watch a serial monitor or send data to another device etc.
ESP32 WiFi Kit32 OLED from www.digitspace.com
I initially had some problems connecting the devices directly to computers via USB cable. I used different cables that work fine with my ESP8266 modules, but these would not work with any Windows, Linux or Mac devices I tried. Response, communication and professionalism was great, but neither I nor the supplier could seem to find a solution, even after contacting the manufacturer.
By chance, I ordered another module of the same model from a different supplier, and it came with a CH340G based USB->TTL module! It is starting to look like this is a known issue. So I tried it with these devices and BAM! Working nicely. Upload speed needed to be 256000 or lower.
The 5v pin on my TTL modules is connected directly to the USB 5v, so it can power the ESP32 via the 5v pin, even when configured to operate at 3.3v. If the RTS and DTR pins are not exposed, you only need to connect TX & RX, then press RST then PGM, then release RST then PGM buttons to put it into upload mode. RST again after complete. If exposed, connect DTR to GPIO0 and RTS to RST per the pinout below, and uploading will be automatic.
ESP32 WiFi Kit32 OLED Pinout
Now with the capabilities here, including a 240Mhz processor, BLE, WiFi and built in OLED, you might not think this module would be cheaper than an AVR based Arduino, but in some cases it is! Per the DigitSpace store these modules offer a great platform at a great price. Beyond the USB->TTL requirement, installation is easy. The Arduino IDE includes this device in the default ESP32 boards, and there are many examples etc. to test with.
I now have some working ESP32 devices and some other hardware that was given to use in upcoming projects! Some will relate to existing libraries and projects, but as usual I'm not sure exactly how or if I am going to be able to pull it off. Its always a challenge to understand the internal workings of newer MCUs and related code to produce the results I am expecting. It will be a challenge, but it seemed like a good idea at the time, so why not?
In any case, stay tuned for more to come with development & projects on ESP32 modules!
Sunday
nRF24l01+ Library Roundup
Overview and Status of RF24 Arduino/Linux Comm. Stack
Six, seven years? Has it really been that long? According to GitHub that is in fact the case, as my first commits to my own fork of the RF24 library took place in early 2014. It all started by identifying existing limitations in the available libraries and working to achieve the highest level of performance and reliability possible. The main fork for RF24 at the time was written and maintained by ManiacBug, but he dropped off the scene shortly after publishing this very nicely designed library. Other users attempted to add support for various devices and address certain bugs, but nobody had really taken a thorough look at the capabilities of the hardware vs what was achievable at the time. Only through a lot of reading, testing and learning to program was I able to make those initial changes to begin work improving the RF24 core library. Looking back now, I am very grateful that I picked ManiacBugs code base as a place to get back into programming in C/C++, as it was well thought out and I discovered ways of doing things that I previously did not understand or know about.
And with that began a long and arduous journey into the internal workings of nrf24l01+, Arduino (AVR) and RPi (Linux) devices with the stated goal of optimizing the library to whatever extent possible. Once the RF24 library began to take shape, I began looking at the RF24Network library, another very nicely designed bit of code by ManiacBug, but it had its limitations and problems. Many issues were addressed, features and functionality added in order to push the nrf24 devices to the test in a multi-device scenario. It performed much better than expected using this OSI Layer 3 (network) library.
At that point, I began to really understand the OSI model, how the different layers actually work together and what they do down the last bit. Based on a number of conversations and user input, I came up with the idea for RF24Mesh, which is basically another layer on top of RF24Network -> RF24 that handles addressing, similar to DHCP, but able to verify connectivity and reconnect nodes at any point in the mesh. This allows nodes to move around and reconnect quickly as required while maintaining the mesh structure. The Network & Mesh layers use a 'master' node that acts like the gateway in a standard IP network, and it provides addressing and resolution for RF24Mesh.
Based on input and discussions with users and the performance of the radio modules when used in a multi-device mesh/network scenario along with the emergence of IoT and sensor networks, it seemed like the thing to do would be to work my way up the OSI model, so that is exactly what I did. Upon discovering the uIP stack for Arduino, I realized it would be possible to add ip4 support to some of the smallest devices like ATmega328 AVRs, and larger devices like RPi could just use their own IP stack and encryption etc. The uIP stack and related software was quite a challenge to understand and implement, as uIP is written in C and is designed to be as small and lightweight as possible as opposed to being human readable and easy to understand. The UIPEthernet library was also essential in helping me to understand and implement RF24Ethernet.
With that, an OSI RF24 comm. stack was established, with RF24, RF24Network, RF24Mesh, RF24Ethernet and RF24Gateway libraries all working together and inter-operational at any of the layers. This means that on a RF24Gateway/RF24Ethernet mesh network incorporating RPi and Arduino nodes, devices can choose to operate using OSI layer 2,3,4,5 and/or 7 (Data Link, Network, Transport, Session, Application). When using the higher layer libraries (RF24Gateway & RF24Ethernet) standard IP connectivity is established and devices can utilize standard encryption & authentication protocols (SSH, HTTPS etc.) to secure their traffic as per the OSI model.
The RF24 Communication Stack vs The OSI Model
With the associated libraries and enhanced functionality, the RF24 comm. stack is able to provide IoT and sensor network connectivity and capabilities to suit many different scenarios at a very low price point with much lower power consumption than WiFi networks. With the production of many devices like RPi, ESP8266 and ESP32 which support WiFi, users can construct and deploy IoT networks very easily, extending them as required.
Even now, it is really kind of cool to witness the system in action, using these little radios on tiny little computers we refer to as 'Arduinos'. The methodology and processes in place to manage a network of devices, fragmentation & reassembly, mitigate wireless data collisions and ensure delivery of data are quite effective, from the radio hardware and auto-ack functionality, all the way up to the TCP/IP and Mesh levels. With TCP/IP for example, using Raspberry Pi devices, the MTU is 1500-bytes, so each packet can require up to 63 sequential, successful data transmissions at the core RF24 layer, but this still works relatively well in real-world testing/usage.
It has been a lot of fun and a great learning experience so far. The creation of RF24Gateway really allowed the limits of the radios to be tested, and provided an excuse to play around with different network/IP configurations, routing scenarios and traffic handling at any level of the OSI model. A full and complete understanding of how devices operate on the internet or similar networks helps dramatically in troubleshooting, testing and development of systems and software that operate using these protocols.
Going forward, the development of the RF24 libraries has slowed significantly along with bug reports and issues. My focus has recently been on cleaning things up, finding and fixing bugs, mostly in the higher layers, and improving the user interface via examples and documentation. I had previously almost given up on the RF24Gateway and Ethernet layers as being too difficult to fix, but with the rise of more powerful MCUs and the potential for expanding their capabilities, it seems worthwhile to take another crack at working out bugs and issues.
It would be nice to find another similar radio device with lower power consumption and/or more advanced capabilities and throughput to create more robust networks and mesh capabilities. The current design allows for speeds up to about 20KB/s over IP (RF24Gateway) using two Linux devices, but it would be nice to have a higher throughput to allow for more nodes, longer range and more advanced communication scenarios.
In conclusion I want to extend a big thank you to everybody who contributed ideas, analysis, information and programming skills along the way. An extra thank you to Avamander (GitHub) who has played a big part in ongoing maintenance, support and development!
Update: Aug 2020
Wow, I mentioned that I would be working on finding and fixing bugs due to the slowdown involving support, dev and maintenance etc, but did not expect to spend so much time, or that I would be able to identify so many issues, their root cause and a solution. Some pretty significant issues have been fixed throughout the RF24 stack, including 1 major bug in RF24, affecting all libraries, a number of bugs/issues affecting functionality and reliability in RF24Network and RF24Mesh, and a memory issue in RF24Ethernet affecting stability.
RF24Gateway has been updated with better handling of interrupts, a few corrections and the first release made, v1.0 due to all the issues that have been addressed. Again, a portion of these improvements are due to assistance, testing and input from a select few users. Users should notice a marked improvement in stability, functionality and the ability to recover from significant errors/hardware issues.
