Saturday

Automation/IoT with NRF24L01+ and MQTT

Automation/IoT with Arduino, RPi, NRF24L01+ and MQTT
Control and monitor Arduino remotely with mobile devices


Overview:

Please see Arduino Project Hub for an up to date version of this tutorial
Preconfigured kits available at https://tmrautomation.myshopify.com/

This is a quick tutorial on how to wirelessly control or monitor Arduino devices using nothing more than NRF24L01+ radio modules, a Raspberry Pi and a mobile device. I have found MQTT to be one of the best methods of controlling these devices wirelessly, in the case of simple input/output scenarios like monitoring sensor values or sending RGB values to wireless lighting systems.

This assumes users are familiar with the core RF24 library as well as installing libraries from the Arduino IDE, and building and running programs on the RPi.

Requirements:

Raspberry Pi with NRF24l01+ device attached
One or more Arduino devices with NRF24L01+ device attached
The RPi should be connected to the same network as your mobile device

Pre-configuration:

Ensure all devices are functional with RF24 core examples before attempting. 

  Install RF24 libraries on RPi from your home directory:
  2. chmod +x install.sh
  3. ./install.sh
  4. Follow the command prompts and install the RF24, RF24Network, RF24Mesh, and RF24Gateway libraries

  Install RF24 libraries on Arduino:
  1. From Arduino IDE select  Sketch > Include Libraries > Library Manager
  2. Install the RF24, RF24Network, RF24Mesh and RF24Ethernet libraries

RPi Setup:




Now that the correct libraries are installed, we need to install an MQTT broker on the Raspberry Pi. Setup is as easy as running the following command:

sudo apt-get install mosquitto

Note: As of mosquitto 2.0 you need to add  listener 1883 and  allow_anonymous true in mosquitto.conf.

Then we need to start the Gateway on the RPi, so it will pass traffic from the connected sensor nodes etc, to the MQTT server.

The RF24Gateway library is installed in ~/rf24libs/RF24Gateway/

In this example, we will use the ncurses example, located at ~/rf24libs/RF24Gateway/examples/ncurses

Edit the file RF24Gateway_ncurses.cpp and modify the radio constructor to suit your pin connections: RF24 radio(22,0); is the default, then type 'make -B' to build the example.

To run the example, type ' sudo ./RF24Gateway_ncurses ', and it should pop up asking for an IP and subnet mask to use. This is mainly arbitrary, and any suitable private IP/Mask can be selected.

Once the gateway is running, we can switch our attention to the Arduino.

Arduino Setup:

1. Open the MQTT example: File > Examples > RF24Ethernet > MQTT > mqtt_basic
2. Edit the radio constructor to suit your chosen CE/CS pins: RF24 radio(7,8); is default.
3. Edit the IP address of the device to match the range chosen when configuring the RPi.
4. Edit the IP of the gateway to match exactly the IP of the RPi
5. Edit the IP of the server to match exactly the IP of the RPI 
6. Upload the sketch to the Arduino.

Note: If using an external MQTT server, the RPI must be configured to forward packets and perform NAT, see the forwarding and routing section here.

You should see serial output like the following:
Attempting MQTT connection...connected

If not, something has gone wrong. Ensure your device is connecting to the mesh and showing up in the address list. If not, troubleshoot using examples from RF24 or RF24Mesh to diagnose the connectivity issues.

Nodes running the MQTT example will publish their NodeID every second to the MQTT server, at outTopic and will receive all messages published to the inTopic.

Setup on iPhone etc:

On my iPhone, I chose to use the program MQTTTerminal (MQTTool is also good with no ads). Setup is easy, just input the external IP address of the MQTT server (the ip of your RPi) and the port (1883). Then set the Publish Topic to inTopic and the Subscribe Topic to outTopic.

You should see messages incoming from the device(s) and can send data to the devices from the app.

From here it is a simple matter of customizing the topics and messages to suit your needs, whether controlling a simple LED on the Arduino, or reporting temperature and humidity from sensors.

This can be expanded using programs like Node Red to create an open-source home automation system. See my more recent blog post.

If problems are encountered etc, feel free to open an issue at https://github.com/nRF24/RF24Ethernet/issues 



19 comments:

theDiver said...

Hi there

Trying to follow your guide.

But having a few questions.

Could you post how exactly to connect the NRF24l01 to the Rapberry pi?

I have tried this: https://forum.mysensors.org/topic/2437/step-by-step-procedure-to-connect-the-nrf24l01-to-the-gpio-pins-and-use-the-raspberry-as-a-serial-gateway-mysensors-1-x

But when i start the nCurse program, I am not sure what to enter as IP and Netmask, but no matter what i enter, the program hangs and then the Raspberry crashes, kicks me of SSH, and i need to turn of the Raspberry to reboot it.

Thanks a lot :)

TMRh20 said...

The connections shown in the mysensors setup guide is correct, you just need to define the correct CE/CSN pins in the ncurses program. Even if they are incorrect, it will not be the cause of the hang etc.

How about if you run the following commands before starting the ncurses program?:

sudo ip tuntap add dev tun_nrf24 mode tun user pi multi_queue

sudo ifconfig tun_nrf24 10.10.3.1/24

theDiver said...

Hi TMRh20

The 2 last commandoes seems to help the issue, now i can start the ncurses tool, and it looks like on your screen shot,

Also updated my Raspberry from Raspbian V8 to V9.1 :-)

Also added a 5V->3.3V converter to get enough power for my NRF24l01+ with antenne

Thanks a lot for your time and help.

smerrett said...

Hi TMRh20,

Thanks so much for ALL your work over the years. Your blog and repository have become the first resources I turn to each time I attempt something new with an NRF24L01-based system. Currently I'm trying to make a low powered remote control (AVR processor) send messages to a Phillips Hue bulb over zigbee2mqtt (which uses Mosquitto) running on a raspberry pi. I can get communication between the Pi and the AVR using gettingstarted.ino and the Pi equivalent. However, I've come unstuck with adapting the mqtt example above to work with zigbee2mqtt. zigbee2mqtt runs using `mqtt://localhost` and your demonstration uses specific IP addresses. Please could you give me some hints about what to try next?

smerrett said...

Update: I got it working with the tips you gave theDiver above AND making sure I changed RF24Network_config.h to allow large packets and recompile, as per your linked video. I don't know how the "arbitrary" IP address of 10.10.3.1 resolves to the localhost MQTT port but I'll take that as a win for now and not worry that I can't explain it! For anyone else interested, the setup in this blog post allows you to do something similar to me in home automation - using a low power Arduino-based microcontroller to talk directly to e.g. Philips Hue equipment without needing bluetooth or WiFi (as many of the ESP8266/32 solutions use). You need zigbee2mqtt running on the Pi and you need a CC2531 USB dongle too, to complete your zigbee coordinator hardware.

Angel said...

Hello there, I am using two nano-combo for a RC car and when the remote emitter battery is too down if the car was running and battery finally empty the car was still running...is there any way by programming to check communication between the two nanos and solve this problem??. Thanks.

smerrett said...

@Angel, I think you are asking about a topic which isn't closely related to the content of this particular blog post. I suggest you'll have more success finding a solution if you post on the Arduino forums.

Angel said...

ok, thanks.

Old Surfer Dude said...

I was looking for, but didn't find, a c++ client for the RPi like the mqtt_basic for the Arduino. Did I miss that?

Yes, I'm one of those programmers that stands on the shoulders of the giants that came before me.

TMRh20 said...

@old surfer dude I am aware of python clients for MQTT but not for c++. I assume there are c++ libraries available but I’m not aware of any myself.

Old Surfer Dude said...

@TMRh20 Thanks! I can make Python work, just didn't want to do that.

My project is wireless sensors-in-the-yard powered by microSolar+microStorage (sensor unit [5W solar panel, 18650 rechargeable battery, TP4056 Battery Charging Board, Nano, nRF24] and RPi/nRF24 as GUI and data cruncher)

I experimented with an ESP8266 and found it very robust, but I think I want to steer clear of WiFi for sensors.

I'll probably have some questions about your gateway.

Thanks for your help in confirming I'd be hard pressed to find a C++ MQTT client for RPi

OSD

Old Surfer Dude said...

Thanks for letting me know to put the radioi.setPALevel() right after the gw.begin();

Now that my power is higher, I'm seeing that I'm getting nodes on my other mesh network connecting to the one I'm setting up for MQTT. I'll need to set the channel to something different. I'm assuming that I would also put the radio.setChannel() in the same place. Is this a good assumption?

Is there a default variable with the channel? Would it be better to change the value of that?

Thanks for your help!

Old Surfer Dude said...

On a different topic, this project is a wireless arduino sensor node powered by solar+storage (5W solar panel and 18650 rechargeable battery)

I will collect data every 15 minutes. In order to keep the battery usage to a minimum I 1) release the mesh address, power down the radio, then go into a sleep mode. Upon wake, I power up the radio (wait 100mSec) and then do a mesh.begin();

after that I connect to the MQTT broker on the RPi. If that fails I try 10 more times (usually connect first or second time). After these 10 attempts, I loop until mesh.begin() (if unsuccessful, "power cycle" the radio [100mSec, power down, 100mSec power up, 100mSec]) is successful

More often than not, at each level, the first or second retry is successful. I have yet to see getting stuck in the mesh.begin()/power-cycle-radio loop, though this would happen if the gateway were down.

Here's the problem, though, and I'm asking if you think this could be a problem. If I have a remote node to far away to connect directly to the broker (connects through other nodes) it would have to wait for an intermediate node to be connected. It would be stuck in the aforementioned loop until the intermediate node woke up. The situation would be compounded if this remote node required two hops.

Points to ponder. I'm considering permanently powered intermediary nodes.

Thanks for all the good quality work you do.

OSD

TMRh20 said...

@Old Surfer Dude The channel needs to be set in gw.begin();

ie: uint8_t nodeID = 0;
uint8_t channel = 5;
gw.begin(nodeID, channel);

Old Surfer Dude said...

My questions are "Where is the IP address of the Gateway stored?" "What other information is stored in this manner?" I would like to start the Gateway on boot; how do I automatically set the IP address and mask?

I found that the children of absent parents tend to get permanently lost.

For example,
o a node with ID 41 connects to the mesh master and receives the address 3.
o Another node with ID 62 wants to connect and finds this node41 and becomes a child at address 43
o this makes the node41 at address 3 a parent and the node62 at address 43 a child.
o The parent node41 subsequently releases its address and goes to sleep
o The child node62 now cannot connect to the master.
o The master still has node62 at address 43 in its address list.
o When the child node62 goes through the process of trying to connect (try to renew, then continuously try to "mesh.begin();") it gets stuck. I suspect that the master is giving the old address of 43, but there is no node with address 3 so it's stuck until a node with address 3 appears.

Therefore I add the line after the mesh.begin(); "mesh.setChild(false);" to prevent any other node from passing through it.

In this manner, a network of 4 transient nodes works well.

To expand the network I created a permanently powered node with the NodeID of 5. In the master code (RF24Gateway_ncurses.cpp) I add the line "mesh.setStaticAddress(5,5);" after the gw.begin(); line. This node connects to the MQTT broker but never subscribes nor publishes, but it does do periodic ping requests to the broker. (running mosquitto -v on the RPi allows me to see these ping requests.)

This works in the perfect case: the RPi boots and starts mosquitto followed by RF24Gateway_ncurses. Then the static node is turned on and it connects to the mesh, then broker. The other sensorNodes, as I call these transient clients, can now come and go as they please: read sensors, power up radio (startListening), connect to the mesh network, connect to the MQTT broker, publish data, check for incoming subscriptions, unsubscribe, disconnect from MQTT broker, release mesh address, power down the radio, sleep CPU.

But things are not perfect. If the static node fails, all of the nodes will have to wait for a spot in the master, which occurs if a node successfully releases its address. Worse yet, the statuc node does not reconnect to the master.

There seems to be some lingering data in the mesh master. I note that if the master node gateway program is stopped, and restarted, the initial IP address is still present. Deleting the dhcp.txt will clear the address list. but not the IP address of the gateway. While this is not important, what is important is why isn't my static node able to reconnect to the mesh?

Thus my questions. Thanks for your help.

OSD

TMRh20 said...

@Old Surfer Dude
The IP address of the Gateway is not really "stored" anywhere. The network interface is configured just like any other interface with an IP and subnet mask. As long as the tun_nrf24 interface exists, the configured IP/Mask will exist. eg: sudo ip link delete tun_nrf24 will remove the interface and IP/Mask.

I use a startup script to start the interface containing the following:
ip tuntap add dev tun_nrf24 mode tun user pi multi_queue
ifconfig tun_nrf24 10.1.3.33/16

And another script to start the gateway in screen:
su - pi -c "screen -S RF24Gw_on_RPi -dm sudo /home/pi/rf24libs/RF24Gateway/examples/ncurses/./RF24Gateway_ncurses&"

Your static node should be running just RF24Network, not RF24Mesh. If everything is working hardware wise etc. it should just keep functioning. I would suggest maybe setting up a second static node both as a test to see if there is an issue with the original static node and to have some redundancy.

I haven't seen the behavior you describe, with a node node being able to renew its address after its parent node goes offline. I would suggest communication or hardware issues in that case. Best to use github.com/nrf24 to post inquiries like this if you are having potential issues with the library though.

gstruwig said...

Hello and Happy New Year!
Please help, I have followed this tutorial but the mqtt_basic sketch fails at mesh.begin().
I am using the ncurses example but notice that the Mesh Info is hardcoded to Address: 01 Id: 8
I have tried commenting out the line mesh.setStaticAddress(8, 1); in RF24Gateway_ncurses.cpp and make -B again but this problem remains.

As I understand from other examples of yours, the ip I have set on the Arduino should show up here.

Thanks
Strugp

TMRh20 said...

@gstruwig If it is failing at mesh.begin(); that usually indicates a problem with the hardware configuration, wiring or power supply etc. You can test with the core RF24 library examples to troubleshoot your radios and make sure they are in working order.
The mesh.setStaticAddress() line can be removed, that was mostly left in there for testing purposes. The mesh node should pick up its own address if the radios are working properly. If you have removed that line from RF24Gateway_ncurses.cpp, then just delete the dhcplist.txt file (saved nodes list) and it should go away.

gstruwig said...

@TMRh20 Thank you, it is working now.

Incorrect CE pin assignment on the Pi. For those new at this like me pin22 is GPIO25 so should be assigned RF24 radio(25, 0);

XIAO BLE SENSE: nrf52840 radio communication with nrf24L01+

XIAO BLE SENSE: nrf52840 radio communication with nrf24L01+ Establishing a communication layer using the RF24 API  I recently received an XI...