Showing posts with label mesh network. Show all posts
Showing posts with label mesh network. Show all posts

Wednesday

RF24Mesh - Dynamic Mesh Networking for NRF24L01+ Radio Modules
An Overview of Progress so Far


Overview:

RF24Mesh is a C++/Arduino/Linux/Raspberry Pi library allowing many devices to simultaneously connect wirelessly and remain connected in a mesh style network using nrf24l01+ radio modules. Nodes can communicate with any other node on the network, including a master node that acts as a gateway for the mesh, providing addressing and lookups.

Once assigned a unique ID, everything is automated, and nodes can move around and rejoin the mesh at any point where another node is in range. RF24Mesh supports some of the smallest and most power efficient devices available.

The RF24 Communication Stack is based on the OSI model, and RF24Mesh makes use of the features of the lower layers (RF24 (Layer 2) , RF24Network (Layer 3), and the overall capabilities can be extended via RF24Ethernet (Layers 4&5 TCP/IP)

Setting it Up:

The main thing when setting up a mesh network is to configure and test your devices accordingly, to make sure they are in a fully operational state. Most if not all of the available radio chips being sold are clones, so I've found it necessary to modify the radio devices slightly in some cases. In many cases, a capacitor (10-100uF) is added directly to the VCC/GND of the module itself to help with common power supply issues.

In the case of these modules, I've found that scraping down the last bit of the antenna, and soldering on a small 3-4" piece of wire greatly improves their performance:
In the case of the high powered PA + LNA modules, I've found it necessary to add shielding as in the post found here. Placing them close to a ground plane seems to help as well. Operating them at a lower power level may be required if the power supply is not adequate.


The modules that are low powered with an antenna generally work out of the box.

I generally recommend using an adapter, to provide a stable power source, even for the low powered modules. I've been using them with many of my modules.

From there it is a matter of installing the Arduino libraries etc. which is covered in the documentation and in earlier blog posts and the official documentation below.

How it works:

RF24Mesh leverages the features provided in the supporting libraries, RF24 and RF24Network to provide connectivity between many devices, even if they are far away from the central node or moving around. Each node is assigned a unique identifier (1-255) and uses that high level-address to request a dynamic address at the network layer, according to the structure of the mesh and proximity to other devices in the mesh. Nodes typically communicate using the RF24Network API, with RF24Mesh working at a higher layer to provide address lookups, dynamic addressing and connectivity.

Nodes arrange themselves using a tree topology around the master node, which provides address lookups etc. for the mesh. At any given time, each node has a path of communication through the network, and can re-establish and verify a new link if the current path fails. In each network there is the master node (the base node), children of the master node (relay nodes) with children nodes themselves, and leaf nodes, (children of the master or its children) with no child nodes attached.

The protocol is fairly simple for each node. On power up or reset, nodes attempt to find a connection by using multicast. Multicasting is arranged into 5 levels, so the master node is contacted first, then its direct children, then their direct children and so on, making the way nodes attach themselves to the network tend towards close proximity to the master node in a tree-like structure. Once contact is made the node requests an address from the master node, either directly or through other nodes. Once an address has been assigned and verified, the node is considered connected to the mesh and can then communicate with all other nodes in the mesh, and beyond to other systems etc.

Nodes have the option of managing their connection however desired, and quite often, a simple timer is suitable to verify connectivity after a set period of time. Transmitting nodes can also detect when communication is failing and re-establish their connectivity to the mesh as required. That is, at any given time, nodes will have a single path to communicate over the network, but if that link fails, they can re-establish their communication path through the network.

The communication layers are interoperable, meaning a network can combine devices running RF24Network and RF24Mesh libraries together, with static nodes running RF24Network, and dynamic nodes running RF24Mesh. Devices running RF24Ethernet (TCP/IP) can also communicate using the other layers of the libraries, interacting directly with RF24Mesh and/or RF24Network nodes. This allows an array of devices to operate on the same wireless mesh network, from ATTiny all the way to the more powerful devices like ESP32 and Raspberry Pi, using the RF24Network, RF24Mesh APIs and/or standard networking tools and protocols. (ICMP/TCP/UDP, etc.)

Current status, testing etc. 

RF24Mesh is now past the development/beta testing stage and considered (by me) to be stable and well tested. The mesh can handle a large number of nodes, theoretically up to 255, but in practice, more like 15-25 nodes sending or receiving data every few seconds is a reasonable bet. It generally depends on the type of traffic and frequency with which it is generated, as a much larger number of nodes reporting data every minute presents only a small load for the network.

The progress made with RF24Mesh is very encouraging, supporting simple communication scenarios, small household sensor networks, home automation systems, and larger expanded networks stretching over relatively long distances with the more powerful RF24 modules.

The RF24 communication stack is generally complete, with bug reports and issues dropping off quite nicely, but development is still ongoing. As always, please report any potential bugs or issues using the GitHub link below. 


ENC28J60 Network Modules, Arduino and the Internet of Things (IoT):

An experiment, overview and analysis of TCP/IP networking with Arduino Ethernet

The module in question was sent to me by the friendly people at ICStation.com , in exchange for a review of the module itself.

Video Review:


ENC28J60 Mini Network Module:
http://www.icstation.com/images/big/productimages/4060/4060.JPG
  • 3.3V operation
  • Create a mini Arduino Web Server or Client
  • Plugs into your home router or modem just like a computer

    The uIP TCP/IP stack is software driven, while the ENC28J60 module handles the data mainly at the network (Ethernet) level. One main difference between UIPEthernet and the official Arduino Ethernet library is the need to keep the TCP/IP stack updated via software with the ENC28J60.

    Hardware: 

    The ENC28J60 was very easy to wire up, using the default SPI pins shown at http://arduino.cc/en/Reference/SPI along with the VCC and Ground pins.

    The device seems to work ok connected to the 3.3v output of the Arduino, but it seems best to use an external power supply or battery
    .

    Software:

    Two main libraries appear to be available: Ethercard and the UIPEthernet libraries. This post will focus on usage via UIPEthernet, since it shares a common API with the official Arduino Ethernet library.

    I am using the UIPEthernet library and examples found at: https://github.com/TMRh20/arduino_uip/tree/fix_errata12

    In attempting to utilize the modules and UIPEthernet for creating a simple webserver, I ran into a number of issues with data being sent or received incorrectly. Over a number of days, and benefiting from the information gained in building the RF24Ethernet library, I have been working to improve UIPEthernet, and also created some examples to demonstrate the issues and current usage.

    (A pull request has been made regarding these suggested changes to the main library)

    Overview of Library Changes (At the time of this post):
    Changes:
    • Introduce client.waitAvailable(); function  - Ensures the IP stack is processing incoming data while waiting for incoming data for a maximum defined duration in milliseconds: client.waitAvailable(3000);
    • Introduce Ethernet.update(); function - Allows user applications to ensure the IP stack is processing data while delaying or performing other tasks: Ethernet.update();
    • Perform additional processing of data while calling client.available();
    • Per testing, reduce max segment size (MSS) and the receive window to 511 bytes from 512 due to unidentified issue.
    • Reduce number of packets per socket to prevent data loss
    • Re-open the TCP window at timed intervals when data has not been sent or received on an open connection to prevent stalling during large downloads
    • Fix for outgoing data corruption
    • Force application to wait while writing data 
    • Better TCP window handling
    • Other minor changes, add examples
    Changes - Technical Summary:

    The first changes introduced added some new functionality, geared towards a software driven IP stack.  With the addition of additional processing and the Ethernet.update() function, the IP stack is more responsive to ICMP requests etc.

    The new examples included with my branch of the library clearly demonstrate some of the issues I encountered when attempting to create a simple web client and/or web server. The first issue I noticed involved the device crashing or stalling, and then failing to reconnect or respond to pings. Testing indicated that the hardware itself was still responding and receving packets, so the issue seemed to be software based. These issues mostly seem to be addressed in the fix_errata12 branch, but testing via large dowloads from a web server indicated there were still some remaining issues, mostly surrounding data corruption and flow control.

    Most of the remaining changes attempt to address issues with corrupted or out of sequence data, and to manage the flow of data better. In some situations, for example, web servers will send TCP payloads equal in size to 1/2 of the TCP Max Segment Size (MSS) which causes problems with flow control. The IP stack will now only close the TCP window if data is being received fast enought to do so, which appears to reduce issues with these transfers. The main issues come when a sending device sends only 1/2 MSS, and the recipient closes the window. The library will now count individual bytes in memory vs packets with data in them, and only close the TCP window if the assigned data buffers are completely full.

    Testing has been very successful, with the device demonstrating full functionality for extended periods of time, downloading 70+MB of data without any errors detected in the data or hardware. The new web server examples and HttpClientTest.ino example seem to be good displays of the current capabilities and error free operation, and can be used to demonstrate the issues, when used with the original library code, or when changes are reverted.

    Summary:
     
    The mini ENC28J60 modules are an inexpensive and effective way to add internet connectivity to any Arduino project, and their size make them very suitable for use with Arduino.

    I found the hardware itself to be very reliable and easy to use, although there seemed to be a few minor issues with the related Arduino library (UIPEthernet) that was tested.
    As noted, I have submitted some workarounds/fixes for review and possible integration into the main library at the time of this post, so users are encouraged to use the library linked above.

    ICStation.com:

    A big thank you to www.ICStation.com for sending me this product, as well as being very patient.

    It has been a number of months since I actually received it, but their small contribution has lead to the creation of the RF24Ethernet TCP/IP Radio Network library, and some potential improvements to the UIPEthernet library. A wealth of Arduino and Raspberry Pi related products, gizmos and devices are available on their website, and their staff seem very friendly.



    RF24Ethernet - For DIY Internet of Things and Home Sensor Networks
    Adding internet connectivity to RF24Networks using nrf24l01+ modules as ethernet cards

    RF24Ethernet, What is it and how does it work?

    For anybody not familiar with my blog, I have done a lot of recent work in improving the radio driver and related network library for NRF24L01+ radio modules, using Arduino and Raspberry Pi or using two or more Arduinos. These very inexpensive and feature-rich devices can be used to create your own home sensor network, with very little cost when compared to most available solutions.

    RF24Ethernet will allow you to use a Raspberry Pi or Arduino as the 'gateway' machine to your network, and lets you connect to your sensors directly, using any device that has a web-browser, whether it is an iPod, PC, etc. or your sensors can connect out to the internet for information.

    Current Libraries:
    RF24 - OSI Layer 2 radio driver for nrf24l01+ modules
    RF24Network - OSI Layer 3 network driver for RF24
    RF24Ethernet - OSI Layer 4 Protocol (TCP/IP) driver for RF24Network
    RF24Mesh - Mesh networking layer for RF24Network (Dynamic config and topology)

    How it works: (Updated)

    The RF24Ethernet library is currently in the testing phase, and uses the UIP TCP/IP stack.
    The library has been modelled after the Arduino Ethernet library, allowing users to create web enabled, wireless devices, without having to learn the RF24 or RF24Network APIs. The addition of a real protocol (TCP) on top of RF24Network provides a level of simplicity, consistency and reliability beyond what was previously possible.

    The IP address of each node needs to be statically assigned. Translation between RF24/RF24Network (MAC) addresses and IP addresses is handled in one of two ways, depending on the configuration. The default configuration uses ARP requests to find the correct node when TCP data is incoming, and users can optionally utilize RF24Mesh to provide MAC/IP translation and/or dynamic address assignment at the network layer.

    Configuration/Testing: (Updated)

    Configuration is now integrated directly into RF24Ethernet, with the RF24Network address specified as the MAC address, and this can be further automated by utilizing the RF24Mesh layer along with RF24Ethernet. Users have the option of creating a static network, or utilizing RF24Mesh to create a dynamic network with nodes capable of moving around physically or utilizing fail-over nodes. The RF24toTUN application running on a Raspberry Pi automatically performs discovery and routing for incoming TCP/IP data, so users only have to configure static IP addresses, and a unique identifier for each node when used with RF24Mesh.

    Setup (Arduino & RPi):
     RPi:
      Install RF24, RF24Network, RF24Mesh & RF24toTUN libraries
      Note: RF24toTUN requires the boost libraries. Run sudo apt-get install libboost1.50-all 
      a: wget http://tmrh20.github.io/RF24Installer/RPi/install.sh
      b: chmod +x install.sh
      c: ./install.sh  (Note: This should be run without sudo or all your base files will belong to root)
    Arduino:
     a:  Install RF24 lib
      c: Install RF24Ethernet library
      d: If using with a RPi, run any of the Getting_Started examples. You should be able to ping and connect to the remote Arduino.
      e: If using with a SLIP interface (non-RPi) :
          1. Run the  SLIP_Gateway.ino example on the Arduino connected to the PC/MAC etc
          2. Run the SLIP_InteractiveServer.ino example on another Arduino.
          3. Once the SLIP interface is configured as per the examples, you should be able to ping and connect to the remote Arduino.


     RF24Ethernet - Setup, config and demo
    The initial proof of concept interface borrowed directly from the SerialIP library, using the uIP TCP/IP stack and uIP proto-sockets/proto-threads. Since testing was very succesful, the library has been developed to provide a simpler user interface, very similar to the Arduino Ethernet library. In its current form, a node running RF24Ethernet can act as a standard TCP server, and can function as a telnet or web server, by modifying the included example.

    How can this be used?

    The possibilities from here are virtually endless, with very inexpensive sensor networks being provided with direct TCP/IP connectivity. One of the more obvious applications involves simple retrieval of information from network or internet sources, such as the time or the current weather. Controlling a connected LED or other attached device can be as simple as opening a bookmark in your browser.
     
    To take things a whole lot further, this could even allow RF24Network to be utilized as a bridge betwen LANs, using sensor networks for emergency commuinications etc, when main connections are down. Testing shows that speeds  across the network are slightly better than or equivalent to a dial-up connection, so it seems to provide a practical solution for low-speed data transfers like sensor data, text-based email or chat.

    Results of Testing:

    The above picture presents an idea of the latency, etc when sending ping requests of varying sizes from the RPi master node to a connected sensor node.

    Documentation/Further Development?
    See http://tmrh20.github.io/RF24Ethernet for available documentation.

    Update Nov24:
    Working with UIP TCP/IP stack has been a bit of a challenge to say the least, but things seem to be coming along nicely. The RF24toTUN library has been updated to route payloads according to the radio MAC address, so integration with RF24Network is coming along nicely. Support for the RPi to send fragmented multicast payloads has been added, so ARP requests are working nicely within RF24Network. This will allow full integration with RF24Mesh, to create a dynamic TCP/IP mesh sensor network if desired, with very little to no configuration needed by the user. I've commited the current code to a new branch (master_dev) to hold the new code until it is cleaned up and more functionality is added. Currently, it only supports incoming data connections as a simple server, and ICMP packets.

    Update Nov30:
    The Server and Client API to match the official Ethernet library is now in place and working for the most part. DHCP, DNS lookups and UDP in generall is not developed yet. There are still some bugs and oddities to work out, but overall, the TCP server and client seem to perform fairly well given the circumstances. Web client and server examples have been included.

    Update Dec 7:
    The library seems to be working very well now, and the API is about 95% complete. Most bugs have been addressed, with only a few minor issues left, generally surrounding timeouts during very large data transfers. The existing examples have been updated, and some new examples have been added, demonstrating a simple web server and interaction with a sensor node via a web browser. Some documentation has also been created and is linked below.

    Update Dec9:
    RF24Mesh has been integrated with RF24toTUN, to provide automated addressing and mesh support for RF24Ethernet. When building RF24toTUN, use 'sudo make install MESH=1' to compile with RF24Mesh support. If using with RF24Ethernet, see the new SimpleServer_Mesh.ino example for usage. An install script has also been created for RPi, to simplify installation of the various libraries.

    Update Dec 14:
    Updated RF24Ethernet to support SLIP or TUN devices as well. This requires RF24Mesh to perform MAC/IP translation. Two new examples have been added, SLIP_Gateway.ino demonstrates how an Arduino can act as a simple USB interface to any device that supports SLIP. SLIP_InteractiveServer demonstrates how to use RF24Ethernet and RF24Mesh with a SLIP or TUN interface.

    Update Dec 28:
    Modified the MAC address format due to issues with standards when using it on RPi/Linux. Modified the timing of uip restarts. Updated RF24toTUN to provide cmd line configuration for all major options including node addressing. Fixes and updates to RF24Network and RF24toTUN seem to have addressed some buggy behaviour.

    Update Dec 30:
    Todays latest updates should be applied along with the latest updates to RF24toTUN and RF24Network. The main change fixes corrupt client requests, which would have been noticed when establishing outgoing connections. The remaining changes revolve around reliability, the order of operations, and the timing of things, and seem to really bring RF24Ethernet closer to a stable, reliable library. 
     
    Update Jan 2, 2015
    This round of updates comes after a very long round of testing and coding. The RF24 library has been updated to provide even better timing, reliability and throughput. Changes to RF24Network improve fragmentation/reassembly. RF24toTUN has been updated to fully support all 3 datarates, and throughput has been improved for all speeds. RF24Ethernet has been updated to provide a timeout for connections, part of better handling for TCP window reopening and failures, and allows users to configure the periodic timer via uip-conf.h to provide faster transfers. These changes bring RF24Ethernet a lot closer to a stable release. Testing shows that these updates provide easily noticeable increases in reliability and speed at all levels, and updating all the mentioned libraries is recommended.

    Update Jan 4, 2015
    Added connection timeouts to recover from hangs during failed client donwloads, which adds improved reliability to the library. Added better TCP window management to prevent those hangs during client downloads, along with configuration options. Documentation updated to include new features and options.

    Update Jan 16, 2015
    Updated to v1.2b - Adds UDP and DNS support along with a pile of fixes and updates. Users should update RF24Network and RF24toTUN along with RF24Ethernet. This will be the final update to this blog post. See the documenation below or check back here at tmrh20.blogspot.com for additional posts.

    See https://github.com/TMRh20 for all related code/library downloads, and http://nrf24.github.io/RF24Ethernet for available documentation.


    My Custom Zephyr + Arduino Build for Arduino Uno Q: Adding Zephyr Networking to the Mix

     My Custom Zephyr + Arduino Build for Arduino Uno Q: Adding Zephyr Networking to the Mix Using RF24Ethernet with native Zephyr networking  F...