One rule-of-thumb in software development is that ‘even the least probable occurrence is guaranteed to occur in large scale’. For example, if code contains a glitch that happens once per million second in one device, in a device network of a million, it occurs, on average, every second. To avoid and mitigate this is one of the challenging aspects of software development.
Working with Wirepas Mesh
Development of the Wirepas Mesh usually starts with small scale, i.e. number of devices. When that is done and we are happy with the results, the device amount is increased rapidly to spot the difficult-to-catch problems. Mechanism itself works fine but understandably contains inherit problem of analysing what is going on. It does not really help to check issues in larger setups if there are no ways to analyse what is going on in the network.
Example case – fast mass inventory
An example of a such that I could go through is the use case of mass inventory. The use case is a pallet containing hundreds, even thousands of individual packages each of which is tracked. Now, when a pallet is entering to a dedicated area, such as a warehouse or a truck loading area, there is a need to get the inventory of each package on the pallet. Each device is equipped with a Wirepas Mesh enabled tag. The inventory is then performed by ensuring that tags find the Wirepas Mesh network routing devices as fast as possible. The routing devices transfer the tracking information to backend via normal operation of Wirepas Mesh.
What makes this use case challenging is the fact that tags are battery-operated. This makes power saving one of the key requirements, as well as the ability to create full inventory as fast as possible. These are basically conflicting requirements for a wireless battery-operated network.
Traditionally, to understand what happens in this kind of set up, is to inject trace harness to devices in the setup. This trace harness would be needed to inject to both tags and Mesh routing devices to see ‘both sides of the coin’. However, sheer amount of these tags makes impossible to create practical means to extract debug harness from each of them.
How to ensure that the needs of the use case are met?
1) One major design feature of the Wirepas Mesh is that devices are intelligent on their operation and can gracefully operate together in mass volumes. To verify this operation, we needed one additional testing aspect, which requires tracking all activity in radio interface. In addition to this it is mandatory that a device is totally non-intrusive and does not affect the existing network, i.e. the tool is listening only. This tool, denoted here as the network scanner, basically listens all packets in the radio interface, for example data transmissions, acknowledgements, control plane signals, network discovery signals, basically everything. Besides that, also metadata such as exact transmission time of the data packet, signal strength etc. are tracked. Also, failures such as packet collisions are tracked. Same radio devices as other devices can be used for implementing network scanner.
Definite pro of the network scanner is that it shows how the devices are working collaboratively in the radio interface. It has been proven many times that to optimize operation in dense scale requires totally different approach than, for example, optimizing traditional point-to-point operation. Optimizing performance by using just two devices would be a total disaster in large scale. By using the network scanner, also the difficult-to-spot occurrences are found. For example:
a) Starvation of single device (one device cannot get transmission through).
b) Ensure that tags do not use excess transmission power.
c) Reveal tendency for ‘mass hysteria’ occurring when a large number of devices decide to perform radio operations synchronously.
2) Another design feature of Wirepas Mesh network is the ability to use multiple radio channels. However, network scanner can only track one radio channel at the time so to get a holistic view of the system, multiple scanners are needed. And to make that feasible, we needed a cost-effective solution. This is achieved by a setup which contains very cost-effective components.
a) First, we chose Raspberry Pi mainly because of it is very commonly used and already familiar. Second reason is that it contains easy built-in SPI (Serial Peripheral Interface) support. This interface is used between Raspberry Pi and Radio device and is fast enough to log all incoming data without becoming a bottleneck.
b) For the radio device Promistel Raspberry Pi HAT device was optimal and an affordable choice and because it resides in the Raspberry Pi casing, it is a sustainable choice.
3) Another requirement for testing is, of course, power consumption measurement. Since tracked items are battery operated, power consumption measurement ensures that battery usage is sustainable. Albeit the network scanner ensures that devices do not waste power on radio interface, devices may waste energy on other aspects. However, this can be done only a very small subset of the tracked items.
To summarize, the testing setup is following:
- Wirepas Mesh network sends data to Wirepas Mesh gateway. This data contains actual use case of the setup, for example, detection of tracked item in the setup.
- Some devices are injected with debug trace harness. In practice, there can be only few such devices.
- Network scanners track the communication between devices in radio interface
- Power consumption measurement is done for tags. Again, in practice, this can be only performed for very few of them.
It is important to be able to combine information from multiple sources. This makes it to ensure operation from different points-of-view and get the ‘big picture’ of how the system is operating.
Note that there are additional use cases that the network scanner enables, since it tracks all the packets in radio channel heard. An example is to track devices in the radio range. By tracking device RSSI and device address, it is possible to find lost devices. When searching for the lost device, all that is necessary is to walk towards the direction where RSSI is stronger. To make use case feasible, you need portable device with user interface capability. We have been using Raspberry Pi with built-in display and USB battery pack.
As a developer of Wirepas Mesh the most rewarding aspect is to make a large number of devices to act collaboratively. To ensure that the system works requires collecting data from various angles. Only when all the collected data is combined into information and analysis, it is possible to truly understand how such setup works. To develop and test a collaborative element in a dense and large-scale network also requires development of new testing techniques, which may even solve some totally unrelated use cases.
This is the first blog post in the series of Wirepas Developer Blogs. The purpose of these blogs is to have a sneak peek to what the development of the Wirepas Mesh or related technologies, such as support tools or testing, looks like.