GSM data usage for sensor monitoring

A large portion of the costs for a remote monitoring installation using the cellular network is linked to communication costs that the provider bills. It is therefore important to correctly estimate the amount of data transmitted in order to budget the system, to select the most appropriate subscription, and make the best technical choices. Here are a few elements which can help you to do so.



To give a concrete basis to this discussion, let's take as an example a small system made of a YoctoHub-GSM-3G-EU, providing connectivity to three Yoctopuce modules:

• A Yocto-PT100 to measure the temperature
• A Yocto-4-20mA-Rx to measure two specific physical quantities
• A Yocto-GPS to locate the system (latitude, longitude, altitude, speed)

All together, there are therefore seven measures that we want to transmit.

Our GSM transmission test system

If we want to monitor these seven measures every minute, we could imagine that, with an ideal system, this costs for example 4 bytes per measure, so roughly 30 bytes per minute.

Unfortunately, Internet is not that efficient for small quantities of data and we must add a good number of auxiliary data needed by the system for it to work.

1. We must minimally encapsulate the data in TCP/IP network packages, in order for them to reach the appropriate server, which adds at least 50 bytes per transmission.
2. The exchange protocol used by the server can add a few hundreds of bytes per transmission. It is the case for example with the HTTP protocol, which uses very verbose text headers.
3. The data themselves are often encoded in a more explicit way than a binary transmission, for example in JSON. This easily multiplies by 3 or 4 the quantity of data to be transmitted.
4. Checking the continuity of the GSM connection in order to post the data without delay can also cost a few tens of bytes per minute, according the desired quality level.

This may seem somewhat vague, but fortunately, to make everything more precise, YoctoHub-GSMs contain a transmitted and received byte counter. We don't guarantee that the number billed by the provider is exactly the same, as rounding can apply, but this allows us to have a more precise idea of data usage. To retrieve the counter values, you can simply call the following methods:

If your YoctoHub-GSM is connected by a USB cable to a computer, you can write a short script to retrieve these two values every minute and perform a few statistic computations. This is what we have done here to document the data usage in a few classic scenarios.

Basic usage

When a YoctoHub-GSM is connected to the TCP/IP network, its default behavior is to check its connectivity every minute, in order to initiate a new connection in case of problem. This behavior has a cost of 60 bytes per minute, in upload and download. If need be, you can space out the checking packets with the method

Moreover, the hub connects to an NTP server every 10 minutes to potentially adjust the real time clock, used to time stamp the measures. This represents 150 bytes per connection, in both directions.

Multiply by 60 minutes, 24 hours and 31 days, and you have a 6.7 MB/month consumption to maintain connectivity and time stamping 24/7, regardless of the number of sensors connected. You can naturally proportionally reduce this volume by putting the hub into sleep mode with the internal clock during some hours of the night, or by waking it up periodically only.

Sending the measures by simple HTTP callback

If you add a minimalist transmission of the current value of each sensor every minute with a simple HTTP GET request, with a CSV coding, you rise up to 0.6 KB for upload and 0.5 KB for download per minute, that is 50 MB/month for day and night operation. If you select the fancier alternative of transmission by HTTP POST with WWW-urlcoded form, you can even reach 70 MB/month.

These numbers correspond to our little test system with seven measured values. If you use more sensors, the data usage will raise a bit, but not proportionally since a large part of the data usage comes from the protocols overhead, which is independent of the number of sensors.

Connecting by Yocto-API HTTP callback

Using the Yocto-API HTTP callback protocol costs more. Indeed, then it's not only one value per sensor which is transmitted every minute, but the current state of each sensor, including minimal and maximal values, the measuring unit, the accuracy, the settings, uptime, and so on. And naturally, the advantage of this protocol is that it allows you to act on the modules as well, to change their configuration remotely, or trigger an actuator.

In its basic version, the Yocto-API HTTP protocol has a costs of 12.1 KM/minute for upload and 0.7 KB/minute for download, so a total of 580 MB/month. Fortunately, if we enable the optimization introduced some time ago for the PHP library, we can cut down the traffic to 300 MB/month.

Connecting by Yocto-API WebSocket callback

Using a persistent WebSocket channel avoids repeating the HTTP headers for each request. However, maintaining this channel open permanently, with a continuous publication of the measures, also has a non-negligible cost.

Without further precaution, a WebSocket permanent connection for our system costs between 3 and 5 KB/minute in upload, between 2 and 3 KB/minute in download, for a total of about 350 MB/month. As each new measure is transmitted instantaneously, the volume of transmitted data varies depending on measure variability. However, this allows you, for example with the help of the GatewayHub, to generate as many simple HTTP callbacks to third-party services that you want, without additional GSM traffic.

In the same way, if you want to forward a WebSocket callback to an Yocto-API HTTP callback with the GatewayHub, you must take into account loading the full state of each sensor through the WebSocket channel, which represents an additional load. For a Yocto-API HTTP callback per minute, we reach 500 MB/month. It's almost double from an optimized Yocto-API callback performed directly from the hub, but it has the advantage to allow you to take full control of the hub remotely.

Fortunately, we can do better than that. Indeed, the 350 MB/month used to transmit instantaneously and 24/7 the current value of each sensor are not necessarily useful, especially if you want to treat the data only once every minute. You simply have to disable on all the sensors the sending of instantaneous values and to use instead measures with a timed interval.

Thanks to using time interval measures, we bring back the traffic to 1.3KB/min for download and 0.9KB/min for upload, so 100 MB/month only with a permanent WebSocket connection. Moreover, for each time interval, you receive not only the average value but also the min/max during the interval. And if there is a temporary GSM connectivity failure, if you have enabled the data loggers on the modules, you can precisely retrieve the missing data, as they are recorded simultaneously with the same granularity.

This solution requires however that you code your solution by exploiting the Yoctopuce library and its WebSocket callback API. If you are looking for something easier to set up, using the GatewayHub and a simple HTTP callback to a third party service such as EmonCMS, you can also simply configure the sensors to propagate the average value per minute, rather than the instantaneous value, as explained in this post.

This probably enables you to benefit from the possibility of a remote connection to your hub, while consuming less than 100 MB/month.

Conclusion

GSM data usage differences between the different options are really significant. It's therefore worth it to take them into account as early as the design stage of the software solution that you are going to implement to read your sensors.

Remember as well that you can drastically reduce the data usage by spacing out the measures and/or disconnecting the hub from the network by putting it to sleep.

And before you sign a contract with a GSM provider, don't forget to check in real life with a test SIM that the provider counts the data in the way you expect it to ...