Optimizing communications with Yoctopuce devices

Optimizing communications with Yoctopuce devices

Experienced users of Yoctopuce modules will sooner or later wonder how much data needs to be transferred to retrieve the measures from Yoctopuce sensors. It can be the case to optimize GSM consumption, as it we discussed in a previous post, but also simply to improve the performances of an application where the modules are connected by USB but where the measures must be read very quickly. So here is a detailed explanation to know everything and optimize everything...


Base protocol


The elementary working mode of all Yoctopuce modules is to receive requests in HTTP format and to answer with data in the JSON format, including the state and the parameters of all the functions of the sensor. This behaviour applies not only to network communications, where these protocols are very common, but also for USB communications, in which case the HTTP/JSON enodings are encapsulated into USB frames.

The advantage of this solution is that it enables a very high degree of flexibility on the type and format of the data transmitted by the modules. For example, a program designed to monitor the temperature put into service 5 years ago can work today with sensors which didn't exist at the time, or for which the features have been extended since with new firmware, without raising compatibility issues.

The disadvantage of this rather simple solution is that it is rather verbose: if the aim is only to read the value of a sensor, it may seem exaggerated to transfer the whole state of the sensor each time the value is read. This will not be a problem if you want to read a sensor through USB once per second or less, because the transfer takes only a few hundredth of seconds, but in some cases this can raise issues.

Therefore we propose here different mechanism which are documented below, to avoid data transfer penalty. It is for you to choose which one best suits your needs...

API cache memory


When you read several values on the same Yoctopuce module, for example to read the six entries of a Yocto-MaxiThermistor, the data exchange occurs only on the first reading. The values are at this time stored in the API cache memory, to be then provided instantaneously. The validity period of this cache memory is normally 5 ms. If you need to increase this length to reduce communications even more, you can use the YAPI.SetCacheValidity() method:


  YAPI.SetCacheValidity(100);
 


However, this solution doesn't help you to increase the reading frequency.

Reading a unique attribute


You can also explicitly read a single attribute of a sensor, instead of transferring the whole state of the module. You can do so with the loadAttribute() method, as we demonstrated it in a previous post:


  sensor.loadAttribute("unit");
 


This is efficient if you need to read only one or two values, but if the requests are numerous, you'll feel the penalty of repeating requests.

Reading by callback


Reading sensors by callback, as described in this previous post, always stays the most efficient method to read sensors. It allows the selective and spontaneous transfer of the bare minimum of wanted data, without the penalty of encapsulating the data into an HTTP or JSON format. Yoctopuce sensors provide the choice of callbacks each time the value changes, or of periodical callbacks at predefined intervals.


  lightSensor.registerValueCallback(myValueChangeCallbackFunction);

  tempSensor.set_reportFrequency("15/m");
  tempSensor.registerTimedReportCallback(myPeriodicCallbackFunction);
 


This method is markedly more efficient that explicit readings, both for working through USB or through the network.

Reading by function enumeration


If you are not at ease with callback functions, there is an alternative function enabling you to obtain at any time the value of the latest signaled value modification, without additional data transfer: enumerating the module functions. Indeed, the Yoctopuce API implicitly stores the latest value of each measure, and you can retrieve them with the functionValue() functions, without any communication cost:


  string serial = module.get_serialNumber();
  for (int i = 0; i < module.functionCount(); i++) {
    string hardwareId = serial + "." + module.functionId(i);
    Console.WriteLine(hardwareId + ": " + module.functionValue(i));
  }
 



Configuration change callback


The only limitation of reading by callback (or by function enumeration) described above is that it is limited to the sensor measured value, and it doesn't allow you to monitor auxiliary attributes such as the unit of the calibration parameters. To avoid having to reintroduce useless periodical accesses with the base protocol, you can henceforth register as well a callback function which is called each time that a configuration change is detected on a module. Thus, the application can selectively reload the configuration of the sensors (unit, and so on), even if it was modified independently by another application:


  tempSensor.registerConfigChangeCallback(myConfigChangeCallbackFunction);
 



Module inventory and Plug-and-Play


On top of the data traffic necessary to read the sensors, there is the auxiliary traffic necessary to enumerate the connected modules and to manage pug-and-play. This traffic is insignificant by USB but can be very significant in case of network connections.

As a rule, each connection and disconnection of a module on a network hub is signaled by an event, which triggers a module connection or disconnection callback. The application periodically calls YAPI.UpdateDeviceList() to be informed of them through dedicated callbacks (see this post).

Nevertheless, to ensure never to miss a connection or disconnection event, once every ten seconds, calls to YAPI.UpdateDeviceList() trigger a complete transfer of the list of connected modules and of each of their functions. The cost of this transfer is meaningless on a local network, but can be significative if performances are of the essence or on a GSM line. In these cases, if you expect few or no changes of modules, or when there is no reason to anticipate loss of plug-and-play events, we recommend to increase the validity duration of the module inventory to an hour or more, using the YAPI.SetDeviceListValidity() method:


  YAPI.SetDeviceListValidity(3600);
 


Thus, the auxiliary traffic is reduced to a strict minimum as well.

Firmware and API revisions


The methods registerConfigChangeCallback, SetDeviceListValidity and SetCacheValidity are available since revision 31701 of our programming libraries.

In order to use configuration change callbacks, you should also ensure to use device firmwares (and possibly VirtualHub) revisions 31745 or up.

To summarize

For maximum reduction of data transmitted over the network:

  • Read the post on data usage again
  • Use periodical callbacks to read data:
    sensor.registerTimedReportCallback()
  • Disable sending instantaneous values:
    sensor.muteValueCallbacks()
  • Lengthen the module inventory validity duration:
    YAPI.SetDeviceListValidity()
  • Avoid all recurrent calls to get_*** functions
  • Use configuration change callbacks:
    sensor.registerConfigChangeCallback()
  • Use a network connection of WebSocket type:
    YAPI.RegisterHub("ws://...")


For reading sensors at maximum frequency:

  • Read the post on value callbacks again
  • Use periodical callbacks to read data:
    sensor.registerTimedReportCallback()
  • Disable sending instantaneous values:
    sensor.muteValueCallbacks()
  • Lengthen the module inventory validity duration:
    YAPI.SetDeviceListValidity()
  • Avoid all recurrent calls to get_*** functions
  • Méfiez-vous des mini-PC ARM bon marché, le controlleur USB est parfois limité

When using a network-based connection,

  • Use configuration change callbacks:
    sensor.registerConfigChangeCallback()
  • Use a WebSocket connection:
    YAPI.RegisterHub("ws://...")


In any event, don't forget that you can't go above the limits imposed by the sensor technology. For each module, there is a documented maximum refresh rate, which depends mainly on the circuit performing the measure. It's useless to transfer light measures 100 times per second if the sensor updates the measure only ten times per second...

Add a comment No comment yet Back to blog












Yoctopuce, get your stuff connected.