Retrieving recorded measures over the Internet

Retrieving recorded measures over the Internet

There is a data logger in almost all of our sensors. This data logger can automatically record values measured by the sensor. The advantage is that it continues to record data even network connectivity is lost or when the application stops. When the application takes control again, it is possible to get back the measures performed during the interval. But up till now, it was unfortunately impossible to retrieve data logger measure using the HTTP callback mode. Thanks to the new WebSocket callback mode, you can now use the data logger over an Internet link without limits. This week, we are writing a Java web application which illustrates this feature.

This web application is a variant of the application that we had created for our weather station. It can display the graph of the values measured by a sensor on a Yocto-MaxiDisplay. The screen and the sensor are each connected on a YoctoHub, which connects itself with a WebSocket callback to the web application serving as a central hub. By hosting the web application on the Internet, we can install the sensor in one building and the display in another building at the other end of the world. There is no need to modify the firewall or router configuration because it is the YoctoHub which performs the connection with the application.

The Yocto-MaxiDisplay and the Yocto-Temperature can be installed in two different buildings
The Yocto-MaxiDisplay and the Yocto-Temperature can be installed in two different buildings


In our previous version of the weather application, the HTTP callback mechanism did not allow us to keep connections open over a long period, nor to use the data logger. So we had configured the YoctoHubs so that they connect to the server every 5 minutes, and the measures were stored in a database. With the WebSocket callbacks, we can now establish a persistent and bidirectional connection with the application. This means that you can display the value of a sensor in real time, but above all that you can do without the database by using the data logger.

In the opposite to the HTTP callback, WebSocket callbacks enable us to maintain a persistant and truly bidirectional connection with the server
In the opposite to the HTTP callback, WebSocket callbacks enable us to maintain a persistant and truly bidirectional connection with the server


Using the Java library with WebSocket support


To implement this application, we are going to use the Java library with WebSocket callback support. Unfortunately, the WebSocket support requires JavaEE (Java Enterprise Edition). In order to maintain forward compatibility with our library, we have decided to duplicate the Java library and to offer two versions of our library. The traditional version located in the yoctolib directory doesn't include WebSocket support but compiles with any 1.7 or 1.8 JDK. The new library, including WebSocket support, requires JavaEE. This new version is available in the yoctolib-jEE directory of the zip file.

Apart from the WebSocket support, both versions are identical and are going to be maintained in the same way and at the same rhythm. They are both located in the same archive. If you don't intend to use WebSockets, you can continue using the same source files or JAR files. If, on the other hand, you need to use WebSocket, you must use the source files located in the yoctolib-jEE directory of the jar file yoctolib-jEE.jar. We have also published this library on Maven Central, and you can use it in you Maven projects by adding in your pom file the following dependency:

<dependency> <groupId>com.yoctopuce.java</groupId> <artifactId>yoctolib-jee</artifactId> <version>1.10.23400</version> </dependency>


The application


Our application is composed of three classes: WebSocketEndpoint, WorkerThread, and SensorCache. The main class is WorkerThread. As its name indicates it, it's a unique thread which is started when there is the first connection. The run() method contains all the logic of our application.

The subtlety of this class is its way to detect the list of usable Yoctopuce modules. Instead of making an explicit enumeration, we have two callback methods which are called as soon as a new module is detected by the Yoctopuce library and as soon as a module is not detected anymore. These callback methods are well known.

Usually, we register a YoctoHub only once and we use UpdateDeviceList() to detect when a module is physically connected or disconnected on a computer or YoctoHub. The present case is slightly different: we are trying to detect when a hub connects itself to our application. The main loop runs UpdateDeviceList() even if no YoctoHub is connected. When an incoming connection is detected, we register the YoctoHub that connected itself and all the modules of this just connected YoctoHub are going to be detected during the next run. In the same way, when the connection is over, we are going to unregister the YoctoHub so that its modules are not listed anymore.


public class WorkerThread implements Runnable, YAPI.DeviceArrivalCallback, YAPI.DeviceRemovalCallback, YSensor.UpdateCallback
{
    private HashMap<String, SensorCache> _sensors_hwid = new HashMap<>();
    private ArrayList<String> _displays_hwid = new ArrayList<>();

    @Override
    public void run()
    {
        YAPI.RegisterDeviceArrivalCallback(this);
        YAPI.RegisterDeviceRemovalCallback(this);
        System.out.printf("worker thread started\n");
        while (_mustRun) {
            try {
                YAPI.UpdateDeviceList();
                for (String disp_hwid : _displays_hwid) {
                    YDisplay ydisplay = YDisplay.FindDisplay(disp_hwid);
                    ...
                        sensorCache.refresh();
                        repaintDisplay(sensorCache, ydisplay);
                }
                YAPI.Sleep(1000);
            } catch (YAPI_Exception e) {
                e.printStackTrace();
            }
        }
    }

    public void addSession(Session session)
    {
        try {
            YAPI.PreregisterHubWebSocketCallback(session);
        } catch (YAPI_Exception e) {
            e.printStackTrace();
        }
    }

    public void removeSession(Session session)
    {
        YAPI.UnregisterHubWebSocketCallback(session);
    }

    @Override
    public void yDeviceArrival(YModule module)
    {
        try {
            ArrayList<String> functionIds = module.get_functionIds("Sensor");
            String serialNumber = module.get_serialNumber();
            for (String funid : functionIds) {
                String hwid = serialNumber + "." + funid;
                if (!_sensors_hwid.keySet().contains(hwid)) {
                    YSensor sensor = YSensor.FindSensor(hwid);
                    sensor.set_logFrequency("12/h");
                    SensorCache sensorCache = new SensorCache(sensor);
                    _sensors_hwid.put(hwid, sensorCache);
                }
            }
            if (functionIds.size() > 0) {
                String hwid = serialNumber + ".dataLogger";
                YDataLogger dataLogger = YDataLogger.FindDataLogger(hwid);
                dataLogger.set_autoStart(YDataLogger.AUTOSTART_ON);
                dataLogger.set_recording(YDataLogger.RECORDING_ON);
                module.saveToFlash();
            }
            functionIds = module.get_functionIds("Display");
            for (String funid : functionIds) {
                String hwid = serialNumber + "." + funid;
                if (!_displays_hwid.contains(hwid)) {
                    YDisplay display = YDisplay.FindDisplay(hwid);
                    display.resetAll();
                    _displays_hwid.add(hwid);
                }
            }
        } catch (YAPI_Exception e) {
            e.printStackTrace();
        }
    }

    ...
}

 



The remainder of the method is a simple loop which updates the connected Yocto-MaxiDisplay every second. The data to be displayed are retrieved directly from the data logger with the sensor method get_recordedData().


public void refresh() throws YAPI_Exception
{
    _currentValue = _ysensor.get_currentValue();
    _max = _ysensor.get_highestValue();
    _min = _ysensor.get_lowestValue();
    _logicalName = _ysensor.get_logicalName();
    YDataSet dataset = _ysensor.get_recordedData(0, 0);
    dataset.loadMore();
    _summary = dataset.get_summary();
    int progress = 0;
    do {
        progress = dataset.loadMore();
    } while (progress < 100);
    _details = dataset.get_measures();
}
 


The WebsocketEndpoint class has only one purpose: it must pass the Session object from the incoming connections to the WorkerThread so that it calls the PreregisterHubWebsocketCallback(session) method to make the modules of this connection available to the Yoctopuce API. In the same way, at the end of the connection, it must pass the Session object to the WorkerThread for it to call the
UnregisterHubWebSocketCallback(session)
method.


@ServerEndpoint("/callback")
public class WebSocketEndpoint
{
    @OnOpen
    public void onOpen(Session session)
    {
        System.out.println("WS Open " + session.getId());
        WorkerThread wt = WorkerThread.getInstance();
        wt.addSession(session);
    }

    @OnClose
    public void onClose(Session session)
    {
        System.out.println("WS Close " + session.getId());
        WorkerThread wt = WorkerThread.getInstance();
        wt.removeSession(session);
    }
}
 


The SensorCache class isn't of great interest, it is used to store data and other information downloaded from the data logger.

As usual, the source code of the application is available on GitHub:
http://github.com/yoctopuce-examples/wsdatalogger

Conclusion


As we have seen it, it is very simple to use the data logger in a WebSocket callback. The library works as if it had performed the connection itself.

In this application, we used the data logger to avoid using a database, but a more realistic use would be to have the data logger as a backup solution for cases in which the server is down for maintenance or when the Internet connection fails. In this scenario, when connecting again, the application could retrieve the missing data by using the data logger.

Add a comment No comment yet
Back to blog












Yoctopuce, get your stuff connected.