How to handle a serial protocol automatically

How to handle a serial protocol automatically

Yoctopuce serial communication modules are more than simple interfaces: they are able to query and analyze autonomously the data coming from any device to then present the results in the way of a Yoctopuce sensor and/or record them in the embedded data logger. But in order to do so, you must tell the Yoctopuce module how to manage the dialog with the device. So here are a few examples as a complement to the documentation, to make this task easier.


For most standard scenarios, the job creation wizard of the Yocto-RS232, Yocto-Serial, Yocto-RS485, and Yocto-SPI directly offers a dedicated interface. For example, the Yocto-RS232 offers the following interface to configure NMEA message decoding:

Simplified interface to configure message decoding
Simplified interface to configure message decoding


In this article, we will explain how to deal with the cases which are not covered by the dedicated interfaces, and for which you need to specify the message format.

Decoding messages sent by the device


To begin, we are going to see how to teach the Yoctopuce module to decode any message. An instructive way to do so is to look at how the standard scenarios we mentioned above do it. To do so, define a task by using the simplified interface, reopen it and select the Use a custom protocol option. In this way, commands are displayed which could have been used to manually define the task.

For example, if you define a task which analyzes two values separated by commas (CSV format), you obtain the following protocol:

expect "($1:FLOAT)[,]($2:FLOAT).*"


The expect command is used to recognize and decode received messages corresponding to a specific format. The elements which interest us particularly are the groups within the parentheses. They correspond to the values which must be read and assigned to the genericSensors. The $1, $2, ... indicates to which genericSensor the value must be assigned. FLOAT indicates the type of the expected value, which influences both the accepted characters for the expression and the decoding which is performed.

The whole command is therefore interpreted as follows:

  • ($1:FLOAT): read a floating point number and assign it to genericSensor1
  • [,]: wait for a comma (square brackets are actually superfluous here)
  • ($2:FLOAT): read a floating point number and assign it to genericSensor2
  • .*: ignore the remainder of the line


Let's take a more complex example, for example decoding an NMEA message given as an example in the above illustration. We see that the following command is used:

expect "$GPGGA,[0-9.]*,($1:DDM),($2:DDM),[1-3],($5:INT),($3:FLOAT),($4:FLOAT),.+"


To better understand the string defining the format, here is a NMEA message example which is recognized by this string:

$GPGGA,101558,3852.1553,N,07703.2147,W,1,14,1.5,345.6,M,46.9,M,,*47


It is interpreted as follows:

  • $GPGGA: identifies a message providing a location
  • 101558: message time stamp (10:15:58 UTC)
  • 3852.1553,N: latitude, in degrees and minutes
  • 07703.2147,W: longitude, in degrees and minutes
  • 1: determination method (1=GPS, 2=DGPS, 3=PPS)
  • 14: number of tracked satellites
  • 1.5: horizontal dilution of the measure
  • 345.6,M: altitude in meters above sea level
  • ...: the remainder of the message is of no interest to us


If we look again at the decoding string, we see that it works on the principle of pattern matching: as long a literal text patterns correspond, the message is accepted. Expressions within square brackets allow you to define a set of authorized characters, and the star indicates iteration at will, as in classic regular expressions.

This decoding string uses other value types besides FLOAT. So you have here for reference the exhaustive list of decoding types to this day. You can also find it in the documentation:

  • ($x:INT) allows you to recognize an integer value (base 10) and assigned it to the corresponding genericSensorX function. For example, {$3:INT} allows you to recognize an integer value and to assign it to the genericSensor3 function.
  • ($x:FLOAT) allows you to recognize a decimal value (floating point number) that is assigned to the genericSensorX. function. Scientific notation (for example 1.25e-1) is supported.
  • ($x:DDM) allows you to recognize a decimal value in degree-minute-decimal format as used by NMEA standard
  • ($x:BYTE) allows you to recognize a value between 0 and 255 represented in hexadecimal, as for instance when using binary protocols. If the value is in the range -128...127, you should use ($x:SBYTE) instead (where the S stands for signed byte. The decoded value is assigned to the genericSensorX.
  • ($x:WORD) or ($x:SWORD) allows you similarly to decode a 16-bit hexadecimal value, unsigned or signed, and assign the result to the genericSensorX. The byte order is expected to be the natural written byte order (big-endian), i.e. the most significant byte first like in 0104 to represent 260.
  • ($x:WORDL) or ($x:SWORDL) allows you to decode in the same way a 16-bit hexadecimal unsigned or signed value with the bytes in little-endian order, i.e. with the least significant byte first (eg. 0401 to represent 260).
  • ($x:DWORD) or ($x:SDWORD) allows you to decode in the same way a 32-bit hexadecimal unsigned or signed value in big-endian byte order.
  • ($x:DWORDL) ou ($x:SDWORDL) allows you to decode in the same way a 32-bit hexadecimal unsigned or signed value in little-endian byte order.
  • ($x:DWORDX) ou ($x:SDWORDX) allows you to decode in the same way a 32-bit hexadecimal unsigned or signed value in mixed-endian byte order: two 16-bit words, each of them in big-endian, but the least significant 16-bit word is put first and the most significant word is put second.
  • ($x:HEX) allows you to decode a variable length hexadecimal number (1 to 4 byes in big-endian order), that is assigned to the genericSensorX.
  • ($x:FLOAT16B) and ($x:FLOAT16L) allows you to decode a 16-bit floating point number encoded in hexadecimal according to IEEE 754, using big-endian or little-endian byte order.
  • ($x:FLOAT16D) allows you to decode a floating point number encoded in hexadecimal on two bytes, with the first byte representing the mantissa and the second byte representing the decimal signed exponent.
  • ($x:FLOAT32B) and ($x:FLOAT32L) allows you to decode a 32-bit floating point number encoded in hexadecimal according to IEEE 754, using big-endian or little-endian byte order.
  • ($x:FLOAT32X) allows you to decode a 32-bit floating point number encoded in hexadecimal according to IEEE 754, using mixed-endian byte order, i.e. two 16-bit words, each of them in big-endian, but the least significant 16-bit word is put first and the most significant word is put second.


As the internal representation of floating point numbers in Yoctopuce devices is limited to 3 decimals, it is possible to change the magnitude of floating point numbers decoded by FLOAT, FLOAT16 and FLOAT32 expressions by prefixing them with a M to count in thousandths, or a U to count in millionths (U like micro), or a N to count in billionths (N like nano). For instance, when the value 1.3e-6 is recognized with expression ($1:UFLOAT), the value assigned to genericSensor3 is 1.3.

Automatic interactions


Often, it's not enough to wait for message to be sent to the device, you must request them. You can add commands which send messages on the serial port. For example, we could define the following task to establish a communication through a serial modem:

writeLine "AT"
expect "OK"
writeLine "ATDT0123456789"
expect "CONNECT"
writeLine "Hello world"
wait 1000
write "+++"
wait 1000
writeLine "ATH"
expect "OK"


This task sends the AT command, waits for an OK answer. If the OK answer doesn't come, the task starts again later from the beginning. Otherwise, the following command ATDT0123456789 is sent, and the tasks waits for the CONNECT answer, and so on. Note that you can add delays between sending operations with the wait command, with a delay given in milliseconds.

Another example: let's try to see now how reading a MODBUS register works. It's typically a task requiring an interaction, as MODBUS devices don't spontaneously send measures but wait for reading commands. For example, if we create a task to read register 40008 and that we reopen it as a custom protocol, we obtain the following commands:

writeMODBUS "010300070001"
expect ":010302($1:WORD).*"


In looking on the internet at the MODBUS Application Protocol Specification document, you find the syntax of MODBUS messages which will enable you to understand this protocol. The sent MODBUS command format is the following:

  • 01: message addressed to device 01
  • 03: command "Read MODBUS register"
  • 00007: register number (0 for 40001, 1 for 40002, and so on.)
  • 00001: number of registers to be read

The format of the expected answer is the following:

  • :01: answer of MODBUS device 01
  • 03: answer to a "Read MODBUS register" command
  • 02: number of reply bytes in the answer (2 bytes per register)
  • xxxx: value of the register (on 2 bytes)
  • ...: an error detection code, that we can ignore


Note that as it is not a text protocol but a binary one, received messages are automatically converted in hexadecimal before being interpreted by the expect command, to make their processing easier.

We now easily see how this task could be modified to read two registers at a time and to interpret them as a floating number on 32 bits, for example. We only have to change the number of registers to be read in the writeMODBUS command, and to change the length of the expected answer and its decoding:

writeMODBUS "010300070002"
expect ":010304($1:FLOAT32).*"


We can naturally read several separate registers in the same task by adding other writeMODBUS and expect commands.

Complex dialogs


In some relatively rare cases, the format of the answer of the device is not always the same and you can therefore not use the simple expect to analyze it. you need several distinct ones which work in parallel. This might be the case for example when a device can answer try again or give an expected answer.

Generally, the best way to solve the issue is to create several distinct tasks:

  1. A periodic task which sends the initial command
  2. A reactive task which manages the try again answer
  3. A second reactive task which manages the answer including the measure

This works because reactive tasks can be triggered at any moment, as soon as the expect command with which they start is satisfied by a received message.

On the other hand, to make sure they don't interfere with one another, periodic tasks don't work in parallel: whatever the configured period, a periodic task never starts before the end to the preceding periodic task.

Conclusion


With these explanations, you should have everything to write your own system to manage personalised protocols. It may require a few trials, but by observing the dialog in the communication window or through a small application provided in a previous post, you should manage. And naturally, Yoctopuce support is always present to help you in case of difficulty.

Add a comment No comment yet Back to blog












Yoctopuce, get your stuff connected.