Sending keystrokes from sensor data

Sending keystrokes from sensor data

The best way to use our devices is to leverage our programming API, available for most programming languages. But there is also a trick to use our sensors to interface existing applications for which you don't have the source code, by emulating keyboard input. If you are building a flight simulator deck or simply try to improve an old-fashioned data entry form interface, this is worth giving a look...



Principle

The idea is simple: we start as a background task a tiny program that reads the Yoctopuce sensors, and depending on the measure (and on the context), sends keystrokes to the active application.

The key to success is to use a reliable key sequence to send the information. For instance, if you want to automatically type in the temperature measured by a Yocto-PT100 into a form, you should start with keys ensures that the temperature will be typed in the right field. Similarly, in order to use a rotating knob connected to a Yocto-Knob instead of up/down arrow keys, it is best to handle the min/max positions of the knob explicitly using specific keystrokes.

Implementation

Emulating keystrokes is very OS-dependant. We will use Windows in this article, since this is still the most popular OS.

Under Windows, it is not possible to emulate keyboard input from a service, since services are not linked to the user session. It is however possible to emulate keyboard input using a system tray application running in background. This application can be automatically launched using the Startup Menu for instance.

We will implement this example in C#, combining various components found on the net:

The only part of the code that needs to be written from scratch is the logic that links sensors to keystrokes to be sent. Since this part depends very much on what you are trying to achieve, we won't write it for you. But to give you a start, here are two examples that can be used for the flight simulator deck.

Two examples

To keep the code simple and robust, we have implemented the logic using callbacks.

As a first example, we map one input of the Yocto-Knob (which reports a value between 0 and 1000) to the breaks command of X-Plane flight simulator. We enable breaks using Ctrl-B, and send B to toggle back off the breaks when they are set. The current state is saved in a global variable named currBreaks:

static void YBreaksCallback(YAnButton b, string value)
{
  int newBreaks = Convert.ToInt32(value) / 501;

  if (newBreaks == 1 && currBreaks != 1) {
    sim.Keyboard.KeyDown(VirtualKeyCode.LCONTROL);
    sim.Keyboard.KeyPress(VirtualKeyCode.VK_B);
    sim.Keyboard.KeyUp(VirtualKeyCode.LCONTROL);
    currBreaks = 1;
  }
  if (newBreaks == 0 && currBreaks != 0) {
    sim.Keyboard.KeyPress(VirtualKeyCode.VK_B);
    currBreaks = 0;
  }
}



Here is a second case, a bit more complex: handling X-Plane throttle using a knob potentiometer.
There are no keys to select a specific throttle values in X-Plane, but only two keys (F1 et F2) to reduce or increase throttle. Moreover, the magnitude of the change depends on the duration of the key press. It is therefore important to properly handle the end values of the potentiometer to make sure that no drift appears over time.

static void YThrottleCallback(YAnButton b, string value)
{
  int newThrottle = Convert.ToInt32(value) * throttleSteps / 1001;
  int delta = newThrottle - currThrottle;

  if (delta > 0) {
    if (newThrottle >= throttleSteps-1) {
      // min throttle, add extra steps to be sure
      delta += throttleSteps / 4;
    }
    sim.Keyboard.KeyDown(VK_MoreThrottle);
    sim.Keyboard.Sleep(30 * delta);
    sim.Keyboard.KeyUp(VK_MoreThrottle);
  }
  else if (delta < 0) {
    delta = -delta;
    if (newThrottle == 0) {
      // min throttle, add extra steps to be sure
      delta += throttleSteps / 4;
    }
    sim.Keyboard.KeyDown(VK_LessThrottle);
    sim.Keyboard.Sleep(30 * delta);
    sim.Keyboard.KeyUp(VK_LessThrottle);
  }
  currThrottle = newThrottle;
}



You will find a complete project for VisualStudio Express 2012 including these fonctions, with an additional example that maps the orientation of a Yocto-3D to arrow keys, to be used in a flash game for instance.

Getting further

If you intend to automate contextual interactions with an application (as to automate entries in a form), you can improve the reliability by using classes from System.Windows.Automation. This framework lets you know exactly which application and which input field has the input focus, and even to change it. It is therefore possible to make a Tray App that automatically enters data into a specific field when the user is prompted to fill it in, without any other interference on the application. To illustrate this API, here is another sample callback handler that automatically appends to a Notepad window the value of any Yoctopuce sensor each time that it changes, regardless of the active application.

static void SensorCallback(YSensor s, string value)
{
    AutomationElement notepad, textarea;    
    notepad = AutomationElement.RootElement.FindFirst(TreeScope.Children,
        new PropertyCondition(AutomationElement.ClassNameProperty, "Notepad"));
    if (notepad == null) return;
    textarea = notepad.FindFirst(TreeScope.Descendants,
        new PropertyCondition(AutomationElement.ClassNameProperty, "Edit"));
    if (textarea == null) return;
    try {
        // Save current focus position
        AutomationElement savedFocus = AutomationElement.FocusedElement;
        // Set focus to notepad
        textarea.SetFocus();
        Thread.Sleep(20);
        // Add text using Windows method (works well for standard applications)
        SendKeys.SendWait("^{END}"+s.get_hardwareId()+"="+value+"{Enter}");
        Thread.Sleep(20);
        // Restore focus
        savedFocus.SetFocus();
    } catch (Exception e) { };
}



And here is the result:

  

Add a comment No comment yet Back to blog












Yoctopuce, get your stuff connected.