Managing power outages

Managing power outages

Here in Switzerland, there is a risk of energy shortages during next Winter, and we can't exclude having scheduled or unscheduled power outages. We took this opportunity to build a small device to help up manage the consequences of potential power outages.





At Yoctopuce, many tasks are automated with computers scattered everywhere, and after a power outage, we must turn them back on one by one without forgetting any. This week's project is thus to build a box which indicates the state of each machine. If, on top of this, the box enabled us to perform a clean shutdown on all the machines, this would allow us to easily power off all of our machines before scheduled power outages.

Principle

The hardware part is dead easy, as we only need a panel of leds and a big button, so we need:

Principle diagram
Principle diagram



The principle consists in running on each computer a small script which takes control of the box and lights in green the led corresponding to it. If for one reason or another the script stops, the said led automatically switches to red. If the script detects that the box switch was engaged, the system sets the led to orange and triggers a machine shutdown. When the computer is off, the led goes off as well.


The electronic part  The raw face
The said box
The said box


Programming

The somewhat complex part of this project is script programming. There are several aspects which must be properly managed. We decided to write a first version of the script in Python because it's a language which is easily available under Windows, Linux and MacOS.

Managing connections

The YoctoHub-Ethernet can manage only a limited number of connections in parallel. The script must therefore connect to the box, do what it has to do as fast as possible before closing the connection, then wait a more or less random time before starting again. Waiting for a random time prevents all the machines from trying to connect to the box at once.
The structure of the code thus looks like this:

  while True:
    if (YAPI.TestHub(CONTROLER_ADDR, 1000, errmsg) != YAPI.SUCCESS):
        print("cannot reach " + CONTROLER_ADDR)
        YAPI.FreeAPI()
    else:
      try:
        if (YAPI.RegisterHub(CONTROLER_ADDR, errmsg) == YAPI.SUCCESS):

          ##  payload

          YAPI.FreeAPI()
      except Exception as e:
        print("Error " + str(e))
    t = random.randint(SLEEP_AVG - 10, SLEEP_AVG + 10)
    print("sleeping for " + str(t) + "sec")
    time.sleep(t)



Configuring the system

You must store somewhere which computer corresponds to which led. We could provide this parameter to each script instance thanks to the command line, but we thought it more appropriate to centralize this information. It so happens that the YoctoHub-Ethernet contains a small file system in which we can save a configuration file. We selected a very simple format where each line corresponds to a computer in the shape LED index=Addresse IP, for example:

1=192.168.1.43 2=192.168.1.72 3=192.168.1.88


As we would like to launch the scripts again after each configuration change while parsing the configuration file only when necessary, we used the fact that the YoctoHub-Ethernet can compute the CRC of each file to detect changes.

        myIPAddr = socket.gethostbyname(socket.gethostname())
        files = YFiles.FirstFiles()
        if files is None:
          sys.exit("No Files on " + CONTROLER_ADDR)
        fileslist = files.get_list(CONFIG_FILE)
        if len(fileslist) > 0:
          if (fileslist[0].get_name() == CONFIG_FILE and fileslist[0].get_crc() != lastCRC ):
            ledIndex = -1
            lastCRC = fileslist[0].get_crc()
            config = files.download(CONFIG_FILE).decode('utf-8')
            lines = config.split()
            for line in lines:
              tokens = line.split("=")
              if len(tokens) == 2:
                if tokens[1] == myIPAddr:
                  ledIndex = int(tokens[0])
            if (ledIndex < 0):
              print("No Ledindex for " + myIPAddr + " in config.txt on " + CONTROLER_ADDR)
            else:
              print("LED index=" + str(ledIndex))
        else:
          print("No config.txt file on " + CONTROLER_ADDR)



Note that on computers with several network adapters, we don't know which IP address your script will detect first. It's not a big issue as long as nothing prevents you from defining several lines for the same led in the configuration file. As long as all the addresses correspond to the same computer, the system will work.

Managing the leds

The main aim of the script is to switch to green the corresponding led on the face of the box, but we also want this led to automatically switch to red if the script stops, typically because the computer shut down unexpectedly. Similarly, when the script detects a shutdown request, it must switch the led to orange, and then automatically turn it off when the script stops because the computer is effectively off.

To make sure that a led automatically changes color when its controlling script has stopped, we use again the Yocto-Color animation trick. It uses the fact that Yocto-Color animations run in a loop, unless they contain an instruction to jump which enables them to go to another animation. So we pre-program two animations:

  • The first one maintains the first color, for example green, for a set period of time, and then jumps to the second animation
  • The second animation maintains the second color, for example red, in a loop.

As long as the script is running, the led is periodically reassigned to the first animation before it has time to jump to the second animation. If the script stops, the led ends up jumping to the second animation and changes color on its own.

You can define animations once and for all and save them on the Yocto-Color system file. But we used a slightly different approach: the script checks at start up if animations are already defined in the Yocto-Color memory and creates them if need be. Thus, if we want to change the animations, we only need to change the script and to restart the module and then the script for the animations to be updated.

And while we are at it, initialization defines the led start up color to red, if not already done. Thus, when power is back on after an outage, all the leds used are red by default.

    cluster = YColorLedCluster.FirstColorLedCluster()
      if cluster is None:
        sys.exit("No ColorLedCluster on " + CONTROLER_ADDR)

      if FirstTime:
        signatures = cluster.get_blinkSeqSignatures(0, 5)

        if (signatures[GREENSEQ] == 0):
          cluster.resetBlinkSeq(GREENSEQ)
          cluster.addRgbMoveToBlinkSeq(GREENSEQ, GREEN, 0)
          cluster.addRgbMoveToBlinkSeq(GREENSEQ, GREEN, DELAY)
          cluster.addJumpToBlinkSeq(GREENSEQ, REDSEQ)
          cluster.startBlinkSeq(GREENSEQ)


        if (signatures[REDSEQ] == 0):
          cluster.resetBlinkSeq(REDSEQ)
          cluster.addRgbMoveToBlinkSeq(REDSEQ, RED, 0)
          cluster.addRgbMoveToBlinkSeq(REDSEQ, RED, 10000)
          cluster.startBlinkSeq(REDSEQ)

        ## define other sequences ...



      if cluster.get_rgbColorArrayAtPowerOn(ledIndex, 1)[0] != RED:
        cluster.set_rgbColorAtPowerOn(ledIndex, 1, RED)
        cluster.saveLedsConfigAtPowerOn()



Managing the shutdown

Managing the shutdown first consists in checking that the switch is not already engaged when the script starts up to avoid that a machine goes back off as soon as it is on because someone forgot to reset the switch to its proper position. When the shutdown command has been sent to the system, the script still runs, maintaining the led to orange. We count on the shutdown process to kill it. When the script is dead, the led goes off thanks to the animation mechanism. Note that the script probably needs adequate privileges to run a shutdown command.


  if (mustShutDown):
    if (ShutDownAlreadySetAtStartUp):
      print("Switch was left in shutdown position")
      cluster.linkLedToBlinkSeq(ledIndex, 1, ORANGEBLINKSEQ, 0)
    else:            
      cluster.linkLedToBlinkSeq(ledIndex, 1, ORANGESEQ, 0)
      if ShutdownAlreadySent:
        print("Shutdown in progress...")
      else:
        print("Initialing shutdown...")
        platformName = platform.system();
        #platformName = "DISABLE SHUTDOWN"
        if platformName == "Windows":
          os.system('shutdown -s')
        elif platformName == "Linux" :
          os.system('systemctl poweroff')
        elif platformName == "Darwin":  #Mac
          os.system("shutdown -h now")  # might not work if not root
        else:
          print("Unknown platform : " + platformName)
        ShutdownAlreadySent = True



Conclusion

We oversaw the most important features of this small application. If you want to know more, you can always download the complete source code here.

We are ready
We are ready



At one point, we could rewrite the script in C++ to make it a Windows service, which will be smarter than having a DOS window open all the time on the Windows machines managed by this system.

Note that if the box enables us to shut down all the computers remotely, it doesn't enable us to turn them back on. We may have the opportunity to talk about this again :-)

Add a comment No comment yet Back to blog












Yoctopuce, get your stuff connected.