Detect and restart hanging programs with Go

Programming Snapshot – Automated Restarts with Go

© Lead Image © xyzproject, 123RF.com

© Lead Image © xyzproject, 123RF.com

Article from Issue 261/2022
Author(s):

Detecting programs where the standard output has frozen can require a deep dive into terminal emulation basics. Go plumber Mike Schilli builds a plunger to free up the pipe works.

If the browser stops halfway while loading a website and then freezes, experienced users know that usually all it takes is clicking the reload button to make it work like clockwork on the next attempt. Or, if an rsync transfer suddenly stalls because the server has fallen asleep, admins will intuitively press Ctrl+C to abort, only to immediately restart the command and see it finish without a hitch in most cases. Scenarios where humans have to manually take control over running processes just because the computer fails to understand that an automatic restart would solve the problem are one of the last hurdles to a fully automated world.

The yoyo Go program presented in this issue supervises programs entrusted to it and will rejigger them like a yo-yo (as you know, yo-yos also need to be pulled up by a human hand to keep them moving). To do this, the utility monitors a supervised program's standard output (along with its standard error), which typically features frequently changing patterns – such as a progress bar that indicates that something is still happening. If the display freezes, for example, because of a network outage or because the server has lost interest, yoyo detects this and restarts the program after a configurable timeout, in hopes of somehow fixing the problem this way.

Feels Like a Terminal

Easy, right? But programs behave differently, depending on whether or not they think they are running in a terminal. Typing git push in a terminal, for example, continuously outputs the transfer progress as a percentage. And this gives the calling user, especially when they need to commit large files, some idea of how long the whole process is going to take (Figure 1, top).

[...]

Use Express-Checkout link below to read the full article (PDF).

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Command Line: Processes

    Innumerable processes may be running on your Linux system. We’ll show you how to halt, continue, or kill tasks, and we’ll examine how to send the remnants of crashed programs to the happy hunting grounds.

  • Command Line: Process Control

    What is happening on your Linux machine? Various shell commands give you details about system processes and help you control them.

  • System Monitoring Center

    The System Monitoring Center combines all the important information you need to monitor a computer in a single state-of-the-art interface.

  • Elixir 1.0

    Developers will appreciate Elixir's ability to build distributed, fault-tolerant, and scalable applications.

  • Perl: Network Monitoring

    To discover possibly undesirable arrivals and departures on their networks, a Perl daemon periodically stores the data from Nmap scans and passes them on to Nagios via a built-in web interface.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More

News