Keep control of goroutines with a Context construct
Programming Snapshot – Go Context
When functions generate legions of goroutines to do subtasks, the main program needs to keep track and retain control of ongoing activity. To do this, Mike Schilli recommends using a Context construct.
Go's goroutines are so cheap that programmers like to fire them off by the dozen. But who cleans up the mess at the end of the day? Basically, Go channels lend themselves to communication. A main program may need to keep control of many goroutines running simultaneously, but a message in a channel only ever reaches one recipient. Consequently, the communication partners in this scenario rely on a special case.
If one or more recipients in Go try to read from a channel but block because there is nothing there, then the sender can notify all recipients in one fell swoop by closing the channel. This wakes up all the receivers, and their blocking read functions return with an error value.
This is precisely the procedure commonly used by the main program to stop subroutines that are still running. The main program opens a channel and passes it to each subroutine that it calls; the subroutines in turn attach blocking read functions to it. When the main program closes the channel later on, the blocks are resolved, and the subroutines proceed to do their agreed upon cleanup tasks – usually releasing all resources and exiting.
Ripcord on Three
As an intuitive example, Listing 1 shows a main program that calls the function work()
three times in quick succession. The function returns almost immediately, but each time it sets off a goroutine internally that counts to 10 in one second steps and prints the current counter value on standard output each time. Each of these goroutines would now continue to run for their entire scheduled 10 seconds, even after the function calling them had long terminated, if it weren't for the main program call to close()
, pulling the ripcord after three seconds in line 15.
Listing 1
grtest.go
01 package main 02 03 import ( 04 "fmt" 05 "time" 06 ) 07 08 func main() { 09 done := make(chan interface{}) 10 work(done) 11 work(done) 12 work(done) 13 14 time.Sleep(3 * time.Second) 15 close(done) 16 time.Sleep(3 * time.Second) 17 } 18 19 func work(done chan interface{}) { 20 go func() { 21 for i := 0; i < 10; i++ { 22 fmt.Printf("%d\n", i) 23 24 select { 25 case <-done: 26 fmt.Printf("Ok. I quit.\n") 27 return 28 case <-time.After(time.Second): 29 } 30 } 31 }() 32 }
The done
channel, used as a mechanism for this synchronization, is created in line 9. interface{}
specifies the data type transported in the channel as generic, since the program does not send any data to the channel or read from it later on, but will only detect a future close
command.
How exactly do the workers get to hear the factory siren at this point? After all, they are still busy doing their job and compiling results or just busily passing time, as in this example.
This "busy waiting" is implemented by the select
construct starting in line 24. Using two different case
statements, select
waits simultaneously for one of two possible events: the channel reader <-done
attempts to obtain data from the done
channel or it picks up an error if main
closes the channel. Or else the timer started by time.After()
in line 28 expires after one second and directs the select
construct to jump to the corresponding empty case, which does nothing except continue the surrounding endless for
loop.
During the first three seconds of running the sample program, it will show timer ticks from all three subroutines. But shortly after, things start happening, because the main program closes the done
channel, which triggers an error in the first case
in line 25; this in turn causes the worker to display the Ok. I quit. message and exit its goroutine with return
. Figure 1 shows the running program's output.
And Now for Real
Instead of counting to 10 and always waiting a second between each step, a work()
function in the real world would, for example, perform time-consuming tasks such as retrieving a web page over the network or using a tailf
-style technique to detect growth in one or more locally monitored files. However, even in these situations, a server program may have to pull the emergency brake – whether because the requesting user has lost patience or data processing on the back end is simply taking too long for the main program and it wants to move on to serve other requests.
At the end of a standalone main
program such as the one in Listing 1, the operating system does indeed clean up any goroutines that are still running and automatically releases the allocated resources such as memory or file handles. However, a server program must not rely on this luxury, because canceling a request – for whatever reason – does not terminate the program. It may have to continue running for weeks, without orphaned functions utilizing more and more memory, until the dreaded out-of-memory killer steps in and takes everything down.
Limits
By the way, functions that send data to each other via channels must be aware of two borderline cases. If they send a message to a channel that has already been closed, the Go program goes into panic mode and aborts with an error. On the other hand, if a function reads from a channel where nobody can send anything anymore because all writers have given up the ghost, the program flow will hang forever. In the case at hand, however, that's the intended behavior: No data will ever flow through the channel used, because the program only takes advantage of the fact that reading from a closed channel generates an error.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs