Performance gains with goroutines
Programming Snapshot – goroutines
In the Go language, program parts that run simultaneously synchronize and communicate natively via channels. Mike Schilli whips up a parallel web fetcher to demonstrate the concept.
I often wonder why some developers seem committed to designing new programming languages. Of course, the young guns today are all hungry for slight improvements in the syntax, while hipsters enthuse over smart ideas for compact code. But the effort of building an ecosystem and setting up a community is immense!
Alas, since processors stopped running faster every year some time ago and only simulate more speed with cores running in parallel, one thing is very important: Your choice of language has to be able to coordinate parallel program parts easily. When I visited the WhatsApp team at Facebook in Menlo Park after work a few months ago, I learned what the secret of the small team's success was when they used a handful of machines to text millions of users. They used the old-fashioned Erlang language, which has parallelism as a native feature.
It's the same with Go. The smart people at Google have not only built process management and threading into the programming language, but also have added new primitives such as goroutines and channels, thus not only making concurrency available, but an integral part of the language.
All Inclusive
To prepare my experiment, Listing 1 first builds a little helper, a library for easy web access [1]. Users later simply call httpsimple.Get()
and receive a success or error code, as well as the text of the retrieved web page. The Get()
function is given an initial capital so that external clients can use it from the package later, as required by Go. As the declaration in line 10 shows, it accepts a URL of the string
type as an argument and returns two values: the result string with the content of the web page and an error value, which is set to nil
in case of successful access.
Listing 1
httpsimple.go
01 package httpsimple 02 import( 03 "fmt" 04 "net/http" 05 "io/ioutil" 06 "time" 07 "errors" 08 ) 09 10 func Get(url string) (string, error) { 11 tr := &http.Transport{ 12 IdleConnTimeout: 30 * time.Second, 13 } 14 client := &http.Client{Transport: tr} 15 resp, err := client.Get(url) 16 17 if err != nil { 18 fmt.Printf("%s\n", err) 19 return "", err 20 } 21 22 if resp.StatusCode != 200 { 23 return "", errors.New(fmt.Sprintf( 24 "Status %v", resp.StatusCode)) 25 } 26 27 defer resp.Body.Close() 28 body, err := ioutil.ReadAll(resp.Body) 29 if err != nil { 30 fmt.Printf("I/O Error: %s\n", err) 31 return "", err 32 } 33 34 return string(body), nil 35 }
Go aficionados can simply create web clients from the net/http
core package with http.Get()
, but for parallel access, the client should also be able to pull the ripcord in case of hanging web pages or sluggish data traffic. According to reports [2], the default client is not suitable for this, therefore lines 11-13 in Listing 1 define a transport that sets the timeout to 30 seconds. And it's great that the client can also speak HTTPS, as if it were the most natural thing in the world!
The rest of Listing 1 is used for error handling, checking the status code (which should be 200
), and requesting and reading the web page text arriving via the socket. The function returns the empty string as the result and an error code in the event of a premature termination. In lines 19 and 31, it only passes on the error values provided by the core libraries net/http
and io/ioutil
, while in lines 22-25, it even compiles a new error type if it receives a status message other than 200
from the web server.
If all goes well, line 34 returns the page text converted to a string and the error value nil
to the caller. For a client to find the helper later on, I have to copy the Go code from Listing 1 into a new directory ~/go/src/httpsimple
and then compile it there using go install
to make it available as a library for other Go code.
One by One
Listing 2 now calls the web servers of some large US companies, one after the other, and fetches their homepages with the new httpsimple
library [3]. To do this, it defines an array of strings with their URLs in lines 9-13 and iterates over this in a for
loop from line 15. Instead of outputting the whole mess of incoming web data, it uses len()
to determine the data length and outputs it for illustrative purposes. Figure 1 shows the call to the compiled binary (created via go build http-serial.go
), wrapped with the command-line timer time
, revealing that the whole action takes a little over two seconds.
Listing 2
http-serial.go
01 package main 02 03 import( 04 "fmt" 05 "httpsimple" 06 ) 07 08 func main() { 09 urls := []string{ 10 "https://google.com", 11 "https://facebook.com", 12 "https://yahoo.com", 13 "https://apple.com"} 14 15 for _, url := range urls { 16 body, err := httpsimple.Get(url) 17 if err == nil { 18 fmt.Printf("%s: %d bytes\n", 19 url, len(body)) 20 } 21 } 22 }

How could this data retrieval be accelerated? The web client is by no means fully loaded but waits patiently until the web server finally serves up the data; this wait must feel pretty much like an eternity to a fast CPU. It would be more effective if the web client were to send the requests to all four web servers at once and then collect the incoming data as it trickles in. This could be done either with several parallel running processes, with lightweight threads, or with an event loop, as in Node.js, for example.
Own Soup
In addition to the above, Go offers goroutines as a concurrency primitive. Their lifetime is planned and executed by the Go runtime. They are even more lightweight than threads, since several goroutines share one thread. The go
keyword – followed by a function call – starts off a parallel goroutine in the background, executing the function, but also jumps to the next line to continue executing the main program. Nice! However, if you execute the example program below, with only a few calls to goroutines that output one letter at a time,
go fmt.Println("a") go fmt.Println("b") go fmt.Println("c")
you will be surprised that nothing appears at all on standard output while the program runs and then ends abruptly! The reason for this is that although Go starts the three routines in parallel, it closes the main program so quickly that none of the spawned program flows reaches its Println()
command.
An interesting race condition occurs when I add a Sleep
statement from the time
package, delaying the program end by a few microseconds in line 10 of Listing 3. The output of the program then varies between nothing, one, two, or three letters, depending on how far the program gets in the given time, but this is obviously not deterministic (Figure 2).
Listing 3
racecond.go
01 package main 02 import "fmt" 03 import "time" 04 05 func main() { 06 go fmt.Println("a") 07 go fmt.Println("b") 08 go fmt.Println("c") 09 // unreliable! 10 time.Sleep( 50 * time.Microsecond ) 11 }
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.
News
-
The GNU Project Celebrates Its 40th Birthday
September 27 marks the 40th anniversary of the GNU Project, and it was celebrated with a hacker meeting in Biel/Bienne, Switzerland.
-
Linux Kernel Reducing Long-Term Support
LTS support for the Linux kernel is about to undergo some serious changes that will have a considerable impact on the future.
-
Fedora 39 Beta Now Available for Testing
For fans and users of Fedora Linux, the first beta of release 39 is now available, which is a minor upgrade but does include GNOME 45.
-
Fedora Linux 40 to Drop X11 for KDE Plasma
When Fedora 40 arrives in 2024, there will be a few big changes coming, especially for the KDE Plasma option.
-
Real-Time Ubuntu Available in AWS Marketplace
Anyone looking for a Linux distribution for real-time processing could do a whole lot worse than Real-Time Ubuntu.
-
KSMBD Finally Reaches a Stable State
For those who've been looking forward to the first release of KSMBD, after two years it's no longer considered experimental.
-
Nitrux 3.0.0 Has Been Released
The latest version of Nitrux brings plenty of innovation and fresh apps to the table.
-
Linux From Scratch 12.0 Now Available
If you're looking to roll your own Linux distribution, the latest version of Linux From Scratch is now available with plenty of updates.
-
Linux Kernel 6.5 Has Been Released
The newest Linux kernel, version 6.5, now includes initial support for two very exciting features.
-
UbuntuDDE 23.04 Now Available
A new version of the UbuntuDDE remix has finally arrived with all the updates from the Deepin desktop and everything that comes with the Ubuntu 23.04 base.