Performance gains with goroutines
Programming Snapshot – goroutines
In the Go language, program parts that run simultaneously synchronize and communicate natively via channels. Mike Schilli whips up a parallel web fetcher to demonstrate the concept.
I often wonder why some developers seem committed to designing new programming languages. Of course, the young guns today are all hungry for slight improvements in the syntax, while hipsters enthuse over smart ideas for compact code. But the effort of building an ecosystem and setting up a community is immense!
Alas, since processors stopped running faster every year some time ago and only simulate more speed with cores running in parallel, one thing is very important: Your choice of language has to be able to coordinate parallel program parts easily. When I visited the WhatsApp team at Facebook in Menlo Park after work a few months ago, I learned what the secret of the small team's success was when they used a handful of machines to text millions of users. They used the old-fashioned Erlang language, which has parallelism as a native feature.
It's the same with Go. The smart people at Google have not only built process management and threading into the programming language, but also have added new primitives such as goroutines and channels, thus not only making concurrency available, but an integral part of the language.
All Inclusive
To prepare my experiment, Listing 1 first builds a little helper, a library for easy web access [1]. Users later simply call httpsimple.Get()
and receive a success or error code, as well as the text of the retrieved web page. The Get()
function is given an initial capital so that external clients can use it from the package later, as required by Go. As the declaration in line 10 shows, it accepts a URL of the string
type as an argument and returns two values: the result string with the content of the web page and an error value, which is set to nil
in case of successful access.
Listing 1
httpsimple.go
01 package httpsimple 02 import( 03 "fmt" 04 "net/http" 05 "io/ioutil" 06 "time" 07 "errors" 08 ) 09 10 func Get(url string) (string, error) { 11 tr := &http.Transport{ 12 IdleConnTimeout: 30 * time.Second, 13 } 14 client := &http.Client{Transport: tr} 15 resp, err := client.Get(url) 16 17 if err != nil { 18 fmt.Printf("%s\n", err) 19 return "", err 20 } 21 22 if resp.StatusCode != 200 { 23 return "", errors.New(fmt.Sprintf( 24 "Status %v", resp.StatusCode)) 25 } 26 27 defer resp.Body.Close() 28 body, err := ioutil.ReadAll(resp.Body) 29 if err != nil { 30 fmt.Printf("I/O Error: %s\n", err) 31 return "", err 32 } 33 34 return string(body), nil 35 }
Go aficionados can simply create web clients from the net/http
core package with http.Get()
, but for parallel access, the client should also be able to pull the ripcord in case of hanging web pages or sluggish data traffic. According to reports [2], the default client is not suitable for this, therefore lines 11-13 in Listing 1 define a transport that sets the timeout to 30 seconds. And it's great that the client can also speak HTTPS, as if it were the most natural thing in the world!
The rest of Listing 1 is used for error handling, checking the status code (which should be 200
), and requesting and reading the web page text arriving via the socket. The function returns the empty string as the result and an error code in the event of a premature termination. In lines 19 and 31, it only passes on the error values provided by the core libraries net/http
and io/ioutil
, while in lines 22-25, it even compiles a new error type if it receives a status message other than 200
from the web server.
If all goes well, line 34 returns the page text converted to a string and the error value nil
to the caller. For a client to find the helper later on, I have to copy the Go code from Listing 1 into a new directory ~/go/src/httpsimple
and then compile it there using go install
to make it available as a library for other Go code.
One by One
Listing 2 now calls the web servers of some large US companies, one after the other, and fetches their homepages with the new httpsimple
library [3]. To do this, it defines an array of strings with their URLs in lines 9-13 and iterates over this in a for
loop from line 15. Instead of outputting the whole mess of incoming web data, it uses len()
to determine the data length and outputs it for illustrative purposes. Figure 1 shows the call to the compiled binary (created via go build http-serial.go
), wrapped with the command-line timer time
, revealing that the whole action takes a little over two seconds.
Listing 2
http-serial.go
01 package main 02 03 import( 04 "fmt" 05 "httpsimple" 06 ) 07 08 func main() { 09 urls := []string{ 10 "https://google.com", 11 "https://facebook.com", 12 "https://yahoo.com", 13 "https://apple.com"} 14 15 for _, url := range urls { 16 body, err := httpsimple.Get(url) 17 if err == nil { 18 fmt.Printf("%s: %d bytes\n", 19 url, len(body)) 20 } 21 } 22 }
How could this data retrieval be accelerated? The web client is by no means fully loaded but waits patiently until the web server finally serves up the data; this wait must feel pretty much like an eternity to a fast CPU. It would be more effective if the web client were to send the requests to all four web servers at once and then collect the incoming data as it trickles in. This could be done either with several parallel running processes, with lightweight threads, or with an event loop, as in Node.js, for example.
Own Soup
In addition to the above, Go offers goroutines as a concurrency primitive. Their lifetime is planned and executed by the Go runtime. They are even more lightweight than threads, since several goroutines share one thread. The go
keyword – followed by a function call – starts off a parallel goroutine in the background, executing the function, but also jumps to the next line to continue executing the main program. Nice! However, if you execute the example program below, with only a few calls to goroutines that output one letter at a time,
go fmt.Println("a") go fmt.Println("b") go fmt.Println("c")
you will be surprised that nothing appears at all on standard output while the program runs and then ends abruptly! The reason for this is that although Go starts the three routines in parallel, it closes the main program so quickly that none of the spawned program flows reaches its Println()
command.
An interesting race condition occurs when I add a Sleep
statement from the time
package, delaying the program end by a few microseconds in line 10 of Listing 3. The output of the program then varies between nothing, one, two, or three letters, depending on how far the program gets in the given time, but this is obviously not deterministic (Figure 2).
Listing 3
racecond.go
01 package main 02 import "fmt" 03 import "time" 04 05 func main() { 06 go fmt.Println("a") 07 go fmt.Println("b") 08 go fmt.Println("c") 09 // unreliable! 10 time.Sleep( 50 * time.Microsecond ) 11 }
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Halcyon Creates Anti-Ransomware Protection for Linux
As more Linux systems are targeted by ransomware, Halcyon is stepping up its protection.
-
Valve and Arch Linux Announce Collaboration
Valve and Arch have come together for two projects that will have a serious impact on the Linux distribution.
-
Hacker Successfully Runs Linux on a CPU from the Early ‘70s
From the office of "Look what I can do," Dmitry Grinberg was able to get Linux running on a processor that was created in 1971.
-
OSI and LPI Form Strategic Alliance
With a goal of strengthening Linux and open source communities, this new alliance aims to nurture the growth of more highly skilled professionals.
-
Fedora 41 Beta Available with Some Interesting Additions
If you're a Fedora fan, you'll be excited to hear the beta version of the latest release is now available for testing and includes plenty of updates.
-
AlmaLinux Unveils New Hardware Certification Process
The AlmaLinux Hardware Certification Program run by the Certification Special Interest Group (SIG) aims to ensure seamless compatibility between AlmaLinux and a wide range of hardware configurations.
-
Wind River Introduces eLxr Pro Linux Solution
eLxr Pro offers an end-to-end Linux solution backed by expert commercial support.
-
Juno Tab 3 Launches with Ubuntu 24.04
Anyone looking for a full-blown Linux tablet need look no further. Juno has released the Tab 3.
-
New KDE Slimbook Plasma Available for Preorder
Powered by an AMD Ryzen CPU, the latest KDE Slimbook laptop is powerful enough for local AI tasks.
-
Rhino Linux Announces Latest "Quick Update"
If you prefer your Linux distribution to be of the rolling type, Rhino Linux delivers a beautiful and reliable experience.