Create a bootable USB stick with terminal UI display
Programming Snapshot – Bootable USB with Go

© Lead Image © Pei Ling Hoo, 123RF.com
A Go program writes a downloaded ISO file to a bootable USB stick. To prevent it from accidentally overwriting the hard disk, Mike Schilli provides it with a user interface and security checks.
To test new Linux distributions on real hardware, a bootable USB stick with a downloaded image in ISO format will help with bootstrapping the installation. Rebooting the computer with the stick plugged in will often bring up a Live system that can be played around with to your heart's content, possibly after having to make some changes to the boot order in the BIOS.
How does the ISO file get onto the stick? Ultimately, this is done quite simply with a dd
command that expects the ISO file as the input (if
) and the device entry of the stick (for example /dev/sdd
) as the output (of
).
Tools like Ubuntu's Startup Disk Creator make things even more convenient with a graphical UI, but there are some concerns. Under no circumstances would you want the tool to have a bug that accidentally overwrites the next hard disk in the device tree instead of the stick.
How hard would it be to write a similar tool in Go – one that actively waits for the USB stick to be plugged in and then asks the user for permission to copy the ISO file to it? As always, by writing tools yourself, you'll learn a few new techniques that you might find helpful going forward to solve new everyday tasks.
The Art of Copying
It is not trivial to copy an ISO file weighing in at several gigabytes to another filesystem like the USB stick. Utilities like cp
and dd
do not read all the data in the source file from the disk in a single pass – this would take up a huge amount of precious RAM without shortening the process. Instead, such copy tools read the data from the source file, typically in megabyte-sized chunks, and keep writing them to the target file opened at the same time.
This is exactly what the code from Listing 1 does. The cpChunks()
function expects the names of the source and the target files as well as an open Go channel as parameters. The caller taps into the latter as a source of information to see how far the copy process has progressed. To do this, cpChunks()
sends a percentage value to the channel after each copied chunk, which indicates the fraction of the bytes already copied in relation to the total number. Starting off, the function obtains the total number of bytes to be copied via the os.Stat()
system function, which gets the source file size from the filesystem.
Listing 1
cpchunks.go
01 package main 02 03 import ( 04 "bufio" 05 "os" 06 "io" 07 ) 08 09 func cpChunks(src, dst string, percent chan<- int) error { 10 11 data := make([]byte, 4*1024*1024) 12 13 in, err := os.Open(src) 14 if err != nil { 15 return err 16 } 17 reader := bufio.NewReader(in) 18 defer in.Close() 19 20 fi, err := in.Stat() 21 if err != nil { 22 return err 23 } 24 25 out, err := os.OpenFile(dst, os.O_WRONLY, 0644) 26 if err != nil { 27 return err 28 } 29 writer := bufio.NewWriter(out) 30 defer out.Close() 31 32 total := 0 33 34 for { 35 count, err := reader.Read(data) 36 total += count 37 data = data[:count] 38 39 if err == io.EOF { 40 break 41 } else if err != nil { 42 return err 43 } 44 45 _, err = writer.Write(data) 46 if err != nil { 47 return err 48 } 49 50 percent <- int(int64(total) * int64(100) / fi.Size()) 51 } 52 53 return nil 54 }
To allow Go programmers to funnel data between different functions without writing too much glue code, many libraries accept the standard Reader
and Writer
interfaces. A library function expects a pointer to a Reader
object from the caller and then uses Read()
to draw data from it chunk by chunk.
The Reader
/Writer
interface builds an abstraction on the data, regardless of its origin, whether it's a JSON stream from a web server or blocks from the local filesystem read in via a file descriptor. The advantage is that things are kept flexible; you don't need to change the code just because the data source changes, because the interface remains the same.
Go Design: Reader/Writer
For example, Listing 1 opens the source file and receives an object of the os.File
type from the Open()
call. This object is passed to NewReader()
from the bufio package, which returns a reader that the caller can in turn use to extract the bytes from the file gradually. Accordingly, the code obtains a writer to the target file, which already exists in the application as a stick device entry – but on Unix, practically everything is a file.
Calling os.OpenFile()
with the O_WRONLY
option in line 25 opens the filesystem entry for writing. As it is a device entry that must already exist (instead of being created by the function), the O_CREATE
option, which is normally used for files, is deliberately missing here. Line 29 creates a new writer object from the file object, and the copying process can begin.
In the for
loop starting in line 34, the reader now fetches 4MB data chunks to match the data
buffer previously defined in line 11. However, the Read()
function does not always return 4MB, because at the end of the file, there may be less bytes available while scraping the bottom of the barrel. It is important to shorten the data
slice in line 37 to reflect the actual number of bytes retrieved and to discard garbage at the end of the buffer.
If the function were to simply pass the buffer to the writer, the writer would write the whole buffer out to the target file. A 5MB source file would be read as two 4MB chunks, written out to create an 8MB target file, with the last 3MB consisting of uninitialized garbage.
Percentages Through the Pipe
Since the data are read in small chunks and line 20 has determined the size of the source file in advance using os.Stat()
, the function knows in each loop pass how far it has progressed with copying and how much remains to be done. Line 50 writes this ratio as a percentage integer between
and 100
to the Go channel passed to the function as percent
by the caller. The caller later reads incoming values from the channel and can move a progress bar to the right while the function is still doing its job – true multitasking.
Now, how does the Flasher detect that the newly connected USB stick has appeared? In Listing 2, the driveWatch()
function, starting in line 14, calls devices()
from line 61 on to see which device entries are visible on the system below /dev/sd*
. Usually the first hard drive is listed there as /dev/sda
, while /dev/sdb
and higher mark other SATA devices. USB sticks usually appear in /dev/sdd
on my system, but this may vary elsewhere.
Listing 2
drive.go
01 package main 02 03 import ( 04 "bytes" 05 "errors" 06 "fmt" 07 "os/exec" 08 "path/filepath" 09 "strconv" 10 "strings" 11 "time" 12 ) 13 14 func driveWatch( 15 done chan error) chan string { 16 seen := map[string]bool{} 17 init := true 18 drivech := make(chan string) 19 go func() { 20 for { 21 dpaths, err := devices() 22 if err != nil { 23 done <- err 24 } 25 for _, dpath := range dpaths { 26 if _, ok := seen[dpath]; !ok { 27 seen[dpath] = true 28 if !init { 29 drivech <- dpath 30 } 31 } 32 } 33 init = false 34 time.Sleep(1 * time.Second) 35 } 36 }() 37 return drivech 38 } 39 40 func driveSize( 41 path string) (string, error) { 42 var out bytes.Buffer 43 cmd := exec.Command("sfdisk", "-s", path) 44 cmd.Stdout = &out 45 cmd.Stderr = &out 46 47 err := cmd.Run() 48 if err != nil { 49 return "", err 50 } 51 52 sizeStr := strings.TrimSuffix(out.String(), "\n") 53 size, err := strconv.Atoi(sizeStr) 54 if err != nil { 55 return "", err 56 } 57 58 return fmt.Sprintf("%.1f GB", float64(size)/float64(1024*1024)), nil 59 } 60 61 func devices() ([]string, error) { 62 devices := []string{} 63 paths, _ := filepath.Glob("/dev/sd*") 64 if len(paths) == 0 { 65 return devices, 66 errors.New("No devices found") 67 } 68 for _, path := range paths { 69 devices = append(devices, path) 70 } 71 return devices, nil 72 }
The standard globbing library in Go, similar to the one used by the shell to convert wildcards like *
into matching filesystem entries, does not report an error if the file path is invalid. It only complains about incorrect glob expressions. Because of this, if a glob expression does not find anything in Go, it is a good idea to look for other causes, such as incorrect paths or missing access permissions.
This is why devices()
searches through all entries starting in line 61, and driveWatch()
adopts these paths to initialize the seen
map variable with the entries that were found. This search is performed asynchronously, because driveWatch()
uses go func()
to start a Go routine in parallel in line 19. Meanwhile, the function proceeds to the end and returns the newly created drivech
Go channel to the caller to report newly discovered drives after the init phase.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.
News
-
The GNU Project Celebrates Its 40th Birthday
September 27 marks the 40th anniversary of the GNU Project, and it was celebrated with a hacker meeting in Biel/Bienne, Switzerland.
-
Linux Kernel Reducing Long-Term Support
LTS support for the Linux kernel is about to undergo some serious changes that will have a considerable impact on the future.
-
Fedora 39 Beta Now Available for Testing
For fans and users of Fedora Linux, the first beta of release 39 is now available, which is a minor upgrade but does include GNOME 45.
-
Fedora Linux 40 to Drop X11 for KDE Plasma
When Fedora 40 arrives in 2024, there will be a few big changes coming, especially for the KDE Plasma option.
-
Real-Time Ubuntu Available in AWS Marketplace
Anyone looking for a Linux distribution for real-time processing could do a whole lot worse than Real-Time Ubuntu.
-
KSMBD Finally Reaches a Stable State
For those who've been looking forward to the first release of KSMBD, after two years it's no longer considered experimental.
-
Nitrux 3.0.0 Has Been Released
The latest version of Nitrux brings plenty of innovation and fresh apps to the table.
-
Linux From Scratch 12.0 Now Available
If you're looking to roll your own Linux distribution, the latest version of Linux From Scratch is now available with plenty of updates.
-
Linux Kernel 6.5 Has Been Released
The newest Linux kernel, version 6.5, now includes initial support for two very exciting features.
-
UbuntuDDE 23.04 Now Available
A new version of the UbuntuDDE remix has finally arrived with all the updates from the Deepin desktop and everything that comes with the Ubuntu 23.04 base.