Debugging with strace
So far, I've used strace with very few options: just -o to send the output to a file. But a variety of options are available that make it easier to understand and debug programs.
If you want to use strace with any complicated programs, you'll probably want to be able to trace their forks as well. The -f switch traces child processes (forks) as they are created. -F will do the same thing, but with vforks. (vfork is a special case of fork that doesn't take certain sorts of information from the parent process.)
Another useful set of options have to do with time. These can come in handy if you're debugging a program that hangs or that seems to be running particularly slowly. In this way, you can find out where the problem actually lies. The -t, -tt, and -ttt switches all put the time of day – to the second, microsecond, or second+microsecond, written as seconds since epoch – at the start of each line. The -T switch shows the time spent in system calls – that is, the time difference between the beginning and end of each system call. Similarly, -r prints a relative time stamp at the start of each system call.
The -c switch counts the time, calls, and errors for each system call and summarizes this information when the program exits. This information is helpful if you're debugging and you're not sure where to start, because it will help you work out where the problem might lie.
Additionally, you can use -e trace=set to trace only the system calls you specify – for example, -e trace=open,read,write (note the lack of spaces) to trace only the open, read, and write system calls. This can minimize the amount of output you have to wade through if you've used -c to identify what you think are the problem calls.
Finally, if you want more gory details for your traces, try -v, which gives full values of stat, environment, and other common calls. (Use the -v flag and then have another look at the fstat call as discussed above to get all the information that's returned about that file.)
Shell Script Wrappers
Although it's not always possible to run your command under "real" conditions when something goes wrong, a possible workaround might be to write a shell script wrapper for the command so that you can run it via strace.
Move your program command out of the way by renaming it to, for example, command.old. Then save the script in Listing 4 to command.
Wrapping Another Command with strace
01 #!/bin/sh 02 # 03 strace -o/tmp/command_out.$$ /usr/bin/command.old $*
This not only calls the real command program with any arguments you used but also runs strace on it and dumps the output to a new file each time the script is called. Then you can check out what's happening under these real circumstances. Don't forget to move the real program back where it belongs once you're done!
Another very useful option is to use the -p PID option: This attaches strace to the process with the specified process ID. So if you suddenly see a process hanging or eating up lots of CPU, you can find out what it's doing in real time.
Knowing what system calls should look like and how they work makes it much easier to notice when something's not quite right. Often it's clear very quickly what's going wrong (e.g., a failed open call to a particular file). Looking at file permissions can also be helpful.
Strace is incredibly useful when debugging, but it can also be educational to run it on programs or commands that are running perfectly well, just to see what's going on under the hood of your working system.
One of the major advantages of running Linux is that the innards of the system are both accessible and (largely) well documented. Strace is one of the tools that can let you make the most of this access.
- strace article part 1: http://www.linuxpromagazine.com/issues/2009/103/bug_bumper
- strace tool: http://sourceforge.net/projects/strace/
Carnegie Mellon researchers say 3 million pages could fall down the phishing hole in the next year.
The US government rolls new best-practice rules for protecting SSH.
Klaus Knopper announces the latest version of his iconic Live Linux system.
All websites that use these popular CMS tools could be vulnerable to denial of service attacks if users don't install the updates.
According to a report, many potential victims of the Heartbleed attack have patched their systems, but few have cleaned up the crime scene to protect themselves from the effects of a previous intrusion.
DARPA and NICTA release the code for the ultra-secure microkernel system used in aerial drones.
Should you trust an online service to store your online passwords?
New B+ board lets you build cool things without the complication of a powered USB hub.
Redmond rushes in to root out alleged malware haven.
New initiative will bring futuristic virtual reality effects to the web surfing experience.