Optimizing and visualizing GPS data

Climbing Aid

Article from Issue 185/2016
Author(s):

Handheld navigation devices point the way and continuously record your position while you are walking. With a few scripts on Linux, extreme climber Mike Schilli attractively visualizes the data from some of his bold first ascents.

Every smartphone features a GPS receiver nowadays, and a generous collection of apps are guiding hikers across hill and dale by displaying maps. Of course, things can be pretty rustic out there in the wild, and it is a good idea to use more robust, water-splash protected devices with more powerful batteries. Some time ago, I purchased a Garmin 62s – on special offer. Although it might be a little outmoded by now, it looks as if you could drive a tank over it without causing any damage.

If you are spoiled by years of intuitive on-screen operation with your smartphone, you will probably be rubbing your eyes in disbelief to discover that LCD displays with weird menu designs – in which the user has to steer the cursor with a dozen plastic buttons on the front of the device – really do still exist (Figure 1).

Figure 1: The Garmin GPSmap 62s.

UI for Steam Punks

Entering a single waypoint to mark the start of the trail nearly drove me mad. This is probably what the future of mobile telephony would have looked like if Bill Gates had asserted his aggressive monopoly policy – heaven forbid! If I were a product designer at Garmin, I would immediately launch a product line with a Mad Max Fury Road-influenced steam punk look.

Moreover, the software for reading and writing data to and from the device only runs on Windows; the Mac version did not even detect the device when I plugged it in, and they don't offer a Linux version at all. However, because my Ubuntu system responded immediately when I plugged in the USB connector (see the syslog in Figure 2), revealing the device's data storage areas as two mounted disks, I calmed down straight away and decided to simply switch on the device and take it with me when I went on a hike.

Figure 2: After plugging in the USB connector, Ubuntu immediately detects the navigation device and mounts its internal storage as a disks.

Even without doing anything, the device writes an entry to what it calls a Tracks file every few seconds, recording the time of day at which the device was located at a certain position. Besides the timestamp, it records the longitude and latitude, as well as the current elevation above sea level.

Open Source Instead of Windows

Garmin devices store these data points in an XML dialect names GPX, which you can easily explore using open source tools. Figure 3 shows the files stored on the device. The Tracks file Track_2015-12-31 130304.gpx contains the locations I was looking for and the matching times.

Figure 3: GPX files on the Garmin 62s.

Parsing the XML is not difficult; in fact, a CPAN author has already taken the trouble of bundling all of this magic into a module named Geo::Gpx, so Listing 1 [1] only took a couple of lines of code – after downloading the module – to convert the whole enchilada to the YAML format, which you can more easily inspect visually and process computationally downstream.

Listing 1

gpx2yaml

01 #!/usr/local/bin/perl -w
02 use strict;
03 use Geo::Gpx;
04 use YAML qw( Dump );
05
06 my( $file ) = @ARGV;
07 die "usage: $0 file" if !defined $file;
08
09 open my $fh, "<$file" or die $!;
10 my $gpx = Geo::Gpx->new( input => $fh );
11
12 print Dump $gpx->tracks->[ 0 ]->
13    { segments }->[ 0 ]->{ points };

For each timestamp (time) in Unix seconds, the output in Figure 4 shows the longitude (lon), latitude (lat), and elevation above sea level in feet (ele). If desired, you could change the settings to metric units on the device.

Figure 4: Listing 1 converts the GPX data from the Tracks file to a YAML format that is far easier on the eye.

Listing 1 uses two modules from CPAN to do this: Geo::Gpx and YAML. On request, the latter exports the Dump() method, which outputs a nested data structure in YAML format; in this case, it is based on the array of hashes below tracks, segments, and finally points in the mess of GPX fed to and processed by Geo::Gpx.

The scripts presented here form a process chain: Each reads the YAML output of the previous step via a Unix pipe and the Load() function from the YAML module and then proceeds to work with and transform the data. The scripts also write their output in YAML format so that any number of scripts of this kind can be strung together in good old Unix style, each focusing on a specific task.

Lazy Operator

The next stage in the process chain takes care of extracting relevant parts from the Tracks file. Between operator inertia and forgetfulness, I didn't access the device's Tracks menu either to delete the tracks or save them under the name of a tour (e.g., "Grand Canyon"). Instead, I just let the device happily save everything to the same Tracks file throughout my entire vacation; later, I even noticed that this had been going on for several years while I had been using the device off and on.

To clean up this mess, the tours script in Listing 2 is called with

gpx2yaml *.GPX | tours

and grabs the GPX data converted to YAML from the Unix pipe. It then breaks down the data points into groups representing different daily tours, which it detects by looking for gaps between recordings. If it finds more than five hours without activity between two entries (i.e., a period in which the device was probably switched off), it assumes it found different tours.

Listing 2

tours

01 #!/usr/local/bin/perl -w
02 use strict;
03 use Getopt::Long;
04 GetOptions( \my %opts, "tour=s" );
05 use YAML qw( Load Dump );
06
07 my @tours           = ( );
08 my $tour;
09 my $tracks          = Load join "", <>;
10 my $tour_split_secs = 60 * 60 * 5;
11
12 for my $point ( @$tracks ) {
13
14     # next tour?
15   if( defined $tour and
16     $point->{ "time" } >
17     $tour->{ end } + $tour_split_secs ) {
18     undef $tour;
19   }
20
21     # start new tour?
22   if( !defined $tour ) {
23     $tour = {
24       start  => $point->{ "time" },
25       points => [] };
26     push @tours, $tour;
27   }
28
29     # next point in current tour
30   $tour->{ end } = $point->{ "time" };
31   push @{ $tour->{ points } }, $point;
32 }
33
34 if( $opts{ tour } ) {
35     print Dump( $tours[
36         $opts{ tour } - 1 ]->{ points } );
37
38 } else {
39   my $idx = 1;
40   for my $tour ( @tours ) {
41     printf "tour %02d: %s - %s (%d)\n",
42       $idx, map( { scalar localtime $_ }
43       ( $tour->{ start }, $tour->{ end } )
44       ), scalar @{ $tour->{ points } };
45     $idx++;
46   }
47 }

Figure 5 shows the tours the script found and enumerated from 01 through 19, each with a start and end time. The script is showing me these because I called it without specifying any parameters. To select a specific daily tour, you need to pass in the tour number. Typing

gpx2yaml *.GPX | tours --tour=18
Figure 5: The tours script extracts the individual tours from the tracking data.

outputs the track data for the tour on December 30, 2015 (again as YAML, of course), when my wife and I hiked a couple of miles down and back up on the icy "Bright Angel Trail" in the Grand Canyon (Figure 6). The output from Listing 2 looks like Figure 4, but with fewer data points because everything apart from December 30 has been caught by the tours filter.

Figure 6: Hiking with a brick-sized GPS device on the Bright Angel Trail in the Grand Canyon.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • GPS Tools

    Almost all manufacturers of GPS devices use proprietary formats to save routes, tracks, and waypoints. Vendors unfortunately rarely offer Linux software for uploading and downloading or processing the data. Four GPS editors keep Linux users on the right track.

  • Perl: Plotting GPS Data

    Perl hackers take to the hills with a navigation system that provides a graphical rendering of a hiking tour.

  • Prune

    Have more fun with your GPS data with this easy-to-use Java-based tool.

  • Google Earth

    Google Earth lets you view the world from space and zoom in on towns, roads, and buildings. We’ll show you how to travel our planet’s surface, its cities, and even the sky from your desktop.

  • Perl: Reading GPS Data

    With a small GPS receiver on his wrist, Mike has been jogging through San Francisco neighborhoods. While catching his breath, safe at home, he visualizes the data he acquired while running with Perl.

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News

njobs Europe
What:
Where:
Country:
Njobs Netherlands Njobs Deutschland Njobs United Kingdom Njobs Italia Njobs France Njobs Espana Njobs Poland
Njobs Austria Njobs Denmark Njobs Belgium Njobs Czech Republic Njobs Mexico Njobs India Njobs Colombia