Go retrieves GPS data from the komoot app
Programming Snapshot – Go GPS Data Retrieval
The hiking and cycling app komoot saves your traveled excursion routes. Mike Schilli shows you how to retrieve the data with Go.
As we all know, the past year and a half were completely lost to the pandemic. Because of various lockdowns, there wasn't much left to do in your leisure time in terms of outdoor activity. Playing soccer was prohibited and jogging with a face covering too strenuous. This prompted my wife and me to begin exploring areas of our adopted city, San Francisco. We hiked hidden pathways, previously unknown to us, every evening on hour-long neighborhood walks. To our amazement, we discovered that even 25 years of living in a city is not enough to explore every last corner. We found countless little hidden stairways, unpaved paths, and sights completely unknown to major travel guides.
Memorizing all the turnoffs for these newly invented, winding city hiking trails is almost impossible, but fortunately a mobile phone can step in as a brain extension here. Hiking apps plan your tours, record your progress during the walk, display traveled paths on a online map, and allow for sharing completed tours with friends (Figure 1). One of the best-known hiking (and biking) apps is the commercial komoot, which is based on OpenStreetMap data and remains free of charge as long as the user gets invited by another (even new) user and limits themselves to one local hiking area.
Staying Safe
This cheap subscription should suffice for basic needs, but I ended up buying the World Pack. The one-time payment of US $30 is not exactly a pittance (Figure 2), but I thought I'd support the komoot whippersnappers, helping them to keep the lights on in the datacenter. But then it dawned on me: What happens if komoot goes out of business at some point? What happens to my painstakingly created trails in that case? Fortunately, komoot lets you export the GPS data from the routes you hiked, and you can restore the tours from these data in an emergency. However, seeing that I have dozens of saved trails (Figure 3), manual downloads would be too labor-intensive, not to mention the discipline required to back up new trails on a regular basis – after all, you never know when an app will fail or be abandoned.
For this reason, a program that logs into komoot once a week via a cron job and stores tours that have not yet been saved locally in a backup directory would be just the thing. Komoot does offer an API for script-driven retrieval of user data but not for ordinary people like me. When I asked about getting an API key, the support team there referred me to a B2B department that only deals with business partners. But with a bit of tweaking, a web scraper can also scrape the GPS data from the web page. That's exactly what the Go program presented in this article does, based on some reverse-engineering work published on GitHub [1].
Cron Mirror
The scraper negotiates the login process on the website, queries all the tours stored under the account, downloads their JSON data, and converts them into the GPX format supported by all GPS trackers [2]. Now, if komoot were to lose the saved routes for some reason, the tour collection could be restored from the backup because the GPX data represents the hikes as a set of GPS coordinates with timestamps, independently of a particular app.
Listing 1 [3] shows the main Go program. A cron job calls the program once a week, and Go downloads the .gpx
files of all tours that exist in a komoot account and saves them in a subdirectory named tours/
. To do this, the Go program logs into the komoot web page with a username and password in the kc.kLogin()
function, uses kc.kTours()
to query a list of all tours saved under that account, and downloads only the .gpx
files of the tours it doesn't already have in a local directory.
Listing 1
kbak.go
01 package main 02 03 import ( 04 "fmt" 05 "io/ioutil" 06 "log" 07 "os" 08 ) 09 10 const saveDir = "tours" 11 12 func main() { 13 kc := NewkColl() 14 15 _ = os.Mkdir(saveDir, 0755) 16 17 err := kc.kLogin() 18 if err != nil { 19 log.Fatalf("Login returned %v", err) 20 return 21 } 22 23 jdata, err := kc.kTours() 24 25 if err != nil { 26 log.Fatalf("Fetching tour ids returned %v", err) 27 return 28 } 29 30 ids := tourIDs(jdata) 31 32 for _, id := range ids { 33 gpxPath := fmt.Sprintf("%s/%s.gpx", saveDir, id) 34 if _, err := os.Stat(gpxPath); err == nil { 35 fmt.Printf("%s already saved\n", gpxPath) 36 continue 37 } 38 39 jdata, err = kc.kTour(id) 40 41 if err != nil { 42 log.Fatalf("Fetching tour %s returned %v", id, err) 43 return 44 } 45 46 gpx := toGpx(jdata) 47 48 fmt.Printf("Saving %s\n", gpxPath) 49 err := ioutil.WriteFile(gpxPath, gpx, 0644) 50 if err != nil { 51 panic(err) 52 } 53 } 54 }
The call to toGpx()
in line 46 converts the JSON data from komoot to the platform-independent GPX tracker format, while the code starting in line 49 saves newly converted data to a .gpx
file in the tours/
subdirectory. The procedure is gentle on the komoot servers and should not bother anyone there, and it helps the user to maintain ownership of routes they created themselves.
Caution, Password
It wouldn't be good style to hard code the username and password into the program, so Listing 2 offloads this to the creds.yaml
file. You don't want to check the file into a GitHub repo; just keep it local. The web scraper later reads this YAML file before fetching your tour data and uses the komoot email, the associated password, and the numeric user ID stored there in RAM only while the program is running. Sample values are shown in Listing 3. For active use of the program, replace them with the values of your komoot account.
Listing 2
creds.go
01 package main 02 03 import ( 04 "gopkg.in/yaml.v2" 05 "io/ioutil" 06 "os" 07 ) 08 09 var credsPath = "creds.yaml" 10 11 func readCreds() map[string]string { 12 creds := map[string]string{} 13 14 f, err := os.Open(credsPath) 15 if err != nil { 16 panic(err) 17 } 18 defer f.Close() 19 20 bytes, err := ioutil.ReadAll(f) 21 if err != nil { 22 panic(err) 23 } 24 25 err = yaml.Unmarshal(bytes, &creds) 26 if err != nil { 27 panic(err) 28 } 29 30 return creds 31 }
Listing 3
creds.yaml.sample
01 email: "foo@bar.com" 02 password: "hunter123" 03 client_id: "2014254181621"
To parse the YAML in creds.yaml
, Listing 2 retrieves the gopkg.in/yaml.v2
package from GitHub, which – in the exported Unmarshal()
function in line 25 – unravels the YAML data structure from Listing 3 and converts it into an internal Go hashmap. The data structure returned as creds
contains the email address serving as the username for the komoot account used in the email
key; password
is unsurprisingly the password, and the client_id
is the numeric ID of the user account with which komoot accesses the user's data. For an active account, the browser displays the numeric user ID in the URL field after you log in (Figure 4), from where it can be easily copied into the YAML file.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.