Go retrieves GPS data from the komoot app
Programming Snapshot – Go GPS Data Retrieval

© Lead Image © Nataliia Natykach, 123RF.com
The hiking and cycling app komoot saves your traveled excursion routes. Mike Schilli shows you how to retrieve the data with Go.
As we all know, the past year and a half were completely lost to the pandemic. Because of various lockdowns, there wasn't much left to do in your leisure time in terms of outdoor activity. Playing soccer was prohibited and jogging with a face covering too strenuous. This prompted my wife and me to begin exploring areas of our adopted city, San Francisco. We hiked hidden pathways, previously unknown to us, every evening on hour-long neighborhood walks. To our amazement, we discovered that even 25 years of living in a city is not enough to explore every last corner. We found countless little hidden stairways, unpaved paths, and sights completely unknown to major travel guides.
Memorizing all the turnoffs for these newly invented, winding city hiking trails is almost impossible, but fortunately a mobile phone can step in as a brain extension here. Hiking apps plan your tours, record your progress during the walk, display traveled paths on a online map, and allow for sharing completed tours with friends (Figure 1). One of the best-known hiking (and biking) apps is the commercial komoot, which is based on OpenStreetMap data and remains free of charge as long as the user gets invited by another (even new) user and limits themselves to one local hiking area.
Staying Safe
This cheap subscription should suffice for basic needs, but I ended up buying the World Pack. The one-time payment of US $30 is not exactly a pittance (Figure 2), but I thought I'd support the komoot whippersnappers, helping them to keep the lights on in the datacenter. But then it dawned on me: What happens if komoot goes out of business at some point? What happens to my painstakingly created trails in that case? Fortunately, komoot lets you export the GPS data from the routes you hiked, and you can restore the tours from these data in an emergency. However, seeing that I have dozens of saved trails (Figure 3), manual downloads would be too labor-intensive, not to mention the discipline required to back up new trails on a regular basis – after all, you never know when an app will fail or be abandoned.
For this reason, a program that logs into komoot once a week via a cron job and stores tours that have not yet been saved locally in a backup directory would be just the thing. Komoot does offer an API for script-driven retrieval of user data but not for ordinary people like me. When I asked about getting an API key, the support team there referred me to a B2B department that only deals with business partners. But with a bit of tweaking, a web scraper can also scrape the GPS data from the web page. That's exactly what the Go program presented in this article does, based on some reverse-engineering work published on GitHub [1].
Cron Mirror
The scraper negotiates the login process on the website, queries all the tours stored under the account, downloads their JSON data, and converts them into the GPX format supported by all GPS trackers [2]. Now, if komoot were to lose the saved routes for some reason, the tour collection could be restored from the backup because the GPX data represents the hikes as a set of GPS coordinates with timestamps, independently of a particular app.
Listing 1 [3] shows the main Go program. A cron job calls the program once a week, and Go downloads the .gpx
files of all tours that exist in a komoot account and saves them in a subdirectory named tours/
. To do this, the Go program logs into the komoot web page with a username and password in the kc.kLogin()
function, uses kc.kTours()
to query a list of all tours saved under that account, and downloads only the .gpx
files of the tours it doesn't already have in a local directory.
Listing 1
kbak.go
01 package main 02 03 import ( 04 "fmt" 05 "io/ioutil" 06 "log" 07 "os" 08 ) 09 10 const saveDir = "tours" 11 12 func main() { 13 kc := NewkColl() 14 15 _ = os.Mkdir(saveDir, 0755) 16 17 err := kc.kLogin() 18 if err != nil { 19 log.Fatalf("Login returned %v", err) 20 return 21 } 22 23 jdata, err := kc.kTours() 24 25 if err != nil { 26 log.Fatalf("Fetching tour ids returned %v", err) 27 return 28 } 29 30 ids := tourIDs(jdata) 31 32 for _, id := range ids { 33 gpxPath := fmt.Sprintf("%s/%s.gpx", saveDir, id) 34 if _, err := os.Stat(gpxPath); err == nil { 35 fmt.Printf("%s already saved\n", gpxPath) 36 continue 37 } 38 39 jdata, err = kc.kTour(id) 40 41 if err != nil { 42 log.Fatalf("Fetching tour %s returned %v", id, err) 43 return 44 } 45 46 gpx := toGpx(jdata) 47 48 fmt.Printf("Saving %s\n", gpxPath) 49 err := ioutil.WriteFile(gpxPath, gpx, 0644) 50 if err != nil { 51 panic(err) 52 } 53 } 54 }
The call to toGpx()
in line 46 converts the JSON data from komoot to the platform-independent GPX tracker format, while the code starting in line 49 saves newly converted data to a .gpx
file in the tours/
subdirectory. The procedure is gentle on the komoot servers and should not bother anyone there, and it helps the user to maintain ownership of routes they created themselves.
Caution, Password
It wouldn't be good style to hard code the username and password into the program, so Listing 2 offloads this to the creds.yaml
file. You don't want to check the file into a GitHub repo; just keep it local. The web scraper later reads this YAML file before fetching your tour data and uses the komoot email, the associated password, and the numeric user ID stored there in RAM only while the program is running. Sample values are shown in Listing 3. For active use of the program, replace them with the values of your komoot account.
Listing 2
creds.go
01 package main 02 03 import ( 04 "gopkg.in/yaml.v2" 05 "io/ioutil" 06 "os" 07 ) 08 09 var credsPath = "creds.yaml" 10 11 func readCreds() map[string]string { 12 creds := map[string]string{} 13 14 f, err := os.Open(credsPath) 15 if err != nil { 16 panic(err) 17 } 18 defer f.Close() 19 20 bytes, err := ioutil.ReadAll(f) 21 if err != nil { 22 panic(err) 23 } 24 25 err = yaml.Unmarshal(bytes, &creds) 26 if err != nil { 27 panic(err) 28 } 29 30 return creds 31 }
Listing 3
creds.yaml.sample
01 email: "foo@bar.com" 02 password: "hunter123" 03 client_id: "2014254181621"
To parse the YAML in creds.yaml
, Listing 2 retrieves the gopkg.in/yaml.v2
package from GitHub, which – in the exported Unmarshal()
function in line 25 – unravels the YAML data structure from Listing 3 and converts it into an internal Go hashmap. The data structure returned as creds
contains the email address serving as the username for the komoot account used in the email
key; password
is unsurprisingly the password, and the client_id
is the numeric ID of the user account with which komoot accesses the user's data. For an active account, the browser displays the numeric user ID in the URL field after you log in (Figure 4), from where it can be easily copied into the YAML file.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Direct Download
Read full article as PDF:
Price $2.95
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
News
-
An All-Snap Version of Ubuntu is In The Works
Along with the standard deb version of the open-source operating system, Canonical will release an-all snap version.
-
Mageia 9 Beta 2 Ready for Testing
The latest beta of the popular Mageia distribution now includes the latest kernel and plenty of updated applications.
-
KDE Plasma 6 Looks to Bring Basic HDR Support
The KWin piece of KDE Plasma now has HDR support and color management geared for the 6.0 release.
-
Bodhi Linux 7.0 Beta Ready for Testing
The latest iteration of the Bohdi Linux distribution is now available for those who want to experience what's in store and for testing purposes.
-
Changes Coming to Ubuntu PPA Usage
The way you manage Personal Package Archives will be changing with the release of Ubuntu 23.10.
-
AlmaLinux 9.2 Now Available for Download
AlmaLinux has been released and provides a free alternative to upstream Red Hat Enterprise Linux.
-
An Immutable Version of Fedora Is Under Consideration
For anyone who's a fan of using immutable versions of Linux, the Fedora team is currently considering adding a new spin called Fedora Onyx.
-
New Release of Br OS Includes ChatGPT Integration
Br OS 23.04 is now available and is geared specifically toward web content creation.
-
Command-Line Only Peropesis 2.1 Available Now
The latest iteration of Peropesis has been released with plenty of updates and introduces new software development tools.
-
TUXEDO Computers Announces InfinityBook Pro 14
With the new generation of their popular InfinityBook Pro 14, TUXEDO upgrades its ultra-mobile, powerful business laptop with some impressive specs.