Go retrieves GPS data from the komoot app
Programming Snapshot – Go GPS Data Retrieval

© Lead Image © Nataliia Natykach, 123RF.com
The hiking and cycling app komoot saves your traveled excursion routes. Mike Schilli shows you how to retrieve the data with Go.
As we all know, the past year and a half were completely lost to the pandemic. Because of various lockdowns, there wasn't much left to do in your leisure time in terms of outdoor activity. Playing soccer was prohibited and jogging with a face covering too strenuous. This prompted my wife and me to begin exploring areas of our adopted city, San Francisco. We hiked hidden pathways, previously unknown to us, every evening on hour-long neighborhood walks. To our amazement, we discovered that even 25 years of living in a city is not enough to explore every last corner. We found countless little hidden stairways, unpaved paths, and sights completely unknown to major travel guides.
Memorizing all the turnoffs for these newly invented, winding city hiking trails is almost impossible, but fortunately a mobile phone can step in as a brain extension here. Hiking apps plan your tours, record your progress during the walk, display traveled paths on a online map, and allow for sharing completed tours with friends (Figure 1). One of the best-known hiking (and biking) apps is the commercial komoot, which is based on OpenStreetMap data and remains free of charge as long as the user gets invited by another (even new) user and limits themselves to one local hiking area.
Staying Safe
This cheap subscription should suffice for basic needs, but I ended up buying the World Pack. The one-time payment of US $30 is not exactly a pittance (Figure 2), but I thought I'd support the komoot whippersnappers, helping them to keep the lights on in the datacenter. But then it dawned on me: What happens if komoot goes out of business at some point? What happens to my painstakingly created trails in that case? Fortunately, komoot lets you export the GPS data from the routes you hiked, and you can restore the tours from these data in an emergency. However, seeing that I have dozens of saved trails (Figure 3), manual downloads would be too labor-intensive, not to mention the discipline required to back up new trails on a regular basis – after all, you never know when an app will fail or be abandoned.
For this reason, a program that logs into komoot once a week via a cron job and stores tours that have not yet been saved locally in a backup directory would be just the thing. Komoot does offer an API for script-driven retrieval of user data but not for ordinary people like me. When I asked about getting an API key, the support team there referred me to a B2B department that only deals with business partners. But with a bit of tweaking, a web scraper can also scrape the GPS data from the web page. That's exactly what the Go program presented in this article does, based on some reverse-engineering work published on GitHub [1].
Cron Mirror
The scraper negotiates the login process on the website, queries all the tours stored under the account, downloads their JSON data, and converts them into the GPX format supported by all GPS trackers [2]. Now, if komoot were to lose the saved routes for some reason, the tour collection could be restored from the backup because the GPX data represents the hikes as a set of GPS coordinates with timestamps, independently of a particular app.
Listing 1 [3] shows the main Go program. A cron job calls the program once a week, and Go downloads the .gpx
files of all tours that exist in a komoot account and saves them in a subdirectory named tours/
. To do this, the Go program logs into the komoot web page with a username and password in the kc.kLogin()
function, uses kc.kTours()
to query a list of all tours saved under that account, and downloads only the .gpx
files of the tours it doesn't already have in a local directory.
Listing 1
kbak.go
01 package main 02 03 import ( 04 "fmt" 05 "io/ioutil" 06 "log" 07 "os" 08 ) 09 10 const saveDir = "tours" 11 12 func main() { 13 kc := NewkColl() 14 15 _ = os.Mkdir(saveDir, 0755) 16 17 err := kc.kLogin() 18 if err != nil { 19 log.Fatalf("Login returned %v", err) 20 return 21 } 22 23 jdata, err := kc.kTours() 24 25 if err != nil { 26 log.Fatalf("Fetching tour ids returned %v", err) 27 return 28 } 29 30 ids := tourIDs(jdata) 31 32 for _, id := range ids { 33 gpxPath := fmt.Sprintf("%s/%s.gpx", saveDir, id) 34 if _, err := os.Stat(gpxPath); err == nil { 35 fmt.Printf("%s already saved\n", gpxPath) 36 continue 37 } 38 39 jdata, err = kc.kTour(id) 40 41 if err != nil { 42 log.Fatalf("Fetching tour %s returned %v", id, err) 43 return 44 } 45 46 gpx := toGpx(jdata) 47 48 fmt.Printf("Saving %s\n", gpxPath) 49 err := ioutil.WriteFile(gpxPath, gpx, 0644) 50 if err != nil { 51 panic(err) 52 } 53 } 54 }
The call to toGpx()
in line 46 converts the JSON data from komoot to the platform-independent GPX tracker format, while the code starting in line 49 saves newly converted data to a .gpx
file in the tours/
subdirectory. The procedure is gentle on the komoot servers and should not bother anyone there, and it helps the user to maintain ownership of routes they created themselves.
Caution, Password
It wouldn't be good style to hard code the username and password into the program, so Listing 2 offloads this to the creds.yaml
file. You don't want to check the file into a GitHub repo; just keep it local. The web scraper later reads this YAML file before fetching your tour data and uses the komoot email, the associated password, and the numeric user ID stored there in RAM only while the program is running. Sample values are shown in Listing 3. For active use of the program, replace them with the values of your komoot account.
Listing 2
creds.go
01 package main 02 03 import ( 04 "gopkg.in/yaml.v2" 05 "io/ioutil" 06 "os" 07 ) 08 09 var credsPath = "creds.yaml" 10 11 func readCreds() map[string]string { 12 creds := map[string]string{} 13 14 f, err := os.Open(credsPath) 15 if err != nil { 16 panic(err) 17 } 18 defer f.Close() 19 20 bytes, err := ioutil.ReadAll(f) 21 if err != nil { 22 panic(err) 23 } 24 25 err = yaml.Unmarshal(bytes, &creds) 26 if err != nil { 27 panic(err) 28 } 29 30 return creds 31 }
Listing 3
creds.yaml.sample
01 email: "foo@bar.com" 02 password: "hunter123" 03 client_id: "2014254181621"
To parse the YAML in creds.yaml
, Listing 2 retrieves the gopkg.in/yaml.v2
package from GitHub, which – in the exported Unmarshal()
function in line 25 – unravels the YAML data structure from Listing 3 and converts it into an internal Go hashmap. The data structure returned as creds
contains the email address serving as the username for the komoot account used in the email
key; password
is unsurprisingly the password, and the client_id
is the numeric ID of the user account with which komoot accesses the user's data. For an active account, the browser displays the numeric user ID in the URL field after you log in (Figure 4), from where it can be easily copied into the YAML file.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.
News
-
elementary OS 7.1 Now Available for Download
The team behind elementary OS has released the latest version of its operating system with a focus on personalization, inclusivity, accessibility, and privacy.
-
The GNU Project Celebrates Its 40th Birthday
September 27 marks the 40th anniversary of the GNU Project, and it was celebrated with a hacker meeting in Biel/Bienne, Switzerland.
-
Linux Kernel Reducing Long-Term Support
LTS support for the Linux kernel is about to undergo some serious changes that will have a considerable impact on the future.
-
Fedora 39 Beta Now Available for Testing
For fans and users of Fedora Linux, the first beta of release 39 is now available, which is a minor upgrade but does include GNOME 45.
-
Fedora Linux 40 to Drop X11 for KDE Plasma
When Fedora 40 arrives in 2024, there will be a few big changes coming, especially for the KDE Plasma option.
-
Real-Time Ubuntu Available in AWS Marketplace
Anyone looking for a Linux distribution for real-time processing could do a whole lot worse than Real-Time Ubuntu.
-
KSMBD Finally Reaches a Stable State
For those who've been looking forward to the first release of KSMBD, after two years it's no longer considered experimental.
-
Nitrux 3.0.0 Has Been Released
The latest version of Nitrux brings plenty of innovation and fresh apps to the table.
-
Linux From Scratch 12.0 Now Available
If you're looking to roll your own Linux distribution, the latest version of Linux From Scratch is now available with plenty of updates.
-
Linux Kernel 6.5 Has Been Released
The newest Linux kernel, version 6.5, now includes initial support for two very exciting features.