Go retrieves GPS data from the komoot app
Cowboys
This handy, if cowboyish, drilling method is wrapped up in the drill()
function starting in line 36 of Listing 5. It takes an array of keys, descends into the sub-hashmaps, and outputs the data found at the end of the keychain. Listing 5 collects the numeric IDs of all the tours the user has in their account. These can be tours planned on a virtual drawing board – which the user has added to the map with komoot's idiosyncratic browser interface – as well as completed hikes that the user has already taken and recorded with the app or even another GPS tracker.
From JSON to GPX
Using the IDs, the main program, supported by the kTour()
function in Listing 4, can then fetch the data of individual tours from komoot. Finally, Listing 6 converts the fetched JSON data into GPX format, which popular trackers from Garmin and others understand, and which komoot also accepts when you are uploading new tours. This is the restore part of the backup solution. The result of the conversion is shown in Figure 6 in the XML typical of GPX, while Figure 7 shows that komoot accepts the downloaded data (after conversion to GPX) as a new tour, without any complaints.
However, converting Go data to XML in a prescriptive manner, much like the JSON conversion before it, requires a huge number of type
declarations linking its sub-elements in the various layers. Ultimately, the Go code would have to declare the entire GPX syntax. The structure could then be populated with the parsed data and turned into GPX-compliant XML using the encoding/xml package. To make things simpler – if also cowboyish – Listing 6 simply writes the XML as a parameterized text string instead.
Lines 13 through 20 of Listing 6 again use the drill()
function defined in Listing 5, which works its way into the sub-hashmaps of the unpacked JSON data. Listing 6 saves the geo-coordinates of the tour in line 19 in the items
array slice after a type assertion confirming that it is an array of unknown content ([]interface{}
).
Timing
The timestamps of the recorded waypoints are a special case because the JSON format in komoot lists the start time of a tour once, only at the beginning, in RFC 3339 format. Komoot then gives the individual times of the track points each in thousandths of a second relative to the start time. Unfortunately, the GPX format lists the respective absolute time for each waypoint, meaning that Listing 6 has to do some math.
Line 21 reads the time found in the tour start field and converts it to Go's internal time format. While the for
loop then rattles through the excavated track points starting in line 27, line 29 divides the time delta found there in milliseconds by 1,000 and calls Add()
to add the resulting seconds value to the start time in line 30. The result is the timestamp for the respective waypoint, which line 37 converts back into RFC 3339 format and injects into the XML as a string.
Added to this are the entries for lat
(latitude) and lng
(longitude, but lon
in XML), which are present as generic interface{}
s but are converted to a float64 format by the %f
placeholder of the Sprintf()
function via type assertion. The same applies to the alt
(altitude) above sea level, which goes into the ele
(elevation) field of the GPX format. What the main program gets back from toGpx()
is a byte array containing the XML data, which it writes to the backup file on disc and that completes the backup. If you upload the generated .gpx
file again by way of a test, you will see with growing enthusiasm that komoot recognizes it as a new tour just as if you had walked it: a perfect backup solution.
It must be said that the disadvantage of officially unmaintained web scrapers such as this one is that even the smallest layout changes to the provider's website can break the program and would require small adaptations. You have to live with that. But maybe komoot will eventually take pity and decide to grant hobbyists access to the API and register an OAuth2 client ID. That would be much cleaner.
Infos
- "Get Komoot tour data without API": https://python.plainenglish.io/get-komoot-tour-data-without-api-143df64e51fa
- GPX format: https://en.wikipedia.org/wiki/GPS_Exchange_Format
- Listings for this article: ftp://ftp.linux-magazine.com/pub/listings/linux-magazine.com/252/
- "Programming Snapshot – Colly" by Mike Schilli, Linux Magazine, issue 223, June 2019, https://www.linux-magazine.com/Issues/2019/223/Data-Scraper/(language)/eng-US
« Previous 1 2 3
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.