Big Data search engine for full-text strings and photos with radius search
Mini-Google with Many Formats
Later, the script in Listing 2 finds files for predefined keywords, just like an Internet search engine. When you call this script with fs-search '*'
, the command matches every document in the index. (The single quotes stop the Unix shell from grabbing the metacharacter *
and turning it into a glob in the local directory.) Anyway, Elasticsearch returns 10 more or less random results for fs-search '*'
, because with no further configuration, the maximum number of hits is set to 10. The script shown a little later changes this value to 100.
Listing 2
fs-search
The search()
method called in line 12 expects the name of the search index under which the data resides (again fs
), and it wants the query string in the body
part of the request. From the documentation [7], you can see that Elasticsearch obviously understands a whole range of historically grown query formats, which necessitates the seemingly absurd nesting (lines 15-17) of query/query_string/query
.
The result of the fact-finding mission is a reference to an array of hits, which the for loop iterates over in lines 20 and 21, printing only the names of the matching files, which are part of the result set.
Searching GPS Image Data
Elasticsearch can do even more, though. For example, the geo_distance
filter [8] extends the classic full-text search, adding an interesting capability. If you configure the filter and store matching geodetic data for each document, the search engine will show you the entries that are located within a certain radius. This feature could be useful, for example, if you find yourself roaming around with your mobile phone late at night looking for a five-star restaurant in the area that is still serving.
Because my iPhone 5, just like any other smartphone, stores the GPS data in the Exif header of the JPEG file for every image I shoot, a search that starts with a given image in the phone's photo album ("Gallery") and finds images that I shot within a 1km radius of the image's location could be fun. As an example, Figure 2 shows a photo of the newly constructed eastern span of the Bay Bridge [9] in my city of residence, San Francisco. Since last year, pedestrians and bike riders can get to the middle of the bridge on a dedicated path, from where I shot a number of photos.
Figure 3 shows the output of the exiftags
command for the photos transmitted from the phone to my Linux machine. Almost at the bottom, you can see that the image was shot at the geolocation 37°48.87' north latitude and 122°21.55' west longitude.
Modern Sextant
The photo_latlon()
function in Listing 3 reads these GPS values with the CPAN Image::EXIF module and uses dm2decimal()
from the Geo::Coordinates::DecimalDegrees module to convert them to floating point numbers. The regular expression in lines 47-53 searches the geodata for a letter (N or S for north or south latitude, W or E for western or eastern longitude), followed by the numerical degree value and the degree symbol encoded in UTF-8. The minutes follow after one or more spaces.
Listing 3
IPhonePicGeo.pm
Thus, 37°48.87'N becomes the value 37.816
and 122°21.55'W becomes the negative floating-point number -122.3555
.
Google Maps confirmed (Figure 4) that the talented photographer really was standing in the middle of San Francisco Bay on the Bay Bridge when he pressed the button. To find out whether other pictures in the photo album were recorded within a radius of 1km, photo-index
in Listing 4 examines all the transferred photos in the ~/iphone
directory and stores their GPS data on the local Elasticsearch server.
Listing 4
photo-index
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.
-
ZorinOS 17.1 Released, Includes Improved Windows App Support
If you need or desire to run Windows applications on Linux, there's one distribution intent on making that easier for you and its new release further improves that feature.
-
Linux Market Share Surpasses 4% for the First Time
Look out Windows and macOS, Linux is on the rise and has even topped ChromeOS to become the fourth most widely used OS around the globe.
-
KDE’s Plasma 6 Officially Available
KDE’s Plasma 6.0 "Megarelease" has happened, and it's brimming with new features, polish, and performance.
-
Latest Version of Tails Unleashed
Tails 6.0 is based on Debian 12 and includes GNOME 43.
-
KDE Announces New Slimbook V with Plenty of Power and KDE’s Plasma 6
If you're a fan of KDE Plasma, you'll be thrilled to hear they've announced a new Slimbook with an AMD CPU and the latest version of KDE Plasma desktop.
-
Monthly Sponsorship Includes Early Access to elementary OS 8
If you want to get a glimpse of what's in the pipeline for elementary OS 8, just set up a monthly sponsorship to help fund its continued existence.