Safer Internet Searches
Setting Up a Search Engine
If you want to set up a search engine for a private network or personal website, Searx and YaCy can help you get started. Both open source packages are simple to install and offer several installation methods, including Docker images, which provide an especially easy way to become familiar with each search engine as well as how the search engine affects your system.
With both Searx and YaCy, you need to pay attention to your system load. With YaCy, you also need to be mindful of your bandwidth due to YaCy's distributed nature.
Both YaCy and Searx can be tested using public instances, and both offer detailed websites with information on how to change settings to match your system, as well as defaults for other settings.
Finally, Searx and YaCy are also available as source code and other binary packages. YaCy, which uses Java, is the simpler option; you run the ant
binary to compile. However, you can also install the Searx package from your distribution's standard repository.
Searx as a Solution
If you like Docker, you can stick with it for installation. You do need to pull the Searx package and select which ports to use:
docker pull searx/searx
All Searx settings are available for both regular and Docker installations. There are advantages to using the locally compiled version with performance being one such reason. To get deeper into the installation and settings, see the Searx installation document [9].
If you want to use another installation method, download the source code from GitHub [10] and use the appropriate script to install. You'll find several scripts. The NGINX and Apache web servers are supported. If you want to add Searx to an existing site, follow the instructions on the NGINX and Apache websites.
You should also be able to find the Searx package in your distribution's repository using a general binary. This works fine unless you want to squeeze all the performance possible out of your system.
Searx Settings
Regardless of installation method, you will use the default settings file (settings.yml
) to set your preferences. You can use the default options, which contains many well-known search engines and some lesser known ones.
You also need to set up a directory for settings.yml
and then point to it. The files in this directory are the ones controlling Searx. Since the settings file is written in YAML, you have access to all of YAML's features for creating links (like adding a site) that work as searches for your service.
When you compile, you will need most expected libraries for handling networking, the build, etc. You should check out the uwsgi
install, which is a minimal binary protocol for communicating between the nodes.
In the main settings, you have one absolutely vital task to perform: setting the secret_key
value (Listing 1). To choose this value, your best option is to let OpenSLL create it for you, but you can also use a password manager. The other options under the main settings are useful but not necessary to get started.
Listing 1
Setting secret_key
01 server: port : 8888 02 bind_address : "127.0.0.1" # address to listen on 03 secret_key : "SuperSecretKey" # change this! 04 base_url : <http://localhost:/> # Set custom base_url. 05 Possible values: False or 06 "<https://your.custom.host/location/>" 07 image_proxy : False # Proxying image results 08 through searx http_protocol_version : "1.0" # 1.0 and 09 1.1 are supported 10 method: "POST" # POST queries are more secure as 11 they don't show up in history but may cause problems when using 12 Firefox containers 13 default_http_headers: 14 X-Content-Type-Options :nosniff 15 X-XSS-Protection : 1; 16 mode=block 17 X-Download-Options : noopen 18 X-Robots-Tag : noindex, nofollow 19 Referrer-Policy : no-referrer
If you look through the rest of the settings file, you will find the valid search providers for your instance. You can also use this file to limit your searches if you have concerns about a particular provider.
To add more providers (you can even add a single page), you need to add a section to the settings file. Copy the default file and edit what you need. As a simple example, you can edit the Wikipedia settings entry in the file:
- name : wikipedia engine : wikipedia shortcut : wp base_url : 'https://{language}.wikipedia.org/'
In addition to the language
variable, you'll find the query
, page
, and params
variables, which can be used to control your searches. You also can control the type of results that are returned. Result options include strings, images, and videos, as well as torrent files.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs