Close Search

Indexing Web Pages?

It is possible, and in fact quite easy, to use Recoll to index and search the web pages you visit. First, download and install the Firefox add-on called Recoll WE [5]. Second, enable web indexing from the GUI Index configuration panel, or by setting the processwebqueue variable to 1 in the configuration file. After that, one click on the Recoll WE button in Firefox will save a complete copy of the current page in a folder, where Recoll will later find, copy, and index it. The only limitation of this method is that Recoll stores all the copies in a cache of fixed size. No matter how big you make it, if that cache fills up, Recoll will automatically remove the older files it contains. You must explicitly archive any web page you want to keep indefinitely.

Index Maintenance

The command-line program that manages the actual indexing is called recollindex. You can launch recollindex from the File menu of the GUI, from the command line, or from a cron job. There are two ways, accessible from the Preferences | Indexing schedule panel, to keep an index up to date. One way is to make recollindex run all the time in the background, indexing and re-indexing files as soon as they are created or modified. This approach may slow down your computer, so avoid it unless you really, really need an index that is as up to date as possible.

In most cases, it is much better to only run recollindex at regular intervals, possibly every few days. By default, this program will analyze and index only the files that change after its previous run. You can, however, rebuild the whole index in any moment – from the GUI or by passing the -z option to recollindex. To update a different index, use the -c option to show recollindex the path to the corresponding configuration directory.

Basic and Advanced Search

Running basic searches in Recoll could not be any simpler: Enter the search term in the top text box, click the Search button, and browse the results (Figure 2). The Advanced Search panel (reachable from the Tools menu) and the web interface (Figure 3) can filter results in many ways, based on keywords or the type, age, and size of the file. All options are quite easy to understand by trial and error, so I will focus on an even more powerful feature: the language by which Recoll understands and can perform very complex queries. Once you know how to write a query, you can pass it to the recollq command line utility or enter it in the text box of the desktop interface. You'll need to set the search mode to Query language.

Figure 3: The Recoll web interface is almost as easy to use as the desktop one, and it provides the same option to save the results of any search as a spreadsheet.
Figure 2: The Recoll search result window: a clean listing, with buttons to preview or open every file discovered, as well as a show query link to check what Recoll did.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Recoll

    Whether you’re looking for a letter to the Internal Revenue Service or an email from an online trader, the Recoll desktop search machine will help you find it with just a few mouse clicks.

  • FSearch

    In a crowded field of search applications, FSearch offers many interesting functions for quickly searching files and folders, with more promised in the future.

  • Paperwork Document Manager

    Paperwork was developed to manage the paperless office – a dream as old as desktop PCs.

  • Tutorial – Manuskript

    The Manuskript editor is all you need to jump start your next writing project.

comments powered by Disqus
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters

Support Our Work

Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.

Learn More

News