Text File Statistics
Linguist: A Statistical Extension for LibreOffice Writer
Linguist offers writers interesting, valuable data to help them make the best of their words.
LibreOffice Writer includes features for both the home user's short files and a publisher's lengthy manuscripts. However, a professional writer would benefit from statistical tools to answer such questions as: How often does each word appear? What is the average sentence length? How readable is a passage? Needing such information for a book-length manuscript, I turned to LibreOffice's extension site, confident that among the hundreds of extensions available I would find at least one that would give me such statistics. Within seconds I had found Linguist, which was able to give me the raw data I needed, although without any filters or more than a summary in a LibreOffice file.
Linguist is installed like every other LibreOffice extension: Download the extension, open Writer |Tools | Extensions to find and add it, and Linguist will be available the next time you reboot Writer. Just be sure that you download the 1.5.1 release at the bottom of Linguist's page rather than use the Download Latest button at the top of the page, which does not do what it says. When you reboot, you will find that, unlike many extensions that add a menu item or a toolbar icon seemingly at random, Linguist adds its own easily found menu. If, like me, you are not fond of extra menus, you can leave Linguist disabled in Tools | Extension and only enable it when you needed during revisions.
Linguist's menu lists four self-explanatory items. Selecting any one generates a Writer document with its results displayed one per line. The analysis is rapid, generating results for a 400-page document in less than 20 seconds on a modern desktop system. However, you may prefer to work chapter by chapter rather than with an entire document in order to minimize the scrolling as you work.
List Unrecognized Words
This item is useful for detecting not only non-standard coinages such as "weirded" not found in most dictionaries, but also foreign words such as "hypocaust" (a Roman method of home heating using hot water pipes) (Figure 1). Typos, too, are detected, and while using Linguist, I also discovered several common typos of my own of which I was previously unaware. Similarly, I found that I had spelled one name in five different ways and another in three. No doubt the spell-check would have queried each one, but I may not have seen the pattern and been more careful in the future. Should a document contain no unrecognized words, the results will of course be blank.
Complete Word List (Alphabetical)
At one result per line, the results for this list are extremely long (Figure 2). I probably had no need to know that my test document uses "the" 252 times, but the results did confirm my suspicion that I am too fond of "but" and "and," and made me eliminate many complex and compound sentences.
The results end with the number of different words, the total word count, and the lexical variety. As a ratio, the number of different words and the word count indicate the complexity of expression, while the lexical variety indicates the number of words derived from the same root such as "lives" and "lively" from "live." My test document had a lexical variety of .38, more complex than most newspaper articles, but simpler than most academic papers.
Sort Words on Frequency
These results give the same information as the complete word list, including the final statistics, but arranged by how often they appear rather than by alphabetical order (Figure 3). The complete word list is useful for checking on a particular word, while the frequency is handy when line editing, telling me words that I might want to swap for synonyms or directing me toward sentences I should consider restructuring. When checking frequency, I begin by deleting frequent common words that I can hardly do without, as well as low-frequency words, and then feed the words that were left, on at a time, into Find and Replace and consider each occurrence of the word. While the complete word list gives me an overview, I use the word frequency list for the heavy editing.
Statistics
The Statistics include the lexical variety and the number of different words, but also the number of words per full stop (that is, per sentence), as well as the percentage of long words, often down to several decimal points (Figure 4). These figures are used to derive the LIX readability score, in which 20 is very easy and 60 is very difficult. My test document was just over 30, which is close to what I always aim for.
Further Enhancements
Lexicon would benefit from filters. For instance, it would save time if you could filter common words from the results. Some users, too, might want to see results by part of speech so that they can see, for example, how many adverbs they use and that can be deleted in favor of stronger verbs. Adjectives could also be minimized in favor of nouns.
Those who want such enhancements might look at LanguageTool. In the last few releases, LibreOffice has partially incorporated LanguageTool into its structure. From Tools | Options | LibreOffice | Language Settings, users can enable either the free or subscription version of LanguageTool (Figure 5). Alternatively, they can also install the self-contained extension version of LanguageTool, which functions locally rather than online. However, both the free and the extension versions of LanguageTool are merely grammar checkers, and the fact that they work as you type may not be to everyone's taste. To get editing statistics, you must pay a monthly or yearly subscription fee, and although LanguageTool's statistics are far more advanced than the latest release of Linguist and have a graphical interface, paying for a subscription may be unacceptable to free software users. With luck, Linguist will evolve to something closer to LanguageTool, but even in its current raw form, Linguist lets writers the job done.
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Fedora 41 Released with New Features
If you're a Fedora fan or just looking for a Linux distribution to help you migrate from Windows, Fedora 41 might be just the ticket.
-
AlmaLinux OS Kitten 10 Gives Power Users a Sneak Preview
If you're looking to kick the tires of AlmaLinux's upstream version, the developers have a purrfect solution.
-
Gnome 47.1 Released with a Few Fixes
The latest release of the Gnome desktop is all about fixing a few nagging issues and not about bringing new features into the mix.
-
System76 Unveils an Ampere-Powered Thelio Desktop
If you're looking for a new desktop system for developing autonomous driving and software-defined vehicle solutions. System76 has you covered.
-
VirtualBox 7.1.4 Includes Initial Support for Linux kernel 6.12
The latest version of VirtualBox has arrived and it not only adds initial support for kernel 6.12 but another feature that will make using the virtual machine tool much easier.
-
New Slimbook EVO with Raw AMD Ryzen Power
If you're looking for serious power in a 14" ultrabook that is powered by Linux, Slimbook has just the thing for you.
-
The Gnome Foundation Struggling to Stay Afloat
The foundation behind the Gnome desktop environment is having to go through some serious belt-tightening due to continued financial problems.
-
Thousands of Linux Servers Infected with Stealth Malware Since 2021
Perfctl is capable of remaining undetected, which makes it dangerous and hard to mitigate.
-
Halcyon Creates Anti-Ransomware Protection for Linux
As more Linux systems are targeted by ransomware, Halcyon is stepping up its protection.
-
Valve and Arch Linux Announce Collaboration
Valve and Arch have come together for two projects that will have a serious impact on the Linux distribution.