A modern compression tool

Command Line – zstd

© Lead Image © Lembit Ansperi, 123RF.com

© Lead Image © Lembit Ansperi, 123RF.com

Article from Issue 250/2021

Like other modern replacement commands, zstd offers significantly faster file compression than the standard archiving tools.

Unix-like systems have been around long enough that replacements are available for time-honored commands. For instance, tree is a substitute for ls, while apt has unified apt-get and the most popular of its associated scripts. Since 2015, one of the most popular substitutes has been Zstandard (zstd) [1], a compression tool that is a simplification of gzip that is significantly faster than standard archiving tools such as tar, zip, and bzip.

An LZ77 lossless data compression algorithm [2] gives zstd its speed. Algorithms in the LZ77 family are a form of sliding window compression, so-called because they encode in chunks of customizable sizes as they copy and verify results. Chunks that are too small do nothing to increase speed, but ones that are too large start to slow compression because they take longer to verify. Finding the right balance maximizes the speed of an operation. To help obtain the most efficient setting, zstd can build and use a dictionary of settings for different types of files (see the "Using a Dictionary" section).

To LZ77 compression, zstd adds two modern, high-speed entropy encoders [3], which are used at the end of compression. Huffman, with its out-of-order (OoO) execution, offers high speed operations, while Finite State Entropy (FSE), a more recent entropy encoder, is designed to ensure the accuracy of compression at high speeds. Armed with LZ77 compression and these two entropy encoders, zstd easily outperforms other archiving commands, especially when the command options are carefully chosen.


Use Express-Checkout link below to read the full article (PDF).

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Command Line – zstd

    In an effort to meet modern computing needs, zstd offers a greater degree of compression at a faster compression rate, with unique options to enhance performance.

  • Optimizing System Startup

    Today's Linux systems boot faster than ever, but many users still get impatient waiting for that first glimpse of desktop. These tweaks will help you get a faster start from the bootloader and kernel.

  • Optimizing the Kernel

    We explore some optimizations designed to deliver a smoother experience for desktop users.

  • BorgBackup

    In Star Trek, the Borg adds individuals to its collective consciousness, an apt metaphor for any backup application that stores individual files in an archive. BorgBackup creates folder repositories for multiple archives, making it an especially befitting description of assimilation.

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95