Optimizing bash scripts for multi-core processors

Swapping Out to Other Machines

If you aren't satisfied with the efficiencies of parallel processing on a local machine, you can even apply this principle to the network. In that case, a first-level dispatcher could use TCP/IP to talk to multiple second-level dispatchers on various machines. The second-level dispatchers then talk to their local worker processes. This approach is only useful if you have a secure network, of course.


With just a couple of lines of code, you can use the techniques described in this article to parallelize existing shell scripts. Other scripting languages can use this approach; however, some languages offer superior alternatives. For example, Python uses explicit forking (os.fork()) in addition to pipes (os.pipe()), which allows for a low-level solution that is very close to the efficiency of the C programming language.


  1. Dynamic dispatcher source code: http://www.linux-magazine.com/resources/article_code

The Author

Bernhard Bablok manages a large data warehouse with technical performance data for machines ranging from mainframes to servers at Allianz Shared Infrastructure Services. Besides listening to music, hiking, and biking, Bernhard enjoys anything related to Linux and object-oriented programming.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Bash 4

    Despite the Bourne-again shell's biblical age and high level of maturity, developers continue to work on it. We take a look at the latest Bash release.

  • Perl: Personal Finances

    A helpful Perl script gives you an immediate overview of your financial status, adding the balances of multiple accounts and share depots. It even allows users to add their own plugins.

  • Scripting Tutorial for Bash 4.0

    The "Advanced Bash Scripting Guide" version 6 has been reworked to incorporate the new features of Bash 4.0.

  • Bash scripting

    A few scripting tricks will help you save time by automating common tasks.

  • Bacula

    When backup jobs become too challenging for a script, the daemon-based free backup tool Bacula may be the answer.

comments powered by Disqus

Direct Download

Read full article as PDF:

056-059_bashscript.pdf  (320.40 kB)


njobs Europe
Njobs Netherlands Njobs Deutschland Njobs United Kingdom Njobs Italia Njobs France Njobs Espana Njobs Poland
Njobs Austria Njobs Denmark Njobs Belgium Njobs Czech Republic Njobs Mexico Njobs India Njobs Colombia