Beowulf cluster

A Beowulf cluster is a computer cluster of normally identical, commodity-grade computers networked into a small local area network with libraries and programs installed that allow processing to be shared among them. The result is a high-performance parallel computing cluster from inexpensive personal computer hardware.

Beowulf originally referred to a specific computer built in 1994 by Thomas Sterling and Donald Becker at NASA. They named it after the Old English epic poem, Beowulf.

No particular piece of software defines a cluster as a Beowulf. Typically only free and open source software is used, both to save cost and to allow customization. Most Beowulf clusters run a Unix-like operating system, such as BSD, Linux, or Solaris. Commonly used parallel processing libraries include Message Passing Interface (MPI) and Parallel Virtual Machine (PVM). Both of these permit the programmer to divide a task among a group of networked computers, and collect the results of processing. Examples of MPI software include Open MPI or MPICH. There are additional MPI implementations available.

Beowulf systems operate worldwide, chiefly in support of scientific computing. Since 2017, every system on the Top500 list of the world's fastest supercomputers has used Beowulf software methods and a Linux operating system. At this level, however, most are by no means just assemblages of commodity hardware; custom design work is often required for the nodes (often blade servers), the networking, and the cooling systems.

A description of the Beowulf cluster, from the original "how-to", which was published by Jacek Radajewski and Douglas Eadline under the Linux Documentation Project in 1998:

As of 2014^[update] a number of Linux distributions, and at least one BSD, are designed for building Beowulf clusters. These include:

• MOSIX, geared toward computationally intensive, IO-low applications
• Rocks Cluster Distribution, latest 2017
• DragonFly BSD OS, latest 2022
• Quantian OS latest 2006, a live DVD with scientific applications, remastered from Knoppix
• Kentucky Linux Athlon Testbed, physical installation at University of Kentucky

The following are no longer maintained:

• Kerrighed (EOL: 2013)
• OpenMosix (EOL: 2008), forked from MOSIX
• ClusterKnoppix OS, forked from Knoppix OS, forked from OpenMosix
• PelicanHPC OS latest 2016, based on Debian Live

A cluster can be set up by using Knoppix bootable CDs in combination with OpenMosix. The computers will automatically link together, without need for complex configurations, to form a Beowulf cluster using all CPUs and RAM in the cluster. A Beowulf cluster is scalable to a nearly unlimited number of computers, limited only by the overhead of the network.

Provisioning of operating systems and other software for a Beowulf Cluster can be automated using software, such as Open Source Cluster Application Resources. OSCAR installs on top of a standard installation of a supported Linux distribution on a cluster's head node.

• Aiyara cluster
• HTCondor
• Grid computing
• Kentucky Linux Athlon Testbed
• Maui Cluster Scheduler
• Open Source Cluster Application Resources
• Stone Soupercomputer
• Oracle Grid Engine

• Beowulf Cluster Computing With Windows by Thomas Lawrence Sterling 2001ISBN 0262692759 MIT Press
• Beowulf Cluster Computing With Linux by Thomas Lawrence Sterling 2001ISBN 0262692740 MIT Press

• The Beowulf Cluster Site
• A detailed description of building a Beowulf cluster