You are here: Home Services IT-Services Overview News GNU Parallel installed on cluster

GNU Parallel installed on cluster

GNU parallel is a shell tool for executing jobs in parallel using one or more computers.

GNU Parallel, a shell tool for executing jobs in parallel using one or more computers, has been installed on the cluster. A typical workload might be handling the pre-
or post-processing of a large set of data files.

Parallel handles the splitting of the workload across the resources available
to it, and also includes logging/resume capability so you can restart partially
completed processing.

To start using it, load the most recent module:

module load parallel

A README and some example code is available in the gnu-parallel subdirectory of PIK IT's HPC cluster-examples GitLab repository:

https://gitlab.pik-potsdam.de/its/hpc/cluster-examples/tree/master/gnu-parallel

Further reading:

1) https://www.gnu.org/software/parallel/

2) man parallel (after module load)

3) man parallel_tutorial (after module load)

Document Actions