Sie sind hier: Startseite » Institute » Organization » IT-Services » Services » High Performance Computing
Benutzerspezifische Werkzeuge

Compute Service Overview

The IT-Services Group currently operates a high performance computing cluster  for scientific calculations. The cluster is available to all PIK scientists which had been registered by their group/research domain and upon request also to external guest scientists. Access is granted via PIK standard computer accounts.*

 

The IBM iDataPlex Cluster (IPLEX)

iplex001s.JPG

 

  • Basic metrics
    • 2.784 CPU cores* in three network domains,
    • 320 nodes with 2 x 4-core Intel Harpertown CPU and 32 GByte RAM each (dx001 - dx320),
    • 56 nodes with 2 x 2-core Intel Woodcrest CPU and 8 GByte RAM each (blade01 - blade56),
    • Infiniband DDR interconnect,
    • 300 TByte total disk space.

  • Login via secure shell (ssh) using publickey authentication to interactive host(s): cluster.pik-potsdam.de.**

  • Batch jobs are controlled by IBM Load Leveler.
  • Open MPI Library for parallel programming installed.

  • Two parallel file systems available:
    • /iplex/01/
      A parallel file system for project groups with storage requirements of 1 to 10 TByte each. These project file systems will be created upon request only. Regular automated backup of data is default.

    • /scratch/01
      A parallel file system for temporary output only. Can be used by any scientist directly. No backup. Files which have not been accessed for a given time - currently 3 months - will be deleted  automatically.

  • User initiated archive/retrieve available via the dsmj command on interactive nodes.

 

 


* Provided a 50% machine utilization about 12 Mio. CPU hours can basically be delivered to scientific applications every year at a approximated cost of about 0.1 EUR/CPU hour.

** Holders of a PIK computer account, which have been registered for the cluster, should copy their public ssh key into their NFS $HOME/.ssh/authorized_keys file (see man page:  ssh-keygen (linux) for more information) in order to be able to access the cluster immediately. For Windows based client computers it is necessary to use the free software package "putty" instead of the commercial version of the SSH client. External guest need to send their public keys to: cluster-admin@pik-potsdam.de to access the system.

Please don't forget to check access permissions of your $HOME/.ssh/ directory and files. The current Open SSH implementation won't allow access if permissions are not properly restricted.

 

 

erstellt von Karsten Kramer zuletzt verändert: May 10, 2010 04:20 PM
Mitwirkende: Ciaron Linstead, Roger Grzondziel, Joachim Glauer, Werner von Bloh
Artikelaktionen

Upcoming Workshops

Kommentar von Karsten Kramer am Aug 28, 2009 02:02 PM
This autumn IBM and the IT-Services group will organize two workshops in order to introduce the new cluster to scientists and to provide "hands on" practical training.

The first workshop - An Introduction to the IPLEX cluster @ PIK - will be held Wednesday and Thursday, September 23rd and 24th of 2009.

For further information and in order to register (registration is mandatory), please go to URL: http://www.pik-potsdam.de/forms/workshops



Slides of 1st Workshop available.

Kommentar von Karsten Kramer am Sep 23, 2009 09:21 PM
Slides of the first workshop are available at URL:

http://www.pik-potsdam.de/[…]/

Configuring memory-intensive jobs

Kommentar von Ciaron Linstead am Apr 19, 2010 01:38 PM
The iPlex compute nodes allow accept user-specified soft memory limits up to the maximum available memory per node (28GB). This can be configured with the 'ulimit' command as follows:

For bash scripts: (#!/bin/bash)
ulimit -Sm [any number between 3670016 and 29360128]
ulimit -Sv [any number between 3670016 and 29360128]

and *additionally* for Korn shell scripts: (#!/bin/ksh)
ulimit -SM [any number between 3670016 and 29360128]

(NOTE: the Bash ulimit accepts different parameters to the Korn shell ulimit!)

You must also specify to LoadLeveler how much free memory your job requires in your job command file thus:

# @ resources = ConsumableMemory(3.5gb)

Note that the LoadLeveler resources parameter only affects job scheduling, whereas ulimits enforce memory usage at runtime.

EXAMPLE

In your job description file add:

# @ resources = ConsumableMemory(8192mb)

Now, assuming you use a shell script to start your executable it shall
look like:

#!/bin/bash
 
ulimit -Sm 8388608
ulimit -Sv 8388608

myexecutable

return 0

ACCOUNTING

Jobs requesting more than 3.5GB per process will be accounted for by multiples of CPU runtime. For example, a job requiring 10GB per process will be recorded as having used 3x CPU time (since 2 extra CPUs will be unavailable for LoadLeveler scheduling).