You are here: Home › Services › IT-Services › High Performance Computing
 

Compute Service Overview

The IT-Services Group operates a high performance cluster computer at the institute for scientific calculations (numerical simulation experiments). The current cluster has been ordered in 2008 and is in operation since spring 2009. The cluster is available to all members of scientific staff of the institute and as well to external scientists in the context of scientific co-operations. Registration is required in order to access the system.

 

The IBM iDataPlex Cluster (IPLEX)

iplex_front.jpg

 

  • Basic metrics
    • 2.560 CPU cores for batch processing.*
      • CPU cores are organized in 320 compute nodes with 2 x 4-core Intel Harpertown CPU and 32 GByte RAM each.
      • Compute nodes are divided into two network domains (green, blue) with 160 nodes each (blue: dx001 - dx160; green: dx161 - dx320).
    • Infiniband DDR interconnect,
    • 800 TByte total disk space.

  • Access

    • Login via secure shell (ssh) using publickey authentication to interactive host(s): cluster.pik-potsdam.de.***

  • File systems and Archiving

    • /iplex/01/
      A parallel file system for project groups with storage requirements of 1 to 20 TByte each. These project file systems will be created upon request only. Regular automated backup of data is default.

    • /scratch/01
      A parallel file system for temporary output only. Can be used by any scientist directly. Current user quota is 10 TByte. No backup. Files which have not been accessed for a given time - currently 3 months - will be deleted  automatically.
    • User initiated archive/retrieve available via the dsmj command on interactive nodes.

  • Documentation

 

  • Statistics (total, group, user) - as sound statistics are indispensable to justify funding for machine enhancements and replacements all users are asked to submit  job files with the proper project group specified.

 

 

* Provided a 50% machine utilization about 12 Mio. CPU hours can basically be delivered to scientific applications every year at a approximated cost of about 0.1 EUR/CPU hour.

 

*** Holders of a PIK computer account, which have been registered for the cluster, should copy their public ssh key into their NFS $HOME/.ssh/authorized_keys file (see man page:  ssh-keygen (linux) for more information) in order to be able to access the cluster immediately. For Windows based client computers it is necessary to use the free software package "putty" instead of the commercial version of the SSH client. External guest need to send their public keys to: cluster-admin@pik-potsdam.de to access the system.

Please don't forget to check access permissions of your $HOME/.ssh/ directory and files. The current Open SSH implementation won't allow access if permissions are not properly restricted.


Document Actions

Comments (5)

Karsten Kramer Jul 08, 2011 05:06 PM
This autumn IBM and the IT-Services group will organize two workshops in order to introduce the new cluster to scientists and to provide "hands on" practical training.

The first workshop - An Introduction to the IPLEX cluster @ PIK - will be held Wednesday and Thursday, September 23rd and 24th of 2009.

For further information and in order to register (registration is mandatory), please go to URL: http://www.pik-potsdam.de/forms/workshops



Karsten Kramer Jul 08, 2011 05:06 PM
Slides of the first workshop are available at URL:

http://www.pik-potsdam.de/[…]/

Ciaron Linstead Jul 08, 2011 05:06 PM
The iPlex compute nodes allow accept user-specified soft memory limits up to the maximum available memory per node (28GB). This can be configured with the 'ulimit' command as follows:

For bash scripts: (#!/bin/bash)
ulimit -Sm [any number between 3670016 and 29360128]
ulimit -Sv [any number between 3670016 and 29360128]

and *additionally* for Korn shell scripts: (#!/bin/ksh)
ulimit -SM [any number between 3670016 and 29360128]

(NOTE: the Bash ulimit accepts different parameters to the Korn shell ulimit!)

You must also specify to LoadLeveler how much free memory your job requires in your job command file thus:

# @ resources = ConsumableMemory(3.5gb)

Note that the LoadLeveler resources parameter only affects job scheduling, whereas ulimits enforce memory usage at runtime.

EXAMPLE

In your job description file add:

# @ resources = ConsumableMemory(8192mb)

Now, assuming you use a shell script to start your executable it shall
look like:

#!/bin/bash
 
ulimit -Sm 8388608
ulimit -Sv 8388608

myexecutable

return 0

ACCOUNTING

Jobs requesting more than 3.5GB per process will be accounted for by multiples of CPU runtime. For example, a job requiring 10GB per process will be recorded as having used 3x CPU time (since 2 extra CPUs will be unavailable for LoadLeveler scheduling).
Ciaron Linstead Jul 08, 2011 05:06 PM
The iPlex Parallel Programming workshop was held on the 28th and 29th September 2010 at PIK.

A rough outline of the agenda was as follows:

Day 1: An update on the Intel tools available at PIK (including an introduction to the new Performance Tuning Utility (PTU))

Day 2: An introduction to the partitioned global address space (PGAS) parallel programming model, in particular UPC (an extended C for PGAS programming) and X10 (a new language for PGAS programming).

The programme included an introduction to using the Eclipse IDE with the PTU and the X10 development toolkit (X10DT), and how to configure VNC to run a remote Eclipse instance.

Course materials are available under /home/pospiech/PIK_course on the cluster.
Ciaron Linstead Jul 08, 2011 05:06 PM
Materials available on the PIK Portal here:

http://www.pik-potsdam.de/i[…]gramming-workshop-materials